Middle school students' self-efficacy, attitudes, and achievement in a computer-enhanced problem-based learning environment.

This study examined the effect of a computer-enhanced problem-based learning (PBL) environment on middle school students' learning, investigating the relationship among students' self-efficacy, attitude toward science, and achievement. As Bandura defined it (1986), self-efficacy refers to the beliefs people have about whether or not they can successfully complete a task. From analyses of quantitative and qualitative data, findings indicated an increase in students' science achievement and self-efficacy for learning science after their engagement in a computer-enhanced PBL environment; however, no significant changes were seen in their attitude toward science. Students' attitude toward science and self-efficacy beliefs were positively related to each other. Self-efficacy was shown to be a statistically significant predictor of science achievement scores but attitude was not. In addition, when groups were formed based on a median split of attitude and self-efficacy scores, a significant interaction was found. Among students with low attitude, science achievement scores were significantly higher for the high self-efficacy than for the low self-efficacy group, while students in the high attitude group showed no difference in the achievement scores regardless of their self-efficacy grouping. Results suggested that students' self-efficacy towards science learning could be used to predict achievement.

Introduction

How middle school students approach science is of interest for two reasons. Early experiences with science can be expected to have an effect on the likelihood that students would pursue science in later grades. It may also be easier to ameliorate students' attitudes toward science and their self-efficacy beliefs (i.e. students' beliefs about completing a task successfully) when they are still in the early stages of exposure to science. In this study, we examined changes in middle school students' achievement, self-efficacy, and attitude toward science as a result of their participation in a computer-enhanced problem-based learning environment. Problem-based learning (PBL) exemplifies student-centered learning where students take responsibility for their own learning processes and build knowledge through their learning experiences (Cobb & Bowles, 1999). Although PBL has been shown to be effective in medical education and with college and gifted students, literature has indicated that implementing such complex and ill-structured learning environments in K-12 classrooms has been challenging. Additional support in the form of computer-based tools can enhance PBL delivery and extend the benefits of PBL to all students. A computer-enhanced PBL provides a new and different means for students to develop problem-solving skills, to reflect on their own learning, and to develop a deep understanding of the content domain (Cognition and Technology Group at Vanderbilt, 1997). In this study, sixth-graders were engaged in a computer-enhanced PBL learning environment as part of their science curriculum. It was hypothesized that the students would exhibit a positive change in their science achievement, science self-efficacy beliefs, and attitude toward science after their participation in this environment. Literature on problem-based learning, self-efficacy, and attitude provided a basis for our investigation.

RESEARCH FRAMEWORK

Problem-Based Learning

Problem-based learning (PBL) is a learning process in which students are presented with a problem and are asked to apply reasoning, questioning, researching, and critical thinking to find a solution to the problem. It emphasizes solving complex problems in resource-rich contexts and aims at developing higher-order thinking skills (Savery & Duffy, 1995). The focus is not on the outcome but on the process by which students become self-reliant and independent. The essential characteristics of PBL include: (a) learning is student-centered; (b) authentic problems form the organizing focus for learning; (c) new information is acquired through self-directed learning; (d) learning occurs in small groups; and (e) teachers act as facilitators (Barrows, 1996).

Originally developed for medical education, its success in that setting has sparked the interest of educators in other fields. PBL has been shown to be more effective than traditional classroom instruction in providing opportunities for transferring knowledge and skills from the classroom to the workplace (Lunyk-Child, Crooks, Ellis, Ofosu, O'Mara, & Rideout, 2001; Stepien, Gallagher, & Workman, 1993). It results in better long-term content retention than traditional instruction (Norman & Schmidt, 1992) and supports the development of problem-solving skills (Gallagher, Stepien, & Rosenthal, 1994; Hmelo & Ferrari, 1997).

Most successful uses of PBL come from non-technology based settings involving college or gifted K-12 students. Educators interested in using PBL in their classrooms recognize the challenges of its implementation (Farnsworth, 1994). Such challenges include ineffective ways to present the central problem through oral or written means, large investment in time and effort to develop PBL units, initial discomfort with the methodology from learners and instructors, and the need for new forms of assessment (Farnsworth, 1994; Hoffman & Richie, 1997). Although challenges are being recognized, technology has been suggested as a means of addressing some of these challenges. Technology creates "new opportunities for curriculum and instruction by bringing real-world problems into the classroom for students to explore and solve" (Bransford, Brown, & Cocking, 2000, p. 195). Interactive video, for example, has been used to present students with real-life problems that require students to understand and apply important concepts in mathematics (Cognition and Technology Group at Vanderbilt, 1997). Interactive and media rich computer tools can assist learners to generate and test hypotheses and can engage students in activities that would be out of their reach otherwise. Computers can enhance PBL delivery and provide necessary scaffolding to learners (Hoffman & Richie, 1997). Such suggested benefits of a computer-enhanced PBL environment on students' learning, however, need to be verified with research evidence.

Self-Efficacy and Achievement

Self-efficacy refers to people's beliefs about their capabilities to perform a task successfully at designated levels (Bandura, 1986, 1997). It is a construct different from the colloquial term confidence because confidence is used to describe the strength of a belief but it does not specify the content of the belief, so that one can be quite confident in being unsuccessful (Bandura, 1986, 1997). Also, the term self-efficacy is to be differentiated from notions of ability or capacity. In Bandura's conception of self-efficacy, the idea is that learners develop a sense of, or a belief about, how well they are likely to perform in a task. This sense of how well they will perform is separate from, though clearly tied to, their actual ability (Lane & Lane, 2001), and has been shown to influence performance. Self-efficacy is concerned not with the skills one has but the judgments of what one can do with whatever skills one possesses. According to Bandura (1986), "If self-efficacy is lacking, people tend to behave ineffectually, even though they know what to do" (p.425).

Motivation is enhanced when students perceive they are making progress in learning or feel they are capable of handling the task. In turn, as students work on tasks and become more skillful, they develop a sense of self-efficacy for performing well (Schunk, 1991). It has been suggested that one's perceived self-efficacy has a powerful influence over one's choice of activity, the kind of effort one expends, and how much one is able to maintain that effort in the face of difficulty (Bandura, 1986, 1997; Schunk, 1995). Consequently, self-efficacy beliefs have been found to influence students' attitude, their engagement in an activity, and how strategic they may be when encountering a task (Pajares & Johnson, 1996; Pajares & Miller, 1994, 1995). Students' self-efficacy beliefs also seem to be significantly related to their academic performance (Lane & Lane, 2001).

Although research supports the positive relationship between self-efficacy beliefs and academic achievement, much of the evidence has been garnered in such areas as sports and traditional math classes. More research is needed in other areas because self-efficacy and its relationship to achievement may vary in different domains of learning. Bandura (1986) cautioned that because judgments of self-efficacy are domain specific, different ways of assessing learners' confidence at completing a given task successfully would be needed to correspond to the assessed performance. In a PBL environment, students not only have more autonomy over their learning, they also have more responsibility for their learning processes and outcomes, putting a heavier burden on students' motivation and capability to complete learning tasks. If students are not equipped with the ability to learn in a self-paced, self-directed environment, their motivation and achievement may suffer. Likewise, without the belief that they are capable at completing tasks successfully in the PBL environment, students may shy away from this activity, which may in turn hurt their attitude and performance in class. Because students' self-directed learning can be assisted by computer tools, it is possible that students may find support in a computer-enhanced PBL environment and gain self-efficacy when tackling the complex problems that are at the heart of a PBL lesson. This study was therefore intended to examine students' initial self-efficacy beliefs and how these beliefs changed as their experiences changed.

Attitude and Achievement

Students' attitude is another factor highly associated with success and motivation. Having high ability or skills does not guarantee that students will enjoy working at an activity or perform successfully on the task. Often students need to have an internal drive and a positive mindset about the learning task in order to sustain motivation. Attitudes and beliefs are regarded as precursors of behaviors and behavioral intentions. Students with positive attitudes are more likely to sustain their efforts and have the desire to be involved in the learning tasks.

Loyd and Gressard (1984) found that students' attitudes about computers were strongly related to their success in computer-related tasks. A study by Mattern and Schau (2002) showed that positive attitude toward science is related positively to achievement. Germann (1988) reported that attitudes toward science influence students' attainment, consistency, and quality of work. In reviewing the literature about attitudes toward science over the past 20 years, Osborne, Simon, and Collins (2003) noted that research has indicated a decline in attitudes toward science from age 11 onward and an apparent contradiction between students' attitudes toward science in general and their attitudes toward school science. Students consider science itself interesting and useful but regard science classes as boring (Ebenezer & Zoller, 1993). Teachers, curricula, and culture, among other factors, were cited as influence on students' attitudes (Osborne, et al., 2003). Thus, how science is taught in classrooms can affect learners' interest in science as a subject and a career.

Research Questions

Three research questions guided this study:

1. Do students' self-efficacy for learning science, attitude toward science, and science achievement change as a result of using a computer-enhanced PBL environment?

2. Is there a relationship between students' science self-efficacy beliefs and their attitude toward science after they engaged in a computer-enhanced PBL environment?

3. How do students' science self-efficacy beliefs and their attitude toward science relate to their science achievement after they engaged in a computer-enhanced PBL environment?

METHOD

Participants

The participants were 549 sixth-graders from two middle schools in a mid-sized southwestern city. Sixth-grade students were selected because the PBL environment was designed for sixth-grade science. The study took place over a three-week time period in the second semester of the school year. The schools were from the same district and shared similar demographics: 16% Hispanic, 6% African-American, 73% Caucasian, and 5% other ethnic backgrounds. Of the 549 students, 49.3% (n = 271) were female students. Approximately 75% of the students (n = 412) were in regular education classes, 15% of the students (n = 82) were in Talented and Gifted (TAG) classes, and 10% of the students (n = 55) were students with special needs or resource students. This study employed a pretest and posttest design because all sixth-grade science classes from these two schools elected to use the computer-enhanced PBL program described below. Five teachers taught these science classes and indicated that the students had sufficient computer skills to use the program.

Treatment: A Computer-Enhanced PBL Environment

Alien Rescue, a CD-based hypermedia PBL environment, was used in this study. Alien Rescue was designed to engage sixth-grade students in a complex task that requires them to use problem-solving skills and acquire knowledge about both our solar system and the tools and procedures scientists use to solve problems (Liu, Williams, & Pedersen, 2002). The program begins with a video presentation of a complex problem. A group of six species of aliens, different in their characteristics, have arrived in Earth's orbit escaping the explosion of their home planets. Students, acting as scientists, are asked to participate in this rescue operation, and their task is to help the aliens find new homes that can support their life forms by determining the most suitable relocation site for each alien species.

To solve this problem, students must engage in a variety of problem-solving activities. They need to study the aliens' needs and research the planets in our solar system to find possible new homes. Students must also take part in planning and decision-making as they determine how to use the resources of the solar system effectively. The program was purposely built to have more information available to students than needed in solving the problem so that students must actively sift through the vast information and decide what is relevant and important. Given the complex nature of the problem and the vast amount of information available, a set of computer tools is provided to scaffold students' problem-solving process. For example, information about the characteristics of each alien species (what different species look like, where they live, the food they eat, etc.), the atmosphere and gravity on a planet, and past NASA missions are provided through carefully constructed knowledge databases enhanced through graphics, animations, and 3-D videos. If students come across a scientific concept with which they are unfamiliar, they can look it up in the concept database that provides visually illustrated tutorials on various science topics. To assist students' cognitive processes, an expert is available at four critical points via the video format to model expert thinking process in solving the central problem. The probe builder and launcher tools allow students to design probes with various scientific instruments and launch them. Students then study the data returned from the probes to see if a planet has the adequate conditions to be a home for a species. Such computer tools enable students to perform cognitive activities that would be out of their reach otherwise. More information about the program can be found on http://alienrescue.edb.utexas.edu/.

Students used Alien Rescue in their daily 45-minute science class for a total of 15 days. Because Alien Rescue was aligned with the national science standards on scientific investigation and problem solving, the teachers used it in place of the regular teaching materials on the solar system unit. Students worked on computers in the computer lab. They were assigned to groups of two or three by their teachers to solve the central problem collaboratively, but each student had his or her own computer.

This was the second year that four of the teachers had used Alien Rescue and the fourth year for one of the teachers. All five teachers participated in a training workshop before implementing Alien Rescue. During the workshop, the philosophy and attributes of student-centered learning environments were discussed in depth. In particular, the role of the teacher as a facilitator was highlighted. In their use of Alien Rescue, the teachers did not do any direct teaching. They did not tell students what to do specifically, and did not teach the science concepts needed to solve the central problem prior to or during the use of Alien Rescue. On the first day, students watched the video scenario in Alien Rescue, and then the teachers and the students held a discussion about the primary task the students faced. For the next 14 days, teachers allowed the students to decide what their learning tasks were for each day and how to approach solving the problem. Typically, each day began with a mini-discussion on what the students had accomplished in the previous day and teachers addressed questions that came up. Students' questions were often answered by more questions from the teachers or answered by other students. Then, the students went to their computer stations. Each group divided tasks among themselves and made their own decisions as to how to proceed. The teachers walked around the room answering students' questions, checking progress, and ensuring students were on task. Most days, the lesson ended with another short discussion on what the students had accomplished that day, any questions that surfaced, and what the goal should be for the next day. Sample lesson plans can be found at http://jabba.edb.utexas.edu/liu/aliendb/Lesson-Plan.htm. Because all the necessary tools for students to work on the problem were provided via computers in this PBL environment, it was possible for the teachers to spend most of the class time interacting with students individually. The teachers provided scaffolding to the students through daily questioning, answering, and discussion.

Data Sources and Analyses

To address the research questions, both quantitative and qualitative data were collected. Qualitative data were analyzed to substantiate the findings from the quantitative data. These data sources are described below.

Science achievement test. Achievement was assessed by measuring students' understanding of the various scientific concepts introduced in Alien Rescue. A 25-item test (KR(20) = .73) was used, which reflected what the designers and subject matter experts consider as important for the students to acquire after using Alien Rescue. It addresses both factual knowledge (n=15) and application (n=10) questions, and was used in previous studies with similar samples (Liu, 2004, 2005; Liu, Bera, Corliss, Svinicki, & Beth, 2004). Examples of the questions are as follows:

Factual: Which of these worlds is a planet (not a moon)?

a. Charon

b. Io

c. Phobos

d. Uranus

Application: You need to design a probe to go to Titan to find out if it has a magnetic field or earthquakes. Which of the following would you choose to include on your probe?

a. a battery and a solar panel

b. an infrared camera and a magnetometer

c. a barometer and a seismograph

d. a magnetometer and a seismograph

Because no direct teaching was involved, a gain in the performance score would indicate that the student had acquired a good understanding of the scientific concepts needed for problem solving through his or her self-directed learning, classroom discussions, and/or peer interaction while using Alien Rescue.

Self-efficacy questionnaire. Students' self-efficacy beliefs were measured using eight items from the Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich, Smith, Garcia, and McKeachie, 1993), which dealt specifically with students' self-efficacy for learning and performance. The reported Cronbach's alpha for the eight items was .93. Students rated themselves on a 5-point Likert scale with 1 being "not at all true of me" and 5 being "very true of me." For each of the eight items, "science class" was used in place of "course" in the original statements. For example, "I am confident I can learn the basic concepts taught in this science class" was used in place of "I am confident I can learn the basic concepts taught in this course." In the post-survey, "science class" in each statement was defined as "science class using Alien Rescue." The alpha for the sample in this study was 0.93.

Attitude toward science questionnaire. Students' attitude toward science as a school subject was measured using the Attitude Toward Science in School Assessment (ATSSA) (Germann, 1988). This instrument consists of 14 Likert scale items, with I being "strongly disagree" and 5 being "strongly agree." Sample statements include "Science is fun" and "During science classes, I usually am interested." According to Germann, the scale has been used with students from seventh to tenth grade in science classes in four different studies, with a reported Cronbach's alpha of .95. The alpha for the sample in this study was 0.96. In the post-survey, "science" in each statement was defined as "science using Alien Rescue." For example, "Science is fun" became "Science with Alien Rescue was fun."

The same science achievement test, self-efficacy and attitude toward science questionnaires were given to the students both before and after their use of Alien Rescue to measure any changes. The pretests were given before the PBL experience and the posttests were given after the three weeks of PBL implementation. Since there were more than 15 days apart between the pretest and posttest, it was unlikely that the students would remember the content in the pretest. The teachers did not teach to the posttest (see "Treatment" section above).

An ANOVA with repeated measures was run to see if there were changes in scores from pre to post in students' science achievement, self-efficacy for science, and attitude toward science (for research question 1). Correlation analyses were conducted to examine the relationship between students' self-efficacy and attitude toward science (for research question 2), and the relationship between science achievement scores, attitude toward science, and self-efficacy at the posttest (for research question 3). A multiple regression analysis was then used to determine whether students' self-efficacy beliefs and attitude toward science, as independent variables, would predict students' science achievement (for research question 3).

Open-ended questions and interviews. After completing the Alien Rescue program, students were asked to respond to three written open-ended questions: "When working with Alien Rescue, do you feel that you can understand science better? Why or why not?" "Do you think that you like science more OR less after you used Alien Rescue? Give 3 reasons why," and "Do you feel more confident about learning science successfully after using Alien Rescue? Give 3 reasons why." Scheduled from the middle of the Alien Rescue unit on, individual and small group interviews were conducted with a selected group of 20 students who were identified by their teachers as either doing well or not well in their regular science class. Interview questions included, "Do you like working on science using Alien Rescue?" "What aspect of it did you like the most or least?" "In the Alien Rescue class, are you confident in finding the answers on your own since your teacher is not teaching you the information directly now?" and "What do you do when you can't find the answers?"

The responses to the open-ended questions and the transcribed interview data were coded and analyzed following the guidelines by Miles and Huberman (1994). Patterns from the data were extracted. The data were sorted into categories and subcategories according to their common themes and shared relationships. Three researchers independently coded one third of the data and then discussed the emerging themes until a consensus was reached on categories and subcategories.

RESULTS

Changes in Science Achievement, Self-Efficacy, and Attitude from Pre- to Post-Assessment

Results of the ANOVA with repeated measures indicated that there was a significant increase in science achievement from pretest to posttest: F (1, 463) = 1214.96, p < .001 (see Table 1). Students' self-efficacy scores had also increased significantly at the posttest: F (1, 463) = 12.61, p < .001, although scores were already above the mid-point of the scale at the pretest.

By contrast, a comparison of scores on the attitude toward science scale did not show a significant difference between pretest and posttest. It is important to note, however, that attitude scores were all above the mid-point of the scale (see Table 1).

Relationships Among Self-Efficacy, Attitude, and Science Achievement

Correlation analysis between attitude toward science and self-efficacy for science showed that the two factors were positively related: r = .60, p < .001. Another correlation analysis showed that attitude toward science and self-efficacy were positively related to the science achievement test: [r.sub.attitude] = .11, p < .05; [r.sub.self-efficacy] = .28, p < .001. Regression analysis, examining whether students' self-efficacy beliefs and attitude toward science would predict students' post-science achievement scores, showed a significant [R.sup.2] of .08, F (2, 489) = 22.30, p < .001. Of the two predictors, self-efficacy was found to significantly predict science achievement ([beta] = .33, t (489) = 6.02, p < .001) while attitude was not (p = .19).

Further analyses were conducted to examine how students with different levels of self-efficacy and attitude performed. Students were divided into high and low self-efficacy groups and high and low attitude groups using the median split method. A two-way ANOVA was run with high and low self-efficacy, and high and low attitude as independent variables, and the science achievement posttest scores as the dependent variable. ANOVA results supported the findings from the regression analysis in that students with high self-efficacy scored higher on the post science achievement test (M = 74.96) than students with low self-efficacy (M = 69.81), F (1, 454) = 7.55, p < .01. There was no significant difference in the science achievement scores between high (M = 73.29) and low attitude groups (M = 71.48).

Although there was no interaction between the two independent variables of attitude and self-efficacy at the pretest, there was an interesting and statistically significant interaction effect between self-efficacy and attitude groups at the posttest, F (1, 454) = 8.94, p < .01 (see Table 2). At the pretest, the self-efficacy and the attitude variables indicated main effects such that the higher groups performed better than the lower groups. However, at the posttest, a large difference in science achievement scores was found for students in the low attitude group divided based on their self-efficacy levels: The high self-efficacy group scored significantly higher than the low self-efficacy group on their achievement scores. On the other hand, students in the high attitude group showed no difference in posttest science achievement scores regardless of their self-efficacy levels.

DISCUSSION

Changes in and Relationship Among Science Achievement, Self-Efficacy, and Attitude

Results on the science achievement test indicated that students gained significantly more science knowledge after engaging in learning with Alien Rescue. This finding supports previous research examining the effectiveness of this computer-enhanced PBL environment, which showed similar positive impact of the Alien Rescue program (Liu, 2004, 2005; Liu, Bera, Corliss, Svinicki, & Beth, 2004). The findings from this and previous research together suggest that a computer-enhanced PBL program such as Alien Rescue is promising in helping sixth-graders gain science knowledge. The indication that well-designed technology tools can assist learning is inline with research conducted on other technology-based interactive learning environments (Cognition & Technology Group at Vanderbilt, 1997; Corderoy, Harper, & Hedberg, 2003; Lajoie, 1993).

Ideally, such positive effects should be verified with a control group. However, this research was conducted in classrooms where the use of Alien Rescue was an integral part of the curriculum. All participating teachers, who had used the program previously and were positive about the learning benefits of the PBL environment, felt strongly that all students should have access to the program at the same time. Consequently, having a control group was not feasible with these two schools. We did attempt to find an intact class of 57 gifted sixth-graders from a different school in a different southwestern city but from the same state. Students in this comparison group had similar demographics as the students in the Alien Rescue group (18% Hispanic, 4% African-American, and 78% Caucasian students, about 50% were female students). These students were asked to complete the same science knowledge test (as described above) after they completed a textbook lesson on the solar system toward the end of their sixth-grade school year. An unpaired t-test indicated that the Alien Rescue group significantly outperformed the comparison group with respective means of 69.98 and 59.02 in their post science test, t (575) = 4.34, p < .001. We were aware, however, that the comparison of these two groups was limited given the two groups were from the intact classes and from different schools. Finding a true comparison group is a challenge in conducting classroom-based research. Thus we made an effort to use multiple data sources, both quantitative and qualitative, to provide a more complete picture of what was going on the classrooms.

The pretest and posttest comparison showed that sixth-graders' self-efficacy for science had significantly increased after being engaged in the computer-enhanced PBL environment. The finding that students' self-efficacy was a significant predictor of students' achievement was in support of self-efficacy research conducted in traditional classroom settings which showed that the belief and students' actual performance was highly correlated (Lane & Lane, 2001; Pajares & Miller, 1994). Interestingly, although students' attitudes toward science were positively related to their self-efficacy beliefs, the small increase in attitude scores was not significant from pretest to posttest and students' attitudes were not a predictor of their achievement. It is possible that this group of students already had high attitudes toward science prior to their PBL experience and there was a ceiling effect for further increase in attitude. More research is needed to replicate such findings especially with students whose attitudes are not as high at pre-assessment.

It is expected that students in the high self-efficacy or high attitude groups would perform better in their science test than students in the low self-efficacy or low attitude groups. This was the case before the students used Alien Rescue. However, after the implementation of the PBL program, it was interesting to note that students with low attitude but high self-efficacy scored the highest on their science achievement posttest while students in the high attitude group showed no difference in science achievement regardless of their self-efficacy levels. This interaction effect of self-efficacy and attitude suggested a strong relationship between self-efficacy and achievement especially when students' attitude toward science was less positive. That is, high self-efficacy played a more important role in improving science achievement for students who did not like science than for students who did like science. Having a positive attitude toward learning science, on the other hand, helped students maintain a certain level of achievement even when they did not feel capable. Attitude, therefore, seemed to mitigate the effect of low self-efficacy.

Such findings have practical implications in that teachers can help students with low attitude or low self-efficacy increase their achievement level by enhancing their self-efficacy beliefs. Enhancing students' self-efficacy appears more likely to lead to an increase in the performance than enhancing their attitude. In Alien Rescue, students were encouraged to take control over their learning, and, more importantly, they were provided computer tools in assisting them to complete learning tasks on their own. When students succeeded in such a self-paced and self-directed environment, their motivation was high and their confidence in completing the task successfully also increased. Such increased self-efficacy appeared to be positively related to the actual performance.

Insights Offered by the Qualitative Data

Learning science. The qualitative data corroborated the findings from the quantitative data and offered more detailed information about what the students were experiencing as they learned science in this new PBL environment. The qualitative data showed that the sixth-graders learned and liked science, and felt more confident after using Alien Rescue successfully. In confirming the findings from the quantitative data, the qualitative data revealed why the students thought they understood science better. Some of the frequently cited reasons included: 1) students can use the tools provided in the program to look up the information, 2) they feel like being a scientist and are charged to solve a challenging problem, 3) they can learn on their own, and 4) Alien Rescue offers a different and interesting way of learning. Below are a few sample responses from the students to the question: "When working with Alien Rescue, do you feel that you can understand science better? Why or why not?"

Yes, because I can click on what I want to know instead of learning in a particular order.

Yes, because it made me feel like a scientist and as if I didn't have a teacher but learn everything on my own.

Yes, because Alien Rescue explains it more deeply so I can understand it more easily.

A student explained in the interview: "It's fun to work on your own and try to experience the real world experience of not having a teacher always by your side telling you what to do. Science is one of my best subjects and I've always liked science. With the Alien Rescue program, I got to see another side of science by seeing how real scientists work to get answers. It is very different from reading off a textbook. It's very interesting. It does make me want to know more about the solar system."

Attitude. Some students mentioned that they liked science more after using Alien Rescue. In addition to the reasons discussed above, they stated that they liked the hands-on activities provided by Alien Rescue, and that they had the chance to work on computers and in groups. Almost all the students stated that Alien Rescue made learning science fun. The quantitative data indicated that there was not a statistically significant increase in students' attitudes toward science from pretest to posttest and students' attitude was not a predictor of achievement. This is perhaps due to the reason that many students already liked science before being introduced to Alien Rescue, and all the students interviewed, whether they had a high or low achievement score or high or low attitude score, liked learning science using Alien Rescue and considered the program fun. This seemed to confirm the ceiling effect as shown in the quantitative data. A few students said their attitude toward science had not changed because they knew they would be going back to textbook learning once the unit was over. One student wrote, "I don't like science more, I just like this unit because it was enjoyable, but I know I still have to learn other things from the textbook. When I think about it, I feel sad." Although there was not a significant shift in their attitude toward science from before to after using Alien Rescue, all students appeared to enjoy the experience with Alien Rescue.

Self-efficacy. In support of the findings from the quantitative data, students said they were confident in learning science successfully after using Alien Rescue and mentioned that the program provided them the necessary tools to find the information and help them learn. One student said, "... it boosts my creativity, makes me want to know more because I know I can find more information." Another responded, "Only if science is always going to be like this. I don't know if I can do well on textbook now, maybe I can, but I really just want to learn using the computer and working with my friends. I will have more time to research on things and then I will be confident I know the information." Another student stated that "... because I know that if I can learn all that I did without teacher, I can learn even more with a teacher."

It is worth noting that all the students interviewed, whether they had a high or low post-achievement score, said they felt confident in learning science successfully using Alien Rescue because they could solve the problem on their own, rather than relying on their teacher. Some students said they were frustrated at first because they did not know what to do and no teacher was there to give answers. However, they became more confident about learning science successfully in the PBL environment after they learned to use the computer tools embedded in the program to help them solve the problem. One student said in the interview, "I wasn't too confident at first when I knew I had to work on my own without the teacher, but now I'm more comfortable because I have already explored the program. I'm not worried about not finding answers. I will just try harder and if I still don't get it, I will ask my group members and they will help me out. My partner and I think very differently but we try to work things out and find what is best for the aliens." Such responses further illustrated the values of a PBL approach and support our hypothesis that building computer tools in PBL provides necessary scaffolding for sixth-graders to enjoy PBL benefits. A few students, however, stated their confidence level in being successful in learning science stayed the same because 1) they knew Alien Rescue was only one unit and they would go back to textbooks, and 2) they already knew quite a bit about science and they already have high self-efficacy beliefs about learning science. For a small number of students in this study, a positive experience of how science is taught in the classrooms did not appear to increase their attitude toward science as a subject or science self-efficacy beliefs. More research is needed to investigate if students' attitude or self-efficacy would change when they are exposed to more programs like Alien Rescue in their curriculum.

SUMMARY

Although there was no control group in this study, the findings, based upon both quantitative and qualitative data, showed that Alien Rescue was effective in assisting students' learning. The computer-enhanced PBL environment presented students an opportunity to have more autonomy and responsibility for their own learning processes. Computer tools embedded in the environment assisted students in solving problems. Such an environment, though complex, offers an exciting opportunity for students to challenge themselves, develop their self-efficacy, and find learning relevant and meaningful. As a result, students had a better understanding of science concepts and felt more confident about being a successful learner after they solved a challenging problem.

In this study, we examined the effect of the computer-enhanced PBL environment on sixth-graders' self-efficacy, attitudes, and the knowledge they had gained about the solar system. Using authentic assessment is an important aspect of PBL instruction. In our research reported elsewhere (Liu & Bera, 2005; Liu et al., 2004), we assessed the problem-solving skills sixth-graders had developed by examining the problem solutions and rationales they provided. Because of our interest in motivational issues, our next step is to investigate if students' goal orientation, a related factor in students' motivation and achievement, is influenced by the PBL environment and if the goal orientation may relate to their self-efficacy beliefs and achievement when they are engaged in a computer PBL environment. Research on such motivational factors can help us better understand what motivates students to learn, how the motivational variables can jointly affect students' science learning, and how to design effective learning environments to assist students in learning.

References

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman.

Barrows, H. S. (1996). Problem-based learning in medicine and beyond: A brief overview. New Directions for Teaching and Learning, 68, 3-12.

Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.) (2000). How people learn: Brain, mind, experience, and school: Expanded Edition. Washington, DC: National Academy Press.

Cobb, P., & Bowles, J. (1999). Cognitive and situated learning perspectives in theory and practice. Educational Researcher, 28, 4-15.

Cognition and Technology Group at Vanderbilt. (1997). The Jasper Project: Lessons in Curriculum, Instruction, Assessment, and Professional Development. Mahwah, NJ: Erlbaum.

Corderoy, R., Harper, B., & Hedberg, J. (2003). Designing simulations that support improved learning outcomes in water catchment management. Proceedings of the World Conference on Educational Multimedia & Hypermedia, USA, 2767-2775.

Ebenezer, J., & Zoller, U. (1993). Grade 10 students' perceptions of and attitudes toward science teaching and school science. Journal of Research in Science Teaching, 30, 175-186.

Farnsworth, C. (1994). Using computer simulations in problem-based learning. In M. Orey, (Ed.), Proceedings of the Thirty-Fifth ADCIS Conference (pp. 137-140). Nashville, TN: Omni Press.

Gallagher, S. A., Stepien, W. J., & Rosenthal, H. (1994). The effects of problem-based learning on problem solving. Gifted Child Quarterly, 36, 195-200.

Germann, P. J. (1988). Development of the attitude toward science in school assessment and its use to investigate the relationship between science achievement and attitude toward science in school. Journal of Research in Science Teaching, 25, 680-703.

Hmelo, C. E., & Ferrari, M. (1997). The problem-based learning tutorial: Cultivating higher order thinking skills. Journal for the Education of the Gifted, 20, 401-422.

Hoffman, B., & Richie, D. (1997). Using multimedia to overcome the problems with problem based learning. Instructional Science, 25, 97-115.

Lajoie, S. P. (1993). Computer environments as cognitive tools for enhancing learning. In S. P. Lajoie and S. J. Derry (Eds.), Computers as cognitive tools (pp.261-288). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Lane, J., & Lane, A. (2001). Self-efficacy and academic performance. Social Behavior and Personality, 29, 687-694.

Liu, M. (2004). Examining the performance and attitudes of sixth-graders during their use of a problem-based hypermedia learning environment. Computers in Human Behavior, 20, 357-379.

Liu, M (2005). The effect of a hypermedia learning environment on middle school students' motivation, attitude, and science knowledge. Computers in the Schools, 22(3/4), 159-171.

Liu, M, & Bera, S. (2005). An analysis of cognitive tool use patterns in a hypermedia learning environment. Educational Technology Research and Development, 53(1), 5-21.

Liu, M., Bera, S., Corliss, S., Svinicki, M., & Beth, A. (2004). Understanding the connection between cognitive tool use and cognitive processes as used by sixth-graders in a problem-based hypermedia learning environment. Journal of Educational Computing Research. 31(3), 309-334.

Liu, M., Williams, D., & Pedersen, S. (2002). Alien Rescue: A problem-based hypermedia learning environment for middle school science. Journal of Educational Technology Systems 30, 255-270.

Loyd, B. H., & Gressard, C. (1984). Reliability and factorial validity of computer attitude scales. Educational and Psychological Measurement, 44, 501-505.

Lunyk-Child, O. L, Crooks, D., Ellis, P.J., Ofosu, C., O'Mara, L., & Rideout, E. (2001). Self-directed learning: Faculty and student perceptions. Journal of Nursing Education, 40, 116-123.

Mattern, N., & Schau, C. (2002). Gender differences in science attitude-achievement relationships over time among white middle-school students. Journal of Research in Science Teaching, 39, 324-340.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). Thousand Oaks, CA: Sage Publications.

Norman, G. R., & Schmidt, H. G. (1992). The psychological basis of problem-based learning: A review of the evidence. Academic Medicine, 67, 557-565.

Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implication. International Journal of Science Education, 25, 1049-1079.

Pajares, F., & Johnson, M. J. (1996). Self-efficacy beliefs in the writing of high school students: A path analysis. Psychology in the Schools, 33, 163-175.

Pajares, F., & Miller, M. (1994). Role of self-efficacy and self-concept beliefs in mathematical problem solving: A path analysis. Journal of Educational Psychology, 86, 193-203.

Pajares, F., & Miller, M. (1995). Mathematics self-efficacy and mathematics outcomes: The need for specificity of assessment. Journal of Counseling Psychology, 42, 190-198.

Pintrich, P. R., Smith, D. A., Garcia, T, & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801-813.

Savery, J. R., & Duffy, T. M. (1995). Problem based learning: An instructional model and its constructivist framework. Educational Technology, 35, 31-38.

Schunk, D. H. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26, 207-232.

Schunk, D. H. (1995). Self-efficacy and education and instruction. In J. E. Maddux (Eds.), Theory, research, and application (pp. 281-303). New York: Plenum Press.

Stepien, W. J., Gallagher, S. A., & Workman, D. (1993). Problem-based learning for traditional and interdisciplinary classrooms. Journal for the Education of the Gifted, 16, 338-357.

MIN LIU

The University of Texas at Austin, USA

[e-mail]

PEGGY (PEI-HSUAN) HSIEH

The University of Texas at San Antonio, USA

[e-mail]

YOONJUNG CHO 

Washington State University, USA

[e-mail]

DIANE L. SCHALLERT

The University of Texas at Austin, USA

[e-mail]

Table 1 Means and Standard Deviations for the Science Achievement Test,
Self-Efficacy, and Attitudes Toward Science from Pretest to Posttest

               Science Achievement                            Attitude
               Test (% correct           Self-efficacy        (Scale of
               from 0-100) Mean          (Scale of 1-5)       1-5) Mean
          n    (SD)                 n    Mean (SD)       n    (SD)

Pretest   464  47 (15.94)           464  3.94 (.73)      464  3.78 (.79)
Posttest  464  70.52 (a) (18.09)    464  4.06 (a) (.72)  464  3.85 (.86)

(a) Posttest significantly different from pretest, p<.001 (partial Eta
Squared for Science Test and Self-efficacy = .72, and .03 respectively.
Non-matching measures were dropped in the analyses).

Table 2 Means and Standard Deviations for the Science Achievement Test
for High and Low Self-Efficacy and Attitude Groups

                                       Low Attitude        High Attitude
                                       Group Mean          Group Mean
                                  n    (SD)           n    (SD)

Pretest       Low Self-efficacy   184  43.82 (15.16)   72  44.28 (13.37)
                Group
              High Self-efficacy   58  48.83 (16.27)  144  52.72 (16.46)
                Group
Posttest (a)  Low Self-efficacy   179  66.10 (19.10)   80  73.51 (15.69)
                Group
              High Self-efficacy   49  76.86 (16.13)  150  73.06 (17.27)
                Group

(a) the interaction was significant at p < .01 (partial Eta Squared =
.02). Non-matching measures were dropped in the analyses.