Enhancing conceptual learning through computer-based applets: the effectiveness and implications.

Several Canadian universities and colleges have been working together for years to build Web-accessible computer-based applets to help students learn physics concepts. This paper reports the findings from a study that evaluated the effectiveness of these applets in enhancing conceptual learning. We integrated quantitative and qualitative methods including tests, surveys, focus groups, interviews, and class observations. The data show that the computer-based applets, which were designed in the light of constructivism, were helpful in fostering conceptual learning, but they should be used in a constructivist teaching environment to be more effective. In addition, based on this study, some suggestions will be given on the use of instructional technology in teacher education.

**********

INTRODUCTION

Conceptual learning has been considered an important topic; much effort has been made to improve physics instruction and to thus help students to better understand concepts (Brouwer, 1995; Redish, Saul, & Steinberg, 1997; Redish & Steinberg, 1999; Thornton & Sokoloff, 1990). Since the spring of 1998, several Canadian universities and colleges have been working together on an ongoing project, known as the "Modular Approach to Physics (MAP)," to help students learn physics concepts. A set of Web-accessible applets has been built. Each applet is focused on one key concept of university first-year physics, and is made up of one or more than one simulations designed in the light of constructivism.

The evaluation of the MAP applets started in 1999 with a pilot test. More formal investigations followed in 2000. This paper focuses on our findings on two questions: (1) Are the MAP applets effective in enhancing conceptual learning? (2) In what teaching environment are these applets more effective? In addition, we also discuss some issues related to teacher education based on our research findings.

CONSTRUCTIVIST APPROACH TO SCIENCE TEACHING

Constructivists take knowledge as a construction by human beings rather than as an absolutely true representation of the world; therefore, they believe that effective learning should be an active process during which students construct and reconstruct the knowledge (von Glasersfeld, 1995). This new interpretation of learning asks us to make a shift in instruction from "what are we teaching and how can we deliver it?" to "what are the students learning and how do we make sense of what they do?" (Redish & Steinberg, 1999)

In the last three decades, intensive studies in science education have demonstrated that students come to school with their own understandings of the world, which greatly influence their learning (Driver, Guesne, & Tiberghien, 1985). Although, in some instances, student preconceptions are in keeping with scientific ones, most often there are huge differences between children's notions and school science. Therefore, how to facilitate the movement of students from intuitive understanding to scientific understanding becomes a key question for science educators.

The ontological study of student preconceptions tells us that many students' preconceptions are reminiscent of well known concepts in the history of science. For example, "a heavier body falls faster than a lighter one," "force can be given to an object under the name of 'impetus," and "force is needed to maintain motion," are conceptions held by people and scientists in pre-Newtonian times and are well documented in the studies of students' preconceptions (Berg & Brouwer, 1991; Clement, 1982; Driver & Easley, 1978; McCloskey, 1983). "Heat is a kind of material that can flow from one object to another" (Erickson, 1979, 1980). This preconception reminds us of scientists' caloric view of heat in the 19th century. The parallelism between the history of science and the psychological process does not exist only at this content level. Scholars are trying to convince us that there is also a parallelism concerning the general features of the process of knowledge acquisition in the history of science and by students. Gopnik (1996) claimed that the fundamental cognitive processes are the same in students and mature scientists. Siegel (1995) further maintained that students like scientists need to assess their own ideas before any new view can be accepted.

The hypothesis of a parallelism between the process of student learning and the development of science has been a fruitful source of pedagogical study. Paralleling the conditions suggested by Kuhn for scientific revolutions (Kuhn, 1970), Posner, Strike, Hewson, and Gertzog (1982) state that several important conditions must be fulfilled before any conceptual change occurs. These conditions can be briefly described in terms of students' dissatisfaction with the old conception and the intelligibility, plausibility, and fruitfulness of the new conception. Although Posner's model is criticized as a 'cool' or 'isolated' model because it ignores the social dimension and the non-rational aspect of learning (Pintrich, Marx, & Boyle, 1993), it correctly addresses the importance of cognitive conflict in conceptual change. Many instructional strategies proposed to address students' preconceptions, such as "Prediction-Observation-Explanation," share a common feature--they confront students with discrepant events that contradict their conceptions (Scott, Asoko, & Driver, 1992). This is intended to invoke cognitive disequilibration that induces students to reconstruct knowledge. Discrepant events can be demonstrations that require students to make predictions or phenomena that require students to explain why and how.

Computer simulations can also be used to provide such discrepant events. To have this function, computer simulations should be able to allow students to interact with the process, make predictions, and test their predictions. With these kinds of simulation, students can freely explore the microworld the program creates by changing parameters and variables; they can immediately visualize the consequence of their manipulations.

There are not many studies that have been done primarily about the effectiveness of computer simulations on conceptual change, and the reported studies have drawn an unclear picture on this topic. For example, Zietsman and Hewson (1986) used a computer simulation to diagnose and remedy alternative conceptions about velocity. Their results indicate that computer simulations can be a credible representation of reality and that the use of simulation produces significant conceptual change in students' alternative conceptions. Carlen and Andre (1992), however, in a study on electrical circuits, found that using text designed to produce conceptual change resulted in better performance on tests, but that using a computer simulation in addition to the text produced no greater change than the text alone.

We assume that the effectiveness of computer simulations is determined by two factors: the simulation design and the context in which simulations are used. In this study, we have used our constructivism-directed simulation project to investigate whether simulations were effective in fostering conceptual change and in what context they were effective.

MODULAR APPROACH TO PHYSICS--SIMULATIONS FOR ENHANCING CONCEPTUAL CHANGE

The MAP applets are Web-accessible computer simulations with a very brief tutorial as the context. They were developed to facilitate conceptual teaching and learning. Each applet is focused on one key concept of the university first year physics curriculum and is designed to address relevant preconceptions that students hold. Applets are relatively independent. Teachers can pick up and use whatever applets they need for their teaching. Students can access applets both in class and after class. This is why the project is named the "Modular Approach to Physics." The MAP project is different from many other computer-based instruction (CBI) programs because the underlying principle of the MAP project is constructivism, instead of instructionism. Many CBI programs use the technology as an alternative way of presenting information but show little advantage in their fundamental educational principles compared with conventional text-based materials. Their simulations are primarily used to present experiences that are not easy to understand; in addition, they use a design that often fails to reflect insights of cognitive science. Salomon, Perkins, and Globerson (1991) critically comment on this point, "No important impact can be expected when the same old activity is carried out with a technology that makes it a bit faster or easier; the activity itself has to change." (p. 8). In contrast, the MAP project integrates the constructivist instructional approach into the design of the simulations and emphasizes the interaction between learners and the computer. Simulations are designed to allow students to input and test their predictions, and students can try more than one prediction by working individually or collaboratively. The following is one example of the MAP applets.

The computer simulates a ball rolling down an incline and then moving on a horizontal table (Figure 1). Students are prompted to draw arrows to indicate the velocity of the ball at three points along the path. Most students will draw the velocity arrows correctly (the lighter arrows in Figure 1). From the literature, however, we know that many students have trouble in the next step--drawing arrows to indicate the acceleration of the ball. Students believe that a high velocity necessarily implies a big acceleration, and they will draw a longer arrow at point 2 than at point 1 (the darker arrows in Figure 1). After students input their ideas (draw the acceleration arrows), they can test their ideas by clicking the "view results" button. The system will generate virtual processes based on students' ideas. The virtual process is shown on the screen beside the realistic one for comparison (in Figure 1, the darker ball represents students' prediction, and the ghost ball represents reality). This ability for students to visually compare the outcome of their predictions with the realistic process can be helpful in creating cognitive conflict and facilitating conceptual change. Traditional laboratory experiments cannot as easily give students this ability to see the results of their predictions. The applet allows students to make a number of different choices, but only the correct choices will duplicate the realistic motion.

[FIGURE 1 OMITTED]

THE IMPLEMENTATION OF EVALUATION

Sample

The evaluation of the MAP applets started several years ago with a pilot test (Zhou, Martin, Brouwer, & Austen, 2000). More formal investigations followed in three terms of 2000: the winter, spring, and fall terms. In total, nine classes, involving over 700 students, took part in the investigation. Participating students in the winter and spring terms were from the seven classes of a university algebra-based, first-year physics course. Those in the fall term were from the two classes of a university calculus-based, first-year physics course. These two courses covered the same topics: kinematics, dynamics, and heat, but the calculus-based course was more challenging. The algebra-based course was generally set up for the first- or second-year university students from different science-related departments while the calculus-based course was limited to the physics and chemistry students. Students registered in the calculus-based course had an overall better background in mathematics than those enrolled in the algebra-based course.

In the winter term, four classes were assigned as control classes (WC1, WC2, WC3, WC4), and two classes were treatment classes whose instructors and students used the MAP applets (WT1, WT2). In the spring term, there was one treatment class (ST), but no control class. In the fall term, there were tow classes--one control class (FC) and one treatment class (FT) (Table 1).

Methods and Instruments

We administered conceptual understanding tests to all the classes at the beginning and end of each course. The instrument we used for the conceptual tests was a combination of the Force Concept Inventory (FCI), which has 30 conceptual questions, and three more conceptual questions from the literature. Therefore, we refer to our instrument as FCI-Plus. The first version of FCI was published by Hestenes, Wells, and Swackhammer (1992), and a slightly modified version was published by Mazur (1997). FCI was designed to be a test of students' conceptual understanding of Newtonian mechanics. One of its outstanding features is that the questions are designed to explore the understanding of basic concepts in a way that is understandable to the novice who has never taken a physics course, while at the same time being rigorous enough for people who have taken a physics course. FCI has been widely and successfully used for testing the effectiveness of physics classes (Hake, 1998; Redish et al., 1997; Redish & Steinberg, 1999). From the results of the pilot trial we did in evaluating MAP applets, we found that students had preconceptions about the independence of components of motion and about gravity in space, which the FCI did not cover. We therefore added three questions into the FCI, one for motion independence and two for gravity. These three questions were taken from literature studying student preconceptions and were of a conceptual nature (Berg & Brouwer, 1991; Whitaker, 1983). Because all the questions in FCI-Plus were developed and widely used by university instructors and researchers, the validity of FCI-Plus in testing students' conceptual understanding was not a problem. For the reliability of FCI-Plus, we used the Odd-Even method to estimate the reliability coefficient, which resulted in 0.90 for both classes in the fall term. We used another method, Kuder-Richardson 20, and obtained a result of 0.89. The index of reliability, which is defined as the correlation between observed scores and true scores, is 0.95. These results confirm that FCI-Plus is a highly reliable test.

We administered a survey to the treatment classes at the end of each course to collect information about students' attitudes toward MAP and the frequency with which they accessed MAP after class. We carried out real-time class observations to collect information about the teacher's class design, applet use, interaction with students, and students' engagement in the class. We interviewed students and instructors individually to understand their attitudes and ideas about the design and the use of the MAP applets. We also held focus group discussions after each course and listened to students' expressions of appreciation or criticism of various components of the teaching of physics.

DATA ANALYSIS

g Values

To investigate and compare the effectiveness of instructional methods, we need a measurable and comparable variable primarily determined by the studied instructional methods. The existence of this kind of variable would allow us to study instructional methods in a wide range of class levels. Hake (1998) did a detailed study of FCI results by investigating 62 introductory physics courses involving over 6,000 students at school, college, and university levels. Based on this longitudinal study, He constructed a statistical measure: normalized gain, which he called g,

g = [absolute gain]/[maximum possible gain] = [[bar.X.sub.post] - [bar.X.sub.pre]]/[100 - [bar.X.sub.pre]]

where the absolute gain is equal to the difference between the post-test mean ([bar.X.sub.pre]) and the pre-test mean ([bar.X.sub.post]) of the class, and the maximum possible gain is equal to the gap between the full mark of the test (assumed 100) and the pre-test mean.

[FIGURE 2 OMITTED]

The g value remained roughly the same for the classes with a similar instructional approach regardless of the various pre-test means (Figure 2). It is equal to the absolute value of the slope of the line graph. Hake calculated the correlation coefficient of g with the pre-test score and got a very low value (0.02). In contrast, the correlation coefficient of the post-test score and the pre-test score was 0.55, and the correlation coefficient of the absolute gain and the pre-test score was -0.49. Clearly, compared with the post-test score and the absolute gain, the ratio g was more suitable for comparing course effectiveness over diverse groups with a wide range of initial levels. Hake (1998) therefore inferred that "a consistent analysis over diverse student populations in high schools, colleges, universities is obtained if a rough measure of the average effectiveness of a course in promoting conceptual understanding is taken to be the average normalized gain <g>" (p. 64).

Our test results for the winter and spring terms are reported in the Table 2. The test results for the fall term are reported in the Table 3. Each table lists the class, pre-test mean, post-test mean, and the g value for each class.

The data demonstrate that every treatment class produced a bigger g than the control classes. Among the four treatment classes, the two in the spring and fall terms did much better than the other two in the winter term. A possible explanation will be explored later.

t Test Results

We used t test to investigate the mean difference between the treatment and control groups. The significance level was set at 0.05 for every term. For the winter term, the mean difference between the treatment and control groups was not significant either at the beginning or the end of the course. That is, the two groups were at a similar level in terms of conceptual understanding when they started the course and when they finished the course. The use of the MAP applets did not result in a significant difference for the treatment group compared with the control group. Since there was no control class in the spring term, we took the control group in the winter term (WCw) as a reference group. The t test result on the pre-test was not significant, but the t test result on the post-test told us that the spring treatment class produced a significantly better mean than the winter control group. For the fall term, the mean difference between the treatment and control classes was not significant for the pre-test but was significant for the post-test. That is, the treatment and control classes in the fall term were at the same level in physics conceptual understanding at the beginning of the course, but they had a significant difference in conceptual understanding after the course. The treatment class performed significantly better on the FCI-Plus than the control class.

DISCUSSION

MAP Applets Were Effective in Enhancing Conceptual Learning

As Tables 2 and 3 demonstrate, the treatment classes created larger g values than the control classes. We therefore concluded that the MAP approach successfully helped students learn physics concepts. The t test results also support this conclusion. For the spring and fall terms, the treatment class produced a significantly higher mean in the post-test than the control class, even though the treatment and control classes were at the same conceptual understanding level in the pre-test. Although the t test results for the winter term test did not show a significant difference between the treatment and control groups in the post-test at an alpha level of 0.05, some difference did exist between these two groups. The pre-test mean of the treatment group was lower than that of the control group, but the order was reversed in the post-test.

The results must be interpreted with care, however, since the treatment classes had different instructors than the control classes. The difference might arise simply from the presence of the different instructors. However, the five instructors of the control classes were assistant, associate, or full professors. They have collectively had many years of teaching experience. In contrast, the instructors of the treatment classes in the winter and fall terms were postdoctoral fellows who had only one or two years of teaching experience. These teachers' lack of teaching experience did not stop their classes from doing better. Hence, we feel that the improvement in conceptual understanding was mainly due to the treatment.

To further investigate this matter, within the fall term treatment class, we studied the relationship between the personal normalized gain (personal absolute gain over personal maximum possible gain) in the conceptual tests and the students' frequency of accessing the MAP applets or the number of MAP applets they visited after class. We obtained positive correlation coefficients, although not large: 0.286 for the conceptual gain and the accessing frequency, and 0.316 for the conceptual gain and the number of visited applets. The positive correlation indicated that students who accessed the MAP applets more frequently tended to have a higher conceptual gain.

The survey results provided further evidence for our argument, since most treatment students reported in the survey that the MAP applets helped them learn physics concepts (Figure 3). In the winter term, 46% of the treatment students reported that the MAP applets significantly helped them learn concepts; 27% reported that the MAP applets helped them a little bit; 16% did not gain benefits from the MAP applets at all; and the remaining 11% were uncommitted. In the spring term, 96% of the treatment students reported that the MAP applets significantly helped them to learn concepts; 4% said that the MAP applets helped only a little bit; there were no students who found no benefit to using the MAP applets; and no students were uncommitted. In the fall term, 52% of the treatment students reported the MAP applets helped them understand concepts a lot; 17% said the MAP applets helped them only a little bit; 7% did not derive benefits from the MAP applets at all; and the remaining 24% were uncommitted. During the interviews and focus group discussions, we heard supporting comments from students as well: "The applet helps. It allows you to see what is going on," "Anything visual, in my opinion, helps," and "The applets help us do the transformation [from the physical situation to physics language]."

[FIGURE 3 OMITTED]

MAP Applets Were More Effective in a Constructivist Teaching Environment

The treatment class in the spring term produced a g value over two times larger than the two treatment classes in the winter term and almost all students felt MAP was helpful. This dramatic difference leads us to question why this happened. To answer this question, we need to examine the instructional processes of the treatment classes. Based on class observations, the main differences that could be detected between the spring treatment class and the winter treatment classes were found in the teaching strategy and in the way of integrating the MAP applets into the lectures. The treatment class instructor in the spring term received his doctoral degree in science education, and he was quite familiar with the student preconception theory. While the winter treatment classes generally had a traditional format of instruction, which stressed problem solving, the spring instructor could explicitly address the preconception issues and promote student involvement in the construction of concepts. The interaction between the spring instructor and the students was more frequent and effective. In one of the winter treatment classes, for instance, the instructor asked students to predict which direction the ball would go after cutting the string. When the first responding student gave the right answer, the instructor moved ahead right away. It would appear that for him the purpose of questioning students was to get a right answer rather than stimulating students' understanding. For the same question, however, the spring instructor waited for a while after a student gave the correct answer. He questioned more students until most of the possible student concepts were addressed, and then he worked on these students' concepts one-by-one.

The treatment class instructor in the spring term was a member of the MAP team. He understood the design and purpose of each applet. He used applets in a quite different way from the two treatment instructors in the winter term. The two winter term instructors used applets as new information-presenting tools. Students saw that the applets simulated, but they were not encouraged to make predictions. Applets were primarily used as examples of knowledge application. In contrast, the spring instructor could smoothly orchestrate applets. He quite often used applets to set the stage for new learning and involved students in thinking.

One of the two treatment instructors in the winter term was also the treatment instructor in the subsequent fall term. Exploring his two sessions is even more informative for us to understand the context under which MAP applets were more effective. In the winter term test, his class produced no significantly different result compared with the control classes, but in the fall term test, his class did significantly better than the control class. To investigate the reason for the improvement, one of the authors (Zhou) interviewed this teacher at the end of the fall term. One part of the recorded dialogue went like this:

Zhou: In the Winter term, your class did better than control classes, but the difference was not significant. In this Fall term, however; the difference [between your class and the control class] turned out to be significant. Can you figure out any possible reasons for these different results?

Teacher: I had better students in this Fall term. They were highly motivated. They knew they would use what they were learning.

Zhou: You are right. But we compared your classes with other classes from the same background. In the Winter term, we compared your algebra-based class with other algebra-based classes. In the Fall term, we compared your calculus-based class with another calculus-based class.

Teacher: I mean, better students can learn more. I put the MAP site onto the course web site. Students could access them anytime they wanted to.

Zhou: You mean better students will get more when we supply them with a chance?

Teacher: Yes.

Zhou: Do you think there is any difference in the way you used MAP in the Fall term compared with the Winter term?

Teacher: Of course. When you first use it, you do not know what it is for. I used applets more often at the beginning of the class in this [Fall] term. If I used applets after the theory, students would not pay attention to them. They already knew what would happen.

Zhou: They are not curious anymore.

Teacher: No.

He was honest when he described his different ways of using MAP applets in two terms. According to our class observation records, in the winter term he taught new concepts in a traditional way and used applets after he taught the concept. On the other hand, in the fall term he quite often used applets in the exploratory stage of conceptual construction. Here is an example to show the different ways he used applets. A friction applet was developed to facilitate teaching and learning such concepts as static friction, maximum static friction, and kinetic friction (Figure 4). The instructor or students can use the computer mouse to pull the force probe. At the beginning, the probe has a reading, but the books do not move because of static friction. When students pull harder and the reading increases to one point (maximum static friction), the pile of books starts to move, and suddenly the probe's reading drops down to a smaller value (kinetic friction). Students can turn on the free body diagram. This diagram allows students to see the change of friction during the whole dynamic process (Figure 5). By increasing or decreasing the number of books, the applet can also be used to investigate the relationship between the friction and the normal force. This applet proved to be an excellent aid for the construction of students' understanding of friction. In the fall term, the teacher launched this applet at the very beginning when he taught friction. He successfully used this applet to echo students' life experiences with friction such as pulling a table on a floor, to visualize the force change in the process, and to offer students detailed guidelines for constructing relevant concepts and knowledge about friction. On the other hand, in the winter term this teacher used the same applet to verify what he taught about friction.

[FIGURE 4 OMITTED]

[FIGURE 5 OMITTED]

The above analysis leads us to conclude that MAP applets are more functional in a constructivist teaching environment. They work best only when they are included in the right spots in the process of teaching and learning. It is no wonder that the same project can produce different results when used by different teachers. They teach using a different style and use applets in a different way.

SUGGESTIONS FOR TEACHER EDUCATION

When we sat in the classroom and observed the communication between the teacher and students, we often felt that some of our university teachers might not be ready to teach although they might know a great deal about the subject matter. They seemingly did not know how to encourage students to participate in the class; they did not pay enough attention to what students learned from the class; and they did not show enough enthusiasm for the new instructional technology. Based on the evaluation results, our class observations, and our communications with teachers and students, we will provide suggestions for the education of today's college teacher in two areas: more pedagogical knowledge, and more interest in the appropriate use of the new instructional technology.

More Pedagogical Knowledge

In Alberta, Canada, students must receive a teaching certificate before they can teach in schools. To get this certificate, students need to have a minimum of 45 credits of course work from a faculty of education. However, people can become college or university lecturers without any training in education. Hence, some university teachers hold high academic degrees in a subject area, but they do not know much about what is going on in cognitive or pedagogical studies. Teaching without pedagogical knowledge quite often limits instructors to rely only on their personal learning experience. Redish (1994) reminded us of the danger of this kind of teaching: "Our (physics teachers') own personal experiences may be a very poor guide for telling us what to do for our students" (p.10). Prospective university physics teachers or researchers will normally obtain more than 10 years of special training in physics and make up a very small number in our introductory physics classes. The experiences of a small group with intensive special training could not tell us much about the learning of students who make up the larger part of the class. Redish cited his own story as a footnote to his argument:

    I will never forget one day a few years ago when a student in my
    algebra-based introductory physics class came in to ask about some
    motion problems. I said: "All right, let's get down to absolute
    basics. Let's draw a graph." The student's face fell, and I realized
    suddenly that a graph was not going to help him at all. I also
    realized that it was going to be hard for me to think without a
    graph and to understand what was going through the student's mind.
    (p. 10)

In this study, we found that the teaching perspectives of those teachers without training in education may lag very far behind compared with the research findings in education. One day, on the way back from the class to the office, the following dialogue occurred between one of the treatment instructors and one of the authors (Zhou):

Teacher: Is it boring? It is old stuff for you. [The teacher asked Zhou who observed his class]

Zhou: No. You know I am not learning physics now. I am learning how students learn and how teachers teach, especially on concepts.

Teacher: Unfortunately, not much is on concepts for this course; lots on problem solving.

This informal talk clearly exposed the teacher's perspective on conceptual learning and teaching. His comment was quite far removed from the research findings that tell us that students have great difficulty in learning concepts of kinematics, dynamics, and heat. We could not expect him to put much emphasis on concept teaching as intensive studies have suggested.

More Interest in the New Instructional Technology

New instructional technology, especially computer-based multimedia, has been stepping into the classroom. However, teachers tend to teach the way they were taught. This fundamental resistance to new things stops teachers from fully exploring the function of new approaches. The treatment instructor in the spring term was one member of the MAP team, while the two treatment instructors in the winter term had no connection with MAP before they used the MAP applets in their classes for the first time. Therefore, the difference between the spring treatment instructor and the winter treatment instructors in attitudes toward the MAP applets was obvious and expected. Sitting in the class, we could clearly feel the passion the spring treatment instructor had for the MAP applets. Applets were an essential part of his lecture design. They were integrated with demonstrations, blackboard draft and drawing, and other components. In the cases in which the computer crashed and applets could not be down loaded, the teacher showed patience and tolerance with technology by commenting, "Well, sometimes technology has problems." We had a very different feeling in the treatment classes in the winter term. The instructors often failed to run applets correctly. They did not know when and how the applets were supposed to be used. They sometimes asked students to make predictions, but failed to show the difference between the prediction and the actual result. The applets were used as a kind of decoration in the traditional framework of instruction rather than as a part of the process leading to the construction of knowledge. In the cases in which the computer crashed, the instructor continued his teaching with a comment of "We'd better get rid of it."

Connecting the instructors' attitudes and students' attitudes toward the MAP applets is very informative for the understanding of how teachers can influence students in the new technology. In the survey, 88% of the treatment students in the spring term reported that they liked the MAP applets much; 12% liked them a little bit; and no student disliked them. In contrast, only 50% of the treatment students in the winter term reported that they liked the MAP applets much; 27% liked them modestly; and 9% did not like them at all. The remaining 14% were uncommitted. These percentages were structurally similar to the student-reported effectiveness of applets in terms of helping them learn concepts (Table 4). Obviously, the treatment students in the spring term liked the MAP applets more and benefited more from them than the treatment students in the winter term. Given the difference between the treatment instructor in the spring term and the treatment instructors in the winter term in terms of their attitudes toward MAP, we concluded that the teachers' attitudes could greatly impact students' attitudes, and subsequently impact students' learning. If we want students to benefit from the new technology, our teachers must like it first.

In Table 4, the treatment class in the fall term lies somewhere between the winter treatment classes and the spring treatment class in terms of students' attitudes toward the MAP applets and the level at which students benefited from them. This provides further evidence for our above argument considering that the appreciation of the fall treatment instructor toward the MAP applets improved with experience but was not yet as positive as the attitude of the spring instructor. As described in the previous section, the fall treatment instructor used the MAP applets for the second time in the fall term and used them more effectively than he did in the earlier winter term. It is reasonable to assume that he had a stronger appreciation of MAP in the fall term because of his prior experience with the applets. However, it is difficult for him to become as committed to MAP as the spring treatment instructor since he was not one of the MAP designers.

CONCLUSION

This study demonstrates that computer-based applets, designed in the light of constructivism, can be helpful in fostering conceptual change/learning, but they should be used in a constructivist teaching environment to be more effective. The effectiveness of computer applets is a function of the applet design, the instructional environment in which they are used, and the teacher's readiness for using new instructional technology. Well-designed new media applications must be used in a constructivist teaching environment by enthusiastic teachers to be effective. University science teachers, normally without a pedagogical background, need to become familiar with the teaching suggestions from cognitive studies and the way that new media works in order to be effective teachers.

This research result on the effectiveness of computer-based applets may have limitations because it is hard to screen out the factor of the instructor. A more reliable research design can be obtained through a random selection of treatment and control instructors from a larger pool of instructors, for example, 10 treatment class instructors and 10 control class instructors out of 20 available instructors. Of course, this kind of research design is hard to implement for many reasons, such as the availability of instructors and classes. One thing needs to be noted for similar studies in the future. The instructors who will try to implement new technological initiatives need training. In the winter term, we held a one-hour long workshop to introduce the MAP applets to the treatment instructors. The study told us that the workshop was too short. Overall, the two treatment instructors in the winter term were not successful in using the applets. We could not count on instructors investigating applets by themselves. They were too busy and lacked the associated pedagogical knowledge to know how applets should be used.

Table 1 Student Numbers in Each Class

Term                          Winter              Spring  Fall
Class               WC1  WC2  WC3  WC4  WT1  WT2  ST      FC  FT

Number of students  81   76   99   105  88   103  26      89  71

Table 2 The g Values for the Control and Treatment Classes (Winter and
Spring)

Term                               Winter
Class      WC1     WC2     WC3     WC4      WCw      WT1     WT2
           (n=81)  (n=76)  (n=99)  (n=105)  (n=361)  (n=88)  (n=103)

Pre mean   16.8    16.6    15.4    16.5     16.3     15.8    15.4
Post mean  18.3    17.6    16.6    18.1     17.6     18.4    18.0
g           0.093   0.064   0.067   0.095    0.078    0.151   0.148

Term       Winter   Spring
Class      WTw      ST
           (n=191)  (n=26)

Pre mean   15.6     16.1
Post mean  18.2     22.1
g           0.149    0.36

Note: WCw represents the whole control group in the winter term, WTw the
whole treatment group.

Table 3 The g Values for the Control and Treatment Classes (Fall)

Class      FC (n=89)  FT (n=71)

Pre mean   17.8       17.7
Post mean  20.5       23.0
g           0.176      0.345

Table 4 Structural Similarity Between Survey Results on How Students
Liked MAP and How Students Benefited From MAP

     Survey results on how students liked MAP (%)
     Much  A little bit  Not at all  Uncommitted

WTw  50    27             9          14
ST   88    12             0           0
FT   61     7             9          24

     Survey results on how students benefited from MAP (%)
     Much  A little bit  Not at all  Uncommitted

WTw  46    27            16          11
ST   96     4             0           0
FT   52    17             7          24

References

Berg, T., & Brouwer, W. (1991). Teacher awareness of student alternative conceptions about rotational motion and gravity. Journal of Research in Science Teaching, 28, 3-18.

Brouwer, W. (1995). Towards a more conceptual way of teaching physics. Physics in Canada, 51, 55-61.

Carlsen, D., & Andre T. (1992). Use of a microcomputer simulation and conceptual change text to overcome student preconceptions about electric circuits. Journal of Computer-based instruction, 19, 105-109.

Clement, J. (1982). Students' preconceptions in introductory mechanics. American Journal of Physics, 50, 66-71.

Driver, R., & Easley, J. (1978). Pupils and paradigms: A review of literature related to conceptual development in adolescent science students. Studies in Science Education, 5, 61-84.

Driver, R., Guesne, E., & Tiberghien, A. (1985). Children's ideas in science. Milton Keynes: Open University Press.

Erickson, G. L. (1979). Children's conceptions of heat and temperature. Science Education, 63, 221-230.

Erickson, G. L. (1980). Children's viewpoints of heat: A second look. Science Education, 64, 323-336.

Gopnik, A. (1996). The scientist as child. Philosophy of Science, 63, 485-514

Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64-74.

Hestenes, D., Wells, M., & Swackhammer, G. (1992). Force concept inventory. The Physics Teacher, 30, 141-158.

Kuhn, T (1970). The structure of scientific revolutions. Chicago University Press.

Mazur, E. (1997). Peer instruction: A user's manual. Englewood Cliffs, NJ: Prentice Hall.

McCloskey, M. (1983). Intuitive physics. Scientific American, 248, 114-122.

Pintrich, P. R., Marx, R. W., & Boyle, R. A. (1993). Beyond cold conceptual change: The role of motivational beliefs and classroom contextual factors in the process of conceptual change. Review of Educational Research, 63, 167-199.

Posner, G. J., Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change. Science Education, 66, 221-227.

Redish, E. F. (1994). Implications of cognitive studies for teaching physics. American Journal of Physics, 62, 796-803.

Redish, E. F., Saul, J. M., & Steinberg, R. N. (1997). On the effectiveness of active-engagement microcomputer-based laboratories. American Journal of Physics, 65, 45-54.

Redish, E. F., & Steinberg, R. N. (1999). Teaching physic: Figuring out what works. Physics Today, 52, 24-30.

Salomon, G., Perkins, D. N., & Globerson, T. (1991). Partners in cognition: Extending human intelligence with intelligent technologies. Education Research, 20(3), 2-9.

Scott, P. H., Asoko, H. M., & Driver, R. H. (1992). Teaching for conceptual change: a review of strategies. In R. Duit, F. Goldberg, & H. Niedderer (Eds.), Research in physics learning: Theoretical issues and empirical studies (pp. 310-329). Kiel: IPN.

Siegel, H. (1995). Why should educators care about argumentation? Informal Logic, 17(2), 159-176.

Thornton, R. K., & Sokoloff, D. R. (1990). Learning motion concepts using real-time microcomputer-based laboratory tools. American Journal of Physics, 58, 858-867.

von Glasersfeld, E. V. (1995). A constructivist approach to teaching. In L. P. Steffe & J. Gale (Eds.), Constructivism in education (pp. 3-15). Hillsdale, NJ: Lawrence Erlbaum Associate.

Whitaker, R. J. (1983). Preconceptions about trajectory motion. American Journal of Physics, 51, 352-357.

Zhou, G., Martin, B., Brouwer, W., & Austen, D. J. (2000). Computer-based physics and student preconceptions. Alberta Science Education Journal, 32(2), 23-35.

Zietsman, A. I., & Hewson, P. W. (1986). Effect of instruction using microcomputer simulations and conceptual change strategies on science learning. Journal of Research in Science Teaching, 23, 27-39.

GEORGE G. ZHOU, WYTZE BROUWER AND NORMA NOCENTE

University of Alberta, Canada

[e-mail]

[e-mail]

[e-mail]

BRIAN MARTIN 

King's University College, Canada

[e-mail]