Ability and critical thinking as predictors of change in students' psychological misconceptions.

Based on the conceptual change literature, this study assessed factors influencing change in students' misconceptions about psychology. We expected students' ability and their critical thinking to predict whether they would change their misconceptions following completion of an introductory psychology course. GPA, scores on a test of critical thinking, and scores on a misconceptions test given at the beginning and end of the course were obtained from 74 introductory psychology students. Analyses indicated that critical thinking made a unique contribution to the prediction of change in student misconceptions but that the effect of GPA was accounted for by its relation with critical thinking. The study suggests that misconceptions can change for students at any level of ability and are particularly likely to change for students who think critically.

**********

A variety of studies indicate both the prevalence of student misconceptions about psychology and the limited effect the introductory psychology course has in correcting these misconceptions (e.g.; Gardner & Dalsing, 1986; McKeachie, 1960; Vaughan, 1977). Although instructors are often surprised at the limited impact of their courses on students' beliefs, the finding is exactly what the conceptual change literature would predict.

Cognitive psychology suggests that learning involves the interpretation of new information based on existing knowledge or schemata. Prior knowledge that is congruent with new information can aid learning. Prior knowledge that is inconsistent with new information, however, can hamper learning (Guzzetti, 2000; Lipson, 1982). Conceptual change learning refers to the kind of learning that occurs when newly presented information is inconsistent with prior knowledge and the learner must reorganize existing schemata and change previously held ideas. This kind of knowledge reorganization is most likely when students are able to engage in effortful processing, evaluating old beliefs and comparing them with new intelligible, plausible, and more useful concepts (Dole & Sinatra, 1998: Posner, Strike, Hewson, & Gertzog, 1982). Because such evaluation involves effort, it is easier and often more likely for people to ignore, reinterpret, or reject competing beliefs rather than to reorganize their belief systems (Chinn & Brewer, 1993). As a result, changing misconceptions is difficult, and students often leave a class believing what they did when they entered. This resistance to change can be particularly prevalent among students with below average ability, who may be less able to comprehend the new information (Guzzetti, 2000), and students with poor metacognitive skills who may be less able to detect inconsistencies between the old and the new information (Dole & Sinatra, 1998: Maki, 1998; Otero, 1998).

Despite several similarities between this conceptual change literature and the studies conducted to date on psychological misconceptions, the link between the two lines of research has never been made. As noted previously, students come into the introductory course with many misconceptions about psychology. Across a variety of studies, scores on tests of misconceptions given at the beginning of a course range from 28% correct (McCutcheon, 1991) to 62% correct (Gardner & Dalsing, 1986). Psychology students also leave their courses having changed few of their original misconceptions. Some researchers report as little as 5.5% improvement in students' scores (Gardner & Dalsing, 1986; McKeachie, 1960: Vaughan, 1977), although an improvement of 25% to 50% is reported when items are explicitly discussed in class (Taylor & Kowalski, 2004; Vaughan, 1977). Even after several psychology courses, 30% of the students still believe someone experiencing schizophrenia demonstrates a "split personality" (Gardner & Dalsing, 1986) and that "most people use only about 10% of their potential brain power" (Higbee & Clay, 1998). As the conceptual change literature suggests, however, change while difficult may be more likely for certain kinds of students.

In the few studies of psychological misconceptions that look at whether student performance on misconceptions tests differs with measures of student ability, results are mixed. Using course grade as a measure of student ability, Best (1982) found no relation and Vaughan (1977) found a relation in only one of four psychology courses tested. McCutcheon (1991) found no relation between students' initial level of misconception and their final course grade but did find a relation between student initial misconception and student GPA (McCutcheon, Apperson, Hanson, & Wynn, 1992). Perhaps the differences between these studies result from the difficulty in measuring student ability by means of course grade (McCutcheon et al., 1992). Nevertheless, these findings suggest that better performing students may be at least slightly less susceptible to common misconceptions.

Even when high ability students hold initial misconceptions, they may be more likely than less able students to change their misconceptions following instruction. In the only study to look at this question, Gutman (1979) found that students who performed at the A/B level in the course changed many of their misconceptions whereas D/F students did not. This finding suggests that, as in the conceptual change literature, the better students may be better at comprehending the new information, evaluating the new information (Otero, 1998), or exerting the effort necessary to process the information (Dole & Sinatra, 1998). Although the data are limited, conceptual change in psychology may be more likely for students who have the tools needed to make the change.

In addition to ability, research suggests that metacognitive skills are an important tool aiding conceptual change (e.g., Baker & Brown, 1984; Dole & Sinatra, 1998). Metacognitive skills include efforts to both monitor and regulate one's cognitive processes. Critical thinking is clearly a metacognitive skill that would influence students' ability to evaluate the evidence for and against concepts presented in introductory psychology. Critical thinking is defined as a logical and rational process of avoiding preconceptions by gathering evidence, contemplating and evaluating alternatives, and coming to conclusions (Smith, 2001). In the only study to look at critical thinking and psychological misconceptions, McCutcheon et al., (1992) found a weak but significant relation between student achievement (as measured by GPA), the Inference and Evidence subscales of the Watson-Glaser Critical Thinking Appraisal (Watson & Glaser, 1980), and student susceptibility to misconception. Both of the critical thinking subscales and GPA made independent contributions to the prediction of student misconceptions with the Inference subscale's contribution being slightly higher than that of the others. Although the study found relations between student thinking, student ability, and students' initial misconceptions, it did not assess the relations between thinking, ability, and change in misconceptions. Thus, we do not know the degree to which the kinds of learner characteristics believed to influence conceptual change predict change in students' beliefs in psychological misconceptions.

The purpose of the present study was to look at how students' ability and critical thinking skills predict whether they correct their misconceptions following an introductory psychology course. The conceptual change literature suggests that both students' ability and their metacognitive skills contribute to their evaluation of new information and the restructuring of their cognitive concepts. We expected both ability and critical thinking, therefore, to make unique contributions to the prediction of change in students' psychological misconceptions. Specifically, we examined whether (1) there was a significant decrease in psychological misconceptions following the introductory psychology course and (2) GPA and critical thinking predicted the reduction in posttest misconception above and beyond pre-course misconceptions.

Method

Participants

Ninety introductory psychology students, primarily traditional (immediately out of high school), volunteered to complete all questionnaires. The sample was 60% female. All students attended a small private university.

Materials

We developed a 36-item true/false questionnaire assessing psychological misconceptions (Taylor & Kowalski, 2004). We selected items from previous instruments (e.g., A schizophrenic is someone with a split personality [Vaughan, 1977]) and from the popular literature (e.g., Mozart's music increases intelligence). The introductory course covered all items through readings or class discussions. As have authors of other misconceptions measures, we worded items so that the correct response was false. Ten additional items were added at posttest, half of which we worded so that the correct response was true. This procedure allowed assessment of response bias at posttest. Split-half reliability at pretest was .69 and at posttest .80.

To assess critical thinking we used a set of items from the Scottsdale Critical Thinking Test (Ricker, 2003). The full test consists of 30 multiple-choice questions. Most questions describe scenarios in which there is a failure of critical thinking. Six questions were both characteristic of critical thinking in psychology (i.e., requiring an understanding of inferences, probability, and coincidence) and demonstrated adequate reliability (Cronbach's alpha= .62). Students' scores on this six-item scale correlated .40 with course grade.

Students' provided us with permission to obtain their first semester GPA from the university registrar. The registrar calculates GPA on a four-point scale (A = 4.00).

Procedure

Participants provided consent prior to completing the misconceptions pretest during the first meeting of the fall semester. About four weeks into the semester, participants completed the critical thinking questionnaire. During the last week of the semester, approximately 15 weeks later, they completed the posttest misconceptions test. Participants required about 20 minutes to complete each of these questionnaires. Students completed all questionnaires in class.

Results

Complete data were available for 74 participants. (We dropped 14 participants with missing data and two participants who demonstrated response bias on the misconceptions questionnaire at posttest.) The percent of the 36 items answered correctly on the misconceptions questionnaire at both pretest and at posttest was calculated, as was the percent of items answered correctly on the six-item Scottsdale Critical Thinking measure.

Table 1 shows means, standard deviations, and correlations for students' GPA, Misconception score at pretest, Misconception score at posttest, and Critical Thinking score. Aone-way repeated measures ANOVA indicated the difference between pre and post test scores (38% correct vs. 65% correct) was statistically significant, F(1, 73) = 232.92, p < .001, [n.sup.2]= .76, suggesting that students changed their misconceptions following the 15 week course.

Zero order correlations indicated that GPA and pretest misconception scores were unrelated. Correlations between GPA, critical thinking, and misconception posttest were all significant.

We used a hierarchical multiple regression procedure to investigate whether GPA and critical thinking make unique contributions to the prediction of change in misconception. Table 2 shows B, [R.sup.2] change for the regression of posttest scores on pretest scores, GPA, and critical thinking. Pretest scores predicted posttest scores at the first step, [r.sup.2] = .24, F(1,73) = 22.86,p < .001. The residual in posttest scores after this step is essentially a measure of change from pretest to posttest (Cohen & Cohen, 1983). Entering GPA and Critical Thinking score at the second step resulted in a significant addition to the prediction of change in misconceptions, [R.sup.2] change = .12, F(1, 70) = 6.63, p < .01, for a total [R.sup.2] of .36, F(1, 70) = 13.23, p < .001. With both Critical Thinking and GPA in the equation, Critical Thinking made a significant contribution to the prediction of change, t (70) = 2.68, p < .01 but GPA did not, t (70) = 1.60, p =. 11. Critical Thinking, therefore, predicted changes in posttest misconception scores above and beyond that accounted for by pre-course misconception scores and students'GPA. Although GPA predicted change without Critical Thinking in the model, [beta] = .22, t (71) = 2.21, p < .05, this relation was apparently accounted for by students' higher level of critical thinking.

Discussion

The purpose of this study was to look at how students' ability and critical thinking skills relate to changes in their misconceptions following an introductory psychology course. Based on the conceptual change literature, we expected both variables to contribute to the prediction of change in student misconceptions.

Correlations indicated that GPA and pretest misconception scores were unrelated. As others have noted (Gutman, 1979; McCutcheon. 1991), this lack of relation suggests that students who perform at all levels of ability come into the introductory psychology class with misconceptions. Despite high levels of initial misconceptions (38% correct at pretest), this study did find significant overall mean change on the misconceptions test following the introductory class (65% correct at posttest). Although students still performed at a relatively low level, the 37% difference in pre and posttest scores is higher than reported in earlier studies on misconceptions (e.g., Vaughan 1977) yet consistent with our previous studies (Taylor & Kowalski, in press). We attribute the difference to our questionnaire reflecting misconceptions we target in class. The importance of targeting has been noted often in the misconception literature (Vaughan, 1977) and directly assessed in our research (Taylor & Kowalski, 2002). Perhaps this targeting makes it easier for students at all levels of ability to comprehend the new information (Baker & Brown, 1984).

Despite this overall level of change, however, it does appear that learner characteristics make it more likely for certain students to change their beliefs. The significant correlations between GPA, critical thinking, and misconception posttest score suggest that students who perform at higher levels and students who think critically are likely to leave the introductory class with fewer misconceptions. This finding is consistent with the conceptual change literature--better students and students who engage in critical evaluation of information are most likely to modify their misbeliefs (Dole & Sinatra, 1998; Posner et al., 1998).

Although it is true that the measures of GPA, critical thinking, and misconception are intercorrelated, the multiple regression analysis provides a more detailed understanding of these relations. The significance of critical thinking with all variables in the equation suggests that at any ability level (as measured by GPA) those students who are better critical thinkers are more likely to change their misconceptions. Although better students are more likely to engage in critical thinking, students of all ability levels can evaluate or fail to evaluate the information-it is that evaluation that makes the difference. This finding differs somewhat from McCutchen et al., (1992) who found that critical thinking was superior to GPA in predicting students' initial misconception scores but that both variables made small but significant contributions. In our study, although better students were more likely to change misconceptions this difference appears to be related to the fact that the better students' in this study also tended to be better critical thinkers. One implication of these findings is that perhaps, as the conceptual change literature suggests (e.g., Baker & Brown, 1984), teaching students thinking strategies may help even students with less ability change their misconceptions.

Admittedly the findings from this study must be interpreted cautiously. The sample size is relatively small for testing variable contributions in the regression equation and the direction of influence is unclear. It is also true that more valid measures of the constructs in this study may exist. Although students' past performance is used in the conceptual change literature to identify "good" and "poor" students, GPA may be only a crude reflection of students' general ability. In addition, there is little previous research with the Scottsdale Critical Thinking Questionnaire (Ricker, 2003). Its reliability in this study was modest and only six items tapping critical thinking about psychological issues were used for this study. Although this focus on critical thinking about psychology was appropriate for the questions we were asking, it would be useful to know whether and how other more established measures of critical thinking predict conceptual change in psychology.

Despite these limitations we believe that this study contributes to the literature. Although some researchers suggest attention to student misconceptions is unnecessary (Griggs & Ransdell, 1987), we disagree. As the conceptual change literature suggests, inconsistent prior knowledge will make the acquisition of new concepts difficult for students. Knowing the level and type of misconceptions students bring to the psychology course is essential if instructors intend to build on these beliefs in a way that helps students achieve more sophisticated understandings (Bransford, Brown, & Cocking, 2000). Our study suggests that not only do students enter the introductory course with many misconceptions but that these misconceptions can change for students at any level of ability and are particularly likely to change for those students who think critically about the material. These findings lay the foundation for further research on factors influencing this change in psychological knowledge and they highlight the importance of engaging students in thinking about the material for meaningful conceptual change. It is this kind of meaningful change that is central if students are to relinquish popular notions of human behavior and develop a more scientific understanding of psychology.

Table 1
Variable descriptive statistics: Correlations, Means,
& Standard deviations

                           Misconceptions   Misconceptions
                           Pre-test         Post-test

Misconceptions Post-test   .46 **
GPA                        .04              .24 *
Critical Thinking          .28 *            .44 **
Mean                       38.16            65.01
SD                         14.21            15.67

                           GPA              Critical
                                            Thinking

Misconceptions Post-test
GPA                        .24 *
Critical Thinking          3.19             74.77
Mean                       0.47             25.47
SD

* p < .01, ** p < .001

Table 2

Hierarchical Multiple Regression of Course Performance and
Critical Thinking on Misconceptions Scores

                             [beta]     sr2
                                        (incremental)

Step 1                                  .24 *
  Misconception Pre-test     .49 **
Step 2                                  .12 *
  GPA                        .18
  Critical Thinking          .30 *

                                   R2 = .36 **
                          Adjusted R2 = .34
                                    R = .60 **

* p < .01, ** p < .001

References

Baker, L., & Brown, A.L. (1984). Meta-cognitive skills and reading. In P. D. Pearson (Ed.), Handbook of reading research (Vol. I, pp. 353-421). New York: Longman.

Best, J.B. (1982). Misconceptions about psychology among students who perform highly. Psychological Reports, 51,239-244.

Bransford, J.D., Brown, A.L., & Cocking, R. R. (2000). How people learn: Brain, Mind, Experience, and School. Washington, D.C.: National Academy Press

Chinn, C.A., & Brewer, W.F. (1993). The role of anomalous data in knowledge acquisition: A theoretical framework and implications for science instruction. Review of Educational Research, 63, 1-49.

Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlational analysis for the behavioral sciences (2nd ed.). Hillsdale, N. J.: Lawrence Erlbaum Associates.

Dole, J.A., & Sinatra, G.M. (1998). Reconceptualizing change in the cognitive construction of knowledge. Educational Psychologist, 33(2/3), 109-128.

Gardner, R.M., & Dalsing, S. (1986). Misconceptions about psychology among college students. Teaching of Psychology, 13, 32-34.

Griggs, R.A., & Ransdell, S.E. (1987). Misconceptions tests or misconceived tests? Teaching of Psychology, 14, 210-214.

Guzzetti, B. (2000). Learning counter-intuitive science concepts: What have we learned from over a decade of research? Reading and Writing Quarterly, 16, 89-98.

Gutman, A. (1979). Misconceptions of psychology and performance in the introductory course. Teaching of Psychology, 14, 210-214.

Higbee, K.L., & Clay, S. L. (1998). College students" beliefs in the ten-percent myth. The Journal of Psychology. 132(5), 469-476.

Kowalski, P., Taylor, A., Guggia, A., & Grande, M. (2003, May). Ability, cognitive engagement, instruction and changes in students' psychological misconceptions. Poster presented at the 83rd Annual Convention of the Western Psychological Association, Vancouver, BC.

Lipson, M.Y. (1982). Learning new information from text: The role of prior knowledge and reading ability. Journal of Reading Behavior; 14, 234-258.

Maki, R.H. (1998). Test predictions over text material. In D. J. Hacker, J. Dunlosky. & A. C. Graesser (Eds.), Metacognition in Educational Theory and Practice, Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

McCutcheon, L.E. (1991). A new test of misconceptions about psychology. Psychological Reports. 68, 647-653.

McCutcheon, L.E., Apperson, J. M., Hanson, E., & Wynn, V. (1992, Relationships among critical thinking skills, academic achievement, and misconceptions about psychology. Psychological Reports. 71, 635-639.

McKeachie, W.J. (1960), Changes in scores on the Northwestern misconceptions test in six elementary psychology courses. Journal of Educational Psychology, 51(4), 240-244.

Otero, J. (1998). Influence of knowledge activation and context on comprehension monitoring of science texts. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in Educational Theory and Practice. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Posner, G. J., & Strike, K. A., Hewson, P. W., & Gertzog, W. A. (1982). Accommodation of a scientific conceptions: Toward a theory of conceptual change. Science Education, 66, 489-508.

Ricker, J. (2003). Scottsdale Test of Critical Thinking. Unpublished instrument, Scottsdale Community College.

Ruble, R. (1986). Ambiguous psychological misconceptions. Teaching of Psychology, 13, 34-36.

Smith, R. A. (2001). Challenging your preconceptions (2nd ed.). Belmont, CA: Wadsworth/Thomson Learning.

Taylor, A.K., & Kowalski, P. (2004). Naive psychological science: The prevalence, strength, and sources of misconceptions. Psychological Record. 54, 15-25.

Taylor, A., & Kowalski, P. (June, 2002). Variables affecting the reduction of misconceptions in psychology. Poster presented at the Ninth Annual APS/STP Teaching Institute. at the 14th American Psychological Society Annual Convention. New Orleans, LA.

Vaughan, E. D. (1977). Misconceptions about psychology among introductory psychology students, Teaching of Psychology, 4, 139-141.

Watson, G., & Glaser, E. M. (1980). Critical Thinking Appraisal Manual. New York: Harcourt Brace Jovanovich.

Patricia Kowalski and Annette Kujawski Taylor, Department of Psychology, University of San Diego.

Correspondence concerning this article should be addressed to Dr. Kowalski, 5998 Alcala Park, San Diego, CA 92110: Email: [e-mail].

This work was supported, in part, by the University of San Diego through Faculty Research Grants to both authors.