Expert teachers' beliefs about use of critical-thinking activities with high- and low-advantage learners.

In a middle-school science class, a teacher asks students to interpret a graph indicating that the average temperature of the Earth's atmosphere has increased in recent decades. Then the teacher asks students to think through what will likely happen should this warming trend continue for another century. After some brainstorming and discussion, aided by the teacher's guidance, the students come up with a set of detailed predictions concerning the impact of global warming. Such a lesson instantiates the concept of critical thinking (CT): "cognitive skills and strategies that increase the likelihood of a desired outcome ... thinking that is purposeful, reasoned, and goal-directed--the kind of thinking involved in solving problems, formulating inferences, calculating likelihoods, and making decisions" (Halpern, 2003, p. 6). The teacher could have opted to share her knowledge of the topic in a lecture format (an approach comparatively low in CT), but the high-CT lesson had the benefits of making students active in their learning and requiring them to reason as scientists do.

Research has shown such lessons to be educationally effective (for a review see Alexander & Murphy, 1998; see also Brown & Campione, 1990; Lambert & McCombs, 1998; Pogrow, 1990, 1994; White & Fredrikson, 2000). Many educational psychologists and teacher educators regard CT-oriented activities as essential to optimal educational practice (e.g., Browne & Keeley, 2001; Ennis, 1987; Halpern, 2003; Henderson, 2001; King & Kitchener, 1994; Kuhn, 1999; Raths et al., 1986; Resnick, 1987; Torff, 2003). Moreover, contemporary testing procedures increasingly require performances emphasizing CT skills (Yeh, 2002); for example, the SAT now includes an analytic essay--a task posing greater CT challenges relative to earlier formats featuring only multiple-choice questions. Similarly, a recent high school biology exit examination in New York State charged students to write essays describing an experiment that would test a particular theory--a task requiring learners to engage in scientific reasoning.

Accordingly, educational researchers have addressed the issue of how to help students gain CT skills (Brown & Campione, 1990; Browne & Keeley, 2001; Ennis, 1987; Henderson, 2001; O'Tuel & Bullard, 1993; Perkins, Jay, & Tishman, 1993; Pogrow, 1990, 1994; Raths et al., 1986; Resnick, 1987; Torff, 2003). In this work, a distinction is typically drawn between high-CT activities (e.g., debate, discovery learning) and low-CT ones (e.g., lecture, drill), although the amount of CT required of learners in a given lesson may also be treated as a continuous variable.

Teachers' beliefs about high-CT and low-CT activities have been the focus of a growing body of literature in teacher education (Pogrow, 1990, 1996; Raudenbush, Rowan, & Cheong, 1993; Torff, 2005; Torff & Warburton, 2005; Warburton & Torff, 2005; Zohar, Degani, & Vaakin, 2001; Zohar & Dori, 2003), based on theory and research indicating that beliefs influence how teachers interact with learners and organize classroom tasks (Anning, 1988, Calderhead, 1996; Fang, 1996; Fenstermacher, 1994; Hollingsworth, 1989; Nespor, 1987; Pajares, 1992; Putman & Borko, 1997, 2000; Richardson, 1994, 1996, 2002; Smylie, 1988). The subset of this work that focuses on teachers' CT-related beliefs has investigated the relationship between such beliefs and teachers' perceptions of learners' advantage level--i.e., learners' academic track, achievement level, or SES advantages (Oakes, 1990; Page, 1990; Pogrow, 1990, 1996; Raudenbush et al., 1993; Torff, 2005; Torff & Warburton, 2005; Warburton & Torff, 2005; Zohar et al., 2001; Zohar & Dori, 2003). Studies investigating differences in CT-related beliefs for high-advantage and low-advantage learners have been motivated by the assertion that teachers judge high-CT activities to be ineffective for low-advantage learners, whom are purportedly seen as ill prepared to handle high-CT activities and in need of a remedial regimen of low-CT ones (Pogrow, 1990, 1996; Raudenbush et al., 1993; Zohar et al., 2001). According to this line of reasoning, a self-fulfilling prophecy may result in which low-advantage learners receive few high-CT activities, which restricts their academic growth, which in turn makes high-CT activities less likely to be used; in contrast, high-advantage learners receive abundant high-CT activities, which enhance their academic growth, which makes still more high-CT instruction likely (Zohar et al., 2001).

In the first study devoted to teachers' CT-related beliefs for different populations of learners, Raudenbush et al. (1993) asked 303 secondary teachers to complete specially designed scales that tapped teachers' emphasis on high-CT instruction in upper-track and lower-track classes. Participants were also asked to identify instructional objectives for these classes. Results indicated that emphasis on high-CT activities and instructional objectives differed significantly across academic tracks. Teachers were significantly less likely to focus on high-CT activities in lower-track classes (relative to high-track ones), especially in math and science. According to the researchers, differentiation of instruction based on academic track is commonplace in modern schools, due to a "transmission style of teaching that avoids challenging teacher-student interactions" in lower-track classes (Radenbush et al., 1993; p. 546).

Zohar et al. (2001) obtained similar results by conducting semi-structured interviews in which teachers described their instructional goals for learners identified by the experimenters as low-achieving or high-achieving. Almost half of the teachers (19 of 40 teachers, or 47.5%) believed high-CT activities to be ineffective for low-achieving learners. According to these 19 teachers, only high-achieving learners possess the academic skills necessary to participate successfully in high-CT activities. The researchers interpreted these findings as reflecting teachers' construal of learning as progressing from simple, lower-order cognitive skills (e.g., comprehension) to more complex ones (e.g., analysis).

At the same time, recent research points to a more complex picture of teachers' CT-related beliefs, by explicitly comparing views about low-CT activities and high-CT ones (Warburton & Torff, 2005). Practicing secondary teachers (N = 145) completed the Critical Thinking Belief Appraisal (CTBA)--a four-factor scale that taps teachers' beliefs about high-CT and low-CT activities for high-advantage and low-advantage learners. (The scale is described in detail below, since it was used in the research reported in this article; for validation data see Torff & Warburton, 2005.) Teachers rated both high-CT and low-CT activities as significantly more effective for high-advantage learners than low-advantage ones. Such "advantage effects" are consistent with prior research (Raudenbush et al., 1993; Zohar et al., 2001). However, teachers also produced "pedagogical-preference effects" in which high-CT activities were rated as significantly more effective than low-CT ones for both for high-advantage and low-advantage learners. Results as such call into question the claim that teachers judge low-CT activities to be preferable to high-CT ones for low-advantage learners. But neither do the results indicate that teachers believe that high- and low-advantage learners should have similar mixtures of high-CT and low-CT activities. The advantage effect was larger for high-CT activities than low-CT ones, and the pedagogical preference effect was considerably larger for high-advantage learners than low-advantage ones. Teachers apparently deemed it appropriate that low-advantage learners receive fewer high-CT activities than their high-advantage peers.

These findings appear to have implications for teacher education, but it is not possible for advocates of appropriate use of CT in schools to determine the extent to which the findings are favorable, given the absence of a comparison group of expert teachers. Only by comparing experts and inservice teachers can teacher educators ascertain whether the findings summarized above are consistent with expert teaching or indicative of a problem meriting a purposeful response in teacher-education practices. This comparison is undertaken in the research described below.

Method

Participants

Participants in this study (N = 194) comprised two groups of teachers employed at secondary schools in New York State and South Carolina: "expert" teachers and "inservice" ones. (See Table 1 for data on participants' ages, years of teaching experience, and educational attainment.) Teachers in the expert group (N = 92, from 27 different schools) were nominated by supervisors in the role of principal or assistant principal. These supervisors were asked to nominate one to three teachers who had proven through classroom observations and student outcomes to be extraordinarily capable of fostering learning for all learners, and also had five or more years of classroom experience--a benchmark for teacher expertise established by Berliner (1987, 1992, 1994). Such a nomination procedure is in accordance with other research in teaching expertise (e.g., Berliner, 1987, 1992, 1994; Clarridge & Berliner, 1991; Torff, 2003) and in expertise studies in other domains (e.g., Bereiter & Scardamalia, 1993; Chase & Simon, 1973; Chi, Glaser, & Farr, 1988). Different supervisors likely have different selection criteria, but as such this study's expert nominations represent views from the teaching profession and avoid use of nomination criteria determined by the researcher. The 53 women and 39 men in the expert group included 18 teachers of English, nine of languages other than English, 15 of mathematics, eight of physical education/health, 10 of science, 18 of social studies, 12 of visual/performing arts, and two of other subjects.

Teachers in the inservice group (N = 102) were picked at random from 30 schools similar in socioeconomic status (SES) to the schools at which the expert teachers were employed. Data on housing costs were used for SES matching. Remarkably, random selection resulted in a group of inservice teachers with 51 females and 51 males. They included four teachers of business, 23 of English, 12 of languages other than English, three of mathematics, two of physical education/health, 28 of science, 20 of social studies, five of visual/performing arts, and five of other subjects.

Some clarification is needed concerning the groups in this study. It was not necessary to classify individual teachers according to a scheme for categorizing the levels of teaching expertise--such as the one presented by David Berliner (1987, 1992, 1994), which sets out five levels of teaching prowess (novice, advanced beginner, competent, proficient, expert). Instead, it was necessary to determine how the beliefs held by a typical blend of these levels in the teacher workforce (gleaned through random selection) compare to the beliefs of experts, thus isolating the pattern of beliefs characteristic of teaching expertise. A suitable method is to compare experts and randomly-selected ("inservice") teachers, who by definition include the full spectrum of levels of teaching expertise. Since at least some teachers that would have been classified as experts had their supervisors been asked to submit such a nomination, it seems inappropriate to distinguish the groups as "expert" and "non-expert." Conversely, experts are employed as teachers and are therefore technically classifiable as inservice teachers. Hence, in this study, experts are the group of teachers with the highest level of teaching skill (even though they work as inservice teachers) and inservice teachers are the group representing the full range of levels of teaching skill (even though some teachers classifiable as experts are likely also included in such a group).

Materials and Procedure

Participants completed the Critical Thinking Belief Appraisal (CTBA) (Torff &Warburton, 2005). The 36-item CTBA includes 12 lesson vignettes ("prompts") describing high-CT and low-CT classroom activities (Figure 1). Each prompt is followed by three items assessing the effectiveness of the prompt for high-advantage or low-advantage learners. To reduce response bias due to self-presentation, three "advantage characteristics" (each highly correlated with SES advantages) are used as indicators of learners' advantage level: ability (learners' capacity for intellectual or academic achievement when dealing with the specific topic the class is studying); prior knowledge (how much learners know about the specific topic the class is studying before they participate in additional lessons); and motivation (how much interest and attention learners show when dealing with the specific topic the class is studying) (Archer & McCarthy, 1988; Dweck, 1986; Givvin, et al., 2001; Nolen & Nichols, 1994; Madon et al., 1998; Moje & Wade, 1997; Pintrich & Schunk, 1996; Tollefson, 2000). Each prompt is followed by either a high-advantage item or a low-advantage one for each advantage characteristic--for example, prompt #1 is followed by a low-ability item, a low prior-knowledge item, and a high-motivation item. Each item is scored on a six-point Likert-type scale where 1 = "highly ineffective" and 6 = "highly effective." The CTBA is balanced as follows: the 12 prompts describe six high-CT and six low-CT activities; it presents 18 items for high advantage learners and 18 for low advantage ones; it includes 12 of each the three advantage characteristics, six for high advantage learners and six for low advantage learners. The four factors yielded by the scale include 1) high-CT activities for high-advantage learners (high-CT/high-adv), 2) high-CT activities for low-advantage learners (high-CT/low-adv), 3) low-CT activities for high-advantage learners (low-CT/high-adv), and 4) low-CT activities for low-advantage learners (low-CT/low-adv).

A series of five validation studies supported the theoretical and practical efficacy of the construct and measure of teachers' beliefs about classroom use of CT activities for different populations of learners (Torff & Warburton, 2005). The scores yielded by the CTBA evinced a stable factor structure comprised of four factors that collectively accounted for 62% of within-group variance and individually yielded satisfactory levels of internal consistency (alpha levels of .88, .76, .90, and .88, respectively). The scale's discriminant validity was supported by low correlations (ranging from .02 to .28, p < .05) obtained between each of the four factors and measures of CT ability (the California Critical Thinking Skills Test; Facione, Facione, & Giancarlo, 2000), CT disposition (the Need for Cognition Scale; Caccioppo & Petty, 1982; Caccioppo et al., 1996), and social desirability (the Marlowe Crowne Social Desirability Scale; Crowne & Marlowe, 1964). Finally, the scale produced scores with satisfactory predictive validity, with an overall correlation of .72 (p < .05) between ratings of observed classroom use of high-CT and low-CT activities and the subset of CTBA items that match the learner characteristics of the classroom observed (as judged by the teacher). Validation research also supported the utility of the three advantage characteristics. Factor-analytic results and internal-consistency correlations (ranging from .74 to .96, p < .05) indicated that ability, prior knowledge, and motivation collectively were reliable indicators of teachers' perception of learners' advantage level but had little effect as independent factors.

The scale was accompanied by six demographic questions for use as independent variables. The two continuous independent variables were age and teaching experience. The four categorical independent variables were gender, content area (business, English, languages other than English, mathematics, physical education/ heath, science, social studies, visual/performing arts, or other), postsecondary educational attainment in the content area (none, bachelors, masters, masters plus 30 credits, or doctoral), and postsecondary educational attainment in pedagogical studies (none, bachelors, masters, masters plus 30 credits, or doctoral).

Teachers in both groups were contacted by telephone and asked to complete an opinion survey. No teachers declined to do so. Neither teachers nor supervisors were compensated for participating. Research assistants administered the CTBA at the schools at which the teachers were employed. The teachers were instructed that the survey had no correct answers and responses were confidential. The teachers, supervisors, and research assistants had no knowledge of the research design or hypothesis.

Results

Table 2 presents means and standard deviations for the dependent variables in the study (high-CT/high-adv, high-CT/low-adv, low-CT/high-adv, and low-CT/ low-adv). Evaluation of assumptions of normality of sampling distributions, linearity, homogeneity of variance, and reliability of covariates were satisfactory. No univariate within-cell outliers were obtained at alpha = .001. The covariates age and teaching experience were highly correlated, with a squared multiple correlation, SMC ([R.sup.2]), of .73. The covariate age was deleted to reduce the potential for multicollinearity and focus the analyses on teaching experience, the more conceptually pertinent variable. ANCOVA and MANCOVA procedures were carried out to control for the effects of covariate measures while examining within-participants and between-participants differences in the dependent variables. None of the covariates contributed significant adjustment to a dependent variable. (Note: throughout this article, the term "significant" refers to statistical significance.)

Within-Participants Analyses

In the first of a series of within-participants ANCOVA analyses, the difference between high-CT/high-adv and high-CT/low-adv was significant among both experts [F(1, 85)= 26.65, p < .0001; eta-squared = .28] and inservice teachers [F(1, 95)= 281.57, p < .0001; eta-squared = .68]. Similarly, the difference between low-CT/high-adv and low-CT/low-adv was significant among experts [F(1, 85) = 8.80, p < .01; eta-squared = .09] and inservice teachers [F(1, 95) = 111.35, p < .0001; etasquared = .52]. Among teachers in both groups, high-CT activities and low-CT ones alike were seen as more effective with high-advantage than low-advantage learners, demonstrating the advantage effects obtained in prior research (Raudenbush et al., 1993; Torff, 2005; Warburton & Torff, 2005; Zohar et al., 2001). It is noteworthy, however, that the effect sizes produced by inservice teachers (.68 and .52 for high-CT and low-CT activities, respectively) were considerably larger than those yielded by expert teachers (.28 and .09), indicating that inservice teachers demonstrated much stronger advantage effects relative to expert teachers.

The difference between high-CT/high-adv and low-CT/high-adv was significant among both experts [F(1, 85) = 171.64, p < .0001; eta-squared = .64] and inservice teachers [F(1, 95) = 8.32, p < .01; eta-squared = .08]. The difference between high-CT/low-adv and low-CT/low-adv was significant among experts [F(1, 85) = 182.69, p < .0001; eta-squared = .69] but insignificant among inservice teachers. In accordance with results obtained in previous research (Torff, 2005; Warburton & Torff, 2005), both experts and inservice teachers evinced a pedagogical-preference effect favoring high-CT activities over low-CT ones for high-advantage learners. This effect was far stronger for experts (effect size of .64) than for inservice teachers (.08). For low-advantage learners, experts (but not inservice teachers) demonstrated a pedagogical-preference effect (with a large effect size of .69).

Between-Participants Analyses

Between-participants MANCOVA procedures yielded significant group differences for the combined dependent variables [F(4, 183) = 104.58, p < .0001; eta-squared = .70]. Post hoc univariate comparisons were performed using the Bonferroni method. In high-CT/high-adv, the difference between groups was not significant. In high-CT/low-adv, significant differences were obtained between (p < .0001; partial eta-squared = .48]. Relative to inservice teachers, experts produced significantly higher ratings for high-CT activities for low-advantage learners (with a substantial effect size of .48).

In low-CT/high-adv, significant group differences were obtained (p < .0001; partial eta-squared = .36]. Similarly, the groups differed significantly in low-CT/ low-adv (p < .0001; partial eta-squared = .15]. Compared to inservice teachers, expert teachers produced significantly lower ratings concerning low-CT activities for both high-advantage learners (with a moderate effect size of .36) and low-advantage learners (with a comparatively modest effect size of .15).

Discussion

Pedagogical-Preference Effects

For both populations of learners, experts were generally more favorable to high-CT activities and less favorable to low-CT ones (relative to inservice teachers). For high-advantage learners, both groups produced pedagogical-preference effects favoring high-CT activities--but the effect size was eight times larger for experts (eta-squared = .64) than inservice teachers (.08). This appears to be attributable to differences in beliefs about low-CT activities, not high-CT ones; experts were lower in low-CT activities for high-advantage learners with an effect size of .36, but the difference in high-CT activities for high-advantage learners was insignificant.

For low-advantage learners, experts favored high-CT activities over low-CT ones with a large effect size of .69, whereas the difference among inservice teachers was insignificant. The result seems to stem from experts' stronger support for high-CT activities and weaker support for low-CT activities, relative to inservice teachers. Experts were higher in high-CT activities for low-advantage learners with an effect size of .48; and they also were lower in ratings for low-CT activities for low-advantage learners with an effect size of .15. For both high-advantage and low-advantage learners, experts were more favorable to high-CT activities and less favorable to low-CT ones relative to inservice teachers.

Advantage Effects

As with inservice teachers, experts produced advantage effects for both high-CT and low-CT activities, but with different effect sizes. For high-CT activities, experts produced an advantage-effect size of .28, whereas inservice teachers produced a much larger effect size of .68. This difference seems to be attributable to beliefs about high-CT items for low-advantage learners; as noted, for this variable experts produced higher ratings than inservice teachers with an effect size of .48. In contrast, there was no group difference for high-CT items for high-advantage learners.

Concerning advantage effects for low-CT items, inservice teachers produced a considerably larger effect size (.52) relative to expert teachers (.09). This result appears to stem from the findings that experts produced lower ratings than inservice teachers concerning low-CT items for both high-advantage learners (with an effect size .36) and low-advantage learners (with an effect size of .15). Both groups produced advantage effects for both high-CT and low-CT items, but inservice teachers produced meaningfully larger ones, indicating stronger beliefs supporting differentiation of the CT-level of instruction based on perceived learner advantages.

These results bear upon the assertion that low-advantage learners are too often afforded less rigorous curriculum relative to their high-advantage counterparts (Aronson, 2004; Barton, 2004). Teacher's beliefs may contribute to this "rigor gap," given the findings that expert teachers were generally more supportive of high-CT activities, less supportive of low-CT ones, and less prone to differentiate instruction according to learner advantages (relative to inservice teachers). The rigor gap has been implicated as a factor in the persistent achievement gap between high- and low-advantage learners (Aronson, 2004; Barton, 2004; Popham, 2004; Rothstein, 2004)--a suggestion that seems especially warranted with testing practices increasingly emphasizing CT skills (Yeh, 2002). As such, teacher-education practices that initiate change in teachers' CT-related beliefs have potential to help alleviate the achievement gap in schools.

Implications for Teacher Education

The results of this study indicate a need for research and practice in teacher education with a pair of aims. First, initiatives are needed to further discount the effectiveness of low-CT activities for both high-advantage and low-advantage learners. Although teacher educators have been vigorous in their efforts to downplay low-CT activities (Blumenfeld, Hicks, & Krajcik, 1996; Lambert & McCombs, 1998; Richardson, 1997), continued efforts appear to be warranted, given that experts produced lower ratings than inservice teachers in low-CT activities for both learner populations. Second, research and practice in teacher education are needed to encourage teachers to support high-CT instruction for low-advantage learners. Similar conclusions were drawn by Pogrow (1990, 1996), Raudenbush et al. (1993), and Zohar et al. (2001). Although there is little theory or research specifically devoted to fostering change in teachers' CT-related beliefs, promising techniques that could be extended from the general literature on teacher change include: a) encouraging teachers to reflect on their CT-related beliefs in discussions, journals, and reaction papers (Feinman-Namser et al., 1989; Hollingsworth, 1989; Holt-Reynolds, 1992; Placier & Hamiliton, 1994; Richardson & Hamilton, 1994; Stallings, 1989); b) assigning preservice teachers to observe, in their fieldwork, the discrepancies between the instructional approaches used by cooperating teachers to teach different learner populations (Feinmen-Nemser et al., 1989; Placier & Hamilton, 1994; Pogrow & ten Brink, (1993); c) asking teachers to analyze case studies of high-CT and low-CT instructional planning, including instances of effective use of high-CT activities for low-advantage learners (Anderson, et al., 1995; Barnett & Sather, 1992; Blumenfeld, Hicks, & Krajcik, 1996; Morine Dershimer, 1993; Richardson & Hamilton, 1994); and d) providing opportunities for guided participation in the design of vehicles for curriculum, instruction, and assessment that promote expert use of high-CT activities (Blumenfeld et al., 1994; Carter, 1990; Smylie, 1988; Russell, 1995; Wilson, 1996; Woolfolk Hoy & Murphy, 2001) (for a review of the literature on teacher change see Richardson & Placier, 2002).

Limitations and Future Research

Several limitations of this study should be noted, at least some of which set out promising directions for future research. To begin with, limitations are inherent in the use of categorical variables (high-CT versus low-CT and high-advantage versus low-advantage) where the variables might otherwise be treated along a more finely grained continuum. Although CTBA validation research supported the use of three factors (ability, prior knowledge, and motivation) collectively as an assessment of teachers' perception of learners' advantage level (Torff & Warburton, 2005), these advantage characteristics individually may differ in influence on teachers' beliefs. Teachers in other geographic regions may hold different beliefs relative to the teachers in New York State and South Carolina who participated in this study. Among secondary teachers, there may be differences across content areas--for example, science teachers and social studies teachers may differ in CT-related beliefs. And the secondary teachers studied here may have different CT-related beliefs relative to other populations of teachers--notably elementary teachers and special education teachers.

It seems plausible that beliefs about use of CT activities with different learner populations might be influenced by a variety of teacher characteristics--including teachers' SES, level of CT ability, and the extent of their experience with high-advantage and low-advantage populations of learners. Future research might well investigate how these teacher characteristics interact with beliefs about use of CT in schools. Moreover, teaching practices inevitably occur in the context of modern schooling, including today's single-minded focus on accountability through testing. High-stakes testing exerted a significant impact on schools before the No Child Left Behind Act and are even more of a presence in the wake of this legislation. As noted, modern testing procedures are increasingly CT-oriented relative to previous testing formats; however, the extent to which tests have been modified as such varies across states, subjects, and grade levels, and it remains unclear how teachers' beliefs and practices are influenced by their awareness of and inclination to respond to new testing procedures. Studies are needed investigating the interaction of these factors with teachers' beliefs about use of CT activities with different learner populations.

Finally, it is unknown which aspects of the experience of the classroom teacher influence their beliefs about CT activities. Interviews with teachers have potential to reveal how and why teachers come to hold a set of CT-related beliefs, given their experiences during and after preservice education. This research promises to shed light on the underlying causes of CT-related beliefs found in this study to be partially incompatible with expertise in teaching.

References

Alexander, P. & Murphy, P. (1998). The research base for APA's learner-centered psychological principles. In Lambert, N., & McCombs. B. (Eds.) (1998), How learners learn: reforming schools through learner-centered instruction. Washington DC: American Psychological Association.

Anderson, L. (2001). Nine prospective teachers and their experiences in teacher education: The role of entering conceptions of teaching and learning. In B. Torff & R. Sternberg, (Eds.), Understanding and teaching the intuitive mind: Student and teacher learning (pp. 187- 216). Mahwah, NJ: Erlbaum Associates.

Anderson, L., Blumenfeld, P., Pintrich, P., Clark, C., Marx, R., & Peterson, P. (1995). Educational psychology for teachers: Reforming our courses, rethinking our roles. Educational Psychologist, 30, 143-158.

Anning, A. (1988). Teachers' theories about children's learning. In J. Calderhead (Ed.), Teachers' professional learning (pp. 128-145). London, UK: Falmer Press.

Archer, J. & McCarthy, B. (1988). Personal biases in student assessment. Educational Research, 30, 142-145.

Aronson, J. (2004). The threat of stereotype. Educational Leadership, 62, 14-20. Barnett, C. & Sather, S. (1992). Using case discussions to promote changes in beliefs among mathematics teachers. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.

Barton, P. (2004). Why does the gap persist? Educational Leadership, 62, 8-13.

Bereiter, C. & Scardamalia, M. (1993). Surpassing ourselves: An inquiry into the nature and implications of expertise. New York: Open Court.

Berliner, D. (1992). The nature of expertise in teaching. In Oser, F., Dick, A. & Patry, J. (Eds.), Effective and responsible teaching (p. 227-248). San Francisco: Jossey Bass.

Berliner, D. (1994). Expertise: The wonders of exemplary performance. In J.N. Mangieri & C.C. Block (Eds.), Creating powerful thinking in teachers and students: Diverse perspectives (pp. 161-186). Fort Worth, TX: Holt, Reinhardt, & Winston.

Berliner, D. C. (1987). In pursuit of the expert pedagogue. Educational Researcher, 15, 5-13.

Blumenfeld, P., Hicks, L., & Krajcik, J.S. (1996). Teaching educational psychology through instructional planning. Educational Psychologist, 31, 51-62.

Blumenfeld, P.C., Krajcik, J.S., Marx, R.W., & Soloway, E. (1994). Lessons learned: How collaboration helped middle-school science teachers learn project-based instruction. Elementary School Journal, 94, 539-551.

Brookhart, S. & Freeman, D. (1992). Characteristics of entering teacher candidates. Review of Educational Research, 62, 37-60.

Brooks, J. & Brooks, M. (1993). The case for constructivist classrooms. Alexandria, VA: Association for Supervision and Curriculum Development.

Brown, A. & Campione, J. (1990). Communities of learning and thinking, or a context by any other name. In D. Kuhn, (Ed.), Developmental perspectives on teaching and learning thinking skills. Contributions to Child Development (Vol. 21, pp. 108-126). Basel, Switzerland: Karger.

Browne, M. & Keeley, K. (2001). Asking the right questions: A guide to critical thinking (Sixth Ed.). Upper Saddle River, N.J.: Merrill/Prentice Hall.

Caccioppo, J. & Petty, R. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116-131.

Cacioppo, J.T., Petty, R.E., Feinstein, J.A., & Jarvis, W.B. (1996). Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychological Bulletin, 119, 197-253.

Calderhead, James (1996). Teachers: Beliefs and knowledge. In D. Berliner & R. Calfee (Eds). Handbook of educational psychology. (pp. 709-725). New York: Macmillan.

Carter, K. (1990). Teachers' knowledge and learning to teach. In W.R. Houston (Ed.), Handbook of research on teacher education (pp. 291-310). New York: Macmillian.

Chase, W. & Simon, H. (1973). The mind's eye in chess. In W. Chase (Ed.), Visual information processing. Mahwah, NJ: Erlbaum.

Chi, M., Glaser, R. & Farr, M. (Eds.) (1988). The nature of expertise. Hillsdale, NJ: Erlbaum. Clarridge, P., & Berliner, D. (1991) Perceptions of learner behavior as a function of expertise. Journal of Classroom Interaction, 26, 1-8.

Crowne, D. & Marlow, D. (1964). The approval motive. New York: Wiley.

Dweck, C. (1986). Motivational processes affecting learning. American Psychologist, 41, 1040-1048.

Ennis, R. (1987). A taxonomy of critical-thinking dispositions and abilities. In J. Baron & R. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26). New York: Freeman.

Facione, P., Facione, N., & Giancarlo, C. (2000). The California critical thinking skills test. Millbrae, CA: California Academic Press.

Fang, Z. (1996). A review of research on teacher beliefs and practices. Educational Research, 38, 47-65.

Feinman-Nemser, S., McDiarmid, G. W., Melnick, S. L., & Parker, M. (1989). Changing beginning teachers' conceptions: A description of an introductory teacher education course (Research Report 89-1). East Lansing, MI: National Center for Research on Teacher Education, College of Education, Michigan State University.

Fenstermacher, G.D. (1994). The knower and the known: The nature of knowledge in research on teaching. In L. Darling-Hammond (Ed.), Review of research in education (Vol. 20; pp. 1-54). Washington, DC: American Educational Research Association.

Fosnot, C. (1989). Enquiring teachers, enquiring learners. New York: Teacher's College Press. Givvin, K.B., Stipek, D.J., Salmon, J.M., & MacGyvers, V.L. (2001). In the eyes of the beholder: Students' and teachers' judgments of students' motivation. Teaching and Teacher Education, 17, 321-331.

Halpern, D. (2003). Thought and knowledge (4th Ed.). Mahwah, NJ: Erlbaum.

Henderson, J. (2001). (3rd Ed.). Reflective teaching: Professional artistry through inquiry. Upper Saddle River, NJ: Merrill/Prentice Hall.

Hollingsworth, S. (1989). Prior beliefs and cognitive change in learning to teach. American Educational Research Journal, 26, 160-189.

Holt-Reynolds, D. (1992). Personal history-based beliefs as relevant prior knowledge in course work. American Educational Research Journal,

44, 279-287.

Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28, 16-26.

Lambert, N & McCombs. B. (Eds.) (1998). How learners learn: reforming schools through learner-centered instruction. Washington DC: American Psychological Association.

Madon, S., Jussim, L., Keiper, S. Eccles, J., Smith, A., & Paolumbo, P. (1998). The accuracy and power of sex, social class, and ethnic stereotypes: A naturalistic study in person perception. Personality and Social Psychology Bulletin, 24, 1304-1318.

Mayer, R. (1991). Cognition and instruction: their historic meeting within educational psychology. Journal of Educational Psychology, 84, 405-412.

Moje, E.B & Wade, S.E. (1997). What case discussions reveal about teacher thinking. Teaching and Teacher Education, 13, 691-712.

Morine Dershimer, G. (1993). Tracing conceptual change in preservice teachers. Teaching and Teacher Education, 9, 15-26.

Nespor, J. (1987). The role of beliefs in the practice of teaching. Journal of Curriculum Studies, 19, 317-28.

Nolen, S.B. & Nicholls, J.G. (1994). A place to begin (again) in research on student motivation: Teachers' beliefs. Teaching and Teacher Education, 10, 57-69.

O'Tuel, F. & Bullard, R. (1993). Developing higher-order thinking in the content areas. Pacific Grove, CA: Critical Thinking Books and Software.

Oakes, J. (1990). Multiplying inequalities: The effects of race, social class, and tracking on opportunities to learn math and science. Santa Monica, CA: Rand Corporation.

Page, R.N. (1990). The lower-track curriculum in a college-preparatory high school. Curriculum Inquiry, 20, 249-281.

Pajares, M.F. (1992). Teacher beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 14, 5-19.

Patrick, H. & Pintrich, P. (2001). Conceptual change in teachers' intuitive conceptions of learning, motivation, and instruction: The role of motivational and epistemological beliefs. In B. Torff & R. Sternberg, (Eds.) Understanding and teaching the intuitive mind: Student and teacher learning (pp. 117-144). Mahwah, NJ: Erlbaum.

Perkins, D.N., Jay, E., & Tishman, S. (1993). New conceptions of thinking: From ontology to education. Educational Psychologist, 28, 67-85.

Pintrich, P. & Schunk, D. (1996). Motivation in education. Upper Saddle River, NJ: Prentice Hall.

Placier, P. & Hamilton, M.L. (1994). Schools as contexts: A complex relationship. In V. Richardson (Ed.), Teacher change and the staff development process: A case in reading instruction (pp. 135-159). New York: Teachers College Press

Pogrow, S. & ten Brink, B. (1993). Where's the beef? Looking for exemplary materials. Educational Leadership, 50, 39-46.

Pogrow, S. (1990). Challenging at-risk learners: findings from the HOTS program. Phi Delta Kappan, 71, 389-97.

Pogrow, S. (1994). Helping learners who "just don't understand." Educational Leadership, 52, 62-66. Popham, W.J. (2004). A game without winners. Educational Leadership, 62, 46-50.

Putman, R & Borko, H. (1997). Teacher learning: implications of new views of cognition. In B.J. Biddle, T.L. Good, & I.F. Goodson, (Eds.), The international handbook of teachers and teaching (Vol. 2, pp. 1223-1296). Dordrecht, the Netherlands: Kluwer.

Putman, R & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 29, 4-15.

Radenbush, S.W., Rowan, B., Cheong, Y.F. (1993). Higher order instructional goals in secondary schools: Class, teacher, and school influences. American Educational Research Journal, 30, 523-553.

Raths, L., Wasserman, S., Jonas, A., & Rothstein, A. (1986). Teaching for thinking. New York: Teachers College Press.

Resnick, L. (1987). Education and learning to think. Washington, DC: National Academy Press.

Richardson, V. & Placier, P. (2002). Teacher change. In V. Richardson (Ed.), Handbook of research on teaching (4rd Ed., pp. 905-947). Washington, DC: American Educational Research Association.

Richardson, V. (1990). Significant and worthwhile change in teaching practice. Educational Researcher, 19, 10-18.

Richardson, V. (1994). The consideration of beliefs in staff development. In V. Richardson (Ed.), Teacher change and the staff development process: A case of reading instruction (pp. 90-108). New York: Teachers College Press.

Richardson, V. (1996). The role of attitudes and beliefs in learning to teach. In J. Sikula (Ed.), Handbook of research in teacher education (2nd Ed., pp. 102-119). New York: Macmillian.

Richardson, V. (Ed.) (1997). Constructivist teacher education: Building a world of new understandings. London, UK: Falmer Press.

Richardson, V. (Ed.) (2002). Handbook of research on teaching (Fourth Edition). Washington, DC: American Educational Research Association.

Rothstein, R. (2004). The achievement gap: A broader picture. Educational Leadership, 62, 40-45.

Russell, T. (1995). Returning to the physics classroom to rethink how one teaches physics. In T. Russell and F. Korthagen (Eds.), Teachers who teach teachers (pp. 95-112). London, UK: Falmer Press.

Smylie, M.A. (1988). The enhancement function of staff development: Organizational and psychological antecedents to individual teacher change. American Educational Research Journal, 23, 1-30.

Sparks, G.M. (1988). Teachers' attitudes toward change and subsequent improvements in classroom teaching. Journal of Educational Psychology, 80, 111-117.

Stallings, J. (1989). School achievement effects and staff development: What are some critical factors? Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.

Strauss, S. (2001). Folk psychology, folk pedagogy, and their relations to subject-matter knowledge. In B. Torff & R. Sternberg, (Eds.) Understanding and teaching the intuitive mind: Student and teacher learning (pp. 217-242). Mahwah, NJ: Erlbaum.

Sykes, G. & Bird, T. (1992). Teacher education and the case idea. In G. Grant (Ed.), Review of Research in Education (Vol. 19, pp. 457-521). Washington, DC: American Educational Research Association,

Tishman, S., Perkins, D. & Jay. E. (1993). The thinking classroom. Boston: Allyn & Bacon.

Tollefson, N. (2000). Classroom applications of cognitive theories of motivation. Educational Psychology Review, 12, 63-83.

Torff, B. (2003). Developmental changes in teachers' use of higher-order thinking and content knowledge. Journal of Educational Psychology, 95, 563-569.

Torff, B. (2005). Developmental changes in teachers' beliefs about critical-thinking activities. Journal of Educational Psychology, 97, 13-22.

Torff, B., & Warburton, E.C. (2005). Assessment of teachers' beliefs about classroom use of critical-thinking activities. Educational and Psychological Measurement, 63, 155-179.

Warburton, E.C., & Torff, B. (2005). The effect of perceived learner advantages on teachers' beliefs about critical-thinking activities. Journal of Teacher Education, 56, 24-33.

White, B. & Fredrikson, J. (2000). Metacognitive facilitation: An approach to making scientific inquiry available to all. In J. Minstrell & E. Van-Zee (Eds.), Inquiry into inquiry learning and teaching in science (pp.331-370). Washington, DC: American Association for the Advancement of Science.

Wilson, B. (Ed.) (1996). Constructivist learning environments: Case studies in instructional design. Englewood Cliffs, NJ: Educational Technology Publications.

Woolfolk Hoy, A. & Murphy, K. (2001). Teaching educational psychology to the intuitive mind. In B. Torff & R. Sternberg, (Eds.) Understanding and teaching the intuitive mind: Student and teacher learning (pp. 145-186). Mahwah, NJ: Erlbaum.

Yeh, S. (2002). Tests worth teaching to: Constructing state-mandated tests that emphasize critical thinking. Educational Researcher, 30, 12-17.

Zohar, A. & Dori, J. (2003). Higher order thinking and low-achieving students: Are they mutually exclusive? The Journal of the Learning Sciences, 12, 145-182.

Zohar, A., Degani, A., & Vaakin, E. (2001). Teachers' beliefs about low-achieving students and higher-order thinking. Teaching and Teacher Education, 17, 469-485.

Bruce Torff is an associate professor of curriculum and teaching in the School of Education and Allied Human Services at Hofstra University, Hempstead, New York.

Table 1
Means and Standard Deviations for Independent Variables

                                                Group

                                Expert     Inservice
                                Teachers   Teachers

Variable                            M       SD           M      SD

Age, in years                     42.23    8.82        37.40   10.07

Teaching experience, in years      1459    8.75        11.55   7.51

Educational attainment in          2.68    1.18        2.56    1.09
  content area
Educational attainment in          3.36    0.97        3.02    1.13
  pedagogy

Note. Educational attainment variables in both conten t and pedagogy
were coded as follows: 1 = no postsecondary educational attainment;
2 = bachelor's degree; 3 = master's degree; 4 = master's plus 30
credits; 5 = doctorate.

Table 2
Means and Standard Deviations for Dependent Variables

                                     Group

                              Expert      Inservice
                             Teachers      Teachers

Variable                      M     SD     M     SD

High-CT activities for      4.59    .45   4.44   .60
  high-advantage learners
High-CT activities for      4.20    .76   3.06   .59
  low-advantage learners
Low-CT activities for       2.84   1.08   4.19   .92
  high-advantage learners
Low-CT activities for       2.49    .76   3.05   .79
  low-advantage learners

Note. CT = critical thinking. All variables were assessed
on six-point scales (1 = highly ineffective, 6 = highly effective).

Figure 1

Sample prompts and items from the Critical Thinking Belief Appraisal

High-CT

A science class is studying the sun. The teacher asks students to
write down several ways in which the sun influences everyday life and
then to predict what would happen if the sun stopped shining.

To what extent would this activity be effective for...

... low-ability learners?

1   2   3   4   5     6

highly           highly
ineffective   effective

... learners with a high level of prior knowledge of the topic?

1   2   3   4   5     6

highly           highly
ineffective   effective

... learners with high motivation?

1   2   3   4   5     6

highly           highly
ineffective   effective

Low-CT

An Italian class is studying the vocabulary involved in ordering food
in a restaurant. The teacher writes several new words on the board,
defines them, asks students to repeat them, and provides a handout on
which students add them to sample sentences.

To what extent would this activity be effective for...

... high-ability learners?

1   2   3   4   5     6

highly           highly
ineffective   effective

... learners with a high level of prior knowledge of the topic?

1   2   3   4   5     6

highly           highly
ineffective   effective

... learners with high motivation?

1   2   3   4   5     6

highly           highly
ineffective   effective