Teacher efficacy, school reform, and state tests.

Abstract

This study examined relationships among school reform, teacher efficacy, and passage rates on state math and reading tests. 1,101 teachers from 59 elementary schools were surveyed, and ten were interviewed. Reports of effective implementation of reforms were positively related to improvement in passage rates on state tests, while teachers' efficacy helping difficult students was negatively related. Interviewees asserted that passage rates do not capture the academic progress of difficult students, because students are so far behind that progress did not equate with passing scores.

**********

Passage of the No Child Left Behind Act (NCLB) has increased pressure to improve achievement. The law requires states to sanction schools not meeting state determined adequate yearly progress (AYP) goals, including improvement on percentages of students passing state reading and math tests. Sanctions for lack of improvement range from mandated school choice to firing staff (NCLB, 2002).

Foolproof plans to improve failing schools remain elusive, but much is known about elements that make schools effective such as high expectations, parental involvement, and strong principal leadership. Research on elements of effective schools, beginning with Edmonds (1979), has been codified in reviews of the literature on effective schools (Taylor, 2001; Teddlie & Reynolds, 2000). Empirical studies confirm that schools with improving test scores contained the elements of effective schools included in these studies. (e.g. Barth, Haycock, Jackson, Mora, Pablo, Robinson, & Wilkins 1999; Carter 2000; Dolejs & Jenkins, 2001; EdSource, 2006; Trimble 2002).

How can failing schools become effective? Teachers' efficacy may be key. Based on Bandura (1977; 1978), efficacy has two dimensions: personal teacher efficacy, i.e. teachers' beliefs about their own abilities, and general teacher efficacy, i.e. teachers' beliefs about the abilities of teachers in general. Gibson and Dembo (1986) developed a survey for measuring efficacy, although the content validity has been questioned (Guskey & Passaro, 1994; Ross, Cousins, & Gadall, 1996; Labone, 2004; Tschannen-Moran, Hoy, & Hoy, 1998). Guskey and Passaro in particular noted that the survey items in general teacher efficacy are all stated negatively and, therefore, may depend more on negative perceptions of teachers' abilities rather than on personal vs. general beliefs. However, several researchers found "personal teacher efficacy" a useful predictor of teachers' willingness to implement changes (Fuchs, Fuchs, & Bishop, 1992), of teachers' ability to involve families (Garcia, 2004), and of student achievement (Moore & Esselman, 1992; Ross, 1992), indicating that Gibson and Dembo's survey provides useful information. The impact of teacher efficacy in times of intense reform efforts remains a less explored topic. Wheatley (2002, 2005) suggested that efficacy may lead to over confidence that could cause teachers to resist reforms.

Purpose

This study examined teachers' assessment of the effectiveness of school reform efforts, their efficacy and their schools' passage rates in the reading and math subtests of state tests. The Cleveland Municipal School District (CMSD) spent the last five years implementing research-based reforms. These reforms had been effective in similar high-poverty, high-minority settings (e.g. Barth, Haycock, Jackson, Mora, Pablo, & Wilkins 1999; Carter 2000; Dolejs & Jenkins, 2001; Trimble 2002). However, results on state tests in CMSD have been mixed. Of particular concern has been uneven progress on the Ohio Fourth-Grade Proficiency Test, which has been a high-stakes test for the past six years. The following questions were addressed:

(1.) To what extend do teachers' assessment of the implementation of school reform and efficacy predict school passage rates on the math and reading subtests of the Fourth-Grade Ohio Proficiency Test in 1998 (initial status) and improvement from 1998 to 2004 (growth rate)?

(2.) How do interviews of selected teachers illuminate the quantitative findings?

Methodology

In all, 1101 teachers from 59 elementary schools participated. These respondents represented 70% of contacted schools and 54% of teachers in participating schools.

The 19 items concerning reform implementation were based on Taylor (2001) and a review of district-wide policies. The efficacy questions were the 16 item Gibson and Dembo survey (1984). All school-level testing and demographic information was gathered from the state website, including passage rates on math and reading subtests of the Fourth Grade Ohio Proficiency Test from 1998 to 2004, teachers' years of experience, and percentages of non-white students. Student poverty was not included. CMSD reports 98.9% of students are economically disadvantaged. When surveys were administered, teachers could return a card indicating they would like to participate in semi-structured phone interviews. (See Appendix).

A principal component factor analysis with a varimax rotation determined dimensions of teachers' perceptions of reform implementation and self efficacy, and Cronbach's Alpha examined reliability. An individual growth model (Raudenbush & Bryk, 2002) examined the extent to which teachers' assessment of the implementation of reform and efficacy predicted school passage rates on the math and reading subtests in 1998 (initial status) and the rate of improvement from 1998 to 2004 (growth rate). To illuminate the quantitative findings, interviews of selected teachers were analyzed using a constant comparison method (Bogden & Biklen, 2003, Glaser & Strauss, 1967). Eight schools, three with low ratings on teacher confidence helping difficult students and two with high ratings, were selected from the list of schools with teachers willing to participate.

Results

Factor analysis revealed three dimensions based on the 19 items representing district reforms and effective school elements. The first, "teacher-controlled reforms," (8 items, alpha =.91), included elements of reform that teachers themselves could put into practice, such as "monitoring student progress frequently," and "having high expectations for all students." Because the other two dimensions seemed to fall together due to the newness of the reform or due to forces outside teachers' control, these dimensions were eliminated from subsequent analysis. See issue website http://rapidintellect.com/AEQweb/fal2006.htm

Although Gibson and Dembo (1984) found two dimensions of teacher efficacy, our analysis revealed three dimensions with eignenvalues greater than one. The first factor contained the same items as Gibson and Dembo's "general teacher efficacy." We called this, "limited teacher influence," (6 items, alpha = .81) because, as Guskey and Passaro (1994) pointed out, all items state that teachers have limited ability to help students. The second dimension "confidence helping difficult students," (6 items, alpha = .76), related to teachers' views of their ability to help difficult students. The third dimension, "confidence with successes," (4 items, alpha = .80), related to teachers' beliefs that students' successes were due to good teaching. See website http://rapidintellect.com/ AEQweb/fal2006.htm To create the most parsimonious model, "limited teacher influence," which in previous studies was not related to student achievement (Moore & Esselman, 1992; Ross, 1992), was eliminated from subsequent analysis.

The individual growth model (Raudenbush & Bryk, 2002) used the dimensions: "confidence with helping difficult students," "confidence with successes," and "teacher-controlled reforms." Teachers' perceptions of teacher-controlled reforms had a statistically significant positive relationship to the rate of improvement in school passage rates between 1998 and 2004 in both reading (beta = 0.06, p < 0.01) and mathematics (beta = 0.07, p < 0.01). In contrast, teachers' reported confidence in their ability to help students overcome challenges had a statistically significant negative relationship to rates of improvement in school passage rates in mathematics (beta =-0.08, p < 0.05). This relationship was not significant in reading but was also negative. Teachers' confidence that their teaching was responsible for students' successes was also negatively related to increases in passage rates but did not reach significance. The initial status of a school's passage rate in reading and math was not significantly predicted by any variable. See http://rapidintellect.com/AEQweb/fal2006.htm

Next, we added two variables to the model discussed above (average years of teaching experience and the percentage of non-white students) and eliminated the variable representing teacher's confidence that their efforts were responsible for student successes since this was not a significant predictor. Teacher experience was not related to the initial status or to growth. The percentage of non-white students was significantly related to the initial status in reading (beta = -0.24, p < 0.01) and in math (beta = -0.32, p < 0.01), indicating the achievement gap on the test. However, the percentage of non-white students was not significantly related to the growth rate, i.e. schools with higher or lower percentages of non-white students did not progress differently during the period of reform. The variable representing teachers' confidence in helping challenging students and the variable representing school based reforms in the first model both remained significant in the second indicating that significance could not be attributed to teacher experience or ethnic make-up of the school: Teacher-controlled reforms in reading (beta = 0.06, p < 0.01) and in math (beta = 0.07, p < 0.01), confidences with difficulties in reading (beta = -0.08, p < 0.05) and in math (beta -0.10, p < 0.01). See issue website http://rapidintellect.com/AEQweb/fal2006.htm

Surveys allowed teachers to volunteer for interviews, and 231 teachers from 40 schools volunteered. The interviews were designed to shed light on the finding that more confidence helping difficult students was negatively related to rising test scores. Therefore, two schools with above average ratings of teacher confidence in helping difficult students and three with below average were chosen, and an attempt was made to contact all volunteers in those schools. Ultimately, 10 teachers participated. All ten teachers who responded were female, which is a limitation of the study; however, most teachers in the district are female. In addition, racial information on these teachers was not collected, which is another limitation of the study. Years of experience ranged from 5 to 29. Most teachers taught all subjects, but both regular and special education teachers were represented. See http://rapidintellect.com/AEQweb/fal2006.htm

Interview questions focused on characteristics of "difficult" students, the impact of reforms on these students, and the reasons teachers believed teacher confidence in helping such students related to less progress on state tests. Data were analyzed using a constant comparison method (Bogden & Biklen, 2003; Glaser & Strauss, 1967). Comments will be attributed to participants using the codes in Table 3, so that readers can connect teachers' demographic information to their comments.

Consensus emerged around the characteristics of difficult students. Most teachers (1,3,4,5,6,7,8,9) defined difficult students as behind academically. As one teacher described, "They're frustrated, and they just want you to kind of leave them alone, because they feel like they're never going to catch up" (4).

Teachers also saw difficult students as behavior problems (3,8,9,10). Behavior problems, however, often overlapped with learning problems. For example, one teacher explained, "I think you have kids who have behavior issues, and I think also these kids have difficulty with learning, so I guess I think of it both ways" (3).

Finally, a smaller number of teachers (2,6,7) said poor parenting and/or disadvantaged backgrounds characterized difficult students. As with behavior problems, however, poor parenting and/or disadvantaged backgrounds coexisted with learning difficulties according to two teachers (6,7). As one noted, "Most children who are labeled difficult in Cleveland have been underexposed to what would give them success in school" (6). So, while behavior problems and difficulties due to home life do appear to be present, learning problems appear to be characteristic of difficult students in most teachers' views.

When asked what reforms had been fully implementing in the past five years, all but one teacher (1,2,3,4,5,7,8,9,10) mentioned implementing standards. However, several of these teachers noted that despite faithful implementation, standards-based teaching was not always effective. One teacher explained, "There's not necessarily enough time because there are too many standards and there're such high expectations of what the kids will achieve" (7).

Two special education teachers found standards-based reform especially unhelpful, because the standards were inappropriate for students. One explained, "You've got the standard here, and you've got the student's skill and ability level down someplace else ... Now try and pull them up to that" (5).

As with other findings, a pattern emerged when teachers' explained why confidence in helping difficult students related to slower improvement on passage rates on state tests. On this subject, however, no clear majority consensus appeared. Instead, four teachers (4,5,6,8) said that passage rates on state tests do not capture the progress of difficult students because these students are so far behind. The secondly most common response was disbelief (2,9). Other explanations included over confidence causing teachers to "wait until the last minute" to teach standards (I), causing teachers to "blame the students" if the students fail to learn (3), and leading teachers to shun "workshops" and "input" from other teachers (7). One teacher also said that confident teachers might have been undermined by parents who did not or could not support their child's learning at home (10).

Conclusion

This study found that teachers' reports of effective implementation of district-led, teacher-controlled reforms were positively related to improvement in passage rates on state tests, while teachers' confidence in their ability to help difficult students was negatively related to improvement. However, teacher interviews conducted in the fall of 2004 shed more light on these perplexing findings. Most teachers defined difficult students as those having trouble academically. Several teachers noted that the recent, standards-based reforms were not helpful for some academically-low students. Explanations of the relationship between teachers' confidence helping difficult students and passage rates varied. However, four teachers asserted that passage rates on state tests do not capture the academic progress of these difficult students who are far behind academically. It may be that teachers who are confident about helping academically-low students, focused on these students and helped them, but the students never make enough progress to pass state tests. However, it may be that one of the less frequently mentioned explanations is correct, or that none of these explanations fully accounts for this phenomenon.

While qualitative findings do not fully explain the qualitative findings, they may be important in making a recommendation. However, this suggestion is tentative due to the limitations of this study. The number of schools included in the quantitative analysis is 59, making the power to find significant variables adequate but weak. In addition, no men participated in the qualitative portion of the study, and we did not ask the race of participants.

Despite limitations, it is clear that many teachers embraced state standards and were confident in their ability to help difficult students. Was the state measuring and, thus, rewarding and building on these efforts? As pressure to improve academic achievement in high-poverty, high-minority schools increases, it seems that Ohio's measurement system may not have been capturing progress. Some teachers interviewed did not trust the test. The U.S. Department of Education has recently allowed Ohio to test a "growth model" which will track students longitudinally. This new model may reveal a more complex relationship between teacher efficacy and student achievement and deserves future research. Therefore, it is recommended that future studies examining the relationship between teacher efficacy and student achievement use growth models for analyses.

References

Bandura, A. (1977). Sell-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215.

Bandura, A. (1978). Reflections on self-efficacy. Advances in Behavioral Research and Therapy, 1,237-269.

Barth, P., Haycock, K., Jackson, H., Mora, K., Pablo, R., Robinson, S., & Wilkins, A. (Eds.). (1999). Dispelling the myth: High poverty schools exceeding expectations. Washington DC: Education Trust.

Bogden, R.C., & Biklen, S.K. (2003). Qualitative research for education: An introduction to theory and methods, (4th ed.). Boston: Pearson Education Group.

Carter, S.C. (2000). No excuses: Lessons from 21 high-performing, high-poverty schools. Washington DC: Heritage Foundation.

Dolejs, A., & Jenkins, R, (Eds.). (2001). Opening doors: Promising lessons from five Texas high schools. Austin: Charles A. Dana Center.

Edmonds, R.R., (1979). Effective schools for the urban poor. Educational Leadership, 37(1) 15-24.

EdSource. (2006, June). Similar students, different results: Why do some schools do better? Mountain View, CA: Author.

Fuchs, L.S., Fuchs, D., & Bishop, N. (1992). Instructional adaptation for students at risk. Journal of Educational Research, 86, 70-84.

Gibson, S., & Dembo, M. (1984). Teacher efficacy: A construct validation. Journal of Educational Psychology, 76(4), 569-582.

Garcia, D.C. (2004). Exploring connections between the construct of teacher efficacy and family involvement practices: Implications tier urban teacher preparation. Urban Education, 39(3), 290-315.

Glaser, B.G., & Strauss, A.L. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine Publishing Co.

Guskey, T.R., & Passaro, P.D. (1994). Teacher efficacy: A study of construct dimensions. American Educational Research Journal, 31, 627-642.

Labone, E. (2004). Teacher efficacy: Maturing the construct through research in alternative paradigms. Teaching and Teacher Education 20(4), 341-359.

Moore, W., & Esselman, M. (1992, April). Teacher efficacy, power, school climate and achievement: A desegregating district's experience. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco. No Child Left Behind Act of 2001, Pub. L. No. 107-110, & 115 Stat. 1425 (2002).

Raudenbush, S.W. & Bryk, A.S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods. Thousand Oaks, CA: Sage Publications.

Ross, J.A. (1992). Teacher efficacy and the effect of coaching on student achievement. Canadian Journal of Education, 17(1), 51-65.

Ross, J.A., Cousins, J.B., Gadalla, T. (1996). Within-teacher predictors of teacher efficacy. Teaching and Teacher Education, 12(4) 385-400.

Taylor, B.O. (2001). The effective school process: Alive and well. Phi Delta Kappan, 83(5) 375-378.

Teddlie, C. & Reynolds, D. (2000). The International Handbook of School Effectiveness. New York: Falmer Press.

Trimble, S. (2002). Common elements of high performing, high poverty middle schools. The Middle School Journal, 33(4)7-21.

Tschannen-Moran, M., Hoy, A.W., & Hoy, W.K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research, 68(2) 202-248.

Wheatley, K.F. (2002). The potential benefits of teacher efficacy doubts for educational reform. Teaching and Teacher Education, 18 5-22.

Wheatley, K.F. (2005). The case for reconceptualizing teacher efficacy research. Teaching and Teacher Education, 21 747-766.

Guofang Wan, Ohio University 

Guofang Wan, Ph.D., is associate professor of education at Ohio University. She has published research articles and books on media literacy education.