School practices that matter: this study proves that what schools do, and what resources they have for doing it, can have a powerful impact on student achievement.

Why do some California elementary schools serving largely low-income students score as much as 250 points higher on the state's Academic Performance Index than other schools with very similar students?

That's the research question asked by a recently released large-scale study that surveyed principals and teachers in 257 such schools across the state. What we learned is that the higher-performing schools tend to have four interrelated practices at the core of their operation--prioritizing student achievement; implementing a coherent, standards-based curriculum; analyzing student assessment data from multiple sources; and ensuring availability of instructional resources.

Many studies have examined successful schools as a group, in an effort to understand their methods or best practices. This study--conducted by EdSource and researchers from Stanford University, the University of California, Berkeley and the American Institutes for Research--took a different tack. Rather than looking at a specific performance band, we examined elementary schools within a specific, fairly narrow socio-economic and demographic band but across the full range of school performance.

Specifically, we surveyed 257 of the 550 elementary schools that fall within the 25th to 35th percentile band of California's School Characteristics Index. Schools in this band have high percentages of English learners and students from low-income families. Our outcome variable was the school's API. We surveyed all principals as well as 80 percent of the K-5 classroom teachers, giving us 257 principal surveys and more than 5,500 teacher surveys.

Our surveys included more than 300 items each, covering seven broad domains often associated with effective schools. The questions reflected California's current standards-based reform environment, asking specifically what principals and teachers were doing and how frequently.

Once the data was collected, we used by Trish Williams and Michael Kirst a weighted analysis to assure that results were statistically representative of all public non-charter elementary schools in the 25th to 35th School Characteristics Index band. We then used several types of regression analyses to determine which school practices differentiated the highest from the lowest performing schools.

What we discovered was that school API scores are highest when the entire team--teachers, principals and district leaders--focuses strongly on helping students meet state standards. It's clear that socioeconomic factors are not the sole predictor of academic performance. What schools do and what resources they have for doing it can have a powerful impact on student achievement.

What the study found: School practices that matter

We've all heard people say that you can guess a school's API score by the zip code of its students. But the 250-point difference in API between schools serving similarly disadvantaged students shows that's not true. What do the higher-performing schools do differently? From the school district through to the classroom, they focus on improving student achievement against the state's academic standards, and they align all their practices around that goal. The closest correlation with a high API occurred among schools where teachers and principals gave strongly positive answers to the questions in these four domains:

1. Prioritizing student achievement. Where teacher and principal answers to multiple survey questions indicated higher expectations for students, their schools had, on average, higher API scores. In more successful schools, teachers and principals alike report that their school has well-defined plans for instructional improvement and that they put priority on meeting the state's API goals and the federal Adequate Yearly Progress goals. Teachers and principals also report that their schools set measurable goals for exceeding the mandated API student subgroup growth targets for improved achievement.

2. Implementing a coherent, standards-based curriculum and instructional program. Teachers who report the following were more likely to be in higher performing schools: instructional consistency within grades; curricular alignment from grade to grade; classroom instruction guided by state academic standards; curriculum materials in math and language arts aligned with the state's standards; and working in a district that addresses the instructional needs of English learners at their school.

Principals were more likely to be in higher performing schools if they report that the district has clear expectations for student performance aligned with the district's adopted curriculum, and the district evaluates the principal based on the extent to which instruction in the school aligns with the curriculum.

3. Using assessment data to improve student achievement and instruction. Strongly correlated with a higher school API was the extensive use of student assessment data by the district and the principal in an effort to improve instruction and student learning.

For example, principals more often report that they and the district use assessment data from multiple sources (including frequent curriculum program tests and annual California Standards Tests) to evaluate teachers' practices and to identify teachers who need instructional improvement. Principals report using this data to develop strategies to follow up on the progress of selected students and help them reach academic goals.

According to these principals, the district expects all of its schools to improve achievement, evaluates principals based on student achievement, and provides support for site-level planning related to improving achievement.

4. Ensuring availability of instructional resources. Where more teachers report having regular or standard certification for teaching in California, schools had, on average, higher API scores. The same was true of schools where principals more often reported that their districts provide sufficient and up-to-date instructional materials as well as support for supplementary instruction for struggling students and for facilities management. Teachers with at least five years of full-time teaching experience were more likely, on average, to be from schools with higher APIs. Principal experience was also correlated with higher school achievement.

The survey also included questions related to schools' efforts to involve parents, teacher collaboration and development, and the enforcement of high expectations for student behavior. Although each of these types of practices made some contribution to a school's API score and were likely practiced by most of the schools in the sample, they were not nearly as strongly correlated with higher school performance as were the four key interactive, interdependent school improvement practices described above.

Interpreting the findings: Leadership makes the difference

In reflecting upon the study's findings, it is clear to the authors that across the four critical interrelated practices, school and district leaders emerged as key actors. What happens in the classroom between teacher and student will always be at the heart of student learning. But school district and school site leadership are pivotal in creating the kind of focused, positive, accountable environment that fosters improved teaching and learning.

1. District leadership provides the support schools need to emphasize and ensure improved student achievement.

Principals' answers to the survey suggest that an important factor in getting their schools focused on state standards and student achievement is a no-excuses approach from district leaders. The message from the district is unequivocal: schools will meet API and AYP growth and subgroup targets. Districts back up those expectations by evaluating principals' and teachers' practices based on an analysis of student test score data.

However, the district doesn't just hold the school accountable. It also provides the structure, resources and support schools need to succeed. For example, principals in the highest-performing schools report that the district ensures that the school's math and language arts curricula are aligned with state standards and that instruction is focused on student achievement.

Teachers at the school report that the district provides them with information about curriculum standards. In some instances, this may mean the district determines the curriculum programs it believes are most closely aligned with state standards and expects all schools in the district to adopt them.

In addition, these districts provide assistance for site-level planning related to improving learning; address the instructional needs of EL students; and ensure that their schools have adequate facilities, current textbooks for every student, and resources for struggling students.

2. Principal leadership is being redefined to include extensive use of student assessment data to evaluate and intervene.

In California's standards-based context, successful elementary principals report that meeting their school-wide API and AYP growth and subgroup goals is a clear priority. But their focus is not on test preparation. These principals have embraced the state's academic standards. They ensure school-wide instructional alignment with the standards via the curriculum program. Consequently, these principals are more likely to be regarded by teachers as knowledgeable about standards and curriculum.

Under the principal's leadership, the school develops well-defined plans for instructional improvement, and teachers map standards to lesson plans, examine the scope and sequence of curriculum topics, and review a grade-level pacing calendar.

Our most striking finding about the role of principals was the level of direct and hands-on use of student assessment data they reported. Principals in the highest-performing schools are using data from multiple sources (the California Standards Tests as well as curriculum program and district-developed assessments) to manage the school improvement process. Moreover, they are using the data in an impressive variety of ways to diagnose student learning problems and to enact changes in instructional approaches.

Examples include:

* Principals ensure that teachers receive CST and CAT-6 data for all students in their classrooms and across their grade level, disaggregated by specific skill or academic content and by student subgroup.

* Principals meet with teachers to review these data, along with teachers' benchmark assessments, to develop strategies to help selected students reach goals.

* Principals are also using test data to evaluate teacher practices and identify teachers who need instructional improvement.

Overall, it appears that principals in the higher performing schools are using data to nurture a school-wide culture of standards. Data help drive instructional priorities. Everyone can see which students are at or below basic levels of achievement, and the data provide evidence about which teaching strategies are helping move those students to proficiency or above.

Good teaching has always been the linchpin of student success. Continual data analysis in higher-performing schools appears to help define the ingredients of a teaching staff's effectiveness.

In higher-achieving schools, principals reported that a higher proportion of teachers had (in order of priority): a demonstrated ability to raise student achievement; strong content knowledge; characteristics that made them a good fit with the school culture; training in curriculum programs; and the ability to map curriculum standards to instruction.

These teachers were also supportive of colleagues' learning and improvement; able to use data from student assessments; familiar with the school community; excited about teaching; and familiar with state academic standards.

One widely held belief about the teaching workforce is that the most qualified teachers will choose to work in the "easier" schools. But in our sample of high-challenge schools, the highest performing have attracted teachers with highly valued qualities and competencies. One explanation could be that having a challenged student population and enforcing high professional and academic expectations are not impediments to attracting good teachers if the district and school site leadership is strong, cohesive and supportive of the importance of the work.

Summary of interviews still to come

A follow-up to the initial findings report will include a summary of the interviews conducted with 21 school district superintendents in 18 school districts as well as a summary of principals' responses to an open-ended question. Both the interview and the open-ended question focused on these leaders' perceptions of the strategies that have most improved student achievement. These new results, as well as those summarized in this article, will be covered in an upcoming EdSource report.

The initial findings report, entitled "Similar Students, Different Results: Why Do Some Schools Do Better," is available at www.edsource.org, along with the technical appendices and copies of the surveys. The upcoming EdSource report on our follow-up analyses will be sent to EdSource subscribers and participating schools and will also be made available online.

Trish Williams, the study's project director, is executive director of EdSource. Professor Michael Kirst of Stanford University, principal investigator, is a founder and former co-director of Policy Analysis for California Education (PACE).