What matters most in closing the gap: some of the differences between schools that have closed the gap and those that have not run counter to general expectations, this study found.

The achievement gap between white or Asian students and their Hispanic/Latino or African-American peers is troubling and persistent. Even when looking at students from similar socioeconomic backgrounds, a gap exists based on race. In this 50th anniversary year of the landmark Brown vs. Board of Education ruling, it is imperative that we examine what's working and what's not.

Over the course of the 2002-2003 school year, the Bay Area School Reform Collaborative, a non-partisan, non-profit organization based in the San Francisco Bay Area, set out to investigate the question, "What set of school-level policies and practices contribute to closing the achievement gap?" The findings paint a clear picture of what's working in a set of Bay Area schools, some of which runs contrary to conventional wisdom and challenges underlying beliefs about what matters in education.

What common assumptions did our study challenge?

In what follows, we sketch our study design to shine the light on four widely held beliefs about "what matters most in closing the gap" that--contrary to our own expectations--were not confirmed, but instead brought into question by our study.

Who's closing the gap?

During the 2002-2003 school year, BASRC collected data through a survey of 32 K-8 San Francisco Bay Area schools and conducted case studies of three schools that were successfully narrowing the achievement gap--specifically, narrowing their gaps between 49 to 86 percent over the time period studied. The survey, designed with help from Stanford's Center for Research in the Context of Teaching, asked questions on a variety of school-level policies, such as school focus, professional development, role of leaders and use of assessments.

We asked three teachers from each school to respond and received surveys back from all 32 schools, with a final total of 82 individual surveys.

To analyze the data, we divided the schools into two groups: "gap-closing" and "non-gap-closing," and compared the results. We used California's Academic Performance Index ranking system, which measures how school subgroups perform on the state's standardized testing package. We looked at data over a four-year period, from the 1998-1999 school year through the 2001-2002 school year. We specifically looked at the racial or ethnic achievement gap, comparing the performance of white or Asian students and their Hispanic/Latino or African-American peers.

We defined "gap-closing" schools as those in which both the top-performing and the bottom subgroup improved, but the bottom did so at a greater rate. Non-closing schools were defined as schools that had widened the gap over the four-year time period, with the high-performing subgroup making more progress than the low-performing subgroup.

At the three case study schools, we conducted interviews with principals, vice principals, at least three teachers per site and other key individuals. We also conducted walk-throughs of all classrooms, and observed individual teachers and staff meetings.

Belief #1: Testing does more harm than good

Some believe that testing only does harm, that testing only confirms what we already know, and that it punishes low performers--whether schools, or worse, children. What we found, however, is that schools that use diagnostic data as part of a continuous improvement process--reflecting, analyzing and altering strategies accordingly--have more success at closing the achievement gap. Teachers in gap-closing schools also use data frequently as a tool to hone their instruction.

Gap-closing schools reported using data more frequently than non-closing schools. About two-thirds (64 percent) of respondents flora gap-closing schools use data to understand the skill gaps of low-achieving students a few times a week or a few times a month. In contrast, more than three-quarters (79 percent) of teachers in non-closing schools use data in this way only a few times a year or never.

Teachers at gap-closing schools also use assessments more frequently. Almost all (91 percent) administer ongoing assessments of students at least monthly, with more than half (55 percent) administering assessments a few times a week. In contrast, only 32 percent of respondents from non-closing schools administer assessments a few times a week.

In addition, teachers in gap closing schools are more likely to discuss low-performing student achievement data with colleagues. More than three-quarters (77 percent) of respondents from gap-closing schools discuss data at least a few times a month, with about one-third (32 percent) talking about data a few times a week. Respondents in non-gap-closing schools discuss data far less frequently--just about one-half (47 percent) only discuss data a few times a year.

These findings are evidenced in our case study schools. In particular, Belle Air Elementary, a K-6 school that's about two-thirds Hispanic/Latino, uses frequent diagnostic assessments.

Importantly, Belle Air teachers are given time on a weekly basis in grade-level, cross-grade-level and full-staff groupings to analyze, discuss and modify strategies based on data.

By principal and teacher report, this infrastructure helps teachers see using data as an integral part of their job, not an add-on.

Our findings don't suggest that testing is a "silver bullet" solution, but they do suggest that regular testing can provide important information, and when teachers are given time after the test to analyze, reflect and revise strategies, testing can lead to positive change.

Belief #2: Focusing on one group means ignoring the rest

Some believe that focusing on one group of kids comes at the cost of the rest of the students. When a school tries to close its achievement gap and so begins devoting time and resources to the lowest performing students, some members of the school community may resist. Teachers feel it runs contrary to their mission, which is to teach all kids. Parents--particularly those of academically successful students--feel this focus is in opposition to their child's needs.

Our research, however, reveals that a focus on a low-performing group can benefit all kids. The focus can help teachers become more adept at differentiating instruction and can prompt the school community to address deficiencies in school services and the academic program for all student groups.

Case studies illuminate the benefits of focusing on one student subgroup. Belle Air Elementary has been making consistent student achievement gains between 1999 and 2002 for all student groups. The school closed its achievement gap by 52 percent, or by 60 API points. Hispanic/ Latino students improved API scores by 86 points, or 14 percent, while white/ Asian students improved API scores by 26 points or 4 percent.

How are they doing it? Belle Air Elementary looked at its student achievement data and identified its lowest performing subgroup--Hispanic/Latino boys. Each teacher then selected an Hispanic/Latino boy from each class as her "focal student" and paid special attention to her interactions and support for this student. Teachers reported that this practice not only helped these focal students improve, but also helped them become better teachers, more adept at differentiating instruction.

Case study school Roosevelt Middle also benefited from a focus on its lowest-performing subgroup: African-Americans. Looking at the subgroup's performance on the Gates MacGinitie assessment, it became clear that reading was a significant weakness. This prompted the faculty to re-evaluate how the school taught reading. The principal gave the staff time to look at reading scores, both for African-Americans and all other subgroups. Faculty could then see for themselves how low reading levels really were, and from that the principal gained support and buy-in to radically alter the daily schedule.

As a result, all students now have three reading classes every day: English class, which has a standard language arts curriculum; reading class, giving explicit instruction in how to best acquire, analyze and understand text; and literature class, where students generally read a book of their choice, though some teachers do journal writing or perform plays.

Four years after making these changes, not only have African-Americans improved their reading levels--closing Roosevelt's achievement gap by 102 points, or 49 percent--but two to three times as many students school-wide are now at or above grade level in reading.

Belief # 3: Teachers' attitudes and beliefs matter more than anything else

The third belief our findings challenge is that teachers' attitudes and beliefs matter more than anything else. Some believe that educators simply don't either have enough faith in their students, or that they don't think that all children can learn and therefore they lack the will to foster equal outcomes in student achievement. This assumption is part of what leads to policies using sanctions, assuming that teachers need more motivation to work with low achieving students.

The research, however, revealed no significant differences between gap-closing and non-gap-closing respondents regarding attitudes and beliefs. Instead, almost all of the significant differences emerged around how teachers were supported to use data. This has little or nothing to do with subjective nuances of belief systems, and everything to do with concrete realities of resources and time.

We asked survey respondents a variety of questions on attitudes and beliefs and found no significant differences on any of the items.

* When asked if it is possible to close the achievement gap, the vast majority in both groups--95 percent in gap-closing and 93 percent in non-closing schools--agreed.

* When asked, "Do low-achieving students have the potential to meet or exceed grade-level standards?" 98 percent of gap-closing respondents and 100 percent of non-closing respondents agreed.

* We saw the same kind of results when we asked respondents if they felt closing the gap was a personal priority. Clearly, from our survey, we saw no discernable pattern regarding will, or belief in students' potential.

Teachers at gap-closing schools also received more professional development about linking data on low performing students with effective instructional strategies. Twice as many gap-closers (50 percent) said they received professional development on this topic on a monthly of a weekly basis, compared to 26 percent of non-closers. Three-quarters (74 percent) of non-closers reported receiving professional development on this a few times a year or never.

Leaders in gap-closing schools are far more likely to encourage or lead inquiry into the achievement gap. Every single respondent (100 percent) from a gap-closing school agreed that their school leaders do this, with 82 percent strongly agreeing. In contrast, just under half (42 percent) of non-closing respondents somewhat agreed and about one in four (21 percent) disagreed.

Attitudes and beliefs do certainly matter. And they're difficult to measure--this survey alone certainly isn't enough to make any conclusive judgments. Our findings, however, do call into question the wisdom of relying solely on strategies to change beliefs if it means omitting concrete steps that schools can take to provide teachers with support to do their jobs better.

Belief #4: Don't talk about race

Our findings suggest that race matters. When reality is decidedly color-conscious, with African-American and Hispanic/Latino children languishing far behind their white and Asian peers, color blindness appears to compound the problem. Race is an explosive issue. Talking about it and taking action with particular groups of students in mind can be controversial or counter-intuitive. However, the findings of this study strongly suggest that addressing race is essential to narrowing the achievement gap in our schools.

According to the survey findings, schools in which teachers have structured time to discuss the complexities and impacts of race/ethnicity on school, self and students have more success closing the gap. Our research clearly pointed to the importance of leaders explicitly encouraging reflection on race/ethnicity and equity.

Leaders in gap-closing schools are more likely to provide structured opportunities for faculty to discuss race and ethnicity. Nine out of ten (91 percent) respondents from gap-closing schools agreed that they had these opportunities, while about two out of three (63 percent) respondents from non-closing schools said they did not have the opportunity.

Respondents in gap-closing schools are more likely to agree that closing the achievement gap between white/Asian students and Hispanic/Latino or African-American students is a primary goal for school leaders. Eighty-six percent of respondents from gap-closing schools strongly agreed their leaders held closing the gap as a priority, compared to 58 percent at non-gap closing schools.

Gap-closing schools are more likely to have developed and agreed upon an explicit definition of equity, with more than three-quarters (82 percent) of respondents from gap-closing schools agreeing with this statement, and just under half (41 percent) strongly agreeing. In non-closing schools, the majority (58 percent) disagreed that their school had a definition of equity, with more than one quarter (26 percent) strongly disagreeing.

An anecdote from case study school Roosevelt Middle illustrates these findings. When Roosevelt's principal came to the school in 1997 she was concerned with low African-American achievement. She looked more closely at the school's suspension data and discovered that over the course of the previous year, 60 percent of African-American students had received suspensions. In other words, three out of every five African-American students were missing valuable classroom time due to the discipline system.

After this discovery, the principal made time for faculty to delve into some heated, honest, data-based discussions about race and their discipline system. She gave teachers plenty of time to look at the numbers and come to their own conclusions about why it was happening. This involved talking about both teachers' own racial/ethnic identities, and the racial component to some of the patterns in the interactions between African-American students and teachers.

The principal brought in external facilitators for these conversations, and also invested in professional development for teachers regarding classroom management styles. Several teachers also cited the value of an in-service from a veteran Roosevelt teacher on classroom management, specifically with African-American students.

As a result of the intensive focus, Roosevelt teachers became aware of the bias in their system and changed it. The number of African-American students suspended decreased by more than two-thirds, falling to 18 percent in 2002-2003. The total suspension rate went down as well, though at a slower rate. This means that the school reduced the over-representation of African-American children being suspended.

Inquiry and scrutiny

When we began our study, we clearly expected to find differences between gap-closing schools and non-closing schools. We did not, however, expect the research to run so profoundly counter to our expectations about what matters most.

Honesty, reflective inquiry and ongoing scrutiny of the data are indeed the three critical keys.

How frequently do you discuss low-performing
student achievement data with colleagues?

                        gap     non gap
                      closers   closers

A few times a year      23%       47%
A few times a month     45%       48%
a few times a week      32%        5%

Note: Table made from bar graph.

How frequently do you receive professional development on linking
low-performing student data to instructional strategies?

                        gap     non gap
                      closers   closers

Never                    5%       11%
A few times a year      45%       63%
A few times a month     45%       26%
a few times a week       5%        0%

Note: Table made from bar graph.

Leaders at our school provide structured opportunities for faculty
to discuss race and ethnicity

                    gap closers   non gap closers

Strong disagree      0%              42%
Somewhat disagree    9%              21%
Somewhat agree      77%              16%
Strongly agree      14%              21%

Ida Oberman is director of research for the Bay Area School Reform Collaborative. Kiley Walsh Symonds, former BASRC policy director, is currently a consultant.