The revelations of value-added: an assessment model that measures student growth in ways that NCLB fails to do.

In the No Child Left Behind era of high-stakes testing, school administrators are facing their toughest challenge ever. They are being held accountable for the performance of their schools, yet current systems in public education typically fail to provide them with the appropriate tools to manage effectively.

Although the classroom is where learning takes place, superintendents and principals often know precious little about what is happening within them. As never before, administrators need the means to measure and evaluate the impact of curricula, new practices and professional development on academic achievement. For unless the quality of classroom instruction and programs can be improved significantly, students will be unlikely to meet the high standards now required.

Fortunately, significant help is available in the form of a relatively new tool known as value-added assessment. Because value-added isolates the impact of instruction on student learning, it provides detailed information at the classroom level. Its rich diagnostic data can be used to improve teaching and student learning. It can be the basis for a needed improvement in the calculation of adequate yearly progress. In time, once teachers and administrators grow comfortable with its fairness, value-added also may serve as the foundation for an accountability system at the level of individual educators.

Identifying Contributers

Used by a growing number of states, value-added assessment provides a new way to measure teaching and learning. Value-added uses the annual test scores that are now being collected for students and analyzes them to reveal the progress students are making each year. In its focus on growth rather than solely on levels of absolute achievement, value-added broadens our understanding of the contribution instruction makes to student learning. While family income remains the best predictor of absolute achievement, good instruction is 10 to 20 times more powerful in predicting student growth.

By following individual students over time, value-added accounts for student background characteristics over which schools have no control and that tend to bias test results. And in what is perhaps its most unique contribution, value-added enables educators and the public to identify not only the progress made by students but also the extent to which individual teachers, schools and districts have contributed to it.

Under the value-added approach, test scores are projected for students and then compared to the scores they actually achieve at the end of the school year. Classroom scores that exceed projected values suggest that the instruction is highly effective. Conversely, scores that are mostly below projections suggest that the instruction was ineffective.

But at the same time this approach considers student-related factors, such as the pattern of prior test scores, both those of the individual student as well as those of other students in the same class. If a student's present performance is below projected scores while students with comparable previous academic history in the same classes have done well, this is evidence of the student effect--external variables such as the home environment that are beyond the control of teachers and schools--that can be ruled beyond the range of a teacher's influence.

Value-added provides educators with two patterns of instruction that characterize their classrooms: which students are the focus of their instruction (previously low, average or high achieving) and how effective their instruction has been in providing students with a year's worth of growth from wherever they started in September.

Armed with these data, teachers can meet regularly to discuss how to change their instructional patterns in desired directions. The ensuing conversations end the isolation of teachers and teaching and, when coupled with strong instructional leadership to guide educators through a process of study, schools can be transformed into true learning communities.

While adequate yearly progress measures show only a vague snapshot of performance from year to year and offer no information about where strengths and weaknesses lie, value-added methodologies, characterized by a focus on individual students and longitudinal cohorts, are instead helping educators close the achievement gap, manage and evaluate innovations and improve teaching and learning. The following examples illustrate how superintendents in three different school districts have used value-added to transform their instructional practice and raise student achievement.

Addressable Issues

Value-added assessment has been in place in Tennessee since the early 1990s when it was created by William Sanders, who was then a statistician at University of Tennessee. Schools use the Tennessee Value-Added Assessment System to identify where their achievement gaps exist and address student needs more expeditiously. One example is the Maryville Middle School in Maryville, Tenn., whose principal, Joel Giffin, used value-added data to improve the school's academic program and help underperforming students make significant gains.

Giffin, now in his 33rd year at the school, believes the analysis and disaggregation of value-added data allowed him to see that 7th grade math performance was not what it should have been for 20 of the lowest-achieving students. The statewide benchmark was 15 scale score points, while these students made a 12.5 scale gain in their test scores. Scale scores are common units of measure on a standardized test.

Giffin and his staff identified five characteristics shared by these students: 1) low socioeconomic status; 2) one-parent families; 3) lack of adults at home at the end of the school day; 4) lack of money for school supplies; and 5) lack of help or positive reinforcement with homework.

"While we recognized that we could not change one through three," Giffin says of the five factors, "we felt we could change four and five. Four was simple. Money was donated once people understood the need. Five was harder because it meant we had to do things differently."

The principal's mission, then, was to change the school to fit the needs of the students. "We created during school time the same support for these students as we would our own kids. We added a daily math class to help them complete homework, provided tutoring, gave encouragement and praise and made sure that they went to class prepared and feeling confident," he says.

The results were outstanding. The value-added gains of these students were 360 percent of the national norm compared to the state benchmark of 100 percent of the national norm. This was the greatest gain ever of any group of students at Maryville Middle School.

Indeed, through Giffin's use of value-added and the myriad of reforms he implemented, the Maryville Middle School now ranks as Tennessee's top-scoring middle school on value-added assessment. The 10-year schoolwide value-added score for the school is 144 percent of the national norm.

Comparing Programs

In Ohio, a value-added pilot project was started two years ago with funding from the Battelle for Kids Foundation. Seventy-eight school districts now are using value-added assessment to better manage program innovations.

James Mahoney, a former school superintendent who leads the pilot, says value-added assessment has been used in multiple ways by the participating districts. (See related story, page 16.)

In one school district in Westerville, Ohio, value-added measures showed that two elementary schools were consistently outperforming the others in 4th grade science. The superintendent, George Tombaugh, visited the two schools to talk to the 4th grade teachers and discovered that because none of these teachers was particularly strong in science, they decided that each would specialize in only one segment of the curriculum. The teacher with the comparatively greater subject expertise then taught that segment to all the 4th grade students. Tombaugh now is exploring the possibility of replicating this practice districtwide. Another Ohio district, Riverview Local Schools, piloted a new 4th and 5th grade math program in two of its elementary schools. They used the district's value-added measures to assess the growth of students in this program as compared to the growth of students in the old program to determine which program was more successful.

A third Ohio district, Miami East, was interested in alternative programs for its elementary schools and investigated programs that worked for students with similar characteristics. They used the value-added "school search" feature to find similar schools that were getting high levels of progress. When they discovered they were getting equal progress to the other schools, they decided to keep their current program.

Measuring Impact

Sharon Kirk, superintendent of the DuBois Area School District in Dubois, Pa., described how her 4,500-student district is making advantageous use of value-added in testimony before the state Senate Education Committee, saying, "The best part of value-added is its measure of teacher and school effectiveness and the resulting potential for increased student achievement."

The DuBois district was among the original 32 Pennsylvania districts in 2002 to implement value-added. The state board of education quickly saw the benefit and voted to require its use in all 501 districts statewide by 2007.

Kirk says she has been impressed by the difference between value-added measures and No Child Left Behind's adequate yearly progress measurement.

In her testimony to state legislators in March 2004, she used a simple analogy: "Imagine a busload of children arriving at a park to play baseball. The aim of the event is to teach every child to hit a triple. As the children arrive their individual abilities are very different. Some children can already hit a double while others are having difficulty even managing the steps on the bus. All the children make progress during this event. Many of the students who could hit a double now hit triples. Some of the students who had trouble with the steps on the bus have worked very hard and can now hit a double.

"How would you gauge the most successful students, or who made adequate progress? If hitting a triple is the measure of success, students with the least progress may be considered the only successful ones even though they have made little progress because they started so far ahead. Those children who worked hard and perhaps made the most progress will be judged as failures."

Making this distinction is important to teachers and one reason why teachers in DuBois have come to trust value-added measures.

This trust helps when it comes time to measure the effectiveness of particular teaching practices or programs. Kirk says she and her staff developed a districtwide instructional improvement plan after reviewing two years of school-level data. The value-added assessment system provided each school with data about progress using proficiency levels and quintiles for math and reading.

In one DuBois school this data revealed that two different cohorts of students were performing in an identical pattern in math for two years in a row. "What that told us is that with two entirely different groups of students we were getting similar achievement. In both years we were failing our top students. Their scores were well above average and certainly proficient but their progress was not," Kirk says.

This information provided the impetus for change. The principal of Dubois Middle School, Daniel Hawkins, minces no words in his praise of the new approach. "Value-added has been the No. 1 most positive motivator for my staff as well as for me.... It has been a catalyst in our schoolwide improvement program."

The DuBois Area School District has eight elementary buildings, some highly successful and above the norm. Kirk believes that the only way you can have 100 percent proficiency is to implement sound instructional practice, which can be greatly enriched by value-added assessment.

Likely Impact

Value-added assessment by itself does not improve student achievement. As these examples illustrate, only when educators understand its power and use what they learn to guide instruction and professional development will schools begin to see significant learning gains in their students.

More educators and policymakers are recognizing the benefit of value-added assessment. In addition to Tennessee, Ohio and Pennsylvania, new legislation in Arkansas and Minnesota calls for implementing a form of value-added meassurement, and the state school boards associations in Iowa and New York are piloting a value-added program this year. Dallas and Seattle are the most prominent urban districts that use the value-added approach, along with several smaller districts in Colorado, North Carolina and Florida.

In coming years, value-added's increasing popularity may result in three significant changes for public schools nationwide.

First, we will see the production of subject and grade-specific professional development modules (what are the key questions to ask and the best readings to pursue) to help educators change their instructional practices to improve student learning. Parallel efforts using simulated data will be undertaken in the colleges and universities where students who desire to become teachers are educated.

Second, with some projections showing that virtually all schools will fail to meet their NCLB targets in the next 4-5 years, significant efforts already are under way to amend how AYP is calculated. Value-added could play a key part in this process by providing schools with an alternative measure that would be able to identify which schools failing AYP in the current year were on a trajectory of growth that would get their students to proficiency by a later grade.

We call this alternative "Growth to Standards." It does not abandon the federal government's commitment to ensure all students reach proficiency.

Finally, as teachers and administrators come to appreciate its many strengths, value-added assessment also will serve as the foundation for an accountability system at the level of individual educators. Studies of value-added models, such as one recently released by RAND, already are struggling with this question.

The RAND review, "Evaluating Value-Added Models for Teacher Accountability," concluded that the effect of teachers on student learning was real, it could be large and it persists for years beyond the year in which it is first evident. While the authors cautioned against using value-added for high-stakes personnel decisions at this time, they felt value-added models "might actually provide less-biased and more precise assessments of teacher effects." As long as "test-based accountability remains an instrument of education policy," they recommend value-added assessment should be given serious consideration even in light of its limitations (for individual level accountability)."

Excellent Timing

The timing for the introduction of value-added could not be better. Although knowing that our schools are not worse than they were 20 years ago may provide comfort in the face of intense school bashing, it does not follow that our schools are good enough for the 21st Century. An open society facing the twin challenges of terrorism and the fiercely competitive global economy of the Information Age requires citizens and workers who can use technology, think critically and solve problems. If America is to remain a democracy anchored by a strong and stable middle class, then our schools must be able to educate all our children, not merely the top 20 percent, to unprecedentedly high standards.

This will involve difficult change that necessitates overhauling the entire system rather than fixing a few parts. Although critics point to teachers' unions or to poor instructional leadership from administrators or to government regulations, it is critical to understand that no one is to blame. In simplest terms, the nation's economy has changed much faster than our schools. It is now time to move our schools to their next level of excellence. Value-added will play a key role in this effort.

Resources

With so many new resources emerging on value-added assessment, readers might find it helpful to look at the proceedings from the Battelle for Kids' National Value-Added Conference, held in late October.

Battelle's website is www.BattelleforKids.com.

RELATED ARTICLE: Operation Public Education.

Operation Public Education, an education reform initiative based at the University of Pennsylvania, is developing assessment and accountability systems to transform America's schools. Using value-added assessment as the key to deep and substantive K-12 education improvement, OPE has developed a strategic set of reforms to improve the performance of public schools.

The initiative has three goals:

* establishing value-added methodology as the basis for measuring classroom performance;

* implementing new accountability systems based on value-added; and

* designing a high-quality standards-based comprehensive assessment system with explicit links to support high quality instruction.

In addition to value-added, Operation Public Education has developed a new and comprehensive approach to accountability that draws on successful reform models from across the nation. The system is characterized by new forms of evaluation and compensation, peer review, more fluid career opportunities, expanded professional development and capacity building for all educators along with mandatory remediation for ineffective teachers and administrators.

The core of this unique system is that it provides a means to hold teachers and administrators accountable as individuals for student learning results. In return for accepting this measure of accountability, teachers would share in all curricular and professional development decisions affecting classrooms.

The third component of OPE's effort is to develop a new, comprehensive approach to assessment through the creation of a model Request for Proposals that states and school districts may adopt. This model will establish the criteria for both an end-of-the-year test and an integrated testing system for use at four-to-six week intervals throughout the year to help students meet specific standards.

The initial funding to introduce these reforms nationwide is from the Annenberg Foundation and the Carnegie Foundation for the value-added component.

More information is available at www.operationpubliced.org.

Ted Hershberg is executive director of Operation Public Education and a professor of public policy and history at the University of Pennsylvania, 3701 Chestnut St., Suite 6E, Philadelphia, PA 19104. E-mail: [e-mail]. Virginia Adams Simon is the program's associate director and Barbara Lea-Kruger is director of development and communications.