Using data to drive policy.

"Using data to drive instruction" is a phrase that is tossed around a lot today in education. In Washington, we could all use more data to drive policymaking. Just as the presidential election of 2000 put a spotlight on the election process, the higher stakes of accountability under No Child Left Behind has shined a light not only on the achievement of all children and all student groups but also on the quality of the data we now use to make key policy decisions.

How shall schools be labeled in need of improvement? How should we continue to improve high schools? These and other questions are open for debate in Washington, and yet the data we have to analyze these issues is far from complete.

The Dropout Debate

The push for high school reform is still rumbling with strong support from the nation's governors and the Gates Foundation, among others, and even Oprah Winfrey got into the game with a two-part series this spring on the supposed dropout crisis. Calls for reform cite dramatic statistics about dropouts, ranging from half of poor students to a third of all students. In fact, a serious debate continues about the dropout rate because all numbers are estimates.

Jay Greene of the Manhattan Institute concludes the dropout rate is a serious matter, after looking at state-reported diploma data and the national Common Core of Data. The overall graduation rate, he claims, is 70 percent; for black male students it is only 48 percent and for Hispanics it is 53 percent, while white students graduate at a rate of 78 percent.

Larry Mishel and Joydeep Roy of the Economic Policy Institute present an alterative data set based on census data, the Current Population Survey and the National Educational Longitudinal Study. They conclude that about 82 percent of students nationwide graduate on time, and among black students, the rate is 75 percent. For Hispanics, they estimate a 74 percent completion rate.

There are several key sticking points: how 9th-grade students are counted, how to account for population changes between 9th grade and graduation and which data sets are used. Greene claims that using state diploma counts is more accurate; Mishel and Roy point to the long-term benefit of using population surveys to account for students who move or take longer to graduate but who eventually receive a diploma.

The implications of the varying estimates are significant. Dropout prevention programs are facing the same limited resources as anything else in education, and efforts to improve the graduation rate must be targeted to have the greatest effect.

Competitiveness Conundrum 

Along with calls for high school reform there has been much hand-wringing over America's competitiveness. Anyone who has had to call technical support for their home computer can understand the immediacy of outsourcing. But do we really need to be worried that, as stated in a National Academies report, China graduated 600,000 engineers last year, India produced 350,000, and America was far behind with just 70,000?

Putting aside for a moment the huge population differences that should remind us that America also makes a commitment to educating every child and has set a goal for proficiency for every child, what should we make of this? Is America falling behind? Researchers from Duke University went behind the numbers and found many of what China and India called engineers would be considered technicians here, having completed the equivalent of an associate's degree, as Gerald Bracey reported in Kappan magazine.

Further, the public itself is undecided whether students need to be getting more math and science instruction. Only 32 percent of parents think their child needs more math and science, while 57 percent thinks things are fine, according to Public Agenda. Yet 67 percent also agree with increasing the number of math and science courses that students take. Apparently things are fine with their own children, while other children might need more. In reality, the number of students taking advanced courses in math and science has been steadily increasing, as have the number of Advanced Placement tests in math and science.

Is American competitiveness in a crisis? The messages are mixed. One thing we do know is that when talking about high school reform, most parents and the public think that schools need minor changes, not a major overhaul--57 percent compared to just 16 percent, according to data from AASA polling.

Student Progress

Will the new pilot growth model systems being used by North Carolina and Tennessee provide any valuable data on student progress? The jury remains out. The two systems approved by the U.S. Department of Education rely on calculating growth trajectories or forecasts for individual students but may fall short of providing detailed, useable feedback about individual student progress. Still, an important step has been taken in the right direction.

Both states are making the most of their state data systems including individual student identifiers. The Data Quality Campaign is one group advocating for better state data systems. Yet such systems are expensive and can be unwieldy to implement--something that can also be said about the use of progress or growth models for achievement.

Until more states have the data systems that can truly track student growth and all the factors involved, including courses taken and graduation/dropout information, policymakers in Washington will remain hampered by gaps in the data. How do we ever truly know what students know? That's the million-dollar question.

Terri Duggan Schwartzbeck is a senior policy analyst at AASA. E-mail: [e-mail]