A most uneven playing field: big differences between state standards threaten the accountability that's at the heart of No Child Left Behind.

Whether you agree or disagree with the accountability called for in No Child Left Behind, one thing is starting to become clear--the standards and the Title I money attached to meeting those standards may depend a lot more on where you live than on how well you teach your students.

Everyone knows the federal law is based on accountability and educating each child. Everyone also knows states are given the latitude to create their own set of standards (with federal government approval) and decide at what level their students will be deemed proficient What everyone doesn't know, but thanks to two studies are beginning to find out, is how wide the variations in standards are from state to state.

What this means, of come, is that a proficient child in Indiana may not be proficient if he or she moved to South Carolina. Going further, that means schools or districts deemed "in need of improvement" in South Carolina, might be fine in Indiana; these schools could face governance changes and the loss of Title I funds down the line.

Defining Proficient

"We didn't expect to find quite as much variation from state to state," says Gage Kingsbury, director of research for the Northwest Evaluation Association.

NWEA, and separately the American Institutes for Research, have compared students across state lines by looking at which students passed a certain state test and how those students fared on the national NAEP test Both groups then plotted those students' NAEP scores against the NAEP test results of other states' proficient students.

To make comparisons easier, NWEA then put each state's proficient score on what it calls a RIT scale. (RIT, or Rasch Unit, is an equal-interval scale that measures growth in learning like a ruler measures height. The RIT scale makes it possible to create tests with different items that measure the same level of achievement) This shows that in the third grade, a proficient student in math in Indiana scores a 194 on RIT, but a proficient math student in Minnesota scores 209, significantly higher. In some cases, the gap widens as the students get older. In the eighth grade, proficient students in Indiana, Oregon and Idaho, scored 230, 231 and 233, respectively. But those in South Carolina, Arizona and Wyoming scored 251, 254 and 257, respectively.

Comparing reading scores, a student deemed proficient for 10th grade in Colorado scores a 223 on the RIT scale, while a proficient students in Washington and South Carolina's seventh grade are required to score 226--three percentage points higher.

On some level, everyone involved in education--and especially state education officials--knew that gaps existed between states. A 1996 report by the Southern Regional Education Board was the first to publicize such differences. But the range between what one state and another consider proficient are suprising even experts.

"In every grade for which we had information on more than two states, the states differed in their proficiency levels by at least 33 percentile points," writes Allan Olson, the executive director of NWEA. "The standards differ in a manner that is significant and persuasive."

The NWEA conducted research for members in six different states, Colorado, Indiana, Minnesota, Oregon, Washington and Wyoming. "One of the worries of No Child Left Behind is that lots of states are managing their school accountability system their own way," says Steve Ferrara, a senior researcher at the American Institutes for Assessment. Before the national law, it wasn't a problem became states used their own assessment and their own decision criteria. "Now, NCLB puts a new set of requirements" on top of each state's idiosyncratic system, he says. Still, he cautions that "it's way too early to decide whether [the differences between states are] good or bad."

"We've kind of painted ourselves into a corner. I anticipate states that set their performance standards a fairly long time ago, might petition to reset their standards," says Kingsbury.

And they are. In three examples, proficient students in Louisiana only have to reach the state's "basic" level; Colorado will count students that are partially proficient on state standards to be proficient for NCLB; and while Connecticut hasn't lowered its standards, it did create a new, lower, category that will allow students to be NCLB proficient even if they don't meet all the state's standards.

In-state problems

What are even more alarming than the differences between states are the differences that occur between grades, or subjects, in the same state. In some cases uncovered by NWEA's research, a student who is proficient at one grade level has low odds of achieving the same distinction just two years later because the bar for proficiency jumps so significantly. The reverse is also true. In Colorado's reading test, a proficient student in seventh grade has to score 218 on the RIT scale. One year later, the bar moves up to 220; but in the ninth grade the proficient level inexplicably dips back to the 218 score.

"One thing you can't tell is which is right. But what you can tell is that they are very, very different," Kingsbury says.

"What that means for a classroom teacher, they're getting inappropriate info about how a student is doing. Pressure is being put on teachers depending on the difficulty of the standards based on the grade they are in," he adds. This could lead to a misallocation of money if a school official simply looks at the percentage of students failing in each grade.

But is all this disparity bad? As eager as President George W. Bush was to implement NCLB, he and the Department of Education have been that unwilling to try to push a national level of standards.

"No Child Left Behind respects local control," says Dan Langan, a DOE spokesman. Every state had standards before the law was passed and "this builds upon what they've been doing for years," Langan adds.

Part of this freedom means that states don't have to change their standards to homogenize them with other states. But Kingsbury and Ferrara expect the new flow of information comparing states to lead to more uniformity, eventually.

The comparisons "in the long run will result in more consistent standards and useful standards, Kingsbury says. "But I don't anticipate quick fixes."

"This could lead to pressure states to bring their states more in line" with other states, says Ferrara. "That's not a simple process."

Not Standard

When state standards are compared, using
NAEP test results as a base, wide variations
between states appear. NWEA compared the
standards of six states, and graded the score
needed to be proficient in each state using a
RIT scale.

Math Scores

Grade 3:
Indiana                       194
Oregon                        199
South Carolina                208
Minnesota                     209

Grade 6
Indiana                       216
Colorado                      223
South Carolina                234

Grade 8
Indiana                       230
Oregon                        231
Colorado                      241
South Carolina                251
Wyoming                       257

English Scores

Grade 3
Arizona                       190
Indiana                       195
South Carolina                205
Minnesota                     206

Grade 6
Idaho                         211
Colorado                      212
Indiana                       219
South Carolina                221

Grade 8
Idaho                         218
Colorado                      220
South Carolina                230
Wyoming                       232

Source: Northwest Evaluation Association

Judging NAEP

Is the National Assessment of Educational Progress an accurate way to judge the country's students? For researchers trying to tell how one state's standards fare against other states, the NAEP test is invaluable. NAEP is the only national test given to students in grades four and eight. It is given to students in every state and has existed for 34 years. So why might it not be a good test to use as a national yardstick? There are three reasons, according to Gage Kingsbury, a senior researcher at NWEA:

* NAEP is taken by a small percentage of students in each state, while state standard tests are taken by nearly every student in every state.

* NAEP is a high-stakes test with no consequence for students. Students never get to see their scores and it doesn't count toward their overall grade. There's no motivation for students to do their best.

* It's designed around a generic curriculum, while each state test is designed around that state's set of standards.

Wayne D'Orio, wdorio@edmediagroup. com, is editorial director.