State standards rise in reading, fall in math.
Current conversations about creating a common national standard largely focus on the substantive curriculum to be taught at various grade levels. Even more important, we submit, is each state's expectations for student performance with respect to the curriculum, as expressed through its proficiency standard. Curricula can be perfectly designed, but if the proficiency bar is set very low, little is accomplished by setting the content standards in the first place.
To see whether states are setting proficiency bars in such a way that they are "lowballing expectations" and have "lowered the bar" for students in 4th- and 8th-grade reading and math, Education Next has used information from the recently released 2009 National Assessment of Educational Progress (NAEP) to evaluate empirically the proficiency standards each state has established. This report is the fourth in a series in which we periodically assess the rigor of these standards (see "Johnny Can Read ... in Some States," features, Summer 2005; "Keeping an Eye on State Standards," features, Summer 2006; and "Few Stales Set World-Class Standards," check the facts, Summer 2008).
The 2009 NAEP tests in reading and math were given to a representative sample of students in 4th and 8th grade in each state. NAEP, called "the nation's report card," is managed by the Department of Education's National Center for Education Statistics and is currently the "gold standard" of assessments. Its proficiency standard is roughly equivalent to the international standard established by those industrialized nations that are members of the Organisation for Economic Co-operation and Development (OECD). If a state identifies no higher a percentage of students as being proficient on its own tests than NAEP does, then the state can be said to have set its standards at a world-class level. To ascertain objectively whether state standards are high or low, and whether they are rising or falling, we compare the percentage of students deemed proficient by each state with the percentage proficient as measured by NAEP. The state assessment data used in this report consist of those compiled in 2009 by the 50 states and the District of Columbia.
States have strong incentives not to set world-class standards. If they do, more of their schools will be identified as failing under NCLB rules, and states will then be required to take corrective actions to bring students' performance up to the higher standard. As a result, the temptation for states to "lowball expectations" is substantial. Perhaps for this reason, a sharp disparity between NAEP standards and the standards in most states has been identified in all of our previous reports. In 2009, the situation improved in reading, but deteriorated further in math.
Every state, for both reading and math (with the exception of Massachusetts for math), deems more students "proficient" on its own assessments than NAEP does. The average difference is a startling 37 percentage points. In Figure 1, we provide a uniform ranking of the rigor of state standards using the same A to F scale used to grade students (see sidebar for the specifics on the methodology we used).
Strength of State Proficiency Standards, 2009 (Figure 1) 4th Grade 8th Grade Math Reading Math Reading Massachusetts A A A A Missouri A A A A Washington A A B+ A Hawaii B+ A B A New Mexico B+ A B A New Hampshire B A B- A Vermont B A B- A Minnesota B- A C+ B+ Maine B- A C+ B+ Montana B- A C+ B+ New Jersey B- A C+ B Rhode Island B- A C+ B Colorado C+ B+ C+ B Utah C B- C+ Nevada C B- C C+ Indiana C B- C C+ District of Columbia C B- C C+ West Virginia C B- C C+ Ohio C B- C C+ Oklahoma C C+ C- C Kentucky C C+ C- C Florida C C+ C- C Wyoming C C+ C- C Mississippi C C+ C- C California C C+ C- C Wisconsin C C+ C- C Alaska C C+ C- C South Dakota C C+ C- C Pennsylvania C C+ C- C North Dakota C- C+ C- C North Carolina C- C D+ C Connecticut C- C D+ C Iowa C- C D+ C Oregon C- C D+ C Kansas C- C D+ C Delaware C- C D+ C Georgia D+ C D+ C Arkansas D+ C D C- Louisiana D+ C D C- South Carolina D+ C D C- Virginia D+ C D C- Idaho D C- D C- Maryland D C- D D+ Arizona D C- D D+ New York D C- D- D+ Texas D C- D- D+ Illinois D D+ D- D+ Michigan F D F D Alabama F D F D- Nebraska F D- F F Tennessee F F F F Overall Average Percentage Change in Ranking 2003 2005 2007 2009 2003 to 2009 Massachusetts A A A A +8.2 Missouri A A A A +4.8 Washington C+ C B- A +33.8 Hawaii B B+ B+ A +9.2 New Mexico B- C+ A +19.4 * New Hampshire B- B+ New Vermont B- B B+ +15.9 Minnesota B- B New Maine A A B- B -11.5 Montana C- C+ C B +50.5 New Jersey C C C B +36.1 Rhode Island B- B- C+ B +2.9 Colorado D D B- B- +57.0 Utah D+ D+ C+ +40.3 * Nevada C C C +2.2 * Indiana C- C- C C +22.1 District of Columbia C C +5.8 * West Virginia D- D- C +46.7 * Ohio C+ C C- C -5.3 Oklahoma F D- F C +44.4 Kentucky B- C+ C C -17.3 Florida C C C+ C -5.0 Wyoming A A C C -41.3 Mississippi D- D- D- C +38.5 California B B B C -35.3 Wisconsin D C- C- C +28.9 Alaska D+ D+ D C +22.1 South Dakota C- D+ C- C +18.5 Pennsylvania C C C C -2.3 North Dakota C C C C -8.9 North Carolina D- D- D+ C +32.8 Connecticut C- C C C +7.2 Iowa D+ C- C- +9.0 * Oregon C C- C- -13.7 * Kansas C- C- C- C- -2.5 Delaware C C- C- C- -4.1 Georgia D- D- F C- +24.2 Arkansas C+ B C+ C- -32.2 Louisiana C- C C- C- -2.4 South Carolina A A A C- -65.2 Virginia D+ D+ D+ D+ +4.2 Idaho D+ D D+ D+ -2.1 Maryland C+ C C D+ -39.1 Arizona B- D+ C- D+ -48.5 New York C C C+ D -36.9 Texas F D+ D D +14.5 Illinois C C D D -30.5 Michigan C C- D D- -27.8 Alabama D- D- F -3.0 * Nebraska D D- F -8.2 * Tennessee F F F F -1.0 NOTE: "*" indicates that data for this state are not available for 2003, so the percentile change is from 2005 to 2009. "New" indicates that data for this state are not available for 2003 or 2005, and we cannot calculate the percentile change from either year. No grade means that either state scores or NAEP results were unavailable. Letter grades in colored type indicate improvement by a full letter grade or more from 2007. SOURCE: Author calculations based on stale tests and NAEP
Racing to the Top?
Ironically, Tennessee received an F and had the lowest standards of all states, despite the fact that it is one of the two winners in the first phase of the bitterly contested Race to the Top (RttT) competition sponsored by the Obama administration's Department of Education. Indeed, Tennessee has had the lowest standards of all states since 2003. Based on its own tests and standards, the state claimed in 2009 that over 90 percent of its 4th-grade students were proficient in math, whereas NAEP tests revealed that only 28 percent were performing at a proficient level. Results in 4th-grade reading and at the 8th-grade level are much the same. With such divergence, the concept of "standard" has lost all meaning. It's as if a yardstick can be 36 inches long in most of the world, but 3 inches long in Tennessee.
Delaware, the other RttT First Phase winner, also had below-average standards, for which we awarded a grade of C- and ranked it 36th of the 50 states and the District of Columbia. Delaware claimed that 77 percent of its 4th-grade students were proficient in math, when NAEP shows that only 36 percent were. In 8th-grade reading, Delaware said 81 percent of its students were proficient, but NAEP put the figure at 31 percent.
From these findings one might conclude that the Obama administration is having a huge policy impact by getting states like Tennessee and Delaware to set standards they have been unwilling to establish in the past. But Tennessee earned almost full marks (98 percent) on the section of the competition (weighted a substantial 14 percent of all possible points) devoted to "adopting standards and assessments," even though its standards have remained extremely low ever since the federal accountability law took hold. The proof will be in the pudding. If Tennessee and Delaware and other states now shift their standards dramatically upward, RttT will win over those who think it is performance, rather than promises, that should be rewarded.
Disparities in State Standards
Despite the incentive to lowball expectations, five states--Hawaii, Massachusetts, Missouri, New Mexico, and Washington--have set their standards at or close to the world-class level, earning them an A. Notice that we award grades purely for the expected standard for performance, not actual proficiency. New Mexico earned the same mark as Massachusetts, even though only about one-quarter of its students are proficient, while half of Massachusetts students score at that level. The two deserve equal grades, however, because both are rigorous in their expectations. Another eight states--Colorado, Maine, Minnesota, Montana, New Hampshire, New Jersey, Rhode Island, and Vermont--earned a B for their standards.
President Obama is undoubtedly correct, however, in suggesting that many states are "lowballing expectations." Of the remaining 38 states, 27 earned a C, and 8--Arizona, Idaho, Illinois, Maryland, Michigan, New York, Texas, and Virginia--a D. Three states--Alabama, Nebraska, and Tennessee--had such low standards that we awarded them an F. All of the states that earned grades of F have been ranked D or below in all three of our previous reports. This suggests that once a standard, however low, has been set, it tends to persist--another reason to be concerned about promises from Delaware and Tennessee.
Changes in Standards
Secretary of Education Duncan is not altogether correct in suggesting that educators are lowering the bar, however. Figure 2 shows that in 2009 the differences between state and NAEP standards shrank by 0.08 standard deviations as compared to the average for the three prior surveys. This is a reversal of the trend of declining standards we observed between 2003 and 2007. Eight states improved the overall rigor of their assessments by a full letter grade or more since 2007: Georgia, Mississippi, Montana, New Jersey, New Mexico, Oklahoma, Utah, and West Virginia. By contrast, we gave just four states--Alaska, California, New York, and South Carolina--grades that were at least a full letter grade worse than they received in 2007.
Rising Standards? (Figure 2) Standards are dearly increasing overall in reading but not in math Conversely, they are converging in math but not in reading. Change in Standards since Convergence in Standards 2003 (All States) since 2003 (All States) OVERALL 2003 - 1.00 2005 -0.15 0.97 2007 -0.11 0.91 2009 0.08 0.91 4th Grade Math 2003 - 1.00 2005 -0.11 0.99 2007 -0.06 0.89 2009 -0.12 0.86 Reading 2003 -- 1.00 2005 -0.15 0.97 2007 -0.07 0.91 2009 0.49 1.01 8th Grade Math 2003 - 1.00 2005 -0.16 0.93 2007 -0.12 0.93 2009 -0.31 0.82 Reading 2003 - 1.00 2005 -0.16 0.97 2007 -0.20 0.90 2009 0.26 0.94 NOTE: Change in standards for a given year is measured in standard deviations. It is the average difference between that year and the previous year for all states. Convergence is measured by the size of the standard deviation for each year. The smaller the standard deviation, the greater the convergence. SOURCE: Author calculations based on state tests and NAEP
The reversal in the overall trend is, however, driven wholly by an improvement in the rigor of reading assessments, which set expectations that are higher by 0.49 standard deviations in 4th grade and by 0.26 standard deviations in 8th grade. As a matter of fact, 17 states increased the rigor of their 4th-grade reading assessments by a whole letter grade since 2007, and 17 states did the same for 8th grade. But math standards have slipped by 0.12 standard deviations in 4th grade and by 0.31 in 8th grade. This means that at least some of the state-reported improvements in mathematics proficiency are misleading.
Converging on a De Facto National Standard?
Most changes to standards, as we noted, have been fairly small: only 12 states have made changes to their standards that alter their standing by a whole letter grade. But since our last report two states, Hawaii and South Carolina, have made major alterations to their assessments. The results of these moves have been at odds: while Hawaii's increased alignment with NAEP raised its grade from a B+ in 2007 to an A, South Carolina dropped from an A to a C-.
States nonetheless seem to be continuing their trajectory of convergence toward standards of similar rigor in math (which, given the slipping standards noted above, constitutes a downward convergence), but are more divergent in reading since 2007, particularly in 4th grade. If the convergence of math standards were to continue, we could gradually attain something like a national standard. But it would take a great deal of national patience to achieve a national standard by convergence creep.
In this report, as in previous ones, we assess the rigor of standards that states set. This is an important task, as it reminds states that whether students have or have not learned cannot be a matter of how the test is designed and where the "proficiency line" is drawn. Rather, setting high standards for proficiency is the first step in the journey toward actually improving the learning of a high percentage of students. According to NAEP, less than one-third of students are proficient in reading and a similar proportion in math nationwide. For the sake of the children of this country, we should be doing much better than that.
RELATED ARTICLE: The Grading
In 2003, 2005, 2007, and 2009, 4th- and 8th-grade students took both state and NAEP tests in math and reading. The grades reported here are based on the comparison of state and NAEP proficiency scores in 2009, and changes for each are calculated relative to 2003 (Figure 2). For each available test, we computed the difference between the percentage of students who were proficient on the NAEP and the percentage reported to be proficient on the state's own tests for the same year. We also computed the standard deviation for this difference. We then determined how many standard deviations each state's difference was above or below the average difference of all observations in 2009, 2007, and 2005 on each test. The scale for the grades was set so that if grades had been randomly assigned and so were in a normal distribution, 10 percent of the states would earn As, 20 percent Bs, 40 percent Cs, 20 percent Ds, and 10 percent Fs. The grade given to each state is based on how much easier it was to be labeled proficient on the state assessment than on the NAEP. For example, on the 4th-grade math test in 2009, West Virginia reported that 60.8 percent of its students had achieved proficiency, but 28.1 percent were proficient on the NAEP. The overall grade is the average of ranking for each subject and year. In the case of West Virginia for 4th-grade math, the difference (60.8 percent - 28.1 percent = 32.7 percentage points) is about 0.02 standard deviations worse than the average difference between the state test and the NAEP over the three years, which is 32.4 percent. This earned West Virginia a C for its standards in 4th-grade math. We are therefore generous in that we do not require the meeting of any stipulated cutoff in the differences with NAEP to award a specific grade: no single state would be ranked A, say, if we required for this a difference with NAEP smaller than 5 percentage points. Instead, we rank states against each other in accordance to their current position in the distribution of differences over all the years for which we have observations (2003, 2005, 2007, and 2009).
Tennessee, a Race to the Top winner, is at the very bottom
Paul E. Peterson is professor of government at Harvard University, senior fellow at the Hoover Institution, and editor-in-chief at Education Next. Carlos Xabel Lastra-Anadon is a research fellow at the Program on Education Policy and Governance at Harvard University.
|Printer friendly Cite/link Email Feedback|
|Author:||Peterson, Paul E.; Lastra-Anadon, Carols Xabel|
|Date:||Sep 22, 2010|
|Previous Article:||2+2=litigation: new front opens in the math wars.|
|Next Article:||Invisible ink in teacher contracts: state policy trumps collective bargaining.|