My Stakes Well Done.

The issue isn't academic benchmarks, it's the misguided use of a single test

A colleague on the National Assessment Governing Board refers to testing as an illusion. By that he means that we ascribe much more power and exactness to tests than they actually possess. In part, that illusion of testing has created many of the problems currently encountered with the assessment portion of the high standards movement.

My home state of Virginia was recognized early on in the standards-setting process as having developed an excellent product in the Standards of Learning, known widely as SOL. Indeed, educators in Virginia embraced the SOLs as quality benchmarks for learning in the 21st century.

Today, the state's SOLs face serious challenges from educators and parents concerned with the eventual impact the tests might have on children and education. The standards themselves remain quality expectations for our children. The problem is with the high-stakes assessment program that is supposed to measure whether or not the standards are being met.

It was not enough to simply raise the bar and expect our children to perform at higher levels. Along with the higher standards came accountability. Our policymakers were determined to hold students and educators accountable for their performance or lack thereof. Enter high-stakes testing.

Most states contracted with reputable, experienced test-developing firms to create the tests the state would use to assess the extent to which their standards were being met. In other instances, existing standardized tests were selected. In most cases, the tests were traditional paper-and-pencil, multiplechoice exams--reliable to the extent that the same results would be obtained with repeated administration of the tests, valid to the extent that the tests measure what they are purported to measure.

Improper Use

It is the question of validity, or how these high-stakes tests are being used and interpreted, that threatens to undermine the whole standards movement. The President's Advisory Commission on Educational Excellence for Hispanic Americans last summer issued a report that accused state leaders of compromising the educational future of Hispanic students by making highstakes decisions based on inaccurate and inadequate testing information. The committee does not place the blame on the tests themselves but rather on the educational context in which they are created and used.

In Michigan, the American Civil Liberties Union challenged that state's scholarship program asserting that using the state's test as the sole basis for awarding the scholarships is discriminatory. The U.S. Education Department's Office for Civil Rights recently published "The Use of Tests When Making HighStakes Decisions for Students: A Resource Guide for Educators and Policymakers." The report is meant to assist policymakers in ensuring that the highstakes tests they implement are educationally sound and comply with nondiscrimination laws.

The federal report relies heavily on the "Standards for Educational Psychological Testing" developed jointly by the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education. The joint standards specifically assert that decisions affecting individual students' life chances or educational opportunities should not be made on the basis of test scores alone.

All of this reinforces the illusory nature of tests: They are extremely useful, but hardly infallible measures of student performance. Even the leading test publishers agree that test scores are not perfect measures and should not be the sole basis for making high-stakes decisions. The teachers' associations, while remaining supportive of the standards movement, also are wading in on the high-stakes testing controversy. Bob Chase, the National Education Association president, said at his annual conference in Chicago this summer, "Testing mania is quite literally devouring whole school systems like some education-eating bacteria."

Sandra Feldman, the president of the American Federation of Teachers, is concerned with the demands that high-stakes testing is placing on teachers. Everyone in the schoolhouse is feeling the stress. A principal of a well-to-do elementary school in Montgomery County, Md., resigned after allegations that students in her school were encouraged to change their wrong responses on the state test. In my own school system in Fairfax County, Va., two teachers resigned after testing irregularities were uncovered. In both cases, the teachers had prepared their students prior to the state tests with questions that closely paralleled those on the test itself.

Harmful Effects

The issue of teaching to these tests has become a major concern to parents and educators. A real danger exists in that the test will become the curriculum and that instruction will be narrow and focused on facts.

Much of the criticism against Virginia's Standards of Learning tests has to do with their emphasis on facts and recall. Teachers believe they spend an inordinate amount of time on drills leading to the memorization of facts rather than spending time on problem solving and the development of critical and analytical thinking skills. Teachers at the grade levels at which the test is given are particularly vulnerable to the pressure of teaching to the test.

Students, too, are affected in harmful ways. I have heard from many parents who are upset with the stress the tests place on their children, and they lament the apparent renewal of school phobia in the younger students.

Rather than a push for higher standards, the tests may be driving the system toward mediocrity. The classroom adaptations of "Trivial Pursuit" and "Do You Want to Be a Millionaire?" may well result in higher scores on these standardized tests, but will students have acquired the breadth and knowledge to do well on other quality benchmarks, such as the SAT and Advanced Placement exams?

A poll sponsored by the American Association of School Administrators last summer found that 63 percent of respondents disagreed with the idea that one test can accurately measure students' progress. The 32nd annual Phi Delta Kappa/Gallup poll on public schools, released this fall, indicated the public's growing unease with standardized testing and test-driven reform. Thirty-four percent of public school parents said standardized testing is emphasized too much in their community, an increase from 19 percent just three years ago. Concern is growing that this emerging testing backlash could seriously hamper the standards movement and other efforts at accountability and education reform.

Subjective Decisions

Because the National Rifle Association has its national headquarters here in Fairfax, Va., I am mindful of the defensive statement they frequently issue: "Guns don't kill people, people kill people." Perhaps we could modify this to say: "Tests don't hurt kids, it's how we use tests that hurts kids."

The truth is that testing always has been a part of teaching. How can we know that children have learned unless we assess their progress? In reality, teacher-developed tests are responsible for most of the decisions that take place in the classroom on a day-to-day basis and those tests hardly stand up to the rigor of reliability and validity that standardized tests meet. The problem is clearly one of test interpretation and usage.

Tests create a sense of objectivity when, in reality, a great deal of subjectivity is involved. That is specifically the case in the determination of the cutoff points that define the achievement levels. The National Assessment of Educational Progress tests are among the best-developed and most-respected assessment instruments available. A highly rigorous and systematic process is followed by the National Assessment Governing Board in establishing the standards of performance applicable to the NAEP tests. The standards are thoroughly reviewed by focus groups and content experts and then discussed at public forums before they are brought to NAGB for adoption.

Even so, standard setting for NAEP has been controversial. In lay terms, the question is, what constitutes passing? What constitutes whether a particular score on a NAEP test falls within one of the three achievement levels: basic, proficient or advanced?

Ignoring Counsel

The percentage of students either passing or failing a high-stakes state test will vary according to the difficulty of the tests and where the bar is set. Few states have followed the rigorous approach used in determining the NAEP performance standards. Unlike NAEP, the state tests are used to make decisions that affect high school graduation, promotion or school accreditation.

In Virginia, panels of teachers, administrators and university professors were convened to, as the standard-setting instructions said, "make a professional judgment on the number of questions on a test that must be answered correctly to signify that a student has reached a specific criterion (i.e., passed the test)." The recommendations of the standard-setting committees were delivered to the state board of education, which then quickly ignored them and proceeded to set its own standards. So much for the scientific approach.

In the first year of the SOL administration, only 3 percent of Virginia schools met the accreditation standards. Seven percent met the standards in year two. Significant improvement is expected in year three, aided in part by not tabulating the scores for limited English proficient students and transient pupils (those entering school after the first 30 days of instruction).

The setting of standards, cutoff points and passing grades is highly judgmental and subject to political manipulation. If we want to look good, we set the bar low. If we want to make the system look bad, we set the bar unreasonably high.

Both approaches ignore that the best use of the test should be to provide feedback that allows us to build on the strengths of our students and to correct weaknesses. It should allow us to praise good performance and identify where improvement is necessary. We should use multiple valid benchmarks, not a single one.

Our public schools are here to teach, support and encourage our students to maximize their potential, not to penalize and destroy their future at the age of 17. Let's have our stakes well done.

Daniel Domenech, a past president of AASA, is superintendent of the Fairfax County Public Schools, 10700 Page Ave., Fairfax, Va. 22030. E-mail: [e-mail]