Full Disclosure.

Who's Tested and Who Isn't

Editor's Note: The Education Writers Association adopted a resolution last spring calling on school districts and states to release information on which groups of students are tested and which are excluded when they report on the results of standardized tests.

The resolution emanated primarily from the work of a free-lance education writer, Bill Zlatos, who spent six months researching the relationship between school districts' test-taking policies and aggregate test scoring reports. He determined the percentage of students who were tested and whose scores were included in test reports ranged from 66 percent to 93 percent among the 14 large school districts studied.

He found school districts inflated their scores, sometimes intentionally, by excluding students with disabilities and students in English as a second language programs from the testing itself, or from the reports given to the public.

"How can you really compare one district to another or one school to another ... when schools are aware of how to manipulate?" asks Zlatos, whose work appeared in The New York Times, American School Board Journal, Education Digest, and other publications.

The adopted resolution by EWA, which consists of education writers for newspapers and other publications, was shared with the Committee to Develop Standards for Educational and Psychological Testing, which is developing revised test-taking and reporting procedures for educators and other professionals. The group has representatives from the American Psychological Association, the American Educational Research Association, and others.

The School Administrator asked three authorities on student assessment at the district and state levels for their opinions on the test-reporting resolution of the journalists' group. Their views follow.

Karen Banks, assistant superintendent for research and evaluation, Wake County Public Schools, Raleigh, N.C.

(Banks also is president of the National Association of Test Directors, but her opinions here are her own and not those of the association.)

"In order to hold schools, school districts, and states accountable for student achievement, we must ensure that educators do not permit or encourage wholesale exemptions of disabled students from standardized testing programs.

Not only do such wholesale exemptions prevent any accountability for the education of students with disabilities, the exemptions can be used to artificially raise average test scores for the remaining students.

"While the goal of the Education Writers Association resolution is worthwhile, it needs revision. The wording 'who is tested and who is not' should be replaced with words that refer to percentages of groups of students, to avoid implying that individual student information should be made public. More importantly, the resolution reflects an oversimplification and will not necessarily lead to the correction of the problem; additional measures are needed.

"Public or media scrutiny--either because a large number of students aren't tested or because data are not released on what percentage are tested--may be embarrassing but no more embarrassing than the lower average test scores that would result from testing more students with disabilities. When faced with two types of embarrassment, educators will probably choose the one that leads to higher test scores, thus defeating the intent of the EWA resolution.

"A regulatory solution to the problem would be more effective. Exemption decisions should be made on an individual basis, based on the individual education plan for each student. Such decisions will include parent input, through the individual education plan. Therefore, education officials should have no objection to releasing data on the proportion of all students and the proportion of disabled students tested. (Reporting more specific information would not only impose additional work on overloaded testing departments, compiling very specific testing information about individual students could erode the confidentiality of those student records.)

"Additional constraints also will provide useful guidance to schools. The state of North Carolina requires that 95 percent of all students take the state tests. Thus, schools can exempt students with severe disabilities, who comprise a small percentage of the population, but cannot exempt large numbers of learning disabled or mildly disabled students. Recently, the state has implemented an additional requirement that 98 percent of all eligible students (those not exempted based on their IEP's) be tested. Thus, schools will have to try harder to make sure all eligible students take the test, even if it requires more effort to conduct makeup exams than in the past."

Thomas H. Fisher, administrator for student assessment services, Florida Department of Education, Tallahassee, Fla.

"I agree with the Education Writers Association that the results of standardized achievement tests should be reported so readers can determine who was tested and who was not.

"State and district student assessment programs exist for two primary purposes. First, they provide information about the extent to which individual students are progressing in school and are learning what they are supposed to learn. To accomplish this, it is necessary for all students to be given the opportunity to participate in an appropriately structured academic assessment. For certain students, appropriate accommodations should be made available so they can fully participate. For example, a vision-impaired student might be tested with a Braille form of the test.

"However, a few students might not be able to participate in ordinary large-group academic assessments even with accommodations because they have a disability that interferes with the administration of the test. For these students, alternative assessments that make allowances for their disability are needed.

"Second, student assessment programs provide parents, citizens, and leaders with information about the extent to which the total student body is succeeding in school. This objective can be met only if all students are included in the testing program, assuming appropriate test accommodations are available as mentioned. Schools are responsible for educating all students, and the public has the right to understand how well this goal is being achieved.

"Some school and district personnel are concerned that public reports of assessment programs can lead to negative publicity because of low scores. This may give an incentive to some people to influence the test results by not testing certain students such as those who receive exceptional education services or have limited English proficiency. However, these are only the two most obvious candidates for exclusion. Once one goes in this direction, what would prevent the exclusion of students who are at risk of failure, are highly mobile, are from low socioeconomic family status, are not performing at grade level, are in dropout prevention programs, or have not been enrolled in the same district since kindergarten?

"While we must acknowledge some students probably should not be included in large-group testing programs for various reasons, we must not forget our primary objective is to teach all children. Assessment methods exist to measure the progress of all children. Our obligation to the public, and to the students themselves, is to assess fairly and accurately and to report the results to the public. One part of such reports can be an analysis of who was not assessed and why."

Lynn Winters, assistant superintendent for research, planning and evaluation, Long Beach Unified. Schools, Long Beach, Calif.

"The education writers are reasonable to expect public agencies to re lease data on who is tested and who is not. Public schools must be responsible for the learning of all students, and reporting on their progress toward this goal is indisputable.

"However, school leaders know only too well that a total school score may not accurately describe a school. Schools with programs targeted to different populations (gifted, bilingual, etc.) find that an overall score presents a less accurate picture than disaggregated results (scores for subgroups). Ideally, the public should receive both.

"When our district releases test reports, we indicate who is excluded--in most cases, students with disabilities whose IEPs state they cannot take a standardized test under modified conditions. Unfortunately, space limitations in the press often cause such explanatory notes to be deleted, leaving the public to assume that scores include all students in a school.

"When school districts follow a policy of full disclosure, these caveats should be kept in mind:

* "A school score that summarizes the performance of students receiving very different curricula is only a rough index of 'school impact' and often hides rather than highlights significant achievement differences by different subpopulations of students.

* "The single score summary needs to be understood in light of the information it summarizes. Therefore, with each 'one score index, we should provide the scores of subgroups contributing to the one-score summary. For instance, School A, which has a mediocre average score, may have a large number of students with limited English proficiency who are performing relatively well when compared with other LEP groups. Yet these students lower the school average when factored into the mix.

"School B, on the other hand, may have a very high score but serves a small number of special education and LEP students who are NOT performing well but whose low performance is masked by the preponderance of high-scoring students.

"Without this backup information about important subgroups of interest, it's really impossible to know much about the school. I would argue that an even better approach would be to report information about the entire distribution of performance for important subpopulations and compare it to the whole-population distribution (or other targets of interest)."