Work in progress: ACSA's advocacy role in the development of California's standards-based assessment and accountability system continues as the system evolves. Here are ACSA's latest recommendations.

Developing a comprehensive assessment system for a state with 1,048 districts that have 8,761 public schools serving 6,050,895 students is no easy task. Prior to the passage of SB 376, the assessment program in each district was unique. Other than efforts like the CAP and CLAS testing programs, statewide assessments had been minimal and voluntary. There was little continuity from district to district across the state.

This article describes ACSA's work to influence the ongoing development of a comprehensive, standards-based assessment and accountability system for California.

Rollout of the assessment program

In 1997, SB 376 authorized the Standardized Testing and Reporting (STAR) program. It mandated that all districts administer a norm-referenced test to students--enter SAT-9. The state assessment program was initiated so data would be available to judge an individual school's success in student achievement, and to compare performance across schools.

The initiative got off to a rocky beginning in part because it was brought online prior to the adoption of statewide content standards. The alignment of the SAT-9 to the standards was reported to be minimal. Educators were confused--do I teach the standards, do I teach what is assessed on the SAT-9, or do I try to do both?

The next step in the development of California's assessment system was integrating items to assess California standards--enter "augmented" tests. In 1999, the California Standards Tests were added for English language arts and mathematics for grades two to 11. History/social science and science assessments for grades nine to 11 as well as a writing assessment for grades four and seven were added in 2001.

One main problem with the rollout of the state assessment program has been the fact it has been layered on top of what already existed. This has been especially problematic at the high school level, where AP, SAT, Golden State and ACT were and still are all in existence.

ACSA responds with a plan

As the state assessment program continues to evolve, ACSA remains involved. In fall 2000, the ACSA Curriculum, Instruction and Evaluation Committee--with its four new assessment representatives--was asked to propose guiding principles for an ACSA policy statement on assessment.

Following a presentation to the ACSA Board, Executive Director Bob Wells lay down the challenge to translate the principles into a template for the ideal California assessment program, with the admonition to "be sure it fits on one piece of paper."

The challenge was met by Linda Kaminski, chief academic officer, Santa Ana USD; Sally Bennett, assistant director, standards, assessment and accountability, San Diego USD; Joann Merrick, deputy superintendent, educational services, Glendale USD; Kathy McCreery, ACSA assistant executive director; and Susan Liberati, director, curriculum, Torrance USD.

A template was indeed designed and approved by the Board in spring 2001. The plan became a valuable tool as ACSA members and staff worked diligently with our partners in the Education Coalition to influence the development of future generations of California's assessment program. In spring 2002, the CIE Committee was asked to review and update the plan as a result of changes to the state assessment and accountability systems.

Themes of ACSA's plan

Several themes formed the basis of the first plan and continue in the latest revision. They include:

* Emphasis on multiple measures. ACSA believes no single assessment should be used for high stakes decisions about students, schools and/or districts.

* Emphasis on having a match between the purpose of the assessment and the assessment itself. ACSA believes it is essential the purposes of the state assessment program are clearly identified and the assessments necessary to achieve those purposes are selected.

* Emphasis on the assessment of California's standards and de-emphasis on the norm-referenced tests. From the beginning, ACSA has recommended the use of the survey form of the norm-referenced test and only at selected grade levels. ACSA believes the purpose of using a norm-referenced test should be for national comparisons and not for any diagnostic purposes, because the tests are not aligned with California standards.

* Delay assessment of ELL students until they have developed sufficient English skills to have valid and reliable results. ACSA believes it is ineffective to have students with a very limited knowledge of English attempt assessments that are incomprehensible.

* Reduce redundancies in the assessment program and minimize the impact on instructional minutes. ACSA believes California needs a streamlined assessment program that is free from redundancies and has minimal encroachment on instructional minutes.

What's new in version two of the plan

Implementation of the High School Exit Exam in spring 2001 and the California English Language Development Test in fall 2001, coupled with the reauthorization of the STAR program for 2003-05, definitely tipped the state scales toward standards-based tests.

ACSA strongly supports this major shift in attention and focus, but also recognizes that continued work is needed. ACSA's revised long-term plan presents several specific recommendations to further improve and enhance the state assessment system.

California Standards Tests

* Fast-track the development/revision of the integrated science CSTs. Discontinue administration of the norm-referenced science test. Consider development of a general science CST if needed to assess students enrolled in courses for which there are no content-specific tests.

* Do not add a writing test to the 11th grade English language arts CST. The CAHSEE provides a statewide high school writing test through the grade 10 census administration. The linked CST/Golden State Exam and the revised SAT I can provide the higher-level writing assessments needed for college placement and/or admission.

California High School Exit Exam 

* Investigate alternatives to the CAH-SEE for students who demonstrate equivalent proficiency via other assessments (the GSE, selected CSTs), and for special needs students who must demonstrate their knowledge and skills via other modalities.

Assessment of English learners

* Continue improving the CELDT content and administration to ensure that English language proficiency is assessed appropriately and consistently across the state.

* Delay administration of the CST and NRT to English learners until they reach the intermediate level of fluency or after completion of two years of instruction in a U.S. school, whichever comes first.

Norm-referenced test

* Administer the NRT at selected grade levels only to further reduce the time and emphasis given to this test. Consider embedding the norm referenced items in the California Standards Tests. This would provide national benchmarking while focusing the state assessment system on standards-based tests.

Multiple measures

* Use multiple measures in evaluating student performance, especially for high-stakes decisions such as promotion/retention of graduation. These measures should represent different assessment formats to provide students with various ways to demonstrate their knowledge and skill.

Accountability recommendations

As part of the revisions to the long-term plan, a whole new section on accountability has been added since assessment and accountability are now intricately intertwined within the educational arena. Locally, teachers and schools use various assessment mechanisms to hold students accountable, and districts use test results to hold schools accountable.

The state Academic Performance Index, with its associated rewards and interventions, is calculated from the state test results. Nationally, the No Child Left Behind Act holds states accountable for the annual progress of each school as measured by state tests.

It became obvious that ACSA could not talk about assessment without talking about accountability, and vice versa. Therefore, the revised assessment plan brings forward the following recommendations to improve the state accountability system.

API as an academic measure

* Maintain the API as an academic performance index. While non-academic indicators such as attendance can provide valuable information related to students' opportunity to learn, they should not be part of an index established to report on student achievement outcomes. This information can be reported through other documents, such as the School Accountability Report Card.

Measures included in the API

* Fast-track the process to include CELDT results in the API. This test provides important information about the performance of English learners, a substantial portion of the state's student population. It is aligned with the English Language Development Standards, which describe what students need to know and be able to do as they acquire English and work toward the grade-level English language arts standards.

* Accelerate the process to include the CST science in the API, and discontinue the administration and use of the NRT science test. Since the CSTs are aligned with the state content standards, their inclusion may encourage high schools to improve science instruction, a critical need given recent NAEP results and the TIMSS report.

Value-added data

* Use both longitudinal (matched) and cross-sectional (unmatched) data in the API. This combination would hold schools accountable for not only the current status of student performance, but also the value added through their instructional programs.

* Move quickly to implement a statewide student information system so the state can establish and maintain longitudinal databases. This would provide the capability to match student academic data from year to year, as well as make it possible to track students across schools and districts.

Student subgroups

* Calculate individual growth targets for subgroups based on the difference between each group's base API and the statewide goal of 800.

* Redefine a numerically significant subgroup as one that represents at least 10 percent of the population tested, with a minimum of 30 students. Currently, a subgroup could represent only two to three percent of enrollment at a large, diverse school.

* Consider carefully how English learners and special education students will be included in the accountability process. English learners are a self-renewing group; as their performance approaches that of fluent English students, they are reclassified and no longer included in the English learner subgroup. "Students receiving special education services" is a widely diverse group whose members range from students instructed primarily in general education settings to students with multiple handicaps taught in specialized classrooms.

Low-performing schools

* Use a specified API score to define a low-performing school, just as 800 is used to define a high-performing school. This is consistent with a standards-based approach in which performance is evaluated in relation to set criteria rather than in relation to the performance of others. It would eliminate the need to use decile rankings for this purpose.

What's next?

Education systems are continually evolving--in response to student achievement outcomes; in response to research and professional study; in response to local, state and national policies and beliefs. The ACSA long-term assessment and accountability plan must also evolve if it is to be useful to the educational community. ASCA has already identified three areas that need to be promptly studied and included in the plan.

* No Child Left Behind. ACSA must carefully study the assessment and accountability requirements of NCLB, paying particular attention to the schoolwide and subgroup criteria used to determine Adequate Yearly Progress. The long-term plan can be used as a resource to help ACSA respond to NCLB proposals emerging from the California Department of Education.

* High school testing. As any high school person can tell you, testing controls the schedule during the spring semester. Add together state-mandated tests, "voluntary" tests (SAT, AP, GSE) and local assessments, and it is no wonder high schools are waving the white flag. ACSA must look carefully at the scope of assessment at the high school level, its impact on instructional time, and its effect on student and staff motivation and morale. The long-term plan can serve as a vehicle to illuminate the issue and bring forward suggestions for improvement.

* Data reporting. Assessment data is valuable to practitioners only to the degree that it can be used to inform instruction and school practices. Currently, the state and publisher reports are useful mostly for general reporting. As a result, teachers, schools and districts individually slice and dice the numbers, and seek out software that will put meaningful data at their fingertips. ACSA must consider the area of data reporting as part of its overall effort to improve assessment and accountability processes in the state.

ACSA's advocacy role

ACSA will continue to use the long-term assessment (and accountability) plan in its "inform and influence" advocacy role across the state. Through member representatives on state-level committees, through the untiring and often unrecognized work of staff and through the Education Coalition, ACSA will continue its efforts to inform legislators and influence legislation; inform the State Board of Education and influence policy decisions; and inform the governor and secretary of education and influence the direction of education in the state of California.

Interested readers can find the entire Long-Term Assessment Plan on ACSA's Web site: www.acsa.org.

Joann Merrick is assistant superintendent in the Glendale Unified School District. Sally Bennett is assistant director in the San Diego City Unified School District. Both authors are members of ACSA's Curriculum, Instruction & Evaluation Committee.