Building state assessment from the classroom up: Why nebraska has forsworn high-stakes testing in favor of district-tailored measures.

Our nation has taken up the call to standards and the results have changed everything about the way our schools and state education agencies do business. These changes have extended from the classroom to the boardroom and from the schoolhouse to the statehouse.

For most states, the biggest change has been a reliance on single, mandatory and often standardized tests to assess whether students meet the standards set for them. It is unfortunate we hold such high regard for these summative measures, which have little power to change what actually happens in the classrooms.

The goal of assessment is to improve teaching and learning and thereby improve student achievement. Therefore, it makes sense to build assessments from classrooms up and to build assessments within and upon the program of curriculum and instruction.

Much of the power of assessment is lost when it is not integrated into classroom activities. Assessment built at the classroom level makes it easier for teachers to identify those who are (or are not) learning and what they are (or are not) learning. With this information at their fingertips, teachers can make changes immediately rather than waiting until the end-of-year assessments are administered and scored--when it is too late to help struggling students.

Alignment Work

Nebraska was one of two states that did not have a comprehensive, state-mandated, single-test system. (Iowa is the other one.) But that has changed. Now Nebraska's plan for assessment and accountability is state-mandated, but it is based on a foundation of formative classroom or school-based assessments. It also calls for districts to create their local assessment systems with guidance from the state department of education.

Nebraska's School-based, Teacher-led Assessment and Reporting System, known as STARS, provides accountability for reporting how well students are doing against the standards while protecting the local curriculum and ensuring that the teachers have the power to decide how they will assess their students' learning. Teachers are encouraged to integrate teaching and assessment rather than regard teaching and assessment as distinct and separate from each other.

Under the STARS plan, local districts create an assessment plan that outlines how they will assess student learning against the standards. The plan has two parts.

First, districts must select a norm-referenced test from the five approved by the department of education. Roughly 30-35 percent of Nebraska's standards at grades 4, 8 and 11 are addressed by the five state-approved norm-referenced tests. The department used independent assessment experts from the Buros Institute of Mental Measurements to determine which standards at which grade levels are met by the NRTs. (School districts may access this information on the Nebraska Department of Education s Web site.)

Second, districts must plan for how they fill in the blanks for the other 65-70 percent of the standards not addressed by the norm-referenced tests. Most districts created "wrap-around" assessments to fill in the gaps where the standards were not met by the approved NRT. Some districts administered one of the approved NRTs as well as a comprehensive curriculum-based assessment created or purchased by the district.

For the STARS plan to succeed, the leadership and effort must come at the local level with guidance and support from the state. The state has provided the framework of state model standards that local districts may use to create their own. Or districts may adopt the model standards. Either action requires a local conversation about what it is that students should know and be able to do.

The state ensures that each intermediate service region in the state has at least one person trained in the assessment development process who is ready to provide training at the local level.

The state also has used its discretionary federal monies to provide grants to schools so that teams of teachers can be paid stipends for developing local assessments. In most districts, teams of teachers work during the summer to develop their standards, align their curriculum and instructional activities and create their local assessments.

In addition, the state has conducted leadership workshops for principals and superintendents on how to lead the process in their schools. Most school districts have either designated a principal or selected a lead teacher to shepherd their district through the process.

Through the University of Nebraska-Lincoln, the first cohort of educators is enrolled in a program of study and mentoring. They will be the first educators to receive certification as assessment leaders. We also worked with all the teacher training and administrator training institutions to ensure future teachers and administrators graduate with the necessary knowledge and skills to lead and implement standards and assessments in their schools and districts.

Testing the Test

Districts are held accountable for the quality of the assessments they develop and are provided guidelines for what good assessment looks like so they know what they are being held accountable for and build their assessments accordingly.

Each district submits an assessment plan created to measure student achievement against the standards at grades 4, 8 and 11. The portfolio is judged by an independent jury representing research, higher education and the state department of education based on how the portfolio meets six criteria developed by a panel of state and national experts. The criteria are:

* Alignment to Standards: Are assessments aligned to either the model standards adopted by the state or the local district's standards (which must be equal to or more rigorous than the model standards)?

* Opportunity to Learn: Is evidence apparent that curriculum and instructional activities are aligned to standards and are provided to the students in a timely manner so students have the opportunity to learn those things that are in the standards?

* Bias Review: Are measures free of bias based on gender, socioeconomic status, racial or ethnic background or handicapping conditions?

* Developmental Level: Are measures appropriate to the age group of students?

* Score Consistency (reliability): Does evidence exist that administration and scoring of this assessment over and over would have the same results?

* Mastery Levels: Do mastery levels determine whether the student's performance can be judged to be proficient on each standard appropriate for the subject and the grade level?

The jury rates the quality of each district's assessment on each of the six criteria. All criteria listed here are critical to good assessment practices. However, as the intent is to create assessments that can be used for school and classroom improvement, the two most important criteria are the alignment to standards and curriculum match.

Formative measures, such as classroom and school-based assessments that are locally developed, place highest priority on these two factors. The other criteria are of little importance until alignment to standards and curriculum match are met.

Trusting the Test

Based on their locally created assessment measures and practices, districts report the percentages of students who are proficient on each standard for grades 4, 8 and 11 and receive a rating based on the percentages of their students who meet or exceed proficiency standards.

Can we trust the results of classroom and school-based measures? Because STARS requires the inclusion of a state-approved standardized, norm-referenced test in the local assessment plan, any inconsistency between the results of the local curriculum-based assessments and the standardized tests raises a red flag. When the results of the two measures are markedly different, districts need to look at the two sets of results and make adjustments.

The STARS plan also includes using external benchmarks of statewide results in ACT scores and National Assessment of Educational Progress to verify the aggregate results of the statewide reporting of local assessment results. Consistency across the results of multiple statewide measures provides a means of judging whether the assessments are appropriate and whether the results are consistent across multiple external measures.

Each November, the Nebraska Department of Education publishes a profile of each school district that includes: (1) percentages of students who meet each standard; (2) the jury's rating of the quality of the district's assessment on each of the six criteria; and (3) other school performance and demographic information.

The profile of each district may be reviewed by school personnel, community members and policymakers to see how well students are doing and how well districts are doing in educating students. The profile shows not only how well students are learning but also how high the district held the bar.

Informing Reforms

We decided early on that we would not design the STARS model with preconceived notions about what actions the state would take if schools either refused to participate or performed poorly.

Schools are required to have standards and to develop assessments and report the results of those assessments for grades 4, 8 and 11. Our accreditation rule is being revised to include these requirements so that refusing districts will be jeopardizing their accreditation status.

With regard to student performance and how schools are judged according to that performance, we assumed that local people armed with information about how their schools and the students in them are doing will spur local efforts to remedy any problems. In Nebraska, schools and communities are tightly connected. Citizens will put pressure on the local leadership and policymakers to fix areas of low performance.

Therefore, we are using this first year of reporting to monitor how communities respond to the data that will be disseminated statewide. If we find that districts do respond and schools do improve, then the state likely will not intervene.

Where school improvement is needed we will converse throughout the year with all stakeholders statewide to discuss what the state should do. We hope the locus of leadership for school improvement will remain at the local level.

Above all, we intend to honor and highlight districts with exemplary performance. We will have an honor roll of schools that achieve high levels of performance on their assessments and/or report high levels of achievement of students, We will also find ways to recognize and disseminate best and promising practices of schools and districts in standards, assessment, reporting and school improvement.

The STARS program provides formative data from the local assessments to ensure that the school improvement process will focus first on curriculum, instruction and teaching rather than on the performance of the student as the locus of accountability. That data, when aggregated to show a picture of group or school performance, will ensure that the entire schooling process will be part of the discussion of school improvement.

Facing Detractors

The STARS program is not without its critics. Some believe that because Nebraska ranks in the top 10 among the states in just about every performance category we don't need to add more testing. Some believe the STARS process is too labor-intensive and time-consuming and that it would be cheaper and easier to have a state-created test. Still others contend that the only true accountability is that which results from ranking school districts against each other based on all students taking the same test at the same time.

There is no question that the STARS process is labor-intensive and time-consuming for local educators who already work very hard. A state-created and administered test would be more efficient for the local leaders. However, it would likely be less effective.

Administering a state test will not provide the critical professional development that the teachers of Nebraska are experiencing. Our local educators are becoming assessment literate. They are engaged in vital professional development as they determine how they know students are learning, what they are supposed to learn and how to change curriculum and instructional activities so students can learn.

Nothing in a state test will create teacher commitment to standards and to student learning without the use of force by sanctions or reward. Nothing in a state test will energize professional commitment to doing whatever is necessary to help students learn.

An Imperfect System

If standards are to spur the changes that most everyone agrees they can and should, then they must be linked to curriculum and instructional programs from which assessments systems are designed. And these assessments must have their base in the classroom.

The STARS program focuses assessment and school improvement efforts on what is best for students and what they need to learn. The local assessment plans requires that student learning be compared to standards, that districts compare their performance to standards and to the criteria for good assessment and that districts compare their performance to external benchmarks and show they are improving over time. In the STARS plan, districts are compared, but they are compared to themselves, to the criteria of standards and to the criteria of good assessment practices.

The Nebraska system is not perfect. However, the preliminary results from our first assessments are encouraging. There is every reason to believe that we will continue to improve on the external benchmarks as evidence that STARS works and that formative measures are the key to the promise that standards will, in fact, change everything for the better.

Doug Christensen is the commissioner of the Nebraska Deportment of Education, P.O. Box 94987, Lincoln, Neb. 68509.

RELATED ARTICLE: Off the Bandwagon in Nebraska: A Local View

STEPHEN C. JOEL

The landscape of school accountability across America is not a pretty picture. It is littered with threats of punitive measures against schools and districts by state education agencies, widespread use of teaching to the test, hostile state takeovers and mounting distrust between local superintendents and state authorities.

That's the portrayal of accountability painted for me by state commissioners and superintendents of public instruction nationwide. Last summer, I had the good fortune to attend their professional organization's three-day institute in Wilmington, N.C., where I heard abundant stories about the negative impact of high-stakes testing on certain student populations and the exodus of local school leaders and staff who no longer felt they had control over student learning.

At the same time, I never felt prouder to be an educator from Nebraska, where we have largely resisted jumping on the national bandwagon of using a single state test with punitive implications to measure accountability. In front of "the chiefs" attending the summer institute of the Council of Chief State School Officers, I joined Doug Christensen, Nebraska's education commissioner, and other invitees in describing our state's unique approach in which all students are pushed to meet high standards, school leaders are trusted to do what they've been trained to do and school districts aren't held accountable through the threat of punishment.

Realistic Expectations

I have been a superintendent for 18 years, the last nine in Nebraska. I cannot remember another time when we have had the conversations that are presently taking place in schools and communities regarding student achievement.

From the board of education to the classroom teacher, there is little doubt that our priorities have shifted dramatically to place the emphasis where it should be: student learning and accountability for that learning. School leaders in our state no longer spend the same amount of time talking about budgets and buses for they now are discussing teaching and learning strategies to improve student achievement. Standards, assessment and accountability have put classrooms in the forefront.

Nebraska school system leaders have risen to the challenge of increased accountability brought on by the introduction of state content standards in 1998. While many of us gnashed our teeth when Christensen unveiled the High Performance Learning Model 10 years ago, we have moved ahead collaboratively to make it work at the district level. As a result, most superintendents would say the changes taking place in our state's classrooms have benefited all children.

Nebraska is considered a rural state with a high-achieving, relatively homogeneous and stable population. However, many Nebraska school districts now educate much larger minority populations and special-needs groups. These demographic changes will require many districts to rethink their educational delivery.

Nebraska also is a local control state with minimal bureaucratic interference even in this day of increased statewide accountability in many places. In fact, the state's expectations for its schools are both reasonable and realistic: Schools must be able to demonstrate to their constituents that they have cohesive and aligned curricula, that they are able to measure what they teach and that academic growth is evident. These are achievable expectations that the state has set for every school district.

The fact that our data needs to be reported locally and will likely be displayed across the state comparatively has created a window for leadership to think "outside the box" when creating strategies and interventions to promote student learning. Many of these interventions -- calendar alterations, summer and other extended programs and summer jump starts -- are a response to the state's use of school improvement data. Interventions can work only when decisions are driven by student learning data.

Standing Tall

Healthy debate is continuing in America over the best approach to improving student learning. At the CCSSO conference this summer, a lively but respectful argument took place between proponents of a single test to measure student success and those who support approaches using multiple assessments, including some measures developed locally. Both sides did their best to convince the other.

Those of us from Nebraska left the meeting with a greater resolve to continue our work at the local level that emphasizes value-added growth for students. We have stood tall in resisting federal pressure and remain committed to accurately measuring our students' learning in classrooms, We believe this is the most accurate gauge of what is successful in our schools and what needs to be improved.

Yes, we had our early concerns about the commissioner's High Performance Learning Model, but we harbor no doubts now. Nebraska has created an environment where conversational synergy between educators and their communities will ultimately benefit all students through better teaching and learning and the improvement of schools.

Steve Joel is superintendent of the Grand Island School District, P.O. Box 4904, Grand Island, Neb. 68803.