Multiple measures: Beginning with ends: One illinois district uses more than scores to assess its students and schools.

It is a miracle of the Internet--a miracle born of one school district's refusal to accept simple solutions to complex problems.

Our problem was one many school districts face: How do we define what we want our students to know and be able to do at the end of their journey with us? How do we determine what kinds of adults we hope they will be? What value will we add to their learning and their lives? How do we establish targets for quality improvement? Are there deeper questions that need to be considered as we serve the needs of the 4,200 students in K-8 in our suburban Chicago school district?

These were the questions that our board of education struggled with as it began to ask what lay beyond test scores.

A Composite Index 

A major step toward a resolution of the dilemma came when an Internet search turned up an article in the May 2000 issue of The Elementary School Journal titled "Toward a Composite Index of School Performance," which addressed the question of what students should know and be able to do. The author, Richard Rothstein, argued that although "dialogue about schools focuses mostly on student test scores in reading and math, and to a lesser degree, science ... American leaders, citizens and parents have repeatedly emphasized that a broader array of outcomes and processes comprise schools' purposes."

Rothstein introduced a concept he called a "composite index of school performance" to address this issue. He added: "School controversies focus mostly on academic test scores. But Americans insist that schools have broader purposes, including also equity of outcome, citizenship, social ethics and health, and Americans want children to be happy in schools."

A nice statement, yet Rothstein moved beyond the rhetoric to propose a model that defines dozens of possible components to the index, grouped in four areas: academic outcomes, non-academic outcomes, processes and consumption measures. His index included the predictable competencies in the core academic areas, but also included ideas such as wellness and social ethics. He suggested looking at teacher quality and parental involvement as well as school facilities, class size and freedom from violence. He further suggested that perhaps it was important for weights to be assigned to these factors.

Rothstein's premise was intriguing. However, he cautioned that many of the factors, perhaps even most of the factors, were beyond the ability of individual schools, school districts or state agencies to control. Nevertheless, our interest was piqued. Could the concept be applied to a single school district? We were not sure, but we were ready to begin the journey. Our school district theme in 1999 had been "If Not Us, Who?" A year later, it became "If Not Now, When?" and in 2001, "Serving the Needs of the Whole Child--Looking Beyond ... ." Easy words to say but tough concepts to make real.

If finding Rothstein's article was a miracle of the Internet, it was just the beginning. We were prepared to let the Internet also help us find Rothstein. Our search engine quickly located him. We found Rothstein was a research associate of the Economic Policy Institute in Washington, D.C., and an adjunct professor of public policy at Occidental College in Los Angeles. It did not take long to locate his e-mail address and fire off a message. To our great surprise, he responded within hours and began a dialogue on our mutual interests.

Rothstein appeared interested in what our district was doing. He came to realize we were genuinely interested in looking beyond standardized test scores, and we realized his composite index could form a template on which to build a series of measures that could be shaped to our district's priorities.

After several e-mails and conference calls, Rothstein visited our district. He met with board members at an early morning breakfast meeting and joined us for another meeting that night. His comments and observations proved both encouraging and disappointing. Rothstein told our board members that what they were attempting to do was difficult. And although many school districts grapple with these issues, our board's commitment to go beyond test scores alone as a measure of success was remarkable and to his knowledge unique. He cautioned, however, that this endeavor was fraught with obstacles. Board members would have to use care that the goals they adopted truly reflected community values and that the community would accept this approach.

Rothstein visited our middle schools, talked with teachers and asked many questions. Out of these visits he wrote one of his weekly Lessons columns in The New York Times. Titled "A School District Refuses to Worship Scores Alone," it described our interest, spotlighted our attempts and encouraged us further.

A Broader Portrait

Looking beyond performance on standardized testing had been discussed in our district since at least 1997. At that time, four school board members managed to be re-elected in spite of a challenge that our test scores were not high enough. The successful candidates acknowledged that test scores did not tell the whole story. As one board member asked, "What is the rest of the story?" Another wondered if we should spend more time addressing bullying at our middle schools than trying to push up our test scores? The concept of "educational ends" was born.

In truth, we had been looking at success indicators for some time. We were interested in the work done by the Oak Park, Mich., schools and reviewed their information-based decision-making process. In conversations with them we learned they had worked with their community to identify indicators for success, determine the data sources for measurement and decide what was acceptable performance at the student, school and district levels, They reported success in moving beyond test scores alone to looking at a variety of objectives. Their work served as an early model.

We had spent several years educating the school board and the broader community to the larger picture of student performance. We decided to present our model of the District 64 Student Profile anytime we reported test scores to inform the board and community on the broader picture. We had our work cut out for us.

The national debate on high-stakes testing continued. Many states, and now the Bush White House, were still pushing harder for high-stakes tests as the gatekeeper for students, schools and districts, We were prepared to join the movement against this approach, led most notably by Alfie Kohn, author of The Case Against Standardized Testing. Board members and other leaders in the district increasingly understood that standardized tests put children into "boxes" and such tests fail to account for the complexity of the school's mission to serve the needs of all students.

The challenge to public schools comes not only in moving parents, school boards and the public beyond high-stakes testing but in specifying ends in non-academic areas. Our journey began when we formed building based quality improvement teams six years ago. Each building was asked to define an academic and a non-academic goal.

In retrospect, we were fortunate that the administrative and teacher leadership at our middle schools began to discuss Daniel Goleman's book, Emotional Intelligence: Why It Can Matter More Than IQ. Trust, stress management and anger management are now taught as important components of the overall middle school curriculum.

Additionally, for the last three years, the district has had the good fortune to have an Elementary Learning Foundation that funds a day-long ethical leadership conference in the summer for students as they enter 6th grade. Larry Nucci, professor of education and psychology at the University of Illinois at Chicago, helped us develop a unique curriculum for this summer conference and other associated activities. The overall concept is to change the middle school culture into one that reflects a more civil atmosphere. Several of our middle school teachers have picked up on Nucci's work and are writing units that integrate ethical leadership and moral development into our existing curriculum.

Beyond Tests

Our board now understands that even though the school cannot control how families, peers, the community and other forces have an impact on children, the school can and must have a positive impact on the lives of our students. During recent elections, some board members actually dared to challenge the point of view that test scores are not the Holy Grail. We took large steps forward. But questions remained: How does a community decide what is important? How does the community make these decisions?

The time seemed right for us as educational leaders to demonstrate to our board members and school community the value of this composite approach, of identifying educational ends that would move us beyond test scores to a much broader arena.

During the past year, curriculum leaders, teachers, principals and district administrators worked to develop educational ends statements in all academic areas, including language arts, mathematics, science, social studies, health and physical development, fine arts, music and foreign language. In language arts, for example, ends statements include students' abilities to read proficiently and with fluency, to read both for pleasure and information, to communicate ideas orally and in writing for a variety of purposes and audiences and to listen effectively and empathetically in formal and informal situations. Assessment tools include standardized tests, surveys, checklists and teacher observations. Long-term goals include surveys of student attitudes toward reading as well as reviews of student literacy (reading and writing) portfolios by parents and principals.

We also included ends statements about critical thinking and problem solving, social and emotional areas and the school environment. We proposed to assess whether students treated themselves and others with dignity and respect and whether they were able to apply conflict resolution strategies to their own life situations. Developing measures for these areas is a continuing need.

Certainly many schools and districts are doing great work similar to this. We do not profess to be on the cutting edge in data collection and manipulation. However, we believe we push the envelope regarding the totality of what we are looking at and how we will use our ends statements to develop action plans to achieve our goals--all the time using data (in an affordable, sustainable, non-intrusive manner) to verify our progress in the journey.

The end result is a synthesis of all these activities. We now survey special education parents after the end of the school year. We do follow-up surveys with our 6th-grade and 9th-grade students. We collaborate with our high school district in a drug and alcohol survey on a recurring basis for all 7th - through 12th-grade students. We have broken new ground. The board of education is empowering the district's leadership to present educational ends as goals in areas beyond traditional academics.

New Ground

The work is not finished. It is a work in progress. We do not have all the answers, but we think we know most of the questions. Our next steps will be to present an administrative recommendation on the ends we believe are important and to suggest specific weights for each end. We will engage the board in this discussion. The community will be invited to be part of the dialogue.

The acceptance by the board and the broader community of this mission and of the realization that standardized test scores alone must not be worshipped indicate progress toward serving the needs of all our students.

This is not the rest of the story.

We are beginning a long journey that will surely be fraught with difficulties. By acting as educators, doing the work that school leaders are charged to do, pulling people together and listening to the community, we are positioned to break new ground for the good of each child in our care.

Fred Schroeder is superintendent of the Park Ridge-Niles Consolidated School District 64, 164 S. Prospect Ave., Park Ridge, Ill. 60068. Sally Pryor is assistant superintendent for curriculum in the school district.

RELATED ARTICLE: Origins of a Composite Index

Richard Rothstein's article that inspired school leaders in the Park Ridge-Niles Consolidated School District argues that schools should be held accountable for the full range of outcomes that the public wants them to generate.

A research associate with the Economic Policy Institute in Washington, D.C., Rothstein proposes a composite index that balances the various measures used to evaluate student and school performance.

His proposal suggests the following:

* 1.0 core academic outcomes (literacy, mathematics, history, foreign language, creative arts, etc.) comprise 40 percent of the index;

* three nonacademic outcomes (citizenship, wellness and social ethics) have a 25 percent weight;

* three process indicators, including school completion and parent involvement, comprise 15 percent; and

* three consumption measures (building conditions, school security, class size) provide 20 percent of the composite.

Rothstein's detailed article, "Toward a Composite Index of School Performance," appeared in the May 2000 issue of The Elementary School Journal, published by the University of Chicago. The publication is not available online in full text, but a single copy of the article can be ordered through the journal's Web site (www.journals.uchicago.edu/cgi-bin/vendor?journal=ESJ).