When raising isn't rising: the failure of accountability systems to measure student growth over time.

Ask any educator in the hallway outside your door: "So what is standards-based reform all about?" In all likelihood, the answer will be something to do with "raising expectations" or "all children can learn to high levels."

In one sense your informants are correct: American educators and policymakers have hashed out the ongoing importance of proficient calculation, defined the role of phonics in balanced literacy and declared an uneasy truce about whose history must be taught. Most also agree that two years of business math is not enough for a graduating high school senior. An 8th grader should be able to write a coherent and persuasive essay. And 3rd graders should read fluently and critically.

Having raised the standards, educational decision makers have moved on to raising the stakes for failing to meet these higher expectations. For students, the consequences run from mandatory summer school to no high school diploma without passing the state test. Likewise, increasing numbers of teachers work in systems where either the school or the individual teacher may receive cash awards based on student performance. At the school level, a staff faces clear benchmarks to hit as well as costs for failing that range from technical assistance to reconstitution or closing.

A harsh or even just a candid critic of the standards-based school reform could say, "Right, OK, so the adults can sleep at night. They 'duked it out' and got the standards and consequences down on paper. Now about the children ...?" This is more than casual irony. Behind the remark lies the most fundamental question in school improvement: Having invested heavily in 'raising' both the standards and the stakes, what investment are we willing to make to support students in 'rising' to meet those standards?

The Dominant Model

The newly enacted Elementary and Secondary Education Act of 2002 (No Child Left Behind or NCLB) puts this issue of rising to meet the standards at the forefront of its framework. The law explicitly requires using annual individual testing of children in grades 3-8 in mathematics and literacy to drive accountability for changing the level of student performance.

Specifically, the NCLB legislation requires schools to establish a base-line performance and then to show "adequate yearly progress" for each of 10 successive years, with the goal of 100 percent of children performing at the proficient level in mathematics and reading. As with any vision, the devil is in the details. The details of NCLB all reflect a specific model of accountability, a model that reigns in districts and states as well.

To illustrate this point:

* The legislation is entirely based on the use of data from standardized tests. While such tests provide an efficient and reliable measure of some aspects of student achievement, as currently formulated, few of the widely used tests probe students' mastery of complex or high-end skills: developing a finished, as opposed to a first-draft essay; interpreting data from an experiment or translating a conversation into another language.

* Testing is designed to look at student achievement at a particular point in time. Even though NCLB testing will soon be yearly, the tests and analyses examine the variance within student performance at a specific grade level. The fundamental questions are: What percentage of 4th graders score at basic, proficient and advanced levels? And how is this pattern of scores different from those of last year's 4th graders? The basic approach is to conduct repeated comparisons of specific years of performance--not successive years of individual or groups of children's achievement.

* Just how test score gains defined in this way fit into mastering the standards is unclear. For instance, it is an open question as to how many years of gain can be covered simply by helping children to answer correctly increasing numbers of relatively low-level items without substantially changing children's command of fundamental concepts or important strategies.

* The NCLB legislation addresses the problem of improving student achievement and closing the gap as if every interval of change were the same. Yet early gains likely can be achieved through teaching the format of the test, while later gains can be achieved by responsibly teaching basic skills. By contrast, we have little knowledge about teaching the thinking skills or cultural capital that will be the substance of later gains.

Competing Models

These are substantial issues, in and of themselves. But a still more fundamental problem exists. As suggested above, the accountability mechanisms proposed for NCLB use an attainment, as compared to a development, model of accountability

In attainment models, no one asks about the longitudinal history of current scores. The danger of this is readily apparent. This year's 8th graders may outstrip last year's 8th graders, but they may be substantially underperforming relative to what they accomplished in 7th grade. Potentially, the school or the district has a bold and broadly successful middle school program for 6th and 7th graders that dead-ends into a test-driven final year that endangers students' transition to high school.

But a cross-sectional attainment model--for instance, only successive testing of 8th graders--would not expose that critical pattern of accelerated and then stunted growth. Instead there would be the puzzle of "rising" 8th grade scores and slipping high school performance--a scenario that could easily, but wrongly, focus on finding out what's wrong with 9th grade. The annual focus at selected ages also pushes educational communities to think only in terms of concurrent or short-term effects. A new literacy program has to affect test scores right away to be sustained. Few districts ask about effects that show up later.

Consider a well-documented example. The Third International Mathematics and Science Study, or TIMSS, showed that U.S. 4th graders performed well in international comparisons, while U.S. graders performed quite poorly. The common inference is that there is a sharp decline in the quality of teaching and learning between 4th and 8th grades. But the 4th-grade performance could actually be a decline from where it was at 3rd grade. Or growth could continue during 5th and 6th grades, declining sharply in 7th when early algebra begins. Only if we bother to research the shape and direction of the growth trajectory do we know where to start looking.

We find equally pressing instances around the "achievement gap"--the divergence in the scores of mainstream, high-status children and children who have been accorded lower status. NAEP and other data sources make it clear that sizable differences are evident by the 4th grade. Typically educators monitor the widening or the closing of that (and other) time-specific gaps.

But we cannot ignore the developmental question of when the distance between the learning trajectories of high- and low-status children diverge. Is it already substantial at kindergarten? Does it suddenly widen when reading replaces talking as a primary mode of learning? At what other points does it grow wider? This is critical data in thinking about where and how to intervene. As these examples indicate, if we are looking for information to inform our actions, then widely spaced age sampling is not enough--we need to understand what happens to students' learning over time.

The Pressing Need

If we are serious about children rising to meet the standards throughout their years in school, we need a developmental approach to accountability systems. Basically, these models follow populations of children over time, along several dimensions (their literacy, their mathematical skill, their engagement with learning outside of school, even their health).

The basic forms of analyses focus on change over time, or growth. The effort is not only to track growth but to develop models of growth that can help educators to identify factors that affect it and identify when children first begin to diverge from continuous development. The data and concerns that flow from developmental models are an essential complement to the attainment data that we now use so exclusively.

Vital as developmental approaches to accountability are, the sheer demands of compliance with NCLB are likely to keep states and districts tied to familiar attainment models. Any move to developmental models for accountability will have to take root at the local level. It will be forward-looking school administrators, teachers and community leaders who will play the major roles in beginning a conversation about achievement that focuses on continuous growth.

Why would any school or district take on a developmental approach to accountability--given the extraordinary demands exerted by increasing accountability for attainment? Only because the educators understand the urgency of understanding rather than tinkering with the challenges they face. Consider this example: Across the country, administrators in many districts are face-to-face with stubborn problems in the area of adolescent literacy. Far too many high school students can't read or write at more than the note or shopping-list level. Many of these same districts are stunned to realize that their 17- and 18-year-olds got this far without such fundamental skills and habits of mind. Yet rarely do districts or schools think about the long-term history of students' literacy learning. Still more rarely does a faculty have the chance to reflect on where these patterns of adolescent literacy may be rooted. But schools can take steps to change this.

It can (and perhaps should) begin at the most local level. Working with protocols developed at the Annenberg Institute for School Reform, the principals and literacy coordinator in a moderate-sized district decided to take faculty inquiry beyond examining the test scores. They called on teachers in all subjects, K-12, to collect assignments where students engaged in informational reading and writing (reports, evidence-based essays) throughout an entire month. She asked teachers to collect completed assignments from currently successful, competent and struggling students.

Using a professional development day, teachers spread out their samples from competent (average) students, making a developmental progression leading from K-12 across the gym floor. Teachers, working together across disciplines, examined the work developmentally, beginning in kindergarten and working up to 12th grade. As they examined the work, teachers took notes about where they observed rising expectations, plateaus of unchanging expectations and cliffs (defined as sharp rises in expectations that led many students to fail or do poorly).

Using these data, teachers sketched the growth trajectory they saw implied in the assignments and the resulting student work. In a debriefing session, teachers shared their observations and were able to isolate major trouble spots where students' growth foundered. Here are some examples from the observations they made as K-12 peers:

K-3rd: All letter and sound practice, no opportunities to create/invent meaningful records. Elementary grades deal in story all the time, no experience with informational writing until 4th or 5th grade -- no foundation for informational reading and writing.

5th-7th The topics for reports change (Aztecs, castles, American history), but the reports are no more demanding than in 4th.

8th: Big research paper appears, as much as one-third of student's grade second semester; sudden appearance of formal research style (footnotes, bibliography). No one is objecting to huge amounts of Internet cut and paste.

9th: Assignments assume kids know how to write in a discipline. But kids are still writing up the "story" of what happened in their lab experiment ("First, we talked about...).

10th-12th: Many report assignments across subjects but no evidence that students are being taught what a report in chemistry is versus one in history.

This is only a brief scenario. However, it provides an example of how one group of teachers broke step with familiar attainment models in order to focus on the longer-term origins of a low level of performance and interest in high school students. It also illustrates an aligned rather than a finger-wagging approach to the origins of stubborn problems in student achievement.

In this model, the level of student reading at 8th grade is the responsibility of teachers, children and families from kindergarten on. It was a conversation that initiated a more diversified elementary literacy program, as well as efforts to teach (not assign) more sophisticated forms of reading and writing during what was previously the "desert" of middle school.

Promoting Growth

This kind of vertical alignment is one example of the professional activity that skilled school administrators will have to practice in order to focus attention on the most critical process in school reform: student growth rising to meet the standards. Certainly other practices will help:

* Promoting strong diagnostic skills. School staff, particularly teachers and counselors, must be able to look at a piece of student work and have good intuitions about what the next step is for that child. Asking for evidence of this kind of diagnostic skill ought to be a part of every hiring or promotion interview.

* Working toward vertical alignment. To teach 5th grade reading, a teacher needs to understand what was accomplished in 4th and what the demands of 6th grade reading are likely to be. It is time to balance time and resources between grade-level and vertical teams.

* Supporting practices that promote development. One example would be the practice of "looping" elementary school teachers with the same group of children across multiple years.

* Protecting children from circumstances that impede growth. These would include high rates of mobility, encountering a series of new or poorly prepared teachers or low-standards tutoring or homework sessions.

* Developing new approaches to intervention designed specifically to deliver support early. For instance, consider a year round middle school designed so that the frequent inter-sessions function as opportunities to catch up before lags are huge and discouraging.

A Local Catalyst

Clearly, we know how to raise standards. However, we are less clear on how to support students in rising to meet those standards. In part this is because our fundamental model for accountability is one of attainment. We are content to measure the number of children who meet a particular standard at a particular moment in time.

But ensuring that over time large numbers of children rise to meet the standards, actually demands a different model of accountability. It demands asking how many children are on a path to meeting the standards and if they have diverged, when and why did that happen.

In the coming years, it will depend largely on local educational leaders to articulate the need for thinking in terms of continuous growth. They will be the source for turning the current habit of raising standards into the reality of students and their teachers steadily rising to meet those standards.

Dennie Wolf directs the Opportunity and Accountability Initiative at the Annenberg Institute for School Reform, Box 1985, Providence, RI 02912. E-mail: [e-mail]. She is a member of the National Assessment Governing Board.