Retaining retention: how Chicago changed--but ultimately saved--its controversial program to end social promotion.

In a remarkable confluence of events last winter and spring, questions about student retention became the swirling center of education debate in New York City, Chicago, and, by virtue of these cities' size and clout, throughout the country.

In New York, Mayor Michael Bloomberg announced in January that city schools would no longer pass 3rd graders on to the next grade if they had not mastered minimal reading skills on an hour-long reading test. Civil-rights and community activists were opposed, and they threatened to sue to block the plan. The district's decision to hire teachers for summer school--part of the retention plan--outside of union hiring rules created another furor. And some parents and community members protested the new initiative as too harsh.

While New York churned, Chicago, where a pioneering student retention policy had been in effect since 1997, was hit in April by two studies from the Consortium on Chicago School Research (CCSR), a nonprofit group affiliated with the University of Chicago. The studies suggested, with seeming definitiveness, that ending social promotion was ineffective, at best, and possibly destructive.

"We should scrap what we're doing and go back to the drawing board," University of Chicago professor Melissa Roderick, one of the lead CCSR researchers, told the Associated Press, in a story reprinted around the country.

Roderick's message in the CCSR press release accompanying the report itself was somewhat more nuanced: "The bottom line is that, without substantial supports, neither social promotion nor retention will improve low-performing students." But this qualification was apparently lost on newspaper editors, who reprinted the story under headlines like, "Social promotion fight didn't help in Chicago" and "Program to curtail 'social promotion' fails."

[ILLUSTRATION OMITTED]

The media also largely ignored other CCSR studies that had indicated that Chicago's retention policy was popular among the city's teachers, principals, and parents for, among other things, setting clear standards for student performance (see "Educators and Students Speak," page 49). Nor did the media consider whether the retention policy had contributed to the dramatic overall achievement gains seen in Chicago since the policy's adoption.

In New York, the critical CCSR studies helped fuel opposition to the new retention program. Mayor Bloomberg had to remove two of his own appointees from the school governing board, a move that brought widespread criticism, in order to ensure adoption. And even before the program was fully implemented, the district was forced to add an extensive appeals process.

The Chicago Board of Education, also responding to the new CCSR reports, immediately modified its student retention program by dropping math scores as a consideration in retention decisions. That move, seeming to herald the end of one of Mayor Richard Daley's signature school reform initiatives, was labeled by the Chicago Tribune as "the social promotion surrender."

Staying Power

Despite the controversy--and the retreats--the programs have survived. But the events of last winter and spring illustrate the roller-coaster ride that awaits any school district that tries to tamper with a core education policy like student retention. Student retention, one of several lightning rod issues that bring parents, union leaders, politicians, and academics together--to fight--is not a task for the faint of heart.

It's thus fitting that the city of broad shoulders, guided by a mayor named Daley, should lead the way. Chicago's pioneering program is now in its eighth year, having withstood its share of icy gusts from multiple directions. For those who have been paying close attention, the brawl that broke out last spring was just the latest in a rolling clamor to control one of education's linchpin topics. The stakes are high; the questions, many and complicated.

Does retention work? For whom? How do you justify a program that may benefit a school district overall, but also presents potentially life-altering challenges to individual students who are flunked? How do you measure its success? What do you do with students who are held back? How do you pay for it? And, if the policy does work, shouldn't you see an eventual reduction in retention rates?

To be sure, Chicago is not the only district that retains students. Over the decades, the practice of holding students back has waxed and waned in the American school system. Recent studies have found that roughly 15 percent of students repeat a grade between kindergarten and graduation. Such student retention seems politically popular, having been endorsed by the past two presidents.

But Chicago's program may be the nation's largest sustained effort by a major school district to end social promotion. It provides an intriguing and revealing view of the multiple issues that confront school systems that seek to change a well-entrenched policy--what it takes to endure and when it is time to change course.

In the Beginning

In the spring of 1995, the Chicago Public School system, the nation's third largest and, arguably, one of the most troubled, made national headlines when it was taken over by Mayor Daley, then starting his third term and the most popular mayor since his father, Richard J. Daley, held the office for 21 years. With the new powers granted by the Illinois legislature, Daley fils quickly formed a new school management team, which, just as quickly, unveiled what one local paper called "a dizzying array of new programs," including principal training, longer school days, expanded preschool, college classes for the gifted, hiring of parents as truant officers, and alternative schools for dropouts.

[FIGURE 1 OMITTED]

But by far the biggest reform introduced by Daley and Paul Vallas, chief executive officer of the Chicago Public Schools from 1995 to 2001, was the education equivalent of just say no: ending the practice of social promotion. The new policy required Chicago's lowest-performing third-, sixth-, and eighth-grade public school students to attend summer school and possibly repeat a grade at least once if they did not meet minimum reading and math test-score cutoffs on the Iowa Tests of Basic Skills (ITBS). This shift toward greater accountability also included putting "low performing" schools, those in which fewer than 15 percent of elementary students met national norms in reading, on an academic watch list. What had been whispered in policy reform circles for years and tested in small public school districts and pockets of reform around the country was now policy in a major city school system. Suddenly 412,921 students in 475 elementary schools--specifically, children in grades three, six, and eight--were to have the progressive rug pulled out from under them.

It was an ambitious accountability agenda, and in the years that followed the policy's implementation, test scores among Chicago's lowest-achieving students rose, particularly in the upper grades, while the proportion of schools with extremely low performance fell. The improved scores were impressive enough to lead several states and other major school districts, including New York, to adopt elements of the Chicago Public Schools (CPS) policy--making student progress toward the next grade dependent on demonstrated achievement on standardized tests. President Bill Clinton visited a Chicago school in 1997, proclaiming, "I want what is happening in Chicago to happen all over America."

Since 1997, the policy has resulted in thousands of Chicago's elementary students being required to go to summer school and being held back every year. Not counting special-education and limited-English-proficient students, from 6.8 to over 20 percent of students have been held back each year, with a peak of nearly 15,000 students retained in 1998 (see Figure 1).

A Little Success Goes a Long Way

Despite these large numbers--which indicate a huge systemic change--and despite the long-standing practice of social promotion in public schools, the rationale for holding students back seemed readily apparent to many parents. According to a 2000 Public Agenda survey, more than two-thirds of all parents nationwide supported retention even if it meant that their child would be held back.

One reason for the popularity of the retention policy in Chicago--aside from the advantages of having a popular mayor to promote it--was the strong and relatively steady improvement in the Chicago schools. Over the past eight years, the dropout rate has decreased from 16 percent to 13 percent. The percentage of elementary students meeting national norms on the ITBS in reading has increased from 36.9 to 43 percent. The percentage of students testing in the bottom quarter of the ITBS has dropped from 32 percent to 24.4--better than the nation as a whole, according to CPS officials.

In fact, even while things were looking bad last spring, a new study by Brian Jacob of Harvard and Lars Lefgren of Brigham Young provided more evidence to support the proponents of Chicago's retention program. Jacob and Lefgren examined the records of some 148,000 students in Chicago's third and sixth grades for 1997-1999, comparing students who had just passed the retention cutoff test with those who had just failed it. The two groups were virtually identical except that the latter was required to attend summer school and faced retention. The study revealed that the policy resulted in substantial academic improvement among the third graders who faced retention. The effect was insignificant for sixth graders.

"Our results indicate the achievement gains that are possible with remedial education," said Jacob in a press release issued at the time, seeming to buttress Roderick's hedge about the need for "substantial supports." "It is obviously important to track the longer-run impacts of these programs. However, in the quest for higher standards and achievement, these programs may offer some hope for students struggling to meet the bar."

An Easy Target

The Jacob/Lefgren study, however, did not receive nearly the attention of the one-two punch effect created by the negative CCSR studies and New York's controversial unveiling of its new retention policy. And it surely did not quiet the critics, who continued to complain that children are too young to be penalized for reading struggles, that retention has a potentially negative effect on children's love of learning, and that the stigma attached to being held back while the rest of your classmates move on is not worth it. The policy could still be cast as an ineffective, politically motivated, and discriminatory approach to school reform: an easy target.

The policy was the subject of a civil-rights lawsuit filed in 1999 by a group called Parents United for Responsible Education (PURE). Retention opponents like Don Moore, executive director of the education reform organization Designs for Change, frequently criticized the policy for being a misuse of standardized test scores that simply overidentified poor and minority children for retention. (The PURE suit was settled in 2001 and resulted in modifications of the policy to take other, behavioral, factors into account and to adopt an appeals procedure.)

In truth, the policy had some glaring flaws: among them, multiple retentions for some students, an undisclosed and ill-defined waiver program that allowed thousands of students to go on to the next grade despite not meeting the official standards during the first three years, late or even last-minute notification about retention decisions, and an inconsistently implemented appeals process. In addition, it wasn't particularly well aligned with the district's other main accountability program, school probation, which focused on second-quartile students rather than the lowest-performing students in a school.

Financially, the reality is that reeducating 10,000 students a year, at roughly $8,400 per pupil, costs no small chunk of change. Districts often ratchet back on retaining students during tight economic times, and, though it was once flush with cash, the Chicago school system faces an ever-growing budget gap. Business support began to show signs of waning even before the pessimistic CCSR studies. Left Behind, a scathing 2003 report by the business-led Civic Committee of Chicago concluded that Chicago's school reform efforts had failed to raise student achievement sufficiently and that a new effort at reform was needed.

Not surprisingly, positive media coverage of the school system began to slide. Though the Tribune remained a strong supporter of the retention policy, in 2003 the Chicago Sun Times published a two-part series, "The Daley Generation," that criticized it. Roughly 100,000 students had been held back since the program began in 1997, the paper reported. It said that more than a quarter of the 36,000 students who entered first grade the year Daley took over the schools in 1995 had been held back at least once. The paper suggested that if the program were working and the system improving, those numbers should be decreasing and more middle-income parents would be sending their children to Chicago public schools. They weren't.

Often too little happened when a student was held back. In theory, retained students were supposed to participate in an enriched, accelerated academic program that would, through additional help and tailor-made interventions, result in a better outcome at the end of the repeated grade or even help the student catch up to his or her classmates. Despite the apparent success of the Summer Bridge program, the school system failed repeatedly to come up with additional supports and adjustments for students who were held back. Efforts to get those support and remediation programs up and running were plagued with problems and never fully implemented. In reality, most retained students simply repeated the previous grade level.

In the meantime, the increases in test scores slowed.

The Devil in the Details

When the two critical CCSR studies were released, the timing couldn't have been worse for the proponents of retention. On top of what seemed a run of desultory news, the University of Chicago researchers reported that, over a period of two years, third graders who were retained did just as poorly as similarly low-performing third graders who weren't held back. Retained sixth graders did worse than their counterparts who were not held back. And retained eighth graders were far more likely to drop out and at a younger age. In fact, 78 percent of the students retained in 8th grade had dropped out by the time they turned 19.

Though longtime supporters of stronger accountability were quick to express their dismay at what seemed to be the school board's even quicker abandonment of the math component of the retention standard, the Tribune's "surrender" editorial predicted what was likely to happen. "Slowly returning to social promotion relieves the pressure on teachers, parents, and students, but ultimately devalues the diploma the kids receive."

All but unnoticed at the time, however, were the CCSR studies that had found benefits in the retention program, as well as the Jacob/Lefgren research, which seemed to dispute some of the substantive negative findings, at least as they applied to the effect of retention on third graders.

None of the studies, though, attempted to measure fully the impact of the policy on students who might have been motivated to work harder to avoid being held back, or on teachers and schools; nor did they parse the effect of student retention on overall system performance.

As some observers have noted, student retention policies are not really about the students who are retained as much as they are about the way the rest of the school system operates when it knows that there is no social promotion. Researchers like G. Alfred Hess, an education professor at Northwestern University, are quick to point out that the justifications for retaining students should not be based solely on their impact on retained students, but rather on the effect of the threat of retention on all students and their families. "This dual intent for ending the social promotion policy is frequently ignored by its opponents and is rarely considered in evaluating the effectiveness of the policy," Hess writes in School Reform in Chicago (2004).

The News Left Behind

Finally, though, a deficiency of the CCSR studies was that they focused on the early years of the retention policy and thus failed to take into account more recent modifications to the program. In fact, a crucial omission in the press coverage of the retention program was just that: the many adjustments to the policy. And a closer look at the 2004 changes suggests that they were actually just the latest in a long series of alterations that have been made over the years, some to strengthen the program, others to soften it.

Minimum test-score requirements have changed a number of times. As initially implemented, the policy affected primarily those who were a year below national norms in 3rd grade, or a year and a half below norms in 6th and 8th grades. The promotion requirements were steadily ratcheted up during the first three years (1997-1999), but eventually were turned into "bands" around minimum test scores.

The waivers process through which students could pass along to the next grade without meeting test score cutoffs was fleshed out in 2000, formalizing procedures through which classroom grades, attendance, and teachers' recommendations could be factored into retention decisions. (Researchers found that some parts of the city were giving more waivers than others, and the total number of waivers was more than reported.)

Classroom behavior, grades, and attendance were all given a more prominent role in the retention decision, and an appeals process was, in theory at least, implemented for parents who wanted to dispute a retention decision--in part the result of an agreement with the U.S. Office for Civil Rights that was brought on by the 1999 PURE lawsuit.

In 2003, the board informally adopted most of the changes codified in 2004, including the focus on reading, but not math scores. Dropping math was not only not new, it may have been less of a change than it seemed. "Math was never a big factor," says Jenny Nagaoka, one of the Consortium report's coauthors. "For the most part, kids ended up being retained because of reading."

In September of 2004, as school opened, Chicago officials announced that fewer students--just under 8,000--would be retained. Was it surrender, retreat, or victory? Did it mean that ending social promotion was working or that the strict retention policies that were the program's hallmark in 1996 had been abandoned?

In the end, the changes made by the Board of Education last spring were neither as new as many thought nor as substantial as they may have seemed at first glance. About this, close observers, both critics and supporters of student retention, seem to agree. According to a May cover story in Catalyst, the Chicago magazine that covers education issues, the board merely "tapped the brakes" in response to the latest research findings. The new policy is "not likely to reduce significantly the number of students who are retained," says Don Moore, who described the policy change as "mere tinkering."

Even Jenny Nagaoka said that while the Board of Education had implemented most of the CCSR's recommendations "beautifully" in regard to preventing the need for retention, it had not backed off on retention itself. "They took all our recommendations but the biggest one," she said.

Getting the Story Straight

Senior officials in Chicago themselves contributed to some of the confusion about the changes to the retention policy. In the same week, for example, Arne Duncan, who had taken over from Paul Vallas in June of 2001, described the changes as "finetuning," while his chief education officer, Barbara Eason-Watkins, said the new policy was "intended to dramatically reduce the number of students who must be retained." And the announced enhancements, which included kindergarten literacy assessments, full-day kindergarten, smaller class sizes, keeping teachers and students together during the early grades, and individualized learning plans for students at risk of being held back, gave no indication of how dramatic the changes were. Another change: summer school would be required for at-risk students as well as those faced with retention.

Whatever the causes--and however they were characterized--one result of the policy shift was that few people were aware of the existence of the other studies, of the overall progress made in Chicago schools over the past decade, or of the fact that teachers and principals think the policy is working and that kids report that teachers and parents are helping them out more.

In the meantime, more recent policy initiatives--such as Mayor Daley's new "Renaissance 2010" plan, announced last summer, to close 60 low-performing schools and in their place create 100 new small schools, many of them charter and contract schools without union teachers or local school councils--seem to have replaced student retention as a focus of community concern and opposition.

Back in New York City, the first year of the Big Apple's retention program seemed to have gone smoothly. In contrast to previous attempts at mandatory summer school, most students showed up for class in the summer of 2004. And more than 40 percent of the students who had failed their reading tests passed after going to the beefed-up, $32 million summer school program, more than doubling the previous year's rate of 19 percent. Thus, somewhat ironically, the overall number of retained students was about 3,600, just 500 higher than the year before, when there was no revolutionary retention policy. Not surprisingly, in September, Mayor Bloomberg announced that the retention program would in the future include 5th grade as well as 3rd grade.

"By now it's a familiar pas de deux in the public schools," wrote New York magazine columnist Robert Kolker about the early months of retention and the relatively small actual impact on retention figures.

Alfred Hess writes that ending social promotion may have been "neither as destructive as some opponents predicted nor as miraculous as Vallas and others hoped." In both Chicago and New York, however, what remains unknown is whether the school systems will do any better by the retained students than they have in the past--thereby eliminating one of the chief remaining arguments against retention.

Alexander Russo is a freelance education writer based in Chicago. He is the editor of School Reform in Chicago: Lessons in Policy and Practice and author of the weekly Internet newsletter "This Week in Education."