New kids on the block: results from the Moving to Opportunity experiment.

If only we could move poor kids out of high-poverty, inner-city neighborhoods. Moving to more affluent neighborhoods would surround children with more educated adult role models, stronger educational values, and better community resources. The children would benefit from higher-quality schools and the peer influences of high-achieving classmates. We would be sure to see improvement in their academic performance. Right? Maybe. Research has in fact found surprisingly little convincing evidence that neighborhoods play a key role in children's educational success.

To assess the influence of neighborhoods on children's education, we need to compare large numbers of children in similar families who reside in different communities. Unfortunately, the task is not as simple as finding families who already live in higher- and lower-poverty neighborhoods. Even if these families share characteristics that we can measure, such as income and parental education levels, there may be differences we cannot measure that both led these otherwise similar families to reside in their respective neighborhoods and influence the academic performance of their children. This possibility leaves us uncertain about how much neighborhoods, as compared to family influences, matter for children's academic performance.

What we really want is to be able to identify low-income families in high-poverty neighborhoods who would all prefer to live in better areas. We need some of these families to relocate to lower-poverty communities, but which families move and which remain must be decided by a process outside their control. If we see differences in academic performance between the children in the relocated families and the children who remained in the higher-poverty neighborhood, we can then be reasonably certain that these changes are due to the new neighborhoods, and by extension the new schools, and not family characteristics.

In 1994, the U.S. Department of Housing and Urban Development (HUD) began a pilot program that provided researchers with just this type of experimental data. From 1994 to 1997, HUD carried out a lottery that awarded housing vouchers to families in five U.S. cities: Boston, Baltimore (see "All Over the Map," features, page 28), Chicago, Los Angeles, and New York. The Moving to Opportunity (MTO) Fair Housing demonstration program was designed to move poor families with children out of public housing developments located in census tracts with 1990 poverty rates of at least 40 percent and into the private housing market of more affluent neighborhoods.

Families had to take the initiative to enter the housing voucher lottery, thus indicating their preference for moving. The lottery process was used to randomly assign each family to one of three groups. The first group received housing vouchers that could be used to rent in the private market in any neighborhood. These vouchers were part of the preexisting Section 8 (now called "Housing Choice Voucher") program, which awards housing vouchers to low-income families. (Families residing in public housing or using vouchers to rent apartments in the private market are generally required to pay 30 percent of their income in rent.) The second group also received vouchers, but, new to the MTO program, the use of the vouchers was restricted in the first year to low-poverty neighborhoods, that is, a neighborhood in a census tract with a poverty rate of less than 10 percent, based on 1990 census data. The third group did not receive either voucher, creating a control group of families against which those in the other two groups could be compared. During the first four years of the program, a total of 4,248 families entered the lottery. Some 1,209 were offered the Section 8 unrestricted vouchers; 1,729 families were offered the restricted vouchers; 1,310 received no voucher. Forty-seven percent of the families who were offered restricted vouchers and 59 percent of the families who were offered unrestricted Section 8 vouchers "leased-up" new apartments under the terms of the program. In our analysis, families who were offered vouchers but did not use them remained part of the experiment in order to maintain the randomly assigned groups. However, our analysis takes account of the fact that not all families who were offered a voucher used it.

We set out to evaluate the impact of the MTO program on neighborhoods, schools, and educational outcomes across all five cities and on the full age range of children affected by the MTO program. We expected that in 2002, the year of our study, children whose families were offered vouchers of either type would, on average, live in lower-poverty neighborhoods, attend higher-performing schools, and would, as a result, have improved educational outcomes. Our approach was to compare the educational outcomes of children whose families were offered housing vouchers to those of children in families who entered the lottery but were not offered vouchers. Using data on more than 5,000 children between the ages of 6 and 20, we examined mid-term outcomes four to seven years after a family entered the lottery.

We found that families offered housing vouchers in the MTO demonstration program on average moved to communities with substantially fewer poor and substantially more educated neighbors, but with only marginally fewer minority neighbors. Despite these neighborhood improvements, the families sent their children to schools where students were performing at only modestly higher levels on state exams than in the schools attended by children in the comparison group. Nor did we find evidence of any improvement in reading scores, math scores, behavior, or school engagement. At least for the children in the Moving to Opportunity experiment, the promise that better neighborhoods would bring greater academic achievement has thus far gone unfilled.

Data

In 2002, in collaboration with HUD and Abt Associates, we were able to achieve an effective response rate of 90 percent in obtaining information on families participating in the program four to seven years after they entered the lottery. We collected data on one adult and up to two children from each family in our study. We interviewed parents and we also interviewed children age 8 or older. In the interviews, parents were asked about their children's academic performance and behavior. Children were asked about their neighborhoods, schools, academic performance, and behavior. Interviewers administered a battery of achievement tests to all children.

Our sample of over 5,000 children is divided roughly equally between girls and boys. The average child is just over 13 years old. The sample consists mainly of minority children; approximately two-thirds are non-Hispanic African Americans and about 30 percent are Hispanic (black or non-black). The majority of the children are from female-headed households.

We identified changes in neighborhoods and schools by constructing address and school histories for each family. We used several sources to compile the address histories, including contacts with the families, the National Change of Address system, and credit reporting bureaus. To determine a neighborhood's level of poverty, we located the street address within the 1990 and 2000 census tracts and used the percentage of residents in the tract who live below the poverty line, as measured by the census data. We interpolated poverty levels for years between 1990 and 2000, and extrapolated post-2000. Our expectation was that neighborhoods have a cumulative impact on children. We incorporated this assumption in our analysis by averaging the characteristics of all the neighborhoods the children lived in between their families' entrance in the lottery and our follow-up in 2002. The longer a child lived in a neighborhood, the more weight we gave that neighborhood's statistics in our calculation.

Interviewers constructed school histories for each child by asking the adult in the family for the names and grades of all the schools the child had attended since the lottery. We were able to use the names and addresses of schools to retrieve information about school enrollment and type from the National Center for Education Statistics' Common Core of Data and Private School surveys. State education departments and the National Longitudinal School-Level State Assessment Score Database provided us with average test scores at the schools.

Our primary measures of each student's academic performance were the reading and math scores of MTO children from the Woodcock-Johnson-Revised (WJ-R) battery of tests, which the interviewers administered. The WJ-R can be used across a wide range of ages, produces scores that are highly correlated with other commonly used achievement tests, and has been standardized on a nationally representative sample.

We obtained additional information about schools and the students from interviews with the adults and children. We asked children about their school's climate, including its safety and the level of disruption by other students. We assessed how engaged the children were with learning by asking them directly about how hard they worked at school, how often they were tardy, and how many hours they spent reading each day.

Interviewers asked the adults whether their children exhibited various behavior problems. Specifically, we measured a child's degree of behavior problems as the fraction of eleven problems, including "is disobedient at school" and "has difficulty concentrating, cannot pay attention for long," that the adult reported as "sometimes" or "often" true for the child. We also used the survey to gather information from the adults about other educational outcomes, such as whether the child had been held back a grade, been suspended, or taken any remedial or advanced classes.

Measuring the Impact of the MTO Program

In a randomized experiment, the baseline characteristics of the treatment and control groups should, on average, be the same. In the case of the MTO program, the random assignment of housing vouchers by lottery does appear to have created experimental and control groups that are similar on characteristics that we can observe, including age, gender, race, frequency of learning and behavior problems, the employment status and education of parents, and the reported reasons for wanting to move, including the desire for better schools. This experimental setup should allow us to attribute any subsequent differences between the groups as the product of the offer of a voucher. To increase the precision of our estimates of the effects of being offered a voucher, we controlled statistically for the influence of other characteristics of the child and parent on educational outcomes.

After estimating the average effect of being offered a voucher by comparing the entire treatment and control groups, we used information on how many families actually moved to gauge the impact on those who relocated to a new neighborhood. Assuming that the families who were offered a voucher were not affected if they did not use the voucher, the size of the impact is essentially the estimated impact of being offered a voucher divided by the fraction of families who moved. Recall that 47 percent of families who were offered restricted vouchers moved to lower-poverty neighborhoods. This means that the impact of actually using a restricted voucher is approximately twice as large as the impact of being offered a restricted voucher.

New Neighborhoods

The MTO program had huge success in moving families out of public housing and into lower-poverty neighborhoods. The offer of a restricted housing voucher, for use in a tract with a poverty rate less than 10 percent, led to children living in neighborhoods with poverty rates averaging 12.6 percentage points lower than those where families that had not been offered a voucher lived, an average poverty rate of 33 percent for the restricted voucher group compared to an average poverty rate of 45.6 percent for the control group. When we calculate the impact for families who actually used the restricted vouchers, we estimate that families moved into neighborhoods with an average poverty rate 28 percent lower than the neighborhood in which they would have lived if they had not used the vouchers (see opening Figure).

The impact of using a Section 8 unrestricted voucher was also sizable, though smaller than the impact for the restricted voucher group. Children whose families actually used unrestricted vouchers moved to neighborhoods where the average poverty rate was 17 percent lower than the average poverty rate of the neighborhood in which they would have lived.

Whether a family used a restricted or unrestricted housing voucher, children lived in neighborhoods with approximately 5 to 7 percentage points lower male unemployment as well as more college-educated adults and fewer minorities. Restricted voucher families resided in neighborhoods with 13 percentage points more adults with a college degree and 17 percentage points fewer minorities. The changes for families who used unrestricted vouchers were about 30 to 40 percent as large.

What we did not see, however, were families moving en masse to affluent, white suburbs. Even among the families who used restricted vouchers, only 18 percent lived in neighborhoods where the poverty rate was below the state median. Additionally, only 21 percent resided in neighborhoods where more than half of the residents were non-Hispanic whites.

New Schools

The changes in the schools children attended were more modest than the improvements in neighborhoods. Children in families who moved with restricted vouchers attended schools with slightly higher average scores on state exams. The average effect was 9.5 percentiles, bringing the average percentile rank for children whose families used restricted vouchers from the state's 15th percentile to the 24th percentile. Students also attended schools with about 10 percentage points fewer minority students and almost 13 percentage points fewer classmates eligible for the federal lunch program. The use of unrestricted vouchers again produced changes about 30 to 50 percent as large.

The schools attended by students in families using vouchers had higher-performing students, but they also had more-advantaged student bodies. As a result, it is unclear whether these schools were actually more effective in raising student achievement. When we compared their performance to other schools in the same states, taking into account differences in the backgrounds of their students, the advantage for the schools attended by housing voucher students disappeared. We therefore cannot rule out the possibility that the increased school test scores simply reflect differences in the students served and are not indicative of more effective schools.

What did students think of their new schools? We found no differences between students from families offered vouchers and those from the control group, when the children themselves were asked to characterize the climate of their schools. We are unsure how to interpret these results. The lack of positive effects on self-reported measures of school climate could reflect a real lack of improvement, but it could also be due to changes in the frame of reference among students in their new neighborhoods. This certainly must be the case among the youngest children, who never attended schools in their original neighborhoods.

Student Test Scores

When it comes to improving student test scores, the MTO program does not appear to have had an overall impact (see Figure 1). Our statistical estimate puts the impact of moving with a restricted voucher at four-hundredths of a standard deviation increase in combined reading and math test scores, and the estimate is not statistically significant. We also failed to find evidence that suggests an advantage to using an unrestricted Section 8 voucher.

Nor were the impacts of the program more favorable for younger children, who had spent fewer years in high-poverty neighborhoods and, we hypothesized, were more amenable to change. We tested for varying impacts by age in two different ways. First, we divided our sample of children into three roughly equal groups, ages 6-10, 11-14, and 15-20. We then estimated the impacts separately for each age group. We also tested for the possibility that the effect of moving to a lower-poverty neighborhood decreases gradually with the child's age. The estimates of the impact of a voucher on student test scores are statistically insignificant regardless of whether we look within age groups or for a year-by-year correlation.

We did find noteworthy impacts on combined reading and math test scores in two of the five MTO sites. In Baltimore, use of a restricted voucher appears to produce a statistically significant increase in test scores of 25 percent of a standard deviation, an effect that is concentrated among elementary school-aged children. Oddly, in New York, use of an unrestricted voucher is associated with a decline of one-half of a standard deviation. For the three other cities, none of the estimates were large or statistically significant.

We estimated the impact on test scores separately for girls and boys, students with different races and ethnicities, and children grouped by characteristics measured at the time of the lottery that are predictive of low, moderate, and high test scores. Again, we usually failed to find that students from voucher families performed differently than students from the control group. The one exception was the statistically significant positive impact of the use of a voucher on the achievement test scores of non-Hispanic African American children. Among these students, restricted-voucher use is correlated with notable improvements in reading scores, with average scores almost one-fifth of a standard deviation higher than for the control group. The estimated impact of using an unrestricted Section 8 voucher among African American children is not statistically significant and is approximately 0.1 standard deviations. When we analyzed this relationship by MTO site, we discovered that these improvements in reading for African Americans appear only in the Baltimore and Chicago sites. In these two sites, our sample includes virtually no other racial or ethnic groups. Considering the lack of evidence for an impact on African Americans in the other three sites, and the increased likelihood of incorrectly identifying a chance occurrence as a real impact when comparing multiple subgroups within a sample, we encourage readers to interpret these findings with caution.

Behavior and Attitudes toward School

As with test scores, we generally did not find large, statistically significant differences in behavior or attitudes toward school between children from families with and without vouchers. Students from both restricted and unrestricted voucher families and the control group had similar numbers of behavior problems and were equally likely to have repeated a grade, been suspended in the last two years, or had a parent or guardian called to the school to discuss a problem the child was having. Additionally, we could not find differences between the children in their engagement with school, which we measured with the children's self-reported attention in class and tardiness. Nor were children from voucher families any more or less likely to attend remedial or advanced classes.

There are three exceptions to the similarity between the voucher and control groups. First, moving with a restricted voucher appears to be associated with increased behavior problems for 11- to 14-year-olds. Second, we find that moving with a restricted, though not an unrestricted, voucher has an adverse impact on the behavior of boys. Boys from families that used restricted vouchers scored, on average, one unit higher on the behavior problems index. The estimate for girls from the same group suggests that they may have fewer behavior problems than girls in the control group, but the estimate is not statistically significant. Third, for boys and girls with characteristics associated with higher test scores, the family's use of a restricted voucher appears to be associated with more behavior problems.

Why Didn't Student Performance Improve?

The Moving to Opportunity program moved families into lower-poverty neighborhoods and children into schools where students performed at modestly higher levels. So why were there virtually no improvements in children's test scores, incidence of behavior problems, and school engagement, even among children who were preschoolers at the time their families moved?

One plausible explanation lies in the fact that families who used vouchers later took steps that undid some of the potential advantages of their initial moves to middle-class neighborhoods. Children in families that moved using a restricted voucher resided an average of 3.1 years at their new addresses. Subsequent moves, however, tended to be to tracts that were considerably less affluent than the neighborhoods in which they first used the voucher. Moreover, for many families who remained in their new tracts, the poverty rate in their neighborhood increased around them. The census tracts to which MTO voucher families initially moved had substantially higher poverty rates on average in 2000 than they did in 1990. Despite this trend, however, voucher families did reside in considerably more affluent neighborhoods, on average, than control families.

Second, although families with vouchers moved to less-impoverished neighborhoods, most did not move to racially or ethnically integrated neighborhoods. By 2002, three-fifths of families that moved with vouchers were living in neighborhoods with 80 percent or more minority populations. For a variety of reasons, these neighborhoods may not have high-quality schools or other public services usually found in more-affluent neighborhoods where whites make up a greater share of the residents. We can do little more than speculate about this here, however, because we do not have the data to test whether this was the case.

Third, today many urban school systems offer school choice. In fact, over 30 percent of the control-group children in Chicago and Los Angeles were attending magnet schools. School choice may have allowed families that moved with vouchers to continue sending their children to schools in their old neighborhoods.

The final plausible explanation is related to the relative affluence of MTO families' new neighborhoods. Certainly, voucher families moved to neighborhoods that were substantially less poor, and the improvement was very likely greater than any that could be produced by community-based neighborhood improvement programs. Nevertheless, the neighborhood improvements did not involve moving to truly affluent neighborhoods. Previous nonexperimental studies have indicated that neighborhood effects on academic performance are primarily found in comparisons of children in affluent areas to those in middle-class neighborhoods. Lacking random assignment, these studies are not fully convincing. Nonetheless, it may be the case that children from low-income families who moved into high-income suburbs would experience notable improvements in academic achievement.

In the end, we were surprised and disappointed by the inability of the Moving to Opportunity experiment to help poor children succeed in school. After following more than 5,000 MTO children over five years, we can offer some tentative conclusions about policy interventions designed to improve student performance. Moving poor families to neighborhoods that, while less poor, have schools that are only marginally better than those in the original neighborhoods is unlikely to solve the children's academic problems. Interventions might be more profitably focused on factors more directly related to the child, family, and school. The question remains whether these same conclusions will hold after a full decade of living in new neighborhoods, especially for the youngest children who, because of the MTO program, will have spent their entire lives outside of public housing projects. We will be collecting additional data next year and look forward to learning more.

Lisa Sanbonmatsu is a postdoctoral fellow at the National Bureau of Economic Research. Jeffrey R. Kling is senior fellow and deputy director of economic studies at the Brookings Institution. Greg J. Duncan is professor of education and social policy at Northwestern University. Jeanne Brooks-Gunn is professor of child development and education at Columbia University. Details on these findings are available in "Neighborhoods and Academic Achievement: Results from the MTO Experiment," Journal of Human Resources 41:4 (Fall 2006), 649-691.

RELATED ARTICLE: Somewhat Different Neighborhoods, but Similar Schools

Compared to similar families who did not win the "Moving to Opportunity" lottery, families who used a restricted housing voucher found neighborhoods with lower poverty rates and a more educated population, but the characteristics of the schools the children attended changed only marginally.

Neighborhood Characteristics

                                                 Percentage
                                     Moved with
                                     restricted voucher  Control Group

Poverty Rate                         19                  47
Adult Male Unemployment Rate         12                  19
Share Minority                       75                  91
Share of Adults with College Degree  25                  12

Note: Table made from bar graph.

School Characteristics

                                                   Percentage
                                       Moved with
                                       restricted voucher  Control Group

Share Eligible for Free Lunch          63                  76
Share Minority                         82                  92
Average Test Performance (percentile)  24                  15

Note: Table made from bar graph.

Note: Neighborhood characteristics for the appropriate control group were calculated by subtracting the estimated effect of using a restricted voucher from the average neighborhood characteristics of families using restricted vouchers.

SOURCE: Authors' calculations from U.S. Department of Housing and Urban Development data

No Impact in the Classroom (Figure 1)

Using either restricted or unrestricted Moving to Opportunity vouchers
to move to a new neighborhood had no effect on student reading or math
performance.

                                 Effect of Moving with an MTO
                                 Voucher on Test Scores*
                                 Reading  Math

Move Restricted to Middle-Class  0.03     -0.01
  Communities
Move Unrestricted                0.03     -0.04

Note: Woodcock-Johnson-Revised test data
SOURCE: Authors' calculations from U.S. Department of Housing and Urban
Development

Note: Table made from bar graph.