The great unknown: does the black-white test-score gap narrow or widen through the school years? It depends on how you measure. (Check the Facts).

Educational Achievement and Black-White Inequality 

By Jonathan Jacobsen, Cara Olsen, Jennifer Kinq Rice, Stephen Sweetland, and John Ralph

National Center for Education Statistics, July 2001.

Through the 1960s, African-Americans earned much less than whites--even when their cognitive abilities (as measured by test scores) were similar. By the end of the century, however, many believed that employment discrimination had attenuated to such a degree that the gap in labor-market outcomes could be explained almost entirely by differences in test scores. Consequently, reducing the well-known gap between the test scores of black and white students is now seen as an important way to reduce economic and other forms of inequality. In response, the U.S. Department of Education's National Center for Education Statistics commissioned Mathemarica Policy Research, a contract research firm, to find out whether 1) employers are now playing fair and 2) schools are doing their part in narrowing the black-white testscore gap. Accepting the challenge, Jonathan Jacobsen and his colleagues discovered that the answer to the first of these questions was both more hopeful and much easier to provide.

Few dispute the dreadful state of affairs that existed in the 1960s. One study found that the black-white gap in scores on the Armed Forces Qualifying Test in 1964 could account for only a quarter of the difference in wages between black men and white men. Quite clearly, widespread racial discrimination helped create a labor market that yielded fewer rewards for blacks than for whites of similar ability. In addition, many jobs did not require a high level of cognitive skill.

Today, cognitive skills and educational credentials are more valuable for workers in part because of changes in production technologies that demand more highly skilled employees. Moreover, the civil-rights legislation of the 1960s and 1970s seems to have caused discrimination in employment to decline. As a result, the black-white gap in academic achievement now seems to account for a sizable share of the black-white gap in wages.

Whether the gap in labor-market outcomes has virtually disappeared depends on how it is measured. In a widely cited 1996 study, economists Derek Neal and William Johnson showed that if test scores are not taken into account, white men's wages are 24 to 28 percent higher than those of black men. But these raw differences narrow substantially when differences in test scores are accounted for. Neal and Johnson, using data from the National Longitudinal Survey of Youth begun in 1979, showed that white men earned hourly wages that are only 7 to 10 percent more than black men with similar scores on the Armed Forces Qualifying Test. However, the gap in annual earnings between white men and black men with similar scores was about three times as large (roughly 30 percent), This is because white men are less likely to be unemployed, work longer hours on average, and are on the job more days of the year. Among women, wages and earnings were actually somewhat higher for blacks than for whites with similar test scores.

Confirmation

Using data from a variety of sources, including the National Longitudinal Survey of Youth, the High School and Beyond study, and the National Longitudinal Study of the High School Class of 1972, Jacobsen and his colleagues at Mathematica essentially confirm Neal and Johnson's findings, providing additional evidence that most of the remaining wage gap is due to differences in cognitive skills, as measured by test scores.

In the Jacobsen study, the hourly wages, employment rates, and annual earnings of black women were at least as high as those of white women with similar test scores and family backgrounds. The researchers also found that black and white men with similar scores and family backgrounds had similar wages and, contrary to the findings of Neal and Johnson, generally experienced similar employment rates and annual earnings as well.

The minor discrepancies between the findings of Jacobsen et al. and those reported by Neal and Johnson for male earnings and employment are presumably due to some combination of differences in sample definitions and outcome measures. For example, the Jacobsen study focused on respondents' earnings over a one-year period and excluded everyone who reported no earnings during that year. Neal and Johnson, by contrast, included in their sample all respondents who reported positive annual earnings for at least one year during the three years covered by their survey questions. Their strategy tends to include workers with more tenuous ties to the labor market than that used by Jacobsen et al, If blacks are likely to be less attached to the labor market than whites, it should come as no surprise that Neal and Johnson would estimate a larger black-white gap in earnings than that found by the Jacobsen report.

In any case, the findings of Jacobsen and his colleagues lead to the same bottom line as previous research. Without a doubt, racial discrimination remains a problem to some degree in labor markets and other aspects of American society. Nevertheless, public policies that reduce differences in skills between blacks and whites before they enter the workforce could substantially reduce racial differences in labor-market outcomes.

Closing the Gaps?

Given this finding, the second question posed to the Jacobsen team is all the more interesting: Are schools gradually eliminating the test-score gap? Do black students and white students continue on in school at the same rate and become more similar in their test scores as they pass from one grade to the next?

One way to improve the skills of black students is to encourage them to stay in school for a longer period of time. In this respect, the Jacobsen report provides some heartening news. Jacobsen and his colleagues found that blacks are at least as likely as whites with similar test scores and family backgrounds to graduate from high school (or earn a GED) and attend college. Among college attendees, blacks are at least as likely to finish their degree program as whites with similar test scores and family characteristics. These findings confirm those from a variety of previous studies, which have shown that adjusting the data for students' socioeconomic status is usually enough to eliminate most or all of the difference between blacks and whites in educational attainment.

This arguably provides grounds for optimism for a number of reasons. First, ethnographic studies have suggested that black students are less inclined to try hard in school because their peers may view academic effort as characteristic of "acting white:' Measures of persistence--how far students go in school--will of course miss the more subtle forms of academic effort, Nevertheless, the fact that blacks stay in school at least as long as whites from similar socioeconomic backgrounds seems inconsistent with the idea that the youth culture of blacks is much more anti-academic than that of whites.

Second, whatever problems colleges have in enrolling and educating students from disadvantaged backgrounds appear to be driven by issues of class more than by race. But what to do about the persistent effects of social class on educational attainment? If being a child in a low-income family mattered primarily because teens were forced to work or could not afford college, the government could reduce the black-white gap in educational attainment through more-generous tuition or income-assistance programs. However, research by economists Stephen Cameron and James Heckman suggests that the real problem may be that poor children simply don't do as well in school, a somewhat discouraging conclusion since sending families bigger checks to help with college is arguably easier than improving academic achievement in elementary and secondary schools.

Narrowing or Widening?

Staying in school is no longer enough, however. Students need to acquire cognitive skills along the way. How well are schools--and other forces outside of school--doing at reducing the blackwhite test-score gap as young people move from one grade to the next? At this point Jacobsen et al. are hard-pressed to find a convincing answer--and the one they do provide seems to depend on a number of analytical choices they made, some of which are debatable.

The best way to answer this question would be to follow a national sample of students through school from kindergarten through 12th grade. Surprisingly, the U.S. Department of Education has never done this. Instead, one cohort of students has been followed for a few years in elementary school, another cohort has been followed for a few years in middle school, and a third cohort has been followed in high school. Jacobsen and company are thus handicapped from the beginning in their search for an answer.

In an attempt to solve this problem, the Jacobsen team cobbled together data from the federal longitudinal "Prospects" study's Cohort 1 to measure children's test scores in 1st and 2nd grades (1992-93); Prospects Cohort 3 for scores in 3rd and 5th grades (1991-93); Prospects Cohort 7 for 7th and 9th grades (1991-93); and the National Education Longitudinal Study to measure 10th- and 12th-grade scores (1990-92). The available data enabled the researchers to derive measures of changes in the black-white gap for only a few years for each cohort's educational career. As a result, measuring changes in the gap over a longer period of time requires some comparison across cohorts,

The Jacobsen report, while hedging its bets, leaves the reader with the impression that the black-white test-score gap closes as students move through school. However, the evidence for the closing of the gap comes from problematic cross-cohort comparisons, not from the tracking of a specific cohort of students from one grade to the next (a"within.cohort" comparison).

Studying the more reliable within-cohort trends elicits less reason for optimism. In reading scores, two of the four within-cohort comparisons reveal a widening of the black-white gap as children move through school: between the 1st and 2nd grades, the gap grew one-third of a standard deviation larger, and another one-fifth of a standard deviation larger between grades 3 and 5. (There was no statistically significant change in the reading gap between grades 7 and 9 or grades 10 and 12.) In math, the within-cohort comparisons show the gap growing by two-fifths of a standard deviation between grades 7 and 9. (There was no statistically significant change in the math gap between grades 1 and 2, 3 and 5, or 10 and 12.)

Another way to assess the performance of the education system with respect to racial disparities in achievement is to compare the subsequent performance of black and white students with similar initial scores. This approach is analogous to comparing the wages of blacks and whites with similar test scores, as discussed above. Again using the more reliable within-cohort comparisons, Jacobsen and his colleagues found that in both math and reading a black-white gap was virtually always present, even for students whose scores were similar just one or two years earlier.

Only in the across-cohort comparisons did the Mathematica researchers find much evidence to suggest that the black-white test-score gap may narrow during the school years. These comparisons seem to suggest that the gap in math scores narrows through elementary school, with some (but not all) of this gain lost during the middle-school years. In reading, the across-cohort comparisons suggest a narrowing of the gap through 7th grade, after which the gap remains stable.

The across- and within-cohort comparisons thus yield different stories about whether the reading gap narrows or grows as children progress through school. Which trend should be believed? An obvious problem with the acrosscohort comparisons, Jacobsen and his coauthors note, is that each cohort of students may exhibit different characteristics, making cross-cohort comparisons difficult. In other words, this year's 3rd graders are not so easily compared with next year's 3rd graders; in one, cartoons like Pokemon may have been all the rage, while reading Harry Potter books might have caught on the next year. Unfortunately, the degree to which the report's findings can be explained by these kinds of across-cohort differences is not easy to assess. Data from the National Assessment of Educational Progress suggest a fairly complicated pattern of changes in the black-white gap across cohorts that depends on the subject area and the age at which the tests are administered,

Further complicating matters is the authors' choice of how to measure the black-white gap in each grade, which seems to drive much of what they find. To wit, the black-white gap in each grade is expressed as a fraction of the standard deviation in test scores observed in the 8th grade. Setting aside the question of whether 8th-grade tests can be compared with the lower-grade tests in the Prospects study and the higher-grade tests in the National Education Longitudinal Survey, a more basic problem with this measure has to do with the statistical properties of test scores.

Say that a large population of black and white 5th graders is given a standardized test. Each group's scores will form a distribution with a certain average and a standard deviation--a measure of the scores' range. Since white students score higher than black students on average, let's say that the average white score is 100, while the average score for black students is 80. Each distribution has a standard deviation of 20 points.

Now say the same group of students is tested again in 8th grade, where the average score of black students turns out to be 90, versus an average of 100 among white students. The difference in average scores is now 10 instead of 20. The test-score gap appears to have narrowed. However, imagine that the standard deviation of these distributions has also been reduced to 10 points, meaning that there is less variability than in the scores of 5th graders.

Has the black-white gap narrowed from 5th grade to 8th grade? The approach adopted by Jacobsen would suggest that it has: measured as a share of the 8th-grade standard deviation, the black-white gap has declined from two standard deviations to one. But this is not the most informative approach to measuring the gap. Notice in my example that the average score of black students lies at the same point in the white distribution in both the 5th and the 8th grades: around 75 percent of white students score higher than the average black student in both grades. If we care about how much the two distributions overlap each other, we should calculate what is known as an "age-standardized" measure of the black-white gap, defined as the difference in the black and white average scores in a given grade divided by the standard deviation in scores in that grade Applying this approach to my example, one would conclude that the black-white gap has not changed across grades.

When University of California at Los Angeles professor Meredith Phillips and her colleagues compared age-standardized measures of the black-white gap across grades, they found that the math gap widened by around one-fifth of a standard deviation between the 1st and the 12th grades, while the reading gap did not change much. Unfortunately, given the limitations of the available data, Phillips also relied on across-cohort comparisons that are vulnerable to the problems mentioned above. Given this currently unavoidable problem, I remain agnostic on whether the black-white gap changes very much one way or the other as children progress through school.

What Would Be Learned?

Suppose that enlightened policymakers eventually fund the type of longitudinal study that would enable the tracking of changes in the black-white test-score gap from 1st grade to 12th grade for a single cohort of students--precisely the type of study Jacobsen and his colleagues call for. It is still not clear what would be gained from such an exercise.

One motivation for studying the change in the black-white gap across grades has been to test the hypothesis that the difference is primarily genetic in origin, since evidence that the black-white gap changes in magnitude over time would suggest a role for environmental influences. But evidence for substantial environmental effects on test scores has been available for some time, including evidence of large IQ gains over time within a number of countries (the so-called Flynn effect, named for the psychologist James Flynn). Moreover, genetic influences need not have the same effect on test scores throughout the lifecycle, since any genetic effects may be amplified over time if they also affect the environmental influences that people seek out for themselves. While there are several compelling points against the genetic argument, variation in the size of the black-white test-score gap across grades is not high on the list.

Another motivation for tracing these changes is to see whether the nation's education system is contributing to equal opportunity. If no reduction in the gap can be detected, one has reason to doubt that schools are having their desired impact. Of course, it may be argued that schools are counteracting socioeconomic factors that would otherwise lead to ever-growing inequality (if, for example, family background becomes increasingly important for scholastic success as children age and homework becomes more important). These data do not help to answer that question. Unfortunately, without information about a single cohort of students throughout their years in school, it is not certain if the black-white test-score gap increases, decreases, or remains roughly the same through the school years. Underlining this fact may be the greatest contribution of the Jacobsen report.

Evidence about how the black-white gap changes as children move through school may help to identify which life stages deserve unusually intense study by policy analysts. However, the specific policy implications of changes in the black-white gap as children age are far from obvious. What no one disputes is that the average black child enters and exits the school system with lower scores than the average white child. This fact alone would seem to leave little doubt that more needs to be done to address this problem.

Jens Ludwig is an associate professor of public policy at Georgetown University.