Choice words: are private schools truly more effective? (Book Review).

Catholic Schools: Private and Social Effects

Kluwer Academic Publishers, 2000, $100; 160 pages

By William Sander

The Education Gap: Vouchers and Urban Schools

Brookings Institution, 2002, $28.95; 275 pages

By William G. Howell and Paul Peterson, with Patrick J. Wolf and David E. Campbell

The advantage of reading The Education Gap and Catholic Schools together is in being able to appreciate their use of diverging research strategies. At the heart of The Education Gap is a large-scale study of privately funded school-voucher systems in three cities, New York, Washington, D.C., and Dayton, Ohio. The authors, Paul Peterson [editor-in-chief] of Education Next] and William Howell, mounted a randomized field trial like those used in medicine in order to rest the effects on achievement of being given a voucher to attend private school, Catholic Schools, by contrast, uses a nonexperimental approach in studying the influence of Catholic schools. The strengths of the experimental approach compensate for the weaknesses of the nonexperimenral approach and vice versa. When both types of studies yield similar conclusions, the results inspire greater confidence.

In The Education Gap, Howell and Peterson call randomized field trials the "gold standard" of social-science research. Randomized trials, built on the model of medical experiments, allow researchers to estimate the effects of a policy change by randomly sorting individuals into two comparable groups--in the case of school vouchers, a test group that receives vouchers and a control group that doesn't. The random allocation ensures that there are no systematic differences--such as income, achievement levels, or parental involvement--between the two groups that might influence the results. Despite their advantages, randomized trials have weaknesses, the most basic being that they do not tell the researcher whether the results may be generalized to other situations, to nonvolunteers, or how the treatment variable will interact with other policy-relevant variables.

Randomized field trials are so expensive that they often must be limited to only a subset of the population of interest. For example, the study might include only a few locations and only a few grades. Even if the outcomes of the treatment and control groups differ significantly, we cannot be sure that the results will generalize to other locations and to other grades. If the experiment was limited to a single income group, we cannot know if the treatment would work equally well for other income groups. Despite these limitations, however, randomized field trials are an extremely useful type of evaluation, and they have an exceptional level of internal validity. We generally can be more confident in conclusions reached by randomized field trials than those produced by nonexperimental research as long as we do not generalize beyond the subgroups studied.

Most social-science research cannot use randomized field trials because of their expense and because it is rarely feasible to assign a sample population to treatment and control groups. Social-science researchers therefore typically observe relationships in nonexperimental settings and attempt to adjust statistically for all the relevant variables, For example, if we are interested in whether private school students and public school students have different outcomes, we try to obtain good measures for all other variables that influence the outcomes and then use regression analysis to discover whether private school students did better than the public school students. In Catholic Schools: Private and Social Effects, William Sander uses this method to study the effects of attending a Catholic school on various academic and nonacademic outcomes.

Nonexperimental research in education has two important methodological problems to overcome; omitted variables and selection bias. Selection bias is a special case of the omitted variable problem. Assume for a moment that two families live next door to each other, The parents have the same occupations, educations, and incomes, and the children have similar abilities. One family pays private school tuition, while the other enrolls its children in public schools. If we observe that the children in the private school learn more, can we conclude that private schools are better? No, because the families obviously have different values and goals. It is likely that the parents paying tuition have other characteristics that encourage their children to value education highly and to work hard in school. Statistical comparisons of public and private school outcomes must find a way to account for these unobserved, but very important, characteristics.

Randomized field trials essentially eliminate the problem of selection bias. At the same time, nonexperimental research can use large sample sizes to test the effects of Catholic schools in a wide variety of situations and to test for interactions between attending a private school and such variables as race, class, religion, and ability level. That these diverging research strategies have common findings lends them serious credibility. The most important common finding was that attending a private school significantly improves the education outcomes of African-American children in the inner cities but not of other students.

A second common finding is that instruction at a religious school has a positive effect on the religiosity of students. A major reason that parents choose to send their children to religious schools is to obtain religious instruction. The Education Gap finds that such instruction increases the religious observance of students. Catholic Schools shows that this increase lasts into adulthood and affects religious behaviors such as the frequency of prayer and the contributions one makes to the church.

Because all of the randomized experiments occurred in urban areas, a question that could not be answered by the Howell and Peterson study is, "Do private schools make a difference in rural areas?" Catholic Schools tells us that they do not. Similarly, because the experiments in The Education Gap included only low-income students, we do not know if children from higher income families who attend private schools learn more than higher income students who attend public schools. Again, Catholic Schools says no.

The Education Gap provides important information concerning the effects of vouchers on political tolerance. Opponents of expanded school choice fear that allowing families to choose private schools, particularly fundamentalist schools, will encourage intolerance among children. Although previous nonexperimental research has found that students in private schools showed greater tolerance and concern for the less fortunate than did public school students, these results may have been caused by selection bias. That the randomized field trials found similar relationships gives us much greater confidence that private schools do not promote intolerance.

Given the weight that the black-white test-score gap carries in the American psyche, the advantage that Catholic schools give black children has substantial policy implications. The most immediate of these is that providing vouchers to low-income children in the inner cities appears to be a cost-effective tool for improving their education outcomes. Such a policy would increase educational equity across racial and ethnic groups. A second implication is that voucher policies that include sectarian schools will not reduce political tolerance in the United States and may increase feelings of civic duty and political participation. This should alleviate the fears expressed by such prominent philosophers of education as Amy Gutmann of Princeton and Eamonn Callan of Stanford. It doesn't appear that vouchers would reduce the quality of democracy in the United States.

-R. Kenneth Godwin is a professor of political science at the University of North Carolina at Charlotte and coauthor, with Frank R. Kemerer, of School Choice Tradeoffs: Liberty, Equity, and Diversity (2002).