Study abroad: what TIMSS teaches us. (Book Review).

Why Schools Matter: A Cross-National Comparison of Curriculum and Learning

By William H. Schmidt et al.

Jossey-Bass. 2001, $29; 400 pages

The International Association for the Evaluation of Educational Achievement (IEA) began with a chance meeting between Torsten Husen of the University of Stockholm and a group of University of Chicago researchers in the mid-1950s. The scholars at Chicago's Comparative Education Center, C. Arnold Anderson, Mary Jean Bowman, and Ben Bloom, believed that the whole world should be seen as an educational "laboratory." The first results from IEA's international surveys appeared in 1964. Since then, there have been 29 IEA-sponsored cross-national studies. The most famous of these was the 1995 Third International Mathematics and Science Study (TIMSS), the main subject of Why Schools Matter.

Up to 45 nations participated in at least one aspect of this cooperative study of schools, teachers, students, and learning in math and science at three age levels, making it perhaps the largest survey ever undertaken in the social sciences. As in previous surveys, the results suggested that the typical U.S. student knows less math and science than do students in many other industrialized nations. What TIMSS has taught us is that these differences are compounded over time: children in the United States drop lower in the rankings as they age.

One explanation is that curricula in U.S. schools are characterized by too much breadth and not enough depth--in more familiar words, they are "a mile wide and an inch deep." William Schmidt and his colleagues, the authors of Why Schools Matter, are largely responsible for this notion. They found that school systems in the United States cover a broader range of curricular topics than any other country save Switzerland. However, Swiss textbooks are more selective than American texts, making the U.S. curriculum the most "splintered" in the world.

Unlike the national curricula used in Europe and East Asia, curricula in the United States are uncoordinated, with insufficient attention paid to what students learn as they progress from grade to grade. In a highly mobile nation like the United States, this is particularly nettlesome when children change schools and discover that their new school district teaches geometry in 7th grade, while they took it the previous year.

Curricular variation complicates international comparisons as well. The TIMSS test for 13-year-olds included 44 different mathematic concepts. Some of these concepts weren't taught in all nations, while many others were taught at one age level but nor at another or in different sequences. Does this mean that the tests were unfair? No. In fact, even when there were differences in coverage and sequence, these differences turned out to yield important information. It may be the case that children in the Netherlands are asked to move at a quicker pace, with a more logical sequence of topics, and thus learn more mathematics by the time they graduate. This information has proven useful for U.S. policymakers as they search for ways to upgrade the education system.

Students can even vary from nation to nation in their approach to these tests, further complicating comparisons. The international tests are low stakes in the sense that no consequences are attached to a student's performance. However, in some cultures, students may view all testing as a challenge to be taken seriously. In other cultures, students may not care in the slightest how they perform unless it

The Size of Minnesota

What Why Schools Matter fails to say is that countries themselves are often an arbitrary choice as a unit of analysis. For instance, the United States has a range of demographic, cultural, and social-policy differences that make a comparison with Sweden, a country with a population the same size as Minnesota, somewhat artificial. If other factors were used in defining a legitimate unit of analysis--size, wealth, and school functions--states might be the only units meeting the criteria in the United States.

Why Schools Matter also fails to discuss--mostly because it was not included in the TIMSS surveys--how teachers' knowledge and skills differed across countries and what effect that might have on achievement. Nor did the TIMSS study have any data on the amount of money nations invested in education. That TIMSS paid no attention to spending was no accident. Decisions concerning which data would be collected were made by IEA committees in which experts had to reach consensus. The committees themselves were overloaded with specialists on subject matter and methodology. No education economist, and few education sociologists, sat on a TIMSS committee. When the time came to decide on the focus of TIMSS, the subject-matter specialists had a virtual monopoly. Consequently, the measurements of the social contexts in which learning takes place are superficial. Schmidt and his colleagues do take a brief and superficial detour to see if their results change when they adjust the data to account for a nation's per-capita in come (they did not change).

Nonschool factors such as a child's social background seem to make a difference on the TIMSS tests at all levels. The book does not ask, however, whether this difference is the same in all countries or whether it varies in some systematic pattern. Research performed by William Loxley and myself in 1983 found that the influence of a student's socioeconomic status on achievement varies from one nation to another. The lower a nation's gross domestic product (GDP), it appears, the more influence the school seems to have. This finding has been a principal rationale for the investment in school quality by the World Bank, USAID, UNESCO, and many other international development agencies. Although recent re-analyses have challenged the strength of the earlier findings, the fact remains that the influence of socioeconomic status on achievement is by no means uniform across nations, age/grade levels, gender, and subject matter.

Why Schools Matter may be of marginal interest to general readers, but it is an excellent book based on a study of unprecedented sophistication and quality. It tells us that if U.S. schools are to improve, much more attention must be paid to the depth and focus of their curricula.

Stephen P. Heyneman is a professor of international education policy at Vanderbilt University.