Teaching and Assessing for Successful Intelligence.

Educators can unearth hidden talents in students by going beyond mastery testing

During a trip to Jamaica, I visited several elementary schools in relatively poor areas of the island. The physical layout of the schools differed drastically from that to which most of us are accustomed. Most were large, one-room schoolhouses. Teachers and students were arrayed around the room, usually with no partitions separating one class from another.

Accordingly, students could listen not only to their own teacher's presentation, but to all the other teachers' lectures as well. If students had the misfortune to sit near the side or back of the class, the students often could hear another teacher better than they could hear their own. Indeed, it was a challenge to discern what their own teacher was saying.

I found myself reflecting on the problem posed to Alfred Binet, the father of intelligence testing, almost a century ago. If one wished to construct tests to predict school achievement, what tests might one construct? I decided that, in the Jamaican setting, two of the most useful test items would measure not skills like vocabulary or arithmetic or spatial visualization--the things Binet measured--but auditory acuity and auditory selective attention.

One needed auditory acuity to hear what the teacher was saying and auditory selective attention to filter out the teachers whose voices competed with one's own teacher. The test probably would predict school achievement well. That's because instruction and even most testing were conducted in the same setting so that one would need the same skills to do well on an auditory ability test in the classroom and on orally administered tests of achievement.

Our Closed System

I believe that schools in other parts of the world, including the United States, echo the Jamaica situation, without being aware of it. In most schools, two abilities are at a premium: memory abilities (those abilities used to memorize, recall and recognize information) and, to a lesser extent, abstract analytical abilities (those abilities used to analyze, judge, evaluate, compare and contrast fairly abstract concepts).

The ability tests measure these skills, learning from instruction in school requires them, and then achievement tests assess the degree to which the abilities have been applied successfully. People who do well on the ability tests tend to do well in school and vice versa because both settings require similar abilities.

The problem is that these abilities are not necessarily the ones that matter most in the life activities for which school is supposed to prepare our children. How many times have you had to memorize a book or a lecture in your job as a school administrator or even during your days as a teacher? Unless you teach math, how many times have you had to remember the theorems you learned in plane geometry? How many times have you seen the extremely obscure words that often occur on vocabulary tests? Probably almost never. But there are other things you have had to do.

Jobs require memory and abstract analytical skills in some degree, but they also require other and arguably more important skills as well. It is no surprise, therefore, that even supporters of current tests who are as traditional as Richard Herrnstein and Charles Murray in The Bell Curve admit that conventional tests of abilities and even of achievement predict only about 10 percent of the variation among people in real-world measures of success. In other words, these tests are poor predictors. What happened to the other 90 percent?

Broad Ability Range

In my book, Successful Intelligence, I argue that intelligence in everyday life requires a broader range of abilities than is measured by conventional tests. The problem with these conventional tests is that they spotlight children who have certain abilities (especially memory and abstract-analytical ones), but leave in the dark children with other abilities, such as creative and practical ones.

Children with other abilities may be derailed from the fast track early in life, with the result that they never get the opportunity to show what they really can do. Not only do we disenfranchise these children, but we provide almost limitless opportunities for those individuals who do not necessarily have the broader range of abilities they will need to take advantage of the opportunities they receive.

The best school administrators are not necessarily going to be those who got straight A's in their educational administration courses. Such grades might not hurt. But the practical skills required to handle subordinates firmly but fairly, to negotiate with administrators at higher and lower levels, to deal with parents, to cope with union demands and to maintain a life outside the school are probably going to be a lot more important than the memory and abstract analytical skills that led to A's in courses.

Similarly, the creative skills required to structure or restructure the school, to create favorable working conditions, to deal with unexpected problems and to instill a sense of vitality in your school are also ones not likely to have been rewarded in most classrooms.

The problem with our present curricula is that these same skills needed to succeed in school administration also are needed to succeed in other jobs. Scientists may need abstract-thinking skills, but without the creative skills to generate new ideas and the practical skills to gain acceptance of their often unconventional ideas, they are lost. Artists need creative skills to do their work and practical skills to get it displayed and accepted. Teachers need the creative skills to bring a fresh approach to their teaching and the practical skills to make meaningful contact with students, parents and administrators alike.

Ironically, the skills we value most of all in the conventional school curriculum seem to be those that often matter least in life.

Measuring Various Abilities

To remedy this situation, I have worked with colleagues at Yale to restructure the processes of ability testing, instruction and assessment of achievement. In collaboration with Michel Ferrari, Pamela Clinkenbeard and Elena Grigorenko, I started by developing a group-administered research version of what I refer to as the Steinberg Triarchic Abilities Test. There are two levels currently available at cost for research purposes--one for children at the high school level (roughly ages 15-18) and one for children at the intermediate, 4th-grade level (roughly ages 9-10). The test measures the conventional abilities, but other abilities as well.

One-third of the test measures the memory and analytical abilities evaluated by conventional tests of intelligence and scholastic abilities. Another third of the test measures abilities more germane to creative thinking and coping with novelty--the ability to think in novel ways. And the last third of the test measures practical abilities needed to adapt to everyday life.

The three abilities--(1) memory-analytical, (2) creative and (3) practical--are each measured in four different ways via (1) verbal, (2) quantitative and (3) figural multiple-choice items and via (4) essays. The multiple-choice items are objectively scored, the essay items subjectively scored via trained raters (who are taught to focus on the abilities being measured and to ignore irrelevant attributes, such as spelling or punctuation). The goal is to obtain a more nearly complete picture of a child's abilities than would be possible from a conventional test.

The memory-analytical section looks pretty much like a conventional test, requiring students to figure out meanings of words in context (verbal), complete number series (quantitative) and complete figural matrices with a missing term (figural). The high school level analytical essay requires students to analyze the advantages and disadvantages of armed security guards in schools.

The creative section is less conventional, requiring students to solve verbal reasoning problems with counterfactual premises (e.g., what would the solution to this analogy be if money fell from trees?), solve quantitative problems with new mathematical operators (e.g., flux: A flux B = A + B, if A is greater than B; and A - B, if A is less than or equal to B) and complete figural series with unusual mappings. The high school creative essay asks the student the design of an ideal school.

The practical section requires students to solve verbally presented everyday problems faced by typical high school (or elementary school) students, solve practical quantitative problems involving recipes, train schedules or purchase of athletic event tickets and plan routes using (figural) maps. The high school practical essay requires students to describe a problem they have and three practical solutions to it.

The Best Predictors

When we selected high school students for a summer program on the basis of this test, some interesting things happened. Students all around the country took the test. We selected five groups: high analytical, high creative, high practical, high in all three abilities, low in all three abilities.

Our first finding was unexpected: The high-analytical group looked pretty much like a standard high-ability group: mostly white, middle-class and attending strong schools. But the high-creative and high-practical groups were much more diverse in terms of ethnic, socioeconomic and educational background. In other words, we found we had selected more minority students not through any program of affirmative action but through a program of recognizing and valuing abilities that schools typically neglect, both in their instruction and in their assessments.

We also found the test to be reliable and predictively valid. In a study of a summer program at Yale, we found the analytical, creative and practical sections all predicted achievement in a high school psychology course. This course had been taught in different ways to value analytical, creative or practical abilities. So, for example, an analytical task might involve analyzing the strengths and weaknesses of a scientific theory or experiment; a creative task might involve generating a new theory or experiment; and a practical task might involve applying a theory or experiment to one's own life.

In our study, the best predictor was analytical abilities and the poorest, practical. In a replication performed by Deborah Coates at the City University of New York with poor, African-American children, the pattern of prediction was reversed, with practical thinking giving the best prediction and analytical thinking the poorest.

We also found that students who were placed in an instructional program that matched their pattern of abilities outperformed those who were mismatched. In other words, if students are taught in a way that at least partially values their strengths, they perform better than if they are taught in standard ways that always value the same abilities, namely, the memory and abstractanalytical ones.

Perhaps you are thinking that it just is not feasible to match instruction to students' patterns of abilities. We anticipated this objection. So together with Bruce Torff and Elena Grigorenko, I designed a study that taught either 3rd-grade social studies or 8th-grade science in one of three ways: in the traditional way, with an analytical (critical-thinking) emphasis or with a three-prong emphasis on creative and practical as well as analytical abilities.

The achievement of all students was assessed via analytical, creative and practical performance assessments but also via standard multiple-choice assessments that emphasized the kinds of memory-learning that are emphasized in most standardized achievement tests and statewide mastery tests. We found the three-prong instruction not only resulted in better performance on the performance assessments, but even on the multiple-choice memory-based assessments. In other words, by letting students learn the material in three different ways, and thereby make the most of their patterns of abilities, students learned better, even when achievement was measured in conventional ways.

Unearthing Hidden Talents

To effect change in education, we need not only change the ability tests but the instruction and the tests of achievement as well. When we emphasize all three abilities--analytical, creative and practical--rather than just one, we will find that many students who now seem rather inept actually have abilities that, under traditional systems of testing and instruction, remain hidden and ultimately go to waste.

We can make the change with relatively little effort because teachers already know how to teach analytically, creatively and practically. Often, they are afraid to do so lest their students perform poorly on mastery or other conventional tests. Our results show that the students actually will perform better on all tests when given a chance to learn in a way that best allows them to bring their strengths to bear on their classroom learning.