Predicting Success in School and at Work.
We are hearing again the rumblings of disaffection over the use of the Scholastic Aptitude Test (SAT) as a basis for college admissions. University of California President Richard Atkinson has called for scrapping altogether the SAT requirement for the state's public undergraduate programs, including Berkeley and UCLA. Elite Mount Holyoke College no longer requires an SAT score from applicants. A survey conducted by FairTest, a group that hopes to de-emphasize the use of standardized tests, finds that about one-fifth of four-year institutions of higher learning do not make the SAT a strict requirement.
One frequent argument against the use of the SAT is that it is not a perfect predictor of academic success. Of course it isn't. Like any measure it yields some error. And personal traits other than raw intelligence can also bear upon scholastic performance. Yet there is no getting around the fact that, of all the information we might have available about an aspiring college student, an SAT score is the single best predictor we have of academic success-and Atkinson, a psychologist, surely knows this.
This is not to say that admission decisions should rest solely on SAT scores. Other kinds of reliable information should be added into the equation to the extent that they add predictive power, net of the SAT. But the established validity of the SAT argues eloquently against the wisdom of eliminating it altogether. Lafayette College, for example, made the test optional for five years, only to find that the task of picking strong prospects became much tougher; the institution recently reinstituted the exam as a requirement for admission.
Another criticism hurled at the SAT is its "unfairness"--an argument that presents us with an irony of sorts. Before World War II, the elite institutions of the land, such as the Ivy League schools and the "Seven Sisters," stood as exclusive enclaves of the wealthy, privileged, and well-connected, virtually without regard to ability. Once the SAT became established as an admission criterion in the postwar years (a prototype of the SAT was used on a limited, more or less experimental basis in the prewar years), the ranks of students in those institutions defined a clear dimension of meritocracy. "In any society," notes George Will, "be it Periclean Athens or Elizabethan England or modern America, the question is not whether elites shall prevail, but which elites shall prevail. So something must perform the predictive function assigned to the SAT." The problem is that no other measure or scrap of information predicts academic success at the college level so well.
Research and practice in the field of human resource management presents us with a parallel issue. The work of industrial psychologists John Hunter and Frank Schmidt has established conclusively that the one best predictor of job performance is a test of general mental ability. Whether we select people for clerical jobs, maintenance work, marketing, accounting, or financial management, a mental ability test is the most accurate means available to forecast success on the job. In other words, regardless of the content of the job, if you had to limit yourself to one single bit of information for personnel selection, you would do best by relying on a test of cognitive ability. Even a 12-minute commercially available test such as the Wonderlic would give you that optimal predictive power.
As with college admissions, however, no one says you have to limit yourself to that one bit of information, only that you should give it substantial weight. Because mental ability test scores do contain some error, and because other personal traits do affect job performance (net of the effect of mental ability), you should add to the test score any other reliable information that provides incremental predictive power.
Schmidt and Hunter (1998) have recently scoured the literature to find out what would be the next best piece of information to add to the mental ability test. They examined research on 18 other factors, including such varied items as integrity tests, graphology, reference checks, biographical data, and interviews. What they found amounted more or less to a three-way tie. Combining either a structured interview, an integrity test, or a work sample test with a mental ability test provided an overall validity of about .60 in predicting job performance. Any one of those three additional items increased the validity of the mental test score alone by about 25 percent. Of the three supplemental items, the researchers recommend the structured interview as the one most generally practicable and useful.
Schmidt and Hunter's conclusion might come as a mild surprise to some. I can remember a time when, among HR professionals, the interview was said to lack validity and lend itself to all manner of distortions and biases. That view probably derived from the fact that most job interviews were unstructured. It now appears that a structured interview--a carefully designed set of questions, ideally drawn from close attention to the job description, and used in a standardized manner and sequence by different interviewers--boasts not only considerable reliability (that is, different interviewers using these same questions arrive at similar assessments) but also substantial validity, net of that provided by the mental ability test. What does the structured interview provide that adds to the information contained in the ability test? Most likely some means of gauging a person's work habits or conscientiousness, as well as a basis for deciding whether the applicant's values fit well with the organization's culture. Thu s, if Person A comes off a lot better in the interview than Person B, who scored only slightly higher on a cognitive ability test, Person A would probably be selected for the job.
Perhaps requiring a structured interview with a college official would be an intolerable expense for institutions of higher learning. If so, maybe a more viable approach would be to use the SAT for identifying some more or less automatic admits and rejects, then add the interview for those "on the bubble." Moreover, colleges could (and in fact do) use loyal alumni in different regions to interview prospective students for their alma mater. Schmidt and Hunter would caution, however, that the interview be in a structured format.
Whether in education or work, the choice between making a decision solely on the basis of a mental ability score versus eliminating the test is a false dichotomy. The predictive power of the test score is just too valuable to toss out. Use it, and add to it whatever combines successfully with it to increase the accuracy of assessments we make about a person's likelihood of success, on the job or in school.
Frank L. Schmidt and John E. Hunter, "The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications," Psychological Bulletin, September 1998, pp. 262-274.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||the use of standardized testing in hiring of employees and admission of college students|
|Date:||May 1, 2001|
|Previous Article:||The Effects of Adopting Cash-Balance Pension Plans.|
|Next Article:||E-Competitive Transformations.|