PCB-induced impairments in older adults: critique of Schantz et al.'s methodology and conclusions. (Correspondence).

Shantz (1) provided a valuable scientific service to the field of the developmental neurotoxicity of polychlorinated biphenyls (PCBs) by offering insightful criticisms of the methodologies used by Jacobson et al. (2) and others. Schantz and colleagues proceeded to study PCBs and dichlorodiphenyl dichloroethane (DDE), shifting the focus from effects on infants and children to effects on a cohort of older adults (3-6). The shortcomings of the research design and data analysis used by Shantz et al. are equivalent to the shortcomings of previous studies (1). In their paper published ill 2001 (3), Schantz et al. a) failed to account adequately for the chance significant findings that occur when many statistical analyses are conducted simultaneously; b) used an outdated measure of memory when a much-improved test was available at the time of testing; c) failed to consider the implications of the experimental interdependency of two key variables that were significantly related to PCBs; and d) controlled IQ (intelligence quotient) only with the Wechsler Adult Intelligence Scale-Revised (WAIS-R) vocabulary subtest.

Schantz et al. (3) conducted 48 multiple regression analyses simultaneously, 24 with DDE and 24 with PCBs, spanning several cognitive domains. They used an alpha level of 0.05, which means that one significant finding is expected to occur by chance alone for every 20 analyses; with 48 analyses, 2-3 significant findings will occur by chance. Schantz et al. identified four significant findings, which is barely above the number expected by chance; however, they focused only on three--the ones that produced the anticipated negative correlation-and they virtually ignored the significant, but opposite, relationship between DDE and delayed recall. Of the three negative associations with PCBs, two were experimentally interdependent--List A, Trial 1, and the semantic cluster ratio--both yielded by the California Verbal Learning Test (CVLT). The semantic cluster ratio is based on performance on Trials 1-5 of List A; consequently, the significant relationship between PCBs and Trial 1 of List A contributed to the relationship between PCBs and the semantic cluster ratio; hence, the two CVLT significant results may be redundant.

Furthermore, the other two significant results--the negative relationship with PCBs and the positive relationship with DDE--occurred on the delayed recall portion of the logical memory subtest of the Wechsler Memory Scale (WMS). These results occurred on a 1975 revision of a long-outdated 1945 test (7). Why did Schantz et al. use such an old measure when the revision (WMS-R) was readily available when they collected their data? More to the point, the WMS did not even include a delayed recall component; Russell added that component in 1975, using a weak sample and producing a delayed recall measure with poor psychometric properties (8). In contrast, the 1987 WMS-R, which produces reliable and valid measures of immediate and delayed memory, has been given exceptional reviews (9). It is conceivable that the two significant results that occurred on the WMS Logical Memory test are more a function of the weak measurement of delayed recall than of any real relationship to PCBs or DDE.

In view of the multiple simultaneous comparisons and other points raised here, the best explanation for the significant results by Schantz et al. (3) is chance. The investigators discounted the impact of the many analyses because the negative results were confined to PCBs, as opposed to DDE, mercury, and lead, and because of alleged consistency with previous findings with children. In the "Discussion" (but not in the abstract), they urged caution in interpreting their results because of multiple analyses. However, the researchers did not consider the overlap in the two CVLT scores or the weakness of the WMS. The consistency in research findings is also open to considerable debate (10).

Most of all, Schantz et al. (3) understate the problem of multiple analyses. They stated that mercury and lead, as well as DDE and PCBs, were evaluated as exposure variables, but that all of the significant negative relationships occurred for PCBs. Consequently, they apparently conducted 96 analyses, not 48. In addition, Schantz's team analyzed motor functioning variables for the same cohort of older adults, but published the results in a separate paper (4). They conducted a variety of parametric and nonparametric analyses, although the exact number is not easily discernible; they found no relationship between PCB/DDE exposure and either hand steadiness or visual-motor coordination. Surprisingly, they blended DDE and PCB exposure to get a joint measure of contamination. Though the merger of the two might be defensible, it is not intuitive. Did the authors look at an array of statistical analyses of PCBs and DDE separately before deciding to combine the two for the published paper?

Additionally, Schantz et al. published a paper in 1996 while their analyses were still partly in the planning stage (5). In that paper, the emphasis was on two groups, fish eaters and non-fish eaters, matched on age and sex. The groups were statistically compared on a diversity of potential confounders and were generally found not to differ significantly. One of the purposes of the study was to relate consumption of contaminated fish to decline in cognitive and motor function. A second purpose was to relate serum PCB and serum DDE levels to the degree of behavioral dysfunction. Schantz and colleagues have published papers on serum levels, but the only papers that featured the fish eaters versus non-fish eaters compared the groups by potential confounders, such as alcohol consumption and general intelligence (5), and by PCB congener profiles (6). Why did they not relate fish-eating status to neuropsychologic decline? Their published research has addressed serum levels, but not fish-eating status. Do the few significant results published by Schantz et al. (3) represent the only significant findings obtained by this team of researchers despite the large number of analyses they conducted?

Schantz et al. (3) used the WAIS-R vocabulary subtest as the measure of general intelligence to control for this important confounding variable. Vocabulary is reliable, stable, and a good measure of general intelligence, and provides excellent measurement of what Horn (11) called crystallized intelligence (Gc) (7). Gc reflects knowledge and problem solving that is dependent on formal schooling and acculturation, and is referred to by Horn as a "maintained" ability, one that is maintained across the adult life span and is generally resistant to brain damage (11). In contrast, fluid intelligence (Gf), which refers to novel problem solving that is not dependent on education (such as solving abstract analogies), is a "vulnerable" ability that declines rapidly with increasing age and is vulnerable to brain damage (11). The growth curves for Gc and Gf are so different across the adult age range that there is really not a single general intelligence for adults, but two general intelligences, Gc and Gf.

The Verbal IQ yielded by the WAIS-R or the third edition of the WAIS (the WAIS-III) is roughly equivalent to Gc, whereas the Performance IQ is roughly equivalent to Gf (7). The difference in the aging patterns for these two types of general intelligence are dramatic. In an education-adjusted cross-sectional study conducted with the WAIS-III across the 20-89 year age span (12) using a common adult reference group, Verbal IQ (Gc) averaged about 98 for ages 20-24, peaked at about 104 for ages 45-54, declined gradually to about 98 for ages 80-84, and reached its low point of 96 for ages 85-89. In striking contrast, Performance IQ (Gf) peaked at ages 20-24 (mean = 100), decreasing successively to 92 (ages 45-54), 85 (ages 65-69), 80 (ages 75-79), and 76 (ages 85-89). These data with the WAIS-III are extremely similar to cross-sectional data on the WAIS-R and to longitudinal data with independent samples (7,12).

When controlling for general intelligence for a group of older adults, such as the sample of 49- to 86-year-olds in Schantz et al.'s study (3), it is essential to control for both Gc and Gf in order to rule out the potential confounding of general intelligence. By controlling only for Gc, these investigators did not provide an adequate control for general intelligence. Because the kinds of memory and learning abilities that relate significantly to PCBs are vulnerable abilities whose aging curves more closely resemble the curves for Gf than Gc, it is especially important to rule out the potential confounding of fluid general intelligence. The investigators should have administered a measure of Gf, most notably Raven's Matrices, a fairly pure measure of fluid reasoning that is included in many epidemiologic studies, usually with a vocabulary test, to control for intelligence (13).

The problems of multiple statistical analyses, the experimental interdependence of the two CVLT tasks that related significantly to PCBs, the outdated nature of the WMS and poor qualities of Russell's delayed memory measure, and the inadequate controlling for general intelligence are flaws in the the methodology used by Schantz et al. (3) and challenge the conclusions that they reached. Certainly, the problem of multiple comparisons and poor measurement of adults' general intelligence extends well beyond PCB research, affecting the interpretation of the relationship of lead exposure to IQ loss ill children (13,14). Although there has been a great deal of research on the relationship of lead to IQ in children, Schantz and colleagues are the only team that has studied PCBs in older adults. Because of the flaws in their methodology, the significant findings reported by Schantz et al. (3) are best attributed to chance.

The author has been remunerated by the General Electric Company for consultation about the quality of PCB studies, but he received no remuneration for preparation of any part of this letter.

Alan S. Kaufman
Yale University School of Medicine
New Haven, Connecticut
E-mail: [e-mail]

REFERENCES AND NOTES

(1.) Schantz SL. Developmental neurotoxicity of PCBs in humans: what do we know and where do we go from here? Neurotoxicol Teratol 18:217-227 (1996).

(2.) Jacobson JL, Jacobson SW, Humphrey HEB. Effects of exposure to PCBs and related compounds on growth and activity in children. Neurotoxicol Teratol 12:319-326 (1990).

(3.) Schantz SL, Gasior DM, Polverejan E, McCaffrey RJ, Sweeney AM, Humphrey HEB, Gardiner JC. Impairments of memory and learning in older adults exposed to polychlorinated biphenyls via consumption of Great Lakes Fish. Environ Health Perspect 109:605-611 (2001).

(4.) Schantz SL, Gardiner JC, Gasior DM, Sweeney AM, Humphrey HEB, McCaffrey RJ. Motor function in aging Great Lakes fisheaters. Environ Res 80 (2 Pt 2):S46-S56 (1999).

(5.) Schantz SL, Sweeney AM, Gardiner JC, Humphrey HEB, McCaffrey RJ, Gasior DM, Srikanth KR, Budd ML. Neuropsychological assessment of an aging population of Great Lakes fisheaters. Toxicol Ind Health 12:403-417 (1996).

(6.) Humphrey HEB, Gardiner JC, Pandya JR, Sweeney AM, Gasior DM, McCaffrey RJ, Schantz SL. PCB congener profile in the serum of humans consuming Great Lakes fish. Environ Health Perspect 108:167-172 (2000).

(7.) Kaufman AS. Assessing Adolescent and Adult Intelligence. Boston, MA:Allyn & Bacon, 1990.

(8.) Erickson RC, Scott ML. Clinical memory testing: a review. Psychof Bull 84:1130-1149 (1977).

(9.) Powel J. Review of the Wechsler Memory Scale--Revised. Arch Clin Neuropsychol 3:397-403 (1988).

(10.) Schell JD Jr, Budinsky RA, Wernke MJ. PCBs and neurodevelopmental effects in Michigan children: an evaluation of exposure and dose characterization. Regul Toxicol Pharmaco133:300-312 (2001).

(11.) Horn JL Cognitive diversity: a framework of learning. In: Learning and Individual Differences (Ackerman PL, Sternberg RJ, Glaser R, eds). New York:Freeman, 1989;61-116.

(12.) Kaufman AS. WAIS-III IQs, Horn's theory, and generational changes from young adulthood to old age. Intelligence 29:131-167 (2001).

(13.) Kaufman AS. Do low levels of lead produce IQ loss in children?: a careful examination of the literature, Arch Clin Neuropsychol 16:303-341 (2001).

(14.) Phelps L. Low-level lead exposure: implications for research and practice. Sch Psychol Rev 28:477-492 (1999).