PCB-induced impairments in older adults: Schantz et al.'s response. (Correspondence).

In his letter critiquing our study of neuropsychologic functioning in older adults exposed to PCBs and DDE (1), Kaufman states that there were some serious flaws in the research design and data analysis. Specifically, he calls into question one of the test instruments we used to assess memory, argues that we did not adequately control for IQ in our analyses, and raises several statistical issues including failure to adequately control for multiple comparisons and failure to consider the implications of the interdependency of two key variables.

With regard to the issue of multiple comparisons, Kaufman charges that we conducted at least 48 and possibly as many as 96 separate multiple regression analyses without correcting for multiple comparisons. He also implies that we may have conducted additional analyses that were not reported in the published paper. Kaufman's calculations assume that each of the four exposure variables we assessed was considered independently. That is not accurate. The four exposure measures (PCBs, DDE, lead, and mercury) were included in the same regression model, which examined PCBs and DDE as the major independent variables and lead and mercury levels as covariates. We freely acknowledged in the paper, and we reiterate here, that we did perform multiple comparisons to look at a number of different cognitive outcomes. However, as shown in Table 4 of our paper (1), the total number of comparisons was 24, not 48 or 96 or some unspecified number beyond that. We were acutely aware that the use of multiple statistical tests to assess multiple cognitive outcomes raised the possibility that one or more of the associations we encountered could be spurious, and we discussed that issue in detail in our paper. As Kaufman acknowledges in his letter, we urged caution in interpreting our results because of the multiple comparisons.

The issue of correcting for multiple comparisons is not as straightforward as Kaufman apparently believes. In recent years this topic has been the subject of spirited debate, and a number of prominent epidemiologists have argued that adjustments are unnecessary (2-4). Also, in his zeal to make his point about Type I error, Kaufman fails to consider the other side of the coin: Type II error--concluding that there is no effect when one does, in fact, exist. [n addressing important public health issues, we believe that Type II error is a serious concern and should not be ignored.

It is important to point out that our study was hypothesis-driven research. We designed our neuropsychologic test battery based on previous research on the neurologic effects of PCBs, and we selected a specific subset of outcome variables from the tests we administered a priori on the basis of our hypotheses about which aspects of neuropsychologic function were likely to be affected. These were the only analyses we performed, and they are all reported in our published papers.

Kaufman also charges that because two key outcome variables (List A, Trial 1, and the semantic cluster ratio), both measures from the California Verbal Learning Test (CVLT), are "experimentally interdependent," two of our significant results may be redundant. The fallacy of this argument becomes apparent if we consider the correlations between several CVLT measures. The Spearman correlation between List A, Trial 1, and semantic cluster ratio in our sample was a modest 0.376, yet both of these outcomes were significantly associated with log PCB in our models. In contrast, the Spearman correlation between List A, Trial 1, and List A, Trials 1-5, was much greater (0.752), but one of these measures was significantly associated with log PCB and the other was not. This illustrates that we cannot assume that two correlated outcomes will both be significant (or nonsignificant). Finally, although we do not feel it is necessarily the best approach to control for other outcome variables in the regression model, we repeated the regression analysis and found that the association between semantic cluster ratio and log PCB remained significant even when we controlled for List A, Trial 1, in the analysis.

According to Kaufman, another flaw in our research design was the use of an outdated version of the Wechsler Memory Scale (WMS), when a newer, more reliable version, the Wechsler Memory Scale-Revised (WMS-R) was available. He cites one paper (5), which gave the WMS-R "exceptional reviews." We based our decision not to use the WMS-R on a careful review of all of the literature that was available at the time. Although favorably reviewed by Powel (5) and Holden (6), the WMS-R has been sharply criticized by a number of others. Loring (7) pointed out that in the development of the WMS-R, advances in cognitive, experimental, and clinical psychology over the decades since the introduction of the original WMS (8) were largely ignored. In addition, the WMS-R has been faulted on basic psychometric properties including small sample size and poor subtest reliability (9) as well as interpolation of scores used as norms for one-third of the population (10,11). Finally, two separate reviews in the Mental Measurements Yearbook (12,13) were both highly critical of the WMS-R. Although we do not feel that Kaufman's concerns about the WMS are valid, we would like to point out the fallacy of his argument. He implicitly assumes that use of an "outdated" measure with "poor psychometric properties" would be more likely to lead to a false positive than a false negative association, but he fails to provide a rationale for this assumption. We argue that an unreliable test would be more likely to attenuate correlations than to result in spurious associations.

Kaufman is also critical of our choice of the WAIS-R vocabulary subtest as a measure to control for general intelligence. He acknowledges that WAIS-R vocabulary is a "reliable, stable and good" measure of general intelligence--precisely the reasons we selected it--but he goes on to argue that vocabulary taps primarily "crystallized intelligence" and does not adequately control for "fluid intelligence," which he considers to be particularly important in older adults. Kaufman raises an interesting point, but in reality this issue is not as simple as he makes it seem. As he himself acknowledges, fluid intelligence is vulnerable to brain damage, so using it as a control for general intelligence ill the presence of exposures that have the potential to damage the brain is of questionable utility. Furthermore, although it is possible that including the Raven Matrices (or alternatively the entire WAIS-R) would have provided better overall control of general intelligence, this would have added significantly to the time required to administer the test battery. The subjects in our study were aging adults who were evaluated for approximately 3-4 hr in their homes. Pilot testing prior to the study indicated that people in this age group were not receptive to a longer testing battery and that fatigue became a factor in test performance if the visit was extended any longer. The homes of the study participants were located 90-200 miles from the research office; thus we did not have the resources for more than one visit per subject. In the selection of dependent variables, we considered the hypotheses to be tested, the instruments available to us at the time, and the amount of time we realistically had available to do the assessments. No single research study will be definitive in every conceivable respect. As Needleman and Bellinger (4) aptly pointed out when Kaufman leveled similar criticisms regarding their lead studies, "... complete control of all confounders is an unattainable goal in real-world epidemiology" (p. 363).

We thank Kaufman for his thorough critique of our manuscript and the editors of EHP for giving us the chance to address the misconceptions concerning the design of our study and the statistical analyses performed on the data. As we stated in the original article (1): "[Our] study suggests ... that PCB exposure during adulthood may [emphasis added] be associated with impairments in certain aspects of memory and learning" (p. 610) and "... it would be prudent to interpret the findings with caution until they have been replicated in an independent exposure cohort" (p. 610).

Susan L. Schantz
University of Illinois at
Urbana-Champaign
Urbana, Illinois
E-mail: [e-mail]

Donna M. Gasior
Elena Polverejan
Harold E.B. Humphrey
Joseph C. Gardiner
Michigan State University
East Lansing, Michigan

Robert J. McCaffrey
University at Albany
State University of New York
Albany, New York

Anne M. Sweeney
University of Texas Health
Science Center at Houston
Houston, Texas

REFERENCES AND NOTES

(1.) Schantz SL, Gasior DM, Polverejan E, McCaffrey RJ, Sweeney AM, Humphrey HEB, Gardiner JC. Impairments of memory and learning in older adults exposed to polychlorinated biphenyls via consumption of Great Lakes fish. Environ Health Perspect 109:605-611 (2001).

(2.) Poole C. Multiple comparisons? No problem! Epidemiology 2:241-243 (1991).

(3.) Rothman K. No adjustments are needed for multiple comparisons. Epidemiology 1:43-46 (1990).

(4.) Needleman HL, Bellinger D. Studies of lead exposure and the developing nervous system: a reply to Kaufman. Arch Clin Neuropsych 16:359-374 (2001).

(5.) Powel J. Review of the Wechsler Memory Scale-Revised. Arch Clin Neuropsych 3:397-403 (1988).

(6.) Holden R. Wechsler Memory Scale-Revised. In: Test Critiques, Vol. VII (Keyser D, ed). Kansas City, MO:Test Corporation of America, 1988;633-638.

(7.) Loring DW. The Wechsler Memory Scale-Revised, or the Wechsler Memory Scale-Revisited? Clin Neuropsychol 3:59-69 (1989).

(8.) Wechsler D. A standardized memory scale for clinical use. J Psycho119:87-95 (1945).

(9.) Elwood R. The Wechsler Memory Scale-Revised: psychometric characteristics and clinical application. Neuropsychol Rev 2:179-200 (1991).

(10.) D'Elia L, Satz P, Schretlen D. Wechsler Memory Scale: a critical appraisal of the normative studies. J Clin Exp Neuropsychol 11:551-568 (1989).

(11.) Chelune GJ, Borenstein RA, Prifitera A. The Wechsler Memory Scale-Revised: current status and applications. In: Advances in Psychological Assessment, Vol. 7 (Rosen J, McReynolds P, Chelune GJ, eds). New York:Plenum Press, 1990;66-95.

(12.) Huebner EC. Review of the Wechsler Memory Scale-Revised. In: The Eleventh Mental Measurements Yearbook (Kramer JJ, Conoley JC, eds). Lincoln, NE: University of Nebraska Press, 1992;1023-1024.

(13.) Reinehr RC. Review of the Wechsler Memory Scale-Revised. In: The Eleventh Mental Measurements Yearbook (Kramer JJ, Conoley JC, eds). Lincoln, NE: University of Nebraska Press, 1992;1024-1025).