Statistical issues in farmworker studies.
Barr et al. (2006) surveyed statistical issues related to farmworker exposure studies. However, they made several factual and conceptual errors that need to be called to the readers' attention.
First, Barr et al. (2006) claimed incorrectly that "representativeness" is optional and not a necessary condition for a well-designed investigation. For convenience samples,
[T]he results only pertain to the sample itself, and should not be used to make quantitative statements about any population-including the population from which the sample was selected." [U.S. Environmental Protection Agency (EPA) 2003]
Barr et al. (2006) stated that "because responses from convenience samples are likely to be better than that for a representative sample, they may actually be more 'representative.'" The fallacy of this statement is shown by a hypothetical CNN CNN
or Cable News Network
Subsidiary company of Turner Broadcasting Systems. It was created by Ted Turner in 1980 to present 24-hour live news broadcasts, using satellites to transmit reports from news bureaus around the world. call-in response to a question from 100% of its viewers that perfectly represents all CNN viewers. In this illustrative example, the 100% response would not represent the entire population of the United States United States, officially United States of America, republic (2005 est. pop. 295,734,000), 3,539,227 sq mi (9,166,598 sq km), North America. The United States is the world's third largest country in population and the fourth largest country in area. as well as a probability-based survey of the U.S. population that included non-CNN viewers that achieved an 80% response rate.
In their article, Barr et al. (2006) claimed that "perfectly random sampling across all relevant factors is therefore almost universally impractical." Acquavella et al. (2004) monitored a probability sample of pesticide applicators; U.S. EPA EPA eicosapentaenoic acid.
n.pr See acid, eicosapentaenoic.
n. provided several TEAM (Total Exposure Assessment Methodology) studies using a scientific probability design (Thomas 1993; Wallace 1991; Wallace et al. 1987), as did the World Health Organization, U.S. EPA, and Harvard University Harvard University, mainly at Cambridge, Mass., including Harvard College, the oldest American college. Harvard College
Harvard College, originally for men, was founded in 1636 with a grant from the General Court of the Massachusetts Bay Colony. for the government of Kuwait during the 1991 oil fires (Mage DT, Wallace LA, Kollander M, personal communication). The Centers for Disease Control and Prevention's (CDC See Control Data, century date change and Back Orifice.
CDC - Control Data Corporation ) National Health and Nutrition Examination Survey study (CDC 2003) is another excellent example of proper probability-based sample selection.
According to according to
1. As stated or indicated by; on the authority of: according to historians.
2. In keeping with: according to instructions.
3. Barr et al. (2006), it is possible to identify and "sample known or anticipated 'hot spots' of [pesticide] exposure." There are only two categories of applicators expected to be at high risk of a high pesticide exposure event: the inexperienced applicators who are still learning how to apply pesticides safely, and those applicators who do not follow the mandatory manufacturer's label requirements in violation of federal law (Mage et al. 2002). Whereas the former cohort might be identified by a screening question about prior numbers of applications, there is no certain way to identify the latter group, who will likely not admit to taking shortcuts See Win Shortcuts. or refusing to use required personal protective equipment, because they might be incriminating in·crim·i·nate
tr.v. in·crim·i·nat·ed, in·crim·i·nat·ing, in·crim·i·nates
1. To accuse of a crime or other wrongful act.
2. themselves. Finally, such an applicator ap·pli·ca·tor
An instrument for applying something, such as a medication.
n a device for applying medication; usually a slender rod of glass or wood, used with a pledget of cotton on the end. may succumb to the Hawthorne effect Hawthorne effect Psychology A beneficial effect that health care providers have on workers in most settings when an interest is shown in the workers' well-being. See Halo effect, Placebo effect, Placebo response. Cf Nocebo. [not mentioned by Barr et al. (2006) as a caveat], defined by Last (1988) as "the effect of being under study upon the persons being studied."
Barr et al. (2006) claimed that "some form of convenience sampling is typically adopted in practice." Unfortunately, this claim is true; some of these authors did use convenience sampling in previous studies (Curwin et al. 2002, 2005) in which subjects were recruited by "word of mouth." A friend or neighbor recruited by an enrolled subject might not be "an independent sample" if he or she has some similar characteristics (e.g., crops grown, acreage, age, race, education, sex) as the recruiter. This haphazard practice of using volunteers for convenience, or even subjects based on expert choice (Hoppin et al. 2006), limits the validity of the study, as theoretical confidence intervals and significance p-values become meaningless.
The weakness of all nonprobability sampling is its subjectivity that precludes the development of a theoretical framework for it. (Kalton 1983)
Finally, as former U.S. EPA scientists who pioneered agency exposure science, we are disappointed that this article was cleared for publication by the U.S. EPA because it is not in accordance with U.S. EPA (and other agency) requirements to follow the Office of Management and Budget's (OMB OMB
Office of Management and Budget
Noun 1. OMB - the executive agency that advises the President on the federal budget
Office of Management and Budget ) data collection policies (OMB 2006) that require "selecting samples using generally accepted statistical methods (e.g., probabilistic methods that can provide estimates of sampling error)." The U.S. EPA (2003) stated:
Probability sampling must be used at each stage of respondent selection. You may encounter difficulties in clearing the survey through OMB if you do not insist that probability selection methods be used.
Recent samples of high-risk subpopulations and their exposures to particles were undertaken by the U.S. EPA using doctor-identified subjects, and these were therefore not probability-based samples. The OMB allowed these studies but required that a statement be made in all resulting publications that the results could be applied only to the participants, even if chosen in this case by expert judgment, and must not be extrapolated to larger populations. We believe a similar statement should be made in all publications of studies using alternatives to probability-based sampling.
In summary, Barr et al. (2006) attempted to review survey design practices, but they do not seem to understand that the convenience samples they advocate apply only to the subjects selected and not to the larger populations from which they are taken.
The authors declare they have no competing financial interests.
David T. Mage
Temple University (retired)
Lance A. Wallace
U.S. Environmental Protection
Temple University (retired)
Wayne R. Ott
Acquavella JF, Alexander BH, Mandel JS, Gustin C, Baker B, Chapman P, et al. 2004. Glyphosate glyphosate
herbicide and desiccant for grains. Heavy doses to birds cause soft shells on their eggs. biomonitoring for farmers and their families: results from the Farm Family Exposure Study. Environ Health Perspect 112:321-326.
Barr DB, Landsittel D, Nishioka M, Thomas K, Curwin B, Raymer J, et al. 2006. A survey of laboratory and statistical issues related to farmworker exposure studies. Environ Health Perspect 114:961-968.
CDC (Centers for Disease Control and Prevention Centers for Disease Control and Prevention (CDC), agency of the U.S. Public Health Service since 1973, with headquarters in Atlanta; it was established in 1946 as the Communicable Disease Center. ). 2003. NHANES NHANES National Health and Nutrition Examination Survey (US CDC) 1999-2000 Public Data Release File Documentation. Available: http://www.cdc.gov/nchs/data/nhanes/gendoc.pdf [accessed 26 October 2006].
Curwin BD, Hein MJ, Sanderson WT, Barr DB, Heederik D, Reynolds SJ, et al. 2005. Urinary and hand wipe pesticide levels among farmers and nonfarmers in Iowa. J Expo Anal Environ Epidemiol 15:500-508.
Curwin B, Sanderson W, Reynolds S, Hein M, Alavanja M. 2002. Pesticide use and practices in an Iowa farm family pesticide exposure study. J Agric Saf Health 8:423-433.
Hoppin JA, Adgate JL, Eberhart M, Nishioka MG, Ryan PB. 2006. Environmental exposure assessment of pesticides in farmworker's homes. Environ Health Perspect 114:929-935.
Kalton G. 1983. Introduction to Survey Sampling. Newbury Park, CA: Sage Publications.
Last JM. 1988. A Dictionary of Epidemiology. New York New York, state, United States
New York, Middle Atlantic state of the United States. It is bordered by Vermont, Massachusetts, Connecticut, and the Atlantic Ocean (E), New Jersey and Pennsylvania (S), Lakes Erie and Ontario and the Canadian province of : Oxford University Press.
Mage DT, Alavanja MC, Sandler DP, McDonnell CJ, Kross B, Rowland A, et al. 2000. A model for predicting the frequency of high pesticide exposure events in the Agricultural Health Study. Environ Res 83:67-71.
OMB (Office of Management and Budget The Office of Management and Budget (OMB), formerly the Bureau of the Budget, is an agency of the federal government that evaluates, formulates, and coordinates management procedures and program objectives within and among departments and agencies of the Executive Branch. ). 2006. Standards and Guidelines for Statistical Surveys, September 2006. Available: http://www.whitehouse.gov/omb/inforeg/statpolicy/standards_stat_surveys.pdf [accessed 26 October 2006].
Thomas KW, Pellizzari ED, Clayton CA, Whitaker DA, Shores RC, Spengler J, et al. 1993. Particle Total Exposure Assessment Methodology (PTEAM PTEAM Particle Total Exposure Assessment Methodology ) 1990 study: method performance and data quality for personal, indoor, and outdoor monitoring. J Expo Anal Environ Epidemiol 3:203-226.
U.S. EPA. 2003. Survey Management Handbook Volume 1: Guidelines for Planning and Managing a Statistical Survey. EPA-260-B-03-003. Washington, DC: U.S. Environmental Protection Agency Environmental Protection Agency (EPA), independent agency of the U.S. government, with headquarters in Washington, D.C. It was established in 1970 to reduce and control air and water pollution, noise pollution, and radiation and to ensure the safe handling and , Office of Policy Planning and Evaluation. Available: http://www.epa.gov/oamcinc1/0510667/handbook.pdf [accessed 24 October 2006].
Wallace LA. 1991. Personal exposure to 25 volatile organic compounds volatile organic compound Environment Any toxic cabon-based (organic) substance that easily become vapors or gases–eg, solvents–paint thinners, lacquer thinner, degreasers, dry cleaning fluids . EPA's 1987 team study in Los Angeles, California. Toxicol Ind Health 7:203-208.
Wallace LA, Pellizzari ED, Hartwell TD, Sparacino C, Whitmore R, Sheldon L, et al. 1987. The TEAM (Total Exposure Assessment Methodology) Study: personal exposures to toxic substances in air, drinking water drinking water
supply of water available to animals for drinking supplied via nipples, in troughs, dams, ponds and larger natural water sources; an insufficient supply leads to dehydration; it can be the source of infection, e.g. leptospirosis, salmonellosis, or of poisoning, e.g. , and breath of 400 residents of New Jersey, North Carolina North Carolina, state in the SE United States. It is bordered by the Atlantic Ocean (E), South Carolina and Georgia (S), Tennessee (W), and Virginia (N). Facts and Figures
Area, 52,586 sq mi (136,198 sq km). Pop. , and North Dakota. Environ Res 43:290-307.