Evaluation and Use of Epidemiological Evidence for Environmental Health Risk Assessment: WHO Guideline Document.

Environmental health risk assessment is increasingly being used in the development of environmental health policies, public health decision making, the establishment of environmental regulations, and research planning. The credibility of risk assessment depends, to a large extent, on the strength of the scientific evidence on which it is based. It is, therefore, imperative that the processes and methods used to evaluate the evidence and estimate health risks are clear, explicit, and based on valid epidemiological theory and practice. Epidemiological Evidence for Environmental Health Risk Assessment is a World Health Organization (WHO) guideline document. The primary target audiences of the guidelines are expert review groups that WHO (or other organizations) might convene in the future to evaluate epidemiological evidence on the health effects of environmental factors. These guidelines identify a set of processes and general approaches to assess available epidemiological information in a clear, consistent, and explicit manner. The guidelines should also help in the evaluation of epidemiological studies with respect to their ability to support risk assessment and, consequently, risk management. Conducting expert reviews according to such explicit guidelines would make health risk assessment and subsequent risk management and risk communication processes more readily understood and likely to be accepted by policymakers and the public. It would also make the conclusions reached by reviews more readily acceptable as a basis for future WHO guidelines and other recommendations, and would provide a more rational basis for setting priorities for future research. Key words: environmental health, environmental health risk assessment, guidelines, international cooperation, World Health Organization. Environ Health Perspect 108:997-1002 (2000). [Online 11 September 2000]

http://ehpnet1.niehs.nih.gov/docs/2000 /108p997-1002krzyzanowski/abstract.html

Environmental health risk assessment contributes increasingly to policy development, public health decision making, the establishment of environmental regulations, and research planning. It also often plays an important role in cost-benefit analysis and risk communication. Its credibility depends, to a large extent, on the strength of the scientific evidence on which it is based. Epidemiology, toxicology, clinical medicine, and environmental exposure assessment all contribute information for risk assessment.

However, epidemiological studies play a unique role in the assessment of the health risk of environmental factors. Unlike laboratory experiments, epidemiology provides evidence based on studies of human populations under real-world conditions. It largely avoids the extrapolations across species and levels of exposure required for the use of data from animal experiments, which contribute large uncertainties. In addition, epidemiology has often contributed to the recognition of new hazards, thereby stimulating new research and identifying new areas for public health action. The contribution of epidemiology to health risk assessment has been widely discussed (1-5). However, epidemiological studies that report associations between measures of the health of populations and the presence of hazardous factors in the environment are frequently difficult to interpret (6,7). Therefore, a careful evaluation of all existing epidemiological evidence is necessary as part of the risk assessment process.

To provide authoritative assessments of environmental epidemiology research, public health and regulatory agencies may rely on expert review groups to evaluate the evidence, draw conclusions on the existence of hazard to health, and estimate the magnitude of associated health risks. These expert reviews may then be used to support actions that are difficult and expensive. It is, therefore, imperative that the processes and methods used to evaluate the evidence and estimate health risks are clear, explicit, and based on valid epidemiological theory and practice.

To improve the methodology used by the expert groups reviewing the evidence, the World Health Organization (WHO) European Centre for Environment and Health, Bilthoven Division, in collaboration with the International Programme on Chemical Safety, initiated the project "Accepting Epidemiological Evidence for Health Impact Assessment." The results of this project are presented in this paper. This report, together with the "Appendix," summarizing the discussion of the working subgroups is also available as a WHO document (8).

Scope and Purpose

The purpose of this project is to develop guidelines that identify a set of processes and general approaches to assess available epidemiological information in a clear, consistent, and explicit manner. The guidelines should also help in the evaluation of epidemiological studies with respect to their ability to support risk assessment and, consequently, risk management.

Conducting expert reviews according to such explicit guidelines would make health risk assessment and subsequent risk management and risk communication processes more readily understood and likely to be accepted by policymakers and the public. From the standpoint of WHO, it would also make the conclusions reached by reviews more readily acceptable as a basis for future WHO guidelines and other recommendations, and would provide a more rational basis for setting priorities for future research.

This project focuses only on approaches to the evaluation and use of epidemiological evidence for health risk assessment. However, this should not be interpreted as implying that only epidemiological studies are important. The working group, and WHO, appreciate that data from toxicological, clinical, and other areas of research often play vital roles in both the characterization of health hazards and the estimation of risks to health, and may, in the absence of suitable epidemiological data, provide the sole basis for such activities.

Public health action (e.g., the reduction of population exposure to a suspected hazard or even its elimination from the human environment) must often proceed even when the scientific evidence is insufficient. Most of the working group members agree that the precautionary principle should play a role in guiding public health action where there is uncertainty.

The project considers two distinct activities of health risk assessment: health hazard characterization and health impact assessment. They correspond to components of risk assessment defined by both the U.S. National Research Council (NRC) and WHO (1,9).

* Health hazard characterization involves the identification of environmental hazards via the collection, evaluation, and interpretation of available evidence from epidemiology and other scientific disciplines concerning the association between an environmental factor and human health.

* Health impact assessment involves the quantification of the expected health burden due to an environmental exposure in a specific population.

Health hazard characterization comprises the hazard identification and elements of the dose-response assessment stages of the NRC risk assessment paradigm, at least as they apply to epidemiological studies. Health impact assessment combines the exposure assessment, dose-response assessment, and risk characterization stages of the NRC and WHO risk assessment paradigm. Thus, the two stages overlap and interlock.

This project focuses on the evaluation and use of epidemiological evidence on associations between environmental factors and health. This evidence is used to support the assessment of health impact of certain exposures. The health impact assessment discussed in this report is, therefore, not synonymous with the broader concept of assessment of health impacts of specified action on the health of a defined population. The latter is an emerging tool in the evidence-based public health policy making. It can be applied to a wide range of actions, policies, or projects on various determinants of health, such as behavioral factors, socioeconomic issues, or health care system reforms. Environmental health risk assessment, with its well-established methodology, is a significant contribution to the wider concept of health impact assessment (10).

The scope and purpose of the project with respect to the audience and the issues that it would address have been further refined as follows:

* The target audiences for these guidelines are expert review groups that WHO (or other organizations) might convene in the future to evaluate epidemiological evidence on the health effects of environmental factors.

* The working group convened to develop this project saw its role as providing future review groups with general recommendations and principles for conducting such evaluations, rather than providing formulae, or lists of approved methods.

* The working group focused on the evaluation of evidence in the context of large-scale public health issues, as opposed to local emergencies. The working group acknowledged that epidemiological studies of local environmental exposures (e.g., clusters of childhood leukemia in the vicinity of nuclear power plants) might provide evidence about large-scale public health concerns, but felt that the evaluation of such outbreaks presents a unique set of problems that warrant attention in their own right.

* The working group agreed that health impact assessments are conducted for a range of purposes and under a variety of conditions, and therefore the purpose of the health impact assessment will, and should, determine its scope, form, and content.

* The principles described apply, in the first place, to chemical pollutants. In reviews concerning some other exposures, adjustments may be proposed. Should such deviations from the principles be applied, a clear justification must be given.

Process

The WHO working group of experts in epidemiology, public health, and environmental policy was assembled at the end of 1998 (see "Appendix"). The experts were selected on the basis of their following qualifications:

* experience in the scientific review of epidemiological evidence for governmental bodies, WHO, or other public and private sector organizations

* involvement in risk assessment of environmental factors

* involvement in communication related to health risk with general public or decision makers

* representation of wide range of countries within the European region of WHO and the United States.

The experts were asked to prepare working papers presenting their views and proposals concerning the process of review of epidemiological evidence, as well as criteria for its acceptance and use in assessment of health risk of environmental factors. The papers were distributed to all working group members and provided a basis for the discussion at the meeting of the working group convened in Il Ciocco, Italy, from 31 May to 2 June 1999. Dr. Robert Maynard chaired the meeting, and Dr. Aaron Cohen acted as its rapporteur. The working papers are not included in this report, although the individual authors may choose to publish them elsewhere.

After a half-day plenary discussion to establish the exact scope of the meeting and the methods of working, two subworking groups were formed: one to consider health hazard characterization and another to consider health impact assessment. A third group met initially to discuss issues related to the broader social and public policy contexts in which environmental health risk assessment is used. Their views were ultimately incorporated into the chapters of the two main subworking groups.

After an iterative process of subgroup discussion and plenary meetings, the subworking groups summarized their discussions and drafted recommendations on their respective topics, which were further refined following the plenary meeting of the working group. These discussions are summarized in two papers prepared by the two working groups and are presented in Annex 3 of the WHO document (8). On the basis of these materials, the rapporteur of the meeting, assisted by the chairmen and rapporteurs of the subgroups, and the secretariat prepared a draft of the meeting report. Prior to its finalization, that draft was presented to all members of the working group to ensure that it correctly reflected the working group's consensus on the recommendations and the rationale for them.

The draft report was discussed at a special WHO symposium, organized at the joint conference of the International Society for Environmental Epidemiology and the International Society of Exposure Analysis in Athens, Greece, in September 1999, and made available through the World Wide Web for review. The comments received from approximately 20 scientists were used in the preparation of the final draft of this guideline report in November 1999. Revisions based on these comments have focused on improving the depth and clarity of presentation of the recommendations and conclusions, rather than on additional detailed discussion of methodological issues of risk assessment. The draft was reviewed by the chairs and rapporteurs of the working group (and subgroups) and accepted, with small editorial changes, as the WHO guideline document in January 2000 (8).

Conclusions and Recommendations

The following text lists the major recommendations made by the working group for the evaluation and use of environmental epidemiology studies for health risk assessment. They constitute the core of the WHO Guidelines on Epidemiological Evidence for Environmental Health Risk Assessment (8). The guidelines comprise a set of general recommendations, followed by specific recommendations for evaluations of epidemiological research for health hazard characterization, and use of epidemiological data for health impact assessment.

General Recommendations

Expert review groups should adopt a systematic and explicit approach to the assessment of epidemiological evidence for health risk assessment. The working group acknowledged that various expert review groups had, in the past, used a variety of methods and standards to assess epidemiological evidence, and that these were often inadequately described.

Expert review groups, and the agencies that sponsor them, should strive for better communication with stakeholders (e.g., citizens, private interests, government agencies) regarding the process of evaluating, and drawing conclusions from, epidemiological evidence. The need for an evaluation of the epidemiological evidence often reflects the existence of divergent views among stakeholders about the true extent of the risk. When expert review groups make explicit and explain in clear terms the methods they use to conduct their evaluations and reach their conclusions, they reduce the potential for those conclusions to be misunderstood and mistrusted by stakeholders.

To improve the applicability of epidemiological research to health risk assessment, future epidemiological studies should seek where possible to provide results in a way that enhances the health risk assessment at the interface of epidemiology, other fields of research, and policymaking. In particular, the study reports should describe as precisely as possible the exposure characteristics and the shape of the exposure-response function, as well as distinguish between the acute and chronic effects of exposure.

The WHO secretariat of the future reviews should assess the feasibility of implementation of these recommendations and the increased time and effort that will be needed, and modify the guidelines as necessary. WHO should also attempt to assess whether their use leads to increased acceptance by stakeholders of the evaluations of environmental epidemiology research produced by expert review groups.

This proposed more rigorous and thorough approach to the review of the evidence and its use in health impact assessment may require increased effort and resources. The transparency of the methods should, however, lead to a wider acceptance and applicability of the reviews, and may reduce the need for duplication of effort and facilitate updating.

The working group did not propose a scale or rating of the evidence with respect to a level of proof required to support risk management decisions. Although health hazard characterization precedes, and is often viewed as a prerequisite for, health impact assessments (because it provides the scientific justification for them, and provides data for the calculation of risk estimates), the existence of a specific level of scientific evidence required to justify either a health impact assessment or subsequent action is controversial. For example, expert judgment that the available evidence is consistent with a causal relationship between exposure and health effect is considered a necessary condition for action by some, but not all (11). While most members of the working group agreed that the precautionary principle should play a role in public health decision-making when there are scientific uncertainties, there was no general agreement that the principle should play a role in the evaluation of epidemiological evidence per se. Discussion of the level of evidence on hazard needed to conduct health impact assessment is summarized in section A3.1 of the WHO guideline document (8).

Recommendations for the Evaluation of Epidemiological Evidence for Health Hazard Characterization

The working group recommended five general guidelines for five aspects of the evaluation of epidemiological research:

Development of a protocol for the review. Expert assessments of epidemiological evidence for health hazard characterization should be conducted systematically according to an explicit protocol defined in advance. The objectives of a systematic review are transparency, avoidance of bias, validity, replicability, and comprehensiveness. A systematic approach provides an efficient way of updating the evidence base as new studies emerge, and will facilitate research planning. A protocol for the systematic review ensures that the expert group has a common understanding of its task and will adhere to the systematic approach recommended by WHO. It is expected that revisions of the protocol may be needed as new aspects of the task emerge during the review. The essential components of the protocol will be the following:

* Specification of the question(s) to be addressed by the health hazard characterization.

* Justification of the expertise represented in the health hazard characterization expert group. The criteria for selection should be based on having the appropriate mix of scientific expertise and experience. Within these criteria, WHO will also consider the need for geographic representation.

* Specification of the methods to be used for identification of relevant studies, assessment of evidence of the individual studies, and interpretation of the entire body of available evidence (see below).

Identification of relevant studies. The assessment should be based on comprehensive identification of all relevant studies. A comprehensive bibliographic search would include the following: a) involvement of qualified searchers (e.g., librarians, trained investigators); b) definition of an explicit search strategy including identification of key words; c) an effort to include all available studies; d) search of bibliographic databases; e) inclusion of non-English reports.

Optional methods that might be considered by the expert group include hand searching of journals, and inclusion of abstracts and unpublished data (including writing to authors of published data).

Systematic assessment of the validity of epidemiological studies. As Hill emphasized (12), this assessment should aim at answering the question, "Is there any other way of explaining the set of facts before us [study results], is there any other answer equally, or more, likely than cause and effect?" The evaluation should consider the following:

* Evidence on strength of association, its temporality, biological plausibility, coherence, consistency and specificity.

* Characteristics of exposure-response relationships. The demonstration of specific patterns of association can provide strong support for causal interpretations if pathophysiological models agree with them. In such cases, more complex, and hence less implausible, patterns of confounding or bias are required as counter-explanations. In addition, the information on exposure-response relationships in particular study populations is an important component in health impact assessments of other populations (see "Recommendations for the Use of Epidemiological Data for Health Impact Assessment").

* Alternative explanations for the observed associations. They fall into three categories: chance, bias (information, selection, analytic), and confounding.

* Results of any sensitivity analysis. In such analysis the outcome variable(s) are examined with respect to a) changes in expression of exposure variables, b) addition of other plausible explanatory variables, and/or c) introduction or removal of confounding variables.

Conduct of systematic overviews of evidence from multiple studies: use of meta-analysis. Although meta-analysis is widely viewed as simply a method for statistically combining the results of multiple studies, it can contribute more to hazard characterization when viewed as a quantitative review of the literature, a "study of studies." Conducted in this way, a meta-analysis looks for consistent patterns among, and sources of discrepancies between, studies (13, 14). Expert groups should consider the following questions when conducting meta-analyses: a) How will heterogeneity among studies be assessed? b) Will summary effect estimates be calculated, and by which methods? The working group recommends that expert review groups consider the following issues when designing and conducting quantitative reviews (meta-analyses) of epidemiological literature or assessing their findings:

Protocol. Each meta-analysis must have its own protocol, perhaps nested within the overall protocol for the health hazard characterization. The protocol should include a clear statement of the objectives of the review and the methods to be employed.

Inclusion criteria. It is desirable for a meta-analysis to be inclusive rather than exclusive. Sensitivity to various inclusion criteria can then be examined.

Use of quality scores. Reducing the features of a set of epidemiological studies to a single measure of quality is not recommended, because these features may affect the results of the studies in different directions and to varying degrees. It is preferable to assess the characteristics of the primary studies individually.

Chance. In meta-analysis, the results are usually weighted by the statistical precision (in general, by the amount of information) of each primary study. Adjustment for the amount of information can be achieved through either inverse-variance weighting or random effects models.

Publication bias. The results of certain kinds of primary studies are more likely to be published than of the others. The impact of the publication bias can be minimized by thorough search and inquiries regarding completed studies. If the number of studies in the analysis is sufficient, their results can be plotted in a funnel graph to check for a possible publication bias. Corrections and tests for publication bias have been proposed, but these must be applied with special caution because of the assumptions involved and low power of the tests. The impact of the publication bias can, and should, also be assessed by sensitivity analysis.

Assessment of overall heterogeneity. Systematic, quantitative assessment of heterogeneity may contribute significantly to the identification of both methodological and natural sources of variability of epidemiological effect estimates, including the identification of susceptible subgroups and exposure conditions.

Meta-analytic methods that may be used to compare studies. An example is stratified analysis or meta-regression.

Sensitivity analyses. Such analyses might, for example, examine the sensitivity of summary estimates to reasonable alternatives with regard to the inclusion and exclusion of particular studies. One can also evaluate the sensitivity to alternative approaches to the extraction of results from published reports.

Methods to obtain summary estimates from different studies (aggregative meta-analysis). Though quantitative summary estimates are not essential for health hazard characterization, they will be a particularly useful input to the health impact assessment (see "Recommendations for the Use of Epidemiologic Data for Health Impact Assessment").

Drawing conclusions from epidemiological evidence. After the epidemiological evidence has been evaluated and appropriately summarized, as discussed above, expert judgment as to whether the observed associations are most consistent with a causal explanation or some alternative is required. This judgment should draw upon all the available epidemiological evidence, as well as on evidence from toxicology, clinical medicine, and other disciplines, as appropriate. The method of choice is critical scientific thinking: there are no formulas or checklists that will suffice, although, as noted above, Hill's attributes can provide useful guidance and focus. It is critical, however, that expert review groups make explicit the process of scientific reasoning that led to a judgment concerning causality. This explanation should include descriptions of a) how expert reviewers weighted particular features of the epidemiological studies (e.g., assessments of bias, confounding, exposure-response) in reaching their judgment; b) how expert reviewers used guidelines such as Hill's attributes; and c) how nonepidemiological sources of evidence figured in their interpretation of the epidemiological evidence, and how that evidence contributed to their overall judgment.

Expert judgments concerning the causal nature of observed associations are often accompanied by qualifications as to the degree of uncertainty. When the product of a health hazard characterization is presented as a conclusion regarding the existence (or nonexistence) of a hazard, the degree of uncertainty is sometimes expressed on a qualitative (weak, moderate, strong evidence for hazard) or on a quantitative scale. If a quantitative scale is devised, it should be capable of being reproduced by other experts. In either case, the use of a particular scale and the meaning of its levels should be clearly explained. More generally, it may be useful in the future to standardize such scales to avoid problems of noncomparability among the reviews produced by different expert review groups.

Recommendations for Use of Epidemiological Data for Health Impact Assessment

The working group made recommendations with regard to the use of epidemiological data for the design, implementation, and the interpretation of health impact assessments:

The design and implementation of health impact assessments. Health impact assessments, which aim to quantify the expected health burden in a specific population(s) should be conducted according to explicit protocols that accomplish the following:

Specify the purpose of the assessment. The purpose(s) of the health impact assessment should be made clear, because decisions concerning the choice of epidemiological and other data and quantitative methods will depend on the objectives of the assessment. Ideally policymakers, scientists, and stakeholders should be involved in defining the scope of the assessment, as different parties may have different questions about, and perspectives of, the same environmental health issue.

Specify the method(s) used to quantify uncertainty. It should be made explicit in each health impact assessment what the uncertainties are likely to be and how the assessors will deal with them. The choice of data and methods by which to quantify uncertainty may be determined by the specific objectives of the impact assessment (e.g., identification of a maximum or minimum potential impact). The quantification of the uncertainty contributed by epidemiological effect estimates should consider both their statistical variability (i.e., precision), and nonstatistical variability resulting from sources of error (e.g., bias and confounding) in the epidemiological data.

Specify the metric of exposure to the specified hazards and methods to identify its distribution in the population for which assessment is requested. A clear and explicit definition of the metric of exposure, i.e., the operationalization of the cause considered in the health impact assessment, should be provided. Health impact assessment will require information on the distribution of exposure in the target population, which will ultimately need to be combined with information on the exposure-response function in order to conduct the assessment. Depending on the available evidence (e.g., from epidemiological studies), the metric may need to incorporate temporal (e.g., induction period or latency) and compositional (e.g., mixtures and surrogates for them) aspects of exposure. The impact assessment should describe and, whenever possible, quantify the uncertainty contributed by the exposure assessment.

The magnitude of the estimated impact will depend strongly on the level and range of exposure used in the health impact assessment. The choice of a reference level for the impact assessment may be particularly complicated and may require the consideration of epidemiological and other data with regard to issues such as the existence of thresholds and natural background levels. If exposures in the target populations exceed or are below those that have been studied epidemiologically, it will be necessary to determine whether effects should be extrapolated. Ultimately, these choices will depend on both expert judgment and the perspective and purpose(s) of the assessment, but the basis for those choices should be clearly explained.

Define the appropriate health outcomes. A particular health impact assessment might focus on one or several health effects. If there is evidence of an environmental hazard being associated with several health effects, then ideally the impact should be assessed separately for each one. In practice, several aspects of the health impact assessment, mainly its purpose and objectives, the definition of exposure, and the availability of the necessary data, will guide the selection of health outcome(s). On the basis of these considerations, assessors may decide not to include all conceivable outcomes. These decisions and their rationales should be made explicit.

Specify methods for estimating the exposure-response relationship. The quantitative association between the exposure and the health effect(s) is an essential component for the calculation of the attributable number of cases, and information about the exposure-response function is potentially the key contribution of epidemiological demiological studies to a health impact assessment.

Due to both uncertainties in epidemiological studies and true variability in the association between exposure and health outcomes within and among human populations, the available body of epidemiological evidence may provide different exposure-response functions for the same general exposure-outcome relationship. Thus, for a given health impact assessment, the process used to derive the exposure-response function(s) must be well defined. It should, at a minimum, include a systematic review of the available epidemiological information to obtain information on exposure-response relationships for every selected health outcome. All studies with quantitative information on exposure or that allow linkage to such information should be considered as potentially providing information for the exposure-response evaluation. The hazard characterization process normally will provide an inventory of the relevant studies.

Epidemiological studies identified as potentially providing useful exposure-response information may need to undergo an additional selection process that considers: a) the quality of exposure measurement; b) whether the exposure metric is the same as that available for the target population of the impact assessment; and c) whether the estimated measures of effect are generalizable to the target population due to the influence of effect modifiers such as local socioeconomic factors or the prevalence of susceptible subgroups.

Projecting exposure-response relationships beyond the range of exposure observed in the available epidemiological studies may be necessary for a given health impact assessment. However, the validity of such extrapolations should not be simply assumed, but rather, arguments for, and the limitations and potential impacts of, extrapolations should be carefully addressed in the health impact assessment, including allowance for additional uncertainty.

An expert group may decide that combining exposure-response information from different epidemiological studies, for example, via meta-analysis of published results or pooling of original data, is the best approach for deriving an exposure-response relationship for a given impact assessment. These approaches can potentially provide not only an overall summary of an exposure-response function, but also (and perhaps as or more important) a range of estimates corresponding to possible sources of heterogeneity in the target population. Care should be taken to present estimates of the statistical and other sources of uncertainty in any combined estimates of the dose-response function.

Specify approach for obtaining measures of baseline frequency of health outcomes in the target population. Estimating the impact of exposure requires information on the baseline occurrence (rate, prevalence) of outcome(s) in the target population. Combined with the estimates of relative effect most often provided by epidemiological studies, it yields an estimate of impact of exposure in absolute terms, e.g., number of cases of disease or deaths (see below). Although exposure-response relationships may be derived from the international literature, the baseline disease occurrence should preferably be obtained from data regarding the target population of the assessment. If such data are unavailable or inadequate, health frequency data from other populations may sometimes be used. In such cases, the potential limitations of such substitutions should be considered and thoroughly discussed in the health impact assessment.

Specify methods for estimating the number of attributable cases. The estimation of the burden of disease or mortality expected in the target population requires three basic elements whose estimation is discussed above: a) the distribution of the exposure in the target population; b) estimates of the epidemiology-based exposure-effect function; and c) epidemiology-based estimates of baseline frequency of the health measure of interest.

Using this information, and under the assumption that exposure causes the health outcome, an epidemiology-based health impact assessment estimates the population-attributable proportion (of disease or death) due to exposure, a measure described in standard epidemiological texts (13). When applied to the target population, the population-attributable proportion yields an estimate of the expected number of cases attributed to the exposure.

In practice, both the estimation and interpretation of the population-attributable proportion and its application to the target population may involve a number of subtleties. These involve, for example, the choice of relative risk estimator when there is evidence of confounding (15, 16). The assumptions underlying the statistical methods used to estimate attributable proportions or other measures of impact, and their implications for interpretation of those estimates, should be discussed.

The uncertainties in the data that contribute to the impact assessment, as well as any natural sources of heterogeneity in the effect of exposure, will often require the calculation of a range of estimates in order to describe fully the likely impact of exposure and to better reflect the uncertainty.

Issues in the interpretation of health impact assessments. The results of the impact assessment require not only clear presentation, but also coherent interpretation, including explicit discussion of assumptions and limitations. Specific components of the overall uncertainty and their potential impact on the results should be addressed, as discussed above. Sensitivity analyses, in which the effects of key assumptions are explored quantitatively, may provide a better sense of the overall uncertainty of the estimates than purely qualitative discussion, and should be performed when appropriate.

In general, the direct effect of the removal of a particular exposure may only rarely be estimated. Depending on the health outcome, the specificity of the exposure, and the time frame of exposure and effect, the benefit (or reversibility) may be realized either much later than predicted, or not to the full extent. In particular, removal of the environmental hazard may not prevent the occurrence of the estimated number of cases due to how competing risks may come into play if one contributing cause (the exposure) is removed or reduced.

REFERENCES AND NOTES

(1.) National Research Council. Committee on the Institutional Means for Assessment of Risks to Public Health. Risk Assessment in the Federal Government: Managing the Process. Washington, DC:National Academy Press, 1983.

(2.) Gordis L, ed. Epidemiology and Health Risk Assessment. New York:Oxford University Press, 1988.

(3.) Federal Focus, Inc. Principles for Evaluating Epidemiologic Data in Regulatory Risk Assessment. Washington, DC:Federal Focus, Inc, 1996.

(4.) Hertz-Picciotto I. Epidemiology and quantitative risk assessment: a bridge from science to policy. Am J Public Health 85:484-491 (1995).

(5.) Samet JM, Burke TA. Epidemiology and risk assessment. In: Applied Epidemiology: Theory and Practice (Brownson RC, Petitti DB, eds). New York:Oxford University Press, 1998;137-175.

(6.) Neutra RR, Trichopoulos D. The place of epidemiology in environmental decisions: needed support for the development of risk assessment policy. Environ Health Perspect 101(suppl 4):67-69 (1993).

(7.) Taubes G. Epidemiology faces its limits. Science 289:164-189 (1995).

(8.) WHO. Evaluation and Use of Epidemiological Evidence for Environmental Health Risk Assessment. Guideline Document. Copenhagen:WHO Regional Office for Europe, 2000.

(9.) WHO. Principles for the Assessment of Risks to Human Health from Exposure to Chemicals. Environmental Health Criteria 210. Geneva:World Health Organization, 1999.

(10.) Scott-Samuel A. Health impact assessment--theory into practice. J Epidemiol Community Health 52:704-705 (1998).

(11.) Rothman KJ, ed. Causal Inference. Chestnut Hill, MA:Epidemiology Resources Inc, 1988.

(12.) Hill AB. The environment and disease: association or causation. Proc R Soc Mod 58:295-300 (1965).

(13.) Rothman KJ, Greenland S. Modern Epidemiology. 2nd ed. Philadelphia, PA:Lippincott-Raven, 1998.

(14.) Greenland S. Quantitative methods in the review of epidemiologic literature. Epidemiol Rev 9:1-30 (1987).

(15.) Greenland S, Robins J. Conceptual problems in definition and interpretation of attributable fractions. Am J Epidemiol 128:1185-1197 (1988).

(16.) Rochill B, Newman B, Weinberg C. Use and misuse of population attributable fractions. Am J Public Health 88:15-19 (1998).

Appendix. Working group members.

Ross Anderson
St. George's Hospital Medical School
London, United Kingdom

Ben Armstrong
London School of Hygiene and Tropical Medicine
London, United Kingdom

Larry Chambers
McMaster University
Hamilton, Ontario, Canada

Aaron Cohen
Health Effects Institute
Cambridge, Massachusetts, USA

Lynne Edwards
European Commission
Brussels, Belgium

David Gee
European Environment Agency
Copenhagen, Denmark

Paul Harrison
MRC Institute for Environment and Health
Leicester, United Kingdom

Karl-Heinz Jockel
Institut for Medizinische Informatik, Biometrie und
  Epidemiologie
Essen, Germany

Pierre Jouannet
Service d'histologie-embryologie oriente en biologie
  de la reproduction, Hospital Cochin
Paris, France

Matti Kamppinen
University of Turku
Turku, Finland

Klea Katsouyanni
University of Athens Medical School
Athens, Greece

Manolis Kogevinas
Universitat Autonoma de Barcelona
Barcelona, Spain

Michal Krzyzanowski
WHO European Centre for Environment and Health
Bilthoven, Netherlands

Nino Kunzli
Institute of Social and Preventive Medicine, University
  of Basel
Basel, Switzerland

Erik Lebret
National Institute of Public Health and the
  Environment (RIVM)
Bilthoven, Netherlands

Rolaf van Leeuwen
WHO European Centre for Environment and Health
Bilthoven, Netherlands

Marco Martuzzi
WHO European Centre for Environment and Health
Rome, Italy

Robert Maynard
Department of Health
London, United Kingdom

Goran Pershagen
Karolinska Institute
Stockholm, Sweden

Charles Poole
University of North Carolina School of Public Health
Chapel Hill, North Carolina, USA

Gerhard Raabe
Mobil Business Resources Corporation
New Hope, Pennsylvania, USA

Simon Rozendaal
Elsevier
Amsterdam, Netherlands

Bernd Seifert
Institute for Water, Soil and Air Hygiene
Berlin, Germany

Jack Spengler
Harvard School of Public Health
Boston, Massachusetts, USA

John Vandenberg
U.S. Environmental Protection Agency
Research Triangle Park, North Carolina, USA

Paolo Vineis
Turin University
Turin, Italy

Douglas Weed
National Cancer Institute
Bethesda, Maryland, USA

Bogdan Wojtyniak
National Institute of Hygiene
Warsaw, Poland

Maged Younes
International Programme on Chemical Safety
Geneva, Switzerland

Denis Zmirou
Faculte de Medecine-Pharmacie
La Tronche, France

WHO Working Group(*)

WHO European Centre for Environment and Health, Bilthoven Division, A. van Leeuwenhoeklaan 9, Bilthoven, The Netherlands

Address correspondence to M. Krzyzanowski, P.O. Box 10, NL-3730 AA De Bilt, Netherlands. Telephone: [phone]. Fax: [phone]. E-mail: [e-mail]. Website: http://www.who.nl

(*) See "Appendix" for list of World Health Organization Working Group Members.

This project has been supported by a special grant from the Swiss Agency for the Environment, Forests and Landscape.

The views expressed in this paper are those of the contributors and do not necessarily represent the decisions or stated policy of the World Health Organization.

Received 2 February 2000; accepted 22 May 2000.