Risk assessment at the EPA: an agency self-exam.

The U.S. Environmental Protection Agency (EPA) has for the first time conducted an internal investigation of its own approach to risk assessment. The investigation's results are contained in a 193-page staff paper titled An Examination of EPA Risk Assessment Principles and Practices, released in final form 25 March 2004. The staff paper is not a guidance document--rather, it is a snapshot of how risk assessments are currently performed at the EPA. The paper also provides recommendations for how the agency can strengthen and improve its risk assessments.

"We're not going to change this document," says Kerry Dearfield, a senior scientist in the EPA Office of the Science Advisor, who played a key role in coordinating the effort. "We want people to look at it and determine if they agree with its conclusions or not. The intent is to create a dialogue about how we can move EPA risk assessment forward."

The inspiration for this effort arose from an unprecedented February 2003 request by the White House Office of Management and Budget (OMB) for public comment on risk assessment procedures across the federal government. The several hundred comments received, many of which focused specifically on methods used at the EPA, were eventually incorporated into the OMB's 2003 annual report to Congress. The OMB's focus on risk assessment galvanized EPA officials, who, at the urging of Paul Gilman, science advisor for the EPA and assistant administrator of the EPA Office of Research and Development, convened a task force to review how risk assessments were being performed at the agency.

"This review morphed into the idea of the staff paper," Dearfield recalls. "We wanted to look at the overarching issues that the comments had raised and look for opportunities to refine or otherwise revise practices."

Hundreds of staffers throughout the EPA contributed to the effort. Dearfield says the task force deliberately sought the input of staff-level risk assessors. "The senior managers did not contribute as much," he explains. "What we were really looking for was the ground-level view--we wanted to know what the risk assessors are actually doing. We wanted the real truth, warts and all, and I think we succeeded in getting that."

Detailed Content

The staff paper itself reflects the EPA's delicate position as a public

health agency positioned between industry and environmental concerns. Much of the document provides detailed descriptions of methods and discussions about how conservatism, uncertainty, and variability influence the risk assessment process. In terms of structure, the staff paper is arranged around a series of themes, including uncertainty and variability, the use of default parameters and extrapolation assumptions, site- and chemical-specific assessments, and ecological assessment.

The paper presents many of the EPA's current practices while describing ongoing efforts to make the risk assessment process more data-intensive and robust. Opportunities to enhance risk assessment are also described. For instance, the report acknowledges that accumulation of more toxicity and exposure data will lessen reliance on defaults. The report highlights the need to increase the transparency and clarity of risk assessment, for the benefit of both risk managers, who make decisions based on the outcomes, and the public, who must understand the assumptions upon which those decisions are based.

"The EPA should be commended on developing and sharing this report," says Christopher Portier, chief of the NIEHS Environmental Toxicology Program. "Other agencies, including our own, can read this document and better understand the limitations of risk assessment as it is currently practiced. This will help us to do the science necessary to aid the EPA in developing a stronger scientific basis for risk assessments."

To a large extent, the staff paper was intended as a way to address concerns posed during the OMB review, particularly those articulated by the American Chemistry Council (ACC), which provided the bulk of comments about EPA risk assessment to the OMB. EPA officials, including Dearfield and Gilman, met with representatives of this leading industry group at the start of the process to discuss a range of issues, particularly agency approaches to defaults and other methods to address uncertainty that the ACC claims are overly conservative and protective.

Peter Preuss, director of the EPA National Center for Environmental Assessment, says he's not surprised at the level of the ACC's involvement. "They are probably the group that has organized itself best to deal with these issues, many of which they have raised previously," he says.

With few exceptions, environmental groups were not engaged in dialogue with the EPA to the same degree as industry. The main environmental concerns were raised by the Natural Resources Defense Council (NRDC), which proposed that conservative defaults and other precautionary measures are justified because populations are typically exposed to contaminant mixtures whose cumulative health impacts are largely unknown. Jennifer Sass, a senior scientist in the NRDC's public health group, describes the staff paper as a "private conversation between industry and the EPA." Nonetheless, Sass points out that, with the staff paper, "EPA makes a scientifically credible defense that its risk assessments are not overly cautious."

The Nature of Conservatism

A degree of caution is deliberately built into EPA risk assessment to protect against the uncertainty generated by data gaps, particularly those relating to exposure and chemical effects in humans. Agency scientists use generic data, obtained from EPA guidance documents such as the Exposure Factors Handbook, to fill in data gaps so that risk assessments can proceed. Among these parameters are age- and population-specific inhalation rates, food and water consumption rates, residential exposure durations, and others.

Conservatism is also achieved with numerical safety factors (or uncertainty factors, as the EPA calls them) that account for the uncertainty of extrapolating from animal data to human effects. Safety factors are incorporated into the EPA's acceptable lifetime human exposure levels for pollutants. These levels typically have their basis in animal data, usually the no-observed-adverse-effect level (NOAEL) from a bioassay using a limited number of test animals. But thanks to statistical uncertainty, a NOAEL dose could conceivably cause effects in 10-20% of a study population, says George Lucier, former associate director of the NIEHS National Toxicology Program and now an adjunct senior scientist for Environmental Defense. Safety factors are therefore applied to account for unknowns such as interspecies differences in response and the potential for heightened sensitivity among some human populations, such as children.

When combined, defaults and safety factors can have the effect of magnifying calculated risk levels to a degree that many in industry believe is unreasonable. For instance, when doing a screening assessment, it's not unusual for the EPA to assume that a hypothetical resident threatened by contaminants at a Superfund site might be exposed to these agents for 30 years. If one were to also assume the resident is maximally exposed to these contaminants via consumption, inhalation, and dermal pathways, then the toxicity threat can appear more extreme than it likely is among most individuals.

In its comments to the OMB and in discussions with the EPA, ACC members suggested the agency's selections of defaults and safety factors constitutes a set of "policy decisions" made with insufficient consideration of the data from which the values were derived. Moreover, the ACC claims the various judgments that go into selecting these variables are not well described in EPA risk assessments, thus undermining the transparency and clarity of the process.

"The EPA needs to quantify the impact of [its] choices," says Leslie Hushka, a toxicologist with ACC member company ExxonMobil. "It's essential to evaluate more options, to be more objective, and to be policy-neutral in developing risk assessments." For example, she says, even though the EPA's risk assessment guidelines recommend an examination of risk ranges and averages (or central tendencies) in hazard and exposure assessments, risk assessors choose numbers on the high end. As a result, she says, "The agency and the public simply do not know what the consequences of these choices are."

Examining Alternatives

In response to these comments, the EPA staff paper acknowledges that clarity and transparency comprise "aspect[s] of EPA's practices that need strengthening." Default parameters, which the agency has no intention of abandoning, are described in the paper as "appropriate ... within the range of plausible outcomes ... and based on published studies, empirical observations, extrapolation from related observations, and/or scientific theory."

The staff paper also highlights ongoing efforts to ensure that "nothing appears hidden or buried in an assessment--so that nothing keeps one from understanding the impact of the elements that go into estimating and characterizing risk." These efforts include, among others, the EPA's new cancer guidelines, expected to be finalized in the fall of 2004, which emphasize a full examination of data before invoking defaults; the development of new models to support risk assessment; and an upgrading of the agency's Integrated Risk Information System, which is the principle source of toxicity values used to describe chemical hazards.

But the paper also concedes a need for "better communication of the data, assumptions, and choices used in risk assessment," noting that "dose attention to our guidance documents will ensure transparency and clarity." An opportunity to better characterize uncertainty, the paper suggests, is provided by probabilistic modeling, a statistics-based method for risk assessment long championed by industry.

Probabilistic models substitute distributions of values for "point estimates," which are single values that describe given variables in the risk equation, such as exposure frequency or duration. In a probabilistic model, computers continually select values from a range of data points for each parameter, running through the calculations thousands of times until measures of central tendency are achieved. Industry has long argued that probabilistic model outputs closely mirror real-world exposure scenarios, in contrast to point estimate methods, which are easily biased high or low, depending on chosen values.

Preuss acknowledges that the agency has been cautious in its use of probabilistic methods thus far. "It's not a method that you apply willy-nilly or in all cases," he says. "You need a fundamental set of data to use it. But we're in favor of it and will apply it where we feel it brings some added value." Consistent with this view, the staff paper notes that the EPA "should encourage greater use and reliance on probabilistic modeling when appropriate."

According to Preuss, the Risk Assessment Forum at EPA, coordinated by his office and comprising scientists from across the agency, is currently looking for opportunities to apply probabilistic methods. This forum studies scientific issues within the agency and advises policy on the basis of their findings. "We're also looking at a spectrum of other issues--for instance, how choices are made in deriving reference doses and concentrations, and how to do a better job of assessing cumulative exposure," he says.

The issue of cumulative exposure to chemical mixtures is described in the paper in some detail. The paper references a stakeholder comment submitted to the OMB that accuses the EPA of "assuming the toxicity of a chemical mixture is equal to the sum of the toxicity of each individual chemical, regardless of the toxicity type [or] competition or antagonism among chemicals." The EPA addresses this charge by noting that current guidance directs risk assessors to consider chemical interaction data "whenever possible." Even so, the staff paper recommends that agency scientists continue to aggressively "flesh out approaches to cumulative risk ... to produce the most scientifically rigorous evaluations that the state-of-the-science can accommodate."

Preuss says a point that became abundantly clear as the internal examination unfolded is that stakeholders are often unaware of technical progress at the agency. "Many of the comments allude to risk assessment practices that people believe we use that in fact we do not use," he says. "These discussions have gone on for many years, and meanwhile the methods and approaches here have become vastly more sophisticated. It's important that people who make these comments are aware of these advances. Frankly, I'm hopeful that with the work that comes out of this staff paper, a lot of these issues can be put to bed, and we can move away from these old discussions."

A Springboard for Change

Looking ahead, EPA officials are optimistic the staff paper will serve as a vehicle to open dialogue among staff, managers, and stakeholders. Dearfield highlights plans to hold workshops with the EPA's Science Advisory Board, in addition to industry and environmental groups, professional societies, and other outside parties. A request for comments on issues for further discussion, made in the Federal Register the day the staff paper was announced, is currently in place, with a deadline for submission of 23 June 2004. Meanwhile, risk assessment will go on as one of the EPA's most important functions--a paradigm through which all that is known about toxicity and human response can be funneled directly into regulatory decision making.

"This document could not have come at a better time," says Portier. "The pace of science has increased over the last ten years, with molecular biology and electronics serving as the key catalysts. As the EPA advances through the next decade, it needs to assess how these technologies will be used to improve risk assessments. Knowing fairly and honestly what is done today will play a critical role in forming what can and will be done in the future."