Subjective science: environmental cost-benefit analysis.

Last February, in a legislative proposal to reduce power plant emissions, the U.S. Environmental Protection Agency (EPA) suggested that the economic value of saving the elderly from early death caused by air pollution was less than that of saving healthy adults. Two cost-benefit analyses for the rule, known as the Clear Skies initiative, were presented by the agency. A base analysis without the so-called "senior discount" yielded benefits of $93 billion. But an alternative analysis in which this age-adjusted factor was applied produced a much lower benefit of just $11 billion, barely twice the cost of the program. According to Reece Rushing, a policy analyst with the Washington, D.C.-based public interest group OMB Watch, health-protective regulations may not be adopted if benefits appear low relative to cost.

The backlash against the senior discount was swift and severe. Advocacy groups, religious organizations, and critics including former EPA administrator Carol M. Browner blasted the agency, charging that it had undervalued aging Americans in its benefits calculations.

Under fire, the EPA withdrew the alternative analysis in May. But while the senior discount may have been shelved for now, cost-benefit analysis for environmental rule making under the Bush administration remains controversial.

At the core of this issue is the growing influence of the White House office with responsibility for cost-benefit review: the Office of Information and Regulatory Affairs (OIRA), within the Office of Management and Budget (OMB). Traditionally, OIRA has had fairly minimal interactions with submitting agencies as they prepare cost-benefit analyses. But under its current administrator, John Graham, OIRA has become intimately involved in all aspects of the cost-benefit process.

Stacking the Deck

Although his technical abilities are highly regarded Graham's relationship with environmentalists is complicated by his industry associations. Prior to taking the reins at OIRA in July 2001, Graham directed the Harvard University Center for Risk Analysis, an organization that applies cost-benefit research to health and environmental decision making. Nearly 60% of the center's budget during Graham's tenure was supplied by corporate sponsors, including chemical companies such as Monsanto and Union Carbide.

Today, Graham is among the most powerful environmental officials in the country: all health and environmental rules valued at $100 million or more cross his desk before they go into force. During the eight years of the Clinton administration, OIRA sent 16 rules back to agencies for rewriting. Graham sent back 19 rules (not all of which were environmental) during his first year alone.

OIRA's growing participation in the cost-benefit process has drawn mixed reactions from EPA officials. Some welcome the infusion of new science they say OIRA brings to the process. For instance, Al McGartland, director of the EPA National Center for Environmental Economics, suggests the collaborations foster innovation on new methods and approaches. But other officials say OIRA has become excessively intrusive. Critics also suggest that OIRA's involvement in the cost-benefit process itself undermines the office's ability to perform an objective review of the outcome, going against the purpose of having OMB as an independent review body.

In a further divergence from past practices, OIRA has also begun to critique the science behind proposed rules in addition to the economics assumptions. Once staffed mainly by economists and policy analysts, OIRA now also employs a variety of health and environmental scientists, present at Graham's request. Among them are Margo Schwab, an epidemiologist previously with the Johns Hopkins Bloomberg School of Public Health, and Nancy Beck, a toxicologist drawn from the EPA National Center for Environmental Assessment. "OIRA's investment in science and engineering has permitted us to ask more informed and penetrating questions about regulatory proposals," Graham says. "It has also allowed us to communicate more effectively with agency specialists."

The addition of scientists to OIRA's staff is another source of controversy. Rushing says OIRA is overstepping its legislative mandate because health and environmental expertise should be concentrated in the agencies that draft legislation. "It's dear that OIRA wants to question agency science in the same way they want to question agency cost-benefit analysis," he says.

Alan Krupnick, a senior fellow with the Washington, D.C.-based research organization Resources for the Future (RFF), disagrees. "I think it's a great idea," he says. "Why should OIRA's review of EPA studies be limited to the quality of their economic analyses? OIRA could help with the science as well. Looking at this logically rather than bureaucratically, it seems like a good thing to do."

Guidelines and Measures

Many of OIRA's proposed methods changes are described in draft OMB guidelines titled "2003 Report to Congress on the Costs and Benefits of Federal Regulations," published in the 3 February 2003 issue of the Federal Register. The guidelines, currently under review, are expected to be finalized in the fall. A key goal of the draft guidelines is to expand on the economic methods used to valuate the health benefits of proposed rules.

The EPA currently quantifies health benefits using a metric called the value of a statistical life (VSL). The VSL is derived mainly from two types of studies: surveys that inquire how much individuals are willing to pay for a reduction in the risk of early death, and studies on the difference in wages between high- and low-risk jobs among workers of the same age and health status--that is, the compensatory "wage premium" paid to high-risk workers.

Scientists use the resultant data in calculations that determine the VSL for a given population. The EPA's calculated VSL, based on 26 separate studies (mainly wage studies), is $6.1 million. Specifically, this figure is an estimate of the amount that a population is willing to pay to save a single hypothetical person (i.e., the statistical life) from dying prematurely. To quantify the benefits of a proposed rule, scientists multiply the VSL by the number of people that the rule is expected to save.

Critics, including Graham, suggest that, as it is currently used, the VSL's limitations distort environmental benefits calculations. For instance, because it is based largely on labor market studies that address the risk of accidental death, the VSL may be inappropriate for valuating the lives of nonoccupational populations, including those who are most vulnerable to environmental pollution, such as the elderly, says Bryan Hubbell, an economist in the EPA Office of Air Quality Planning and Standards. The age-adjusted factor used in the Clear Skies analysis was an attempt to correct for this discrepancy, Hubbell says.

At OIRA's urging, the EPA revised its VSL using new willingness-to-pay data collected from surveys in the United Kingdom and Canada. But these studies found that seniors would pay approximately 35% less for risk reductions than would healthy adults. Therefore, the VSL applied to elderly populations under the Clear Skies initiative was commensurately lower ($3.7 million) than the standard VSL of $6.1 million applied by the EPA to the rest of the population.

Environmental groups seized on this analysis as evidence that OIRA and Graham were making subjective decisions about the living value of one group versus another--a charge OIRA officials vehemently deny. In the midst of this uproar, EPA and OIRA officials quietly revisited the Canadian and U.K. data and found them to be unacceptable in quality. Furthermore, a willingness-to-pay analysis of U.S. seniors performed by RFF could not replicate the age-adjusted factor derived from the U.K. data. And so in a memorandum to the President's Management Council dated 30 May 2003, Graham formally advised the EPA and other agencies to "discontinue use of this factor as an adjustment to the economic value of a statistical life."

In the meantime, critics are increasingly claiming the VSL is a flawed tool. In general, the public tends to be uncomfortable with the whole notion of assigning dollar values to life and health. And VSLs vary widely depending upon the studies from which they are derived. The Food and Drug Administration's VSL, for instance, is $5 million. RFF has conducted willingness-to-pay surveys in the United States and Canada that identify VSLs ranging from $1 million to $3 million. Are any of these values more valid than the others? Experts concede this is a difficult question to answer. "OMB is concerned that agencies are using different VSL figures without clear justifications for the variability," Graham says. "[Therefore], OMB is encouraging agencies to discuss these issues and find the most technically reasonable figures."

Cost-Effectiveness Analysis 

OMB's draft guidelines propose an alternative approach that foregoes the economic valuation of life altogether. This approach, which has a long history in public health, is called cost-effectiveness analysis. In cost-effectiveness analysis, scientists quantify the benefits of proposed legislation not by the economic value of the lives it saves but by the life years it saves in the population as a whole. Public health experts call this the "effect on life expectancy." Graham explains, "A rule that adds an expected ten years of life for at-risk individuals may deserve greater priority than a rule that adds only one year of life."

According to Hubbell, scientists calculate life years saved by first determining the risk of mortality from a given environmental threat according to both age and health status, then estimating the resultant life years lost from the population. The inverse of that figure represents the life years saved if the threat is eliminated. For instance, assuming a natural life span of 75 years, a reduction in air pollution that saves a 50-year-old and a 70-year-old can be said to have saved 30 life years (25 years for the 50-year-old and 5 years for the 70-year-old). This approach allows regulators to rank regulatory options that protect public health according to the life years the options predict will be saved.

Graham emphasizes that the draft guidelines don't call for abandonment of the VSL, a deeply entrenched value still favored by many stakeholders. Rather, they propose that it be combined with cost-effectiveness analyses so that agency administrators can compare policy options and approve those that give the most bang for the buck in terms of health protections.

According to Hubbell, the life years approach has a key drawback, however: it assumes that individuals will value life years in the same way, regardless of the status of their health or their remaining life expectancy. Economists have sought to account for these quality-of-life parameters with a variety of controversial techniques.

Although not proposed in the current draft guidelines, Graham has suggested that quality-of-life measures can be incorporated into agency cost-benefit evaluations. Specifically, remaining life years can be weighted according to subjective valuations regarding their overall quality, he says. The key measures in these analyses are quality-adjusted life years (QALYs)--the dominant metric in the public health economics field, says McGartland--which are derived from surveys of how different age and disease groups rate the quality of their existence. Zero represents death, and I describes a state of perfect health. A QALY of 0.5 might therefore be applied to the life year of someone with chronic lung disease. So a policy that saves 10,000 people with such an illness would be predicted to yield 5,000 QALYs.

Most stakeholders tend to agree that cost-effectiveness approaches can be useful. But not surprisingly, the QALY approach has its own numerous detractors. For example, Wesley Warren, a senior fellow for environmental economics at the Natural Resources Defense Council, claims the approach is nonviable, slamming it as "voodoo economics." At present, OIRA is working with the Institute of Medicine to examine the advantages of QALYs and other alternative measures, Graham says.

In the meantime, it is clear that cost-benefit approaches as they apply to environmental rule making are evolving and becoming more sophisticated. The extent to which they actually lead to better decisions that protect public health is likely to be debated extensively throughout the remainder of the Bush presidency and beyond. "Ultimately, OIRA under Graham is shining a bright light on procedures that have previously been taken for granted, and that's a good thing," says Krupnick. "It's good to get better science into the rule-making analysis. Some see a problem in that these newer approaches are pointing us in a downward direction in terms of benefits estimations. However, what is needed is a debate over appropriate procedures for ranking policies that improve health."