Printer Friendly

Choosing quality of care measures based on the expected impact of improved care on health.

Consumers, payers, and policymakers are demanding to know more about the quality of the services they are purchasing or might purchase. The information provided, however, is often driven by data availability rather than by epidemologic and clinical considerations. In this article, we present an approach for selecting topics for measuring technical quality of care, based on the expected impact on health of improved quality. This approach employs data or estimates on disease burden, efficacy of available treatments, and the current quality of care being provided. We use this model to select measures that could be used to measure the quality of care in health plans, but the proposed framework could also be used to select quality of care measures for other purposes or in other contexts (for example, to select measures for hospitals). Given the limited resources available for quality assessment and the policy consequences of better information on provider quality, priorities for assessment efforts should focus on those areas where better quality translates into improved health.

Coinciding with policy initiatives to control health care costs through competition, policymakers have been increasing their attention in the last decade to the need to collect more information on the quality of care. Such information on quality could be used to regulate providers, or the information could be used by providers themselves to monitor and improve quality of care (Berwick 1989). Alternatively, the information could be used to stimulate competition by better informing consumers and major purchasers of health care (Office of Technology Assessment 1988; Davies and Ware 1988). This growing interest in measuring quality of care has spawned discussions about identifying the data that should be collected (Office of Technology Assessment 1988).

A variety of approaches to measuring quality are available. One can use structural measures that describe the health care settings and providers of medical care, but these structural measures may not be sufficiently valid to detect clinically significant differences in quality. An alternative approach might be to measure patient satisfaction. Clearly, patient satisfaction is an important outcome of care; however, knowing about satisfaction alone yields only limited information about the appropriateness of resource-intensive technical aspects of care (the physician visits, diagnostic tests, and treatments).

For information on these technical aspects of care, much attention has focused on quality of care indicators for which data are easily available. Thus, interest has concentrated on hospital mortality, read-mission, and procedure volume as potential indicators of quality of care. While data availability or the feasibility of data collection are important, these should not be the central considerations in selecting indicators of quality. The selection of quality of care measures for public policy purposes should also be based on epidemiologic, medical, and health care delivery considerations.

Ideally, the choice of measures should be based on knowledge of disease burden, efficacy of available treatments, the current type and amount of care being provided, the expected health impact of improvement in care, and the costs associated with making these improvements. Based on this information, one would then select those quality of care measures where improvement in care was associated with the greatest expected health impact for a given cost. Many of these considerations have been used in previous attempts to select topics for measuring the technical quality of care (Brook, Davies-Avery, Greenfield, et al. 1977). This earlier work is exemplified by Williamson's pioneering work on selecting priorities for quality assurance based on the expected health impairment associated with a medical condition, the preventability of this impairment through medical care, and the extent to which this prevention is not achieved (Williamson 1978; Williamson, Alexander, and Miller 1968).

This article elaborates on such earlier work by proposing an approach that more explicitly specifies and quantifies the parameters and assumptions made in estimating the expected impact of improvements in care. We apply our approach to problems related to primary, secondary, and tertiary prevention, and we extend the approach to deal with overuse or inappropriate use of medical care. The methods also consider the expected impact of varying degrees of improved quality of care, and the approach can be extended to estimate the expected impact of compliance with specific quality of care indicators or criteria.

The methods we describe can be applied in a variety of contexts. For illustration, we describe this process in the context of selecting quality of care measures that could be used in HMO health plans either for routine quality assurance activities, for continuous quality improvement (Berwick 1989), or for meeting the information needs of major purchasers. We use our estimates of the expected impact of improved care as our primary consideration in selecting topics for measuring quality. Then considering the context in which the measures will be used, we base a final choice of quality of care measures on other important considerations, such as cost effectiveness, relevance to the health care delivery setting, and feasibility.


The quality of medical care has been defined in many ways. One definition considers quality of care as the difference "between efficacy and effectiveness that can be attributed to care providers" (Brook and Lohr 1985, 711). Efficacy here is defined as the benefits to patients achievable when medical technology is applied to a given clinical problem under ideal circumstances, and effectiveness is defined as the benefits achieved under more ordinary or actual circumstances.

Considering this definition, interest in assessing the quality of care is sensible only when interventions of known efficacy exist for treating a given medical problem. If medical care for a given problem is known to be efficacious, reducing missed opportunities to apply the intervention (and/or improving the execution of the intervention) should lead to improvements in health. The missed opportunities to apply efficacious interventions are the major focus of this article, but other types of quality problems also merit consideration. If an intervention has no effects or potentially harmful effects, it would be more efficacious to desist from the intervention. Improved health would result from reducing these inappropriate interventions. Although reduced use of inappropriate care is not the major focus of this study, we illustrate the applicability of our approach to this situation as well.

By these arguments, selection of clinical problems as topics for measuring quality of care should focus not only on the prevalence and health impact of a condition, but also on the efficacy (or lack thereof) of available treatments. Such information could be used to estimate the expected impact of improved care.

To make such estimates, we began by identifying the major causes of mortality and morbidity in the United States. For each of the identified conditions, we asked a series of questions. Are there efficacious interventions for primary, secondary, or tertiary prevention of these causes of mortality and morbidity? If so, is the full efficacy of these interventions being realized? If not, what is the potential or expected impact of improved care for these conditions?

Using data from Vital Statistics and the Health Interview Survey (Collins 1986), we identified the major sources of mortality and morbidity limited activity days) for five age and gender groups: birth-17, 18-44 (by gender), 45-64, and over 65 (Table 1). These data sources were chosen for their national scope and comprehensive detailed listing of medical problems. But the use of these surveys also had disadvantages: for instance, because of reliance on physician- or self-reporting, these data sources underreport or unreliably classify a number of conditions (such as mental health problems, sexually transmitted diseases, dementia, incontinence, and osteoporosis). Additionally, we recognize that the impact of some important conditions (for example, sensory impairment) was reduced by the disaggregation of diseases in the surveys. With these considerations in mind, we added a number of additional conditions and health issues that were not originally identified (Table 1) in consultation with a panel of clinicians (authors J. B., K. C., J. L., B. P., H. S., W. W., A. W.).
Table 1: Major Causes of Mortality and Morbidity
Identified from Vital Statistics and Household Interview Survey(*)
 Infant mortality and related conditions (< 1)
 Otitis media (1-17)
 Asthma (1-17, 18-44)
 Accidents and injuries (1-17, 18-44, 45-64)
 Suicide (18-44)
 Acute respiratory conditions, including influenza (all ages)
 Breast cancer (18-44, 45-64)
 Back conditions (18-44, 45-64)
 Coronary artery disease (18-44, 45-64, 65+)
 Arthritis (18-44, 45-64, 65+)
 Chronic bronchitis and emphysema (45-64, 65+)
 Colorectal cancer (45-64, 65+)
 Lung cancer (45-64, 65+)
 Stroke and cerebrovascular disease (45-64, 65+)
 Diabetes (45-64, 65+)
 Pneumonia (65+)
Additional Conditions
 Vaccine-preventable childhood infectious diseases
 Mental health problems
 Sexually transmitted diseases
 Dementia and incontinence
 Osteoporosis and hip fractures
 Sensory impairment
(*) Age categories in parentheses.

For each of these conditions, we reviewed the medical literature to determine whether medical care or health-related behaviors were known to mitigate disease-specific outcomes. We first reviewed what was known about causation of each condition, what risk factors were associated with its development, and what was known about interventions for primary prevention -- that is, the promotion of health by personal and communitywide efforts (Last 1983). We then considered the potential impact of secondary and tertiary prevention by reviewing interventions known to reduce mortality or morbidity among those who have already developed the condition. Secondary prevention is the early detection and treatment of disease, and tertiary prevention is the reduction of impairment, disability, and suffering due to established illness (Last 1983).

If the literature indicated that effective interventions existed for prevention, we estimated the impact of such interventions annually at the national level in the United States. Thus, for specified interventions, we estimated expected reductions in deaths, bed-days, or other adverse outcomes. Depending on the availability of data, similar estimates could be made at other population-based levels such a states, regions, or a given health plan as appropriate. These estimates were intended to illustrate the potential effect of individual medical interventions. Where more than one intervention was discussed for a given condition, the impact estimate was made independent of that for any other intervention(s) considered. Thus, attempts to combine impact estimates made for individual interventions may overestimate the magnitude of the total impact.

Estimates of the potential impact of specific interventions were made using adaptations of previously described methods (Morgenstern and Bursic 1982; Ouellet et al. 1979; Browner 1986). For each intervention, we calculate an impact fraction (IF) or the expected proportional reduction in the occurrence of a disease or outcome following the intervention in a particular target population. For example, if the risk of an outcome event in the target population is reduced from 10 percent before an intervention to 8 percent after the intervention, the IF would be (0.10 - 0.08)/0.10 or 20 percent. (Note that the absolute reduction in risk is 0.10 - 0.08 or 2 percent.) From the vantage point of the target population, the IF summarizes the impact of a specific intervention given (a) how much of the population is exposed to a risk factor, (b) the risk associated with exposure, (c) the estimated incremental participation rate in the intervention, and (d) the efficacy of the intervention. If the intervention is designed to modify a causal risk factor (i.e., a factor that increases outcome risk), IF is estimated as

IF = f(P)(RR)(1 - RR')/f(RR - ) + 1 (1) where

f = the proportion of the population currently exposed to

the risk factor;

P = the proportion of the exposed population that is

expected to receive the intervention, that is, the participation


RR = the relative risk for a given outcome comparing the

exposed to the nonexposed in the target population,

that is, the risk in the exposed divided by the risk in

the unexposed; and

RR' = the relative risk (for the forementioned outcome)
 comparing exposed individuals participating in the
 intervention to exposed individuals not participating
 in the intervention, that is, the risk among exposed
 individuals participating in the intervention divided
 by the risk among exposed individuals not participating
 the intervention.(1)

If there is more than one exposure category (e.g., different categories of elevated serum cholesterol), we use an extension of this equation to estimate the impact of interventions that deal with exposure to more than one (k + 1) exposure category (indicated by the subscript 1 where i = 0 is the reference category) (Browner 1986):

(2) [Mathematical Expression Omitted] If the risk factor under consideration is protective (i.e., RR < 1), equation 1 or 2 is used with the exposure categories reversed so that f is the proportion of the population currently unexposed and P is the participation rate among the currently unexposed.

Where the intervention involves the direct application of a protective risk factor (e.g., a vaccine), we assume that the intervention's benefit for newly treated patients is equal to the benefit for previously treated patients. In this special case, Equation 1 reduces to

IF = f(P)(1 i RR')/f(1 - RR') + RR' (3) where f is the proportion of the population currently unexposed (or untreated) and P is the participation rate among the currently unexposed or untreated.

Apart from the validity of the four specified parameters, we make two additional assumptions. We assume that participation in the intervention program is unrelated to outcome risk, and we assume that the only risk factor modified by the intervention is the one exposure under consideration. We make these assumptions for practical reasons. We have no information by which to estimate how participation might vary by risk category in most cases, and simultaneous consideration of multiple risk factors is limited by data availability in many cases. When these assumptions are not tenable, however, the multiple exposure category method can be modified to address these considerations. For example, the participation rate can be varied by risk category. Similarly, exposure to more than one risk factor can be addressed by separate exposure categories for combinations of risk factors.

The estimated number of outcome events (per unit time) expected to be prevented by the intervention is the product of IF and the number of outcome events (e.g., incident cases or deaths) from that condition in the population. If only one subgroup of individuals (e.g., diabetics) with the given condition comprises candidates for the intervention, the expected number of outcome events prevented as a result is the product of the IF and the number of outcome events within that subgroup (e. g., the number of strokes in patients with diabetes).

Estimates of exposure prevalences (f), risk-factor effects (RR), and intervention effects (RR') were obtained from the literature. If no estimate of these parameters could be obtained (primarily in the case of exposure prevalence and intervention effect), we made preliminary assumptions and tested the sensitivity of our results to alternative assumptions.

After considering the expected impact of identified preventive strategies, we based decisions on the desirability of measuring quality of care on three issues that were context specific (i.e., in our particular case, specific to measuring quality of care in HMOs). First, if information was available in the literature, we considered whether or not the relevant improvements in medical care were known to be cost effective. Second, we considered whether the relevant improvements in quality of care could be influenced by the providers we were evaluating. Third, we considered the availability and feasibility of collecting quality of care information.

This process was undertaken for all the conditions identified in Table 1. We present our overall conclusions and detailed discussions on four conditions that illustrate key issues.


Based on the expected impact of improved care and on context-specific (i.e., measuring quality of care in HMOs) issues, we selected a number of conditions and topics for evaluating the quality of care within a group of HMO health plans (Table 2). We estimated that improved care for the conditions on this list could have a substantial impact on health, based on the prevalence of risk factors, the efficacy of relevant interventions, the level of current quality of care, and the incidence of adverse events (Table 3). A report with detailed discussions of all conditions is available from the authors (Siu, McGlynn, Beers, et al. 1992). The four conditions discussed now in detail illustrate the considerations that went into selecting this list of measures. [TABULAR DATA 2 and 3 OMITTED]


Infant mortality is one of the traditional indicators of the health of the population. In the last decade, however, attention has focused on the problem of low birthweight (LBW) (less than 2,500 grams) infants, one of the major determinants of infant and particularly neonatal mortality and morbidity. The risk of mortality increases with decreasing birth weight so that LBW babies account for two-thirds of neonatal deaths (Behrman 1985). Surviving LBW infants are also at increased risk of neurodevelopmental handicaps, lower respiratory tract infections, learning disorders, behavior problems, and iatrogenic complications of neonatal intensive care unit interventions (McCormick 1985).

Recent interest in reducing infant mortality is directed at the primary prevention of LBW births. Some maternal risk factors (such as race, education, and socioeconomic status) for LBW are not amenable to medical intervention. On the other hand, birth spacing and teenage pregnancy are examples of risk factors that might be modified by preconception planning. Further, the increased risks related to smoking, drinking, and failure to receive adequate prenatal care are thought be highly modifiable (Behrman 1985). Indeed, the early initiation of prenatal care is considered to be the most cost-effective strategy for improving pregnancy outcomes (Joyce, Corman, and Groosman 1988).

The potential impact of interventions directed at primary prevention is likely to be large. Virtually unchanged since 1980, an estimated 20.6 percent of white and 38.9 percent of black women in the United States (1987) fall to receive prenatal care in the first trimester (National

Center for Health Statistics 1990b). Based on a multivariate analysis of national data, timely and appropriate prenatal care is estimated to reduce the risk of LBW by 15 percent (i.e., RR' = 0.85) among white and by 12 percent among black women (Behrman 1985). Thus, we estimate that reducing the percentage of women failing to get prenatal care in the first trimester to 5 percent (P = .156/.206 = .757 for white women) could result in 2.7 percent(2) or 4,521 fewer LBW births among white women, and 4.4 percent or 3,580 fewer LBW births among black women annually. A more modest goal of 10 percent failure in the first trimester would result in 3,050 fewer white and 3,050 fewer black LBW births.

Methods for secondary prevention are also available. These methods include specialized treatment of high-risk infants in neonatal intensive care units and the establishment of specialized centers for the delivery of high-risk infants. The availability of such units is thought to be responsible for much of the reduction in neonatal mortality since 1960 (McCormick 1985). Most experts believe, however, that the major gains in secondary prevention have been achieved already, and that efforts should now be directed to primary prevention of LBW (Behrman 1985).

Thus, in the prevention of infant mortality, quality of care measurement is best directed at the primary prevention of low birth weight and related problems, because the expected impact of improved neonatal care is likely to be small. Specifically, the proposed measures should focus on the timeliness, frequency, and content of prenatal care. Alternatively, one could measure outcomes such as rates of LBW births. Because these outcomes are known to be strongly affected by socioeconomic and other variables beyond the control of individual providers, evaluations of such outcomes need to be adjusted for complexity of the pregnancy (e.g., multiparity), maternal age, comorbidity, socioeconomic status, and ethnicity.


Although mortality from coronary artery disease has declined in the United States in the last two decades, coronary disease and related diagnoses remain one of the major causes of mortality and morbidity among adults. Primary prevention of ischemic heart disease is based on modification of several well-established risk factors, including hypercholesterolemia, cigarette smoking, hypertension, obesity, and physical inactivity. Indeed, modification of risk factors (primarily changes in serum cholesterol and smoking) may have accounted for more than half of the decline in mortality from 1968-1976 (Goldman and Cook 1984). Nevertheless, the potential for further improving health through risk factor modification remains large. An estimated 26.5 percent of American adults smoke (Centers for Disease Control 1987). Over one-third of the smokers seen in university internal medicine practices report never having been asked to stop smoking (Kosecoff, Fink, Brook, et al. 1985), and physicians (as recently as the mid-1980s) dismiss hypercholesterolemia as a problem in an estimated 71 percent of patients with cholesterol levels ranging from 221-260 mg/dl (Wynder, Field, and Haley 1986).

The potential impact of improved primary prevention is substantial. Consider just one possible prevention strategy--reducing cholesterol by 10 percent for adults with values in the top quintile. Although there has been considerable controversy over the recommendations for broad-based cholesterol screening and treatment made by a recent national advisory body, screening and attempts to lower cholesterol in individuals with very high cholesterol is much less controversial (Olson 1989). Men in the top cholesterol quintile have a 3.4-fold relative risk of coronary death (i.e., RR = 3.4 and f = .2) compared to the lowest quintile (Martin, Hulley, Browner, et al. 1986), and reducing cholesterol by 10 percent is achievable and proportionally reduces the incidence of coronary heart disease by 20 percent (i. e., RR' = 0.80) (Lipid Research Clinics Program 1984; Lavie et al. 1988). If these results generalize to women as well, reducing cholesterol by 10 percent in half of high-risk individuals (i.e., P = .5) would reduce coronary mortality (510,000 cases in 1988 [National Center for Health Statistics 1990a]) by 3.5 percent(3) or 17,850 cases annually (9,300 of which are men). Alternatively, getting just 10 percent of individuals in the top quintile to lower their cholesterol by 10 percent would also have important health effects, reducing coronary mortality by 0.7 percent or 3,570 cases (1,850 of which would be men).

Progress has also been made in secondary and tertiary prevention where the potential for improvement in health exists at several levels. Lowering plasma cholesterol can slow the progression of established coronary artery disease (Blankenhorn, Nessim, Johnson, et al. 1987). Effective medications are available for the treatment of chronic (Strauss and Parisi 1988) and unstable angina (Theroux, Ouimet, McCans, et al. 1988). Improved prehospital care for acute myocardial infarctions has been shown to reduce mortality (Eisenberg, Bergner, and Hallstrom 1979, Crampton, Aldrich, Gascho, et al. 1975). Major advances have been made in both the acute (Wilcox, von der Lippe, Olsson, et al. 1988; ISIS Steering Committee 1987; ISIS-2 1988; GISSI 1986, 1987; Yusuf, Peto, Lewis, et al. 1985; Goldman, Sia, Cook, et al. 1988) and postacute (Oldridge et al. 1988) treatment of myocardial infarction. Furthermore, a study suggests that some deaths among patients hospitalized for acute myocardial infarction may be preventable, perhaps by the more adequate treatment of postinfarction angina (Dubois and Brook 1988).

Because of rapid technological changes, it is difficult to estimate the level of current use of some beneficial services. Nevertheless, the potential impact of secondary and tertiary prevention is also large. To illustrate, we consider the potential impact of two different treatments following myocardial infarction: thrombolytic therapy and betablockade. Studies have shown that thrombolytic therapy reduces mortality from hospitalized myocardial infarction (RR' = 0.76) (Yusuf, Wittes, and Friedman 1988). Thrombolytic therapy is perceived to be one of the more rapidly diffusing technologies ever introduced; however, it is probably still being underused, especially in smaller hospitals and in patients treated by generalist physicians (Tate and Dehmer 1989; Hlatky, Cotugno, O'Connor, et al. 1988). Inpatient mortality would otherwise be 12 percent (ISIS 1987) among the 15 percent of patients who are eligible for such treatment (Lee, Weisberg, Brand, et al. 1989). If we assume only 40 percent of eligible patients currently receive thrombolytic therapy (i.e.,f = .6), we estimate that increasing use of thrombolytic treatment to 75 percent of eligible patients (i.e., P = (.60 - .25)/.60 = .58) would reduce mortality by 9.3 percent(4) among eligible patients and reduce the annual number of deaths by 1,172. The expected impact of treating 75 percent of eligible patients with thrombolytic therapy is slightly larger or smaller depending on the level of current use (Table 4).


Similarly, treatment with beta-blockers after myocardial infarction proportionally reduces the risk of death (RR' = 0.78) (Yusuf, Wittes, and Friedman 1988). Treatment with beta-blockers is contraindicated in some patients, and other patients will experience mild side effects from these drugs; however, approximately 50 percent of myocardial infarction patients are eligible for treatment (ISIS-1 1986), and annual mortality is estimated to be 8 percent among these patients (Yusuf, Peto, Lewis, et al. 1985). If we assume that only 60 percent (i.e., f = .4) of eligible patients are currently treated,(5) we estimate that 851 deaths(6) could be averted annually by increasing the percentage of patients treated to 75 percent of eligibles [i.e., P = (.4 - .25)/.4 = .375]. The expected impact of improved use of beta-blockade is shown in Table 4 for different levels of current use.

Thus, because the burden of coronary artery disease is great and many clinical studies are available, we are able to estimate that the expected impact of improved care is large in primary prevention and in secondary and tertiary prevention; the choice of topics for this condition depends largely on the context for these quality assessment efforts. If one is interested in studying hospitals, the estimates just cited indicate that the potential impact of improved hospital care is sufficiently great to warrant such efforts. If, instead, one is interested in health plans, the choice is wider and one can focus on various aspects of either or both primary prevention and hospital care.


Breast cancer is one of the leading causes of death among women in the United States, with nearly 40,000 deaths occurring yearly. Unfortunately, primary prevention is not feasible. Although several risk factors for developing breast cancer have been identified, most of the important risk factors (previous breast cancer, family history of breast cancer, nulliparity and late first pregnancy) are not modifiable.

On the other hand, early detection of breast cancer is feasible and beneficial. Evidence on the efficacy of breast self-examination is far from conclusive, but studies have shown that women who perform breast self-examination present with cancers at earlier stages (Hill et al. 1988) and have higher survival rates (Huguley et al. 1988). The efficacy of mammography in specific groups of women has been more convincingly demonstrated in a number of studies (Tabar, Fagerberg, Gad, et al. 1985; Verbeek, Hendriks, Holland, et al. 1984; Andersson, Aspergren, Janzon, et al. 1988; UK Trial of Early Detection of Breast Cancer Group 1988). The most widely cited work was a randomized trial at the Health Insurance Plan of New York (HIP) that showed that mammography reduced breast cancer mortality by 30 percent in women over 50 (Shapiro 1977). Although mammography may be effective in women under 50, the benefits are smaller and the decision to screen is likely to be based much more on a patient's risk factors and preferences (Eddy and Hasselbald 1988).

With respect to tertiary prevention, the primary treatment of early breast cancer is surgical (axillary dissection with either mastectomy or local excision with radiation) (Veronesi, Saccozzi, Del Vecchio, et al. 1981; Fisher, Bauer, Margolese, et al. 1985). Although these surgical alternatives may have different effects on psychosocial function (Steinberg, Juliano, and Wise 1985), mortality is equivalent for either approach. After surgery, adjuvant therapy is useful in some patients. Available evidence indicates that adjuvant chemotherapy is effective in prolonging survival of premenopausal women with cancer extending to the axillary lymph nodes (Stockdale 1988; Bonadonna and Valagussa 1987), but while a number of recent trials have indicated that adjuvant therapy for node-negative cancer reduces recurrence, no impact has yet been found on overall survival (Ludwig Breast Cancer Group 1989; Fisher et al. 1989a; Fisher, Constantino, Redmond, et al. 1989; Mansour, Gray, Shatila, et al. 1989).

Because of ongoing advances in adjuvant chemotherapy, it is difficult at this time to estimate the impact of improved treatment of breast cancer. If we consider the one subgroup where adjuvant therapy is most indicated, however, the impact of improved care can be shown to be relatively small on a national basis. Adjuvant chemotherapy proportionally reduces mortality by 24 percent (i.e., RR' = 0.76) in premenopausal women with involved axillary nodes (Bonadonna and Valagussa 1987). Increasing the proportion of these women getting appropriate adjuvant chemotherapy by 20 percent would reduce annual mortality from breast cancer by only 5.4 percent(7) among women who presented with positive nodes, or by about 100 cases nationally (Table 5).
Table 5: The Expected Impact of a 20 Percent Increase in
Adjuvant Chemotherapy Use in Premenopausal Women with
Breast Cancer Extending to the Axilla
 A([dagger]) B C
Assumed percent currently 30 50 70
receiving adjuvant treatment
Impact fraction (%) 5.2 5.4 5.8
Prevented number of deaths 99 103 111
expected per year(*)
(*) There are 3,826 breast cancer deaths among women under 45 (National Center f
Health Statistics 1990a). We assume that 50 percent of these women present with
positive nodes, because 50 percent of women who do not do breast self-examinatio
present with positive nodes (Hill et al. 1988).
([dagger]) Similar results are obtained for the use of adjuvant therapy in node-
negative cancer.
Extrapolating from National Surgical Adjuvant Breast Project (NSABP) trial resul
(Fisher, Redmond, Dimitrov, et al. 1989; Fisher, Costantino, Redmond, et al. 198
increasing the proportion of women receiving adjuvant chemotherapy from 30 to 50
percent would prevent 85 recurrences among premenopausal women and 67
recurrences among postmenopausal women annually.

By contrast, the potential impact of secondary prevention through early detection may be substantial on a national basis. Currently, only 50 percent of women seeing physicians report having been taught how to perform self-examination (Brook, Fink, Kosecoff, et al. 1987). Only 20 percent of women over 50 are screened regularly with mammography (i.e., f = .8) (Hayward et al. 1988), and increasing screening levels to over 50 percent is feasible (Nattinger, Panzer, and Janus 1989). Extrapolating from the HIP study (i.e., RR' = 0.7), we estimate that increasing to 50 percent the proportion of women over 50 who undergo screening mammography (i.e., P = (.8-.5)/.8 = .375) would reduce breast cancer mortality by 9.6 percent)(8) or 2,400 cases annually. Alternatively, increasing the proportion screened to 75 percent would reduce mortality by 17.6 percent or 4,400 cases.

At first glance, the estimates of impact might suggest focusing quality measurement on the early detection of breast cancer. Because mammography involves more women, however, increasing mammography rates can require greater resources than increasing the use of adjuvant chemotherapy in the relatively smaller number of women diagnosed with breast cancer. Indeed, recent studies have suggested that adjuvant chemotherapy in some cases may be more cost effective than screening mammography (Hillner and Smith 1991; Eddy 1989).

Thus, depending on the context and purpose of quality assessment, it might be reasonable to focus on either mammography or breast cancer treatment. A cancer hospital or referral center interested in monitoring quality might focus on the use of adjuvant therapy. On the other hand, a health plan with an enrollment of 100,000 would have only about 60 cases of breast cancer in a given year and only a small fraction of these cases would be candidates for adjuvant chemotherapy (at least for those indications where it has been proved beneficial). this case, the health plan might focus on mammography rates as a more routine quality assessment or continuous improvement activity. Meanwhile, the health plan could use a more ad hoc case-by-case peer review approach to monitoring quality in breast cancer treatment.


Annually, 138,000 new cases of colorectal cancer are diagnosed, and 60,000 deaths occur. Commensurate with its public health impact, a large number of studies have looked at the prevention, early screening, and treatment of colon cancer. Although several dietary risk factors (Zaridze 1983) for colorectal cancer have been identified, much of the evidence is inconclusive and the efficacy of diet modification has yet to be demonstrated. Thus, primary prevention is theoretically possible but not scientifically confirmed at this time.

Most efforts at secondary prevention of colorectal cancer have focused on screening and early detection of cancer or premalignant adenomas. Screening can be beneficial for both reducing disease incidence (by removal of premalignant lesions) and improving disease stage at presentation (and hence improving survival). Several screening strategies have been advocated; however, data on the efficacy of screening are inconclusive. Preliminary findings from trials of fecal occult-blood testing suggest that testing results in the diagnosis of a greater proportion of new cancers at an earlier stage (Winawer, Leidner, Miller, et al. 1977; Hardcastle, Thomas, Chamberlain, et al. 1989), but no results are available on mortality. The efficacy of screening sigmoidoscopy is suggested by the results of two large, nonrandomized studies (Gilbertsen and Nelms 1978; Crespi, Weissman, Gilbertsen, et al. 1984). In one study of 21,150 patients screened with sigmoidoscopy, Gilbertsen and Nelms (1978) found that the incidence of colorectal cancer was only 15 percent of what might have been anticipated in the absence of sigmoidoscopy (or an 85 percent reduction). However, this is probably an overestimate of the efficacy of screening: if one includes cases identified in the study's initial screening examination, the reduction in incidence is actually only 56 percent. Because of design and methodologic flaws, the true reduction in incidence is likely to be even smaller (Selby and Friedman 1989). Thus, estimates of the expected impact of improved screening for colorectal cancer would be highly speculative.

With respect to tertiary prevention, the primary treatment of colorectal cancer is surgical. Attempts to improve outcomes after surgery have focused on testing various combinations of adjuvant radiotherapy and chemotherapy. A review of trials of adjuvant treatment suggests that benefits, if any, are likely to be quite small (Buyse, Zeleniuch-Jacquotte, and Chalmers 1988). Very recently, however, one form of adjuvant chemotherapy has been shown to reduce mortality proportionally by 33 percent (i.e., RR' = 0.67) for patients with Stage C colorectal cancer (i.e., cancer extending to the local lymph nodes) at the time of surgery (Laurie, Moertel, Fleming, et al. 1989; Moertel, Fleming, Macdonald, et al. 1990).

The expected impact of the use of these adjuvant chemotherapy approaches is likely to be reasonably large. An estimated 21,000 patients have surgery for Stage C colorectal cancer annually (Moertel, Fleming, Macdonald, et al. 1990), and five-year survival for these patients ranges from 45-60 percent depending on the extent of lymph node involvement (Mayer et al. 1989). Hence, we estimate that patients initially diagnosed as Stage C account for approximately 10,000 of the 60,000 annual deaths from colorectal cancer. Administering appropriate adjuvant therapy to 50 percent of Stage C patients would reduce colorectal cancer deaths by 1,650 cases(9) annually, assuming that only a negligible number of patients were receiving this relatively newly described treatment until very recently. The estimated impact of alternative assumptions of improvement are shown in Table 6.
Table 6: The Expected Impact of Improved Care
for Colorectal Cancer
 A B C
Increased Use of Adjuvant Chemotherapy
 Assumed percent to be treated 30 50 70
 Impact fraction (%) 10.2 16.5 23.8
 Prevented number of cases 1,020 1,650 2,380
 expected per year

Thus, the expected impact of improved screening (by fecal occult blood testing or any other method) is very uncertain, and assessing quality of care relative to screening is not recommended at this time. On the other hand, if one is interested in measuring quality of care particularly as it relates to the diffusion of new therapies, assessing quality of care for newly diagnosed cases of Stage C colorectal cancer would be a reasonable approach.


On the subject of our final list of conditions (Table 2), a number of issues merit special comment. First, our illustrations thus far have focused on increasing the use of interventions that are underused, but the same framework could be extended to include overuse or performance of services for clinically inappropriate indications. For example, an estimated 32 percent of carotid endarterectomies are performed for inappropriate indications, and major postoperative complications either death within 30 days or stroke with residual deficit) occur in 10.8 percent of these inappropriate procedures (Winslow, Solomon, Chaissin, et al. 1988). The volume of carotid endarterectomies performed in the United States reached a peak of 107,000 procedures in 1985 (Pokras 1987); hence, we estimate that 3,700 (107,000 x 0.108 x 0.32) major postoperative complications occurred in conjunction with inappropriate procedures in 1985. In many of these inappropriate surgeries, the short-term risk of death or stroke without surgery is very much lower than the risk with surgery (Chambers and Norris 1984; Warlow 1984), and we conservatively assume that the risk of short-term complication reaches only as high as 5 percent (i.e., RR' = 0.05/ 0.108) with nonsurgical management, or about half the surgical complication rate. Receipt of appropriate nonsurgical management by half of the patients who would otherwise have had inappropriate surgery i.e., P = .5 and f = 1) would prevent 26.8 percent or 992 "postoperative" strokes or deaths annually. If the short-term nonsurgical complication rate is only 2 percent, instead, eliminating half of inappropriate surgeries would prevent 1,508 strokes or deaths. Because many of the parameters required for making similar estimates are not available in the literature for numerous procedures and conditions, we could not make formal calculations of expected impact in many cases. Nevertheless, because we wished to balance considerations of underuse and overuse in our work, we included topics and measures focusing on appropriate use of procedures on our list of proposed measurement areas (Table 2).

Second, a number of important health problems are not included in Table 2. In some cases, this exclusion is due to insufficient evidence on the efficacy of interventions to improve outcomes. For example, although suicide is one of the leading causes of death among young men and women, there is little evidence on the efficacy of methods to prevent suicide (Eddy, Wolpert, and Rosenberg 1987). Similarly, back pain is the most common cause of chronic physical limitation among adults aged 17 to 45 (HIS), but evidence on the efficacy of methods of primary prevention or treatment of back pain is insufficient (Deyo 1983).

Either because of feasibility or relevance, a few conditions were excluded from our final list even though the expected impact of improved care was potentially great. We did not include accidents and injuries because the health plans we wished to evaluate had limited or varying roles in the primary prevention of accidents and in prehospital care for victims of trauma. We did not include smoking cessation counseling to prevent lung cancer at this time, because of feasibility problems in ascertaining the occurrence of counseling by either chart review or patient recall. Finally, mental health problems and sexually transmitted diseases were not included because of problems (privacy concerns, among them) in identifying individuals with these conditions in a population.


There is growing interest in holding providers more accountable for the quality of care they deliver. In this article, we propose an approach for selecting topics for measuring technical aspects of quality, and we apply this approach to selecting quality of care measures for possible use in HMO health plans. Based on epidemiologic considerations, the process we propose can be used to narrow the conditions and topics being considered. Then, depending on the goals of the quality measurement process, the population being measured, and the degree of feasibility, the list of possible topics can be further narrowed. One needs to consider whether the providers being evaluated have the power to provide the relevant interventions that might be expected to improve outcomes. For example, if an organization contracts with another group for inpatient services, it may not be appropriate to focus on topics related to inpatient care. On the other hand, if that organization has the ability to influence future contracting decisions, it may be more desirable to select topics dealing with inpatient care.

Similarly, one needs to consider the cost effectiveness of the proposed improvements in care. Our work is limited in that we could not select those medical conditions where improvement in care has the greatest expected impact for a given cost. Unfortunately, this is difficult to do for two reasons. First, most clinical studies fail to use or give enough information to enable construction of a common measure of effectiveness (e.g., quality-adjusted life years) so that different types of outcomes can be evaluated across a variety of diseases and treatments. We attempted in this project to use a common unit of benefit based on the Index of Well-Being (Kaplan and Anderson 1988), but we were not successful in most cases because of insufficient information in the clinical literature; sufficiently detailed information on symptoms frequently was unavailable. Rather than reporting results only for the most commonly reported outcome (i.e., mortality), we elected to report on a variety of disease-specific outcomes because mortality was not very relevant for a number of medical problems. Although different types of outcomes are admittedly sometimes difficult to compare, the outcome estimates are explicit and available for others to consider, weigh, and discuss.

In addition to difficulty in obtaining common units of effectiveness, data on the costs of treatment are rarely included in clinical trials. Although it is possible to perform cost-effectiveness analyses for many of these interventions using other data sources, this is an unreasonable undertaking for an activity encompassing a large number of medical diagnoses and conditions. Because of these problems, we relied on published studies of the cost effectiveness of various interventions as a guide to selecting conditions. To deal with cost effectiveness more explicitly we propose, as part of collecting quality of care data, that information be collected on the costs of the care provided. This information can then be used retrospectively to estimate the cost effectiveness of incremental improvements in quality of care.

The process we advocate in this article is iterative. Estimates and the appropriateness of quality measurement topics need to be reassessed as information becomes available on the cost effectiveness of incremental improvements in quality. As new clinical studies become available (as in the colorectal cancer example), estimates must be revised. Indeed, our proposed approach provides a method and context for estimating the public health impact of newly reported therapies. For example, one can easily compare the relative expected impact of the newly advocated adjuvant chemotherapies for breast versus colon cancer. Estimates and choices need to be revised as new technologies and therapies are developed, but also as new information becomes available on established therapies. Even though we believe sufficiently strong evidence is available, there are those who question the efficacy of coronary disease prevention (McCormick and Skrabanek 1988), early detection of breast cancer (Warren and Skrabanek 1989), and aggressive diagnosis and treatment of hypertension O'Brien and O'Malley 1988; Cruickshank, Thorp, and Zacharias 1987). Thus, the process for selecting quality of care measures must be as current as the clinical evidence on which it is based and as responsive as the providers whose care is being evaluated.

Finally, although we have illustrated the use of this epidemiologic model in choosing topics for quality measurement, the same methods could also be extended to choose the specific criteria and indicators used to assess technical quality. For example, one could estimate the expected effect of a two-hour delay by a provider in administering thrombolytic therapy, if one were considering the use of this issue as a specific quality of care criterion. Evidence from clinical studies to support efficacy estimates may be more difficult to find for these more detailed criteria, and subjective estimates (Robins et al. 1985) may need to be substituted. However, assumptions of efficacy are implicit whenever such specific criteria are selected, and the proposed approach only stipulates that these estimates be made explicit to aid in the discussion of the merits of specific quality of care criteria.

Quality of care measurement is done to identify medical care delivery problems that can be remedied. Knowing this information presumably permits appropriate steps to be taken to remedy those problems. For this reason, we have argued in this article that one should select topics for measurement based on the expected impact of these quality assessment and assurance efforts. Although precise estimates of the expected impact of improved quality are not always possible, enough is known to incorporate epidemiologic and clinical evidence into public policy decisions about which aspects of quality of care should be measured.


(1.) RR' reflects the efficacy of the intervention. Specifically, "efficacy" (i.e., the proportion of potential outcome events that is prevented by the intervention under ideal conditions) is 1 -RR'. "Relative efficacy" (i.e., the extent to which full participation in the intervention reduces risk to the level of those who were never exposed) is [RR (I - RR')]I(RR - 1). (2.) IF = 0.206(0.757)(1 - 0.85)/0.206(1 - 0.85) + 0.85 = 0.027 (3.) RR for coronary heart disease death for individuals in the five serum cholesterol quintiles (lowest to highest) is 1.0, 1.3, 1.7, 2.2, and 3.4, respectively (Martin, Hulley, Browner, et al. 1986). Thus, (4.) IF = 0.6(0.583) (1 - 0.76)/0.6(1 - 0.76) + 0.76 = 0.093 (5.) In a nationally representative sample of Medicare patients with acute myocardial infarction in the mid-1980s, beta-blockers were received at some point during the hospitalization in 27 percent of patients (Rubenstein 19) (or 54 percent of eligible patients if we assume that half the patients had contraindications to treatment). Younger patients may be more likely to be treated with beta-blockers; thus, we assume that 60 percent of patients (both Medicare and younger patients) are currently receiving beta-blockers post-MI. (6.) IF = 0.4 (0.375)(1 - 0.78)/0.4(1 - 0.76) + 0.76 = 0.038 (7.) Assuming 50 percent are currently receiving treatment (i.e., P = (0.5 - 0.3)/0.5 = 0.4), IF = 0.5(0.4)(1 - 0.76)/0.5(1 - 0.76) + 0.76 = 0.054 (8.) There are approximately 25,000 deaths from breast cancer in women aged 50-75 (National Center for Health Statistics 1990a) and IF = 0.8(0.375)(1 - 0.070)/1.8(1 - 0.70) + 0.70 = 0.096 (9.) IF = 1.0(0.50)(1 - 0.67)/1.0(1 - 0.67) + 0.67 = 0.165

This manuscript was collaboratively written with the HMO Quality of Care Consortium Ad Hoc Clinical Subcommittee. The Consortium consists of 12 managed care plans, in alphabetical order: AV-MED, CIGNA Health Plans, Community Health Care Plan, Group Health Cooperative of Puget Sound, Harvard Community Health Plan, The Health Care Plan of Buffalo, The Health Plan of America, Kaiser-Permanente Colorado Region, Kaiser-Permanente Northwest Region, Kaiser-Permanente Southern California Region, MedCenters Health Plan, and United Healthcare Corporation. This work was supported by the Hartford Foundation, the National Institute on Aging (Academic Award to Dr. Siu), and the participating plans.


Andersson, I., K. Aspergren, L. Janzon, T. Landberg, K. Lindholm, F. Linell, O. Liungberg, J. Ranstam, and B. Sigfusson. "Mammographic Screening and Mortality from Breast Cancer: The Malmo Mammographic Screening Trial." British Medical Journal 297 (15 October 1988): 943-48. Behrman, R. E. Chairman, Committee to Study the Prevention of Low Birthweight. Preventing Low Birthweight. Washington, DC: Institute of Medicine, National Academy Press, 1985. Berwick, D. M. "Continuous Improvement as an Ideal in Health Care." New England Journal of Medicine 320, no. 1 (1989): 53-56. Blankenhorn, D. H., S. A. Nessim, R. L. Johnson, M. E. SanMarco, S. P. Azen, and L. Cashin-Hemphill. "Beneficial Effects of Combined Colestipol-Niacin Therapy on Coronary Atherosclerosis and Coronary Venous Bypass Grafts." Journal of the American Medical Association 257, no. 23 (1987): 3233-40. Bonadonna, G., and P. Valagussa. "Current Status of Adjuvant Chemotherapy for Breast Cancer." Seminars in Oncology 14, no. 1 (1987): 8-22. Brook, R. H., A. Davies-Avery, S. Greenfield, L. J. Harris, T. Lelah, N. E. Solomon, and J. E. Ware, jr. "Assessing the Quality of Medical Care Using Outcome Measures: An Overview of the Method." Medical Care 15 (1977 supplement): 1-84. Brook R. H., A. Fink, J. Kosecoff, L. S. Linn, W. E. Watson, A. D. Davies, V. A. Clark, and C. Kamberg. "Educating Physicians and Treating Patients in the Ambulatory Setting." Annals of Internal Medicine 107, no. 3 1987): 392-98. Brook, R. H., and K. N. Lohr. "Efficacy, Effectiveness, Variations and Quality Boundary-Crossing Research." Medical Care 23, no. 5 (1985): 710-22. Browner, W. S. "Estimating the Impact of Risk Factor Modification Programs." American Journal of Epidemiology 123, no. 1 (1986): 143-51. Buyse, M., A. Zeleniuch-Jacquotte, and T. C. Chalmers. "Adjuvant Therapy of Colorectal Cancer. Why We Still Don't Know." Journal of the American Medical Association 259, no. 24 (1988): 3571-78. Centers for Disease Control. U.S. Department of Health and Human Services. "Cigarette Smoking in the United States, 1986." Morbidity and Mortality Weekly Report 36 (11 September 1987): 581-85. Chambers, B. R., and J. W. Norris. "The Case against Surgery for Asymptomatic Carotid Stenosis." Stroke 15, no. 6 (1984): 964-67. Collins, J. G. "Prevalence of Selected Chronic Conditions: United States, 1979-81." In Vital and Health Statistics Series 10, no. 155. DHHS Publication no. (PHS) 86-1583. National Center for Health Statistics, Public Health Service. Washington, DC: U.S. Government Printing Office, July 1986. Crampton, R. S., R. F. Aldrich, J. A. Gascho, J. R. Miles, jr., and R. Stillerman. "Reduction of Prehospital, Ambulance and Community Coronary Death Rates by the Community-wide Emergency Cardiac C are System." American Journal of Medicine 58 (February 1975): 151-65. Crespi, M., C. S. Weissman, V. A. Gilbertsen, S. J. Winawer, and P. Sherlock. "The Role of Proctosigmoidoscopy in Screening for Colorectal Neoplasia." CA-A Cancer Journal for Clinicians 34, no. 3 (1984): 158-66. Cruickshank, J. M., J. M. Thorp, and F. J. Zacharias. "Benefits and Potential Harm of Lowering High Blood Pressure." Lancet 1 (March 1987): 581-83. Davies, A. R., and J. E. Ware, jr. "Involving Consumers in Quality of Care Assessment." Health Affairs 7, no. 1 (Spring 1988): 33-48. Deyo, R. A. "Conservative Therapy for Low Back Pain. Distinguishing Useful from Useless Therapy." Journal of the American Medical Association 250, no. 8 (1983): 1057-62. Dubois, R. W., and R. H. Brook. "Preventable Deaths: Who, How Often, and Why?" Annals of Internal Medicine 109 (October 1988): 582-89. Eddy, D. M. "Screening for Breast Cancer." Annals of Internal Medicine 111, no. 5 (1989): 389-99. Eddy, D. M., R. L. Wolpert, and M. L. Rosenberg. "Estimating the Effectiveness of Interventions to Prevent Youth Suicides." Medical Care 25, no. 12 (1987): S57-S65. Eddy, D. M., and V. Hasselbald. "The Value of Mammography Screening in Women under Age 50 Years." Journal of the American Medical Association 259, no. 10 (1988): 1512-19. Eisenberg, M., L. Bergner, and A. Hallstrom. "Paramedic Programs and Out-of-Hospital Cardiac Arrest: 11. Impact on Community Mortality." American Journal of Public Health 69, no. 1 (1979): 39-42. Fisher, B., C. Redmond, N. V. Dimitrov, D. Bowman, S. Legautt-Poisson, D. L. Wickerham, N. Wolmark, E. R. Fisher, R. Margolese, C. Sutherland, A. Glass, R. Foster, and R. Caplan. "A Randomized Clinical Trial Evaluating Sequential Methotrexate and Fluorouracil in the Treatment of Patients with Node-Negative Breast Cancer Who Have Estrogen-Receptor-Negative Tumors." New England Journal of Medicine 320, no. 8 (1989): 473-78. Fisher, B., J. Costantino, C. Redmond, R. Poisson, D. Bowman, C. Couture, N. V. Dimitrov, N. Wolmark, D. L. Wickerham, E. R. Fisher, R. Margolese, A. Robidoux, H. Shibata, J. Terz, A. H. G. Paterson, M. I. Feldman, W. Farrar, J. Evans, H. L. Lickley, and M. Ketner. "A Randomized Clinical Trial Evaluating Tamoxifen in the Treatment of Patients with Node-Negative Breast Cancer Who Have Estrogen-Receptor-Positive Tumors.' New England Journal of Medicine 320, no. 8 1989): 479-84. Fisher, B., M. Bauer, R. Margolese, R. Poisson, Y. Pilch, M. Deutsch, E. Montague, E. Saffer, L. Wickerham, H. Lerner, C. Redmond, E. Fisher, N. Wolmark, A. Glass, H. Shibata, P. Deckers, A. Ketcham, R. Oishi, and I. Russell. "Five-Year Results of a Randomized Clinical Trial Comparing Total Mastectomy and Segmental Mastectomy with or without Radiation in the Treatment of Breast Cancer." New England Journal of Medicine 312, no. 11 (1985): 665-73. Gilbertsen, V. A., and J. M. Nelms. "The Prevention of Invasive Cancer of the Rectum." Cancer 41, no. 3 (1978): 1137-39. Goldman, L., and E. F. Cook. "The Decline in Ischemic Heart Disease Mortality Rates: An Analysis of the Comparative Effects of Medical Interventions and Changes in Lifestyle." Annals of Internal Medicine 101, no. 6 1984): 825-36. Goldman, L., S. T. B. Sia, E. F. Cook, J. D. Rutherford, and M. C. Weinstein. "Costs and Effectiveness of Routine Therapy with Long-term Beta-Adrenergic Antagonists after Acute Myocardial Infarction." New England Journal of Medicine 319, no. 3 (1988): 152-57. Gruppo Italiano per lo Studio della Streptochinasi Nell'infarto Miocardico (GISSI). "Effectiveness of Intravenous Thrombolytic Treatment in Acute Myocardial Infarction." Lancet 1, no. 22 (February 1986): 397-401. --. Long-Term Effects of Intravenous Thrombolysis in Acute Myocardial Infarction: Final Report of the GISSI Study." Lancet 1, no. 17 (October 1987): 871-74. Hardcastle, J. D., W. M. Thomas, J. Chamberlain, G. Pye, J. Sheffield, T. W. Balfour, N. C. Armitage, P. D. James, S. S. Amar, and S. M. Moss. "Randomised, Controlled Trial of Faecal Occult Blood Screening for Colorectal Cancer. Results for First 107,349 Subjects." Lancet (27 May 1989):1160-64. Hayward, R. A., M. F. Shapiro, H. E. Freeman, and C. R. Corey. "Who Gets Screened for Cervical and Breast Cancer?" Archives of Internal Medicine 148 (May 1988): 1177-81. Hill, D., V. White, D. Jolley, and K. Mapperson. "Self Examination of the Breast; Is It Beneficial? Meta-Analysis of Studies Investigating Breast Self Examination and Extent of Disease in Patients with Breast Cancer." British Medical Journal 297 (23 July 1988): 271-75. Hillner, B. E., and T. J. Smith. "Efficacy and Cost Effectiveness of Adjuvant Chemotherapy in Women with Node-negative Breast Cancer." New England Journal of Medicine 324, no. 3 (1991): 160-68. Hlatky, M. A., H. Cotugno, C. O'Connor, D. B. Mark, D. B. Pryor, and R. M. Califf. "Adoption of Thrombolytic Therapy in the Management of Acute Myocardial Infarction." American Journal of Cardiology 61 (1 March 1988):510-14. Huguley, C. M., R. L. Brown, R. S. Greenberg, and W. S. Clark. "Breast Self-Examination and Survival from Breast Cancer." Cancer 62, no. 5 (1988):1389-96. ISIS Steering Committee. "Intravenous Streptokinase Given within 0-4 Hours of Onset of Myocardial Infarction Reduced Mortality in ISIS-2." Lancet 1 (28 February 1987): 502. ISIS-1 (First International Study of Infarct Survival) Collaborative Group. "Randomized Trial of Intravenous Atenolol among 16,027 Cases of Suspected Acute Myocardial Infarction: ISIS-1." Lancet 1 (12 July 1986): 57-66. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. "Collaborative Group Randomized Trial of Intravenous Streptokinase, Oral Aspirin, Both, or Neither among 17,187 Cases of Suspected Acute Myocardial Infarction: ISIS-2." Lancet 13 (August 1988): 349-60. Joyce, T., H. Corman, and M. Groosman. "A Cost-Effectiveness Analysis of Strategies to Reduce Infant Mortality." Medical Care 26, no. 4 (1988) 348-60. Kaplan, R. M., and J. P. Anderson. "A General Health Policy Model: Update and Applications." Health Services Research 23, no. 2 (1988): 203-35. Kosecoff, J., J. Fink, R. H. Brook, A. R. Davies, G. Goldberg, L. S. Linn, V. A. Clark, and P. C. Salisbury. "General Medical Care and the Education of Internists in University Hospitals: An Evaluation of the Teaching Hospital General Medicine Group Practice Program." Annals of Internal Medicine 102, no. 2 (1985): 250-56. Last, J. M. A Dictionary of Epidemiology. New York: Oxford University Press, 1983. Laurie, J. A., C. G. Moertel, T. R. Fleming, H. S. Wieand, J. E. Leigh, J. Rubin, G. W. McCormack, J. B. Gerstner, J. E. Malliard, D. I. Twito, R. F. Morton, L. K. Tschetter, and J. F. Barlow. "Surgical Adjuvant Therapy of Large-Bowel Carcinoma: An Evaluation of Levamisole and the Combination of Levamisole and Fluorouracil." Journal of Clinical Oncology no. 10 (1989): 1447-56. Lavie, C. J., G. T Gau, R. W. Squires, and B. A. Kottke. "Management of Lipids in Primary and Secondary Prevention of Cardiovascular Diseases." Mayo Clinical Proceedings 63 (June 1988): 605-21. Lee, T. H., M. C. Weisberg, D. A. Brand, G. W. Rouan, and L. Goldman. "Candidates for Thrombolysis among Emergency Room Patients with Acute Chest Pain: Potential True- and False-Positive Rates." Annals of Internal Medicine 110, no. 12 (1989): 957-62. Lipid Research Clinics Program. "The Lipid Research Clinics Coronary Primary Prevention Trial Results." Journal of the American Medical Association 251, no. 3 (1984): 351-74. Ludwig Breast Cancer Study Group. "Prolonged Disease-Free Survival after One Course of Perioperative Adjuvant Chemotherapy for Node-Negative Breast Cancer." New England Journal of Medicine 320, no. 8 1989): 491-96. Mansour, E. G., R. Gray, A. H. Shatila, C. K. Osborne, D. C. Tormey, K. W. Gilchrist, M. R. Cooper, and G. Faulkson. "Efficacy of Adjuvant Chemotherapy in High-Risk Node-Negative Breast Cancer: An Intergroup Study." New England Journal of Medicine 320, no. 8 (1989): 485-90. Martin, M. J., S. B. Hulley, W. S. Browner, L. H. Kuller, and D. Wentworth. "Serum Cholesterol, Blood Pressure, and Mortality: Implications from a Cohort of 361,622 Men." Lancet 25 (October 1986): 933-36. Mayer, R. J., M. J. O'Connell, J. E. Tepper, and N. Wolmark. "Status of Adjuvant Therapy for Colorectal Cancer." Journal of the National Cancer Institute 81, no. 18 (1989): 1359-64. McCormick, J., and P. Skrabanek. "Coronary Heart Disease Is Not Preventable by Population Interventions." Lancet 1 (27 October 1988): 839-41. McCormick, M. C. "The Contribution of Low Birth Weight to Infant Mortality and Childhood Morbidity." New England Journal of Medicine 312 (10 january 1985): 82-90. Moertel, C. G., T. R. Fleming, J. S. Macdonald, D. G. Haller, J. A. Laurie, P. J. Goodman, J. S. Ungerleider, W. A. Emerson, D. C. Tormey, M. H. Veeder, and J. A. Mailliard. "Levamisole and Fluorouracil for Adjuvant Therapy of Resected Colon Carcinoma." New England Journal of Medicine 322, no. 6 (1990): 352-58. Morgenstern, H., and E. S. Bursic. "A Method for Using Epidemiologic Data to Estimate the Potential Impact of an Intervention on the Health Status of a Target Population." Journal of Community Health 7, no. 4 (1982): 292-309. National Center for Health Statistics. "Advance Report of Final Mortality Statistics, 1988." Monthly Vital Statistics Report 39, no. 7 (1990a): 1-48. . Prevention Profile. Health, United States, 1989. Hyattsville, MD: Public Health Service, 1990b. Nattinger, A. B., R. J. Panzer, and J. Janus. "Improving the Utilization of Screening Mammography in Primary Care Practices." Archives of Internal Medicine 149 (September 1989): 2087-92. O'Brien, E., and K. O'Malley. "Overdiagnosing Hypertension." British Medical Journal 297 (November 1988): 1211-12. Office of Technology Assessment. The Qualify of Medical Care: Information for Consumers. Summary. Washington, DC: U.S. Government Printing Office, 1988. Oldridge, N. B., G. H. Guyatt, M. E. Fischer, and A. A. Rimm. "Cardiac Rehabilitation after Myocardial Infarction." Journal of the American Medical Association 260 (1988): 945-50. Olson, R. E. "A Critique of the Report of the National Institutes of Health Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol." Archives of Internal Medicine 149, no. 7 (1989): 1501-1503. Ouellet, R. P., A. Y. Apostolides, G. Entwisle, and J. R. Hebel. "Estimated Community Impact of Hypertension Control in a High Risk Population." American Journal of Epidemiology 109, no. 5 (1979): 531-38. Pokras, R. "Detailed Diagnoses and Procedures for Patients Discharged from Short-Stay Hospitals, United States, 1985." In Vital and Health Statistics Series 13, no. 90. DHHS Publication no. (PHS) 87-1751. National Center for Health Statistics. Public Health Service. Washington, DC: U.S. Government Printing Office, April 1987. Robins, J. M., P. J. Landrigan, T. J. Robins, and L. J. Fine. "Decision Making under Uncertainty in the Setting of Environmental Health Policy." Journal of Public Health Policy 6 (September 1985): 322-28. Rubstein, L. V. Telephone conversation. 19 . Selby, J. V., and G. D. Friedman. "Sigmoidoscopy in the Periodic Health Examination of Asymptomatic Adults." Journal of the American Medical Association 261, no. 4 (1989): 595-601. Shapiro, S. "Evidence on Screening for Breast Cancer from a Randomized Trial." Cancer 39 (June 1977, supplement): 2772-82. Siu, A. L., E. McGlynn, M. H. Beers, H. Morgenstern, D. M. Carlisle, E. B. Keeler, R. H. Brook, J. Beloff, K. Wirtin, J. Leaning, B. C. Perry, H. P. Selker, W. Weiswasser, and A. Wiesenthal. The Expected Impact of Improved Quality of Care for the Major Causes of Mortality and Morbidity. Santa Monica, CA: The RAND Corporation, 1992. Steinberg, M. D., M. A. Juliano, and L. Wise. "Psychological Outcome of Lumpectomy versus Mastectomy in the Treatment of Breast Cancer." American Journal of Psychiatry 142, no. 1 (1985): 34-39. Stockdale, F. E. "Breast Cancer." In Scientific American Medicine. Edited by E. Rubenstein and D. Federman. New York: Scientific American, Inc., 1988. Strauss, W. E., and A. F. Parisi. "Combined Use of Calcium-Channel and Beta-Adrenergic Blockers for the Treatment of Chronic Stable Angina. Rationale, Efficacy, and Adverse Effects." Annals of Internal Medicine 109, no. 7 (1988): 570-81. Tabar, L., C. J. G. Fagerberg, A. Gad, L. Baldetorp, L. H. Holmberg, O. Grontoft, U. Ljungquist, B. Lundstrom, J. C. Manson, G. Eklund, and N. E. Day. "Reduction in Mortality from Breast Cancer after Mass Screening with Mammography. Randomized Trial from the Breast Cancer Screening Working Group of the Swedish National Board of Health and Welfare." Lancet (13 April 1985): 829-32. Tate, D. A., and G. J. Dehmer. "New Challenges for Thrombolytic Therapy." Annals of Internal Medicine 1 10, no. 12 (1989): 953-55. Theroux, P., H. Ouimet, J. McCans, J. Latour, G. Levy, E. Pelletier, M. Juneau, J. Stasiak, P. Deguise, G. B. Pelletier, D. Rinzler, and D. D. Waters. "Aspirin, Heparin, or Both to Treat Acute Unstable Angina." New England Journal of Medicine 319, no. 17 (1988): 1105-1 1. UK Trial of Early Detection of Breast Cancer Group. "First Results on Mortality Reduction in the UK Trial of Early Detection of Breast Cancer." Lancet (20 August 1988): 411-16. Verbeek, A. L. M., J. H. C. L. Hendriks, R. Holland, M. Mravunac, F. Sturmans, and N. E. Day. "Reduction of Breast Cancer Mortality through Mass Screening with Modern Mammography. First Results of the Nijmegen Project, 1975-1981." Lancet (2 June 1984): 1222-24. Veronesi, U., R. Saccozzi, M. Del Vecchio, A. Banfi, C. Clemente, M. DeLena, G. Gallus, M. Greco, A. Luini, E. Marubini, G. Muscolino, F. Rilke, B. Salvadori, A. Zecchini, and R. Zucali. "Comparing Radical Mastectomy with Quadrantectomy, Axillary Dissection, and Radiotherapy in Patients with Small Cancers of the Breast." New England Journal of Medicine 305, no. 1 (1981): 6-11. Warlow, C. "Carotid Endarterectomy: Does it Work?" Stroke 15, no. 6 (1984): 1068-76. Warren, R., and P. Skrabanek. "The Debate over Mass Mammography in Britain. The Case For, the Case Against." British Medical Journal 297 (October 1989): 969-72. Wilcox, R. G., G. von der Lippe, C. G. Olsson, G. Jensen, A. M. Skene, and J. R. Hampton. "Trial of Tissue Plasminogen Activator for Mortality Reduction in Acute Myocardial Infarction. Anglo-Scandinavian Study of Early Thrombosis (ASSET)." Lancet 3 (September 1988): 525-30. Williamson, J. W. "Formulating Priorities for Quality Assurance Activity. Description of a Method and Its Application." Journal of the American Medical Association 239, no. 7 (1978): 631-37. Williamson, J. W., M. Alexander, and G. E. Miller. "Priorities in Patient-Care Research and Continuing Medical Education." Journal of the American Medical Association 204, no. 4 (1968): 303-308. Winawer, S. J., S. D. Leidner, D. G. Miller, D. Schottenfeld, B. Befler, R. C. Kurtz, P. Sherlock, and M. Stearns. "Results of a Screening Program for the Detection of Early Colon Cancer and Polyps Using Fecal Occult Blood Testing." Gastroenterology 72, no. 5 (1977): 1150. Winslow, C. M., D. H. Solomon, M. R. Chassin, J. Kosecoff, N. J. Merrick, and R. H. Brook. "The Appropriateness of Carotid Endarterectomy." New England Journal of Medicine 318, no. 12 (1988): 721-27. Wynder, E. L., F. Field, and N. J. Haley. "Population Screening for Cholesterol Determination. A Pilot Study." Journal of the American Medical Association 256, no. 20 (1986): 2839-42. Yusuf, S., J. Wittes, and L. Friedman. "Overview of Results of Randomized Clinical Trials in Heart Disease. I: Treatments following Myocardial Infarction." Journal of the American Medical Association 260, no. 14 (1988): 2088-93. Yusuf, S., R. Peto, J. Lewis, R. Collins, and P. Sleight. "Beta Blockade during and after Myocardial Infarction: An Overview of the Randomized Trials." Progress in Cardiovascular Diseases 27 (1985): 335-71. Zaridze, D. G. "Environmental Etiology of Large-Bowel Cancer." Journal of the National Cancer Institute 70 (1983): 389-400.

Albert L. Siu, M.D., M.S.P.H., lead author, is affiliated with the Health Sciences Program at RAND, Santa Monica, CA, and the UCLA School of Medicine, Department of Medicine, Los Angeles, CA. Elizabeth A. McGlynn, Ph.D. and Emmett B. Keeler, Ph. D. are in the Health Sciences Program at RAND. Mark H. Beers, M. D. and David M. Carlisle, M.D., M.P.H. are in the UCLA School of Medicine, Department of Medicine. Hal Morgenstern, Ph.D. is in the UCLA School of Public Health, Los Angeles, CA. Robert H. Brook, M.D., Sc.D. is in the Health Sciences Program at RAND; the UCLA School of Medicine, Department of Medicine; and the UCLA School of Public Health. Members of the Consortium Subcommittee are Jerome Beloff, M.D. of AV-MED Health Plan, Miami, FL; Kathleen Curtin, M.B.A. of The Health Care Plan, Buffalo, NY; Jennifer Leaning, M.D. of The Harvard Community Health Plan, Boston, MA; Bruce C. Perry, M. D. of Group Health Cooperative of Puget Sound, Seattle, WA; Harry P. Selker, M.D., M.S.P.H. of New England Medical Center, Boston, MA; Warren Weiswasser, M.D. (deceased) of The Community Health Care Plan, New Haven, CT; and Andrew Wiesenthal, M.D. of Kaiser-Permanente Colorado Region, Denver, CO.

Address correspondence and requests for reprints to Albert L. Siu, M.D., M.S.P.H,, RAND, P.O. Box 2138, Santa Monica, CA 90407-2138. This article, submitted to Health Services Research on July 3, 1990, was revised and accepted for publication on January 13, 1992.
COPYRIGHT 1992 Health Research and Educational Trust
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 1992 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Siu, Albert L.; McGlynn, Elizabeth A.; Morgenstern, Hal; Beers, Mark H.; Carlisle, David M.; Keeler,
Publication:Health Services Research
Date:Dec 1, 1992
Previous Article:Commentary: are we splitting hairs over splitting DRGs?
Next Article:How the elderly fare in HMOs: outcomes from the medicare competition demonstrations.

Terms of use | Copyright © 2016 Farlex, Inc. | Feedback | For webmasters