Combining information about process and outcomes to improve medical care.

For some managers, clinical quality improvement begins and ends with measuring the clinical outcomes of care. For others, analyzing and altering processes of care are the central focus. To remove quality improvement from the realm of guesswork, these approaches must be intertwined. In health care, this linkage has appeared elusive. Now, using decision theory, there is a way.

Several ophthalmologists in a group practice became concerned that one of their partners was "losing his touch." Each of them recalled an anecdote or two in which this partner's patients had complications or did not seem to recover as quickly as had been expected.

Not many years ago, several unfavorable anecdotes and an agreement among colleagues might have been enough to chastise this ophthalmologist or even begin to ease him out of the group. This was a common recourse when quality determinations were based only on perception, expert judgment, and adherence to rules. Modern computer technology and analytic capabilities, along with total quality management and decision theory techniques borrowed from manufacturing industries, are changing that. The ophthalmologists mentioned above were hesitant to approach their partner without supporting facts. Instead, they proposed a general study of complication rates throughout the practice. The study centered on where and when complications were occurring and included a comparative analysis of each partner's clinical outcomes of care.

The results of the outcomes analysis surprised them: All the doctors had similar complication rates, but most of the difficulties were occurring among patients seen in a particular examining room. There, it turned out, a major piece of equipment was miscalibrated and had been producing incorrect measurements.

As this incident demonstrates, when quality problems are suspected, both process and outcomes need to be examined. As Deming has observed, if you do what you've always done, you'll get what you always got. On the other hand, health care delivery contrasts with manufacturing widgets in that changing what you've always done does not necessarily produce a different or better outcome. Analysis of clinical outcomes becomes necessary both to identify opportunities for improvement and to evaluate changes in process.

Analyzing Clinical Outcomes

A pioneer in the technology of using clinical outcomes data to monitor quality of care has been the Health Care Financing Administration (HCFA). Its methods of risk adjustment, though not perfect, have been refined each year since publication of the first Medicare mortality report in 1986.

Despite criticism, secondary analyses have indicated that the Medicare mortality data are both valid and reliable. When the risk-adjusted mortality rates for Medicare patients at more than 5,000 hospitals nationwide were compared over two years, 35 hospitals had outcome rates that were at least 20 percent better than the national average. The odds of this occurring by chance are less than one in a million. Nearly all of the hospitals with the lowest mortality rates already were widely recognized as highly respected institutions with excellent reputations for the quality of their patient care. Another study divided hospitals into eight categories according to their risk-adjusted mortality rates for each of three successive years. More than 99 percent of hospitals remained in the same or an adjacent category from one year to the next--a remarkable degree of consistency.

Although the technology of clinical outcomes measurement is still in its infancy, clear and reliable distinctions in performance have been made repeatedly, not only among hospitals but also among clinical departments within the same facility.

One such study motivated analysis of a hospital's open heart surgical service. The analysis showed considerable room for improvement. Physicians and managers greeted this news with two possible explanations, based on differing philosophies of care. One was that a group of physicians was too aggressive about exposing moderately ill patients to the risks of operation. The other was that their competitors operated only on the sickest patients, who then had difficulty surviving the procedure. (The group considered responsible for the high mortality rates, of course, was the one with the opposing philosophy of care.) The goal of analysis was to discover which group was at fault so that the physicians in the wrong could be reformed.

Results for the two groups of practitioners were compared, and mortality and complication rates were delineated at specified intervals after surgery. Those who came to point sat on them when they learned that deaths were occurring at about the same rates for patients in both practices. Data revealed that the bulk of problems began, not at surgery, but in the recovery room. Both sides of the controversy reluctantly agreed to bury the hatchet (at least on this issue) and to cooperate in examining recovery room processes to find the common or systemic cause.

Demands for information about clinical outcomes increasingly are coming from consumers and third-party payers who seek more value for their health care dollar. In one large industrial community, health care providers have joined forces with large corporate purchasers of health benefits to obtain more information about comparative quality of care. They are compiling a large database and, at the same time, undertaking patient satisfaction surveys. Large corporate purchasers plan to use the results to shift patients to providers of choice. Providers will use the information to establish a baseline for future performance and to improve quality to increase their share of the health care market.

A large insurer in a midwestern state already is using mortality data to establish its preferred provider network. As a result, one of the providers that had hoped to contract with this insurer as a center of excellence found itself in danger of being dropped from the network altogether. It now is endeavoring to improve its performance so that it can continue to qualify to serve this important customer.

Analyzing Processes of Care

While clinical outcomes analysis has gained attention only since the 1980s, processes of care have been scrutinized for inspection purposes during the past several decades. Screening checksheets first were designed to determine whether certain steps had been taken and whether specified techniques had been used. Later, branching logic was incorporated to allow for alternatives in diagnosis or treatment. The resulting utilization review protocols became more complex and more authoritative as nationally representative groups were assembled to reach expert consensus about particular protocols and to grant their approval.

These detailed descriptions of process have been called algorithms, protocols, practice parameters, guidelines, standards of care, or critical pathways. In chart form, they are strikingly similar to the flow charts used by industry in total quality management. By studying the flow in a manufacturing plant, a quality improvement team can discover where in the process critical decisions are made and what steps might be streamlined, altered, or eliminated to reduce errors and rework and to make the process more efficient. In health care organizations, critical pathways describe pivotal steps in the clinical processes of care. Their goal ordinarily has been to define generally desirable practice patterns and to reduce variation from these patterns and techniques. Their use has evolved from inspection to education, communication, and process analysis.

Health care teams today are using various combinations of total quality management tools to improve and supplement critical pathways in examining the processes of care. In one hospital, presurgical medication dosages came under scrutiny. Fishbone and cause-and-effect diagrams revealed significant variation in the number of times per day medications were being administered before surgery and in the time elapsing between administration of medication and surgery. The quality improvement team consulted with physicians and reviewed the medical research literature to establish optimal medication dosages and times. They incorporated this information in the hospital's critical pathways and used the new pathways to communicate the optimal schedule for medications. As a result, variations in practice patterns were reduced and unnecessary medications were avoided. Major cost savings as well as improved quality of care were the outcomes.

Such examples, while impressive, are not numerous. The path from first encounter to final outcome is considerably more complex for health care delivery than, for example, manufacture of a light bulb. The relationship between process and outcome is less direct and the outcome is less predictable.

Using Decision Theory to Relate Outcomes and Processes of Care

A lesser-known but emerging method of addressing, this ambiguity is decision theory.[*] In industry, decision theory has been used for many years to examine risk/benefit ratios and to weigh various alternatives when exact outcomes of business decisions are not predictable. Decision theory is heavily dependent upon decision trees, which are similar to protocols or critical pathways. At each juncture in a critical pathway, decision theory adds weights that represent the risk/benefit ratio for each set of alternatives. It also incorporates the probabilities of occurrence of each outcome. Efforts to make these determinations for clinical care have revealed how little basic information is really available to health care providers about the probabilities of various outcomes and the relative effectiveness and efficiency of medical techniques. This underscores the need for further epidemiologic studies that relate processes to the outcomes of care. Such studies employ large databases to assess the rates at which expected outcomes of care actually are being achieved.

Probabilities drawn from epidemiological research about possible outcomes and the degrees of risk associated with various characteristics of patients will someday be incorporated into decision trees or sophisticated clinical pathways. This will permit interactive computer programs in which progressive practitioners will be able to obtain instant verification of diagnoses and statistical probabilities of various treatment modalities. Patients will be able to participate more knowledgeably in decisions about their care when they have more precise information about the probabilities of all the potential risks and outcomes associated with each alternative.

While TQM presently receives the funds and holds the spotlight, work of this nature proceeds quietly behind the scenes. One surgical specialty already has created and distributed to its members a detailed set of parameters of care that were carefully designed to serve as the foundation for data collection. Data on indications for care, therapeutic goals, factors affecting risk, standards of care, indicators of desirable outcomes, and indicators of undesirable outcomes will be collected and analyzed to produce a profile of current practice patterns. The profile will become the baseline for epidemiological studies to assess the relative effectiveness and efficiency of various surgical procedures and techniques. The resulting insights will be disseminated promptly to the members of the specialty. As practitioners alter their practice patterns to take advantage of the research results, the parameters will be revised. As outcomes data are continually being analyzed, feedback is received, practice patterns are altered, and the parameters are revised, quality of care in this surgical specialty will continue to improve.

As the field of outcomes measurement and clinical decision analysis matures, health care organizations will become more adept at creating and using critical pathways and decision trees in combination with analyses of risk-adjusted outcomes to improve practice patterns. As the comparative performances of individual practitioners continues to become more visible, those who can document and demonstrate value to their customers will be the winners. To do so, greater sophistication in monitoring both process and clinical outcomes will be required.

* Weinstein, M., and others. Clinical Decision Analysis. Philadelphia, Pa. W.B. Saunders Company, 1980.

Michael Pine, MD, MBA, is President of Michael Pine and Associates, Chicago, Ill., and Research Associate, Department of Medicine, University of Chicago, Chicago, Ill., and Joan Pine is Director of Communications, Michael Pine and Associates. Dr. Pine is a member of the College's Society on Corporate Medical Services and its Forum on Quality Health Care.