Clinical decision analysis.

Clinical Decision Analysis When they make clinical decisions, physical therapists face a difficult challenge. These decisions often have important consequences for patients and require valuable resources to carry out, yet clinical choices usually are made under conditions of great uncertainty. The scientific basis for practice is incomplete, and complex variations in individual characteristics make it difficult to predict exactly how a particular patient will respond to treatment. Clinical decision analysis is a systematic method for making wise choices under just such circumstances. Developed originally by Howard Raiffa at Harvard Business School, in the past decade decision analysis has been adapted for application to medicine. [1] [2] The rationale for using a similar process in physical therapy has been discussed previously. [3] This article will present an overview of six major steps in clinical decision analysis and illustrate these steps with examples from physical therapy practice.

Procedure and Examples

Step 1--Defining the

Decision Problem

Each major step in decision analysis involves several component activities. For example, to define a problem for analysis, we must both select a patient care decision that presents some special problem to us and specify the patient population and treatment situation to which we want the analysis to apply.

Because of the detail they involve, analyses are most useful if they focus on a fairly specific type of diagnostic or therapeutic action, rather than on design of a patient's overall plan of care. For example, if you were responsible for treating patients with head injury in the acute neurological unit of a general hospital, a specific component of treatment that presents a decision-making problem might be: "How to manage hypertonicity in comatose patients with head injury?"

Because the methods we are free to consider and the results we get from treatment may be influenced strongly by variations in the patient's characteristics and in the resources and constraints of the treatment situation, we need to spell out our major assumptions about these variables as a framework for the analysis. This defining of major assumptions helps to rule out unusual cases that could complicate and confuse our thinking about the majority of patients. It also allows colleagues to decide whether our conclusions can be applied to their patients and treatment situation. In defining the problem just presented as an example, you might want to specify the following: "The analysis will be limited to patients who are: comatose, one week to three months post-head injury, out of intensive care, free of extracranial injuries that limit management options, and between 5 and 60 years of age."

Characteristics of the treatment setting you might want to standardize could include: "In the hospital where care is provided, all comatose patients are on the acute neurological unit, this unit occasionally has a shortage of experienced nursing staff, and family members are permitted to be with patients between 8 AM and 9 PM."

Specifying such assumptions greatly clarifies and simplifies a decision analysis. It also, however, restricts the range of patients and situations to which the completed analysis can be applied. We try, therefore, to be selective in adopting assumptions and to avoid adding any that are not essential in order to complete an understandable analysis in a reasonable period of time.

Step 2--Defining Successful and

Unsuccessful Outcomes

When our decision analysis is complete, we will use it to select a preferred course of action, the strategy that seems most likely to lead to a good outcome. At the start of an analysis, therefore, we must decide exactly what we hope to accomplish, what we hope to avoid, and when it will be realistic to judge how successful we have been. This step has three components: 1) identification of the factors that concern us most in judging the results of our decision making, 2) development of a clearly defined set of outcome categories we can use to classify the results we actually achieve, and 3) determination of a specific time at which we will make this evaluative classification.

The list of concerns should reflect our rationale for working with this disorder and name the factors that must be included in a definition of success. For example, in selecting a method for managing hypertonicity in the comatose patient with head injury, we might have the following concerns: patient comfort, ease of nursing care for skin care and hygiene, freedom from musculoskeletal injuries, and level of independent motor function.

Simply knowing what factors we consider important in making this decision is only part of the job. To arrive at a method for judging results, we must translate these concerns into a set of categories that are relevant to our stated concerns, operationally defined, all inclusive, and mutually exclusive. In addition, the categories must be time based; that is, they must specify at what time in the future results of the actions we choose will be assessed.

For example, we might establish the following set of outcome categories for judging results in the head-injury analysis: "At the end of three months, the patient may be 1) dead; 2) dependent (vegetative or severely disabled), either with care or comfort problems or free of problems; or 3) independent (moderately disabled or good recovery), either with care or comfort problems or free of problems." This is only a skeleton description of the categories. Additional detail would be needed to make them operational and unambiguous. For example, in categories "dependent with care or comfort problems" and "independent with care or comfort problems," we would need to specify which types of problems we will consider attributable to hypertonus and the level of severity with which such problems must be present for us to assign a patient to these outcome groups.

Before undertaking this additional work, we should stop to consider whether the analysis really requires five different categories. In many cases, the preferred approach can be selected using only two possible outcome groupings: satisfactory and unsatisfactory. For example, the five categories listed earlier could be collapsed into two, with "satisfactory" defined as an outcome in which the patient is still alive and, regardless of whether he has regained a functional level of consciousness, remains free of any problems of comfort or personal care attributable to hypertonus. All patients who do not meet this standard would be classified as having an unsatisfactory outcome.

Several aspects of this process are likely to be troubling. As realists, we recognize that even if we do our job superbly, the patient may have an unsatisfactory result. We cannot be confident, for example, that if we manage the hypertonus effectively, all patients will surfive for three months. We also know we are not the only ones who may be responsible for a good result. Even if we neglect or mismanage the patient's hypertonus, he may remain free of related problems because of effective medication or good nursing care or simply because the neurological damage done by the initial head injury was temporary and the patient's overall status improved rapidly. Because we cannot identify the specific effects physical therapy produces, in most analyses we must compare the effectiveness of alternative courses of action simply by saying we believe the patient is more likely to have a satisfactory result if we make wise decisions than if we do not. The outcome classification system we develop then allows us to compare the outcomes actually achieved when large numbers of patients receive care designed using different decision-making approaches.

Step 3--Describing

Alternative Approaches and

The Consequences

This is the most complex and difficult phase in an analysis. It requires us to carry out the following series of activities:

1. List the options for treatment or evaluation we want to consider.

2. Identify at least two different approaches to deciding which option to use.

3. Identify the sequence of choices each approach will involve and the possible short-term consequences each action may have.

4. Summarize these events using a standardized graphic format called a "decision tree."

5. Define each event (action and consequence) in operational terms.

These activities can be illustrated using the head-injury example. We would need to begin by listing all the methods we want to consider as options for managing the patient's hypertonicity. This list might include passive alignment and range-of-motion exercises, serial casting, posturing and other sensory input techniques to modify tone, and no treatment at this time. As we begin thinking about the different approaches we might take to choosing an option, we might consider which option is easiest, least expensive, most readily reversed or modified, most aggressive, least aggressive, usually successful, or will be most helpful in deciding what to do next.

Our preference will be influenced by the situation and by variations in the patient's status. This preference may make us feel the first step we take should be to evaluate key characteristics of the patient and the treatment situation. Whatever first step we select, we are never entirely sure what the evaluation will reveal or how a patient will respond to our initial intervention. To proceed with the analysis, therefore, we must anticipate what the consequences of each initial action might be and then go on to consider what actions we might take next under each set of contingencies. For example, if the first action we choose in the head-injury analysis were to manage the hypertonicity with positioning and passive ROM and if the initial response were satisfactory, our next action might be to continue without change, try a different method to determine whether the results will be even better, or reduce the frequency of treatment to determine whether the results will remain satisfactory. If the initial response were unsatisfactory, however, our next action might be to try the initial method a little longer, try a different method, discontinue treatment, or evaluate why the initial intervention was unsuccessful.

As with the sample list of first steps, these are only a few of the many options open to us for follow-up action, and it is easy to see that the terms we are using to describe possible actions and consequences will need careful definition before the analysis is complete. Even at this early state, however, we should begin to summarize our thinking by putting it on paper using the decision-tree format.

Figures 1 and 2 show the beginning and intermediate stages, respectively, of a decision tree diagram and demonstrate several of the format conventions that make these summaries easy for other decision analysts to understand:

1. All decision trees read from left to right across the page, beginning with the first choice we must make and showing the key events that follow in sequence until the point is reached at which outcomes are assessed.

2. Events that represent a decision by the therapist about which of two or more actions to take are called "choice points" and are represented on the tree by a square.

3. Events that represent the consequences or immediate results of an evaluative or therapeutic action are called "chance points" and are designated by a circle.

4. Choice points and chance points alternate with each other as time progresses and are connected on the tree by a line called a "path."

5. At least two paths branch out following each choice point because making a choice always involves considering at least two alternatives.

6. Each chance point also is followed by two or more branches because these represent points at which we cannot be completely sure what the response to our action or results of evaluation will be.

7. Each path is labeled with a brief symbol or legend to give a general idea of what it represents.

8. When the decision tree is complete, each path leads to the set of possible outcomes identified at the start of the analysis.

As the analysis progresses, and branches are added, the decision tree may come to resemble a tangled thicket and require judicious "pruning" to remove events and paths that are not essential and to organize the diagram so it is easy to follow. This task is easiest to do if the paths are used to represent different overall approaches to decision making. The decision tree shown in Figure 2 is still quite incomplete, but illustrates how several different approaches might look when summarized in a diagram. Additional examples of decision trees for physical therapy problems may be seen in Figure 3 and in the literature.[3,4]

The end result of a thoughtful analysis may seem deceptively simple. Identifying choices and predicting consequences can be difficult and time consuming, especially when the person doing the analysis is a highly experienced clinician who often makes decisions intuitively. Examining the decision-making process in a conscious, analytical way often requires the expert to work through a sort of professional aphasia to review what he or she actually does in practice and to describe this process so others can understand.

Fortunately, even a somewhat messy and incomplete decision tree can be a useful tool for clarifying our own thinking and communicating with colleagues about an important decision problem. When this is true, it will seem worthwhile to continue the process and take on the time-consuming task of defining the choice and chance alternatives we are considering. If the decision analysis is to proceed to the next step of probability analysis, each branch on the tree must be defined with the same operational clarity as the outcome categories, and we must be equally certain the consequence branches are all inclusive and mutually exclusive.

Step 4--Estimating and

Analyzing Probabilities

A completed decision tree tells us only what each of several different approaches to decision making might involve; it does not tell us which approach is best. Before we can use the analysis as a guide to selecting a preferred decision-making strategy, we need to estimate the probability of achieving a successful outcome using each approach considered.

This process begins with estimates of probabilities for alternative consequences at each chance point in the tree. Starting with the earliest chance points at the far left of the tree, and working our way through all chance points along each path, we must try to estimate the probability that each possible consequence actually will occur. At each chance point the 100% probability that something will happen is divided among the possible branches we have diagrammed. These values are then written in parentheses on the tree

as shown in Figure 4.

A similar process is used at the end of each path to estimate the proportion of all patients who go through that particular chain of events who will end up in each outcome category. Because it makes later calculations easier, I like to write the probabilities for final outcomes on the tree using rates per 1,000 rather than percentages.

The process of recording our expectations in quantitative terms makes us painfully aware of the obvious questions: "How do we know what values to assign at chance points and in predicting outcomes?" "How can we justify such estimates?" Ideally, we would do this by turning to clinical science and basing our estimates on published reports of soundly designed clinical research in which the population studied and the methods of intervention and measurement used all were identical to those in our analysis. In reality, we seldom are lucky enough to find such convincing data. Research on the clinical outcomes of physical therapy still is scanty, and we often hae doubts about the applicability of findings even when we do find a published study concerned with the decision we are analyzing. In designing and evaluating research, most of us have been taught to give careful attention to the internal validity fo a study. Internal validity, of course, is important, but when research findings are to be used to guide practical decisions about care of real patients, we must be equally concerned about their external validity.[5] For example, even the best single-subject studies and research on healthy subjects may have little applicability to the patients and treatment setting in our analysis. Assignment of probabilities in decision analysis thus begins with a thoughtful and suspicious review of the research literature, but then moves on to incorporate ideas from several other sources. Research findings can be supplemented with objective data on observed frequencies of key patient characteristics and responses systematically documented in other databases, such as patient records and audits; logical inferences from research on underlying mechanisms and from application of theoretical models; and subjective estimates based on personal experience, intuition, and discussions with experienced colleagues.

Combining the data we derive from these varied sources to arrive at quantitative probability estimates is not an easy or a comfortable process. We may doubt the value of performing a formal decision analysis when it relies so heavily on incomplete and subjective evidence.

We cannot make our uncertainty in predicting the results of treatment disappear, however, simply by refusing to confront it. Each day clinicians must make important choices about what to do on the basis of their "best guesses" about the probable results of alternative actions. Expressing these guesses quantitatively when a decision is especially important, controversial, and frequently encountered in practice has several advantages. It forces us to consider competing hypotheses and plan treatment not simply on the basis of what we hope will happen, but on a basis that prepares for the possibility of failure. It provides a mechanism for comparing and integrating the guidance we get from conscious logic and documented research with that derived from experience and intuition. Perhaps most important of all, formal probability analysis can help us set realistic priorities for future research. It reminds us of the need to test our theoretical models and treatment hypotheses through clinical studies of large groups of real patients treated under real clinical conditions. It also helps us see where the evidence supporting key clinical decisions is most questionable and the need for additional research most urgent.

Although we can learn a great deal from thinking about the probabilities at individual chance points, these probability estimates may not be enough to show us clearly which approach to decision making is most likely to be effective. Especially when each of the approaches we want to compare involves a series of branching pathways, the overall picture may be difficult to interpret. The probability estimates for individual points within the tree, therefore, often need to be analyzed further to help us make a strategic series of choices about how to proceed. In decision analysis, the method used for this step is a straightforward application of the summation principle for calculating joint probabilities referred to as "averaging out and folding back." This article will not attempt to describe the procedure in detail, but clear explanations of rationale and step-by-step instructions for the calculations are readily available in the decision analysis literature, and an example of completed calculations is shown in Figure 3. [1,2,6]

The arithmetic for these calculations is simple, but it can be time consuming and tedious if the tree is complex or if we want to test the sensitivity of our analysis by varying the probability estimates at key change points to determine how great a change is needed to change our choice of a preferred action. Fortunately, once the tree has been drawn and values estimated for all individual chance-point probabilities, these calculations of combined probability can be done efficiently using a computerized spreadsheet program, so this need not be a major chore. [7]

The end result of this quantitative analysis is a tree that shows at each decision point which action is most likely to lead to a successful outcome. A pair of diagonal slashes is placed across the path(s) that should be rejected at each choice point. The remaining path indicates not only the best strategy for achieving success but also the best way to respond to problems and failures that occur along the way.

Step 5--Estimating Costs

Identifying the most effective approach to patient care represents only one side of wise clinical decision making. Selection of a preferred course of action also must take into account differences in the resources alternative approaches require. This process involves four different activities: 1) identification of the different types of resources each approach involves, 2) quantification to estimate the amount of each resource needed, 3) valuation to analyze how much these resources are worth, and 4) summarization of this value in terms of some common denominator that will let us compare the overall demands of the approaches considered.

Major categories of resources that usually need to be considered include

Tangible resources. These resources are goods and services to which it is relatively easy to assign a monetary value based on their fair market price. Costs for these resources may be either of the following:

1. Direct costs--expenditures for time and skills of health care professionals and laymen who provide services, facilities, and equipment used or purchased, supplies consumed, and access costs. Direct costs also include expenditures for such things as transportation, telephone calls, child care, or hospitalization needed to receive care.

2. Indirect costs--lost productivity of the patient or others because of time taken off work for treatment or because of the health care professional's recommendations to reduce or avoid certain work activities.

Intangible resources. These resources are demands associated with the process of health care to which it is difficult to assign a monetary value because they represent psychological and social factors that vary widely from individual to individual. Such costs may include

1. Satisfactions lost--interesting, enjoyable, rewarding activities that must be reduced or given up entirely in order to use health services or follow the recommendations of health care professionals (eg, giving up after-work recreational activities in order to go for physical therapy).

2. Dissatisfactions added--uncomfortable, frightening, boring, or embarrassing activities that are necessary to receive care or follow health recommendations (eg, having to perform uncomfortable exercises, being around seriously ill people during visits to the hospital).

Estimating costs in both the tangible and intangible categories can be complex and is an unfamiliar process for many health care professionals. Fortunately, clear instructions for economic analyses are readily available. [2,8] Guidelines for evaluating intangible costs are not as easy to find, but some very helpful suggestions are provided by Weintein and coauthors in their discussion of utility analysis. [2] Clear examples of important intangible costs, and of the need to consider these costs in selecting a strategy the individual patient will accept, are presented in a thought-provoking and humanistic discussion of decision analysis by Bursztain et al. [9]

Step 6--Selecting a

Preferred Strategy

The end result of a decision analysis is a preferred strategy that identifies the best action to take at each choice point in the tree. To arrive at this result requires a two-dimensional comparison of alternative approaches that synthesizes the results of effectiveness analysis with the results of alternative costs analysis. A simple way to see what such comparisons may reveal is to summarize them on the sort of table shown in Figure 5. Obviously, the ideal strategy would be one that was more effective and less costly than all others. In practice, however, the best we can do may fall short of that and still be our most cost-effective alternative. An approach that is of equal cost but more effective than competing alternatives, or one that lowers cost without reducing effectiveness, may be the overall "best buy," even if it is not superior on all counts.

Some combinations of cost and effectiveness are more difficult to interpret and require additional analysis of values before we can decide what is preferable. This sort of combination is represented in Figure 5 by a question mark.

As with other steps in the analysis, excellent references are available to help us think through the issues involved. [10,11] Analyses completed by colleagues in other clinical fields, such as orthopedic surgery, also provide encouraging examples of just how informative a full cost-effectiveness comparison can be. [12]

Summary

This condensed review of a complex method can provide only a general view of what clinical decision analysis is and why it might be useful. It also should help clarify, however, what decision analysis is not: This is not a practical method for making most decisions in everyday clinical practice. The process of carrying out all the steps in full, formal analysis is much too time consuming to be applied to the dozens of important decisions clinicians make each day. The method should be used selectively to ensure that the time invested in analysis will pay off in a practical way in practice. Formal analyses should be attempted only for decisions that are made frequently, have important consequences, and provoke some sort of significant controversy, uncertainty, or discontent with the results less formal decision making achieves. Decision analysis is not a substitute for all other methods of decision making. To suggest that this is the only tool clinicians need would be a little like suggesting that a balanced diet could consist exclusively of broccoli. Intuitive judgments, use of theoretical models, knowledge-problem coupling, and hypothesis-oriented algorithms for clinicians all provide important tools for improving patient care. Often these tools can be combined effectively with decision analysis, but at other times they will be superior on their own. Decision analysis is not an entirely unfamiliar method, and it does not always need to include all component steps to be useful. Good clinicians have always asked themselves (and their students) such questions as these: What results are you trying to produce, and what are you trying to avoid? How will you judg e the results you actually get, and when should you be able to do this? What is most likely to happen if you try this, and what might happen instead? Is there anything unusual about this patient that might change the results? What evidence do you have to support these expectations, and how confident do you feel about them? What will do if things go wrong? How much will this cost? What else could you do? Is there a better way?

We can address these questions one at a time, but now and then we should consider them in combination, using a comprehensive, systematic method of thinking about practice such as decision analysis. When we do this for carefully selected problems, it reminds us these questions all need to be asked, helps us hone our skills in finding answers, let us share our thinking and experience with each other, and provides a framework for future research and for utilization of research to improve practice. As I look at the growing number of decision analyses now reported in the medical literature, I dream of the day when we may see decision trees as a regular feature of our own professional writing and relish the thought of how such analyses could enrich and enliven our practice.

References

[1] Raiffa H: Decision Analysis: Introductory Lectures on Choices Under Uncertainty. Reading, MA, Addison-Wesley Publishing Co Inc, 1968

[2] Weinstein MC, Fineberg HV, Elstein AS, et al: Clinical Decision Analysis. Philadelphia, PA, W B Saunders Co, 1980

[3] Watts NT: Decision analysis: A tool for improving physical therapy practice and education. In Wolf SL (ed): Clinical Decision Making in Physical Therapy. Philadelphia, PA, F A Davis Co, 1985, pp 7-23

[4] Coogler C: Clinical decision making among neurological patients: Spinal cord injury. In Wolf SL (ed): Clinical Decision Making in Physical Therapy. Philadelphia, PA, F A Davis Co, 1985, pp 149-170

[5] Sterler C, Maram G: Evaluating research findings for applicability in practice. Nurs Outlook 24:559-563, 1976

[6] Cebul RD, Bek LH (eds): Teaching Clinical Decision Making. New York, NY, Praeger Publishers, 1985

[7] Francis K: Computer communication: Decision analysis using a spreadsheet. Phys Ther 68:1409-1410, 1988

[8] Drummong MF: Principles of Economic Appraisal in Health Care. Oxford, England, Oxford University Press, 1980

[9] Bursztajn H, Feinbloom RI, Hamm RM, et al: Medical Choices, Medical Chances: How Patients, Families, and Physicians Can Cope with Uncertainty. New York, NY, Delacorte Press, 1981

[10] Drummond MF, Ward H: Assessing the "value for money" from rehabilitation programmes. Physiotherapy Practice 4:30-40, 1988

[11] Levin HM: Cost-effectiveness: A Primer. Beverly Hills, CA, Sage Publications Inc, 1983

[12] Soboroff SH, Pappius MS, Komaroff AL: Benefits, risks, and costs of alternative approaches to the evaluation and treatment of severe ankle sprain. Clin Orthop 183:160-168, 1984

N Watts, PhD, FAPTA, is Professor, Program in Physical Therapy, MGH Institute of Health Professions, 15 River St, Boston, MA 02108-3402 (USA).