Outcomes research: shifting the dominant research paradigm in physical therapy.

Key Words: Outcome and process assessment (health care), Quality of health care, Research, Research design.

Introduction: The Third

Revolution in the

Modern Medical Era

Arnold Relman,[1] past editor of the New England Journal of Medicine, has described the ascendancy of outcomes research as the last of three revolutions in the modem age of medicine as practiced in the United States. First, he explains, came the Era of Expansion, from the end of World War II through the late 1960s, the hallmark of which was the expanding role of the US federal and state governments in the financing of medical facilities and medical services. This first era was highlighted by a fundamental change in the federal government's Social Security Act: the passage of Titles 18 and 19 (Medicare and Medicaid), which fundamentally changed the role of the federal government in financing health care for many of its citizens. By 1990, the 25th anniversary of Medicare and Medicaid, the total expenditure of the US federal government for health care was in excess of $150 billion, up from $5.6 billion in 1965.[2]

This first modern era of medicine, Relman[1] explains, was followed by the Era of Cost Containment, as the increase in health care expenditures grew relentlessly through the 1970s and 1980s (reaching $620 billion in 1989, 11.2% of the gross national product). A series of cost-containment regulations and legislative actions were enacted during the 1970s and 1980s in response to the growth in health care expenditures. One example is the introduction of the diagnostic related groups (DRGs) as the basis for hospital reimbursement under the Medicare program. As noted by Relman, we are currently in the third stage of this modern revolution, the Era of Assessment and Accountability, in which the focus has been directed toward the quality and effectiveness of health care. The emphasis is no longer on unbridled growth nor on blind cost containment, but on a balance between assessment of gains achieved for certain costs and an accountability for those costs incurred.

Relman[1] has described the outcomes movement in medicine" as characterizing this third stage in modern medical care. Its proponents have defined outcomes management in medicine as the centerpiece of this era of assessment and accountability.[3,4] It is a technology of patient experience designed to help patients, payers, and providers make rational medical care-related choices based on better insight into the effect of these choices on the patient's life. Today, outcomes research is being used in medicine, and increasingly in physical therapy, to justify policies regulating practice, including consensus statements, practice guidelines, and practice protocols, which are being offered increasingly as standards for third-party reimbursement and malpractice protection.[5,6]

The outcomes movement within medicine has also stimulated its share of controversy. As Anderson recently commented in the journal Science, "...5 years and $200 million later, critics are asking: Where's the beef.?"[2.sup.1080] Has the US Congress been sold as a bill of goods? Critics have argued that after spending nearly $200 million on outcomes research, the Agency for Health Care Policy and Research cannot point to a single case in which database studies have changed general clinical practice.[2] Or, as Tanenbaum[7] has argued, in contrast to many European nations that have opted for budgetary regulation, US physicians have chosen the path of behavioral regulation - through utilization review, practice guidelines, and outcomes research - as the major regulatory strategy for accountability and cost containment. And in doing so, they have sacrificed considerable clinical autonomy in favor of more economic autonomy.

What lessons, if any, do these outcomes research developments in medicine have for physical therapy.? Is outcomes research a revolutionary approach, representing a new paradigm for clinical research as its many vocal advocates have argued.[1,3,4,8] IS it the direction that physical therapy research should be heading toward? Or, as others[2,7] have suggested, is it merely a regulatory strategy that devalues clinical judgment?

To present my perspective on these questions, I will focus on several points in this article. First, I will describe the conceptual roots of outcomes research within a quality-of-care framework. Next, I win contrast outcomes research against the traditional impairment research often conducted in physical therapy and show how clinical outcomes of importance in physical therapy research are shifting from impairments to disabilities. Finally, I will present some thoughts on the potential for outcomes research to shift the dominant research paradigm within physical therapy toward the development and testing of theories related to the pathogenesis of disability and how this could have a lasting and profound impact on clinical practice.

Determining Quality of Physical

Therapy

Donabedian[9] has provided a framework that can be used to discuss assessing quality of physical therapy care. Quality of care, according to Donabedian, can be described as the ability to achieve desirable objectives (eg, various states of health) using legitimate means (in this case, various aspects of physical therapy care). His conceptual approach to assessing quality of care provided by any health provider includes measurement of three elements: structure, process, and outcomes of that care.

Structural evaluation deals with stable resources needed to provide care. Structural evaluation criteria address issues such as provider qualifications, administrative organization, and facilities. One example of structural quality-of-care criteria in physical therapy in the United States is the Commision on Accreditation of Rehabilitation Facilities' (CARF) guideline on a facility utilizing competent, ethical, and qualified personnel who contribute to shaping and accomplishing its mission. Another example is the Joint Commission on Accreditation of Healthcare Organizations' (JCAHO) guideline that hospitals have the required number and mix of staff members in each unit, area, and department to provide for patient needs. The major criticism of structural criteria is that they are necessary but not sufficient criteria for adequate care.

Process evaluation of quality of care consists of evaluating the degree to which services provided to patients meet professional standards of quality. Quality assurance systems using process measures (such as the professional standards review organizations) are in widespread use throughout the medical care system in the United States. A process evaluation example in physical therapy would be the development and use of practice guidelines or practice screens designed to identify outlier practice and to bring them into compliance with existing standards for practice in the field. A major criticism of the process approach to determining quality of care, however, is that such practice standards could actually increase the delivery of certain aspects of care that may not ultimately affect the patient's outcome. In other words, optimal process of care does not ensure an optimal outcome unless the link between the two has been established.

Ultimately, if one wants to improve the outcome of care, one needs to address Donabedian's third element in quality of care, the determination of outcomes directly in terms of death, morbidity, disability, or quality of life.[9] A positive patient outcome, as this is ultimately the general goal of medical care, has considerable face validity as a measure of the quality of care. Defining a positive patient outcome, however, is no easy task.

Physical therapy, like other therapies, is deemed valuable when it provides important benefits. One of the important contributions of the outcomes research in medicine has been the expansion of the scope of outcomes deemed useful.[10] This research has focused attention on the consideration of outcomes of value to patients, to payers, and to clinicians as an aid to clinical decision making and to influencing health care policy. Outcomes research in medicine has forced medical researchers to examine outcomes other that disease and mortality rates[10,11] to include impacts such as patient function and health-related quality of life. Such a broadening of outcomes can certainly be viewed as revolutionary within medicine; the situation, however, is different within physical therapy, where considerations of patient function and disability outcomes are anything but revolutionary.

Because the broadening of relevant and valued outcomes may not be as nontraditional in physical therapy as it is in medicine, one might ask whether outcomes research in physical therapy really offers something new? Or alternatively, is it anything more than what has always been considered as "good" clinical research? Is there something important going on in outcomes research in physical therapy, or is it the latest fad? I argue that the ascendancy of outcomes research in physical therapy does have relevance and importance to physical therapy, not only in similar ways to the ascendancy of outcomes research in medicine but also in ways that are unique to research to our profession.

Physical therapists have always advocated and respected patient-level goals, as reflected in our traditional emphasis on maximizing patient function.[12] The importance of patient function is certainly seen in contemporary definitions of physical therapy such as that put forth by Sahrmann,[13] who defined physical therapy as focusing on movement dysfunction, defined as imbalance or restriction in the movement of segments, limbs, or the whole body [emphasis added]. Physical therapy research, however, has not always matched the profession's own rhetoric. Although it is axiomatic within physical therapy to advocate patient-level outcomes, these outcomes have not been widely reflected in our research.

Research Paradigms in

Physical Therapy

Kuhn,[14] in his classic work on scientific change, 7be Structure of Scientific Revolutions, introduces the concept of research paradigms. Kuhn argued that one of die characteristics of science is what he calls a paradigm, an accepted framework that governs the way in which the science operates. Dominant paradigms guide the development and conduct of research within fields of inquiry. Advances within an area of science, Kuhn argues, are typically characterized by a shift from the current one to a new paradigm. He describes, for example, the shift from an earth-centered astronomy advocated by the astronomer Ptolemy (where the sun was believed to revolve around the earth) to die Copernican view of a sun-based astronomy as a classic illustration of a major paradigm shift that had major consequences in astronomy.

Although designed to describe change in pure and not applied sciences, Kuhn's concept of shifting research paradigms may be helpful in describing traditional physical therapy clinical research and what outcomes research could contribute to the science and practice of physical therapy. I believe a shift in the dominant research framework (or paradigm) guiding physical therapy research needs to occur, and if it does, this shift could have profound implications for the development and testing of theory and ultimately could contribute to the body of knowledge guiding physical therapy practice. I will try to illustrate how a research paradigm shift could stimulate the development of physical therapy research that examines the relationship between impairments and disability, thereby developing new knowledge about the pathogenesis of disability.

To describe the shift in research paradigm being advocated, I first will illustrate what I see as the traditional framework for physical therapy research. Concepts in the disablement model developed by Saad Nagi,[15,16] or the World Health Organization's International Classification of Impairments, Disabilities, and Handicaps,[17] provide an excellent lens through which we can view the traditional research paradigm guiding physical therapy research. Viewed from the terminology used within these conceptual frameworks, the dominant research paradigm guiding physical therapy research and practice can be characterized as one that focuses on impairment outcomes, defined as loss or abnormalities in the physiologic or anatomic structure within specific body organs or systems within the organism. Examples familiar to physical therapists would be research that focuses on outcomes such as restriction in range of motion, muscle weakness, or pulmonary function.

Less commonly included as outcomes within research conducted by and of interest to physical therapists is another concept within these disablement frameworks, variously described as functional limitations, disability, or handicap. This concept, which I will refer to as disability, focuses outcome attention at the level of the individual's behavior or his or her functioning in social roles within society (see Jette[18] for a full discussion of this concept).

One example of the traditional impairment research paradigm guiding research in physical therapy is drawn from an article in applied physiology by Frontera et al[19] titled "Strength Conditioning in Older Men: Skeletal Muscle Hypertrophy and Improved Function." The title of the article clearly suggests that the study focused on strength conditioning's effects on two levels of outcome: skeletal muscle hypertrophy (a traditional impairment outcome) and improved function (an apparent disability outcome). The major finding of the study was that strength training in older men resulted in an increase in muscle force production (one repetition maximum) associated with muscle hypertrophy. In contrast to the impression conveyed in the article's tide, the functional outcome" addressed and reported by these investigators was muscle force generated by the knee extensors, a classic impairment outcome. The study did not address strength training's impact on individual function or disability level (eg, walking, mobility).

The development of outcomes research in medicine is affecting clinical research in physical therapy and related disciplines by moving research beyond traditional impairment outcomes (as illustrated in the article by Frontera et al[19]) such as force, range of motion, and pain to include the measurement and evaluation of broader patient outcomes such as functional limitations and disability. One example of this broadening of outcomes is illustrated by an article by Fisher et also titled "Quantitative Effects of Physical Therapy on Muscular and Functional Performance in Subjects With Osteoarthritis of the Knees." In evaluating a physical therapy-led exercise intervention with patients who had osteoarthritis of the knee, these investigators examined not only the outcome on force production (as was done in the 1988 study by Frontera et al'9) but also outcomes in patients' degree of assistance, pain, and difficulty in climbing stairs, rising from a chair, walking, and other individual behaviors (examples of disability outcomes). These investigators reported positive impacts of the exercise program on force production and individual functioning. A review of clinical investigations recently published in Physical Therapy reveals several illustrations of research that included disability as well as impairment outcomes. Examples include cardiac rehabilitation,[21] low back pain treatment,[22,23] and rehabilitation following anterior cruciate ligament reconstruction.[24]

Changing the Dominant

Impairment Research Paradigm

Although useful, clinical research in physical therapy that includes an examination of potential benefits at the level of disability outcomes as well as that of more traditional impairment outcomes, at best, brings physical therapy research more in line with the profession's rhetoric. This development does not portend a fundamental shift in the framework guiding clinical research in the profession. I believe a more fundamental shift, if it occurs in physical therapy clinical research, may ultimately have a profound impact on clinical practice. Such a shift would require a change in a fundamental assumption underlying much of clinical research relevant to physical therapy. This assumption, more often implicit in such research, is that there is a linear relationship between impairments and broader patient-level outcomes. The assumption of a positive linear relationship between reduction of impairment and reduction in disability simply hypothesizes, either implicitly or explicitly, that the more that impairments are decreased, the more disability will be reduced. Under such an assumption, for example, demonstrating that a physical therapy intervention in patients who have suffered a hip fracture produces improvements m muscle strength (a reduction m an impairment outcome) assumes that a commensurate reduction in disability (or improvement in patient functioning) will follow.

Clinicians know a linear relationship between an impairment and disability is not necessarily the case for at least two reasons. First, the relationship between improvement in force production and reduction of disability, for example, is often not linear, and beyond a certain threshold, improvement in force production may not affect function. Second, the relationship between impairments and disability is complex and is affected by many additional factors. The degree to which the impact of physical therapy on a muscle's ability to function also affects disability levels depends on a host of other factors such as the individual's psychologic response to the injury, the attitudes of significant others in the patient's social networks, and the patient's physical environment. Clinicians intuitively realize that physical therapy's impact on impairments and disabilities is affected by multiple factors such as those mentioned. This appreciation of the complex clinical reality in physical therapy, however, has not been reflected in traditional clinical research, which has remained largely bivariate in design and analysis.

What is needed in clinical outcomes research, therefore, is direct evidence of the degree to which physical therapy that affects an impairment (eg, muscle force) will also reduce disability and improve the functional outcomes of the patient (ie, in activities such as transfers, walking ability, and improved quality of life), as well as more information on the hypothesized factors affecting the impairment-to-disability relationship. This latter element requires the development and subsequent testing of theoretical formulations on the pathogenesis of disability (for an illustration of one framework, see Verbrugge and Jette's recent work[25]). This type of research will require more multivariate designs in physical therapy clinical research. Evidence from multivariate clinical research could provide direct and important clinical implications for physical therapy.

One illustration of the relevance and implications of investigating our implicit assumptions underlying the impairment and disability relationship is seen in the geriatrics literature through the work of Buchner and colleagues.[26,27] As Figure 1 adapted from Buchner et al[26] illustrates, a curvilinear relationship between measures of physical fitness (eg, muscle force) and functional status of the older person (reduced disability) is hypothesized based on the underlying theoretical concept of physiologic reserve. The curvilinear relationship in Figure 1 proposes a threshold effect exists in the force and disability relationship: Above a certain threshold of muscle force that represents the adequate physiologic reserve for certain functions, individual behavior is not limited; below that threshold level, the individual is disabled. Specific clinical hypotheses can be derived from such formulations. For example, a curvilinear relationship between muscle force and disability in elderly persons suggests that benefit from physical therapy (eg, exercise in frail older persons), in part, will depend on the status of the target group. in Buchner and colleagues' hypothetical study 1 with frail adults (displayed in Fig. 1), an exercise intervention produces a large improvement in functioning of the individuals. In study 2 with nearfrail adults, the impact on function through improvement m force production is modest, whereas in study 3 with asymptomatic adults, no improvement in individual function is observed in the face of improvements in force production. Figure 2, adapted from Buchner and de Lateur,[27] displays some actual data illustrating that a curvilinear relationship may exist between knee extensor torque and health status (assessed using a health status instrument called the Sickness impact Profile) in a sample of elderly individuals. Data such as these begin to shed light on how specific impairments and interventions that affect them may have subsequent impacts on disability levels.

Examples such as the work of Buchner and colleagues in geriatrics[26,27] illustrate several potential directions for physical therapy research. First, clinical research in physical therapy needs to continue the examination of disability outcomes in addition to impairment outcomes. This step is necessary but not sufficient to move research in our field forward. Second, physical therapy clinical research needs to explicitly state and then investigate the nature of the hypothesized relationship between different impairments and specific disabilities. included in such research is an examination of the impact of changes in impairments on change in disability and the investigation of important covariates that alter these relationships. There is a paucity of examples of such research in all the health professions' literature, not only in physical therapy. Finally, moving physical therapy clinical research in this second direction highlights the need of further theoretical frameworks on the pathogenesis of disability in various target groups seen by physical therapists. Various discussions of disability theory were illustrated in several articles in the recent special issue of Physical Therapy devoted to the topic of physical disability.[28] This work needs to be extended and further developed.

Conclusion

The outcomes research movement in medicine has already stimulated clinical researchers in physical therapy to address broad disability outcomes in addition to its traditional impairment outcomes. The belief that disability outcomes are important in physical therapy research, however, does not mean that impairment outcomes are no longer important. Both are relevant to physical therapy research and practice and to the development of the body of scientific knowledge in the profession. Although this broadening of outcomes is an important beginning, I have tried to illustrate why this step will not be sufficient. The field of physical therapy research should move beyond the mere broadening of outcomes addressed in its research and expand the nature of its inquiry about these outcomes. Research needs to include both impairments and disabilities when appropriate, and most importantly, it needs to explicity investigate the nature of the relationship assumed to exist between impairments and disability-level outcomes across various target groups. The development of theory regarding the pathogenesis of disability will guide the progression of this research, drawing upon, where appropriate, theoretical work from other fields. Shifting the dominant physical therapy research framework in this direction, I believe, could produce dramatic findings that could conceivably have profound impacts on clinical practice in our profession.

References

[1] Relman A. Assessment and accountability: the third revolution in medical care. N Engl J Med. 1988;319:1220-1222. [2] Anderson C. Measuring what works in health care. Science. 1994;263:1080-1082. [3] Ellwood P. Outcomes management: a technology of patient experience. N Engl J Med. 1988;318:1549-1556. [4] Epstein A. The outcomes movement: Will it get us where we want to go? N Engl Med. 1990;323:266-270. [5] Institute of Medicine. Guidelines for Clinical Practice: From Development to Use. Washington, DC: National Academy Press; 1992. [6] Garnick DW, Hendricks AM, Brennan TA. Can practice guidelines reduce the number and costs of malpractice claims? JAMA. 1991; 266:2856-2860. [7] Tanenbaum SJ. Sounding board: what physicians know. N Engl J Med. 1993;329:1268-1270. [8] Wennberg JE. Dealing with medical practice variations: a proposal for action. Health Aff (Millwood). 1984;3(2):6-32. [9] Donabedian A. Quality assessment and assurance: unity of purpose, diversity of means. Inquiry. 1988;25:173-192. [10] McDermott W. Absence of indicators of the influence of physicians on a society's health. Am J Med. 1981;70:833-843. [11] Tarlov AR, Ware JE, Greenfield S, et al. The medical outcomes study: an application of methods for monitoring the results of medical care. JAMA. 1989;262:925-930. [12] Rothstein JM. Editor's note: Disability and our identity. Phys Ther. 1994;74:375-378. [13] Sahrmann SA. Diagnosis by the physical therapist - a prerequisite for treatment: a special communication. Phys Ther. 1988;68:1703-1706. [14] Kuhn TS. The Structure of Scientific Revolutions. Chicago, Ill: University of Chicago Press; 1962. [15] Nagi S. Some conceptual issues in disability and rehabilitation. In: Sussman M, ed. Sociology and Rehabilitation. Washington, DC: American Sociological Association; 1965:100-113. [16] Nagi S. Disability concepts revisited: implication for prevention. In: Pope A, Tarlov A, eds. Disability in America: Toward a National Agenda for Prevention. Washington, DC: National Academy Press; 1991:309-327. [17] International Classification of impairments, Disabilities, and Handicaps. Geneva, Switzerland: World Health Organization; 1980. [18] Jette AM. Physical disablement concepts for physical therapy research and practice. Phys Ther. 1994;74:380-386. [19] Frontera WR, Meredith CN, O'Reilly KP, et al. Strength conditioning in older men: skeletal muscle hypertrophy and improved function. J Appl Physiol. 1988;64:1038-1044. [20] Fisher NM, Gresham GE, Abrams M, et al. Quantitative effects of physical therapy on muscular and functional performance in subjects with osteoarthritis of the knees. Arch Phys Med Rehabil. 1993;74:840-847. [21] Jette DU, Downing J. Health status of individuals entering a cardiac rehabilitation program as measured by the Medical Outcomes Study 36-Item Short-Form Survey (SF-36). Phys Ther. 1994;74:521-527. [22] Stratford PW, Binkley J, Solomon P, et al. Assessing change over time in patients with low back pain. Phys Ther. 1994;74:528-533. [23] Erhard RE, Delitto A, Cibulka MT. Relative effectiveness of an extension program and a combined program of manipulation and flexion and extension exercises in patients with acute low back syndrome. Phys Ther. 1994;74: 1093-1100. [24] Tovin BJ, Wolf SL, Greenfield BH, et al. Comparison of the effects of exercise in water and on land on the rehabilitation of patients with intra-articular anterior cruciate ligament reconstructions. Phys Ther. 1994;74:710-719. [25] Verbrugge L, Jette A. The disablement process. Soc Sci Med. 1994;38:1-14. [26] Buchner DM, Beresford SA, Larson EB, et al. Effects of physical activity on health status in older adults, II: intervention studies. Annu Rev Public Health. 1992;13:469-488. [27] Buchner DM, de Lateur BJ. The importance of skeletal muscle strength to physical function in older adults. Ann Behav Med. 1991;13:91-97. [28] Physical Disability. Phys Ther. 1994; 74(special issue):375-505.

AM Jette, PT, is Chief Research Scientist, New England Research Institutes, 9 Galen St, Watertown, MA 02172 (USA) ([e-mail]), and Professor of Social & Behavior Sciences, Boston University School of Public Health, MA 02215.

An earlier version of this article was presented at the Second Joint Congress of the American Physical Theraphy Association and the Canadian Physiotheraphy Association; Toronto, Ontario, Canada; June 5, 1994.

This work was supported, in part, by the Edward R Roybal Research Center on Applied Geronthology (AG11669).