Outcome first target of technology assessment.

HEALTH CARE TECHNOLOGY

In the evaluation of any type of medical service, a broad range of technology assessment methodologies can be used, ranging from sophisticated meta-analyses, to reviews of the published literature, to consensus panels, or even to the intuitive unarticulated process the provider may use at the bedside. However, no matter what the methodology, all technology assessments have a similar first step: identification of the outcome, the yardstick by which the safety and effectiveness of any medical service will be measured. On the surface, this may seen a straightforward process; however, this seemingly simple first step can be the most controversial aspect of a technology assessment. For some researchers, an intermediate outcome, such as tumor response to a novel chemotherapy, may represent the target outcome of a technology assessment project. Some diagnostic tests may be advocated because they produce stunning images of the vasculature, the brain, or whatever. However, unlike researchers, managed care companies are primarily interested in final patient outcomes, not intermediate outcomes. It is not enough to know that chemotherapy may provide a tumor response in x percent of patients. What is required is data demonstrating that the tumor response will translate into an improved survival rate, or at least will palliate some specific symptom. Likewise, unless the information from diagnostic tests can be translated into altered management of the patient, with ultimate health benefits, the beauty of diagnostic tests is considered unnecessary.

But even the focus on final patient outcomes can be controversial. Rarely can safety and effectiveness be considered in isolation; typically they are combined into some form of risk (i.e., safety)-benefit (i.e., effectiveness) ratio. Particularly when considering cancer therapies, an acceptable risk-benefit ratio may vary according to the point of view of the patient, the physician, and managed care company. For example, in the evaluation of various applications of high-dose chemotherapy and bone marrow transplantation, the patient's willingness to assume a high level of risk with uncertain but potential benefits has clashed with managed care companies' greater emphasis on knowledge of potential benefits.

In addition, technology assessment must try to keep pace with the phenomenon of shifting outcomes--i.e., as experience with a technology accumulates, expectations and relevant outcomes change. For example, at its outset, high-dose chemotherapy was presented and advocated as the only possible curative therapy for patients with metastatic or high-risk breast cancer. Given the high level of potential benefit, a high level of risk was acceptable. However, as data have accumulated, it has become apparent that high-dose chemotherapy can be expected to have curative potential in only carefully selected patients. At the same time, the advent of hematopoietic growth factors and the use of peripheral stem cell transplants has lowered the risk of the procedure. Thus, a whole new risk-benefit ratio is created, leading some to consider high-dose therapy not primarily as a curative procedure, but perhaps as an effective palliative procedure, either by prolonging survival or, as in the case of multiple myeloma, palliating the specific debilitating symptoms of bone pain.

This shift in outcome creates a whole new framework for evaluating high-dose therapy, and the technology assessment process must return to square one. When palliation is the goal, relevant outcomes must focus more on quality of life measures in conjunction with overall survival. What degree of treatment-related morbidity is acceptable in a setting of palliation? What length of increase in survival is needed to justify this therapy? This type of data is rarely included in the initial studies of a new technology.

Shifting outcomes are an inevitable by-product of the natural evolution of a technology. Only when a technology becomes widely diffused and used can its limitations and added capabilities be fully explored, leading to new niches and new expected outcomes. The Food and Drug Administration (FDA) clearly recognizes this phenomenon. Once a device or drug has received FDA approval for a specific indication, additional "off-label" uses are not regulated by the FDA. In certain instances, the evolution is incremental, with individual refinements so minor that an additional technology assessment may not seem warranted. However, over time the evolution of a technology may extend so far that it is only tenuously tethered to its original scientific and outcomes data.

The experience with continuous passive motion (CPM) machines is an example of the phenomenon above. CPM machines are a type of orthopedic rehabilitation device that continuously and passively moves the joint through a range of motions. The technique was pioneered in the 1970s by Robert Salter, MD, who did a series of animal experiments showing that the use of continuous motion promoted healing of articular cartilage. The device was widely adopted by orthopedic surgeons, who used it for a broad range of immediate postoperative indications but primarily after ligament surgery and total knee arthroplasty. The device was initially used in the immediate postoperative period, with the stated outcome being a reduction in length of hospital stay and improved long-term range of motion. Even in this initial application, CPM had shifted from its original basic science roots, because articular cartilage defects are not a component of knee arthroplasty or ligament repair.

As a reflection of the volume of business CPM represents, the Medical Device Register lists some 19 different manufacturers. CPM machines are also available for most appendicular joints. The bulk of the early literature regarding CPM consisted of case studies describing CPM as a component of various rehabilitation techniques, but initially there was little research that isolated the contribution of CPM as compared to active physical therapy and other rehabilitation techniques. In the late '80s and early '90s, it appeared that the literature was beginning to catch up to clinical practice. Several randomized controlled trials appeared that, by and large, showed that, when CPM was used in the immediate postoperative period, there was no decrease in hospital length of stay or improvement in long-term range of motion.

However, by this time, CPM had moved on, evolving into a device to be used in the home after hospital discharge, often for a prolonged time. The rationale was that CPM could be used to "jump start" the patient into active physical therapy. Some of the rationale for this application may have been related to limitations on the number of physical therapy visits imposed by managed care companies. Thus, a new outcome of CPM was its role as an alternative to active physical therapy, rather than as an adjunct. At this time, there are no controlled clinical trials that have explored and validated this new outcome, which is far afield from the original outcome of improved healing of articular cartilage.

Many providers and manufacturers assume that, once a technology has received FDA approval or has become widely diffused, any shift in outcome represents only the natural evolution of technology and thus may not warrant additional scrutiny. However, in this era of accountability and outcomes-based decision making, shifting outcomes signal that a fresh look at a technology is warranted.

Elizabeth Brown, MD, is Director, Technology Assessment and Clinical Guidelines, Aetna Health Plans, Aetna Life and Casualty Chicago, Ill. The opinions expressed in the article are those of the author and not necessarily those of Aetna Health Plans