How should treatments be critiqued for scientific merit?

The desire to embrace a new treatment approach, often simply because it is new and different and represents a change from the clinician's standard everyday practice, is a behavior that is probably common to physical therapists throughout their careers. As Rothstein reflected in recalling his first exposure to manual therapy:

Suddenly hands could do what they could not do before. I had a heady sense of power and a burst of confidence -- and a deep-seated feeling that my practice would never be the same. I knew in my heart that I could do more than I had ever hoped to do before and that my patients would benefit from my newfound abilities. So it was when I was first introduced to the world of manual therapy, a world that had not been part of my entry-level education.(1) (p939)

As members of a helping profession, our tremendous desire to improve the quality of lives of the clients with whom we work is exemplified by Rothstein's enthusiasm about his introduction to manual therapy.

Unfortunately, our desire to fervently embrace new treatments often overshadows our responsibility as ethical practitioners to examine the theoretical grounding and evidence underlying these new techniques. Too often we are satisfied that if we learn the new treatment approach by attendance at a continuing education course given by an acknowledged "expert" in the field, our role as responsible consumers of this new knowledge has been achieved. As Rothstein, however, concluded in his editor's note on manual therapy, "We do a disservice to the pioneers of manual therapy when we worship their words and fail to advance the scientific basis of what they first developed."[1] (p841)

In a provocative editorial entitled "Science or Cult?" published more than 30 years ago in this journal, one of the foremost leaders in our profession challenged the devotees of widely advocated methods of physical therapy to record, document, and publish data-based evidence for the procedures that they were advocating.(2) Hislop's editorial engendered lively debate and commentary from a number of clinicians and academicians, most of whom took issue with her criticism of the lack of scientific evidence for physical therapy practice.(3-6) Even those who wrote in support of Hislop's comments stated that the responsibility for evaluating our treatment methods rested with the researchers in physical therapy rather than with the clinicians.4

As both a practicing clinician and a clinical researcher, I believe that the responsibility to deliver evidence-based treatment techniques rests with all physical therapists. To fully understand what constitutes evidence-based treatment, it is helpful first to examine the characteristics of "nonstandard" or alternative therapies.(7) Unfortunately, there are a fair number of nonstandard therapies currently promulgated by physical therapists and other health care professionals that provide representative examples of many of these characteristics. The purposes o this perspective, therefore, are (1) to list and discuss characteristics of nonstandard therapies, with representative examples of these characteristics that are pertinent to physical therapists, and (2) to provide criteria for evaluating the scientific merit of a new (or existing) treatment approach. The overall goal of this article is to provide physical therapist clinicians, educators, and researchers with straightforward and pragmatic strategies for critically analyzing the scientific merit of physical therapy treatments.

Characteristics of Nonstandard Therapies In an article reviewing nonstandard therapies for children with developmental disabilities, Golden7 outlined five characteristics that typify nonstandard treatment approaches. Although written in reference to therapies for children with developmental disabilities, these characteristics are applicable to any type of nonstandard treatment, including several that are used by physical therapists. These five characteristics are listed together with specific examples of how each characteristic can be applied to nonstandard treatments currently used by physical therapists.

1. Treatments are based on theories that are incongruent with anatomical or physiological function. The first characteristic of a nonstandard treatment approach is that the theory underlying the treatment techniques is not supported by current concepts of human anatomy and physiology.(7) For example, a recent review article describing craniosacral therapy stated that the craniosacral system is "the primary respiratory mechanism" of the body.[8] Needless to say, this represents a significant departure from what I was taught in my undergraduate anatomy and physiology course more than 25 years ago (or what is published in anatomy and physiology texts today). This article goes on to state that emotional feelings and memories of past traumas are stored in the connective tissue. Once again, current neuroanatomy and neurophysiology texts such as that by Gilman and Newman[9] would suggest that emotions and feelings are stored in neural tissue, not in the connective tissue.

The failure to provide theoretical support based on current concepts of anatomy and physiology should provide the first warning sign that the treatment approach lacks scientific credibility.

2. Treatment is said to be effective for a broad range of diagnoses. The second characteristic of a nonstandard therapy is that it claims to be effective for a broad range of diagnoses. A 1995 continuing education brochure advertising myofascial release seminars states that myofascial release can be used to evaluate and treat the following conditions: birth trauma, cerebral palsy, headaches, temporomandibular problems, scoliosis, cervical and back pain, fibromyalgia, menstrual pain, neurological dysfunction, sports injuries, recurring injuries, restriction of motion, and chronic fatigue syndrome.(10) Similarly, a relatively new rehabilitation treatment labeled therapeutic electrical stimulation (TES), which involves the nighttime application of low-level electrical stimulation, is reported to reverse bladder incontinence and improve bowel sensation in individuals with incomplete spinal cord injury or spina bifida, stimulate new muscle and bone growth and decrease spastic "lock-up" in clients with spasticity, and enable patients with cerebral palsy to walk without crutches and improve their fine motor skills." Although the primary proponent of TES is a physician, a number of pediatric physical therapists are embracing this approach enthusiastically for their young clients - even though there is, as yet, no controlled experimental support for its efficacy.

This "one-size-fits-all" mentality for any treatment approach should provoke immediate skepticism in practitioners who are considering its use.

3. Treatments are incapable of causing harm. When, as an entry-level physical therapy student, I first learned about different physical therapy modalities, we typically were taught the indications, contraindications, and side effects of that treatment. Recent and ongoing research by physical therapists continues to document potentially serious side effects of some of our most conventional treatments.12 In contrast, proponents of nontraditional treatment approaches often minimize the risk or potential for harmful side effects.

In the recent review article describing craniosacral therapy, for example, no specific side effects of this treatment were mentioned, although a few contraindications were listed; craniosacral therapy was deemed "a gentle, potent, and safe treatment.[8](p531) Although side effects are usually thought of as harmful, a proponent of TES has reported positive "side effects," such as the reversal of bladder incontinence and improved bowel sensation in patients with incomplete spinal cord injury and spina bifida.[11]

4. No adequately controlled studies in peer-reviewed journals. The lack of scientific support, as evidenced by the failure to publish well-controlled studies in peer-reviewed research journals, is an all-too-common characteristic of nontraditional treatment approaches. More typically, the proponents of nonstandard approaches cite publications in newsletters or bulletins to support their treatments.13,14 Often, the therapy approaches are described in glowing terms in family magazines[15] or are highlighted in clinical journals that lack a research focus.[11,16]

Although some proponents of nonstandard treatments have succeeded in publishing their work in peer-reviewed journals, the "studies" may appear to be more rigorous than they actually were. The developers of TES, for example, recently published a study with a purported single-subject research design examining the effects of TES on gross motor skills in five children with mild cerebral palsy.[17] This "pilot study" failed to satisfy some of the most basic criteria required of a single-subject research design, such as using repeated measures throughout each phase of the study, including visual analysis of graphed data, and establishing interrater reliability of the outcome measures during the course of the study. Basically, this article described a series of case reports and was not a single-subject design, as was claimed by the authors.

Another problem that occurs with nonstandard treatment approaches is when proponents of those approaches fail to cite studies that have been published in reputable peer-reviewed journals because those studies failed to support the efficacy of the treatment. At a major motor control conference in 1990 that attracted nearly 400 physical therapists from around the world, the speaker who had been invited to discuss and review current research on motor control problems in the elderly limited her entire presentation to a description of the Feldenkrais method, an exercise approach for enhancing movement awareness. Not a single controlled study or peer-reviewed reference was cited in this presentation nor in its published version in the conference proceedings.[18] Noticeably absent from the reference list was a study published 13 years earlier in the Journal of Gerontoloqy that had failed to find any significant objective benefits of the Feldenkrais method (or of a conventional exercise approach) when compared with a no-treatment control group; objective outcomes assessed in this study included a variety of fitness measures and activities of daily living.[19] The omission of a well-controlled study that had been published in a respected peer-reviewed journal is an important one - particularly when the only references include in the published proceedings were five publications from sources that were not peer reviewed (eg, two books written Feldenkrais, the originator and major proponent of the treatment approach, and an article from Physical Therapy Forum).

The lack of scientific support for some of today's more popular manual therapies, such as myofascial release, cranio-sacral mobilization, and somatoemotional release techniques, does not seem to limit the enthusiasm with which physical therapists embrace these treatments.[20] A recent well-designed and well-controlled study, published in this journal, suggested that the interrater reliability of measurements of craniosacral rate made by experienced craniosacral therapists was very poor and that the measures of craniosacral rate bore little relationship to heart rate and respiratory rate.[21] The lack of consistency in measuring craniosacral rate, one of several bases on which treatment decisions are made, suggests that this measure has limited validity as an evaluation technique for making subsequent treatment decisions.[21]

Although this important study did not address the question of whether craniosacral therapy actually works," the results certainly cast doubt on the reliability of one of the primary assessment procedures that is used for treatment planning. As the authors concluded in their response to the commentaries on their article, "If we have succeeded in demonstrating the necessity for physical therapists to continue to examine craniosacral motion empirically, then we will be satisfied that our article has achieved its primary purpose."[22](p920)

5. The therapies have an emotional appeal, and studies that fail to confirm their effectiveness are attacked. Rothstein's "heady sense of power and burst of confidence" after being introduced to "the world of manual therapy"[1](p839) succinctly describes the emotional appeal engendered by many of the nontraditional treatment approaches. These nonstandard therapies are often described as a salvation for persons who, according to the promulgators of those therapies, are unable to receive ongoing, standard treatments. As Dr Karen Pape, the primary proponent of TES, has explained:

Because there is nothing offered to patients with chronic disorders, we decided to look at the technologies that we could develop and then train the disabled to train themselves. You can strengthen muscle with electricity, or you can go to a gym and strengthen it. Since most of these people can't go to gyms, and there are no physiotherapists available for chronic, long-term, lifetime care, it seems a whole lot more sensible to use the technology.[23](p46)

In response to the comment that some physicians remain skeptical about the value of low-intensity electrical stimulation for persons with chronic neuromuscular disorders and suggest that it may only be a placebo effect, Pape remarked, "I say, bless it. I don't care if it is a placebo effect. I don't believe that this makes a bit of difference, as long as the patients get better."[23](p47) What she failed to add when making these comments is that consumers pay $950 for the stimulator that delivers what may be a "placebo effect" as well as $200 per year for the electrodes. These are merely the equipment costs, of course, and do not include the professional fees for evaluations and training in the use of the equipment.

Not surprisingly, recent studies published in this journal that cast doubt on the reliability of the assessment of craniosacral motion[2l] and of Cyriax's soft-tissue diagnosis system for the knee[24] resulted in letters to the editor suggesting that the studies were somehow flawed.[25,26] In his response to the study showing that measures of craniosacral motion have poor reliability, Upledger stated, "I am convinced that we should not allow strict adherence to the rules of experimental design to fetter human intelligence, nor should we allow it to stifle our creativity."[25](p329)

It will come as no surprise that I strongly disagree with Dr Upledger's comments. It is high time for physical therapists to ensure that the treatments that they are endorsing and providing for their clients are based on the strictest rules of experimental design and scientific evidence. Each physical therapist has a responsibility not only to his or her clients but also to his or her health professional colleagues and health care funding sources to provide interventions that are sound in theory and also supported by well-controlled experimental studies.

To that end, physical therapists should be able to judge the scientific merit of any new (or existing) treatment approach. The next section will list and discuss six specific criteria for evaluating the scientific merit of a new or existing treatment. Not surprisingly, these criteria are almost the reverse of the characteristics that define nonstandard therapies.

Criteria for Evaluating the Scientific

Merit of a Treatment

In order to ethically practice physical therapy, we must offer our clients treatments that have sound theoretical and scientific bases. Before administering a treatment, particularly a new technique, to clients in your caseload, determine first whether the treatment meets the following six criteria. Furthermore, before signing up for a continuing education course that provides training in a particular treatment approach, ask the instructors or course coordinators to provide written evidence (eg, a bibliography of peer-reviewed articles) that the approach satisfies these same criteria. As Dean commented a decade ago in discussing the role of continuing education for physical therapists, A course based on clinical experience alone has little place in contemporary continuing education programs in the profession."[27](p1066)

In examining these six criteria, one of the standard treatments used in physical therapy - namely, therapeutic ultrasound - will be used as an example. Several recent studies involving the use of therapeutic ultrasound will be cited to highlight these criteria.

1. The theories underlying the treatment approach are supported by valid anatomical and physiological evidence. Used in medicine for more than 50 years, ultrasound's biophysical effects can be classified as either thermal or nonthermal.[28] Therapeutic ultrasound is most commonly used by physical therapists for its deep-heating effects. The theoretical basis for the clinical use of ultrasound is that it will increase tissue metabolism and blood flow to skeletal muscles,[29] increase the extensibility of collagen tissue, increase the pain threshold, and promote changes in contractile activity of skeletal muscle.[28] Extensive animal research over the past half-century has provided valid anatomical and physiological evidence of ultrasound's effects.

2. The treatment approach is designed for a specific type of patient population. Because ultrasound has been shown to selectively increase temperatures of periarticular structures, it is used clinically in the treatment of joint contractures, scar tissue, tendinitis, and bursitis.[28] These are specific types of impairments that can result from either degenerative diseases or traumatic injuries.[29] Other clinical uses of ultrasound have been in the treatment of pressure sores and stasis ulcers and in phonophoresis (to drive topical drugs through the skin and into underlying tissue).[28]

3. Potential side effects of the treatment are presented. Precautions in the use of ultrasound have been well described and well documented over the years.[28,29] Recent experimental research[12] has demonstrated that continuous therapeutic ultrasound increases both the volume and weight of solid malignant tumors in mice, leading the researchers in this important and well-controlled study to express concern about the use of continuous therapeutic ultrasound in humans with tumors or suspected tumors.

The importance of these study results in outlining potentially serious side effects of ultrasound treatment was highlighted as well in an accompanying commentary: "Because ultrasound is a very commonly used modality in physical therapy practice, it is important to know that ultrasound may indeed exacerbate cancers."[30](p11)

4. Studies from peer-reviewed journals are provided that support the treatment's efficacy. A recent study reported in Physical Therapy compared the effects of several different treatment combinations on wound healing in clients with spinal cord injury.[31] The effects of nursing care alone (control group) were compared with the effects of nursing care in conjunction with either laser treatment or ultrasound combined with ultraviolet-C. Results of this randomized controlled trial showed that the combined ultrasound/ ultraviolet-C treatment had a greater effect on wound healing than nursing care alone or nursing care in combination with laser treatment. This well-controlled study provides support for the use of ultrasound in conjunction with ultraviolet-C in the treatment of clients with spinal cord injury.

5. Peer-reviewed studies include well-designed, randomized, controlled clinical trials or well-designed single-subject experimental studies. Ideally, published studies should include standardized, well-described intervention techniques as well as objective and reliable outcome measures. Inclusion of a control group, or a control condition in the case of single-subject experimental studies, is also important. Blinded outcome evaluations are a necessity in any type of efficacy research in order to minimize examiner bias. Finally, when assessing a new treatment approach, comparison with existing treatment approaches for this condition is desirable.

Most of these criteria were attained in a recent randomized, controlled clinical trial by Ciccone and colleagues[32] that was designed to examine the effects of ultrasound and ultrasound with trolamine salicylate (administered by phonophoresis) on delayed-onset muscle soreness (DOMS) in 40 young adult women who under-went repeated eccentric contractions of the elbow flexors. The subjects who received only the ultrasound experienced an increase in DOMS, whereas the subjects who received the ultrasound in conjunction with the salicylate did not experience an increase in muscle soreness.

The authors succeeded in standardizing and clearly describing their treatment procedures. Reliability of the outcome measures was addressed. A control group receiving sham ultrasound and placebo cream was included. Subjects were blinded to both the ultrasound treatment (whether the machine was on or off) and the type of cream used (salicylate versus placebo). Finally, the use of groups that received standard ultrasound versus ultrasound combined with salicylate administered through phonophoresis provided an opportunity to compare a newer treatment with an existing or standard treatment.

6. The proponents of the treatment approach ore open and willing in discussing its limitations. In presenting their results, conclusions, and clinical implications, the authors of each of the studies cited[12,31,32] provided careful and balanced perspectives on both the advantages and limitations of therapeutic ultrasound. Accompanying commentaries to each of these articles, written by acknowledged experts in the use of therapeutic ultrasound, also openly discussed both the benefits and limitations of this widely used modality.[30,33,34]

Readers are encouraged to review these articles and the commentaries that follow them in order to derive their own conclusions on the scientific merit of these studies.

Concluding Comments

Whose responsibility is it to decide whether or not to use a new or existing treatment approach? According to Golden,[7] it is the responsibility of the proponents of a nonstandard treatment approach to demonstrate the treatment's effectiveness. It should also be the responsibility of continuing education instructors to provide written evidence (eg, a bibliography of peer-reviewed studies) that the treatment about which they are teaching has been empirically examined and that the measures used to evaluate its effectiveness have satisfied accepted standards in physical therapy practice.[35]

The ultimate responsibility, however, in refraining from using treatment approaches that have not been subjected to scientific scrutiny resides with all physical therapists. I believe that we, as a profession, are at an important crossroads in our development. We can choose to remain nonreflective practitioners, learning about and practicing new treatment techniques because they are new and different and exciting, or we can move forward as professionals - with sound theoretical and scientific support underlying our treatment practices.

My personal belief is that we have a moral and ethical responsibility to take the more challenging road and move forward as professionals. As Golden concluded in his discussion of nonstandard therapies:

The scientific approach to scientific problems cannot be replaced by untested hypotheses leading to undocumented therapies. Bronowski[36] stated that, "Magic is a technology, technology without science." Each person must decide whether to practice science or magic.[7](p491)

If we truly believe in providing client-centered care, then it is our duty "...both to the patient and to the funding source to be knowledgeable and to be in possession of the relevant scientific facts in order to act in the best interest of the patient."[37](p755) Each physical therapist must decide whether to provide ethical and responsible treatments that are based upon the latest scientific evidence or to provide therapies whose value is based solely upon anecdotal testimonials. I hope that you will join me in traveling down the more challenging road and that you, too, will decide to practice science rather than magic.

Finally, it is my hope that this perspective will assist physical therapists to become the kinds of ethical, reflective, and caring practitioners, teachers, and researchers that our clients, health care colleagues, and funding agencies expect and deserve.

References

[1] Rothstein JM. Editor's note: Manual therapy - a special issue and a special topic. Phys Ther. 1992;72:839-841. [2] Hislop HJ. Editorial: Science or cult? J Amer Phys Ther Assoc. 1963;43:163 - 165. [3] McGarrett AL. Letter to the editor on "Editorial; Science or cult" J Amer Phys Ther Assoc. 1963;43:455. [4] Forward E. Letter to the editor on 'Editorial: Science or cult?" J Amer Phys Ther Assoc. 1963;43:455. [5] Keelan VB. Letter to the editor on "Editorial: Science or cult?"J Amer Phys Ther Assoc. 1963;43:455-456. [6] Jones MB. Letter to the editor on "Editorial: Science or cult? " J Amer Phys Ther Assoc. 1963;43:456. [7] Golden GS. Nonstandard therapies in developmental disabilities. Am J Dis Child. 1980;134:487-491. [8] Hollenberg S, Dennis M. An introduction to craniosacral therapy. Physiotherapy. 1994;80:528-532. [9] Gilman S, Newman SW. Manterand Gatz's Essentials of Clinical Neuroanatomy and Neuropysiology 8th ed. Philadelphia, Pa: FA Davis Co; 1992:225-226. [10] The John F Barnes' Myofascial Release Approach. Paoli, Pa: Myofascial Release Treatment Centers & Seminars; 1995. [11] Pape KE, Kirsch SE, Bugaresti JM. New therapies in spastic cerebral palsy. Contemporary Pediatrics. May/June 1990:6-13. [12] Sicard-Rosenbaum L, Lord D, Danoff JV, et al. Effects of continuous therapeutic ultrasound on growth and metastasis of subcutaneous murine tumors. Phys Ther. 1995;75:3-11. [13] Schacht WD. Cited by: Jackson OL. The Feldenkrais method: a personalized learning model. In: Lister MJ, ed. Contemporary Management of motor Control Problems: Proceedings of the II STEP Conference. Foundation for Physical Therapy Inc; 1991:131-136. [14] Barnes JF. Pediatric myofascial release. Physical Therapy Forum. July 19, 1991:9-10. [15] Lechky O. New technique helps reverse effects of brain injuries. Canadian Living. September 1992:153. [16] Reynolds JP. Profiles in alternatives. PT - magazine of Physical Therapy. 1994;2(9):52-59. [17] Pape KE, Firsch SE, Galil A, et al. Neuromuscular approach to the motor deficits of cerebral palsy: a pilot study. J Pediatr Orthop. 1993; 13:628-633. [18] Jackson OL. The Feldenkrais method: a personalized learning model. In: Lister MJ, ed. Contemporary Management of motor Control Problems: Proceedings of the II STEP Conference. Alexandria, Va: The Foundation for Physical Therapy Inc; 1991:131-136. [19] Gutman GM, Herbert CP, Brown SR. Feldenkrais versus conventional exercises for the elderly. J Gerontol 1977;32:562-572. [20] Reynolds JP. Profiles in alternatives. Peeling the layers of the onion - myofascial release. PT - Magazine of Physical Therapy. 1994;2(9):55-57. [21] Wirth-Pattullo V, Hayes KW. Interrater reliability of craniosacral rate measurements and their relationship with subjects' and examiners' heart and respiratory rate measurements. Phys Theraphy. 1994;74:908-916. [22] Wirth-Pattullo V, Hayes KW. Author response to invited commentaries on "Interrater reliability of craniosacral rate measurements and their relationship with subjects' and examiners' heart and respiratory rate measurements." Phys Ther. 1994;74:920. [23] Pape KE. Quoted by: Lowry F. The Magee clinic: new approaches to self-care. Wellness MD. 1992;2(4):46-47. [24] Hayes K, Petersen C, Falconer J. An examination of Cyriax's passive motion tests with patients having osteoarthritis of the knee. Phys Ther. 1994;74:697-707. [25] Upledger JE. Letter to the editor on "Interrater reliability of craniosacral rate measurements and their relationship with subjects' and examiners' heart and respiratory rate measurements." Phys Ther. 1995;75:328-329. [26] Bowling RW, Erhard RE. Letter to the editor on An examination of Cyriax's passive motion tests with patients having osteoarthritis of the knee." Phys Ther. 1994;74:1073-1074. [27] Dean E. Psychobiological adaptation model for physical therapy practice. Phys Ther. 1985;65:1061-1068. [28] Ziskin MC, McDiarmid T, Michlovitz S. Therapeutic ultrasound. In: Michlovitz, S, ed. Thermal Agents in Rehabilitation. 2nd ed. Philadelphia, Pa: FA Davis Co; 1990:134-169. [29] Lehmann JF, DeLateur BJ. Therapeutic heat. In: Lehmann JF, ed. Therapeutic Heat and Cold. 3rd ed. Baltimore, Md: Williams & Wilkins; 1982:404-562. [30] Reed BV. Invited commentary on "Effects of continuous therapeutic ultrasound on growth and metastasis of subcutaneous murine tumors." Phys Ther. 1995;75:11-12. [31] Nusbaum EL, Biemann I, Mustard B. Comparison of ultrasound/ ultraviolet-C, and laser for treatment of pressure ulcers in patients with spinal cord injury. Phys Ther. 1994;74:812-823. [32] Ciccone CD, Leggin BG, Callamaro JJ. Effects of ultrasound and trolamine salicylate phonophoresis on delayed-onset muscle soreness." Phys Ther. 1991;71:666-675. [33] Michlovitz S. Commentary on "Effects of ultrasound and trolamine salicylate phonophoresis on delayed-onset muscle soreness." Phys Ther. 1991;71:675-676. [34] Michlovitz S, Buinewicz B. Invited commentary on "Comparison of ultrasound/ultraviolet-c and laser for treatment of pressure ulcers in patients with spinal cord injury." Phys Ther. 1994;74:824-825. [35] Task Force on Standards of Measurement in Physical Therapy. Standards for tests and measurements in physical therapy practice. Phys Ther. 1991;71: 589-622. [36] Bronowski J. Magic, Science, and Civilization. New York, NY: Columbia University Press; 1978. [37] Hays RM. Health care ethics and pediatric rehabilitation. Physical Medicine and Rehabilitation Clinics of North America. 1991;2:743-763.