Use evidence cautiously.

To the Editor:

I am writing this letter for 2 purposes. First, I want to comment on 2 articles in the March 2004 issue of Physical Therapy--the Research Report by Dumas et al titled "Recovery of Ambulation During Inpatient Rehabilitation: Physical Therapist Prognosis for Children and Adolescents With Traumatic Brain Injury" (1) and the Update by Aldrich and Hunt titled "When Can the Patient With Deep Venous Thrombosis Begin to Ambulate?" (2) Second, in the context of those articles, I want to briefly address the issue of evidence-based practice. Each article addressed important clinically relevant issues. Specifically, Dumas et al addressed ambulation prognosis after traumatic brain injury in children, and Aldrich and Hunt addressed the decision to ambulate after deep venous thrombosis (DVT). Each article also referred to the presence or absence of relevant evidence in literature.

In the systematic retrospective study of ambulation prognosis, the authors concluded that lower-extremity (LE) hypertonicity, brain injury, and LE injury were the best predictors of ambulation ability. Is this conclusion anything other than confirmation of an obvious commonsense appraisal of gait potential? That is, the more severe the injury and the LE dysfunction, the worse the prognosis for ambulation. Could it be otherwise?

I commend the authors for noting and defining the concepts of sensitivity and specificity, as well as acknowledging the high number of false positives and false negatives. I would like to translate the sterile notion of false positives and false negatives into clinical terms. A false positive would mean telling a child's family that their injured child is not expected to walk, but then the child does; a false negative would mean lolling them that their injured child is expected to walk, but then the child does not. Should the family be told about the incidence of these "false" inaccurate predictions?

As stated in the Editor's Note for this issue, outcome (especially predicting it accurately) "depends on a lot of things." (3) I take exception to the notion that a prognosis is anything more than an educated guess with varying probability of accuracy. Regardless of evidence from correlational studies and statistical information, each patient is a unique individual-whose outcome may or may not conform to that of the statistically "average" one. Prognosis is no more or less than predicting the future, and that simply cannot be done with consistent, 100% accuracy. It is ultimately a best guess, based on probability, available evidence, logic, and common sense I am sure that many clinicians have seen individuals defy the odds and the available evidence by unexpectedly achieving outcomes well beyond expectations or prognosis. Again, the ultimate outcome "depends on a lot of things."

The authors of the article about ambulation after DVT concluded--after extensively searching the medical literature--that there was "inadequate evidence" and a need for "more definitive evidence." Ultimately, the "clinical judgment" of the physician and the physical therapist was deemed "critical." In light of those statements, I contend that the decision to ambulate after DVT also "depends on a lot of things," which the available evidence does not always provide.

Regarding evidence-based practice, I am definitely a strong advocate, How could a clinician practice responsibly without due regard for the available current evidence on which to base intervention and other decisions? On the other hand, there are huge considerations regarding the sources of evidence and the quality of the evidence, among other things. The so-called "literature" is not necessarily the best source of evidence, for a variety of reasons. First, there is much variation in its quality. Second, populations addressed in the "literature" do not always resemble individual patients encountered in daily practice. Third, evidence is often couched or clouded in statistical terms that are misleading and easily subject to misinterpretation. Fourth, the evidence may not be in any way "definitive" and, even if so, may not have been adequately replicated or critiqued or relined, Fifth, as stated in a recent Physical Therapy editorial, (4) evidence is often "equivocal." Evidence, as found in the literature, sometimes has real shortcomings.

Is clinical experience to be disregarded as a source of evidence? Certainly, it has limitations, but it does have some genuine value. If there is a hierarchy of evidence, shouldn't a patient's performance be ranked high? Might a clinician gain a kind of evidence through the simple process of trial and error, or through application of standardized tests? I simply want to raise questions about the strengths, weaknesses, and nature of so called evidence based practice. I suspect that many--if not most--clinicians are continually seeking evidence in support them in their quest to most effectively help their patients. I believe that all evidence must be evaluated and applied cautiously to individuals.

David J Smyntek, PT, MA, NCS 

[e-mail]

References

(1) Dumas HM, Haley SM, Ludlow LH, Carey TM. Recovery of ambulation during inpatient rehabilitation: physical therapist prognosis for children and adolescents with traumatic brain injury. Phys Ther. 2004;84:232-242.

(2) Aldrich D, Hunt DP. When can the patient with deep venous thrombosis begin to ambulate? Phys Ther. 2004;84:268 273.

(3) Rothstein JM. What will be, won't necessarily be [editor's note]. Phys Ther. 2004;84: 230-231.

(4) Ciccone CD. Evidence in practice: answers are within your reach [note from the Editorial Board]. Plus Ther. 2004;84:6-7.

Author Response:

We would like to thank Smyntek for his comments in response to our article, "Recovery of Ambulation During Inpatient Rehabilitation: Physical Therapist Prognosis for Children and Adolescents With Traumatic Brain Injury." We are pleased that he agrees that our research addresses a clinically relevant issue for physical therapists. This study was intended to provide therapists with confirmatory research data to use, in conjunction with their own experience and with the goals of the child and family to determine a reasonable prognosis and to develop and justify an individualized plan of care. For children with traumatic brain injury (TBI), these group research data are a critically missing piece in evidence-based practice.

Smyntek wonders whether our conclusions--that lower-extremity (LE) hypertonicity, brain injury severity, and LE injury are the best predictors of ambulation ability--were anything other than confirmation of "an obvious commonsense appraisal of gait potential." We actually were quite surprised that hypertonicity was the strongest predictor of ambulation following TBI, as this had not been reported previously. We hope that our findings, and future prognostic studies, will add specificity to the tests and measures section of the Guide to Physical Therapist Practice (1) for children with TBI.

Over the years, we have been struck b the proclivity of physical therapists to annunciate a prognosis for individual clients, accompanied with a noticeable silence when asked, "How did you arrive at that prognosis?" A therapist's "common-sense prognosis" often derives from a series of individual experiences with patients, some extensive, some just beginning, some more or less relevant--and most likely to be somewhat different from those experienced by other therapists. We believe that common sense guided by shared experiences as reported in the literature is the best way for each therapist to sharpen the ability to make an informed prognosis for individual clients.

Helene Dumas, PT, MS

Research Center for Children with Special Health Care Needs

Franciscan Hospital for Children

Boston, Mass

[e-mail]

Stephen M Haley, PT, PhD

Health and Disability Research Institute

Boston University 

Boston, Mass

[e-mail]

References

(1) Guide to Physical Therapist Practice. 2nd ed. Phys Their. 2001;81:9-746.

Author Response:

Smyntek raises a number of important issues regarding evidence-based practice. We agree that the clinical application of the medical literature to patient problems requires a combination of both clinical judgment and critical analysis of the available evidence. Proven support for treatment choices helps clinicians provide patients with reliable and effective interventions. Although the medical literature is often incomplete or does not directly address our patients' specific circumstances, we feel strongly that the evidence should still be sought to assist with difficult clinical questions.

We agree that there is variation in the quality of the medical literature and in the validity and strength of primary research articles. This variability mandates that the clinician be able to assess the literature or that the clinician rely on valid "second-party" assessments of the literature as done through systematic reviews. We applaud the editors of Physical Therapy far their insistence that reviews strive to meet the established criteria for valid systematic reviews. (1,2) We agree with Smyntek that individual patients do not always match the patients included in studies. This situation requires judgment by the clinician as to whether or not a patient is so different from those included in a study that the study does not apply.

As to the question of whether clinical experience should be disregarded as a source of evidence, we agree that experience has genuine value. However, clinical experience derived from insufficient self-reflection leads to poor clinical judgment. We have been struck by the number of times we have heard the phrase, "In my experience ..." uttered by junior clinicians who have seen 1 or 2 similar cases. Certainly, clinicians can obtain valid experience through trial and error, a process sometimes termed an "N-of-1" trial. In general, one should pursue this approach only if literature evidence is insufficient, there is a pathophysiological rationale for the approach, and the intervention has been discussed carefully with the patient.

We also would like to believe that clinicians are constantly seeking evidence to guide decision making. We hope that over time this evidence will become more robust and plentiful, but we expect that there will always be a major role for clinical judgment in medical decision making. In addition, appraisal of available evidence should stimulate questions for future research.

Dana Aldrich, PT, MS

The Institute for Rehabilitation and Research

1333 Moursund

Houston, TX 77030

[e-mail]

Daniel P Hunt, MD

Baylor College of Medicine 

Section of General Internal Medicine

Ben Taub General Hospital 

Houston, Tex

References

(1) Oxman A, Guyatt G, Cook D, Montori V. Summarizing the evidence. In: Guyatt G, Rennie D, eds. Users' Guides to the Medical Literature. A Manual for Evidence-based Clinical Practice. Chicago, III: AMA Press; 2002:155-173.

(2) Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126: 376-380.