Dialogue on evidence in practice.

Dialogue on Evidence in Practice: We encourage Journal readers to respond to articles in "Evidence in Practice." E-mail your contribution to the dialogue to [e-mail].

On "Does acetic acid iontophoresis accelerate the resorption of calcium deposits in calcific tendinitis of the shoulder?":

I looked forward to reading the "Evidence in Practice" article by Charles D Ciccone, which asked the question "Does acetic acid iontophoresis accelerate the resorption of calcium deposits in calcific tendinitis of the shoulder?" (1)

This article describes a physical therapy intervention that I have not used yet; but I have heard, from several colleagues whom I respect, that this intervention had good results when other interventions had failed. I was disappointed to find out that there was scant evidence to support this intervention. I was more disappointed, however, that the article misapplied the principles of evidence-based practice and misinterpreted what little evidence there was.

From his search Dr Ciccone was only able to find 2 articles that he felt might provide evidence on the question. The first article, by Perron and Malouin, (2) discussed a randomized controlled trial (RCT) of 22 adults with calcifying tendinitis in the shoulder. Half the group was treated with acetic acid iontophoresis and ultrasound, and half the group received no treatment. Perron and Malouin reported that there was a reduction in the area and density of the calcium deposit in both groups. The authors were unable to find any difference between groups for any of the variables measured. They therefore concluded that the reduction in the area and density of the calcium deposit was likely a result of a natural process, rather than the intervention. It should be noted that this is a conclusion based on speculation rather than on evidence.

The second article, by Rioja Toro et al, (3) was a cohort study rather than an RCT. It looked at 34 patients who were treated with acetic acid iontophoresis and ultrasound and found that, in all those patients, the calcification size was modified and pain was reduced. With no control group, it was of course difficult to definitely attribute the outcome to the treatment.

Dr Ciccone chose to disregard the Rioja Toro et al study because it lacked a control group. Based on the Perron and Malouin study, he decided not to include acetic acid iontophoresis in the plan of care for his patient. He concluded that "the study by Perron and Malouin provides evidence that this intervention is not effective in accelerating resorption of calcific lesions in the supraspinatus tendon." (1(p73)) His conclusion was not based on evidence. His conclusion was a misreading or misinterpretation of the Perron and Malouin study. The study did not, in any way, provide "evidence that this intervention is not effective." The study failed to provide any conclusive evidence of anything.

Ciccone and Perron and Malouin are not alone. Perron and Malouin made the classic error of over-interpreting the data. Although it is true that their study failed to provide evidence of efficacy for this treatment, it is not true that it also proved the null hypothesis (ie, that evidence was found that the treatment was not effective). There is a big difference between the two. Unfortunately many people who teach about and even purport to use evidence-based practice do not understand the difference between studies that provide no evidence regarding effectiveness and those that provide evidence of ineffectiveness.

If the acetic acid iontophoresis treatment were merely 20% more effective than no treatment or treatment with ultrasound alone, it could take a cohort of as many as 200 patients with the same exact diagnosis to demonstrate that effectiveness. A smaller cohort, such as the one used by Perron and Malouin, is not capable of proving the null hypothesis (ie, that the treatment is ineffective) with a sufficient power rating of at least 80%.

Essentially, the study on which Dr Ciccone based his clinical decision provided no evidence. The clinical decision, therefore, was not evidence based. It was no more than a stab in the dark or, at best, in the dim.

In employing the concept of evidence-based practice, we are told to find the best evidence possible to guide us. (4) Cormack (5,6) suggests a hierarchy of evidence as follows:

1 Meta-analysis

2 Systematic reviews

3 Clinical practice guidelines 

4 Randomized control trial (RCT).

5 Cohort studies

6 Case control studies

7 Case studies

8 Opinions from respected authorities based on clinical practice

9 Basic science research

In this case, Dr Ciccone's careful literature searches produced no meta-analyses, no systematic reviews, and no clinical practice guidelines. Let us therefore look at the evidence that is available:

* The search produced only one RCT, (2) which failed to provide any meaningful evidence, and one cohort study, (3) which suggested that there might be some benefit to the use of this intervention.

* In his introduction, Ciccone also referred to a case report by Wieder, (7) which indicated the possibility of some benefit from the treatment on a similar condition (myositis ossificans).

* He had apparently heard opinions from other clinicians whom he respected that, based on their clinical practice, this intervention might have been helpful--or else he wouldn't have been investigating the intervention in the first place.

* There was a scientific rationale for the treatment described in the case report by Wieder, (7) a rationale that Ciccone says was based on a known "chemical reaction where insoluble calcium carbonate molecules combine with acetic acid to form calcium acetate, which is more soluble." (1(p69))

This scant evidence, the only available evidence on which to make a clinical decision, suggests that acetic acid iontophoresis treatment might be helpful. There was no evidence to the contrary, nor was there any evidence that the treatment could be harmful.

Did Dr Ciccone make the right clinical decision? We really don't know. This was not a case study. He did not have to report whether the patient got better. If the patient got better, it really doesn't matter whether he made the right clinical decision. If the patient didn't get better, however, the patient might have been denied a beneficial intervention because of Dr Ciccone's failure to recognize and properly use what little evidence there was.

What disappointments me the most is that Physical Therapy, the official journal of the American Physical Therapy Association, would publish this article as an example of evidence-based practice. This perpetuates practice decisions made on non-evidence and does so in the name of evidence-based practice. This does more to set back our professional growth and deny advancements in care for our patients than it does to advance us.

I am not writing this letter in support of acetic acid iontophoresis. As I stated before, I am unfamiliar with this intervention; therefore, I have no personal opinion one way or the other on whether it works. I am writing in support of appropriate understanding and use of the concept of evidence-based practice. Evidence-based practice does not mean that clinicians can use only those treatment procedures that have been proven effective in RCTs or higher rated studies. If that were the case, our patients would have never benefited when their physical therapists used the works and ideas of other innovative clinicians, from Bobath and Brunnstrom to McKenzie and McConnell.

In my opinion, evidence-based practice means that the thinking clinician looks at all possible evidence starting with the basic science evidence at the bottom of the hierarchy and then working up the ladder, through what other clinicians have done, to what the research community has been able to find. That evidence, extensive or limited as it might be, then has to be set against the template that has been created by the clinician's own education and experience and then finally applied to the individual patient's unique presentation.

Certainly, when the effectiveness of an intervention is demonstrated by a good RCT or even a higher study on the hierarchy, we should do our best to use that intervention. In the absence of such conclusive studies (but also in the absence of any studies that prove that an intervention is not beneficial or that the intervention is harmful), good clinicians should continue to use their intuition and explore the use of these interventions on a trial-and-error basis. That is good clinical practice. It is the type of judgment call that sets the clinician apart from the technician.

The Journal would serve us better if it would present articles on evidence-based practice that show clinical choices based on evidence of what works in practice rather than on speculation based on studies that fail to show evidence. I recognize that this is difficult because of the paucity of good studies, particularly in orthopedics and physical medicine. We have to recognize that this paucity tells us more about the nature, quality, and outcomes of the research than it does about nature, quality, and outcomes of physical therapist clinical practice.

Philip Paul Tygiel, PT, MTC 

Tygiel Physical Therapy

6606 E Carondelet Dr

Tucson, AZ 85710

References

(1) Ciccone CD. Does acetic acid iontophoresis accelerate the resorption of calcium deposits in calcific tendinitis of the shoulder? Phys Ther. 2003;83:68-74.

(2) Perron M, Malouin F. Acetic acid iontophoresis and ultrasound for the treatment of calcifying tendinitis of the shoulder: a randomized control trial. Arch Phys Med Rehabil. 1997;78:379-384.

(3) Rioja Toro J, Romo Monje M, Cantalapiedra Puentes E, et al. Treatment of calcifying tendinitis of the shoulder by acetic acid iontophoreseis and ultrasound. Rehabilitacion. 2001;35(3)166-170.

(4) Sackett DL, Straus SE, Richardson WS, et al. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd ed. New York, NY: Churchill Livingstone Inc; 2000.

(5) Cormack JC. Evidence-based practice ... what is it and how do I do it? [guest editorial.] J Orthop Sports Phys Ther. 2002;32:484-487.

(6) Cormack JC. Evidence-based practice [author's response to letter to the editor]. J Orthop Sports Phys Ther. 2003:33:146-148.

(7) Wieder DL. Treatment of traumatic myositis ossificans with acetic acid iontophoresis. Phys Ther. 1992;72:133-137.

Editor/Author responds:

I thank Mr Tygiel for his thoughtful comments. His letter addresses several important issues that need to be considered.

Mr Tygiel takes particular exception to the interpretation of the results from the Perron and Malouin study. He acknowledges that this study failed to provide evidence of treatment efficacy but fell short of providing evidence of treatment ineffectiveness. I disagree. Perron and Malouin reported a reduction in size and density of calcium deposits in both the untreated control group and the group treated with a regimen of acetic acid iontophoresis and ultrasound. This study likewise failed to find a difference between the groups in any other dependent variables (pain and range of motion). Because using no treatment was just as effective as using the intervention, I must interpret these results as providing evidence that the treatment did not work and was therefore ineffective. If I knowingly provided an ineffective treatment, I would be violating APTA's Code of Ethics and wasting health care resources.

Moreover, Mr Tygiel maintains that there is a big difference between those studies that fail to provide evidence of efficacy and those that provide evidence of ineffectiveness. I agree with this principle; absence of evidence does not mean that an intervention is ineffective. We should not say that a treatment is ineffective simply because there is no evidence available to support or refute use of the intervention (ie, because the studies have yet to be conducted). In the situation described in my article, however, we do not have absence of evidence. Perron and Malouin clearly provided evidence that the intervention did not work in the 21 subjects in their study. Mr Tygiel is incorrect in saying that this study did not provide conclusive evidence of ineffectiveness because responses in the control (untreated) group were similar to those in the treated group. Consider, for example, what would happen if a new medication was developed, but this medication was no more effective in resolving a medical condition than no treatment. Would Mr Tygiel advocate use of this medication even though people seem to improve just as quickly if they do not take it? If a randomized controlled trial tails to find a difference between treatment and control groups, I have no compelling reason to incorporate this intervention into my treatment regimen.

Mr Tygiel mentions the small cohort size of the Perron and Malouin study, and I applaud his awareness of issues related to sample size and statistical power. A study must have adequate sample size and statistical power to guard against a type II error (ie, making the erroneous conclusion that there was no difference between the groups when a difference might be discovered if more subjects were included). Regrettably, Perron and Malouin do not provide measures of effect size or statistical power directly in their paper. I cannot understand, however, how Mr Tygiel can state categorically that as many as 200 subjects would be needed to find a specific effect (a 20% improvement) without also taking into account all the factors that influence the statistical power of these results. Without considering other factors such as the variance of their data, Mr Tygiel is incorrect in assuming that Perron and Malouin were not capable of making the correct conclusion (ie, that there really was no difference between the groups).

Mr Tygiel also overlooked an important finding from that study that I used to guide my clinical decision. Perron and Malouin found a weak correlation between radiological changes (decreased size of the calcific lesion) and improvement in shoulder function (decreased pain, increased range of motion). This finding suggests that reducing the size of the lesion might not be a prerequisite for improved outcomes. We have always assumed that the size of the calcium deposit must be reduced, and acetic acid iontophoresis offered a chemical (albeit theoretical) method that might accelerate resorption. Perron and Malouin, however, suggest that changes in the size of the lesion are not associated closely with functional improvement. This finding further convinced me that it might not be worthwhile to use an intervention that is focused exclusively on reducing the size of the calcium deposit.

Mr Tygiel also suggests that my clinical decision was not evidence based. On the contrary, I decided to use ultrasound and therapeutic exercise because research from other studies (1,2) documented the effectiveness of these interventions in people with calcific tendinitis. I chose to exclude acetic acid iontophoresis because there is no evidence to suggest that adding this intervention would provide additional benefits for my patient. Hence, my decision was evidence based because I weighed my options according to the preponderance of the evidence that I had at hand.

I agree with Mr Tygiel that evidence-based practice should be based on a hierarchy of evidence. He suggests the use of such a hierarchy based on some common study designs. Mr Tygiel, however, appears to ignore his own suggestion. Early in his letter he claims to have heard from several respected colleagues that acetic acid iontophoresis had good results in accelerating the resorption of recalcitrant calcific tendinitis, and he later makes a vague reference to opinions from other (unnamed) authorities. At best, these claims are ranked at the eighth level of his hierarchy (opinions from respected authorities based on clinical practice). I cannot understand why these opinions should supercede the findings from Perron and Malouin (a randomized controlled trial that corresponds to level 4 of his hierarchy). A hierarchy such as the one proposed by Mr Tygiel is very useful, but only if we have the confidence to accept evidence from higher levels that might contradict the opinions of clinical experts.

Mr Tygiel also noted that my search retrieved only one RCT (Perron and Malouin) and that the search failed to discover any evidence from the highest levels of this hierarchy (meta-analysis, systematic reviews, and clinical guidelines). It is true that I did not retrieve any of these types of studies, presumably because they do not yet exist. Nonetheless, the fact that such higher levels might not currently exist does not detract from evidence provided by the one relevant RCT that I did retrieve.

Mr Tygiel then states that the only evidence that can be used to make a clinical decision suggests that acetic acid iontophoresis might be helpful and that there is no evidence to the contrary. This statement is actually the opposite of what we know to be true. I found one RCT that provided evidence that this intervention has the same effect as no treatment. More importantly, I found no indication that the evidence from the "respected authorities" actually exists in our peer-reviewed literature. Again, I am not sure who these authorities are, but it is their professional obligation to publish their opinions via case reports and other studies so that these opinions can be subjected to the scrutiny and critical analysis that constitutes peer review. Otherwise, these experts are doing a great disservice to our profession by keeping their results to themselves or by sharing their opinions through word-of-mouth and anecdotal reports in non-peer-reviewed sources.

Finally, Mr Tygiel expressed his disappointment that the Journal would publish this "Evidence in Practice" article as an example of evidence-based practice. As we have indicated many times in the past, these articles are intended to illustrate the search process rather than establish best practice or serve as clinical guidelines. Still, my clinical decision in this case was based on the evidence that is currently available and relevant to this particular patient. Rather than impairing our professional growth or denying advancements in care, use of evidence as described in this article represents an objective and mature way to practice. We must continually reexamine long-held beliefs and opinions when higher levels of evidence suggest that these beliefs might not be true.

Charles D Ciccone, PT, PhD

Editor--Evidence in Practice and Reviews

Physical Therapy

References

(1) Ebenbichler GR, Erdogmus CB, Resch KL, et al. Ultrasound therapy for calcific tendinitis of the shoulder. N Engl J Med. 1999;340:1533-1538.

(2) Ginn KA, Herbert RD, Khouw W, Lee R. A randomized, controlled clinical trial of a treatment for shoulder pain. Phys Ther. 1997;77:802-811.

In the next Evidence in Practice: Lisa Riolo, PT, PhD, NCS, and Kimberly Fisher will seek evidence to answer the question, "Does strength training interfere with function, or is it a useful component of a rehabilitation program for a man who has had a stroke?" (Coming in September 2003)