Measurement accuracy. (Letters to the Editor).

To the Editor:

As an ultrasound device manufacturer, I read with great interest "A Calibration Study of Therapeutic Ultrasound Units" in the March 2002 issue of Physical Therapy. I applaud the efforts of the authors in promoting an awareness that therapeutic ultrasound devices need to be calibrated annually as per the recommendations of the Food and Drug Administration (FDA).

The results of the study, however, should be taken with a grain of salt. The wattmeter used to check the calibration of the equipment is not approved by the FDA, and I contend that measuring [+ or -] 20% with a meter that has an accuracy of [+ or -] 10% is really not good science when FDA-approved meters with an accuracy of [+ or -] 3% (eg, a device produced by Ohmic Instruments *) are available.

I would encourage a new study that utilizes state-of-the-art technology to perform the measurements. I think the findings would show a strong correlation between the age of devices and their displayed intensities.

Ken Coffey
President/CEO
Rich-Mar Corp

To the Editor:

As a professional who is interested in the calibration of devices and who works in the field of ultrasound power measurement, I would like to take exception with the technical aspect of the study by Artho et al--specifically, with the use of a power meter with an accuracy of [+ or -] 10%.

Standard practice in instrument calibration--as defined by the American National Standards Institute (ANSI) (1)--requires that the reference instrument (in this case, the power meter) have a minimum of 4 times the accuracy of the unit under test to obtain dependable data. That would mean that, to measure the tolerance of [+ or -] 20% as required by the US Food & Drug Administration, (2) a minimum accuracy of 5% is required. At 10 W, FDA limits are 8 to 12 W of a meter with an accuracy [+ or -] 10%, which results in a range of uncertainty (or error range) of 7.2 to 13.2 W. I believe that a meter with such tolerance diminishes the credibility of the study and its conclusions.

Ultrasound power meters with accuracies of [+ or -] 3% are available from companies such as the one with which I am affiliated. Using such a meter would have reduced the range in which there might be some error to 7.8 to 12.4 W, which would result in an improvement that I believe would make the study more credible.

The issue of risk management and protection of patients should be emphasized, because tissue damage may occur at excessive output levels. Tissue damage may occur at power densities as low as 3 W/[cm.sup.2]. (3) Depending on the size of the therapy head, this level could be reached at the power levels used in the test. Note that the therapist may have to use power levels approaching dangerous levels in order to get sufficient penetration of the therapeutic signal. If the signal is too low, however, the effectiveness of the therapy could be significantly reduced. (3)

The authors' effort to bring technical shortcomings to the professional's attention is welcomed by all risk-management personnel.

Denes Roveti
Technical Director
Ohmic Instruments Company

* Ohmic Instruments, 508 August St, Easton, MD 21601. www.ohmicinstruments.com

References

(1) Calibration Laboratories and Measuring and Test Equipment General Requirements [ANSI-Z540.1]. Washington, DC: American National Standards Institute; July 1994.

(2) US Food & Drug Administration. 21 CFR 1050.10(c)(1).

(3) Kitchen SS, Partridge CJ. A review of therapeutic ultrasound:background and physiological effects. Physiotherapy. 1990;76: 593-595.

Author Response:

We appreciate Mr Coffey's and Mr Roveti's comments regarding our article on the calibration of therapeutic ultrasound devices in use in clinical settings. They have raised a very interesting question regarding the accuracy of the wattmeter (Bio-Tek Digital Ultrasound Wattmeter, model UW-2 *) that was used in our study to test the percentage of error of power output of ultrasound devices.

The UW-2 instrument used in our study was calibrated just prior to the beginning of testing and was guaranteed by the manufacturer for accuracy within [+ or -] 10% for a period of 1 year. As Mr Coffey and Mr Roveti indicate, an instrument with an accuracy of [+ or -] 3% would be better than one with a [+ or -] 10% accuracy. However, the calibration program of the ultrasound wattmeter used in our study met the requirements of the US Food and Drug Administration's (FDA) Good Manufacturing Practices/ Quality System Regulations, ISO 9001, (1) and EN 46001. (2) The UW-2 calibration was also traceable directly or indirectly to the US National Institute of Standards and Technology (NIST) or to nationally accepted measuring systems, through equipment calibration at planned intervals.

When the UW-2 instrument was sent to the manufacturer for calibration 2 weeks prior to the beginning of data collection, the manufacturer tested it at 3 different times within 1 week for measurements of reliability at a frequency of 1 MHz and powers of 0 to 20 W. In fact, the UW-2 instrument we used was found to be accurate within a [+ or -] 2% range and did not require any adjustments from the manufacturer. Although the UW-2 instrument was guaranteed for an accuracy of [+ or -] 10%, it was found to be accurate within a [+ or -] 2% range during its calibration testing.

We believe that our study design further enhanced the accuracy of the application of the wattmeter because one investigator (who had a perfect correlation coefficient of 1.0, with zero variance during test-retest protocol of the pilot study) took all of the measurements and because optimum conditions such as degassed water with oxygen content of less than 2 ppm was used (3) and a clamp attached to a ring stand was used to eliminate any motion of the transducer during testing. We firmly believe that, based on the design of our study, similar results would have been found if the instrument Mr Coffey and Mr Roveti refer to had been used.

As Mr Roveti points out, some authors have suggested the potential deleterious effect on tissue of therapeutic ultrasound at intensities above 3 W/[cm.sup.2], (4,5) and findings of our study showed that 18% of the machines tested were at least 20% above the output setting displayed on the ultrasound machine. For example, this means that an intensity of 2 W/[cm.sup.2] displayed on the ultrasound device was measured to produce an intensity greater than 2.4 W/[cm.sup.2] (one device was found to have a measured intensity 50% greater than the displayed intensity; in this case, a displayed intensity of 2 W/[cm.sup.2] resulted in a measured intensity of 3 W/[cm.sup.2]). We agree with Mr Roveti that this lack of calibration of intensity of ultrasound devices resulting in higher intensities that those displayed by the devices might potentially harm patients. This emphasizes the need for yearly calibration of intensity of therapeutic ultrasound devices.

The argument raised by Mr Coffey regarding FDA approval of wattmeters merits some clarification. A common misconception is that the FDA approves ultrasound calibration instruments (or wattmeters). No therapeutic ultrasound calibration instruments (such as those manufactured by Ohmic Instruments ([dagger]) or Bio-Tek Instruments *) are FDA-approved (personal communication, Gerald R Harris, PhD, Center for Devices and Radiological Health, FDA, 9200 Corporate Blvd, Mail Code HFZ-132, Rockville, MD 20850, e-mail: [e-mail]). Manufacturers of therapeutic ultrasound devices are required by the FDA to comply with the Quality System Regulation and Electronic Product Performance Standard for Ultrasound Products. (6,7) These regulations require establishment of testing procedures, limits for accuracy and precision, appropriate instrument selection, and calibration. However, these requirements do not necessarily extend to manufacturers of testing and calibration instrumentation and, therefore, therapeutic ultrasound calibration instruments are not "approved" by the FDA.

Mr Coffey also suggests that the use of a more accurate testing device would have led to results indicating strong correlations between the age of the therapeutic ultrasound units and their accuracy for power output. Such results were not found in our study. We found no correlation between the age of the machines and the accuracy of power output at intensities of 0.5, 1.0, and 1.5 W/[cm.sup.2]. Because we guaranteed anonymity of the participants and their clinic or hospital (some clinicians were reluctant to participate in the study unless anonymity was guaranteed), it was not possible for us to evaluate the factors responsible for the large percentage of ultrasound devices (39%) found to be out of calibration for power output. We were stunned by the lack of knowledge of the clinicians concerning the calibration of the ultrasound devices in their clinics. Forty-two percent of clinicians reported not knowing what was being checked on the ultrasound devices in their facility, and 19% reported that the machines were checked for electrical supply and safety only.

Although a greater percentage of certain brands of ultrasound devices were out of calibration for power output, we could not infer that the brand itself was the sole parameter correlated with the lack of accuracy, because we were unable to control for some possible confounding factors such as the potential absence of yearly calibration for power output by the facility.

It is imperative, in our opinion, that clinicians request yearly calibration of their ultrasound devices for power output, frequency, timer, electrical supply, and safety and that they request a written notice from the technician stating the accuracy of the devices for these parameters. We would also suggest that the FDA require yearly calibration of the ultrasound devices for these parameters and that devices not within the 20% accuracy standard (7) should be fixed for accurate readout. This would enhance appropriate ultrasound output in a time of controversy concerning the effectiveness of therapeutic ultrasound. (8,9)

Jean-Michel Brismee, PT, MS, OCS
Assistant Professor
Physical Therapy Program
Texas Tech University Health Sciences
  Center
Amarillo, TX 79106
([e-mail])

Neal Latman, PhD
Associate Professor
Texas Tech University Health Sciences
  Center

Paul Artho, PT, MPT
Physical Therapist
BSA Hospital
Amarillo, Tex

Steven Sawyer, PT, PhD
Associate Professor
Texas Tech University Health Sciences
  Center

* Bio-Tek Instruments Inc, Highland Park, Box 998, Winooski, VT 05404-0998.

([dagger]) Ohmic Instruments, 508 August St, Easton, MD 21601. www.ohmicinstruments.com

References

(1) Quality Systems-Model for Quality Assurance in Design, Development, Production, Installation, and Servicing. ISO Designation: ISO 9001. Geneva, Switzerland: International Organization for Standardization (ISO); 2000.

(2) Quality System-Medical Devices Particular Requirement for the Application of ISO 9001. CEN Designation: EN 46001. Brussels, Belgium: European Committee for Standardization (CEN); 1996.

(3) Snow CJ. Ultrasound therapy units in Manitoba and Northwestern Ontario: performance evaluation. Physiotherapy Canada. 1982;34:185-189.

(4) Payton OD, Lamb RL, Kasey ME. Effects of therapeutic ultrasound on bone marrow in dogs. Phys Ther. 1975;55:20-27.

(5) Kitchen SS, Partridge CJ. A review of therapeutic ultrasound: background and physiological effects. Physiotherapy. 1990;76:593-595.

(6) US Department of Health and Human Services, Food and Drug Administration, Center for Devices and Radiological Health. Quality System Regulation. 21 CFR 820.72. Revised 04/01/2001. Available at: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPCD/ ShowCFR.cfm?FR=820.72.

(7) US Department of Health and Human Services, Food and Drug Administration, Center for Devices and Radiological Health. Performance Standards for Sonic, Infrasonic, and Ultrasonic Radiation-Emitting Products. 21 CFR 1050.10. Revised 04/01/2001. Available at: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPCD/ ShowCFR.cfm?FR=1050.10.

(8) Robertson VJ, Baker K. A review of therapeutic ultrasound: effectiveness studies. Phys Ther. 2001;81:1339-1350.

(9) Baker K, Robertson VJ, Duck FA. A review of therapeutic ultrasound: biophysical effects. Phys Ther. 2001;81:1351-1358.