There are 3 types of health-related quality-of-life instruments: discriminative, predictive, and evaluative.

There are 3 types of health-related quality-of-life instruments: discriminative, predictive, and evaluative. (1) Each type of instrument has different characteristics and is selected based on its intended use. The purpose of a discriminative instrument is to differentiate individuals based on the score they achieve when no superior, "gold" standard is available. (1) These instruments also can be used to determine the seriousness of the impairment, functional limitation, and disability compared with other individuals. Predictive instruments are used in conjunction with a gold standard to categorize individuals. (1) Evaluative instruments measure an individual's change in status over time. Evaluative instruments, as opposed to discriminative and predictive instruments, can be used to assess the effectiveness of treatment (1) and commonly serve as outcome instruments. If treatment outcomes are to be appropriately measured, clinicians need to select a suitable evaluative instrument and properly interpret the obtained scores.

When selecting and using an evaluative outcome instrument, a number of factors must be considered. Not only is evidence for content validity, construct validity, reliability, and responsiveness required to properly interpret scores from evaluative measures but also a useful description of the applicable test conditions. Test conditions include information about the subjects' characteristics, timing of data collection, and construct of change. (2) Appropriate score interpretation requires that the test conditions associated with research that offers evidence for score interpretation must be similar to the clinical conditions in which the instrument will be implemented.

Instruments that satisfy the requirement for evidence for content validity, construct validity, reliability, and responsiveness will allow obtained scores to be more comprehensively interpreted. Information obtained about reliability and responsiveness can be useful for clinicians when a patient's score is interpreted over time. Related to test-retest reliability, an intraclass correlation coefficient value by itself offers little information for clinical interpretation. The minimal detectable change (MDC) may be a more useful measure because it will represent the amount of change necessary to conclude that the changes in scores are beyond measurement error. (3) The difference between repeated measures must be greater than the MDC to be confident that a score has truly changed over time.

Related to responsiveness, the score on an evaluative instrument should respectively increase or decrease if the individual's condition improves or worsens. (4) Information for interpreting score changes of a particular individual can come from receiver operating characteristic (ROC) curves. These ROC curves can be used to determine the cutoff value for the change score that has the highest sensitivity and specificity for change. (5) Sensitivity of change is defined as the proportion of subjects who have improved according to a criterion of change and have a change score above a cutoff value. (4) Specificity of change is defined as the proportion of subjects who have not improved according to a criterion measure of change and have a change score below a cutoff value. (4) The cutoff value is the change in score that best discriminates between those who have improved from those who have not improved.

This cutoff value is referred to as the "minimum clinically important difference" (MCID). (5) The results of a responsiveness study at the individual level can be used to determine whether a particular individual has improved or has not improved, provided conditions are similar to those of the responsiveness study (eg, patient group, type of treatment, and timing of data collection). (5)

The Movement Ability Measure (MAM) is a self-report instrument created to assess an individual's perception of how he or she moves. The articles by Allen outline evidence for content validity, construct validity, reliability, and responsiveness. The method used incorporated item response theory (IRT). Item response theory has advantages over classical methods because IRT analysis is item directed as opposed to test directed. While being mathematically complex, it potentially provides a precise measure that is a more direct function of the individual's ability. (6)

Allen should be commended for her research by providing the 4 categories of evidence for the MAM. There are many different ways to provide this evidence, each with advantages and disadvantages. Therefore, my comments should not be interpreted as overly critical but rather as an attempt to encourage further research in using the MAM. Although the MAM was found to yield reliable data over a 2-week period, specific information regarding the MDC may be helpful to allow for clinicians to directly interpret whether score changes over time are outside of measurement error. The MAM also was found to be responsive, and the MCID was calculated. However, the methods used make it difficult to make clinical decisions regarding changes in an individual's status. Calculations done at an individual level, such as with ROC curves, may facilitate interpreting whether an individual's condition has improved or worsened.

Clinicians also need to evaluate the subjects to make sure their sample is similar to those used in Allen's studies. However, information about the samples used in the reliability and responsiveness analyses is vague. Thirty-four (11%) of the 318 subjects were included in the test-retest analysis. The diagnosis for the entire sample of 318 included "musculoskeletal (including upper-extremity, lower-extremity, or spine problems), neurological, endocrine, and cardiovascular pathologies," with 39 subjects (12%) either involved in or about to begin physical therapy intervention. Thirty-five subjects involved in physical therapy intervention were included in the analysis of responsiveness. This group was involved in physical therapy intervention because of "low back pain; neck or upper back pain; shoulder pain; knee, wrist, or foot surgery; ankle tendinitis; carpal tunnel syndrome; and thoracic outlet syndrome." More detailed information about these subjects might assist clinicians in determining whether the MAM could be reliable and responsive for their population.

One other comment relates to the dimensions the 24-item MAM represents. If the 6 dimensions assess flexibility, strength, accuracy, speed, adaptation, and endurance, subscale scores, in conjunction with the total combined score, may provide useful information. Subscale scores could allow professionals to direct treatment toward the dimensions that were perceived as most problematic. Additionally, certain dimensions could be more responsive to individuals with different diagnosis and impairments.

In summary, Allen should be congratulated for her efforts to develop and provide evidence to support the use of the MAM. One should remember: evidence for validity, reliability, and responsiveness continue indefinitely as information supporting interpretation of obtained scores is presented through ongoing research. Additionally, issues related to validity, reliability, and responsiveness are never completely established for all uses of the instrument. The results from research studies can only offer supporting evidence under particular conditions. Therefore, the interpretation of the score obtained by the instrument is determined to be valid, reliable, and responsive; the instrument itself is not. (7)

References

(1) Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chronic Dis. 1985;38:27-36.

(2) Martin RL, Irrgang JJ, Lalonde KA, Conti SF. Current concepts review: foot and ankle outcome instruments. Foot Ankle Int. 2006;27:383-390.

(3) Stratford PW, Binkley JM, Solomon P, et al. Defining the minimum level of detectable change for the Roland-Morris Questionnaire. Phys Ther. 1996;76:359-365; discussion 366-368.

(4) Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis. 1986;39:897-906.

(5) Beaton DE. Understanding the relevance of measured change through studies of responsiveness. Spine. 2000;25:3192-3199.

(6) Hambleton RK, Jones RW. Comparison of classical test theory and item response theory and their applications to test development. Education Measurement: Issues and Practices. 1993;12:38-47.

(7) Messick S. Meaning an values in test validation: science and ethics of assessment. Educational Researcher. 1989;18:5-11.

RL Martin, PT, PhD, CSCS, is Assistant Professor, Department of Physical Therapy, Duquesne University, 114 Rangos School of Health Sciences, Pittsburgh, PA 15282. Address all correspondence to Dr Martin at: [e-mail].

DOI: 10.2522/ptj.2006.0182.0197.0198.ic2