Reliability of the Fugl-Meyer assessment for testing motor performance in patients following stroke.

Key Words: Fugl-Meyer assessment, Motor performance, Reliability, Stroke.

Stroke is a major cause of mortality and disability in many countries. The average annual mortality rate for stroke per 100,000 people is 100.4 in the United States, 116.2 in England and Wales, and 96.6 in Canada.[1] Data compiled from a number of surveys in the United States show that the annual incidence is approximately 100 to 150 per 100,000 people.[1] Following a stroke, patients usually receive intensive therapy to promote motor recovery and help them cope with their disability.

Reliable and valid measurements of sensorimotor status are required for clinical decision-making and research purposes. One measure that has been extensively discussed is the Fugl-Meyer evaluation of physical performance.[2] The Fugl-Meyer assessment was developed to assess physical recovery following stroke. It was primarily developed from the earlier works of Twitchell[3] and Brunnstrom.[4]

Twitchell examine the ontogenetic concept of motor recovery, through the assessment of sequenced reflex and synergistic patterned movements seen in patients recovering from stroke.[3] Based on this recovery sequence, Brunnstrom identified six sequences of temporal, stepwise stages as a method of assessing motor recovery in patients with hemiplegia following stroke.[4] In addition to motor performance, balance, sensation, range of movement, and pain are also assessed by the Fugl-Meyer items, which examine volitional movement within synergies, partially out of synergy, and independent of synergies.

The Fugl-Meyer motor assessment includes items dealing with the shoulder, elbow, forearm, wrist, and hand in the upper extremity and the hip, knee, and ankle in the lower extremity. Reflex activity is assessed in the upper and lower extremities at the beginning and end of the motor assessment. Balance is examined in sitting and standing. Sensation, evaluated by light touch, is examined on two surfaces in both the upper and lower extremities, and position sense (kinesthesia) and range of motion (ROM) are tested on eight joints, four in each extremity.[2]

The Fugl-Meyer assessment, which consists of 155 items, is an impairment measure. Impairment is defined as any loss or abnormality in psychological, physiological, or anatomical structure or function.[5] With the Fugl-Meyer assessment, each item is rated on a three-point ordinal scale (2 points for the detail being performed completely, 1 point for the detail being performed partially, and 0 for the detail not being performed). The maximum score that can be attained is 226. The maximum motor performance score is 66 points for the upper extremity, 34 points for the lower extremity, 14 points for balance, 24 points for sensation, and 44 points each for passive joint motion and joint pain. Joint pain is assessed by moving the joint through its available ROM to assess whether pain occurs at any point in the range. Fugl-Meyer assigned motor function scores to items that assessed motor function alone, with a total possible score of 100 points. Scores were grouped according to the various levels of impairment, which were as follows: < 50 points = severe motor impairment, 50-84 points = marked motor impairment, 85-95 points = moderate motor impairment, and 96-99 points = slight motor impairment.[6]

The properties of this instrument have been described by various authors [2,7-15] (Tab. 1). Most of these authors examined the validity of the instrument. In the original study by Fugl-Meyer et al,[2] 28 patients were assessed on five occasions during a 1-year period following stroke. The mean correlation coefficient between the upper- and lower-extremity scores was .88.[2] Other researchers compared scores from the Fugl-Meyer assessment with data from the Barthel Index [10,11]; a Bobath assessment[12]; upper-extremity function tests[9]; and tests for balance in standing, walking performance, and postural stability.[10] The correlation coefficients reported from these studies varied from .54 for the total score to .94 for the upper extremity, thus providing evidence of construct validity.

Studies to date have examined the construct validity of the Fugl-Meyer assessment in samples of chronically disabled patients following stroke. Less work has been done to determine the reliability of measurements obtained with this assessment. Duncan et al[14] reported the interrater reliability of motor performance in the upper and lower extremities and the intrarater reliability of all sub-scores and the total Fugl-Meyer score using four physical therapists as evaluators. The 19 patients in their study were more than 1 year poststroke, with a mean time of 51 months since the onset of their stroke. Intrarater Pearson correlations for each subscore and the total score varied from .86 to .99. The interrater Pearson correlations for the motor scores of the upper and lower extremities varied from .79 to .99. Because Duncan et al studied subjects who were chronically disabled following stroke, the generalizability of their results is limited to similar patient populations.

The study described in this report examined the overall interrater reliability and the reliability of each subsection of the Fugl-Meyer assessment, administered by three raters to patients undergoing active rehabilitation following stroke.

The purposes of our study were (1) to examine the interrater reliability of measurements obtained with the Fugl-Meyer assessment and (2) to determine the relevance of the assessment for clinical and research purposes. The specific aims of the study were to determine whether the Fugl-Meyer assessment is able to discriminate among patients when it is used to evaluate patients' motor recovery following stroke and to determine the magnitude of the measurement error for the total score and score of the subsections of the assessment. The a priori level of acceptable reliability among raters was set at an intraclass correlation coefficient (ICC[2,1]) of greater than .80 for the total score and subscores.

Method

This study was part of a larger study that examined the measurement properties of the Chedoke-McMaster Stroke Assessment.[16]

Subjects

Twelve patients consecutively admitted to the Chedoke-McMaster Rehabilitation Centre, Hamilton, Ontario, Canada, were each assessed by three therapists (see Tab. 2 for descriptions of the patients) on separate occasions. The Chedoke-McMaster Rehabilitation Centre is a tertiary care setting at which patients are treated on a daily basis by a multidisciplinary team. The average length of stay per patient was 10 weeks. Patients were included if they were less than 80 years of age, were less than 6 months poststroke, and gave their consent to participate in the study.

Raters

Three physical therapists who had between 10 and 20 years of experience in stroke rehabilitation administered the assessment. They reviewed the original article by Fugl-Meyer et al[2] and designed a form to record the results (Appendix). Prior to commencing the study, the physical therapists discussed the assessment and tested three patients. Each therapist performed and rated one assessment while the other therapists rated the patient's responses. They compared their results after each assessment. Approximately 2 1/2 hours was spent in this preparation.

Design

The order in which the therapists saw the patients was randomized and balanced (equal numbers of subjects were assigned to the therapists). All patients were tested within one working day of the previous assessment (ie, theoretically, there should have been no change in the patients) and were seen on only one occasion by each therapist; that is, each patient was assessed by all three physical therapists on separate occasions. Assessments took approximately 30 minutes to administer. After the study began, the therapists did not discuss the results of the assessments.

Data Analysis

Descriptive statistics, including the mean and standard deviation for the total Fugl-Meyer score and for the subscores, were calculated for each rater and for each testing occasion. A three-way analysis of variance (ANOVA) was performed to test for differences between raters (three levels) and occasions (three levels). The third factor in the analysis was subjects (ie, subjects x raters x occasions). Differences were formally tested by calculating the F statistic and the associated probability value. The level of statistical significance was set at .05. Reliability was assessed by calculating an ICC (type 2,1), a random-effects model described by Shrout and Fleiss.[17] Reliability is defined as the ability of an instrument to measure attributes in a reproducible and consistent manner.[18]

A measurement is considered to be consistent if it produces similar results over a number of occasions in a stable situation.[19] The ICC used to calculate the reliability coefficients represents the ratio of variance due to subjects compared with the total variance; thus, its most direct interpretation relates to its ability to discriminate among patients. It is directly reliant on the heterogeneity of the patients. Ninety-five percent confidence limits were calculated to determine the precision of the estimate of reliability.[20] Calculating the reliability of an assessment also involves calculating the variability in an individual's score due to factors irrelevant to the purpose of the test. This variability is known as the error of measurement.[20,21] The standard error of measurement (SEM) was calculated from an error term that combined rater, occasion, and error variances.18

Results

The summary statistics (means and standard deviations) for both time and rater are shown in Tables 3 and 4. The ANOVA did not reveal any statistically significant systematic differences attributable to raters or occasions. The overall mean scores of the raters varied from 161 to 167 points. All raters had a mean score of 40 points for the upper extremity. The sensation subsection had the greatest difference between means, a 3-point difference. The mean total scores according to occasion varied from 162 to 165 points, with the largest difference between the means in any section being 2 points.

The ICCs, confidence limits, variance components, and SEMs are shown in Table 5. The interrater reliability for the total score was .96.

The reliability coefficients for the subsections of the Fugl-Meyer assessment varied from .61 for the pain measurements to .97 for the upper-extremity items (Tab. 5). The SEM was 9 points for the total score and varied from 1 for balance to 4 for pain. Table 5 provides the maximum total score for each section as a reference point for each of the SEMs.

The total scores for the patients varied from 79 to 220 points. Using the guidelines developed by Fugl-Meyer for characterizing the various levels of motor performance[6] to analyze our results, we found that 92% of the patients in our study were in the first three categories (severe impairment, n=3; marked impairment, n=4, and moderate impairment, n=4). Only one patient had scores that would indicate slight motor impairment.

Discussion

Overall, the reliability estimates in our study were similar to those reported by Duncan et al (r=.86-.99)[14] and Di Fabio and Badke (rho=.95).[15] In our study, the reliability coefficient for the upper extremity was higher than that for the lower extremity. Recovery of function often occurs faster and to a greater extent in the lower extremity than in the upper extremity. Therefore, many of the patients may have been unable to perform components of the upper-extremity assessment, reducing the possibility of rater error. Most patients were able to perform almost all of the lower-extremity items. The lower-extremity performance may have been less consistent from day to day, which may account for a relatively greater amount of residual error. The relative heterogeneity of the scores for the upper limb may account for the higher reliability coefficient for the upper extremity compared with the lower extremity.

Pain assessment was the least reliable section. This finding may be due to the natural (random) variation in pain from day to day or to variation of rater interpretation of the patients' pain. Although sensation had a reasonable degree of reliability (ICC=.86), it had the greatest variance due to rater error. The rater is reliant on the patient's verbal ability to describe his or her perceptions. This is in contrast to other items in the Fugl-Meyer assessment, which involve scoring through direct observation.

There was very little variation in the measurements attributable to occasion. The small variances associated with rater and occasion (Tab. 5) demonstrate the absence of systematic differences related to these variables.

The main component of the assessment is motor recovery following stroke. The inclusion of sections to assess range of movement and pain allows for the assessment of impairments that may be attributed to causes other than the stroke. The impairment measures of the Fugl-Meyer assessment have been shown to correlate with activities of daily living (ADL) measures (r=.92).[22] This finding suggests that patients with similar scores on the impairment measures of the Fugl-Meyer assessment will also have similar scores on ADL measures. Because sensation has been noted as a predictor of sensorimotor recovery,[23] assessment of this area by the Fugl-Meyer assessment increases the strength of the tool.

The SEM provides an interpretation of the magnitude of measurement error and is more appropriate than the reliability coefficient for interpreting individual scores.[21] One standard error encompasses 68% of obtained scores around the true score, and a standard error of 1.96 encompasses 95% of obtained scores. The SEM for the Fugl-Meyer assessment is 9.4 points, and the 95% confidence interval would be [+ or -] 18 points. Given that a clinician obtains a total score of 150 points for the overall observed total score at the 95% confidence level with any single administration of the test, the score obtained will lie between 140.6 and 159.4 points.

In order to evaluate change in an individual patient, Ottenbacher et al[24] recommended that the "reliability change index" be used. The standard error of the difference is used to determine whether the change score exceeds the score that would be expected on the basis of measurement error. Given that no true change occurred 95% of the time, the score obtained by a second rater at a different point in time would be within [+ or -]26 points (13 x 1.96 x 2)[24] of the score of the first rater. This difference represents about 11.5% of the maximum possible score. On the basis of this result, the Fugl-Meyer assessment is a moderately reliable tool. In our study, evaluation using the instrument involved more than one therapist; thus, both the reliability coefficients and the SEMs are conservative estimates compared with the situation of a single assessment by a therapist.

How can these results be interpreted by a therapist in a clinical setting? The results of this study indicate the reliability and the SEM that would likely be obtained among several raters with a similar patient population in a clinical setting. If two clinicians assessed a patient who was receiving active rehabilitation following a stroke (ie, similar to the patients in our sample) on different occasions using the Fugl-Meyer assessment and derived total scores that were 10 points apart, we would not be sure whether true change had occurred or whether the difference in scores was due to rater or residual error. Although the assessment of intrarater reliability was not part of our study, we would expect it to be equal to or greater than the interrater reliability coefficient. Fugl-Meyer et al[2] reported scores of 13 patients they assessed on five occasions over a 12-month period. Patients' total scores from time 1 to time 5 improved from 7 to 83 points on the assessment, with the average change for the group calculated at 43.08 points (SD=29.13). If a large amount of "true" change occurs, then a SEM of 9.4 points is acceptable. if smaller change in the patient occurs, however, the magnitude of the error may be unacceptable. Error can be reduced by averaging over raters or occasions when change is not occurring. By increasing the raters or occasions on which a patient is assessed, the SEM would be reduced, as the error would be averaged across raters or occasions.[16]

The Fugl-Meyer assessment is well suited as a research tool because it is relatively easy to establish a high degree of reliability among several raters. In a clinical or research setting, standardization needs to be considered prior to using this assessment. A limitation of this assessment is the lack of administration guidelines. One strategy for reducing the measurement error is to standardize the administration guidelines of this assessment.

Anastasi[21] suggests that separate reliability coefficients should be reported for the sample subgroups, as the coefficients obtained are more likely to be applicable to clinical practice. Our sample was too small to calculate tight confidence limits for the reliability coefficients for subgroups, but this would be a worthwhile direction for further study.

Conclusions

The Fugl-Meyer assessment is designed to assess motor recovery following

stroke. The interrater reliability of measurements obtained with this assessment was tested by three experienced physical therapists on 12 patients in a rehabilitation population. It is a relatively simple assessment to administer and requires minimal training. The overall reliability for this instrument was high (ICC=.96), as were the reliability measurements for the subsections of this assessment, with the exception of pain. Error measured in absolute terms may be significant, however, when only small changes in the patient's level of motor performance are expected. These results would suggest that the Fugl-Meyer assessment is a moderately reliable measure for assessing impairment in a population of patients undergoing rehabilitation following stroke.

References

[1] Kurtzke JF, Kurland LT. Epidemiology of cerebrovascular disease. In: Joint Council on Cerebrovascular Disease, National Institute of Neurological Diseases and Stroke and National Heart and Lung Institute. Rochester, Minn: Whiting Press; 1970:163-175. [2] Fugl-Meyer AR, Jaasko L, Leyman I, et al. The post-stroke hemiplegic patient, I: a method for evaluation of physical performance. Scand J Rehabil Med. 1975;7:13-31. [3] Twitchell TE. The restoration of motor function following hemiplegia. Brain. 1951;74:443-40. [4] Brunnstrom S. Motor testing procedures in hemiplegia: based on sequential recovery stages. Phys Ther. 1966;46:357-375. [5] International Classification of Impairments, Disabilities, and Handicaps A Manual of Classification Relating to the Consequences of Disease. Geneva, Switzerland: World Health Organization; 1980. [6] Fugl-Meyer AR. Post-stroke hemiplegia: assessment of physical properties. Scand J Rehabil Med Suppl. 1980;7:85-93. [7] Badke MB, Duncan PW. Patterns of rapid motor responses during postural adjustments, when standing, in healthy subjects and hemiplegic patients. Phys Ther. 1983;63:13-20. [8] De Weerdt WJG, Harrison MA. Measuring recovery of arm-hand function in stroke patients: a comparison of the Brunnstrom-Fugl-Meyer test and Action Research Arm test. Physiotherapy Canada, 1985;37:65-70. [9] Berglund K, Fugl-Meyer AR. Upper extremity function in hemiplegia. Scand J Rehabil Med. 1986;18:155-157. [10] Dettman MA, Linder MT, Sepic SB. Relationships among walking performance, postural stability, and functional assessments of the hemiplegic patient. Am J Phys Med. 1987; 66:77-90. [11] Wood-Dauphinee S, Williams JI, Shapiro SH. Examining outcome measures in a clinical study of stroke. Stroke. 1990;21:731-739. [12] Arsenault B, Dutil E, Lambert J, et al. An evaluation of the hemiplegic subject based on the Bobath approach. Scand J Rehabil Med. 1988;20:13-16. [13] Kusoffsky A, Waddell I, Nilsson BY. The relationship between sensory impairment and motor recovery in patients with hemiplegia. Scand J Rehabilmed. 1982;14:27-32. [14] Duncan PW, Propst M, Nelson SG. Reliability of the Fugl-Meyer assessment of the sensorimotor recovery following cerebrovascular accident. Phys Ther. 1983;63:1606-1610. [15] Di Fabio RP, Badke MB. Relationship of sensory organization to balance function in patients with hemiplegia. Phys Ther. 1990;70; 542-548. [16] Gowland C, Torresin W, Stratford PW; et al. Chedoke-McMaster Stroke Assessment: a comprehensive clinical and research measure. In: Proceedings of the 11th International Congress of the World Confederation for Physical Therapy, Barbican Centre, London. 1991;2: 851-853. [17] Shrout PE, Fleiss J. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420-428. [18] Streiner DL, Norman GR. Health Measurement Scales A Practical Guide to Their Development and Use. New York, NY: Oxford University Press Inc; 1989:88-89. [19] Rothstein JM. Measurement in Physical Therapy. New York, NY: Churchill Livingstone Inc; 1985:1-46. [20] Cohen J. Statistical Power Analysis for the Behavioural Sciences. New York, NY: Academic Press Inc; 1977:286-276. [21] Anastasi A. Psychological Testing. 6th ed. New York, NY: PF Collier Inc; 1988:102. [22] Lindmark B, Hamrin E. Evaluation of functional capacity after stroke as a basis for active intervention. Scand J Rehabil Med. 1988;20: 111-115. [23] Anderson TP, Bourestorn N, Greenberg FR, Hilyard VG, Predictive factors in stroke rehabilitation. Arch Phys Med Rehabil. 1974;55:545-553. [24] Ottenbacher KJ, Johnson MB, Hojem M. The significance of clinical change and clinical change of significance: issues and methods. Am J Occup Ther. 1988;42:156-163.