Accuracy of observational kinematic assessment of upper-limb movements.

Physical therapists commonly use observation to evaluate the movement control deficits of people with neurological impairments following a cerebrovascular accident (CVA).[1-6] Such observations are used to generate hypotheses regarding impaired muscle activity due to central nervous system damage and to form bases for clinical decision making about treatment strategies.[4-7] Sound clinical decisions rely in part on the accuracy and reliability of therapists' observational assessment of movement. The importance of observational assessment to physical therapists is evidenced by studies that have examined the reliability of observational assessment of gait abnormalities in people with a variety of impairments.[8-11]

These studies[8-11] have demonstrated that observational assessment is, at best, only moderately reliable. The absence of strong evidence of reliability also suggests that observational assessment is likely to have poor validity. One of the few studies to examine the concurrent validity of observational assessment,[12] however, has demonstrated that judgments based on observation can indeed be valid. Spencer et all[12] examined the accuracy of physical therapists' judgments of the temporal symmetry of gait of 14 patients following CVA. Time spent in stance on the weaker lower extremity, as determined using a Clinical Stride Analyzer, formed the criterion measure or gold standard. The terapists' judgments of gait symmetry were found to be accurate when viewed from either the front (average r = .88) or the side (r = .87). These results suggested that observational assessment warranted further investigation. Moreover, support for the highly skilled nature of human perception is provided by many studies outside the field of physical therapy that have demonstrated the ability of observers to use kinematic cues from objects and humans to make accurate judgments while the subjects were driving[13, 14] or participating in sports[15,16] or to aid them in recognizing human activity, such as walking, running, or dancing, when only points of light from reflective markers positioned over major joints were visible.[17] Studies using point light displays have clearly demonstrated that the kinematic information in these displays is sufficient to enable observers to not only recognize activities but make accurate inferences about kinetic features of motion such as the weight of a lifted box[18] or the weight lifted during a one-arm curl.[19] These findings suggest that physical therapists should have the capacity to make accurate judgments about the kinematic patterns of their patients' movements.

The advent of instrumented kinematic measurement has created the opportunity for more detailed study of the accuracy of observational assessment of kinematics. The primary purpose of our study was to investigate how accurately physical therapists could make visual judgments about the upper-limb kinematics of people without neurological impairments and of individuals with impairments following a CVA. We termed this form of assessment observational kinematic assessment (OKA).

Selecting Variables for the Study of OKA Accuracy

An important design issue in studies of visual judgments of observed phenomena is the selection of cues to which observers are directed to attend when making judgments. As Scholz pointed out in his discussion of the analysis of motor performance in people with movement disorders, "there is a plethora of potential variables that could be measured."[20(p77)] Two noticeable features of previous studies of OKA have been the diversity of variables studied and the lack of consensus among researchers in selecting variables to be rated by therapists. For example, 38 variables were selected to be rated in previous studies of the reliability of observational assessment of gait.[8-11] These variables included evaluation of lower-limb excursion during step and stance phases and rating of upper-body movement, step width, and cadence of gait. Lack of consensus among researchers about variable selection is reflected in the fact that abnormality in knee range of motion during the stance phase was the only variable common to all four studies. The lack of consensus shown in these studies is also reflected in clinical practice. Observational assessment of gait is widely practiced, yet no standardized assessment protocol is universal use.[2]

The disagreement in selection of kinematic variables used for studies of observational assessment suggested to us that consensus methods or expert opinion would not be helpful in selecting variables to study. We chose instead to review the literature for studies identifying kinematic variables that clearly discriminated the performances of people with neurological impairments from those of individuals without neurological impairments and that also reflected changes in performance. These criteria are consistent with clinical goals of observational assessment, that is, to identify movement abnormalities and to monitor recovery after neurological damage.[4] Three kinematic variables emerged as potential variables for the study of OKA of upper-limb movement control: speed of movement, jerkiness, and hand path indirectness.

Speed of movement, defined by authors as either peak movement speed[21-26] or movement time,[23] has been shown to discriminate between the performances of individuals without neurological impairments and those of individuals who have had a CVA. People with upper-limb hemiparesis were found to have marked reduction in movement speed when compared with individuals without neurological impairments during the performance of either self-paced elbow movements[21] or single-joint tracking tasks.[22] The study of reaching by Fisk and Goodale[23] also distinguished between the performances of the less affected arm of individuals with left- and right-sided hemiplegia and control subjects without neurological impairments. In addition, movement speed has been found to be a sensitive measure of recovery in motor performance following a CVA.[24-26]

Lack of movement smoothness, or movement jerkiness, has also been shown to differentiate degrees of neurological impairment.[27-30] The speed traces of individuals without neurological impairments typically contain a single peak, or a single movement unit (where one movement unit is equivalent to one acceleration followed by a deceleration).[27] In contrast, people with impairments after a CVA[28-30] or cerebral palsy[31] have been shown to exhibit multiple peaks or movements units during performances of a variety of upper-limb tasks. Two of these studies[28,31] demonstrated that the variety of movements units decreased over time, and the authors in these studies suggested that jerkiness is a useful indicator of improved movement control. The path of the hand during a movement also has been shown to be useful for discriminating between the performances of people without neurological impairments and those of individuals following CVAs and other neurological injuries and for reflecting changes in performance over time.[28,30,32,33] These studies[28,30,32,33] showed that the hand paths of individuals with neurological impairments were less direct than those of people without neurological impairments over a range of upper-limb tasks. Movement speed, jerkiness, and hand path indirectness were consequently selected as kinematic variables for this study of the accuracy of therapists' OKA of upper-limb movements. A further aim of our study was to identify the stability (reliability) of these visual judgments over time.

Method

Subjects

Ten physical therapists with between 7 and 14 years (X = 10.5, SD = 2.2) of experience since graduation and at least 5 years (X = 8.2, SD = 2.6, range = 5-13) of experience working with people who have had CVAs served as observers in this study. Eleven people with a primary diagnosis of cortical or subcortical CVA and 4 older people without neurological impairments who were friends or relatives were recruited from the North West Hospital (Melbourne, Victoria, Australia) Rehabilitation Program to act as "performers" in the study (Tab. 1). Individuals with CVA were required to be able to perform item 4 of the "Upper-Arm Function" section and item 1 of the "Hand Movements" section of the Motor Assessment Scale.[34] These criteria ensured that these subjects were able to grasp and raise an object to at least 90 degrees of shoulder flexion. All therapists and performers gave informed consent prior to commencement of the study.

[TABULAR DATA 1 NOT REPRODUCIBLE IN ASCII]

Apparatus

The task (item 1 from the "Subtest Grasp" section of the Action Research Armtest[35]) required the seated subjects to grasp a wooden block from a table and lift it onto a shelf. This task was modified in that the block rested on pressure-sensitive boxes at the beginning and the end of the movement. Position and height of the shelf were adjusted to each person's arm length such that the subject potentially achieve full extension of the elbow and had moved through a shoulder flexion range of 115 degrees (25 [degrees] above the horizontal) at the completion of the task. Using adhesive tape, a reflective marker was fixed to the skin on the dorsal surface of the wrist crease in line with the base of the third metacarpal.

Performances were videotaped using three cameras (Fig. 1). Videotaped data were collected at 50 Hz by Panasonic WV F15E video cameras(*) on SONY Super VHS MQSE-120 videotape [dagger] with a shutter speed of 1/500 of a second. To obtain the criterion measurements of limb kinematics, data from two cameras, place 90 degrees apart and 1.6 m away from the seated performer, were processed using the Peak Motion Measurement System. [double dagger] The Peak Motion Measurement System was calibrated to achieve accuracy of position on a standard fixed reference frame within [+ or -] 0.5 mm. Positional data were filtered with a low-pass, fourth-order zero-lag Butterworth digital filter using a cutoff frequency of 6 Hz. A third camera, also placed 1.6 m away from the seated subject, recorded the view of the performance to be shown to the therapists. This view was equivalent to the therapist standing on the weaker side of the seated performer, a little behind the subject's shoulder. Camera position remained constant across subjects. Movement time was measured as the time between voltage changes in a circuit containing two pressure-sensitive switch boxes.

[FIGURE 1 ILLUSTRATION OMITTED]

Procedure

All subjects who served as performers attended a single session at the movement laboratory of La Trobe University (Victoria, Australia), during which they were video-taped performing a number of trials of the transport task. Each subject complete a maximum of 10 trials. Seven trials were completed by all performers, with trials repeated when technical problems arose. Data collection took approximately 1 hour for each subject.

Identifying peak movement speed, jerkiness, and path indirectness. Peak movement speed was identified from resultant speed traces of each performance. Path indirectness (the directness index) was defined as the difference between the actual path and the shortest path of the hand.[36] The more direct a subject's movement was, the closer the directness index value was to 1; the more indirect the hand path was, the higher the directness index value was. Jerks were identified iteratively from the resultant speed traces, based on the method described by Jagacinski et al.[37] Values for all maxima and minima in the movement speed profiles, and the times of these peaks, were recorded. A jerk was considered to be present when the period between two troughs was longer than 75 milliseconds and when the average speed ([V.sub.av]) between two adjacent peaks ([V.sub.1] and [V.sub.2]) minus the value of the minimum speed ([V.sub.min]) between the two peaks ([V.sub.diff]) was equal to or greater than 10% of the average speed (Fig. 2).

[FIGURE 2 GRAPH OMITTED]

Obtaining therapists' data. Videotape from the therapist's camera view was edited to generate three video-tapes from which therapists made their visual judgments. One videotape was generated for each variable: movement speed, jerkiness, and path indirectness. These videotapes included only those parts of each subject's movement from the frame in which the block was raised from the first pressure-sensitive box to the frame in which the block touched down on the second box. Each videotape contained 10 performances, which were selected from all performances of the 11 people with CVAs and the 4 people without neurological impairments. Although the performers completed several trials of the transport task, only one trial as selected to be edited onto the videotape to be viewed by the therapists. These trials represented the widest possible range and even spread of values for the variable. The 10 performances were arranged on the videotape in random order and then videotape in random order and then repeated in reverse order to form a counterbalanced judgment structure.[38] Values for all three variables included on the three videotapes in order of presentation to the therapists are shown in Table 2.

Table 2. Values of Peak Movement Speed, Number of Jerks, and Directness Index of Performances Rated by Physical Therapists in Order of Presentation

Peak Movement   Directness   No. of
Speed (cm/s)   Index        Jerks

 76.4           1.28          5
107.1(a)        1.20          0(a)
 40.0           1.32          1(a)
 47.8           1.16         10
 54.9           1.12(a)       8
136.8(a)        1.35          3
 72.4           1.24          6
 35.8           1.18(a)       6
 64.0           1.53          4
 79.4           1.17(a)       6

(a) Values represent performances by performers without neurological impairments.

All 10 therapists attended two viewing sessions 1 week apart. These sessions were held in a quiet room at La Trobe University, with each session taking 1.5 hours. The therapists received no formal training, but they were familiarized with the scales they would use to rate performances and with the nature of the task prior to commencement of data collection. The therapists recorded their judgments of movement speed, jerkiness, and path indirectness by marking a 100-mm line. This line represented a visual analog scale (VAS). The ends of the VAS were labeled "very slow" and "very fast" for movement speed, "very jerky" and "very smooth" for jerkiness, and "very direct" and "very indirect" for path indirectness. Comparisons with a normal model were encouraged by the presence of an additional VAS at the top of each page, on which therapists recorded where they considered normal performances would lie with respect to each kinematic variable. The therapists were allowed to view each performance up to five times before making their judgments. Response sheets were collected after each set of 10 performances.

Data Analysis

Distances along the VAS for the two judgments of each performance were averaged for each therapist. The relationships between the therapist's mean judgment of each performance shown for each variable and the criterion kinematic values were then investigated by regression methods. This method allowed careful investigation of the nature of the errors associated with observational assessment. Separate examination of constant error, demonstrated in the slopes and intercepts of the regression model, and variable error, demonstrated by the degree of scatter around the line of best fit, were deemed useful in this exploratory study of OKA. The relationship between the therapists' scores and the criterion measures represented the judgment model of the therapists. Linear and nonlinear regression models were investigated for each therapist's data and examined for goodness of fit. Log-transformed data were examined in order to investigate the possibility of a nonlinear relationship between visual judgments and the physical scales made up of 10 performances. Nonlinear relationships have been found in other psychophysical studies of perceptual or sensory phenomena.[39-41]

The accuracy of the therapists' visual judgments was evaluate by inspection of the regression models for scatter around the line of best fit and was expressed as the coefficient of determination (r). The standard error of estimate (SEE) provided a metric index of error.[42] The mean SEE was reported for each therapist's judgments of the three variables.

Intertherapist and test-retest reliability were also evaluated using regression methods in order to partition error into constant and variable sources. The slope and intercept variability resulting from comparisons between the regression models for each therapist with each of the other therapists' models are described. Average inter-therapist agreement was computed for each therapist. A summary estimate of mean variable error of any single therapist's model against that of all other therapists is expressed as the mean coefficient of determination, which represented how much the pattern of judgments and associated errors of that therapist corresponded with those of all other therapists. The resulting coefficients of determination were converted to z scores using the Fischer z transformation[43] before the mean of these nine scores was calculated. The mean z score was then converted back to a coefficient of determination, which reflected the average correlation between any single therapist and all other therapists in the group. Therapists' scores for each variable during sessions 1 and 2 (test-retest reliability) were also examined using regression methods. In addition, intraclass correlation coefficients (ICC[2,1]) were calculated to provide a useful measure of the total error associated with the test-retest and intertherapist reliability of observational assessment.

Results

Judgment Model Characteristics and Accuracy

Examination of the regression scattergrams and the coefficients of determination of the regression equations constructed for raw data and log-transformed values of the therapists' scores revealed that a linear model best described the relationships between the criterion measures and the therapists' visual judgments of movement speed, jerkiness, and path indirectness. Figure 3 demonstrates the linear relationship between therapists' judgments and the criterion measures for all three variables. and the criterion measures for all three variables. Examples of therapists' models with both lowest residual errors (greatest accuracy) and highest residual errors (lowest accuracy) are shown in Figure 3.

[FIGURE 3 ILLUSTRATION OMITTED]

Correlations between therapists' scores and the criterion measures for movement speed and path indirectness for session 1 are shown in Table 3. In all cases, data from session 2, performed 1 week later, closely resembled data from session 1. The correlations (r) for jerkiness (Tab. 4) were much lower than the correlations for movement speed and path indirectness and ranged from .22 to .70 in session 1. In contrast, intertherapist agreement for jerkiness showed that therapists' scores were highly correlated, with coefficients of determination ranging between .82 and .95 (Tab. 5). The good inter-therapist agreement suggested that a common judgment model was operating. The low correlations between the therapists' judgments and the instrumented data indicated that the operational criterion for jerkiness that we had developed did not represent the therapists' model. In an attempt to better match the model, the number of jerks in the movement speed profile was thus recalculated using alternative threshold values for the difference ([V.sub.diff]) between [V.sub.av] and [V.sub.min]. Originally set at 10%, iteration through threshold values in 5% steps was conducted. Following each incremental calculation, correlations between visual judgments and the new jerkiness scores that were created were examined. Optimal agreement occurred when only those jerks with greater than 25% difference between [V.sub.av] and [V.sub.min] were counted. Coefficients of determination of .78 to .90 were then found between visual judgments of jerkiness and the criterion values (Tab. 4), representing good levels of accuracy. This revised operational definition of jerkiness was used for the remainder of the analyses.

Table 3. Correlations (r) and Standard Errors of Estimate (SEE) Between Therapists' Scores an Criterion Measures for Peak Movement Speed and Path Indirectness for Session 1

            Peak             Path
            Movement         Indirectness
Therapist   Speed (cm/s)     (Ratio A/S)(a)
No.         r         SEE    r           SEE

 1          .88       15.0   .72         .08
 2          .96        8.4   .70         .09
 3          .94       10.7   .79         .07
 4          .89       14.2   .93         .04
 5          .95        9.4   .68         .09
 6          .88       15.1   .95         .04
 7          .87       15.4   .75         .08
 8          .88       14.8   .69         .09
 9          .87       15.7   .77         .08
10          .94       11.0   .68         .09

(a)Ratio of actual path length divided by shortest path length.

[TABULAR DATA 4 NOT REPRODUCIBLE IN ASCII]

Table 5. Intertherapist Reliability Expressed as Mean (r) Across Therapists for Movement Speed, Jerkiness, and Path Indirectness and Intraclass Correlation Coefficients (ICC)

Therapist   Movement                    Path
No.         Speed (r)   Jerkiness (r)   Indirectness (r)

 1          .90         .89             .92
 2          .93         .91             .91
 3          .90         .92             .88
 4          .89         .91             .89
 5          .86         .88             .91
 6          .92         .89             .87
 7          .92         .95             .84
 8          .91         .92             .90
 9          .89         .82             .90
10          .87         .93             .91

ICC         .85         .65             .75

In general, therapists with lower levels of accuracy showed higher residual errors when judging individuals with more abnormal performances. For example, those therapists who judged movement speed less accurately than their peers showed higher residual errors when rating performances with peak movement speeds below 64.0 cm/s (Fig. 3). The trend for more abnormal performances to be associated with higher residual errors also appeared to be present in path indirectness scatters. Average levels of error, represented by SEEs, are shown in Tables 3 and 4.

Reliability

Intertherapist agreement for all three variables was high, with coefficients of determination of .82 or higher (Tab. 5). Differences in scale use were reflected in variable mean slopes and intercepts across therapists. Slopes for visual judgments ranged from 0.5 to 1.8 for movement speed, from 0.5 to 1.5 for jerkiness, and from 0.4 to 2.1 for path indirectness. Variability between therapists was also found for intercepts, which ranged from -- 18.9 to 26.5 for movement speed, from -- 20.9 to 39.8 for jerkiness, and from -- 113.9 to 61.2 for path indirectness. This variability in constant error represented therapists using different units of measurement or a different range of VAS scores, and was reflected in lower ICCs (Tab. 5). Highly consistent and stable judgments (test-retest reliability) were obtained for judgments of all three variables. Slopes of 1 and intercepts of 0 were commonly found. The high ICCs reflect this finding (Tab. 6).

Table 6. Stability of Therapists' Judgments Over Time (Test-Retest Reliability) Expressed as Coefficient of Determination (r) and Intraclass Correlation Coefficients (ICC[2, 1])

            Movement                 Path
Therapist   Speed       Jerkiness    Indirectness
No.         r     ICC   r      ICC   r        ICC

 1          .97   .96   .98    .96   .97      .93
 2          .98   .92   .98    .93   .94      .95
 3          .97   .96   .98    .94   .98      .92
 4          .97   .92   .92    .85   .93      .83
 5          .89   .88   .93    .93   .82      .79
 6          .94   .93   .99    .98   .98      .96
 7          .98   .92   .97    .96   .85      .82
 8          .95   .82   .90    .90   .97      .88
 9          .93   .77   .93    .93   .89      .88
 10         .94   .76   .98    .95   .95      .81

Discussion

The main aims of our study were to investigate the accuracy of physical therapists' judgments of the kinematics of upper-limb movements and to examine the nature of the regression model that best described the relationship between these visual judgments and physical reality. A simple linear model described the relationship between the therapists' scales and the physical scales constructed using the performances of people with CVAs and people without neurological impairments. This finding supports the notion that therapists view abnormal movement along a continuum that includes normal movement.

Accuracy of Therapists' Judgments

The accuracy of therapists' visual judgments varied somewhat for movement speed, jerkiness, and path indirectness. Of particular note were therapists' judgments of speed of movement, which showed the lowest residual errors and the highest levels of accuracy. These findings were reflected in strong correlations between therapists' scores and the physical scales. Therapists' judgments of jerkiness were also found to be moderately to highly accurate following post hoc iteration of the criterion measure. The new operational definition for jerkiness derived from iteration discounted jerks in the speed trace that were below a threshold amplitude and resulted in marked improvements in correlations between visual judgments of jerkiness and the criterion measure. Following iteration, accuracy levels for jerkiness approached those for speed of movement. This finding suggested that visually detectable jerkiness was highly dependent on the amplitude, as well as the frequency, of jerks occurring in a movement performance.

The accuracy of visual judgments of path indirectness appeared to be lower and more variable (r=.68-.93) than for movement speed and jerkiness. As with jerkiness, however, high levels of intertherapist agreement indicated that something other than the criterion definition of path indirectness was driving human perception. This mismatch between therapists's scores and physical scales represent a dilemma in the study of the accuracy of physical therapists' judgments of kinematics. To clarify this issue, consider path indirectness, which was defined on the basis of physics as the difference between the shortest possible path of the object to the target (representing normal movement strategies) and the actual path of the object that incorporated substantial deviation away from the shortest path by up to 50%. The data showed that levels of accuracy were lower in the presence of uniformly high levels of intertherapist agreement (average r=.84-.92). Allowing for individual variability in therapists' scale units, this result strongly indicated that therapists were judging path indirectness using cues that did not appear to be fully represented in the criterion measure adopted for this study.

Inspection of the regression models confirmed that therapists' residual errors for the 10 performances were very similar in magnitude and direction (Fig. 4). This tightly clustered pattern of errors across therapists indicates that residual errors were not random but instead represented potentially meaningful differences between the cues accessed by therapists to determine path indirectness and the criterion measure of path indirectness. Further research to elucidate the nature of this mismatch has the potential to lead to judgments of improved accuracy for path indirectness. Therapists with higher mean residual errors were usually found to have larger errors associated with visual judgments of the more abnormal performances (ie, they appeared to be less able to discriminate among individuals with neurological impairments who had slow movement speeds or very indirect hand paths). Alternatively, this finding may reflect and unconscious threshold value for movement abnormality, below which some therapists see little importance in making discriminating judgments, for example, between very slow (and highly abnormal) and extremely slow (and highly abnormal) movements. This explanation may account for the groupings of performances at the very slow end of the scattergrams for peak movement speed shown in Figure 3. The consistent pattern in direction and magnitude of the errors associated with judgments of path indirectness (Fig. 4), however, are more readily attributable to a mismatch between our operational definition and the cues used by therapists to determine path indirectness than to the lack of a perceived need to discriminate between more abnormal performances.

[FIGURE 4 ILLUSTRATION OMITTED]

Stability of Therapists' Judgment Models

We believe that the highly stable nature of the judgment models generated by the therapists is important. Correlations between visual judgment scores across sessions were uniformly high, with slopes of close to 1 commonly found. This finding indicates that although therapists used differently scaled units (some therapists marked performances at the extreme ends of the VAS, whereas other therapists used more restricted ranges), there was little variability in the units used by individual therapists. The stable nature of these judgments models is particularly encouraging, and we believe that their stability supports the future development of methods to calibrate individual models, which may result in further improvements in intertherapist agreement.

OKA Research Methodological Issues

Our study departed in method from previous studies of observational assessment in a number of ways. Visual analog scales provided therapists with a framework for making judgments about movement kinematics. These scales had anchor labels that represented "very much" or "very little" of each of the three kinematic variables: movement speed jerkiness, and path indirectness. This method of accessing therapists' judgments contrasted with scales that required therapists to rate the presence or absence of abnormality of a kinematic variable[11] or the degrees of abnormality or normality using either three-point[8-10] or five-point[12] scales. Nevertheless, the units used by therapists in our study were also likely to reflect units of kinematic abnormality.

Differences in therapists' scales that are apparent from the variability in slopes and intercepts of individual therapist model may represent individual calibration of these units of kinematic abnormality. Individual differences in calibration of interval units of abnormality may have resulted from differing therapists' experiences with respect to the variety of movement abnormalities that they have seen in people who have had CVAs and may have led to the scaling variability found among the therapists. We used experienced therapists as observers in an attempt to diminish these effects; however, heterogeneity in scale usage remained. Although some degree of heterogeneity is to be expected in any study of human subjects, some therapists not only differed in scale use but were more accurate at OKA than other therapists were.

Prior training of therapists to rate movement variables with which they may have been unfamiliar could have improved both judgment accuracy and intertherapist agreement. There was no prior training of observers in our study. We aimed instead to identify the baseline accuracy of judgments of physical therapists who work with people with neurological impairments as they applied existing clinical observation techniques to the assessment of the upper-limb transport task. The results of previous studies of observational assessment that used some form of training of observers[8,10,11] did little to encourage us to use training. These studies failed to produce more than poor to moderate levels of reliability, even under conditions of extensive (2 hours) training prior to observational assessment.[11] The question of whether perceptual training enhanced the ability of therapists to make more accurate and reliable visual judgments about movement warrants further study.

The most important methodological difference between our study and the majority of earlier studies was our inclusion of criterion measures. The use of instrumented kinematics enabled the construction of physical scales of known magnitudes against which therapists' judgments could be compared. Although we have demonstrated that the development of appropriate operational definitions for observable kinematics is not simple, the advantages gained from our ability to closely inspect the resulting regression models representing therapists' visual judgments are clear. We have been able to determine the accuracy of therapists' judgments of three potentially important kinematic cues for evaluating movement abnormality in the upper limb following a CVA. Moreover, examining the nature of the errors made by therapists for patterns or trends has enable us to determine when mismatches between visual judgments and the physical scales have occurred and, in the case of movement jerkiness, stimulated successful development of a more representative kinematic definition.

Clinical Implications and Directions for Further Research

Observational assessment is commonly practiced for three main reasons: (1) It ease of application permits immediate evaluation of changes in performance, (2) there is a lack of alternative (and affordable) methods of instrumented measurement, particularly with respect to evaluation of control of the upper limb after neurological injuries, and (3) physical therapists believe that they are skilled at accurately assessing movement through observation.

Our study represents an early step in the process of evaluating observational assessment. We believe that it is important to determine whether the moderately to highly accurate judgments for movement speed, jerkiness, and path indirectness demonstrated by experienced therapists in our study can be produced in the clinical domain. An important question is whether OKA can be usefully applied as a measurement technique to discriminate change in patient performance, particularly in view of the potentially variable nature of upper-limb movements in some types of patients. In our study, in order to evaluate errors associated solely with the process of observation, we deliberately eliminated performance variability from our analysis. Videotaped performances affords the possibility of proportioning total variability in scores into components due to observer error and variability in patient performance. If patient performance is variable, however, OKA (or indeed any other form of measurement of kinematics) may not provide sufficiently stable data to permit the sensitive detection of treatment effects.

An additional question for future research is: What movement variables are most important for evaluation of upper-limb performance? We investigated only three movement variables in our study. These variables were selected from the literature because of their ability to reflect differences between the performances of individuals with CVAs and those of people without neurological impairments and to sensitively measure change in performance over time. Clinical support for the usefulness of observations of these upper-limb performance characteristics can be found in the physical therapy literature.[3,44-46] Other indexes of upper-limb performance, however, should be tested. Before standardized assessment (instrumented or observational) can be developed, physical therapists need to determine which of the multitude of indexes currently used are most representative of upper-limb performance and provide useful evaluation of treatment effectiveness. Furthermore, the issue of which movement tasks should be included in a test needs to be addressed. Research to identify the best indexes of performance, and the most representative upper-limb tasks, should precede application of OKA in the clinical arena.

Conclusion

Given the important contribution of assessment of movement pattern to therapists' decision making about treatment, the current finds are exciting. We now know that under conditions in which therapists can make standardized responses after reviewing several repetitions of patients' movements, accurate and reliable judgments are possible. It would be premature to attempt to generalize the current findings to the entire range of visual judgments needed in clinical practice. Rather, the findings point to the need for systematic investigation into the effect of individual differences, and additional kinematic variables, on judgments accuracy. Studies also need to be conducted to determine the kinematic variables that should be adopted for thorough, standardized evaluation of upper-limb movements using observational assessment. This research could be conducted within the present paradigm.

(*) Matsushita Electric Industrial Co Ltd, Central PO Box 288, Osaka, Japan 539-91.

[dagger] SONY Corp. Tokyo, Japan.

[double dagger] Peak Performance Technologies Inc. 7388 S Revere Pkwy, Ste 603, Englewood, CO 80112.

References

[1] Patla A, Proctor J, Morson B. Observations on aspects of visual gait assessment: a questionnaire study. Physiotherapy Canada. 1987;39:311-316.

[2] Malouin F. Observational gait analysis. In: Craik RL, Oatis CA, eds. Gait Analysis: Theory and Applications. St Louis, Mo: Mosby; 1995:112-124.

[3] Bobath B. Adult Hemiplegia: Evaluation and Treatment. 3rd ed. London, England: William Heinemann Medical Books Ltd; 1990.

[4] Carr J, Shepherd R. A Motor Relearning Programme for Stroke. London, England: William Heinemann Medical Books Ltd; 1982.

[5] Moore S, Schurr K, Wales A, et al. Observation and analysis of hemiplegic gait: swing phase. Australian Journal of Physiotherapy. 1993; 39:271-278.

[6] Moseley A, Wales A, Herbert R, et al. Observation and analysis of hemiplegic gait: stance phase. Australian Journal of Physiotherapy. 1993; 39:259-267.

[7] Herbert R, Moore S, Moseley A, et al. Making inferences about muscle forces from clinical observations. Australian Journal of Physiotherapy. 1993:39:195-201.

[8] Eastlack M, Arvidson J, Danoff J, McGarvey C. Interrater reliability of videotaped observational gait-analysis assessment. Phys Ther. 1991;71: 465-472.

[9] Goodkin R, Diller L. Reliability among physical therapists in diagnosis and treatment of gait deviations in hemiplegics. Percept Mot Skills. 1973;37:727-734.

[10] Krebs DE, Edelstein JE, Fishman S. Reliability of observational kinematic gait analysis. Phys Ther. 1985;65:1027-1033.

[11] Patla AE, Clouse SD. Visual assessment of human gait: reliability and validity. Rehabilitation Research. October 1988:87-96.

[12] Spencer KI, Goldie PA, Matyas TA. Criterion-related validity of visual assessment of the temporal symmetry of hemiplegic gait. In: Proceedings of the Australian Physiotherapy Association National Congress, 1992, Adelaide, Australia. Melbourne, Victoria, Australia: Australian Physiotherapy Association; 1992:68.

[13] Schiff W, Oldak R. Accuracy of judging time to arrival: effects of modality, trajectory, and gender. J Exp Psychol Hum Percept Perform. 1990;16:303-316.

[14] McLeod R, Ross H. Optic-flow and cognitive factors in time-to-collision estimates. Perception. 1981;12:417-423.

[15] Rippol H, Fleurance P. What does keeping one's eye on the ball mean? Ergonomics. 1988;13:1647-1654.

[16] Bootsma RJ, van Wieringen PCW. Timing an attacking forehand drive in table tennis. J Exp Psychol Hum Percept Perform. 1990;16:21-29.

[17] Johansson G. Visual perception of biological motion and a model for its analysis. Perception and Psychophysics. 1973;14:201-211.

[18] Runeson S, Frykholm G. Visual perception of lifted weight. J Exp Psychol Hum Percept Perform. 1981;7:733-740.

[19] Bingham GP. Kinematic form and scaling: further investigation on the visual perception of lifted weight. J Exp Psychol Hum Percept Perform. 1987;13:155-177.

[20] Scholz JP. Commentary on "Quantification of control: a preliminary study of effects of neurodevelopmental treatment on reaching in children with spastic cerebral palsy." Phys Ther. 1990;70-76-78.

[21] Corcos DM. Strategies underlying the control of disordered movement. Phys Ther. 1991;71:25-38.

[22] Jones RG, Donaldson IM, Parkin PJ. Impairment and recovery of ipsilateral sensory-motor function following unilateral cerebral infarction. Brain. 1989;112:113-132.

[23] Fisk JD, Goodale MA. The effects of unilateral brain damage on visually guided reaching: hemispheric differences in the nature of the deficit. Exp Brain Res. 1988;72:425-435.

[24] Broberg RA. Functional arm movement: the recovery of motor function following stroke. In: Proceedings of the Eleventh International Congress of the World Confederation for Physical Therapists, 1991, London, United Kingdom. London, England: The Chartered Society of Physiotherapy; 1991:1028-1030.

[25] Wing AM, Lough S, Turton A, et al. Recovery of elbow function in voluntary positioning of the hand following hemiplegia due to stroke. J Neurol Neurosurg Psychiatry. 1990;53:126-134.

[26] Trombly CA. Deficits of reaching in subjects with left hemiparesis: a pilot study. Am J Occup Ther. 1992;46:887-896.

[27] Fetters L, Todd J. Quantitative assessment of infant reaching movements. Journal of Motor Behavior. 1987;19:147-166.

[28] Lough S, Wing AM, Fraser C, Jenner JR. Measurement of recovery of function in the hemiplegic upper limb following stroke; a preliminary report. Human Movement Science. 1984;3:247-256.

[29] Trombly CA. Observations of improvement of reaching in five subjects with left hemiparesis. J Neurol Neurosurg Psychiatry. 1993;56:40-45.

[30] Charlton JL. Motor control considerations for assessment and rehabilitation of movement disorders. In: Summers JJ, ed. Approaches to the Study of Motor Control and Learning. Amsterdam, the Netherlands: Elsevier Science Publishers BV; 1992-441-467.

[31] Kluzic J, Fetters L. Coryell J. Quantification of control: a preliminary study of effects of neurodevelopmental treatment on reaching in children with spastic cerebral palsy. Phys Ther. 1990;70:65-76.

[32] Chieffi S, Gentilucci A, Allport A, et al. Study of selective reaching and grasping in a patient with unilateral parietal lesion. Brain. 1993;116:1119-1137.

[33] Inzelberg R, Flash T, Schechtman E, Korcyn A. Kinematic properties of upper limb trajectories in idiopathic torsion dystonia. J Neurol Neurosurg Psychiatry. 1995;58:312-319.

[34] Carr JH, Shepherd RB, Nordholm L, Lynne D. Investigation of a new motor assessment scale for stroke patients. Phys Ther. 1985;65:175-180.

[35] Lyle RC. A performance test for assessment of upper limb function in physical rehabilitation and research. Int J Rehabil Res. 1981;4:483-492.

[36] Grant C. 3D Path Calculation Software. Melbourne, Victoria, Australia: La Trobe University; 1992.

[37] Jagacinski RJ, Reppenger DW, Moren MS, et al. Fitt's law and the microstructure of rapid discrete movements. J Exp Psychol Hum Percept Perform. 1980;6:309-320.

[38] Poulton EC. Bias in Quantifying Judgments. East Sussex, England: Lawrence Erlbaum Associates Ltd: 1989.

[39] Maher C, Adams R. A psychophysical evaluation of manual stiffness discrimination. Australian Journal of Physiotherapy. 1995;41:161-167.

[40] Brown JF. The visual perception of velocity. In: Spigel IM, ed. Readings in the Study of Visually Perceive Movement. New York, NY: Harper & Row; 1965:64-107.

[41] Spigel IM, ed. Readings in the Study of Visually Perceived Movement. New York, NY: Harper & Row; 1965.

[42] Polgar S, Thomas S. Introduction to Research in the Health Sciences. Edinburgh, Scotland: Churchill Livingstone; 1988.

[43] Altman D. Practical Statistics for Medical Research. London, England: Capman and Hall; 1991.

[44] Carr JH, Shepherd RB, Gordon J, et al. Movement Science: Foundations for Physical Therapy in Rehabilitation. Gaithersburg, Md: Aspen Publishers Inc; 1987.

[45] Mulder T, Pauwels F, Nienhuis B. Motor recovery following stroke: towards a disability-orientated assessment of motor dysfunction. In: Harrison M, ed. Physiotherapy in Stroke Management. Edinburgh, Scotland: Churchill Livingstone; 1995:275-282.

[46] Sawner KA, La Vigne JM. Brunnstrom's Movement Therapy in Hemiplegia: A Neurophysiological Approach. 2nd ed. Philadelphia, Pa: JB Lippincott Co; 1992.

J Bernhardt, PT, is a doctoral candidate in the Schools of Physiotherapy and Psychological Science, La Trobe University, Melbourne, Australia. She completed this research in partial fulfillment of her degree requirements. Address all correspondence to Ms Bernhardt at Physiotherapy Department, Faculty of Health Sciences, La Trobe University, Melbourne, Victoria, Australia 3083 (jbernhardt@latrobe,edu.au).

PJ Bate, PhD, PT, is Reader, School of Psychological Science, La Trobe University.

TA Matyas, PhD, is Reader, School of Psychological Science, La Trobe University.

This study was approved by the North West Hospital Human Research and Ethics Committee and by the La Trobe University Human Research Ethics Committee.

This research was supported by a grant from the Australian Physiotherapy Association (Victorian Branch) Research Committee.