Reliability and validity of the WATSMART Three-dimensional Optoelectric Motion Analysis System.

Reliability and Validity of the WATSMART[TM] Three-dimensional Optoelectric Motion Analysis System

Reliability and validity of the WATSMART[TM] (Waterloo Spatial Motion Analysis Recording Technique) system was evaluated under static and dynamic conditions. In experiment 1, infrared light-emitting diodes (IREDs) were placed at the axis and along the arms of a clinical goniometer. Twelve angles in 5-degree increments were each recorded 10 times at each of three spatial locations. Reliability was assessed using intraclass correlation coefficients (ICCs) and analysis-of-variance procedures to determine within-trial variability. The ICCs for all spatial locations exceeded .99. The 95% confidence interval for each angle was less than 0.5 degree in all cases. Criterion-referenced instrument validity was assessed with regression analysis. Slopes of the regression of reconstructed angle on reference angle were close to unity for each spatial location. A systematic error, however, that increased as the goniometer was rotated 45 degrees away from the cameras was evident. In experiment 2, a robotic arm was fitted with four IREDs and made to repeat a defined movement trajectory 10 times at each of three spatial locations. The ICCs for portions of each trajectory ranged from .20 to .99. The results show that reliable and valid results can be obtained from this motion analysis system if adequate precautions are taken to reduce unwanted light reflections. Reliability and validity decreased somewhat as the object was rotated further away from the plane in which the cameras were mounted. [Scholz JP: Reliability and validity of the WATSMART[TM] three-dimensional optoelectric motion analysis system. Phys Ther 69:679-689, 1989] Key Words: Equipment, general: Kinesiology/biomechanics, general; Motion; Tests and measurements, general. Recent technological advances have greatly enhanced our capabilities for quantifying human movement. Complementing older methods for obtaining objective kinematic data, such as 16-mm film and uniaxial electrogoniometry, are more sophisticated procedures that include various forms of videography, triaxial electrogoniometry, accelerometry, and optoelectric techniques. Many of the newer motion analysis systems are computer controlled and include "user-friendly" software for the control of data collection and analysis. Moreover, increased competition among manufacturers and technological improvements have reduced the cost of such systems so that they are becoming more readily available for the clinical assessment of movement disorders and rehabilitation research. Thus, there is hope that the assessment of patient dysfunction can progress beyond traditional observational techniques that are relatively subjective and often have questionable reliability and validity.(1,2) Although the use of such instrumentation in rehabilitation has primarily been for gait analysis,(3-8) objective studies of the control and coordination of upper extremity function and other behaviors are the focus of many rehabilitation specialists.(9-13) Thus, the choice of a particular technology is dependent on the particular application. For example, electrogoniometry may be useful for evaluating gait dysfunction but has limited applicability for the assessment of upper extremity function. Both 16-mm film and videography provide a greater range of application. The monetary cost of purchasing and processing 16-mm film and the time required for hand digitization of each frame, however, make this technique an impractical choice for clinical applications. Video equipment is readily available in many clinical settings, is less costly, and is easier to use. The quality of the instrumentation available, however, varies widely and can dramatically affect the reliability and validity of the results. Moreover, in most instances, hand digitization of the captured movement sequence is required, which may be too time-consuming for clinical application. Numerous newer video-based and optoelectric instruments are designed to provide objective spatial and temporal measurments of movement performance that can be acquired rapidly, digitized automatically, and stored on a computer disk.(14) Kinematic data obtained with such recording techniques can be reviewed, analyzed, and further manipulated (eg, determination of movement derivatives) off-line. Each type of system has its own advantages, and the system of choice will depend on the user's applications. For instance, video-based systems that provide for automatic digitization require the use of passive reflective markers on the subject. Thus, patient setup is relatively easy, and there is little likelihood of markers interfering with patient movement. Optoelectric systems require more careful attention to subject setup than video-based systems to minimize interference with the movement from wires that carry an electric signal to active markers. The spatial resolution obtained with optoelectric recording techniques, however, is often significantly better than that of video systems, and the available sampling rate far exceeds the 60-Hz standard of video systems (unless very expensive video components are purchased.) The choice of a particular system, therefore, requires a number of important considerations. These considerations should include not only the cost, the environment in which the system is to be used, and the particular applications, but also the type of questions that the researcher or clinician desires to address.(15) In addition, the careful examination of information about the resolution, reliability, and validity of each system is paramount. Unfortunately, although most manufacturers provide data about the resolution and accuracy of their systems, it is often difficult to interpret exactly what those data mean in an experimental or clinical context. The purpose of this study was to examine the accuracy and consistency of the WATSMART[TM] (Waterloo Spatial Motion Analysis Recording Technique),(*) a commercially available motion analysis system that has been used recently in several basic and clinically relevant studies of human movement. (9,12,16-18) The WATSMART[TM] system is a relatively new version of an optoelectric technique that uses active infrared light-emitting diodes (IREDs) for determining the position of limb segments in space. Infrared-based motion analysis systems use special cameras with optical lenses that focus the light pulse from the IREDs onto a special semiconductor diode surface.(14) This photodetector produces electrical signals that are proportional to the horizontal and vertical coordinates of the center of light intensity on the detector surface. This technique requires that only one IRED be visible at each sampling instant, which is guaranteed when multiple IREDs are used by firing them sequentially at intervals of several microseconds. Three-dimensional coordinates of each IRED can be obtained if two or more synchronized cameras, mounted in nonparallel planes, simultaneously view each IRED. A direct linear transformation (DLT) algorithm is used to obtain the three-dimensional coordinates from the combined two-dimensional coordinates of each camera.(14) In a recent article by Samuelson et al, the reliability of one such optoelectric, infrared-based system--the Selspot II[TM ]--was evaluated.(19) These investigators repeatedly recorded the static position of eight infrared markers located at various distances up to 10 m from two cameras. The cameras were first calibrated using a specially designed frame. The coefficient of variation of the three-dimensional coordinates of each IRED was generally well below 5%, although when the IRED was located near the extremes of the measurement area, variability was as high as 17%. The position of the eight IREDs within the measurement volume, however, was not varied by the investigators. A dynamic test of the system was also performed, but test results were presented qualitatively. Finally, although the authors discussed the loss of IRED signal power that occurs with rotation of an IRED away from the plane in which the cameras are mounted, they presented no quantitative information on the amount of measurement noise introduced by such rotation. This study provides a quantitative and functionally relevant evaluation of the reliability and criterion-referenced validity of the WATSMART[TM] system for measuring angular motion.

General Instrumentation and Setup

The WATSMART[TM] system is a three-dimensional motion digitizing and analysis system that can track as many as 64 individual IREDs attached to a subject. The active IREDs are detected by two or more optoelectric measurement cameras at distances from 1.1 to 8 m, without lens adjustment. Sequential firing of the IREDs is governed by a central controller unit via lightweight strobers that are attached to the subject. (The IREDs used in these experiments were actually "triple" IREDs, consisting of three individual light-emitting diodes connecting in a ring in series.) The maximum sampling rate that can be achieved is dependent on the number of IREDs used and the length of each trial, but for most applications can be in excess of 200 Hz if desired. The manufacturer reports camera accuracies of 0.2% across the 33-degree field of view for each axis (resolution: 1:4096). A calibration frame measuring 1 m(3) with 40 embedded IREDS is placed in the field of view of both cameras and used to calculate the camera's relative position. Calibration of a volume larger than the calibration cube is obtained by using a multiple calibration procedure in which the cube is moved to various locations and calibrated repeatedly. The multiple calibrations are then combined to yield a single calibration for the entire measurement volume. The WATSMART[TM] system was set up in an identical manner for both experiments reported in this article. Two cameras were attached to a bar that was mounted on one wall of a 4.6- X 7.3-m room. The cameras were mounted 2 m above the floor, 2 m apart, and were angled approximately 15 degrees inward and 15 degrees downward toward the recording area located on the opposite side of the room (4 m from the cameras' lenses). Camera position was chosen to minimize the average error in estimating the actual three-dimensional position of an IRED of the calibration frame from the coordinates obtained via the optoelectric cameras. The floor of the recording area was covered with 25-mm thick fiberboard that had been sprayed with a flat black solar paint to reduce light reflections. A background frame built of three 1.22- X 2.44-m painted sheets of the fiberboard, hinged so that the side sheets could be folded forward, created an alcove within which all calibration and testing was performed. The measurement field was approximately 1.2 m wide X 1.2 m deep X 1.8 m high. For the static tests performed in experiment 1, an IRED was placed at the axis and at the midline of each arm, 0.23 m from the axis, of a standard clinical goniometer. The goniometer had a 12.5-cm diameter hub. Care was taken to ensure that the IREDs were centered on the axis line of each arm. Because the goniometer is made of highly reflective clear plastic, the center hub and arms were covered with a 0.32-cm thick nonreflective black rubber covering to an extent that did not restrict movement of the arms nor obscure the angle marks (1 increments) on the hub. The goniometer arm continuous with the center hub was clamped to a small vise positioned between the axis and the arm's terminal marker, which provided a stable base for recording when the angle was selected by the investigator (JPS). The vise was placed on a wooden box (Fig. 1), and both were covered with the nonreflective rubber material. A Mini Mover-5 robot arm (Fig. 2) was used for the dynamic test of experiment 2. The robot provided a means of generating a predictable movement trajectory. A microcomputer controlled movement of the robot arm via an algorithm supplied by the manufacturer. Thus, the same joint and end-point trajectories could be reproduced accurately across all trials. The shoulder and elbow joints were constrained to move in the same plane, although the base could be rotated. The IREDs were mounted at the shoulder and elbow joint axes of the arm, at the base along a line projecting from the shoulder joint, and at the terminal end of the forearm along a line projecting from the elbow joint. These IRED locations allowed for later determination of shoulder and elbow joint angles, as well as the trajectory of the terminal IRED in space. Because the arm is made of highly reflective metal, it was covered with 0.32-cm thick nonreflective rubber material to an extent possible without interfering with the arm's motion. The motors controlling the joints, located at the base of the arm, were also covered. It was not possible, however, to cover the motor edges completely, and these uncovered areas may have served as a source of reflections from light emitted from the base IRED.

Experiment 1

Method

The first experiment was designed to address the following questions: 1) How reliable are individual measurements of known angular positions obtained by the WATSMART[TM] system? 2) Does measurement reliability change with the spatial location of the object within the measurement volume? 3) How much variation occurs over time in the reconstructed data of a given angular position? 4) Does this variability increase, and if so to what extent, as the object is rotated away from the plane of the cameras (ie, the plane containing the bar on which the cameras were mounted)? 5) How accurately does the reconstructed angle obtained from the WATSMART[TM] analysis reflect the true angle of the goniometer? and 6) To what extent is this accuracy affected, if at all, by the spatial location of the object? Procedure. Prior to the experiment, the cameras were calibrated for five different positions of the calibration cube (multiple calibration procedure) from a position in which the base of the cube rested on the floor to one in which the base rested on a box 0.91 m above the floor. The average error in detecting the position of each cube IRED with respect to the cube's origin was 2.3 mm for this experiment. Twelve different goniometric angles were selected and recorded for each of 10 trials at three different spatial locations (blocks). The angles selected ranged from 45 to 100 degrees in 5-degree increments. The order of angle presentation was randomized for each trial, and trials were randomized for each block. The experimenter positioned himself behind the box so that he could change the goniometer position without obstructing the cameras' view of the IREDs. The initially chosen angle was recorded for three seconds, after which the goniometer angle was changed. This procedure was repeated until all 12 angles were presented in a trial. In block 1 of each trial, the vise-held goniometer was placed on a wooden box positioned on its side, a distance of 0.6 m above the floor. The goniometer was oriented parallel to the plane of the cameras. For block 2, the box was placed on end so that the goniometer was 0.91 m above the floor and 0.3 m to the right of its location for block 1. The orientation with respect to the cameras remained the same. This same location was maintained for block 3, except that the goniometer was rotated 45 degrees away from the plane of the cameras. This rotation had the effect of moving the IREDs located on the goniometer arms further away from the cameras, decreasing the signal intensity of the IREDs to the cameras. Theoretically, this procedure should increase the detected signal's noise.(19) Block 3 measurements were performed to determine the extent of this effect. Following data collection, the digitized (100 Hz), two-dimensional coordinates of each IRED were converted to three-dimensional positions using a DLT algorithm supplied by the manufacturer. The data were then filtered with a forward-pass--backward-pass, second-order Butterworth filter having a cutoff frequency of 10 Hz. Following this procedure, angular positions of the goniometer were derived from the coordinates of the three IREDs using an algorithm provided by Northern Digital Inc. The investigator felt that angular measures would be more relevant kinematic measures than the IRED positions alone because angular joint excursions are often of primary interest in kinesiological and pathokinesiological research. In addition, any error in the coordinate determination of the infrared markers should be compounded in the angle calculations. Data analysis. Separate intraclass correlation coefficients (ICCs) were computed to determine the reliability (ie, of a single observation) of the angular data obtained for each spatial location (ie, position in the measurement field).(20) For each three-second epoch of recording an angle, the mean reconstructed angle and its standard deviation were determined. This temporal mean was obtained for each angle on each of the 10 trials, and these values were used in the data analysis. Ninety-five percent confidence intervals (CIs) for the difference between the ICCs were computed to determine whether reliability differed across spatial locations. The standard deviation of the reconstructed angle for each three-second epoch (ie, while the position of the goniometer remained stationary) was obtained to estimate the amount of variation (noise) in the data. A location-X-angle repeated-measures analysis of variance (ANOVA) for the temporal standard deviation was performed. The average standard deviation for each angle across trials and the 95% CI for the standard deviation were also calculated. Accuracy of the system was evaluated by performing a least-squares regression analysis, with the reference angle treated as a continuous independent variable, spatial location as a categorical independent variable, and the reconstructed angle as the dependent variable. This procedure allowed for the assessment of various types of error in the data.(21) For example, constant error is reflected by the deviation of the intercept of the resulting regression equations from zero, and proportional error by deviations in the slopes from unity.

Results

Mean reconstructed angles and their standard deviations for the three spatial locations are presented. Intertrial variability was slightly higher at the larger angles, although the standard deviation in all cases was less than 0.2 degree. The reliability of a single observation, determined by the ICC,(5) was .99 for each of the three spatial locations of the goniometer. Because the ICCs and their degrees of freedom were identical, the CI for differences between the ICCs was not derived. Table 2 presents the mean within-trial, or temporal, variability for each angle as well as the 95% CI rounded down (first value) or up (second value) to two places. These values provide a measure of the average variability of the reconstructed angle within each three-second epoch during which the goniometer was held stationary. Fluctuations may derive, in part, from digitization noise and other sources, although light reflections are the most likely source of error. The location-X-angle ANOVA revealed significant effects for location (F = 562.7; df = 2,27; p < .001), angle (F = 34.1; df = 11,297; p < .001), and the location-X-angle interaction (F = 1.7; df = 22,297; p < .05). Mean temporal variability tended to be larger for the largest angles at the first two spatial locations. The pattern of standard deviations was consistent across these two locations. For the third location, where the goniometer was rotated 45 degrees to the plane of the cameras, the standard deviation did not change significantly with angle. The overall level of fluctuation, however, was higher than when the goniometer was parallel to the plane of the cameras. Nevertheless, the 95% CI for angle variability was within 0.3 to 0.4 degree, even when the goniometer was rotated 45 degrees. Intercepts and slopes from the regression of reconstructed angle on reference angle are presented for each spatial location in Table 3. All slopes were approximately in unity, suggesting that proportional error was not present in the data. In all cases, 1.0 was contained in the interval defined by the 95% CI for the slope. The intercepts, however, were systematically lower than zero, indicating the presence of a constant error in the measurement system. The mean data (Tab. 1) indicate that the reconstructed angle was always smaller than the reference angle. This difference was also larger for the rotated position of the goniometer (location 3) with reconstructed values being from 0.4 to 1.4 degrees smaller than the reference angle, depending on the angle, in this spatial location. Moreover, the constant error was slightly greater at larger reference angles in the rotated condition.

Experiment 2

Method

Experiment 2 attempted to address the reliability of the WATSMART[TM] system in a more dynamic fashion. The questions asked were 1) How reliably can this system reconstruct known trajectories of joint and end-effector motion? and 2) Does reliability differ in different spatial locations within the measurement volume? Procedure. At the beginning of this experimental session, which occurred on a different day than experiment 1, the cameras were recalibrated. The average error in detecting the position of each cube IRED with respect to the cube's origin was 2.19 mm for this experiment. The robot arm was placed on the same box used in experiment 1. It was oriented in a plane parallel to that of the camera mounting bar. Two different movement trajectories of the robot arm were produced in this experiment. Ten repetitions of the first trajectory occurred at each of two spatial locations, mirroring those used in the first two blocks of experiment 1. The starting position had the "forearm" vertical and the "hand," or gripper, of the robot touching the box surface. Control of the arm's actuators by a computer caused the arm to reach upward toward a predefined spatial location, which was achieved by initial "humeral" extension, followed by simultaneous humeral flexion and forearm extension. The arm's trajectory was slightly different for the final block of 10 trials. In this block, the initial position and spatial location of the arm was the same as in the second block of the trials. Rotation of the robot arm's base, however, was added to the trajectory so that the arm rotated away from the cameras. Although motion of the shoulder and elbow joints was nearly the same as that during the previous two blocks, slight differences occurred because of the minimization function that calculates the individual joint trajectories needed to achieve a particular spatial location. Initial data processing (ie, conversion to three-dimensional data, smoothing) was the same as that in experiment 1. Data analysis. Intraclass correlation coefficients were calculated separately for each block of trials for 1) the shoulder joint angular trajectory; 2) the elbow joint angular trajectory; and 3) the x-coordinate (horizontal), y-coordinate (depth), and z-coordinate (vertical) trajectories of the arm's end point. Reliability of the y coordinate was assessed only for the last block of trials because movement was restricted to the x,z plane (ie, a plane parallel to the plane of the cameras) in the first two blocks.

Results Ten overlaid trajectories are shoulder and elbow angle and the x and z coordinates of the end-effector, respectively, for block 1 trials (note the differences in resolution). The y-coordinate trajectories for block 3 are presented in Figure 7. These trajectories involved movement away from the cameras, slight motion back toward the cameras, movement away again, and then return of the arm to its initial orientation along this coordinate. Trajectories for other trial blocks did not differ substantially and were less noisy. Visually, the trajectories appear rather consistent from trial to trial, with the greatest variability occurring at the movement end points (plateaus), especially for shoulder joint motion. The ICCs for overall joint and end-effector coordinate trajectories were remarkably high for each spatial location. The least reliable trajectory was that of the shoulder, although the lowest ICC was .97. These reliability coefficients, however, were calculated over 790 data samples, spanning rather large changes in the trajectory coordinates. It is clear that for at least some trajectories, movement plateaus were rather noisy. Additional ICCs, therefore, were calculated for trial segments having a length of no greater than 100 samples, including movement plateaus. These ICCs are presented for block 1 trials and for the y coordinate of block 3 trials in Table 5. Calculations for the same trajectories from the other trial blocks were either equal to or higher than those reported in this article and are available from the author. Overall, the ICCs are still rather high and generally acceptable, including those for portions of the trajectory spanning the movement plateaus. There are exceptions, however, most notably for movement plateaus reached following shoulder extension and those following achievement of 45 degrees of terminal IRED rotation away from the plane of the cameras.

General Discussion All motion analysis systems are subject to sources of error that depend on the technology used (eg, improper lighting, errors in digitizing body landmarks, camera lens distortion). It is important, therefore, that sources of error be identified and minimized as much as possible if interpretable data are to be obtained. A potential source of error that is especially problematic with infrared-based systems is light reflections from the floor, walls, or other objects (eg, highly reflective clothing) located in the vicinity of the IREDs. Because the camera's photo detector cannot separate simultaneous sources of light, reflections of emitted light from such surfaces detected by the camera diode will result in a "virtual" marker location, that is, an image somewhere between the actual position of the IRED and that produced by its reflections.[19] The magnitude of the error will depend on the intensity of the reflections because determination of the virtual coordinate is given by the center of intensity of all light sources. It is imperative, therefore, that highly reflective material be kept from the field of view. Nonetheless, reflections cannot be eliminated entirely and may vary in different parts of the field of view. Because functional movement analysis using such a system will necessarily involve the motion of IREDs through different regions of the camera's measurement volume, it is important to determine the degree of error present throughout that volume. The present experiments attempted to determine the extent to which such error affects the reliability and validity of motion detection using the WATSMART [TM] system under conditions where precautions had been taken to reduce reflections. The ICC was used to assess intertrial reliability of the WATSMART [TM] system because of its ability to estimate the extent to which multiple measures agree with one another, which is not true of more common correlation methods (eg, Pearson product-moment correlation).[2] The ICC, therefore, provides an estimate of the extent to which repeated measures of the same phenomenon are equal. In addition, 95% CIs were calculated to determine the actual amount of error to be expected in estimating known angles from reconstructed angles with this system. In general, results of both experiments 1 and 2 indicate that the position and movement trajectory of a limb in different parts of a measurement volume can be reconstructed reliably from infrared markers applied to the limb, assuming that adequate precaution has been taken to reduce reflected light in the measurement area. In experiment 1, ICCs calculated for 10 repeated trials at each spatial location revealed minimal deviation of the reconstructed angle from trial to trial, whether the goniometer remained parallel to the wall on which the cameras were mounted or was rotated 45 degrees away from that plane. All calculated ICCs were above .99. The variability of the reconstructed angles during each three-second epoch of angle stationarity, however, is more revealing, Although mean within-epoch variability increased with the size of the angle in blocks 1 and 2 and variability was higher overall in block 3 (when the goniometer was rotated 45 degrees), the 95% CI for angle fluctuations was in all cases within 0.3 to 0.4 degree. That is, the reconstructed angle did not vary by more than 0.5 degree (Tab. 2) in any part of the field of view. It is not clear whether the reconstructed angle's variability would have continued to increase in blocks 1 and 2, and, if so, to what extent, with further increases in the goniometer's angle beyond 100 degrees. The increase that did occur apparently was due to reflections from the moveable arm's IRED in that part of the field of view and not to the increased separation of the markers with increasing angle because no such trend was found when the goniometer was rotated 45 degrees away from the plane of the cameras. Data from experiment 1 also indicate that the WATSMART [TM] system is generally accurate in determining angular information, at least under static conditions. A small constant error of approximately 1 degree was present for all three trial blocks. The experimenter was very careful to position the moveable goniometer arm consistently from trial to trial. Although the contribution of experimenter error to this difference cannot be ruled out, such error would more likely be random than systematic. Systematic error could have been introduced by the goniometer itself. The goniometer used in this study was not tested against another source for accuracy. Goniometer error, however, would not account for the greater inaccuracies found at larger reference angles when the goniometer was rotated 45 degrees (Tab. 1, location 3). In this case, an additional factor--weaker signal power from the IREDs--came into play as well. The ICCs for selected portions of movement trajectories from experiment 2 varied, depending on the trajectory examined (Tab. 5). The ICCs for the elbow and the terminal IRED x- and z-coordinate trajectories were all greater than .9, even for movement plateaus. These plateaus are not equivalent to the stationary epochs in experiment 1 because movement was reciprocal and the robot arm never came to a complete stop. Movement extent was limited, however, and the direction of movement was reversing during these plateaus. Reliability of the shoulder trajectory was highest for flexion and reasonably high for the plateau following flexion and for shoulder extension. (Tab. 5). Reconstruction error, however, was unacceptably large for the movement plateau following shoulder extension (ICC = .20). This unreliability is most likely due to light reflections. The robot base contained three small actuators that controlled joint motion. Although the motor housings were covered with a low-reflection rubber material, movement of the covering away from the edge of the housing was noticed during some portions of the arm's trajectory. This movement occurred primarily at the extension plateau, and the base or shoulder IRED was optimally positioned to reflect off of this surface at this point. Nevertheless, this result dramatizes the potential effects of reflections on motion analysis with a system of this type. Finally, the terminal IRED's y-coordinate trajectory was examined for block 3 trials because in this block the arm rotated away from the cameras. As with shoulder movement, some portions of this trajectory were very unreliable (Tab. 5). The most unreliable trajectory segments occurred when the arm was maximally rotated away from the cameras and the direction of the motion was reversing (Fig. 7). In this case, ICCs were less than .6. One advantage of a motion analysis system of this type is that information about movement in the third dimension can be obtained. Results from both experiments suggest that information obtained about the third dimension can have acceptable reliability. For example, portions of the y-coordinate trajectory of the robot's terminal IRED (valley plateau; Tab. 5; Fig. 7) were relatively reliable. Intraclass correlation coefficients above .9 were obtained for most segments of the shoulder (except for the extension plateau) and for elbow angle trajectories of block 3, despite rotation of the IREDs (from which these angles were derived) away from the plane of the cameras. Nonetheless, reliability was generally lower for the end points of movement in this dimension and in some instances was unacceptably low, even under the carefully controlled conditions of this study. Moreover, information about this dimension may be slightly less accurate (Tab. 3). It should be emphasized, however, that these conclusions apply only to the experimental setup used in this study and when the subject or segment is rotated approximately 45 degrees away from the plane of the cameras. Many motor tasks of interest to physical therapists (eg, human locomotion) occur with significantly less rotation of the limb segments.[22] For example, the extent of hip rotation during gait is usually less than 15 degrees.[22] The addition of a third camera, mounted above the subject, would likely increase the reliability of such trajectory segments. Errors attributable to other sources cannot be eliminated completely as possible contributors to any unreliability found in WATSMART [TM] performance. The system uses a high-quality camera lens, which minimizes lens distortion as a source of error. The digitization of IRED positions occurs electronically and on-line, without user intervention. Digitization error, therefore, is likely to be minimal. The WATSMART [TM] system, however, provides for automatic camera gain changes whenever the detected signal intensity falls below a given criterion, which is most likely to occur when an IRED rotates or actually moves away from a camera, as was the case for the third block of trials of experiment 2. Although such gain changes are designed to improve signal strength and thus to reduce error in determining the IRED's actual position in the measurement volume, it is not clear what effect these changes have on the reliability of data capture. Interestingly, the lowest ICCs for the movement trajectories of experiment 2, where the robot's "shoulder" rotated away from the plane of the cameras, were obtained on the plateaus where the velocity and direction of movement were changing. How much, if any, of this unreliability is related to gain changes occuring at these points in the trajectory, as compared with increased reflections from surrounding surfaces, is unknown. Although this study did not attempt to assess the degree of error introduced into the measurements when no precautions are taken to reduce reflections, the low reliability obtained for the plateau of shoulder motion in experiment 2 (Tab. 5), which was most likely due to reflections off of the arm's actuators, suggests that careful attention to the experimental environment is required if useful information is to be obtained with infrared systems of this type. Painting or covering the walls and floor of the room with nonreflective paint greatly reduces this source of error. Unfortunately, this solution is not always practical in clinical settings. Nonetheless, relatively inexpensive rubber matting can be placed on the floor and draped temporarily from the ceiling in the recording area. Subjects should also wear dark blue or black textured clothing that has low reflectivity. The skin of the subject is probably also a source of reflections, although how much error results from skin reflections is unknown. One possible way to reduce this source of reflections is to surround each IRED with a doughnut of thin nonreflective rubber, applied with an adhesive, as long as the center hole is large enough to prevent jarring of the IRED during movement. Nonetheless, it is recommended that the extent of this source of error be formally assessed prior to data collection. For example, two IREDs could be mounted on each end of a rigid bar and attached to relevant body segments. The subject would then perform the experimental task. The distance between the IREDs could be calculated and the variability of this estimate examined. The estimate can also be compared with the actual distance, measured with a ruler, for systematic errors. In this way, an estimate of the influence of errors (eg, light reflections from skin and clothing) can be obtained that has direct relevance to the task being studied.[23]

Conclusions The results of these experiments provide an indication of the precision, accuracy, and limits of the WATSMART [TM] system. The extent to which unreliability will affect an investigator's conclusions will depend on the question being addressed and the resolution required to answer the question. Failure to take adequate precautions against errors such as those attributable to light reflections may lead to the acquisition of spurious or uninterpretable information about the movement task under investigation. Moreover, differences in the distance between the cameras and measurement area used in this study could lead to much better or much worse reliability. It is the responsibility of the investigator, therefore, to ensure that any such system is providing reliable and accurate results in that particular experimental situation. [Tabular Data 1 to 5 Omitted] [Figures 1 to 7 Omitted]

Acknowledgments I thank Michael Burrows of the Robotics and Microcomputer Instrumentation Laboratory, Georgia Institute of Technology, for providing and operating the robotic arm for experiment 2. I also thank Linda Fetters, PhD, PT, for her valuable comments and suggestions on an earlier version of this manuscript.

(*)Northern Digital Inc, 403 Albert St, Waterloo, Ontario, Canada N21 3V2. ([dagger])Selspot AB, Flojelbergsgatan 14, S-431 37 Molndal, Sweden. ([double dagger])Model LD242-3, Electro Sonic Inc, 1100 Gordon Baker Rd, Willodale, Ontario, Canada M2H 3B3. ([subsection])Therapeutic Equipment Corporation, 60 Page Rd, Clifton, NJ 07012. ([~~])Microblot Inc, 453-a Ravendale Dr, Mountain View, CA 94043.

References [1]Craik RL, Oatis CA: Gait assessment in the clinic: Issues and approaches. In Rothstein JM (ed): Measurement in Physical Therapy. New York, NY, Churchill Livingstone Inc, 1985, pp 169-205 [2]Rothstein JM: Measurement and clinical practice: Theory and application. In Rothstein JM (ed): Measurement in Physical Therapy. New York, NY, Churchill Livingstone Inc, 1985, pp 1-46 [3]Burdett RG, Borello-France D, Blatchly C, et al: Gait comparison of subjects with hemiplegia walking unbraced, with ankle-foot orthosis, and with Air-Stirrup[R] brace. Phys Ther 68:1197-1203, 1988 [4]Logan L, Byers-Hinkley K, Ciccone C: Anterior vs posterior walkers for children with cerebral palsy: A gait analysis study. Abstract. Phys Ther 68:868, 1988 [5]McGonagle L, Oatis CA: Structural and kinematic asymmetry of the lower extremities. Abstract. Phys Ther 68:804, 1988 [6]Nichols P, Slaton D: Temporal characteristics of gait in children with CP at free and fast walking speeds. Abstract. Phys Ther 68:867, 1988 [7]Winter DA: Concerning the scientific basis for the diagnosis of pathological gait and for rehabilitation protocols. Physiotherapy Canada 37:245-252, 1985 [8]Winter DA, Sienko SE: Biomechanics of below-knee amputee gait. J Biomech 21:361-367, 1988 [9]Daleiden SK, Fetters L: Kinematic analysis of reaching by active elders: Apreliminary report. Abstract. Phys Ther 68:810, 1988 [10]Fetters L, Fernandez L, Cerbak S: The relationship of proximal and distalcontrol of movement during development. Abstract. Phys Ther 68:839, 1988 [11]Heriza CB: Comparison of leg movements in preterm infants at term with healthy fullterm infants. Phys Ther 68:1687-1693, 1988 [12]Kluzik J, Fetters L, Coryell J: Short term effects of neurodevelopmental treatment (NDT) on reaching in children with spastic cerebral palsy. Abstract. Phys Ther 68:848, 1988 [13]VanSant AF: Age differences in movement patterns used by children to risefrom a supine position to erect stance. Phys Ther 68:1330-1338, 1988 [14]Winter DA: Biomechanics of Human Movement. New York, NY, John Wiley & Sons Inc, 1979, pp 16-25 [15]Phillips SJ, Jensen JL: Timing variables in the investigation of developmental questions: Methodological issues. In Clark J, Humphrey J (eds): Advances in Motor Development Research 1. New York, NY, AMS Press Inc, 1987, pp 179-195 [16]Gracovetsky S: The Spinal Engine. New York, NY, Springer-Verlag New York Inc, 1988 [17]Scholz JP, Kelso JAS: Intentional switching between patterns of bimanual coordination is dependent on the intrinsic dynamics of the pattern. Journal of Motor Behavior, to be published [18]Thelen E, Ulrich BD, Niles D: Bilateral coordination in human infants: Stepping on a split-belt treadmill. J Exp Psychol [Hum Percept] 13:405-410, 1987 [19]Samuelson B, Wangenheim M, Wos H: A device for three-dimensional registration of human movement. Ergonomics 30:1655-1670, 1987 [20]Winer BJ: Statistical Principles in Experimental Design, ed 2. New York, NY, McGraw-Hill Book Co, 1971, pp 283-289 [21]Westgard JO, Hunt MR: Use and interpretation of common statistical tests in method-comparison studies. Clin Chem 19:49-57, 1973 [22]Soderberg GL: Kinesiology: Application to Pathological Motion. Baltimore,MD, Williams & Wilkins, 1986, pp 317-321 [23]Haggard P, Wing AM: Assessing and reporting the accuracy of position measurements made with optical tracking systems. Journal of Motor Behavior, to be published

J Scholz, PhD, PT, is Assistant Professor, Program in Physical Therapy, School of Life and Health Sciences, University of Delaware, 009 McKinly Lab, Newark, DE 19716 (USA).