Reliability of performance measurements obtained using the Stability Testing and Rehabilitation Station.

As technology continues to develop, practitioners are faced with decisions regarding the applicability of devices in the evaluation and treatment of various patient populations. Even though these devices appear to be applicable to a wide variety of patients, little information is available in the literature or from the manufacturers relative to the validity or reliability of measurements obtained with equipment proposed for use in the clinic. One such recently marketed device is the Stability Testing and Rehabilitation Station (STARStation[R]) (*1) (Fig. 1). This product is similar to "balance boards," which combine exercise with the concept of the closed kinetic chain (ie, a series of linked segments that combine to provide variable movement patterns). Balance boards are used primarily for the purpose of improving joint range of motion (ROM), muscle tension-generating capabilities, and proprioception in patients with lower-extremity dysfunction.

A number of reports have addressed the clinical efficacy of the balance board for rehabilitation of lower-extremity disorders. Tropp nd Askling [1] recently reported that ankle disk training is beneficial for athletes with functional instability. Glick et al [2] and Karlsson et al [3] have proposed the use of a tilt board as part of their rehabilitation program. Exercises on a "sprain board" and on balance board for purposes of coordination and proprioceptive conditioning have also been proposed. [4] Soderberg et al [5] have reported the use of a balance board to determine that a nondisabled population did not differ significantly from a group with chronic ankle sprains in muscular control exerted during board rotation. None of these accounts addresses the reliability of measurements obtained during the imposed exercise tasks.

The STARStation[R] yields data on the force exerted by the board on a platform in addition to information about the velocity of board rotation. Exercises can be performed by rotating the board in either a clockwise or a counterclockwise direction while the board is elevated to various heights (levels) from the supporting surface.

Although this and similar devices are purported to provide the clinician with a more definitive diagnosis, leading to better and more specific rehabilitation and conditioning, no data have been reported on validity or reliability of the measurements obtained with such systems. We have designed a series of investigations to establish the clinical utility of this type of device. These investigations to establish the clinical utility of this type of device. These investigations include (1) an analysis of the instrument output characteristics; (2) an analysis of the reliability of force and velocity measurements for nondisabled subjects; and (3) assuming that the validity and reliability of the measurements are found to be satisfactory as a result of the preceding studies, a series of studies with patients. We believe that, without completing the first two studies, the variability associated with patient performance cannot appropriately be compared with data resulting from the performance of a sample of nondisabled subjects. Knowledge of the performance characteristics of individuals without pathology may serve to improve interpretation of the output characteristics in the evaluation of pathological conditions.

In order to proceed with the study reported in this article, a preliminary investigation was conducted to examine the performance of the instrument and error characteristics associated with the pressure-sensing elements of the STARStation[R] system (B Ballantyne, DA Nawoczenski, unpublished data, December 1989). Eight different positions on the platform were selected for evaluation. The investigators believed that the data from these locations, testing closure of individual switches embedded in the surface of the platform, were sufficient to represent the output of the device. A series of 10 known weights, accurate to within 1 g, were used to apply a load. At each of the eight positions, three calibration cycles were performed, beginning at 50% of the full-scale (FS) load (45,624 g). Weights were added in approximately 10% increments in ascending and descending order throughout each of the three cycles. Digital decimal values were available through the Utilities program of the proprietary software package supplied with the device and were recorded as each weight was added or removed from the platform. Values of the measured (in grams) and output digital values were used for determination of regression equations, hysteresis, and linearity and repeatability of the measurements.

Linearity was determined using the least-squares method and defined as the maximum deviation of the output of the measurement system from a specified straight line applied through the data points. The total system linearity (using all data points at all positions on the platform) was determined to be 1.11% of the FS load. Hysteresis, which is the maximum difference between upscale and downscale values within a cycle for the specified range of measurand values, resulted in an error of 0.55% of the FS load. Repeatability was the maximum deviation experienced at any measurand value during repeated trials. The worst case of repeatability was determined to be 0.72% of the FS load. These values indicate a small error when compared with the changes detected during the experimental testing procedures.

No specific "dynamic" testing was completed. The investigators felt that the loading described earlier produced an output similar to that produced when the balance board was rotated over the surface. We believe that the velocity data were accurate, because the board position is reflected on the display on the computer screen and because the software directly uses and reports this value both during and at the end of each trial. During this preliminary study, the investigators observed a marked fall in velocity at one point in the subjects' ROM as they performed on the device. Subsequently, a frame structured on a wheel was manufactured so that this device could fit on the stem on the STARStation[R] with the board removed. A series of revolutions was then completed at near-constant velocities throughout the 360-degree cycle, and values were read from the screen. Plots of the velocities are also available in the proprietary software package supplied with the device. Visual evaluation yielded consistent responses for the velocities tested during the multiple trials performed. As a result of both the formal and informal tests performed, we were confident of both the force and velocity output of the device.

These data are the first results obtained for analysis of the output characteristics of this device. To date, however, there are no published data as to whether within- and between-day measurements of subjects are similar, a finding that is of import to any clinician using these techniques in the assessment and management of lower-extremity disorders. Therefore, this study was undertaken to determine within- and between-day reliability of measurements of nondisabled subjects for the variables of force and velocity during clockwise rotation of the board when positioned at heights of 4.5 and 7.5 cm.

Method

Subjects

The subjects in this study were nondisabled men (N=24) betwen the ages of 20 and 35 years (X=25.0, SD=3.9). The subjects' height ranged from 177 to 191 cm (X=182, SD=10), and their weight ranged from 57 to 99 kg (X=78.0, SD=11.6). A brief orthopedic screening examination was administered to each subject to exclude subjects with a history of functional impairment or surgical repairs of the right lower limb and back.

Instrumentation

The STATStation[R] (Fig. 1) consists of an ellipsoidal board 63 cm in the anterior-posterior dimension and 71 cm in the medial-lateral dimension. The board is mounted on a shaft that raises the center of the board from 4.5 cm to 10.5 cm in 1.5-cm increments. A change in ankle ROM necessary to complete board rotation. This shaft projects through the center of a platform instrumented with strain gauges optimally aligned in a Wheatstone-bridge configuration on four cantilevered arms supporting the platform. The compressive force created by a load applied to the platform as the board rotates through 360 degrees alters the resistance of the bridge and produces a change in voltage output, which is in turn converted to a digital value as the board makes contact with the platform. Additionally, 120 switches are mounted at 3-degree intervals in a circular pattern under the rubber-matting cover of the platform. Together with the strain guages, the switches provide information about the load, the position of the board on the platform, and the rate of change of board position.

Output was transmitted to a computer comprising an 80M AST hard disk and a 20-286/87 processor and math co-processor. (*2) An NEC Multisync 2A color monitor (*3) was used as the display device. The STARStation[R] software, which provides a menu-driven program capable of real-time animation or replay analysis of the various data forms available, was operated in accordance with the manufacturer's instructions. The values provided by the software package represented the mean force and velocity over the 120 switches. Values for individual switches, representing selected locations on the platform, were available through the ASCII file.

Procedure

Prior to data collection, the system was calibrated in accordance with the manufacturer's instructions. Subjects, dressed in everyday clothes and footwear, wear informed of the protocol of the study. After consent forms were signed, one of the investigators (DJM) demonstrated the board-rotation technique. All subjects were instructed to rotate the board at a smooth, self-selected velocity in order to simulate clinical conditions. For all trials, subjects were instructed to maintain the right knee in slight flexion, to minimize the motion of the hips and trunk, and to allow the ankle to provide the majority of the motion. The protocol also required that the backs of the hands be maintained, at shoulder level, in contact with the vertical bars attached to the frame of the device (Fig. 1). Positioning of the hands in this manner minimized the influence of forces exerted by the arms during the activity. During all testing, one investigator (JEP), who was seated behind the subjects, provided verbal cues to maintain the appropriate posture and to minimize hip and trunk movement. After any questions were answered, the subjects positioned their right foot on the template incorporated into the board and then practiced board rotation at the two board heights until they felt competent to perform the test. A 5-minute rest interval was provided to avoid fatigue and to allow for answering further questions before data collection.

For the data-collection phase, the board was positioned randomly at either level 1 (lowest board position=4.5 cm height) or level 3 (intermediate board position=7.5-cm height). Ten revolutions of the board in the clockwise direction were completed in each of six trials. A 45-second rest was provided between each trial. When the trials associated with the first randomly ordered board level were completed, a 5-minute rest was provided and the same protocol was completed for the remaining board level. If, during any trial, a deviation from the original foot and board position occurred, the trial was repeated. When the six trials at each of the two board levels were completed, the data were stored in the computer and the subject was scheduled for a second session conducted 48 hours after the first session. The second session consisted of the same protocol, but with the board levels administered in reverse order.

Data Analysis

In addition to the data available from the automated system, we chose to select data from 8 of the 120 switches to determine repeatability of force and reliability measurements at every 45 degrees of board rotation. The independent variables were level (1 and 3), day (1 and 2), trial (1-6), and switch position (1-8). The dependent variables were force and velocity. Means and standard deviations for force and velocity over the six trials were obtained from the proprietary software package supplied by the manufacturer. Force and velocity values for each of the eight switch data points were available through an ASCII file subroutine.

Descriptive statistics were calculated for the dependent variables for the six trials for each levelxday combination. Any effect over the repeated trials was examined using repeated-measures analyses of variance (ANOVAs). Linear or quadratic trends were evaluated with polynomial contrasts in order to examine the stabilization in force and velocity values over the trials. Helmert's contrasts [6] were used to determine whether and where a plateau for force and velocity occurred.

For each board level and switch position, nested ANOVAs (completely randomized design) for within-subject values for the independent variable day and within-day values for the independent variable trial were analyzed. Reliability analysis consisted of examining the variance components of the intraclass correlation coefficients (ICC[1,1]) [7] and expected deviation values for each subject. The expected deviation is the value, in actual units of measure, that may occur between any 2 days or any two trials for the respective analysis.

All analyses were performed using the Statistical Analysis System (SAS) software program (*4) available through the main computer center at The University of Iowa (Iowa City, Iowa). The level of significance adopted for all analyses was .05.

Results

Representative data plots (Figs. 2-4) show intertrial, between-day, and between-level variability for every 45 degrees of board rotation. Descriptive statistics for force on the platform surface and velocity of rotation of the board for each day are shown in Figure 5. The results of the polynomial contrasts were significant for linear and quadratic trends for force at level 1 on day 1 and level 3 on both days 1 and 2, indicating that changes occurred over the trials during a session (Tab. 1). When subjected to further analysis using Helmert's contrasts, a plateau was evident after the second and third trials.

Intraclass correlation coefficients for within-subject reliability for the independent variable day ranged from .72 to .81 (Tab. 2), and within-day ICC values for the independent variable trial ranged from .46 to .81 (Tab. 3). Because of the potential learning effect suggested by the polynomial contrasts for force values over repeated trials, ICCs were examined using all six trials of a session as well as the last three trials of a session. Little change was evident in ICC values when including only the last three trials in the analysis. An ICC difference of .11 for velocity at level 3 was the maximum difference observed when comparing the six- versus three-trial force and velocity analyses. The ICCs for velocity, ranging from .70 to .81, were considered good by the investigators, whereas those for force, ranging from .46 to .79, were considered only fair. For the individually selected switch positions, the ICCs showed a wide range of values for both force and velocity at each level for both three and six trials (Tabs. 4, 5).

Discussion

Data illustrated consistent patterns of response for force at selected board rotations across the trials (Fig. 2). Average force and velocity patterns between days also showed little variability (Fig. 3). Representative data for between-level comparison showed greater variability, as would be expected when comparing performance at different board heights (Fig. 4). Because no other reports for performance on the balance board have appeared in the literature, these data represent the initial information for nondisabled subjects performing this activity at a self-selected velocity.

The mean values for force and velocity were calculated from the readings of all 120 switches. Between-day comparisons for mean force and velocity values demonstrated higher values for both force and velocity on day 2 than on day 1 (Fig. 5). The increase in mean values from day 1 to day 2 in a population without pathology may indicate an improvement in an individual's ability to perform or learn the test rather than an improvement in strength or mobility over a 48-hour period. These findings suggest that caution should be exercised when interpreting the results between test sessions. Adding another test session to the protocol of this study may have provided information regarding improved performance on the STARStation[R] that was based on learning the activity.

One of the limitations of the STARStation[R] that may affect interpretation of the force data is the variability associated with an individual's position on the platform during the test. Although the force values represent a certain percentage of body weight applied to the switches on the force platform, variations in the placement of the subject's hands, trunk, and non-weight-bearing lower extremity will alter the percentage of body weight applied to the individual switches on the platform. Consequently, an increase in force values may be due to an alteration in arm, trunk, or opposite lower-extremity positioning on the device, rather an individual's ability to bear more weight through the foot during certain, or all, positions of board rotation. This observation, which we attempted to control in this study, emphasizes the importance of maintaining consistency of positioning on the platform during each test session.

The potential effect of learning that could occur between the first and last trials of a test session was evaluated in order to determine whether data from all trials should be included in the reliability analysis. The polynomial

Table 1. Probability Polynomial Contrasts to Examine Learning Effect Over Six Trials at Each Board Level

          Force                Velocity
          Linear   Quadratic   Linear     Quadratic
Level 1
  Day 1   .01 (a)  .84         .05 (a)    .26
  Day 2  .06       .11         .18        .43
Level 3
  Day 1  .02 (a)   .01 (a)     .12        .44
  Day 2   .003 (a) .02 (a)     .07        .84
  (a) Significant at P<.05.

Table 2. Intraclass Correlation Coefficients (ICCs) and Expected Deviations (a) for Within-Subject Reliability for Independent Variable Day and Dependent Variables Force and Velocity

                       6 Trials              3 Trials
                       Force      Velocity   Force      Velocity
Level 1
  ICC                   .74         .73       .79         .77
  Expected deviation   1.52       11.34      1.41       11.02
Level 3
  ICC                   .77         .81       .72         .80
  Expected deviation   1.52       7.88       1.16        4.01
  (a) Expected deviation=value in actual units that reflects
between-day differences.

contrasts indicated a linear trend for force and velocity at level 1 for day 1, and both linear and quadratic trends for force at level 3 on days 1 and 2 (Tab. 1). When Helmert's contrasts were used to indicate when intertrial variability tended to plateau, the majority of plateauing was shown to occur after trials 2 and 3. If an individual requires a few performance trials before results stabilize during the testing session, then it would follow from these results that a more accurate representation of the subject's responses may be made by the inclusion of only the last three trials in the analysis. The ICCs showed little difference when using six trials versus three trials for analysis, with the possible exception of velocity at level 3 (r=.70-.81, Tab. 3). These findings support the manufacturer's recommendation of including six trials for analysis during one test session. Although no significant differences were found among the ICCs, however, the linear and quadratic trends that were revealed, particularly at level 3 for force, suggest a need for adequate practice prior to the testing session at this higher board level. Clinicians can use these findings as a guide in selecting the number of trials that should be completed before performance can be considered an accurate representation.

The ICCs for within-subject reliability for the independent variable day

Table 3. Intraclass Correlation Coefficients (ICCs) and Expected Deviations (a) for Within-Day Reliability for Independent Variable Day and Dependent Variables Force and Velocity

                       6 Trials              3 Trials
                       Force      Velocity   Force      Velocity
Level 1
  ICC                   .55        .74        .57        .77
  Expected deviation   1.35       6.60       1.22         6.09
Level 3
  ICC                   .54        .70        .46        .81
  Expected deviation   1.41       5.11       1.16         4.01
  (a) Expected deviation=value in actual units that reflects
intetrial differences.

showed slightly higher values at board level 3 than at board level 1, especially for velocity (Tab. 2). One explanation for the higher coefficients at level 3 may be related to the observation of the investigators that subjects appeared to have less difficulty in generating lower velocities at this board level. Perhaps gravity plays an assistive role. There was also less expected deviation between days for velocity at level 3 (ie, 7.88 rpm) than at level 1 (ie, 11.34 rpm) (Tab. 2).

Intraclass correlation coefficients for within-day reliability for the independent variable trial were markedly less for force than those for velocity at both levels 1 and 3 (Tab. 3). This finding may mean that minor shifts in body or foot position relative to the supporting axis altered the force perceived by the switches. The ICCs for within-day reliability for the independent variable velocity were comparable to those for within-subject reliability for the independent variable day and the dependent variable velocity. These findings lend support to the manufacturer's suggestion of incorporating mean force and velocity values (of six trials) rather than attempting to examine the repeatability of responses from one trial to another within a test session.

Because a clinician may be more interested in force and velocity patterns at certain points in an individual's ROM, eight switches, representing every 45 degrees of board rotation, were selected for further analysis. The ICCs for within-subject reliability for the independent variable day and for within-day reliability for the independent variable trial (Tabs. 4 and 5, respectively) demonstrated a large range of values across individual switch positions, with a greater range of values for velocity. Trials within a session generally resulted in poor to marginal ICCs at the selected switch positions and in fair ICCs between days. As demonstrated by the range of ICCs for the independent variable day (Tab. 4), some ICC values at selected switches were greater than those reported

Table 4. Intraclas Correlation Coefficients (ICCs) for Within-Subject Reliability for Independent Variable Day and Dependent Variables Force and Velocity for Eight Switch Positions.

              6 Trials              3 Trials
              Force      Velocity   Force      Velocity
Level 1
  ICC range   .59-.95    .42-.98    .51-.82    .54-.91
  ICC X       .75        .65        .72        .70
Level 3
  ICC range   .57-.83    .13-.89    .46-.85    -.15-.84
  ICC X       .74        .67        .69        .69

Table 5. Intraclass Correlation Coefficients (ICCs) for Within-Day Reliability for Independent Variable Trials and Dependent Variables Force and Velocity for Eight Switch Positions

              6 Trials              3 Trials
              Force      Velocity   Force      Velocity
Level 1
  ICC range   .03-.55    .02-.42    .27-.54    .03-.46
  ICC X       .35        .26        .45        .29
Level 3
  ICC range   .12-.47    .06-.50    .11-.71    -.11-.50
  ICC X       .37        .28        .48        .27

for all switches (Tab. 2). We believe that the variability of ICCs at the individual switches may be due, in part, to the ellipsoidal shape of the board on the platform, resulting in different force and velocity patterns throughout the 360 degrees of board rotation. Further evaluations of force and velocity patterns at specific board positions is necessary if these patterns are to be used by clinicians to extrapolate information regarding certain pathologies of the lower limb.

The results of this study provided information about subjects' performance at two selected board levels, with rotation of the board in the clockwise direction and the foot in a relatively neutral position of rotation. Numerous options are available for testing and rehabilitation using this platform, and recommended protocols need to be developed and tested for reliability if this device is to be considered a useful rehabilitation tool.

Conclusions

The results of this study provide an indication of the between-day and intertrial reliability of the same measurements obtained with the STARStation [R] at the two board levels tested on nondisabled subjects, using both mean force and velocity values and individual switch-position values. Reliability values for mean force and velocity measurements were higher between days than when compared between trials within a session. When individual switch positions were analyzed, wide ranges of ICC values were seen for both force and velocity. Additional reliability analyses performed on patient groups are needed before this device can be considered to yield reliable clinical measurements. If one of the purposes of this device is to identify pathologies based on certain force and velocity patterns at different board locations on the platform, then further evaluation is needed to identify those locations as well as the patterns of response specific to those locations.

(*1) At the time this study was conducted, the STARSTation [R] was a product of Camp International (PO Box 89, Jackson, MI 49204) and the device was distributed by Camp International to selected sites, including The University of Iowa, for testing Camp International has subsequently announced that STARStation [R] has not been identified.

(*2) AST Research Inc, 2121 Alton Ave, Irvine, CA 92714.

(*3) NEC Home Electronics (USA) Inc, NEC Corp, Tokyo, Japan.

References

[1] Tropp H, Askling C. Effects of ankle disc training on muscular strength and postural control. Clin Biomech. 1988;3:88-91.

[2] Glick JM, Gordon RB, Nishimoto D. The prevention and treatment of ankle injuries. Am J Sports Med. 1976;4:136-141.

[3] Karlsson J, Bergsten T, Lansinger O, et al. Reconstruction of the lateral ligaments of the ankel for chronic ankle lateral instability. J Bone Joint Surg [Am]. 1988; 70:581-588.

[4] Smith RW, Reischel SF. Treatment of ankle sprains in young athletes. AM J Sports Med. 1986;14:465-471.

[5] Soderberg GL, Cook TM, Rider SC, Stephenitch BL. Electromyographic activity of selected leg musculature in subjects with normal and chronically sprained ankles performing on a BAPS[R] Board. Phys Ther. 1991;71:514-522.

[6] Finn JD. A General Model for Multivariate Analysis. New York, NY: Holt, Rinehart & Winston Inc; 1984:232, 249, 255.

[7] Shrout PE, Fleiss JL. Intraclass correlation coefficients: uses in assessing rater reliability. Psychol Bull. 1979;86:420-428.

DA Nawoczenski, MEd, PT, is a doctoral student in the Physical Therapy Graduate Program, College of Medicine, The University of Iowa, 2600 Steindler Bldg, Iowa City, IA 52242 (USA). Address all correspondence to Ms Nawoczenski.

WB Sharp, MPT, PT, is Staff Physical Therapist, Sioux Center Community Hospital, Sioux Center, IA 51250.

DJ Maiers, MPT, PT, is Staff Physical Therapist, Christ Hospital, 4440 W 95th St, Oak Lawn, IL 60453.

JE Patterson, MPT, PT, is Staff Physical Therapist, National Jewish Center for Immunology and Respiratory Medicine, 1400 Jackson St, Denver, CO 80206.

GL Soderberg, PhD, PT, FAPTA, is Professor and Director, Physical Therapy Graduate Program, College of Medicine, The University of Iowa.