Interrater reliability of lumbar accessory motion mobility testing.

Key Words: Accessory motion, Low back pain, Manual therapy, Mobility.

Accessory motion is defined as movement at a joint that cannot be performed voluntarily but can be performed passively by an external force.[1] A goal of accessory motion testing is to evaluate aspects of tissue compliance, including amount of available range of motion (ROM), degree of resistance to motion, change in resistance relative to ROM, and end-feel of the motion.[1] In addition, spinal accessory motion testing is used to assist in detecting a symptomatic level, the behavior of pain relative to ROM, and the presence of spasm. Accessory motion testing is commonly included in the physical therapy assessment of the spine.

The information gained from accessory motion testing is used in conjunction with other findings to establish a diagnosis and plan treatment. As a diagnostic tool, accessory motion evaluation is performed to establish relative segmental stiffness (hypermobility and hypomobility) as a cause of low back pain (LBP). Posterior-anterior (P-A) accessory motion is a commonly used accessory motion in which pressure is exerted in a posterior to anterior direction over the spinous process of the selected segmental level.[1] In the lumbar spine, the pressure is generally applied with the patient in a prone position, using a point just distal to the examiner's pisiform.[1] As a test for determining relative motion and reproduction of symptoms at a particular spinal segment, accessory motion testing must be reliable and valid. To date, there is only one study published in peer-reviewed journals that has addressed the issue of reliability of accessory motion testing in subjects with LBP.[2]

Maher and Adams[2] evaluated the interrater reliability of P-A accessory motion testing in 90 patients with LBP. Three pairs of physical therapists each evaluated the P-A accessory motion at the L-1 to L-5 levels in a subset of 30 subjects. Each therapist held a graduate diploma in manipulative therapy in Australia and had a minimum of 5 years of experience in manipulative physical therapy. The therapist treating a given subject completed the usual patient evaluation, including the assessment of P-A accessory motion. The treating therapist marked the spinous processes tested, and the second therapist in the pair, blind to the first tester's results, then evaluated the P-A accessory motion. The order of testing was alternated between the therapists. Stiffness was recorded on an 11-point scale, from -5 (markedly reduced stiffness) to 5 (markedly increased stiffness), with 0 being normal stiffness. Pain produced during the assessment of each level was recorded on an 11-point scale, from 0 (no pain) to 10 (intense pain). Intraclass correlation coefficients (ICCs) for agreement between raters on stiffness were reported as .03 to .37, depending on the level tested. The 95% confidence intervals were .18 to .53, indicating that the population ICC value would be in this range. Intraclass correlation coefficients for judgments of pain were .67 to .73, with 95% confidence intervals of .55 to .81. The authors concluded that the interrater reliability of judgments of P-A stiffness in patients with LBP is not acceptable and that there appears to be better reliability when pain reproduction at a given level is the goal of the test.[2]

Maher and Adams' study[2] was the first study published in a peer-reviewed journal to document the reliability of P-A accessory motion testing in a patient population. The population was defined as subjects with nonspecific mechanical LBP and excluded postsurgical patients and patients with neurological signs or definable pathology such as spinal stenosis or other serious pathology. The evaluators were well qualified and appeared to be familiar with the scale. Repeated testing may introduce systematic change in subjects, such as increased pain or mobility. The effect of order of testing on the results was not addressed. The comparison of results between a physical therapist knowledgeable of the subject's clinical signs and symptoms (treating therapist) and an evaluator blind to the clinical signs and symptoms (second rater) may have introduced bias. Although the order of the evaluators was alternated each time, the effect of this knowledge of clinical signs and symptoms by one rater on the results is not clear.

Matyas and Bach[3] reported on reliability of lumbar accessory motion testing in a review of unpublished graduate studies supervised by the authors. The report on three studies that examined the reliability of spinal accessory motion for determining tissue compliance and segmental ROM is relevant. In one study, the interrater reliability in detecting the onset of resistance (R1) and end of resistance (R2) during a P-A spinal accessory motion was evaluated. The levels tested were C-2, C-6 T-2, T-10, L-2, and L-4. Average reliability coefficients for all levels combined as .3 for R1 and .28 for R2. The best coefficients were obtained at L-2 for R1 (.64) and L-5 for R2 (.58). In a second study, the test-retest reliability of detecting the range at which there is an onset of resistance (RI) during P-A accessory motion at C-2, T-4, and L-5 was evaluated. The reliability averaged across joints and physical therapists was .46 during one session but dropped to .09 between test sessions conducted 1 week apart. In a third study, the interrater and test-retest reliability of P-A accessory motion testing at the L-4 to T-11 levels was evaluated. The test-retest reliability, reported as a percentage of agreement, was 31%, slightly better than chance alone. The interrater reliability was reported to be similarly low at 25.7%. In each of these studies reported by Matyas and Bach,[3] the subjects had no previous back pain and were symptom-free at the time of the study. This may have the effect of restricting the range of variability and reducing the reliability coefficient. In addition, the results cannot be generalized to a clinical population. Finally, the methodology and statistical analysis of each study were not detailed, and the studies were not published in a peer-reviewed journal. For these reasons, therapists should draw conclusions from these studies with caution.

Gonnella et al[4] and Keating et al[5] reported on the reliability of passive intervertebral motion testing in the spine. Passive physiological intervertebral motion (PPIM) testing is the evaluation of the motion available at a spinal segmental level during the application of a passive physiological motion (flexion, extension, side flexion, or rotation). While the therapist passively moves the spine in a given direction, the spinal segment is palpated and a judgment is made with respect to the amount of available ROM and end-feel at that segment.[1] Although the technique of evaluating PPWM is distinct from that of evaluating accessory motion, the literature is relevant, as the goal in each technique is to evaluate segmental mobility.

Gonnelia et al[4] evaluated the test-retest and interrater reliability of PPIVMs in five female subjects without symptoms. Five physical therapists evaluated the intervertebral motion on a seven-point ordinal scale, from ankylosed (0) to unstable (6), during forward-bending, side-bending, and rotation maneuvers. To evaluate test-retest reliability, therapists evaluated the motion under a blinded condition on two occasions 13 days apart. Therapists also performed the evaluation under a "normal" condition to simulate a more realistic clinical condition. The analysis of reliability consisted of reports of mean and standard deviations for each spinal level, therapist, session, maneuver, and blind versus normal condition. The authors concluded that intratherapist reliability was good for all maneuvers and in both normal and blind conditions. The intertherapist reliability, however, was reported to be poor.[4] It is difficult to draw conclusions, as descriptive statistics are presented and the specific training and clinical experience of the therapists was not well described. In addition, the study was performed on subjects without symptoms and cannot be generalized to a patient population.

Keating et al[5] examined the interrater reliability of measurements obtained by three chiropractors on eight spinal examination techniques. One of these techniques was the evaluation of "passive motion palpation." The chiropractors evaluated 21 subjects with symptoms and 25 subjects without symptoms. The passive motion palpation technique, referenced to Cassidy and Potter,[6] was performed with each subject in a sitting position. The examiner moved the subject passively to the end of the subject's active ROM, then stressed the segment to the end of passive ROM using thumb pressure against a spinous process. The presence of a joint "fixation" is determined when no motion is felt and a hard end-feel is noted on stressing the segment passively. Testing was performed on levels from T11-12 to L5-S1, inclusively, and spinous processes were marked on each subject by one evaluator prior to testing. Kappa coefficients were presented for the presence or absence of a fixation, ranging from .00 to .23.[5] The subjects with symptoms in this study were not well defined, and it is difficult to judge the degree of similarity in technique and experience between chiropractic and physical therapy.

Several studies have examined the usefulness of passive testing for identifying a symptomatic segmental level. Jull et al[7] reported on the accuracy of one therapist's passive evaluation (accessory motions and PPIVMs) in detecting an involved segmental level in a group of 20 patients with neck pain. Segmental level was determined by the physical therapist based on abnormal end-feel, abnormal quality of resistance to motion, and reproduction of the subject's pain. The therapist's judgment was compared with the level at which a nerve block performed under fluoroscope completely relieved symptoms. The authors reported that the therapist could determine the symptomatic level in all cases.[7] The identification of an abnormal level did not include the evaluation of accessory ROM, making direct application to the validity of accessory motion testing for determining segmental mobility difficult. Evaluations were performed by one therapist, making the generalizability of the results questionable. The study suggests that passive motion testing in the cervical spine may be a valid method of determining a symptomatic cervical level, but the study did not address the issue of reliability of mobility testing.

Potter and Rothstein[8] examined the reliability of 13 sacroiliac tests, including 2 tests in which a positive test was considered to be reproduction of pain by passive stressing of the joint. Seventeen patients with unilateral buttock pain, with or without leg pain or paresthesia, were evaluated by eight physical therapists specializing in orthopedics. The authors then evaluated the intertester reliability of the tests. The supine iliac gapping test and the side-lying iliac compression test showed good intertester reliability, with 900/o and 70% agreement, respectively.[8] Reliability coefficients were not calculated. These results are similar to those of Jull et al,[7] suggesting that passive testing may be reliable when pain production is the criterion measured.

In summary, there is one published report documenting the interrater reliability of spinal accessory motion mobility testing in a patient population. The available literature indicates that therapists' judgments on pain reproduction during accessory motion testing are reliable both within and between raters.[2,7] The available literature on the reliability of PPIVMs for determining segmental mobility indicates poor intertester reliability and acceptable intratester reliability, although it is difficult to generalize the findings of Gonnella et al[4] to a patient population and the findings of Keating et al[5] to the physical therapy evaluation of accessory motion. The review of unpublished studies by Matyas and Bach3 on the reliability of accessory motion testing does not provide sufficient methodological detail to critically review and interpret the results. Because accessory motion testing is used clinically as a diagnostic test for determining the probable cause of LBP, the intertester reliability of accessory motion testing is critical. The purpose of this study was to determine the interrater reliability of accessory motion mobility testing in a group of subjects with LBP.

Method

Subjects

Subjects were 18 patients referred to an outpatient physical therapy setting in a university teaching hospital. Subjects were English-speaking patients referred with nonspecific mechanical LBP.[9] This definition excludes the following pathologies: postsurgical, serious pathology and neurological signs, including tendon reflex abnormality and myotome weakness. Patients unable to lie prone for longer than 10 minutes at a time were excluded. Informed consent was obtained.

Subjects ranged in age from 23 to 62 years ([bar]X= 36, SD = 10.4). There was an equal distribution of male and female subjects, which was not planned a priori. All subjects stated that they had LBP in the previous week, and seven subjects reported numbness or tingling in the lower extremity in the past week. All subjects reported episodes of back pain prior to the current episode for which treatment was sought. The mean duration of LBP since original onset ranged from 2 months to 10 years ([bar]X=3.2 years, SD=3.4).

Procedure

Six orthopedic physical therapists evaluated the lumbar P-A accessory motion mobility of each subject. All of the therapists met the following criteria considered to be representative of an average experienced clinician working in an outpatient setting: (1) working in an outpatient orthopedic setting for at least 3 years, (2) performing one or more detailed lumbar assessments per week, and (3) postgraduate training in manual therapy (Canadian [V.sub.2]/[V.sub.3] level).[10] The [V.sub.2]/[V.sub.3] course levels require a minimum of 40 hours of postgraduate theoretical and practical course work in spinal manual therapy. Three of the therapists met the minimum criteria, in addition to having advanced postgraduate specialization training in manual therapy, and were certified Canadian Orthopaedic Manipulative Physiotherapists.[10] The demographics of the physical therapists are presented in Table 1.

[TABULAR DATA 1 OMITTED]

Subjects were positioned prone and were draped to conceal their identity. The therapists evaluated the mobility of the P-A accessory motion of six levels, from L-1 to the sacral base, inclusive. The motion was recorded on a nine-point scale developed for the study.[11] The scale ranged from 1 (severe excess motion) to 9 (no motion) (Fig. 1). The central point on the scale (5) was normal. The reproduction of pain at any level was noted. Each therapist used the scale in practice on a minimum of five patients prior to the study and attended a 2-hour training session on interpretation of the scale. There was no attempt to standardize the angle of application of the P-A accessory motions in order to simulate clinical practice. For each level, the therapist made a judgment as to whether there were mobility and pain findings that were of significance to treat. In cases where there were judged to be mobility findings of significance to treat, the therapist noted whether this intervention would be aimed at treating excess or reduced motion. This final question was asked to maximize the clinical relevance of the study. We felt that, regardless of agreement on the mobility scale, that agreement on the decision making to treat a given level was a critical issue. Disagreement on the level of the lumbar spine being assessed could confound the results. In order to evaluate agreement on the level of the lumbar spine being assessed, one spinous process of each subject was arbitrarily marked by an investigator. As part of the accessory motion testing, each therapist identified the level of the mark and assessed the mobility of that level as one of the routine lumbar segments. Thus, the degree to which therapists agreed on the level that was marked and the mobility at that standardized level could be determined.

A Latin square design was used to balance for subjects, order of testing, and raters.[12] The 18 subjects were divided into three groups of 6 subjects. A table was devised for each subject group (Fig. 2). Subjects were randomly assigned a number to determine placement in the table. Raters were randomly assigned to a letter designation, which directed the order of testing of subjects. The procedure was repeated for each group of 6 subjects. The Latin square design allowed the analyses of order effect, such as systematic change of pain or mobility produced by repeated testing. All evaluations were performed on 1 day. Although there was no limit on the number of repetitions for each level, a maximum of 5 minutes was allotted for each assessment. A 5-minute break was given to subjects and therapist after each second evaluation. Therapists were given a half-hour break between each group of 6 subjects.

Data Analysis

To determine whether systematic change in subjects occurred during testing, a three-factor analysis of variance (subject, rater, order of testing) was performed to examine order effect.[13] generalized Kappa statistic that accommodates more than two ratings per subject was calculated to determine agreement on levels, motion, and decision to treat. For this analysis, the seriousness of all magnitudes of disagreement was treated as being equal.[14] The second analysis used the ICC (11) as the expression of reliability. The magnitude of this statistic is equal to the value of weighted Kappa when quadratic weights are assigned to the disagreement cells.[10] The a priori minimal levels for identifying the marked level and the degree of mobility were set at .90 and .70, respectively. Finally, the standard error of measurement (SEM) was calculated to express the magnitude of intertherapist disagreement in the same units as the original measurement. In order to find the extent to which two raters disagree, we used a calculation based on the reliability change. index described by Ottenbacher et al.[15] Specifically, the SEM is multiplied [square root of]2 (acknowledging a comparison of two measurements), and this value is then multiplied by the tabled z value for the confidence level of interest. We present a confidence level of 95% (tabled z=1.96).

Results

A summary of the results is provided in Table 2. No systematic difference due to order of testing was observed for identifying the marked level or for grading mobility (P--.39 for both analyses). The magnitude of the Kappa statistic for determining agreement on identification of the marked level was below the a priori level of declared significance. Our analysis of the error estimate indicates that 95% of the time two raters would be within 1.4 levels of each other when making a judgment as to a particular segmental level. Due to the poor interrater reliability the identification of the marked segment, ratings for mobility and decision to treat are reported for the marked segment only. Intraclass correlation coefficients for motion testing of the marked segment were below the a priori level of declared significance. The error estimate indicates that 95% of the time, two raters would agree to within 3.3 levels on the mobility scale. The analysis to determine agreement on the therapists' decision to treat the marked level yielded a Kappa value of .09. The results of the three physical therapists with formal manual therapy qualifications were similar to those of the other physical therapists.

[TABULAR DATA 2 OMITTED]

Discussion

There was poor interrater agreement on determination of the segmental level of a marked spinous process for subjects positioned prone lying. This finding has implications for clinical and educational practice. We demonstrated that 95% of the time physical therapists will be able to agree only to within 1.4 segmental levels in the lumbar spine. Consideration of these results must be made when assigning specific levels as the origin of a patient's dysfunction and in evaluating students on their ability to identify segmental levels.

Due to the poor interrater reliability found on determining the segmental level, conclusions regarding agreement on mobility testing can be based on the marked segment only. There was poor interrater agreement on mobility judgments and the decision whether to treat the marked segment. These results indicate that P-A accessory motion testing is not an appropriate test when the goal is to determine segmental mobility. These results are similar to those reported for interruter reliability by Maher and Adams[2] and Matyas and Bach3 and for PPIVM testing by Gonnella et al.[4] In a review article on accessory motion measurement, Riddle[16] addressed the relationship of the available literature on passive mobility testing in the extremity joints to the literature on passive mobility testing of spinal joints. The interrater reliability of passive motion testing for determining ligamentous stability at the knee has been reported to be poor.[17] it could be expected that interrater reliability of measures of smaller, less accessible spinal joints is similarity poor.[16] Further work is needed to determine whether procedures or conditions exist that will enhance the reliability of measures of spinal segmental mobility.

Accessory motion testing is also used to determine a symptomatic spinal level. The work of Jull et al[7] and Potter and Rothstein[8] suggests that passive testing for determining symptom reproduction is reliable between physical therapists In the cervical and lumbar spine, respectively. In light of our findings, however, assigning a segmental level to the symptoms elicited may be inaccurate between raters.

Repeated segmental mobility testing may alter patients' mobility and pain levels. Accordingly, studies that do not accommodate for a potential order effect may underestimate the true magnitude of interrater reliability or inadvertently attribute a systematic variation within patients to rater change. The Latin square design is balanced such that the unique effect of patient, rater, and order of testing can be assessed independently. The analysis of the order effect allows investigators to determine whether a systematic difference is occurring that cannot be attributed to interrater difference. In our study, the lack of significance of the findings with respect to testing order indicates that subjects remained stable during the testing period. Accordingly, the low levels of agreement cannot be attributed to a systematic change within the subjects over time, such as might be produced by increasing pain, spasm, or mobility with repeated testing.

Analysis of the ICCs for the three specialist physical therapists revealed that there was no difference in results in this group versus the orthopedic physical therapists without this additional training. This finding suggests that there may not be a direct relationship between raters' experience and interrater reliability of the evaluation of motion by P-A accessory testing. This finding thus indicates that factors other than experience, such as test standardization, appear to contribute more to the error associated with judging segmental motion.

Several issues in standardizing the approach to determining a segmental level and testing spinal segmental mobility require consideration. The precise technique of testing for P-A accessory motion was not standardized in our study or by Maher and Adams.[2] Protocols for testing, including specific angle and method of force application, patient position, and standard rating scales may improve reliability. Therapists, for example, were required to judge segmental level with the subject in a prone position, which may not be the position that maximizes accuracy in judging the level. An alternative method, such as positioning the patient in side lying to allow the therapist to simultaneously palpate and passively flex and extend the spine through the hips, may improve the identification of interspinous spaces. A weakness of our study is that the therapists were restricted to using the method of palpation with the subject in a prone position to determine segmental level.

The scale used to rate motion affects the reliability of judgments. Scale reliability increases as the number of points available and used by raters increases.[11] Thus, a 3-point scale (with points of hypomobility, normal mobility, and hypermobility) will be inherently less reliable. Collapsing our data in this manner led to lower levels of reliability. The scales used in our study and by Maher and Adams [2] and Gonnella et al[4] differ in number of points on the scale and in the orientation of the scale anchors. Maher and Adams,[2] for example, used an 11-point scale, whereas our raters reported mobility on a 9-point scale. Gonnella et modified a 7-point scale originally defined by Kaltenborn. Clinicians responded with plus and minuses, effectively expanding the scale to 13 points.4 The orientation of the scale anchors reported is not consistent. In our study and that of Maher and Adams,[2] higher numbers on the scale represented greater stiffness, whereas Gonnella et al[4] used higher numbers to represent greater mobility. The use of negative numbers on the scale may be confusing to raters. The effect of the lack of consistency in scaling is to make comparisons among studies difficult. Lack of a single scale that is widely used clinically also means that raters in these studies, although experienced in evaluating accessory motion, do not have the same depth of experience with the scale. A related issue is that normal values for a given segmental level have not been established. This problem was highlighted by the finding of Gonnella et al[4] that, in a group of young asymptomatic subjects, ratings on the mobility scale were hypomobile on average. In summary, the effect of scaling and establishment of "normal" is critical to future reliability studies. We suggest that an 11-point numerical rating scale be adopted for studies of segmental mobility, where 0 represents no motion, 5 represents normal motion, and 10 represents severe excessive motion.

Physical therapists were asked to make decisions regarding the mobility of a spinal segment with no additional clinical data or other objective information. Conclusions must, therefore, be made in this context. Additional data, such as history, lumbar ROM, and PPIVMs, may increase the agreement on accessory motion mobility and decisions to treat specific segmental levels in a clinical setting. All therapists involved stated that they would consistently use additional clinical data to determine diagnosis and treatment, including subjective data, overall lumbar ROM, PPIVM testing, and segmental stress testing.

Conclusions

There was poor interrater reliability in the identification of a marked lumbar spinal level in our subjects. There is poor interrater reliability in the determination available passive ROM in the absence of corroborating clinical data. Future study should address the issue of testing protocol, including scaling for mobility ratings. In determining segmental mobility, reliability of measurements of P-A accessory ROM may be enhanced by combining the results with other subjective and objective clinical data, such as overall physiological ROM, level of symptom reproduction, and ROM on corroborating tests such as PPIVMs. Future study should include evaluation of combinations of tests for determining segmental mobility and standardization of test protocols and scaling.

Acknowledgments

We acknowledge Elizabeth Black, Director of Physiotherapy at St Joseph's Hospital, for supporting this project. We also thank the staff of St Joseph's Hospital and the physical therapists and patients who participated in this study.

Figure 1. Mobility rating scale.

[Figure 1 ILLUSTRATION OMITTED]

Figure 2. Order of Testing

[Figure 2. ILLUSTRATION OMITTED]

References

[1] Maitland GD. Vertebral Manipulation. 5th ed. Boston, Mass: Butterworth-Hienemann; 1986:2, 74-78, 282-283. [2] Maher C, Adams R. Reliability of pain and stiffness assessments in clinical manual lumbar spine examination. Phys Ther. 1994;74:801-811. [3] Matyas TA, Bach TM. The reliability of selected techniques in clinical arthrometrics. Australian Journal of Physiotherapy. 1985;31: 175-199. [4] Gonnella C, Paris SV, Kutner M. Reliability in evaluating passive interverteral motion. Phys Ther. 1982;62:436-444. [5] Keating JC, Bergmann TF, Jacobs GE, et al. Interexaminer reliability of eight evaluative dimensions of lumber segmental abnormality. J Manipulative Physiol Ther. 1989;12:93-97, [6] Cassidy JD, Potter GE. Motion examination of the lumbar spine. J Manipulative Physiol Ther. 1979;2:151-158. [7] Jull G, Bogduk N, Marsland A. The accuracy of manual diagnosis for cervical zygapophysial joint pain syndromes. Med J Aust. 1988;148:233-236. [8] Potter NA, Rothstein JM. Intertester reliability for selected clinical tests of the sacrodiac joint. Phys Ther. 1989;69:268-275. [9] Nachemson A. Newest knowledge of lowback pain: a critical look. Clin Orthop. 1992; 279:8-20. [10] Orthopaedic Division Education Syllabus. Toronto, Ontario, Canada: Canadian Physiotherapy Association; 1991. [11] Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. New York, NY: Oxford University Press Inc; 1989. [12] Armitage P. Statistical Methods in Medical Research. New York, NY: John Wiley & Sons Inc; 1973:239-245. [13] Winer BJ. Statistical Principles in Experimental Design. 2nd ed. New York, NY: McGraw-Hill Inc; 1971;693-700. [14] Fleiss JL. Statistical Methods for Rates and Propotiions. New York, NY: John Wiley Sons Inc; 1981:225-232. [15] Ottenbacher KJ, Johnson MB, Hojem M. The significance of clinical change and clinical change of significance. Am J Occup Ther. 1988;42:156-162, [16] Riddle DL. Measurement of accessory motion: critical issues and related concepts. Phys Ther. 1992;72:865-874. [17] McClure PW, Rothstein JM, Riddle DL. Intertester reliability of clinical judgments of medial knee ligament integrity. Phys Ther. 1989;69:268-275.

J Binkley, PT, MCISc, COMP, is an educational and clinical consultant and Assistant Professor, Department of Physical Therapy, North Georgia College, Barnes Hall, Dahlonega, GA 30597 (USA). Address all correspondence to Ms Binkley.

PW Stratford, MSc, PT, is Assistant Professor, School of Occupational Therapy and Physiotherapy, McMaster University, 1200 Main St W, Hamilton, Ontario, Canada L8N 3Z5.

C Gill, PT, is Orthopaedic Physical Therapist, St Joseph's Hospital, Hamilton, Ontario, Canada.

Ethics approval was obtained from St Joseph's Hospital.

This article was submitted July 1, 1994, and was accepted May 8, 1995.