Issues in measuring change in motor function in children with cerebral palsy: a special communication.

Accurate detection of change in function is the essential purpose of an evaluative or outcome measure.1 Recent interest in techniques to measure functional change has led to a number of approaches to establishing the responsiveness of evaluative clinical measures.1-5 This trend is particularly true in areas of experience or function (eg, pain, health status, motor behavior) not easily measured by a physiological or other standardized assessment. The purpose of this special communication is to review various ideas about how to measure functional change validly. Although most of the concepts presented in this article have been developed and implemented in adult medicine, the ideas are generalizable to any clinical population. We make specific reference to the challenges posed in evaluating change in motor function in children with cerebral palsy (CP). We also critically examine measures of motor behavior used as outcomes in published clinical trials of physical therapy for CP.

Cerebral palsy is defined as a group of non-progressive disorders occurring in young children in which disease of the brain causes impairment of a motor function. The impairment of motor function ma),, be the result of paresis, involuntary movement or incoordination, but motor disorders which are transient or are the result of progressive disease of the brain or attributable to abnormalities of the spinal cord are excluded.[6](pl)

Most clinicians would agree with Bax's observation that CP is a "not necessarily unchanging" disorder,[7] despite the apparently static nature of the underlying central nervous system impairment. Change in the clinical picture over time is probably a function of CNS development, the evolution of motor patterns at reflex and volitional levels, motor learning, and therapy. As a result, the clinical manifestations and motor behaviors of individuals with CP often change and evolve, particularly in children during the first several years of life. Although the neurophysiological basis of CP has been the subject of much study,[8-11] at a clinical level descriptive or evaluative measures of observed motor function can be undertaken, regardless of one's theory of the pathophysiology of CP, using the observed behavior as the "raw material" for the measurement instrument.

General Issues in Measurement

Kirshner and Guyatt have provided a methodological framework for assessing health measures. They point out that these measures may be used for one purpose or for various purposes. A discriminative index distinguishes between individuals with and without a particular characteristic or function (eg, the Peabody Developmental Motor Scales[13] are used to categorize children via percentile rank scores, standard scores, or age-equivalent scores). A predictive index classifies people into categories based on what is believed or expected will be their future status (eg, the Bleck scale predicts ambulation at age 7 years on the basis of a preschool child's postural and tonic reflex activity). An evaluative index is used to measure the magnitude of change in function over time or after treatment (eg, the measure used by Wright and Nicholson[15] assessed several motor-related clinical functions before and after a period of physical therapy). Construction and validation of a measure must be predicated upon its ultimate purpose, and methodological rigor is essential from the initial conception of the measure to the completion of its development and testing. Note that measures generally are developed and validated specifically to fulfill one of the functions described above. A measure cannot automatically be used for a purpose other than the one for which it was created or applied to a population dissimilar to that on which it was developed and validated. It should be self-evident that Bleck's[14] predictive measure will not suffice as a discriminative instrument. Bleck's explicit purpose was to develop a test to predict future ambulation based on the presence or absence of seven postural and reflex items, and the scale has excellent measurement properties to accomplish this task (sensitivity = .98; specificity = .84). Seven items, however, would scarcely be adequate to distinguish or discriminate a population of children with CP into anything other than very crude and probably clinically meaningless categories, because the scope and subtlety of motor behavior in this population require much finer distinctions than can be achieved with such a small subsample of the domain of motor function. Similarly, as an evaluative measure, the Bleck scale is limited both in the number of items potentially responsive to change in motor function with time and treatment and in the narrow range of response options (present-absent) by which to annotate improved motor function. The Peabody Developmental Motor Scales were designed specifically to enable clinicians and others to determine the relative developmental skill level of a child, identify skills that were not completely developed or not in the child's repertoire, and then plan an instructional program to develop those skills.[13](pl)

The large number of items and the three-level response options make it suitable for this purpose, and validation studies have established its usefulness.13 This measure might be valuable as either an evaluative or a predictive motor function measure, but to our knowledge it has not yet been validated for either of these functions.

Creation and Validation of an Evaluative Measure

To assess the clinical motor function outcome of interventions for children with CP, an evaluative measure with specific structural characteristics is required. Items in the measure must be selected on the basis of both clinical relevance and potential responsiveness to change. For example, a therapist might reasonably anticipate a change in duration of independent standing following a physical therapy program, but he or she would be less likely to expect a change in the asymmetric tonic neck reflex (ATNR). It would be appropriate, therefore, to include duration of standing but not ATNR in an evaluative measure, despite the clinical importance of both to overall motor behavior. If the ATNR is indeed not responsive to change, its presence in the evaluative measure serves no purpose and can add error to the instrument. The measure must be feasible to use, with clearly described response options and explicit scaling. The measure must be reliable; that is, it must give consistent responses or scores when used repeatedly with "stable" subjects. The measure must be valid, that is, responsive to real change and stable in the absence of change. The essential component in validating an evaluative instrument is to establish its responsiveness to change.

Any evaluative motor assessment instrument used to measure outcome in children with CP must be capable of detecting change in individual children. Responsiveness can be enhanced by one or both of two strategies. First, by increasing the number of responsive items and eliminating those that are unresponsive, the therapist increases the chance that change will be detected if it is actually occurring. Second, by offering several response options for partial as well as complete accomplishment of each item, there is an increased potential to identify finer gradations of change than is possible with a simple present-absent response choice.

The reliability and validity of a responsive measure should be established before a measure is used. Reliability refers to the ability of a measure to give consistent responses on repeated assessments in the absence of change in the characteristic being evaluated.[16] Potential sources of variability that contribute to unreliability of an index include those attributable to observers, subjects, environments, timing, and the index itself. These potential sources of variability should be quantified and controlled as fully as possible.

Validity refers to the appropriateness (truth) of inferences made when interpreting a measure, a score, or the results of a test.[17] Several components of validity have been described, including criterion, content, and construct validity.[18] Readers are referred to standard textbooks on research methodology such as Principles of Medical Statistics: A Short Textbook of Medical Statistics by Bradford Hill[19] for a more complete discussion of these measurement concepts.

In summary, an evaluative measure must contain relevant items and must be applicable to the population for whom it is developed, feasible to use, reliable, and valid for that purpose. The most essential feature of an evaluative measure, however, is it: responsiveness to clinically important change over time. Ideally, the therapist should "titrate" the measure by using it to assess change in individuals known to be susceptible to treatments of proven efficacy. Guyatt et al, for example, were able to evaluate the responsiveness of a new functional status measure in patients with chronic lung disease before and after a therapeutic intervention thought likely to produce improvement and to compare that amount of (real) change with the relatively much smaller variability in stable subjects tested repeatedly with the same measure but not receiving any treatment.[1]

As Meenan et al point out, however, when a relatively new outcome measure is used in a situation where the effectiveness of the treatment is unknown, it is difficult to determine whether the findings, be they positive or negative, are due primarily to the properties of the treatment )r the properties of the measurement approach.[2](pl351)

This uncertainty pervades most areas of developmental medicine, where few, if any, treatments are proven to be efficacious and hence are not useful as "proven interventions" to be used with newly developed measures. Furthermore, none of the available measures of motor function have been validated for their capacity to detect change. We will return to this point in our review of measures used in published clinical trials of physical therapy interventions for children with CP.

Assessing Responsiveness in Absence of a "Gold Standard"

If change cannot be measured because of the absence of measures of known responsiveness, and if no current interventions are of proven value, how can any measure-new or established-be shown to be responsive to clinically important change? Several approaches have been applied, mainly in the area of adult medicine, and these methods, with appropriate adaptations, are relevant to the field of developmental pediatrics.

Meenan et al correlated functional score changes on a newly applied clinical measure with changes in physiological function to evaluate the responsiveness of the new measure.2 Even though they had the advantage of using a treatment of known efficacy (eg, gold therapy for rheumatoid arthritis), the concurrent changes on the functional, physiological, and clinical indexes provided evidence for the responsiveness of the clinical health status measure.

Deyo and Inui suggested that, in the absence of a "gold standard" for determining what constitutes a clinically important change, the independent, concurrent agreement about change by patient and physician can serve as a basis for defining the presence and magnitude of functionally important changes.3 Using this clinically defined change criterion, they then assessed and contrasted several clinical methods of rating change among people for whom there was agreement that change had actually occurred.

Devo and Centor argued that there was an analogy between assessing the responsiveness of a functional scale and determining the discriminating properties of a diagnostic test using sensitivity and specificity.[4] They created receiver operating characteristic (ROC) curves on which they plotted on the ordinate the true positive rate (sensitivity) of the measure to detect clinically important change" against the false positive rate (1 - specificity) on the abscissa. Visually, the ROC curve is plotted from the lower left to the upper right of a standard X-Y plot. The curve rises more or less steeply above the 45-degree diagonal, curving to the right and gradually flattening out at the extreme right end. A perfect test, therefore, would be represented by a vertical line along the ordinate, with no horizontal component, whereas a useless test would have a response "curve" along the diagonal at 45 degrees.20

Using data from several related measures of change in patients with low back pain, with an improvement criterion of "return to full activity," Deyo and Centor were able to demonstrate the value of short, disease-specific indexes over longer, more general functional scales.[4] They suggest, as Kirshner and Guyatt[12] have stated, that the removal of unresponsive items from an evaluative measure appears to increase its responsiveness. A second important value of the ROC-curve approach is the capacity to choose a cutoff point of clinical change that best discriminates improved from unimproved patients.

Guyatt et al recommend a new statistically based index of responsiveness for clinical measures designed to detect change., in their approach, issues pertinent to establishing responsiveness are 1) the variability of change scores in subjects considered stable and 2) the amount of change considered to be clinically important. The first issue is related to the test-retest reliability of the measure, the second issue concerns the validity, of the measure, specifically its ability to detect clinically important change in function when that change is believed to have taken place. Statistically, responsiveness is derived by the ratio of the clinically important difference (delta) to the variability in stable subjects (square root of twice the mean square error). Their article contains data-based evidence to support this technique, and they illustrate the utility of this method for sample-size calculation.

The notion of construct validity[l8] posits that one puts forward a priori hypotheses about how a measure should perform if it is doing the task for which it was created. In the absence of either a criterion measure against which to compare a new measure or a treatment of known efficacy by which to titrate the new scale, this indirect approach attempts to aggregate data in support of the measure. Russell et al have relied on construct validity to establish the responsiveness to change over time of a measure of gross motor function in children with CP.[21] They hypothesized 1) that if the measure were responsive, it should be capable of detecting change in children judged by their parents and therapists to have changed; 2) that children with "mild" CP should show more change per unit of time than age-matched children with "severe" CP; 3) that younger children with CP should change more than older children in the same time period; and 4) that children recovering from acute head injury should show more change than children with CP. Each of these hypotheses has clinical face validity, and each provided a basis for assessing the responsiveness of our new measure (ie, for testing the performance of the measure as an evaluative instrument).

Results of a validation study to establish the responsiveness of the Gross Motor Function Measure (GMFM) have been published elsewhere.21 Briefly, the GMFM assesses 88 items of motor function in five areas: 1) lying and rolling; 2) crawling and kneeling; 3) sitting; 4) standing; and 5) walking, running, and jumping. The GMFM items were derived from several published sources (notably reports by Hoskins and Squires(22) and Steel and Spasoff(23)) and organized by body position to allow for ease of testing. All items would usually be accomplished by a 5-year-old with normal motor abilities. Items are scored using a four-point scale (O = cannot initiate task, 1 = initiates task, 2 = partially completes task, and 3 = completes task), each task being explicitly described in a manual.' in addition to a total score, scores can be calculated for each of the five dimensions described above. What makes this measure different from other (discriminative) measures such as the Peabody Developmental Motor Scales'3 and the Bayley Motor Scale24 is the validation process by which its responsiveness to change was established. The scale was applied to 140 children of various ages with a wide range of neuromotor disabilities and to 30 healthy preschool-aged children on two occasions separated by a three- to six-month interval. Concurrently, judgments about motor function were made independently by the disabled children's parents and physical therapists. After the second administration of the GMFM, parents and therapists independently made structured judgments about the direction and amount of change in motor function of each child. These judgments of change correlated strongly with GMFM change scores, in support of the hypotheses outlined previously, thus establishing the responsiveness of the GMFM.

In a variation of construct validity, Lipsey recommends the use of criterion groups whose function can be measured and compared to define a "clinically relevant" effect size.5 He suggests that the groups chosen should differ at least as much as the minimal therapeutic effect expected by a intervention. For example, if a therapeutic program were designed to produce one grade year of improvement, criterion groups might be children in grades 3 and 4, with the achievement difference between them representing the effect size to be sought by the therapeutic intervention. The choice of criterion groups-and a criterion difference so defined-enables any measured effects of an intervention to be put into perspective. Lipsey's arguments are complex and largely statistical, but the ideas are clearly presented.5 To determine what respondents considered a "clinically important" effect size, Russell and colleagues asked both the therapists and parents independently to make judgments about the importance of any changes in function they reported at the time of reassessment of each child with the GMFM.(21) This approach allowed the authors to detect a gradient of perceived importance with gradations of measured functional change. However, only with continued use of the measure, under varied conditions, will it be possible to refine our understanding of what actually constitutes clinical importance.

In summary, a variety of clinical, psychometric, and statistical approaches for the assessment of responsiveness of clinical measures has been formulated. Combinations of these techniques will generally be required to establish that the measure is responsive to clinically important change in function. Like validity, responsiveness is an abstract property established incrementally over time by the accumulation of evidence about the performance of a measure.

Review of Measures Used in outcome Studies

We critically reviewed the measures used in published controlled clinical trials of physical therapy interventions for children with CP to highlight the limitations of currently available instruments. Review articles by Parette and Hourcade(25) and Ottenbacher et al(26( and a survey of the recent developmental pediatric literature provided source papers for this analysis. Studies such as those of Goodman et al (27) and Piper et al(28), evaluating therapy for at-risk infants were excluded because those studies used discriminative measures to assess the effectiveness of interventions. (They did not measure the same functions before and after an intervention, for which an evaluative measure is required, but instead undertook postintervention assessments of groups of infants is an effort to discover whether treatment and control groups differed on motor function outcomes of interest [a discriminative task].)

The assessment scheme used by Wright and Nicholson(15) was largely based on Holt's work.(29) Three dimensions were studied: motor function, range of motion of selected joints, and presence or absence of primary automatic reflexes. The items appear applicable to children with CP up to 6 years of age. They presented no justification concerning the selection of this group of items, apart from the assumption that they "were those common to the normal developmental pattern for locomotion."(15)(p147) Although a scoring system is presented, the authors did not state whether they used a manual to define the items to provide clarity for an observer required to make such judgments.

No evidence is given of the interrater or intrarater reliability. The validity of the functional component of Wright and Nicholson's(15) measure and its value as a discriminative instrument are suggested by the low initial mean scores of the youngest infants (reflecting both young age and severity of CP). They presented no evidence, however, that the measure was capable of detecting change in function, ROM, or reflex patterns in children with CP. The lack of differential change in the treatment group may reflect either lack of efficacy of the treatment or a failure of the measure to detect clinically important change. In the absence of proper validation of the measure for responsiveness to change, one simply cannot tell.

The Bayley Motor Scale24 was used by Carlsen(3O) and Palmer et al(31) to evaluate change in motor function. The Bayley Motor Scale has been extensively developed for use as a discriminative measure with infants and very young children, and Bayley herself points out that the scales "have limited value as predictors of later abilities."(24)(p4) More significantly, she writes that because of rapidly changing development in infancy, "measurement procedures must differ in significant ways for infants only a few months apart in age."(24)(p5) No responsiveness data are presented to establish the Bayley Motor Scale as an evaluative instrument.

Carlsen(3O) also used the Denver Developmental Screening Test, which was created as a discriminative screening instrument.32 Motor scale items should be a highly selective subsample of motor functions, designed to differentiate children into those judged to be developing normally, those whose motor function is suspect, and those with impaired motor development. Although the measure is developmental in structure, there is no evidence that it is responsive to clinically important changes in motor function in children with CP. The measure offers neither detailed discriminative nor evaluative functions and has not been demonstrated to be useful as an outcome measure in clinical trials.

Scherzer et al used a measure especially constructed for their study but not described in their report.33 Their measure indicates change in relation to position, tone, movement and overall "function" and was used to evaluate motor outcome. No measurement properties of the instrument are provided to attest to its applicability, feasibility of use, reliability, or validity. In particular, there is no discussion of evidence that the measure is capable of detecting change in motor function if that change actually exists. Overall judgments are made of "definite improvement" or "no improvement," but the basis of these judgments is not explained. On the of the evidence presented, one cannot be confident that the measure can detect change in motor function. et al(34) used measures of reflexes,(35) a gross motor assessment compiled by one of the authors from the developmental literature, and goniometric measurements of selected joints. The reflex and goniometric measurements were made according to published methods for testing and recording, but no evidence is presented that the motor function items were applied in a standardized manner. The investigators pretested their measures for test-retest reliability over a two-week period (r = 37-.99). They then selected only those items from the assessment instruments with acceptable evidence of reliability (r = >.70). Choosing only highly reliable (ie, stable) items, however, may decrease the chance of demonstrating the measure's responsiveness.

Sommerfeld et al(34) presented no discussion or evidence that any of these measures is responsive to change over time. As with the previous studies, the absence of change in any of the treatment (or control) groups may reflect the lack of responsiveness of the measures.

Bertoti measured stride length, stride width, foot angle, and footprint clarity using an ink-print ambulation pattern on paper.36 She was able to evaluate changes in those variables in children treated with short-leg casting, but it is instructive to quote from the discussion of her article:

I believe the measure chosen in this study yielded valuable information on ambulation changes comparing casted and uncasted children with CP. Subjectively, the therapists involved in this study observed additional proximal changes exhibited by the casted children that could not be documented by this measure four emphasis]. These changes were judged to be significant clinically and included improvements in trunk control and mobility of the pelvis in positions such as side lying, sitting, and quadruped. Trunk control appeared to be improved.... A follow-up study using this ambulation measure and an additional measure studying the pelvis or trunk probably would yield valuable information.(36)(p528)

Bertoti's subjective impressions about change illustrate precisely the type of clinically important information for which we have need of well-designed, validated, responsive measures.36 In addition to the physical therapy techniques aimed at the facilitation of normal postural control and movement patterns,3-39 a variety of mechanical and surgical treatments have been used in the management of CP. Orthoses, orthopedic surgery, and occasionally drugs to promote muscle relaxation are used in conjunction with therapy. Since the early 1970s, neurosurgical procedures (ie, chronic cerebellar stimulation,(40,41) spinal cord stimulation,(42) selective dorsal rhizotomy(43)) have been introduced in further attempts to control disordered movement in patients with CP. These invasive neuro-augmentation and ablative procedures are not widely available, in part because, like virtually all interventions, their value has yet to be clearly established and measured in carefully designed studies.44 Until we have valid measures responsive to clinically important change, we will be severely compromised in attempting to determine whether any of our interventions do more good than harm.

Conclusions

Developmental medicine is a new field, and until recently, relatively little attention has been paid to the scientific basis of current practice. For example, there have been few rigorous clinical trials evaluating the efficacy of the many clinical interventions for children with CP, and those available are often beset by methodological difficulties.45 One of the principal reasons for this situation is the absence of appropriately validated measures that are responsive to clinically important functional change. Such instruments are essential to evaluate the effects of interventions designed to improve the function of physically disabled children. The science of measurement is well established in the fields of education and psychology, and it is becoming recognized in clinical medicine. We have tried to show in this article that techniques are available for the development of measures responsive to clinically important functional change. As clinicians and investigators, we have an obligation to be as creative about the design and validation of measures for assessing change as we are in undertaking clinical trials of the treatments we use. The challenge is great, and the rewards will be a harvest of measures useful for both clinical and investigative activities.

Acknowledgments

We gratefully acknowledge the advice and ideas of Dr Gordon H Guyatt and Dr Charles Goldsmith during the development of this article. Joanne Miles prepared multiple drafts of this paper with efficiency and good humor.

References

1 Guyatt GH, Walter SD, Norman G; Measuring change over time: Assessing the usefulness of evaluative instruments. j Chronic Dis 40:171-180, 1987

2 Meenan RF, Anderson , JJ, Kazis LE, et al: Outcome assessment in clinical trials: Evidence for the sensitivity of a health status measure. Arthritis Rheum 27:1344-1352, 1984

3 Devo RA, Inui TS: Toward clinical applications of health status measures: Sensitivity of scales to clinically important changes. Health Serv Res 19:275-289, 1984

4 Devo RA, Centor RM: Assessing the responsiveness of functional scales to clinical change: An analogy to diagnostic test performance. J Chronic Dis 39:897-906, 1986

5 Lipsey MW: A scheme for assessing measurement sensitivity in program evaluation and other applied research. Psychol Bull 94:152165, 1983

6 Ingram TTS: A historical review of the definition and classification of the cerebral palsies. in Stanley F, Alberman E (eds): The Epidemiology of the Cerebral Palsies. Philadelphia, PA, J B Lippincott Co, 1984, pp 1-11

7 Bax MCO: Terminology and classification of cerebral palsy. Dev Med Child Neurol 6:295297, 1964

8 Bishop B: Spasticity: Its physiology and management-Part I. Neurophysiology of spasticity: Classical concepts. Phys Ther 57:371-376, 1977

9 Bishop B: Spasticity: Its physiology and management-Part II. Neurophysiology of spasticity: Current concepts. Phys Ther 57:377-384, 1477

10 Bishop B: Spasticity: Its physiology and management-Part III. Identifying and assessing the mechanisms underlying spasticity. Phys Ther 57:385-395, 1977

11 Harris SR: Neuropathology in cerebral palsy. Physical and Occupational Therapy in Pediatrics 1(4):45-52, 1981

12 Kirshner B, Guyatt GH: A methodologic framework for assessing health indices. J Chronic Dis 38:27-36, 1985

13 Folio RM, Fewell RF: Peabody Developmental Motor Scales. Allen, TX, DLM Teaching Resources, 1983

14 Bleck EE: Locomotor prognosis in cerebral palsy. Dev Med Child Neurol 17:18-25, 1975

15 Wright T, Nicholson J: Physiotherapy for the spastic child: An evaluation. Dev Med Child Neurol 15:146-163, 1973

16 Mitchell SK; Interobserver agreement, reliability, and generalizability of data collected in observational studies. Psychol Bull 86:376-390, 1979

17 Cronbach LJ: Test validation. In Thorndike RL (ed): Educational Measurement, ed 2. Washington, DC, American Council on Education, 1971, pp 221-237

18 Kaplan RM, Bush JW, Berry CC: Health status: Types of validity and index of well-being. Health Serv Res 11:478-507, 1976

19 Hill AB: Principles of Medical Statistics: A Short Textbook of Medical Statistics. London, England, Hodder & Stoughton Ltd, 1977

20 Metz CE: Basic principles of ROC analysis. Semin Nucl Med 8:283-298, 1978

21 Russell Dj, Rosenbaum PL, Cadman DT, et al: The Gross Motor Function Measure: Validating the responsiveness of an evaluative measure. Dev Med Child Neurol 31:341-352, 1989

22 Hoskins TA, Squires JE: Developmental assessment: A test for gross motor and reflex development. Phys Ther 53:117 126, 1973

23 Steel KO, Spasoff RA: The Testing of a Motor Control Assessment for Physically Handicapped Children. Toronto, Ontario, Canada, Ministry of Health, Final Report of Grant 677, Project No. 00643, 1986

24 Bayley N: Bayley Scales of Infant Development. New York, NY, The Psychological Corp, 1969

25 Parette HP, Hourcade JJ: A review of therapeutic intervention research on gross and fine motor progress in young children with cerebral palsy. Am J Occup Ther 38:462-468, 1984

26 Ottenbacher KJ, Biocca Z, DeCremer G, et al: Quantitative analysis of the effectiveness of pediatric therapy: Emphasis on the neurodevelopmental treatment approach. Phys Ther 66:1095-1101, 1986

27 Goodman M, Rothberg A, HoustonMcMillan JE, et al: Effect of early neurodevelopmental therapy in normal and at-risk survivors of neonatal intensive care. Lancet, December 14, 1985, pp 1327-1330

28 Piper MC, Kunos VI, Willis DM, et al: Early physical therapy effects on the high-risk infant: A randomized controlled trial. Pediatrics 78:216-224, 1986

29 Holt KS: Assessment of Cerebral Palsy, London, England, Lloyd-Luke (Medical Books) Ltd, 1965

30 Carlsen PN: Comparison of two occupational therapy approaches for treating the young cerebral-palsied child. Am J Occup Ther 29:267-272, 1975

31 Palmer FB, Shapiro BK, Wachtel RC, et al: The effects of physical therapy on cerebral palsy. N Engl J Med 318:803-808, 1988

32 Frankenburg WK, Dobbs JB, Fandel AW: Denver Developmental Screening Test. Denver, CO, University of Colorado Medical Center, 1970

33 Scherzer AL, Mike V, Ilson J: Physical therapy as a determinant of change in the cerebral palsied infant. Pediatrics 58:47-51, 1976

34 Sommerfeld D, Fraser BA, Hensinger RN, et al: Evaluation of physical therapy service for severely mentally impaired students with cerebral palsy. Phys Ther 61:338-344, 1981

35 Wilson J: A developmental reflex test. In Vulpe SG (ed): Vulpe Assessment Battery. Toronto, Ontario, Canada, National Institute on Mental Retardation, 1977, pp 342-358

36 Bertoti DB: Effect of short leg casting on ambulation in children with cerebral palsy. Phys Ther 66:1522-1529, 1986

37 Bobath K, Bobath B: The facilitation of normal postural reactions and movements in the treatment of cerebral palsy. Physiotherapy 50:246-262, 1964

38 Jones RB: The Vojta method of treating cerebral palsy. Physiotherapy 61:112-113, 1975

39 Levitt S: Treatment of Cerebral Palsy and Motor Delay. Oxford, England, Blackwell Scientific Publications Ltd, 1977

40 Cooper IS, Crighel E, Amin I: Clinical and physiological effects of stimulation of the paleocerebellum in humans. J Am Geriatr Soc 21:40-43, 1973

41 Davis R, Barolat-Romana G, Engle H: Chronic cerebellar stimulation for cerebral palsy: A five-year study. Acta Neurochir [Suppl] (Wien) 30:317-332, 1980

42 Waltz JM: Computerized percutaneous multi-level spinal cord stimulation in motor disorders. Appl Neurophysiol 45:73-92, 1982

43 Peacock WJ, Arens LJ, Berman B: Cerebral palsy spasticity: Selective posterior rhizotomy. Pediatr Neurosci 13:61-66, 1982

44 Penn RD: Chronic cerebellar stimulation for cerebral palsy: A review. Neurosurgery 10:116-121, 1982

45 Bax MCO: Aims and outcomes of therapy for the cerebral palsied child. Dev Med Child Neurol 28:695-696, 1986