Development of a quality-of-movement measure for children with cerebral palsy.

Research in the field of developmental pediatrics has lagged behind investigation in other areas of clinical medicine. We believe this lag has been due to the numerous conceptual, methodological, and practical difficulties that require significant ingenuity if progress is to be made. Foremost among these problems in cerebral palsy research is the lack of reliable, valid, and responsive clinical measures of physical activity. [1] These measures must be available before outcome studies of treatment effectiveness can be properly implemented. In particular, it is essential to be able to assess change in gross motor behavior, because gross motor dysfunction is the primary problem in children with cerebral palsy. [2]

Gross motor behavior has two main features: function and performance. [3] Gross motor function describes the achievement of particular motor activities, for example, sitting independently for 10 seconds. Gross motor performance describes the quality of a motor activity, for example, postural alignment or stability while sitting. Campbell [4] notes that movement quality is difficult to measure becaust it does not consist of a single factor, but rather is a jargonistic term inclusive of many elements.

Numerous reviewers of the therapeutic effectiveness literature have emphasized the need for improved assessments of gross motor performance in children who have cerebral palsy. [1,2,4-6] Parrette and Hourcade note that

Those studies that failed to document the effectiveness of therapeutic intervention may have used instrumentation that was not sensitive to the subtle motoric progress that was exhibited in these children. The too frequent sole evaluative dependency on motor progress (function) scales may result in these qualitative motor (performance) gains being obscured. [7(p467)]

The literature repeatedly supports the need for measurement instruments that can objectively assess quality of movement (ie, the performance of gross motor activities) in children with cerebral palsy.

From 1965 to 1990, various gross motor function instruments were developed that included elements of motor performance. Performance elements such as postural control, coordination, and balance, however, have not been well defined or measured. Moreover, many of these instruments have not been adequately tested for validity, reliability, and responsiveness in detecting clinically important changes in gross motor performance. It is necessary that an instrument being used to evaluate the effects of treatment over time have a high likelihood of detecting a clinically important treatment effect, even if that effect is small. [8] Responsiveness to change is a crucial component of a gross motor measure being considered for use in evaluating treatments in cerebral palsy. [9] Since 1984, the Gross Motor Measures Group (a group of researchers and therapists in Hamilton, Toronto, and Kingston, Ontario, Canada) has been working together to develop and validate measures for evaluating gross motor behavior in children who have cerebral palsy (see our accompanying literature review in this issue). Initially, the Gross Motor Function Measure (GMFM), with 85 items in the five functional areas of lying, sitting, crawling and kneeling, standing, and walking-running-jumping activities, was developed using a four-point scale for each item. Physical therapists used the GMFM to assess 111 children with cerebral palsy, 25 children with head injury, and 34 nondisabled children on two occasions, 4 to 6 months apart. Parents and therapists independently rated the children's function within 2 weeks of each assessment, and a sample of paired assessments was videotaped for "blind" evaluation by the therapists. Correlations between scores for change on the GMFM and the judgments of change by parents, therapists, and "blind" evaluators supported the hypothesis that the GMFM would be responsive to both negative and positive changes in gross motor function. Intrarater and interrater reliability was demonstrated with intraclass correlations of .87 to .99. This measure has been shown to be valid, reliable, and responsive in detecting clinically important change in children who have cerebral palsy. [10]

The Gross Motor Measures Group is currently developing a companion instrument, the Gross Motor Performance Measure (GMPM), using a combination of traditional measurement development techniques and new consensus methods. Five performance attributes have been identified, defined, and scaled: alignment, coordination, dissociated movement, stability, and weight shift. The measure uses 20 items selected from the GMFM. For each of the 20 function items, three attributes, or aspects of the motor performance, are assessed. For some items, an attribute may be assessed in more than one body area (eg, alignment of head and neck, alignment of shoulder girdle and arm). A five-point scale has been constructed for use with each attribute. (See Appendix 1 for attribute definitions; see Appendix 2 for an example of a GMPM item). A multitude of reasons exist why a suitable measure of gross motor performance that demonstates adequate reliability, validity, and responsiveness has not yet been developed. These reasons are conceptual, methodological, and practical and will be outlined first in this article. Subsequently, we will report on the development of an observational gross motor performance measure that addresses these issues. A validation study, for which data are currently being collected, will be reported in a future publication. At that time, the GMPM will become available for use.

Issues in Instrument

Development

Conceptual Issues

At some stage in the development of scientific knowledge, it is necessary to describe and classify observable phenomena. There is currently some disagreement in the physical therapy literature regarding the role of theory in the development of measurement systems. Michels [11] sttes that theories are needed before measurement systems can be developed so as to provide a context for item development. Rose, [12] however, stated that classification of phenomena to produce a clinically relevant body of knowledge must precede the development of theories regarding treatment. At this point in the neurodevelopmental field, treatment methods are intimately connected to theories of the Bobaths, Rood, Peto, and others. The development of a measure system based on the theory of one of these treatment methods may preclude that measurement system's use in the evaluation of other treatment methods. Thus, development of generic measures that use scientifically accepted pathokinesiological and motor control terminology may be most appropriate at this time. [13] There are also a number of other conceptual issues that must be addressed.

Defining quality. How is quality of movement defined? Is quality a single feature of a movement (eg, "poor" quality), or is quality multidimensional? Is quality merely a more specific analysis of patterns of gross motor function, or does qualty imply something about the aesthetics and fluidity of a movement in addition to its pattern? Methods to develop standardized definitions of quality are urgently required in this field, which traditionally has relied heavily on jargonistic terms and doctrinaire statements.

Observational context. In what context is quality of movement relevant? Is movement quality important in isolated test situations, or is it important only in functional activities? For example, does coordination exist as a unique, measurable construct, or can coordination only be observed during functional antigravity activities such as crawling or walking? If the latter is the case, then the question of choosing relevant motor activities in which to observe various aspects of movement quality is crucial.

Static and dynamic quality. Does quality of movement involve different features in static activities versus dynamic activities? Are unique attributes of movement quality, such as weight shift, observable only in dynamic activities? Does stability in sitting relate to stability in crawling? Is postural control needed in static activities as well as in dynamic activities?

Scaling. Is immature motor behavior "normal" in movement quality? For example, should a toddler's widebased gait be classified as "normal" in coordination? How is measurement of quality of movement in children confounded with their age? How does one assess quality in the wide variety of motor patterns normal for many functional activities, for example, in rising to a standing position? Can changes in quality of movement in children with cerebral palsy, who have a wide range of neurological involvement, be assessed with one generic scale? Should a scale be norm-referenced or criterion-referenced? What are the benefits of each approach? [14]

Methodological Issues

In addition to the usual methodological issues in instrument development, there are two particular challenges that need to be addressed in the construction of an instrument for the assessment of movement quality.

Instrument purpose. What type of measure is required to assess quality of movement? Kirshner and Guyatt [15] suggest that steps in instrument development and validation depend on the ultimate purpose for which the measure is to be used. In cerebral palsy research and in the clinic, there is a crucial need for instruments that can be used to evaluate change in motor behavior over time to study the effectiveness of treatments. An evaluative measure differs from instruments used to discriminate between children with cerebral palsy and nondisabled children (eg, Movement Assessment of Infants [16]) or those used to predict eventual motor outcome (eg, Bleck Scale of Locomotor Prognosis [17]). An evaluative measure must demonstrate responsiveness to change, which can be defined as a high likelihood of detecting a clinically important treatment effect--even if that effect is small.

One of the issues that arises out of a decision to develop an evaluative measure is concerned with choosing and developing test items amenable to change. [9] For example, a test item assessing "coordination" might be appropriate to include in an evaluative measure of motor performance, because coordination may change over time. A test item assessing the deep tendon reflexes, however, may not be appropriate to include, as these reflexes are less likely to change over time. The inclusion of nonchanging items in an evaluative measure would detract from the power of the measure to detect any clinically important changes.

Participation in instrument development. Pediatric therapists have great interest and skills in assessing quality of movement in children who have cerebral palsy and consider this to be one of their special areas of expertise. Eventual utilization of an evaluative measure in the clinic or for therapeutic effectiveness studies is contingent on therapists' acceptance of the measure as both appropriate and relevant. As the knowledge base of clinicians is essential to the content of an assessment of quality of movement, therapists' input must be solicited. A variety of consensus development methods can be used in these situations. [18]

Practical Issues

For many years, therapists have observed and described the quality of children's movement. Therapists have confidence in their judgments regarding change in motor performance. They have not been able, however, to document these changes in an objective, quantifiable manner that relates closely to functional activities. A number of practical issues have contributed to this situation.

Clinical observation. Can quality of movement be validly and reliably assessed in a clinical setting? Is quality of movement a phenomenon that can only be measured by precise laboratory instrumentation, or is it possible to assess quality of movement with an observational scale? Hopkins and Prechtl [3] have discussed some of the psychological and practical implications of these issues in the visual assessment of quality of movement in neonates. These authors contend that, at this stage in the development of instruments for assessing quality of movement, it is most appropriate to use observers' gestalt perception of motor phenomena as the basis of a classification system. Gestalt perception is defined as

... a perceptual computing mechanism which picks out relevant configurations from inessential data through unconscious statistical assessment. [19(p281)]

More simply, a child's general movements can be quantified in terms of speed, amplitude, force, and so forth, but these data might not easily display whether the movements are smooth or jerky, easy or labored. For these assessments, the therapist needs direct visual observation based on gestalt perception combined with an appreciation of aesthetics, fluidity, and form.

Assessor skills. What are the limitations on observational skills required to assess quality of movement? How many attributes of movement quality can be simultaneously scored? Can therapists evaluate total-body quality of movement, or is it only feasible to assess quality of movement by focusing on particular, relevant parts of the body?

Inconsistent performance. What is the clinical relevance of inconsistent quality of movement in a child who has cerebral palsy? Should therapists be interested in the best performance, the worst performance, or the average performance? How does one operationalize the decision making around which performance to record? What is the effect of child behavior on the assessment of movement quality?

These conceptual, methodological, and practical issues were all apparent during the development of the GMPM. These issues relate not only to theory, correct methodological procedures, and questions of instrument validity, but also ultimately to the clinical acceptability of the measure.

Development of the Gross

Motor Performance Measure

Three major steps are involved in the development of a new instrument: planning, construction, and validation. [20] The methods used in the first two phases of development of the GMPM are described next.

Planning Phase 

Determining need. The need for a measure of movement quality was established during the construction and validation of the GMFM as an evaluative instrument. [10] During the GMFM validation study, many comments were received from therapists and parents that this functional measure was not capturing all of the changes in motor behavior that children with cerebral palsy were demonstrating. In particular, the quality of movement, or performance of motor activities, was not being documented. For example, a child's improvement in stability while standing was not evident in the scoring.

The need for a measure of motor performance led to the establishment of a larger multidisciplinary and multicenter project. The research group included personnel with backgrounds in physical therapy, pediatrics, epidemiology, kinesiology, research design, and biostatistics. Over 30 pediatric physical therapists and occupational therapists were recruited from five children's treatment centers in Ontario to participate in the development and testing of the GMPM.

Therapist involvement. At an early stage in planning, it was decided that a formalized approach would be required to solicit and incorporate input from therapists and experts in the field of clinical child development. A variety of consensus methods and ranking methods were considered to structure this input. [18,21] In particular, nominal group process meetings, modified Delphi procedures, and Q-sort methods were used. Lomas [22] discusses some of the strengths and limitations of consensus methods in clinical medicine. Strengths include the possibility of bringing together various professions and conflicting viewpoints in the discussion of topical issues. Obtaining clinician input concerning feasibility issues and improving clinician awareness of research are also facilitated, with resultant feelings of "ownership" of the research. Finally, consensus methods can assist in translating a large, diverse body of knowledge into practical clinical terms.

Some weakness of consensus methods are also noted. Too much emphasis on compromise between opposing viewpoints may product ambiguous or overly generalized results. There are often inadequate time and resources available to devote to a consensus process. Finally, dominance of the consensus process by a few outspoken individuals may compromise the outcome.

In the development of the GMPM, a series of nominal group process meetings were used to develop consensus from therapists regarding the proposed instrument. The purpose of a nominal group meeting is to obtain prespecified levels of agreement on controversial subjects or statements of clinical policy. When properly used, nominal group process consensus strategies can create structured environments in which individuals are given the best available information on a topic and are encouraged to discuss the information fully before voting on particular statements. [23]

Nominal group process meetings were planned to develop consensus on selection of performance attributes, attribute definitions, instrument format, scoring system, and test administration guidelines. Agreement by at least two thirds (66%) of the therapists was prespecified as the consensus target. The results of this planning process will not be presented.

Construction Phase

Each step in teh derivation of performance attributes shown in Table 1 refers to methodological stages in instrument development and reduction of the number of attributes from 33 to 5.

Initial attribute selection and definitions. At the outset, relevant descriptors or features of gross motor performance were selected from the literature and other instruments (step 1). From these descriptors, 33 comprehensive attributes of gross motor performance such as "antigravity control" and "dissociation" were selected. Operational definitions of attributes were written by a team of three pediatric physical therapists and reviewed

[TABULAR DATA OMITTED]

by the study investigators (step 2). A variety of resources, including scientific journals, textbooks, and dictionaries, were used to write definitions that were free of jargon and that could be readily accepted by others in the scientific community (see Appendix 1 for final GMPM attributes and definitions). A nominal group process meeting was held with 30 therapists to develop consensus on definitions of these attributes (step 3). During this nominal group process meeting, the number of relevant, definable attributes was reduced from 33 to 21. This reduction was the result, in some cases, of a failure to reach a two-thirds consensus on the definition and, in other cases, by the realization that an attribute (eg, muscle tone) could not actually be observed in a clinical situation. Raw agreement levels on acceptable attributes varied from 66% to 90%.

Instrument format. Development of the format, or structure, of the GMPM instrument was a particularly difficult problem. It was evident that attributes of gross motor performance could be relevant in a wide spectrum of gross motor activities. It was unclear, however, which functional behaviors should be evaluated for performance. Availability of the GMFM allowed the investigators to use it as a basis for the development of the new measure. In addition, it was not believed to be important to assess all possible attributes for each gross motor functional behavior. Therapists also could feasibly observe and evaluate only a few attributes during the performance of any particular gross motor activity. To solve the problem of selecting functional activities and performance attributes, a Q-sort study was performed with 30 therapists (step 4). Respondents were asked to sort and rank those performance attributes that were most appropriate for each gross motor item in the GMFM. This procedure involved ranking 21 attributes for each of the motor function items. Ranking was achieved by sorting the performance attributes into five groups varying from "extremely useful" to "not useful" in assessing change in performance in the functional item. One result of the Q-sort procedure was that 15 attributes considered to be "extremely useful" or "very useful" were retained, whereas 6 attributes were eliminated from further consideration.

Item matching. The results of the Q-sort attribute matching allowed the investigators to determine a sample of three attributes of performance judged to be most relevant to each function item on the GMFM. A sample of 20 representative gross motor function items from the GMFM for use in the GMPM was also determined from the Q-sort analysis. Function items associated with the 15 most commonly chosen attributes were identified. These items were then reduced to 4 gross motor items from each of the five dimensions of the GMFM. This selection of items was achieved by balancing items in each dimension that required either maintenance of a static position (eg, sitting unsupported for 10 seconds) or dynamic achievement of a position (eg, coming to a half-kneeling position). The final 20 function items were matched with their 3 corresponding attributes from the Q-sort.

Administration guidelines. Instrument administration guidelines were written to specify environmental conditions under which the tests should be performed (room temperature and test equipment), child-state conditions (healthy, alert), therapist behavior (no handling), standardized allowable instructions to be given during testing (verbal and visual cues), sequence of testing (order not essential), and operational definitions of attributes. A final nominal group process meeting with the 30 therapists was held to achieve consensus on the instrument format, preliminary scoring system, and administration guidelines.

Pilot tests. The draft instrument was pilot-tested with 15 children who had cerebral palsy to elucidate issues of administration feasibility, clarity, and preliminary reliability (step 5). Three pairs of therapists each assessed 5 children. The estimated mean Kappa score for the therapist pairs was .51 for individual items. During this testing, the number of suitable attributes was reduced to 9.

Content validity. Once an initial draft of the GMPM was ready, content validity studies were conducted with a panel of 11 international experts in the fields of developmental therapy and research (step 6) (Tab. 2). These experts were asked to judge the suitability and representativeness of the attributes, attribute definitions, test format, and preliminary scoring system. A modified Delphi consensus approach was used, via a repeated mail survey, to obtain consensus. This

[TABULAR DATA OMITTED]

method used feedback from the panel to revise the instrument, which was then judged again by the panel. Criteria used by the judges included clarity of expression, potential for evaluating change in the target population, and instrument completeness. Individual attributes and scales were evaluated on each criterion using a scale from 1 (low agreement) to 5 (high agreement), with a mean score of at least 3.0 required for acceptance. After the first content validity study, revisions were made to the instrument and the revised instrument was sent to the experts for consensus. Results of this second content validity study demonstrated that the GMPMP has satisfactory clarity, completeness, and potential for evaluation of change (Tab. 3). At this stage, 6 attributes remained in the instrument.

Instrument scoring. Several variations of an instrument scoring system were considered for the GMPM (step 7). Generic scaling was achieved with a five-point scale, varying from "severely abnormal" to "consistently normal." A separate specific scale was constructed for each attribute in combination with each function item. This scaling method may improve reliability without sacrificing the ability of the measure to be responsive to change. Effort was made to construct scales that would be able to evaluate change in performance for a variety of children who have cerebral palsy and motor involvement varying from mild to severe. Individual scales were constructed for five attributes (see Appendix 2 for a sample GMFM item with the GMPM attributes and generic scale).

Clinical Relevance

Clinical pediatric therapists have long had a particular interest in assessment of quality of movement in children.

[TABULAR DATA OMITTED]

This interest has been demonstrated by the popularity of various certification courses in the methods of the Bobaths, Rood, Peto, and others. These courses provide training in theory, descriptive assessment of movement problems, and application of appropriate treatment methods. The descriptive assessment of movement quality, however, does not lend itself to quantification for purposes of evaluation of change in the child over time or for purposes of comparison with other children. Without such quantification, it becomes very difficult to demonstrate treatment effectiveness conclusively.

The individual clinician might be interested in the development of the GMPM for several reasons. First, the therapist can use the GMPM to profile the changes in movement quality in a single child, or in group of children, over time. This profile may be used to document the rate of change in performance when frequency of therapy is altered or when a more invasive intervention such as survery is undertaken.

Second, attributes of movement performance such as alignment, coordination, dissociated movement, stability, and weight shift are important from the therapists' viewpoint. [4] Measures capable of assessing the status of these attributes are useful in treatment planning and in monitoring progress.

Third, as clinicians use a wide variety and combinations of treatment methods, the availability of the GMPM as a generic measure, using accepted pathokinesiological and motor recovery terminology, may be of value in clinical research. Ottenbacher [24] has noted that early intervention research has a relatively poor chance of demonstrating effectiveness, because the effect size of most treatment is small. By using a reliable measure that is responsive to small changes in motor performance, the likelihood of demonstrating treatment effectiveness may be improved.

Finally, clinical managers may benefit in their program evaluation by having an objective measure of movement quality on the outcome side of cost-effectiveness studies. This is also important to therapists, because it is movement performance that many therapists feel they are able to affect with their treatment. In addition, quality assurance programs may benefit by the availability of objective measures for clinical databases.

Summary and Current Work

We have planned and constructed a new observational instrument for the evaluation of gross motor performance, or quality of movement, in children who have cerebral palsy. This instrument is designed to be used in conjunction with the previously developed GMFM. Principles involved in development of the GMPM have been (1) a collaborative multicenter and multidisciplinary approach, (2) use of standard methodological steps in instrument development, and (3) use of consensual methods with therapists and experts. Many of the conceptual, methodological, and practical issues in measuring quality of movement were addressed in planning and construction of the GMPM. The development of this measure has been a complex undertaking. We are now in the midst of a multicenter validation study to determine whether the GMPM is a valid, reliable, and responsive measure. It would be premature to publish the GMPM in its entirety until this study is completed. Data from the validation study, which involves repeated administrations of the GMFM and GMPM to 120 children who have cerebral palsy, 30 children who have head injuries, and 30 non-disabled children, will allow preliminary answers to many questions. In particular, the validation study may provide insights into the relationships between function and performance of movement. Examination of motor performance in static and dynamic activities as well as over a wide range of functional activities will be possible. The nature and development of movement performance in nondisabled children may also be studied

Detailed descriptive assessments of movement quality and overall gestalt impressions of quality have been the primary method for investigation of these phenomena to date. Our current work will determine the suitability and feasibility of an observational, objective assessment of movement performance. In addition, feedback from therapists will give valuable information on the mental processes therapists use to observe, classify, and score perceptions of motor performance.

The results of this validation study will hopefully provide the basis for further theoretical work in quality of movement as well as provide a valid, reliable, and responsive assessment instrument to be used in clinical trials of treatment for children who have cerebral palsy.

Acknowledgments

We gratefully acknowledge advice and assistance from Dr David Cadman and Sheila Jarvis during the development of this project. We particularly thank the international experts and pediatric therapists from Hotel Dieu Hospital, Chedoke-McMaster Hospitals, Hugh MacMillan Rehabilitation Centre, Ongwanada, and Belleville General Hospital for their interest in and support and reviews of our work. Special thanks to Anne Turner for preparation of this manuscript.

References

[1] Campbell SK. Measurement in developmental therapy: past, present, and future. In: Miller L, ed. Developing Norm-referenced Standardized Tests. Binghamton, NY: The Haworth Press Inc; 1989:1-13.

[2] Scherzer AL, Tscharnuter I. Early Diagnosis and Therapy in Cerebral Palsy. New York, NY: Marcel Dekker Inc; 1982:87-97.

[3] Hopkins B, Prechtl H. A qualitative approach to the development of movements during early infancy. In: Prechtl H, ed. Continuity of Neural Function from Prenatal to Postnatal Life. Philadelphia, Pa: JB Lippincott Co; 1984:143-145.

[4] Campbell SK. Assessment of the child with central nervous system dysfunction. In: Rothstein JM, ed. Measurement in Physical Therapy. New York, NY: Churchill Livingstone Inc; 1985;7:207-228.

[5] Lewko JO. Current practices in evaluating motor behavior of disabled children. Am J Occup Ther. 1976;30:413-419.

[6] Hourcade JJ, Parrette HP. Motoric change subsequent to therapeutic intervention in infants and young children who have cerebral palsy: annotated listing of group studies. Percept Mot Skills. 1984;58:519-524.

[7] Parrette HP, Hourcade JJ. A review of therapeutic intervention research on gross and fine motor progress in young children with cerebral palsy. Am J Occup Ther. 1984;38:462-468.

[8] Guyatt GH, Walter SD, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis. 1987;40:171-180.

[9] Rosenbaum PL, Russell DJ, Cadman DT, et al. Issues in measuring change in motor function in children with cerebral palsy: a special communication. Phys Ther. 1990;70:125-131.

[10] Russell DJ, Rosenbaum PL, Cadman DT, et al. The gross motor function measure: a means to evaluate the effects of physical therapy. Dev Med Child Neurol. 1989;31:341-352.

WF Boyce, MSc, PT, is Lecturer, Department of Pediatrics, Queen's University, Kingston, Ontario, Canada K7L 3N6. At the time this study was conducted, he was Physiotherapy Research Coordinator, Child Development Centre, Hotel Dieu Hospital, Kingston. Address correspondence to Mr Boyce.

C Gowland, MHSc, PT, is Assistant Professor, School of Occupational Therapy and Physiotherapy, Faculty of Health Sciences, McMaster University, and Research Manager, Physiotherapy Department, Chedoke-McMaster Hospitals, Hamilton, Ontario, Canada L8N 3Z5.

S Hardy, MSc, PT, is Senior Physiotherapist, Hugh MacMillan Rehabilitation Centre, 350 Rumsey Rd, Toronto, Ontario, Canada M4G 1R8.

PL Rosenbaum, MD, FRCP(C), is Professor, Department of Pediatrics, Faculty of Health Sciences, McMaster University, and Director of Pediatrics, Chedoke Child and Family Centre, Chedoke-McMaster Hospitals.

M Lane, DipPT-OT, PT, is Physiotherapist, Halton Parent-Infant Program, Oakville, Ontario, Canada L6J 6E1.

N Plews, BHSc, PT, is Research Physiotherapist, Chedoke-McMaster Hospitals.

C Goldsmith, PhD, is Professor, Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, McMaster University.

DJ Russell, MSc, is Research Coordinator, Department of Clinical Epidemiology and Biostastistics, Faculty of Health Sciences, McMaster University.

This study was made possible by a research grant from the Easter Seal Research Institute of Ontario.

This study was approved by Queen's University Faculty of Medicine.

[11] Michels E. Measurement in physical therapy. Phys Ther. 1983;63:209-215.

[12] Rose SJ. Description and classification: the cornerstones of pathokinesiological research. Phys Ther. 1986;66:379-381.

[13] Campbell SK. On the importance of being earnest baout measurement, or, How can we be sure that what we know is true? Phys Ther. 1987;67:1831-1833.

[14] Montgomery PC, Connolly BH. Norm-referenced and criterion-referenced tests: uses in pediatrics and application to task analysis of motor skill. Phys Ther. 1987;67:1873-1876.

[15] Kirshner B, Guyatt GH. A methodological framework for assessing health indices. J Chronic Dis. 1985;38:27-36.

[16] Chandler LS, Andrews MS, Swanson MW. Movement Assessment of Infants: A Manual. Rolling Bay, Wash: Child Development and Mental Retardation Center; 1980.

[17] Bleck EE. Locomotor prognosis in cerebral palsy. Dev Med Child Neurol. 1975;17:18-25.

[18] Fink A, Kosecoff J, Chassim M, et al. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74:979-983.

[19] Lorenz K. Gestalt perception as a source of scientific knowledge. In: Lorenz K, ed. Studies in Animal and Human Behaviour. London, England: Methuen & Co Ltd; 1971;2:281-322.

[20] Benson J, Clark F. A guide for instrument development and validation. Am J Occup Ther. 1982;36:789-800.

[21] Woodward CA, Smith KD. A Guide to scaling Techniques Useful for Health Care Research. Hamilton, Ontario, Canada: McMaster University; 1980.

[22] Lomas J. The consensus process and evidence dissemination. Can Med Assoc J. 1986;134:1340-1341.

[23] Jacoby I. The consensus development program of the National Institutes of Health: current practices and historical perspectives. Int J Tech Assess Health Care. 1985;2:420-432.

[24] Ottenbacher KJ. Statistical conclusion validation of early intervention research with handicapped children. Except Child. 1988;55:534-540.