# Effects of alternative goal structures within curriculum-based measurement.

ABSTRACT: This study assessed the effects of alternative good
structures within curriculum-based measurement (CBM) in the area of
math. Subjects were 30 special education teachers, assigned randomly to
a dynamic goal CBM, static goal CBM, or control group for 15 weeks. Each
teacher selected two mildly or moderately handicapped pupils for
participation. Analyses of variance conducted on fidelity of treatment
measures indicated that dynamic goal teachers increased goals more
frequently and, by the study's completion, employed more ambitious
goals. Multivariate analyses of covariance indicated that students in
the dynamic goal group had better content mastery than did control
students, whereas students in the static goal group did not. Content
coverage for the three groups was comparable. implications for special
education goal-setting practice are discussed. Curriculum-based
assessment is a generic term that most commonly refers to the evaluation
of instructional needs by measuring pupil performance within the local
school curriculum (Gickling, 1981; Tucker, 1985). Defined within the
framework of measurement theory, where curricular validity represents
the correspondence between tests and programmatic goals (Yalow &
Popham, 1983), curriculum based assessment links the parameters of
testing to the goal statements that constitute school curricula. So,
within special education, where a student's curriculum is specified
in the Individualized Education Program (IEP), curriculum-based
assessment refers to measurement of student performance with respect to
IEP goals.

Consequently, within curriculum-based assessment, meaningful goal specification is critical. Specification of a goal precedes and defines the curriculum-based assessment process. The goal dictates (a) the material and conditions with which measurement will occur, (b) the behavior to be observed, and (c) the criteria for judging attainment. For example, if a goal specified that a student would read third-grade material with fluency and comprehension by year's end, the curriculum-based assessment would be conducted in the following way. Stimulus materials for the assessment would comprise third-grade passages from the basal text; the behaviors observed and scored during measurement would be oral reading fluency and comprehension (e.g., words read correctly per minute and/or content words retold from the passage); and the criteria for judging proficiency would be standards for fluency (e.g., 70 words correct per minute) and comprehension (e.g., 40 content words retold). Given the close connection between goals and measurement in this process, it would appear that selection of appropriate goals, which are both realistic and ambitious, is essential to effective curriculum-based assessment.

Despite the apparent importance of goals to curriculum-based assessment, the literature on appropriate goal specification is extremely limited (see, e.g., Fuchs, 1986b). In one long-term, large-scale project, curriculum-based assessment improved pedagogy and enhanced student achievement (Fuchs, Deno, & Mirkin, 1984). However, post-hoc analyses indicated that the ambitiousness of the goal employed within the ongoing curriculum-based assessment may have mediated student achievement outcomes (Fuchs, Fuchs, & Deno, 1985). Although this finding supports use of ambitious goals, it fails to provide useful guidelines for determining ambitious goals for individual students. An absense of such guidelines has serious implications for practice. When a special education team formulates an IEP for a poorly achieving pupil, the broad outline and direction of its mission are clear: The child's rate of progress must be accelerated. But the extent to which special education services can enhance the student's learning rate remains unclear, and corresponding procedures for identifying appropriately ambitious, but realistic, IEP goals are unknown.

Consequently, the first purpose of this study was to explore alternative goal structures within one particular variant of curriculum-based assessment, known as Curriculum-Based Measurement (CBM) Deno, 1987). CBM has been developed empirically during the past decade and provides clear guidelines for many dimensions of the assessment process, including measurement methods, graphing procedures, and aspects of data use (see Deno, 1985, 1986; Deno & Fuchs, 1987; Fuchs, 1986a). However, CBM development has not adequately addressed procedures for goal specification.

The current study contrasted a conventional CBM process, wherein teachers measure and evaluate student progress with respect to a fixed annual goal, to a contrasting CBM procedure, requiring teachers to measure and evaluate student progress in relation to a dynamic goal. In this dynamic goal condition, goals are adjusted upward whenever actual student progress indicates that a more ambitious rate of progress, or a more difficult goal, may he attainable. Within this dynamic goal CBM process, the notion of goal use is reconceptualized: Instead of viewing it as a static, terminal component that fixes dimensions of assessment, the goal is related dynamically to evaluation and instruction. This study contrasted static and dynamic CBM goal structures with a control group, which did not employ systematic student monitoring procedures. Results should clarify for the special education community (a) how goals affect the curriculum-based assessment process and (b) how IEP teams might develop ambitious but realistic IEP goal statements.

The second purpose of this study was to investigate the scope of skills which CBM affects. Some have proposed (see Slavin, 1987, for a related discussion) that, although closely linked measurement and instruction may improve student mastery of the skills specified for assessment, they may reduce the extent to which teachers cover a broader curricular scope. Within this framework, the hypothesis favors the performance of CBM groups on content mastery measures, which tap the extent to which students achieve skills closely related to CBM systems. However, the hypothesis predicts that performance on content coverage indices, which assess the extent to which students achieve a broader set of related skills, would favor control groups. The current study investigated this hypothesis by including content mastery and content coverage measures. Additional outcomes were teachers' fidelity of treatment, including indices of their goal-setting behavior. METHOD Subjects Teachers. Participants were 30 special education teachers in 16 schools in a southeastern metropolitan area, who taught self-contained and resource programs for students in Grades 2-9. Teachers were assigned randomly to three treatment groups: dynamic goal CBM, static goal CBM, and control. One-way analyses of variance revealed no significant differences among teacher groups on the following variables: age level, total years teaching, years teaching special education, years computer use, years in current school, and the self-reported number of times per year teachers had evaluated students' math performance before the study began. Descriptive and inferential statistics on these variables are displayed in Table 1. Additionally, a chi-square test applied to teachers' highest educational degree revealed no statistically significant relation, X2 (4, N = 30) = 2.54, ns. In the dynamic goal CBM, static goal CBM, and control groups, respectively, there were 4, 5, and 6 teachers with bachelor's degrees; 5, 5, and 4 with master's degrees; and 1, 0, and 0 with specialist's certificates.

Information also was collected on two aspects of teacher efficacy, personal teaching efficacy and general teaching efficacy. These factors of the Teacher Efficacy Scale (Gibson & Dembo, 1984), which correspond to Bandura's (1977, 1978, 1982) two-factor theoretical model of self-efficacy, (a) demonstrate adequate convergent and discriminant validity and (b) are associated with effective teaching variables, including size of instructional group, use of criticism, and persistence in failure situations (Gibson & Dembo, 1984). A multivariate one-way analysis of variance (MANOVA) was conducted on these two scales, using Wilks' lambda to test for equality of group centroids. This analysis revealed no significant differences among groups, F (4,52) = .33, ns (Wilks' lambda = .951). Univatiate F values and descriptive statistics for these efficacy factors are shown in Table 1. Students. Each special education teacher selected for participation two mildly or moderately handicapped students, each of whom had a current goal in reading, spelling, and math. These pupils were in Grades 2-9, and had been classified as learning disabled, emotionally disturbed, or educable mentally retarded according to state regulations.

One-way analyses of variance ANOVAS) conducted on students' chronological age, grade placement, level of math performance as estimated by their teachers, and years in special education revealed no significant differences. Descriptive and inferential statistics on these variables are shown in Table 1. Additionally, chi-square tests applied to students' race, sex, and handicapping condition indicated the experimental groups were comparable. In each group, 19 boys and I girl participated. In the dynamic goal CBM, static goal CBM, and control groups, respectively, there were 9, 9, and 8 minority; 14, 14, and 18 learning disabled; 6, 4, and 2 emotionally disturbed; and 0, 2, and 0 educable mentally retarded students. Measures Two types of math achievement tests were employed: (a) a curriculum-based Math Computation Test, which served as the content mastery measure, and (b) a math concepts test, the Concepts of Number subtest of the Stanford Achievement Test, which served as the content coverage measure. Curriculum-Based Test. The Math Computation Test (MCT; Fuchs, 1987a) samples math problems across Grades 1-6 from the computation objectives of the Tennessee Basic Skills First Math Curriculum. This curriculum encompasses a statewide set of competencies that are (a) expected for promotion across grades and (b) tested annually in a statewide criterion-referenced testing program. Pupils are provided directions in standard format and have 10 minutes to complete 36 problems (or 78 digits). Performance is scored in terms of number of correct digits written in answers. Digits allow credit for partially correct problems and, as an index of achievement, appear to be more sensitive than correct problems (see Deno, 1985 for related discussion).

Criterion validity of the MCT with respect to the Concepts of Number and Math Computation subtests of the Stanford Achievement Test was .82 and .83, as calculated on this sample. Internal consistency reliability (Cronbach's alpha) obtained in this study was .93. interscorer agreement, calculated on 30% of the protocols in this study, was 98%. (Percentage of agreement = [agreement between Rater A and Rater B/(agreements between A and B, disagreements between A and B, and omissions)) x 100; see Coulter, cited in Thompson, White, & Morgan, 1982.)

This measure was deemed a useful and appropriate measure of content mastery for the following reasons. Because it was derived from a statewide curriculum equally applicable to the experimental and control groups, it should have assessed achievement on curriculum targeted for all groups, including experimental and control students (see Slavin, 1987, for related discussion). Moreover, the MCT represents more than a simple index of mastery of each individual's CBM curriculum: It encompasses the entire Grade 1-6 computation curriculum, not just the subset of the curriculum incorporated into any particular child's goal. Commercial Standardized Test. Form E of the Primary 2, Primary 3, and Intermediate I levels of the Concepts of Number (CN) subtest of the Stanford Achievement Test (Gardner, Rudman, Karlsen, & Merwin, 1982) was employed. The level administered to each pupil was determined by matching the teacher's estimate of pupil math level as closely as possible to the grade levels corresponding to a form of the test. The CN requires pupils to respond to multiple-choice questions concerning number concepts by marking their selected answers. At the Primary 2 and 3 levels, examiners read the question numbers and stems aloud, while pupils follow along silently. Examiners pause for 10 seconds between questions to allow pupils to mirk answers. At the Intermediate I level, pupils read questions to themselves and complete the questions at their own pace, but within a maximum of 20 minutes.

For the Primary 2 level of CN, Kuder-Richardson Formula #20 and alternate-form reliability coefficients were both .86. Intercorrelations among the Stanford Tests and correlations with the Otis-Lennon School Ability Test were between .62 and .78. For the Primary 3 level, Kuder-Richardson and alternateform reliability coefficients also were .86; intercorrelations and coefficients with the Otis-Lennon ranged from .62 to .72. Reliability figures for the Intermediate I form were .88 and .82, with validity coefficients between .59 and .81. Standard scores, which are comparable across levels of the test, were employed in analysis. Interscorer agreement (see previous formula), calculated on 30% of the protocols in this study, was 98%.

In this investigation, this outcome was conceptualized as an index of content coverage, rather than content mastery. See Slavin, 1987 for related discussion.) As an index of content coverage, it assessed the extent to which teachers covered material beyond the scope of the CBM computation curriculum. Treatments Curriculum-Based Measurement. For 15 weeks, each experimental teacher (including dynamic and static goal teachers) employed CBM to track their target pupils' progress toward math goals. The CBM system was rooted in the Tennessee Basic Skills First Math Program (BSF). The math computation objectives tested at each grade level within the BSF were listed. Teachers inspected these lists and determined an appropriate grade level on which to establish each student's goal. This level included the pool of math objectives the teacher hoped the student would master by the year's end. [Goals and instructional programs employed for this study were not part of the student's IEP, but rather used as an adjunct to the IEP, i.e., as "lesson plans or more detailed objectives based on the IEP" (Office of the Federal Register, 1986, p. 84). Therefore, modifying these instructional programs and objectives did not require IEP meetings (Office of the Federal Register, 1986).]

Using a standard measurement task, teachers were required to assess each pupil's math performance at least twice weekly, for 2 minutes, each time on a different probe representing the type and proportion of problems from the BSF goal level they had designated. So, if the teacher set a student's goal as third grade, the teacher was provided with 50 alternate test forms, each of which sampled the BSF third-grade computation objectives in the proportion tested on the BSF third-grade criterion-referenced end-of-year test. (Teachers actually assessed pupils' math performance I to 3 times per week with an average of 28.73 assessments across the 15-week study. No teacher ran out of alternate test forms.)

Each CBM test comprised 36 problems, displayed in random order, encompassing randomly generated numerals. Therefore, each CBM test could be conceptualized as a short form of the BSF third-grade computation test. So, as teachers monitored pupil growth on these tests, they could estimate progress toward mastery of the corresponding grade level of the BSF computation curriculum. Performance was scored in terms of number of correct digits per 2 minutes.

Teachers administered and scored these CBM tests in one of two ways. Half the teachers administered and scored the assessments themselves; the other half employed software that automatically administers tests to pupils and scores performance (see Fuchs, Hamlett, & Fuchs, 1987, for description of this software). With automatic data collection, children are assessed at a computer and scores are saved to a teacher's data-management disk for decision making. (Number keyboarding skills did not affect measurement because students completed the timed probes on paper at the computer and then entered their responses into the computer in an untimed fashion. The math assessments were completed on paper so that keyboarding and other skills required to complete complex algorithms on a computer screen would not confound the measurement of math performance.) Teachers administering the tests themselves were required to enter pupil performance scores onto the data-management disk by hand. However, the administration and scoring procedures for the teacher- and computer-administrations were completely analogous.

Additionally, within the two experimental groups (dynamic and static goal), half the teachers conducted testing themselves while the other half used computerized testing. A two-way analysis of covariance (ANCOVA) (computer vs. teacher administrations and dynamic vs. static goal, with pretreatment scores as the covariate) indicated no significant achievement effects for the administration factor or for the administration x goal interaction. Therefore, this administration factor is not discussed further here.

Teachers collected 9 initial measurements (see teacher training that follows) to master the measurement procedures and to provide a training and acclimation period to students in the test-taking procedure, before using the assessment information for instructional decision making. Teachers then collected three measurement points to calculate a median baseline performance. Teachers next set a performance criterion, representing their best estimate of what the student might accomplish by year's end. Teachers were instructed to be ambitious but realistic in setting goals; no other guidelines were offered.

Then, each week, teachers employed datamanagement software (Fuchs et al., 1987) that automatically (a) stores and graphs the math assessment data, (b) applies a set of data-evaluation decision rules to the graphed performance, and (c) provides feedback to teachers to communicate those decisions. This data-management software displays a graph on the computer screen, showing (a) the pupil's performance over time, (b) an aimline reflecting the desired slope of improvement from baseline to goal, and (c) a regression line superimposed over the data points that have been collected since the last instructional change and extrapolated to the goal date. Figure I shows a sample computerized graph. Depending on the group to which teachers had been randomly assigned, the software applied either dynamic or static goal decision rules to the graphed data. Dynamic Goal Decision Rules. The dynamic goal decision rules guided the development of the student's instructional program in the following way. First, 7 to 10 measurements were collected. Then, if the regression line was less steep than the aimline, the decision was that the teacher (a) introduce an instructional change to the pupil's program in an attempt to improve the rate of progress, and (b) collect 7 to 10 additional data points before reapplying the decision rules. If the regression line was as steep as the aimline, the decision was that the teacher (a) collect additional data points and (b) within the next 2 weeks, reapply the decision rules to the data that had been collected since the last intervention change. If the regression line was steeper than the aimline, the decision was for the teacher to (a) raise the goal to a higher level (this new goal criterion was predicted on the basis of the student's current rate of progress) and (b) collect 7 to 10 additional data points before reapplying the decision rules. Additionally, once the student's performance approached a criterion reflecting nearly maximum performance on the curriculum level (determined by having proficient adults complete the assessments), the decision was that the teacher conduct CBM on the next higher grade level of the math curriculum. Static Goal Decision Rules. The static goal decision rules guided instructional programming in the same manner as the dynamic goal decision rules, with one exception. When the regression line projected that the student would surpass the goal criterion performance, the decision rule was to collect additional data points and, within the next 2 weeks, reapply the decision rules to the data that had been collected since the last intervention change. Therefore, although teachers were free to increase their goals whenever they deemed it appropriate and wished to do so, teachers were not directed to raise their goals when the slope of the regression line exceeded that of the aimline.

The data-management software displayed the graph with regression line and aimline. Below the graph, on the same screen, a statement communicated the appropriate decision: "Uh-oh! Make a teaching change"; Ok! Collect more data"; Ok! Raise the goal to X" (only for dynamic goal group; X = projected goal); "Move to a higher grade level for measurement"; or "Insufficient data for analysis." (See example in Figure 1.) (See Fuchs, 1988, for additional information on the decision rule employed.) Control. Control group teachers set goals for their pupils using standard IEP forms. Additionally, self-reports provided on an open-ended posttreatment questionnaire Fuchs, 1987b) indicated these teachers monitored pupil progress toward goals using end-of unit math test results, unsystematic observation of performance, and workbook/worksheet performance for their database. Such descriptions match those reported by larger samples of special educators (Fuchs, Fuchs, & Warren, 1982; Mirkin & Potter, 1982). Consequently, the control group served as a benchmark representing conventional special education practice in monitoring pupil progress toward goals. CBM Training CBM teachers were trained to implement the CBM procedures over a period of approximately 7 weeks. Training included two 2-hour after school workshops, teacher practice, and staff observation of teachers. After training, staff met with teachers in their classrooms once every 2 weeks for 20 minutes to inspect graphs, discuss pupil performance patterns, and assist teachers in problem solving concerning treatment implementation. On average, during the 15-week study, teachers received 7 visits (range 5 to 13). Fidelity of Treatment The accuracy with which teachers implemented the treatment was assessed using the Math Modified Accuracy of Implementation Rating Scale MMAIRS; Fuchs, 1987a), which comprises three subscales: Initial Set Up (taking baseline, graphing data, writing goals, and drawing aimlines), Measurement (task administration, reliability of scoring, and frequency of measurement), and Data Utilization (entering information into the data-management system, describing instructional procedures, delineating instructional modifications, and timing instructional changes). Each item is rated on a 5-point Likert-type scale O = low; 4 = high), in accordance with detailed scoring guidelines. Staff were trained in scoring the M-MAIRS during one 3-hour session. Percentage of agreement (see previous formula), calculated on 15% of the protocols, was 96. (See Fuchs, 1987a, for descriptions of items and scoring guidelines.)

To explore complementary aspects of treatment implementation, two other indexes were summarized. First, the number of goal changes introduced by the teacher, following the specification of an initial goal, was counted by inspecting the data editor of the software, which recorded these changes. Second, the final goal ratio (final goal level divided by baseline median) was calculated for each pupil. Interrater agreement on these indices was calculated for 20% of the students, with percentage of agreement (see previous formula) of 98 and 100, respectively. Data Collection M-MAIRS data were collected for the two experimental groups (dynamic and static goal) 10 weeks into the study. One student per teacher was selected randomly as the target on whom M-MAIRS was assessed. The additional indexes of treatment implementation were collected after treatment implementation. Achievement data on the MCT were collected for the experimental and control groups immediately before and after the study: These data-collection periods are referred to as "Time 1" and "Time 2. " CN data were collected immediately following treatment implementation. Project staff administered the MCT and the CN in groups of one to six pupils, in standard format. Data Analysis Because teachers had been assigned randomly to treatment groups, "teacher" was designated the statistical unit of analysis. When necessary, measurements were aggregated across pupils for each teacher before data were analyzed. Although Time I MCT scores for the three groups were not reliably different, F (2, 27) = 1.87, p = .17, multivariate analyses of covariance (MANCOVAS) were employed for subsequent analysis of achievement data because of the large raw-score differences among groups. Assumptions of equal regression slopes and linearity of Y on X appeared tenable. Follow-up univariate ANCOVAs and related tests were employed, as required. RESULTS Fidelity of Treatment The one-way MANOVA was conducted on the M-MAIRS subscales. Wilks' lambda criterion was used to test for equality of group centroids, and the value calculated with the Wilks' lambda procedure was transformed into an F value through Rao's approximation (Cooley & Lohnes, 1962). The test for lambda revealed no significant differences associated with the treatment factor, F (3, 16) = 1.02, ns (Wilks' lambda = .840). Univariate F (1, 18) values for the Measurement Set Up, Measurement, and Data Evaluation subscales, respectively, were 1.00, .23, and 1.75. Descriptive statistics for the experimental groups are shown in Table 2.

Descriptive statistics for the additional indexes of treatment implementation also are displayed in Table 2. ANOVAs yielded the following statistical values: F (1, 18) = 12.81, p < .01 for number of goal changes, and F (1, 18) = 9.42, p < .01 for final goal ratio. Inspection of the means indicates the dynamic goal group introduced a greater number of goal changes. They increased goals for an average of more than one out of every two pupils (.60 or 12 out of 20 students). By contrast, in the static goal group, only one teacher increased one goal for one student (.05). Moreover, the dynamic goal group utilized a higher final goal ratio than the static goal group. Achievement The MANCOVA on the MCT and CN revealed a significant effect, F (4, 48) = 2.7 1, p < .05 Wilks' lambda = .675). Scores on the math achievement tests are shown in Table 2. The ANCOVA on the MCT produced a significant effect, F (2, 26) = 4.67, p < .05. Follow-up tests indicated the dynamic goal CBM group's adjusted achievement level was greater than that of the controls; no other difference was statistically significant. The statistically significant effect for the dynamic goal versus control groups on the MCT was associated with an effect magnitude of .52 (where effect magnitude derived from ANCOVA = adjusted experimental mean minus adjusted control mean, divided by the quantity (equation not shown) see Glass, McGaw, & Smith, 1981). Other effect magnitudes were: .25 for the contrast between the static goal CBM and control groups and .28 for the contrast between the dynamic and static goal CBM groups.

The ANCOVA on the CN indicated no reliable differences among groups, F (2,26) = 1.63, p = .21. However, contrary to the hypothesis that predicted that achievement for the control group on this content coverage measure would exceed that of the experimental groups see Slavin, 1987 for discussion), the direction of the means favored CBM. The effect magnitudes (see previous formula) for the dynamic goal versus the control groups was .39; for the static goal versus the control groups, 35. DISCUSSION As indicated on the M-MAIRS measure, the CBM teachers complied with the overall CBM procedures with comparable accuracy. Nevertheless, findings on two additional fidelity of treatment indexes revealed that their methods of using goals to structure their measurement and instructional planning activities did differ. The dynamic goal CBM teachers, whose decision rules required them to increase goals whenever actual progress exceeded the anticipated rate of improvement, increased goals more frequently than did the static goal CBM teachers, who were always free, but never required, to increase goals. Moreover, by the study's completion, the goals of the dynamic goal group were more ambitious than those of the static goal group. These accuracy of implementation findings document the integrity of the experimental treatment: CBM teachers employed the monitoring procedures as required, and the goal-setting behaviors of teachers in the contrasting CBM groups differed and conformed to the specified treatments.

Concurrent with teachers' use of alternative goal-setting processes, the CBM procedures resulted in differential student achievement. Students in the dynamic goal CBM group achieved better than the controls on the MCT, the measure of content mastery, whereas the achievement of the static goal CBM group did not exceed that of the controls. The effect magnitude associated with the dynamic goal CBM procedures was .52, or approximately one-half standard deviation. This indicates that, in terms of the standard normal curve and an achievement test scale with a population mean of 100 and standard deviation of 15, one might expect the use of CBM with dynamic goals to increase the typical achievement outcome score from 100 to approximately 107.5. This finding supports previous research in psychology indicating that adults in work settings perform better with difficult goals (Locke, Shaw, Saari, & Latham, 1981). Additionally, the current study corroborates a post-hoc special education analysis Fuchs et al., 1985), where teachers who employed more difficult CBM goals effected better student achievement outcomes.

The current study contributes to this previous literature in at least four important ways. First, within a controlled special education investigation, it documents the importance of ambitious goals for use by teachers in their assessment and instructional planning with handicapped youngsters.

Second, this study suggests that, despite the importance of ambitious goals, special educators' typical goal-setting standards may underestimate many students' potential: The study procedures allowed teachers to establish their initial goals freely, in line with the progress rates they deemed ambitious but realistic. However, with these initial goals, teachers in the dynamic goal group were required to increase goals for more than one out of every two pupils. This goal-increasing behavior was prompted by students exceeding the rates of progress that teachers anticipated.

Third, in addition to demonstrating that teachers' goals may underestimate potential progress rates, the current study indicates that, without systematic prompting to raise goals, practitioners cannot be expected to do so: Among the 20 students participating in the static goal group, there was only one instance of a teacher raising a goal.

A fourth and important way in which this study contributes to the related literature is by providing an example of a workable methodology the special education community might employ for empirically deriving ambitious, but realistic, goals. A persistent problem for special education has been that during the IEP development process, before the efficacy of special education intervention has been established for a particular student, it is difficult, if not impossible, to anticipate the scope of attainable, but ambitious, goals. The current study provides a process by which goals can be developed dynamically, so that progress toward mastery is monitored closely and goals are adjusted upward whenever possible. Given (a) the finding that such goal adjustment, specifically, and goal ambitiousness, generally, may enhance student achievement, along with (b) the relative ease and efficacy of CBM with automatic data collection and management systems see Fuchs, Hamlett, Fuchs, Stecker, & Ferguson, 1988), the special education community might consider adoption of CBM systems that incorporate dynamic goal-setting procedures.

A second purpose of the current study was to assess the extent to which CBM affected not only a measure of content mastery, but also an index of content coverage. As proposed by critics of the mastery learning field, a close link between measurement and instruction may result in better content mastery of the measured domain, but may decrease content coverage-that is, the extent to which teachers address skills related to, but not synonomous with, the measured domain (Slavin, 1987). To investigate this hypothesis, the current investigation employed a computation test as a measure of content mastery and a number concepts test as a measure of content coverage. Although findings favored the growth of the dynamic goal CBM over the control group on the content computation mastery measure, no significant differences were found on the number concepts mastery-coverage index. Nevertheless, the hypothesis suggests that the control students' achievement on the mastery-coverage test should have exceeded that of the CBM groups. In fact, even though number concepts were not the focus of the CBM measurement domain, the effect magnitude on the number concepts tests favored students in the CBM groups, with an effect magnitude of .37. This suggests that (a) CBM teachers did cover a curriculum broader than that encompassed in the CBM procedures, and/or (b) students who grew in computation concurrently progressed in or generalized their newly acquired skills to a broader curricular scope. Either way, results do not support the contention that a close connection between measurement and instruction leads to more limited achievement on content coverage measures. REFERENCES Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 8.4,

191-215. Bandura, A. 1978). Reflections on self-efficacy. Advances in Behavioral Research and Theory, 1, 237-269. Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37, 122-147. Cooley, W. W., & Lohnes, R. R. (1962). Multivariate procedures for behavioral sciences. New York: Wiley. Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219232. Deno, S. L. (1986). Fortnative evaluation of individual student programs: A new role for school psychologists. School Psychology Review, 15, 358-374. Deno, S. L. (1987). Curriculum-based measurement. TEACHING Exceptional Children, 20(i), 41-42. Deno, S. L., & Fuchs, L. S. (1987). Developing curriculum-based measurement systems for data-based special education problem solving. Focus on Exceptional Children, 19(8), 1-16. Fuchs, L. S. (I 986a). Monitoring the performance of mildy handicapped students: Review of current practice and research. Remedial and Special Education, 7, 5-12. Fuchs, L. S. (1986b). Use of goals with handicapped learners. [Spencer Foundation Seed Grant]. Available from L. S. Fuchs, Box 328, Peabody College, Vanderbilt University, Nashville, TN 37203.) Fuchs, L. S. (1987a). Improving data-based instruction through computer technology: Continuation application. (Available from L. S. Fuchs, Peabody College, Vanderbilt University, Nashville, TN 37203.) Fuchs, L. S. (I 987b). Instructional planning with curriculum-based measurement. Unpublished manuscript. Available from L. S. Fuchs, Peabody College, Vanderbilt University, Nashville, TN 37203.) Fuchs, L. S. (1988). Effects of computer-managed instruction on teachers' implementation of systematic monitoring programs and student achievement. Journal of Educational Research, 81, 294-304. Fuchs, L. S., Deno, S. L., & Mirkin, P. K. (1984). The effects of frequent curriculum-based measurement and evaluation on pedagogy, student achievement, and student awareness of learning. American Educational Research Journal, 21, 449-460. Fuchs, L. S., Fuchs, D., & Deno, S. L. 1985). The importance of goal ambitiousness and goal mastery to student achievement. Exceptional Children, 52, 63-71. Fuchs, L. S., Fuchs, D., & Warren, L. M. (1982). Special education practice in evaluating student progress toward goals (Research Report No. 81). Minneapolis: University of Minnesota Institute for Research on learning Disabilities. (ERIC Document Reproduction Service No. ED 224 197) Fuchs, L. S., Hamlett, C., & Fuchs, D. 1987). Computer

applications to curriculum-based measurement. [Unpublished

computer program]. Nashville, TN. Fuchs, L. S., Hamlett, C., Fuchs, D., Stecker, P. M., &

Ferguson, C. (1988). Conducting curriculum-based

measurement with computerized data collection: Effects

on efficiency and teacher satisfaction. Journal of Special

Education Technology, 9(2), 73-86. Gardner, E. F., Rudman, H. C., Karlsen, B., & Merwin,

J. C. (1982). Stanford achievement test. Stanford, CA:

Psychological Corporation. Gibson, S. & Dembo, M. H. (1984). Teacher efficacy: A

construct validation. Journal of Educational Psychology,

76, 569-582. Gickling, E. E. (1981). The forgotten learner. Nevada

Public Affairs Review, 1, 19-22. Glass, G. V., McGaw, B., & Smith, M. L. (1981).

Meta-analysis in social research. Beverly Hills, CA:

Sage. Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G.

P. (1981). Goal-setting and task performance: 1969 - 1980. Psychological Bulletin, 90, 125-152. Mirkin, P. K., & Potter, M. L. (1982). A survey of

program planning and implementation practices of LD

teachers (Research Report No. 80). Minneapolis: University

of Minnesota Institute for Research on Learning

Disabilities. Office of the Federal Register (1986). Code of federal

regulations, Parts 300 - 399. Washington, DC: Author. Slavin, R. E. (1987). Mastery learning reconsidered.

Review of Educational Research, 57, 175-213. Thompson, R. H., White, K. R., & Morgan, D. P. (1982).

Teacher-student interaction patterns in classrooms with

mainstreamed mildly handicapped students. American

Educational Research Journal, 19, 220-236. Tucker, J. (1985). Curriculum-based assessment: An

introduction. Exceptional Children, 52, 199-204. Yalow, E., & Popham, J. (1983). Content validity at the

crossroads. Educational Researcher, 12(g), 10-14, 2 1.

Consequently, within curriculum-based assessment, meaningful goal specification is critical. Specification of a goal precedes and defines the curriculum-based assessment process. The goal dictates (a) the material and conditions with which measurement will occur, (b) the behavior to be observed, and (c) the criteria for judging attainment. For example, if a goal specified that a student would read third-grade material with fluency and comprehension by year's end, the curriculum-based assessment would be conducted in the following way. Stimulus materials for the assessment would comprise third-grade passages from the basal text; the behaviors observed and scored during measurement would be oral reading fluency and comprehension (e.g., words read correctly per minute and/or content words retold from the passage); and the criteria for judging proficiency would be standards for fluency (e.g., 70 words correct per minute) and comprehension (e.g., 40 content words retold). Given the close connection between goals and measurement in this process, it would appear that selection of appropriate goals, which are both realistic and ambitious, is essential to effective curriculum-based assessment.

Despite the apparent importance of goals to curriculum-based assessment, the literature on appropriate goal specification is extremely limited (see, e.g., Fuchs, 1986b). In one long-term, large-scale project, curriculum-based assessment improved pedagogy and enhanced student achievement (Fuchs, Deno, & Mirkin, 1984). However, post-hoc analyses indicated that the ambitiousness of the goal employed within the ongoing curriculum-based assessment may have mediated student achievement outcomes (Fuchs, Fuchs, & Deno, 1985). Although this finding supports use of ambitious goals, it fails to provide useful guidelines for determining ambitious goals for individual students. An absense of such guidelines has serious implications for practice. When a special education team formulates an IEP for a poorly achieving pupil, the broad outline and direction of its mission are clear: The child's rate of progress must be accelerated. But the extent to which special education services can enhance the student's learning rate remains unclear, and corresponding procedures for identifying appropriately ambitious, but realistic, IEP goals are unknown.

Consequently, the first purpose of this study was to explore alternative goal structures within one particular variant of curriculum-based assessment, known as Curriculum-Based Measurement (CBM) Deno, 1987). CBM has been developed empirically during the past decade and provides clear guidelines for many dimensions of the assessment process, including measurement methods, graphing procedures, and aspects of data use (see Deno, 1985, 1986; Deno & Fuchs, 1987; Fuchs, 1986a). However, CBM development has not adequately addressed procedures for goal specification.

The current study contrasted a conventional CBM process, wherein teachers measure and evaluate student progress with respect to a fixed annual goal, to a contrasting CBM procedure, requiring teachers to measure and evaluate student progress in relation to a dynamic goal. In this dynamic goal condition, goals are adjusted upward whenever actual student progress indicates that a more ambitious rate of progress, or a more difficult goal, may he attainable. Within this dynamic goal CBM process, the notion of goal use is reconceptualized: Instead of viewing it as a static, terminal component that fixes dimensions of assessment, the goal is related dynamically to evaluation and instruction. This study contrasted static and dynamic CBM goal structures with a control group, which did not employ systematic student monitoring procedures. Results should clarify for the special education community (a) how goals affect the curriculum-based assessment process and (b) how IEP teams might develop ambitious but realistic IEP goal statements.

The second purpose of this study was to investigate the scope of skills which CBM affects. Some have proposed (see Slavin, 1987, for a related discussion) that, although closely linked measurement and instruction may improve student mastery of the skills specified for assessment, they may reduce the extent to which teachers cover a broader curricular scope. Within this framework, the hypothesis favors the performance of CBM groups on content mastery measures, which tap the extent to which students achieve skills closely related to CBM systems. However, the hypothesis predicts that performance on content coverage indices, which assess the extent to which students achieve a broader set of related skills, would favor control groups. The current study investigated this hypothesis by including content mastery and content coverage measures. Additional outcomes were teachers' fidelity of treatment, including indices of their goal-setting behavior. METHOD Subjects Teachers. Participants were 30 special education teachers in 16 schools in a southeastern metropolitan area, who taught self-contained and resource programs for students in Grades 2-9. Teachers were assigned randomly to three treatment groups: dynamic goal CBM, static goal CBM, and control. One-way analyses of variance revealed no significant differences among teacher groups on the following variables: age level, total years teaching, years teaching special education, years computer use, years in current school, and the self-reported number of times per year teachers had evaluated students' math performance before the study began. Descriptive and inferential statistics on these variables are displayed in Table 1. Additionally, a chi-square test applied to teachers' highest educational degree revealed no statistically significant relation, X2 (4, N = 30) = 2.54, ns. In the dynamic goal CBM, static goal CBM, and control groups, respectively, there were 4, 5, and 6 teachers with bachelor's degrees; 5, 5, and 4 with master's degrees; and 1, 0, and 0 with specialist's certificates.

Information also was collected on two aspects of teacher efficacy, personal teaching efficacy and general teaching efficacy. These factors of the Teacher Efficacy Scale (Gibson & Dembo, 1984), which correspond to Bandura's (1977, 1978, 1982) two-factor theoretical model of self-efficacy, (a) demonstrate adequate convergent and discriminant validity and (b) are associated with effective teaching variables, including size of instructional group, use of criticism, and persistence in failure situations (Gibson & Dembo, 1984). A multivariate one-way analysis of variance (MANOVA) was conducted on these two scales, using Wilks' lambda to test for equality of group centroids. This analysis revealed no significant differences among groups, F (4,52) = .33, ns (Wilks' lambda = .951). Univatiate F values and descriptive statistics for these efficacy factors are shown in Table 1. Students. Each special education teacher selected for participation two mildly or moderately handicapped students, each of whom had a current goal in reading, spelling, and math. These pupils were in Grades 2-9, and had been classified as learning disabled, emotionally disturbed, or educable mentally retarded according to state regulations.

One-way analyses of variance ANOVAS) conducted on students' chronological age, grade placement, level of math performance as estimated by their teachers, and years in special education revealed no significant differences. Descriptive and inferential statistics on these variables are shown in Table 1. Additionally, chi-square tests applied to students' race, sex, and handicapping condition indicated the experimental groups were comparable. In each group, 19 boys and I girl participated. In the dynamic goal CBM, static goal CBM, and control groups, respectively, there were 9, 9, and 8 minority; 14, 14, and 18 learning disabled; 6, 4, and 2 emotionally disturbed; and 0, 2, and 0 educable mentally retarded students. Measures Two types of math achievement tests were employed: (a) a curriculum-based Math Computation Test, which served as the content mastery measure, and (b) a math concepts test, the Concepts of Number subtest of the Stanford Achievement Test, which served as the content coverage measure. Curriculum-Based Test. The Math Computation Test (MCT; Fuchs, 1987a) samples math problems across Grades 1-6 from the computation objectives of the Tennessee Basic Skills First Math Curriculum. This curriculum encompasses a statewide set of competencies that are (a) expected for promotion across grades and (b) tested annually in a statewide criterion-referenced testing program. Pupils are provided directions in standard format and have 10 minutes to complete 36 problems (or 78 digits). Performance is scored in terms of number of correct digits written in answers. Digits allow credit for partially correct problems and, as an index of achievement, appear to be more sensitive than correct problems (see Deno, 1985 for related discussion).

Criterion validity of the MCT with respect to the Concepts of Number and Math Computation subtests of the Stanford Achievement Test was .82 and .83, as calculated on this sample. Internal consistency reliability (Cronbach's alpha) obtained in this study was .93. interscorer agreement, calculated on 30% of the protocols in this study, was 98%. (Percentage of agreement = [agreement between Rater A and Rater B/(agreements between A and B, disagreements between A and B, and omissions)) x 100; see Coulter, cited in Thompson, White, & Morgan, 1982.)

This measure was deemed a useful and appropriate measure of content mastery for the following reasons. Because it was derived from a statewide curriculum equally applicable to the experimental and control groups, it should have assessed achievement on curriculum targeted for all groups, including experimental and control students (see Slavin, 1987, for related discussion). Moreover, the MCT represents more than a simple index of mastery of each individual's CBM curriculum: It encompasses the entire Grade 1-6 computation curriculum, not just the subset of the curriculum incorporated into any particular child's goal. Commercial Standardized Test. Form E of the Primary 2, Primary 3, and Intermediate I levels of the Concepts of Number (CN) subtest of the Stanford Achievement Test (Gardner, Rudman, Karlsen, & Merwin, 1982) was employed. The level administered to each pupil was determined by matching the teacher's estimate of pupil math level as closely as possible to the grade levels corresponding to a form of the test. The CN requires pupils to respond to multiple-choice questions concerning number concepts by marking their selected answers. At the Primary 2 and 3 levels, examiners read the question numbers and stems aloud, while pupils follow along silently. Examiners pause for 10 seconds between questions to allow pupils to mirk answers. At the Intermediate I level, pupils read questions to themselves and complete the questions at their own pace, but within a maximum of 20 minutes.

For the Primary 2 level of CN, Kuder-Richardson Formula #20 and alternate-form reliability coefficients were both .86. Intercorrelations among the Stanford Tests and correlations with the Otis-Lennon School Ability Test were between .62 and .78. For the Primary 3 level, Kuder-Richardson and alternateform reliability coefficients also were .86; intercorrelations and coefficients with the Otis-Lennon ranged from .62 to .72. Reliability figures for the Intermediate I form were .88 and .82, with validity coefficients between .59 and .81. Standard scores, which are comparable across levels of the test, were employed in analysis. Interscorer agreement (see previous formula), calculated on 30% of the protocols in this study, was 98%.

In this investigation, this outcome was conceptualized as an index of content coverage, rather than content mastery. See Slavin, 1987 for related discussion.) As an index of content coverage, it assessed the extent to which teachers covered material beyond the scope of the CBM computation curriculum. Treatments Curriculum-Based Measurement. For 15 weeks, each experimental teacher (including dynamic and static goal teachers) employed CBM to track their target pupils' progress toward math goals. The CBM system was rooted in the Tennessee Basic Skills First Math Program (BSF). The math computation objectives tested at each grade level within the BSF were listed. Teachers inspected these lists and determined an appropriate grade level on which to establish each student's goal. This level included the pool of math objectives the teacher hoped the student would master by the year's end. [Goals and instructional programs employed for this study were not part of the student's IEP, but rather used as an adjunct to the IEP, i.e., as "lesson plans or more detailed objectives based on the IEP" (Office of the Federal Register, 1986, p. 84). Therefore, modifying these instructional programs and objectives did not require IEP meetings (Office of the Federal Register, 1986).]

Using a standard measurement task, teachers were required to assess each pupil's math performance at least twice weekly, for 2 minutes, each time on a different probe representing the type and proportion of problems from the BSF goal level they had designated. So, if the teacher set a student's goal as third grade, the teacher was provided with 50 alternate test forms, each of which sampled the BSF third-grade computation objectives in the proportion tested on the BSF third-grade criterion-referenced end-of-year test. (Teachers actually assessed pupils' math performance I to 3 times per week with an average of 28.73 assessments across the 15-week study. No teacher ran out of alternate test forms.)

Each CBM test comprised 36 problems, displayed in random order, encompassing randomly generated numerals. Therefore, each CBM test could be conceptualized as a short form of the BSF third-grade computation test. So, as teachers monitored pupil growth on these tests, they could estimate progress toward mastery of the corresponding grade level of the BSF computation curriculum. Performance was scored in terms of number of correct digits per 2 minutes.

Teachers administered and scored these CBM tests in one of two ways. Half the teachers administered and scored the assessments themselves; the other half employed software that automatically administers tests to pupils and scores performance (see Fuchs, Hamlett, & Fuchs, 1987, for description of this software). With automatic data collection, children are assessed at a computer and scores are saved to a teacher's data-management disk for decision making. (Number keyboarding skills did not affect measurement because students completed the timed probes on paper at the computer and then entered their responses into the computer in an untimed fashion. The math assessments were completed on paper so that keyboarding and other skills required to complete complex algorithms on a computer screen would not confound the measurement of math performance.) Teachers administering the tests themselves were required to enter pupil performance scores onto the data-management disk by hand. However, the administration and scoring procedures for the teacher- and computer-administrations were completely analogous.

Additionally, within the two experimental groups (dynamic and static goal), half the teachers conducted testing themselves while the other half used computerized testing. A two-way analysis of covariance (ANCOVA) (computer vs. teacher administrations and dynamic vs. static goal, with pretreatment scores as the covariate) indicated no significant achievement effects for the administration factor or for the administration x goal interaction. Therefore, this administration factor is not discussed further here.

Teachers collected 9 initial measurements (see teacher training that follows) to master the measurement procedures and to provide a training and acclimation period to students in the test-taking procedure, before using the assessment information for instructional decision making. Teachers then collected three measurement points to calculate a median baseline performance. Teachers next set a performance criterion, representing their best estimate of what the student might accomplish by year's end. Teachers were instructed to be ambitious but realistic in setting goals; no other guidelines were offered.

Then, each week, teachers employed datamanagement software (Fuchs et al., 1987) that automatically (a) stores and graphs the math assessment data, (b) applies a set of data-evaluation decision rules to the graphed performance, and (c) provides feedback to teachers to communicate those decisions. This data-management software displays a graph on the computer screen, showing (a) the pupil's performance over time, (b) an aimline reflecting the desired slope of improvement from baseline to goal, and (c) a regression line superimposed over the data points that have been collected since the last instructional change and extrapolated to the goal date. Figure I shows a sample computerized graph. Depending on the group to which teachers had been randomly assigned, the software applied either dynamic or static goal decision rules to the graphed data. Dynamic Goal Decision Rules. The dynamic goal decision rules guided the development of the student's instructional program in the following way. First, 7 to 10 measurements were collected. Then, if the regression line was less steep than the aimline, the decision was that the teacher (a) introduce an instructional change to the pupil's program in an attempt to improve the rate of progress, and (b) collect 7 to 10 additional data points before reapplying the decision rules. If the regression line was as steep as the aimline, the decision was that the teacher (a) collect additional data points and (b) within the next 2 weeks, reapply the decision rules to the data that had been collected since the last intervention change. If the regression line was steeper than the aimline, the decision was for the teacher to (a) raise the goal to a higher level (this new goal criterion was predicted on the basis of the student's current rate of progress) and (b) collect 7 to 10 additional data points before reapplying the decision rules. Additionally, once the student's performance approached a criterion reflecting nearly maximum performance on the curriculum level (determined by having proficient adults complete the assessments), the decision was that the teacher conduct CBM on the next higher grade level of the math curriculum. Static Goal Decision Rules. The static goal decision rules guided instructional programming in the same manner as the dynamic goal decision rules, with one exception. When the regression line projected that the student would surpass the goal criterion performance, the decision rule was to collect additional data points and, within the next 2 weeks, reapply the decision rules to the data that had been collected since the last intervention change. Therefore, although teachers were free to increase their goals whenever they deemed it appropriate and wished to do so, teachers were not directed to raise their goals when the slope of the regression line exceeded that of the aimline.

The data-management software displayed the graph with regression line and aimline. Below the graph, on the same screen, a statement communicated the appropriate decision: "Uh-oh! Make a teaching change"; Ok! Collect more data"; Ok! Raise the goal to X" (only for dynamic goal group; X = projected goal); "Move to a higher grade level for measurement"; or "Insufficient data for analysis." (See example in Figure 1.) (See Fuchs, 1988, for additional information on the decision rule employed.) Control. Control group teachers set goals for their pupils using standard IEP forms. Additionally, self-reports provided on an open-ended posttreatment questionnaire Fuchs, 1987b) indicated these teachers monitored pupil progress toward goals using end-of unit math test results, unsystematic observation of performance, and workbook/worksheet performance for their database. Such descriptions match those reported by larger samples of special educators (Fuchs, Fuchs, & Warren, 1982; Mirkin & Potter, 1982). Consequently, the control group served as a benchmark representing conventional special education practice in monitoring pupil progress toward goals. CBM Training CBM teachers were trained to implement the CBM procedures over a period of approximately 7 weeks. Training included two 2-hour after school workshops, teacher practice, and staff observation of teachers. After training, staff met with teachers in their classrooms once every 2 weeks for 20 minutes to inspect graphs, discuss pupil performance patterns, and assist teachers in problem solving concerning treatment implementation. On average, during the 15-week study, teachers received 7 visits (range 5 to 13). Fidelity of Treatment The accuracy with which teachers implemented the treatment was assessed using the Math Modified Accuracy of Implementation Rating Scale MMAIRS; Fuchs, 1987a), which comprises three subscales: Initial Set Up (taking baseline, graphing data, writing goals, and drawing aimlines), Measurement (task administration, reliability of scoring, and frequency of measurement), and Data Utilization (entering information into the data-management system, describing instructional procedures, delineating instructional modifications, and timing instructional changes). Each item is rated on a 5-point Likert-type scale O = low; 4 = high), in accordance with detailed scoring guidelines. Staff were trained in scoring the M-MAIRS during one 3-hour session. Percentage of agreement (see previous formula), calculated on 15% of the protocols, was 96. (See Fuchs, 1987a, for descriptions of items and scoring guidelines.)

To explore complementary aspects of treatment implementation, two other indexes were summarized. First, the number of goal changes introduced by the teacher, following the specification of an initial goal, was counted by inspecting the data editor of the software, which recorded these changes. Second, the final goal ratio (final goal level divided by baseline median) was calculated for each pupil. Interrater agreement on these indices was calculated for 20% of the students, with percentage of agreement (see previous formula) of 98 and 100, respectively. Data Collection M-MAIRS data were collected for the two experimental groups (dynamic and static goal) 10 weeks into the study. One student per teacher was selected randomly as the target on whom M-MAIRS was assessed. The additional indexes of treatment implementation were collected after treatment implementation. Achievement data on the MCT were collected for the experimental and control groups immediately before and after the study: These data-collection periods are referred to as "Time 1" and "Time 2. " CN data were collected immediately following treatment implementation. Project staff administered the MCT and the CN in groups of one to six pupils, in standard format. Data Analysis Because teachers had been assigned randomly to treatment groups, "teacher" was designated the statistical unit of analysis. When necessary, measurements were aggregated across pupils for each teacher before data were analyzed. Although Time I MCT scores for the three groups were not reliably different, F (2, 27) = 1.87, p = .17, multivariate analyses of covariance (MANCOVAS) were employed for subsequent analysis of achievement data because of the large raw-score differences among groups. Assumptions of equal regression slopes and linearity of Y on X appeared tenable. Follow-up univariate ANCOVAs and related tests were employed, as required. RESULTS Fidelity of Treatment The one-way MANOVA was conducted on the M-MAIRS subscales. Wilks' lambda criterion was used to test for equality of group centroids, and the value calculated with the Wilks' lambda procedure was transformed into an F value through Rao's approximation (Cooley & Lohnes, 1962). The test for lambda revealed no significant differences associated with the treatment factor, F (3, 16) = 1.02, ns (Wilks' lambda = .840). Univariate F (1, 18) values for the Measurement Set Up, Measurement, and Data Evaluation subscales, respectively, were 1.00, .23, and 1.75. Descriptive statistics for the experimental groups are shown in Table 2.

Descriptive statistics for the additional indexes of treatment implementation also are displayed in Table 2. ANOVAs yielded the following statistical values: F (1, 18) = 12.81, p < .01 for number of goal changes, and F (1, 18) = 9.42, p < .01 for final goal ratio. Inspection of the means indicates the dynamic goal group introduced a greater number of goal changes. They increased goals for an average of more than one out of every two pupils (.60 or 12 out of 20 students). By contrast, in the static goal group, only one teacher increased one goal for one student (.05). Moreover, the dynamic goal group utilized a higher final goal ratio than the static goal group. Achievement The MANCOVA on the MCT and CN revealed a significant effect, F (4, 48) = 2.7 1, p < .05 Wilks' lambda = .675). Scores on the math achievement tests are shown in Table 2. The ANCOVA on the MCT produced a significant effect, F (2, 26) = 4.67, p < .05. Follow-up tests indicated the dynamic goal CBM group's adjusted achievement level was greater than that of the controls; no other difference was statistically significant. The statistically significant effect for the dynamic goal versus control groups on the MCT was associated with an effect magnitude of .52 (where effect magnitude derived from ANCOVA = adjusted experimental mean minus adjusted control mean, divided by the quantity (equation not shown) see Glass, McGaw, & Smith, 1981). Other effect magnitudes were: .25 for the contrast between the static goal CBM and control groups and .28 for the contrast between the dynamic and static goal CBM groups.

The ANCOVA on the CN indicated no reliable differences among groups, F (2,26) = 1.63, p = .21. However, contrary to the hypothesis that predicted that achievement for the control group on this content coverage measure would exceed that of the experimental groups see Slavin, 1987 for discussion), the direction of the means favored CBM. The effect magnitudes (see previous formula) for the dynamic goal versus the control groups was .39; for the static goal versus the control groups, 35. DISCUSSION As indicated on the M-MAIRS measure, the CBM teachers complied with the overall CBM procedures with comparable accuracy. Nevertheless, findings on two additional fidelity of treatment indexes revealed that their methods of using goals to structure their measurement and instructional planning activities did differ. The dynamic goal CBM teachers, whose decision rules required them to increase goals whenever actual progress exceeded the anticipated rate of improvement, increased goals more frequently than did the static goal CBM teachers, who were always free, but never required, to increase goals. Moreover, by the study's completion, the goals of the dynamic goal group were more ambitious than those of the static goal group. These accuracy of implementation findings document the integrity of the experimental treatment: CBM teachers employed the monitoring procedures as required, and the goal-setting behaviors of teachers in the contrasting CBM groups differed and conformed to the specified treatments.

Concurrent with teachers' use of alternative goal-setting processes, the CBM procedures resulted in differential student achievement. Students in the dynamic goal CBM group achieved better than the controls on the MCT, the measure of content mastery, whereas the achievement of the static goal CBM group did not exceed that of the controls. The effect magnitude associated with the dynamic goal CBM procedures was .52, or approximately one-half standard deviation. This indicates that, in terms of the standard normal curve and an achievement test scale with a population mean of 100 and standard deviation of 15, one might expect the use of CBM with dynamic goals to increase the typical achievement outcome score from 100 to approximately 107.5. This finding supports previous research in psychology indicating that adults in work settings perform better with difficult goals (Locke, Shaw, Saari, & Latham, 1981). Additionally, the current study corroborates a post-hoc special education analysis Fuchs et al., 1985), where teachers who employed more difficult CBM goals effected better student achievement outcomes.

The current study contributes to this previous literature in at least four important ways. First, within a controlled special education investigation, it documents the importance of ambitious goals for use by teachers in their assessment and instructional planning with handicapped youngsters.

Second, this study suggests that, despite the importance of ambitious goals, special educators' typical goal-setting standards may underestimate many students' potential: The study procedures allowed teachers to establish their initial goals freely, in line with the progress rates they deemed ambitious but realistic. However, with these initial goals, teachers in the dynamic goal group were required to increase goals for more than one out of every two pupils. This goal-increasing behavior was prompted by students exceeding the rates of progress that teachers anticipated.

Third, in addition to demonstrating that teachers' goals may underestimate potential progress rates, the current study indicates that, without systematic prompting to raise goals, practitioners cannot be expected to do so: Among the 20 students participating in the static goal group, there was only one instance of a teacher raising a goal.

A fourth and important way in which this study contributes to the related literature is by providing an example of a workable methodology the special education community might employ for empirically deriving ambitious, but realistic, goals. A persistent problem for special education has been that during the IEP development process, before the efficacy of special education intervention has been established for a particular student, it is difficult, if not impossible, to anticipate the scope of attainable, but ambitious, goals. The current study provides a process by which goals can be developed dynamically, so that progress toward mastery is monitored closely and goals are adjusted upward whenever possible. Given (a) the finding that such goal adjustment, specifically, and goal ambitiousness, generally, may enhance student achievement, along with (b) the relative ease and efficacy of CBM with automatic data collection and management systems see Fuchs, Hamlett, Fuchs, Stecker, & Ferguson, 1988), the special education community might consider adoption of CBM systems that incorporate dynamic goal-setting procedures.

A second purpose of the current study was to assess the extent to which CBM affected not only a measure of content mastery, but also an index of content coverage. As proposed by critics of the mastery learning field, a close link between measurement and instruction may result in better content mastery of the measured domain, but may decrease content coverage-that is, the extent to which teachers address skills related to, but not synonomous with, the measured domain (Slavin, 1987). To investigate this hypothesis, the current investigation employed a computation test as a measure of content mastery and a number concepts test as a measure of content coverage. Although findings favored the growth of the dynamic goal CBM over the control group on the content computation mastery measure, no significant differences were found on the number concepts mastery-coverage index. Nevertheless, the hypothesis suggests that the control students' achievement on the mastery-coverage test should have exceeded that of the CBM groups. In fact, even though number concepts were not the focus of the CBM measurement domain, the effect magnitude on the number concepts tests favored students in the CBM groups, with an effect magnitude of .37. This suggests that (a) CBM teachers did cover a curriculum broader than that encompassed in the CBM procedures, and/or (b) students who grew in computation concurrently progressed in or generalized their newly acquired skills to a broader curricular scope. Either way, results do not support the contention that a close connection between measurement and instruction leads to more limited achievement on content coverage measures. REFERENCES Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 8.4,

191-215. Bandura, A. 1978). Reflections on self-efficacy. Advances in Behavioral Research and Theory, 1, 237-269. Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37, 122-147. Cooley, W. W., & Lohnes, R. R. (1962). Multivariate procedures for behavioral sciences. New York: Wiley. Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219232. Deno, S. L. (1986). Fortnative evaluation of individual student programs: A new role for school psychologists. School Psychology Review, 15, 358-374. Deno, S. L. (1987). Curriculum-based measurement. TEACHING Exceptional Children, 20(i), 41-42. Deno, S. L., & Fuchs, L. S. (1987). Developing curriculum-based measurement systems for data-based special education problem solving. Focus on Exceptional Children, 19(8), 1-16. Fuchs, L. S. (I 986a). Monitoring the performance of mildy handicapped students: Review of current practice and research. Remedial and Special Education, 7, 5-12. Fuchs, L. S. (1986b). Use of goals with handicapped learners. [Spencer Foundation Seed Grant]. Available from L. S. Fuchs, Box 328, Peabody College, Vanderbilt University, Nashville, TN 37203.) Fuchs, L. S. (1987a). Improving data-based instruction through computer technology: Continuation application. (Available from L. S. Fuchs, Peabody College, Vanderbilt University, Nashville, TN 37203.) Fuchs, L. S. (I 987b). Instructional planning with curriculum-based measurement. Unpublished manuscript. Available from L. S. Fuchs, Peabody College, Vanderbilt University, Nashville, TN 37203.) Fuchs, L. S. (1988). Effects of computer-managed instruction on teachers' implementation of systematic monitoring programs and student achievement. Journal of Educational Research, 81, 294-304. Fuchs, L. S., Deno, S. L., & Mirkin, P. K. (1984). The effects of frequent curriculum-based measurement and evaluation on pedagogy, student achievement, and student awareness of learning. American Educational Research Journal, 21, 449-460. Fuchs, L. S., Fuchs, D., & Deno, S. L. 1985). The importance of goal ambitiousness and goal mastery to student achievement. Exceptional Children, 52, 63-71. Fuchs, L. S., Fuchs, D., & Warren, L. M. (1982). Special education practice in evaluating student progress toward goals (Research Report No. 81). Minneapolis: University of Minnesota Institute for Research on learning Disabilities. (ERIC Document Reproduction Service No. ED 224 197) Fuchs, L. S., Hamlett, C., & Fuchs, D. 1987). Computer

applications to curriculum-based measurement. [Unpublished

computer program]. Nashville, TN. Fuchs, L. S., Hamlett, C., Fuchs, D., Stecker, P. M., &

Ferguson, C. (1988). Conducting curriculum-based

measurement with computerized data collection: Effects

on efficiency and teacher satisfaction. Journal of Special

Education Technology, 9(2), 73-86. Gardner, E. F., Rudman, H. C., Karlsen, B., & Merwin,

J. C. (1982). Stanford achievement test. Stanford, CA:

Psychological Corporation. Gibson, S. & Dembo, M. H. (1984). Teacher efficacy: A

construct validation. Journal of Educational Psychology,

76, 569-582. Gickling, E. E. (1981). The forgotten learner. Nevada

Public Affairs Review, 1, 19-22. Glass, G. V., McGaw, B., & Smith, M. L. (1981).

Meta-analysis in social research. Beverly Hills, CA:

Sage. Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G.

P. (1981). Goal-setting and task performance: 1969 - 1980. Psychological Bulletin, 90, 125-152. Mirkin, P. K., & Potter, M. L. (1982). A survey of

program planning and implementation practices of LD

teachers (Research Report No. 80). Minneapolis: University

of Minnesota Institute for Research on Learning

Disabilities. Office of the Federal Register (1986). Code of federal

regulations, Parts 300 - 399. Washington, DC: Author. Slavin, R. E. (1987). Mastery learning reconsidered.

Review of Educational Research, 57, 175-213. Thompson, R. H., White, K. R., & Morgan, D. P. (1982).

Teacher-student interaction patterns in classrooms with

mainstreamed mildly handicapped students. American

Educational Research Journal, 19, 220-236. Tucker, J. (1985). Curriculum-based assessment: An

introduction. Exceptional Children, 52, 199-204. Yalow, E., & Popham, J. (1983). Content validity at the

crossroads. Educational Researcher, 12(g), 10-14, 2 1.

Printer friendly Cite/link Email Feedback | |

Author: | Fuchs, Lynn S.; Fuchs, Douglas; Hamlett, Carol L. |
---|---|

Publication: | Exceptional Children |

Date: | Feb 1, 1989 |

Words: | 5715 |

Previous Article: | Effects of social integration on preschool children with handicaps. |

Next Article: | The regular education initiative debate: its promises and problems. |

Topics: |