CBM with goal setting: impacting students' understanding of reading goals.

This pilot study investigated if goal setting with curriculum-based measurement (CBM) was effective in increasing student awareness of goal knowledge and if students could set realistic daily reading goals. Nineteen 6th and 7th grade students with learning disabilities participated in a goal setting treatment group or a control group. During the intervention period, all students completed computerized CBM assessments. For pre- and posttesting, students completed measures of goal knowledge. During posttesting more students in the goal setting group were able to specifically state goals: however, students in the goal setting condition had difficulty setting realistic daily CBM goals.

**********

Students with learning disabilities benefit from setting learning and behavior goals, yet they often have difficulty setting realistic goals (Graham & Harris, 1989) and accurately self-evaluating their academic skills (Stone & May, 2002). Curriculum-based measurement (CBM) may be one instrument to help students' understand their academic goals and progress. CBM refers to a systematic, standardized, reliable procedure for documenting a student's progress in reading, mathematics, spelling, or written language (Deno, 1985, Deno, 1989) Deno (2003) reported many uses of CBM such as predicting performance on important criteria, enhancing teacher instructional planning, developing norms, increasing ease of communication, screening to identify students academically at risk, recommending and evaluating inclusion and several other purposes. In the area of increasing the ease of communication, research has shown the effectiveness of CBM with assisting students to understand and accurately state reading goals (Fuchs, Deno, & Mirkin, 1984).

Student goal setting appears to fit very well with CBM as CBM involves setting daily and long-term goals. Students who participate in goal setting may increase their performance (Fuchs, Fuchs, & Deno, 1985; Johnson, Graham, & Harris, 1997). Studies of goal setting and CBM have found that students have increased knowledge of goals (Fuchs, Butterworth, & Fuchs, 1989a) and respond better to self-set goals as opposed to teacher-set goals (Fuchs, Bahr, & Rieth, 1989), and respond with more effort to task-focused goals (Fuchs, Fuchs, Karnes, Hamlett, Katzaroff & Dutka, 1997a).

Although CBM is an alternative assessment that has proven to be effective in monitoring students' ongoing progress in basic skills (Deno, 1985), the effects of student involvement during these assessments needs further investigation. The active involvement of students in the assessment procedure is an area that is often overlooked. Goal setting during CBM is an area that might be beneficial to students for the following reason: For students to have an understanding of their academic progress, they need to understand their reading goals and be able to work with their teachers to determine if they are making progress. Although proximal or daily student goal setting is a possible feature of CBM, CBM usually involves teachers setting end-of-year goals and connecting the student's baseline performance to the end-of-year goal. The goal line or aim line visually provides daily goals for each time the student is assessed. Students may be aware of their proximal goals by looking at their graphs; however, training students on goal setting procedures is often not part of the CBM routine.

Although CBM and student goal setting appear to fit together well, few studies have examined the impact that student goal setting has on CBM performance. Fuchs, et al. (1997a) found that elementary students who were low achieving chose more challenging goals and increased their effort. This study found that only students who were low achieving without learning disabilities showed greater mathematics learning. Students with learning disabilities did not show the same increase in learning.

The purpose of this pilot study was to examine the effectiveness of goal setting with CBM on increasing students' understanding of their reading goals. The ability of middle school students to set realistic goals for CBM reading assessments was also examined for students who participated in the goal setting group. This study explored how these procedures could be implemented with middle school students and practical suggestions for future application are given. Limitations of the present study and future research directions are discussed.

Method

Participants

Teachers. Four special education teachers who taught language arts in a special education setting from two suburban middle schools in a large Midwestern metropolitan area participated in this investigation. The two middle schools had enrollments of 874 and 896. Teachers from each school were randomly assigned to one of two conitions: CBM with goal setting or the control group. Teachers at the second middle school were matched in order to have one sixth and one seventh grade teacher in each treatment group. The two teachers in the Goal Setting condition had a mean of 9.5 years (SD = 0.71) of special education teaching experience and the two teachers in the Control Group had a mean of 9.5 years (SD = 9.2).

Students. Participants in the study were 19 sixth (n = 10) and seventh (n = 9) grade students who had met the following stepwise criteria: a) identified as having a learning disability by their local school district, b) Intelligence Quotient (IQ) scores between 80 and 120 on an intelligence test given by a school psychologist, c) special education language arts classroom placement, and d) an absence of a visual or hearing impairment. A series oft-tests revealed no significant differences between treatment groups with respect to students' (a) chronological age, t (17) = 0.31, p= .76, (b) IQ, t (17) = 0.78, p = .44, and (c) broad reading standard scores, t (17) = 0.19, p = .85. A chi square test revealed no significant difference between treatment groups with respect to students' gender, [chi square] (1, N = 19) = 1.80, p = .18. A chi square test revealed no significance between treatment groups and student grade level, [chi square] (1, N = 19) = .05, p = .82. Table 1 provides student characteristics by experimental condition.

Instrumentation

CBM. Teachers in both conditions employed CBM to track their students' progress toward reading goals for 7 weeks. During the study all students were assessed using the computerized maze CBM twice weekly. These assessments took approximately 10 minutes per week per child. CBM monitoring consisted of goal selection and ongoing measurement of the goal material. The computerized CBM procedure, Monitoring Basic Skills Progress: Basic Reading (Fuchs, Hamlett, & Fuchs, 1997b) was used to document ongoing progress in reading. As CBM was new to all of the teacher participants in this study, teachers were instructed not to make instructional decisions based on the information gained through CBM. Teachers were instructed on how to make educational decisions based upon CBM results upon the conclusion of this study.

Student Interview. The student interview assessed students' awareness of individual reading goals. Students in both experimental conditions were asked twelve questions modified from the Student Awareness of Goals and Progress Interview (SAGPI; Fuchs et al., 1989a) that indicated his or her awareness of individual reading goals. Students' answers to the interview questions required "yes", "no" or "'don't know" responses for seven of the items, three items provided continuous data, and two items provided categorical data. Categorical questions included items such as "Does your teacher have a graph or record of how well you are doing in reading?" and "Can you tell whether you are improving in reading by your grades?" Continuous data was obtained from questions such as "How often does your teacher tell you your grade in reading?" and "What would your goal be for reading tomorrow?"

Procedures. Three months prior to the implementation of the study, teachers received structured training in CBM during three one-hour after school workshops. One week prior to the study, teachers met with the researcher to learn the procedures involved with their specific intervention and to review CBM procedures. Ongoing assistance was provided during the 7 weeks of the study.

Two weeks prior to the implementation of the study, all students from both experimental groups were taught how to use the computerized CBM program Monitoring Basic Skills Progress: Basic Reading (Fuchs et al., 1997b). During this training, all students (a) reviewed how to use the keyboard; all students had previously received keyboard training through the school district; (b) were taught how to use the computer software; (c) were observed using the software until they demonstrated correct use on two separate occasions. Mastery of the software was determined through teacher observation of two concurrent successful completions of the CBM program. Corrective feedback was provided as necessary until all students reached mastery of the CBM program. Baselines were established by using the last three scores from these training sessions. End-of-intervention goals were established by the teachers with assistance from the researcher.

CBM with Goal Setting. Students in this condition attended a training session on how to set goals using CBM data. Students were taught (a) the purpose of a goal and goal line, (b) how to identify their daily goal, (c) to try to reach their daily goal, and (d) how to determine if they were successful in reaching their daily goal. To ensure that the goal training was consistent, a training script was used at each of the two middle schools. Students in this condition saw the teacher-set goal line before and after the computerized assessment. Students were instructed to set daily goals by writing down the number of correct word replacements that the students felt they could meet. Students were instructed during training that their daily goal should be 1 to 2 word choices higher than their last CBM score. A daily goal of I to 2 additional word choices was considered to be a realistic goal based on research of goal setting in CBM. Each time a student completed a CBM assessment, he or she completed the goal setting form (see Figure 1).

The CBM Graph and Goal Line Test was used for students in the goal setting condition after students had completed CBM and goal setting training to assess student understanding of graphed performance feedback (Stecker, Whinnery, & Fuchs, 1988). This test assessed student knowledge of dates and scores of graph components, understanding goal lines and daily and long-term goals. The mean accuracy for the group was 96.7% with a range of 88% to 100%.

Control group. Students in this condition did not see a teacher-set goal line or a graph of their CBM scores, and were not taught to self-set goals. Students wrote down their score from the CBM assessment on the CBM form and gave the score to the teacher immediately following the assessment. Upon completion of the CBM assessment, the teacher signed the CBM form and the student placed the form in his or her CBM folder.

Results

After baseline data, students completed two computerized assessments per week. Treatment integrity was assessed by examining the number of completed CBM assessments. The percentage of completed CBM assessments for students in the goal setting group (M = 77.22, SD = 9.9) was not significantly different than the students in the control group (M = 82.60, SD = 10.50), t(17) = 1.43, p = .27.

Setting realistic goals. Students in the goal setting group met their self-set daily goals 38% of the time during the 7 week intervention period. Students set appropriate goals according to the training and guidelines 46% of the time and met the realistically set goals 48% of the time. Eight (89%) of the 9 students in the goal setting group met the end-of-intervention goal set by the teacher.

Effects on student understanding of their reading goals. Students were individually administered the Student Awareness of Goals and Progress Interview (SAGPI; Fuchs et al., 1989a). For this 12 item interview, no significant differences between groups were found for pretreatment. For posttreatment, significant differences were found on two items. When asked, "Does your teacher have a graph or record of how well you're doing in reading?" During the pretest session, the percentage of students who answered yes was 44% (n = 4) and 50% (n = 5) for the goal setting and control group, respectively. During posttesting, the percentage of students who responded yes was 100% (n = 9) for the goal setting group and 50% (n = 5) for the control group. Achi square test indicated a significant relation between group and response, [chi square] (1, N = 19) = 5.70, p = .02.

The other question in which a significant difference was found during posttesting was, "What would your goal be for reading tomorrow?" The pretest responses for 95% of the students were vague such as, "read a chapter", or "do better." During posttesting, 44% of the students in the goal setting group specified a specific number of words while no students in the control group gave a specific answer. A chi square test revealed a significant difference between responses and treatment conditions, [chi square] (1, N = 19) = 5.33, p = .02.

Discussion

The students in the goal setting group had difficulty setting realistic daily goals despite specific guidelines and upper limits given during training. Students only met their self-set daily goals 38% of the time and only met 48% of the goals that they set that met the goal training guidelines. These findings are supported by Graham and Harris (1989) in which the difficulty students with learning disabilities have setting realistic goals is described. The current study supports the need for ongoing training and teacher monitoring of student set goals. Without ongoing training it is likely that students will experience frustration caused by self-setting unrealistic and unobtainable goals. The current study had an intervention period of 7 weeks. A longer intervention in conjunction with ongoing training may increase students understanding of setting realistic goals. Despite the difficulty students had setting daily goals, 78% of the students in the goal setting group met the end-of-intervention goal set by the teacher.

These findings support the need for teacher involvement in CBM assessments. It may not be enough to have students independently complete CBM assessments. It appears that students need teacher feedback regarding their progress in order to make assessments understandable and more meaningful. Further discussion with the teacher would possibly be beneficial for students on completion of CBM assessments.

Although students had difficulty setting realistic goals, the posttest results of the student interview showed that significantly more students in the goal setting group were able to specifically state a specific reading goal. This study looked at whether or not the goal was specific and not vague such as "Do better." Therefore, students may be reporting specific yet unrealistic goals. This is an area that needs to be examined further through research to determine how well students understand daily goals and the relationship that daily goals have with long-term goals.

Utilizing student goal setting as a component of CBM is a feature that could possibly assist students with learning disabilities to realize that they are making progress. Students with learning disabilities often believe that they are unable to overcome their difficulties and end up feeling helpless (Licht, 1983). It is important for educators to realize that students with learning disabilities are continually experiencing failure, which often leads to the belief that they lack ability. Due to this lack of ability, students may lower their expectations for future successes (Weiner, 1979). Stone and May (2002) found that although students with learning disabilities in their study had a significantly less positive self-concept than their average-achieving peers, these same students over-estimated their academic skills. Since all students in the goal setting group recognized that his/her teacher had a graph of their reading, this information could be used to better estimate academic skills.

There are several limitations to the current study. Increasing the number of students and the length of the intervention would provide valuable information as to the benefits of goal setting training. Students need to be taught specific goal setting procedures and many students will need ongoing training in goal setting procedures. Although students had difficulty self-setting reading CBM goals, students in the goal setting condition were better able to specifically state reading goals. It is also recommended that students meet with their reading teacher at least once a week to provide feedback regarding their progress toward reading goals and discuss possible reading strategies that may need to be reinforced. Another limitation to this study is that groups were randomly assigned by teachers, not students. It is recommended that a follow-up study be completed that randomly assigns students to treatment groups.

Due to the vague responses that students initially provided when asked what their goal for reading was, it is suggested that teachers work to increase student awareness about individual long and short term goals.

References

Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 18, 19-32.

Deno, S. L. (1989). Curriculum-based measurement and special education services: A fundamental and direct relationship. In M. R. Shinn (Ed.). Curriculum-based measurement: Assessing special children (pp. 1-17). New York: Guilford Press.

Deno, S. L. (2003). Developments in curriculum-based measurement. The Journal of Special Education, 37, 184-192.

Fuchs, L. S., Bahr, C. M., & Rieth, H. J. (1989). Effects of goal structures and performance contingencies on the math performance of adolescents with learning disabilities. Journal of Learning Disabilities, 22, 554-560.

Fuchs, L. S., Butterworth, J. R., & Fuchs, D. (1989a). Effects of ongoing curriculum-based measurement on student awareness of goals and progress. Education and Treatment of Children, 12, 63-72.

Fuchs, L. S., Deno, S. L., & Mirkin, P. (1984). Effects of frequent curriculum-based measurement and evaluations on pedagogy, student achievement, and student awareness of learning. American Educational Research Journal, 21, 449-460.

Fuchs. L. S., Fuchs, D., & Deno, S. L. (1985). Importance of goal ambitiousness and goal mastery to student achievement. Exceptional Children, 52, 63-71.

Fuchs, L. S., Fuchs, D., Karnes, K., Hamlett, C. L., Katzaroff, M., & Dutka, S. (1997a). Effects of task-focused goals on low-achieving students with and without learning disabilities. American Educational Research Journal, 34. 513-543.

Fuchs, L. S., Hamlett, C. L., & Fuchs, D. (1997b). Monitoring Basic Skills Growth: Basic Reading. Austin, TX: PRO-ED.

Graham, S., & Harris, K. R. (1989). Cognitive training: Implications for written language. In J. Hughes & R. Hall (Eds.). Cognitive behavioral psychology in the schools: A comprehensive handbook. (pp. 247-279). New York: Guilford.

Johnson, L., Graham, S., & Harris, K. (1997). The effects of goal setting and self-instruction on learning a reading comprehension strategy: A study of students with learning disabilities. Journal of Learning Disabilities, 30, 80-91.

Licht, B. G. (1983). Cognitive-motivational factors that contribute to the achievement of learning-disabled children. Journal of Learning Disabilities, 16, 483-490.

Stecker, P. M., Whinnery, K. W., & Fuchs, L. S. (1988). Curriculum Based Measurement Graph Test. Unpublished instrument.

Stone, C. A., & May, A. L. (2002). The accuracy of academic self-evaluations in adolescents with learning disabilities. Journal of Learning Disabilities, 35, 370-383.

Weiner, B. (1979). A theory of motivation for some classroom experiences. Journal of Educational Psychology, 71, 3-25.

Kristine D. Swain, Ph.D., Assistant Professor, Department of Special Education and Communication Disorders, University of Nebraska-Omaha.

Correspondence concerning this article should be addressed to Dr. Kristine D. Swain, Assistant Professor, University of Nebraska-Omaha, 115 Kayser Hall, Omaha, NE 68182-0054; Email: [e-mail].

Table 1
Student Characteristics by Experimental Conditions

Condition         GS     Control   t-value   [chi square]

Age in Years
    M            12.26     12.36   0.31 *
    SD            0.41      0.64
IQ
    M            98.00    101.60   0.78 *
    SD            9.63      9.58
Reading Standard Score
    M            88.50     86.25   0.85 *
    SD            7.80      7.23
Gender                                          1.80 *
    Male (n)      8         6
    Female (n)    1         4
Grade                                            .05 *
    6 (n)         7         3
    7 (n)         2         7

* not significant

Figure 1. Students in the CBM with Goal Setting condition completed
the goal setting
form each time he/she completed a CBM assessment.

Goal Setting Form

End of year goal: 22    (set by teacher and researcher)

            Your Goal   Your Score   Goal Met
Date        for today   for today    (yes or no)

10/26       15          16           yes
10/28       17          17           yes
11/2        19          17           no
11/4        8           18           yes