Evaluation of a changing criterion intervention to increase fluent responding with an elementary age student with autism.
At the core of fluency is rate of responding (Eckert, Dunn, & Ardoin, 2006; Martens et al., 2007; Martens & Witt, 2004). During the acquisition of a skill the focus is on accurate responding, regardless of the latency to respond. The goal of acquisition is to bring responding under appropriate stimulus control of instructions (Martens & Witt, 2004). After accurate responding is acquired the next goal is to "perform an acquired skill rapidly" (p. 24). Fluency building has been applied to reading and math instruction (Codding, Eckert, Fanning, Shiyko, & Solomon, 2007; Hartnedy, Mozzoni, & Fahoum, 2005: Martens et al., 2006), although it can be applied to any acquired learning objective.
One method of increasing fluent responding is to make rewards contingent on rapid responding. Noell and colleagues (1998) examined the effects of contingent rewards on reading fluency and found that student's speed to reading increased when rewards were made contingent on an increase in correct words per minute from the previous reading session. Similar results were found by Martens and colleagues (2007) who examined the effects of contingencies on fluency. Specifically, these researchers provided students with a reading passage, some training on that passage, and a post-assessment reading passage. Rewards were contingent on a 50% increase in fluency from the first to last reading assessment.
The difficulty of reading passages was systematically increased so that students became more fluent with increasingly difficult reading passages.
The changing criterion design (Cooper, Heron, Heward, 1987) is appropriate for systematically increasing the difficulty of a subject matter by examining "the effects of reinforcement or punishment contingencies as they are applied in a graduated or stepwise fashion to a single target behavior" (p.219). Regarding fluency and contingent rewards, this design can be used to systematically increase the number of fluent responses that are needed to obtain rewards, first by conducting a baseline of fluent responses and then making the mean number of fluent responses the first criterion for obtaining rewards during intervention. The criterion can be systematically increased until the desired number of fluent responses is demonstrated. This design can be further strengthened by adding a baseline reversal so that causal statements can be made regarding the effects of the intervention on performance (Cooper et al., 1987).
The current study examined the effects of contingent rewards on building fluent responding in a student diagnosed with autism. The effects were evaluated using a changing criterion design whereby the number of consecutive fluent responses needed to earn a reward was systematically increased based on the student's performance. This intervention was conducted in a public school setting and implemented by a bachelor's level teacher's aide and supervised by a behavioral consultant attending a doctoral school psychology program.
Participant, Setting, and Roles
Frank was an 11 year old boy diagnosed with Autism, enrolled in a 4th grade at a public school. The student spent the majority of his day in a substantially separate classroom next to the 4th grade mainstream classroom. He was assigned a one-on-one aide who followed him throughout the day. The student attended the mainstream classroom for morning announcements and homeroom. He attended lunch, recess, and specials (e.g., art and music) with his typically-developing peers. Frank's special education teacher conducted 45-minutes of direct instruction (Engelmann & Carnine, 1982) for reading on a daily basis. The remainder of his time in school was spent in a quiet classroom away from his typical peers receiving discrete-trial training (Lovaas, 1987) from his personal aide. Additionally, Frank received approximately 10-hours of discrete-trial training in his home during 2-hour blocks five days per week.
The primary author was assigned as an outside behavioral consultant to Frank for 4-hours a week. The author spent this time in a single consecutive block one time per week. During these consultation sessions the author met with Frank's personal aide, regular education teacher, and special education teacher. A behavioral consultation problem-solving model (Bergan & Kratochwill, 1990) was followed. The consultant observed Frank across several activities during each visit and engaged Frank's team in problem-solving issues as they came up on a week to week basis.
During the spring semester issues related to fluent responding arose regarding Frank's performance during discrete-trial training. It was casually observed that Frank would respond to questions with a latency of five or more seconds. The data reported in this study represent the intervention designed in collaboration with school personnel to remedy this issue of slow, non-fluent responding.
The target behavior was student fluent responding to questions delivered during discrete trial training. Fluent responding was operationally defined as responding to a question within 3-seconds. A timer was used by the consultant to give feedback to the student's personal aide as to whether the student responded within 3-seconds. A stop watch was used to record latency to respond during baseline.
A data sheet was used to record the number of consecutive correct fluent responses. In the upper left corner of the sheet was a column made up of 11 boxes that extended to the middle of the sheet of paper. The first 10 boxes we numbered in order one through 10. The eleventh box was left blank so that a reward could be written in the box. Ten of these columns were lined up next to each other and another 10 of these columns were on the bottom half of the piece of paper. This allowed for 20 data recording columns per sheet of paper.
The sheet of paper was folded to indicate the number of consecutive correct fluent responses that were needed to receive the reward. For instance, if the criterion was four consecutive correct fluent responses then the paper was folded at the bottom of the block with four typed in it and folded over so that the four was followed by the reward box. This procedure of folding the paper was implemented so that one data sheet would be sufficient for the entire project.
Discrete-trial training materials involved written questions and stimuli related to calendar and time skills. For calendar skills, questions such as "What day is today?" were written on index cards. Additionally, a calendar was presented which showed all of the days of the respective month. All questions required the student to look at the calendar and answer the given question. For time skills, index cards displayed pictures of clocks at various times and the student responded to the question, "What time is it?"
Baseline. During baseline the student's personal aide engaged him in discrete trials for calendar and time related concepts. Before starting, the aide would ask the student to choose a reward he would receive for work completion. A variety of edibles and toys were available. The student would vocally choose a reward which was to be delivered after 10 responses. These reinforcers were identified by talking to the student and his parents. No formal preference assessment was conducted, which leaves open the possibility that more potent reinforcers were not identified. The aide would deliver an SD, or instruction, and wait for the student to respond. The calendar and time programs were already acquired, the student had previously reached mastery criterion for all of the questions delivered for these programs. In other words, the student demonstrated high levels of accuracy on these questions and was ready for fluency training. Following the delivery of an instruction the aide would wait for a student response. Following 10 responses she delivered the reward that the student had requested.
[FIGURE 1 OMITTED]
During baseline the consultant recorded the latency for the student to respond following the delivery of an SD, that is, the number of seconds between the instruction and the student beginning a vocal utterance. From these data the consultant determined the number of consecutive fluent responses the student was emitting.
Intervention. The point data sheet was used to indicate to the student how many consecutive fluent responses were needed to earn a reward. If the student responded to an instruction within three seconds he earned a point. When enough points were earned (based on the current specified criterion) the aide delivered the reward to the student that the student had chosen. If the student's response did not occur within three seconds the aide said, "We have to start over," crossed out the current column, and pointed to the new column of points just to the right (from the perspective of the student). "Starting over" could occur during any time a response was requested from the student during the calendar and time programs. For instance, if the student was required to make five consecutive fluent responses, if any one of the five responses was too slow, the process was started over and an additional five fluent responses were required to earn a reward and terminate work demands. In the end, each series of discrete trials ended with a string of fluent responses equal to the criterion set at the time. For ease of implementation the student's personal aide was trained to evaluate whether a response was fluent by counting in her head. Prior to the intervention phase the consultant met with the personal aide and the aide was instructed to look at a stop watch and count along with the stop watch at a sub-vocal level (i.e., in her head). This was to ensure that she could judge whether a response was fluent to the best of her ability without having to start and stop a stopwatch on each trial. For implementation, the aid would deliver the SD and begin counting to three in her head. If the student responded before three then the child earned a point for emitting a fluent response.
[FIGURE 2 OMITTED]
A changing criterion design was used to evaluate the intervention and increase the number of consecutive fluent responses required to attain a reward. The criterion for both programs, calendar and time, began at three consecutive fluent responses. The criterion was raised by one following three consecutive sessions of reaching criterion. A session was a series of trials that resulted either in "starting over" or in a reward. At the start of intervention a session lasted a maximum of three trials since the criterion was set at three. To move up three sessions at criterion were required, or 9 trials. As the criterion moved up, mastering a criterion became more difficult. For instance, when the criterion was set at eight, 64 consecutive fluent responses across three sessions was needed in order to move the criterion up to 9. Since the calendar and time programs were already acquired, no criterion was set for moving the criterion down should the student experience difficulty with a particular criterion. The goal of the intervention was to increase fluent responding to 12 consecutive fluent responses, which is two more responses in a session than were previously targeted. As an additional control a baseline reversal was conducted following successful intervention behavior.
Figures 1 and 2 display the results from this intervention. Baseline was only conducted on the calendar program. The student averaged 2.24 consecutive fluent responses during baseline. The initial criterion was set at three consecutive responses for both the calendar and time program. With the calendar program the student quickly acquired up to five consecutive fluent responses. When the criterion was between five and eight the student experiences some difficulty in reaching criterion during sessions. However, the student quickly acquired 9 through 12 fluent responses with the calendar program. During the 55 sessions at criterion of 12, only two times did Frank not respond fluently at criterion. Following stable responding at 12 consecutive fluent responses a baseline reversal was implemented. During the return to baseline fluent responding dropped to baseline levels. The intervention was reimplemented and Frank again attained stable responding at 12 consecutive fluent responses. Throughout the intervention, Frank averaged 96.9% accuracy in responding to calendar-related questions. The high level of accuracy indicates that the intervention targeting fluency did not adversely impact acquisition.
Figure 2 displays the results for the time program. No formal baseline was conducted for this program, however reports from the aid and observations of the consultant indicated that Frank did not respond fluently to time questions prior to the intervention. The number of consecutive fluent responses for this program began at three, which was the same as for the calendar program. The student experienced some difficulty at first with the intervention but quickly acquired four and five consecutive fluent responses. Frank experienced some difficulty when the criterion was set at six but readily acquired seven through 12 consecutive fluent responses. After the intervention was in place for some time the intervention was withdrawn. Similar to the results with the calendar program, consecutive fluent responses decreased in the absence of the intervention. The intervention was re-implemented and Frank again demonstrated 12 consecutive fluent responses during the time program. Throughout the intervention, Frank averaged 98% accuracy in responding to time-related questions.
Overall the intervention, that is, placing a contingency on fluent responding, was successful in increasing consecutive fluent responses in a elementary school-aged student diagnosed with Autism. Before the intervention the child was responding to 10 consecutive questions in a program and reinforcement was contingent on making 10 responses, regardless of how quickly the student responded to questions. This resulted in slow, but accurate responding. From the perspective of the instructional hierarchy, this type of responding is suitable for acquisition but is not sufficient for maintenance and generalization (Martens & Witt, 2004). The intervention targeted fluency, with the goal to increase the rate of responding. The intervention structured reinforcement to be contingent on quick responding. This was accomplished by lowering the number of consecutive fluent responses to an attainable level. Using a changing criterion design the number of consecutive fluent responses was systematically increase and moved beyond 10 consecutive responses--which is the number of trials run per program during baseline--to 12 consecutive fluent responses. This study joins other studies in demonstrating the positive effects of contingent rewards on increasing fluency (Martens et al., 2007; Noell et al., 1998).
The student acquired fluent responding rather quickly. According to the instructional hierarchy (Martens & Witt, 2004), this is probably due to the fact that the student had previously acquired the skills that were targeted for fluency training. Responding matched the criterion throughout most of the intervention. For calendar skills Frank experienced some difficulty with acquiring 6, 7, and 8 consecutive fluent responses and experienced difficulty with acquiring 6 consecutive fluent responses with the time program. However, during these times the criterion was not lowered to a more attainable goal, which is one variant of the changing criterion design (Cooper, Heron, Heward, 1987). Before and during the intervention Frank demonstrated high levels of accuracy during these two programs. The decision to not lower criterion was made through consultation with Frank's educational team (e.g., personal aide, special education teacher, etc.) who believed he was capable of achieving each goal set. Additionally, since the requirement to terminate questions was to reach the criterion, for each sitting the student demonstrated the current criterion. On the graph, consecutive data points below criterion indicate that the correction procedure, "starting over," was implemented multiple times before criterion was met. Even with data points below criterion, the student was regularly demonstrating criterion-level responding.
The changing criterion design coupled with a baseline reversal design added experimental control that allows for causal statements to be applied between the application of the intervention and the positive effect on consecutive fluent responses. With this design it is appropriate to attribute the positive effects of the dependent variable (number of consecutive fluent responses) to the independent variable (intervention) in that the intervention increased fluent responding.
This project benefits from the fact that it was conducted, almost solely, by school personnel in the student's natural learning environment. This was accomplished using relatively few outside resources. The aide who conducted the intervention was not extensively trained in applied behavior analysis, data collection, or research design. The planning of this intervention was the result of consultation with a professional with master's level training in applied behavior analysis who was currently in a doctoral program for school psychology. Through this collaboration, an intervention was designed that was easy to implement in the student's natural learning environment using school personnel who were already working with the student. The results of the intervention were verified through observation by the consultant each week, although no formal treatment integrity measure was completed.
Unfortunately, the evaluation of these results are limited by the absence of formal treatment integrity data and inter-observer agreement data (IOA). When the consultant observed the implementation of the intervention no formal data were collected to verify two important variables. The first variable is treatment integrity. No formal data were collected that evaluated the percentage of correct implementation of intervention procedures. However, case notes reveal that there was little need for corrective feedback to be provided to the aide regarding this program, indicating that in general, it was implemented correctly. The second variable is IOA. No independent observations of correct/incorrect and fluent/non-fluent responding was gathered to be compared with the student's aide's recording of data. Although the consultant observed the implementation of the treatment, no formal measures of reliability were obtained. Treatment integrity and IOA are important aspects of research and serve as a major limitation to the current investigation. Despite the limitations, this data set provides further support that placing a contingency on fluent responding can be done effectively with a child with autism during the instructional school day utilizing available school personnel.
Bergan, J. R. & Kratochwill, T. R. (1990). Behavioral consultation and therapy. New York: Plenum Press.
Codding, R. S., Eckert, T. L., Fanning, E., Shiyko, M., & Solomon, E. (2007). Comparing mathematics interventions: The effects of cover-copy-compare alone and combined with performance feedback on digits correct and incorrect. Journal of Behavioral Education, 16, 125-141.
Cooper, J. O., Herson, T. E., & Heward, W. L. (1987). Applied behavior analysis. New York: Macmillan Publishing Company.
Eckert, T. L., Dunn, E. K., & Ardoin, S. P. (2006). The effects of alternate forms of performance feedback on elementary-aged students' oral reading fluency. Journal of Behavioral Education, 15, 149-162.
Engelmann, S., & Carnine, D. (1982). Theory of instruction: Principles and applications. New York: Irvington Publishers.
Hartnedy, S. L., Mozzoni, M. P., & Fahoum, Y. (2005). The effects of fluency training on math and reading skills in neuropsychiatric diagnosis children: A multiple baseline design. Behavioral Interventions, 27-36.
Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3-9.
Martens, B. K., Eckert, T. L., Begeny, J. C., Lewandowski, L. J., DiGennaro, F. D., Montarello, S. A., Arbolino, L. A., Reed, D. D., & Fiese, B. H. (2007). Effects of a fluency-building program on the reading performance of low-achieving second and third grade students. Journal of Behavioral Education, 16, 39-54.
Martens, B. K., & Witt, J. C. (2004). Competence, persistence, and success: The positive psychology of behavioral skills instruction. Psychology in the Schools, 41, 19-30.
Noell, G. H., Gansle, K. A., Witt, J. C., Whitmarsh, E. L., Freeland, J. T., LaFluer, L. H., Gilbertson, D. N., &
Northup, J. (1998). Effects of contingent reward and instruction on oral reading performance at differing levels of passage difficulty. Journal of Applied Behavior Analysis, 31, 659-663.
The May Institute, Inc.
The May Center for Education and Neurorehabilitation
596 Summer St.
Brockton, MA 02302
Illinois State University
Department of Psychology
Campus Box 4620
Normal, IL 61790-4620