Effect of drill ratios on recall and on-task behavior for children with learning and attention difficulties.

The instructional level, a measure of optimal instructional challenge, addresses the amount of review material interspersed with new. The current study further attempted to identify the instructional level for drill tasks by teaching sight-words to five fourth grade students with reading disabilities and documented attention difficulties. Four different drill ratios. 0% known. 50% known, 83% known, and 90% known, were used with a 1-week retention interval and both recall and on-task behavior being the dependent variables. Results suggested that the 90% known condition led to the highest percentage of on-task behavior and the highest retention, but required much more time to complete. Limitations and implications for future research are included.

**********

"The most effective device that can be applied to learning is to increase the amount of drill or practice" (p. 289) and efforts to improve academic motivation serve to increase the amount of practice (Symonds & Chase, 1992). Thus, scholars suggested the need for high repetition of new items during initial learning (Daly, Hintze, & Hamler, 2000: Gickling & Thompson, 1985: Tucker, 1989) and Shapiro (1996) recommended that practitioners turn to various drill formats for academic remediation.

Gickling and colleagues (Gickling & Armstrong, 1978; Gickling & Rosenfield, 1995, Thompson, Gickling & Havertape, 1983) demonstrated that students presented with an appropriate level of challenge exhibited higher task completion, task comprehension, and on-task behavior. They labeled this appropriate level of challenge the instructional level, a term first corned by Betts (1946), and theorized that it involved presenting a child with enough review material interspersed with new material to be adequately challenging without being frustrating. Most academic tasks fall into one of two categories, 1) gaining meaning from print (reading), and 2) rehearsing tasks such as mathematics computation, spelling, and sight-word recognition (drill; Gravois & Gickling, 2002). In order for a reading task to be at a child's instructional level, it should include 93% to 97% words that the child can read without effort (Gickling & Thompson, 1985). Drill tasks were theorized to be presented at an instructional level when 70% to 85% of the items have already been learned to the point that they can be recalled without effort (Gickling & Thompson, 1985). Research has consistently supported that providing reading instruction at Gickling's proposed instructional ratio improved reading skills (Burns, 2002; Gickling & Rosenfield, 1995; Shapiro, 1992; Shapiro & Ager, 1992), but the instructional level for drill tasks was not derived from empiricism and data regarding instructional ratios for drill tasks have not been conclusive.

Although research has demonstrated the need to provide instruction that is at each student's instructional level, what that level is for drill tasks has not been conclusively defined. Different ratios of known to unknown items for drill tasks have been suggested including 70% to 85% known and 15% to 30% unknown (Gickling & Thompson, 1985), 10% unknown to 90% known (Tucker, 1989), 30% unknown to 70% known (Coulter & Coulter, 1990), and 50% unknown to 50% known (Neef; Iwata. & Page, 1980). Roberts, Turco, and Shapiro (1991) suggested that the more challenging ratios such as 50% unknown and 40% unknown to 60% known resulted in more unknown sight words being acquired during drill sessions, but the 20% unknown to 80% known level was linked to better retention. Roberts and Shapiro (1996) also found that the 20% unknown to 80% known condition resulted in a higher percentage of material learned, but resulted in less total material learned compared to more challenging ratios. However, research comparing drill tasks containing 90% known, 50% known and 0% known items found that the 90% known condition led to significantly better retention (MacQuarrie, Tucker, Burns, & Hartman, 2002). Others have suggested the need to individualize the ratio of unknown to known material based on academic area (Cooke, Guzaukas, Pressley, & Kerr, 1993; Cooke & Reichard, 1996). Meta-analytic research found that drill ratios containing at least 50% known led to a strong effect, but no specific ratio within that parameter was decisively more effective than the others (Burns, 2004).

A related line of research has consistently demonstrated a link between academic learning time (ALTL "proportion of instructional time allocated to a content area during which students are actively and productively engaged in learning" (Gettinger & Seibert, 2002. p. 774), and academic outcomes (Gettinger & Stoiber, 1999). Gickling and Armstrong (1978) defined off-task behaviors as those that are irrelevant to the immediate academic task. This definition appears more consistent with procedural engagement (observable behaviors; Nystrand & Gamaron, 1991) than substantive engagement (prolonged personal commitment). However, procedural engagement has been linked to a high level of academic success (Gickling & Thompson, 1985), and increasing academic success increases ALT (Gettinger & Seibert, 2002).

Most studies (e.g. Cooke et al., 1993: Cooke & Reichard, 1996; MacQuarrie et al., 2002; Roberts, et al. 1991) that examined instructional ratios for drill tasks generally used the number of items learned and later recalled as the dependent variable. Gickling proposed that instruction provided at an appropriate level increases academic behaviors such as task completion and on-task behavior (Gickling & Thompson, 1985: Gravois & Gickling, 2002). Students identified as learning disabled increased time on task during a reading assignments from a range of approximately 30% to 70% to a range of approximately 80% to 95% (Gickling & Armstrong, 1978). However, no studies could be found in the literature that examined drill ratios using academic behaviors such as time on task as the dependent variable.

Skinner and colleagues (Cates & Skinher, 2000; Skinner, Fletcher. Wildmon, & Belfiore, 1996: McCurdy, Skinner. Grantham, Watson & Hindman, 2001) have consistently demonstrated that interspersing known, or easier, items within an assignment increased student preference for that assignment. In addition, McCurdy et al. (2001) examined the effect of interspersing easier problems every third within a mathematics worksheet had on increasing time on task as compared to a control condition. However, none of these studies compared different ratios of known, or easier tasks, to unknown items. Therefore, the current study examined the effect that drill-task instructional ratios have on academic behaviors in addition to recall. It was hypothesized that teaching students new words at a ratio of 17% unknown to 83% known, which fell within Gickling's suggested 70% to 85% known ratio, would lead to the largest number of retained items and would also lead to the highest percentage of on-task behaviors. Finally, little research has been found that compared the amount of time needed to complete various drill-task ratios. Therefore, an exploratory comparison between times to complete the different ratios was also conducted.

Method

Participants and Settings

Five fourth-grade students, three boys and two girls, receiving special education services served as the participants for the study. Each student was Caucasian, participated in special education services since second grade, and was either 9 or 10 years old. Additionally, each was administered a Wechsler Intelligence Scale for Children: Third Edition within the previous 2 years and the resulting age-based Full-Scale Quotients were all between 90 and 95. The five students were from one elementary school in a rural community in Michigan in which 58.65) of the students were eligible for the federal free or reduced lunch program. Each student had a current Individualized Educational Program (IEP) and met the Michigan criteria for Specific Learning Disability (R340.1713; Michigan State Board of Education, 1997) in basic reading skills. Further, difficulties sustaining attention during academic tasks were mentioned as an IEP goal for each student, which suggested attention difficulties during instructional activities with attempted classroom interventions. However, none of the students were taking prescription medication for attention difficulties.

Students were individually taken to a quiet area within their elementary school, but away from their classroom and peers. However, the workspace was in an area where school activity could be observed (e.g., back of the classroom or in the connecting hallway). The student and primary researcher were seated at a table across from each other, and three school psychology graduate students were seated close enough to observe the student, but far enough away so as not to be a significant distracter (approximately eight to 10 feet).

Procedure

Gickling and Thompson's (1985) initial definition of the instructional level was based on research that used academic behaviors, such as time on task, as the dependent variable. Thus, the current study observed time on-task as a dependent variable by having three school-psychology graduate students trained in behavioral observations observe each student.

An interval scoring method was used with a momentary time-sampling schedule as recommended by Hintze, Volpe, and Shapiro (2002). Observers timed the sessions while dividing them into 15-second intervals. Next the student's behavior was observed on the 15-second intervals and recorded as off-task according to the following definition: the student not "having his head and/or eyes oriented toward assigned material, an appropriate speaker, or another presentation medium" (Skinner, Rhymer, & McDaniel, 2000, pg. 23). The behavior had to be rated as off-task by two of the three observers in order to be accepted. A comparison of the three observers' ratings found over 95% agreement between the judges' rating of off-task behaviors. Because length of the conditions was not controlled, on-task behavior was converted to a percentage by counting those rated as on-task, or more accurately not rated as off-task, and dividing the total by the total number of intervals and multiplying by 100.

Semb and Ellis (1994) criticized previous recall research for not controlling prior experience with the new items or experience with the material external to the study. Therefore, words from the Esperanto International Language (Richardson, 1988) were used to control for prior and external experience, under the assumption that no external instruction in the language would be conducted. Words selected from Esperanto for use in the study were concrete nouns with five letters to control for the size and imagery level of the word. Known words were obtained from the fourth-grade list of the Fry (1980) reading list. Students were presented the Esperanto and known words before beginning to assess if each was appropriately unknown or known. Each word was written on a 3 x 5 index card and presented to the child, who was asked to verbally state the word within 2 seconds. Fry words that were correctly identified within 2 seconds were used as known words. None of the Esperanto words were correctly read or translated within 2 seconds, and thus, served as the unknown words.

Students were seen individually at their elementary school once a week for 5 weeks. Each session consisted of teaching 10 new individual Esperanto words, and the corresponding English translations, by writing the word on a 3 x 5 index card, and having the primary author present the unknown word on the card while verbally stating its English translation. The student was next asked to read the word, provide its meaning, and then use it in a sentence. Once the student correctly read the unknown word, it was rehearsed using one of the four conditions consisting of different ratios of new words and review words. The four conditions were 0% known words, 50% known (Neef et al., 1980), 83% known (Gickling & Thompson, 1985), and 90% known (Tucker, 1989). The order in which the four conditions were presented was counter balanced in order to reduce any potential order effect for the instructional conditions.

The 0% known condition contained 10 cards with all unknown Esperanto words, which were presented in sequence three times. The 50% known condition had two unknown Esperanto words and two known words, the 83% known condition had one unknown Esperanto word and five known words, and the 90% known condition contained one unknown Esperanto word and nine known words. The Esperanto words were the first two words presented in the 50% known condition, and were presented in the following sequence, A) Esperanto word 1, Esperanto word 2; B) Esperanto word 1, Esperanto word 2, known word 1; C) Esperanto word 1, Esperanto word 2, known word 1, known word 2. This procedure, called folding in (Shapiro, 1996), was also used for the other two conditions, except there was only one unknown Esperanto word presented at a time and the number of known words was five (83% unknown) or nine (90% unknown) as compared to two for the 50% known condition. The cards were presented within the following sequence:

1st Unknown, 1st Known; 1st Unknown, 1st Known, 2nd Known; 1st Unknown, 1st Known, 2nd Known, 3rd Known 1st Unknown, 1st Known, 2nd Known, 3rd Known, 4th Known; 1st Unknown, 1st Known, 2nd Known, 3rd Known, 4th Known, 5th Known; (83% known stopped here) 1st Unknown, 1st Known. 2nd Known, 3rd Known, 4th Known. 5th Known, 6th Known: 1st Unknown. 1st Known, 2nd Known, 3rd Known, 4th Known, 5th Known, 6th Known, 7th Known; 1st Unknown, 1st Known, 2nd Known, 3rd Known, 4th Known. 5th Known, 6th Known. 7th Known, 8th Known; 1st Unknown, 1st Known, 2nd Known, 3rd Known, 4th Known, 5th Known, 6th Known, 7th Known, 8th Known, 9th Known.

After completing the above sequence, the 1st Unknown word became the new 1st Known, and the previous 5th known word (83% known condition) or 9th Known word (90% known) was removed. Therefore, the total number of words remained six for the 83% known condition and 10 for the 90% known condition.

The students were individually taught 10 Esperanto words during each instructional session for four sessions with a 1-week interval between sessions. Students participated in one instructional session each day. Subsequent sessions began by testing recall of the words learned in the previous session, with the total number of new words recalled being recorded. After testing for retention. the new instructional condition began. A fifth session was conducted to test the retention of words taught in the previous (fourth) session, without teaching any new words.

The amount of time needed to complete each session was recorded starting from the first presentation of the first unknown word and ending with the last presentation of the last known word. Percentage of on-task behavior was recorded for each condition. The number of words retained and the percentage of on-task behaviors were compared between the four conditions. Finally, the amount of time needed to complete the four conditions was compared as well.

Results

Gravois and Gickling (2002) suggested that 70% to 85% known material was needed during drill tasks in order to assure adequate challenge. Thus, it was hypothesized that the 83% known condition would lead to the largest number of retained items. Figure 1 graphically displays the number of words retained by the students for each instructional condition. The 90% known (and 10% new) condition resulted in the most words retained, with only one of the five points equaling the highest data point for the other three conditions. Three students did not retain any words using the 50% known condition, and one did not retain any words from the 0% known condition as well. Therefore, data did not support the first hypothesis, but did offer support for the 90% known condition.

[FIGURE 1 OMITTED]

It was also hypothesized that the 83% known condition would lead to the highest rate of on-task behavior, given that this ratio fell within Gickling's (Gickling & Thompson, 1985) proposed ratio for drill tasks. The rate of on-task behavior for each condition is displayed in Figure 2. The 90% known condition had the highest rate of on-task behavior for all of the students, with all data points exceeding 90% on task. Further, the five data points for the 90% known condition exceeded the highest points for the other conditions. The range of on-task behavior was approximately 20 percentage points for 0% condition, 25 points for the 50% known condition, 40 percentage points for the 83% known condition, but the range was only approximately 10% on-task intervals for the 909k known condition. Thus, the data did not support the second hypothesis and in fact suggested that the 90% known condition led to the highest rate of on-task behavior.

[FIGURE 2 OMITTED]

The mean length of time needed to complete sessions for the four conditions was computed by adding the number of minutes for the condition for each student and dividing by five. Mean number of minutes for the conditions was 6.1 (SD = 1.4) for the 0% known condition, 7.9 (SD = 2.8) minutes for the 50% known condition. 13.9 (SD = 3.3) minutes for 83% known, and 27.5 (SD = 8.7) minutes for the 90% known condition.

Discussion

The current study attempted to examine Gickling's instructional level for drill tasks by using both number of words recalled and time on-task as the dependent variables. Results supported the effect of instructional ratios on recall and on-task behavior, but were not consistent with Gickling's suggestions. These data offer some support for the ratio of 10% unknown to 90% known for both recall and increasing on-task behavior, but it was the 83% known condition that fell within Gickling's hypothesized instructional level range of 70% to 85% known. The level of on-task behavior (e.g., exceeding 90% of the intervals) was equal to or somewhat higher than what Gickling and Armstrong (1978) found for reading tasks presented at the instructional level. MacQuarrie, et al. (2002) used similar procedures to examine retention and found that a 10% unknown/90% known condition led to significantly better retention than did conditions containing 50% unknown/50% known or 100% unknown. They theorized that this could have been due to an increase in opportunities to respond (OTR: Greenwood, Delquadri, & Hall, 1984) for the 10% unknown/90% known condition, but only used recall as the dependent variable. The 90% known condition for the current study involved one unknown to nine known, which compared to the other conditions, resulted in more opportunities to rehearse the given unknown word. Therefore, these current recall data seem consistent with previous research (Daly et al., 2000; Logan & Klapp, 1991: MacQuarrie et al., 2002) that demonstrated a link between OTR and recall. Increases in percentage of time on task are more difficult to explain with OTR, but it does make some intuitive sense that the condition that led to the highest rate of off-task behavior also led to the highest retention. It would seem that in order for a student to learn new material, he or she has to attend to it.

The 90% known condition took considerably longer to complete than did the other three, and the 0% known condition consistently required the shortest amount of time. Given that the 90% known condition contained 10 cards total, where the 83% unknown contained six total cards and the 50% known condition contained only four total cards, it is not surprising that the least challenging condition required the most time to complete. However, the 90% known condition required almost twice as much time, approximately 30 minutes, as the next least challenging (83% known) condition. This is somewhat inconsistent with Darch and Gersten (1985), who found that the more time spent on a task, the greater likelihood of off-task behavior because the task that took the longest also generally had the highest amount of on-task behavior.

The students who participated in the current study met the Michigan criteria for Specific Learning Disability in basic reading skills, and had documented attention difficulties. interspersing 90% known material in drill tasks was linked to increased on-task behavior and retention for these students, but as noted above, the 90% known condition also required more time to complete than the more challenging options. Although drill tasks have been used to remediate various academic difficulties with students in general and special education (Bums. 2002), practitioners should weigh the time-costs against the potential benefits for each student, considering academic and behavioral needs.

Limitations

Although the current stud,/ suggested data with potential utility and implications for future research, several limitations should be noted. This study was designed to examine the research questions in an applied setting to primarily develop areas for future research, with only cautiously suggested implications for practice. Furthermore, these data only compare the rate of on-task behavior between four conditions, but do not suggest a criterion for this variable. In essence, it can be suggested that using 90% known material in drill task can increase on-task behavior, but it is not known if the level of on-task behavior increased to an adequate or acceptable rate. Additionally, artificial stimuli were used for instruction and the instructional sessions did not take place within a classroom among other students. These measures were taken to control the variability within the instructional setting, and although controlled conditions are desirable in research, this may limit the external validity of the data.

The current study does not provide conclusive answers, but may lay groundwork for future studies. Questions that could be addressed in future research include: What level of off-task behavior is significant enough to interfere with learning, and can interventions based on drill ratios be implemented at a small-group or classroom level? Research is also needed that investigates drill ratios among different academic tasks and student disabilities. Further, future researchers may wish to compare Gickling's suggested ratios of 70% to 85% known material for drill tasks and 93% known to 97% known material for reading tasks while engaging in various drill and comprehension activities. Because the data on ratios for drill tasks has been largely inconclusive, additional research in this area, particularly with children with special needs and long-term effects, is needed. Finally, future researchers may wish to compare this intervention to reduce off-task behavior frequency and duration with other approaches currently discussed in the research literature.

References

Burns, M. K. (20047. Empirical analysis of drill ratio research: Refining the instructional level for drill tasks. Remedial and Special Education, 25, 167-175.

Burns, M. K. (2002), Comprehensive system of assessment to intervention using curriculum-based assessments. Intervention in School and Clinic, 38, 8-13.

Cates. G. L., & Skinner, C. H. (2(/02). Getting remedial mathematics students to prefer homework with 20% and 40% more problems: An investigation of the strengths of the interspersing procedure. Psychology in the Schools, 37, 339-347.

Cooke, N. L., Reichard. S. M. (19967. The effects of different interspersal drill ratios on acquisition and generalization of multiplication and division facts. Education & Treatment of Children, 19, 124-142.

Cooke, N. L., Guzaukas, R., Pressley, J. S., & Kerr. K. (1993). Effects of using a ratio of new items to review items during drill and practice: Three experiments. Education and Treatment of Children, 16, 212-234.

Coulter. W. A., & Coulter, E. M. (1990). Curriculum-based assessment for instructional design. Unpublished manuscript.

Daly, E. J., III. Hintze, J. M., & Hamler, K. R. (2000). Improving practice by taking steps toward technological improvements in academic interventions in the new millennium. Psychology in the Schools, 37, 61-72.

Darch, C. & Gersten, R. (1985). The effects of teacher presentation rate and praise on LD students' oral reading performance. British Journal of Educational Psychology, 55, 295-303.

Fry, E. (1980). The new instant word list. The Reading Teacher, 34, 284-289.

Gettinger, M., & Seibert, J. K. (2002). Best practices in increasing academic learning time. In Thomas & J. Grimes (Eds,) Bestpractices in school psychology (4'hed.; pp. 773-788). Bethesda, MD: National Association of School Psychologists.

Gettinger, M., & Stoiber, K. C. (1999). Excellence in teaching: Review of instructional and environmental variables. In C. R. Reynolds & T. B. Gutkin (Eds.), The handbook of school psychology (3rd ed.; pp.933-958). New York: John Wiley.

Gickling, E. E., & Armstrong, D. L. (1978). Levels of instructional difficulty as related to on-task behavior, task completion, and comprehension. Journal of Learning Disabilities. 11, 559-566.

Gickling, E., & Rosenfield, S. (1995). Best practices in curriculum-based assessment. In A. Thomas, & J. Grimes (Eds.7. Best Practices in School Psychology. (3rd ed., pp. 587-595). Washington D.C.: National Association of School Psychologists.

Gickling, E., & Thompson, V. (1985). A personal view of curriculum-based assessment. Exceptional Children, 52, 205-218.

Gravois, T. A., & Gickling, E. E. (2002). Best practices in curriculum-based assessment. In A. Thomas & J. Grimes (Eds.) Bestpractices in school psychology (4th ed.: pp. 885-898). Bethesda, MD: National Association of School Psychologists.

Greenwood, C. R., Delquadri, J., & Hall, R. V. (1984). Opportunity to respond and student academic performance. In W. Heward, T. Heron, D. Hill, & J. Trap-Porter (Eds.), Focus on behavior analysis in education (pp. 58-887. Columbus, OH: Charles E. Merrill Publishing.

Hintze, J. M., Volpe, R. J., & Shapiro, E. S. (2002). Best practices in the systematic direct observation of student behavior. In A. Thomas & J. Grimes (Eds.) Best practices in school psychology (4th ed.: pp. 993-1006). Bethesda, MD: National Association of School Psychologists.

Logan, G. D., & Klapp, S. T. (1991). Alphabet arithmetic: Is extended practice necessary to produce automaticity? Journal of Experimental Psychology: Learning, Memory, and Cognition, 17, 179-195.

MacQuarrie. L. L., Tucker, J. A., Burns, M. K., & Hartman, B. (2002). Comparison of retention rates using traditional, Drill Sandwich, and Incremental Rehearsal flashcard methods. School Psychology Review, 31, 584-595.

McCurdy, M., Skinner, C. H., Grantham, K., Watson, T. S., & Hindman, E. G. (2001). Increasing on-task behavior in an elementary student during mathematics seatwork by interspersing additional brief problems. School Psychology Review, 30, 23-32.

Michigan State Board of Education. (1997). Revised administrative rules for special education. Lansing. MI: Author.

Neef, N. A., Iwata, B. A., & Page, T. J. (1980). The effects of interspersal training versus high-density reinforcement on spelling acquisition and retention. Journal of Applied Behavior Analysis, 13, 153-158.

Nystrand, M., & Gamaron. A. (1991). Instructional discourse, student engagement, and literature achievement. Research in the Teaching of English, 25, 261-290.

Richardson, D. (1988). Esperanto: Learning and using the international language. El Cerrito, CA: Esperanto League for North America.

Roberts. M. L., & Shapiro, E. S. (1996). Effects of instructional ratios on students' reading performance in a regular education program. Journal of School Psychology, 34, 73-91.

Roberts, M. L., Turco. T. L., & Shapiro, E. S. (1991). Differential effects of fixed instructional ratios on students" progress in reading. Journal of Psychoeducational Assessment, 9, 308-318.

Semb, G.B., & Ellis, J.A. (1994). Knowledge taught in school: What is remembered? Review of Educational Research, 64, 253-286.

Shapiro, E. S. (1992). Use of Gickling's model of curriculum-based assessment to improve reading in elementary age students. School Psychology Review, 21, 168-176.

Shapiro, E. S. (1996). Academic skills problems. Direct assessment and intervention (2nd ed.). New York: Guilford Press.

Shapiro, E. S., & Ager, C. (1992). Assessment of special education students in regular education programs: Linking assessment to instruction. Elementary School Journal, 92, 283-296.

Skinner, C. H., Fletcher, P. A., Wildmon, M., & Belfiore, P. J. (1996). Improving assignment preference through interspersing additional problems: Brief versus easy problems. Journal of Behavioral Education. 6, 427-437.

Skinner, C. H., Rhymer, K. N., & McDaniel, E. C. (20001. Naturalistic observation in educational settings. In E. S. Shapiro & T. R. Kratochwill (Eds.) Conducting school-based assessments of child and adolescent behavior (pp. 21-54). New York: Guilford.

Symonds, E. M., & Chase, D. H. (1992). Practice vs. motivation. Journal of Educational Psychology, 84, 282-289.

Thompson, V. P., Gickling. E., & Havertape, J. F. (1983). The effects of medication and curriculum management on task-related behaviors of attention deficit disordered and low achieving peers. Monographs in behavioral disorders." Severe behavior disorders of children and youth (Series # 6). Council for Children with Behavioral Disorders, Arizona State University.

Tucker, J. A. (1989). Basic flashcard technique when vocabulary is the Goal. Unpublished teaching materials, School of Education, University of Tennessee at Chattanooga. Chatanooga, TN.

Matthew K. Burns, University of Minnesota, Vincent J. Dean, Michigan State University.

Correspondence concerning this article should be addressed to Dr. Matthew Burns, Department of Educational Psychology, College of Education and Human Development, 346 Elliott Hall, 75 East River Road, Minneapolis, MN 55455; Email: [e-mail].