Printer Friendly

Monitoring children's growth in early literacy skills: effects of feedback on performance and classroom environments.


The study examined the benefits of providing kindergarten teachers with feedback about students' performance on early literacy progress-monitoring probes. Students were administered the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) in fall, winter, and spring; classroom environment was evaluated using the Early Language and Literacy Classroom Observation (ELLCO) in fall and spring. Teachers received either (a) specific information about students' performance on the DIBELS or (b) descriptive information about the DIBELS without performance feedback. Students whose teachers received feedback (n=55) made greater improvements on two DIBELS subtests compared to students whose teachers did not receive feedback (n=48). Feedback did not lead to greater change in classroom environment. Implications for using progress-monitoring to promote early literacy skill development are discussed.


Children enter kindergarten with a wide range of early literacy skills, such as phonemic awareness and letter naming, which are strongly predictive of later success in reading (Burns, Griffin, & Snow, 1999; Neuman & Dickinson, 2001). Most children develop skills rapidly during their first 2 years of school. For some children, however, early reading skill deficits at the beginning of kindergarten tend to remain, or even worsen, throughout elementary school Quel, 1988; Scarborough, 1998; Simmons, Kame'enui, Coyne, & Chard, 2002; Snow, Burns, & Griffin, 1998). Children who are poor readers at the end of elementary school are most often those who fail to show typical progress in developing early literacy skills during kindergarten and first grade.

In recent years, Responsiveness to Intervention (RTI) has been recommended as an approach for identifying children who may be at risk for reading problems (Fuchs & Fuchs, 2006). RTI is an alternative to a discrepancy model, or so-called "wait and fail" approach, in which students must fall significantly behind in reading before they receive special services. RTI hinges on the use of systematic progress-monitoring to (a) collect information about how children are performing, and (b) enable teachers to respond with well-targeted instruction and individualized support as soon as delays are evident. The major premise of RTI is that intervening early can prevent the development of serious academic problems. Recently, researchers have developed a model known as Recognition and Response (R&R) which adapts RTI to early childhood education (Coleman, Buysse, & Neitzel, 2006). This model assumes that once teachers recognize when children are falling below expected benchmarks, they will respond in ways to promote children's success, such as making adjustments in the literacy environment or providing additional assistance as needed. The aim of the current study was, in part, to test this critical assumption underlying RTI.

Monitoring Growth in Early Literacy Skills

A key element of RTI is the use of appropriate progress-monitoring tools to identify children who fail to make adequate progress in acquiring early literacy skills and, as a result, may be at risk for reading problems. Most RTI implementation models rely on two cycles of progress-monitoring (Good, Gruba, & Kaminski, 2002). The first cycle involves measuring periodically (three times per year) skills that predict future reading achievement including phonemic awareness, alphabet knowledge, and alphabet code (or phonics) fluency (Casey & Howe, 2002). Typically, all students participate in periodic progress-monitoring which uses benchmark assessments to identify children who are at risk for reading difficulty and to evaluate the overall effectiveness of classroom instruction. The second cycle of progress-monitoring measures the same skills and often uses the same assessment tools; however, only students who fall below benchmarks based on periodic progress-monitoring are included. This type of progress-monitoring occurs with greater frequency (up to twice per week) and is designed to help educators provide individualized instruction and monitor the progress of at-risk students toward important reading goals. For purposes of the current study, our focus was on the use of periodic benchmark assessments for all children.

The Dynamic Indicators of Basic Early Literacy Skills (DIBELS) (Good & Kaminski, 2002) are a set of short, individually-administered probes used for evaluating children's growth in early literacy skills for both benchmark assessments and individual progress-monitoring. Each DIBELS subtest is designed to assess a skill that predicts future reading performance and is functionally related to reading acquisition. The DIBELS benchmark probes can be administered three times each year, with criteria to help identify individual students who require greater support. Research has shown that the DIBELS have adequate reliability and predictive validity for use in school settings (Good et al., 2002; Kaminski & Good, 1996). Furthermore, data from a large national sample have been used to construct a rubric of risk-level cut points (DIBELS benchmarks) to guide educational decision-making (Good, Kaminski, Smith, Kame'enui, & Wallin, 2003). The cut points for each testing period are designed to reflect a distribution whereby 60-80% of students are "at benchmark," 15-20% of students have "some risk," and 5-15% are "at risk" for reading problems.

The benefit of using a progress-monitoring tool such as DIBELS lies in the extent to which teachers use the data to inform their classroom practices. Only a handful of studies have directly investigated the utility of progress-monitoring for purposes of modifying instruction, and nearly all research has examined the benefits of frequent progress-monitoring rather than periodic benchmark assessment. To date, the results of these studies have been mixed (see Scott, Vitale, & Masten, 1998; Stecker, Fuchs, & Fuchs, 2005 for reviews). For example, Stecker and Fuchs (2000) documented improvement in academic performance when teachers used information from curriculum-based measurement (CBM) to provide students with instruction based on their individual needs. Fuchs, Fuchs, Hamlett, Phillips, and Karns (1995), however, found that making instructional adaptations based on CBM performance depended heavily on the provision of structured recommendations from consultants, prompts to use adaptations, and ongoing support for implementation. Most recently, Graney and Shinn (2005) found that students whose teachers received feedback about CBM performance made no more significant reading gains than students whose teachers received no feedback. Moreover, when teachers received individual feedback, students actually made less progress over time than students whose teachers received aggregated group feedback or no feedback at all. Graney and Shinn hypothesized that teachers may have either relaxed their instructional intensity for students who were making adequate progress or "given up" on children who repeatedly made little progress.

Overall, research shows that the use of progress-monitoring procedures, such as CBM, has the potential to help teachers make changes in their classroom instruction to promote academic progress among students. Using CBM alone, however, is likely to be less effective for improving students' performance than using information in combination with specific instructional recommendations and consultation. Teacher training and support (e.g., resources, time) may be critical for ensuring that progress-monitoring information leads to high-quality instructional adaptations for children whose progress is inadequate (Scott et al., 1998; Stecker et al., 2005).

Classroom Environment and Early Literacy Development

Kindergarten classrooms provide critical contexts for the development of early literacy skills, especially among children who may be at risk due to language or environmental factors (Whitehurst & Lonigan, 1998). By focusing on classroom variables that promote early literacy development, kindergarten teachers have the opportunity to support reading achievement for all children. Many factors combine and interact to create a literacy-rich classroom environment. Collectively, literacy-rich environmental variables include (a) structural components, (b) language opportunities and exposure, and (c) classroom management strategies.

Structural components include teachers' organization of classroom materials, supplies, and furniture, all of which have a significant impact on the quality of the literacy environment (Searfoss, Readence, & Mallette, 2001). A teacher's choice of literacy tools and activities influences the way in which children learn information as well as the specific skills they acquire. For example, the availability of writing utensils and aids, paper, and books allows children to independently and routinely incorporate literacy activities into their day. Likewise, spreading literacy-related materials around the room prompts children to use their reading and writing skills in many situations. Finally, designated work spaces that include seating, books, and writing materials allow children to feel comfortable engaging in literacy-related activities.

Two aspects of language in early literacy classrooms are the opportunities for teacher-student interaction and exposure to print (Searfoss et al., 2001). Opportunities for teacher-student interactions in kindergarten classrooms abound during shared book reading, small-group activities, and free-play periods. According to Massey (2004), one-third of adult-child interactions should purposefully challenge and extend young children's language, such as explaining new vocabulary and encouraging children to make predictions about stories. Teachers also create a print-rich environment by providing exposure to diverse types of print throughout the classroom (Makin, 2003). Posting daily schedules, labeling important objects, and displaying printed directions help children understand the different functions of print and the need to rely on written language to proceed through daily routines.

Finally, effective classroom management comprises the third dimension of literacy-rich environments, even though it does not relate directly to early literacy development. Although a comprehensive review of management strategies is beyond the scope of this paper, research demonstrates that effective literacy teachers make their behavioral expectations clear for all literacy and learning activities. They also teach, facilitate, and strengthen appropriate classroom behavior through consistent reinforcement and feedback (Kame'enui & Simmons, 1990).

Despite knowledge of variables that influence the development of children's early reading skills, little is known about the relationship between performance on early literacy probes and observable aspects of classroom environments. One study examined the link between the classroom literacy environment {as measured by the English Language Learner Classroom Observation Instrument) and DIBELS performance, specifically among English Language Learners (ELLs) (Graves, Ger-sten, & Haager, 2004). Graves et al. found a moderately strong correlation (r = .65) between the quality of the classroom environment and growth in early literacy skills among 186 first-grade ELLs in 14 classrooms. Another recent study examined early literacy skill growth among 69 preschoolers using Individual Growth and Development Indicators (IGDIs) (Missal, McConnell, & Cadigan, 2006). The percentage of time children were engaged in literacy-related learning situations, as measured by the Ecobehavioral System for the Complex Assessment of Preschool Environments (ESCAPE; Carta, Greenwood, & Atwater, 1985), was correlated with their rate of growth on three IGDIs, specifically Picture Naming, Rhyming, and Alliteration. The authors reported correlations between ESCAPE variables and IGDI growth rates ranging from .31 to .71 across four samples of children with varying levels of risk for reading problems.

Purpose of Study

In sum, the use of systematic and periodic progress-monitoring in kindergarten classrooms has begun to receive significant attention due to recent federal legislation and a growing interest among educators in promoting early literacy development to prevent later reading problems. Although a number of models have provided guidelines for the use of periodic progress-monitoring within the context of a multi-tiered approach, relatively little attention has been given to how such data are actually used by classroom teachers.

The current study was designed to examine the extent to which kindergarten teachers use information about children's periodic progress to promote early literacy skill development. Specifically, the study sought to address three questions. First, does providing teachers with specific feedback about students' performance on early literacy benchmark probes (DIBELS) lead to greater progress on the DIBELS compared to students of teachers who receive no feedback? Second, does the perceived usefulness of periodic progress-monitoring information differ between teachers who receive feedback about DIBELS performance and those who do not? And, finally, does providing teachers with feedback about student performance lead to greater changes in the overall quality of the literacy environment than no feedback? There is some evidence that having access to progress-monitoring data for all students is more useful than not having such information (Stecker & Fuchs, 2000). Thus, for each question, we predicted that feedback about student performance would lead to higher performance on DIBELS probes, higher ratings among teachers concerning the usefulness of periodic progress-monitoring data, and more positive changes in the overall literacy environment.



Participants included kindergarten teachers (N = 8) and their students (N = 103) in eight classrooms. Contacts were made via e-mail with building principals, who facilitated the recruitment of individual teachers. Teachers from two public (n = 6) and two private (n = 2) schools located in rural or suburban communities in the Midwest agreed to participate in the study. All participating teachers were Caucasian females. According to teacher reports, a system for monitoring children's early literacy skills was not being used in any kindergarten classroom during the time of the study.

Consent forms were distributed to parents of all students in the participating classrooms. The return rate ranged from 75% to 100% across classrooms. In all, parental consent was given for 135 children. Students were excluded from the final sample if they (a) had incomplete DIBELS data due to absence or administration errors, (b) were not proficient in English, or (c) refused to participate. Based on these criteria, 32 students were excluded from the study, resulting in a final sample of 103 children (76% of the original sample). Approximately 80% of the child participants were Caucasian, and fewer than 25% qualified for subsidized school lunch; 60% of the children were female. The eight classrooms (two private school and six public school classrooms) were non-systematically assigned to either the Feedback (n = 55 students) or No Feedback (n = 48 students) condition. Each condition included one private school classroom and three public school classrooms.


Three measurement procedures were implemented over a nine-month period (September through May). Procedures were used to (a) periodically monitor children's progress in early literacy skill development using the Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good & Kaminski, 2002), (b) evaluate classrooms in terms of key components of literacy-rich environments using the Early Language and Literacy Classroom Observation toolkit (ELLCO; Smith, Dickinson, Sangeorge, & Anastasopoulos, 2002), and (c) solicit teacher input regarding the utility of progress-monitoring data using an informal survey developed for this study.

Dynamic Indicators of Basic Early Literacy Skills. The DIBELS benchmark probes included four measures. The first measure, Letter Naming Fluency (LNF), requires children to name as many printed upper- and lower-case letters as possible in 1 minute. The score is the total number of letters named correctly in 1 minute (possible range = 0-110). The second measure, Initial Sound Fluency (ISF), is administered by asking students to select a picture that begins with a target sound after the examiner says the names of the pictures (12 items), or to produce the beginning sound of a pictured object that is named by the examiner (4 items). The total score depends on both the number of correct responses (0-16) and the speed with which the student responds (0-80 s). The score is calculated as follows: (60 x number correct) / seconds. Although ISF is designed for administration during the fall and winter for kindergarten students (Good & Kaminski, 2002), the current study included a spring administration to obtain a measure of growth across the school year. The third measure is Phoneme Segmentation Fluency (PSF). This task is administered by asking students to segment a word into individual phonemes or sounds. The total score is the number of correct phonemes given in 1 minute (possible range = 0-72). PSF is designed for administration with kindergarten children during the winter and spring; however, the current study included a fall administration to obtain a measure of growth across the school year. Finally, Nonsense Word Fluency (NWF) is administered by having students read nonsense words, or say individual letter sounds, as quickly as possible. The overall score is the number of correct phonemes given in 1 minute (range = 0-145). Similar to PSF, the NWF task is designed for administration during the winter and spring in kindergarten; however, we included a fall administration to examine growth across the school year.

Early Language and Literacy Classroom Observation. The ELLCO is an observation tool for assessing the quality of preschool and kindergarten literacy environments. It measures multiple variables (structural, language, management) to provide a picture of the overall quality of a classroom in relation to early literacy skill development. Available research indicates acceptable construct validity and inter-rater agreement for the ELLCO, as well as utility for measuring environmental changes across time (Smith et al., 2002).

The ELLCO yields a total maximum score ranging from 15 to 124 distributed across three parts: (a) Literacy Environment Checklist (LEC), (b) one 40-minute classroom observation and follow-up teacher interview, and (c) Literacy Activities Rating Scale (LARS). Consistent with the author guidelines, the LEC was completed when children were not present in the classroom. The checklist focuses on classroom organization and materials and can be completed in about 10 minutes. It consists of 24 items that are scored using either a yes-no format (e.g., "Is an area set aside just for book reading?") or a rating indicating the number of literacy materials available (e.g., "How many varieties of teacher dictation are on display in the classroom?"). The total score on the checklist ranges from 1 to 41.

The classroom observation occurred during 40 minutes of early literacy instruction in each classroom (i.e., structured, teacher-directed time focusing on literacy activities). Six items on the observation protocol reflect aspects of the general classroom environment (e.g., organization of the classroom, classroom management strategies), and eight items relate specifically to language development and literacy instruction (e.g., oral language facilitation, approach to book reading). Each item is scored using a five-point scale (1 = Deficient, 3 = Basic, 5 = Exemplary), for a total possible score ranging from 14 to 70. Descriptive criteria are provided for scores of 1, 3, and 5; observers are encouraged to reserve ratings of 2 or 4 for situations in which evidence exists for multiple ratings. The teacher interview occurred within 1 to 2 days following the observation. The interview protocol consists of six questions designed to clarify the observation and gather additional information to assist in the scoring of observation items.

Finally, the LARS was completed at the end of each observation. Five items on the LARS focus on book reading, using either a yes-no format (e.g., "Did you observe an adult engaged in one-to-one book reading or small-group book reading?") or a 0-2 rating (e.g., "What was the total number of minutes spent on full-group book reading?"). Four LARS items focus on writing activities, using either a yes-no format (e.g., "Did an adult model writing?") or a 0-2 rating (e.g., "How many times did you see an adult help a child write?"). The nine items on the LARS yield a total possible range of scores from 0 to 13.

Teacher survey. An informal survey was given to teachers in May, at the conclusion of all data collection. The survey included eight items for teachers in the Feedback condition and seven items for teachers in the No Feedback condition. Five items requested ratings of the (a) usefulness of feedback for modifying the classroom environment, (b) usefulness of feedback for modifying instructional practices, (c) link between DIBELS probes and the curriculum, (d) accuracy of predicting student performance, and (e) likelihood of continuing to use the probes. Survey items were rated on a Likert scale ranging from 1 to 5, with low scores reflecting low perceptions of usefulness. Teachers in both conditions responded to two open-ended questions asking them to identify the strengths and drawbacks of the study. Finally, teachers in the Feedback condition were asked (a) whether or not knowledge of students' performance affected their classroom environments or instruction, and (b) to explain their reasons for using or not using DIBELS information to make changes.


The four DIBELS probes were administered at three times during the school year - fall (September), winter (January), and spring (May). Each session used alternate forms of the probes, which were administered individually by either the first author or a trained tester. Testers, including the first author, were 10 graduate students in school psychology with experience conducting individual assessments with young children. All testers received a 30-minute training session that included a review of the DIBELS procedures and practice in administering individual probes. Testers other than the first author were unaware of which classrooms were assigned to the Feedback and No Feedback conditions.

A subset of student participants (20%) was randomly selected from each classroom at the outset of the study; their probes were simultaneously double-scored for all measurement times. Complete data were available for 15% of ISF administrations and 18% of LNF, PSF, and NWF probes. The agreement rate was calculated for ISF by dividing the lower total score by the higher score. For LNF, PSF, and NWF, agreement was calculated based on the number of items for which there was agreement divided by the total number of items. Mean agreement estimates reflected an adequate level of inter-rater agreement: 86% for ISF (ranging from 34% to 100%), 96% for LNF (64% to 100%), 90% for PSF (51% tol00%), and 89% for NWF (63% to 100%).

Within 1 week following each testing period, the four classroom teachers assigned to the Feedback condition were given specific feed-back regarding their students' performance on the DIBELS probes. The first feedback meeting included (a) an explanation of the DIBELS subtest, what they attempted to measure, and how the benchmarks were established; and, (b) sharing of information regarding individual students' performance. Teachers also received a packet of results indicating each child's performance and his/her level of risk on each DIBELS subtest. The winter and spring feedback meetings included only the data-sharing portion of the original meeting. By design, the feedback meetings did not include discussions or guidelines about using progress-monitoring data to modify instruction or to make changes in the classroom environment. This design feature was intended to preserve a focus on measuring the effects of performance feedback only, without the influence of ongoing support or recommendations for teachers. Following the spring testing (and prior to completing the survey), teachers in the No Feedback condition participated in a meeting with the first author which included a description of the types of information provided by the DIBELS and a set of mock feedback. The purpose of this meeting was to simulate a feedback session as a reference for completing the teacher survey.

Classroom observations using the ELLCO were conducted twice during the year, prior to the fall feedback meetings and following the spring feedback meetings. All classroom observations were conducted independently and simultaneously by two observers (either the first author and/or a trained observer) to establish inter-rater agreement. Disagreements in scoring were resolved through discussion, and consensus coding was used to derive a final rating. Observers were seven graduate students in school psychology with experience conducting classroom observations. All observers participated in a 30-minute training session that included a detailed, step-by-step explanation and practice with the observation protocol. Observers were not informed of which classrooms were assigned to the Feedback and No Feedback conditions.


We predicted that students whose teachers received feedback would make differentially greater gains on DIBELS probes compared to students whose teachers did not receive feedback. To test this prediction, DIBELS scores of students in the Feedback versus No Feedback conditions were compared using a 2 x 3 analysis of variance (ANOVA) with condition (Feedback versus No Feedback) as a between-subjects variable and measurement time (Fall, Winter, and Spring) as a within-subjects variable on each probe. For within-group and interaction effects, Wilks' A was transformed to an F statistic and used to establish significance; for between-group effects, the F statistic was derived through typical ANOVA procedures.

DIBELS Performance in Feedback and No Feedback Conditions

Figures 1 through 4 display graphs showing performance among children in the Feedback and No Feedback conditions at each testing time. There were no significant main effects for feedback on DIBELS scores. In partial support of our prediction, however, the Condition x Measurement Time interaction was significant for two of the four probes: (a) LNF, F (2,100) = 7.33, p < .001, [[eta].sup.2] = .13, and (b) PSF, F (2,100) = 4.42, p < .02, [[eta].sup.2] = .08. Each significant interaction effect is depicted in Figures 1 and 3. Figure 1 shows the differential gain in the number of letters named correctly in one minute (LNF) among children in the Feedback condition, and Figure 3 depicts the differential gain in the number of correct phonemes in one minute for children in the Feedback versus No Feedback condition.





Follow-up univariate analyses were conducted to evaluate group differences in fall-to-spring gains on each DIBELS probe. The improvement from September to May was significantly higher for children in the Feedback classrooms on three of four probes: (a) LNF, F (1,101) = 12.96, p < .001, [[eta].sup.2] = .11; (b) ISF, F (1,101) = 5.60, p < .02, [[eta].sup.2] = .05; and, (c) PSF, F (1,101) = 8.02, p < .006, [[eta].sup.2]= .07. As shown in Figure 4, change in NWF performance did not differ significantly between the two conditions, F (1,101) = 1.07, p > .05, [[eta].sup.2] = .01.

DIBELS Performance during Fall, Winter, and Spring Testing

Table 1 summarizes children's performance on the DIBELS probes across fall, winter, and spring testing periods. The data presented in Table 1 reveal an increasing trend in performance over time among students in the Feedback and No Feedback conditions, which is expected given the level of maturation and development of early literacy skills that typically occurs during kindergarten. Specifically, there was a significant main effect for time on each DIBELS probe: (a) LNF, F (2,100) = 165.24, p < .001, [[eta].sup.2] = .77; (b) ISF, F (2,100) = 41.55, p < .001, [[eta].sup.2] = .45; (c) PSF, F (2,100) = 116.49, p < .001, [[eta].sup.2] = .70; and, (c) NWF, F (2,100) = 98.52, p < .001, [[eta].sup.2] = .66. Follow-up, pairwise comparisons revealed significant fall-to-winter improvement (p < .001) on each probe. Students evidenced significant winter-to-spring gains (p < .001), however, on only two probes, PSF and NWF. Interestingly, there was no improvement in children's LNF scores (p = .17), and children demonstrated a significant decline in performance on the TSF probe (p < .001) from winter to spring.
Table 1
DIBELS Performance for Feedback and No-Feedback Conditions Across Time

Literacy Skill: Feedback No Condition x
 M(SD) Feedback Time
 M (SD) (F value)

Letter Naming
Fluency (LNF) (a)
 Fall (Sep) 18.1 (12.1) 22.4 (15.0)
 Winter (Jan) 35.0 (14.9) 34.2 (16.5) 7.33 **
 Spring (May) 38.5 (15.4) 34.7 (17.1) [[eta].sup.2] = .13

Initial Sound
Fluency (ISF) (b)
 Fall (Sep) 11.7 (8.5) 13.3 (8.3) 2.89
 Winter (Jan) 22.5 (9.4) 20.7 (10.5) [[eta].sup.2] = .06
 Spring (May) 18.7 (8.6) 16.1 (6.7)

Fluency (PSF)
 Fall (Sep) 9.6 (9.4) 12.0 (14.5) 4.42 *
 Winter (Jan) 20.2 (10.2) 18.9 (11.4) [[eta].sup.2] = .08
 Spring (May) 35.1 (9.3) 29.0 (15.1)

Nonsense Word
Fluency (NWF) (d)
 Fall (Sep) 7.4 (6.5) 6.7 (8.9) .59
 Winter (Jan) 19.6 (12.8) 17.1 (14.3) [[eta].sup.2] = .01
 Spring (May) 27.4 (15.7) 23.9 (16.0)

* p < .05
** p < .001

Whereas Table 1 presents the average scores for literacy skill probes, Table 2 summarizes children's performance relative to DIBELS benchmark criteria. Specifically, Table 2 reports the percentage of students in the Feedback and No Feedback conditions who fell within each risk category based on DIBELS benchmarks. According to the DI-BELS authors, typical classroom scores should reflect about 60-80% of students "at benchmark" (or displaying skills that are "established"), with an additional 15-20% having "some risk" ("emergent" skills) and 5-15% "at risk" ("deficit" skills). Using these normative guidelines, the fall performance patterns shown in Table 2 reflect a distribution of scores for both Feedback and No Feedback classrooms that is considered acceptable. At least 60% of students met or exceeded the benchmark criteria for ISF (62% and 71%) and for LNF (76% and 77%) performance.
Table 2
DIBELS Performance Relative to Fall, Winter, and Spring Benchmarks
across Feedback and No-Feedback Conditions

 ISF (a)

 % Established % Emerging % Deficit

Fall Benchmark Distribution:
 Feedback (n = 55) 62 25 13
 No-Feedback (n = 48) 71 23 6

Winter Benchmark Distribution:
 Feedback (n = 55) 38 56 6
 No-Feedback (n = 48) 25 65 10

Spring Benchmark Distribution:
 Feedback (n = 55) -- -- --
 No-Feedback (n = 48) -- -- --


 % Benchmark % Some Risk % At-Risk

Fall Benchmark Distribution:
 Feedback (n = 55) 76 18 6
 No-Feedback (n = 48) 77 19 4

Winter Benchmark Distribution:
 Feedback (n = 55) 71 20 9
 No-Feedback (n = 48) 69 21 10

Spring Benchmark Distribution:
 Feedback (n = 55) 42 36 22
 No-Feedback (n = 48) 35 27 38

 PSF (a)

 % Established % Emerging % Deficit

Fall Benchmark Distribution:
 Feedback (n = 55) -- -- --
 No-Feedback (n = 48) -- -- --

Winter Benchmark Distribution:
 Feedback (n = 55) 64 24 13
 No-Feedback (n = 48) 56 23 21

Spring Benchmark Distribution:
 Feedback (n = 55) 49 49 2
 No-Feedback (n = 48) 44 35 21

 NWF (a)

 % Benchmark % Some Risk % At-Risk

Fall Benchmark Distribution:
 Feedback (n = 55) -- -- --
 No-Feedback (n = 48) -- -- --

Winter Benchmark Distribution:
 Feedback (n = 55) 71 18 11
 No-Feedback (n = 48) 54 19 27

Spring Benchmark Distribution:
 Feedback (n = 55) 51 33 16
 No-Feedback (n = 48) 40 29 21

(a) Benchmark criteria are not established for ISF in spring
or for PSF and NWF in fall for kindergarten.

During winter testing, however, the distribution of PSF and NWF performance for children in the No Feedback condition was lower than established criteria. For both PSF and NWF, fewer than 60% of students in the No Feedback condition scored "at benchmark" (skills are "established), and the proportion of students in the "at risk" category (skills are "deficit") was above 20%. In contrast, the percentages of children in the Feedback condition performing at benchmark versus at risk were consistent with normative criteria.

Finally, as shown in Table 2, the performance on the ISF measure for both Feedback and No Feedback conditions revealed that many students failed to demonstrate sufficient progress according to normative expectations. Fewer than 40% of students had "established" this skill as expected by the middle of kindergarten. This trend of insufficient progress, particularly among children in No Feedback classrooms, was even more noticeable during the spring testing period. With one exception, less than half (< 50%) of students across both conditions demonstrated performance at "established" or "benchmark" levels on DIBELS probes as expected by the end of kindergarten. Parallel increases in the proportion of students in the two risk categories also occurred during the spring testing. In summary, although children generally demonstrated significant improvement across time (Table 1), the benchmark performance (Table 2) suggests that overall progress relative to DIBELS normative data may have been insufficient to ensure future reading success for most children.

Perceptions of Usefulness of Progress-Monitoring Data and Effect on Classroom Environment

Table 3 presents the scores on the end-of-year survey for each teacher in the Feedback and No Feedback conditions. A Mann-Whitney U test was used to compare survey responses of teachers between the two conditions. The results from this analysis revealed no significant differences between groups for the total score or for any individual survey item. Teachers' perceptions of the utility of progress-monitoring data were similar, regardless of whether they actually received information about student progress.
Table 3
Teacher Survey and ETLCO Total Scores for Classrooms in
Feedback and No Feedback Conditions

Condition: Classroom Fall Spring Fall-Spring Survey (b)
 (Sep) (May) Change

Feedback Condition:
 Classroom 1 72 75 +3 15
 Classroom 2 66 80 +14 15
 Classroom 3 80 91 +9 12
 Classroom 4 86 95 +9 18

No-Feedback Condition:
 Classroom 5 85 96 +11 23
 Classroom 6 76 78 +2 22
 Classroom 7 67 68 +1 17
 Classroom 8 72 99 +27 13

(a) ELLCO total score range: 15-124.
(b) Teacher survey total score range: 5-25.

Table 3 also presents the ELLCO total scores for classrooms in the Feedback and No Feedback conditions during fall (September) and spring (May) measurement. A Wilcoxon Signed Ranks test showed significant change across time for all classrooms (Z = -2.52; p < .01). Change scores (see Table 3) were calculated for each classroom and compared using a Mann-Whitney U test. Again, contrary to what we predicted, there were no significant differences between groups. The classrooms of teachers in both conditions demonstrated similar improvement in ELLCO scores over time. Despite improvement in the overall quality of the early literacy environment from fall to spring for classrooms, it is interesting to note that the correlations between ELLCO scores and median DIBELS scores in the spring were low to moderate, ranging from .17 (ISF) to .31 (LNF).


This study was designed to evaluate the effects of providing feedback to kindergarten teachers about children's progress in early literacy skill development. The study examined the effects of feedback on children's performance on the DIBELS, the quality of classroom literacy environments, as well as teachers' perceptions of the utility of progress-monitoring data. Students of teachers in the Feedback condition demonstrated greater improvement on literacy skills compared to students of teachers in the No Feedback condition; these observed differences were statistically significant for two subtests, LNF and PSF. These findings suggest that providing feedback to teachers about children's early literacy progress served to increase the performance of their students on subsequent testing, more so than students of teachers who received no feedback. Importantly, these effects were achieved without providing training, recommendations for using the feedback, or ongoing consultative support to help teachers implement instructional changes.

Despite statistically significant differences between the two conditions, the proportion of students who met DIBELS benchmark goals at the end of kindergarten did not exceed 51% for either condition. Although feedback led to differentially greater improvement among students, the overall educational significance of providing performance feedback appears to be limited. Specifically, feedback alone was not sufficient for teachers to meet the instructional needs of all children in their classrooms given that most students were below benchmark at the end of the year. Further evidence that feedback alone has limited benefit was observed in teachers' perceptions of the utility of the information they received. Teachers in the Feedback condition reported that information about children's progress had only "limited" to "moderate" influence on their instructional practices and that it was "limited" or "somewhat useful" in terms of altering their classroom environments. Overall, teachers did not find feedback highly useful, either because they did not know how to use the information or because they did not have the resources necessary to make instructional modifications. This finding is consistent with previous research on CBM which underscores the limitations of simply providing teachers with performance data, without efforts to demonstrate instructional adaptations, provide training to teachers, or monitor classroom changes (Stecker et al., 2005).

Based on the ELLCO scores, there were no differences in the degree to which the quality of classroom environments improved over time between the Feedback versus No Feedback conditions. Although contrary to our original prediction, this finding is not surprising in light of the information teachers received about their students' DIBELS performance during the fall (September) and winter (January) feedback meetings. When feedback meetings occurred in the fall, DIBELS scores reflected an acceptable distribution of students meeting benchmark criteria (see Table 2). At least 60% of students were "at benchmark" for ISF and LNF, and less than 15% fell into the "at risk" category. Based on these data, teachers most likely concluded that students were making adequate progress and that major changes to their classroom environment were not warranted. Similarly, when feedback was provided in the winter, more than 60% of students were at benchmark on three subtests (LNF, PSF, and NWF), and only 6% were "at risk" on the fourth subtest (ISF). Again, these performance data did not signal to teachers the need for classroom modifications. In the spring, however, most students (more than 50%) scored below benchmark on all subtests. These spring performance data were shared with teachers too late for them to modify their classrooms or to make instructional adaptations, especially for students who were considered to be "at risk." In general, this suggests that more frequent monitoring may be required to maintain an appropriate number of students at benchmark throughout the school year.

The findings from this study extend existing research in two important ways. First, unlike previous research (e.g., Graney & Shinn, 2005), feedback in this study included information about the performance of all students in the classroom, not exclusively students with low reading performance. The universal progress-monitoring and whole-class feedback provided to teachers may be more beneficial than feedback targeting individual students in terms of maximizing achievement for all learners. Second, the results of this study suggest that providing teachers with feedback from periodic, classwide progress-monitoring can lead to greater gains in students' performance than providing no feedback at all. At the same time, however, the fact that students lost ground compared to DIBELS benchmarks throughout the year indicates that more frequent monitoring and more useful feedback may have been required to make important classwide changes that help most students. Thus, this study is an important step toward further research that explores the effects of more frequent feedback and/or different levels of support with feedback (e.g., feedback with instructional recommendations) to maximize the educational impact of the information contained in DIBELS scores.

As with any research, this study has limitations. First, the sample was comprised of teachers and students who were fairly homogeneous with respect to racial (Caucasian) and socioeconomic background (middle- to upper-middle class), thus limiting the external validity of the study. As a result, there are restrictions in the extent to which the findings can be generalized to other samples or contexts, including high-poverty schools, English Language Learners, or racially diverse teachers and students. Second, the sample size was small. Although we included a relatively large number of student participants (N = 103), the number of teacher participants was much lower (N = 8). This presents a potential problem in terms of our unit of analysis. Specifically, treatment condition was randomly assigned at the classroom or teacher level; outcomes, however, were assessed at the student level. Although this approach to data analysis is fairly common in educational research, it is important to bear in mind that the observations of students within classrooms are not independent. Rather, they are inextricably linked to classroom and teacher variables. This increases the possibility that the results we found were produced by a classroom or teacher variable, other than providing feedback, that was not directly measured.

An additional limitation of this study relates to the use of the DIBELS as an outcome measure. This study failed to include an independent measure of early reading skills. As such, the feedback measure (DIBELS) was also the outcome measure. This potentially allowed teachers in the Feedback condition to "teach to the test." In this study, however, teacher self-reports suggested it was unlikely they made overt attempts to improve students' performance on the DIBELS. A second potential problem in using the DIBELS is evident in the observation that students seemed to get worse across time compared to the benchmark goals. This phenomenon is somewhat common for measures like the DIBELS, which utilize fluid benchmark scores. That is, the DIBELS' criterion for achieving "benchmark" performance changes over time, and students must make regular progress throughout the year to maintain their benchmark status. Some alternative measures utilize specific fixed criteria, which hold the criterion constant across time and emphasize children's progress toward the ultimate goal. The use of such an alternative criterion-referenced measure to assess early literacy outcomes at pre- and post-intervention may have reflected normative gains in early literacy skills, rather than an apparent decline. Although many standardized measures of literacy development assess similar skills as the DIBELS, a measure that relies on specific fixed criteria rather than fluid benchmark cutoff scores may provide a more stable indication of student growth.

The results of this study point to some possible directions for future research. First, with the increased focus on implementation of RTI, research should continue to examine the application of periodic progress-monitoring in classrooms. In particular, given the pattern of student performance on DIBELS across time (i.e., performance was acceptable in fall and winter, but inadequate by spring), further research may determine whether three data collection points are sufficient for teachers to adequately evaluate their classroom practices and make classroom adjustments. Overall, using DIBELS as a progress-monitoring tool within an RTI framework has the potential to help teachers make efficient changes in classroom instruction to help students make progress in reading at the critical early stages. Based on the mixed results of this study, however, more research appears to be needed to make the provision of DIBELS performance feedback a strong intervention.

The effect of progress-monitoring data on both classroom instruction and students' development of early literacy skills remains an important area for continued research. Teachers in the current study who received feedback reported that student progress information did not influence their instructional practices. Moreover, the ELLCO data confirmed that teachers in the Feedback condition did not make more adjustments in their classroom environments compared to teachers who received no feedback. Nevertheless, their students demonstrated greater improvement in the development of certain early literacy skills. In light of these findings, future research should take a closer look at the processes by which these differences emerged, given that neither teacher reports nor classroom observations revealed any significant indicators of differential environmental change.

Finally, future research should focus more explicitly on classroom instructional strategies. Although general literacy instruction as captured by the ELLCO may not have been influenced by feedback, it remains possible that teachers initiated other types of changes in response to feedback about children's performance. For example, in many kindergarten classrooms, students who fall below expected performance levels often receive additional practice opportunities, individual tutoring from an aide or volunteer, or a stronger focus on literacy skills. Teachers may not regard these types of adaptations as actual changes in their instruction or environment, and the ELLCO is certainly not sensitive to such adjustments at the individual student level. Future research, therefore, should obtain classroom data related directly to instruction (e.g., amount, intensity, frequency). Collecting such data as part of this study may have detected individualized modifications teachers made in response to student performance feedback and helped to explain the greater improvement of students in the Feedback condition. Careful selection and collection of data related to instructional techniques would also allow for an analysis of the relationship between instruction and DIBELS performance. Future research that attempts to replicate this study would be sharpened by including a direct measure of instructional strategies.

In sum, this study addressed questions relevant to periodic progress-monitoring of early literacy skills among kindergarten children. The results suggested that providing teachers with feedback does lead to greater improvements in student performance on some DIBELS subtests compared to no feedback. The methods teachers use to bring about these improvements, however, remain unclear. Teachers who received feedback reported limited utility of the information for implementing changes in classroom practice; likewise, classroom observations showed that changes to the environment were neither systematically influenced by feedback nor strongly correlated with DIBELS performance. Further knowledge about how teachers can be supported in their efforts to make use of student progress data and to promote early literacy development for all children is critical for advancing an RTI model and preventing reading failure.


Burns, M. S., Griffin, P., & Snow, C. E. (Eds.). (1999). Starting out right: A guide to promoting children's reading success. Washington, DC: National Academy Press.

Carta, J. J., Greenwood, C. R., & Atwater, J, (1985). Ecobehavioral System. for the Complex Assessment of Preschool Environments. Kansas City, KS: Juniper Gardens Children's Project, Bureau of Child Research Project, University of Kansas.

Casey, A., & Howe, K. (2002). Best practices in early literacy skills. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 721-735). Bethesda, MD: National Association of School Psychologists.

Coleman, M. R., Buysse, V., & Neitzel, J. (2006). Response and recognition: An early intervening system for young children at-risk for learning disabilities. Full report. Chapel Hill, NC: The University of North Carolina at Chapel Hill, FPG Child Development Institute.

Fuchs, D., & Fuchs, L. S. (2006). Introduction to responsiveness-to-intervention: What, why, and how valid is it? Reading Research Quarterly, 41, 92-99.

Fuchs, L. S., Fuchs, D., Hamlett, C. E., Phillips, N. B., & Karns, K. (1995). General educators' specialized adaptation for students with learning disabilities. Exceptional Children, 61, 440-459.

Good, R. H., Gruba, J., & Kaminski, R. A. (2002). Best practices in using Dynamic Indicators of Basic Early Literacy Skills (DIBELS) in an outcomes-driven model. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 699-720). Bethesda, MD: National Association of School Psychologists.

Good, R. H., & Kaminski, R. A. (Eds.). (2002). Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: University of Oregon, Institute for the Development of Education Achievement.

Good, R. H., Kaminski, R. A., Smith, S. B., Kame'enui, E. J., & Wallin, J. (2003). Reviewing outcomes: Using DIBELS to evaluate kindergarten curricula and interventions. In S.R. Vaughn & K.L. Briggs (Eds.), Reading in the classroom: Systems for the observation of teaching and learning (pp. 221-266). Baltimore: Brookes.

Graney, S. B., & Shinn, M. R. (2005). Effects of reading curriculum-based measurement (R-CBM) teacher feedback in general education classrooms. School Psychology Review, 34, 184-201.

Graves, A. W., Gersten, R., & Haager, D. (2004). Literacy instruction in multiple-language first-grade classrooms: Linking student outcomes to observed instructional practice. Learning Disabilities Research & Practice, 19, 262-272.

Juel, C. (1988). Learning to read and write: A longitudinal study of 54 children from first to fourth grades. Journal of Educational Psychology, 80, 437-447.

Kame'enui, E. J., & Simmons, D. C. (1990). Designing instructional strategies: The prevention of academic learning problems. Columbus, OH: Merrill.

Kaminski, R. A., & Good, R. H. (1996). Toward a technology for assessing basic early literacy skills. School Psychology Review, 25, 215-227.

Makin, L. (2003). Creating positive literacy learning environments in early childhood. In N. Hall, J. Larson, & J. Marsh (Eds.), Handbook of early childhood literacy (pp. 327-337). Thousand Oaks, CA: Sage.

Massey, S. L. (2004). Teacher-child conversation in the preschool classroom. Early Childhood Education Journal, 31, 227-231.

Missal, K. N., McConnell, S. R., & Cadigan, K. (2006). Early literacy development: Skill growth and relations between classroom variables for preschool children. Journal of Early Intervention, 29, 1-21.

Neuman, S. B., & Dickinson, D. K. (Eds.). (2001). Handbook of early literacy research. New York: Guilford Press.

Scarborough, H. S. (1998). Predicting the future achievement of second graders with reading disabilities: Contributions of phonemic awareness, verbal memory, rapid serial naming, and IQ. Annals of Dyslexia, 48, 115-136.

Scott, B. J., Vitale, M. R., & Masten, W. G. (1998). Implementing instructional adaptations for students with disabilities in inclusive classrooms: A literature review. Remedial and Special Education, 19, 106-119.

Searfoss, W., Readence, J., & Mallette, M. (2001). Helping children learn to read: Creating a classroom literacy environment, Toronto: Allyn & Bacon.

Simmons, D. C, Kame'enui, E. J., Coyne, M. D., & Chard, D. J. (2002). Effective strategies for teaching beginning reading. In E.J. Kame'enui, D.W. Carnine, R.C. Dixon, D.C. Simmons, & M.D. Coyne (Eds.), Effective teaching strategies that accommodate diverse learners (2nd ed., pp. 53-92). Columbus, OH: Merrill Prentice Hall.

Smith, M. W., Dickinson, D. K., Sangeorge, A., & Anastasopoulos, L. (2002). Early Language and Literacy Classroom Observation Toolkit, research edition. Baltimore: Brookes.

Snow, C. E., Burns, S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press.

Stecker, P. M., & Fuchs, L. S. (2000). Effecting superior achievement using curriculum-based measurement: The importance of individual progress monitoring. Learning Disabilities Research & Practice, 15, 128-134.

Stecker, P. M., Fuchs, L. S., & Fuchs, D. (2005). Using curriculum-based measurement to improve student achievement: Review of research. Psychology in the Schools, 42, 795-819.

Whitehurst, G. J., & Lonigan, C. J. (1998). Child development and emergent literacy. Child Development, 69, 848-872.

Carrie Ball

Ball State University

Maribeth Gettinger

University of Wisconsin-Madison

Correspondence to Carrie Ball, PhD, Department of Educational Psychology, Ball State University, Muncie, IN 47306; e-mail:
COPYRIGHT 2009 West Virginia University Press, University of West Virginia
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2009 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Ball, Carrie; Gettinger, Maribeth
Publication:Education & Treatment of Children
Article Type:Report
Geographic Code:1USA
Date:May 1, 2009
Previous Article:Advances in Evidence-Based Education: A Roadmap to Evidence-Based Education.
Next Article:The use of audio prompting to assist mothers with limited English proficiency in tutoring their pre-kindergarten children on English vocabulary.

Terms of use | Privacy policy | Copyright © 2021 Farlex, Inc. | Feedback | For webmasters |