Printer Friendly

Exploring the relationship between fidelity of implementation and academic achievement in a third-grade gifted curriculum: a mixed-methods study.

With educational practices currently receiving deep scrutiny, researchers increasingly focus on the impact of research-based curricula and teacher practices on student achievement. The need to attend to research-based practice extends to the field of gifted education where research should be directed toward understanding and measuring classroom environments in which gifted children learn, as well as the educational practices that best respond to the learning characteristics of these students (Plucker & Callahan, 2007).

To measure the impact of research-based curricula designed for the gifted classroom, researchers must consider the degree to which teachers who implement such curricula do so with fidelity to the research-based design (Lillehoj, Griffin, & Spoth, 2004; Lynch & O'Donnell, 2005). Logically, if teachers do not implement research-based curricular interventions as designed, measured outcomes cannot be attributed to the effectiveness of the intervention. However, few studies measure fidelity of implementation (FOI) or "the extent to which delivery of an intervention adheres to the protocol or program model originally developed" (Mowbray, Holter, Teague, & Bybee, 2003, p. 315). Even fewer explore the ways in which teachers' beliefs, expectations, and perceptions influence their adherence to an intervention (Kennedy, 2004).

The purpose of this study was twofold. First, we examined ways in which teacher beliefs, expectations, and perceptions influenced teacher implementation of research-based curriculum. Then, we conducted follow-up analyses of student achievement test scores to assess the relationship between teachers' fidelity to the curricular model and student performance.

Research-Based Curriculum and Context

The research described here is embedded within a Javits Grant-funded study conducted by The National Research Center on the Gifted and Talented (NRC/GT) to develop empirical and descriptive understandings of effective curriculum for gifted students. This larger study is called "What Works in Gifted Education?" (WWIGE). As described more fully below, WWIGE studies the impact on student achievement of two challenging units in language arts/reading developed by experts in the field of gifted education from the University of Virginia for third-grade, self-contained, and pull-out gifted classrooms.

The units are based on three well-known models in gifted education: (a) Tomlinson's Differentiated Instruction Model, (b) Renzulli and Reis's Schoolwide Enrichment Model (SEM), and (c) Kaplan's Depth and Complexity Model. The Differentiated Instruction Model articulates and organizes curricular and instructional approaches designed to meet the needs of individual learners (Tomlinson, 2001). SEM encourages talent development and deliberately and systematically provides students with the skills, knowledge, and attitudes necessary for authentic work and original products (Renzulli & Reis, 2010). The Depth and Complexity Model provides a vocabulary and strategies for designing rich and challenging curriculum specifically for gifted learners (Kaplan, 2005).

For purposes of the WWIGE study, distinct components of these models were synthesized and integrated into the Challenge Leading to Engagement, Achievement, and Results (CLEAR) Curriculum Model. The CLEAR Curriculum Model was developed to incorporate and reflect best practices in gifted education and to provide a framework for designing high-quality, authentic curriculum appropriate for diverse learners, including students identified as gifted and those capable of advanced work.

The CLEAR Curriculum Model served as the theoretical and philosophical underpinning for two language arts/reading units. One, "The Magic of Everyday Things," focuses on poetry. The other, "Exploration and Communication," focuses on expository, nonfiction text and research skills (referred to as the "poetry" and "research" units, respectively). Lessons for both units were designed for 45- to 60-minute instructional blocks. Both units are fully differentiated to challenge gifted learners with varying skills, interests, and readiness levels in a variety of settings. Additional principles of differentiation, such as formative and ongoing assessment and flexible grouping, are applied throughout the units.

In the broader study, the impact of the two units will be based on measured outcomes of students exposed to the intervention using standards-based assessments and structured performance assessments. To evaluate whether these measured outcomes are attributable to the intervention, the broader study and this more limited study consider FOI, specifically how beliefs and expectations influence a teacher's adherence to the research-based curriculum.

Review of the Literature


Educators are increasingly called on to use a variety of curricular and instructional approaches to meet the diverse needs of students and improve student achievement. The No Child Left Behind Act (2001) requires educators to "use only research-based teaching methods and programs 'proven' to be effective" (O'Donnell, 2008, p. 35). Therefore, researchers and educators should be concerned about the extent to which the educational interventions they design, implement, and adopt are evidence based (Lynch & O'Donnell, 2005; Mowbray et al., 2003; Slavm, 2003).

To determine the effectiveness and impact of a research-based curricular intervention, assessing the degree to which its implementation adheres to the model of curriculum and instruction inherent in the research design becomes important (O'Donnell, 2004), a concept referred to as FOI. Although there are many competing definitions, we broadly address FOI as the extent to which delivery of an intervention adheres to the protocol or program model originally developed (Mowbray et al., 2003). Assessing FOI in the context of any intervention study is important because "[f]ailure to establish fidelity can severely limit the conclusions that can be drawn from any outcome evaluation" (Dumas, Lynch, Laughlin, Smith, & Prinz, 2001, p. 39).

In studies assessing FOI, some empirical evidence suggests significant correlations between the extent to which intervention studies are implemented with fidelity and the level of treatment outcomes (Blakely, Mayer, Gottschalk, Schmitt, & Davidson, 1987; Dane & Schneider, 1998; Ruiz-Primo, 2005). Although most of these studies have taken place in the context of public health and counseling research (O'Donnell, 2008), FOI studies evaluating the effectiveness of K-12 curricular interventions have been increasingly reported (Lillehoj et al., 2004; Lynch & O'Donnell, 2005; O'Donnell, 2008). When evaluating teacher fidelity in the context of such research-based curricula, a five-component framework has been proposed to include adherence, exposure, program differentiation, quality of delivery, and participant responsiveness (see Table 1; Lynch & O'Donnell, 2005; O'Donnell, 2008). Given the "compelling need to understand how curricula, instruction, and student diversity affect student achievement in the K-12 classroom" (Lynch & O'Donnell, 2005, p. 2), fidelity studies examining the effectiveness of K-12 core curriculum interventions interpreting the impact of teacher FOI on outcome measures warrant attention and have been encouraged (O'Donnell, 2008).

Notably, to assess FOI, it is not necessary to evaluate all five components. Researchers should attend to those components that are of interest to their study (Lillehoj et al., 2004; Lynch & O'Donnell, 2005), and few report measuring all five. For example, Lynch and O'Donnell (2005) monitored adherence, quality of delivery, and participant responsiveness in evaluating teacher fidelity during the implementation of a research-based middle school science curriculum. Similarly, Lillehoj et al. (2004) focused on adherence and quality of delivery while assessing fidelity in the context of a middle school problem behaviors study.

For purposes of our study, the components of adherence and quality of delivery are of primary interest. Specifically, we are interested in how teacher beliefs and experiences influence these two fidelity components. The research-based curriculum conforms to the gifted education field's current knowledge of practices enjoying the greatest support in the literature. As such, the curriculum is distinct from standard curriculum, obviating the need to examine program differentiation. Moreover, the researchers were not able to fully assess and measure participant responsiveness or student engagement during delivery of the curriculum. Finally, investigation of adherence and quality of delivery precluded the need for inquiry into exposure as a distinct component. As indicated in Table 1, exposure (as one dimension of fidelity) looks at the number of lessons taught, time spent, and concepts taught. These variables were considered in our examination of adherence, as well as teachers' reasons for their decisions about adherence and how these factors influenced delivery.

Best Practices in Gifted Education

To reflect best practices in the field, the National Association for Gifted Children (NAGC) issued the Pre-K-Grade 12 Gifted Programming Standards (the Standards; NAGC, 2010). The stated purpose for the Standards is to help "teachers and other educators in Pre-K-12 settings to be effective in working with" gifted learners by helping them "understand the characteristics and needs of the population for whom they are planning curriculum, instruction, assessment, programs, and services" (NAGC, 2010, p. 1).

The Standards are based on literature identifying well-supported characteristics of gifted students (NAGC, 2010). These characteristics include, among others, a desire to focus on the essential knowledge, skills, and "big ideas" within a unit of study; a preference for open-ended and ill-defined academic experiences and tasks; the ability to proceed through curriculum at an individualized and accelerated pace; and a preference for independent work and in-depth pursuit of topics of interest to the student (Renzulli & Reis, 2010; Tomlinson, 2005). Although these characteristics are broadly associated with the gifted learner, educators must also recognize that "gifted learners themselves are anything but formulaic," (Tomlinson, 2005, p. 160) and they present with varying strengths, interests, and needs that require differentiation.

Enrichment and acceleration are two principle approaches for differentiating gifted curricula, and both are incorporated into the Standards (NAGC, 2010; Renzulli & Reis, 2010; VanTassel-Baska & Little, 2011). Enrichment broadly refers to broadening the scope of the curriculum beyond what is typically covered and/or increasing the depth of content studied (Reis & Renzulli, 2003). Enrichment in the context of rich curriculum allows students to analyze the layers of a discipline or content by going from concrete to abstract concepts, and from familiar to unfamiliar academic experiences (Reis & Renzulli, 2003; Tomlinson, 2005). It is distinguished from repeating material already taught and learned and from those classroom activities that fail to provide students the "opportunity to delve into advanced issues and content" (Reis, 2003, p. 195).

Acceleration, however, broadly refers to faster academic progress that matches the "level, complexity, and pace of the curriculum to the readiness and motivation of the student" (Colangelo, Assouline, & Gross, 2004, p. xi). Although acceleration is often associated solely with more radical academic interventions such as grade-skipping and early entrance, it can and should be used in a variety of ways through curriculum compacting and flexible grouping strategies to meet the range of student skills and readiness levels within a classroom (NAGC, 2010; VanTassel-Baska & Little, 2011).

SEM is a prominent model for enrichment programming with a deep theoretical basis (Reis & Renzulli, 2003). Recognizing the diversity of skills and interests that gifted students possess, SEM focuses on the development of creative-productive giftedness through a variety of enrichment activities emphasizing "the use and application of information (content) and thinking skills in an integrated, inductive, and real-problem oriented manner" (Renzulli & Reis, 2010, p. 142).

Tomlinson's Differentiated Instruction Model also enjoys wide use in educational programming for gifted students. Tomlinson (2001) summarized this model as follows: "At the core of the classroom practice of differentiation is the modification of four curriculum-related elements--content, process, product, and affect--which are based on three categories of student need and variance--readiness, interest, and learning profile" (p. 15). Like SEM, the Differentiated Instruction Model incorporates a variety of curricular and instructional strategies to meet the diverse needs and profiles of gifted learners. They include focusing on the "big ideas" and concepts in a discipline, matching the pace, degree of challenge, and interests of each student with instructional tasks and allowing students to create individual products that reflect learning (Tomlinson, 2001).

Kaplan's Depth and Complexity Model provides for using standards-based curriculum as the foundation for creating additional challenge and extended understanding by integrating elements of depth (big ideas, layers of the discipline, details, patterns, and rules) and complexity (multiple perspectives, interdisciplinary connections, unanswered questions, ethical issues, and changes over time; Kaplan, 2005).

Finally, the Standards also state that assessment is integral to the decision making and teaching of gifted students (NAGC, 2010). Thus, the Standards urge educators of the gifted to use the results of formative, performance-based, and pre-assessments to adjust instruction and planning and to enhance ongoing learning progress (NAGC, 2010).

Teacher Expectancy Theory

Teacher expectancy theory generally refers to teachers' differential expectations about the learning capability of students within a classroom. These expectations are often based on perceptual bias rather than on students' actual performance or capabilities (Good & Nichols, 2001; Kuklinski & Weinstein, 2000). Within teacher expectancy theory is the notion that teachers often "hold a deficit-oriented framework when considering the characteristics of the primary-grade learner" (Moon & Brighton, 2008, p. 474). In the gifted education classroom, teachers holding a deficit orientation attend to "remediation for these deficits before suggesting any enrichment, acceleration, or other gifted intervention strategies for their evident strengths" (Moon & Brighton, 2008, p. 474). The results of studies investigating the effect of teacher expectancy suggest that students often confirm the expectations teachers hold for them, and teacher beliefs about the abilities of their students appear to influence student achievement (Kolb & Jussim, 1994). Alternatively, teacher expectations may produce perceptual biases when "teachers perceive, evaluate, or remember their students' behaviors in ways that are consistent with their erroneous beliefs" (Kolb & Jussim, 1994, p. 28). Consequently, when teachers have low expectations for their students' capabilities, student achievement is lower.

Although teacher expectancy theory is generally applied to differential expectations about the learning capability of minority or of high- versus low-achieving students and the resulting differential treatment of those students within the same classroom, some research posits this theory may have applicability to the expectations applied to whole classrooms (Reis, 2003). However, this research is scant. In addition, one would expect stereotypes of gifted learners to be reflected in high teacher expectancies for classrooms of gifted students. Pilot studies using the curriculum in our study suggested that universally high expectancies for gifted students are an unwarranted assumption. Thus, the impact of teacher expectancies in the gifted classroom, particularly when research-based curricular interventions are being implemented, merits consideration.

Using teacher expectancy theory as a conceptual framework, we sought to understand teacher beliefs about and expectations of student capabilities in the context of the gifted education classroom. We further sought to understand how those beliefs and expectations affected teacher adherence to and delivery of the educational practices and strategies reflected in a research-based advanced language arts curriculum. Finally, we sought to determine whether adherence and quality of delivery, as influenced by teacher expectations and beliefs, were related to measured student outcomes.


The researchers employed a sequential mixed-methods research design, beginning with the qualitative component of the study. In doing so, researchers relied primarily on interpretivist methods and techniques (Erickson, 1986) for the major purpose of understanding the experiences and beliefs of the participants and how these beliefs and expectations influenced their adherence to and delivery of the research-based curriculum (Maxwell, 2005). Second, we used quantitative analysis to illuminate the qualitative component of the study and to further understand whether the degree to which a teacher exhibited FOI (i.e., adherence and quality of delivery) is associated with student outcomes on researcher-designed, criterion-referenced postassessments.

Participant Sample

Teachers were randomly assigned to experimental and treatment classrooms in the larger WWIGE study. Participants in the experimental condition included 55 teachers with a total of 740 students in urban, suburban, and rural schools across 10 states: Virginia, Maryland, South Carolina, Colorado, Kentucky, Florida, Pennsylvania, Connecticut, Georgia, and Texas. The units were implemented in self-contained and pull-out classrooms for identified gifted students during the 2009-2010 school year. (An additional 32 classrooms, which included 520 students, served as a comparison group for the broader study.) The qualitative analysis of data from the 55 experimental classrooms resulted in these teachers being assigned to one of three groups: high fidelity, medium fidelity, and low fidelity. The basis for this assignment is described later in this article.

After qualitative analysis of observation and interview data from the 55 experimental teachers, researchers used another purposive sampling strategy, maximum variation sampling, to understand a wide range of teacher fidelity levels and their impact on another phenomenon of interest, namely, student outcomes (Creswell, 2007; Miles & Huberman, 1994). Maximum variation sampling is appropriate in emergent or sequential methodological approaches such as the one used here where conclusions drawn at one phase of a study (the influence of teacher beliefs and expectations on fidelity) can then be used to inform further direction of the study (student outcomes; Creswell, 2007). This sampling strategy resulted in a reduced sample size of 26 teachers, 12 of whom were categorized as high fidelity and 14 as low fidelity, for subsequent quantitative analysis. This methodological approach allowed the researchers to more precisely explore the relationship between teacher fidelity and student outcomes in a manner consistent with the purposes of maximum variation sampling.

Data Sources

Qualitative sources of data included observations in a field setting while teachers implemented the reading units and intensive, follow-up participant interviews. Researchers observed and interviewed 96% of teachers in the experimental condition at least once; 67% were observed and interviewed on three separate occasions. Observations were recorded on a semistructured protocol specifically designed to note where teachers adhered to, or did not adhere to, the lessons. Following the observations, researchers interviewed participants using semistructured interview protocols to gain an understanding of the fidelity and instructional choices teachers made during implementation. These interviews were recorded and transcribed for later analysis.

Quantitative data sources included pre- and postassessments. To ensure equity of achievement levels prior to intervention, student performance on the Iowa Tests of Basic Skills (ITBS) Reading Comprehension subtest (Level 9, Form A) was assessed at the beginning of the 2009-2010 school year. The Reading Comprehension subtest measures how students derive meaning from a variety of reading passages representing narrative, poetry, and nonfiction material from science and social studies. Reliability estimates of the Reading Comprehension subtest are generally in the range of .95 (Hoover et al., 2003). For the student outcome measurement, two criterion-referenced posttests with 35 items each were constructed by the research team and piloted at treatment sites in different states to determine appropriate level of difficulty and discrimination for each item. Posttests were administered in May 2010 at treatment and comparison sites. Cronbach's alpha and Spearman-Brown reliability estimates ranged from .72 to .79 for both of the tests.

Data Analysis

Qualitative data analysis. A research team, including a research scientist and two graduate research assistants, analyzed the qualitative data. The team members were experienced educators and had a specialty area in gifted education, qualitative methodology, and reading instruction. Analytic induction (Erickson, 1986), which included reading the data corpus (observations and interviews), coupled with initial memo writing on themes emerging from the data occurred in three stages.

During the first stage of the data analysis, the researchers read the data corpus to "develop tentative ideas about categories and relationships" (Maxwell, 2005, p. 96). All initial categories synthesized the observational data with its companion interview data. This allowed the research team to compare the researcher-generated observational data with the related teacher interview data. For example, if a researcher observed that a teacher had omitted the formative assessments incorporated into the intervention or had failed to use the grouping strategies set forth therein, the research team categorized those practices as lacking adherence. The research team then reviewed interview data to ensure that these nonadherent practices or modifications to the intervention were discussed and explained during the follow-up interview. This process allowed the researchers to identify those components of the intervention from which the participants deviated while understanding the teacher beliefs undergirding the deviations.

The initial reading of the data allowed the researchers to capture adherence to the intervention as well as a set of beliefs and experiences that indicated a teacher's adherence and quality of delivery. These beliefs and experiences, which provided initial coding categories, included a teacher's perception of the amount of time she (1) had to implement the intervention (time), the expectations she held for students' abilities to understand and learn the content and skills in the intervention (teacher expectations), the degree of autonomy and support she felt while implementing the intervention (autonomy), and the degree to which she felt professionally competent in the areas of poetry and research instruction (professional expertise).

To promote coder agreement, research team members independently coded the interview transcripts of three teachers. They then met to compare the application of codes and to refine their understanding about code applicability. The team divided the remaining data for coding with frequent interaction and feedback across the team. This exercise was repeated until coders aligned on the application of the codes. Periodically, the data were recoded by a different team member to check for consistent coding. In all, three iterations of the coding process were required to reach interreliability of .90.

The research team found that each coding category (time, autonomy, teacher expectations, and professional expertise) represented a continuum reflecting a range of beliefs and expectations that could facilitate or serve as a barrier to adherence and quality of delivery. For example, the researchers observed that teachers devoted varying amounts of time to implement lessons contained in the unit. During the interviews, researchers learned that the amount of time a teacher spent on a lesson depended on her perception of time available. The participants' perception of time as captured in interview data varied from "I have as much time as I need to teach these units just as you asked" (teacher interview, March 23, 2010), to "I really have to be creative in finding the time I need to teach these lessons because I have so many students" (teacher interview, October 14, 2009), to "Of course my whole year had to be adjusted because we had a whole spate of tests that we had to give, that kind of thing and that's made it more difficult to implement the units" (teacher interview, April 16, 2010), to "I just don't have time to do it all" (teacher interview, October 27, 2009). The teachers' varying perspectives of time that differentially impacted their adherence were ultimately coded under a perception of time category.

Similarly, researchers observed that some participants adhered strictly to the lessons with virtually no modifications. Participants' beliefs about autonomy captured from interview data ranged from "Since I agreed to participate in this study and my principal supports me, I taught the units exactly as you asked" (teacher interview, March 16, 2010), to "Although I wouldn't necessarily have chosen to do this, my district coordinator has given me all I need to make it happen" (teacher interview, October 9, 2009), to "If my principal hadn't told me I have to participate in this study, I would not have taken it on" (teacher interview, March 26, 2010). The teachers' varying beliefs about their autonomy were ultimately coded under an autonomy category.

Another example is seen in the degree to which teachers used grouping and differentiation strategies incorporated into the intervention. The researchers observed that teachers adjusted the pace of instruction to the readiness levels of individual students in varying degrees with some adopting a dominant "one size fits all" pacing strategy; others used strategies allowing students to move through the curriculum at their own pace. When asked about their pacing choices, teachers offered varying explanations ranging from "I noticed some students were having problems" so all students repeated an activity (teacher interview, February 16, 2010) to "Since almost all of my students understood the concepts, I let them move forward while I conferenced with two girls who I had to get up to speed" (teacher interview, December 4, 2009). Generally, these comments reflected differential expectations held by teachers for their students' ability. The teachers' varying expectations were ultimately coded under a teacher expectations category.

After developing the initial set of categorizing codes reflecting teacher beliefs and experiences, which contributed to adherence and quality of delivery, the research team completed a second reading of the qualitative data corpus to further develop codes. Again, the team reviewed both observational and interview data to confirm that observations regarding fidelity were corroborated or explained by teacher interview statements. The researchers looked for confirming and disconfirming evidence of the codes to ensure that the analytic process plausibly captured the beliefs and experiences that contributed to FOI. The team then subjected the data to further categorizing strategies and thematic analysis.

In the second stage, the research team repeated the steps described above. They then inductively developed, through a close "open coding" of the data to describe more substantively the theoretical categories, a more refined set of codes reflecting more differentiated beliefs and experiences contributing to adherence and quality of delivery (Maxwell, 2005, p. 97). These subcodes helped to further delineate the initial coding categories and became the final codes reflected in Table 2.

Researchers applied the codes to observational and interview data. For example, in coding the observational data, we observed that Mrs. Robbins2 adhered to the lessons in the units consistently. When Mrs. Robbins was asked to explain the reasons for not modifying the lessons during follow-up interviews, she replied, "I've been trying to stick to the lessons exactly as they're written because it's research, and I signed on to this" (Robbins interview, March 16, 2010). This data received the "AUT-P" (Autonomy-Participant commitment) code because it reflected Mrs. Robbins' beliefs about her autonomous role as participant.

As another example, in coding the observational data, we found that Mrs. Baker made extensive modifications to the intervention in, among other things, the grouping strategies she used. Specifically, when the intervention prescribed small-group or pair-share activities, Mrs. Baker consistently used whole-group practices. In follow-up interviews seeking to understand these modifications, Mrs. Baker explained, "My kids really want to hear from each other. They just don't have fun if they don't get to share with each other. And if they're not having fun, they're not going to learn" (Baker interview, March 8, 2010). As these statements indicated Mrs. Baker's expectations that her students were less likely to succeed without modifications, this data received Teacher Expectations-Fun (TE-F) and Teacher Expectations-Share (TE-S) codes.

Finally, in a third stage of qualitative analysis, we found that teachers could be grouped into three categories: high adherence, mixed adherence, and low adherence. From the exhaustive coding and inductive analytic process described above, teachers were grouped according to their overall fidelity to the lessons as written and to the theoretical strategies embedded within. Researchers relied on observational data that found broad support in interview statements in making these decisions. See the "Findings" section for details. This finding is explicated here to provide the context of the analysis of student outcome data which follows.

Multilevel analysis of student outcome data. Once the qualitative data analysis was complete, the researchers used the high and low teacher categories to further inform the quantitative data analysis to examine the association between teachers' adherence to the curriculum and student outcomes.

Maximum variation sampling (Creswell, 2007; Miles & Huberman, 1994) provided a complimentary method to illuminate the broadest differences in overall beliefs and experiences between teachers in the high-adherence group and teachers in the low-adherence group. Student outcomes from the high-adherence teachers' classrooms were compared with outcome measures from low-adherence teachers' classrooms. Although the qualitative analysis resulted in the placement of 12 teachers in the "high-adherence" group and 14 in the "low-adherence" group, ultimately, the sample for the quantitative component of the study included 11 teachers from the "high-adherence" group, and 13 from the "low-adherence" group because one teacher from each group did not return posttest data for quantitative analysis. These two groups were compared for purposes of analyzing student outcome data. Consistent with the principles of maximum variation sampling, "mixed-adherence" teachers were not included in the quantitative analysis.

Multilevel analyses for both units allowed the nested nature of the data set to be taken into account and prevented issues with aggregation bias and the misinterpretation of standard errors in the analysis (Raudenbush & Bryk, 2002). All the analyses used Mplus 6, which allows maximum likelihood parameter estimates with standard errors and chi-square test statistics that are robust to nonnormality and non-independence of observations in clustered data (Harnqvist, Gustafsson, Muthen, & Nelson, 1994). Prior to data analysis, intraclass correlation (ICC) and design effect (DE) statistics were calculated to decide whether multilevel modeling was necessary. There was significant between-classroom variance: The ICC ranged from .15 to .33 for both units. In addition, the DEs, which ranged from 3.02 to 5.87, also supported the need for multilevel modeling for outcome measurement for both units. To examine the relationship between teachers' adherence to the curricular intervention and student outcome measures, multilevel models for each unit were generated. The Level 1 model contained students' ITBS scores. The Level 2 model included teacher's adherence to the curriculum. Based on the qualitative data analysis with maximum variation sampling (Creswell, 2007; Miles & Huberman, 1994), teachers' adherence to the curriculum was coded as 0 = low-fidelity group and 1 = high-fidelity group. The pretest ITBS scores were grand mean centered, and they were a proxy for previous unaccounted influences. The other categorical variables were dummy coded and left uncentered.

The final models for analysis appear below:

Level 1 model:

[Y.sub.ij] = [[beta].sub.0j] + [[beta].sub.1j][(Grand_ITBS).sub.ij] + [r.sub.ij].

Level 2 model:

[[beta].sub.0j] = [[gamma].sub.00] + [[gamma].sub.01] [(Teacher Adherence).sub.j] + [u.sub.0j].

[[beta].sub.ij] = [[gamma].sub.10] + [u.sub.1j].


Qualitative Findings

Data analysis provided insights into the ways the beliefs and experiences of teachers influence fidelity during the implementation of the research-based curriculum. In other words, these beliefs and experiences influence the degree to which teachers adhere to the units as written as well as the degree to which the quality of their delivery comports with the theoretical and pedagogical ideals contained in them.

Specifically, the research findings revealed that a teacher's sense of autonomy and professional expertise, expectations of student capability, and perceptions of time all interact with each other to either facilitate or constrain adherence and quality of delivery. These interactions are dynamic in that each tends to exert an influence on the others. However, the interactions among beliefs and experiences varied across teachers so that no one indicator influenced adherence and quality of delivery in exactly the same way for all teachers. Nevertheless, indicators of fidelity, including teacher perceptions of autonomy, perceptions of time, expectations of student capability, and beliefs about their professional experience, support the following six general assertions:

1. Adherence falls along a continuum of high adherence to low adherence.

2. Quality of delivery falls along a continuum of high quality to low quality.

3. Teacher beliefs about their degree of professional autonomy fall along a continuum of high autonomy to low autonomy. The more professional autonomy a teacher experiences, the higher the teacher's adherence and quality of delivery tend to be.

4. Teachers' perceptions of the instructional time they have to devote to the intervention fall along a continuum of high perceptions to low perceptions. The higher a teacher's perception of available instructional time, the higher the teacher's adherence and quality of delivery tend to be.

5. Teacher beliefs and expectations fall along a continuum from high to low. The higher a teacher's expectation for student performance or the higher the teacher's belief about her students' abilities, the higher the teacher's adherence and quality of delivery tend to be.

6. Teachers' beliefs about their own professional expertise fall along a continuum of high beliefs to low beliefs. The higher a teacher's belief about her own professional expertise, the higher the teacher's adherence and quality of delivery tend to be.

High-adherence teacher. Overall, a combination of teacher beliefs reflecting high levels of autonomy, high expectations of student performance and belief in students' capability, perceptions of sufficient time to implement the units, and/or strong beliefs about their own professional expertise interacted with each other to indicate high adherence and high quality of delivery. We refer to these participants as high-adherence teachers as their beliefs and experiences combined to facilitate fidelity. A total of 12 participants can be described as adhering closely to the research-based curriculum and delivering the WWIGE units with high quality. High adherence was indicated by overall fidelity to the units as written as observed by the researchers and as understood by teachers in interviews. High quality was indicated by the use of best practices, including (a) appropriate accelerative strategies that set the pace of instruction to the readiness level of students, (b) appropriate enrichment strategies that were directed toward individual student interests, and (c) integrated instructional strategies that differentiated based on individual student need. Incorporating specific interview data, the following vignette illustrates the typical experiences and beliefs of a high-adherence teacher.
   I've been a gifted resource teacher for 6 years. Our students are
   identified for gifted program services in second grade based on
   their abilities test scores. This year I have 12 third-grade
   students. I see them for an hour and a half everyday. I usually do
   language arts 3 days per week and math and critical thinking
   activities 2 days per week. My students get most of their core
   third-grade curriculum instruction in their regular education

   These reading units fit perfectly in my program. They are quite
   different from the language arts instruction my students get in
   their regular classroom where the kids spend a lot of time
   preparing for standardized tests, which I don't have to worry
   about. For my students, this is hardly challenging or engaging. So,
   when my students come to me I try to provide academic experiences
   that challenge them, and that allow them to go more deeply into
   areas of interest to them. They are very curious and thoughtful
   students who I can push. There's really almost nothing I can't give

   I feel like one of the luckiest teachers in the world. My principal
   and district coordinator really trust me to run my program as I
   think is best. I have everything I need and more from our
   librarian, our technology lab assistant, and the classroom
   teachers. They respect the time I have with my students and always
   make themselves available when I need them.

   When I heard about the research you were doing, I thought it would
   be a great fit for my students. After I read the units, I was so
   impressed with the high level and strong quality that I immediately
   wanted to be a participant in the study. I've taught poetry and
   research skills before so I was comfortable teaching this material.
   I jumped right on board and proposed participating in the study to
   my district coordinator. I described both units, and I explained
   how much skills development and critical thinking were included in
   them. Without hesitation, my coordinator and principal said go for
   it. Ever since I committed to the study, they've completely
   supported me. My students love the units also.

   I can usually fit at least two lessons from your study into our
   week. While there are about 24 students in their regular
   classrooms, I only have 12. It makes instruction much more
   manageable. Although my students are all identified gifted, they
   still have varied levels of skills and abilities, so I do need to
   differentiate for them, even in these units. I've found the
   formative assessments in the units to be really helpful, and have
   discovered that the groups are fluid. My expectations for my
   students' capabilities are high. For example, when students raise
   their hands and say, "I need help" or "this is so hard," I don't
   help them right away. Instead I'll say, "you've been at it for one
   minute. It's too soon to give up" or "it's supposed to be hard, but
   I know you can do it." I don't hesitate to let my students know
   when their work product is strong. I say things like "What a great
   metaphor. I've never thought of a snake in that way but I sure will
   now." I also let them know when they haven't reached their
   potential. I'll tell them, "I know you can describe that scene with
   more detail. Use your senses. Keep going." Of course, sometimes I
   have to give one or two students a little more attention because
   they're all at different levels. But in those instances I schedule
   some one on one time while the rest of my students keep moving

   I taught the Explorers unit first. My students really had to
   stretch with some of the independent work, but they got through it.
   Gifted kids shouldn't expect school to be without challenges. I
   can't remember making any modifications to the unit, except once or
   twice asking the kids to finish some of the research at home if
   they didn't finish during class time. I'll be honest here. I really
   considered modifying the poetry unit because when I read the poems,
   I thought they were so abstract and far off from their experiences
   that they'd get very little out of them. I even had supplemental
   materials ready to go. However, since I signed on to your study, I
   felt committed to sticking to the plans and I didn't use that

   I'm glad I didn't change the lessons and I must say I've been
   really impressed. With everything I've taught them they've been
   like sponges. They're using the vocabulary and strategies
   appropriately and developing reading skills. I realized from
   teaching the poetry unit that when you give learning back to the
   kids, they'll understand much more than we might otherwise expect.
   Everyday I come to work and I get to be amazed at the work my
   students produce. My students are in the gifted program for a
   reason. They're incredibly capable.

It is important to note that high-adherence teachers exhibited varying beliefs about their levels of autonomy, expectations of student capability, perceptions of time, and professional expertise. Thus, for example, it was still possible for a teacher to express low beliefs about her professional expertise ("I have never taught poetry before and I was uncomfortable during that unit," [teacher interview, March 14, 2010]) and still be considered a high-adherence teacher based on her overwhelmingly high expectations for student capability that more forcefully influenced her implementation of the intervention. In other words, high-adherence teachers were those teachers for whom implementation of the intervention was more forcefully guided by high levels of adherence and quality of delivery indicators.

Mixed-adherence teacher. A total of 29 participants, or approximately half, can be described as exhibiting mixed adherence to the research-based curriculum and delivering the intervention with mixed quality. Overall, these teachers expressed varying beliefs about their respective levels of autonomy, expectations of student capability, perceptions of time, and professional expertise. In turn, their varying beliefs interacted with each other to indicate mixed levels of adherence and quality of delivery to the research-based curriculum. We refer to these participants as mixed-adherence teachers. The following vignette illustrates the typical experiences and beliefs of a mixed-adherence teacher.
   I teach 22 gifted third-grade students. At my school, the gifted
   program is a self-contained model. To be identified, students have
   to have IQ and achievement scores at the 95th percentile. For low
   SES and minority students, we include students at the 90th
   percentile. Students also produce a portfolio which is taken into
   consideration when we identify for the program.

   I first found out about the study from our district coordinator.
   She presented it to the three GRTs in the county and asked if we
   were interested. My coworkers were very interested, but they have
   fewer students. I said I'd do it because the other two wanted to
   participate, but to be honest I probably would not have done so if
   it were left to me. I think the units are great and the kids have
   gotten a lot out of them. However, on top of everything else I have
   to do I'm really stretched. In fact, next year I'll probably have
   to go back to the classroom because my workload is so intense. My
   principal is great. However, his biggest concern is that our
   students pass the state tests. It's not what I call gifted
   education, and it takes away from what I think is appropriate
   curriculum for these kids, but I know we have to improve our test

   Even though I have my students all day, I don't have a lot of time
   to do these units on top of my other responsibilities. We have so
   much district testing we're still responsible for. I feel like I
   spend most of my day covering our state-mandated curriculum and the
   testing that goes along with it before I get the chance to
   implement the research units. So when I'm running short on time,
   the first thing that I drop is these units. Sometimes a couple of
   weeks go by between lessons, which makes it necessary to cover some
   of the material again.

   One way I try to get through the units is to blend the lessons with
   some of the curricula I'm already teaching. That way, I can kill
   two birds with one stone. For example, in our district we have an
   art history unit. I think it's useful to blend the art unit with
   the poetry unit because so many of the same concepts, like
   abstraction, imagery, and point of view, are in both. I can really
   enrich both units by integrating them. When we went to the art
   museum a few weeks ago, I was so happy to hear my students
   discussing what was abstract and what was concrete in the
   paintings. It showed me how transferrable your research units are.

   We ran into a few difficulties getting library time, which impacted
   how I taught the Explorers unit. Our librarian is so swamped with
   her responsibilities that asking her to assist me on the research
   unit wasn't easy. We also have a limited number of computer
   terminals. Consequently, my students had to take turns doing their
   research, which slowed us down. Fortunately, our regular research
   curriculum had many of the strategies and skills the students
   learned in the Explorers unit. So, I skipped over some lessons in
   Explorers so we could get through it more efficiently. Plus, I'm
   very comfortable with the research process because I've taught the
   skills and strategies contained in the Explorers unit.

   When I first started teaching the poetry unit, I really thought the
   material might be a bit too much for most of my students. The poems
   are so difficult that I worried they might be lost. Sometimes
   third-grade students, even gifted students, need more accessible
   material. At first, I went ahead and added some poetry that was
   much simpler so I'd be sure they understood the concepts. However,
   as I went through the unit I was pleasantly surprised because these
   children-even when I thought this was going to be too hard for
   them-came up with some of the most insightful discussions. I was
   very pleased that even though the children might not get every
   meaning of every single poem they certainly did get a lot out of
   them and could pick out the imagery and poetic devices. So I bit my
   tongue and stepped back because they proved me wrong. Now, I stick
   to your poems.

As reflected in the above vignette, mixed-adherence teachers by definition articulated a full range of beliefs indicating adherence and quality of delivery. For example, a mixed-adherence teacher might have a combination of beliefs such as low autonomy and perceptions of time, high sense of professional expertise, and mixed expectations of student capability, all of which interact to yield a teacher showing mixed levels of adherence and quality of delivery. Thus, a mixed-adherence teacher may have adhered highly to the grouping strategies in the units but not those addressing assessments.

Low-adherence teacher. A total of 14 participants can be described as exhibiting low adherence to the research-based curriculum and low quality of delivery. Overall, these teachers expressed broadly compromised beliefs on their respective levels of autonomy and expectations of student capability, perceptions of insufficient time to implement the units, and doubts as to their own professional expertise. In turn, their low beliefs interacted in such a way as to indicate low adherence and quality of delivery to the research-based curriculum. In other words, beliefs and expectations served as barriers to fidelity. Low adherence was indicated by the inconsistent implementation of numerous unit components as observed by the researchers and as acknowledged by these teachers in interviews. Low quality of delivery was indicated by the weak use of best practices reflected by (a) pacing instruction to the readiness level of the slowest learner(s), (b) limiting enrichment opportunities directed toward individual interests, and (c) inconsistent integration of differentiation strategies. The following vignette illustrates the typical experiences and beliefs of a low-adherence teacher.
   I'm a gifted resource teacher at my school. At my school, the
   gifted program utilizes both pull-out and push-in components. I
   work with students in third, fourth, and fifth grade who have been
   identified as gifted. To be identified, students have to have IQ
   scores at the 95th percentile, or at the 90th percentile if they
   have a strong teacher recommendation.

   I work with third-grade students during my pull-out time with them.
   I have 25 students. In theory, I see them twice a week for an hour.
   However, whenever there is an assembly or field trip, I lose my
   time slot with them. It seems like this class is not always the

   I'm not sure how my school became involved in this study. My
   district coordinator thought it was a great idea and told me I was
   going to do it. Although I really like the units, I feel like I
   don't really have the time to do them the way I've been asked. So,
   I probably would have preferred to not participate in your study.
   I'm just too buried. As you can see, I haven't even had time to
   fill in my teacher fidelity log.

   I really love working with these kids. They are all very sweet and
   bright. They're still at the age where they're not afraid to be
   silly. They can work on material that is more difficult than most
   third-grade students can handle. You would be amazed at how
   creative my students can be, and how they like to think deeply and

   However, they're still children who want to enjoy what they're
   doing. In my opinion, some of the lessons are so challenging that I
   get concerned that they won't enjoy themselves. If my students
   aren't having fun with their work, they're just not going to grasp
   complex material. So I try to incorporate games and catchy
   activities into my lessons to keep them interested. For example, in
   the poetry unit, I let my students bring in their favorite song
   lyrics to see if we could detect poetic devices in them. Plus, it
   gives each student time to share their interests with the class.
   They always want to hear what their classmates think. In fact, even
   though all the poetry lessons asked students to share their
   thoughts about the poems with only one partner, I preferred to do a
   whole-group share. In addition to making class more fun, I believe
   sharing with the whole class helps reinforce the work we've done.
   Otherwise, they'll just tune out. I'll tell you, though, it really
   takes so much time to do it right. These units are taking a lot
   longer than I expected.

   Now that we've gotten through the poetry unit, I must say the
   poetry in this unit is wonderful. I wouldn't necessarily have
   chosen this poetry for them because I thought it would have been
   too difficult. That could be a reflection of my own discomfort with
   poetry, I've just never taught poetry before. I didn't understand
   some of the poems you used, and I knew my students wouldn't either.
   I found I needed to take time to discuss the meaning, context, and
   vocabulary with them, and also did some author study work so they
   could learn more about the poets. I felt they needed more than just
   focusing on images the poem presented. But I was so surprised to
   see that they are capable of this. I still think they just need a
   little more explanation or a lower level of abstraction with
   practical examples of poetry most of the time before we move on to
   new lessons. So I supplement the poems in the lessons with my own
   that are more appropriate for third graders. If I think a worksheet
   is too hard for them, I'll help the group along with whole class
   instruction, or make one up on my own that I think works better for
   everyone. I also really tried to help kids come up with their
   research questions. I thought they would need guidance on that
   because it just seemed too complex for these kids.

   I didn't use your formative assessments very much. Sometimes, the
   groups the assessments would probably create based on ability would
   just be too difficult in my class. Some of my students don't work
   well together and I'd rather have some of the higher students help
   some of the lower students. With so many kids in the class, I need
   to make the classroom work smoothly.

Again, it was possible for low-adherence teachers to demonstrate adherence in some areas. However, their low beliefs in other categories more strongly influenced their overall adherence and quality of delivery. Thus, for example, a teacher with low perceptions of time and low autonomy could demonstrate low adherence and quality of delivery even if she believed she had high levels of professional expertise.

Quantitative Findings

The multilevel analyses indicated that teachers' adherence to the curricular intervention has a significant association with student outcome measures after controlling for pretest scores (on the ITBS Reading Comprehension subtest) for both units (p < .05). Students in high-adherence teachers' classrooms scored 1.98 points higher in the poetry unit and 3.38 points higher in the research unit than students in low-adherence teachers' classrooms. Model fit tests corroborate the significant association between teachers' adherence to the curriculum and student outcome for both units: [DELTA][[chi square].sub.df=1] = 10.00, p < .01 for the poetry unit and [DELTA][[chi square].sub.df=1] = 6.12, p < .01 for the research unit. As effect size indices for the teacher adherence to the curricular intervention, global effect size for each unit was calculated. Results indicated that a significant amount of the variance in student outcome (8.5% for the poetry unit and 12.3% for the research unit) can be explained by teachers' adherence to the curricular intervention. See Tables 3 and 4.


The results of this study have important implications for the development and implementation of research-based curricula. Specifically, our results suggest that the beliefs and experiences teachers bring to the classroom impact their instructional practices, notwithstanding the availability of quality curriculum. Here, the participants all had reading curricula available to them that conformed to best practices in gifted education. All were participants in a study measuring the impact of research-based curricula on student achievement. Nevertheless, the beliefs teachers held about their students, their own expertise, their autonomy, and the time required for implementing the units ultimately produced varying degrees of fidelity.

For example, the use of accelerative and enrichment strategies appropriate to gifted learners depended largely on the expectations teachers held for students. Thus, if a teacher expected that any of her students would struggle with the content and skills embedded within the units, she would often slow the pace of instruction for all students or limit the individual choices available to them. In turn, these beliefs appear to have ultimately affected measured student outcomes. Conversely, if a teacher expected her students to be capable of engaging successfully with a challenging curriculum, she based her instruction on the readiness levels of individual students in a manner consistent with best practices. These results are consistent with prior studies investigating teacher expectancy effects on student outcomes (Moon & Brighton, 2008).

Oftentimes, barriers derived from external factors, such as instructional time with students or demanding district testing requirements. In other cases, low adherence derived from internal factors, such as a teacher's deficit-oriented expectations of her students or her own professional expertise. By comparing these two groups--"high-adherence" teachers who emphasized facilitators for implementing the curriculum versus "low-adherence" teachers who emphasized barriers to implementation--we were able to determine that a teacher's level of adherence is related to student outcomes. Although we do not suggest that these findings are causal in nature, it is an important point of further inquiry.

Implications and Directions for Future Research

The results relating to influences on teacher implementation of curricular interventions and the effects on outcomes are important for researchers who develop research-based classroom interventions as they support the notion that if teachers are not personally committed to participating in a study, lack the time and resources necessary to meaningfully implement an intervention with fidelity, or do not expect students to be capable of high-quality curriculum, they will be less likely to adhere to research protocols. As the results strongly suggest that teacher beliefs and expectations influence fidelity, creating and delivering professional development specifically addressing beliefs and expectations for research participants could be an important first step in mitigating these potentially adverse influences.

Data related to teacher characteristics, such as teaching experience, educational experience, years teaching gifted students, and coursework directed toward gifted learners, were collected. Although one might assume that more classroom experience would yield enhanced adherence, we did not find that to be evident in this study (see Table 5). In fact, teachers in the "low-fidelity" group had more total years experience teaching, teaching third grade, and teaching gifted learners. Similarly, teachers with advanced degrees did not represent a larger number in the "high-fidelity" group (see Table 6). Although these findings provide limited, descriptive insights, they do suggest that the variables influencing a teacher's adherence and quality of delivery in a given curriculum are complex and multifaceted. Although these characteristics were not the foci of this study, they do provide the opportunity for subsequent analyses to investigate the interrelationships and the degree to which these personal characteristics influence teacher beliefs, expectations, FOI, and student outcomes.

Finally, barriers to implementing research interventions with fidelity pose pragmatic challenges to researchers who endeavor to understand curriculum and instruction appropriate for diverse learners. The findings from this study contribute to not only the field of gifted education but also the research community's understanding of FOI. Finally, by addressing how barriers to implementation may in turn influence student academic achievement, this mixed-methods study helps to answer a breadth of questions that lead to empirical and descriptive understandings of "what works in gifted education" in the area of reading instruction.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.


The authors disclosed receipt of the following financial support for the research and/or authorship of this article: Research for this study was supported under the Javits Act Program (Grant No. R305A060044) as administered by the ED/Institute of Education Sciences, U.S. Department of Education. Grantees undertaking such projects are encouraged to express freely their professional judgment. This study, therefore, does not represent positions or policies of the Government, and no official endorsement should be inferred.

DOI: 10.1177/1932202X11424878


Blakely, C., Mayer, J. P., Gottschalk, R. G., Schmitt, N., & Davidson, W. S. (1987). The fidelity--Adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253-269.

Colangelo, N., Assouline, S. G., & Gross, M. U. M. (Eds.). (2004). A nation deceived: How schools hold back America's brightest students. Iowa City: University of Iowa.

Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five approaches (2nd ed.). Thousand Oaks, CA: SAGE.

Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23-45.

Dumas, J., Lynch, A., Laughlin, J., Smith, E., & Prinz, R. (2001). Promoting intervention fidelity: Conceptual issues, methods and preliminary results from the EARLY ALLIANCE prevention trial. American Journal of Preventive Medicine, 20, 38-47.

Erickson, F. (1986). Qualitative methods in research on teaching. In M. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 119-161). New York, NY: Macmillan.

Good, T. L., & Nichols, S. L. (2001). Expectancy effects in the classroom: A special focus on improving the reading performance of minority students in first-grade classrooms. Educational Psychologist, 36, 113-126.

Harnqvist, K., Gustafsson, J. E., Muthen, B., & Nelson, G. (1994). Hierarchical models of ability at class and individual levels. Intelligence, 18, 165-187.

Hoover, H. D., Dunbars, S. B., Frisbie, D. A., Oberley, K. R., Ordman, V. L., Naylor, R. J., Shannon, G. P. (2003). The Iowa tests: Guide to research and development. Itasca, IL: Riverside.

Kaplan, S. (2005). Layering differential curricula for gifted and talented. In F. Karnes & S. Bean (Eds.), Methods and materials for teaching gifted students (2nd ed., pp. 107-132). Waco, TX: Prufrock Press.

Kennedy, M. M. (2004). Reform ideals and teachers' practical intentions. Education Policy Analysis Archives. Retrieved from mkennedy/publications/docs/Teaching%20Practice/PracticalIntentions/ Kennedy%20Ideals%20and%20Intentions.pdf

Kolb, K. J., & Jussim, L. (1994). Teacher expectations and underachieving gifted children. Roeper Review, 17, 26-30.

Kuklinski, M. R., & Weinstein, R. S. (2000). Classroom and grade level differences in the stability of teacher expectations and perceived differential teacher treatment. Learning Environments Research, 3, 1-34.

Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education & Behavior, 31, 242-257.

Lynch, S., & O'Donnell, C. (2005, April). "Fidelity of implementation" in implementation and scale-up research designs : Applications from four studies of innovative science curriculum materials and diverse populations. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.

Maxwell, J. A. (2005). Qualitative research design: An interactive approach (2nd ed.). Thousand Oaks, CA: SAGE.

Miles, M., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: SAGE.

Moon, T. R., & Brighton, C. M. (2008). Primary teachers' conceptions of giftedness. Journal for the Education of the Gifted, 31, 447-480.

Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement, and validation. American Journal of Evaluation, 24, 315-340.

National Association for Gifted Children. (2010). Pre-K-Grade 12 standards. Retrieved from

No Child Left Behind Act, Title IX, General Provisions, Part A, Section, 9101, 20 U.S.C. [section] 7801 et seq. (2001). Retrieved from esea02/pg107.html

O'Donnell, C. L. (2004). Fidelity of implementation: Background, definitions, and components for measuring (Internal document). Washington, DC: The George Washington University.

O'Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78, 33-84.

Plucker, J., & Callahan, C. (Eds.). (2007). Critical issues and practices in gifted education: What the research says. Waco, TX: Prufrock Press.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). London, England: SAGE.

Reis, S. M. (2003). Reconsidering regular curriculum for high-achieving students, gifted underachievers, and the relationship between gifted and regular education. In J. A. Borland (Ed.), Rethinking gifted education (pp. 186-200). New York, NY: Teachers College Press.

Reis, S. M., & Renzulli, J. S. (2003). Research related to the Schoolwide Enrichment Triad Model. Gifted Education International, 18, 15-39.

Renzulli, J. S., & Reis, S. R. (2010). The Schoolwide Enrichment Model: A focus on student strengths and interests. Gifted Education International, 26, 140-157.

Ruiz-Primo, M. A. (2005, April). A multi-method and multi-source approach for studying fidelity of implementation. In S. Lynch & C. L. O'Donnell (Chairs), "Fidelity of implementation" in implementation and scale-up research designs: Applications from four studies of innovative science curriculum materials and diverse populations. Symposium conducted at the annual meeting of the American Educational Research Association, Montreal, Canada. Slavin, R. (2003). A reader's guide to scientifically based research: Learning how to assess the validity of education research is vital for creating effective, sustained reform. Educational Researcher, 60(5), 12-16.

Tomlinson, C. A. (2001). How to differentiate instruction in mixed-ability classrooms (2nd ed.). Upper Saddle River, NJ: Pearson Education.

Tomlinson, C. A. (2005). Quality curriculum and instruction for highly able students. Theory Into Practice, 44, 160-166.

VanTassel-Baska, J., & Little, C. A. (2011). Content based curriculum for high ability learners (2nd ed.). Waco, TX: Prufrock Press.

Amy Azano (1), Tracy C. Missett (1), Carolyn M. Callahan (1), Sarah Oh (1), Marguerite Brunner (1), Lisa H. Foster, and Tonya R. Moon(1)

(1) University of Virginia, Charlottesville, USA


(1.) Pseudonyms have been assigned for all people and places.

Corresponding Author:

Amy Azano, National Research Center on the Gifted and Talented, Curry School of Education, University of Virginia, 1912 Thomson Road, Charlottesville, VA 22903, USA Email:

Amy Azano, PhD, is a research scientist at the University of Virginia's Curry School of Education and serves as a project manager for the "What Works in Gifted Education" grant with The National Research Center on the Gifted and Talented. Her research interests include the development of reading and writing curricula for high-ability students, writing initiatives for struggling students, and place-based literacy interventions for rural students.

Tracy C. Missett, MAEd, JD, is a doctoral candidate in educational psychology, gifted education at the University of Virginia's Curry School of Education where she works as a graduate research assistant with The National Research Center on the Gifted and Talented. She received her BA from the University of Virginia, her JD from the University of California, Hastings College of the Law, and her MAEd from Columbia University, Teachers College. She taught history at the middle school level and as a gifted resource teacher at the elementary level. Her research interests include creativity, gifted students with emotional disabilities, fidelity of implementation, and program evaluation.

Carolyn M. Callahan, PhD, is a commonwealth professor of education at the University of Virginia and has been a principal investigator on projects of The National Research Center on the Gifted and Talented for the past 20 years. She has published more than 200 refereed articles and 50 book chapters across a broad range of topics, including the areas of identification of gifted students, program evaluation, the development of performance assessments, and curricular and programming options for highly able students. She received recognition as outstanding faculty member in the Commonwealth of Virginia, outstanding professor of the Curry School of Education, distinguished higher education alumnae of the University of Connecticut, and was awarded the distinguished scholar award and the distinguished service award from the National Association for Gifted Children. She is a past-president of The Association for the Gifted and the National Association for Gifted Children. She is currently editor of Gifted Child Quarterly.

Sarah Oh, MEd, is a graduate student in gifted education at the University of Virginia where she works as a graduate research assistant with The National Research Center on the Gifted and Talented. She is a former elementary classroom teacher and earned her MEd in curriculum and instruction from the University of Virginia. She has been working with pre- and in-service teachers at the graduate level on differentiating instruction and curriculum for the gifted learners. Her research interests include curriculum development, differentiating instruction, program evaluation, and identification of diverse gifted students.

Marguerite Brunner is a doctoral student in gifted education at the University of Virginia, Curry School of Education, where she also works in The National Research Center on the Gifted and Talented. Her previous work in education includes teaching in both elementary and middle schools and serving as a math specialist in the Charlottesville City Schools. In addition to her teaching, Marguerite has 15 years of experience working with gifted students in various settings. She earned her BA in English at the University of Virginia and earned a postgraduate degree in teaching at the University of Texas. Her research interests include differentiated instruction, fidelity of implementation, identification of gifted students, and mentoring programs for gifted children and their families.

Lisa H. Foster, PhD, is a postdoctoral research associate for Project READS at Harvard University's Graduate School of Education. She is currently developing and overseeing all training, program and treatment fidelity measures/assessments for the project in addition to studying the relationship between fidelity, program evaluation, data sharing, and sustainability of an intervention. As a doctoral student in Educational Leadership at the University of Virginia, she worked as project coordinator for The National Research Center on the Gifted and Talented. She taught Spanish for 20 years and is National Board Certified. Lisa holds licensure in K-12 administration, Spanish, Russian, and government. She earned BAs in international studies and political science from Virginia Tech, and a MEd in curriculum and instruction from Liberty University. She is engaged with research involving fidelity of implementation, effective leadership practices, improved literacy, and reduction of summer reading loss.

Tonya R. Moon, PhD, is a professor in the Curry School of Education at the University of Virginia and a Principal Investigator for the National Research Center on the Gifted and Talented. In addition to her research and teaching responsibilities at the University, she is the Chair of the Institutional Review Board for the Social and Behavioral Sciences, is a member of the Board of Directors of the Virginia Association for Gifted, and is a Past President of the Virginia Educational Research Association. She is also an Associate Editor for Gifted Child Quarterly and editor for the GCQ Methodological Briefs.
Table 1. FOI Five-Component Framework

Component          Definition

Adherence          Whether the unit is delivered as designed.

Exposure           Whether the number of lessons taught, the length
                   of time spent by the teacher, and the type of
                   concepts and skills delivered match the intent
                   of the unit developer.

Program            The existence or absence of "critical features"
differentiation    that distinguish the unit from standard curriculum.

Quality of         Whether a teacher implements a unit in a manner
delivery           consistent with the "theoretical or pedagogical
                   ideals" and techniques embedded within the unit.

Participant        Whether the extent of student involvement or
responsiveness     engagement in the unit's activities matches the
                   intent of the unit developer.

Note: FOI = fidelity of implementation.

Table 2. Codes and Subcodes for Qualitative Data Analysis

Codes                       Subcodes          Subcodes description

Autonomy (AUT)--            Participant       Degree to which
Autonomy for this study     commitment        teacher feels
broadly refers to the       (AUT-P)           personally
level of administrative                       committed as
support for and personal                      participant in
commitment to                                 research
participate throughout
implementation of the       Support           Degree to which teacher
WWIGE study                 (AUT-S)           believes she has what is
                                              needed in terms of
                                              materials, space,
                                              technology, etc.

                            Administration/   Degree to which
                            principal         teacher
                            (AUT-A/P)         believes she is
                                              supported by

Perceptions of time         Program           Degree to which
(logistics; T)--            delivery (T-PD)   pull-out versus
Perceptions of time                           self-contained program
refers to the degree to                       delivery impacted
which a teacher felt she                      perceptions of time
was or was not impacted
by time constraints
implementation of the
WWIGE study

                            Testing (T-T)     Degree to which
                                              district/state testing
                                              requirements impacted
                                              perceptions of time

                            Number of         Degree to which number
                            students (T-NS)   of students impacted
                                              perceptions of time

                            Library (T-L)     Degree to which library
                                              resources (librarian,
                                              books, and technology)
                                              impacted perceptions of

Teacher expectations        Fun (TE-F)        Degree to which teacher
(TE)--Teacher                                 expects student learning
expectations refer to a                       requires having fun
teacher's beliefs about                       during intervention
the capability of her
students to understand
and access WWIGE
curriculum. These are
tied to the perceived
cognitive, affective,
and behavioral traits of

                            Share (TE-S)      Degree to which teacher
                                              expects students must
                                              share academic
                                              experiences during
                                              intervention to

                            Equal (TE-E)      Degree to which teacher
                                              expects students must
                                              have equal academic
                                              experiences during
                                              intervention to learn

                            Enrichment        Degree to which teacher
                            (TE-En)           expects students need
                                              supplemental educational
                                              experiences to learn

Professional experience     Comfort level     Degree to which teacher
(PE)--Professional          (PE-CL)           believes professional
experience refers to                          experiences have
overall professional                          prepared him or her to
experiences of teacher                        teach WWIGE units
which impact
                            Groups (PE-G)     Degree to which
                                              teacher's professional
                                              experiences inform
                                              decisions about
                                              grouping strategies

                            Formative         Degree to which
                            assessments       teacher's professional
                            (PE-FA)           experiences inform
                                              decisions about when/
                                              how to use formative

                            Management        Degree to which
                            (PE-M)            teacher's professional
                                              experiences inform
                                              classroom management
                                              decisions which impact
                                              implementation of WWIGE

Note: WWIGE = What Works In Gifted Education.

Table 3. Model Summaries for Poetry Unit

                                           Unconditional model

Parameters                                  estimates      SE

Intercept ([[gamma].sub.00])               24.323 ***    0.511
ITBS ([[gamma].sub.10])
Teacher adherence ([[gamma].sub.01])
Residual variance ([[sigma].sup.2])         19.807 **    2.069
Intercept variance ([[tau].su.sub.00])       3.588 *     2.112
Deviance statistics                               2,008.260

                                              Level 1 model

Parameters                                  estimates      SE

Intercept ([[gamma].sub.00])               24.173 ***     .766
ITBS ([[gamma].sub.10])                     0.019
Teacher adherence ([[gamma].sub.01])
Residual variance ([[sigma].sup.2])        17.998 ***    2.147
Intercept variance ([[tau].su.sub.00])      6.00         5.018
Deviance statistics                               1,621.142

                                                Full model

Parameters                                  estimates      SE

Intercept ([[gamma].sub.00])               25.293 ***    0.684
ITBS ([[gamma].sub.10])                       0.019      0.012
Teacher adherence ([[gamma].sub.01])        1.983 **     0.906
Residual variance ([[sigma].sup.2])        18.118 ***    2.153
Intercept variance ([[tau].su.sub.00])      4.479        4.291
Deviance statistics                               1,611.142

Note: ITBS = Iowa Tests of Basic Skills.
* p < .05. **p < .01. ***p < .001.

Table 4. Model Summaries for Research Unit

                                           Unconditional model

Parameters                                  estimates      SE

Intercept ([[gamma].sub.00])               25.208 ***    0.622
ITBS ([[gamma].sub.10])
Teacher adherence ([[gamma].sub.01])
Residual variance ([[sigma].sup.2])         14.669 **    2.370
Intercept variance ([[tau].su.sub.00])       7.161 *     3.531
Deviance statistics                               1,882.45

                                              Level 1 model

Parameters                                  estimates      SE

Intercept ([[gamma].sub.00])               25.409 ***    0.731
ITBS ([[gamma].sub.10])                     0.004        0.008
Teacher adherence ([[gamma].sub.01])
Residual variance ([[sigma].sup.2])        11.529 ***    2.125
Intercept variance ([[tau].su.sub.00])      8.707 *      4.198
Deviance statistics                               1,420.31

                                                Full model

Parameters                                  estimates      SE

Intercept ([[gamma].sub.00])               27.032 ***    0.755
ITBS ([[gamma].sub.10])                     0.005        0.007
Teacher adherence ([[gamma].sub.01])        3.380 **     1.292
Residual variance ([[sigma].sup.2])        11.527 ***    2.132
Intercept variance ([[tau].su.sub.00])      5.905        2.744
Deviance statistics                              1,414.194

Note: ITBS = Iowa Tests of Basic Skills.
* p < .05. ** p < .01. *** p < .001.

Table 5. Teaching Experience of Participants

                            High-fidelity     Low-fidelity
                            group (n = 11)    group (n = 13)

                              M        SD       M        SD

Years of teaching total     14.25     8.78    20.46     9.66

Years of teaching third      4.44     3.33     8.58     6.29

Years of teaching gifted     3.87     1.96     6.62     6.27

Table 6. Number of Teachers With Advanced Degrees

                             High-fidelity     Low-fidelity
                             group (n = 11)   group (n = 13)

Master's degree in gifted          0                2

Master's degree in other           6                8
COPYRIGHT 2011 Sage Publications, Inc.
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2011 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Azano, Amy; Missett, Tracy C.; Callahan, Carolyn M.; Oh, Sarah; Brunner, Marguerite; Foster, Lisa H.
Publication:Journal of Advanced Academics
Article Type:Report
Geographic Code:1USA
Date:Nov 1, 2011
Previous Article:A reliability generalization of the overexcitability questionnaire--two.
Next Article:High-ability students' time spent outside the classroom.

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters