Printer Friendly

Tier 2 supports to improve motivation and performance of elementary students with behavioral challenges and poor work completion.

Abstract

We offer a methodological illustration for researchers and practitioners of how to conduct a development study consistent with the parameters delineated by the Institute of Education Sciences (IES; U.S. Department of Education [USDE], 2010) to explore the utility of an existing Tier 1 intervention applied as a Tier 2 support within a three-tiered model. Although the present study was not submitted to or funded by IES, it was developed as a pilot study to address two objectives. First, we explored the utility of a character development intervention focused on improving goal setting, decision making, and self-management skills to meet the behavioral and instructional needs of fourthgrade students who have behavioral challenges and limited work completion, but who have adequate literacy skills. The main questions pertained to treatment integrity and social validity, with a goal of having a fully-developed intervention ready for testing as part of an efficacy trial. Specific research questions were as follows: To what extent was this type of Tier 2 support implemented within the regular school day with integrity? What did teachers and students think about the goals, procedures, and outcomes? Did this initial evidence suggest this intervention resulted in students learning the content taught (proximal outcomes) translating into improved skill sets (intermediate outcomes) and ultimately improved motivation (distal outcomes)? Second, we conducted this study to field test the required components specified in the IES 2010 RFA to assist in the development of a proposal to be submitted to IES for funding.

KEYWORDS: Tier 2, Intervention, Character Education, Emotional and Behavioral Disorders.

Students with emotional and behavioral disorders (EBD) are a diverse group known for pronounced behavioral and social skill excesses and deficits (Walker, Ramsey, & Gresham, 2004). Not only are behavioral and social skill deficits of concern for these students, but academic underachievement is often a concern as well (Nelson, Benner, Lane, & Smith, 2004). Academic performance is negatively impacted by low academic engagement, limited motivation, and limited self-determined behaviors (Carter, Lane, Crnobori, Bruhn, & Oakes, 2011; DiPerna & Elliott, 2002). As such, students with cooccurring needs represent one of our most at-risk student populations (Reinke, Herman, Petros, & Ialongo, 2008). Conservative estimates suggest 6% of school-age students have EBD (Kauffman & Landrum, 2009). With only about 1% of these students served by special education programs, a need exists to proactively identify and support students within general education. Three-tiered prevention models address this need and are being adopted by school-sites and districts across the country providing an early response to the needs of students before these difficulties become persistent.

Benefits of a Comprehensive, Integrated, Three-Tiered Model of Prevention

A comprehensive, integrated, three-tiered (CI3T) model of prevention offers benefits for supporting students academically, behaviorally, and socially. The CI3T model resembles practices of the Response to Intervention (RtI; D. Fuchs & Fuchs, 2006) model which identifies and responds to students' academic needs and the Positive Behavior Interventions and Supports (PBIS; Sugai & Homer, 2009) model attending primarily to students' behavioral and social skill needs. The CI3T model of prevention combines practices of RTI and PBIS models offering an approach with a comprehensive primary plan and use of behavioral and academic data to design, implement, and evaluate interventions (Tier 2 and Tier 3) supporting students across all domains (Lane, Oakes, & Menzies, 2010; McIntosh, Bohanon, & Goodman, n.d.). Integrated models are particularly beneficial for students with or at risk for emotional or behavioral difficulties since they provide a seamless delivery of increasingly-intensive supports (McIntosh, Chard, Boland, & Homer, 2006; Stewart, Benner, Martella, & Marchand-Martella, 2007).

School-site teams design a CI3T model of prevention based on the unique needs and priorities of their students and school communities. The primary plan addresses priorities of academic, behavioral, and social skill development for all students. This approach allows for defined responsibilities and participation for students, teachers, administrators, parents, and community members across the three domains (academics, behavior, and social skills). For example, administrators may be responsible for providing the instructional materials and professional development for teachers to help students meet academic expectations. Further, administrators may be charged with communicating academic priorities to parents and the community. Teachers are commonly expected to plan and deliver differentiated instruction addressing the state, district, or Common Core State Standards (National Governors Association Center for Best Practices [NGA Center] and the Council of Chief State School Officers [CCSSO], 2011) and conduct ongoing assessment of student performance. Teachers are also expected to explicitly teach the defined behavioral expectations and provide reinforcement to students meeting those expectations. Schools select a curriculum for character or social skill development that is taught to all students as part of the school's general education instruction. Once these practices are established and integrity of delivery is monitored ensuring all students have the opportunity to benefit from the primary plan, more intensive supports are considered for some students (Lane, Kalberg, & Menzies, 2009).

At Tier 2, students who are in need of additional instruction or supports are identified using academic and behavioral measures (e.g., curriculum-based measures, formative assessments, report card grades, attendance, behavior screening tools, and office discipline referrals [ODRs]). School-site teams delineate the low intensity or small group supports they have the resources to offer students in an intervention grid, defining the intervention, the criteria for being considered for the intervention, how students' progress will be monitored for response, and the exit criteria for deciding whether students have benefitted from the intervention and no longer require it or are in need additional supports (see Lane, Menzies, Oakes, & Kalberg, 2012). Tier 2 interventions may be existing supports at the school level (e.g., tutoring, peer-supported learning) or may require additional professional development or resources (e.g., intensive reading programs, Check-In/ Check-Out [CICO] via School-wide Information System [SWIS]; May et al., 2010).

Tier 3 supports are also identified and developed by the school-site team. Tier 3 or tertiary supports are the most intensive supports available to students at school-sites and are often reserved for those with the greatest need. Tier 3 supports may be delivered through one-to-one instruction or counseling or in small groups at higher frequency or longer duration than in Tier 2. For example, students may receive small group social skills instruction five days per week compared to three days at Tier 2 or for 60 min per day compared to 30 min at Tier 2. School-sites may elicit assistance from district-level support specialists at Tier 3 (e.g., behavior specialists, literary coaches). Tier 3 supports may require additional resources and professional development for teachers and other school-based personnel who provide direct supports to students.

A Need to Focus on Tier 2

Within the three-tiered framework, Tier 2 interventions and supports hold promise for intervening before the behavioral and academic challenges of students become resistant to intervention efforts (Walker et al., 2004). Tier 2 interventions are less intensive in terms of time and resources and can often be delivered to students by general education teachers within the structure of the school day during a designated intervention block of time (e.g., Oakes, Mathur, & Lane, 2010). Much attention has been devoted to primary and tertiary supports within the context of three-tiered models of prevention; less research examines supports and interventions offered at the secondary level of prevention (Bruhn, Lane, & Hirsh, 2012; Mitchell, Stormont, & Gage, 2011). Integrated models pose a particular benefit at Tier 2 given that behavioral, social, and academic concerns influence one another (Wentzel, 1993). For example, the strongest predictor of student achievement, after current level of academic achievement, is academic motivation followed by study skills (for older students) and then social skills (DiPerna, Volpe, & Elliott, 2002). So it follows that if students have the academic ability (acquired skills) but are not performing academically (e.g., not completing work), interventions should target improving academic enablers (e.g., interpersonal skills, motivation, engagement, study skills; DiPerna & Elliott, 2000). The critical nature of the relation between academic achievement and academic enablers supports the use of tiered models of prevention integrating academic, social, and behavioral supports in the school setting. Thus, interventions addressing improved student outcomes in all three areas must be explored.

Recently, Mitchell and colleagues (2011) conducted a review of the literature examining the evidence for Tier 2 interventions specifically within three-tiered models of prevention. The review focused on the extent to which fidelity of Tier 1 was assessed prior to implementing Tier 2 interventions; student characteristics and identification methods; targeted outcomes, types of intervention, and effectiveness within tiered models; and issues of sustainability and social validity. All included studies met several criteria, two of which were: (a) interventions were conducted within three-tiered models; and (b) outcomes were measured for social or behavioral, or academic and social or behavioral outcomes, thus the review excluded studies reporting only academic outcomes. Thirteen studies met criteria and were included in the review, all of which were conducted within the last 10 years. About half (n = 6) of the studies reported levels of treatment integrity of the primary plan, predominately reporting Schoolwide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Homer, 2001) scores. Further, few (n =3) of the studies reported a description of Tier 1 practices. Students were identified for participation in these studies by referral (parents, teachers, or school teams) or school data (e.g., ODRs or behavior screening results). In most cases, the nomination of students was vaguely defined (e.g., no set criteria for inclusion were provided). None were reported to have used school data alone to determine participation; however, three studies used behavior screening tools to determine student need. The overwhelming majority (12 out of 13) targeted behavioral outcomes. Only one identified study (Lane et al. 2002) measured academic and behavioral outcomes. Decreased problem behaviors, increased engagement or social skills, and improved academic performance were found in 12 studies. Limited information on treatment integrity was reported across studies; in contrast, social validity results were reported in 9 studies predominately from the student and teacher perspectives.

Collectively, the results of the review (Mitchell et al., 2011) found initial evidence for the benefit of Tier 2 supports (behavioral, social, and combined with academics). Yet, further inquiry is needed to determine which of these Tier 2 supports have sufficient evidence to establish them as an evidence-based practice (Cook, Tankersley, & Landrum, 2009).

We must continue to develop the scope of these Tier 2 supports for assisting students in acquiring the skill sets necessary to engage more fully in school by using practices with sufficient evidence to warrant the investment of teachers' and students' time. To date, limited attention has been devoted to secondary supports for students who have the requisite skills needed for the instructional activities, but who lack the motivation and self-regulation skills to participate.

Knowledge base: Positive Action. Positive Action[R] is a Tier 1 character development curriculum shown to have positive effects on improving behavior and academic outcomes of students K-12 (USDE, 2007b). The Positive Action program uses scripted lessons, role-play, songs, games, activity books, and discussion to promote positive feelings and habits. The basis of the program is that when students feel good about themselves they "think and do positive actions" (USDE, 2007b, pp.1). Positive Action kits contain all of the materials needed to teach the curriculum. There are 6 units with 140 lessons per grade level which may be selected for differing levels of implementation allowing for flexibility based on needs and resources (Positive Action, 2004). In addition to teaching materials, resources are available to measure pre and post skill levels (e.g., Skills for Success), gather perceptions of teachers and parents, and structure lessons to achieve fidelity of implementation.

Positive Action has been evaluated as a Tier 1 intervention and shown to have improved standardized academic outcomes and decreased office discipline referrals (Flay, Allred, & Ordway, 2001). A large-scale, matched-sample, archival data study found these improvements maintained during middle and high school (Flay & Allred, 2003). In a similar experiment, Positive Action implementation at Tier 1 was found to effect improvements in attendance and standardized academic outcomes and reductions in suspensions and retentions, all with moderate to large effect sizes (Snyder et al., 2010). In another large randomized trial of students in urban elementary schools (gr. 3--5), significant reductions were found in self-report of substance abuse, violence-related behaviors, and bullying behaviors for the Positive Action schools compared to control schools (Li et al., 2011). Evidence shows strong support for this curriculum as a Tier 1 program with positive effects on students' behavioral, academic, and social outcomes. The utility of Positive Action as a Tier 2 support has yet to be tested, but poses an interesting next step in this line of inquiry.

Purpose

In characterizing education research, Shavelson and Towne (2002) suggest questions can be conceptualized into three, interrelated categories: descriptive inquiry (exploring the question of, "What is happening?"), causal inquiry (exploring the question of, "Is there an effect?", or "Does X cause Y?"), and process or mechanism inquiry (exploring the question of, "Why or how is it happening?"). IES has an established goal structure offering a framework for addressing these three categories of questions. The goal structure consists of the following: Goal 1 Exploration, Goal 2 Development and Innovation, Goal 3 Efficacy and Replication, Goal 4 Effectiveness, and Goal 5 Measurement.

In terms of intervention research, the sequence of the goal structure is potentially very useful. For example, Goal 1 inquiry can be conceptualized as providing a forum for descriptive inquiry which could address a number of functions. For example, such projects could "(a) generate hypotheses regarding the potential causal relations between malleable factors and education outcomes, (b) contribute to theories of change for education interventions, (c) contribute to the development of interventions that can improve student outcomes or identify the conditions that are associated with better implementation of interventions, and (d) identify potentially beneficial interventions." (USDE, 2011b,. p. 7)

Information gleaned from Goal 1 inquiry can be used to inform Goal 2. Development and Innovation inquiry focuses on developing "innovative education intervention and improving existing education interventions" which includes curricula, technology, instructional approaches, education practices, programs as well as policies. Interventions are to be developed using an iterative process defined as "a cycle of development, implementation, observation, and revision" (USDE, 2011b, p. 7). The iterative process, which begins with a clearly-articulated theory of change grounded in the literature, continues until evidence is established to indicate the intervention can be implemented as intended (evidence of treatment integrity), is feasible within the context of an authentic setting (evidence of social validity), and offers promise that implementing the intervention will yield the desired outcomes (evidence of the promise of achieving the desired student outcomes). The goal of such interventions is to improve student outcomes when said "intervention" is implemented in actual educational contexts (e.g., classrooms; USDE, 2011b). The end result is a fully developed intervention ready for efficacy testing in as part of Goal 3 inquiry.

Goal 3 Efficacy and Replication inquiry offers a framework for causal inquiry, to test a newly developed practice or strategy or well established intervention implemented in a new context (e.g., with a new group of students, by a different interventionist, or other variation). Goal 4 Effectiveness inquiry provides a forum for determining "whether or not fully developed interventions, with prior evidence of efficacy, produce a beneficial impact on student outcomes relative to a counterfactual when implemented in authentic education delivery settings" (USDE, 2011b, p. 8).

In the current paper, we offer a methodological illustration for researchers and practitioners of how to conduct a development study (Goal 2) consistent with the parameters delineated by the Institute of Education Sciences (USDE, 2010) to explore the utility of an existing Tier 1 intervention applied as a Tier 2 support within a three-tiered model. Although the present study was not submitted to or funded by IES, it was developed as a pilot study to address two objectives. First, we examined the utility of a character development intervention focused on improving self-regulation skills to promote goal setting, decision making, and self-management skills to meet the behavioral and instructional needs of fourth-grade students who have behavioral challenges and limited work completion, but who have adequate literacy skills. The main questions explored pertained to treatment integrity and social validity, with a goal of having a fully-developed intervention for testing as part of an efficacy trial (Goal 3) as outline by parameters established by the IES. Specific research questions were as follows: To what extent was this type of Tier 2 support implemented within the regular school day with integrity? What did teachers and students think about the goals, procedures, and outcomes? Is there initial evidence to suggest this intervention resulted in students learning the content taught (proximal outcomes) translating into improved skills sets (intermediate outcomes) and ultimately improved motivation (distal outcomes)? Second, we conducted this study to field test the required components specified in the IES 2010 RFA to assist in the development of a proposal to be submitted for funding.

Method

Participants

Nine fourth-grade students participated in this study (see Table 1). They attended an inclusive, rural elementary school in a southeastern state. They were identified by school-wide data as being capable yet underperforming academically and having behavioral concerns (inclusion criteria to follow). All students were White and the majority were male (6 boys). Students ranged in age from 9.03 to 10.02 years (M = 9.29). An estimate of cognitive ability was obtained using a short form of the Wechsler Intelligence Scale for Students--Fourth Edition (WISC-IV; Wechsler, 2003) which included vocabulary and block design subtests (split half reliabilities of .82 for both). Sattler's (1991) conversion formula was used to obtained a deviation quotient (r = 0.91). Average IQ was 101.93 (SD = 9.83); which was in the average range. Students also performed in the average range in terms of reading comprehension skills (M = 95.00; SD = 5.61). Two (22.22%) students received special education services under the categories of Other Health Impaired (OHI) and Speech and Language Impairment (SLI) (IDEA, 2004). One student was referred to the multidisciplinary team before the onset of the intervention for concerns about attention and hyperactivity, yet was not found eligible for services.
Table 1 Student Participants' Characteristics
 Intervention
 Group

 Positive Book Study (n
 Action (n = 5)
 = 4

Variable Level n (%) M (SD) n (%) M(SD)

Gender Male 3 (75.00) 3 (60.00)

 Female 1 (25.00) 2 (40.00)

Ethnicity White 4 5 (100.00)
 (100.00)

Age 9.31 9.27
 (0.46) (0.41)

Special 2 (50.00) 0 (00.00)
Education

IQ 97.10 105.80
 (10.05) (8.70)

WJ Passage 91.50 97.80
Comprehension (4.51) (5.07)

Student Risk
Screening

Sale (SRSS)

Pre 7.50 8.20
 (3.70) (3.11)

 Moderate

 Pre 3 (75.00) 2 (40.00)

 High

 Pre 1 (25.00) 3 (60.00)

 Total (n =
 9)

Variable Level N (%) M(SD)

Gender Male 6 (66.67)

 Female 3 (33.33)

Ethnicity White 9
 (100.00)

Age 9.29
 (0.4O)

Special 2
Education (22.22)

IQ 101.93
 (9.83)

WJ Passage 95.00
Comprehension (5.61)

Student Risk
Screening

Sale (SRSS)

Pre 7.89
 (3.18)

 Moderate

 Pre 5 (55.56)

 High

 Pre 4 (44.44)

Note. IQ refers to estimated intelligence
quotient. SRSS refers to Student Risk
Screening Scale (Drummond, 1994). One
student in the Book Study condition was
referred to the multidisciplinary team
prior to the intervention.


Setting

The school was a large rural: distant elementary school located near a large southern city (USDE, 2011a). The school served students in PK--5th grades and was in good standing according to No Child Left Behind requirements, meeting annual yearly progress criteria (Tennessee Department of Education [TNDE], 2011). The student body was predominately White (96.4%; n = 669), with 16.7% of students economically disadvantaged. About 12% of students were eligible for free or reduced-priced lunches. The district served 9.06% of students across 37 schools in special education programs through a full inclusion model (TNDE, 2011).

The elementary school was in their fifth year of implementing a CI3T model of prevention. The CI3T plan included intervention and monitoring of student progress at all three tiers. The primary plan included academic, social, and behavioral components. The academic component involved teaching of academics by planning and delivering instruction aligned with state standards and district learning targets. An emphasis was placed on differentiated instruction and the use of formative assessment to adjust instructional levels and pacing. As part of their social skills component, teachers provided monthly social skill lessons adapted from the Social Skills Intervention Guide (Elliott & Gresham, 1991). In terms of the behavioral framework, school-wide behavioral expectations of Respect, Responsibility, Cooperation, and Best Effort, were explicitly taught to students and reinforced through a school-wide ticket system tied to reward assemblies and a school store. The school's leadership team first developed this plan in the 2005-2006 school year through a year-long training series offered by a local university. In 2010-2011, at the time of this study, the school's leadership team again attended a year-long series to revise their CI3T plan and update their current knowledge of evidence-based practices related to their plan.

The school-site leadership team had systems in place to monitor student progress. They used academic benchmarking and formative assessments (e.g., AIMSweb, ThinkLink) in conjunction with behavior screening data (Student Risk Screening Scale [SRSS]; Drummond, 1994; descriptions to follow) to determine students in need of Tier 2 and 3 supports. Students participated in interventions during a 30 min intervention block built into the regular school day.

Treatment integrity of the primary plan was measured through a partnership with Project Support and Include, a state-funded technical assistance project at Vanderbilt University. Treatment integrity of the primary plan was measured in three ways: (a) direct observation from two perspectives - outside observer (research assistant) and teacher (30 min observation rated on the same criteria); (b) self-assessment by teachers of their use of practices and procedures in the school plan from the first day of the academic year to mid-spring semester; and (c) the School-wide Evaluation Tool (SET; Sugai et al., 2001). Direct observations showed teachers were utilizing the plan nearing optimal integrity (80%; Homer et al., 2004) with 79.32% (teacher perspective) and 72.70% (outside observer) of the elements in place. The teachers' self-assessment was reported at 81.76% and the SET measured 91.43% integrity of the PBIS components. Social validity as indicated by spring-semester scores were high, with an overall mean of 90.59% (SD = 8.80) on the Primary Intervention Rating Scale (PIRS; Lane, Kalberg, Bruhn et al., 2009). The percentage of students at low risk as measured by the SRSS increased from 81.3% in Fall 2005 to 90.06% in the Fall of 2010. Thus, the primary plan was implemented with fidelity, viewed favorably by their faculty, and the level of students' behavioral risk was decreasing.

Participants Selection Criteria

Screening. School-wide data, collected as part of regular school practices approximately 6-8 weeks after the school year began, were used to identify students for possible participation. Fourth and fifth-grade students were invited to participate if they (a) scored in the high or moderate risk range on the SRSS, (b) had average or above average reading skills according to AIMSweb (Pearson, 2010) or Thin-kLink (Discovery Education, 2000), (c) earned a progressing or needs improvement in one or more of the personal and growth assessment areas on the first nine-weeks report card, and (d) were nominated by teachers as having difficulty completing assignments according to teacher-maintained records of class and homework. The screening measures are described below.

SRSS. The SRSS (Drummond, 1994) is a systematic screening tool designed to identify elementary-age students at risk for antisocial behavior. Teachers rated each student in their class on seven items: steal; lie, cheat, sneak; behavior problems; peer rejection; low academic achievement; negative attitude; and aggressive behavior using a 4-point Likert-type scale (0= never, 1 = occasionally, 2= sometimes, 3 = frequently). Total scores ranged from 0-21, yielding three categories: low risk (0-3), moderate risk (4-8), or high risk (9-21). Ratings of an entire class are completed by teachers in about 10-15 min. This measure has strong internal consistency (>.80), test-retest stability, and predictive validity (Oakes, Wilder et al., 2010).

AIMSweb. AIMSweb (Pearson, 2010) is a benchmark and progress monitoring system available for grades kindergarten through eight. AIMSweb generates data to identify students who may not make adequate progress without supplemental instruction. The reading measures used to monitor progress in grades two through eight are Oral Reading Fluency (Reading--Curriculum-Based Measurement [R-CBM]) and MAZE, a measure of comprehension. AIMSweb benchmarking probes are administered 3 times per academic year at equal intervals beginning in the first few weeks of the school year. AIMSweb R-CBM probes have adequate reliability with alternate-form reliabilities for the mean score of three probes, as used for benchmarking, ranging from .92-.97 and test-retest reliabilities ranging from .88-.95 (Daniel, 2010). Inclusion criteria required students to score in the Established, Upper Established, or Accelerated levels during the Fall 2010 benchmark.

ThinkLink. ThinkLink (Discovery Education, 2000) is a formative assessment available for grades K-12 measuring student achievement identifying areas for additional instruction and predicting student performance on state achievement assessments (80 - 90% accuracy; Discovery Education, n.d.). ThinkLink assessments occur three times per year. Individual student scores are reported as percent correct and in categories of proficiency (Above Proficient, Proficient, and Below Proficient) according to grade level standards. Developers report adequate reliability of the assessment items with Cronbach's alpha = .86 for fourth grade reading and language items (Discovery Education, n.d.). Inclusion criteria required students to score in the proficient or above proficient range.

Students' identified and random assignment. Of the 252 fourth-(n = 120) and fifth-grade (n = 132) students screened, 14 (13 fourth-grade and 1 fifth-grade) students met all four inclusion criteria. All fourth-grade (n = 5) and fifth-grade (n = 1) teachers invited to participate consented. Therefore, parental consent was sent home to parents of 14 students. Parental consent and student assent were secured for 11 (78.57%) participants. Initially, the intent had been to have two intervention groups (one for fourth-and one for fifth-grade students) for each condition (Positive Action and Book Study). Because only one fifth-grade student consented, it was necessary to include the fifth-grade student in the fourth-grade group during the fourth-grade intervention block. However, the teacher of the fifth-grade student was unable to adjust the instructional day for the student to participate in the intervention without missing core academic content. The remaining 10 students were randomly assigned to one of two conditions: experimental (n = 5; Positive Action) or control (n = 5; Book Study). Parents of one student in the Positive Action condition withdrew their child from the study indicating she was not enjoying being in the small group. Nine students completed the study: 4 in the Positive Action and 5 in the Book Study control condition.

Intervention Procedures

Positive Action. Positive Action (2008) was chosen as the intervention because it was being piloted by the school district as a possible PK--12 Tier 1 character education program. Positive Action is one of four programs or practices showing medium to large positive effects on personal/social development as reviewed by What Works Clearinghouse (USDE, 2007a). Positive Action curriculum is available for purchase in grade level kits from the publisher. These kits have all materials needed to implement the intervention such as a training DVD, lesson plan manual, student books, activity sheets, music, posters, and manipulatives (e.g., pipe cleaners, cotton balls). Lessons for each grade level are organized into seven units: (1) Self-Concept, (2) Physical and Intellectual Positive Actions for a Healthy Self, (3) Managing Yourself Using Social/Emotional Positive Actions, (4) Getting Along with Others Using Social/Emotional Positive Actions, (5) Telling Yourself the Truth Using Social/Emotional Positive Actions, (6) Improving Yourself Continually Using Social/Emotional Positive Actions, and (7) a Positive Action Review.

A subset of lessons was chosen for this Tier 2 intervention. Units 2, 3, and 6 were selected by the research team (with over 20 collective years of teaching experience in general and special education) as being most closely aligned to the behaviors of interest: improved academic performance by improving self-regulation skills (e.g., goal setting, self-management, decision making), engagement, and motivation. Unit 2 contained lessons such as the importance of learning something new, being curious, using resources, and strengthening your memory. Unit 3 taught students to use their talents and resources (e.g., time, energy, possessions, money, talents, thoughts, actions, and feelings) for succeeding in school. Unit 6 targeted motivation and self-management including: self-improvement, persistence, and turning problems into opportunities.

Of the 21 Positive Action intervention lessons taught, nine lessons were revised by the research team to combine two lessons from the published curriculum into one lesson. These lessons were combined to maximize the content of lessons that were taught during the intervention. The lesson sequence remained as designed by the publishers. Transitions were planned to accommodate for omitted lessons, as needed. Further, lessons were modified to fit a direct instruction format with the following components: learning objective, starter activity to activate prior knowledge, modeled instruction, guided practice, independent practice, assessment and closing activity (Hunter & Hunter, 2004; Sugai & Lewis, 1996). Behavioral supports were embedded in the lessons by setting expectations at the start of the lesson, giving behavior specific praise paired with the school-wide reinforcement ticket system, and students setting and monitoring the achievement of a daily goal. Activities were included to program for generalization.

A typical lesson began with the daily schedule displayed on the board for students as a visual prompt. Students began by selecting a behavioral goal for the day that aligned with school-wide behavioral expectations (e.g., What behavioral goal would you like to work on today: respect, responsibility, cooperation, or best effort?). The instructor activated background knowledge through a warm-up question or activity from the previous day's lesson. Then a new topic was taught and students engaged in an activity on that topic. For example, when learning about Managing Your Time students created a schedule for their time after school and for Strengthening Your Memory students played a memory game taking turns repeating names for each letter of the alphabet. Students also engaged in stories or skits, such as The Little Engine that Could when studying the concept of persistence. After the new concept and activity, students set personal goals and wrote them on goal cards. For example, when learning about Strengthening Your Memory, students made a goal to remember one fact learned in a class the following day. Students were praised for meeting their goals and would celebrate by drawing a checkmark on their goal card. Finally, the lesson closed with an assessment in the form of closing questions and examples for applying the new skills in class before the group met for the next session.

The goal of the Positive Action intervention at the Tier 2 level was to improve academic performance and motivation in the general education classroom for these students whom exhibited the academic skill-sets to do so. Lessons were designed to make explicit connections between the Positive Active curriculum and the group activities and objectives. The daily personal goals that students set served to facilitate generalization of the Positive Action skills. For example, for the lesson topic, Taking Advantage of Resources in your Life, students set goals such as using a dictionary when reading a word they did not know or using the internet after school to find information about their future job. Students received reinforcement (e.g., school-wide PBIS tickets and behavior specific praise) for using the skills in the intervention group and sharing evidence of generalizing the skills (e.g., Last night I ...).

Control condition. Students in the control group participated in a Book Study to account for issues of time in a small group instructional setting and teacher attention. The students' teachers were asked for recommendations of an appropriate book for the group. They suggested three grade-level books, one of which was selected: the Tale of Despereaux (DiCamillo, 2003). The book was selected for its interest, reading level, length, and richness of the vocabulary (Beck, McKeown, & Kucan, 2002). The Tale of Despereaux: The Graphic Novel (Smith & Tilton, 2008) was also used in the Book Study as a supplemental text to examine text related images. Lesson objectives were matched to the state's curriculum standards for fourth grade reading. Students engaged in reading practice daily by choral reading, reading with a partner, or silent reading. The teacher also frequently read sections of the book aloud to the group to model fluency, prosody (Dowhower, 1991), and to engage students in the text.

Intervention lessons focused exclusively on reading skills such as literary devices (i.e. onomatopoeia, metaphors, and similes) and avoided discussions related to character development avoiding possible overlap (also referred to as treatment contamination) with the Positive Action lesson objectives. The instructor's notes prompted her to support student behavior by greeting students, maximizing instructional time, differentiating activities, using a positive tone, and using behavior-specific praise.

Lessons followed the same format as the Positive Action lesson plans. Lessons began with students choosing a behavioral goal for the day (selected from the school-wide expectations), followed by starter activities for activating prior knowledge, modeled instruction, guided practice, independent practice, and ending with an assessment and closing activities. The teacher posted a daily visual schedule so students would know the activities and schedule for each lesson. Lesson topics were vocabulary development, point of view, predicting, ordering of events, comparing and contrasting, summarizing, and dramatic reading. Each day, the previous readings were summarized. To facilitate vocabulary development, a word wall was constructed by students throughout the intervention using drawings, collages, and writing. Reading of the text and vocabulary development activities were in each lesson, as well as one new skill activity. The lessons often included hands-on activities such as using highlighting tape to locate interesting words in text, sorting and matching activities with note-cards, creating a collage, performing sections of the book, and singing songs. Choice was built into most activities to allow students to show their strengths and preferences (Lane, Menzies, Bruhn, & Crnobori, 2011). For example, when art activities were planned, students had the choice of developing their art through drawing, using magazine pictures, and/or using premade line drawings. Lessons ended with a check for understanding or other assessment activity. For example, the instructor ended one lesson by asking students to add an additional event from the story onto the timelines they constructed during the lesson. After each lesson, students were reinforced for meeting their behavioral goal with a PBIS ticket.

Dosage and schedule. Each condition consisted of 21 lessons. Interventions were taught three days per week for eight week. Sessions >were held during the fourth grade intervention block from 1:40-2:10 (30 min sessions). Accounting for schedule changes due to inclement weather, the total dosage was 10.5 instructional hours between January 4 and March 3, 2011.

Training procedures. The research team attended a two-day training with the Positive Action developer, Carol Allred, as guests of the participating school district. The purpose was to learn about the program for supporting schools using the program as the primary prevention social skills/character development component of their CI3T model of prevention. Two members of the research team also attended a half-day training at a pilot school site that was also led by the program developer. Book study lessons were written by a Master's student with a degree in English education and a researcher with more than 10 years of teaching experience in elementary reading. A Master's student with teaching certificates in elementary and special education was the instructor for students in the Book Study group and a Master's student with a teaching certificate in special education taught the students in the Positive Action group.

Intervention instructors practiced the Positive Action and Book Study lessons, using the treatment integrity checklists, with the research team. The research team worked collaboratively, brainstorming ideas for strengthening the daily activities and the pacing of each lesson. After lessons were fully developed, instructors taught three lessons each to the research team who gave immediate feedback. The feedback included input on adhering to the lesson plan, using positive interactions with the students, and pacing. The research team completed the treatment integrity forms for the Positive Action (Figure 1) and Book Study (Figure 2) practice lessons for guiding the feedback. This also gave the reliability observers practice completing the treatment integrity forms accurately. Instructors made final revisions to the lessons based on feedback provided.

[FIGURE 1 OMITTED]

[FIGURE 2 OMITTED]
 Ratine Items to be
 Observed

 T 1 S 1 S 2 S 3 S 4 S 5

1. Teacher greeted
 students
 appropriately

2. Teacher
 reviewed
 school-wide
 expectations
 before starting
 the lesson.

3. Teacher
 displayed daily
 schedule of
 activities

4. Teacher
 presented a
 starter
 activity

5. Teacher
 activated prior
 knowledge

6. Teacher
 followed lesson
 procedures

7. Teacher
 presented an
 assessment/
 dosing activity
 through
 reviewing the
 main takeaway
 points from the
 lesson.

8. Teacher
 prompted
 journal use.
 Student used
 journals

9. Teacher
 programmed for
 generalization
 and students
 part id pat ed

10. Teacher had all
 materials
 prepared before
 lesson

11. Teacher
 provided
 opportunities
 to respond;
 Students
 Responded

12. Teacher kept
 positive tone
 during student
 interactions:
 Students
 responded in
 positive lone

13. Teacher made
 use of behavior
 specific
 praise

14. Teacher awarded
 tickets to
 students who
 followed the
 school-wide
 expectations.

Notes:

 Rating Items in be
 Observed

 T 1 S 1 S 3 S 4 S 5

1. Teacher
 greeted
 students
 appropriately

2. Teacher
 reviewed
 school-wide
 expectations
 he lore
 starting the
 lesson

3. Teacher
 displayed
 doily schedule
 of activities

4. Teacher
 presented a
 starter
 activity

5. Teacher
 activated
 prior
 knowledge

6. Teacher
 modeled
 instruction

7. Teacher
 provided
 opportunity
 for guided
 practice;
 Students
 participated
 in guided
 practice

8. Teacher
 provided
 feedback on
 guided
 practice

9. Teacher
 provided
 directions for
 independent
 practice;
 Students
 participate!
 in independent
 practice

10. Teacher
 presented an
 assessment/
 closing
 activity

11. Teacher had
 all materials
 prepared
 before lesson

12. Teacher
 provided
 opportunities
 to respond:
 Students
 Responded

13. Teacher kept
 positive tone
 during student
 interactions;
 Students
 responded in
 positive tone

14. Teacher made
 use of
 behavior
 specific
 praise

15. Teacher
 awarded
 tickets Hi
 students who
 followed the
 school-wide
 expectations.

16. Teacher was
 not discussed
 as a part of
 the lesson

Notes:


Treatment integrity procedures. Measuring treatment integrity was an integral component to the experiment. The following techniques were used to ensure Positive Action and Book Study instruction were delivered with fidelity. First, as mentioned above, instructors were instructed in how to teach the content of each intervention condition. Second, the research team held weekly meetings to discuss the interventions including implementing the lessons. The project director and principal investigator were also available by phone or in person daily if any issue needed attention (e.g., behavior management concerns, scheduling conflicts). Third, instructors completed daily treatment integrity checklists prompting each intervention component to measure intervention compliance. Fourth, once a week a second research team member (an outside observer, a masters' student who was a certified special education teacher) evaluated the lesson using the same treatment integrity form as the instructor.

Treatment integrity forms for both intervention conditions were aligned for a majority of components (n = 11), yet varied in areas in which a social skills curriculum would differ from a book study. For example, one item on the book study form not on the Positive Action form was Character was not discussed as a part of the lesson. This was monitored to ensure character development was not address in the control condition. Items were rated 1 or 0 if implemented by the instructor or not, respectively. Also, items in which student participation (e.g., Teacher provided opportunities to respond; Student responded) or teacher to student interaction (e.g., Teacher greeted students appropriately) were required were also measured for each student (with a 1 or 0). This was to ensure each student was engaged in the lessons and was receiving similar instruction and positive feedback. For both intervention groups, session integrity was computed by dividing the total number of components completed by the total number of components possible, multiplied by 100. Likewise, to measure the integrity of each student's participation, the total number of components the student was taught or participated in was divided by the total number of expected components, multiplied by 100.

Measures

In addition to screening measures, the study included descriptive measures, treatment integrity measures, social validity measures, and measures to assess student outcomes (proximal, intermediate, and distal). The research team members were trained to criteria in assessing and scoring each of the measures. Training presentations were developed with an overview of the measure, its purpose, and psychometrics, and procedures for administration, scoring, and reliability of scoring. Members of the research team with experience using these tools in research and school settings developed the training presentations. As part of training activities, research team members practiced giving the subtests three times with feedback from the trainers. Further, each team member was taught to enter data into Excel, score measures with the publisher's software, and ensure reliability of data (two independent scorers and checking 100% of entry). Practice was conducted on sample data and accuracy assessed. Research team members were trained to criteria (> .90).

Descriptive measures. Descriptive measures were administered to confirm the accuracy of the inclusion procedures and describe participant characteristics.

Woodcock Johnson III Test of Achievement: Passage Comprehension Subtest (WI-III; Woodcock, McGrew, & Mather, 2001). The WJ-III is a norm-referenced measure of achievement. Students were individually administered the passage comprehension subtest prior to beginning the intervention to confirm they were independent readers as both intervention conditions required students to read. The passage comprehension subtest has adequate psychometrics with one year test-retest reliability of .86 (ages 8 to 10).

Wechsler Intelligence Scale for Children, Fourth Edition (WISC-IV; Wechsler, 2003). Description provided in participants' description section. Subtests were administered individually to students prior to participating in the intervention.

Treatment integrity. Behavioral component checklists were used by the instructors (all sessions) and outside observers, as previously described (33.33% of sessions). Raters evaluated each component using the following scale: 0 = expected to be completed but omitted; 1= expected to be completed and actually completed (shaded items were not expected to be completed by students), and 7= purposely omitted. The percentage of possible components completed by the instructors and each individual student was computed with a range of 0-100%. If the instructor intentionally skipped a component (e.g., "Boy and girls, because we took a little longer with the last activity, we are going to ..."), this was indicated as a 7 and the total number of components possible was adjusted in the denominator. Three sets of percentages were computed for the instructor and each participating student: (1) percentage of treatment integrity for all sessions according to the instructor perspective, (2) percentage of treatment integrity for sessions observed by the outside observer, and (3) percentage of treatment integrity for sessions observed by an outside observer according to the instructor's perspective.

Social validity. Social validity was assessed from teacher and student perspectives pre and post intervention using the following measures.

Intervention Rating Profile--15 (IRP-15; Witt & Elliott, 1985). The IRP-15 is a 15-item questionnaire used to determine the acceptability of intervention procedures and intended goals. Teachers' rated items on a 5-point Likert-type scale ranging from 1 = Strongly Disagree to 5 = Strongly Agree IRP-15 scores range from 15 to 90 with higher scores indicating greater social validity. Internal-consistency reliabilities, of the IRP-15, range from .88 to .98 (Witt & Elliott, 1985).

Children's Intervention Rating Profile (CIRP; Witt & Elliott, 1985). The adapted CIRP is a 7-item questionnaire used to assess students' acceptability of the intervention procedures, goals, and outcomes on a 6-point continuum of 1 = I agree to 6 = I do not agree. Positively worded items (e.g., I think I will like being in this program.) were reverse-scored. The CIRP scores range from 7 to 42 points, higher scores indicated greater social validity. Internal consistency reliabilities, for the CIRP, range from .75 to .89 (Witt & Elliott, 1985).

What I Think About Positive Action. The Positive Action social validity form assessed students' and teachers' perceptions of participation in the Positive Action intervention. First, participants responded to 4 questions by circling Not Much, A Little, or A Lot. The questions were: (1) I think I will learn a lot from my Positive Action program; (2) I think I will use what I learn from my Positive Action program; (3) The things I will learn from my Positive Action program will be helpful; and (4) I think there will be things that I will want to change about the Positive Action program. Next, teachers and students responded to five open-ended items: what they liked, did not like, how they thought they would use what they learned, what they thought were the most helpful things they learned, and what changes would improve the program. The post-intervention form asked the same questions, phrased in the past tense, to assess whether or not participants' expectations were met.

Student outcomes. Student outcome measures included proximal, intermediate, and distal measures. The outcome measures were administered pre and post intervention.

Proximal: Criterion Reference Assessment: Positive Action. Students completed an assessment of the most salient aspects of the Positive Action lessons that were selected for the intervention that was developed by the researchers. The assessment directly measured concepts taught through the lessons. The criterion-referenced assessment is a 50-item, multiple-choice and true-false assessment targeting objectives taught in the Positive Action intervention. Raw scores ranged from 0-50 with percentage scores calculated. Higher scores indicated greater knowledge of the content taught.

Intermediate: Skills for Success Checklist (Positive Action, 2010)--Teacher and student versions. The Skills for Success checklist was a tool designed by the developers of Positive Action. It consisted of 50 items to allow students to self-assess and teachers to assess students on the key constructs of the Positive Action curriculum. Items are scored on a scale of DK = Don't Know (teacher version only), 0 = Not True, 1 = Somewhat or Sometimes True, and 2= Very or Often True. Individual scores ranged from 0 - 100. Higher scores are indicative of greater application of key skills. For the current study, we examined items specific to Units 2, 3, and 6 only (29 items). As such, individual scores for this subset ranged from 0 to 58. Due to teachers marking a large number of items as Don't Know, we report only student findings.

Distal: Academic Competence Evaluation Scale (ACES; DiPerna & Elliott, 2000)--Teacher version. ACES assessed students' academic skills and academic enablers as observed by their teachers in the classroom. ACES employed a 2-part rating scale to measure academic skills (reading/language arts, mathematics, and critical thinking) and academic enablers (interpersonal skills, engagement, motivation, and study skills). The teacher version of the form contained 73 items rated on two 5-point scales: Frequency (or Proficiency) and Importance. The item anchors for the Academic Skills scale were 1 = far below grade level to 5 = far above grade level. The Academic Enablers scale anchors were 1 = never to 5 = almost always. Items were summed and raw scores were recorded on the Scoring Summary placing the student's performance into categories: developing (0), competent (1), or advanced (2). The authors reported internal consistency coefficients for scales ranging from .92 to .98. The test-retest reliabilities for the teacher form of the Academic Skills and Academic Enablers were .95 and .96 respectively. In the current study, we focus solely on the engagement and motivation academic enablers as distal measures of student outcomes.

Experimental Design

The experimental design was a two group, pretest-posttest design. Data were not analyzed using multivariate procedures as the purpose of this development study was to examine the feasibility and effectiveness of implementing this program within the context of a three-tiered model. The sample size for this developmental study was intentionally small to obtain initial information on the feasibility, utility, and practicality of using Positive Action as a Tier 2 support. We used descriptive procedures to examine treatment integrity and social validity data. Difference scores were computed for social validity measures (IRP-15, PA teacher perspective, CIRP, and PA student perspective) as well as intervention outcomes (content knowledge [proximal outcomes], skills for success acquired [intermediate outcomes] and ultimately improved engagement and motivation [distal outcomes]) by subtracting Time 1 scores from Time 2 scores to obtain a measure of growth. Using difference scores allowed us to account for any variability between groups prior to intervention onset. We computed effect sizes comparing pre-intervention and post-intervention scores for each group (Positive Action and Book Study) as well as change scores between groups. We calculated effect sizes using Cohen's d formula by subtracting mean scores and dividing by the pooled standard deviation. In the results section we provide a brief description of the data analyses procedures used to address each research question.

Results

Integrity of Implementation

Statistical analysis. Descriptive procedures (means and standard deviations) were used to analyze the extent to which this type of Tier 2 support could be implemented within the regular school day with integrity. Treatment integrity data are reported from the perspectives of the instructor and an outside observer for the Positive Action and Book Study conditions. Three sets of mean scores are reported as described in the method section: (a) instructors' perspectives for all lessons completed, (b) instructors' perspective for lessons conducted when the outside observer was evaluating treatment integrity, and (c) outside observers' perspectives, which occurred for 33.3% for each group (see Table 2).
Table 2 Percentage of Treatment Integrity by Observer and
Group

Intervention Observations Observer Participant

 Instructor Student Student
 1 2

Positive
Action

 All Lessons

 Instructor M 99.66 (1.56) 99.52 100.00
 (SD) (2.18) (0.00)

 Reliability
 Lessons

 Instructor M 98.98 (2.70) 98.57 100.00
 (SD) (3.78) (0.00)

 Observer M 96.94 (3.82) 95.71 93.33
 (SD) (5.35) (8.16)

 Instructor Student Student
 5 6

Book Study

 All Lessons

 Instructor M 100.00 98.44 100.00
 (SD) (0.00) (5.00) (0.00)

 Reliability
 Lessons

 Instructor M 100.00 95.56 100.00
 (SD) (0.00) (8.01) (0.00)

 Observer M 100.00 96.79 100.00
 (SD) (0.00) (5.54) (0.00)

Intervention Observations Observer

 Student Student
 3 4

Positive
Action

 All Lessons

 Instructor M 100.00 98.57
 (SD) (0.00) (3.59)

 Reliability
 Lessons

 Instructor M 100.00 98.57
 (SD) (0.00) (3.78)

 Observer M 95.71 94.29
 (SD) (7.87) (5.35)

 Student Student Student
 7 8 9

Book Study

 All Lessons

 Instructor M 99.47 100.00 97.94

 (SD) (2.29) (0.00) (4.22)

 Reliability
 Lessons

 Instructor M 100.00 100.00 96.48
 (SD) (0.00) (0.00) (5.46)

 Observer M 100.00 100.00 94.63
 (SD) (0.00) (0.00) (9.17)


Findings. The Positive Action and Book Study conditions were implemented with a high degree of fidelity. For the Positive Action condition, overall integrity across all lessons for the instructor's components was 99.66%, with students' percentages ranging from 98.57% (Students 1 and 4) to 100% (Students 2 and 3). Integrity was slightly lower for the instructor's components during the observed sessions according to the outside observer's perspective (96.94%) compared to instructor's perspective during these same lessons (98.98%). Student components' percentages of integrity were also lower for each student, yet, the differences were relatively minor.

For the Book Study condition, overall integrity for the instructor's components was 100%, with students' percentages ranging from 97.94% (Student 9) to 100% (Students 6 and 8). Integrity was the same for instructor's components during the observed sessions according to the outside observer's perspective and the instructor's perspective during these same lessons (100%). For Students 5 and 9, there were minor variations in treatment integrity scores.

Social Validity

Statistical analysis. Descriptive procedures (means and standard deviations) were used to examine teachers' and students' perceptions regarding their respective intervention's goals, procedures, and outcomes. Means and standard deviations were computed for pre-and post-intervention scores on the IRP-15 by teachers and CIRP by students participating in each condition. Teachers and students also completed a social validity measure specific to the Positive Action curriculum. This measure was completed by all teachers and students to reduce the possibility the teacher would know which students were randomly assigned to the intervention condition. IRP-15 and CIRP data are reported for both groups; the Positive Action social validity data are reported only for teachers and students who participated in this condition. Effect sizes are reported to describe the magnitude of the change scores between Positive Action and Book Study groups, pre-and post-intervention scores for Positive Action, and pre-and post-intervention scores for the Book Study condition (see Table 3).
Table 3 Social Validity

Variable Intervention Effect Size
Level Group

Phase Positive Book [DELTA] PA(pre: BS(pre:
 Action(n = 4) Study(n = Scores(PA: post) post)
 5) BS)

 M(SD) M(SD)

Social
Validity

IRP-15 1.95 3.90 1.28

Pre 79.00 (4.62) 80.62
 (10.40)

Post 88.00 (0.00) 87.67
 (0.58)

[DELTA] 5.00 (0.00) 13.30
 (8.52)

PA-Teacher 3.00

Pre 6.50 (1.00)

Post 8.00 (0.00)

[DELTA] 1.00(1.41)

CIRP -0.07 -0.19 -0.19

Pre 37.50 (3.32) 38.00
 (5.43)

Post 36.75 (4.50) 37.00
 (5.15)

[DELTA] -0.75 (4.03) -1.00
 (2.65)

PA-Student -0.90

Pre 6.00 (0.82)

Post 5.00 (1.41)

[DELTA] -1.00 (1.41)

Note. LRP-15 = Intervention Rating Profile-15, range = 15
to 90; CIRP = Children's Intervention Rating Profile, range
= 7 to 42. PA refers to Positive Action; BS refers to Book Study.


Findings. For the Positive Action condition, teachers' mean IRP-15 scores increased from 79.00 to 88.00 (effect size = 3.90), suggesting the intervention was highly acceptable and actually exceeded their initial expectations. The effect size for pre-and post-intervention scores for the Book Study condition also indicated a positive effect (1.28), although the magnitude of the change was substantially greater for the Positive Action group compared to IRP-15 scores for the Book Study group (1.95). The same pattern was observed between teachers' pre-and post-intervention social validity scores for information pertaining solely to perceptions of Positive Action. The Positive Action intervention exceeded teachers' expectations (3.00).

For the Positive Action condition, students rated the intervention favorably at intervention onset and following intervention completion. Students' average CIRP scores decreased slightly from 37.50 to 36.75 (effect size = -0.19), suggesting low-magnitude decreases in students' initial expectations. The effect size for pre-and post-intervention scores for the Book Study condition also indicated a similar pattern (-0.19), with the magnitude of the change comparable between the Positive Action and Book Study groups' CIRP scores (-0.07). The same pattern was also observed between students' pre-and post-intervention social validity scores for information pertaining solely to perceptions of the Positive Action curriculum. The Positive Action intervention fell short in terms of students' initial expectations (-0.90).

Student Outcomes

Statistical analysis. Descriptive procedures were also used to determine the extent to which initial evidence suggested the Positive Action intervention yielded proximal, intermediate, and distal outcomes. Measures included: (a) percentage of items correct on the criterion-referenced test of content taught in the Positive Actions (proximal outcomes; units 2, 3, and 6); (b) average scores on the Skills for Success tool from student perspectives (intermediate outcomes; items measuring skills addressed); and (c) engagement and motivation subscales on the ACES (distal outcomes) examining teachers' perceptions of students' generalization of content learned. Multivariate models with moderating and mediating variables were not possible in this development study. Instead, initial evidence was examined by means and effect sizes. Means and standard deviations were computed for pre-intervention, post-intervention, and change scores (post-minus pre-intervention scores) on each measure by intervention group. Effect sizes are reported to describe the magnitude of the change scores between Positive Action and Book Study groups, pre-and post-intervention scores for Positive Action, and pre-and post-intervention scores for the Book Study condition.

Findings. In the following sections we document findings of proximal, intermediate, and distal outcomes (see Table 4).
Table 4 Performance of Student Participants

Variable Intervention
Level Croup

Phase Positive Action Book Study [DELTA]
 Scores

 (n = 4) M(SD) (n = 5) M (SD) (PA: BS)

Content -2.08
Knowledge

 Pre 52.00(12.75) 60.00 (7.07)

 Post 66.50 (6.61) 59.60 (12.12)

 [DELTA] 14.50 (6.40) -0.40 (7.92)

Skills for 0.25
Success:
Student
Perspective

 Pre 89.22(12.78) 79.31 (16.40)

 Post 85.78(11.56) 76.90(21.21)

 [DELTA] -3.45 (2.30) -2.41 (6.17)

Engagement -0.30

 Pre 0.75 (0.96) 1.00(0.71)

 Post 1.00 (0.00) 1.00 (0.71)

 [DELTA] 0.25 (0.96) 0.00 (0.71)

Motivation 0.41

 Pre 0.00 (0.00) 0.20 (0.45)

 Post 0.75 (0.50) 1.20 (0.45)

 [DELTA] 0.75 (0.50) 1.00 (0.71)

Variable Effect
Level Sizes

Phase PA BS

 (pre: (pre:
 post) post)

Content 1.50 -0.04
Knowledge

 Pre

 Post

 [DELTA]

Skills for -0.28 -0.13
Success:
Student
Perspective

 Pre

 Post

 [DELTA]

Engagement 0.52 0.00

 Pre

 Post

 [DELTA]

Motivation 3.00 2.22

 Pre

 Post

 [DELTA]


Content knowledge. For the Positive Action condition, students' mean content knowledge scores increased from 52.00 to 66.50 (effect size = 1.50), suggesting content knowledge increased. Yet, mastery was not achieved given students learned less than 70% of skills taught. The effect size for pre-and post-intervention scores for the Book Study condition also indicated a negligible effect (-0.04), with the magnitude of change substantially greater for the Positive Action group's content knowledge scores compared to change scores of the Book Study group (-2.08).

Skills for Success: Student perspective. For the Positive Action condition, students' Skills for Success mean scores decreased from 89.22 to 85.78 (effect size = -0.28), suggesting a small decline in their reported skill sets. Pre-intervention scores were quite high, close to 90% indicating students perceived little room for improvement. The effect size for pre-and post-intervention scores for the Book Study condition also suggested a small decline (-0.13), with the magnitude of the change suggesting comparable changes between the Positive Action group and the Book Study group (0.25).

Engagement. For the Positive Action condition, teacher ratings of engagement increased from 0.75 to 1.00 (effect size = 0.52), suggesting a moderate effect. Following intervention, students' engagement was in the competent range. The effect size for pre-and post-intervention scores for the Book Study condition showed no change (0.00); yet, they had higher pre-intervention scores compared to the Positive Action group. The magnitude of the change was greater for the Positive Action group's engagement scores compared to change scores of the Book Study group (-0.30).

Motivation. For the Positive Action condition, average teacher ratings of motivation increased from 0.00 to 0.75 (effect size = 3.00), suggesting a strong effect. Although improvement was substantial, engagement was in the developing range following intervention completion. The effect size for pre-and post-intervention scores for the Book Study condition was also substantial (2.22), yet not as strong as for the Positive Action group. The magnitude of the change was moderately lower for the Positive Action group engagement scores compared to change scores of the Book Study group (0.41).

Discussion

In this paper, we provided a methodological illustration for conducting a development study consistent with the parameters delineated by the Institute of Education Sciences (USDE, 2010) to examine an existing Tier 1 character development intervention applied as a Tier 2 support within a three-tiered prevention model. In the method we provided a detailed description of how specific units were selected to focus on skill sets associated with improving self-regulation skills for promoting goal setting, decision making, and self-management skills to meet the behavioral and instructional needs of fourth-grade students. Despite having adequate literacy skills, these students had behavioral challenges and limited work completion. The main questions explored pertained to treatment integrity and social validity, with a goal of having a fully-developed intervention ready for testing as part of an efficacy trial.

Treatment Integrity

The first question explored was: To what extent can this type of Tier 2 support be implemented within the regular school day with integrity? In answering this first question, we established the importance of collecting treatment integrity data from multiple perspectives. Consistent with other studies, outside observers tended to report slightly lower levels of implementation compared to those delivering the instruction (Wickstrom, Jones, LaFleur, & Witt, 1998). Due to the variation according to observer, we learned it is necessary to not only have teachers evaluate integrity (which can also serve as a prompt for addressing all components and involving all students), but to include outside observers. Information from the outside observer could also be helpful in coaching interventionists to meet targeted levels of integrity (Oliver & Wehby, 2011; Sheridan, Swanger-Gange, Welch, Kwon, & Garbacz, 2009).

Although the level of integrity was quite high when observing instructor and student participation, there was variability. It is essential to not assume all students are participating in the intervention components to the same degree simply because it is a small group intervention. Data illustrate that just because an instructor is implementing the intervention with intended integrity does not mean students are accessing the intervention experience to the same degree. For example, Student 4's participation involved accessing fewer intervention components compared to Students 2 and 3. While the level of detail measured in the treatment forms may seem overly detailed, we contend this level of detail is needed to not only study variations in fidelity, but also provide insight on potential moderating and mediating variables to be used in more sophisticated analyses. For example, we may explore the extent to which teacher tone (item 13) mediated intervention outcomes. In sum, it is important to collect treatment integrity data from multiple perspectives and on all individuals involved in the intervention as equal access is not assumed.

Social Validity

The second question explored was: What did teachers and students think about the goals, procedures, and outcomes? To assess this question, we sought information from the classroom teacher and student participants. We used general measures of social validity from teacher (IRP-15) and student (CIRP) perspectives, as well as a measure specific to the content of the Positive Action intervention.

While the intervention actually exceeded the classroom teachers' expectations, it is important to consider teachers were not the intervention agents. Although they had awareness level knowledge of the conditions, they were not intimately familiar with the instruction. Favorable ratings are more reflective of the procedures employed and how it translated to student performance in the classroom. Teachers reported they liked "the time spent in small group instruction to target the students' needs" and they "tried to give students extra praise as [they] saw student performance improve [in class]". Teachers also reported they "noticed several students improve class participation partly because they felt 'special' or important because they were being pulled out." On the post IRP-15, teachers reported comments for four students. For the two students in the Positive Action group, the teachers reported, "I have noticed that [the student] is always prepared and her off-task behavior is improving" and "[The student's] absences did not decrease, but she was extremely responsible in getting work she missed and making up work." For two students in the Book Study group, teachers reported, "I did not notice a change in [the student] positive or negative" and "I have not noticed any changes in [the student's] behavior or work habits or organization."

In contrast, both groups of students reported slight decreases in their social validity scores suggesting the intervention fell short of their initial expectations. Several students made informal comments to the instructor indicating they were dissatisfied because other students in their homeroom classes were using the time to work on homework and classwork during the intervention block. For example, two students commented that they did not like that they "missed intervention" in the classroom. When asked what they would change about the program one student reported, "Nothing except leaving." However, only one student elected to withdraw.

In sum, while the procedures and intervention goals were acceptable to the teachers, we learned "missed" activities are an important consideration when designing interventions involving students being removed from class. Great care was taken to avoid missing instructional time in core content areas. Additional consideration should be given to ensuring students are not missing other experiences viewed as preferable or highly valued (e.g., time to complete homework with support).

Preliminary Evidence of Outcomes

The final question explored was: Is there initial evidence to suggest this intervention results in students learning the content taught (proximal outcomes) translating into improved skill sets (intermediate outcomes) and ultimately improved motivation (distal outcomes)? In this development study, we explored initial evidence of proximal, intermediate, and distal outcomes with an emphasis on selecting reliable and feasible measures to monitor student performance. Results indicated while content knowledge increased, mastery was not achieved. Although students in the Positive Action condition showed high magnitude changes on the percentages of items within (pre-to post-) and between groups (change scores for Positive Action and Book Study groups), the learning outcomes were not sufficient. It is quite likely the dosage of intervention was insufficient and subsequent studies would need to examine contact hours needed to reach mastery (Kalberg, Lane, & Lambert, 2012; Robertson & Lane, 2007).

In terms of Skills for Success scores, students' pre-intervention scores were relatively high (79.31 for the Book Study group and 89.22 for the Positive Action group). These scores indicated students reported they had mastery of the vast majority of the skills to be taught in the Positive Action curriculum. Yet, this runs contrary to teachers' data (report card and work completion) that determined students' initial eligibility for participation. This is consistent with the work of Gresham and colleagues suggesting students with and at risk for EBD tend to have a positive illusory bias meaning they tend to see themselves as more competent than do their teachers, parents, and peers (Gresham, Lane, MacMillan, Bocian, & Ward, 2000). It is unfortunate that many of the items were marked as do not know by teachers for the Skills for Success survey as it would have been helpful to compare teacher and students' perceptions of their skills before and following intervention completion to further explore this notion of illusory bias. When asked why many items were rated as do not know, teachers indicated they did not feel they had sufficient information to accurately rate several of the physical items (e.g., exercises or is active in healthy ways) associated with the Physical and Intellectual Positive Actions for a Healthy Self. This measure continues to be under development by the publishers of Positive Action. Future versions should be tested to ensure teachers can rate items on the survey.

An initial attempt was also made to collect data on work completion based on teachers' records as another intermediate outcome variable. Teachers were provided with data collection sheets to obtain data on the number of assignments requested and completed as well as work accuracy of the completed products. Due to variability in teachers' record keeping procedures and the manner in which class and homework were assigned and graded, these data were not reliable and, therefore, not reported.

Two distal measures were examined: engagement and motivation. In terms of engagement, there were initial differences in mean scores with students in the Positive Action group performing in the developing range and students in the Book Study group performing in the competent range. It should be noted neither group of students moved into the advanced range following intervention completion. However, students in the Positive Action group showed moderate improvement between pre-and post-intervention conditions (0.52).

For the Positive Action condition, average teacher ratings of motivation increased from 0.00 to 0.75 (effect size = 3.00), suggesting a strong effect. Although improvement was substantial, motivation was in the developing range following intervention completion. The effect size for pre-and post-intervention scores for the Book Study condition was also substantial (2.22), yet not as strong as for the Positive Action group. The magnitude of the change was moderately lower for the Positive Action group motivation scores compared to change scores of the Book Study group (0.41), influenced in part by the absence of variability in pre-intervention scores for the Positive Action group. While the improvements in motivation are encouraging, additional inquiry will be needed using larger sample sizes to explore other moderating and mediating variables such as the role of actual participation in the small group, the effect of additional attention in small group format, and levels of praise used by the classroom teacher for demonstrating newly learned skills.

In sum, the preliminary evidence regarding student outcomes provided useful direction on issues of measurement. For example, it will be important to select measures teachers feel comfortable completing and use systems closely aligned to current record keeping practices. Student-completed measures and teacher-completed rating scales were the most complete and most reliable in the current study.

Limitations and Future Directions

When interpreting the outcomes of this development study, it is important to recognize the associated limitations. First, because this was a development study, the study involved a very small sample size-only 9 students. It is imperative readers not generalize these findings beyond the intended purpose: to conduct the preliminary research necessary to have a fully-developed intervention and the associated measurement tools ready for testing using experimental procedures. Subsequent studies may involve single-case or group methodologies using the core quality indicators necessary to establish internal and external validity (Gersten et al., 2005; Horner et al., 2005).

Second, this study attempted to obtain additional proximal measures (e.g., work completion and work accuracy), which were unsuccessful for the previously explained reasons. There were also attempts to obtain information from parents regarding their perceptions of their child's competencies (Skills for Success-Parent Perspective) and views about the interventions goals, procedures, and outcomes (social validity). However, only a couple parents returned these measures despite multiple prompts. Additional venues for encouraging parental participation will need to be explored in subsequent studies.

A third limitation pertains to intervention dosage. Due to inclement weather (e.g., snow) a few lessons had to be shortened and/or combined due to students being released early or school closings. As such, we did not obtain our original target of 15 intervention hours. Prior to identifying students for possible participation in similar Tier 2 studies, it is important to allow students adequate time to respond to primary prevention efforts before determining the primary plan was insufficient. Yet, we also must ensure sufficient time remains in the academic year to offer Tier 2 supports of adequate dosage.

Finally, as discussed previously, the issue of when to provide Tier 2 interventions is important. While students indicated they enjoyed spending time with the instructors, they expressed concern about missing the opportunity to complete homework in class with assistance. Because their classmates were using the intervention block as a time to complete homework, the students in this and other school-based Tier 2 supports had more homework relative to classmates. Yet, none of the students elected to withdraw for this reason. Future research should consider system-wide options for addressing this concern. For example, perhaps the intervention block should not be used as a time for homework, but as a time for enrichment.

Concluding Thoughts

Despite these limitations, we view this study as meeting the desired outcomes. Our first objective was to explore the utility of a character development intervention initially developed for use as Tier 1 instruction when it was applied as a Tier 2 support focused on improving goal setting, decision making, and self-management skills to meet the behavioral and instructional needs of fourth-grade students who have behavioral challenges and limited work completion, but who have adequate literacy skills. The answers to the research questions--which serve as the cornerstone of all educational research (Shavelson & Towne, 2002), resulted in a fully-developed intervention with a set of measures ready to assess treatment integrity, social validity, as well as proximal, intermediate, and distal student outcomes.

The output from this development study addressed the second objective: to field test the required components specified in the IES 2010 RFA to assist in the development of a proposal to be submitted for funding. The practices and procedures employed in the present study enabled this research team to study the challenges associated with using an iterative process grounded in a defined theory of change, to develop an "innovative education intervention" (USDE, 2011b, p. 7). We hope this methodological illustration will be useful to researchers and practitioners interested in conducting similar development inquiry.

This research was supported by in part by Project Support and Include, a technical assistance grant from the Tennessee Department of Education (#GR-10-27642-00).

References

Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing words to life: Robust vocabulary instruction. New York, NY: Guilford Press.

Bruhn, A., Lane, K. L., & Hirsch, S. (2012). A review of secondary interventions conducted within multi-tiered models of behavioral prevention Manuscript submitted for publication.

Carter, E. W., Lane, K. L., Crnobori, M. E., Bruhn, A. L., & Oakes, W. P. (2011). Self-determination interventions for students with and at risk for emotional and behavioral disorders: Mapping the knowledge base. Behavioral Disorders, 36, 100-116.

Cook, B. G., Tankersley, M., & Landrum, T. J. (2009). Determining evidence-based practices in special education. Exceptional Children, 75, 365-283.

Daniel, M. H. (2010). Reliability of AIMSweb Reading Curriculum-Based Measurement (R-CBM) (Oral Reading Fluency). Clinical Assessment. Bloomington, MN: NCS Pearson.

DiCamillo, K. (2003). Tale of Despereaux. Somerville, MA: Candlewick Press.

DiPerna, J. C., & Elliott, S. N. (2000). Academic Competence Evaluation Scales. San Antonio TX: The Psychological Corporation.

DiPerna, J.C., & Elliott, S.N. (2002). Promoting academic enablers to improve student achievement: An introduction to the miniseries. School Psychology Review, 31, 293-297.

DiPerna, J. C., Volpe, R. J., & Elliott, S. N. (2002). A model of academic enablers and elementary reading/language arts achievement. School Psychology Review, 31, 298-312.

Discovery Education. (n.d.). A multi-state look at what is predictive assessment. Retrieved from http://www.thinklinklearning.com/research_predict.php

Discovery Education. (2000). ThinkLink Learning. Nashville, TN: Author.

Dowhower, S. L. (1991). Speaking of prosody: Fluency's unattended bedfellow. Theory into Practice, 30, 165-175.

Drummond, T. (1994). The Student Risk Screening Scale (SRSS). Grants Pass, OR: Josephine County Mental Health Program.

Elliott, S. N., & Gresham, F. M. (1991). Social skills intervention guide: Practical strategies for social skills training. Circle Pines, MN: American Guidance Service.

Flay, B. R., & Allred, C. G. (2003). Long-term effects of the Positive Action program. American Journal of Healthy Behavior, 27, 6-21.

Flay, B. R., Allred, C. G., & Ordway, N. (2001). Effects of the Positive Action program on achievement and discipline: Two matched-control comparisons. Prevention Science, 2, 71-89.

Fuchs, D., & Fuchs, L. (2006). Introduction to response to intervention: What, why, and how valid is it? Reading Research Quarterly, 41, 93-99.

Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71, 149-164.

Gresham, F. M., Lane, K. L., MacMillan, D. L., Bocian, K. M., & Ward, S. L. (2000). Positive and negative illusory biases: Comparisons across social and academic self-concept domains. Journal of School Psychology, 38, 151-175.

Homer, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179.

Homer, R., Todd, A., Lewis-Palmer, T., Irvin, L. K, Sugai, G., & Boland, J. B. (2004). The school-wide evaluation tool (SET): A research instrument for assessment of school-wide PBS. Journal of Positive Behavior Interventions, 6, 3-12.

Hunter, R., & Hunter, M. (2004). Updated editions Madeline Hunter's mastery teaching. Thousand Oakes, CA: Corwin Press.

Individuals with Disabilities Education Improvement Act of 2004, 20 U.S.C. 1400 et esq. (2004). (reauthorization of Individuals with Disabilities Act 1990).

Kalberg, J. R., Lane, K. L., & Lambert, W. (2012). The utility of conflict resolution and social skills interventions with middle school students at risk for antisocial behavior: A methodological illustration. Remedial and Special Education, 22, 23-38. doi: 10.1177/0741932510362514

Kauffman, J. M., & Landrum, T. (2009). Characteristics of emotional and behavioral disorders of children and youth (8th ed.). Columbus, OH: Merrill.

Lane, K. L., Kalberg, J. R., Bruhn, A. L., Driscoll, S. A., Wehby, J. H., & Elliott, S. (2009). Assessing social validity of school-wide positive behavior support plans: Evidence for the reliability and structure of the Primary Intervention Rating Scale. School Psychology Review, 38, 135-144.

Lane, K. L., Kalberg, J. R., & Menzies, H. M. (2009). Developing school-wide programs to prevent and manage problem behaviors: A step-by-step approach. New York, NY: Guilford Press.

Lane, K. L., Menzies, H. M., Bruhn, A. L., & Crnobori, M. (2011). Managing challenging behaviors in schools: Research-based strategies that work. New York, NY: Guilford Press.

Lane, K. L., Menzies, H. M, Oakes, W. P., & Kalberg, J. R. (2012). Systematic screenings of behavior to support instruction: From preschool to high school. New York, NY: Guilford Press.

Lane, K. L., Oakes, W. P., & Menzies, H. M. (2010). Systematic screenings to prevent the development of learning and behavior problems: Considerations for practitioners, researchers, and policy makers. Journal of Disability Policy Studies, 21, 160-172.

Lane, K. L., Wehby, J. H., Menzies, H. M., Gregg, R. M., Doukas, G. L., & Munton, S. M. (2002). Early literacy instruction for first-grade students at risk for antisocial behavior. Education and Treatment of Children, 25, 438-458.

Li, K.-K., Washburn, I., DuBois, D. L., Vuchinich, S., Ji, P., Brechling, V., ... Flay, B. R. (2011). Preventing disruptive behaviors, substance use, and violent behaviors among elementary students: Effects of the Positive Action program in Chicago. Psychology & Health, 26, 187-204. doi: 10.1080/08870446.2011.531574

May, S., Ard, W., Todd, A., Homer, R., Glasgow, A., Sugai, G. & Sprague, J. (2010). School-wide Information System v4.4.75. Eugene, OR: University of Oregon.

McIntosh, K., Bohanon, H., & Goodman, S. (n.d.). Integrating academic and behavior supports within an RtI framework, part 3: secondary supports. RtI Action Network. Retrieved from http://www.rtinetwork.org/learn/behavior-supports/integrating-academic-and-behavior-supports-secondary-supports

McIntosh, K., Chard, D. J., Boland, J. B., & Horner, R. H. (2006). Demonstration of combined efforts in school-wide academic and behavioral systems and incidence of reading and behavior challenges in early elementary grades. Journal of Positive Behavior Interventions, 8, 146-154.

Mitchell, B. S., Stormont, M., & Gage, N. A. (2011). Tier two interventions implemented within the context of a tiered prevention framework. Behavioral Disorders, 36, 241-261.

National Governors Association Center & the Council of Chief State School Officers for Best Practices. (2011). Common Core State Standards Initiative. Washington, DC: Author.

Nelson, J. R., Benner, G. J., Lane, K. L., & Smith, B. W. (2004). An investigation of the academic achievement of K-12 students with emotional and behavioral disorders in public school settings. Exceptional Children, 71, 59-73.

Oakes, W. P., Mathur, S. R., & Lane, K. L. (2010). Reading interventions for students with challenging behavior: A focus on fluency. Behavioral Disorders, 35, 120-139.

Oakes, W. P. Wilder, K., Lane, K. L., Powers, L., Yokoyama, L., O'Hare, M. E., & Jenkins, A. B. (2010). Psychometric properties of the Student Risk Screening Scale: An effective tool for use in diverse urban elementary schools. Assessment for Effective Intervention, 35, 231-239.

Oliver, R.M. & Wehby, J.H. (2011). Sustaining high fidelity of classroom evidence-based practices: Performance feedback with teacher self-monitoring. Submitted for publication.

Pearson Education. (2010). AIMS Web. San Antonio, TX: Author.

Positive Action. (2004). Positive Action levels of implementation plans: Bronze, silver, gold, and platinum. Twin Falls, ID: Author.

Positive Action. (2008). Positive Action: Positive Development for Schools, Families and Communities. Twin Falls: ID: Author.

Positive Action. (2010). Skills for success [teacher, parent, student surveys]. Twin Falls: ID: Author.

Reinke, W. M., Herman, K. C., Petros, H., & 'alongo, N. (2008). Empirically-derived subtypes of child academic and behavior problems: Co-Occurrence and distal outcomes. Journal of Abnormal Child Psychology, 36, 759-777.

Robertson, E. J., & Lane, K. L. (2007). Supporting middle school students with academic and behavioral concerns within the context of a three-tiered model of support: Findings of a secondary prevention program. Behavioral Disorders, 33, 5-22.

Sattler, J. (1991). Assessment of students (3rd ed.). San Diego, CA: Sattler.

Shavelson, R. J., & Towne, L. (2002). Scientific research in education. Washington, DC: National Academy Press.

Sheridan, S. M., Swanger-Gange, M., Welch, G. W., Kwon, K., & Gar-bacz, S. A. (2009). Fidelity measurement in consultation: Psychometric issues and preliminary examination. School Psychology Review, 38, 476-495.

Smith, M., & Tilton, D. (2008). The tale of Depereaux: The graphic novel. Somerville, MA: Candelwick Press.

Snyder, F. J., Flay, B. R., Vuchinich, S., Acock, A., Washburn, I. J., Beets, M., & Li, K. (2010). Impact of the Positive Action program on school-level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched-pair, cluster randomized, controlled trial. Journal of Research on Educational Effectiveness, 3, 25-55. doi: 10.1080/19345740903353436

Stewart, R. M., Benner, C. J., Martella, R. C, & Marchand-Martella. N. E. (2007). Three-tier models of reading and behavior: A research review. Journal of Positive Behavior Interventions, 9, 239253.

Sugai, G., & Horner, R. H. (2009). Responsiveness-to-intervention and school-wide positive behavior supports: Integration of multi-tiered system approaches. Exceptionality, 17, 223-237. doi: 10.1080/09362830903235375

Sugai, G., & Lewis, T. J. (1996). Preferred and promising practices for social skills instruction. Focus on Exceptional Children, 29, 1-16.

Sugai, G., Lewis-Palmer, T., Todd, A., & Homer, R. H. (2001). School-wide evaluation tool. Eugene, OR: University of Oregon.

Tennessee Department of Education. (2011). Annual Report Card. Nashville, TN: Author.

U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2007a). Retrieved from http://ies.ed.govincee/wwc/findwhatworks.aspx

U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2007b). What Works Clearinghouse: Positive Action 2007. Washington, DC: Author.

U.S. Department of Education, Institute of Education Sciences. (2010). Request for application. Education research grants CFDA number: 84.305A. Washington, DC: Author.

U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. (2011a). Common Core of Data: Public school data. Washington, DC: Author.

U.S. Department of Education, Institute of Education Sciences. (2011b). Request for application. Education research grants CFDA number: 84.305A. Washington, DC: Author.

Walker, H., Ramsey, E., & Gresham, F. M. (2004). Antisocial behavior in school: Evidence-based practices (2nd ed.). Belmont, CA: Wadsworth.

Wechsler, D. (2003). Wechsler Intelligence Scale for Students--Fourth Edition. San Antonio, TX: Harcourt Brace Jovanovich.

Wentzel, K. R. (1993). Does being good make the grade? Relations between academic and social competence in early adolescence. Journal of Educational Psychology, 85, 357-364.

Wickstrom, K. F., Jones, K. M., LaFleur, L. H., & Witt, J. C. (1998). An analysis of treatment integrity in school-based behaviral consultation. School Psychology Quarterly, 13, 141-154.

Witt, J. C., & Elliott, S. N. (1985). Acceptability of classroom intervention strategies. In T. R. Kratochwill (Ed.). Advances in School Psychology. (Vol. 4, pp. 251-288). Mahwah, NJ: Erlbaum.

Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock--Johnson III tests of achievement. Itasca, IL: Riverside Publishing.

Wendy Peia Oakes, Kathleen Lynne Lane, Meredith Cox, Ashley Magrane, Abbie Jenkins, and Katy Hankins Vanderbilt University

Correspondence to Kathleen Lynne Lane, Ph.D., Kansas University; Kathleen.Lane@ku.edu or Wendy P. Oakes, Ph.D., Arizona State University; Wendy.Oakes@asu.edu.
COPYRIGHT 2012 West Virginia University Press, University of West Virginia
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2012 Gale, Cengage Learning. All rights reserved.

 
Article Details
Printer friendly Cite/link Email Feedback
Author:Oakes, Wendy Peia; Lane, Kathleen Lynne; Cox, Meredith; Magrane, Ashley; Jenkins, Abbie; Vanderbilt,
Publication:Education & Treatment of Children
Article Type:Report
Geographic Code:1USA
Date:Nov 1, 2012
Words:13916
Previous Article:Effects of tiered training on general educators' use of specific praise.
Next Article:Exclusionary discipline practices across students' racial/ethnic backgrounds and disability status: findings from the Pacific Northwest.
Topics:

Terms of use | Privacy policy | Copyright © 2018 Farlex, Inc. | Feedback | For webmasters