Assessing antecedent variables: the effects of instructional variables on student outcomes through in-service and peer coaching professional development models.

Abstract

A student's behavioral performance within the classroom is influenced, in large part, by the interaction that student has with the learning environment and the people in that environment. Identifying antecedent variables that effect student performance enables practitioners to set the stage for prosocial behavior and academic learning. One targeted strategy that has been recommended in the literature as essential for insuring student learning is increasing the number of student opportunities to respond (OTR). The current study utilized two separate professional development models, traditional in-service and peer coaching, to assess the effects of optimizing teacher levels of four instructional strategies, often defined as core to OTR, on students' academic and social behavior. Data were collected across a total of 16 teachers and 16 target students across two diverse elementary schools. Standardized, survey, direct observation, and work product measures were collected and are reported across academic, behavior, and social student outcomes. Implications for next steps research regarding instructional strategies as potent antecedents for academic and social behavior outcomes and professional development models are discussed.

**********

One of the central tenets of behavioral psychology is that behavior is functionally related to events that occur within the environment prior to and following an individual's behavior. The literature is replete with examples demonstrating the manipulation of key antecedent and consequent variables and concomitant change in behavior such as "prompting" to occasion a response or delivering a "reinforcer" following a desired behavior to increase and maintain behavior. This simple tenet is the cornerstone in creating classroom environments to promote academic and social learning as exemplified by the effective school literature (e.g., Brophy & Good, 1986; Cotton, 1999). What a child learns both academically and socially is influenced through the interactions the child has with his or her environment including adults and other students (Carta, Atwater, Schwartz & Miller; 1990; Stichter, Lewis, Johnson & Trussell, 2004; Wallace, Bellamy & Hupp, 2002). The literature seems to be less clear about which key antecedent and consequent variables, and the amount and proportion of each, lead to improved learning in the most efficacious manner (Stichter et al., 2004). A related concern repeatedly expressed in the literature focuses on the optimal manner to increase and maintain teacher use of effective research-validated teaching practices (Carnine, 1995; Kauffman, 1996).

The effective schools' literature represents a wide ranging collection of practices that have been demonstrated, largely through evaluative and descriptive studies, to lead to improved academic and social outcomes for students (Kern & Manz, 2004). Likewise, many of the studies conducted that serve as the foundation of the effective schools literature have primarily explored effects among typically developing children (e.g., Brophy & Good, 1986). However, data is emerging through more experimental research methodologies with respect to specific practices among students who are at-risk and those with disabilities. One such practice is increasing the number of opportunities children have to respond to academic or social prompts thereby increasing fluency with material, overall levels of academic engaged time, and reductions in off-task behavior (Bulgren & Carta, 1983; Cooper & Speece, 1990; Englert, 1983; Gunter, Coutinho, & Cade, 2002; Sutherland, Adler, & Gunter, 2003). Increasing student opportunities to respond (OTR) has demonstrated increases in learning among typically developing peers (Slavin, 1994) and those with mild disabilities (Sutherland & Wehby, 2001).

Within the literature, opportunities to respond is typically characterized as a variation of four key variables; 1) specific amounts of teacher instructional talk, 2) prompts, 3) sufficient wait time for student response, and 4) contingent praise for correct responding (Sutherland, Adler, & Gunter, 2003; Stichter et al, 2004). Teacher instructional talk incorporates the essential information necessary to allow student learning and should include demonstration as well as guided practice (Good, Grouws & Ebmeier, 1983). Research suggests that teachers should engage in instructional talk 40-50% of the instructional period to promote maximal student learning (Good, 1984; Roberson, Woolsey, Seabrooks, & Williams, 2004). Prompts are specific requests (oral, written or gesture) for student behavior to demonstrate their understanding of instruction (Englert, 1983). The limited research to date on optimal rates of prompting within an instructional period appear to indicate that an appropriate average is 3.5 prompts per minute during active instruction (Englert, 1983; Sutherland, Adler, & Gunter, 2003).

Following a prompt for student response, sufficient wait time is essential to allow students time to process the request and deliver a response. A wait time of 3 or more seconds following a prompt, or when a student pauses within their response, has been linked to improved student learning (Rowe, 1974; Tobin, 1983). Following a correct or appropriate response on the students' part, contingent "praise," or other reinforcing events should be given to increase and maintain student learning. In examining the research relative to praise, rate does not appear to be the critical factor; meaningful contingent reinforcement is necessary. Amount of praise is typically reported relative to a ratio of corrective to reinforcing statements made by the teacher. A ratio of 4:1 positive to corrective statements has been linked to improved student outcomes (Cameron & Pierce, 1994; Pfeffner, Rosen, & O'Leary, 1985). The sequence of environmental events that make up an OTR have been demonstrated to improve academic responding (DePaepe, Shores, Jack, & Denny, 1996; Gunter & Denny, 1998), increased rates of academic engaged time (Ferguson & Houghton, 1992; Gunter, Hummel & Conroy, 1998) and a decrease in disruptive social behavior (DePaepe et al, 1996; Skinner, Ford, & Yunker, 1991; Wehby, Symons, Canale, & Go, 1998).

While the research relative to OTR is encouraging, similar to many practices advocated within the effective schools literature as well as other research-based practices, insufficient overall levels of OTR and incorrect component sequences are common in both general and special education settings (Sanders, 1998). The issue is best characterized as a "research to practice" gap (Carnine, 1995). A myriad of explanations have been posited for the low level use of research-based practices in schools. A common theme among all the reasons given for the failure to implement research-based practices is the simple fact that, like their students, educators' behaviors are also functionally related to their environment. As stated by Kauffman (1995), the challenge is to ".... construct a culture of support for research-based practices in education" (p. 59). The "culture," or environment in which education of children takes place must not only introduce research-based practices but also provide frequent and on-going support over time to insure teacher success (DePaepe et al., 1996). Presently, most of the professional development efforts put in place within education, and the research examining their overall impact, point to repeated failure in implementing sustained change in practice (Guskey, 1986, 2000). In his seminal work on professional development in education, Guskey (2000) points to three essential features that must be considered to insure sustained implementation of research-based practices: 1) change is a slow, difficult, gradual process for teachers, 2) teachers need to receive regular feedback on student learning outcomes, and 3) continued support and follow-up are necessary after initial training. Unfortunately, the field continues to rely on traditional professional development models primarily relying on "one-shot" in-service with limited or no follow-up support (Guskey, 2000).

Proposed professional development practices that reflect necessary components to insure teacher use of effective classroom practices include the development of "communities of practice" (Boudah, Logan, & Greenwood, 2001; Gersten & Domino, 2001; Greenwood & Abbott, 2001), team-based implementation (Klinger, Ahwee, Pilonieta, & Manadaz, 2003; Sugai et al. 2001), and performance support in the form of "coaching" (Boudah, Logan, & Greenwood, 2001; Guskey, 2000; Klinger, Arguelles, Hughes, & Vaughn, 2001). A variation that attempts to incorporate all three, as well as other professional development strategies, is the use of reciprocal peer coaching (Ackland, 1991). Peer coaching involves dyads of teachers who observe each other during periods of instruction and then share performance feedback relative to specific target skills or overall benchmarks on a range of effective practices. To date, while peer coaching is often advocated as a cost-effective professional development strategy that maps well with common educational practices and is reported as well received among educators, little research has been conducted on the impact related to both teacher and student behavior change. For example, while nineteen reports employing some form of research design have been found in the literature to date, only six report changes in teacher behaviors (Bowman & McCormick, 2000; Hasbrouck, 1997; Hasbrouck & Christen, 1997; Kohler, Crilley, Shearer, & Good, 1997; Morgan, Menlov, Salzberg & Hudson, 1994; Showers, 1984), only two report student outcomes (Morgan, Menlov, Salzberg & Hudson, 1994; Showers, 1984), and the majority (90%) used pre-service teachers as subjects. In summary, little is known empirically about the effectiveness of this popular model of professional development on change in teacher behavior and potential impact on student outcomes.

While the emerging research literature on advocated instructional strategies such as opportunities to respond and its component steps is encouraging, additional research is needed clearly linking specific rates of implementation for these instructional strategies to student outcomes. Likewise, research-based strategies that fail to be implemented with integrity in schools pose a continued educational risk for students (Kauffman, 1996). The purpose of this descriptive study was two-fold. First, the study explored the impact of increased appropriate opportunities to respond on changes to measures of student academic and social behavior. Second, the study compared peer coaching to a more common model of professional development, the "one shot" in-service" also commonly found in educational settings. Specifically, the focus was on the assessment of any change on measured teacher and student behavior related to variance of rates of OTR variables employed in the classroom, as well as information related to those changes on the amount and type of environmental supports put in place to facilitate teacher use of these effective practices.

Method

Overview and Design

The study was conducted through three primary phases. Within the first phase, teacher and student subjects were selected, baseline measures were collected, and in-service on essential features of OTR was provided. The second phase of the study provided additional support to half of the teacher participants in the form of a peer coaching protocol. The final phase consisted of post-intervention data collection and social validity measures. The study employed an overall descriptive evaluative design, with a nested single subject design on a sub-sample of participants, to examine overall and comparative efficacy of the OTR intervention and professional development delivery. Within the nested study, a multiple treatment design was employed using direct observation data to assess both the integrity of implementation as well as the impact of two separate models of professional development on student outcomes,. According to Kazdin (1982), the multiple treatment design allows for the staggered introduction of various interventions across time and participants. With this design, as each intervention is introduced, changes in the dependent measures can with increased confidence be attributed to the intervention by accounting for extraneous variables that can occur through natural maturation and simultaneous influences (Kazdin, 1982). Therefore, reducing threats to internal validity for single subject research.

Subjects and Setting

Two public elementary schools located in a mid-sized city in the Midwest were invited to participate in the current study. Building 1 was identified as low socio-economic status (SES) due to its designation as a Title 1 school. Building 2 had a proportionally low percentage of students eligible for free and reduced lunch and was commonly referred to as one of the higher SES elementary buildings in the district. At the time of the study both schools were successfully implementing School-Wide Positive Behavioral Support (SW-PBS). SW-PBS (see Lewis & Sugai, 1999 for a full description) is technically defined by the implementation of six essential elements commonly assessed through the School-wide Evaluation Tool (SET; Horner, Todd, Lewis-Palmer, Irvin, Sugai, & Boland, 2004). Each of the schools had received SW-PBS professional development training from a team at the authors' university and each had established a previous research agreement with the institution. Therefore, all members of the school staff and student body were familiar with the presence of researchers in a variety of school settings and in a variety of roles. Research continues to support that this type of familiarity can positively impact the validity of research findings whereby participant behaviors are less likely to be affected by the presence of observers with whom the participants are familiar (Fox, Gunter, Davis, & Brall, 2000; Kazdin, 1982).

Consistent with the SW-PBS initiative in both buildings, school administration agreed to incorporate the four targeted OTR instructional strategies as a building wide initiative for all faculty and staff. However, due to resource limitations, the study design only included 16 teachers (8 in each building). The participants were divided into two groups across the two schools and assigned to a traditional in-service activity and in-service plus peer coaching. The two groups of 8 were further divided into cohorts of 4 with respect to additional direct observation data collection activities conducted within the nested study. The direct observation cohort was developed to provide a form of treatment integrity data (unprecedented in the peer coaching literature) and to ensure that comparisons could be made across both professional development models. The targeted 16 teachers were chosen based on willingness to participate and equal representation across grades kindergarten through fifth. From this group of 16, teachers were assigned to equal groups of either peer coaching or in-service only.

Building 1 had one teacher representing grades kindergarten, first, fourth, and fifth and two teachers representing grades second and third. The teacher participants in this building had teaching experience ranging from 3 to 33 years, with a mean of 13 years. Two of the teachers had earned a Bachelor's degrees, five Master's degrees, and one had attained a Doctorate. Building 2 had one teacher representing grades kindergarten, second, and third, two teachers representing fourth grade, and three teaching fifth grade. Their experience ranged from 2 years of teaching to 25 years and a mean of 14 years. Two of the representative teachers had completed Bachelor's degrees, four had Master's degrees, and two hold Doctorate degrees.

Data on 16 target students from each of the 16 teacher classrooms were also collected. Teachers were asked to choose students for whom there were academic and social behavior concerns. In building 1, participating students included 6 African American and 2 Caucasian students with an equal representation of male and female students. Building 2 included 1 African American student, 7 Caucasian students, 7 males, and only 1 female. Building 1 had no students on Individualized Education Plans (IEP) that were represented in this study. However, Building 2 had four children with IEP's. The students with IEPs were receiving special education services under the following disability categories: Attention Deficit Disorder, Learning Disability, and Other Health Impairment and the students' diagnoses were not mutually exclusive.

Procedures

The lead author met with each administrator to present a letter of explanation for the study and to answer any questions concerning the details of the study. Each administrator was told that the study was designed to assess the interaction between the provision of the four OTR strategies at optimal levels, as identified in the literature, and students' social and academic behavior. Additionally, the administrator was informed that the study would assess variations in achieving optimal levels of strategy implementation across the two models of professional development selected for use in the study. A brief description of each teaching strategy was provided as well as details concerning when, types of, and how often data would be collected.

Teachers who indicated an interest in participation were asked to submit their classroom schedule. It was decided that data would be taken during literacy instruction, which provided for the best standardization across content of instruction. The sixteen participants were chosen according to consistent availability for participation throughout the 16 week period and were then matched to teams of data collectors. Teachers selected one student who had behavioral concerns and a permission form was sent to the student's home. After the permission forms were returned, each teacher was interviewed and an operational definition of each participating student's primary behavior of concern was established.

In-service on OTR. Before the study began, the lead author provided a 2-hour in-service for all faculty and staff in each building to review the four instructional variables of OTR and the timeline for their introduction. In addition, the lead author provided a half-hour "refresher in-service" before the introduction of each of the four teaching strategies. The in-services consisted of a presentation and a handout. The handout, titled the 'Teacher Cool Tool' tip sheet, described each strategy and presented the research support, academic and behavioral examples, and a quantified optimal level of implementation for each strategy. Each cool tool strategy was introduced every four weeks, resulting in four phases totaling 16 weeks. All faculty and staff in both buildings were expected, by building administration, to target both the newest 'cool tool' strategy as well as any previously introduced cool tool strategy in their daily instructional practices. Reminders about the 'cool tool of the month' were provided via the school's public announcement system each day and the principal sent reminders via email every two weeks.

Peer Coaching. In addition to the in-service listed above, as was also typical for peer coaches, an additional 3-hour initial training was provided to the eight teachers involved in peer coaching. The lead author instructed the peer coaches on specific strategies regarding 1) observation of peers, 2) pre and post observation conferencing and feedback procedures, and 3) data collection procedures. In order to ensure that observation data could be taken, peer coaches were provided assistance in scheduling substitutes to cover class. They were also provided with supplemental literature on incorporating peer coaching for improvements in instruction and student learning and with access to ongoing email assistance from the lead author throughout the duration of the study. The eight core teachers involved in peer coaching were placed into dyads and were engaged in a total of eight 'peer coaching sessions' for each peer coach, 2 sessions per each cool tool strategy. Peer coaches sessions included an observation of their partner peer as well as both a pre- and post-conference for each peer coaching observation. Pre-conference procedures were clearly outlined and provided a minimum of 15 minutes within 24 hours of the observation to identify the lesson plan goals, and to refresh target student behavior. This way the peer coach who would be doing the observation was correctly oriented toward the lesson they were observing (Ackland, 1991). Post-conferencing also followed a set protocol designed to ensure that the peer coach could debrief with the observed teacher and share their data and observations. Both procedures were highly scripted for the peer coaches in the form of conferencing sheets that were to be filled out for each observation. Peer coaches, just like all teachers in the building, would spend the first two weeks of each cool tool month working on the target strategy (as well as any previously introduced cool tool) and then, during the last two weeks of each phase, would engage in the peer coaching sessions.

Data Collection and Measures

Table 1 provides an overview of the teacher participants and corresponding measures used across the principal and nested studies. Measures are further delineated by participant and target.

Teacher instructional behaviors. Direct observation data was collected using the Multiple Option Observation System for Experimental Studies (MOOSES) software program on hand-held computers (Tapp, Wehby & Ellin, 1995) for the subset of 8 participants. A total of 9 codes was measured in real time, including codes for both teacher and student behavior. Teacher behavior codes included both duration and frequency codes, representing the four OTR instructional strategies, as well as default codes for when teachers were engaged in other non-targeted behaviors. Direct observation data was collected across the study for a total of 256 hours. Two complete peer coaching cycles per cohort for each four-week period was collected; including data on each participant teacher and target student during literacy instruction period. There were two primary and two secondary data collectors involved in the study.

Training for all four data collectors was conducted using the MOOSES program and through observing videotapes of classroom instruction. Each data collector was trained until a reliability criterion of 80% or higher agreement was met. They were trained using a modified version of the Setting Factor Assessment Tool (SFAT), (Stichter, et al., 2004) which is designed to concurrently assess multiple commonly researched classroom-based instructional variables involving both typically developing and at risk students as well as students with mild disabilities (Brophy & Good, 1986; Greenwood et al, 1994; Sutherland, Alder & Gunter, 2003).

Reliability was also monitored for both teacher and student behavior (described below) throughout the entire study. Data was collected on implementation of the teaching strategies, by direct observation two times a week, during sessions ranging from 45 minutes to 1 hour, during literacy instruction.

Peer coaching. Peer coaching data was collected from the peer coaches' pre-and post-conferencing data sheets. These were coded for verification of occurrences of peer coaching sessions as scheduled, for length of peer coaching observations as scheduled and for accuracy of implementation of conferencing processes. Any discrepancies in procedures were viewed as requirements for re-training of peer coaching dyads. This was necessary on two occasions with two separate dyads. University data collectors, for the purpose of integrity of implementation, observed two of the 8 sessions for each teacher including accompanying conferences.

Student social behavior. Participating teachers completed the Social Skills Rating Scale (SSRS; Gresham & Elliott, 1990), which provides a multi-rater assessment of student social behavior and how it can affect student-teacher relations for each of the 16 subjects pre- and post-intervention. The elementary level teacher version was used to assess three behavioral domains: social skills, problem behavior, and academic competence.

Direct observation data were collected simultaneously for a sub set of eight students whose teachers were in the nested study. Therefore students direct observation data could be collected simultaneously with their teacher's direct observation data. Observed student behaviors were individually defined for each student but included three general categories. The first was "disruptive" behavior which included attempting to distract the teacher, interrupting seatwork time or circle time, outbursts, crying, refusal, talking out during teacher instruction, or with others, and interfering with someone else's ability to listen and learn. A second category was general "off-task" behavior which was defined as behaviors that included talking with others, fidgeting with items/objects, making noises or silly faces, and interfering with one's own ability to listen and learn but not creating overall disruptions to the learning process. The third was an "on-task" category which was defined as the student being appropriately engaged in learning as defined by the task (e.g., listening during teacher instruction, responding to teacher prompts).

The School Archival Records Search (SARS; Walker, Block-Pedego, Todis, & Severson, 1991) was used to provide a record of each student's school adjustment and academic performance. For the purposes of this study, we selected school office discipline referrals as an additional measure of student social behavior performance.

Student academic behavior. Through the SARS, student standardized reading scores were compared pre- and post-intervention. In addition, a weekly work sample of graded literacy related work products for each target student in the participating classrooms was also collected. For consistency in analysis, each work sample was coded for student accuracy using an "A-B-C" scale. Most of the participating elementary teachers did not provide A-B-C grades so feedback from teachers on the graded work was converted via teacher directions. Each teacher provided the metric or rubric (e.g., what would constitute a 90%) for the work sample by which the work was coded.

Intervention integrity. Each month, peer coaches were provided with a timer and data collection sheets that included codes for each new cool tool strategy. The peer coaches were asked to do momentary time sampling data procedures every 10 seconds by circling the corresponding code if, at the 10 second timer, the teacher was engaged in any of the current cool tool strategies and if the target child was engaged in the target behavior of concern. Wait time was a duration measure and the coding procedure reflected appropriate modifications. Each peer coach was trained as part of the initial training mentioned previously. Additionally, as each new cool tool was introduced, they were given new data sheets including the new code and explicit written directions (1/4 page containing two statements and sub bullets). University data collectors took data simultaneously on 2 of the total 8 sessions for each peer coach to insure correct teacher measurement. Additionally, data collectors sat in on 25% of all pre and post peer coaching conferences to insure that procedural integrity of that the process was accurately being followed.

Social Validity. One of the impetuses for simultaneously studying the peer coaching and the standard in-service models was based on the interest by the staff to initiate a peer coaching process. Additionally, the majority of the literature on peer coaching has relied highly on this interest and enthusiasm for such a professional model. Therefore, it was essential to assess the feasibility of the two professional development models as well as the effects on the implementation of the optimal levels of OTR strategies. A likert scale social validity survey was given to all teachers in both buildings and coded to separate participants of the study and the remaining staff that had engaged in the school wide initiative of working on optimizing the four OTR variables through the traditional in-services. Since all staff in both buildings received the in-services and had been asked to engage in optimization of the OTR variables, the majority of the social validity survey included 2 categories of questions: the usefulness of the 'Cool Tool' guides: and the ease of implementing the instructional strategies. The survey was given to all members of each building, including participants within the study. A one (very useful) to five (not at all useful) rating scale was used for each question. Mean scores were calculated, by building, for both the usefulness of the tip sheets and the ease of the implementation dimension questions. An additional set of questions were given to the sub-sample who participated in the peer coaching process.

Results

As this study represents an initial examination of the two types of professional development models related to changes in OTR variables, data are summarized in a descriptive manner. Teacher and student data summarized in the following section include changes in rates OTR variables (instructional talk, prompting, feedback, and wait time) teachers as well as student academic and behavioral measures. All measures were descriptively compared pre-post while direct observation data was also collected to explore possible trends. Given the pilot nature of this descriptive study and the small number of teachers involved, statistical analysis was not conducted. However, where appropriate, data from this study are compared to the literature in this area.

Teacher Outcomes: Opportunities to Respond

Across the sub-sample of 8 teachers, direct observation data across the four instructional strategies were compared to optimal levels identified in the literature of the four indicator variables. Mean instructional talk in the current study (46%), is very close to the optimal goal (45%). Mean prompts (2.2) did not meet the 'recommended' optimal prompts per minute (3.63). The mean positive to negative feedback ratio (4.5:1) is close to the optimal ratio (4:1). Finally, mean wait time in the current study (5.6) seconds, is slightly higher than the recommended 3 to 5 seconds. All 8 of the teachers observed in this study had been part of a larger descriptive study during the previous year. This study assessed 36 different teaching variables during literacy instruction time as well, using the same direct observation system over 5 days for a total of 5 hours (Stichter, Lewis, Whittaker, Richter, Johnson, & Trussell, 2004). The four OTR variables were among the variables measured in the previous study and provided baseline data for the instructional variables. Teacher's use of Instructional Talk at baseline ranged from 47% to 95% and dropped to a range of 36% to 57%, after targeted implementation of instructional strategies. The mean range of prompts at baseline was 1.91 to 3.65 and dropped slightly after introduction of the prompting tip sheet (1.6 to 2.9). The positive to negative feedback mean ratio ranged from .81 to 12.37, at baseline, and steadied at 3.0 to 7.5 across phases. The mean range of wait time was 0 to 6.5 at baseline and rose to 3.5 to 11.7 by the end of the study. Table 2 provides summaries of instructional change and displays whether the preset criterion for each instructional strategy was attained across the subset of 8 teachers who were directly observed.

Generally, results indicate an improvement in the use of instructional strategies making up OTR for most teachers. Improvements were most evident with teachers' use of instructional talk and feedback. Specifically, 5 of the 8 teacher participants met the criterion goal for instructional talk, 4 of the 8 for feedback, 2 of the 8 for wait time, and none for prompting. Teacher 1 showed the greatest improvement, by improving the use of all four instructional variables and meeting three of the four goals.

The mean for both Building 1 (Title) and Building 2 (Non-Title) were calculated to examine the relationship among instructional variables. Teachers in Building 1 met 38% of the instructional goals and made improvements in 81% of their potential opportunities across phases of the instructional strategies. Building 2 teachers met 31% of the instructional goals (denoted by * in Table 2) and made improvements in 63% of the potential opportunities. Overall, school teachers in Building 1 met slightly more goals (by one) and improved their use of the instructional strategies more (by one) than teachers in Building 2. Teachers involved in peer coaching, together, met a total of 6 goals (across the four instructional domains), whereas teachers not involved in peer coaching met a total of 5 goals. The peer coaching group also had a slightly larger number of teachers making improvements in their use of the instructional strategies (12) than the comparison group (11).

Student Academic and Social Outcomes

Work samples. Student work products (permanent literacy products), assessed during the first phase of the study, were used as baseline data for subsequent comparisons of growth for each student during phases 2-4. It should be noted since the design of the study called for 16 weeks of intervention in staggered phases, it was important to begin as early in the school year as possible. However, since K-5 was represented and these grades could potentially introduce confounds due to other beginning of the year learning curves, the initial 6 weeks of the school year were avoided, The majority of students improved in permanent work product across all phases as compared to baseline. Additionally, directional changes in teacher behavior, as shown in Table 2, directly corresponded with the same directional change in students' permanent product scores for 18 of 24 overlapping teacher/student data points. Non corresponding data points were twice as likely to occur, for teachers involved in peer coaching, as compared to those only receiving the in-service only (see Table 2). On a scale of 1-4 with 4 being the equivalent of above 95% correct or "A" work, the mean for the two student groups, students whose teachers participated in peer coaching (mean=2.77) or those who did not (mean=2.52), were almost equal. Similarly, when comparing across the two schools, no difference for academic improvement was evident between the two. Building 1(the title school) was calculated as having a mean score of 2.65 and building 2 (the non-title school), a mean score of 2.68. This was particularly interesting given the fact that one-fourth of the students in building 2 were on IEPs.

Standard literacy scores. Literacy levels were gathered from district wide literacy assessments at each grade level at the beginning of the year and at the end of the year. District ranges for 'below,' 'at' or 'above' grade level for each corresponding assessment were used to assign each child one of these categories for their pre and post assessment scores. Table 3 displays changes in literacy levels based on these scores as well as office referral data which will be discussed in a forthcoming section. Grade level standards for literacy were used to measure marked change because different grade levels used different standardized assessments and it was not feasible to measure growth across students based on changes in raw scores. The students of teachers in the in-service only group demonstrated greater improvements in literacy scores when compared to the students of teachers in the Peer Coaching (PC) group, with 5 in-service only students showing improvements in literacy levels and 3 students showing no change in literacy levels. In contrast, only 2 students in the PC group demonstrated improvements in literacy scores, while the levels for 5 students in this group remained the same and for one student, decreased.

Student social behavior. Table 4 summarizes changes that were recorded through direct observation of students' problem behavior within, the nested study. Despite indications that overall change reflected an increase in rates of problematic behavior over baseline, comparison of the mean data analyses indicates minimal change observed across direct observation probes. Mean scores were further calculated to compare the two student groups' percent of problem behavior. Students whose teachers were involved in peer coaching had a mean duration of 18.1% problem behavior compared to 17.8% problem behavior for students whose teachers were not involved in peer coaching. Only one student had a decrease in problem behavior, while the rest of the students demonstrated an increase, ranging from 6% to 30%.

For purposes of comparison, broad behavior categories were developed from the original definition of each student's target behaviors. Students were grouped based on overall levels of "disruptive" or "general off-task." Differences were evident between the behavioral means of the two identified groups. Students who displayed off-task behavior had a behavioral mean of 37% (peer coaching group) and 20.5% (non peer coaching group). Students who were defined as disruptive had a behavioral mean of 9.25% (peer coaching group) and 9.75% (non peer coaching group).

There was a surprising difference in the mean score of problem behavior between the Title (9.1%) and Non-Title schools (30.7%). Although there are no statistical or empirical data to indicate why this difference existed, two factors may have logically contributed to this discrepancy. First, the non-title building had a larger proportion of students on IEPs then did the title building. Second, the teachers in the title building were proportionately more likely to have children at risk for behavioral and academic challenges in their classrooms. Anecdotal observation by university staff suggests that they may have selected students to which support plans were not in place or other mitigating factors had not already been attributed to the behaviors of concern, resulting in students with less chronic problems participating from that building.

Table 5 provides a visual representation of changes in student percentiles based on the pre- and post-study SSRS scores, according to the three subscales of social skills, problem behavior, and academic competence. Improvements in both academic and social skill behaviors are defined by an increase in change of percentile of behaviors. Problem behavior is defined as a decrease in change of percentile of behavior. According to percentile ranks from normative post-test data for the SSRS, the mean for academic competence is 26 and for problem behavior, 77. In this study the pre-test mean score for academic competence was 22 and 80 for problem behavior. A total of 9 students improved in social skills behavior, while social skill behavior for 6 students declined, and one student stayed the same. Ten students decreased their percent of problem behavior, 5 increased, and 1 remained the same. According to the SSRS scale, by the end of the study, students whose teachers were involved in peer coaching displayed less problem behavior. Finally, 11 students improved in the area of academic competence, 3 decreased, and 2 remained the same. According to the SSRS scale, 7 of the 8 students whose teachers did not receive peer coaching made academic improvements. Only half of the students whose teachers did receive peer coaching showed academic improvement.

The number of office discipline referrals was pulled from the SARS data base for each child prior to the start of the study and after each phase and at the end of the year. The number of referrals, however, was generally low (total of 34 referrals across the two groups for the year) with one student accounting for nearly a third of the referrals for both groups. Therefore changes across phases were barely notable with occasionally more than a difference of one referral. Therefore, only pre- and post-data are displayed in Table 3. Though the students from the in-service only group demonstrated a slightly higher increase in the number of office referrals, the number of office referrals for both groups remained relatively constant for the year.

Social Validity Data

Social validity data for the usefulness and ease of implementation for the tip sheet was overall positive across both buildings. This was noteworthy as all faculty and staff in both building participated in this part of the initiative and therefore answered this component of the social validity survey. For the Title 1 building, the mean score for the usefulness of the tip sheets was 39% for ratings 1 and 2; 22% for rating 3; and 0% for ratings 4 & 5. Mean scores for ease of implementation were 42% for rating 1; 48% for rating 2; 5% for ratings 3 & 4; and 0% for rating 5. In the Non-Title building, the mean scores for usefulness of the tip sheets were 20% for rating 1; 34% for rating 2; 31% for rating 3; 9% for rating 4; 3% for rating 5; and 3% answered 'not applicable.' The mean scores for ease of implementation were 29% for rating 1; 26% for ratings 2 & 3; 13% for rating 4; and 6% for rating 5.

The peer coaching dyads were then asked additional questions pertaining to the ease of usefulness of the peer coaching methodology in addition to the in-service. All 8 teachers in this group indicated that there were advantages as reflected in the following positive attributes listed: increased dialogue between teachers, opportunity for feedback, increased reflection and opportunities to observe other classrooms. They also all listed at least one disadvantage and included the following limitations: too much time, challenges with data collection and a dislike with pre and post conference procedures. Two of the teachers in the title building and two in the non-title building indicated that peer coaching changed their behavior through increased awareness of teaching practices. One of the teachers in the non-title building indicated that s/he had become more positive and the other teacher in that building cited an increase in teacher cool tool use because of it. Two of the teachers in the title building gave no response to that question. Three of the four teachers in the non-title and one of the title teachers indicated that they felt positive changes had occurred for their students as a result of their own engagement in peer coaching. Three of the teachers (two non-title) rated recommending peer coaching to a colleague as very much (rating 1) and another teacher from the same building rated it as a 2 (somewhat) with three rating it a 3(neutral). One teacher (title building) rated it a 5 (not at all).

Reliability

Reliability data were collapsed and are described below by phases. Reliability measures were taken during 45% (range 40-50%) of all observations across all phases and were calculated by dividing the number of intervals the two observers agree by the number of intervals they agree plus the number of intervals they disagree, then multiplying by 100 (Kazdin, 1982). Reliability mean for Phase 1 was 91% (range 74-100%); Phase 2, 92% (range 71-100%); Phase 3, 93% (range 66-100%); and Phase 4, 93% (range 71-100%).

Discussion

The purpose of this study was to examine the overall impact and component analyses of providing opportunities to respond by teachers on student behavior relative to the amount and type of environmental support put in place to support teacher use of this effective practice. The teachers' use of the identified strategies, as well as the professional development model employed to support the teachers, including any feedback they receive on their instruction (peer coaching), is an important variable to consider when analyzing students' academic and social outcomes. The degree to which these serve as a moderator for treatment effectiveness remains under investigated related to outcomes for students (Sutherland, Palmer, Stichter & Morgan (in press). Using a variety of assessments, the effects of two professional development models on both teachers' behavior as well as students' academic and social behavior outcomes were assessed.

Data from the current study support previous work that demonstrates that teachers can reach improved levels of OTR variables with further implications for positively associated changes in student behavior and academic outcomes (Good, 1984; Gunter, 1998; Stichter et al, 2004). Despite unremarkable changes in directly observed social behavior, work product data, and SSRS, literacy scores suggest that a majority of the students demonstrated academic growth. When outcome data was compared across the two different professional development models, neither model was rendered more effective in this study. Changes in teacher behavior occurred at a slightly rate with those engaged in peer coaching, yet positive changes in student academic outcome was more closely associated with those teachers engaged only in the in-service model. Most teachers improved their rates of each strategy as compared to baseline levels. Both peer coaching and in-service only teachers were similar in the degree to which they changed their behavior and the number of whom met optimal ranges. Peer coaching dyads met slightly more (difference of one) of the target ranges. Social validity data indicated a similar distribution regarding perceptions of the utility of the peer coaching model. Academic performance gains for students did improve more incases when teachers reached optimized rates of the OTR strategies. Yet in cases where it did not, this was twice as likely for the peer coaches. Due to the small number of participants for this study (16 total) this data is no more than a point of interest to note for future replications. Changes in district literacy levels across the two professional models presented similar results indicating almost 2 and a half times as many children improving their literacy levels by the end of the year in the non-peer coaching classrooms than in the peer coaching classrooms. No other factors such as SES or special education status seemed to account for this trend. Again, sample size in conjunction with age ranges and teacher volunteerism could have all been mitigating factors to name a few.

Identifying ways to optimize teachers' use of specific instructional antecedents designed to enhance the amount and type of student response is considered essential to realizing effective instruction for students with and without disabilities (Carta et al, 1990; Stichter et al, 2004; Wallace et al, 2002). The current study contributes to the research literature in several fundamental ways. First, it adds to the literature base on OTR by simultaneously, and, in real time, assessing four key variables, instructional talk, prompts, wait time and feedback, as they occur naturally within literacy instruction for students with and without disabilities. Teachers in this study demonstrated that previously defined "optimal" levels can be achieved and maintained during literacy instruction across K-5 grade levels. Future research needs to determine whether this is consistently true for literacy as well as other content areas, a comparison, not fully discussed within the effective schools literature. Secondly, this study has contributed to a limited amount of research available that examines both teacher and student behavior in relation to teaching practices and professional development models. For example, the literature on professional development is replete with support for communities of practice and coaching (Boudah et al, 2001; Guskey, 2000). Yet, little empirical data is available on the impact of peer coaching models on teacher behavior and even less is available on student outcomes, particularly measures taken concurrently. Ultimately refining measures that can concurrently assess the level of skill transference and then the concomitant impact of the level of that new skill on student outcomes in applied settings would provide a meaningful triangulation of data toward closing the research to practice gap.

Limitations

The current study was an exploratory study designed to examine teachers' ability to effectively execute the components of OTR within the context of two professional development models. However, given the limited experimental research on peer coaching models, few conclusions can be made regarding the appropriateness of the peer coaching protocol employed for this study without extensive replication with a larger number of teachers in a similar study. Additionally, the design of the current study incorporated two schools already engaged in SW-PBS, and therefore, by definition, actively committed to pursuing a culture of systematic change. Their ongoing efforts as part of this change process may have uniquely impacted their perceptions of both models and their ability to employ change under each format. Similarly teachers who were part of the nested multi-element design chose the professional development model in which they were actively engaged (i.e., peer coaching) and their partners. A stronger design for subsequent studies could explore randomly assign volunteers to the two separate professional development models, observe more often and over more time. However, it should be noted, to date this has not been done because by definition, there exist several purposes for why peer coaching is undertaken and as a result most forms of reciprocal coaching involve teachers volunteering to learn or refine application of skills and knowledge simultaneously (Ackland, 1991). To remove volunteerism completely and randomly assign teachers to peer coaching dyads may pose a unique threat to construct validity. Finally, the measures utilized in this study were a combination of those commonly accessed through the schools (i.e., district literacy scores) and those that represented a breadth of measures for each dependent variable. Although, sufficient for an initial exploratory study, these measures were limited in their sensitivity to changes in student behavior related to changes in teacher behavior, and in some cases lack standardization, creating threats to external validly.

Implications

There remains a critical need to further define not only effective practices for classroom instruction, but particularly those that seem most universal to occasion and reinforce desired student behavior. OTR and the component variables warrant ongoing investigation that further defines the impact of each variable and the cumulative effect of the variables on one another. This research would include further investigation of optimal levels of each variable, in isolation, in combination and across content areas which often promote varied instructional formats. Additionally, more sensitive measures would be included that allow for specific monitoring of changes in student outcomes as they respond to teacher behavior change. Specifically designed curriculum based measures may be useful and practical for such investigations in conjunction with pre--post ratings and screenings to compare product data, observation data and perceptions. Understanding how well these forms of data do and do not triangulate will be essential to quantify decades of effective schools literature as well as assist schools in developing meaningful measures to monitor their own professional development and skills acquisition. Finally, further research comparing professional development models is necessary to understand if, and or when, types of professional development as compared to factors related to school building culture or faculty preference effect implementation and maintenance of identified variables for effective instruction. By pursuing and combining these lines of inquiry, the goal will be toward identification of specific variables that elicit and reinforce enhanced student engagement most efficaciously and consistently.

References

Ackland, R. (1991). A review of the peer coaching literature. Journal of Staff Development, 12(1), 22-27.

Brophy, J. H. & Good, T. (1986) Teacher behavior and student achievement. In M.C. Wittrock (Ed.), Handbook of research in teaching. (3rd ed., pp 328-375). New York: Macmillian.

Bulgren, J. A. & Carta, J.J. (1993). Examining the instructional contexts of students with learning disabilities. Exceptional Children, 59, 182-191.

Carta, J. J., Atwater, J. B., Schwartz, I. S., & Miller, P. A. (1990). Applications of ecobehavioral analysis to the study of transitions across early childhood settings. Education and Treatment of Children, 13, 298-315.

Boudah, D.J., Logan, K.R., & Greenwood, C.R. (2001). The research to practice projects: Lessons learned about changing teacher practice. Teacher Education and Special Education, 24(4), 290-303.

Bowman, C.L., & McCormick, S. (2000). Comparison of peer coaching versus traditional supervision effects. The journal of Education Research, 93(4), 256-261.

Cameron, J., & Pierce, W.D. (1994). Reinforcement, reward, and intrinsic motivation: A meta-analysis. Review of Educational Research, 64(3), 363-423.

Carnine, D.W. (1995). The professional context for collaboration and collaborative research. Remedial and Special Education, 16, 368-371.

Cooper, D.H. & Speece, D.L. (1990). Maintaining at-risk children in general education settings: Initial effects of individual differences and classroom environment. Exceptional Children, 54, 117-128.

Cotton, K. (1999). Research You Can Use To Improve Results. Northwest Regional Educational Lab., Portland, OR.; Association for Supervision and Curriculum Development, Alexandria, VA.

DePaepe, P.A., Shores, R. E., Jack, S.L. & Denny, R.K. (1996). Effects of task difficulty on disruptive and on-task behavior of students with severe behavior disorders. Behavioral Disorders, 21, 216-225.

Englert, C. S. (1983). Measuring special education teacher effectiveness. Exceptional Children, 50(3), 247-254.

Ferguson, E. & Houghton, S. (1992). The effects of contingent teacher praise, as specified by Canter's Assertive Discipline program, on children's on-task behavior. Educational Studies, 18, 83-94.

Gersten, R. & Domino, J. (2001). The realities of translating research into classroom practice. Learning Disabilities Research and Practice, 16(2), 120-130.

Good, T. (1984). Teacher Effects. Making Our Schools More Effective: Proceedings of Three State Conferences, p.25 ERIC Clearinghouse: EA017110.

Good, T., Grouws, D., & Ebmeier, H. (1983). Active mathematics teaching. New York: Longman.

Greenwood, C. R., & Abbott, M. (2001). The research to practice gap in special education. Teacher Education and Special Education, 24, 276-289.

Gunter, P.L., & Denny, R.K. (1998). Trends and issues in research regarding academic instruction of students with emotional and behavioral issues. Behavioral Disorders, 24, 44-50.

Gunter, P.L., Coutinho, M.J., & Cade, T. (2002). Classroom Factors Linked with Academic Gains among Students with Emotional and Behavioral Problems. Preventing School Failure, 46(3), 126-32.

Gunter, P.L., Hummel, J.H., & Conroy, M.A. (1998). Increasing correct academic responding: An effective intervention strategy to decrease behavior problems. Effective School Practices, 17(2), 55-62.

Guskey, T.R. (1986). Staff Development and the Process of Teacher Change. Educational Researcher; 15(5), 5-12

Guskey, T. R. (2000). Evaluating Professional Development. Thousand Oaks, CA: Corwin Press.

Hasbrouck, J. E. (1997). Mediated peer coaching for training preservice teachers. The Journal of Special Education, 31(2), 251-277.

Hasbrouck, J. E., & Christen, M.H. (1997). Providing peer coaching in inclusive classrooms: A tool for consulting teachers. Intervention in School and Clinic, 32(3), 172-177

Kauffman, J. M. (1996). Research to practice issues. Behavioral Disorders, 22, 55-60.

Kern, L. & Manz, P. (2004). A look at current validity issues of school-wide behavior support. Behavioral Disorders, 30(1), 47-59.

Klinger, J.K., Arguelles, M.E., Hughes, M. T., & Vaughn, S. (2001). Examining the schoolwide "spread" of research-based practices. Learning Disability Quarterly, 24, 221-234.

Klinger, J.K., Ahwee, S., Piloneta, P. & Menendez, R. (2003). Barriers and facilitators in scaling up research-based practices. Exceptional Children, 69(4), 411-429.

Kohler, F. W., Crilley, K.M., Shearer, D.D., & Good, G. (1997). Effects of peer coaching on teacher and student outcomes. The Journal of Educational Research, 90(4), 240-250.

Morgan, R.L., Menlove, R., Slazberg, C.L., & Hudson, P. (1994). Effects of peer coaching on the acquisition of direct instruction skills by low performing preservice teachers. The Journal of Special Education, 28(1), 59-76.

Pfeffner, L. J., Rosen, L. A., & O'Leary, S. G. (1985). The efficacy of an all-positive approach to classroom management. Journal of Applied Behavioral Analysis, 18, 257-261.

Roberson, L., Woolsey, M. L., Seabrooks, J., & Williams, G. (2004). An ecobehaviroal assessment of the teaching behaviors of teacher candidates during their special education internship experiences. Teacher Education and Special Education, 27, 264-275.

Rowe, M. (1974). Wait time and rewards as instructional variables, their influence on language, logic and fate control: Part I--Wait time. Journal of Research in Science Teaching, 11, 81-94.

Sanders, W.L. (1998). Value-Added Assessment. School Administrator, 11(55),24-27.

Showers, B. (1984). Peer coaching: A strategy for facilitating transfer of training. Report to the U.S. Department of Education. Eugene, Oregon: Senter for Educational Policy and Management, University of Oregon. ERIC EC 271 849.

Skinner, C. H., Ford, J. M., & Yunker, B. D. (1991). A comparison of instructional response requirements on the multiplication performance of behaviorally disordered students. Behavioral Disorders, 17, 56-65.

Slavin, R.E. (1994). Quality, Appropriateness, Incentive, and Time: A model of instructional effectiveness. International Journal of Educational Research, 21, 141-157.

Stichter, J. P., Lewis, T. J., Johnson, N., & Trussell, R. (2004). Toward a structural assessment: Analyzing the merits of an assessment tool for a student with E/BD. Assessment for Effective Intervention, 30, 25-40.

Sutherland, K.S., Adler, N. & Gunter, P.L. (2003). The effects of varying rates of opportunities to respond to academic requests on the classroom behavior of students with EBD. Journal of Emotional & Behavioral Disorders, 11, 239-248.

Tobin, K. (1983). The influence of wait time on classroom learning. European Journal of Science Education, 5(1), 35-48.

Wallace, T., Anderson, A. R., Bartholomay, T., & Hupp, S. (2002). An ecobehaviroal examination of high school classrooms that include students with disabilities. Exceptional Children, 68, 345-359.

Wehby, J. H., Symons, F. J., Canale, J. A., & Go, F. J. (1998). Teaching practices in classrooms for students with emotional and behavioral disorders. Behavioral Disorders, 24, 51-56.

Janine P. Stichter, Timothy J. Lewis, Mary Richter, Nanci W. Johnson, and Linda Bradley

University of Missouri

Janine Stichter, PhD, University of Missouri-Columbia, Department of Special Education, 303 Townsend Hall, Columbia, MO 65211; e-mail: [e-mail].

Table 1 Overview of Participant groupings and corresponding measures

                        Treatment
Teacher     School      Condition          Represented Measures

Teacher 1   Building 1  In-service & Peer  - Pre/Post Social Skills
Teacher 2   Building 1  Coaching           Rating Scale (SSRS),
Teacher 3   Building 2  with Direct        - School Archival Records
Teacher 4   Building 2  Observation        Search (SARS),
                                           - Pre/Post Conferencing Form,
                                           - Student Work Sample,
                                           - Direct Observation
                                           - Social Validity Data
Teacher 5   Building 1  In-service         - Pre/Post Social Skills
Teacher 6   Building 1  with Direct        Rating Scale
Teacher 7   Building 2  Observation        - School Archival Records
Teacher 8   Building 2                     Search,
                                           - Student Work Sample,
                                           - Direct Observation
                                           - Social Validity Data
Teacher 9   Building 1  In-service & Peer  - Pre/Post Social Skills
Teacher 10  Building 1  Coaching           Rating Sea
Teacher 11  Building 2  with NO Direct     - School Archival Records
Teacher 12  Building 2  Observation        Search,
                                           - Pre/Post Conferencing Forms
                                           - Student Work Sample,
                                           - Social Validity Data
Teacher 13  Building 1  In-service with    - Pre/Post Social Skills
Teacher 14  Building 1  NO Direct          Rating Scale,
Teacher 15  Building 2  Observation        - School Archival Records
Teacher 16  Building 2                     Search
                                           - Student Work Sample,
                                           - Social Validity Data

Table 2 Summary of Teacher and Student academic change across four
phases

                     In-service plus Peer Coaching Group
                                                  (Change in students'
Participant  Teacher Change : Direct Observation  Academic Scores)
             (Phase 1)           (Phase 2)        (Phase 3)  (Phase 4)
             Instructional Talk  Prompts          Feedback   Wait Time

 1           +*  (/)             +  (+)           +*  (+)    +*  (+)
 2           +*  (/)             -  (-)           +   (+)    +   (-)
 3           +*  (/)             +  (+)           -   (+)    +   (+)
 4           +*  (/)             +  (+)           -   (+)    -   (+)
 9               (/)                (-)               S          (+)
10               (/)                (-)               (-)        (+)
11               (/)                (+)               (+)        (-)
12               (/)                (+)               (-)        (+)

                         In-service Only Group
             Instructional Talk  Prompts          Feedback   Wait Time

 5           +   (/)             -  (-)           +   (+)    +*  (+)
 6           +   (/)             -  (+)           +*  (+)    +   (+)
 7           -   (/)             -  (-)           +*  (+)    +   (+)
 8           +*  (/)             -  (+)           +*  (+)    +   (+)
13               (/)                (-)               (-)        (-)
14               (/)                (+)               (-)        (-)
15               (/)                (+)               (+)        (+)
16               (/)                (+)               (+)        (-)

Note: Teacher change scores are highlighted in grey. The + represents a
change toward the desired goal; the - represents no change in the
desired direction; The asterisk * represents teacher achievement of the
predetermined goal. The (+) represents a positive increase in the
students' academic scores; the (-) represents a decrease in the
students' academic score; The (S) indicates that the student's academic
score remained the same as baseline.

Table 3 Summary of Student Change in Pre/Post Literacy and Office
Referrals for School Year

          In-service plus Peer Coaching Group
Student Participant  Literacy  Office Referrals

 1                   S         S
 2                   S         -
 3                   -         S
 4                   S         +
 9                   S         S
10                   +         S
11                   +         -
12                   S         S

             In-service Only Group
Student Participant  Literacy  Office Referrals

 5                   +         +
 6                   +         S
 7                   +         +
 8                   +         S
13                   S         +
14                   S         S
15                   S         -
16                   +         S

Note: The + represents an increase in literacy or office referrals;
the - represents a decrease in literacy or office referrals; The S
indicates no change in the student's literacy or office referrals.

Table 4 Summary of Change in Student's Problem Behaviors Across Four
Phases

                                 Strategies
Student      Instructional Talk  Prompts     Feedback   Wait time
Participant  (Baseline)          (Phase 2)   (Phase 3)  (Phase 4)

1            /                   +           +          +
2            /                   +           S          +
3            /                   +           +          +
4            /                   +           +          +
5            /                   -           +          +
6            /                   -           +          +
7            /                   -           +          -
8            /                   +           +          +

Note: The / represents that the baseline score was taken at that point
in time; the - represents a decrease in the student's problem behavior;
the + represents an increase in the student's problem behavior; the 'S'
means that the child's percentage of problem behavior remained the same
as baseline. Student's 1-4 were taught by teachers involved in peer
coaching and student's 5-8 were taught by teachers receiving the Teacher
Cool Tool only.

Table 5 SSRS: Differences between Pre & Post-Test Scores

Changes in Percentile Rank Within Each Domain

Student                                       Academic
Participant  Social Skills  Problem Behavior  Competence

 1            -4            -13                +2
 2            +9            -15               -20
 3           +24            +22               -20
 4            -4             -1               -21
 5           -14            +14                +1
 6           +23             +2                +8
 7           +38            -18               +10
 8           +10            +26                +1
 9            +2            -23                +6
10            +6            S                 +13
11           +14             -2               S
12            -8            -14                -5
13           -16             +9               -20
14           -20             -6                +9
15           S              +31                +3
16            +8            -16               +20

Note: The table represent differences between pre and post-test scores
in each area assessed. For social skills and academic competence,
numbers above zero, representing an increase in the domain, are
preferred; for problem behaviors, numbers below zero, representing a
decrease in the domain, are preferred.