Preventing problem behaviors: primary, secondary, and tertiary level prevention interventions for young children.

Abstract

The purpose of this report is to compare changes in social skills, problem behaviors, and academic competence for kindergarten or first grade students identified as being at risk for serious behavior problems who received primary, secondary, or tertiary level preventive interventions. Of the 93 participants in this study, 73% were male; 86% were Caucasian, and 65% were characterized as having externalizing behavior problems. A repeated measures analysis of variance indicated statistically significant differences (p < .01) between the groups based on type of intervention received the Self-Control subscale (e.g., controlling temper, responding appropriately to teasing) of the Social Skills Rating System (Gresham & Elliott, 1990). School-wide Positive Behavior Support is an effective primary prevention intervention, even for young children with serious internalizing or externalizing behavior problems.

Key words: Social skills, positive behavior support, self-control, externalizing problem behaviors, prevention.

**********

Behavior problems interfere with success in school for many children and create unpleasant situations for teachers and other children. Although traditionally schools often responded reactively to problem behaviors, with punishments and exclusion, today many educators are seeking to prevent problem behaviors by using proactive, early interventions.

A three-tiered model of prevention, popular in many fields, involves primary, secondary, and tertiary levels of prevention. When problem behaviors at school are the concern, Positive Behavior Support (PBS) is an approach that offers these three levels of prevention (see http://pbis.org). Schoolwide discipline, at the primary prevention level, emphasizes teaching, prompting, and reinforcing appropriate behavior proactively and universally, to all children in the school (Sugai & Horner, 2002). When well implemented, it can be expected to enable most students to behave well in school.

Some students may be given additional, support, at the secondary prevention level, if they are considered "at risk" for problem behaviors. Decisions about who is "at risk" may be based on a variety of factors and assessed in different ways. One way of deciding that a child needs some extra behavioral support is simply to notice that the child seems to have some difficulties with peers or in following instructions. In the early elementary grades, teachers often notice behaviors that, while in themselves not particularly severe, might lead to greater problems later, and the teachers may decide to implement some type of early intervention. A more formal way of making this decision is to use a rating scale or other instrument that indicates which children have known risk factors.

A few children come to school, from the beginning, with serious behavior problems that are already well established. These children are in need of tertiary level prevention interventions, which are preventive in the sense of preventing the problem from getting worse.

What evidence supports the use of three levels of preventive interventions in Positive Behavior Support in schools? Although the concept is logical, data are needed to show specific effects. In addition, other questions beg for answers:

* What is the nature of secondary and tertiary level prevention of problem behavior?

* How does secondary and tertiary level prevention relate to other factors in the educational environmental context?

* How should the assessment of the effects of behavioral interventions be similar to, and different from, assessment of academic interventions?

Information is needed to clarify the relative effects of primary, secondary, and tertiary prevention level behavioral interventions and to identify unique aspects of applying this approach to behavior problems. A 3-tiered model that works well for changes in academic skills, such as learning to read, may not generalize to situations where resistance to behavioral interventions can only be understood in the relationship to the environmental context. For example, increasing the dosage, or amount, of secondary or tertiary interventions might be related to improved academic outcomes, but not necessarily to behavioral outcomes. One can see how additional instruction in reading would likely lead to improved reading skill. However, additional behavior support typically is related to resistance to intervention (RTI) shown by individuals with chronic and serious behavior challenges. Behavior problems, particularly school discipline problems, unlike reading problems, tend to increase in frequency as children get older, at least through elementary and middle school.

Across the United States, a number of Centers are developing and testing K-3 Behavior and/or Reading Intervention Models, funded by the Office of Special Education Programs (OSEP). Special studies of "Tracking Samples" (i.e., students identified as being at high risk of internalizing or externalizing behavior problems) have been added to larger research projects involving three-tiered, school-wide approaches to PBS that have primary, secondary, and tertiary levels of interventions to prevent behavior problems (CCE, 2003). In Oregon, the larger research that provides the foundation for the Behavior Tracking Study (BTS) is examination of the Positive Behavior Support (PBS) approach (e.g., Horner, Sugai, Todd, & Lewis-Palmer, 1999-2000; Sugai & Horner, 2002; Sugai, Horner, & Gresham, 2002) in what has come to be known as the 90 Schools Study, or, more formally, as the Research and Demonstration Center on School-Wide Behavior Support. Although the larger study involves 90 schools and a randomized design with a delayed start control group, BTS is focused on seven schools know to have well established school-wide, primary interventions in place. In line with the PBS approach recommended by the Oregon Behavior Center, these schools that have demonstrated effective implementation of school-wide positive behavior support and are in an excellent position to turn their attention to the development and implementation of systematic secondary and tertiary interventions for students who need more support than school-wide systems can provide. In addition, these seven schools are in districts that have worked with the University of Oregon for several years to provide staff development, consultation, and inservice training related to the types of secondary and tertiary behavioral interventions used in the PBS approach (e.g., Crone & Horner, 2003; Crone, Horner, & Hawken, 2003; Tobin, Lewis-Palmer, & Sugai, 2002; Todd, Horner, Sugai, & Colvin, 1999).

The purpose of this preliminary analysis at the local level is to compare changes in social skills, problem behaviors, and academic competence for students who had been identified as having serious problem behaviors and who received different types of preventive interventions.

Method

In this section, we first describe the behavioral measures, because they are key to understanding how and why the participants were selected and the procedures that were used.

Key Behavioral Measures

Systematic Screening for Behavior Disorders (SSBD). The SSBD (Walker & Severson, 1990) identifies children who are at risk for behavioral disorders. A special version of the SSBD, the Early Screening Project (ESP) is used for kindergarten students (Walker, Severson, & Feil, 1995; Feil, Severson, & Walker, 2002). Two main categories of behavior disorders are addressed: internalizing and externalizing. The internalizing concept is based on earlier work with preschool screening of children at risk for social withdrawal (Greenwood, Walker, Todd, & Hops, 1979). The externalizing concept is based on earlier work in the identification of adolescents at risk delinquency (Loeber, Dishion, & Patterson, 1984).

For the SSBD, teachers are given a description of typical characteristics of these two behavioral categories, asked to look at the list of children in their classroom, to consider each child's characteristics, and then to list the ten children who best exemplify a description of externalizing characteristics and the ten who best exemplify a description of internalizing characteristics. The two lists are mutually exclusive, so a child can only be put on one of the lists. This is a universal procedure because each student is considered and thus has an equal chance to be identified for further assessment. The students in the lists of 10 are then rank ordered according to how closely they match the profile of an internalizer or externalizer. In this project, for each classroom, the three students with the highest ranks for internalizers and the three students with the highest ranks for externalizers are said to pass through Gate 1. These six students per class, who are nominated by their teachers as being potentially at-risk for behavior problems, form the intended sample for Tracking Study, and enter Gate 2 assessment. At Gate 2 of the SSBD, each student is rated by his/her teacher on both the SSBD Adaptive Behavior Scale and the SSBD Maladaptive Behavior Scale. The teacher marks a Likert-like scale for occurrence and frequency of specific behaviors. Normative data from the development of the SSBD indicates that, typically, at least one student per classroom is at risk for severe externalizing type behavior problems (e.g., aggressive, disruptive, oppositional) and one student at risk for severe internalizing type behavior problems (e.g., fearful, depressed, anxious, neglected by peers) in every two or three classrooms. The original SSBD included a third "gate" involving direct observations but that was not used in the identification of students for the tracking study. Instead, students who scored in the 30% percentile for "at-risk" on either the Adaptive (i.e., were lacking in adaptive social behavior skills) or the Maladaptive (i.e., had excessive maladaptive social behaviors) scales of the SSBD (or ESP) become eligible for on-going participation in the Tracking Study, and formal informed consent was recruited from their family.

Social Skills Rating System (SSRS). The SSRS provides norms for boys and girls from ages 3 to 18, based on a large national sample and it is designed to assist educators in planning interventions (Gresham & Elliott, 1990). Three behavioral domains are covered: competing problem behaviors (e.g., externalizing, internalizing, and hyperactivity problems) that interfere with the performance of appropriate behaviors, academic competence (e.g., reading, mathematics, motivation, and general intelligence), and social skills. Three social skills domains are included: cooperation, assertion, and self-control. Examples of these domains are given in Gresham (2001): (a) "Cooperation" includes "follows classroom rules; complies with teacher instructions" (p. 344); (b) "Assertion" includes "introduces self, questions rules that may be unjust" (p. 344); and (c) "Self-control" includes "controls temper in conflict situations, responds appropriately to teasing" (p. 344). Items are rated by frequency. The SSRS has forms for teachers, parents, and students. In this project, only teacher ratings were used. In addition, we used a developer-approved short form that contains only 18 key items.

Student Intervention Record (SIR, CCE, 2003). This instrument collected information from the teachers regarding the interventions the students received for behavioral support. Of particular interest for this analysis was the prevention level of the support provided.

Participants and Setting

The study took place in a medium sized city in the Northwest, during the 2002-2003 and 20032004 school years. Seven elementary schools, three from one school district and four from another school district, participated. All had previously documented fidelity with implementation of school-wide PBS on the SET . Figure 1 shows the SET scores for one of the schools; these types of scores were typical for all seven schools.

[FIGURE 1 OMITTED]

All children in kindergarten and in Grade 1 in these schools, a total of 1066 students, were screened using Gates 1 and 2 of the SSBD or ESP. Informed consent for participation was obtained by the schools for students who qualified, in accord with a protocol approved by the participating university's Institutional Review Board. Of the 93 students who received permission and were eligible to participate at the time the study started, 25 (27%) were female and 68 (73%) were male. The ethnic distribution was as follows: 86.02% White, 5.38% Latino/Hispanic, 3.23% Pacific Islander/Asian, 1.08% Black, and 4.30% Other. Most of the students, 60 (65%), qualified as Externalizers; 33 (35%) qualified as Internalizers. In order to qualify, students had to meet certain "cut off" scores on the adaptive and/or the maladaptive behaviors scales of the SSBD (or ESP). An indication of the seriousness of the students' behavior problems was that 76.34% of the sample met the cut off scores on both the adaptive and the maladaptive scale. About eleven percent met criteria only on the basis of maladaptive behaviors and about thirteen percent, only on the basis of a lack of adaptive behaviors. As of January, 2004, the percentage of the sample at each grade levels was: Kindergarten, 48.39%, first grade, 34.41%, and second grade, 17.20%. Over the course of time during the first two years of the project, due to school transfers, 7 students (all from the Externalizing group, 6 male) were lost to attrition, leaving a total of 86 at the end of the 2003-2004 school year.

Independent Variable: Prevention Level of Intervention

Using information from the SIR, four groups of students were identified, based on the level of intervention needed and/or received:

* Primary Only: The school-wide PBS universal intervention (Sugai & Horner, 2002) was the only behavior support needed (n = 52).

* Secondary: Added Behavior Education Plans (Crone, Horner, & Hawken, 2003), more commonly known as the "Check In, Check Out (CICO)" intervention (n = 8).

* Tertiary: Added individualized function-based support (n = 22).

* Time Lacking: Needed more than the primary prevention level intervention but time & resources were lacking so did not receive special intervention (n = 4)

Dependent Variables: Subscales of the Social Skills Rating Scale(SSRS)

The dependent variables were the raw scores on the seven SSRS subscales:

* Cooperation, Assertion, Self-Control (the Social Skills)

* Externalizing, Internalizing, Hyperactive Behaviors (the Problem Behaviors)

* Academic Competence

Interventions Described in More Detail

Primary Only. The following description of the primary prevention intervention is taken from School Climate and Discipline: Going to Scale (Sugai & Horner, 2001), a paper presented at the National Summit on the Shared Implementation of IDEA, in Washington, DC, June 23, 2001:

"Schools that adopt a school-wide PBS approach have the following features:

* An agenda of primary prevention has priority and is visible school-wide.

* All students and staff members are taught the school-wide expectations and received regular and frequent opportunities to practice them and to be positively acknowledged when they use them.

* A majority (>80%) of students, staff, and families can state the school-wide positive expectations and give a specific behavioral example for each.

* Positive school-wide behavioral expectations are defined, taught, and encouraged for all students using a range of positive and negative examples.

* Most contacts between teachers and students are prosocial (positive and preventive) rather than corrective and punishing (i.e., 5-8 positives for every negative interaction).

* A full continuum of PBS is available for all students at the school and district levels.

* Behaviorally competent personnel are readily available.

* A function-based approach serves as the foundation for addressing problem behaviors.

* All staff members actively participate in the implementation of school-wide PBS approach.

* Accurate and consistent implementation of PBS practices by all staff members is emphasized.

* The school administrator is an active participant and leader in the PBS effort.

* A school-wide leadership team guides the systemic adoption and sustained use of research-validated practices.

* School data are reviewed at least monthly to guide decision making and planning.

Schools that adopt a school-wide PBS approach consider four distinct and necessary implementation elements: (a) specification of clearly defined and measurable results, (b) use of data for decision making, (c) adoption of evidence-based practices and processes, and (d) provision of supports for high fidelity implementation." (Sugai & Horner, 2001, p. 3).

Secondary. Check-in/Check-out (CICO), also known as "Behavior Education Plan" (Crone, Horner, & Hawkins, 2004), is a type of secondary prevention or targeted behavioral support that is widely used in various forms with students whose behavior problems do not respond sufficiently to primary prevention in the form of universal, school-wide positive behavior support. Key features of the CICO intervention are (a) being readily available, (b) increasing monitoring and adult contact, (c) providing contingent and frequent feedback, and (d) increasing coordination between school and home support. Previous studies have indicated that many students benefit from CICO as shown by higher rates of academic engagement and fewer office discipline referrals (Hawken & Horner, 2003; March & Horner, 2002). CICO is relatively cost-effective and efficient, in comparison with comprehensive, individualized, tertiary prevention level interventions, for several reasons. First, CICO can be implemented quickly with minimal training for staff. Second, it does not require extensive assessments or consultations prior to use. Third, although the amount of time and resources may vary, generally it requires only a few minutes per day of staff time and only materials that most schools have on hand. After a brief, initial meeting of a behavior support team or a teachers' assistance team, to clarify the nature of the behavioral issues for this student, students participating in CICO typically follow a routine, described by Crone, Horner, and Hawken (2004) as follows:

* Each morning the student will check-in with a designated school staff person (e.g., teacher, secretary, counselor, educational assistant). That check-in will determine if the student has materials needed for class and if the student is physically prepared to attend classes. The student is given a form (e.g., point sheet or card) to use throughout the day that lists the student's behavioral goals and a matrix showing classes or time periods. This check-in usually takes less than 5 minutes and includes verbal prompts and encouragement.

* A key feature of CICO is that the teachers continually monitor the behavior of the student throughout the day. Each class period (or at other designated times), the student brings the form to the teacher, who marks a rating of how well the student met his or her behavioral goals. In some cases, the student also self-monitors.

* At the end of the school day, the student takes the form back to the staff person who conducted the morning check-in, for the afternoon check-out, which consists of a quick review of the form, verbal feedback, and, in some cases, small reinforcers if certain goals have been met (e.g., 80% of possible points on the teachers' ratings). The afternoon check-out typically takes less than 5 minutes.

* Students take the form (sometimes called a daily behavior report card) home to show their parents, who will sign it. It will be returned to school the next day.

* The school staff member who is monitoring the child maintains a record of progress, which can be charted and used to make decisions about maintaining, fading, or strengthening the intervention over time.

Tertiary. A tertiary prevention intervention would begin with a functional behavioral assessment (FBA) to identify the function or need underlying a student's challenging behavior and enable teachers and other school personnel to develop individualized positive behavior support (Crone & Horner, 2003; Sugai, 1998; Sugai et al., 2000). FBAs vary according to individual needs and local school circumstances. The process of FBA may include a student interview (Ervin et al., 98; Kern, Dunlap. Clarke, & Childs, 1994; Jolivette, Lassman, & Wehby; 1998; Kearney & Tillotson, 1998; Reed, Thomas, Sprague, & Horner, 1997). Descriptive information often is obtained from other sources, such as: teacher or parent interviews; rating scales, direct observations in natural settings, and a review of school records and incident reports (Artesani & Mallar, 1998; Fox, Gunter, Davis, & Brall, 2000; McConnell, Hilvitz, & Cox, 1998; Todd, Horner, & Sugai, 1999). Descriptive assessments, which depend on naturally occurring variations in environmental factors, are valuable for "planning treatments that will succeed in the natural environment" (Schill, Kratochwill, & Gardner, 1996, p. 93).

Using FBA, adults can understand a child's perspective and, with that in mind, develop interventions that will effectively decrease occurrences of inappropriate behavior while at the same time, provide the child with respect and consideration. Function-based support takes into account what Montague and Warger (1997) called "the desired outcome from the student's perspective (e.g., attention from peers, teacher attention, avoidance of a math task because the student lacks the skills" (p. 7). Many students who behave inappropriately have found that it provides quick access to powerful reinforcers, such as attention (even if negative) or opportunities to avoid difficult tasks. Teaching an alternative behavior that is functionally equivalent to the inappropriate behavior, in that it leads to the same consequence, eliminates problem behavior when the replacement behavior is a more efficient and/or more effective way to meet the need associated with the problem behavior (Carr, 1988; Carr, Reeve, & Magito-McLaughlin, 1996; Horner & Billingsley, 1988; Horner & Day, 1991; Neef, Bicard, & Endo, 2001; Neef & Lutz, 2001a, 2001b). In some cases, teachers include elements of CICO within a multi-component intervention based on an FBA (e.g., Condon & Tobin, 2001). The teachers in the schools involved in this study had access to assistance from a district behavior specialist and support from school and district behavior teams in conducting FBAs and developing interventions based on them. In addition, many of the teachers had previously received enough training and support that they were able to develop and implement basic function-based support on their own.

How does a student become eligible for a secondary or tertiary level behavioral intervention?

* Teachers decide to implement these interventions, either on their own or with the help and advice of a behavior support team or specialist.

* There is not a "formula" or "score" for making these decisions about interventions for individual students.

Results

The primary prevention level intervention, schoolwide PBS, met the behavioral support needs of most of the students in the tracking study. The majority (60%) of the tracking study participants did not need secondary or tertiary level interventions, according to their teachers. Some teachers commented that, although the child had a behavior problem, their typical classroom management strategies were sufficient. Thirty-five percent were given some type of additional behavioral support, either secondary or tertiary level prevention. Five percent were considered to be in need of some type of additional support but time and/or resources were lacking so no special intervention was developed. Of the students who received additional support, 27% were at the secondary prevention level and 63% at the tertiary. The targeted intervention used for secondary prevention was CICO. The intensive intervention used for tertiary prevention was function-based support. Of those at the tertiary prevention level, 26% received CICO as one part of a multi-component intervention based on an FBA.

What were the behavior problems? Teachers selected from a list of types provided on the SIR; more than one behavior could be selected. Self-control was always listed, either alone or with other behavior problems. Cooperation was listed 16 times, always in combination with some other behaviors. Attention/Hyperactivity was mentioned 9 times. Externalizing difficulties were listed 14 times but Internalizing difficulties, only twice. Other was used 4 times (e.g., group participation, talk outs, following directions, and transitions).

Some of the students (n = 47), at the end of Year 2, had been rated on the SSRS twice. The SSRS was administered to students in the first cohort in the Spring of Year 1 and again in the Spring of Year 2. Of the students in the second cohort, some attended schools where the SSRS was given in the Fall and in the Spring of Year 2; others attended schools where it was given only in the Spring of Year 2. To study changes over time, the students who had been rated twice on the SSRS were grouped by their SIR category. The number of students with Time 1 and Time 2 SSRS ratings in each group was as follows: 26 in the Primary Only group, 6 in the Secondary group, 13 in the Tertiary group, and 2 in the Time Lacking group.

Changes in Self-Control Social Skills

A repeated measures analysis of variance indicated statistically significant differences (p < .01) between the groups on raw scores for the Self-Control subscale of the SSRS (Table 1).

Tukey's Studentized Range Test was used to control the Type I experimentwise error rate, while following up the ANOVA to identify mean differences. For the Self-Control subscale, statistically significant differences (p < .05) between the means were found for Tertiary and Primary at Time 2 (but not at Time 1). Figure 2 illustrates the changes for the different groups on the Self-Control measure.

[FIGURE 2 OMITTED]

Other SSRS subscales, while not showing statistically significant change (p < .05) on mean scores, given the small n with two SSRS scores at this time, show interesting trends.

Cooperation Social Skills. The group that received the secondary prevention level intervention stood out as making the greatest change in on the Cooperation subscale, in comparison with the other groups, as shown in Figure 3.

[FIGURE 3 OMITTED]

Assertion Social Skills. The Primary and the Tertiary groups improved more than the other groups on the Assertion subscale, as shown in Figure 4.

[FIGURE 4 OMITTED]

Internalizing Problem Behaviors. The Primary, Secondary, and Tertiary groups all decreased internalizing problem behaviors. The two students in the "Time Lacking" group, however, increased internalizing problem behaviors. In addition, this group had the highest scores on the SSRS internalizing subscale at Time 1 (Figure 5).

[FIGURE 5 OMITTED]

Externalizing Problem Behaviors. The Primary, Secondary, and Tertiary groups all slightly decreased externalizing problem behaviors. The two students in the "Time Lacking: group, who initially had low scores on the externalizing SSRS subscale, were rated as having increased externalizing problem behaviors, surpassing the Primary group and almost reaching the level of the Secondary group (Figure 6).

[FIGURE 6 OMITTED]

Hyperactive Problem Behaviors. The Secondary and Tertiary groups decreased their average scores on the SSRS subscale for hyperactive problem behaviors (Figure 7).

[FIGURE 7 OMITTED]

Academic Competence. All of the groups improved, slightly, on the Academic Competence subscale of the SSRS. The Tertiary group, although the lowest at both Time 1 and Time 2, still improved the most and at Time 2, almost caught up with the Primary group (Figure 8).

[FIGURE 8 OMITTED]

Out of School Suspensions and Discipline Problems

Although young children in kindergarten and Grades 1 and 2 typically are not often sent to the principal's office for disciplinary reasons or suspended out of school, some of the students in the tracking study had these types of problems. Table 2 shows the number of student in each SIR group who were suspended and Figure 9 shows the range and distribution of office discipline referrals (ODRs) for the tracking sample students, by grade level.

[FIGURE 9 OMITTED]

Discussion

The results indicate that (a) school-wide PBS is a beneficial intervention, particularly for increasing the social skill of self-control, even for children identified on the SSBD as having serious behavior problems and (b) students identified by their teachers as needing Tertiary interventions are resistant to interventions the amount of support they needed must be adjusted accordingly.

Description of a New Tool

A new tool is being used to study systems level implementation of positive behavioral support in schools for students who need more than the universal schoolwide primary intervention. The Individual Student Systems Evaluation Tool (I-SSET) (Lewis-Palmer, Todd, Horner, & Sugai, 2003) measures the extent to which a school has the capacity to provide secondary (e.g., Behavior Education Plans such as those described in Crone et al., 2003) and tertiary (e.g., function-based support, see Crone & Horner, 2003) preventive interventions related to positive behavior support. It includes an administrator interview and a behavior specialist interview. Also, about five teachers or other staff members are briefly interviewed. In addition, there is a review of written documents related to individualized support.

Big Ideas

* Most of these young students, even though identified as having serious behavior problems (by inclusion criteria), did not need secondary or tertiary level prevention interventions.

* Of those who did need secondary or tertiary level interventions, most received support.

* Receiving support at a higher level and for a longer period of time does not necessarily translate immediately into improved behavior because intensive support is given to the students with the most serious and most resistant, chronic behavior problems.

* School-wide Positive Behavior Support is an effective primary prevention intervention, even for young children with serious internalizing or externalizing behavior problems.

Author Note

George Sugai was at the University of Oregon when this research was conducted and is now at the University of Connecticut. Dr. Sugai continues in his role as co-director of the Center on Positive Behavior Interventions and Supports (http://pbis.org). Preparation of this manuscript was funded, in part, by the Office of Special Education Programs, Award #H324X010015. Portions of the report were presented by Tary Tobin at the Second International Conference on Positive Behavior Support, in Tampa, Florida (March, 2005) in "Research at the tip of the triangle from K to 12."

References

Artesani, A. J., & Mallar, L. (1998). Positive behavior support in general education settings: Combining person-centered planning and functional analysis. Intervention in School and Clinic, 34(1), 33-38.

Carr, E. G. (1988). Functional equivalence as a mechanism of response generalization. In R. Horner, R. L. Koegel, & G. Dunlap (Eds.), Generalization and maintenance: Lifestyle changes in applied settings (pp. 194-219). Baltimore: Paul H. Brooks.

Carr, E. G., Reeve, C. E., & Magito-McLaughlin, D. (1996). Contextual influences on problem behavior in people with developmental disabilities. In L. K. Koegel, R. L. Koegel, & G. Dunlap (Eds.), Positive behavioral support: Including people with difficult behavior in the community (pp. 403-424). Baltimore: Paul H. Brookes.

Coordination, Consultation, and Evaluation (CCE) Center for K-3 Reading and Behavior Intervention Models. (2003). Student Intervention Record. Madison: University of Wisconsin, School of Education, Wisconsin Center for Education Research.

Condon, K. A., & Tobin, T. J. (2001). Using electronic and other new ways to help students improve their behavior. Teaching Exceptional Children, 34(1), 44-51.

Crone, D. A., & Horner, R. H. (2003). Building positive behavior support systems in schools: Functional behavioral assessment. New York: Guilford.

Crone, D. A., Horner, R. H., & Hawken, L. S. (2004). Responding to problem behavior in schools: The Behavior Education Program. New York: Guilford.

Ervin, R. A., Radford, P. M., Bertsch, K., Piper, A. L., Ehrhardt, K. E., & Poling, A. (2001). A descriptive analysis and critique of the empirical literature on school-based functional assessment. School Psychology Review, 30(2), 193-210.

Feil, E. G., Severson, H. H., & Walker, H. M. (2002). Early screening and intervention to prevent the development of aggressive, destructive behavior patterns among at-risk children. In M. R. Shinn, G. Stoner, & H. M. Walker (Eds.), Interventions for academic and behavior problems: Preventive and remedial approaches (pp. 143-166). Silver Spring, MD: National Association of School Psychologists.

Fox, J. J., Gunter, P., Davis, C. A., Brall, S. (2000). Observational methods in functional behavioral assessment: Practical techniques for practitioners. Preventing School Failure, 44, 152-157.

Greenwood, C., Walker, H. M., Todd, N., & Hops, H. (1979). Selecting a cost-effective device for the assessment of social withdrawal. Journal of Applied Behavior Analysis, 12, 639-652.

Gresham, F. M. (2001). Assessment of social skills in children and adolescents. In J. J. W. Andrews, D. H.

Saklofske, & H. L. Janzen, (Eds.), Handbook of psychoeducational assessment: Ability, achievement, and behavior in children (pp. 325-355). San Francisco: Academic Press.

Gresham, F. M., & Elliott, S. N. (1990). Social skills rating system. Circle Pines, MN: American Guidance Service.

Hawken, L. S., & Horner, R. H. (2003). Evaluation of a targeted intervention within a schoolwide system of behavior support. Journal of Behavioral Education, 12, 225-240.

Horner, R. H., & Billingsley, F. F. (1988). The effect of competing behavior in the generalization and maintenance of adaptive behavior in applied settings. In R. H. Horner, G. Dunlap, & R. L. Koegel (Eds.), Generalization and maintenance: Life-style changes in applied settings (pp. 197-220). Baltimore, MD: Paul H. Brookes.

Horner, R. H., & Day, H. M. (1991). The effects of response efficiency on functionally equivalent competing behaviors. Journal of Applied Behavior Analysis, 24(4), 719-732.

Horner, R. H., Todd, A., & Lewis-Palmer, T., Irvin, L. K., Sugai, G., & Boland, J. B. (2004). The School-Wide Evaluation Tool (SET): A research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6(1), 3-12.

Horner, R. H., Sugai, G., Todd, A., & Lewis-Palmer, T. (1999-2000). Elements of behavior support plans: A technical brief. Exceptionality, 8(3), 205-215.

Jolivette, K., Lassman, K. A., & Wehby, J. H. (1998). Functional assessment for academic instruction for a student with emotional and behavioral disorders: A case study. Preventing School Failure, 43(1), 19-23.

Kearney, C. A., & Tillotson, C. A. (1998). School attendance. In T. S. Watson & F. M. Gresham (Eds.), Handbook of child behavior therapy (pp. 143-161). New York: Plenum Press.

Kern, L., Dunlap, G., Clarke, S., & Childs, K. (1994). Student-Assisted functional assessment interview. Diagnostique, 19(2-3), 29-39.

Lewis-Palmer, T., Todd, A., Horner, R. H., & Sugai, G. (2003). Individual Student Systems Evaluation Tool (I-SSET). Eugene: University of Oregon, Educational and Community Supports.

Loeber, R., Dishion, T. J., & Patterson, G. R. (1984). Multiple gating: a multi-stage assessment procedure for identifying youths at risk for delinquency. Journal of Research in Crime and Delinquency, 21(1), 7-32.

March, R. E., & Horner, R. H. (2002). Feasibility and contributions of functional behavioral assessment in schools. Journal of Emotional and Behavioral Disorders, 10, 158-170.

McConnell, M. E., Hilvitz, P. B., & Cox, C. J. (1998). Functional assessment: A systematic process for assessment and intervention in general and special education classrooms. Intervention in School and Clinic, 34(1), 10-20.

Montague, M., & Warger, C. (1997). Helping students with attention deficit hyperactivity disorder succeed in the classroom. Focus on Exceptional Children, 30(4), 1-16.

Neef, N A., Bicard, D. F., & Endo, S. (2001). Assessment of impulsivity and the development of self-control in students with attention deficit hyperactivity disorder. Journal of Applied Behavior Analysis, 34, 397-408.

Neef, N. A., & Lutz, M. N. (2001a). Assessment of variables affecting choice and application to classroom interventions. School Psychology Quarterly, 16, 239-252.

Neef, N. A., & Lutz, M. N. (2001b). A brief computer-based assessment of reinforcer dimensions affecting choice. Journal of Applied Behavior Analysis, 34, 57-60.

Reed, H., Thomas, E., Sprague, J. R., & Horner, R. H. (1997). The student guided functional assessment interview: An analysis of student and teacher agreement. The student guided functional assessment interview: An analysis of student and teacher agreement. Journal of Behavioral Education, 7, 33-49.

Schill, M. T., Kratochwill, & Gardner, W. I. (1996). Conducting a functional analysis of behavior. In M. J. Breen & C. R. Fiedler (Eds.), Behavioral approach to assessment of youth with emotional/behavioral disorders (pp. 83-180). Austin, TX: Pro-Ed.

Sugai, G. (1998). The development of individualized behavior support plans. In M. M. Kerr & C. M. Nelson, Strategies for managing behavior problems in the classroom (3rd ed., pp. 139-145). Upper Saddle River, NJ: Prentice-Hall.

Sugai, G., Horner, R. H., Dunlap, G., Hieneman, M., Nelson, C. M., Scott, T., Liaupsin, C., Sailor, W., Turnbull, A. P., Turnbull, H. R., Wickham, D., Wilcox, B., & Ruef, M. (2000). Applying positive behavior support and functional behavioral assessment in schools. Journal of Positive Behavior Interventions, 2(3), 131-143. [Can be downloaded from http://pbis.org]

Sugai, G., & Horner, R. (2001, June 23). School climate and discipline: Going to scale. Paper presented at the National Summit on the Shared Implementation of IDEA, Washington, DC.

Sugai, G. & Horner, R. (2002a). Evaluating the Impact of School-wide Positive Behavior Support. Grant proposal submitted to the Office of Special Education Programs. Eugene: University of Oregon, Special Education Area, Educational and Community Supports.

Sugai, G., & Horner, R. (2002b). The evolution of discipline practices: School-wide positive behavior supports. Child and Family Behavior Therapy, 24, 23-50.

Tobin, T. J., Lewis-Palmer, T., & Sugai, G. (2002). School-wide and individualized effective behavior support: An explanation and an example. Behavior Analysis Today, 3(1), 51-75. Retrieved May 1, 2002, from http://www.behavior-analyst-online.org/BAT/BAT31.pdf

Todd, A. W., Horner, R. H., Sugai, G. (1999). Self-Monitoring and self-recruited praise: Effects on problem behavior, academic engagement, and work completion in a typical classroom. Journal of Positive Behavior Interventions, 1(2), 66-76.

Todd, A. W., Horner, R. H., Sugai, G., & Colvin, G. (1999). Individualizing school-wide discipline for students with chronic problem behaviors: A team approach. Effective School Practices, 17(4), 72-82.

Walker, H. M., & Severson, H. H. (1990). Systematic Screening for Behavior Disorders: User's guide and administration manual. Longmont, CO: Sopris West.

Walker, H. M., Severson, H. H., & Feil, E. G. (1995). Early Screening Project: A proven child-find process. Longmont, CO: Sopris West.

Address correspondence to:

Tary J. Tobin, Ph.D.

Educational and Community Supports

1235 University of Oregon

Eugene, OR 97403-1235

E-mail: [e-mail]

Phone: [phone]

FAX: [phone]

Table 1. Repeated Measures Analysis of Variance for Self-Control

                                       Partial Eta
Source         df          F             Squared      P

                    Between Subjects

 Prevention
Intervention
   Level        3        5.16 *             .27      .004
   Error       43       (32.79)

                    Within Subjects

    Time        1        0.05             .001       .826
Time * Level    3        2.20             .133       .102
Error (Time)   43        (9.08)

Note. Values enclosed in parenthesis represent mean square
errors. * p < .01

Table 2.  Number of Students Receiving Out of School Suspensions by
SIR Category

               Suspended   Not Suspended

Primary Only       0            52
Secondary          0             8
Tertiary           5            17
Time Lacking       0             4
Total              5            81