A preliminary study on the effects of training using Behavior Support Plan Quality Evaluation Guide (BSP-QE) to improve positive behavioral support plans.

Abstract

The purpose of this research was to evaluate the effects of two trainings designed to increase the competencies of professionals to develop high quality positive behavior support plans for students that engage in problem behaviors that interfere with theirs and/or others' ability to learn. Training one consisted of training attendees on six key concepts of behavior analysis, and team functioning, that are supported by the research as best practice for effective behavior change. Training two concentrated on training attendees how to evaluate and rate the quality of PBS plans using an evidence-based rating instrument. Results of the professional trainings revealed that participants were nearly four times more likely to develop PBS plans that were rated as good or superior after receiving training on how to evaluate and rate the quality of PBS plans than receiving training on the six key concepts alone. The implications for professional pre- and in-service training to enhance the skills of educators in developing PBS plans based on functional behavioral assessments are discussed.

**********

The Individuals with Disabilities Education Improvement Act (IDEIA, 2004) was signed into law on December 3, 2004, renewing several key commitments to special education students who engage in persistent, problematic behavior. Two of the most significant commitments embedded within the language of IDEIA 2004 that are most relevant to disciplinary practice in the schools relate to conducting a functional behavior assessment (FBA) and developing a positive behavior support plan (PBS plan). The specific language in IDEIA 2004 states that the IEP Team shall develop a PBS plan: (a) In the case of a child whose behavior impedes the child's learning or that of others; (b) when a disciplinary action is taken that results in an involuntary placement change, and the behavior is a manifestation of the disability; and (c) in the situation where a behavioral intervention plan has been developed, review the behavioral intervention plan if the child already has such a behavioral intervention plan, and modify it, as necessary, to address the behavior.

Despite the continuation of these requirements from the Individuals with Disabilities Education Act of 1997 (IDEA 97), recent data indicate that schools in general have made little to marginal progress in adequately meeting these mandates and fulfilling the intent of the law (Cook et al., in press; Smith, 2000; Van Acker, Boreson, Gable, & Potterton, 2005). The current inadequacy is not surprising considering that several researchers anticipated this inadequacy at the outset of the policy debate in Congress during the nascent stages of IDEA 97 (Conroy, Clark, Gable, & Fox, 1999; Drasgow & Yell, 2001; Gresham, Quinn, & Restori 1999; Smith, 2000). Researchers initially feared that school personnel lacked the necessary knowledge and skills to conduct adequate FBAs, and to develop legally defensible and educationally appropriate PBS plans. Sadly, research over the past few years (Conroy, Katsiyannis, Clark, Gable, & Fox, 2002; Horner, Sugai, Todd, Lewis-Palmer, 1999-2000; Scott, Nelson, & Zabala, 2003) has validated these fears by revealing a state of affairs in American education where (a) inadequate FBAs were conducted (Gable, 1999), (b) there was little or no correspondence between FBA data and the content of PBS plans (Van Acker et al., 2005), and (c) the majority of PBS plans from typical school teams were rated as legally indefensible and substantively inadequate (Cook et al., in press; Yell, 2002). This latter issue, improving the substantive adequacy of PBS plans, represents the primary focus of this paper.

In an attempt to begin to develop an understanding of the quality of PBS plans developed by educators in today's schools, Cook et al. (in press) performed a study that focused on comparing the substantive adequacy of actual PBS plans developed by typical school teams without demonstrated knowledge and experience to teams including a member with advanced knowledge and skills in behavioral theory and practice. The quality of PBS plans were rated with the Behavior Support Plan Quality Evaluation Guide (Browning-Wright, Saren, & Mayer, 2003), which includes 12 rating items assessing various substantive aspects of plans (e.g., quality of behavior definition, specification of antecedents, identification of behavior function). Results from the investigation showed that only 11% (8 out of 76) of the plans developed by typical school teams were rated as "Adequate" compared to 65% of the plans from teams with a more experienced and skilled member. Based on the disparate results, the authors concluded that by delivering training with the intention of improving the behavioral competencies of school personnel, the substantive quality and educational relevance of PBS plans could be improved.

The findings from Cook et al. and similarly those from Van Acker et al. (2005) indicate that the majority of PBS plans developed in today's schools are inadequate and potentially legally invalid due to substantive and procedural violations. This is particularly disturbing in light of the fact that courts have already, and continue to, render decisions based upon procedural violations (e.g., missing mandatory personnel, not developed within the team context, etc.). More recently, however, courts have started to rule on the substantive aspects of PBS plans as well, making it even more important to develop substantively adequate PBS plans (see Etscheidt, 2006). Two recent cases illustrate this trend.

In the first case, a hearing officer in Iowa upheld the premise that positive behavioral supports and strategies were a necessary element of a legally tenable PBS plan (People v. Mason City Community School District, 2000). In his final decision, the hearing officer ordered that the school district develop a new PBS plan consistent with the mandates of IDEA 1997, because the former plan was deemed insufficient due to its emphasis on punitive strategies and general failure to specify how the student was going to be taught appropriate replacement behaviors. The second case, which was the closest case to date to ruling on the substantive aspects of PBS plans, was decided in Pennsylvania and was appealed to the 7th Circuit Court of Appeals (Alex R. v. Forrestville Valley Community Unit School District, 2004). In this case, the mother of a special education student contested her son's PBS plan on the grounds that it was "substantively insufficient." The judge issued the following ruling:

  Alex, nevertheless, urges us to follow the lead of the administrative
  judge in Mason City, who manufactured the substantive criteria of a
  sufficient behavioral intervention plan ... We decline the invitation.
  Although we may interpret a statue and its implementation regulations,
  we may not create out of cloth substantive provisions for the
  behavioral intervention plan contemplated by [section] 1415(k)(1) or
  [section] 1414(d)(3). In short, the District's behavioral intervention
  plan could not have fallen short of substantive criteria that do not
  exist. (p. 19)

Despite this judge's reluctance to levy a decision on the substance of the behavior plan, hearing officers and judges of other cases involving PBS plans have been far less reluctant in that they have made decisions regarding the substantive aspects of plans (Etscheidt, 2006; Maag & Katsiyannis, 2006). Moreover, research has for some time now elucidated substantive components of a legally defensible and educationally appropriate PBS plan. The authors hope to demonstrate that these components do indeed exist in the research base and can be effectively taught to educators in the field to produce better quality plans for students. The key concepts disseminated in the trainings that are described in this paper were drawn from basic texts and empirical research within the field of applied behavior analysis and team performance (Cooper, Heron, & Heward, 1987; Crone & Horner, 2003; O'Neill et al., 1997; Sulzer-Azaroff & Mayer, 1991). What remains to be answered, though, is whether trainings emphasizing such concepts lead to improved PBS plan development.

Several researchers have suggested that intensive in-service training is the key to enhancing the behavioral competencies of school personnel to meet the discipline mandates of IDEA 1997 and IDEIA 2004 (Conroy et al., 2002; Gresham, 2003; Scott & Nelson, 1999; Van Acker et al., 2005). However, the findings from Scott et al. (2005) on the effects of a "crash course" training on FBA and behavior intervention planning to improve school-based teams ability to use FBA data to develop effective intervention strategies revealed that such a training was not sufficient to move teams' performance into the range of best practice. Specifically, after training, the school-based teams were reported to rely overly on reactive and negative interventions and ignore largely the identified function of behavior. These findings fly in the face of other research that has suggested that educators can be trained successfully to conduct FBAs and develop interventions that match behavior function (Iwata et al., 2000; Moore et al., 2002).

Recently, as part of a statewide research initiative in California, researchers from the Positive Environment Network of Trainers (PENT) attempted to shed light on the issue by empirically evaluating the effects of two types of intensive inservice training. In the first case, training consisted of a review of key concepts in behavior analysis, including case studies and plan reviews that illustrated positive behavior support concepts. In the latter, provided by the same trainer, those who had received the initial training were then provided a specific training agenda using the Behavior Support Plan Quality Evaluation Guide on the substantive aspects of PBS plans. This paper describes an empirical investigation of these trainings to enhance the quality of PBS plan development for students in typical educational settings.

The researchers recognize that many behavior plans will be written by educators who have not had extensive coursework in behavior analysis. Many educators did not, in fact, have any coursework in behavior analysis prior to employment, and yet are assigned job duties in which skills in this area are required. Additionally, educational budgetary concerns and logistics often dictate trainings in school districts that are of limited time. Nevertheless, designing a six-hour training for existing educators in school districts that will result in behavior plans being written with greater skill and internal consistency is of practical importance and value to today's educators.

Methods

The California PENT was established in 1998 by the California Department of Education, Diagnostic Center-Southern California (http://www.pent.ca.gov/whoarewe.htm) in order to provide a statewide dissemination of best practice in addressing problem behavior in alignment with discipline mandates of IDEA 1997 and IDEIA 2004. Members of the PENT organization represented the focal point of the trainings.

Participants

Beginning in 1997 and continuing through 2002, training on functional behavioral assessment and behavior plan development to address behavior interfering with learning, was conducted across the state of California by the senior author who was the PENT Director. A total of approximately 10,000 educators across the state participated in trainings. From that number, a cadre of local behavior specialists was selected. The selection process was developed in collaboration between the California Special Education Local Plan Area (SELPA) Directors and the California Department of Education, Diagnostic Center South's Director and PENT Project Manager, Deborah Holt, and PENT Director/Trainer. Eligibility was based on student attendance in each region, with more populous areas eligible to send more behavior specialists. Each SELPA was requested to send behavior specialists who had received the training on the six key concepts during the previous year and who were considered to be the most skillful in conducting FBAs and developing PBS plans. A total of 200 behavior specialists attended the first training of which 169 submitted both pre- and post-PBS plans. Each participant had their Master's degree and self-reported to have previously attended a minimum of two trainings on behavior management and taken a minimum of four to five classes in behavior analysis. Most participants were practicing school psychologists, resource specialists, or behavior specialists.

In order to assess how well these local "experts" were incorporating best practices presented at the initial training, prior to further training, all participants were required to submit a PBS plan developed by the school IEP teams in which they participated. A second requirement was that they submit another PBS plan developed for a student in association with an IEP team within six months of completing the second training. A total of 338 PBS plans were collected at pre- and post-training to evaluate the effects of training (Pre-Second Training, n = 169 and Post-Second Training, n = 169). Some participants attrited between pre- and post-plan submission. Moreover, since there was no contingency in place to strictly enforce post-second training plan submission, some participants that turned in pre-plans did not turn in post-plans even though they attended the second training. Nevertheless, 84.5% of the participants who submitted pre-plans also submitted their post-plans within the 6-month period.

Behavior Support Plan-Quality Evaluation Guide

After a preliminary review of the PBS plans obtained from earlier trainings, the authors determined that most plans did not specify interventions that were logically consistent with the behavior assessment data, and that key concepts were often missing. This observation is consistent with other reports of inadequate PBS plans in the field (Cook et al., in press; Smith, 2000; Van Acker et al., 2005).

Therefore, in order to more objectively evaluate each plan's quality, including internal consistency and presence of key concepts, an instrument was developed by Browning-Wright et al (2003): The Behavior Support Plan-Quality Evaluation Guide (BSP-QE). The BSP-QE is based on six key concepts that represent critical aspects of PBS plan development based on literature from applied behavior analysis, team functioning, team performance, and the law. The six key concepts were identified by conducting a comprehensive search of the literature and systematically identifying commonalities across the research, see Table 1.

In addition to key concepts, the BSP-QE included related items that evaluated whether the interventions matched the behavior assessment (e.g., Is the intervention based on function?; Are the environmental changes related to identified predictors?). In all, the BSP-QE includes 12 items that embody the six key concepts and logical relationship between items. The 12 items are rated on a scale of 0 to 2 to produce a maximum score of 24. Plans were determined to be "adequate" if 17/24 points were earned (71% of possible points) whereas "inadequate" plans earned 16 or fewer points (67% or less). Further divisions of "adequate" into "superior" or "good" and "inadequate" into "underdeveloped" or "weak" were made with corresponding guidelines discussed in Tools to Develop, Implement and Score Behavior Support Plans (Browning-Wright et al., 2003), see Table 2.

Reliability. The 200 pre-second training PBS plans were evaluated by 52 graduate students. A minimum of one team of two graduate students enrolled in advanced courses in behavior analysis at California State University, Los Angeles evaluated each plan. These students had completed between two and six courses in applied behavior analysis. The students were trained on the BSP-QE by the first and second authors, and had opportunities for dialogue with the trainers for several weeks during the evaluation. During training students were provided full, partial, and non-examples of each item (Browning-Wright et al., 2003). Students then practiced rating plans and were provided feedback based on performance until they agreed to at-least an 83% or better level with the trainer's rating. During reliability estimation, raters first scored plans individually, which were used to estimate inter-rater reliability. To derive a final score for the 200 PBS plans, raters then compared scores with each other to agree on a final score for each evaluated plan.

A second group of 46 advanced graduate students were subsequently trained one year later to evaluate the 169 post-PBS plans using the same procedures as above. Once trained, they followed the same rating sequence in order to estimate inter-rater reliability and agree on a final plan score. Data were then submitted to us for evaluation and summary.

Evidence in support of the reliability of the BSP-QE was established by calculating item-total correlations, internal consistency, and inter-rater reliability (IRR) statistics. The item-total correlations for the rating items had a range of .45 to .67, with an average of .59. In terms of the internal consistency, the BSP-QE obtained an alpha of .80, which indicated sufficient internal consistency across the rating items. To assess IRR, a total of 140 (58% of PENT Cadre) pairs of scores across different plans were generated. Due to the three point Likert-type metric of the rating items, the most appropriate method of computing an estimate of IRR was to calculate a Pearson Product Moment Correlation between the pairs of scores for each item and the total plan score. IRR estimates for the total plan score exceeded .80. Stemler (2004) contends that values greater than 0.70 are acceptable estimates of IRR and, thus, the BSP-QE serves as a consistent instrument across raters. These reliability data suggest that the BSP-QE produces adequate internal consistency and inter-rater reliability estimates.

Trainings

First training: Six hours on key concepts ("Behavior Plans That Work"). All trainings were conducted by the first author who had a MS degree in school psychology and five courses in applied behavior analysis. Though she was not a Board Certified Behavior Analyst, she did meet the requirements to sit for the certification exam. In accordance with the literature on the importance of multiple exemplars (Kinder & Carnine, 1991; Sprague & Horner, 1984), the first author reviewed the six key concepts illustrated in Table 1, explaining each concept with a minimum of four examples. Furthermore, potential functional hypotheses were reviewed for twelve disciplinary infractions. Activities were provided for participants to identify the "function" of four cases commonly encountered in the field: (a) an emotionally disturbed teenager bullying and threatening weaker peers for favors and money; (b) a middle school student with low academic skills refusing to do written work; (c) a socially inept fourth grader attempting to initiate social interactions through aversive interactions such as calling peers names and running away; and (d) a first grader with autism who swears or hits herself if corrected, or when frustration occurs in an activity. Participants then were given eight cases with corresponding functional hypotheses. Participants were required to design interventions in teams of three to five. Each team was required to develop interventions that (a) taught a functionally equivalent replacement behavior and (b) altered the environment to make the problem behavior irrelevant, inefficient, and ineffective (O'Neill et al., 1997). Potential interventions for behaviors by students with exceptional needs were described and methods of selecting appropriate reinforcers and teaching behaviors were reviewed, with multiple case examples given. Finally, complete behavior plans were reviewed and critiqued. For illustrated training materials, see: http://www.pent.ca.gov/03Training/TrainingTOC/TOC_BehPlans.htm

Second training: Three hours on key concepts and three hours on scoring using the BSP-QE (Behavior Plan Practicum: Developing and Scoring High Quality Behavior Plans). The PBS plan was reformatted to align the analysis and interventions in sequence for key concept three: "A complete PBS plan must address both strands, i.e., make environmental changes that support acceptable behavior, AND specify how to teach or promote functionally equivalent acceptable behavior." The first author described these changes and the rationale: too many cases demonstrated a lack of connection between the described analysis and the selected interventions. Next, the six key concepts in behavior analysis were reviewed (as above) for three hours, demonstrating their application to three cases: (a) A fifth grade student with task refusals, (b) a middle school student with disruptive verbal behaviors, and (c) a student with moderate mental retardation using profanity to initiate social interactions. Participants were cautioned about the use of the simple BSP form (see: http://www.pent.ca.gov/10Forms/BSPcolor06.doc) for addressing "serious" behavior problems. They were encouraged to seek further training, seek additional professional assistance, and use more complete data gathering strategies and tools if the behavior in question was assaultive, self injurious, caused severe property damage or constituted a pervasive, maladaptive behavior.

The second three hours were spent learning the scoring rubric and practicing scoring two cases in teams of three or four participants. Following completion, each case was reviewed any disparities in the teams' charted scores in the twelve areas were discussed as a large group. For training materials, see: http://www.pent.ca.gov/03Training/TrainingTOC/TOC_Practicum.htm. Participants in the second training included, but were not limited to (due to personnel changes), those who had attended the initial training.

Data Analytic Strategy

The inferential statistics employed in this research consisted of a paired t-test to examine within participant change and a chi-square analysis to determine the effects of training on PBS plan development for the 169 participants who submitted both pre and post PBS plans. In addition, effect sizes were computed in the forms of a [r.sub.BESD] and an odds ratio statistic to provide an index of the practical and social significance of the change produced by the training. The [r.sub.BESD] was transformed into the binomial effect size display (BESD), which further highlights the practical importance of an effect size. The BESD addresses the question, what is the effect of training (key concept training alone vs. key concept training plus training on the BSP-QE) on the success/nonsuccess rates of a given outcome (i.e., substantive quality of PBS plan). In essence, the BESD is a 2 X 2 contingency table with the columns representing training status (key concept training alone vs. key concept training plus training on the BSP-QE) and the rows representing success and nonsuccess rates, respectively. Success rate is defined as the percentage of individuals expected to be above the mean and nonsuccess rates are simply the reciprocal of the success rate (i.e., 1--success rate). The following are the results from the analyses of the data.

Results

Inferential Data Analysis

To determine whether training on the BSP-QE significantly improved the total mean substantive score of PBS plans from pre-training to post-training a paired t-test was computed. Results revealed that BSP-QE training, indeed, had a significant impact on the quality of PBS plan development, t (168) = 5.01, p < .001, two-tailed. The BSP-QE training improved the mean score of 14.91 before training to 17.47 after training which results in an associated [r.sub.BESD] of .36. When cast into the BESD, an [r.sub.BESD] of .36 indicates that 68% of those who receive BSP-QE training are likely to produce plans in the adequate range compared to only 32% of those that do not receive this training, but had received training on the key concepts. Hence, over one third more educators are likely to produce PBS plans in the adequate range following training on the BSP-QE than without.

A chi-square analysis was also performed to examine changes in the proportion of plans from "Inadequate" to "Adequate" after exposure to BSP-QE training. The chi-square analysis corroborated the results from the t-test and indicated that the proportion of plans falling in the categories "Inadequate" and "Adequate" significantly changed as a function of the BSP-QE training, [chi square](1) = 17.21, p < .001. The accompanying odds ratio statistic was 3.7, which indicated that PENT Cadre members were almost four times more likely to develop plans in the "Adequate" range following training on the BSP-QE.

In order to depict the amount of change that occurred across the categories of "Weak," "Underdeveloped," "Good," and "Superior," the frequency of plans in each category at pre- and post-Second training was graphed (see Figure 1). As one can see, the number of plans in the "Weak" and "Underdeveloped" ranges decreased quite dramatically while the plans in the "Good" and "Superior" ranges increased. Most notably, a 267% increase in the number of the plans in the "Superior" range was obtained following BSP-QE training. These results suggest that improvement in the knowledge and skills of school personnel to develop substantively adequate PBS plans can be achieved via training in using the BSP-QE.

Discussion

Although prior research has suggested that certain trainings may not be effective ways of improving PBS services for students (Scott et al., 2005), this study has indicated that the substantive quality of PBS plans developed by IEP teams for actual students can actually be improved by providing behavior specialists who serve on these teams a six hour training to learn how to evaluate a behavior plan using the BSP-QE. This result is consistent with the findings reported by other researchers who have found that educators can be trained effectively in FBA and behavior intervention methodology (Iwata et al., 2000; Moore et al., 2002). The BSP-QE was specifically designed to address internal consistency between analysis and intervention design on the BSP form used extensively across California (see: www.pent.ca.gov for forms and manual: Tools to Develop, Evaluate and Score a Behavior Support Plan). It should be noted, though, that the BSP-QE can be applied to any PBS plan form, since the rating content of the BSP-QE--six key concepts--should be present in every plan.

[FIGURE 1 OMITTED]

It appears that training on the BSP-QE may result in better plans, i.e., plans in alignment with the six key concepts identified as critical for any behavior plan that addresses positive interventions to change problem behavior. However, the reader should keep in mind that this effort was to improve the skills of individuals working in the field across California who have not necessarily had extensive training in applied behavior analysis, yet are required to develop behavior plans as part of their job assignment. The investigators thus attempted to enhance staff skills in developing better plans that address problem behavior that "interferes with the learning of the student or others." The authors did not envision this BSP-QE training to be sufficient for extreme behavior (e.g., severe self-injurious behavior). Safe and effective treatment for such extreme behavior requires a high level of expertise and/or supervision from an identified professional with extensive experience and certification. However, when multiple functions are suspected or confirmed, the BSP-QE training principles still apply.

After reviewing the PBS plans from the initial training condition, it was determined that most plans did not specify interventions that had a logical relationship with the antecedents and consequences analysis, and that the key concepts were often missing. This observation is consistent with other reports of inadequate PBS plans in the field (Cook et al., in press; Smith, 2000; Van Acker et al., 2005). Thus, we determined that the BSP-QE contained all the necessary training ingredients to instruct individuals how to better develop logically consistent and substantively meaningful PBS plans. In addition, the BSP-QE provides good examples and non-examples of particular concepts, which has been demonstrated as a vital component for effective instruction (Kinder & Carnine, 1991). The results of this investigation confirm our hypothesis by revealing that the skills of school personnel can be increased with training to improve the quality of PBS provided to individual students.

Moreover, the results from this research are particularly important because the training did not consist of rating case simulations or vignettes. Rather each participant was asked to submit the most recent PBS plan they developed for an actual student within the context of an IEP. Therefore, the results have direct generalizability to practice in the "real" world. Although there are certainly internal validity limitations inherent in the design of this study, the external validity, or the generality, of this study is quite high. The high external validity of this study notwithstanding, an important next step is to perform a randomized trial with adequate internal validity to determine robustly whether the training produces improvements in PBS plan quality.

Readers should also keep in mind the importance of conducting trainings to improve PBS plan quality, given the number of cases that have been taken to due process over procedural and substantive issues (Etschdeit, 2006; Maag & Katsiyannis, 2006). Although some districts may not see the added value in such trainings, the costs of providing a six hour pre- or in-service training to staff to improve the quality of services provided to students is low compared to the benefits that will likely be seen (e.g., avoidance of due process hearings, improved delivery of PBS services).

Limitations

This study had several limitations that are worth noting. Foremost, the study did not have a control group, so it is difficult to conclude that the obtained changes were due to the provided program or were due to some other variable. However, based on the current state of correct applications of behavior analysis, it is doubtful that the obtained changes were due to something else. Also, it is impossible to determine if the increase in skills is due to the BSP-QE training or a combination of the BSP-QE training and the first training of six hours on "key behavioral concepts." Subsequent research will need to address these issues.

An additional limitation of this study was the failure to assess concomitantly student outcomes and to collect data that would describe in detail characteristics of the participants. Without a doubt, the most important criterion of the behavioral support process is whether desirable change in the student's behaviors is produced. Although student outcomes were not evaluated as part of this investigation, this study did examine the ability to increase educators' skills to develop sound components of behavioral planning that are empirically supported elements of effective behavioral change. In this way, one would hope that the gains produced by the training would correspond directly to improved outcomes for students. One should, however, be cautious in making this conclusion until research has demonstrated that PBS plans of better substantive quality are associated with better outcomes than plans of poorer quality.

Future Directions

The investigators view this study as an initial step to establishing a method of evaluating PBS plans. Additional study phases are in need of investigation, which include: (a) establishing the relationship between the quality of plans and the extent to which they are implemented as intended; and, (b) determining the degree to which quality plans and their implementation result in desired behavior change. A follow-up study is currently in progress that will shed light on the relationships between these variables, as well as reveal other potential correlates of plan quality. In addition, the BSP-QE is under revision based on empirical findings and professional feedback to clarify concepts and provide better examples in an attempt to further improve its ease of use and reliability. Also, research is currently underway to reveal item discrimination indices that identify strengths and weaknesses that can be used to inform further training.

Future research should also look at educator characteristics to determine which characteristics moderate the effectiveness of training. An understanding of moderating factors will allow us to better identify which professionals are likely to benefit from pre-service and in-service training. Moreover, it would be useful to determine the mediational mechanisms that take place during PBS plan meetings that account for improved plan quality. An understanding of the mediational factors, such as structural or semantic processes, of PBS plan meetings would permit the development of a specific training on how to carry out effective meetings to develop PBS plans, as well as other team-based plans.

Some researchers contend that significant coursework in applied behavior analysis in addition to in-service training is needed before an individual is capable of competently addressing the critical elements of an effective PBS plan (Scott & Nelson, 1999). Thus, an additional follow up study has been conducted to examine the extent to which educators familiar with moderate to severe disabilities and who have received extensive, formal ABA training are likely to produce better quality PBS plans and student outcomes than those educators who have simply received training on the BSP-QE. Also, whether additional training of those educators on the BSP-QE statistically improves their plan quality is currently under investigation.

In the meantime, the authors hope that readers will avail themselves of training materials and forms at www.pent.ca.gov to provide BSP-QE training for their staff to improve the quality of behavior plans in their regions. Trainings can be provided by individuals trained in both education and applied behavior analysis. Preferably, trainings would be provided by behavior specialists who are Board Certified Behavior Analysts or qualify to sit for the exam. It is strongly recommended that district administrators serving students with persistent forms of behavior problems consider requiring that at least one educator, skilled in behavior analysis and PBS plan development, be available to serve in every educational building. This way, knowledge about how best to develop and implement positive behavioral support strategies is readily available in all schools.

References

Alex R. v. Forrestville Valley Community Unit School District. No. 03-3858[7th Cir. July, 2004].

Retrieved on April 9, 2005 from http://caselaw.lp.find.com/data2/circs/7th/033588p.pdf.

Browning-Wright, D., Saren, D., & Mayer, G. R. (2003). The behavior support plan-quality evaluation guide. Retrieved on April 2005 from http://www.pent.ca.gov.

Conroy, M. A., Clark, D., Gable, R. A., & Fox, J. J. (1999). Building competence in the use of functional behavioral assessment. Preventing School Failure, 43(4), 140-144.

Conroy, M., Katsiyannis, A., Clark, D., Gable, R.A., & Fox, J. M. (2002). State office of education practices: Implementing the IDEA disciplinary provisions. Behavioral Disorders, 27, 98-108.

Cook, C. R., Crews, S. D., Brown-Wright, D., Mayer, G. R., Gale, B., Kraemer, B., & Gresham, F. M. (in press). Establishing and evaluating the substantive adequacy of positive behavior support plans. Journal of Behavioral Education.

Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Upper Saddle River, NJ: Prentice Hall.

Crone, D. A., & Horner, R. H. (2003). Building positive behavior support systems in schools. New York, London: Guilford Press.

Drasgow, E., & Yell, M. L. (2001). School-wide behavior support: Legal implications and requirements. Child & Family Behavior Therapy, 24, 129-145.

Etschdeit, S. (2006). Behavior intervention plans: Pedagogical and legal analysis of issues. Behavioral Disorders, 31, 89-103.

Gable, R. A. (1999) Functional assessment in school settings. Behavioral Disorders, 24, 249-252.

Gresham, F. M. (2003). Establishing the technical adequacy of functional behavioral assessment: Conceptual and measurement challenges. Behavioral Disorders, 28, 282-298.

Gresham, F. M., Quinn, M. M., & Restori, A. (1999). Methodological issues in functional analysis: Considerations in generality to other disability groups. Behavioral Disorders, 24(2), 180-182.

Horner, R. H., Sugai, G., Todd, A. W., & Lewis-Palmer, T. (1999-2000). Elements of behavior support plans: A technical brief. Exceptionality, 8, 205-215.

Individuals with Disabilities Education Act Amendments of 1997, 20 U.S.C. [section] 1400 et seq.

Individuals with Disabilities Education Improvement Act Amendments of 2004, 20 U.S.C. [section] 1462(h) et seq.

Iwata, B.A., Wallace, M.D., Kahng, S.W., Lindberg, J.S., Roscoe, E.M., Conners, J., et al. (2000). Skill acquisition in the implementation of functional analysis methodology. Journal of Applied Behavior Analysis, 33, 181-194.

Kinder, D., & Carnine, D. (1991). Direct instruction: What it is and what it is becoming. Journal of Behavioral Education, 1, 193-213.

Maag, J. W., & Katsiyannis, A. (2006). Behavioral intervention plans: Legal and practical considerations for students with emotional and behavioral disorders. Behavioral Disorders, 31, 348-362.

Moore, J. W., Edwards, R. P., Sterling-Turner, H. E., Riley, J., DuBard, M., & McGeorge, A. (2002). Teacher acquisition of functional analysis methodology. Journal of Applied Behavior Analysis, 35, 73-77.

O'Neill, R. E., Horner, R. H., Albin, R. W., Sprague, J. R, Storey, K, & Newton, J. S. (1997). Functional assessment and program development for problem behavior: A practical handbook (2nd ed.). Pacific Grove, CA: Brooks/Cole.People v. Mason City Community Sch. Dist., (Iowa) 32 IDELR 216 (2000).

Scott, T. M., & Nelson, M. C. (1999). Functional behavioral assessment: Implications for training and staff development. Behavioral Disorders, 24, 249-252.

Scott, T. M., Nelson, C. M., & Zabala, J. (2003). Functional behavior assessment training in public schools: Facilitating systemic change. Journal of Positive Behavior Interventions, 5, 216-224.

Scott, T. M., McIntyre, J., Liaupsin, C., Nelson, C. M., Conroy, M., & Payne, L. D. (2005). An examination of the relation between functional behavior assessment and selected intervention strategies with school-based teams. Journal of Positive Behavior Interventions, 7, 205-215.

Smith, C. R. (2000). Behavioral and discipline provisions of IDEA '97: Implicit competencies yet to be confirmed. Exceptional Children, 66, 403-412.

Sprague, J., & Horner, R. H. (1984). An experimental analysis of generalized vending machine use with severely handicapped students. Journal of Applied Behavior Analysis, 17, 273-278.

Sulzer-Azaroff, B., & Mayer, G. R. (1991). Behavior analysis for lasting change. Fort Worth, TX: Harcourt, Brace, and Jovanovich.

Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research & Evaluation, 9, 1-14.

Van Acker, R., Boreson, L., Gable, R. A., & Potterton, T. (2005). Are we on the right course? Lessons learned about current FBA/BIP practices in schools. Journal of Behavioral Education, 14, 35-56.

Yell, M. L. (2002). The law and special education. Upper Saddle River, NJ: Merrill/Prentice Hall.

Diana Browning Wright, Southern California Diagnostic Center

G. Roy Mayer, California State University, Los Angeles

Clayton R. Cook, University of California, Riverside 

S. Dean Crews, Eastern Washington University 

Bonnie Rawlings Kraemer, San Diego State University 

Bruce Gale, BehaviorTech Solutions

Correspondence to G. Roy Mayer, Division of Special Education and Counseling, California State University, Los Angeles, 5151 State University Drive, Los Angeles, CAA 90032; e-mail: [e-mail].

Table 1 Descriptions and Reasons for Inclusion of the Six Key Concepts
of Positive Behavior Support Planning

Key Concept        Description                 Reason

Behavior function  Behavior serves a           The PBS plan must
                   particular purpose for the  identify the function
                   student (e.g., positive or  of the problem behavior
                   negative reinforcement).    in order to develop a
                                               plan that teaches a
                                               functionally equivalent
                                               replacement behavior
                                               (FERB).
Situational        Behavior is related to the  Something is either in
  specificity      context/environment in      the environment, or NOT
                   which it occurs.            in the environment
                                               which increases the
                                               likelihood the behavior
                                               will occur.
Behavior change    Changing behavior           A complete BSP must
                   involves addressing         address both strands:
                   both the environmental      make environmental
                   features AND teaching a     changes that support
                   functionally-equivalent     acceptable behavior,
                   behavior that student can   AND specify how to
                   use to satisfy the          teach or prompt
                   function of the behavior    functionally equivalent
                   in a acceptable way.        acceptable behavior.
                                               When a plan is
                                               implemented well and
                                               change is not
                                               occurring, evaluating
                                               whether both strands
                                               were addressed is a
                                               first step.
Reinforcement      New behavior must be        PBS plan must specify
  tactics          reinforced to result in     reinforcement for new
                   behavioral increases,       functionally equivalent
                   generalized performance     behavior. (PBS plan
                   and maintenance.            may also wish to
                                               specify general
                                               reinforcement for
                                               positive behaviors.)
Reactive           Implementers need to        PBS plan must specify
  strategies       know how to handle          reactive strategies
                   problem behavior if it      ranging from prompting
                   occurs again.               the alternative
                                               replacement behavior
                                               through distraction,
                                               redirection, response-
                                               cost tactics, and
                                               school and district
                                               disciplinary required
                                               actions.
Team coordination  For optimal team            PBS plan must specify
  and              performance, it is          who is responsible for
  communication    important to indicate       implementing each of
                   who is responsible          the plan components in
                   for carrying out each       order to build a system
                   element of the plan. And,   of accountability and
                   communication needs to      evaluate the fidelity
                   be between all important    of the plan. PBS plan
                   stakeholders, frequently    must specify who
                   enough to result in the     communicates with
                   progress monitoring         whom, how frequently
                   necessary to achieve        and in what manner.
                   success.

Table 2 Categories of Plan Quality According to the BSP-QE

                Points and Percentage
Category        Range                  Description

Weak            Fewer than 12 points   This plan may affect some change
                (<50%)                 in problem behavior but the
                                       written plan only weakly
                                       expresses the principles of
                                       behavior change. This plan should
                                       be rewritten.
Underdeveloped  13-16 points (51% to   This plan may affect some change
                69%)                   in problem behavior but would
                                       require a number of alterations
                                       for the written plan to clearly
                                       embody best practice. Consider
                                       alterations.
Good            17-21 points (70% to   This plan is likely to affect a
                90%)                   change in problem behavior and
                                       elements of best practice are
                                       present.
Superior        22-24 points (>91%)    This plan is likely to affect a
                                       change in problem behavior and
                                       embodies best practice.