J Behav Educ (2008) 17:93110 DOI 10.

1007/s10864-007-9053-y ORIGINAL PAPER

Comparing Individual Behavior Plans from Schools With and Without Schoolwide Positive Behavior Support: A Preliminary Study
Natasha S. Medley Steven G. Little Angeleque Akin-Little

Published online: 2 October 2007 Springer Science+Business Media, LLC 2007

Abstract School-wide positive behavior support (SWPBS) has been proposed as a proactive and preventive method to reduce problematic behavior in schools. Under this approach, educators and administrators seek to create a school environment that fosters prosocial behavior and attempts to systematically deter problem behaviors before they happen. To date, the relationship between SWPBS and individualized positive behavior support (PBS) plans has not been examined. Specically, it is unclear whether an atmosphere of SWPBS facilitates the functional behavioral assessment process and the design of PBS plans for students exhibiting severe behavior problems. The purpose of the present study was to investigate whether behavior support plans created in schools employing SWPBS systems were more technically adequate than support plans created in schools utilizing traditional approaches to behavior problems. Results indicated that support plans created at schools with SWPBS systems were more technically adequate than support plans produced at non-SWPBS schools as measured by the Behavior Support Plan-Quality Evaluation (BSP-QE). However, support plans from schools with SWPBS systems were still considered underdeveloped. Limitations and future research are discussed. Keywords Positive behavior supports Behavior support plans BSP-QE

Schools have become increasingly interested in identifying strategies to reduce disruptive and violent behaviors and raise prosocial behaviors in students. School-wide
N. S. Medley (&) Graduate School of Education, University of California, Riverside, Riverside, CA 92521, USA e-mail: Natasha.medley@email.ucr.edu S. G. Little A. Akin-Little Walden University, Minneapolis, MN, USA

123

94

J Behav Educ (2008) 17:93110

positive behavior support (SWPBS) is a proactive model that utilizes preventive strategies at three levels to reduce problematic behavior within school settings (Scott et al. 2002; Anderson and Kincaid, 2005; Eber et al. 2002; Safran and Oswald 2003; Sugai and Horner 2002). The three-tier model employs empirically-based interventions to promote prosocial behavior in (a) the general school population of students who lack chronic behavior problems (primary prevention); (b) students who are at risk of developing chronic behavior problems (secondary prevention); and (c) students with major behavioral problems (tertiary prevention). In an effective SWPBS system, changes in attitudes and behavior occur across both students and staff. Data suggest that an effectual SWPBS system will foster a school climate where appropriate behavior is acknowledged and appreciated by all staff, within-child psychopathology is not viewed as an unchangeable entity, and expectations of staff are universal (Horner and Carr 1997; Crone and Horner 19992000). At the core of this model is the problem-solving team whose purpose, at the tertiary prevention level, is to identify the environmental factors supporting the students problem behavior and produce strategies and interventions that will decrease those behaviors (Colvin et al. 1993). The functional behavior assessment (FBA) process plays a key role in the problem-solving teams ability to uncover the function of the behavior, and the antecedents and consequences that facilitate its occurrence. In the context of a SWPBS system, collaborative teams should be comprised of individuals that are knowledgeable in the area of behavior support. Furthermore, it can be expected that a comprehensive problemsolving team will have a team leader who has expertise in applied behavior analysis (Sugai et al. 19992000; Crone and Horner 19992000). When these foundational components of the SWPBS system are in place, it can be anticipated that behavior teams are equipped to navigate through the FBA and Behavior Support Plan (BSP) process. It may be further expected that these teams will create BSPs that will lead to more effective behavioral interventions than BSPs created by teams utilizing a more traditional approach and/or lacking in specic PBS-FBA training. For example, Benazzi et al. (2006) found that BSPs formulated by a team with a behavior specialist were rated higher in technical adequacy and contextual appropriateness than BSPs created by a team without a behavior specialist. Further, other researchers have found that availability of FBA information does not necessarily lead to more effective BSPs (Hsiao and Albin 2000) and that teams that use FBA information effectively need at least one member specially trained in behavioral theory (Mitachi and Albin 2001). Consequently, the assumption is that teams with more behavioral training will produce better quality BSPs than teams without that training. This study attempts to ascertain the veracity of that assumption. The success of a BSP clearly depends on the quality of the plan (Horner et al. 2000; Sugai et al. 1999). Producing accurate and technically sound BSPs is important not only because it is best practice, but also because it is mandated by federal law. IDEA (1997) clearly mandates that when disciplinary action is taken by school, the Individualized Education Plan (IEP) team should conduct an FBA and develop a related BSP. The 2004 IDEA revision continues the emphasis on BSPs and use of the FBA process (20 U.S.C. 1400 et seq.). Although inadequate

123

J Behav Educ (2008) 17:93110

95

plans may result in behavior change, this generally is not the case. Furthermore, when a BSP is inadequate it may actually exacerbate student problems (e.g., reactionary methods are only included, focus on hypothesized within-child psychopathology), leading to higher rates of problem behavior (e.g., Mayer et al. 1983). In recent years, several measures have been developed to judge the adequacy of the BSP (e.g., Lewis-Palmer et al. 2004). The Behavior Support Plan QualityEvaluation (BSP-QE) scoring guide (PENT 2003) is the only instrument that evaluates the quality of the BSP (the purpose of the present study), offers concrete denitions and examples for each component of the BSP, and allows the evaluator to calculate a total score that indicates the overall quality of the plan. The evaluator rates the support plans across 12 concepts, ranging from the denition of problem behavior to the relationship of functional assessment results to intervention strategies. The purpose of the present study was to evaluate whether individualized BSPs created in schools employing SWPBS were more technically adequate than those created in schools without SWPBS training, utilizing traditional approaches to behavior problems. It was hypothesized that individualized BSPs in the context of SWBS would be more technically adequate than those created in non-SWPBS schools, given that historically, interventions for students with behavior problems that rely on reactive procedures have not had positive outcomes (Walker et al. 1995; Walker and Shinn 2002). To date, it has been assumed that individuals with training in assessment-based intervention should develop improved BSPs due to a better understanding of the principles of behavior change and the FBA process. The intent of this study was to test this hypothesis. A second hypothesis was that the quality of support plans in SWPBS schools would be related to length of time the system had been in place, with the longer duration of SWPBS resulting in a higher score on technical adequacy of SWPBS plans. These two hypotheses are based on two assumptions: (a) a systemic view of the importance of proactive and preventative approaches to addressing problem behavior permeating to all levels of support, and (b) behavioral expertise of the collaborative team in schools with SWPBS will lead to higher quality plans. It is assumed that because staffs at SWPBS schools are required to receive training on PBS strategies, they should display a better understanding of behavior, its function, and environmental factors that maintain the problem behavior and thus produce more technically adequate BSPs.

Method Participants Schools Nine middle schools from a single district in an urban community located in Southern California participated in this study. The study included support plans

123

96

J Behav Educ (2008) 17:93110

from all nine middle schools in the district. Each school served between 1,300 and 2,200 students. The ethnic breakdown of the district is as follows: (a) Hispanic or Latino62%, (b) African-American19%, (c) White14%, (d) Asian2%, and (e) other3%. On average, 89% of students qualify for free or reduced lunch. At the time of the study, two of the nine schools had implemented SWPBS systems. SWPBS support plans were obtained from the two SWPBS schools and the seven middle schools not employing a SWPBS system. The demographics of the schools from which support plans were obtained were similar. The implementation dates of SWPBS system varied for both schools. At the time of the study School 1 had a SWPBS system in place for 22 months, while School 2 had a SWPBS system in place for 10 months. The composition of team members for both schools was similar; however, School 1 had a psychology intern on the team with training in applied behavior analysis.

SWPBS System and Training Each school implementing SWPBS established two collaborative teams. The Behavior Team (BT) was trained to lead the new school-wide policies and the Student Study Team (SST) assumed the role of handling behavioral concerns for nonresponsive students. Under the model, the SST was responsible for the implementation of behavior goals and writing support plans for at-risk students. Generally, the SST and BT were composed of the school psychologist, general education and special education teachers, and the vice principal. Some team members, such as general and special education teachers, participated in the meetings based on their involvement with the student under evaluation. Schools implementing a SWPBS system received four full days of onsite trainings focused on the framework for a Positive School-Wide Discipline program and the creation of the Tier 1 and Tier 2 interventions. Both SST and behavioral team members received a half-day training on the fundamentals of BSP writing, based on the Behavior Support Plan-QE that was developed by Positive Environments, Network of Trainers (PENT 2003). During these trainings, teams were taught how to use the information provided by the functional assessment to create a BSP. Team members were explicitly trained on the differences between an effective and poor BSP. The training addressed each component of the BSP and gave examples of how various answers would be scored. During the session, trainees were given the opportunity to practice writing a BSP and were given feedback. Additionally, after the training, team members were allowed to submit sample BSPs for critique and feedback. In order to ensure that all staff was familiar with the principles of SWPBS, an overview of the program was presented on site for all school employees. These trainings were separate from the 4-day training attended by collaborative teams and school administration. Furthermore, throughout the year district coaches offered a variety of positive discipline trainings for all staff to attend. Both SWPBS schools adopted three main rules on campus. The three broad rules were: (a) be safe, (b) be responsible, and (c) be respectful.

123

J Behav Educ (2008) 17:93110

97

District-wide Behavior Support Plan Training A basic BSP training is offered quarterly to all district employees. The training is offered as an option for professional development hours. These workshops are generally attended by school psychologists, resource specialists, and special education teachers. Typical attendance for each workshop is approximately 20 staff. Staff from SWPBS and non-SWPBS schools was welcomed to attend the basic training. The 4-h training is voluntary and takes place over a 2-day period. The training focuses on the purpose of the BSP and explains the various components of the BSP. The rationale behind the BSP and its role in special education placement are discussed. Trainees are provided with a blank template created to mirror the layout of the BSP-QE rubric. While each component of the BSP is mentioned, the training centers on teaching educators how to identify environmental correlates and function of problem behavior. Denitions of each construct are given and trainees are provided with four examples to show the possible functions of the behavior and environmental factors that increase the likelihood of the behavior occurring. However, trainees are not shown the types of responses that would be included in a technically adequate BSP. No additional assistance for BSP writing is provided after the workshop.

Materials All BSPs were evaluated using the BSP-QE scoring guide. The BSP-QE was created in 2003 by Diana Browning-Wright and Dru Saren, with input from Rob Mayer (PENT 2003). In its development, the rubric was used by over 200 behavior specialists and was revised to improve the educators ability to effectively evaluate BSPs and produce scores that accurately indicate the quality of the BSP plan. The purpose of the rubric is to establish whether the BSP developed by the team aligns with the principles of behavioral change found in applied behavior analysis. The BSP-QE does not assess the appropriateness of the BSP in relationship to the developmental needs of the students. Specically, the BSP-QE does not determine whether the identied function of the behavior is correct or whether the interventions selected were appropriate or implemented as intended. The guide is based on six key concepts posited to be important in the creation of an effective BSP. The six key concepts, as outlined by Positive Environments, Network of Trainers (2003) are as follows: (a) all behavior, including problem behavior serves a purpose for the student; (b) the behavior is related to the context or environment in which it occurs; (c) in order to change behavior, the environment must be changed such that the problem behavior is no longer effective and a functionally-equivalent replacement behavior must be taught; (d) to increase maintenance of behavior over time, the new behavior must be reinforced; (e) implementers must have a uniform method regarding how problem behavior will be addressed if it reoccurs; and (f) frequent two-way communication between all stakeholders must occur and staff training must be ongoing.

123

98

J Behav Educ (2008) 17:93110

The BSP-QE measures the key components over 12 categories. These 12 factors are: (a) problem behavior, (b) predictors of problem behavior, (c) the relationship between environmental changes and problem behaviors, (d) the logical relationship between environmental changes and events supporting the problem, (e) the relationship of predictors to the function of the behavior, (f) the relationship of function to replacement behavior, (g) the relationship of teaching strategies to replacement behavior, (h) the quality of reinforcers to be used during the intervention, (i) the adequacy of reactive strategies to be utilized when the child exhibits the problem behavior, (j) goals and objectives of the intervention, (k) team coordination and implementation, and (l) outline of communication. Each category can be scored as zero, one, or two, with two indicating the objectives of the category were met and zero meaning the objectives were either minimally met or absent. The rubric operationally denes the characteristics required to accompany scoring for each category. The total BSP-QE score ranges from 0 to 24. BSPs yielding fewer than 12 points are categorized as weak. The PENT cadre suggests that while this plan may affect some change in problem behavior, the written plan only weakly expresses the principles of behavior change. It is suggested that any plan scoring in this category should be rewritten. BSPs receiving scores between 13 and 16 represent underdeveloped plans. While there is the possibility that this plan could produce some change in behavior, it would require many modications to embody best practice. Plans producing scores between 17 and 21 points are categorized as good. Plans rated as good are likely to produce positive changes in behavior and incorporates elements of best practice. BSP yielding scores of 22 points or more are considered superior. The PENT cadre postulates that this plan is likely to produce positive changes in behavior and embodies best practice. Recent research suggests adequate reliability and validity of the BSP-QE. Cook et al. (in press) indicated a Cronbach alpha greater than .80 and an interrater reliability estimate that exceeded .80. Furthermore, content validity was reviewed by experts in PBS and applied behavior analysis, who reported that the BSP-QE had adequate content validity (Cook et al. in press).

Procedure Behavior support plans from SWPBS and non-SWPBS schools were obtained from the district. Several steps were taken to identify support plans for this study. First, a list was generated with the names of students for whom support plans had been created at each school. Next, students were randomly selected from the list and their BSPs were obtained from the schools. Two support plans for randomly selected students could not be found despite reports indicating that support plans had been developed. Forty support plans (21 SWPBS; 19 non-SWPBS) were evaluated in this study. Prior to evaluation, all identifying information was removed from the plans. In order to control for response bias, any information indicating whether the support plan

123

J Behav Educ (2008) 17:93110

99

was from a SWPBS or non-SWPBS school was removed. The only coding information on the support plan was the student identication number. Each plan was evaluated by the rst author using the BSP-QE. The standards outlined by the rubric were applied to each item. The score and the rationale for the score were recorded for each item. After all components were rated, the item scores were summed and assigned a categorical evaluation based on the total score. This process was repeated for all support plans. Prior to rating the plan, the rst author received 4 h of training on the BSPQE and how to evaluate plans by two behavior specialists. Training focused on each component of the BSP, offering examples of the types of responses that would yield a score of zero, one, or two. During this training the author practiced applying the principles of the rubric to a sample BSP. Then, she created her own support plan and evaluated it using the BSP-QE. The trainers concluded that the rst author adequately applied the guidelines of the rubric to the evaluation of the BSP.

Interobserver Agreement In order to obtain an interobserver agreement estimate, 25% (10 support plans) of the sample support plans were evaluated by a second reviewer. The second reviewer received the same 4-h training as the rst author. The support plans were compared on each of the 12 components outlined on the BSP-QE. Interobserver agreement was calculated by dividing the number of exact numeric agreements on each component by the total number of items and multiplying by 100%. Reliability was calculated using Cohens kappa (Cohen 1960). Agreement across all components was .61. According to Landis and Koch (1977), the strength of agreement based on a score of .61 is good.

Data Analysis Procedures An independent samples t-test was used to determine whether or not there were signicant mean differences between SWPBS and non-SWPBS total scores. Specically, support plans were compared on how closely they adhered to the guidelines outlined on the BSP-QE. Support plans were compared on each of the 12 components of the BSP-QE and the total score. A t-test was employed because there were only two levels of the independent variable. Furthermore, t-tests are robust to minor departures of normality and homogeneity of variance. Graphs of the total scores did not reveal any outliers in the distributions of SWPBS and non-SWPBS scores. Additionally, Fishers z transformation was used to identify any signicant differences in correlations amongst variables based on whether the school was a SWPBS or non-SWPBS school. Finally, a Pearson correlation was calculated to determine whether there was a relationship between length of SWPBS implementation and BSP quality.

123

100

J Behav Educ (2008) 17:93110

Results Mean Comparisons Forty support plans (21 SWPBS; 19 non-SWPBS) were evaluated. An independent samples t-test was preformed to compare the total scores yielded by SWPBS and non-SWPBS schools. A comparison of mean total score of SWPBS schools and non-SWPBS schools resulted in a mean of 13.95 for SWPBS schools (SD = 6.67) and a mean of 7.84 for non-SWPBS schools (SD = 3.76). A Levenes test for the equality of variances for the total BSP score yielded an F-value of 16.769 (p \ .001) therefore equal variances were not assumed for the t-test. Analysis revealed a t of 3.606 with 32.175 degrees of freedom and a two-tailed p-value of .001. This indicates that there was a signicant difference between the total scores yielded by SWPBS and non-SWPBS schools, such that SWPBS schools received signicantly higher total scores on the BSP-QE. When total scores were differentiated by school type and BSP-QE effectiveness categories, very different patterns of score distributions emerged. In SWPBS schools the following pattern emerged: a) nine of 21 support plans were rated as weak, b) three of 21 were rated as underdeveloped, c) 6 of 21 were rated as good, and d) three of 21 were rated as superior. In non-SWPBS schools, 16 of 19 support plans were categorized as weak and three of the 19 non-SWPBS support plans were categorized as underdeveloped.

Correlational Analyses We hypothesized that there would be a positive relationship between the length of SWPBS implementation and support plan total scoresthat is longer the implementation of a three-tier SWPBS model, the higher the total scores. A Pearson correlation was utilized to assess the relationship between length of time and total scores. Results did not conrm this hypothesis. A signicant correlation was not found for length of SWPBS implementation and total scores, r = 0.169. Analysis of SWPBS schools yielded moderate to large correlations between the BSP total score and problem behaviors (r = .491), predictors of behavior (r = .834), environmental changes (r = .618), predictors related to function (r = .536), replacement behaviors (r = .792), teaching strategies (r = .856), reinforcement (r = .952), reactive strategies (r = .871), goals and objectives (r = .918), and communication (r = .820). The variable predictors of behavior yielded moderate to strong correlations with environmental changes (r = .476), replacement behaviors (r = .678), teaching strategies (r = .736), reinforcement (r = .782), reactive strategies (r = .691), goals and objectives (r = .775), and communication (r = .616). Results are listed in Table 1. Bivariate correlations were calculated for SWPBS and non-SWPBS schools. According to Cohens (1988) guidelines, analysis yielded moderate to large correlations. Results for non-SWPBS schools are listed in Table 2. Moderate to large correlations were found for the BSP total score and problem behaviors (r = .492), predictors of behavior (r = .564), environmental changes (r = .697),

123

Table 1 Intercorrelations among components and total score on the BSP-QE for SWPBS Schools 2 3 4 5 6 7 8 9 10 11 12 13

Measure

1. Problem behavior

J Behav Educ (2008) 17:93110

2. Predictors of behavior .165 .237 .160 .389 .264 .144 .382 .200 .171 .386 .200 .224 .168 .037 .618** .536* .792** .477* .488* .636** .132 .175 .175 .014 .669** .856** .064 .611** .509* .661** .724** .568** .536* .757** .673** .578** .487* .771** .850** .857** .832** .023 .847** .952** .212 .834** .288 .617** .871** .383 .116 .464* .285 .702** .212 .462* .256

.293

3. Supporting problem behavior

.160

.353

4. Environmental changes

.212

.476*

5. Predictors related to function

.327

.378

6. Replacement behaviors

.394

.678**

7. Teaching strategies

.327

.736**

8. Reinforcement

.345

.782**

9. Reactive strategies

.426

.691**

10. Goals and objectives

.509*

.775**

11. Team coordination

.175

.209

12. Communication

.302

.616**

.673* .918** .247

.181 .070 .213 .820** .124 .169

13. BSP total score

.491*

.834**

14. Length of PBS implementation

.159

.232

* p \ .05, two-tailed; ** p \ .01, two-tailed

101

123

102

123
2 3 4 5 6 7 8 9 10 11 12 .123 .007 .160 .418 .220 .474* .205 .223 .233 .245 .697** .581** .062 .062 .317 .765** .182 .102 .425 .093 .054 .056 .190 .262 .469* .208 .231 .393 .362 .303 .428 .065 .147 .341 .179 .167 .038 .240 .337 .102 .217 .190 .562* .055 .244 .303 .422 .369 .079 .070 .171 .224 .354 .517* .308

Table 2 Intercorrelations among components and total score on the BSP-QE for Non-SWPBS Schools

Measure

1. Problem behavior

2. Predictors of behavior

.470*

3. Supporting problem behavior

.120

.041

4. Environmental changes

.197

.507*

5. Predictors related to function

.243

.218

6. Replacement behaviors

.351

.305

7. Teaching strategies

.122

.239

8. Reinforcement

.436

.576**

9. Reactive strategies

.023

.215

10. Goals and objectives

.224

.105

11. Team coordination

.014

.248

12. Communication

.149

.217

13. BSP total score

.492*

.564*

J Behav Educ (2008) 17:93110

* p \ .05, two-tailed; ** p \ .001, two-tailed

J Behav Educ (2008) 17:93110

103

predictors related to functions (r = .581), replacement behaviors (r = .765) and reactive strategies (r = .562). A moderate correlation was found between problem behavior and predictors of behavior (r = .470). A large correlation was found between predictors of behavior and environmental changes (r = .507) and reinforcement (r = .576). A moderate correlation was found between supporting problem behaviors and reactive strategies (r = .474). Lastly, a strong correlation was suggested between predictors related to function and replacement behaviors (r = .517). Since the Pearson correlation coefcient of SWPBS schools was not being compared to zero, but to a known r (non-SWPBS), a Fishers z transformation was used to compare the correlations produced by non-SWPBS and SWPBS schools. Results indicated signicant differences in correlations between total BSP scores and predictors of behavior (Z = 2.352), teaching strategies (Z = 4.085), reinforcement (Z = 6.288), reactive strategies (Z = 2.966), goals and objectives (Z = 5.284), and communication (Z = 4.563). Results were in the predicted direction. Schools with SWPBS systems yielded higher positive correlations than non-SWPBS schools. Interestingly, the correlation for reactive strategies and supporting problem behavior was higher for non-SWPBS schools (r = .474) than SWPBS schools (r = .144). This was the only correlation in which the difference between the schools displayed different results such that the correlation for a non-PBS school was higher. Results are listed in Table 3.

Discussion The purpose of this study was to determine if there were differences in the technical quality of BSPs written at non-SWPBS and SWPBS schools. Specically, the study sought to investigate whether schools employing a SWPBS system produced better support plans than schools utilizing traditional methods to address students displaying problematic behaviors. It was hypothesized that SWPBS schools would yield signicantly higher total scores as measured by the BSP-QE than non-SWPBS schools. Mean comparisons between the two types of school supported this hypothesis. BSP total scores on average were higher for SWPBS schools than nonSWPBS schools. However, despite yielding results in the hypothesized direction, the scores still indicated that over half of the plans in the SWPBS schools were rated as underdeveloped or weak according to the BSP-QE rubric. Amongst the two SWPBS schools, it was expected that length of implementation would inuence the quality of the support plan. It was presupposed the school that had SWPBS in place longer would produce higher BSP total scores than the school with the shorter duration. The difference in implementation dates of the school-wide PBS system for the two SWPBS schools was 1 year. Results indicated that there was not a signicant relationship between length of implementation and overall score on the BSP-QE. This was also true for individual components of the BSP-QE. No signicant correlations were found between length of implementation and the 12 components of the BSP. These results seem counterintuitive given that according to the districts schedule for training SWPBS schools, the school with the longer

123

104

J Behav Educ (2008) 17:93110

Table 3 Item-total score product-moment correlations for SWPBS and non-SWPBS schools Variables 1. Replacement behaviors-predictors of behavior 2. Teaching strategies-predictors of behavior 3. Reactive strategies-predictors of behavior 4. Goals and objectives-predictors of behavior 5. Communication-predictors of behavior 6. BSP total score-predictors of behavior 7. Reactive strategies-supporting problem behavior 8. Goals and objectives-environmental changes 9. Reinforcement-predictors related to function 10. Goals and objectives-predictors related to function 11. Communication-predictors related to function 12. Replacement behaviors-teaching strategies 13. Reinforcement-replacement behaviors 14. Reactive strategies-replacement behaviors 15. Goals and objectives-replacement behaviors 16. Reinforcement-teaching strategies 17. Reactive strategies-teaching strategies 18. Goals and objectives-teaching strategies 19. BSP total score-teaching strategies 20. Reactive strategies-reinforcement 21. Goals and objectives-reinforcement 22. Communication and reinforcement 23. BSP total score-reinforcement 24. Goals and objectives-reactive strategies 25. Communication-reactive strategies 26. BSP total score-reactive strategies 27. Communication-goals and objectives 28. BSP total score-goals and objectives 29. Communication-BSP total score * p \ .05, two-tailed; ** p \ .01, two-tailed SWPBS .679 .736 .691 .775 .616 .834 .144 .611 .487 .509 .488 .702 .771 .757 .661 .850 .673 .724 .856 .857 .832 .847 .952 .834 .617 .871 .673 .918 .820 Non-SWPBS [.305 [.239 [.215 [.105 [.217 [.564 \.474 [.093 [.065 [.054 [.062 [.224 [.147 [.469 [.056 [.341 [.208 [.231 [.303 [.179 [.167 [.240 [.337 [.102 [.190 [.562 [.244 [.303 [.079 Z 2.178* 2.945** 2.669** 3.932** 2.114** 2.352* 2.119* 2.623** 1.992* 2.144* 1.992* 2.725** 3.703** 2.051* 3.120** 3.822** 2.555* 2.856** 4.085** 4.648** 4.326** 4.208** 6.288** 4.610** 2.225* 2.966** 2.398* 5.284** 4.563**

implementation date would have received more specialized training on support plan writing, which should have led to higher prociency in BSP writing. The anticipated result may not have been observed for two reasons. First, there may have been a difference between the school staff in the overall acceptance of the SWPBS system. It has been recommended that at least 80% of school staff must be willing to participate and adhere to the policies of the SWPBS system (Sugai et al. 2003). In places where the school climate is not supportive of the SWPBS system, the development of the program may be stied, therefore limiting collaborative team growth. A second factor that may account for the results were the previous skills of the collaborative team members. The effect of length of implementation may have

123

J Behav Educ (2008) 17:93110

105

been muted if the members of the collaborative team of the school with the shorter implementation date had a strong behavioral foundation prior to training. Because data were not collected on the professional background of the team members or staff, this explanation can only be viewed as speculation. The total score on the BSP-QE is based on 12 different factors. However, many of these factors are interrelated, such that incorrect assessments on one factor may negatively affect subsequent ratings. Given these interrelations, it can be expected that several of the individual components will be highly correlated. Overall, the correlations in SWPBS schools followed the anticipated pattern; however, some factors did not correlate that should have. For example, the BSP-QE the component predictors of behavior is supposed to directly impact recommendations in environmental changes and predictors of function. This expectation was supported for environmental factors, but not for predictors related to function. Environmental changes referred to environmental, curriculum, and/or interaction changes that would alleviate the need to engage in the problem behavior. These environmental modications were supposed to be related to the identied antecedents reported in predictors of behavior. The correlation yielded for these variables was moderate, suggesting that there was a positive relationship between these variables. However, the expected relationship between predictors of behavior and predicted function was not observed. The antecedents noted in predictors of behavior should have informed the teams perception of why the student was engaging in the target behavior. A nonsignicant relationship between these variables suggests that teams failed to integrate these two concepts. For example, in one BSP the identied predictors of behavior were: (a) academic tasks involving cursive writing, (b) timed math assignments, (c) dictation tasks, and (d) whole group instruction. In order to gain the highest score of two on the BSP-QE the environmental changes must be logically related to the predictors. The environmental changes recommended on the BSP were: (a) seat student next to the door for access to alternative work areas, (b) allow the student to select a lunch buddy and leave class early for lunch, (c) consider the students sensory sensitivities and need for movement, and (d) front-loading or telling the student what the next activity will be and what he needs to do. Upon review, only one of the proposed environmental modications was logically related to the reported predictors of behavior. Although these types of errors were less frequent on the support plans produced at SWPBS schools, it is important to note that they also had problems integrating the various components of the BSP. Fewer correlations were observed between variables on support plans written at non-SWPBS schools. This partially explains why the total scores of non-SWPBS schools were lower than SWPBS school. As mentioned earlier, many of the items on the BSP-QE build on each other so that is there is a relationship between many of the variables. In order to achieve a high total score, several of the items must be logically related to one another. Only ve correlations between variables were observed in non-SWPBS schools. Interestingly, contrary to SWPBS schools, a strong correlation was observed between predictors of behavior and environmental factors; however, the correlation amongst predicted function and predictors of behavior was not signicant. In non-SWPBS schools, the predictors identied often

123

106

J Behav Educ (2008) 17:93110

centered around internal or within-child psychopathology as opposed to environmental triggers; therefore, functions were typically unrelated to antecedents. For example a support plan listed mood as an antecedent and hypothesized the function of the behavior to be a place of belong. Furthermore, the strong correlation for environmental changes and predictors of behavior may have been found because of the logical link between medicine and psychopathology. Often the recommended environmental change would be to ensure that the child receive his or her medication, which despite not being a school factor, is related to a predictor stating the child has bipolar disorder and refuses to take medication. Signicant differences were observed between correlations from SWPBS and non- SWPBS schools. Overall, signicant differences were found in 29 correlations for SWPBS and non-SWPBS schools. These differences suggest that SWPBS schools were better at integrating the components the BSP, therefore producing stronger relationships amongst the items. Differences in non-SWPBS and SWPBS ?tul?> correlations were statistically signicant for: (a) function of behavior, (b) reinforcement, (c) reactive strategies and (d) communication. The positive relationship observed between these variables indicates that SWPBS schools were better at identifying the function of behavior, and were found to use appropriate reinforcement and reactive strategies. The signicant difference in reactive strategies between the two types of schools, suggest that SWPBS schools produced more strategies that were supportive, corrective, and assisted in deescalating the child than reverting to punishment procedures. Non-SWPBS schools often used traditional disciplinary methods, such as time out, following student problem behavior.

Limitations Several limitations existed in this study. First, the training and implementation of the SWPBS systems were not directly observed. All of the information about training was obtained from interviews with the training facilitators and reviews of training manuals. Although it is assumed that all trainings were implemented as reported, no documentation exists showing exactly what took place or who attended the trainings. Furthermore, due to other district commitments, several promised supports such as BSP evaluation and feedback by district trainers, were not offered on a consistent basis to SWPBS schools. Second, the actual composition of the collaborative teams was unknown for all of the schools included in the study. Similar to the limitations identied by Scott et al. (2005a), the author had no knowledge of the teams skill level prior to training; therefore, it cannot be assumed that the differences observed in quality of support plans was due solely to training and presence of SWPBS. It is possible that collaborative team members in SWPBS schools had expertise in behavioral assessment and intervention construction preceding implementation of the schoolwide program. Additionally, the extent to which SWPBS was being implemented in the two SWPBS schools was not formally evaluated. Future research should use the

123

J Behav Educ (2008) 17:93110

107

School-wide Evaluation Tool (Horner et al. 2004) to evaluate the adequacy of SWPBS implementation. Third, although the BSP-QE has been widely used to evaluate BSPs, very little research regarding its validity and reliability exist. Currently, there is only one known study indicating high reliability and validity for the BSP-QE (Cook et al. in press). The interobserver agreement reliability (Kappa) yielded for this study was only .61. The raters found several areas of the rubric to be unclear, which led to discrepancies in scores. In order to effectively apply the rubric extensive training is necessary. However, the BSP-QE is the only metric available to date that operationally denes and quanties the quality of BSPs. Lastly, this study focused solely on the technical adequacy of support plans and did not address treatment integrity or effectiveness. Technical adequacy is not analogous to treatment integrity; therefore, research needs to be conducted investigating the link between technical adequacy and integrity. Gresham (1989) suggested that treatment integrity may be increased by writing out the intervention plan. The underlying assumption of this study is that a technically adequate BSP will most likely lead to higher integrity because it clearly identies the problem behavior and the proposed intervention procedures. In addition, a technically adequate plan is presumably more effective, also increasing the likelihood of implementation with integrity.

Relevant Findings and Implications The overall hypothesis that schools employing SWPBS systems would produce more technically adequate BSPs than SWPBS schools was conrmed. This supports assertions suggesting that SWPBS systems promote increased understanding of behavior and environmental factors that can support the maintenance of the problem behavior (Walker et al. 1996, Sugai et al. 2003). However, it is important to note that even with higher total scores; the support plans produced at the SWPBS schools were still evaluated as underdeveloped. These ndings support previous research suggesting that acquisition level trainings are not sufcient to produce accurate FBA based support plans or interventions (Scott et al. 2005a, b). Scott et al. (2005a) found that despite extensive training, functional behavior assessment teams still did not consistently connect assessed function of behavior with logically corresponding strategies related to function. The same problem was observed in this study. Despite training, collaborative teams were not consistent in their integration of the BSP components. Inconsistency across support plans may be the result of an inadequate functional behavioral assessment or lingering attitudes related to traditional methods of discipline. The ndings of this study suggest that ongoing staff training is necessary to increase the quality of BSPs in both SWPBS and non-SWPBS and schools. Given that many of the components of the BSP are interrelated, strategically focusing on items that inform other variables may assist in increasing the overall quality and consistency of the plan. For example, since predictors of behavior informs the variables predicted function of the behavior and environmental modications,

123

108

J Behav Educ (2008) 17:93110

by ensuring that staff understands how to correctly identify predictors of behavior, the scores in subsequent areas may increase.

Future Research This study focused solely on the quality of BSPs produced at SWPBS and nonSWPBS schools. Future research should conduct evaluations of the functional behavioral assessment process, in addition to the quality of resulting BSPs. Perhaps the problems observed in this study were related to a awed FBA process. Future studies also should focus on determining the amount of training necessary to produce consistency across assessments and support plans. In addition, specic factors that lead to team success in the FBA-BSP process should be studied. Another important measure of effectiveness is how well the BSP translates into effective interventions in SWPBS and non-SWPBS schools. Future research should investigate how the technical adequacy of the BSP inuences the implementation and success of behavioral interventions in the presence or absence of SWPBS. Treatment delity also may be of interest. An interesting research question would be whether treatment delity is higher in SWPBS schools. Also, additional research is needed on the BSP-QE. For example, research should seek to uncover whether the categories yielded by the BSP-QE (i.e., weak, underdeveloped, good, superior) differentially predict the success of the intervention. The importance of function-based interventions for children with behavioral problems has been well established (Ingram et al. 2005; Anderson and Kincaid 2005, Horner and Carr 1997). With IDEA (1997), the federal government acknowledged the importance of creating individualized, functionally based intervention plans. Given these requirements, problem-solving collaborative teams must possess the knowledge and skills to appropriately navigate through the FBA process and gather meaningful information to produce an effective BSP. As observed in this study and previous research, these skills are not easily acquired. Ongoing research in training is necessary if educators are expected to use FBA and PBS strategies to decrease problem behavior.

References
Anderson, C. M., & Kincaid, D. (2005). Applying behavior analysis to school violence and discipline problems: Schoolwide positive behavior support. Behavior Analyst, 28, 4963. Benazzi, L., Horner, R. H., & Good, R. H. (2006). Effects of behavioral support team composition on the technical adequacy and contextual t of behavior support plans. The Journal of Special Education, 40, 160170. Cohen, J. (1988). Set correlation and contingency tables. Applied Psychological Measurement, 12(4), 425434. Cohen, J. (1960). A coefcient of agreement for nominal scales. Educational and Psychological Measurement, 20, 3746. Colvin, G., Kameenui, E. J., & Sugai, G. (1993). Reconceptualizing behavior management and schoolwide discipline in general education. Education & Treatment of Children, 16, 361381.

123

J Behav Educ (2008) 17:93110

109

Cook, C. R., Crews, S. D., Wright, D. B., Mayer, G. R., Gale, B., Kraemer, B., & Gresham, F.M. (in press). Establishing and evaluating the substantive adequacy of positive behavior support plans. Journal of Behavioral Education. Crone, D. A., & Horner, R. H. (19992000). Contextual, conceptual and empirical foundations of functional behavioral assessment in schools. Exceptionality, 8, 161172. Eber, L., Sugai, G., Smith, C. R., & Scott, T. M. (2002). Wraparound and positive behavioral interventions and supports in the schools. Journal of Emotional and Behavioral Disorders, 10, 171180. Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18, 3750. Horner, R. H., Todd, A. W., Lewis-Palmer, T., Irvin, L. K., Sugai, G., & Boland, J. B. (2004). The School-wide Evauation Tool (SET): A research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6(1), 312. Horner, R. H., Albin, R. W., Sprague, R. R., & Todd, A. W. (2000). Positive behavior support. In M. E. Snell & F. Brown (Eds.), Instruction of students with severe disabilities (pp. 4783). Columbus: Merrill. Horner, R. H., & Carr, E. G. (1997). Behavioral support for students with severe disabilities: Functional assessment and comprehensive intervention. Journal of Special Education. Special Issue: Research in Severe Disabilities, 31(1), 84109. Hsiao, Y., & Albin, R. W. (2000, May). The effects of functional assessment information on the behavioral support recommendations for school personnel. Paper presented at the Association for Behavior Analysis Convention. Washington, DC. Individuals with Disabilities Education Act Amendments of 1997, Public Law 10517, 20, U. S. C. Chapter 33, Section 1415 et seq.. Individuals with Disabilities Education Act Reauthorization of 2004, 20 U.S.C. 1400 et seq. Ingram, K., Lewis-Palmer, T., & Sugai, G. (2005). Function-based intervention planning: Comparing the effectiveness of FBA function-based and non-function-based intervention plans. Journal of Positive Behavior Interventions, 7, 224236. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159174. Lewis-Palmer, T., Todd, A. W., Horner, R. H., Sugai, G., & Sampson, N. (2004). Individual student system evaluation tool. Eugene: Educational Community Support, University of Oregon. Mayer, G. R., Butterworth, T., Nafpaktitis, M., & Sulzer-Azaroff, B. (1983). Preventing school vandalism and improving discipline: A three year study. Journal of Applied Behavior Analysis, 16, 55369. Mitichi, M., & Albin, R. W. (2001, May). The effects of functional assessment information on the behavioral support recommendations of school personnel. Paper present at the Association of Behavior Analysis Convention, New Orleans. Positive Environments, Network of Trainers. (2003). http://www.pent.ca.gov/behBbspqe.htm Safran, S. P., & Oswald, K. (2003). Positive behavior supports: Can schools reshape disciplinary practices? Exceptional children, 69, 361373. Scott, T. M., Liaupsin, C., Nelson, C. M., & McIntyre, J. (2005a). Team-based functional behavior assessment as a proactive public school process: A descriptive analysis of current barriers. Journal of Behavioral Education, 14, 5771. Scott, T. M., McIntyre, J., Liaupsin, C., Nelson, C. M., Conroy, M., & Payne, L. D. (2005b). An examination of the relation between functional behavior assessment and selected intervention strategies with school-based teams. Journal of Positive Behavior Interventions, 7, 205215. Scott, T. M., Nelson, C. M., Liaupsin, C. J., Jolivette, K., Christle, C. A., & Riney, M. (2002). Addressing the needs of at-risk and adjudicated youth through positive behavior support: Effective prevention practices. Education & Treatment of Children, 25, 532551. Sugai, G., & Horner, R. (2002). The evolution of discipline practices: School-wide positive behavior supports. Child & Family Behavior Therapy, 24, 2350. Sugai, G., Horner, R. H., & Gresham, F. M. (2003). Behaviorally effective school environments. In M. R. Shinn, H. M. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II: Preventative and remedial approaches (pp. 315351). Betheseda: National Association of School Psychologists. Sugai, G., Horner, R. H., & Sprague, J. R. (1999). Functional-assessment-based behavior support planning: Research to practice to research. Behavior Disorders, 24, 253257.

123

110

J Behav Educ (2008) 17:93110

Sugai, G., Lewis-Palmer, T., & Hagan-Burke, S. (19992000). Overview of the functional behavioral assessment process. Exceptionality, 8, 149160. Walker, H. M., Colvin, G., & Ramsey, E. (1995). Antisocial behavior in schools: Strategies and beast practice. Pacic Grove: Brooks-Cole. Walker, H. M., Horner, R. H., Sugai, G., Bullis, M., Sprague, J. R, Bricker, D., & Kaufman, M. (1996). Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. Journal of Emotional and Behavioral Disorders, 4, 194209. Walker, H. M., & Shinn, M. R. (2002). Structuring school-based interventions to achieve integrated primary, secondary, and tertiary prevention goals for safe and effective schools. In M. R. Shinn, H. M. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II: Preventative and remedial approaches (pp. 315351). Betheseda: National Association of School Psychologists.

123