Exploring SWPBIS Implementation Outcomes in Rural and Urban Schools in Florida

Abstract

Rural schools experience unique challenges, including teacher quality and teacher retention, limited resources, and availability of funding. Furthermore, access to professional development and, subsequently, implementation of evidence-based practices may also be limited in rural settings. One evidence-based framework for implementing evidence-based practices, School-Wide Positive Behavioral Interventions and Supports (SWPBIS), has been widely implemented, including in rural and urban schools. Yet, very little research has explicitly compared rural and urban schools implementing SWPBIS with regard to implementation and discipline. Therefore, we examined statewide data to evaluate differences between rural and urban schools implementing SWPBIS in Florida. We found that both rural and urban schools were equally likely to implement all the components of SWPBIS. When comparing schools, we found that rural schools implementing SWPBIS had more out-of-school suspensions than nonimplementing rural schools, while the opposite was true for urban schools. Limitations and future research are discussed.

Keywords

School-Wide Positive Behavior Interventions and Supports suspensions rural fidelity of implementation

Rural schools face unique challenges, often including geographic isolation, concerns about teacher quality and teacher retention, limited resources and availability of funding, meeting new federal and state accountability requirements, and low student achievement (Arnold et al., 2005; Rude & Miller, 2018; Steed et al., 2013). Therefore, research is needed to identify specific issues constraining the implementation of evidence-based practices in rural schools and, subsequently, how to address those unique challenges for rural schools. Unfortunately, there is limited research that explicitly compares the implementation of evidence-based practices in rural and urban settings, and even fewer employing rigorous research methods (Cicchinelli & Beesley, 2017). To address this gap, we compared rural and urban schools’ implementation of universal School-Wide Positive Behavior Interventions and Supports in Florida using a post hoc quasi-experimental design and extensive statistical controls. The purpose of this article was to identify whether rural schools demonstrate implementation challenges and whether there are differences in implementation effects on school suspensions by urban locale (e.g., rural or urban).

Rural Education

According to the 2017–2018 data from the National Center for Education Statistics, a third of the approximately 130,000 K–12 schools in the United States are located in rural areas whereas a quarter of our nation’s schools reside in urban areas (De Brey et al., 2019). While rural schools outnumber urban schools across the country, rural student enrollment averages 40% fewer students than their counterparts. Yet, all states contain both rural and urban school districts regardless of size. For example, 51% of Florida’s school districts are rural, yet the state contains 50% of the top 10 largest urban school districts in the country. Extreme variabilities within a single state can present unique challenges, especially in rural settings. A review of research found that rural schools are challenged by geographic isolation, restricted budgets, limited salaries, high parent expectations of school leadership, and difficulty recruiting and retaining highly qualified teachers and principals (Preston et al., 2013). Furthermore, rural schools have limited access to the consistent, ongoing professional development needed to implement evidence-based practices with high levels of fidelity (Cavanaugh & Swan, 2015; Steed et al., 2013). Therefore, understanding similarities and differences in the implementation of evidence-based practices between rural and urban schools is needed. This information can then be used to develop novel professional development approaches targeting rural schools’ needs and addressing the unique geographic challenges, potentially through new technologies (e.g., Schmidt et al., 2015).

The quality of educational research has also been noted as a concern. Arnold et al. (2005) reviewed 498 research studies conducted in rural settings and found limited experimental or quasi-experimental research explicitly comparing rural and nonrural settings. The authors found only 21% of the studies used a comparative design and only 2% of studies were considered “high quality” comparative research. More recently, Cicchinelli and Beesley (2017) reviewed the research and found modest increases in rigorous research but apparent gaps between rural and urban education in identifying specific concerns and professional development targets. Thus, more research is needed in explicitly comparing and contrasting rural and urban implementation of evidence-based practices and determining if differences exist across the two locales for adequate support planning.

School-Wide Positive Behavioral Interventions and Supports (SWPBIS)

School-Wide Positive Behavioral Interventions and Supports (SWPBIS) is an evidence-based multitiered framework for implementing prevention and intervention practices in schools to reduce the occurrence of disruptive behaviors (Sugai & Horner, 2002). Prevention and intervention practices are delivered to all students at the universal level (Tier 1), to students nonresponsive to universal prevention (Tier 2), and to students with intensive problem behaviors (Tier 3). SWPBIS is implemented in more than 29,000 schools across the country (Center on PBIS, 2020), and extensive research has explored implementation processes (Mathews et al., 2014), scale-up efforts (Bradshaw & Pas, 2011), and student and school impacts (Pas et al., 2019). Lee and Gage (2020) conducted a meta-analysis of experimental and quasi-experimental research and found schools implementing SWPBIS had fewer discipline outcomes, including suspensions (d = -0.26), and higher academic achievement (d = 0.11).

SWPBIS has been implemented in Florida for nearly 20 years under the support of the statewide training and technical assistance project, Florida Positive Behavioral Interventions and Support Project (FLPBIS). FLPBIS has supported 62 of the 67 Florida school districts and trained more than 2,100 schools in Tier 1 PBIS. Given the expansive reach of PBIS across Florida districts, FLPBIS has worked with numerous schools of every demographic. The FLPBIS Project maintains a comprehensive evaluation system (i.e., implementation fidelity measures, student outcome measures) with a high response rate; an average of at least 70% of active PBIS schools submit a fidelity and outcome measure each year. In addition, schools that implement SWPBIS with fidelity have realized improved outcomes (FLPBIS, 2020).

SWPBIS in Rural Settings

Although there is extensive research exploring implementation and impacts of SWPBIS, there is limited research focusing on the implementation of SWPBIS in rural schools. For example, McCrary et al. (2012) described the implementation of SWPBIS in four rural West Texas schools. During their first year of implementation, the authors monitored implementation efforts and found schools reduced time spent in restrictive environments (i.e., in-school suspension, alternative school placement) and increased time in classroom instruction, yet fidelity data were not reported. McDaniel et al. (2020) conducted a systematic review of the SWPBIS literature base on underrepresented educational settings (e.g., schools serving majority non-White students and/or those from low-income backgrounds) and coded across 13 variables (e.g., geographic region, % race, % SPED, % ELL, underserved indicator). Criteria for inclusion did not imply rural locale yet shared many of the same variables that rural environments typically experience. Across the 46 eligible studies, SWPBIS outcomes demonstrated effectiveness in reducing disciplinary issues, but only four of the studies explicitly described “rural setting.”

One challenge in exploring the effects of urban locale in the SWPBIS research is that few studies describe SWPBIS differences by urban locale. Researchers may have described a setting as “rural” (e.g., Kartub et al., 2000; Lane et al., 2008) but not how the context influenced the implementation. For example, Curtis et al. (2010) describe a SWPBIS training for a leadership team in a rural school that showed reductions in out-of-school suspensions and fewer instructional days lost but did not specify how the “rurality” influenced the training. Others have adapted SWPBIS to reflect the local rural culture, such as shifting SWPBIS practices for a population that incorporated Native American cultural adaptations to its school-wide expectations (Jones et al., 2006). Nese et al. (2019) estimated predictors of latency to achieving implementation fidelity based on the measure used in each school’s state (e.g., Tiered Fidelity Inventory [TFI], Benchmarks of Quality [BoQ], Schoolwide Evaluation Tool [SET]). Using data from 708 schools across the United States, the authors examined urban locale as a predictor of latency to fidelity. The authors found that rural schools had the largest latency, taking an average of 2.3 years after initial training, but were less likely to abandon SWPBIS than urban schools. Others, including Gage et al. (2019) and Kim et al. (2018), include urban locale as a covariate in their statistical analyses to control for differences but do not explicitly compare implementation or effects by urban locale.

As limited research specifically comparing the implementation of SWPBIS in rural schools exists, we designed this study to fill this gap and explore differences in both fidelity of implementation and school suspensions by urban locale. We compare rural and urban schools as there is a clear geographic distinction between the two urban locales. Furthermore, we extend the rural education literature by using a rigorous post hoc quasi-experimental design to compare rural and urban schools implementing SWPBIS and explore interaction effects for urban locale and SWPBIS implementation. Our study was guided by the following research questions:

Research Question 1: Are there differences in implementation fidelity between rural and urban schools?

Research Question 2: Are there differences in the rate of suspensions between rural and urban schools implementing SWPBIS?

Research Question 3: Are there differences in the rate of suspensions between rural and urban schools implementing and not implementing SWPBIS?

Method

We combined three different datasets from the 2015–2016 school year to create the analysis sample. First, we obtained data from the U.S. Department of Education’s Office of Civil Rights (OCR). Every other year, the OCR conducts the Civil Rights Data Collection (CRDC), which requires all U.S. public schools to report on specific discipline outcomes by gender, race, and disability. The CRDC includes hundreds of variables, but we focused exclusively on in- and out-of-school suspensions for this study. Second, we obtained data from the U.S. Department of Education’s National Center for Educational Statistics (NCES). NCES collects demographic data for all U.S. public schools, including urban locale based on U.S. Census criteria that define locales as (a) urban, (b) suburban, (c) town, and (d) rural (Geverdt, 2015). Finally, we obtained SWPBIS fidelity of implementation data and the number of years schools were implementing SWPBIS from the FLPBIS Project. We used both NCES identification numbers and school and district names to merge the three data sets. The full Florida data set contained 3,950 schools after merging.

Next, we reduced the data set to only schools in rural and urban locales based on the U.S. Census definition. The final analytic sample consisted of 1,627 schools. Of those, 511 were located in rural settings, while 1,116 schools were located in urban settings. Table 1 provides demographic characteristics for all schools, schools by rural or urban settings, and schools by rural or urban setting and SWPBIS implementation. In summary, there was a greater percentage of White students and students receiving free/reduced-price lunch (FRL) in rural schools than in urban schools. There were also more rural schools with different grade levels (e.g., K–12, 3–8), more alternative schools, and fewer charter schools in rural settings. Schools implementing SWPBIS had more students than schools not implementing SWPBIS. There were greater percentages of Hispanic students and smaller percentages of Black students in rural schools implementing SWPBIS than in urban schools implementing SWPBIS. Overall, there were very few charter schools implementing SWPBIS compared with schools not implementing SWPBIS.

Table 1.

School Characteristics by Urban Locale and School-Wide Positive Behavior Interventions and Supports Implementation.

Characteristics	Full sample	Urban	Rural	No SWPBIS		SWPBIS
Characteristics	Full sample	Urban	Rural	Urban	Rural	Urban	Rural
Number of Schools	1,627	1,116	511	701	363	415	148
Enrollment	635 (540)	644 (556)	618 (506)	577 (593)	559 (514)	757 (465)	761 (456)
% White	41.0% (27.2%)	33.2% (24.1%)	58.0% (25.8%)	31.9% (25.4%)	57.8% (26.4%)	35.4% (21.7%)	58.5% (24.1%)
% Black	30.4% (27.2%)	36.3% (28.1%)	17.7% (19.9%)	35.0% (28.8%)	19.0% (21.7%)	38.3% (26.8%)	14.6% (13.9%)
% Hispanic	22.4% (21.4%)	24.1% (22.3%)	18.9% (18.9%)	27.2% (24.9%)	17.8% (18.1%)	18.8% (15.6%)	21.6% (20.5%)
% FRL	41.0% (26.4%)	33.2% (26.9%)	58.0% (25.1%)	31.9% (29.8%)	57.8% (26.7%)	35.4% (20.2%)	58.5% (17.1%)
% SPED	13.7% (33.4%)	14.8% (27.6%)	11.2% (43.5%)	14.4% (28.9%)	9.4% (51.2%)	15.5% (25.2%)	15.8% (8.3%)
% LEP	7.8% (11.4%)	9.0% (12.0%)	5.3% (9.3%)	9.8% (13.3%)	4.8% (9.0%)	7.6% (9.4%)	6.5% (9.7%)
Elementary (K–5)	53.8%	56.0%	48.9%	51.8%	44.6%	63.1%	59.5%
Middle School (6–8)	14.4%	14.7%	13.7%	12.1%	11.8%	19.0%	18.2%
High School (9–12)	15.4%	15.1%	16.0%	17.4%	17.4%	11.3%	12.8%
“Other” Grade Levels	16.4%	14.2%	21.3%	18.7%	26.2%	6.5%	9.5%
SPED School	3.7%	4.2%	2.5%	5.3%	3.3%	2.4%	0.7%
Alt School	11.1%	10.4%	12.5%	13.6%	17.4%	5.1%	0.7%
Charter School	16.6%	18.2%	13.1%	28.4%	17.9%	1.0%	1.4%

Note. All values are means except those in parentheses, which are standard deviations. SWPBIS = School-Wide Positive Behavioral Interventions and Supports, FRL = free/reduced-price lunch, SPED = special education services, LEP = limited English proficiency.

Measures

Benchmarks of quality

We used the BoQ (Cohen et al., 2007; Kincaid et al., 2005, 2010) to measure treatment schools’ fidelity of implementation. The BoQ is a 53-item rating scale completed by a school’s SWPBIS team in consultation with a SWPBIS coach to measure Tier 1 implementation fidelity. The BoQ items are organized into 10 SWPBIS elements defined as critical indicators of adherence: (a) the SWPBIS team, (b) faculty commitment to SWPBIS implementation, (c) effective procedures for dealing with discipline, (d) data entry and analysis plan established, (e) expectations and rules developed, (f) reward/recognition program established, (g) lesson plans for teaching expectations/rules, (h) implementation plan, (i) classroom management systems, and (j) evaluation. The 53 items are scored between 0 and 3 points, with 12 items allowing for a score of 3 for fully-in-place and 43 items using a 2 as fully-in-place. The scores are then summed and divided by the total number of points possible (107) and scaled as a percentage. Cohen et al. (2007) defined fidelity as a score of 70% or above as they evaluated the psychometric properties of the BoQ. They found adequate total score internal consistency (α = .96), test–retest reliability (r = .94, p < .01), and interrater reliability (r = .87, p < .01). As noted earlier, the BoQ is part of standard practice for the FLPBIS Project’s comprehensive evaluation and is collected from all SWPBIS schools every spring.

Suspension from school

We focused on two types of suspensions: in-school suspensions (ISS) and out-of-school suspensions (OSS). The CRDC data set provides the number of students with one or more ISS and separates OSS into two types: (a) the number of students with only one suspension and (b) the number of students with more than one suspension. We retained both types of OSS and summed them for an overall OSS count. The CRDC also collects the number of days missed for OSS, which we used in this study. All data were from the CRDC database (see https://ocrdata.ed.gov/flex/Reports.aspx?type=school).

Covariates

We included a series of school- and district-level demographic variables. First, we included an indicator for the rural or urban setting of each school using NCES data based on the U.S. Census (Geverdt, 2015). Next, we identified schools by the grade levels they served as elementary, middle, high, and “other.” “Other” was defined as any grade span that did not fall within typical grade spans (e.g., grades 1–5, 6–8), such as schools serving K–12th or 5th–12th grades. We then included the total number of students enrolled in each school and the total number of full-time employees in the school. Student demographics included the percentage of students classified as (a) White, Black, or Hispanic; (b) receiving FRL, (c) receiving special education services, and (d) receiving limited English proficiency services. We aggregated the student demographic variables at the district-level to capture district-wide demographics. Finally, we included an indicator for the number of years a school had been implementing SWPBIS and the percentage of schools within a school district implementing SWPBIS.

Data Analyses

We used a series of multilevel models to answer all three research questions. All models were estimated with schools nested in districts. To answer the first two research questions, we limited the data to only schools implementing SWPBIS (n = 563), while models addressing Research Question 3 used all schools (N = 1,627). We transformed the suspension outcomes because they were significantly skewed. The skewness of total OSS, single OSS, multiple OSS, and ISS was 5.1, 4.6, 5.5, and 4.6, respectively, indicating a significant positive skew. We explored both square root and cube root transformations and found that cube root transformation improved skew for total OSS, single OSS, multiple OSS, and ISS was 0.8, 0.7, 0.6, and 0.6, respectively. The skewness of a normally distributed outcome is zero; thus, the closer to zero, the more normally distributed the variable is. We did not transform the fidelity measures as all skewness values were <2.0.

First, we estimated an intercept-only model to calculate the intraclass correlation coefficient (ICC), which describes the amount of variance within and between school districts. Next, we estimated four models for each outcome: (a) treatment only, (b) a treatment only with random slope, (c) school-level covariates, and (d) school- and district-level covariates. We evaluated model fit for each successive model using the loglikelihood ratio test. If no difference was found between the random intercept and the random intercepts and slopes models, we used the most parsimonious model (i.e., random intercept model) for subsequent models with covariates. We grand mean centered all continuous variables; all other variables were dichotomous or categorical. We also included an interaction term for school level (e.g., elementary) to compare differences by the grade levels served. All models were estimated in lme4 (Bates et al., 2015) in R.

Results

First, we examined the descriptive statistics of all dependent variables (see Table 2). The average fidelity of implementation score was 77%. The subscale with the highest percentage was Effective Procedures, while the lowest percentage subscale was Implementation Plan. Overall, there were very few differences between rural and urban schools on BoQ scores. The largest difference between rural and urban schools was for the Lesson Plan subscale, while the smallest difference was for Reward Program. Based on the total score, 73% of the schools implemented SWPBIS with fidelity (i.e., received a total BoQ score higher than 70%). Seventy-three percent of urban schools implemented with fidelity, while 74% of rural schools implemented with fidelity. The average OSS per 100 students was 10, while the average ISS per 100 students was nine. Based on the raw data, urban schools used more OSS and ISS than rural schools.

Table 2.

Descriptive Statistics for Fidelity of Implementation and Suspensions.

Outcome measure	All		Urban schools		Rural schools
Outcome measure	M	SD	M	SD	M	SD
PBS Team	86.6	17.8	87.3	17.6	84.5	18.3
Faculty Commitment	73.4	26.6	73.9	26.1	71.8	27.8
Effective Procedures	86.9	16.7	86.2	17.5	88.8	14.2
Data Entry Plan	72.0	25.1	70.7	25.5	75.5	23.6
Expectations	85.5	19.7	85.6	19.5	85.3	20.3
Reward Program	76.1	22.0	76.1	22.2	76.0	21.3
Lesson Plans	70.9	30.3	72.6	29.5	66.2	32.1
Implementation Plan	69.1	25.8	69.6	25.9	67.6	25.4
Classroom	81.6	21.4	81.1	21.8	82.9	20.4
Evaluation	71.3	24.0	70.6	24.3	73.3	23.4
BoQ Total Score	77.2	18.5	77.1	18.7	77.2	18.2
Total OSS	10.5	23.0	11.0	24.4	9.4	19.5
Single OSS	5.2	8.7	5.3	9.2	4.8	7.4
Multiple OSS	5.3	15.3	5.7	16.1	4.6	13.4
ISS	8.6	16.0	8.8	16.9	8.0	13.8

Note. All fidelity subscales are percentage of items present; suspension outcomes are rate per 100 students. PBS = positive behavior support, BoQ = Benchmarks of Quality, OSS = out-of-school suspension, ISS = in-school suspension.

Fidelity of Implementation in Rural and Urban Schools

We estimated a series of multilevel models using only data from schools implementing SWPBIS and reporting a BoQ score. There were 563 schools in 48 school districts that had available BoQ scores and were located in rural or urban settings. ICC values for the empty models ranged from 0.02 for the PBIS Team subscale to 0.10 for Evaluation. The ICC for the total BoQ score was 0.09. Overall, the values suggest that, on average, <10% of the variance for BoQ scores was between districts. Although the ICC values were low, we still used multilevel models to increase the accuracy and precision of the standard errors (Huang, 2018). (Full tables of results can be obtained from the first author.) The only subscale with significant differences between all rural and urban schools was the PBIS Team subscale (b = −6.09, p < .015). The interaction effect for school type revealed that rural high schools had significantly lower scores on the PBIS Team, Faculty Commitment, and Classroom subscales than urban high schools. Finally, we estimated a multilevel logistic regression model to evaluate differences in the likelihood a school implements SWPBIS with fidelity by urban locale (rural or urban). Results suggest that urban and rural schools have the same odds of implementing SWPBIS with fidelity (i.e., receiving a BoQ total score >70%; log[OR] = -0.15, p = .709).

Suspension Outcomes for Schools Implementing SWPBIS

Next, we estimated a series of models to compare the suspension outcomes for rural and urban schools implementing SWPBIS. The model results are presented in Table 3. However, as noted, we included interaction effects for school grade level but did not report them in the tables for parsimony. Overall, we found no statistically significant difference when comparing rural and urban schools implementing SWPBIS. However, we did find a significant effect for fidelity, indicating that schools with BoQ scores >70% had significantly fewer suspensions. We also found that middle and high schools used more suspensions than elementary schools. Alternative schools used suspensions more frequently, and schools with greater percentages of free/reduced-price lunch had more suspensions. Schools with greater percentages of students receiving special education services had more OSS but equal numbers of ISS. The only significant interaction effect was the three-way interaction for rural schools with the “Other” grade-level structure implementing with fidelity. These schools had significantly fewer ISS than other schools (b = −0.28, p = .009).

Table 3.

Rural and Urban Suspension Outcomes for Schools Implementing School-Wide Positive Behavior Interventions and Supports.

Parameters	All OSS		Single OSS		Multiple OSS		ISS
Parameters	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
(Intercept)	0.33***	0.02	0.33***	0.02	0.30***	0.02	0.29***	0.03
Rural	0.00	0.03	0.00	0.03	–0.01	0.04	–0.01	0.04
Middle	0.17***	0.02	0.17***	0.02	0.17***	0.03	0.17***	0.03
High	0.09**	0.03	0.09**	0.03	0.11**	0.04	0.12**	0.04
“Other”	0.07	0.05	0.07	0.05	0.05	0.06	–0.09	0.06
Fidelity	–0.03*	0.01	–0.04*	0.02	–0.04*	0.02	–0.05*	0.02
SPED School	–0.16	0.08	–0.15	0.08	–0.18	0.10	–0.11	0.11
ALT School	0.32***	0.03	0.32***	0.03	0.34***	0.04	0.32*	0.04
Charter School	0.07	0.04	0.08	0.05	0.10	0.06	0.04	0.06
% FRL	0.26***	0.04	0.26***	0.04	0.30***	0.05	0.30***	0.05
% SPED	0.26***	0.08	0.26**	0.08	0.27**	0.10	0.17	0.10
% LEP	–0.05	0.08	–0.08	0.09	0.00	0.10	–0.12	0.11
% White	0.05	0.14	0.05	0.15	0.09	0.18	–0.01	0.19
% Black	0.20	0.14	0.21	0.14	0.24	0.17	0.14	0.18
% Hispanic	–0.02	0.15	–0.01	0.15	–0.05	0.18	–0.04	0.20

Note. Models included interaction effects not reported here for parsimony. All dependent variables are cube root transformed. OSS = out-of-school suspension, ISS = in-school suspension, SPED = special education, ALT = Alternative, FRL = free/reduced-price lunch, LEP = Limited English Proficient; SE = standard error.

p< .05. **p< .01. ***p< .001.

Suspension Outcomes for Schools Implementing and Not Implementing SWPBIS

Last, we compared schools implementing SWPBIS with schools not implementing SWPBIS in rural and urban settings. These models included treatment as an independent variable; therefore, we also estimated models with random slopes for treatment and compared the random intercept models to random intercept and random slope models. First, we estimated ICC values for all four outcomes. The average ICC for the three OSS outcomes was ~0.10, suggesting that only 10% of the variance in OSS was between districts. The ICC for ISS was 0.15, suggesting that only 15% of the variance was between districts. Next, we examined random slope models. The loglikelihood test suggested that the random slope models were significantly different than the random intercept-only models; therefore, we included a random slope for treatment in all subsequent models.

We ran a series of four models for each outcome. The first model estimated differences between schools implementing SWPBIS and schools not implementing SWPBIS in rural and urban schools, while the second added a third interaction for school grade level. We then reestimated the same models but used fidelity as the treatment indicator, meaning schools with BoQ scores >70%. Overall, we found no significant effects for the three-way interactions that included school grade level, so we focused on the models with the two-way interactions for SWPBIS and urban locale (rural or urban). Table 4 presents the results for the first models with SWPBIS implementation compared with non-implementation. We found that schools in rural settings used significantly fewer OSS than urban schools, while middle and high schools used significantly more OSS and ISS. We found a significant, positive interaction effect between rural settings and SWPBIS implementation. This suggests that schools implementing SWPBIS in rural settings use significantly more OSS than rural schools not implementing SWPBIS. However, this also indicates, and we confirmed, that urban schools implementing SWPBIS had statistically significantly fewer OSS than urban schools not implementing SWPBIS. No differences were found for ISS.

Table 4.

Rural and Urban Suspension Outcomes in Schools Implementing and Not Implementing School-Wide Positive Behavior Interventions and Supports.

Parameter	All OSS		ISS
Parameter	Estimate	SE	Estimate	SE
(Intercept)	0.29***	0.02	0.24***	0.02
Rural	–0.05**	0.01	–0.02	0.02
SWPBIS	0.03	0.02	0.03	0.02
Middle	0.16***	0.01	0.17***	0.01
High	0.09***	0.01	0.12***	0.01
“Other”	0.12***	0.02	0.12***	0.02
SPED School	–0.07	0.05	–0.16**	0.05
ALT School	0.19***	0.02	0.18***	0.02
Charter School	–0.01	0.01	–0.02	0.01
% FRL	0.28***	0.02	0.26***	0.03
% SPED	0.05	0.04	0.10*	0.04
% LEP	–0.11	0.06	–0.09	0.07
% White	0.03	0.11	–0.11	0.11
% Black	0.13	0.10	0.00	0.10
% Hispanic	0.05	0.11	–0.12	0.11
Rural*SWPBIS	0.05*	0.02	0.03	0.02

Note. All dependent variables are cube root transformed. OSS = out-of-school suspension, ISS = in-school suspension, SWPBIS = School-Wide Positive Behavioral Interventions and Supports, SPED = special education, ALT = Alternative, FRL = free/reduced-price lunch, LEP = Limited English Proficient.

p< .05. **p< .01. ***p< .001.

Discussion

This study compared both implementation and suspensions for schools using SWPBIS in rural and urban schools. As noted, very little research has specifically compared rural and urban schools with regard to SWPBIS to determine if there are, in fact, differences in both implementation and outcomes. Overall, we found very few differences in universal implementation fidelity and that the odds of implementing with fidelity were equivalent for both rural and urban schools. We found that rural schools use fewer OSS than urban schools, regardless of SWPBIS implementation (i.e., implementing SWPBIS or not implementing). Finally, we found that rural schools implementing SWPBIS had more OSS than rural schools not implementing SWPBIS, while the opposite was true for urban schools; fewer OSS in schools implementing SWPBIS. Below, we connect these findings to research and recommendations for adapting professional development and planning technical assistance supports for rural schools.

Using a relatively large sample of schools implementing SWPBIS, we found the likelihood of reaching implementation fidelity at the universal level was the same for rural and urban schools. This suggests that the training strategies employed by FLPBIS are successfully meeting the implementation needs of rural schools. Yet, it is interesting that we did find a BoQ subscale difference. Notably, rural schools had significantly lower scores on the PBIS Team subscale, which includes only 3 items: “Team has administrative support,” “Team has regular meetings (at least monthly),” and “Team has established a clear mission/purpose.” There are several potential reasons for the difference when reviewing previous studies. In a study that looked at high needs schools, including rural schools, McDaniel et al. (2018) identified five factors to consider when implementing in high needs schools, including SWPBIS Readiness (e.g., buy-in and foundational skills) and Stakeholder Support (e.g., parents, administrators, teachers, and community). The results suggest that technical assistance with rural schools should consider these two factors as a high priority.

Adding to the complexity of perceptions versus outcomes, rural schools often have limited infrastructure such as personnel to facilitate a strong implementation process. It is common for SWPBIS teams to include specialized support personnel such as school psychologists, behavior analysts, and/or other district-supported staff. Therefore, limited support availability may lead to one person being responsible to do more things and serve multiple schools in a district, which may impact school teams meeting regularly. Berry et al. (2012) reported that rural districts struggle with hiring and retaining highly qualified special educators and Doss et al. (2017) indicated that there is a scarcity of school psychologists to meet the state’s needs. Despite limitations, some studies in rural schools have recognized and responded to the needs of a team by providing funding, time, and resources for SWPBS implementation in schools (Menendez et al., 2008). Oswald et al. (2005) reported that a middle school training included bi-monthly team meetings after school to reflect when they were able to meet and limited the meetings to 45 min. The need for dedicated team planning with skilled personnel appears to be critical for active SWPBIS implementation.

The absence of personnel with behavioral expertise to prevent and respond to behaviors may also contribute to our finding of higher OSS in rural schools compared with urban schools. Administrators in rural schools may be less likely to “buy-in” to SWPBIS and refrain from playing an active role in the implementation and team processes when consumed with discipline issues. Furthermore, Preston et al. (2013) noted that rural principals are less likely to be “highly qualified” and may have different, less evidence-based perspectives on school discipline. On the contrary, rural schools that implemented SWPBIS have reported strong administration support and buy-in (Curtis et al., 2010; Menendez et al., 2008). In one training across nine teams in a district, one of the schools was highlighted for its work and effective outcomes, including a 50% reduction in OSS (Fitzgerald et al., 2014). Among other practices, several included active principal participation: the principal was involved directly with the team, serving as the facilitator, and held a Principal’s Lunch once every quarter for students (one from each Grade 6–12) randomly selected for good behavior (Fitzgerald et al., 2014). The findings in this study both support and contradict earlier studies. Therefore, more research is needed, including qualitative and case study analyses, to better understand why there are discrepancies in the PBIS Team subscale, including any possible relation to administrative buy-in and support when compared to the level of implementation fidelity and student outcomes achieved. It is possible that SWPBIS training in rural schools may need to explicitly incorporate unique teaming approaches and incentives to support staff given the smaller number of personnel available in rural schools. For example, one rural district provided stipends to internal coaches for participation on the SWPBIS team (Cavanaugh & Swan, 2015). With more tasks expected across less personnel, careful consideration in providing effective and efficient support in rural settings is essential.

It is noteworthy that we found less faculty commitment and less classroom implementation in rural high schools than in urban high schools. Research on SWPBIS in high schools suggests it typically takes at least 3 years in high schools to reach implementation fidelity compared to 2 years in elementary schools (Nese et al., 2019). In addition, secondary educators often focus exclusively on academics rather than integrating behavior support, which might impact the adoption of classroom implementation (Flannery et al., 2013). Yet, when Bohanon et al. (2006) examined the SWPBIS implementation process in an urban high school, they found that staff perceived implementation as increasing in school-wide and classwide areas as the implementation progressed. Replication in a rural high school could provide insights as to whether the same perceptions exist and what might be needed to specifically support high schools in the implementation of SWPBIS in rural settings. Overall, more research is needed to explore differences in implementation levels and staff perceptions in rural high schools.

One of the primary reasons for implementing SWPBIS is to reduce disruptive behavior and, conjointly, reduce exclusionary discipline events, such as ISS and OSS. The results for schools implementing SWPBIS suggest that there are no differences in the rates of either of these disciplinary exclusions by urban locale (rural or urban). We also found that, although fidelity of implementation predicted decreased OSS and ISS, meaning better implementation results in fewer ISS and OSS, there was no significant interaction effect on fidelity by urban locale. Put differently, schools that implement with fidelity see fewer ISS and OSS, regardless of whether the school is rural or urban. This result becomes more relevant when paired with the finding for all rural and urban schools in Florida, which indicates urban schools use more OSS than rural schools. Contextualized as such, the results from the SWPBIS-only models would suggest that SWPBIS in urban schools must have a larger effect on OSS, and the models with all schools supported this finding. The full sample models’ results indicate that urban schools implementing SWPBIS are significantly reducing their use of OSS, while rural schools are not. We did not control OSS for prior years; therefore, the rural schools implementing SWPBIS may have used more OSS even before beginning to implement SWPBIS. This finding is intriguing given that Gage et al. (2018) found schools implementing SWPBIS with fidelity in Florida had reductions in OSS, specifically when disaggregated by race and disability status. It might be prudent to examine if there are differences in OSS rates that reflect the different demographical make-up of the rural settings compared with urban settings. It is also possible that as schools are trained in SWPBIS and map out their continuum of disciplinary practices, the suspension is included as an option that is less punitive than expulsion. If other alternatives are nonexistent and simply a more positive approach was adopted in the school setting, this might impact the number of suspensions used. Finally, schools that are trained in SWPBIS are asked to engage in data-based decision-making that promotes a system that collects and analyzes data. As schools reach higher rates of fidelity, ideally their outcomes should be more accurate. With that being said, it does not account for the reduction in the use of suspensions in urban schools, suggesting more research is needed to explore these differences in rural settings.

We believe that the last finding, indicating that rural schools implementing SWPBIS use more OSS than rural schools not implementing SWPBIS, suggests that rural schools may require more support to address OSS. This might include professional development that focuses on alternatives to suspension combined with additional training in preventive systems that would negate the use of suspensions as a disciplinary go-to. Gage and colleagues (2020) developed a series of recommendations for reducing suspensions, specifically when implementing SWPBIS. Among them are developing systematic alternatives for OSS and pairing any disciplinary practice with targeted intervention supports. For example, schools could emphasize more instructional approaches that include addressing the problem behavior in ISS, such as social skills or anger management programming, and then suggest that OSS be used for only the most extreme cases. In addition, one of the features of SWPBIS is that a continuum of approaches is used to address problem behavior, ideally starting with the least restrictive. It may be necessary to increase the training time for addressing this continuum in rural school settings to ensure positive instructional approaches are emphasized rather than more restrictive practices, such as the use of OSS. Given the findings from this study, training in SWPBIS for rural schools may benefit from intentionally emphasizing reductions in OSS as the desired outcome and systematically planning for explicit alternatives to suspensions.

Limitations

Although this study was designed to be statistically rigorous, some limitations necessitate discussion. First, all of the data were obtained from administrative records and cannot be independently confirmed. It is possible that schools under-reported their actual use of ISS and OSS, which would impact the results of the study. Second, we have no information about what behavior-related initiatives may have been implemented in the schools not implementing SWPBIS. Third, we do not have any information about the implementation of SWPBIS beyond universal (Tier 1) practices. OSS often results from more significant behaviors targeted by universal prevention practices. Therefore, schools implementing Tier 2 and Tier 3 interventions may actualize more significant impacts on OSS. Fourth, we included a series of covariates in all of our models, but there are other unmeasured or unavailable predictors. For example, having measures of OSS and ISS before schools began implementing SWPBIS would ensure that prior suspensions are not confounding the results. Last, we conducted this study in Florida. While the number of rural schools in this study was substantial, there are other states with more remote rural schools, such as Oregon or Montana. Therefore, the results may not generalize to other, more rural states.

Conclusion

To date, very little research has compared rural and urban schools’ implementation of SWPBIS. This study filled a clear gap in the literature, finding that overall, rural schools are just as likely to implement SWPBIS with fidelity as urban schools. However, some nuanced differences emerged, which should be explored with more refined research methods. We also found that there were no differences in OSS or ISS for urban and rural schools implementing SWPBIS. Yet, we also found that rural schools implementing SWPBIS use OSS more often than rural schools not using SWPBIS. As noted, urban schools implementing SWPBIS were using OSS significantly less than comparison urban schools. Therefore, with additional training, we believe that rural schools using SWPBIS can also decrease their OSS use.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Nicholas A. Gage

Heather Peshak George

Karen Robbie

References

Arnold

M. L.

Newman

J. H.

Gaddy

B. B.

Dean

C. B.

(2005). A Look at the condition of rural education research: Setting a direction for future research mid-continent research for education and learning. Journal of Research in Rural Education, 20(6), 1–25. http://www.umaine.edu/jrre/20-6.pdf

Bates

Maechler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Berry

A. B.

Petrin

R. A.

Gravelle

M. L.

Farmer

T. W.

(2012). Issues in special education teacher recruitment, retention, and professional development: Considerations in supporting rural teachers. Rural Special Education Quarterly, 30, 3–11. https://doi.org/10.1177/875687051103000402

Bohanon

Fenning

Carney

K. L.

Minnis-Kim

M. J.

Anderson-Harriss

Moroz

K. B.

Hicks

K. J.

Kasper

B. B.

Culos

Sailor

Pigott

T. D.

(2006). Schoolwide application of positive behavior support in an urban high school: A case study. Journal of Positive Behavior Interventions, 8(3), 131–145. https://doi.org/10.1177/10983007060080030201

Bradshaw

C. P.

Pas

E. T.

(2011). A statewide scale up of positive behavioral interventions and supports: A description of the development of systems of support and analysis of adoption and implementation. School Psychology Review, 40(4), 530–548. https://doi.org/10.1080/02796015.2011.12087528

Cavanaugh

Swan

(2015). Building SWPBIS capacity in rural schools through building-based coaching: Early findings from a district-based model. Rural Special Education Quarterly, 34(4), 29–39. https://doi.org/10.1177/875687051503400404

Center on Positive Behavioral Interventions and Supports. (2020). www.pbis.org

Cicchinelli

L. F.

Beesley

A. D.

(2017). Introduction: Current state of the science in rural education research. In Nugent

Kunz

Sheridan

Glover

Knoche

(Eds.), Rural Education Research in the United States (pp. 1–14). Springer. https://doi.org/10.1007/978-3-319-42940-3_1

Cohen

Kincaid

Childs

(2007). Measuring school-wide positive behavior support implementation: Development and validation of the Benchmarks of Quality (BoQ). Journal of Positive Behavior Interventions, 9(4), 203–213. https://doi.org/10.1177/10983007070090040301

10.

Curtis

Van Home

J. W.

Robertson

Karyonen

(2010). Outcomes of a school-wide positive behavioral support program. Professional School Counseling, 13(3), 159–164. https://eric.ed.gov/?id=EJ880426

11.

De Brey

Snyder

T.D.

Zhang

Dillow

S.A.

(2021). Digest of Education Statistics 2019 (NCES 2021-009). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC. https://files.eric.ed.gov/fulltext/ED611019.pdf

12.

Doss

K. M.

Krach

S. K.

Vickers

(2017). Alabama and Florida school psychologists shortage: Significance for training programs. Trainers’ Forum, 34(3), 34–42.

13.

Fitzgerald

C. B.

Geraci

L. M.

Swanson

(2014). Scaling ip in rural schools using positive behavioral interventions and supports. Rural Special Education Quarterly, 33, 18–22. https://doi.org/10.1177/875687051403300104

14.

Flannery

Frank

Kato

Doren

Fenning

(2013). Implementing schoolwide positive behavior support in high school settings: Analysis of eight high schools. The High School Journal, 96, 267–282. https://doi.org/10.1353/hsj.2013.0015

15.

Florida Positive Behavioral Interventions and Support Project. (2020). Florida PBIS MTSS: 2018-2019 annual report. https://www.livebinders.com/play/play?id=2237091

16.

Gage

N. A.

Beahm

Kaplan

MacSuga-Gage

A. S.

Lee

(2020). Using positive behavioral interventions and supports to reduce school suspensions. Beyond Behavior, 29(3), 132–140. https://doi.org/10.1177/1074295620950611

17.

Gage

N. A.

Grasley-Boy

Peshak George

Childs

Kincaid

(2019). A quasi-experimental design analysis of the effects of school-wide positive behavior interventions and supports on discipline in Florida. Journal of Positive Behavior Interventions, 21(1), 50–61. https://doi.org/10.1177/1098300718768208

18.

Gage

N. A.

Lee

Grasley-Boy

George

H. P.

(2018). The impact of School-Wide Positive Behavior Interventions and Supports on school suspensions: A statewide quasi-experimental analysis. Journal of Positive Behavior Interventions, 20, 217–226. https://doi.org/10.1177%2F1098300718768204

19.

Geverdt

(2015). Education Demographic and Geographic Estimates Program (EDGE): Locale boundaries user’s manual (NCES 2016-012). U.S. Department of Education. National Center for Education Statistics. https://nces.ed.gov/programs/edge/docs/NCES_LOCALE_USERSMANUAL_2016012.pdf

20.

Huang

F. L.

(2018). Multilevel modeling myths. School Psychology Quarterly, 33(3), 492–499. https://doi.org/10.1037/spq0000272

21.

Jones

Caravaca

Cizek

Horner

R. H.

Vincent

C. G.

(2006). Culturally responsive schoolwide positive behavior support: A case study in one school with a high proportion of Native American students. Multiple Voices for Ethnically Diverse Exceptional Learners, 9(1), 108–119. https://doi.org/10.5555/muvo.9.1.0311x7477113q741

22.

Kartub

D. T.

Taylor-Greene

March

R. E.

Horner

R. H.

(2000). Reducing hallway noise: A systems approach. Journal of Positive Behavior Interventions, 2(3), 179. https://doi.org/10.1177/109830070000200307

23.

Kim

McIntosh

Mercer

S. H.

Nese

R. N.

(2018). Longitudinal associations between SWPBIS fidelity of implementation and behavior and academic outcomes. Behavioral Disorders, 43(3), 357–369. https://doi.org/10.1177/0198742917747589

24.

Lane

K. L.

Kalberg

J. R.

Bruhn

A. L.

Mahoney

M. E.

Driscoll

S. A.

(2008). Primary prevention programs at the elementary level: Issues of treatment integrity, systematic screening, and reinforcement. Education & Treatment of Children, 31(4), 465–494. https://doi.org/10.1353/etc.0.0033

25.

Lee

Gage

N. A.

(2020). Updating and expanding systematic reviews and meta-analyses on the effects of School-Wide Positive Behavior Interventions and Supports. Psychology in the Schools, 57, 783–804. https://doi.org/10.1002/pits.22336

26.

Mathews

McIntosh

Frank

J. L.

May

S. L.

(2014). Critical features predicting sustained implementation of School-Wide Positive Behavioral Interventions and Supports. Journal of Positive Behavior Interventions, 16(3), 168–178. https://doi.org/10.1177/1098300713484065

27.

McCrary

Lechtenberger

Wang

(2012). The effect of Schoolwide Positive Behavioral Supports on children in impoverished rural community schools. Preventing School Failure, 56(1), 1–7. https://doi.org/10.1080/1045988X.2010.548417

28.

McDaniel

Bruhn

George

H. P.

(2020). Establishing the literature base of SWPBIS research in schools with underserved students. Preventing School Failure, 64(4), 290–300. https://doi.org/10.1080/1045988X.2020.1762158

29.

McDaniel

Sunyoung

Kwon

Choi

(2018). Stakeholder perceptions of contextual factors related to PBIS implementation in high needs schools. Journal of Children and Poverty, 24, 109–122. https://doi.org/10.1080/10796126.2018.1518777

30.

Menendez

A. L.

Payne

L. D.

Mayton

M. R.

(2008). The implementation of positive behavioral support in an elementary school: Processes, procedures, and outcomes. The Alberta Journal of Educational Research, 54, 448–462. https://doi.org/10.11575/ajer.v54i4.55250

31.

Nese

R. N. T.

Nese

J. F. T.

McIntosh

Mercer

S. H.

Kittelman

(2019). Predicting latency of reaching adequate implementation of tier I Schoolwide Positive Behavioral Interventions and Supports. Journal of Positive Behavior Interventions, 21(2), 106–116. https://doi.org/10.1177/1098300718783755

32.

Oswald

Safran

Johanson

(2005). Preventing trouble: Making schools safer places using positive behavior supports. Education and Treatment of Children, 28, 265–278. https://www-jstor-org-443.web.bisu.edu.cn/stable/42899849

33.

Pas

E. T.

Ryoo

J. H.

Musci

R. J.

Bradshaw

C. P.

(2019). A state-wide quasi-experimental effectiveness study of the scale-up of School-Wide Positive Behavioral Interventions and Supports. Journal of School Psychology, 73, 41–55. https://doi.org/10.1016/j.jsp.2019.03.001

34.

Preston

J. P.

Jakubiec

B. A.

Kooymans

(2013). Common challenges faced by rural principals: A review of the literature. The Rural Educator, 35(1), 1–12. https://doi.org/10.35608/ruraled.v35i1.355

35.

Rude

Miller

K. J.

(2018). Policy challenges and opportunities for rural special education. Rural Special Education Quarterly, 37(1), 21–29. https://doi.org/10.1177/8756870517748662

36.

Schmidt

Gage

A. M.

Gage

Cox

McLeskey

(2015). Bringing the field to the supervisor: Innovation in distance supervision for field-based experiences using mobile technologies. Rural Special Education Quarterly, 34(1), 37–43. https://doi.org/10.1177/875687051503400108

37.

Steed

E. A.

Pomerleau

Muscott

Rohde

(2013). Program-wide positive behavioral interventions and supports in rural preschools. Rural Special Education Quarterly, 32(1), 38–46. https://doi.org/10.1177/875687051303200106

38.

Sugai

Horner

R. H.

(2002). The evolution of discipline practices: School-wide positive behavior supports. Child and Family Behavior Therapy, 24(1-2), 23–50. https://doi.org/10.1300/J019v24n01_03