Validation of a Measure of District Systems Implementation of Positive Behavioral Interventions and Supports

Abstract

District leadership teams perform key roles in building the systems to support schools in the implementation of Positive Behavioral Interventions and Supports (PBIS). However, there is a lack of measures for assessment and progress monitoring specific to district PBIS systems. To address this gap, we evaluated the validity of a measure of implementation of district PBIS systems, the District Systems Fidelity Inventory (DSFI). Using 183 school districts and 760 schools implementing PBIS, we found the DSFI to have good evidence of structural validity for measuring nine aspects of district systems (Leadership Teaming, Stakeholder Engagement, Funding and Alignment, Policy, Workforce Capacity, Training, Coaching, Evaluation, and Local Implementation Demonstrations). We also found DSFI subscales to be moderately related to school-level PBIS implementation fidelity, providing evidence of convergent validity. We describe how leadership teams can use the DSFI to improve PBIS implementation and student outcomes.

Keywords

Positive Behavioral Interventions and Supports district systems implementation fidelity

For the past 20 years, Positive Behavioral Interventions and Supports (PBIS) has been among the most scaled-up innovations in U.S. schools (Sugai & Horner, 2019). A key reason for the successful adoption of PBIS is the focus on the implementation of organizational systems to support the initial and sustained use of evidence-based practices (EBPs) for students with and without disabilities (George et al., 2018; McIntosh et al., 2018). When district and school leaders commit to implementing PBIS, they are committing to implementing not a specific EBP, but rather a framework comprised of organizational systems to support the implementation of an array of EBPs to improve student behavior and academic outcomes across multiple levels of intensity: Tier 1, universal supports for all students; Tier 2, supports for moderate student needs; and Tier 3, supports for the complex and intensive needs of students. To ensure organizational systems at each tier are being implemented as designed, district and school leaders are strongly encouraged to use validated tools to regularly measure the fidelity. These organizational systems include establishing district and school leadership teams to oversee implementation of EBPs, using professional development strategies (e.g., training and coaching) to implement EBPs with fidelity and investing in data collection and data decision systems to monitor implementation of EBPs and organizational systems (George et al., 2018; McIntosh et al., 2013).

Empirical Support for PBIS

When PBIS is implemented with fidelity, research has shown effects on numerous academic and behavior outcomes for students with and without disabilities receiving support at Tiers 1, 2, and 3 (K. Algozzine & Algozzine, 2007; Blair et al., 2020; Gage et al., 2018; Grasley-Boy et al., 2022; Lee & Gage, 2020; Walker et al., 2018). For example, Lee and Gage (2020) conducted a meta-analysis examining the effect of Tier 1 PBIS on student outcomes. Of the 29 studies included in their review (seven randomized controlled trials and 22 quasi-experimental group designs), the authors found significant decreases in exclusionary discipline (i.e., office discipline referrals and suspensions) and increases in student academic outcomes (i.e., reading and math achievement), with the majority of studies reporting small effect sizes. Blair et al. (2020) recently conducted a meta-analysis of Tier 2 EBPs implemented within PBIS and utilized single-case designs. Of the 26 studies, significant and small-to-medium effects were found across multiple Tier 2 EBPs (i.e., Check-In/Check-Out, self-monitoring, and group contingencies) for decreasing problem behaviors, improving appropriate behaviors, and improving academic engagement. Finally, Walker et al. (2018) conducted a meta-analysis with 27 studies for Tier 3 EBPs implemented in inclusive settings in schools and included students with disabilities utilizing single-case designs. The authors found moderate-to-large intervention effects for decreasing problem behaviors and increasing appropriate behaviors using nonoverlap single-case design approaches to calculate intervention effect sizes.

School and Practice Variables Predicting Implementation of PBIS

Much of what is known about factors facilitating implementation of PBIS systems (e.g., training, coaching, data collection and use) comes from longitudinal research examining the relation between-school (context) and practice (intervention specific) variables on PBIS implementation fidelity (Kittelman et al., 2022; Mathews et al., 2014; Nese et al., 2018; Schaper et al., 2016). For example, using a sample of 353 schools implementing Tier 1 for 1 to 4 years, Schaper et al. (2016) found rural schools to be more likely to have higher Tier 1 implementation fidelity scores and larger schools and high schools to be more likely to have lower Tier 1 implementation fidelity scores at the beginning of the school year. In addition, schools with a higher proportion of students eligible for free and reduced-price lunch had less within-year growth in Tier 1 implementation fidelity scores. Relatedly, using a sample of 708 schools, Nese et al. (2018) found elementary schools (compared to middle and high schools), non-Title 1 schools, and suburban schools (compared to city schools) more likely to reach adequate Tier 1 implementation fidelity after initial training.

Perhaps more important for school improvement, longitudinal implementation studies have also identified practice variables predictive of PBIS implementation fidelity (Kittelman et al., 2022; Mathews et al., 2014; McIntosh et al., 2018). For example, using a sample of 860 schools, McIntosh et al. (2018) found teams who collected, summarized, and shared data in Year 1 was predictive of higher Tier 1 implementation fidelity scores in Year 3. Finally, using a sample of 776 schools, Kittelman et al. (2022) demonstrated schools with higher Tier 1 implementation fidelity scores the year before implementing Tiers 2 and 3 had higher implementation fidelity scores in the first year of Tier 2 and 3 implementation.

Limited Knowledge of District Variables Predictive of PBIS Implementation Fidelity

Compared to school variables, less is known about district variables predictive of PBIS implementation fidelity (Kittelman et al., 2019; McIntosh et al., 2018). Kittelman et al. (2019) used a sequential cohort of 552 districts to identify predictors of the rate of district adoption of PBIS. The proportion of students who received free or reduced-price lunch (FRL) was a positive predictor and district size (number of schools in the district) and districts located in cities and towns were negative predictors of the rate of district adoption. McIntosh et al. (2018) also found the proportion of schools in the district implementing PBIS predicted higher Tier 1 implementation fidelity scores for schools 4 years later and the proportion of schools in the district newly implementing PBIS predicted stronger Tier 1 implementation fidelity scores in schools 3 years later.

Unfortunately, a major limitation of the current PBIS implementation research is that most district-level variables found to predict school implementation (e.g., number of schools implementing PBIS, district size) are non-malleable (e.g., Kittelman et al., 2019; McIntosh et al., 2018). As such, this information does not provide district leadership teams with practical guidance on how to improve school implementation fidelity or better support students who have multiple needs (e.g., speech or language impairment, specific learning disabilities, and social-behavioral needs) through stronger tiered systems at the district level. This research gap is unfortunate because district leadership teams can have a significant positive effect in supporting school leadership teams to support students with or at risk for disabilities across all three tiers. For example, district leadership teams are often responsible for identifying and providing school leadership teams with positive model exemplars during the implementation of PBIS at all three tiers (also referred to as initial demonstrations; Horner & Sugai, 2015). At Tier 1, district leadership teams can provide school leadership teams with local exemplars to ensure Tier 1 EBPs are effective and inclusive for students with severe disabilities (e.g., ensure schoolwide expectations are taught and modeled for all students, have adapted schoolwide acknowledgment systems for students with serve disabilities (e.g., adjusted performance criteria; Walker et al., 2022). In addition, due to the limited resources for PBIS implementation across tiers, many school leadership teams rely on initial and ongoing professional development from district trainers and coaches (George et al., 2018). When supporting students with or at risk for disabilities, district leadership team members often provide professional development to school Tier 2 and 3 teams to ensure EBPs are correctly matched to the function of student’s unwanted behavior(s) and implemented with fidelity. Another responsibility of district leadership teams is to identify or develop data systems (e.g., screening systems and evaluation systems) to ensure district and school leadership teams can access and use school and student-level data for decision-making and reporting (George et al., 2018). For students with or at risk for disabilities, these data systems are crucial for (a) ensuring students are being adequately nominated for and receive support, (b) disaggregating data to assess outcomes by student group (e.g., by disability) and monitoring student data during intervention, and (c) reporting on student behavior and academic outcome data to educational leaders and local communities.

Identifying malleable district-level facilitators is important because there are substantial between-district differences in school PBIS implementation fidelity scores (McIntosh et al., 2016). For example, McIntosh et al. (2016) examined the extent to which there were between-school, between-district, and between-state differences in Tier 1 PBIS implementation. Using a sequential cohort of over 3,000 schools, the authors found nearly as much variance in PBIS implementation scores at the district level (17%–34%, depending on year of PBIS implementation) compared to the school level (18%–39%). One reason for the limited identification of district variables predictive of PBIS implementation fidelity is because there is a lack of valid measures for district leadership teams to assess implementation of district systems facilitating PBIS implementation fidelity in schools.

A Lack of District Measures of PBIS Implementation

PBIS implementation fidelity measures used in the field primarily focus on measuring aspects of school PBIS implementation (primarily school and practice-level factors; R. F. Algozzine et al., 2014; Kincaid et al., 2010). For example, the Schoolwide (SW) PBIS Tiered Fidelity Inventory (TFI; R. F. Algozzine et al., 2014), the most commonly used school-level PBIS implementation fidelity measure with strong evidence of validity (Mercer et al., 2017), includes subscales for measuring school teams, implementation, interventions, evaluation, resources, and support plans. Another school-level PBIS implementation fidelity measure, the Schoolwide Evaluation Tool (SET; Sugai et al., 2005) has a district-level support subscale but includes only two items (i.e., district funding for PBIS, identifying a district, or state facilitator/technical assistance coach). In addition, Ward et al. (2022) conducted a study examining the development and validation of the District Capacity Assessment (DCA; Ward et al., 2015), a measure designed to assess district implementation capacity of multiple EBPs (e.g., multi-tiered systems of support, early literacy practices) and guide leadership teams in action planning and improvement. Although not specific to PBIS, the measure is designed to assess and support implementation of capacity elements important for all EBPs. The authors found the measure to have a three-factor structure for assessing scale scores across district leadership, competency, and data systems for decision-making.

Although there is a lack of valid measures for assessing implementation of district PBIS systems, a number of qualitative and descriptive case studies have identified malleable district variables perceived to be strongly associated with PBIS implementation in schools (George et al., 2018; Kittelman et al., 2020; Netzel & Eber, 2003). As an example, George et al. (2018) conducted an exploratory study to identify malleable district variables perceived to facilitate PBIS implementation in high implementing districts (districts with high proportions of schools implementing with fidelity and producing positive student outcomes). Through rigorous semi-structured interviews with district personnel, the authors identified eight major themes (and 23 subthemes) perceived to facilitate PBIS implementation in schools. These included district coordinators, coaches, district teaming, district team activities (visibility and funding), leadership buy-in and support, district data infrastructure, direct support to schools, and communication (George et al., 2018). Although George et al. (2018) provided novel contributions and identified malleable district systems, the authors noted additional research is needed to validate these systems as predictive of PBIS implementation and identify other variables facilitating PBIS implementation. These district-level variables identified by George et al. (2018) are unique variables perceived to facilitate PBIS implementation in schools and are largely not measured using school-level PBIS implementation fidelity measures. By having valid measures of implementation fidelity at both the district and school levels, teams can assess and improve upon the implementation of organizational systems implemented at different levels to facilitate stronger implementation of PBIS in schools. In addition to improving overall implementation, teams at both levels can identify and target specific organization systems for improvement (e.g., training, coaching on implementing Tier 3 EBPs) to have the most significant positive effects on improving outcomes for students with and at risk for disabilities.

The District Systems Fidelity Inventory

Designed to facilitate district action planning, the District Systems Fidelity Inventory (DSFI; Center on Positive Behavioral Interventions and Supports, 2020) guides district leadership teams by identifying strengths, needs, and priorities for change. The instrument includes 56 items, related to PBIS implementation at all three tiers, organized into nine categories: Leadership Teaming, Stakeholder Engagement, Funding and Alignment, Policy, Workforce Capacity, Training, Coaching, Evaluation, and Local Implementation Demonstrations. District leadership teams can use the DSFI to determine district-level capacity to initiate PBIS implementation, progress monitor ongoing implementation efforts, and assess overall implementation fidelity. The tool includes references to data sources and a rubric to assist in scoring each item.

The DSFI evolved through an iterative development process, with the use of a preliminary and pilot version. Researchers from the University of Missouri-Columbia (MU) Center for Schoolwide Positive Behavior Support used the TFI and the PBIS Implementer’s Blueprint Self-Assessment Tool (Center on Positive Behavioral Interventions and Supports, 2015) as references to align DSFI subscales and items with those measures focused on school PBIS implementation. Next, an eight-person workgroup of researchers and technical assistance providers, each with over a decade of experience in PBIS implementation and technical assistance, reviewed initial DSFI items and suggested content edits and recommendations. Using this feedback, the authors made changes to the wording of items, data sources recommended for district team members to review, and the scoring rubric resulting in the first working version of the instrument (preliminary version). Next, the DSFI was reviewed by co-directors and implementing partners from the Center on PBIS. Based on the feedback, minor additional revisions were made to the DSFI and included removing an item in one subscale (Coaching), adding an item in another subscale (Workforce Capacity), and revising the language of some items and rubric statements which resulted in the pilot version of the DSFI.

Study Aims

The objective of this study was to evaluate the psychometric properties of the DSFI as a measure of implementation of district PBIS systems (e.g., professional development systems, leadership systems, and evaluation systems). This research is important because it would provide PBIS leadership teams with a validated measure to assess implementation of district systems and could be used by researchers to evaluate the effects of malleable district systems (e.g., training and local implementation demonstrations) on school PBIS implementation fidelity. The first aim was to evaluate whether the DSFI exhibited evidence of structural validity. The second aim was to evaluate whether the DSFI exhibited evidence of convergent validity with the TFI. As both measures are designed to facilitate PBIS implementation in schools (one at the district level and one at the school level), we hypothesized both measures should be significantly correlated.

Method

Participants and Settings

The samples included 183 U.S. school districts that completed the DSFI during the 2018–2019 and/or 2019–2020 school year and 760 schools within those districts that assessed PBIS implementation fidelity using the TFI. District leadership team members from 147 of the districts completed the DSFI during the 2018–2019 school year, 141 during the 2019–2020 school year, and 105 of the 183 (57.4%) completed the DSFI both years. The 183 districts were located in 22 states: Washington (n = 22, 12%), California (n = 21, 11.5%), Pennsylvania (n = 19, 10.4%), Georgia (n = 17, 9.3%), Missouri (n = 16, 8.7%), Michigan (n = 14, 7.7%), Illinois (n = 14, 7.7%), Florida (n = 13, 7.1%), Wisconsin (n = 12, 6.6%), Nevada (n = 8, 4.4%), Alabama (n = 5, 2.7%), Oregon (n = 4, 2.2%), Arizona (n = 3, 1.6%), Idaho (n = 3, 1.6%), Virginia (n = 2, 1.1%), Indiana (n = 2, 1.1%), Iowa (n = 2, 1.1%), New York (n = 2, 1.1%). Maryland (n = 2, 1.1%), Massachusetts (n = 1, 0.5%), Nebraska (n = 1, 0.5%), and North Carolina (n = 1, 0.5%). Table 1 includes characteristics for the 2018–2019 school year disaggregated by districts that completed the DSFI in 2018–2019, 2019–2020, and the total 183 districts. Prior to determining the final sample described above, we removed one DSFI completion from 2018 to 2019 because the DSFI was completed for an alternative education program, and we removed one DSFI completed in 2019–2020 because the DSFI was completed for a specialized services program within a larger district that completed the DSFI in 2018–2019 and 2019–2020.

Table 1.

Characteristics of the 183 Public School Districts That Completed the DSFI.

Characteristic	2018–19	2019–20	Total
Districts	147	141	183
States	20	22	22
Public schools, M (SD)	24.9 (44.5)	28.7 (48.8)	25.4 (43.9)
Student enrollment, M (SD)	15,428 (33,456)	18,571 (37,483)	16,110 (33,469)
% Students IEP, M (SD)	14.8 (6.1)	14.6 (3.5)	14.9 (5.8)
% Students LEP/ELL, M (SD)	8.7 (9.9)	8.9 (9.6)	8.7 (9.8)
% Non-White students, M (SD)	46.1 (27.9)	48.9 (27.9)	46.8 (27.9)
Student/teacher ratio, M (SD)	16.6 (3.5)	17.2 (3.9)	16.9 (3.8)
Districts by locale
% Cities	23.8	27.0	25.1
% Suburbs	37.4	37.6	37.2
% Towns	19.0	17.0	17.5
% Rural	19.7	18.4	20.2

Note. Missing National Center for Educational Statistics (NCES) district data ranged from 1% to 10.2%. DSFI = District Systems Fidelity Inventory; IEP = Individualized Education Program; LEP = limited English proficient; ELL = English language learner.

School leadership teams in the 760 schools that completed the TFI during the 2018–2019, 2019–2020, or both years were located in 18 states.Table 2 includes additional school characteristics from the 2018–2019 school year disaggregated by schools that completed a Tier 1, Tier 2, and Tier 3 scale of the TFI and all total 760 schools. Most schools were elementary schools (69.5%) and located in suburban areas (42.6%). On average, 73.8% of the schools were identified as Title 1% and 58.9% of students were receiving FRL.

Table 2.

Characteristics of the 760 Schools That Completed the TFI.

Characteristic	Tier 1	Tier 2	Tier 3	Total
Schools	610	575	341	760
States	17	17	16	18
Student enrollment, M (SD)	564 (294)	594 (325)	595 (332)	588 (328)
% Students FRL, M (SD)	58.6 (25.9)	58.9 (24.9)	59.9 (22.6)	58.9 (24.7)
% Non-White students, M (SD)	52.8 (27.8)	53.6 (27.1)	53.2 (25.8)	51.1 (27.8)
School type
% Elementary	71.1	72.2	70.7	69.5
% Middle	18.5	18.3	18.5	18.8
% High	8.9	7.3	7.9	9.7
% Other	1.5	2.3	2.9	2.0
Schools by locale
% Cities	29.7	27.1	26.9	27.6
% Suburbs	41.0	46.3	48.5	42.6
% Towns	10.7	8.9	8.3	10.5
% Rural	18.7	17.3	16.3	18.8
% Title 1	72.6	75.4	78.2	73.8

Note. Missing National Center for Educational Statistics (NCES) school data ranged from less than 1% to 6.2%. TFI = SWPBIS Tiered Fidelity Inventory; FRL = free or reduced-price lunch.

Measures

District implementation fidelity

District PBIS systems implementation data were obtained using the DSFI (Center on Positive Behavioral Interventions and Supports, 2020). The DSFI is freely available and can be found on the Center on PBIS website (https://www.pbis.org/resource/pbis-district-systems-fidelity-inventory-dsfi-pilot-version-v0-1). The measure includes 56 items divided across nine subscales (Leadership Teaming = 9, Stakeholder Engagement = 3, Funding and Alignment = 8, Policy = 5, Workforce Capacity = 3, Training = 6, Coaching = 8, Evaluation = 11, Local Implementation Demonstrations = 3). Each question includes a rating scale with three response options (0, 1, or 2) specific to each question. For example, one Workforce Capacity subscale item is “Job descriptions: PBIS implementation activities (e.g., training, coaching) are embedded into job descriptions and dedicated time is allocated.” The response options include

0 = No PBIS implementation activities are embedded in job descriptions, 1 = PBIS implementation activities are assumed as part of the job descriptions, but are not formally included, 2 = PBIS implementation activities are embedded into job descriptions and time is allotted for them.

School implementation fidelity

PBIS school implementation fidelity data were obtained using the TFI. The TFI includes 45 items divided across three scales (Tier 1 scale = 15 items, Tier 2 scale = 13 items, Tier 3 scale = 17 items). The psychometric properties of the TFI have been evaluated across multiple studies (Massar et al., 2017; McIntosh et al., 2017; Mercer et al., 2017). The TFI was found to have strong internal consistency across the Tier 1 (α = .87), Tier 2 (α = .96), and Tier 3 scales (α = .98; McIntosh et al., 2017). The TFI has a strong factor structure as a three-factor model (Comparative Fit Index [CFI] = .993, Tucker–Lewis index [TLI] = .993, root mean square of error approximation [RMSEA] = .038; Massar et al., 2017). In addition, when compared to other Tier 1 implementation fidelity measures (e.g., SET, Team Implementation Checklist [TIC]; Sugai et al., 2001), the Tier 1 scale of the TFI has been found to have moderate-to-high convergent validity (r values = .59–.96; Mercer et al., 2017).

District and school demographic data

District and school demographic characteristics (see Tables 1 and 2) were obtained from the National Center for Educational Statistics (NCES) database for the 2018–2019 school year. Missing NCES data across district characteristics and district samples ranged from 1% (student enrollment and number of public schools across total 183 districts) to 10.2% (% of students limited English proficient [LEP]/English language learner [ELL] for 147 districts that completed DSFI in 2018–2019). Missing NCES data across school characteristics ranged from less than 1% (student enrollment, school locale) to 6.2% (% of schools identified as Title 1 for 321 schools that complete a Tier 3 TFI).

Procedure

Upon approval from the lead author’s institutional review board, DSFI and TFI data were obtained through two databases. District DSFI scores for 2018–19 (preliminary version) and 2019–2020 (pilot version) were extracted from an online survey administered to a national sample of district leadership teams participating in a larger, 3-year longitudinal study funded by the Institute of Education Sciences (IES) focused on identifying factors facilitating initial and sustained implementation of Tier 2 and 3 behavior systems in schools (Kittelman et al., 2021). TFI implementation fidelity scores were extracted from a free, web-based application called PBIS Assessment (www.pbisapps.org) maintained by the Educational and Community Supports, a research unit at the University of Oregon. School teams use the web-based application to enter their school implementation fidelity scores for annual evaluation and/or more frequent progress monitoring. For this study, if school teams competed more than one TFI per year in PBIS Assessment, we used their last assessment of each tier that was completed with the guidance of an external PBIS coach.

Data Analysis

For study aim 1, to evaluate the structural validity of the DSFI, we first conducted an exploratory factor analysis (EFA) based on DSFIs completed in the 2018–2019 school year (n = 147; preliminary version) and then conducted confirmatory factor analyses (CFAs) of competing models, based on the EFA results, using DSFIs completed in the 2019–2020 school year (n = 141; pilot version). To account for the ordered, categorical response format of the DSFI items, we analyzed the polychoric correlation matrix in the EFA, with parallel analysis used to determine the number of factors (Timmerman & Lorenzo-Seva, 2011). Consistent with typical parallel analysis procedures, we considered whether the amount of explained common factor variance for real data exceeded (a) the mean amount for randomly permuted data and (b) the amount at the 95th percentile for randomly permuted data. All CFA models were specified with DSFI items as ordinal indicators, with the following values indicating good model fit to the data (Hu & Bentler, 1999): CFI > .95, RMSEA < .06, and the standardized root mean squared residual (SRMR) < .08. We evaluated the reliability (internal consistency) of DSFI scores based on ordinal α (Zumbo et al., 2007).

For study aim 2, we fit a series of three-level models to evaluate the correlations between DSFI scores and implementation fidelity (convergent validity). The districts’ mean scores across DSFI items (by subscale and overall) and the schools’ mean scores across TFI items (by tier) were analyzed. Separate analyses were conducted by TFI tier, with each school’s TFI scores for 2018–2019 and 2019–2020 included in analyses. Random intercepts for fidelity scores were included at the school and district levels. At level 1 (time), year was entered as an effects-coded fixed effect (2018–2019 = −0.5; 2019–2020 = +0.5) to evaluate differences in fidelity scores by year. At level 3 (district), the correlations between DSFI scores and district-level fidelity were estimated. The earliest available DSFI administration for each district (2018–2019 or 2019–2020) was included in these analyses. Schools were included if they had a completed TFI at that tier in either 2018–2019 or 2019–2020. The Tier 1 analyses were based on 610 schools in 91 districts, with 575 schools in 94 districts included in Tier 2 analyses, and 341 schools in 71 districts for Tier 3 (see Table 2).

The EFA and parallel analysis were conducted using the FACTOR program (Lorenzo-Seva & Ferrando, 2006), and the CFAs and correlational models were fit using Mplus 8.6 (Muthén & Muthén, 2017). Missing TFI data (for example, if a school completed a Tier 1 TFI in 2018–2019 but not 2019–2020) were handled using multiple imputation in Mplus, with results aggregated across the 1,000 multiply imputed data sets for each tier. The imputation models included district and school demographic characteristics, DSFI scores, and TFI scores by year (separate models by tier). Of the 610 schools that completed a Tier 1 TFI, 490 (80.33%) completed it in 2018–2019 and 408 (66.89%) completed it in 2019–2020. Of the 575 schools that completed a Tier 2 TFI, 456 (79.30%) completed it in 2018–2019 and 343 (59.65%) completed it in 2019–2020. Of the 341 schools that completed a Tier 3 TFI, 265 (77.71%) completed it in 2018–2019 and 206 (60.41%) completed it in 2019–2020.

Results

Factor Structure

Means and standard deviation by year for DSFI items are presented in the online supplemental materials (Table S1). Results of the EFA parallel analysis with the 2018–2019 DSFI data indicated either one-factor or two-factor solutions are optimal, depending on the criterion used. When the number of factors is based on exceeding the mean amount of explained common factor variance in random data, two factors would be indicated. By contrast, when the criterion is exceeding the amount at the 95th percentile for random data, only one factor would be indicated. The very large eigenvalue for the first factor relative to the second factor (20.80 vs. 2.05) suggests the DSFI may be unidimensional enough to specify it as such in analyses (Bonifay et al., 2015); however, the two-factor EFA solution is theoretically meaningful. Items with larger loadings on the first factor, Executive Functions, were mostly from the following five DSFI subscales: Leadership Teaming, Stakeholder Engagement, Funding and Alignment, Policy, and Workforce Capacity. Items with larger loadings on the second factor, implementation functions, were mostly from the following four DSFI subscales: Training, Coaching, Evaluation, and Local Implementation Demonstrations. In the two-factor EFA model, these two factors, Executive Functions and Implementation Functions, were highly correlated (r = .59).

Based on the EFA results, we evaluated the relative fit of three competing factor structures for the DSFI in CFAs: (a) a unidimensional model, (b) a two-factor model with items loading on either Executive Functions or Implementation Functions, and (c) a second-order factor model, with items loading on factors representing the nine DSFI subscales and the nine subscale factors loading on the Executive Functions or Implementation Functions factors (see Figure 1).

Figure 1.

Second-order confirmatory factor analysis model for the District Systems Fidelity Inventory.

Results indicated the second-order model fit better than both the two-factor model, χ²(9) = 85.65, p < .001, and the one-factor model, χ²(10) = 118.03, p < .001. Fit for the second-order factor model was good on all indicators other than the model chi-square, with χ²(1474) = 1691.56, p < .001; CFI = .96; RMSEA = .04, 90% CI [.03, .05]; SRMR = .08. For these reasons, we identified the second-order factor model as best representing the factor structure of the DSFI, but also challenging to use in research and practice due to its complexity. Further adding complexity, most substantive research questions involving the DSFI will involve multilevel analyses, for example, district-level characteristics such as DSFI scores as predictors of school- and student-level outcomes.

For these reasons, we believe the following scoring options represent key aspects of the EFA and CFA results while also being more feasible for use in research and practice: (a) an Overall score based on all DSFI item scores, (b) Executive Functions and Implementation Functions scores based on the items aligning with the second-order factors (as in Figure 1), and (c) scores based on the nine DSFI subscales (see Discussion). Reliability was good for each of these options, with ordinal α = .97 for the Overall score; ordinal α = .95 and .96 for the Executive Functions and Implementation Functions scores; and ordinal α = .90, .80, .87, .78, .87, .87, .89, .92, and .82 for the nine DSFI subscale scores.

Correlations With Implementation Fidelity

The correlations of DSFI scores in three configurations (Overall score, Executive and Implementation Functions scores, and nine subscale scores) with TFI scores at Tiers 1–3 are in Table 3. Means and standard deviations for DSFI and TFI scores are presented in the online supplemental materials (Table S2). At Tier 1, 18.4% of the variance in TFI scores was between school districts, with 29.0% and 23.3% of the variance in Tier 2 and Tier 3 TFI scores between school districts, respectively. There were no statistically significant differences between 2018–2019 and 2019–2020 TFI scores at Tier 1 (β = −.08, p = .12), Tier 2 (β = .05, p = .31), or Tier 3 (β = .05, p = .35).

Table 3.

Correlations of DSFI and TFI Implementation Fidelity Scores.

Model	DSFI score	Tier 1		Tier 2		Tier 3
Model	DSFI score	r	SE	r	SE	r	SE
1 Factor	Overall	.49**	.15	.34*	.13	.32*	.15
2 Factors	Executive functions	.50***	.14	.29*	.14	.27	.15
2 Factors	Implementation functions	.44**	.15	.36*	.12	.32*	.14
9 Subscales	Leadership teaming	.28	.15	.07	.14	.10	.16
	Stakeholder engagement	.38*	.15	.18	.14	.07	.17
	Funding and alignment	.57***	.12	.42**	.13	.36*	.14
	Policy	.48***	.11	.34**	.13	.24	.15
	Workforce capacity	.32*	.13	.25	.14	.11	.16
	Training	.35*	.14	.29*	.13	.29*	.14
	Coaching	.31*	.15	.41***	.11	.23	.15
	Evaluation	.42**	.15	.32**	.12	.26	.15
	Local implementation demonstrations	.34*	.13	.33**	.13	.40**	.14

Note. Sample sizes were the following: Tier 1 (610 schools in 91 districts), Tier 2 (575 schools in 94 districts), and Tier 3 (341 schools in 71 districts). DSFI = District Systems Fidelity Inventory; TFI = SWPBIS Tiered Fidelity Inventory.

p < .05. **p < .01. ***p < .001.

As detailed in Table 3, the Overall score had statistically significant, positive correlations with TFI scores at all three tiers, whereas the Executive and Implementation Functions factor scores and the nine DSFI subscales had varied relations with fidelity by tier. Although the Implementation Functions score was positively related to fidelity at all tiers, the Executive Functions score was only related to fidelity at Tiers 1 and 2. For the DSFI subscale scores, eight of the nine were statistically significantly correlated with Tier 1 fidelity (Stakeholder Engagement, Funding and Alignment, Policy, Workforce Capacity, Training, Coaching, Evaluation, and Local Implementation Demonstrations), with six of nine positively correlated with Tier 2 fidelity (Funding and Alignment, Policy, Training, Coaching, Evaluation, and Local Implementation Demonstrations) and three of nine positively correlated with Tier 3 fidelity (Funding and Alignment, Training, Local Implementation Demonstrations). The Funding and Alignment, Training, and Local Implementation Demonstrations subscale scores were correlated with fidelity at all three tiers.

Discussion

Although district leadership teams play critical roles in building district capacity and supporting schools to implement PBIS (George et al., 2018; Netzel & Eber, 2003), district teams lack valid and reliable instruments to guide implementation of district systems. To address this, we evaluated the psychometric properties of the DSFI. Through the structural validity analyses, we identified support for several scoring configurations (Overall score, Executive Function and Implementation Function scores, and nine subscale scores) that may have utility for both research and practice, with evidence of good reliability for the DSFI subscale scores likely to be used for evaluation and action planning. For research studies, the Overall and Executive Functions and Implementation Functions scores are likely most useful to assess stability in scores over time, compare these scores across school districts, and identify potential state, district, and school factors predictive of implementation of district systems. For practice, district leadership teams may want to use the Overall, Executive Functions, and Implementation Functions scores, and subscale scores to assess district implementation fidelity and identify areas for improvement. District leadership teams may find more value in assessing scores on the Overall and Executive and Implementation Functions at the beginning of the school year, as a marker for overall implementation. Then they can focus on assessing the nine subscale scores multiple times during the year for progress monitoring. The decisions on which subscales to focus on for improvement may depend on which DSFI subscales are scored lowest and/or which subscales are most closely connected to improving district student outcomes (e.g., academic, social-behavioral, mental health for students with or at risk for disabilities).

Moreover, through the convergent validity analyses, DSFI scores in the three configurations (Overall score, Executive and Implementation Functions scores, and the nine subscale scores) demonstrated some evidence of convergent validity across all PBIS tiers. Although the Overall score and Implementation Functions scores were related to fidelity at all three tiers, Executive Functions scores were related to fidelity only at Tiers 1 and 2. For the DSFI subscales, only the Funding and Alignment, Training, and Local Implementation Demonstrations scores were significantly correlated with fidelity at Tiers 1, 2, and 3.

It is not surprising that Implementation Functions scores were significantly correlated with TFI scores across all three tiers and the Executive Functions scores were only significantly correlated at Tiers 1 and 2. One explanation for this pattern is because strong implementation of Tier 3 systems require additional and more intensive technical assistance (e.g., individualized coaching to support school personnel implementing Tier 3 EBPs, training personnel to monitor implementation fidelity of Tier 3 EBPs, and outcomes for students with or at risk for disabilities during implementation; Robertson et al., 2020; Sanetti et al., 2015), which is largely captured within the subscales of the Implementation Functions scale (Training, Coaching, and Local Implementation Demonstrations). It is interesting the Coaching subscale was only significantly correlated with Tier 1 and 2 TFI scores because of the intensity involved in the implementation of Tier 3 EBPs by general and special educators for students with multiple needs (e.g., adequate coaching on fidelity of data collection and plan implementation; Kittelman et al., 2021); however, it could be that some districts have limited capacity for coaching and rely more on training and providing schools with strong model exemplars at Tier 3.

This study focused on addressing calls to identify malleable district factors predictive of PBIS implementation in schools (George et al., 2018; Kittelman et al., 2019). Previous research identified non-malleable district variables predictive of PBIS implementation (Kittelman et al., 2019; McIntosh et al., 2018) and malleable variables perceived to facilitate PBIS implementation (George et al., 2018; McIntosh et al., 2016). For example, of the major themes identified by George et al. (2018), we found similar themes to be significantly correlated with school fidelity at Tiers 1 to 3. For example, our findings are consistent with themes identified by George et al. (2018) including district coaches (Coaching), district data infrastructure (Evaluation), and direct support to schools (Training).

Implications for Practice

Findings of this study have several implications for future practice. For administrators and educational practitioners at regional, state, and local levels, these findings indicate the DSFI provides a multidimensional representation of a district’s capacity to implement PBIS. Previously, PBIS implementation measures were focused on the practice level for individual educators or systems level for individual schools. As well, other implementation measures for district leaders (e.g., DCA; Ward et al., 2015) are designed to be general enough a district could measure capacity for implementing a broad set of practices (e.g., early literacy and PBIS). Now, district-level administrators and leaders can use the DSFI to assess and monitor the specific infrastructure needs for building and sustaining capacity for high-fidelity implementation of multi-tiered social, emotional, and behavioral systems of support and practices (i.e., PBIS).

Moreover, the DCA includes three subscales to measure factors related to district leadership, competency, and decision-making. In contrast, the DSFI includes nine subscales and measures factors related to Leadership Teaming, Stakeholder Engagement, Funding and Alignment, Policy, Workforce Capacity, Training, Coaching, Evaluation, and Local Implementation Demonstrations. These additional subscales on the DSFI (a) confirm the importance of similar factors measured on the DCA and (b) provide district leadership teams with additional subscales to measure unique factors found to be significantly correlated with school-level PBIS implementation fidelity. Leadership teams can use the results of the DSFI, in combination with additional district data (e.g., number of schools implementing PBIS across grade levels, number of students with disabilities supported at each level and tier, professional development offerings and evaluation, implementation fidelity by school) to develop a multi-year action plan to advance implementation. The item descriptions and detailed scoring criteria in the DSFI provide insight into actionable steps and strategies for enhancing systemic implementation.

Another implication is annual evaluation with the DSFI, in concert with school-level data, provides guidance on continuous improvement for district and school teams with implementation. For example, these data can be used to identify the number of schools in districts implementing PBIS (reach), professional development strategies provided to schools (process), and whether core features of PBIS are in place (implementation fidelity). Using these data, district leadership teams can create a cycle of continuous quality improvement to assess implementation and plan for new/enhanced supports. This cycle provides the district leadership teams and school teams with regular feedback for timely adjustments to implementation activities.

Given district-wide implementation of PBIS is intended to affect every student proactively and preventively, there is a need to address implications for students with or at risk for disabilities. Establishing a district-wide infrastructure to support effective implementation of the tiered framework of PBIS and the empirically supported practices within it benefits all students. Research has shown inclusion of students with disabilities within a schoolwide system of positive behavior support positively impacts their academic and social-emotional-behavioral outcomes (K. Algozzine & Algozzine, 2007; Walker et al., 2018). As well, the proactive and instructional responses to challenging behavior inherent within PBIS reduce the use of exclusionary and reactive discipline practices for students with and without disabilities (Gage et al., 2018; Lee & Gage, 2020). District leadership teams can use the DSFI to identify current implementation strengths and weaknesses and establish improvement goals and action plans. These improvement goals and action plans, using the DSFI, can be developed specifically to better support and improve outcomes for students with or at risk for disabilities across the district. For example, district leadership teams can use the DSFI to evaluate whether school leadership teams have (a) strong model demonstrations showing how students with disabilities are being included in the implementation of practices at all three tiers, (b) professional development (training and coaching) is provided to school personnel to support these students on Tier 2 and 3 practices with high fidelity, and (c) data systems are in place and being used to disaggregate, monitor, and report on student outcome data (George et al., 2018; Horner & Sugai, 2015; Walker et al., 2022). Collectively, district leadership teams can use the DSFI to build the systemic supports and establish supportive contexts for high-fidelity implementation of PBIS across all schools for the benefit of all students.

Limitations and Directions for Future Research

Findings from this study should be interpreted with consideration for limitations and recommendations for future research. To begin, the structural validity analyses were conducted using DSFIs completed in 183 districts. Therefore, all districts were implementing some district-level PBIS systems, and their data may not reflect districts not implementing PBIS. Although the DSFI is designed to measure district implementation fidelity, future research could examine measurement invariance across DSFI scores for districts at different stages of PBIS implementation (e.g., initial implementation vs. full implementation). Next, because district leadership team members self-assessed their own implementation of PBIS district systems, with recommended guidance of an internal or external facilitator (Center on Positive Behavioral Interventions and Supports, 2020), it is possible that scores on the DSFI could be higher or lower than if completed by an external assessor. Relatedly, demographic information was not collected on individual team members; therefore, we could not evaluate team composition or competency. Future research could examine whether meaningful differences in DSFI scores occur depending on team composition and competency. For example, future research could examine the test–retest reliability of the DSFI when district leadership teams complete the DSFI with and without an internal or external facilitator.

The convergent validity analyses were completed using TFI scores from predominately elementary schools. Although it is common for the majority to be elementary schools, in part because there are more elementary schools in the United States than middle and high (Kittelman et al., 2022; McIntosh et al., 2018), the correlations between DSFI and TFI scores may be skewed toward district systems most supportive of PBIS implementation in elementary schools. For example, the Coaching subscale of the DSFI was significantly correlated only with Tier 1 and 2 implementation fidelity scores in this study; however, the majority of schools included in the study were elementary schools. It could be the correlations between Coaching and fidelity at each tier may be stronger for middle and high schools because of the additional length of time and support middle and high school leadership teams need in reaching adequate PBIS implementation fidelity (Nese et al., 2018). Relatedly, as shown in Table 2, fewer schools were assessing Tier 2 and 3 implementation fidelity and smaller samples sizes were used to examine the correlations between DSFI subscales scores and TFI Tier 2 and 3 implementation fidelity scores. It is possible, using a larger sample of schools measuring implementation of Tiers 2 and 3 PBIS, and middle and high schools, additional DSFI subscales would be significantly correlated with Tier 2 and 3 implementation fidelity scores.

Findings from this study provide future researchers with a validated measure to identify malleable district factors predictive of school PBIS implementation and district and school outcomes for students with and without disabilities (e.g., percent of students receiving and benefiting from Tier 2 and 3 EBPs, improved academic and social-behavioral outcomes district-wide). This is key because previous measures assessing district malleable factors have been brief (limited to a few items and subscales) and primarily completed by school leadership teams (e.g., SET). In addition to examining whether DSFI subscale scores are predictive of school PBIS implementation fidelity and improved student outcomes, it would also be worthwhile to examine if the relation between these variables is also moderated by certain contextual variables. For example, it may be the strength of the relations between DSFI subscale scores and school implementation fidelity scores vary based on whether schools are in the initial implementation phase of Tier 1, 2, or 3 compared to the full implementation phase. If so, these findings could provide district leadership teams with research-based guidelines on when to stagger implementation of certain district systems to best support schools with implementation at the different tiers.

Finally, although studies have examined the association between overall school-level implementation fidelity scores and student outcomes (Grasley-Boy et al., 2022), examining the relation between DSFI subscales scores and student outcomes (e.g., discipline outcomes, graduation rates, adequate supports for students with disabilities with multiple needs) could provide novel insights into how district systems support students. For example, the Workforce Capacity subscale of the DSFI includes items assessing the hiring, job descriptions, and performance evaluations of school personnel implementing PBIS at various positions. It would be worthwhile to evaluate whether districts with high scores on this subscale also have (a) more capacity to support students with disabilities with intensive needs, (b) are able to develop and implement effective Tier 3 supports (aligned to the function of the behavior), and (c) demonstrate higher fidelity of implementing Tier 3 EBPs in schools. In theory, if district leadership teams provide guidance and policy for school leaders to hire educators with the knowledge and skills in supporting students on Tier 3 EBPs (e.g., implementing function-based supports in the classroom), more educators should have the skills and capacity to support students needing intensive intervention.

Supplemental Material

sj-docx-1-rse-10.1177_07419325221114472 – Supplemental material for Validation of a Measure of District Systems Implementation of Positive Behavioral Interventions and Supports

Supplemental material, sj-docx-1-rse-10.1177_07419325221114472 for Validation of a Measure of District Systems Implementation of Positive Behavioral Interventions and Supports by Angus Kittelman, PhD, Sterett H. Mercer, PhD, Kent McIntosh, PhD, Kelsey R. Morris, EdD and Heather L. Hatton, PhD in Remedial and Special Education

Supplemental Material

sj-docx-2-rse-10.1177_07419325221114472 – Supplemental material for Validation of a Measure of District Systems Implementation of Positive Behavioral Interventions and Supports

Supplemental material, sj-docx-2-rse-10.1177_07419325221114472 for Validation of a Measure of District Systems Implementation of Positive Behavioral Interventions and Supports by Angus Kittelman, PhD, Sterett H. Mercer, PhD, Kent McIntosh, PhD, Kelsey R. Morris, EdD and Heather L. Hatton, PhD in Remedial and Special Education

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through grant no. R324A1800027 to the University of Oregon. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

ORCID iDs

Sterett H. Mercer

Kent McIntosh

Kelsey R. Morris

Supplemental Material

Supplemental material for this article is available on the Remedial and Special Education website with the online version of this article.

References

Algozzine

(2007). Classroom instructional ecology and school-wide positive behavior support. Journal of Applied School Psychology, 24(1), 29–47. https://doi.org/10.1300/J370v24n01_02

Algozzine

R. F.

Barrett

Eber

George

Horner

R. H.

Lewis

T. J.

Putnam

R. F.

Swain-Bradway

McIntosh

Sugai

(2014). SWPBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. http://www.pbis.org

Blair

K. C.

Park

Kim

(2020). A meta-analysis of tier 2 interventions implemented within school-wide positive behavioral interventions and supports. Psychology in the Schools, 58(1), 141–161. https://doi.org/10.1002/pits.22443

Bonifay

W. E.

Reise

S. P.

Scheines

Meijer

R. R.

(2015). When are multidimensional data unidimensional enough for structural equation modeling? An evaluation of the DETECT multidimensionality index. Structural Equation Modeling: A Multidisciplinary Journal, 22(4), 504–516. https://doi.org/10.1080/10705511.2014.938596

Center on Positive Behavioral Interventions and Supports. (2015). Positive behavioral interventions and supports (PBIS) implementation blueprint. University of Oregon. https://www.pbis.org/resource/pbis-implementation-blueprint.

Center on Positive Behavioral Interventions and Supports. (2020). Positive behavioral interventions and supports District Systems Fidelity Inventory (DSFI)—Version 0.2. University of Oregon. https://www.pbis.org

Gage

N. A.

Whitford

D. K.

Katsiyannis

(2018). A review of schoolwide positive behavior interventions and supports as a framework for reducing disciplinary exclusions. The Journal of Special Education, 52(3), 142–151. https://doi.org/10.1177/0022466918767847

George

H. P.

Cox

K. E.

Minch

Sandomierski

(2018). District practices associated with successful SWPBIS implementation. Behavioral Disorders, 43(3), 393–406. https://doi.org/10.1177/0198742917753612

Grasley-Boy

N. M.

Gage

N. A.

Lombardo

Anderson

(2022). The additive effects of implementing advanced tiers of SWPBIS with fidelity on disciplinary exclusions. Journal of Positive Behavior Interventions, 24, 183–195. https://doi.org/10.1177/10983007211011767

10.

Horner

R. H.

Sugai

(2015). School-wide PBIS: An example of applied behavior analysis implemented at a scale of social importance. Behavior Analysis in Practice, 8(1), 80–85. https://doi.org/10.1007/s40617-015-0045-4

11.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

12.

Kincaid

Childs

George

(2010). School-wide benchmarks of quality (Revised). https://www.pbis.org/resource/boq

13.

Kittelman

McIntosh

Hoselton

(2019). Adoption of PBIS within school districts. Journal of School Psychology, 76, 159–167. https://doi.org/10.1016/j.jsp.2019.03.007

14.

Kittelman

Mercer

S. H.

McIntosh

Hoselton

(2022). Optimal timing for launching installation of tiers 2 and 3 systems of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 24, 171–182. https://doi.org/10.1177/1098300721996084

15.

Kittelman

Mercer

S. H.

McIntosh

Nese

R. N. T.

(2021b). Development and validation of a measure assessing sustainability of tier 2 and 3 behavior support systems. Journal of School Psychology, 85, 140–154. https://doi.org/10.1016/j.jsp.2021.02.001

16.

Kittelman

Strickland-Cohen

M. K.

Pinkelman

S. E.

McIntosh

(2020). Variables contributing to abandonment and readoption of SWPBIS. Journal of Positive Behavior Interventions, 22(2), 67–77. https://doi.org/10.1177/1098300719888748

17.

Lee

Gage

N. A.

(2020). Updating and expanding systematic reviews and meta-analyses on the effects of school-wide positive behavior interventions and supports. Psychology in the Schools, 57(5), 783–804. https://doi.org/10.1002/pits.22336

18.

Lorenzo-Seva

Ferrando

P. J.

(2006). FACTOR: A computer program to fit the exploratory factor analysis model. Behavior Research Methods, 38(1), 88–91. https://doi.org/10.3758/BF03192753

19.

Massar

McIntosh

Mercer

S. H.

Hoselton

(2017). Factor analysis of a fidelity of implementation measure for social behavior systems. Remedial and Special Education, 40(1), 16–24. https://doi.org/10.1177/0741932517736515

20.

Mathews

McIntosh

Frank

J. L.

May

S. L

. (2014). Critical features predicting sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 16(3), 168–178. https://doi.org/10.1177/1098300713484065

21.

McIntosh

Massar

M. M.

Algozzine

R. F.

George

H. P.

Horner

R. H.

Lewis

T. J.

Swain-Bradway

(2017). Technical adequacy of the SWPBIS Tiered Fidelity Inventory. Journal of Positive Behavior Interventions, 19(1), 3–13. https://doi.org/10.1177/1098300716637193

22.

McIntosh

Mercer

S. H.

Hume

A. E.

Frank

J. L.

Turri

M. G.

Mathews

(2013). Factors related to sustained implementation of schoolwide positive behavior support. Exceptional Children, 79(3), 293–311.

23.

McIntosh

Mercer

S. H.

Nese

R. N. T.

Strickland-Cohen

M. K.

Hoselton

(2016). Predictors of sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 18(4), 209–218. https://doi.org/10.1177/1098300715599737

24.

McIntosh

Mercer

S. H.

Nese

R. N. T.

Strickland-Cohen

M. K.

Kittelman

Hoselton

Horner

R. H.

(2018). Factors predicting sustained implementation of a universal behavior support framework. Educational Researcher, 47(5), 307–316. https://doi.org/10.3102/0013189X18776975

25.

Mercer

S. H.

McIntosh

Hoselton

(2017). Comparability of fidelity measures for assessing tier I school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 19(4), 195–204. https://doi.org/10.1177/1098300717693384

26.

Muthén

L. K.

Muthén

B. O.

(2017). Mplus user’s guide (8th ed.).

27.

Nese

R. N. T.

Nese

J. F. T.

McIntosh

Mercer

S. H.

Kittelman

(2018). Predicting latency of reaching adequate implementation of tier I schoolwide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 21(2), 106–116. https://doi.org/10.1177/1098300718783755

28.

Netzel

D. M.

Eber

(2003). Shifting from reactive to proactive discipline in an urban school district: A change of focus through PBIS implementation. Journal of Positive Behavior Interventions, 5(2), 71–79. https://doi.org/10.1177/10983007030050020201

29.

Robertson

R. E.

Kokina

A. A.

Moore

D. W.

(2020). Barriers to implementing behavior Intervention plans: Results of a statewide survey. Journal of Positive Behavior Interventions, 22(3), 145–155. https://doi.org/10.1177/1098300720908013

30.

Sanetti

L. M. H.

Collier-Meek

M. A.

Long

A. C. J.

Byron

Kratochwill

T. R.

(2015). Increasing teacher treatment integrity of behavior support plans through consultation and implementation planning. Journal of School Psychology, 53(3), 209–229. https://doi.org/10.1016/j.jsp.2015.03.002

31.

Schaper

McIntosh

Hoselton

(2016). Within-year fidelity growth of SWPBIS during installation and initial implementation. School Psychology Quarterly, 31(3), 358–368. https://doi.org/10.1037/spq0000125

32.

Sugai

Horner

R. H.

(2019). Sustaining and scaling positive behavioral interventions and supports: Implementation drivers, outcomes, and considerations. Exceptional Children, 86(2), 120–136. https://doi.org/10.1177/0014402919855331

33.

Sugai

Horner

R. H.

Lewis-Palmer

T. L.

(2001). Team Implementation Checklist (TIC). Educational and Community Supports.

34.

Sugai

Lewis-Palmer

T. L.

Todd

A. W.

Horner

R. H.

(2005). School-wide evaluation tool (SET) version 2.1. Educational and Community Supports. http://www.pbis.org

35.

Timmerman

M. E.

Lorenzo-Seva

(2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209–220. https://doi.org/10.1037/a0023353

36.

Walker

V. L.

Chung

Y.-C.

Bonnet

L. K.

(2018). Function-based intervention in inclusive school settings: A meta-analysis. Journal of Positive Behavior Interventions, 20(4), 203–216. https://doi.org/10.1177/1098300717718350

37.

Walker

V. L.

Loman

S. L.

Mickelson

A. M.

Lyon

K. J.

(2022). Participation of students with extensive support needs in SWPBIS: Administrator and educator perspectives. Psychology in the Schools, 59, 1623–1642. https://doi.org/10.1002/pits.22711

38.

Ward

C. S.

Harms

A. L.

St. Martin, K. Cusumano

Russell

Horner

R. H.

(2022). Development and technical adequacy of the District Capacity Assessment. Journal of Positive Behavior Interventions, 24, 87–98. https://doi.org/10.1177/1098300721990911

39.

Ward

C. S.

St. Martin, K. Horner

R. H.

Duda

Ingram-West

Tedesco

Putnam

Buenrostro

Chaparro

(2015). District capacity assessment. The University of North Carolina at Chapel Hill. https://nirn.fpg.unc.edu

40.

Zumbo

B. D.

Gadermann

A. M.

Zeisser

(2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6, 21–29. https://doi.org/10.22237/jmasm/1177992180

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB