Longitudinal Associations Between SWPBIS Fidelity of Implementation and Behavior and Academic Outcomes

Abstract

The purpose of this study was to examine associations between implementation fidelity of school-wide positive behavioral interventions and supports (SWPBIS) and student outcomes over time. The sample included 477 K–12 schools across 10 states implementing SWPBIS for varying numbers of years and at varying levels of fidelity (but with 78% of schools at or above established fidelity criteria). Results indicate that, in general, schools showed a decline in office discipline referrals (ODRs) and out-of-school suspensions (OSSs) over 3 years. SWPBIS fidelity of implementation was positively associated with initial levels of ODRs and OSSs, and the relations between fidelity and OSSs varied based on years of SWPBIS implementation. Levels of fidelity did not predict change in ODRs or OSSs. Despite no statistically significant associations between fidelity and student academic outcomes, schools that had implemented SWPBIS for 3 years or more had higher achievement in mathematics after controlling for prior achievement.

Keywords

SWPBIS school-wide positive behavioral interventions and support student outcomes

Get full access to this article

View all access options for this article.

References

Algozzine

Horner

R. H.

Sugai

Barrett

Dickey

C. R.

Eber

. . . Tobin

(2010). Evaluation blueprint for school-wide positive behavior support. Eugene, OR: National Technical Assistance Center on Positive Behavior Interventions and Support. Retrieved from http://www.pbis.org/blueprintguidestools/blueprint/evaluation-blueprint

Andreou

T. E.

McIntosh

Ross

S. W.

Kahn

J. D.

(2015). Critical incidents in the sustainability of school-wide positive behavioral interventions and supports. The Journal of Special Education, 49, 157–167. doi:10.1177/0022466914554298

Blase

Fixsen

(2013). Core intervention components: Identifying and operationalizing what makes programs work. Washington, DC: Office of the Assistant Secretary for Planning and Evaluation, Office of Human Services Policy, U.S. Department of Health and Human Services. Retrieved from http://aspe.hhs.gov/hsp/13/KeyIssuesforChildrenYouth/CoreIntervention/rb_CoreIntervention.cfm

Bradshaw

C. P.

Debnam

Koth

C. W.

Leaf

(2009). Preliminary validation of the Implementation Phases Inventory for assessing fidelity of schoolwide positive behavior supports. Journal of Positive Behavior Interventions, 11, 145–160. doi:10.1177/1098300708319126

Bradshaw

C. P.

Mitchell

M. M.

Leaf

P. J.

(2010). Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133–148. doi:10.1177/1098300709334798

Century

Rudnick

Freeman

(2010). A framework for measuring fidelity of implementation: A foundation for shared language and accumulation of knowledge. American Journal of Evaluation, 31, 199–218. doi:10.1177/1098214010366173

Childs

K. E.

Kincaid

George

H. P.

Gage

N. A.

(2016). The relationship between school-wide implementation of positive behavior intervention and supports and student discipline outcomes. Journal of Positive Behavior Interventions, 18, 89–99. doi:10.1177/1098300715590398

Cohen

Kincaid

Childs

K. E.

(2007). Measuring school-wide positive behavior support implementation development and validation of the benchmarks of quality. Journal of Positive Behavior Interventions, 9, 203–213. doi:10.1177/10983007070090040301

Dane

A. V.

Schneider

B. H.

(1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45. doi:10.1016/S0272-7358(97)00043-3

10.

Duncan

G. J.

Dowsett

C. J.

Claessens

Magnuson

Huston

A. C.

Klebanov

. . . Brooks-Gunn

(2007). School readiness and later achievement. Developmental Psychology, 43, 1428–1446. doi:10.1037/0012-1649.43.6.1428

11.

Enders

C. K.

(2010). Applied missing data analysis. New York, NY: Guilford Press.

12.

Flannery

K. B.

Fenning

Kato

M. M.

McIntosh

(2014). Effects of school-wide positive behavioral interventions and supports and fidelity of implementation on problem behavior in high schools. School Psychology Quarterly, 29, 111–124. doi:10.1037/spq0000039

13.

Freeman

Simonsen

McCoach

D. B.

Sugai

Lombardi

Horner

(2016). Relationship between school-wide positive behavior interventions and supports and academic, attendance, and behavior outcomes in high schools. Journal of Positive Behavior Interventions, 18, 41–51. doi:10.1177/1098300715580992

14.

Gage

N. A.

Leite

Childs

Kincaid

(2017). Average treatment effect of school-wide positive behavioral interventions and supports on school-level academic achievement in Florida. Journal of Positive Behavior Interventions, 19, 158–167. doi:10.1177/1098300717693556

15.

George

H. P.

Childs

K. E.

(2012). Evaluating implementation of schoolwide behavior support: Are we doing it well? Preventing School Failure: Alternative Education for Children and Youth, 56, 197–206.

16.

Hagan-Burke

Burke

M. D.

Martin

Boon

R. T.

Fore

III Kirkendoll

(2005). The internal consistency of the school-wide subscales of the Effective Behavioral Support Survey. Education and Treatment of Children, 28, 400–413.

17.

Horner

R. H.

Sugai

Smolkowski

Eber

Nakasato

Todd

A. W.

. . . Esparanza

(2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–144. doi:10.1177/1098300709332067

18.

Horner

R. H.

Todd

A. W.

Lewis-Palmer

Irvin

L. K.

Sugai

Boland

J. B.

(2004). The school-wide evaluation tool (SET) a research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6, 3–12. doi:10.1177/10983007040060010201

19.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

20.

Kim

McIntosh

Hoselton

(2014). Do schools with adequate Tier I SWPBIS implementation have stronger implementation at Tiers II and III? (PBIS Evaluation Brief). Eugene: OSEP National Technical Assistance Center on Positive Behavioral Interventions and Supports, University of Oregon.

21.

Kincaid

Childs

George

H. P.

(2005). School-wide benchmarks of quality (Unpublished instrument). Tampa: University of South Florida.

22.

Lee

Reeves

(2012). Revisiting the impact of NCLB high-stakes school accountability, capacity, and resources: State NAEP 1990–2009 reading and math achievement gaps and trends. Educational Evaluation and Policy Analysis, 34, 209–231. doi:10.3102/0162373711431604

23.

McIntosh

Goodman

(2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. New York, NY: Guilford Press.

24.

McIntosh

Mercer

S. H.

Nese

R. N. T.

Strickland-Cohen

M. K.

Hoselton

(2016). Predictors of sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 18, 209–218. doi:10.1177/1098300715599737

25.

McIntosh

Mercer

S. H.

Nese

R. N. T.

Strickland-Cohen

M. K.

Kittelman

Hoselton

. . . Horner

R. H.

(2017). Factors predicting sustained implementation of a universal behavior support framework. Manuscript submitted for publication.

26.

McIntosh

Predy

L. K.

Upreti

Hume

A. E.

Turri

M. G.

Mathews

(2014). Perceptions of contextual features related to implementation and sustainability of school-wide positive behavior support. Journal of Positive Behavior Interventions, 16, 31–43. doi:10.1177/1098300712470723

27.

Mercer

S. H.

McIntosh

Hoselton

(2017). Comparability of fidelity measures for assessing tier 1 school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions, 19, 195–204. doi:10.1177/1098300717693384

28.

Muthén

L. K.

Muthén

B. O.

(1998–2012). Mplus user’s guide: Statistical analysis with latent variables: User’s guide (7th ed.). Los Angeles, CA: Author.

29.

Muthén

L. K.

Muthén

B. O.

(2009, March). Mplus short courses topic 8: Multilevel modeling with latent variables using Mplus: Longitudinal analysis (Handout in the Mplus Short Course Video). Retrieved from https://www.statmodel.com/download/Topic%208-%20v17.pdf

30.

National Center for Education Statistics. (2014). Public elementary/ secondary school universe survey data, 2012-13 [Database]. Retrieved from http://nces.ed.gov/ccd/

31.

O’Donnell

C. L.

(2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of Educational Research, 78, 33–84. doi:10.3102/0034654307313793

32.

Pas

E. T.

Bradshaw

C. P.

(2012). Examining the association between implementation and outcomes. The Journal of Behavioral Health Services & Research, 39, 417–433.

33.

Pas

E. T.

Waasdorp

T. E.

Bradshaw

(2015). Examining contextual influences on classroom-based implementation of positive behavior support strategies: Findings from a randomized controlled effectiveness trial. Prevention Science, 16, 1096–1106.

34.

Reinke

W. M.

Herman

K. C.

Stormont

(2013). Classroom-level positive behavior supports in schools implementing SW-PBIS: Identifying areas for enhancement. Journal of Positive Behavior Interventions, 15, 39–50. doi:10.1177/1098300712459079

35.

Simonsen

Eber

Black

A. C.

Sugai

Lewandowski

Sims

Myers

(2012). Illinois statewide positive behavioral interventions and supports: Evolution and impact on student outcomes across years. Journal of Positive Behavior Interventions, 14, 5–16. doi:10.1177/1098300711412601

36.

Solomon

B. G.

Tobin

K. G.

Schutte

G. M.

(2015). Examining the reliability and validity of the effective behavior support self-assessment survey. Education and Treatment of Children, 38, 175–191.

37.

Steiger

J. H.

(2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42, 893–898. doi:10.1016/j.paid.2006.09.017

38.

Sugai

Horner

R. H.

Todd

A. W.

(2000). PBIS Self Assessment Survey 2.0. Eugene, OR: OSEP Center for Positive Behavioral Supports.

39.

Sugai

Horner

R. R.

(2006). A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review, 35, 245–259.

40.

Sugai

Lewis-Palmer

Todd

A. W.

Horner

R. H.

(2001). School-wide evaluation tool (Version 2.0). Eu-gene: Educational and Community Supports, University of Oregon.

41.

Sugai

Todd

A. W.

Horner

R. H.

(2001). Team implementation checklists (Version 2.2). Eugene, OR: OSEP Center for Positive Behavioral Supports.

42.

Turri

M. G.

Mercer

S. H.

McIntosh

Nese

R. N.

Strickland-Cohen

M. K.

Hoselton

(2016). Examining barriers to sustained implementation of school-wide prevention practices. Assessment for Effective Intervention, 42, 6–17. doi:10.1177/1534508416634624