A Meta-Analysis of Performance Feedback as an Evidence-Based Practice in Special Education Teacher Preparation

Abstract

This meta-analysis systematically examines the evidence supporting the effectiveness of performance feedback as an evidence-based practice in the training of preservice special education teachers. A total of 27 studies were evaluated using the rigorous What Works Clearinghouse (WWC) design standards for Single-Case Experimental Research. Results indicate that 15 studies, encompassing 56 preservice special education teachers, met the WWC design standards with and without reservation. The weighted aggregated mean treatment effect estimates were .91 (Tau-U) and 2.59 (Between-Case Standardized Mean Difference), demonstrating a strong impact of performance feedback interventions on enhancing preservice teachers’ instructional practices. This study contributes to the extant literature by providing robust evidence supporting the classification of performance feedback as an evidence-based practice for preservice special education teacher preparation, meeting the stringent criteria set forth by the What Works Clearinghouse (WWC). Analyses also include methodological quality, risk of bias, and treatment effect estimates, addressing key gaps in extant literature and supporting recommendations for future research. The findings highlight the potential of performance feedback to shape effective teaching behaviors among preservice special education teachers.

Keywords

performance feedback preservice special education teachers evidence-based practice single-case experimental research teacher training

Get full access to this article

View all access options for this article.

References

Adams

Hillier-Brown

F. C.

Moore

H. J.

Lake

A. A.

Araujo-Soares

White

Summerbell

(2016). Searching and synthesising “grey literature” and “grey information” in public health: Critical reflections on three case studies. Systematic Reviews, 5(1), 1–11. https://doi.org/10.1186/s13643-016-0337-y

*Auld

Belfiore

Scheeler

(2010). Increasing pre-service teachers’ use of differential reinforcement: Effects of performance feedback on consequences for student behavior. Journal of Behavioral Education, 19, 169–183. https://doi.org/10.1007/s10864-010-9107-4

*Barton

E. E.

Chen

C. I.

Pribble

Pomes

Kim

Y. A.

(2013ab). Coaching preservice teachers to teach play skills to children with disabilities. Teacher Education and Special Education, 36(4), 330–349. https://doi.org/10.1177/0888406413505113

*Barton

E. E.

Fuller

E. A.

Schnitz

(2016). The use of email to coach preservice early childhood teachers. Topics in Early Childhood Special Education, 36(2), 78–90. https://doi.org/10.1177/0271121415612728

Billingsley

G. M.

Scheuermann

B. K.

(2014). Using virtual technology to enhance field experiences for pre-service special education teachers. Teacher Education and Special Education, 37(3), 255–272. https://doi.org/10.1177/0888406414530413

Brock

M. E.

Carter

E. W.

(2017). A meta-analysis of educator training to improve implementation of interventions for students with disabilities. Remedial and Special Education, 38(3), 131–144. https://doi.org/10.1177/0741932516653477

Brownell

M. T.

Benedict

A. E.

Leko

M. M.

Peyton

Pua

Richards-Tutor

(2019). A continuum of pedagogies for preparing teachers to use high-leverage practices. Remedial and Special Education, 40(6), 338–355. https://doi.org/10.1177/0741932518824990

Chand

Alasa

V. M.

Chitiyo

Pietrantoni

(2022). Preparation of pre-service teachers: Assessment of generation Z students. In Keengwe

(Ed.), Handbook of research on digital-based assessment and innovative practices in education (pp. 116–130). IGI Global.

Chen

Pustejovsky

J. E.

Klingbeil

D. A.

Van Norman

E. R.

(2023). Between-case standardized mean differences: Flexible methods for single-case designs. Journal of School Psychology, 98, 16–38. https://doi.org/10.1016/j.jsp.2023.02.002

10.

Cleaver

Detrich

States

(2019). Overview of performance feedback. The Wing Institute. https://www.winginstitute.org/teacher-evaluation-feedback

11.

*Coogle

C. G.

Ottley

J. R.

Storie

Rahn

N. L.

Kurowski-Burt

(2020). Performance-based feedback to enhance preservice teachers’ practice and preschool children’s expressive communication. Journal of Teacher Education, 71(2), 188–202. https://doi.org/10.1177/0022487118803583

12.

*Coogle

C. G.

Rahn

N. L.

Ottley

J. R.

(2015). Pre-service teacher use of communication strategies upon receiving immediate feedback. Early Childhood Research Quarterly, 32, 105–115. https://doi.org/10.1016/j.ecresq.2015.03.003

13.

Cook

Odom

(2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79(2), 135–144. https://doi.org/10.1177/001440291307900201

14.

Cornelius

K. E.

Nagro

S. A.

(2014). Evaluating the evidence base of performance feedback in preservice special education teacher training. Teacher Education and Special Education, 37(2), 133–146. https://doi.org/10.1177/0888406414521837

15.

Darling-Hammond

Sykes

(2003). Wanted, a national teacher supply policy for education: The right way to meet the “Highly Qualified Teacher” challenge. Education Policy Analysis Archives, 11, Article 33. http://epaa.asu.edu/ojs/article/view/261

16.

Fallon

L. M.

Collier-Meek

M. A.

Maggin

D. M.

Sanetti

L. M.

Johnson

A. H.

(2015). Is performance feedback for educators an evidence-based practice? A systematic review and evaluation based on single-case research. Exceptional Children, 81(2), 227–246. https://doi.org/10.1177/0014402914551738

17.

Gage

N. A.

Cook

B. G.

Reichow

(2017). Publication bias in special education meta-analyses. Exceptional Children, 83(4), 428–445. https://doi.org/10.1177/001440291769101

18.

Garet

M. S.

Heppen

Walters

Smith

Yang

(2016). Does content-focused teacher professional development work? Findings from three Institute of Education Sciences studies. National Center for Education Evaluation and Regional Assistance, Institute for Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee/pubs/20174010/pdf/20174010.pdf

19.

*Garland

D. P.

(2022). Individualized clinical coaching with Bug-in-Ear: Enhancing fidelity of implementation of behavior specific praise among novice teachers of students with developmental disabilities in rural classrooms. Journal of Inquiry and Action in Education, 11(1), n1.

20.

Harn

Meline

(2021). Developing critical thinking and reflection in teachers within teacher preparation. In Mariano

G. J.

Figliano

F. J.

(Eds.), Research Anthology on Developing Critical Thinking Skills in Students (pp. 232–251). IGI Global.

21.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179. https://doi.org/10.1177/001440290507100203

22.

Huwaldt

Steinhorst

(2015). Plot Digitizer (Version 2.6. 8). Biostat.

23.

Individuals With Disabilities Education Improvement Act of 2004, Pub. L. No. 108-446 § 118 Stat. 2647.

24.

Kazdin

A. E.

(2010). Single-case research designs: Methods for clinical and applied settings (2nd ed.). Oxford University Press.

25.

Klingner

J. K.

Boardman

A. G.

McMaster

K. L.

(2013). What does it take to scale up and sustain evidence-based practices? Exceptional Children, 79(2), 195–211. https://doi.org/10.1177/001440291307900205

26.

Kratochwill

T. R.

Horner

R. H.

Levin

J. R.

Machalicek

Ferron

Johnson

(2021). Single-case design standards: An update and proposed upgrades. Journal of School Psychology, 89, 91–105. https://doi.org/105.10.1016/j.jsp.2021.10.006

27.

Kratochwill

T. R.

Horner

R. H.

Levin

J. R.

Machalicek

Ferron

Johnson

(2023). Single-case intervention research design standards: Additional proposed upgrades and future directions. Journal of School Psychology, 97, 192–216. https://doi.org/10.1016/j.jsp.2022.12.002

28.

Kretlow

A. G.

Bartholomew

C. C.

(2010). Using coaching to improve the fidelity of evidence-based practices: A review of studies. Teacher Education and Special Education, 33(4), 279–299. https://doi.org/10.1177/0888406410371643

29.

Maggin

D. M.

Barton

Reichow

Lane

K. L.

Shogren

K. A.

(2022). Commentary on the What Works Clearinghouse Standards and Procedures Handbook (v. 4.1) for the review of single-case research. Remedial and Special Education, 43(6), 421–433. https://doi.org/10.1177/07419325211051317

30.

Martens

B. K.

Witt

J. C.

Elliott

S. N.

Darveaux

D. X.

(1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16, 191–198. https://doi.org/10.1037/0735-7028.16.2.191

31.

*McKinney

Vasquez

III . (2014). There’s a bug in your ear! Using technology to increase the accuracy of DTT implementation. Education and Training in Autism and Developmental Disabilities, 49(4), 594–600. https://www-jstor-org-443.web.bisu.edu.cn/stable/24582354?seq=1

32.

*McLeod

R. H.

Kim

Resua

K. A.

(2019). The effects of coaching with video and email feedback on preservice teachers’ use of recommended practices. Topics in Early Childhood Special Education, 38(4), 192–203. https://doi.org/10.1177/0271121418763531

33.

*Miller

R. D.

Uphold

(2021). Using content acquisition podcasts to improve preservice teacher use of behavior-specific praise. Teacher Education and Special Education, 44(4), 300–318. https://doi.org/10.1177/0888406421994336

34.

Moeyaert

Dehghan-Chaleshtori

(2023). Evaluating the design and evidence of single-case experimental designs using the What Works Clearinghouse standards. Evidence-Based Communication Assessment and Intervention, 17(1), 78–96. https://doi.org/10.1080/17489539.2022.2139148

35.

Moher

Liberati

Tetzlaff

Altman

D. G.

, & Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLOS Medicine, 6, e1000097. https://doi.org/10.7326/0003-4819-151-4-200908180-00135

36.

*Morgan

R. L.

Gustafson

K. J.

Hudson

P. J.

Salzberg

C. L.

(1992). Peer coaching in a preservice special education program. Teacher Education and Special Education, 15(4), 249–258. https://doi.org/10.1177/088840649201500403

37.

*Morgan

R. L.

Menlove

Salzberg

C. L.

Hudson

(1994). Effects of peer coaching on the acquisition of direct instruction skills by low-performing preservice teachers. The Journal of Special Education, 28(1), 59–76. https://doi.org/10.1177/002246699402800105

38.

Mortenson

B. P.

Witt

J. C.

(1998). The use of weekly performance feedback to increase teacher implementation of a pre referral academic intervention. School Psychology Review, 27(4), 613–627. https://doi.org/10.1080/02796015.1998.12085942

39.

Nudelman

Otto

(2020). The development of a new generic risk-of-bias measure for systematic reviews of surveys. Methodology, 16(4), 278–298. https://doi.org/10.5964/meth.4329

40.

Parker

R. I.

Vannest

K. J.

Davis

J. L.

Sauber

S. B.

(2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42(2), 284–299. https://doi.org/10.1016/j.beth.2010.08.006

41.

Pustejovsky

J. E.

Chen

Klingbeil

Van Norman

(2023, April). Empirical benchmarks for between-case standardized mean differences from single-case multiple baseline designs examining academic interventions [Paper presentation]. Annual Convention of the American Educational Research Association, Chicago, IL.

42.

Pustejovsky

J. E.

Hedges

L. V.

Shadish

W. R.

(2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics, 39(5), 368–393. https://doi.org/10.3102/1076998614547577

43.

Rakap

(2015). Effect sizes as result interpretation aids in single-subject experimental research: Description and application of four nonoverlap methods. British Journal of Special Education, 42(1), 11–33. https://doi.org/10.1111/1467-8578.12091

44.

*Rakap

(2017). Impact of coaching on preservice teachers’ use of embedded instruction in inclusive preschool classrooms. Journal of Teacher Education, 68(2), 125–139. https://doi.org/10.1177/0022487116685753

45.

*Rakap

Balikci

(2023ab). Training preservice teachers to use evidence-based practices: Effects of coaching with performance feedback on teacher and child outcomes. Learning and Instruction, 86, 101755. https://doi.org/10.1016/j.learninstruc.2023.101755

46.

Rakap

Evran

Cig

(2016). Comparative evaluation of the reliability and validity of three data extraction programs: UnGraph, GraphClick, and DigitizeIt. Computer in Human Behavior, 55, 159–166. https://doi.org/10.1016/j.chb.2015.09.008

47.

Reichow

Barton

E. E.

Maggin

D. M.

(2018). Development and applications of the single-case design risk of bias tool for evaluating single-case design research study reports. Research in Developmental Disabilities, 79, 53–64. https://doi.org/10.1016/j.ridd.2018.05.008

48.

Rock

M. L.

Schumacker

R. A.

Gregg

Howard

P. W.

Gable

R. A.

Zigmond

(2014). How are they now? Longer term effects of eCoaching through online bug-in-ear technology. Teacher Education and Special Education, 37(2), 161–181. https://doi.org/10.1177/0888406414525048

49.

Schaefer

J. M.

Ottley

J. R.

(2018). Evaluating immediate feedback via bug-in-ear as an evidence-based practice for professional development. Journal of Special Education Technology, 33(4), 247–258. https://doi.org/10.1177/0162643418766870

50.

Scheeler

M. C.

McKinnon

Stout

(2012). Effects of immediate feedback delivered via webcam and bug-in-ear technology on preservice teacher performance. Teacher Education and Special Education, 35(1), 77–90. https://doi.org/10.1177/0888406411401919

51.

Scheeler

M. C.

Ruhl

K. L.

McAfee

J. K.

(2004). Providing performance feedback to teachers: A review. Teacher Education and Special Education, 27(4), 396–407. https://doi.org/10.1177/088840640402700407

52.

Schles

R. A.

Robertson

R. E.

(2019). The role of performance feedback and implementation of evidence-based practices for preservice special education teachers and student outcomes: A review of the literature. Teacher Education and Special Education, 42(1), 36–48. https://doi.org/10.1177/0888406417736571

53.

Shepley

Zimmerman

K. N.

Ayres

K. M.

(2020). Estimating the impact of design standards on the rigor of a subset of single-case research. Journal of Disability Policy Studies, 32(2), 108–118. https://doi.org/10.1177/1044207320934048

54.

Sinclair

A. C.

Gesel

S. A.

LeJeune

L. M.

Lemons

C. J.

(2020). A review of the evidence for real-time performance feedback to improve instructional practice. The Journal of Special Education, 54(2), 90–100. https://doi.org/10.1177/0022466919878470

55.

Stahmer

Rieth

Lee

Reisinger

Mandell

Connell

(2015). Training teachers to use evidence-based practices for autism: Examining procedural implementation fidelity. Psychology in the Schools, 52, 181–195. https://doi.org/10.1002/pits.21815

56.

Van den Noortgate

Onghena

. (2024). Harnessing available evidence in single-case experimental studies: The use of multilevel meta-analysis. Psychologica Belgica, 64(1), 166–184. https://doi.org/10.5334/pb.1307

57.

Van Houten

. (1980). Learning through feedback. Human Science Press.

58.

Warren

S. F.

Fey

M. E.

Yoder

P. J.

(2007). Differential treatment intensity research: A missing link to creating optimally effective communication interventions. Mental Retardation and Developmental Disabilities Research Reviews, 13(1), 70–77. https://doi.org/10.1002/mrdd.20139

59.

What Works Clearinghouse. (2017). What Works Clearinghouse: Standards Handbook (Version 4.0). U.S. Department of Education. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_standards_handbook_v4.pdf

60.

What Works Clearinghouse. (2020). What Works Clearinghouse standards handbook (Version 4.1). U.S. Department of Education. https://ies.ed.gov/ncee/wwc/handbooks

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB