Effects of a University-Led High-Impact Tutoring Program on Low-Achieving High School Students: A 3-Year Randomized Controlled Trial

Abstract

This study uses a randomized controlled trial designed to examine a university-led high-impact tutoring program at seven high schools. The treatment group (n = 525) participated in high-impact tutoring (i.e., groups of 2:1 or 3:1) while the control group (n = 438) attended a remedial mathematics course. The treatment group showed a difference of nearly a half-year of learning (0.13 SD) compared with the control group. We also found no evidence that 2:1 student–tutor groups were more effective than 3:1 groups. Although the university-led program produced strong effects, it was delivered at a high cost. Future work is needed to investigate strategies for reducing the cost of high-impact tutoring while maintaining effectiveness.

Keywords

high-impact tutoring high-dosage tutoring remedial instruction instructional time randomized controlled trial

Get full access to this article

View all access options for this article.

References

Adams

Khojasteh

(2018). Igniting students’ inner determination: The role of a need-supportive climate. Journal of Educational Administration, 56(4), 382–397. https://doi.org/10.1108/JEA-04-2017-0036

Angrist

J. D.

Cohodes

S. R.

Dynarski

S. M.

Pathak

P. A.

Walters

C. R.

(2016). Stand and deliver: Effects of Boston's charter high schools on college preparation, entry, and choice. Journal of Labor Economics, 34(2), 275–318. https://doi.org/10.1086/683665

Aughinbaugh

(2012). The effects of high school math curriculum on college attendance: Evidence from the NLSY97. Economics of Education Review, 31(6), 861–870. https://doi.org/10.1016/j.econedurev.2012.06.004

Banerjee

Banerji

Berry

Duflo

Kannan

Mukherji

Shotland

Walton

(2016). Mainstreaming an effective intervention: Evidence from randomized evaluations of “Teaching at the Right Level” in India (NBER Working Paper No. w22746). National Bureau of Economic Research. https://doi.org/10.3386/w22746

Bryk

A. S.

Gomez

L. M.

Grunow

LeMahieu

P. G.

(2015). Learning to improve: How America's schools can get better at getting better. Harvard Education Press.

Cattaneo

M. A.

Oggenfuss

Wolter

S. C.

(2017). The more, the better? The impact of instructional time on student performance. Education Economics, 25(5), 433–445. https://doi.org/10.1080/09645292.2017.1315055

Cerasoli

C. P.

Ford

M. T.

(2014). Intrinsic motivation, performance, and the mediating role of mastery goal orientation: A test of self-determination theory. Journal of Psychology, 148(3), 267–286. https://doi.org/10.1080/00223980.2013.783778

Cha

S. H.

(2015). Exploring disparities in taking high level math courses in public high schools. KEDI Journal of Educational Policy, 12(1), 3–17. https://doi.org/10.22804/kjep.2015.12.1.001

Chetty

Friedman

J. N.

Rockoff

J. E.

(2014). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104(9), 2633–2679. https://doi.org/10.1257/aer.104.9.2633

10.

Cohodes

S. R.

Parham

K. S.

(2021). Charter schools’ effectiveness, mechanisms, and competitive influence (NBER Working Paper No. w28477). National Bureau of Economic Research. https://doi.org/10.3386/w28477

11.

Colvin

J. W.

(2007). Peer tutoring and social dynamics in higher education. Mentoring & Tutoring, 15(2), 165–181. https://doi.org/10.1080/13611260601086345

12.

Cortes

K. E.

Goodman

J. S.

Nomi

(2015). Intensive math instruction and educational attainment: Long-run impacts of double-dose algebra. Journal of Human Resources, 50(1), 108–158. https://doi.org/10.3368/jhr.50.1.108

13.

Coyle

T. R.

Purcell

J. M.

Snyder

A. C.

Richmond

M. C.

(2014). Ability tilt on the SAT and ACT predicts specific abilities and college majors. Intelligence, 46, 18–24. https://doi.org/10.1016/j.intell.2014.04.008

14.

de Ree

Maggioni

M. A.

Paulle

Rossignoli

Walentek

. (2021). High impact tutoring in pre-vocational secondary education: Experimental evidence from Amsterdam. SocArXiv. https://doi.org/10.31235/osf.io/r56um

15.

Deci

E. L.

Ryan

R. M.

(2012). Self-determination theory. In Van Lange

P. A. M.

Kruglanski

A. W.

Higgins

E. T.

(Eds.), Handbook of theories of social psychology (pp. 416–436). Sage. https://doi.org/10.4135/9781446249215.n21

16.

Dougherty

Fleming

(2012). Getting students on track to college and career readiness: How many catch up from far behind?ACT Research Report Series. https://files.eric.ed.gov/fulltext/ED542022.pdf

17.

Duncan

G. J.

Dowsett

C. J.

Claessens

Magnuson

Huston

A. C.

Klebanov

Pagani

L. S.

Feinstein

Engel

Brooks-Gunn

Sexton

Duckworth

Japel

(2007). School readiness and later achievement. Developmental Psychology, 43(6), 1428–1446. https://doi.org/10.1037/0012-1649.43.6.1428

18.

Duran

(2017). Learning-by-teaching. Evidence and implications as a pedagogical mechanism. Innovations in Education and Teaching International, 54(5), 476–484. https://doi.org/10.1080/14703297.2016.1156011

19.

Egan

K. L.

Davidson

A. H.

(2017, November 14). Alignment of the NWEA MAP growth and MAP growth K–2 to the common core state standards: English language arts and mathematics. EdMetric.

20.

Flores

(2007). Examining disparities in mathematics education: Achievement gap or opportunity gap? The High School Journal, 91(1), 29–42. https://www.jstor.org/stable/40367921

21.

Gershenson

(2016). Linking teacher quality, student attendance, and student achievement. Education Finance and Policy, 11(2), 125–149. https://doi.org/10.1162/EDFP_a_00180

22.

Guryan

Ludwig

Bhatt

M. P.

Cook

P. J.

Davis

J. M.

Dodge

Farkas

Frye

R. G.

Jr. Mayer

Pollack

Steinberg

(2021). Not too late: Improving academic outcomes among adolescents (NBER Working Paper No. w28531). National Bureau of Economic Research. https://doi.org/10.3386/w28531

23.

Hanushek

E. A.

(2023). Generation lost: The pandemic's lifetime tax. Education Next. https://www.educationnext.org/generation-lost-the-pandemics-lifetime-tax/

24.

Heckman

J. J.

Stixrud

Urzua

(2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411–482. https://doi.org/10.1086/504455

25.

Heinrich

C. J.

Darling-Aduana

Good

Cheng

(2019). A look inside online educational settings in high school: Promise and pitfalls for improving educational opportunities and outcomes. American Educational Research Journal, 56(6), 2147–2188. https://doi.org/10.3102/0002831219838776

26.

Hickey

A. J.

Flynn

R. J.

(2019). Effects of the TutorBright tutoring programme on the reading and mathematics skills of children in foster care: A randomised controlled trial. Oxford Review of Education, 45(4), 519–537. https://doi.org/10.1080/03054985.2019.1607724

27.

Howard

J. L.

Bureau

Guay

Chong

J. X.

Ryan

R. M.

(2021). Student motivation and associated outcomes: A meta-analysis from self-determination theory. Perspectives on Psychological Science, 16(6), 1300–1323. https://doi.org/10.1177/1745691620966789

28.

Imbens

G. W.

Rubin

D. B.

(2015). Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.

29.

Institute of Education Sciences. (2022). What works clearinghouse standards handbook (Version 5.0). U.S. Department of Education. https://ies.ed.gov/ncee/WWC/Docs/referenceresources/Final_WWC-HandbookVer5_0-0-508.pdf

30.

Joensen

J. S.

Nielsen

H. S.

(2009). Is there a causal effect of high school math on labor market outcomes? Journal of Human Resources, 44(1), 171–198. https://doi.org/10.3368/jhr.44.1.171

31.

Karsenty

(2010). Nonprofessional mathematics tutoring for low-achieving students in secondary schools: A case study. Educational Studies in Mathematics, 74, 1–21. https://doi.org/10.1007/s10649-009-9223-z

32.

Kotok

(2017). Unfulfilled potential: High-achieving minority students and the high school achievement gap in math. High School Journal, 100(3), 183–202. https://www.jstor.org/stable/90024211

33.

Kraft

M. A.

(2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253. https://doi.org/10.3102/0013189X20912798

34.

Kraft

M. A.

Falken

G. T.

(2021). A blueprint for scaling tutoring and mentoring across public schools. AERA Open, 7(1), 1-21. https://doi.org/10.1177/23328584211042858

35.

Kraft

M. A.

Novicoff

(2024). Time in school: A conceptual framework, synthesis of the causal research, and empirical exploration. American Educational Research Journal, 61(4), 724–766. https://doi.org/10.3102/00028312241251857

36.

Kraft

M. A.

Schueler

Falken

(2024). What impacts should we expect from tutoring at scale? Exploring meta-analytic generalizability (EdWorkingPaper No. 24-1031). Annenberg Institute at Brown University. https://doi.org/10.26300/zygj-m525

37.

Larson

Boswell

(2019). Big ideas math: Algebra 1. A common core curriculum. Big Ideas Learning.

38.

Lohmeier

J. H.

Lee

S. W.

(2011). A school connectedness scale for use with adolescents. Educational Research and Evaluation, 17(2), 85–95. https://doi.org/10.1080/13803611.2011.597108

39.

Long

M. C.

Iatarola

Conger

(2009). Explaining gaps in readiness for college-level math: The role of high school courses. Education Finance and Policy, 4(1), 1–33. https://doi.org/10.1162/edfp.2009.4.1.1

40.

Moffatt

(2020). Experimetrics: Econometrics for experimental economics. Bloomsbury Publishing.

41.

National Assessment of Education Progress (NAEP). (2024). NAEP US math score trends. https://www.nagb.gov/naep/mathematics.html

42.

National Assessment of Education Progress (NAEP). (2022). Reading and mathematics scores decline during the pandemic. https://www.nationsreportcard.gov/highlights/ltt/2022/

43.

National Center for Education Statistics (NCES). (2022). Revenues and expenditures for public elementary and secondary education: FY 20. U.S. Department of Education. https://nces.ed.gov/pubs2022/2022306.pdf

44.

National Council of Teachers of Mathematics. (2014). Principles to actions: Ensuring success for all. National Council of Teachers of Mathematics. https://www.nctm.org/Store/Products/Principles-to-Actions–Ensuring-Mathematical-Success-for-All/

45.

National Research Council and Mathematics Learning Study Committee. (2001). Adding it up: Helping children learn mathematics. National Academies Press.

46.

Nickow

Oreopoulos

Quan

(2020). The impressive effects of tutoring on pre-K–12 learning: A systematic review and meta-analysis of the experimental evidence (NBER Working Paper No. w27476). National Bureau of Economic Research. https://doi.org/10.3386/w27476

47.

Nomi

Allensworth

(2009). Double-dose Algebra as an alternative strategy to remediation: Effects on students’ outcomes. Journal of Research on Educational Effectiveness, 2(2), 111–148. https://doi.org/10.1080/19345740802676739

48.

Northwest Evaluation Association (NWEA). (2019). MAP Growth technical report. https://www.nwea.org/uploads/2021/11/MAP-Growth-Technical-Report-2019_NWEA.pdf

49.

Office of State and Grantee Relations (OESE). (2024). Elementary and secondary school emergency relief fund. Formula Grant. https://oese.ed.gov/offices/education-stabilization-fund/elementary-secondary-school-emergency-relief-fund/

50.

Oklahoma Educational Quality and Accountability. (2024). Oklahoma school report cards. https://schoolreportcards.ok.gov/

51.

Randall

Engelhard

(2010). Examining the grading practices of teachers. Teaching and Teacher Education, 26(7), 1372–1380. https://doi.org/10.1016/j.tate.2010.03.008

52.

Robinson

C. D.

Kraft

M. A.

Loeb

Schueler

B. E.

(2021). Accelerating student learning with high impact tutoring. EdResearch for Recovery Project. https://files.eric.ed.gov/fulltext/ED613847.pdf

53.

Roscoe

R. D.

Chi

M. T.

(2007). Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research, 77(4), 534–574. https://doi.org/10.3102/0034654307309920

54.

Rose

Betts

J. R.

(2004). The effect of high school courses on earnings. Review of Economics and Statistics, 86(2), 497–513. https://doi.org/10.1162/003465304323031076

55.

Siegler

R. S.

Braithwaite

D. W.

(2017). Numerical development. Annual Review of Psychology, 68, 187–213. https://doi.org/10.1146/annurev-psych-010416-044101

56.

Taylor

Jungert

Mageau

G. A.

Schattke

Dedic

Rosenfield

Koestner

(2014). A self-determination theory approach to predicting school achievement over time: The unique role of intrinsic motivation. Contemporary Educational Psychology, 39(4), 342–358. https://doi.org/10.1016/j.cedpsych.2014.08.002

57.

Thum

Y. M.

Kuhfeld

(2020). NWEA 2020 MAP growth: Achievement status and growth norms tables for students and schools. NWEA. https://teach.mapnwea.org/impl/MAPGrowthNormativeDataOverview.pdf

58.

Trusty

Niles

S. G.

(2003). High-school math courses and completion of the bachelor's degree. Professional School Counseling, 7(2), 99–107. https://www.jstor.org/stable/42732549

59.

Tyson

Lee

Borman

K. M.

Hanson

M. A.

(2007). Science, technology, engineering, and mathematics (STEM) pathways: High school science and math coursework and postsecondary degree attainment. Journal of Education for Students Placed at Risk, 12(3), 243–270. https://doi.org/10.1080/10824660701601266