Tutoring the Tutors: Piloting Online Modules for Tutor Training

The Rise of Tutoring Programs

Although there is some variation in describing tutoring, a predominant definition is “one-on-one or small-group human (i.e., non-computer) instruction aimed at supplementing classroom-based education” (Nickow et al., 2020). Tutoring incorporates learning interactions that focus on an area of curriculum content that needs improvement or strengthening (Roe & Vukelich, 2001; Woolley & Hay 2007), and although it involves aspects of pedagogy, tutoring is not teaching and should not dilute or replace teaching (Topping, 1998). Tutoring can also be implemented through technology (Robinson & Loeb, 2020) but still requires interactions between a tutor and tutee.

Evidence has overwhelmingly affirmed the positive effects of tutoring on student learning (Cortes et al., 2024; Guryan et al., 2023; Loeb et al., 2023; Ritter et al., 2009). This staunch evidence base prompts persistent calls to expand tutoring services across U.S. public schools (Kraft & Falken, 2021) to reduce COVID-19 learning loss (Kraft & Goldstein, 2020; White House Council of Economic Advisors, 2023). For instance, in a meta-analysis of rigorous causal evidence, Nickow et al. (2020) found consistent and large pooled effect size of 0.37 standard deviations for tutoring, though there was meaningful variation across individual studies. Recently, the White House Council of Economic Advisors (2023) released analyses of the effects of an Illinois statewide tutoring effort, finding a significant jump in the National Assessment of Educational Progress test scores relative to other states. Importantly, positive effects were strongest in programs that were conducted by professionals (e.g., teachers), in earlier grades, and within school.

The heterogeneity of who carries out the tutoring has often been explained as understanding the tutor skill, role, or experience. That is, programs vary in who serves as a tutor; tutors include older students, educators, and parents. However, there is a difference in effect by these roles. For example, studies that examine difference in tutors have identified that professional educators (e.g., teachers) have larger effects than those who have no education background (e.g., parents, nonprofessionals; Lopez et al., 2020; Robinson et al., 2021). Further, evidence indicates that trained tutors are more effective than untrained tutors (Fresko & Chen, 1989). This emphasis on tutor training changes the discussion from differentiating who should serve as a tutor, to how they should be trained, described next.

Tutor Training

Evidence has established that preparing tutors is valuable for student learning (Fountas & Pinnell, 1996; Roe & Vukelich, 2001; Waltz, 2019; Woolley & Hay 2007) and has shown some positive effects on the tutors themselves (Cohen et al., 1982; Topping, 1998). For instance, Skelley et al. (2022) identified that tutor training (debriefing sessions and reflections) increases tutors’ confidence. Specifically, these tutors were initially hesitant and insecure about their role and responsibility as a tutor, but after training, there were significant increases in their self-perception after they realized that they could learn from their peer tutors. However, the training was individually designed and provided to the sample of only eight tutors for elementary literacy.

Despite the efficacy of and demand for tutoring (Hampel & Stickler, 2005), there are some challenges to establishing tutoring programs from a structural standpoint. White et al. (2022) found through a systematic literature review that successful tutoring implementation often hinges on the support of key school leaders who have the power to direct the use of school funding, space, and time. Additionally, the authors found that tutoring setting and schedule, tutor recruitment and training, and curriculum identification influence whether students can access tutoring services, and the quality of the instruction provided.

In terms of what high-quality tutor training should entail, some have pointed to key characteristics. First, ongoing training and support throughout the tutoring duration are more beneficial than a one-time introductory training session (Hill, 2016; Skelley et al., 2022). Second, training should include endorsement from the tutors, meaning that tutors should find value in the training as applied to their work (Hampel & Stickler, 2005; Stickler & Hampel, 2007); however, this is less assured, given that evidence was from individual case studies and only within online contexts. Finally, training must be specific to tutors’ work and include strategies for potential challenges that may arise (Topping, 1998).

Perhaps the most comprehensive suggestions come from the National Student Support Accelerator (NSSA), a leading source for tutoring research in the United States. Specifically, NSSA offers recommendations for tutor preparation, including tutor expectations, content proficiency, and supporting students with learning and thinking differences. In fact, this is the only framework that we have encountered that offers explicit guidance on how to train and support tutors. However, these recommendations are built from a limited empirical base, and NSSA does not provide any citations that support these recommendations for training tutors. This, in conjunction with the aforementioned studies, largely draws from weak data from a hyper-specific context and offers little empirical evidence for how to effectively train tutors. That is, programs often draw from anecdotal best practices, as opposed to research-based strategies on how to train tutors. To successfully realize tutoring as both a scalable, high-quality educational intervention and a pipeline to the teaching profession, we need to establish best practices around the training and support of new tutors. To contribute to this emerging evidence base, our study examines one model to prepare tutors for their work.

Deans for Impact Intervention

Deans for Impact (DFI) is a national nonprofit organization committed to ensuring that every child is taught by a well-prepared teacher. DFI works toward this vision through policy advocacy and programmatic efforts to ensure that pathways into teaching are affordable, practice based, and focused on high-quality instruction. Guided by principles of learning science, DFI aims to help teachers create rigorous, equitable and inclusive classrooms so that all children thrive. In the fall of 2023, DFI launched the Aspiring Teachers as Tutors Network, a national collaborative of 28 tutoring initiatives across 15 states that are working to increase the number of aspiring teachers serving as tutors for K–12 students and to improve preservice teachers’ instructional skills through field experiences and training.

As part of the Aspiring Teachers as Tutors Network, DFI sought to examine the efficacy of tutor training for aspiring teachers and the conditions associated with implementing the training across diverse tutoring programs. DFI designed a series of training modules for aspiring teachers serving as tutors that were adapted from those originally developed in partnership with educator preparation programs as part of a high-quality instructional materials network. The training focused on key components of instructional decision-making and were designed to be embedded into educator preparation programs, enabling programs to integrate tutoring experiences within existing coursework required to complete a preparation program, or into ongoing training provided by tutoring providers outside of teacher certification programs.

DFI offered a series of four asynchronous training modules to prepare tutors on elements of effective high-impact tutoring. Each of the modules follows a research-based sequence that allows tutors to understand, analyze, and apply the concepts covered in the materials. Specifically, tutors (1) build background knowledge by exploring a key concept or practice, (2) explore strategies related to the practice, grounded in authentic instructional artifacts, and (3) complete a culminating activity in which they respond to a real-world example.

The four modules included approximately 24 hours of content in K–8 mathematics, ¹ representing the knowledge and skills requisite for developing successful high-impact tutors. The modules are intended to be completed in sequence and in full, and prepare aspiring teachers and tutors to do the following (see Appendix A for a more detailed description of each module):

Participating Program Context

Four sites partnered with DFI to complete the modules for this pilot study. A convenience sample was used from the DFI network of programs to identify sites that wanted to leverage more robust training, had a decent sample size, and were committed to piloting in sequence and in full with their candidates and tutors. The sites are anonymized but briefly described here (full details are in Appendix B):

Each program was provided with links to the four asynchronous training modules. Training was provided to faculty and staff on best practices for incentives, support, accountability structures, sequencing, and pacing of the modules to ensure completion. DFI staff also provided ongoing support via email communication, phone, and Zoom, including an initial onboarding and subsequent training session around module content. Tutors completed all training modules during the spring 2023 semester.

Data Collection

DFI staff collected two main types of data throughout the pilot study. A pre-assessment and post-assessment were electronically administered to all participating tutors at each of the four sites; descriptive statistics of participants are shown in Table 1. In total, 211 participants completed at least one of the assessments (with 166, or 77.7%, completing both); the overwhelming majority were female (91.0%), and nearly half were white (46.5%). However, only 65% of individuals were currently enrolled in an educator preparation program, indicating the variety in programs and career aspirations.

OPEN IN VIEWER

Table 1.

Sample Descriptives by Site.

	Site A	Site B	Site C	Site D	Total
N (%)	61 (28.9)	88 (41.7)	46 (21.8)	16 (7.6)	211 (100)
Gender
Female	59 (96.7)	84 (95.5)	34 (73.9)	15 (93.8)	192 (91.0)
Male	1 (1.6)	1 (1.1)	11 (23.9)	0 (0)	13 (6.2)
Other	0 (0)	2 (2.3)	0 (0)	0 (0)	2 (1.0)
Prefer not to answer	1 (1.6)	1 (1.1)	1 (2.2)	1 (6.3)	4 (1.9)
Race and Ethnicity
Black, Afro-Caribbean/American	1 (1.6)	13 (14.8)	14 (30.4)	1 (6.3)	29 (13.7)
Latinx or Hispanic American	1 (1.6)	60 (68.2)	3 (6.5)	3 (18.8)	67 (31.8)
Other	1 (1.6)	1 (1.1)	6 (13.0)	1 (6.3)	9 (4.3)
Prefer not to answer	0 (0.0)	3 (3.4)	5 (10.8)	0 (0.0)	8 (3.8)
White or European American	58 (95.1)	11 (12.5)	18 (39.1)	11 (68.8)	98 (46.5)
Current enrollment in an EPP
No	4 (6.6)	26 (29.6)	30 (65.2)	14 (87.5)	74 (35.1)
Yes	57 (93.4)	62 (70.5)	16 (34.8)	2 (12.5)	137 (64.9)

Note. EPP = education preparation program.

The pre-assessment was completed before starting the modules, and the post-assessment upon completion of the modules. ³ These assessments collected changes in tutor knowledge and skills, inquiring about participant understanding and application of the core knowledge and skills necessary to make effective instructional decisions that support all learners. These were predominantly through detailed case study illustrations, in which participants considered how to approach various tutoring scenarios. Although the pre- and post-assessment used the same questions, tutors did not receive a grade or any feedback on right or wrong answers. Further, sufficient time passed (i.e., a semester), such that the change in the percentage of correct answers between pre and post was likely not simply an artifact of increased familiarity with the assessment. After the assessment, DFI also captured descriptions of changes in tutor mindset and beliefs about (1) students, (2) the work of teaching mathematics, (3) tutoring, and (4) standards-aligned instruction and high-quality instructional materials. Finally, they queried changes in their preparedness to teach mathematics and plans to stay in teaching.

DFI also requested interviews with all training leads and at least one tutor per program. Training leads were individuals, often program administrators or faculty members, responsible for setting up, implementing, and monitoring progress of the tutoring modules. A 30-minute semi-structured interview was conducted using an interview protocol designed to identify participant experiences related to module implementation. Following are descriptions of interviewed participants by program; all were given pseudonyms.

Data Analysis

We took a concurrent mixed-methods approach (Tashakkori & Teddlie, 2021) to allow for separate related analyses of different type of data: quantitative analysis of the pre-/post-assessments, and qualitative analysis of participant interviews.

Quantitatively, we examined changes on the pre-/post-assessment, and we considered variation across key tutor demographics and other relevant characteristics—with our focus being on understanding change in their measured skills, career plans, and knowledge. In addition to reporting the raw pre–post changes, we also estimated least squares regression models that predicted post-assessment measures as a function of tutor demographics, controlling for the relevant pre-assessment measure. Because we examined pre–post changes, our analytic sample was restricted to the 166 tutors who responded to both assessments. Appendix Table C1 shows a comparison of characteristics between included and excluded tutors. Excluded tutors were much more likely to be from Site C and report their race as something other than Black, Latino, or White. ⁴ We report results from two model specifications—with and without site fixed effects. The inclusion of site fixed effects accounts for the nonrandom sorting of tutors, given that sites differed substantially in terms of their demographic composition. However, we could not rule out any other program or campus-level experiences that may have accounted for changes in participant assessment.

One open-response item queried tutor knowledge on HQIM elements. HQIM are predesigned vetted pedagogical materials (e.g., lesson plans) that are provided to instructors to reduce time in preparation, and are increasingly being throughout the United States. We used the four elements explicitly stated from the modules and categorized responses by each element. We allowed for the idea to be conveyed, as opposed to direct use of the terms; in practice, though, participants mostly used the actual terms in their responses.

Qualitatively, we transcribed the interview data verbatim and imported the transcripts into the qualitative software, Dedoose. From there, we engaged in constant comparative analysis (Glaser & Strauss, 2017) to identify main patterns of tutors’ experiences of the tutor training intervention. We began by open coding within each program, which allowed us to highlight individual programmatic differences. After identifying patterns within program, we conducted axial coding to identify key patterns across tutoring programs for characteristics of module implementation, suggestions for improvement for the tutoring training, and major areas of value for trainers and tutors.

RQ1: How did tutor knowledge, feelings of preparedness, and career plans change throughout implementation?

We probed the participant experience by examining what tutors found valuable for their own development and how the modules accentuated their contextual teaching and learning. Participants improved their instructional knowledge and skills as measured by the pre- and post-assessments (see Figure 1). On average, participants improved by roughly 20 percentage points (in terms of the percentage of correct responses on the assessment). Using a dependent t test with heteroskedasticity-robust standard errors, we found that this change was statistically significant at a 99% confidence level. The pre–post change was approximately the same across each of the four sites, evidenced by the parallel lines in Figure 1. Although there was substantial variability in terms of the average score on the pre-assessment, tutors from each of the four sites grew by the same amount, on average.

OPEN IN VIEWER

Figure 1.

Improvement in score from pre- to post-assessment by site.

We also examined changes in the dispositional items on the pre- and post-assessments, shown in Table 2. These items asked tutors to rate their agreement with the statement on a 1–10 scale, with 10 being most in agreement. We compared the average rating for the pre- and post-assessments using the common sample of tutors who responded to both instruments. In general, we observed changes in attitudes and beliefs that were consistent with the assessment results showing improved skills. Tutors were more likely to respond negatively to statements that did not align with HQIM principles, such as prioritizing materials based on fun or engagement rather than alignment with standards (4.0 pre-assessments to 2.5 post-assessments). The smallest observed change was in taking ownership over student mastery. Here, there was a slight increase, from 6.5 to 7.1.

OPEN IN VIEWER

Table 2.

Pre–Post Responses to Dispositional Questions.

Question	Pre-Assessment	Post-Assessment
If I could only prioritize only one thing, I would select materials based on how fun and engaging they look over the degree to which they align with a particular standard.	4.0	2.5
One of the most helpful ways you can prepare to support students with a math lesson is to think about how you were taught to solve similar problems and come ready to teach those strategies.	5.2	3.9
When students struggle with grade-level content, effective instructional supports include giving them tasks from a lower grade and calling on them for the easiest questions in your lesson. By meeting them where they are, your students are more likely to stay engaged and feel comfortable and successful.	6.0	4.6
If at the end of our lesson a student hasn’t mastered what we were working on, that probably has more to do with how I structured the time and supported them than their effort or ability.	6.5	7.1
To be successful in mathematics, all students really need are the standard algorithms for each operation. Most of the time, learning multiple strategies to solve a problem is more confusing than it is helpful.	4.2	3.2
Please indicate how well-prepared you currently feel to teach K–8 mathematics.	5.3	7.9

Note. Scores are mean of 1–10 scale.

To understand variability by tutor characteristics, we estimated regression models using the post-assessment response as a dependent variable (controlling for the pre-assessment response) shown in Table 3. Panels A and B show results without and with site fixed effects, respectively. Focusing first on Panel A, we found several statistically significant differences based on tutor race and ethnicity, though not for gender. On average, Black and Latinx tutors scored lower on the post-assessment relative to White tutors. Column 1 shows the magnitude of these gaps: 15 percentage points for Black tutors and 8 percentage points for Latinx tutors. The remaining columns examine the 1–10 scale responses to the dispositional questions. There were statistically significant differences between White and Black tutors for two questions: taking responsibility for lack of student mastery (Black tutors responded 1.5 points less favorably, on average), and belief that students do not need multiple strategies for math problem-solving (White tutors responded 0.7 points less favorably, on average). Latinx tutors were more likely to endorse making things easier for students struggling with grade-level content (1.5 points greater than White tutors) and believed that students did not need multiple strategies for math problem-solving (0.9 points greater). These differences were quite large relative to the overall variability in responses (the standard deviation was roughly 2 points for each). Although we could not pinpoint their origin, these gaps suggest that nonwhite tutors approached the training and work differently. Note that there were insufficient numbers of tutors from other racial or ethnic groups to conduct statistical analyses.

OPEN IN VIEWER

Table 3.

Predicting Post-Assessment Score and Dispositional Response Growth Using Tutor Demographics.

	Post-assessment score (proportion correct)	Prioritizefunmaterials	Support using prior experience	Make easier for strugglingstudents	Take responsibility for lack of mastery	Students don’t need multiple strategies
	(1)	(2)	(3)	(4)	(5)	(6)
Panel A: No Site Fixed Effects
Pre-assessment measure	0.53*** (0.07)	0.34*** (0.08)	0.47*** (0.09)	0.38*** (0.09)	0.36*** (0.08)	0.47*** (0.09)
Female	0.09 (0.08)	-0.97 (0.73)	1.41** (0.66)	0.07 (0.63)	0.32 (0.50)	-0.13 (0.64)
Black	-0.15** (0.06)	0.09 (0.49)	-0.23 (0.60)	0.48 (0.48)	-1.49*** (0.45)	0.74* (0.44)
Latinx	-0.08** (0.04)	0.45 (0.31)	0.32 (0.41)	1.49*** (0.47)	-0.18 (0.32)	0.93** (0.43)
Panel B: Site Fixed Effects
Pre- assessment measure	0.42*** (0.08)	0.31*** (0.09)	0.47*** (0.09)	0.35*** (0.09)	0.35*** (0.08)	0.43*** (0.09)
Female	0.11 (0.07)	-1.19* (0.71)	0.79 (0.63)	0.22 (0.69)	0.27 (0.55)	-0.51 (0.75)
Black	-0.08 (0.06)	-0.30 (0.56)	-0.04 (0.58)	-0.13 (0.59)	-1.17** (0.48)	0.52 (0.42)
Latinx	0.03 (0.05)	-0.34 (0.49)	0.04 (0.53)	0.74 (0.64)	0.29 (0.42)	0.38 (0.45)
N	166	166	166	166	166	166

Heteroskedasticity-robust standard errors in parentheses. For column 1, the dependent variable is the proportion of correct answers. For the remaining columns, the dependent variable is the respondents’ 1-10 rating for the item. See Table 1 for the full text of each item in Columns 2–6. Results in Panel B add controls for site fixed effects, such that gender and race, and ethnicity differences are estimated via within-site comparisons only.

*

p < .1. ^** p < .05. ^*** p < .01.

It is important to note that we could not pinpoint the origin of these gaps, and they should be interpreted with caution. As one way to probe these gaps, we could reestimate the models, controlling for site fixed effects. Black and Latinx tutors, in particular, were concentrated in just one or two sites, such that the results in Panel A could reflect site-level differences that were correlated with race and ethnicity. Controlling for site fixed effects (Panel B) restricts these comparisons to tutors who were in the same site, which effectively controls for any between-site differences that could affect the outcome. Doing this eliminates most of these gaps in terms of their statistical significance; given the small sample size, the gaps must be quite large to be statistically significant. These results may suggest that the differences by race and ethnicity in Panel A are explained by between-site differences, though we cannot speak definitively to what these between-site differences actually are.

To assess changes in feelings of preparedness, we examined pre–post differences in items that asked tutors to rate their preparedness to teach mathematics on a scale from 1 to 10. There was an additional item that explicitly asked tutors to rate whether their preparedness had increased as a result of the HQIM modules. We found strong alignment between both of these measures.

The bottom row of Table 4 shows that the mean rating of preparedness increased by 2.6 points (statistically significant at a 99% level) between the pre-assessment and the post-assessment (5.3 to 7.9 points). Figure 2 disaggregates this change by site. Here we found substantial differences. Specifically, there was large variability across sites in the feeling of preparedness on the pre-assessment. For instance, Site C tutors rated their preparedness at 7.3 (out of 10), on average, compared with 4.6 for Site A tutors. The juxtaposition with the assessment results is interesting: Site A tutors felt less prepared at the time of the pre-assessment but actually had the highest scores. However, we see a convergence in perceptions of preparedness to teach after completing the training modules. Despite the large differences on the pre-assessments, the post-assessments means were very similar. They were also quite high, on average, hovering around 8 out of 10, indicating that tutors felt fairly well prepared to teach.

OPEN IN VIEWER

Table 4.

Predicting Pre- and Post-Assessment Feelings of Preparedness and Commitment Using Tutor Demographics.

	Feel Prepared(pre)	Feel Prepared(post)	Feel Prepared(post)	More Commitment
	(1)	(2)	(3)	(4)
Panel A: No Site Fixed Effects
Pre- assessment measure			0.21*** (0.04)
Female	-1.70** (0.71)	-0.63(0.46)	-0.27(0.44)	0.19(0.69)
Black	0.53(0.44)	-1.04*** (0.38)	-1.15*** (0.37)	-0.60(0.59)
Latinx	0.33(0.36)	0.13(0.23)	0.06(0.22)	0.62** (0.27)
Panel B: Site Fixed Effects
Pre-assessment measure			0.26*** (0.05)
Female	-0.96(0.71)	-0.63(0.45)	-0.38(0.46)	-0.07(0.72)
Black	0.07(0.48)	-0.69* (0.41)	-0.70* (0.39)	-0.28(0.59)
Latinx	0.33(0.51)	0.65** (0.33)	0.56* (0.30)	0.82* (0.44)
N	166	166	166	166

Heteroskedasticity-robust standard errors in parentheses. The dependent variable is the respondent’s 1–10 rating for the item. Results in Panel B add controls for site fixed effects, such that gender, and race and ethnicity differences are estimated via within-site comparisons only.

*

p < .1. ^** p < .05. ^*** p < .01.

OPEN IN VIEWER

Figure 2.

Change in feeling of preparedness from pre- to post-assessment by site.

Tutors’ feelings of preparedness increased—what about their commitment to teaching? Figure 3 shows the frequency distribution based on the pre-assessment and post-assessment responses. Here, instead of a 1–10 rating, tutors choose one of several possible time ranges, from “I do not plan on teaching” to “I plan to stay more than 20 years.” Overall, tutors’ intended commitment to teaching was high at baseline and did not change much after they completed the training modules. Just under half of tutors reported that they planned to teach for more than 20 years, with most of the remaining respondents in the intermediate ranges of five to 20 years. Only a small handful of tutors reported that they did not plan on going into teaching or that they would remain for just one year. As noted earlier, there was essentially no change in the distribution at the post-assessment. Further, the stability of individuals’ responses was high, with a correlation coefficient of 0.82. This indicates strong alignment between what an individual tutor reported on the pre-assessment and what they reported on the post-assessment. Thus, the training did not appear to appreciably change commitment, but it also started from a very high baseline.

OPEN IN VIEWER

Figure 3.

How long do you plan to stay in teaching? Pre-/post-assessment differences.

Table 4 shows results from regression models predicting feelings of preparedness and commitment by tutor demographics. Here, we observe that women felt substantially less prepared than men, on average, at the time of the pre-assessment (gap of 1.7 points). This gap was largely eliminated in the post-assessment. We found the opposite pattern for Black tutors; they reported slightly higher preparedness on the pre-assessment (0.5 points, not statistically significant) but substantially lower preparedness on the post-assessment (-1.0 points, p < .01), relative to White tutors. Latinx tutors had no significant differences relative to White teachers for preparedness, but they reported feeling more committed to teach after completing the training modules.

Despite some evidence of differences by demographics, there was strong evidence that overall, tutors found the training valuable in terms of increasing their preparedness for and commitment to teaching. Figure 4 shows mean responses from a post-assessments scale that asked respondents to specifically evaluate changes in these constructs as a result of the training. For each of the four, the mean rating was slightly above 8 out of 10, indicating strong agreement. This evidence is consistent with the pre–post changes for feeling of preparedness, which also showed a substantial increase.

OPEN IN VIEWER

Figure 4.

After completing the HQIM modules.

Next, we considered the relationship among the various constructs. Do higher scoring tutors (in terms of the assessment) feel more prepared to teach? Are they also more committed? We examined these relationships using a simple correlation matrix for the post-assessment responses, which is shown in Table 5. A higher positive number (up to 1.0) indicates a stronger relationship. Generally, we found that commitment and feelings of preparedness to teach were only weakly correlated with performance on the post-assessment. These correlations ranged from .08 to .15 and were only marginally statistically significant. We did find, however, evidence of a moderate positive relationship between feelings of preparedness and commitment to teaching or plans to stay. That is, tutors who perceived themselves as more prepared to teach were more likely to report that the training increased their commitment to teaching (r = .54) and that they planned to stay in teaching for a longer time (r = .22). Plans to stay and feeling more committed as a result of the training were also positively related (r = .41).

OPEN IN VIEWER

Table 5.

Correlation Matrix of Post-Assessment Items.

	(1)	(2)	(3)	(4)
1. Score
2. Prepared to teach	0.15
3 .Plan to stay	0.08	0.22
4. More committed	0.10	0.54	0.41

Finally, to examine the extent to which tutors processed the underlying intent of tutoring modules, we analyzed responses to the open-ended item, “What are the key elements of high-quality instructional materials?” We coded it against the four elements listed throughout modules: aligned to standards, coherence, focus, and rigor. Figure 5 illustrates the percentage of respondents who answered each element correctly by assessment. Tutors increased in their knowledge of each element across assessments, though minimally for the element of focus. This indicated some learning of these elements from the modules. Relatedly, of the key elements, tutors best understood that high-quality instructional materials should be aligned to standards. Overall, though, a large contingent of tutors were unable to directly name these key elements, drawing into question the tutoring knowledge gained.

OPEN IN VIEWER

Figure 5.

Percentage of respondents for each key element of HQIM, pre- and post-assessment.

Figure 6 similarly examines the number of key elements that participants got correct by program. On average, tutors increased in the number of correct key elements listed from pre- to post-assessment. However, the mean number correct by program hovered around one, with little difference by program, indicating minimal participant knowledge of four elements of identifying high-quality instructional materials.

OPEN IN VIEWER

Figure 6.

Change in key elements across pre-/post-assessment by program.

RQ2: What are participant experiences with the tutor training intervention?

Because we identified some tutor development from our quantitative findings, we aimed to explore how the modules may have facilitated this growth. Through participant interviews, three themes arose across their responses: module accessibility, pedagogical development, and training considerations. In other words, participants spoke about how accessible the modules were and what they learned, and they offered suggestions to improve modules.

Module Accessibility

The most salient feature among trainers was the ease of module integration into existing course structures. ⁵ Often, content within modules readily aligned with concurrent course content, reiterating established course goals for developing teaching skills. For instance, one trainer noted, "There were a lot of connections between the modules and the course as already designed. . . . The modules reinforced what we were already talking about in class” (Dr. Shannon). Some of the content alignment was facilitated by the trainers, complemented through consistent in-class discussion: “Each particular week or every two weeks, [the module content] was embedded in our reflections for the week . . . [I was] continuing to try to weave it into our weekly conversations with our tutors” (Ms. Jessica). The modules bolstered the coursework or training program, supporting connections between theory and application:

We don’t want them to just learn about the principles but to see it in real time. A large part of their course grade is designing a study and testing an intervention, so that directly linked to the work they were doing with tutoring. (Dr. Kelly)

Trainers appreciated how the modules engaged tutors by having them

read, check out this video, process and try it out. I think that was a lot better than read an article, answer some multiple-choice questions. So they thought about it a lot more. That performance-based approach is really good and one of the big strengths. (Dr. Thomas)

On the other hand, accessibility problems—often around technology— were present throughout the learning management system that housed the training modules. Simple navigational issues prevented access to the content of the modules, where tutors “had to have a lot of tabs open with the links. So much clicking around because tabs were so embedded” (Lyndsey). There were also challenges in saving work, resulting in tutors having to redo or resubmit assignments: “Even though supposedly they were able to complete it piecemeal, they were completing it two or three times, just because they were opening different tabs, I think. So, I think it was a little bit frustrating” (Dr. Kelly). Finally, support was not always available to troubleshoot larger technical issues, as stated by Dr. Roberts: "The biggest obstacle was not having university support for learning management system." Thus, participants seemingly wasted valuable time dealing with undue technological challenges.

Pedagogical Development

Tutors reported positive tutoring experiences that facilitated their pedagogical development. First, tutors valued learning a plethora of practical skills. By the end of the training, tutors could list an array of strategies that they had learned:

Doing things step-by-step and giving specific praise was a good thing that I learned and integrated into my tutoring. For sure the how and why questions were really helpful as well as think aloud. I had never thought about that before. And also modeling. I didn’t see the point of reading something for them or showing them how you do it, so that was helpful. Allowing them to think of the answer themselves before jumping in and giving them the answer. And coming prepared with questions and how to address misconceptions. (Katie)

Likewise, Lyndsey was able to list various teaching strategies that she found helpful: “Breaking it down and explaining it to the kids is really helpful. Using the anchor charts and visual models. And explaining step by step. And then the turn and talks. Modeling and thinking aloud piece.”

Of these pedagogical strategies, tutors found particular value in lesson internalization; this was defined in the modules as “the preparation a teacher engages in to make sense of a unit and prepare to teach a particular lesson,” including learning “how curricula are structured, how to navigate curricular resources, and how high-quality instructional materials can support the development of content knowledge and pedagogical content knowledge of teachers.” This essentially means understanding what students will process throughout a particular lesson, and it was best described by Angela:

The biggest takeaway for me was the internalizing. You have to put yourself in your students’ shoes, and as I was doing that with tutoring, I could see where they have misconceptions and I knew how to help them. . . . More than the self-management skills, it really helped me think about what I need to do to help my students.

Internalizing the lesson allowed tutors to focus their mental efforts on supporting students’ learning rather than on the procedural aspects of lesson implementation. The lesson internalization process also helped tutors explicitly understand the work that goes behind preparing for teaching:

Overall, it shows me how important lesson planning is—you have to think about how to explain this in ways, not just the ways you learned, but how students might be thinking. It taught me that teaching is a lot, but if you prepare yourself, your students will be successful. (Jason)

The value of lesson internalization was supported by trainers as well. As Dr. Roberts stated, “The lesson internalization was really helpful for the ‘why’—you don’t lesson plan just to do an activity. This lesson internalization allowed them to think more about the process of planning, what it looks like, why it looks like that.”

Tutors also expressed their enjoyment in building student relationships and witnessing their academic growth. As an example, Katie explained,

My favorite part was the relationship with the kids and seeing them grow in their confidence. I liked it when they were proud of themselves or responded well to praise and constructive feedback, when I would scaffold them into learning something new.

Another tutor reiterated his appreciation of “the kids being excited to come! You could see the development of relationships between February and May too” (Jason). Most poignantly, Jasmin reflected, “The wonderful thing was the amount of growth I saw in such a limited amount of time—less than 40 minutes, and the amount of growth I saw was just amazing. It was truly amazing.”

Training Considerations

Amid this training pilot, opportunities for module improvement arose. Foremost, tutors were often busy with full-time schedules and working on the modules took low priority or were difficult to finish in a timely manner. Jason explained, “We were tutoring for about eight hours a day and then trying to do this, so it was a lot.” Others went on to add, “[the modules were] labor intensive, and we had to fit that into 25 hours a week, that includes 15 hours of tutoring and prep time and this” (Jasmin). Trainers saw this burden and attempted to ameliorate it. For example, several programs adjusted expectations to accommodate tutors; “[we recognized] the length of time was really hard for our staff to juggle in addition to their workloads, so we were often more lax in changing deadlines” (Ms. Chelsea & Ms. Ashley), whereas another site “had them submit plans for two weeks at a time” (Dr. Roberts) to reduce the consistent workload.

Another improvement was to reduce the material throughout modules. Participants appreciated the content but suggested that it could be better consolidated. They stated, “The content could be really impactful, but it needs to be broken down in a different way” (Jason), and although “it was lengthy . . . [it was] still good information but felt like a slog” (Lyndsey). Rather, participants suggested using more creative modes of engagement: “more videos of teachers teaching kids—I can see it and hear it and how they are saying it” (Lyndsey), or “maybe a little less open-ended questions. Maybe you could do some game format to keep it interesting. . . . More varied and more choice in question response” (Jason).

A third recommendation was to consider transferring tutoring skills to other content areas. Because the modules were situated solely in mathematics, some tutors who were unfamiliar with the content or were placed in tutoring assignments for other content areas found the math content challenging to understand. Trainers specifically requested more support with math for their tutors. They expressed, “math language was overwhelming—felt very fish out of water” (Ms. Jessica) and that “one thing we are thinking about is supporting students with actual math content. . . . A lot of our candidates are not comfortable with the math content—getting into fraction concepts—in higher grades” (Dr. Shannon). Tutors also said, “[I] felt like I didn’t have the content knowledge, so I had an added step which took a very long time” (Jasmin), and “Since I was tutoring literacy, it was a bit difficult to make connections to math” (Katie). Further support in content or content transfer could accommodate more tutors.

Finally, tutors wished to learn more about differentiation to adapt content to individual students’ needs. Jason said, “I wish I would have had the learning about looking across grade-level standards to support the teaching.” Although Jessica appreciated the module on equitable instruction, she wanted to extend her learning:

I also think this program raises a lot of good points and ideas that we aren’t really talking about in other areas. Like being equitable beyond just SES, in terms of disability etc., which I experience. Some scholars were at different levels and learning how I accommodate all.

In this way, tutors could enhance their craft.

Limitations

Despite some promising findings, this study faces a number of limitations. First, the four tutoring programs are not nationally representative of tutoring programs. Rather, the sites included in this study were connected to and sampled by DFI as well as part of an aspiring teachers’ network; further research needs to identify how modules would be implemented in other contexts. Specifically, to get a better idea of whether tutoring and the module intervention have a relationship with tutors’ career plans, studies should consider identifying tutoring programs separate from educator preparation programs. Because of our sample’s relatively high commitment to the profession already, it was difficult to disentangle effects of professional commitment from individuals who had already largely committed to entering teaching (Redding & Baker, 2019). Studies can also follow whether tutors indeed enter the profession and for how long, particularly in comparison with individuals who either did not go through the tutoring intervention or were not tutors.

Second, although the pre–post design supports a cautious interpretation that tutors’ understanding of HQIM improved, there was no comparison group, which could have helped to rule out competing hypotheses, such as increased familiarity with the assessment items or the effects of other interventions (e.g., the other coursework that tutors may have been completing). Future studies using an experimental or quasi-experimental design could speak more credibly to the extent to which the modules improved tutors’ skills. Research could also consider additional measures of tutor engagement and effectiveness to better capture growth and to potentially validate our study instruments. Third, qualitative data could be more representative and triangulated. Site B did not have a tutor available to interview, Site C had multiple participants, and overall, tutoring experiences were retrospective in nature. This allowed for slight imbalance of reporting by program. Future studies could incorporate consistent interviewing throughout implementation, collect qualitative data on more participants, and incorporate observations, sample work, or other types of data to get closer to tutors’ experiences.

Implications

Despite limitations, our findings offer implications for policy, research, and practice. For policy, the intense attention to the effects of tutoring must now be balanced with supporting tutors. Because programs are looking to further expand, it is an opportune time to intentionally consider how to train and develop tutors. Tutor training appears to help tutors feel more prepared, and thus, policymakers need to invest in programs that do exactly that. Policymakers should ensure that tutoring programs are equipped to support tutors and offer appropriate curriculum so that tutors and ultimately, students can succeed.

For researchers, continued studies on tutors, their experiences, their growth and development, and their career trajectories remains underexamined. Although tutoring impacts student achievement, it has also been touted as a potential entry point for the teacher workforce. Studies need to investigate the extent to which tutors become teachers, as evidence would suggest (Kraft & Falken, 2021). Additionally, further studies are needed that focus on what supports may be needed to help tutors. Given empirical evidence on good tutoring practices, researchers need to identify how that information gets disseminated throughout programs and ensures tutor learning so that best practices can occur, and student learning acceleration can happen at scale. Finally, more data are needed on the context and experiences of tutors. Models should consider accounting for information, such as the types of schools and students that tutors work with, interactions they may have with school personnel, the quality of the curriculum that they use, and amount of modular engagement, to more precisely measure the effect of a tutoring intervention.

For practitioners, content, curriculum, and tutoring experience must be constantly evaluated and self-reflected on. Programs need to consider how they are training their tutors, identify appropriate tutoring curriculum, or create content around empirical strategies. They cannot assume that tutors have the requisite knowledge and skills to accelerate student learning and need to proactively provide or enhance training to ensure tutoring delivers consistent learning benefits for all students. There should also be some consideration of tutor selection, particularly around individuals who take ownership in student mastery. Identifying individuals who would want to invest in student success could be a distinct trait that expedites tutor development.