Teacher-Implemented Modified Schema-Based Instruction with Middle-Grade Students with Autism and Intellectual Disability

Setting

After receiving human subjects approval from the university and local school district, researchers recruited special education teachers by contacting all middle school principals of one school district in the southeastern United States. Three teachers from two schools within the same district participated in the study. School A was located in a suburban area and had 1,013 students enrolled in Grades 6 to 8 (70% White, 15% Black, 7% Hispanic), and 10.8% were identified as students with disabilities. School B was located in an urban area in the same city and met federal requirements for designation as Title 1. School B had 956 students enrolled in Grades 6 to 8 (30% White, 55% Black, 6% Hispanic), and 15.5% of the school population were identified as students with disabilities. All sessions, to the extent possible, were completed in a workroom attached to the teachers’ main classrooms during regularly scheduled math instruction.

Participants

Table 1 contains demographic information on teacher and student participants. Three special education teachers expressed interest and consented. The first teacher, Mr. Gibbs, was a White male with a bachelor’s degree in elementary education, 10 years of special education teaching experience, and no prior experience with MSBI. When asked about his current math instruction, Mr. Gibbs reported that his priority was for his students to think through problems and solve them independently, beginning with a focus on addition and subtraction. He discussed individualizing instruction based on student needs and being an “experimenter” who would try different things with students to see what helped them to understand concepts. Researchers observed students completing worksheets with a variety of mathematical content that Mr. Gibbs had downloaded from the internet, selected based on their Individualized Education Program (IEP) goals. The second teacher, Ms. Gonzo, was a Latinx female with a bachelor’s degree in special education. She was finishing requirements for her master’s degree in special education and was completing her student teaching under the supervision of Mr. Gibbs. Ms. Gonzo had research experience with the authors and had implemented MSBI to teach a different problem type to older students with ASD in a prior single-case study. Both Mr. Gibbs and Ms. Gonzo taught at school A.

OPEN IN VIEWER

Table 1.

Participant Demographics.

Teacher and dyad	Student	Race/ethnicity	Age	Grade	Sex	Eligibility area	Key Math a	Social and communication observations	Instructional setting
Ms. Capp, Dyad 1	Drake	Black	13	8th	M	ASD	BC: <1% OP: <1% APP: <1%	Observed joking with friends, responded verbally to questions, and initiated conversations	Academics: SE Electives: GE
	Carl	Black	13	8th	M	ASD	BC: <1% OP: 1% APP: <1%	Tentative, preferred written communication to verbal, and did not initiate conversations	Academics: SE Electives: GE
Ms. Gonzo, Dyad 2	Cole	White	13	8th	M	ASD/ID	BC: <1% OP: 1% APP: 1%	Enjoyed assisting peers in the classroom and communicating with adults, responded verbally to questions, and initiated conversations	Academics: SE b Academics: GE c Electives: GE
	Josh	White	12	6th	M	ASD/ID	BC: <1% OP: <1% APP: <1%	Easy going, responded verbally to questions but did not initiate conversations	Academics: SE Electives: GE
Mr. Gibbs, Dyad 3	Dane	White	13	6th	M	ASD	BC: <1% OP: 1% APP: 1%	Expressed confidence in academic abilities, responded verbally to questions, and initiated conversations	Academics: SE Electives: GE
	Eric	Asian	13	6th	M	ASD/ID	BC: <1% OP: <1% APP: <1%	Imitated behaviors modeled by peers, primarily communicated using scripting and echolalia	Academics: SE Electives: GE

Note. M = male; ASD = autism spectrum disorder; BC = basic concepts; OP = operations; APP = applications; SE = special education; GE = general education, ID = intellectual disability.

a

Scores reported as percentile rank by age equivalent.

b

English, Math, Science.

c

History.

The third teacher, Ms. Capp, was a White female with a bachelor’s degree in psychology and 3 years of experience in special education; she was enrolled in a master’s degree program in special education. She did not have prior experience implementing MSBI. When asked about her current math instruction, Ms. Capp reported she focused on functional skills and real-world applications such as time and money, but that a lack of resources and training was a barrier to her effectiveness in mathematics instruction. She reported that she had access to Unique Learning System curriculum (n2y, 2021) and sometimes used those materials but supplemented frequently based on her students’ IEP goals and current needs.

Student participants were recruited from these two classrooms. The special education teachers sent home recruitment fliers and consent forms to all students enrolled in the class who took the state’s alternate assessment aligned with alternate achievement standards. All students who returned consent forms and assented to participation were screened for inclusion. Student participants were screened by researchers to determine whether they had the prerequisite skills, including: (a) using one-to-one correspondence to count 1 to 20 objects, and (b) receptive and expressive identification of numerals up to 99. Details on the screening are provided in the procedures of this manuscript. A total of seven participants met inclusion criteria. Due to the coronavirus disease 2019 (COVID-19) pandemic, schools were closed before the seventh participant entered intervention; therefore, only six participants are reported in this manuscript. Standardized and diagnostic test information were not provided by the school to researchers. To obtain a standardized measure of students’ math performance, the second author administered the Key Math-3 (Connolly, 2007), a diagnostic assessment that provides details about a student’s current mathematics abilities, and is appropriate to use with students with disabilities, as they were included in the normative sample (Connolly, 2007). Heterogenous student dyads were created based on researcher observation.

Materials

Student participant materials for baseline and intervention sessions included: (a) researcher-created anchor videos; (b) worksheets displaying a problem-solving heuristic, multiplicative comparison word problem, and schematic diagrams; (c) manipulatives (i.e., unifix cubes); (d) calculators; and (e) writing utensils. During generalization sessions, materials included: (a) anchor videos, (b) generalization worksheets displaying a problem-solving heuristic and schematic diagrams, (c) researcher-created generalization problem videos, (d) manipulatives (i.e., unifix cubes), (e) calculators, and (f) writing utensils.

Anchor videos were 20 to 30 videos that depicted real-life multiplicative comparison scenarios in community locations (e.g., local lake, food trucks, plant nursery). All videos were created by student members of the research team in the local community and used locations participants would be familiar with. For example, one video featured a local lake and had voice-over narration stating the lake was home to ducks and turtles; another video showed two people in exercise apparel walking and had voice-over narration stating that visiting the lake with friends was a fun activity.

Worksheets used during baseline and intervention sessions displayed a heuristic for solving the problem and one word problem printed at the top of the page. Researchers wrote six multiplicative comparison word problems for the 12 community locations following guidelines by Spooner et al. (2017). The first line gave the context of the problem (e.g., “Eric went to the park to feed ducks.”). The second line gave the referent unit (e.g., “Eric saw a family of 3 people under one picnic shelter.”). The third sentence stated the compared value or scalar (e.g., “At another picnic shelter he saw 8 times as many people for a family reunion.”). The final sentence contained the problem statement (e.g., “How many people did he see at the family reunion under the second picnic shelter?). Once students selected a theme, that theme was crossed out. Therefore, problems were never repeated. Three schematic diagrams were given as choices to students (multiplicative comparison and two distractors). See Figure 1 for an example worksheet, problem, and schematic diagram.

Worksheets used during generalization sessions mirrored those used in intervention except they did not display a word problem at the top. Generalization problem videos aligned with the community locations from the anchors and word problems. For example, the generalization video aligned with the gardening theme depicting two young adults shopping at a local plant nursery and asked a question comparing the number of cacti purchased to the number of herbs purchased. Participants were able to pause and re-watch the video as needed.

Materials and procedures were reviewed for content validity. Specifically, the word problem guidelines, heuristic, teaching procedures, and schematic diagrams were reviewed by an expert in SBI who was not a member of the research team. Minor suggestions were made and followed regarding teaching procedures. An expert in middle-grade mathematics reviewed all word problems to ensure they met the guidelines and represented realistic multiplicative comparison scenarios. Problems were deemed similar in difficulty (i.e., followed the same four-sentence structure and used quantities between 0 and 100).

Procedures

Similar to prior MSBI research, participants that assented were given a screening to determine whether participants would benefit from the intervention (i.e., had prerequisite skills but could not already solve the targeted word problems). Screening tasks included: (a) counting with one-to-one correspondence; (b) receptive and expressive identification of single- and double-digit numbers; (c) receptive and expressive identification of mathematics symbols (i.e., percent, fraction, multiplication); (d) using a calculator to complete addition, subtraction, multiplication, and division calculations and write solutions; and (e) solving word problems.

Experimental Design and Measurement

A multiple probe across participants (dyads) design (Horner & Baer, 1978; Ledford & Gast, 2018) was used to evaluate the effects of teacher-implemented MSBI on word problem-solving. There were four experimental conditions: baseline, intervention, generalization, and maintenance. All student dyads began baseline simultaneously and three consecutive baseline data points were collected before researchers determined Ms. Capp’s students (Carl and Drake) would be the first student dyad to begin intervention due to scheduling needs among the three teachers. Once the first student dyad demonstrated a clear change in level in the primary dependent variable, the second student dyad was introduced to intervention. This procedure was repeated for the third student dyad. A minimum of five baseline data points were collected in the primary dependent variable for all participants in all phases in adherence to the What Works Clearinghouse (WWC) standards for single-case research designs (What Works Clearinghouse [WWC], 2020). Generalization was assessed once in baseline and once in intervention for each participant. Mastery criteria were 11 out of 12 problem-solving behaviors independently correct across the two independent practice problems (6 behaviors measured per problem) for two sessions. Maintenance data were intended to be collected 1 and 2 weeks after intervention. Data collection ended prematurely due to the COVID-19 pandemic’s closure of schools and cessation of research with human subjects.

The primary dependent variable was independence in solving word problems, measured by the number of problem-solving behaviors completed independently correct across two independent practice problems. Six behaviors were measured for each problem, giving students an opportunity to demonstrate 12 behaviors in each session. The operational definitions for each of the six measured behaviors are displayed in Table 2. The second dependent variable was cumulative problems solved, measured by the cumulative number of problems that had a correct numerical answer with a label (e.g., 12 pizzas). The third dependent variable was the generalization of problem-solving, measured in the same way as the primary dependent variable.

OPEN IN VIEWER

Table 2.

Operational Definitions for Each Measured Behavior of the Task Analysis.

Step	Behavior	Operational definition
1	Discover the problem type	Student states or writes correct problem type (multiplicative comparison) on the problem-solving heuristic.
2	Diagram the relationship	Student visually represents the problem by diagraming the values in an appropriate schema.
3	Write the equation	Student writes a mathematical equation with two equal sides including an equal sign and operation that represents the relationship in the problem.
4	Solve the problem	Student uses an appropriate strategy (e.g., multiplies with counters, tallies, etc.) to solve for an answer.
5	Write the answer a	Student states or writes the correct numerical answer with appropriate label (e.g., 12 pizzas).
6	Confirm answer makes sense	Student justifies their answer with reasoning. Methods include using a calculator, estimation, and writing a number sentence.

a

Behavior must be completed independently correct for the problem to be considered “solved correctly.”

Data Analysis

Visual analysis was the primary method for analyzing data. The first and second authors independently followed the following four-step process outlined by the WWC (2020) guidelines: (a) document predictable baseline pattern of data, (b) examine data within each phase to assess within-phase patterns (i.e., level, trend, variability), (c) compare data from each phase with data from adjacent phase to assess for presence of an effect (i.e., overlap, immediacy of effect, and consistency of patterns in similar phases), and (d) integrate all information to determine if there are at least three demonstrations of effect at three different points in time. In addition, an effect size measure (i.e., between-case standardized mean difference [BC-SMD]; Hedges et al., 2012) was used to evaluate the effect of the intervention. The online BC-SMD application (Gierut et al., 2015; Pustejovsky et al., 2021) was used to calculate an estimated effect size. BC-SMD is a method to estimate the magnitude of effect in a way that is comparable to a group-design Cohen’s d by comparing participant mean performance in adjacent comparable conditions (e.g., baseline to intervention). Effects can be interpreted as small (1.4), medium (3.6), or large (10.1).

Reliability and Fidelity

Interobserver agreement (IOA) was assessed for a minimum of 30% of sessions for each participant in each condition, exceeding WWC (2020) standards for single-case research designs. IOA was calculated using the item-by-item method by dividing the number of agreed items by the total number of agreements plus disagreements and multiplying by 100. During researcher-administered baseline probes, IOA was collected for an average of 52% of sessions across dyads and agreement was 100%. During teacher-led intervention lessons, IOA was collected for an average of 64% of sessions across dyads and mean agreement was 98.52% (range, 96%–100%) for Dyad 1, 97% (range 90%–100%) for Dyad 2, and 100% for Dyad 3.

Researchers assessed fidelity during baseline for the same sessions as IOA using a 10-item checklist to assess whether participants were provided with materials they needed, procedures were followed, and students were not provided with confirmatory or corrective feedback. After teachers completed or observed a session, they were provided an opportunity to debrief with a researcher. The teacher and researcher discussed any errors in fidelity, individual student strengths and any patterns evident in data collection. Fidelity was assessed for an average of 39% of baseline sessions across tiers and with a mean of 93%. Researchers assessed treatment integrity during intervention for the same sessions as IOA using a checklist to assess the degree to which the intervention was implemented as intended. The checklist included a total of 24 behaviors divided between the model (n = 7), guided practice (n = 8), and independent practice (n = 9) segments of the lessons (see Supplementary File 1). Errors were primarily a result of teachers not following the script closely and omitting steps. Coaching and feedback were provided to address fidelity errors, most frequently including reading the problem, reviewing X as a variable, and reviewing the vocabulary for each shape in the schema. Treatment integrity was assessed for an average of 68% of sessions across dyads and mean integrity was 97% (range, 88%–100%) for Dyad 1, 96% (range, 88%–100) for Dyad 2, and 100% for Dyad 3.

Word Problem-Solving

Figure 2 contains a graph of student problem-solving behaviors completed correctly during the two independent practice problems (closed circles) and generalization problems (open circles) across the three student dyads (tiers). The three sessions with no data following introduction of the intervention for each participant represent the 3 days of introduction lessons during which no data were taken as students did not have an opportunity to independently solve problems. Using procedures described above for visual analysis of the data shown in the graph in Figure 2, we concluded there was a functional relation between MSBI and participants’ independent problem-solving behaviors. Researchers further evaluated the effectiveness of the intervention using BC-SMD. Researchers used the restricted maximum likelihood (REML) model on the online calculator to allow for a more flexible analysis. BC-SMD calculates an average estimated effect of the magnitude of observed changes across participants. Mean differences between baseline and intervention performance estimate a medium effect (BC-SMD = 3.5, 95% confidence interval [CI] = [1.9, 5.1]).

OPEN IN VIEWER

Figure 2.

Graph of student problem-solving behaviors.

No participants were able to correctly solve problems in baseline, defined as the correct numerical answer and label. All participants were able to correctly solve problems during intervention. Sessions in which one problem was solved correctly are marked with a carrot (^) in Figure 2, and an asterisk (*) is above data points for sessions in which participants solved both problems. Given that each participant had a different number of opportunities to solve problems depending on how quickly they reached mastery criteria, cumulative totals are reported by participant below. The percent of total problems solved ranged from 22% to 90% (mean 56%) in intervention, 25% to 75% (mean 50%) in maintenance, and 0% to 100% (mean 70%) in generalization.

Baseline data demonstrate participants across all three student dyads were unable to independently solve the multiplicative comparison word problems with the materials alone (i.e., heuristic, schematic diagram, manipulatives, and calculator). Participants in the first student dyad participated in a research study on the effects of the virtual-representation-abstract (VRA) framework to teach word problem-solving in the prior school year (Root, Cox, et al., 2021), which may be the reason that their performance in baseline was slightly higher than their peers. With the exception of Drake, participants had stable patterns of behavior with no trend or counter-therapeutic trend in baseline. Drake did immediately improve after his first baseline session, when he observed his partner using the schema and calculator to multiply the numbers in the word problem, with his performance stabilizing after the second session. After introduction to MSBI, both participants in Dyad 1 demonstrated an immediate jump in level of performance, and minimal variability (within one point difference) and no overlap between baseline and intervention conditions. Neither Drake nor Carl solved any problems in baseline. Drake solved a cumulative total of 9/10 problems in intervention and 1/4 problems in maintenance. Cole solved a cumulative total of 9/10 problems in intervention and 3/4 problems in maintenance.

Dyad 2 also exhibited no overlap in performance between baseline and intervention, with Cole demonstrating an immediate jump in level, while Josh followed an increasing trend in performance. Josh needed the greatest number of intervention sessions (dosage) at 13. The teacher and researchers observed Josh was rushing through the problem-solving process (e.g., watching peer as he completed, then tossing his marker down and shouting “done”), and therefore, we decided to implement a positive reinforcement system contingent on behaviors completed independently correct. He earned 5 min of leisure time (e.g., playing basketball with researcher or independent tablet time) based on the number of boxes of the heuristic completed correctly on his independent practice problem. Josh’s performance immediately increased thereafter (10/12 steps), and while his performance dropped for one session (6/12), he then increased and maintained performance for the final four intervention sessions (range = 11–12 points). Josh solved a cumulative total of 5/22 problems in intervention. Cole solved a cumulative total of 8/12 problems in intervention.

Similar performance was demonstrated by both student participants in Dyad 3, with an immediate level change after introduction of MSBI. Although these participants demonstrated slightly higher levels of variability than the other two dyads, their ability to independently complete the problem-solving steps remained higher in intervention than baseline. Researchers and Mr. Gibbs observed both Dane and Eric in Dyad 3 were eager to complete the task but raced each other through the problems. Mr. Gibbs and researchers agreed to implement the contingent reinforcement system for both participants after spring break but were unable to do so due to COVID-19 school cancellations. Neither Dane nor Eric solved any problems in baseline. Dane solved a cumulative total of 4/14 problems in intervention. Eric solved a cumulative total of 6/14 problems in intervention.

Generalization to Video-Based Problems

Participants had two opportunities to demonstrate generalization to video-based problems, once during baseline and once during intervention, as shown by open circles in Figure 2. In baseline, Drake’s performance was consistent when presented with typed word problems or video word problems, and then increased to ceiling performance after intervention and he correctly solved both problems. Carl’s performance during baseline generalization was slightly lower but improved after intervention and he correctly solved both problems. Unfortunately, due to a school trip and then COVID-19, generalization data were not able to be collected for Cole after intervention. Josh had less success generalizing problem-solving from word problems to video problems, increasing from 0 correct in baseline to 2 after intervention but not solving the problem correctly. Dyad 3 student participants also exhibited similar performance for generalization probes during both conditions. During baseline, both Eric and Dane completed 0/12 independently correct for the video generalization word problems. After intervention, Eric was able to correctly solve one of the two problems and Dane solved both problems.

Social Validity

At the conclusion of the study, the three teachers were sent an electronic survey and had the opportunity to participate in interviews via Zoom with the second author who was their primary point of contact throughout the study. The teachers reported positive experiences with the intervention, and all would recommend involvement in similar studies to future students and other teachers. Two teachers stated the most beneficial aspect of participating in the research was the ongoing coaching sessions with the research team, where they were able to ask specific questions and were given immediate feedback. Teachers expressed continued insecurity with their pedagogical and content knowledge related to mathematics and felt they would benefit from additional professional development in this area. Finally, two teachers (Ms. Gonzo and Ms. Capp) shared realizations that their students were capable of more than they had imagined. Both teachers were unsurprised by the performance of one of the two students in each of the dyads they worked with (Carl and Cole), as they had high expectations for both of these students at the onset of the study. Yet they both expressed feelings of amazement with the performance of the other students in the dyads (Drake and Josh) as they had each been initially doubtful of the appropriateness of the intervention for these students given that they lacked fluency in addition and subtraction. Postintervention surveys and interviews were planned for student participants but could not take place due to sudden COVID-19 school closures.

Limitations and Suggestions for Future Research

Several limitations of the current study should be kept in mind when interpreting the results and can serve as avenues of exploration for future research. This study took place in the student and teacher participants’ natural classroom context, within which the teachers had multiple demands on their time over the course of the study. As a result, researchers led math instruction on days that the teacher had to fulfill other duties. As a result, the aim of evaluating a fully teacher-implemented intervention was sacrificed to maintain a 3-day per week dosage and complete the study prior to what researchers saw as an impending shut down of schools as the COVID-19 virus spread. Future research teams should make feasibility of teacher implementation a focal point, as it is a critical component of social validity. Prior research has found that middle-grade peers can deliver MSBI with fidelity to teach additive problems (Ley Davis, 2016). Future research evaluating the effects of peer-mediated instruction may increase the utility of MSBI as an intervention, expanding its use beyond one-to-one or small group settings with teachers, enabling instruction to continue when teachers’ time must be diverted.

The second limitation relates to generalization, which was only assessed once in baseline and once in intervention. Frequent measures of generalization would have provided data on whether this growth is concurrent or lags word problem-solving. This information would benefit teachers as they support student development of problem-solving across phases of learning (i.e., acquisition, fluency, maintenance, generalization). This leads to the final consideration for future investigation we will discuss here, and that is the issue of the pre-service training and in-service professional development in the area of mathematics for students with ASD/ID. Teacher participants made multiple comments to researchers before, during, and after the study regarding how little training they had in his area. Discussions about current practices reflected a developmental approach. They did not think students were “ready for” multiplication because they had not yet mastered addition or subtraction. Additional research is needed to formally explore and document teachers’ preparation, priorities, and beliefs related to mathematics for students with ASD/ID, as this developmental paradigm is likely (but unknowingly) detrimental to meaningful access to the general curriculum and inclusive opportunities.

Implications for Practice

Teachers do not need to wait for middle-grade students with ASD/ID to be “ready for” multiplication or division, as this limits inclusive opportunities and progress in grade-level standards. Instead, teachers can use MSBI to explicitly teach students with ASD/ID to identify multiplicative problem structures. Students benefit from using both the calculator and manipulatives to solve problems. Manipulatives assist in developing a conceptual understanding of multiplication through concrete representations of the problem. Calculators are more efficient, especially with large quantities (Spooner et al., 2017). Teachers can intentionally sequence the quantities in problems so that manipulatives are used for smaller quantities when initial concept development is taking place, and as they demonstrate conceptual understanding of multiplication, calculators can be used as larger quantities are introduced. This would maximize instructional time and ensure students develop functionally meaningful problem-solving skills.