Effects of Using Virtual Manipulatives for Students With Disabilities: Three-Level Multilevel Modeling for Single-Case Data

Abstract

We synthesized studies published since 2000 that assessed the effects of using virtual manipulatives to increase the mathematical accuracy of students with disabilities. We extracted a total of 1,796 raw data points from 114 cases across 35 single-case studies. By applying three-level multilevel modeling, we analyzed both immediate effects and trends during the intervention phase as well as moderation effects related to student characteristics (case level) and intervention features (study level). Both the average immediate effect and trend during the intervention were statistically significant. The average immediate effect varied significantly by student grade, disability type, developer, device, type of virtual manipulative, and visual model embedded in virtual manipulatives. Neither student characteristics nor intervention feature–related moderators significantly influenced the average trend during the use of virtual manipulatives.

Many elementary and secondary school students, including those with disabilities, obtain low scores when taking standardized mathematics tests. According to the National Assessment of Education Progress results (Nation’s Report Card, 2019), only 19% of fourth-grade students with disabilities performed at or above a proficient level in 2009; this performance declined slightly to 17% in 2019. Eighth-grade students with disabilities exhibited more limited mathematics performance than the elementary group; only 8% performed at or above the proficient level in 2009, and this rate remained unchanged in 2019. The persistent struggle faced by students with disabilities demonstrates an urgent need for educators to determine how to provide students with disabilities intensive, tailored mathematics interventions.

To this end, educators have implemented various digital technologies as instructional tools for teaching mathematics (e.g., Özdemir, 2017; Reneau, 2012). Researchers (e.g., Kiru et al., 2018; Ok et al., 2020) have validated the positive effects of digital technology in improving the performance of students who struggle with mathematics. The software used for mathematics education includes various programs, such as auto-generated practice exercises; tutorials providing guidance and human-like tutoring support; games providing time constraints, simulations replicating real or imaginary systems, problem-solving activities that include specific steps and additional opportunities to practice (Doering & Veletsianos, 2009); and programs providing virtual calculators or virtual manipulatives (VMs). VMs are technology-based interactive visual models, allowing students to manipulate virtual objects to understand mathematical concepts (Moyer-Packenham & Bolyard, 2016).

Virtual manipulatives are technology-based interactive visual models, allowing students to manipulate virtual objects to understand mathematical concepts.

VMs and Recommendations for Use in Teaching Mathematics

VMs were initially developed as Flash or Java programs (see the Utah State University’s [n.d.] National Library of Virtual Manipulatives) and have been available since the late 1990s. Since the implementation of HTML5 web standards, VMs have also become available on mobile devices (see Math Learning Center, n.d.). The key component of VMs is that the representations are not static but are instead designed to be manipulated. Highlighting the function of interactive visualization of images, Moyer-Packenham and Bolyard (2016) suggested two other types of software that are suitable for VM environments, single representation (displaying only interactive visual images) and multirepresentation (presenting numerical or textual information and images to single-representation images), in addition to existing instructional software types, such as tutorials, games, simulations, and problem solving (Doering & Veletsianos, 2009). In designing visual models embedded in VMs, Beckmann (2013) and the National Council of Teachers of Mathematics (2010) suggested several appropriate visual models, including set models (a countable discrete quantity), area models (a continuous quantity comparing the magnitude of area), linear models (a continuous quantity comparing lengths), base-10 blocks (an object representing place value of 10 numerals or digits in a number), algebra scales (balance of two sides presenting numerical values for algebraic equations), and multiple models (more than two visual models).

Previous Literature on the Use of VMs in Mathematics Interventions

In the years since they were introduced, research has indicated that the use of VMs is effective in teaching mathematics to students with disabilities (Bouck & Park, 2018; Satsangi & Miller, 2017; Shin et al., 2017). Four reviews and syntheses have discussed the use of VMs in mathematics interventions. First, Moyer-Packenham and Westenskow (2013) reviewed 66 research reports as part of their examination on the effects of VMs on learning. They observed a moderate overall effect (Cohen’s d = 0.35) of VMs alone or in combination with other instructional treatments (e.g., physical manipulatives and textbooks). However, because Moyer-Packenham and Westenskow’s meta-analysis involved interventions with all students, conclusions about the effects of VMs on specific groups were not possible.

More recently, other researchers (Bouck & Park, 2018; Lafay et al., 2019; Peltier et al., 2020) reviewed studies of VM use in mathematics interventions that included the joint use of virtual and physical manipulatives for students with disabilities. Lafay et al. (2019) synthesized 38 articles to examine the effects of different types of mathematics manipulatives (e.g., physical, pictorial, and virtual) and discovered considerable variations in the types of manipulatives. Bouck and Park (2018) noted an increase in VM use in the education field and found five of the 36 articles were about examining the academic effects of incorporating VMs (e.g., concrete-representational-abstract). Peltier et al. (2020) quantified the effects of mathematics manipulatives on learning by applying a meta-analytic method to 53 single-case design (SCD) articles or dissertations targeting students with disabilities or mathematics difficulties. Although Peltier et al. did not find any significant moderation effects associated with the manipulative type, they observed a higher average effect for VMs (Tau-U = 0.98) than for physical manipulatives (Tau-U = 0.89).

Moderators Influencing the Effects of VMs

Previous research revealed several potential moderators that might influence the effects of VMs on mathematics teaching. Such effects may differ depending on student characteristics (e.g., student grade and disability type) and intervention features (e.g., pretraining purpose, instructional methods of mathematics interventions, developer, device, VM type, and visual model embedded in VMs). For example, Moyer-Packenham and Westenskow (2013) found mixed effects of VM use across grade levels, with moderate effects for preschoolers through fourth graders, small or no effects for middle school students, and large effects for high school students. Ok and Kim (2017) indicated that the effects of using technology (e.g., iPads and iPods) might differ among students with different types of disabilities; most studies have focused on teaching students with low-incidence disabilities (e.g., autism spectrum disorders) and that research including individuals with other types of disabilities is limited.

Furthermore, researchers of studies incorporating technology into academic content, including mathematics, suggested that intervention features can serve as potential moderators. For example, Meyer et al. (2019) noted that a period of pretraining before using a virtual reality platform added significantly to the degree of knowledge attained and reduced the cognitive load necessary for multimedia lessons. Additionally, researchers incorporating technology in their mathematics interventions suggested three different types of instructional methods, leading to various student achievement: teacher-led instruction (providing explicit teacher modeling with a combination of multiple practice opportunities; Bouck, Park, Levy, et al., 2020), teacher-guided instruction (providing scaffolds and prompts when students struggle with independent mathematical problem solving; Satsangi, Hammer, & Hogan, 2018), or technology-assisted instruction (supporting students’ mathematical learning through software programs, commonly combined with instructors’ prompts for instructional steps; Shin & Bryant, 2017). Commercial applications that have been continuously field-tested by researchers maintained the high quality specified by the tool (Koumpouros & Kafazis, 2019). The type of device (e.g., computer or iPad) also led to variations in the mathematical achievement of students with learning disabilities or mathematics difficulties (Shin, 2018; Sung et al., 2016). Other reviews (Ok et al., 2020; Seo & Bryant, 2009) have demonstrated that students’ mathematical achievement and attitudes varied depending on the type of educational technology used (e.g., games, drill and practice, and tutorials). The game type was particularly effective in sustaining students’ motivation for learning, whereas the tutorial type was effective in providing support and feedback within the program.

Furthermore, the use of visual models can be a potential moderator that affects the impact of mathematics intervention effects for students with disabilities. Mazzocco et al. (2013) noted that one consistent challenge for students with disabilities is visualizing mathematical concepts. Thus, educators have implemented multiple visual models (e.g., linear, area, and set models) for various topics to assist students in making sense of different mathematical concepts (van Garderen, 2006). Hamdan and Gunderson (2017) recommend that teachers incorporate a number line (linear model) when teaching fractions to prevent students’ tendency to rely on whole-number counting.

Rationale for the Study

Despite the number of syntheses on technology implementation for mathematics interventions for students with disabilities (Kiru et al., 2018; Ok et al., 2020; Seo & Bryant, 2009) and syntheses on the use of mathematics manipulatives (Bouck & Park, 2018; Lafay et al., 2019; Peltier et al., 2020), no previous meta-analyses have investigated the moderation effects of VMs on students with disabilities in terms of student characteristics and intervention features. To fill this gap in the literature, we implemented three-level multilevel modeling and analyzed the effects of VMs used by students with disabilities.

We utilized only studies that implemented SCDs, a quantitative experimental design in which each student served as their own control (Horner et al., 2005). Previously, there have been two group studies (Bouck et al., 2015; Xin et al., 2017) that have incorporated VMs in their mathematics interventions for students with disabilities. However, the design structure of these studies did not provide case-level data involving repeated measurements on different occasions to examine changes in levels and trends during the intervention; therefore, we excluded these group studies.

The purpose of this study was to synthesize the effects of using VMs to increase the mathematical accuracy of students with disabilities. Using three-level multilevel modeling, we examined the immediate effects and trends during the intervention as well as the moderation effects of case-level (student characteristics) and study-level (intervention features) variables. Multilevel models for individual case data allowed for flexibility and addressed methodological considerations within SCD studies, such as changes in levels and trends between baseline and intervention phases (Baek et al., 2020), variability in intervention effects through moderators at both the case and study levels (Moeyaert et al., 2020), and within-group errors that can be autocorrelated (Petit-Bois et al., 2016). Three primary research questions guided the current study:

What are the average immediate effect and the average trend during the use of VMs for the mathematical accuracy of students with disabilities?

To what extent is the average immediate effect of using VMs moderated by student characteristics and intervention features?

To what extent is the average trend during the use of VMs moderated by student characteristics and intervention features?

Method

Inclusion Criteria and Search Strategies

We selected studies that met four inclusion criteria: (a) studies included students with disabilities in Grades K–12, (b) the research designs of the studies were SCDs (reversal or ABAB, multiple baseline or multiple probe, changing criterion, and alternating treatment with baselines), (c) the independent variable was the use of VMs (operationally defined as interactive visual representations on a digital screen that are used for mathematical teaching and learning tools), and (d) the dependent variable was mathematical accuracy (correct percentages on outcome measures in mathematics, depicted in graphical data).

We employed four search strategies to identify the studies. Figure S1 (in the online supplemental materials) depicts the Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart (Moher et al., 2009). The first was a search of peer-reviewed articles and dissertations published between January 2000 and September 2020. We designated the year 2000 as the starting point because we sought to examine the latest trends in VM use in our field of interest. Also, beginning in 2000, we observed a gradual increase in articles about VM use in special education journals. Therefore, using an electronic database search of Academic Search Complete (n = 2,919), Education Source (n = 1,360), PsycInfo (n = 1,235), and ERIC (n = 1,347)—with keywords “virtual manipulative*,” “game,” “simulation,” “interactive*,” “touchscreen,” “digital,” “mobile,” “tablet,” “iPad,” “computer,” or “app*,” and “mathematic*” and “disab*”—we archived a total of 6,861 articles. After the removal of 2,536 duplicates, this article search resulted in 4,325 studies. The next step was to review the titles, keywords, and abstracts of the 4,325 studies. Also, 3,942 studies were excluded for being unrelated to the research topic. The third step was to retrieve full-text copies of the remaining 383 studies and to carefully review them to determine whether they met the inclusion criteria. A total of 352 of the 383 studies were excluded for the following reasons: (a) they did not target students with disabilities (n = 11), (b) they did not implement single-case research designs (n = 98), (c) they did not use VMs within mathematics interventions (n = 87), or (d) they did not focus on mathematical accuracy (n = 156). Thus, the full-text review resulted in a total of 31 studies. The fourth and final step was to conduct an ancestral search of the 31 included studies. After a careful review of their reference lists, we found five additional studies that met the inclusion criteria, resulting in a total of 36 studies. Two authors checked the interrater reliability for the selection of included studies and achieved 100% agreement using the percentage agreement calculation between the two coders. After checking for outliers (perfect fit with residual standard error as almost zero; 0% accuracy in the baseline and 100% accuracy in the treatment phases), one study was removed, resulting in 35 studies for the data analysis.

Coding Procedures

Using an Excel spreadsheet, we developed a coding tool by reviewing previous studies of the use of VMs (e.g., Moyer-Packenham & Bolyard, 2016) and three-level multilevel modeling for SCD data and effects (e.g., Moeyaert et al., 2021). The primary coder was the first author, who has more than 10 years of teaching and research experience in the field of special education. The first author provided coding training to another coauthor in two hour-long sessions, the first focused on how to code each categorical variable, and the second focused on how to code the SCD raw data. The two coders conducted four individual remote follow-ups to establish an agreement about the meaning of each coding variable. After the training, the coders independently coded two articles and achieved 100% interrater reliability for all categories of the coding tool. The primary coder coded student characteristics, intervention feature–related variables, study quality, and SCD raw data. The independent coder coded approximately 34% (n = 12) of the 35 included studies to check the interrater reliability of coding. By applying the percentage agreement calculation established by the two coders, we achieved 98% agreement for student characteristics and intervention feature–related variables and 100% agreement for study quality and SCD raw data.

Student characteristics and intervention features

To examine the overall features of the 35 included studies, we coded student characteristics and intervention features (see Tables S1 to S3 in the online supplemental materials for the summary of each coding). For the categorical variables that were included in the moderator analysis, each variable was dummy coded. To code student characteristics, we identified students’ gender, ethnicity, grade level (Grades 1 to 5 as elementary school, Grades 6 to 8 as middle school, and Grades 9 to 12 as high school), and disability (learning disability [LD], intellectual disability [ID], autism spectrum disorder [ASD], emotional and behavioral disorders [EBD], and other health impairments [OHI]). In cases of students that were identified with multiple disabilities, we coded only the primary disability.

Next, we identified the research designs and the Common Core State Standards in Mathematics (CCSSM) content standards (National Governors Association Center for Best Practices [NGA Center] & Council of Chief State School Officers [CCSSO], 2010) for grade-level mathematics topics. We coded additional intervention features that were proposed as potential moderators: pretraining purpose (no pretraining, device use only, and device use with teacher-led instruction), instructional method (technology-assisted instruction with teachers’ prompts, teacher-guided instruction, and teacher-led instruction), VM developer (researcher developed and commercially developed), device for VM use (iPad and computer), VM type (single representation, multirepresentation, tutorial, and game), and the visual model embedded in the VMs (set model, area model, linear model, base-10 block, algebra scale, and multiple models).

Assessment of study quality

Before analyzing the data for three-level multilevel modeling, we assessed the quality of the included studies using the quality indicators (QIs) established by the Council for Exceptional Children (CEC; 2014). The eight CEC QI categories for SCD studies were context and setting (1.1), participants (2.1 and 2.2), intervention agent (3.1 and 3.2), description of practice (4.1 and 4.2), implementation fidelity (5.1, 5.2, and 5.3), internal validity (6.1, 6.2, 6.3, 6.5, 6.6, and 6.7), outcome measures and dependent variables (7.1, 7.2, 7.3, 7.4, and 7.5), and data analysis (8.2). We coded each QI that was met as 1, and we coded each criterion that was not met as 0, leading to a total of 22 possible points. Table S4 (in the online supplemental materials) displays study quality by the QIs of the CEC (2014), which was quite high. Out of a total of 22 possible points, study quality scores ranged from 19 to 22 points: 19 (n = 1), 20 (n = 2), 21 (n = 11), and all 22 (n = 21). Of the 35 studies, 13 did not meet the QI related to the intervention agent. The fidelity of implementation was 100% (n = 27) or 80% to 100% (n = 4); in one study, fidelity was descriptively reported as being high in the study, and three studies did not properly report implementation fidelity information. Interrater reliability was 100% (n = 23) or 83% to 100% (n = 10); one study evaluated interrater reliability yet did not report the actual value, and the other remaining study did not collect interrater reliability. Although there were a few missing variables described already, these were not the primary concern in the current study; therefore, we decided to include all 35 studies for data analysis.

Extraction of SCD raw data

Regarding student outcomes, each SCD data point was extracted using the GetData Graph Digitizer 2.26 web-based application (GetData Graph Digitizer, 2013). The reliability and validity of this software were significantly high, r = 1.00, p < .001 (Shin & Jung, 2018). Shin and Jung (2018) calculated correlation coefficients between coders’ extracted data points to examine the interrater reliability. They also evaluated the concurrent validity between data points from previously published articles and hypothetical graphs.

To ensure accurate data collection, we checked the baseline phase first and then moved to the intervention phase. We extracted the data using the following step: opening a captured image graph from the folder, setting up the minimum and maximum coordinate values of the x- and y-axes, selecting the data in the baseline phase and transforming them into numerical values, and selecting the data in the intervention phase and transforming them into numerical values. Finally, the y-axis values were copied and pasted into the Excel file.

Data Analysis

SCD data are hierarchically structured and can be displayed using three-level multilevel modeling (Moeyaert et al., 2021; Pustejovsky & Ferron, 2017). The levels examine repeated-measurement occasions (Level 1) nested within cases (Level 2) and cases nested within studies (Level 3). In analyzing SCD data, visual analysis techniques are frequently used to determine the quality and effects of interventions (CEC, 2014). In an attempt to quantify the level and trend between phase data patterns among visual analysis approaches (What Works Clearinghouse, 2017), we focused on the analysis of immediate effects and trends during the use of VMs.

As the focus of the current study was to determine mathematical accuracy, all outcome measures were instructional probes of correct percentages or correct scores. The scores from 12 of the 35 studies (34.3%) were first converted to percentages to apply uniform metric measures across studies. Because the raw SCD data were all correct percentages ranging from 0% to 100%, we did not standardize each case data point but, rather, used raw data for three-level multilevel modeling. To postulate individuals’ immediate effects and trends during the use of VMs, we applied a piecewise linear regression approach by separately modeling time trends in the baseline and intervention phases (Singer & Willett, 2003). Specifically, the following equation was used in Model 1, which includes no moderators:

\begin{array}{l} y_{i j k} = β_{00 j k} + β_{01 j k} I n t e r v e n t i o n_{i j k} \\ + β_{02 j k} I n t e r v e n t i o n_{i j k} \times T i m e_{i j k} + e_{i j k}, \end{array}

where y_ijk is the mathematical accuracy at the measurement occasion, ith (I = 0, 1, . . . I), for the jth case (j = 1, 2, . . . J) in study k when β_00jk is the baseline level, β_01jk is the change in level when the intervention phase starts (immediate effect), β_02jk is the change in slope between the baseline and the intervention phase (trend during the use of VMs), and e_ijk is an error term that is assumed to be independent and is normally distributed with a mean of 0 and variance of σ². Intervention_jk was dummy coded (0 for baseline phase, 1 for intervention phase), and Time_ijk was centered at the first measurement occasion in the intervention phase. Assuming that errors were associated with repeated observations (autocorrelation) and heterogeneous across phases (Baek et al., 2020; Petit-Bois et al., 2016), we applied a first-order autoregressive structure, corCAR1 (Pinheiro et al., 2020). We also assumed heterogeneous variance across phases (Moeyaert et al., 2020), incorporating the varIdent variance structure (Pinheiro et al., 2020). Based on the initial examination of SCD data, we found stable baseline data and assumed no trend in the baseline phase. Therefore, we assumed only that baseline levels, immediate effects, and trends during an intervention would vary across cases and studies. Table 1 depicts the parameters and standard errors of Model 1’s fixed and random effects.

Table 1.

Parameters for the Three-Level Multilevel Model Without Moderators.

Parameter	Model 1
Fixed effects
Immediate effect (γ₀₁₀₀)	70.95*** (3.74)
Trend during intervention (γ₀₂₀₀)	1.73*** (0.39)
Random effects
Between-study variance
Immediate effect ( $σ_{v_{1}}^{2}$ )	453.10* (121.07)
Trend during intervention ( $σ_{v_{2}}^{2}$ )	3.04* (1.23)
Between-case variance
Immediate effect ( $σ_{u_{1}}^{2}$ )	92.60* (34.78)
Trend during intervention ( $σ_{u_{2}}^{2}$ )	0.68* (0.43)

Note. Standard errors are in parentheses. The means and variances of the intercept (baseline level) are not presented.

p < .05. **p < .01. ***p < .001.

Furthermore, the between-case and between-study variances of immediate effects and trends during the use of VMs were examined. As depicted in Figure S2 and Figure S3 (in the online supplemental materials), the random effects of immediate effects and trends during the intervention phase were in the larger dispersion. Therefore, as a follow-up analysis, we investigated whether case-level (student characteristics) and study-level (intervention features) moderators could explain these variances.

To examine moderation effects, an interaction between immediate effects and dummy-coded moderators was added to Model 2. In a follow-up analysis, another interaction between trends during interventions and dummy-coded moderators was added to Model 3. Table 2 depicts the parameters and standard errors of all fixed effects as well as variances across cases and studies. For the analysis of three-level multilevel modeling, we primarily used the lme function from the nlme R package (Pinheiro et al., 2020). Standard errors in the standard deviation scale were calculated using the delta method function (Bruin, 2006) from the msm R package (Jackson, 2011).

Table 2.

Parameters for the Three-Level Multilevel Model With Moderators.

Parameter	Model 2^a	Model 3^b
Fixed effects
Immediate effect (γ₀₁₀₀)	88.21* (37.99)	70.44*** (3.73)
Trend during intervention (γ₀₂₀₀)	1.99*** (0.44)	−1.01 (6.99)
Grade^c
Middle school (γ₀₃₀₀)	−18.62 (12.79)	3.28 (2.12)
High school (γ₀₄₀₀)	27.78* (13.50)	1.99 (2.73)
Disability
Intellectual disabilities (γ₀₅₀₀)	−7.58 (4.63)	−0.24 (0.88)
Autism spectrum disorders (γ₀₆₀₀)	−3.52 (4.87)	0.32 (0.91)
Emotional-behavioral disorders (γ₀₇₀₀)	30.83* (12.95)	3.71 (3.49)
Other health impairment (γ₀₈₀₀)	−6.66 (5.35)	−0.89 (1.01)
Pretraining purpose
Device use (γ₀₉₀₀)	−8.05 (13.42)	3.36 (3.20)
Device use, instruction (γ₁₀₀₀)	−25.23 (14.41)	1.21 (2.27)
Instructional method
Teacher guided (γ₁₁₀₀)	−14.95 (31.08)	1.32 (5.96)
Teacher led (γ₁₂₀₀)	−25.64 (31.35)	1.43 (5.98)
Developer
Commercially developed (γ₁₃₀₀)	46.60*** (12.39)	−0.21 (2.09)
Device
Computer (γ₁₄₀₀)	28.00* (11.83)	−0.26 (1.96)
Type of virtual manipulative
Single representation (γ₁₅₀₀)	−7.69 (10.62)	−2.01 (1.91)
Tutorial (γ₁₆₀₀)	−44.15* (17.81)	−2.43 (3.18)
Game (γ₁₇₀₀)	−66.15 (37.40)	0.71 (6.80)
Visual model
Area model (γ₁₈₀₀)	−7.55 (12.02)	−0.26 (2.39)
Linear model (γ₁₉₀₀)	−4.37 (10.09)	−1.54 (1.83)
Base-10 block (γ₂₀₀₀)	2.63 (10.64)	−1.00 (1.91)
Algebra scale (γ₂₁₀₀)	27.64 (19.70)	0.95 (3.04)
Multiple models (γ₂₂₀₀)	−45.95** (15.82)	4.17 (2.46)
Random effects
Between-study variance
Immediate effect ( $σ_{v_{1}}^{2}$ )	147.62* (72.40)	448.32* (125.36)
Trend during intervention ( $σ_{v_{2}}^{2}$ )	3.73* (1.57)	3.05* (2.17)
Between-case variance
Immediate effect ( $σ_{u_{1}}^{2}$ )	94.33* (34.17)	91.28* (40.21)
Trend during intervention ( $σ_{u_{2}}^{2}$ )	0.66* (0.38)	0.71* (0.92)

Note. Standard errors are in parentheses. The means and variances of the intercept (baseline level) are not presented.

Interactions between immediate effects and moderators.

Interactions between trends during intervention and moderators.

The reference group was given to elementary school students with learning disabilities who received technology-assisted instruction with teachers’ prompts, but without pretraining, using iPads to run the researcher-developed multirepresentation-with-sets virtual manipulative.

Publication Bias

To reduce publication bias, we included unpublished dissertations within our target studies. Any potential publication bias was assessed using funnel plot asymmetry with Egger’s statistical test (Egger et al., 1997). As depicted in Figure S4 and Figure S5 (in the online supplemental materials), funnel plots were created using the metaphor R package (Viechtbauer, 2010). An asymmetric plot with potential publication bias was observed for immediate effects (t = −3.77, p < .001). Regarding the trends during the intervention, a somewhat symmetrical plot was observed with no significant publication bias (t = 1.53, p = .14). This confounding result regarding publication bias requires careful attention when interpreting the results. The extracted raw data points, coded data, and R codes for data analysis were posted through an online data repository (Shin et al., 2021).

Results

Summary of Student Characteristics and Intervention Features

A total of 114 students participated across 35 studies (seven dissertations and 28 peer-reviewed journal articles), published between 2012 and 2020 (see Table S1 in the online supplemental materials). Participants included 86 male and 28 female students: 21 in elementary school, 72 in middle school, and 21 in high school grade levels. Of these, 68 students were classified as White, 20 as Black, 13 as Hispanic, and four as multiracial (nine students did not report their race-ethnicity). Additionally, 34 students were identified as having LD; 26, ID; 32, ASD; 11, EBD; and 11, OHI.

As depicted in Table S2 and Table S3 (in the online supplemental materials), 21 of the 35 studies employed a multiple-probe design across subjects (n = 17) or behaviors (n = 4), eight used a multiple-baseline design across subjects (one also employed an alternating treatment design), and six others implemented an alternating-treatment design. Regarding the CCSSM content standards (NGA Center & CCSSO, 2010), the largest number of studies focused on numbers and operations–fractions (e.g., equivalent fractions, adding and subtracting fractions, multiplying fractions; n = 10), followed by number and operations in base 10 (e.g., place value, adding and subtracting whole numbers with regrouping; n = 6) mixed with other domains (n = 1), operations and algebraic thinking (e.g., single-digit and double-digit multiplication and division; n = 7), algebra–reasoning with equations and inequalities (e.g., solving linear algebraic equations; n = 3), measurement and data (solving area and perimeter) or counting and cardinality (single-digit number comparison; n = 3), expressions and equations (e.g., one-step division algebraic equations; n = 1), the number system (e.g., adding integers; n = 2), geometry (e.g., partition a rectangle; n = 1), and ratios and proportional relationships (n = 1).

Regarding intervention features, about half of the reviewed studies (n = 19) provided no pretraining before the intervention; in 13 other studies, interventionists delivered pretraining on both device use and instruction with (n = 10) or without explicit instruction (n = 3); in the remaining three studies, device use was the only purpose of pretraining. Interventionists implemented VMs through teacher-guided instruction with (n = 11) or without (n = 3) incorporating the system of least prompts, teacher-led instruction with (n = 15) or without (n = 1) the use of explicit instruction, and as part of technology-assisted instruction with teachers’ prompts (n = 5). When teacher-guided instruction was incorporated, instructional prompts were given to each student at the independent testing time with records of students’ independence on mathematical task completion. In most studies, commercially developed VM programs were used (n = 21), although 13 other studies used researcher-developed VMs, and one study included both VM programs. The interventions using VMs were conducted via iPads (n = 24) or computers (n = 11). The most frequently incorporated type of VM was multirepresentations (n = 20), followed by tutorials (n = 6), single representations (n = 4), and games (n = 3). In the other two studies, a single representation was used in combination with multirepresentation or a tutorial. The most frequently observed visual model embedded in VM was the linear model (n = 11), followed by the base-10 block (n = 5; one with a combination of set models), set model (n = 8), area model (n = 4), algebra scale (n = 3), and multiple models within VM apps (n = 4).

The Immediate Effect and Trend During the Use of VMs

In total, 1,796 single-case raw data points for 114 cases across 35 SCD studies were analyzed. Model 1 in Table 1 depicts the parameter estimates of a three-level multilevel model that assumes autocorrelation. According to the multilevel model without moderators, the average immediate effect was statistically significant, γ₀₁₀₀ = 70.95, p < .001. Specifically, when VMs were used, there was an average increase of 70.95% in mathematical accuracy of the level, from baseline to intervention phases, in the first intervention session. Furthermore, the average trend during the use of VMs was also statistically significant and positive (γ₀₂₀₀ = 1.73, p < .001). There was a significant interaction between the use of VMs and time; with one session increase during the intervention phase, there was an average of 1.73% increase in mathematical accuracy.

The between-case ( $σ_{u_{1}}^{2}$ = 92.60, p < .05) and between-study ( $σ_{v_{1}}^{2}$ = 453.10, p < .05) variances of the immediate effect for mathematical accuracy of students with disabilities were statistically significant. The between-case ( $σ_{u_{2}}^{2}$ = 0.68, p < .05) and between-study ( $σ_{v_{2}}^{2}$ = 3.04, p < .05) variances of the trend during the use of VMs were also statistically significant. These results support the need to add moderators at both case and study levels to determine how these predictors explain variability in the immediate and trend effects.

The Influence of Moderators on the Immediate Effect of Using VMs

As depicted in Model 2 in Table 2, we added case-level (student characteristics) and study-level (intervention features) moderators. Because all moderators were categorical variables, the reference group was designated to have the following aspects: elementary school students with LD who received technology-assisted instruction with teachers’ prompts, but without pretraining, using iPads to run the researcher-developed multirepresentation-with-sets VM.

We found statistically significant interactions between the immediate effect of using VMs and student characteristics–related variables: student grade and disability type. When controlling for main effects and other variables in the model (e.g., disability type, developer, and device), the average immediate effect of using VMs was significantly greater for high school students than for elementary school students (γ₀₄₀₀ = 27.78, p = .04). Nonsignificant and smaller average immediate effect was observed for middle school students than for elementary school students. Additionally, although not statistically significant, the average immediate effect was larger for students with LD than for students with ID, ASD, and OHI. Contrariwise, there was a significantly greater average immediate effect of using VMs for students with EBD than for those with LD (γ₀₇₀₀ = 30.83, p = .02).

There were also statistically significant interactions between the immediate effect of using VMs and intervention feature–related variables: developer, device, VM type, and the visual model embedded in VMs. Notably, the larger average immediate effect of using VMs was associated with the use of commercially developed VMs rather than researcher-developed VMs (γ₁₃₀₀ = 46.60, p < .001). Furthermore, the incorporation of the computer was significantly more effective than the use of the iPad (γ₁₄₀₀ = 28.00, p = .02). A larger average immediate effect was observed for the use of the multirepresentation type than for single-representation, tutorial, and game VM types. In particular, the average immediate effect was significantly smaller for tutorials than for the multirepresentation VM type (γ₁₆₀₀ = −44.15, p = .01). The average immediate effect of the use of the set model was larger than the effect of the area model, linear model, and multiple models but smaller than the effect of the base-10 block and algebra scale. Compared with the use of multiple models in VMs, a statistically significant and larger average immediate effect was associated with the use of the set model (γ₂₂₀₀ = −45.95, p = .004).

After controlling for all other variables, studies showed no significant difference in the average immediate effects of using VMs between providing pretraining (on the use of the device or device use combined with instruction) and not providing pretraining. Moreover, although studies demonstrated slightly larger average immediate effects on the use of technology-assisted instruction with the teachers’ prompts than on the teacher-guided instruction (some with the system of the least prompts) and teacher-led instruction, including explicit instruction, these differences were not statistically significant.

After adding moderators to the analysis in Model 2, both the immediate effect of using VMs (γ₀₁₀₀ = 88.21, p = .02) and the trend during the use of VMs (γ₀₂₀₀ = 1.99, p < .001) remained significant. Relative to Model 1, there was a 1.87% increase in the between-case variance ( $σ_{u_{1}}^{2}$ = 94.33) and a 67.42% reduction in the between-study variance ( $σ_{v_{1}}^{2}$ = 147.62) of the immediate effect. There was also a 2.94% reduction in the between-case variance ( $σ_{u_{2}}^{2}$ = 0.66), yet there was a 22.70% increase in the between-study variance ( $σ_{v_{2}}^{2}$ = 3.73) of the trend during the intervention.

Influences of Moderators on Trends During the Use of VMs

As displayed in Model 3 in Table 2, we examined another three-level multilevel model, one including all moderators, to analyze the variation of the trend effect of using VMs on the mathematical accuracy of students with disabilities. We assigned the same reference group as Model 2 to examine the interactions between trend effects and moderators.

None of the moderators related to student characteristics (student grade and disability type) significantly influenced the trend about the use of VMs. Although the average trend was slightly larger for middle and high school students than for elementary school students, these differences were not statistically significant after controlling for other variables. The difference in the average trend for students with LD compared with students with other disabilities was also minimal.

All intervention feature–related moderators (pretraining purpose, instructional method, VM developer, device for the use of VMs, VM type, and visual model) did not significantly influence the trend during the use of VMs. In other words, there were no statistically significant interactions between moderators and the trend during the intervention. Although not statistically significant, after controlling for all other variables, the slightly larger average trend during the use of VMs was associated with groups provided with pretraining on the device or on both device use and instruction together compared with the groups with no pretraining. Moreover, a slightly larger average trend was found during the intervention phase associated with teacher-guided instruction and teacher-led instruction compared with the use of technology-assisted instruction. Over time, the use of researcher-developed VMs turned out to be slightly more effective in improving the mathematical accuracy of students with disabilities than that of commercially developed VMs. A slightly higher trend effect was associated with the use of iPads than with the use of computers. Additionally, the incorporation of the multirepresentation VM type was associated with a slightly higher average trend than the single-representation and tutorial VM types but a slightly smaller average trend than the game VM type. The average trend was slightly higher for the set model than for the area model, linear model, and base-10 block. Relative to the use of the algebra scale and multiple models embedded in VMs, a lower average trend was observed for the use of the set model.

By adding moderators to the analysis in Model 3, the trend during the use of VMs turned out to be nonsignificant (γ₀₂₀₀ = −1.01, p = .88), but the average immediate effect remained significant (γ₀₁₀₀ = 70.44, p < .001). Compared with Model 1, there was a 1.43% reduction in the between-case variance ( $σ_{u_{1}}^{2}$ = 91.28) and a 1.05 % reduction in the between-study variance ( $σ_{v_{1}}^{2}$ = 448.32) of the immediate effect. However, there was a 4.41% increase in the between-case variance ( $σ_{u_{2}}^{2}$ = 0.71) and a 0.33% increase in the between-study variance ( $σ_{v_{2}}^{2}$ = 3.05) of the trend during the intervention

Discussion

The purpose of the current study was to synthesize the effects of using VMs on the mathematical accuracy of students with disabilities. Applying three-level multilevel modeling, we investigated trends during the intervention phase as well as immediate intervention effects among SCD studies. The influences of moderators related to student characteristics (case level) and intervention features (study level) on immediate effects and trends during the use of VMs were also evaluated.

Both the average immediate effect and trend during the use of virtual manipulatives were statistically significant.

The Immediate Effect and Trend During the Use of VMs

The results support that the use of VMs was highly effective in teaching mathematical accuracy for students with disabilities. There was a statistically significant average immediate effect of using VMs in mathematical accuracy from the baseline to the intervention phase. This finding is consistent with previous syntheses of research on mathematics manipulatives (Bouck & Park, 2018; Lafay et al., 2019; Peltier et al., 2020) or technology-mediated interventions (Kiru et al., 2018; Ok et al., 2020) in the use of VMs, which have been found to be generally effective for improving the mathematical outcomes of students with disabilities. These previous research included VMs as a subset of targeted mathematics interventions and validated the positive effects of using mathematics manipulatives or technology-mediated mathematics interventions. Additionally, the use of VMs produced a significantly positive average trend during the intervention, indicating a growth over time due to the interventions incorporating VMs in their mathematics lessons. These findings add value to the existing syntheses and reviews on the effects of using VMs on students with disabilities (Bouck & Park, 2018; Lafay et al., 2019; Peltier et al., 2020), particularly in terms of positive changes in trends and levels between the baseline and intervention phases.

The Influence of Moderators on the Immediate Effect of Using VMs

To reduce the unexplained variance between cases and studies concerning the immediate effects of using VMs for mathematical accuracy, we added student characteristics and intervention feature–related moderators. Although not all categorical variables turned out to be significant moderators influencing the immediate effects of using VMs, we found that student grade, disability type, developer, device, VM type, and visual models embedded in VMs are potential moderators.

After controlling for all other variables in the model, student grade and disability type were significant moderators influencing the immediate effect of using VMs. Consistent with the findings of Moyer-Packenham and Westenskow (2013), high school students showed the largest immediate effects, with a significantly greater effect than elementary school students. Moreover, students with EBD exhibited the highest average immediate effect of using VMs, a significantly greater improvement than even those with LD. This finding is somewhat different from that of Peltier et al. (2020); from studies using both physical and virtual manipulatives, students with EBD exhibited the least improvement compared with students with LD, OHI, ID, and ASD. However, in the present study, when we disaggregated these mathematical manipulatives and evaluated the effect of VMs, we found the opposite results for this student population.

The average immediate effect significantly differed by intervention features (developer, device, VM type, and visual model embedded in VM). After controlling for the main effects and other variables (e.g., student disability, device, and VM type), the use of commercially developed VMs had a larger average immediate effect on mathematical accuracy than the use of researcher-developed VMs. This finding indicates that commercially available applications (e.g., Langhorne, 2017), which receive continuous updates and validations by many other researchers and users, might be able to maintain higher quality and more accessibility for users than researcher-developed applications that are not easily accessible (e.g., Özdemir, 2017).

Students with disabilities exhibited statistically significant and larger average immediate effects when they used VMs with a computer than with an iPad. This finding is somewhat contradictory to previous reviews on the effects of device type (Ok et al., 2020; Sung et al., 2016). Sung et al. (2016) found more effective usage of handheld devices, including iPads, than that of laptops as teaching and learning tools across subjects (e.g., mathematics, science). However, when exclusively focused on the mathematics domain, Ok et al. (2020) did not find a significant difference in the mathematical improvement of students with LD. Thus, the moderation effect by different device types should be further validated within various technology-mediated learning settings, including the use of VMs.

The higher average immediate intervention effect with the use of the multirepresentation VM type compared with the other types of VMs (i.e., single representation, tutorial, and game) was notable. When selecting visual models for students struggling with mathematics, the built-in function of multirepresentations, such as pictorial, numeric, and text representations, can accommodate the problem-solving process and extend learners’ experiences to mathematical connections across various representations (Shin et al., 2017). When compared with the immediate effect of using the tutorial type (e.g., providing feedback or tutoring), this difference was statistically significant. Although the tutorial VM type includes an additional function of feedback beyond the features of the multirepresentation type, the current findings indicate that the additional feature does not always lead to better mathematical outcomes.

The visual model embedded in VMs was another moderator influencing the immediate effect on students’ mathematical accuracy. The use of the set model embedded in VMs had a larger average immediate effect than the effect of the area model, linear model, and multiple models. As shown in the present study, the set model was applied to various VM types and mathematical topics. For example, the set model was embedded in the single-representation VM type (e.g., Color Tiles; Brainingcamp) in teaching multiplication and division (Bouck, Park, & Shurr, 2019) or multirepresentation VM type (e.g., two-color counter) in teaching integer addition (Bouck & Park, 2020). The effect of using set-model VMs was statistically significant and larger compared with the effect of using multiple models. This finding indicates that providing more than two different visual models (e.g., linear model and set model) does not guarantee better immediate effects.

The Influence of Moderators on Trends During the Use of VMs

With the incorporation of three-level multilevel modeling, we were able to examine how the trend changes over time during the use of VMs in the intervention phase. Observing a change in slope to a describable direction, as well as a shift in level, is an essential component of the visual analysis of SCD studies (Ledford & Gast, 2018).

After controlling for all other variables, the trend during the use of VMs did not significantly differ by student grade or disability status. However, the current multilevel modeling indicated some notable patterns in the students’ performances. Although statistically nonsignificant, regarding the use of VMs, middle and high school students with disabilities demonstrated slightly higher average trends during the intervention phase than the elementary students’ groups. This finding is consistent with Peltier et al.’s (2020) study, in which secondary students with disabilities or mathematics difficulties exhibited larger effects than those at the elementary level when using mathematical manipulatives.

Furthermore, consistent with the moderation effect on the immediate effect, the growth of students with EBD over time was larger than the growth of students with LD. Although statistically not significant, during the intervention, there was a notable improvement in students with ASD. Students with ASD exhibited a smaller average immediate effect than those with LD. However, during the intervention phase, over time, they exhibited a slightly larger average trend than students with LD.

After controlling for main effects and other moderators, study-level moderators did not show significant interactions with students’ growth of mathematical accuracy over time. Despite the overall nonsignificant moderation effects, some findings were notable. The average trend during the use of VMs was slightly larger when researchers provided pretraining. This finding is consistent with Meyer et al. (2019) regarding the importance of providing pretraining in technology-mediated learning environments to familiarize learners with the use of technology before the actual intervention. In the present study, researchers (e.g., Shin et al., 2017; Xin et al., 2020) who implemented technology-assisted instruction with teachers’ prompts primarily provided pretraining on how to use the device. Other researchers (e.g., Bouck, Shurr, et al., 2020; Saunders, 2014) provided pretraining on both device use and how to solve mathematical problems with task analysis steps. Despite a larger average immediate effect of using technology-assisted instruction with teachers’ prompts, the average trend during the use of VMs was smaller than when using the other instructional methods, either teacher-guided (e.g., Jimenez & Besaw, 2020) or teacher-led mathematics instruction (e.g., Park et al., 2021).

The influence of the developer and device type of VMs on trends during the use of VMs was contrary to those of the immediate effects. Students with disabilities demonstrated slightly more positive growth when they used researcher-developed VMs than when they used commercially developed VMs. Although nonsignificant, the trend during the use of VMs was more positively associated with the use of the iPad device than with the use of a computer. The use of the multirepresentation VM type was more effective than the use of the single-representation and tutorial types in improving students’ mathematical growth over time. Moyer-Packenham and Westenskow (2013) pointed out various desirable features of VMs, such as allowing students to simultaneously link two or more representations (e.g., graphic or numeric information). Zbiek et al. (2007) also noted that with the implementation of interactive and dynamic technology tools, such as VMs, students can increase their representational fluency.

The effect of the visual model embedded in VMs on trends during the intervention phase was distinguished from the visual model’s impact on the immediate effects. Although the base-10-block model of VM was associated with a slightly higher average immediate effect than the set model, the set model was more effective in improving students’ mathematical growth during the intervention than the base-10 block. Furthermore, although the incorporation of multiple models led to a significantly lower immediate effects than the set model, a slightly higher average trend was found for the use of multiple models during the intervention phase. These deviations in the findings corroborate the need to examine trends during the use of VMs, even after the immediate effects have been observed after the introduction of mathematics interventions.

Deviations in the moderation effects corroborate the need to examine trends during the use of virtual manipulatives, even after the immediate effects have been observed.

Limitations and Future Research

The findings of the current study have several limitations that should be taken into account in future research. First, of the 35 included studies, various SCD studies were included (multiple-baseline and multiple-probe designs across subjects or behaviors and alternating-treatment designs). These results indicate that studies can demonstrate intervention effects across various student populations and mathematics topics, depending on the study design. Thus, future research needs to consider how students’ immediate learning and academic growth can differ by various types of SCDs. Furthermore, for Reneau’s (2012) study, we disaggregated data and selectively coded only the two students with disabilities, excluding the other three cases of students without disabilities. When we evaluated the study’s QIs by CEC (2014), we considered these variations across studies. Furthermore, the results of the study’s QIs can vary according to different QI guidelines. In future research, it is suggested that QIs are further compared across multiple guidelines to validate whether consistent results are obtained across various QI standards (e.g., CEC, What Works Clearinghouse).

Second, there was a limited focus on the target students, limiting the generalization of the current study’s findings across students from various demographics. Specifically, the researchers in the included studies primarily targeted middle school students (n = 20) versus elementary school students (n = 8) and high school students (n = 7). Furthermore, the participants in the studies were predominantly White and those with LD, ASD, or ID. There is a need to investigate the effects of using VMs with students from other racial and ethnic groups (e.g., Black, Hispanic, Asian) who have other types of disabilities (e.g., EBD, OHI).

Third, there was a limited focus on CCSSM content standards (NGA Center & CCSSO, 2010). Of the 35 studies, researchers most frequently targeted numbers and operations–fractions (n = 10) or numbers and operations in base 10 (n = 7). However, limited research has been conducted on other mathematics concepts and skills, including expressions and equations, the number system, geometry, ratios, and proportional relationships.

Fourth, because the primary concern of the current study was to examine the average level and trend changes during the use of VMs over time, we did not conduct formal visual analysis for each study (e.g., What Works Clearinghouse, 2017). Thus, a functional relation between mathematics interventions with VMs and mathematical accuracy for each study and case remains unknown. Thus, future synthesis needs to conduct a formal visual analysis (level, trend, variability, immediacy of the effect, overlap, and consistency of data in similar phases) for each study and case, assessing data patterns within and across phases (What Works Clearinghouse, 2017).

Last, in the current multilevel modeling, pretraining and instructional methods were found to have no statistically significant moderating effects that influenced both immediate effects and trends during the use of VMs. However, these results could have been influenced by the relatively small sample sizes. As a result, low power in the model may have failed to detect such interaction terms between moderators and each intervention effect (Aguinis, 1995). In future research, researchers should continuously implement VMs, comparing the effects of the different purposes of pretraining before the intervention and exploring the influence of various types of instructional methods in the educational technology environment (teacher-guided, teacher-led, or technology-assisted instruction) in an effort to further validate the role of moderators.

Practical Implications

The results of this study revealed that the use of VMs during mathematics instruction with students with disabilities immediately had a positive influence on mathematical accuracy and supported students’ mathematical growth over time during mathematics interventions. The effects of student characteristics and study feature–related moderators on immediate effects and trends during the use of VMs were somewhat different. For example, the use of commercially developed VMs and computer devices was found to be significantly more effective in improving the immediate effects of mathematical accuracy than researcher-developed VMs used with iPads. However, when observing students’ mathematical growth over time during the use of VMs, the opposite results were found. Thus, when planning mathematics interventions incorporating VMs, teachers need to consider both immediate and gradual growth over time and accommodate various design features in their mathematics lessons.

As learning tools, the use of VMs provides benefits for students with disabilities. As an alternative mathematical tool to physical manipulatives, VMs are considered practical tools for teaching mathematics to students both with and without disabilities in inclusive settings (Bouck & Park, 2018; Moyer-Packenham & Westenskow, 2013). The findings of the present study highlighted favorable effects on students’ growth over time as well as increased mathematical accuracy through the use of VMs. VMs can be flexibly used to fit the various sizes, colors, and unlimited objects provided during mathematics instruction.

VMs are a promising and viable option for applying technology in K–12 schools in a flexible and convenient way. Furthermore, several VMs are free or low-cost, enabling easy access for educators for planning differentiated instruction for diverse learners. As previously noted, various types and visual models embedded in VMs can be used flexibly in mathematics interventions. Teachers can apply authoring tools (e.g., SMART Notebook software) to create interactive materials (Root et al., 2020) across mathematical domains. From the perspective of assistive technology, VMs can provide a new means of representation that interactively supports students’ visualization of mathematical concepts.

Supplemental Material

sj-zip-1-ecx-10.1177_00144029211007150 – Supplemental material for Effects of Using Virtual Manipulatives for Students With Disabilities: Three-Level Multilevel Modeling for Single-Case Data

Supplemental material, sj-zip-1-ecx-10.1177_00144029211007150 for Effects of Using Virtual Manipulatives for Students With Disabilities: Three-Level Multilevel Modeling for Single-Case Data by Mikyung Shin, Jiyeon Park, Rene Grimes and Diane P. Bryant in Exceptional Children

Footnotes

Authors’ Note

Our work was supported by the Killgore Faculty Research Grant, West Texas A&M University.

Open Science Practices

For publishing data, code, and materials, this article earned badges for Open Data and Open Materials. The public content for the data, code, and materials may be retrieved from .

ORCID iD

Mikyung Shin

Supplemental Material

Supplemental material for this article is available online.

References

Aguinis

(1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21(6), 1141–1158. http://doi.org/10.1016/0149-2063(95)90026-8

Baek

Beretvas

S. N.

Van den Noortgate

Ferron

J. M.

(2020). Brief research report: Bayesian versus REML estimations with noninformative priors in multilevel single-case data. Journal of Experimental Education, 88(4), 698–710. https://doi.org/10.1080/00220973.2018.1527280

*Bassette

Bouck

Shurr

Park

Cremeans

(2019). Comparison of concrete and app-based manipulatives to teach subtraction skills to elementary students with autism. Education and Training in Autism and Developmental Disabilities, 54(4), 391–405.

*Bassette

Bouck

Shurr

Park

Cremeans

Rork

Miller

Geiser

(2020). A comparison of manipulative use on mathematics efficiency in elementary students with autism spectrum disorder. Journal of Special Education Technology, 35(4), 179–190. https://doi.org/10.1177/0162643419854504

Beckmann

(2013). Mathematics for elementary teachers with activity manual. Pearson.

Bouck

Flanagan

Bouck

(2015). Learning area and perimeter with virtual manipulatives. Journal of Computers in Mathematics and Science Teaching, 34(4), 381–393. https://www.learntechlib.org/primary/p/114409

*Bouck

E. C.

Bassette

Shurr

Park

Kerr

Whorley

(2017). Teaching equivalent fractions to secondary students with disabilities via the virtual–representational–abstract instructional sequence. Journal of Special Education Technology, 32(4), 220–231. https://doi.org/10.1177/0162643417727291

*Bouck

E. C.

Chamberlain

Park

(2017). Concrete and app-based manipulatives to support students with disabilities with subtraction. Education and Training in Autism and Developmental Disabilities, 52(3), 317–331. https://doi.org/10.2307/26420403

*Bouck

E. C.

Long

Park

(2021). Using a virtual number line and corrective feedback to teach addition of integers to middle school students with developmental disabilities. Journal of Developmental and Physical Disabilities, 33, 99–116. https://doi.org/10.1007/s10882-020-09735-z

10.

*Bouck

E. C.

Maher

Park

Whorley

(2020). Learning fractions with a virtual manipulative based graduated instructional sequence. Education and Training in Autism and Developmental Disabilities, 55(1), 45–59.

11.

Bouck

E. C.

Park

(2018). A systematic review of the literature on mathematics manipulatives to support students with disabilities. Education & Treatment of Children, 41(1), 65–106. https://doi.org/10.1353/etc.2018.0003

12.

*Bouck

E. C.

Park

(2020). App-based manipulatives and the system of least prompts to support acquisition, maintenance, and generalization of adding integers. Education and Training in Autism and Developmental Disabilities, 55(2), 158–172.

13.

*Bouck

E. C.

Park

Cwiakala

Whorley

(2020). Learning Fraction Concepts Through the Virtual-Abstract Instructional Sequence. Journal of Behavioral Education, 29, 519–542. https://doi.org/10.1007/s10864-019-09334-9

14.

*Bouck

E. C.

Park

Levy

Cwiakala

Whorley

(2020) App-base manipulatives and explicit instruction to support division with remainders. Exceptionality, 28(1), 45–59. https://doi.org/10.1080/09362835.2019.1586709

15.

*Bouck

E. C.

Park

Maher

Levy

Cwiakala

(2019). Acquiring the skill of identifying fractions through the virtual-abstract framework. Journal of Developmental and Physical Disabilities, 31(4), 435–452. https://doi.org/10.1007/s10882-018-9650-9

16.

*Bouck

E. C.

Park

Satsangi

Cwiakala

Levy

(2019). Using the virtual-abstract instructional sequence to support acquisition of algebra. Journal of Special Education Technology, 34(4), 253–268. https://doi.org/10.1177/0162643419833022

17.

*Bouck

E. C.

Park

Shurr

(2019). Using the virtual-representational instructional sequence to support the acquisition and maintenance of mathematics for students with intellectual disability. International Journal of Developmental Disabilities. Advance online publication. https://doi.org/10.1080/20473869.2019.1640999

18.

*Bouck

E. C.

Park

Shurr

Bassette

Whorley

(2018). Using the virtual–representational–abstract approach to support students with intellectual disability in mathematics. Focus on Autism and Other Developmental Disabilities, 33(4), 237–248. https://doi.org/10.1177/1088357618755696

19.

*Bouck

E. C.

Park

Sprick

Shurr

Bassette

Whorley

(2017). Using the virtual-abstract instructional sequence to teach addition of fractions. Research in Developmental Disabilities, 70, 163–174. https://doi.org/10.1016/j.ridd.2017.09.002

20.

*Bouck

E. C.

Park

Stenzel

(2020). Virtual manipulatives as assistive technology to support students with disabilities with mathematics. Preventing School Failure: Alternative Education for Children and Youth, 64(4), 281–289. https://doi.org/10.1080/1045988X.2020.1762157

21.

*Bouck

E. C.

Satsangi

Doughty

T. T.

Courtney

W. T.

(2014). Virtual and concrete manipulatives: A comparison of approaches for solving mathematics problems for students with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44(1), 180–193. https://doi.org/10.1007/s10803-013-1863-2

22.

*Bouck

E. C.

Shurr

Park

(2020). Virtual manipulative-based intervention package to teach multiplication and division to secondary students with developmental disabilities. Focus on Autism and Other Developmental Disabilities, 35(4), 195–207. https://doi.org/10.1177/1088357620943499

23.

Bruin

(2006). Newtest: Command to compute new test. University of California–Los Angeles Statistical Consulting Group. https://stats.idre.ucla.edu/stata/ado/analysis

24.

Council for Exceptional Children. (2014). CEC standards for evidence-based practices in special education.

25.

Doering

Veletsianos

(2009). Teaching with instructional software. In Roblyer

M. D.

Doering

(Eds.), Integrating Educational Technology Into Teaching (pp. 73–108). Pearson.

26.

Egger

Smith

G. D.

Schneider

Minder

(1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629

27.

GetData Graph Digitizer. (2013). GetData Graph Digitizer (Version 2.26) [Computer software]. http://www.getdata-graph-digitizer.com/index.php

28.

Hamdan

Gunderson

E. A.

(2017). The number line is a critical spatial-numerical representation: Evidence from a fraction intervention. Developmental Psychology, 53(3), 587–596. https://doi.org/10.1037/dev0000252

29.

*Hammer

(2018). Virtual manipulatives with technology-based graphic organizers (VM+TBGO): A combined mathematics intervention to support secondary students with learning disabilities to solve proportion word problems (Publication No. 13420380) [Doctoral dissertation, George Mason University]. ProQuest Dissertations and Theses Global.

30.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71(2), 165–179. https://doi.org/10.1177/001440290507100203

31.

Jackson

C. H.

(2011). Multi-state models for panel data: The msm package for R. Journal of Statistical Software, 38(8), 1–29. http://www.jstatsoft.org/v38/i08

32.

*Jimenez

B. A.

Besaw

(2020). Building early numeracy through virtual manipulatives for students with intellectual disability and autism. Education and Training in Autism and Developmental Disabilities, 55(1), 28–44.

33.

Kiru

E. W.

Doabler

C. T.

Sorrells

A. M.

Cooc

N. A.

(2018). A synthesis of technology-mediated mathematics interventions for students with or at risk for mathematics learning disabilities. Journal of Special Education Technology, 33(2), 111–123. https://doi.org/10.1177/0162643417745835

34.

Koumpouros

Kafazis

(2019). Wearables and mobile technologies in autism spectrum disorder interventions: A systematic literature review. Research in Autism Spectrum Disorders, 66, Article 101405. https://doi.org/10.1016/j.rasd.2019.05.005

35.

Lafay

Osana

H. P.

Valat

(2019). Effects of interventions with manipulatives on immediate learning, maintenance, and transfer in children with mathematics learning disabilities: A systematic review. Education Research International, 2019, Article 2142948. https://doi.org/10.1155/2019/2142948

36.

*Langhorne

(2017). Improving foundational math skills in students with autism using virtual instruction (Publication No. 13876800) [Doctoral dissertation, Johns Hopkins University]. ProQuest Dissertations and Theses Global.

37.

Ledford

J. R.

Gast

D. L.

(2018). Single case research methodology: Applications in special education and behavioral sciences. Routledge.

38.

Math Learning Center. (n.d.). Free math apps. Retrieved March 31, 2021, https://www.mathlearningcenter.org/apps

39.

Mazzocco

M. M.

Myers

G. F.

Lewis

K. E.

Hanich

L. B.

Murphy

M. M.

(2013). Limited knowledge of fraction representations differentiates middle school students with mathematics learning disability (dyscalculia) versus low mathematics achievement. Journal of Experimental Child Psychology, 115(2), 371–387. https://doi.org/10.1016/j.jecp.2013.01.005

40.

Meyer

O. A.

Omdahl

M. K.

Makransky

(2019). Investigating the effect of pre-training when learning through immersive virtual reality and video: A media and methods experiment. Computers & Education, 140, Article 103603. https://doi.org/10.1016/j.compedu.2019.103603

41.

Moeyaert

Klingbeil

D. A.

Rodabaugh

Turan

(2021). Three-level meta-analysis of single-case data regarding the effects of peer tutoring on academic and social-behavioral outcomes for at-risk students and students with disabilities. Remedial and Special Education, 42(2), 94–106. https://doi.org/10.1177/0741932519855079

42.

Moeyaert

Manolov

Rodabaugh

(2020). Meta-analysis of single-case research via multilevel models: Fundamental concepts and methodological considerations. Behavior Modification, 44(2), 265–295. https://doi.org/10.1177/0145445518806867

43.

Moher

Liberati

Tetzlaff

Altman

D. G.

(2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151(4), 264–269. https://doi.org/10.7326/0003-4819-151-4-200908180-00135

44.

Moyer-Packenham

P. S.

Bolyard

J. J.

(2016). Revisiting the definition of a virtual manipulative. In Moyer-Packenham

P. S.

(Ed.), International perspectives on teaching and learning mathematics with virtual manipulatives (pp. 3–23). Springer.

45.

Moyer-Packenham

P. S.

Westenskow

(2013). Effects of virtual manipulatives on student achievement and mathematics learning. International Journal of Virtual and Personal Learning Environments, 4(3), 35–50. https://doi.org/10.4018/jvple.2013070103

46.

National Council of Teachers of Mathematics. (2010). Focus in Grade 4: Teaching with curriculum focal points.

47.

National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for Mathematics. http://www.corestandards.org/Math

48.

Nation’s Report Card. (2019). NAEP report card: Mathematics. https://www.nationsreportcard.gov/mathematics/nation/achievement

49.

M. W.

Bryant

D. P.

Bryant

B. R.

(2020). Effects of computer-assisted instruction on the mathematics performance of students with learning disabilities: A synthesis of the research. Exceptionality, 28(1), 30–44. https://doi.org/10.1080/09362835.2019.1579723

50.

M. W.

Kim

(2017). Use of iPads and iPods for academic performance and engagement of preK–12 students with disabilities: A research synthesis. Exceptionality, 25(1), 54–75. https://doi.org/10.1080/09362835.2016.1196446

51.

*Özdemir

(2017). The investigation of virtual concrete-representational-abstract model to teach one digit multiplication word problem to students with emotional and behavioral disorders (Publication No. 10284142) [Doctoral dissertation, State University of New York at Buffalo]. ProQuest Dissertations and Theses Global.

52.

*Park

Bouck

E. C.

Fisher

M. H.

(2021). Using the virtual–representational–abstract with overlearning instructional sequence to students with disabilities in mathematics. The Journal of Special Education, 54(4), 228–238. https://doi.org/10.1177/0022466920912527

53.

*Park

Bouck

E. C.

Smith

J. P.

(2020). Using a virtual manipulative intervention package to support maintenance in teaching subtraction with regrouping to students with developmental disabilities. Journal of Autism and Developmental Disorders, 50(1), 63–75. https://doi.org/10.1007/s10803-019-04225-4

54.

Peltier

Morin

K. L.

Bouck

E. C.

Lingo

M. E.

Pulos

J. M.

Scheffler

F. A.

Suk

Mathews

L. A.

Sinclair

T. E.

Deardorff

M. E.

(2020). A meta-analysis of single-case research using mathematics manipulatives with students at risk or identified with a disability. The Journal of Special Education, 54(1), 3–15. https://doi.org/10.1177/0022466919844516

55.

Petit-Bois

Baek

E. K.

Van den Noortgate

Beretvas

S. N.

Ferron

J. M.

(2016). The consequences of modeling autocorrelation when synthesizing single-case studies using a three-level model. Behavior Research Methods, 48(2), 803–812. https://doi.org/10.3758/s13428-015-0612-1

56.

Pinheiro

Bates

DebRoy

Sarkar

, & R Core Team. (2020). nlme: Linear and nonlinear mixed effects models (R package Version 3) [Computer software]. https://CRAN.R-project.org/package=nlme

57.

Pustejovsky

J. E.

Ferron

J. M.

(2017). Research synthesis and meta-analysis of single-case designs. In Kauffman

J. M.

Hallahan

D. P.

Pullen

P. C.

(Eds.), Handbook of special education (2nd ed., pp. 168–186). Routledge.

58.

*Reneau

J. L.

(2012). Using the concrete-representational-abstract sequence to connect manipulatives, problem solving schemas, and equations in word problems with fractions (Publication No. 3530320) [Doctoral dissertation, West Virginia University]. ProQuest Dissertations and Theses Global.

59.

*Root

J. R.

Cox

S. K.

Gilley

Wade

(2020). Using a virtual-representational-abstract integrated framework to teach multiplicative problem solving to middle school students with developmental disabilities. Journal of Autism and Developmental Disorders. Advance online publication. https://doi.org/10.1007/s10803-020-04674-2

60.

*Satsangi

Bouck

E. C.

(2015). Using virtual manipulative instruction to teach the concepts of area and perimeter to secondary students with learning disabilities. Learning Disability Quarterly, 38(3), 174–186. https://doi.org/10.1177/0731948714550101

61.

*Satsangi

Bouck

E. C.

Taber-Doughty

Bofferding

Roberts

C. A.

(2016). Comparing the effectiveness of virtual and concrete manipulatives to teach algebra to secondary students with learning disabilities. Learning Disability Quarterly, 39(4), 240–253. https://doi.org/10.1177/0731948716649754

62.

*Satsangi

Hammer

Evmenova

A. S.

(2018). Teaching multistep equations with virtual manipulatives to secondary students with learning disabilities. Learning Disabilities Research & Practice, 33(2), 99–111.

63.

*Satsangi

Hammer

Hogan

C. D.

(2018). Studying virtual manipulatives paired with explicit instruction to teach algebraic equations to students with learning disabilities. Learning Disability Quarterly, 41(4), 227–242. https://doi.org/10.1177/0731948718769248

64.

Satsangi

Miller

(2017). The case for adopting virtual manipulatives in mathematics education for students with disabilities. Preventing School Failure: Alternative Education for Children and Youth, 61(4), 303–310. https://doi.org/10.1080/1045988X.2016.1275505

65.

*Saunders

A. F.

(2014). Effects of schema-based instruction delivered through computer-based video instruction on mathematical word problem solving of students with autism spectrum disorder and moderate intellectual disability (Publication No. 3636163) [Doctoral dissertation, University of North Carolina at Charlotte]. ProQuest Dissertations and Theses Global.

66.

Seo

Y. J.

Bryant

D. P.

(2009). Analysis of studies of the effects of computer-assisted instruction on the mathematics performance of students with learning disabilities. Computers & Education, 53(3), 913–928. https://doi.org/10.1016/j.compedu.2009.05.002

67.

*Serianni

(2014). Supporting middle school students with emotional or behavioral disorders in blended learning: A fraction intervention using virtual manipulatives [Doctoral dissertation, University of Central Florida]. Electronic Theses and Dissertations. https://stars.library.ucf.edu/etd/4723

68.

Shin

(2018). Developing a “Fun Fraction” website of virtual manipulatives and the effects of explicit and systematic instruction with the virtual-representational-abstract instructional sequence for elementary school students with mathematics difficulties. Korea Journal of Learning Disabilities, 15(1), 57–91. https://doi.org/10.47635/KJLD.2018.15.1.57

69.

*Shin

Bryant

D. P.

(2017). Improving the fraction word problem solving of students with mathematics learning disabilities: Interactive computer application. Remedial and Special Education, 38(2), 76–86. https://doi.org/10.1177/0741932516669052

70.

Shin

Bryant

D. P.

Bryant

B. R.

McKenna

J. W.

Hou

M. W.

(2017). Virtual manipulatives: Tools for teaching mathematics to students with learning disabilities. Intervention in School and Clinic, 52(3), 148–153. https://doi.org/10.1177/1053451216644830

71.

Shin

Jung

(2018). Reliability, validity, and usability of a graph extraction computer program for the meta-analysis of single-case studies. Special Education Research, 17, 185–207. https://doi.org/10.18541/ser.2018.08.17.3.185

72.

Shin

Park

Grimes

Bryant

(2021). Effects of using virtual manipulatives for students with disabilities: Three-level multilevel modeling for single-case data [Data files and scripts]. Center for Open Science. https://doi.org/10.17605/OSF.IO/WVTXG

73.

*Simsek

(2016). Use of a game-based app as a learning tool for students with mathematics learning disabilities to increase fraction knowledge/skill (Publication No. 10145359) [Doctoral dissertation, University of South Florida]. ProQuest Dissertations and Theses Global.

74.

Singer

J. D.

Willett

J. B.

(2003). Applied longitudinal data analysis: Modeling change and event occurrence. Oxford University Press.

75.

Sung

Y.-T.

Chang

K.-E.

Liu

T.-C.

(2016). The effects of integrating mobile devices with teaching and learning on students’ learning performance: A meta-analysis and research synthesis. Computers & Education, 94, 252–275. https://doi.org/10.1016/j.compedu.2015.11.008

76.

Utah State University. (n.d.). National library of virtual manipulatives. Retrieved March 31, 2021, from http://nlvm.usu.edu

77.

van Garderen

(2006). Teaching visual representation for mathematics problem solving. In Montague

Jitendra

A. K.

(Eds.), Teaching middle school students with mathematics difficulties (pp. 72–88). Guilford Press.

78.

Viechtbauer

(2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03

79.

*Weng

P. L.

Bouck

E. C.

(2016). An evaluation of app-based and paper-based number lines for teaching number comparison. Education and Training in Autism and Developmental Disabilities, 51(1), 27–40. https://www-jstor-org-443.web.bisu.edu.cn/stable/26420362

80.

What Works Clearinghouse. (2017). What Works Clearinghouse: Procedures handbook (Version 4.0). Institute of Education Sciences.

81.

*Xin

Y. P.

Park

J. Y.

Tzur

(2020). The impact of a conceptual model-based mathematics computer tutor on multiplicative reasoning and problem-solving of students with learning disabilities. Journal of Mathematical Behavior, 58, Article 100762. https://doi.org/10.1016/j.jmathb.2020.100762

82.

Xin

Y. P.

Tzur

Hord

Liu

Park

J. Y.

(2017). An intelligent tutor-assisted mathematics intervention program for students with learning difficulties. Learning Disability Quarterly, 40(1), 4–16. https://doi.org/10.1177/0731948716648740

83.

Zbiek

R. M.

Heid

M. K.

Blume

G. W.

Dick

T. P.

(2007). Research on technology in mathematics education: The perspective of constructs. In Lester

F. K.

(Ed.), Second handbook of research on mathematics teaching and learning (pp. 1169–1207). Information Age.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.15 MB

0.00 MB