A Meta-Analysis on Computer Technology Intervention Effects on Mathematics Achievement for Low-Performing Students in K-12 Classrooms

Abstract

This meta-analysis extended the current literature regarding the effects of computer technology (CT) on mathematics achievement, with a particular focus on low-performing students. A total of 45 independent effect sizes extracted from 31 empirical studies based on a total of 2,044 low-performing students in K-12 classrooms were included in this meta-analysis. Consistent with previous reviews, this study suggested a statistically significant and positive effect of CT ( $\bar{d}$ = 0.56) on low-performing students’ mathematics achievement. Of four CT types, the largest CT effect was found with problem-solving system ( $\bar{d}$ = 0.86), followed by tutoring $(\bar{d}$ = 0.80), game-based intervention ( $\bar{d}$ = .58), and computerized practice ( $\bar{d}$ = .23). Furthermore, other moderators were found to explain variation in CT effects on low-performing students’ mathematics achievement. Study findings contributed to clarifying the effect of CT for low-performing students’ mathematics achievement. Implications for instructional design and practices are also discussed.

Keywords

computer technology mathematics instructional design meta-analysis low-performing K-12

The use of computer technology in K-12 classrooms has been growing at a stunning pace over the past several decades. In the late 20th century, 85% of U.S. schools had computers, and 64% of U.S. schools had access to the internet (Coley et al., 1997). Another national report published in 2010 showed that almost all schools owned computers, and all teachers had access to one or more computers in classrooms (Gray et al., 2010). As evident in the increasing use of computer technology in schools, technology has become a critical instructional technology in teaching and learning.

Several policies documents—including the Common Core State Standards for Mathematics (National Governors Association Center for Best Practices & Council of Chief State School Offices, 2010), Principles and Standards for School Mathematics (NCTM, 2000), and Adding It Up (NRC, 2001)—have emphasized that technology plays a vital and centralized role in mathematics teaching and learning. Incorporating technology in mathematics classrooms makes substantial contributions to student mathematics learning in various ways, including aiding for mathematics conceptualizing and modeling (NCTM, 2000; Tucker, 2018); allowing students to make conjectures, collect and record data (Cullen et al., 2020; Martinez, 2017); and providing chance to engage in complex problem-solving or learning activities (Hobbs, 2012). Given a wide array of technology (i.e., computer program, graphing calculators, visual reality) applied in mathematics classrooms, in this work, we focused on computer technology (CT), which refers to the use of computer software/applications or web-based programs for providing instruction, managing or monitoring student learning, or any other learning purposes. Online tutoring or learning environments that could provide differentiated and individualized interventions are included, while some types of distance learning (i.e., watching recorded lectures) are excluded.

The increasingly popular use of CT in mathematics education has stirred up a huge amount of research that focuses on the effectiveness of different types of CT across multiple aspects of mathematics learning. For example, Chang et al. (2006) found that a computer-based problem-solving system produced a large significant effect (d = 0.78) on fifth-grade students’ problem-solving tests. Another widely used problem-solving software, Go Solve Word Problems, was also found to have a large effect of improving problem-solving skills for fifth graders (Fede et al., 2013) and for second graders (Leh & Jitendra, 2013). The possible explanation for the big impact is that this software was designed to include instructional elements that are particularly beneficial for student problem-solving, such as using schematic diagrams, strategy instruction, and instructional feedback. Hwang et al. (2006) found that a web-based multimedia whiteboard system significantly increased student performance of mathematics problem solving, especially when students’ critical thinking behind solutions was elicited and collaborative peer learning was encouraged.

Besides, improving students’ arithmetical skills and fluency is a big concern for the CT usage in mathematics classroom, particularly in lower grades. Researchers have conducted a series of studies focusing on how game-based technology or software is supported for early number skills development. Wilson et al. (2009) found that an adaptive computer game (The Number Race) had a positive effect for kindergarteners’ number sense. Other researchers found this game intervention was also effective for improving kindergarten students’ number skills, such as number comparison (Räsänen et al., 2009) and basic arithmetic (Salminen et al., 2015a). Salminen et al. (2015b) detected that another similar adaptive computer game (GraphoGame Math) had positive impact for kindergarten students’ verbal counting and dot counting fluency.

In sum, CT interventions were found to be effective in fostering student mathematics learning. As these CT interventions have become more sophisticated and more accessible, the expectations that CT can improve schooling mathematics outcomes have increased. Many researchers now see CT as a potential solution to the issues of low-performing mathematics students.

Improving Mathematics Achievement for Low-Performing Students

The No Child Left Behind Act placed great emphasis on high mathematics achievement for all students, especially for subgroup populations who have traditionally performed below expectations (Ohnemus, 2002). Addressing the need for improving mathematics achievement for low-performing students remains a big concern. Years of effort failed to enhance students’ proficiency levels in mathematics achievement. The report from National Assessment of Educational Progress (National Center for Education Statistics, 2005) showed that 63% of fourth graders and 70% of eighth graders performed below proficiency in mathematics. The updated report of NAEP in 2011 showed that there were still 60% of fourth-grade students and 65% of eighth-grade students who failed to reach the proficient level in mathematics (National Center for Education Statistics, 2011).

CT is highly anticipated to increase the success of academically disadvantaged students. Before we dive into how CT can address the issues of low-performing (LP) students, it is necessary to first explain our position regarding this term since there is inconsistent usage around the meaning and scope of low-performing (LP) and learning disability (LD) or mathematical learning disability (MLD). For instance, Robotti and Baccaglini-Frank (2017) defined students with MLD as students with persistent low achievement in mathematics and students who have been diagnosed with LD. While students are labeled as LD when they are clinically diagnosed with situations, such as mental or emotional disorder, or having severe difficulties in obtaining or retaining information (Bateman, 1992); hence, they are possible to be at risk for mathematics learning disability and tend to perform low in mathematics. In the current study, LP students means those who score below the basic proficiency level of mathematics tests but which is not caused by learning disability or mental disability. Given the inconsistent criteria of LP in literature, for this study, as long as students are clearly defined as scoring below basic proficiency level based on a mathematics test in the included study, they are incorporated as participants.

One early review found that LP students ( $\bar{d}$ = 0.44) profited most from the exposure of CT intervention compared to students with average achievement ( $\bar{d}$ = 0.39) and high achievement ( $\bar{d}$ = 0.19) in the combinational outcomes of math, reading, and science (Niemiec et al., 1987). In order to test whether the effect of CT interventions was differential based on the content areas, Swan et al. (1990) conducted an empirical study focusing only on educationally disadvantaged students and they found that CT interventions had slightly higher effect for increasing mathematics achievement than for reading performance. Decades passed, a great deal of research work involving the effects of CT interventions on LP students’ mathematics achievement emerged.

However, literature shows inconsistent results regarding CT intervention effects on LP students’ mathematics achievement. While CT interventions have significant and positive effects on LP students’ mathematics gains (Burns et al., 2012; Chappell et al., 2015; Kanive et al., 2014; Karner, 2016; Kulik et al., 1980; Ok & Bryant, 2016; Outhwaite et al., 2017; Perkins & Gilman, 2002; Räsänen et al., 2009; Rutherford et al., 2014; Zunker, 2008), some fail to show significant effects (Arnold, 2013; Cook, 2008; Leh & Jitendra, 2013).

Further, Brown (2018) found that instructions with laptops were less effective for LP students. Tienken and Maher (2008) reported a large and negative effect ( $\bar{d}$ = −1.63) of computer drill and practice for academically weak students. Insignificant results of CT intervention found in Brown’s study might stem from rigor in research design, which is based on a relatively small sample size in each group. Such issues might also explain Tienken and Maher’s study findings (14 in the experiment group vs. 22 in the control group). In addition, the misalignment between CT types and the needs of LP students might also explain ineffective results. For instance, the negative effect of computer drill and practice may indicate the practice of assignments could not satisfy low performers’ needs of conceptual understanding. However, other researchers have found a positive influence of drill and practice on LP students’ achievement (Burns et al., 2012; Fuchs et al., 2006; Reynolds, 2010). The inconsistent study findings regarding the effectiveness of CT interventions for LP students call for updated systematic research for this specific population in K-12 levels.

CT Interventions on Mathematics Achievement in Previous Reviews

In the past four decades, over twenty meta-analyses have been performed to examine the effects of CT interventions on student learning (Bayraktar, 2001; Blok et al., 2002; Demir & Başol, 2014; Fletcher-Flinn & Gravatt, 1995; Goldberg et al., 2003; Hsu, 2003; Kulik, 1994; Kulik et al., 1985; Niemiec et al., 1987; Schenker, 2007). These reviews examined a wide range of subjects (i.e., reading, science, writing, mathematics, social study, and statistics) and different grade levels from kindergarten to college stage. The majority of the reviews include students from K-12 and only three reviews focused on college students’ statistics achievement (Larwin & Larwin, 2011; Schenker, 2007; Sosa et al., 2011). Very few reviews (only eight) included LP students or students with special needs. Thirteen out of 23 reviews focused only on mathematics achievement. All the reviews reported that there were positive effects of technology on students’ mathematics achievement, with effect sizes ranged widely, from 0.09 to 1.80. Table 1 presents a summary of findings for mathematics outcomes for these major 23 reviews.

Table 1.

Summary of Results From Previous Meta-Analyses on CT Effect.

Author	Years covered	Grades	LP cover	NS	Subjects	Technology	ES
Hartley (1977)	1960–1975	Elementary	No	22	Mathematics	CBI	0.40
Kulik et al. (1985)	1966–1984	Elementary	Yes	28	Mathematics/Others	CBI	0.54
Bangert-Drowns et al. (1985)	1968–1982	Secondary	Yes	22	Mathematics/Others	CBI	0.26
Niemiec et al. (1987)	1966–1983	Elementary	Yes	NA	Mathematics/Others	CBI	0.45
Kulik & Kulik (1991)	1966–1986	Elementary to college	No	9	Mathematics/Others	CBI	0.39
Xin & Jitendra (1999)	1986–1996	Grade 1-12	Yes	25	Mathematics	CAI	1.80
Kroesbergen & Van Luit (2003)	1985–2000	K-12	Yes	58	Mathematics	CAI	0.52
Christmann & Badgett (2003)	1966–2001	Elementary	No	12	Mathematics/Others	CAI	0.34
Schenker (2007)	1974–2005	College	No	47	Statistics	Technology	0.24
Liao (2007)	1983–2003	Elementary to college	No	12	Mathematics/Others	CAI	0.29
Slavin & Lake (2008)	1971–2006	Elementary	No	38	Mathematics	CAI	0.19
Slavin et al. (2009)	1971–2007	Secondary	No	38	Mathematics	CAI	0.1
Rakes et al. (2010)	1968–2008	Elementary to college	No	36	Mathematics	Technology	0.16
Li & Ma (2010)	1990–2006	K-12	Yes	46	Mathematics	Technology	0.71
Sosa et al. (2011)	NA	College	No	45	Statistics	CAI	0.33
Larwin & Larwin (2011)	1960–2010	College	No	70	Statistics	CAI	0.56
Cheung & Slavin (2013)	1980–2012	K-12	No	74	Mathematics	Technology	0.15
Steenbergen-Hu and Cooper (2013)	2001–2010	K-12	Yes	26	Mathematics	ITS	0.09
Demir & Başol (2014)	NA	Secondary	No	40	Mathematics	CAI	0.89
Jitendra et al. (2018)	1990–2017	Grade 6-12	Yes	19	Mathematics	CBM	0.39
Chauhan (2017)	2001–2016	Elementary	No	41	Mathematics/Others	Technology	0.54
Byun & Joung (2018)	2005–1014	K-12	No	24	Mathematics	DGBL	0.37
Higgins et al. (2019)	1985–2013	K-G8	No	24	Mathematics	Technology	0.49

Note: Effect sizes were extracted from mathematics outcomes for studies focusing on multiple subjects. NS refers to the number of studies for mathematics. LP Coverage refers to studies cover low-performing students as separate groups. CAI: Computer-assisted instruction, CBI: Computer-based instruction, CBM: Computer-based modules, DGBL: Digital game-based learning, ITS: Intelligent tutoring systems.

Various CT Interventions

Meta-analyses before 2010 mainly focused on computer-assisted instruction (CAI) and computer-based instruction (CBI), which were the most prevalent technology-supported teaching and learning in the early years. CAI is a method of instruction in which computer contained instruction is designed to teach, guide, or aid students until they reach the desired level (AECT Task Force on Definition & Terminology, 1977). CBI is a broad concept that refers to either stand-alone computer learning activities or computer-mediated activities that teachers are also involved in (Cotton, 1991). Several meta-analyses found CBI had a medium effect on mathematics achievement for elementary and secondary students (Bangert-Drowns et al., 1985; Hartley, 1977; Jitendra et al., 2018; Kulik et al., 1985; Kulik & Kulik, 1991; Niemiec et al., 1987) and for college students (Kulik et al., 1980). While the effects of CAI on mathematics achievement varied from small to large (Kroesbergen & Van Luit, 2003; Liao, 2007; Slavin et al., 2009; Slavin & Lake, 2008; Xin & Jitendra, 1999). After 2010, the focus of review studies extended to technology that is beyond the traditional CAI or CBI. Another trend is to focus on reviewing specific types of technology, such as Intelligent tutoring systems (Steenbergen-Hu & Cooper, 2013) and digital game-based learning (Byun & Joung, 2018).

Factors Matter to CT Application

The literature on the usage of CT in mathematics classrooms suggests that the impact of CT on mathematics learning may rely on many other contextual and methodological factors. Of them, CT type is one of most crucial factors that has been widely examined. Lou et al. (2001) meta-analysis found that CBI-tutor programs ( $\bar{d}$ = 0.20), when combined with drill and practice, were more effective than computer tool programs ( $\bar{d}$ = 0.04). Another meta-analysis indicated that drill and practice were most effective in raising students’ mathematics scores compared to CAI, tutorial, and problem-solving systems (Niemiec et al., 1987). Cheung and Slavin (2013) have found that CAI had the largest effect with an effect size of 0.18 on K-12 students’ mathematics achievement compared with computer-management learning ( $\bar{d}$ = 0.09) and comprehensive programs ( $\bar{d}$ = 0.06). However, Li and Ma (2010) found that all types of CT (tutoring, communication media, tools, and exploratory environment) demonstrated the same effects on mathematics achievement.

Another two important factors related to treatment have been found to moderate the effects of CT interventions—that is intervention duration and intervention intensity. Consistent results were found that the shorter intervention had a larger effect on students’ mathematics achievement than the longer one despite the disunifying criteria that define whether an intervention is short or long. For example, Li and Ma (2010) found that shorter duration (less than one-year intervention) had a larger effect on students’ mathematics achievement than interventions which lasted longer (more than one year). Chauhan (2017) detected that technology produced the highest effect when intervention continued less than one week, compared to intervention lasting between one week and four weeks or intervention more than one month. Other previous reviews also found that longer interventions were less effective than shorter interventions (Gersten et al., 2009; Kroesbergen & Van Luit, 2003; Kulik et al., 1985). Regarding the intervention intensity, much fewer meta-analyses have focused on it. Only one meta-analysis found that higher intensity with more than 30 min CT intervention in a week had a bigger effect on students’ mathematics achievement than lower intensity with less than 30 min treatment in a week (Cheung & Slavin, 2013). Based on these findings, it seems that high intensity within shorter intervention duration was possibly more effective, but no studies had been conducted to check the combination effect of duration and intensity.

Intervention contents have also been found to moderate technology effect. Kroesbergen and Van Luit (2003) found that interventions on basic facts had the largest effect than on problem-solving or arithmetic skills for increasing the gain scores of students with special needs. This finding may be explained by the fact that it may be easier to teach basic skills to students with special needs than to teach problem-solving skills. However, when compared to other research syntheses, the effect of computer interventions on problem solving has also been detected (Xin & Jitendra, 1999).

Other factors that may contribute to mathematics learning with technology are the participant characteristics including, students’ grade level and student type. A consistent result by grade levels has been found that CT interventions are more effective for elementary students than for secondary students (Bangert-Drowns et al., 1985; Cheung & Slavin, 2013; Kulik et al., 1985; Li & Ma, 2010). As for student type, researchers found the effectiveness of CT interventions across different student populations. For instance, Traynor (2003) showed that CT improved mathematics achievement among various student populations, including students in special education, students in regular education, and students with less proficiency in English.

However, when comparing the effects of CT interventions between students with special needs and general education students, results have not been consistent. Some reviews found that CT was noticeably more effective for students with special needs than for general education students (Li & Ma, 2010; Niemiec et al., 1987; Sivin-Kachala & Bialo, 1994), while the review from Steenbergen-Hu and Cooper (2013) found that the effects of CT for helping LP students were smaller than for helping general education students.

Considering the inconsistent results regarding CT effects for LP students that widely existed in both review studies and a large number of empirical studies, as well as the vacuum of meta-analytical research only for LP students, it is emergent to conduct a meta-analysis merely for this particular population. The two existing meta-analyses conducted by Kroesbergen and Van Luit(2003) and Xin and Jitendra (1999) included LP students as participants, but they also used other non-computer intervention studies. In order to extend these reviews and provide a complete picture about CT effects for LP students, our meta-analysis focuses on computer intervention studies published after 2000.

Goals and Research Questions

Our meta-analysis is to explore the effects of CT interventions on K-12 LP students’ mathematics achievement. We are particularly interested in understanding how student characteristics and methodological features moderate the effect of CT on low-performers’ mathematics achievement. In order to accomplish our research goals, the following two specific research questions are addressed in our meta-analysis.

What is the overall effectiveness of CT interventions on the mathematics achievement of K-12 LP students, compared to students without CT interventions?

What moderators (i.e., grade levels, types of CT, intervention components, duration of intervention, intervention intensity, and types of testing instrument) explain variation in the effect of CT interventions on the mathematics achievement of K-12 LP students?

Method

Inclusion Criteria

The following inclusion and exclusion criteria were developed based on our research purposes.

Study involves CT in the instruction or learning process.

Study measures mathematics achievement as an outcome.

Study allows extraction of LP student data as a separate set.

Study includes treatment group where students received the CT intervention and control group where students did not.

Study reports sufficient data for calculating effect sizes.

Study involves students from K-12.

One challenging part of applying the inclusion criteria was to determine whether “at-risk” students had been identified because they were, in fact, low-performing or for some other reasons. Studies that explicitly defined at-risk students as being low performing were included; otherwise, those studies were excluded. Moreover, studies that reported students as being either “at-risk” or “low performing” as a consequence of learning disabilities were excluded.

Search Process

The literature search and screening process followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol (Moher et al., 2009), as shown in Figure 1.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Diagram of Selection Process.

Studies for our meta-analysis were identified by the following three major steps: (1) electronic database search, (2) ancestry searches, and (3) manual search on peer-reviewed journals. Our first step was a comprehensive literature search of electronic databases, including ERIC, Primary Search, Educational Administration Abstracts, Public Administration Abstracts, PsycBOOKS, PsycARTICLES, PsycINFO, and PsycEXTRA via EBSCOhost. In this electronic database search, several combinations of the following keywords were used: mathematics outcome keywords including mathematics achievement, mathematics performance, problem solving in mathematics, mathematics education, and mathematics research; intervention keywords including computer technology, computer-assisted instruction, computer-based instruction, technological Interventions, and computer interventions; and target population keywords including low-achieving students, LP students, low performance, low achievement, underachievement, students at risk, struggling learners, slow learner, below-average, poor mathematics performance, learning problems, and learning difficulties. The initial search resulted in 354 studies.

Then in the second step, references of studies in our initial step were searched for locating any additional studies. Eighty-six potential studies were found in this second step. In the third step, we manually searched 11 peer-reviewed journal outlets, including Educational Technology Research and Development, British Journal of Educational Technology, Journal of Research on Technology in Education, Journal of Computers in Mathematics and Science Teaching, Journal of Research in Mathematics Education, Educational Studies in Mathematics, Journal of Mathematics Behavior, Mathematics Thinking and Learning, Australasian Journal of Educational Technology, Computer and Education, and Journal of Educational Computing Research. Twelve of the related studies were found in this third step. As a result, with three different searches, we identified 452 studies as potential literature to be included in our meta-analysis.

Of more than 450 studies, the first step to remove 60 replicated records. The second screening process is based on titles and abstracts; as a result, 103 studies were excluded and 289 full-text studies were produced. The last step was a detailed review by applying our inclusion/exclusion criteria. A further 258 studies were excluded, of which 59 studies did not use CT as interventions, 31 studies did not measure mathematics as outcomes, 84 studies cannot allow to extract LP students as a separate set, six studies were not conducted for K-12 students, 54 studies had no control conditions, and 24 studies did not report sufficient data so that effect sizes can be computed. As a result, a total of 24 independent journal articles and seven dissertations were included in our meta-analytic review.

Study Coding

In order to examine the methodological and substantive features that might moderate the effect between interventions and student mathematics achievement, a set of study features was coded based on theoretically defined constructs and categories that emerged from the coding process. The six aspects were coded for each primary study, including grade level, types of CT, intervention components, duration of intervention, intervention intensity, and types of testing instrument.

Grade Levels (Kindergarten; Primary, 1–6; Junior High, 7–10; Senior High, 11–12, and Other)

We adopted the common way to categorize grade 1 to grade 12 into three groups, including primary school (grade 1 to grade 6), junior high school (grade 7 to grade 10), and senior high school (grade 11 to grade 12). We separated kindergarten as an individual group, considering that there is a certain number of CT particularly designed for improving kindergarten students’ number sense and skills. “Other” is the group that represented grade in other forms (i.e., age 9–18).

Type of CT (Tutoring; Game-Based; Computerized Practice; Problem-Solving System)

Different types of CT—ranging from early CAI, or computer-mediated communication, or computer or tablet game to recent intelligent tutoring system—have been developed and applied for enhancing mathematics teaching and learning. Of many classifications of CT types, the common one is suggested by Means (1994), which includes four main categories: (a) tutorial, (b) communication media, (c) exploratory environment, and (d) tools. These CT types have been widely examined by other meta-analytical research (Li & Ma, 2010; Lou et al., 2001). However, it is possible that some CT types have not yet been implemented with LP students. Hence, the existing categories are not the best way to capture the updated CT for improving students’ low performance. Therefore, based on what has been empirically used in the current literature, we categorized CT into four groups: (1) tutoring, (2) game-based technology, (3) computerized practice, and (4) computer-based problem-solving system.

The first category is tutoring, referring to web-based programs that provide instructional guidance or practices, especially for students who have difficulties in learning (Karner, 2016; Tsuei, 2012). Gamed-based technology covers mathematics concepts or presenting learning activities by using educational games in a computer-based environment. Compared with the traditional drill and practice, we defined computerized practice differently, which refers to software or applications that provide adaptive, iterative, and continuous practice on a stream of mathematics skills. Hence, it markedly differed from the traditional one in multiple ways (Cibulka & Cooper, 2017), including: (a) practice is adaptive to students’ performance level, (b) intensive practice and students’ performance report are provided, and (c) students receive immediate and corrective feedback based on their performance. Additionally, we added a new category—computer-based problem-solving system, considering the increasing trend of embedding problem-solving complexity into CT. This system refers to a computer-based learning environment or program that provides problem-solving instructions or guidance to assist students’ solving process (i.e., word problem solving procedures).

Intervention Components (Arithmetic Skill; Problem Solving; Mathematical Reasoning; Multiple Skills)

Kroesbergen and Van Luit (2003) claimed that it might be easier to teach basic arithmetic skills to students with special needs than to teach problem-solving skills. For testing whether this claim still remains the same when newly complex CT interventions were applied for LP students, we basically follow the definitions about intervention components from their meta-analysis. Arithmetic skills are the skills to do basic mathematical operations (i.e., addition, subtraction, and multiplication). For mathematics problem solving, it not only involves basic mathematical skills, but also involves how to apply knowledge into new situations. Another intervention component, multiple skills, includes more than one mathematics skill, such as basic computations and problem solving, etc. Mathematical reasoning is a kind of high-order thinking skill, that refers to integrate skills and knowledge to reason mathematically.

Duration of Intervention (1. 0–11 Weeks; 2. 12–23 Weeks; 3. 24–35 Weeks; 4 ≥ 36 Weeks)

The duration of intervention program is a very important contextual factor for the CT effects. Cheung and Slavin (2013) claimed that studies with less than 12-week intervention are brief experiment, which can often easily create experimental conditions that could not be maintained for a whole school year. Hence, we adopted this 12-week period as a cut-off point to divide the duration category.

Intervention Intensity (Low, Less Than 60 Min per Week; Medium, Between 60 and 150 Min per Week; High, More Than 150 Min per Week)

Intervention intensity is the frequency of CT usage. We used the common unit by how many minutes per week. Cheung and Slavin (2013) categorized less than 30 minutes per week as the low intensity program, between 30 and 75 min per week as the medium, and more than 75 min per week as the high intensity. The criteria are not suitable for our meta-analysis as the range of intensity is much bigger from 0 min per week to 275 min per week. We decided to use 150 min (near half of the highest dosage) as the cut-off point to differentiate medium and high intensity, and 1 hour as the point to cut off medium and low intensity.

Types of Testing Instrument (Standardized Tests, Teacher-Developed; Researcher-Developed)

When used to measure student mathematics outcomes, studies with non-standardized tests showed greater effects of CT than studies with standardized tests (Li & Ma, 2010). Standardized tests could present in different forms, such as state-standardized tests, district-standardized tests, or national standardized tests. In our work, non-standardized tests were specifically grouped into teacher-developed tests and researcher-developed tests.

Based on these six study features, thirty-one primary studies were coded. A detailed summary of primary studies with the author of the study, publication year, sample size (number of low performing), the cut-off points for identifying LP students, grade levels, types of CT, duration of intervention, intervention intensity, test types, intervention components, and effect sizes is provided in Table 2. To ensure consistency in interpretation of coding criteria, the weighted Kappa (k = 0.90) was used to measure interrater reliability. It showed there was a substantial agreement between two coders based on Cohen’s (1960) criteria.

Table 2.

Coded Information for the Included Studies.

Study	Pub. year	N (low-performing)	Cut-off	Grade level	Types of technology	Duration of intervention (week)	Intervention dosage (min)	Test types	Components	ES (d)
Arnold	2013	26	15%	5	Tutoring	24	250	SST	Multiple	0.00
Karner	2016	369	NA	9	Tutoring	28	NA	CBT	Math reasoning	0.11
Chappell et al.	2015	49	14.30%	5–6	Tutoring	20	60	SST	Multiple	0.95
Chappell et al.	2015	50	14.30%	5–6	Tutoring	20	74	SST	Multiple	1.50
Harris	2011	192	11.40%	9	Tutoring	72	N/A	SST	Multiple	1.20
Baroody et al.	2013	21	25th	1	Game-based	10	60	NST	Arithmetic	0.45
Baroody et al.	2013	22	25th	1	Game-based	10	60	NST	Arithmetic	0.96
Baroody et al.	2013	21	25th	1	Game-based	10	60	NST	Arithmetic	0.64
Outhwaite et al.	2017	27	NA	K	Problem-solving	13	150	TDT	Math reasoning	1.20
Kanive et al.	2014	28	25th	4–5	Practice	2	30	RDT	Problem solving	0.37
Räsänen et al.	2009	15	8.5%	K	Game-based	3	62.5	RDT	Arithmetic	0.19
Räsänen et al.	2009	15	8.5%	K	Game-based	3	62.5	RDT	Arithmetic	0.09
Chang et al.	2006	24	37%	5	Problem-solving	6	80	RDT	Problem solving	0.57
Chang et al.	2006	24	37%	5	Problem-solving	6	80	RDT	Problem solving	0.78
Burns et al.	2012	65	15th–25th	3	Practice	8 to 15	45	RDT	Arithmetic	0.33
Burns et al.	2012	74	15th–25th	4	Practice	8 to 15	45	RDT	Arithmetic	0.28
Burns et al.	2012	34	15th	3	Practice	8 to 15	45	RDT	Arithmetic	0.37
Burns et al.	2012	43	15th	4	Practice	8 to 15	45	RDT	Arithmetic	0.46
Leh & Jitendra	2013	13	50th	3	Problem-solving	6	125	DST	Problem solving	1.03
Leh & Jitendra	2013	13	50th	3	Problem-solving	6	125	SST	Multiple	−0.35
Shin et al.	2012	10	50%	2	Game-based	9	37.5	RDT	Arithmetic	2.24
Salminen et al.	2015b	13	43%	K	Game-based	3	57.5	RDT	Arithmetic	1.27
Huang et al.	2012	17	25%	2–3	Problem-solving	4	NA	RDT	Problem solving	1.61
Moyer-Packenham & Suh	2012	13	22.4%	5	Game-based	2	250	RDT	Arithmetic	0.64
Leh	2011	13	50%	3	Problem-solving	6	97.5	RDT	Problem solving	2.01
Salminen et al.	2015a	9	10th	K	Game-based	3	52.5	RDT	Arithmetic	1.29
Salminen	2015	16	11–25th	K	Game-based	3	52.5	RDT	Arithmetic	0.22
Salminen	2015	17	11–25th	K	Game-based	3	57.5	RDT	Arithmetic	0.03
Salminen	2015	8	11–25th	K	Game-based	3	57.5	RDT	Arithmetic	−0.04
Salminen	2015	6	11–25th	K	Game-based	3	57.5	RDT	Arithmetic	0.50
Hassler Hallstedt et al.	2018	151	60th	2	Practice	20	63	RDT	Arithmetic	0.60
Alexis	2015	231	40%	12	Tutoring	144	240	NST	Arithmetic	0.52
Afzal	2014	24	NA	6	Practice	10	NA	RDT	Arithmetic	1.28
Bos	2007	48	30th	11	Problem-solving	1	275	SST	Multiple	0.68
Fede et al.	2013	16	25th	5	Problem-solving	12	90	RDT	Problem solving	0.78
Fuchs et al.	2006	16	50th	1	Practice	18	25	RDT	Arithmetic	1. 35
Reynolds	2010	26	50th	2–6	Practice	9-13	52.5	CBT	Arithmetic	0.54
Reynolds	2020	30	NA	2–6	Practice	9-13	52.5	CBT	Arithmetic	0.26
Pellerito	2011	30	25th	9	Tutoring	36	N/A	SST	Multiple	0.30
Tienken & Maher	2008	14	20%	8	Practice	20	90	NST	Multiple	−1.63
Ysseldyke et al.	2003	35	NA	3–5	Practice	36	N/A	DST	Multiple	0.22
Brown	2018	55	NA	9	Tutoring	72	N/A	CBT	Multiple	−0.73
Brown	2018	47	NA	9	Tutoring	72	NA	CBT	Multiple	−1.05
Luft et al.	2013	51	24%	9 to 18 years	Game-based	NA	N/A	RDT	Arithmetic	0.71
Tsuei	2012	23	27th	3–4	Tutoring	36	120	RDT	Problem solving	1.15

Note: For test type, SST is the state standard test, NST is the national standard test, DST is the district standard test, TDT is the teacher-developed test, RDT is the researcher-developed test, and CBT is the course-based test.

Effect Size

The primary effect size for this study is a standardized mean difference (d), which quantifies the effect of CT intervention on the outcome. First, for studies using post-test control group design, the effect sizes were computed as the formula (Cooper & Hedges, 1994):

d = \frac{{\bar{X}}_{\exp} - {\bar{X}}_{con}}{\sqrt{S_{pooled} / (n_{\exp} + n_{con} - 2)}}

(1)

where

{\bar{X}}_{\exp}

is mean of outcome for experiment group with CT interventions and

{\bar{X}}_{con}

is mean of outcome for control group without CT intervention; The

S_{pooled} = {SD}_{\exp}^{2} (n_{\exp} - 1) + {SD}_{con}^{2}

(

n_{con} - 1)

, where SD is standard deviation of the outcome variable; and

n_{\exp}

is sample size for experiment group, and

n_{con}

is sample size for control group.

The associated variance for d computed via Equation (1) was calculated as

v_{d} = \frac{n_{\exp} + n_{con}}{n_{\exp} n_{con}} + \frac{d^{2}}{2 (n_{\exp} + n_{con})}

(2)

Second, when Eta-squared (η²) is reported in the studies for estimation of the effects of CT, the η² is converted into Cohen’s d via this formulation provided by Cohen (1988):

d = 2 \sqrt{(\frac{η^{2}}{1 - η^{2}})}

(3)

Lastly, when Pearson Product Moment Correlation coefficient (r) was reported in the studies, the reported r was transformed to Cohen’s d via the following formulation provided by Rosenthal (1994).

d = \sqrt{(\frac{{a r}^{2}}{1 - r^{2}})}

(4)

where

a

is correction factor for cases when sample sizes are unequal (n₁ ≠ n₂) and

a

is given by

a = \sqrt{(\frac{{(n_{1} + n_{2})}^{2}}{n_{1} n_{2}})}

(5)

when the sample sizes are equal, it will yield

a

= 4. If n₁ and n₂ are not known precisely, then the n₁ = n₂ is used for producing the value of

a

The associated variance for d computed by Equation (4) was calculated via

v_{d} = \frac{v_{r} {(d^{2} + a)}^{3}}{a^{2}}

(6)

where

v_{r}

is the variance of correlation and computed through the following equation:

v_{r} = \frac{{(1 - r^{2})}^{2}}{n - 1}

(7)

Handling Dependent Effect Sizes

Multiple effect sizes can be generated from each study based on different experimental conditions, sample population, outcome measures (i.e., problem solving and mathematical reasoning), or different duration of intervention. Generally, there are two approaches used for extracting findings from studies in a meta-analysis: a single effect size per study and multiple effect sizes per study. According to Lou et al. (2001) and Li and Ma (2010), extracting one effect size per study can guarantee the independence of each finding. However, this approach may cause the loss of difference within a study between different sample groups, between different treatment conditions, or between different outcome domains. Among several available methods of handing dependent issues, Borenstein et al. (2011) suggested to averaging the dependent effect sizes if they come from the same outcome. Along with this method, we extracted multiple findings only if a study had multiple effect sizes from different sample groups, such as different grade levels (e.g., grade 3 vs. grade 4) or sample sources (school 1 vs. school 2). In other cases, such as the same subjects for different mathematics domains or different experimental conditions, we computed an overall average effect size for each study. This approach of taking the arithmetical average of effect sizes allows us to eliminate the problem of dependencies due to multiple effect sizes, and it ensures the usage of all study features for each primary study (Cheung & Slavin, 2013; Li & Ma, 2010). In our study, following this approach, we produced 45 effect sizes from 31 studies.

Publication Bias

Funnel plots, displaying in Figure 2, are simple scatterplots of effect sizes estimated from individual studies against the inverse values of their associated standard errors. Publication bias might not be a concern on the outcome for this study due to the symmetrical appearance of the funnel plot, as shown in Figure 2. Similar results were drawn from both Egger’s regression test (z = 1.08, p = .28) and Mazumdar’s rank test (p = .25), suggesting that little evidence exists to support for the publication bias.

Figure 2.

Funnel Plots of Effect Sizes of Mathematics Achievement.

Data Analysis

The R statistical software (Core Team R, 2013) with the metafor package (Viechtbauer, 2010) was used to estimate the overall effect and a series of moderator effects.

The Overall Effect

The overall effect of CT interventions on LP students’ mathematics achievement was estimated. The Q_total was first conducted to examine whether the effect sizes are consistent across studies. When Q_total is found to be statistically significant, it indicates that the observed individual effects come from different populations, then the random-effects models were used to estimate the overall effect, where the between-study variance is estimated using Restricted Maximum Likelihood (REML) estimation method (Veroniki et al., 2016). Otherwise, the fixed-effects model is used to estimate the overall effect.

Moderator Analysis

For the moderator analysis, we first examine the significance of between-class homogeneity (Q_within). When Q_within was found to be significant, the mixed-effects moderator model was used. Otherwise the fixed-effects moderator model was used. If there is a significant group difference based on Q_model, a pairwise comparison using Tukey adjustment was used to find the significant pairs of groups.

Results

Description of Studies

In total, forty-five effect sizes were extracted from 31 primary studies. In those 31 studies, a total of 2,044 LP students with individual ranges from 9 to 369 was included. Across 31 studies, six studies were published before 2010, while all other studies were published between 2010 and 2018. Of 2,044 LP students, there were 142 kindergarteners, 865 primary-school students, 707 junior high school students, and 279 senior high school students, and the remaining 51 students’ grade-level information was not reported. In terms of types of CT used in 31 studies, eight studies adopted tutoring, game-based technology, and computerized practice as interventions, respectively, and seven studies used a problem-solving system as instruction interventions.

In terms of intervention duration, most of the studies (16 out of 31) conducted the CT interventions with a range from one to 11 weeks, six studies with an intervention of between 12 to 23 weeks, three with a range from 24 to 36 weeks, five studies implemented more than 36-week interventions, and the intervention duration is not available in one study. The highest intervention lasted four years. In terms of intervention intensity, seven studies implemented a low intensity intervention with less than 60 min per week, nine studies with a medium intensity intervention of between 60 min and 150 min per week, and seven studies with a high intensity intervention of more than 150 min per week, while eight studies did not report the intensity information. The majority of studies (14 out of 31) implemented CT interventions on students’ arithmetic skills, nine studies focused on intervening on the aspect of problem solving, and only two studies worked on students’ mathematical reasoning. In addition, six studies adopted a comprehensive CT intervention with a combination of concepts, problem solving, or mathematical application. As for the tests that were used to test students’ performance, most of the studies (17 out of 31) preferred to use a researcher-developed test, and ten studies gained students’ mathematics scores by using the standardized tests and four studies by using teacher-developed tests.

Overall Effect of CT on Mathematics Achievement

The overall effect of CT on LP students was evaluated by the outcome variable of mathematics achievement. The test of the overall homogeneity of the effect size was significant (Q (44) = 291.28, p < .01), indicating that more than one population exists underlying the included studies. Therefore, the random-effects model was used to compute the overall effect size.

Under the random-effects model, CT was found to be significant in mathematics achievement for LP students (z = 5.83, p < .01). The standardized mean difference was 0.56, with a 95% CI of 0.37 and 0.75. The I-squared value of 85.34% suggests that around 85% of the total variation in effect sizes is due to between-study variation (Higgins & Thompson, 2002).

The significant estimated effect size value indicates that mathematics achievement and CT are significantly and positively related to each other with a large magnitude ( $\bar{d}$ = 0.56, p < .01). These results suggest that CT interventions were largely effective for LP students’ mathematics achievement.

Moderating Effect of CT

A series of moderator analyses were performed to examine whether the relationship between CT usage and mathematics achievement of LP students was significantly different depending on six substantive features of the current study: types of CT, grade levels, duration of intervention, intensity, intervention components, and types of testing instrument. The estimation of mean effect sizes for each moderator factor under the mixed-effect model is provided in Table 3.

Table 3.

Summary for Moderator Analyses.

	Mixed effect analysis for moderating factors
	k	ES	SE	95% CI				Test of heterogeneity in effect sizes
	k	ES	SE	LL	UL	z	p	Q	df(Q)	p
Technology
1. Tutoring	8	0.803	0.209	0.394	1.212	3.85	0.00
2. Game-based	14	0.588	0.169	0.256	0.919	3.47	0.00
3. Problem solving	9	0.867	0.205	0.466	1.268	4.24	0.00
4. Practice	14	0.225	0.152	−0.0743	0.523	1.47	0.14
Total between								46.96	4	<.01
Grade level
1. Kindergarten	9	0.48	0.21	0.05	0.91	2.22	0.02
2. Primary	27	0.71	0.11	0.48	0.94	6.16	0.00
3. Junior high	6	−0.20	0.26	−0.72	0.31	−0.77	0.44
4. Senior high	2	0.59	0.38	−0.16	1.34	1.53	0.12
5. Other	1	0.70	0.56	−0.39	1.79	1.25	0.21
Total between								47.51	5	<.01
Duration
1. 0–11 weeks	28	0.60	0.12	0.37	0.84	5.07	0.00
2. 12–23 weeks	7	0.83	0.23	0.38	1.29	3.59	0.00
3. 24–35 weeks	3	0.54	0.43	−0.29	1.38	1.26	0.20
4. >36 weeks	6	0.08	0.23	−0.36	0.53	0.38	0.70
Total between								40.46	4	<.01
Intensity
1. Low (<60 min/wk)	15	0.55	0.17	0.21	0.89	3.23	0.001
2. Medium (60–150 min/wk)	16	0.65	0.16	0.33	0.97	3.99	0.00
3. High (>150 min/wk)	5	0.59	0.27	0.04	1.13	2.13	0.03
4. Other	9	0.37	0.21	−0.05	0.80	1.72	0.08
Total between								33.94	4	<.01
Components
1. Problem solving	8	0.95	0.22	0.52	1.39	4.27	0.00
2. Math reasoning	2	1.18	0.60	0.003`	2.35	1.96	0.06
3. Arithmetic skill	24	0.58	0.12	0.33	0.82	4.59	0.00
4. Multiple	11	0.20	0.17	−0.14	0.55	1.16	0.24
Total between								44.63	4	<.01
Test
1. Teacher-developed	6	−0.01	0.25	−0.51	0.47	−0.07	0.94
2. Researcher-developed	25	0.68	0.12	0.44	0.92	5.61	0.00
3. Standardized	14	0.56	0.15	0.25	0.86	3.57	0.00
Total between								44.35	3	<.01

Type of CT

Under the mixed-effects model, results showing that Q statistics of 8.47 with 3 degrees of freedom was statistically significant, suggesting that intervention effects are found to be different depending on types of CT. The estimated intervention effects for three types of CT—tutoring, game-based technology, and problem-solving system—were found to be statistically significant ( $\bar{d}$ = 0.80, SE = 0.20, z = 3.84, p < .01 by tutoring, $\bar{d}$ = 0.58, SE = 0.17, z = 3.47, p < .01 by game-based technology, and $\bar{d}$ = 0.86, SE = 0.20, z = 4.23, p < .01 by problem-solving system). A post-hoc Tukey test showed that students significantly performed higher when problem-solving system ( $\bar{d}$ = 0.86, SE = 0.20) was given to LP students’ achievement than when computerized practice was provided ( $\bar{d}$ = 0.22, SE = 0.15).

Grade Levels

Results from the mixed-effects model indicate that the effect of CT usage on LP students’ mathematics achievement significantly differs depending on grade levels (Q (4) = 10.38 p < .05). Specifically, the estimated intervention effects were significant for kindergarteners ( $\bar{d}$ = 0.48, SE = 0.21, z = 2.22, p < .05) and students in primary education ( $\bar{d}$ = 0.71, SE = 0.11, z = 6.16, p < .01). The results of the Tukey test indicated that CT interventions were significantly more effective for primary students ( $\bar{d}$ = 0.71, SE = 0.11) than for junior high school students ( $\bar{d}$ = −0.20, SE = 0.26).

Duration of Intervention

Results from the mixed-effects model showed that the duration of intervention did not statistically moderate the effect of CT interventions on mathematics achievement (Q_model (3) = 5.76, p = .12).

Intensity of Duration

Results from the mixed-effects model suggested that the effect of CT did not significantly differ depending on the levels of intensity (Q_model (3) = 1.04, p = .79).

Intervention Components

Results from the mixed-effects model indicate that the effect of CT usage on LP students’ mathematics achievement significantly differs depending on intervention components (Q_model (3) = 8.18, p < .05). The estimated intervention effects were all found to be significant on problem solving ( $\bar{d}$ = 0.95, SE = 0.22, z = 4 .27, p < .01) and arithmetic skills ( $\bar{d}$ = 0.58, SE = 0. 12, z = 4.59, p < .01). Results from a post-hoc analysis suggest that a significant difference existed between problem solving and multiple skills (p = .04). The results showed that CT interventions for problem solving ( $\bar{d}$ = 0.95, SE = 0.22) has a larger impact on mathematics achievement than for multiple skills ( $\bar{d}$ = 0.20, SE = 0.17).

Types of Testing Instrument

The mixed-effects model results show that the types of test have a significant effect on LP students’ mathematics performances under the CT usage as an intervention (Q (2) = 6.30, p < .05). The estimated intervention effects were found to be significant for the researcher-developed tests ( $\bar{d}$ = 0.68, SE = 0.12, z = 5.61, p < .01) and for the standardized tests ( $\bar{d}$ = 0.56, ES = 0.15, z = 3.57, p < .01). The significant difference was found between the levels of researcher-developed tests and teacher-developed tests (p = .03). The results showed that CT intervention has a higher effect on students’ mathematics achievement when using researcher-developed tests ( $\bar{d}$ = 0.68, SE = 0.12) than teacher-developed tests ( $\bar{d}$ = −0.01, SE = 0.25).

Discussion

The overall findings of this review indicated that CT has a significantly positive and large effect ( $\bar{d}$ = 0.56) on the mathematics achievement for LP students. Compared to several recent meta-analyses that applied CT for general education students in K-12 classrooms (Byun & Joung, 2018; Chauhan, 2017; Higgins et al., 2019), we found a little higher effect for LP students. This direction displayed the high potential of leveraging CT for enhancing LP students’ mathematics achievement. This claim was also strongly supported by an earlier review (Niemiec et al., 1987), which found that LP students benefited more from the CT compared to high-performing students.

However, Steenbergen-Hu and Cooper (2013) discovered a reverse comparison that the effectiveness of CT is higher for improving general education students than for LP students. The opposite direction might be due to the use of a single CT intervention (i.e., intelligent tutoring systems) in their review.

Our overall effect size seems to be smaller when compared to reviews that both included general education students and students with special needs. For example, Li and Ma (2010) found a bigger overall effect size of 0.71 for improving K-12 students’ mathematics achievement. One possible reason for this difference may arise from sampling. In our study, we merely focused on LP students. However, Li and Ma (2010) included both general education and special needs students in their research, that included a wide range of populations, such as LP students, mental/physical/emotional disability students, and at-risk students.

Besides student types, the number of studies used for reviews had a great influence on the overall findings. Xin and Jitendra (1999) reported CT greatly impacts the mathematics learning of students with learning problems ( $\bar{d}$ = 1.80). In their review, they only included four CT studies that all had large positive effect sizes. In our study, more recent studies that involved the usage of CT interventions—even studies with negative effect sizes, such as studies from Tienken and Maher (2008), and Brown (2018)—were included, which might produce a robust and comprehensive evidence about the CT effects for LP students’ mathematics achievement.

The significant results from our review are also supported by some ideas coming from primary studies. For instance, Leh and Jitendra (2013) found that teachers were willing to apply CT in their classrooms as it was more efficient in providing immediate feedback and afforded them more time to address the needs of LP students. In addition, Outhwaite et al. (2017) claimed that the features of CT, such as immediate feedback and continuous assessment, are beneficial for LP students’ mathematics learning by tailoring task levels based on students’ abilities and allowing them to repeat tasks. Overall, with supportive arguments from primary studies, our results provide well-founded evidence about the effectiveness of CT on mathematics achievement for LP students by adding more recent studies focusing on one specific student type.

In addition to these overall findings from our review, we also look at the different impacts of CT on LP students’ mathematics achievement by using various study features: types of CT, grade levels, implementation time (duration and intensity of intervention), intervention components, and types of testing instrument, respectively.

First, among four types of CT, problem-solving systems had the largest effect on mathematics achievement compared to the tutoring program, game-based intervention, and computerized practice. Our results conflicted with an earlier review (Niemiec et al., 1987), which found drill and practice was the most effective in raising students’ mathematics scores among tutorial and problem solving. This should come as no surprise for at least two reasons. First, CT has updated a lot since the last century, which allows for more targeted and individualized instruction or support during the problem-solving process, including creating more realistic problem scenarios that are more related to students (Lantz-Andersson et al., 2009), the use of schematic diagrams and solution trees (Leh & Jitendra, 2013; Reusser, 1996), and providing explicit procedure instruction and metacognitive scaffolding (Leh & Jitendra, 2013). Second, a series of representation techniques or strategies made the learning activities more appealing to students, at the same time, maintaining an appropriate cognitive level for LP students.

Second, our findings indicated that CT interventions had significant effect for kindergarten and primary school students. However, we found that CT did not significantly influence on high school students. The multiple comparison also showed that CT interventions had a significant higher effect for primary students than for those from high schools; in other words, interventions with CT are especially beneficial to students in lower grades. The direction of CT influence is consistent with previous reviews that focused on general education students (Bangert-Drowns et al., 1985; Cheung & Slavin, 2013; Li & Ma, 2010; Niemiec et al., 1987), suggesting that CT intervention had a higher effect on elementary students than on secondary students. The possible reason for less effect on high school students seems to be their different learning needs, as Kulik et al. (1985, p. 71) argued: “high school and college students apparently have less need for the highly structured, highly reactive instruction provided in computers.”

Third, the previous views indicated that the intervention time regarding the duration and intensity were critical moderator factors that influences the CT effect. Specifically, the shorter invention had a larger effect than the longer one (Gersten et al., 2009, Kroesbergen & Van Luit, 2003; Kulik et al., 1985, Li & Ma, 2010) and the high intensity produced a bigger effect than the low intensity on students’ mathematics achievement (Cheung & Slavin, 2013). However, in our review we found that both the levels of two intervention time factors (duration and intensity) did not significantly differentiate the CT effects on LP students’ mathematics achievement. Those insignificant findings might suggest that what matters less for low performers’ mathematics learning are learning time compared to other aspects, such as learning contents or needs that certain types of CT could afford.

The fourth interesting and contradictory finding, compared to the review by Kroesbergen and Van Luit (2003), is that CT interventions on problem-solving seem to be more effective than on arithmetic skills or on multiple skills. There might be two factors related to this different finding. First, in their review, they included both LP students and students with severe learning disabilities; it is possible that learning problem solving by using CT is very challenging for LD students, while LP students without LD may benefit from technology-aided problem solving. Second, the high effect of intervening on problem solving stems from, as mentioned above, the affordance of recent technology development. The advance of CT could realize the design of a complex computer-assisted problem-solving system, which could incorporate real-world problem-solving scenarios that are more understandable for LP students (NASEM, 2018). Chang et al. (2006) claimed that this kind of system was beneficial for students’ problem solving in two aspects: Decreasing the cognitive loading by providing systematic guidance and feedback, and providing cognitive scaffolding by using a step-by-step approach. Our findings provide some suggestive evidence that using CT for problem-solving intervention is feasible and effective.

Fifth, in terms of test types and effectiveness of CT, researcher-developed tests revealed larger effect than standardized tests or teacher-developed tests. This finding is consistent with Li and Ma (2010) study that found non-standardized tests show greater effect sizes than do standardized tests. Standardized tests may have good psychometric properties but fail to align with the content being studied, while teacher-developed tests may have some issues of reliability and validity, and as a consequence, it may inflate the effects of CT on mathematics achievement.

Implication for Practice

Quite a few reviews have explored the effects of CT on the mathematics performance of general education students or students with special needs. However, they did not specifically focus on LP students. In the end, low-performing students seem to have become the “forgotten middle” in the literature. An increasing number of empirical studies have explored the effects of CT on LP students’ mathematics achievement, while these studies offer conflicting results at numerous levels. This meta-analysis integrated the findings from relevant studies and identified the moderating factors that could influence the effects of CT, which could provide useful information or guidance for instructional design and practices.

It is encouraging to find that type of the problem-solving system is the most useful in determining students’ mathematics learning. This finding represented an important strand in the mathematics research community—that is, CT can be used to design and create advanced and appealing problem-solving activities or learning experiences in the technology-supported environment for better satisfying the learning needs of diverse students. This review proved that CT—specifically, a computer-based problem-solving system—has been a very valuable tool to help students with LP or at-risk for academic failure in mathematics to improve their mathematics learning.

Another implication speaks to the higher effect of CT in promoting mathematics achievement for students from lower grades. It demonstrates that CT seems to work well for elementary students. Along with the development and design of technology by incorporating more adaptive and visual support and attractions that fit the learning styles of elementary students, the impact of CT on mathematics achievement for lower graders could even be more dramatic. However, whether CT is more attractive to younger children and hence, further influences their engagement in learning activities, is beyond the scope of this study. Future research needs to investigate the effects of CT on students’ attitudes toward mathematics.

Limitations and Future Research

There are a number of methodological limitations that need to be discussed for the current meta-analysis. First, locating LP students based on percentage or percentile was not consistent across all studies we collected due to the variation in the essential standards. For instance, some studies labeled students as a low performer when they were at or below the 25th percentile, while some studies used the 15th percentile as their cut-off scores. Though we did not exclude studies based on their percentile cut-off scores, these different standards may influence the effect sizes, especially in studies just focusing on one specific group of students. Future meta-analyses that are based on a common percentile may allow for more specific conclusions. Moreover, demographic information such as ethnicity and gender were available for the general sample population in most of the studies, but the information was not provided for the group of LP students. Since we are focusing on LP students, we could not use gender or ethnicity information in our study. We recommend that detailed demographic information should be reported separately in future studies to be used for further reviews. Finally, because we focused on LP students, we could not investigate specific mathematics outcomes. Future research might look at the effectiveness of CT on specific outcomes such as problem solving, mathematical reasoning, or arithmetic skills, etc. Instead, the results of specific mathematic aspects would be more beneficial for educators.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Hua Ran is a third year PhD student at the department of Teaching and Learning, University of Miami. Her research interest is related to low-performing students in mathematics, computer interventions, CGI (cognitively guided instruction) teaching practices, and problem-solving struggles.

Murat Kasli is a third-year PhD student at the department of Educational and Psychological Studies, University of Miami. His research interests include IRT, Test Fraud, and Response time patterns of students. He is currently working on developing a detecting cheating model that includes IRT and Lognormal Response Time models.

Walter G. Secada is a professor at the department of Teaching and Learning, University of Miami. His research interests include equity in education, mathematics education, bilingual education, school restructuring, professional development of teachers, student engagement, and reform. Secada was the editor of the Review of Research in Education published by the American Educational Research Association (AERA) and series editor for Changing the Faces of Mathematics published by the National Council of Teachers of Mathematics (NCTM).

ORCID iD

Hua Ran

References

References marked with an asterisk indicate studies included in the meta-analysis.

AECT Task Force on Definition and Terminology. (1977). Educational technology: Definition and glossary of terms. Association for Educational Communications and Technology.

*Afzal

Gondal

Fatima

(2014). The effect of computer based instructional technique for the learning of elementary level mathematics among high, average and low achievers. International Journal of Education and Development Using ICT, 10(4), 3–51. https://www.learntechlib.org/p/150712/

*Alexis

E. G.

(2015). State math and English assessments as predictors of high school students’ success in an online credit recovery program (Publication No. 3664494) [Doctoral dissertation, St. John’s University]. ProQuest Dissertations Publishing.

*Arnold

K. T.

(2013). A comparison of Georgia criterion referenced competency test math scores between at-risk fifth grade students receiving computer-based math instruction and at-risk students not receiving computer-based math instruction (Publication No. 3602284) [Doctoral dissertation, Liberty University]. ProQuest Dissertations Publishing.

Bangert-Drowns

R. L.

Kulik

J. A.

Kulik

C.-L C.

(1985). Effectiveness of computer-based education in secondary schools. Computers in Human Behavior, 1(1), 59–68.

*Baroody

A. J.

Eiland

M. D.

Purpura

D. J.

Reid

E. E.

(2013). Can computer-assisted discovery learning foster first graders’ fluency with the most basic addition combinations? American Educational Research Journal, 50(3), 533–573. https://doi.org/10.3102/0002831212473349

Bateman

(1992). Learning disabilities: The changing landscape. Journal of Learning Disabilities, 25(1), 29–36. https://doi.org/10.1177/002221949202500105

Bayraktar

(2001). A Meta-analysis of the effectiveness of computer-assisted instruction in science education. Journal of Research on Technology in Education, 34(2), 173–188. https://doi.org/10.1080/15391523.2001.10782344

10.

Blok

Oostdam

Otter

M. E.

Overmaat

(2002). Computer-assisted instruction in support of beginning reading instruction: A review. Review of Educational Research, 72(1), 101–130. https://doi.org/10.3102/00346543072001101

11.

Borenstein

Hedges

L. V.

Higgins

J. P.

Rothstein

H. R.

(2011). Introduction to meta-analysis. John Wiley & Sons.

12.

*Bos

(2007). The effect of the Texas instrument interactive instructional environment on the mathematical achievement of eleventh grade low achieving students. Journal of Educational Computing Research, 37(4), 351–368. https://doi.org/10.2190/EC.37.4.b

13.

*Brown

(2018). Using laptop technology to improve mathematical achievement rates: A quasi-experimental study. Journal of Computers in Mathematics and Science Teaching, 37(3), 217–238. https://www.learntechlib.org/primary/p/178474/

14.

*Burns

M. K.

Kanive

DeGrande

(2012). Effect of a computer-delivered math fact intervention as a supplemental intervention for math in third and fourth grades. Remedial and Special Education, 33(3), 184–191. https://doi.org/10.1177/0741932510381652

15.

Byun

Joung

(2018). Digital game‐based learning for K–12 mathematics education: A meta‐analysis. School Science and Mathematics, 118(3–4), 113–126. https://doi.org/10.1111/ssm.12271

16.

*Chang

K.-E.

Sung

Y.-T.

Lin

S.-F.

(2006). Computer-assisted learning for mathematical problem solving. Computers & Education, 46(2), 140–151. https://doi.org/10.1016/j.compedu.2004.08.002

17.

*Chappell

Arnold

Nunnery

Grant

(2015). An examination of an online tutoring program’s impact on low-achieving Middle school students’ mathematics achievement. Online Learning, 19(5), 37–53. https://files.eric.ed.gov/fulltext/EJ1085790.pdf

18.

Chauhan

(2017). A Meta-analysis of the impact of technology on learning effectiveness of elementary students. Computers & Education, 105, 14–30. https://doi.org/10.1016/j.compedu.2016.11.005

19.

Cheung

A. C. K.

Slavin

R. E.

(2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K–12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113. https://doi.org/10.1016/j.edurev.2013.01.001

20.

Christmann

E. P.

Badgett

J. L.

(2003). A meta-analytic comparison of the effects of computer-assisted instruction on elementary students’ academic achievement. Information Technology in Childhood Education Annual, 2003(1), 91–104. https://www.learntechlib.org/primary/p/18866/

21.

Cibulka

J. G.

Cooper

B. S.

(Eds.). (2017). Technology in school classrooms: How it can transform teaching and student learning today. Rowman & Littlefield.

22.

Cohen

(1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.

23.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.

24.

Coley

Cradler

Engel

P. K.

(1997). Computers and classrooms: The status of technology in U.S. schools. Educational Testing Service, Policy Information Center. https://files.eric.ed.gov/fulltext/ED412893.pdf

25.

Cook

L. L.

(2008). Increasing middle grades math achievement through effective teaching practices (Publication No. 3288765) [Doctoral dissertation, Walden University]. ProQuest Dissertations Publishing.

26.

Cooper, H., & Hedges, L. V. (Eds.). (1994). Handbook of research synthesis. Russell Sage Foundation.

27.

Core Team R. (2013). R: A language and environment for statistical computing. (Version 4.0.2). The R Foundation for Statistical Computing. http://www.R-project.org/

28.

Cotton

(1991). May). Computer-assisted instruction. School Improvement Research Series, Close-up #10. https://educationnorthwest.org/sites/default/files/Computer-AssistedInstruction.pdf

29.

Cullen

C. J.

Hertel

J. T.

Nickels

(2020). The roles of technology in mathematics education. The Educational Forum, 84(2), 166–178. https://doi.org/10.1080/00131725.2020.1698683

30.

Demir

Başol

(2014). Effectiveness of computer-assisted mathematics education (CAME) over academic achievement: A meta-analysis study. Educational Sciences: Theory and Practice, 14(5), 2026–2035.

31.

*Fede

J. L.

Pierce

M. E.

Matthews

W. J.

Wells

C. S.

(2013). The effects of a computer-assisted, schema-based instruction intervention on word problem-solving skills of low-performing fifth grade students. Journal of Special Education Technology, 28(1), 9–21. https://doi.org/10.1177/016264341302800102

32.

Fletcher-Flinn

C. M.

Gravatt

(1995). The efficacy of computer assisted instruction (CAI): A meta-analysis. Journal of Educational Computing Research, 12(3), 219–241. https://doi.org/10.2190/51D4-F6L3-JQHU-9M31

33.

*Fuchs

L. S.

Fuchs

Hamlet

C. L.

Powell

S. R.

Capizzi

A. M.

Seethaler

P. M.

(2006). The effects of computer-assisted instruction on number combination skill in at-risk first graders. Journal of Learning Disabilities, 39(5), 467–475. https://doi.org/10.1177/00222194060390050701

34.

Gersten

Chard

D. J.

Jayanthi

Baker

S. K.

Morphy

Flojo

(2009). Mathematics instruction for students with learning disabilities: A meta-analysis of instructional components. Review of Educational Research, 79(3), 1202–1242. https://doi.org/10.3102/0034654309334431

35.

Goldberg

Russell

Cook

(2003). The effect of computers on student writing: A meta-analysis of studies from 1992 to 2002. The Journal of Technology, Learning and Assessment, 2(1), 3–51.

36.

Gray

Thomas

Lewis

(2010). Teachers’ use of educational technology in U.S. public schools: 2009. (NCES 2010-040). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. https://files.eric.ed.gov/fulltext/ED509514.pdf

37.

*Harris

T. D.

(2011). An integrated learning system: Impact on at-risk students’ ninth grade TAKS mathematics achievement (Publication No. 539330) [Doctoral dissertation, Texas A&M University]. ProQuest Dissertations Publishing.

38.

Hartley

S. S.

(1977). Meta-analysis of the effects of individually paced instruction in mathematics [Unpublished doctoral dissertation]. University of Colorado.

39.

*Hassler Hallstedt

Klingberg

Ghaderi

(2018). Short and long-term effects of a mathematics tablet intervention for low performing second graders. Journal of Educational Psychology, 110(8), 1127–1148. https://doi.org/10.1037/edu0000264

40.

Higgins

J. P.

Thompson

S. G.

(2002). Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186

41.

Higgins

Huscroft-D’Angelo

Crawford

(2019). Effects of technology in mathematics on achievement, motivation, and attitude: A meta-analysis. Journal of Educational Computing Research, 57(2), 283–319. https://doi.org/10.1177/0735633117748416

42.

Hobbs

R. M.

(2012). Improving problem-solving techniques for students in low-performing schools (Publication No. 3493910) [Doctoral dissertation, Temple University]. ProQuest Dissertations Publishing.

43.

Hsu

Y.-C.

(2003). The effectiveness of computer-assisted instruction in statistics education: A meta-analysis [Doctoral Dissertation, University of Arizona]. Dissertation Abstracts International Section A: Humanities and Social Sciences, 64(5-A), 1526.

44.

*Huang

T.-H.

Liu

Y.-C.

Chang

H.-C.

(2012). Learning achievement in solving word-based mathematical questions through a computer-assisted learning system. Journal of Educational Technology & Society, 15(1), 248–259. https://www-jstor-org-443.web.bisu.edu.cn/stable/jeductechsoci.15.1.248

45.

Hwang

W.-Y.

Chen

N.-S.

Hsu

R.-L.

(2006). Development and evaluation of multimedia whiteboard system for improving mathematical problem solving. Computers & Education, 46(2), 105–121. https://doi.org/10.1016/j.compedu.2004.05.005

46.

Jitendra

A. K.

Lein

A. E.

S.-H.

Alghamdi

A. A.

Hefte

S. B.

Mouanoutoua

(2018). Mathematical interventions for secondary students with learning disabilities and mathematics difficulties: A meta-analysis. Exceptional Children, 84(2), 177–196. https://doi.org/10.1177/0014402917737467

47.

*Kanive

Nelson

P. M.

Burns

M. K.

Ysseldyke

(2014). Comparison of the effects of computer-based practice and conceptual understanding interventions on mathematics fact retention and generalization. The Journal of Educational Research, 107(2), 83–89. https://doi.org/10.1080/00220671.2012.759405

48.

*Karner

M. L.

(2016). An analysis of the effectiveness of ALEKS within a high school mathematics setting (Publication No. 10194365) [Doctoral dissertation, Aurora University]. ProQuest Dissertations Publishing.

49.

Kroesbergen

E. H.

Van Luit

J. E. H.

(2003). Mathematics interventions for children with special educational needs: A meta-analysis. Remedial and Special Education, 24(2), 97–114. https://doi.org/10.1177/07419325030240020501

50.

Kulik

C.-L C.

Kulik

J. A.

(1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behavior, 7(1–2), 75–94. https://doi.org/10.1016/0747-5632(91)90030-5

51.

Kulik

J. A.

(1994). Meta-analytic studies of findings on computer-based instruction. In E. L. Baker & H. F. O’Neil (Eds.), Technology assessment in education and training (pp. 9–34). Lawrence Erlbaum Associates.

52.

Kulik

J. A.

Kulik

C.-L C.

Bangert-Drowns

R. L.

(1985). Effectiveness of computer-based education in elementary schools. Computers in Human Behavior, 1(1), 59–74. https://doi.org/10.1016/0747-5632(85)90007-X

53.

Kulik

J. A.

Kulik

C.-L C.

Cohen

P. A.

(1980). Effectiveness of computer-based college teaching: A meta-analysis of findings. Review of Educational Research, 50(4), 525–544. https://doi.org/10.3102/00346543050004525

54.

Lantz-Andersson

Linderoth

Säljö

(2009). What’s the problem? Meaning making and learning to do mathematical word problems in the context of digital tools. Instructional Science, 37(4), 325–343. https://dx-doi-org.web.bisu.edu.cn/10.1007/s11251-008-9050-0

55.

Larwin

(2011). A meta-analysis examining the impact of computer-assisted instruction on postsecondary statistics education: 40 years of research. Journal of Research on Technology in Education, 43(3), 253–278. https://doi.org/10.1080/15391523.2011.10782572

56.

*Leh

(2011). Mathematics word problem solving: An investigation into schema-based instruction in a computer-mediated setting and a teacher-mediated setting with mathematically low-performing students (Publication No. 3439742) [Doctoral dissertation, Lehigh University]. ProQuest Dissertations Publishing.

57.

*Leh

J. M.

Jitendra

A. K.

(2013). Effects of computer-mediated versus teacher-mediated instruction on the mathematical word problem-solving performance of third-grade students with mathematical difficulties. Learning Disability Quarterly, 36(2), 68–79. https://doi.org/10.1177/0731948712461447

58.

(2010). A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review, 22(3), 215–243. https://dx-doi-org.web.bisu.edu.cn/10.1007/s10648-010-9125-8

59.

Liao

Y.-K C.

(2007). Effects of computer-assisted instruction on students’ achievement in Taiwan: A meta-analysis. Computers & Education, 48(2), 216–233. https://doi.org/10.1016/j.compedu.2004.12.005

60.

Lou

Abrami

P. C.

d’Apollonia

(2001). Small group and individual learning with technology: A meta-analysis. Review of Educational Research, 71(3), 449–521. https://doi.org/10.3102/00346543071003449

61.

*Luft

C. D. B.

Gomes

J. S.

Priori

Takase

(2013). Using online cognitive tasks to predict mathematics low school achievement. Computers & Education, 67, 219–228. https://doi.org/10.1016/j.compedu.2013.04.001

62.

Martinez

A. R.

(2017). The effects of using GeoGebra on student achievement in secondary mathematics [Master’s thesis, California State University]. https://digitalcommons.csumb.edu/caps_thes_all/137

63.

Means, B. (Ed.). (1994). The technology and education reform: The reality behind the promise. Jossey-Bass.

64.

Moher

Liberati

Tetzlaff

Altman

D. G.

; PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097

65.

*Moyer-Packenham

Suh

(2012). Learning mathematics with technology: The influence of virtual manipulatives on different achievement groups. Journal of Computers in Mathematics and Science Teaching, 31(1), 39–59. https://www.learntechlib.org/primary/p/37643/

66.

National Academies of Sciences, Engineering, and Medicine [NASEM]. (2018). How people learn II: Learners, contexts, and cultures. National Academies Press.

67.

National Center for Education Statistics. (2005). The nation’s report card: Mathematics 2005 (NCES 2006–453). U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/nationsreportcard/pdf/main2005/2006453.pdf

68.

National Center for Education Statistics. (2011). The nation’s report card: Mathematics 2011 (NCES 2012-458). U.S. Department of Education, Institute of Education Sciences. https://nces.ed.gov/nationsreportcard/pdf/main2011/2012458.pdf

69.

National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. http://gen.lib.rus.ec/book/index.php?md5=D8E144616E131F23FF015291AE77D10B

70.

National Governors Association Center for Best Practices & Council of Chief State School Offices. (2010). Common core state standards for mathematics. National Governors Association Center for Best Practices, Council of Chief State School Officers.

71.

National Research Council (2001). Adding it up: Helping children learn mathematics. National Academies Press. https://doi.org/10.17226/9822

72.

Niemiec

Samson

Weinstein

Walberg

H. J.

(1987). The effects of computer-based instruction in elementary schools: A quantitative synthesis. Journal of Research on Computing in Education, 20(2), 85–103. https://doi.org/10.1080/08886504.1987.10781826

73.

Ohnemus, E. (Ed.). (2002). No child left behind: A desktop reference. U.S. Department of Education, Office of the Under Secretary. https://files.eric.ed.gov/fulltext/ED471334.pdf

74.

M. W.

Bryant

D. P.

(2016). Effects of a strategic intervention with iPad practice on the multiplication fact performance of fifth-grade students with learning disabilities. Learning Disability Quarterly, 39(3), 146–158. https://doi.org/10.1177/0731948715598285

75.

*Outhwaite

L. A.

Gulliford

Pitchford

N. J.

(2017). Closing the gap: Efficacy of a tablet intervention to support the development of early mathematical skills in UK primary school children. Computers & Education, 108, 43–58. https://doi.org/10.1016/j.compedu.2017.01.011

76.

*Pellerito

F. L.

(2011). The effects of technology enriched mathematics instruction on at-risk secondary school students [Doctoral dissertation, University of Kansas]. https://kuscholarworks.ku.edu/bitstream/handle/1808/7934/Pellerito_ku_0099D_11416_DATA_1.pdf

77.

Perkins

Gilman

D. A.

(2002). Discourse technology: An evaluation of math technology and proficiency scores for Worthington City schools. Office of Educational Research and Improvement, U.S. Department of Education. https://files.eric.ed.gov/fulltext/ED475145.pdf

78.

Rakes

C. R.

Valentine

J. C.

McGatha

M. B.

Ronau

R. N.

(2010). Methods of instructional improvement in algebra: A systematic review and meta-analysis. Review of Educational Research, 80(3), 372–400. https://doi.org/10.3102/0034654310374880

79.

*Räsänen

Salminen

Wilson

A. J.

Aunio

Dehaene

(2009). Computer-assisted intervention for children with low numeracy skills. Cognitive Development, 24(4), 450–472. https://doi.org/10.1016/j.cogdev.2009.09.003

80.

Reusser

(1996). From cognitive modeling to the design of pedagogical tools. In S. Vosnadiou, E. De Corte, R. Glaser, & H. Mandl (Eds.), International perspectives on the design of technology supported learning environments (pp. 81–104). Lawrence Erlbaum Associates.

81.

*Reynolds

J. L.

(2010). The effects of computerized instruction and systematic presentation and review of math fact acquisition and fluency (Publication No. 3440726) [Doctoral dissertation, Central Michigan University]. ProQuest Dissertations Publishing.

82.

Robotti

Baccaglini-Frank

(2017). Using digital environments to address students’ mathematical learning difficulties. In E. Faggiano, F. Ferrara, & A. Montone (Eds.), Innovation and technology enhancing mathematics education: Perspectives in the digital era (pp. 77–106). Springer.

83.

Rosenthal

(1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 232–243). Russell Sage Foundation.

84.

Rutherford

Farkas

Duncan

Burchinal

Kibrick

Graham

Richland

Tran

Schneider

Duran

Martinez

M. E.

(2014). A randomized trial of an elementary school mathematics software intervention: Spatial-temporal math. Journal of Research on Educational Effectiveness, 7(4), 358–383. https://doi.org/10.1080/19345747.2013.856978

85.

*Salminen

(2015). Response to computer-assisted intervention in children most at risk for mathematics difficulties [Academic Dissertation, University of Jyväskylä]. Jyväskylä Studies in Education, Psychology and Social Research. https://jyx.jyu.fi/bitstream/handle/123456789/48010/1/978-951-39-6432-0_vaitos_20151210.pdf

86.

*Salminen

Koponen

Räsänen

Aro

(2015a). Preventive support for kindergarteners most at-risk for mathematics difficulties: Computer-assisted intervention. Mathematical Thinking and Learning, 17(4), 273–295. https://doi.org/10.1080/10986065.2015.1083837

87.

*Salminen

J. B.

Koponen

T. K.

Leskinen

Poikkeus

A. M.

Aro

M. T.

(2015b). Individual variance in responsiveness to early computerized mathematics intervention. Learning and Individual Differences, 43, 124–131). https://doi.org/10.1016/j.lindif.2015.09.002

88.

Schenker

J. D.

(2007). The effectiveness of technology use in statistics instruction in higher education: A meta-analysis using hierarchical linear modeling (Document No. kent1194979182) [Doctoral dissertation, Kent State University]. http://rave.ohiolink.edu/etdc/view?acc_num=kent1194979182

89.

*Shin

Sutherland

L. M.

Norris

C. A.

Soloway

(2012). Effects of game technology on elementary student learning in mathematics. British Journal of Educational Technology, 43(4), 540–560. https://doi.org/10.1111/j.1467-8535.2011.01197.x

90.

Sivin-Kachala

Bialo

E. R.

(1994). Report on the effectiveness of technology in schools, 1990–1994. Software Publishers Association. https://files.eric.ed.gov/fulltext/ED371726.pdf

91.

Slavin

R. E.

Lake

(2008). Effective programs in elementary mathematics: A best evidence synthesis. Review of Educational Research, 78(3), 427–455. https://doi.org/10.3102/0034654308317473

92.

Slavin

R. E.

Lake

Groff

(2009). Effective programs in middle and high school mathematics: A best evidence synthesis. Review of Educational Research, 79(2), 839–911. https://doi.org/10.3102/0034654308330968

93.

Sosa

G. W.

Berger

D. E.

Saw

A. T.

Mary

J. C.

(2011). Effectiveness of computer-assisted instruction in statistics: A meta-analysis. Review of Educational Research, 81(1), 97–128. https://doi.org/10.3102/0034654310378174

94.

Steenbergen-Hu

Cooper

(2013). A Meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students’ mathematical learning. Journal of Educational Psychology, 105(4), 970–987. https://psycnet.apa.org/doi/10.1037/a0032447

95.

Swan

Guerrero

Mitrani

Schoener

(1990). Honing in on the target: Who among the educationally disadvantaged benefits most from what CBI? Journal of Research on Computing in Education, 22(4), 381–403. https://doi.org/10.1080/08886504.1990.10781929

96.

*Tienken

C. H.

Maher

J. A.

(2008). The influence of computer-assisted instruction on eighth grade mathematics achievement. RMLE Online, 32(3), 1–13. https://doi.org/10.1080/19404476.2008.11462056

97.

Traynor

P. L.

(2003). Effects of computer-assisted-instruction on different learners. Journal of Instructional Psychology, 30(2), 137–143.

98.

*Tsuei

(2012). Using synchronous peer tutoring system to promote elementary students’ learning in mathematics. Computers & Education, 58(4), 1171–1182. https://doi.org/10.1016/j.compedu.2011.11.025

99.

Tucker

S. I.

(2018). Applying the modification of attributes, affordances, abilities, and distance for learning framework to a child’s multi-touch interactions with an idealized number line. In L. Ball, P. Drijvers, S. Ladel, H.-S. Siller, M. Tabach, & C. Vale (Eds.), Uses of technology in primary and secondary mathematics education (pp. 35–57). Springer.

100.

Veroniki

A. A.

Jackson

Viechtbauer

Bender

Bowden

Knapp

Kuss

Higgins

J. P. T.

Langan

Salanti

(2016). Methods to estimate the between‐study variance and its uncertainty in meta‐analysis. Research Synthesis Methods, 7(1), 55–79. https://doi.org/10.1002/jrsm.1164

101.

Viechtbauer

(2010). Conducting Meta-analyses in R with the Metafor package. Journal of Statistical Software, 36(3), 1–48. https://dx-doi-org.web.bisu.edu.cn/10.18637/jss.v036.i03

102.

Wilson

A. J.

Dehaene

Dubois

Fayol

(2009). Effects of an adaptive game intervention on accessing number sense in low‐socioeconomic‐status kindergarten children. Mind, Brain, and Education, 3(4), 224–234. https://doi.org/10.1111/j.1751-228X.2009.01075.x

103.

Xin

Y. P.

Jitendra

A. K.

(1999). The effects of instruction in solving mathematical word problems for students with learning problems: A meta-analysis. The Journal of Special Education, 32(4), 207–225. https://doi.org/10.1177/002246699903200402

104.

*Ysseldyke

Spicuzza

Kosciolek

Teelucksingh

Boys

Lemkuil

(2003). Using a curriculum-based instructional management system to enhance math achievement in urban schools. Journal of Education for Students Placed at Risk, 8(2), 247–265. https://doi.org/10.1207/S15327671ESPR0802_4

105.

Zunker

L. J.

(2008). Computer-based instruction and mathematics skills of elementary students with learning disabilities (Publication No. 3332683) [Doctoral dissertation, Texas A&M University-Kingsville]. ProQuest Dissertations Publishing.