Effects of Robot-Assisted Language Learning on English-as-a-Foreign-Language Skill Development

Abstract

While numerous studies of robot-assisted language learning (RALL) for English-as-a-foreign-language (EFL) learners’ language skill development have been done, a comprehensive and theoretically-driven meta-analysis on its effects is still in paucity. To fill the gap, drawing on Activity Theory (AT), this study reported a meta-analysis from 47 independent studies out of 29 literature samples involving 1791 EFL learners on RALL for language skill development published during 2004–2023. The results indicated that the overall effect size was g = .69, 95% CI [.49, .90], suggesting that RALL outperforms non-RALL conditions. In addition, educational levels and intervention durations were found to be significant moderators. Based on the results, implications for practice were discussed.

Keywords

activity theory English-as-a-foreign-language evidence-based applied linguistics meta-analysis robot-assisted language learning

Introduction

English-as-a-foreign-language (EFL) learners’ language skill development is defined as the progress in English language ability for listening, speaking, reading, and writing skills, which is crucial to improving their communications and future careers in an increasingly globalized society (El Shazly, 2021). However, EFL learners often perceived language learning to be difficult due to the lack of sufficient exposure to authentic language contexts (Tai & Chen, 2022), the lack of EFL learning motivation (Lee et al., 2011; Meng & Li, 2023; Tsai, 2019), and the mismatch between teaching strategies and learning style (Huang, 2005). To solve these problems, researchers (e.g., Alemi et al., 2015; Banaeian & Gilanlioglu, 2021; Engwall & Lopes, 2022; Jeon, 2023) have adopted robot-assisted language learning (RALL) tools to facilitate language learning and teaching, given that social robots can imitate normal human-to-human conversations (El Shazly, 2021), play various social roles (Hsu & Liang, 2021), teach learners according to their aptitude (Belpaeme et al., 2013), and provide personalized feedback and authentic interactions (Li et al., 2021; Wang et al., 2013).

By definition, RALL refers to “the use of social robots to teach people language expression or comprehension skills” (Randall, 2020, p. 2). The pedagogical affordances of RALL have been well-documented, including facilitating domain-specific language skill development, e.g., vocabulary (Banaeian & Gilanlioglu, 2021), speaking (El Shazly, 2021; Iio et al., 2019), listening (Dizon, 2020; Hsiao et al., 2015) and reading (Kim, 2018; Tai & Chen, 2022), along with domain-general EFL development (Hsu & Liang, 2021; Lee et al., 2011; Wu et al., 2015). While previous studies (e.g., Lee et al., 2011; Park et al., 2011) empirically explored the use of RALL for EFL skill development, and most of them obtained the positive effects of RALL on language acquisition (Kory Westlund et al., 2017; Mubin et al., 2013; Vogt et al., 2019), there is still a lack of research that provides comprehensive, synthesized evidence of its overall effects for EFL learning, let alone from a more theoretical perspective. Because a holistic theoretically-driven understanding of RALL’s overall effects can not only offer new insights into its effectiveness, but also scrutinize the influence of potential moderators that informs pedagogy. To narrow the gap, this study aims to (a) meta-analyze the results based on an exhaustive retrieval of the previous RALL research, and (b) gain a deeper understanding of how potential moderators have an impact on the effects of RALL under the framework of Activity Theory (AT, Engeström, 2001), since such AT-related factors as subjects, objectives, community, rules, division of labor, and tools are crucial in RALL (Banaeian & Gilanlioglu, 2021; Tai & Chen, 2022).

Related Studies on Robot-Assisted Language Learning for English-as-a-Foreign-Language Skill Development

Currently, given the important facilitative pedagogical affordances of RALL, researchers have adopted the quasi-experimental design to explore the effects of RALL on language skill development along two academic strands of inquiry–the pedagogical effects as compared with traditional methods (e.g., ordinary paper-and-pencil) and other educational technologies (e.g., noncomputer-based media or web-based instruction).

One strand of the related studies has examined the effects of RALL on EFL learners’ language performance compared with traditional learning methods (Dizon, 2020; Hsiao et al., 2015; Hsieh et al., 2023), but consensus has not been reached thus far. For instance, Hong et al. (2016) conducted a quasi-experiment to investigate EFL learners’ language performance in a RALL setting, and found that learners of the experimental group using social robots had lower anxiety levels, better self-esteem, and stronger learning motivation than their counterparts in the control group. Similarly, Hsieh et al. (2023) explored how the use of social robots would promote EFL learners’ speaking skills, and posited that social robots provide more contextualized and personalized feedback than the traditional repeat-after-me pronunciation instruction. Despite the promising results, not everyone could be convinced (e.g., Banaeian & Gilanlioglu, 2021; Kanda et al., 2004; Kanero et al., 2022). For instance, Banaeian and Gilanlioglu (2021) compared the effect of RALL with traditional learning methods on college EFL learners’ vocabulary learning, and the results indicated that learners using traditional methods outperformed their counterparts who used RALL for EFL vocabulary learning. To elucidate, they attributed the discrepancies to such moderators as participants’ educational levels, the role of the robot and intervention duration. Kanero et al. (2022) examined the effect of RALL on learners’ language performance and individual difference variables (viz. attitude, anxiety, and personality). In doing so, they randomly assigned EFL learners into groups of NAO robot tutor and human tutor, and the results revealed that learners with negative attitudes and higher anxiety towards robots would learn fewer words than those with more positive attitudes and lower anxiety.

The other strand has compared the effects of RALL on language skill development with other technologies, e.g., web-based instruction (Han et al., 2005, 2008) and tablet-PC (Hsiao et al., 2015; Konijn et al., 2021). For instance, Han et al. (2008) compared the effects of web-based instruction and RALL on EFL learners’ language skills, and identified the more positive effects of social robots than computers for EFL learning, as social robots can also offer various forms of expression, such as motions, gestures, and facial expressions. Liang and Hwang (2023) compared the effect of web-based instruction and RALL on language learning outcomes, and found that RALL outperformed the web-based instruction, because it could provide EFL learners with rich multimodal materials, more contextual interactions and embodied learning experiences. Hsiao et al. (2015) conducted a quasi-experiment to compare the effectiveness of table-PC and RALL on EFL learners’ language skills. Results also reported that RALL was a more effective learning companion than tablet-PC, since learning materials installed on highly interactive RALL tool could significantly promote learner motivation. In another quasi-experiment, Konijn et al. (2021) observed that children who were trained with a social robot could learn more new words than a tablet, because they could perceive more social robot’s humanness than the tablet and thus were more engaged in the learning task.

Related Reviews of Robot-Assisted Language Learning for English-as-a-Foreign-Language Skill Development

Aside from the aforementioned empirical attempts, researchers (Uslu et al., 2022; Randall, 2020; van den Berghe et al., 2019) have to date adopted an evidence-based applied linguistics (EBAL) account that “pedagogical intervention should be supported with sound evidence available” (Li, 2023a, p. 36), and begun to synthesize the current trends of RALL for learners’ language skill development. For instance, van den Berghe et al. (2019) reviewed 33 RALL studies and discussed the possibilities and limitations of using social robots to improve language skills (e.g., vocabulary, reading, and speaking). Mixed results showed that social robots could offer interaction possibilities in a real-life environment, but some other issues remained to be addressed. That is, more research is needed to determine the most effective type of robot role (e.g., tutor or teaching assistant), the age groups for which social robots are most beneficial (e.g., preschool children or college students), and the optimal intervention durations. Likewise, Randall (2020) conducted a qualitative study to examine RALL’s effectiveness by reviewing previous RALL studies published between 2004 and 2017, and claimed that social robots can facilitate foreign language acquisition. Nevertheless, this review seems to raise more questions than it answers because the qualitative review might fail to systematically analyze how moderators (e.g., different EFL language skills, control conditions, and types of robot form) affect the effectiveness of RALL.

Although the aforementioned qualitative reviews may contribute to the understanding of the current status and the potential moderators regarding the effectiveness of RALL, such as age groups, robot roles, interaction types, and intervention durations, they might fail to provide a comprehensive quantitative analysis of RALL’s pedagogical effects, and it remains unclear how those moderators affect RALL’s effectiveness. To our knowledge, only Lee and Lee (2022) have begun to meta-analyze the previous experimental studies on RALL with a limited sample k = 12. While this meta-analysis may shed some light on the RALL’s effectiveness, we argue that this paper goes beyond the aforementioned reviews or meta-analysis in the following aspects: First, Lee and Lee’s (2022) meta-analysis only included 12 independent samples. A scrutiny of those 12 selected studies showed that they were sporadically distributed in each year and the latest were published in 2019 (Tsai, 2919; Vogt et al., 2019), which may result in the limited generalizability. In contrast, given the rapid development of generative artificial intelligence (AI)-empowered (e.g., ChatGPT) social robots in the recent three years (2020–2023), our paper would provide updated and timely synthesis on RALL research published from the time span of 2004–2023 that can inform scholarly activity on the state-of-the-art of social robots in language education. Second, some potential moderators had to be overlooked due to the small sample sizes, such as intervention duration and specific language domains. Third and importantly, while Lee and Lee (2022) had included limited moderators (e.g., roles of robots, interaction type, and control conditions), this meta-analysis had not yet been grounded with a solid theoretical framework. Existing studies (Banaeian & Gilanlioglu, 2021; Hsu & Liang, 2021; Tai, 2022; Tai & Chen, 2022) demonstrated that such factors as subjects, objectives, community, rules, division of labor, and tools play a crucial role in RALL, necessitating an Activity Theory- (AT-)driven meta-analysis having those dimensions with larger sample sizes for further investigation.

Current Study

Based on the existing gaps, this meta-analysis that includes more studies aims to examine the overall effects of RALL on EFL learning while addressing some of the complexities regarding the outcomes of RALL. Moreover, compared with previous relevant studies that were not supported by a well-grounded theoretical framework, this study adopted AT as a framework to provide a more systematic analysis of some key moderators of RALL.

AT originally referred to desired outcomes mediated by psychological tools, including subjects, objects, and mediating artifact (Vygotsky, 1978). Later, considering the interrelationships between the subjects and their community, the community was added (Leont’ev, 1981). Furthermore, to describe the interactivity between two or more activity systems, Engeström (1987) proposed that the interaction process includes not only the subjects, objects and tool, but also rules, community and division of labor. The rules refer to norms, guidelines, and social relations within a community. Additionally, the division of labor involves the task distribution among authhcommunity members (Engeström, 2001).

Considering that the advantage of AT is to bridge the gap between the subjects and the community through the mediating activity, it serves as a working framework for the general human-technology interaction (Lin et al., 2019) and the specific human-robot interaction (Tlili, et al., 2020). Tlili et al. (2020) adopted AT to conduct the content analysis of robot-assisted special education, which would shed some theoretical light on this study. In this study (as shown in Figure 1), subjects refer to participants with different educational levels in the RALL research (Li, 2022a). Tool includes type of robot form due to its important impact on the effectiveness of RALL (Randall, 2020). Objects consist of specific language skills, e.g., listening, speaking, writing, and reading, etc. (Li, 2022b, 2024). Labor division refers to the distribution of duties among teachers and robots, including the types of teacher role and the types of robot role because various social roles of teachers or robots might influence the process of RALL (Alemi et al., 2015). Community is defined as the interactions between students and social robots in a community, involving in groups and one-one-one types (Uslu et al., 2022). Rules refer to principles in implementing RALL interventions, including intervention durations and control conditions.

Figure 1.

The proposed framework of Activity Theory for RALL.

To reiterate, this study aims to comprehensively meta-analyze the effects of RALL for EFL learners’ language skill development. Specifically, two research purposes are to be achieved. First, it aggregates the overall effect sizes of RALL for EFL learning performance. Second, drawing on the theoretical underpinnings of AT, it reports the moderating effects of educational levels, types of robot form, types of robot role, language skills, types of teacher role, interaction types, intervention durations, and control conditions on the overall effect. Consequently, two questions are to be addressed as follows:

Research question 1

What is the overall effect size of RALL for EFL skill development?

Research question 2

How do AT-related moderators, such as educational levels, types of robot form, types of robot role, language skills, types of teacher role, interaction types, intervention durations, and control conditions, affect the aggregated effect size?

Methodology

Data Collection

Motivated by the preferred reporting items for systematic reviews and meta-analyses (PRISMA) (Moher et al., 2009), the following retrieval procedures were strictly observed. First, drawing on the insights of the recently published reviews (Uslu et al., 2022; Randall, 2020; Tlili et al., 2020; van den Berghe et al., 2019), a combination of the following robot-related and language learning-related keywords integrated with Boolean operators was performed, i.e., (robot-assisted language learning OR RALL OR robot applications OR education robots OR social robots OR chatbot) AND (EFL OR English OR language OR language performance OR language achievement OR reading OR writing OR listening OR speaking OR vocabulary OR education OR language skill OR language ability). Second, the systematic literature retrieval was conducted via several online databases (e.g., Web of Science, ScienceDirect, Springer, ProQuest, Wiley, ERIC, Scopus), search engines (Google Scholar and Baidu Scholar), and related journals on EduTech (e.g., Journal of Educational Computing Research, Journal of Computer Assisted Learning, Educational technology and Society, Interactive Learning Environments, Computers and Education, British Journal of Educational Technology, Computer-Assisted Language Learning, ReCALL, Language Learning and Technology, System, Computational Linguistics, Learning Media and Technology, Education and Information Technologies, Journal of Computing in Higher Education, Technology Pedagogy and Education, Mobile Media and Communication, International Journal of Social Robotics, International Journal of Humanoid Robotics, IEEE Transactions on Learning Technologies) and language education (e.g., Language Teaching, Language and Education, Linguistics and Education, Journal of Language Identity and Education, Foreign Language Annals, Reading and Writing, Journal of Research in English). Third, to avoid the insufficient search of a significant portion of the relevant literature, backward and forward citation search based on the related reviews (e.g., Uslu et al., 2022; Mubin et al., 2013; van den Berghe et al., 2019) was used. The inclusion and/or exclusion criteria were diagramed in Figure 2 as follows:

(1) Thirty-four studies were excluded for the following reasons: Thirty studies are inaccessible to acquire full tests, and four studies are conference abstracts.

(2) Studies included should be (quasi-) experimental that examined the effectiveness of RALL on EFL skill development. These studies (k = 16) that used social robots in other disciplines (e.g., mathematics, technology, native language) were excluded. For instance, the study (Mubin et al., 2012) that aimed to evaluate robot artificial language was excluded.

(3) Studies should report experimental results of language skills measured by standardized examinations or researcher-designed tests. Those publications (k = 24) that investigated learners’ perceptions and pedagogical or theoretical recommendations were excluded. For instance, a qualitative study (Jeon, 2022) that explored young learners’ perspective of AI chatbot affordances in the EFL classroom was excluded.

(4) Studies should have sufficient data to calculate effect sizes, such as mean, standard deviations (SDs), sample sizes, t value, or F value. The studies (k = 10) without sufficient data for effect size calculation were excluded. For instance, Lin et al. (2022) examined the impact of educational robots on enhancing EFL vocabulary learning. However, they only reported total test scores rather than mean or SDs. Therefore, this study was excluded.

Figure 2.

Flow diagram for the search and inclusion of studies.

Coding Scheme

A coding scheme was developed to identify potential moderators of RALL from the included studies. First, drawing on AT, seven moderators were coded, including educational levels, intervention durations, types of robot form, types of robot role, types of teacher role, interaction types, and control conditions. Second, multiple effect sizes were coded separately when a single publication involves different intervention durations or different types of measurement (Plonsky & Oswald, 2014). For example, as Vogt et al. (2019) conducted two quasi-experiments that compared the effects of RALL with traditional methods and other technology on EFL vocabulary learning outcomes, its effect sizes were coded separately. Third, two trained raters selected 10 included studies to reach a common understanding of coding options and independently coded all included studies. All articles were double coded for inter-rater reliability and the interrater reliability (percentage of agreement) was 95%. The coding characteristics for the study samples are shown in Table 1.

Table 1.

The Descriptive Information of Coding Scheme.

Dimensions	Moderators	Subtypes	Operational definitions	References
Subjects	Educational levels	1. Primary	Preschool or primary school students	Li (2022a)
		2. Secondary	Junior high school or senior high school students	Li (2022a)
		3. Tertiary	College students or adults
Objects	Language skills	1. Listening	Listening development	Li (2022b)
	Language skills	2. Speaking	Speaking skill	Li (2022b)
		3. Reading	Reading skill
		4. Writing	Writing skill
		5. Vocabulary	Vocabulary skill
		6. Language in general	General language skill
Rules	Intervention durations	1. Short	≤1 week	Randall (2020)
	Intervention durations	2. Intermediate	>1 week, ≤4 weeks
		3. Long	>4 weeks, ≤one semester
	Control conditions	Other technologyTraditional	Participants in the control group using other technology (e.g., computer or tablet-PC)Participants in the control group using traditional methods (e.g., a paper-and-pencil method)	Li (2023a)
Community	Interaction types	1. In groups	Students interacting with social robots in groups	Uslu et al. (2022)
		2. One-on-one	Students interacting with social robots one-on-one
Labor division	Types of teacher role	1. No teachers	No teachers available in RALL	Engwall and Lopes (2022)
		2. Assistant	Teachers assisting social robots to teach a language
		3. Tutor	Teacher-led RALL lessons
	Types of robot role	1. Assistant	Social robots acting as teacher assistants	Randall (2020)
		2. Peer	Social robots acting as student peers
		3. Tutor	Robot-led RALL lessons
Tool	Types of robot form	1. Anthropomorphic	Social robots having a human-like torso or facial features (e.g., NAO)	Randall (2020)
		2. Zoomorphic	Social robots having animal-like or caricature-like features (e.g., iCat)
		3. Mechanomorphic	Social robots having machine-like qualities (e.g., chatbots)

Effect Size Calculation

Hedges’ g was used to calculate effect sizes due to small sample size cases included in the present meta-analysis (Lipsey & Wilson, 2001). The equation for calculation is formulated as follows

Hedges ’ g = J_{correction factor} \times \frac{{Mean}_{T} - {Mean}_{C}}{\sqrt{(n_{T} - 1) {S D}_{T}^{2} + (n_{c} - 1) {S D}_{C}^{2} / n_{T} + n_{c} - 2}}

Mean_T, n_T, and SD_T represent the mean, sample size, and standard deviation of the treated group, respectively; Mean_C, n_C, and SD_C represent the mean, sample size, and standard deviation of the control group, respectively (Hedges & Olkin, 1985).

Outlier Diagnosis

Potential outliers that yielded extremely large effect sizes should be excluded from calculation (Higgins et al., 2019). According to Lipsey and Wilson (2001), potential outliers refer to extreme effect sizes that are more than three SDs from the mean of all the effect sizes. In this case, one study (g = 3.99, Hsu et al., 2021) was excluded, resulting in a total of 29 valid studies for the forthcoming analysis.

Publication Bias Analysis

Studies with positive findings are more likely to be published, which may result in publication bias (Rosenthal, 1991). For this purpose, visual (i.e., funnel plot, see Figure 3) and mathematical (i.e., Egger’s test) checks were both employed. Egger’s test is a linear regression method, which was used to test the funnel-plot symmetry. “It uses the standardized estimate with size effect as a dependent variable and the inverse of the standard error as an independent variable. If the intercept is significantly different from zero, the estimate of the effect is considered biased” (Shi et al., 2017, p. 15). In this study, Egger’ s test provided statistical evidence (p = .001), suggesting the existence of publication bias. Moreover, the ‘trim and fill’ method was used to test and adjust publication bias (Møller & Jennions, 2001). The mean effect size after trimming and filling for zero or five missing studies respectively was g = .692 (95% CI [.486, .897]) and g = .816 (95% CI [.604, 1.029]) (both ps < .05) compared with an original estimate of g = .692 (95% CI [.486, .897]). The conclusion is therefore reliable even if it is assumed that five studies are missing owing to publication bias.

Figure 3.

Funnel plot of the selected studies.

Results

Overall Analysis Results

Results were aggregated from the 47 effect sizes out of 29 studies with a total of 1791 participants. The eligible studies include 19 studies with one effect size and 10 with multiple effect sizes. As shown in Table 2, the overall effect size is moderate (g = .69, 95% CI [.49, .90]) and significant (Z = 6.60, p < .001), indicating a positive and medium effect for using social robots for EFL skill development.

Table 2.

Overall Effect Size Results of the Included Studies.

			95% CI			Heterogeneity
k	N	g	LL	UL	Z	Q	I ²
47	1791	.69	.49	.90	6.60***	325.71***	87.90

Note. k = number of effect sizes; N = number of participants; g = Hedges’ g; LL = lower limit, UL = upper limit; 95% CI = 95% confidence interval; ***p < .001.

Homogeneity Analysis Results

A between-study Q test was executed to examine whether there was substantial variability in the outcomes of the primary studies and the need for moderator analyses (Borenstein et al., 2005, 2009). In this study (see Table 2), Q-value was 325.71 with p < .001, indicating that moderator analysis was needed.

Moderator Analysis Results

The moderator analysis was conducted to examine the moderating effect of six moderators, including educational levels, intervention durations, types of robot form, types of robot role, types of teacher role, and interaction types. In Table 2, two moderators–educational levels and intervention durations–were found to have a moderating effect on the overall effect sizes, whereas the other moderators did not find a significant moderating effect.

Subjects

Educational Levels

There were three educational levels, primary (k = 26), secondary (k = 6), and tertiary (k = 15). The significant between-group difference of educational levels was found, Q _between = 5.56, p = .038. According to Table 2, RALL was found to be significantly effective among learners of secondary (g = 1.098, 95% CI [.580, 1.617]), and primary (g = .791, 95% CI [.535, 1.047]) levels, but those of tertiary education level (g = .329, 95% CI [−.043, .701]) did not obtain a significant result.

Objects

Language Skills

There were six language skills involved in previous RALL research, listening (k = 5), reading (k = 2), speaking (k = 11), writing (k = 1), vocabulary (k = 18), and language in general (k = 10). Table 2 showed that RALL was effective to develop learners’ listening (g = .772, 95% CI [.307, 1.236]), speaking (g = .709, 95% CI [.245, 1.172]), vocabulary (g = .637, 95% CI [.244, 1.103]), and language in general (g = .797, 95% CI [.393, 1.201]), rather than other two skills, viz. reading (g = .591, 95% CI [−.279, 1.461]), and writing (g = .398, 95% CI [−.143, .939]). However, there was no statistical difference among these language skills, Q _between = 1.61, p = .900.

Rules

Intervention Durations

Three categories of intervention durations were involved: short (k = 19), intermediate (k = 14), and long (k = 14). As shown in Table 2, both intermediate (g = .933, 95% CI [.487, 1.379]), and long intervention duration (g = .871, 95% CI [.466, 1.277]) achieved large effect sizes, much higher than that of short intervention duration (g = .377, 95% CI [.162, .592]) with a significant between-group difference, Q _between = 7.73, p = .021.

Control Conditions

The moderator effect of control conditions did not reach a significant level, Q _between = .32, p = .572, with moderate effect sizes for both the traditional methods (g = .713, 95% CI [.480, .946]) and other technology (g = .577, 95% CI [.164, .989]), suggesting that the effects of RALL studies are unlikely to be biased by the varying configurations of the control groups.

Community

Interaction Types

Interaction types can be roughly divided into two categories, in groups (k = 16) and one-on-one (k = 31). The effect size of in groups (g = .944, 95% CI [.560, .329]) was large, and one-on-one (g = 0.568, 95% CI [.333, .803]) had a medium effect size. In addition, there was a marginally significant between-group difference, Q _between = 2.67, p = 1.102.

Labor Division

Types of Teacher Role

Teacher roles consisted of three categories, no teacher (k = 34), tutor (k = 11), and assistant (k = 2). While the effect of assistant (g = 1.023, 95% CI [−.216, 2.263]) was not observed, tutor (g = .790, 95% CI [.469, 1.110]) achieved a large effect size, and no teachers (g = .643, 95% CI [.384, .903]) had a medium effect. No between-group difference was found, Q _between = .74, p = .692.

Types of Robot Role

Social robots can play three roles in RALL, including assistant (k = 14), peer (k = 7), and tutor (k = 26). While peer (g = .604, 95% CI [−.035, 1.243]) was found to be non-significant, significant effects of assistant role (g = .946, 95% CI [.642, 1.251]), and tutor role (g = .579, 95% CI [.300, .859]) had been obtained. No between-group difference was observed, Q _between = 3.20, p = .202.

Tool

Types of Robot Form

According to social robots’ appearances, robot forms can be divided into three categories, anthropomorphic (k = 31), mechanomorphic (k = 14), and zoomorphic (k = 2). Table 2 showed that mechanomorphic (g = .976, 95% CI [.519, 1.432]) and anthropomorphic social robots (g = .554, 95% CI [.335, .772]) were found to be effective, while the effectiveness of zoomorphic social robots (g = .867, 95% CI [−.547, 2.282]) was not achieved. Moreover, there was no between-group difference, Q _between = 2.77, p = .250.

Discussion

Considering that numerous RALL studies have been accumulated over the past two decades, the current meta-analysis aimed to explore the effects of RALL on EFL learners’ language skill development and understand how the effects were influenced by potential AT-related moderators. Results showed that RALL had a positive and moderate effect on EFL development. Furthermore, the moderator analysis results indicated that the educational levels and intervention durations were significant moderators, while significant effects of other moderators were not observed. (Table 3).

Table 3.

Moderator Analysis Results.

Moderators	k	g	95% CI	Q _between	p
Subjects
Educational levels				6.56*	.038
primary	26	.791	[.535, 1.047]
secondary	6	1.098	[.580, 1.617]
tertiary	15	.329	[-.043, .701]
Objects
Language skills				1.61	.900
listening	5	.772	[.307, 1.236]
speaking	11	.709	[.245, 1.172]
reading	2	.591	[-.279, 1.461]
writing	1	.398	[-.143, .939]
vocabulary	18	.637	[.244, 1.103]
language In general	10	.797	[.393, 1.201]
Rules
Intervention durations				7.73*	.021
short	19	.377	[.162, .592]
intermediate	14	.933	[.487, 1.379]
long	14	.871	[.466, 1.277]
Control conditions				.32	.572
traditional Methods	40	.713	[.480, .946]
other Technology	7	.577	[.164, .989]
Community
Interaction types				2.67	1.102
in Groups	16	.944	[.560, 1.329]
one-On-one	31	.568	[.333, .803]
Labor division
Types of teacher role				.74	.692
no Teachers	34	.643	[.384, .903]
assistant	2	1.023	[-.216, 2.263]
tutor	11	.790	[.469, 1.110]
Types of robot role				3.20	.202
assistant	14	.946	[.642, 1.251]
peer	7	.604	[-.035, 1.243]
tutor	26	.579	[.300, .859]
Tool
Types of robot form				2.77	.250
anthropomorphic	31	.554	[.335, .772]
mechanomorphic	14	.976	[.519, 1.432]
zoomorphic	2	.867	[-.054, 2.282]

Note. k is the number of independent studies available for the certain variable; Hedges’ g is effect size; CI is short for confidence interval; p-value indicates significance ^†p < .10; *p < .05.

The first research question dealt with the overall effect size of RALL for EFL skill development compared with non-RALL conditions, which is in line with most previous RALL research (e.g., Aidinlou, 2014; Kanero et al., 2018; Kory Westlund et al., 2017; Mubin et al., 2013), suggesting social robots can serve as effective EFL tools. The beneficial effects of RALL might be explained by a number of reasons: First, RALL affords EFL learners with rich authentic language contexts and human-humanoid interactions. Confronted with the limited exposure to authentic language contexts in the traditional methods, social robots can play various social roles (Randall, 2020), and provide natural communication for EFL learners through imitating human-to-human conversations (Han et al., 2008; Lin et al., 2022; Tai & Chen, 2022). Second, RALL promotes EFL learners’ positive FL emotions and motivation. In other words, social robots have positive effects on EFL learners’ affects, such as motivation (Donnermann et al., 2020; Wu et al., 2015), interest (Han et al., 2005, 2008; Lee et al., 2011), and confidence (Lee et al., 2011; Hong et al., 2016; Tsai, 2019), which have been found to be positively related to learning achievements (Chen et al., 2020). Social robots can support interactive learning activities and allow learners to practice repeatedly in a relaxed atmosphere (Lee et al., 2011), which might be beneficial to increase learner motivation, gain confidence, and lower anxiety (Alemi and Bahramipour, 2019; Vogt et al., 2019). Third, Dual Coding Theory (DCT) assumes that the interconnection of a verbal channel and a non-verbal channel can decrease the cognitive load on working memory (Paivio, 1991). Social robots can relieve the cognitive load on working memory through associating verbal behaviors (e.g., offering feedback, saying someone’s name) (Tlili et al., 2020; van den Berghe et al., 2019) and nonverbal behaviors (e.g., gesture, body movement) (Banaeian & Gilanlioglu, 2021; Donnermann et al., 2020), which can facilitate language development in turn (Li, 2021). Fourth, social robots can support personalized learning, since social robots can update timely learning content and afford feedback to meet personalized needs and preferences (Belpaeme et al., 2013).

The second research question was to gain a deeper understanding of how potential AT-related moderators have an impact on the effectiveness of RALL. The significant moderators of educational levels and intervention durations are discussed first. On the one hand, moderator analysis of educational levels showed that RALL was effective for learners of the primary and secondary education levels, rather than those of tertiary level, resonating with previous studies that RALL might be more beneficial for young learners than older learners (Kanda et al., 2004; Zhang et al., 2021). To further elucidate, while adults are cognitively mature (Li, 2021), children who are cognitively premature are easily attracted by the RALL’s entertainment features, such as game-based activities, and various presentation types (Eimler et al., 2010; Park et al., 2011). Thus, compared with younger generations, adult learners are likely to psychologically regard social robots as childish toys rather than effective learning tools (Kanda et al., 2004). On the other hand, moderator analysis results of intervention durations indicate that longer durations are favored, as both intermediate and long durations achieved large effect sizes, while short duration had a small effect size, echoing Sung et al. (2015), because learners need time to be acquainted with educational technologies (Li, 2022a). Furthermore, while both intermediate and long durations achieved significant effect sizes, the effect of intermediate duration was larger than that of long duration might be accounted for by the novelty effect (Li, 2023b). In other words, after learners get acquainted with the RALL tools in a short duration period, they would have a great interest in social robots at the initial use during the intermediate duration, but their interest would wear off if they are familiar with social robots and continue using them for the long duration, warranting the needs to overcome RALL’s novelty effect in the future research.

Apart from those significant moderators, other non-significant moderators of language skills, control conditions, interaction types, types of teacher role, types of robot role and types of robot form, are valuable to be discussed. The moderator analysis of language skills showed RALL was effective to develop learners’ listening, speaking, vocabulary, and language in general, rather than other skills, viz. reading and writing. The limited effects of RALL on reading and writing might rest on RALL’s technological limitations. For instance, Hong and colleagues (2016) investigated the effect of RALL on EFL learners’ overall (listening, speaking, reading and writing) performance using a quasi-experiment design, and claimed that RALL could only enhance their listening and speaking, as opposed to reading and writing skills, which is partly due to technological unavailability of reading and writing tasks. Likewise, Lin and Chang (2020) adopted a mixed method to explore the effect of RALL on post-secondary writers’ writing skills. Results demonstrated that writing social robots still bear some complex technological realities, such as difficulties in understanding abstract writing topics and learners’ natural language. Another possible reason might be due to the small number of selected studies (k_reading = 2 vs. k_writing = 1), which might limit the generalizability of moderator analysis results drawn for these two language skills, warranting further empirical attempts. For control conditions, while both traditional methods and other technologies have reached moderate effect sizes, there was no statistical difference between these two conditions, suggesting that RALL for EFL development is robustly effective, regardless of the difference in control conditions. In other words, although researchers adopted different methods (e.g., ordinary paper-and-pencil, noncomputer-based media, or web-based instruction) as control conditions, all those included studies were conducted under rigorous quasi-experimental designs, lending support to the accuracy and reliability of the quasi-experiment results. For interaction types, “in groups” type obtained a larger effect size than the “one-on-one” type, because language learners in groups can communicate and cooperate with their group peers, and collaborative learning could sustain and increase their motivation, which leads to improved language performance (Chen et al., 2020). When it comes to types of teacher role and robot role, both results indicated the only involvement of robot (viz. no availability of teachers and RALL serving as tutors) generates the smallest effect sizes as compared other types. One possible reason is that social robots are unlikely to fully replace human teachers due to technology limitations (Belpaeme et al., 2013), and other internet-related problems (Li, 2022c; 2023c). For instance, learners who lack information literacy skills would be puzzled about how to use social robots without teachers’ assistance (MacIntyre & Vincze, 2017). For types of robot form, besides the ineffective of zoomorphic social robots, the beneficiary effects of anthropomorphic and mechanomorphic social robots were obtained, and mechanomorphic social robots had a higher effect size that of anthropomorphic social robots. Compared with anthropomorphic social robots having a human-like torso or facial features, the ineffective of zoomorphic robots having animal-like features might be explained by the poverty of resemblance to real-life world (Yang & Li, 2023), as social robots that bear higher resemblance to real-life world would bring young learners with more immersive and embodied learning experiences (Banaeian & Gilanlioglu, 2021; Tsai, 2019), along with enhanced learning motivation and performance (Randall, 2020). Moreover, the highest effect of mechanimorphic social robots might be due to the fact that most mechanomorphic social robots are chatbots, which would customize courses for language learners (Hsu et al., 2021).

Implications

Implications for Teachers

First, to effectively integrate RALL into teaching activities, teachers should understand the pedagogical advantages of RALL. For instance, teachers can use social robots to attract learners’ attention (Aidinlou, 2014; Eimler et al., 2010), and provide personalized feedback through recording and saving learners’ language performance (Randall, 2020), which might enhance learners’ second language (L2) engagement (Chang et al., 2010). Second, what is noteworthy is that teachers should realize the challenges when employing social robots to facilitate language learning, such as the limited positive effect of short intervention duration (Engwall & Lopes, 2022; Uslu et al., 2022), difficulties in using social robots as tutors (van den Berghe et al., 2018), and suffering from a lack of interest in RALL for adult learners (van den Berghe et al., 2019). Third, teachers should avoid the novelty effect of RALL use and try to maximize the utilization of RALL, a possible solution is to optimize curriculum design by integrating language learning content well with the RALL’s functionalities to sustain learners’ motivation, interest and engagement in the long durations (Li, 2023b).

Implications for Designers

First, RALL designers should pay more attention to personalized learning materials based on users’ educational levels, and integrate interactive and collaborative features into RALL (Kim et al., 2022), since collaborative learning with interaction features can support social interactivity and enable learners to communicate with peers, thus enhancing their language skills (Alemi & Haeri, 2020; de Wit et al., 2018), and develop more functions for assistant robots because social robots as teacher assistants have the potential to facilitate language learning. Second, designers should be aware of the importance of different robot forms on learning outcomes according to learners’ individual differences (e.g., age, gender, and preference, etc.). For instance, anthropomorphic social robots having a human-like torso or facial features that bear a high resemblance to the real-life world would be psychologically more acceptable among young learners, which would improve their L2 learning outcomes in turn. Third, as RALL tools for reading and writing are scarce and have limited effects, the increasing demand in RALL for L2 development calls for designers to pay a particular eye on learners’ gratification of personalized needs in terms of reading and writing skill development.

Implications for Researchers

First, to further explore the potential of RALL, researchers should attempt to include large sample sizes to guarantee compelling statistical power since most related studies tend to have small sample sizes, which led to the lack of a large-scale field study on this topic (van den Berghe et al., 2019). Second, few studies focused on EFL writing and reading skills, suggesting researchers should pay more attention to these skills in further research. Third, given that most relevant experiments were conducted in kindergarten or primary schools and few focused on secondary education, warranting further investigations in this regard.

Conclusion

This study meta-analyzed the effects of RALL on EFL learners’ language skill development based on the theoretical framework of AT. The results revealed that the overall effect size was significantly moderate, suggesting RALL has more positive effects on EFL language development than non-RALL conditions. Moderator analysis results showed that educational levels and intervention durations were significant moderators.

Notwithstanding the meaningful findings that have been obtained, this study is not without its limitations. A main limitation lies in the limited studies that have been included. While we have followed the PRISMA guidelines to conduct the systematic literature retrieval, it is inevitable to miss some studies due to the limited number of search pathways or search techniques. Further endeavors that include more studies should be done in the future.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Scientific Research Funds from Hunan Provincial Ministry of Education (grant number 23A0040).

ORCID iD

Rui Li

Data Availability Statement

Data are available from the corresponding author upon reasonable request.

Author Biographies

Xueqing Wu is a PhD student at School of Foreign Languages, Hunan University. Her research interest includes computer assisted language learning. ORCID: .

Rui Li is currently a full professor, also serving as a doctoral and post-doctoral supervisor at School of Foreign Languages, Hunan University. His research interests include computer assisted language learning, applied linguistics and psycholinguistics. His recent publications appear in top-tier SSCI-indexed journals, such as Computers and Education, Computer Assisted Language Learning, Language Learning and Technology, ReCALL, British Journal of Educational Technology, Journal of Computer Assisted Learning, Journal of Computing in Higher Education, Journal of Educational Computing Research, Educational Technology and Society, Reading and Writing, Journal of Psycholinguistic Research, Porta Linguarum, among others. He also works as an associate editor of Humanities and Social Sciences Communications (SSCI/A&HCI, since 2022), and an anonymous reviewer for over 30 highly ranked SSCI-indexed journals, such as Computer Assisted Language Learning, Language Learning and Technology, and ReCALL, among others. ORCID: .

References

Articles marked with an asterisk (*) were analysed in the meta-analysis.

Aidinlou

N. A.

(2014). Applications of robot assisted language learning (RALL) in language learning and teaching. International Journal of Language and Linguistics, 2(3), 12–20. https://doi.org/10.11648/j.ijll.s.2014020301.12

* Alemi

Bahramipour

(2019). An innovative approach of incorporating a humanoid robot into teaching EFL learners with intellectual disabilities. Asian-Pacific Journal of Second and Foreign Language Education, 4(1), 1–22. https://doi.org/10.1186/s40862-019-0075-5

* Alemi

Haeri

N. S.

(2020). Robot-assisted instruction of L2 pragmatics: Effects on young EFL learners’ speech act performance. Language Learning and Technology, 24(2), 86–103. https://hdl.handle.net/10125/44727

* Alemi

Meghdari

Ghazisaedy

(2015). The impact of social robotics on L2 learners’ anxiety and attitude in English vocabulary acquisition. International Journal of Social Robotics, 7(4), 523–535. https://doi.org/10.1007/s12369-015-0286-y

* Banaeian

Gilanlioglu

(2021). Influence of the NAO robot as a teaching assistant on university students’ vocabulary learning and attitudes. Australasian Journal of Educational Technology, 37(3), 71–87. https://doi.org/10.14742/ajet.6130

Belpaeme

Baxter

P. E.

Read

Wood

Cuayáhuitl

Kiefer

Racioppa

Humbert

(2013). Multimodal child-robot interaction: Building social bonds. Journal of Human-Robot Interaction, 1(2), 33–53. https://doi.org/10.5898/JHRI.1.2.Belpaeme

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2005). Comprehensive meta-analysis version 2.0 (Computer software). Biostat.

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2009). Introduction to meta-analysis. John Wiley and Sons, Ltd.

10.

Chang

C. W.

Lee

J. H.

Wang

C. Y.

Chen

G. D.

(2010). Improving the authentic learning experience by integrating robots into the mixed-reality environment. Computers and Education, 55(4), 1572–1578. https://doi.org/10.1016/j.compedu.2010.06.023

11.

Chen

H. L.

Vicki

W. G.

Sutrisno

(2020). A chatbot for learning Chinese: Learning achievement and technology acceptance. Journal of Educational Computing Research, 58(6), 1161–1189. https://doi.org/10.1177/0735633120929622

12.

* de Wit

Schodde

Willemsen

Bergmann

de Haas

Kopp

Krahmer

Vogt

(2018). The effect of a robot’s gestures and adaptive tutoring on children’s acquisition of second language vocabularies. In Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction (pp. 50–58). Association for Computing Machinery. https://doi.org/10.1145/3171221.3171277

13.

* Dizon

(2020). Evaluating intelligent personal assistants for L2 listening and speaking development. Language, Learning and Technology, 24(1), 16–26.

14.

Donnermann

Schaper

Lugrin

(2020). Integrating a social robot in higher education: A field study. In 2020 29th IEEE international conference on robot and human interactive communication (RO-MAN) (pp. 573–579). IEEE. https://doi.org/10.1109/RO-MAN47096.2020.9223602

15.

* Eimler

von der Pütten

Schächtle

(2010). Following the white rabbit—a robot rabbit as vocabulary trainer for beginners of English. In Leitner

Hitz

Holzinger

(Eds.), Proceedings of the 6th symposium of the workgroup human-computer interaction and usability engineering (pp. 322–339). Springer. https://doi.org/10.1007/978-3-642-16607-522

16.

* El Shazly

(2021). Effects of artificial intelligence on English speaking anxiety and speaking performance: A case study. Expert Systems, 38(3), 1–15. https://doi.org/10.1111/exsy.12667

17.

Engeström

(1987). Learning by expanding: An activity-theoretical approach to develop mental research. Orienta-Konsultit.

18.

Engeström

(2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work, 14(1), 133–156. https://doi.org/10.1080/13639080020028747

19.

Engwall

Lopes

(2022). Interaction and collaboration in robot-assisted language learning for adults. Computer Assisted Language Learning, 35(5–6), 1273–1309. https://doi.org/10.1080/09588221.2020.1799821

20.

* Han

Park

Kim

(2005). The educational use of home robots for children. In ROMAN 2005 IEEE International Workshop on Robot and Human Interactive Communication, 2005, Nashville, TN, 13–15 Aug. 2005, 378–383. https://doi.org/10.1109/ROMAN.2005.1513808

21.

* Han

J. H.

Jones

J. H.

(2008). Comparative study of effects of language instruction program using intelligence robot and multimedia on linguistic ability of young children. In RO-MAN 2008 - the 17th IEEE international symposium on robot and human interactive communication (pp. 187–192). IEEE. https://doi.org/10.1109/ROMAN.2008.4600664

22.

Hedges

Olkin

(1985). Statistical methods for meta-analysis. Acedamic Press, INC.

23.

Higgins

Huscroft D’Angelo

Crawford

(2019). Effects of technology in mathematics on achievement, motivation, and attitude: A meta-analysis. Journal of Educational Computing Research, 57(2), 283–319. https://doi.org/10.1177/0735633117748416

24.

* Hong

Z. W.

Huang

Y. M.

Hsu

Shen

W. W.

(2016). Authoring robot-assisted instructional materials for improving learning performance and motivation in EFL classrooms. Educational Technology and Society, 19(1), 337–349. https://www-jstor-org-443.web.bisu.edu.cn/stable/10.2307/jeductechsoci.19.1.337

25.

Hsiao

H. S.

Chang

C. S.

Lin

C. Y.

Hsu

H. L.

(2015). “iRobiQ”: The influence of bidirectional interaction on kindergarteners’ reading motivation, literacy, and behavior. Interactive Learning Environments, 23(3), 269–292. https://doi.org/10.1080/10494820.2012.745435

26.

* Hsieh

W. M.

Yeh

H. C.

Chen

N. S.

(2023). Impact of a robot and tangible object (R&T) integrated learning system on elementary EFL learners’ English pronunciation and willingness to communicate. Computer Assisted Language Learning, forthcoming. https://doi.org/10.1080/09588221.2023.2228357

27.

Hsu

M. H.

Chen

(2021). Proposing a task-oriented chatbot system for EFL learners speaking practice. Interactive Learning Environments, 23(3), 269–292. https://doi.org/10.1080/10494820.2021.1960864

28.

Hsu

Liang

Y. S.

(2021). Simultaneously improving computational thinking and foreign language learning: Interdisciplinary media with plugged and unplugged approaches. Journal of Educational Computing Research, 59(6), 1184–1207. https://doi.org/10.1177/0735633121992480

29.

Huang

(2005). A diary study of difficulties and constraints in EFL learning. System, 33(4), 609–621. https://doi.org/10.1016/j.system.2005.04.001

30.

* Iio

Maeda

Ogawa

Yoshikawa

Ishiguro

Suzuki

Aoki

Maesaki

Hama

(2019). Improvement of Japanese adults’ English speaking skills via experiences speaking to a robot. Journal of Computer Assisted Learning, 35(2), 228–245. https://doi.org/10.1111/jcal.12325

31.

* Jeon

(2023). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis. Computer Assisted Language Learning, 36(7), 1338–1364. https://doi.org/10.1080/09588221.2021.1987272

32.

Jeon

(2022). Exploring AI chatbot affordances in the EFL classroom: Young learners’ experiences and perspectives. Computer Assisted Language Learning, forthcoming. https://doi.org/10.1080/09588221.2021.2021241

33.

* Jia

Chen

Ding

Ruan

(2012). Effects of a vocabulary acquisition and assessment system on students’ performance in a blended learning class for English subject. Computer Education, 58(1), 63–76. https://doi.org/10.1016/j.compedu.2011.08.002

34.

* Kanda

Hirano

Eaton

Ishiguro

(2004). Interactive robots as social partners and peer tutors for children: A field trial. Human-Computer Interaction, 19(1), 61–84. https://doi.org/10.1207/s15327051hci1901&2_4

35.

Kanero

Geçkin

Oranç

Mamus

Küntay

A. C.

Göksun

(2018). Social robots for early language learning: Current evidence and future directions. Child Development Perspectives, 12(3), 146–151. https://doi.org/10.1111/cdep.12277

36.

* Kanero

Oranç

Koşkulu

Kumkale

G. T.

Göksun

Küntay

A. C.

(2022). Are tutor robots for everyone? The influence of attitudes, anxiety, and personality on robot-led language learning. International Journal of Social Robotics, 14(2), 297–312. https://doi.org/10.1007/s12369-021-00789-3

37.

* Kim

N. Y.

(2018). A study on chatbots for developing Korean college students’ English listening and reading skills. Journal of Digital Convergence, 16(8), 19–26. https://doi.org/10.14400/JDC.2018.16.8.019

38.

Kim

Yang

Shin

Lee

J. H.

(2022). Design principles and architecture of a second language learning chatbot. Language, Learning and Technology, 26(1), 1–18. https://hdl.handle.net/10125/73463

39.

* Kim

Jeongjo

(2018). Chatbots and Korean EFL students’ English vocabulary learning. Journal of Digital Convergence, 16(2), 1–7. https://doi.org/10.14400/JDC.2018.16.2.001

40.

* Konijn

E. A.

Jansen

Mondaca Bustos

Hobbelink

V. L. N. F.

Preciado Vanegas

(2021). Social robots for (second) language learning in (migrant) primary school children. International Journal of Social Robotics, 14(3), 827–843. https://doi.org/10.1007/s12369-021-00824-3

41.

Kory Westlund

J. M.

Dickens

Jeong

Harris

P. L.

DeSteno

Breazeal

C. L.

(2017). Children use non-verbal cues to learn new words from robots as well as people. International Journal of Child-Computer Interaction, 13, 1–9. https://doi.org/10.1016/j.ijcci.2017.04.001

42.

Lee

Yeo

(2022). Developing an AI-based chatbot for practicing responsive teaching in mathematics. Computers and Education, 191, 1–17. https://doi.org/10.1016/j.compedu.2022.104646

43.

Lee

J. H.

(2022). The effects of robot-assisted language learning: A meta-analysis. Educational Research Review, 35, 1–13. https://doi.org/10.1016/j.edurev.2021.100425

44.

* Lee

Noh

Lee

G. G.

Sagong

Kim

(2011). On the effectiveness of robot-assisted language learning. ReCALL, 23(1), 25–58. https://doi.org/10.1017/S0958344010000273

45.

Leont’ev

A. N.

(1981). The problem of activity in psychology. In Wertsch

J. V.

(Ed.), The concept of activity in soviet psychology (pp. 37–71). Sharpe.

46.

(2021). Does game-based vocabulary learning APP influence Chinese EFL learners’ vocabulary achievement, motivation, and self-confidence? Sage Open, 11(1), 1–15. https://doi.org/10.1177/21582440211003092

47.

(2022a). Effects of mobile-assisted language learning on EFL/ESL reading comprehension. Educational Technology and Society, 25(3), 15–29. https://www-jstor-org-443.web.bisu.edu.cn/stable/48673721

48.

(2022b). Effects of blended language learning on EFL learners’ language performance: An activity theory approach. Journal of Computer Assisted Learning, 38(2), 1273–1285. https://doi.org/10.1111/jcal.12697

49.

(2022c). Research trends of blended language learning: A bibliometric synthesis of SSCI-indexed journal articles during 2000–2019. ReCALL, 34(3), 309–326. https://doi.org/10.1017/S0958344021000343

50.

(2023a). Effects of mobile-assisted language learning on EFL learners’ listening skill development. Educational Technology and Society, 26(2), 36–49. https://www.j-ets.net/collection/forthcoming-articles/26_2

51.

(2023b). Investigating effects of computer-mediated feedback on L2 vocabulary learning. Computers and Education, 198(2), 1–11. https://doi.org/10.1016/j.compedu.2023.104763

52.

(2023c). Still a fallible tool? Revisiting effects of automated writing evaluation from activity theory perspective. British Journal of Educational Technology, 54(3), 773–789. https://doi.org/10.1111/bjet.13294

53.

(2024). Effects of mobile-assisted language learning on foreign language learners’ speaking skill development. Language Learning and Technology, 1–24.

54.

Meng

Z. K.

Tian

Zhang

Z. Y.

C. B.

Xiao

(2021). Modelling Chinese EFL learners’ flow experiences in digital game-based vocabulary learning: The roles of learner and contextual factors. Computer Assisted Language Learning, 34(4), 483–505. https://doi.org/10.1080/09588221.2019.1619585

55.

* Liang

J. C.

Hwang

G. J.

(2023). A robot-based digital storytelling approach to enhancing EFL learners’ multimodal storytelling ability and narrative engagement. Computers and Education, 201(3), 1–24. https://doi.org/10.1016/j.compedu.2023.104827

56.

Lin

Liu

Kou

Kulikova

Lin

(2019). Mobile-assisted reading development: A review from the activity theory perspective. Computer Assisted Language Learning, 33(8), 833–864. https://doi.org/10.1080/09588221.2019.1594919

57.

Lin

M. P. C.

Chang

(2020). Enhancing post-secondary writers’ writing skills with a chatbot: A mixed-method classroom study. Educational Technology and Society, 23(1), 78–92. https://creativecommons.org/licenses/by-nc-nd/3.0/

58.

Lin

Yeh

H. C.

Huang

H. H.

Chen

N. S.

(2022). Enhancing EFL vocabulary learning with multimodal cues supported by an educational robot and an IoT-Based 3D book. System, 104(6), 102–121. https://doi.org/10.1016/j.system.2021.102691

59.

Lipsey

M. W.

Wilson

D. B.

(2001). Practical meta-analysis. Sage.

60.

* Liu

Y. F.

Hwang

W. Y.

C. H.

(2023). Investigating the impact of context-awareness smart learning mechanism on EFL conversation learning. Interactive Learning Environments, forthcoming. https://doi.org/10.1080/10494820.2023.2194931

61.

MacIntyre

Vincze

(2017). Positive and negative emotions underlie motivation for L2 learning. Studies in Second Language Learning and Teaching, 7(1), 61–88. https://doi.org/10.14746/SSLLT.2017.7.1.4

62.

Meng

Z. K.

(2023). Understanding Chinese teachers’ informal online learning continuance in a mobile learning community: An intrinsic-extrinsic motivation perspective. Journal of Computing in Higher Education, 1–23. https://doi.org/10.1007/s12528-023-09352-7

63.

Moher

Liberati

Tetzlaff

Altman

D. G.

(2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151(4), 264–269. https://doi.org/10.7326/0003-819-151-4-200908180-00135

64.

Møller

A. P.

Jennions

M. D.

(2001). Testing and adjusting for publication bias. Trends in Ecology and Evolution, 16(10), 580–586. https://doi.org/10.1016/S0169-5347(01)02235-2

65.

Mubin

Bartneck

Feijs

Hooft van Huysduynen

Muelver

(2012). Improving speech recognition with the robot interaction language. Disruptive Science and Technology, 1(2), 79–88. https://sci-hub.et.fine.com/10.1089/dst.2012.0010

66.

Mubin

Stevens

C. J.

Shahid

Mahmud

A. A.

Dong

J. J.

(2013). A review of the applicability of robots in education. Technology for Education and Learning, 1(1), 1–7. https://doi.org/10.2316/Journal.209.2013.1.209-0015

67.

Paivio

(1991). Dual coding theory: Retrospect and current status. Journal of Psychology, 45(3), 255–287. https://doi.org/10.1037/h0084295

68.

Park

S. J.

Han

J. H.

Kang

B. H.

Shin

K. C.

(2011). Teaching assistant robot, ROBOSEM, in English class and practical issues for its diffusion. Advanced robotics and its social impacts (pp. 8–11). Menlo Park. https://doi.org/10.1109/ARSO.2011.6301971

69.

Plonsky

Oswald

F. L.

(2014). How big is “big”? Interpreting effect sizes in L2 research: Effect sizes in L2 research. Language Learning, 64(4), 878–912. https://doi.org/10.1111/lang.12079

70.

Randall

(2020). A survey of robot-assisted language learning (RALL). ACM Transactions on Human-Robot Interaction, 9(1), 1–36. https://doi.org/10.1145/3345506

71.

Rosenthal

(1991). Meta-analytic procedures for social research. Sage.

72.

Shi

X. Q.

Nie

Shi

Wang

Yang

H. J.

Zhou

(2017). Effect comparison between Egger’s test and Begg’s test in publication bias diagnosis in meta-analyses: Evidence from a pilot survey. International Journal of Research Studies in Biosciences, 5(5), 14–20. https://doi.org/10.20431/2349-0365.0505003

73.

Spolaôr

Benitti

F. B. V.

(2017). Robotics applications grounded in learning theories on tertiary education: A systematic review. Computers and Education, 112(5), 97–107. https://doi.org/10.1016/j.compedu.2017.05.001

74.

Sung

Y. T.

Chang

K. E.

Yang

J. M.

(2015). How effective are mobile devices for language learning? A meta-analysis. Educational Research Review, 16, 68–84. https://doi.org/10.1016/j.edurev.2015.09.001

75.

* Tai

T. Y.

(2022). Effects of intelligent personal assistants on EFL learners’ oral proficiency outside the classroom. Computer Assisted Language Learning, forthcoming. https://doi.org/10.1080/09588221.2022.2075013

76.

* Tai

T. Y.

Chen

H. H.

(2022). The impact of intelligent personal assistants on adolescent EFL learners’ listening comprehension. Computer Assisted Language Learning, forthcoming. https://doi.org/10.1080/09588221.2022.2040536

77.

Tlili

Lin

Chen

N.-S.

Huang

Kinshuk (2020). A systematic review on robot-assisted special education from the activity theory perspective. Educational Technology and Society, 23(3), 95–109. https://creativecommons.org/licenses/by-nc-nd/3.0/

78.

* Tsai

(2019). Adapting an anthropomorphized robot for enhancing EFL learning motivation and performance at an elementary school in Taiwan. The New Educational Review, 58(4), 22–35. https://doi.org/10.15804/tner.2019.58.4.02

79.

Uslu

A. N.

Yavuz

G. Ö.

Koçak

U. Y.

(2022). A systematic review study on educational robotics and robots. Interactive Learning Environments. 31(9), 5874–5898. forthcoming. https://doi.org/10.1080/10494820.2021.2023890

80.

van den Berghe

van der Ven

Verhagen

Oudgenoeg-Paz

Papadopoulos

Leseman

(2018). Investigating the effects of a robot peer on L2 word learning. In Companion of the 2018 ACM/IEEE international conference on human-robot interaction (pp. 267–268). ACM. https://doi.org/10.1145/3173386.3176986

81.

van den Berghe

Verhagen

Oudgenoeg-Paz

van der Ven

Leseman

(2019). Social robots for language learning: A review. Review of Educational Research, 89(2), 259–295. https://doi.org/10.3102/0034654318821286

82.

* Vogt

Berghe

R. V.

Haas

M. D.

Hoffmann

Kanero

Mamus

Montanier

Oranç

Oudgenoeg-Paz

García

D. H.

Papadopoulos

Schodde

Verhagen

Wallbridge

C. D.

Willemsen

Wit

J. D.

Belpaeme

Göksun

Kopp

Pandey

A. K.

(2019). Second language tutoring using social robots: A large-scale study. In 2019 14th ACM/IEEE international conference on human-robot interaction (HRI) (pp. 497–505). IEEE. https://doi.org/10.1109/HRI.2019.8673077

83.

Vygotsky

(1978). Mind in society: The development of higher mental processes. Harvard University Press.

84.

* Wang

Y. H.

Young

S. C.

Jang

R. J. S.

(2013). Using tangible companions for enhancing learning English conversation. Educational Technology and Society, 16(2), 296–309. https://www-jstor-org-443.web.bisu.edu.cn/stable/10.2307/jeductechsoci.16.2.296

85.

* Wu

C. V.

Wang

R. J.

Chen

N. S.

(2015). Instructional design using an in-house built teaching assistant robot to enhance elementary school English-as-a-foreign-language learning. Interactive Learning Environments, 23(6), 696–714. https://doi.org/10.1080/10494820.2013.792844

86.

Yang

(2023). Contextualized game-based language learning: Retrospect and prospect. Journal of Educational Computing Research, 1–19. https://doi.org/10.1177/07356331231189292

87.

Zhang

Luo

Zhu

Yin

(2021). Educational robots improve K-12 students’ computational thinking and STEM attitudes: Systematic review. Journal of Educational Computing Research, 59(7), 1450–1481. https://doi.org/10.1177/0735633121994070