The Impact of Immersive Virtual Reality on EFL Learners’ Listening Comprehension

Abstract

Virtual reality via mobile-rendered head-mounted displays (MVR) has emerged as a valuable language learning tool. However, research has yet to fully access its effects on English as a Foreign Language (EFL) listening. Therefore, this study investigated the impact of MVR on EFL learners’ listening comprehension. Seventy-two seventh graders in Taiwan were recruited and randomly assigned to an experimental (MVR player) or control (video watcher) group. The MVR players played the language learning VR app Mondly, using a mobile-rendered head-mounted display. The video watchers watched a walkthrough video of Mondly on a PC screen. Listening comprehension tests, a presence questionnaire and interviews were used to evaluate the participants’ listening comprehension, sense of presence, and perception of MVR-assisted EFL listening. The results showed the MVR players’ listening comprehension and retention was significantly higher than the video watchers’. The interviews revealed that the majority of the MVR players found MVR-assisted EFL listening engaging and beneficial. MVR offered learners access to simulated, interactive, and immersive virtual environments to perform authentic learning activities, helping them activate prior knowledge and make appropriate inferences. Furthermore, virtual presence in MVR brought learner involvement from the fringe to the center, prevented cognitive overload, reduced anxiety, and thus aided comprehension.

Keywords

virtual reality mobile-rendered HMD EFL listening immersion presence

Being a precursor to acquiring other skills, listening is crucial to language learning (Vandergrift & Baker, 2018). However, listening comprehension is often considered difficult and causes anxiety among EFL learners (Chen, 2019; Jiang & Dewaele, 2019). An enduring challenge for language instructors is how to create authentic contexts for L2 listening and prepare learners for real-world communicative contexts (Lan, 2016). To deal with L2 listening obstacles, many researchers (e.g., Lan & Liao, 2018; Liao & Lu, 2018) advocate the use of the virtual world (VW), often termed the desktop virtual environment (Lan, 2020), which provides a simulated environment for interactive, learner-centered listening.

As technology has developed, numerous virtual reality (VR) platforms have emerged and quickly migrated from the desktop to Head Mounted Displays (HMDs), creating more immersive, interactive, and engaging virtual learning environments (Fransson et al., 2020). Further, driven by advances in mobile learning devices, learners can view VR content via mobile-rendered HMDs, referred to as immersive VR or mobile VR (Ladendorf et al., 2019; Lan, 2020). Mobile VR (MVR) provides “a 3D mobile-based, virtual environment that simulates a realistic environment while simulating a physical feeling of being in an authentic location or situation” (Ladendorf et al., 2019, p.3). Learners are offered a visuospatial view, which offers sensory immersion and consequently a strong sense of presence. According to Radianti et al. (2020), MVR has the advantages of being lightweight, portable, and ubiquitous, delivering rich and fully immersive VR experiences. Therefore, MVR is widely recognized as the most suitable and affordable VR technology for schools (Fransson et al., 2020; Stojšić et al., 2019).

However, empirical evidence supporting specific L2 learning benefits of MVR is still limited, focusing mainly on motivation and vocabulary learning (Alfadil, 2020; Tai et al., 2020; Xie et al., 2019). Research has yet to determine its effects on EFL listening comprehension. The purpose of the present study, therefore, was to investigate the impact of MVR, particularly regarding the effects of virtual presence, on EFL learners’ listening comprehension and their perception of MVR-assisted EFL listening. It is hoped that the application of MVR and analysis of its pedagogical affordances offer concrete methods for the EFL listening activities involving MVR and help practitioners make informed decisions on integrating MVR into curricula.

Literature Review

Virtual World and Virtual Reality

Driven by recent advances in VR technology, numerous VR platforms have emerged. It is essential to differentiate VW experiences on computer screens from VR experiences via computer driven and wired or mobile HMDs, as they may immerse users differently (Fransson et al., 2020). A VW is a virtual space experienced through standard audio-visual equipment such as a desktop computer, where users can create an identity through an avatar and interact with other avatars (Lan, 2020). The online virtual world Second Life is a popular example of a VW setting. In contrast, VR provides a computer-generated 360° virtual space that can be spatially realistic and offers users visual, auditory, and sometimes a tactile sense of simulation (Kaplan-Rakowski & Gruber, 2019). Head-motion tracking is crucial in VR because it enables the user to choose what is viewed and therefore take control of the learning process.

Two types of wearable VR HMDs currently exist. Tethered HMDs are tethered to powerful gaming desktops and include the Oculus Rift and HTC Vive. Mobile-rendered HMDs, the second type, are self-contained, untethered HMDs that run on mobile phones such as Google Cardboard and the Samsung Gear VR. According to Ladendorf et al. (2019), MVR provides 3D mobile-based, virtual environments that enable stereoscopic views of scenes and simulates a physical presence in an authentic location or situation. Mobile-rendered HMDs have the potential to be as ubiquitous as the smartphones they run on, with lightweight, affordable, and portable characteristics. Therefore, MVR is regarded as the most suitable and affordable technology for K-12 schools (e.g., Fransson et al., 2020).

To date, most studies have used VW tools, especially Second Life, and desktop-based VR. By contrast, empirical studies exploring the effects of MVR integration in language learning are scant (Radianti et al., 2020; Xie et al., 2019). Further research on this topic is needed to help practitioners decide when to use MVR, for which purpose to use it, and which technological steps to take.

VR for Second Language Acquisition

According to Lan (2020), immersion, active learner participation, social interaction, and authenticity are the crucial components of successful language learning, which can be satisfied by mediating the three core characteristics of VR technologies: Immersion, presence, and interactivity (Radianti et al., 2020). VR can provide realistically simulated environments which allows L2/FL learners to immerse themselves and thus enhances the situated experience (Lan et al., 2019). Furthermore, VR provides learners with a special channel of interpersonal communication (Wang et al., 2020). Learners can explore the contexts and interact with the objects, avatars, and other learners. Most importantly, highly immersive virtual environments help learners achieve a strong sense of presence, which increases learners’ positive feelings during task execution (Huang et al., 2020). L2/FL learners can learn a language with an intercultural experience beyond geographical limitations with no need to step out of the classroom or leave their countries. Therefore, the specific features of VR have great potential for L2/FL learning.

Theoretical Foundations of MVR-Assisted Language Learning

With the advances in VR hardware, learners can view VR content via mobile-rendered HMDs, which had the advantages of being lightweight, portable, and ubiquitous. According to Radianti et al. (2020), MVR-assisted language learning has foundations in multiple theoretical concepts, including constructivist learning, situated learning, and embodied cognition theories. Constructivism posits that learning is an active, constructive process. Learners are information constructors who actively construct their subjective representations and comprehensions of reality. Situated cognition theory indicates that learning should be embedded in the social and physical contexts within which it will be used so that learning could be appropriately associated with the contextual experience and the applicable environment (Brown et al., 1989). With regard to embodied cognition theory, Varela et al. (2017) advocated that learners’ bodily experiences help to form and build cognition.

Recently, Ladendorf et al. (2019) proposed the Hypothetical Model of Immersive Cognition (HMIC), which brings together the Information Processing theory (Atkinson & Shiffrin, 1968), the Situated Cognition theory, and the Embodied Cognition theory. MVR connects multiple stimuli across the sensory register, activates cross-channel connections, and fully immerses learners in a 3D world. This creates a sense of presence, tricking the brain into believing that physical stimuli is present. A strong sense of presence is crucial for two reasons. First, it can enable direct interaction between embodied memories in long-term memory (LTM) and the sensory registers, creating a bypass of the working memory. This ensures better activation of LTM and a deeper learning experience (Pan et al., 2017) (Figure 1). Second, the sense of presence created by MVR can reduce the perceived distance and stimulate the brain to activate the schema, which streamlines the cognitive path and enhances the learning experience by processing objects in a more thorough and complex manner (Huang et al., 2020). Ladendorf et al.'s (2019) HMIC is adopted to provide the theoretical foundation for the application of MVR in this study because it has the advantages of the Information Processing theory, Situated Cognition theory, and Embodied Cognition theory combined.

Figure 1.

Information Processing in an MVR-Assisted Learning Environment.

MVR-Assisted Language Learning Research

MVR-assisted language learning has attracted the attention of L2/FL researchers due to its potential to bring a whole new dimension to the realm of language education (Alizadeh, 2019; Fransson et al., 2020; Lan, 2020). For example, Alfadil (2020) examined the effects of MVR on ESL vocabulary learning. Sixty-four intermediate school students in Saudi Arabia were recruited. The experimental group used the VR app House of Languages to learn English vocabulary via Samsung Gear VR. The control group was taught using a traditional ESL vocabulary learning method. The results showed that the experimental group performed better than the control group and had positive attitudes towards MVR-assisted language learning. Similar to Alfadil’s (2020) findings, Tai et al. (2020) also found MVR to have positive impact on motivation and vocabulary learning. Additionally, Xie et al. (2019) provided Chinese L2 students with oral practice using Google Cardboard and Expeditions. Twelve students role-played tour guides in six locations for one semester. The findings revealed that the students’ vocabulary and the contents of their oral presentations were significantly enhanced due to the use of MVR tools.

The aforementioned studies indicate that MVR is motivational and benefits L2/FL learning. Nevertheless, other researchers (e.g., Bailey et al., 2016; Huang et al., 2020) point out that the plausibility in a VR learning environment can promote learner motivation; however, more stimulating immersive VR experiences require more mental effort and, which can cause cognitive and physical overload and thus reduce learning performance. For example, Dolgunsöz et al. (2018) examined the effects of MVR (i.e., Samsung Gear VR with Samsung S7 Edge Mobile Phone) on developing EFL writing skills. They compared Samsung Gear VR usage against viewing content on 2D traditional video in different time periods. The results indicated that the participants enjoyed the MVR-assisted writing experience, but this did not lead to improved writing performance.

Based on the literature reviewed above, as Lan (2020) and Radianti et al. (2020) indicate, MVR in language education is not yet a fully-fledged field of research. Empirical evidence supporting specific L2 learning benefits of MVR is still limited, focusing mainly on motivation and vocabulary learning. As such, research has yet to extrapolate its effects on EFL listening comprehension. In particular, some researchers (e.g., Cheng et al., 2017; Huang et al., 2020; Ladendorf et al., 2019) indicate that highly immersive virtual environments help learners achieve a high sense of presence, which improves learning outcomes. However, other researchers (Huang et al., 2020; Hu et al., 2017) suggest that in the virtual learning environment, the sense of presence takes up learners’ attention, resulting in an increase in cognitive load, and thus hinders learning. Due to the inconsistency between the sense of presence and learning and its absence in related listening studies, it is crucial to explore the influence of sense of presence on learning in immersive virtual environments. The current study was thus motivated by a lack of empirical research regarding the application of MVR, particularly regarding the effectiveness of virtual presence in EFL listening. In addition, the study focuses on an under-represented learner group—junior high school EFL students. To achieve this goal, three research questions were addressed,

Does MVR significantly improve listening comprehension and retention in adolescent EFL learners?

How does MVR facilitate adolescent EFL learners’ listening comprehension, particularly regarding the effects of virtual presence?

What are EFL learners’ perceptions of using MVR for English listening comprehension?

Method

Research Design

This study investigated the impact of MVR on adolescent EFL learners’ listening comprehension, particularly regarding the effects of virtual presence. The participants were randomly assigned to either an experimental (MVR player) or control (video watcher) group. The MVR players played the language learning VR app Mondly, using mobile-rendered HMDs, giving them a stereoscopic view of the virtual environment. The spatial audio, high-caliber 3D graphics, and movement of the virtual characters provided visual immersion, allowing the learners a feeling of “being there” in the virtual environment. In contrast, the video watchers watched a walkthrough video of the VR app Mondly. This was identical in content but had a different presentation mode as it was watched on a standard PC screen. The walkthrough video could be paused and replayed.

Participants

Seventy-two EFL eighth graders (38 males, 34 females, aged 13–15) were recruited in Northern Taiwan. All had received parental approvals before participating. They were all native Chinese speakers and had undergone 7–8 years of formal English education. According to their performance on five monthly English exams, they had similar levels of language proficiency (t =.29, p = .97) and listening comprehension ability (t = .13, p = .73). Audio materials delivered via a CD player were the main stimulus for listening comprehension in their regular English class and none of them had any previous experience of using MVR for language learning.

Instruments

The instruments used in this study were learning devices (i.e., Samsung Gear VR, a mobile phone, and a PC), a language learning VR app, a listening comprehension test, free recall of learning contents, questionnaires, and interview.

Learning Devices and Materials

The Samsung Gear VR and Samsung Galaxy Note 8 were used as the learning devices for the MVR players. A personal computer was used for the video watchers. A foreign-language learning VR app, Mondly: Learn Languages in VR, produced by Ati Studios (http://www.mondlylanguages.com), was selected for this study due to two distinguishing characteristics: the high fidelity of its representation (e.g., realistic display of the virtual environment and spatial audio) and a high degree of interactivity (e.g., embodied actions including view control and navigation and verbal and non-verbal communication), which might lead to high degree of immersion, making learners feel as if they are “there” with the virtual characters. As detailed in Table 1, five simulated real-life scenarios (i.e., Train Station, Department Store, Hospital, Hotel Reception, and Restaurant) were selected for the study. Each scenario involved one communicative activity. The topics and contents were appropriate for the participants’ age and proficiency level.

Table 1.

Five Listening Activities in the Study.

Scenario
Scenario	Train station	Department store	Hospital	Hotel reception	Restaurant
Topic	Buy tickets	Go shopping	Make an appointment	Check into a hotel	Order dinner
Duration
VR	4′45′′	4′40′′	5′00′′	6′30′′	7′02′′
Video	3′54′′	3′45′′	4′04′′	5′42′′	6′17′′
Word token	105	108	102	203	233

The automatic speech recognition (ASR) system enables learners to interact with virtual characters and receive immediate feedback. During conversations, learners are prompted with responses to what the virtual characters say, and every answer learners give is transcribed on the screen. The app listens to the learners’ words, analyzes the accuracy of their pronunciation, and gives immediate feedback. For example, virtual characters nod if an answer is correct and a green check mark will hover over the transcription of what they have just said (Figure 2). For incorrect or unclear answers, users receive linguistic or kinesics signals (e.g., gesturing or staring).

Figure 2.

Example Interaction With a Virtual Character.

Listening Comprehension Test

To gauge the participants’ listening comprehension, a listening comprehension test (online Appendix A) was administered after each listening activity. The listening skills tested involved (1) deducing meaning from context, (2) detecting main and supporting ideas, and (3) understanding linguistic information and making inferences from the content of the text. Two experienced EFL teachers and a native English teacher were asked to rate the test validity based on the relevance and appropriateness of the listening materials. Each correctly answered multiple-choice question scored 1 point. To ensure the quality of the spoken message, the test items were pre-recorded by native English teachers. The question prompt was in spoken form. The speech rate of the spoken message was approximately 130 words per minute.

Free Recall of Learning Contents

Free recall section, which took place immediate after the intervention, was designed to assess the participant’s comprehension and retention of the listening content. Each participant was asked to write down as many idea units as possible from the learning content. They could write in English or Chinese. Recall protocols were independently scored by two experienced junior high school English teachers based on two criteria: (1) the total number of correct idea units given, and (2) the number of main ideas and details given. Each idea entry was scored 1 point if the entry was accepted by two raters

Presence Questionnaire

A presence questionnaire (online Appendix B) was used to survey the MVR players’ sense of presence while playing the VR app Mondly. This questionnaire was adapted from Silva et al. (2016) and Witmer et al. (2005) presence questionnaires. Four factors involved in presence are covered in the questionnaire, with a total of 18 closed-ended items: (1) involvement, measuring the amount of attention devoted to the virtual environment; (2) immersion, referring to a sense of being enveloped; (3) sensory fidelity, related to the subjective experience of realism of VR scenarios; and (4) Interface quality, evaluating the influence of visual and control interfaces in the virtual environment. The participants were asked to rate the items on a 5-point Likert scale, anchored by the end points strongly disagree (1) and strongly agree (5). The estimated Cronbach’s α coefficient was 0.87.

Semi-Structured Interviews

Semi-structured interviews were conducted with the MVR players individually to obtain their opinions on the value of VR as a listening tool. The interviews were conducted in their native language (Mandarin Chinese) and lasted for approximately 10 minutes.

Procedure

As shown in Figure 3, each participant completed a demographic questionnaire 2 weeks before the intervention. At the onset of the VR app listening session, the participants were given a brief orientation about the activities to be completed. The MVR players were given instructions on how to use the Gear VR and play the VR app. Next, the VR app listening activity was individually administered to each participant by one of the researchers in the school’s computer laboratory. There are five play sessions, each taking approximately 5-7 minutes. The participants had a 5-minute break between sessions. The duration of the experiment was approximately 50-60 minutes to complete. A listening comprehension test and recall were administered to each participant immediately after each listening activity. A presence questionnaire and an interview were conducted with each MVR player by the same researcher immediately followed the free recall.

Figure 3.

Experimental Procedure of the Study.

Data Analysis

Both quantitative and qualitative data were collected. For the quantitative data, a listening comprehension test was administered immediately after each listening session. One presence questionnaire and a semi-structured interview were used to investigate the MVR players’ sense of presence and learning experience in the Mondly virtual learning environment. An independent-samples t test was calculated to compare the MVR players’ and video watchers’ English proficiency level and listening comprehension ability in five monthly English exams prior to the intervention to determine any initial differences between the two groups. The English listening comprehension test scores were analyzed using independent-samples t test. Interview data were analyzed and coded. Codes with common themes or similar meanings were categorized together. The frequency and percentage of each category were calculated for analyses of participants’ perceptions. Findings from the interview data were used to triangulate and cross-validate the quantitative results obtained from the listening comprehension test.

Results

Listening Comprehension Test

The first research question concerns the impact of MVR on EFL learners’ listening comprehension. The results shown in Table 2 indicate that a significant difference (t = 2.13, p = .04, d = .50) in listening comprehension between the MVR players (M = 7.75, SD = 2.21) and the video watchers (M = 6.42, SD = 3.04). The MVR players performed much better.

Table 2.

Results of the Independent Samples t-Test on the Participants’ Listening Comprehension.

Variable	Group	N	Mean	SD	t	p	d
Listening test	MVR	36	7.75	2.21	2.13	.04	.50
	Video	36	6.42	3.04

p < .05.

Free Recall of Learning Contents

As demonstrated in Table 3, a significant difference was noted between the VR players (M = 14.11, SD = 8.86) and the video watchers (M = 5.36, SD = 6.12) in recall of idea units (t = 4.88, p = .00, d = 1.15). The VR players outperformed the video watchers in listening comprehension and retention.

Table 3.

Results of Independent Samples t-Test of Free Recall between Two Groups.

Variable	Group	N	Mean	SD	t	p	d
Recall	MVR	36	14.11	8.86	4.88	.00	1.15
	Video	36	5.36	6.12

***p < .001.

Specifically, as detailed in Table 4, the MVR players demonstrated significantly better recall of the main ideas (t = 4.29, p = .00, d = .01) and details (t = 2.62, p = .01, d = .62) than the video watchers. Furthermore, the results reveal that the MVR players recalled more main ideas than details (M = 7.64, SD = 8.08 for main ideas; M = 6.25, SD = 4.52 for details). This is illustrated in Figure 4. However, the video watchers recalled more details than main ideas (M = 1.33, SD = 3.51; M = 3.67, SD = 3.82).

Table 4.

Results of the Independent t-Test Regarding Recall of Main Ideas and Details.

		MVR players		Video watchers
		Mean	SD	Mean	SD	t	p	d
Recall	Main ideas	7.64	8.08	1.33	3.51	4.29	.00	.01
	Detail	6.25	4.52	3.67	3.82	2.62	.01	.62

***

p < .001.

Figure 4.

Results Regarding the Mean Scores of the Participants’ Recall of Main Ideas and Details.

The Presence Questionnaire

As shown in Table 5, the average values of the four factors involved in presence were greater than 4.00 for the MVR players. Sensory fidelity was the most highly valued factor (M = 4.15; SD = .56), while involvement (M = 4.11; SD = .59) came second, followed by interface quality (M = 4.08; SD = .61), and finally immersion (M = 4.04; SD = .60).

Table 5.

MVR Players’ Responses to the Presence Questionnaire.

Items	Score
Immersion	4.04
1. I adjusted to the virtual environment experience quickly.	4.00
2. I could concentrate on the listening task and virtual environment rather than on the mechanisms used to perform those tasks or activities.	4.17
3. My senses were engaged in this experience completely.	4.00
4. I adjusted to the control devices used to interact with the virtual environment easily.	4.11
5. The information provided through different senses in the virtual environment was consistent.	3.94
Involvement	4.11
6. My sense of objects moving through space was very compelling.	4.63
7. My experiences in the virtual environment seem consistent with my real world experiences.	4.08
8. I am able to actively survey or search the environment.	4.19
9. I could move or manipulate objects in the virtual environment very well.	4.19
10. I was involved in the virtual environment experience very well.	4.08
Interface quality	4.08
11. I experienced no delay between actions and expected outcomes.	4.14
12. The visual display quality never distract me from required activities.	4.03
13. The control devices never interfere with listening activities.	4.14
Sensory Fidelity	4.15
14. The auditory aspects of the environment involved me very much.	4.11
15. I could localize sounds very well.	4.31
16. I was able to examine objects very closely.	4.03
17. I could examine objects from multiple viewpoints very well.	4.19
18. It seems to be in a real place, able to sense and interact with the environment.	4.00

Regarding sensory fidelity, a majority of the MVR players found the virtual environment in Mondly to be a real place. They could interact with the environment (M = 4.00), examine objects from multiple viewpoints (M = 4.19), and localize sounds well (M = 4.31). As for the MVR players’ involvement, the objects moving through space was found to be very compelling (M = 4.63). They could manipulate such objects (M = 4.19) and explore the environment actively (M = 4.19). Regarding interface quality, the VR players experienced no delays between taking action and experiencing the expected outcomes (M = 4.14). The visual display quality (M = 4.03) and control devices (M = 4.14) did not distract the MVR players from listening activities. Being immersed in the virtual environment, they reported that they were able to concentrate on the listening tasks (M = 4.17) and felt completely engaged in the listening process (M = 4.11).

Interviews With the MVR Players

Table 6 shows the MVR players’ perceptions of MVR-assisted English listening.

Table 6.

Summary of the MVR Players’ Responses to Interview Questions (N = 36).

Category	N	%
Motivation
Yes
Interesting, motivating, beneficial (e.g., realism, presence)	29	80.56
Reduced listening anxiety	18	50.00
Stereoscopic views of scenes	17	47.22
No
Lack of diversity in interaction (e.g., linear order)	2	5.56
Not beneficial for English listening tests	2	5.56
Neutral	1	2.78
Perceived Usefulness of VR
Yes
Simulated real-life scenarios	28	77.78
A sense of presence	26	72.22
Immersive 3D virtual environment	21	58.33
Interactive listening (first-person perspective) and instant feedback	17	47.22
Multisensory input and learning support	17	47.22
No
Distracted attention	2	5.56
Too much functionality	2	5.56
Neutral	3	8.33
Perceived Usability of VR
Easy to use	27	75.00
Mobility (e.g., portable, flexible, and convenient)	16	44.44
Technical problems
ASR system	8	22.22
Immediacy of interaction	2	5.56
Simultaneous manipulation of multiple stimuli	2	5.56

Four major themes emerged through qualitative data analysis.

Theme 1: The high fidelity of its representation sparked interest in EFL listening. The realism and spatial audio made listening more motivating and interesting for the majority of the MVR players (77.78%). Virtual presence was also a critical factor affecting the MVR players’ enjoyment (72.22%). When they had the sense that they were in the same place as the virtual characters and were participants in the interaction, they felt excited and engaged in the listening task. The following was derived from one interview:

我喜歡, 有臨場感, 感覺自己在現場。很像真的看著一個人, 和她聊天, 很好玩!讓我更投入虛擬世界裡。

(MVR player 71210)

I liked (MVR) because it provided a sense of presence, just like being there. I felt like I was ’looking’ at a person at eye-level. It was interesting to talk with her! I was more engaged within the virtual world.

Theme 2: Presence and real-time interactivity in MVR prevented cognitive overload and enhanced listening comprehension. Regarding the usefulness of MVR for listening comprehension, over 70% of the MVR players appreciated the simulated real-life virtual environment, which helped them make appropriate inferences, instead of wild guessing. For example,

以前, 聽力的時候, 我會試著聽懂每個單字再去想像情境。但是, 一旦我沒猜對就完了。它提供了情境。主題也很清楚。一開始方向大概正確, 從情境可以猜到大概意思。VR讓我壓力(怕猜錯)和負擔小一點。

(MVR player 70728)

In the past, when listening, I tried to understand every word and then imagined the context. However, once I failed, everything was over. Mondly provided the context. The topic of each scenario was clear to me. I could start with the right direction. I didn’t have to worry about going in the wrong direction. As for the word meaning, it was not wild guessing but based on contextual cues. VR really helped reduce my cognitive load in listening comprehension.

Furthermore, the immersive virtual environment focused learners’ attention and energy on the virtual environment stimuli, as one MVR player stated,

能夠不受外界干擾, 增加專注度, 學習力集中。

(MVR player 70736)

Being fully immersed in the VR world, I was able to immune to outside interference. It helped me concentrate on listening and thus enhance comprehension.

Specifically, the sense of “being there” with virtual characters was regarded as a crucial factor enhancing listening comprehension (72.22%). Some MVR players expressed amazement at “how realistic it was,” noting that “it felt like I was there.” The following responses were an indication that the MVR players had a sense of being present, acting in a place other than their real environment.

哇…好可怕!走開!走開!不要一直看我嘛!你好醜, 不要一直看我!

(MVR player 70729)

Oh…It is so terrible! Go away! Go away! Don’t stare at me all the time! You’re so ugly! Don’t stare at me all the time!

我感覺真的在裡面, 身歷其境, 增加專注度。例如計程車那一幕, 很像真的車, 我坐在車子裡, 可以感覺車子在動, 從後照鏡可以看到司機, 從車窗可以看到外面, 不過裡面有點擠!

(MVR player 71236)

For example, in the scenario “Taxi”, the taxi was like a real one. I felt I really sat inside a car, having a strong sense of immersion, which increased concentration. I could feel the car moving. I could see the driver from the rear view mirror. I really could see outside from the car window. However, it was a little bit crowded.

Furthermore, perhaps the best indication of virtual presence came in the form of deictic expressions, such as I, me, we, here, and now, used by the MVR players’. They seemed to regard themselves as participants when interacting with virtual characters and recalled communicative tasks from a first-person perspective. This contrasts with the video watchers who never used I when recalling their experiences, using the third-person pronouns, he, she, or they frequently instead, followed by the second person pronoun you. The following extracts are taken from the recall data:

有個服務生問我有關房間的問題。問我是否要幫助?問我是否要換房間?

(MVR player 71231, Scenario 4)

A waiter asked me some questions about the room. He asked me whether I needed help and changed the room.

他們在講他有沒有要幫忙?他想要換房間所以他要求服務生幫他。

(Video watcher 70829, Scenario 4)

They were talking about whether he (the customer) needed help. He wanted to change rooms so he asked the waiter to help him .

服務生叫我點餐, 後來另桌還有一個女生要跟我對話。

(MVR player 71231, Scenario 5)

The waitress asked me to order some food. Then, a female customer talking to me .

他們在講他要點什麼, 還有他和一個也在店裡的女生聊天。

(Video watcher 70808, Scenario 5)

They talked about the food he would like to order. Also, he talked with a girl in the restaurant.

In brief, virtual presence allows for a first-person view of the environment. As a participant, the MVR players’ real-time interactivity with the virtual characters in the immersive virtual environment facilitated listening comprehension.

Theme 4: Easy to use and intention to use MVR. Regarding the usability of VR, 75% of the MVR players perceived the mobile-rendered HMDs as being easy to use and control. Over 44% of the MVR players expressed their preference for the portability of MVR, allowing for flexible and convenient learning. They hoped to use MVR to practice English in the future, if possible. For example, one MVR player stated,

因為攜帶方便, 很適合課堂上使用。例如介紹外國時, 一戴上VR, 彷彿就到了那個國家和當地的人聊天, 很方便。

(MVR player 70707)

MVR is very suitable for the English class because of its portability. For example, it is very convenient for the teacher to use mobile VR to introduce a foreign country. It would give students a sense of “being in that country” and “having a talk with people there”.

Nevertheless, several participants perceived some difficulties with the MVR tools. Challenges arose mainly from the ASR system. They (22.22%) experienced glitches where they had to repeat themselves. Another challenge reported by two MVR players was the immediacy of interaction. They were unable to manipulate so many stimuli simultaneously, which hindered their sense of presence and comprehension. For example,

我的英文不是很好。同時要聽懂, 又要回答, 又要看字, 我也很好奇虛擬實境裡的東西, 想看一下, 但真的來不及。有些字, 我不知道怎麼唸, 我要重覆聽好幾次。雖然有提供字幕, 我卻來不及讀完。

(MVR player 71230)

I am not good at English. I had to listen, respond, and read the caption simultaneously. Also, I was curious about the objects in the virtual world. I would like to take a look at them. There’s not enough time. I didn’t know how to say some words. I had to listen to these words again and again. Although it (Mondly) provided caption, I didn’t have enough time to read it thoroughly.

To sum up, MVR-assisted EFL listening was effective due to its situational authenticity, a strong sense of immersion and presence, dynamic interaction with virtual characters and immediate feedbacks, and mobility. However, MVR was not beneficial and easy for all of the MVR players to use.

Discussion

The first research question concerned listening comprehension with the aid of a VR app via mobile-rendered HMDs. The results revealed that the MVR players significantly outperformed the video watchers on listening comprehension. It is also important to note that the MVR players and video watchers differed significantly regarding recall with the MVR players demonstrating better comprehension and retention. MVR seemed to help the players recall more of the main ideas and with greater clarity. However, the video viewers recalled more details than main ideas. The findings were in line with Lan and Liao’s (2018) study. They observed that the listening lesson in the immersive VR context was enjoyable for L2 learners and had facilitated listening comprehension.

The second research question asked how MVR would facilitate listening comprehension, particularly regarding the effects of virtual presence. The interview data revealed that most MVR players appreciated the fidelity of the representation (i.e., the realistic environment and spatial audio) and interactivity (e.g., embodied actions which included view control and navigation), which led to a high degree of immersion and consequently, a sense of presence. The here-and-now orientation that allowed the MVR players to make use of context to interpret the meaning of utterances enhanced listening comprehension. Furthermore, interacting with virtual characters helped the players develop feeling of “being there” within the virtual environment and “being participants” in interactions. This excited the players, helped engage them in the listening process and thus aid comprehension (Rost, 2016; Vandergrift & Baker, 2018). The findings corroborate Ladendorf et al. (2019, p.2) study that a sense of presence can enhance learning. Nevertheless, the findings provide evidence in direct contrast to Huang et al. (2020) and Hu et al.’s (2017) studies that a high sense of presence in an immersive VR learning environment can cause learners’ motivation; however, it may distract learners’ attention, generate cognitive overload and thus hinder learning.

The third research question concerned the participants’ perceptions of MVR-assisted EFL listening. The majority of the MVR players regarded the use of MVR for listening comprehension motivating, engaging, and effective. The presence, participation, interactivity, and mobility provided by MVR are found to be the main features that affect players’ positive attitudes. The findings are consistent with previous research which found that MVR-mediated learning environments elevate learner motivation and interest (Alfadil, 2020; Tai et al., 2020; Xie et al., 2019). However, not all participants made favorable comments on the value of MVR for listening comprehension. Several of the MVR players reported that the different stimuli distracted them and that they were unable to manipulate so much functionality simultaneously. This impeded listening comprehension. The unfavorable comments echoed problems caused by distractions and cognitive overloading mentioned in previous research (Bailey et al., 2016; Dolgunsöz et al., 2018; Huang et al., 2020). This finding suggests that while choosing MVR apps, instructors carefully consider the degree and type of interaction required from learners to avoid overwhelming them, disrupting the sense of presence and thus having a negative impact on learning.

Conclusion

The purpose of this study was to investigate the impact of MVR, especially regarding presence, its unique feature, on adolescent EFL learners’ listening comprehension and their perceptions of MVR as applied to EFL listening. The results indicate that MVR facilitated EFL learners’ listening comprehension and retention. The MVR players’ listening efficiency might have been due to the main variables involved in presence offered by the platform. Namely, a high fidelity of representation, interaction with virtual characters, and a first-person perspective for examining objects and exploring the virtual environment (Slater, 2018). This finding is suggestive of the facilitative effect of virtual presence on EFL listening comprehension. The qualitative analyses of the participants’ perceptions further revealed that a majority of the MVR players regarded MVR-assisted EFL listening as motivating, beneficial, and good preparation for real-world communication. More importantly, virtual presence prevented cognitive overload, reduced listening anxiety, and thus enhanced listening comprehension.

From a pedagogical perspective, the study supports the integration of MVR in EFL learning. Scenarios available on interactive MVR platforms offer readily assessable teaching materials in authentic contexts and provided a sense of presence in the target language community. This is especially important for EFL learners who lack access to authentic English learning environments. Furthermore, the study can help educators make informed decision on appropriate priorities based on their teaching goals through MVR-assisted instruction.

Nevertheless, this study was limited by its short intervention time, small-scale sample of participants, and limited selection of MVR apps. To provide more substantial evidence, it is recommended that future studies be longitudinal, undertaken with larger, more diverse samples and incorporate various types of MVR apps. Furthermore, future research can hopefully investigate the development of listening strategies in MVR-based learning environments and the transfer of such strategies to real-world communication.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Howard Hao-Jan Chen

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Tzu Yu Tai is an assistant professor in the Center of Language Education, National Tsing Hua University. She is also a teacher in Taipei Municipal Hong Dao Junior High School. Her research interests include CALL and TESOL.

Howard Hao-Jan Chen is a professor in the Department of English, National Taiwan Normal University. He is also the president of English Teaching and Research Association (ETRA). His research interests include SLA, Technology-Assisted Language Learning (e.g., VR, AI, CALL, mobile learning, and digital game-based learning), and Corpus Linguistics.

References

Alfadil

(2020). Effectiveness of virtual reality game in foreign language vocabulary acquisition. Computers and Education, 153, 1–13.

Alizadeh

(2019). Virtual reality in the language classroom: Theory and practice. CALL-EJ, 20(3), 21–30.

Atkinson

R. C.

Shiffrin

R. M.

(1968). Human memory: A proposed system and its control processes. In Spence

K. W.

Spence

J. T.

(Eds.), The psychology of learning and motivation: Advances in research and theory (pp. 89–105). Academic Press.

Bailey

J. O.

Bailenson

J. N.

Casasanto

(2016). When does virtual embodiment change our minds? Presence: Teleoperators and Virtual Environments, 25(3), 222–233.

Brown

J. S.

Collins

Duguid

(1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32–42.

Chen

C. W.

(2019). Guided listening with listening journals and curated materials: A metacognitive approach. Innovation in Language Learning and Teaching, 13(2), 133–146.

Cheng

Yang

Andersen

(2017). Teaching language and culture with a VR game. In Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems (pp. 541–549). https://dl.acm.org/citation.cfm?id=3025857

Dolgunsöz

Yıldırım

(2018). The effect of virtual reality on EFL writing performance. Journal of Language and Linguistic Studies, 14(1), 278–292.

Fransson

Holmberg

Westelius

(2020). The challenges of using head mounted virtual reality in K-12 schools from a teacher perspective. Education and Information Technologies, 25(4), 3383–3404. https://doi.org/10.1007/s10639-020-10119-1

10.

P. J.

H. F.

Fang

(2017). Examining the mediating roles of cognitive load and performance outcomes in user satisfaction with a website: A field quasi-experiment. MIS Quarterly, 41(3), 975–987.

11.

Huang

C. L.

Luo

Y. F.

Yang

S. C.

C. M.

Chen

A. S.

(2020). Influence of students’ learning style, sense of presence, and cognitive load on learning outcomes in an immersive virtual reality learning environment. Journal of Educational Computing Research, 58(3), 596–615.

12.

Jiang

Dewaele

J. M.

(2019). How unique is the foreign language classroom enjoyment and anxiety of Chinese EFL learners? System, 82, 13–25.

13.

Kaplan-Rakowski

Gruber

(2019). Low-immersion versus high-immersion virtual reality: Definitions, classification, and examples with a foreign language focus. In Proceedings of the 12th international conference innovation in language learning (pp. 552–555). Filodiritto Editore.

14.

Ladendorf

Schneider

Xie

(2019). Mobile-based virtual reality: Why and how does it support learning? In Zhang

Cristol

(Eds.), Handbook of mobile teaching and learning (pp. 1–19). Springer.

15.

Lan

Y. J.

(2016). The essential components of game design in 3D virtual worlds: From a language learning perspective. Learning, Design, and Technology, 1, 1–18.

16.

Lan

Y. J.

(2020). Immersion, interaction and experience-oriented learning: Bringing virtual reality into FL learning. Language Learning & Technology, 24(1), 1–15. http://hdl.handle.net/10125/44704.

17.

Lan

Y. J.

Liao

C. Y.

(2018). The effects of 3D immersion on CSL students’ listening comprehension. Innovation in Language Learning and Teaching, 12(1), 35–46.

18.

Lan

Y. J.

Lyu

B. N.

Chin

C. K.

(2019). Does 3D immersive experience enhance mandarin writing by CSL students? Language Learning & Technology, 23(2), 125–144.

19.

Liao

(2018). Exploring the affordances of telepresence robots in foreign language learning. Language Learning & Technology, 22(3), 20–32. http://doi.org/10125/44652.

20.

Pan

Bingham

(2017). Embodied memory allows accurate and stable perception of hidden objects despite orientation change. Journal of Experimental Psychology. Human Perception and Performance, 43(7), 1343–1358.

21.

Radianti

Majchrzak

T. A.

Fromm

Wohlgenannt

(2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147, 103778–103729.

22.

Rost

(2016). Teaching and researching listening (3rd ed.). Routledge.

23.

Silva

G. R.

Donat

J. C.

Rigoli

M. M.

de Oliveira

F. R.

Kristensen

C. H.

(2016). A questionnaire for measuring presence in virtual environments: Factor analysis of the presence questionnaire and adaptation into Brazlian Portuguese. Virtual Reality, 20(4), 237–242.

24.

Slater

(2018). Immersion and the illusion of presence in virtual reality. British Journal of Psychology (London, England : 1953), 109(3), 431–433.

25.

Stojšić

Ivkov-Džigurski

Maričić

(2019). Virtual reality as a learning tool: How and where to start with immersive teaching. In Daniela

(Ed.), Didactics of smart pedagogy (pp. 353–369). Springer.

26.

Tai

T. Y.

Chen

H. J.

Todd

(2020). The impact of a virtual reality app on adolescent EFL learners’ vocabulary learning. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2020.1752735

27.

Vandergrift

Baker

S. C.

(2018). Learner variables important for success in L2 listening comprehension in French immersion classrooms. Canadian Modern Language Review, 74(1), 79–100.

28.

Varela

F. J.

Thompson

Rosch

(2017). The embodied mind: Cognitive science and human experience. MIT Press.

29.

Wang

C. P.

Lan

Y. J.

Tseng

W. T.

Lin

Y. T.

Kao

C. L.

(2020). On the effects of 3D virtual worlds in language learning: A meta-analysis. Computer Assisted Language Learning, 33(8), 891–915. https://doi.org/10.1080/09588221.2019.1598444

30.

Witmer

B. G.

Jerome

Singer

M. J.

(2005). The factors structure of the presence questionnaire. Presence: Teleoperators and Virtual Environments, 14(3), 298–312.

31.

Xie

Chen

Ryder

L. H.

(2019). Effects of using mobile-based virtual reality on Chinese L2 students’ oral proficiency. Computer Assisted Language Learning, 1–21. https://doi.org/10.1080/09588221.2019.1604551