Acceptability of RPG Simulators for Foreign Language Training in Japanese Higher Education

Definition of Terms

The three perceived attributes were defined with reference to Kebritchi’s (2010) findings on teachers’ perceptions of a digital game-based learning tool, and phrased to fit within the scope of the present investigation.

Pre-Existing Knowledge

The literature suggests that the life experiences of educators greatly influence their perceptions of digital games and learning. For instance, perceptions of pedagogical use correlate with an individual’s own use of games (Schrader, Zheng, & Young, 2006). Also, personal interest in related technologies can influence perceptions of educational computer games (Can & Cagiltay, 2006). In addition, negative views of computer games in the broader cultural context may be reflected in teacher opinions of educational computer games (Kenny & McDaniel, 2011). Thus, various contextual and experiential factors influence perceptions of new technological applications.

With regard to Japan, Joshi, Pan, Murakami, and Narayanan (2010) found that, in contrast to American kindergarten teachers, only a small fraction of Japanese counterparts believed that computers and the internet could play an educational role. The authors suggested that this trend may be due to a cultural emphasis on human interaction in education, or an abnegation of mechanically mediated contact. It should be noted that the ability to draw inferences from an analogous relationship between kindergarten and higher education instructors is debatable. However, teachers in Japan are trained in higher education, as is the case in other countries, and their values and beliefs regarding pedagogy may reflect cultural norms in the larger educational context.

Policy Influence and Training

The literature indicates that a well-designed policy intervention, which includes faculty education, has a positive influence on perceptions of technology-supported educational approaches. For instance, Alshumainmeri (2008) found that EFL teachers who had undergone CALL training had more positive attitudes toward the approach, and were much more likely to use it in the classroom. Also, Kenny and McDaniel (2011) found that training was effective in overcoming biases against educational applications of a motion-sensor game. Thus, formal training in the rationale and use of new technologies can positively influence attitudes among educators.

Top-down policies are influential, but may not be successful in implementation. In the backdrop of a program to promote technology integration in the Turkish educational system, Uzunboylu and Ozdamli (2011) found that secondary public school teachers felt positively toward mobile learning and that adequate infrastructure was present. Similarly, Aydin (2013) found that Turkish primary and secondary teachers had a generally positive view of computer application in instruction. In a counter example from Singapore, also in the context of a top-down policy of technology integration, Koh, Kin, Wadhwa, and Lim (2011) found that although most teachers felt educational games are useful in fostering a positive attitude among students with regard to school, most also cited a lack of organizational support for not utilizing them more often. Koh and colleagues attributed this in part to the absence of adequate training programs.

With regard to the Japanese context, top-down policy mandates to promote the adoption and use of learning technologies have largely been ineffective. In 2001, the government began providing funding and guidelines for infrastructure and promoting IT/ICT skill acquisition and the ability to integrate technology and pedagogy (Sakamoto, 2002). However, training and faculty development were neglected in actual implementation (Uchida, 2004). As a result, the infrastructure provided has been widely under-utilized, and pedagogy has remained largely unchanged from conventional instructor-centered approaches (Aoki, 2010; Bachnik, 2003; Latchem et al., 2008). According to a survey on ICT integration in higher education conducted by the Open University of Japan (2011), the emphasis placed on execution is sporadic among institutions, and few have explicit plans for implementation. Thus, it can be assumed that, while organizational interventions may be effective in overcoming biases based in pre-existing knowledge, with the absence of effective policies in Japanese higher education such preconceptions may continue to influence perceptions of RPG Sims.

Professional Networks and Informal Information Sharing

The importance of peer networks is emphasized in Diffusion Theory (DT). According to Rogers (2003), individuals are more likely to adopt an innovation when they observe its use by other individuals perceived as being homophilous, or similar in social status and sharing similar values. Accordingly, Kessler (2007) found that a positive attitude with regard to CALL correlated more closely with informal learning among peers than formal training in degree programs. That is, knowledge sharing among peers has a considerable effect on perceptions of new technologies. The influence of homophilous groups is particularly important to understand in the Japanese context because a large portion (up to 45%) of FL educators in higher education is non-Japanese (Yonezawa & Ishida, 2012). Some scholarly work indicates the presence of dichotomies based on nationality and/or work status in higher education (Cowie, 2011; McVeigh, 2002; Whitsed & Volet, 2010; Whitsed & Wright, 2011). Further, these divisions may influence professional networking (Yonezawa & Ishida, 2012). Thus, cultural views influencing the perception of RPG Sims may be delineated along lines of cultural background.

Marginalization of CALL

The treatment of CALL materials in general is relevant to understanding how FL educators may perceive RPG/Sims because of the DT construct of technology cluster (Rogers, 2003, p. 249), which predicts that potential adopters will treat innovative technologies as they would familiar technologies, which are perceived as being related. If RPG Sims are perceived as being related to CALL, then variation may exist in perceptions based on existing research interests or pedagogical approaches.

The field of FL education appears divided between a minority who embrace and a majority who eschew CALL, and this dichotomy may be reflected in the educator community in Japan. Computer-mediated learning materials in general have never exhibited widespread popularity because, as Bax (2003) argues, the technology has failed to meet exaggerated expectations, and so has failed to realize a mainstream status. This situation persists (Bax, 2011), because in the minds of most FL educators, CALL is synonymous with “drill” software (Bush, 2008). Accordingly, Kim (2008) found that preservice EFL/ESL instructors tended to favor a teacher-centered classroom approach and marginalize CALL as supplementary material. The prevalence of this perception in Japan is indicated in the consignment of computer-supported learning systems to language labs as self-study material (Brown, Campbell, & Weatherford, 2008). At the same time, a CALL research community does exist in the form of academic societies such as the Japan Association of Language Teachers and CALL (JALTCALL), and the Japanese Association of Language Education and Technology (LET). The concept of technology cluster predicts that RPG Sims would be more acceptable to members of such academic societies, or educators with similar research interests.

Summation and Research Questions

In sum, it would be reasonable to infer from the literature that variation in the acceptability of RPG Sims may exist based on demographic traits. Furthermore, variation in perceptions may be based in experience with or interest in CALL approaches to FL education. To further the analysis of FL educator perceptions of RPG Sims in Japanese higher education, the present study adopted the following research questions to guide data collection.

Sources of Data

The target population for the present study was FL instructors working in higher education in Japan. A sample of convenience was employed in the collection of both quantitative and qualitative data.

In the collection of quantitative data, FL instructors were solicited via email to participate in an online self-completion questionnaire. This type of instrument was chosen based on the availability of resources and the assumption that an electronic survey would not significantly limit participation for the following reasons. Japan is a technologically advanced country, the target population consists of highly educated academic professions, and IT infrastructure for general use purposes (i.e., document sharing, email, word processing, etc.) is for the most part ubiquitous in Japanese higher education (Open University of Japan, 2011). Thus, the principal investigator compiled a direct mailing list of FL faculty members at Japanese institutions of higher education using the Google search engine. Email addresses were recorded from faculty profile pages only when an examination of the instructor’s profile indicated that they were engaged in FL education. The examination of cases resulted in a list of 850 individual email addresses.

In the gathering of qualitative data, snowball sampling was used to recruit informants for individual asynchronous email interviews from the principal investigator’s professional network. A total of seven informants participated in the interviews. Four of these participants also participated in the online self-completion questionnaire.

Data Collection Strategies and Procedures

Acceptability research involves first providing an example of an innovation, then collecting data on perceptions of its attributes. Hence, the online self-completion questionnaire was comprised of a brief video presentation introducing a state-of-the-art RPG Sim for FL and cultural training, and a questionnaire eliciting opinions thereof along with demographic information.

Video Permission was obtained to use the ISLET (Integrated System for Language Education and Training) promotional video produced by Alelo Inc (2013) (Figure 1). The ISLET video was chosen primarily because it is designed for promotional purposes to provide a brief but informative overview for an audience that is likely unfamiliar with RPG Sim solutions for FL and cultural instruction.

OPEN IN VIEWER

Figure 1.

Screenshot of ISLET video.

Questionnaire The self-completion questionnaire contained 10-items (Appendix A). Three items were designed to gather data on the perceptions of relative advantage, compatibility and complexity, and followed a five-point Likert response format. Five items elicited demographic information of job title, rank, term of office, nationality, research interest and teaching objectives. The questionnaire contained an additional item consisting of a text field where respondents were invited to comment freely.

Survey instrument The researcher created two versions of the questionnaire, one in Japanese and one in English. The two versions along with matching videos, one dubbed in Japanese and one in English, were embedded in two separate webpages, and a parent page was created to contain links to both Japanese and English survey instruments. An email soliciting participation and containing a link to the parent page was sent to addresses on the direct mailing list, and recipients were invited to click on the link and select the language in which they chose to respond.

Individual interviews Interviews were conducted asynchronously via email. These interviews involved providing informants with a link to the ISLET video, and having them write responses to three writing prompts designed to frame the discussion in the three perceived attributes (see Appendix B). Participants were asked to respond to each prompt with at least a paragraph. Following initial responses, the researcher followed up when necessary with requests for elaboration and/or clarification.

The Analysis Pertaining to Research Question 1

To investigate the perceived attributes of RPG Sims, response frequencies for items 1-3 were summarized in bar charts, and a content analysis was performed on the qualitative data following the procedure laid out by Saldaña (2009). Specifically, the text was coded first according to comments as they related to relative advantage, compatibility and complexity. These comments were further sub-categorized based on stance (positive, negative or neutral) and reasons explaining the stance if any were provided. For example, Informant 1’s comment, “If [the instructor] must know everything including the content, then it couldn’t be used because it would take too much time,” was coded as: complexity-neut, (instructor burden). Following Saldaña (2009), the sub-categories were refined in an iterative process involving defining and re-labeling. Emergent themes are summarized in Table 1 with definitions.

OPEN IN VIEWER

Table 1.

Categories With Definitions and Number of Occurrences.

Category	Definition	Occurrences
Administration	The set of organizational conditions under which an instructor must work, including the specific role that the instructor fulfills	10
Cost	The financial burden born by instructor, learner or institution	16
Infrastructure	The availability and type of physical equipment used for RPG Sims	17
Instructor burden	The cost in time and effort for the instructor to implement RPG Sims	15
Instructor literacy	The level of technical literacy among instructors	9
Learner attitude	Values, beliefs and norms among the learners	23
Learner motivation	The engaging effect of RPG Sims on learners	18
Learning objective	Skills or knowledge to be acquired in a particular course of study	28
Learning practice	The behaviors, activities or sets of conditions that learners participate in to facilitate learning	32
Utilization	The appropriate role that RPG Sims should fulfill in a course	14

These themes were examined in conjunction with responses to items 1-3, which elicited the three perceived attributes of innovation proposed by Rogers (2003), and relationships that offered explanations of response trends were noted.

Relative Advantage Survey respondents were largely neutral regarding the relative advantage of RPG Sims compared to conventional approaches. As shown in Figure 2, nearly half of the responses were noncommittal, but over twice as many responses were positive relative to negative. A theme in the qualitative data that explains the large portion of neutral responses is an uncertainty regarding the ability of RPG Sims to produce real learning outcomes (see Table 2). This is not a surprising result considering that game-based learning for adult FL is largely unknown outside of government and military contexts in North America and Europe, and educational applications for games and simulations do not yet present a fully recognized or matured discipline (Crookall, 2011). Yet, even though the respondents seem to consider RPG Sims unproven, educators are unwilling to dismiss them outright as indicated by the low percentage of negative responses.

OPEN IN VIEWER

Figure 2.

Relative advantage.

OPEN IN VIEWER

Table 2.

Comments Expressing Skepticism Regarding Outcomes.

Record	Comment
I4	Some studies have claimed improved outcomes, but I do not believe that there is a general consensus on this.
J31	Just from this video, I can’t tell concretely what kind of conversation takes place in the game, so I can’t judge whether or not there would be a real effect.
J17	Whether linguistic ability improves or not is unpredictable.
I2	I think learners would get used to virtual communication, and this is unrelated to actual communication
E3	I doubt that they can really improve some logical and critical thinking and reasonable communication skills through that way.

Note. E = English; J = Japanese (Translated by Author).

The high rate of positive relative to negative responses is attributable to a belief among educators that game-based learning would have a positive effect on learner motivation. Participants cited various reasons for their beliefs, but the most common was an assumption pertaining to values and norms among the learner population. That is, many educators assume that since digital games are popular among the younger generation of learners, game-based learning materials would be as well (see Table 3).

OPEN IN VIEWER

Table 3.

Comments Attributing a Positive Impact on Learner Motivation to Learner Attitude.

Record	Comment
I1	The advantage is that the generation of learners and students that has become used to games can learn as though it were a game.
E18	As many students have now grown up with games such as this, I think it would be an ideal way of getting students motivated to learn.
J25	I felt that learning through computer games could be effective for learners and students of the younger generation in that they can learn in a fun manner at their own pace.
J37	Many young learners love games, so this kind of material is a breakthrough.
J32	. . . there are many learners who are interested in [Japanese] subculture, and many learners with an interest in games, so I think this approach is very useful.
J48	Computer games might be easy to use as learning material for the younger generation because there is already a strong interest and games are used habitually.

It is worth noting that although many respondents who considered RPG Sims to be unproven regarding actual improvement of linguistic skills still acknowledge a motivation-based advantage. Therefore, learner motivation appears to be a salient issue in FL education in Japanese higher education, and instructors believe that RPG Sims could have a beneficial impact in this regard.

Compatibility Respondents were more divided regarding the suitability of RPG Sims in their teaching contexts. As shown in Figure 3, the responses to compatibility were less noncommittal and more polarized than those for relative advantage. This is attributable to a diversity of teaching objectives reported among survey respondents, in conjunction with a belief that RPG Sims may be effective for meeting some of these goals but not others. For instance, Informant 6 made a direct connection between the existence of various learning goals one instructor may have and the effect on compatibility:

. . . the courses I have taught most recently were mostly content courses; the courses on tourism might have benefitted from RPGs simulating tourist-industry situations . . . My only language skills courses in recent years were for writing; RPGs would not have been suitable for these courses unless they had a large writing component . . . Finally, it is easy to imagine a RPG that could match the teaching goals of my speaking/listening classes. Even the software shown in the videos incorporates goals such as learning what to say in different situations.

OPEN IN VIEWER

Figure 3.

Compatibility.

The highest frequency of teaching objective was conversation at 43 occurrences, or roughly half of the total number of respondents. Conversation as a class focus entails the learning goals of acquiring or improving the speaking and listening skills required to utilize a target language in various types of oral/aural communicative acts. The qualitative data indicated a perception among respondents that the ISLET system may be useful for improving speaking and listening skills, but not for other linguistic modalities such as reading and writing. Therefore, the half of participants who indicated conversation as a teaching objective would likely perceive the RPG Sim as being more compatible than those focusing on other learning goals.

It should be noted, however, that a relationship between conversation as a teaching objective and perceptions of compatibility was not observed in the statistical analysis described in the following section. One possible explanation for this is that FL educators appear to concurrently be in charge of multiple classes with multiple teaching objectives. The item eliciting data on teaching objective was multiple-select, and 62.5% of respondents selected two or more choices. Therefore, the perception of compatibility may be contingent on courses that are most emphasized by individual instructors.

Complexity Respondents were slightly positive regarding their technical ability to implement RPG Sims within a reasonable time frame. As shown in Figure 4, the largest minority of responses regarding complexity were positive, and the second largest neutral. At the same time, one emergent theme in the qualitative data suggested that participants consider RPG Sims to be highly complex in terms of the cost in time and effort it would take to implement them as core material in a course of study. This contrast could be explained by another emergent theme, which is a belief that RPG Sims would be useful primarily for supplementary or self-study material (see Table 4).

OPEN IN VIEWER

Figure 4.

Complexity.

OPEN IN VIEWER

Table 4.

Comments Recognizing a Supplementary/Extracurricular Role for RPG Sims.

Record	Comment
I1	There are possibilities in the application of computer games for supplementing classroom learning
I3	I think there is plenty of room to consider it as learning material for off-campus or over long breaks.
J38	I have no problem with university students trying to learn Japanese at home or in their free time while enjoying a game. Live and let live.
E15	. . . as a supplemental tool—outside of the classroom environment, I can see the efficacy of such a method/tool for language acquisition.

This theme is consistent with literature concerning common treatment of CALL systems (Bax, 2003, 2011; Brown et al., 2008; Bush, 2008). Instructors may be assuming that if RPG Sims are relegated to a supplementary or extracurricular role, they themselves would have very little direct interaction with such systems in practice. That is, they would have little need to understand the operation and contents or explain them to learners. The instructor’s role would presumably be limited to monitoring progress through reports, a task that is not considered overly complex.

The Analysis Pertaining to Research Question 2

To address the differences in perceptions based on demographic characteristics, responses to items 1-5 were cross tabulated with responses to items 6-11 and tested using a Chi-Square test of independence that was considered valid if no cell had an expected frequency of zero and 80% of cells had an expected count of five or greater (Yates, Moore, & McCabe, 1999). Under these conditions, no valid test results were found due to the diversity of responses. The author then collapsed the strongly agree with the agree variable, and the strongly disagree with the disagree variable and reran the tests. Under these conditions, only one statistically significant relationship was noted between research interest and perceptions of relative advantage; X² (2, N = 88) = 16.54, p = 0.0005. Examination of the cell frequencies showed that about 74% (14 out of 19) of respondents who listed CALL among their research interests had a positive perception of relative advantage, while none responded negatively and the remaining 26% were noncommittal. On the other hand, only about 25% (17 out of 69) of educators with other research interests gave a positive response, the majority (55%) were noncommittal, and about 20% were negative. Thus, a positive perception of ISLET’s relative advantage was disproportionately high among researchers focusing on CALL.

The tendency to associate positive perspectives with innovations bearing a resemblance to technologies in which potential adopters are already interested is predicted by the construct of technology cluster, proposed in Rogers’ DT (2003, p. 249). This result is also compatible with Can and Cagiltay’s (2006) argument that a prior interest in computers can predict a favorable perception of the educational application of computer games. Also, the literature suggested that familiarity with technology-supported instructional approaches based on training or informal information sharing through peer networks fosters a positive perspective of its relative advantage (Alshumainmeri, 2008; Kessler, 2007). Thus, it is not surprising that CALL researchers in Japanese higher education would have more favorable views of RPG Sims than those interested in other fields of research.

No other significant relationships were found between perceptions and demographics, but one relationship in particular may deserve special note. No evidence was found indicating cultural background as a factor in variation of perceptions. When nationality (Japanese or Non-Japanese) was cross-tabulated with perceptions of RPG Sims, no significant relationships were apparent. Thus, regarding Joshi and colleague’s (2010) suggestion that Japanese culture may favor human over machine mediated learning interactions, no evidence was found in the present data to suggest that this cultural factor, if accurate, influences perceptions of RPG Sims for FL instruction.