A Meta-Analysis of the Effectiveness of English-Medium Education in Hong Kong

Abstract

To facilitate second language learning, it has become increasingly popular to use a second language as the medium of instruction for content subjects for majority language students. Although numerous research studies have shown the advantages of such kind of programs in North America and Europe, those investigating English as the Medium of Instruction (EMI) schools in Hong Kong yielded inconclusive results. This meta-analysis is the first attempt to synthesize the research evidence on EMI education in Hong Kong since 1970. Based on 24 studies, this meta-analysis shows that students in EMI secondary schools were more proficient in second language and performed better on measures of affective variables. Yet their learning in other content subjects suffered. The differences between the effectiveness of EMI education in Hong Kong and that of similar programs in other contexts will be discussed, thereby illuminating second language acquisition theories and bilingual education.

Keywords

L2 as the medium of instruction majority language students meta-analysis Hong Kong English medium

Medium of instruction (MoI) refers to the language used when teaching nonlanguage academic/content subjects (e.g., mathematics, science, history). Students’ first language (L1) is normally the default MoI in schools, yet for various reasons, students’ second language (L2) may be adopted instead (Ho & Ho, 2004). Some examples include immersion programs in Canada, Content and Language Integrated Learning in Europe, and bilingual programs in the United States. In Hong Kong, a former British colony reverted to China as a special administrative region in 1997, there is a strong preference for learning through the L2, English, and these programs are mainly implemented in English as the Medium of Instruction (EMI) secondary schools.

Owing to some complicated political, socioeconomic, and educational considerations, the MoI adopted in secondary schools in Hong Kong has always been a source of major debates. Unlike other contexts where similar programs are practiced, there seems to be no conclusive research evidence revealing the advantages and disadvantages of using English (L2) over Chinese (L1) as the MoI in Hong Kong. By synthesizing the research evidence of EMI education in Hong Kong over the past four decades, this meta-analysis sheds some light on MoI issues in Hong Kong. EMI education in Hong Kong is an interesting context for research as the two languages involved are typologically more distant than the languages involved in similar programs in Western contexts (Genesee, 2006). By comparing the effectiveness of using L2 as the MoI in Hong Kong with other educational contexts, this study can in turn illuminate broader questions regarding L2 acquisition and bilingual education.

The Development and Rationale for Using L2 as the MoI

Most people in the world learn at least one language other than their mother tongue (Tucker, 1999), and so the major issue of L2 acquisition is how to learn the language more effectively. In traditional L2 learning programs, the L2 is usually taught in isolation as a subject, which usually occupies several 30- to 40-minute periods per week in the school curriculum. The exact amount of time varies from one school to another and from one country to another, depending on the role of the L2 in a particular context.

Since the St. Lambert experiment in Canada, which was an initiative to help Anglophone children learn French more effectively (Lambert & Tucker, 1972), immersion programs seem to be an attractive alternative to traditional L2 learning programs. The basic principle is to teach some or all nonlanguage content subjects through the language that students are learning as an L2 (Lyster & Ballinger, 2011; Stoller, 2008). In this way, students are exposed to more L2 input in meaningful, communicative contexts, and hence they learn the L2 incidentally (Genesee, 2006; Snow, Met, & Genesee, 1989).

In the psycholinguistic strand, the effectiveness of such kinds of programs in L2 learning is supported by several L2 learning hypotheses. According to the comprehensible input hypothesis (Krashen, 1982), when learning content subjects through L2, learners are inevitably exposed to more comprehensible L2 input, which then triggers the innate language acquisition device (Chomsky, 1959). Learners also tend to produce more L2 through interacting with their teachers and peers, so they can obtain more interactionally modified input that better suits their needs, receive feedback on their language use, as well as test their own hypotheses about the language. These processes are likely to contribute to L2 learning according to the interaction hypothesis (Long, 1996) and output hypothesis (Swain, 1995). In the psychological strand, the use of L2 as the MoI is likely to motivate students to enhance their L2 proficiency, which significantly affects their academic achievement. It is also argued that students will be more interested in learning L2, as that is the medium through which they can understand the subject content (Gardner, 1985; Genesee, 1991; Snow et al., 1989). Based on these theoretical underpinnings, using L2 as the MoI provides a favorable environment for L2 learning.

Using L2 as MoI for Majority Language Versus Minority Language Students

However, the potential benefits of using L2 as the MoI have been challenged by the contrast between the success of using French as the MoI when teaching Anglophone children in the Canadian immersion programs (reviewed in more detail below) and the limitations of using EMI when teaching immigrant children in U.S. schools (Skutnabb-Kangas & Toukomaa, 1976, as cited in Cummins, 1979). Although the former resulted in additive bilingualism without decrements in academic achievement, the latter resulted in partial bilingualism, where students were proficient in neither L1 nor L2 and suffered negative effects on their academic performance (Cummins, 1979).

Such differential effects of using L2 as the MoI for majority and minority language students can be explained by the theoretical model of bilingual education proposed by Cummins (1979, 2000). In this model, educational outcomes (including cognitive, academic, linguistics, and affective) are the function of the interaction of background, students’ input, and education programs. The two hypotheses incorporated into this model, namely, the developmental interdependence hypothesis and the threshold hypothesis, provide significant insights into the necessary conditions for effective bilingual programs. The developmental interdependence hypothesis proposes that the level of L2 competence is partially a function of students’ L1 competence when intensive exposure to L2 begins (Cummins, 1979). The threshold hypothesis suggests that students have to attain a minimum level of L2 and L1 competence in order to enjoy the potential benefits that bilingualism may bring to their cognitive and academic functioning (Cummins, 1979).

Following these two hypotheses, when immigrant children (e.g., Spanish-speaking students) enter schools in Anglophone countries, they are immersed in an all-English learning environment from Day 1, although reaching the threshold level of English to fully benefit from such a learning environment may take 5 to 7 years (Collier, 1989). Meanwhile, their L1 is not supported in the school or the community. Although immigrant children gradually learn English in their daily life, such conversational language proficiency is different from academic proficiency, which is essential for academic development (Cummins, 2000). These students then suffer cognitive and academic disadvantages in schools. In contrast, when majority language students start learning through L2, their L1 is often well developed, since it is the home and community language. It is also likely that they have developed some proficiency in the L2, which has been taught as an isolated subject in school. Hence, those majority language students tend to enjoy the academic and cognitive benefits of bilingual programs adopting L2 as the MoI.

The focus of the current study is bilingual programs that use L2 as the MoI for majority language students (hereafter called L2-medium education¹). Some well-known examples include the immersion programs in Canada noted above and Content and Language Integrated Learning springing up in Europe since the 1990s, where English or other L2s are often adopted as the MoI for several content subjects in schools (Coyle, Hood, & Marsh, 2010). In these contexts, plenty of research studies have provided evidence that using L2 as the MoI for majority language students brought about different advantages.

Students learning through L2 outperformed their counterparts in traditional language learning programs in various L2 skills, without any adverse effects on their L1 development (e.g., Admiraal, Westhoff, & de Bot, 2006; Jiménez Catalán & Ruiz de Zarobe, 2009; Swain & Lapkin, 1982). When compared with native speakers at similar stages, students in L2-medium education could achieve native-like proficiency in receptive skills such as reading and listening, though they could only attain near-native proficiency in productive skills such as writing and speaking, and were not comparable with native speakers in terms of phonology, vocabulary, grammar, and communicative or sociolinguistic competence (Cummins & Swain, 1986; Genesee, 1987; Harley & King, 1989).

Most important, the more advanced L2 proficiency of students in L2-medium education was achieved at no cost to their academic achievement (e.g., Bergroth, 2006; Turnbull, Hart, & Lapkin, 2001). In other words, compared with their peers in traditional L2 learning programs, students in L2-medium education were not adversely affected in their learning of academic subjects, even though those subjects were learned through L2. In addition, researchers have found other benefits of L2-medium education, including such cognitive benefits as mental manipulation, flexibility, divergent thinking, creative thinking, and originality (Cummins, 1977; Jäppinen, 2005; Peal & Lambert, 1962).

Social and psychological benefits have also been observed, in the sense that students learning through L2 felt more confident when speaking to native speakers and developed more positive attitudes toward L2 (Lasagabaster & Sierra, 2009; Merisuo-Storm, 2006). Furthermore, it has been noted that L2-medium education was suitable for all students, irrespective of IQ, learning disabilities classification, and socioeconomic status (e.g., Genesee, 1987; Holobow, Genesee, & Lambert, 1991). There are some recent reviews of L2-medium education in different contexts, such as Lazaruk’s (2007) article on immersion programs in Canada and Pérez-Cañado’s (2012) synthesis of research on Content and Language Integrated Learning in Europe. However, the research evidence for using L2 as the MoI is not so conclusive in the Hong Kong educational context. The following section will briefly review the issues regarding MoI in Hong Kong and the debate to date.

Using L2 as MoI in Hong Kong

Policy Development

In Hong Kong, Chinese (spoken Cantonese and written Modern Standard Chinese) is the mother tongue of the majority of population, although English is considered high status (Hoare & Kong, 2008; Tsui, 2004). Chinese is used as the MoI for primary education and students learn English as a separate subject. On the other hand, English is the MoI for tertiary education. It is in secondary schools where the Hong Kong government faces a constant dilemma when deciding which language, Chinese or English, is to be used as the MoI, given the interplay of political, socioeconomic, and educational factors (Lin, 2006; Tsang, 2004; Tsui & Tollefson, 2004).

Before the 1997 political handover, the Hong Kong government adopted a noninterventionist policy, under which schools were allowed to decide on their own MoI (Poon, 2010). At this time, over 90% of secondary schools claimed to be EMI schools (Falvey, 1998), where all subjects (except Chinese, Chinese History, and Chinese Literature) were taught in English. However, observations revealed that in many of these ostensible EMI schools, the use of mixed Chinese and English was prevalent (Johnson, 1983). In 1997, shortly after the political handover, the Education Department stated that all secondary schools should adopt Chinese as the MoI, except schools that had applied for and been granted an exemption. As a result, all except 114 secondary schools in Hong Kong (≈25%) became Chinese as the Medium of Instruction (CMI) schools. However, this policy aroused vigorous opposition in society (Falvey, 1998). In 2009, the government decided to “fine-tune” the mother tongue policy (Education Bureau, 2009) by allowing CMI schools that met certain requirements to have some approved classes. In those classes, schools can adopt a different MoI for different subjects, groups, or time periods, according to the needs and ability of the students and teachers. These changes in the government’s MoI policy have not resolved the controversy over the issue (Hoare & Kong, 2008).

Research Evidence to Date

Judging from the contextual factors, Hong Kong, Canada, and Europe should be rather similar, in the sense that students admitted to L2-medium education are self-selected, majority language students with a solid foundation in their L1. Considering such contextual similarities, So (1998) was convinced that many students in Hong Kong possessed “the necessary sociolinguistic capital” (p. 171) to enjoy the benefits of EMI education. Nevertheless, although the research evidence reviewed above shows the benefits of L2-medium education in North America and Europe, the effectiveness of EMI education implemented in Hong Kong is not so clear.

Since the 1970s, there have been quite a number of empirical studies evaluating EMI education in Hong Kong, and the conclusions drawn from these are confusing or intriguing. For example, K. K. Ho’s (1985) experimental study revealed that EMI students enjoyed no significant advantages in English proficiency over their CMI counterparts, and there were no significant differences in the learning of most content subjects between the two groups. Marsh, Hau, and Kong (2000, 2002), after following a large number of students for 5 years, suggested that EMI students benefited in Chinese and English proficiency but were disadvantaged in content subjects such as science, geography, and history.

Several researchers (e.g., Hoare & Kong, 2008; Yip, Tsang, & Cheung, 2003) have critically reviewed the previous studies and identified some potential factors that may have led to the discrepancies in the findings in previous studies. The first one is the time period when the studies were conducted. As Yip et al. (2003) suggested, studies done before the 1997 compulsory mother tongue policy bore the risk of investigating mixed-code schools, instead of EMI or CMI ones. The language use in classrooms after the compulsory mother tongue policy was under more stringent control of the government and so the types of schools in studies conducted after 1997 were more clearly identified.

Second, the age of students under investigation and the research design of the studies are potential moderating factors. Collier (1989) argued that immersion students normally needed 5 to 7 years to catch up with those studying in their L1, because immersion students needed a certain period of time to reach the threshold level of L2 proficiency to benefit from immersion education. Therefore, the age of students, which implies how many years they have been experiencing L2-medium education, may affect the findings of the studies. For instance, junior form students (Grade 7 to 9) in Hong Kong EMI schools have typically experienced 1 to 3 years of L2-medium education, whereas their senior form counterparts (Grade 10 to 13) should have received 4 to 7 years of L2-medium education. In a similar vein, research studies adopting longitudinal designs following students for a few years may be in a better position to capture the long-term effects of bilingual programs.

Third, the types of outcome measures may influence students’ performance. Different types of measurements were adopted in different studies—some adopted low-stakes tests, on which students were not encouraged to do revision in advance, and the test results would not affect students’ official academic results; some used high-stakes tests such as public examinations, on which students were highly motivated to do a great deal of revision. Students’ motivation and efforts associated with different types of outcome measures may therefore moderate the effects of MoI on achievement (Marsh et al., 2002). In addition, some researchers specially designed tests for their studies, whereas others simply adopted standardized tests or measures, such as public examinations and validated vocabulary tests. The nature and scope of those different types of outcome measures may also affect the evaluation of students’ performance in different bilingual programs.

Fourth, most studies evaluating the impact of MoI selected students from intact schools/groups. Such a design ran the risk of having confounding variables, one of which is students’ initial abilities before entering EMI or CMI secondary schools (which were normally measured in terms of students’ academic results by the end of primary education). Therefore, whether studies have controlled for prior differences in abilities may influence the findings of studies.

More systematic reviews of research studies on the MoI issue in Hong Kong were done by Lin and Man (2009) and Poon (2009). Lin and Man (2009) provided a general review of empirical research studies on MoI-related issues since 1970s, whereas Poon (2009) reviewed English language education in Hong Kong over the past 25 years. However, both these reviews are narrative in nature, and they mainly reported the findings of studies available, without synthesizing them and drawing conclusions on the impact or effectiveness of EMI education in Hong Kong.

The Current Study

In the current study, we synthesize research studies examining the EMI education in Hong Kong over the past four decades (1970–2010), using the technique of meta-analysis, which is “a procedure for integrating the results of empirical research studies” (McGaw & Glass, 1980, p. 1). By doing so, we hope to come to a more definitive conclusion on the potential effectiveness of EMI education, which can then shed light on the long-lasting debate over the MoI issues in Hong Kong and inform policymakers on future policy formulation.

Furthermore, the synthesized results of studies examining EMI education in Hong Kong can be compared with those investigating L2-medium education in other contexts (e.g., Canada and Europe). Such a comparison will be particularly insightful for L2 acquisition and bilingual programs, since the languages involved in those contexts are considerably different in nature—in Hong Kong, the languages involved are Chinese and English, which are typologically distant and belong to distinct language families; in the Canadian and European contexts, the languages involved are very often English, French, German, and other European languages, which share common origins and are considered as more similar in typology (Genesee, 2006; Janson, 2012). A comparison of the research studies in those contexts may then reveal the potential moderating effect of language typology on the effectiveness of L2-medium education.

Research Questions

As reviewed above, the potential benefits of using L2 as the MoI include development in L2 proficiency without sacrificing L1 proficiency or academic achievement as well as benefits in affective variables such as learning motivation, strategy, interest, self-efficacy, and so on. Hence, to evaluate students’ learning in EMI education in Hong Kong, the three primary research questions of this meta-analysis are as follows.² First, what is the difference in academic achievement between students studying in EMI and CMI education? In particular, we were interested in differences in students’ Chinese and English language proficiency and in students’ achievement in such content subjects as mathematics, science, history, and geography.

Second, are there differences in affective variables, including self-concept, motivation, learning strategies, and interest, between students studying in EMI and CMI education? Third, to further examine the variables that may moderate the differences between students studying in EMI and CMI education, we examined the moderating effect of the following variables on the differences between students studying in EMI and CMI education: (a) the time period of carrying out the study (nonintervention, mother tongue policy), (b) the research design (longitudinal, cross-sectional, experimental), (c) the age of students (junior forms, senior forms), (d) the type of outcome measures (standardized tests, self-designed tests), and (e) control over initial abilities (with control, without control).

Method

Locating the Studies

The literature search was conducted with the keywords medium of instruction or immersion and Hong Kong, with studies published between 1970 and 2010. The databases that were searched included ERIC, SCOPUS, Hong Kong Education Bibliographic Database, and Education Bureau Central Resources Centre Catalogue, and the initial search yielded 178 studies. These were then screened with several additional criteria. First, the study must be empirical, rather than conceptual, theoretical, or a review. Second, the study had to compare students’ learning in EMI education with those in CMI education. Third, the study had to examine students’ learning in secondary schools, as the major dispute over MoI is at the secondary level (Tsang, 2004). Fourth, they had to have independent student outcome measures, such as achievement, self-concept, or motivation. In other words, the outcomes could not be teachers’ or students’ reflections, attitudes, or opinions.

Supplementary hand searches of the reference lists of some studies were also conducted. To ensure that the criteria of including studies were reliable, a sample of 20 of the 178 studies was screened by the second author, and the decisions to include or discard showed 100% agreement. The final sample consisted of 31 studies published in English. Although there were 11 Chinese articles that matched the searching keywords, none was empirical. The 31 studies were coded by the first author according to the variables of interest in the current meta-analysis, including (a) the year of study, (b) research design, (c) types of experimental and comparison groups, (d) independent variable(s) apart from MoIs, (e) dependent variable(s), (f) age of student participants, (g) types of tests/measures used, and (h) features of statistical analysis. Again, to ensure the reliability of the coding, one third of the studies were given to the second author for recoding. By dividing the number of items that were coded in the same way between the two authors by the total number of coded items, the estimated interrater reliability was 87%. The discrepancies were discussed between the authors and the coding of the other studies was modified wherever necessary. The coding protocol is in the appendix (available online at http://RER.sagepub.com/supplemental).

Statistical Analyses

Computing the Effect Sizes

For 13 studies, which included experimental studies and studies that compared EMI and CMI students without any statistical control, we estimated their effect sizes with Cohen’s d, which was calculated by dividing the differences in the means of the experimental group and comparison group with the pooled standard deviation (Lipsey & Wilson, 2001). Among those studies, one experimental study (Education Department, 1985) presented the adjusted means and standard deviations after considering students’ pretest scores. We therefore used those adjusted figures in calculating the Cohen’s d.

Two studies (Education Department, 1992; Lin & Morrison, 2010) compared EMI and CMI students with indicators other than means and standard deviations (e.g., success rate in public examination), so we adopted other formulas of effect size calculation (Wolf, 1986). Seven studies used hierarchical linear modeling to control for students’ socioeconomic background, initial abilities, and other factors. In those studies, the raw mean scores and standard deviations were normally not reported. Instead, those raw scores were all standardized so that the means became 0 and the standard deviations became 1. Hence, the simple formula for calculating Cohen’s d could not be applied and our preferred measure of effect size was the path coefficient of the variable, medium of instruction, in the hierarchical linear models. For three studies (C. C. Ho, 1989; Lau & Yuen, 2011; Lin & Morrison, 2010), several ways of comparing students’ academic achievement were reported; hence, more than one estimation of effect size was used.

During the process of computing effect sizes, 7 studies from the original pool of 31 were discarded because there was insufficient information for estimation of effect size or the data reported were also reported in another study. In the end, 24 studies were included in the final meta-analysis, 15 of which focused on academic achievement, 2 of which examined affective outcomes, and 7 of which measured both.

The basic principle of calculating effect sizes was to treat students studying in EMI schools as the experimental group and those in CMI schools as the comparison group. In some studies, there were some mixed-medium groups (e.g., mixed-medium, Chinese by subject, Chinese by class). These groups were excluded in the current analysis as the term, mixed, was used in different ways across studies, and including these groups would inevitably result in ambiguities. For some studies, the differences in students’ initial abilities were taken into account while they were not in the other studies. To avoid loss of information, we decided to include all the effect sizes first, and had one moderating variable, control over initial abilities, to determine its impact on the effect sizes.

Combining Effect Sizes

Each effect size was first multiplied by the inverse of its variance to yield the weighted effect size. Then the sum of all the weighted effect sizes was divided by the sum of the inverse variances (i.e., the weights) to generate the overall mean effect size (Lipsey & Wilson, 2001). For most studies, there was more than one effect size. This outcome occurred when, for example (a) the experimental group (EMI students) was compared with the comparison group (CMI students) on different variables such as Chinese, English, mathematics, history, self-concept, motivation, and so on; (b) when students studying in CMI schools were further categorized into different subgroups according to their abilities (e.g., CMI-High group, CMI-Middle group, and CMI-Low group), and then comparisons were made between EMI students and each of those subgroups, respectively; or (c) when the development of students was traced for several years by testing the students at regular intervals (e.g., at the end of each grade).

To capture the maximum amount of information without violating the assumption of independence of effects, we decided to adopt a shifting unit of analysis approach (Cooper, 1998; Cooper, Robinson, & Patall, 2006). In this approach, every effect size of a comparison in a study was first recorded. For example, students in one study were compared on their Chinese and mathematics achievement. Thus, two effect sizes were calculated for that study. When analyzing the difference in students’ overall academic achievement between EMI and CMI education, the two effect sizes in that study were averaged before entry into the analysis so that the sample only contributed one effect size (Borenstein, Hedges, Higgins, & Rothstein, 2009). On the other hand, when analyzing the difference in students’ achievement in different academic subjects, that study would contribute one effect size to the analysis of Chinese and mathematics achievement, respectively.

Testing for Publication Bias

Both published and unpublished studies (e.g., master dissertations, government reports) were included in this meta-analysis. Yet publication bias could still exist. Therefore, we generated two funnel plots (one for academic achievement and the other for affective variables) to examine the distribution of effect sizes in relation to the studies’ sample sizes. These funnel plots are presented in Figures S1 and S2 (available online at http://RER.sagepub.com/supplemental). Our impression was that there was little publication bias, since quite a number of studies with small sample sizes were included, and most studies were distributed symmetrically about the mean effect size (Borenstein et al., 2009). Nevertheless, we noted that Salili and Lai (2003) and Ripple, Jaquish, Lee, and Salili (1984) were potential outliers for the analysis of academic achievement and affective variables, respectively. We ran the main analyses without the outliers and found very minor differences in the mean effect sizes (±.01) when compared with the analyses for all studies (see Tables S1 and S2 for the results of main analyses without the outliers, available online at http://RER.sagepub.com/supplemental). We therefore decided to report the synthesized results for all 24 studies in this article.

Addressing the Research Questions

In the primary research questions, the independent variable is MoI (i.e., EMI vs. CMI) and the dependent variables are the mean effect sizes of academic achievement and affective variables in the 24 studies. To provide a more fine-grained picture of the differences between EMI and CMI education, the mean effect sizes of six core subjects—namely Chinese, English, mathematics, science, history, and geography—were also computed. Similarly, for the affective variables, the mean effect sizes of self-concept, motivation, learning strategy, and learning interest were calculated. For the subcomponents of self-concept and learning interest, apart from the overall mean effect sizes, those for some major subjects (e.g., Chinese, English, mathematics, science) were also estimated. In a similar vein, the mean effect sizes of three major types of motivation and learning strategies, namely, surface, deep, achieving (Biggs, 1987), were calculated.

However, not all included studies interpreted or measured subcomponents in the same way. For instance, although Marsh’s (1989) Self-Description Questionnaire and Biggs’s (1987) Learning Process Questionnaire were adopted by quite a number of studies to measure students’ self-concept and learning motivation and strategy, respectively, some other measurements were used in other studies (e.g., the PISA questionnaires that estimated students’ learning interest in Ho & Man, 2007, study). Also, although learning strategies were generally categorized into surface, deep, and achieving in a majority of studies, different categories were found in some other studies (e.g., the addition of rote learning strategy in the report of Education Bureau, 2004; the categorization of strategies into rehearsal, organization, and elaboration, among others, in Salili & Lai’s, 2003, study). In view of these differences in measurements and categorization, we first estimated the overall mean effect sizes of those studies that included those subcomponents (i.e., self-efficacy, motivation, strategy, and interest) and then only included the mean effect sizes of those studies that had similar categorization of subjects or types of those subcomponents.

The direction of the mean effect sizes shows how students in EMI education performed on the measures of different dependent variables when compared with their CMI peers and the magnitude of the mean effect sizes reflects the strength of such differences. Cohen (1988) suggested an effect size of 0.2 was small, 0.5 moderate, and 0.8 or over large. Apart from estimating the mean effect sizes, the 95% confidence intervals of the mean effect sizes were calculated. If the confidence intervals do not contain zero, the null hypothesis of no difference between EMI and CMI students can be rejected (Cooper et al., 2006). Z tests were also conducted to infer the statistical significance of the results (Lipsey & Wilson, 2001).

Next, the homogeneity of effect sizes was evaluated by computing Hedges’ Q statistics for academic achievement and affective variables (Card, 2012; Hedges & Olkin, 1985; Lipsey & Wilson, 2001). Significant heterogeneity would suggest that factors other than sampling error may be contributing to the variation in the effect sizes. To interpret such heterogeneity, two moderating tests, including (a) between-group heterogeneity and (b) one-way analysis of variance (ANOVA) on mean effect sizes, were conducted. The former test partitioned total heterogeneity (Q_Total) into between-group heterogeneity (Q_Between) and within-group heterogeneity (Q_Within; Card, 2012; Lipsey & Wilson, 2001). A significant Q_Between (with df = number of categorical groups − 1, p < .05) would indicate that effect sizes vary significantly among the studies for a particular moderating variable. The magnitude of the true variation of effect sizes was further estimated by T and I² (Borenstein et al., 2009).

The other test (i.e., one-way ANOVA) mainly evaluated the heterogeneity of effect sizes in terms of differences in magnitude and direction of mean effect sizes. A significant F statistic (or adjusted F in Brown–Forsyth in case of heterogeneous variance across categorical groups) would indicate a significant mean difference across groups for that particular moderating variable. These two tests addressed the third research question, which asks whether the five variables moderate the differences in students’ learning between EMI and CMI education.

Results

A summary of the effect size data for each study is presented in Table S3 (available online at http://RER.sagepub.com/supplemental). In the tables of this section, as the EMI students were treated as the experimental group, positive mean effect sizes reveal that EMI students performed better than the CMI students on the measures of the dependent variables and vice versa.

Differences in Students’ Academic Achievement

This section addresses the first research question: What is the difference in academic achievement between students studying in EMI and CMI education? The mean effect sizes of the 22 studies that measured students’ academic achievement are presented in Table 1. Generally speaking, EMI students had lower overall academic achievement, with a small and significant negative effect size, M = −.28, SE = .01; Z = −33.0, p < .01, and the homogeneity test shows that these effect sizes varied significantly across studies, Q(21) = 1,240, p < .01; T = .32; I² = 98.31%.

Table 1

Mean effect sizes (ESs) of the differences in academic achievement between EMI and CMI students

				95% CI
Variables	N of studies	M ESs	SE ESs	Lower	Upper	Z Sig.
All academic achievement	22	−.28	.01	−.29	−.26	Z Sig.	−33.00**
Chinese	8	−.12	.01	−.14	−.10	−12.90**
English	10	+.42	.01	+.40	+.44	+41.13**
Mathematics	13	−.01	.01	−.02	+.01	−0.86
Science	13	−.55	.01	−.58	−.54	−65.67**
History	9	−.86	.02	−.89	−.83	−53.18**
Geography	7	−.78	.02	−.81	−.75	−47.15**

Note. EMI = English as the medium of instruction; CMI = Chinese as the medium of instruction; CI = confidence interval.

**p < .01.

Differences in Students’ Chinese- and English-Language Proficiency

One major aim of EMI education or other similar kinds of L2-medium programs for majority language students is additive bilingualism, that is, the learning of L2 without sacrificing development in L1. Therefore, it is important to investigate students’ L1 (Chinese) and L2 (English) proficiency in EMI and CMI education. Table 1 shows that students in EMI education lagged slightly behind their CMI peers in Chinese, M = −.12, SE = .01; Z = −12.90, p < .01, but outperformed their CMI peers in English proficiency. The difference in students’ English achievement was significant and fairly large, with moderate effect size, M = .42, SE = .01; Z = 41.13, p < .01.

Differences in Students’ Achievement in Other Content Subjects

While facilitating additive bilingualism or L2 learning, effective EMI education should not hamper students’ learning of other content subjects when compared with their counterparts studying through L1. In the studies included in this meta-analysis, four major content subjects were consistently found, namely, mathematics, science, history, and geography. Table 1 shows that there was no significant difference in students’ learning of mathematics, M = −.01, SE = .01; 95% CI = −.02, .01; Z = −.86, p = .19. However, the performance of EMI students on science, history, and geography was significantly worse than the CMI group, with moderate to large effect sizes, M = −.55 to −.86.

Differences in Affective Variables

In addition to academic achievement, we compared EMI and CMI students on measures of self-concept, motivation, strategy, and interest. The mean effect sizes of all affective variables and their subcomponents are presented in Table 2. The synthesis shows that students in EMI schools generally had more positive affective outcomes, though the effect size was small, M = .08, SE = .01; Z = 8.97, p < .01. Again, the homogeneity test indicates that the effect sizes varied significantly across studies, Q(9) = 111, p < .01; T = .11; I² = 91.94%.

Table 2

Mean effect sizes (ESs) of the differences in affective variables between EMI and CMI students

				95% CI
Variables	N of studies	M ESs	SE ESs	Lower	Upper	Z Sig.
All affective variables	9	+.08	.01	+.06	+.10	Z Sig.	+8.97**
Self-concept	9	+.10	.01	+.09	+.12	+11.46**
Chinese	4	+.10	.01	+.08	+.12	+10.76**
English	5	+.28	.01	+.27	+.30	+30.56**
Mathematics	6	+.12	.01	+.11	+.14	+13.07**
Science	3	−.08	.01	−.10	−.05	−6.71**
Motivation	5	+.02	.01	+.00	+.04	+2.18*
Surface	4	+.02	.01	+.00	+.04	+2.23*
Deep	4	+.01	.01	−.01	+.03	+.94
Achieving	4	+.02	.01	+.00	+.04	+2.30*
Strategy	5	+.07	.01	+.05	+.09	+7.37**
Surface	4	+.05	.01	+.03	+.07	+5.05**
Deep	4	+.08	.01	+.06	+.10	+7.69**
Achieving	4	+.11	.01	+.09	+.13	+10.65**
Interests	3	+.05	.01	+.02	+.07	+3.83**
Mathematics	3	+.11	.01	+.09	+.14	+9.95**
Science	2	−.05	.01	−.08	−.03	-4.06**

Note. EMI = English as the medium of instruction; CMI = Chinese as the medium of instruction; CI = confidence interval.

p < .05. **p < .01.

With regard to the affective subcomponents, EMI students had higher self-concept in English learning than their peers in CMI schools, M = .28, SE = .01; Z = 30.56, p < .01, but EMI students reported lower self-concept in science subject, M = −.08, SE = .01; Z = −6.71, p < .01. In addition, EMI students performed slightly better on measures of learning motivation, M = .02, SE = .01; Z = 2.18, p < .05, irrespective of the types of motivation. EMI students were also found to score higher on measures of learning strategies, M = .07, SE = .01; Z = 7.37, p < .01. Among the three kinds of strategy investigated in this meta-analysis, the mean effect size was the largest for the achieving strategy, M = .11, SE = .01; Z = 10.65, p < .01, which refers to strategy that maximizes cost-effectiveness of time and effort (i.e., study skills; Biggs, 1987). Finally, students in EMI schools showed slightly more interest in learning as a whole, M = .05, SE = .01; Z = 3.86, p < .01, though they were less interested in learning science, M = −.05, SE = .01; Z = −4.06, p < .01, when compared with their peers in CMI schools.

Moderating Effects

As previously reported, homogeneity tests show that the effect sizes of studies measuring academic achievement and affective variables varied significantly. This implies that some factors other than sampling error have contributed to the variance in the effect sizes. In this section, we examine the moderating effect of five variables on the differences between students studying in EMI and CMI education. As described in the Method section, Hedges’ Q tests with T and I² and one-way ANOVA were used. Some studies (e.g., Marsh et al., 2002; Salili & Lai, 2003; Siu et al., 1979) included students in both junior and senior forms, and hence the effect sizes for these two groups of students were entered separately when analyzing the students’ grades, resulting in more than 24 effect sizes. In addition, one study (C. C. Ho, 1989) included both standardized tests and self-designed tests and was not included in the analysis of type of outcome measures. Similarly, two studies (C. C. Ho, 1989; Lau & Yuen, 2011) compared EMI and CMI students partly with statistical control over students’ initial abilities and partly without. Thus, they were not included when examining the impact of the variable, control over initial abilities.

The results of the moderator analyses regarding academic achievement are presented in Tables 3 and 4, and results of the moderator analyses regarding affective variables are presented in Tables 5 and 6. The type of outcome measures is missing in Tables 5 and 6, because all the studies examining affective outcomes, except Salili and Lai (2003), used standardized tests and so we decided not to investigate the impact of that particular moderating variable.

Table 3

Homogeneity tests of effect sizes (academic achievement)

Moderating variable	Hedges’ Q	N of ES	df	T (95% CI)	I² (95% CI)
Implementation period
Total heterogeneity	1239.97**	22	21
Within-group: Nonintervention (pre-1997)	304.62**	13	12
Within-group: Mother tongue policy (post-1997)	892.42**	9	8
Between-group heterogeneity	42.93**		1	.04 (.02–.06)	51.08% (20.18% to 70.02%)
Research design
Total heterogeneity	1239.97**	22	21
Within-group: Longitudinal	394.05**	5	4
Within-group: Cross-sectional	616.08**	13	12
Within-group: Experimental	1.38	4	3
Between-group heterogeneity	160.62**		2	.13 (.11–.15)	90.81% (87.43% to 93.27%)
Grade of students
Total heterogeneity	1668.42**	33	32
Within-group: Junior form (Grades 7 to 9)	1344.88**	25	24
Within-group: Senior form (Grade 10 or above)	278.71**	8	7
Between-Group heterogeneity	64.99**		1	.04 (.03–.06)	50.76% (26.47% to 67.02%)
Type of outcome measures
Total heterogeneity	1236.07**	21	20
Within-group: Standardized test	748.26**	14	13
Within-group: Self-designed test	26.14**	7	6
Between-group heterogeneity	461.67**		1	.19 (.17–.21)	95.67% (94.41% to 96.64%)
Control over initial abilities
Total heterogeneity	1236.04**	20	19
Within-group: With control^a	558.17**	11	10
Within-group: Without any control	677.77**	9	8
Between-group Heterogeneity	.10		1	.00 (.00–.00)	00.00% (00.00% to 00.00%)

Note. ES = effect size; CI = confidence interval.

With control: including studies with statistical control and experimental studies.

p < .01.

Table 4

Results of one-way ANOVA for moderators (academic achievement)

Moderating variable	N of ESs	M ESs	SD ESs	F	df
Implementation period				2.18^a	1, 9.53^a
Nonintervention (pre-1997)	13	−.25	.30
Mother tongue policy (post-1997)	9	+.16	.81
Research design				.653	2, 19
Longitudinal	7	+.02	.74
Cross-sectional	11	−.04	.59
Experimental	4	−.38	.12
Grade of students				.00	1, 31
Junior form (Grades 7 to 9)	25	+.03	.75
Senior form (Grade 10 or above)	8	+.02	.44
Type of outcome measures				8.55*^a	1, 15.58^a
Standardized test	14	+.10	.66
Self-designed test	7	−.44	.15
Control over initial abilities				4.12^a	1, 8.83^a
With control^b	11	−.32	.20
Without any control	9	+.24	.81

Note. ES = effect size.

Significant Levene’s statistic for testing homogeneity of variance across groups was found (p < .05). Thus, F, df, and significant value with Brown–Forsyth’s adjustment are reported.

With control: including studies with statistical control and experimental studies.

p < .05.

Table 5

Homogeneity tests of effect sizes (affective variables)

Moderating variable	Hedges’ Q	N of ESs	df	T (95% CI)	I² (95% CI)
Implementation period
Total heterogeneity	111.72**	9	8
Within-group: Nonintervention (pre-1997)	25.04**	4	3
Within-group: Mother tongue policy (post-1997)	38.99**	5	4
Between-group heterogeneity	47.67**		1	.07 (.04–.09)	83.22% (69.61% and 90.73%)
Research design
Total heterogeneity	111.72**	9	8
Within-group: Longitudinal	45.21**	4	3
Within-group: Cross-sectional	41.84**	5	4
Between-group heterogeneity	24.66**		1	.04 (.02–.07)	67.56% (34.64% to 83.90%)
Grade of students
Total heterogeneity	317.41**	19	18
Within-group: Junior form (Grades 7 to 9)	253.06**	13	12
Within-group: Senior form (Grade 10 or above)	50.95**	6	5
Between-group heterogeneity	13.40**		1	.00 (.00–.02)	0.00% (00.00% to 31.42%)
Control over initial abilities
Total heterogeneity	111.72**	9	8
Within-group: With control^a	53.23**	5	4
Within-group: Without any control	7.47**	4	3
Between-group heterogeneity	51.02**		1	.07 (.05–.09)	84.32% (71.91% to 91.25%)

Note. ES = effect size; CI = confidence interval.

With control: including studies with statistical control and experimental studies.

p < .01.

Table 6

Results of one-way ANOVA for moderators (affective variables)

Moderating variable	N of ESs	M ESs	SD ESs	F	df
Implementation period				2.94	1, 7
Nonintervention (pre-1997)	4	−.06	.29
Mother tongue policy (post-1997)	5	+.16	.09
Research design				.01	1, 7
Longitudinal	4	+.07	.15
Cross-sectional	5	+.06	.28
Grade of students				.77	1, 17
Junior form (Grades 7 to 9)	13	+.08	.21
Senior form (Grade 10 or above)	6	−.00	.08
Control over initial abilities				.00	1, 7
With control^a	5	+.06	.15
Without any control	4	+.06	.32

Note. ES = effect size.

With control: including studies with statistical control and experimental studies.

When academic achievement was considered, Table 3 shows that, according to Hedges’ Q, the effect sizes of the included studies were significantly heterogeneous for all the moderating variables, except for control over initial abilities. The magnitude of heterogeneity for each moderating variable was then evaluated. The resulting T fell in a range of .00 to .19, with the variables, type of outcome measures, and research design yielding moderate Ts of .19 and .13, respectively. Their I² of 95.67% and 90.81% also indicated that most of the observed variance in the effect sizes is real (Borenstein et al., 2009).

Table 4 contains the effect of the moderating variables based on the results of the one-way ANOVA. In academic achievement, most of the moderating variables (except grade of students) appeared to affect the magnitude and direction of the differences between EMI and CMI students. For instance, the studies conducted after the compulsory mother tongue policy showed that EMI students possessed better academic achievement, M = .16, SD = .81, whereas those conducted during the nonintervention period indicated the opposite, M = −.25, SD = .30. The studies using longitudinal designs indicated slightly better performance of EMI students, M = .02, SD = .74, whereas the experimental studies indicated that EMI students performed worse than CMI students, M = −.38, SD = .12. Such differences in the magnitude and direction of effect sizes also applied to type of outcome measures and control over initial abilities. However, Table 4 shows that there was no significant difference between or among groups for most variables, except type of outcome measures.

With regard to affective variables, such as strategy, interest, motivation, and self-efficacy, Hedges’ Q statistics in Table 5 show that the effect sizes of the included studies were significantly heterogeneous for all the moderating variables. However, the resulting T fell in a range of .00 to .07, reflecting low heterogeneity. Parallel to these findings, Table 6 indicates that most of the variables did not have any significant moderating effects on the differences in affective outcomes between EMI and CMI students. Despite this, the variable, implementation period, did seem to affect the magnitude and direction of the effect sizes of affective variables. In particular, EMI students were found to perform slightly worse on measures of affective variables in studies conducted during the nonintervention period, M = −.06, SD = .29, yet they performed better on those measures in studies carried out under the mother tongue policy, M = .16, SD = .09.

Discussion

This meta-analysis systematically combines the empirical evidence of 24 studies investigating the EMI education in Hong Kong. We acknowledge that a majority of the studies included were cross-sectional and correlational in nature and thus we cannot make any strong causal claims. In the following sections, we discuss some potential implications of the findings of this synthesis.

Academic Achievement

First, this meta-analysis shows that in the Hong Kong educational context, students in secondary schools that adopted English (L2) as the MoI performed better in their L2 proficiency, but not L1. This group of students also lagged behind in content subjects, especially in science, history, and geography. These results suggest that EMI education in Hong Kong may fail to achieve the goal of additive bilingualism. In addition, the findings imply the fact that EMI students in Hong Kong may have sacrificed academic achievement for L2 proficiency. Further analyses reflect that the variables, implementation period, research design, type of outcome measures, and control over initial abilities had some moderating effects on the magnitude and direction of the differences in students’ academic achievement between EMI and CMI education.

Although these variables were found to lead to small and significant heterogeneity of effect sizes across studies, only type of outcome measures was a significant moderator in the ANOVA tests. It was found that EMI students performed slightly better than their CMI peers in studies adopting standardized tests, yet the EMI group performed worse than the other group when self-designed tests were used. Such a significant moderating impact of the type of outcome measures can be attributed to the fact that self-designed tests were mainly used with experimental designs (e.g., Siu et al., 1979) and some smaller scale studies (e.g., K. K. Ho, 1985). These tests were usually more specifically designed to measure students’ learning on certain topics taught during the research period, which may have favored CMI students who learnt through their mother tongue.

Comparison with Other Contexts

As noted previously, research studies on L2-medium education in other educational contexts (e.g., Canada, Europe) provide fairly conclusive evidence of the positive impact of using L2 as the MoI on students’ language development without hindering academic achievement. Therefore, the findings of this meta-analysis reveal that the effectiveness of EMI education in Hong Kong appear inconsistent with findings in other educational contexts. There are several potential factors for this discrepancy.

The first factor is the proportion of students enrolling in L2-medium schools in Hong Kong. Owing to the general preference for English in Hong Kong, over 90% of students were studying in EMI schools before the mother tongue policy (Johnson, 1997). Even after the implementation of compulsory mother tongue education, there were still around 25% of students receiving EMI education, and this proportion is on the rise again after the fine-tuning policy since the 2010/2011 academic year (Kan, Lai, Kirkpatrick, & Law, 2011). Such a relatively large proportion of students studying in EMI schools in Hong Kong puts great pressure on schools (Hoare & Kong, 2008), because successful implementation of L2-medium education requires teachers with near-native proficiency in the L2 as well as effective pedagogical skills of teaching through the L2 (Navés, 2009; Tsui, 1992).

Closely related to the large proportion of students in EMI education is students’ L2 proficiency when L2-medium education starts. Hong Kong students are generally admitted to EMI schools in Grade 7, by which time they should have been learning English for at least 6 years in primary schools. However, Tsui (1992) argued that most students are not proficient enough in English to cope with the all-English curriculum, as the English they have learned in primary schools is mainly for communicative purposes, which is different from the academic language required to study content subjects through English (Cummins, 2000; Johnson & Swain, 1994). The Hong Kong government and secondary schools fail to help students bridge this proficiency gap in English (Hoare & Kong, 2008). Without reaching the threshold level of L2, Hong Kong students are very likely to suffer in academic achievement, according to the threshold hypothesis (Cummins, 1979, 2000).

However, students in other educational contexts may also face similar problems with L2 proficiency gaps. Why do the problems look more severe in Hong Kong? There may be two reasons. The first reason is the languages involved. In similar programs in Canada and Europe, the languages involved are likely to be European languages, such as English, French, German, Swedish, and Dutch (Pérez-Cañado, 2012). Tracing their origins and histories, English, German, Dutch, and Swedish all belong to Germanic languages, and many English words—especially academic vocabulary related to religion, science, medicine, and literature—were based on Latin, from which French developed (Janson, 2012). On the other hand, the languages involved in EMI education in Hong Kong are Chinese (Cantonese) and English, which are two distinct language systems, with different writing and spelling systems (Janson, 2012). Thus, it is reasonable to conclude that EMI students in Hong Kong may encounter greater difficulties in bridging the proficiency gap.

Second, the situation in Hong Kong may be related to teachers’ pedagogical skills in teaching effectively through L2. As mentioned previously, Hong Kong teachers were observed to rely heavily on switching to Chinese to help students better understand the lesson content, especially when they tried to cover the whole public exam syllabus, which is the same for all schools irrespective of the MoI. Teachers simply cannot afford to spend more time teaching English in content subject lessons nor can they provide support for students’ L2 development (Hoare, 2003; Hoare & Kong, 2008). This context may not be the most conducive to students’ language development or learning subject content through the L2.

Affective Outcomes

Though students studying in EMI education in Hong Kong appeared to suffer in academic achievement, they performed better in almost all measures of affective variables. In other words, EMI students’ self-concept, motivation, and interest were not undermined. Instead, they may rely on their higher initial abilities and employ different learning strategies (e.g., achieving strategy) to overcome the difficulties they encountered in EMI schools. Such psychological benefits of EMI education is likely because of the general social preference for English proficiency in the Hong Kong society, so students admitted to EMI schools feel more privileged (Salili & Tsui, 2005).

These benefits became even more obvious in studies carried out during the mother tongue policy period, as the moderating variable implementation period was found to have affected the magnitude and direction of effect sizes measuring affective outcomes. This finding can be explained by the strong labeling effect of the compulsory mother tongue policy, since only 114 schools remained as EMI schools, which were then regarded as more prestigious. Thus, students in those schools may feel better about themselves, be more motivated and interested to learn, and more aware of using different types of strategy to study. By synthesizing the results concerning both academic achievement and affective variables, this meta-analysis highlights the fact that when evaluating the effectiveness of EMI education in Hong Kong, we should not focus only on tangible learning outcomes but also some less tangible variables, such as self-concept, learning motivation, and interest.

The differing findings on EMI education in Hong Kong and those L2-medium education programs in other educational contexts can yield some important implications for L2 acquisition. Backed up by psycholinguistic theories of L2 learning, using L2 as the MoI in some or all content subjects has become an appealing alternative to traditional L2 learning programs and has been practiced worldwide. However, this meta-analysis suggests that using L2 as the MoI does not guarantee successful L2 learning without sacrificing academic achievement. Rather, whether such kind of programs can achieve its dual aims depends on many other factors, which may include the sociolinguistic contexts, the actual program implementation in schools, students’ language proficiency, teachers’ pedagogical practices, and the typological differences between the languages involved. These claims echo what Cummins (1979, 2000) proposed in his theoretical model—the outcomes of bilingual education are a function of the interaction of different variables. Hence, though L2-medium instruction is a global trend, there are challenges in making it an effective practice in different educational contexts, an aspect to which more attention should be paid.

Moderators

Several potential moderating variables have been examined in this meta-analysis. Though only a few of them were found to have statistically significant effect on the differences between EMI and CMI students, a closer look into those moderating variables can perhaps illuminate both the MoI policy in Hong Kong and the broader methodological issues of conducting research on L2-medium education or other bilingual programs.

When the studies were categorized according to the period of implementation, it was found that while EMI students tended to lag behind in academic achievement during the nonintervention period, they outperformed the CMI students after the compulsory mother tongue policy was implemented. Bearing in mind that the mother tongue policy greatly reduced the number of EMI schools by stipulating the necessary conditions that EMI schools had to fulfill, the discrepancy in the magnitude of effect sizes of studies conducted in periods with different policies may then provide some evidence of the effective streaming effects of the compulsory mother tongue policy. That is, students with sufficient English language proficiency admitted to schools with the necessary resources and support are more likely to benefit from EMI education. Such a finding may shed light on the recent fine-tuning MoI policy, which has increased the number of EMI schools (or classes) in Hong Kong. The government and schools may need to carefully consider whether teachers are well supported in developing effective pedagogical skills and whether students are sufficiently assisted with their L2 proficiency.

In addition, EMI students performed slightly better than CMI students in longitudinal research studies, which traced the development of students throughout their secondary schooling (usually for 5 years). Perhaps students entering EMI schools need at least 5 to 7 years to reach the threshold level of L2 (Collier, 1989). Therefore, when longitudinal studies follow students over several years, the negative impact of EMI education gradually diminishes (Marsh et al., 2002). To evaluate the impact of any bilingual programs, it seems more reasonable to follow the development of students’ learning throughout the program, instead of simply taking snapshots of their learning.

Some studies controlled for students’ initial abilities (i.e., students’ abilities or achievement before they were admitted to EMI or CMI secondary schools), and some did not. However, students’ initial abilities are a potential confounding variable when comparing students’ learning in different types of education programs. The current meta-analysis showed that when the initial abilities are not controlled, EMI students outperform CMI students academically. Yet when students’ initial abilities were taken into account, the former group was not on par with the latter group. Such a contrast implies that studies evaluating the effectiveness of bilingual programs should consider this important variable.

Limitations

This meta-analysis is not without limitations. First, as illustrated above, there was considerable heterogeneity in the effect sizes across studies. The variance in effect sizes across the relatively small number of studies may explain why most of the moderating variables were found to be nonsignificant in the ANOVAs. Such heterogeneity may also reflect the variance in the quality of the studies included. Whereas some studies (e.g., Education Bureau, 2004, 2006; Marsh et al., 2000, 2002) were large-scale longitudinal studies tracing thousands of students throughout their secondary education, some were cross-sectional comparisons with fairly small samples of students (e.g., Lau & Yuen, 2011; Lo & Murphy, 2010). The number of experimental studies was also very limited, thereby raising concerns about the internal validity of some studies.

Second, even though this meta-analysis attempted to examine the effectiveness of EMI education in students’ learning by controlling some input variables, such as students’ prior abilities and socioeconomic status, there are still some variables that can hardly be captured or controlled by any studies. Examples of these variables include parental support and effects of self-selection into the programs. Third, this study focused on the learning outcomes, whereas teaching and learning processes (e.g., classroom interaction and classroom discourse) were not included. Teaching and learning processes in EMI education are also important for evaluating bilingual programs, through understanding what actually happens in lessons using an L2 as the MoI (Lo & Macaro, 2012; Yip, Coyle, & Tsang, 2007). Finally, this meta-analysis only compared EMI and CMI students and excluded mixed-code groups. However, as some researchers may argue (e.g., Lin, 2006; Macaro, 2009; Probyn, 2009), judicious and principled code-switching may actually be helpful and effective for some learners. Therefore, excluding mixed-code groups may have precluded a comprehensive picture of the impact of using different languages as the MoI. This is an area worth further investigation.

Recommendations and Conclusion

From the insights of this meta-analysis, we have generated a list of recommendations for future research on the effectiveness of L2-medium education or other bilingual programs:

Longitudinal investigation into students’ performance can better capture and trace the impact of different bilingual programs on students’ development

Large-scale studies can provide a more comprehensive picture, but individual factors such as initial academic abilities and socioeconomic status should be taken into account or controlled for

Validated standardized measures of students’ performance may be less sensitive to the effect of other confounds, such as sample size and period of intervention; if similar standardized measures are adopted across studies, the findings of different studies can be better interpreted, compared, and synthesized

The various groups of participants (e.g., EMI group, CMI group, mixed-MoI group) should be clearly defined, not only in terms of the written materials adopted but also with regard to the specific language that teachers use when delivering lessons

To the best of our knowledge, this study is the first to systematically synthesize research studies of EMI education in Hong Kong. This meta-analysis, therefore, helps provide more evidence about students’ learning in EMI education in Hong Kong. The results can in turn shed light on the MoI debate in Hong Kong society and the government’s formulation of MoI policies in the future. In addition, through discussing the differences in the effectiveness of L2-medium education in different educational contexts, this study may have implications for L2 learning and bilingual education programs more generally.

Footnotes

Notes

Authors

YUEN YI LO is an assistant professor at The University of Hong Kong, teaching on the Bachelor of Education, Postgraduate Diploma in Education, and Masters programs. She has a strong research interest in second language acquisition, language-in-education policy, and teacher development.

ERIC SIU CHUNG LO is a part-time lecturer at Hong Kong Baptist University, teaching research method courses on the Bachelor of Psychology program.

References

Admiraal

Westhoff

de Bot

(2006). Evaluation of bilingual secondary education in the Netherlands: Students’ language proficiency. Educational Research and Evaluation, 12, 75–93.

Bergroth

(2006). Immersion students in the matriculation examination three years after immersion. In Bjorklund

Mard-Miettinen

Bergstrom

Sodergard

(Eds.), Exploring dual-focused education: Integrating language and content for individual and societal needs (pp. 123–134). Vaasa, Finland: Centre for Immersion and Multilingualism, University of Vaasa.

Biggs

J. B.

(1987). Learning process questionnaire manual: Student approaches to learning. Hawthorn, Victoria, Australia: Australian Council for Educational Research.

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2009). Introduction to meta-analysis. Chichester, England: John Wiley.

Card

N. A.

(2012). Applied meta-analysis for social science research. New York, NY: Guilford Press.

*Chan

(1981). A crossroads in language of instruction. Journal of Reading, 24, 411–415.

Chomsky

(1959). Review of verbal behaviour. Language, 35, 26–58.

*Chung

C. H.

(1996). The medium of instruction on the academic achievement of secondary one students (Unpublished master’s thesis). The University of Hong Kong.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

10.

Collier

V. P.

(1989). How long? A synthesis of research on academic achievement in a second language. TESOL Quarterly, 23, 509–531.

11.

Cooper

(1998). Synthesizing research: A guide for literature reviews (3rd ed.). Thousand Oaks, CA: Sage.

12.

Cooper

Robinson

J. C.

Patall

E. A.

(2006). Does homework improve academic achievement? A synthesis of research, 1987–2003. Review of Educational Research, 76, 1–62. doi:10.3102/00346543076001001

13.

Coyle

Hood

Marsh

(2010). CLIL: Content and language integrated learning. Cambridge, England: Cambridge University Press.

14.

Cummins

(1977). Cognitive factors associated with the attainment of intermediate levels of bilingual skills. Modern Language Journal, 61, 3–12.

15.

Cummins

(1979). Linguistic interdependence and the educational development of bilingual children. Review of Educational Research, 49, 222–251.

16.

Cummins

(2000). Language, power, and pedagogy: Bilingual children in the crossfire. Clevedon, England: Multilingual Matters.

17.

Cummins

Swain

(1986). Bilingualism in education: Aspects of theory, research and practice. London, England: Longman.

18.

*Education Bureau. (2004). Evaluation on the implementation of the medium of instruction guidance for secondary schools: Final report (1999–2002). Hong Kong, China: Government Printer.

19.

*Education Bureau. (2006). Further evaluation on the implementation of the medium of instruction guidance for secondary schools: Final report (2002–2004). Hong Kong, China: Government Printer.

20.

Education Bureau. (2009). Fine-tuning the medium of instruction for secondary schools (Education Bureau Circular No. 6/2009). Hong Kong, China: Government Printer.

21.

*Education Department. (1985). The effects of the medium of instruction on the achievement of Form 2 students in Hong Kong secondary schools. Hong Kong, China: Government Printer.

22.

*Education Department. (1992). A study on the relation between initial language proficiency at S1 level and subsequent HKCEE performance for medium-of-instruction grouping. Hong Kong, China: Government Printer.

23.

*Education Department. (1994). Research on change of medium of instruction in secondary schools. Hong Kong, China: Government Printer.

24.

*Education Department. (1998). Evaluation study on the implementation of medium of instruction grouping in secondary schools (1994/95–1996/97). Hong Kong, China: Government Printer.

25.

*Education Department. (2000). Evaluation study on the implementation of medium of instruction grouping in secondary schools (1994/95–1998/99). Hong Kong, China: Government Printer.

26.

Falvey

(1998). ESL, EFL & language acquisition in the context of Hong Kong. In Asker

(Ed.), Teaching language and culture: Building Hong Kong on education (pp. 73–85). Hong Kong, China: Addison Wesley.

27.

Gardner

R. C.

(1985). Social psychology and second language learning: The role of attitudes and motivation. London, England: Edward Arnold.

28.

Genesee

(1987). Learning through two languages: Studies of immersion and bilingual education. Rowley, MA: Newbury House.

29.

Genesee

(1991). Second language learning in school settings: Lessons from immersion. In Reynolds

A. G.

(Ed.), Bilingualism, multiculturalism and second language learning: The McGill Conference in honour of Wallace E. Lambert (pp.183–201). Hillsdale, NJ: Lawrence Erlbaum.

30.

Genesee

(2006). What do we know about bilingual education for majority-language students? In Bhatia

T. K.

Ritchie

W. C.

(Eds.), The handbook of bilingualism (pp. 547–576). Oxford, England: Blackwell.

31.

Harley

King

M. L.

(1989). Verb lexis in the written compositions of young L2 learners. Studies in Second Language Acquisition, 11, 415–439.

32.

Hedges

L. V.

Olkin

(1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.

33.

K. K.

(2004). The developmental trend of the medium of instruction in secondary schools of Hong Kong: Prospect and retrospect. Language and Education, 18, 400–413.

34.

*Ho

C. C.

(1989). An evaluation of the effects of changing the medium of instruction in a Hong Kong secondary school (Unpublished master’s thesis). The University of Hong Kong.

35.

*Ho

E. S. C.

Man

E. Y. F.

(2007). Student performance in Chinese medium-of-instruction (CMI) and English medium-of-instruction (EMI) schools: What we learned from the PISA study? Hong Kong, China: Chinese University of Hong Kong.

36.

*Ho

K. K.

(1985). The paradox of immersion in a second language. NABE Journal, 10, 51–64.

37.

Hoare

(2003). Effective teaching of science through English in Hong Kong secondary schools (Unpublished doctoral dissertation). The University of Hong Kong. Retrieved from Proquest Dissertations & Theses A&I Database (Proquest No. 305241259)

38.

Hoare

Kong

(2008). Late immersion in Hong Kong: Still stressed or making progress? In Fortune

T. W.

Tedick

D. J.

(Eds.), Pathways to multilingualism: Evolving perspectives on immersion education (pp. 242–263). Clevedon, England: Multilingual Matters.

39.

Holobow

Genesee

Lambert

W. E.

(1991). The effectiveness of a foreign language immersion program for children from different ethnic and social class background: Report 2. Applied Psycholinguistics, 12,179–198.

40.

Janson

(2012). The history of languages: An introduction. New York, NY: Oxford University Press.

41.

Jäppinen

A. K.

(2005). Thinking and content learning of Mathematics and Science as cognitional development in Content and Language Integrated Learning (CLIL): Teaching through a foreign language in Finland. Language and Education, 19, 147–168.

42.

Jiménez Catalán

R. M.

Ruiz de Zarobe

(2009). The receptive vocabulary of EFL learners in two instructional contexts: CLIL versus non-CLIL instruction. In Zarobe

Y. Ruiz de

Catalàn

R. M. Jiménez

(Eds.), Content and language integrated learning: Evidence from research in Europe (pp. 81–92). Bristol, England: Multilingual Matters.

43.

Johnson

R. K.

(1983). Bilingual switching strategies: A study of the modes of teacher-talk in bilingual secondary school classrooms in Hong Kong. Language Learning and Communication, 2, 249–350.

44.

Johnson

R. K.

(1997). The Hong Kong education system: Late immersion under stress. In Johnson

R. K.

Swain

(Eds.), Immersion education: International perspectives (pp. 171–189). New York, NY: Cambridge University Press.

45.

*Johnson

R. K.

Chan

R. M. K.

Lee

L. M.

J. C.

(1985). An investigation of the effectiveness of various language modes of presentation, spoken and written, in form III in Hong Kong Anglo-Chinese secondary schools. Hong Kong, China: Government Printer.

46.

Johnson

R. K.

Swain

(1994). From core to content: Bridging the L2 proficiency gap in late immersion. Language and Education, 8, 211–229.

47.

Kan

Lai

K. C.

Kirkpatrick

Law

(2011). Fine-tuning Hong Kong’s medium of instruction policy. Hong Kong, China: Hong Kong Institute of Education.

48.

Krashen

(1982). Principles and practice in second language acquisition. New York, NY: Prentice Hall.

49.

Lambert

W. E.

Tucker

G. R.

(1972). The bilingual education of children: The St. Lambert experiment. Rowley, MA: Newbury House.

50.

Lasagabaster

Sierra

J. M.

(2009). Language attitudes in CLIL and traditional EFL classes. International CLIL Research Journal, 2, 4–17.

51.

*Lau

W.W. F.

Yuen

A. H. K.

(2011). The impact of the medium of instruction: The case of teaching and learning of computer programming. Education and Information Technologies, 16, 183–201. doi:10.1007/s10639-009-9118-8

52.

Lazaruk

(2007). Linguistic, academic, and cognitive benefits of French immersion. Canadian Modern Language Journal, 63, 605–628. doi:10.3138/cmlr.63.5.605

53.

Lin

A. M. Y.

(2006). Beyond linguistic purism in language-in-education policy and practice: Exploring bilingual pedagogies in a Hong Kong Science classroom. Language and Education, 20, 287–305.

54.

Lin

A. M. Y.

Man

E. Y. F.

(2009). Bilingual education: Southeast Asian perspectives. Hong Kong, China: Hong Kong University Press.

55.

*Lin

L. H. F.

Morrison

(2010). The impact of the medium of instruction in Hong Kong secondary schools on tertiary students’ vocabulary. Journal of English for Academic Purposes, 9, 255–266. doi:10.1016/j.jeap.2010.09.002

56.

Lipsey

M. W.

Wilson

D. B.

(2001). Practical meta-analysis. Thousand Oaks, CA: Sage.

57.

*Lo

L. F.

(1991). Academic achievement, medium of instruction and teacher characteristics. New Horizons: Journal of Education, 32, 18–26.

58.

Long

M. H.

(1996). The role of the linguistic environment in second language acquisition. In Ritchie

Bhatia

(Eds.), Handbook of second language acquisition (pp. 413–468). San Diego, CA: Academic Press.

59.

Y. Y.

Macaro

(2012). The medium of instruction and classroom interaction: Evidence from Hong Kong secondary schools. International Journal of Bilingual Education and Bilingualism, 15, 29–52. doi:10.1080/13670050.2011.588307

60.

*Lo

Y. Y.

Murphy

V. A.

(2010). Vocabulary knowledge and growth in immersion and regular language-learning programmes in Hong Kong. Language and Education, 24, 215–238. doi:10.1080/09500780903576125

61.

Lyster

Ballinger

(2011). Content-based language teaching: Convergent concerns across divergent contexts. Language Teaching Research, 15, 279–288. doi:10.1177/1362168811401150

62.

Macaro

(2009). Teacher use of codeswitching in the second language classroom: Exploring optimal use. In Turnbull

Dailey-O’Cain

(Eds.), First language use in second and foreign language learning (pp. 35–49). Bristol, England: Multilingual Matters.

63.

Marsh

H. W.

(1989). Self-Description Questionnaire II: Manual. Campbelltown, New South Wales, Australia: University of Western Sydney.

64.

*Marsh

H. W.

Hau

K. T.

Kong

C. K.

(2000). Late immersion and language of instruction in Hong Kong high schools: Achievement growth in language and nonlanguage subjects. Harvard Educational Review, 70, 302–346.

65.

*Marsh

H. W.

Hau

K. T.

Kong

C. K.

(2002). Multilevel causal ordering of academic self-concept and achievement: Influence of language of instruction (English compared with Chinese) for Hong Kong students. American Educational Research Journal, 39, 727–763. doi:10.3102/00028312039003727

66.

McGaw

Glass

G. V.

(1980). Choice of the metric for effect size in meta-analysis. American Educational Research Journal, 17, 325–337.

67.

Merisuo-Storm

(2006). Development of boys’ and girls’ literacy skills and learning attitudes in CLIL education. In Bjorklund

Mard-Miettinen

Bergstrom

Sodergard

(Eds.), Exploring dual-focused education: Integrating language and content for individual and societal needs (pp. 176–188). Vaasa, Finland: Centre for Immersion and Multilingualism, University of Vaasa.

68.

Navés

(2009). Effective content and language integrated learning (CLIL) programmes. In Zarobe

Y. Ruiz de

Catalàn

R. M. Jiménez

(Eds.), Content and language integrated learning: Evidence from research in Europe (pp. 22–40). Bristol, England: Multilingual Matters.

69.

Peal

Lambert

W. E.

(1962). The relation of bilingualism to intelligence. Psychological Monographs, 76, 1–23.

70.

Pérez-Cañado

M. L.

(2012). CLIL research in Europe: Past, present, and future. International Journal of Bilingual Education and Bilingualism, 15, 315–341. doi:10.1080/13670050.2011.630064.

71.

Poon

A. Y. K.

(2009). A review of research in English language education in Hong Kong in the past 25 years: Reflections and the way forward. Education Research Journal, 24, 7–40.

72.

Poon

A. Y. K.

(2010). Language use, language policy and planning in Hong Kong. Current Issues in Language Planning, 11, 1–66. doi:10.1080/14664201003682327

73.

Probyn

(2009). Smuggling the vernacular into the classroom: Conflicts and tensions in classroom codeswitching in township/rural schools in South Africa. International Journal of Bilingual Education and Bilingualism, 12, 123–146. doi:10.1080/13670050802153137

74.

*Ripple

Jaquish

Lee

H. W.

Salili

(1984). Cognitive and affective costs of bilingual education: A look at the Hong Kong experience. New Horizons: Journal of Education, 25, 74–81.

75.

*Salili

Lai

M. K.

(2003). Learning and motivation of Chinese students in Hong Kong: A longitudinal study of contextual influences on students’ achievement orientation and performance. Psychology in the Schools, 40, 51–70. doi:10.1002/pits.10069

76.

Salili

Tsui

A. B. M.

(2005). The effects of medium of instruction on students’ learning motivation and learning. In Hoosain

Salili

(Eds.), Language in multicultural education (pp. 135–156). Greenwich, CT: Information Age.

77.

*Siu

P. K.

Chung

S. C.

Hinton

Cheng

Y. N.

L. F.

Luk

H. K.

Chung

Y. P.

(1979). The effects of the medium of instruction on student cognitive development and academic achievement. Hong Kong, China: Chinese University of Hong Kong.

78.

Snow

Met

Genesee

(1989). A conceptual framework for the integration of language and content in second/ foreign language instruction. TESOL Quarterly, 23, 201–218.

79.

D. W. C.

(1998). One country, two cultures and three languages: Sociolinguistic conditions and language education in Hong Kong. In Asker

(Ed.), Teaching language and culture: Building Hong Kong on education (pp.152–177). Hong Kong, China: Addison Wesley.

80.

Stoller

F. L.

(2008). Content-based instruction. In Deusen-Scholl

N. Van

Hornberger

N. H.

(Eds.), Encyclopedia of language and education: Vol. 4. Second and foreign language education (2nd ed., pp. 59–70). New York, NY: Springer.

81.

Swain

(1995). Three functions of output in second language learning. In Cook

Seidlhofer

(Eds.), Principle and practice in applied linguistics: Studies in honour of H. G. Widdowson (pp. 125–144). Oxford, England: Oxford University Press.

82.

Swain

Lapkin

(1982). Evaluating bilingual education: A Canadian case study. Clevedon, England: Multilingual Matters.

83.

Tsang

W. K.

(2004, August). Evaluating the medium of instruction policy in a post-colonial society: The case of Hong Kong Special Administration Region. Paper presented at the annual meeting of the American Sociological Association, San Francisco, CA.

84.

Tsui

A. B. M.

(1992). Using English as a medium of instruction and English language acquisition. In Kwong

L. K.

(Ed.), Into the twenty first century: Issues of language in Hong Kong (pp. 135–145). Hong Kong, China: Hong Kong Linguistics Society of Hong Kong.

85.

Tsui

A. B. M.

(2004). Medium of instruction in Hong Kong: One country, two systems, whose language? In Tollefson

Tsui

A. B. M.

(Eds.), Medium of instruction policies: Which agenda? Whose agenda? (pp. 97–106). Mahwah, NJ: Lawrence Erlbaum.

86.

Tsui

A. B. M.

Tollefson

(2004). The centrality of medium of instruction policy in sociopolitical processes. In Tollefson

Tsui

A. B. M.

(Eds.), Medium of instruction policies: Which agenda? Whose agenda? (pp. 1–20). Mahwah, NJ: Lawrence Erlbaum.

87.

Tucker

G. R.

(1999). A global perspective on bilingualism and bilingual education. Washington, DC: ERIC Clearinghouse on Languages and Linguistics.

88.

Turnbull

Hart

Lapkin

(2001). Grade 3 immersion students’ performance in literacy and mathematics: Province-wide results from Ontario (1998–99). Canadian Modern Language Review, 58, 9–26.

89.

Wolf

F. M.

(1986). Meta-analysis: Quantitative methods for research synthesis. Beverly Hills, CA: Sage.

90.

Yip

D. Y.

Coyle

Tsang

W. K.

(2007). Evaluation of the effects of medium of instruction on the science learning of Hong Kong secondary students: Instructional activities in science lessons. Educational Journal, 35, 77–107.

91.

*Yip

D. Y.

Tsang

W. K.

(2007). Evaluation of the effects of the medium of instruction on science learning of Hong Kong secondary students: Students’ self-concept in science. International Journal of Science and Mathematics Education, 5, 393–413. doi:10.1007/s10763-006-9043-x

92.

*Yip

D. Y.

Tsang

W. K.

Cheung

S. P.

(2003). Evaluation of the effects of medium of instruction on the Science learning of Hong Kong secondary students: Performance on the science achievement test. Bilingual Research Journal, 27, 295–331. doi:10.1080/15235882.2003.10162808

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB

0.08 MB

0.20 MB