Exploring Effectiveness and Moderators of Language Learning Strategy Instruction on Second Language and Self-Regulated Learning Outcomes

Theoretical Underpinnings

In her seminal works, Rubin (1975, 1981) argued that observing “good language learners” (i.e., those identified as effective in acquiring a new language) and studying processes that help them become successful may inform theories of language processing. This information, in turn, could be taught to less successful language learners. The goals of SI, then, are to (a) increase learners’ awareness of the most effective methods of language learning (Cohen, Weaver, & Li 1996; Dabarera, Renandya, & Zhang, 2014; De Silva, 2014; Hu et al., 2009) and (b) develop the independent, self-regulated learners actively participating in their own learning processes (Graham & Macaro, 2008; Oxford, 1999).

Focus on self-regulated learning, Zimmerman (1990) argued, is needed because the “initial optimism that teaching students various learning strategies would lead to improved self-regulated learning has cooled with mounting evidence that strategy use involves more than mere knowledge of a strategy” (p. 9; see also Dörnyei, 2003, 2005; Grenfell & Macaro, 2007). He further argued that to promote self-regulated learning, instruction should focus on supporting three component processes: (a) behavioral, the knowledge and use of learning strategies; (b) metacognitive, self-feedback regarding the effectiveness of learning and learning strategies and student responsiveness to such feedback; and (c) motivational, the interdependence between learning and motivational processes. Figure 1 summarizes these theoretically formulated relationships among SI, self-regulated learning, and achievement.

OPEN IN VIEWER

Figure 1

A model of strategy instruction effects on self-regulated learning and achievement

With regard to the behavioral component, much theoretical and empirical evidence supports the need for the learner to both know and use LLS. Cohen (1998) and Genesee, Lindholm-Leary, Saunders, and Christian (2005) observed that conscious awareness and use of LLS is characteristic of L2 development. This is because L2 learners—beginning to learn a new language at a more mature age—are more aware of language features they need to learn and can consciously draw on explicit LLS to enhance their learning. According to Macaro (2006), LLS do not simply make learning more efficient, but are “the raw material without which L2 learning cannot take place” (p. 332). Indeed, despite some inconsistencies (Nisbet, Tindall, & Arroyo, 2005; Takeuchi, 1993) and evidence to the contrary (Gardner, Tremblay, & Masgoret 1997), a positive relationship between LLS and L2 proficiency has been documented in a number of studies (e.g., Lan & Oxford, 2003; Nahavandi & Mukundan, 2014; Peacock & Ho, 2003; see also Cohen & Macaro, 2007). In synthesizing findings from 12 studies, Oxford (1999), for example, reported that LLS use accounted for substantial amount of variance in L2 proficiency ranging from 21% (a study of Taiwanese students learning English in secondary and tertiary institutions) to 58% (a study of first-year English learners in a Japanese women’s college).

With regard to the metacognitive component, Zimmerman (1990) argued that learners need to engage a cyclic “self-oriented feedback loop” through which they monitor LLS effectiveness and respond to self-feedback in varied ways, “ranging from covert changes in self-perception to overt changes in behavior” (e.g., altering LLS use; p. 5). Many LLS researchers (e.g., Cohen, 1998; Hsiao & Oxford, 2002; Vandergrift & Tafaghodtari, 2010; Zenotz, 2012) argued that L2 success did not depend on the number or frequency of LLS use, but rather on the learner’s ability to select and orchestrate LLS that are most appropriate for completing a given learning task. After all, Grenfell and Macaro (2007) noted, if inappropriately used, any LLS may result in failure. Indeed, metacogntive knowledge—including knowledge of self, knowledge of task, knowledge of learning goals, and strategic competence—has been linked with higher L2 outcomes (Ardasheva, 2016; Ardasheva & Tretter, 2013; Graham & Macaro, 2008; Huang et al., 2009; Kolic-Vehovec & Bajsanski, 2007; Schoonen, Hulstijn, & Bossers, 1998; van Gelderen et al., 2004).

Finally, the motivational component of self-regulated learning relies on learners’ willingness to commit time, effort, and vigilance to initiate and regulate LLS (Zimmerman, 1990). In other words, Zimmerman (1990) contends, self-regulated learning is not only “self-determined in a metacognitive sense” (p. 6) but also self-motivated. With some inconsistencies (Takahashi, 2005; Vandergrift, 2005), research supported the existence of a direct (e.g., Ehrman & Oxford, 1995; Wang, 2008) or mediated (e.g., Ardasheva, 2016; Pae, 2008) relationship between motivation and L2 achievement. Examples of investigated individual difference characteristics mediating motivation include metacognitive strategies (Ardasheva, 2016), effort (Bernaus & Gardner, 2008), and intensity and self-confidence (Pae, 2008). Furthermore, studies found a robust association between motivation and LLS (e.g., MacIntyre & Noels, 1996; Schmidt & Watanabe, 2001; Vandergrift, 2005). Peacock and Ho (2003), for example, found that high LLS users reported being highly motivated and perceived learning an L2 as personally important and enjoyable. Other studies comparing high- versus low-motivation students found that the former group used LLS with greater frequency (Oxford & Nyikos, 1989) and reported knowing more LLS and tended to find LLS more effective and easier to use (MacIntyre & Noels, 1996). Importantly, research indicated that motivation may be stimulated by instructional environments that satisfy human needs for competence (the know-how), autonomy (self-initiation, self-regulation), and relatedness (“secure and satisfying relationships with others;” Deci, Vallerand, Pelletier, & Ryan, 1991, p. 327; see also Noels, 2001; Noels, Clément, & Pelletier, 1999; Wu, 2003). Taken together, these findings suggest that motivation is an important, manipulable-by-instruction component of self-regulated learning; its impact on L2 development may be direct or mediated by LLS and related individual difference characteristics.

Although, as noted above, a number of statistical modeling studies examined mediating effects of different aspects of self-regulated learning on L2 achievement, SI studies per se typically examine only the direct SI impacts on self-regulated learning and/or on L2 achievement, without considering the theoretically posited mediating effects. Following the typical SI study design, this meta-analysis will separately estimate SI effects and their moderators for two domains: language and self-regulated learning. Below, we first describe a typical SI study that exemplifies the common structures of SI studies included in this meta-analysis and then discuss variables that may moderate SI effectiveness depending on the operationalization of such common structures across individual studies.

A Typical SI Study Showcasing Common SI Structures

Macaro and Erler (2008) conducted a 14-month SI study in reading with a sample of young (11–12 year olds), beginner (Year 1) learners of French enrolled in six secondary schools in England. Six intact classrooms were matched on teachers’ years of experience and nonrandomly assigned to either treatment or control conditions (one control classroom dropped from the experiment before the end of the study contributing to an attrition rate of 30.2%; participants and nonparticipants did not differ in reading on pretest, suggesting that attrition was not an issue). Whereas 62 students in the treatment group received awareness raising of 12 strategies listed in a pre-intervention questionnaire plus six additional strategies, 54 students in control groups received regular instruction. Strategies targeted in the treatment group included cognitive (e.g., sounding out, using context clues, background knowledge) and socioaffective (e.g., asking for help, not giving up easily) strategies. Both treatment and control groups were taught by their regular teachers; treatment teachers used researcher-developed reading/SI materials, in addition to regular textbooks. SI lasted, on average, about 10 minutes per day with instructional procedures including the following steps: (a) awareness raising, (b) modelling of strategies, (c) scaffolded practice, (d) removal of scaffolding, and (e) evaluation. The latter component included individualized feedback by the teacher and pair and whole-group discussions. Importantly, “implicit in all these activities was the concurrent development of metacognition with regard to making decisions about clusters of strategies available, and evaluating strategies used” (Macaro & Erler, 2008, p. 105).

Participants were assessed pre- and postintervention on three researcher-developed measures: (a) a reading comprehension test, (b) a reading strategy use questionnaire, and (c) a French attitudes scale. The reading test—the posttest was administered 1 month after the intervention—included narrow (translation) and broad (idea unit identification) reading tasks. The complexity and length of L2 text increased from pre- to postintervention to reflect growing proficiency of the students; test responses were provided in the first language (L1; also referred to as the native language). To allow for results’ comparability, reading tests were scored as percent correct. Strategy use and attitudes measures were both self-report, with 3- and 5-point Likert-type scales, respectively. The reported reliabilities for all three measures were acceptable (.7 and above). Results indicated that SI (a) significantly improved comprehension with an effect size (ES)—corrected for small sample size—of 1.17, (b) “brought about changes in strategy use” (Macaro & Erler, 2008, p. 90; with some of the changes favoring control group), and (c) improved reading attitudes.

Yet, no benefits of SI were found in at least two other recently published L2 reading comprehension studies (Gladwin & Stepp-Greany, 2008; Takallou, 2011) qualifying for this meta-analysis; similar discrepancies in SI impacts were documented for other language (e.g., listening; Cross, 2009; Vandergrift & Tafaghodtari, 2010) and self-regulated learning (e.g., strategy use; Ranalli, 2013; Zenotz, 2012) outcomes. Cross (2009) and Vandergrift and Tafaghodtari (2010), for example, found no versus positive SI impacts, respectively. Notably, although both studies were similar in terms of SI instructional features and focused on adult FL learners, the two studies differed in terms of SI duration, participants’ proficiency levels, and L1–L2 linguistic proximity. These findings suggest that variations in common SI structures across individual studies may play a moderating role on SI effectiveness. We discuss such moderators next, including both known and novel (not explored by previous meta-analytic research) moderators.

Outcomes

Research on spontaneous strategy use linked LLS with higher L2 achievement across a broad range of language and self-regulated learning outcomes. Examples of latter include reading (Huang et al., 2009; Kolic-Vehovec & Bajsanski, 2007; Schoonen et al., 1998; van Gelderen et al., 2004), listening (Dreyer & Oxford, 1996; Peacock & Ho, 2003; Takeuchi, 1993; Vandergrift, Goh, Mareschal, & Tafaghodtari, 2006), speaking and writing (Peacock & Ho, 2003), overall proficiency (Nisbet et al., 2005; Takeuchi, 1993), and vocabulary and grammar knowledge (Dreyer & Oxford, 1996; Fraser, 1999; Peacock & Ho, 2003; Takeuchi, 1993). Examples of former include motivation (MacIntyre & Noels, 1996) and self-efficacy (Magogwe & Oliver, 2007).

Research on instructed strategy use, in turn, suggested that SI effectiveness varied depending on the targeted outcome, both for language and self-regulated learning domains. Hassan et al.’s (2005) systematic review, for example, found that SI “works for reading comprehension and writing skills, and [that] the research evidence for this is stronger than it is for listening, speaking and overall proficiency” (p. 6). Evidence regarding SI effectiveness for vocabulary, in turn, was judged as being weak. In line with Hassan et al.’s (2005) synthesis, Plonsky’s (2011) meta-analysis estimated that SI effects on overall proficiency and listening were negligible. In contrast to Hassan et al.’s (2005) work, however, Plonsky (2011) found that SI effectiveness was the strongest for speaking; the effects for reading and vocabulary were average and the effect for writing was the smallest. Furthermore, out of the two SI effects on self-regulated learning outcomes—namely, strategy use and attitudes—only that on strategy use was statistically significant and large in size; the effect on attitudes was estimated to be negligible.

Two reasons may account for such discrepancies scope of work included in each synthesis (published over half a decade later, Plonsky’s [2011] study included a worth of eight more years of research) and differences in analytic approaches (meta-analytic techniques in Plonsky [2011] and team-assigned weights of evidence regarding study trustworthiness, methodological soundness, and relevance in Hassan et al. [2005]). In addition to these obvious reasons, the between-syntheses discrepancies in conclusions may be attributed to methodological advancements in the field itself, such as the differentiation between generic and task-specific LLS (e.g., speaking LLS, listening LLS; see Dörnyei, 2005; Hsiao & Oxford, 2002; Oxford, 2011; Oxford, Cho, Leung, & Kim, 2004). The latter trend, in particular, may explain the drop in the percentage of SI studies targeting overall proficiency over time. That is, among all synthesized studies targeting a specific language outcome, studies that focused on overall proficiency were 5 out of 27 in Hassan et al. (2005; see Table 4.1) and 4 out of 95 in Plonsky (2011), roughly 18% and 4% of all qualifying studies, respectively. (We discuss other advancements in the field later in the paper.) Taken together, these findings highlight language and self-regulated learning outcomes as an important moderator of SI effectiveness and suggest a need to periodically reevaluate SI effectiveness across outcomes, considering most recent trends and evidence.

Furthermore, it is important to recognize that language outcomes, specifically, may be broadly categorized into language outcomes per se (speaking, listening, vocabulary, and grammar) and literacy outcomes (reading and writing). Although the two categories may somewhat overlap in adult L2 learning, this may not be the case for young learners who typically develop language skills first (for this reason, bilingualism does not necessarily imply biliteracy) and, as noted earlier, few SI interventions place an equal emphasis on both language and literacy skills. These considerations suggest that both learners’ age and proficiency (language and literacy skills) may moderate SI effectiveness, along with other learner and study characteristics discussed next.

Context

Researchers have long recognized that context-related variables may differentially affect the effectiveness of any intervention. Literature suggests that when it comes to SI interventions, such variables may include L2 versus FL setting, age, educational level, proficiency, and language typology.

Treatment

In an earlier systematic review of SI studies, Hassan et al. (2005) noted that between-study differences in structural features of SI treatments—such as the selection of strategy type (metacognitive, cognitive, or socioaffective strategies), strategy scope (single strategy or packaged together strategies), treatment duration (from up to 2 weeks to up to a school year), and instructional approach (awareness-raising vs. behavior-modeling; defined later in the article)—“limited the degree to which studies could be combined cumulatively” (p. 64) to assess the overall effectiveness of SI interventions. The authors called for greater standardization of intervention frameworks in the future and for more research comparing effectiveness of awareness-raising versus behavior-modeling approaches.

Subsequently, the use of meta-analytic techniques—as contrasted with Hassan et al.’s (2005) weight-of-evidence judgments—did allow Plonsky (2011) to quantify the extent to which differences in structural features of interventions, but not in the selection of an instructional approach, contributed to differences in SI effectiveness. In particular, Plonsky (2011) found that teaching fewer strategies (eight or less), targeting cognitive rather than metacognitive strategies, and providing longer interventions (over 2 weeks) were associated with larger SI impacts. Echoing Hassan et al. (2005), Plonsky (2011) noted the continued lack of SI standardization, which he attributed to a lack of comprehensive SI theory leaving “researchers and practitioners to design studies of SI based largely on convenience, intuition, and/or some level of idiosyncrasy” (p. 998).

It is worthwhile to note, however, that although Plonsky (2011) incorporated more recent research, both Plonsky’s (2011) and Hassan et al.’s (2005) reviews included an overlapping data subset of early (1980s and 1990s) studies. Due, in part, to recommendations for research and practice emanating from 30 years of SI research (Cohen & Macaro, 2007), the proliferation of pedagogical, how-to literature (e.g., Chamot, 2009; Rivera-Mills & Plonsky, 2007; see a discussion of SI models in Chamot, 2004), and an emergent theoretical focus on self-regulated learning—particularly on the metacognitive component of self-regulated learning—as an enabling mechanism of SI effectiveness (e.g., Grenfell & Macaro, 2007; Macaro, 2006), a notably greater standardization of intervention frameworks has gradually emerged in the past decade.

In particular, most recent SI interventions adopt an awareness-raising instructional model targeting task-specific strategy clusters—rather than single strategies—often across metacognitive, cognitive, and socioaffective strategy types (e.g., Dabarera et al., 2014; Lam, 2009; Macaro & Erler, 2008; Takallou, 2011; Vandergrift, & Tafaghodtari, 2010). With some minor variations, the awareness-raising model typically includes the following steps: (a) consciousness raising—students reflect on learning and their current and potential strategies; (b) modeling—teachers introduce and model new strategies appropriate for the learning goal; (c) guided practice—students are given opportunities to practice strategies with gradual removal of teacher prompts to do so; and (d) evaluation and goal-setting—“students identify problem areas, select strategies that might help remedy them and evaluate their success” (Graham & Macaro, 2008, p. 753). Yet, a number of studies still employ a behavior-modeling approach including only modeling and practice (Kim, 2013; Morett, 2012; Urlaub, 2012).

Accordingly—beyond duration and strategy number moderators examined in Plonsky (2011)—we will explore whether the selection of SI approach (awareness-raising vs. behavior modeling) affects SI effectiveness. Other moderators possibly affecting SI effectiveness and not investigated in previous meta-analyses, will include SI scope and delivery mode. With regard to former, we will examine whether or not larger strategy selection scope (clustering strategies across metacognitive, cognitive, and socioaffective strategy types; e.g., Dabarera et al., 2014; Lam, 2009; Takallou, 2011) would be associated with greater gains than narrower strategy selection scope (only a single strategy type; e.g., Ding & Songphorn, 2012; Gladwin & Stepp-Greany, 2008). With regard to delivery mode, we will estimate whether technology-delivered instruction (an emergent trend in recent SI research; e.g., e-learning tools: Morett, 2012; Ranalli, 2013; video: Kim, 2013) would be associated with greater or lesser gains when compared with human-delivered instruction and whether greater familiarity with SI theories and procedures and a greater control over fidelity of implementation would benefit researcher-led (e.g., Baleghizadeh & Mortazavi, 2013; Bozorgian & Pillay, 2013; Chan, 2014; Dabarera et al., 2014) versus teacher-lead (e.g., Grenfell & Harris, 2013; Gunning, 2011; Hu et al., 2009) interventions.

Methodological Features

Past meta-analytic research, across educational fields, has demonstrated that research design and implementation features (e.g., assignment to conditions) may serve as sources of variance in the estimated outcomes. Plonsky (2011), for example, found that although studies with stronger design (pretest, random groups, reported reliability) consistently yielded larger effects than did studies with weaker designs (no pretest, random groups, reported reliability), on average, all studies included in his meta-analysis yielded statistically detectable effects, regardless of the strengths or weakness of their designs. By contrast, Adesope, Lavin, Thompson, and Ungerleider (2010) reported that, for at least some of the examined outcomes, weaker design studies (reliability not reported) “produced a statistically detectable mean effect size” whereas stronger design studies (reliability reported) did not (p. 226). Thus, in addition to exploring substantively and practically meaningful moderators described in the literature review, we will also investigate methodological moderators. In particular, building on previous research, we will investigate the extent to which individual study quality markers (reliability reported: e.g., Adesope et al., 2010; Plonsky, 2011; and pretest and random assignment implemented: Plonsky, 2011) covary with study ESs.

Data Sources

Data Coding

After running electronic and complementary literature searches, the titles and abstracts of identified studies were scanned using a Screening Guide that aligns with our inclusion criteria. The Screening Guide served as a basis for rater judgments regarding the likely relevance of the studies. If study relevance could not be determined from the readings of the titles and abstracts, full-length study reports were retrieved and reviewed. Final relevance decisions for all studies was based on the reading of the full texts.

The initial electronic and complementary literature searches yielded a total of 2,628 potentially relevant articles. Following a screening of titles and abstracts, 2,528 studies were removed. A full-text screening of the remaining 98 studies (2 studies could not be retrieved) was conducted independently by two authors; a third author was consulted in case of a disagreement. Reasons for excluding studies during the full-text screening process included irrelevant treatment (e.g., LLS were part of a larger scope intervention with no possibility of ascertaining the unique effects of LLS; n = 43), irrelevant design (e.g., two SI interventions were compared with no true control; results for language learners were reported in aggregate with those of native speaking participants; n = 15), and irrelevant outcomes (i.e., neither FL/L2 nor self-regulated learning outcomes were assessed; n = 1). The final sample of eligible studies across two research questions was 39: 37 studies (47 independent samples) for first research question and 16 studies (17 independent samples) for second research question (two studies were nonoverlapping, focusing exclusively on self-regulated learning outcomes).

Next, studies meting the inclusion criteria were coded by two independent raters using a pre-established coding protocol. The protocol was developed based on Valentine and Cooper’s (2003, 2008) recommendations for study quality assessment, piloted in a scoping review study, and refined by the research team. It included eight sections that captured information about the review process (e.g., reviewer and study identification), the study setting (e.g., educational level), sampling procedures (e.g., assignment to treatment), sample characteristics (e.g., age), treatment details (e.g., intervention duration), data collection (e.g., outcome measures), data analyses (e.g., unit of analysis), and the results (e.g., means, standard deviations). One of the outcomes of data extraction procedures is to generate a set of ESs to be synthesized in a meta-analysis. Another outcome is to code studies—that is, to generate a list of categorical codes along four dimensions: outcomes, learning contexts, structural features of SI treatments, and methodological characteristics of individual studies—for subsequent moderator analyses. After the initial training, the average interrater reliability on the coding protocol was about 92% with most disagreements falling under the treatment details category. All disagreements were resolved between the two coding authors, consulting with a third author as needed.

Data Analyses

Effect Size Estimation

The standardized mean difference (Cohen’s d) served as the measure of ES; its estimation for individual studies and integration across studies followed the common and best practices recommended by Cochrane Handbook (Higgins & Green, 2011). For each individual study, a Cohen’s d index (Lipsey & Wilson, 2001) was computed initially. This index is a standardized measure of difference between two group means divided by the pooled standard deviation. Cohen’s d index was computed as

d = Y ¯ T − Y ¯ C S p ,

where Y ¯ T is the treatment group mean, Y ¯ C is the comparison group mean, and S_p is the pooled standard deviation. Then, because of the inherent bias in the standardized mean difference ESs with small sample studies, we applied Hedges’s g (Hedges, 1981) as the usual correction for this issue. When both pretests and posttests were available, we used the adjusted posttest ESs by computing the “difference in differences” in the means from pretest to posttest. This strategy allowed for more comparable groups, thus addressing some of the shortcomings of studies with quasi-experimental designs:

d p o s t - p r e = ( Y ¯ p o s t T − Y ¯ p r e T ) − ( Y ¯ p o s t C − Y ¯ p r e C ) S p ,

where Y ¯ p o s t T is the treatment group mean at posttest, Y ¯ p r e T is the treatment group mean at pretest, Y ¯ p o s t C is the comparison group mean at posttest, Y ¯ p r e C is the comparison group mean at pretest, and S_p is the pooled (posttest) standard deviation. When primary studies presented only inferential statistics (e.g., F ratio), we used formulas given in Borenstein, Hedges, Higgins, and Rothstein (2009) to convert the statistics to the appropriate ES metric. For studies reporting results from analyses of covariance, we used adjusted means (if they were reported).

When studies reported more than one ES from the same sample (e.g., multiple measures of the same construct), we either selected the most relevant outcome (e.g., test rather than self-assessment) or used a weighted average ES (e.g., synthesizing results reported individually by LLS), so that each study only contributed one independent ES to the main analysis, and used the shifting unit of analysis approach (Cooper, 2010) when testing moderators. The shifting unit of analysis approach involves averaging effects when appropriate (e.g., averaging related language outcome effects when testing the overall SI effect), then splitting the effects when testing the dimension on which they differ. In this example, when asking whether program effects appear to be larger for one language outcome than for the other, a study with two or more language outcome measures contributed to both levels of that moderator.

SI Effectiveness for Language

Six contextual variables—setting, age, educational level, proficiency, and two language typology variables—were assessed with respect to their relationship with the effectiveness of SI on language learning (see Table 2). Moderate to large effects were obtained for all subgroups with the smallest effect being associated with the ESL setting (ES = 0.58) and the largest effect being associated with elementary education level (ES = 1.31). The overall trends in the results show that larger effects on language learning were obtained in foreign language (EFL, FL) settings, for younger learners (both in terms of age and educational level), for leaners above minimum proficiency, and for learners studying target languages belonging to the same family—but not necessarily having the same script—as their native languages.

In addition, six treatment-related variables were assessed (strategy scope, number of strategies, instructional procedures, duration, and two delivery variables). Medium to large effects were detected for all these subgroups with only two medium ES exceptions: SI interventions targeting more than eight strategies (ES = 0.55) and not focusing on metacognition (i.e., behavior modeling approach; ES = 0.57). The overall trends in the results show that larger effects on language learning were obtained for interventions that were shorter, focused on eight or fewer strategies, incorporated more than one strategy type, and used an awareness-raising approach. The effect for interventions targeting eight or fewer strategies (ES = 1.01) was significantly larger than that for interventions targeting more than eight strategies (ES = 0.55; Q = 6.08, p = .01).

Last, the three methodological moderators (reliability reported, random assignment, and pretest) were assessed. Medium to large effects were detected across different methodological features, with the only medium-sized effect associated with studies not using pretest (ES = 0.53) significantly smaller than that for studies using pretest (ES = 0.72; Q = 4.69, p = .03).

SI Effectiveness for Self-Regulated Learning

With regard to self-regulation, the moderating effects for all—but language family—contextual variables were statistically significant (see Table 2). Prior to further discussion of the results for this learning domain, however, it is worthwhile to note that moderator analyses for educational level, duration, and procedures should be interpreted with caution as at least one of the comparison groups has a small sample size (i.e., a single study).

Overall, contrasting with results for language learning, to some extent, larger effects were obtained in ESL settings, for older learners (both in terms of age and educational level), for leaners above minimum proficiency, and for learners studying target languages that differ from their native language to a greater extent (belonging to a different language family and not sharing the same script). A post hoc analysis conducted for a three-level settings moderator revealed that both the ESL and EFL effects were significantly larger than the FL effect (Q = 12.41, p < .001, and Q = 5.33, p < .05, respectively). There was no significant difference between the ESL and the EFL settings effects (Q = 2.45, p = .117). Another post hoc analysis for a three-level moderator revealed that the effect for university/adult learners was larger than those for middle/high school (Q = 8.19, p < .001) and elementary school (Q = 30.99, p < .001) learners. In addition, the effect for middle/high school learners was larger than that for elementary school learners (Q = 6.15, p < .05). Notably, the effect on self-regulated learning was significantly and substantively (by about 3.4 times) larger for older participants (12 years and older; ES = 0.98) than it was for younger participants (younger than 12 years; ES = 0.29; Q = 7.25; p = .01). A statistically and practically larger SI effects were found for learners studying languages with different from—rather than similar to—their native languages scripts (ESs of 1.12 and .26, respectively; Q = 9.83, p = .00).

For treatment-related variables, larger effects on self-regulated learning were obtained for interventions that were longer in duration, focused on a single strategy type, used an awareness-raising approach, were delivered by technology, and were researcher-led. Two of the moderating effects were statistically significant. Specifically, the effects were significantly larger for studies with SI focusing on only one strategy type (ES = 2.30) as contrasted with SI focusing on a combination of strategies (ES = 0.69; Q = 4.91; p = .03) and with SI being led by the researcher (ES = 0.97) as contrasted with SI being led by the teacher (ES = 0.31; Q = 9.75; p = .02).

For method, there was only one significant moderator effect associated with assignment to treatments. That is, studies employing random assignment designs reached a statistically larger ES (ES = 1.19) than did studies with nonrandom assignment designs (ES = 0.37; Q = 6.89; p = .01).

Correlations Among Moderators

Correlation analyses among statistically significant moderator results by domain and group indicated that for self-regulated learning, nonsurprisingly, age and educational level positively correlated, r = .54, p < .05. Interestingly, there was a significant negative correlation between proficiency and script, r = −.67, p < .001. Script was coded as 0 for different and 1 for same; proficiency was coded as 1 for low and 2 for intermediate/high. No other correlations for statistically significant moderator results were found.

Moderation by Context

Moderation by Treatment

The results regarding treatment characteristics highlight a number of variables affecting SI effectiveness. First, as expected, SI effects associated with awareness-raising approach tended to be larger than those associated with behavior-modeling approach. The lack of statistically significant difference, however, is somewhat unexpected and will be discussed later in the article. Furthermore, similar to Plonsky (2011), our meta-analysis provides strong support for less-is-more approach when it comes to deciding on the number of strategies to teach to improve learners’ language skills. Although focusing on a greater number of strategies appeared to benefit SI impacts on self-regulated learning to a greater extent, this advantage was not statistically significant. Rather, a significant less-is-more approach’s benefit for self-regulated learning outcomes demonstrated itself in studies that focused on a single type of LLS. This may be attributed to more focused instruction and a closer conceptual or procedural similarity among strategies of the same type making it easier for the learner to internalize.

Unlike Plonsky’s (2011) study, our results show a trend in means benefiting shorter interventions for improving language skills (but longer interventions for improving self-regulated learning skills). A lack of statistically significant difference between shorter versus longer intervention durations, however, suggests that both short-term (intensive) and longer term (incremental development over time) interventions may be equally beneficial for the learner and that the decision regarding the treatment length may be best decided by the teacher depending on learners’ needs. This finding—although far from being conclusive necessitating more longitudinal research with delayed posted to gauge long-term impacts—alleviates some of the earlier concerns regarding SIs taking valuable time away from direct language instruction (Bialystok & Hakuta, 1990, as cited in Plonsky, 2011; Rees-Miller, 1993) and provides support for current pedagogical recommendations for instruction balanced between language and (meta)cognitive skills (Chamot, 2009; Peregoy & Boyle, 2012).

With regard to SI delivery methods and agents, two interesting findings emerged. First, the statistically similar ESs for methods suggest that technology-delivered SI is not inferior to teacher-delivered SI. In fact, the trending in the means suggests greater benefits for the latter method, particularly when it comes to self-regulated learning outcomes. In turn, the use of technology to deliver SI has the potential to further alleviate the cost–benefit considerations noted earlier. That is, delivering SI entirely, or some aspects of it, on a computer (Ranalli, 2013; Urlaub, 2012) or using videos (Kim, 2013) has the potential to minimize demands on the teacher and on instructional time as learners may access these resources independently and proceed at their own pace. Additionally, “online SI might also facilitate more rigorous and ecologically valid research designs” by allowing for more accurate data collection of both products’ and processes’ data (Ranalli, 2013, p. 76). These findings, however, need to be treated with caution, as they are based only on a few studies using technology and there is clearly a need for more primary research in this area.

Second, the results indicate that, when it comes to language outcomes, researcher-led SI and teacher-led SI are equally effective, a nontrivial finding providing some support for ecological validity of SI research. However, the SI effect on self-regulated learning was significantly and sizably (by three times) smaller in teacher-led versus researcher-led interventions. This may be due to at least some of the educators’ coming from traditional, behavior-based teacher preparation programs. Educators with such preparation are likely to believe in the value of and be comfortable with providing LLS modeling and practice opportunities (basic components of the behavior-modeling approach), but may be less skilled with or willing to invest classroom time in consciousness-raising and effectiveness evaluations regarding LLS (two components essential for the awareness-raising model). This, in turn, is likely to lead to learners’ adopting LLS, but without either conscious awareness or attribution of their successes to LLS use.

Taken together, these considerations provide a plausible explanation to the lack of statistically significant differences between awareness-raising and behavior-modeling approaches noted earlier and highlight a need for further investigation of the issue in primary studies. From a practical perspective, these findings suggest a need for a greater emphasis on the awareness-raising model and on self-regulated learning and metacognition as underlying mechanisms of SI effectiveness in educator preparation and professional development programs as well as in SI curricular materials. From a research perspective, these findings highlight a need for further research on teachers’ implementing SI, in particular, it terms of “what teachers know and how they use it” (Cohen & Griffiths, 2015, p. 426).

Another concern regarding SI delivery agent is that the number of studies in which the researcher (or the teacher/researcher) led SI was notably larger than that of studies with regular teachers’ leading SI (respectively, about 53% and 21% of studies for language and about 47% and 24% for self-regulated learning) and a substantial number of studies did not clearly state who delivered SI. To better ascertain ecological validity of SI—after all, teachers are the ones who typically deliver SI—more research on teacher-led SI interventions is needed.

Conclusions and Recommendations

Given a continued interest in SI among educators and researchers (Cohen & Griffiths, 2015), a growing decline in FL-learning motivation (Grenfell & Harris, 2013; Macaro & Erler, 2008), and a simultaneous need to improve L2/FL proficiency “to boost stability, employability, and prosperity” in globalized world (British Council, 2013, p. 3), examining the extent to which SI may affect learners’ L2/FL learning and self-regulated learning is essential.

The overall results of this meta-analysis indicate that SI works for improving both learning domains. From a theoretical perspective, this study provides empirical grounding for a theoretically postulated need for SI to be directed toward developing dimensions of self-regulated learning (Zimmerman, 1990; see Grenfell & Macaro, 2007). From a practical point of view, this study offers several insights. In particular, this study (a) corroborates “empirical justification for integrating learner training programs into L2 curricula” (Plonsky, 2011, p. 1013); (b) provides guidelines for more effective SI designs by identifying student populations that are more prone to benefit from SI and treatment features that are associated with greater gains; and (c) suggests a need for a greater emphasis on self-regulated learning as an underlying mechanism of SI effectiveness in teacher preparation and professional development programs as well as in SI curricular materials.

Additional research is needed to further investigate the relationships among self-regulated learning, language learning, and SI. In particular, there is a need for primary studies that would instructionally incorporate and assess—in addition to language outcomes—all three dimensions (metacognitive, motivational, behavioral) of self-regulated learning and assess not only the direct effects of SI (SI on language learning and on self-regulated learning) but also its mediating effects (SI on language learning via self-regulated learning). When assessing the behavioral dimension of self-regulated learning, more primary research is needed to gage SI impacts on efficacy (e.g., Lam, 2009; Ranalli, 2013)—rather than on frequency—of LLS use. After all, L2 success does not depend on frequency of LLS use but rather on the learner’s ability to select and orchestrate LLS effectively (Macaro, 2006). Finally, more primary studies with teacher-led and technology-delivered interventions are needed to better ascertain ecological validity of SI and to further explore the benefits of brining technology to the field.