The reliability of language-switch costs in bilingual one- and two-digit number naming

Abstract

Aims and objectives/purpose/research questions:

Language-switch costs, which denote worse performance in language-switch than in language-repetition trials, appear to be a robust finding in bilingual language switching. The aim of the present study was to examine the intraindividual reliability of language-switch costs by means of a number-naming task with German-English bilinguals.

Design/methodology/approach:

In a cued language-switching paradigm, participants (n = 36) switched between German and English. They performed a number-naming task in three different conditions: one-digit numbers; two-digit numbers with decade 10; and two-digit numbers with decade 20.

Data and analysis:

We examined the experimental effects with an analysis of variance (reaction time and error rate as the dependent variables), using trial language, language sequence and number condition as independent variables. In addition, we calculated the split-half reliability of language-switch costs (across all conditions) as well as the correlations of language-switch costs between the different conditions.

Findings/conclusions:

While significant language-switch costs emerged in all three number conditions, our results demonstrate a medium-sized correlation between the three experimental conditions. The split-half reliability shows a moderate to strong correlation between the odd- and even-numbered trials in the experiment.

Originality:

On the one hand, the present study extends the observation of language-switch costs from one-digit number naming to the more complex naming of two-digit numbers. On the other hand, and theoretically even more important, we explored the reliability of language-switch costs in a bilingual number-naming task by calculating both split-half reliability and correlations between different number conditions.

Significance/implications:

The results indicated that while language-switch costs are a robust experimental effect on the group level, they appear to be less well suited for correlational approaches. This also suggests that caution should be exerted when language-switch costs are used to diagnose the ability of an individual to perform language control.

Keywords

Bilingualism language switching split-half reliability correlation

Introduction

Language-switching situations occur in our everyday life and even more so in an academic setting. Accordingly, there is a large research interest in the potential mechanisms of language control when participants switch between languages. Most of these studies use the language-switching paradigm in its different variants (for a review, see Declerck & Philipp, 2015). Irrespective of the individual experimental setting, the language-switching paradigm usually leads to the observation of language-switch costs, which reflect better performance in language-repetition trials as compared to language-switch trials (see, e.g. Meuter & Allport, 1999). Thus, language-switch costs are often seen as an empirical marker for language control processes (e.g. Declerck & Philipp, 2015; Green, 1998; Kroll et al., 2008).

Language switching

When we understand language-switch costs as a marker for language control processes, it is tempting to interpret the difference between language switches and language repetitions (i.e. language-switch costs) as a quantitative indicator for the involvement of language control processes. This, for example, becomes relevant when interpreting language-switch costs in terms of persisting inhibition (cf. the inhibitory control model of Green, 1998). The inhibitory control model assumes that language control processes in bilingual language-production tasks operate by means of inhibiting the currently irrelevant language. When a word has to be produced in one certain language, the other, non-required (i.e. non-target) language needs to be inhibited to allow production in the target language. If the target language was again relevant in the next trial (language-repetition trial), persisting inhibition of the non-target language helps language production in the target language. However, if the target language in the next trial was the previously inhibited language, the now relevant language still suffers from persisting inhibition. Activating the now relevant language and overcoming the persisting inhibition in such a language-switch trial takes time, which leads to a worse performance (slower reaction times (RTs) and more errors) in language-switch than in language-repetition trials—and thus to the occurrence of language-switch costs.

Language-switch costs have proven to be a very robust empirical finding in language switching, especially when switching between languages in a language-production task, such as digit naming or picture naming (for a review, see Declerck & Philipp, 2015; for a review on the neural basis of language control, see Calabria et al., 2018). Further, language-switch costs can be observed in different experimental paradigms, such as cued language switching, in which the relevant language for each trial is indicated by an explicit cue (e.g. Costa & Santesteban, 2004; Meuter & Allport, 1999; Philipp et al., 2007), language switching with pre-defined language sequences (e.g. Declerck, Koch, et al., 2015; Festman et al., 2010; Jackson et al., 2001) or voluntary language switching in which the participant decides which language to use in the next trial (e.g. Gollan & Ferreira, 2009; Gollan et al., 2014; Gross & Kaushanskaya, 2015). Furthermore, language-switch costs occur when the stimuli are presented visually (as in most studies), but they also occur with auditory stimulus presentation (i.e. sound naming, cf. Declerck, Stephan, et al., 2015). In summary, language-switch costs appear to be a very robust finding in language switching.

Language-switch costs as individual-differences measures

Given that language-switch costs apparently represent a robust finding in language switching, it is tempting to believe that they could also be useful as an individual-differences measure of bilingual switching ability. Thus, if the relative size of language-switch costs represents the language control abilities of an individual, those individuals with large language-switch costs in one language-switching experiment should also show large language-switch costs in a different language-switching experiment.

Further, if switching between languages was controlled by a rather general ability to switch among changing situations, those individuals with large language-switch costs in a language-switching experiment should also have large switch costs when switching between non-linguistic tasks, such as categorizing the shape or the colour of an object (i.e. task switching). Yet, in studies in which task-switch costs and language-switch costs were correlated within the same participants (e.g. Branzi, Calabria, et al., 2016; Calabria et al., 2015; Declerck et al., 2017; Klecha, 2013; Prior & Gollan, 2013; Timmer et al., 2018), the data appear relatively mixed. Results ranged from no correlation (Branzi, Calabria, et al., 2016) and partial correlations (Calabria et al., 2012; Klecha, 2013; Prior & Gollan, 2013) to significant correlations (Declerck et al., 2017; Timmer et al., 2018). The study of Declerck et al. (2017) additionally demonstrated that the correlation coefficient increased from r = .383 to r = .572 when the task-switching and the language-switching paradigm became more similar. Taken together, these studies do not support the notion that language switching and task switching, that is, language control and cognitive control, are controlled by the very same mechanism (see also Calabria et al., 2012; Jylkkä et al., 2018; Segal et al., 2019)—but they indicate that there might be some overlap. Accordingly, a number of studies demonstrated a partial overlap in the neural networks activated in language control and cognitive control (see, e.g. Branzi, Della Rosa, et al., 2016; De Baene et al., 2015; Timmer et al., 2017; Weissberger et al., 2015; for a review, see Calabria et al., 2018).

However, before interpreting a possible correlation between language-switch costs and task-switch costs, that is, before taking the relative size of language-switch costs as a measurement that can describe inter-individual differences, it would be important to first know more about the reliability of language-switch costs. A first empirical examination was provided by Timmer et al. (2018), who demonstrated a good test–retest reliability of language-switch costs (r = .739). The present study aimed at extending this line of research by calculating the split-half reliability of language-switch costs as well as the correlation of language-switch costs in different conditions of a number-naming task.

Reliability of individual-differences measures

Reliability of individual-differences measures refers to the degree of consistency of scores or measures of the same thing by the same individuals. In this way, the reliability is directly related to the usefulness of a measurement tool or empirical marker—as, for example, language-switch costs in the present study (for a review, see Berchtold, 2016; Guttman, 1945; Howell, 2009; Yen & Lo, 2002). The most common methods employed for estimation of reliability are the test–retest reliability and the split-half reliability (see, e.g. Cole et al., 1989; Doty et al., 1985; Ezpeleta et al., 1997; Serra-Mayoral & Peña-Casanova, 2006; Wear & Pratz, 1987). The test–retest reliability can be used to describe the properties of scores or measurement tools evaluated twice on different occasions (for test–retest reliability of language-switch costs, see Timmer et al., 2018). For calculating the split-half reliability, the test (or experiment) is divided into two equivalent halves, for example by separating even and odd trials, and the score or measurement (e.g. language-switch costs) in odd versus even trials are correlated.

Importantly, the obtained correlation depends on the test length (i.e. the number of trials) so that the correlation is often corrected for test length by applying the Spearman–Brown formula (Bland & Altman, 1997; Gliem & Gliem, 2003). By increasing the number of test items/trials, the reliability of the test increases. The number of trials examined in studies correlating task-switch costs and language-switch costs was variable, from about 70 repetition trials and switch trials (Branzi, Calabria, et al., 2016; Prior et al., 2013) to 256 repetition trials and 64 switch trials (Calabria et al., 2015) or 280 repetition trials and 120 switch trials (Klecha, 2013). In the study of Timmer et al. (2018), the test–retest correlation of language-switch costs included approximately 70 language-repetition trials and 250 language-switch trials.

In the present study, we intend to have an equal distribution of repetition and switch trials with a total of 540 trials per participant (i.e. about 270 repetition trials and 270 switch trials; please note that the first trial in each block is neither a switch nor a repetition trial). This equal probability of language-repetition and -switch trials results from a major difference between the present study and that by Timmer et al. (2018). While their study included switching among three languages, participants in the present study only switched between two languages (German and English). This different set-up did not only increase the number of language-repetition trials (resulting in an equal distribution of repetition and switch trials), but it also allowed us to study language-switch costs without the influence of n−2 repetition costs. When switching among three languages, n−2 repetition costs can be measured as performance difference in a language A between ABA and CBA language sequences (e.g. Philipp et al., 2007). In the task-switching literature, there is some evidence that the size of n−2 repetition costs was reduced by introducing task repetitions (Philipp & Koch, 2006; Scheil & Kleinsorge, 2019), indicating that the mechanisms behind n−1 switch costs and n−2 repetition costs are not identical and might even influence each other. Thus, switching among three languages (rather than only two) may have an influence on language-switch costs, which we avoid in the present design.

There is another important difference between the study of Timmer et al. (2018) and the present study. That is, whereas Timmer et al. (2018) used a picture naming task, in the present study, we aimed at exploring language switching in a number-naming task. Just as is the case for the number of languages, also the type of stimuli may have an effect on language switching. Declerck et al. (2012) observed that number naming resulted in smaller switch costs than picture naming and suggested that the difference in the language-switch costs can be attributed to phonological priming, as most one-digit numbers are cognates between languages (German and English in this study).

Next to one-digit numbers (i.e. the digits from 1 to 9; cf. Declerck et al., 2012), we also used blocks with two-digit numbers (i.e. the digits from 11 to 19 and from 21 to 29) in the present study. For two-digit numbers, the complexity is higher than for one-digit numbers as, in the number word, also the decade and the unit position have to be considered. Thus, when switching between two languages using two-digit numbers, performance may not be affected only by language switching but also by considering the correct composition rule of the number (i.e. naming the unit or the decade first).¹ If the size of language-switch costs was representative of the necessary control processes to perform a specific condition, switch costs might differ in two-digit number naming compared with one-digit number naming. Yet, we suppose that language control is necessary for language switching irrespective of number-naming complexity. In this way, we expect to observe language-switch costs in all three conditions and the size of language-switch costs should be correlated between conditions (i.e. participants with relatively small language-switch costs in one condition should also have relatively small language-switch costs in the other conditions).

Taken together, we examined the reliability of language-switch costs on the one hand by calculating the split-half reliability (odd versus even trial numbers) of the complete experiment and, on the other hand, by correlating the size of language-switch costs in the three different conditions (i.e. switching between one-digit numbers, two-digit numbers with decade 10 and two-digit numbers with decade 20). With this design, we did not only intend to replicate the study of Timmer et al. (2018) but to generalize it in several points: Firstly, we calculate the split-half reliability and correlations between different conditions rather than the test–retest reliability. Secondly, participants switched between only two languages (rather than three languages as in Timmer et al., 2018), so that we could calculate language-switch costs based on an equal distribution of language-switch and language-repetition trials. Thirdly, instead of picture naming, we used number naming and also included a manipulation of number-naming complexity by using one-digit and two-digit numbers.

Method

Participants

Bilingual native German participants were included in this study (n = 36; three male and 33 females with an average age of 21 years; the handedness distribution determined via self-report of participants was three left- and 33 right-handers). All participants were students of psychology at RWTH Aachen University and received partial course credit for participation. Participants switched between German and English. We assessed the level of language proficiency using the Lexical Test for Advanced Learners of English, LexTALE (Lemhöfer & Broersma, 2012) for English and German. Participants reached on average a score of 85% (range: 58–94%) in German and 66% (range: 53–83%) in English. The age of acquisition for English was between 4 and 14 years.

Materials

The stimulus set consisted of nine one-digit numbers (1–9), nine two-digit numbers with decade 10 (11–19) and nine two-digit numbers with decade 20 (21–29). All numbers were presented in Arabic number format in black colour in the center of the screen. We used Arial font, font size 20, which resulted in an approximate size of 2 cm height/1.5 cm width for single-digit numbers and 2 cm height/2.5 cm width for two-digit numbers. We used coloured flag images (with a size of 1.5 cm height × 3.5 cm width) presented above the number stimuli as cues for the language to be used in a trial.

Stimulus and cue presentation was controlled using E-Prime 2.0 software. All stimuli and cues were presented against a white background on a LG Flatron L1710B monitor (17 inches) at a viewing distance of approximately 60 cm. A microphone was used to record participants’ vocal responses (voice onset time). Errors were coded by the experimenter offline, using a list with the correct answers. We considered two kinds of errors: response errors and microphone errors. Response errors are all answers where participants used an incorrect language, named an incorrect number, where the utterance was unintelligible or where the answer was a language combination. A microphone error was noted when an external sound (e.g. a sneeze or a loud breath) activated the microphone, replacing a proper answer, or when the answer was not loud enough to activate the microphone at the start of the response, recording a wrong RT.

Procedure

The language cue was visually presented for 100 ms before the stimulus (number in Arabic format) was visually presented. The cue and stimulus remained on the screen until a voice key response was registered (i.e. no response deadline). After a blank screen for 900 ms, the next trial started. The experiment was run in a single session of approximately 45 minutes. Instructions were given both via the monitor and orally. Participants were informed that they would have to name the one-digit number or the two-digit number on the screen in the language that was indicated by the cue. The instruction emphasized speed as well as accuracy.

The experiment contained a short practice block with nine trials and six experimental blocks with 90 trials each. Out of the six experimental blocks, two consecutive blocks included one-digit numbers (1–9), two consecutive blocks included two-digit numbers with decade 10 (11–19) and two consecutive blocks included two-digit numbers with decade 20 (21–29). The order of these three conditions was counterbalanced across participants. The sequence of languages in each block was unpredictable for the participants but had the same number of German and English trials in each block. The sequence of trials was further controlled for an equal number of languages switches and language repetitions. Immediate number repetitions were avoided. Based on these restrictions, 18 novel trial sequences were created (i.e. each sequence was used by two participants).

Design

Data analysis proceeded in two parts. Firstly, we examined the experimental effects. We conducted a 2×2×3 analysis of variance (ANOVA) with repeated measures. Language sequence (language repetition versus language switch), trial language (German versus English) and number condition (one-digit numbers versus two-digit numbers with decade 10 versus two-digit numbers with decade 20) were used as within-subjects variables. The RT and error rate were the dependent variables.

Secondly, we conducted a split-half reliability analysis that included all odd-numbered trials (split-half A) and all even-numbered trials (split-half B) as well as a correlation analysis on the size of switch costs across the three number conditions. We performed these reliability analyses on the size of language-switch costs in the RT data.

Results

For all analyses, the first trial of each block (1.1%), microphone errors and trials following a microphone error (3.0%) and trials with a RT below 100 ms (0.3%) or above 3000 ms (<0.1%) were discarded. In RT analyses, also error trials and trials following an error were excluded (6.5%). For the RT, 89% of the data were included in the analyses, while 95% of the data were included in the error rate analysis.

Experimental effects

The 2×2×3 ANOVA included language sequence (language repetition versus language switch), trial language (German versus English) and number condition (one-digit numbers versus two-digit numbers with decade 10 versus two-digit numbers with decade 20) as within-subjects independent variables. We observed a significant main effect of language sequence in the RT analysis (F(1,35) = 102.53, p < .001, η²_p = .746) and for error rate analysis (F(1,35) = 6.50, p = .015, η²_p = .157). The performance in language-repetition trials was better (585 ms/2.9%) than in language-switching trials (640 ms/3.8%), demonstrating language-switch costs (55 ms/0.9%; see Table 1).

Table 1.

Overall mean reaction time (RT) in ms and mean error rate percentage (ER) (and standard deviation) as a function of language sequence (language switch versus language repetition), trial language (German versus English) and number condition (one-digit numbers versus two-digit numbers with decade 10 versus two-digit numbers with decade 20).

		Language switch		Language repetition
		RT	ER	RT	ER
German language
One-digit number	Mean	624 ms	4.7%	570 ms	3.2%
One-digit number	SD	94	7.2%	77	3.4%
Two-digit number 10	Mean	647 ms	4.0%	580 ms	3.4%
Two-digit number 10	SD	101	3.9%	80	3.7%
Two-digit number 20	Mean	650 ms	3.1%	598 ms	2.3%
Two-digit number 20	SD	97	3.6%	85	2.7%
English language
One-digit number	Mean	619 ms	4.0%	572 ms	3.3%
One-digit number	SD	89	3.8%	70	4.2%
Two-digit number 10	Mean	645 ms	3.8%	605 ms	2.8%
Two-digit number 10	SD	108	4.2%	86	4.0%
Two-digit number 20	Mean	654 ms	3.2%	582 ms	2.2%
Two-digit number 20	SD	107	4.2%	95	3.5%

We observed a significant main effect of number condition in the RT analysis (F(2,70) = 6.01, p = .004, η²_p = .147) and for error rate analysis (F(2,70) = 4.57, p = .014, η²_p = .116), but these represent a speed–accuracy tradeoff, with faster but more erroneous responses for one-digit numbers (596 ms/3.8%) then for two-digit numbers with decade 10 (620 ms/3.5%) and two-digit numbers with decade 20 (621 ms/2.7%). The main effect of trial language was not significant for either RT (F < 1) or error analysis (F < 1).

The two-way interaction between trial language and number condition was significant for the RT analysis (F(2,70) = 4.28, p = .023, η²_p = .109) but not for the error analysis (F < 1). While RTs numerically increased from one-digit numbers (597 ms) to two-digit numbers with decade 10 (614 ms) and two-digit numbers with decade 20 (624 ms) for German trials, two-digit numbers with decade 20 (618 ms) showed a shorter RT than two-digit numbers with decade 10 (625 ms) in English trials. Still, two-digit numbers had a longer RT than one-digit numbers (596 ms) in English trials.

In addition, this data pattern was qualified by the three-way interaction between trial language, language sequence and number condition, which was significant for RT (F(2,70) = 7.37, p = .001, η²_p = .174) but not for error analysis (F < 1). While the language-switch costs were significant in each condition,² the RT pattern indicates that for both one-digit numbers and two-digit number numbers with decade 10 the language-switch costs were numerically larger for German than for English (54 ms versus 47 ms and 67 ms versus 40 ms, respectively), while the pattern was reversed for two-digit number numbers with decade 20, indicating smaller language-switch costs in German (52 ms) than in English (72 ms). Please note that the two-way interaction between language sequence and trial language was not significant (F(1,35) = .506, p < .482, η²_p = .014 for the RT analysis and F(1,35) = .001, p < .976, η²_p = .000, for the error analysis), indicating that across all number conditions the language-switch costs for German trials (58 ms) were only marginally larger than for English trials (53 ms).

Reliability analyses

We calculated the split-half reliability (odd versus even trials) of the overall language-switch cost, averaged across the number conditions and across languages. To this end, language-switch costs were calculated separately for odd-numbered and even-numbered trials and then correlated. The Pearson’s product-moment correlation between the two halves was r = .707. The Spearman–Brown coefficient, which provides an estimate of reliability of the test as a whole (i.e. in this case, the experiment consisting of 540 trials in total), equals r_c = .828.

In addition, we conducted analyses over the RT language-switch costs of each number-naming condition (see Table 2). The correlation between one-digit numbers and two-digit numbers with decade 10 was r = .576, p < .001 (see Figure 1), the correlation between one-digit numbers and two-digit numbers with decade 20 was r = .597, p < .001 (see Figure 2) and the correlation between two-digit numbers with decade 10 and two-digit numbers with decade 20 was r = .537, p < .001 (see Figure 3).

Table 2.

Pearson correlations (r) of language-switch costs in reaction time between the different number conditions (one-digit number versus two-digit number decade 10 versus two-digit number decade 20).

	One-digit numbers	Two-digit decade 10	Two-digit decade 20
One-digit numbers	1	.576**	.597**
Two-digit numbers decade 10		1	.537**
Two-digit numbers decade 20			1

Correlation is significant with p < 0.01 (one-tailed).

Figure 1.

Scatterplot visualizing the correlation of language-switch costs in the conditions with one-digit numbers (mean language-switch cost across languages, 50 ms) and two-digit numbers with decade 10 (mean language-switch cost across languages, 52 ms). Each data point represents the values of one individual.

Figure 2.

Scatterplot visualizing the correlation of language-switch costs in the conditions with one-digit numbers (mean language-switch cost across languages 50 ms) and two-digit numbers with decade 20 (mean language-switch cost across languages 62 ms). Each data point represents the values of one individual.

Figure 3.

Scatterplot visualizing the correlation of language-switch costs in the conditions with two-digit numbers with decade 10 (mean language-switch costs across languages 52 ms) and two-digit numbers with decade 20 (mean language-switch costs across languages 62 ms). Each data point represents the values of one individual.

Discussion

The present study examined language switching in a number-naming task in three different naming conditions: naming one-digit numbers; naming two-digit number with decade 10; and naming two-digit number with decade 20. The data pattern demonstrated language-switch costs in all three conditions. Thus, in the present study, we extended the observation of language-switch costs from number naming with one-digit numbers (cf. Declerck et al., 2012) to the more complex naming of two-digit numbers.

Importantly, the main aim of the present study was to examine the reliability of language-switch costs (cf. Timmer et al., 2018). Our results demonstrate a medium to high correlation between the three experimental conditions (number naming with one-digit numbers versus two-digit numbers with decade 10 versus two-digit numbers with decade 20). In addition, the split-half reliability shows a moderate to strong correlation between the two “test” halves (i.e. between the odd- and even-numbered trials in the experiment).

That is, along with the highly robust observation of language-switch costs (averaged across participants) in our experiment and in other studies, language-switch costs showed moderate to high evidence of reliability when applied as an inter-individual differences measure (i.e. split-half reliability was moderate to strong (r = .707)). Timmer et al. (2018) observed a similar, but stronger, test–retest reliability of language-switch costs (r = .739). The reasons for this difference between our study and that by Timmer et al. (2018) might be due to a different measure (split-half reliability versus test–retest reliability), a different proportion of language-switch and repetition trials, the different stimulus material (pictures versus numbers) or the difference in the sample size (i.e. 36 participants in this study and 53 participants in the study by Timmer et al., 2018). However, as the sample size in both studies is relatively small, the difference in reliability scores should not be over-interpreted. Thus, we suppose that both studies—despite the differences in the specific designs—can be summarized as showing a moderate to strong reliability of language-switch costs.

Yet, the present data pattern is also in line with an observation reported by Hedge et al. (2018). In their study, the authors describe a reliability paradox, which means that empirical measures that are very robust on a group level had relatively weaker reliability when used as inter-individual difference measures. More specifically, language-switch costs have proven to be a very robust finding in language-switching studies (for a review, see Declerck & Philipp, 2015)—yet, they might be less well suited to be used at the individual level. Thus, language-switch costs should only be used very carefully when used to diagnose the ability of an individual to perform language control. The (only) moderate to strong reliability of language-switch costs should also be kept in mind in studies that correlate the size of language-switch costs to, for example, the size of task-switch costs, as the reliability constitutes an upper bound for any such correlations (for a corresponding discussion, see Timmer et al., 2018).

In the present study, we also calculated the reliability of language-switch costs across different number conditions. Especially when referring to the correlation across different conditions, it is important to take into account that the conditions were similar in the sense that they all requested number naming. However, there was also a difference with respect to the complexity of the number names (see Contreras-Saavedra et al., 2020). Whereas the names of one-digit numbers have no specific composition rule, a language-specific composition rule has to be followed when naming two-digit numbers. The data pattern shows that the size of switch costs was larger for a bilingual naming task involving two-digit numbers (especially with decade 20) than in a naming task involving one-digit numbers. This may suggest that the complexity of two-digit numbers, involving two constituents (unit and decade), might add an additional influence on language switching (Contreras-Saavedra et al., 2020). Although this difference was small, it might nevertheless indicate that language-switch costs are not a pure empirical marker for language control processes, but always depend on a multitude of factors specific for each individual setting. This notion is in line with the concept of task impurity suggested by Miyake et al. (2000). These authors already mentioned that different aspects of cognitive or executive control are never pure measures, as each task includes different control mechanisms. Thus, when adding the necessity to take into account the composition rule of a two-digit number (which also differs between German and English), additional cognitive control processes come into play that influence the size of switch costs. As these numerical control processes could be relatively independent from the language control processes, combining both factors could reduce the correlation between conditions as compared to a direct repetition of a condition (i.e. when measuring test–retest reliability).

Yet, the additional influence might not have been due to the increased morphological complexity of the two-digit numbers but due to phonological priming effects. In the present set of stimuli, phonological priming might have taken place not only between languages (as, for example, most one-digit numbers are cognates in German and English), but also within languages. More specifically, within-language priming might have taken place in English language-repetition trials in the two-digit number condition with decade 20, as all number names start with the decade morpheme twenty (i.e. 22, 23, etc.). That is, there might be an additional benefit in this specific condition for repeat trials in English as the onset of the two responses was identical. This suggestion is supported by the present results (see Table 2), as the relative high switch costs in this condition seems to be driven by fast responses in repetition trials rather than by slow responses in switch trials. This interpretation could also explain the reversal of the switch-cost asymmetry from larger switch costs for the dominant language in the one-digit numbers and two-digit numbers with decade 10 conditions to smaller switch costs for the dominant language in the two-digit numbers with decade 20 condition. However, please note that this is a post-hoc explanation and the possible influence of phonological priming in language switching and its potential influence on the reliability of language-switch costs should be explored in future studies.

Conclusion

Our investigation examined the reliability of the bilingual number-naming task with one- and two-digit numbers to assess language switching. In all number conditions, we replicated the switch costs in language switching that were already reported in previous language-switching studies (see Declerck & Philipp, 2015, for a review). Split-half reliability revealed a moderate to high level of consistency of language-switch costs in the present number-naming task. Also, correlations between the different number-naming conditions (one-digit versus two-digit numbers) were moderate to high. Together, these results suggest that language-switching costs represent a robust experimental finding on the group level that may be, albeit to a lower degree, suited for correlational approaches at the individual level.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by a DAAD/BECAS Chile Grant to Carla Contreras.

ORCID iD

Carla E Contreras-Saavedra

Notes

Author biographies

Carla E Contreras-Saavedra is PhD student at the Institute of Psychology, RWTH Aachen University, Germany. Her research focuses on language switching and multilingual two-digit number processing.

Iring Koch is full professor of psychology at RWTH Aachen University, Aachen, Germany since 2005. In 1998 he received his Ph.D. of the University of Würzburg, Würzburg, Germany. His main research interests are in cognitive control, bilingualism, action, and learning.

Stefanie Schuch is a senior research scientist and interim professor of work and cognitive psychology at the Institute of Psychology, RWTH Aachen University, Germany. Her research focuses on cognitive control, including inhibitory control in task switching as well as methodological questions.

Andrea M Philipp is a senior research scientist at the Institute of Psychology, RWTH Aachen University, Germany. Her research focuses on cognitive control and multilingual language control.

References

Berchtold

(2016). Test–retest: Agreement or reliability? Methodological Innovations, 9, 2059799116672875.

Bland

J. M.

Altman

D. G.

(1997). Statistics notes: Cronbach’s alpha. BMJ, 314(7080), 572.

Branzi

F. M.

Calabria

Boscarino

M. L.

Costa

(2016). On the overlap between bilingual language control and domain-general executive control. Acta Psychologica, 100(166), 21–30.

Branzi

F. M.

Della Rosa

P. A.

Canini

Costa

Abutalebi

(2016). Language control in bilinguals: Monitoring and response selection. Cerebral Cortex, 26(6), 2367–2380.

Calabria

Branzi

F. M.

Marne

Hernández

Costa

(2015). Age-related effects over bilingual language control and executive control. Bilingualism: Language and Cognition, 18(1), 65–78.

Calabria

Costa

Green

D. W.

Abutalebi

(2018). Neural basis of bilingual language control. Annals of the New York Academy of Sciences, 1426(1), 221–235.

Calabria

Hernández

Branzi

F. M.

Costa

(2012). Qualitative differences between bilingual language control and executive control: Evidence from task-switching. Frontiers in Psychology, 2, 399.

Cole

K. N.

Mills

P. E.

Dale

P. S.

(1989). Examination of test-retest and split-half reliability for measures derived from language samples of young handicapped children. Language, Speech, and Hearing Services in Schools, 20(3), 259–268.

Contreras-Saavedra

C. E.

Willmes

Koch

Schuch

Benini

Philipp

A. M.

(2020). Multilingual two-digit number naming: The influence of composition rules on language switching. Quarterly Journal of Experimental Psychology, 73(9), 1481–1494. https://doi.org/10.1177/1747021820916108

10.

Costa

Santesteban

(2004). Lexical access in bilingual speech production: Evidence from language switching in highly proficient bilinguals and L2 learners. Journal of Memory and Language, 50(4), 491–511.

11.

De Baene

Duyck

Brass

Carreiras

(2015). Brain circuit for cognitive control is shared by task and language switching. Journal of Cognitive Neuroscience, 27(9), 1752–1765.

12.

Declerck

Grainger

Koch

Philipp

A. M.

(2017). Is language control just a form of executive control? Evidence for overlapping processes in language switching and task switching. Journal of Memory and Language, 100(95), 138–145.

13.

Declerck

Koch

Philipp

A. M.

(2012). Digits vs. pictures: The influence of stimulus type on language switching. Bilingualism: Language and Cognition, 15(4), 896–904.

14.

Declerck

Koch

Philipp

A. M.

(2015). The minimum requirements of language control: Evidence from sequential predictability effects in language switching. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(2), 377–394.

15.

Declerck

Philipp

A. M.

(2015). A review of control processes and their locus in language switching. Psychonomic Bulletin & Review, 22(2), 1630–1645.

16.

Declerck

Stephan

D. N.

Koch

Philipp

A. M.

(2015). The other modality: Auditory stimuli in language switching. Journal of Cognitive Psychology, 27(6), 685–691.

17.

Doty

R. L.

Newhouse

M. G.

Azzalina

J. D.

(1985). Internal consistency and short-term test-retest reliability of the University of Pennsylvania Smell Identification Test. Chemical Senses, 10(3), 297–300.

18.

Ezpeleta

De La Osa

Domenech

J. M.

Navarro

J. B.

Losilla

J. M.

(1997). Fiabilidad test-retest de la adaptación española de la Diagnostic Interview for Children and Adolescents (DICA-R). Psicothema, 9(3), 529–539.

19.

Festman

Rodriguez-Fornells

Münte

T. F.

(2010). Individual differences in control of language interference in late bilinguals are mainly related to general executive abilities. Behavioral and Brain Functions, 6(1), 5.

20.

Gliem

J. A.

Gliem

R. R.

(2003). Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales. In Midwest research-to-practice conference in adult, continuing, and community education. Ohio State University, 1992, 82–88. https://scholarworks.iupui.edu/bitstream/handle/1805/344/Gliem+&+Gliem.pdf?sequence=1

21.

Gollan

T. H.

Ferreira

V. S.

(2009). Should I stay or should I switch? A cost–benefit analysis of voluntary language switching in young and aging bilinguals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(3), 640–665.

22.

Gollan

T. H.

Kleinman

Wierenga

C. E.

(2014). What’s easier: Doing what you want, or being told what to do? Cued versus voluntary language and task switching. Journal of Experimental Psychology: General, 143(6), 2167–2195.

23.

Green

D. W.

(1998). Mental control of the bilingual lexico-semantic system. Bilingualism: Language and Cognition, 1(2), 67–81.

24.

Gross

Kaushanskaya

(2015). Voluntary language switching in English–Spanish bilingual children. Journal of Cognitive Psychology, 27(8), 992–1013.

25.

Guttman

(1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255–282.

26.

Hedge

Powell

Sumner

(2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186.

27.

Howell

D. C.

(2009). Statistical methods for psychology. Cengage Learning, Thomson Wadsworth.

28.

Jackson

G. M.

Swainson

Cunnington

Jackson

S. R.

(2001). ERP correlates of executive control during repeated language switching. Bilingualism: Language and Cognition, 4(2), 169–178.

29.

Jylkkä

Lehtonen

Lindholm

Kuusakoski

Laine

(2018). The relationship between general executive functions and bilingual switching and monitoring in language production. Bilingualism: Language and Cognition, 21(3), 505–522.

30.

Klecha

(2013). Language and task switching in Polish-English bilinguals. Psychology of Language and Communication, 17(1), 17–36.

31.

Kroll

J. F.

Bobb

S. C.

Misra

Guo

(2008). Language selection in bilingual speech: Evidence for inhibitory processes. Acta Psychologica, 128(3), 416–430.

32.

Lemhöfer

Broersma

(2012). Introducing LexTALE: A quick and valid lexical test for advanced learners of English. Behavior Research Methods, 44(2), 325–343.

33.

Meuter

R. F. I.

Allport

(1999). Bilingual language switching in naming: Asymmetrical costs of language selection. Journal of Memory and Language, 40(1), 25–40.

34.

Miyake

Emerson

M. J.

Friedman

N. P.

(2000). Assessment of executive functions in clinical settings: Problems and recommendations. Seminars in Speech and Language, 21(2), 169–183.

35.

Philipp

A. M.

Gade

Koch

(2007). Inhibitory processes in language switching: Evidence from switching language-defined response sets. European Journal of Cognitive Psychology, 19(3), 395–416.

36.

Philipp

A. M.

Koch

(2006). Task inhibition and task repetition in task switching. European Journal of Cognitive Psychology, 18(4), 624–639.

37.

Prior

Gollan

T. H.

(2013). The elusive link between language control and executive control: A case of limited transfer. Journal of Cognitive Psychology, 25(5), 622–645.

38.

Scheil

Kleinsorge

(2019). Effects of global and local task repetition proportion on n−2 repetition costs. Quarterly Journal of Experimental Psychology, 72(3), 579–588.

39.

Segal

Stasenko

Gollan

T. H.

(2019). More evidence that a switch is not (always) a switch: Binning bilinguals reveals dissociations between task and language switching. Journal of Experimental Psychology: General, 148(3), 501–519.

40.

Serra-Mayoral

Peña-Casanova

(2006). Test-retest and interrater reliability of Barcelona test. Neurología, 21(6), 277–281.

41.

Timmer

Calabria

Branzi

F. M.

Baus

Costa

(2018). On the reliability of switching costs across time and domains. Frontiers in Psychology, 9, 1032.

42.

Timmer

Grundy

J. G.

Bialystok

(2017). Earlier and more distributed neural networks for bilinguals than monolinguals during switching. Neuropsychologia, 106, 245–260.

43.

Wear

R. W.

Pratz

(1987). Test-retest reliability for the eating disorder inventory. International Journal of Eating Disorders, 6(6), 767–769.

44.

Weissberger

G. H.

Gollan

T. H.

Bondi

M. W.

Clark

L. R.

Wierenga

C. E.

(2015). Language and task switching in the bilingual brain: Bilinguals are staying, not switching, experts. Neuropsychologia, 66, 193–203.

45.

Yen

L. H.

(2002). Examining test-retest reliability: An intra-class correlation approach. Nursing Research, 51(1), 59–62.