Abstract
This article examined the integration of semantic and morphosyntactic information by Korean learners of English as a second language (L2). In Experiment 1, L2 learners listened to English active or passive sentences that were either plausible or implausible and translated them into Korean. A significant number of Korean translations maintained the original passive/active structure, but switched the thematic roles of the actors in the sentences. In Experiment 2, the direction of translation was reversed and participants made very few translation errors, showing that the errors in Experiment 1 were not due to participants’ lack of control over the English passive morphosyntax. The results are strikingly similar to previous results in the first language (L1) psycholinguistics literature, and support a view of L2 processing (like L1 processing) that is ‘good enough’ in nature: misinterpretations arise from only a ‘good enough’ integration of semantic and morphosyntactic information in the input.
Keywords
I Introduction
It has often been reported that it is common for adult second language (L2) learners to face difficulties in ultimate attainment, unlike first language (L1) acquisition. A great deal of second language acquisition (SLA) research has focused on understanding what s learned or what is not learned by non-native speakers and the development of their linguistic system (competence). These studies have often been informed by offline grammaticality judgment tasks, comprehension, or L2 production tasks. Recent studies have started to examine whether the difficulties in ultimate attainment may lie instead in different underlying processing systems from native speakers (performance). SLA researchers have begun to shift focus from L2 linguistic knowledge to L2 processing, using various time-sensitive psycholinguistic methods such as self-paced reading, eye-tracking, and event-related brain potentials (ERPs) to explore on-line L2 processing mechanisms (Dussias, 2010; Felser et al., 2003; Frenck-Mestre, 2005).
A critical question about L2 sentence comprehension and processing is how L2 learners process different types of linguistic information during on-line comprehension compared to native speakers. The current study adds to this line of research investigating how Korean learners of English comprehend L2 active/passive sentences in a translation paradigm. Many studies have investigated this question by examining non-native speakers’ ambiguity attachment resolution (Dussias, 2001; Felser et al., 2003; Papadopoulou and Clahsen, 2003), processing of wh-dependencies (Marinis et al., 2005), or sensitivity to morphosyntactic information (Hopp, 2006; Jackson, 2008; Jiang, 2004, 2007). Although there has been extensive recent work on L2 morphosyntactic processing, results have still not converged to agreement on the issue of whether L2 processing is qualitatively different from native processing, specifically with regard to grammatical processing. While some studies have concluded that non-native speakers necessarily underuse syntactic information during on-line processing compared to native speakers due to real-time computational constraints inherent in L2 processing (Felser et al., 2003; Marinis et al., 2005), others have shown that L2 learners are able to process linguistic input similarly to native speakers (Frenck-Mestre and Pynte, 1997; Hopp, 2006; Jackson and Dussias, 2009; Juffs, 1998). This discrepancy in existing research necessitates more behavioral studies with various types of structures, and considerations of other factors, such as different populations, proficiency levels, task differences, and individual differences, all of which should be considered if L2 processing mechanisms are to be fully understood.
Furthermore, even though reading time data provide considerable information about L2 speakers’ reading patterns, it is questionable whether reading time data alone fully represent how L2 speakers integrate different types of linguistic information during comprehension. In this respect, an application of L2 learners’ unique linguistic activity, namely translation, may be useful in examining the integration of linguistic information when processing an L2.
The present study aims to address the question of how Korean learners of English integrate syntactic and semantic information into a single, coherent representation via a translation paradigm. In two experiments, the information sources used by L2 learners and the extent to which they are influenced by the sources in processing L2 are investigated in L2 to L1 and L1 to L2 translation. L2 proficiency is also considered as a potential factor that modulates L2 learners’ comprehension processes. This study further seeks to examine whether L2 processing mechanisms can be explained by good-enough (GE) processing (Christianson et al., 2001, 2006; Ferreira, 2003; Ferreira and Patson, 2007; Ferreira et al., 2002), as it relates to shallow processing in the monolingual sentence comprehension literature. The translation paradigm used in the current study is well suited to show whether L2 speakers access syntactic representations in their comprehension because the translation output provides explicit evidence of L2 speakers’ comprehension of linguistic input. For example, if a to-be-translated L2 input sentence contains a difficult or non-canonical syntactic structure, and that structure is equally non-canonical in the translation language but nevertheless is carried over faithfully into the translation output, then it is reasonable to assume that the structure was both recognized and processed in the L2 input language. The following sections include brief summaries of the key studies in the monolingual and L2 sentence processing literature, along with a rationale for employing a translation paradigm.
1 Sentence processing research: Monolingual studies
In the L1 sentence comprehension literature, when people process a language, syntactic representations that they compute during comprehension have traditionally been assumed to be complete and veridical, barring occasional random errors. Recently, Ferreira et al. (2002) proposed a good-enough (GE) processing approach to language comprehension, suggesting that the language processor sometimes employs only partial or incomplete representations, which under certain (but, crucially not all, or even most) conditions result in misinterpretations of the input. Several studies have provided evidence for GE processing when native speakers process garden-path sentences (Christianson et al., 2001, 2006; Ferreira et al., 2001, 2002) and passive sentences (Christianson et al., 2010; Ferreira, 2003).
In an influential study, Ferreira (2003) examined how native speakers comprehend unambiguous passive sentences by manipulating syntactic structure and plausibility information as in The mouse chased the cat versus The cat was chased by the mouse. The study found that native speakers of English misinterpreted non-canonical (i.e. passive) syntactic structures more than canonical (i.e. active) structures, and that the rate of misinterpretation was larger when the semantic content of the sentence was inconsistent with world knowledge. Specifically, native English speakers were less accurate in implausible sentences (80%) than plausible sentences (90%) for passives, whereas they were as accurate with plausible sentences (98%) as with implausible sentences (95%) for actives. Importantly, there was also a significant interaction between syntactic structure and plausibility, indicating that syntactic representations can be overwhelmed by semantic information, especially when semantic information is inconsistent with world knowledge. These results were interpreted as showing that native comprehenders have a tendency to employ a heuristic parser along with a syntactic parser. These heuristics include semantic plausibility and a Noun–Verb–Noun (NVN) word order template (see Townsend and Bever, 2001) that maps the NVN order onto Agent–Verb–Patient interpretations. The passive structure, which has a non-canonical patient-before-agent order, was apparently harder to process than the active structure due to the conflict between the outputs of the syntactic algorithm and the heuristic parser. The syntactic representation in the comprehender’s mind can thus be described as ‘fragile’, and the heuristic parser is described as ‘fast and frugal’ at reaching an interpretation.
A recent study by Christianson et al. (2010) also investigated the interaction of semantic and syntactic processing routes in normal comprehension by combining a comprehension paradigm and a structural priming paradigm. The active and passive structures used were taken from Ferreira (2003), yielding a 2 (active, passive) × 2 (plausible, implausible) design. Native English speakers were asked to listen to these sentences, which served as prime sentences for later production, to answer comprehension questions (similar to those in Ferreira, 2003), and then to describe target pictures. These descriptions were recorded. Comprehension error rates again increased when structure conflicted with plausibility, that is, passive-implausible sentences yielded the most incorrect answers, replicating the results of Ferreira’s study. A more important finding was that both plausible-passive and implausible-active conditions tended to prime the production of passive structures in the picture description task. The authors took this to suggest that plausibility information overrode the actual syntactic structure in the input (prime) sentences, and in the process of comprehension the structure was ‘flipped’ to accommodate an interpretation within participants’ world knowledge. Evidence of this structural reorganization appeared in the production data, where passive production was primed by plausible passives, but not implausible passives, and implausible actives. This pattern of results is striking, as it is the first to show a semantic influence on syntactic priming (see Bock, 1986). Altogether, the Ferreira (2003) and Christianson et al. (2010) data were interpreted as evidence that the language processor strives to reconcile the output of both syntactic and semantic/heuristic processing routes, and that sometimes the representation resulting from this reconciliation ends up being only good enough to make sense, not good enough to remain faithful to the input.
2 L2 studies
L2 researchers have started to apply the same psycholinguistics methods that have been used with native speakers and, over the past decade, there have been extensive studies on how non-native speakers process L2 input – in particular, whether they are able to process grammatical information in the same manner as native speakers, and how they are influenced by different types of information during comprehension.
Felser et al. (2003) found that whereas L2 learners were able to show patterns similar to native controls in reading ambiguous relative clauses when sentences contained the lexical preposition with, they did not exhibit any attachment preferences in the condition linked by the genitive preposition of. This result was interpreted as showing that L2 learners lack the ability to process full-fledged grammatical information, but rather rely on lexical–semantic information more than native speakers (see also Papadopoulou and Clahsen, 2003). In addition, Marinis et al. (2005) showed that non-native speakers failed to display detailed syntactic representations in real-time L2 processing, that is, they were not apparently sensitive to intermediate syntactic gap information in long-distance wh-phrases, whereas native speakers were facilitated by the availability of an intermediate landing site when processing the fronted wh-pronouns.
These results provide evidence for the Shallow Structure Hypothesis (SSH; Clahsen and Felser, 2006), which proposes that adult L2 learners have fundamentally different processing mechanisms in which adult L2 learners underuse syntactic structure due to their ‘less detailed’ syntactic representations during on-line comprehension, thus making more use of lexical–semantic or plausibility information in L2 sentence processing.
One thing that needs to be pointed out, however, is that the conclusions from the above studies were derived from null results. The fact that the L2 learners failed to show any preferences to attach to noun phrase (NP)1/NP2 may not necessarily mean that they do not build full syntactic representations during comprehension. Indeed, native speakers are, under certain circumstances, predicted to show no preferences for certain attachments not only within the GE framework discussed above, but also by the Construal Theory (Frazier and Clifton, 1996). Specifically, within Construal, non-syntactic information can be used to adjudicate between competing syntactic options; if those non-syntactic information sources do not resolve the competition, attachments may be left underspecified (see also the very different approach of constraint-based theories, which, in principle, allow for the same sort of underspecification; e.g. MacDonald et al., 1994). So the failure to make certain syntactic attachments can be explained just as well by differences in how various information sources are integrated as it can by postulating necessarily ‘shallow’ syntactic structures. In addition, there is a growing body of studies showing that non-native processing is essentially identical to native processing (Frenck-Mestre and Pynte, 1997; Jackson and Dussias, 2009; Juffs, 1998) and that the differences may possibly be due to a matter of proficiency (Hopp, 2006; Jackson, 2008; Van Hell and Tokowicz, 2010), degree of L1–L2 similarity (Sabourin and Stowe, 2008; Tolentino and Tokowicz, 2011), or a task difference (Jackson and Bobb, 2009; Leeser et al., 2011; Lim and Christianson, in press).
Frenck-Mestre and Pynte (1997) showed that French–English bilinguals read sentences containing temporarily ambiguous prepositional phrases in the same manner as monolingual counterparts. Using main verb/reduced relative clause ambiguities, Juffs (1998) also found that English as a Second Language (ESL) learners were sensitive to different argument structure and lexical–semantic cues during on-line comprehension similar to native English speakers. Recently, Jackson and Dussias (2009) investigated L1 English and L2 German speakers’ processing patterns in reading unambiguous wh-subject-extractions and wh-object-extractions in German. Even though case marking is largely absent in English, the L2 German learners were able to use case-marking information during German comprehension. Hopp (2006) also examined L2 processing of subject–object ambiguities in German with advanced and near-native L2 learners (L1 English and Dutch). The study showed that on-line reading patterns varied depending on L2 proficiency; for instance, while near-native L2 speakers disambiguated sentences using both types of syntactic cues (case marking and verbal agreement), advanced learners were not able to distinguish the cue types, although they showed overall slowdowns at the end of the sentences. This L2 proficiency effect was replicated in a study by Jackson (2008), in which intermediate proficiency L2 German learners did not exhibit sensitivity to case-marking information when reading temporarily ambiguous wh-questions, unlike advanced learners and native speakers.
In summary, existing studies in the L2 processing literature are mostly dependent on reading time data to draw their conclusions, regardless of whether the results are toward qualitative differences or similarities. These studies have revealed much information on L2 learners’ reading patterns, especially with regard to whether they are able to map linguistic strings onto syntactic frames in real-time comprehension in the same way as native speakers. Although existing studies usually have also employed grammaticality judgment tasks or comprehension questions as an off-line measure in combination with on-line measures, these measures are not always clearly informative about how lexical–semantic and syntactic information are integrated in the mind. If various information sources are not completely integrated into final meaning representations, these representations can be, to use Ferreira et al.’s (2002) term, simply good enough. The next section describes how a translation paradigm can shed further light on the integration of morphosyntactic and semantic information, along with brief reviews of several translation studies.
3 Translation
The goal of the present study was to use translation as a means of evaluating how L2 learners integrate morphosyntactic and semantic information in L2 language comprehension. Translation is an important process that L2 speakers undertake when dealing with two languages, providing information about cognitive and linguistic development (Campbell, 1998; Malakoff, 1992). Translation was also once considered to be an accurate way of verifying learners’ comprehension, although it is no longer popular in language teaching situations (Cordero, 1984; Malakoff, 1992). In the L2 psycholinguistics literature, however, translation has not yet been used to address questions relating to whether syntactic, morphological, and pragmatic knowledge are accurately processed or fully integrated; rather, comprehension questions and implicit reading tasks are common methods in the literature. One drawback of these common tasks is that they cannot precisely detect where and how people are incorrect. For example, an L2 learner might spend longer reading a given word in a sentence because they do not know the word, or, alternatively, they might have trouble integrating that word into existing context. In the same vein, a learner might get a comprehension question wrong because he or she misunderstood a single word in the input, or understood all the lexical items but not the syntactic structure of the sentence. In a translation, a mis-translated word provides evidence of a lexical-level problem in comprehension. On the other hand, a translation with all the lexical items translated correctly but a mis-translated syntactic structure would suggest processing difficulty at the syntactic level. In addition, off-line measures of L2 comprehension administered in the L2 are vulnerable to L2 processing effects of the measures themselves. For example, a learner might, in fact, have difficulty comprehending the comprehension question itself (i.e. the question must also be read and processed). Translation – or at least translation from L2 to L1 – does not display this same vulnerability. We do not claim that translation is the best possible measure of L2 processing, or even the best possible off-line measure of L2 comprehension. Rather, we simply propose that it can offer unique insights into L2 processing and comprehension and is therefore a useful task in this line of research.
Translation, a prevalent cognitive strategy used in L2 reading (Chamot and Kupper, 1989), can show how morphosyntax is being understood and how successfully people decode one language and encode into the other, through the output of the translation. Thus, when it comes to how the lexical–semantic level of representation is incorporated with the morphosyntactic level during L2 processing, translation can provide substantial data about L2 learners’ comprehension. If the direction of the translation is from L1 to L2 (i.e. L1 input, L2 translation output), then the comprehension of the input is assumed to be relatively straightforward, as the link between L1 and the concept is argued to be strong (e.g. Kroll and Stewart, 1994). The output, however, might be constrained by the translator’s command over the L2 grammar and/or lexicon. For example, if a translator encounters a passive sentence in the L1 but does not know how to construct a passive in the L2, the L2 translation output may well appear with the simpler, more canonical active morphosyntax. Conversely, if the translation input is the L2, the difficulty for the translator lies in comprehending the input accurately; assuming this can be done, formulating the L1 output is relatively easy. If comprehension of a given lexical item in the L2 input fails, one would not expect to see that lexical item translated correctly in the L1 output. Similarly, if comprehension of the L2 input morphosyntax fails, one would not expect to see it translated correctly into its L1 counterpart. This is especially true if the L2 input is relatively difficult, infrequent, or non-canonical. Therefore, if a given morphosyntactic structure is accurately translated from L2 input to L1 output, it is reasonable to infer that the structure in the L2 input was recognized and, at least on the morphosyntactic level, comprehended.
Although a considerable body of research has examined how two languages are represented in bilingual memory using single-word translation paradigms (Kroll and Stewart, 1994; Kroll and Tokowicz, 2001), there have not been many studies to examine sentence-level translation; only recently have experimental studies been conducted to explore the processes of sentence-level translation from a psycholinguistic perspective (Hatzidaki and Pothos, 2008; Macizo and Bajo, 2004, 2006). These studies in sentence-level translation explored two conflicting psycholinguistic theories of translation which were proposed decades ago. Seleskovitch (1976) construed translation as a ‘vertical process’, in which people comprehend a source language first and then produce a target language from memory in a serial order. Conversely, other researchers (Danks and Griffin, 1997; Gerver, 1976) have advocated a ‘horizontal perspective’, where translation requires direct, interactive links between source (input) language and target (output) language while reading the source language text. Under this view, translations areconstructed relatively incrementally, more or less in real time.
Macizo and Bajo (2004) investigated which of the two theories of translation better explain the process involved in sentence translation. The study compared reading for repetition and reading for translation using a self-paced word-by-word reading task. Professional translators (L1-Spanish–L2-English) performed these tasks in both directions, from Spanish to English (L1 to L2) and English to Spanish (L2 to L1), followed by global comprehension questions. Results showed that reading times were significantly slower in reading for translation than in reading for repetition. The authors interpreted the results as supporting the horizontal view because translators used the direct connections between the two language representations, which increased reading times, in reading for translation more so than in reading for repetition. Another study by Macizo and Bajo (2006) also provided evidence that reading times were significantly slower in reading for translation than in reading for repetition. The results also favored the horizontal approach of translation because participants used working memory resources to ‘activate and switch between the two languages’ when instructed to translate (Macizo and Bajo, 2006: 15).
Since translation is technically not an on-line processing task, one could raise the question as to whether translation can actually reflect the kinds of information L2 learners use during on-line sentence comprehension. With regard to this issue, the research cited above supports the idea that translation proceeds in a relatively incremental fashion during reading – that is, horizontally – such that one must ‘process’ linguistic input in real-time, and that the translations are reflections of the processed input. Alternatively, even if translation does not occur in a horizontal fashion – that is, if translation is merely an indicator of the ‘memory’ of a sentence – readers still need to process the input sentence to yield an accurate memory trace of the input, and the translation must derive at least in part from the processed input upon which that memory trace was built. This is especially true if the to-be-translated input contains a non-canonical, less frequentstructure and the translation output explicitly reflects that very same non-canonical, less frequent structure. Thus, translation is an appropriate representative measure of learners’ comprehension during on-line processing.
II The current study
The idea of ‘shallow’ or ‘good-enough’ processing leads to new ways of looking at human language processing mechanisms and might also suggest the possibility of a somewhat integrated account of L1 and L2 processing. One of the main aims of the current study is to examine whether L2 learners’ processing is qualitatively different from native processing, by investigating whether non-native processing, like native processing, can be accounted for within the GE framework. The GE processing account assumes that the human language processing system has two different routes to compute sentence meaning: a syntactically-driven algorithmic route and a semantics-based heuristic route (see Jackendoff, 2007; Kuperberg, 2007). To be specific, the syntactic algorithm is hypothesized to be fully operational alongside semantic-based heuristics such as lexical/pragmatic information, world knowledge, or the NVN heuristic. But the syntactic representation is fragile and can be overridden by the heuristics, leading to an interpretation that is unfaithful to the original text. Figures 1 and 2 illustrate the mechanisms of GE processing in comparison to SSH (Figure 1), which are clearly related in language processing. Figure 1 shows the SSH, which proposes that L2 learners are limited in their access to the syntactic route (dotted line), which results in over-reliance on the heuristic route to compute the meaning of a sentence. On the other hand, GE (Figure 2) proposes that even if the syntactic route is available, misinterpretation can be possible due to the incompleteness of the integration of the outputs from the two routes, which is represented by the dotted integrative line between the good-enough and full representation boxes.
Shallow Structure Hypothesis for L2 processing, taken from Clahsen and Felser (2006b).
Good-enough processing, for both L1 and L2 processing.
Considering the fact that there have been very few attempts to examine L2 processing mechanisms using a translation paradigm (see Hatzidaki and Pothos, 2008; Macizo and Bajo, 2004), and that translation has the potential to provide an explicit output representation of L2 speakers’ comprehension, this study used an off-line translation paradigm to specifically address the question of L2 speakers’ integration of different kinds of information. Specifically, if it were to be found that translations sometimes contained accurate reflections of the input language morphosyntactic features, yet inaccuracies in semantic role information, it would be difficult to argue that the semantic errors were due to incomplete morphosyntactic processing of the input language. To take a concrete example, imagine a situation in which the to-be-translated sentence was The cat was chased by the mouse, and the translation was in passive voice (in the output language) yet had the cat chasing the mouse. Such a reversal of thematic roles could not have come from a failure to encode the passive structure, as evidence for accurate computation of the passive morphosyntax is explicitly included in the translation.
Based on the study of Ferreira (2003), Experiment 1 was designed to examine whether language learners show similar patterns of misinterpretations as native speakers, manipulating syntax (active, passive) and plausibility (plausible, implausible) information. If L2 processing is ‘good enough’ and, like native speakers, characterized by a complete syntactic parse yet incomplete mapping of syntactic structure to semantic interpretation (Christianson et al., 2010; Ferreira, 2003), then we should observe a nontrivial proportion of translations that maintain the correct morphosyntax, but not the thematic roles of the nouns from the input. In this scenario, it is hypothesized that plausibility can interfere with integration of the output from the syntactic processing route and the output from the heuristic/semantic route. As such, sometimes the output from the semantic/heuristic route will be chosen, and sometimes the output from the syntactic route will be chosen. However, if we observe that inaccurate translations of meaning exclusively co-occur with inaccurate translations of the input language morphosyntax, then it can be concluded that L2 speakers simply have very limited access to the syntactic parsing route in the L2 and translation errors derive from an attempt to cobble together meaningprimarily from lexical semantics.
III Experiment 1
Method
Participants
Twenty-eight native speakers of Korean who began learning English as a second language at or after puberty participated in this experiment. All of them were from the University of Illinois community. English proficiency was measured via a cloze test. 1 All participants were compensated for their participation with either US$7 or course credit. Table 1 presents participants’ English background information, including years of residence in the USA, cloze test scores (out of 40), and self-ratings of English proficiency (out of 10). Two native speakers of English scored the cloze tests and the mean of the two scores for each participant was used for dividing participants into two proficiency groups. Inter-rater reliability between the two scorers was 91%. Based on the median split of the cloze test result, those who scored greater than 22.5 were assigned to the high-proficiency group and those who scored less than 22.5 were assigned to thelow-proficiency group. A t-test performed on the cloze scores revealed that the high-proficiency group was significantly higher on the cloze test than the low-proficiency group (t(27) = 2.40, p = .02).
Participants’ English background information in Experiment 1.
Notes: * p < .01; † p < .10.
Materials
Twenty-four experimental stimuli were adapted from Ferreira (2003), which contained active and passive structures in either plausible or implausible versions (as normed by Ferreira, 2003). In the plausible versions, the information was congruent with how events would normally occur in reality, whereas implausible sentences depicted events that are atypical in the real world. All items are provided in Appendix 1. A sample sentence is shown in the four conditions in (1).
(1) a. The cat chased the mouse. (active-plausible) b. The mouse was chased by the cat. (passive-plausible) c. The mouse chased the cat. (active-implausible) d. The cat was chased by the mouse. (passive-implausible)
The experimental items were equally distributed across four fully-crossed lists in a Latin square design, such that each participant read six sentences in each condition and saw only one version of each item in the experiment. There were also 59 fillers, containing different types of structures that were minimally related to the trial sentences, such as locative sentences (e.g. The book was on the shelf). The length of the fillers was approximately 4 to 10 words, which was quite similar to the number of words of the stimuli sentences. All items and fillers were recorded digitally by a male native speaker of American Midwestern English at a rate of 10 kHz on the Praat program (Boersma and Weenink, 2005).
c Procedure
All participants were tested individually in a quiet lab, and the experimental procedures were explained verbally and in a written form. First, a Reading Span Task (RST) (Conway et al., 2005) was performed in English before the main translation task in order to examine whether working memory capacity (WMC) influences translation performance, as working memory has been assumed to be an important individual difference in ability to process linguistic input (e.g. Just and Carpenter, 1992; King and Just, 1991). Participants were instructed to read a set of sentences out loud, make a semantic anomaly decision for each sentence, and recall a letter presented after the sentence. Then, each participant briefly reviewed a vocabulary sheet, which contained 20 English words 2 from the stimuli sentences along with each word’s meaning in Korean, in order to make sure that a lack of vocabulary knowledge did not impede the translation process. The main task was to listen (over headphones) to an aurally presented sentence in English (L2) and translate it into Korean (L1) verbally; translations were recorded on a Sony ICD-ST25 digital voice recorder. A short practice session preceded the main task to allow participants to familiarize themselves with the task. After the main translation task, participants completed the cloze test and language background survey.
Results
A 2 × 2 fully-crossed, within-participants design was employed: syntactic form was either active or passive, and meaning was either plausible or implausible. Participants’ Korean translations were the dependent variables, scored as combinations of translation meaning accuracy (correct or incorrect) and the morphosyntax (active or passive). They were coded as correct-active, correct-passive, incorrect-active, and incorrect-passive, resulting in a multinomial distribution. Inappropriate translations (30% of total data) were treated as outliers, which were excluded from further analyses. Most of the inappropriate translations included incorrect meanings of words in the to-be-translated sentences (e.g. the horse threw the rider was translated into the rider touched the horse), incomplete structures (e.g. truncated passives or intransitives, which could not be evaluated for semantic accuracy), or missing information. The reason for the inappropriate translations seemed to be partly due to the aural presentation of the input, which posed a challenge for some participants. In addition, the implausible conditions yielded the largest proportion of inappropriate translations, which implies that participants had the most difficulty in these conditions, actually mirroring the structural priming data pattern from Christianson et al. (2010). 3 A Poisson loglinear analysis with repeated measures (GEE in SAS software) was performed, because all data were categorical variables and each of the 28 subjects translated all of the 24 items across the four conditions (Agresti, 2007).
Table 2 shows the number of translations along with the percentage of total translations produced in each condition. From the contingency table, it can be seen that the proportion of incorrect translations was greater for passive sentences than for active sentences. Participants were overall 85% accurate in active sentences, but just 69% accurate in passive sentences. Also, there was a 33% difference in accuracy between active-plausible and active-implausible sentences (100% versus 67%, respectively), which was less than the 39% difference in translation accuracy between passive-plausible and passive-implausible sentences (88% versus 49%, respectively).
Number of translations in each condition in Experiment 1 (percentages in parentheses).
A Poisson loglinear regression yielded main effects of both syntax (z = 13.37, p < .01) and plausibility (z = 6.81, p < .01), indicating that both syntax and plausibility information had significant effects on translation performance. The estimated odds of a correct translation when the syntax was active were approximately two times larger than when the syntax was passive. These main effects were qualified by an interaction between syntax and plausibility (z = −3.16, p < .01), implying that participants were more likely to mistranslate implausible events than plausible ones for passives than they were for actives. These results mirror those observed previously for native speakers (Christianson et al., 2010; Ferreira, 2003) with respect to direction and pattern, although more errors were observed here for L2 speakers.
Recall that one of the purposes of this experiment was to examine whether L2 learners are able to compute full-fledged syntactic representations during processing. The small number of translations that contained correct passive morphosyntax but with reversed thematic roles were particularly interesting, because these demonstrate that L2 speakers actually used their morphosyntactic knowledge to comprehend sentences in at least some instances, but their comprehension was just ‘good enough’ to reach final interpretations with correct morphosyntactic representations mapped onto plausible semantic representations. In a residual analysis (Agresti, 2007), the 95% confidence intervals of residuals for the particular non-empty cells (incorrect-passive cell in passive-implausible condition and incorrect-passive cell in active-implausible condition) showed significance (ps < .01), meaning that the cells were not merely random errors. These instances are quite difficult to explain if one assumes that L2 learners’ morphosyntactic representations are less detailed during processing. In addition, even low-proficiency learners produced incorrect-passive translations (with accurately translated morphosyntax) in implausible conditions. This constitutes direct evidence that the syntactic route is available to even low-proficiency L2 speakers (see Table 3).
Number of translations in each condition for high- and low-proficiency group in Experiment 1.
Separate analyses for the high- and low-proficiency groups were performed to explore whether English proficiency modulated the influences of syntax and plausibility information on translations. With regard to the high-proficiency group, both syntactic (z = 5.73, p < .01) and plausibility information (z = 2.35, p = .02) significantly affected translation performance. No interaction was found in the high-proficiency group (z = −.19, p = .85), indicating that highly proficient learners comprehended sentences using independent syntactic and semantic processing routes, and the syntactic representation was not significantly affected by semantic information. The low-proficiency group was similarly affected by both syntax (z = 11.54, p < .01) and plausibility (z = 4.52, p < .01). However, unlike high-proficiency learners, low-proficiency learners showed a reliable interaction (z = −2.14, p = .03), indicating that they produced more inaccurate translations in implausible events (71%) than plausible events (17%) for passive than for active sentences (41% versus 6%, respectively).
A further correlation analysis, as presented in Table 4, revealed a significant relationship between proficiency and translation performance in the passive-implausible condition (r = −.43, p = .02), showing that proficiency and the number of incorrect translations had a strong negative relation in this condition. In other words, as proficiency rose, the number of incorrect translations of passive-implausible sentences decreased. In the active-implausible condition, the relationship between proficiency and performance was marginally significant (p = .07) as well, indicating that plausibility was overall a relatively stronger contributor to translation errors by lower-proficiency speakers than by higher-proficiency learners.
Correlations between translation performance and proficiency in each condition in Experiment 1.
Note; * p < .05. † p < .10.
In summary, when the content of the sentence was not consistent with world knowledge, low-proficiency participants tended to have more difficulty mapping the correct morphosyntactic representation onto the semantic representation even in the simple active structure, thereby yielding frequent misinterpretations. These results suggest that low-proficiency L2 speakers are more reliant on plausibility information than syntactic information in deriving interpretations of L2 input compared to the high-proficiency group. An alternative interpretation can also be proposed, however: it is possible that both groups used plausibility information to a similar extent, but what changes as proficiency increases is the ability to use morphosyntactic information to override the plausibility information, and this ability improves with increased proficiency.
To examine whether (WMC) affected the way the L2 learners comprehended the L2 input, a correlation analysis was performed between working memory span scores and translation performance in each condition. The RST was scored according to procedures described by Conway et al. (2005); a participant received 1 point for each letter string recalled in the correct order, up to 42 possible points (one for each item). There were no significant relations in any conditions between WMC and the way the L2 learners translated English sentences into Korean (all ps > .60). It thus appears that working memory was less of a factor than L2 proficiency in the L2 to L1 translation of this active/passive structure.
Discussion
The purpose of Experiment 1 was to use a translation task to investigate how morphosyntactic and plausibility information are integrated during L2 comprehension. Participants translated English active and passive sentences into Korean active and passive sentences, respectively, demonstrating that L2 learners generally had command of this aspect of English morphosyntactic knowledge. However, they were more likely to misinterpret implausible sentences than plausible sentences with passive structures, suggesting that morphosyntactic processing occurs along with semantic processing. The results obtained here for L2 speakers patterned similarly to results obtained previously for native speakers (Christianson et al., 2010; Ferreira, 2003), but the overall number of misinterpretations for L2 speakers was larger. This indicates a quantitative rather than qualitative difference between native and non-native speaker processing of these structures. A notable finding was that participants produced a small but significant number of translations with reversed thematic roles that maintained the correct morphosyntax. This appears to suggest that non-native processing in this experiment is best characterized by an incomplete mapping of syntactic structure to the semantic interpretation.
Experiment 2 was designed as a follow-up to Experiment 1, reversing input and target languages in the same translation paradigm, to provide a picture of the opposite direction of translation. L2 to L1 translation is important in this context, as it provides a measure of the extent to which L2 speakers drawn from the same subject pool as those in Experiment 1 have productive control over the English passive structure. It could be argued, for instance, that translation errors observed in Experiment 1 derived not from a failure to integrate the morphosyntactic and lexical–semantic information contained in the English L2 input, but rather a general lack of command of English passive morphosyntax. Under this interpretation, the apparently systematic translation errors that occurred in Experiment 1 might simply stem from poor comprehension of the input due to unfamiliarity with or lack of access to English passive morphosyntax during listening. The purpose of Experiment 2, then, was to examine Korean (L1) to English (L2) translations for evidence that would support this alternate interpretation of Experiment 1’s results. Specifically, if participants are consistently able to produce grammatical English translations of Korean passives, irrespective of plausibility, then we would have positive evidence that the errors observed in Experiment 1 were not due to lack of mastery over English passive constructions.
IV Experiment 2
Method
Participants
Participants were 28 Korean-speaking learners of English who had not participated in Experiment 1, drawn from the same University of Illinois subject pool as in Experiment 1. The same English cloze test that was used in Experiment 1 was used to measure the participants’ proficiency level (scored according to the same procedures as in Experiment 1; inter-rater reliability was again 91%). Participants’ English background information and their cloze test scores are presented in Table 5. Based on the median split of the cloze test result, those who scored higher than 24 out of 40 were assigned to the high-proficiency group and those who scored lower than 24 out of 40 were assigned to the low-proficiency group. A t-test revealed that the high-proficiency group performed significantly better than the low-proficiency group (t(24) = 2.34, p = .03).
Participants’ English background information in Experiment 2.
Note; * p < .05.
Materials
The materials in Experiment 2 were identical to those used in Experiment 1, except that the input language was now Korean, the participants’ first language. As in English, only transitive verbs can be passivized in Korean, having both agent and theme roles, although sometimes the agent role may not be overtly expressed (as in English truncated passives). Korean passives contain passive morphemes such as -i-, -hi-, -li-, -ki-, such that Korean passives are morphologically distinct from their corresponding actives (Lee and Lee, 2008; for a full review of the Korean passive structure, see Sohn, 2001). All of the 24 English experimental sentences and 59 filler items used in Experiment 1 were translated into Korean as the linguistic input source, which was to be translated in this experiment into English. As in Experiment 1, all items and fillers were recorded digitally by a female native speaker of Korean at a rate of 10 kHz on the Praat program (Boersma and Weenink, 2005). Syntax (active, passive) and plausibility (plausible, implausible) were manipulated, resulting in four conditions. A sample Korean stimulus is displayed in the four conditions in (2).
(2) a. Goyangyi-ga chwi-lul ccoc-ass-ta. (active-plausible) The cat-Nom the mouse-Acc chase-Past-Dec. b. Chwi-ga goyangyi-uyhay ccoc-ki-ess-ta. (passive-plausible) The mouse-Nom the cat-by chase-Passive-Past-Dec. c. Chwi -ga goyangyi-lul ccoc-ass-ta. (active-implausible) The mouse-Nom the cat-Acc chase-Past-Dec. d. Goyangyi-ga Chwi-uyhay ccoc-ki-ess-ta. (passive-implausible) The cat-Nom the mouse-by chase-Passive-Past-Dec.
Procedure
Procedures were nearly identical to those in Experiment 1, except that the translation direction was reversed. Because Korean was the input language of this experiment, the participants completed a Korean version of the RST instead of an English version (Kim, 2008). After the working memory task, participants were asked to listen to Korean sentences and perform a verbal translation into English. As in Experiment 1, participants briefly reviewed the same 20 English words identified in Experiment 1 as potentially problematic before the translation task began. Translations were recorded for later transcription. A practice session was performed at the beginning of the experiment.
Results
As in Experiment 1, Poisson loglinear regressions with repeated measures (GEE in SAS software) were performed to determine the effects of the two factors on translation performance. Two levels of syntax (active, passive) and two levels of plausibility (plausible, implausible) were used as independent variables. All translations were coded into four categories of correct-active, incorrect (reversed thematic role)-active, correct-passive, and incorrect-passive. As in Experiment 1, inappropriate translations (6% of the total data, defined as in Experiment 1) were excluded from further analyses.
Table 6 presents the number of English translations that participants produced in each condition. As shown in the contingency table, there were few incorrect translations and significantly large numbers of translations where structure and meaning were both correct, regardless of the syntax or plausibility. Results showed a main effect of syntax (z = 9.18, p < .01) and a marginal effect of plausibility (z = 1.81, p = .07), with no interaction. This indicates that the Korean structure had a reliable influence on the way the participants formulated their English translations. However, the participants were not as strongly affected by the plausibility information in L1 to L2 translation as in L2 to L1 translation.
Number of translations in each condition in Experiment 2 (percentages in parentheses).
Separate analyses for high- and low-proficiency groups were performed as in Experiment 1 to determine whether L2 proficiency affects L1 to L2 translation. Table 7 shows the number of translations in each condition by proficiency group. There was a main effect of syntax for both high-proficiency (z = 16.58, p < .01) and low-proficiency (z = 19.98, p < .01) groups, but there was neither an effect of plausibility nor an interaction for either group. Proficiency did not seem to play a crucial role in L1 to L2 translations, unlike L2 to L1 translations. In addition, L2 translations were not related to WMC in any condition (all ps > .10). In sum, it is likely that neither L2 proficiency nor WMC significantly influenced the L1 to L2 translations of these simple sentence structures. This lack of a proficiency effect is likely due to translation accuracy hitting ceiling levels; however, it also supports the view of the translation task that we have proposed here. Namely, the task appears to be primarily a reflection of comprehension processes for these participants, rather than of production processes. As such, L2 comprehension resulted in L1 translation errors in Experiment 1, whereas L1 comprehension was trivial in Experiment 2, thus resulting in few L2 translation errors.
Number of translations in each condition for high- and low-proficiency group in Experiment 2a.
Discussion
In Experiment 2, L2 learners performed L1 to L2 translation with no significant difficulties no matter what the morphosyntax or plausibility were. The morphosyntactic information affected the formulation of the English translations; that is, most active Korean sentences were translated into active sentences in English, and Korean passive structures into English passive structures. Unlike in Experiment 1, both the plausible and the implausible sentences were correctly translated into English, implying that the participants were not reliably influenced by the plausibility information in L1 to L2 translation.
The observation that L2 learners produced correct passive structure independent of plausibility information provides evidence that these language learners have sufficient morphosyntactic knowledge to be tested with this translation paradigm. Moreover, it falsifies the alternative hypothesis that the L2 to L1 translation errors in Experiment 1 derived from a lack of command over English passive morphosyntax. If L2 learners from this participant pool were able to correctly produce English passive structures, it seems very unlikely that demographically homogenous participants from the same pool would not be able to comprehend English passive structures. The results of Experiment 2 thus reinforce the interpretation that the small but significant number of misinterpreted translations in Experiment 1 did not result from ‘shallow’ or incomplete syntactic representations built online while listening to the L2 input, but rather that the misinterpretations were attributable to the conflicting outputs from the two proposed processing routes. The incorrect translations (reversed thematic roles) using passive morphosyntax observed in Experiment 1 can thus be taken as evidence of GE processing. That is, L2 learners appear to have been able to compute adequate syntactic information during processing, but the syntactic representation was overridden by semantic knowledge that was inconsistent with the output of syntactic route. This competition between and failure to integrate different sources of information is proposed to be a hallmark of GE processing.
Another point of discussion is why Korean learners of English did not produce mistranslations in Experiment 2 in the passive-implausible condition to a similar extent that the native English speakers misinterpreted these sentences in Ferreira’s (2003) study, given that the materials were the same and the input language was participants’ first language. This difference seems to provide an important insight into the cognitive processes of translation. When translation is involved during comprehension, people make additional effort to search linguistic entries in two languages (Danks and Griffin, 1997; Gerver, 1976; Macizo and Bajo, 2006), resulting in a more concrete and detailed representation being built during comprehension. This pattern has been observed recently in a direct comparison of reading for comprehension and reading for translation in a self-paced reading study (Lim and Christianson, in press). We speculate that translation strengthens the link between L1 and conceptual memory, such that the syntactic representations built during comprehension were not significantly disturbed by the semantic information even when the structure was atypical, that is, passive. Thus, the L2 learners in Experiment 2 were able to produce mostly correct translations even for the implausible passive sentences.
V General discussion and conclusions
The results from Experiment 1 can be accounted for within the GE language processing framework (Ferreira et al., 2002). First, L2 speakers here were influenced by both syntax and plausibility information in parallel, which suggests that L2 processing does indeed proceed along both syntactic and semantic routes, at least the constructions to be tested in this study, namely English active/passive structures (for similar evidence for English–Korean subject-/object-relative clauses, see also Lim and Christianson, in press). Second, L2 learners’ translations showed flipped structures (either active to passive or passive to active) when the sentences were implausible (see Table 2). For instance, the passive-implausible condition yielded 46% incorrect translations using active structures and 12% incorrect translations using passive structures when translating active-implausible sentences. This pattern is strikingly similar to results in Christianson et al. (2010), where native English speakers were more likely to describe target pictures using passive structures when the prime sentences were plausible passives and implausible actives. That study suggests that the interaction of morphosyntactic and semantic information affected implicit processing, that is, the processes involved in structural priming. Similarly, the L2 learners reconciled the output of both syntactic and semantic routes, in the same manner as native speakers, showing an interaction in explicit processing, that is, the cognitive processes involved in translation.
The translations signaling misinterpretations in Experiment 1 were produced most often when the semantic content of passive sentences was implausible. Importantly, however, there were incorrect translations in Experiment 1 that contained reversed thematic roles with correct passive morphosyntax. These instances represent very explicit evidence that the morphosyntactic parser was operating in L2 processing, but world knowledge overrode the syntactic representation. This pattern is consistent with the previous studies with native speakers, reporting that incorrect responses increased when plausibility information conflicted with the syntactic parse (Ferreira, 2003). GE processing proposes that syntactic computation can run parallel to semantic/heuristic processing (Jackendoff, 2007; Kuperberg, 2007), so this pattern of results is not unexpected. Even though GE was framed for L1 processing, based on the present findings, it appears that L2 speakers are not qualitatively different from native speakers with regard to the use of these information sources during comprehension.
Another issue concerns the question of how proficiency modulates the way L2 speakers used both syntactic and semantic information in translation. Low-proficiency learners were influenced by semantic information to a greater extent than the high-proficiency learners, especially with passive structures, although both groups experienced difficulties in implausible conditions. This is suggestive of the role of proficiency involved in L2 processing; namely, proficiency leads to a more automatic processing of syntactic information and a better ability to integrate conflicting outputs from the morphosyntactic and semantic processing routes. Consequently, the high-proficiency group was less disturbed by plausibility information, experiencing less difficulty than low-proficiency speakers in translation of implausible-passive sentences: incorrect answers were 38% (high) versus 71% (low). Thus, it seems that as proficiency increases, the ability to compute syntactic information during comprehension improves, with performance eventually becoming more like that of native speakers.
A closer look at the difference between the translation accuracy in this experiment and the comprehension accuracy rates in Ferreira (2003) point to a reason for the absence of an interaction between syntax and plausibility in the high-proficiency group. The high-proficiency learners were more inaccurate than the native speakers in Ferreira’s study even when the active sentences were implausible, demonstrating that the L2 learners relied on how the world usually works rather than the representation of the syntactic parse, even in the simple active structure. This suggests that L2 learners are more influenced by world knowledge during the stage at which semantic and morphosyntactic information are integrated. Although the output of the morphosyntactic parse may be accurate, it is more fragile than that of native speakers. This observation is not only consistent with GE processing, but also with previous L2 studies showing over-reliance on lexical–semantic knowledge (SSH; Clahsen and Felser, 2006a). It is also congruent with the idea of ‘semantic bootstrapping’ (Fisher et al., 1994; Pinker, 1984), which posits that the use of existing world knowledge in adult language learners can boost incomplete L2 grammar in language acquisition. Importantly, however, the translations in Experiment 1 showing correct morphosyntax with reversed thematic roles along with the results of Experiment 2, where full control of the L2 passive morphosyntax was demonstrated, both argue against characterizing L2 processing as syntactically ‘shallow’. Rather, it seems that L2 morphosyntax would be better described as ‘fragile’; susceptible to intrusions from semantic or heuristic sources in the same way as native speakers’ morphosyntax, just more so. In other words, the difference between L1 and L2 processing appears to be quantitative rather than qualitative.
Footnotes
Appendix
Acknowledgements
An earlier version of this research was presented at the CUNY conference on Human Sentence Processing (University of California at Davis, 2009). The authors thank the members of the Educational Psychology Psycholinguistics Lab for helping with data collection. and Dr. Caroline Anderson for advice about data analysis.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded in part by a University of Illinois at Urbana-Champaign Campus Research Board grant and a summer GA support award from the University of Illinois at Urbana-Champaign Bureau of Educational Research.