The effect of language on recognition memory in first language and second language speakers: The case of placement events

Abstract

Aims and Objectives/Purpose/Research questions:

German and Spanish differ in lexicalization of object position in placement events (e.g. They stand/lay-put the binoculars on the shelf). Do native (L1) speakers of these languages show different recognition memory for object position in placement scenes (“Thinking for Speaking” (TFS))? And if so, can learning German as a second language (L2) improve memory accuracy?

Originality:

There is very little research on the effect of language on memory in L2 speakers and no such studies have focused on placement events. By adopting a short time course (750 ms) between the prime and recognition phase this study makes a methodological advancement.

Design/Methodology/Approach:

We employed a design with L1 speakers (N = 54) of German and Spanish, and a group of Spanish L2 learners (N = 123) of German. Participants were presented with a two-phased memory task with minimum delay, with language and pictures showing placement events. Following the direction indicated by German placement verbs we changed position of objects in the picture recognition phase. L2 German speakers received a form-focused instruction on German placement verbs (stand/lay) before the memory task.

Data and Analysis:

We analysed recognition accuracy for object position changes.

Findings/Conclusions:

Results showed that L1 German speakers had more accurate recognition memory for object position changes than L1 Spanish speakers. When Spanish learners of L2 German performed the experiment in German, their accuracy exceeded L1 German speakers’ scores.

Significance/Implications:

The findings provide support for TFS effects on memory for object position in placement events for L1 speakers and show accuracy advantages for L2 speakers. Future studies should consider employing tasks with short time courses as the one used in this paper, in order to establish a base of controlled and reliable findings to unravel the linguistic relativity literature.

Keywords

Thinking for Speaking placement events recognition memory German Spanish

Introduction

Imagine seeing a picture of a man placing a pair of binoculars on a shelf. Later, you encounter a second, similar picture and are asked to compare it with the first one. Will the language one speaks affect what is remembered about the first picture and the latter comparison as a result? This question relates to the notion of linguistic relativity. Linguistic relativity can be said to comprise a family of related proposals on the relation between languages and thought (for discussions, see Bylund & Athanasopoulos, 2014; Malt, Sloman, & Gennari, 2003; Wolff & Holmes, 2010). The Thinking for Speaking (TFS) hypothesis (Slobin, 1996, 2003) poses that effects of language on thought appear when individuals are engaged in language-driven activities (e.g. speaking, listening, writing, reading). A significant number of studies with functional monolingual (first language; L1) speakers has documented that TFS is indeed language-specific (e.g. Slobin, 2003). In recent years, studies with second language (L2) learners have shown that learning to re-think for speaking in a L2 can be difficult even for advanced L2 learners (e.g. see Cadierno, 2017, for a recent review on research conducted in this area). L2 learning provides a compelling case to document behavioural shifts that occur with a change of specific language (Pavlenko, 2014).

Most of the previous TFS research has dealt with the study of verbalization and categorization, while few monolingual and hardly any bilingual studies have dealt with memory (Filipović, 2016). Existing work has focused on different lexicalization patterns of voluntary motion (Filipović, 2011) or intentionality in caused motion (Filipović, 2016). However, few studies have exploited cross-linguistic differences in the expression of one type of caused motion event, namely placement events (Kopecka & Narasimhan, 2012; Talmy, 1985, 2000). Placement events are events where an agent moves an object to a certain location (e.g. He puts the book on the shelf) and are a ubiquitous part of everyday human experience. This domain is an ideal platform to investigate TFS and memory, since German and Spanish (and other Germanic versus Romance languages) clearly differ in the semantic information that is coded in the verb when describing placement events. In German, one employs a set of semi-obligatory placement verbs that indicate whether the final position of the moved entity is either vertical (stellen [stand]) or horizontal (legen [lay]) (Fagan, 1991; Lemmens, 2006). In contrast, Spanish employs a single placement verb poner [put] or dejar [leave in a place] that does not indicate a certain position of the placed object (Cadierno, Ibarratxe-Antunano, & Hijazo-Gascón, 2016; Ibarretxe-Antunano, 2012).

Considering the methodology used to document effects of language on memory, previous studies have mainly employed memory tasks that involved a delay between prime and recognition phase, varying from 10 minutes to 1 day (Billman, Swilley, & Krych, 2000; Bosse & Papafragou, 2010; Gennari, Sloman, Malt, & Fitch, 2002). In contrast, studies such as Filipović (2011, 2016) and Coventry, Valdés, and Guijarro-Fuentes (2010) designed two-phased memory tasks with minimum delays of 120 and 750 ms, respectively (see Figure 1). Adopting a shorter time course ensures that language is less likely to be used as a strategy (e.g. descriptions that interfere with effects of critical language in the encoding phase) at encoding and on retrieval to remember a previously encountered image. As a result, tasks with a short time course will yield more reliable results than longer time courses employed previously. A memory task with minimal delay may be well able to reveal language-specific effects of placement verbs on recognition of object position in placement events.

Figure 1.

Example of a two-phased experimental trial in the present study, adapted from Coventry et al. (2010).

A well-cited theoretical framework on memory and attention is that of Shiffrin and Schneider’s (1977). They showed that the learning of categories (e.g. verbal labels) is shown to improve controlled search performance, leading to higher detection accuracy for changes in memory tasks. Their framework allows for making precise predictions about memory accuracy for object position in placement events in L1 and L2 speakers. L1 German speakers have learned to categorize placement actions in terms of horizontality and verticality from childhood, whereas L1 Spanish speakers have not learned to structurally make this distinction in their native language. We predict that German placement verbs will lead to improved search performance and higher detection accuracy for changes in object position in L1 German speakers as compared with L1 Spanish speakers (reading Spanish placement verbs). In the case that Spanish learners of L2 German learn about the German placement verbs, however, we predict that they too will show high search performance when performing the task in German. Moreover, we predict that if Spanish speakers are instructed on the meanings of German placement verbs, this instruction will lead them to notice the verbs in the linguistic input (de Bot & Jaensch, 2013; Sanz, 2000; Schmidt, 1990). In its turn, this will exhibit a higher level of detection accuracy for changes in object position as compared with L1 German speakers.

The current paper provides new empirical data on TFS and memory in L1 and L2 speakers using a fine-grained method. We address TFS hypotheses by exploiting cross-linguistic differences in verb semantics in the description of placement events. We employ a behavioural computer task adapted from Coventry et al. (2010) with minimum delay, where participants were presented with language that described people placing objects and accompanying images (see Figure 1). In critical recognition trials we altered object position with respect to preceding prime pictures and examined recognition accuracy. Following the TFS (Slobin, 1996) and Shiffrin and Schneider (1977) frameworks, we predict that L1 German speakers will have more accurate recognition memory for changes in object position than L1 Spanish speakers, and Spanish learners of L2 German will show accuracy advantages for changes in object position as compared with L1 German speakers.

Theoretical background

Previous work: Recognition memory for (motion in) space

In psychological research, recognition memory is considered to be a subcategory of declarative memory, consisting of the ability to recognize previously encountered stimuli. It thus involves a matching process by which the content of a given stimulus in the environment is compared to the content stored in memory (Gardiner, 1988; Tulving, 1985; Yonelinas, 2002). Recognition memory is different from memory recall, which refers to the act of retrieving information from the past without being cued. Thus, recognition memory tasks require the recognition of a correct answer among several options being offered to the participant, while recall memory tasks involve accessing what is remembered without any stimulus as a prompt (Filipović, 2016).

Previous studies have yielded mixed results with respect to language effects on recognition memory of human motion and objects. This may be explained by differences in methodology, for example, the complexity of the task (e.g. Filipović, 2011, 2016) or the (lack of) involvement of verbalization (e.g. Gennari et al., 2002; Papafragou & Selimis, 2010). Another important methodological factor in relation to recognition memory is the time course between prime and recognition phases (see Table 1). Some studies have found that the more specific the language is, the more accurate speakers’ recognition memory is. For example, Billman et al. (2000) and Gennari et al. (2002) let participants describe in writing or verbally videos of motion events before doing a memory task with a day or 10 minutes delay (see Table 1). Results showed that English speakers had more accurate recognition memory for (videos of) human motion (e.g. clamber, stride, creep) in the case that they heard or verbalized verbs that encode manner of motion as compared with Spanish speakers, whose verbs do not typically encode manner of motion. Filipović (2011) also found that L1 English speakers had more accurate recognition memory for voluntary motion than L1 Spanish speakers and balanced Spanish-English bilingual participants who adhered to Spanish, less specific lexicalization patterns (see Table 1). In another study, Filipović (2016) examined intentionality in caused motion and memory in Spanish learners of L2 English. Results showed that their memory recall was mostly informed by their L1-entrenched (more specific) preferences even when speaking in the (less specific) L2. They performed better on a recall memory task than English learners of L2 Spanish. Filipović’s (2011, 2016) participants performed the memory tasks after a 120 ms distractor task.

Table 1.

Overview of methodologies and results in Thinking for Speaking (TFS) studies into memory of (motion in) space.

Authors, Year	Focus	Skill	Memory	Stimuli	TFS?
Billman et al. (2000)	Motion V	Writing	1 day	Videos	Yes, more accurate
Gennari et al. (2002)	Motion V	Speaking	10 min	Videos	Yes, more accurate
Feist & Gentner (2007)	Spatial PPs	Reading	10 min	Drawings	Yes, less accurate
Bosse & Papafragou (2010)	Positional V	Writing	10 min	Pictures	No
Coventry et al. (2010)	Spatial PPs	Reading	750 ms	Drawings	No
Filipović (2011)	Motion V	Speaking	120 ms	Videos	Yes, more accurate
Filipović (2016)	Motion V	Speaking	120 ms	Videos	Yes, more accurate

V: verbs; PP: preposition.

Contrary to the studies just mentioned, Bosse and Papafragou (2010) found no effects of the German positional verbs sitzen [sit] and liegen [lie] versus the English be on memory for pictures of scenes where objects were positioned horizontally or vertically. This was the case despite participants describing pictures in writing before doing a memory task with 10 minutes delay (see Table 1). Feist and Gentner (2007) presented participants with sentences and drawings of spatial relationships and took a memory task with 10 minutes delay (see Table 1). Their monolingual English participants falsely recognized stronger versions of spatial relationships (e.g. a block on top of a building) as identical to preceding weaker ones (e.g. a block on the edge of a building) after reading spatial prepositions (e.g. “The block is on the building.”). The spatial prepositions presented at encoding were inappropriate descriptions of presented images. The authors argue that participants thus encoded the scene in terms of a competing (and more applicable) spatial prepositional category, which caused the indirect effect of language on memory. Coventry et al. (2010) adapted the methodology of Feist and Gentner to further investigate spatial prepositions. They designed a two-phased memory task with minimum delay (750 ms) administered with a computer (see Figure 1). The authors argue that a shorter time course yields more fine-grained results than methods employed previously, as it ensures that language is less likely to be used as a tool at encoding and on retrieval to remember a previously encountered image. Coventry et al. found no effects of reading spatial prepositions in English (in and on) and Spanish (en [in/on]) on recognition memory for drawings showing containment and support relations (see Table 1). The drawings showed hands that were opened to varying degrees and contained different amounts of entities. Although the methodology led to null results for spatial prepositions, it may be well able to reveal language-specific effects with a different language phenomenon, that is, placement verbs, on recognition of object position in placement events.

Learning placement verbs: Language and cognition

In a large cross-linguistic volume, researchers have documented that languages vary greatly in their descriptions of putting and taking actions (Kopecka & Narasimhan, 2012). The basic components of a placement event are as follows: Figure (what is moved); Agent (the causer of the movement); Ground (the location where an object is placed); Causation (what triggers the placement); Motion (the act of moving); and Path (the trajectory followed by the Figure) (Jackendoff, 1990; Talmy, 1985). Languages vary greatly in the manner in which they encode all these components of putting actions. As mentioned, German and Spanish verbs differ in their expression of spatial position of the Figure object in relation to the Ground (Cadierno et al., 2016; Lemmens, 2006), which provides an interesting TFS testing case. It has been posed that verbs of putting and taking are amongst the most frequent, basic verbs in a language and that they are amongst the earliest verbs learned by children (Levinson, 2012), which gives a critical reason to study their effects on cognition.

Although memory has been left unconsidered so far, research has documented effects of placement verbs on cognition. A study of particular relevance is that of van Bergen and Flecken (2017). They showed that variation in the semantics expressed by placement verbs may affect how fast L2 speakers move their eyes towards Figure objects upon reading Dutch placement verbs (e.g. zetten/leggen [stand/lay]). They found that German learners of L2 Dutch anticipated objects that matched the verbally encoded position (similar to L1 Dutch speakers). However, a sample of equally proficient English and French users of L2 Dutch did not predict objects on the basis of verbal semantics. They attribute findings to transfer of L1 processing routines, as German, like Dutch, encodes the position of objects in placement verbs (e.g. stellen/legen [stand/lay]), whereas English and French do not. What remains unclear, however, is whether and to what extent the advanced L2 Dutch learners in this study had learned the critical placement verbs.

Several studies suggest learning difficulties with learning the specific verbal distinction marked in placement verbs. Narasimhan and Gullberg (2011) report overextension of the Dutch verb leggen [lay] by 4- and 5-year-old children acquiring Dutch. In L2 production, Gullberg (2009) found that English L2 users of Dutch overgeneralized the verb zetten [stand]; Berthele (2012) reports an overuse of the verb legen [lay] in Romansh L2 users of German. A recent study by Cadierno et al. (2016) also found that L2 placement verb meaning reconstruction was difficult both in moving from a general (Spanish) to a specific (Danish) system and vice versa.

Several researchers have stressed the important role of instruction in getting learners to notice linguistic forms in the input and to understand their meaning (de Bot & Jaensch, 2013; Sanz, 2000; Schmidt, 1990). In fact, de Knop and Perrez (2014) and de Knop and Mollica (2017) have shown that instructing French and Italian learners of L2 German leads to improved understanding of placement verbs. There is a rich literature on the pros and cons of different types of instruction (Bell, 2015; Hawkins, 1984; Loewen, 2012). Yet, Norris and Ortega (2001) concluded, in a meta-analysis of studies into the effectiveness of different types of instruction, that more important than type is the mere fact that instruction takes place. In the present study we examine how a short period of instruction on the meaning of placement verbs affects recognition memory for object position in Spanish learners of L2 German in comparison with German native speakers. Our instruction can be classified as focus-on-form instruction (Ellis, 2001; Spada & Lightbown, 1993). It involves drawing learners’ attention to the target forms within a meaningful context.

L1 and L2 information processing and memory

Previous research has investigated whether bilinguals process and store information in a language-specific way. This has also been phrased in terms of independence versus interdependence, or separate versus common store of the bilingual’s languages. As there are many types of bilinguals (e.g. differing in age of acquisition; context of acquisition; levels of proficiency of the spoken languages; current use of both languages) the answers to these questions are not uniform (Abutalebi, Cappa, & Perani, 2009; Myers-Scotton, 2003). Maigiste (1982, 1985), for example, noted that dominant bilinguals reacted significantly faster to pictured objects in their dominant language than did balanced bilinguals in either of their two languages. This suggests more interdependence for the latter type of bilingual. Abutalebi et al. (2009) also argue that in early bilinguals with equal practice from birth, a single common language system appears to be responsible for processing both languages. The current paper does not aim to provide clarity in this debate, as bilinguals would need to be tested in both languages and perform tasks in the bilingual mode (as in Filipović, 2011). We do assume that a bilingual’s both languages are always active while processing information, although the activation may be to a different degree (Grosjean, 2001; Grosjean & Soares, 1984).

We employ Shiffrin and Schneider’s (1977) theory on information processing to make predictions about memory accuracy for object position in placement events in L1 and L2 speakers. The framework has informed theory on processing approaches to L2 learning (see Pienemann, 2003, for an overview) and previous studies on bilingual memory (e.g. Maigiste 1982, 1985). Shiffrin and Schneider showed, in a classical series of experiments on recognition memory, that the learning of categories (e.g. digit versus consonant) is shown to improve controlled search performance, which in turn leads to higher detection accuracy. Their participants were presented with a memory frame with four digits or consonants. In successive trials, 20 frames with digits and/or consonants in immediate succession, participants had to detect any memory-set items that appeared. Half of the trials contained a target (item from the memory set), and one half contained no target (distractors). An important independent variable was the relation between target and distractor items. Results showed that in the case that there was consistent mapping, that is, cases where meaning targets were never distractors and belonged either to the category digits or consonants, participants had higher accuracy scores than in the case of varied mapping where targets/distractors and digits/consonants were mixed. Thus, categorical distinctions between the memory ensemble and distractor set benefited search accuracy:

When all of the members of a memory set are members of the same category [e.g., digits] and no distractors [e.g., consonants] are in that category, then a controlled search can bypass the individual memory-set items and utilize comparisons that involve matching the single category against the category of each display item. (Shiffrin & Schneider, 1977:142)

Importantly, they point out that verbal labels are a common class of categories. We thus predict that German placement verbs, which categorize placement events along a vertical or horizontal axis, will lead to improved search performance that will be reflected in higher detection accuracy of changes in object position for L1 German speakers as compared with L1 Spanish speakers (Hypothesis 1).

For Spanish L2 German speakers, however, another hypothesis needs to be formulated. We also predict that Spanish L2 German speakers, who have received instruction on the placement verbs, will show high detection accuracy. However, due to the instruction factor, we may expect differences between L1 and L2 German speakers. Focusing on attention tasks, Shiffrin and Schneider (1977) argue that if instruction takes place before a memory task, which tells that certain inputs are relevant (and others or not), the memory load is decreased. As the size of the memory and distractor set is decreased, detection accuracy increases. We argue that by instructing L2 German learners on the meaning of verb semantics before the memory task, the size of the potential memory set is decreased. In other words, they are more likely to make object position a focus of attention in comparison with L1 German speakers. Therefore, we may predict higher accuracy for L2 German learners as compared with L1 German speakers (Hypothesis 2).

Method

Participants

The participants in this study were L1 speakers of Spanish (N = 27); L1 speakers of German (N = 27); and Spanish learners of L2 German (N = 123). We recruited L1 speakers of German at the University of Bremen and L1 speakers of Spanish at Seville University; the Spanish learners of L2 German were recruited in Spain, at the Goethe Institut in Granada, and at the German Department at the Universities of Seville and Granada. The learners’ level of proficiency varied from level A2 to B2 (and four C1 candidates) according to the Common European Framework of Reference (CEFR).¹ The CEFR indicates that from level A2 onwards, learners are able to comprehend simple sentences and instructions, which was the task at hand in our experiment. Participant details are provided in Table 2. Participants received a nominal fee for their participation.

Table 2.

First language (L1) German and Spanish and second language (L2) German participants’ gender, age and pre- and posttest scores.

Language	N	% Female	Mean age	Pretest % correct	Posttest % correct
L1 German	27	51.9	23.5	-	-
L2 German A2	58	60.3	21.9	55.7	86.3
L2 German B1	45	77.3	20.5	65.1	95.1
L2 German B2+	20	55.0	21.5	61.7	93.8
L1 Spanish	27	74.0	21.3	-	-

Spanish learners of L2 German received a form-focused instruction on the German verbs legen/stellen [lay/stand] before the experiment (see the Appendix). In the class-fronted instruction, the teacher discussed the meaning and conjugations of the verbs and students role-played a meaningful example. The instruction was preceded and concluded with an 18-item test, where scores could range from 0% to 100% correct (Appendix). The aim of the tests was to examine the extent to which the learners knew the meaning of the L2 German placement verbs before and after the instruction. In the tests, learners had to indicate whether six forms of legen; six forms of stellen; and six neutral verbs implied horizontality, verticality or neither or both. The average score (N = 123) on the pretest was M = 60.0 % (SD = 21.3%, min = 11.1%, max = 100%). This showed that irrespective of proficiency level, a large number of learners did not understand placement verbs well before the instruction. The average score on the posttest was M = 90.8% (SD = 17.2%, min = 22.2%, max = 100%), indicating that after instruction, most learners had improved their understanding of placement verbs. Analyses excluding the 18 students that had less than 75% (cut-off point) of the items correct in the posttest yielded results that were not reliably different from the ones reported below.

Materials

Participants completed a computer-based task adapted from Coventry et al. (2010) with 96 trials. As compared with Coventry et al. (2010), we used a different type of stimuli (pictures versus drawings); different manipulations (different degrees of inclination versus degrees of openness); and studied a different language phenomenon (motion verbs versus spatial preposition). However, the following was the same. Each trial started with a prime display, which was present until a response was made. Hereafter a fixation cross (+) was presented for 750 ms, followed by the recognition display, which remained on screen until a response was made (see Figure 1). In the prime display, participants were asked to verify whether two identical pictures depicting a placement scene matched a sentence describing the placement scene (e.g. They put the binoculars on the shelf).² The purpose of this task was to ensure that participants read the sentences and studied the pictures. In the recognition display, participants were asked to verify whether the presented picture was identical to the picture(s) just seen. The purpose here was to determine whether the language presented in the prime display would affect their recognition memory accuracy.

In the prime display, participants were exposed to language and pictures. The critical language stimuli were 48 sentences describing placement scenes (e.g. They put the binoculars on the shelf). The sentences described four critical placement scenes, which were as follows: men put binoculars on shelves; men put glue sticks on plates; women put flashlights on pieces of paper; women put lipsticks on cutting boards. Through a pilot study, we chose object combinations with no “typical” placing orientation of the figure object.³ This was to ensure that, in German, participants would only be primed by the verbs legen/stellen [lay/stand] and not by a preferred placing orientation. We created two prime sentences and one “neutral” sentence describing the four scenes (see Table 3). The neutral sentence (e.g. There are men, binoculars and shelves) did not indicate object position. It was created to examine whether any language effects on memory for German speakers were due to the German placement verbs only or whether they extended to neutral sentences.

Table 3.

Examples of two prime sentences and one “neutral” sentence for German and Spanish with English translations.

Sentence type	German	Spanish
Prime1	Sie stellen das Fernglas auf das Regal. They stand the binoculars on the shelf.	Ellos ponen un par de binoculares en el estante. They(-masculine) put the binoculars on the shelf.
Prime2	Sie legen das Fernglas auf das Regal. They lay the binoculars on the shelf.	Ellas ponen una barra de labios en la tabla de cortar. They(-feminine) put the lipstick on the cutting board.
Neutral	Es gibt Männer, Ferngläser und Regale. There are men, binoculars and shelves.	Hay hombres, pares de binoculares y estantes. There are men, binoculars and shelves.

In the prime and probe display, the critical picture stimuli were 16 colour pictures. The standard prime pictures (prime display) showed an agent placing an object, with the object held at an angle of 45 degrees. Each prime picture was followed by a recognition picture (recognition display) that appeared in one of four conditions. The recognition picture was either identical to the prime pictures or the object disappeared (control conditions); facial gender was changed (this condition is not discussed in this paper, as its focus is on object position); or we changed the object angle (critical condition) (see Figure 2). The object angle was changed from 45 to 65 or 25 degrees, always congruent with the German verb in the prime display. Forty-eight (48) filler sentences and pictures described and showed agents performing other actions with other objects (e.g. They open the door of the office).

Figure 2.

Examples of stimuli in the present study, showing a man putting a pair of binoculars on a shelf at an angle of 25 degrees (left panel) 45 degrees (middle panel) and 65 degrees (right panel).

Procedure

Participants were tested in groups of 10–20 in a computer room. L1 and L2 German speakers were given instructions in German; L1 Spanish speakers were instructed in Spanish. The experimental instructions employed a cover story on binocular vision so participants would not guess the aim of the study. There were three practice trials to familiarize participants with the task. Each participant completed a 3 sentence (prime1, prime2, neutral) × 4 picture (identical, orientation change, facial gender change, object disappears) × 4 scene (binoculars, glue stick, flashlight, lipstick) design, which sums to 48 critical items. In combination with the 48 filler items, there were a total of 96 (randomized) trials. The materials and the data reported below have been uploaded in the IRIS database (www.iris-database.org) and are publicly accessible.

Results

L1 German versus L1 Spanish accuracy

We first ran a 2 (group: German, Spanish) × 3 (picture change: orientation change, facial gender change, object disappears) analysis of variance (ANOVA) on % of noticed picture changes (see Table 4). There was no significant main effect of group, F(1,52) = 3.220, p = .079, η_p² = .059. There was a significant main effect of picture change condition, F(2,104) = 455.529, p < .001, η_p² = .898. Overall, participants were most accurate for the object disappearance condition (M = 90%) and less accurate for the orientation change (M = 14%) and the gender change (M = 7%) conditions. Pairwise comparisons (least significant difference (LSD)) showed that differences between all conditions were significant, p < .04. There was also a reliable interaction between picture change condition and group, F(2,104) = 3.855, p = .024, η_p² = .069. Thus, German and Spanish speakers had different accuracy scores for different picture changes. Pairwise comparisons (LSD) showed a significant difference for the orientation condition for German (M = 22%) and Spanish (M = 6%), p = .002, but not for the other conditions, p > .7 (see Figure 3). This analysis provides support for Hypothesis 1. It shows that L1 German speakers had better memory for object position than L1 Spanish speakers.

Table 4.

Picture recognition task accuracy percentages and reaction times for first language (L1) and second language (L2) speakers.

Group	N	1. % accuracy change in object orientation		2. % accuracy change in facial gender		3. % accuracy object disappears		Average reaction time (1–3) (ms)
		M	SD	M	SD	M	SD	M	SD
L1 German	27	0.22	0.32	0.08	0.15	0.91	0.12	1668	501
L2 GermanA2	58	0.38	0.29	0.14	0.19	0.85	0.22	1894	734
L2 GermanB1	45	0.37	0.30	0.08	0.16	0.90	0.15	1833	678
L2 GermanB2+	20	0.30	0.29	0.22	0.29	0.83	0.23	1769	617

Figure 3.

Interaction between group (first language (L1) German, L1 Spanish) and picture change (orientation, gender, object disappears). Error bars represent 95% confidence intervals.

L2 versus L1 German accuracy

Secondly, we ran a 4 (German group: L1, A2, B1, B2+) × 3 (picture change: orientation change, facial gender change, object disappears) ANOVA on the % of noticed picture changes (see Table 4). There was no significant main effect of group, F(3,146) = 1.082, p = .359, η_p² = .022. There was a significant main effect of picture change, F(2,292) = 434.865, p < .001, η_p² = .749, with accuracy scores for conditions as follows: orientation (M = 32%), gender (M = 13%) and object disappears (M = 87%). Pairwise comparisons (LSD) showed that differences between all conditions were significant with p < .01. There was also a reliable interaction between picture change and group, F(6,292) = 2.365, p = .030, η_p² = .046. To assess whether performance for each picture change condition differed between language proficiency groups, we ran follow-up analyses comparing language groups for each condition separately. Pairwise comparisons (LSD) showed a significant difference between the L1 and L2 German groups for orientation changes, p < .05, but not for other comparisons, p > .05 (see Figure 4).

Figure 4.

Interaction between group (L1, A2, B1, B2+ German) and picture change (orientation, gender, object disappears). Error bars represent 95% confidence intervals.

L1 and L2 German accuracy per sentence type

We also compared L2 and L1 German accuracy for the different sentence types: legen [lay] (prime1), stellen [stand] (prime2) and es gibt [there are] (neutral) (see Table 3). We distinguished sentences with spatial language (legen [lay] and stellen [stand]) from sentences without spatial language (es gibt [there are]). We calculated accuracy percentages with respect to the number of trials (respectively eight and four) (see Table 5). We ran a 4 (group: L1, A2, B1, B2+) × 2 (sentence type: spatial, not spatial) on % of noticed orientation changes. There was no main effect of group, F(3,146) = 2.392, p = .071, η_p² = .047. There was, however, a main effect of sentence type, F(1,146) = 13.357, p < .001, η_p² = .084. On average, accuracy was higher for sentences with spatial language (M = 35.8) as compared with sentences with no spatial language (M = 30.2). The interaction between group and sentence type was not significant, F(3,146) = 2.584, p = 0.056, η_p² = .05 (see Figure 5).

Table 5.

Picture recognition task accuracy percentages for first language (L1) and second language (L2) German speakers, for trials with changes in object orientation.

Group	N	Spatial language		No spatial language
		M	SD	M	SD
L1 German	27	0.24	0.31	0.19	0.34
A2 German	58	0.38	0.30	0.38	0.32
B1 German	45	0.41	0.29	0.30	0.32
B2+ German	20	0.34	0.29	0.23	0.30

Figure 5.

Main effect of sentence type (spatial, not spatial) for first language (L1) and second language (L2) German speakers. Error bars represent 95% confidence intervals.

Analyses in the L2 versus L1 German accuracy and L1 and L2 German accuracy per sentence type sections provide support for Hypothesis 2, as Spanish L2 German learners’ memory for object position was affected by position marked by German verbs and better than that of L1 German speakers. Analyses of sentence type provided further support for the notion that spatial language triggered more accurate memory.

Discussion

The aim of this paper was to provide novel empirical evidence regarding TFS and memory in L1 and L2 speakers.⁴ German and Spanish differ with respect to the semantic information that is coded in their placement verbs, with German expressing the position of the Figure object in relation to the Ground (stellen [stand] versus legen [lay]), a semantic distinction that is absent in the Spanish verbs poner [put] or dejar [leave on a place]. We thus asked: do L1 speakers of these two languages show different recognition memory for object position in placement scenes? And if so, can learning the meaning of L2 German placement verbs improve memory accuracy? In a fine-grained experimental task we registered in critical trials whether participants recognized changes in pictures that were linked to linguistic cues. Following Shiffrin and Schneider (1977), we predicted that L1 German speakers’ recognition memory of object orientation would be more accurate than that of L1 Spanish speakers (Hypothesis 1). Also, we predicted that when Spanish learners of L2 German received instruction on placement verbs, they would show accuracy that exceeded that of L1 German speakers (Hypothesis 2).

With respect to placement verbs and recognition of changes in object position, we found support for both Hypotheses 1 and 2. We found that L1 German speakers were significantly more accurate at spotting changes in object orientation than L1 Spanish speakers, whose score was close to 0% correct. We also found that Spanish learners of L2 German were more accurate than L1 German speakers to spot changes in object orientation. Further support for the spatial effect came from an analysis that showed higher accuracy scores for trials with spatial language as compared with trials with non-spatial language.

Comparison with previous empirical work

The results on placement verbs and recognition of changes in object position comply with previous findings on motion verbs in L1 speakers (Billman et al., 2000; Gennari et al., 2002), balanced bilinguals (Filipović, 2011) and L2 learners (Filipović, 2016) and can be interpreted as evidence in favour of TFS. The results are contrary to those of Feist and Gentner (2007), who found that spatial prepositions led L1 English speakers to falsely recognize changed positions of objects in pictures; to those of Coventry et al. (2010), who found no effects on recognition memory for Spanish-English linguistic instances describing relations of containment and support; and to Bosse and Papafragou (2010), who found no effects on memory for the German verbs stehen [sit] and liegen [lie].

How do we explain the difference between positive (Billman et al. (2000); Gennari et al. (2002); Filipović (2011, 2016); current study) versus negative (Feist & Gentner, 2007) or null (Coventry et al., 2010; Bosse & Papafragou, 2010) effects of language on memory accuracy?

Let us start by looking at the appropriateness of critical language at encoding. The only study that documented negative within-language effects (Feist & Gentner, 2007) is also the only study reporting that the critical language (e.g. spatial prepositions) presented was inappropriate to describe the presented images. One can argue that in studies with positive findings, critical language (e.g. motion verbs) was (more) appropriate, which may explain the difference in directionality of the effect. In studies with null findings (Coventry et al., 2010; Bosse & Papafragou, 2010), however, it can be argued that critical language was appropriate (enough) as well. Do these studies differ from the other studies in different aspects?

Language skill offers no satisfactory explanation, as reading as well as writing led to positive, negative and null results (speaking has only led to positive results so far). A more plausible cause of differences is the time course employed between encoding and memory phase, with short time courses leading to similar results. Yet, we employed the same short time course as that by Coventry et al. (2010), but found different results. Different types of stimuli (drawings versus pictures); the different inclinations employed (different degrees of open–closed versus different degrees of inclination); or the different language phenomenon under study (spatial prepositions versus motion verbs) may then explain the difference between the latter and current findings. We also note that all previous studies employing dynamic stimuli (videos) found positive effects, whereas studies with static stimuli (drawings or pictures) found negative or null effects. This study is the first to document positive TFS effects for caused motion verbs with static stimuli.

What is most stringently clear, however, is that studies with positive effects concern motion verbs, and the negative/null effects appeared in work on spatial prepositions or positional verbs, that is, static language. Is it thus the case that motion verbs systematically affect memory, whereas static language does not? Support for this idea comes from behavioural, electrophysiological, neuropsychological and imaging studies (see Pulvermüller, 2005; Meteyard, Bahrami, & Vigliocco, 2007; Vigliocco, Vinson, Druks, Barber, & Cappa, 2011, for overviews). Wallentin, Ellegaard, Ostergaard, Ostergaard and Roepstorff (2005) and Wallentin et al. (2011), for example, showed that motion verbs in sentences activate the temporal cortex despite a static context (e.g. “The path comes into the garden”), whereas static verbs do not. This finding indicates that dynamic language has powerful effects on parts of our brains. As empirical (bilingual) studies on language and memory for motion and space will culminate, the validity of the dynamic versus static thesis could be examined by means of a meta-analysis in future work.

Implications for learning and teaching placement verbs

Our findings comply with and add to studies that show learning difficulties for placement verbs, but learning success through L2 instruction. Next to English L2 Dutch users; Dutch children; Romansh users of L2 German; Spanish L2 Danish and Danish L2 Spanish users; and French and Italian learners of L2 German, we have documented that Spanish L2 German learners also struggle with the correct meaning of German placement verbs, at beginner, but also at intermediate and advanced levels. Similar to de Knop and Perrez (2014) and de Knop and Mollica (2017), a form-on-focus instruction was successful in increasing learners’ understanding of German legen and stellen. Consequently, learners’ memory for spatial position exceeded that of L1 German speakers when tested immediately after the instruction phase. Future work should investigate whether, in the long run, L2 learners still correctly use and remember placement verbs and, if so, whether recognition memory is still affected.

Filipović (2016) did not instruct her English learners of L2 Spanish and Spanish learners of L2 English on intentionality (un)marked in motion verbs prior to the memory task. Different from our results, she found that Spanish learners of L2 English adhered to L1-entrenched (more specific) patterns in speech. Consequently, their recall memory accuracy did not differ from that of L1 Spanish speakers, but was better than that of English learners of L2 Spanish. It would be interesting to give Spanish learners of L2 English instruction on the lack of expression of intentionality in English causation verbs and vice versa for English learners of L2 Spanish prior to a memory task. In this way we could document whether instruction leads to memory shifts as documented in this study, instead of an upholding L1 dominance while speaking the L2. Further, the present study could be replicated while adding a group of German learners of L2 Spanish that are instructed on Spanish placement verbs. In this way we would find out whether a TFS effect holds when moving in the opposite language direction. If the TFS hypothesis holds, one would expect low memory accuracy scores for object orientation, like those of L1 Spanish speakers in the present study.

Relevance to theory on L1 and L2 information processing and memory

We have shown that our predictions about TFS outcomes could be correctly predicted with Shiffrin and Schneider’s (1977) theory on information processing. Although their investigations did not focus on the effects of language on recognition memory as such, they did focus on the effects of categories on memory accuracy. As language, in our case placement verbs, functions as a categorical distinction marker, in our case the categories horizontal versus vertical, we correctly predicted that spatial language would improve controlled search performance for L1 German speakers. In its turn, this led to higher memory accuracy as compared with L1 speakers, who did not have the benefit of categorical distinctions in language that could be used to encode prime pictures. Consequently, they did not perform a controlled search performance, which lead to low accuracy scores in the recognition phase. Spanish learners of L2 German who received instruction on German placement verbs prior to the memory experiment, however, did succeed in performing controlled search performance with high memory accuracy in turn. Instead of the sheets with digits and consonants employed by Shiffrin and Schneider, this study shows that effects of (language) categories on memory accuracy hold as well for recognition of object position in pictures of actions that humans perform every day, that is, placing objects. Future studies could build on the present study to add insight into the independence–interdependence debate of processing and memory in bilinguals (Abutalebi et al., 2009; Myers-Scotton, 2003). This could be done by testing bilingual participants in both languages and having participants perform tasks in bilingual mode (as in Filipović, 2011). A comparison with monolingual speaker groups can then be made to determine whether results support interdependent or independent processing and storing of information in foreign language learners.

Conclusion

This study has revealed new empirical evidence on language-specific effects on recognition memory for pictures in L1 speakers and L2 learners, using a fine-grained methodology. The main finding is that spatial language affects recognition memory for object position in L1 and L2 speakers. This result can be interpreted as support for the TFS (Slobin, 1996) hypothesis and explained in Shiffrin and Schneider’s (1977) framework on information processing. Future TFS and memory studies should consider employing fine-grained tasks with short time courses to establish a base of controlled and reliable findings to unravel and consolidate the results within this paper and the linguistic relativity literature in general.

Footnotes

Appendix

Acknowledgements

We thank Kenny Coventry for his prominent role in shaping the theoretical, methodological and analytical foundations of this study. We thank our collaborators, hosts and local assistants who enabled data collection at the various test locations. We are also grateful for the insightful comments from an anonymous reviewer that helped us make significant improvements.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the EU 7th Framework Programme Marie Curie Initial Training Networks grant Nr. 316748 under the project Language and Perception.

Notes

Author biographies

Dietha Koster is a postdoctoral Researcher at the department of Dutch studies at the University of Münster, where she lectures on bilingualism, bilingual education and gender linguistics. Her research interests include bilingual cognition, language pedagogy and sociocultural linguistics. She received a PhD in Linguistics at University of Southern Denmark. She completed her doctoral studies within the Marie Curie Initial Training Network “Language and Perception” funded by the 7th European Union Framework Programme.

Teresa Cadierno is a Professor of Second Language Acquisition at the Institute of Language and Communication, University of Southern Denmark. She is a leading scholar on the acquisition and teaching of L2 constructions for the expression of motion events by adult foreign language learners. She has published in journals as Language Learning, the Modern Language Journal and Applied Linguistics and is author of the book Linguistic relativity in second language acquisition: Thinking-for-speaking (co-authored with Han, Z., 2010).

References

Abutalebi

Cappa

Perani

(2009). What can functional neuroimaging tell us about the bilingual brain? In Kroll

de Groot

(Eds.), Handbook of bilingualism (pp. 479–515). Oxford: Oxford University Press.

Bell

(2015). Explicit and implicit learning: Exploring their simultaneity and immediate effectiveness. Applied Linguistics, 38(3), 297–317.

Berthele

(2012). On the use of PUT verbs by multilingual speakers of Romansh. In Kopecka

Narasimhan

(Eds.), Events of putting and taking: A crosslinguistic perspective (pp. 145–166). Amsterdam, Philadelphia: John Benjamins.

Billman

Swilley

Krych

(2000). Path and manner priming: Verb production and event recognition. Proceedings of the 22nd Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum Associates.

Bosse

Papafragou

(2010). Spatial position in language and visual memory: A cross-linguistic comparison. Proceedings from the 32nd annual meeting of the cognitive science society. Hillsdale, NJ: Erlbaum.

Bylund

Athanasopoulos

(2014). Linguistic relativity in SLA: Toward a new research program. Language Learning, 64(4), 952–985.

Cadierno

(2017). Thinking for speaking about motion in a second language: Looking back and forward. In Ibarretxe-Antuñano

(Ed.), Motion and space across languages and applications. Amsterdam: John Benjamins.

Cadierno

Ibarratxe-Antunano

Hijazo-Gascón

(2016). Semantic categorization of placement verbs in L1 and L2 Danish and Spanish. Language Learning, 66(1), 191–223.

Cook

Bassetti

(Eds.). (2011). Language and Bilingual Cognition. New York, NY: Psychology Press.

10.

Coventry

Valdés

Guijarro-Fuentes

(2010). Thinking for speaking and immediate memory for spatial relations. In Han

Z. -H.

Cadierno

(Eds.), Linguistic relativity in L2 acquisition: Evidence of L1 thinking for speaking (pp. 84–101). Clevedon: Multilingual Matters.

11.

de Bot

Jaensch

(2013). What is special about L3 processing? Bilingualism: Language and Cognition, 18(2): 130–144.

12.

de Knop

Mollica

(2017). Construction-based teaching of the German caused motion and intransitive constructions to Italian learners. Deutsch als Fremdsprache: Zeitschrift für Theorie und Praxis des Deutschunterrichts für Ausländer. Retrieved from http://hdl.handle.net/2078/183390

13.

de Knop

Perrez

(2014) Conceptual metaphors as a tool for the efficient teaching of Dutch and German posture verbs. Review of Cognitive Linguistics, 12(1), 1–29.

14.

Ellis

(2001). Investigating form-focused instruction. Language Learning, 51(1), 1–46.

15.

Fagan

(1991). The semantics of the positional predicates liegen/legen, sitzen/setzen, and stehen/stellen. Die Unterrichtspraxis / Teaching German, 24, 136–145.

16.

Feist

Gentner

(2007). Spatial language influences memory for spatial scenes. Memory & Cognition, 35(2), 283–296.

17.

Filipović

(2011). Speaking and remembering in one or two languages: Bilingual vs. monolingual lexicalization and memory for motion events. International Journal of Bilingualism, 15(4), 466–485.

18.

Filipović

(2016). Speaking in a second language but thinking in a first language: Language-specific effects on memory for causation events in English and Spanish. International Journal of Bilingualism, 20(1), 1–19.

19.

Gardiner

(1988). Functional aspects of recollective experience. Memory and Cognition, 16, 309–313.

20.

Gennari

Sloman

Malt

Fitch

(2002). Motion events in language and cognition. Cognition, 83, 49–79.

21.

Grosjean

(2001). The bilingual’s language modes. In Nicol

(Ed.), One mind, two languages: Bilingual language processing (pp. 1–22). Oxford: Blackwell.

22.

Grosjean

Soares

(1984). Processing mixed languages: Some preliminary findings. In Vaid

(Ed.), Language processing in bilinguals: Psycholinguistic and neurological perspectives (pp. 145–179). Hillsdale, NJ: Lawrence Erlbaum Associates.

23.

Gullberg

(2009). Reconstructing verb meaning in a second language. How English speakers of L2 Dutch talk and gesture about placement. Annual Review of Cognitive Linguistics, 7, 221–224.

24.

Hawkins

(1984). Awareness of language: An introduction. Cambridge: Cambridge University Press.

25.

Ibarretxe-Antunano

(2012). Placement and removal events in Basque and Spanish. In Kopecka

Narasimhan

(Eds.), Events of putting and taking: A crosslinguistic perspective (pp. 145–166). Amsterdam, Philadelphia: John Benjamins.

26.

Jackendoff

(1990). Semantic structures. Cambridge, MA: MIT Press.

27.

Kopecka

Narasimhan

(Eds.). (2012). Events of Putting and Taking, a crosslinguistic perspective. Amsterdam: Benjamins.

28.

Lemmens

(2006). Caused posture: Experiental patterns emerging from corpus research. In Stefanowitsch

Gries

(Eds.), Corpora in cognitive linguistics. Vol II: The syntax-lexis interface. Berlin: Mouton De Gruyter.

29.

Levinson

(2012). Preface. In Kopecka

Narasimhan

(Eds.), Events of putting and taking: A crosslinguistic perspective. Amsterdam: Benjamins.

30.

Loewen

(2012). Instructed second language acquisition. In Chapelle

(Ed.), Encyclopaedia of applied linguistics. UK: Wiley Blackwell.

31.

Maigiste

(1982). Automaticity and interference in bilinguals. Psychological Research, 44, 29–43.

32.

Maigiste

(1985). Development of intra- and interlingual interference in bilinguals. Journal of Psycholinguistic Research, 14, 137–154.

33.

Malt

Sloman

Gennari

(2003). Speaking versus thinking about objects and actions. In Gentner

Goldin-Meadow

(Eds.), Language in mind: Advances in the study of language and thought (pp. 81–111). Cambridge, MA: MIT Press.

34.

Meteyard

Bahrami

Vigliocco

(2007). Motion detection and motion verbs. Psychological Science, 18(11), 1007–1013.

35.

Myers-Scotton

(2003). Multiple voices: An introduction to bilingualism. Oxford: Blackwell.

36.

Narasimhan

Gullberg

(2011). The role of input frequency and semantic transparency in the acquisition of verb meaning: Evidence from placement verbs in Tamil and Dutch. Journal of Child Language, 38, 504–532.

37.

Norris

Ortega

(2001). Does type of instruction make a difference? Substantive findings from a meta-analytic review, Language Learning, 51(1), 157–213.

38.

Papafragou

Selimis

(2010). Event categorization and language: A cross-linguistic study of motion. Language and Cognitive Processes, 25(2), 224–260.

39.

Pavlenko

(2014). The bilingual mind. And what it tells us about language and thought. Cambridge: Cambridge University Press.

40.

Pienemann

(2003). Language processing capacity. In Doughty

C. J.

Long

M. H.

(Eds.), The handbook of second language acquisition (pp. 679–714). Malden, MA: Blackwell.

41.

Pulvermüller

(2005). Brain mechanisms linking language and action. Nature Reviews Neuroscience, 6(7), 576–582.

42.

Sanz

(2000). Bilingual education enhances third language acquisition: Evidence from Catalonia. Applied Psycholinguistics, 21, 23–44.

43.

Schmidt

(1990). The role of consciousness in second language learning. Applied Linguistics, 11(2), 129–158.

44.

Shiffrin

Schneider

(1977). Controlled and automatic human information processsing: II perceptual learning, automatic attending, and a general theory. Psychological Review, 84(2), 127–190.

45.

Slobin

(1996). From “thought and language” to “thinking for speaking.” In Gumperz

Levinson

(Eds.), Rethinking linguistic relativity. Studies in the social and cultural foundations of language (Vol. 17, pp. 70–96). Cambridge: Cambridge University Press.

46.

Slobin

(2003). Language and thought online. Cognitive consequence of linguistic relativity. In Gentner

Goldin-Meadow

(Eds.), Language in mind: Advances in the study of language and thought (pp. 157–192). Cambridge, MA: MIT Press.

47.

Spada

Lightbown

(1993). Instruction and the development of questions in L2 classrooms. Studies in Second Language Acquisition, 15, 205–224.

48.

Talmy

(1985). Lexicalization patterns: Semantic structure in lexical forms. In Shopen

(Ed.), Language typology and syntactic description, Vol. 3: Grammatical categories and the lexicon, (pp. 36–149). Cambridge: Cambridge University Press.

49.

Talmy

(2000). Toward a cognitive semantics: Typology and process in concept structuring (Vol. II). Cambridge, MA: MIT Press.

50.

Tulving

(1985). Memory and consciousness. Canadian Psychology, 26, 1–12.

51.

van Bergen

Flecken

(2017). Putting things in new places: Linguistic experience modulates the predictive power of placement verb semantics. Journal of Memory and Language, 92, 26–42.

52.

Vigliocco

Vinson

Druks

Barber

Cappa

(2011). Nouns and verbs in the brain. A review of behavioural, electrophysiological, neuropsychological and imaging studies. Neuroscience and Biobehavioral Reviews, 35(3): 407–426.

53.

Wallentin

Ellegaard

Ostergaard

Roepstorff

(2005). Motion verb sentences activate left posterior middle temporal cortex despite static context. Cognitive Neuroscience and Neuropsychology, 16(6), 649–652.

54.

Wallentin

Hojlund Nielsen

Vuust

Dohn

Roepstorff

Ellegaard Lund

(2011). BOLD response to motion verbs in left posterior middle temporal gyrus during story comprehension. Brain and Language, 119(3), 221–215.

55.

Wolff

Holmes

(2010). Linguistic relativity. WIREs Cognitive Science, 2(3), 253–265.

56.

Yonelinas

(2002). The nature of recollection and familiarity: A review of 30 years of research. Journal of Memory and Language, 46(3), 441–517.