Abstract
Four experiments examined perceptuo-motor associations involved in spatial knowledge encoding and retrieval. Participants learned spatial information by studying a map or by navigating through a real environment and then verified spatial descriptions based on either egocentric or cardinal directional terms. Participants moved the computer mouse to a YES or NO button to verify each statement. We tracked mouse cursor trajectories to examine perceptuo-motor associations in spatial knowledge. An encoding hypothesis predicts that perceptuo-motor associations depend on the involvement of perceptions and actions during encoding, regardless of how spatial knowledge would be used. The retrieval hypothesis predicts that perceptuo-motor associations change as a function of retrieval demands, regardless of how they are learned. The results supported the retrieval hypothesis. Participants showed action compatibility effects with egocentric retrieval, regardless of how spatial information was learned. With well-developed spatial knowledge, a reliable compatibility effect emerged during egocentric retrieval, but no or limited compatibility effects emerged with cardinal retrieval. With less-developed knowledge, the compatibility effects evident during cardinal retrieval suggest a process of egocentric recoding. Other factors of environment learning, such as location proximity and orientation changes, also impacted the compatibility effect, as revealed in the temporal dynamics of mouse movements. Taken together, the results demonstrate that retrieval demands differentially rely upon perceptuo-motor associations in long-term spatial knowledge. This effect is also modulated by environment experience, proximity of learned locations, and experienced orientations.
Keywords
Introduction
People learn environments by navigating through them or from symbolic models, such as maps or virtual environments. When learning via navigation, information you gather is grounded in experienced actions and perceptions. This affords situated simulations of these perceptual and motor associations when later retrieving environment information (Wang, Taylor, & Brunyé, 2012). What happens when learning an environment from a symbolic model, like commonly used two-dimensional maps? Maps depict an environment from a survey perspective, making relative landmark locations readily apparent, and likely not engaging the same degree of perceptuo-motor involvement as learning through navigation. However, people often use maps to inform and guide navigation between locations (Bjerva & Sigurjónsson, 2017; Taylor, Naylor, & Chechile, 1999), transforming a survey perspective into physical navigation experiences from an egocentric perspective, a task heavily reliant on perceptual and motor activity (Montello, 2005). In general, people use spatial knowledge, regardless of its source, for different purposes that may have varying degrees of perceptual or motor involvement. This leads to a critical question. Are perceptual and motor associations evident in memory as a function of how spatial information is learned (encoding) or how it will be used (retrieval)? Specifically, the present work examines spatial representations encoded from maps and through real-world navigation that are then accessed relative to egocentric or cardinal directions.
Embodied spatial cognition
Spatial knowledge is crucial to nearly all species; to survive, human and non-human animals must remember where food is cached or where safe-hiding places can be found. Psychological research has long explored the nature of spatial representations. Traditional theories of spatial knowledge development propose that well-developed spatial representations take on an image-like, survey format, similar to a map (Siegel & White, 1975; Thorndyke & Hayes-Roth, 1982). Recent studies suggest that navigators’ representations also involve sensorimotor information (Brunyé, Mahoney, & Taylor, 2010; Wang et al., 2012; Zetzsche, Wolter, Galbrath, & Schill, 2009), consistent with the Theory of Event Coding (TEC) by Hommel, Müsseler, Aschersleben, and Prinz (2001). TEC proposes that perceptions and actions are coded in a common representational system and both would likely be activated if either were retrieved (Hommel, 2007, 2011). In other words, if the perceptual representation of the spatial relation between two landmarks was retrieved, spontaneous activation of related motor representations would occur (e.g., Wang et al., 2012), or vice versa. For instance, moving to the left could influence the visual perception of directional information, such as a left-pointing arrow (Wykowska, Schubö, & Hommel, 2009).
One essential difference between humans and other species is that human spatial cognition has become intertwined with language (Arsenijević, 2008; Libby, Shaeffer, & Eibach, 2009; Tversky & Lee, 1998). While spatial cognition influences meaning in many domains, like space, time or even social status (Feist, 2009; Feist & Gentner, 2007; Gagnon, Brunyé, Robin, Mahoney, & Taylor, 2011; Landau & Jackendoff, 1993), language may also structure spatial knowledge (Pyers, Shusterman, Senghas, Spelke, & Emmorey, 2010; Shusterman, Ah Lee & Spelke, 2011). Theories on grounded or embodied cognition suggest that language comprehension activates perceptuo-motor experiences (e.g., Glenberg & Kaschak, 2002; Kemmerer, Gonzalez, Talavage, Patternson, & Wiley, 2008). For example, when judging the sentence “close the drawer,”, participants responded faster with the “Yes” button farther away compared to when it was closer to the body. The farther location matches the movement one would make when closing a drawer (Glenberg & Kaschak, 2002). These types of compatibility effects, together with TEC, predict that reading spatial terms (e.g., left or right) would activate associated perceptions and actions, as observed in Wang et al. (2012). Accordingly, in the present studies, we expect that when participants retrieve spatial knowledge to understand directional terms, they will spontaneously reactivate perceptuo-motor.
Nevertheless, the debate on the extent to which abstract, intangible concepts also integrate and reuse bodily experiences keeps appearing in embodied cognition theories. Spatial knowledge can engage more abstract reference systems, ones more distant from perceptual-motor experiences, such as cardinal directions (north, south, east or west). While egocentric terms readily involve concrete body-centred perception and actions (Brunyé, Gardony, Mahoney, & Taylor, 2012; Wang et al., 2012), few studies have explored perceptuo-motor processing associated with more abstract spatial terms, especially when retrieving from long-term representations. In one exception, Tower-Richardi, Brunyé, Gagnon, Mahoney, and Taylor (2012) suggest that people associate cardinal directions to a body-centred reference frame in pointing tasks, for example, north-up or east-right. If so, explicit associations between abstract and concrete spatial concepts could engage perceptuo-motor processing. However, it is unclear whether retrieving environmental knowledge from long-term memory would evoke such perceptuo-motor associations.
Action-compatibility paradigm
To examine perceptuo-motor associations, we used an action-compatibility paradigm (Glenberg & Kaschak, 2002; Wang et al., 2012; Zwaan & Taylor, 2006). This paradigm can provide behavioural evidence of perceptual and/or motor re-instantiation if memory retrieval interacts with the motor processes needed to respond. Language research has used action-compatibility paradigms to explore perceptuo-motor reactivation with action word comprehension. For instance, in Zwaan and Taylor (2006), participants made sensibility judgements about sentences such as “Mark turned left at the intersection,” more quickly when turning the response knob leftward compared to rightward. If the direction of perceptuo-motor reactivation is compatible with the response motion, an action-compatibility effect (ACE), or response enhancement, occurs (Kaschak & Borreggine, 2008; Knoblich & Flach, 2001; Repp & Knoblich, 2004).
Our action compatibility paradigm employs mouse tracking (Freeman & Ambady, 2010). Hints from gesture literature suggest a connection between spatial knowledge retrieval and hand movements (Alibali, 2005; Chu & Kita, 2011; Emmorey, Tversky, & Taylor, 2000; Hostetter & Alibali, 2008; Novack & Goldin-Meadow, 2017; So, Ching, Lim, Cheng, & Ip, 2014; So, Shum, & Wong, 2015). People use gestures to describe spatial information, with or without a conversational partner (Emmorey et al., 2000). Spatial problem solving, even without the goal to communicate to others, also elicits gestures (Chu & Kita, 2011; Tversky & Kessel, 2014). When gestures are impeded, spatial descriptions become slower and less fluent. Moreover, people adopt different gestures when describing spatial information from different perspectives or for different reasons (Emmorey et al., 2000; Galati, Weisberg, Newcombe, & Avraamides, 2018; So et al., 2014). While the mapping between hand movements and spatial representations might not be as direct as with other typical actions and responses, such as giving or grasping objects, the spontaneous use of gestures with spatial descriptions, even to an imaginary conversational partner, strongly suggests a connection between hand movements and spatial representations. So et al. (2015)’s work suggested hand movements are more effective in encoding spatial information than spatial language, supporting that gestures reflect spatial mental representations rather than the semantics of the spatial description. As such, tracking mouse movements when verifying spatial relations should reveal motor associations integrated with spatial knowledge (Barsalou, 1999; Beilock & Holt, 2007; Fischer & Zwaan, 2008; Glenberg, 1997; Goldin-Meadow & Beilock, 2010). Further, mouse tracking can reveal the magnitude and time course of any ACEs (Freeman & Ambady, 2010; Richardson, Dale, & Spivey, 2007; Spivey, Richardson, & Dale, 2009).
Using this methodology, our previous work showed ACEs at retrieval for spatial knowledge acquired from navigation (Wang et al., 2012). Participants verified, by moving the mouse either leftward or rightward, whether egocentric direction terms (e.g., left/right) correctly described the spatial relation between two campus locations. Mouse movements in a direction incompatible with the real spatial relation or with the directional term showed bias towards the compatible direction. Take the Student Union and Library in Figure 1 as an example. Suppose that the front door of each landmark is located at the building’s south side. The probe description “Student Union-Library-right” asks participants to verify that the Library is to the right of the Student Union (when facing the front door of Student Union). In this case, the Library (target building) is actually to the left of Student Union (reference building), given the facing direction. A spatially based ACE would attract mouse movements towards left, the direction defining the spatial relationship. In contrast, a semantically based ACE would attract mouse movement towards right, as the direction the term “right” describes. Note that only with incorrect descriptions can we disentangle spatially based and semantically based ACEs, as the spatially and semantically based ACEs generally deviate mouse movements towards opposite directions in incorrect descriptions. That is, the spatially congruent condition is semantically incongruent, and vice-versa. Responses in Wang et al. (2012) showed ACEs that differed in their underlying sources, depending on the extent of participant’s knowledge. Participants with more extensive experience navigating the tested environment had a spatially driven ACE, evident when response direction matched the direction of the spatial relation between locations. Participants with less experience showed a semantically driven ACE, that is, related to the spatial term. These findings show activation of perceptuo-motor associations as suggested by TEC (Hommel, 2004, 2007).
One (of two) modified map used in Experiments 1–3.
Wang et al.’s (2012) finding suggests that spatial representations can differ both in the extent to which they involve perceptuo-motor associations and the basis of the associations. It also showed that the extent of environment experience impacted these associations. What other factors might impact the extent to which spatial representations incorporate perceptuo-motor associations? Taylor et al.’s (1999) studies suggest that both encoding source and retrieval goal impact spatial representations. With encoding, navigating through an environment involves extensive perceptual and motor processes, while studying a map usually only involves visual perception (Evans & Pezdek, 1980; Montello, Waller, Hegarty, & Richardson, 2004; Presson & Hazelrigg, 1984; Sholl, 1987; Thorndyke & Hayes-Roth, 1982). Retrieval differences emerge based on how spatial information is or will be used (Brunyé, Rapp, & Taylor, 2008; Taylor et al., 1999). For example, to explain how to get to one’s house, someone may give route directions, draw a map, or both. Generating the route direction likely involves mentally simulating the route, including the actions and sights involved. Drawing a map may instead involve creating a mental picture of relative locations. The present studies explore how encoding source, retrieval demands, and the extent of environment exposure impact evidence of perceptuo-motor associations in spatial knowledge, extending Wang et al. (2012). We proposed the encoding and retrieval hypotheses in considering how perceptuo-motor associations may be involved with spatial knowledge.
Encoding hypothesis
The encoding hypothesis proposes that perceptuo-motor associations are built during encoding and are reactivated at retrieval. Thus, learning via navigation should show greater perceptuo-motor associations than learning from a map. As described earlier, some extant research (Wang et al., 2012) showed ACEs after navigation. Here, we explore map learning. Learning from a map does not necessarily require actions (although some people rotate their map), but does involve visually examining the map (Evans & Pezdek, 1980; Montello, Waller, Hegarty, & Richardson, 2004; Presson & Hazelrigg, 1984; Sholl, 1987; Thorndyke & Hayes-Roth, 1982). Accordingly, spatial representations acquired from maps would not have the same level of vestibular or proprioceptive input. Thus, the encoding hypothesis predicts minimal perceptuo-motor associations, i.e., minimal ACE, when environmental knowledge is gained from maps.
Retrieval hypothesis
The retrieval hypothesis predicts that activation of perceptuo-motor associations depends on how the information is used, regardless of how it was learned. In other words, whether an ACE would be observed depends on anticipated perceptual or motor processes when using spatial knowledge. If anticipating navigation, retrieval of the spatial representations would engage navigation-related perceptuo-motor processes (Brunyé, Ditman, Mahoney, Augustyn, & Taylor, 2009; Brunyé, Mahoney, Gardony, & Taylor, 2010) and interact with body-defined processes preparing to respond, e.g., moving right or left to press a response button. Wang et al. (2012) showed ACEs when people retrieved spatial knowledge in response to egocentric coordinates. In contrast, mental imagery studies suggest image-based retrieval of map knowledge (Kosslyn, 1994; Thorndyke & Hayes-Roth, 1982). More specifically, Sholl (1995) proposed that map knowledge is retrieved based on a 2-D retinal coordinate system and that spatial direction and distance information are represented in allocentric vector space with vertical and horizontal axes. Thus, when anticipating map-based information verification, no motor and some perceptual associations would be activated. In this case, evidence of an ACE would be absent or weak.
Studies on stimulus-response spatial congruency, known as the spatial Stroop or Simon effect, predict some perception-based associations with map knowledge (De Houwer, 2003; Hommel, 2011; Hommel & Müsseler, 2006; Müsseler & Hommel, 1997; Simon, 1990). With the spatial Stroop, people respond more slowly when a visual stimulus and a response key are in opposite directions, even when the target position is irrelevant to the response.
In the present work, participants retrieved spatial knowledge in response to either egocentric (left, right, forward, or back) or cardinal (north, south, west, or east) coordinate system. The retrieval hypothesis predicts that egocentric coordinate system would evoke perceptuo-motor associations (perhaps in preparation for navigation), leading to an ACE. However, cardinal coordinate retrieval would have fewer associations to one’s body (Levinson, 1996; Majid, Bowerman, Kita, Haun, & Levinson, 2004; Taylor & Tversky, 1992) and consequently should interact less with response motions, i.e., an absent or reduced ACE.
Location proximity: implications for encoding and retrieval hypotheses
Responses to locations of different relative proximity should help differentiate between the encoding and retrieval hypotheses. When navigating, one interacts perceptually and motorically with locations near their current location, and consequently near one another. With maps, because they take a bird’s-eye viewpoint, spatial information for both distant and proximal landmarks is readily evident. Accordingly, the encoding hypothesis should show greater perceptuo-motor processing with proximal compared to distal locations learned through navigation (Carlson & Kenny, 2006; Longo & Lourenco, 2007; Wang et al., 2012). Proximity should impact perceptuo-motor associations with map knowledge to a lesser degree. While distance information can be evident in map knowledge, it emerges with specific task demands, such as mental scanning (Kosslyn, Ball, & Reiser, 1978). If the retrieval hypothesis holds, proximity would affect egocentric, but not cardinal, retrieval, regardless of encoding. Supporting this view for cardinal retrieval, proximity does not seem to affect response compatibility with the Spatial Stroop (Lamberts, Tavernier, & D’Ydewalle, 1992; Nicoletti & Umiltà, 1989, 1994; Rizzolatti, Riggio, & Sheliga, 1994).
Spatial knowledge development
It takes time to fully develop a mental representation of a complex environment. With representations based on navigation, some work suggests that navigation-based representations systematically progress from including landmarks to routes to configurations (Siegel & White, 1975). A counter claim suggests that these information types develop in parallel (Evans, Marrero, & Butler, 1981; Hermer & Spelke, 1996, 1994; Montello, 1998; Yeap & Jefferies, 2000). Another view suggests that spatial representations preserve perceptuo-motor information derived from the navigation experience (Regier & Carlson, 2001; Taylor & Brunyé, 2013; Tversky, 2005; Tversky & Hard, 2009). This embodied notion contrasts with more abstract, amodal network codes for spatial knowledge (Kuipers, 1978; McNamara, 1991; Stevens & Coupe, 1978). Representations based on maps, in contrast, have the flexibility to incorporate landmark, route, and configuration information relatively early in development (Brunyé et al., 2008; Brunyé & Taylor, 2008; Taylor & Tversky, 1992; Thorndyke & Hayes-Roth, 1982). While navigators need more experience to develop survey representations of environments, map learners might need more practice applying that knowledge to actual navigation (Brunyé et al., 2008). These findings suggest greater change as navigation-based representations develop, compared to map-based representations. To the extent that spatial representations involve perceptuo-motor associations, we should see more change as a function of environment exposure with navigation-based than map-based representations.
Summary
The present work explores how encoding and retrieval impact the involvement of perceptual and/or motor processing with spatial representations and their development. Experiments 1 through 3 use map encoding, but differ in terms of retrieval demands: egocentric (Experiment 1), cardinal (Experiment 2), or mixed (Experiment 3) coordinates. Experiment 4 explores cardinal coordinate retrieval after navigation encoding. Cross-experiment comparisons, including those to Wang et al. (2012) holding constant either encoding source or retrieval coordinates allow exploration of the encoding and retrieval hypotheses. We collected mouse movement data using Mouse Tracker (Freeman & Ambady, 2010; Richardson et al., 2007; Spivey et al., 2009). The encoding hypothesis predicts ACE with navigation learning, but map learning should yield either a weak or no ACE. The retrieval hypothesis predicts an ACE when retrieving with egocentric, but not with cardinal coordinates. We have summarised the presence of ACE predicted by hypotheses based on retrieval demands in Table 1.
The presence of ACE predicted by hypotheses based on retrieval demands.
Experiment 1: egocentric coordinate retrieval of map knowledge
In Experiment 1, participants learned a map and then judged egocentrically defined spatial relations. By comparing to navigation-based learning in Wang et al. (2012), the study examines how encoding affects perceptuo-motor associations.
Our encoding hypothesis predicts that map learning should yield either a weak or no ACE. A weak ACE may emerge based on perceptual processing, promoted by maintaining a stable map orientation during study. People generally keep a map in one orientation when looking at it, leading to orientation-specific representations (Montello et al., 2004; Rieser, 1989; Sholl, 1995, 1999; Sholl & Nolin, 1997; Tlauka & Wilson, 1996). Then, verifying spatial relations from a novel, unlearned orientation takes longer and involves reorientation (Rieser, 1989; Sholl, 1995; Sholl & Nolin, 1997). This orientation specificity has two implications for the current work. First, with the map kept at a stable orientation, people may visually relate map locations to one another using body-based coordinates (e.g., “the Eiffel Tower is to the left of the Museé de Louvre on the Paris map”). If so, retrieving information from the studied orientation may result in some perceptuo-motor interference, akin to the Spatial Stroop. Second, thinking about the environment from a different orientation (mental reorientation) may engage perceptual-motor processes, similar to those shown with mental rotation (Amorim, Isableu, & Jarraya, 2006; Chu & Kita, 2008, 2011; Richter et al., 2000). Such perceptuo-motor processing driven by mental rotation may interfere with that driven by spatial knowledge, leading to a weaker ACE, compared to the situation without mental rotation involvement.
To examine perceptuo-motor processes tied to the learned orientation, participants retrieved map knowledge from either the studied or an unstudied orientation. In both cases, participants imagined facing a particular direction and then verified whether a verbal description accurately described a spatial relation. The encoding hypothesis predicts that ACEs would either not be evident or would be weak with map knowledge. However, responses may reflect some mental reorientation processing with the novel orientation. The retrieval hypothesis predicts ACEs at retrieval with the egocentric frame of reference.
Method
Participants
Forty-nine college-age volunteers (20 male, 29 female) participated individually for monetary compensation. One male participant reported not remembering the map after a long learning period and did not complete the study.
Materials
We created two versions of a fictitious college campus map by modifying the University of California, Riverside map (see Figure 1). The maps depicted 31 buildings, located within a square area; other buildings on the original map were removed. We named all buildings by functions (e.g., dormitory, biology, admissions, and gymnasium) familiar to college students. The two versions differed in label placement, thus controlling for association between specific buildings, locations, and function names. Both maps included a compass rose with north pointing towards the top of the page.
The study used two tasks, a labelling task during learning and a spatial verification task during test. Materials for the labelling task included the maps without labels. Materials for the spatial verification task involved pairs of building function names and spatial relation terms defined egocentrically (left, right, back and forward). Buildings in a location pair were either proximal to or distant from each other. For example, the Education building may be proximal to library, but distant from administrative office. The experiment also used two questionnaires, one demographics and the other for self-rating of sense of direction and memory.
Design
The study used a 2 (Description correctness: correct vs. incorrect description) × 2 (Congruency: congruent vs. incongruent response) × 2 (Proximity: proximal vs. distant locations) × 2 (Facing direction: imagine facing-north vs. facing-east) × 2 (Familiarity: low- vs. high-familiarity) mixed-design.
Description Correctness reflected whether the verification description correctly described the spatial relation between landmarks. An incorrect description always described the opposite spatial relationship from the actual one, rather than an orthogonal direction.
Congruency, used to examine ACE, reflected the relation between mouse movement direction for an accurate response and the direction defined either by the spatial relationship between locations or the directional term. Response movement is spatially congruent if it is in the same direction as that defined by the spatial relationship between buildings. Semantic congruency compares mouse movement direction to the direction named by the spatial term. For incorrect descriptions, spatial congruent condition for spatial congruency is semantic incongruent condition for semantic congruency, vice versa for the spatial incongruent condition and semantic congruent condition. As fillers, we included trials unrelated to leftward or rightward movement (“forward” and “back”), but did not analyse them.
Proximity defines the physical proximity between buildings. Proximal trials described buildings physically adjacent to each other, while with distant trials there are more than one building or landmark between building pairs.
Facing direction oriented participants through instructions, such as “imagine you are on the west side of the building facing east.” Half of participants imagined facing north during retrieval while the other half imagined facing east.
Familiarity with the environment was manipulated by having different study criteria. Half of the participants (low-familiarity) had to correctly label a blank map one time during learning, while the other half (high-familiarity) had to correctly label the blank map three times. Description correctness, congruency, and proximity served as within-participant variables; facing-direction and familiarity served as between-participant variables.
Response button position and map version served as control variables. Half of the participants had the “Yes” button at the upper right corner and the “No” button at the upper left corner, and half had the reverse placement. Half of participants studied each version of the map.
Procedure
Participants completed the study in two phases. In the learning phase, they studied the campus map for 2 min. To test learning, they labelled a blank map. The experimenter then checked their map. Participants continued a 2-min study and then label cycle until they could correctly label (100%) the map one time (low-familiarity) or three times (high-familiarity). Once participants reached their criterion, they moved on to the test phase.
For each spatial verification trial, a start button appeared, centred at the bottom of the 22’ monitor. Participants clicked the start button and the reference building name appeared, centred on the screen for 1000 ms, followed below it by the target building name, for another 1000 ms, and finally below the target name a directional term (“left,” “right,” “forward” or “back”). Participants were instructed to imagine being south/west side of the first presented building and facing north/east, and then judge if the second building is on their left/right/forward/back. That is to judge whether the directional term correctly described the relative location of the target building to the reference building (e.g., Parking garage, Gymnasium, Left). In all, 500 ms after the directional term appeared, the “YES”/”NO” response buttons appeared in the upper left and right corners and the mouse became active and visible. The building names, directional term, and response buttons remained visible until the participant responded. If participants responded accurately, the next trial started. For inaccurate responses, an “X” appeared for 1000 ms. When participants took longer than 5 s to respond, “time out” appeared. After each trial, the screen was blank for 500 ms before the next trial.
At the beginning of spatial verification test, participants completed 6 practice trials to familiarise themselves with the procedure and reinforce the imagined facing direction. The actual test involved 96 trials, presented in random order, including 10 trials each in the 8 conditions defined by the three within-participants variables (description correctness, congruency and proximity) plus 16 forward/back filler trials.
Dependent variables and coding
Dependent variables included accuracy, response time (RT), and mouse trajectory data, all recorded by MouseTrackerTM (Freeman & Ambady, 2010, http://www.mousetracker.org/). MouseTrackerTM records the mouse’s raw position data (x-y coordinates) over time, allowing us to calculate three trajectory-related variables—initial time, area under the curve (AUC), and proportional Euclidean proximity (PEP). Initial time measured the time between when the mouse became active and when the participant first moved it. Initial time may indicate action planning and/or confidence. Both RT and initial time are measured in milliseconds. AUC is the area between the actual and an idealised (a straight line between the start and response button) trajectory. Thus, positive AUC, with the mouse trajectory above the idealised trajectory, indicates mouse movement attracted towards the opposite response and negative AUC, with the mouse trajectory under the idealised one, indicates mouse movement attracted by the expected response. PEP also measures the extent to which the mouse moved closer to the opposite response and is calculated as, 1-distance/max (distance). “Distance” in the equation represents the Euclidean proximity between the mouse position at each time-step and the wrong response button. The AUC and PEP data correlate highly with each other, as both of them reflect the extent of mouse attraction to the opposite response. While the AUC showed a general pattern of the deviation of mouse movements, PEP data could demonstrate the temporal dynamic of mouse movement, as it analysed the mouse position by time-steps. To do so, MousetrackerTM rescales individual trial trajectories into a standard coordinate space and normalises them into 101 time-steps, using linear interpolation. We combined the time-steps into five time bins (time-steps: 1–20, 21–40, 41–60, 61–80, 81–101) and averaged PEP for each time bin. Trajectory plots show the 101 time-steps.
Results
One high-familiarity participant was eliminated from analyses due to low verification accuracy (less than .55). For the data analysis of between-participant conditions, there were 12 participants in low-familiarity & facing-north condition, 12 participants in low-familiarity & facing-east condition, 11 participants in high-familiarity & facing-north condition, and 12 participants in high-familiarity & facing-east condition. A t-test indicated that the hit rate (M = .91, SD = .07) and correct rejection rate (M = .92, SD = .05) were greater than our elimination criteria, .55; t(46) = 37.68, p < .001; t(46) = 49.57, p < .001. During the map study task, as would be expected high-familiarity participants required significantly more repetitions of map study and testing (M = 4.83, SD = .65) than low-familiarity participants (M = 3.08, SD = .83), t(45) = 7.91, p < .001, to reach criterion.
A preliminary analysis showed no effect of map version, response button position and gender. Thus, we collapsed data across these variables, and in the case of button position, reverse scored for one button position. Correct and incorrect description trials were analysed separately due to their interaction with spatial and semantic definition of congruency.
The data were submitted into a 2 (Proximity: proximal vs. distant location pairs) ×2 (Congruency: congruent vs. incongruent) × 2 (Facing direction: imaging facing-north vs. facing-east) × 2 (Familiarity: low-familiarity vs. high-familiarity group) multivariate analysis of variance (ANOVA) with accuracy, RT, initial time and AUC as dependent measures. The multivariate approach provides protection against inflation of type I errors (Simmons, Nelson, & Simonsohn, 2011). For RT, initial time and AUC, only data of accurate responses were submitted into analysis. We analysed PEP data separately, using time bins as an additional within-participant variable. This allowed us to examine temporal dynamics of responses. We also report the Bayes Factors (BF01) for each univariate test result to show the extent that the observed data favour the null hypothesis over the alternative hypothesis. When BF01 < 1, the data are more likely under the alternative hypothesis than the null hypothesis. When BF01 < .033, the data very strongly favour the alternative hypothesis over the null hypothesis. Accordingly, when BF01≈1, the data show no evidence for either null or alternative hypothesis. When BF01 > 3, the data show moderate evidence for the null hypothesis over the alternative hypothesis (Jarosz & Wiley, 2014; Masson, 2011; Rouder, Morey, Verhagen, Swagman, & Wagenmakers, 2017; Wagenmakers et al., 2018). For example, BF01 = 10 indicates that the observed data are 10 times more likely under the null hypothesis that postulates the absence of the effect then under the alternative hypothesis that postulates the presence of the effect. Only significant results are reported; effects not reported can be assumed non-significant (ps > .1 and BF01 > 3).
Correct descriptions
The multivariate results showed main effects of proximity, F(4, 40) = 19.56, p < .001, and congruency, F(4, 40) = 11.77, p < .001, and a proximity × familiarity interaction, F(4, 40) = 2.64, p = .048. The univariate analyses, described below, further explain these effects.
Accuracy & RT. Accuracy, F(1, 43) = 53.21, p < .001, mean square error (MSE) = .01, BF01 < .001, and RT, F(1, 43) = 11.36, p = .002, MSE = 72,273.4, BF01 = .026, data showed proximity main effects. Participants made more errors when judging proximal (M = 0.87, 95% confidence interval (CI) = [.84, .90]) than distant locations (M = .96, 95% CI = [.94, .99]) and took more time to judge proximal (M = 1512.4 ms, 95% CI = [1433.2, 1591.5]) than distant relations (M = 1380.1 ms, 95% CI = [1278.4, 1481.7]). The RT effect was qualified by an interaction between proximity and familiarity, F(1, 43) = 4.60, p = .038, BF01 = .559. Low-familiarity, d =Mproximal – Mdistant = 216.5 ms, F(1, 45) = 15.94, p < .001, but not high-familiarity participants, d = Mproximal – Mdistant = 48.1 ms, F(1, 45) = .74, p = .395, showed the proximity effect.
Mouse Trajectory Results. Mouse trajectory data also showed a proximity effect in initial time, F(1, 43) = 4.35, p = .043, MSE = 23,375.8, BF01 = .616, and AUC, F(1, 43) = 5.45, p = .024, MSE = .20, BF01 = 2.263. Participants took more time to initially move the mouse with proximal (M = 315.0 ms, 95% CI = [237.1, 392.9]), compared to distant locations (M = 268.5 ms, 95%CI = [207.3, 329.6]). For AUC, trajectories converged on the “Yes” response more directly, i.e., smaller AUC, with distant (M = .70, 95% CI = [.56, .85]) compared to proximal locations (M = .86, 95% CI = [0.69, 1.02]).
The AUC data also showed a congruency effect, F(1, 43) = 45.81, p < .001, MSE = .49, BF01 < .001. As Figure 2a illustrates, mouse movements gravitated towards the wrong response, leading to a larger AUC when response direction was incongruent (M = 1.13, 95% CI = [0.93, 1.34), with the spatial relationship, compared to the congruent condition (M = .43, 95% CI = [.30, .57]).
(Exp.1) (a) Main effect of congruency on real-time mouse trajectories in correct description trials. Mouse trajectories for incongruent trials gravitated in the direction defined by the spatial relationship. Note that mouse trajectories for participants with “YES” and “NO” buttons in opposite positions have been collapsed for ease of illustration. For ease in interpreting the graphs, the x-coordinates ranged from -1 to 1 and the y-coordinate ranged from 0 to 1.5 (with x- and y-axes intersecting at “-1, 0”). (b) Interaction between congruency and time bin in proportional Euclidean proximity (PEP) for correct description trials. PEP is plotted as a function of normalised time. Mouse trajectories for incongruent trials show a greater attraction to the opposite response than do congruent trials, peaking during time bin 2.
The analysis of the PEP data elucidated the congruency effect, showing an interaction between congruency and time bin, F(4, 172) = 13.15, p < .001, MSE = .004, BF01 < .001. Figure 2b shows that mouse trajectories on incongruent trials moved in the opposite direction to a greater extent throughout the response (time Bins 1–4) compared to movements during congruent trials (see comparisons in Table 2).
Comparisons of PEP between the congruent and incongruent condition.
T: time bin; MANOVA: multi-variate analysis of variance.
The tables show results of repeated measures MANOVA of PEP data for comparing the congruent to incongruent condition.
Significant at α < .05; **Significant at α < .01.
The PEP data also showed a four-way interaction between proximity, congruency, facing direction and time bin, F(4, 172) = 4.55, p = .002, MSE = .003, BF10 = 1.382. To illustrate this complex interaction, PEP data are plotted as difference scores (PEP of incongruent trials—PEP of congruent trials) at each time-step. Positive difference scores reflect greater mouse movements towards the wrong response on spatially incongruent compared to spatially congruent trials; negative scores mean greater mouse movement towards the wrong response on spatially congruent trials compared to incongruent trials. These scores would indicate the source of ACE, with negative scores reflecting semantic congruency. As Figure 3a illustrates, with proximal locations, a congruency × facing direction × time bin interaction was found, F(4, 172) = 3.11, p = .017. Participants imagining facing north (learned orientation) showed an early compatibility effect, F(4, 180) = 6.73, p < .001, that then largely disappeared. When imagining facing east (novel orientation), the compatibility effect occurred later and was weaker, F(4, 180) = 2.55, p = .041, (see comparisons in Table 3). For distant locations, this three-way interaction was not evident, F(4, 172) = .80, p = .529. Instead, results showed congruency × time bin interaction in both facing-east, F(4, 180) = 5.59, p < .001, and north, F(4, 180) = 36.29, p < .001, conditions, suggesting that compatibility effects emerged with similar time-courses regardless of the facing direction.
Difference scores between incongruent and congruent conditions in PEP plotted as a function of normalised time. (a) (Exp.1) The figures compare the degree to which the mouse trajectories were attracted towards opposite response with proximal or distant locations when imagining facing east and north in correct descriptions. (b) (Exp.2) The figures compare the degree to which the mouse trajectories were attracted towards opposite response in the high-familiarity group to that in the low-familiarity group in correct descriptions. (c) (Exp.4) The figures compare the degree to which the mouse trajectories were attracted towards opposite response when oriented east and north in correct descriptions. (d) (Exp.3) The figures compare the degree to which the mouse trajectories were attracted towards opposite response with proximal or distant locations when imagining facing east and north in correct descriptions. (e) (Exp.3) The figures compare the degree to which the mouse trajectories of the low- and high-familiarity groups were attracted towards opposite response with proximal or distant locations in correct descriptions.
Comparisons of PEP between the congruent and incongruent condition for interactions.
MANOVA: multivariate analysis of variance; T: time bin.
The tables show results of repeated measures MANOVA of PEP data for comparing the congruent to incongruent condition.
Significant at α < .05; **Significant at α < .01.
Incorrect descriptions
The multivariate analysis of incorrect description trials showed a consistent proximity effect, F(4, 40) = 15.83, p < .001, across the dependent measures.
Accuracy & RT. The proximity effect emerged for accuracy, F(1, 43) = 23.09, p < .001, MSE = .01, BF01 < .001, and RT, F(1, 43) = 21.51, p < .001, MSE = 68,026.5, BF01 < .001. As with correct descriptions, participants judged proximal locations (M = .91, 95% CI = [.89, .93]) less accurately than distant ones (M = .97, 95% CI = [.95, .98]) and took more time to judge proximal (M = 1618.0 ms, 95% CI = [1507.4, 1728.6]) than distant ones (M = 1441.4 ms, 95%CI = [1340.5, 1541.9]). The effect was qualified by a proximity and familiarity interaction for RT, F(1, 43) = 4.80, p = .034, BF01 = .439, like the correct description results. Low-familiarity participants took longer to judge proximal, compared to distant locations, d =Mproximal – Mdistant = 260.0 ms, F(1, 45) = 24.91, p < .001, while this difference was not evident for high-familiarity participants, d = Mproximal – Mdistant = 93.1 ms, F(1, 45) = 3.10, p = .085.
Mouse Trajectory Results. Mouse trajectories showed the proximity effect, F(1, 43) = 8.51, p = .006, MSE = 22,719.1, BF01 < .09, and proximity × familiarity interaction, F(1, 43) = 4.72, p = .035, BF01 = .047, with initial time. Consistently with results of RT, participants took longer to initially move when judging proximal locations, especially low-familiarity participants, dlow = Mproximal – Mdistant = 122.0 ms, F(1, 45) = 13.83, p = .001; dhigh = Mproximal –Mdistant = 16.4 ms, F(1, 45) = .29, p = .592.
No other effects with mouse trajectory measures were found in the analysis.
Discussion
Congruency effect
Experiment 1 explored perceptuo-motor associations evident when egocentrically retrieving map knowledge. Our encoding hypothesis predicts that perceptuo-motor associations incorporated during learning will be reactivated at retrieval. Because map learning involves some perceptual, but no explicit motor processes, we would expect either no or a limited ACE. The retrieval hypothesis suggests that perceptuo-motor associations will be activated based on the retrieval requirements and goals, regardless of learning conditions. Egocentric terms used to describe spatial relations should prime body and/or action associations (Brunyé et al., 2009; Brunye, Mahoney, & Taylor, 2010). The mouse trajectory data indicated an ACE when attempting to solve egocentric problems after studying a map, supporting the retrieval hypothesis. However, the story is more complex when taking other variables into consideration.
Impact of location proximity
The influence of location proximity on responses suggests a greater role of encoding earlier in spatial representation development. Our results demonstrate a reliable proximity effect in the form of a symbolic distance effect, rather than one indicative of mental simulation. Participants showed an advantage in thinking about distant spatial relations, by performing better (more accurate, faster, smaller AUC) with distant than proximal locations (Brunyé et al., 2012; Brunye, Mahoney, & Taylor, 2010; Denis, 2008; Noordzij & Postma, 2005). Alternatively, this finding could reflect categorical processing, emerging from strategic encoding (Friedman & Montello, 2006; Hommel, Gehrke, & Knuf, 2000; Huttenlocher, Hedges, & Duncan, 1991; McNamara, Halpin, & Hardy, 1992; Wang, Taylor, Brunyé, & Maddox, 2014). Participants informally reported a map memorization strategy that involved dividing the map into clusters and associating landmarks within each cluster. This strategy groups landmarks into spatial categories; people make faster judgements for landmarks located in different, compared to the same, spatial clusters (Huttenlocher et al., 1991; Wang et al., 2014).
Interestingly, proximity interacted with familiarity. Low-familiarity participants showed the symbolic distance effect, while high-familiarity participants did not. This suggests that distance information impacts map representations as they develop, but less so once they are solidified. The present proximity effect on developing map knowledge is opposite of that we observed with navigation learning (Wang et al., 2012) wherein travellers with less-developed spatial knowledge responded faster and more accurately when judging spatial relations between proximal landmarks, while those with well-developed spatial knowledge did not show the proximity effect. This distinction confirms different mechanisms or strategies underlying development of spatial knowledge acquired from map or navigation learning. Nevertheless, as the spatial representation becoming steady and sound, the proximity effect disappears.
Impact of map familiarity
Map familiarity affected how participants used location proximity information, but did not affect ACEs more generally. Unlike learning from navigation, learning from a survey perspective allows greater representational flexibility earlier. Brunyé et al.’s (2008) work on spatial descriptions showed that learning from a route description required more time to gain the flexibility to change perspectives (verify from an unlearned perspective) than learning from a survey description. In the same vein, learning from maps more readily allows perspective switching compared to navigation learning (Thorndyke & Hayes-Roth, 1982). With map representations relatively well formed early on, perceptuo-motor associations based on egocentric retrieval would be evident, regardless of familiarity (Brunyé et al., 2009; Brunye, Mahoney, & Taylor, 2010). This point is supported in the data.
Impact of reorientation at retrieval
The fact that people keep maps stable while studying allows them to associate map coordinates (on the page or screen) to body coordinates (e.g., “the Eiffel Tower is to the left of the Museé de Louvre when looking at the Paris map”). This congruence between map and body coordinates has been shown to aid both learning and retrieval of new spatial knowledge (Gagnon et al., 2014). These encoding-based associations may lead to perceptuo-motor effects, akin to the spatial Stroop, which could be misinterpreted as retrieval effects. To distinguish between these as encoding or retrieval effects, we included mental reorientation at retrieval. If map knowledge is represented in an image-like pattern (Kosslyn, 1994; Sholl, 1995; Thorndyke & Hayes-Roth, 1982) and people relate map locations using body-based coordinates at encoding, then reorientation should break this association. The time course of the PEP data addresses this issue. ACEs emerged with both the studied and novel orientation, but the temporal dynamics of the effect differed. With proximal locations, we observed a strong and early ACE when imaging facing north (studied orientation), but a weaker and later ACE when imaging facing east (novel orientation). Thus, both orientations engaged perceptuo-motor processing. With the learned orientation, the effect could be related to encoding or retrieval, but the novel orientation findings suggest a retrieval-based influence. Whether encoding or retrieval-based, the finding lends support to the notion that perceptuo-motor associations will be activated when demanded of a retrieval task, and vary as a function of relations between learning and testing experiences (Hommel, 2007, 2011).
Summary
To summarise, Experiment1 suggested perceptuo-motor involvement when egocentrically retrieving map knowledge. This lends more support for the retrieval than the encoding hypothesis. The perceptuo-motor processes associated with map knowledge might be like the spatial Stroop effect, i.e., based on perceptual stimulus-response compatibility. If so, proximity and retrieval orientation may not influence use of perceptuo-motor processes, but would interact and vary their temporal dynamic and strength. Maps not only represent spatial information from a survey perspective, but also place it within cardinal coordinates. However, cardinal coordinates are more abstract and less likely to be associated with perceptual and motor processes, compared to the egocentric coordinates. To further explore the role of retrieval, Experiment 2 explores map knowledge retrieval using cardinal coordinates.
Experiment 2: action compatibility in cardinal coordinate retrieval of map knowledge
To further explore the retrieval hypothesis, Experiment 2 asked participants to retrieve map memory using cardinal terms. As addressed earlier, the encoding hypothesis would predict no or limited perceptuo-motor compatibility from map learning. In the case of cardinal terms, which describe spatial information based on an abstract reference frame and are less connected with perceptual or motor experiences (Levinson, 1996; Majid et al., 2004; Taylor & Tversky, 1992), the retrieval hypothesis would also predict limited ACEs. Since we observed reliable ACEs with egocentric retrieval in Experiment 1, to further support the retrieval hypothesis, we should be able to dissociate this result with cardinal retrieval.
If retrieving spatial representations from cardinal coordinate also shows ACEs, then it may involve other manipulations of spatial representations while completing the task. An exception to the above predictions may arise if map learners engage in egocentric recoding, i.e., associating cardinal directions to body coordinates (Tower-Richardi et al., 2012). For example, the standard north-up compass rose points “west” leftward and learners may associate “west” with the map’s left side. Later, the west is left association may be retrieved, leading to a spatial Stroop-like effect (De Houwer, 2003; Hommel, 2011). Egocentric recoding may elicit an ACE when retrieving map knowledge based on cardinal coordinates. However, this ACE should be weaker and/or have a later time course compared to egocentric retrieval, due to the additional level of abstraction.
To explore the possible effects of egocentric recoding, we manipulated the compass orientation during study. One compass rose had a typical “north-up” orientation and the other pointed “north” to the right, breaking the conventional “left-west” association. Map knowledge learned with atypical compass rose does not have the default cardinal-to-egocentric associations (e.g., “west” is left). If participants create cardinal-egocentric mappings with the atypical compass, then we may attribute ACEs based on egocentric recoding. If not, ACEs with only the typical compass would suggest associations based on mapping conventions.
Method
Participants
Forty-nine college-age volunteers (18 male, 31 female) participated for monetary compensation. Participants were tested individually. One female participant could not complete the test due to a technical error.
Materials
The campus maps used in Experiment 1 were used in this experiment. We created two versions of each by including one of two compass roses, either north-up or north-right. The study used Experiment 1’s labelling task as a criterion for learning. The spatial verification task was modified to use cardinal directional terms (East, West, North, or South). Again, location pairs were either proximal or distant.
Design
The study used a 2 (Description Correctness: correct vs. incorrect) ×2 (Proximity: proximal vs. distant locations) × 2 (Congruency: congruent vs. incongruent) ×2 (Familiarity: low-familiarity vs. high-familiarity) ×2 (Compass Orientation: north-up vs. north-right) mixed-design. Compass orientation and familiarity served as between-participant variables while description correctness, congruency and proximity, served as within-participant variables. Except compass orientation, all variables shared the same definitions as Experiment 1. Compass orientation described the compass rose orientation during learning, north-up or north-right, with half of participants studying each. As in our previous studies, we included non-competition trials describing spatial locations unrelated to leftward or rightward mouse movement (“east” and “west” in north-up map, “north” and “south” in north-right map). These trials were not included in any analyses. Control variables were manipulated as in Experiment 1.
Procedure
The overall procedure and data collection matched Experiment 1. The experimenter pointed out the compass orientation before studying. Low- and high-familiarity criteria matched Experiment 1.
Prior to the spatial verification task, participants were informed that they would verify the spatial relations based on cardinal coordinates and then completed 6 practice trials to familiarise themselves with the procedure. The task involved 96 trials, presented in random order, including 10 trials in each the 8 conditions defined by the three within-participants variables (description correctness, congruency and proximity) plus 16 non-competition filler trials.
Result
Four low-familiarity participants and three high-familiarity participants were eliminated from analyses due to low accuracy (less than .55). For the data analysis of between-participant conditions, there were 10 participants in low-familiarity & north-up condition, 10 participants in low-familiarity & north-right condition, 12 participants in high-familiarity & north-up condition, and 9 participants in high-familiarity & north-right condition. A t-test indicated that the hit (M = .79, SD = .15) and correct rejection rates (M = .80, SD = .15) were greater than our elimination criteria, .55; t(40) = 10.10, p < .001; t(40) = 11.17, p < .001. High-familiarity participants took significantly more learning cycles (M = 4.90, SD = .77) than low-familiarity participants (M = 3.35, SD = 1.04), t(39) = 5.46, p < .001. A preliminary analysis showed no effect of map version, response button position, or gender. Thus, data were collapsed across these control variables and in the case of button position, reverse scored for one button position.
Correct and incorrect description trials were analysed separately as in Experiment 1. The data were submitted into a 2 (Congruency: congruent vs. incongruent) × 2 (Proximity: proximal vs. distant location pairs) × 2 (Compass orientation: north-up vs. north-right) × 2 (Familiarity: low-familiarity vs. high-familiarity) multivariate ANOVA with dependent measures of accuracy, RT, initial time and AUC. RT and mouse trajectory analyses used only accurate responses. PEP analyses also included time bins as a within-participant variable.
Correct descriptions
The multivariate results showed an effect of proximity, F(4, 34) = 4.77, p = .004, and congruency, F(4, 34) = 7.02, p < .001. Congruency interacted with familiarity, F(4, 34) = 3.83, p = .011. The univariate analyses further explained these effects.
Accuracy & RT. As in Experiment 1, results showed an effect of proximity for accuracy, F(1, 37) = 6.18, p = .018, MSE = .02, BF01 = .161, and RT, F(1, 37) = 13.48, p = .001, MSE = 100,421.6, BF < .001. Participants made more errors judging proximal (M = .76, 95% CI = [.71, .81]) compared to distant spatial relationships (M = .81, 95% CI = [.76, .85]) and also took more time judging proximal (M = 2066. 8 ms, 95% CI = [1879.3, 2254.3]) than distant locations (M = 1884.1 ms, 95% CI = [1719.9, 2048.3]). In addition, participants with the north-up compass (M = .84, 95% CI = [.78, .90]) showed a higher accuracy than those with north-right compass (M = .72, 95% CI = [.65, .78]), F(1, 37) = 8.01, p = .07, MSE = .08, BF01 = .131
Mouse Trajectory Results. AUC data showed a proximity effect, F(1, 37) = 7.03, p = .012, MSE = .30, BF01 = .781, AUC when judging distant (M = .84, 95% CI = [0.66, 1.03]) compared to proximal locations (M = 1.07, 95% CI = [0.88, 1.27]). The analysis also showed an effect of congruency, F(1, 37) = 16.95, p < .001, MSE = .558, BF01 < .001. Mouse trajectories for incongruent trials had a larger AUC (M = 1.21, 95% CI = [0.97, 1.44]), compared to those for congruent trials (M = .71, 95% CI = [.54, .88]). The congruency main effect was qualified by an interaction with familiarity, F(1, 37) = 9.02, p = .005, BF01 = .025. As seen in Figure 4, low-familiarity participants showed a substantial congruency effect, d = Mincongruent – Mcongruent = .86, F(1, 39) = 23.85, p < .001, while high-familiarity participants showed little difference based on congruency, d = Mincongruent – Mcongruent = .13, F(1, 39) = 1.12, p = .296.
(Exp.2) Interaction between familiarity and congruency in AUC (a) of correct descriptions. The mouse trajectories were plotted as real-time figures; (b) For high-familiarity participants, mouse trajectories for congruent and incongruent trials did not differ; (c) For low-familiarity participants, mouse trajectories on incongruent trials showed a reliable attraction to the “No” response button compared to congruent trials.
A three-way interaction between congruency, familiarity and time bin was found in PEP data, F(4, 148) = 3.70, p = .007, MSE = .006, BF01 = 1.081. To interpret the interaction, we plotted PEP difference scores by subtracting the PEP of congruent trials from that of incongruent ones, as in Experiment 1. As a reminder, positive scores reflect an ACE. Figure 3b illustrates that low-familiarity participants showed an ACE throughout the response, F(4, 156) = 5.33, p < .001 while high-familiarity participants’ difference scores remained around zero throughout, F(4, 156) = .07, p = .991 (see Table 3 for comparison results).
Incorrect descriptions
Multivariate analyses on incorrect descriptions also revealed effects of proximity, F(4, 34) = 4.75, p = .004, and congruency, F(4, 34) = 2.79, p = .042. The univariate results further elaborate on these effects.
Accuracy & RT. Incorrect description trials showed a proximity effect for both accuracy, F(1, 37) = 5.36, p = .026, MSE = .018, BF01 = .991, and RT, F(1, 37) = 7.71, p = .009, MSE = 137,145.3, BF01 = .017. Consistent with Experiment 1, participants made more errors when judging proximal (M = .78, 95% CI = [.73, .83]) compared to distant locations (M = .83, 95%CI = [.78, .88]) and needed more time to judge proximal (M = 2121.2 ms, 95% CI = [1954.8, 2287.7]) compared to distant locations (M = 1959.7 ms, 95% CI = [1782.9, 2136.6]).
Mouse Trajectory Results. Initial mouse movement time also showed a proximity effect, F(1, 37) = 4.59, p = .039, MSE = 51,843.3, BF01 = .614. Participants took more time before initially moving the mouse when processing proximal (M = 489.1 ms, 95% CI = [357.1, 621.1]) than distant locations (M = 412.4 ms, 95% CI = [289.0, 535.8]).
The AUC analysis showed a congruency effect, F(1, 37) = 4.71, p = .036, MSE = .615, BF01 = .256. Mouse trajectories in incongruent trials moved closer towards the wrong, but spatially congruent, response (M = 1.23, 95% CI = [0.93, 1.54]), compared with those in congruent trials (M = .97, 95% CI = [0.73, 1.20]).
Discussion
Congruency effect
In Experiment 2, participants studied maps and then verified relative locations defined by cardinal coordinates. Both encoding and retrieval hypothesis predicted absent or limited perceptuo-motor reactivation, as little explicit perceptuo-motor processing might be involved in either map learning or retrieval with the abstract, cardinal reference frame. As an exception to this, egocentric recoding of cardinal directions may drive a spatial Stroop-like effect, especially with the conventional North-up compass. The results partially support the retrieval hypothesis and egocentric recoding.
While high-familiarity participants did not show the ACE in any cases, supporting the retrieval hypothesis, low-familiarity participants showed a spatially based ACE. When responses required a motion incompatible with the spatial relation between locations, mouse trajectories moved in the direction defined by the spatial relation (see, Figure 4). Unlike Experiment 1 (egocentric retrieval) wherein familiarity did not impact ACEs, the above result suggests egocentric recoding, mapping cardinal directions to one’s egocentric reference frame, when map knowledge is weak. These associations are evident when retrieving map knowledge, although the effect appears later than those with egocentric retrieval. The ACEs’ temporal dynamic showed that the deviation of mouse trajectories peaked midway through the response (time bin 3) with cardinal terms, later than the peak with egocentric retrieval (time bin 2) in Experiment 1. Egocentric recoding may represent a strategy for cardinal retrieval as map information is learned. As the spatial representation develops more fully, people may not have to associate cardinal coordinates to their egocentric reference frame. Supporting this, high-familiarity participants did not show ACEs. Furthermore, for low-familiarity participants, egocentric recoding does not seem dependent on mapping conventions. The atypical compass breaks the conventional “left-west” associations, yet evidence of perceptuo-motor associations did not interact with compass orientation.
Impact of location proximity
As in Experiment 1, proximity between locations yielded a symbolic distance effect. Participants had higher accuracy, shorter RT and initial times, and smaller AUCs with distant compared to proximal locations.
Proximity did not interact with congruency, which indicates proximity processing would not impact ACEs with map knowledge. As discussed earlier, map as a symbolic representation could provide proximal and distant spatial information simultaneously. Accordingly, perceptuo-motor associations based on egocentric recoding may build up for proximal and distant landmarks with similar processes.
Summary
Again, results lent support to the retrieval hypothesis, with influences from other factors, notably map familiarity. Retrieving well-developed map knowledge based on cardinal reference frame did not drive ACEs, consistent with the retrieval hypothesis. With less-developed map knowledge, participants showed reliable, spatially based ACEs, consistent with egocentric recoding and the spatial Stroop. This suggests that people may strategically connect abstract spatial concepts to their egocentric reference frame during learning and before map knowledge can be used flexibly. Proximity and learning orientation did not influence effects of perceptuo-motor processes, although proximity affected responses overall, as in Experiment 1.
Experiments 1 and 2 specifically explored differences in perceptuo-motor associations as a function of retrieval demands. Usually, people learn and use maps for many reasons that may involve both egocentric and cardinal systems. When the retrieval coordinates cannot be predicted, would reactivation of perceptuo-motor associations change? Experiment 3 explored retrieving map knowledge with either egocentric or cardinal coordinates, but participants could not predict which.
Experiment 3: action compatibility in mixed-coordinate retrieval of map knowledge
In Experiments 1 and 2, participants learned a map and then verified spatial relations from a single coordinate system (egocentric for Experiment 1 and cardinal for Experiment 2). By knowing the retrieval demand, participants could adopt retrieval strategies. For instance, high-familiarity participants showed a significant ACE when retrieving egocentrically, but not when verifying canonically defined relations. While this supports our retrieval hypothesis, it could alternatively reflect strategic processing. Low-familiarity participants demonstrated ACEs when retrieving map knowledge from both egocentric and cardinal coordinates, suggesting egocentric recoding for cardinal retrieval, a potentially strategic process.
Two questions should be asked: (1) For well-developed map knowledge, does the distinction between egocentric and cardinal retrieval reflect different retrieval processes or strategic processing? and (2) For less-developed map knowledge, does egocentric recoding for cardinal coordinates lead to perceptuo-motor associations similar to egocentric retrieval?
Experiment 3 examined ACEs in map knowledge when retrieval coordinates could not be predicted. To do so, we combined the ACE paradigms used in Experiments 1 and 2. The encoding hypothesis would still predict no or weak ACEs with map knowledge. The retrieval hypothesis would predict ACEs consistent with the retrieval coordinates. The retrieval hypothesis, however, may also need to take into account the extent of environment familiarity and the potential for strategic retrieval. Our predictions take these factors into account. For high-familiarity, if participants show ACEs with both egocentric and cardinal retrieval, this would suggest that with well-developed map knowledge different retrieval strategies could be flexibly employed. If, however, high-familiarity participants show performance differences with egocentric and cardinal coordinates (Experiments 1 and 2, respectively), this would suggest different retrieval processes, rather than strategies, underlying egocentric and cardinal retrieval. For low-familiarity, egocentric recoding may play a role (Experiment 2). If low-familiarity participants again show ACEs for both types of retrieval coordinates, this would suggest egocentric recoding and also different mechanisms for less-developed map knowledge. To fully examine re-coding, Experiment 3, like Experiment 1, included different retrieval orientations. Participants learned maps with a typical compass rose (north-up) and then retrieved from either the studied or an unstudied orientation. Results of Experiment 1 suggested that reorientation affected the temporal dynamics, but not the occurrence of ACEs with less-developed map knowledge.
Method
Participants
Fifty college-age volunteers (26 male, 24 female) participated for monetary compensation. Participants were tested individually. Two participants (one male and one female), who reported not being able to remember the map after extensive study, were eliminated from analyses.
Materials
The maps used in Experiment 1 were again used here. The study again used the labelling and spatial verification tasks. New to this study, the spatial verification task mixed trials defining spatial relationship egocentrically (left, right, back, forward) and using cardinal direction terms (East, West, North, South).
Design
The study used a 2 (Description correctness: correct vs. incorrect) × 2 (Congruency: congruent vs. incongruent) ×2 (Proximity: proximal vs. distant locations) × 2 (Directional term: egocentric vs. cardinal terms) ×2 (Facing direction: imagining facing-north vs. facing-east) × 2 (Familiarity: low-familiarity vs. high-familiarity) mixed-design. Description correctness, congruency, proximity, and directional term, served as within-participant variables and familiarity and facing direction served as between-participant variables. Except directional term, these variables shared definitions with Experiment 1. The directional term, designed as a within-participant variable, reflected whether the described relationship between location pairs used egocentric or cardinal terms. The non-competition filler (egocentric and cardinal) terms were again used, but not analysed. Control variables were manipulated as before.
Procedure
The overall procedure and data collection matched Experiments 1 and 2. Prior to the spatial verification test, participants completed 6 practice trials (3 egocentric and 3 cardinal). The experiment involved 96 trials, presented in random order, that included 5 trials each in the 16 conditions defined by the 4 within-participants variables (description correctness, congruency, proximity and direction term) plus 16 filler trials.
Result
Four low-familiarity participants were eliminated from analyses due to low accuracy (less than .55). For the data analysis of between-participant conditions, there were 10 participants in low-familiarity & facing-north condition, 10 participants in low-familiarity & facing-east condition, 12 participants in high-familiarity & facing-north condition, and 12 participants in high-familiarity & facing-east condition. A t-test indicated that the hit rate (M = .84, SD = .09) and correct rejection rate (M = .86, SD = .10) was greater than our elimination criteria, .55; t(43) = 21.49, p < .001; t(43) = 20.81, p < .001. High-familiarity participants took significantly more times (M = 5.42, SD = 1.06) of learning round than low-familiarity participants did (M = 3.45, SD = .89), t(42) = 6.59, p < .001. A preliminary analysis showed no effect of map version, response button position and gender. Thus data were collapsed across these control variables, and in the case of button position, reverse scored for one button position. Responses for correct and incorrect descriptions were analysed separately as before. The data were submitted into a 2 (Proximity: proximal vs. distant location pairs) × 2 (Congruency: congruent vs. incongruent) × 2 (Directional term: egocentric vs. cardinal term) × 2 (Facing direction: imaging facing-north vs. imaging facing-east in task) × 2 (Familiarity: low- vs. high-familiarity) multivariate ANOVA to examine effects on accuracy, RT, initial time and AUC. For RT and mouse trajectory variables, only accurate responses were submitted into analyses. Time bins was included as an additional within-participant variable in PEP analyses.
Correct descriptions
Multivariate analyses showed effects of proximity, F(4, 36) = 22.86, p < .001, and congruency, F(4, 36) = 13.66, p < .001. The results also showed effects of familiarity, F(4, 36) = 3.46, p = .017, and facing direction, F(4, 36) = 4.39, p = .005, as well as a familiarity × facing direction interaction, F(4, 36) = 3.03, p = .031. Congruency interacted with familiarity, F(4, 36) = 2.89, p = .036. The results also included a directional term × facing direction interaction, F(4, 36) = 2.89, p = .036.
Accuracy & RT. Analyses showed an effect of proximity for accuracy, F(1, 39) = 51.79, p < .001, MSE = .021, BF01 < .001, and RT, F(1, 39) = 29.55, p < .001, MSE = 251,967.1, BF01 < .001. As before, participants made more errors judging proximal (M = .79, 95% CI = [.75, .82]) than distant locations (M = .90, 95% CI = [.88, .93]) and also judged proximal locations (M = 2081.2 ms, 95% CI = [1904.7, 2257.7]) more slowly than distant locations (M = 1786.1 ms, 95% CI = [1656.3, 1915.9]).
Accuracy data also showed an effect of facing direction, F(1, 39) = 4.48, p = .041, MSE = .050, BF01 = 1.037. Participants made fewer errors when imagining facing north (M = .87, 95% CI = [.84, .91]) than imagining facing east (M = .81, 95% CI = [.78, .85]). The accuracy effect was qualified by an interaction between facing direction and directional term, F(1, 39) = 4.68, p = .037, MSE = .052, BF01 = .038, demonstrating that directional term did not affect response accuracy when imagining facing north, d = Mcardinal – Megocentric = -.04, F(1, 42) = 1.05, p = .310, but participants responded more accurately with cardinal than egocentric terms when imagining facing east, d = Mcardinal – Megocentric = .09, F(1, 42) = 5.84, p = .020. The RT data supported the facing direction × directional term interaction, F(1, 39) = 9.97, p = .003, MSE = 333,362.3, BF01 = .012. When imaging facing east, participants responded faster with cardinal than egocentric terms, d = Mcardinal – Megocentric = –198.7 ms, F(1, 41) = 5.04, p = .030, while the opposite was true when imagining facing north, d = Mcardinal – Megocentric = 195.5 ms, F(1, 41) = 4.38, p = .043.
Mouse Trajectory Results. As seen previously, initial time for correct descriptions showed a main effect of proximity. Participants took longer before initially moving the mouse when processing proximal (M = 411.7 ms, 95% CI = [317.8, 505.6]) versus distant locations (M = 324.8 ms, 95% CI = [260.7, 388.9]), F(1, 39) = 12. 17, p = .001, MSE = 53,056.9, BF01 = .084. Initial time data also showed a facing direction × direction term interaction, F(1, 39) = 5.10, p = .030, MSE = 72,086. 2, BF01 = .396. While the pair-wise comparisons did not reach significance, the pattern matched the RT data.
The initial time results showed a facing direction × familiarity interaction F(1, 39) = 6.91, p = .012, MSE = 487,376.3, BF01 = .228. High-familiarity participants took more time to initially move when imagining facing east, d = Mfacing-east – Mfacing-north = 260.1 ms, F(2, 39) = 29.15, p < .001. In contrast, low-familiarity participants took more time before initially moving when imagining facing north, d = Mfacing-east – Mfacing-north = −136.8 ms, F(2, 39) = 22.04, p < .001.
AUC results showed an effect of facing direction, F(1, 39) = 16.01, p < .001, MSE = 2.431, BF01 < .001, with a larger AUC when imagining facing east (M = 1.33, 95% CI = [1.08, 1.57]) compared to facing north (M = .65, 95%CI = [.41, .89]). AUC also showed a familiarity effect, F(1, 39) = 11.89, p = .001, MSE = 2. 431, BF01 = .126, low-familiarity participants had a larger AUC (M = 1.28, 95% CI = [1.03, 1.53]) than did high-familiarity participants (M = .70, 95% CI = [.47, .93]). Notably, AUC data demonstrated a marked congruency effect, F(1, 39) = 57.61, p < .001, MSE = .605, BF01 < .001. Participants had a much larger AUC with incongruent (M = 1.31, 95% CI = [1.91, 1.52]) than congruent trials (M = .67, 95% CI = [.51, .83]). The effect was qualified by a congruency × familiarity interaction, F(1, 39) = 9.54, p = .004, BF01 = .073. Both high- and low-familiarity participants showed the congruency effect, but it was much larger for low-familiarity participants (dlow = Mincongruent – Mcongruent = .90, F(1, 41) = 52.39, p < .001; dhigh = Mincongruent – Mcongruent = .38, F(1, 41) = 10.68, p = .002.
The PEP data showed a congruency × time bin interaction, F(4, 156) = 4.99, p = .001, MSE = .008, BF01 = 1.002; the congruency effect peaked mid-way through the response (time bins 2 to 4; see Table 2 for comparison results).
An important four-way interaction emerged between proximity, congruency, facing direction and time bin, F(4, 156) = 3.19, p = .015, MSE = .010, BF01 = .1.033. Further analysis showed a three-way interaction between facing direction, congruency and time bin with proximal locations, F(4, 156) = 2.62, p = .037. As Figure 3d illustrates, a compatibility effect appeared later and peaked at time bin 4 when processing proximal relations while imagining facing east. While the pattern was similar when imagining facing north, it did not reach significance (see comparisons in Table 3). The facing direction × congruency × time bin interaction was not significant for distant locations.
The PEP data also demonstrated a four-way interaction between proximity, congruency, familiarity and time bin, F(4, 156) = 3.64, p = .007, MSE = .010, BF01 = 1.016. Figure 3e plots PEP difference scores at each time-step, calculated by subtracting the PEP of congruent trials from that of incongruent ones. Positive values indicate an ACE. Low-familiarity participants showed a proximity× congruency × time bin interaction, F(4, 164) = 4.46, p = .002. With proximal locations, the compatibility effect developed later in the response, peaking at time bin 4, compared to a peak between time bins 2 and 3 for distant locations. High-familiarity participants did not show these effects (see Table 3 for comparison results).
In addition, the PEP analyses showed a directional terms × facing direction × time bin interaction, F(4, 156) = 3.55, p = .008, MSE = .007, BF01 = 1.019. When participants imagined facing east, mouse trajectories with egocentric terms peaked later than those with cardinal terms, F(4, 164) = 3.64, p = .007 (see comparisons in Table 4). As the Figure 5 illustrates, when imagining facing north, mouse trajectory time course did not differ based on directional term type, F(4, 164) = 1.01, p = .403.
Comparisons of PEP between the cardinal and egocentric condition for facing directions in correct descriptions (Exp.3).
MANOVA: multivariate analysis of variance; T: time bin.
The table shows results of repeated measures MANOVA of PEP data for comparing the congruent to incongruent condition.
Significant at α < .05; **Significant at α < .01.
(Exp.3) Difference scores between trials with cardinal and egocentric terms (cardinal- egocentric) in PEP to the opposite response are plotted as a function of normalised time. The figures compare the degree to which the mouse trajectories were attracted towards opposite response when imagining facing east and north in correct descriptions.
Incorrect descriptions
Multivariate analyses showed an effect of proximity, F(4, 37) = 9.34, p < .001, and a directional term × facing direction interaction, F(4, 37) = 4.43, p = .005. The results also demonstrated a four-way interaction between proximity, congruency, familiarity and facing direction, F(4, 37) = 5.13, p = .002. The univariate results further explored these effects.
Accuracy & RT. As with correct descriptions, participants judged proximal locations (M = .82, 95% CI = [.79, .86]) less accurately than distant ones (M = .90, 95% CI = [.87, .93]), F(1, 40) = 26.15, p < .001, MSE = .019, BF01 < .001, and took more time processing proximal (M = 2146.3 ms, 95% CI = [1993.2, 2299.3]) than distant locations (M = 1971.6 ms, 95% CI = [1801.8, 2141.5]), F(1, 40) = 9.22, p = .004, MSE = 288,818.6, BF01 = .042.
The accuracy results also showed the directional term × facing direction interaction, F(1, 40) = 4.15, p = .048, MSE = .026, BF01 = .668. As Figure 6a illustrates, participants imagining facing east responded more accurately with cardinal than egocentric terms, d = Mcardinal – Megocentric = .06, F(1, 42) = 6.57, p = .014, while those who imagined facing north responded equally accurately to both directional term types, d = Mcardinal – Megocentric = .01, F(1, 42) = .23, p = .637. The RT analysis supported this interaction, F(1, 40) = 14.92, p < .001, MSE = 351,675.4, BF < .001. Processing egocentric terms took longer than cardinal terms when imaging facing east, d = Mcardinal – Megocentric = –234.3 ms, F(1, 42) = 7.40, p = .009, while participants imagining facing north had the opposite tendency, d = Mcardinal – Megocentric = 256.1 ms, F(1, 42) = 7.64, p = .008, see Figure 6b.
(Exp.3) Interaction between directional term and facing direction in (a) accuracy, (b) RT, and (c) initial time of incorrect descriptions.
Mouse Trajectory Results. Initial time showed an interaction between directional terms and facing direction, F(1, 40) = 4.67, p = .037, MSE = 112,849.2, BF01 = .753. As with correct descriptions, the initial time was longer with egocentric than cardinal terms when imagining facing east, d = Mcardinal – Megocentric = –94.8 ms, F(1, 42) = 3.99, p = .052. Participants imagining facing north did not show this difference, d = Mcardinal – Megocentric = 60.5 ms, F(1, 42) = 1.29, p = .263 (see Figure 6c).
The initial time results also demonstrated a three-way congruency × proximity × familiarity interaction, F(1, 40) = 4.38, p = .036, MSE = 99,411.6, BF01 = .860. Low-familiarity participants showed a congruency ×proximity interaction, F(1, 42) = 7.45, p = .009. With proximal locations, low-familiarity participants took more initial time with spatially congruent trials (semantically incongruent), d = Mincongruent – Mcongruent = 168.2 ms, F(1, 42) = 4.54, p = .039, while the difference was not evident for distant locations, d = Mincongruent – Mcongruent = 106.5 ms, F(1, 42) = 2.00, p = .165. No such interaction was found for high-familiarity participants, F(1, 42) = 0.04, p = .837 (see Figure 7).
(Exp.3) Three-way interaction between proximity, congruency and familiarity in initial time of incorrect descriptions.
No effects were found with AUC and PEP data for the incorrect description trials.
Discussion
Experiment 3 examined whether retrieval expectations and/or strategies contributed to perceptuo-motor associations in map memory. By mixing egocentric and cardinal retrieval, participants could not adopt a coordinate-based retrieval strategy. The encoding hypothesis predicts no ACEs with map knowledge. The retrieval hypothesis predicts perceptuo-motor associations for egocentric, but not cardinal, retrieval. Consistent with this prediction and with Experiment 1, the results demonstrated ACEs for both high and low-familiarity participants during egocentric retrieval.
Familiarity affects congruency effect
The extent of familiarity with a map affected the retrieval of environmental knowledge. Low-familiarity participants consistently showed evidence of perceptuo-motor associations during retrieval (Experiments 1, 2, and 3), regardless of retrieval conditions. In contrast, high-familiarity participants could flexibly retrieve perceptuo-motor associations depending on retrieval conditions. Although previous research has suggested that people quickly gain representational flexibility when learning from a survey perspective, including maps (Thorndyke & Hayes-Roth, 1982) and survey descriptions (Brunyé et al., 2008), the present work suggests that map representations also change with increased experience.
Egocentric recoding, i.e., mapping cardinal to egocentric coordinates, may help scaffold map knowledge as it develops. Egocentric recoding would also lead to perceptuo-motor associations. Low-familiarity participants showed ACEs with both egocentric and cardinal retrieval, replicating Experiments 1 and 2, and removing retrieval expectations as an influencing factor in Experiment 3. The manipulation of unpredictable retrieval coordinates does not assume that participants would change their strategies of egocentric recoding. Participants are likely to do the recoding of cardinal terms for each trial of cardinal retrieval. Supporting this possibility, the results of initial time of incorrect description trials showed a semantic-based ACE with proximal locations for low-familiarity participants. This suggests that low-familiarity participants might be more affected by the semantic process before they moved the mouse. Further, the temporal analysis of mouse trajectories showed a strong but late ACE with proximal locations for low-familiarity participants, while high-familiarity participants did not show such an effect. It may take more time for low-familiarity participants to complete the recoding process and the embodied process might be more necessary for them to complete the task, compared to the high-familiarity group. When spatial knowledge has not been solidified, reliance on egocentrically defined spatial relations may be more important (Thorndyke & Hayes-Roth, 1983).
Otherwise, participants may develop alternate strategies that could help solve the problems more efficiently. High-familiarity participants showed a relatively weak ACE with cardinal retrieval, inconsistent with the retrieval hypothesis and with Experiment 2. Their adoption of a retrieval strategy explains this result. Egocentric recoding would facilitate spatial verification for trials shifting between using cardinal and egocentric terms, providing a bridge between the retrieval conditions. Participants with well-developed spatial knowledge could use it more flexibly and figure out quickly the equivalence between egocentric and cardinal terms. Accordingly, it would take less processing to recode the terms into equivalent meanings than to continually adjust the entire reference frame.
Impact of location proximity
Proximity between locations again affected responses, with an advantage for distant locations – the classic symbolic distance effect. In addition, with mixed coordinates, proximity interacted with congruence in ways different from Experiments 1 and 2. Previous research, including Experiment 1 and 2, suggests that perceptuo-motor associations based on map knowledge should be less affected by proximity (Lamberts et al., 1992; Nicoletti & Umiltà, 1989, 1994; Rizzolatti et al., 1994). However, this experiment found reliable ACEs, particularly with proximal locations. With unpredictable retrieval coordinates, mental processing is more complicated. Perceptuo-motor processes may have been evoked with proximal locations to reduce mental workload (Carlson & Kenny, 2006; Longo & Lourenco, 2007; Tversky, 2005; Wang et al., 2012).
Impact of reorientation at retrieval
People study maps in a single orientation. This affords cardinal-to-egocentric associations at encoding. As in Experiment 1, we again included mental reorientation at retrieval to distinguish between perceptuo-motor effects at encoding and retrieval. Further, orientation should differentially impact retrieval as a function of directional term. Cardinal coordinates are stable and do not change during reorientation; egocentric coordinates are directly tied to retrieval orientation necessitating a translation from the learned orientation if different. Experiment 3 consistently showed interactions between retrieval orientation and directional term. From the learned orientation, participants had slower responses for cardinal term verification, perhaps supporting egocentric associations. However, participants responded more accurately and faster with cardinal than egocentric terms when imagining a novel orientation.
Retrieval orientation also interacted with location proximity, impacting the temporal dynamics of perceptuo-motor associations. When retrieving proximal locations egocentrically (Experiment 1), participants had an early ACE when imagining the studied orientation (north), and a later and weaker ACE when imagining a novel orientation (east). Experiment 3 showed a similar pattern, a later ACE when verifying proximal locations from a novel orientation. This supports the suggestion that participants had to mentally rotate to the novel orientation to process egocentric relations. Unlike Experiment 1, the ACE was not evident with distant locations, which may be due to the increased mental load with mixed coordinates retrieval.
Summary
Perceptuo-motor associations in map knowledge when retrieval coordinates are unpredictable depend on how well the map has been learned. Well-developed map knowledge supported the retrieval hypothesis. With well-developed knowledge, egocentric-only retrieval led to ACEs, while cardinal-only retrieval did not. Surprisingly, with mixed coordinates, ACEs were evident with both egocentric and cardinal retrieval. This suggests cognitively flexible retrieval strategies when map knowledge is well developed. For less-developed knowledge, ACEs appeared regardless of retrieval coordinates, suggesting that less-developed knowledge may be more reliant on cardinal-to-egocentric associations.
Experiment 4: action compatibility in cardinal retrieval after navigation
Experiments 1 through 3 investigated perceptuo-motor associations in map knowledge, varying retrieval coordinates. Together with Wang et al. (2012), the involvement of perceptuo-motor associations in spatial processing appears contingent on retrieval demands, particularly when the spatial information has been learned well. The results, as the retrieval hypothesis proposes, suggest that egocentric retrieval induced greater ACEs compared to cardinal coordinate retrieval. Low-familiarity participants showed a somewhat different pattern, suggesting the involvement of egocentric recoding with map representations. They demonstrated ACEs with cardinal and egocentric retrieval. Perceptuo-motor associations could be activated during cardinal retrieval if map learners associate cardinal coordinates to their egocentric axes, even though map learning may not explicitly engage perceptuo-motor processes. To fully investigate the role and interaction between encoding and retrieval, participants in Experiment 4 verified cardinally defined spatial relations for an environment they learned primarily through navigation.
Spatial studies have documented that with sufficient navigation experience, people can access their spatial representation from alternate perspectives, for example, a “survey perspective” (Brunyé et al., 2008; Taylor et al., 1999; Taylor & Tversky, 1992; Thorndyke & Hayes-Roth, 1982). The direct experience during navigation involves referencing locations to the navigator’s own position, but the survey perspective delineates spatial information from a “bird’s-eye” view, usually based on an absolute, fixed reference frame (Levinson, 1996; Taylor & Tversky, 1992). Thus, because developing a survey representation through navigation requires abstraction, it is unclear the extent to which perceptuo-motor associations would be involved with cardinal coordinate retrieval.
As discussed previously, the encoding hypothesis would predict ACEs with navigation learning, regardless of retrieval demands. Navigation involves many perceptual and motor experiences. Furthermore, as perceptuo-motor associations are more likely formed between nearby locations during navigation (Carlson & Kenny, 2006; Longo & Lourenco, 2007; Parsons, 1994; Regier & Carlson, 2001; Tversky, 2005), ACEs should be more evident with proximal locations (Wang et al., 2012).
In contrast, the retrieval hypothesis predicts that perceptuo-motor associations are activated in response to retrieval demands. As survey representations encode spatial information from a “bird’s-eye” viewpoint, retrieving spatial knowledge from survey perspective might mimic the perception of spatial relations on a map (Brunyé et al., 2008; Taylor et al., 1999; Taylor & Tversky, 1992). As such, ACEs should not be evident or be weak when verifying cardinally defined spatial relations. Experiments 1 and 2 supported the retrieval hypothesis by showing ACEs in egocentric, but not cardinal retrieval, when map knowledge was well developed. With less developed knowledge, both egocentric and cardinal retrieval evoked perceptuo-motor associations. With map knowledge, proximity did not affect these ACEs.
Experiment 4 also examines the effect of retrieval orientation. Research is mixed regarding the orientation-specificity of spatial knowledge acquired through varied experiences. Some research suggests that spatial knowledge acquired by actively navigating through a large-scale environment is less orientation-specific than map knowledge (Sun, Chan, & Campos, 2004; Tlauka & Wilson, 1996). In other cases, spatial knowledge resulting from navigation can also be highly orientation-specific, for instance, showing reliance on principle reference vectors, such as the first path experienced during navigation or long straight roads through a large-scale environment (Brunyé, Burte, Houck, & Taylor, 2015; Marchette, Yerramsetti, Burns, & Shelton, 2011; McNamara, Rump, & Werner, 2003; Werner & Schmidt, 1999). Following the former research, retrieving from different orientations should have less of an influence than was seen in Experiments 1 and 3 (Sun et al., 2004; Tlauka & Wilson, 1996). Following the latter research, orientations experienced during retrieval should impact performance similarly to that seen in Experiments 1 and 3. Furthermore, if survey representation based on individual navigation experience is image-like and adopts a north-up orientation, alignment effects may arise (Gagnon et al., 2014). If so, then we would expect the similar pattern of ACE observed in Experiment 2.
To summarise, the present study examines perceptuo-motor involvement in spatial knowledge developed from navigation, but retrieved using the cardinal coordinate system. By comparing with egocentric retrieval after real navigation (Wang et al., 2012) and cardinal retrieval of map knowledge (Experiment 2), we further investigate the role of encoding and retrieval in activating perceptuo-motor associations in spatial knowledge.
Method
Participants
Forty-seven Tufts undergraduates (21 males, 26 females) participated in the study. Of 47 participants, 24 first-year students comprised the low-familiarity group and 23 seniors made up the high-familiarity group.
Stimuli and design
The stimuli consisted of names and pictures of 50 Tufts University buildings familiar to undergraduates. Building pictures depicted the front-door side of the building, but avoided showing adjacent locations. These pictures were used to confirm that students knew a building and its front door location. Test trials used building names. On a given trial, participants saw two building names and a spatial term relating them (e.g., Dowling Hall, Eaton Hall, south).
Cardinal directional terms (east, west, north and south) were used to describe spatial relations. The study used a 2 (Description correctness: correct vs. incorrect) ×2 (Congruency: congruent vs. incongruent) × 2 (Proximity: proximal vs. distant locations) × 2 (Compass Orientation: north-up vs. east-up) × 2 (Familiarity: low-familiarity vs. high-familiarity) mixed-design. Description correctness and proximity matched Experiment 1. Familiarity was operationalized by how long participants had been at Tufts, first-year student or senior. As perceptuo-motor associations may play a role in reorientation when retrieving spatial knowledge from survey perspective, compass orientation instructed participants to imagine looking at the campus map with a north- or east-up compass before the spatial verification task. The full explanation to participants used location examples aligned with the assigned orientation. This manipulation also attempted to match Experiment 2 wherein two types of compasses were used, a conventional north-up and an atypical east-up compass. Twenty-three participants completed the task facing-north and 24 completing it facing-east. Congruency, defined as in earlier studies, differed for the two facing directions. A rightward mouse movement would be congruent with an eastward relative direction when facing north, but with a southward relative direction when facing east. Thus, congruency was defined for each facing direction, taking into account both spatially based and semantically based congruency. As before, filler trial, unrelated to leftward or rightward movement, was included in the study, but not analysed. Thus, description correctness, congruency and proximity served as within-participant variables and compass orientation and familiarity served as between-participant variables. Response button position, as a control variable, was manipulated as in Experiment 1.
Procedure
Before beginning the formal experiment, participants viewed a list of Tufts buildings and verbally confirmed each building’s location. If they did not remember one, the experimenter showed them the building picture. Then participants were instructed to imagine looking at a campus map and oriented to the assigned facing direction by four location pairs not included in the main study (e.g., “Eaton Hall is to the west of Psychology building,” “Campus centre is to the south of Library”). Buildings in these examples were directly north-south or east-west of one another. After these examples, participants began the spatial verification task, similar to Experiment 2 but specific to this environment. They would imagine looking at the first presented building on a campus map and then judge whether the directional term described the location of the second building. They completed 6 practice trials to familiarise themselves with the procedure. The experiment involved 96 trials, presented in random order, that included 10 trials each in the 8 conditions defined by the 3 within-participants variables (description correctness, congruency and proximity), plus 16 non-competition filler trials. Data collection matched Experiment 2.
Results
We eliminated three low-familiarity and two high-familiarity participants for low accuracy (less than .60). For the data analysis of between-participant conditions, there were 9 participants in low-familiarity & facing-north condition, 12 participants in low-familiarity & facing-east condition, 11 participants in high-familiarity & facing-north condition, and 10 participants in high-familiarity & facing-east condition. As undergraduates were more familiar with the campus than the artificial map used in Experiments 1–3, we increased the criteria to .60 in this study, which also matched the criteria used in Wang et al. (2012). A t-test indicated that the hit (M = .82, SD = .10) and correct rejection rates (M = .82, SD = .13) were greater than our elimination criteria, .60; t(41) = 13.86, p < .001; t(41) = 11.19, p < .001. Preliminary analysis showed no effect of response button position or gender, so analyses collapsed across these variables. We analysed correct and incorrect description trials separately as before. Analyses, for both correct and incorrect description trials, consisted of 2 (Familiarity: low, high) × 2 (Congruency: congruent, incongruent) × 2 (Proximity: proximal, distant) × 2 (Compass Orientation: north-up vs. east-up) multivariate ANOVA on accuracy, RT, initial times, and AUC data. RT, initial time, and AUC data only included accurate responses. The PEP analyses also included Time bin with 5 levels.
Correct descriptions
The multivariate analysis showed effects of proximity, F(4, 35) = 5.33, p = .002, compass orientation, F(4, 35) = 2.69, p = .047, and familiarity, F(4, 35) = 2.93, p = .035. Univariate results further explored the above effects.
Accuracy & RT. The accuracy results showed a proximity effect, F(1, 38) = 16.17, p < .001, MSE = .016, BF01 = .006, with higher accuracy for distant, (M = .85, 95% CI = [.81, .89]) than proximal locations (M = .78, 95% CI = [.74, .81]). The RT analyses demonstrated an effect of compass orientation, F(1, 38) = 9.21, p = .004, MSE = 631,155.3, BF01 = .084, with longer RT when oriented north (M = 2734.9 ms, 95% CI = [2554.2, 2915.6]), compared to east (M = 2360.8 ms, 95% CI = [2188.7, 2533.0]).
Mouse Trajectory Results. Analyses of initial time demonstrated an effect of familiarity. High-familiarity participants (M = 694.6 ms, 95% CI = [559.6, 829.5]) took longer to initially move than low-familiarity participants (M = 366.3 ms, 95% CI = [230.1, 502.5]), F(1, 38) = 12.02, p = .001, MSE = 372,388.8, BF01 = .061. The results also showed an effect of compass orientation, F(1, 38) = 4.33, p = .044, BF01 = .569. Consistent with the RT results, participants took longer to initially move when oriented north (M = 628.9 ms, 95% CI = [490.1, 767.7]) than east (M = 432.0 ms, 95% CI = [299.7, 564.2]).
No other effects were found with correct description trials.
Incorrect descriptions
Multivariate analyses showed effects of proximity, F(4, 35) = 2.64, p = .050, and compass orientation, F(4, 35) = 4.97, p = .003.
Accuracy & RT. The results showed that participants responded faster, F(1, 40) = 7.25, p = .011, MSE = 613,348.6, BF01 = .183, when oriented east (M = 2436.4 ms, 95%CI = [2266.6, 2606.7]) than north (M = 2763.5 ms, 95% CI = [2585.4, 2941.7]).
Mouse Trajectory Results. The initial time analyses showed a familiarity effect, F(1, 38) = 9.21, p = .004, MSE = 522,105.6, BF01 = .157, with high-familiarity participants (M = 731.5 ms, 95% CI = [571.8, 891.3]) taking longer to initially respond than low-familiarity participants (M = 391.2 ms, 95% CI = [230.0, 552.5]). Compass orientation also influenced initial time, F(1, 38) = 6.97, p = .012, MSE = 522,105.6, BF01 = .263. Participants took longer to initially respond when oriented north, (M = 709.4 ms, 95% CI = [545.0, 873.8]) than east (M = 413.4 ms, 95% CI = [256.8, 569.9]). The AUC results did not show any significant effects.
The PEP analyses showed an interaction between congruency, compass orientation and time bin, F(4, 152) = 2.73, p = .031, MSE = .003, BF01 = .141. When participants oriented north, the mouse trajectories’ temporal dynamics differed between congruent and incongruent trials, F(4, 160) = 3.17, p = .015. As Figure 3c illustrates, when oriented north, mouse trajectories moved slightly towards the opposite response in congruent trials in time bin 3, but then later (time bin 5) they moved towards the opposite response in incongruent trials (see comparisons in Table 3). The congruency by time bin interaction was not evident when participants oriented east, F(4, 160) = .65, p = .625.
Discussion
Experiment 4 participants learned their campus by navigating. Navigation involves many perceptual and motor experiences that can be integrated into a spatial representation. The encoding hypothesis predicts perceptuo-motor associations with navigation, showing ACEs with any retrieval coordinates. The present results did not show consistent ACEs. The retrieval hypothesis ties perceptuo-motor processes to retrieval and predicts they would be less involved when verifying canonically defined spatial relations, because cardinal terms require an abstraction beyond what may be directly experienced during learning.
Congruency effect
Notable in Experiment 4 results is the virtual absence of ACEs, consistent with the retrieval hypothesis. Further, comparing to Experiment 2, the absence of ACEs suggests that perceptuo-motor processes involved in map knowledge differ from those involved with navigation-based knowledge (Rieser, 1989; Sholl & Bartels, 2002; Sholl & Nolin, 1997). Experiment 2, in which participants studied maps and verified cardinally defined spatial relations, showed ACEs for low-familiarity participants. The use of egocentric recoding as map knowledge developed can explain this finding. Developing a representation from navigation that allows verification of cardinally defined spatial relations would be unlikely to involve egocentric recoding, particularly when the representation is not well developed.
Experience with mapping conventions, i.e., north pointing to top of the page, may explain the only congruency effect seen in Experiment 4. This effect appeared late in the response (PEP data) when participants imagined the map with a north-up compass. Based on map experience and the need to verify cardinally defined spatial relations, people engage existing associations between cardinal and egocentric directions (e.g., west is left on a map; Gagnon et al., 2014).
Impact of reorientation at retrieval
One notable finding here is, even for spatial knowledge acquired from navigation, survey representations based on cardinal directions may still be orientation-specific, as seen in prior research (Brunyé et al., 2015; Frankenstein, Mohler, Bülthoff, & Meilinger, 2012; Gagnon et al., 2014), although not always tied to north-up mapping conventions. The present results demonstrated a benefit of east-up compass, compared to north-up compass. This indicates that navigators build survey representations based on a specific orientation, which might be north (Frankenstein et al., 2012; Gagnon et al., 2014) or may reflect specifics of the environment’s organisation (e.g., its alignment with cardinal axes; Brunyé et al., 2015). Take the Tufts campus as an example. The Tufts University buildings are organised along a west-east axis, an organisation reinforced by experience (See the campus map: http://campusmaps.tufts.edu/docs/Tufts_Medford-Som_Map.pdf). Students usually encode and represent the spatial environment based on the campus structure by primary roads that form principle reference vectors in spatial memory. When this vector is not parallel with the north-south cardinal axis, participants show reliance on it for structuring their understanding and retrieval of spatial knowledge. Accordingly, the present results indicate that orientation change affects cardinal retrieval, as seen with knowledge acquired from both maps and large-scale navigation (Brunyé et al., 2015; Frankenstein et al., 2012; Marchette et al., 2011; McNamara et al., 2003; Sun et al., 2004; Werner & Schmidt, 1999).
The impact of familiarity to the environment
Familiarity impacted responses generally, but did not interact with congruency. Senior undergraduate students, with more navigation experience on campus, responded consistently slower than did first-year students. As the senior class students have more experience of navigating through the campus, their survey representations of the campus might be connected more stably to the preferred orientation, like the east-west axis in the present study. Once the spatial representations are retrieved based on an unfamiliar orientation or pattern, it may take longer for senior students to make responses than freshman students did. This difference likely reflects the abstraction processes when developing a survey representation from navigation (Brunyé et al., 2008; Taylor et al., 1999; Taylor & Tversky, 1992). However, as we did not control over the extent or nature of participant navigation through the campus, it needs further study to examine whether it reflects the abstraction processes or possible artefacts of yet unknown group differences that may modulate our results.
The effect of familiarity on perceptuo-motor processing showed different patterns with cardinal retrieval between navigation-based representation and map knowledge (Experiment 2). With less-developed map knowledge, people may need to associate cardinal directions to their own egocentric system to better understand the more abstracted cardinal relations. These associations likely become unnecessary as map knowledge solidifies. For this reason, congruency may differentially impact map knowledge based on familiarity. However, we did not observe any effects of familiarity on perceptuo-motor processing when retrieving navigation-based survey representations. Such differences may suggest the distinction of survey representation acquired from map learning and real navigation. Cardinal directions are readily evident during map learning and can be perceptually associated with egocentric references, particularly early in learning. In contrast, a traveller may not have to recognise cardinal directions when navigating through an environment. Thus, even though people can form survey representation with sufficient navigation, there might not necessarily be tied to cardinal directions.
Wang et al. (2012) also reported familiarity by congruency interaction during egocentric retrieval of navigation-based knowledge. Low-familiarity participants showed ACEs driven by semantic processing, while high-familiarity participants showed ACEs driven by spatial representations. We did not have such interaction in this study. While egocentric terms automatically reactive perceptuo-motor experience, the more abstracted cardinal terms involved weaker perceptuo-motor associations for both low- and high-familiarity participants. Such differences may support again that the involvement of perceptuo-motor processing in spatial knowledge depends on retrieving representation, although people could use both egocentric and survey representation regardless of information sources.
Impact of location proximity
Experiment 4 again showed a symbolic distance effect, i.e., greater accuracy, shorter initial times for distant landmarks. The symbolic distance effect is consistent with our other map-based knowledge findings (Experiments 1, 2, and 3). However, it is inconsistent with our navigation learning study (Wang et al., 2012), wherein participants verified navigation-based spatial knowledge defined egocentrically. In Wang et al. (2012), people retrieved proximal locations more accurately than distant ones. These findings would appear to reflect how people structure spatial information to reduce memory load, interacting with retrieval demands.
Summary
To summarise, the present experiment examined whether cardinal retrieval of navigation-based spatial knowledge involves perceptuo-motor processing by showing ACEs. The results again support the retrieval hypothesis by showing largely absent ACEs. The only exception is when participants retrieved cardinal relations based on an atypical, east-up compass, which is not consistent with a conventional map reference frame.
General discussion
ACEs in spatial knowledge development support retrieval hypothesis
Four studies explored the role of encoding and retrieval on perceptuo-motor associations involved in spatial knowledge and its development. In the first three studies, participants studied a map and then verified spatial relations based on egocentric (Experiment 1), cardinal (Experiment 2), or mixed (Experiment 3) coordinates. In Experiment 4, spatial knowledge was acquired through navigation and retrieved based on cardinal coordinates. We also compared this work to Wang et al. (2012), wherein navigation-based spatial knowledge was verified egocentrically. The encoding hypothesis predicts that spatial representations based on navigation would have a stronger perceptuo-motor basis than those learned from maps. The retrieval hypothesis predicts that the necessities of how information will be used, and therefore retrieved, may differentially evoke perceptuo-motor associations. In our studies, ACEs changed as a function of retrieval demands, supporting the retrieval hypothesis.
While the results generally supported the retrieval hypothesis, environment familiarity also played a role. With well-developed spatial knowledge, participants demonstrated ACEs when verifying egocentrically defined relations (Experiments 1 and 3, and Wang et al., 2012), but not those defined with cardinal coordinates (Experiments 2 and 4). One exception to this emerged. When high-familiarity participants could not predict the retrieval coordinates of a given trial (Experiment 3), weak ACEs emerged to cardinal coordinates. Together, these results suggest strategic retrieval when spatial knowledge is well-learned. Such results are consistent with the intentional weighting mechanism proposed by Hommel and his colleagues (Hommel, 2017; Memelink & Hommel, 2013). According to the intentional weighting theory, in the present study, when participants were required to verify spatial relations from an egocentric or survey perspective, the top-down cognitive control would intentionally increase weight on the required perspective based on their prior spatial knowledge. Consequently, perceptuo-motor processing would be reactivated with egocentric retrieval, while it would be not necessarily involved in survey retrieval, as we found in the present study with high-familiarity group.
Participants with less-developed knowledge showed a different pattern. With map learning, they demonstrated ACEs on both egocentric and cardinal trials, whether or not the retrieval demands could be predicted. As map knowledge develops, people may engage in egocentric recoding, spontaneously associating cardinal directions with egocentric coordinates and reactivating these associations at retrieval. This cardinal-egocentric mapping may initially facilitate map encoding. It should be noted that the ACEs to cardinal coordinates were weaker when mixed with egocentric coordinates. Constantly switching between egocentric and cardinal coordinates may confuse low-familiarity participants, leading to difficulty maintaining the cardinal-egocentric mapping. With navigation, no ACEs emerged when verifying cardinal coordinates. Cardinal coordinates are likely not as available to navigators as egocentric coordinates. When navigational knowledge is less-developed, mapping cardinal directions to the available egocentric reference frame is difficult, a finding also shown in other work (Brunyé & Taylor, 2008; Thorndyke & Hayes-Roth, 1982). These results suggest that the information source (maps vs. navigation) impacts spatial knowledge development and the ability to flexibly retrieve spatial knowledge in response to varied task demands.
Spatial proximity affects perceptuo-motor associations
Although the present work and Wang et al. (2012) consistently suggest that egocentric retrieval led to ACEs, proximity interacted with ACEs in different ways. The differences between Wang et al. (2012) and the current Experiments 1 and 3 indicate that the nature of ACEs may differ based on spatial information source. ACEs emerged only with proximal locations when spatial knowledge was acquired from navigation (Wang et al., 2012). During navigation, people have more perceptual and motoric interactions with nearby, compared to distant, locations (Carlson & Kenny, 2006; Hölscher, Tenbrink, &Wiener, 2011; Logan & Sadler, 1996; Regier & Carlson, 2001; Wiener & Mallot, 2003). These perceptuo-motor associations then give rise to action-based compatibility effects. However, proximal and distant information is readily available on maps, even with limited study, and can be retrieved as a visual image (Brunyé & Taylor, 2008; Kosslyn, 1994; Thorndyke & Hayes-Roth, 1982). The interaction between the simulated spatial perception and concurrent actions would lead a perception-based compatibility effect, akin to the spatial Stroop effect (De Houwer, 2003; Hommel, 2011). With map knowledge, verifying distant spatial relations showed more stable ACEs, which were influenced little by familiarity or orientation, compared to proximal locations. People may mentally structure distant locations in different clusters to make them more distinguishable, leading to a stable spatial Stroop-like effect (Hommel et al., 2000; Stevens & Coupe, 1978; Wang et al., 2014). One exception emerged in our data. When participants had to deal with mental reorientation, coordinate transfer and spatial verification simultaneously (Experiment 3), processing proximal rather than distant locations demonstrated more stable ACEs. This effect might indicate that with high mental workload, processing proximal locations involves more automatic perceptuo-motor associations, even for map representations.
On the other hand, the proximity did not interact with ACEs in cardinal retrieval either after map learning (Experiment 2) or real navigation (Experiment 4). Although participants in Experiment 4 shared the navigation experience with those in Wang et al. (2012), retrieving cardinal relations based on survey representation was less affected by proximity, akin to retrieving map representations. Our retrieval hypothesis may explain this pattern: retrieval demands may play more important roles in the involvement of perceptuo-motor processing in spatial knowledge. Although people acquire more perceptual and motoric interaction with proximal locations during real navigation, they treat proximal and distant locations within one map-like, survey representation when required to retrieve cardinal relations.
In addition, the relative proximity of locations itself consistently impacted verification performance. With both map and navigation-based knowledge, participants showed a symbolic distance effect, i.e., an advantage in thinking about distant spatial relations (Friedman & Montello, 2006; Hommel et al., 2000; McNamara et al., 1992). People may divide the map into several clusters as in categorical effect, so responses to distant landmarks could be categorically based (Hommel et al., 2000; McNamara, 1991). More importantly, the proximity effect also appeared when retrieving navigation-based spatial knowledge with cardinal coordinates (Experiment 4). This result lends strong support to the categorically feature of survey representation, regardless of spatial information sources.
Effect of orientation change on perceptuo-motor processing
The present study explored interactions between orientation changes and perceptuo-motor processing. Online mouse tracking afforded additional insights into how people mentally process orientation change with spatial knowledge. Analysis of the mouse trajectory temporal dynamic allows the present work to extend previous studies. Studies on stimulus-response compatibility with visual stimuli, such as spatial Stroop paradigm, have suggested that perceptuo-motor processes activated by visual perception interfere with concurrent motoric processing, even when stimuli or display orientation changes (Hommel & Lippa, 1995; Kerzel, Hommel, & Bekkering, 2001).
The present work additionally extended previous research in two ways. First, here participants responded from memory, rather than perception, but still demonstrated ACEs regardless of retrieval orientation. This suggests image-like properties for map knowledge, leading to a perceptual-based compatibility effect. Second, we differentiated temporal dynamics of ACEs in the studied orientation from those in novel orientations. For egocentric only retrieval (Experiment 1), ACEs appeared early and strong with the studied orientation, but were later and weaker with the novel orientation. Mental reorientation may interfere or delay the interaction between perceptuo-motor processes associated with map retrieval and action planning to make a response. The difference disappeared when using mixed coordinates (Experiment 3), implicating task demands. Furthermore, the effects of reorientation suggested a different influence on navigation-based knowledge. In Experiment 4, where participants verified cardinal relations after real navigation, an early and weak semantically based ACE emerged when instructed to retrieve from a northward orientation, followed by a spatially based ACE later in the response. The northward orientation is inconsistent with mapping conventions, which people appear to use when developing spatial representations (Brunyé et al., 2015; Gagnon et al., 2014; Marchette et al., 2011). Reorienting likely induces a cognitive load. With a heavier cognitive load, the automaticity of language comprehension might engage first, followed by more controlled spatial information retrieval. This also suggests the difference between navigation-based survey representation and map knowledge, as we did not observe semantically based ACE with map knowledge.
The difference between cardinal and egocentric retrieval with orientation changes may also reflect the nature of spatial representations. Our results may shed light on whether the egocentric or survey representations acquired from different sources share the same features. Our results show higher accuracy and shorter RT and initial times with the studied orientation, especially with egocentric retrieval. However, cardinal directions do not change with orientation changes. Consistent with this, Experiment 3, with mixed coordinates, suggested an advantage (higher accuracy, shorter RT and initial times) for cardinal coordinate retrieval when retrieving from a novel orientation, compared to egocentric coordinate retrieval. Meanwhile, the temporal dynamic of cardinal coordinate retrieval showed an earlier modulation of trajectories than those with egocentric retrieval when retrieving from a novel orientation (see Figure 5). The above discrepancy between egocentric and cardinal retrieval with orientation changes lends support to different mechanism underlying egocentric and survey representations, even though they may be structured based on the same learning sources.
Of course, survey representations based on navigation likely differ in some ways from map-based representations. When navigating through an environment, travellers have firsthand egocentric experience. As the representation builds, they may spontaneously select a preferred orientation to help structure their spatial knowledge, like the case of Tufts campus. If the retrieving orientation is not consistent with the preferred orientation, it may take longer to identify spatial relations. However, here we did not observe any benefit of a particular map orientation (north-up or east-up) with map learning (Experiment 2). Maps show locations relative to cardinal directions, generally designated by a compass. The cardinal directions would not change when map orientation changes, whether the compass is conventional or atypical. This may explain why participants were not affected by the compass orientation change when retrieving map knowledge with cardinal coordinates.
Limitations
Our study has some limitations within which our findings need to be interpreted carefully. First, the mouse tracking as an online methodology is still a type of observation of overt behaviours, although it could reveal temporal dynamic of hand movement during processing spatial representations. For instance, while we assumed that the data of initial time might indicate action planning and/or confidence, it may need more evidence to support such argument. We are conducting further electroencephalography research to disclose the substrates underlying overt behaviours. Second, it might be valuable for future studies to consider individual differences in perspective preference and how they might modulate spontaneous development of egocentric or allocentric coordinate knowledge regardless of input modality. Third, we have no control over the extent or nature of participant navigation through the Tufts campus in Experiment 4; that likely contributes some noise to the data and it would be beneficial to know a bit more about the frequency, extent, and type (foot, bike, car) of campus navigation, and any other differences between participant groups that might modulate our results. Finally, though we refer to perceptuo-motor associations, we cannot effectively disentangle independent and interactive contributions of perceptual versus motor associations. Though our dependent task primarily targeted the latter, more research is warranted on effectively isolating the respective roles of these systems in driving spatial memory formation and retrieval.
Summary
The present studies investigated perceptuo-motor associations in spatial knowledge, expanding the work of Wang et al. (2012). Overall, our results support a greater role for retrieval demands, than for encoding process, in engaging perceptuo-motor associations. In addition to supporting the retrieval hypothesis, the present results show how learning experience interacts with perceptuo-motor processing. When spatial knowledge comes from navigation, perceptuo-motor associations are activated with egocentric, but not with cardinal retrieval. When map knowledge is well developed, people can adopt cognitive strategies consistent with perspective demands of a retrieval task. But when map knowledge is less developed, the ACEs seen when retrieving with cardinal coordinates suggest effortful egocentric recoding of cardinal directions. Overall, the influence of retrieval, rather than encoding, also suggests that flexibility in accessing spatial representations can emerge early in their development.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the National Natural Science Foundation of China (31400865).