Abstract
This study investigated to what extent humans can encode spatial relations between different surfaces (i.e., floor, walls, and ceiling) in a three-dimensional (3D) space and extend their headings on the floor to other surfaces when locomoting to walls (pitch 90°) and the ceiling (pitch 180°). In immersive virtual reality environments, participants first learned a layout of objects on the ground. They then navigated to testing planes: south (or north) walls facing Up, or the ceiling via walls facing North (or South). Participants locomoted to the walls with pitch rotations indicated by visual and idiothetic cues (Experiment 1) and only by visual cues (Experiment 2) and to the ceiling with visual pitch rotations only (Experiment 3). Using the memory of objects’ locations, they either reproduced the object layout on the testing plane or did a Judgements of Relative Direction (JRD) task (“imagine standing at object A, facing B, point to C”) with imagined headings of south and north on the ground. The results showed that participants who locomoted onto the wall with idiothetic cues showed a better performance in JRD for an imagined heading from which their physical heading was extended (e.g., imagined heading of North at the north wall). In addition, the participants who reproduced the layout of objects on the ceiling from a perspective extended from the ground also showed a sensorimotor alignment effect predicted by an extended heading. These results indicate that humans encode spatial relations between different surfaces and extend headings via pitch rotations three-dimensionally, especially with idiothetic cues.
Introduction
Humans and non-human animals are living in three-dimensional (3D) spaces. In a natural environment, the ground provides two horizontal dimensions and the sky above the ground adds the third dimension (i.e., vertical dimension). Different species vary in navigational experiences in 3D spaces. While some animals navigate often in all three dimensions (e.g., rats travel underground pathways and bats fly in the air), humans primarily navigate on the horizontal planes (i.e., the ground). Recently, there have been growing interests in investigating how animals represent 3D spaces and update their headings in 3D spaces (Jeffery et al., 2013, 2015) and whether different navigation styles between species affect spatial memory and navigation (Du, Mahdi, et al., 2016; Du, Spetch, & Mou, 2016; Flores-Abreu et al., 2014). It is still not clear to what extent humans represent spatial relations in 3D spaces and update their locations relative to 3D spaces.
People develop two-dimensional (2D) maps, printed or on GPS devices, to represent locations of important places on the horizontal surface. People also use maps to pinpoint their locations and headings (self-localisation) on the horizontal surface (Levine, 1982). Similarly, most studies on human spatial memory and navigation focus on the format of mental maps of the horizontal surface and the mechanisms of updating one’s self-localisation horizontally. One important finding of research on human spatial cognition on the horizontal surface is that mental representations of different local spaces might not be integrated in a single global mental map (Friedman & Brown, 2000; Marchette et al., 2014; McNamara, 1986; Stevens & Coupe, 1978; Tversky, 1981; Wang & Brockmole, 2003). A local space here refers to a vista space proposed by Montello (1993). In a vista space, people can view locations of all objects with minimal locomotion and viewpoint change, in contrast to a large-scale space, in which people need significant locomotion to view all objects’ locations (Montello, 1993). In addition to the scale, a local space is usually a space within a boundary (e.g., within a room), whereas a large-scale environment is usually across boundaries (e.g., two rooms) (Mou & Wang, 2015).
In a local space, people represent interobject spatial relations (Easton & Sholl, 1995; Mou & McNamara, 2002; Shelton & McNamara, 2001). People also encode spatial relations between two separate arrays of objects (Greenauer & Waller, 2010; Holmes et al., 2018; Kelly & McNamara, 2010; Kelly et al., 2018). People update their self-localisation automatically while locomoting in the local space even without vision (Rieser, 1989). In contrast, in across-boundary spaces, people may not spontaneously encode spatial relations between two spaces across the boundary (Marchette et al., 2014). Moreover, people may not automatically update their self-localisation relative to spaces that are separated from their current local space (Wang & Brockmole, 2003). With significant cognitive efforts, people may be able to develop spatial relations between two across-boundary spaces. Participants formed global spatial relations when they were explicitly required to learn spatial relations between two headings inside different rooms (Shine et al., 2016), when they were primed by a prior learning of the global environment in which the two local rooms were located (Lei et al., 2020), or when the two spaces were connected by similar visual appearance (Han & Becker, 2014; Kelly et al., 2007). In these studies, participants also travelled between the two local spaces for many times.
Why do people not spontaneously mentally represent spatial relations of across-boundary spaces while they represent spatial relations of items in the same local space? There are two possible reasons. First, it is difficult to perceive spatial relations between across-boundary spaces. People can directly see interobject spatial relations within a boundary but they cannot directly see spatial relations across boundaries (Mou & Wang, 2015). Second, people directly interact with the immediate local space but not with the spaces outside the current boundary. Thus, people should prioritise updating their spatial relations relative to the immediate local space. People need to have mental representations of the locations of objects in the same local space so that they can effectively update their spatial relations relative to the local space. It might not be efficient to encode spatial relations across boundaries or to update ones’ self-localisation relative to spaces other than the immediate local space (Wang, 2004, 2017).
Inspired by the different natures of spatial memory and updating for within-boundary and between-boundary spaces, we ask whether the nature of spatial memory and updating in a 3D local space are the same as that in the horizontal local space. More specifically, the floor, the walls, and the ceiling of a room are different surfaces in the same local space. Do people represent different surfaces separately or in an integrative manner? When people move to different surfaces (e.g., walk onto a wall from the floor with a 90° pitch rotation 1 ), do they update their headings relative to the previous surfaces? Answers to these questions can advance our understanding of the format of human mental representations of 3D spaces and the mechanisms of updating one’s self-localisation, in particular one’s orientation/heading, in 3D navigation. They can also shed light on commonality and differences in spatial memories and navigation across species.
One possible format of spatial memories of different surfaces in the same room is non-integrative. People may treat different surfaces in the same room just as they treat across-boundary spaces. They form separate spatial memories of individual surfaces but do not integrate spatial memories of individual surfaces in a global manner. The rationale of this speculation is given as follows. People primarily navigate on the horizontal surface (i.e., floor). Encoding spatial relations between the floor and other surfaces is not relevant to navigation on the floor. For example, differentiating different walls with unique objects on them (e.g., a wall with a clock) can tell one’s moving direction on the horizontal surface. However, whether the unique objects (e.g., the clock) are on the up or down side of the walls is not informative of one’s moving direction. In addition, the edges of surfaces provide clear visual boundaries of surfaces (e.g., the floor is covered by a dark blue carpet whereas the wall is painted in green). Just as the boundaries separating rooms significantly impair the integration of spatial memories of different rooms, the edges of two surfaces also prevent people from encoding spatial relations between two surfaces. Thus, to conceptualise spatial memories of different surfaces in the same room, we first hypothesise that people encode individual surfaces in separate local mental maps but do not encode the spatial relations between different surfaces. Accordingly, people only update their 2D heading on the current navigating surface and do not update their headings relative to other surfaces or across the horizontal-vertical boundaries. This hypothesis is referred to as the non-integration hypothesis.
One hypothesis alternative to the non-integration hypothesis is inspired by the findings of animal neuroscience. Some recent research in animal neuroscience showed that some animals can update their heading representation in 3D space by extending their headings from grounds to other surfaces (Jeffery et al., 2013, 2015). When rats climbed onto vertical walls from a horizontal plane, the head direction cells maintained the same firing patterns as if the walls were an extension of the floor (Calton & Taube, 2005; Stackman et al., 2000; Taube et al., 2013). When flying bats performed pitch rotation movements in all 360°, the movements did not affect the head direction cells’ firing patterns (Finkelstein et al., 2015). These studies suggest that for these animals, the represented heading on vertical and inverted planes is an extension of the heading on the ground by pitch rotations.
A toroidal model (Figure 1) has been proposed to explain how pitch rotation movements do not affect the azimuth head direction cells’ firing patterns (Finkelstein et al., 2015). If the animal locomotes to the vertical planes via a pitch rotation (e.g., from the ground to the walls), head direction cells treat the vertical plane as an extension of the allocentric reference directions on the floor (e.g., cardinal directions) rotated by the 90° pitch up. As a result, the direction up on the north wall is the extension of the north on the ground whereas up on the south wall is the extension of the south on the ground. If the animal locomotes to the inverted plane (e.g., ceiling), head direction cells treat the ceiling as an extension of the allocentric reference directions on the floor rotated by the 180° pitch up. Consequently, the direction north on the ceiling is south on the floor. A dual-axis model has been proposed recently which accounts for locomoting between two vertical walls (Page et al., 2018). According to this theory, if the animal locomotes via pitch rotations across vertical walls, the head direction cells’ firing will change, which is different from what is predicted by the toroidal model. However, these two models are identical if animals only travel between horizontal and vertical planes. In this study, human participants only locomoted between horizontal and vertical planes. We do not differentiate these two models and use the extended-heading hypothesis to refer to both.
Coordinate systems and reference frames in a 3D environment according to the extended-heading hypothesis.
To our best knowledge, there is no human study providing clear empirical data that can distinguish the extended-heading hypothesis and the non-integration hypothesis for human 3D spatial memory and navigation. On one hand, although the findings of rats and bats favour the extended-heading hypothesis over the non-integration hypothesis, humans have quite different navigation habits from rats or bats. In particular, humans primarily locomote on the ground and lack 3D navigation experiences compared with rats and bats. People may occasionally navigate on a slope but do not afford a large degree of pitch rotation before falling. People may pitch their bodies on a small slope. For a steep slope, they reduce the pitch rotation by climbing instead. Although people lie down (pitch 90°) while sleeping, sleeping is not a navigational activity. Hence, people (except some professionals like fighter pilots) do not have a lot of experiences in updating their 3D heading using pitch rotations in navigation. The rare experiences of pitching our body 90° or more in navigation might prevent humans from spontaneously extending their headings on the ground to vertical and inverted surfaces (Calton & Taube, 2005).
On the other hand, some studies showed that humans distinguish horizontal and vertical movements in a virtual reality (VR) environment where only visual stimuli are available to show the movements (Indovina et al., 2016). Humans can use slope as a cue to orient themselves (Nardi et al., 2011). Moreover, humans also sometimes treat the directions of up and front as equivalent (Levine et al., 1982). For example, when participants read a map on a vertical surface and then used the map to judge goal directions, their judgements of goal direction were easier from the heading aligned with the up direction of the map than from the direction that is contra-aligned with it. This suggests that at least in some circumstances like using a map, humans treat gravity up on the wall as equivalent to the front of their body and therefore extend a horizontal coordinate system to the vertical wall. Therefore, evidence so far cannot tell whether humans can extend their headings in 3D navigation.
The primary purpose of this study was to differentiate the extended-heading hypothesis from the non-integration hypothesis. In this study, participants learned an object array on the floor in a rectangular room with environmental cues. Later they navigated to a wall or the ceiling of the room via pitch rotations. According to the extended-heading hypothesis (Figure 1), people can extend their headings on the ground to walls and ceiling by pitch rotations while locomoting to those surfaces. According to the non-integration hypothesis, people neither encode the relations between headings on the different surfaces nor extend headings across surfaces. Instead, people encode different surfaces in separate local mental maps; when they move to different surfaces, they lose track of the headings relative to the original surface just like they move to a remote space, which is not relevant to the current space (Wang, 2017).
We examined these two hypotheses by using two tasks. One task was to reconstruct the object array in the testing plane (on the wall or on the ceiling) after pitch rotations from the ground to the testing plane (the placement task). Using the responses on the testing plane, we calculated the relationship between the directions on the testing plane (i.e., Up, North/South) and those on the ground (i.e., North). For example, if the upside of the reproduced layout on the north wall corresponds to the north side of the original layout on the ground, it indicates that participants treat Up on the north wall as the North on the ground. We then used the calculated relationship to differentiate the two hypotheses. If participants treat Up on the north wall as the North on the ground and treat Up on the south wall as the South on the ground, then these results support the extended-heading hypothesis. If participants treat Up on the south and north walls the same direction on the ground, then these results support the non-integration hypothesis. We note that using the extended heading might not be obligatory in the placement task. Participants who have extended their headings may also store the original orientation while learning objects on the floor and then just reconstruct the array from the original learning orientation regardless of the representations of the extended headings. To test this possibility, we used the second task.
The second task was a Judgements of Relative Direction (JRD) task using the memories of the layout. Participants were asked to imagine north or south directions on the ground and then make JRDs of objects on the ground. Previous studies have showed that the JRD performance depends on the relations between the imagined heading and the physical heading (Kelly et al., 2007; Mou et al., 2004; Rieser, 1989). JRD is easier when these two headings are aligned than when these two headings are misaligned. The alignment effect is attributed to the interference of the sensorimotor representations of the physical headings with the mental representations of the imagined headings in the JRD task (Riecke & McNamara, 2017). The sensorimotor alignment effect has been widely used to examine the obligatory spatial updating of self-localisation during physical body rotations on the horizontal surface (i.e., yaw rotations) (Farrell & Robertson, 1998; Farrell & Thomson, 1998; Kelly et al., 2007; Mou et al., 2004; Rieser, 1989; Waller et al., 2002). We assume it can also examine obligatory spatial updating of self-localisation in the 3D space (i.e., extending heading) during physical pitch rotation, if extending headings in the 3D space occurs. Critically, participants could not inhibit the sensorimotor representations of actual headings to eliminate sensorimotor alignment effects even when they were instructed to do so (Farrell & Robertson, 1998). Therefore, the sensorimotor alignment effect can be used to examine the relations between the represented physical headings on the testing surfaces and the imagined headings on the ground.
The extended-heading hypothesis predicts an extended sensorimotor alignment effect, that is, the performance would be better when the imagined heading was the same as the extended heading during retrieval. For example, for the imagined heading of North on the ground, facing Up on the north wall is easier than facing Up on the south wall, and facing South on the ceiling is easier than facing North on the ceiling (see Figure 1). In contrast, according to the non-integration hypothesis, participants do not encode the spatial relations between the physical headings on the testing surfaces and the imagined heading on the ground. As a consequence, participants might act as if they were tested in a remote space, not relevant to the original learning space (Shelton & Marchette, 2010). Thus, they would not show any sensorimotor alignment effect. The predictions according to the two hypotheses are summarised in Table 1.
Two hypotheses and the corresponding predictions for the placement and JRD tasks.
JRD: Judgements of Relative Direction.
The top rows of the table show the predictions on the placement task according to each of the hypotheses, which are specified in the correspondence between Up on the north/south wall and the directions on the ground and between North/South on the ceiling and the directions on the ground. The bottom rows of the table show the predictions on the JRD task according to each of the hypotheses, which are specified in the comparisons between the conditions when the participant was on the north/south wall or when the participant was facing north/south on the ceiling.
In addition, the extended-heading hypothesis does not claim that there is a true 3D spatial representation and updating. While the extended-heading hypothesis treats Up on the wall as the extended moving direction on the ground, a hypothesis claiming a true 3D spatial representation and updating treats the vertical surface orthogonal to (i.e., independent of) both directions on the floor. Consequently, while the extended-heading hypothesis predicts that there will be sensorimotor alignment effect in the JRD tests, a hypothesis claiming a true 3D spatial representation and updating predicts no such an effect.
There were three experiments in this study. In Experiment 1, we examined whether humans can extend their headings or not from the ground to the vertical walls with physical body pitch rotations using both placement and JRD tasks. In Experiment 2, we examined heading extending from the ground to the vertical walls with only visual pitch rotations to examine whether visual pitch rotations are sufficient for heading extending (Klatzky et al., 1998). In Experiment 3, we examined the heading extending from the ground to the ceiling during visual pitch rotations using both tasks.
Experiment 1
The primary purpose of Experiment 1 was to test whether participants extend their headings from the floor to the vertical walls using both the placement task and JRD task. Participants physically pitched their bodies onto two opposite vertical walls (North or South walls). According to the extended-heading hypothesis, the upside of the reproduced layout on the North (South) wall should be the North (South) side of the original layout on the ground, respectively. The JRD task should show the sensorimotor alignment effect. According to the non-integration hypothesis, the upside of the reproduced layouts on the North and South walls should be the same. The JRD task should not show the sensorimotor alignment effect. In addition, a model claiming a true 3D spatial representation also predicts no sensorimotor alignment effect.
Method
Participants
A total of 48 university students participated in the experiment (age range = 18–53 years, M = 20.19 years) with an equal number in each gender. One additional participant dropped out during the experiment due to VR sickness. All participants received course credits in an introductory psychology course for their participation. Written informed consent was obtained from all individual participants. The power of using 48 participants is 0.67 to test a medium sensorimotor alignment effect (Cohen’s d = 0.50) at the alpha level of .05 for a two-tailed t test.
Apparatus
The experiment was conducted in a 4 m × 4 m physical room. Participants saw the virtual experimental environments presented by an immersive virtual reality system that used Vizard software (WorldViz, Santa Barbara, CA), a head-mounted display (HMD, Oculus Rift, Oculus VR, LLC., Irvine, CA), and an InterSense-900 motion-tracking system (InterSense, Inc., Billerica, MA). Screen resolution within the display was 2,160 × 1,200 pixels. The diagonal field of view (FOV) was 110°. Participants’ physical viewing orientation was tracked by the IS-900 motion-tracking system so that they could physically rotate their bodies to change their viewing orientations in the virtual environment.
A joystick (Logitech ATK3, Logitech, Newark, CA) was used for translation in the virtual environment. The travel direction was determined by the projection of the direction of participants’ viewing orientation onto the current travelling plane. The joystick was used for making a response in the JRD task. The resolution of the joystick was 1°.
A swivel recliner was used for pitch rotations. The chair could be reclined from vertical upright position to a horizontal, flat position (i.e., 90° pitch) so that the participants could lie on their back to view the environment. Meanwhile, the leg rest could give extra support to participants’ legs so that they could lie on their back in a flat body position.
To make a pointing response in the placement task, participants pointed a virtual wand at locations by controlling a physical pointer that was tracked by the InterSense motion-tracking system. Participants could move the wand to indicate the response location and click a button on the wand to confirm the response.
Environment
The virtual experimental environment was a rectangular virtual room (Figure 2a). The room’s size was 4 m wide, 8 m long, and 4 m high. The walls were homogeneously green with textures to increase the perceived optic flow during navigation. For simplicity, in rest of this article, we use north, south, east, and west to distinguish the directions on the ground. During the experiment, participants were never instructed with these terms. At each corner, to serve as feature or landmark cues, furniture was presented: a door, a bookshelf, a table, and a picture. The ceiling was light grey with a pattern of square textiles. Two lights were presented on the ceiling.
(a) Top view of the experimental environment. The arrow denotes the viewing direction in the learning phase. The X denotes the learning position. (b) An example of the object array. The main axis of the array was always oriented towards North on the ground. Object locations were the same across participants. For simplicity, object names are used to show the locations. For each participant, the objects were randomly assigned to the locations on the mat.
Materials and design
In the learning phase, a light grey round mat (2 m in diameter, 0.1 m in height) was presented at the centre of the room on the ground to show the seven target objects (Figure 2b). The objects were a clock, a phone, a paperclip, a bottle, a lock, a candle, and a piece of wood. The association between the locations and objects was randomised across participants. A small blue platform (0.5 m in diameter, 0.1 m in height) was presented at the southwest side of the mat and 45° away from the orthogonal south to guide participants to locomote to the pre-determined learning position. In the testing phase, the mat and the platform (without objects) were displaced to the testing walls (north or south). The mat was presented at the centre of the testing wall. The platform was presented at the middle bottom of the wall (0.8 m from the ground) and exactly below the mat. Participants needed to navigate to the platform and then perform the JRD tests and replace objects to the platform (placement task) based on their memory of the objects’ locations.
The participants maintained an upright perspective when navigating on the ground. Whenever the participants reached the boundary between the ground and the north or south wall, the experimenter pulled a lever to move the back and the leg rest of the recliner to the horizontal, flat position so that the participants lay down on their back. While participants were lying down, the participants’ perspective was rotated towards 90° upwards (pitch up) continuously. Therefore, participants faced Up when they were on the testing walls. When the participants navigated from the wall to the ground, they physically got up and then the experimenter pulled the lever to return the chair to the vertical, upright position. The participants’ perspective was also rotated 90° (pitch down) so that they could return to an upright perspective when navigating on the ground.
We also added a ground placement test phase after participants learned the layout of objects and before they navigated to the walls. In the ground test phase, participants navigated to a location either north or south of the mat and then replaced the objects. The ground test was used to ensure that participants could update their headings when locomoting on the ground.
In the placement task, the primary independent variable is the testing wall (north or south). All participants navigated to both walls. The order of the testing walls was counterbalanced across participants.
In the JRD task, each test consisted of 12 trials, 6 for imagining north and 6 for imagining south on the ground, presented in a randomised sequence (see Table 2). On each trial, the participants were presented the texts on the screen of the HMD: “Imagine you are standing at Object A, facing Object B.” Then they clicked a button on the joystick to see the text: “point to Object C.” Objects A, B, and C were from the object array in the learning phase as shown in Table 2. Object A was the imagined position. Object B was the imagined facing object. Object C was the target. The participants were asked to respond as quickly as possible without sacrificing accuracy. The response direction and the latencies for orientation and pointing response were recorded. The participants were instructed not to turn their heads when performing the JRD task.
Trial list in the JRD task.
JRD: Judgements of Relative Direction.
For simplicity, object names are used in the table to show the locations. See Figure 2b for the object locations. For each participant, the objects were randomly assigned to the locations on the mat.
Table 3 shows the design of the JRD task. As in previous literature using JRD tasks, we labelled the conditions into two independent variables: (a) the angular difference between the actual facing direction and the imagined heading direction at the time of test (denoted Actual-Imagined or AI) and (b) the angular difference between the learning heading (i.e., North) and the imagined heading direction at the time of test (denoted Learning-Imagined or LI; for this method, see Mou et al., 2004). Using this method, the main effect of AI would reflect the sensorimotor alignment effect and the main effect of LI would reflect the memory or the learning effect. For the sake of composition, we denote LI = 0° as imagined North condition and LI = 180° as imagined South condition.
The design of the JRD task.
JRD: Judgements of Relative Direction.
Procedure
Participants were tested individually. They were led into the experiment room blindfolded. They sat on the swivel recliner and wore the HMD to view the virtual environment. First, they were introduced to the feature cues in the virtual environment and were asked to point to them with the wand. Then they practised using the joystick to navigate in the environment, including locomoting on the north and south walls along the gravity up-down and the east-west axes. After that, they entered the formal experiment.
In the learning phase, the participants navigated to the platform and faced the mat and stayed in this position during the entire learning phase. Therefore, the learning perspective was northeast. Before being presented with the objects, the participants once again pointed to the feature cues in the environment. Then, the participants were asked to learn the array of seven target objects presented on the mat (Figure 2b). There were two learning blocks. In each block, the object array was presented for 1 min and then removed. The participants used the wand to place the objects, which were probed at the bottom of the HMD screen sequentially in a random order. Their response locations were recorded. After the participants replaced all of the objects, the replaced objects were removed. Because in each trial of the JRD task, the names of objects were presented to the participants, the experimenter named each object in the first learning block.
In the ground test phase, the participants navigated to the platform that had been displaced to either the north or the south of the mat on the ground. This position was counterbalanced between participants. The participants faced the mat and replaced the objects with the wand. Their response locations were recorded. There was no feedback regarding the accuracy of their placement. After all of the objects were placed, the replaced objects and the mat were removed.
In the wall test phase, the participants climbed up to each testing wall from the ground and navigated to the platform. The participants performed the JRD tests and then replaced the objects on the mat. There was no feedback regarding the accuracy of the JRD pointing directions or their placement of the objects. The JRD tests were conducted earlier than the placement tests to avoid having participants rely on the visual memories of their replaced layout on the walls in the JRD tests. Participants alternately navigated to the north or south wall across the four testing phases (two JRD tests followed by two placement tests) and their first testing wall was counterbalanced across participants. The participants navigated back to the ground between tests.
Finally, participants were administered with two spatial transformation ability tests, 3D mental rotation test (Vandenberg & Kuse, 1978) and perspective taking test (Hegarty & Waller, 2004). After the wall tests, the participants were led to another testing room to perform the mental rotation test and the perspective taking test sequentially. They were also asked to report their training history of pitch rotations in real life like gymnastics training. They reported a score of 1–5 in which 1 means minimal training and 5 means extensive training. For the details of the spatial transformation tests, see supplementary materials.
Calculation of estimated heading (H′)
For the wall placement task, we calculated the participants’ estimated heading on the testing wall relative to North on the ground using the least square of angular discrepancy. For the sake of composition, we calculated the heading as if the participants were facing the direction of gravity Up. For every two objects (A and B, see Figure 3) of the learned layout, we calculated the bearing between the replaced locations of those objects on the testing wall (A′ and B′) relative to the direction of Up on the testing wall. As there were seven objects, there were 21 possible pairs of objects, which makes 21 estimated bearings in total. Similarly, we calculated the bearing between the correct locations of the objects on the ground relative to any possible reference direction on the ground (we refer to this bearing as the correct bearing). For each corresponding pair of objects (e.g., A and B), we calculated the bearing error, which is the difference between the response bearing (the bearing of A′B′ relative to the Up of the testing wall) and the correct bearing (the bearing of AB relative to the hypothetical reference direction on the ground). The hypothetical reference direction that leads to the least square of the bearing error across all 21 bearings is defined as the heading of Up on the testing wall relative to North on the ground. We refer to this estimated heading as H′. If participants treated the heading of facing Up as North on the floor, H′ would be close to North according to this method.
Correspondence between the estimated heading and the actual direction. A and B denote the actual positions of two objects. A′ and B′ denote the response positions of the two objects. H denotes an allocentric heading on the testing plane (e.g., in Experiment 1, Up on the wall). H′ denotes the estimated heading corresponding to H (e.g., in Experiment 1, North on the ground). The vector A′B′ relative to H is equal to the vector AB relative to H′.
To confirm the validity of this method, we also calculated the estimated heading using participants’ responses in the learning phase and ground testing phase. For the sake of composition, H′ for the learning phase and ground testing phase was presented as if participants were facing North. In particular, we calculated the bearing between the replaced locations of any two objects (A′ and B′) relative to the direction of north rather than participants’ physical facing direction. If this method works well, H′ during the learning phase and ground testing phase should be very close to North. In addition, the finding that H′ corresponding to the facing direction of North in the ground testing phase was close to North would also support that participants updated their headings while navigating on the ground. To specify H′, we use North as the direction of 0°.
Results and discussion
Placement test results
We calculated the estimated heading (H′) as described above for each participant and for the responses in each phase. Representative responses are plotted in Figures S2 and S3. The results were plotted in Figure 4 with the circular mean and 95% confidence intervals (see Table 4) for each phase. In the learning phases and the ground test, the group mean of H′ was very close to 0° (i.e., North on the ground), which indicates that the participants successfully reconstructed the object array from their learning heading and updated their headings of facing the testing direction in the ground test. These results also indicate that the methods of calculating H′ are valid and participants updated their headings when navigating to the viewing position of the ground test.
Estimated heading (H′) and angular difference in Experiment 1 (placement task on the wall).
Summary of 95% circular confidence intervals of H′ in the placement task.
N/A: not applicable.
Rayleigh tests were conducted for each wall test. The results showed that in both tests, the H′s were not uniformly distributed (ps < .001). However, in both wall tests, there was some variance among the participants. This result could have occurred because different participants might have selected the different reference directions using egocentric, allocentric, or intrinsic cues to encode the object array (e.g., Kelly & McNamara, 2008) but each individual participant always mapped the Up on the testing wall to the reference direction in encoding the object array.
To test this possibility, we examined the angular difference between the H′s in the north wall test and the south wall test for each participant. We also plotted the angular difference in Figure 4 (N-S difference). We found that for most of the participants, the angular difference between H′s in the north and south wall tests was around 0° (angle range [0°, 45°] or [315°, 360°]) whereas for the rest, the angular difference was around 180° (angle range [135°, 225°]). Clearly, most of the participants had the same or similar H′ in both wall tests. This type of response was found in 42 participants (87.5%). These participants considered Up on the north and south walls to be the same or a similar direction on the ground, which indicates that they did not have 3D extended headings on the testing planes. This contradicts the extended-heading hypothesis. These participants were categorised as “same-up” (see the cyan dots in Figure 4). We also summarised the percentages of different categories in Table 5.
Summary of the frequencies and percentages of different categories in the placement task.
Note: The top row of the table shows the results in Experiment 1. The bottom row shows the results in Experiment 3. The numbers in the parentheses show the percentage of the frequency out of the total number of participants in that group.
Several of the rest of the participants showed a pattern of extending their headings (see the purple dots in Figure 4) according to the extended-heading hypothesis. Specifically, according to their replaced locations, Up on the north wall was close to North on the ground (i.e., H′s were around 0°; angle range [0°, 45°] or [315°, 360°]) and Up on the south wall was close to South on the ground (i.e., H′s were around 180°; angle range [135°, 225°]). This type of response was found in three participants (6.25%). These participants were categorised as “extension.”
There were three participants (6.25%) whose responses could not fit into any of the two categories. These participants were categorised as “other” (see the orange dots in Figure 4).
JRD test results
For the JRD tests, we calculated the averaged absolute pointing error and the response latency for each participant and then calculated the mean for the whole group. The mean and standard deviation for the absolute pointing error and the response latency were shown in Table 6. In all experiments, the absolute pointing error showed the same general pattern as the response latency but was noisier for significance tests. There was no evidence of speed-accuracy trade-offs. For brevity, here we only report the results from the statistical tests using response latency as the dependent variable.
Mean and standard deviation for absolute pointing error and response latency in the JRD task in Experiment 1.
JRD: Judgements of Relative Direction; SD: standard deviation.
Response latency as a function of LI (0°, 180°) and AI (0°, 180°) is plotted in Figure 5. A 2 × 2 repeated-measures analysis of variance (ANOVA; LI [0°, 180°] × AI [0°, 180°]) was conducted. The main effect of AI was significant, F(1, 47) = 4.61, p = .037, partial η2 = .09. Imagining the directions aligned with the actual heading was faster than imagining misaligned directions, which indicates a sensorimotor alignment effect. The main effect of LI was also significant, F(1, 47) = 4.69, p = .035, partial η2 = .09. Imagining North was faster than imagining South, which indicates a memory effect. The interaction effect was not significant, F(1, 47) = 0.05, p = .830, partial η2 < .01. These results indicate that overall, participants extended their headings on the ground to the wall by a physical pitch rotation of 90° so that their headings of facing Up on the testing wall were the same as the heading as indicated by the traversed route from the ground to that wall, producing the sensorimotor alignment effect.
Response latency in Experiment 1 (JRD task on the wall).
Experiment 1 indicates that human adults can extend their headings from ground to vertical walls when performing physical pitch rotations. As the participants could not inhibit the sensorimotor representations (Farrell & Robertson, 1998), the sensorimotor alignment effect in the JRD task provides direct evidence in favour of the extended-heading hypothesis over the non-integration hypothesis. In the placement task, although not instructed to do so, some participants spontaneously reproduced the object array in the “extended-heading” manner while majority of participants placed the same direction of the array on the upsides of both walls (the “same-up” category).
To interpret the differences between the placement results and the JRD results, it seems to be intuitive to say the placement task miscategorised participants into both the “extension” and the “same-up” groups. However, we think the “extended heading” participants did extend their headings. There was no “false alarm” in this category. In the placement task, the participants were allowed to replace the objects to anywhere they wanted to on the mat and arrange the array in any orientation. If they did not have the extended-heading representation, it would be impossible to respond in the “extension” way (Figure S3) as there was an infinite number of ways to complete the task. Therefore, it is not unwarranted to conclude that at least some participants can extend their headings from the floor to the walls spontaneously.
However, some of the “same-up” participants may also have done that. They were not in the “extension” category due to our strict criteria for “extended heading.” First, there has to be a clear angular difference (180°) between the two calculated H′ in the two wall tests. Some participants might have decided to reconstruct the array in one orientation when they were on one wall, but when on the other wall, they might have simply retrieved the immediate visual memory of the reconstructed array due to inertia (as they were performing the same task for the second time). Second, the criteria also ask for H′ on the north wall around 0° and H′ on the south wall around 180°. However, to reproduce the object array, the participants only need to retrieve spatial memories of the array from a preferred orientation encoded during learning, which may or may not be the same as the reference directions (the North/South on the ground). As shown in Figure 4, some of the cyan dots were not around 0 or 180 in the individual North/South wall tests, which indicates the variety of choosing preferred directions. Furthermore, anecdotally, when the participants were asked to report how they solved the wall placement task after the experiment, they reported that they hesitated about how to place the objects because they thought there were a few possible, different orientations that could be used to arrange the array on the walls. It is possible that they had the extended-heading representations but did not use them in the placement task.
Therefore, the placement task is an effective tool for identifying people who extended their headings spontaneously but not for identifying people who did extend their headings but chose other strategies for the placement task. As a result, the placement task might have missed a lot of participants who extended their headings. Therefore, the finding that majority of participants placed the same direction of the array on the upsides of both walls (“same-up”) is less likely to be used to support the non-integrated hypothesis.
As the individual difference in the heading extending ability is not the focus of this study, we present the results on the spatial transformation tests in the supplementary section. To briefly sum up the results, there was no significant correlation between the spatial tests performance and the placement or the JRD task performance.
Experiment 2
In Experiment 2, we used the JRD task to further examine whether participants could extend their headings when they locomoted from the floor to the walls only using visual pitch rotations instead of physical pitch rotations. Previous research suggested that spatial updating of headings with physical body rotations may differ from spatial updating with rotations indicated by visual optic flow (Klatzky et al., 1998). Animal research also showed that active physical pitch rotations are more likely to lead to extension of headings than passive rotations (Taube et al., 2013), which suggests that visual cues alone seem insufficient for extending headings. Experiment 2, together with Experiment 1, examined the critical role of idiothetic cues of body rotations in obligatory updating of headings during 3D navigation (i.e., pitch rotations).
Method
Participants
A total of 48 university students participated in the experiment (age range = 17–28 years, M = 19.33 years) with an equal number in each gender. One additional participant dropped out during the experiment due to VR sickness. All participants received course credits in an introductory psychology course for their participation.
Apparatus and environment
The apparatuses were exactly the same as in Experiment 1 with the following exceptions. First, to translate in the virtual environment, the participants pushed the forward button of a wireless controller (Oculus Touch, Oculus VR, LLC., Irvine, CA). Second, the joystick (Logitech ATK3) was only used for making a response in the JRD task. Third, an office swivel chair was used instead of the swivel recliner. The experimental environment was exactly the same as Experiment 1.
Materials, design, and procedure
There are several differences between this experiment and Experiment 1. First, only visual pitch rotation but no physical rotation was involved in Experiment 2. The participants maintained an upright perspective when navigating on the ground. However, whenever they reached the boundary between two planes (e.g., the ground and the wall, or the wall and the ceiling), by pushing the joystick, their perspective was rotated 90° upwards (pitch up; see supplementary material for a rotation video) and moved onto the place they were looking at. For example, when they reached a wall, their perspective was rotated 90° (pitch up) as if they were stepping on the wall and their body was perpendicular to the wall. When they navigated from the wall to the ground, their perspective was also rotated 90° (pitch down) so that they could return to an upright perspective when navigating on the ground. Second, only the JRD tests on the testing walls were used. The order of the two JRD tests on different walls was counterbalanced across participants. Third, no spatial ability tests were conducted.
Results and discussion
As in Experiment 1, for the responses in each placement phase (learning and the ground test), we calculated the estimated heading (H′). The results were plotted in Figure 6 with the circular mean and 95% confidence intervals (see Table 4) for each phase. The mean of H′ was around 0°(i.e., North on the ground), which indicates that the participants successfully reconstructed the object array from their learning heading and updated their headings of facing the testing direction on the ground test.
Estimated heading (H′) and angular difference in Experiment 2 (placement task).
For the JRD tests, the mean and standard deviations of absolute pointing error and those of response latency are shown in Table 7. Response latency as a function of LI (0°, 180°) and AI (0°, 180°) is plotted in Figure 7. A 2 × 2 repeated-measures ANOVA (LI [0°, 180°] × AI [0°, 180°]) was conducted. There was no significant main effects or interaction (ps > .43, partial η2 < .02).
Mean and standard deviation for absolute pointing error and response latency in the JRD task in Experiment 2.
JRD: Judgements of Relative Direction; SD: standard deviation.
Response latency in Experiment 2 (JRD task on the wall).
We did not find the sensorimotor alignment effect in this experiment. Together with the sensorimotor effect reported in Experiment 1, this result indicates that visual cues were not sufficient for extending their headings via pitch rotations. Participants in Experiment 2, while performing a pitch rotation between two planes, only perceived their rotation from the visual cues. Previous research has found that idiothetic cues (e.g., proprioceptive and vestibular systems) differ from optic flow in spatial updating (Chrastil & Warren, 2013; Klatzky et al., 1998; Rieser, 1989; Ruddle et al., 2011; Taube et al., 2013; Waller et al., 2004; Zhang & Mou, 2019; but see Riecke et al., 2007). The current results are consistent with this point.
Experiment 3
The purpose of Experiment 3 was to further test the extended-heading hypothesis when participants navigated from the ground to an inverted horizontal plane (i.e., ceiling). Finkelstein et al. (2015) showed that bats can extend their heading after 180° pitch rotations (e.g., a head direction cell sensitive to north for an upright position fired when the inverted bat faced south), treating the inverted plane as an extension of the heading on the ground by pitch rotations. The extension hypothesis would be much strengthened if we could demonstrate that people could also extend their heading after 180° pitch rotations from the ground to the ceiling.
As in Experiment 1, we tested the same participants with both the placement and JRD tasks. Participants navigated to the ceiling via the north or south wall and then did JRD task and replaced objects on the ceiling from two opposite facing directions (South or North). As summarised in Table 1, the placement responses on the ceiling would be different across participants who had an extended heading (extended-heading hypothesis), and those who merely retrieved their preferred heading encoded on the ground regardless of their facing direction (non-integration hypothesis).
We acknowledge that some participants who had an extended heading may have been miscategorised to other groups by the placement task in Experiment 1. We still used the placement task in Experiment 3 for the following reasons.
It is not practical to let participants physically pitch 180° in our lab due to the safety concerns, so Experiment 3 still used visual pitch rotations as in Experiment 2. Although visual navigation from the floor to the ceiling may create more unusual (in real life), greater visual changes than from the floor to the walls, according to the results in previous experiments, the JRD task may not show the sensorimotor alignment effect for the whole group of participants with visual rotation alone. The JRD task alone may not reveal clear evidence of extended headings. Using the placement task can identify people who spontaneously extend their headings. With a larger sample size than Experiment 1, it would be possible to get a sample of participant in the “extension” category. Together with the JRD task, it would be possible to test whether these participants showed the sensorimotor alignment effect.
Thus, the convergent evidence from both tasks may be able to reveal clear evidence of extended headings. We first used the placement task to categorise participants and then used the JRD task to examine whether each category of participants could show any sensorimotor alignment effect. We are especially interested in examining the sensorimotor alignment effect for the “extension” group categorised by the placement task. Participants who used extended 3D headings in placement task should have extended their headings although we could not tell whether or not other groups had extended their headings. According to the extended-heading hypothesis, the “extension” participants would show a reversed sensorimotor alignment effect. In particular, the performance would be better when the imagined heading (e.g., South) was the same as the extended heading but opposite to the facing direction (e.g., an extended heading of south when facing north on the ceiling) than when the imagined heading was opposite to the extended heading but the same as the facing direction (e.g., extended heading of north when facing south on the ceiling). The extended-heading hypothesis will be disconfirmed if there lacks evidence of extension groups in the placement task and lacks evidence of a reversed sensorimotor alignment effect in the JRD tasks (Table 1).
Method
Participants
A total of 80 university students participated in the experiment (age range = 17–60 years, M = 20.19 years) with an equal number in each gender. Two additional participants dropped out during the experiment due to VR sickness. All participants received course credits in an introductory psychology course for their participation. The sample size was about twice that of each of the previous experiments because we intended to increase the number of participants in each category to examine the sensorimotor alignment effect in the JRD task for each category.
Apparatus and environment
They were exactly the same as in Experiment 2.
Materials, design, and procedure
They were the same as Experiment 1 with the following exceptions. First, the participants performed visual pitch rotations when navigating between the ground and the ceiling. The rotation effect was the same as in Experiment 2. Second, the wall placement and JRD tests were replaced by the ceiling placement and JRD tests. After the ground testing phase, the mat was moved to the centre of the ceiling. On each ceiling test, the participants navigated to the platform at the north or south of the mat via the north or south wall (which was closer to the platform) and faced the mat and then did the tasks. Before the formal experiment, participants practised using the controller to navigate on the walls horizontally and vertically and also navigated on the ceiling. Finally, no spatial ability tests were conducted.
Results and discussion
Placement test results
For the placement tests, we calculated the estimated heading (H′) from the responses in each placement phase when the participants were facing a direction on the testing plane relative to North of the ground. In the learning and ground tests, the estimated heading still corresponded to the facing direction of North. Thus, estimated headings should have been close to North as participants accurately encoded and updated their headings on the ground. In the ceiling tests, the estimated heading corresponded to the participants’ actual facing direction. In particular, when participants faced South, we calculated the estimated heading of facing South. Thus, if participants extended their headings, then the estimated heading of facing South would be close to North on the ground (0°) or participants placed north side of the original layout on the south side of the ceiling. If participants used the facing direction, then the estimated heading of facing South would be close to South on the ground (180°) or participants placed south side of the original layout on the south side of the ceiling. If participants used the preferred heading regardless of the facing direction, then the estimated heading of facing South would be similar to that of facing North in the ceiling test (Table 1).
Representative responses are plotted in Figures S4, S5, and S6. The results were plotted in Figure 8 with the circular mean and 95% confidence intervals (see Table 4) for each phase. In the learning phases and the ground test, the mean of H′ was very close to 0°(i.e., North on the ground), which indicates that the participants successfully reconstructed the object array from their learning heading and updated their headings of facing the testing direction on the ground test. Rayleigh tests were conducted for each ceiling test. The results showed that in both tests, the H′s were not uniformly distributed (ps < .007).
Estimated heading (H′) and angular difference in Experiment 3 (placement task on the ceiling).
Again, we examined the angular difference between H′s in the ceiling-facing-north test and the ceiling-facing-south test for each participant and then categorised participants who showed different types of responses (see Table 5 and Figure 8).
We found that some participants had the same or similar H′s in both tests (i.e., the angular differences were around 0°; angle range [0°, 45°] or [315°, 360°]). In other words, from the participants’ egocentric view, their responses were basically the same, regardless of their actual facing directions. This type of response was found in 32 participants (40%). These participants did not use the environmental cues or their movements from the floor to the ceiling and instead likely just retrieved a fixed, preferred heading on the ground from their memories. They were categorised as “same-front” (see the cyan dots in Figure 8). The “same-front” participants were similar to those in the “same-up” category in Experiment 1.
For the rest of participants, we further categorised those who showed that the angular difference of H′ in the two ceiling tests was around 180° (angle range [135°, 225°]). Some participants’ responses indicate that facing North on the ceiling was equal to facing South on the ground (i.e., H′s were around 180°; angle range [135°, 225°]) and that facing South on the ceiling was equal to facing North on the ground (i.e., H′s were around 0°; angle range [0°, 45°] or [315°, 360°]). This type of response was found in 17 participants (21.25%). Such participants could have used the pitch rotations to extend their headings on the floor to the ceiling. Therefore, the participants who showed this type of response were categorised as “extension” (see the purple dots in Figure 8).
All other 31 participants (38.75%) were categorised as “other”; that is, those whose response could not fit into any of the two categories (see the orange dots in Figure 8). However, we found some interesting systematic patterns in their responses. For 22 of these “other” participants, their responses indicate that facing North on the ceiling equalled facing North on the ground (i.e., H′s were around 0°; angle range [0°, 45°] or [315°, 360°]) and facing South on the ceiling equalled facing South on the ground (i.e., H′s were around 180°; angle range [135°, 225°]). Note that such a pattern of responses is different from those of the “extended heading” participants. Clearly, these 22 participants could distinguish North and South on the ceiling so that these participants did not directly retrieve a preferred heading from their memories. They might have used environmental cues (e.g., furniture) to find reference directions (e.g., North) to get reoriented but they seemed to ignore their inverted perspective (i.e., upside down) and the pitch rotations.
JRD test results
For the JRD tests, we calculated the averaged absolute pointing error and the response latency for each participant and then calculated the mean for the whole group. The mean and standard deviation for the absolute pointing error and the response latency were shown in Table 8.
Mean and standard deviation for absolute pointing error and response latency in the JRD task in Experiment 3.
JRD: Judgements of Relative Direction; SD: standard deviation.
Response latency as a function of LI (0°, 180°) and AI (0°, 180°) is plotted in Figure 9. A 2 × 2 repeated-measures ANOVA (LI [0°, 180°] × AI [0°, 180°]) was conducted. The main effect of LI was significant, F(1, 79) = 5.63, p = .020, partial η2 = .07. Imagining North was faster than imagining South regardless of the physical facing direction, which indicates a memory effect. There was no significant main effect of AI or the interaction (ps > .15, partial η2 < .03).
Response latency in Experiment 3 (JRD task on the ceiling). (a) The results for all the participants. (b) and (c) The results for each category of participants according to the placement task results.
As shown above, we found two categories of participants (extension and same-front) according to the responses in the placement task. For each category, we conducted a 2 × 2 repeated-measures ANOVA (LI [0°, 180°] × AI [0°, 180°]) with response latency as the dependent variable. We are particularly interested in the “extension” participants.
For the “extension” participants, the main effect of AI was significant, F(1, 16) = 8.25, p = .011, partial η2 = .34 (large effect). The participants responded significantly faster when their physical facing direction on the ceiling was opposite to the imagined heading on the ground than when their physical facing direction was the same as the imagined heading, which shows a reversed sensorimotor alignment effect. This indicates that participants extended their headings on the ground to the ceiling by pitch rotations of 180° so that their headings of facing the same direction on the ceiling and the ground were opposite, producing the reversed sensorimotor alignment effect. The main effect of LI was not significant, F(1, 16) = 0.99, p = .335, partial η2 = .06. The interaction effect was not significant, F(1, 16) = 3.32, p = .087, partial η2 = .17.
In addition, for the “same-front” participants, the main effect of LI was significant, F(1, 31) = 7.19, p = .012, partial η2 = .19. Imagining north was faster than imagining south regardless of the physical facing direction, which indicates a memory effect. The main effect of AI was not significant, F(1, 31) = 0.97, p = .332, partial η2 = .03, which indicates no sensorimotor alignment effect. The interaction was not significant, F(1, 31) = 1.59, p = .217, partial η2 = .05. These results indicate that the “same-front” participants directly retrieved a fixed heading from memory they acquired from the ground phases.
For the 22 participants from the “other” category who used environmental cues to find reference directions, there was no significant main effect or interaction (ps > .23, partial η2 < .07). Therefore, there was no sensorimotor alignment effect for these participants. This result, combining with previous placement results, further reveals that these 22 participants did not fit “extended-heading” hypothesis according to the null sensorimotor alignment effect. They were more likely to reorient themselves using room cues after the pitch rotations rather than updating their headings as the “extended heading” participants did.
Resampling simulation on JRD test results
To further consolidate the validity of the placement categorisation method, we analysed our data using a resampling simulation method. We randomly sampled 17 participants from all 80 participants without replacement and conducted the same ANOVA for the JRD response latencies for the selected 17 participants. We repeated this process for 1,000 times. Here, we are most interested in whether there was a true main effect of AI (the sensorimotor alignment effect). If our placement categorisation method was effective, then in 1,000 times, there would be a very small percentage of samples that would show a significant main effect of AI. If our placement categorisation method was not effective, and any random sample from the 80 participants could lead to a significant main effect of AI, then there would be a large percentage of samples that would show a significant main effect of AI.
The results showed that there were only 43 times (4.3%) that the main effect of AI was significant (p < .05) and 23 times (2.3%) that partial η2 was larger than .25 (large effect). These results suggest that the significant main effect of AI in the observed data (p = .011, partial η2 = .34) was a very rare case and the extension category therefore was reliable.
In Experiment 3, we found convergent results from the placement and the JRD tasks regarding the extended headings. Participants in the “extension” category of the placement task restored the headings on the ground via mental pitch rotations, producing the reversed sensorimotor alignment effect. However, most randomly sampled 17 of the 80 participants did not show this effect. The simulation analysis indicates that the placement categorisation was effective in identifying “extended heading” participants.
General discussion
This study investigated how humans update headings in 3D navigation on different surfaces with pitch rotations. Inspired by the literature of human spatial memory on the horizontal plane and animal studies in 3D spaces, we tested two hypotheses, the extended-heading hypothesis and the non-integration hypothesis (see Table 1). We found the first evidence that people extend their headings via physical pitch rotations from the floor to the walls; we also found partial evidence of extended heading via visual pitch rotations from the floor to the ceiling. We discuss the theoretical significances and the methodological implications of these findings as follows.
Humans can extend their 3D headings, especially with idiothetic cues
The most important finding in this study is that humans can extend their 3D headings from the ground to the vertical walls, as indicated by the sensorimotor alignment effect in Experiment 1. This finding is consistent with the prediction of the extended-heading hypothesis but inconsistent with the non-integration hypothesis. This finding is the first demonstration in the literature that humans, to some degree, encode different surfaces in the same 3D space in a global manner. When people navigate across surfaces (from the floor to the walls), they also update their headings relative to other surfaces. If participants had not encoded spatial relations between different surfaces, they would have treated the testing surfaces and the learning surfaces as two separate or irrelevant spaces, producing no sensorimotor alignment effect in the JRD tasks.
One may argue that this finding is not that surprising as participants could visually perceive spatial relations between different surfaces during navigation. However, people do not always encode information that is directly visible to them as indicated by the literature of change blindness (Henderson & Hollingworth, 1999; Simons & Levin, 1997). In Experiment 2 of this study, participants who were presented with the same visual information as in the Experiment 1 did not extend their headings on the floor to the walls. Therefore, the idiothetic cues of physical pitch rotations might play critical roles in associating spatial memories of different surfaces into the same global representation, supporting the sensorimotor alignment effect. Without physical rotation, people might just treat different surfaces as separate spaces, showing no sensorimotor alignment effect. This speculation is consistent with the theoretical view stipulating that path integration plays a critical role in forming global spatial memories of a large-scale environment (Gallistel, 1990) but extends this theoretical view to 3D spaces. Physical rotations involve multiple modalities (vision, proprioception, etc.), which seems to provide more channels to encode more information of the movements. Similar to horizontal movements, multiple sources of perceptual inputs usually lead to more accurate representations (e.g., Chrastil & Warren, 2013; Klatzky et al., 1998; Rieser, 1989; Ruddle et al., 2011; Waller et al., 2004; Zhang & Mou, 2019).
Note that extended heading from one surface to the other surface is not a characteristic of a true 3D spatial representation and updating. Extending their headings from the floor to the wall, participants would act as if they only performed translations on the same surface and ignore their rotation across surfaces, causing the sensorimotor effect. In contrast, for participants who have a true 3D spatial representation and update their true 3D representations, the vertical surface is orthogonal to both directions on the floor, so there will be no sensorimotor alignment effect in the JRD tests. Furthermore, participants with a true 3D representation would be more likely to update their 3D representations with physical rotations than with visual rotations only because of the additional idiothetic cues in the physical rotations. Therefore, according to this true 3D possibility, participants with physical rotations will be less likely to show extended heading than those with visual rotations. But our results indicate the opposite. Although this study was designed to test whether people can extend their headings instead of separately encoding different surfaces, it also provides some preliminary evidence against a theory of true 3D spatial representations and updating. Further research should more precisely conceptualise and test other models of 3D spatial representations and updating, differentiating from the model of the extending heading.
As we discussed in the Introduction, humans primarily navigate on the horizontal surface without involving many pitch rotations. Participants in Experiment 1 did not have much 3D navigation experience from the floor to walls via pitch rotations. They only had a brief practice of moving from the floor to each wall before the real experiment. We propose two reasons why humans can extend their 3D headings as do rats (Calton & Taube, 2005; Stackman et al., 2000; Taube et al., 2013) and bats (Finkelstein et al., 2015) although humans have much less 3D navigation experiences involving a pitch rotation of a large angle than rats and bats. First, although humans as surface-based animals most times navigate on the horizontal plane, humans still live in 3D spaces. In ancient times, humans may spend much more time navigating three-dimensionally for survival (e.g., forage on the tree). Sometimes humans perform 3D physical activities such as scuba diving and rock climbing. Second, people still quite often pitch their bodies, for example, lying down for sleep or getting up from bed, although this pitch rotation is not navigation relevant. Such pitch rotations, however, might share the same cognitive and neural mechanisms as the pitch rotations used in navigation. Hence, even with minimal experience of using idiothetic cues from pitch rotations, participants could extend their 3D headings as do rats and bats.
Importantly, idiothetic cues are also critical to 3D heading extension of rats. When passively moved from the horizontal floor to the vertical walls, the rats did not extend their headings, whereas when actively locomoted to the walls, they did (Taube et al., 2013). Although different neural and cognitive mechanisms might be developed to accommodate the natural locomotion style for different species, there are also common neural and cognitive mechanisms across species as all species are living in the same 3D spaces. One should also note that this study cannot conclude humans have the same spatial representation mechanisms as rats. For example, rats navigate along a vertical surface after pitch rotations more often than humans do but rats reply less on vision than humans. Updating their headings after visual pitch rotations task in this study may be even more difficult to rats. Given many aspects of differences across species, future research should design comparable tasks for humans and non-human mammals to further investigate the similarities and differences in representations across species in 3D navigation.
In Experiment 3, using both JRD and placement tasks, we intended to find some evidence of extended headings when participants only could perform visual pitch rotations from the floor to the ceiling. This experiment did not use physical pitch rotations due to the lack of some safe equipment for physically rotating a participant’s body 180°. The results showed a sensorimotor alignment effect for participants who extended their headings in the placement task. These results indicate that some participants extended their headings from the ground to the ceiling via a 180° pitch rotation indicated by visual cues. Therefore, at least some participants can extend their 3D headings from ground to the ceiling even only using visual pitch rotations. Although we did not include both tasks in Experiment 2, we would predict that if we did that, the placement task could select the “extended heading” participants out for visual rotations of 90° and they would be able to show a sensorimotor alignment effect in the JRD task.
We note that results across all participants in Experiment 3 did not show any sensorimotor alignment effect. We do not attribute this result to the speculation that humans cannot update their 3D headings in pitch rotations more than 90° (Calton & Taube, 2005). In Experiment 2, without idiothetic cues, participants did not show sensorimotor alignment effect even for 90° rotations. Instead, the cause may be lack of the idiothetic cues in Experiment 3. Future research should systematically investigate this issue using some special equipment in which 180° physical pitch is safe to reach decisive conclusions.
This study also investigated the origins of the individual differences in updating 3D headings with the spatial transformation ability tests and individual history (see supplementary material). However, we did not find any correlation between these measures and the responses in the placement task or those in the JRD task. There was also no gender difference in either JRD or placement tasks in any of the experiments. We speculate that individuals’ physical experience of pitch rotations might contribute to the individual differences observed in this study, as training can improve spatial abilities (Feng et al., 2007). For example, participants who practised gymnastics might have more experiences in pitch rotations of their body. However, the participants in this study were university students and most of them did not have such training experience. Future research may test some athletes, jet pilots, and astronauts who do a lot of training with physical pitch and roll rotations. Such research might examine whether these people more likely extend their 3D heading from the ground to walls and the ceiling using the experimental paradigm of this study. We may also test whether human infants more likely update their headings of yaw rotation before and after they start crawling, which might be a crucial time point for the development of spatial ability (Crowther et al., 2000).
One further interesting question is whether humans can update headings via roll rotations (i.e., rotations along the front-back body axis) in self-locomotion. Intuitively, updating heading via roll rotations might be even harder than via pitch for humans as humans usually do not have such experience in daily activities. One study suggests that monkeys may have head direction cells that are tuned to roll rotations (Laurens et al., 2016). It is likely that humans also have head direction cells that are sensitive to roll rotations. Searching behavioural evidence of updating headings in roll rotations could further determine whether daily pitch rotations like lying down for sleep is critical to extending heading via pitch rotations.
Our findings suggest that humans can integrate spatial representations across boundaries at least in some certain scenarios (like in this study), although it might be difficult for some people. Encoding different surfaces in a 3D room is possible because people can directly see at least some spatial relations across boundaries (Mou & Wang, 2015). This helps people to more easily encode the spatial relations between the surfaces than those between multiple rooms on the floor. Previous research proposed that encoding spaces across boundaries may impact efficiency of navigation because there is redundant information (Wang, 2004, 2017). But in our study, encoding different surfaces in a 3D room was useful for navigation in that room. Our findings seem consistent with the efficiency view.
Methodologies of the placement task
Aiming to examine human 3D heading extending using behavioural measures, we used JRD tasks (examining sensorimotor alignment effects), which has been widely used to examine heading updating on the horizontal plane (e.g., Rieser, 1989). In addition to JRD tasks, this study developed a placement task to measure the heading before and after moving to another plane. We think the placement task is informative and can be used in future research. The calculated estimated heading (H′) from the placement task indicates a specific direction shown in the range of [0, 360], which is a continuous dependent variable. Therefore, it reflects more than the categorised variable in the JRD task (aligned or misaligned direction with the learning direction or with the current direction).
This study shows that the placement task is effective, especially for measuring the heading updating on the same surface (i.e., ground). In all experiments, in the ground placement test, the estimated heading (H′) clearly indicated that the participants updated their headings on the horizontal ground. In particular, they could accurately recover the reference direction that had been used to encode objects’ locations (e.g., north) to reconstruct the objects layout accurately from a new perspective in the ground tests. For most participants, the estimated headings (H′) corresponding to facing north in the ground test were very close to the north direction on the ground (Figures 4, 6, and 8). This clearly indicates that participants updated their headings when they turned their bodies (i.e., yaw) on the ground as shown in previous research (e.g., Kelly et al., 2007). In other words, when people only move on the same plane with only yaw rotations, the placement task can accurately measure whether people update their 2D headings or not and can indicate their estimated headings after the movements.
We acknowledge that, for measuring the heading after movements between different planes (i.e., in 3D navigation), the placement task may underestimate the number of extended-heading people or the “extension” category. In Experiment 1, we found only 6% of the participants were in the “extension” category according to the placement task results whereas overall participants showed an extended sensorimotor alignment effect in the JRD task. As we discussed in the Introduction, the placement task measures how people spontaneously choose an orientation (the learning heading or the updated heading) to reproduce the layout, whereas the JRD task directly requires people to use the updated heading on that plane (e.g., Farrell & Robertson, 1998). The placement task can identify extended-heading people, but with very strict criteria, it will miss some people and miscategorise them. For measuring the updated heading in 3D navigation, the placement task might be conservative. Therefore, in future research, this task should be used with other tasks independent of participants’ strategies (e.g., the JRD task in the current study) to get converging evidence about extended headings.
Conclusion
As an early behavioural study to examine human spatial memories and updating in 3D navigation, this study raised more questions than answered. However, this study answered the main questions raised in the Introduction. The findings of this study indicate that human adults can extend their headings from the floor to the walls while locomoting from the floor to the walls involving physical pitch rotations. In addition, some but not all human adults can extend their headings from the floor to the ceiling while locomoting from the floor to the ceiling involving only visual rotations. These findings suggest that people can develop 3D spatial memories during 3D navigation involving both yaw and pitch rotations.
Supplemental Material
sj-docx-1-qjp-10.1177_1747021820978973 – Supplemental material for Updating headings in 3D navigation
Supplemental material, sj-docx-1-qjp-10.1177_1747021820978973 for Updating headings in 3D navigation by Yu Karen Du, Weimin Mou and Xuehui Lei in Quarterly Journal of Experimental Psychology
Footnotes
Acknowledgements
We thank Eugini Ann Tolentino, Ojas Srivastava, Lydia Jiang, and Rachel Mustaklem for their contributions to our data collection.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by Natural Sciences and Engineering Research Council of Canada to W.M.
Compliance with ethical standards
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
