Complexity of postural sway affects affordance perception of reachability in virtual reality

Abstract

Visual perception of whether an object is within reach while standing in different postures was investigated. Participants viewed a three-dimensional (3D) virtual reality (VR) environment with a stimulus object (red ball) placed at different egocentric distances. Participants reported whether the object was reachable while in a standard pose as well as in two separate active balance poses (yoga tree pose and toe-to-heel pose). Feedback on accuracy was not provided, and participants were not allowed to attempt to reach. Response time, affordance judgements (reachable and not reachable), and head movements were recorded on each trial. Consistent with recent research on perception of reaching ability, the perceived boundary occurred at approximately 120% of arm length, indicating overestimation of perceived reaching ability. Response times increased with distance, and were shortest for the most difficult pose—the yoga tree pose. Head movement amplitude increased with increases in balance demands. Unexpectedly, the coefficient of variation was comparable in the two active balance poses, and was more extreme in the standard control pose for the shortest and longest distances. More complex descriptors of postural sway (i.e., effort-to-compress) were predictive of perception while in the tree pose and the toe-to-heel pose, as compared with control stance. This demonstrates that standard measures of central tendency are not sufficient for describing multiscale interactions of postural dynamics in functional tasks.

Keywords

Perception affordance posture virtual reality complexity

The hallmark of efficacious behaviour is the ability to perceive and act on opportunities for action termed affordances (Gibson, 1979). In most everyday settings, multiple affordances are simultaneously available and performing a given behaviour requires perceiving (and acting upon) multiple affordances. For example, if a person intends to pick up an object from a table, that person has to perceive the nested behaviours that must be performed such that the goal can be achieved. That is, the person must perceive how the present state of affairs must be modified to bring about the intended state of affairs (Wagman, 2012). For example, the person must (simultaneously) perceive whether (and how) it would be necessary to change body position, arm extension, and grasp width to successfully pick up the object. Typically, leaning and reaching are behaviours that bring effectors into the vicinity of a target object, to allow for subsequent (or simultaneous) grasping. That is, affordances for grasping are (typically) nested within affordances for leaning and reaching.

The multiple affordances available to a given person at any given moment are nested across multiple levels of a means–ends hierarchy (Vicente & Rasmussen, 1990; Wagman, Caputo, & Stoffregen, 2016; see Ye et al., 2009). Higher levels of the hierarchy function as ends, or goals, whereas lower levels function as means to those ends. At every level of the hierarchy, affordances are simultaneously constrained by lower levels (i.e., by lower order affordances or means of performing a given behaviour) but also by higher levels (i.e., higher order affordances or goals for performing a given behaviour). In the context of reaching for an object, affordances for reaching are nested within affordances for maintaining (or changing) posture (i.e., how the reaching is to occur) and the goal of reaching (i.e., why the reaching is occurring; Wagman et al., 2018).

Postural control can facilitate performance in suprapostural tasks such as focusing attention (or behaviour) on a target object (Balasubramaniam et al., 2000; Stoffregen et al., 1999, 2000). Postural control can also maintain stability and support perceptual performance under dynamically changing environmental constraints (Stoffregen et al., 2009; Walter et al., 2017, 2019). Such studies have clearly demonstrated that postural demands influence affordance perception. However, the nature of the relationship is unclear. Posture exhibits dynamical stability through continuous body sway. Such minute fluctuations of the body’s centre of mass cause corresponding complex, but subtle head movements. These head movements generate subtle optic flow and gravito-inertial patterns. In principle, the patterning of information in the global ambient energy array generated by postural sway ought to be informative for affordance judgements (Stoffregen et al., 2017; Y. Yu et al., 2010). How can this pattern be described? In a virtual reality (VR) study, Mantel et al. (2015) demonstrated that perception of affordances for reaching is specific to higher-order patterns of ambient energy arrays that include optical and gravito-inertial patterns. Importantly, Mantel and colleagues noted that such information is only useful to the extent that perceptual systems are able to detect it. They hypothesised that self-selected complex exploratory activity enhances both the availability of and the ability to detect the information. Our aims in this study were to (a) characterise such ambient energy patterns using a global measure of movement variability and (b) demonstrate that perception of affordances for reaching is a function of complex exploratory activity. As such, our current investigation is complementary to Mantel et al.’s in that we focused on the circumstances that permit optimal detectability of information, rather than on the information itself. We hypothesised that optimal detectability is promoted by complex¹ exploratory activity including postural sway and subsequent head movements, and thus focused on measuring the complex variability of head movements in a reaching task.

Postural sway, and in turn the optic, haptic, and gravito-inertial flow patterns that are generated by it, are nonstationary signals. This means that classical measures of central tendency (e.g., mean magnitude and standard deviation) describe the movement at a very coarse level. By calculating the (stationary) mean of the fluctuations of the centre of mass during upright stance, researchers can miss the rich structure of variability that exists in the movement. To remedy this shortcoming a complexity measure called effort-to-compress (ETC) was calculated for each trial’s time series of head movements. ETC is a measure of the heterogeneity of the time series and the ease with which it can be converted into a homogeneous series (Nagaraj & Balasubramanian, 2017a, 2017b). ETC is especially well suited for the description of short time series (less than 500 samples) in a variety of fields, such as neuroscience (e.g., neural spikes, heart rate) and engineering (e.g., structural complexity of materials, Virmani & Nagaraj, 2019). ETC measures the heterogeneity by identifying “streaks” in the time series. These repeated occurrences (streaks or patterns) are then labelled as a unit, effectively shortening the time series. This logic is also used in engineering technology and computer science to compress data files such as music files and digital images. The number of steps involved in compressing the time series into its smallest possible length is a measure of how complex the original series was. In the present experiment, we used ETC as a measure of complexity of head movements by analysing the Euclidean distance series for each trial. More complex head movements may enhance the availability of global energy patterns and the effective sampling of the ambient energy array. This may yield perception of affordances that more closely matches action capabilities.

Does complexity of postural sway inform perception of affordances for other behaviours such as reaching to grasp a target from a standing posture? One way to test this is to manipulate the difficulty of the posture adopted during an ostensibly perceptual task (cf. Mark et al., 1990). It is hypothesised that the difficulty of the posture will influence postural sway that will in turn affect perception of whether a target object is within reach. Importantly, it is also hypothesised that ETC (the movement parameter that describes the complexity of postural sway variability) will modulate perception of affordances.

Perception of affordances in VR

VR has become a widely used tool in several areas of research, particularly perception (e.g., Durgin & Li, 2010; Flach & Holden, 1998; Geuss et al., 2016; Pointon et al., 2018). Due to the ease of manipulating task demands and stimuli within a virtual environment and the convenience for designing and running experiments that could not be conducted otherwise, researchers utilise it regularly. Specifically, VR has become a useful tool in investigating affordance judgements. For example, Geuss et al. (2010) examined accuracy of affordance judgements in the real world versus a virtual world. They modelled the virtual environment after the real-world environment and investigated perception of affordances for fitting through an aperture, following up on a study by Warren and Whang (1987). Judgements of affordances for passage between two poles were compared at matching distances in each setting. It was found that accuracy in participants’ responses were no different between the real world and the VR. Perception of affordances for reaching have also been shown to be comparable in both the real world (Carello et al., 1989) and virtual environments (Bhargava et al., 2020; Day et al., 2019). In real-world judgements of reachability, participants typically overestimate their reaching ability, even when reaching is part of the response (Weast & Proffitt, 2018). As in real-world settings, participants who are asked to judge whether an object in a virtual world is within reach tend to overestimate their actual reaching abilities (Doyon et al., under review). It is not clear what the source of the overestimation is. Although VR technology cannot simulate the full richness of the multimodal (visual, haptic, and gravito-inertial) patterns that are generated by global ambient energy patterns in the real environment, it serves as a compelling approximation that drives a lot of contemporary perceptual research.

In this study, observers wore a head-mounted display (HMD) that showed a target object (red ball) placed at different egocentric distances in a virtual room. Participants viewed this object while (actually) performing one of three postures: normal pose, heel-to-toe pose, and a yoga tree pose. They reported whether the object would be reachable at that distance from the performed posture. Participants’ verbal reports were recorded, along with response latencies. Head movements were recorded by the HMD during each trial. It was predicted that response times would be longer during more difficult² postural tasks. We based our prediction on past research on response latencies in tasks that require physical effort (Hajnal et al., 2014, 2016). Second, given that previous research has shown that performing a balance task inhibits the ability to perceive affordances (Mark et al., 1990; Wagman & Hajnal, 2014), it was predicted that affordance judgements would be less accurate (would less closely reflect reaching ability) as balance demands increase. Third, it was expected that head movements would increase with more difficult balance tasks to meet the demands of maintaining stable posture.

Our general predictions considered two main sources of influence on affordance judgements: task demands and organismic factors. The three poses constituted the main task demand. The placement of the stimuli at different distances was a (relative) spatial variable that was determined by the π-ratio, an intrinsic measure of affordance capability. In this sense, the π-ratio was a combination of external spatial task demands and organismic constraints, spanning both task and organismic variables. Each pose was grouped into blocks of trials, defining a temporal task demand. Given the differential effort requirements of maintaining some postures for an extended period of time, we expected that both the ability to maintain posture and the perception of reaching affordance would change across blocks of trials.

The second class of factors that were predicted to influence perceptual performance were organismic factors that described postural sway during trials: mean head movement (Mean), variability of head movement expressed as the coefficient of variation (CV), and ETC, indicating the complexity of postural sway. We assumed that these variables would differentially account for variability in affordance judgements given the nature of each variable. Specifically, we assumed that the Mean would be the least useful predictor, given the nonstationary nature of postural sway, CV would be significantly better, and ETC would be the best predictor. Spatiotemporal task demands (π, Block) were predicted to differentially interact with organismic factors (Mean, CV, and ETC) in the context of the three poses. Specifically, we expected that ETC would be the best predictor of affordance judgements and latencies, and that high values of ETC would improve the accuracy of perception.

Method

Participants

Students were recruited through the Sona participant pool at the University of Southern Mississippi. Data were collected from a total of 38 participants. Five participants were excluded due to misinterpretation of experimental instructions (N = 33). This is a sufficient sample size based on an approximate power analysis performed using the G*Power software package (Version 3.1.9.2; Faul et al., 2007) to obtain a medium $(η_{p}^{2} = 0.09)$ to large $(η_{p}^{2} = 0.25)$ effect size, and is consistent with what has been obtained in similar research (Doyon et al., under review). Participants included 29 women and four men, ranging from ages 18 to 26 (M = 18.97, SD = 1.69). Individuals were required to be 18 years of age or older and have normal or corrected-to-normal vision as well as no existing physical injuries (e.g., broken bones, sprained joints).

Materials and apparatus

This study employed a VR environment administered by a consumer version Oculus Rift HMD. Participants recorded their responses using two wireless handheld controllers, a button on the right controller was used to indicate a “yes” response and a button on the left controller was used to indicate a “no” response. The Unity game engine software (Version 2017.1.1f1) was used to programme and deliver the environment, along with the C# programming language to script events and data recordings. Two table-mounted Oculus motion sensors as well as sensors contained in the HMD tracked participant’s movement. The data drawn from the HMD were the data used to record head movement and assess postural instability.

The virtual environment consisted of a room with textured walls and natural lighting. The visual stimulus was a sphere (approximately the size of a tennis ball) that was suspended on a wire at the specific shoulder height of each participant (see Figure 1). This allowed for comfortable judgements of reachability. Reachability was defined as the ability to grasp the object with both the thumb and forefinger without leaning or bending forward at the hip or ankle.

Figure 1.

Virtual reality environment: ball hanging from ceiling at shoulder height.

Experimental design

This study employed a 3 Pose (Normal, Tandem, and Tree) × 5 Distance (π-Ratios of 0.9, 1.0, 1.1, 1.2, and 1.3) repeated-measures design. The stimulus was placed at different relative distances in front of participants. These distances were determined by dimensionless π-ratios (Carello et al., 1989) ranging from 0.9 to 1.3 (reflecting proportions of a participant’s maximum reaching distance). It was originally proposed that the distances be set at a range of 0.8–1.2. However, after analysing pilot data, it was determined that a shift in distances was necessary to achieve greater variability in responses due to overestimation observed in recent research.

The equation for these ratios is as follows

π = \frac{d}{a}

The equation takes into account both environmental- and participant-specific measurements. Here, d equals the physical distance to the target or visual stimulus, and a equals the length of the individual’s arm. Thus, a ratio of π = 1.00 represents the individual’s maximum reaching distance. Therefore, ratios of π ⩽ 1.00 will be within the participant’s reach, and ratios of π > 1.00 will be out of reach. Participants were randomly exposed to all five distances (π-Ratios of .9, 1.0, 1.1, 1.2, and 1.3) three times in each pose for a total of 45 trials. The repetitions were grouped into three sequences for each pose to minimise back-to-back trials being presented with the same distances.

Over the course of the study, participants were required to perform three separate balance positions to the best of their ability. The first was a normal pose (see left panel of Figure 2) where both feet were comfortably placed on the floor, the second was a toe-to-heel (tandem) pose (see middle panel of Figure 2) where one foot was placed directly in front of the other so that the toes of one met the heel of the other. Finally, there was a tree pose (commonly used in yoga practice, see right panel of Figure 2; S. S. Yu et al., 2012) where the sole of one foot was brought to rest on the alternate calf.

Figure 2.

Control pose (left panel), tandem pose (middle panel), and tree pose (right panel).

Procedure

After providing informed consent, physical measurements (e.g., shoulder height, eye height, and arm length) were taken for each participant and entered into the VR software. Arm length was measured from the shoulder joint to the tip of the thumb. Verbal instructions were given on how to operate the VR equipment as well as what to expect within the virtual environment. Demonstrations were given on how to perform the appropriate poses. After the participant had been fitted with the HMD and had each of the wireless controllers in hand, they began a series of practice trials. There were 15 total practice trials. At each increment of five trials, verbal instructions were given instructing a transition into the next pose. This allowed participants to become acquainted with the virtual environment as well as all three different poses. At all points of verbal instruction throughout the experiment, participants were allowed to rest if needed.

Once the practice trials were complete, participants were assigned a beginning pose. This differed depending on the counterbalancing order into which they were randomly assigned. The first group performed the following order of poses: normal, tandem, and tree; the second group: tandem, tree, and normal; whereas the third group: tree, normal, and tandem. Before beginning the experimental trials, participants were given verbal instructions on which pose to perform first. After each sequence of 15 trials, participants were allowed the opportunity to rest as additional verbal instructions were provided indicating which pose they would transition to next. Once they were comfortable in that pose, they pushed a button to proceed. Each individual trial concluded after participants reported their perceived ability to reach the ball in the current pose by pushing the relevant button. After finishing all 45 experimental trials, the experiment was complete. Participants were then asked to answer a brief demographic questionnaire and were given the opportunity to ask any questions. They were then granted credit for participation and excused.

Measurements and data analysis

Response times were recorded in milliseconds for each trial. Response time recording began with a button press marking the start of the trial and continued until the participant again pressed a button giving a response. There was a 500-ms inter-stimulus interval (ISI) between trials. Head and body movements were not restricted in any way. Participants were asked to not perform any type of reaching or leaning while making judgements. In the event that the participant had to step out of a pose and regain balance during a trial, the researcher recorded this by the press of a button. These recordings were documented in an excel file accompanied by a time stamp.

Head movements were recorded by tracking motion via the Oculus Rift headset’s position sensor in a three-dimensional coordinate system. The position sensor sampled the data at 30 Hz. The time series of head position coordinates were converted into Euclidean distances by computing the straight-line distance between each adjacent sample’s position coordinates. These timeseries were processed in MATLAB using the ETC algorithm (Nagaraj & Balasubramanian, 2017a). This analysis assesses the heterogeneity of variability across the timeseries and characterises the efficiency with which the time series can be turned into a completely homogeneous signal. The resulting parameter ETC is a reliable descriptor of the level of disorder, or randomness, in the signal, rather than raw variability. Accordingly, the signal might be highly variable, but not very complex, or it may be highly complex, but not very variable. Smaller values of ETC indicate lower complexity, whereas higher values of ETC indicate higher complexity.

Results

Probability data

As affordance judgements are measured with a dichotomous variable (yes/no), we used a mixed-effects hierarchical logistic regression (Bates et al., 2014) as it is a more appropriate analysis than analysis of variance (ANOVA) for this type of data. The following model was used

\begin{array}{l} Response ~ Trial + Pose \times π \times Block \\ \times Mean + Pose \times π \times Block \times CV + Pose \\ \times π \times Block \times ETC + (Trial | Participant) \end{array}

Trial and participant were set as random effects; all other variables were fixed effects. Pose was coded as a categorical variable with three levels: 1 = normal (control), 2 = tandem, and 3 = tree pose. The model was built to test how affordance responses were affected by postural demands (pose) along with spatial aspects of the task (distance ratio π), and temporal aspects of the task (blocks of trials). In addition, the model tested the contributions of various measures of head movement: magnitude (Mean), variability (CV), and complexity (ETC). Table 1 shows the output of the statistical analysis. Due to the constraints of the lmer statistical package in R, the main effects of pose and interactions involving the pose variable are always based on the comparison with the control pose.

Table 1.

Best fitting mixed-effects logistic regression model of affordance judgements.

Predictor	β	SE	p
Intercept	−4.31	43.29	.920
Trial	−0.02	0.03	.465
Block	9.39	19.07	.623
π	4.37	35.41	.902
Tandem pose	16.65	51.63	.747
Tree pose	69.5	58.76	.237
Mean	−32,485.65	45,300.05	.473
CV (coefficient of variation)	26.07	48.36	.589
ETC	142.21	81.66	.082
π × Mean	33,564.84	36,188.07	.354
π × Block	−7.86	15.75	.618
π × CV	−19.78	39.51	.617
π × ETC	−128.18	67.19	.057
Block × ETC	−44.82	35.68	.209
Block × CV	−5.29	21.05	.802
Block × Mean	18,285.52	20,491.22	.372
π × Mean × Block	−17,151.48	16,204.34	.29
π × CV × Block	3.75	17.33	.829
π × ETC × Block	40.89	29.63	.168
Interactions of tandem pose with other terms
Tandem Pose × Mean	30,174.96	47,833.84	.528
Tandem Pose × π	−16.5	42.7	.699
Tandem Pose × Block	−5.37	26.71	.841
Tandem Pose × CV	−51.68	55.44	.351
Tandem Pose × ETC	−92.71	99.9	.354
Tandem Pose × π × Mean	−27,698.81	38,624.71	.473
Tandem Pose × π × Block	5.33	22.19	.810
Tandem Pose × π × CV	39.35	45.71	.389
Tandem Pose × π × ETC	88.38	83.01	.2872
Tandem Pose × Block × Mean	−23,137.76	21,475.48	.282
Tandem Pose × Block × CV	30.44	29.45	.302
Tandem Pose × Block × ETC	28.96	48.87	.554
Tandem Pose × π × Mean × Block	19,016.48	17,168.93	.268
Tandem Pose × π × CV × Block	−22.93	24.3	.346
Tandem Pose × π × ETC × Block	−27.97	40.75	.493
Interactions of tree pose with other terms
Tree Pose × Mean	19,220.4	45,683.49	.674
Tree Pose × π	−64.33	48.78	.187
Tree Pose × Block	−28.71	27.02	.288
Tree Pose × CV	−41.01	67.61	.544
Tree Pose × ETC	−198.85	101.7	.051
Tree Pose × π × Mean	−22,032.8	32,314	.496
Tree Pose × π × Block	−21,509	36,545.36	.556
Tree Pose × π × CV	37.66	56.03	.502
Tree Pose × π × ETC	187.93	84.28	.026
Tree Pose × Block × Mean	−16,243.04	20,752.58	.434
Tree Pose × Block × CV	11.74	30.16	.697
Tree Pose × Block × ETC	89.37	47.09	.058
Tree Pose × π × Mean × Block	15,093.58	16,440.37	.359
Tree Pose × π × CV × Block	−10.63	25.05	.671
Tree Pose × π × ETC × Block	−80.66	39.24	.04

SE: standard error; ETC: effort-to-compress.

Significant effects (p < .05) are in bold font.

Overall, there was no effect of Mean or CV on affordance judgements. There was a significant positive three-way Tree Pose × π × ETC interaction (β = 187.93, standard error [SE] = 84.28, p = .026). There was also a significant negative four-way Tree Pose × Block × π × ETC interaction (β = −80.66, SE = 39.24, p = .04). There were no interactions for tandem pose. The four-way interaction is presented in Figure 3. A schematic diagram of all significant interactions is presented in Figure 4 to visualise the apportionment of the total explained variance.

Figure 3.

The four-way Tree Pose × Block × π × ETC interaction on affordance judgements in the hierarchical logistic regression. The plots show the comparison between tree pose (continuous lines) and control pose (dashed lines) over blocks and across low and high values of ETC. The points represent average probability of reaching (based on yes/no affordance judgements) at each value of π. ETC is a continuous variable, but for the purposes of better visualisation, ETC was dichotomised by a median split (LOW and HIGH ETC) in the plots.

Figure 4.

Schematic diagram presenting significant effects of the logistic regression on affordance judgements. The shaded oval represents a negative effect, and the unfilled oval is a positive effect. C3 represents the comparison between tree pose and control pose. The arrow indicates how the variance explained is apportioned from lower- to higher-order interactions. Each new row represents the addition of a new dimension by the significant interactions. The boldface font indicates which new term was added at each, more complex level of interactions.

A linear mixed-effects model was created to predict response time. The model was built using the same combination of predictors as the logistic model (random effects are not shown, but included)

\begin{array}{l} Response Time ~ Pose \times π \times Block A \\ \times Mean + Pose \times π \times Block \times CV \\ + Pose \times π \times Block \times ETC \end{array}

Table 2 shows the output of the statistical analysis.

Table 2.

Best fitting mixed-effects linear regression model of response time.

Predictor	β	SE	p
Intercept	−0.548	4.375	.9004
Block	−0.852	2.19	.6975
π	4.102	3.772	.277
Tandem pose	−12.986	6.698	.0527
Tree pose	−4.36	6.768	.5195
Mean	−4,782.12	3,917.017	.2223
CV (coefficient of variation)	−2.743	4.7	.5595
ETC	8.124	7.561	.2828
π × Mean	6,105.579	3,517.489	.0828
π × Block	0.546	1.922	.7766
π × CV	1.61	4.045	.6907
π × ETC	−13.584	6.657	.0415
Block × ETC	−0.072	3.817	.9849
Block × CV	1.824	2.291	.4261
Block × Mean	1,840.701	1,427.372	.1974
π × Mean × Block	−2,223.7	1,290.595	.0851
π × CV × Block	−1.238	1.993	.5346
π × ETC × Block	0.256	3.421	.9404
Interactions of tandem pose with other terms
Tandem Pose × Mean	13,282.71	5,325.755	.0127
Tandem Pose × π	11.422	5.926	.0541
Tandem Pose × Block	6.303	3.235	.0516
Tandem Pose × CV	19.12	6.712	.0045
Tandem Pose × ETC	2.462	11.691	.8333
Tandem Pose × π × Mean	−12,574.4	4,780.974	.0086
Tandem Pose × π × Block	−5.461	2.883	.0584
Tandem Pose × π × CV	−16.094	5.976	.0072
Tandem Pose × π × ETC	−2.992	10.444	.7745
Tandem Pose × Block × Mean	−4,060.67	2,011.021	.0437
Tandem Pose × Block × CV	−8.341	3.261	.0106
Tandem Pose × Block × ETC	−4.339	5.656	.4431
Tandem Pose × π × Mean × Block	4,158.464	1,814.68	.0221
Tandem Pose × π × CV × Block	6.802	2.91	.0196
Tandem Pose × π × ETC × Block	4.126	5.094	.4181
Interactions of tree pose with other terms
Tree Pose × Mean	9,583.523	4,134.994	.0206
Tree Pose × π	3.255	5.996	.5873
Tree Pose × Block	3.431	3.188	.2821
Tree Pose × CV	8.638	7.585	.255
Tree Pose × ETC	−6.261	10.781	.5615
Tree Pose × π × Mean	−10,695.4	3,728.789	.0042
Tree Pose × π × Block	−2.943	2.819	.2967
Tree Pose × π × CV	−6.363	6.741	.3454
Tree Pose × π × ETC	6.724	9.672	.487
Tree Pose × Block × Mean	−4,032.94	1,566.61	.0101
Tree Pose × Block × CV	−5.359	3.479	.1237
Tree Pose × Block × ETC	−0.018	5.185	.9972
Tree Pose × π × Mean × Block	4,470.826	1,415.767	.0016
Tree Pose × π × CV × Block	4.307	3.073	.1612
Tree Pose × π × ETC × Block	−0.212	4.643	.9637

SE: standard error; ETC: effort-to-compress.

Significant effects (p < .05) are in bold font.

There were no significant main effects. π interacted with ETC (β = −13.58, SE = 6.65, p = .042). Tandem pose interacted with Mean (β = 13,282.71, SE = 5,325.76, p = .013) and CV (β = 19.12, SE = 6.71, p = .005), indicating that response times increased as the mean magnitude and variability of head movement increased in the tandem pose compared with the control pose. There were four significant negative three-way interactions for tandem pose. Tandem Pose × π × Mean (β = −12,574.4, SE = 4,780.97, p = .009), Tandem Pose × π × CV (β = −16.09, SE = 5.98, p = .007), Tandem Pose × Block × Mean (β = −4,060.67, SE = 2,011.02, p = .044), and Tandem Pose × Block × CV (β = −8.34, SE = 3.26, p = .011). There were two significant positive four-way interactions: Tandem Pose × π × Block × Mean (β = 4,158.46, SE = 1,814.68, p = .022) and Tandem Pose × π × Block × CV (β = 6.80, SE = 2.91, p = .019).

There was a significant positive two-way interaction of Tree Pose and Mean (β = 9,583.52, SE = 4,134.99, p = .021). There were also two significant negative three-way interactions including tree pose. These include the Tree Pose × π × Mean interaction (β = −10,685.4, SE = 3,728.80, p = .004) as well as Tree Pose × Block × Mean (β = −4,032.94, SE = 1,566.61, p = .01. Finally, there was a significant positive four-way interaction of Tree Pose × π × Block × Mean (β = 4,470.83, SE = 1,415.76, p = .002). A schematic diagram of all significant main effects and interactions was presented in Figure 5 to visualise the apportionment of the total explained variance.

Figure 5.

Schematic diagram presenting significant effects of the mixed effects model on response time. The shaded ovals are negative effects, and the unfilled ovals are positive effects. C2 represents the comparison between tandem pose and control pose. C3 represents the comparison between tree pose and control pose. The arrows indicate how the variance explained is apportioned from lower- to higher-order interactions. Each new row represents the addition of a new dimension by the significant interactions. The boldface font indicates which new term was added at each more complex level of interactions.

Mean, CV, and ETC were used as predictor variables in mixed effects models. However, just like with affordance judgements and response time, we expected these variables to be influenced by the postural manipulation as well. To check our specific hypotheses, we conducted 3 Pose (Normal, Tandem, and Tree) × 5 Distance (π-Ratios of 0.9, 1.0, 1.1, 1.2, and 1.3) repeated-measures ANOVAs on response time, Mean, CV, and ETC.

For response time, there was a statistically significant main effect of both pose, F(2, 64) = 3.23, p = .046, $η_{p}^{2} = 0.09$ , and distance, F(2.44, 78.13) = 11.29, p < .01, $η_{p}^{2} = 0.26$ (Greenhouse–Geisser correction, due to significant Mauchly’s test on sphericity). Overall, participants took less time to respond while maintaining the tree pose than the tandem pose (p = .027, Bonferroni correction was applied to mitigate the potential for Type I error by multiplying the actual obtained p value by 3, the number of comparisons conducted). See Figure 6 for details. Response times increased with distance.

Figure 6.

Mean of response time across π (distance) for each pose. Distance was expressed as ratio of arm length to actual distance of target object. Response times increase with distance and are smallest for the most difficult tree pose. Error bars represent the standard error of the mean.

The mean magnitude of head movements (Mean) was calculated to estimate the magnitude of postural sway in each pose. There was a significant main effect for distance, F(3.17, 101.5) = 2.83, p = .04, $η_{p}^{2} = 0.08$ (Greenhouse–Geisser correction). In all three poses, the largest head movements occurred at the largest distance. There was also a significant main effect of pose, F(1.36, 43.58) = 59.76, p < .001, $η_{p}^{2} = 0.65$ (Greenhouse–Geisser correction). The most head movement occurred in the tree pose, followed by the tandem pose and the normal control pose (see Figure 7 for details). The CV for head movements was analysed to assess variability. Once again, we found a significant main effect for distance, F(3.25, 107.33) = 4.83, p = .003, $η_{p}^{2} = 0.13$ (Greenhouse–Geisser correction) as well as a significant interaction of distance and pose, F(8, 256) = 2.19, p = .03, $η_{p}^{2} = 0.06$ . The CV was most extreme for the shortest and longest distances in the normal pose (see Figure 8).

Figure 7.

Mean of head movements across π (distance) for each pose. Head movement increased with pose difficulty. Error bars represent the standard error of the mean.

Figure 8.

Coefficient of variation (CV) of head movements across π (distance) for each pose. The CV was most extreme for the shortest and longest distances in the normal stance. Error bars represent the standard error of the mean.

The ANOVA on ETC returned a significant main effect of distance, F(2.81, 89.9) = 16.81, p = .001, $η_{p}^{2} = 0.34$ , as well as a significant main effect of pose, F(2, 64) = 3.26, p = .045, $η_{p}^{2} = 0.09$ . Participants’ head movements were more complex while maintaining the tree pose (M = 0.335, SD = 0.10) than the standard control pose (M = 0.314, SD = 0.09; p = .02, Fisher’s protected least significant difference [PLSD] post hoc test). The data are plotted in Figure 9.

Figure 9.

Effort-to-compress (ETC) of head movements across π (distance) for each pose. ETC was the largest in the tree pose, and significantly different from the standard control pose. Error bars represent the standard error of the mean.

Discussion

The purpose of this study was to investigate (a) the effects of balance demands on perception of affordances for reaching, and (b) the contribution of body movements on predicting affordance judgements. The results showed that more complex movement patterns are better predictors of affordance judgements than less complex movement patterns. This could mean that complex movements with large ETC values more effectively generated ambient information patterns that specify whether certain actions are possible (e.g., whether a target object is within reach from a given pose).

As a reminder, this study included five separate hypotheses for the five variables that were used as dependent measures in ANOVA designs, listed in Table 3.

Table 3.

Overview of pose effect predictions for reachability, response time, and movement parameters using ANOVA designs.

	Normal (control)	Tandem	Tree
Reachability judgements	Most accurate	Less accurate	Least accurate
Response time	Least time	More time	Most time
Head movement (Mean)	Least movement	More movement	Most movement
Head movement (CV)	Least variability	More variability	Most variability
Head movement (complexity)	Least complex	More complex	Most complex

ANOVA: analysis of variance; CV: coefficient of variation.

The ANOVA analyses showed that increased postural demands during perceptual tasks result in more overall postural sway and faster judgements of affordances. As such, the second hypothesis about response times was not supported. It is possible that the tree pose was so demanding that participants sped up their responses to minimise energy expenditure or to avoid losing balance. This contradicts the “posture first” principle (Horak, 2006) which predicts longer latencies when the postural task is difficult. However, increases in response time did occur in all three poses as object distance increased.

Movement variability (CV) exhibited a more complex pattern of dependency on postural demands. The significant π × Pose interaction showed that the two difficult poses (tandem and tree) produced the same level of variability across distances, and that variability steadily increased over distances only in the control pose. This latter finding is consistent with past research on quiet stance where viewing more distant targets caused more variability in postural sway (Bonnet et al., 2010; Stoffregen et al., 1999, 2000). It is still unclear why more difficult poses used in the present experiment did not follow the same effect of distance. Future research is needed to disentangle the interaction between distance and postural demands.

The magnitude and variability of head movements were largest while participants maintained the tree pose. This is consistent with our hypotheses. In all three poses, it was found that as object distance increased, head movement also increased. This was also found in past research that showed that increased object distance is associated with increased postural sway (Bonnet et al., 2010).

The complexity of head movements (as measured by ETC) was the highest in the tree pose, which was consistent with our hypothesis. It appears that more demanding postures facilitated more complex movement patterns in the service of detecting information patterns that guide prospective action. Whether complex movement patterns are a byproduct of increased postural instability, or a part of intentional exploratory activity to ensure optimal information detection is an open question for future research.

To get a more complete description of the data, regression models were constructed to predict responses. The models combined both spatial (π) and temporal (block) aspects of the task to assess movement parameters.

Affordance judgements are a function of task demands and complexity of postural sway

Mixed effects hierarchical logistic regression modelling showed that the strongest predictor of affordance judgements was a pattern of significant interactions among ETC and Pose. Mean head position and CV of head position did not interact with Pose. Thus, affordance judgements in the most difficult balance task were predicted by the most complex descriptor of head movements.

The four-way interaction of Tree Pose × π × Block × ETC is important to consider (see Figure 5). Participants who maintained the tree pose and exhibited high ETC showed gradually increasing sensitivity in differentiating possible from impossible actions, as indicated by the increase in the slope of logistic curve around the 50th percentile across blocks of trials. This pattern was noticeable less consistent when ETC was low. In other words, on trials with high ETC, participants exhibited more certainty that the object was within reach at the two closest distances and more certainty that the object was out of reach at the two farthest distances compared with trials with low ETC. The fact that this perceptual sensitivity increased over blocks suggests that the most accurate responses while maintaining a difficult pose occurred when participants explored their environment through complex movements (i.e., high ETC). This finding is congruent with recent findings showing that increases in movement complexity yielded greater reliance on multimodal information and resulted in more accurate affordance judgements (Hajnal et al., 2018). Responses were not as accurate (i.e., showing overestimation) and not as sensitive (indicated by shallow slope of psychometric curve) during the tree pose in Blocks 1 and 2 for high ETC (see bottom panel of Figure 5). The fact that affordance judgements changed over blocks means that performance was influenced by temporal factors. As mentioned before, this could be attributed to practice effects for either familiarity with repeated distances or balance maintenance. This finding is interesting because response times were shorter overall for the tree pose than for the other poses. This could suggest that perception of affordances is more accurate when judgements are made without taking too much time to dwell on the task at hand (see Heft, 1993; Wagman, Bai, & Smith, 2016). One could argue that this is due to participant’s underestimation of abilities based on being in an unstable standing position. However, in this circumstance, it is still the case that an overestimation of reachability occurs for closer distances. In sum, participants who held the tree pose as the final portion of the experiment responded faster than those in the control pose and were more likely to be accurate.

Response times are affected by increased task demands and more complex movements

In a linear mixed effects model of response time, there were influences of Mean and CV on response times in the tandem pose as compared with the control pose. As mean magnitude and variability of movement increased, the differences in response times between the tandem pose and control pose became smaller, as indicated by the positive four-way interactions containing Mean and CV, respectively. Increasing movement magnitude resulted in less deliberation of affordances, that is, shorter response times (see Figure 7 for details). For the tree pose, the pattern of results was such that only Mean magnitude (but not CV) modulated response latencies. In general, this means that increased postural demands go hand in hand with increased postural sway, but not necessarily complexity as they influence the time course of affordance judgements. In short, large movements in the tree pose resulted in comparably longer response times. It is possible that this effect may have been due an artefact of larger movements demanding more time to be performed. Notably, absent was the effect of ETC on response time. This is not congruent with the original hypothesis, because it was originally predicted that greater postural instability and complexity would result in longer latency of response decisions.

We concluded that postural complexity contributes to what is perceived (affordance judgements), but not to latency of responding. Responses during the tree pose posture were the shortest and were modulated simply by the magnitude of postural sway. Although we did not collect any information on strategy, it is possible that participants did not want to hold the difficult tree pose for extended periods of time due to fatigue (or concern for losing balance), and that the brevity of the trials may have overshadowed or masked an effect of complexity on latencies. This result does not exclude the possibility that other affordance tasks that require longer encounters per trial may affect latencies through the complexity of postural sway.

Contribution of the complexity of the haptic system to affordance perception

In the ecological approach to perception and action, perception and behaviour are reciprocal and mutually constraining. Exploratory behaviours aid the perceiver in detecting the invariants within a structured energy array that specify affordances. Postural sway serves this exploratory function and is thus a key contributor to perception of affordances (e.g., Mark et al., 1990; Stoffregen et al., 2005). Importantly, exploratory postural sway involves movements of the eyes, head, torso, and whole body (Eddy & Kelty-Stephen, 2015; Palatinus et al., 2013, 2014; Y. Yu et al., 2010). Such movements create flow patterns both within and across energy arrays, the detection of which supports perception of affordances.

Movements of any kind (even those of the eyes) implicate the haptic perceptual system (Cabe, 2019). Recently, the haptic system has been described as a multifractal biotensegrity system (Turvey & Fonseca, 2014)—a system that efficiently and immediately redistributes applied forces across the tension and compression elements of the system at all levels of organisation from the individual cell to the body as whole. The multifractal nature of the movement system may be one reason why complexity is a better measure of the variability of postural sway than more traditional (stationary) measures of central tendency. We have shown that, at least to some extent, the complexity of postural sway predicted affordance judgements. Due to the shortness of trials, we were unable to use multifractal measures, but ETC provided insight into the role of complexity in tying perception and action together in an exemplary affordance task.

Footnotes

Authors’ note

The experiments reported here were part of a Master’s thesis at the University of Southern Mississippi by H.M. Portions of the data were presented at the Vision Sciences Society meeting (May 2019, St. Pete Beach, FL, USA) and at the International Conference on Perception and Action (July 2019, Groningen, Netherlands).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported in this paper was supported by the Higher Education Excellence Program of the Ministry of Human Capacities within the Biotechnology research area of Budapest University of Technology and Economics (BME FIKP-BIO).

ORCID iD

Hannah Masoner

Notes

References

Balasubramaniam

Riley

M. A.

Turvey

M. T.

(2000). Specificity of postural sway to the demands of a precision task. Gait & Posture, 11(1), 12–24.

Bates

Maechler

Bolker

Walker

(2014). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Bhargava

Lucaites

K. M.

Hartman

L. S.

Solini

Bertrand

J. W.

Robb

A. C.

. . . Babu

S. V.

(2020). Revisiting affordance perception in contemporary virtual reality. Virtual Reality. Advance online publication. https://doi.org/10.1007/s10055-020-00432-y

Bonnet

C. T.

Temprado

J. J.

Berton

(2010). The effects of the proximity of an object on human stance. Gait & Posture, 32(1), 124–128.

Cabe

P. A.

(2019). All perception engages the tensegrity-based haptic medium. Ecological Psychology, 31(1), 1–13.

Carello

Grosofsky

Reichel

F. D.

Solomon

H. Y.

Turvey

M. T.

(1989). Visually perceiving what is reachable. Ecological Psychology, 1(1), 27–54.

Day

Ebrahimi

Hartman

L. S.

Pagano

C. C.

Robb

A. C.

Babu

S. V.

(2019). Examining the effects of altered avatars on perception-action in virtual reality. Journal of Experimental Psychology: Applied, 25(1), 1–24.

Doyon

J. K.

Clark

Hajnal

Legradi

Effects of surface luminance and texture discontinuities on reachability in virtual reality. (Under Review)

Durgin

F. H.

(2010). Controlled interaction: Strategies for using virtual reality to study perception. Behavior Research Methods, 42(2), 414–420.

10.

Eddy

C. L.

Kelty-Stephen

D. G.

(2015). Nesting of focal within peripheral vision promotes interactions across nested time scales in head sway: Multifractal evidence from accelerometry during manual and walking-based Fitts tasks. Ecological Psychology, 27(1), 43–67.

11.

Faul

Erdfelder

Lang

A. G.

Buchner

(2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191.

12.

Flach

J. M.

Holden

J. G.

(1998). The reality of experience: Gibson’s way. Presence, 7(1), 90–95.

13.

Geuss

M. N.

McCardell

M. J.

Stefanucci

J. K.

(2016). Fear similarly alters perceptual estimates of and actions over gaps. PLOS ONE, 11(7), Article e0158610.

14.

Geuss

M. N.

Stefanucci

J. K.

Creem-Regehr

Thompson

W. B.

(2010, July 23–24). Can I pass? Using affordances to measure perceived size in virtual environments [Conference session]. Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization, Los Angeles, CA, United States.

15.

Gibson

J. J.

(1979). The ecological approach to visual perception. Houghton Mifflin.

16.

Hajnal

Bunch

D. A.

Kelty-Stephen

D. G.

(2014). Going for distance and going for speed: Effort and optical variables shape information for distance perception from observation to response. Attention, Perception, & Psychophysics, 76(4), 1015–1035.

17.

Hajnal

Bunch

D. A.

Kelty-Stephen

D. G.

(2016). Pulling out all the stops to make the distance: Effects of effort and optical information in distance perception responses made by rope pulling. Attention, Perception, & Psychophysics, 78(2), 685–699.

18.

Hajnal

Clark

J. D.

Doyon

J. K.

Kelty-Stephen

D. G.

(2018). Fractality of body movements predicts perception of affordances: Evidence from stand-on-ability judgments about slopes. Journal of Experimental Psychology: Human Perception and Performance, 44(6), 836–841.

19.

Heft

(1993). A methodological note on overestimates of reaching distance: Distinguishing between perceptual and analytical judgments. Ecological Psychology, 5(3), 255–271.

20.

Horak

F. B.

(2006). Postural orientation and equilibrium: What do we need to know about neural control of balance to prevent falls? Age and Ageing, 35(Suppl. 2), ii7–ii11.

21.

Kantelhardt

J. W.

Zschiegner

S. A.

Koscielny-Bunde

Havlin

Bunde

Stanley

H. E.

(2002). Multifractal detrended fluctuation analysis of nonstationary time series. Physica A: Statistical Mechanics and Its Applications, 316(1), 87–114.

22.

Mantel

Stoffregen

T. A.

Campbell

Bardy

B. G.

(2015). Exploratory movement generates higher-order information that is sufficient for accurate perception of scaled egocentric distance. PLOS ONE, 10(4), Article e0120025. https://doi.org/10.1371/journal.pone.0120025

23.

Mark

L. S.

Balliett

J. A.

Craver

K. D.

Douglas

S. D.

Fox

(1990). What an actor must do in order to perceive the affordance for sitting. Ecological Psychology, 2(4), 325–366.

24.

Nagaraj

Balasubramanian

(2017a). Dynamical complexity of short and noisy time series. The European Physical Journal Special Topics, 226(10), 2191–2204.

25.

Nagaraj

Balasubramanian

(2017b). Three perspectives on complexity: Entropy, compression, subsymmetry. The European Physical Journal Special Topics, 226(15), 3251–3272.

26.

Palatinus

Dixon

J. A.

Kelty-Stephen

D. G.

(2013). Fractal fluctuations in quiet standing predict the use of mechanical information for haptic perception. Annals of Biomedical Engineering, 41(8), 1625–1634.

27.

Palatinus

Kelty-Stephen

D. G.

Kinsella-Shaw

Carello

Turvey

M. T.

(2014). Haptic perceptual intent in quiet standing affects multifractal scaling of postural fluctuations. Journal of Experimental Psychology: Human Perception and Performance, 40(5), 1808–1818.

28.

Pointon

Thompson

Creem-Regehr

Stefanucci

Joshi

Paris

Bodenheimer

(2018, August 10–11). Judging action capabilities in augmented reality [Conference session]. Proceedings of the 15th ACM Symposium on Applied Perception, Vancouver, BC, Canada.

29.

Stoffregen

T. A.

Mantel

Bardy

B. G.

(2017). The senses considered as one perceptual system. Ecological Psychology, 29(3), 165–197.

30.

Stoffregen

T. A.

Pagulayan

R. J.

Bardy

B. G.

Hettinger

L. J.

(2000). Modulating postural control to facilitate visual performance. Human Movement Science, 19(2), 203–220.

31.

Stoffregen

T. A.

Smart

L. J.

Bardy

B. G.

Pagulayan

R. J.

(1999). Postural stabilization of looking. Journal of Experimental Psychology: Human Perception and Performance, 25(6), 1641–1658. https://doi.org/10.1037/0096-1523.25.6.1641

32.

Stoffregen

T. A.

Villiard

(2009). Body sway at sea for two visual tasks and three stance widths. Aviation, Space, and Environmental Medicine, 80(12), 1039–1043.

33.

Stoffregen

T. A.

Yang

C. M.

Bardy

B. G.

(2005). Affordance judgments and nonlocomotor body movement. Ecological Psychology, 17(2), 75–104.

34.

Turvey

M. T.

Fonseca

S. T.

(2014). The medium of haptic perception: A tensegrity hypothesis. Journal of Motor Behavior, 46(3), 143–187.

35.

Vicente

K. J.

Rasmussen

(1990). The ecology of human-machine systems II: Mediating “direct perception” in complex work domains. Ecological Psychology, 2(3), 207–249.

36.

Virmani

Nagaraj

(2019). A novel perturbation based compression complexity measure for networks. Heliyon, 5(2), Article e01181.

37.

Wagman

J. B.

(2012). Perception of maximum reaching height reflects impending changes in reaching ability and improvements transfer to unpracticed reaching tasks. Experimental Brain Research, 219(4), 467–476.

38.

Wagman

J. B.

Bai

Smith

P. J.

(2016). Nesting in perception of affordances for stepping and leaping. Attention, Perception, & Psychophysics, 78(6), 1771–1780.

39.

Wagman

J. B.

Caputo

S. E.

Stoffregen

T. A.

(2016). Sensitivity to hierarchical relations among affordances in the assembly of asymmetric tools. Experimental Brain Research, 234(10), 2923–2933.

40.

Wagman

J. B.

Cialdella

V. T.

Stoffregen

T. A.

(2018). Higher order affordances for reaching: Perception and performance. Quarterly Journal of Experimental Psychology, 72(5), 1200–1211.

41.

Wagman

J. B.

Hajnal

(2014). Getting off on the right (or left) foot: Perceiving by means of a rod attached to the preferred or non-preferred foot. Experimental Brain Research, 232(11), 3591–3599.

42.

Walter

H. J.

Wagman

J. B.

Stoffregen

T. A.

(2019). Adaptive perception of changes in affordances for walking on a ship at sea. Human Movement Science, 64, 28–37.

43.

Walter

H. J.

Wagman

J. B.

Stergiou

Erkmen

Stoffregen

T. A.

(2017). Dynamic perception of dynamic affordances: Walking on a ship at sea. Experimental Brain Research, 235(2), 517–524.

44.

Warren

W. H.

Jr. Whang

(1987). Visual guidance of walking through apertures: Body-scaled information for affordances. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 371–383.

45.

Weast

R. A.

Proffitt

D. R.

(2018). Can I reach that? Blind reaching as an accurate measure of estimated reachable distance. Consciousness and Cognition, 64, 121–134. https://doi.org/10.1016/j.concog.2018.02.013

46.

Cardwell

Mark

L. S.

(2009). Perceiving multiple affordances for objects. Ecological Psychology, 21(3), 185–217.

47.

S. S.

Wang

Samarawickrame

Hashish

Kazadi

Greendale

G. A.

Salem

G. J.

(2012). The physical demands of the tree (vriksasana) and one-leg balance (utthita hasta padangusthasana) poses performed by seniors: A biomechanical examination. Evidence-Based Complementary and Alternative Medicine, 2012, e971896. https://doi.org/10.1155/2012/971896

48.

Bardy

B. G.

Stoffregen

T. A.

(2010). Influences of head and torso movement before and during affordance perception. Journal of Motor Behavior, 43(1), 45–54.