Abstract
We investigated the effect of an observer’s hand postures on visual motion perception using the stream/bounce display. When two identical visual objects move across collinear horizontal trajectories toward each other in a two-dimensional display, observers perceive them as either streaming or bouncing. In our previous study, we found that when observers put their palms together just below the coincidence point of the two objects, the percentage of bouncing responses increased, mainly depending on the proprioceptive information from their own hands. However, it remains unclear if the tactile or haptic (force) information produced by the postures mostly influences the stream/bounce perception. We solved this problem by changing the tactile and haptic information on the palms of the hands. Experiment 1 showed that the promotion of bouncing perception was observed only when the posture of directly putting one’s palms together was used, while there was no effect when a brick was sandwiched between the participant’s palms. Experiment 2 demonstrated that the strength of force used when putting the palms together had no effect on increasing bounce perception. Our findings indicate that the hands-induced bounce effect derives from the tactile information produced by the direct contact between both palms.
Introduction
The stream/bounce display provides a means of investigating fundamental motion perception, object representation, and multisensory interaction (e.g., Bertenthal, Banton, & Bradbury, 1993; Bushara et al., 2003; Dufour, Touzalin, Moessinger, Brochard, & Després, 2008; Goldberg & Pomerantz, 1982; Metzger, 1934; Michotte, 1946/1963; Mitroff, Scholl, & Wynn, 2005; Sekuler & Sekuler, 1999; Sekuler, Sekuler, & Lau, 1997; Watanabe & Shimojo, 2001). The stream/bounce display involves a two-dimensional display in which two identical moving objects approach, overlap, and then separate from each other along collinear trajectories. The display produces visual ambiguity and bistability in that the objects can be seen as either streaming through or bouncing off one another despite there being no change in motion trajectory (as shown in Figure 1(a)).
(a) Schematic illustration of the visual stimuli. (b and c) Box plots showing the percentages of bouncing perception in each condition of Experiments 1 and 2. The cross and line within the box show the mean and median. The box boundaries represent the interquartile range, the whiskers represent the minimum and maximum, and the dots represent outliers that are more than 1.5 times the interquartile range. Asterisks between the bars indicate significant differences (*p < .05, **p < .01).
The two moving objects tend to be more frequently perceived as streaming than bouncing. Yet, it has been demonstrated that several factors, such as an auditory cue (Sekuler et al., 1997), endogenous and exogenous distraction of attention (Watanabe & Shimojo, 1998), intramodal grouping (Kawachi & Gyoba, 2006; Watanabe & Shimojo, 2001), and mental imagery (Berger & Ehrsson, 2013, 2017), are capable of modulating the motion perception. Notably, Sekuler et al. (1997) demonstrated that a click sound presented when the two moving objects converge mostly induces bouncing perception. This phenomenon, called the audiovisual bounce-inducing effect, is a clear example that the perception of visual motion is influenced by an auditory stimulus. Furthermore, it has been reported that the motion perception is dependent on the auditory context at the moment of coincidence of the two objects (Grassi & Casco, 2009, 2010; Watanabe & Shimojo, 2001) and on the presence of an additional moving object near the two objects (Kawachi & Gyoba, 2006). These findings have demonstrated that audio and visual contexts can influence the intermodal and intramodal interactions in the stream/bounce perception.
We further focused on the effect of the observer’s hand postures on the perceptual interpretation of a visual motion event. Several studies have shown the effects of one’s own body posture on bistable visual motion perception (Mitsumatsu, 2009; van Elk & Blanke, 2012; Wohlschläger, 2000). In our previous study, we found that when observers put their palms together just below the coincidence point of the two moving objects, the percentage of bouncing responses increased, which is referred to as the hands-induced bounce (HIB) effect (Saito & Gyoba, 2016). Although it has been reported that the HIB effect is mainly based on proprioceptive information regarding the location and posture of one’s hands, the question remains of whether the tactile or haptic (force) information derived from the posture is the main cause of the HIB effect.
Therefore, the aim of this study was to more precisely identify the critical factors responsible for producing the HIB effect. In the following two experiments, we manipulated the tactile information on the palms by sandwiching a brick between the hands and controlled the haptic information by asking participants to change their physical force when they put their hands together.
Experiment 1
In Experiment 1, we examined whether the presence or absence of tactile information produced by the posture of putting one’s palms together would be a main factor for increasing the bouncing perception in the bistable motion display.
Method
Participants
Fourteen students (six men and eight women, aged between 19 and 25 years; mean age 21 years) from Tohoku University gave informed consent and participated in Experiment 1 individually. All participants had normal or corrected-to normal vision. They were naive with regard to the purpose of this study. All experiments were approved by the local ethics committee of the Graduate School of Arts and Letters at Tohoku University and were undertaken in accordance with the Declaration of Helsinki.
Stimuli
The visual motion stimuli, which were displayed on a CRT monitor (frame rate 60 Hz), were two identical white disks (0.7° in diameter) on a black background. The two disks appeared separated by approximately 20° and moved at 10°/second toward each other horizontally (Figure 1(a)).
Procedure
The participants’ task was to watch the stream/bounce display from a distance of 40 cm with their head on a chin rest and to report verbally whether the two moving objects appeared to stream through or bounce off each other. The five pictures in Figure 1(b) show the five conditions used in Experiment 1. In the control condition, the participants rested their hands in their laps (no hands condition). In the direct touch condition, the participants put their palms together. In the indirect touch condition, the participants sandwiched a rectangular brick (width: 3 cm × depth: 10 cm × height: 20 cm; weight: 44 g) between their palms. In the no touch condition, the participants left an empty space between their palms, without the brick in between. In the brick condition, the brick was placed in front of the monitor and the participants rested their hands in their laps in the same manner as in the no hands condition. The five conditions, which consisted of 20 trials each, were conducted in a random order among the participants. Following a practice session of 10 trials using the no hands condition, all participants were presented with 100 trials (5 Conditions × 20 Trials). In the practice session, each participant watched the stream/bounce display and then reported stream through or bounce off in the same manner as in the no hands condition.
Results and Discussion
Figure 1(b) shows the results of Experiment 1. For each participant, we calculated the percentage of trials in which the bouncing perception was reported for each of the five conditions. The percentages of bouncing perception were compared using a one-way repeated measures analysis of variance (ANOVA) with the five levels of condition. There was a significant main effect of condition, F(4, 52) = 3.53, p = .013,
The increased rate of bouncing perception in the direct touch condition replicates the basic results of our previous work (Saito & Gyoba, 2016). Furthermore, the direct contact of the palms was found to be an important factor for producing the HIB effect, because the other sensory information concerning the hands and arms was almost the same among all of the conditions. In addition, we confirmed that, in the brick condition, the presence of an additional nearby visual object (the brick) had little effect on the streaming/bouncing judgment.
Experiment 2
In order to investigate the origin of the HIB effect further, Experiment 2 was designed to check whether the haptic information (the strength of force) in putting one’s palms together would modulate the stream/bounce perception. To test this possibility, we asked the participants to change the force of their hands during the meeting of their palms while they observed the stream/bounce display.
Method
Participants
Fourteen students (seven men and seven women, aged between 20 and 27 years; mean age, 21.6 years) from Tohoku University gave informed consent and participated individually in Experiment 2. All participants had normal or corrected-to-normal vision. They were naive with regard to the purpose of this study.
Stimuli
The visual stimuli were the same as those used in Experiment 1.
Procedure
In Experiment 2, the participants’ tasks were the same as those in Experiment 1, while the no hands, the weak force, and the strong force conditions were used, as can be seen in the three pictures in Figure 1(c). The procedure of the no hands and weak force conditions was the same as that of the no hands and direct touch conditions used in Experiment 1, respectively. In the strong force condition, the participants were asked to keep pushing both of their palms strongly together during the condition. Following a practice session of 10 trials of the no hands condition, all participants were presented with 60 trials (3 Conditions × 20 Trials). We measured the strength of force on the palms in the weak and strong force conditions for 20 s using a force sensor (Flexiforce with 8 Hz) using 7 of the 14 participants after the experiment. In the weak condition, the force ranged from 0 (unmeasurable by the force sensor) to 2 N (newton), while for the strong condition, the force ranged from 3.1 to 7.5 N.
Results and Discussion
Figure 1(c) shows the mean percentage of bouncing perception in each condition. These percentages were compared using a one-way repeated measure ANOVA with the three levels of condition. There was a significant main effect of condition, F(2, 26) = 6.62, p = .005,
The results of Experiment 2 clearly indicate that there was no significant effect of the strength of force when the participants put their hands together, while both conditions produced a significant HIB effect. Thus, the tactile information from the participant’s own palms plays a more crucial role in the HIB effect than the haptic information.
General Discussion
The present experiments examined whether the HIB effect mainly depends on the tactile or haptic (force) information produced when putting one’s palms together. In Experiment 1, the direct contact between the hands promoted a greater HIB effect than the indirect and no contact situation. In Experiment 2, the HIB effect was observed regardless of the strength of force during putting the palms together. Thus, our results indicate that the tactile information, especially the direct contact between the palms, promotes the perception of the collision event.
In the case of the audiovisual bounce-inducing effect, it is known that a bounce-congruent sound, such as a real impact sound, produces a higher proportion of bouncing perception compared with bounce-incongruent sounds (Grassi & Casco, 2009, 2010). Thus, the stimulus congruency plays an important role in the stream/bounce perception. We speculate that the posture with the palms together might be congruent with the bouncing event, if we consider the following points. There is a possibility that the palm-to-palm posture of the hands implies the event of a collision and such an event is usually associated with a sound in the natural environment, just like clapping. Moreover, mental imagery of such an event and a sound might modify the stream/bounce perception in this study. In fact, previous studies have indicated that the imagined sound at the overlap of the two objects promoted the bouncing perception (Berger & Ehrsson, 2013, 2017). However, it seems that the congruency suggested earlier is an indirect one and that it is weaker than the audiovisual congruency, since the bouncing perception in the audiovisual bounce-inducing effect (e.g., Kawachi & Gyoba, 2006; Watanabe & Shimojo, 2001) was generally more frequent than in the HIB effect.
The problem of decision bias should also be discussed here. Our previous study (Saito & Gyoba, 2016) demonstrated that the HIB effect has modality specific and spatially limited properties by using rubber hands and manipulating the distance between the display and the hands. The results of this study indicated that the strength of force when the participants put their hands together did not affect the magnitude of the HIB effect, while a clapping sound usually occurs with a stronger force of the hands. Thus, we think that it is difficult to explain all of these results by the single factor of decision bias, including the participant’s simple guessing. Nevertheless, it is important to conduct a future study using signal detection methodology (e.g., Grassi & Casco, 2012; Grove, Ashton, Kawachi, & Sakurai, 2012; Zeljko & Grove, 2017) in order to clarify the role of perceptual and decisional processes in the HIB effect.
It has been shown that the audiovisual bounce-inducing effect is based on multimodal networks (Bushara et al., 2003; Maniglia, Grassi, Casco, & Campana, 2012). For example, by means of functional magnetic resonance imaging, Bushara et al. (2003) found that the perception of bouncing produced by the interaction between a click sound and a visual motion stimulus was associated with increased activity in multisensory areas, whereas the perception of streaming that occurs regardless of the presence of both auditory and visual input was associated with high activity in unimodal areas. In response to these findings, it is important for future research to examine whether the HIB effect would also derive from multimodal networks, rather than from unimodal (tactile and visual) areas, in order to clarify the mechanism underlying the HIB effect.
Footnotes
Acknowledgements
The authors would like to thank Masahiro Ohka for teaching us how to use the force sensor and Massimo Grassi and Philip Grove for their helpful comments on earlier versions of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a Grant-in-Aid for JSPS Research Fellow (No. 17J06218).