Softness perception intrinsically relies on haptic information. However, through everyday experiences we learn correspondences between felt softness and the visual effects of exploratory movements that are executed to feel softness. Here, we studied how visual and haptic information is integrated to assess the softness of deformable objects. Participants discriminated between the softness of two softer or two harder objects using only-visual, only-haptic or both visual and haptic information. We assessed the reliabilities of the softness judgments using the method of constant stimuli. In visuo-haptic trials, discrepancies between the two senses' information allowed us to measure the contribution of the individual senses to the judgments. Visual information (finger movement and object deformation) was simulated using computer graphics; input in visual trials was taken from previous visuo-haptic trials. Participants were able to infer softness from vision alone, and vision considerably contributed to bisensory judgments (∼35%). The visual contribution was higher than predicted from models of optimal integration (senses are weighted according to their reliabilities). Bisensory judgments were less reliable than predicted from optimal integration. We conclude that the visuo-haptic integration of softness information is biased toward vision, rather than being optimal, and might even be guided by a fixed weighting scheme.
AlaisD.BurrD. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14(3), 257–262. doi:10.1016/j.cub.2004.01.029
2.
BackusB. T.BanksM. S. (1999). Estimator reliability and distance scaling in stereoscopic slant perception. Perception, 28(2), 217–242. doi:10.1068/p2753
3.
TiestW. M. BergmannKappersA. M. (2006). Analysis of haptic perception of materials by multidimensional scaling and physical measurements of roughness and compressibility. Acta Psychologica, 121(1), 1–20. doi:10.1016/j.actpsy.2005.04.005
4.
TiestW. M. BergmannKappersA. M. (2008). Kinaesthetic and cutaneous contributions to the perception of compressibility. In FerreM. (Ed.), Haptics: Perception, devices and scenarios (pp. 255–264). Berlin: Springer
5.
CalvertG. A.SpenceC.SteinB. E. (Eds.). (2004). The handbook of multisensory processes. Cambridge, MA: MIT Press
6.
DiFrancoD.BeauregardG.SrinivasanM. A. (1997). The effect of auditory cues on the haptic perception of stiffness in virtual environments. Proceedings of the ASME Dynamic Systems and Control Division, 61, 17–22
7.
DrewingK.ErnstM. O. (2006). Integration of force and position cues for shape perception through active touch. Brain Research, 1078, 92–100. doi:10.1016/j.brainres.2005.12.026
8.
DrewingK.RamischA.BayerF. (2009). Haptic, visual and visuo-haptic softness judgments for objects with deformable surfaces. In EuroHaptics Conference, 2009 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics 2009. Third Joint, Salt Lake City, UT (pp. 640–645). IEEE
9.
DrewingK.WieckiT. V.ErnstM. O. (2008). Material properties determine how force and position signals combine in haptic shape perception. Acta Psychologica, 128(2), 264–273. doi:10.1016/j.actpsy.2008.02.002
10.
ErnstM. O.BanksM. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870), 429–433. doi:10.1038/415429a
11.
ErnstM. O.BulthoffH. H. (2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8(4), 162–169. doi:10.1016/j.tics.2004.02.002
12.
ErnstM. O. (2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7(5). doi:10.1167/7.5.7
13.
GepshteinS.BurgeJ.ErnstM. O.BanksM. S. (2005). The combination of vision and touch depends on spatial proximity. Journal of Vision, 5(11), 1013–1023. doi:10.1167/5.11.7
14.
HelbigH. B.ErnstM. O. (2007). Optimal integration of shape information from vision and touch. Experimental Brain Research, 179(4), 595–606. doi:10.1007/s00221-006-0814-y
15.
HelbigH. B.ErnstM. O. (2008). Visual-haptic cue weighting is independent of modality-specific attention. Journal of Vision8(1):21, 1–16. doi:10.1167/8.1.21
16.
HillisJ. M.WattS. J.LandyM. S.BanksM. O. (2004). Slant from texture and disparity cues: Optimal cue combination. Journal of Vision, 4(12), 967–992. doi:10.1167/4.12.1
17.
HollinsM.BensmaiaS.KarlofK.YoungF. (2000). Individual differences in perceptual space for tactile textures: Evidence from multidimensional scaling. Perception and Psychophysics, 62(8), 1534–1544. doi:10.3758/BF03212154
18.
JacobsR. A. (2002). What determines visual cue reliability?Trends in Cognitive Sciences, 6(8), 345–350. doi:10.1016/S1364-6613(02)01948-4
19.
KuschelM.Di LucaM.BussM.KlatzkyR. L. (2010). Combination and integration in the perception of visual-haptic compliance information. IEEE Transactions on Haptics, 3(4), 234–244. doi:10.1109/TOH.2010.9
20.
LandyM. S.MaloneyL. T.JohnstonE. B.YoungM. (1995). Measurement and modeling of depth cue combination: In defense of weak fusion. Vision Research, 35(3), 389–412
21.
MascaroS. A.AsadaH. H. (2001). Photoplethysmograph fingernail sensors for measuring finger forces without haptic obstruction. IEEE Transactions on Robotics and Automation, 17(5), 698–708
22.
NardiniM.JonesP.BedfordR.BraddickO. (2008). Development of cue integration in human navigation. Current Biology, 18(9), 689–693. doi:10.1016/j.cub.2008.04.021
23.
OrucI.MaloneyL. T.LandyM. S. (2003). Weighted linear cue combination with possibly correlated error. Vision Research, 43(23), 2451–2468
24.
PerottiV. J.ToddJ. T.LappinJ. S.PhillipsF. (1998). The perception of surface curvature from optical motion. Perception and Psychophysics, 60(3), 377–388
25.
RosasP.WagemansJ.ErnstM. O.WichmannF. A. (2005). Texture and haptic cues in slant discrimination: Reliability-based cue weighting without statistically optimal cue combination. Journal of the Optical Society of America a-Optics Image Science and Vision, 22(5), 801–809
26.
SerweS.DrewingK.TrommershäuserJ. (2009). Combination of noisy directional visual and proprioceptive information. Journal of Vision, 9(5), 28 21–14. doi:10.1167/9.5.28
27.
SrinivasanM. A.LamotteR. H. (1995). Tactual discrimination of softness. Journal of Neurophysiology, 73(1), 88–101
28.
SrinivasanM. A.BeauregardG. L.BrockD. L. (1996). The impact of visual information on the haptic perception of stiffness in virtual environments. Proceedings of the ASME Dynamic Systems and Control Division, 58, 555–559
29.
SteinB. E.MeredithM. A. (1993). The merging of the senses. Nature, 364(6438), 588–588
30.
StevensS. S. (1956). The direct estimation of sensory magnitudes – loudness. American Journal of Psychology, 69(1), 1–25. doi:10.2307/1418112
31.
van BeersR. J.SittigA. C.GonJ. J. (1996). How humans combine simultaneous proprioceptive and visual position information. Experimental Brain Research, 111(2), 253–261. doi:10.1007/BF00227302
32.
van BeersR. J.SittigA. C.GonJ. J. (1999). Integration of proprioceptive and visual position-information: An experimentally supported model. Journal of Neurophysiology, 81(3), 1355–1364
33.
WichmannF. A.HillN. J. (2001). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception and Psychophysics, 63(8), 1293–1313. doi:10.3758/BF03194544
34.
WuW. C.BasdoganC.SrinivasanM. A. (1999). Visual, haptic, and bimodal perception of size and stiffness in virtual environments. Proceedings of the ASME Dynamic Systems and Control Division, 67, 19–26
35.
YoungM. J.LandyM. S.MaloneyL. T. (1993). A perturbation analysis of depth perception from combinations of texture and motion cues. Vision Research, 33(18), 2685–2696
