The Cognitive Neuroscience of Crossmodal Correspondences

Abstract

In a recent article, N. Bien, S. ten Oever, R. Goebel, and A. T. Sack (2012) used event-related potentials to investigate the consequences of crossmodal correspondences (the “natural” mapping of features, or dimensions, of experience across sensory modalities) on the time course of neural information processing. Then, by selectively lesioning the right intraparietal cortex using transcranial magnetic stimulation, these researchers went on to demonstrate (for the first time) that it is possible to temporarily eliminate the effect of crossmodal congruency on multisensory integration (specifically on the spatial ventriloquism effect). These results are especially exciting given the possibility that the cognitive neuroscience methodology utilized by Bien et al. (2012) holds for dissociating between putatively different kinds of crossmodal correspondence in future research.

Keywords

crossmodal correspondence cognitive neuroscience ventriloquism effect TMS ERP

References

Bien

ten Oever

Goebel

Sack

A. T.

(2012). The sound of size: Crossmodal binding in pitch-size synesthesia: A combined TMS, EEG, and psychophysics study. NeuroImage, 59, 663–672. doi:10.1016/j.neuroimage.2011.06.095 ◂

Bremner

Caparos

Davidoff

de Fockert

Linnell

Spence

(in Press). Bouba and Kiki in Namibia? Western shape-symbolism does not extend to taste in a remote population. Manuscript submitted for publication ◂

Ernst

M. O.

(2007). Learning to integrate arbitrary signals from vision and touch. Journal of Vision, 7(5), 7.1–14. doi:10.1167/7.5.7 ◂

Gallace

Spence

(2006). Multisensory synesthetic interactions in the speeded classification of visual size. Perception & Psychophysics, 68, 1191–1203. doi:10.3758/BF03193720 ◂

Köhler

(1929). Gestalt psychology. New York, NY: Liveright ◂

Kovic

Plunkett

Westermann

(2009). The shape of words in the brain. Cognition, 114, 19–28. doi:10.2298/PSI0901005K ◂

Marks

L. E.

(2004). Cross-modal interactions in speeded classification. In Calvert

G. A.

Spence

Stein

B. E.

(Eds.), Handbook of multisensory processes (pp. 85–105). Cambridge, MA: MIT Press ◂

Muggleton

Tsakanikos

Walsh

Ward

(2007). Disruption of synaesthesia following TMS of the right posterior parietal cortex. Neuropsychologia, 45, 1582–1585. doi:10.1016/j.neuropsychologia.2006.11.021 ◂

Oberman

L. M.

Ramachandran

V. S.

(2008). Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: The mirror neuron hypothesis. Social Neuroscience, 3, 348–355. doi:10.1080/17470910701563681 ◂

10.

Parise

C. V.

Spence

(2009). When birds of a feather flock together: Synesthetic correspondences modulate audiovisual integration in non-synesthetes. PLoS One, 4(5), e5664. doi:10.1371/journal.pone.0005664 ◂

11.

Parise

C. V.

Spence

(in press). Audiovisual crossmodal correspondences. To appear in Simner

Hubbard

(Eds.), Oxford handbook of synaesthesia. Oxford, England: Oxford University Press ◂

12.

Ramachandran

V. S.

Hubbard

E. M.

(2003). Hearing colors, tasting shapes. Scientific American, 288(May), 43–49. doi:10.1038/scientificamerican0503-52 ◂

13.

Ramachandran

V. S.

Oberman

L. M.

(2006). Broken mirrors. Scientific American, 295(5), 62–69. doi:10.1038/scientificamerican0503-52 ◂

14.

Sadaghiani

Maier

J. X.

Noppeney

(2009). Natural, metaphoric, and linguistic auditory direction signals have distinct influences on visual motion processing. Journal of Neuroscience, 29, 6490–6499. doi:10.1523/JNEUROSCI.5437-08.2009 ◂

15.

Seo

H.-S.

Arshamian

Schemmer

Scheer

Sander

Ritter

(2010). Cross-modal integration between odors and abstract symbols. Neuroscience Letters, 478, 175–178. doi:10.1016/j.neulet.2010.05.011 ◂

16.

Spence

(2011). Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73, 971–995. doi:10.3758/s13414-010-0073-7 ◂

17.

Spence

(2012). Managing sensory expectations concerning products and brands: Capitalizing on the potential of sound and shape symbolism. Journal of Consumer Psychology, 22, 37–54. doi:10.1016/j.jcps.2011.09.004 ◂