We tested the semantic coding hypothesis, which states that cross-modal interactions observed in speeded classification tasks arise after perceptual information is recoded into an abstract format common to perceptual and linguistic systems. Using a speeded classification task, we first confirmed the presence of congruence interactions between auditory pitch and visual lightness and observed Garner-type interference with nonlinguistic (perceptual) stimuli (low-frequency and high-frequency tones, black and white squares). Subsequently, we found that modifying the visual stimuli by (a) making them lexical (related words) or (b) reducing their compactness or figural ‘goodness’ altered congruence effects and Garner interference. The results are consistent with the semantic coding hypothesis, but only in part, and suggest the need for additional assumptions regarding the role of perceptual organization in cross-modal dimensional interactions.
Get full access to this article
View all access options for this article.
References
1.
BékésyG V, 1959“Similarities between hearing and skin sensations”Psychological Review661–22
2.
Ben-ArtziEMarksL E, 1995“Visual–auditory interaction in speeded classification: Role of stimulus difference”Perception & Psychophysics571151–1162
3.
Ben-ArtziEMarksL E, 1999“Processing linguistic and perceptual dimensions of speech: Interactions in speeded classification”Journal of Experimental Psychology: Human Perception and Performance251–17
4.
BernsteinI HEdelsteinB A, 1971“Effects of some variations in auditory input upon visual choice reaction time”Journal of Experimental Psychology87241–246
5.
BraatenR, 1993“Synesthetic correspondence between visual location and auditory pitch in infants”, paper presented at the Annual Meeting of the Psychonomic Society, Washington, DC, November
6.
ClarkH HBrownellH H, 1976“Position, direction, and their perceptual integrality”Perception & Psychophysics19328–334
7.
GrauJ WKemler-NelsonD G, 1988“The distinction between integral and separable dimensions: Evidence for the integrality of pitch and loudness”Journal of Experimental Psychology: General117347–370
8.
GreenE JBarberP J, 1981“An auditory Stroop effect with judgments of speaker gender”Perception & Psychophysics30459–466
9.
HattaTKirsnerK, 1983“Language processing in partial English – Japanese bilinguals: Evidence from Stroop-test results”International Journal of Neuroscience20241–248
10.
LaBergeD A, 1962“A recruitment theory of simple behavior”Psychometrika27375–396
11.
LewkowiczDTurkewitzG, 1980“Cross-modal equivalence in early infancy: Auditory – visual intensity matching”Developmental Psychology16597–607
12.
LoganG D, 1980“Attention and automaticity in Stroop and priming tasks: Theory and data”Cognitive Psychology12523–553
13.
LuceR DGreenD M, 1972“A neural timing theory for response times and the psychophysics of intensity”Psychological Review7914–57
14.
MarksL E, 1978The Unity of the Senses: Interrelations among the Modalities (New York: Academic Press)
15.
MarksL E, 1982“Bright sneezes and dark coughs, loud sunlight and soft moonlight”Journal of Experimental Psychology: Human Perception and Performance8177–193
16.
MarksL E, 1987“Auditory–visual interactions in speeded discrimination”Journal of Experimental Psychology: Human Perception and Performance13384–394
17.
MarksL E, 1996“On perceptual metaphors”Metaphor and Symbolic Activity1139–66
18.
MarksL EHammealR JBornsteinM H, 1987“Perceiving similarity and comprehending metaphor”Monographs of the Society for Research in Child Development521–93
19.
MelaraR D, 1989“Dimensional interactions between color and pitch”Journal of Experimental Psychology: Human Perception and Performance1569–79
20.
MelaraR DMarksL E, 1990a“Dimensional interactions in language processing: Investigating directions and levels of crosstalk”Journal of Experimental Psychology: Learning, Memory, and Cognition16539–554
21.
MelaraR DMarksL E, 1990b“Interaction among auditory dimensions: Timbre, pitch, and loudness”Perception & Psychophysics48169–178
22.
MelaraR DMarksL E, 1990c“Processes underlying dimensional interactions: Correspondences between linguistic and nonlinguistic dimensions”Memory and Cognition18477–495
23.
MelaraR DO'BrienT P, 1987“Interaction between synesthetically corresponding dimensions”Journal of Experimental Psychology: General16323–336
24.
OsgoodC ESuciG JTannenbaumP H, 1957The Measurement of Meaning (Urbana, IL: University of Illinois Press)
25.
PetrusicW M, 1990“Semantic congruity effects and theories of the comparison process”Journal of Experimental Psychology: Human Perception and Performance18962–986
26.
PomerantzJ R, 1983“Global and local precedence: Selective attention in form and motion perception”Journal of Experimental Psychology: General112515–540
27.
PomerantzJ R, 1985“Selective attention: Differences between Stroop and Garner interference”, paper presented at the 26th Annual Meeting of the Psychonomic Society, Boston, MA
28.
PomerantzJ R, 1986“Visual form perception: an overview”, in Pattern Recognition by Humans and Machines: Visual Perception Eds SchwabE CBusbaumH C (New York: Academic Press) pp 1–30
PristachE APomerantzJ RCarsonC E, 1987“Are Stroop and Garner interference equivalent?”, paper presented at the 58th Annual Meeting of the Eastern Psychological Association, Washington, DC, April
31.
ShorR E, 1975“An auditory analog of the Stroop test”Journal of General Psychology93281–288
32.
StevensJ CMarksL E, 1965“Cross-modality matching of brightness and loudness”Proceedings of the National Academy of Sciences of the USA51407–411
33.
VickersD, 1970“Evidence for an accumulator model of psychophysical discrimination”Ergonomics1337–58
34.
VirziR AEgethH E, 1985“Toward a translation model of Stroop interference”Memory and Cognition13304–319
35.
WagnerSWinnerECicchettiDGardnerH, 1981“‘Metaphorical’ mapping in human infants”Child Development52728–731
36.
WalkerPSmithS, 1984“Stroop interference based on the synaesthetic qualities of auditory pitch”Perception1375–81
37.
WoodC C, 1975“Auditory and phonetic levels of processing in speech perception: Neurophysiological and information processing analyses”Journal of Experimental Psychology: Human Perception and Performance13–20
