Concurrent musical pitch height biases judgment of visual brightness

Abstract

The integration of music sounds with concurrent visual scenes or objects is a occurrence in our daily lives, attracting the attention of researchers investigating how music influences our perceptions of simultaneous visuals. This study specifically investigates the role of musical pitch height in shaping our judgments of visual brightness during concurrent music–visual events. Participants were presented with pitch-modified versions of a wide range of emotional music pieces alongside various visual stimuli. The results demonstrate that lower-pitched music tends to elicit darker judgments of visual objects than higher-pitched music, when the actual visual brightness level remains constant. These findings suggest the influence of musical pitch height on introducing biases in our evaluations of visual brightness within the context of concurrent music-visual experiences, contributing to the advancement of our theoretical understanding of the complex audiovisual integration involving music in our everyday lives.

Keywords

pitch functions of music listening audiovisual multisensory integration

Get full access to this article

View all access options for this article.

References

Ansani

Marini

D’Errico

Poggi

(2020). How soundtracks shape what we see: Analyzing the influence of music on visual scenes through self-assessment, eye tracking, and pupillometry. Frontiers in Psychology, 11, Article 2242.

Baumgartner

Lutz

Schmidt

C. F.

Jäncke

(2006). The emotional power of music: How music enhances the feeling of affective pictures. Brain Research, 1075(1), 151–164.

Bhattacharya

Lindsen

J. P.

(2016). Music for a brighter world: Brightness judgment bias by musical emotion. PLOS ONE, 11(2), Article e0148959. https://doi.org/10.1371/journal.pone.0148959

Bond

Stevens

S. S.

(1969). Cross-modality matching of brightness to loudness by 5-year-olds. Perception & Psychophysics, 6(6), 337–339. https://doi.org/10.3758/BF03212787

Braun Janzen

de Oliveira

Ventorim Ferreira

Sato

J. R.

Feitosa-Santana

Vanzella

. (2023). The effect of background music on the aesthetic experience of a visual artwork in a naturalistic environment. Psychology of Music, 51(1), 16–32.

Burns

Shepp

B. E.

(1988). Dimensional interactions and the structure of psychological space: The representation of hue, saturation, and brightness. Perception & Psychophysics, 43(5), 494–507.

Chen

Y. C.

Spence

(2017). Assessing the role of the “unity assumption” on multisensory integration: A review. Frontiers in Psychology, 8, Article 445. https://doi.org/10.3389/fpsyg.2017.00445

Chiou

Rich

A. N.

(2012). Cross-modality correspondence between pitch and spatial location modulates attentional orienting. Perception, 41(3), 339–353. https://doi.org/10.1068/p7161

Choi

Lee

J. Y.

Lee

S. H.

(2018). Bottom-up and top-down modulation of multisensory integration. Current Opinion in Neurobiology, 52, 115–122. https://doi.org/10.1016/j.conb.2018.05.002

10.

Collier

W. G.

Hubbard

T. L.

(1998). Judgments of happiness, brightness, speed and tempo change of auditory stimuli varying in pitch and tempo. Psychomusicology: A Journal of Research in Music Cognition, 17(1–2), 36–55. https://doi.org/10.1037/h0094060

11.

Eitan

Timmers

(2010). Beethoven’s last piano sonata and those who follow crocodiles: Cross-domain mappings of auditory pitch in a musical context. Cognition, 114(3), 405–422. https://doi.org/10.1016/j.cognition.2009.10.013

12.

Fraser

Bradford

J. A.

(2013). Music to your brain: Background music changes are processed first, reducing ad message recall. Psychology & Marketing, 30(1), 62–75.

13.

Hagtvedt

Brasel

S. A.

(2016). Cross-modal communication: Sound frequency influences consumer responses to color lightness. Journal of Marketing Research, 53(4), 551–562. https://doi.org/10.1509/jmr.14.0414

14.

Hanser

W. E.

Mark

R. E.

Zijlstra

W. P.

Vingerhoets

A. J.

(2015). The effects of background music on the evaluation of crying faces. Psychology of Music, 43(1), 75–85.

15.

Herget

A. K.

Albrecht

(2022). Soundtrack for reality? How to use music effectively in non-fictional media formats. Psychology of Music, 50(2), 508–529.

16.

Holm

(1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70. https://www-jstor-org.web.bisu.edu.cn/stable/4615733

17.

Hong

Y. J.

Choi

Lee

(2023). When background music in audiovisual advertisements can boost the perceived competence of the advertised brands—An empirical study from South Korea. Asia Pacific Journal of Marketing and Logistics, 35, 1991–2011.

18.

Huang

Labroo

A. A.

(2020). Cueing morality: The effect of high-pitched music on healthy choice. Journal of Marketing, 84(6), 130–143. https://doi.org/10.1177/0022242918813577

19.

Hubbard

T. L.

(1996). Synesthesia-like mappings of lightness, pitch, and melodic interval. The American Journal of Psychology, 109(2), 219–238. https://doi.org/10.2307/1423274

20.

International Organization for Standardization. (2003). Acoustics: Normal equal-loudness-level contours (ISO 226:2003). https://www.iso.org/standard/34222.html

21.

Laeng

Endestad

(2012). Bright illusions reduce the eye’s pupil. Proceedings of the National Academy of Sciences of the United States of America, 109(6), 2162–2167. https://doi.org/10.1073/pnas.1118298109

22.

Logeswaran

Bhattacharya

(2009). Crossmodal transfer of emotion by music. Neuroscience Letters, 455(2), 129–133.

23.

Lowe

M. L.

Haws

K. L.

(2017). Sounds big: The effects of acoustic pitch on product perceptions. Journal of Marketing Research, 54(2), 331–346. https://doi.org/10.1509/jmr.14.0300

24.

Ludwig

V. U.

Adachi

Matsuzawa

(2011). Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humans. Proceedings of the National Academy of Sciences of the United States of America, 108(51), 20661–20665. https://doi.org/10.1073/pnas.1112605108

25.

Maeda

Kanai

Shimojo

(2004). Changing pitch induced visual motion illusion. Current Biology, 14(23), R990–R991. https://doi.org/10.1016/j.cub.2004.11.018

26.

Marks

L. E.

(1974). On associations of light and sound: The mediation of brightness, pitch, and loudness. The American Journal of Psychology, 87(1–2), 173–188. https://doi.org/10.2307/1422011

27.

Marks

L. E.

(1987). On cross-modal similarity: Auditory–visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 384–394. https://doi.org/10.1037/0096-1523.13.3.384

28.

Marks

L. E.

(1989). On cross-modal similarity: The perceptual structure of pitch, loudness, and brightness. Journal of Experimental Psychology: Human Perception and Performance, 15(3), 586–602. https://doi.org/10.1037/0096-1523.15.3.586

29.

Marks

L. E.

Hammeal

R. J.

Bornstein

M. H.

Smith

L. B.

(1987). Perceiving similarity and comprehending metaphor. Monographs of the Society for Research in Child Development, 52(1), i–100. https://doi.org/10.2307/1166084

30.

Mathôt

Grainger

Strijkers

(2017). Pupillary responses to words that convey a sense of brightness or darkness. Psychological Science, 28(8), 1116–1124. https://doi.org/10.1177/0956797617702699

31.

Meier

B. P.

Robinson

M. D.

Crawford

L. E.

Ahlvers

W. J.

(2007). When “light” and “dark” thoughts become light and dark responses: Affect biases brightness judgments. Emotion, 7(2), 366–376.

32.

Melara

R. D.

(1989a). Dimensional interaction between color and pitch. Journal of Experimental Psychology: Human Perception and Performance, 15(1), 69–79.

33.

Melara

R. D.

(1989b). Similarity relations among synesthetic stimuli and their attributes. Journal of Experimental Psychology: Human Perception and Performance, 15(2), 212–231.

34.

Mishra

Martinez

Sejnowski

T. J.

Hillyard

S. A.

(2007). Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. The Journal of Neuroscience, 27(15), 4120–4131.

35.

Mondloch

C. J.

Maurer

(2004). Do small white balls squeak? Pitch-object correspondences in young children. Cognitive, Affective, & Behavioral Neuroscience, 4(2), 133–136.

36.

Nosal

A. P.

Keenan

E. A.

Hastings

P. A.

Gneezy

(2016). The effect of background music in shark documentaries on viewers’ perceptions of sharks. PLOS ONE, 11(8), Article e0159279. https://doi.org/10.1371/journal.pone.0159279

37.

Plack

C. J.

Oxenham

A. J.

Fay

R. R.

(2006). Pitch: Neural coding and perception. Springer.

38.

Sekuler

A. B.

Lau

(1997). Sound alters visual motion perception. Nature, 385(6614), 308–308.

39.

Shams

Kamitani

Shimojo

(2000). What you see is what you hear. Nature, 408(6814), 788–788.

40.

Spence

(2011). Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73(4), 971–995. https://doi.org/10.3758/s13414-010-0073-7

41.

Spence

(2020). Assessing the role of emotional mediation in explaining crossmodal correspondences involving musical stimuli. Multisensory Research, 33(1), 1–29. https://doi.org/10.1163/22134808-20191469

42.

Steffens

(2020). The influence of film music on moral judgments of movie scenes and felt emotions. Psychology of Music, 48(1), 3–17.

43.

Sunaga

(2018). How the sound frequency of background music influences consumers’ perceptions and decision making. Psychology & Marketing, 35(4), 253–267. https://doi.org/10.1002/mar.21084

44.

Takeshima

Gyoba

(2013). Changing pitch of sounds alters perceived visual motion trajectory. Multisensory Research, 26(4), 317–332. https://doi.org/10.1163/22134808-00002422

45.

Teramoto

Hidaka

Sugita

(2010). Sounds move a static visual object. PLOS ONE, 5(8), Article e12255.

46.

Toiviainen

Eerola

(2016). MIDI toolbox 1.1. https://github.com/miditoolbox/1.1

47.

Vatakis

Spence

(2007). Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli. Perception & Psychophysics, 69(5), 744–756. https://doi.org/10.3758/BF03193776

48.

Vieillard

Peretz

Gosselin

Khalfa

Gagnon

Bouchard

(2008). Happy, sad, scary and peaceful musical excerpts for research on emotions. Cognition and Emotion, 22(4), 720–752. https://doi.org/10.1080/02699930701503567

49.

Watanabe

Shimojo

(2001). When sound affects vision: Effects of auditory grouping on visual motion perception. Psychological Science, 12(2), 109–116.

50.

Weissgerber

T. L.

Milic

N. M.

Winham

S. J.

Garovic

V. D.

(2015). Beyond bar and line graphs: Time for a new data presentation paradigm. PLOS Biology, 13(4), Article e1002128. https://doi.org/10.1371/journal.pbio.1002128

51.

Wicker

F. W.

(1968). Mapping the intersensory regions of perceptual space. The American Journal of Psychology, 81(2), 178–188.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.14 MB