I “hear” what you're “saying”: Auditory perceptual simulation,reading speed,and reading comprehension

Abstract

Auditory perceptual simulation (APS) during silent reading refers to situations in which the reader actively simulates the voice of a character or other person depicted in a text. In three eye-tracking experiments, APS effects were investigated as people read utterances attributed to a native English speaker, a non-native English speaker, or no speaker at all. APS effects were measured via online eye movements and offline comprehension probes. Results demonstrated that inducing APS during silent reading resulted in observable differences in reading speed when readers simulated the speech of faster compared to slower speakers and compared to silent reading without APS. Social attitude survey results indicated that readers’ attitudes towards the native and non-native speech did not consistently influence APS-related effects. APS of both native speech and non-native speech increased reading speed, facilitated deeper, less good-enough sentence processing, and improved comprehension compared to normal silent reading.

Keywords

Auditory perceptual simulation Embodied cognition Good-enough processing Language comprehension Reading Eye movements

Get full access to this article

View all access options for this article.

References

Alexander

J. D.

, & Nygaard

L. C.

(2008). Reading voices and hearing text: Talker specific auditory imagery in reading. Journal of Experimental Psychology: Human Perception and Performance, 34, 446–459.

Ashby

, & Clifton

Jr. (2005). The prosodic property of lexical stress affects eye movements during silent reading. Cognition, 9(3), B89–B100.

Ashby

, Sanders

L. D.

, & Kingston

(2009). Skilled readers begin processing sub-phonemic features by 80 ms during visual word recognition: Evidence from ERPs. Biological psychology, 80(1), 84–94.

Baayen

R. H.

(2008). Analyzing linguistic data: A practical introduction to statistics using R. Cambridge: Cambridge University Press.

Baddeley

, Eldridge

, & Lewis

(1981). The role of subvocalisation in reading. The Quarterly Journal of Experimental Psychology, 33(4), 439–454.

Bader

(1998). Prosodic influences on reading syntactically ambiguous sentences. Reanalysis in Sentence Processing Studies in Theoretical Psycholinguistics, 21, 1–46.

Barr

D. J.

, Levy

, Scheepers

, & Tily

H. J.

(2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278.

Barsalou

L. W.

(1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577–660.

Barsalou

L. W.

, & Hale

C. R.

(1993). Components of conceptual representation: From feature lists to recursive frames. In Van Mechelen

, Hampton

, Michalski

, & Theuns

(Eds.), Categories and concepts: Theoretical views and inductive data analysis (pp. 97–144). San Diego, CA: Academic Press.

10.

Bergen

, & Wheeler

(2010). Grammatical aspect and mental simulation. Brain & Language, 112, 150–158.

11.

Breen

(2014). Empirical investigations of the role of implicit prosody in sentence processing. Language and Linguistics Compass, 8(2), 37–50.

12.

Breen

, & Clifton

Jr. (2011). Stress matters: Effects of anticipated lexical stress on silent reading. Journal of memory and language, 64(2), 153–170.

13.

Callen

V. J.

, Callois

, & Forbes

P. A.

(1983). Evaluation Reactions to accented English: Ethnicity, Sex Role, and Context. Journal of Cross-Cultural Psychology, 14, 407–426.

14.

Carlson

, Clifton

Jr. , & Frazier

(2001). Prosodic boundaries in adjunct attachment. Journal of Memory and Language, 45(1), 58–81.

15.

Christianson

(2008). Sensitivity to Syntactic changes in garden path sentences. Journal of Psycholinguistic Research, 37, 391–403.

16.

Christianson

, Hollingworth

, Halliwell

J. F.

, & Ferreira

(2001). Thematic roles assigned along the garden path linger. Cognitive Psychology, 42, 368–407.

17.

Christianson

, & Luke

S. G.

(2011). Context strengthens initial misinterpretations of text. Scientific Studies of Reading, 15(2), 136–166.

18.

Christianson

, Luke

S. G.

, & Ferreira

(2010). Effects of plausibility on structural priming. Journal of Experimental Psychology: Learning, Memory, & Cognition, 36, 538–544.

19.

Christianson

, Williams

C. C.

, Zacks

R. T.

, & Ferreira

(2006). Misinterpretations of garden-path sentences by older and younger adults. Discourse Processes, 42, 205–238.

20.

Cutler

, Dahan

, & Van Donselaar

(1997). Prosody in the comprehension of spoken language: A literature review. Language and speech, 40(2), 141–201.

21.

De Villiers

J. G.

, Tager

H. B.

, Flusberg

, Hakuta

, & Cohenl

(1979). Children's comprehension of relative clauses. Journal of psycholinguistic Research, 8, 499–518.

22.

Drumm

A. M.

, & Klin

C. M.

(2011). When story characters communicate: readers'representations of characters’ linguistic exchanges. Memory & cognition, 39(7), 1348–1357.

23.

Edfelt

A. W.

(1960). Silent speech and silent reading. Chicago: University of Chicago Press.

24.

Edwards

J. R.

(1977). Students’ reaction to Irish regional accent. Language & Speech, 20, 280–286.

25.

Ferreira

(2003). The misinterpretation of noncanonical sentences. Cognitive Psychology, 47, 164–203.

26.

Ferreira

, Bailey

K. G. D.

, & Ferraro

(2002). Good-enough representations in language comprehension. Current Directions in Psychological Science, 11, 11–15.

27.

Ferreira

, & Patson

N. D.

(2007). The “good-enough” approach to language comprehension. Language and Linguistics Compass, 1, 71–83.

28.

Filik

, & Barber

(2011) Inner speech during silent reading reflects the reader's regional accent. PLoS ONE, 6(10), e25782. doi:10.1371/journal.pone.0025782

29.

Fodor

J. D.

(2002, April). Psycholinguistics cannot escape prosody. Paper presented at the meeting of Speech Prosody, Aix-en-Provence, France.

30.

Gass

, & Varonis

(1984). The effect of familiarity on comprehension of the non-native speech. Language and Speech, 34, 65–89.

31.

Gennari

S. P.

, & MacDonald

M. C.

(2008). Semantic indeterminacy in object relative clause. Journal of Memory and Language, 58, 161–187.

32.

Gibson

, Piantadosi

S. T.

, Brink

, Bergen

, Lim

, & Saxe

(2013). A noisy-channel account of crosslinguistic word-order variation. Psychological science, 24(7), 1079–1088.

33.

Gibson

E. A. F.

(1998). Linguistic complexity: Locality and syntactic dependencies. Cognition, 68, 1–76.

34.

Giles

(1972). Evaluation of personality content from accented speech as a function of listeners’ social attitudes. Perceptual and Motor Skills, 34, 168–170. doi:10.2466/pms.1972.34.1.168

35.

Glenberg

A. M.

, & Kaschak

(2002). Grounding language in action. Psychonomic Bulletin & Review, 9, 558–565.

36.

Glenberg

A. M.

, & Robertson

D. A.

(2000). Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning. Journal of Memory & Language, 43, 379–401.

37.

Gordon

P. C.

, Hendrick

, & Johnson

(2004). Effects of noun phrase type on sentence complexity. Journal of Memory and Language, 51, 97–114.

38.

Gordon

P. C.

, Hendrick

, Johnson

, & Lee

(2006). Similarity-based interference during language comprehension: Evidence from eye tracking during reading. Journal of Experimental Psychology: Learning, Memory and Cognition, 32, 1304–1321.

39.

Gordon

P. C.

, Hendrick

, & Levine

W. H.

(2002). Memory-load interference in syntactic processing. Psychological Science, 13, 425–430.

40.

Hubbard

T. L.

(2010). Auditory imagery: Empirical findings. Psychological Bulletin, 136, 302–329.

41.

Huey

E. B.

(1968). The psychology and pedagogy of reading. Cambridge, MA: MIT Press. (Original work published 1908).

42.

Jaeger

T. F.

(2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards Logit Mixed Models. Journal of Memory and Language, 59, 434–446.

43.

Kim

J. H.

, & Christianson

(2013). Sentence complexity and working memory effects in ambiguity resolution. Journal of Psycholinguistics Research, 42, 393–411.

44.

Kosslyn

S. M.

, & Matt

M. C.

(1977). If you speak slowly, do people read your prose slowly? Person-particular speech recoding during reading. Bulletin of the Psychonomic Society, 9, 250–252.

45.

Kurby

C. A.

, Magliano

J. P.

, & Rapp

D. N.

(2009). Those voices in your head: Activation of auditory images during reading. Cognition, 112, 457–461.

46.

Levy

B. A.

(1975). Vocalization and suppression effects in sentence memory. Journal of Verbal Learning and Verbal Behavior, 14(3), 304–316.

47.

Levy

, Bicknell

, Slattery

, & Rayner

(2009). Eye movement evidence that readers maintain and act on uncertainty about past linguistic input. Proceedings of the National Academy of Sciences, 106(50), 21086–21090.

48.

Lim

J. H.

, & Christianson

(2013a). Integrating meaning and structure in L1–L2 and L2–L1 translations. Second Language Research, 29, 233–256.

49.

Lim

J. H.

, & Christianson

(2013b). Second language sentence processing in reading for comprehension and translation. Bilingualism: Language and Cognition, 16, 518–537.

50.

Louwerse

M. M.

, & Connell

(2011). A taste of words: Linguistic context and perceptual simulation predict the modality of words. Cognitive Science, 35, 381–398.

51.

Louwerse

M. M.

, & Jeuniaux

(2010). The linguistic and embodied nature of conceptual processing. Cognition, 114, 96–104.

52.

Mecklinger

, Schriefers

, Steinhauer

, & Friederici

A. D.

(1995). Processing relative clauses varying on syntactic and semantic dimensions: An analysis with event-related potentials. Memory & Cognition, 23, 477–494.

53.

Patson

N. D.

, Darowski

E. S.

, Moon

, & Ferreira

(2009). Lingering misinterpretations in garden-path sentences: Evidence from a paraphrasing task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(1), 280–285.

54.

Pecher

, Zeelenberg

, & Barsalou

L. W.

(2003). Verifying different-modality properties for concepts produces switching costs. Psychological Science, 14, 119–124.

55.

Pecher

, Zeelenberg

, & Barsalou

L. W.

(2004). Sensorimotor simulations underlie conceptual representations: Modality-specific effects of prior activation. Psychonomic Bulletin & Review, 11, 164–167.

56.

Pecher

, & Zwaan

R. A.

(Eds.). (2005). Grounding cognition: The role of perception and action in memory, language, and thinking. New York: Cambridge University Press.

57.

Rayner

, Pollatsek

, Ashby

, & Clifton

C. E.

Jr. (2012). The psychology of reading. New York: Psychology Press. ISBN: 978-1-84872-975-9.

58.

Sachs

J. S.

(1967). Recopition memory for syntactic and semantic aspects of connected discourse. Perception & Psychophysics, 2(9), 437–442.

59.

Slattery

T. J.

, Sturt

, Christianson

, Yoshida

, & Ferreira

(2013). Lingering misinterpretations of garden path sentences arise from competing syntactic representations. Journal of Memory and Language, 69(2), 104–120.

60.

Slowiaczek

M. L.

, & Clifton

Jr. (1980). Subvocalization and reading for meaning. Journal of Verbal Learning and Verbal Behavior, 19(5), 573–582.

61.

Solomon

K. O.

, & Barsalou

L. W.

(2004). Perceptual simulation in property verification. Memory & Cognition, 2004, 32, 244–259.

62.

Speer

S. R.

, Kjelgaard

M. M.

, & Dobroth

K. M.

(1996). The influence of prosodic structure on the resolution of temporary syntactic closure ambiguities. Journal of Psycholinguistic Research, 25(2), 249–271.

63.

Stites

, Luke

S. G.

, & Christianson

(2013). The psychologist said quickly, “dialogue descriptions modulate reading speed!” Memory and Cognition, 41, 137–151.

64.

Swets

, Desmet

, Clifton

, & Ferreira

(2008). Underspecification of syntactic ambiguities: Evidence from self-paced reading. Memory & Cognition, 36(1), 201–216.

65.

Townsend

D. J.

, & Bever

T. G.

(2001). Sentence comprehension: The integration of habits and rules. Cambridge, MA: MIT Press.

66.

Traxler

M. J.

, Morris

R. K.

, & Seely

R. E.

(2002). Processing subject and object relative clauses: Evidence from eye movements. Journal of Memory and Language, 47, 69–90.

67.

Van Dyke

J. A.

, & McElree

(2006). Retrieval interference in sentence comprehension. Journal of Memory & Language, 55, 157–166.

68.

van Gompel

R. P.

, Pickering

M. J.

, Pearson

, & Jacob

(2006). The activation of inappropriate analyses in garden-path sentences: Evidence from structural priming. Journal of Memory and Language, 55(3), 335–362.

69.

Vornik

, Sharman

, & Garry

(2003). The power of the spoken word: Sociolinguistics cues influence the misinformation effect. Memory, 11, 101–109.

70.

Watson

, & Gibson

(2005). Intonational phrasing and constituency in language production and comprehension. Studia Linguistica, 59(2–3), 279–300.

71.

Weckerly

, & Kutas

(1999). An electrophysiological analysis of animacy effects in the processing of object relative sentences. Psychophysiology, 36, 559–570.

72.

Wells

J. B.

, Christiansen

M. H.

, Race

D. S.

, Acheson

D. J.

, & MacDonald

M. C.

(2009). Experience and sentence processing: Statistical learning and relative clause comprehension. Cognitive Psychology, 58, 250–271.

73.

Yao

, & Scheepers

(2011). Contextual modulation of reading rate for direct versus indirect speech quotations. Cognition, 121, 447–453.

74.

Zhou

, & Christianson

(2014, September). Mechanisms underlying auditory perceptual simulation: reading speed or speech biases? Poster presented at Architectures and Mechanisms for Language Processing. Edinburg, UK.

75.

Zhou

, & Christianson

(2015, in preparation). Comparison of attitudes towards native and non-native English speech.

76.

Zwaan

R. A.

(2004). The immersed experiencer: Toward an embodied theory of language comprehension. In Ross

B. H.

(Ed.), Psychology of learning and motivation (Vol. 44, pp. 35–62). San Diego, CA: Academic.

77.

Zwaan

R. A.

(2008). Experiential traces and mental simulations in language comprehension. In DeVega

, Glenberg

A. M.

, & Graesser

A. C.

(Eds.), Symbols, embodiment, and meaning (pp. 165–180). Oxford, UK: Oxford University Press.

78.

Zwaan

R. A.

(2009). Mental simulation in language comprehension and social cognition. European Journal of Social Psychology, 39(7), 1142–1150.

79.

Zwaan

R. A.

, & Radvansky

G. A.

(1998). Situation models in language comprehension and memory. Psychological Bulletin, 123, 162–185.