Effects of classroom acoustics on speech intelligibility and response time: A comparison between native and non-native listeners

Abstract

This study evaluates the speech reception performance of native (L1) and non-native (L2) normal-hearing young adults in acoustical conditions containing varying amounts of reverberation and background noise. Two metrics were used and compared: the intelligibility score and the response time, taken as a behavioral measure of listening effort. Listening tests were conducted in auralized acoustical environments with L1 and L2 English-speaking university students. It was found that even though the two groups achieved the same, close to the maximum accuracy, L2 participants manifested longer response times in every acoustical condition, suggesting an increased involvement of cognitive resources in the speech reception process.

Keywords

Speech intelligibility response time cross-language speech perception subjective testing classroom acoustics

Introduction

The speech perception of non-native listeners (L2) under unfavorable acoustical conditions is a multi-faceted problem known to depend on the L2 language proficiency and age of acquisition, on the amount of contextual information in the speech material, on linguistic factors of the L2 language, and on the type of energetic or informational masking of noise too.^1–3 As a result, the accuracy of speech-item identification in speech-in-noise tests under critical acoustical conditions is lower for L2 relative to their native-listener, age-matched counterparts (L1).⁴ To highlight the effect of context, comparative studies employed highly predictable or unpredictable sentences and tested a panel of normal-hearing adults with various degrees of L2 proficiency.⁵ By doing so, predictions of L2 speech reception accuracy could be made based on corrections to the normative Speech Transmission Index (STI) scheme⁶ for speech intelligibility assessment. The problem of non-native listeners is especially relevant in the school settings and, for instance, in the North American countries where a substantial number of students has L2 characteristics. The issue was recently addressed in the context of speech comprehension inside typical classrooms with a panel of university students⁷ and the conclusions pointed to the need for a revision of the normative limits for heating, ventilation, and air conditioning (HVAC) noise and reverberation in classrooms to avoid L2 disadvantages.

Current standards are based on the assessment of speech intelligibility, which has to be optimized for learning environments including university classrooms.⁸ Anyway, even at a high level of accuracy in the item recognition, there might be differences in the amount of speech processing required by L2 compared to L1, and none of the literature studies so far has shaded light on this as regards specifically university classrooms.

This is the first research question of the present work. In the experiment, the cognitive-behavioral metric of the response time (RT) to the auditory speech stimulus was taken as a measure of the speech processing rate. Although not being used as a clinical measure, in the field of audiology, RT has been interpreted as a possible correlate of listening effort, since it increases when listening conditions become more difficult^9–13 or the stimulus complexity increases.^14,15 A more challenging task calls for greater cognitive processing,¹⁶ and RT may provide information on the amount of cognitive resources involved in processing the incoming signal. Thus, it was suggested¹⁷ that RT is an important factor to consider when characterizing the speech communication process. The second research question addressed in this work is the effect of room-acoustic conditions on the L1 and L2 listening effort. Both points are relevant to the assessment of the compliance of classroom acoustics to the L2 needs.

The study involves a panel of L1 and L2 university students, and, in order to focus on basic speech reception, the role of sentence context was eliminated by employing standardized word-test material. Binaural advantage was also excluded by operating monaurally. Furthermore, optimal and only slightly worse acoustical conditions were created, with the latter split into two cases having the same objective STI, but differing in reverberation and signal-to-noise (S/N) ratio. This was done to comply with realistic sound fields experienced inside university classrooms and especially to address the second research question as regards the combination of reverberation and noise.

Experimental procedures

Speech material and listening conditions

The Diagnostic Rhyme Test (DRT)¹⁸ was used for this experiment. The test material consisted of 96 phonetically balanced word pairs (i.e. cat/mat), split into 16 lists of six pairs each. For each word pair, one of the words was recorded embedded in a carrier phrase (i.e. “The next word is cat”). As restated by the ISO 9921 standard,¹⁹ the test lists were recorded in an anechoic room by five native English speakers (two male and three female). The recorded material was processed as to obtain the same level across the speakers.

The first listening condition A, with STI equal to 1, was completely anechoic without any background noise. Then, two other conditions were presented under auralization: one with reverberation dominant (condition B) and one with some reverberation mixed with noise (condition C). The noise background consisted of a classroom field recording of babble and activity noise from an adjacent corridor. The impulse responses used for auralization were collected inside two identical parallelepiped classrooms of volume V = 250 m³ with acoustically untreated (B) and treated (C) ceiling, respectively, which were occupied during measurements. The receivers were located 3.86 m far from a directional source. Moreover, the detailed description of the noise and its temporal and spectral representations is provided in an earlier work.¹¹ The reverberation times (averaged across the 500–2000 Hz octave bands) for conditions B and C were 1.08 s and 0.68 s, respectively. The S/N ratio was >+15 dB for condition B where the noise effect was thus negligible. The resulting STI for condition B was equal to 0.57. In order to achieve the same STI also for condition C, the S/N was set to +7.5 dB. Both conditions B and C corresponded to realistic acoustical scenarios experienced inside running classrooms.

Test procedure and participants

All tests took place in an anechoic room with negligible background noise (L_eq < 20 dBA). Playback of the test material was done using a single KRK Rokit 5 G2 loudspeaker, placed at head height, 1.5 m in front of the seated participant. This playback mode provided only monaural cues but was preferred with respect to headphones presentation to keep a more natural listening experience. The signal level was calibrated to 66 dB (A) at the listening position. A wireless test bench was used to manage the experiment: the system simultaneously controlled the audio rendering and the presentation of the test to the participants, recording the word choices and the RTs. The participants were given a touchscreen handset, to be used for the item selection by means of a soft pen.

During the test, participants listened to each word with carrier phrase and then selected the word that they had heard from one of three options (the two words in the word pair, or “none of the above”). This process was repeated for each pair in the list and hence the intelligibility scores (IS) could be obtained. First, each participant completed one word list from condition A as a training session, then the experiment went on by reproducing one list each from conditions A, B, and C (counterbalanced across participants with a Latin-square design). Lists and talkers were balanced across all participants. Following the listening tests, participants were asked to fill out a brief questionnaire, on which their age of acquisition, time of exposition, and self-rating of proficiency in the English language were collected, as well as information on country of origin and other languages spoken.

A total of 37 university students aged 18–33 years (mean: 21.6 years, standard deviation (SD): 4.3 years) were included in this study. All participants assessed themselves to have normal hearing ability, with no self-reported hearing loss. Based on the age of first acquisition of the English language, the participants were divided in two groups: 24 individuals (14 male and 10 female) were identified as L2 and 13 (9 male and 4 female) identified as L1. An individual was defined as L2 if he or she was not exposed to the English language until after 3 years of age. This age cut-off has previously been used to differentiate between “simultaneous bilingual” and “sequential bilingual” children, the former of whom were exposed to both L1 and L2 from birth, and the latter exposed to L2 only after entering the education system (e.g. attending nursery school) in a primarily L2-speaking country.²⁰ This delay in initiating L2 acquisition has been shown to have consequences in the ability to discriminate between languages at a phonetic level, beginning at infancy (6–12 months of age) and continuing further in childhood.²¹ Among the L2 participants included in the experiment, the most common native language was Mandarin (nine participants), followed by Korean (four participants). The other native languages spoken were Cantonese, Spanish, Farsi and Bengali (two participants each) and Arabic, Portuguese, and Romanian (one participant each). The mean age of first exposure to the English language was 9.3 years (SD: 4.3 years); the amount of time in which L2 participants have lived in a primarily English-speaking environment ranged from 6 months to 15 years.

In Figure 1, the experimental procedure is resumed.

Figure 1.

Outline of the experimental design from the participants selection to the retrieval of the IS and RT metrics. The panel on the right corresponds to a word pair (e.g. “veal” vs “feel”) presented immediately after the end of the audio playback. The response time is taken from end of audio presentation to the selection on the touchscreen by the tester. Upon selection, the chosen item changes to red background.

Results

Prior to analysis, missing data and excessively long RTs (RT > 5 s) were excluded from the data set; the latter criterion was adopted in order to remove slow outliers due to participants’ inattention, as in previous studies on RT.⁹ The procedure led to the elimination of the 0.6% of all the RT data. The statistical analysis was conducted with the R software²² assuming a significance level of 0.05. Due to the small sample size and concerns regarding the normality of the IS data (undergoing a ceiling effect in condition A), a non-parametric analysis was applied to test the main effect and the interactions of the different factors. Specifically, the univariate effect of listening condition and listeners’ group was analyzed by applying a permutation approach to a mixed-model analysis of variance (ANOVA) design, with either IS or RT as the dependent variable. The model included one between-subject variable (listeners’ group: L1 vs L2) and one within-subject variable (listening condition, A vs B vs C). In case of significant results, multiple pairwise comparisons were carried out with a permutation approach, using the Bonferroni correction to adjust the significance level.

In the case of IS, for which results are reported in Table 1, the analysis showed that the main effects of both listeners’ group (p = 0.49) and condition (p = 0.14) were not significant, as well as the interaction between them (p = 0.82).

Table 1.

Median values and interquartile ranges (IQR) for intelligibility results (%) in the three listening conditions (A: anechoic, no noise; B: reverberation only; and C: reverberant noise). Results are presented for native (L1) and non-native (L2) English listeners.

	Condition A		Condition B		Condition C
	L1	L2	L1	L2	L1	L2
Median	100	100	100	87.5	100	100
IQR	0.0	12.0	13.0	16.7	17.0	17.0

Concerning RT, the box plots of the results for each group of participants and for each condition are showed in Figure 2. The statistical analysis revealed significant main effects of listeners’ group (p = 0.02) and listening condition (p < 0.001); no interaction was found between the two factors (p = 0.58). For the main effect of listeners’ group, the post hoc comparison indicated that RTs were significantly faster for the L1 participants (median = 1.33 s, interquartile range (IQR) = 0.42 s) than for the L2 participants (median = 1.65 s, IQR = 0.60 s). As regards the main effect of condition, a statistically significant increase in the RTs was found between conditions A and B (A: median = 1.36 s, IQR = 0.41 s; B: median = 1.66 s, IQR = 0.51 s; A vs B: p < 0.001) and between conditions A and C (C: median = 1.66 s, IQR = 0.76 s; A vs C: p < 0.001). In contrast, no difference was found in RTs between conditions B and C (p = 0.86).

Figure 2.

Box plots of the response times RT (s) of native (L1) and non-native (L2) listeners for the three listening conditions (A: anechoic, no noise; B: reverberation only; and C: reverberant noise). Median values are depicted by the bold, black line; the whiskers extend to 1.5*IQR of the corresponding hinge. Outliers are plotted as points.

Discussion

The listening conditions presented in the experiment were specifically chosen to ensure a high speech reception accuracy as expected in classrooms thanks to an appropriate acoustical design. In particular for the measured STI values and the selected DRT speech material, IS higher than 95% were envisaged for the L1 participants in each case.²³ This was done to comply with the research questions, given the L2 disadvantage reported in literature under more unfavorable conditions. The choice was confirmed by the analysis of the IS results presented in section “Results”: L1 participants achieved an accuracy statistically indistinguishable from 100%, which did not vary across the listening conditions. Furthermore, no effect of listeners’ group was found on IS, indicating that with highly positive SNR, both groups achieved the same speech reception accuracy. The absence of differences between L1 and L2 participants could be further explained by the characteristics of the L2 panel of participants. In particular, although there was a reasonably wide variety in the age of acquisition and in the amount of exposure to the English language among them, the variation was constrained by the fact that all of them were already students at an English-speaking university and had passed the minimum language requirements set by the institution. Consistently, all of them reported a high self-rating of proficiency and selected a minimum of 4 points on the 5-point scale given in the questionnaire (mean: 4.73, SD: 0.53), corresponding to “I understand almost everything that is said to me.”

Another implication of the non-significant IS differences is that every possible differentiation between conditions and listeners could be determined by the behavior of RT alone. In fact, contrary to IS, the presence of a main effect of the listeners’ group on RT results provided evidence that, notwithstanding the same high-accuracy achieved, the speech processing rate of L2 participants increased. In detail, when averaged across conditions, an additional RT of 320 ms was required to non-native listeners. Recalling that a slowing down of RTs can be interpreted as an involvement of more cognitive capacity,^16,24 it can be hypothesized that this occurrence will make word recognition more effortful for L2.

Concerning the presence of differences between listening conditions for both groups, it was found that when listeners are faced with more challenging listening conditions, their RT slows down: an average increase of 300 ms was measured passing from condition A (anechoic, no noise) to the other two. Interestingly, no difference was found in the RTs measured in condition B (reverberation only) and C (reverberant noise); this finding might imply that acoustic scenarios with a similar STI impair RT results to the same extent. Anyway, it should be noted in Figure 1, the high IQR of the RT results for L2 participants in condition C, almost doubling the IQRs in the other conditions. Indeed, the presence of a background noise (even with a positive SNR) increases the individual variability in RT results. Characteristics such as attention and susceptibility to interference (that come into play in presence of noise, but not with reverberation alone²⁵) may influence this variability. Further dedicated experiments are needed to address the specific role of room acoustics and the relative effect of noise and reverberation on the listening effort of L2.

The absence of a significant interaction between the factors listeners’ group and condition suggests that the RT increase in the L2 participants does not depend on the listening condition and that it is simply carried over from condition A to the other two. In fact, looking at Figure 2, it can be seen that differences between L1 and L2 are already present in condition A, where the speech signal is not degraded by the room acoustics. Thus, it is reasonable to trace this result back to a residual and unavoidable difference in the language-processing capabilities between L1 and L2. Specifically, the slowing down of L2 responses could be explained by the basic mechanisms of L2 listening at the lexical level, which involves L1 language interference too. In general, based on the acoustical input, the process of word recognition consists in a selection among multiple candidate words that are simultaneously activated and give raise to a competition.²⁶ In the case of L2 listeners, more phantom words are elicited because their command of the second language is incomplete and also candidates from their L1 vocabulary show up.²⁷ Therefore, since L2 face the management of many more alternative words compared to L1 listeners, they suffer from longer latencies to resolve such competitions. However, it has to be remarked that, since a component of this disadvantage stems from an incomplete availability of lexical access to the L2 language, it is not warranted that L2 listeners could influence the RT increase at all by engaging further explicit cognitive resources or attention. A more specific experimental design would be needed to elucidate this point, which is beyond the scopes of this work.

It is important to recall that the present experiment involved only energetic masking (no informational masking was present due to the nature of the noise) and did not include contextual information, which is especially useful to L2. This latter point is important to transpose the above findings from the lexical level to sentence intelligibility and listening comprehension. Thus, a direct analogy cannot be established, and specific experiments should be realized to investigate this further. However, the relative simplicity of RT measurement in word-choice tests, with respect to more complex paradigms, makes this method attractive for investigating how deficits in basic L2 speech reception possibly result in loss of accuracy and fatigue during more structured listening tasks.

From the point of view of acoustical planning, it has to be remarked that using RT as a design metric might be appropriate to optimize the reception of speech especially in the cases when intelligibility is near ceiling. In fact minimizing this quantity appears as a viable strategy to operate a choice between design alternatives whose optimal intelligibility is already ascertained. In this perspective, further studies are needed to investigate how the RT metric is sensitive to both the diverse changes of room acoustics preparation and the peculiar types of noise.

Conclusion

This study was the first to investigate the performance of L2 listeners in terms of both RT and accuracy in a context of acoustical conditions characterized by almost fully intelligible words and compared them to an L1 group having equivalent characteristics. L2 subjects presumably had a high proficiency, as they fulfilled the requirements to enter university and reported high self-rated scores. Even so, L2 participants manifested higher RTs compared to L1 in each acoustical condition, suggesting an intrinsic and unavoidable increase in the cognitive engagement experienced by non-native listeners, which is carried over all the listening scenarios. Basic reasons for that can be traced back to the increase in the number of phantom words in L2 compared to L1 during word processing. Further research is however needed to highlight the effect of noisy and/or reverberant conditions on L2 speech reception. The implications of the present findings for normal classroom listening activities cannot be firmly established by this initial investigation. Reasonably, it can be hypothesized that the present disadvantage could result in a deprivation of cognitive resources for L2 in the context of communication during a sufficiently long lesson.

Footnotes

Acknowledgements

This work was developed in the framework of a collaboration between the University of Ferrara (Italy) and The University of British Columbia. The authors greatly acknowledge the late Professor Murray Hodgson for his precious support to this research and for his long-standing and invaluable contributions to the development of room acoustics studies. The authors thank Katie O’Brien for the assistance in organizing and conducting the listening tests and Professor Stefano Bonnini for his advice on the statistical analysis.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Garcia Lecumberri

Cooke

Cutler

Non-native speech perception in adverse conditions: a review. Speech Commun 2010; 52: 864–886.

Kilman

Zekveld

Hällgren

et al . The influence of non-native language proficiency on speech perception performance. Front Psychol 2014; 5: 651.

van Wijngaarden

Steeneken

HJM

Houtgast

Quantifying the intelligibility of speech in noise for non-native listeners. J Acoust Soc Am 2002; 111: 1906–1916.

Florentine

Non-native listeners’ perception of American-English in noise. In: Proceedings of the 1985 international conference on noise control engineering, Munich, 18–20 September 1985, pp. 1021–1024. Institute of Noise Control Engineering.

van Wijngaarden

Bronkhorst

Houtgast

et al . Using the Speech Transmission Index for predicting non-native speech intelligibility. J Acoust Soc Am 2004; 115: 1281–1291.

EN 60268-16:2011. Sound system equipment—part 16: objective rating of speech intelligibility by Speech Transmission Index.

Peng

Wang

LM.

Effects of noise, reverberation and foreign accent on native and non-native listeners’ performance of English speech comprehension. J Acoust Soc Am 2016; 139: 2772–2783.

ANSI/ASA S12.60:2010. Acoustical performance criteria, design requirements, and guidelines for schools, part 1: permanent schools.

Prodi

Visentin

Farnetani

Intelligibility, listening difficulty and listening efficiency in auralized classrooms. J Acoust Soc Am 2010; 128: 172–181.

10.

Houben

van Doorn-Bierman

Dreschler

WA.

Using response time to speech as a measure for listening effort. Int J Audiol 2013; 52: 753–776.

11.

Prodi

Visentin

Feletti

On the perception of speech in primary school classrooms: ranking of noise interference and of age influence. J Acoust Soc Am 2013; 133: 255–268.

12.

Pals

Sarampalis

van Rijn

et al . Validation of a simple response time measure of listening effort. J Acoust Soc Am 2015; 138: EL187–EL192.

13.

Mealings

Demuth

Buchholz

et al . The development of the Mealings, Demuth, Dillon, and Buchholz Classroom Speech perception test. J Speech Lang Hear R 2015; 58: 1350–1362.

14.

Uslar

Carroll

Hanke

et al . Development and evaluation of a linguistically and audiologically controlled sentence intelligibility test. J Acoust Soc Am 2013; 134: 3039–3056.

15.

Lewis

Schmid

O’Leary

et al . Effects of noise on speech recognition and listening effort in children with normal hearing and children with mild bilateral or unilateral hearing loss. J Speech Lang Hear R 2016; 59: 1218–1232.

16.

Hällgren

Larsby

Lyxell

et al . Evaluation of a cognitive test battery in young and elderly normal-hearing and hearing-impaired persons. J Am Acad Audiol 2001; 12: 357–370.

17.

McGarrigle

Munro

Dawes

et al . Listening effort and fatigue: what exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group “white paper.” Int J Audiol 2014; 53: 433–440.

18.

ANSI S3.2-2009. Method for measuring the intelligibility of speech over communication systems.

19.

ISO 9921:2003. Ergonomics—assessment of speech communication.

20.

McCarthy

Mahon

Rosen

et al . Speech perception and production by sequential bilingual children: a longitudinal study of voice onset time acquisition. Child Dev 2014; 85: 1965–1980.

21.

Darcy

Krüger

Vowel production and perception in Turkish children acquiring L2 German. J Phonetics 2012; 40: 568–581.

22.

R Core Team. R: “a language and environment for statistical computing.” Vienna: R Foundation for Statistical Computing, 2017, https://www.R-project.org/

23.

Steeneken

HJM

. On measuring and predicting speech intelligibility. Soesterberg: TNO Institute for Perception, 1992, p. 162.

24.

van den Tillaart-Haverkate

de Ronde-Brons

Dreschler

et al . The influence of noise reduction on speech intelligibility, response times to speech, and perceived listening effort in normal-hearing listeners. Trends Hear 2017; 21: 1–13.

25.

Kjellberg

Effects of reverberation time on the cognitive load in speech communication: theoretical considerations. Noise Health 2004; 7: 11–21.

26.

Norris

McQueen

Cutler

Competition and segmentation in spoken-word recognition. J Exp Psychol Learn 1995; 21: 1209–1228.

27.

Broersma

Cutler

Phantom word activation in L2. System 2008; 36: 22–34.