Reciprocal auditory attenuation affects looking behaviour and playing level but not ensemble synchrony: A psychoacoustical study of violin duos

Abstract

Evidence suggests that musicians may be more susceptible to developing a hearing impairment due to increased exposure to loud sounds over the lifespan. Hearing impairments can affect musical performance behaviours, yet research suggests they do not significantly affect ensemble synchrony unless the hearing loss is severe or profound. This study investigated the effect of reduced auditory feedback on ensemble synchrony, looking behaviour and playing level. Four violinists, with self-reported normal hearing, formed two duos in acoustically-isolated rooms separated by a glass window. Each player received feedback from their own and their co-performer’s playing attenuated by 0, 10, 20, 30 and 40 dB. Video recordings of their looking behaviours were coded and signed asynchronies were identified in the audio files. The strongest effects found were bi-directional changes to playing levels as a result of auditory feedback levels, which increased when a player’s own feedback was reduced and reduced when co-performer feedback was attenuated. Violinists’ looking behaviour was found to increase when co-performer feedback was attenuated by 20 dB or more relative to their own, such that they glanced more frequently and looked for longer towards their partners. There were no effects of auditory attenuation on ensemble synchrony, even with 40 dB attenuation. The results indicate that “self-to-other” sound level ratios are more likely to prompt compensatory musical performance behaviours than an individual’s hearing ability.

Keywords

attenuation auditory ensemble synchrony hearing musical performance performance behaviours violinists

Musicians have been shown to be at an increased risk of Noise Induced Hearing Loss (NIHL) due to their exposure to high sound pressure levels over long periods of time (Schink, Kreutz, Busch, Pigeot, & Ahrens, 2014; Phillips, Henrich, & Mace, 2010). In a recent national survey of musicians’ health, 47% of musicians reported experiencing some kind of hearing problem (Help Musicians UK, 2014). The Control of Noise at Work Regulations were extended in 2005 to include the musical and entertainment industries in Great Britain, putting the onus on employers to minimise risk to employees’ hearing. In practice the only realistic approach to protect orchestral musicians’ hearing is to use earplugs (Wenmaekers, Nicolai, Hornikx, & Kohlrausch, 2017). This paper concerns how reductions in auditory feedback, as an experimentally manipulated variable, affect the interpersonal behaviours of musicians and their performance together. Outside the laboratory, such reductions may be caused by a congenital or acquired hearing impairment (HI) or the use of hearing protection to reduce the risk of hearing damage.

Hearing impairments increase the threshold level of perception for intensity or loudness. Cochlear damage can also lead to loudness recruitment where not only is the threshold increased, but sounds above threshold become rapidly louder such that they “catch up” with the normal ear (Moore, 2003, p. 152). They can affect the perception of frequency and/or pitch and also temporal aspects of music, such as timing and rhythm, which hamper listeners’ ability to recognise different instruments due to the impaired perception of harmonic profiles and attack transients (Frost, 2015). In turn, the altered perception of auditory parameters in the impaired ear affects higher-level abstractions, such as our ability to segregate and integrate auditory streams of information (Bregman, 1990). For example, a common early warning sign of deafness is having trouble following a conversation in a noisy environment, which can affect people with thresholds of only 20 dB HL (Action on Hearing Loss, 2016). The perception of music can be considered as a complex form of auditory scene analysis, which a HI, and digital amplification provided by modern hearing aids, makes especially complex.

Musicians with naturally occurring, acquired or congenital deafness report relying on a variety of visual and physical cues to help them stay in time and in tune with their co-performers including seeking eye contact with co-performers and feeling vibrations (Fulford, Ginsborg, & Goldbart, 2011). For some, this dynamic sensory attending extends to sensory compensation over time but, for most, necessitates a constant shifting of attentional focus during performance itself. Participants also reported the challenge of balancing their playing level with that of co-performers (Fulford et al., 2011). Four musicians who had taken part in Fulford et al.’s 2011 study were invited to participate in a subsequent observational study and paired in flute–piano duos. Participants with congenital, profound deafness were found to use a wide range of visual cues, including musical shaping gestures in rehearsal talk (Fulford & Ginsborg, 2013). During play, these musicians also looked towards their co-performers significantly more frequently, and for longer, than the moderately deaf or hearing players, exhibiting a strongly visual sensory attending style (Fulford & Ginsborg, 2014). When both members of the duo were profoundly deaf, however, ensemble synchrony was (at times) badly affected, to the extent that it could break down completely. Whilst the players immediately, and successfully, increased their visual contact to compensate, it was inferred that profound deafness represented by hearing thresholds above 80 dB (Action on Hearing Loss, 2016) may pose a significant challenge to ensemble synchrony (Fulford, 2013), particularly when both players are deaf, even though they may be using hearing aids and able to see each other. A further observational experiment was carried out with violin players with normal hearing by using earplugs to provide ~35 dB of auditory attenuation. This time, results revealed no behavioural changes when using the plugs nor detrimental effects on ensemble synchrony, suggesting that mild or moderate levels of deafness should not adversely affect ensemble synchrony (Fulford, Ginsborg, & Goldbart, 2012).

It is possible that profoundly deaf musicians have a higher propensity to make use of visual information as a result of sensory compensation although, outside musical contexts, there is little empirical evidence to support this hypothesis. Proksch and Bavelier (2002), for example, found that deaf individuals “possessed greater attentional resources in the periphery [of the visual field] but less in the centre when compared to hearing individuals” (p. 687). Bavelier, Dye, and Hauser (2006) identified “enhanced function” effects in a small sample of congenitally deaf users of sign language, while longitudinal research has shown that visual compensation for deafness may not develop until adulthood (Rettenbach, Diller, & Sireteanu, 1999). Taken together, these studies suggest that there may be a combined effect of sensory compensation over time, and “in the moment” situational demands on sensory information. This may explain the strong visually-attending style of profoundly deaf musicians observed in musical performance (Fulford & Ginsborg, 2014).

Empirical research has also been undertaken on the influence of auditory feedback on musical ensemble synchrony with musicians who have normal hearing. Goebl and Palmer (2009) examined temporal synchrony in piano duos creating two-way, one-way and self-only auditory feedback conditions, and observed that reduced auditory feedback led to poorer synchrony and more reliance on visual cues such as finger height and head nods, suggesting that attentional shifts from the auditory to the visual are demand-sensitive and occur as the need arises. Nonetheless, it is possible for highly skilled pianists to achieve synchrony with a duo partner in the absence of auditory self-feedback when parts are in unison; where they are temporally offset, however, synchrony is negatively affected (Zamm, Pfordresher, & Palmer, 2014). Davidson and Good (2002) observed that the “gestural marking of exits and entrances” (p.197) served to co-ordinate ensemble synchrony within a string quartet, a channel best recruited in well-established ensembles where increased familiarity between players may promote more eye contact between them (Williamon & Davidson, 2002). In a recent study involving piano duos, familiarity with the co-performer’s part was shown to benefit synchrony at the phrase level but familiarity with the co-performer themselves was needed to benefit the timing of individual keystrokes (Ragert, Schroeder, & Keller, 2013). Even in the absence of any HI, musicians use the movements of their co-performers and the conductor to stay in time with each other (Boyes Braem & Bram, 2000; Luck & Sloboda, 2009).

The positive effect of congruent, cross-modal sensory perception in promoting ensemble synchrony may be explained by the contribution of different sensory modalities to rhythmic entrainment. Rhythmic entrainment relies on dynamic sensory attending to external rhythmic stimuli using synchronised attentional pulses (Grondin, 2010) and is inherently social, working better in interactive settings than artificial ones, and supporting social cohesion (Knight, Spiro, & Cross, 2016). Furthermore, entrainment is not wholly dependent on auditory information; visual contact alone produces powerful, unintentional coupling in a variety of joint action tasks (Richardson, Marsh, & Schmidt, 2005). Visual and physical cues are also strong enablers of group synchrony, especially when coupled with auditory and vestibular information (Phillips-Silver & Trainor, 2005, 2007). Keller conceptualises a special form of multi-modal sensory attending in music: “attentional resource allocation in musical ensemble performance”, in which metric cues allow musicians attentional flexibility in achieving successful ensemble performance (Keller, 2001). Keller, Novembre, and Loehr (2016) outline the ways in which self–other integration and segregation occurs and emphasise that social factors affect this process, as the findings outlined above have already indicated. To summarise, musical ensemble synchrony remains robust to changes in auditory feedback, especially with familiar co-performers, as rhythmic entrainment itself is not wholly auditory and is strongest in multi-sensory, social environments, where visual information can successfully be recruited. Although synchrony can be achieved where the performer has no feedback from his or her own playing, by using visual cues from the co-performer, contexts must be favourable and do not reflect the realities faced by musicians with HIs during interactive rehearsal and performance. Furthermore, the effects of reduced or impaired auditory feedback of both player and partner on temporal synchrony, as is the case in the presence of a HI, remain unclear.

Aims and research questions

Beyond categorical manipulations of auditory feedback as either present or absent (Goebl & Palmer, 2009; Zamm, Pfordresher, & Palmer, 2014), this study aimed to clarify the level of auditory attenuation at which ensemble synchrony in duos is compromised, and/or yields changes in looking behaviours, replicating Fulford et al.’s (2012) study in an acoustically-controlled environment. The study also aimed to quantify reported effects of auditory attenuation on musicians’ playing level (or “loudness” in lay terminology). Three research questions were posed with hypotheses deriving from the findings of previous research:

What is the effect of attenuating auditory feedback on ensemble synchrony? It was hypothesised that ensemble synchrony would worsen the more:

their own feedback was attenuated

their co-performers’ feedback was attenuated

their co-performers’ feedback was attenuated in relation to that of their own.

What is the effect of attenuating auditory feedback on looking behaviours? It was hypothesised that musicians would look towards each other the more:

their own feedback was attenuated

their co-performers’ feedback was attenuated

their co-performers’ feedback was attenuated in relation to that of their own.

What is the effect of attenuating auditory feedback on the sound levels produced by musicians? It was hypothesised that playing levels would be higher:

the more their own feedback was attenuated

the less their co-performers’ feedback was attenuated

the less their co-performers’ feedback was attenuated in relation to that of their own.

Method

Participants

Two male and two female violinists were recruited to form two, mixed duos; in both cases the male violinist opted to play the first violin part. All four players were students on BMus or MMus degree courses at the Royal Northern College of Music, aged between 23 and 26, and self-reported that they had normal hearing and no diagnosed HI.

Design

Previous studies manipulating auditory feedback and/or investigating ensemble synchrony between musicians used MIDI outputs from electronic pianos, allowing researchers to obtain and quantify temporal data (Goebl & Palmer, 2009; Keller & Appel, 2010). To address the challenge of manipulating auditory feedback from acoustic instruments in this study, each violinist played in an acoustically isolated room in the recording studio at the Royal Northern College of Music. The violinists faced each other through an observation window between the two rooms, allowing each performer to see the head, torso and arms of their co-performer as well as their violin and bow. Video cameras (Panasonic NV-GS280) were used to record all the performances. Audio-recordings of each violinist were made as follows: the AC output from a sound level meter (Brüel and Kjær Type 2231) in each room was sent to the mixing desk and a two-channel Digital Audio Tape (DAT) recorder. Auditory feedback was relayed into the ears of each violinist using in-ear headphones (TDK EB900) to allow control of an audio mix comprising the sound from the performer’s and the co-performer’s violin. The mixing desk was used to alter the auditory feedback conditions and the calibrated DAT recording was used to post-process the recordings to determine the levels played by each violinist during each performance. The in-ear headphones were inserted in the ear canal but did not provide sufficiently high attenuation from the player’s own violin. It was therefore necessary for each violinist to wear a combination of in-ear headphones and hearing defenders (Peltor H540A (L)) that, in combination, provided sufficiently high attenuation.

Acoustic tests

The following steps were taken to assess and quantify the acoustic environment for the violinists. First, the airborne sound insulation between the two studio rooms was determined using pink noise played through an omnidirectional loudspeaker in one room. The sound pressure level was measured in both rooms at positions near the performer’s heads using a Brüel and Kjær Type 4165 microphone on a Brüel and Kjær Type 2231 sound level meter. The sound insulation results in Figure 1 indicate that there was at least 43 dB of attenuation over the frequency range from 200 Hz to 2 kHz. At frequencies above 2 kHz, the sound insulation was sufficiently high that no measurements were possible. Second, sound pressure levels were estimated at the entrance to the ear canals of each violinist. A violinist was asked to play a single note (mezzo-forte) for ~3 seconds in order to measure the equivalent continuous sound pressure level, L_eq, in one-third octave bands. The measurements were made using a Head and Torso Simulator (HATS, Brüel and Kjær Type 4100), a manikin designed to record noise levels at the entrance to the ear canal that simulates the separation of human ears and includes the effect on the sound field from the head and upper body. As the experiment required violinists to wear in-ear headphones with hearing defenders, it was necessary to measure sounds at the entrance to the ear canal to estimate the sound field that would initially be attenuated by the hearing defenders. The HATS microphones were calibrated using a Brüel and Kjær Type 4230 calibrator. The head of the HATS faced forwards and the violinist stood behind it trying to maximise the distance between the HATS and their own head and body. The violin was positioned under the chin of the HATS with the body of the violin closest to the microphone in the left ear of the HATS. Figure 2 shows the sound pressure level difference between the two ears (left ear minus right ear) confirming that the levels are higher in the left ear. Figure 3 shows the absolute sound pressure levels for the left ear. Measurements are reported using bands of one-third of an octave, ranging from 200 Hz to 10 kHz because the lowest note on the violin is G3 (≈196 Hz) and above 10 kHz there is relatively low sound radiation from the violin (see Figure 3).

Figure 1.

Sound pressure level difference (in terms of L_eq) between the two studio rooms.

Figure 2.

Measured sound pressure level (L_eq) difference between the microphone in the left and right ears of the HATS (i.e. left minus right) in one-third of an octave bands measured for single notes between G3 and C7. The player was holding the violin as if being played by the HATS itself. Different colours are used to represent notes within a single octave.

Figure 3.

Measured sound pressure levels (L_eq) in one-third of an octave bands for single notes between G3 and C7 in the left ear of the HATS where the player was holding the violin as if being played by the HATS itself. Different colours are used to represent notes within a single octave.

The attenuation provided by the hearing defenders was assessed by placing the HATS in a reverberation chamber (a 123m³ volume containing diffusers) with pink noise from an omnidirectional loudspeaker. Sound pressure level measurements in one-third of an octave bands were taken using the HATS with and without the hearing defenders. The ears in the HATS do not have ear canals and it was therefore not possible to measure the attenuation provided by the combination of hearing defenders and in-ear headphones. For this reason, the attenuation of the in-ear headphones was estimated separately. Audiometric tests were carried out on four participants with and without in-ear headphones using an audiometer (Octovation Amplitude T-Series with TDH-39 audiometric earphones). This provided the attenuation for the in-ear headphones in octave bands from 125 Hz to 8 kHz. The one-third octave band results for the attenuation of the hearing defenders were converted to octave bands as follows:

X_{O B} = - 10 {log}_{10} (\frac{1}{3} \sum_{n = 1}^{3} 10^{- X_{T O B, n} / 10})

(1)

where X_OB represents an octave band value formed from the three one-third octave band values, X_TOB. Conversion to octave bands allowed a direct comparison between the hearing defenders and in-ear headphones, and an estimate could be made of the attenuation provided by the combination of hearing defenders and in-ear headphones. Figure 4 shows the attenuation provided by the hearing defenders, in-ear headphones and the combination of the two.

Figure 4.

Attenuation provided by hearing defenders, in-ear headphones and the combination of hearing defenders and in-ear headphones.

The combined attenuation of hearing defenders and in-ear headphones meant that it was feasible to use a maximum attenuation of 30 dB for the violinist’s own playing. This ensured that the average level of the radiated sound from a performer’s violin was at least 10 dB below the listening level at the ear when the combination of hearing defenders and in-ear headphones was worn. The combined attenuation of hearing defenders and in-ear headphones with the sound insulation of the studio meant that it was feasible to use greater attenuation than 30 dB for the violinist’s co-performer, so 40 dB was chosen as a reasonable lower limit that gave a sufficiently large matrix of test conditions. It was acknowledged that during the performance there would be short time periods where peaks in the sound pressure level would be much higher than the L_eq, the energy average over the time of the performance. During the performance the Fast time-weighted maximum sound pressure level, L_Fmax, was ~16 dB higher than the L_eq. It was decided that feasible attenuation based on L_eq rather than L_Fmax would be the most robust measure because (a) L_Fmax only gives an indication of the maximum level, perhaps occurring in only a few fractions of a second during a piece of music, and (b) L_eq provides a measure of the entire performance, in common with the other measured variables in this study: frequency and duration of glances and gazes between the performers and ensemble synchrony.

In practice, a violin player’s left ear receives higher sound levels than the right (Figure 2). However, in the experimental set-up used in the present study, the in-ear headphones provided the same sound pressure level to each ear. It was not considered practical to try and replicate this contralateral difference because the position of both the violin and the head usually move during performance. In addition, bone conduction from the vibration of the violin body to the inner ear could change the sound perceived by the violinist. For these reasons, the condition corresponding to playing at a “normal” sound level when wearing hearing defenders and in-ear headphones was initially set to measured values from previous recordings of the first and second violin parts using an ear simulator (Brüel and Kjær Type 4157). The violinist changed the gain on the mixing desk so that their subjective impression was that the normal situation (corresponding to 0 dB attenuation) was the same with and without the combination of hearing defenders and in-ear headphones.

Procedure

Each participant learned a new two-minute piece, Sketch, which had been commissioned from the composer Emma-Ruth Richards specifically for the study (Figure 5). The piece included tempo changes, a dynamic range between ppp and ff, and entry points for each player individually and both players together. The composer was asked to ensure that, while the violinists would take turns with melodic or rhythmic variety, both parts should be of equal importance across the entire piece. The players used a full score showing both parts while practising their own part individually to enable them to internalise the part their co-performer would be playing. They were given time to rehearse together before the experiment started.

Figure 5.

Final 12 bars of Sketch by Emma-Ruth Richards, depicting examples of tempo change, and both shared and solo entry points. One of four “entry markers”, at which temporal synchrony was coded, is highlighted with a red asterisk.

Twenty experimental conditions (see Table 1) were defined based on conditions where the player heard his or her own playing attenuated by 0, 10, 20 and 30 dB; and in which the player heard their co-performer’s playing attenuated by 0, 10, 20, 30 and 40 dB. The lower viable limit for co-performer feedback established by HATS measurements meant it was possible to use levels much lower than -30 dB for co-performer feedback so -40 dB was chosen as a further condition. The conditions were presented to each player in the duo simultaneously and in random order. Each duo performed the full piece in every condition.

Table 1.

Matrix of attenuation conditions for the player and co-performer.

		Attenuation level of player
		0 dB	10 dB	20 dB	30 dB
Attenuation level of co-performer	0 dB	1	2	3	4
	10 dB	5	6	7	8
	20 dB	9	10	11	12
	30 dB	13	14	15	16
	40 dB	17	18	19	20

Analysis

The dependent variables of ensemble synchrony, looking behaviour and overall dynamic level of playing were assessed in all 20 conditions. Post-processing from the recordings of each violinist was carried out by converting the DAT recordings to wav files which were then played using Adobe Audition and into a sound level meter (Brüel & Kjær Type 2260). Calibration tones on the DAT recording allowed measurements of (a) the A-weighted equivalent continuous sound pressure level, L_Aeq, and (b) the A-weighted Fast time-weighted maximum sound pressure level, L_AFmax. A-weighted sound pressure levels are used to represent the frequency response of human hearing where not all frequencies at the same sound pressure level are perceived as being equally loud. Baseline levels were established by calculating the mean of the sound levels produced in a practice performance and performance in the 0 dB/0 dB condition (“Condition 1”). The mean difference between the sound levels (L_Aeq) in the practice run and Condition 1 was +0.7 dB, most likely due to a slight warm-up effect as Condition 1 occurred later in the random succession of conditions. Relative average and peak loudness levels were calculated in relation to the baseline. Four points of joint entry for both parts were identified as “entry markers” and synchrony was analysed at these points. Signed asynchronies were extracted, using onsets in the waveform determined by viewing the wav files in Audacity, and calculating synchrony manually, using the first violin part as a reference; where the second violin part lagged behind, the signed asynchrony was negative, and if it “overtook” the first, the signed asynchrony was positive. The frequency and duration of the two violinists’ glances and gazes towards each other were quantified using Noldus Observer XT9 software. A “looking” state was coded manually from the point at which a performer raised their eyes from the score towards their partner to the point at which their eyes returned to the score, with an analysis resolution of 6 frames per second.

Spearman’s correlations were calculated, initially, to explore associations between the continuous dependent variables and the ordinal levels of auditory attenuation. New categorical variables were then computed to explore further any associations. Player and co-performer conditions were collapsed into three groups of auditory attenuation level; for the player these were:

0 dB attenuation (conditions 1, 5, 9, 13, 17),

Grouping of 10 and 20 dB attenuation (conditions 2, 3, 6, 7, 10, 11, 14, 15,18, 19),

30 dB attenuation (conditions 4, 8, 12, 16, 20).

For the co-performer these were:

0 dB attenuation (conditions 1, 2, 3, 4),

Grouping of 10 and 20 dB attenuation (conditions 5, 6, 7, 8, 9, 10, 11, 12), and

Grouping of 30 and 40 dB attenuation (conditions 13, 14, 15, 16, 17, 18, 19, 20).

A further categorical variable was computed to reflect the relative difference in sound level between the player and co-performer, again across three groups:

Grouping of 30, 20 and 10 dB difference (co-performer’s sound level between 30 and 10 dB higher than the player’s; conditions 2, 3, 4, 7, 8, 12).

Grouping of 0 and -10 dB difference (co-performer’s sound level equal to or 10 dB lower than the player’s; conditions 1, 5, 6, 10, 11, 15, 16, 20).

Grouping of -20, -30 and -40 dB difference (co-performer’s sound level between 20 and 40 dB lower than the player’s; conditions 9, 13, 14, 17, 18, 19).

The effects on the dependent variables of each condition were explored using unrelated analyses of variance, one-way independent ANOVAs and, where parametric assumptions were violated, Kruskal-Wallis tests. Directional a priori hypotheses were tested with planned contrasts or the Jonckheere Terpstra test with ascending groups to confirm the relationships identified.

Results

Spearman’s correlations between auditory feedback and measured variables are shown in Table 2. These indicate that there is no relationship between the attenuation of the player’s own feedback and their looking behaviour or ensemble synchrony. However, it did affect the level of their playing, with a significant negative correlation such that as auditory feedback from their own playing decreased, the musicians played more loudly. Attenuating the feedback from the co-performer revealed effects on looking behaviour such that as feedback was reduced the frequency and duration of gazes or glances towards the co-performer increased. Attenuation of feedback from the co-performer also affected the level of sound players produced such that as the level of their co-performer’s playing was decreased the more quietly they played. Similar, but stronger relationships were found between looking behaviour and playing levels and the co-performer-to-player difference level (see Figures 6 and 7). No relationship was found between the dependent variables and ensemble synchrony.

Table 2.

Spearman’s correlations between auditory feedback and measured variables.

	Auditory feedback level
	Player	Co-performer	Difference
Looking (frequency)	.154	−.297**	−.331**
Looking (duration in s)	.110	−.311**	−.315**
L_Aeq (dB)	−.281***	.344***	.469***
L_AFmax (dB)	−.251***	.188**	.320***
Signed asynchronies (ms)	−.092	−.007	.041
Unsigned asynchronies (ms)	.071	−.004	−.061

Note. *p < .05. **p < .01. ***p < .001.

Figure 6.

Box plots of sound level deviations from baseline by the difference in auditory feedback level between co-performer and player (-40 dB indicates that the co-performer feedback was attenuated by 40 dB relative to the player).

Figure 7.

Box plots of looking frequencies (number of times a performer glanced or gazed towards their co-performer during performance) and looking duration (total amount of time in seconds spent by players looking towards their co-performer) by the difference in auditory feedback level between co-performer and player.

Subsequent analyses of variance testing the effects of attenuating auditory feedback as grouped in Table 3 were used to answer the research questions as follows:

Table 3.

Results of ANOVAs and Kruskal-Wallis tests showing the effects of auditory feedback levels on dependent variable.

	Player level M (SD)			Test statistic (effect size)	p
	0 dB	10 and 20 dB	30 dB	Test statistic (effect size)	p
Looking: Frequency (-)	10.6 (7.5)	9.0 (5.6)	8.2 (4.9)	F(2, 81) = 0.84	.439
Looking: Duration (s)	12.8 (9.0)	10.9 (6.9)	10.3 (5.2)	F(2, 81) = 0.66	.522
Sound level: L_Aeq (dB)	−0.6 (1.9)	0.2 (2.2)	0.5 (2.1)	H(2) = 16.35 (r ² = .24)	.001
Sound level: L_AFmax (dB)	−0.4 (2.1)	0.7 (2.6)	0.9 (2.7)	F(2, 249) = 5.85 (ω ² = .04)	.003
Asynchronies: Signed (ms)	0.0 (0.1)	0.0 (0.1)	0.0 (0.1)	F (2, 249) = 0.52	.597
Asynchronies: Unsigned (ms)	0.1 (0.1)	0.1 (0.1)	0.1 (0.1)	H (2) = 2.88	.237
	Co–performer level M (SD)
	0dB	10 and 20 dB	30 and 40 dB
Looking: Frequency (-)	7.2 (5.3)	8.2 (4.9)	11.6 (6.9)	F(2, 81) = 8.01 (ω ² = .08)	.006
Looking: Duration (s)	8.7 (5.9)	10.2 (6.2)	14.1 (8.2)	F(2, 81) = 8.17 (ω ² = .07)	.005
Sound level: L_Aeq (dB)	1.0 (1.6)	0.4 (1.8)	−0.9 (2.3)	H(2) = 28.91 (r ² = –.34)	.001
Sound level: L_AFmax (dB)	1.1 (2.5)	0.7 (2.4)	−0.3 (2.6)	H(2) = 8.76 (r ² = –.18)	.007
Asynchronies: Signed (ms)	0.2 (1.0)	0.0 (0.8)	0.0 (0.1)	F(2, 249) = 0.64	.527
Asynchronies: Unsigned (ms)	0.1 (0.1)	0.1 (0.1)	0.1 (0.1)	H(2) = 2.27	.321
	Co–performer to player level difference M (SD)
	30, 20 and 10 dB	0 and −10 dB	−20, −30 and −40 dB
Looking: Frequency (-)	7.8 (4.9)	7.8 (5.1)	12.9 (7.0)	F(2, 81) = 9.80 (ω ² = .08)	.002
Looking: Duration (s)	9.6 (5.7)	9.6 (6.1)	15.7 (8.5)	F(2, 81) = 9.83 (ω ² = .08)	.002
Sound level: L_Aeq (dB)	1.3 (1.8)	0.0 (1.6)	−1.2 (2.3)	H(2) = 55.18 (r ² = –.48)	.001
Sound level: L_AFmax (dB)	1.7 (2.7)	0.2 (2.1)	−0.5 (2.6)	F(2, 249) = 16.67 (ω ² = .11)	.001
Asynchronies: Signed (ms)	−0.0 (0.1)	0.0 (0.1)	0.0 (0.1)	H(2) = 1.39	.499
Asynchronies: Unsigned (ms)	0.1 (0.1)	0.1 (0.1)	0.1 (0.1)	H(2) = 0.75	.688

What is the effect of attenuating auditory feedback on players’ ensemble synchrony?

No significant effects were found on ensemble synchrony when attenuating the players’ own feedback, their co-performers’, or the players’ own feedback in relation to that of their co-performer, such that none of the hypotheses was supported.

2. What is the effect of attenuating auditory feedback on players’ looking behaviours?

No significant effect was found on looking behaviours of attenuating the players’ own feedback, so the first hypothesis for looking behaviour was not supported. However, significant linear trends were found supporting the second hypothesis: as the attenuation of co-performer feedback increased so too did looking frequency, F(2, 81) = 8.01, p = .006, ω² = .08, and looking duration, F(2, 81) = 8.17, p = .005, ω² = .07. Planned contrasts revealed that effects of attenuating co-performer feedback on looking behaviour were only observed at 30 dB and 40 dB levels of attenuation: frequency: t(81) = 2.38, p = .01 (one-tailed), r = .26; duration: 40 dB t(81) = 2.24, p = .14 (one-tailed), r = .24. Looking frequency and duration was not affected by attenuating co-performer feedback by 10 dB and 20 dB. Similar but stronger effects were observed on looking behaviour and the co-performer-to-player difference level in the form of significant linear trends supporting the third hypothesis (see Figure 6): as co-performer feedback was attenuated relative to the player’s own looking frequency, looking frequency, F(2, 81) = 9.80, p = .002, ω² = .08, and looking duration also increased, F(2, 81) = 9.83, p = .002, ω² = .08, also increased. Planned contrasts revealed no significant effects on looking frequency or duration where co-performer feedback levels were 10 dB lower, equal to, or up to 30 dB higher than the player’s own feedback. However, when co-performer feedback was between 20 dB and 40 dB lower than the player’s own, there were significant increases in looking frequency: t(81) = 3.73, p < .001, r = .38; and duration: t(81) = 3.74, p < .001, r = .38.

3. What is the effect of attenuating auditory feedback on players’ sound pressure levels?

Relationships between the dependent variables and sound pressure levels supported all three hypotheses. Sound levels in terms of L_Aeq were significantly affected by attenuating the player’s auditory feedback, H(2)= 16.35, such that playing level increased with increasing attenuation, J = 12,466, z = 3.86, r = .24, supporting the first hypothesis. Likewise, a linear trend was observed for peak playing levels, L_AFmax: F(2, 249) = 5.851, p = .003, ω² = .037. Planned contrasts revealed a significant increase in peak playing levels between 0 dB and 10-20 dB, t(249) = 3.42, p < .001 (one-tailed), r = .21, but no further increase at the 30 dB attenuation level. Sound levels were also significantly affected by co-performer attenuation, L_Aeq: H(2) = 28.91; L_AFmax: H(2) = 55.18, and, as predicted, this effect was in the opposite direction such that as co-performer playing was attenuated the violinists played more quietly, L_Aeq: J = 7,005.5, z = -5.38, r = -.34; L_AFmax: J = 5,578.0, z = -7.67, r = -.48. Finally, sound levels were affected by the relative difference between co-performer and player levels such that as co-performer feedback was increasingly attenuated against their own, average sound levels, L_Aeq: J = 5,578.0, z = -7.67, r = -.48, and peak sound levels, L_AFmax: F(2, 249) = 16.67, p < .001, ω² = .11, decreased relative to baseline, yielding a bi-directional effect (see Figure 5). In other words, players increased their playing level relative to baseline when co-performer feedback was between 10 dB and 30 dB higher than their own, L_AFmax: t(249) = 5.59, p < .001 (one-tailed), r = .33, and decreased their playing level relative to baseline when it was between 20 dB and 40 dB lower than their own, L_AFmax: t(249) = 2.04, p = .043 (one-tailed), r = .13.

Discussion and conclusions

Previous research (Fulford et al., 2011) suggested that balancing an appropriate playing level with co-performers is a challenge for musicians with HIs. In the present study, attenuating auditory feedback resulted in a bi-directional effect: musicians played more loudly when their own feedback was attenuated and played more quietly when their co-performers’ playing was attenuated. Post-hoc tests suggest that players may limit compensatory increases in playing level either to maintain optimal balance or because it is not feasible to play any louder. Bi-directional effects were strongest when using an analysis of relative differences between the loudness of “self” and “other” in the duos, suggesting that the motivation to achieve the desired “self-to-other ratio” (SOR) may explain present findings, in particular with regard to adjustments to sound level at lower SORs. Ternström (1999) explored the SOR in the context of choral singing and found that, on average, choral singers preferred a SOR of 6 dB, that is, their own feedback was 6 dB higher than that of their fellow singers (Ternström, 1999, 2003). While increases of 3 dB represent a doubling of sound energy, a 10 dB increase represents a doubling in the perceived loudness. In the present study, playing levels were affected by SORs up to the grouping of 30 dB and 40 dB attenuation, suggesting that players will proportionally reduce their playing level as necessary when co-performer feedback is attenuated to maintain temporal synchrony. As performances with lower average sound levels also yielded lower maximum sound level measurements, it may be the case that by attenuating their playing level, players also attenuated their expressivity, although further research would be needed to explore this effect systematically.

Regarding looking behaviour, again, the strongest effects found were those resulting from an analysis of the relative difference of playing levels, rather than their absolute levels. The harder it was for the participants to hear their co-performers in relation to their own playing, the more they looked towards them, supporting existing research showing that reduced auditory feedback prompts increased looking behaviour (Fulford & Ginsborg, 2014; Goebl & Palmer, 2009). In the present study, post-hoc tests revealed that visual contact between players increased significantly only when SORs were equal to or less than -20 dB, that is, perceived as over twice as quiet as their own. Thus, changes to visual sensory attending may occur when reductions in playing level become insufficient to maintain an ideal SOR and when ensemble synchrony itself is at risk. In group performance, visual contact is a key strategy for maintaining ensemble synchrony when players anticipate difficulties arising from poor auditory feedback and this finding supports existing knowledge that hearing musicians alter their sensory attending style as needed during group musical performance. Although baseline levels of looking behaviour in hearing musicians may not be as high as those of congenitally deaf musicians, they nonetheless benefit from the visually perceived movements and gestures of their co-performers to maintain ensemble synchrony, supporting previous research by Davidson and Good (2002) and Williamon and Davidson (2002).

In the present study, ensemble synchrony was found to be unaffected by auditory feedback level, even when the sound pressure levels produced by co-performers were attenuated by 40 dB relative to the player’s own. The finding supports prior observations that musicians with mild or moderate deafness have little difficulty in maintaining good ensemble synchrony (Fulford, 2013). The result suggests that, by adapting playing levels to maintain a good SOR and increasing visual sensory attending where necessary, musicians can maintain temporal synchrony so as to accommodate wide variances in co-performer feedback. It is likely that the amount of physical, proprioceptive, vibrotactile and even bone-conducted sensory information that is available to musicians in interactive performance supports the robustness of temporal ensemble synchrony to adapt to relatively extreme changes in the auditory modality.

There are limitations to this study. Quantitative methods adopt positivist, realist approaches that assume that patterns in observed data can be generalised. Yet our sense of hearing, and HIs themselves, are unique to the individual and therefore more interpretivist approaches and methodologies must be adopted to explore musicians’ lived experiences of joint musical performance with a HI. In the positivist tradition, the subjective nature of human experience and behaviour leads to error that must be controlled and there are ways in which the present analysis could have been extended in this regard. Although the parts were composed to have equal importance, future work could explore effects between the two violin parts, for example if one part was slightly more challenging than the other. Analysis could systematically explore the direction of asynchronies to identify leader–follower effects. Keller et al. (2016) describe a study showing how leader–follower dynamics are mediated by shifts in the balance of self–other integration and segregation; they argue that leaders, being less adaptive, keep a steady pace via segregation whereas followers synchronise via integration. It is possible that the presentation of conditions simultaneously to both duo performers caused an interaction effect as both players either reduced or increased their playing level to compensate for manipulations to audio level. Reassuringly, the standard deviation of average playing level (L_Aeq) across all conditions was only 2.12 dB and overcompensating would have been counter-productive for the players. Other methodological issues relate to the control of auditory feedback. Audiometric data were not gathered for these participants to eliminate the possibility of undiagnosed HI. Artificial attenuation of the “normal” ear cannot fully simulate natural deafness and does not incur perceptual phenomena such as loudness recruitment or bone conduction. It was not feasible to replicate the naturally-occurring, bi-lateral differences experienced by violinists in their playing and attenuations were themselves subject to weaknesses in measurement using both defenders and in-ear headphones. Finally, it is simplistic to assume that a single “level” of attenuation or deafness governs a musician’s ability to perform in groups. A gradual-onset impairment may allow for changes to sensory attending styles to be developed over time, consciously or unconsciously to the musician, so that they remain just as capable, if not more so, in ensemble performance as an artificially deafened, hearing musician receiving objectively less auditory feedback.

The findings of the present study indicate that musicians are able to adapt their sensory styles in performance to mitigate the effects of a mild or even moderate HI, primarily by increasing their reliance on visual cues. Nonetheless, maintaining a good balance of sound in ensemble playing may be difficult with a HI and contrasting effects may result from not sufficiently being able to hear oneself (playing louder) or one’s co-performer (playing quieter). This study underlines the importance of balance in the audibility of co-performers. A uniform amplification of all available musical sound may not be helpful if it does not promote auditory stream segregation and digital hearing aids do not provide a simple solution to this challenge (Fulford, Greasley, & Crook, 2016). In future, vibrotactile technology could be used to compensate by helping musicians with HIs regulate the loudness and expressivity of their own playing compared to that of co-performers (Hopkins, Mate-Cid, Fulford, Seiffert, & Ginsborg, 2016). There is evidence, for example, that when an auditory signal is presented simultaneously as sensory information to the skin, it is perceived to be louder by an average of 1 dB (Merchel, Altinsoy, & Stamm, 2012). Similarly, Schurmann, Caetano, Jousmaki, and Hari (2004) found that participants matching an auditory probe and reference tone for loudness chose levels that were 12% lower when they were also touching a vibrating tube. The present study extends previous work by the research team exploring perception thresholds and relative pitch perception in the vibrotactile domain and the extent to which it may be possible to use vibrotactile feedback to assist musicians with HIs in ensemble musical performance.

Footnotes

Declaration of conflicting interest

The authors declare that there is no conflict of interest.

Funding

This research was funded by the Arts and Humanities Research Council (AHRC) in the United Kingdom under project AH/H008926/1 “Interactive performance for musicians with a hearing impairment”.

ORCID iD

Jane Ginsborg

References

Action on Hearing Loss. (2016). Facts and figures on hearing loss and tinnitus. Retrieved from https://www.actiononhearingloss.org.uk/about-us/media/facts-and-figures/

Bavelier

Dye

M. W. G.

Hauser

P. C.

(2006). Do deaf individuals see better? Trends in Cognitive Sciences, 10(11), 512–518.

Boyes Braem

Bram

. (2000). A pilot study of the expressive gestures used by classical orchestral conductors. In Emmorey

Lane

(Eds.), The signs of language revisited: An anthology to honor Ursula Bellugi and Edward Klima (pp. 143–167). Hillsdale, NJ: Lawrence Erlbaum Associates.

Bregman

A. S.

(1990). Auditory scene analysis. Cambridge, MA: MIT Press.

Davidson

Good

(2002). Social and musical co-ordination between members of a string quartet: An exploratory study. Psychology of Music, 30(2), 186–201.

Frost

(2015, July). Consequences of hearing impairment on the perception of music. Paper presented at the Hearing Impairment, Enjoyment and Performance of Music conference, Institute of Acoustics, London, UK.

Fulford

(2013). Interactive performance for musicians for musicians with a hearing impairment (Unpublished doctoral thesis). Royal Northern College of Music, Manchester, UK.

Fulford

Ginsborg

(2013). The sign language of music: Musical shaping gestures (MSGs) in rehearsal talk by performers with hearing impairments. Empirical Musicology Review, 8(1), 53–67.

Fulford

Ginsborg

(2014). Can you hear me? Effects of hearing impairments on verbal and non-verbal communication during collaborative musical performance. Psychology of Music, 42(6), 846–855.

10.

Fulford

Ginsborg

Goldbart

(2011). Learning not to listen: The experiences of musicians with hearing impairments. Music Education Research, 13(4), 429–446.

11.

Fulford

Ginsborg

Goldbart

(2012, July). Functions and uses of auditory and visual feedback: Exploring the possible effects of a hearing impairment on music performance. Paper presented at the 12th International Conference of Music Perception and Cognition and 8th Triennial Conference of the European Society for the Cognitive Sciences of Music, Thessaloniki, Greece.

12.

Fulford

Greasley

A. E.

Crook

(2016). Music amplification using hearing aids. Acoustics Bulletin, 41(1), 49–51.

13.

Goebl

Palmer

(2009). Synchronization of timing and motion among performing musicians. Music Perception, 26(5), 427–438.

14.

Grondin

(2010). Timing and time perception: A review of recent behavioral and neuroscience findings and theoretical directions. Attention, Perception & Psychophysics, 72(3), 561–582.

15.

Help Musicians UK. (2014). Health and well-being survey. Retrieved from https://www.helpmusicians.org.uk/news/latest-news/health-and-wellbeing-see-the-results-of-our-survey

16.

Hopkins

Mate-Cid

Fulford

Seiffert

Ginsborg

(2016). Vibrotactile presentation of musical notes to the glabrous skin for adults with normal hearing or a hearing impairment: Thresholds, dynamic range and high-frequency perception. PLOS ONE, 11(5). doi:10.1371/journal.pone.0155807

17.

Keller

P. E.

(2001). Attentional resource allocation in musical ensemble performance. Psychology of Music, 29(1), 20–38.

18.

Keller

P. E.

Appel

(2010). Individual differences, auditory imagery and the co-ordination of body movements and sounds in musical ensembles. Music Perception, 28(1), 27–46.

19.

Keller

P. E.

Novembre

Loehr

(2016). Musical ensemble performance: Representing self, other and joint action outcomes. In Cross

E. S.

Obhi

S. S.

(Eds.), Shared representations: Sensorimotor foundations of social life. Cambridge, UK: Cambridge University Press.

20.

Knight

Spiro

Cross

(2016). Look, listen and learn: Exploring effects of passive entrainment on social judgements of observed others. Psychology of Music, 45(1), 99–115.

21.

Luck

Sloboda

J. A.

(2009). Spatio-temporal cues for visually mediated synchronization. Music Perception, 26(5), 465–473.

22.

Merchel

Altinsoy

Stamm

(2012). Touch the sound: Audio-driven tactile feedback for audio mixing applications. Journal of the Audio Engineering Society, 60(1/2), 47–53.

23.

Moore

B. C. J.

(2003). An introduction to the psychology of hearing. San Diego, CA: Academic Press.

24.

Phillips

S. L.

Henrich

V. C.

Mace

S. T.

(2010). Prevalence of noise-induced hearing loss in student musicians. International Journal of Audiology, 49(4), 309–316.

25.

Phillips-Silver

Trainor

L. J.

(2005). Feeling the beat: Movement influences infant rhythm perception. Science, 308(5727), 1430. doi:10.1126/science.1110922

26.

Phillips-Silver

Trainor

L. J.

(2007). Hearing what the body feels: Auditory encoding of rhythmic movement. Cognition, 105(3), 533–546.

27.

Proksch

Bavelier

(2002). Changes in the spatial distribution of visual attention after early deafness. Journal of Cognitive Neuroscience, 14(5), 687–701.

28.

Ragert

Schroeder

Keller

P. E.

(2013). Knowing too little or too much: The effects of familiarity with a co-performer’s part on interpersonal coordination in musical ensembles. Frontiers in Psychology, 4, 368. doi:10.3389/fpsyg.2013.00368

29.

Rettenbach

Diller

Sireteanu

(1999). Do deaf people see better? Texture segmentation and visual search compensate in adult but not in juvenile subjects. Journal of Cognitive Neuroscience, 11(5), 560–583.

30.

Richardson

M. J.

Marsh

K. L.

Schmidt

R. C.

(2005). Effects of visual and verbal interaction on unintentional interpersonal coordination. Journal of Experimental Psychology: Human Perception and Performance, 31(1), 62–79.

31.

Schink

Kreutz

Busch

Pigeot

Ahrens

(2014). Incidence and relative risk of hearing disorders in professional musicians. Occupational and Environmental Medicine, 71, 472–476.

32.

Schurmann

Caetano

Jousmaki

Hari

(2004). Hands help hearing: Facilitatory audiotactile interaction at low sound-intensity levels. Journal of the Acoustical Society of America, 115(2), 830–832.

33.

Ternström

(1999). Preferred self-to-other ratios in choir singing. Journal of the Acoustical Society of America, 105(6), 3563–3574.

34.

Ternström

(2003). Choir acoustics: An overview of scientific research published to date. International Journal of Research in Choral Singing, 1, 3–11.

35.

Wenmaekers

Nicolai

Hornikx

Kohlrausch

(2017). Why orchestral musicians are bound to wear earplugs: About the ineffectiveness of physical measures to reduce sound exposure. Journal of the Acoustical Society of America, 142(5), 3154–3164.

36.

Williamon

Davidson

J. W.

(2002). Exploring co-performer communication. Musicae Scientiae, 6(1), 53–72.

37.

Zamm

Pfordresher

P. Q.

Palmer

(2014). Temporal coordination in joint music performance: Effects of endogenous rhythms and auditory feedback. Experimental Brain Research, 233(2), 607–615.