Evaluation of the Influence of Head Movement on Hearing Aid Algorithm Performance Using Acoustic Simulations

Delay-and-Subtract Beamformer

The D&Sb is a fixed (nonadaptive) monaural beamformer, which was applied to the left and right hearing aid microphone recordings separately. On each side, it consisted of a first-order differential microphone using the front and rear omnidirectional microphones that were separated by a distance of 14.9 mm. The rear microphone signal was delayed by the time it takes sound to travel over the microphone separation distance (4.3824e–5 s) and subtracted from the front signal to achieve a cardioid pattern. The delay was achieved by applying a corresponding linear phase in the frequency domain using an overlap-add method as described in Grimm et al. (2006). The resulting high-pass characteristics were not equalized.

Adaptive Differential Microphone

The ADMb is a monaural algorithm based on two D&Sbs using a single pair of omnidirectional microphones as described earlier. These two beamformers generated a front-facing and a back-facing microphone signal (Elko & Pong, 1995). A mixing weight was adapted to steer a spatial zero toward the most prominent sound source in the rear hemisphere, as the target was assumed to be in the front and the distractor in the back hemisphere. The optimum value of the mixing weight was found by minimizing the mean-square amplitude value of the output. The algorithm was applied to the left and right hearing aid microphone recordings separately.

Static Binaural Beamformer

The binaural six-microphone beamformer (SBb) algorithm (Rohdenburg et al., 2007) aims to minimize the overall noise output power while preserving the desired speech component in the frontal hearing aid microphone channels. It therefore assumed that the target was coming from the front. The beamformer is a fixed minimum variance distortionless response (MVDR) beamformer without a generalized sidelobe canceller. Binaural cues of both target and noise signals were preserved by applying a real-valued time-variant postfilter of the front microphones controlled by beamformer output, instead of returning the beamformer output directly. A wave propagation model with a sampled head-related impulse responses for frontal direction from the database described in (Kayser et al., 2009) was used as propagation vector.

Adaptive MVDR Beamformer

This is the adaptive version (AMVDRb) of the binaural multimicrophone beamformer algorithm described earlier (Rohdenburg et al., 2007). It used the generalized sidelobe canceler, which could adapt to a varying noise field, and used only four microphones instead of six (front and rear on each side). In our version, the beamformer output was returned directly, resulting in a diotic output signal.

Binaural Noise Reduction

The BNR scheme filtered out signals based on their interaural coherence (Grimm et al., 2009; Luts et al., 2010). The algorithm used the left and right front hearing aid microphones. Target sources were assumed to have a high interaural coherence because their position changes only slowly and their direct signal path dominates the reverberant part (at least within the critical distance; Grimm et al., 2009). Distractor sources were assumed to be incoherent because they are either beyond the critical distance or consist of many similar uncorrelated sources distributed around the listener. This algorithm was thus designed to filter out diffuse noise. The interaural coherence was estimated from fluctuations in the interaural phase difference, in third-octave bands with a time constant of 40 ms. From the measure of coherence, the attenuation for the corresponding frequency band and time frame was calculated so that incoherent signals were filtered out.

Single-Channel Noise Reduction

The SCNR algorithm (Breithaupt et al., 2008) used the left and right front hearing aid microphones. The algorithm aimed to improve the SNR by identifying speech components in the signal and filtering out nonspeech components. This was done by applying temporal smoothing in the cepstral domain. In the cepstral domain, the noisy speech signal was decomposed into coefficients related to the speech envelope, the excitation, and noise. Cepstral coefficients related to the speech excitation were identified using fundamental frequency estimation. Strong temporal adaptive recursive smoothing was applied to the cepstral coefficients that are dominated by noise and little smoothing to the cepstral coefficients representing speech.

Head Movement Traces

The simulated head movement traces were synthesized such that the azimuth followed a cosine ramp from the starting angle to the target angle (0°). Although a listener’s head movements might be better characterized by a sigmoidal curve (Brimijoin et al., 2014), a cosine was chosen here, because a precise time was needed for when the movement ended. The head movement traces started between −180° and 180°, in steps of 30° away from the target and ended at 0°, facing the target. In all cases, the head stayed oriented toward the initial direction for 5 s before the movement started, to give the hearing aid algorithms time to adapt. Peak velocities of 50 deg/s, 100 deg/s and 150 deg/s were implemented, as ±50 deg/s has been reported as the average rotational head velocity for small movements (±30°) and ±150 deg/s is the average rotational head velocity for larger movements (±105°; Brimijoin et al., 2010). In total, 37 traces were created of which one trace was the control trace without movement (i.e., starting facing the target; see Figure 2).

OPEN IN VIEWER

Figure 2.

Simulated Cosine Head Movement Traces That Were Implemented. The parameters starting angle [−180°:30°:180°] and peak velocity [50 deg/s, 100 deg/s, 150 deg/s] were changed to create a total of 37 traces, including a control trace without movement, as indicated by different colors. A 200 ms analysis window is located immediately after the end of the movement in which the algorithm performance related to noise suppression is analyzed.

Stimuli

The target signals were 20 fragments from the DAPS database (Mysore, 2015), using 10 male and 10 female speakers. The fragments were chosen so that the signals in the analysis windows were between +1 dB and +5 dB above the root-mean-squared value for the whole signal without pauses. This was to make sure that the signal in the analysis windows contained speech, not just speech pauses. Noise sources were either unmodulated speech-shaped noise or distractor speakers. Speech-shaped noise signals were created using the long-term average speech spectrum of the target signals. Multiple signals were created so that uncorrelated speech-shaped noise sources could be presented simultaneously. The distractor speakers were different fragments from the DAPS database, always of the opposite sex to the target speaker. The noise signal was started 200 ms before the target signal, allowing for better algorithm adaptation. All scenarios were measured at −5 dB and +5 dB long-term input SNR because the SNR can also influence the algorithm benefit.

Database of Hearing Aid Head-Related Impulse Responses

Hearing aid head-related impulse responses were used that matched the loudspeaker positions for a circular array of 90 loudspeakers (starting at 0° azimuth with 4° spacing and at 0° elevation). They were taken from the database of Thiemann et al. (2015).

Algorithm Settings

To study the effect of the adaptation time on maladaptation, different settings were tested for the two adaptive beamformers (ADMb and AMVDRb). For the ADMb, the parameter µ, which determines the adaptation step size, was set to 1e−4 (standard), 1e−5, and 1e−6. A smaller value for µ results in slower adaptation: respectively, 0.23, 2.3, and 23 s. The values were chosen so that the approximate adaptation time fits the three categories of faster than movement duration, same order of magnitude as movement duration, and slower than movement duration.

For the AMVDRb, two parameters affect the adaptation time: µ has a direct effect on the adaptation time, where larger values result in a faster adaptation, and α influences the variation of the adaptation across time. In the experiment, only µ was changed (0.4, 0.04, and 0.0004) and α was kept constant at the default value of 0.5. For this more complex algorithm, calculation of the approximate adaptation time for the different settings is not straightforward, but it can be estimated from the effect duration (Figure 9).

Analysis

First, the SNR improvement was calculated for the static control condition. From this, it could be seen how well the algorithms worked in each scenario. The movement effect was quantified by calculating the difference in output SNR (termed “ΔSNR”) between the static control condition and the movement conditions. A negative ΔSNR indicated that any improvement in SNR was reduced by the head movement. Situations with a large negative ΔSNR were selected for further investigation by plotting ΔSNR for different peak velocities and start angles. Finally, to analyze how long it took for the algorithms to recover from maladaptation, new recordings were made in which further time was added between the end of the movement and the start of the analysis window while keeping the same signal in the analysis window.

Effect of Head Movement Parameters

For the ADMb and AMVDRb algorithms, a significant negative effect of head movement was found, and this effect was largest in the anechoic scenario with one noise source at −5 dB long-term SNR. In this scenario, the influence of the movement parameters, start angle, and peak velocity on ΔSNR was further investigated. The influence of the movement parameters is shown in Figure 6 for ADMb and in Figure 7 for AMVDRb. For ADMb with the intermediate setting, both the start angle and peak velocity had an influence on ΔSNR, whereas for the slowest setting only the start angle had an influence. For the AMVDRb algorithm, the peak velocity had a larger influence for the intermediate setting, and the start angle had a larger influence for the slowest setting, similar to the ADMb algorithm. The fastest setting also gave a significant ΔSNR, which was influenced mainly by the peak velocity.

OPEN IN VIEWER

Figure 6.

Influence of Movement Start Angle and Peak Velocity on the ΔSNR for the ADMb in Selected Situations. The ΔSNR is plotted for different start angles of the movement (x-axis) and peak velocities (line color). The movement traces indicated with the black circle are selected for the analysis of the effect duration in the next section. ADMb = adaptive differential microphone; SNR = signal-to-noise ratio.

OPEN IN VIEWER

Figure 7.

As per Figure 6, but for AMVDRb. AMVDRb = adaptive minimum variance distortionless response beamformer; SNR = signal-to-noise ratio.

Effect Duration

To study how long the algorithms took to recover from movements, further recordings were made in which extra time was added between the end of the movement and the start of the analysis window. This was done for the individual movement traces that were shown in Figures 6 and 7: movement starting at −60° and +60° with 150 deg/s for the ADMb and movement starting at +120° and +60° with 150 deg/s for the AMVDRb. The ΔSNR in the shifted analysis windows was plotted against the time shift on a logarithmic axis (Figures 8 and 9). We found that the intermediate and slowest settings, having longer adaptation times, took longer to recover; for the slowest setting, it could take 2 or more seconds to recover. The different settings for the ADMb and AMVDRb algorithms show similar recovery times. It is noticeable that the AMVDRb algorithm did not fully recover in the anechoic scenario with one noise source for the movement trace starting at +120° (facing the noise source) with a peak velocity of 150 deg/s. This means that the AMVDRb algorithm potentially has problems in real life when the attention of the hearing aid user is shifted to a different source, but this needs to be checked in a more realistic scenario.

OPEN IN VIEWER

Figure 8.

Movement Effect Duration for the ADMb Algorithm in the Selected Situations and for the Different Settings: Fast Setting (1), Intermediate Setting (2), and Slow Setting (3). Plotted are the median ΔSNR (circles), the 25th and 75th percentiles (thick lines), and the range (thin vertical lines) over all target signals for increasing times between the end of the movement and the start of the analysis window (x-axis, logarithmic) and different algorithm settings (line color). ADMb = adaptive differential microphone; SNR = signal-to-noise ratio.

OPEN IN VIEWER

Figure 9.

As Figure 8, but for AMVDRb in the Selected Situations and for the Different Settings: Fast Setting (1), Intermediate Setting (2), and Slow Setting (3). AMVDRb = adaptive minimum variance distortionless response beamformer; SNR = signal-to-noise ratio.

Virtual Environments

For this experiment, we used head movement data and VEs from Hendrikse et al. (2019b). The six VEs used in this experiment were as follows: a cafeteria (including both a single-task condition and a dual-task condition having an additional hand-eye-coordination task), a lecture hall, a living room, a street, and a train station. In all VEs, the participants had to listen to a specified target, while their head movements were measured. Brief descriptions of the environments are given in the following paragraphs, more details and a technical analysis can be found in (Hendrikse et al., 2019b). The range of SNR in the VEs is plotted in Figure 10 (left panel).

OPEN IN VIEWER

Figure 10.

Input SNR Mean Across Participants (Left) and Range Across Participants (Right). The left panel shows the dynamics of the input SNR over time. The right panel shows the difference in input SNR due to head movement. SNR = signal-to-noise ratio.

Cafeteria: the listener was sitting at the edge of a table at which four persons (azimuths of −28°, −4°, 8°, and 34°, negative azimuths are to the right) were having a conversation. The noise consisted of several point noise sources, such as competing conversations at neighboring tables, music and laughter. Diffuse noise, consisting of babble noise and noise of plates and cutlery, was also present. In the cafeteria_{listeningonly} condition, the only task was to listen to the four-person conversation at the table. In the cafeteria_dualtask condition, the task was to listen to the conversation and at the same time put pins in the holes on a Purdue Pegboard (Tiffin & Asher, 1948), to simulate eating and listening at the same time. In this situation, performing the dual task significantly influenced the head movement behavior (Hendrikse et al., 2019b).

Lecture hall: the listener was sitting in the audience while a lecture was given. The lecturer’s speech (target) was presented directly (−15°) and through loudspeakers (51° and −38°). Presentation slides were shown on a screen (25°). The noise level was low and consisted of occasional noises from the audience, such as coughing, sneezing, sighing and noise of pencil writing, and turning pages.

Living room: the listener was sitting on a sofa while the news was playing on a TV (target, at an azimuth of −4°). A person sitting on a chair to the left (45°) was occasionally commenting on the news. A person sitting on the sofa next to the listener was eating crisps (−90°). On the left side of the listener was an active fire place. Through the open door to the kitchen, noises of the dishwasher, water cooker, and fridge could be faintly heard.

Street: the listener was standing at a bus stop, where four people (−17°, 4°, 23°, and 42°) were having a conversation (target). On the sidewalk, a bicycle was driving by and a mother with pram was walking by while singing lullabies to her baby. On the road, cars were driving past, as well as a truck, a bus, and an emergency vehicle. Diffuse noise consisted of distant traffic noise and birds singing. There was also a train driving past in the distance.

Train station: the listener was standing on the platform. Occasionally, announcements were made over the loudspeaker system about trains arriving/departing (target). The noise consisted of a conversation on a neighboring platform (98°), people walking past with trolleys, beeps of ticket validation machines, train engines, and brakes. Diffuse noise was a recording from a real train station.

Database of Hearing Aid Head-Related Impulse Responses

For this experiment, we wanted to make sure that our simulated signals represented the signals that were played to the participants while recording their head movement. Therefore, hearing aid head-related impulse responses were recorded in the setup described in Hendrikse et al. (2019b). This setup consisted of 28 loudspeakers that were divided between a 16-loudspeaker horizontal ring array at ear level (first loudspeaker at 11.25° from frontal direction, with 22.5° spacing) and two 6-loudspeaker ring arrays at +45° and −45° elevation (first loudspeaker at 0° and 30° azimuth from frontal direction, respectively, with 60° spacing). Four subwoofers were positioned on the floor at 45°, 135°, −135°, and −45°. The room in which this setup was placed was sound-treated and had a reverberation time (T60) of 0.13 s and an early decay time of 0.04 s. The impulse responses were recorded from the position of the target source in the virtual environments at a head-and-torso simulator with multichannel hearing aid dummies behind the ears as described earlier. This was standing at the position of the participant in the laboratory. In the environments in which the participants were seated, they were sitting on a chair on a platform. Two sets of impulse responses were recorded (with and without platform) to match the environments in which the participants were seated as well as those where they were standing. A logarithmic frequency sweep (Farina et al., 2001) was used to eliminate nonlinear distortions of the loudspeakers.

Analysis

First, we checked whether the observed variation across the movement traces of the participants led to differences in SNR because of the head shadow effect. Therefore, differences in input SNR caused by the different movement traces were analyzed.

Subsequently, the algorithm benefit was analyzed to see whether there was a variance over the different movement traces. However, potentially there were also differences due to the movement in the input SNR. It could be possible that the algorithms just compensate for the differences in input SNR, which would result in the same output SNR for all movement traces. This would mean that head movement does not affect the performance of the algorithms. To check this, the variance in output SNR over the movement traces was also analyzed. To examine the hypothesis of differences in directional algorithm performance between older and younger people, a repeated-measures ANOVA was carried out for the mean algorithm benefit over time with the environment and algorithm type as between-participant factors and the age-group as within-participant factor.

Finally, in order to find out how much of the variance in algorithm benefit could be contributed to misalignment and maladaptation, the algorithm benefit was plotted per rotational head speed and per head direction. A potential dependency of the algorithm benefit on the rotational head speed would be caused by maladaptation. A potential dependency of the algorithm benefit on the head direction would be caused by the head shadow effect, which affects the input SNR, and the misalignment effect. The mean horizontal angular rotational head speed and the mean head direction were calculated for the head movement traces of the participants in the same time windows as used for the SNR calculation. The absolute head direction was taken, and for the VEs with multitalker conversation, the head direction relative to the target was taken, to correct for the changing target position. This also shows in which situations a potential effect of head movement is problematic.

Effect of Movement on Input SNR

The range in input SNR across participants was calculated in all time windows and plotted as a histogram to show the variance due to head movement (Figure 10, right panel). It shows that the mean range in input SNR was between 2 and 6 dB for all environments. SNR differences this large can be assumed to influence speech perception and also the hearing aid algorithm performance. This shows that we have enough variation in the input SNR across movement traces to check the influence on algorithm benefit. In the left panel of Figure 10, the mean input SNR across participants in all time windows is plotted as a histogram in the left panel, to show the dynamics of the input SNR over time in the different environments.

Algorithm Benefit

The mean algorithm benefit over time is plotted in Figure 11 for all algorithms, movement traces, and environments. It can be seen that not all algorithms provided a benefit in all environments. Moreover, it can be seen that there is a variance in the algorithm benefit over the different movement traces. This variance indicates an effect of head movement and was also visible in the output SNR (see Supplementary Materials D). Therefore, we can be sure that the algorithms did not just compensate for the differences in input SNR due to head movement, which would lead to the same output SNR, but that head movement really did affect the performance of the algorithms. The following sections investigate whether the variance in algorithm benefit is related to misalignment and maladaptation. Figure 11 shows that the range in algorithm benefit was up to 6 dB for some algorithms in some environments. The differences in single time windows could be larger. To find this out, histograms were plotted of the range in benefit across the participants in each time window (Figure 12). It can be seen that differences in head movement patterns could cause differences in algorithm benefit up to 15 dB in single time windows. For the AMVDRb, a large range in algorithm benefit occurred more often than for the other algorithms, whereas the SCNR and BNR algorithms were less affected by head movement. An effect of head movement could be seen in all VEs, but in the street_active and train station VEs, the range in algorithm benefit was larger.

OPEN IN VIEWER

Figure 11.

Segmental SNR Benefit Over Time of All Algorithms Over All Movement Traces in All Environments. Box plots show the median (different symbol and color for each algorithm), 25th and 75th percentiles (thick line), and the range (thin line). The range in algorithm benefit shows that different movement traces result in a different algorithm benefit. The reverberation time (T60) of each environment is listed on top, as measured in (Hendrikse et al., 2019b), together with the mean input SNR in each environment. ADMb = adaptive differential microphone; AMVDRb = adaptive minimum variance distortionless response beamformer; D&Sb = delay-and-subtract beamformer; SBb = a static binaural beamformer; BNR = binaural noise reduction; SCNR = single-channel noise reduction; SNR = signal-to-noise ratio.

OPEN IN VIEWER

Figure 12.

The Algorithm Benefit Range Across Participants (and Across Head Movement Traces) Calculated in Each Time Window and Plotted as a Histogram for Each Environment Separately. Different colors indicate different algorithms. This shows how the head movement can affect the algorithm benefit in each time window. ADMb = adaptive differential microphone; AMVDRb = adaptive minimum variance distortionless response beamformer; D&Sb = delay-and-subtract beamformer; SBb = a static binaural beamformer; BNR = binaural noise reduction; SCNR = single-channel noise reduction.

The repeated-measures ANOVA, to examine differences in algorithm performance between the age groups based on their different movement behaviors, showed that only the AMVDRb algorithm with the slowest setting gave a significant overall effect of age-group on the algorithm benefit, F(1, 38) = 4.5; p < .05. However, there were very small but significant differences for some algorithms in some environments. These are listed in Table 3.

OPEN IN VIEWER

Table 3.

Significant Differences in Algorithm Benefit Between Younger and Older Participants.

Environment	Algorithm	Difference in algorithm benefit younger–older (dB)	Statistics F (1, 38)
Cafeteriadualtask	ADMb	0.3	12.4***
	SBb	−0.1	4.4*
Lecture hall	D&Sb	0.2	8.5**
	ADMb	0.2	5.0*
Living room	SBb	0.4	14.8***
Train station	D&Sb	−0.4	9.9**
	ADMb	−0.5	16.3***
	SBb	−0.4	9.4**
	AMVDRb	−0.6	17.9***

Note. ADMb = adaptive differential microphone; AMVDRb = adaptive minimum variance distortionless response beamformer; D&Sb = delay-and-subtract beamformer; SBb = a static binaural beamformer.

*p < .05. **p < .01. ***p < .001.

Algorithm Benefit per Rotational Head Speed

To examine whether maladaptation contributed to the variance in algorithm benefit, the correlation between the algorithm benefit and the horizontal angular rotational head speed was investigated using linear regression. No dependency was found, and the correlations were below .15 for all algorithms in all environments. Thus, maladaptation did not contribute to the movement effect. Scatter plots of the algorithm benefit versus the horizontal angular rotational head speed, including regression lines, can be found in Supplementary Materials E.

Algorithm Benefit per Head Direction

To examine how the input SNR and misalignment contributed to the variance in algorithm benefit, the algorithm benefit per time window was plotted as a function of the horizontal head direction of the participants in the corresponding time window (Figure 13). Figure 13 shows that there was a strong dependency of the algorithm benefit on the head direction for the directional algorithms, especially for the AMVDRb algorithm. The omnidirectional BNR and SCNR algorithms did not depend much on the head direction. Only in the train station VE was there a strong increase in algorithm benefit toward azimuths of −90°, for the SCNR algorithm. The dependency of the directional algorithms on the head direction was strongest in the street_active and cafeteria VEs. The direction of optimal benefit for the AMVDRb algorithm was clearly toward the target in these VEs; for the other directional algorithms, it was somewhat off-axis.

OPEN IN VIEWER

Figure 13.

Algorithm Benefit as a Function of Head Direction for All Algorithms (Different Colors) in All Environments. The algorithm benefit is plotted as a function of the absolute head direction for the living room, lecture hall and train station environments, and as a function of the head direction relative to the target for the cafeteria and street environments, to compensate for changing target speaker positions. The plots show a dependency of the algorithm benefit on the head direction. ADMb = adaptive differential microphone; AMVDRb = adaptive minimum variance distortionless response beamformer; D&Sb = delay-and-subtract beamformer; SBb = a static binaural beamformer; BNR = binaural noise reduction; SCNR = single-channel noise reduction.