A variant of SWEMDH technique based on variational mode decomposition for speech enhancement

Abstract

In Speech Enhancement (SE) techniques, the major challenging task is to suppress non-stationary noises including white noise in real-time application scenarios. Many techniques have been developed for enhancing the vocal signals; however, those were not effective for suppressing non-stationary noises very well. Also, those have high time and resource consumption. As a result, Sliding Window Empirical Mode Decomposition and Hurst (SWEMDH)-based SE method where the speech signal was decomposed into Intrinsic Mode Functions (IMFs) based on the sliding window and the noise factor in each IMF was chosen based on the Hurst exponent data. Also, the least corrupted IMFs were utilized to restore the vocal signal. However, this technique was not suitable for white noise scenarios. Therefore in this paper, a Variant of Variational Mode Decomposition (VVMD) with SWEMDH technique is proposed to reduce the complexity in real-time applications. The key objective of this proposed SWEMD-VVMDH technique is to decide the IMFs based on Hurst exponent and then apply the VVMD technique to suppress both low- and high-frequency noisy factors from the vocal signals. Originally, the noisy vocal signal is decomposed into many IMFs using SWEMDH technique. Then, Hurst exponent is computed to decide the IMFs with low-frequency noisy factors and Narrow-Band Components (NBC) is computed to decide the IMFs with high-frequency noisy factors. Moreover, VVMD is applied on the addition of all chosen IMF to remove both low- and high-frequency noisy factors. Thus, the speech signal quality is improved under non-stationary noises including additive white Gaussian noise. Finally, the experimental outcomes demonstrate the significant speech signal improvement under both non-stationary and white noise surroundings.

Keywords

Speech enhancement non-stationary noises additive white gaussian noise intrinsic mode functions SWEMDH technique variational mode decomposition narrow-band components

1. Introduction

In digitalized world, the removal of audio distortion in noisy vocal signals has been basically obligatory for enhancing the vocal signals. Different speech improvement methods and algorithms have been developed by several researchers for diminishing the noise from the vocal signals [1]. In current non-stationary surroundings, the foremost issue in speech improvement is disturbed with the evaluation of the noise data correctly. The traditional evaluators are based on Voice Activity Detectors (VAD) [2, 3]. Afterwards, the energy spectrum of the noise factors is decided as a smoothed adjustment of its preceding rates achieved for the period of vocal intermissions. This method recommends a reasonable precision for fixed environment noises; however, they cannot specifically evaluate the time-varying spectrum. The complicatedness in tracking the non-stationary noises develops into more understandable for lengthy vocal fragments and low Signal-to-Noise Ratio (SNR) [4]. These conditions have been dealt with various power spectrum-based techniques [5, 6].

Over the previous years, a Time-Frequency (TF)-based vocal improvement technique [7] has been proposed by using the Empirical Mode Decomposition (EMD) scheme [8]. In general, the EMD decomposes the vocal signals into a sequence of oscillatory IMF and a remaining factor [9]. This scheme does not require a group of fundamental functions for appropriately analyzing the target signal. Also, it does not constrain the stationary signals. As a result, Zao et al. [10] proposed a novel EMD-based vocal improvement method for solving the difficulties in non-stationary noisy surroundings. In this method, the noise factors of each IMF were recognized and decided by its Hurst exponent data. As well, the choice of IMF and the vocal restoration were executed on the frame-by-frame manner by taking into consideration of the feature and precision target estimates. Nonetheless, this method has high time and resource consumptions. In addition, a considerable enhancement under Babble noise situations was not effective.

Therefore, SWEMDH technique [11] was proposed based on the computation of EMD in a relatively small window which is sliding along with the time axis. The size of the window was depending on the frequency spectrum of the vocal signal. The potential discontinuities in IMF among windows were avoided by means of the sum amount of modes and the amount of filtering iterations that have be assign a priori. The amount of filtering iterations must be modified for each component and depends on the sampling frequency, analyzed signal, its difficulty and band. This was computed by decomposing the signal windows based on the general algorithm and also the typical amount of filtering iterations for each module was computed. However, this technique was not effective in white noise surroundings.

Hence in this article, a VVMD with SWEMDH technique is proposed to reduce the complexity in real-time applications. Initially, SWEMDH technique is applied for decomposing the noisy vocal signals into IMFs. After that, the Time Delay Estimation (TDE)-based VVMD is applied on the addition of chosen IMF. The main aim of this SWEMD-VVMDH technique is to choose the IMFs based on Hurst exponent and then apply the VVMD technique which is appropriate for reducing the low- and high-frequency noise components in the vocal signal. Thus, the vocal signal quality is further enhanced under different noises such as additive white Gaussian noise, street noise, babble noise, airport noise, etc.

The remaining part of this article is prepared as follows: Section II surveys the related researches on SE techniques. Section III explains the methodology of SWEMD-VVMDH technique. Section IV demonstrates the performance efficiency of SWEMD-VVMDH technique compared to the SWEMDH technique. Section V concludes the research work.

2. Related works

Swami et al. [12] focused on adaptive scales for computing the Continuous Wavelet Transform (CWT) coefficients and adaptive thresholding of these coefficients for SE. In this technique, the adaptive scales and thresholds were chosen based on the noise level of the noisy speech signal. Then, the CWT coefficients were soft-thresholded by generating adaptive thresholds. However, it needs to adapt the threshold values separately for the speech regions and also this technique was limited to use single microphone recordings.

Mai et al. [4] proposed a novel technique, namely Extended-DATE (E-DATE) that extend the d-Dimensional Amplitude Trimmed Estimator (DATE) for noise power spectrum opinion in SE. It relies on the statement that the Short-Time Fourier Transform (STFT) of noisy speech signals was sparse in the sense that converted vocal signals can be denoted by a comparatively miniature amount of coefficients with huge amplitudes in the time-frequency domain. However, it needs advanced techniques to increase the quality of reconstructed speech signals. Tavares and Coelho [13] proposed a novel time domain SE method for non-stationary acoustic noise signals. In this technique, the noisy factors were identified and attenuated. Then, these factors were employed for defining a noise range threshold. But, it executes frame-by-frame noise opinion and also it requires threshold values for removing the noise from the speech signals.

Ji et al. [14] proposed a robust noise Power Spectral Density (PSD) estimator for binaural SE in time-varying noise environments. Initially, the noise PSD was acquired by an eigenvalue of the input covariance matrix. After that, the basic estimator was developed via an approximation method. Then, an eigenvalue compensation method was proposed for enhancing the precision of noise PSD approximation. However, computational cost was moderately increased. Messaoud and Bouzid [15] proposed a novel articulation decision algorithm based on the Multi-scale Product (MP) features to filter the noisy signals. In this algorithm, enhanced subspace decomposition and a multi-scale principal component analysis were performed on the noisy vocal signals and the unvoiced segments of similar signal, respectively. However, computational complexity was high.

Figure 1.

Block diagram for SWEMD-VVMDH technique.

Ghahabi et al. [16] proposed a robust VAD to filter the non-speech segments based on the zero-order Baum-Welch statistics which are compared with the robust threshold value. However, Equal Error Rate (EER) was high. Dwijayanti et al. [17] proposed an improvement of speech dynamics for VAD by Deep Neural Network (DNN). The dynamics were pointed by vocal time candidates computed by the heuristic system for the patterns of the primary and secondary derivatives of the input signals. Then, these candidates combined with the log power spectra were given as input to the DNN for obtaining VAD results. However, the performance of VAD was degraded while it eliminates the subbands and its neighbors.

Saleem and Ijaz [18] proposed a SE algorithm based on low-rank and sparse matrix decomposition scheme that uses grammatone filterbank and Kullback-Leibler divergence to decompose the noisy vocal magnitude spectra into the low-rank noise and sparse vocal segments. Based on this technique, the noise signals were considered as low-rank factors and vocal signals were taken as sparse elements. Then, a modified SE algorithm was developed for separating the speech and noise magnitude spectra by requiring the rank and sparsity limits where the improved time-domain speech was created from the sparse matrix. However, the speech recognition accuracy was reduced while background noise was not preserved well.

3. Proposed methodology

In this section, the proposed SWEMD-VVMDH technique is described briefly. The VVMD is mainly applicable to minimize the low-frequency noise as well as the high-frequency noise. In contrast, SWEMDH fails to accurately extracting the high-frequency components. Also, this can be applied on entire length of noisy speech signals for enhancing the speech signals under white noise environment and real-time noisy environments. Similarly, NBS is used for removing unwanted wideband noises in the speech signals. As well, the benefit of selecting the narrow-band components is lower noise bandwidth and therefore better sensitivity and range. Figure 1 illustrates the fundamental processes in this technique.

The steps in this SWEMD-VVMDH technique for speech signal enhancement are described below:

•
Initially, the noisy vocal signal is decomposed into set of IMFs by SWEMD technique. Consider $N$ number of IMFs are obtained. Then, the Hurst exponent $(H)$ is computed for all the obtained IMFs. Then, the IMFs whose $H\leqslant 0.5$ are chosen. Therefore, $H>0.5$ is essential for the low-frequency noisy factors which are removed from the input signal. Consider $K$ IMFs are chosen on the basis of $H$ value.
•
After that, VVMD technique is applied on addition of all chosen $K$ IMFs with input parameters. The input speech signal is decomposed into set of NBCs. The mean $(\mu_{{\overline{f}}_{C}})$ and their corresponding standard deviations $(\sigma_{{\overline{f}}_{C}})$ of obtained NBCs are computed. Then, the NBCs whose $\mu_{{\overline{f}}_{C}}+\sigma_{{\overline{f}}_{C}}\leqslant 3.6$ KHz are chosen. Consider $P$ NBCs are chosen. Therefore, the high-frequency noisy factors are removed.
•
VVMD technique is performed on each chosen NBCs with input parameter to obtain DC components which are eliminated.
•
Finally, an improved speech signal is obtained from the addition of all the residual factors.

3.1 SWEMDH technique

The SWEMDH method is used to obtain the IMFs from an input signal $x(t)$ . At first, the input noisy speech signal is decomposed into a set of windowed IMFs by SWEMDH technique [11]. This is achieved according to the small sliding window that depends on the signal’s time frequency. For each input single frame, the successive IMFs are computed by the number of iterations which is calculated by decomposing signal’s windows and computing the average number of sifting steps. If all the windowed IMFs are acquired, then Hurst exponent $(H)$ is applied to choose the most optimal IMF low-frequency noisy factors i.e., the IMFs whose $H\leqslant 0.5$ are chosen.

3.2 VVMD technique

The main purpose of VVMD is decomposing the signal $x(t)$ into set of NBCs which are determined in the region of counterpart centre frequencies. In this technique, the limited variational optimization problem is utilized for computing the bandwidth of NBCs as follows:

$\displaystyle\min\limits_{\{\gamma_{P}\},\{{\omega}_{P}\}}\sum_{P}\left\|{% \partial_{t}\left[\left(\delta(t)\!+\!\frac{j}{\pi t}\right)*\gamma_{P}(t)% \right]e^{-j{\omega}_{P}t}}\right\|^{2}_{2}$ $\displaystyle\text{Such that }\sum_{P}{\gamma_{P}(t)=x(t)}$ (1)

In Eq. (3.2), $\{\gamma_{P}\}=\{\gamma_{1},\gamma_{2},\ldots,\gamma_{P}\},\{\omega_{P}\}=\{{% \omega}_{1},$ ${\omega}_{2},$ $\ldots,{\omega}_{P}\},\delta(t)$ and $P$ are the set of acquired NBCs and their counterpart centre frequencies, Dirac distribution function and total amount of NBCs, correspondingly. The Lagrangian multiplier $(\lambda)$ and penalty aspect of restoration reliability term $(\alpha)$ is utilized for converting Eq. (3.2) into an unrestrained optimization problem as follows:

$\displaystyle\mathcal{L}(\{\gamma_{P}\},\{{\omega}_{P}\},\lambda)=$ $\displaystyle\alpha\sum_{P}\left\|{\partial}_{t}\left[\left(\delta(t)+\frac{j}% {\pi t}\right)*\gamma_{P}(t)\right]e^{-j{\omega}_{P}t}\right\|^{2}_{2}+$ $\displaystyle\left\|{x(t)-\sum_{P}{\gamma_{P}(t)}}\right\|^{2}_{2}+\left% \langle\lambda(t),x(t)-\sum_{P}{\gamma_{P}(t)}\right\rangle$

By using Eq. (3.2), the NBCs and their related centre frequencies is determined by using the alternating direction scheme of multipliers which is described in Algorithm 3.2.

Initialize $\{{\hat{\gamma}}^{1}_{P}\},\{{\omega}^{1}_{P}\},{\hat{\lambda}}^{1},n\leftarrow 0$ Do again

$n\leftarrow n+1$ ;

$(P=1:P)$

$(i=1:P;i\neq P)$

Update $\gamma_{P}$ ;

$\displaystyle{\gamma}^{n+1}_{P}\leftarrow$

$\displaystyle{\mathop{\operatorname{arg\,min}}\limits_{\gamma_{P}}\mathcal{L}(% \{{\gamma}^{n+1}_{i<P}\},\{{\gamma}^{n}_{i\geqslant P}\},\{{\omega}^{n}_{i}\},% {\lambda}^{n})}$ ;

Update ${\omega}_{P}$ ;

$\displaystyle{\omega}^{n+1}_{P}\leftarrow$

$\displaystyle{\mathop{\operatorname{arg\,min}}\limits_{{\omega}_{P}}\mathcal{L% }(\{{\gamma}^{n+1}_{i}\},\{{\omega}^{n+1}_{i<P}\},\{{\omega}^{n}_{i\geqslant P% }\},{\lambda}^{n})}$ ;

Dual ascent;

$\displaystyle{\lambda}^{n+1}\leftarrow{\lambda}^{n}+\tau\left(x(t)-\sum_{P}{{% \gamma}^{n+1}_{P}}\right)$ ;

Until convergence: $\displaystyle\sum_{P}{{\|\gamma^{n+1}_{P}-\gamma^{n}_{P}\|^{2}_{2}}/{\|\gamma^% {n}_{P}\|^{2}_{2}}<\tau}$

The determination of NBCs in frequency-domain and their centre frequencies are denoted by,

$\displaystyle{\hat{\gamma}}^{n}_{P}(\omega)=\frac{\hat{x}(\omega)+({\hat{% \lambda}(\omega)}/{2})-\sum_{i\neq P}{{\hat{\gamma}}_{i}(\omega)}}{1+2\alpha{(% \omega-{\omega}_{P})}^{2}}$ (3) $\displaystyle{\omega}^{n}_{P}=\frac{\int^{\infty}_{0}{\omega{|{\hat{\gamma}}_{% P}(\omega)|}^{2}{\rm d}\omega}}{\int^{\infty}_{0}{{|{\hat{\gamma}}_{P}(\omega)% |}^{2}{\rm d}\omega}}$ (4)

Then, the mean and standard deviation from Eq. (4) are computed for estimating the centre frequencies ${\overline{f}}_{C}$ for respective NBC as:

$\displaystyle\mu_{{\overline{f}}_{C}}=({1}/{L})\sum^{K}_{n=1}{{\omega}^{n}_{P}}$ (5) $\displaystyle\sigma_{{\overline{f}}_{C}}=\sqrt{\frac{1}{K}\sum^{L}_{n=1}{{({% \omega}^{n}_{P}-\mu_{{\overline{f}}_{C}})}^{2}}}$ (6)

In Eqs (5) and (6), $L$ denotes the number of iterations in VVMD technique for convergence to acquire the NBCs and counterpart centre frequencies. The input factors of VVMD such as penalty aspect of restoration reliability term, the number of NBCs to be extracted, the step length of the dual ascent, the number of DC element to be extracted, the tolerance of convergence measure and the initialization of frequencies of the extracted NBCs are represented by $\alpha,P,\tau,DC,tol$ and ${\omega}_{ini}$ , correspondingly. The step length of dual ascent $\tau$ is used for controlling $\lambda$ and $t o l$ is used for controlling the virtual inaccuracy between the acquired NBCs and respective original NBCs. Once this process is completed, the NBCs whose $\mu_{{\overline{f}}_{C}}+\sigma_{{\overline{f}}_{C}}\leqslant 3.6$ KHz are chosen as high-frequency noisy factors. The entire optimization of VVMD technique is described in Algorithm 3.2.

Initialize $\{{\hat{\gamma}}^{1}_{P}\},\{{\omega}^{1}_{P}\},{\hat{\lambda}}^{1},n\leftarrow 0$ Do again

$n\leftarrow n+1$ ;

$(P=1:P)$

$(i=1:P;i\neq P)$

Update ${\hat{\gamma}}_{P}$ for all $\omega\geqslant 0$ ;

$\displaystyle{\hat{\gamma}}^{n+1}_{P}(\omega)\leftarrow\frac{\displaystyle\hat% {x}(\omega)-\sum_{i<P}{{\hat{\gamma}}^{n+1}_{i}(\omega)-\sum_{i>P}{{\hat{% \gamma}}^{n}_{i}(\omega)+\frac{{\hat{\lambda}}^{n}(\omega)}{2}}}}{% \displaystyle 1+22\alpha{(\omega-{\omega}^{n}_{P})}^{2}}$ ;

Update ${\omega}_{P}$ ;

$\displaystyle{\omega}^{n+1}_{P}=\frac{\displaystyle\int^{\infty}_{0}{\omega{|{% \hat{\gamma}}^{n+1}_{P}(\omega)|}^{2}{\rm d}\omega}}{\displaystyle\int^{\infty% }_{0}{{|{\hat{\gamma}}^{n+1}_{P}(\omega)|}^{2}{\rm d}\omega}}$ ;

Dual ascent for all $\omega\geqslant 0$ ;

$\displaystyle{\hat{\lambda}}^{n+1}(\omega)\leftarrow{\hat{\lambda}}^{n}(\omega% )+\tau(\hat{x}(\omega)-\sum_{P}{{\hat{\gamma}}^{n+1}_{P}(\omega)})$ ;

Until convergence: $\displaystyle\sum_{P}\|{{\hat{\gamma}}^{n+1}_{P}-{\hat{\gamma}}^{n}_{P}}\|^{2}% _{2}/\|{{\hat{\gamma}}^{n}_{P}}\|^{2}_{2}<\tau$

In the restoration fidelity term, the centre frequencies ${\omega}_{P}$ is not appeared; however, only in the bandwidth prior. Thus, Lagrangian multiplier $\mathcal{L}$ reconstructs the speech signal as dual ascent by updating centre frequencies and NBCs efficiently.

4. Experimental results and discussions

In this section, the SWEMD-VVMDH technique is implemented to analyze its effectiveness compared with the SWEMDH technique by using MATLAB 2014a. By using audio tools, VVMD function and Hurst exponent function in MATLAB 2014a, the SWEMD-VVMDH technique is implemented. In this experiment, TIMIT database [19] is considered for acquiring the speech signals which are directly digitized at a sample rate of 20 KHz using a Digital Sound Corporation (DSC) 200 with the anti-aliasing filter at 10 KHz. The speech is then digitally filtered, debiased and downsampled to 16 kHz. From this database, a subset of 12 speakers (a total of 120 vocal signal segments) consisting 7 men and 5 women is chosen at random. As well, eight are fused from each of 10 signals available per speaker and used for training the speaker models whereas the other two are split for testing. Each of $12\times 2=24$ test signals are corrupted by different noises such as additive Gaussian noise, airport, babble, car, exhibition, restaurant, station, street and train with various SNR values of 0 dB, 5 dB, 10 dB and 15 dB. Such noises are gathered from the NOISEX-92 database [20]. So, a total of $36\times 24=864$ testing are executed (9 type of noises $\times 4$ SNR values $=36$ ). The performance metrics used for analysis are described below:

•
SNR: It defines the percentage of the average power of speech signal $(P_{\text{signal}})$ to the average power of corrupted noise $(P_{\text{noise}})$ .

$\displaystyle\textit{SNR}(\text{dB})=10\log_{10}\left(\frac{P_{\text{signal}}}% {P_{\text{noise}}}\right)$ (7)

It can be rewritten as:

$\displaystyle\textit{SNR}(\text{dB})=20\log_{10}\left(\frac{A_{\text{signal}}}% {A_{\text{noise}}}\right)$ (8)

In Eq. (8), $A_{\text{signal}}$ and $A_{\text{noise}}$ are the Root Mean Square (RMS) amplitude of signal and noise, respectively.
•
Mean Square Error (MSE): It defines the cumulative squared error between the restored and actual speech signal. It is computed as:

$\displaystyle\textit{MSE}=\frac{1}{l}\sum^{n}_{i=1}{e^{2}_{i}};∼{}e=\hat{x}(t)% -x(t)$ (9)

In Eq. (9), $l$ is the signal length and $e$ is the error between the actual signal $x(t)$ and restored signal $\hat{x}(t)$ .
•
Peak Signal-to-Noise Ratio (PSNR): It is the percentage of the highest signal power to the noise power. It is calculated as:

$\displaystyle\textit{PSNR}(dB)=10\log_{10}\frac{{255}^{2}}{\textit{MSE}}$ (10)
•
Mean Absolute Error (MAE): It defines the total inexactness between the restored speech signal and actual signal. It is calculated as:

$\displaystyle\textit{MAE}=\frac{1}{l}\sum^{n}_{i=1}{e_{i}}$ (11)
•
Perceptual Evaluation of Speech Quality (PESQ): It is used for evaluating an end-to-end quality to characterize the listening quality as apparent by users.

$\displaystyle\textit{PESQ}=\alpha_{0}-\alpha_{1}.D-\alpha_{2}.A;\alpha_{0}=0.1,$ $\displaystyle\alpha_{1}=0.1,\text{and}∼{}\alpha_{2}=0.0309$ (12)
•
Weighted Spectral Slop Measure (WSSM): It is defined as the time averaged WSSM where only the good frames are averaged. It measures the weighted differences of spectral slope over 25 critical frequency bands between the two corresponding signal frames.

$\displaystyle\textit{WSSM}=$ (13) $\displaystyle\frac{1}{N}\sum^{N}_{k=1}\left[\frac{\sum^{24}_{f=1}{W(f){[\Delta E% _{s}(f)-\Delta E_{\overline{s}}(f)]}^{2}}}{\sum^{24}_{f=1}{W(f)}}\right]$

In Eq. (13), $\Delta E_{s}(f)$ and $\Delta E_{\overline{s}}(f)$ are the spectral slope at frequency band $S$ and $\overline{S}$ , $W(f)$ is the weight in each band and $N$ is the number of good frames.

Table 1
MSE comparison

EMDH SWEMDH VVMDH

Noise 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB

Airport 0.005775 0.003425 0.002664 0.003672 0.001823 0.001043 0.000747 0.001199 0.001623 0.000843 0.000547 0.000999

Babble 0.005829 0.003546 0.002682 0.002428 0.001765 0.001084 0.000749 0.000661 0.001565 0.000884 0.000549 0.000461

Car 0.005574 0.003438 0.002683 0.002434 0.002012 0.001057 0.000777 0.000665 0.001812 0.000857 0.000577 0.000465

Exhibition 0.007421 0.004681 0.003809 0.002418 0.003258 0.001835 0.001342 0.000667 0.003058 0.001635 0.001142 0.000467

Restaurant 0.009604 0.003437 0.002693 0.002452 0.003394 0.001058 0.000753 0.000665 0.003194 0.000858 0.000553 0.000465

Station 0.005772 0.003167 0.002629 0.004093 0.002724 0.001036 0.000742 0.001682 0.002524 0.000836 0.000542 0.001482

Street 0.004502 0.003227 0.003843 0.002392 0.001923 0.001041 0.001172 0.000657 0.001723 0.000841 0.000972 0.000457

Train 0.004846 0.003074 0.002567 0.004568 0.001877 0.001008 0.000748 0.002151 0.001677 0.000808 0.000548 0.001951

Figure 2.
MSE comparison.

Table 1 gives the comparison results of MSE for EMDH, SWEMDH and VVMDH using different acoustic noises that corrupt the speech signal during transmission.

Figure 2 shows the graphical representation of comparison results obtained from MSE values for EMDH, SWEMDH and VVMDH using different acoustic noises. From this analysis, it is identified that the proposed VVMDH approach can minimize the MSE compared to the EMDH and SWEMDH approaches. For example, consider the Babble noise environment with SNR of 15 dB. In this case, the MSE of VVMDH is 81.01% reduced than the EMDH approach and 72.78% reduced than the SWEMDH approach. This is achieved by facilitating the backward error correction and having the ability to properly cope with noise.

Table 2
MAE comparison

EMDH SWEMDH VVMDH

Noise 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB

Airport 0.058149 0.041596 0.032694 0.036451 0.032602 0.022529 0.016521 0.019249 0.029002 0.018929 0.012921 0.015649

Babble 0.058241 0.040286 0.032865 0.028596 0.031588 0.021409 0.016503 0.013812 0.027988 0.017809 0.012903 0.010212

Car 0.056816 0.041438 0.031981 0.028534 0.034311 0.022757 0.016660 0.013912 0.030711 0.019157 0.013060 0.010312

Exhibition 0.064147 0.046558 0.037877 0.028060 0.042292 0.028699 0.021420 0.013980 0.038692 0.025099 0.017820 0.010380

Restaurant 0.072716 0.039627 0.032467 0.028333 0.042734 0.021803 0.016257 0.013627 0.039134 0.018203 0.012657 0.010027

Station 0.054356 0.038677 0.032034 0.036713 0.038153 0.022387 0.016401 0.021438 0.034553 0.018787 0.012801 0.017838

Street 0.049660 0.038047 0.037239 0.027848 0.032773 0.021502 0.020063 0.013732 0.029173 0.017902 0.016463 0.010132

Train 0.050743 0.037712 0.030818 0.035917 0.031988 0.021730 0.016348 0.022726 0.028388 0.018130 0.012748 0.019126

Figure 3.
MAE comparison.

Table 3
SNR comparison (in dB)

EMDH SWEMDH VVMDH

Noise 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB

Airport 2.659574 2.334045 2.106246 1.831171 5.759574 3.834045 2.414662 2.028799 9.959574 8.034045 3.914662 5.428799

Babble 2.605948 2.453267 2.123635 2.018528 6.805948 5.553267 2.413667 2.632736 11.005948 9.753267 6.613667 4.132736

Car 2.437381 2.324013 2.074241 2.018716 4.737381 6.524013 2.307112 2.617482 8.137381 10.724013 6.507112 6.017482

Exhibition 1.803051 1.740852 1.686372 1.972072 4.903051 3.240852 1.844750 2.619774 9.103051 7.440852 3.344750 6.019774

Restaurant 2.778967 2.272254 2.165528 2.035596 6.978967 4.572254 2.367432 2.630781 11.178967 8.772254 5.767432 4.130781

Station 1.432190 1.941313 2.003922 2.239730 1.829495 5.041313 2.487985 5.339730 6.029495 9.241313 3.987985 9.539730

Street 1.550461 2.068816 2.502633 1.948060 5.350461 5.168816 6.302633 2.660235 8.450461 7.868816 8.602633 4.160235

Train 1.812100 1.851648 1.901905 2.216282 4.112100 1.989755 2.452413 3.716282 5.612100 5.089755 6.652413 7.116282

Figure 4.
SNR comparison.

Table 2 gives the comparison results of MAE for EMDH, SWEMDH and VVMDH using different acoustic noises that corrupt the speech signal during transmission.

The graphical representation of MAE values for existing EMDH, SWEMDH and proposed VVMDH using different acoustic noises is shown in Fig. 3. Through this analysis, the proposed VVMDH approach achieves less MAE when compared to the existing EMDH and SWEMDH approaches. For considering the case that Babble noise environment with SNR of 15 dB, the MAE of VVMDH is 64.29% less than the EMDH and 26.06% less than the SWEMDH. This is due to proper tradeoff for errors between the relevant bands and their corresponding modes which are simultaneously estimated by using VVMDH approach.

Table 3 gives the comparison results of SNR for EMDH, SWEMDH and VVMDH using different acoustic noises that corrupt the speech signal during transmission.

Figure 4 shows the graphical representation of comparison results obtained from SNR values for EMDH, SWEMDH and VVMDH using different acoustic noises. From this analysis, it is identified that the proposed VVMDH approach can maximize the SNR while compared to the EMDH and SWEMDH approaches. For the scenario of Babble noise environment with SNR of 15 dB, the SNR of VVMDH is 4.132736 dB which is less than EMDH and SWEMDH approaches whose SNR values are 2.018528 dB and 2.632736 dB, accordingly. This is because of ensuring the narrow-band properties corresponding to the current estimate of the mode’s center-frequency to the signal estimation residual of all other modes.

Table 4 gives the comparison results of PSNR for EMDH, SWEMDH and VVMDH using different acoustic noises that corrupt the speech signal during transmission.

Table 4
PSNR comparison (in dB)

EMDH SWEMDH VVMDH

Noise 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB

Airport 12.683157 15.530436 16.300209 14.790743 17.690017 20.695833 21.821117 19.650713 18.194567 21.620806 23.173878 20.443124

Babble 14.247063 14.667079 16.513134 16.526397 19.436154 19.814285 22.050436 22.177661 19.958561 20.700123 23.398880 23.743065

Car 13.462103 15.318908 16.243443 16.677009 17.887396 20.439911 21.624795 22.313207 18.342077 21.350462 22.916937 23.867782

Exhi- bition 13.667139 13.552145 15.493075 16.793648 17.242225 17.619207 20.024197 22.385494 17.517344 18.120348 20.725250 23.933081

Restau- rant 11.277032 15.192311 16.195447 16.823203 15.793952 20.308583 21.728408 22.489580 16.057703 21.218546 23.068690 24.042886

Station 14.414169 16.044026 16.306159 15.510117 17.675854 20.897496 21.798066 19.372952 18.007094 21.829052 23.161181 19.922814

Street 15.329903 15.487624 15.634679 16.817311 19.024172 20.399830 20.792186 22.425606 19.501070 21.326096 21.604826 24.000680

Train 15.672559 15.702125 16.751990 14.977004 19.790482 20.543528 22.106308 18.248440 20.279658 21.503889 23.457328 18.672344

Figure 5.
PSNR comparison.

Table 5
PESQ comparison

EMDH SWEMDH VVMDH

Noise 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB

Airport 3.546681 3.663083 3.590500 3.507061 3.756681 3.813083 3.790500 3.607061 3.916681 3.973083 3.940500 3.787061

Babble 3.671733 3.718452 3.581719 3.839045 3.549439 3.928452 3.731719 4.039045 3.709439 4.088452 3.891719 4.189045

Car 3.334989 3.765506 3.716819 3.598346 3.434989 3.925506 3.816819 3.758346 3.614989 4.085506 3.976819 3.938346

Exhibition 3.437603 3.466507 3.515634 3.681904 3.647603 3.616507 3.715634 3.781904 3.807603 3.776507 3.865634 3.961904

Restaurant 3.615765 3.763355 3.882143 3.881704 2.694153 3.863355 3.805933 4.091704 2.854153 4.023355 3.985933 4.241704

Station 2.960255 3.400217 3.780931 3.736622 3.090255 3.610217 3.980931 3.946622 3.250255 3.770217 4.130931 4.106622

Street 3.451811 3.473544 3.566137 3.708424 3.651811 3.683544 3.766137 3.908424 3.861811 3.813544 3.866137 4.058424

Train 3.285802 3.309595 3.595872 3.718945 3.385802 3.459595 3.725872 3.868945 3.535802 3.669595 3.885872 4.048945

Figure 6.
PESQ comparison.

Table 6
WSSM comparison

EMDH SWEMDH VVMDH

Noise 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB 0 dB 5 dB 10 dB 15 dB

Airport 0.332341 0.338280 0.430822 0.398105 0.245846 0.511824 0.808057 0.724320 0.786297 1.113143 0.919730 1.338562

Babble 0.362627 0.392673 0.351485 0.390662 0.831609 0.560862 0.724097 1.142055 0.765996 0.922019 0.903414 1.211347

Car 0.319555 0.387499 0.362310 0.359008 0.346420 0.923313 0.826205 0.533941 1.314956 1.056934 0.913469 1.039662

Exhibition 0.341916 0.363030 0.381856 0.387227 0.378026 0.668930 0.746578 0.809059 0.947483 1.019481 1.233894 1.241083

Restaurant 0.320504 0.408478 0.386399 0.384391 0.233470 1.062985 0.666848 0.677155 1.267292 1.267389 0.989482 1.142901

Station 0.371684 0.367435 0.379917 0.402582 0.556546 0.434825 0.682379 0.770057 0.897548 0.735570 1.260015 1.372852

Street 0.333730 0.391852 0.474195 0.411521 0.411576 0.700130 0.829830 0.912615 1.014969 1.418772 1.857531 1.388048

Train 0.367200 0.437585 0.380439 0.349913 0.698813 0.441755 0.590337 0.450032 1.182462 1.006379 1.089460 1.004935

Figure 7.
WSSM comparison.

The graphical representation of PSNR values for existing EMDH, SWEMDH and proposed VVMDH using different acoustic noises is shown in Fig. 5. Through this analysis, the proposed VVMDH approach achieves higher PSNR when compared to the existing EMDH and SWEMDH approach. For considering the case that Babble noise environment with SNR of 15 dB, the PSNR value for the proposed VVMDH approach is 43.67% increased than the EMDH approach and 7.06% increased than the SWEMDH approach. Since the VVMDH approach has high robustness to sampling and noise compared to the other approaches such as EMDH and SWEMDH.

Table 5 gives the comparison results of PESQ for EMDH, SWEMDH and VVMDH using different acoustic noises that corrupt the speech signal during transmission. Also, Fig. 6 shows its graphical representation. From this analysis, it is identified that the proposed VVMDH approach can maximize the PESQ while compared to the EMDH and SWEMDH approaches. For example, consider the Babble noise environment with SNR of 15 dB. In this case, the PESQ of proposed VVMDH approach is 9.12% higher than the EMDH approach and 3.71% higher than the SWEMDH approach. This is achieved because of accurately tuning the center-frequency of both low- and high-frequency harmonics which are detected at acceptable quality and recovered at almost without errors.

Table 6 gives the comparison results of WSSM for EMDH, SWEMDH and VVMDH using different acoustic noises that corrupt the speech signal during transmission.

Figure 7 shows the graphical representation of comparison results obtained from WSSM values for EMDH, SWEMDH and VVMDH using different acoustic noises. From this analysis, it is identified that the proposed VVMDH approach can maximize the WSSM while compared to the EMDH and SWEMDH approaches. For the scenario of Babble noise environment with SNR of 15 dB, the WSSM of VVMDH is 1.211347 which is higher than EMDH and SWEMDH approaches whose WSSM values are 0.390662 and 1.142055, respectively. Thus, it is concluded that the proposed VVMDH approach achieves higher performance i.e., high PSNR, SNR, PESQ and WSSM with less MSE, MAE while compared to the EMDH and SWEMDH approaches.
5. Conclusion

	EMDH	SWEMDH	VVMDH
Noise	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB
Airport	0.005775	0.003425	0.002664	0.003672	0.001823	0.001043	0.000747	0.001199	0.001623	0.000843	0.000547	0.000999
Babble	0.005829	0.003546	0.002682	0.002428	0.001765	0.001084	0.000749	0.000661	0.001565	0.000884	0.000549	0.000461
Car	0.005574	0.003438	0.002683	0.002434	0.002012	0.001057	0.000777	0.000665	0.001812	0.000857	0.000577	0.000465
Exhibition	0.007421	0.004681	0.003809	0.002418	0.003258	0.001835	0.001342	0.000667	0.003058	0.001635	0.001142	0.000467
Restaurant	0.009604	0.003437	0.002693	0.002452	0.003394	0.001058	0.000753	0.000665	0.003194	0.000858	0.000553	0.000465
Station	0.005772	0.003167	0.002629	0.004093	0.002724	0.001036	0.000742	0.001682	0.002524	0.000836	0.000542	0.001482
Street	0.004502	0.003227	0.003843	0.002392	0.001923	0.001041	0.001172	0.000657	0.001723	0.000841	0.000972	0.000457
Train	0.004846	0.003074	0.002567	0.004568	0.001877	0.001008	0.000748	0.002151	0.001677	0.000808	0.000548	0.001951

	EMDH	SWEMDH	VVMDH
Noise	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB
Airport	0.058149	0.041596	0.032694	0.036451	0.032602	0.022529	0.016521	0.019249	0.029002	0.018929	0.012921	0.015649
Babble	0.058241	0.040286	0.032865	0.028596	0.031588	0.021409	0.016503	0.013812	0.027988	0.017809	0.012903	0.010212
Car	0.056816	0.041438	0.031981	0.028534	0.034311	0.022757	0.016660	0.013912	0.030711	0.019157	0.013060	0.010312
Exhibition	0.064147	0.046558	0.037877	0.028060	0.042292	0.028699	0.021420	0.013980	0.038692	0.025099	0.017820	0.010380
Restaurant	0.072716	0.039627	0.032467	0.028333	0.042734	0.021803	0.016257	0.013627	0.039134	0.018203	0.012657	0.010027
Station	0.054356	0.038677	0.032034	0.036713	0.038153	0.022387	0.016401	0.021438	0.034553	0.018787	0.012801	0.017838
Street	0.049660	0.038047	0.037239	0.027848	0.032773	0.021502	0.020063	0.013732	0.029173	0.017902	0.016463	0.010132
Train	0.050743	0.037712	0.030818	0.035917	0.031988	0.021730	0.016348	0.022726	0.028388	0.018130	0.012748	0.019126

	EMDH	SWEMDH	VVMDH
Noise	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB
Airport	2.659574	2.334045	2.106246	1.831171	5.759574	3.834045	2.414662	2.028799	9.959574	8.034045	3.914662	5.428799
Babble	2.605948	2.453267	2.123635	2.018528	6.805948	5.553267	2.413667	2.632736	11.005948	9.753267	6.613667	4.132736
Car	2.437381	2.324013	2.074241	2.018716	4.737381	6.524013	2.307112	2.617482	8.137381	10.724013	6.507112	6.017482
Exhibition	1.803051	1.740852	1.686372	1.972072	4.903051	3.240852	1.844750	2.619774	9.103051	7.440852	3.344750	6.019774
Restaurant	2.778967	2.272254	2.165528	2.035596	6.978967	4.572254	2.367432	2.630781	11.178967	8.772254	5.767432	4.130781
Station	1.432190	1.941313	2.003922	2.239730	1.829495	5.041313	2.487985	5.339730	6.029495	9.241313	3.987985	9.539730
Street	1.550461	2.068816	2.502633	1.948060	5.350461	5.168816	6.302633	2.660235	8.450461	7.868816	8.602633	4.160235
Train	1.812100	1.851648	1.901905	2.216282	4.112100	1.989755	2.452413	3.716282	5.612100	5.089755	6.652413	7.116282

	EMDH	SWEMDH	VVMDH
Noise	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB
Airport	12.683157	15.530436	16.300209	14.790743	17.690017	20.695833	21.821117	19.650713	18.194567	21.620806	23.173878	20.443124
Babble	14.247063	14.667079	16.513134	16.526397	19.436154	19.814285	22.050436	22.177661	19.958561	20.700123	23.398880	23.743065
Car	13.462103	15.318908	16.243443	16.677009	17.887396	20.439911	21.624795	22.313207	18.342077	21.350462	22.916937	23.867782
Exhi- bition	13.667139	13.552145	15.493075	16.793648	17.242225	17.619207	20.024197	22.385494	17.517344	18.120348	20.725250	23.933081
Restau- rant	11.277032	15.192311	16.195447	16.823203	15.793952	20.308583	21.728408	22.489580	16.057703	21.218546	23.068690	24.042886
Station	14.414169	16.044026	16.306159	15.510117	17.675854	20.897496	21.798066	19.372952	18.007094	21.829052	23.161181	19.922814
Street	15.329903	15.487624	15.634679	16.817311	19.024172	20.399830	20.792186	22.425606	19.501070	21.326096	21.604826	24.000680
Train	15.672559	15.702125	16.751990	14.977004	19.790482	20.543528	22.106308	18.248440	20.279658	21.503889	23.457328	18.672344

	EMDH	SWEMDH	VVMDH
Noise	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB
Airport	3.546681	3.663083	3.590500	3.507061	3.756681	3.813083	3.790500	3.607061	3.916681	3.973083	3.940500	3.787061
Babble	3.671733	3.718452	3.581719	3.839045	3.549439	3.928452	3.731719	4.039045	3.709439	4.088452	3.891719	4.189045
Car	3.334989	3.765506	3.716819	3.598346	3.434989	3.925506	3.816819	3.758346	3.614989	4.085506	3.976819	3.938346
Exhibition	3.437603	3.466507	3.515634	3.681904	3.647603	3.616507	3.715634	3.781904	3.807603	3.776507	3.865634	3.961904
Restaurant	3.615765	3.763355	3.882143	3.881704	2.694153	3.863355	3.805933	4.091704	2.854153	4.023355	3.985933	4.241704
Station	2.960255	3.400217	3.780931	3.736622	3.090255	3.610217	3.980931	3.946622	3.250255	3.770217	4.130931	4.106622
Street	3.451811	3.473544	3.566137	3.708424	3.651811	3.683544	3.766137	3.908424	3.861811	3.813544	3.866137	4.058424
Train	3.285802	3.309595	3.595872	3.718945	3.385802	3.459595	3.725872	3.868945	3.535802	3.669595	3.885872	4.048945

	EMDH	SWEMDH	VVMDH
Noise	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB	0 dB	5 dB	10 dB	15 dB
Airport	0.332341	0.338280	0.430822	0.398105	0.245846	0.511824	0.808057	0.724320	0.786297	1.113143	0.919730	1.338562
Babble	0.362627	0.392673	0.351485	0.390662	0.831609	0.560862	0.724097	1.142055	0.765996	0.922019	0.903414	1.211347
Car	0.319555	0.387499	0.362310	0.359008	0.346420	0.923313	0.826205	0.533941	1.314956	1.056934	0.913469	1.039662
Exhibition	0.341916	0.363030	0.381856	0.387227	0.378026	0.668930	0.746578	0.809059	0.947483	1.019481	1.233894	1.241083
Restaurant	0.320504	0.408478	0.386399	0.384391	0.233470	1.062985	0.666848	0.677155	1.267292	1.267389	0.989482	1.142901
Station	0.371684	0.367435	0.379917	0.402582	0.556546	0.434825	0.682379	0.770057	0.897548	0.735570	1.260015	1.372852
Street	0.333730	0.391852	0.474195	0.411521	0.411576	0.700130	0.829830	0.912615	1.014969	1.418772	1.857531	1.388048
Train	0.367200	0.437585	0.380439	0.349913	0.698813	0.441755	0.590337	0.450032	1.182462	1.006379	1.089460	1.004935

In this article, a SWEMD-VVMDH technique is proposed for enhancing the speech signal quality by eliminating both low- and high-frequency noisy factors in real-time applications. Initially, SWEMDH method is applied to decompose the input signal into number of IMFs based on the sliding window. Then, Hurst exponent is computed to decide the IMFs low-frequency noisy components which are discarded from the input signal. After that, VVMD technique is applied to addition of all IMFs by considering NBCs and their centre frequencies for deciding high-frequency noisy components which are also removed from the signal. Thus, the speech signal quality is enhanced by removing both non-stationary and white noisy components efficiently. Finally, the experimental outcomes proved that the VVMDH technique has better performance than the SWEMDH technique for SE under both non-stationary and white noise backgrounds. The SWEMD-VVMDH technique provides better performance in terms of MSE, MAE, WSSM, SNR, PSNR and PESQ measures which correlated with speech quality and perceptual abilities as compared with SWEMDH technique.

References

Kulkarni

D.S.

et al., A review of speech signal enhancement techniques, International Journal of Computer Applications 139(14) (2016).

Kasap

and Arslan

M.L.

, A unified approach to speech enhancement and voice activity detection, Turkish Journal of Electrical Engineering & Computer Sciences 21(2) (2013), 527–547.

Zhang

et al., A hierarchical framework approach for voice activity detection and speech enhancement, The Scientific World Journal, (2014).

Mai

V.K.

et al., Robust estimation of non-stationary noise power spectrum for speech enhancement, IEEE/ACM Transactions on Audio, Speech, and Language Processing 23(4) (2015), 670–682.

Zhao

et al., Speech enhancement method based on sparse reconstruction of power spectral density, Computers & Electrical Engineering 40(4) (2014), 1080–1089.

Y. Jin

et al., Decision-directed speech power spectral density matrix estimation for multichannel speech enhancement, The Journal of the Acoustical Society of America 141(3) (2017), EL228–EL233.

Soni

M.H.

et al., Time-frequency masking-based speech enhancement using generative adversarial network, in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5039–5043.

Mert

and Akan

, Detrended fluctuation thresholding for empirical mode decomposition based denoising, Digital Signal Processing 32 (2014), 48–56.

Mandic

D.P.

et al., Empirical mode decomposition-based time-frequency analysis of multivariate signals: The power of adaptive data analysis, IEEE Signal Processing Magazine 30(6) (2013), 74–86.

10.

Zao

et al., Speech enhancement with EMD and hurst-based mode selection, IEEE/ACM Transactions on Audio, Speech, and Language Processing 22(5) (2014), 899–911.

11.

Poovarasan

and Chandra

, Speech enhancement using sliding window empirical mode decomposition and hurst-based technique, Archives of Acoustics 44(3) (2019), 429–437.

12.

Swami

P.D.

et al., Speech enhancement by noise driven adaptation of perceptual scales and thresholds of continuous wavelet transform coefficients, Speech Communication 70 (2015), 1–12.

13.

Tavares

and Coelho

, Speech enhancement with nonstationary acoustic noise detection in time domain, IEEE Signal Processing Letters 23(1) (2016), 6–10.

14.

et al., Robust noise power spectral density estimation for binaural speech enhancement in time-varying diffuse noise field, EURASIP Journal on Audio, Speech, and Music Processing 2017(1) (2017), 25.

15.

Messaoud

M.A.B.

and Bouzid

, Sparse representations for single channel speech enhancement based on voiced/unvoiced classification, Circuits, Systems, and Signal Processing 36(5) (2017), 1912–1933.

16.

Ghahabi

et al., A robust voice activity detection for real-time automatic speech recognition, in: Proceedings of ESSV 2018, 2018.

17.

Dwijayanti

et al., Enhancement of speech dynamics for voice activity detection using DNN, EURASIP Journal on Audio, Speech, and Music Processing 1 (2018), 10.

18.

Saleem

and Ijaz

, Low rank sparse decomposition model based speech enhancement using gammatone filter bank and kullback–leibler divergence, International Journal of Speech Technology 21(2) (2018), 217–231.

19.

Garofolo

J.S.

et al., DARPA TIMIT acoustic-phonetic continuous speech corpus CD-ROM. NIST speech disc 1–1.1, NASA STI/Recon Technical Report N, (1993), 93.

20.

Varga

and Steeneken

H.J.

, Assessment for automatic speech recognition: II. NOISEX-92: A Database and an experiment to study the effect of additive noise on speech recognition systems, Speech Communication 12(3) (1993), 247–251.