A novel FastICA algorithm based on improved secant method for Intelligent drive

Abstract

Blind Source Separation(BSS) is one of the research hotspots in the field of signal processing. In order to improve the accuracy of speech recognition in driving environment, the driver’s speech signal must be enhanced to improve its signal to noise ratio(SNR). Independent component analysis (ICA) algorithm is the most classical and efficient blind statistical signal processing technique. Compared with other improved ICA algorithms, fixed-point algorithm (FastICA) is well known for its fast convergence speed and good robustness. However, the convergence of FastICA algorithm is comparatively susceptible to the initial value selection of the original demixing matrix and the calculation of the iterative process is relatively large. In this paper, the gradient descent method is used to reduce the effect of initial value. What’s more, the improved secant method is proposed to speed up the convergence rate and reduce the amount of computation. As the results of mixed speech separation experiment turn out, the improved algorithm is of better performance relative to the standard FastICA algorithm. Experimental results show that the proposed algorithm improves the speech quality of the target driver. It is suitable for speech separation in driving environment with low SNR.

Keywords

Blind source separation fixed-point algorithm gradient descent improved secant method speech separation

1 Introduction

Blind source separation (BSS) has always been one of the most important focuses for signal processing. The main task of BSS is to separate out the different estimation signals from the observed signals which were mixed with multiple signals [1]. As voice control technology is more and more widely used in the field of on-board electronic products, noise elimination in driving environment has become an urgent problem to be solved. The recognition rate of speech recognition system is very high in quiet environment, but it is far from this standard in noisy environment. There are many kinds of noise in high-speed cars, such as engine noise, body vibration noise, tire noise and wind noise. These noises will cause great interference to the voice commands issued by the driver, resulting in the low recognition rate of on-board electronic products. Therefore, the speech recognition in driving environment must first go through the speech enhancement to remove the external noise, so as to have a better recognition effect. Although there are many kinds of enhancement algorithms, the enhancement effect in driving environment is not ideal. Traditional speech enhancement algorithms assume that speech and noise are independent of each other, and the form of noise is simple and stable. In driving environment, the form of noise signal is complex and contains many unstable noises, and the noise and speech are not mutually independent [2]. ICA is a classical and excellent signal processing method based on data statistics. ICA is widely used in many fields, such as signal separation, digital image processing, data analysis and so on [3].

In the early 1980 s, ICA concept was initially proposed by Comon at a meeting of neurophysiology communication, which successfully separated out two disparate signals [4]. The emergence of ICA provided a new method which was initially proposed to solve the blind source separation. Various ICA improved algorithms were proposed by many scholars successively. Aapo Hyvarinen advanced the fast fixed-point algorithm (FastICA) which is famous for its fast convergence and good robustness [5]. The idea of FastICA is to tweak the demixing matrix by iterations constantly until it reaches local optimization, which is a quick and steady iterative algorithm that finds the optimal demixing matrix.

Score of scholars carries out research and intend to make FastICA algorithm more faster and stable after it was put forward. Parts of scholars have presented a few ways to optimize the convergence speed and the amount of calculations for FastICA algorithm. Kim et al. [6] proposed the core iterative process of six-order Newton’s method to make FastICA algorithm improvement. Aiming at the shortcomings of the basic Newtonian iterative FastICA algorithm, which is sensitive to the selection of initial values, the modified form of sixth-order Newton iteration is introduced to improve the core iterative process of FastICA algorithm by maximizing negative entropy as the objective function. The convergence of the improved algorithm is no longer dependent on the selection of initial values, and the convergence speed is faster. The improved algorithm has achieved good performances in mixed image separation which the experimental results showed that it has faster convergence speed and less computational cost. Eight-order Newton’s method was brought forward to optimize the FastICA algorithm, which greatly enhanced the convergence speed of the FastICA algorithm [7]. Higher-order FastICA algorithms have the advantages of the fast convergence rate. However, they are sensitive to the initial values selection. If the initial values are not chosen appropriately, which will lead to affecting convergence performance and even result in no convergence. Hence, a novel method called sparseFastICA algorithm was presented, which treated the sparsity property of the independent component hidden in mixed signals as the constraint of FastICA algorithm. The improved FastICA algorithm in functional magnetic resonance imaging (fMRI) achieved faster computation speed and spatial detection capability [8].

In addition, quite a few researchers have made an effort to decrease the sensitivity of the initial value solution of the original demixing matrix while the FastICA algorithm was iterating and converging. An efficient fixed-point ICA algorithm based on a special gradient iteration was proposed, which is different from the gradient descent [9]. Theoretical analysis and experimental results for comparison showed that the initial demixing matrix was of faster convergence and better stability under Gaussian noise. Successive over relaxation factor was brought in processing demixing matrix. On the premise of insuring convergence speed, the improved FastICA algorithm could effectively decrease the initial value sensitivity [10]. Using Newton downhill method to instead of the standard Newton iteration method was suggested to deal with the sensitivity of the initial value problem. It’s about more stability to separate mixed signals of speech and music under different initial values [11]. A weight initialization approach is proposed for optimizing the convergence speed and bringing down the sensitivity of the initial values. The experimental results reflected that the proposed algorithm can obtain better separation quality, speed up the convergence and avoid the uneven convergence speed [12]. After 20 years of development, much improved FastICA algorithms have been proposed. Whereas there is no better way to improve the convergence speed and overcome the initial value sensitivity problem at the same time. Therefore, in this paper it try to come up with a method that cannot only reduce the sensitivity of initial value and but also accelerate the convergence speed.

With the rapid development of industry, the automobile manufacturing industry is also developing vigorously. As an indispensable means of transportation for people to travel, the number of cars continues to increase. However, there are complex noises in the process of driving, which has caused many experts and scholars at home and abroad to pay attention to the research of automobile noise reduction. The initial noise reduction is based on the causes of the internal noise of cars, and a series of methods of physical noise reduction are proposed. People divide the driving environment noise sources into two types: one is the external noise which is directly transmitted to the vehicle through the air. Secondly, the vehicle vibration noise is transmitted and radiated to the vehicle through the structure. Therefore, the noise source is generally determined first, and then targeted performance optimization, sound insulation and absorption measures are taken to eliminate the noise. In addition, many automobile manufacturers have adopted improved manufacturing techniques to reduce the noise of automobiles, so as to achieve the goal of noise reduction. In the driving environment, in addition to all kinds of external noise, there is also the voice of other speakers in the car. Because these sounds are similar to the driver’s voice in time domain and frequency domain, it is difficult to separate the driver’s voice with the common blind source separation algorithm. At present, most of the blind source separation methods applied in driving environment are based on kurtosis and negative entropy maximization of FastICA algorithm.

In this first section, it briefly introduced the problem of blind source separation it were concerned about, and reviewed some different FastICA algorithms which were proposed to solve the BSS. A thorough discussion of the previous FastICA algorithms with their merits and weaknesses had been carried out from two sides. On the one hand, the sensitivity of initial value was discussed after reviewing a series of improved FastICA algorithms, and the stability of the FastICA algorithm was enhanced by different methods. On the other hand, the calculation amount was discussed and analyzed, and several efficient FastICA algorithms were carried out to reduce the amount of calculation and improve the calculation speed. In this work, it put forward a relatively novel FastICA algorithm to overcome the shortcomings. In the second section, it focused on the analysis of the fundamental principle of FastICA algorithm, and tried to explain the shortcomings of the algorithm theoretically. In the third section, it put forward our innovation in view of the deficiency of FastICA algorithm, and theoretically deduce and prove the feasibility and innovation of our method. In the fourth section, it proposed a novel FastICA algorithm, which was based on the original FastICA algorithm, and it’s very effective to enhance the stability of the algorithm and improve the iterative speed. In Section 5, it validated and tested our novel FastICA algorithm with several sets of comparative experiments. According to the analysis of experimental results, it discuss the advantages and disadvantages of our algorithm. The Section 6 summarize our proposed algorithm and generalize the experimental results, and forecast the next research direction about BSS.

The main contributions of this article include the following, 1) The gradient descent method is introduced to reduce the sensitivity of the initial value. 2) Combining secant method, a new FastICA algorithm based on improved secant method is proposed to reduce the computation amount. 3) On the condition of guaranteeing the separation performance, the new algorithm can enhance the convergence speed notably and improve the stability of convergence remarkably compared with FastICA.

2 Previous work

In the previous section, it introduced the recent advances of ICA algorithms. Generally, the ICA method combines the choice of an objective function and an suitable optimization algorithm. In other word, the ICA method is estimated by formulating a objective function which is an indicator of independence in some way and then minimizing or maximizing it. Many improved ICA algorithms have been proposed to enhance numerical stability and increase convergence speed in some way, such as Infomax ICA [13], JADE [14], SOBI [15], FastICA [5 , 17], and so on. One of the most famous and effective solutions for BSS problem is FastICA, owing to its simplicity and fast convergence. It is based on Newton iterative method, which is in order to realize the maximization of the Non-Gaussianity of the independent components hidden in the mixed signals. In this section, it analyze the fundamental principle of FastICA algorithm. The FastICA algorithm mainly consists of signal preprocessing and independent components extraction.

A. Signal preprocessing

It is very useful and necessity to do some signal preprocessing (centering and whitening), which can make the estimation of FastICA algorithm simpler and convergence performance better. Therefore, discussions of signal preprocessing are carried out before proposing new FastICA algorithm to cope with the BSS problem.

1) Centering

Centering is intent to simplify the mixed signal and reduce the influence of noise. Zero-mean vector is obtained by subtracting the mean of mixed signals from the original mixed vector. Centering is as follows: $x \leftarrow x - \frac{1}{n} \sum_{i = 1}^{i = n} x_{i}$ (1) where x = [x₁, x₂, . . . , x_n] ^T is mixed signal, also called as observed signal. Zero-mean vector, which is also called by centered data, is used to instead of the original mixed vector after the centering.

2) Whitening

Whitening is a good idea to remove the correlativity of the centering data and reduce the computational dimension at the same time. The whitening can be regarded as a linear multiplication of the new obtained zero mean vector x with the whitening matrix V. The vector that it get after the whitening is called whitening vector z. Whitening is as follows: $z = Vx$ (2) where the whitening matrix V is usually obtained using principal component analysis (PCA). In this way, it can reduce the dimension of the mixed signals. In addition, when the number of mixed signals is more than the number of source signals, the number of independent components which hid in mixed signals can be approximately estimated as the same number as the source signals through the PCA algorithm. The whitening matrix V can be obtained as follows: $V = {ED}^{- 1 / 2} E^{T}$ (3) where E is the orthogonal matrix consisting of the eigenvectors of the centering data. D = diag (d₁, . . . , d_n) denotes the diagonal matrix consisting of the eigenvalues which corresponds to the eigenvectors. $D^{- 1 / 2} = diag (d_{1}^{- 1 / 2}, . . ., d_{n}^{- 1 / 2})$ means to square the elements of the diagonal matrix. Therefore, the whitening vector z of the centering data can be obtained through the whitening process. $z = Vx = {ED}^{- 1 / 2} E^{T} x$ (4)

b. Independent components extraction

The central limit theorem plays a significant role in ICA model. As the central limit theorem stated, the sum of even two independent identically distributed random variables are more Gaussian than the original random variables. This implies that independent random variables are more non-gaussian than their mixtures. Thus, nongaussianity is a measure of independence.

The characteristic property component of the source signals is statistically independent and has non-gaussian characteristics.

A optimal quantitative measure of nongaussianity is negentropy which is based on the information theoretic differential entropy. The entropy of data is related to the information that is observed. The more random and unpredictable the data is, the larger entropy it will have. Therefore, FastICA algorithm takes the negentropy as an approximate and optimal measure of nongaussianity to complete the estimation of the source signals. Negentropy is the maximum for nongaussian random variable, which is approximatively defined as: $J (y) \approx ρ [E {G (y) - E {G (v)}}]^{2}$ (5) where J (.) is the negentropy of a gaussian random variable. y represents a random variable which is assumed to be of zero mean and unit variance. ρ is a positive constant. v is a standard Gaussian variable of zero mean and unit variance. E {.} is the mean of data.

According to the central limit theorem, the independent component is of non-gaussian characteristics respectively, which belongs to the Gauss distribution in the whole mixed signals. Hence, for the sake of obtaining more robust estimators, the function G (.) is usually nonquadratic function. In particular, it is better to choose the function which grows slowly as the variable growing. The following functions had been proved very useful [8, 17]: $G_{1} (y) = \frac{1}{a_{1}} log cosh (a_{1} y)$ (6) $G_{2} (y) = - exp (- \frac{y^{2}}{2})$ (7) $G_{3} (y) = \frac{1}{4} y^{4}$ (8) where a₁ ∈ [1, 2] is a suitable constant which is randomly selected.

The FastICA algorithm can be simply constructed as an iterative algorithm for finding an optimal demixing matrix w. In this way, maximum of the nongaussianity of w^Tz is of availability to estimate the independent components of source signal hidden in the mixed signals. The local maximum value of the approximation of the negentropy of w^Tz is usually obtained at a certain maximal maxima point of E {G (w^Tz)}. According to the Lagrangian condition, the optimal maxima of E {G (w^Tz)} under the condition E {(w^Tz) ²} = ∥ w ∥ ² = 1 is obtained at the point where the gradient of the Lagrange multiplier function is equal to zero. Therefore, obtaining the local maximum of the negentropy J_G (w) can be transformed into the optimal maxima problem of E {G (w^Tz)}. J_G (w) approximately is represented as: $J_{G} (w) = E {G (w^{T} z)} + β ({∥ w ∥}^{2} - 1)$ (9) where β, called as Lagrange multiplier, is a constant. Calculating partial derivative of formula on both sides respectively, it make partial derivatives equal to zero for obtaining the maxima value point. $E {z g (w^{T} z)} + β w = 0$ (10)

among g (.) = G (.) ′.

The objective function F (w) is expressed as: $F (w) = E {z g (w^{T} z)} + β w$ (11)

Newton iterative method, a suitable optimization algorithm, is used to optimize the objective function. Newton iterative method is as follows: $x_{n + 1} = x_{n} - \frac{f (x_{n})}{f^{'} (x_{n})}$ (12)

Thus, the following approximative Newton iteration formula of w is obtained as: $w_{n + 1} = w_{n} - \frac{E {z g (w_{n}^{T} z)} + β w_{n}}{E {{zz}^{T} g^{'} (w_{n}^{T} z)} + β}$ (13) where g′ (.) is the derivative of g (.). To simplify the operation of this matrix, after the centering and whitening, a rational and approximate formula seems to be as: $E {{zz}^{T} g^{'} (w_{n}^{T} z)} = E {{zz}^{T}} \cdot E {g^{'} (w_{n}^{T} z)} = E {g^{'} (w_{n}^{T} z)} I$ (14)

Hence, get the approximate iterative formula: $w_{n + 1} = w_{n} - \frac{E {z g (w_{n}^{T} z)} + β w_{n}}{E {g^{'} (w_{n}^{T} z)} + β}$ (15)

Further to simplify the formula by multiplying the denominator. The w_n+1 was normalized to improve the stability of value iteration. Therefore, the iterative formula of FastICA algorithm based on negentropy as follows: ${\begin{matrix} w_{n + 1} = E {z g (w_{n}^{T} z)} - E {g^{'} (w_{n}^{T} z)} w_{n} \\ w_{n + 1} = w_{n + 1} / ∥ w_{n + 1} ∥ \end{matrix}$ (16)

The purpose of FastICA is to find the optimal demixing matrix w. The separated signals Y = [y₁, . . . , y_n] ^T, which are also called as independent components, are obtained by multiplying the demixing matrix with the mixing matrix. The basic principle of FastICA algorithm is as follows: $Y = w^{T} x$ (17)

The independent components, which are excellently estimated by the FastICA algorithm, can be regarded as the best approximation signals of the source signals.

The basic steps of FsatICA algorithm based on negentropy are given as follows:

Step 1: Preprocessing for mixed signals.

Step 2: Randomly choose an initial demixing matrix(weight vector) w₀ with unit norm.

Step 3: Let $w_{n + 1} = E {z g (w_{n}^{T} z)} - E {g^{'} (w_{n}^{T}$ z)} w_n

Step 4: Let w_n+1 = w_n+1/ -∥ w_n+1 ∥

Step 5: If it is not converged, then n = n + 1 and get back to step 3.

From the basic form of the algorithm, it can see that it depend on the choice of initial demixing matrix and the convergence of the iterative process. Therefore, it want to do some improvements to enhance the performance of the algorithm in these two aspects.

C. Noise characteristics in driving environment

There are many kinds of noise in driving environment, but it can be divided into two kinds: non-speech noise and speech noise.

Non-speech noise. The non-speech noise in driving environment mainly includes two parts: the internal noise and the external noise in the process of driving.

Voice noise. The on-board electronic products need to recognize the driver’s voice, but in the process of driving, the voice of other passengers in the car will inevitably affect the driver’s voice recognition effect. This kind of speech noise is the most difficult to eliminate. Because the voice signals of driver and passenger overlap in time domain and frequency domain. Therefore, speech noise is removed from the perspective of blind source separation.

3 Improved Secant Method

As is known to all, the deficiency of the Newton iterative method is that the value of the derivative of the function f (x) need to be calculated at each iteration. Some computation processes are either very difficult to complete or time-consuming [18]. In order to overcome the shortcomings, it try to propose a new method instead of it to optimize the objective function.

A. S ecant M ethod

The secant method is also called the difference quotient method, which is a method of improvement based on Newton iterative method. It can be simply described as that using the slope of the secant formed by two points (x_n, f (x_n)) and (x_n-1, f (x_n-1)) of the function f (x) to approximatively instead of the slope of the tangent at the point (x_n, f (x_n)). An approximative process is as follows: $f^{'} (x_{n}) \approx \frac{f (x_{n}) - f (x_{n - 1})}{x_{n} - x_{n - 1}}$ (18)

Combining (12) and (18), the new iterative formula can be deduced as follow: $x_{n + 1} = x_{n} - \frac{f (x_{n}) (x_{n} - x_{n - 1})}{f (x_{n}) - f (x_{n - 1})}$ (19)

This iterative formula is referred to as the secant method.

B. Improved Secant Method

The order of convergence of the secant method is 1.618, which is slightly lower than the second order of convergence for Newton iterative method. The iteration speed of the secant method is less than Newton iterative method sometimes. Neglecting all other costs, the conjecture of the two iterations secant method was put forward, and the rough guess is that the order of convergence of it is about 2.6 [19, 26]. In view of this conjecture, it modify it and present an improved secant method to make up for the shortcoming of the lack of convergence order and the slow iteration speed of the secant method. The improved secant method is as follows: ${\begin{matrix} {\bar{x}}_{n + 1} = x_{n} - \frac{(x_{n} - x_{n - 1}) f (x_{n})}{f (x_{n}) - f (x_{n - 1})} \\ x_{n + 1} = x_{n} - \frac{({\bar{x}}_{n + 1} - x_{n}) f (x_{n})}{f ({\bar{x}}_{n + 1}) - f (x_{n})} \end{matrix}$ (20)

Where the ${\bar{x}}_{n + 1}$ is the solution of secant method and x_n+1 is the solution of improved secant method. In the next section, proof is provided to verify the significance of the improved secant method it proposed.

C. Order of convergence of improved secant method

Reference to the proof of the order of convergence of the secant method [18 –20], it prove that the order of convergence of the improved secant method is 2.414. Proof is as follows:

Definition: Suppose that it are solving the equation f (x) =0 using the secant method, which has a real root x^*. It will assume that the sequence {x_n} converges to x^*, which the iteration value x_n is the variate of each iteration and lies in a sufficiently small neighborhood of x^*. Let e_n = x_n - x^*, which represents the error of the n times iteration. If there exists a positive constant p ⩾ 1 and k ≠ 0, such that: $lim_{n \to \infty} \frac{e_{n + 1}}{e_{n}^{p}} = k$ . Thus, the sequence {x_n} is said to converge to x^* with order of convergence p. The number p is called the convergence factor and the error $e_{n + 1} = {ke}_{n}^{p}$ is called accumulative error. Obtained by the first fraction of (20): ${\bar{x}}_{n + 1} = \frac{f (x_{n}) x_{n - 1} - f (x_{n - 1}) x_{n}}{f (x_{n}) - f (x_{n - 1})}$ (21)

Combined the iterative error of e_n = x_n - x^* and e_n-1 = x_n-1 - x^*: ${\bar{e}}_{n + 1} = \frac{f (x_{n}) e_{n - 1} - f (x_{n - 1}) e_{n}}{f (x_{n}) - f (x_{n - 1})}$ (22)

According to the Lagrange Mean Value Theorem, the existence of x = ξ_n between x_n and x^* makes the formula tenable. $f^{'} (ξ_{n}) = \frac{f (x_{n}) - f (x^{*})}{x_{n} - x^{*}}$ (23)

So there is the following deformation formula: $f (x_{n}) = e_{n} f^{'} (ξ_{n})$ (24)

Similarly, $f (x_{n - 1}) = e_{n - 1} f^{'} (ξ_{n - 1})$ (25)

Obtained by bringing (24) and (25) into (22):

$\begin{matrix} {\bar{e}}_{n + 1} = \frac{e_{n} f^{'} (ξ_{n}) e_{n - 1} - e_{n - 1} f^{'} (ξ_{n - 1}) e_{n}}{f (x_{n}) - f (x_{n - 1})} \\ = e_{n} e_{n - 1} \frac{f^{'} (ξ_{n}) - f^{'} (ξ_{n - 1})}{f (x_{n}) - f (x_{n - 1})} \end{matrix}$ (26)

Simplify the formula, ${\bar{e}}_{n + 1} = {\bar{ke}}_{n} e_{n - 1}$ (27) where $\bar{k}$ is a real number. By definition $e_{n + 1} = {ke}_{n}^{p}$ , which can be deduced from the two formulas is as follows: ${\bar{e}}_{n + 1} = {ke}_{n}^{p} = {\bar{ke}}_{n} e_{n - 1} = {\bar{ke}}_{n} (\frac{e_{n}}{k})^{\frac{1}{p}} = {\bar{kk}}^{- \frac{1}{p}} e_{n}^{1 + \frac{1}{p}}$ (28)

Thus, $p = 1 + \frac{1}{p}$ (29)

Due to p ⩾ 1, $p = (1 + \sqrt{5}) / 2 \approx 1.618$ . Therefore, this is the order of convergence of the secant method.

Analogously, obtained by the second fractions of (20).

$\begin{matrix} e_{n + 1} = \frac{f ({\bar{x}}_{n + 1}) e_{n} - f (x_{n}) {\bar{e}}_{n + 1}}{f ({\bar{x}}_{n + 1}) - f (x_{n})} \\ = {\bar{e}}_{n + 1} e_{n} \frac{f^{'} ({\bar{ζ}}_{n + 1}) - f^{'} (ζ_{n})}{f ({\bar{x}}_{n + 1}) - f (x_{n})} \end{matrix}$ (30) Homoplastically, $e_{n + 1} = k^{*} {\bar{e}}_{n + 1} e_{n}$ (31) where k^* is a real number. Similar to (28),

$\begin{matrix} e_{n + 1} = k^{*} ({\bar{ke}}_{n} e_{n - 1}) e_{n} = k^{*} {\bar{ke}}_{n}^{2} e_{n - 1} \\ = k^{*} {\bar{ke}}_{n}^{2} (\frac{e_{n}}{k})^{\frac{1}{p}} = {Ce}_{n}^{2 + \frac{1}{p}} \end{matrix}$ (32) where C is a real number. By definition $e_{n + 1} = {ke}_{n}^{p}$ , which can be obtained by the two formulas is as follows: $p = 2 + \frac{1}{p}$ (33)

Hence, $p = 1 + \sqrt{2} \approx 2.414$ . The improved secant method has at least 2.414 orders of convergence, and the order of convergence is obviously larger than the two order of convergence of the Newton iteration method.

4 Improved FastICA Algorithm

A. Optimized initial demixing matrix

In order to make the initial demixing matrix convergence as soon as possible at the beginning of the iteration and reduce the influence of initial value sensitivity, it introduce the gradient method to optimize the initial matrix. The gradient descent method is also called steepest descent method, which is in order to get the minimalization by the fastest direction of the gradient descent [9, 21]. Simply, it can be described as: $f (x_{n + 1}) = min f (x_{n} + λ p (x_{n}))$ (34) where λ is the iteration step, $λ = - [\begin{matrix} \frac{\partial E {z g (w^{T} z)}}{\partial w_{1}} & 0 & \dots \\ 0 & ⋱ & 0 \\ ⋮ & 0 & \frac{\partial E {z g (w^{T} z)}}{\partial w_{n}} \end{matrix}]$ (35)

In addition, p (x) is the gradient direction, which is used to constantly correct the direction x_n+1 = x_n + λp (x). Combining (11) it can get: $p (w) = E {z g (w^{T} z)}$ (36)

Therefore, the optimized formula of w can be acquired. $w_{n + 1} = w_{n} + λ p (w_{n}) = w_{n} + λ E {z g (w_{n}^{T} z)}$ (37)

The basic pattern of the steepest descent method is as follows:

Step 1: Generate the initial demixing matrix w randomly.

Step 2: Calculate the negative slope of E {zg (w^Tz)} for the iteration step λ.

Step 3: Update the defining matrix, $w_{n + 1} = w_{n} + λ E {z g (w_{n}^{T} z)}$ .

Step 4: If |w_n+1 - w_n| < ɛ, stop. Otherwise, go back to step3 and continue iterating.

The independent components belong to the gaussian distribution, according to the normal distribution principle 3 - σ, so ɛ = 0.00135.

The initial demixing matrix is optimized by gradient descent method. The simulation results show that the suggested algorithm is capable to bring down the sensitivity of initial values, accelerate the convergence speed of FastICA algorithm at the beginning stage of iteration and enhance the stability of convergence of the algorithm overall.

B. FastICA algorithm based on secant method

In order to reduce the calculation of FastICA algorithm during the iteration process, a novel FastICA using the secant method is proposed. Combining the target function (11) and the secant method (19), it can get the iterative form of demixing matrix. $w_{n + 1} = w_{n} - \frac{F (w_{n}) (w_{n} - w_{n - 1})}{F (w_{n}) - F (w_{n - 1})}$ (38)

Therefore, the iterative formula of FastICA algorithm based on secant method (SFastICA) is as follows: $w_{n + 1} = E {z g (w_{n}^{T} z)} w_{n - 1} - E {zg (w_{n - 1}^{T} z)} w_{n}$ (39)

The secant method has been shown to reduce the computation and time-consuming [18, 22]. The SFastICA algorithm avoids the derivation, and reduces the calculation. However, the order of convergence of SFastICA algorithm is slightly lower than that of FastICA algorithm. Thus, it will make further improvements.

C. FastICA algorithm using improved secant method

For the sake of improving the convergence order of the secant method and speeding up the convergence, it propose the FastICA algorithm based on improved secant method (ISFastICA). The improved secant method has proven that the order of convergence is 2.414, which has higher convergence order and faster convergence speed than that of Newton iterative method. The new iterative formula can be obtained by combining the target function (11) and improved secant method (20). ${\begin{matrix} {\bar{w}}_{n + 1} = w_{n} - \frac{(w_{n} - w_{n - 1}) F (w_{n})}{F (w_{n}) - F (w_{n - 1})} \\ w_{n + 1} = w_{n} - \frac{({\bar{w}}_{n + 1} - w_{n}) F (w_{n})}{F ({\bar{w}}_{n + 1}) - F (w_{n})} \end{matrix}$ (40)

Thus, the basic iterative formula of FastICA algorithm based on improved secant method (ISFastICA) can be derived. ${\begin{matrix} {\bar{w}}_{n + 1} = E {z g (w_{n}^{T} z)} w_{n - 1} - E {z g (w_{n - 1}^{T} z)} w_{n} \\ w_{n + 1} = E {z g (({\bar{w}}_{n + 1})^{T} z)} w_{n} - E {z g (w_{n}^{T} z)} {\bar{w}}_{n + 1} \\ w_{n + 1} = w_{n + 1} / ∥ w_{n + 1} ∥ \end{matrix}$ (41)

The FastICA algorithm based on improved secant method is as follows:

Step 1: Centering preprocessing, $x \leftarrow x - \frac{1}{n} \sum_{i = 1}^{i = n} x_{i}$ .

Steps: Whitening preprocessing, z = ED^-1/2E^Tx.

Step 3: Randomly generate the initial demixing matrix and optimize it using the gradient descent method.

Step 4: update the demixing matrix from (41) ceaselessly.

Step 5: if |w_n+1 - w_n| < ɛ, stop. Else, go back to step4. Where ɛ = 0.0000315 according toprinciple.distribution principle 4 - σ.

Step 6: Estimate separated signals, Y = w^Tx.

In theory, convergence performance of ISFastICA algorithm is obviously better than that of SFastICA algorithm and FastICA algorithm. ISFastICA algorithm not only reduces the sensitivity of initial values effectively, but also has higher convergence order and faster convergence rate. In conclusion, ISFastICA algorithm has the advantage of separation performance and robustness.

5 Simulation results and analysis

It carry out simulation experiments in Grid corpus [22 –24] to compare the separation performance of ISFastICA algorithm, SFastICA algorithm and FastICA algorithm, which the speech segments are selected from Grid corpus composed of 17000 sentences (500 from each of the 34 talkers) with the average length of 2 seconds. Firstly, three clean voice signals are selected, which belong to three different talker’s voices respectively, such as s2_bbbs5p.wav, s5_bbiy7 s.wav, s6_bbwz7n.wav. Secondly, multiply the clean speech by a mixed noise matrix which is randomly generated and get mixed signals. Thirdly, three algorithms are used to separate the mixed signals, and isolate three estimation signals. Finally, the signal-to-noise ratio, iteration time, separation time of the three algorithms is statistically calculated. In addition, the convergence characteristics and the sensitivity of the initial value are analyzed.

A. Comparison of Separation performance

The sample waveform of the clean speech signal is shown in Fig. 1. It is represented as the source signals of three talkers. The horizontal axis is denoted the sampling point, and the vertical axis is signified the waveform amplitude.

Fig. 1

Three clean speeches of different talker.

It deal with three source signals by multiplying a generated mixture matrix randomly, and get the waveform of the mixed signal as shown in Fig. 2.

Fig. 2

Three mixed speeches.

Three algorithms are used to separate and estimate the mixed signals, and the separation results of different algorithms are obtained as shown in Fig. 3.

Fig. 3

(a) Separation the speeches of FastICA. (b) Separation the speeches of SFastICA. (c) Separation the speeches of ISFastICA.

In general, the speech waveforms cannot show the stand or fall of the separation performance directly. Therefore, it should use some performance parameters to evaluate the performance of the algorithms, such as the signal to noise ratio(SNR), the number of iterations, the separation time.

The signal to noise ratio is used to evaluate the effect of the separation, which can be simply described as the ratio of the effective components to noise components in the signal [24, 25]. Generally, the signal to noise ratio is great, and the separation effect is better. The SNR metric is defined as (42), where s represents the clean source signals, and $\hat{s}$ represents the separated estimation signals.

$SNR = 10 * {log}_{10} (\frac{\sum_{i = 0}^{n} s_{i}^{2}}{\sum_{i = 0}^{n} (s_{i} - {\hat{s}}_{i})^{2}})$ (42)

In this section, a repeated experiment is designed to measure separation performance with calculating the average SNR, average iteration time and average separation time of the three separation algorithm. Firstly, three people’s speeches from 34 different people are selected randomly, which one people had 500 speeches. Secondly, three different speeches from three different people multiply with the same mixed noise matrix generated before the 100 times repeated experiment. Thirdly, the average SNR, average iteration time and average separation time of the three separation algorithm are calculated by repeating 100 times. Fourthly, in turn, the remaining speeches of the 500 speeches of each people are used for counting 100 times respectively. Finally, the different mean value of each different algorithm is used for the comparison of separation performance. The purpose of this repeated experiment is to prevent some randomly generated noise matrix from being easy to separate for a person’s speech, which leads to inaccurate iteration times. Each person’s speech is repeated for 100 times and multiplied by the same noise matrix in order to better measure the separation performance under relatively fixed initial conditions. The results are presented in Table 1.

Table 1

Comparison of the separation performance

Algorithm	SNR[dB]	Iteration [times]	Separation time[s]
FastICA	40.5719	19	0.2521
SFastICA	41.6004	16	0.2203
ISFastICA	42.8621	10	0.1368

As shown in the results, the mean SNR of ISFastICA, SFastICA and FastICA are 42.8621 dB, 41.6004 dB, 40.5719 dB respectively. The SNR of ISFastICA increases by 5.64% compared with that of FastICA, and improves by 3.03% than SFastICA. The average iteration time of them is 19,16,10 severally. The iteration time of ISFastICA decrease by 47.37% related to that of FastICA, and reduce by 37.5% than SFastICA. The average separation time of them are 0.2521 s, 0.2203 s, 0.1368 s respectively. The average separation time of ISFastICA is less than that of FastICA by 45.74%, and is lower than SFastICA by 37.9%.

B. Comparison of Convergence

For the purpose of evaluating the calculation of each iteration and the speed of convergence, it made a detailed comparison. Customarily, the number of iterations can be used as a standard for measuring the complexity of algorithms. Therefore, it perform the experiment to compare iteration times of the different algorithms and the difference between the front-back two iteration values in the iterative process. it record the difference of the renewed demixing matrix between each iteration, and draw a line graph with the number of iterations and the variation of the front-back difference, which is as shown in Fig. 4.

Fig. 4

Comparison of convergence.

As it can see, the ISFastICA algorithm is denoted using a bold line with square notation. The SFastICA algorithm is represented by a standard solid line with triangular marks. The FastICA algorithm is showed by a thin solid line marked by the rhombus. The number of iterations of ISFastICA, SFastICA and FastICA are 11,16 and 18 respectively. The descent speed of ISFastICA algorithm is much faster than that of SFastICA algorithm and FastICA algorithm, which means that it converges faster. It is obvious that ISFastICA algorithm and SFastICA algorithm decrease faster than FastICA at the beginning of the iteration. However, the iteration speed of SFastICA in the mid-iteration is slightly lower than that of FastICA, and the fold lines of them form a cross. Eventually, the iterations of SFastICA algorithm and FastICA algorithm are similar. Thus, compared with the other two algorithms, ISFastICA has better convergence performance and faster convergence speed.

C. Comparison of the sensitivity of initial value

It conducted 10 times experiments on the mixed signals of three talkers, which is to compare the sensitivity of initial value. Different initial values of the demixing matrix are generated at random, and then the number of iterations of different algorithms is recorded and analyzed as shown in Table 2.

Table 2

Comparison of the sensitivity

No.	Iteration times
	FastICA	SFastICA	ISFastICA
1	18	16	11
2	24	13	7
3	23	15	10
4	21	13	8
5	16	17	11
6	20	15	7
7	18	11	9
8	25	14	7
9	22	13	10
10	16	17	7
Mean	20.3	14.4	8.7
Variance	10.46	3.82	2.90

As shown on the conditions of 10 different initial demixing matrix, the mean iteration times of ISFastICA, SFastICA and FastICA are 20.3, 14.4, 8.7 respectively, and the variance of the number of iterations of them are 10.46, 3.82, 2.90 severally. ISFastICA has the fewest numbers of iterations and the fastest rate of convergence, which means that the sensitivity of initial value of ISFastICA is smallest. In addition, the rate of convergence of ISFastICA is faster than that of FastICA, which illustrates that the gradient method proposed optimizing the initial demixing matrix is very effective for reducing the sensitivity.

In order to visually display the influence of the initial value of demixing matrix generated by each iteration on the convergence performance of the iteration, the number of iterations of these three algorithms is plotted as a line graph in Fig. 5.

Fig. 5

Comparison of iteration times.

As the graph shows, the ISFastICA algorithm is denoted using a black bold line with square notation. The SFastICA algorithm is represented by a solid blue line with triangular marks. The FastICA algorithm is showed by a thin green line marked by the rhombus. The iterations times of ISFastICA has a more dense distribution. The fluctuation of the broken line amplitude is the least than the other two algorithms. Therefore, convergence performance of ISFastICA algorithm is the most stable for different initial value of demixing matrix.

In order to study the effect of initial values on the separation results, three groups of initial matrices were randomly selected to compare the iteration time and similarity coefficient matrix of the different separation algorithm between the FastICA (before improvement) and ISFastICA (after improvement). The experimental results as shown in Table 3.

Table 3

Comparison of algorithms before and after improvement

		The iteration time (s)		Similarity coefficient matrix
The initial matrix		Before improved	After improved	Before improved			After improved
0.833	0.053			1.001	0.010	0.003	1.000	0.004	0.003
0.122		1.23	1.12	0.002	1.000	0.002	0.003	1.000	0.001
0.097	0.855			0.004	0.003	1.000	0.002	0.003	1.000
0.903	0.017			0.011	1.010	0.001	0.011	1.001	0.001
0.143		1.67	1.13	0.072	0.002	0.997	0.021	0.002	0.999
0.106	0.873			0.997	0.005	0.073	0.999	0.006	0.001
0.914	0.107			0.083	0.263	0.966	0.001	0.003	1.001
0.077		1.77	1.17	0.998	0.003	0.089	1.000	0.001	0.002
0.099	0.885			0.034	0.972	0.265	0.004	1.001	0.003

It can be seen from Table 3 that the improved separation algorithm has less iteration time and iteration times than the improved algorithm before, and the iteration time and iteration times fluctuate in a small range, which solves the problem that the FastlCA algorithm based on negative entropy is sensitive to the initial value. It can be seen from the improved similarity coefficient in Table 3 that the improved algorithm has slightly improved the separation effect. In each iteration process of Newton’s down-hill method, the current value and the calculation result of the previous step need to be weighted and averaged, which is a little more complicated than that of Newton’s iteration method. However, it can be seen from the results that the number of iterations of the improved algorithm is reduced, so the running time does not become longer.

D. Analysis of Results

Compared with the experimental results of three algorithms, the separation performance of ISFastICA is significantly better than that of SFastICA and FastICA. Not only the SNR is higher, but also the number of iterations and separation time are less. Compared with the convergence of these three algorithms, ISFastICA algorithm has the lowest convergence order and the fastest convergence rate. ISFastICA requires the fewest numbers of iterations over different initial values. In summary, ISFastICA algorithm can reduce the effect of initial value and speed up the convergence rate, and enhance the stability of convergence.

6 Conclusion

In this paper, different existing algorithms for FastICA are briefly illustrated. After analyzing the basic principle of FastICA algorithm, it propose a novel method to overcome the shortcomings of it. First, the gradient descent method which put forward to optimize initial demixing matrix is very good to reduce the effect of initial value. Second, the improvement of the secant method is greatly proposed raising the convergence speed and reduce the time-consuming. What’s more, it has proven that the convergence order and convergence rate of the new FastICA algorithm based on improved secant method (ISFastICA) are obviously better than other FastICA algorithms in theory. Finally, according to the experimental results, it can obtain that the proposed ISFastICA algorithm is better than other FastICA algorithm in both separation performance and separation efficiency. Although the performance of our algorithm has improved in some degree, it find that the separated signal maybe distorted or even damaged in the experiment. High fidelity for blind source separation is our further investigation. The algorithm in this paper is applicable to a limited number of scenarios and performs well in driving environment with low SNR. It can be further optimized to make the application scenarios more extensive.

Footnotes

Acknowledgments

This work was supported, the National Natural Science Foundation of China (Grant No. 61871039, 61932012, 61906017, 61802019), the Beijing Municipal Commission of Education Project (No.KM202111417001, KM201911417001), National Engineering Laboratory for Agriproduct Quality Traceability Project (No. AQT-2020-YB2), the Supporting Plan for Cultivating High Level Teachers in Colleges and Universities in Beijing (Grant No. IDHT20170511), the Academic Research Projects of Beijing Union University(No. BPHR2019AZ01, ZK80202001, XP202015, BPHR2020EZ01).

References

Xie

, Xie

, Yang

, Wu

and Xie

, Underdetermined Reverberant Audio-Source Separation Through Improved Expectation–Maximization Algorithm, Circuits Systems and Signal Processing 38(6) (2019), 2877–2889.

Singh

and Singh

, Separation of Linearly Mixed Speech Signals using DWT based ICA, International Journal of Computer Applications 70(28) (2013), 27–31.

Qian

, Li

and Liao

, Performance Analysis of the nc-FastICA Algorithm, Circuits Systems and Signal Processing 34(2) (2015), 441–457.

Jutten

and Taleb

, Source separation: from dusk till dawn, In Proc. 2nd Int. Workshop on Independent Component Analysis and Blind Source Separation (ICA2000) (2000), 15–26.

Hyvärinen

and Oja

, Independent component analysis: algorithms and applications, Neural Networks 13(4-5) (2000), 411–430.

Zhang

, Improved FastICA algorithm using a sixth-order Newton’s method, IEICE Electronics Express 6(13) (2009), 904–909.

Ahmad

, Alias

, Ghanbari

and Askaripour

, Improved Fast ICA Algorithm Using Eighth-Order Newton’s Method. Research Journal of Applied Sciences, Engineering and Technology 6(10) (2013), 1794–1798.

Luo

W.J.

, Yuan

L.F.

and He

Y.G.

, Improved fastICA algorithm based on fifteen-order newton iteration, Comput Eng Appl 52(20) (2016), 108–113.

Voss

J.R.

, Rademacher

and Belkin

, Fast algorithms for Gaussian noise invariant independent component analysis, In Advances in neural information processing systems (2013), 2544–2552.

10.

Zhang

R.F.

and Ye

J.M.

, Adaptive quasi-newton algorithm for blind source separation based on canonical correlation analysis, Journal of Jilin University (Science Edition) 55(3) (2017), 559–563.

11.

Wei

G.U.O.

and Fengqin

Y.U.

, Speech-music separation based on improved algorithm of negentropy maximization, Computer Engineering and Applications (4) (2015), 45.

12.

Yadav

R.K.

, Mehra

and Dubey

, Blind audio source separation using weight initialized independent component analysis. In 2015 1st International Conference on Next Generation Computing Technologies (NGCT) (pp. 563–566). IEEE. (2015).

13.

Khaliq

A.A.

, Qureshi

I.M.

, Malik

S.A.

and Shah

J.A.

, A Modified Infomax ICA Algorithm for fMRI Data Source Separation, Research Journal of Applied Sciences, Engineering and Technology 5(20) (2013), 4862–4868.

14.

Zhang

, Tian

and Lan

, Blind source separation based on JADE algorithm and application. In 3rd International Conference on Mechatronics, Robotics and Automation. Atlantis Press. (2015).

15.

Zhang

, Vialatte

F.B.

, Chen

, Rathi

and Dreyfus

, Embedded implementation of second-order blind identification (sobi) for real-time applications in neuroscience, Cognitive Computation 7(1) (2015), 56–63.

16.

Yuan

and Zhang

L.Y.

, Blind image separation based on an optimized fast fixed point algorithm, Advanced Materials Research 756–759 (2013), 3578–3583.

17.

X.S.

, He

and He

A.L.

, Super-Gaussian BSS Using Fast-ICA with Chebyshev–Pade Approximant, Circuits Systems, and Signal Processing 37(1) (2018), 305–341.

18.

Ehiwario

J.C.

, Comparative Study of Bisection, Newton-Raphson and Secant Methods of Root-Finding Problems, IOSR Journal of Engineering 4(4) (2014), 01–07.

19.

Kumar

, Gupta

D.K.

and Srivastava

, Influence of the Center Condition on the Two-Step Secant Method, International Journal of Analysis 2017 (2017), 1–9.

20.

Long

N.N.

, Salimi

, Sharifi

and Ferrara

, Developing a new family of Newton–Secant method with memory based on a weight function, SeMA Journal 74(4) (2017), 503–512.

21.

Yadav

R.K.

, Mehra

and Dubey

22.

Czyzewski

, Kostek

, Bratoszewski

, Kotus

and Szykulski

, An audio-visual corpus for multimodal automatic speech recognition, Journal of Intelligent Information Systems 49(2) (2017), 167–192.

23.

Mowlaee

, Saeidi

, Christensen

M.G.

, Tan

Z.H.

, Kinnunen

, Franti

and Jensen

S.H.

, A joint approach for single-channel speaker identification and speech separation, IEEE Transactions on Audio Speech and Language Processing 20(9) (2012), 2586–2601.

24.

Mowlaee

and Kulmer

, Harmonic phase estimation in single-channel speech enhancement using phase decomposition and SNR information, IEEE/ACM Transactions on Audio Speech and Language Processing 23(9) (2015), 1521–1532.

25.

and Wang

D.L.

, An iterative model-based approach to cochannel speech separation, EURASIP Journal on Audio Speech and Music Processing 2013(1) (2013), 14.

26.

Lin

, Wu

, Chen

, Khan

and Liu

, The convergence ball and error analysis of the two-step Secant method, Applied Mathematics-a Journal of Chinese Universities Series B 32(4) (2017), 397–406.