Optimal preconditioned regularization of least mean squares algorithm for robust online learning 1

1 Introduction

The least mean squares (LMS) algorithm is an iterative solver of the online estimation problem that adaptively minimises the mean squares error (MSE) to generate a steady state solution of the problem. The conventional LMS algorithm can not perform well under high eigenvalue spread (condition number) of input signal’s autocorrelation [1, 2]. A stylish collection of preconditioned LMS algorithms is realized by Newton’s method that offers fast convergence speed, but at high computational cost [3–5]. However, recent studies have shown that these algorithms may have poor misalignment hence exhibiting deviation from optimal solution [6]. Variable step-size methods offer a good remedy for unbiased solution of LMS algorithm [7], but such methods may slowdown the convergence speed of the algorithm [8].

Deterministic approach, on the other hand, offers direct methods for least squares (LS) solution of the problem. The recursive least-squares (RLS) algorithms have fast deterministic methods, however, this family of algorithms have numerical instabilities and high computational cost [9]. The problems related to numerical instability are overcome by QRD-RLS based techniques, which employ stable orthogonalization tools of numerical linear algebra in finding the Cholesky factor or inverse Cholesky factor of the input data correlation matrix, and use it to obtain a direct solution of the adaptive filtering problem [10, 11]. A noticeable thing in these direct methods is the toeplitz structure of the input signal’s correlation matrix, that can be optimized to regularize the LMS algorithm. Regularization methods play an important role in improving the performance of iterative solvers of inverse problem [12]. Recently, S. Cipolla et. al. [13] have shown that regularized preconditioner for toeplitz matrices can be optimized at a very low cost. In this paper, the inverse QRD-RLS technique is optimized for known statistics of input signals to design a regularization matrix for improving the characteristics of the input signals. The toeplitz structure of the input signal’s correlation matrix that provides an ease in regularizing preconditioned LMS algorithms optimally. The input tap-vector is regularized by the optimal inverse Cholesky factor of the autocorrelation matrix of a stationary stochastic process, like autoregressive (AR) process. The regularized input signals have low eigenvalue spread that guaranty the fast convergence of the proposed method with nominal additional computational complexity. Furthermore, since the regularization matrix is optimized from the inverse Cholesky factor, the proposed optimal regularized LMS (OPRLMS) algorithm has the numerical stability similar to that of inverse QRD-RLS algorithm. Additionally, to have an unbiased solution without compromising on robustness, computational cost and mean squares deviations, an adaptive step-size is used for the development of OPRLMS algorithm. The convergence criterion of LMS algorithm sets an optimal value for step-size parameter to have minimum mean squares deviation that is a measure of convergence in misalignment. However, step-size parameter is bounded by the spectral power of input signals, and its optimal value cannot be determined a priori because statistics of input signals are not always known a priori. This problem is controlled to great extent by the use of variable step-size, or more precisely by an adaptive step-size [8]. Major focus of this paper is on stationary stochastic environment, and therefore the adaptive step-size of proposed OPRLMS algorithm is realized by an optimal power of input signal. It is adaptive in the sense that its value is optimized adaptively according to the statistics of input signals. The proposed algorithm has linear complexity as that of conventional LMS algorithm, but is much faster than it and exhibit minimum mean square deviation from optimal solution as compared with other modifications of LMS algorithms. These characteristics make is highly suitable for applications with highly correlated input signals of known statistics. This paper is organized as follows: A brief summary of the conventional LMS algorithm is given in section 2. In section  3, a Cholesky factor based preconditioning technique is described along with the main idea of optimizing it to form an optimal preconditioned regularization in section 3.1. The highly efficient optimal preconditioned regularized LMS (OPRLMS) algorithm is developed in section 4. Experimental results of section 5 present the comparative performance of proposed algorithm in application of unknown system identification and PLI noise cancelation from real ECG signal. Both applications show preference of proposed algorithm over the rest, and make is highly efficient for applications with stationary environments.

2 A review of the LMS Algorithm

The least mean squares (LMS) algorithm is an adaptive filtering methods for solving the online estimation problem that is modeled by A n x n ≈ b n ,(1) where A _n is n × N (n ⪢ N) data matrix of rank r ≤ N, formed by a sequence of input signals { θ ( n ) } n = 1 ∞ . The vector b n ∈ R n consists of desired signals, and x n = [ x n ( 0 ) , … , x n ( N - 1 ) ] T ∈ R N is tap-weight vector of length N. The objective is to predict desired signal s (n) with an online learning algorithm at time n, by estimated output signal y ( n ) = ∑ i = 0 N - 1 x n ( i ) θ ( n - i ) = x n T Θ n = Θ n T x n ,(2) where, Θ n = [ θ ( n ) θ ( n - 1 ) … θ ( n - N + 1 ) ] T ,(3) is the input vector at instant n. So that the error incurred is given by e ( n ) = s ( n ) - y ( n ) .(4) This process requires an adaptive algorithm to update filter tap-weight vector x _n recursively as new signal comes in. The least mean squares (LMS) algorithm does so by the update equation: x n + 1 = x n + 2 μ e ( n ) Θ n .(5) where μ is the step-size parameter, and controls the convergence speed and stability of the algorithm. The autocorrelation matrix Φ = E { Θ n Θ n T } statistically determines the spectral bounds for μ to control convergence behavior of the algorithm. For each n the characteristics of input signal vector Θ _n measure the deviation of filter tap weight x _n from the optimal Wiener solution x _o , and therefore this deviation can be taken as a function of input signal’s autocorrelation matrix Φ. It is evident from the past studies [1, 14] that power of input signal appears along the diagonal of Φ, therefore, eigenvalue decomposition of Φ is a very useful indicator of the properties of input signal, and hence performance of the algorithm. These eigenvalues can be used to analyze the stability of LMS based algorithms.

Let ϒ = Diag (γ₁, γ₂, …, γ _N ) be the diagonal matrix, having eigenvalues γ₁ ≤ … ≤ γ _N of Φ as its diagonal entries, then corresponding eigenvectors h₁, h₂, …, h _N make columns of an orthogonal matrix H such that an eigenvalue decomposition for Φ is Φ = H ϒ H T .(6) Furthermore, since Φ^-1 = H ϒ^-1 H ^T , the condition number (or eigenvalue spread) χ _Φ of Φ comes out to be χ Φ = γ max γ min .(7) Hence eigenvalue spread of correlation matrix Φ depends upon maximum and minimum eigenvalues only. The eigenvalue spread of Φ, given by (7), determines the convergence speed of the algorithm, and algorithm exhibits fastest convergence for χ _Φ  ≈ 1. If misalignment m _n is defined in terms of weight error vector as: m n = x n - x o ,(8) then adaptive tap weights x _n , generated with an appropriate initial estimate, will relax in a close neighborhood of optimal Wiener solution x _o if statistical mean of m _n converges to zero and variance is stable. Equations (5) and (8) yield E { m n + 1 } = ( I - 2 μ Φ ) E { m n } .(9) It shows that E {m _n } makes a geometric progression, which converges to zero if convergence parameter μ satisfies the condition μ < 1 2 γ j ; 1 ≤ j ≤ N .(10) It yields μ < 1 2 γ max ,(11) where γ _max is the largest eigenvalue of Φ. Since value of γ _max depends upon the characteristics of the input signal, and is large for highly correlated inputs, therefore, the stability of the algorithms depends upong μ whose value satisfies (11). Stability of the LMS algorithm determines the adaptation speed and an appropriate value of μ is required to make to algorithm convergent. Analysis of [1] has shown that although LMS algorithm is robust and a computationally linear, with O (N) complexity, however its rate of convergence is highly sensitive to the eigenvalue spread of input signal’s autocorrelation matrix. This draw back has lead researchers to look for some faster algorithms which are less dependent on the correlation properties of the input signals.

LMS-Newton algorithm is an ideally preconditioned algorithm, that is able to yield optimal Wiener solution in a single step in exact arithmetic [2]. In this algorithm the input vector Θ _n , in correction term μ e (n)  Θ _n of (5), is preconditioned by the inverse Φ^-1 of input signal’s autocorrelation matrix Φ. The modified update equation is: x n + 1 = x n - 2 μ e ( n ) Φ - 1 Θ n .(12) However, LMS-Newton is not suitable for practical purposes, because of the unavailability of exact autocorrelation matrix and its inverse. Nevertheless, a regularized LMS-Newton method can be designed by approximating correlation matrix Φ by Φ _n in (12),

x n + 1 = x n + 2 μ e ( n ) Φ n - 1 Θ n .(13) The normalized LMS (NLMS) algorithm is a regularized algorithm of this type [14]. The correction term of NLMS algorithm, at instant n + 1, is preconditioned by the spectral power of input vector Θ _n at instant n. Since Θ n Θ n T has rank one, it has at most one nonzero eigenvalue which is given by: ∥ Θ n ∥ 2 2 = Θ n T Θ n , which is infact spectral power of Θ _n . This power can be very close to zero for numerically small values of input signals, in which case the algorithm might get unstable. In order to avoid division by zero, it is good to choose ( ψ I + Θ n Θ n T ) - 1 as a regularized inverse of Θ n Θ n T , where ψ ≈ 0. The update equation of NLMS algorithm is given by:

x n + 1 = x n + μ e ( n ) ( ψ I + Θ n Θ n T ) - 1 Θ n .(14) NLMS has fast convergence speed, with preconditioned input vector u ¯ n = ( ψ I + Θ n Θ n T ) - 1 Θ n .

If data matrix A _n is ill-conditioned, then convergence of iterative methods to optimal Wiener solution is not fast enough. One reason is high eigenvalue spread of input signal’s autocorrelation matrix, and the solution is an efficient preconditioner. Consider the normal equation associated with the adaptive LS problem of the form (1): A n T A n x n = A n T b n .(15) Since Φ is symmetric and positive definite matrix, a Cholesky factorization Φ = C ^T C exists for an upper triangular matrix C, its approximation C _n will be a Cholesky factor of Φ n = A n T A n . Then C n - T can be used as a preconditioner for the normal equation (15) such that ( C n - T A n T A n C n - 1 ) C n x n = C n - T A n T b n , or

Φ ˆ n x ˆ n = p ˆ n ,(16) where Φ ˆ n = C n - T A n T A n C n - 1 = ( A n C n - 1 ) T ( A n C n - 1 ) , x ˆ n = C n x n , p ˆ n = C n - T A n T b n = ( A n C n - 1 ) T b n . The following observation, realized from [15], is sufficient to show the efficiency of a factorization preconditioner C n - T to reduced the eigenvalue spread of input signals’s correlation matrix.

Observation 3.1. For all n, eigenvalue spread of Φ ˆ n remains smaller than or equal to the eigenvalue spread of Φ _n .

Proof. Since Φ _n  = H ϒ H ^T , an EVD for preconditioned correlation matrix can be Φ ˆ n = H ˆ ϒ ˆ H ˆ T , where ϒ ˆ = Diag ( γ ˆ 1 , γ ˆ 2 , … , γ ˆ N ) is the diagonal matrix of eigenvalues of Φ ˆ n . With Φ ˆ n = C n - T Φ n C n - 1 , ϒ ˆ = H ˆ T Φ ˆ n H ˆ = H ˆ T C n - T Φ n C n - 1 H ˆ = ( H ˆ T C n - T H ) ϒ ( H ˆ T C n - T H ) T = G ϒ G T , where G = H ˆ T C n - T H . If c _jk is (j, k)th element of G, then jth eigenvalue of Φ ˆ n is γ ˆ j = ∑ k = 1 N c jk 2 γ k ; j = 1 , … , N . Furthermore, since eigenvalues of both Φ _n and Φ ˆ n are real, 0 ≤ γ min ∑ k = 1 N c jk 2 ≤ γ ˆ min ≤ γ ˆ max ≤ γ max ∑ k = 1 N c jk 2 . Hence γ ˆ max γ ˆ min ≤ γ max γ min .□

3.2 Formation of optimal preconditioner (OP)

To understand the concept of optimal preconditioner, a first order autoregressive (AR) input process θ (n) = a₁θ (n-1) + ν (n) of the type described in [16] is considered.

Here a₁ is a real number, and ν (n) a white noise process of zero mean. The autocorrelation matrix T _N of a stochastic input process can be formed by the autocorrelation function of AR process r _n  (j) = E [θ (n) θ (n-j)] for j = 0, 1, …, N-1, such that T N = ( r n ( 0 ) r n ( 1 ) ⋯ r n ( N - 1 ) r n ( 1 ) r n ( 0 ) ⋯ r n ( N - 2 ) ⋮ ⋱ ⋮ r n ( N - 1 ) r n ( N - 2 ) ⋯ r n ( 0 ) ) .(20) If variance of noise process ν (n) is one, then AR process θ (n) can be taken as a white noise passed from an all pass filter, whose transfer function is:

H ( z ) = 1 - α n 2 1 - α n z - 1 ,(21) where |α _n |  <  1, α _n is the pole of the filter, and controls the spectral power of the input signals. Since spectral power of input signals is measured by the eigenvalue spread of autocorrelation matrix, therefore α _n can be measured directly from the autocorrelation function r (j) as

r n ( j ) = E [ θ ( n ) θ ( n - j ) ] = c 1 α n j .(22) Here c₁ is a nonzero constant, and determines the power of input process. Once an appropriate value of correlation parameter α _n is computed from (22), for an incoming process θ (n), its autocorrelation matrix can be computed from (20) as, T N ( α n ) = ( 1 α n α n 2 ⋯ α n N - 1 α n 1 α n ⋯ α n N - 2 α n 2 α n 1 ⋮ ⋮ ⋱ ⋮ α n N - 1 α n N - 2 ⋯ 1 ) .(23) It is a symmetric toeplitz matrix having Cholesky factor C _N  (α _n ), and inverse Cholesky factor C _N  (α _n ) ^-1 as: C N ( α n ) = ( 1 α n α n 2 ... α n N - 1 0 1 - α n 2 α n 1 - α n 2 ... α N - 2 n 1 - α n 2 0 0 1 - α n 2 ⋱ α N - 3 n 1 - α n 2 ⋮ ⋮ ⋮ ⋱ ⋮ 0 0 0 ... 1 - α n 2 )

C N ( α n ) - 1 = ( 1 - α n 1 - α n 2 0 ... 0 0 1 1 - α n 2 - α n 1 - α n 2 ... 0 ⋮ ⋮ ⋱ ⋱ ⋮ 0 0 ... 1 1 - α n 2 - α n 1 - α n 2 0 0 ... 1 1 - α n 2 )(24) Clearly C _N  (α _n ) ^-1 is very sparse having nonzero entries in the main diagonal and upper band only. Furthermore it is independent of N, and depends upon α _n only.

Theorem 3.3. The inverse Cholesky factor C _N  (α _n ) ^-1 of T _N  (α _n ) converges asymptotically to C_∞ (α _n ) ^-1.

Proof. The pattern of C _N  (α _n ) ^-1 in (24) yields that for N→ ∞, C _N  (α _n ) ^-1 ≈ C_∞ (α _n ) ^-1.□

Observation 3.4. As n→ ∞, α _n  → α_∞, and C_∞ (α _n ) ^-1 yields an approximation C ∞ ( α ∞ ) - 1 = C ∞ - 1 of the Cholesky factor of input signal’s autocorrelation matrix.

As the value of α _n increases, the ability of C ∞ - T to decorrelate input signals increase, as is shown in Fig. 1. It is clear that the percentage of reduction in the eigenvalue spread increases with the increase in value of α _n , which in turn guarantees that the condition number of input autocorrelation matrix remains close to 1.

OPEN IN VIEWER

Fig. 1

Percentage of reduction in the eigenvalue spread of input autocorrelation matrix corresponding to parameter α _n .

This section contains application of optimal preconditioner C ∞ - T on least mean squares algorithm for online learning, that is further followed by a regularized step-size technique to develop an optimal preconditioned reqularized LMS (OPRLMS) algorithm.

Using C ∞ - T in Eq-(19) to transform input vector Θ _n in preconditioned input, u ˆ n = C ∞ - T Θ n . Similarly, (18) gives the preconditioned equation,

C ∞ x n + 1 = C ∞ x n + 2 μ e ( n ) C ∞ - T Θ n ,(26) with error signal e ( n ) = s ( n ) - ( C ∞ x n ) T ( C ∞ - T Θ n ) = s ( n ) - x n T Θ n .

Convergence Criteria:. Rewriting equation (26) as,

x n + 1 = x n + 2 μ e ( n ) C ∞ - 1 C ∞ - T Θ n = x n + 2 μ e ( n ) ( C ∞ T C ∞ ) - 1 Θ n ,(27) where ( C ∞ T C ∞ ) - 1 is an optimal inverse of T_∞ (α _n ) and is associated with correlation matrix Φ ˆ n such that Φ ˆ n = E [ u ˆ n u ˆ n T ] = C ∞ - T E [ Θ n Θ n T ] C ∞ - 1 ≈ C ∞ - T Φ n C ∞ - 1 . The following definition is straight froward.

Definition 4.1. The preconditioned misalignment is defined as the translation of the preconditioned tap-weight vector x ˆ n , with respect to the shifted origin x ˆ o , i.e., m ˆ n = x ˆ n - x ˆ o .

With x ˆ o = C ∞ x o as the preconditioned optimal weight vector, an obvious observation is

Observation 4.1. m ˆ n = x ˆ n - x ˆ o = C ∞ x n - C ∞ x o = C ∞ m n

Solving (27) for misalignment, and taking expectation E [ m ˆ n + 1 ] = ( I - 2 μ Φ ˆ n ) E [ m ˆ n ] = ( I - 2 μ C ∞ - T Φ n C ∞ - 1 ) E [ m ˆ n ]

It shows that E [ m ˆ n ] forms a geometric progression, whose convergence to 0 is subject to the spectral radius ∥ 2 μ C ∞ - T Φ n C ∞ - 1 - I ∥ < 1 .(28) As n→ ∞, C ∞ - T Φ n C ∞ - 1 → I , so that (28) yields 0 < μ < 1 .(29)

In order to verify analytical results, and examine the performance of proposed OPRLMS algorithm, simulations are performed for unknown system identification and noise cancelation from real ECG signals.

5.1 Unknown system identification

For this application an autoregressive process process of unit variance is passed through a coloring filter with frequency response given by (21). The correlation parameter 0 ≤ α _n  < 1 is a measures of the correlation characteristics of incoming signal at instant n,as is shown in Fig. 1. It is required to identify a system whose exact output s (n) is given by

s ( n ) = S n Θ n + △ s ( n ) ,(33) where S n ∈ R N , and △s (n) is an output noise with SNR ≈ 30dB. Performance measures is convergence in normalized misalignment 10 log 10 ( ∥ x n - x o ∥ 2 2 ∥ x o ∥ 2 2 ) , and convergence behavior of proposed OPRLMS algorithm is compared with NLMS and TDLMS algorithms.

Choice of μ _init . Initial estimate of step size plays an important role in parameter estimation for starting iterations. Setting ε₁ = 10^-4α _n , Fig. 2 shows the results of three simulations corresponding to μ _init  = 0.05, 0.1 & 0.15. It is clear from these results that value of μ _n decreases adaptively with an increase in time n. It is visible from Fig. 3 that larger the value of μ _init , higher the value of initial normalized misalignment. It is therefore convenient to choose a smaller value of μ _init to have better convergence in normalized misalignment. The results of Fig. 3 are obtained by an ensemble average of 500 independent runs.

OPEN IN VIEWER

Fig. 2

Adaptive change in μ _n with successive iterations for α = 0.6059931.

OPEN IN VIEWER

Fig. 3

Learning curve of normalized misalignment [dB] for OPRLMS algorithm with N = 6 and 0.5 < α < 0.75.

Comparison. Simulation results of this section contain comparison of OPRLMS algorithm with conventional LMS algorithm, NLMS algorithm and TDLMS algorithm. Filter length for this simulation is N = 6, and all the results are obtained by taking average of 500 independent runs.

The experimental results are obtained after performing 1200 iterations. The first results is obtained for normalized misalignment, and this comparison is shown in Fig. 4(a). A mean value of α _n  = α is used for this experiment and this value in the interval (0.5, 0.75) for all n.

Another observation that is made during this experiment is the computation of the mean squares error (MSE) whose learning curve is shown in Fig. 4(b). This observation show the comparative convergence behavior for first 200 iterations. It is evident that although MSE behavior have slight improvement with OPRLMS algorithm, but misalignment behavior shows clear preference of proposed algorithm over the rest.

OPEN IN VIEWER

Fig. 4

OPRLMS and conventional algorithms with N = 6 and 0.5 < α < 0.75.

5.2 Noise cancelation

This application presents the cancelation of powerline interference (PLI) from real ECG signals. ECG signal include valuable clinical information, but frequently this valuable information is corrupted by various kinds of noise presented in the recordings. For simulations real ECG signals are obtained from MIT-BIH database of physionet.com [17]. The MIT-BIH database contains 48 half hour waveforms of two channel ambulatory ECG recordings, which were obtained from 47 patients, including 25 men aged 32-89 years, and women aged 23-89 years. The recordings are digitized at 360 samples per second per channel with 11-bit resolution over a 10 mV range, and have different heartbeat frequencies depending upon the ages of the patients [18]. MIT-BIH signal ECG 101, of a female with age = 75 years, is used for performance study of proposed adaptive filtering algorithms. Performance of these algorithms is analyzed in terms of their efficiency to denoise ECG signal, and computation time. Since medical technicians are interested in accuracy of detecting events of ECG waveform, therefore a minimum norm (MN) analysis is provided to examine the efficiency of an adaptive algorithm to identify clean signal from the noisy one.

Cancelation of PLI Noise. PLI is an environmental noise that is often recorded along with the ECG signals and interrupt the exact diagnosis. However, it may be canceled by some efficient noise cancelation technique. Such noise is generated by changes in the alternating current (AC) in response to poor connection of recording machine. Owing to the AC characteristics, PLI is harmonic, and therefore if sampling frequency is 360Hz, PLI comes up with a frequency of 60Hz, 120Hz, 180Hz, etc. For experimental purposes, a PLI noise is simulated in Matlab, using formula PLI ( n ) = A PL sin ( 2 π f PL n ) ,(34) where A _PL is the amplitude, and f _PL is the frequency of PLI noise that may be higher than or equal to 40Hz. A 60Hz PLI noise is used to generate a harmonic PLI noise for simulation results. The harmonic nature of PLI noise make OP based preconditioned algorithm a suitable choice for PLI removal from real ECG.

OPEN IN VIEWER

   Fig.5

Clean and noise contaminated waveforms of ECG 101.

This simulation involves a comparative study of the performance of LMS, NLMS and OPRLMS algorithms in terms of their efficiency in noise cancelation. A common performance measure in signal processing is SNR, however, this tool is not effective in ECG signals. It is so because medical technician and scientists are generally interested in an accurate detection of clean ECG waveform out of noisy one in minimum possible time. For this reason the performance analysis of this section is done by computing the processing time taken by an algorithm in cancelation of PLI noise. Then Euclidean 2-norm of the difference between denoised ECG and clean ECG signal is computed, to observe the mean square deviation of the algorithm. Furthermore, denoised waveforms are recorded to observe the efficiency of a particular algorithm in cancelation of PLI noise from ECG. All simulations are performed for a filter length N = 5 and 10 seconds duration time, with n = 0 :0.002778 : 10.

The real ECG 101, obtained from MITBIH, is considered as the clean signal, and its waveform is shown in Fig. 5(a), while the PLI contaminated ECG 101 is shown in Fig. 5(b). The waveforms of denoised ECG signals, shown in Fig. 5a, are obtained by the application of LMS, NLMS and OPRLMS algorithms. An overall analysis of the results of Table 1, and denoised waveforms of Fig. 5a show preference of OPRLMS algorithm over the rest. Computational time for OPRLMS algorithm is slightly larger than LMS algorithm, however it is smaller than NLMS algorithm. On the other hand, value of 2-norm is least for OPRLMS algorithm. These results prove high efficiency of OPRLMS algorithm at nominal additional cost of computing optimal preconditioner and adaptive step-size. Time cost for these computations is far less than that of NLMS algorithms, while efficiency is much better than both LMS and NLMS algorithms

OPEN IN VIEWER

Fig. 6

PLI denoised ECG waveforms for N = 5.

OPEN IN VIEWER

Table 1

Processing time of each algorithm for noise cancelation, and norm of the deviation of denoised ECG from clean ECG 101.

Algorithm	Time (in sec.)	Norm of Deviation
LMS	0.06337	2.96603
NLMS	0.08771	2.79772
OPRLMS	0.07177	2.12722

In this paper, an insight into the need of preconditioning the regularization techniques in for online learning is highlighted. Setting a framework of factorization preconditioner for pre-whitening input data signals is shown to regularize the eigenvalue spread of input autocorrelation matrix by inverse Cholesky factor. Then representation of LMS-Newton algorithm as a factorization preconditioned algorithm developed the idea of using inverse Cholesky factor of correlation matrix to precondition LMS algorithm. Afterwards a stylized reformation of NLMS algorithm as a diagonally preconditioned algorithm and then as a step-preconditioned algorithms has lead to the importance of preconditioning. These realizations, and efficiency of optimal preconditioned regularization has produced an efficient OPRLMS algorithm with fast and unbiased convergence behavior. The optimal preconditioner, presented in this paper, is formed using optimizing the inverse Cholesky factor of inverse QRD-RLS algorithm for known statistics of input signals. The proposed regularization technique is developed at nominal computational cost, that is able to achieve a fast convergence rate for adaptive LMS filter without increasing too much in the complexity. Convergence analysis and experimental results have proved the preference of newly proposed OPRLMS algorithm over the existing preconditioned algorithms. These results show high efficiency of OPRLMS algorithm in the applications of unknown system identification and cancelation of PLI noise from real ECG signals.