Improved method for detecting weak abrupt information based on permutation entropy

Abstract

As a dynamic detecting method for abrupt information, permutation entropy could effectively reflect the subtle change in time series data, which is also simple and can be computed conveniently. Based on the permutation entropy, some improved methods for detecting weak abrupt information hidden in time series data are presented, such as permutation entropy spectrum, second permutation entropy, and second permutation entropy spectrum. Through some simulation examples, these new methods are compared with the existing single permutation entropy and approximate entropy, and the results show that these methods can more effectively detect the much weaker abrupt information. Especially, the second permutation entropy spectrum is very robust even if the periodic abrupt information is very weak.

Keywords

Permutation entropy permutation entropy spectrum second permutation entropy weak abrupt information condition monitoring second permutation entropy spectrum

Introduction

It is very important to detect the abrupt information in various signals because the abrupt points often contain abundant characteristic information. In mechanical system, the reason for signal abruption is possibly due to impact shock, speed fluctuation, structure deformation or fracture, and so on, which would generally lead to mechanical fault and affect the safety seriously. Thus, the study on detecting abrupt information is very important and meaningful.

There are many kinds of methods for detecting abrupt information, such as Fourier transform, wavelet analysis, singular value decomposition, higher-order cumulant, detrended fluctuation analysis, and approximate entropy.^1–17 Fourier transform could only determine the overall nature of the abrupt information, which was difficult to determine the exact time position of the abrupt point embedded in signal. Wavelet analysis could meet the requirements for detection of signals with multiple frequencies. But in the procedure of detecting abrupt point, the results would be affected by the selections of wavelet transform coefficients and wavelet basis functions. Singular value decomposition was just used to analyze the time domain signals, which had not used the information in frequency domain.⁸ He and colleagues^11–15 presented several efficient methods for detection of abrupt information, and some mathematical tools such as the higher-order cumulant, detrended fluctuation analysis, approximate entropy, conditional entropy, and permutations slopes were used in these methods. They^11–15 applied those methods to detect abrupt change in different dynamic structures and verified the feasibility and robustness by some different examples.

The permutation entropy (PE) was also useful tool in signal processing.¹⁸ Bandt and Pompe¹⁹ proposed PE to measure the complexity of one dimension time series and found that its performance was similar to Lyapunov exponent. Comparing with Lyapunov exponent, PE was more simple and anti-noising in the calculation. As a new detecting method for abrupt information of the complex system, PE had been widely used in many fields, such as medicine, biology, and climate.^20–25 In recent years, it had received more and more attention in the field of mechanical fault diagnosis^26–28 and used to detect the abrupt information from different mechanical devices.

In this article, some improved methods based on PE for detecting weak abrupt information are proposed, such as permutation entropy spectrum (PES), second permutation entropy (SPE), and second permutation entropy spectrum (SPES). This article is organized as follows. First, the properties of PE are presented and simply summarized. Second, the three presented methods are introduced, and the property of doubling frequency in PES is illustrated through an example of sinusoidal signal. Then, the simulation examples of detecting weak abrupt information in time series are studied, and these new methods are compared with the existing single PE and approximate entropy. The results demonstrate that these methods can more effectively detect the much weaker abrupt information hidden in regular signals. At last, the main conclusions are drawn.

Basic principle of PE

The basic principle of PE was described in detail in Bandt and Pompe¹⁹ and Cao et al.²⁹ To illustrate the basic idea, an arbitrarily scalar time series $[x (i), i = 1, 2, \dots, n]$ is introduced. According to the reconstruction theory, the time series can be reconstructed as

[\begin{matrix} x (1) & x (1 + τ) & \dots & x (1 + (m - 1) τ) \\ ⋮ & ⋮ & \dots & ⋮ \\ x (j) & x (j + τ) & \dots & x (j + (m - 1) τ) \\ ⋮ & ⋮ & \dots & ⋮ \\ x (n - (m - 1) τ) & x (n - (m - 2) τ) & \dots & x (n) \end{matrix}] j = 1, 2, \dots, n - (m - 1) τ

(1)

where m is the embedded dimension and $τ$ is the time delay. Each row of the reconstruction matrix can be considered as a reconstructive component, and $n - (m - 1) τ$ is the total number of reconstructive components. Those values contained in the jth reconstructive component can be rearranged in an ascending order as

x [j + (j_{1} - 1) τ] \leq x [j + (j_{2} - 1) τ] \leq \dots \leq x [j + (j_{m} - 1) τ]

(2)

where $j_{1}, j_{2}, \dots, j_{m}$ represent the serial number of each element in the jth reconstructive component.

If two or more elements in the jth reconstructive component have the same value, for example

x [j + (j_{1} - 1) τ] = x [j + (j_{2} - 1) τ]

(3)

they could be arranged according to the value of $j_{1}$ and $j_{2}$ . That is to say, if $j_{1} < j_{2}$ , then

x [j + (j_{1} - 1) τ] \leq x [j + (j_{2} - 1) τ]

(4)

Accordingly, each row of the reconstruction matrix can be mapped into a group of symbols as

S (l) = (j_{1}, j_{2}, \dots, j_{m})

(5)

where $l = 1, 2, \dots, k$ and $k \leq m!$ . Obviously, $m!$ is the largest number when those m elements are arranged or permuted, and $S (l)$ is one of the symbol permutations. Supposing the probabilities for all the symbols are $P_{1}$ , $P_{2}$ , …, $P_{k}$ , respectively, the PE for the time series $[x (i), i = 1, 2, \dots, n]$ can be defined as the Shannon entropy for the k symbol sequences

H_{P} (m) = - \sum_{l = 1}^{k} P_{l} \ln P_{l}

(6)

Obviously, if all the symbol sequences have the same probability as $P_{l} = 1 / m!$ , the PE $H_{P} (m)$ attains the maximum value $\ln (m!)$ . For convenience, the PE $H_{P} (m)$ is always normalized by $\ln (m!)$ , and denoted as

0 \leq {\bar{H}}_{P} = H_{P} (m) / \ln (m!) \leq 1

(7)

The value ${\bar{H}}_{P}$ can represent the complexity degree of the time series $[x (i), i = 1, 2, \dots, n]$ . The smaller the value of ${\bar{H}}_{P}$ , the more regular the time series. On the contrary, if the value of ${\bar{H}}_{P}$ is closer to 1, the time series is more complicated or random. The changes in the value of ${\bar{H}}_{P}$ can reflect and amplify the weak abrupt information in time series.

Improved methods for detecting weak abrupt information based on PE

When applying PE in detection of abrupt information hidden in periodic signal, the authors found it is difficult to detect the characteristics of the abrupt information by directly observing the change in PE if the abrupt information is weak. Accordingly, we put forward some improved methods to effectively detect the weak abrupt information hidden in periodic signal, which are named as PES, SPE, and SPES respectively.

PES

The PES is defined as the frequency spectrum of PE through Fourier transform. The specific calculation procedure is as follows.

Given the time series as $x (i)$ and selecting an appropriate length of the subsequence, one could divide the time series into n-subsequences. In the dividing procedure, one could select overlapping or non-overlapping. After calculating the PE of every subsequence and endowing the corresponding subsequence with the middle time, one can get a PE series defined as ${\bar{H}}_{p} (n)$ with the length as n. Then, applying the Fourier transform to the PE series

H (ω) = \sum_{n = 0}^{N - 1} \bar{H_{p}} (n) {W_{N}}^{n ω}, ω = 0, 1, 2, \dots, N - 1; W_{N} = e^{- j 2 π / N}

(8)

one could obtain the frequency spectrum called as PES.

SPE

In the study on PES, it may be difficult to detect some weak periodic abrupt signal. Hence, the method named as SPE is proposed. As the name implies, SPE is the PE of the obtained PE series.

Hence, the specific calculation procedure is very similar to the PE. Given the time series $x (i)$ , one can obtain the PE series ${\bar{H}}_{p} (n)$ based on the aforementioned procedure. Considering ${\bar{H}}_{p} (n)$ as a general time series, one can get the PE of ${\bar{H}}_{p} (n)$ , which is called as the SPE ${\bar{SH}}_{P}$ of $x (i)$ .

SPES

As for the very weak periodic abrupt information hidden in other signal, the aforementioned methods may be invalid. Accordingly, the SPES is further presented in this article.

The calculation procedure of SPES is also similar to the calculation of PES. After the PE series ${\bar{H}}_{p} (n)$ is obtained, ${\bar{H}}_{p} (n)$ is considered as a general time series. Similarly, dividing the time series into q-subsequences (overlapping or non-overlapping), one can get a SPE series defined as ${\bar{SH}}_{P} (q)$ with the length as q. Then, one could apply Fourier transform to ${\bar{SH}}_{P} (q)$ and establish the corresponding frequency spectrum of ${\bar{SH}}_{P} (q)$ . By the calculating formula as

\begin{array}{l} S H (ω) = \sum_{n = 0}^{N - 1} \bar{S H_{p}} (q) W_{N}^{n ω}, \\ ω = 0, 1, 2, …, N - 1; W_{N} = e^{- j 2 π / N} \end{array}

(9)

one could obtain the SPES.

Summarization on the three new methods

Obviously, the aforementioned three new methods are intuitively provided based on the properties of PE. Maybe it is difficult to establish rigorous mathematical theory for those methods. Accordingly, different numerical examples and comparison with other methods are presented in the next section, which could be used to illustrate the effectiveness of those new methods.

Simulation analysis of the new methods

When calculating the PE, it is important to choose the embedding dimension m, time delay $τ$ , and the length of the subsequence. Based on Takens’ theorem for reconstruction theory,³⁰ one could easily obtain the embedding dimension. The time delay could be determined based on the autocorrelation method or mutual information method.³⁰ As for choosing the length of the subsequence, some important statistical significance will be lost if it is too short. In order to accurately reflect the abrupt situation of the signal, it should not be too long. In all the presented examples, the maximal overlapping is used. That is to say, each subsequence move backward a data point to get the next subsequence.

Periodic pulse hidden in the sinusoidal signal

Choosing the simulation signal as $x = \sin (2 π t)$ which is coupled with periodic pulse, where the amplitude and frequency of pulse are 0.008 and 25 Hz, respectively, the sampling frequency is 500 Hz and the sampling time is 10 s. From Takens’ theorem and autocorrelation method, one could select embedding dimension $m = 5$ and time delay $τ = 1$ . After some trial computation, the length of subsequence is selected as 1.5 times of basic periodic. The results for pure sinusoid signal and sinusoid signal with periodic pulse are illustrated in Figures 1 and 2, respectively. In the two figures, Figures 1(a) and 2(a) show the time history of the two signals, and the frequency spectra for the two simulation signals are shown in Figures 1(b) and 2(b). From the figures, it could be found that the frequency of the sinusoidal signal can be observed obviously, whereas the frequency of the periodic pulse signal can hardly be seen. The PEs of the simulation signals from 1 to 8 s are given in Figures 1(c) and 2(c). Figures 1(d) and 2(d) present the PESs of the simulation signals from 1 to 8 s, and it is easy to find the frequency of the periodic pulse signal. Moreover, it should be emphasized that the characteristic frequencies will be doubled in PESs as shown in Figures 1(d) and 2(d). The reason lies in there are two abrupt changes in one periodic of the sinusoidal signal. It could also be found that there are some weak higher-order doubling frequencies in the spectrum.

Figure 1.

Simulation results of the pure sinusoidal signal: (a) time history of the simulation signal, (b) frequency spectrum of the simulation signal, (c) PE of the simulation signal, and (d) PES of the simulation signal.

Figure 2.

Simulation results of periodic pulse hidden in the sinusoidal signal: (a) time history of the simulation signal, (b) frequency spectrum of the simulation signal, (c) PE of the simulation signal, and (d) PES of the simulation signal.

Moreover, approximation entropy is selected as a tool compared with the method in this article. The approximation entropy and the corresponding spectrum of the above signal are shown in Figure 3. It could be found that the characteristic frequency f = 25 Hz almost cannot be seen from the spectrum, and only the frequency for sinusoid signal and its doubling frequency could be distinctly observed. It illustrates the better performance of the methods in this article.

Figure 3.

Results of periodic pulse hidden in the sinusoidal signal by approximation entropy: (a) AE of the simulation signal and (b) AES of the simulation signal.

Modulated signal

In this subsection, the simulation signal $x = \sin (120 π t) [1 + β \sin (26 π t)]$ is chosen, where the modulation index is $β = 0.008$ , the sampling frequency is 2000 Hz, and the sampling time is 1 s. Figure 4(a) shows the time history of the simulation signal, and the frequency spectrum of the simulation signal is shown in Figure 4(b). It can be seen that the dominant frequency is evident, but the modulation frequency can hardly be observed. Figure 4(c) presents the PE of the simulation signal from 0.1 to 0.9 s. The PES of the simulation signal from 0.1 to 0.9 s is given in Figure 4(d). Obviously, the modulation frequency is hardly observed yet. Figure 4(e) shows the SPE of the simulation signal from 0.1 to 0.9 s, and the SPES of the simulation signal from 0.1 to 0.9 s is shown in Figure 4(f). It is easy to find the doubled characteristic frequency 120 and 26 Hz in SPES as shown in Figure 4(f). And it is interesting to find that the fractional frequency of the dominant frequency and the modulation frequency in the spectrum. This may be generated from the sampling procedure.

Figure 4.

Simulation results of modulation signal (modulation index β = 0.008): (a) time history of the simulation signal with modulation, (b) frequency spectrum of the simulation signal, (c) PE of the simulation signal, (d) PES of the simulation signal, (e) SPE of the simulation signal, and (f) SPES of the simulation signal.

In the studying procedure, it is also found that the main features could still be observed by the SPES even if the modulation index is very small. The simulation results are as shown in Figure 5, where the modulation index is β = 0.000001, the sampling frequency is 2000 Hz, and the sampling time is 1 s. It can be seen that Figure 5 is similar to Figure 4. The time history of the simulation signal is presented in Figure 5(a), and the frequency spectrum of the simulation signal is given by Figure 5(b). The dominant frequency can be found obviously from this figure, but the modulation frequency can hardly be seen. Figure 5(c) shows the PE of the simulation signal from 0.1 to 0.9 s, and the PES of the simulation signal from 0.1 to 0.9 s is shown in Figure 5(d). Obviously, the modulation frequency can hardly be seen. Figure 5(e) shows the SPE of the simulation signal from 0.1 to 0.9 s, and the SPES of the simulation signal is shown in Figure 5(f). It is easy to find the characteristic frequency of the dominant frequency and modulation frequency in SPES. That means the SPES is robust even if the abrupt information is very weak compared with the regular signal.

Figure 5.

Simulation result of modulation signal (modulation index β = 0.000001): (a) time history of the simulation signal, (b) frequency spectrum of the simulation signal, (c) PE of the simulation signal, (d) PES of the simulation signal, (e) SPE of the simulation signal, and (f) SPES of the simulation signal.

From the above simulation results, it can be found that the frequency component of sinusoidal signal is obvious in the frequency spectrum, but the frequency component of the periodic pulse signal can hardly be observed. While they are presented in PES, the two frequency components of the sinusoidal and periodic pulse signals could be observed simultaneously. In the simulation results of the modulation signal, it is very difficult to find the modulated frequency component directly from the frequency spectrum and the PES. But the doubling frequency for the modulated component can be found clearly in SPES, and it is interesting to find the fractional frequency of the dominant frequency and modulation frequency. Especially, when the modulation index of modulation signal is reduced to a very small level, the expected frequency components could be still easy to be found in SPES.

Conclusion

Based on PE, some new methods for detecting weak abrupt information in time series data are proposed, such as PES, SPE, and SPES. Through some simulation examples, it is illustrated that these new methods can more effectively detect weak abrupt information. Especially, the SPES is very robust even if the periodic abrupt information is very weak.

Footnotes

Academic Editor: James Lam

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors are grateful to the support by National Natural Science Foundation of China (No. 11372198), the Cultivation plan for Innovation team and leading talent in Colleges and universities of Hebei Province (LJRC018), the Program for advanced talent in the universities of Hebei Province (GCC2014053), and the Program for advanced talent in Hebei Province (A201401001).

References

Satish

Short-time Fourier and wavelet transforms for fault detection in power transformers during impulse tests. IEE Proc A 1998; 145: 77–84.

Mohanty

Kar

Vibration and current transient monitoring for gearbox fault detection using multiresolution Fourier transform. J Sound Vib 2008; 311: 109–132.

Xie

ZJ.

Singularity detection of wavelet transform and its application in mechanical failure diagnosis. Mech Eng 2007; 6: 63–65.

Sun

Tang

Singularity analysis using continuous wavelet transform for bearing fault diagnosis. Mech Syst Signal Pr 2002; 16: 1025–1041.

Shark

. Weak transient signal detection based on continuous wavelet and cumulant analysis. In: Proceedings of the 6th international conference on signal processing, Beijing, China, 26–30 August 2002, vol. 2, pp.1851–1854. New York: IEEE.

Mallat

Hwang

WL.

Singularity detection and processing with wavelets. IEEE T Inform Theory 1992; 38: 617–643.

Yang

Tse

PT.

Development of an advanced noise reduction method for vibration analysis based on singular value decomposition. NDT&E Int 2003; 36: 419–432.

Liu

Yang

Shen

YJ.

New method of detecting abrupt information based on singularity value decomposition and its application. Chin J Mech Eng 2002; 38: 102–105.

Hou

Chen

JF.

An analytical method for Mathieu oscillator based on method of variation of parameter. Commun Nonlinear Sci 2016; 37: 326–353.

10.

Hou

JY.

Bursting phenomenon in a piecewise mechanical system with parameter perturbation in stiffness. Int J Nonlin Mech 2016; 81: 165–176.

11.

Feng

. A new method for abrupt change detection in dynamic structures. Nonlinear Progress Geophys 2008; 15: 601–606.

12.

Wan

Jiang

. Detecting abrupt change on the basis of skewness: numerical tests and applications. Int J Climatol 2013; 33: 2713–2727.

13.

Feng

. A new method for abrupt dynamic change detection of correlated time series. Int J Climatol 2012; 32: 1604–1614.

14.

Jin

Liu

. The applicability of research on moving cut data-approximate entropy on abrupt climate change detection. Theor Appl Climatol 2016; 124: 475–486.

15.

Jin

Hou

. Effects of different trends on moving cut data-approximate entropy. Acta Phys Sin 2012; 61: 069201.

16.

Unakafova

Kellera

Conditional entropy of ordinal patterns. Physica D 2014; 269: 94–102.

17.

Armand

Koepf

Detecting regular dynamics from time series using permutations slopes. Commun Nonlinear Sci 2015; 27: 216–227.

18.

Feng

Rao

. Application and development of permutation entropy algorithm. J Acad Armor Force Eng 2012; 26: 34–38.

19.

Bandt

Pompe

Permutation entropy: a natural complexity measure for time series. Phys Rev Lett 2002; 17: 21–24.

20.

Nicolaou

Georgiou

Detection of epileptic electroencephalogram based on permutation entropy and support vector machines. Expert Syst Appl 2012; 39: 202–209.

21.

Frank

Pompe

Schneider

. Permutation entropy improves fetal behavioural state classification based on heart rate analysis from biomagnetic recordings in near term fetuses. Med Biol Eng Comput 2006; 44: 179–187.

22.

Nadia

Aime

. Analysis of absence seizure EEG via permutation entropy spatio-temporal clustering. In: Proceedings of the international joint conference on neural networks, San Jose, CA, 31 July–5 August 2011, pp.1417–1422. New York: IEEE.

23.

Liu

YD.

Permutation entropy applied to movement behaviors of

Drosophila melanogaster. Mod Phys Lett B 2011; 25: 1133–1175.

24.

Sun

XL.

The complexity of gene expression dynamics revealed by permutation entropy. BMC Bioinformatics 2010; 11: 607–621.

25.

Hou

Feng

Dong

WJ.

A technique for distinguishing dynamical species in the temperature time series of north China. Acta Physica Sinica 2006; 55: 2663–2668.

26.

Yan

Liu

Gao

RX.

Permutation entropy a nonlinear statistical measure for status characterization of rotary machines. Mech Syst Signal Pr 2012; 29: 474–484.

27.

Feng

Rao

. Research and application of the arithmetic of PE in testing the sudden change of vibration signal. J Vib Eng 2012; 25: 221–224.

28.

Ouyang

Liang

ZH.

Complexity measure of motor current signals for tool flute breakage detection in end milling. Int J Mach Tool Manu 2008; 48: 371–379.

29.

Cao

Tung

Cao

. Detecting dynamical changes in time series using the permutation entropy. Phys Rev 2004; 70: 1–7.

30.

Kantz

Schreiber

Nonlinear time series analysis. Cambridge: Cambridge University Press, 2000.