Synchrosqueezing with short-time fourier transform method for trinary frequency shift keying encoded SSVEP

Abstract

BACKGROUND:

The frequencies that can evoke strong steady state visual evoked potentials (SSVEP) are limited, which leads to brain-computer interface (BCI) instruction limitation in the current SSVEP-BCI. To solve this problem, the visual stimulus signal modulated by trinary frequency shift keying was introduced.

OBJECTIVE:

The main purpose of this paper is to find a more reliable recognition algorithm for SSVEP-BCI based on trinary frequency shift keying modulated stimuli.

METHODS:

First, the signal modulated by trinary frequency shift keying is simulated by MATLAB. At different noise levels, the empirical mode decomposition, singular value decomposition, and synchrosqueezing with the short-time Fourier transform are used to extract the characteristic frequency and reconstruct the signal. Then, the coherent method is used to demodulate the reconstructed signal. Second, in the paradigm of BCI using trinary frequency shift keying modulated stimuli, the three methods mentioned above are used to reconstruct EEG signals, and canonical correlation analysis and coherent demodulation are used to recognize the BCI instructions.

RESULTS:

For simulated signals, it is found that synchrosqueezing with short-time Fourier transform has a better effect on extracting the characteristic frequencies. For the EEG signal, it is found that the method combining synchrosqueezing with short-time Fourier transform and coherent demodulation has a higher accuracy and information translate rate than other methods.

CONCLUSION:

The method combining synchrosqueezing with short-time Fourier transform and coherent demodulation proposed in this paper can be applied in the SSVEP system based on trinary frequency shift keying modulated stimuli.

Keywords

Steady state visual evoked potentials (SSVEP)trinary frequency shift keying (TFSK)synchrosqueezing with the short-time Fourier transform (SSTFT)canonical correlation analysis (CCA)coherent demodulation

1. Introduction

According to different electrophysiological signals detected, the current brain-computer interface (BCI) systems based on EEG can be divided into the following three categories: steady state visual evoked potentials (SSVEP) 13 [1], motor imagination potentials (MI) [2] and event-related potentials (ERP) [3]. Among them, SSVEP is a periodic signal. This refers to a continuous response of the visual cortex when a person is stimulated by a fixed frequency signal. SSVEP has time-locked and phase-locked characteristics, which means that the phase and the frequency of the stimulus signal are similar to SSVEP [4]. When the phase of the stimulus signal changes, the phase of SSVEP will also change. Therefore, compared to other EEG evoked potentials, it has better stability. Compared to other potentials, experimental paradigms based on SSVEP have the advantage of requiring fewer electrodes and subjects without training [5]. Therefore, SSVEP has been widely used in the BCI system.

A basic SSVEP-BCI system is mainly composed of three parts: the generator of the visual stimulus signal, acquisition device of EEG signals and recognition algorithms. Frequency modulation (FM) is the main modulation mode of the visual stimulus signal [6], in which each stimulus signal has a fixed single frequency. Multiple different BCI instructions correspond to multiple visual stimulus signals with different frequencies [7]. The implementation of BCI instructions is realized by recognizing the characteristic frequencies of SSVEP. FM modulation is simple, and the SSVEP-BCI system based on FM modulation (FM-SSVEP system) has a simple experimental design and high recognition accuracy. However, the effective frequency of the visual stimulus signal that can stimulate the SSVEP response ranges between 6 Hz and 30 Hz. Although in recent years some researchers have found that frequency bands lower than 3 Hz [8] and higher than 80 Hz [9] can also stimulate SSVEP response. But, for some complex BCI systems, the available frequency band is still too narrow. The traditional frequency modulation greatly limits the number of BCI instructions and thus affects the performance of the SSVEP-BCI system.

In order to increase BCI instructions, frequency shift keying (FSK) is introduced [10]. FSK is a common digital modulation method in the field of communication. Based on the time-locked of SSVEP, FSK can also be used in the modulation of the visual stimulus signal. It becomes a stimulus by using a binary coding strategy, which means that a single visual stimulus signal can be composed of two frequencies. For the SSVEP-BCI system based on FSK modulation, FSK signals modulated by different digital baseband information can be assigned to different BCI instructions. Therefore, compared with FM, the number of BCI instructions has increased greatly.

When coding a stimulus of FSK method, the problem of ‘coding repetition’ will appear. For example, when the stimulus is coded as ‘10’, the baseband signal is constructed by the cycles of ‘10’, that is ‘10101010 …’. But when the binary coding strategy is used to encode the stimulus, confusion between the end of one stimulus and the next stimulus arises, and the output may be misreaded as ‘01010101 …’. To solve this problem, we introduced the symbol ‘2’, which encoded the flag by third frequency to judge the start and end of the stimulus cycle during the experiment [11]. This coding strategy is called trinary frequency shift keying (TFSK). Based on this improvement, the SSVEP-BCI system based on TFSK (TFSK-SSVEP system) has a higher recognition rate than the FSK-SSVEP system.

At the same time, compared to other modulation methods, it has the advantage of easy implementation, strong anti-attenuation ability, and anti-noise ability. In terms of the recognition methods, power spectral density analysis (PSDA) [12] and canonical correlation analysis (CCA) [13] can be applied in the TFSK-SSVEP system. CCA is used to recognize BCI instructions by calculating the correlation coefficient between multichannel EEG signals and the stimulus signal. At the same time, traditional demodulation methods in the field of communication can also be applied to the TFSK-SSVEP system. The common demodulation methods are coherent demodulation and incoherent demodulation. Compared to incoherent demodulation, coherent demodulation has a complex implementation but a good demodulation effect. Coherent demodulation is realized by multiplying carrier information and input signal to demodulate the baseband signal. Therefore, using visual stimulus signal modulated by TFSK is reasonable in the SSVEP-BCI system.

EEG is characterized by its nonlinearity, weakness, and complexity. Under the real environment with various kinds of noise, it is difficult to recognize the characteristic frequencies of SSVEP by using CCA or PSDA in the TFSK-SSVEP system. Therefore, it is necessary to extract characteristic frequencies of SSVEP by denoising the raw EEG signal. This paper proposes a method based on synchrosqueezing with short-time Fourier transform (SSTFT) to extract the characteristic frequencies of SSVEP. This is a time-frequency processing algorithm proposed by Thakur et al. in 2011 [14]. Short-time Fourier transform (STFT) provides a tool to analyze the signal time-frequency [15]. The length of the window determines the accuracy of the time-frequency spectrum [16]. For example, the longer the length of the window, the better the frequency resolution and the worse the time resolution. Compared to STFT, SSTFT can be used to reconstruct the signal by extracting the instantaneous frequency information of the time-frequency spectrum and reallocate the time-frequency energy in the frequency direction, which makes the time-frequency curve clearer and increases the accuracy of the time-frequency localization [17].

Based on the above research, this paper uses TFSK to modulate the visual stimulus signal for increasing the number of BCI instructions. Then, the characteristic frequencies of SSVEP are extracted by using SSTFT. Finally, coherent demodulation and CCA are used to recognize BCI instructions in the TFSK-SSVEP system.

2. Materials and methods

2.1 The modulation method of TFSK

The modulation method of TFSK is shown in Fig. 1. First, the number of symbols corresponding to carrier frequencies needs to be determined, which will form the baseband signal. Here, the baseband signal is ‘102102102 …’. Symbol ‘0’ corresponds to frequency f1, symbol ‘1’ corresponds to frequency f2, and symbol ‘2’ corresponds to frequency f3. Three independent oscillators generate three carrier signals of different frequencies. Next, three different carrier signals are multiplied by baseband signals including their encoded parts to obtain their encoded modulation parts respectively. Finally, the signal $S_{\textit{fsk}}(t)$ modulated by TFSK can be obtained by adding three encoded modulation parts.

Figure 1.

The modulation method of TFSK.

2.2 The feature extraction methods

Empirical mode decomposition (EMD) and singular value decomposition (SVD) are two traditional methods for signal reconstruction, and are widely used in feature extraction of EEG signals. The concept of EMD is that a complex signal can be decomposed into the infinite intrinsic mode functions (IMFs) by processing the mean curve of the signal. In these IMFs, local characteristic signals under different time scales of the original are included. The components of SSVEP can be extracted by reserving the IMFs related to stimulation frequencies [18]. Its advantage is that there is no need to set the basis function, and it is suitable for the analysis of nonlinear signals. The disadvantage is that the mode aliasing of IMFs may cause a bad feature extraction effect [19].

SVD has a better feature extraction effect for periodic signals, such as EEG signals.It can be realized by constructing an EEG signal into matrix A and calculating eigenvalues of $\Lambda$ by SVD of A: $A=U\Lambda V^{T}$ . Then the reconstructed signal is obtained by keeping the first k large eigenvalues and their corresponding vectors. Because the first k larger eigenvalues are considered to contain the components of the characteristic frequencies of SSVEP [20]. In this paper, we define k $=$ the numbers of stimulation frequencies*3, because, through spectrum analysis, k has a better extraction effect under this value. So k is set to 9. However, when the signal-to-noise ratio is low, the distribution of singular values tends to be a straight line, and it is difficult to extract the signal feature. In addition, the processing time of SVD is longer than EMD.

We propose SSTFT as a feature extraction method to avoid the problems that the above methods present. The traditional time-frequency spectrum obtained by short-time Fourier transform (STFT) is fuzzy, because of the divergence of energy along the ridge direction. Based on the theory that the phase of the signal is not affected by the length of the window, SSTFT is a good reconstruction method for the EEG signal. Through calculating the instantaneous frequency to redistribute the energy, time-frequency readability can be greatly improved. Compared to EMD and SVD, it has the advantage of better feature extraction effect and eliminating the interference of modal aliasing [21].

Hence, it is effective to use SSTFT for extracting the characteristic frequencies of SSVEP and improving the SNR of the EEG signal. The steps to reconstruct the EEG signal by SSTFT are outlined below:

First, the EEG signal is processed by STFT:

$\displaystyle G_{s}(\omega,t)=\int s(u)g(u-t)e^{-j\omega(u-t)}du=\frac{1}{2\pi% }\int_{0}^{\infty}e^{j\xi t}\hat{s}(\xi)\hat{g}(\omega-\xi)d\xi$ (1)

$g(u)$ is the window function, and $\hat{g}(\xi)$ is the FFT of $g(u)$ .

According to the phase $\phi_{i}(t)$ , the instantaneous frequency can be expressed as:

$\displaystyle f_{i}(t)=\frac{1}{2\pi}\frac{d\phi_{i}(t)}{dt}$ (2)

According to Eqs (1) and (2), the instantaneous frequency $f_{G}(\omega,t)$ in the time-frequency spectrum can be deduced as:

$\displaystyle f_{G}(\omega,t)=\frac{\partial}{\partial t}\text{arg}[G_{s}(% \omega,t)]=\text{Im}\left[G_{s}^{-1}(\omega,t)\frac{\partial G_{s}(\omega,t)}{% \partial t}\right]$ (3) $\displaystyle\frac{\partial G_{s}(\omega,t)}{\partial t}=\int{s(u)\frac{% \partial g(u-t)}{\partial t}e^{-j\omega(u-t)}du}$ (4)

At the same time, a threshold $r$ ( $r>$ 0) is introduced to avoid calculation error when $|G_{s}(\omega,t)|\approx 0$ :

$\displaystyle f_{G}(\omega,t)=\left\{{{\begin{array}[]{ll}f_{G}(\omega,t),&% \left|{{G}_{s}(\omega,t)}\right|>r\\ \infty,&\left|{{G}_{s}(\omega,t)}\right|\leqslant r\\ \end{array}}}\right.$ (5)

Through Eqs (3)–(5), $(t,\omega)$ is mapped to $(t,f_{G}(t,\omega))$ , which means synchro-squeezing. Hence, more precise time-frequency distribution can be obtained.

The next step is to reconstruct the signal. Time-frequency rearrangement can be expressed as:

$\displaystyle f_{s}(\omega,t)=\frac{1}{\pi g(0)}\int\delta(\omega-f_{G}(\tilde% {\omega},t))G_{s}(\omega,t)d\tilde{\omega}$ (6)

$\delta(t)$ is the unit pulse function.

Finally, the following time-frequency reconstruction method is used to extract the characteristic frequencies of SSVEP for obtaining the reconstructed signal:

$\displaystyle C_{r}=\sum\int_{\omega_{i}-0.5\Delta\omega}^{\omega_{i}+0.5% \Delta\omega}f_{s}(\omega,t)d\omega$ (7)

where $C_{r}$ is the reconstructed signal, $\omega_{i}$ is the $i^{\text{th}}$ characteristic frequency of SSVEP, and $\Delta\omega$ is the extract range of frequency.

2.3 Coherent demodulation based on TFSK-SSVEP

The coherent demodulation is also called synchronous detection because it requires the frequency and phase of the reference signal to be the same as that of the modulated signal [22]. But in the TFSK-SSVEP system, the phase information of the stimulus signal is unknown at the beginning when the visual cortex receives the stimulus signal. It is not easy to construct the reference signal. Therefore, there is a certain phase difference between the modulated signal and the reference signal for coherent demodulation. In order to eliminate the influence of the phase difference, the following processing method is adopted.

Suppose that the phase difference between the reference signal and the modulated signal is $\phi$ , A is the baseband signal and $\cos(\omega t)$ is the carrier frequency signal. The modulated signal $s(t)$ can be obtained as follows:

$\displaystyle s(t)=A\cos(\omega t+\varphi)$ (8)

The cosine and sine components of the reference signal are:

$\displaystyle v_{1}(t)=\cos\omega t,v_{2}(t)=\sin\omega t$ (9)

The sine and cosine components of the reference signal are multiplied by the modulated signal respectively to obtain following formula:

$\displaystyle h_{1}(t)=s(t)v_{1}(t)=A\frac{\cos(2\omega t+\theta)+\cos\theta}{2}$ (10) $\displaystyle h_{2}(t)=s(t)v_{2}(t)=A\frac{\sin(2\omega t+\theta)+\sin\theta}{2}$

Next, $h_{1}(t)$ and $h_{2}(t)$ are filtered by a low-pass filter. After filtering out the high-frequency components, $h_{1}(t)$ and $h_{2}(t)$ become $\frac{A\sin\theta}{2}$ and $\frac{A\cos\theta}{2}$ respectively. Finally, the filtered $h_{1}(t)$ and $h_{2}(t)$ is substituted into the following formula:

$\displaystyle\sqrt{\textit{LPFh}_{1}(t)^{2}+\textit{LPFh}_{2}(t)^{2}}=\sqrt{% \frac{A^{2}(\sin\theta^{2}+\cos\theta^{2})}{4}}=\frac{A}{2}$ (11)

According to Eq. (11), the baseband signal can be demodulated and the influence of phase difference on coherent demodulation can be eliminated. Similarly, in the TFSK-SSVEP system, the coherent demodulation steps are shown in Fig. 2.

Figure 2.

The steps of coherent demodulation in the TFSK-SSVEP system.

2.4 Materials

2.4.1 The simulated signal

As can be seen in Fig. 1, the simulated signal modulated by TFSK can be obtained. Here, the sampling rate was set to 250 Hz, the length of each symbol was set to 1.2 s, and symbols ‘0’, ‘1’, and ‘2’ correspond to 7 Hz, 11 Hz, and 15 Hz respectively. The baseband signal consists of 7 cycles of ‘012’. Hence, the symbol number of the baseband signal was 21, and the number of sampling points was 21 $\times$ 1.2 $\times$ 250 $=$ 6300. By setting the above parameters, we can obtain the simulation signal by MATLAB.

2.4.2 The EEG signal

We chose LED as a visual stimulator, because, compared with CRT or LCD, visual stimulation produced by LED has stronger brightness and can evoke stronger SSVEP [23]. Here, the symbols ‘0’, ‘1’, and ‘2’ of the TFSK signal correspond to 7 Hz, 11 Hz, and 15 Hz respectively, and the length of each symbol was also set to 1.2 s. As shown in Table 1, the BCI instructions consist of these symbols. Then, the TFSK signal was used to control the flickering frequency of the LED.

Five volunteers, aged 22–28 years including two females and three males, participated in our experiment. They were informed about the experimental procedures and written consent was provided prior to participation. Subjects with no visual impairment were required to stare at the LED. During this period, the EEG signals were recorded by SynAmps2 of Neuroscan at 1000 Hz with a 0.15 Hz to 200 Hz bandpass filter, and O1, O2, Oz were selected as electrodes. For each subject, the experiment consisted of five blocks. Each block contained six trials corresponding to the six BCI instructions indicated in a random order, and each trial lasted for 15 s. At the end of the trial, a 5 s rest time was included before the next trial.

In order to facilitate the processing, the EEG signals were down-sampled to 250 Hz by MATLAB. Based on the above operation, 10 groups of EEG signals for each instruction were recorded.

Table 1
The BCI instructions

Instructions	Symbol sequence
#1	0012
#2	0102
#3	0112
#4	1002
#5	1012
#6	1102

Figure 3.

The simulated signal and its spectrum.

3. Results

3.1 The reconstruction performance of SSTFT in simulation

In this section, we compare the performance of SVD, EMD and SSTFT using the simulated signals. The simulated signal obtained in Section 2 and its spectrum is shown in Fig. 3. Then, different intensity white Gaussian noises are added to the simulated signal to obtain the noisy signals with different SNR ranging from 0 dB to $-$ 17dB. Finally, for the noisy signal, we use three methods to extract characteristic frequencies to reconstruct the signal. The SNR and root mean square error (RMSE) of the reconstructed signal are calculated respectively.

Figure 4.

SNR comparison of the reconstructed signal.

Figure 5.

RMSE comparison of the reconstructed signal.

The SNR reflects the signal-to-noise ratio. The higher the SNR, the less the components of noise are in the reconstructed signal. The RMSE reflects the deviation between the predicted value and the real value. For the signal, the lower the RMSE, the closer the reconstructed signal is to the original signal. As shown in Figs 4 and 5, the SNR of the reconstructed signal by SSTFT is higher than that of the reconstructed signal by EMD and SVD. At the same time, the RMSE of the reconstructed signal by SSTFT is lower than that of the reconstructed signal by EMD and SVD. Therefore, SSTFT is better than SVD and EMD in signal reconstruction. Finally, we use the coherent method to demodulate the reconstructed signal and calculate the bit error rate (BER). The results are shown in Fig. 6.

Figure 6.

BER comparison of the reconstructed signal.

As shown in Fig. 6, the curve of BER based on SSTFT is lower than that of BER based on EMD and SVD. Hence, SSTFT with better characteristic frequencies extraction has lower BER for the coherent demodulation. At the same time, if the SNR of the original signal is lower than $-$ 14dB, we can see that the curve of BER of the reconstructed signal sharply increases. Therefore, when there is too much noise in the simulated signal, the ability of SSTFT to reconstruct the signal will be restricted.

3.2 Application and effect of SSTFT in TFSK-SSVEP system

Figure 7.

Flowchart of the TFSK-SSVEP system.

The flowchart of the TFSK-SSVEP system is shown in Fig. 7. Three methods are used to extract the characteristic frequencies of SSVEP to obtain the reconstructed signal. The same reconstructed signal by three methods and its frequency spectrum are shown in Figs 8–10 respectively.

Figure 8.

Reconstruction of EEG based on EMD.

Figure 9.

Reconstruction of EEG based on SVD.

Figure 10.

Reconstruction of EEG based on SSTFT.

We then calculate the average SNR of the characteristic frequencies of the Oz channel. The formula is as follows:

$\displaystyle\text{SNR}(f_{n})=\frac{p(f_{n})}{\frac{1}{m}\sum_{r=\frac{m}{2}}% ^{\frac{m}{2}}p(f_{n+r})}$ (12) $\displaystyle\text{SNR}(\bar{f})=\frac{1}{n+1}\sum_{n=0}^{2}p(f_{n})$ (13)

$f_{n}$ is the characteristic frequency, $p(f_{n})$ is the power of the signal, and $m$ is the number of frequency points around $f_{n}$ . Here, $m$ is set to 6, and the interval between adjacent frequency points is 0.4 Hz. Then, according to Eq. (12), the results of the average SNR by three methods are calculated, which are shown in Fig. 11.

Figure 11.

The average SNR of characteristic frequencies by three methods.

From Figs 8–10 it can be seen that SSTFT can extract the characteristic frequencies of SSVEP accurately. There may be some defects in EMD and SVD. For the SVD, it is realized by using the size of eigenvalues to determine whether it has the components mixed with characteristic frequencies of SSVEP. The larger eigenvalues may also be mixed with noise components. For EMD, because there may exist modal aliasing between IMFs, the noise will be introduced into signal reconstruction by using IMFs related to characteristic frequencies of SSVEP. Hence their spectrum contains some noise frequencies. It can also be seen that the average SNR of characteristic frequency extracted by SSTFT is higher than the other two methods displayed in Fig. 11. Therefore, it can be concluded that SSTFT is better than SVD and EMD in the extraction of the characteristic frequencies.

Finally, we use CCA and coherent demodulation to identify the reconstructed EEG signals by three methods respectively. The steps are as follows:

Step 1:

The reconstructed signal is identified by sliding window L1, whose the window length is 0.5 ${}^{*}$ , the length of the unit symbol and the sliding step length of 1.

Step 2:

Coherent demodulation or CCA is used to obtain the recognition sequence for the data in each sliding window L1. The maximum number of 0, 1, 2 in the recognition sequence as the recognition result of each sliding window L1.

Step 3:

Repeat steps 1 and 2 several times until the sliding window L1 moves to the end of the reconstructed signal.

Step 4:

Add the sliding window L2 at the value jump point of the recognition results of window L1. The length and sliding step length of the window L2 are the length of the unit symbol. The maximum number of 0, 1, 2 in each window L2 is counted as the recognition symbols of the TFSK-SSVEP system.

The result of one trial is shown in Fig. 12. The baseband signal is 10121012 …, and each symbol length is 1.2 s. The length of the window L2 is 1.2 s. The recognition symbols are ‘012101210121’, so the recognition result is 1012.

Table 2

The recognition accuracy statistics by different methods in TFSK-SSVEP system

Subjects	SSTFT $+$ coherent demodulation	SVD $+$ coherent demodulation	EMD $+$ coherent demodulation	SSTFT $+$ CCA	SVD $+$ CCA	EMD $+$ CCA
S1	93.3	66.7	60	76.7	60	53.3
S2	90	73.3	56.7	86.7	53.3	63.3
S3	96.6	60	76.7	83.3	56.7	66.7
S4	96.7	73.3	66.7	83.3	73.3	56.7
S5	100	80	63.3	80	66.7	53.3
Average	93.32 $\pm$ 5.25	70.66 $\pm$ 7.59	64.68 $\pm$ 7.68	82 $\pm$ 3.79	62 $\pm$ 8.02	58.66 $\pm$ 6.07

Table 3

The ITR (bit/min) statistics by different methods in TFSK-SSVEP system

Subjects	SSTFT $+$ coherent demodulation	SVD $+$ coherent demodulation	EMD $+$ coherent demodulation	SSTFT $+$ CCA	SVD $+$ CCA	EMD $+$ CCA
S1	25.65	11.12	8.53	15.72	8.54	6.28
S2	23.40	14.04	7.36	21.24	6.28	9.75
S3	24.5	8.54	11.1	19.24	7.37	11.03
S4	20.24	14.01	9.78	19.28	14.06	7.36
S5	25.31	17.37	10.33	17.26	11.15	8.03
Average	23.82	13.02	9.42	18.55	9.48	8.49

Figure 12.

The recognition results of the TFSK-SSVEP system.

Then, the results of the average recognition accuracy of the TFSK-SSVEP system are calculated by combining the three reconstruction methods (SVD, EMD, SSTFT) and two recognition methods (coherent demodulation, CCA) respectively. The results are shown in Table 2.

Compared to the other two reconstruction methods, SSTFT has a better effect on the simulation, so the recognition accuracy in the TFSK-SSVEP system is higher than others. At the same time, comparing the two recognition methods, we found that the coherent demodulation has a higher accuracy in the TFSK-SSVEP system. When combing SSTFT and coherent demodulation, the average recognition accuracy of the TFSK-SSVEP system is as high as 93.32%. Among them, the accuracy of subject 4 can reach 100%.

Finally, under different methods, we counted the ITR (bit/min) of five subjects. As shown in Table 3, using the SSTFT reconstruction method or coherent demodulation can effectively improve the ITR of the TFSK-SSVEP system. But, the method proposed in this paper can increase the ITR of the TFSK-SSVEP system significantly. Its ITR is as high as 23.82 (bit/min), which can improve the performance of the TFSK-SSVEP system effectively.

4. Discussion

The SSVEP-BCI system has less available instructions, because the frequency band which induces SSVEP is narrow. Therefore, in order to increase the number of instructions in the BCI system, a variety of experimental paradigms are adopted. In this paper, we proposed an experimental paradigm based on TFSK modulated stimuli, which achieved the recognition of six targets in the case of three stimulating frequencies. Then, the main problem addressed in this work was to pursue a recognition method with high accuracy in the SSVEP-BCI system based on TFSK modulated stimuli. First, SSTFT was adopted to reconstruct the EEG signal, which can eliminate mode mixing caused by EMD. By processing the stimulated signal, BER, RMES and SNR are compared to other reconstruction methods (Figs 4–6). The feasibility of SST to extract SSVEP components was demonstrated. Second, based on the experimental paradigm of TFSK modulated stimuli, the SSVEP components have the same properties as the TFSK signal. Therefore, this paper improved the coherent demodulation in the field of communications and applies it to the identification of the TFSK-SSVEP system. Finally, in the case of the O1, O2, Oz channel, the accuracy of the improved coherent demodulation is higher than CCA (Table 1), which achieves a higher recognition rate of the TFSK-SSVEP system in the case of fewer channels. It improves the portability and reliability of BCI.

In this paper, the IIR band-pass filter was used to filter the noise in coherent demodulation. In future work, the filter structure can be optimized to reduce the in-band noise and improve the signal-to-noise ratio of the EEG signal, which may improve the accuracy of the TFSK-SSVEP system. Second, incoherent demodulation is used in [11]. The filtered signal will introduce an inconsistent phase error, which will interfere with subsequent identification. In this paper, the sine and cosine components of the reference signal are used to correct the phase difference, in order to improve the accuracy of recognition.

Finally, this paper has some limitations. The coherent demodulation only used the basic frequency component of SSVEP. If combined with its frequency doubling component, the accuracy of the coherent demodulation may be further improved. But correspondingly, the complexity of the algorithm will increase and the computing power of the SSVEP-BCI system will decrease.

5. Conclusion

In the current BCI system, the traditional SSVEP-BCI system using the visual stimulus signal modulated by FM has fewer instructions, whereas the TFSK-SSVEP system can greatly increase the number of instructions and improve the practicability of the SSVEP-BCI system.

For the TFSK-SSVEP system, we proposed SSTFT to reconstruct the EEG signal. This can refine the time-frequency spectrum of the EEG signal to extract the characteristic frequencies of SSVEP, which will lead to higher recognition accuracy compared to EMD and SVD. The feasibility of SSTFT in the TFSK-SSVEP system is shown by comparing the results of simulation and experiment with EMD and SVD.

Footnotes

Acknowledgments

This work was financially supported by the Natural Science Foundation of Chongqing, China (no. cstc2018jcyjAX0163) and the Special Research Project of Philosophy and Social Science of Chongqing Medical University, China (no. 201712).

Conflict of interest

No potential conflict of interest was reported by the authors.

References

Maye

Zhang

Engel

. Utilizing retinotopic mapping for a multi-target SSVEP BCI with a single flicker frequency. IEEE Transactions on Neural Systems & Rehabilitation Engineering. 2017; 25: 1026–1036.

Jafarifarmand

Badamchizadeh

Khanmohammadi

, et al. A new self-regulated neuro-fuzzy framework for classification of EEG signals in motor imagery BCI. IEEE Transactions on Fuzzy Systems. 2018; 26: 1485–1497.

Xiao

Wang

, et al. A brain-computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Transactions on Biomedical Engineering. 2018; 65: 1166–1175.

, H.

, C

,Classification of visual stimuli with different spatial patterns for single-frequency, multi-class SSVEP BCI. Electronics Letters. 2013; 49: 1374–1376.

Chen

Wang

Zhang

, et al. A novel stimulation method for multi-class SSVEP-BCI using intermodulation frequencies. Journal of Neural Engineering. 2017; 14: 1741–2552.

Nakanishi

Wang

, et al. A high-speed brain speller using steady-state visual evoked potentials. International Journal of Neural Systems. 2014; 24: 1–17.

Bradley

Cunnington

. Stimulus specificity of a steady-state visual-evoked potential-based brain – computer interface. Journal of Neural Engineering. 2012; 9: 1–18.

Vialatte

Maurice

Dauwels

, et al. Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives. Progress in Neurobiology. 2010; 90: 418–438.

Asano

Nishida

Fukuda

, et al. Differential visually-induced gamma-oscillations in human cerebral cortex. NeuroImage. 2009; 45: 477–489.

10.

Kimura

Tanaka

Higashi

, et al. SSVEP-based brain – computer interfaces using FSK-modulated visual stimuli. J. IEEE Transactions on Biomedical Engineering. 2013; 60: 2831–2838.

11.

Xing

Dechun

Xia

, et al. A SSVEP stimuli encoding method using trinary frequency-shift keying encoded SSVEP (TFSK-SSVEP). Frontiers in Human Neuroscience. 2017; 11: 1–9.

12.

Wang

Gao

, et al. A practical VEP-based brain computer interface. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2006; 14: 234–240.

13.

Lin

Zhang

, et al. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Transactions on Biomedical Engineering. 2006; 53: 2610–2614.

14.

Thakur

. Synchrosqueezing-based recovery of instantaneous frequency from nonuniform samples. SIAM Journal on Mathematical Analysis. 2011; 43: 2078–2095.

15.

Krawczyk

Gerkmann

, et al. STFT phase reconstruction in voiced speech for an improved single-channel speech enhancement. IEEE/ACM Transactions on Audio Speech & Language Processing. 2017; 22: 1931–1940.

16.

Yin

Shen

, et al. Selection of optimal window length using STFT for quantitative SNR analysis of LFM signal. Journal of Systems Engineering and Electronics. 2013; 24: 26–35.

17.

Herrera

Han

Baan

MVD

. Applications of the synchrosqueezing transform in seismic time-frequency analysis. J. Geophysics. 2014; 79: 55–64.

18.

Chang

Lee

, et al. Frequency recognition in an SSVEP-based brain computer interface using empirical mode decomposition and refined generalized zero-crossing. Journal of Neuroscience Methods. 2011; 196: 170–181.

19.

Wang

Zhang

. Empirical mode decomposition based on instantaneous frequency boundary. Electronics Letters. 2017; 53: 781–783.

20.

Schanze

. Compression and noise reduction of biomedical signals by singular value decomposition. IFAC-PapersOnLine. 2018; 51: 361–366.

21.

. A Concentrated Time-Frequency Analysis Tool for Bearing Fault Diagnosis. IEEE Transactions on Instrumentation and Measurement. 2019: 1–11.

22.

Fang

Liu

Zheng

, et al. FSK/ASK re-modulation WDM-PON with image rejection receiver. Proceedings of SPIE – The International Society for Optical Engineering. 2010; 7988: 146–147.

23.

Songyun

Chang

Klaus

, et al. Stimulator Selection in SSVEP-Based Spatial Selective Attention Study. Computational Intelligence and Neuroscience. 2016: 1–9.

Synchrosqueezing with short-time fourier transform method for trinary frequency shift keying encoded SSVEP

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSION:

Keywords

1. Introduction

2. Materials and methods

2.1 The modulation method of TFSK

2.4.1 The simulated signal

2.4.2 The EEG signal

Table 1 The BCI instructions

3.1 The reconstruction performance of SSTFT in simulation

5. Conclusion

Footnotes

Acknowledgments

Conflict of interest

References

Table 1
The BCI instructions