Motor Imagery (MI) electroencephalographic (EEG) signal classification is a pioneer research branch essential for mobility rehabilitation. This paper proposes an end-to-end hybrid deep network “Spatio Temporal Inception Transformer Network (STIT-Net)” model for MI classification. Discrete Wavelet Transform (DWT) is used to derive the alpha (8–13) Hz and beta (13–30) Hz EEG sub bands which are dominant during motor tasks to enhance the performance of the proposed work. STIT-Net employs spatial and temporal convolutions to capture spatial dependencies and temporal information and an inception block with three parallel convolutions extracts multi-level features. Then the transformer encoder with self-attention mechanism highlights the similar task. The proposed model improves the classification of the Physionet EEG motor imagery dataset with an average accuracy of 93.52% and 95.70% for binary class in the alpha and beta bands respectively, and 85.26% and 87.34% for three class, for four class 81.95% and 82.66% were obtained in the alpha and beta band respective EEG based motor signals which is better compared to the results available in the literature. The proposed methodology is further evaluated on other motor imagery datasets, both for subject-independent and cross-subject conditions, to assess the performance of the model.
Al-QaysiZTAhmedMAHammashNM, et al.A systematic rank of smart training environment applications with motor imagery brain-computer interface. Multimed Tools Appl. 2023;82(12):17905-17927. doi: https://doi.org/10.1007/s11042-022-14118-x
2.
AltaheriHMuhammadGAlsulaimanM, et al.Deep learning techniques for classification of electroencephalogram (EEG) motor imagery (MI) signals: A review. Neural Comput Appl. 2023;35(20):14681-14722. doi: https://doi.org/10.1007/s00521-021-06352-5
3.
AfrakhtehSMosaviMR. Chapter 2 - applying an efficient evolutionary algorithm for EEG signal feature selection and classification in decision-based systems. In: MohamedA, ed. Energy efficiency of medical devices and healthcare applications. Academic Press; 2020:25-52. https://doi.org/10.1016/B978-0-12-819045-6.00002-9
4.
Al-SaeghADawwdSAAbdul-JabbarJM. Deep learning for motor imagery EEG-based classification: A review. Biomedical Signal Processing and Control. 2021;63(102172):1-19. doi: https://doi.org/10.1016/j.bspc.2020.102172
5.
SharmaRKimMGuptaA. Motor imagery classification in brain-machine interface with machine learning algorithms: Classical approach to multi-layer perceptron model. Biomed Signal Process Control. 2022;71(Part A):1-10. doi: https://doi.org/10.1016/j.bspc.2021.103101
6.
SubasiAGursoyMI. EEG Signal classification using PCA, ICA, LDA and support vector machines. Expert Syst Appl. 2010;37(12):8659-8666. doi: https://doi.org/10.1016/j.eswa.2010.06.065
7.
XuMHanJWangY, et al. Implementing over 100 command codes for a high-speed hybrid brain-computer interface using concurrent P300 and SSVEP features. IEEE Trans Biomed Eng. 2020;67(11):3073-3082. doi: https://doi.org/10.1109/TBME.2020.2975614
GaurPGuptaHChowdhuryA, et al. A sliding window common spatial pattern for enhancing motor imagery classification in EEG-BCI. IEEE Trans Instrum Meas. 2021;70:1-9. doi: https://doi.org/10.1109/TIM.2021.3051996
10.
RithwikPBenzyVKVinodAP. High accuracy decoding of motor imagery directions from EEG-based brain computer interface using filter bank spatially regularised common spatial pattern method. Biomed Signal Process Control. 2022;72:1-7. doi: https://doi.org/10.1016/j.bspc.2021.103241
11.
JinJQuTXuR, et al. Motor imagery EEG classification based on riemannian sparse optimization and dempster-shafer fusion of multi-time-frequency patterns. IEEE Trans Neural Syst Rehabil Eng.2023;31:58-67. doi: https://doi.org/10.1109/TNSRE.2022.3217573
12.
Blanco-DiazCFAntelisJMRuiz-OlayaAF, et al. Comparative analysis of spectral and temporal combinations in CSP-based methods for decoding hand motor imagery tasks. J Neurosci Methods. 2022;371:109495. doi: https://doi.org/10.1016/j.jneumeth.2022.109495
13.
JinJXiaoRDalyI, et al. Internal feature selection method of CSP based on L1-norm and dempster-shafer theory. IEEE Trans Neural Netw Learn Syst. 2021;32(11):4814-4825. doi: https://doi.org/10.1109/TNNLS.2020.3015505
14.
AlnaanahMWahdowMAlrashdanM. CNN Models for EEG motor imagery signal classification. Signal Image Video Process. 2023;17(3):825-830. doi: https://doi.org/10.1007/s11760-022-02293-1
15.
LeeDYJeongJHLeeBH, et al. Motor imagery classification using inter-task transfer learning via a channel-wise variational autoencoder-based convolutional neural network. IEEE Trans Neural Syst Rehabil Eng.2022;30:226-237. doi: https://doi.org/10.1109/TNSRE.2022.3143836
HwaidiJFChenTM. Classification of motor imagery EEG signals based on deep autoencoder and convolutional neural network approach. IEEE Access. 2022;10:48071-48081. doi: https://doi.org/10.1109/ACCESS.2022.3171906
18.
MirzaeiSGhasemiP. EEG motor imagery classification using dynamic connectivity patterns and convolutional autoencoder. Biomed Signal Process Control. 2021;68:1-8. doi: https://doi.org/10.1016/j.bspc.2021.102584
19.
LiuTYangD. A densely connected multi-branch 3d convolutional neural network for motor imagery eeg decoding. Brain Sci. 2021;11(2):1-24. doi: https://doi.org/10.3390/brainsci11020197
20.
KhademiZEbrahimiFKordyHM. A transfer learning-based CNN and LSTM hybrid deep learning model to classify motor imagery EEG signals. Comput Biol Med. 2022;143:1-14. doi: https://doi.org/10.1016/j.compbiomed.2022.105288
21.
WangJChengSTianJ, et al. A 2D CNN-LSTM hybrid algorithm using time series segments of EEG data for motor imagery classification. Biomed Signal Process Control. 2023;83:1-11. doi: https://doi.org/10.1016/j.bspc.2023.104627
22.
JinJChenWXuR, et al.Multiscale spatial-temporal feature fusion neural network for motor imagery brain-computer interfaces. IEEE J Biomed Health Inform. 2024;29(1):198-209. doi: https://doi.org/10.1109/JBHI.2024.3472097
23.
ShanmugamSDharmarS. A CNN-LSTM hybrid network for automatic seizure detection in EEG signals. Neural Comput Appl. 2023;35(28):20605-20617. doi: https://doi.org/10.1007/s00521-023-08832-2
24.
LiuCJinJDalyI, et al.SincNet-Based hybrid neural network for motor imagery EEG decoding. IEEE Transactions on Neural Systems and Rehabilitation Engineering.2022;30:540-549. doi: https://doi.org/10.1109/TNSRE.2022.3156076
25.
LiYGuoLLiuY, et al. A temporal-spectral-based squeeze-and- excitation feature fusion network for motor imagery EEG decoding. IEEE Transactions on Neural Systems and Rehabilitation Engineering.2021;29:1534-1545. doi: https://doi.org/10.1109/TNSRE.2021.3099908
26.
ZhangDChenKJianD, et al. Motor imagery classification via temporal attention cues of graph embedded EEG signals. IEEE Journal of Biomedical and Health Informatics. 2020;24(9):2570-2579. doi: https://doi.org/10.1109/JBHI.2020.2967128
27.
AminSUAltaheriHMuhammadG, et al. Attention-inception and long- short-term memory-based electroencephalography classification for motor imagery tasks in rehabilitation. IEEE Trans Industr Inform. 2022;18(8):5412-5421. doi: https://doi.org/10.1109/TII.2021.3132340
28.
LiYZhangXRZhangB, et al. A channel-projection mixed-scale convolutional neural network for motor imagery EEG decoding. IEEE Trans Neural Syst Rehabil Eng.2019;27(6):1170-1180. doi: https://doi.org/10.1109/TNSRE.2019.2915621
29.
TanXWangDChenJ, et al. Transformer-based network with optimization for cross-subject motor imagery identification. Bioengineering. 2023;10(5):1-20. doi: https://doi.org/10.3390/bioengineering10050609
30.
ZhaoWJiangXZhangB, et al. CTNet: a convolutional transformer network for EEG-based motor imagery classification. Sci Rep. 2024;14(1):1-22. doi: https://doi.org/10.1038/s41598-024-71118-7
31.
MaYSongYGaoF. A novel hybrid CNN-transformer model for EEG motor imagery classification. In Proceedings of the International Joint Conference on Neural Networks. Institute of Electrical and Electronics Engineers Inc.; 2022. doi: https://doi.org/10.1109/IJCNN55064.2022.9892821.
32.
ZhangJLiKYangB, et al. Local and global convolutional transformer-based motor imagery EEG classification. Front Neurosci. 2013;17:1219988. doi: 10.3389/fnins.2023.1219988
33.
LuoJWangYXiaS, et al.A shallow mirror transformer for subject-independent motor imagery BCI. Computers in Biology and Medicine. 2023;164:1-12. doi: https://doi.org/10.1016/j.compbiomed.2023.107254
34.
ShiXLiBWangW, et al. EEG-VTTCNet: A loss joint training model based on the vision transformer and the temporal convolution network for EEG-based motor imagery classification. Neuroscience. 2024;556:42-51. doi: https://doi.org/10.1016/j.neuroscience.2024.07.051
GaoDYangWLiP, et al.A multiscale feature fusion network based on attention mechanism for motor imagery EEG decoding. Appl Soft Comput. 2024;151:1-16. doi: https://doi.org/10.1016/j.asoc.2023.111129
37.
GoldbergerALAmaralL A.N.GlassL, et al.PhysioBank, PhysioToolkit, and PhysioNet components of a new research resource for complex physiologic signals. 2000. [Online]. Available: http://www.physionet.org
38.
SchalkGMcFarlandDJHinterbergerT, et al. BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng. 2004;51(6):1034-1043. doi: https://doi.org/10.1109/TBME.2004.827072
39.
AliyuILimCG. Selection of optimal wavelet features for epileptic EEG signal classification with LSTM. Neural Comput Appl. 2023;35(2):1077-1097. doi: https://doi.org/10.1007/s00521-020-05666-0
40.
SzegedyCVanhouckeVIoffeS, et al. 2016. Rethinking the inception architecture of computer vision. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp. 2818-2826
41.
VaswaniAShazeerNParmarN, et al.Attention is all you need. Jun. 2017Advances in Neural Information Processing Systems[Online]. Available: http://arxiv.org/abs/1706.03762
42.
BrunnerCLeebRMüller-PutzGR, et al. BCI competition 2008-Graz data set A experimental paradigm. [Online]. Available: http://biosig.sourceforge.net/.
43.
LeebRBrunnerCMüller-PutzGR, et al. BCI Competition 2008-Graz data set B experimental paradigm. [Online]. Available: http://biosig.sourceforge.net/.
44.
LiuKYangMYuZ, et al. FBMSNet: A filter-bank multi-scale convolutional neural network for EEG-based motor imagery decoding. IEEE Trans Biomed Eng. 2023;70(2):436-445. doi: https://doi.org/10.1109/TBME.2022.3193277
45.
ManeRChewEChuaK, et al. 2021. FBCNet: A multi-view convolutional neural network for brain-computer interface. Mar. 2021, [Online]. Available: http://arxiv.org/abs/2104.01233.
46.
XieJZhangJSunJ, et al.A transformer-based approach combining deep learning network and spatial-temporal information for raw EEG classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering.2022;30:2126-2136. doi: https://doi.org/10.1109/TNSRE.2022.3194600
47.
Perez-VelascoSSantamaria-VazquezEMartinez-CagigalV, et al. EEGSym: Overcoming inter-subject variability in motor imagery based BCIs with deep learning. IEEE Trans Neural Syst Rehabil Eng.2022;30:1766-1775. doi: https://doi.org/10.1109/TNSRE.2022.3186442
48.
AliOSaif-ur-RehmanMGlasmachersT, et al. Contranet: A hybrid network for improving the classification of EEG and EMG signals with limited training data. Comput Biol Med. 2024;168:107649. doi: https://doi.org/10.1016/j.compbiomed.2023.107649
49.
HuangYZhengJXuBinxing, et al.An improved model using convolutional sliding window-attention network for motor imagery EEG classification. Front Neurosci. 2023;17:1-17. doi: https://doi.org/10.3389/fnins.2023.1204385
