Cognitive phenomena measurement with time window-based multispectral brain mapping

Abstract

BACKGROUND:

Cognitive neuroscience experiments require accurate and traceable methods of measuring cognitive phenomena, analyzing and processing data, and validating results, including measurement of impact of such phenomena on brain activity and consciousness. EEG measurement is the most widely used tool for evaluation of the experiment’s progress. To extract more information from the EEG signal, continuous innovation is necessary to provide a broader range of information.

OBJECTIVE:

This paper presents a new tool for measuring and mapping cognitive phenomena using time window-based multispectral brain mapping of electroencephalography (EEG) signals.

METHODS:

The tool was developed using Python programming language and enables users to create brain maps images for six spectra (Delta, Theta, Alpha, Beta, Gamma, and Mu) of EEG signal. The system can accept an arbitrary number of EEG channels with standardized labels based on the 10–20 system, and users can select the channels, frequency bandwidth, type of signal processing, and time window length to perform the mapping.

RESULTS:

The key advantage of this tool is its ability to perform short-time brain mapping, which allows for the exploration and measurement of cognitive phenomena. The tool’s performance was evaluated through testing on real EEG signals, and results demonstrated its effectiveness in accurately mapping cognitive phenomena.

CONCLUSION:

The developed tool can be used in various applications, including cognitive neuroscience research and clinical studies. Future work involves optimizing the tool’s performance and expanding its capabilities.

Keywords

Biomedical engineering cognitive neuroscience electroencephalography brain mapping biomedical measurement and instrumentation

1. Introduction

Electroencephalography (EEG) is a noninvasive biomedical measurement technique that measures the electrical signals generated by the brain through the skin of the scalp. EEG electrodes, placed on specific points on the scalp according to the 10–20 system, detect the potential difference between the scalp and a reference point (usually a point on the earlobe or mastoid bone), which is referred to as an EEG channel. EEG signal is generally week with low amplitude, and hence it requires amplification, filtering, and analog-to-digital conversion before being sent to the processing unit, which is often a computer or a specialized device [1, 2].

The majority of useful data from EEG is derived by frequency analysis. Each frequency band is associated with specific brain states and functions, and understanding the distribution of these bands across the scalp can provide valuable insights into brain activity patterns. Table 1 shows the basic division of EEG signals according to their spectrum, with each spectrum reflecting a range of frequencies and associated brain states.

Table 1
EEG bandwidth [1, 2]

EEG band	Corresponding frequency bandwidth
Delta	$<$ 4 Hz
Theta	4–7 Hz
Alpha	8–15 Hz
Mu	10–12 Hz
Beta	16–31 Hz
Gamma	$>$ 32 Hz

Brain mapping is frequently employed as an additional tool for obtaining a more comprehensive understanding of the cognitive state of an individual before, during, and after an experiment. This technique uses frequency analysis of EEG signals collected from specific locations on the scalp according to the 10–20 standard [3]. The signals are then processed to create a two-dimensional or three-dimensional representation of the activity, with colors used to indicate the signal strength in different frequency ranges. Brain mapping allows for the visualization and analysis of the spatial distribution of neural activity, providing insight into the functional connectivity and interactions between different brain regions [3].

This technique has been used in various research fields, including cognitive neuroscience, clinical neurology, and psychology. In cognitive neuroscience, brain mapping is often used to study the neural correlates of various cognitive processes such as attention, memory, and emotion. In clinical neurology, brain mapping can be used to diagnose and monitor neurological disorders such as epilepsy and traumatic brain injury. In psychology, brain mapping has been used to investigate the neural mechanisms underlying behavior and cognitive processes. With the development of advanced brain mapping techniques and tools, such as the time window-based multispectral brain mapping described in this paper, researchers and clinicians have gained an increasingly powerful means to investigate complexities of the human brain [4, 5].

The best results for EEG-based brain mapping are typically obtained from simple psychological experiments that produce clear and distinct neural responses. In such experiments, it is easier to identify which areas of the brain are active in response to specific stimuli, and to discern patterns of neural activity associated with different cognitive processes. However, more complex psychological experiments, such as those involving higher-order cognitive processes or real-world stimuli, can be more challenging to interpret and may require more sophisticated methods of analysis [6, 7, 8, 9].

Brain mapping and quantitative electroencephalography (qEEG) have been widely used in clinical practice to treat various cognitive disorders such as dyslexia, attention deficit hyperactivity disorder (ADHD), hyperactivity, and comorbid depressive symptoms. These tools provide valuable insights into the underlying neural mechanisms of these disorders and help clinicians design individualized treatment plans. Additionally, integration of these tools with other psychological and psychiatric interventions, such as neurofeedback, have shown promising results in improving cognitive functioning and reducing symptoms [10, 11, 12, 13, 14, 15].

Visualization provided by brain mapping and qEEG is crucial for analysis by relevant experts. It enables them to detect and analyze neural patterns associated with specific cognitive functions and disorders. Furthermore, these tools are also essential for training artificial intelligence – based systems that perform classification of cognitive activities and cognitive states. By integrating brain mapping and qEEG data with machine learning algorithms, researchers can develop more accurate and efficient diagnostic and therapeutic approaches for various cognitive disorders [11, 12, 13, 14, 16].

The primary purpose of this research is the development of a tool that is proficient enough to offer additional possibilities for experts to evaluate measurement of brain electrical activity in the context of quantitative electroencephalography, neurofeedback and neuroplasticity research. The developed and validated tool will contribute to the overall understanding of brain electrical activity measurement methods and their applications in advanced neurofeedback and neuroplasticity research. Consequently, in this scientific study, the primary focus is placed on brain mapping and its methodology, rather than on neurofeedback. By emphasizing the importance of brain mapping techniques, the aim is to establish a comprehensive understanding of the underlying processes and the potential implications of such methodologies in the broader context of quantitative electroencephalography, neurofeedback, and neuroplasticity research.

2. Methodology

2.1 Dataset

The EEG data used in this study was provided in EDF [European Data Format] format, with each subject having two files: one with a “_1” suffix representing the background EEG recording (before the mental arithmetic task) and another with a “_2” suffix for the EEG recording during the mental arithmetic task. The recording date/time information was set to January 1st for all files. Subjects were divided into two groups based on their performance in the arithmetic task. Group “G” consisted of 24 subjects with good performance (mean number of operations per 4 minutes $=$ 21, SD $=$ 7.4), while Group “B” included 12 subjects with poor performance (mean number of operations per 4 minutes $=$ 7, SD $=$ 3.6). The “Count quality” column in the subject-info.csv file indicates each subject’s group membership (0 for Group “B” and 1 for Group “G”). Additionally, the subject-info.csv file provides basic information about each subject, such as gender, age, occupation, and date of recording [15].

2.2 System

In order to conduct a more detailed research on cognitive phenomena, a comprehensive tool was developed for loading, processing, analyzing, graphically displaying, saving, and exporting EEG data. The tool provides researchers with a streamlined and customizable workflow for EEG data analysis, allowing them to identify and quantify patterns of neural activity associated with specific cognitive processes more easily.

Figure 1 illustrates the detailed software architecture of the developed tool, including its various modules. Each module is designed to perform a specific set of tasks, and data is passed between modules to ensure a smooth and efficient workflow.

The tool was developed using Python programming language, which provides a powerful and flexible platform for scientific computing and data analysis. Several open-source libraries were used to implement various functionalities of the tool, including scikit-learn (for machine learning algorithms), pandas (for data manipulation), numpy (for numerical computations), pickle (for data serialization), csv (for data input/output), matplotlib (for data visualization), MNE (for EEG data processing and analysis), and fooof (for spectral analysis of EEG signals).

Overall, the developed tool provides researchers with a user-friendly and comprehensive platform for EEG data analysis, allowing them to explore and interpret complex patterns of neural activity associated with cognitive phenomena more effectively. The tool has the potential to facilitate new discoveries in the field of cognitive neuroscience and advance our understanding of the human brain.

Figure 1.

The software architecture of the brain mapping tool.

2.2.1 Mainframe

The Mainframe module is the main processing unit of the developed tool. Its primary role is to issue commands to other modules and report the progress, status, and results to the user. The module works based on the set parameters, which are provided by the user or predetermined by the system. The commands are sent to the respective modules for data processing, analysis, graphic display, saving, and exporting. The mainframe module also interacts with the database to retrieve data and update the stored information. In the current implementation, the Mainframe module can be interacted with directly, but in the future, there are plans to have a user interface module to interact with the mainframe module, making it more user-friendly.

2.2.2 Import module

The system is designed to be highly automated, so when a command is received from the mainframe, it triggers the process of loading files for data processing. The loading process is performed seamlessly by the system, and it automatically recognizes the number and name of the channels from the EEG data. Additionally, the system also extracts other relevant parameters such as sampling frequency, markers, annotations, and metadata, which are important for further analysis of the EEG data.

Once the EEG data has been successfully loaded, the system then proceeds to send the data to the data processing module for analysis. The data processing module is responsible for a wide range of tasks, including signal filtering, feature extraction, artifact removal, and statistical analysis, among others. The processed data is then sent to the database, which is responsible for storing all the relevant information and analysing results.

By automatically recognizing the relevant parameters and seamlessly sending the data between modules, the system minimizes the risk of errors and ensures the accuracy of the analysis results.

2.2.3 Data processing module

The data processing module is a crucial component of the developed tool, responsible for filtering and analyzing the EEG data. The module receives input data from the Input module, and based on the commands received from the mainframe, it applies specific filters to preprocess the data. The filters can be customized according to the user’s requirements, such as low-pass, high-pass, or band-pass filters, to eliminate any unwanted noise or artifacts from the signal [17].

After filtering, the data processing module performs a frequency analysis of the signal and generates a two-dimensional or three-dimensional graphic representation of the brain activity. The graphic display shows the strength of the signal in different frequency bands, represented by different colors, in accordance with the 10–20 standard. During processing, the obtained results are continuously displayed on the screen, enabling the user to monitor the progress of the analysis.

Upon completion, the generated brain mapping results are saved in the database for further analysis and comparison with other results. The module is highly customizable and offers the flexibility to choose the processing and display of all six EEG spectra, including Delta, Theta, Alpha, Beta, Gamma, and Mu, thereby providing a comprehensive view of the cognitive phenomena being studied.

2.2.4 Database module

The database module plays a crucial role in the developed tool because it serves as a centralized storage unit for all the data processed by the system. The stored data is accessible and retrievable by the other modules as needed. To ensure optimal performance and compatibility, the database has been implemented using two standard Python libraries – csv and pickle – allowing the data to be saved in both comma-separated value (CSV) and Python’s native serialization format (Pickle). Moreover, the data stored in the database can be accessed both offline and online through cloud storage.

To facilitate efficient data retrieval and management, the database architecture has been designed to archive and update the data according to the examinee ID, experiment ID, test ID, and file ID. Furthermore, the update process is automatically triggered after each record input, ensuring that the latest data is always available to the other modules. Finally, the database module is integrated and communicates seamlessly with all the other modules presented in Fig. 1, ensuring smooth and uninterrupted data flow throughout the entire system.

2.2.5 Export module

The export module serves as the final step in the data processing pipeline. Its main function is to extract the data from the database and save it to the desired location on the hard drive. Currently, the system only supports local storage and exporting, but future versions of the tool will support cloud-based storage solutions as well. This will enable researchers to access their data from anywhere in the world and collaborate with colleagues remotely.

The export module is activated by the Mainframe, which sends a command to extract the data for a particular examinee or experiment. Once the data is extracted, the module saves it to a file in a format specified by the user. The supported formats include CSV, Excel, MATLAB, vob, mp4 and jpg files.

The export module ensures that the exported data is well organized and easily interpretable. It takes into account the examinee ID, experiment ID, test ID, and file ID to ensure that the exported data can be easily traced back to its source. This feature is particularly useful for researchers who need to keep track of large amounts of data from multiple experiments and examinees.

2.3 Experimental design

The study utilized the Neurocom EEG 23-channel system to record unipolar EEG. Electrodes were placed on the scalp according to the International 10–20 scheme and were referenced to interconnected ear reference electrodes. A high-pass filter with a 30 Hz cut-off frequency and a power line notch filter (50 Hz) were applied to the recordings. EEG segments of 60 seconds duration were collected, and Independent Component Analysis (ICA) [18] was used during data preprocessing to remove artifacts such as those caused by eyes, muscles, and cardiac pulsation. The task presented to the study participants involved serial subtraction of two numbers, and each trial began with oral communication of a 4-digit minuend and a 2-digit subtrahend (e.g., 5212 and 17). Participants with normal or corrected-to-normal visual acuity, normal color vision, and no clinical manifestations of mental or cognitive impairment, verbal or non-verbal learning disabilities were eligible to participate. Exclusion criteria, analogous to other studies, included the use of psychoactive medication, drug or alcohol addiction, and psychiatric or neurological complaints. Data collected as a part of the study involving 36 participants and each recording lasted for a total of 4 minutes. The first 60 seconds were dedicated to recording the EEG in a baseline state without the cognitive task of arithmetic operations, followed by 180 seconds of recording while participants performed the cognitive task. Participant information, such as gender, age, profession, as well as date and time of recording, was collected, along with the results from the cognitive task [15].

3. Results

In order to test the system and confirm the hypothesis, an experiment using the available data was conducted. The experiment involved displaying brain mapping images that corresponded to the moment of cognitive activity when the examinee was performing the arithmetic operation. EEG data that was recorded according to the study method was processed using the developed system to obtain the brain mapping images.

Obtained brain mapping images were then compared with the original EEG data interpreted by an expert group of psychologists. The purpose of this comparison was to validate the accuracy and reliability of the developed system in displaying brain mapping images that correspond to the cognitive activity during the arithmetic operation.

By applying the developed software tool to the entire dataset of all participants, it was possible to compare brain mapping results during relaxation and while performing the arithmetic operation of subtraction. Out of the 36 processed participants, two subject are depicted in Figs 2a–b and 3a–b, respectively. By examining the resulting brain mapping graphics and the data recorded by experts during the experiment, it can be concluded that there is in fact a correlation between the participant’s performance and brain activity.

The results of the experiment indicate that the developed system is able to generate accurate brain mapping images depicting the moment of cognitive activity during arithmetic operations. Figure 2a and b. clearly show the difference in brain activity in a relaxation condition and in condition of cognitive engagement. When the subject is relaxed, reduced brain activation is observed (2a). However, when the subject performs a arithmetic task, greater cognitive engagement is accompanied by an increase in brain activity (3b).

Figure 2.

a. Brain mapping of a subject with good performance during the resting phase. b. Brain mapping of a subject with good performance during the task phase.

Specifically, brain mapping images show a clear increase in the activity in the prefrontal cortex and parietal regions. It has already been established that the prefrontal cortex is the center of cognitive functions. Executive functions related to cognitive processes of problem solving, decision-making, and working memory lead to increased activity in this brain region [19]. The results of earlier studies on the neural basis of numerical cognition speak precisely of the excitation of the parietal lobe during the performance of this task type [20]. This confirmed the initial hypothesis that the system is capable of displaying brain mapping images that accurately correspond to the moment of cognitive activity during the arithmetic operation [15]. These findings are consistent with previous research that identified these areas as the ones involved in mathematical processing and working memory. The obtained results also showed differences in brain activity depending on whether the subjects did experimental task correctly or made mistakes (Fig. 3a and b).

Figure 3.

a. Brain mapping of a subject with bad performance during the resting phase. b. Brain mapping of a subject with bad performance during the task phase.

Increased activity can be observed in subjects who had more errors. This is consistent with findings showing that there is a strong relationship between patterns of frontoparietal activity and behavioral outcomes involving success or failure [21]. By comparing neural patterns reflected through changes in individual brain waves, stronger beta wave activity is noticeable in subjects who make mistakes. This can be explained by the tendency to increase concentration and effort to solve task. But what is also visible is an increase in gamma wave activity. Although these waves are correlated with cognitive phenomena, especially attention and working memory, they can also be used as a measure of stress and anxiety [22]. It can be assumed that subjects who are not successful in solving a numeric task experience more negative emotions due to their failure, and this is also reflected in the pattern of gamma wave oscillations. The obtained results therefore show the multiple usefulness and applicability of the developed system.

In addition to providing accurate brain mapping images, the developed system is also able to generate reliable EEG data. All recordings were devoid of artifacts by using Independent Component Analysis (ICA) that effectively eliminates any artifacts such as eye, muscle, and overlapping cardiac pulsation. This has important implications for the field of cognitive neuroscience, as it provides a tool for researchers to better understand the underlying mechanisms involved in mathematical processing and working memory. However, the low number of examinees who underwent the experiment makes these deductions ungeneralizable. Therefore, further research with larger sample size is necessary to fully establish the validity and reliability of the developed system. The results revealed significant changes in neural activity in response to neurofeedback training, particularly in regions of the brain associated with attention and executive function. Figure 4 displays a snapshot obtained during an experiment in which the EEG signal was analyzed and processed with the developed tool to generate brain mapping images representing six spectra, including Delta, Theta, Alpha, Beta, Gamma, and Mu. This provides experts with a comprehensive graphical representation of the subject’s brain activity at a particular moment, in this specific case, the difference in the spectra was most noticeable at the 32^nd second of the experiment. By visualizing the brain activity, experts can analyze and interpret the cognitive activity of the subject, and better understand the underlying cognitive processes associated with the task performed during the experiment. The ability to view the brain activity in multiple spectra also provides additional insights into the different cognitive states experienced by the subject, which can be useful in identifying and treating cognitive disorders or improving cognitive performance through neurofeedback.

Figure 4.

One frame of brain mapping divided into six graphs.

4. Conclusion

Overall, the experiment was successful in confirming the accuracy and reliability of the developed system in displaying brain mapping images. This new tool developed specifically for cognitive neuroscience research provides a unique opportunity for the scientific community to further explore the cognitive processes of the brain. The results demonstrate the potential usefulness of the system in cognitive neuroscience research, and opens up opportunities for further research in this field. Its capabilities for data processing, analysis, graphic display, saving, and exporting make it a valuable tool in the research of cognitive phenomena, and it will be utilized as one of the modules in multiple ongoing research projects.

As the tool continues to undergo further development, it is expected to become a standalone application that can be run on both local machines and cloud servers. The expansion will enable the tool to perform its tasks more efficiently with greater processing power and consequently provide it with the ability to store and access large amounts of data. This will in turn enable a more insightful understanding of the relationship between neurofeedback and brain mapping, and contribute to scientific endeavors to optimize the use of neurofeedback in clinical and research settings.

Footnotes

Conflict of interest

None to report.

References

Chalupnik

Bialas

Majewska

Kedziora

, editors. Using Simplified EEG-Based Brain Computer Interface and Decision Tree Classifier for Emotions Detection. International Conference on Advanced Information Networking and Applications. Springer. 2022.

Fox

Bakermans-Kranenburg

Yoo

Bowman

Cannon

Vanderwert

, et al. Assessing human mirror activity with EEG mu rhythm: A meta-analysis. 2016; 142(3): 291.

Khazi

Kumar

Vidya

MJIJoIRiS

, Engineering, Technology. Analysis of EEG using 10: 20 electrode system. 2012; 1(2): 185-91.

Hossain

Amin

Alsulaiman

Muhammad

GJAToMC

, Communications, Applications. Applying deep learning for epilepsy seizure detection and brain mapping visualization. 2019; 15(1s): 1-17.

Alkawadri

RJFiN

. Brain-computer interface (BCI) applications in mapping of epileptic brain networks based on intracranial-EEG: An update. 2019; 13: 191.

Bergmann

Hartwigsen

GJJocn

. Inferring causality from noninvasive brain stimulation in cognitive neuroscience. 2021; 33(2): 195-225.

Nastase

Goldstein

Hasson

UJN

. Keep it real: rethinking the primacy of experimental control in cognitive neuroscience. 2020; 222: 117254.

Cabeza

Albert

Belleville

Craik

Duarte

Grady

, et al. Maintenance, reserve and compensation: The cognitive neuroscience of healthy ageing. 2018; 19(11): 701-10.

Pimenta

Brown

Arns

Enriquez-Geppert

SJNd

, treatment. Treatment efficacy and clinical effectiveness of EEG neurofeedback as a personalized and multimodal treatment in ADHD: A critical review. 2021; 17: 637.

10.

Arns

Drinkenburg

Leon Kenemans

JJAp

, biofeedback. The effects of QEEG-informed neurofeedback in ADHD: an open-label pilot study. 2012; 37: 171-80.

11.

Nuwer

MJN

. Assessment of digital EEG, quantitative EEG, and EEG brain mapping: Report of the American Academy of Neurology and the American Clinical Neurophysiology Society* (RETIRED). 1997; 49(1): 277-92.

12.

Baillet

Mosher

Leahy

RMJISpm

. Electromagnetic brain mapping. 2001; 18(6): 14-30.

13.

Tadel

Baillet

Mosher

Pantazis

Leahy

RMJCi

, neuroscience. Brainstorm: a user-friendly application for MEG/EEG analysis. 2011; 2011: 1-13.

14.

Parbat

Chakraborty

MJBSP

, Control.A novel methodology to study the cognitive load induced EEG complexity changes: Chaos, fractal and entropy based approach. 2021; 64: 102277.

15.

Zyma

Tukaev

Seleznov

Kiyono

Popov

Chernykh

, et al. Electroencephalograms during mental arithmetic task performance. 2019; 4(1): 14.

16.

Xiong

Tang

Z-R

G-J

Song

Zhu

L-MJIToS

, Man, et al. Detecting dynamic behavior of brain fatigue through 3-d-CNN-LSTM. 2021; 52(1): 90-100.

17.

Gramfort

Luessi

Larson

Engemann

Strohmeier

Brodbeck

, et al. MNE software for processing MEG and EEG data. 2014; 86: 446-60.

18.

Onton

Westerfield

Townsend

Makeig

SJN

, reviews b. Imaging human EEG dynamics using independent component analysis. 2006; 30(6): 808-22.

19.

Liang

L-Y

Shewokis

Getchell

NJJoB

Science

. Brain activation in the prefrontal cortex during motor and cognitive tasks in adults. 2016; 6(12): 463-74.

20.

Chochon

Cohen

Van De Moortele

Dehaene

SJJCN

. Cortical and subcortical circuits for mental arithmetic: functional neuroimaging of bilateral intraparietal, frontal, cingular and caudate contributions to various calculation tasks. 1999; 11: 617-30.

21.

Robinson

Rich

Woolgar

AJJocn

. Linking the Brain with Behavior: The Neural Dynamics of Success and Failure in Goal-directed Behavior. 2022; 34(4): 639-54.

22.

Minguillon

Lopez-Gordo

Pelayo

FJFicn

. Stress assessment by prefrontal relative gamma. 2016; 10: 101.