An intelligent wearable embedded architecture for stress detection and psychological behavior monitoring using heart rate variability

Abstract

This study quantifies individual stress levels through real-time analysis of wearable sensor data. An embedded setup utilizes artificial neural networks to analyze R-R intervals and Heart Rate Variability (HRV). Emotion recognition of happiness, sadness, surprise, fear, and anger is explored using seven normalized HRV features. Statistical analysis and classification with a neural network model are performed on approximately 20,700 segments,with participants within the age ranged from 23 to 40, mixed gender, and normal health status, along with other pertinent demographics included. Findings show stress observation’s potential for mental well-being and early detection of stress-related disorders. Three classification algorithms (LVQ, BPN, CART) are evaluated, comparing ECG signal correlation features with traditional ones. BPN achieves the highest emotional recognition accuracy, surpassing LVQ by 5.9% – 8.5% and CART by 2% – 6.5%. Maximum accuracy is 82.35% for LVQ and 97.77% for BPN, but does not exceed 95%. Using only 72 feature sets yields the highest accuracy, surpassing S1 by 17.9% – 20.5% and combined S1/S2 by 11% – 12.7%. ECG signal correlation features outperform traditional features, potentially increasing emotion recognition accuracy by 25%. This study contributes to stress quantification and emotion recognition, promoting mental well-being and early stress disorder detection. The proposed embedded setup and analysis framework offer real-time monitoring and assessment of stress levels, enhancing health and wellness.

Keywords

Psychological behavior stress monitoring artificial neural networks wearable embedded sensors heart rate variability ECG

1 Introduction

Constant exposure to pressurized circumstances can really put an individual in a dangerous position. Hence stress in the working environment needs to be measured and monitored, providing feedback to line managers to take action and bring it back to safer levels. The stress-associated illness should be detected earlier [1–3]. The present research hints at the usage of wearable low-cost solutions with embedded back propagation algorithms in neural networks to identify stress from ECG signals. A variety of psychological disorders and physical problems stem out of the exposition to chronic stress which may include cardiovascular diseases like cardiac arrest and immune deficiency. This may even affect the functioning of the most important organs of the human body, like for example the liver. Emotions which have an undeniable impact on every aspect of daily life are a complicated psychological and physiological process. Attempts were made by psychologists to identify different types of emotions experienced by people. Various theories have emerged to categorize these emotions. Paul Eckman identified six basic emotions during the 1970 s, which were claimed to be universally experienced by human beings across all cultures. These are happiness, sadness, disgust, fear, surprise, and anger [4–6]. This list was afterward further expanded by including new emotions like pride, shame, embarrassment, and excitement. An open challenge studied over the last two decades is the automatic identification of human emotions by computers, which would strengthen the quality of human– computer interaction and has immediate applications in medicine, education, entertainment, commerce, and other fields [7].

Emotion recognition technology is based on human behavior patterns (mainly facial expressions and voice) and on physiological signals. As regards the first approach emotions are detected according to the facial expressions, voice, and body movements. However, there is a deceiving possibility that people can conceal their real emotions and control their behavior intentionally [8–10]. The second approach detects emotions according to the physiological changes, which cannot be consciously controlled, like for example the heartbeat or skin sweat. Therefore, emotion recognition systems that are based on physiological signals of the human body can capture the affective status of a subject.

Previous studies have indicated that the emotional state of an individual can be analyzed through various channels, including physiological reactions, facial characteristics (such as facial expressions, eye gaze, and eye blinks), and voice [11–13]. Consequently, researchers have examined features based on body expressions to explore these channels. Additionally, the investigation of physiological responses has involved the analysis of individual physiological signals, such as Electrocardiography (ECG), Electromyography (EMG), and Electro Dermal Activity (EDA), across different affective states [14–16].

Electrical impulses from muscles and skin are recorded for EMG and EDA studies, respectively. For EMG, skin-surface electrodes are used, and for EDA, two electrodes are fastened to the fingers or palm. Relevant features are identified and evaluated using statistical or machine learning approaches to find patterns related to muscle activity or emotional changes after data acquisition and processing to remove noise and artifacts. Biofeedback, stress assessment, and a number of other disciplines, including sports science, rehabilitation, psychology, and human-computer interaction, can all benefit from these assessments. Before applying these measures in particular applications, validation and calibration are crucial for ensuring accuracy.

1.1 Literature review for emotion recognition from physiological signals with emphasis in ECG

ECG acquisition involves capturing the heart’s electrical activity using electrodes placed on the skin. Different methods are utilized for obtaining ECG signals, each serving specific purposes. The standard 12- lead ECG is the most widely used approach, providing a comprehensive perspective of the heart’s activity from different angles. Single or 3-lead portable devices are commonly employed in wearable health trackers, while Holter and event monitors allow continuous or intermittent recording over extended periods. Exercise stress tests evaluate the heart’s response to physical activity, while signal-averaged ECG enhances signal clarity. Ambulatory ECG monitoring facilitates realtime remote tracking, and implantable cardiac monitors provide continuous monitoring of heart activity. The choice of acquisition method depends on the particular clinical or research requirements, and ongoing technological advancements have resulted in the creation of various wearable and portable ECG devices, significantly enhancing cardiac monitoring accessibility and convenience.

The primary challenges in emotion modeling (affective computing) using ECG data revolve around discovering representations that remain unaffected by inter- and intra-subject variations, as well as the inherent noise present in ECG data recordings. The most frequent invariant features were recovered from the unprocessed ECG data (in frequency and temporal domain) [6]. A system or device [7] that uses affective computing is useful for the mental health and wellness of those who are under stress, suffering from anxiety, or experiencing depression. Affective computing is made possible by essential technology called emotion recognition systems. Heart sound signals can be used for emotion detection [17], and wavelet signal analysis to enhance the performance of emotion recognition from ECG signals [18], The subtler emotional variations that can be noticed in healthy ECG data are typically captured via a mix of non-linear analysis and HOS (Higher Order Statistics) [19], decision tree (J48) and IBK classifiers to recognize a few emotional states, such as “Sad,” “Dislike,” “Joy,” “Stress,” “Normal,” “No-Idea,” “Positive,” and “Negative,” using the eHealth platform [20], a visual tool called Graphical Assessment of Real-life Application-Focused Emotional Dataset [21]. There are numerous methods for creating emotion recognition systems today using different methodologies and algorithms.

The identified method prefers neural network analysis- based stress monitoring. The evaluation of short time series of R – R intervals by artificial neural network analysis is suggested in this method [22]. The features extracted from each sample in the ECG signal are fed to the neural network. A manual switch is employed to select the ECG generator signals which are further fed to the neural network sub-system. The maximum of all past in-puts (u) is obtained as output from the Min-Max Running resettable block. An external reset signal (R) can reset the block’s state. ANN filter detects the peaks which then are fed to the heart rate counter [23]. This counter counts the number of peaks in a given time interval. This counting continues until the specified upper limit is reached. Presence of stress in the given signal results in the output – ‘1’ while the absence of stress results in the output – ‘0’.

The remainder of this article is organized as follows. In Section 2 we present wearable embedded architecture developed for stress detection, which includes the preprocessing strategies, feature extraction, and emotion recognition models by the developed ANN model. In Section 3, the results were presented with special emphasis on the classification of emotions. Finally in Section 4 conclusions are provided.

2 Wearable embedded architecture

The relationship between stress and emotions in people and heart rate is based on how the autonomic nervous system of our body reacts to stress or strong emotions by causing the release of stress hormones like cortisol and adrenaline. The body changes as a result of this physiological reaction, also referred to as the “fight-or-flight” response, including a rise in heart rate [24]. A tough task at work or a crucial presentation might cause the sympathetic nervous system to be activated, which increases heart rate. This reaction gets the body ready for action by boosting blood flow to the muscles and sharpening focus and awareness. Similarly to this, our heart rate tends to increase when we feel powerful emotions like fear, excitement, or rage. Heart rate and stress/emotions are correlated in both directions. Changes in heart rate can affect our emotional state in addition to how stress and emotions affect it. For example, a quick heartbeat might amplify anxiety or panic attacks, whereas a moderate, steady heartbeat can promote tranquility and relaxation [25].

Heart rate sensors on fitness trackers and smart-watches, among other wearable embedded systems, can be quite helpful in tracking and analyzing these physiological reactions. These gadgets offer insightful data regarding the wearer’s emotional and stress levels by continuously monitoring their heart rate [26]. They can use optical sensors or chest straps to obtain real-time heart rate information from the wearer’s wrist. After that, this data is processed and analyzed using machine learning algorithms. The device can identify changes in heart rate that signify increased stress or certain emotional states by developing patterns and correlations. For instance, the wearable gadget may deduce that the user is under severe stress or intense emotions if it notices a sudden, large spike in heart rate along with other physiological changes. Through notifications or alerts, this information can be communicated to the user, enabling them to become more aware of their mental health and take the necessary steps to manage stress [27].

The block diagram of the wearable embedded architecture for stress detection and psychological behavior monitoring is illustrated in Fig. 1. Wearable skin contact sensors are acquiring the ECG signal, which is processed on the embedded board. The processing of the ECG signal includes pro-processing of the signal, extraction of parametric features, and processing of the corresponding ECG signal parameters by a machine learning-based classification module. The detected psychological conditions as well as the acquired data and the location of the subjects can be transmitted wirelessly to a backend server of a cyber-physical system for further processing and management. The preprocessing, feature extraction and classification steps of the ECG-acquired data are described in detail below.

Fig. 1

Block diagram of the wearable architecture for stress and psychological conditions detection based on heart rate monitoring.

2.1 Preprocessing of ECG signal

Figure 2 illustrates the substantial influence of noise cancellation in the electrocardiogram (ECG) signal, allowing for the identification of essential signal components that may be obscured by disturbances. Power Line Interference (PLI) commonly constitutes the main source of noise in many bio-electric signals. To mitigate PLI in ECG signals, notch filters have been employed [28, 29]. These filters are specifically designed to eliminate the 50 Hz power line interference, while a low pass filter aids in removing baseline wander.

Fig. 2

Notch filter for noise cancellation in ECG.

2.2 Feature extraction

The choice of appropriate features plays a crucial role in classification tasks. Incorrect feature selection can lead to poor results, even when using the best classifier. Therefore, feature selection holds significant importance in classification. In this study, the morphological features of the ECG signal, including energy, peak power, and spectral entropy, are considered. These features are presented in Tables 1 to 3 [14]. To obtain a total of 28 features, the standard deviation, mean value, maximums, and minimum values of the first seven morphological features are calculated [30]. Consequently, a total of 32 features are extracted from the ECG signal.

Table 1
State definition for table cleaning scenario

Parameter Unit Description

SDNN ms The standard deviation of NN intervals.

SDRR ms The standard deviation of RR intervals

SDANN ms The standard deviation of the mean NN intervals calculated for every 5-minute segment within a 24-hour heart rate variability (HRV) recording.

pNN50 % The percentage of consecutive RR intervals that exhibit a difference exceeding 50 milliseconds.

HR MaxHR Min bpm The average difference between the highest and lowest heart rates observed during each respiratory cycle.

RMSSD ms The Root mean square (RMS) of successive RR interval differences

HRV triangular index The integral of the density of the RR interval histogram, normalized by its height.

TINN ms The width of the baseline in the RR interval histogram.

Parameter	Unit	Description
SDNN	ms	The standard deviation of NN intervals.
SDRR	ms	The standard deviation of RR intervals
SDANN	ms	The standard deviation of the mean NN intervals calculated for every 5-minute segment within a 24-hour heart rate variability (HRV) recording.
pNN50	%	The percentage of consecutive RR intervals that exhibit a difference exceeding 50 milliseconds.
HR MaxHR Min	bpm	The average difference between the highest and lowest heart rates observed during each respiratory cycle.
RMSSD	ms	The Root mean square (RMS) of successive RR interval differences
HRV triangular index		The integral of the density of the RR interval histogram, normalized by its height.
TINN	ms	The width of the baseline in the RR interval histogram.

Table 2

State definition for table cleaning scenario

Parameter	Unit	Description
ULF power	ms2	The absolute power of the ultra low frequency band (0.003 Hz)
VLF power	ms2	The absolute power of the very low frequency band. (0.0033– 0.04 Hz)
LF peak	Hz	The peak frequency of the low frequency band (0.04– 0.15 Hz)
LF power	ms2	The absolute power of the low frequency band (0.04– 0.15 Hz)
LF power	nu	The relative power of the low frequency band (0.04– 0.15 Hz) in normal units
LF power	%	The relative power of the low frequency band (0.04– 0.15 Hz)
HF peak	Hz	The peak frequency of the high frequency band (0.15– 0.4 Hz)
HF power	ms2	The absolute power of the high frequency band (0.15– 0.4 Hz)
HF power	nu	The relative power of the high frequency band (0.15– 0.4 Hz) in normal units
HF power	%	The relative power of the high frequency band (0.15– 0.4 Hz)
LF/HF	%	The ratio of LF to HF power

Table 3

State definition for table cleaning scenario

Parameter	Unit	Description
S	ms	The area of the ellipse, which represents the total heart rate variability (HRV).
SD1	ms	The standard deviation of the Poincar $\overset{´}{e}$ plot measured perpendicular to the line of identity.
SD2	ms	Poincar $\overset{´}{e}$ The standard deviation of the Poincar $\overset{´}{e}$ plot measured along the line of identity.
SD1/SD2	%	Ratio of SD1-to-SD2
ApEn		Approximate entropy quantifies the level of regularity and complexity within a time series.
SampEn		Sample entropy evaluates the level of regularity and complexity exhibited by a time series.
DFA 1		Detrended fluctuation analysis characterizes the short-term fluctuations present in a time series.
DFA 2		Detrended fluctuation analysis characterizes the long-term fluctuations present in a time series.
D2		Correlation dimension, which estimates the minimum number of variables required to construct a model of system dynamics
S	ms	The area of the ellipse, which represents the total heart rate variability (HRV).
SD1	ms	The standard deviation of the Poincar $\overset{´}{e}$ plot measured perpendicular to the line of identity.

2.3 Process of emotion recognition

The ANN is fed with values of the emotion features so as to train the model to generate an individual emotion classification model [31]. The feature values when fed to the classification model, the emotion type is interpreted and produced as output. The classification process involves three steps. A part of the data in each emotion was used as the training data at the first instance during the training process. The evaluated feature values are normalized and fed to the ANN as input to train the classification model. The remaining data were used as the testing data during the subtesting process. The ANN was then fed with the evaluated feature values after normalization, to determine the classification [32].

2.4 Emotion recognition by ANN

In Fig. 3, one or more hidden layers form a multilayer artificial neural network. This kind of network is capable to connect the feedforward with the arrival of the input signal into the network. An error value is calculated if the output signal deviates from the expected output [33, 34]. This error value is fed forward to the input to adjust the weight. The ANN-based classification is influenced by the number of hidden layer neurons, the number of hidden layers, and the learning rates.

Fig. 3

Back Propagation Neural Network architecture.

A neuron in a hidden layer is associated with each neuron in the neuron in the layer above it and beneath it. In Fig. 3, weight w_ij associates input node x_i to concealed node h_j, and weight v_jk interfaces h_j to yield hub y_k. Classification begins by allotting the information nodes x_i, lil equivalent to the related data vector segment. At that stage, information flows forward through the neurons until reaching the output nodes y_k, where lin. The network has the capability to differentiate between 2n distinct classes, as long as its outputs are assigned binary values of 0 and 1.

Table 4, shows Confusion Matrix. For the classifier, the input is the feature dataset matrix (79×44) and its output result is compared with pre-defined binary targeted values. The selected record number with the feature set is randomly assigned into training and testing subsets using training data as 90% and testing data as 45%. The simulation has been performed by using BPN. One represents the output layer. The hidden layer structure of (10-3-1) is used to train BPN with and without. Results in the structure with more than one hidden layer are significantly better. This fits with the ECG signal’s nature, which is distinctly nonlinear. In this test, a total of 25 patterns were applied. The patterns are extracted from normal ECG signals. With a classification performance of 93.7%, the classifier was able to identify mostly all the patterns.

Table 4

State definition for table cleaning scenario

Confusion Matrix	Normal ECG	Classification
Actual	25	93.7%

3 Results and discussion

Real-time monitoring is expected by the regulators. The manually driven conventional compliance monitoring programs rely on sampling in order to identify possible risks and anomalies. The model used in this study was derived from the training database. Test information is then inputted into the model, which generates test results. These results are used to assess the effectiveness of the model in determining whether an individual is experiencing stress or is in a normal state. Stress identification experiments were conducted using ECG data. The collected data were trained using a neural network with the backpropagation algorithm implemented in MATLAB software. In healthy individuals, heart rates fluctuate from beat to beat due to changes in the activity of the autonomic nervous system at the sinus node. During periods of mental or physical stress, heart rate variability (HRV) tends to decrease, while it increases during relaxation [35]. As a result, HRV is regarded as a non-invasive indicator of autonomic nervous system activity. Diabetes neuropathy has long been identified by the absence of HRV. Low HRV has been demonstrated to be prognostic in patients with myocardial infarction in more recent years. Low HRV is also linked to death and the risk of cardiac events in the general population. To investigate the relationship between the HRV responses, physical position, negative emotion, and time of day in periodic diary entries. As predicted, their findings demonstrated that RR interval reductions were related to increases in stress. Additionally, a large rise in the LF/HF ratio was significantly correlated with psychological stress, indicating higher SNS (Sympathetic Nervous System) activity during stressful times of the day [36].

3.1 Hardware set up of hear rate monitoring device

In Tables 5 and 6 hardware components and device descriptions are given. (AD8232) with an Arduino development board and an ESP 32 Bluetooth and WiFi board to obtain an individual’s ECG on a serial plotter, or over Bluetooth and the internet using ESP 32. The sensor board used to measure the electrical activity of the heart is reasonably priced. An ECG or electrocar-diogram can be used to track this electrical activity and produce an analog reading. The AD8232 Single Lead Heart Rate Monitor functions as an op-amp to assist in obtaining a clean signal from the PR and QT Intervals with ease. ECGs can be very noisy.

Table 5
Hardware components

Device Description

Arduino Board ECG Sensor Wires UNO R3 Development board AD8232

Breadboard Jumper wires

Experimental

Device	Description
Arduino Board ECG Sensor Wires	UNO R3 Development board AD8232
Breadboard	Jumper wires
Experimental

Table 6

Hardware components

Label Name	Description	Connection
Vcc	3.3 V Power supply	Vcc
GND	Ground	GND
LO+	Leads off Detect+	10
LO-	Leads off Detect -	11
SDN	Power OFF	NA
Output	Signal Output	A0

The AD8232 module enables the capture of the heart’s electrical activity, enabling the acquisition of an electrocardiogram (ECG). ECG sensors detect signals generated by the heart’s electrical impulses, which initiate the heartbeat. These electrical signals travel through specific pathways within the heart. To record this electrical activity, electrodes are placed on the skin, typically on the arms, legs, and front of the chest. The AD8232 sensor module incorporates carefully calibrated signal amplifiers and noise filters specifically designed for ECG signals. The module blocks out the 60 Hz noise that is produced by standard household electricity. Since the module’s output is analog in nature, the header pins must be soldered in order to connect it to a microprocessor with analog inputs, such as an Arduino, device.

In Fig. 4, a Heart Rate monitoring device is presented. The presence or absence of stress is represented by the values (1) or (0) in the block respectively. The heart rate of ECG signal 2 is represented by the value 151 while that of ECG signal 1 is represented by the value 100. The manual switch is used for selecting the ECG generator signals which are fed to the neural network subsystem. The Min-Max Running resettable block gives the minimum or maximum output of all the past inputs (u).

Fig. 4

Heart rate monitoring interfacing with human body.

HRV and heart rate are physiological parameters that are influenced by age, gender, and circadian rhythm. Your HRV must be taken into account in brief time windows of a few minutes to a few hours. Your HRV greatly increases while you sleep and dramatically decreases in the moments before you wake up. Two-minute rhythm strips were used to calculate HRV. Reports of stress have a detrimental effect on heart rate variability 63% of the time. Additionally, women (64%) are slightly more likely than males (62%) to experience it. There is a 4-millisecond decrease in the overall average difference. At the 25th percentile, the decrease is 8 ms. Stress actually has a slightly less negative impact on HRV as people get older. We collected the data from different age groups like (i) 18-25 (i) 26-30 (iii) 30-40 (iv) 40-50 and (v) 50 -60. The experiments are done with personal behavior like happiness, sadness, anger, fear, and surprise with the above- aged person group, and collected data are stored in the database for analysis. For nearly every group 20 persons’ data are taken as references and compared with three different algorithms. The obtained results are analyzed with parameters like accuracy, precision, recall, F-measures, sensitivity, and specificity. The model was taken from the test and training databases. information is provided to the model, which then produces the test data. Through age 29, the median difference is -5 ms, -4 for ages 30-39, and barely -3 for ages 40– 69.

The peaks detected by BPN are fed as input to the heart rate counter which counts the number of peaks for a given interval of time. The Counter Limit block restricts the counting to a pre-set upper limit and then the counter is reset to zero, restarting the counting up process. After the counter is initialized to zero, the upper limit parameter and the sample time parameter were used to specify the upper limit and sample time respectively. The resulting output is an unsigned integer that can be either 8, 16, or 32 bits long, utilizing the minimum number of bits required to represent the upper limit. The counter subsystem was employed to compute the R-R intervals within a given timeframe. The signals were directed to an Up-counter. The ECG signals are compared with the value of constant 2, in the relational operator. The pulse generator generated pulses till the value of Constant 2 was reached and fed to the multiport switch. The up counter was used for the determination of the value of Constant 2 again and the signal was checked for a certain time period. A (1) output is displayed if the stress is present else (0) is displayed.

In Fig. 5, the ECG signal of the human body is measured by connecting three wire connections: RA (Right Arm), LA (Left Arm), and RL (Right Leg). The value of N represents the number of bits, with the maximum possible value being 2N-1. The Counter Free Running block counts up until it reaches this maximum value. Once the counter overflows, it resets to zero and starts counting up again. The counter is always initialized to zero. The Maximum Heart Rate (Max HR) is defined as the highest number of beats a person’s heart contracts during a one-minute measurement. It serves as a valuable tool for measuring training intensities and assessing exercise and stress levels. To select the ECG generator signals, a multiport switch is utilized, which are then fed to the Ana-log filter. The Min – Max Running Resettable block determines the minimum or maximum output from past inputs (u). One method for determining the maximum heart rate is by using an age-predicted formula. $MEN : 220 - age = age - adjusted Max HR$ $WOMEN : 226 - age = age - adjusted Max HR$

Fig. 5

Human wearing the device.

In Fig. 6, if a person is 30-year, then the age-adjusted maximum heart rate is 226 – 30 years = 196 bpm (beats per minute). The ECG generator signals are selected using a multi-port switch by specifying the constant values like 1, 2, and 3. The Signal is fed to the Analog filter. The parameters displayed in the dialog box vary for different design/band combinations. Only some of the parameters listed below are visible in the dialog box at any one time The Min-Max Running Resettable block outputs the minimum or maximum of all past inputs u. Select the relational operator connecting the two inputs with the Relational Operator parameter. The block updates to display the selected operator. The input with the smaller positive range is converted to the data type of the other input offline using round-to-nearest and saturation. This conversion is performed prior to comparison.

Fig. 6

Heart rate of male and female as per age.

The counter is initialized to zero at all times. The Upper limit parameter specifies the maximum value, while the Sample time parameter determines the time interval between samples. If the Sample time is set to -1, it means that the sample time is inherited. The output is an unsigned integer with a bit length of 8, 16, or 32 bits, depending on the minimum number of bits required to represent the upper limit. To calculate the R-R intervals for a period, a counter subsystem is utilized. The ECG signals are inputted to the up counter and compared with a constant value of 2 using a relational operator. The pulse generator generates pulses until the value of 2 is reached, and these pulses are then routed to the multiport switch. Subsequently, the up counter is employed again, with the constant value of 2 determined, and the signal is evaluated over a specific time period. The output value is set to 1 if the given signal indicates the presence of stress, and a value of 0 is displayed if stress is not detected.

Psychologists and other mental health experts participated in the data labeling process and used clinical evaluations and standardized tests to measure stress levels and emotions. Participants also self-reported their feelings using scales and questionnaires that had undergone validation, which added transparency to our methodology. We described the precise procedures and standards utilized for data labeling in order to guarantee consistency and dependability. These improvements are intended to address the crucial data collecting issue and increase the precision and authority of our study findings.

Table 7 provides the specific information regarding Tachycardia and Bradycardia. The ECG signal corresponding to a normal heart rate is denoted by the value 101. The structure of human heart rate (Fig. 6) and the characterization of short-term (ST, approximately 5 minutes) or brief, as well as ultra-short-term (UST, less than 5 minutes) durations, are described using measurements in the time domain, frequency domain, and non-linear domain. It is important to emphasize that the values obtained for short-term and ultra-short-term analysis cannot be used interchangeably with the values derived from 25-hour measurements.

Table 7

Human heart rate [34]

Name	Symptoms	Observation
Bradycardia	Patient have heart rates in the normal sinus range. A heart rate of 60–70 min is too low for a patient.	These conditions hear rates are inappropriate or insufficient for humans.
Tachycardia	Typically, the resting heart rate for adults ranges between 60 and 100 beats per minute. It is worth noting that some doctors may consider a healthy limit of 90 beats per minute, so they might diagnose tachycardia at a slightly lower threshold than 100 beats per minute.	The determination of a normal heart rate (pulse) threshold typically takes into account the individual’s age.
Common Heart Function	Normal Heart Rate	In the majority of individuals, a heart rate ranging from 60 to 100 beats per minute during periods of rest is regarded as within the normal range.

Table 8

Performance analysis for classification methods based on accuracy

2*Expression	Accuracy (%)
	LVQ	CART	BPN
Happy	81	84	94
Sad	84	82	93
Fea	82	85	94
Surprise	85	84	91
Anger	84	82	92

Figure 7, output generated signal is obtained as output from the neural network for the given input ECG signal 1. The output generated by ECG signal 2 is given as the input to the neural network. It illustrates changes in heart rate over the recording duration, capturing variations in response to diverse activities, events, or physiological shifts. For example, during rest or relaxation, the heart rate tends to display a relatively stable pattern with slight fluctuations. In contrast, physical activity or stress can result in an elevated heart rate, leading to more pronounced oscillations in the signal.

Fig. 7

Heart rate signal.

Fig. 8

Performance analysis for classification methods based on accuracy.

A Matlab-based open-source modular application for calculating heart rate variability (HRV) is called the PhysioNet Cardiovascular Signal Toolbox [37, 38]. The study’s objective was to create a beat annotation file from.dat files that would be used to classify stress using HRV measurements. The output is an annotation file, and the input is an ECG signal. The annotation file is used to extract RR intervals and their associated timestamps using the PhysioNet HRV toolbox [39]. The dataset was divided into 30-second bins, and the RR intervals were used to extract the HRV features [40]. The second component of this work aimed to develop BPN models that could classify stress using HRV features taken from HRV measurements gathered from the kit.

The performance of several classification techniques utilized for different emotions based on recall is shown in Table 9 and Fig. 9. According to a comparison of classification methods, BPN outperforms all others with the precision of 90 percent, 92 percent, 93 percent, 92 percent, and 92 percent for each emotion.

Table 9

Performance analysis for classification methods based on precision

2*Expression	Precision (%)
	LVQ	CART	BPN
Happy	77	83	90
Sad	76	82	92
Fea	75	84	93
Surprise	75	82	92
Anger	74	83	92

Fig. 9

Performance analysis for classification methods based on precision.

The overall recall performance of classification techniques utilized for different emotions are shown in Table 10 and Fig. 10. In a comparison of classification methods, BPN outperformed all others with recalls of 91 percent, 90 percent, 92 percent, 92 percent, and 91 percent for each emotion, respectively.

Table 10

Performance analysis for classification methods based on recall

2*Expression	Recall (%)
	LVQ	CART	BPN
Happy	81	87	91
Sad	84	88	90
Fea	85	86	92
Surprise	86	87	92
Anger	87	88	91

Fig. 10

Performance analysis for classification methods based on recall.

The overall performance of the F-measure of classification techniques utilized for different emotions is shown in Table 11 and Fig. 11. In a comparison of classification methods, BPN outperformed all others with an F-measure of 90 percent, 92 percent, 91 percent, 92 percent, and 91 percent for each emotion.

Table 11

Performance analysis for classification methods based on F-measure

2*Expression	F-measure
	LVQ	CART	BPN
Happy	78	82	90
Sad	78	84	92
Fea	79	82	91
Surprise	82	81	92
Anger	81	83	91

Fig. 11

Performance analysis for classification methods based on F-measure.

The performance of sensitivity of classification techniques utilized for different emotions are shown in Table 12 and Fig. 12. In a comparison of classification approaches, BPN outperformed all other methods with a sensitivity of 88 percent, 86 percent, 85 percent, 89 percent, and 87 percent for each emotion.

Table 12

Performance analysis for classification methods based on sensitivity

2*Expression	Sensitivity
	LVQ	CART	BPN
Happy	76	81	88
Sad	75	79	86
Fea	72	78	85
Surprise	74	80	89
Anger	71	81	87

Fig. 12

Performance analysis for classification methods based on sensitivity.

The performance of classification techniques utilized for different emotions is shown in Table 13 and Fig. 13. In a comparison of classification methods, BPN outperformed all others with a specificity of 89 percent, 88 percent, 87 percent, 89 percent, and 88 percent for each emotion.

Table 13

Performance analysis for classification methods based on specificity

2*Expression	Specificity
	LVQ	CART	BPN
Happy	72	78	89
Sad	72	79	88
Fea	75	75	87
Surprise	73	76	89
Anger	74	78	88

Fig. 13

Performance analysis for classification methods based on specificity.

Table 14

Performance analysis of stress and non-stress by proposed model

Test	Accuracy	Precision	Recall	F Measures	Sensitivity	Specificity
Stress	93.2	92.8	90.2	91.7	87.2	88.7
Not Stress	94.1	94.0	92.1	93.9	88.4	90.6

To determine the effectiveness of the viewpoint, test information is used. It can make it appear as though the person is under stress or normal. In Table 14, studies are conducted on the identification of stress of patients using the collected database who neural network training using a back propagation program in MatLab.

The study aimed to assess the effectiveness of three classification algorithms (LVQ, BPN, and CART) in detecting stress through emotion recognition. The focus was on examining the feasibility of utilizing correlation features of ECG signals for emotion recognition and comparing them with traditional time/frequency domain features. The data was divided into 80% for training and the remaining data for sentiment classification experiments. The classification algorithms were applied using different feature sets: S1, S2, and S1 and S2. The BPN algorithm exhibited the highest accuracy in emotional recognition. Compared to LVQ, it achieved an accuracy of approximately 5.9% to 8.5% higher, and in the case of CART, the increase was around 2% to 6.5%. However, it should be noted that the recognition accuracy did not surpass 95%. The maximum accuracy achieved was 82.35% for LVQ with 80% training data, while BPN achieved 97.77%. When comparing different feature sets, using only the 72 feature sets resulted in the highest accuracy. It out-performed the S1 case by approximately 17.9% to 20.5% and exceeded the accuracy of using both S1 and S2 feature sets by 11% to 12.7%. Consequently, it can be concluded that correlation features of ECG signals applied to emotion recognition yield better performance than traditional time domain and frequency domain features. The use of correlation features can increase the recognition rate by 25%.

4 Conclusions

This work focuses on validating the ability to classify heartbeats using machine learning algorithms and features related to physiological signals, with participants within the age range from 23 to 40, mixed gender, and normal health status, along with other pertinent demographics included. The classification process involves utilizing a neural network in a single node, aiming at heartbeat counting and stress level monitoring for diagnosis. The study emphasizes the importance of identifying the relationships between physiological signal features and stress through neural models. The findings indicate that the stress estimation algorithm based on evaluating short time series of heartbeat intervals achieves promising results. The classification accuracy using machine learning algorithms, such as LVQ, BPN, and CART, ranges from 82.35% to 97.77%. The utilization of 72 feature sets yields the highest accuracy, surpassing the S1 case by 17.9% – 20.5% and the combined S1 and S2 sets by 11% – 12.7%. These results demonstrate the efficacy of the proposed approach in accurately classifying heart-beats and estimating stress levels. The stress estimation algorithm shows promise for mobile telemedical applications, with high classification accuracy and the potential to enhance mental well-being. Future extensions could involve the analysis of additional biomedical signals, such as EMG and EEG, using neural network techniques, further advancing our understanding of physiological signals in relation to stress.

References

Chandola

, Brunner

and Marmot

, Chronic stress at work and the metabolic syndrome: prospective study, BMJ 332 (2006), 521–525.

Kumar

, Neubert

, Behrendt

, Rieger

, Weippert

, Stoll

, Thurow

and Stoll

, Stress monitoring based on stochastic fuzzy analysis of heartbeat intervals, IEEE Transactions on Fuzzy Systems 20 (2012), 746–759.

Montano

, Porta

, Cogliati

, Costantino

, Tobal-Dini

, Casali

K.R.

and Iellamo

, Heart rate variability explored in the frequency domain: a tool to investigate the link between heart and behavior, Neuroscience & Biobehavioral Reviews 33 (2009), 71–80.

Hassan

M.A.

, Malik

A.S.

, Fofi

, Saad

, Karasfi

, Ali

Y.S.

and Meriaudeau

, Heart rate estimation using facial video: A review, Biomedical Signal Processing and Control 38 (2017), 346–360.

Gudi

, Bittner

and van Gemert

, Real-time webcam heart-rate and variability estimation with clean ground truth for evaluation, Applied Sciences 10 (2020), 8630.

Nikolova

, Petkova

, Manolova

and Georgieva

, ECG-based emotion recognition: Overview of methods and applications, ANNA’18; Advances in Neural Networks and Applications 2018 (2018), 1–5.

Hasnul

M.A.

, Aziz

N.A.A.

, Alelyani

, Mohana

and Aziz

A.A.

, Electrocardiogram-based emotion recognition systems and their applications in healthcare—a review, Sensors 21 (2021), 5015.

Zhang

, Wang

, Li

, Chen

and Shao

, Research on emotion recognition based on ECG signal, In Proceedings of the Journal of Physics: Conference Series. IOP Publishing 1678 (2020), 012091.

Rammohan

R.A.

, Medikonda

and Pothiyil

D.I.

, Speech signal-based modelling of basic emotions to analyse compound emotion: Anxiety. In Proceedings of the 2020 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics (DISCOVER). IEEE, 2020, pp. 218–223.

10.

Yang

C.J.

, Fahier

, Li

W.C.

and Fang

W.C.

, A convolution neural network based emotion recognition system using multimodal physiological signals. In Proceedings of the 2020 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan). IEEE, 2020, pp. 1–2.

11.

Hayano

, Kisohara

, Ueda

and Yuda

, Impact of heart rate fragmentation on the assessment of heart rate variability, Applied Sciences 10 (2020), 3314.

12.

McCraty

and Shaffer

, Heart rate variability: new perspectives on physiological mechanisms, assessment of self-regulatory capacity, and health risk, Global Advances in Health and Medicine 4 (2015), 46–61.

13.

Quintana

D.S.

, Elstad

, Kaufmann

, Brandt

C.L.

, Haatveit

, Haram

, Nerhus

, Westlye

L.T.

and Andreassen

O.A.

, Resting-state high-frequency heart rate variability is related to respiratory frequency in individuals with severe mental illness but not healthy controls, Scientific Reports 6 (2016), 1–8.

14.

Shaffer

and Ginsberg

J.P.

, An overview of heart rate variability metrics and norms, Frontiers in Public Health 2017, p. 258.

15.

Kreibig

S.D.

, Samson

A.C.

and Gross

J.J.

, The psychophysiology of mixed emotional states, Psychophysiology 50 (2013), 799–811.

16.

Lombardi

and Stein

P.K.

, Origin of heart rate variability and turbulence: an appraisal of autonomic modulation of cardio-vascular function, Frontiers in Physiology 2 (2011), 95.

17.

Xiefeng

, Wang

, Dai

, Zhao

and Liu

, Heart sound signals can be used for emotion recognition, Scientific Reports 9 (2019), 6486.

18.

Sepúlveda

, Castillo

, Palma

and Rodriguez-Fernandez

, Emotion recognition from ECG signals using wavelet scattering and machine learning, Applied Sciences 11 (2021), 4945.

19.

Selvaraj

, Murugappan

, Wan

and Yaacob

, Classification of emotional states from electrocardiogram signals: a non-linear approach based on hurst, Biomedical Engineering Online 12 (2013), 1–18.

20.

Khan

A.M.

and Lawo

, Developing a system for recognizing the emotional states using physiological devices. In Proceedings of the 2016 12th International Conference on Intelligent Environments (IE). IEEE, 2016, pp. 48–53.

21.

Larradet

, Niewiadomski

, Barresi

, Caldwell

D.G.

and Mattos

L.S.

, Toward emotion recognition from physiological signals in the wild: approaching the methodological issues in real-life data collection, Frontiers in Psychology 11 (2020), 1111.

22.

Azucena

, Ríos

, Peña

and Díaz

, Design and implementation of a simple portable biomedical electronic device to diagnose cardiac arrhythmias, Sensing and Bio-Sensing Research 4 (2015), 1–10.

23.

Nguyen

T.H.

and Chung

W.Y.

, Detection of driver braking intention using EEG signals during simulated driving, Sensors 19 (2019), 2863.

24.

Fernández

A.P.

, Leenders

, Aerts

J.M.

and Berckmans

, Emotional states versus mental heart rate component monitored via wearables, Applied Sciences 13 (2023), 807.

25.

Weibel

R.P.

, Kerr

J.I.

, Naegelin

, Ferrario

, Schinazi

V.R.

, La Marca

, Hoelscher

, Nater

U.M.

and von Wangenheim

, Virtual reality-supported biofeedback for stress management: Beneficial effects on heart rate variability and user experience, Computers in Human Behavior 141 (2023), 107607.

26.

Sureja

, Mehta

, Shah

and Patel

, Machine Learning in Wearable Healthcare Devices. In Machine Learning for Advanced Functional Materials; Springer, 2023; pp. 281–303.

27.

Luo

, Kramer

, de Kort

, Rense

, Adam

and Lichtenbelt

W. van Marken

, Personal comfort systems and cognitive performance: Effects on subjective measures, cognitive performance, and heart rate measures, Energy and Buildings 278 (2023), 112617.

28.

Chen

, Li

, Cao

, Sun

and He

, Removal of power line interference from ECG signals using adaptive notch filters of sharp resolution, IEEE Access 7 (2019), 150667–150676.

29.

Bansal

, Design of 50Hz notch filter circuits for better detection of online ECG, International Journal of Biomedical Engineering and Technology 13 (2013), 30–48.

30.

Mohammadi

, Fakharzadeh

and Baraeinejad

, An integrated human stress detection sensor using supervised algorithms, IEEE Sensors Journal 22 (2022), 8216–8223.

31.

El-Khafif

S.H.

and El-Brawany

M.A.

, Artificial neural network-based automated ECG signal classifier, International Scholarly Research Notices 2013, 2013.

32.

Adriati

S.F.

, Setiawidayat

and Rofii

, Identification Of ECG signal by using backpropagation neural network, In Proceedings of the Journal of Physics: Conference Series. IOP Publishing 1908 (2021), 012014.

33.

Nainwal

, Kumar

and Jha

, Comparative study of LVQ and BPN ECG classifier, International Journal of Computational Systems Engineering 4 (2018), 140–145.

34.

Zalabarria

, Irigoyen

, Martinez

, Larrea

and Salazar-Ramirez

, A low-cost, portable solution for stress and relaxation estimation based on a real-time fuzzy algorithm, IEEE Access 8 (2020), 74118–74128.

35.

Sloan

, Shapiro

, Bagiella

, Boni

, Paik

, Bigger

Jr , Steinman

and Gorman

, Effect of mental stress throughout the day on cardiac autonomic control, Biological Psychology 37 (1994), 89–99.

36.

Kim

H.G.

, Cheon

E.J.

, Bai

D.S.

, Lee

Y.H.

and Koo

B.H.

, Stress and heart rate variability: a meta-analysis and review of the literature, Psychiatry Investigation 15 (2018), 235.

37.

Vollmer

, HRVTool– an open-source matlab toolbox for analyzing heart rate variability. In Proceedings of the 2019 Computing in Cardiology (CinC). IEEE, 2019 pp. Page–1.

38.

Vest

A.N.

, Da Poian

, Li

, Liu

, Nemati

, Shah

A.J.

and Clifford

G.D.

, An open source benchmarked toolbox for cardiovascular waveform and interval analysis, Physiological Measurement 39 (2018), 105004.

39.

Goldberger

A.L.

, Amaral

L.A.

, Glass

, Hausdorff

J.M.

, Ivanov

P.C.

, Mark

R.G.

, Mietus

J.E.

, Moody

G.B.

, Peng

C.K.

and Stanley

H.E.

, PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals, Circulation 101 (2000), e215–e220.

40.

Vest

A.N.

, Li

, Liu

, Nemati

, Shah

and Clifford

G.D.

, Benchmarking heart rate variability toolboxes, Journal of Electrocardiology 50 (2017), 744–747.

An intelligent wearable embedded architecture for stress detection and psychological behavior monitoring using heart rate variability

Abstract

Keywords

1 Introduction

1.1 Literature review for emotion recognition from physiological signals with emphasis in ECG

2 Wearable embedded architecture

2.4 Emotion recognition by ANN

3.1 Hardware set up of hear rate monitoring device

Table 5 Hardware components Device Description Arduino Board ECG Sensor Wires UNO R3 Development board AD8232 Breadboard Jumper wires Experimental

References

Table 5
Hardware components

Device Description

Arduino Board ECG Sensor Wires UNO R3 Development board AD8232

Breadboard Jumper wires

Experimental