Linked adaptive neuro-fuzzy inference system for biosignal distortion detection system

Abstract

This paper proposes a biosignal distortion detection algorithm for a driver healthcare system based on a contact biosensor and a linked adaptive neuro-fuzzy inference system (ANFIS), and demonstrate its superiority using actual vehicle experiments. Contact biosensors are highly sensitive to vehicle vibration and turning. Although vehicle suspension contributes significantly to ride quality, vibration transfers to the driver and contact between the driver and biosensor can become unstable when executing a turn, causing the driver’s biosignal to not be measured well. This study estimated the driver’s biosignal state using acceleration, angular velocity, and slip ratio measurements obtained from sensor fusion. When the measurement exceeded a defined threshold, the driver healthcare system removed unreliable biosignal data. We adopted ANFIS to improve the proposed sensor fusion algorithm estimate accuracy for the driver’s biosignal state and improved the healthcare system robustness to road conditions. The effectiveness of the proposed algorithm was demonstrated experimentally by comparing the system using sensor fusion and linked ANFIS.

Keywords

Linked adaptive neuro fuzzy inference system biosignal distortion detection driver healthcare system sensor fusion

1 Introduction

As cars are becoming more intelligent, drivers can utilize many new information services, one of which is the driver healthcare system. Personal identification and verification can be useful for these new services and biometrics provide an automatic method to recognize a person based on intrinsic physical or behavioral features. For example, biometrics based on electrocardiogram (ECG) waveforms [1 –3].

The driver healthcare system is based on a contact biosensor that is highly sensitive to vehicle vibration and turning. Although suspension contributes significantly to ride quality when isolated from road noise, vehicle vibration transfers to the driver, and contact between the driver and biosensor can become unstable when executing a turn. Hence the driver’s biosignal may not be well measured, and this unreliably biosignal data may result in faulty healthcare service. Thus, we need to detect driver biosignal distortion in real time and remove unreliably biosignal data. Current approaches for biosignal distortion detection measure left and right wheel speeds and compares the difference to predetermined reference. However, this method poor accuracy and environmental limits, particularly speed difference with time delays and differences caused by vibrations. Therefore, current methods are vulnerable to changing road conditions and biosignal distortion could become significant. This paper proposes a linked adaptive neuro fuzzy inference system (ANFIS) [4–6 , 11–14], which is quite distinct from previous approaches. We employed an inertial measurement unit (IMU) sensor to consider road conditions, derived the slip ratio from wheel speed sensors and vehicle speed sensors to detect changes in friction between the tires and road, and used a steering angle sensor (SAS) to detect vehicle turning. The driver’s biosignal state is estimated using the obtained acceleration, angular velocity, and slip ratio measurements along with fused IMU, SAS, wheel speed sensor, vehicle speed sensor data. The proposed algorithm incorporated the linked ANFIS system to accurately estimate the driver’s biosignal state, improving the health care system robustness to road conditions. We verified the proposed method effectivenessexperimentally.

The remainder of this paper is organized as follows. Section 2 explains the contact sensor biosignal detection system, and Section 3 discusses the distortion detection system using linked ANFIS. We also show experimental results and demonstrate the algorithm performance. Section 4 summarizes the outcomes and concludes the paper.

2 Biosignal detection based on a contact sensor

This section defines a contact sensor for the driver healthcare system and discusses the main sources of disturbance (vehicle shaking and turning). Figure 1, shows that the driver healthcare system comprises three parts.

ECG and IMU sensors. ECG sensors are mounted in the steering wheel and the IMU sensor is mounted in the center console. We acquire the driver’s biosignal via the ECG sensors and vehicle movement via the IMU sensor.

The controller area network (CAN) bus allows a host computer to communicate with devices, such as the ECG and IMU sensors, and other vehicle sensors (steering angle, wheel speed, and vehicle speed), to acquire synchronized sensor data.

The healthcare service application resides in the host computer. The application analyzes driver ECG data to produces driver’s emotionalquotient (EQ), stress level, and heart rate.

Fig.1

Proposed driver healthcare system.

Thus, driver monitoring is possible and drivers can be notified by suitable alarm(s) in cases of emergency.

Vehicles vibrated depending on the road conditions, and grip is reduced when turning. Grip force can be calculated from the vehicle slip rate, which is inversely proportional to the grip force. The IMU provide information regarding the vehicle’s state: position, velocity, and attitude, and provides two main advantages [7 –9]:

object position and velocity can be evaluated in any timeframe, and

readings are not affected by wheel slip.

Figures 2 and 3 shows typical vehicle driving experiments and corresponding IMU sensor values [10], respectively. The vehicle was a Hyundai Genesis DH, and the experiments included 4 road conditions: paved, bumpy, unpaved, and cobblestone roads. When the vehicle was turning, ECG sensor contact (on the steering wheel) became unstable, causing biosignal distortion. Figure 4 shows the typical SAS results for turning the vehicle.

Fig.2

Vehicle and typical paved, bumpy, unpaved, and cobblestone road surfaces employed for the experiments.

Fig.3

Biosignal distortion according to vehicle shaking.

Fig.4

Biosignal distortion according to vehicle turning.

Sensors for wheel speed, vehicle speed, and steering angle were already embedded in the vehicle, and we added an in-house developed IMU sensor, as shown in Fig. 5, integrating STM32F407 (Micro controller unit: MCU) and MPU6050 (IMU). Communication from the MCU and IMU was via an integrated circuit (I2C). Sampling rate for the vehicle speed sensor, SAS, and IMU were all 100 Hz, whereas wheel speed and ECG sample rates were 50 and 333 Hz, respectively.

Fig.5

In-house developed IMU sensor board combining (a) STM32F407 for the MCU and (b) MPU6050 for the IMU.

The IMU sensor board provided biosignals by communicating through the CAN bus. Figure 6 shows the sensor modules applied to the vehicle and corresponding typical sensor values.

Fig.6

(a) In-house developed IMU sensor module and (b) typical corresponding sensor values.

3 Distortion detection

3.1 Distortion detection using sensor fusion

Figure 7 shows the proposed contact sensor based driver healthcare system. Slip ration was calculated from wheel and vehicle speed, IMU sensor measurements were compensated based on road conditions, and a threshold value was applied to the biosignal corresponding to the vehicle speed and acceleration to determine if the obtained biosignal wasdistorted.

Fig.7

Biosignal distortion detection using sensor fusion.

The SAS detects vehicle turning and slip ratio detects friction changes between the wheels and road. The slip ratio can be expressed as $Slip = \frac{| R_{W -} V_{X} |}{R_{W}} \times 100 (%)$ (1) when accelerating, and $Slip = \frac{| V_{X -} R_{W} |}{V_{X}} \times 100 (%)$ (2) when braking, where R_W and V_X are obtained from the wheel and vehicle speed sensors, respectively.

The IMU sensor measures vehicle acceleration (lateral/longitudinal/vertical) and angular rate (roll/pitch/yaw) using a combination of accelerometers and gyroscopes. Accelerometers detect the magnitude and direction of the force applied to the sensor (including gravity) as a vector, and gyroscopes measure the rate of rotation around a particular axis.

The proposed biosignal distortion algorithm used vertical acceleration, roll, and pitch to detect vehicle vibrations due to road conditions. Vertical acceleration reflects the vertical behavior of the vehicle when passing over speed bumps or off-road. Pitch represents rotation around the side-to-side axis and roll represents rotation around the front-to-rear axis of the car.

Figure 8 shows pitch and roll data corrections considering road conditions. The corrected gyro output, w_i can be derived as $W_{i} = W_{m} + I_{I}$ (3)

Fig.8

Pitch and roll correction based on road conditions.

$I_{i} = I_{(i - 1)} + Sign (W_{(i - 1)} - Th .) i_{c},$ (4)

$\begin{matrix} I_{i} & = & I_{(i - 1)} + i_{c} for (W_{(i - 1)} > Th . \\ = & I_{(i - 1)} - i_{c} for W_{(i - 1)} < Th . \end{matrix}$ (5) where I_i is the correction value; w_m is the raw gyro output; Th is the threshold value for biosignal distortion; i_c is a fixed increment; and SIGN() is a sign function, $Sign (X) = \begin{matrix} (1, & X > 0 \\ (0, & X = 0 \\ (- 1, & X < 0 \end{matrix}$ (6)

Figure 9 shows the corrected pitch and roll data, decreased fault rates for paved road and improved distortion detection rates for unpaved road conditions.

Fig.9

Pitch and roll correction affect.

The biosignal state can be estimated in real time using combined sensor data (IMU, SAS, wheel speed sensor, and vehicle speed sensor). Each data set (vertical acceleration, pitch, roll, steering angle, and slip ratio) has individual threshold values. When data exceeds the threshold value, the driver’s biosignal is distorted due to vehicle shaking or turning. Distortion detection performance of varies depending on the driving environment. Speed and acceleration are the main detection factors. To obtain more reasonable values and good detection performance, each sensor threshold was optimized using simulation based on experimental data, calculated as $Th . = Fixed Th . + ABS (Speed - 30) \times s_{c},$ (7) $Th . = Fixed Th . + Accel pedal \times a_{c},$ (8) ${Th}_{slip} = {Fixed Th}_{slip} + Accel pedal \times a_{c},$ (9) ${Th}_{pitch} = {Fixed Th}_{pitch} + Accel pedal \times a_{c},$ (10) ${Th}_{roll} = {Fixed Th}_{roll} + Accel pedal \times a_{c},$ (11) and

$\begin{matrix} {Th}_{vertical} = {Th}_{vertical} + ABS (speed - 30) \times \\ s_{c} + Accel pedal \times a_{c}, \end{matrix}$ (12) where s_c = 0 and a_c = 5 are adjustment constants for vehicle speed and acceleration, respectively.

We employed 10 datasets to detect biosignal distortion to verify the proposed sensor fusion method. Figure 10 shows fused sensor outputs for shaking and turning cases, and Table 1 shows the degree of accuracy (DOA) using Equation (13) for the 10 datasets, $DOA = \frac{\frac{a}{b} + \frac{c}{d}}{2} (%),$ (13) where, a and b are the number of successful distortion detections and number of distortions, respectively; and c and d are the number of successful normal detections and number of normal cases, respectively.

Fig.10

Sensor fusion outputs for vehicle shaking and turning.

Table 1

Proposed sensor fusion and previous general methods

Dataset	Road feature	Accuracy (%)
		General	Sensor fusion
0	pavement+speed bump	79.927	84.692
1	pavement+speed bump	82.102	85.100
2	pavement+speed bump	78.927	84.914
3	off-road	59.199	66.243
4	off-road	58.944	71.538
5	off-road	61.272	84.573
6	off-road	61.265	92.907
7	off-road	61.289	86.054
8	off-road	61.065	87.938
9	off-road	56.950	82.585
Average		66.094	82.654

A total of 188,000 data points were corrected from the various sensors with 21,100 from each of steering angle, wheel speed, vehicle speed, slip ratio, acceleration, vertical acceleration, and CAN bus interface collected at 0.01 s sampling rate. These data were also utilized to train the linked ANFIS system, as shown in Fig. 11.

Fig.11

Experimental datasets for different road conditions as detailed in Table 1.

3.2 Linked ANFIS system design and experimental results

As discussed in Section 2, and shown in Table 1, biosignal distortion detection based on a contact-type bio sensor is general method The contact biosensor is highly sensitive to vehicle vibration and turning, resulting in impaired accuracy. The sensor fusion method also exhibited severely impaired accuracy for different driving environments, because optimized thresholds depended on the experimental data used in the optimization. Therefore, we propose ANFIS to improve driver healthcare system robustness to driving environments. Figure 12 shows that the ANFIS is architecture equivalent to a two-input Sugeno fuzzy model with four rules, where each input is assumed to have two associated membership functions (MFs). Thus, Sugeno fuzzy inference rules were applied, and gradient descent and recursive least-squares estimation algorithms were employed to iteratively adjust the initial and subsequent ANFISparameters.

Fig.12

ANFIS architecture equivalent to the Sugeno fuzzymodel.

The network architecture can be summarized by considering individual layer functionality.

Layer 1: fuzzification

Each node in this layer generates membership grades with a linguistic value. For example, the ith node function is bell shaped, i.e., $μ_{A_{i}} (x) = \frac{1}{1 + {| \frac{x - c}{a} |}^{2 b}},$ (14) where x is the input to node i, A_i is the linguistic value associated with this node function, a and c are the MFs’ width and center, and b is generally positive.

Layer 2: rules

Every node in this layer multiplies the incoming signals and forwards the out, e.g. $w_{j} = μ_{A_{i}} (x_{1}) \times μ_{B_{i}} (x_{2}),$ (15) where i= 1, 2; and j = 1, 2, 3, 4. Each node output represents the firing strength of a rule.

Layer 3: normalization

The jth node of this layer calculates the ratio of the jth rule firing strength to the sum of the firing strengths for all rules, i.e., ${\bar{w}}_{j} = \frac{w_{j}}{w_{1} + w_{2} + w_{3} + w_{4}},$ (16)

Layer 4: defuzzification

Node j in this layer has node function $z_{j} = {\bar{w}}_{j} f_{j} = {\bar{w}}_{j} (c_{j 0} + c_{j 1} x_{1} + c_{j 2} x_{2}),$ (17)

Layer 5: output

The output from this layer has node function $y = \sum_{j} {\bar{w}}_{j} f_{j} = \frac{\sum_{j} w_{j} f_{j} \sum w_{jj}}{\sum_{j} w_{j}},$ (18)

Input selection is very important in the linked ANFIS design to increase model performance. We considered every individual input accuracy for the linked ANFIS, as shown in Fig. 13. The best distortion detection performance resulted from using slip ratio and acceleration input variables, whereas input speed exhibited the poorest performance. Since speed is a very important factor for calculating sensor thresholds, it should be considered a dominant input for ANFIS. Figure 14 shows the Linked ANFIS degree of accuracy from two inputs, where input speed was very important for several inputs, but the corresponding ANFIS accuracy was very poor. Similarly, ANFIS accuracy using speed coupled inputs was also generally poorer. However, accuracy improved when vertical acceleration and speed variables were included.

Fig.13

ANFIS accuracy using a single input variable.

Fig.14

ANFIS accuracy using two inputs.

Thus, we defined vertical acceleration and speed as the two main input variables for the first ANFIS model, and other input variables (acceleration pedal, slip ratio, steering angle, and steering angular velocity) were considered as the main input variables for the 2nd ANFIS model. Each input had two bell shaped membership functions. Figure 15 shows the final linked ANFIS model structure.

Fig.15

Final linked ANFIS model structure.

Tables 2 and 3 show the premise and consequent parameters of tuned linked ANFIS, respectively. Figure 16 shows the before and after learning MF status for mf_2,1 and mf_2,2 from Table 2 (Epoch 3 and 150, respectively).

Fig.16

Membership functions before and after learning.

Table 2

Premise parameters of the Linked ANFIS Model

MF	a	b	c
mf _1,1	– 3.31e-05	1.985	209.5
mf _1,2	418.9	1.958	209.5
mf _2,1	– 0.1416	1.914	1.464
mf _2,2	2.82	2.019	1.639
mf _3,1	0.0001447	2.008	450
mf _3,2	900	1.958	450
mf _4,1	– 0.004153	1.97	16.99
mf _4,2	33.97	1.968	17.01
mf _5,1	2.353e-05	1.988	195.2
mf _5,2	390.4	1.977	195.2

Table 3

Consequent parameters of the linked ANFIS

f _j	c ₀	c ₁	c ₂	c ₃	c ₄	c ₅
f ₁	20.24	1.207	18.2	0.1867	1.994	0.345
f ₂	– 77.9	– 2.105	– 115.1	– 3.423	76.86	1.728
f ₃	30.56	– 18.34	– 177.7	– 1.272	– 9.991	– 3.79
f ₄	– 53.72	27.36	1418	51.42	311.6	– 10.33
f ₅	– 281.9	– 18.43	– 121.5	– 5.028	– 74.63	– 4.904
f ₆	– 735	34.89	– 757.5	18.81	– 332.9	– 7.092
f ₇	– 534	290.9	87.98	71.19	– 0.6506	56.41
f ₈	– 80.75	– 517.3	38.89	– 32.06	– 2185	– 108.2
f ₉	– 79.52	– 2.707	22.13	0.05995	– 24.69	0.3796
f ₁₀	962.1	6.527	– 555.5	0.6659	156.2	0.6774
f ₁₁	112.1	41.11	– 259.2	– 4.541	5.796	– 10.86
f ₁₂	51.44	– 89.77	315.7	4.355	– 28.34	– 53.87
f ₁₃	1294	35.45	– 327.5	10.11	234.8	– 0.02049
f ₁₄	– 244.1	– 1.822	– 628.8	– 21.8	– 589.2	40.06
f ₁₅	– 80.34	– 556.4	879.9	– 144.2	– 56.02	67.93
f ₁₆	– 26.32	– 86.28	44.48	– 157.4	– 186.6	– 92.22
f ₁₇	81.46	– 3.131	31.16	– 0.9342	– 51.02	– 2.177
f ₁₈	145.9	– 18.24	– 51.76	41.63	– 829.1	– 20.68
f ₁₉	– 77.53	54.17	1120	15.23	15.51	32.49
f ₂₀	2.197	278.3	88.16	– 678.7	– 2291	330.2
f ₂₁	51.21	55.21	383.5	67.48	1440	39.93
f ₂₂	246.9	13.94	269.9	– 5.483	12.89	5.306
f ₂₃	– 38.46	– 920.4	– 43.56	– 1078	1856	– 611.2
f ₂₄	12.24	– 72.05	12.63	– 161.3	– 186.4	– 11.82
f ₂₅	– 1212	7.288	797.5	– 3.981	316.4	– 7.692
f ₂₆	– 63.66	– 9.008	– 228.2	0.07426	– 1461	– 11.43
f ₂₇	– 8.634	– 125.3	2033	63.52	– 83.84	130.2
f ₂₈	0.1932	184.3	146.3	8.719	130	198
f ₂₉	– 315.2	– 54.43	– 653.2	– 151.2	– 2963	– 1.863
f ₃₀	63.67	– 26.33	– 21.52	26.14	– 326.5	– 13.06
f ₃₁	– 20.59	940.5	186.2	2413	1255	– 53.99
f ₃₂	3.809	66.14	15.46	132.2	76.18	5.774

As discussed in Section 3, 10 experimental datasets were used to test biosignal distortion detection. We included the IMU sensor with 100 Hz sampling rate; the ECG biosignal was sampled at 333 Hz; and steering angle, wheel speed, and vehicle speed data were acquired via the CAN bus interface. We used CANoe software (Vector Inc.) acquire the synchronized sensor data. Experiments were performed over paved roads with speed bumps and off-road at 0–90 km/h. Datasets 0 and 5 were used for training with bell shaped membership functions.

Table 4 and Fig. 17 show linked ANFIS outputs for various road conditions and driver status. Table 4 includes DOA for the datasets, and average linked ANFIS detection accuracy is compared with general and sensor methods in Fig. 16.

Fig.17

Overall method accuracies.

Table 4

Linked ANFIS and sensor fusion method accuracies

Dataset	Road feature	Accuracy (%)
		Sensor fusion	Linked ANFIS
0	pavement+speed bump	84.692	86.910
1	pavement+speed bump	85.100	89.631
2	pavement+speed bump	84.914	88.501
3	off-road	66.243	76.375
4	off-road	71.538	76.719
5	off-road	84.573	88.651
6	off-road	92.907	88.557
7	off-road	86.054	86.477
8	off-road	87.938	87.800
9	off-road	82.585	85.089
Average		82.654	85.471

Using sensor fusion, the Accuracy for dataset 3 using sensor fusion = 66.243%, whereas accuracy for dataset 6 = 92.907%, but the large accuracy variation also strongly depended on the choice of training datasets. In contrast, the proposed linked ANFIS improved deviation accuracy for all datasets without requiring training datasets, i.e., performance of the proposed method was guaranteed for differing driving environments.

4 Conclusions

This paper proposed a biosignal distortion detection algorithm using sensor fusion and linked ANFIS. We employed vehicle speed and vertical acceleration inputs for the first ANFIS; with slip ration, steering angle, steering angular velocity and acceleration pedal as inputs for the second ANFIS. Two bell shaped MFs were used for each input variable.

The general and sensor fused methods were significantly sensitive to vehicle vibration and movement, and road conditions, causing detection accuracy variances. In contrast, the proposed method improved distorted driver biosignal detection accuracy and increased the healthcare system robustness to differing driving environments.

We verified the proposed method provided superior performance than the general or sensor fusion approaches with experimental data.

Future work will consider more extensive experiments with a wider range of conditions to evaluate the proposed linked ANFIS model.

Footnotes

Acknowledgments

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education (grants NRF-2016R1D1A1B01016071 and NRF-2017R1D1A1B03031467).

References

Chan

A.D.C.

, Mohyeldin

, Hamdy

A.B.

and BadeeWavelet

, Distance Measure for Person Identification Using Electrocardiograms, IEEE Transactions on Instrumentation and Measurement 57(2) (2008), 248–253.

Pan

and Tompkins

W.J.

, A Real-Time QRS Detection Algorithm, IEEE Transactions on Biomedical Engineering 32(3) (1985), 230–236.

Elgendi

, Jonkman

and DeBoer

, Frequency, Bands Effects on QRS Detection, Proceedings Springer-Verlag, Biomedical Engineering Systems and Technologies, Spain (2010), 428–431.

Jang

J-S.R.

, Adaptive-Network-Based Fuzzy Inference System, IEEE Transactions on Systems, Man, and Cybernetics 23(3) (1993), 665–685.

Guler

İ.

and Ubeyli

E.D.

, Adaptive neuro-fuzzy inference system for classification of EEG signals using wavelet coefficients, Journal of Neuroscience Methods 148(2) (2005), 113–121.

Khezri

, Jahed

and Neuro-Fuzzy

, Inference System for sEMG-Based Identification of Hand Motion Commands, IEEE Transactions on Industrial Electronics 58(5) (2011), 1952–1960.

Saadeddin

, American University of Sharjah, Estimating vehicle state by GPS/IMU fusion with vehicle dynamics, Master’s thesis of Science in Mechatronics Engineering, 2013.

Spry

S.C.

and Girard

A.R.

, Gyroscopic Stabilization of Unstable Vehicles: Configurations, Dynamics, and Control, International Journal of Vehicle Mechanics and Mobility 46(1) (2008), 247–260.

Saadeddin

, Abdel-Hafez

M.F.

, Jarrah

M.A.

, Estimating. Vehicle State by GPS/IMU Fusion with Vehicle Dynamics, Journal of Intelligent and Robotic Systems 47(1) (2014), 147–172.

10.

Kavianipour

, Montazeri-Gh

and Moazamizadeh

, Road profile measurement using the two degrees of freedom response-type mechanism, Journal of Mechanical Engineering Science 229(6) (2015), 1074–1087.

11.

Balakishan

and Roshiah

, Development and Implementation of a Simplified Self-Tuned Neuro-Fuzzy-Based IM Drive, 7(8) (2015), 1446–1451.

12.

Khodayari

, Ghaffari

, Kazemi

, Alimardani

and Braunstingl

, Improved adaptive neuro fuzzy inference system car-following behaviour model based on the driver– vehicle delay, IET Intelligent Transport Systems 8(4) (2014), 323–332.

13.

Zheng

, Ling

, Chen

and Xue

, A Hybrid Neuro-Fuzzy Network Based on Differential Biogeography-Based Optimization for Online Population Classification in Earthquakes, IEEE Transactions on Fuzzy Systems 23(4) (2015), 1070–1083.

14.

Zheng

, Chen

, Xue

and Xue

, A Pythagorean-Type Fuzzy and Deep, Denoising Autoencoder for Industrial Accident Early Warning, IEEE Transactions on Fuzzy Systems 25(6) (2017), 1561–1575.