Controlling of smart home system based on brain-computer interface

Abstract

BACKGROUND:

Brain computer interface (BCI) technology is a communication and control approach. Up to now many studies have attempted to develop an EEG-based BCI system to improve the quality of life of people with severe disabilities, such as amyotrophic lateral sclerosis (ALS), paralysis, brain stroke and so on. The proposed BCIBSHS could help to provide a new way for supporting life of paralyzed people and elderly people.

OBJECTIVE:

The goal of this paper is to explore how to set up a cost-effective and safe-to-use online BCIBSHS to recognize multi-commands and control smart devices based on SSVEP.

METHODS:

The portable EEG acquisition device (Emotiv EPOC) was used to collect EEG signals. The raw signals were denoised by discrete wavelet transform (DWT) method, and then the canonical correlation analysis (CCA) method was used for feature extraction and classification. Another part is the control of smart home devices. The classification results of SSVEP can be translated into commands to control several devices for the smart home.

RESULTS:

Here, the Power over Ethernet (PoE) technology was utilized to provide electrical energy and communication for those devices. During online experiments, four different control commands have been achieved to control four smart home devices (lamp, web camera, guardianship telephone and intelligent blinds). Experimental results showed that the online BCIBSHS obtained 86.88 $\pm$ 5.30% average classification accuracy rate.

CONCLUSION:

The BCI and PoE technology, combined with smart home system, overcoming the shortcomings of traditional systems and achieving home applications management rely on EEG signal. In this paper, we proposed an online steady-state visual evoked potential (SSVEP) based BCI system on controlling several smart home devices.

Keywords

Brain computer interface smart home system Power over Ethernet steady-state visual evoked potential discrete wavelet transform canonical correlation analysis

1. Introduction

Brain computer interface (BCI) technology is a communication and control approach that does not depend on the brain’s normal output channels of peripheral nerves and muscles [1]. Hence, the initial and most important research motivation of BCI is to provide a pathway of communication or control with the outside world for those disabled patients whose brain is not damaged, while central nervous system or motor system is seriously impaired. Moreover, with increasing aged population, a system that can make the home environment more intelligent, assistive and electricity security is also needed for elderly people. Brain activity can be recorded with several noninvasive BCI techniques, such as electroencephalography (EEG), magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). Among these, EEG is the most effective manner because of its portability. Up to now many studies have attempted to develop an EEG-based BCI system to improve the quality of life of people with severe disabilities, such as amyotrophic lateral sclerosis (ALS), paralysis, brain stroke and so on [2, 3, 4, 5].

For decades, several EEG-based typical BCI systems have been proposed, including slow cortical potential (SCP) [6], motor imagery (MI) [7], steady-state visual evoked potential (SSVEP) [8], the P300 potential [9] and hybrid BCI [10]. Each type of these BCI systems has its unique characteristic. Among these typical BCI systems, the SSVEP signal that is a repeated response to visual stimulation at frequencies greater than 6 Hz [11]. Besides frequency, the SSVEP signal is also affected by spacing, shape and color of the stimulus [12, 13, 14]. The SSVEP has many advantages such as less training, higher signal-to-noise ratio (SNR) and higher information transfer rates (ITR) [15, 16]. Therefore, an increasing number of international research groups began to engage in this research and achieved remarkable results [8, 11, 17, 18].

Recently, with the advancement in sensor technology and information technology, the idea of the smart home environment is gradually becoming a reality. Many research teams have achieved several works which use BCIs in the smart home system. However, most research groups focus on using BCI in virtual smart home system [19, 20, 21, 22]. In this case, maybe it is difficult to reconstruct in the real environment [23]. Compared with these, a few studies are trying to use BCI in a real smart home [24, 25, 26]. For example, Carabalona et al. used P300 potential to construct a real smart home system [24]. Kosmyna et al. achieved a realistic smart home system with 77% task accuracy [25]. However, in both case, their BCIBSHS were achieved by g.tec cap. Therefore, most users can not benefit from this work due to expensive and bulky EEG acquisition equipment. And it is difficult to control devices with high classification accuracy. Lin et al. set up a smart home system based on brain-computer interface in UPnP Home Networking [26]. Although their BCIBSHS was effective, an EEG electrode-based the international 10–20 EEG system was used to recognize cognitive state. As such, few smart home applications (light) were controlled by monitoring the theta and alpha rhythms.

Power over Ethernet (PoE) is a technology which based on the Cat.5 wiring infrastructure, only using ethernet transmission cable to transmit the network signal and current simultaneously. Compared with standard power supply, this method reduces the onerous process and cost of setting the power cord. Moreover, only need to consider the management of the network cable in the maintenance work without a power cord, power adapter and power socket. In a word, PoE technology, combined with home equipment, can potentially make the home environment more secure and reliable. A typical PoE structure includes a power sourcing equipment (PSE) and a powered device (PD). Cisco proposed the PoE technology and solved the problem of IP telephony power supply at the first time in 1999. The first IEEE 802.3af PoE standard was introduced in 2003 [27]. Wu et al. achieved four-pair architecture with input current balance function for PoE System [28]. Besides, in our previous study, we have achieved intelligent lighting system using PoE technology to control LED [29]. Up to now PoE technology has been successfully used in several applications, such as Voice over Internet Protocol (VoIP), web camera, wireless access points (AP), monitoring system, sales terminal, home automation and other fields, as is shown in Fig. 1 [28].

The goal of this paper is to explore how to set up a cost-effective and safe-to-use online BCIBSHS to recognize multi-commands and control smart devices based on SSVEP. To address these issues, the cost-effective EEG device (Emotiv EPOC) and PoE device were used to control electric home appliances with wireless manner. And the offline results of the Emotiv EPOC was used to select the optimal parameters for the online BCIBSHS. Moreover, the BCIBSHS with PoE technology has the advantages of safety, convenience, low cost, etc. Hence, in this paper, the combination of the BCI technology and PoE technology will provide a new way for supporting daily life of paralyzed people and elderly people in the smart home system.

The structure of the remaining parts of this paper is as follows. The materials and methods are introduced in Section 2. Experimental results and analysis are displayed in Section 3. The discussion and conclusion are summarized in Section 4.

Figure 1.

General structure of a typical PoE.

Figure 2.

Schematic architecture of the proposed BCIBSHS.

2. Materials and methods

2.1 The architecture of BCIBSHS

The BCIBSHS consists of EEG signal acquisition, preprocessing, signal feature extraction, signal classification, command translation, controlled smart home devices and feedback, as is shown in Fig. 2. In SSVEP stimulus paradigm, EEG signal acquisition is performed by selecting locations channels which are used for data recording after ensuring good conductivity in the headset setup status monitoring part of the software. The collected EEG signals are transmitted to the computer through Bluetooth for processing (preprocessing, signal feature extraction and classification). Preprocessing steps are performed, especially for eliminating artifacts and extracting relevant frequencies information. DWT is used to denoise and reconstruct useful information for the next step. In feature extraction and classification, canonical correlation analysis (CCA) is used to find the largest correlation coefficient among many correlation coefficients. Because only the maximum correlation coefficient can best describe the ability of the typical variable correlation. After calculation, the classification results are converted to the corresponding instruction signals. Then the command signal is sent to the PoE device. Finally, PoE device controls smart home devices to perform corresponding tasks, and the results of tasks are fed back to subjects.

2.2 Design of the smart home

The smart home consists of four main rooms, including a living room, two bedrooms and a bathroom, as is shown in Fig. 3. The apartment with PoE technology is equipped with several smart home devices to perform some tasks. In BCIBSHS, lamp, guardianship telephone, web camera, and intelligent blinds are selected for elderly people and paralyzed people who cannot live independently and suffer severe motor disabilities can benefit from both BCI and PoE technology. Therefore, The objective of the BCIBSHS is to enhance elderly people and paralyzed people’s self-confidence, increase their ability to live independently and improve their quality of life.

Figure 3.

The smart home.

Figure 4.

Experimental layout for SSVEP.

2.3 Experimental paradigm

Visual stimulation is performed by LCD computer monitor, but it is limited by the monitor 60 Hz refresh rate. Therefore, the choice of stimulus frequencies is very important due to the monitor refresh rate and principle of SSVEP. Four different stimulus frequencies, namely, 6 Hz, 7.5 Hz, 8.57 Hz and 10 Hz are chose to turn on/off lamp, web camera, guardianship telephone and intelligent blinds. Each selected frequency is not harmonic of another chosen frequency. Stimuli is delivered to participants through laptop screen, and the distribution of the four targets stimulus are shown in Fig. 4. There are four white-black stimuli of shape in black board. The flickering frequencies of the squares at the top, bottom, left, and right positions were 6 Hz, 7.5 Hz, 8.57 Hz and 10 Hz, respectively.

For example, in this BCIBSHS, in order to adjust the brightness of the room, paralyzed people and elderly people can turn the lamp on or off, they directly control the current rather than toggling switch status. Guardianship telephone was used to call his assistant or family when they need help such as drinking water, having a rest and eating. Videocorder was used to record video of rooms when they turn on the web camera. In order to avoid the sun, they can turn the intelligent blinds on or off. Moreover, assistant or family can also use the mobile phone to monitor the rooms and control other devices.

According to the SSVEP type, the BCIBSHS combined with four virtual keys was designed. The SSVEP-based EEG signals generated by the four different frequencies, and each frequency corresponds to the unique virtual key. When a virtual key was pressed, a controlled smart home device will be turned on or off, as is shown in Table 1.

Table 1
The control commands of BCIBSHS

Virtual key	SSVEP-based BCI frequency	Control commands
1	6 Hz	Turn telephone on or off
2	7.5 Hz	Turn lamp on or off
3	8.57 Hz	Turn web camera on or off
4	10 Hz	Turn intelligent blinds on or off

Eight healthy subjects, including six males and two females. All participants were aged from 21 to 26 years old (average age is 24 $\pm$ 1.6), had normal or corrected to normal vision, and signed their written informed consent prior to the experiments. The subjects were seated in a comfortable chair with 45 cm away from the computer screen. In order to analyze EEG signals by Emotiv EPOC, each subject performed six runs and a run was composed of trials of each of the four stimulus frequencies, that is, each subject carried out 24 trials in total. To avoid fatigue, 8 s stimulation phase, as well as 1 s cue with a short beep, 3 s rest were included in a trial. From second 1 to second 9, the trial entered stimulation phase. In stimulation phase, the marker was generated in the TestBench software after a short beep and the EEG signals were recorded. Four visual stimulators, flickering at the frequencies of 6 Hz, 7.5 Hz, 8.57 Hz and 10 Hz respectively. And the subjects were asked to gaze a stimulus frequency (6 Hz, 7.5 Hz, 8.57 Hz or 10 Hz) for 8 s. In other words, for each subject, six sessions of 8 s EEG data were acquired at each stimulus frequency, so in total 48 s EEG data at each stimulus frequency were collected for feature extraction and classification in Matlab, as is shown in Fig. 5.

Figure 5.

EEG recording of the experiment.

The EEG data were collected by Emotiv EPOC, which was a low-cost, portable and wireless headset that consists of 14 channels. The EEG data were internally sampled at a frequency of 2048 Hz, then got down-sampled to 128 Hz sampling frequency in each channel, and send the data via Bluetooth or Wi-Fi. Each electrode connects to the scalp by a felt pad which must be damped with saline solution before positioning on the head. The electrodes were placed at 10–20 system locations, that is, they are AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8 and AF4, as well as two reference electrodes located above the subject’s ears (CMS and DRL), as is shown in Fig. 6. After ensuring good conductivity on all the electrodes, all electrodes were used to capture EEG signals. Furthermore, the Emotiv Software Development Kit (SDK) provides a writable marker trace to tag time period of stimulus signal. And the Application Programming Interface (API) can use C++, C#, Java, Matlab to write program [30]. In order to collect the effective EEG signals on online BCIBSHS, Matlab and C code mixed program was used to compile API.

Table 2

Frequency band range of each level of wavelet decomposition

Each component	Decomposition level	Frequency band (Hz)
$D_{1}$	1	32 $\sim$ 64
$D_{2}$	2	16 $\sim$ 32
$D_{3}$	3	8 $\sim$ 16
$D_{4}$	4	4 $\sim$ 8
$D_{5}$	5	2 $\sim$ 4
$A_{5}$	5	0 $\sim$ 2

Figure 6.

(a) Emotiv EPOC headset and (b) distribution of electrode position.

2.4 Processing algorithm of EEG signal

In the BCIBSHS, discrete wavelet transform (DWT) and canonical correlation analysis (CCA) method were utilized to process EEG signals. DWT was used to analyze the collected EEG signals and provide information in the time-frequency domain. It has advantage of multi-resolution decomposition [31, 32]. The multi-scale characteristics of wavelet transform can decompose the signal into multiple sub-bands, which can analyze the characteristics of the signal in each sub-band. The sub-bands of the respective components are shown in Table 2. Assuming that $x(n)$ is the EEG signal, the DWT of $x(n)$ is defined as:

$\displaystyle C_{j,k}=2^{-j/2}\sum_{n=-\infty}^{\infty}x(n)\bar{\varphi}_{j,k}% (2^{-j}n-k)=\langle x(n),\varphi_{j,k}\rangle,\quad j,k\in Z$ (1)

Where $\varphi(n)$ is the wavelet basis function, $j$ is the resolution of the frequency, and $k$ is the amount of time translation.

Taking the Mallat algorithm to achieve multilayer signal decomposition, the formula follows that:

$\displaystyle x(n)=A_{L}+\sum_{j=1}^{L}D_{j}$ (2)

Where $A_{L}$ is the low frequency component, $D_{j}$ is the high frequency component at different scales, and L is the number of decomposition layers. Assuming that the sampling frequency of $x(n)$ is $f_{s}$ , the discrete wavelet transform can decompose the signal into multiple sub-bands. In the above equation, the sub-bands of $A_{L}$ , $D_{L}$ , $D_{L-1}$ , …, $D_{1}$ are $\left[0,\frac{f_{s}}{2^{L+1}}\right]$ , $\left[\frac{f_{s}}{2^{L+1}},\frac{f_{s}}{2^{L}}\right]$ , $\left[\frac{f_{s}}{2^{L}},\frac{f_{s}}{2^{L-1}}\right]$ , …, $\left[\frac{f_{s}}{2^{2}},\frac{f_{s}}{2}\right]$ . Therefore, the use of wavelet basis function db4 to achieve 5-layer decomposition of EEG signals.

CCA is a multivariable statistical method, which is used to analyze the potential correlation between two sets of data [33]. Suppose that two multidimensional random variables $X, Y$ , that is $X\in R^{H\times J}$ , $Y\in R^{I\times J}$ . CCA finds a pair of weight vectors $w_{X}\in R^{H\times 1}$ and $w_{Y}\in R^{I\times 1}$ , respectively, which maximize the correlation between linear combinations $x=w_{X}^{T}X$ and $y=w_{Y}^{T}Y$ . It is defined as

$\displaystyle\max\limits_{{w_{X}},{w_{Y}}}\rho(x,y)=\frac{E\left[xy^{T}\right]% }{\sqrt{E\left[xx^{T}\right]E\left[yy^{T}\right]}}=\frac{E\left[w_{X}^{T}XY^{T% }w_{Y}\right]}{\sqrt{E\left[w_{X}^{T}XX^{T}w_{X}\right]E\left[w_{Y}^{T}YY^{T}w% _{Y}\right]}}$ (3)

Where the maximum of $\rho$ is the maximum canonical correlation. $x$ and $y$ are projected onto $w_{X}$ and $w_{Y}$ .

Figure 7.

The power scalp distribution.

In this paper, the CCA method was used in our BCIBSHS, as is shown in Fig. 7. $k$ represents $k$ visual stimulators with different stimulus frequencies. $X$ refers to the set of $n$ channel of $t$ s long EEG signals in a trial. $Y_{i}$ refers to the set of reference signals with certain frequency $f_{i}$ , which have the same length as $X$ . $\rho_{i}$ represents the ith canonical correlation of EEG signals and each of reference signals $Y_{i}$ . Compare the values of $\rho_{1},\rho_{2},\cdots,\rho_{k}$ , and find the maximum canonical correlation. It follows that

$\displaystyle Y_{i}=\left(\begin{array}[]{c}\sin(2\pi f_{i}t)\\ \cos(2\pi f_{i}t)\\ \vdots\\ \sin(2\pi N_{h}f_{i}t)\\ \cos(2\pi N_{h}f_{i}t)\end{array}\right),\quad t=\frac{1}{S},\frac{2}{S},% \cdots,\frac{N}{S}$ (4)

Where $N$ is the number of sampling points, $S$ is the sampling frequency, $N_{h}$ is the number of harmonics.

Figure 8.

The channels selected for subject 4 in the experiment. The selected channels are located in the occipital region according to the power scalp distribution. Where red indicates that the region has larger contribution to the SSVEP signal. On the contrary, blue indicates that the region has smaller contribution to the SSVEP signal.

Figure 9.

Time domain of the original signal and reconstructed signal. The first subject was gazing a certain visual stimulus, the collected four seconds of EEG signals were denoised. Discrete wavelet transform with wavelet basis function db4 was utilized to achieve 5-layer decomposition of EEG signals and $D_{2}$ , $D_{3}$ , $D_{4}$ and $D_{5}$ four parts to achieve the reconstruction of EEG signal.

2.5 Overall evaluation of BCIBSHS experiment

In this online experiment section, we designed an overall evaluation experiment for BCIBSHS. All subject were required to complete 4 tasks, including turning on or off lamp, guardianship telephone, web camera, and intelligent blinds. Each subject performed 5 experiments on each home equipment, and a total of 20 trials. After a set of trials (4 trials), an USE questionnaire were given to each subject to evaluate the performance of the system [34]. Usability was used to assess each of the devices in the BCIBSHS. And the questions were based on a 7-point Likert rating scales which ranging from strongly disagree to strongly agree. Furthermore, objectively, the correct rate and ITR were also considered in this experiment. Therefore, the following points are used to evaluate the performance of the BCIBSHS:

(1)
Usability;
(2)
Correct rate: accuracy of 20 online trails in the BCIBSHS;
(3)
ITR: rate of information transfer, It is defined as

$\displaystyle\textit{ITR}=\frac{60}{T}\times\left[\log_{2}N+P\log_{2}P+(1-P)% \log_{2}\left(\frac{1-P}{N-1}\right)\right]$ (5)

where, $N$ is the number of targets, $P$ is the accuracy rate, and $T$ is the time window length.
3. Experimental results and analysis

There are three important parameters in SSVEP-based system, including channel location, window length $T$ and number of harmonics $N_{h}$ . The first harmonic part has great effect on SSVEP-based BCI system performance [11]. In order to find the most appropriate combination of parameters for Emotiv EPOC, channel location and window length $T$ were used to analyze in offline experiments. Consequently, the optimal parameter was selected for online BCIBSHS.

3.1 Channel location

EEG signals of all channels of eight subjects were collected in offline experiments. And the channels selected by analyzing the power scalp distribution of each subject. For example, the power scalp distribution of the subject 4 in four frequencies are shown in Fig. 8. We can see from the Fig. 6b and the Fig. 8 that the significant region is located P7, P8, O1 and O2. So locations P7, P8, O1 and O2 were selected to collect EEG signals for online BCIBSHS.

3.2 Wavelet denoising analysis

DWT was used to improve performance in BCIBSHS. For example, EEG signal of the subject 1, as is shown in Fig. 9. By utilizing DWT method to reconstruct EEG signal, the drift phenomenon of EEG signal disappears in the time domain.

Figure 10.

The relationship between accuracy rate and time window length.

3.3 Analysis of the choice of time window length

Classification accuracy of eight subjects, with the time window length being 1 s, 2 s, 3 s, 4 s, respectively, as is shown in Fig. 10. The longer the time window is, the higher the classification accuracy of the signal is. When the time was more than 2 s (do not contain 2 s), subjects 1, 2, 4, 6, 8 had a higher accuracy rate in classification, and subjects 3, 5, 7 had classification accuracy of not less than 79%. So, through the analysis of experiment data, 3 s or 4 s (time window length) was used for the online BCIBSHS.

Table 3
The relationship between average accuracy with $T$ and ITR

$T$ (s)	Average accuracy rate (%)	ITR (bit/min)
1 s	64.07	29.31
2 s	79.17	27.95
3 s	86.98	24.71
4 s	93.23	23.02

3.4 Analysis of ITR

Any two of the three variables ( $T, N, P$ ) are inversely related in Eq. (5). Therefore, we select appropriate $T$ to reach the highest ITR when the $N$ was determined. The theoretical ITR of the EEG signals of eight subjects in different $T$ are shown in Table 3. By comparing the average accuracy rate, ITR and other reasons, 3 s was selected in BCIBSHS.

Table 4
Result of the online BCIBSHS

Subjects	Correct times	Correct rate (%)	ITR (bit/min)
Subject 1	17	85.0	23.05
Subject 2	16	80.0	19.22
Subject 3	18	90.0	27.45
Subject 4	19	95.0	32.69
Subject 5	18	90.0	27.45
Subject 6	18	90.0	27.45
Subject 7	16	80.0	19.22
Subject 8	17	85.0	23.05
Mean $\pm$ SD	17.38 $\pm$ 1.06	86.88 $\pm$ 5.30	24.95 $\pm$ 4.65

Figure 11.

Average correct times and SD counted by users gaze visual stimulation of guardianship telephone, lamp, web camera and intelligent blinds, respectively.

3.5 Threshold analysis

When subjects did not gaze any target or only watching the target switch process (no task state), CCA method still outputs result, which is not a control command. Therefore, threshold detection method was used to detect idle state. Moreover, this method makes the online BCIBSHS more practical when subjects take a break or do other things for a short while. In our previous study, we have analyzed the thresholds of different frequencies by Emotiv EPOC headset [10]. Therefore, in this paper, the online BCIBSHS selected 0.22 as CCA threshold.

Figure 12.

Average scores and SD counted by subjects about usability on a scale from 1 to 7.

3.6 Results of online BCIBSHS

Through the comprehensive analysis among channel location, time window length $T$ , threshold value, accuracy rate $P$ and ITR, locations P7, P8, O1, O2, 3 s time window length and 0.22 threshold value were selected for the online BCIBSHS. The online results of 8 subjects are shown in Table 4. The average accuracy and average ITR of all participants were 86.88 $+$ 5.30% and 24.95 $+$ 4.65 bit/min, respectively. And subject 4 correct rate was 95% and ITR was 32.69 bit/min. And the correct times of toggling of each device is shown in Fig. 11. The average of correct times for each device was higher than 4. Moreover, The results also indicated that the SSVEP detectability of web camera, and intelligent blinds (8.57 Hz and 10 Hz) was clearly higher than guardianship telephone and lamp (6 Hz and 7.5 Hz). On the other hand, Fig. 12 displays the summary of the answers to the questionnaire in the BCIBSHS. The average scores of each device was higher than 4.5 points. Most the subjects considered the web camera to be useful and made a significant evaluation. And most people believed that lamp was less usable than other devices. Moreover, the subject 4 scored no less than 6 points for each device, and the subject 2 scored no more than 6 points for each device.

4. Discussion and conclusion

In this paper, this study aims at developing a smart home system based BCI technology and PoE technology. The proposed BCIBSHS could help to provide a new way for supporting life of paralyzed people and elderly people. The results demonstrate that the proposed BCIBSHS could enhance their self-confidence, increase their ability to live independently and improve their quality of life.

This study also demonstrates that effect of BCIBSHS by Emotiv EPOC with wireless manner. This study utilized a relatively simple and evidently practical classification method. By using DWT to reconstruct EEG signal and using CCA with threshold value as the core to classify EEG data. So the BCIBSHS implements 5 kinds of tasks (four control commands and one idle status command) without excluding the noise of the family environment. Eight subjects were recruited to do the experiments to control lamp, web camera, guardianship telephone and intelligent blinds. In offline experiments, we analyzed some important parameters for the online BCIBSHS. This research worked with an average correct rate of 86.88 $\pm$ 5.30% and an average ITR of 24.95 $\pm$ 4.65 bit/min in the online BCIBSHS. And the mean of correct times for each device was higher than 4. These results showed that us proposed the BCIBSHS was robust and efficient for real-time smart home applications control. The USE questionnaire results suggested that people can benefit from the use of BCIBSHS. Maybe for the subjects with no BCI experience, it is an interesting method to control equipment. Moreover, the evaluation of usability may be related to the accuracy of the device and the ITR in the human-computer interaction.

In most of the previous works [8, 11, 35], their systems have a high ITR. The reasons are as follows: first, compared with NeuroScan, g.tec, the Emotiv EPOC has a lower sampling rate and fewer channels. Second, the experimental SSVEP paradigm uses LCD method to stimulate. Compared with LED-based stimulation method, this SSVEP paradigm may be limited by the monitor 60 Hz refresh rate. But compared with them [8, 11, 35], this paper provided a new way to bring the BCI technology out of lab, and the combination of BCI and PoE technology gives us a new perspective on the smart home system.

Moreover, in recent years, Emotiv EPOC is rapidly evolving in BCI applications, may be due to its convenience, low price, wireless, secondary development. Comparing with the current SSVEP-based BCI system with Emotiv EPOC [36, 37, 38], the proposed BCIBSHS shows higher average correct rate or ITR than others. Moreover, In order to solve a difficult problem [38], we wrote program to build a stable connection between Emotiv EPOC and Matlab in online experiment. Furthermore, compared with these [24, 25, 26], the BCIBSHS is also an innovative application. Not only that, PoE technology has the advantage of safety, low energy consumption, low cost and convenience. The BCI and PoE technology, combined with smart home system, overcoming the shortcomings of traditional systems and achieving home applications management rely on EEG signal. In the future work we may pay more attention to improving BCIBSHS usability.

Footnotes

Acknowledgments

This work was supported by the Young and Middle-Aged Innovation Talents Cultivation Plan of Higher Institutions in Tianjin (grant no. 20130830) and the National Natural Science Foundation of China (grant no. 61502340).

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this manuscript.

References

Wolpaw

Birbaumer

Heetderks

Mcfarland

. Brain-computer interface technology: A review of the first international meeting. Rehabilitation Engineering IEEE Transactions on 2013; 8: 164-173.

Birbaumer

Murguialday

Cohen

. Brain-computer-interface (BCI) in paralysis. Current Opinion in Neurology 2010; 21: 634-638.

Kübler

Kotchoubey

Kaiser

Wolpaw

Birbaumer

. Brain-computer communication: Unlocking the locked in. Psychological Bulletin 2001; 127: 358-375.

Riccio

Simione

Schettini

Pizzimenti

Inghilleri

Belardinelli

, et al. Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis. Frontiers in Human Neuroscience 2013; 7: 732-741.

Mao

Niu

Xian

Zeng

, et al. Progress in EEg-based brain robot interaction systems. Computational Intelligence and Neuroscience 2017; 2017: 1-25.

Birbaumer

Ghanayim

Hinterberger

Iversen

Kotchoubey

Kübler

, et al. A spelling device for the paralysed. Nature 1999; 398: 297-298.

Pfurtscheller

Lopes

. Event-related EEG/MEG synchronization and desynchronization: Basic principles. Clinical Neurophysiology 1999; 110: 1842-1857.

Lin

Zhang

Gao

. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Transactions on Biomedical Engineering 2006; 53(12): 2610-2614.

Sutton

Braren

Zubin

John

. Evoked-potential correlates of stimulus uncertainty. Science 1965; 150(3700): 1187-1188.

10.

Gao

Dou

Belkacem

Chen

. Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system. Biomed Research International 2017; 2017: 1-8.

11.

Bin

Gao

Yan

Hong

Gao

. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. J Neural Eng 2009; 6(4): 046002.

12.

Resalat

Saba

Afdideh

Heidarnejad

. High-speed SSVEP-based BCI: Study of various frequency pairs and inter-sources distances. BHI’12: Proceedings of the IEEE-EMBS International Conference on Biomedical and Health Informatics; 2012 Jan 5-7; Hong Kong, China. IEEE; 2012. p. 220-223.

13.

Faller

Müller-Putz

Schmalstieg

Pfurtscheller

. An application framework for controlling an avatar in a desktop-based virtual environment via a software SSVEP brain-computer interface. Presence 2010; 19(1): 25-34.

14.

Singla

Khosla

Jha

. Influence of stimuli color on steady-state visual evoked potentials based BCI wheelchair control. Journal of Biomedical Science and Engineering 2013; 6(11): 1050-1055.

15.

Diez

Torres Müller

Mut

Laciar

Avila

Bastos-Filho

, et al. Commanding a robotic wheelchair with a high-frequency steady-state visual evoked potential based brain-computer interface. Medical Engineering and Physics 2013; 35(8): 1155-1164.

16.

Zhang

Huang

Cheng

Yao

. SSVEP response is related to functional brain network topology entrained by the flickering stimulus. Plos One 2013 Sep 9; 8(9): e72654.

17.

Middendorf

Mcmillan

Calhoun

Jones

. Brain-computer interfaces based on the steady-state visual-evoked response. IEEE Trans Rehab Eng 2000; 8(2): 211-214.

18.

Müller-Putz

Pfurtscheller

. Control of an electrical prosthesis with an SSVEP-based BCI. IEEE Transactions on Biomedical Engineering 2007; 55(1): 361-364.

19.

Luo

Bian

Yang

, et al. A hybrid brain-computer interface control strategy in a virtual environment. Frontiers of Information Technology and Electronic Engineering 2011; 12(5): 351-361.

20.

Simoens

Coninck

Vervust

Wijmeersch

JFV

Ingelbinck

Verbelen

. Vision: Smart home control with head-mounted sensors for vision and brain activity. International Workshop on Mobile Cloud Computing and Services 2014; 44(1): 29-33.

21.

Wei

Nisar

Malik

Yea

. A brain computer interface for smart home control. IEEE International Symposium on Consumer Electronics 2013 Jun: 35-36.

22.

Edlinger

Holzner

Groenegress

Guger

Slater

. Goal-oriented control with brain-computer interface. Schmorrow

Estabrooke

Grootjen

Edlinger

, editors. Proc FAC 2009: Proceedings of the 5th International Conference on Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience: Held as Part of HCI International 2009; 2009 July 19-24; San Diego, CA. Berlin: Springer; 2009. p. 732-740.

23.

Kjeldskov

Skov

. Studying usability in sitro: Simulating real world phenomena in controlled environments. International Journal of Human-Computer Interaction 2007; 22(1-2): 7-36.

24.

Carabalona

Grossi

Tessadri

Castiglioni

Caracciolo

de Munari

. Light on! Real world evaluation of a P300-based brain-computer interface (BCI) for environment control in a smart home. Ergonomics 2012; 55(5): 552-563.

25.

Nataliya

Franck

Nicolas

Bertrand

. Feasibility of BCI control in a realistic smart home environment. Frontiers in Human Neuroscience 2016 Aug; 10: 416-426.

26.

Lin

Chang

. Brain computer interface-based smart living environmental auto-adjustment control system in UPnP home networking. IEEE Systems Journal 2014; 8(2): 363-370.

27.

IEEE 802.3af standard(2003)[homepage on the Internet], Available from: http//www.ieee802.org/3/af

28.

. Advanced four-pair architecture with input current balance function for power over Ethernet (PoE) system. IEEE Transactions on Power Electronics 2012; 28(5): 2343-2355.

29.

Gao

Sui

Zhang

, et al. Design and implementation of LED intelligent lighting system based on the technology of PoE. 2015 CCDC: The 27th Chinese Control and Decision Conference; 2015 May 23-25; Qingdao, China. USA: IEEE Computer Society; 2015. p. 2628-2633.

30.

Ranky

Adamovich

. Analysis of a commercial EEG device for the control of a robot arm. Bioengineering Conference: Proceedings of the 2010 IEEE 36th Annual Northeast; 2010 Mar 26-28; New York, NY. USA: IEEE Computer Society; 2010. p. 1-2.

31.

Lin

Jiang

. An EEG-based BCI system to facial action recognition. Wireless Personal Communications 2016; 94(3): 1-15.

32.

Unser

Aldroubi

. A review of wavelets in biomedical applications. Proc IEEE 1996; 84(4): 626-638.

33.

Hotelling

. Relations between two sets of variates. Biometrika 1936; 28(3-4): 321-377.

34.

Lund

. Measuring usability with the USE questionnaire. Usability and User Experience 2001; 8(2): 3-6.

35.

Gao

Cheng

Gao

. A BCI-based environmental controller for the motion-disabled. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2003; 11(2): 137-140.

36.

Martišius

Damaševičius

. A prototype SSVEP based real time BCI gaming system. Computational Intelligence and Neuroscience 2016; 2016(3): 3861425.

37.

Holewa

Nawrocka

. Emotiv EPOC neuroheadset in brain-computer interface. Proceedings of the 2014 15th International Carpathian Control Conference (ICCC); 2014 May 28-30; Velke Karlovice, Czech Republic. USA: IEEE Computer Society; 2014. p. 149-152.

38.

Liu

Jiang

Cao

Wan

Mak

, et al. Implementation of SSVEP based BCI with Emotiv EPOC. 2012 IEEE International Conference on Virtual Environments Human-Computer Interfaces and Measurement Systems (VECIMS) Proceedings; 2012 July 2-4; Tianjin, China. USA: IEEE Computer Society; 2012. p. 34-37.

Controlling of smart home system based on brain-computer interface

Abstract

BACKGROUND:

OBJECTIVE:

METHODS:

RESULTS:

CONCLUSION:

Keywords

1. Introduction

2.1 The architecture of BCIBSHS

2.2 Design of the smart home

Table 1 The control commands of BCIBSHS

3.1 Channel location

3.2 Wavelet denoising analysis

Table 3 The relationship between average accuracy with T and ITR

Table 4 Result of the online BCIBSHS

4. Discussion and conclusion

Footnotes

Acknowledgments

Conflict of interest

References

Table 1
The control commands of BCIBSHS

Table 3
The relationship between average accuracy with $T$ and ITR

Table 4
Result of the online BCIBSHS