Brain computer interface for smart hardware device

1 Biographical notes

Dr.G. Nagarajan has received his Diploma in Electronic & Communication Engineering from Directorate Of Technical Education 1997. He has received his BE degree in Electrical & Electronic Engineering from Manonmaniam Sundaranar University 2000. He received his ME degree in Applied Electronic Engineering from Anna University 2005. He also received his ME degree in Computer Science Engineering from Sathyabama University 2007 He obtained his Ph.D. degree in Computer Science Engineering from Sathyabama University 2015. He has 15 years of teaching experience. Now, he is working as Professor in department of CSE in Sathyabama University. He has published more than 50 paper in the research areas of Wireless Sensor Network, Intelligent Irrigation Systems, Artificial Intelligent, Ontology Learning, Machine Leaning, IoT and Computer Vision

Dr.R.I. Minu obtained her B.E., degree in Electronics and Communication Engineering from Bharathidasan University in the year 2004 and received her M.E., degree in Computer Science Engineering from Anna University in the year 2007. She obtained her Ph.D. degree in the Faculty of Information and Communication from Anna University in the year 2015. She has 11 years of teaching experience. Now, she is working as Associate Professor in the Department of Computer Science and Engineering in SRM Institute of Science and Technology. She has published more than 40 papers in International/National Journals and Conferences. She has involved in many UG and PG projects in the area of Computer Vision, Internet of Things,Wireless Sensor Networks, Ontology Engineering, E-Learning and Artificial Intelligent.

1 Introduction

Man machine interface has been one of the growing field of research and development in recent years in which it designed as a friendly user or innovative interface such as voice recognition, virtual reality. Presently brain computer interface, sometimes called as a direct neutral interface in which it id direct communication between brain and a external device. Here the BCI has both hardware and software components in which the control signals are generated from the electroencephalographic (EEG) activities as referred by (Kryger et al., 2017; Tanka, 2005). The Brain Computer Interface (BCI) reads the waves produced from the brain at different locations in the human head, translates these signals into actions, and commands that can control the computer. The BCI can lead to many applications especially for disabled persons such as social interactions, enabling social applications to capture feelings and emotions, helping partially or fully-disabled people to interact with different computational devices, and helping understanding more about brain activities and human neural networks. These applications depend on the basic understanding of how the brain works. Research on (Brunner, et al., 2015) BCIs has been going on for more than 30 years but from the mid 1990’s there has been dramatic increase working experimental implants.

In order to help physically challenged people control a computer, a communication device or a Robot, various input devices are available. The only alternative for locked-in people is to establish communication and control channels directly from the brain, bypassing the malfunctioning brain’s normal output channels of peripheral nerves and muscles. In a Brain Computer Interface (BCI), signals from the brain are acquired and processed to extract specific features that reflect the user’s intent. These features are then translated into commands to operate a device. The Brain Controlled Robot was designed to provide some motion capability to locked-in people.

A common feature between all BCIs is that, since the recorded brain signal is very noisy and has a large variability, either the uncertainty on the command will be high, or the time between consecutive commands will be long, in the order of seconds. Our goal in this work is to propose a strategy to control a Robot from a BCI. This requires a robotic Robot able to assist the user with the navigation task, and a BCI together with a control scheme adapted to the task.

2 Literature review

Melody. M et al. (2003), A new ways for gamers to play games using their heads, social interactions, enabling social applications to capture feelings and emotions, helping partially or fully-disabled people to interact with different computational devices and helping understanding more about brain activities and human neural networks. These applications depend on basic understanding of how the brain works. BCI applications utilize the brain and its nervous system functions.

Tanaka et al. (2005), come with a discrete approach to the navigation problem: the environment is discredited in squares of 1 m and the user is prompted where to move next. They use an EEG BCI based on motor imagery: by imagining left or right limb movements, thus activating the corresponding motor cortex, the user selects where to move next.

By imagining left or right limb movements, the user decides the next move of the Robot. Although simple and safe, the system requires series of decisions to complete even a simple movement and may thus exhaust the subject. Therefore this strategy clearly breaks the ergonomic constraint, although it respects the safety and low cost constraints.

In the MAIA (Mental Augmentation through Determination of Indented Action) project the asynchronous IDIAP BCI has been integrated with the intelligent Robot Sharioto of the KU Leuven (2007).The BCI continuously analyzes the user’s EEG to detect a pattern associated with one of three learned mental states. After a short evaluation, the user selects the three mental tasks that he/she finds easier out of the following choices: “relax”, imagination of “left” and “right” hand (or arm) movements, “cube rotation”, “subtraction”, or “word association”. This approach proved very successful with non-disabled subjects. However, it requires the user to be constantly alert, which is likely to cause stress and fatigue. The ergonomy constraint is therefore only half fulfilled: the strategy provides an efficient way of navigating toward the goal, however it imposes a heavy burden on the subject. The safety constraint is fully respected.

Allison, B.Z et al. (2008), They demonstrated significant two-dimensional control with scalp recorded EEG signals in human subjects, and in current work they are achieving EEG-based two-dimensional control comparable to that reported in monkey single-neuron studies. This EEG-based control relies on substantial training to establish two independent control signals (i.e., for vertical and horizontal movements, respectively) that are not evident prior to training. The two dimensional joystick task used in the present study was similar to a wide variety of planar hand movements typically coordinated by the brain, and analysis revealed directionally specific ECoG activity at frequencies well above those usually discernible in scalp-recorded EEG.

Graimann B et al. (2009) BCIs can control any application that other interfaces can control, provided these applications can function effectively with the low information throughput of BCIs Most of these applications run on conventional computers that host the BCI system and their application as well. Typically, the application is specifically tailored for a particular type of BCI, and often the application is an integral part of the BCI system. BCIs that can connect and effectively control a range of already existing assistive devices, software, and appliances are rare. An increasing number of systems allow control of more sophisticated devices, including orthoses, prostheses, robotic arms, and mobile robots.

The sophisticated Robot system recently developed by Iturrate, I et al. (2009) where a virtual reconstruction of the surrounding environment (as inferred from laser range scanner data) is displayed with a set of points in the free space that can be selected using a P300 EEG BCI, and these short term goals are reached automatically. As with Tanaka, the system requires a large number of steps to reach a destination, which might exhaust the subject. For instance, Minguez reports that it took 11 minutes and 9 decision steps to realize a 40 meters long path with this system.

Alwasiti H et al. (2010), The BCI technology has achieved many goals in different areas of medical and nonmedical during the last ten decades. Since the current trend are focused on utilizing of motor system which is related to: the electroencephalogram (EEG) signals for neuro rehabilitation, controlling of robot and exoskeleton based on EEG signal, implementation of BCI component on field programmable and reconfigurable computing system, and solving the interoperability and standardization issues of BCI software platforms.

Arindam.D et al. (2013), Emotiv EEG neuro headset is a wireless BCI set; this set is a neuro-signal acquisition and processing wireless neuro headset. The set can be wirelessly connected to a computer Modular EEG is another EEG hardware created by the Open EEG hardware developers. The modular EEG has two or more EEG amplifiers, and a 6-channel signal capture board that connects to a PC via a standard serial cable. The modular EEG has two types of electrodes which are active and passive electrodes. Some skin preparation is required while there is no preparation is required when active electrodes are used a Voluntary control of SCPs may be used for letter selection. With extensive training, completely paralyzed patients are able to produce positive and negative changes in their SCP to drive the vertical movement of a cursor. Based on this kind of control signal, developed a spelling device with an on-screen display, which used a cursor to select letters of the alphabet. Trials involving two patients at advanced stages of ALS showed that they achieved a rate of about 2 characters per minute when texting text messages. Other types of control signals, such as detection of eye blinks, which normally represent an artifact in EEG signals, or classification of three mental tasks, are also used to select the blocks or characters in a virtual keyboard. Both approaches are nearly the same apart from the control signal.

3 Brain computer interface and electroencephalography

Forming a BCI system requires following three main steps as preferred by Melody (2003). Step 1 is the signal acquisition, Step 2 is the signal processing, and Step 3 is the data manipulation. Signal acquisition process is required to capture the brain electric signals. Signal processing could be done through some other sub operations as preprocessing, Feature extraction and signal classification. Electroencephalography (EEG) measures electric brain activity caused by the flow of electric currents during synaptic excitations of the dendrites in the neurons and is extremely sensitive to the effects of secondary currents. EEG signals are easily recorded in a non-invasive manner through electrodes placed on the scalp, for which that reason it is by far the most widespread recording modality. EEG comprises a set of signals which may be classified according to their frequency. Well-known frequency ranges have been defined according to distribution over the scalp or biological significance. These frequency bands are referred to as delta (δ), theta (θ), alpha (α), beta (β), and gamma (γ) from low to high, respectively. The delta band lies below 4 Hz, and the amplitude of delta signals detected in babies decreases as they age. Delta rhythms are usually only observed in adults in deep sleep state and are unusual in adults in an awake state.Theta waves lie within the 4 to 7 Hz range. In a normal awake adult, only a small amount of theta frequencies can be recorded. A larger amount of theta frequencies can be seen in young children, older children, and adults in drowsy, meditative or sleep states. Alpha rhythms are found over the occipital region in the brain. These waves lie within the 8 to 12 Hz range. Their amplitude increases when the eyes close and the body relaxes and they attenuate when the eyes open and mental effort is made. Beta rhythms, within the 12 to 30 Hz range, are recorded in the frontal and central regions of the brain and are associated with motor activities. Gamma rhythms belong to the frequency range from 30 to 100 Hz. The presence of gamma waves in the brain activity of a healthy adult is related to certain motor functions or perceptions, among others.

4 Implementation

The main motivation is to control the movement of our robot through the brain signals captured by the hardware component namely NeuroSky Mindwave sensor. Firstly the brain signals captured, in our case we are using EEG technique for signal acquisitions. Further signals pre-processed followed by feature extraction where brain signals are converted useful commands which can be sent to the computer to perform some predefined actions which is completely dependent on the output of the brain wave sensor. As mentioned earlier some output is fed to our computer as input from brain wave sensor. This input is processed via MATLAB programming in our computer. And output of MATLAB will transmitted to the Arduino microcontroller via pair of ZigBee transmitter and ZigBee receiver. Arduino will run the preloaded program using the output of MATLAB as input and generate outputs, which will instruct the L293D circuit driver to drive the motors respectively.

As shown in Fig. 1, the brain wave signals captures the brain signals and interrupting them through patented as Attention and Meditation algorithm in which get ranges of Attention and Meditation level. These level acts as a input to the pc(computers, laptops). A certain MATLAB program is designed to use these Attention, Meditation and Blink level to run the motor or the robotic(proto type). The output of the MATLAB acts as a output to the robotic proto type. These commands is transmitted to the Arduino via zigbee transmitter(Tx) which also connected to the pc through the USB cables. This is shown in Fig. 2.

OPEN IN VIEWER

Fig.1

System design.

OPEN IN VIEWER

Fig.2

The inter modal design.

The zigbee receiver situated on our prototype receiver these signals and transmit it to the Arduino through transmitter (Tx) pin. Hence these signals are received by the Arduino through transmitter (Tx) pins. And these commands serves as an input for the Arduino program which is uploaded in the microcontroller. This microcontroller these input pins and sets its own pins. Hence these outputs A, B, C, D serves as an output to the L293D and it gives command to the motors. These commands are send to the L293D in the form of logic 0 OR 1. The signal generated in forward, right and left direction is shown in Fig. 3 and tabulated the corresponding values in Table 1.

OPEN IN VIEWER

Fig.3

Empirical results (a) Left turn (b) Right turn (c) Forward movement.

Our goal is to develop a system usable in hospitals and homes with minimal infrastructure modifications, which can help these people regain some mobility. The main challenge is to provide continuous and precise 2D control of the Robot from a Brain Computer Interface, which is typically characterized by a very low information transfer rate. Besides, as design constraints, we want our BCR to be safe, ergonomic and relatively low cost. The strategy we propose relies on 1) constraining the motion of the Robot along predefined guiding paths, and 2) a slow but accurate P300 EEG brain interface to select the destination in a menu.

This strategy reduces control to the selection of the appropriate destination, thus requires little concentration effort from the user. Besides, the trajectory is predictable, which contributes to reduce stress, and eliminates frustration that may be associated with trajectories generated by an artificial agent.

The principle of operation is quite simple. Two dry sensors are used to detect and filter the EEG signals. The sensor tip detects electrical signals from the forehead of the brain. At the same time, the sensor picks up ambient noise generated by human muscle, computers, light bulbs, electrical sockets and other electrical devices. The second sensor, ear clip, is a grounds and reference, which allows Think Gear chip to filter out the electrical noise. The device measures the raw signal, power spectrum (alpha, beta, delta, gamma, theta), attention level, mediation level and blink detection. The raw EEG data received at a rate of 512 Hz. Other measured values are made every second. Therefore, raw EEG data is a main source of information on EEG signals using Mind Wave MW001. In the Robot section we can read the value of raw EEG signal with the maximum frequency of 512 Hz. Sampling frequency is set on 512 Hz, and we control time delays in sampling. The value of the signal and time are written to the array data. The data which are stored in array will be compared with the threshold points given by the user. In this project, the MATLAB section waits for three consecutive blink in order to send the Robot activation signal. Then based on the attention level value Robot Move Forward Command will be send to the Robot module through ZigBee transmission. After three consecutive blink, the program will scan for a left blink and right blink to turn the Robot right and left respectively. The overall hardware implementation was shown in Fig. 4.

OPEN IN VIEWER

Fig.4

Hardware implementation.

Finally, the overall strategy was evaluated and compared to other brain controlled Robot projects. Despite the overhead required to select the destination on the interface, our Robot is faster than others (36% faster than MAIA): thanks to the motion guidance strategy, the Robot always follows the shortest path and a greater speed is possible. Comparison was also performed using a cost function that takes into account traveling time and concentration effort; our strategy yields by far the smallest cost (the best other score is 72% larger).This work resulted in a novel brain controlled Robot working prototype.

The brain-controlled mobile robots have a great deal of attention because they can help bring mobility back to people with devastating neuromuscular disorders and thus improve their quality of life. A comprehensive of the brain controlled mobile robot, uses a brain wave sensor which can collect EEG based brain signals of different frequency and amplitude and it will convert these signals into packets and transmit through Wireless medium in to the level splitter section to check the attention level. Level splitter section (LSS) analyses the level and gives the robot movement for the person who is sitting in the Robot.