Abstract
Despite the increase in research interest in the brain–computer interface (BCI), there remains a general lack of understanding of, and even inattention to, human factors/ergonomics (HF/E) issues in BCI research and development. The goal of this article is to raise awareness of the importance of HF/E involvement in the emerging field of BCI technology by providing HF/E researchers with a brief guide on how to design and implement a cost-effective, steady-state visually evoked potential (SSVEP)–based BCI system. We also discuss how SSVEP BCI systems can be improved to accommodate users with special needs.
Keywords
More HF/E science needs to be incorporated into this emerging field to design better systems and accommodate users with special needs.
However, HF/E issues in BCI research and development, such as usability, interface design, performance modeling, and individual differences, have not been addressed. Although the concepts of the BCI systems that are used to assist users with disabilities are great in theory, the reality is that this technology presents many obstacles for the user and system. Despite the fact that BCI systems require only neurological signals to function, BCI performance decreases as physical deficiency increases (Nam, Woo, & Bahn, 2012). Felton and her colleagues (Felton, Williams, Vanderheiden, & Radwin, 2012) maintained that given that different BCI applications present varying levels of mental stress for the user, attention to HF/E can assist in minimizing such stress and subsequently lead to optimal results. Although it is true that many of the high-end, commercial-grade BCI systems can provide accurate results, offer untethered mobility, and are noninvasive, the fact remains that fatigue occurs with extended wear, the systems are difficult to use, and many devices are not suitable for real-world applications (Campbell et al., 2010; Ekandem, Davis, Alvarez, James, & Gilbert, 2012). A number of variables can hurt performance, so it is essential to factor in HF/E when designing, developing, evaluating, and implementing BCI systems.
To raise awareness of the importance of HF/E involvement in the emerging field of BCI technology, we discuss how to create a cost-effective SSVEP BCI system and how to improve SSVEP BCI systems using HF/E principles of investigation to accommodate users with special needs. This article is offered as a foundation toward a better understanding of BCI technology and to spark interest among those who were not previously aware of how this field can relate to HF/E. An example experiment is also presented to introduce how a SSVEP-based BCI can be used to investigate HF/E issues associated with the users and their ability to operate the system appropriately. Through the example experiment, we show that even a budget-friendly machine can produce many benefits.
Factoring in HF/E
The experiment described here is only one area of HF/E research that can be studied using SSVEP BCI technology; a variety of research topics can be further explored by utilizing SSVEP-based BCI paradigms. At present, many areas of research utilize SSVEP BCI technology in a very efficient manner, such as visual attention and binocular rivalry. As one might expect, visual attention is a common cognitive mechanism that can be studied through use of SSVEPs elicited from light-emitting diode (LED) lights. For example, SSVEP magnitudes elicited from a flickering stimulus on which participants focus have been measured to investigate an electrophysiological correlate of visual spatial attention (e.g., Kelly, Lalor, Reilly, & Foxe, 2005) or to assess neural responses to flicker stimuli that fall within the “spotlight” of spatial attention (e.g., Morgan, Hansen, & Hillyard, 1996).
The use of SSVEPs has also proven to be an effective method to study binocular rivalry, because it does not require active participation from human subjects. Binocular rivalry is the simultaneous presentation of two incongruent visual stimuli, such as two LEDs flickering at different frequencies, one stimulus in one visual hemifield and the second in the other hemifield (Vialatte, Maurice, Dauwels, & Cichocki, 2009). As the visual stimulus in one eye becomes dominant perceptually, the amplitude of the SSVEP to the selectively attended stimulus is enhanced (Sutoyo & Srinivasan, 2009). SSVEP-based BCI that utilizes an LED-based system, such as the one we describe in this article, allows for these visual topics to be explored relatively easily. Not only can the system be created quickly, but during testing, accurate results can be produced with minimal training.
Another topic of increased focus in the realm of SSVEP BCI is working memory. For example, SSVEPs can be used as a frequency tag, which can help in better estimating a person’s memory load through the SSVEP amplitude (Silberstein, Nunez, Pipingas, Harris, & Danieli, 2001). Given that working memory is also a popular topic of study in HF/E, it seems logical to speculate that the application of SSVEP can produce data that will ultimately help further HF/E research.
It is important for HF/E researchers to understand the required software and hardware systems so they can explore HF/E issues involved in designing, developing, evaluating, and implementing BCI systems for users with and without severe motor disabilities. However, few BCI research and development tools are available. Here we provide HF/E researchers with a brief guide on how to design and implement a cost-effective SSVEP BCI system. The example experiment shows how an SSVEP-based BCI can be studied to better accommodate users with special needs.
SSVEP-Based BCI
SSVEP BCI is a noninvasive system that uses electroencephalography (EEG) to record visually evoked potentials from the user’s scalp that result from ionic current flows within the neurons of the brain (Niedermeyer, 2005). The principle of an SSVEP BCI is shown in Figure 1A. Control input activators are presented to the user in the form of several blinking light sources with unique frequencies. While the user looks at the target light source blinking at a fixed frequency (e.g., 7 Hz), EEG signals are recorded from the electrodes positioned according to the International 10/20 system (Figure 1B). Brain signals of the same frequency (Figure 1C) are then produced in addition to corresponding second and third harmonic frequencies of the stimulus frequency (e.g., 14 Hz and 21 Hz).

(A) Principle of a steady-state visually evoked potential (SSVEP)–based brain–computer interface. Electroencephalography signals are amplified and digitized, signal processing and classification methods are applied, and a control signal is generated for the user to interact with applications. (B) The location of electrodes on the scalp, known as montage, is applied according to the International 10/20 systems. (C) SSVEP is the oscillatory wave appearing on the scalp near the occipital area (O1, Oz, and O2) of the brain in response to a visual stimulus modulated at a certain frequency (7 Hz, fundamental frequency or first harmonic) and its harmonics (14 Hz, second harmonic; and 21 Hz, third harmonic).
After processing the signals and extracting the feature, the researcher can classify the brain signals, which are finally translated into control command for the user to operate the external system. SSVEP-based BCIs have many distinct advantages over other BCI systems because they require little to no training while producing a high information transfer rate (ITR; Zhu, Bieger, Molina, & Aarts, 2010).
Hardware
Hardware systems required to implement SSVEP BCIs mainly consist of equipment to acquire brain signals, a visual stimulation device, and an application system controlled by the BCIs. Please visit http://labs.ise.ncsu.edu/bci/ for more details.
Data acquisition
Recording the electrical activity of the brain from the scalp generally requires an amplifier, an EEG cap, a set of electrodes, and other disposable items used to assist in setting up the procedure. First, an amplifier is used to record, digitally convert, and amplify bioelectric potentials associated with neuronal activity of the brain. After this information is extracted, it is transferred to a computer for further processing (Figure 1A).
Second, for multichannel recordings, an EEG cap embedded with a varying number of electrodes is often used and placed on the subject’s scalp (Figure 1A). The location of electrodes on the EEG cap is determined according to the International 10/20 systems (Figure 1B). Third, commonly used scalp electrodes consist of sintered silver–silver chloride (Ag/AgCl) disks. These disks are less than 3 mm in diameter, and long flexible leads are utilized to plug them into an amplifier (Figure 1A).
There are three primary types of electrodes: active electrodes, which do not require checking the impedances; dry electrodes, which do not require the use of electrolyte gels; and passive electrodes, which are the least expensive of the three types but require an impedance check (Fonseca et al., 2007). The active electrodes are preferred for common neurophysiologic applications because they have an output impedance of 1 Ohm, ensuring an optimal signal-to-noise ratio (i.e., passive electrodes output ranges from tens or even hundreds of KOhms).
Finally, disposable items are required to properly administer the testing. One such item is a syringe with a blunt needle, used to apply the electrolyte gels onto the scalp. One of the most effective methods to achieve low impedances is to abrade the area of skin underneath each electrode and then to apply an electrolytic gel that bridges the gap between the skin and the electrode; this results in better scalp conductivity.
Visual stimuli
SSVEP BCIs enable users to select commands by focusing their attention on the visual stimuli that change one of the stimuli’s properties (e.g., color, intensity, or pattern) with a certain frequency. Different stimulation devices can be used, such as an LCD, a CRT, and an LED. The properties of the various stimulation devices that can greatly affect the performance, applicability, comfort, and safety of the SSVEP BCIs (for details, see Bieger, Garcia-Molina, & Zhu, 2010). The findings from the Bieger et al. study suggest that when designing an SSVEP-based BCI, one should choose the properties of the visual stimuli with great care.
In short, the HF/E researcher needs to employ a visual stimulation device so that she or he can easily change its properties to investigate the settings of properties or combinations that provide the best results. The ability to easily manipulate the LED light in this device - that is, alter the color, light intensity, blink frequency, and device time - enables the researcher to test for many variables that may significantly affect not only user performance but user satisfaction as well.
Figure 2 shows a low-cost LED-based stimulation device tool kit, which our research group developed, and its circuit diagram. With this toolkit, HF/E researchers can easily build as many devices as they want. The device itself costs no more than $5 to build, which is as small as 2.1 cm × 1.27 cm. A flash memory programmed microcontroller unit (MCU) is required to control the LED-flickering function.

A low-cost LED-based stimulation device and its circuit diagram.
A PICkit 3, manufactured by Microchip Technology, was used in this system as a programmer/debugger for the programmable interface controllers (PICs). The programmed PIC is mounted onto the IC socket, which enables the developer to change the frequency of the PIC. The two-pin receptacle is used as a socket for the LEDs and makes it easy for the developer to quickly alter the LEDs for color, intensity, and even angle. The variable resistor is used to match the brightness of various LED lights, as the brightness level can vary depending on the LED manufacturer and charge level of the batteries. The battery (CR122) and battery holder make it possible for the device to function wirelessly, and the LED spacer allows the LEDs to be raised to a desired position and/or angle.
Our stimulation device tool kit is just one of the many methods an HF/E researcher can use to make a stimulation device. Many alternative devices, such as a timer IC instead of an MCU or an Arduino, would accommodate more functions.
Application system
Figure 3 shows an application system to be controlled by the user’s brain activity. As an example system, a toy robot, called Brainbot, was built using the Lego Mindstorms NXT™ kit. Brainbot is a crane-shaped structure with claw extensions with three degrees of freedom (horizontal, vertical, and grab and release) enabled by three servo motors. The station is constructed on top of a 24 in. × 26 in. veneer board coated with black paper to reduce the effect of the color of the board itself. The items on top of this foundation consist of a robotic arm, three target locations (Station 1 [S1], Station 2 [S2], and Station 3 [S3]) made of foam coffee cups, five LEDs, and a rubber ball.

Brainbot.
This setup enables users to perform five basic control movements, which include movement to the three stations (S1, S2, and S3), the grasping of a ball, and the releasing of a ball. The movement, grasp, and release commands are initiated by focusing on five corresponding LEDs, which are assigned to each individual task. Three LEDs are located on the side of the cup of each station, and the two on the board are designated for the grasping and releasing tasks. In the example task, the default frequencies for grabbing, releasing, moving to S1, moving to S2, and moving to S3 are 8 Hz, 11 Hz, 7 Hz, 6 Hz, and 13 Hz, respectively.
Software
A software system is required to acquire and process brain signals, extract features, classify the user’s intent, and translate control commands. Any programming language (e.g., C++, LabVIEW, MATLAB) can be used, but the basic framework will be the same. A LabVIEW-based program developed by our research group, called Wolfpack BCI Research and Development Tool Kit, is briefly introduced; this tool kit can support a few third-party EEG signal acquisition systems (e.g., g.tec). Please visit http://labs.ise.ncsu.edu/bci/ for more details.
An Example Study
In the following example experiment, we sought to further display the possibilities of using the SSVEP BCI system we have described as a means of researching HF/E-based issues with an emphasis on improving an interface that meets the needs of potential disabled users. As previously noted, the LED device allows the researcher to manipulate many things. In this experiment, LED color and frequency were accounted for to show how each affects not only the user’s performance but his or her satisfaction as well. In this example, we used both healthy subjects and those with ALS to show how using an SSVEP BCI–based device allows for a wider variety of participants to be tested, which can possibly lead to truly universal (even with regard to health) results in HF/E studies.
Experiment design
Twenty participants (10 able-bodied and 10 age-matched ALS patients) were recruited from the local community. A 2 × 2 × 3 mixed-factors design was used in which three independent variables were manipulated, with LED color (red and blue) and frequency (low, medium, and high) as a within-subjects factor and motor disability as a between-subjects factor.
Two dependent variables were measured: performance and brain activity. First, the user’s performance measurement takes into account accuracy, task completion time, and ITR. Accuracy is the ratio of the correct moves over the total of successful moves, and task completion time measures the time duration to perform a specified task. Moreover, ITR conveys how many bits of data can be transferred per minute, and the ITR can be calculated using the following formula (Pierce, 1980):
where N is the number of possible targets, A is the probability that the target was accurately classified (horizontal movement, vertical movement, and grab and release), and T is task completion time.
Second, brain activity during BCI operation was analyzed through STFT and low-resolution brain electromagnetic tomography (LORETA).
Procedure
Upon completing a demographic questionnaire and consent form, the participants were given basic instructions on how they should respond to the visual stimuli. Each participant’s task was to control the Brainbot to grab the ball at S2, move and release the ball at S3, grab the ball again, and move and release the ball at S1 (see Figure 3).
Data analysis
A three-way ANOVA can be performed to determine the main effects as well as two- and three-way interactions of LED color, frequency, and motor disability on performance and brain activity. For example, one of our previous studies showed that participants without motor disabilities performed significantly better in the low- and medium-frequency conditions compared with the high-frequency condition (Li et al., 2011).
Next, we turn our attention to two of the most commonly used methods of analyzing brain activity: STFT and LORETA. First, an STFT technique can be used to determine the change over time of a signal’s frequency and phase content of local sections (Figure 4A). STFT converts time domain signals into time frequency signals, which provide simultaneous time and frequency information (Flandrin, 1999). As the color renders the magnitude of the spectral power, the STFT plot provides a vivid visualization of the hotspot of brain signals in the frequency domain over time.

(A) Short-time Fourier transform (STFT) plot for five separate tasks, each graph showing changes in amplitude over time on the x-axis and evoked frequency on the y-axis. The x-axis also represents the frequency of the signal presented (i.e., target signal) and the task type performed by participants. 8 Hz (G) = 8 Hz for grab; 11 Hz (R) = 11 Hz for release; 7 Hz (S1) = 7 Hz for move to Station 1; 6 Hz (S2) = 6 Hz for move to Station 2; and 13 Hz (S3) = 13 Hz for move to Station 3. (B) Back view of the brain: low-resolution brain electromagnetic tomography image showing activated occipital areas in response to visual stimuli.
Second, LORETA is used to create a 3-D distribution of electric neuronal activity in the brain. As shown in Figure 4B, brain-imaging methods like LORETA help to identify positions where evoked potentials have occurred, such as on the scalp near the occipital area (O1 and O2) in response to visual stimuli, as in the present study.
HF/E implications
This example experiment is just one of numerous tests that can be conducted using the described SSVEP-based BCI device. Here we used the LED-based visual stimulation device to elicit brain signals as a means of controlling the Brainbot robot without the need for any physical movement by the participants. This experiment can test for differences in responses not only among healthy people but also between healthy patients and those with medical conditions. Because the experiment requires only that the subject have cognitive-based abilities, one could even go as far as to test quadriplegic subjects. Utilizing SSVEP BCI technology would enable researchers in the HF/E field to create, test, and improve interfaces for a wider variety of subjects, including those who are often overlooked because of their disabilities.
Conclusion
As with many areas of technology, it is important for researchers in different disciplines to work together to truly optimize a system. To fully utilize the emerging technology being created in the realm of BCI, it is important for more HF/E researchers to become involved. The SSVEP-based BCI paradigm can also be used as a test bed to investigate important HF/E topics; for example, visual attention and binocular rivalry.
External applications could be developed and connected to the SSVEP-based BCI system, such as robot control, navigation, communication, and entertainment games, which would make it possible to explore a multitude of HF/E topics using BCI technology. For example, the SSVEP-based BCI paradigm can be used to evaluate the brain activity of group members in the decision-making process (Eckstein et al., 2012; Poli, 2013). Regarding the applications themselves, usability testing can be conducted, and the user experience can be studied and enhanced.
Because the system works for people and without with motor disabilities, universal design guidelines can be summarized and utilized effectively. As a bridge, the SSVEP-based BCI can connect people with motor disabilities to other people, greatly improving their quality of life, empowering them with a new level of independence, and even broadening their social interactions. However, it should also be noted that as the number of visual stimuli increases, determining how to optimize their layout would be a problem for future researchers.
In conclusion, we hope that this article will raise awareness about the importance of HF/E involvement in the emerging field of BCI technology. Ultimately, we would like to promote collaboration with researchers from a variety of disciplines who have similar interests, such as usability, interface design, performance modeling, and individual differences. For more information, feel free to visit http://labs.ise.ncsu.edu/bci/.
