Brain Control: Human-computer Integration Control

Based on Brain-computer Interface Approach

1 1 1 1
Xing-Yu WANG Jing JIN Yu ZHANG Bei WANG
Abstract: Recently, a new system called brain control system has been developed rapidly. Brain control system is
a human-computer integration control system based on brain-computer interface (BCI), which relies on human0 s ideas and
thinking. Brain control system has been successfully applied in wide fields, assisting disabled patients daily life, training
patients with stroke or limb injury, monitoring the state of human operator, as well as entertainment and smart house etc.
In this paper, the background, basic principle, system structure and developments are firstly introduced briefly. The current
research status focusing on the problems of electroencephalograph (EEG) signal pattern, control signal transfer algorithm and
system application is summarized and analyzed in detail. The further research direction and problems are discussed. Finally,
the future development of brain control is analyzed and prospects are given.
Keywords: Brain control, brain-computer interface (BCI), human-computer integration control, electroencephalogram
(EEG)

Recently, a new system based on brain signal has been
developed rapidly. Comparing with the usual systems con-
trolled by computer, this system is controlled by “human
brain”. That is the brain signal is utilized to realize the
system control based on brain-computer interface (BCI)[1] .
The system of such human-computer integration control
by brain-computer interface is called brain control system.
“Brain control” relates to several fields including neuro-
science, cognitive science, control science, medicine, com-
puter science, and psychology etc. It represents a new fron-
tier as an interdisciplinary research direction and attracts
wide attentions of researchers[2] .
The original motivation of brain control is to meet the
requirement of rehabilitation for the patients with amy-
otrophic lateral sclerosis (ALS), spinal cord injury (SCI)
and other motion function disorders. The brain control sys-
tem helps these patients to operate a wheelchair or robot
by using their brain signal in order to realize some essential
movement function. Therefore, the patients can be partly
independent from daily care[3] . Besides, the development in
the fields of neuroscience, information science, biomedical
engineering, and especially brain signal and brain function
detection techniques, gives scientific evidence and techni-
cal support for brain control. During the past thirty years,
brain control has been developed rapidly and applied to
wide fields, not only in assisting daily life for disabled pa-
tients, helping rehabilitation training for stroke and extrem-
Manuscript received November 2, 2012; accepted November 8, 2012
Supported by National Natural Science Foundation of China
(61074113, 60674089, 60543005, 61203127) and Fundamental Re-
search Funds for the Central Universities (WH1114038)
Recommended by Academician Lin HUANG
Citation: Xing-Yu Wang, Jing Jin, Yu Zhang, Bei Wang. Brain con-
trol: human-computer integration control based on brain-computer
interface approach. Acta Automatica Sinica, 2013, 39 (3): 208−221
1. Key Laboratory of Advanced Control and Optimization for
Chemical Processes, Ministry of Education, School of Information
science and Technology, East China University of Science and Tech-
nology, Shanghai 200237, China
ity injury, monitoring the mental status for human opera-
tors, but also in offering new techniques for entertainment,
smart home, and military applications.
1 Brain control system
Human0 s ideas or thinking activity is not in physical form
but belonging to psychology. Therefore, it cannot be used
directly for actual system control. In order to achieve brain
control, a communication and control technique has to be
developed between human brain and computer or other
electronic devices. This technique must have two basis: sci-
entific evidence and technical support.
From the control-science point of view, brain is the con-
trol center of all the functions such as sports and language,
and gives instructions through peripheral nerves and mus-
cles to the body. Scientists of neuroscience had found
that even if the peripheral nerves and muscles are bro-
ken, the function of brain still could work normally. The
instruction performed by brain can be reflected into the
electroencephalograph (EEG). It was also found that EEG
showed some corresponding and rhythmic pattern related to
human0 s thought or evoked by external stimulus[4] . There-
fore, it is possible to translate the abstract and virtual form
of human intention into actual and physical form of EEG
signal. EEG becomes the “bridge” connecting human brain
and outside devices. The achievements in the field of neu-
roscience reveal the scientific evidence and give the working
principle for brain control.
In 1875, Richard et al. discovered EEG using rabbit0 s
scalp. In 1924, Berger recorded the EEG from human0 s
scalp, which is the milestone of the technologies based on
EEG. The effective brain activities are always covered in
EEG, which lead to low signal-to-noise ratio (SNR). It is
hard to extract the information of cognitive task from raw
EEG. In the middle of 20 th century, scientist averaged the
 Xing-Yu WANG et al./ Brain Control: Human-computer Integration Control Based on · · ·
EEG and extracted the time-locked brain response evoked
by cognitive task, which was called event-related potential
(ERP). When human received the special external stim-
ulus through his or her sensory system, ERP would be
evoked. In 1962, Galambos et al. reported the averaged
ERP obtained by computer technique[5] . In 1964, British
psychologist Walter presented the first ERP component
evoked by cognitive task called contingent negative vari-
ation (CNV)[6] . ERP was regarded as the window which
could show the brain0 s activity and has been widely used in
brain control system.
On the other hand, the development of BCI strengthens
the technique foundation for brain control. BCI is a channel
between brain and outside world without using peripheral
nerves and muscles. BCI could translate the brain pattern
containing the “mind” of human into control command to
control the external device. The control command reflects
the information from original brain activity and the inter-
active feedback between brain and computer. As a commu-
nication system, BCI mainly contains the following parts:
signal recording, signal processing and interface between
computer and external device.
Fig. 1 Structural diagram of brain control system
There are two kinds of ERP components used for brain
computer interface, which are called exogenous component
and endogenous component, respectively. Exogenous com-
ponent is evoked by physical stimuli (sound, light, and
electric) while endogenous component is related to men-
tal activity. BCI could be categorized based on the way of
EEG recordings from the human brain (invasive and non-
invasive). Invasive BCI needs a medical doctor to put elec-
trodes on the surface of cortex and record the electrocor-
ticogram (ECoG), which will involve ethical issues. Non-
invasive BCI is to put electrodes on the scalp to record
the EEG. However, the EEG is usually ten times weaker
than ECoG and requires more work to extract the features
of brain patterns. Beside ECoG and EEG, functional mag-
netic resonance imaging (fMRI), near-infrared spectroscopy
(NIRS) and magnetoencephalogram (MEG) were also used
209
for BCI. However, due to the cost of the device and the per-
formance of the signal, EEG is mostly used for BCI system.
In this paper, we will give a review for EEG-based BCI.
At present, the brain control system mainly contains
three aspects. The first aspect is the relationship between
human0 s thinking or ideas and the EEG signal. We should
know what kind of EEG pattern corresponds to one certain
thought or stimulus. The second is the design and construc-
tion of brain-computer interface. The third is the develop-
ment of application system. The proper systems should be
designed and developed according to different tasks or envi-
ronments. The structural diagram of brain control system
is shown in Fig. 1.
2 Overview of BCI development
In 1973, Vidal et al. from University of California, Los
Angeles (UCLA) did a research on the communication be-
tween brain and computer, which is the first step for real
BCI system[3] . In 1991, Wolpaw et al., reported that the
amplitude of mu rhythm could be controlled by subjects
and could be used to control external device (i.e., brain
control)[7] . In 1995, the number of BCI teams was not more
than six. And it became more than twenty in 1999, more
than forty in 2002, more than one hundred in 2007 and even
more until now[8] . There are two important conferences for
BCI, one is held by Jonathan R Walpow every three years
in America called “BCI meeting”. The first conference of
“BCI meeting” was held in June 1999[1] and the second one
in June 2002. There were about thirty BCI teams and one
hundred researchers who joined this meeting[9] . Another
one is held by Gert Pfurtscheller every two years in Graz
Austria. Its fifth conference was held in 2011.
With the development of brain control, some important
research results were obtained. In 1999, Nature published
the paper on BCI for the first time. This work presented
the BCI spelling system based on SCP[10] . In 2003, Na-
ture reported an experiment from Duke University, which
was done by Nicolelis[11] . In this experiment, they guided
a monkey to control robot using its brain signal. In the
same year, Pfurtscheller from Graz University of Technol-
ogy made a non-invasive restoration of hand grasp function
in a tetraplegic patient by EEG-recording and functional
electrical stimulation (FES) using surface electrodes. After
five months0 training, the patient could use this device to
catch a cup and drink[12] .
This kind of brain control system is based on EEG.
The brain patterns recently used for BCI system contain
slow cortical potential (SCP), motor imagery (MI), steady
state visual evoked potential (SSVEP) and so on. SCP
and event-related (de)synchronization (ER(D)S) are spon-
taneous brain activities related to specific mental tasks,
while SSVEP and P300 are evoked potential caused by ex-
ternal stimuli. SCPs are negative or positive polarizations
of the EEG or magnetic field changes in the MEG that
usually last from 300 ms to several seconds. Subjects could
210 Acta Automatica Sinica, 2013, Vol. 39, No. 3
be trained to control the SCP by biofeedback. In 1957,
Brookhartd et al. found the SCP for the first time[13] . In
1990, Birbaumer et al. described the SCP in detail[14] . In
1999, Birbaumer et al. first applied the SCP to BCI and
built a spelling system based on the SCP[10] .
P300 potential is evoked around 300 ms after low proba-
bility stimulus. If a rare stimulus happened in a sequence of
similar stimuli (i.e. oddball paradigm), a positive deflection
called P300 potential would be evoked at central parietal
area for around 300 ms after this stimulus. P300 mainly ap-
pears at central vision area (Pz). In 1965, Sutton first found
P300 potential[15] . In 1988, Farwell and Donchin developed
the first P300 spelling system[16] . A 6 × 6 matrix consisting
of characters and numbers was used as a stimulus layout, in
which every row and column would be flashed once in each
run. P300 has been widely applied to BCI system. For ex-
ample, driving based on traffic light[17] , dialling telephone
number[18] , robot arm control[19] , mouse control[20−21] ,
wheelchair control[22] , robot control[23] , household electri-
cal device control[24] and so on.
In 1977, Professor Pfurtscheller found the phenomenon
of ERD by asking the subject to press the button using
right and left thumbs. This experiment explained the ERD
phenomenon of mu component in alpha band at the cen-
ter of brain area[25] . In 1993, the BCI system based on
ERD was developed and obtained the accuracy of 85 %[26] .
In 1999, Professor Pfurtscheller found the phenomenon of
ERS[27] , which mainly occurred at central motion region
(C3, C4, and CZ). Now, ERD(S) has been used to develop
the system for assisting limb injury patients[12] , controlling
keyboard[28] , controlling virtual wheelchair[29] and so on.
Steady state visual evoked potentials (SSVEPs) are nat-
ural responses to visual stimulus at specific frequencies be-
tween 3.5 Hz and 75 Hz. When the retina is excited by a
visual stimulus, the brain generates electrical activity at
the same (or harmonic) frequency of the visual stimulus.
In 1973, Vidal first introduced SSVEP to BCI[3] . Now,
SSVEP-based BCI has been mainly applied to computer
game[30−31] , mouse control[32] , speller[33] and robot arm
control[34] . Movement related potentials (MRPs) appeared
when subject did the motor imagery of left hand, right
hand, foot, or tongue. Birch and Levine applied MRPs
to offline BCI system, respectively[35−36] .
Since BCI is interdisciplinary technology, researchers on
brain control are with different background and do the re-
search with different perspective. Researchers who are ma-
jor in neurophysiological, neural information engineering,
rehabilitation medicine, biomedical engineering, control sci-
ence and engineering, electronic engineering, instrumenta-
tion, machine learning, mathematics commit themselves to
the research of brain control which has been a hot topic.
Ten years ago, our team with the background of control
science and engineering started to do the research on brain
control. Now, beside some famous Universities and insti-
tutes, there are also some big companies that invested for
developing BCI system. Furthermore, there are many news
reports about BCI. For example, Discovery magazine re-
garded BCI as one of the seven most important technologies
in the world in 2006. Time magazine also reported that BCI
was one of the fifty most valuable inventions in the world
in 2009.
3 Current developments and main
problems
From the view of control science, the problems that need
to be solved for brain control bring new challenges. At
present, brain control is still in its primary stage and faces
many problems that are hard to be solved including ba-
sic science and technique problems. The following is the
analysis based on current developments and existing main
problems.
3.1 Research on brain patterns
As mentioned above, several brain patterns were used
to build the BCI. However, it is hard to decode ideas and
thoughts from EEG and only a few brain patterns could be
successfully applied to the BCI. Therefore, the research on
surveying the new brain pattern is very important. Now,
there are mainly three ways to solve this problem, i.e. im-
proving the evoking paradigm of existing brain patterns,
combining several brain patterns called “hybrid”, and find-
ing new brain pattern based on neuroscience and cognitive
science.
Slow cortical potential (SCP) is one of the brain patterns
used in BCI. However, the users need to be trained for sev-
eral months before he/she can control the amplitude of SCP.
It may be impractical for the real application of BCI. The
ERD (S) or MRPs based on motor imagery is one of the hot
research topics in BCI, which is related to the physiological
phenomenon and has advantage in direction control. The
ERD (S) or MRPs can be used to build the BCI system to
train the patients who are recovering from the stroke[37−39] .
However, this kind of signals has drawbacks: unstable, low
recognizing speed, few patterns, long training time, narrow
group of users and so on.
The advantages of visual evoked potentials (VEP) are
rich coded commands, stable pattern, high classification ac-
curacy and high information transfer rate, which make VEP
become a hot research topic in BCI. Bin and Liu et al. did
much work on SSVEP and presented many new methods
for SSVEP-based BCI. For example, they applied canonical
correlation analysis (CCA) to SSVEP recognition[40−41] and
used SSVEP-based BCI to dial telephone number. Zhang
et al., surveyed new paradigm for SSVEP and developed
a system using coded multi-frequency with few frequencies
to realize multi-target control[42] . Jin et al., combined the
time and frequency to control multi-target with few frequen-
cies and decreased the interference of neighbor stimuli[43] .
However, SSVEP may be affected by the form, configura-
tion and sequence of stimulus, and also by the human0 s
 Xing-Yu WANG et al./ Brain Control: Human-computer Integration Control Based on · · ·
sensory capacity. The VEP is not suitable for blind users.
Although auditory sense and somatosensory could be used
by blind users, they are relatively weak and hard to be
classified[44−45] .
P300 potential has wide group of users, could be classi-
fied with high accuracy, has plenty of coded orders and need
no training. However, there are still some unsolved prob-
lems for P300-based BCI. The paradigm for evoking P300
potential will affect the quality of evoked potential, and
hence classification accuracy. Now, the researches mainly
focus on the delay of flash time[46−52] , the form of stim-
uli (stimuli matrix[53−54] , configuration of stimuli[51, 55−56] ),
the characteristics of stimuli [46, 57−58] and the sequence of
stimuli[59−61] and so on.
In 2002, Gonsalvez and Polich found that target to
target interval (TTI) will affect the amplitude of P300
potential[47] . Based on previous work, Hill et al.[48] , in 2008
found that using the same TTI without traditional row and
column pattern could not obtain the accuracy as high as
that using traditional row and column pattern. They also
presented a code book to show the interval and sequence
of flashes. Martens et al., found that the overlap of P300
wave was stronger when using short TTI than using long
TTI. If the TTI for the same target is shorter than 500 ms,
repetition blindness would happen[49−51] . In order to solve
these problems, reference [52] presented a method to avoid
double flashes. In 2010, McFarland et al.[62] , did the ex-
periment which used different flash frequencies to find the
best flash frequency. The configuration of flashes affects
the information transfer rate for P300-based BCI. The non-
target flashes around target flashes will distract the atten-
tions of subjects, when the interval between target flashes
and non-target flashes is less than 500 ms, which is called
attentional blink[50−51] . However, we can decrease the non-
target flashes around target flashes to avoid the attentional
blink[55] . In 2010, Liu et al.[56] developed the gaze indepen-
dent P300-based BCI.
The research on the stimulus sequence is a new topic in
BCI. In 2006, Gerson researched the problem where it was
required from the subject to search the different picture
from a sequence of similar pictures, which would evoke ERP
when subject caught the target picture. This paradigm
is called rapid serial visual presentation (RSVP)[60] . This
kind of paradigm will be suitable for the patients who could
not move their eyes. In 2011, Treder et al. improved this
method and applied it to the speller through which the
severely locked-in patients also could use speller BCI[61] .
The research team of East China University of Science
and Technology has done a lot of work on BCI[63] . In view
of above problems, Jin et al. used math combination to
reduce the flash time for each flash run and increase the
information transfer rate[64] . The goal of this work is to
optimize the large matrix to obtain the best flash time for
each run and solve the “double flashes” problem without de-
creasing the speed of BCI system[65] . Optimized stimulus
matrix was used to decrease the interference from neigh-
211
bor flashes and adaptive strategy was used to increase the
information transfer rate. The method for avoiding dou-
ble flashes and the method for decreasing the interference
from neighbor flashes were integrated together[55] . The op-
timized interval of flashes was used to improve P300-based
BCI system and increase the quality of evoked potential,
which would decrease the overlap of the potentials[66] . Jin
et al. developed the Chinese characters input BCI system
with T9 Chinese input method to realize the communica-
tion between computer and mobile phone[67] .
Multi-ERP and Hybrid BCI could improve the stability
and flexibility of BCI, and also enlarge the group of users.
In 2011, Kaufmann et al.[68] used the face picture as stim-
ulus and their results showed that the familiar face could
improve the classification accuracy of BCI. In 2010, Alli-
son et al. combined the motor imagery and SSVEP to im-
prove the online performance of BCI[69] . In 2011, Brunner
compared the SSVEP-based BCI with the hybrid BCI of
SSVEP and motor imagery[70] . In the same year, Horki et
al. applied the hybrid BCI of SSVEP and motor imagery to
robot arm control[71] . In 2012, Allsion et al. applied hybrid
BCI of SSVEP and motor imagery to mouse control using
synchronization control mode[72] .
The Tsinghua University BCI team used moving stimu-
lus to replace flashing stimulus and reported that this kind
of BCI could be better than P300-based BCI. In 2012, Jin
et al. presented a method combining the flash and move-
ment characteristics, which showed that it could improve
the performance of BCI further[76] . In 2011, Kaufmann et
al.[68] used familiar face as stimulus and their results showed
that familiar face could evoke large N400 which would in-
crease the classification accuracy a lot. Professor Li Yuan-
Qing combined the P300 and motor imagery and succeeded
in controlling the direction and speed of wheelchair[77] . In
2012, Zhang et al.[78] presented that inverse face could evoke
larger vertex positive potential, (VPP), which was com-
bined with N170 and P300 to improve the performance
of BCI system. After that, Jin et al.[79] compared the
paradigm of shaking face with the paradigm of stable face
and their results showed that late ERP could be classified
with high accuracy, while it could not improve the classi-
fication accuracy further when it was combined with early
ERP.
3.2 Control signal translation algorithm
Another critical issue in the BCI researches is how to
translate the EEG signals into corresponding control com-
mands. Algorithm realizing the translation is called con-
trol signal translation algorithm. Brain activities contain
the information of human0 s ideas and thought. Translation
algorithm is an important tool to detect these brain activi-
ties. If the BCI is referred as a pattern recognition system,
it will mainly consist of three research aspects: preprocess-
ing, feature extraction and classification. Feature extrac-
tion and classification are the two crucial factors for BCI
performance. In principle, most methods that have been
212 Acta Automatica Sinica, 2013, Vol. 39, No. 3
introduced for feature extraction and classification in ma-
chine learning could be applied to the BCI. However, the
application of these methods should be fit to the partic-
ularities of the EEG signals and the BCI system. Such
particularities are mainly reflected by the three facts: 1)
the EEG signals are very weak and easy to be affected by
other electrical signals, such as electrocardiogram (ECG),
electromyography (EMG) and electrooculography (EOG);
2) the EEG signals are usually varied among different sub-
jects since they are not only closely related to the thoughts
or intents, but also affected by the mental state of the in-
dividuals; 3) the extracted EEG features are closely related
to the channel configuration. Therefore, the translation al-
gorithm for control command should be developed by con-
sidering the above particularities in EEG signals.
3.2.1 Feature extraction algorithm
The aim of feature extraction is to extract the informa-
tion reflecting certain brain activity. For an EEG-based
BCI, we should first consider how to extract the most effec-
tive features and what the characteristics of such features
are, so that we can design an appropriate classification al-
gorithm to classify the features with high performance. At
present, the commonly used features in BCI are: temporal
amplitude, power spectral density (PSD), adaptive autore-
gressive (AAR) parameter, time-frequency representation,
etc. According to different types of BCIs, the features re-
flecting the most strongly exploited EEG pattern should be
extracted for the subsequent classification.
The temporal features are usually extracted by using a
specific filter with down-sampling that removes the tempo-
ral noises to enhance the signal-to-noise ratio (SNR). The
most popular temporal features in BCIs are amplitude and
band power (BP). The commonly used filtering methods
contain band-pass filtering[21] , Kalman filtering[80] , moving-
average filtering[78] , etc. In addition, continuous/discrete
wavelet transform (CWT/DWT) is also usually adopted to
extract the temporal features from the EEG within a short
duration[81−82] .
Spatial feature extraction, also called spatial filtering,
aims to extract the features with higher SNRs through
multiple channels optimization to improve the classifica-
tion performance. Due to the effects of volume conduc-
tion, the EEG signals recorded from the scalp usually have
poor spatial resolution[86−87] . Common average reference
(CAR) and Laplace filter are the two simplest and most
used spatial filtering methods. The spatial filtering algo-
rithms used frequently for feature extraction include prin-
ciple component analysis (PCA)[88] , independent compo-
nent analysis (ICA)[89] , common spatial pattern (CSP)[90] ,
Fisher0 s criterion (FC)[86, 91] , canonical correlation analysis
(CCA)[40−41] , etc. Both PCA and ICA are unsupervised
spatial filtering algorithms. PCA extracts the features that
are orthogonal to each other with the minimal energy loss,
while ICA extracts the features that are independent to
each other. CSP, FC, and CCA are supervised spatial fil-
tering algorithms. CSP extracts features having the max-
imal variance difference between two classes; FC extracts
features having the maximal discriminant information be-
tween two classes; CCA extracts features having the max-
imal correlation within the same class. Note, the features
extracted by the aforementioned algorithms may not be the
real source signals in EEG, but the features fitting to the
constraints by these algorithms.
The spatial filtering algorithms mentioned above are
basically matrix factorization methods based on two-
dimensional subspace. The EEG signals used in a brain
control system contain typically multi-way data informa-
tion, such as time domain, frequency domain, spatial (chan-
nel) domain, variability among trials, and specificity among
subjects. Simultaneous analysis for the multi-dimensional
information could be effective to excavate some underlying
characteristics in the EEG signals, and hence assist to clas-
sify features more accurately, which is, however, hard to
be implemented using these matrix factorization methods.
Thus researches on multiway analysis algorithm are very
important for improving the BCIs.
A class of multiway algorithms based on tensor analy-
sis was designed to extract collaboratively more effective
features from multi-dimensional information of the EEG
signals. Such multiway analysis process provides neu-
rophysiologically meaningful interpretation for extracted
feature[91−93] . The multi-way algorithms which have been
applied to feature extraction in BCIs mainly include: mul-
tilinear PCA[94] and multilinear CSP[95] for ERD/ERS
feature extraction, multiway CCA for SSVEP feature
extraction[96] , etc. Tensor discriminant analysis was in-
troduced to motor-imagery based BCI and improved the
pattern recognition performance[97] . Zhang et al.[96] devel-
oped a multiway CCA-based SSVEP recognition algorithm,
which not only optimized the spatial features but also re-
duced effects resulted from variability among trials, through
maximizing alternatively the correlation between the spa-
tial dimension as well as trial dimension of the EEG signals
and the reference signals.
These multiway extension versions outperform the tra-
ditional matrix factorization algorithms in varying degree
for feature extraction of the corresponding BCIs. Superior
results indicate that the multiway algorithms will likely ex-
tract more discriminative features to improve classification
performance. However, such multiway algorithms still have
not been widely applied to BCIs. More systematic stud-
ies are required to investigate the advantages and disad-
vantages of the multiway algorithms for the EEG feature
extraction further.
3.2.2 Classification algorithm
After extracting effective features from the EEG signals,
another critical issue is how to select or design an appropri-
ate classifier for classification. Currently, the classification
algorithms which have been used in BCIs mainly are: lin-
ear discriminant analysis (LDA), regularized LDA (RLDA),
 Xing-Yu WANG et al./ Brain Control: Human-computer Integration Control Based on · · ·
support vector machine (SVM), Bayesian classifier, hidden
Markov model (HMM), neural network (NN), and ensemble
classifier, etc. This section will give a brief review for these
classification algorithms, and provide a basic knowledge for
further analysis.
Linear discriminant analysis (LDA)[98−100] is to learn
an optimal hyperplane separating the features from two
classes. Currently, the LDA has been widely adopted in
various BCI systems[100−102] since it has relatively low com-
putational requirement and usually provides good classifi-
cation results. Unfortunately, the LDA is hard to classify
some complex and nonlinear EEG signals accurately, and
also will most probably result in poor generalization perfor-
mance for the classifier due to the curse-of-dimensionality
in small sample size setting.
Regularized LDA (RLDA) tries to enhance the gener-
alization capacity of classifier by imposing specified con-
straint to penalize the training process in the traditional
LDA. Several typical RLDAs used in BCIs are Stepwise
LDA (SWLDA)[16] , Shrinkage LDA (SKLDA)[87] , Bayesian
LDA (BLDA)[24] , and LASSO regression model[103] . The
SWLDA performs feature dimensionality reduction through
a combination of forward and backward stepwise analy-
sis in which the discriminant model is estimated by it-
eratively adding and removing features from the model
based on statistical tests until the maximal number of
features is achieved. The SWLDA has recently been a
state-of-the-art classification algorithm used in the P300-
based BCIs[104−105] . The SKLDA adjusts the estimated
covariance matrix to the real one using a shrinkage
technique[106] to improve the generalization capacity of the
trained classifier. The analytical solution of the shrinkage
parameter can be computed according to one method in-
troduced in literature[87] . The SKLDA has been applied to
the motor-imagery based BCI[107] and ERP-based BCI[87]
with outstanding classification performance for small train-
ing sample size. The BLDA prevents overfitting to high
dimensional and possibly noisy datasets using regulariza-
tion whose degree is estimated automatically and quickly
from the framework of Bayesian analysis[24] . Recently, the
BLDA has been successfully applied to the P300-based BCI
and outperformed the traditional LDA[3, 67] . The LASSO is
to penalize the regression weight vector by imposing the l1 -
norm (sparsity) regularization on a standard linear regres-
sion model[103] . Larger regularization parameter will result
in more zero entries in the regression weight vector. Zhang
et al.[103] designed a LASSO-based frequency component
recognition model for SSVEP by exploiting such sparsity.
The LASSO-based model learns the optimal discriminant
vector to recognize directly the SSVEP component, by im-
plementing simultaneously penalized linear regression be-
tween the EEG signals and reference signals corresponding
to different stimulus frequencies.
Support vector machine (SVM) is also to find an op-
timal hyperplane separating features from different two
213
classes. However, the selected hyperplane in the SVM is the
one that maximizes the margins between the two classes,
which is also named maximum margin hyperplane. The
SVM are known to be insensitive to overtraining and have
good generalization capacity in small sample size. Cur-
rently, the SVM has been widely applied to various BCIs
and achieved good classification results[108−109] . In addi-
tion, several variants of the SVM have also been developed
and introduced to classification in BCIs, such as the sensor
weighting SVM (sw-SVM) for P300 classification[110] and
fuzzy theory-based SVM for ERD/ERS classification[111] ,
etc.
Bayesian classifier assigns to a feature vector the class it
belongs to with the highest posterior probability[98] . The
Bayesian classification is equivalent to the standard LDA
when the two classes are Gaussian with the same covari-
ance, while it forms a Bayesian quadratic when the two
classes have different Gaussian distributions. Although the
Bayesian classifier has still not been widely adopted in
BCIs, it has been successfully applied to the ERD/ERS
classification[112] .
Hidden Markov model (HMM) is a kind of probabilistic
automaton that can provide the probability of observing
a given sequence of feature vectors, where the probability
usually is Gaussian mixture model for BCI[113] . The HMM
has recently been applied to the classification of tempo-
ral sequences of BCI features since EEG components used
to drive BCI have specific time courses[113] . However, the
HMM still has not been widely applied to BCIs.
Multilayer perceptron (MLP) is probably the most popu-
lar one neural network algorithm for BCIs. The MLP is uni-
versal approximator which can approximate any continuous
function when composed of enough neurons and layers, and
hence has been successfully used in various BCIs[114−115] .
However, the MLP is sensitive to overtraining, especially
with such noisy and non-stationary data as EEG, thereby
deteriorating the generalization capacity of the trained clas-
sifier to some extent[116] . Thus structures of neurons should
be carefully designed when applying the MLP to BCIs.
At present, most classification schemes adopted in BCIs
are based on one single classifier, which may, however,
hardly achieve the optimal performance for a specific classi-
fication issue due to the intrinsic defects of the used classi-
fier. To overcome this problem, ensemble learning has been
considered as a feasible approach. The ensemble methods
used in BCIs are either combination of multiple classifiers
that have the same type but different model parameters[117] ,
or combination of several types of classifiers[118] . The en-
semble classifier may help to remedy the defects of each
single classifier by combining multiple classifiers with good
diversity, and hence to improve the classification perfor-
mance for EEG signals.
In summary, although a few characteristic classification
algorithms have been proposed for translation of control sig-
nal, the most challenging problems, i.e. curse of dimension-
214 Acta Automatica Sinica, 2013, Vol. 39, No. 3
ality, overfitting and low adaptability still have not been ef-
fectively solved. The number of training EEG data acquired
from the calibration stage is usually limited when taking
into account the practicability of the BCI system. The
EEG feature vectors are generally high dimensional since
the EEG data are recorded from multi-point and multi-
channel. This will most probably result in the so-called
curse of dimensionality. That is, it is hard to learn good
classifier with high generalization capacity using insufficient
but high-dimensional features. On the other hand, the rel-
atively complicated classifiers (e.g., MLP) outperform some
simple classifier (e.g., LDA) in EEG classification with non-
linear features, whereas they are sensitive to noise and be-
come overfitting to the training data with poor generaliza-
tion capability. In addition, the EEG features are usually
subject-specific and also vary along with time to some ex-
tent. Although the commonly used LDA and SVM gener-
ally work well with low variance, they can hardly keep good
performance after long time use of BCI due to their poor
adaptability.
We think there are two important aspects which should
be mentioned for designing the translation algorithm: 1) In
order to improve the practicability of the BCI, the system
calibration time should be kept as short as possible with
minimal classification performance degeneration. This re-
quires an effective algorithm to learn a classifier with good
generalization capacity from limited training sample set; 2)
During long time use of the BCI, the EEG feature signals
may vary to some extent along with time. The classifier
trained at the initial calibration stage will hardly provide
good classification results for the feature signals recorded
after long time use. Thus the designed classifier should up-
date adaptively its model parameters based on the variety
of EEG features to achieve accurate classification. To this
end, we could either design an ensemble classifier by com-
bining multiple existing classifiers to improve the robust-
ness to variety of EEG features, or introduce the adaptive
learning[119] into classifier design to enhance the adaptabil-
ity and prevent the effects resulting from variety of EEG
features.
3.3 Application system
So far, the researches of the brain control system rarely
focus on the actual application, but still mostly are at the
laboratory study stage. However, compared to the early
studies of offline BCIs, the real-time control and feedback
of many BCI systems have been realized recently, and some
actual application systems have also been developed. The
actual applications of BCI are mainly related to medical
field and non-medical field. In the medical field, the BCI
aims to re-establish communicative and environmental con-
trol approaches, such as intelligent wheelchair, neural pros-
thesis, virtual speller and humanoid robot, etc. for disabled
users with normal brain function. In the non-medical field,
the BCI has been applied to the detection of operator func-
tion state, game entertainment and smart home, etc. The
successful application in game entertainment may be the
most attractive, since the users can operate the mouse, key-
board and control stick directly without muscle activity to
realize the collaboration and competition in the game.
In terms of practicability, the current BCI development
also faces some other technical problems, beside the afore-
mentioned issues about EEG pattern investigation and
translation algorithm design. These problems can be sum-
marized as:
1) Improvement of the EEG recording devices. The most
important problem for BCI is to record the EEG signals
effectively, which is the one of key points deciding whether
the system can work efficiently. EEG recording devices,
such as sensors and amplifier, etc. could be innovated to
prevent interference from the non-EEG signals and effects of
the skull on the EEG signals, and hence enhance the quality
of recorded EEG signals. In addition, the practicability
and portability of the devices need further improvement.
Recently, several companies have developed portable EEG
recording devices[120−121] that make important progress in
the practicability of BCI, although these portable devices
still require further improvement.
2) Security of the brain control system. The personal
safety is the most important factor since the brain control
system is a high-level human-machine integration system.
In recent years, a concept of “brain switch” has been pro-
posed to meet the requirement of security, which could play
an important role in BCI to avoid invalid commands re-
ceived at the idle state. For instance, some mistakes and
unnecessary commands may bring unexpected danger to the
users when they are operating the BCI-based wheelchair or
prosthesis control system. Such things are particularly dan-
gerous to disabled users since it is difficult for them to stop
the control system immediately by physical motion. Thus,
how to choose a safe and reliable EEG pattern as brain
switch signal for specific subject is worth researching[122] .
3) Human-machine adaptability of the brain control sys-
tem. a) The performance of BCI is usually subject-specific,
which should be addressed to improve the universality of
the BCI; b) The status of the users should also be taken
into account to improve the performance of BCI. The sta-
tus and feelings of the users may affect the performance of
BCI, since the BCI system is inherently based on human.
Therefore, the human factor should be considered when de-
signing the paradigm and selecting the parameters of BCIs,
to develop a highly user-friendly brain control system.
4) A unified standard for BCI performance evaluation.
There is still no complete theoretical framework and reg-
ularity for BCI performance evaluation, since the current
researches on BCIs are still mostly at the laboratory study
stage. The current used evaluation standards mainly in-
clude information transfer rate (ITR), system delay and
response time, mental workload and user friendliness, etc.
However, in terms of commercialization, more scientific
standard for BCI performance evaluation is required.
 Xing-Yu WANG et al./ Brain Control: Human-computer Integration Control Based on · · ·
4 Prospect
After nearly four decades, the research of brain control
system has been developed rapidly. The brain control, as an
interdisciplinary new research field, not only has the broad
development prospects but it also faces many challenges.
The followings are the prospects of future development for
brain control system.
1) Investigation of novel EEG patterns. a) A feasible
method for improving BCI is to elicit more discrimina-
tive brain activities by optimizing or combining the exist-
ing brain patterns; b) Simultaneous evoking through vi-
sual, auditory, and tactile approaches may elicit brain ac-
tivity with higher discriminability, thereby improving the
BCI performance further; c) Another important approach
is to discover novel EEG patterns based on psychophysi-
ology, in which psychological factor is regarded as the in-
dependent variable, physiological factor is regarded as the
dependent variable and human is the control object. Af-
ter 50 years research, scientists have discovered various
ERP components[123] that are related to attention, per-
ception, decision and working memory updating, etc., re-
spectively. Although only several ERP components have
been exploited for BCI development, other more ERP com-
ponents also would be applied to BCI in the near future.
Thus, the research on the psychophysiology could assist to
investigate and discover new EEG patterns for developing
new BCl paradigm.
2) Study on new translation algorithms. Effective trans-
lation algorithm with capabilities of multiway analysis and
adaptation should be designed by exploiting the new the-
ories and methods of the pattern recognition and machine
learning[124−125] , to overcome problems caused by the curse
of dimensionality, overfitting and low adaptability, etc. The
multiway analysis methods (consist of feature extraction
and classification) could provide more real and natural
structure representation for the EEG signals. Specific con-
straint or regularization (e.g., independence, maximal cor-
relation, highest discriminability, and sparsity, etc.) could
be imposed on the corresponding dimension of the multi-
way structure to extract features with reliable neurophysi-
ological meaning, and hence enhance further robustness of
feature classification. On the other hand, the distribution
of EEG features is not only subject-specific but also varies
along with time to some extent during the real-time applica-
tion. To overcome these problems, the concept of adaptive
learning in the cybernetics could be exploited to optimize
the classification model, so that it has not only reliable self-
learning ability for individual but also high generalization
capability within group. Combination of multiway analy-
sis and adaptive learning may improve the performance of
the trained classifier further. Thus how to design a multi-
way analysis algorithm with excellent adaptation is consid-
erably important for BCI research. Recently, Jin et al.[126]
proposed an adaptive calibration method using online LDA
with a pre-trained classification model. This adaptive cal-
215
ibration method works successfully for most subjects and
avoids the time-consuming procedure of offline calibration
for individual effectively.
3) Intelligence of brain control system. The close com-
bination between human brain and computer in the brain
control system is reflected by the integration of biological
intelligence and artificial intelligence. Hence, the brain con-
trol system could be a medium for researches on both the
biological intelligence and artificial intelligence. Revealing
the mechanism of biological intelligence and exploring the
limitation of artificial intelligence are always the challenges
to science. The study on intelligence of brain control system
may contribute to solve the aforementioned problems, and
provide a new insight in the intelligent control and artificial
intelligence machine. On the other hand, improvement and
optimization for the brain control system may also benefit
from the intelligence study. For instance, most of the ex-
isting BCIs simply implement tasks step-by-step with poor
execution efficiency and low safety. Intellectualization of
control device could optimize the control procedure to re-
alize the required tasks from the user with only a few com-
mands, and enhance the overall performance of the brain
control system.
4) Networking of brain control system. Currently, the
brain control systems have mainly been developed for indi-
vidual operation. However, a network system is required for
joint control or remote control. The networking integrates
human brain, machine and network into a control system,
which may lead to some network control problems on feed-
back, stability, robustness and delay, and hence affect the
status of the user to yield poor classification performance.
Such complex control system will bring more difficult but
valuable research issues.
5) Expanded new application fields. The current appli-
cation of BCI technique is mainly in the field of rehabilita-
tion. Recently, there is also a direct brain controlled robotic
technique developed for robot control field[127] . Actually,
the BCI technique not only brings benefits to disabled pa-
tients, but also assists normal persons by providing another
“imaginary hand”. Such novel function could fully take
advantage of human ability and make the brain control sys-
tem more useful. For instance, according to special require-
ments under some working environments, multi-objective
and concurrent control operations can also be realized by
brain control when both of hands are occupied, in order to
save the labor and improve the convenience of operation
effectively.
6) Theory and methods on human-machine integra-
tion system. The brain control system is a very special
human-machine integration system. In view of the con-
trol mechanism, this system obeys the human0 s “mind” and
“thought”. In terms of the system itself, the human is not
only the controller but also the object for controlling while
the machine is not only the executor of control command
but also the guide of control generation. In general con-
trol system, human is independent from system structure
216 Acta Automatica Sinica, 2013, Vol. 39, No. 3
and the factors related to human usually could be ignored.
However, with regard to the brain control system, there is
still no in-depth study about how to model, analyze and
evaluate the human factors, which, however, are worth fur-
ther investigating. New theory and approach should be
developed to model and optimize the human-machine in-
tegration system since the traditional control theory and
method cannot be directly applied to solve such problems.
As Wiener, the founder of cybernetics, wrote: “If a new
scientific subject has real vitality, the center of interest in
it must and should shift in the course of years · · · . Thus
it behooves the cyberneticist to move on to new fields and
to transfer a large part of his attention to ideas which have
arisen in the developments of the last decade”[128] . Brain
control is generated and developed based on multidiscipline
interaction, and its further development will certainly un-
dergo new progress remarkably.
Acknowledgement
The authors appreciate Academician Lin Huang et al. for
their valuable comments and advices.
References
[1] Wolpaw J R, Birbaumer N, Heetderkd W J, McFarland
D J, Peckham P H, Schalk G, Donchin E, Quatrono L
A, Robinson C J, Vaughan T M. Brain-computer inter-
face technology: a review of the first international meeting.
IEEE Transactions on Rehabilitation Engineering, 2000,
8(2): 164−173
[2] Wang X Y. Automatic control: virtuality vs. reality. Acta
Automatica Sinica, 2002, 28 (Suppl): 77−84
[3] Vidal J J. Toward direct brain-computer communication.
Annual review of Biophysics and Bioengineering, 1973, 2:
157−180
[4] Gazzaniga M S, Ivry R, Mangun G R. Cognitive Neuro-
science. New York: W. W. Norton and Company, Inc., 2002
[5] Galambos R, Sheatz G C. An electroencephalography study
of classical conditioning. American Journal of Physiology,
1962, 203(1): 173−184
[6] Walter W G, Cooper R, Aldridge V J, McCallum W C,
Winter A L. Contingent negative variation: an electric sign
of sensori-motor association and expectancy in the human
brain. Nature, 1964, 203(4943): 380−384
[7] Wolpaw J R, McFarland D J, Neat G W, Forneris C A.
An EEG-based brain-computer interface for cursor con-
trol. Electroencephalography and Clinical Neurophysiology,
1991, 78(3): 252−259
[8] Mason S G, Bashshati A, Fatourechi M, Navarro K F, Birch
G E. A comprehensive survey of brain interface technol-
ogy designs. Annals of Biomedical Engineering, 2007, 35(2):
137 −169
[9] Vaughan T M , Heetderks W J, Trejo L J, Rymer W Z,
Weinrich M, Moore M M, Kübler A, Dobkin B H, Bir-
baumer N, Donchin E, Wolpaw E W, Wolpaw J R. Brain-
computer interface technology: a review of the second in-
ternational meeting. IEEE Transactions on Rehabilitation
Engineering, 2003, 11(2): 94−109
[10] Birbaumer N, Ghanayim N, Hinterberger T, Iversen I,
Kotchoubey B, Kübler A, Perelmouter J, Taub E, Flor H. A
spelling device for the paralysed. Nature, 1999, 398(6725):
297−298
[11] Nicolelis M A L. Brain-machine interfaces to restore motor
function and probe neural circuits. Nature Reviews Neuro-
science, 2003, 4(5): 417−422
[12] Pfurtscheller G, Müller G R, Pfurtscheller J, Gerner H J,
Rupp R. “Thought”-control of functional electrical stimu-
lation to restore hand grasp in a patient with tetraplegia.
Neuroscience Letters, 2003, 351(1): 33−36
[13] Brookhart J M, Arduini A, Mancia M, Moruzzi G. Slow
cortical potentials induced by thalamic stimulation. Bollet-
tino Della Societa Italiana di Biologia Sperimentale, 1957,
33(10−11): 1629−1630
[14] Birbaumer N, Elbert T, Canavan A G, Rockstroh B. Slow
potentials of the cerebral cortex and behavior. Physiological
Reviews, 1990, 70(1): 1−41
[15] Sutton S, Braren M, Zubin J, John E R. Evoked-
potential correlates of stimulus uncertainty. Science, 1965,
150(3700): 1187−1188
[16] Farwell L A, Donchin E. Talking off the top of your head:
toward a mental prosthesis utilizing event-related brain po-
tentials. Electroencephalography and Clinical Neurophysi-
ology, 1988, 70(6): 510−523
[17] Bayliss J D, Ballard D H. Single Trial P300 Recognition in
a Virtual Environment. New York: University of Rochester,
1998
[18] Miller A R, Rosenfeld J P, Soskins M, Jhee M. P300
amplitude and topography in an autobiographical odd-
ball paradigm involving simulated amnesia. Journal of Psy-
chophysiology, 2002, 16(1): 1−11
[19] Renterghem V D, Wyns B, Devlaminck, D. Adding machine
vision to enhance P300 BCI speller. Advanced Materials
Research, 2011, 271: 102−107
[20] Piccione F, Giorgi F, Tonin P, Priftis K, Giove S, Silvoni
S, Palmas G, Beverina F. P300-based brain computer in-
terface: reliability and performance in healthy and paral-
ysed participants. Clinical Neurophysiology, 2006, 117(3):
531−537
[21] Citi L, Poli R, Cinel C, Sepulveda F. P300-based BCI mouse
with genetically-optimized analogue control. IEEE Trans-
actions on Neural Systems and Rehabilitation Engineering,
2008, 16(1): 51−61
 Xing-Yu WANG et al./ Brain Control: Human-computer Integration Control Based on · · ·
[22] Rebsamen B, Burdet E, Guan C, Zhang H H, Teo C L, Zeng
Q, Ang M, Laugier C. A brain-controlled wheelchair based
on P300 and path guidance. In: Proceedings of the 1st
IEEE/RAS-EMBS International Conference on Biomedical
Robotics and Biomechatronics. Pisa, Tuscany, Italy: IEEE,
2006. 1101−1106
[23] Bell C J, Shenoy P, Chalodhorn R, Rao R P N. Control of a
humanoid robot by a noninvasive brain-computer interface
in humans. Journal of Neural Engineering, 2008, 5(2): 214−
220
[24] Hoffmann U, Vesin J M, Ebrahimi T, Diserens K. An effi-
cient P300-based brain-computer interface for disabled sub-
jects. Journal of Neuroscience Methods, 2008, 167(1): 115−
125
[25] Pfurtscheller G, Aranibar A. Evaluation of event-related
desynchronization (ERD) preceding and following volun-
tary self-paced movement. Electroencephalography and
Clinical Neurophysiology, 1979, 46(2): 138−146
[26] Pfurtscheller G, Flotzinger D, Kalcher J. Brain-computer
interface — a new communication device for handicapped
persons. Journal of Microcomputer Applications, 1993,
16(3): 293−299
[27] Pfurtscheller G, Lopes da Silva F H. Event-related EEG/
MEG synchronization and desynchronization: basic princi-
ples. Clinical Neurophysiology, 1999, 110(11): 1842−1857
[28] Scherer R, Müller G R, Neuper C, Graimann B, Pfurt-
scheller G. An asynchronously controlled EEG-based vir-
tual keyboard: improvement of the spelling rate. IEEE
Transactions on Biomedical Engineering, 2004, 51(6):
979−984
[29] Leeb R, Pfurtscheller G. Walking through a virtual city by
thought. In: Proceedings of the 26th Annual International
Conference on IEEE Engineering in Medicine and Biology
Society. San Francisco: IEEE, 2004. 4503−4506
[30] Lalor E C, Kelly S P, Finucane C, Burke R, Smith R, Reilly
R B, McDarby G. Steady-state VEP-based brain-computer
interface control in an immersive 3D gaming environment.
EURASIP Journal on Applied Signal Processing, 2005, 19:
3156−3164
[31] Martinez P, Bakardjian H, Cichocki A. Fully online multi-
command brain-computer interface with visual neurofeed-
back using SSVEP paradigm. Computational Intelligence
and Neuroscience, 2007 2007: 1−9
[32] Trejo L J, Rosipal R, Matthews B. Brain-computer inter-
faces for 1-D and 2-D coursor control: designs using voli-
tional control of the EEG sperctrum or steady-state visual
evoked potentials. IEEE Transactions on Neural Systems
and Rehabilitation Engineering, 2006, 14(2): 255−229
[33] Firman O, Lüth T, Volosyak I, Graser A. Spelling with
steady-state visual evoked potentials. In: Proceedings of the
3rd International IEEE/EMBS Conference on Neural Engi-
neering. Kohala Coast, Hawaii, USA: IEEE, 2007. 354−357
217
[34] Müller-Putz G R, Pfurtscheller G. Control of an electrical
prosthesis with an SSVEP-based BCI. IEEE Transactions
on Biomedical Engineering, 2008, 55(1): 361−364
[35] Birch G E, Mason S G. Brain-computer interface research
at the neil squire foundation. IEEE Transactions on Reha-
bilitation Engineering, 2000, 8(2): 193−195
[36] Levine S P, Huggins J E, BeMent S L, Kushwaha R K,
Schuh L A, Rohde M M, Passaro E A, Ross D A, Elise-
vich K V, Smith B J. A direct brain interface based on
event-related potentials. IEEE Transactions on Rehabilita-
tion Engineering, 2000, 8(2): 180−185
[37] Daly J J, Cheng R, Roqers J, Litinas K, Hrovat K, Dohring
M. Feasibility of a new application of noninvasive brain
computer interface (BCI): a case study of training for re-
covery of volitional motor control after stroke. Journal of
Neurologic Physical Therapy, 2009, 33(4): 203−211
[38] Broetz D, Braun C, Weber C, Soekadar S R, Caria A, Bir-
baumer N. Combination of brain-computer interface train-
ing and goal-directed physical therapy in chronic stroke: a
case report. Neurorehabilitation and Neural Repair, 2010,
24(7): 674−679
[39] Silvoni S, Ramos-Murquialday A, Cavinato M, Volpato C,
Cisotto G, Turolla A, Piccione F, Birbaumer N. Brain-
computer interface in stroke: a review of progress. Clinical
EEG and Neuroscience, 2011, 42(4): 245−252
[40] Bin G Y, Gao X R, Yan Z, Hong B, Gao S K. An online
multi-channel SSVEP-based brain-computer interface using
a canonical correlation analysis method. Journal of Neural
Engineering, 2009, 6(4): 046002
[41] Lin Z L, Zhang C S, Wu W, Gao X R. Frequency recognition
based on canonical correlation analysis for SSVEP-based
BCIs. IEEE Transactions on Biomedical Engineering, 2006,
53(12): 2610−2614
[42] Zhang Y S, Xu P, Liu T J, Hu J, Zhang R, Yao D Z. Mul-
tiple frequencies sequential coding for SSVEP-based brain-
computer interface. PLoS One, 2012, 7(3): e29519
[43] Jin J, Zhang Y, Wang X Y. A novel combination of time
phase and EEG frequency components for SSVEP-based
BCI. In: Proceedings of the 18th international confer-
ence on Neural Information Processing. Berlin, Heidelberg:
Springer, 2011. 273−278
[44] Belitski A, Farquhar J, Desain P. P300 audio-visual speller.
Journal of Neural Engineering, 2011, 8(2): 025022
[45] Muller-Putz, Scherer R, Neuper C, Pfurtscheller G. Steady-
state somatosensory evoked potentials: suitable brain sig-
nals for brain-computer interfaces? IEEE Transactions
on Neural Systems and Rehabilitation Engineering, 2006,
14(1): 30−37
[46] Martens S M M, Hill N J, Farquhar J, Schölkopf B. Overlap
and refractory effects in a brain-computer interface speller
based on the visual P300 event-related potential. Journal
of Neural Engineering, 2009, 6(2): 026003
218 Acta Automatica Sinica, 2013, Vol. 39, No. 3
[47] Gonsalvez C L, Polich J. P300 amplitude is determined
by target-to-target interval. Psychophysiology, 2002, 39(3):
388−396
[48] Hill N J, Farquhar J, Martens S, Biebetamann F, Schölkopf
B. Effects of stimulus type and of error-correcting code de-
sign on BCI speller performance. In: Proceedings of the
2008 Conference of Advances in Neural Information Pro-
cessing Systems 21. Cambridge, MA, USA: MIT Press,
2009. 665− 672
[49] Woldorff M G. Distortion of ERP averages due to overlap
from temporally adjacent ERPs: analysis and correction.
Psychophysiology, 1993, 30(1): 98−119
[50] Kranczioch C, Debener S, Maye A, Engel A K. Temporal
dynamics of access to consciousness in the attentional blink.
NeuroImage, 2007, 37(3): 947−955
[51] Fazel-Rezai R. Human error in P300 speller paradigm
for brain-computer interface. In: Proceedings of the 29th
Annual International Conference on IEEE Engineering in
Medicine and Biology Society. Lyon, France: IEEE, 2007.
2516−2519
[52] Townsend G, LaPallo B K, Boulay C B, Krusienski D J,
Frye G E, Hauser C K, Schwartz N E, Vaughan T M, Wol-
paw J R, Sellers E W. A novel P300-based brain-computer
interface stimulus presentation paradigm: moving beyond
rows and columns. Clinical Neurophysiology, 2010, 121(7):
1109−1120
[53] Sellers E W, Krusienski D J, McFarland D J, Vaughan
T M, Wolpaw J R. A P300 event-related potential brain-
computer interface (BCI): the effects of matrix size and in-
ter stimulus interval on performance. Biological Psychology,
2006, 3(3): 242−252
[54] Shishkin S L, Ganin I P, Kaplan A Y. Event-related po-
tentials in a moving matrix modification of the P300 brain-
computer interface paradigm. Neuroscience Letters, 2011,
496(2): 95−99
[55] Jin J, Allison B Z, Sellers E W, Brunner C, Horki P, Wang
X Y, Neuper C. An adaptive P300-based control system.
Journal of Neural Engineering, 2011, 8(3): 036006
[56] Liu Y, Zhou Z T, Hu D W. Gaze independent brain-
computer speller with covert visual search tasks. Clinical
Neurophysiology, 2010, 112(6): 1127−1136
[57] Takano K, Komatsu T, Hata N, Nakajima Y, Kansaku K.
Visual stimuli for the P300 brain-computer interface: a
comparison of white/gray and green/blue flicker matrices.
Clinical Neurophysiology, 2009, 120(8): 1562−1566
[58] Salvaris M, Sepulveda F. Visual modifications on the P300
speller BCI paradigm. Journal of Neural Engineering, 2009,
6(4): 046011
[59] Gerson A D, Parra L C, Sajda P. Cortically Coupled com-
puter vision for rapid image search. IEEE Transactions
on Neural Systems and Rehabilitation Engineering, 2006,
14(2): 174−179
[60] Bigdely-Shamlo N, Vankov A, Ramirez R R, Makeig S.
Brain activity-based image classification from rapid serial
visual presentation. IEEE Transactions on Neural Systems
and Rehabilitation Engineering, 2008, 16(5): 432−441
[61] Treder M S, Schmidt N M, Blankertz B. Gaze-independent
brain-computer interfaces based on covert attention and
feature attention. Journal of Neural Engineering, 2011,
8(6): 066003
[62] McFarland D J, Sarnacki W A, Townsend G, Vaughan
T, Wolpaw J R. The P300-based brain-computer interface
(BCI): effects of stimulus rate. Clinical Neurophysiology,
2010, 122(4): 731−737
[63] Wang Xing-Yu. Human-computer integration control based
on EEG signal. In: Proceedings of the 23rd Chinese Control
Conference. Shanghai: Press of East China University of
Science and Technology, 2004. 13−14
[64] Jin J, Horki P, Brunner C, Wang X Y, Neuper C,
Pfurtscheller G. A new P300 stimulus presentation pat-
tern for EEG-based spelling systems. Biomedizinische Tech-
nik/Biomedical Engineering, 2010, 55(4): 203−210
[65] Jin J, Allison B Z, Sellers E W, Bruner C, Horki P, Wang
X Y, Neuper C. Optimized stimulus presentation patterns
for an event-related potential EEG-based brain computer
interface. Medical and Biological Engineering and Comput-
ing, 2011, 49(2): 181−191
[66] Jin J, Sellers E W, Wang X Y. Targeting an efficient target-
to-target interval for P300 speller brain-computer inter-
faces. Medical and Biological Engineering and Computing,
2012, 50(3): 289−296
[67] Jin J, Allison B Z, Brunner C, Wang B, Wang X,
Zhang J, Neuper C, Pfurtscheller G. P300 Chinese in-
put system based on Bayesian LDA. Biomedizinische Tech-
nik/Biomedical Engineering, 2010, 55(1): 5−18
[68] Kaufmann T, Schulz S M, Grünzinger C, Kübler A. Flash-
ing characters with famous faces improves ERP-based
brain-computer interface performance. Journal of Neural
Engineering, 2011, 8(5): 056016
[69] Allison B Z, Brunner C, Kaiser V, Müller-Putz G, Neu-
per C, Pfurtscheller G. Toward a hybrid brain-computer
interface based on imagined movement and visual atten-
tion. Journal of Neural Engineering, 2010, 7(2): 026007
[70] Brunner C, Allison B Z, Altstätter C, Neuper C. A com-
parison of three brain-computer interfaces based on event-
related desynchronization, steady state visual evoked po-
tentials, or a hybrid approach using both signals. Journal
of Neural Engineering, 2011, 8(2): 025010
[71] Horki P, Solis-Escalante T, Neuper C, Müller-Putz G. Com-
bined motor imagery and SSVEP based BCI control of a 2
DoF artificial upper limb. Medical and Biological Engineer-
ing and Computing, 2011, 49(5): 567−577
 Xing-Yu WANG et al./ Brain Control: Human-computer Integration Control Based on · · ·
[72] Allison B Z, Brunner C, Altstätter C, Wagner I C, Griss-
mann S, Neuper C. A hybrid ERD/SSVEP BCI for contin-
uous simultaneous two dimensional cursor control. Journal
of Neuroscience Methods, 2012, 209(2): 299−307
[73] Guo F, Hong B, Gao X R, Gao S K. A brain-computer in-
terface using motion-onset visual evoked potential. Journal
of Neural Engineering, 2008, 5(4): 477−485
[74] Liu T, Goldberg L, Gao S K, Hong B. An online brain-
computer interface using non-flashing visual evoked poten-
tials. Journal of Neural Engineering, 2010, 7(3): 036003
[75] Hong B, Guo F, Liu T, Gao X R, Gao S K. N200-speller us-
ing motion-onset visual response. Clinical Neurophysiology,
2009, 120(9): 1658−1666
[76] Jin J, Allison B Z, Wang X Y, Neuper C. A combined
brain computer interface based on P300 potentials and
motion-onset visual evoked potentials. Journal of Neuro-
science Methods, 2012, 205(2): 265−276
[77] Long J Y, Li Y Q, Wang H T, Yu T Y, Pan J H, Li F. A
hybrid brain computer interface to control the direction and
speed of a simulated or real wheelchair. IEEE Transactions
on Neural Systems and Rehabilitation Engineering, 2012,
20(5): 720−729
[78] Zhang Y, Zhao Q B, Jin J, Wang X Y, Cichocki A. A
novel BCI based on ERP components sensitive to configural
processing of human faces. Journal of Neural Engineering,
2012, 9(2): 026018
[79] Jin J, Allison B Z, Kaufmann T, Kübler A, Zhang Y, Wang
X Y, Cichocki A. The changing face of P300 BCIs: a com-
parison of stimulus changes in a P300 BCI involving faces,
emotion, and movement. PLoS ONE, 7(11): e49688
[80] Vedel-Larsen E, FugløJ, Channir F, Thomsen C E, Sorensen
H B D. A comparative study between a simplified Kalman
filter and sliding window averaging for single trial dynami-
cal estimation of event-related potentials. Computer Meth-
ods and Programs in Biomedicine, 2010, 99(3): 252−260
[81] Bostanov V. BCI competition 2003-Data sets Ib and IIb:
feature extraction from event-related brain potentials with
the continuous wavelet transform and the T-value scalo-
gram. IEEE Transactions on Biomedical Engineering, 2004,
51(6): 1057−1061
[82] Sellers E W, Donchin E. A P300-based brain-computer in-
terface: initial tests by ALS patients. Clinical Neurophysi-
ology, 2006, 117(3): 538−548
[83] Gao X R, Xu D F, Cheng M, Gao S K. A BCI-based envi-
ronmental controller for the motion-disabled. IEEE Trans-
actions on Neural Systems and Rehabilitation Engineering,
2003, 11(2): 137−140
[84] Schlögl A, Lee F, Bischof H, Pfurtscheller G. Characteriza-
tion of four-class motor imagery EEG data for the BCI-
competition 2005. Journal of Neural Engineering, 2005,
2(4): 14−22
219
[85] Wu Z H, Yao D H. Frequency detection with stability co-
efficient for steady-state visual evoked potential (SSVEP)-
based BCIs. Journal of Neural Engineering, 2008, 5(1): 36−
43
[86] Pires G, Nunes U, Castelo-Branco M. Statistical spatial fil-
tering for a P300-based BCI: tests in able-bodied, and pa-
tients with cerebral palsy and amyotrophic lateral sclerosis.
Journal of Neuroscience Methods, 2011, 195(2): 270−281
[87] Blankertz B, Lemm S, Treder M, Haufe S, Muller K R.
Single-trial analysis and classification of ERP components
— A tutorial. NeuroImage, 2011, 56(2): 814−825
[88] Boye A T, Kristiansen U Q, Billinger M, Nascimento O F,
Farina D. Identification of movement-related cortical po-
tentials with optimized spatial filtering and principal com-
ponent analysis. Biomedical Signal Processing and Control,
2008, 3(4): 300−304
[89] Xu N, Gao X R, Hong B, Miao X B, Gao S K, Yang F S.
BCI competition 2003-data set IIb: enhancing P300 wave
detection using ICA-based subspace projections for BCI ap-
plications. IEEE Transactions on Biomedical Engineering,
2004, 51(6): 1067−1072
[90] Wang X H. Optimizing spatial filters for single-trial EEG
classification via a discriminant extension to CSP: the
Fisher criterion. Medical and Biological Engineering and
Computing, 2011, 49(9): 997−1001
[91] Cichocki A, Zdunek R, Phan A H, Amari S. Nonnega-
tive Matrix and Tensor Factorizations: Applications to Ex-
ploratory Multi-Way Data Analysis and Blind Source Sep-
aration. Chichester, UK.: John Wiley and Sons, 2009
[92] Cichocki A, Washizawa Y, Rutkowski T, Bakardjian H,
Phan A H, Choi S, Lee H, Zhao Q B, Zhang L Q, Li Y
Q. Noninvasive BCIs: multiway signal-processing array de-
compositions. IEEE Computer, 2008, 41(10): 34−42
[93] Kolda T G, Bader B W. Tensor decompositions and appli-
cations. Siam Review, 2009, 51(3): 455−500
[94] Washizawa Y, Higashi H, Rutkowski T, Tanaka T, Cichocki
A. Tensor based simultaneous feature extraction and sam-
ple weighting for EEG classification. In: Proceedings of the
17th International Conference on Neural Information Pro-
cessing. Berlin, Heidelberg: Springer, 2010. 25−33
[95] Zhao Q B, Zhang L Q, Cichocki A. Multilinear general-
ization of common spatial pattern. In: Proceedings of the
2009 IEEE International Conference on Acoustics, Speech
and Signal Processing. Washington, DC, USA: IEEE, 2009.
525 −528
[96] Zhang Y, Zhou G X, Zhao Q B, Onishi A, Jin J, Wang X Y,
Cichocki A. Multiway canonical correlation analysis for fre-
quency components recognition in SSVEP-based BCIs. In:
Proceedings of the 18th International Conference on Neural
Information Processing. Berlin, Heidelberg: Springer, 2011.
287−295
220 Acta Automatica Sinica, 2013, Vol. 39, No. 3
[97] Li J, Zhang L Q, Tao D C, Sun H, Zhao Q B. A prior
neurophysiologic knowledge free tensor-based scheme for
single trial EEG classification. IEEE Transactions on Neu-
ral Systems and Rehabilitation Engineering, 2009, 17(2):
107−115
[98] Duda R O, Hart P E, Stork D G. Pattern Classification
(2nd edition). New York: Wiley, 2001
[99] Bishop C M. Pattern Recognition and Machine Learning.
New York: Springer, 2006
[100] Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B.
A review of classification algorithms for EEG-based brain-
computer interfaces. Journal of Neural Engineering, 2007,
4(2): R1-R13.
[101] Pfurtscheller G, Neuper C. Motor imagery and direct
brain-computer communication. Proceedings of the IEEE,
2001, 89(7): 1123−1134
[102] Müller-Putz G R, Scherer R, Brauneis C, Pfurtscheller G.
Steady-state visual evoked potential (SSVEP)-based com-
munication: impact of harmonic frequency components.
Journal of Neural Engineering, 2005, 2(4): 123−130
[103] Zhang Y, Jin J, Qing X Y, Wang B, Wang X Y. LASSO
based stimulus frequency recognition model for SSVEP
BCIs. Biomedical Signal Processing and Control, 2012,
7(2): 104−111
[104] Krusienski D J, Sellers E W, Cabestaing F, Bayoudh S,
McFarland D J, Vaughan T M, Wolpaw J R. A comparison
of classification techniques for the P300 speller. Journal of
Neural Engineering, 2006, 3(4): 299−305
[105] Krusienski D J, Sellers E W, McFarland D J, Vaughan
T M, Wolpaw J R. Toward enhanced P300 speller per-
formance. Journal of Neuroscience Methods, 2008, 167(1):
15−21
[106] Schäfer J, Strimmer K. A shrinkage approach to large-scale
covariance matrix estimation and implications for func-
tional genomics. Statistical Applications in Genetics and
Molecular Biology, 2005, 4: Article32
[107] Vidaurre C, Kramer N, Blankertz B, Schlogl A. Time
domain parameters as a feature for EEG-based brain-
computer interfaces. Neural Networks, 2009, 22(9):
1313−1319
[108] Kaper M, Meinicke P, Grossekathoefer U, Lingner T, Rit-
ter H. BCI competition 2003-Data set IIb: support vector
machines for the P300 speller paradigm. IEEE Transactions
on Biomedical Engineering, 2004, 51(6): 1073−1076
[109] Thulasidas M, Guan C, Wu J K. Robust classification of
EEG signal for brain-computer interface. IEEE Transac-
tions on Neural Systems and Rehabilitation Engineering,
2006, 14(1): 24−29
[110] Jrad N, Congedo M, Phlypo R, Rousseau S, Flamary R,
Yger F, Rakotomamonjy A. sw-SVM: sensor weighting sup-
port vector machines for EEG-based brain-computer inter-
faces. Journal of Neural Engineering, 2011, 8(5): 056004
[111] Xu Q, Zhou H, Wang Y J, Huang J. Fuzzy support vec-
tor machine for classification of EEG signals using wavelet-
based features. Medical Engineering and Physics, 2009,
31(7): 858−865
[112] Lemm S, Schafer C, Curio G. BCI competition 2003-
Data set III: probabilistic modeling of sensorimotor mu
rhythms for classification of imaginary hand movements.
IEEE Transactions on Biomedical Engineering, 2004, 51(6):
1077−1080
[113] Obermaier B, Guger C, Neuper C, Pfurtscheller G. Hid-
den Markov models for online classification of single trial
EEG data. Pattern Recognition Letters, 2001, 22(12):
1299−1309
[114] Haselsteiner E, Pfurtscheller G. Using time-dependent neu-
ral networks for EEG classification. IEEE Transactions on
Rehabilitation Engineering, 2000, 8(4): 457−463
[115] Palaniappan R. Brain computer interface design using
band powers extracted during mental tasks. In: Proceed-
ings of the 2nd International IEEE EMBS Conference on
Neural Engineering. Arlington, VA: IEEE, 2005. 321−324
[116] Balakrishnan D, Puthusserypady S. Multilayer percep-
trons for the classification of brain computer interface data.
In: Proceedings of the IEEE 31st Annual Northeast on Bio-
engineering Conference. Hoboken, New Jersey: IEEE, 2005.
118−119
[117] Rakotomamonjy A, Guigue V. BCI competition III:
dataset II-ensemble of SVMs for BCI P300 speller. IEEE
Transactions on Biomedical Engineering, 2008, 55(3):
1147−1154
[118] Lee H, Choi S. PCA + HMM + SVM for EEG pattern
classification. In: Proceedings of the 7th International Sym-
posium on Signal Processing and Its Applications. Paris,
France: IEEE, 2003. 541−544
[119] Vidaurre C, Schlogl A, Cabeza R, Scherer R, Pfurtscheller
G. Study of on-line adaptive discriminant analysis for EEG-
based brain computer interfaces. IEEE Transactions on
Biomedical Engineering, 2007, 54(3): 550−556
[120] Neurosky: BCl technology grounded in laboratory research
[Online], available: http://company.neurosky.com/univer-
sity/, August 13, 2012
[121] Emotiv forum thread about the epochs sensors [Online],
available: http: //emotiv.com/ forum/ forum10/ topic697,
August 13, 2012
[122] Pfurtscheller G, Solis-Escalante T. Could the beta rebound
in the EEG be suitable to realize a “brain-switch”? Clinical
Neurophysiology, 2009, 120(1): 24−29
 Xing-Yu WANG et al./ Brain Control: Human-computer Integration Control Based on · · ·
[123] Wei J H, Luo Y J. Principle and Technique of Event-
Related Brain Potentials. Beijing: Science Press, 2010
[124] Zhou J Y, Wang F Y, Zeng D J. Hierarchical dirichlet
processes and their applications: a survey. Acta Automatica
Sinica, 2011, 37(4): 389−407
[125] Wang H Q, Sun F C, Cai Y N, Chen N, Ding L G. On
multiple kernel learning methods. Acta Automatica Sinica,
2010, 36(8): 1037−1050
[126] Jin J, Sellers E W, Zhang Y, Daly I, Wang X Y, Cichocki A.
Whether generic model works for rapid ERP-based BCI cal-
ibration. Journal of Neuroscience Methods, 2013, 212(1):
94−99
[127] Fu Y F, Wang Y C, Li H Y, Xu B L, Li Y C. Direct
brain-controlled robot interface technology. Acta Automat-
ica Sinica, 2012, 38(8): 1229−1246
[128] Wiener N [Author], Hao J R [Translator]. Cybernetics; or,
Control and Communication in the Animal and the Ma-
chine (2nd edition). Beijing: Science Press, 1963
Xing-Yu WANG Professor at East
China University of Science and Tech-
nology. He received his bachelor degree
in mathematics from Fudan University in
1967, and his Ph. D. degree in industrial
automation from East China Chemical In-
stitute in 1984. His research interest cov-
ers brain control and its application, intelli-
gent control, pattern recognition, and con-
trol theory. Corresponding author of this paper. E-mail: xy-
wang@ecust.edu.cm
221
Jing JIN Assistant professor at East
China University of Science and Technol-
ogy. He received his bachelor degree from
School of Information and Electrical Engi-
neering of Hunan University of Science and
Technology in 2004, and his Ph. D. degree
in control theory and control engineering
from East China University of Science and
Technology in 2010. His research interest
covers brain-computer interface, signal processing, and pattern
recognition.
E-mail: jinjing@ecust.edu.cn
Yu ZHANG Ph. D. candidate at East
China University of Science and Tech-
nology. He received his bachelor degree
in electrical engineering and automation
from East China University of Science
and Technology in 2008, His research in-
terest covers machine learning and pat-
tern recognition, multiway signal process-
ing, and brain-computer interface. E-mail:
zhangyu0112@gmail.com
Bei WANG Associate professor at East
China University of Science and Technol-
ogy. She received her bachelor degree in
electrical and electronics from East China
Normal University in 1998, and her Ph. D.
degree in system control from Saga Uni-
versity, Japan in 2009. Her research in-
terest covers intelligent signal processing
and pattern recognition, complex system
and its application in artificial life science. E-mail: bei-
wang@ecust.edu.cn