IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 19, NO.

5, OCTOBER 2011 465

Guest Editorial
Brain Training: Cortical Plasticity and Afferent
Feedback in Brain-Machine Interface Systems
B
RAINMACHINE interfaces (BMI) hold great promise
for studying sensorimotor processes in the brain, as
well as for restoring independence to persons with profound
physical disability. Over the past decade, signicant advances
have been made in neural interface technology and signal pro-
cessing methods [1]. The resulting increase in yield and quality
of neural recordings has allowed more reliable and accurate
translation of neural signals into motor commands. Parallel
efforts by mechatronics researchers have yielded virtually
anthropomorphic prosthetic arms and hands that are nally
beginning to rival the capabilities of the human arm [2], [3].
While progress in these areas continues, two new areas have
emerged recently that may be essential elements for advancing
the quality and complexity of movement that can be controlled
by a BMI. First, is the need for tactile and proprioceptive
feedback to augment vision. Second, is the need to leverage the
brains natural adaptive mechanisms to improve BMI training
and operation. Tactile and proprioceptive feedback may be
particularly important for tasks involving forceful interactions,
such as object grasping and manipulation when vision is inade-
quate or distracted by a parallel task (as in drinking coffee while
answering e-mails; see cover illustration). However, feedback
and learning are not independent processes. Somatosensory
feedback is known to play a critical role in motor learning [4].
Consequently, it is reasonable to anticipate that afferent feed-
back may be particularly important during the early phases of
BMI use. Understanding the relation between afferent feedback
and learning and applying these principles to BMI control may
be crucial for realizing its full potential.
This special section is devoted to research into the develop-
ment of somatosensory afferent interfaces for conveying tactile
and proprioceptive feedback to users and the role of afferent
feedback in facilitating motor learning in the BMI framework.
This collection of papers demonstrates several different strate-
gies for providing somatosensory feedback through patterned
electrical stimulation of central and peripheral targets in the ner-
vous system. While it may be unrealistic to expect any of these
approaches to yield truly natural and complete restoration of so-
matosensory function, even crude somatosensory feedback may
be useful given the brains remarkable capacity for learning and
plasticity. By understanding how best to train the brain, we
can improve prosthetic control. Beyond such immediate effects,
it may even further yield therapeutic benets by strengthening
spared neural connections in the injured sensorimotor system
[5][7].
Digital Object Identier 10.1109/TNSRE.2011.2168989
I. NEURAL SUBSTRATES FOR CREATING A SOMATOSENSORY
AFFERENT INTERFACE
Two of the studies in this special issue examine the de-
tectability of intracortical microstimulation (ICMS) inputs to
sensory regions of the cerebral cortex, investigating the effect
of varied current and waveform in awake animal subjects.
Lee Millers group at Northwestern University [8] studies the
somatosensory system of monkeys, while Kevin Ottos group
at Purdue [9] uses rat auditory cortex. Evaluating the efcacy
of sensory stimulation is challenging, particularly in animal
models because of the indirect methods that must be employed
to determine the perceptual effects of stimulation. Nevertheless,
studies like these will be critical to understand the nature of
percepts evoked by activation of an inevitably heterogeneous
population of neurons by single electrodes, and how multiple
electrodes interact. A third study from the lab of Zelma Kiss
at the University of Calgary [10] tested the ability of human
subjects to detect and interpret thalamic stimulation delivered
through a DBS electrode. Working with humans makes it
possible to inquire directly into the nature of the evoked per-
cepts. In this case, while the quality of percepts was stable and
persisting, they were mostly unnatural, a mixture of tingling,
vibration and movement across the skin. The authors reach
the sobering conclusion that evoking natural percepts with
conventional DBS electrodes will be challenging. However, it
is possible that microstimulation with arrays of microelectrodes
may prove more effective [11].
It is well established that somatosensation is not a simple pas-
sive operation, but is instead an active, exploratory process in-
volving a complex interplay with the motor system. Two studies
pursued the use of electrical stimulation in the active sensing of
real or virtual objects. Work by Jose Carmenas group at the
University of California, Berkeley [12]
1
showed that rats could
learn to locate virtual objects while whisking, by replacing ac-
tual whisker contact with appropriately-timed ICMS delivered
to the barrel cortex. These studies and those mentioned above
show that animals are able to detect sensory cues conveyed by
ICMS, however a more detailed description of the perceptual
qualities of these sensory inputs cannot be obtained from ani-
mals.
Studies with human subjects provide the best opportunity for
evaluating the perceptual qualities of afferent interfaces. Ken
Horch and colleagues at the University of Utah have been testing
strategies for restoring sensory feedback to amputees through
1
It should be noted that as an unintended result of its early submission and
acceptance, this paper, which was to have been included in this special issue,
was inavertently published in the July issue.
1534-4320/$26.00 2011 IEEE
466 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 19, NO. 5, OCTOBER 2011
electrical stimulation of peripheral nerves [13], [14]. In [15],
two human participants with transradial amputations used a my-
oelectric hand prosthesis to grasp objects of varied size and
hardness. Force feedback was conveyed by electrical stimula-
tion of cutaneous afferents in the nerve stump and enabled one of
the subjects to correctly identify objects as being soft, medium,
or hard. This subject also reported sensations of nger motion
from electrical stimulation of putative proprioceptive afferents
and was able to discriminate object sizes as large or small, but
was unable to reliably discriminate intermediate sizes.
II. NEURAL CODING OF SOMATOSENSORY INFORMATION
Each of these studies required subjects to learn completely
arbitrary or potentially unnatural stimulus patterns. The neces-
sity for a subject to learn a large number of stimulus patterns
is an important potential limitation of these approaches. This
approach is unlikely to scale adequately without adding an un-
acceptable cognitive burden on subjects. To address this con-
cern, several studies have been devoted to determining the nat-
ural mapping between limb state and neural state, and to de-
vising stimulus methods that might best imitate it. Two studies
in awake monkeys addressed the encoding of tactile and pro-
prioceptive information and the close interplay between these
modalities. Work at Arizona State University in the lab of Steven
HelmsTillery showed that many cutaneous neurons in areas
3b and 1 encode elements of both touch and motion [16]. The
tactile signals arising during haptic exploration appear to un-
dergo dynamic modication by an underlying proprioceptive
map, yielding interesting postural dependencies in the response
properties of putative tactile afferents.
Miller et al. showed that most neurons in area 2, whether with
cutaneous or deep muscle receptive elds, encode the direction
and speed of hand movement, enabling accurate reconstruction
of end point kinematics during movement. They also found that
neurons with similar receptive elds tend to cluster together,
which may give rise to a more coherent motion percept upon
stimulation than would otherwise be the case [8]. Whether the
intermingled proprioceptive and tactile representation may ulti-
mately prove advantageous or detrimental is an important ques-
tion that remains to be answered.
Weber et al. (University of Pittsburgh, [8])recorded from pri-
mary afferent neurons in the dorsal root ganglia (DRG) and
S1 cortex of anesthetized cats during passive movement of the
hindlimb. Most DRG neurons were found to encode both posi-
tion and velocity information for the limb, consistent with pre-
vious observations that both cutaneous and muscle afferent neu-
rons convey proprioceptive signals for the limb [17]. These nd-
ings may explain the intermingling of tactile and proprioceptive
information observed in Millers studies of area 2. Webers lab
also examined the response of S1 neurons to limb movement
and DRG microstimulation using stimulus patterns that were
set to mimic the spatiotemporal pattern of activity recorded in
the DRG during movement. The S1 responses to microstimula-
tion were fairly similar to the S1 responses evoked during move-
ment, suggesting that naturally patterned stimulation applied on
a relatively small number of channels may be effective in deliv-
ering limb-state information to the brain.
Since it is practically impossible to create a truly biomimetic
afferent interface, it will be important to determine how de-
viations from naturalistic patterns degrade performance. A
modeling study of the responses of cutaneous mechanorecep-
tive afferents innervating the glabrous skin of the hand offers
some encouragement (Sliman Bensmaia, University of Chicago,
[18]). Despite the heterogeneity of the responses of different af-
ferents, a single canonical model could be used to accurately
predict the population response to mechanical stimulation. This
suggests that even a reduced set of inputs may be capable of
conveying rich sensory information. To this end, the work of
Karim Oweiss lab at Michigan State University [19] is devoted
to the development of novel stimulation protocols designed to
combat the lack of selectivity and the heterogeneity of neural
responses evoked by electrical stimulation. Stimulation parame-
ters are continuously optimized through a feedback control loop
driven by motor system recordings that optimize the activation
of large networks of neurons using a relatively small number of
electrodes and at or near threshold currents.
III. BRAIN-TRAINING WITH A BMI
Most of the invasive, spike-based BMI approaches begin by
attempting to compute an optimal transformbetween neural dis-
charge and desired motor output. In contrast, the work of Eb
Fetz and Andy Jackson is predicated on the assumption that
recording and decoding biomimetic motor commands or en-
coding naturalistic feedback through electrical stimulation is
essentially impossible [20]. They review a range of their own
and others work, and argue that performance using brain or
myoelectric interfaces depends almost entirely on the subjects
ability to learn the mapping fromthe efference copy of the motor
command to its consequence in the task space, a process that
may serve as a proxy for missing somatosensation. A key to the
difculty of this task may be in the relative dimensionality of the
control signals and the task space. Learning to control a pros-
thesis can be quite rapid if control is mapped from the activity
of muscles to functionally related motions. However, even non-
intuitive maps can be learned with enough practice.
The more limited dynamic range and spatial resolution of
noninvasive BMI systems has traditionally caused more em-
phasis to be placed on learning to modulate cortical rhythmic ac-
tivity in ways that may not closely mimic those of natural move-
ments. Leo Cohens group at the National Institute of Neurolog-
ical Disorders and Stroke has been studying the ability of both
healthy subjects and stroke patients to learn to control a hand or-
thosis using event related desynchronization (ERD) of the sen-
sorimotor rhythm (SMR; 1114 Hz band of EEG or MEG) [21].
In this issue, Soekadar et al. [22] show that by coupling in-
creasing levels of ERD with different speeds of hand closure,
both subject groups adapted dramatically faster than when they
had only binary feedback. The question of whether brain ac-
tivity closely coupled with somatosensory feedback may give
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 19, NO. 5, OCTOBER 2011 467
rise to longer term therapeutic effects in addition to improved
BMI performance is of great interest.
LEE E. MILLER, Guest Editor
Departments of Physiology and Physical Medicine and
Rehabilitation, Feinberg School of Medicine, and the
Department of Biomedical Engineering
Northwestern University
Chicago, IL 60611 USA
DOUGLAS J. WEBER, Guest Editor
Department of Physical Medicine and Rehabilitation
and the Department of Bioengineering
University of Pittsburgh
Pittsburgh, PA 15213 USA
REFERENCES
[1] A. B. Schwartz, X. T. Cui, D. Weber, and D. W. Moran, Brain-con-
trolled interfaces: Movement restoration with neural prosthetics,
Neuron, vol. 52, pp. 205220, Oct. 5, 2006.
[2] C. Cipriani, M. Controzzi, and M. C. Carrozza, The SmartHand tran-
sradial prosthesis, J. Neuroeng. Rehabil., vol. 8, p. 29, 2011.
[3] L. Resnik, Research update: VA study to optimize DEKA arm, J.
Rehabil. Res. Develop., vol. 47, p. ixx, 2010.
[4] T. Sakamoto, K. Arissian, and H. Asanuma, Functional role of the
sensory cortex learning motor skills cats, Brain Res., vol. 503, pp.
258264, 1989.
[5] N. Birbaumer and L. G. Cohen, Brain-computer interfaces: Commu-
nication and restoration of movement paralysis, J. Physiol., vol. 579,
pp. 62136, Mar. 15, 2007.
[6] W. Wang, J. L. Collinger, M. A. Perez, E. C. Tyler-Kabara, L. G.
Cohen, N. Birbaumer, S. W. Brose, A. B. Schwartz, M. L. Boninger,
and D. J. Weber, Neural interface technology for rehabilitation:
Exploiting and promoting neuroplasticity, Phys. Med. Rehabil. Clin.
N Am., vol. 21, pp. 157178, Feb. 2010.
[7] T. A. Thrasher, V. Zivanovic, W. McIlroy, and M. R. Popovic, Reha-
bilitation of reaching and grasping function severe hemiplegic patients
using functional electrical stimulation therapy, Neurorehabil. Neural
Repair, vol. 22, pp. 706714, Nov.Dec. 2008.
[8] D. J. Weber, B. M. London, J. A. Hokanson, C. A. Ayers, R. R. Torres,
B. Zaiimi, and L. E. Miller, Limb-state information encoded by pe-
ripheral and central somatosensory neurons: Implications for an af-
ferent interface., IEEE Trans. Neural. Syst. Rehabil. Eng., vol. 19, no.
5, pp. 501513, Oct. 2011.
[9] A. Koivuniemi and K. Otto, Asymmetric versus symmetric electric
pulses for intracortical microstimulation, IEEE Trans. Neural. Syst.
Rehabil. Eng., vol. 19, no. 5, pp. 468476, Oct. 2011, 2011.
[10] E. Heming, R. Choo, J. Davies, and Z. Kiss, Designing a thalamic so-
matosensory neural prosthesis: Consistency and persistence of percepts
evoked by electrical stimulation, IEEE Trans. Neural. Syst. Rehabil.
Eng., vol. 19, no. 5, pp. 477482, Oct. 2011, 2011.
[11] S. Ohara, N. Weiss, and F. A. Lenz, Microstimulation the region of the
human thalamic principal somatic sensory nucleus evokes sensations
like those of mechanical stimulation and movement, J. Neurophysiol.,
vol. 91, pp. 736745, Feb. 2004.
[12] S. Venkatraman and J. M. Carmena, Active sensing of target location
encoded by cortical microstimulation, IEEE Trans. Neural Syst. Re-
habil. Eng., vol. 19, no. 3, pp. 317324, Jun. 2011.
[13] G. S. Dhillon and K. W. Horch, Direct neural sensory feedback and
control of a prosthetic arm, IEEE Trans. Neural. Syst. Rehabil. Eng.,
vol. 13, no. 4, pp. 468472, Dec. 2005.
[14] G. S. Dhillon, T. B. Kruger, J. S. Sandhu, and K. W. Horch, Effects
of short-term training on sensory and motor function severed nerves of
long-term human amputees, J. Neurophysiol., vol. 93, pp. 26252633,
May 2005.
[15] K. Horch, S. Meek, T. Taylor, and D. Hutchinson, Object discrimina-
tion with an articial hand using electrical stimulation of peripheral tac-
tile and proprioceptive pathways with intrafascicular electrodes, IEEE
Trans. Neural. Syst. Rehabil. Eng., vol. 19, no. 5, pp. 483489, Oct.
2011.
[16] L. Rincon-Gonzalez, J. Warren, D. Meller, and S. Helms-Tillery,
Haptic interaction of touch and proprioception: Implications for
neuroprosthetics, IEEE Trans. Neural. Syst. Rehabil. Eng., vol. 19,
no. 5, pp. 490500, Oct. 2011.
[17] R. B. Stein, D. J. Weber, Y. Aoyagi, A. Prochazka, J. B. Wagenaar, S.
Shoham, and R. A. Normann, Coding of position by simultaneously
recorded sensory neurones the cat dorsal root ganglion, J. Physiol.,
vol. 560, pp. 883896, Nov. 1, 2004.
[18] S.-S. Kim, S. Mihalas, A. Russel, Y. Dong, and S. J. Bensmaia, Does
afferent heterogeneity matter conveying tactile feedback through pe-
ripheral nerve stimulation?, IEEE Trans. Neural. Syst. Rehabil. Eng.,
vol. 19, no. 5, pp. 514520, Oct. 2011.
[19] J. Liu, H. K. Khalil, and K. G. Oweiss, Neural feedback for instan-
taneous spatiotemporal modulation of afferent pathways bi-directional
bra machine interfaces, IEEE Trans. Neural. Syst. Rehabil. Eng., vol.
19, no. 5, pp. 521533, Oct. 2011.
[20] A. Jackson and E. Fetz, Interfacing with the computational brain,
IEEE Trans. Neural. Syst. Rehabil. Eng., vol. 19, no. 5, pp. 534541,
Oct. 2011.
[21] G. Pfurtscheller and A. Aranibar, Evaluation of event-related desyn-
chronization (ERD) preceding and following voluntary self-paced
movement, Electroencephalogr. Clin. Neurophysiol., vol. 46, pp.
138146, Feb. 1979.
[22] S. R. Soekadar, M. Witkowski, J. Mellinger, A. Ramos, N. Birbaumer,
and L. G. Cohen, ERD-based online brain-machine-interface (BMI)
the context of neurorehabilitation: Optimizing BMI learning and per-
formance, IEEE Trans. Neural. Syst. Rehabil. Eng., vol. 19, no. 5, pp.
542549, Oct. 2011.
Lee E. Miller (M08) received the B.A. degree in
physics from Goshen College, Goshen, IN, in 1980,
and the M.S. degree in biomedical engineering and
the Ph.D. degree in physiology from Northwestern
University, Evanston, IL, in 1983 and 1989, re-
spectively. He completed two years of postdoctoral
training in the Department of Medical Physics, Uni-
versity of Nijmegen, Nijmegen, The Netherlands.
He is currently the Edgar C. Stuntz Distinguished
Professor of Neuroscience in the Departments of
Physiology, Physical Medicine and Rehabilitation,
and Biomedical Engineering at Northwestern University, Chicago, IL. His
primary research interests are in the cortical control of muscle activity and limb
movement, the representation of limb state by the somatosensory system, and
in the development of brainmachine interfaces that attempt to mimic normal
physiological systems.
Douglas J. Weber (M94) received the B.S. degree
in biomedical engineering from the Milwaukee
School of Engineering, Milwaukee, WI, in 1994, and
the M.S. and Ph.D. degrees in bioengineering from
Arizona State University, Tempe, in 2000 and 2001,
respectively.
He is currently an Assistant Professor in the De-
partment of Physical Medicine and Rehabilitation,
University of Pittsburgh, Pittsburgh, PA. He is also a
faculty member in the Department of Bioengineering
and the Center for the Neural Basis of Cognition.
Previously, he was a Postdoctoral Fellow and then an Assistant Professor in
the Centre for Neuroscience at the University of Alberta. His primary research
area is neural engineering, including studies of motor learning and control of
walking and reaching with particular emphasis on applications to rehabilita-
tion technologies and practice. Specic research interests include functional
electrical stimulation, activity-based neuromotor rehabilitation, neural coding,
and neural control of prosthetic devices. Active projects in his lab include
development of somatosensory neural interfaces (SSNI) to record from or
stimulate primary afferent neurons in cats and humans, and brainmachine
interface studies with magnetoencephalography and electrocorticography in
humans.
Dr. Weber has been a member of the Society for Neuroscience since 1995.