MEDICAL ROBOTICS

Finger Movement Classification based on Neural Networks

Alvarez Delgado Julio
November, 2016

INTRODUCTION
During the past decades, various research teams along the world, brings important advances
in the biomedical field, especially in Electromyographic (EMG) signals acquisition and analysis. They
use those signals has control inputs for an extensive list of applications such diagnoses of
neuromuscular diseases, rehabilitation robotics, games, prosthetic devices, etc.
EMG signals can be described (roughly) as the electrical impulses coming from the brain
through the neural system which control the skeletal muscles, and they are the milestone of any
advanced robotic prosthetics. Several methods to use those signals as control inputs for robotic
prosthesis devices (Mu, 2007) (B, Parker, & RN, 1993) has been developed, such hybrid systems with
EMG signals and accelerometers, EMG signal train triggers and pure based EMG control.
However, developing myoelectric control based interfaces using the pattern recognition of
EMG signals is not a trivial task. There are large variations in EMG signals depending on the factors
such age, muscles activity, skin-fat layer, and movement performed. If we compare EMG signals with
other biosignals (i.e ECG) we can realize that it has a considerable amount of noise related by the
environment, the equipment, electromagnetic radiation, motion, etc. This behavior makes difficult to
determine the proper features of the signal for the classification problem.
Considering that a functional robotic prosthetic should be able to perform a proper
classification in the shortest time possible, and taking into account that features such wavelets or
other frequency-domain based features are time consuming at considerable computational cost, we
will only focus on time domain features. Along the following report we will discuss how time-domain
features can be used by Neural Network techniques in order to easily identify movements performed
by the user.

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

ELECTROMYGRAPHIC SIGNALS (EMG)

Neurons are specialized cells (fig. 1.1) which transmit and generate electrical impulses and
they conform the foundation of the human nervous system. When there is a stimulus in a neuron or
in a sensorial cell, an electrical wave (called nervous impulse) is generated.

Figure 1.1 Neuron (peripheral nervous system)

A motor unit (MU) is the smallest functional subdivision of a muscle. It consists of the
motoneuron, its axon and all the muscle fibers that are innervated by its branches (De Luca, 2016).
When motor units are activated, the corresponding muscle fibers contract.

Figure 1.2 Motor Unit and muscular fibers

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

The Motor Unit Action Potential (MUAP) is defined as the electrical signal how trigger the
activation of muscular fibers on a MU. The presence of a MUAP produce a contraction into the
muscular fibers, however, in order to maintain this contraction along the time, is necessary to
continuously firing the MU. This continuous firing generate a Motor Unit Action Potential Train
(MUAPT) which remain active as the muscle requires generate Force.

Figure 1.3 MUAP and its corresponding MUAPT

Finally the sum of all the active MUAPT at the same time period create what is called an EMG
signal (fig 1.4). As the force requirement (in the muscle) increases, the amount of activated MU
increases and obviously the firing rate of the activation of the muscular fibers.

Figure 1.4 Sum of MUAPT to conform an EMG signal

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

OBTAIN INFORMATION FROM EMG SIGNALS

In the previous section we discuss how the EMG signal is generated, and is easy to derive that
obtain information (patterns) from this kind of signals is not a trivial task, in fact this type of signals
has more in common with noise than other biological signals such ECG (fig 2.1).

Figure 2.1 EMG spectral

In order to use this signals in advanced robotic prosthetics, various authors propose several
approaches to use EMG signals as control inputs for those devices (Mu, 2007) (B, Parker, & RN, 1993),
such hybrid systems with EMG signals and accelerometers, EMG signal train triggers and pure based
EMG control. However, In order to develop a user friendly prosthetic, we decide to design an
Artificial Neural Network which learn from the training data set the movement performed by the
human.
However, developing myoelectric control based interfaces using the pattern recognition of
EMG signals is not a trivial task. There are large variations in EMG signals depending on the factors
such age, muscles activity, skin-fat layer, and movement performed. If we compare EMG signals with
other biosignals (i.e ECG) we can realize that it has a considerable amount of noise related by the
environment, the equipment, electromagnetic radiation, motion, etc. This behavior makes difficult to
determine the proper features of the signal for the classification problem.
Considering that a proper functional prosthetic should be able to perform a proper
classification in the shortest time possible, and taking into account that features such wavelets or
other frequency domain based features are time consuming at considerable computational cost; we
will only focus on time domain features.

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

There are several proposed time domain features to work with classification problems
(K.Englehart, 2001) (Z. K. Mahyar, 1995) (B, Parker, & RN, 1993), however most of those research limit
the amount of used features to 5 or 6 features for classification, in our proposed approach in order to
obtain a better characterization of the signal, we will work with 9 features: Mean absolute value
(MAV), variance (VAR), number of zero crossing (ZCC), skewness (SKW), root mean square (RMS),
standard deviation (STD), wavelength (WL), maximum and minimum value (MAX, MIN).
The data from the EMG signals was obtained from the EMG data set repository gently
provided by Phd. R. N. Khushaba (R. N. Khushaba, 2012). Eight subjects, six males and two females,
aged between 20 and 35 years were recruited to perform the required fingers movements. The
subjects were all normally limbed with no neurological or muscular disorders. All participants provided
informed consent prior to participating in the study. Subjects were seated on an armchair, with their
arm supported and fixed at one position to avoid the effect of different limb positions on the
generated EMG signals (Scheme, Founger, Stavdahl, Chan, & Englehart, 2010). The EMG data was
collected using two EMG channels (Delsys DE 2.x series EMG sensors) and processed by the Bagnoli
Desktop EMG Systems from Delsys Inc. A 2-slot adhesive skin interface was applied on each of the
sensors to firmly stick the sensors to the skin. A conductive adhesive reference electrode (Dermatrode
Reference Electrode) was utilized on the wrist of each subject. The positions of these electrodes are
show in Fig 2.2.

Figure 2.2 Electrodes Placement

The EMG signals collected from the electrodes were amplified using a Delsys Bagnoli-8
amplifier to a total gain of 1000. A 12-bit analog-to-digital converter (National Instruments, BNC-2090)
was used to sample the signal at 4000 Hz; the signal data were then acquired using Delsys EMGWorks
Acquisition software. The EMG signals were then bandpass filtered between 20 and 450 Hz with a
notch filter implemented to remove the 50 Hz line interference. Ten classes of individual and
combined fingers movements were implemented including: the flexion of each of the individual
fingers, i.e., Thumb (T), Index (I), Middle (M), Ring (R), Little (L) and the pinching of combined Thumb
Index (TI), ThumbMiddle (TM), ThumbRing (TR), ThumbLittle (TL), and finally the hand close
(HC).

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

Figure 2.3 Different movements included in the data set

To extract the features we only use a window of 0.2 seconds from the measurements,
because the goal of the project (which is successfully identify a movement and use that information
as input for a robotic prosthetic device) and finally we only try to identify what we consider the most
important movements which are: every finger movement, hand-close and Thumb-Index. Those
features will be used on a neural network (NN) in order to train and test a proper model to classify
the all the 7 proposed movements.
Through the following sections we will discuss our project development. We explain more
deeply the used features, the signal processing, the neural network design and results.

SIGNAL PROCESING
The EMG signals was recorded in periods of 5 seconds per measurement and we have 6
measurement per movement on every subject, however, if we perform feature extraction of the
whole signal we face two undesired problems: first the computational cost of extraction features of
20000 pieces of data on every measurement and the important delay between the feature extraction
and the action that the robotic prosthetic should achieve, (no matter how precise is the classification
if we need to maintain for 5 seconds the muscle contracted).
To avoid those undesired behaviors we propose work only on the very first 0.2 seconds (800
samples) of every movement measurement, thats make sense if we consider the overall goal:
measure an EMG impulse, determine which movement it is and sent the control input to the
prostheses, take into account that the human eye can barely distinguish delays under 0.5 sec. As fast
all this process is done is better the sensation of real time processing of the robotic hand.

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

Figure 3.1 EMG signals

As it is shown in Fig. 3.1 EMG signals are very similar, no matter the movement described the
signal does not shown evident difference between each other. As a first attempt to reduce
computational cost, since the measurement from both channels are related with the same movement
at the same time, we differentiate those inputs in order to obtain a unique value of movement data,
by doing that we also remove some noise in the process.
There are plenty information of proposed features to achieve this task, but most of them only
uses 5 or 6 features (Md. Rezwanul Ahsan, 2011), (Mohd Harris, 2015), In order to obtain the most
relevant information to uniquely identify what make an EMG signal from a movement different to
another one we propose the following set of features to extract:
1. Mean Absolute Value (MAV) where the average on the i-segment made of k samples is

This parameter is also useful to control the velocity of the movement of the prosthesis
(Matteo Arvetti, 2001).

2. Variance (VAR), uses the power of the signal as a feature

3. Number of zero crossing (ZCC), it is the number of times that the amplitude value of EMG
crosses the zero y-axis, this feature provides an approximate estimation of frequency domain
properties.

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

4. Skewness (SKW), it is a measure of asymmetry in the probability distribution of random

variable about it mean.

5. Root mean square (RMS), it is related to the constant force and non-fatiguing contraction.

6. Standard deviation (STD), it can be used to find the threshold level of the muscle contraction
activity.

7. Wavelength (WL), it is the cumulative length of the waveform over a time segment, with this
feature we can obtain a rough view if the relationship between the waveform amplitude,
frequency and time.

8. Maximum value (MAX), return the maximum value on the analyzed signal
9. Minimum value (MIN), return the minimum value on the analyzed signal

The feature extraction allows to obtain a 9x70 matrix of data from the 5 of the six
measurements of every movement performed by two subjects. This information will be necessary to
build the neural network.

CLASSIFICATION PROCESS
In order to classify those signals, we use an artificial neural network (ANN) to successfully
identify every one of the proposed target movements. There are different methods to achieve this
task, depending on the goal and the features we can use several approaches to obtain the better
result.
Our proposed NN is a scaled conjugate gradient backpropagation, which is a two-layer
feedforward network, with a sigmoid transfer function in the hidden layer, and a softmax transfer
function in the output layer (the same used by Matlab nprtool).

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

Figure 4.1 Proposed Neural Network

The feature matrix was divided in 70% of the data for training, 15% for testing and 15% for
validation, the resulting, the best fit occurs at epoch 53 (Fig 4.2).

Figure 4.2 Network performance

With the used training data we obtain ROC results very accurate, (closest to 100%), however
it is recommended to use more data in order to properly test the accuracy of the neural network
model.

Figure 4.3 Resulting ROC from the training stage

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

MEDICAL ROBOTICS

Finally the model was tested with new data (not used on training data), the results where
the following ones:
MOVEMENT
Index
Ring
Middle
Little
Thumb
Hand-Close
Thumb-Index

ACCURACY
89%
90%
88%
85%
89%
90%
88%
Table 1. Results from testing

As we can see on Table 1. The results during the testing stage was very satisfactory, other
research groups achieved similar results with different approaches.

CONCLUSIONS
Even if the resulting model is able to successful identify the movement of the fingers in rates
closest to 90%, we have to specify that the measurements were taken from a repository which uses
a very specialized device to record EMG signals, this is not usual on EMG robotic prosthesis. That
means that the obtained model is valid on similar quality input signals.
Most of the EMG signals acquired by those prosthetic requires most sophisticated signal
processing methods to achieve the same quality from the repository, also there are more added noise
coming not only from the acquisition but also from the movement itself.
Despite those disadvantages we consider that the implementation of a neural network as a
movement classifier is a good asset to add to a robotic prosthetic hand, this allows the use of only one
measurement of the signal to drive 7 different actions (most of the actual prosthetic hand use train
impulses to perform the same actions).

FUTURE WORK:
Along this report we were working with data from repositories, however is necessary to move
from the concept proof to a real scenario working with EMG signals acquired with a not so high-level
acquisition device.

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

10

MEDICAL ROBOTICS

The following step is to capture signals from Arduino and test how behaves this algorithm in
a real time scenario.

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

11

MEDICAL ROBOTICS

REFERENCES
B, H., Parker, P., & RN, S. (1993). A new strategy for multifunction myoelectric control. Biomedical
Engineering, IEEE Transactions, (pgs. 82,94).
De Luca, C. (11 de 2016). Delsys. Obtenido de
http://www.delsys.com/Attachments_pdf/Practicum%20on%20sEMG%20v1.5.pdf
K.Englehart, B. H. (2001). A wavelet based based continuous classification scheme for multifunction
myoelectric control. IEEE Transactions on Biomedical Engineering, (pgs. 302-311).
Matteo Arvetti, G. G. (2001). Classification of EMG signals through wavelet analysis and neural.
Md. Rezwanul Ahsan, M. I. (2011). Electromygraphy (EMG) Signal based Hand Gesture Recognition
using Artificial Neural Network (ANN). 4th International Conference on Mechatronics (ICOM).
Mohd Harris, P. C. (2015). EMG Signal Based Finger Movement Recognition for Prosthetic Hand
Control. International Conference on Communication, Control and Intelligent Systems.
Mu, M. O. (2007). Myolectric control system - A survey. Biomedical Signal Processing and Control,
(pgs. 275-294).
R. N. Khushaba, M. T. (2012). Toward Improved Control of Prosthetic Fingers Using Surface
Electromyogram (EMG) Signals. Expert Systems with Applications, (pgs. 10731-10738).
Z. K. Mahyar, W. C. (1995). feature evaluation for movement control of Upper extremity prostheses.
IEEE Transactions on Rehabilitation Engineering.

UNIVERSIDAD POLITECNICA DE CATALUNIA| MASTERS DEGREE IN AUTOMATIC CONTROL AND ROBOTICS

12