Biomedical Signal Processing and Control 55 (2020) 101669

Contents lists available at ScienceDirect

Biomedical Signal Processing and Control

journal homepage: www.elsevier.com/locate/bspc

An experimental study on upper limb position invariant EMG signal

classification based on deep neural network
Anand Kumar Mukhopadhyay a , Suman Samui b,∗
a
Department of Electronics and Electrical Communication Engineering, Indian Institute of Technology, Kharagpur, India
b
Advanced Technology Development Centre, Indian Institute of Technology, Kharagpur, India

a r t i c l e i n f o a b s t r a c t

Article history: The classification of surface electromyography (sEMG) signal has an important usage in the man-machine
Received 30 May 2018 interfaces for proper controlling of prosthetic devices with multiple degrees of freedom. The vital research
Received in revised form 29 April 2019 aspects in this field mainly focus on data acquisition, pre-processing, feature extraction and classifica-
Accepted 17 August 2019
tion along with their feasibility in practical scenarios regarding implementation and reliability. In this
article, we have demonstrated a detailed empirical exploration on Deep Neural Network (DNN) based
Keywords:
classification system for the upper limb position invariant myoelectric signal. The classification of eight
sEMG signal classification
different hand movements is performed using a fully connected feed-forward DNN model and also com-
Deep neural network
Electromyogram
pared with the existing machine learning tools. In our analysis, we have used a dataset consisting of
Upper-limb invariant the sEMG signals collected from eleven subjects at five different upper limb positions. The time domain
Hand movement classification power spectral descriptors (TDPSD) is used as the feature set to train the DNN classifier. In contrast to
Prosthetic application the prior methods, the proposed approach excludes the feature dimensionality reduction step, which
in turn significantly reduce the overall complexity. As the EMG signal classification is a subject-specific
problem, the DNN model is customized for each subject separately to get the best possible results. Our
experimental results in various analysis frameworks demonstrate that DNN based system can outper-
form the other existing classifiers such as k-Nearest Neighbour (kNN), Random Forest, and Decision Tree.
The average accuracy obtained among the five subjects for DNN, SVM, kNN, Random Forest and Decision
Tree is 98.88%, 98.66%, 90.64%, 91.78%, and 88.36% respectively. Moreover, it can achieve competitive
performance with the state-of-the-art SVM based model, even though the proposed DNN model requires
minimal processing in feature engineering. This study provides an insight into the detailed step-by-step
empirical procedure to achieve the optimum results regarding classification accuracy using the DNN
model.
© 2019 Elsevier Ltd. All rights reserved.

1. Introduction task in which the real-time signal is processed through the pre-
processing blocks involving filters, feature extractor and classifier
The classification of Electromyogram (EMG) signal is an impor- units in a short period (≈300 ms) to provide the desired output with
tant task for various applications such as clinical diagnosis, acceptable accuracy. Therefore, it requires a proper selection of the
Human-Computer Interaction (HCI) systems for prosthetic devices, size of the analysis window for real-time applicability [3–6]. The
myoelectric controllers used in wireless controlling devices such as imposed time constraint is an essential design criterion to ensure
video game controller and security check-in for authorized access the flawless functioning of the HCI systems for the end users.
[1]. In order to verify whether a person is suffering from any neuro- The shape and firing rate of the Motor Unit Action Potentials
muscular disorder, namely neuropathy and myopathy, the offline (MUAPs), contained in the EMG signal, carry essential information
classification methods are often employed by collecting myoelec- about the characteristic of the signal which is often contaminated
tric signals from the diagnosed region of the muscles [2]. However, by noise. Henceforth, one can exploit the characteristic of MUAPs
the real-time prediction is considered to be a far more challenging in analyzing the signal for clinical diagnosis by employing any effi-
cient noise suppression technique which will allow the acquisition
of useful content from surface EMG (sEMG) signal. Furthermore,
∗ Corresponding author. with the advancement of signal processing and machine learning
E-mail addresses: [email protected] (A.K. Mukhopadhyay), techniques, various new methods have been introduced for the
[email protected] (S. Samui).

https://doi.org/10.1016/j.bspc.2019.101669
1746-8094/© 2019 Elsevier Ltd. All rights reserved.
2 A.K. Mukhopadhyay and S. Samui / Biomedical Signal Processing and Control 55 (2020) 101669

EMG signal classification. In recent years, the supervised learning ensuring a distinct muscle activity pattern. ICA has been a useful
based methods are advancing by considering mainly three aspects: tool for separating MUAPs from multi-channel sEMG [19], viz., to
data collection techniques and its pre-processing, feature extrac- estimate the muscle force from surface HD-EMG with a significant
tion methods and suitable classifier design. However, the proper reduction in error [20], to remove the contaminated ECG artifacts
deployment of these methods on the hardware also requires sig- by using ICA with wavelet analysis [21,22] and hence is a superior
nificant research attention for the real-time application [1,7]. feature extraction technique. However, ICA which is an iterative
A vital aspect lies in the acquisition of data through electrodes algorithm will increase the complexity of the feature extraction
from the specific region of muscles of the subject which can either process and can work only under certain stringent conditions, for
be invasive or non-invasive depending on the end user system. instance, the number of recording channels has to be more than or
In invasive methods, the prosthetic device needs to be connected equal to the number of independent source signals and the source
directly to the targeted muscle tissues surgically whereas, in non- signals should be independent of each other.
invasive methods, signals are recorded from the surface of the Recently, a sophisticated deep neural network termed as deep
muscle regions. Invasive methods prove to be successful for real- time growing neural network (DTGNN) has been shown to classify
time control of artificial arms, but they are less preferred compared biological signals having cyclic time series (CTS) properties with a
to the non-invasive case by the end users, so as to avoid the sur- one-versus-all classification approach [23]. CTS of a signal resem-
gical process which may increase the chances of causing infection bles a time series having repetitive characteristics over a set of
[8]. Now, for the non-invasive method of extracting the signals, successive temporal intervals. The DTGNN classifier can overcome
it is quite evident that multi-channel electrodes are more effec- the complexities of the CTS biological signals through the cyclic
tive compared to a single-channel case because it can cover more learning algorithms involving three different schemes namely for-
regions around the muscle area of a subject. Hence, in this work, ward, bilateral and backward time growing schemes which are
we are mainly focusing on the sEMG data collected from the upper then fed to a time growing neural network (TGNN) [24]. A TGNN
limb of the subjects through the multi-electrode channels at differ- classifier, which is a sub-component of DTGNN has shown bet-
ent limb positions. During the last decade, the EMG classification ter results when compared to time delay neural network (TDNN)
task is investigated extensively, which has led to the development and MLP for classification of short-duration heart sounds [24,25].
of numerous methods which mainly includes various statistically These classifiers (DTGNN, TGNN, and TDNN) increase the number
induced mathematical models [9], discriminative learning models of weight coefficients depending on the number of window seg-
[1] and genetic algorithm [10] based techniques. The important ments for better learning and considers the spectral content of the
approaches for classification of sEMG signals for controlling upper signals for better performance. However, in this work, we have
limb prostheses are Linear Discriminant Analysis (LDA), Support omitted enhanced feature extraction processes and analyzed the
Vector Machines (SVMs), and Hidden Markov Models (HMM) which performance of DNN using TDPSD features extracted from the time
provide a marginal enhancement in performance regarding classifi- domain [26] for sEMG classification. We have not chosen sophis-
cation accuracy [11]. Even though these are proven classifiers, their ticated deep neural networks having the capability to solve CTS
shortcomings are, viz., (1) LDA needs more computational time for problems because it will increase the complexity of the online clas-
determination of eigenvalues which is required to reduce the fea- sifier due to the increased number of weight coefficients and the
ture dimension, (2) Though SVM gives good result with a proper complex feed-forward classification process. The main advantage
combination of simple time domain features [12] such as MAV of DNN lies in the fact that it does not need a very sophisticated fea-
(Mean Absolute Value), WL (Waveform Length), ZC (Zero Crossing) ture extraction pipeline. In other words, it is an excellent tool for
and SSC (Slope Sign Change) which have a low variance for differ- classifying multiple classes without undergoing a complex feature
ent choice of window lengths, data segmentation is performed for extraction process.
better results. In addition to that an appropriate choice of kernel for In this paper, we have examined the application of deep learning
SVM is needed which is a challenging task [13], (3) HMM, on the to EMG classification task. A recent work by Atzori et al. have used
other hand, does not need segmentation of EMG data and uses root Convolutional Neural Network for classifying 50 hand movements
mean square with Auto-Regressive (AR) coefficients as features, has for 67 intact subjects and 11 trans-radial amputees to find accu-
shown higher accuracy than Multi-Layer Perceptron (MLP) and is rate results compared to the other classification methods like k-NN,
suitable for real-time implementation [14]. The AR model helps in SVM, Random Forest [27]. Just like any other supervised learning
generating less number of features; and was also used on myoelec- tasks in many other domains [28–30], deep learning has emerged
tric signal collected from a single electrode channel for upper limb to be a useful tool in the field of medical informatics in recent years
prostheses and later feeding it to a Fuzzy Clustering Neural Network due to its remarkable performance [31]. Furthermore, D. Yang et al.
Classifier which proved to be better than its counterpart neural net- did a classification of multiple finger motions considering dynamic
works consisting of MLP, back propagation NN and conic-section upper limb movements [32]. This study differs from previous stud-
function NN [15,16]. ies in several ways. The major contributions of this manuscript are
The use of artificial neural networks (ANNs) for sEMG signal given as follows:
classification has been attempted earlier for hand gesture recogni-
tion, either by combination with Independent Component Analysis • We focused on fully connected deep neural network architecture
(ICA) or by using both time and frequency based features [17,18]. It for the classification of hand movements using EMG signal. For
is to be noted that the sEMG signal can be analyzed either by con- this study, we have used the dataset consisting of the myoelectric
sidering the gross signal or by extracting the useful information signal obtained from the upper limb of different subjects using
contained in the form of MUAPs from it. The gross signal is a com- multi-electrode channels at five different limb positions, which
bination of signals generated from overlapping muscles whereas were recorded by Khushaba et al. [33].
the useful information content is obtained by separating the sig- • Moreover, most of the previous studies relied on some form of
nals generated from the overlapping muscles. However, due to the high-level features, while in this paper we strive for minimal
similarity in the properties of the signals from different muscle signal processing and rely on deep learning to automate the pro-
regions, the task of separating the overlapping signals becomes cess of feature extraction. We have chosen a fused time domain
a blind source separation (BSS) problem. ICA has been proven to descriptors (fTDD) method with multiple myoelectric channels
be successful in solving BSS tasks in bio-signal applications [17]. while avoiding the feature dimension reduction process [26,34].
In ICA, some prior knowledge of muscle anatomy is required for fTDD is a fusion of the extracted features from the current and
 A.K. Mukhopadhyay and S. Samui / Biomedical Signal Processing and Control 55 (2020) 101669 3

Table 1
List of abbreviations.

Abbreviations Full meaning

ANN Artificial neural networks

AR Auto regressive
BSS Blind source separation
cTDD Correlated time domain descriptors
CTS Cyclic time series
DNN Deep neural network
DTGNN Deep time growing neural network
Fig. 1. The major blocks involved in the classification of EMG signal in real time.
fTDD Fused time domain descriptors
HCI Human Computer Interaction
HMM Hidden Markov Models ing the amplitude of the sEMG signal for accurate simultaneous
ICA Independent Component Analysis
and proportional prosthetic control [35]. After, filtration of the
IF Irregularity Factor
kNN k-Nearest Neighbors raw sEMG signal, relevant features are to be extracted and fed to
LDA Linear discriminant analysis the classifier block for predicting the outcome class of movement,
MLP Multi-Layer Perceptron which is used by the following prosthetic device. To perform all
MUAPs Motor unit action potentials the operations within a stipulated amount of time with accept-
ReLU Rectified linear unit
able performance, the time duration of the window size (twinsize )
ROC Receiver operating characteristics
sEMG Surface electromyography and window increment time (tinc ) for analyzing the data should be
SSC Slope Sign Change chosen wisely.
SVM Support vector machines In this work, an appropriate twinsize and tinc is chosen for accept-
TDNN Time delay neural network
able real-time controllability, i.e., controller delay tcontroller smaller
TDPSD Time domain power spectral descriptors
TGNN Time growing neural network
than 300 ms. We have performed the classification offline while
WL Waveform length considering the constraints such as the twinsize and tinc to be suit-
ZC Zero crossing able for online classification. Another important point to be noted is
that though twinsize corresponds to each of the data sample size, tinc
decides the throughput of processing the signal. We have consid-
previous window for enhancing the robustness of the extracted ered twinsize = 100 ms and tinc = 25ms for creating the data samples
set of features [26]. for analysis which is an acceptable condition required for online
• Furthermore, to the best of our knowledge, our work provides the classification. Further details for choosing the features from the
first step-by-step detailed empirical exploration of deep learning data samples is described in Section 4.
methods applied to EMG classification task. Please note that our
objective is not to achieve the state-of-the-art performance, but 3. EMG materials: dataset
rather to explore the deep learning framework for EMG classifi-
cation to the other existing EMG classification schemes whose We have used the dataset consisting of the myoelectric signal
performance often depends on the complex feature set. Fur- obtained from the upper limb of subjects via seven electrode chan-
ther improvement may straightforwardly be attained with larger nels at five different limb positions from different subjects, which
datasets and more complex models like RNN and CNN. were recorded by Khushaba et al. [33].1 The eight different classes
of hand movements considered are wrist flexion, wrist extension,
The paper is organized as follows: Initially, the motivation of wrist pronation, wrist supination, power grip, pinch grip, open hand
classifying myoelectric signal for different applications and a brief and rest. This dataset is considered to be useful from the fact that
survey of work related to it is elucidated in Section 1. In Section it considers the variation in limb position which does affect the
2, a system level overview of sEMG classification in real time is characteristics of the sEMG signal [36,37]. Here, the data has been
described. The materials comprising of the sEMG dataset is pre- recorded from the circumference of the forearm of healthy subjects.
sented in Section 3. The TDPSD features used are elaborated in The data collection approach is valid for trans-radial amputees as
Section 4. In Section 5, a brief theory of the deep learning architec- well in which the data would be recorded from the muscles in their
ture and the learning method is described. The analysis procedure residual forearm. Hence, with appropriate training, while taking
leading to the experimental results is explained in a systematic into account the different limb positions, we can expect to get a
manner in Section 6. In Section 7, the performance of DNN com- better test accuracy in real time scenarios where the user usually
pared with four baseline classifiers is discussed. Finally, the paper performs the different hand gestures at different limb positions. The
is concluded by summarizing the key points in Section 8. Table 1 applications benefited from this dataset are, viz., (1) different EMG
shows the list of abbreviations used in the article. based control applications such as exoskeleton/robotic hands, EMG
based video game controllers, and (2) prosthetic hand controllers
2. System level overview of real-time sEMG classification for trans-radial amputees.

An overview of the blocks used to describe the process of EMG 4. Feature selection
classification is illustrated in Fig. 1. Initially, the training of a par-
ticular subject is done offline, and then the generated trained Fused time domain descriptors (fTDD) are the features which
parameters are used in the online (embedded integrated circuit combine the essential information between the current window
based) classifier. The function of the filter block is to retrieve the (nth ) and the previous window ((n − step)th ), in which step = 1, 2,. . .,
relevant portion of the sEMG signal (between 40 Hz to 450 Hz) by N, determines the separation between the current and previous
eliminating the high-frequency noise and power line interference windows [26]. The advantage of this scheme is that if the two win-
frequency (50 Hz) from the raw sEMG signal. The process needs dows occur during the same class, it will enhance the correlation
to be fast and accurate for better performance of the subsequent
blocks. Recently, Hofmann proposed a Bayesian Filter which has
shown to be superior to the conventional linear filters in estimat- 1
https://www.rami-khushaba.com/electromyogram-emg-repository.html.
4 A.K. Mukhopadhyay and S. Samui / Biomedical Signal Processing and Control 55 (2020) 101669

between the fused features, whereas for non-similar classes the 5. Classification method: deep learning architecture
correlation will be diminished. In further analysis, we have consid-
ered fusion of features from two consecutive windows (i.e., step = 1) Deep learning in neural networks is the approach of compos-
which will also determine the total number of data samples as given ing networks into multiple layers of processing to meaningfully
in Eq. (8). transform data to learn a useful representation of the input data.
fTDD is formed by the fusion of correlated time domain descrip- The essence of deep learning lies in this idea of successive layers
tors (cTDD) which is extracted from individual windows. The of representations which in turn helps to learn higher-level fea-
feature set of cTDD consists of root squared zero-order moment, tures from low-level ones in a hierarchical manner, nullifying the
root squared fourth and eighth order moments, sparseness, Irreg- over-dependence of shallow networks on feature engineering [40].
ularity Factor (IF), and WL ratio. The six-time domain descriptors For this reason, the deep learning is often referred to as hierarchical
used can be mathematically expressed as follows (Eqs. (1)–(7)): representations learning in several contexts in the literature.
In the current work, we have adopted a feed-forward neural
f1 = log(m0 ) (1) network architecture which comprises of multiple layers of trans-
formations and non-linearity with the output of each layer feeding
f2 = log(m0 − m4 ) (2)
the subsequent layer. Mathematically, this DNN model can be inter-
f3 = log(m0 − m8 ) (3) preted as given in Eq. (9) and Eq. (10):

Z (l) = y (l−1) W (l) + b(l) (9)

f4 = log(S) (4)
(l) (l)
y = g(Z ) (10)
f5 = log(IF) (5)
where l ∈ [1,. . ., L] denotes the layer, lth Zl
is the vector of pre-
f6 = log(WL) (6) activations of layer l, y(l−1) is the output of previous layer (l − 1)
and input to layer l. W l ∈ Rni ×no is a matrix of learnable biases of
mn layer l, y (l) ∈ Rno is the output of layer l, y(0) is the input to the
mn = ; n = 0, 2, 4, 8. (7)
 model and y(L) is the output of the final layer L of the model, g(.)
 denotes the non-linear activation function. ReLU has been used in
N−1 2
where the parameters  = 0.1, m0 = 2
x[j] , the hidden layers due to its computational efficiency and faster
    j=0
learning convergence over other activation functions. To provide a
2 1 N−1 2 2 1 N−1 2
m2 = N
(x[j]) , m4 = N
(2 x[j]) , m8 = probabilistic interpretation of the model’s output, the output layer
  j=0 j=0

2 1 N−1 2 m0 m L utilizes a softmax non-linearity as in Eq. (11):

N j=0
(4 x[j]) , S= √ √
m −m m −m
, IF = √ 4
m m
, WL =
N−1 2 0 4 0 8 0 8
expZ k
| x[j]| softmax(Z (L) ) = K (11)
j=0
N−1 . The parameter N indicates the total number of expZ k
4 | x[j]| k=1
j=0
samples of the EMG signal. In addition, , 2 , 3 and 4 repre- where K is the number of output classes, i.e., the output layer con-
sent the first, second, third and fourth derivatives, respectively. tains K number of neurons.
The EMG signal and its nonlinear version extracted directly from
the time-domain can aid in reducing the computational cost 5.1. Learning
[38,39]. Basically, a set of six features were obtained per channel,
implying a total of 42 (6 features × 7 channels) feature components The learning of a DNN is formulated as an optimization problem
representing each data sample which is formed by concatenation to minimize a cost-function [41]. As we are dealing with classifica-
of features from all the channels present [26]. The set of 6 features tion problem, the cross-entropy loss function has been used in this
from each channel is computed from two consecutive windows of work as given in Eq. (12):
size, twinsize = 100 ms with an increment in shift of time window,
tinc = 25 ms. This is a reasonably good approximation of window 
K
C=− ŷk log(yk (L) ) (12)
length for real-time classification of myoelectric signals [4,5]. The
generated feature set from the multiple channels are fed directly to k=1

the DNN trainer for finalizing the trained parameters (weight/bias where ŷ ∈ {0, 1}K is one-hot-encoded label and y(L) is the output of
coefficients) to be used by the DNN classifier. Each type of hand the model. In order to train the model, the gradients are computed
movement (class) was performed for 5 s with a rest of 3 s interval by differentiating the cost-function w.r.t. the model parameters
for 6 trials. Hence, for a particular subject, class, and limb position using a mini-batch of data sampled from the training data and back-
the number of data extracted can be calculated using Eq. (8) which propagated to prior layers using the backpropagation algorithm
results in 196 samples for the current case. [41].
T − (twinsize − tinc )

dur
Ndata = − step (8) 6. Experimental results
tinc

where Ndata , Tdur , twinsize and tinc are the number of data samples, The earlier studies by Khushaba et al. have shown the impor-
the total duration of movement, window size, and window incre- tance of the feature extraction methods and compared between
ment time respectively. Further, for 8 different type of movement different types of classifiers such as SVM, LDA, kNN and Extreme
classes and 6 trials for a particular limb position, the number of data Learning Machine (ELM) with a good amount of accuracy [33]. The
samples is 9408 (8 classes × 6 trials × 1 limb position × 196 data current work is not to compete with the state of the art results
samples). The features representing the 8 classes of hand move- obtained, but to do an extensive analysis with essential TDPSD fea-
ments are non-linearly separable which can be confirmed from the tures without undergoing the feature dimension reduction process,
scatter plot as shown in Fig. 2. Only the first three features from the using DNN based classifier with an optimum configuration and a set
feature vector are considered for visualization purpose for the five of parameters on the similar set of data. In Fig. 3, a schematic model
limb positions individually. of the DNN classifier is depicted which takes the feature set as input
 A.K. Mukhopadhyay and S. Samui / Biomedical Signal Processing and Control 55 (2020) 101669 5

1 1

0.95

0.95 0.95

0.9
0.9 0.9 1
1 1 1
1 0.98 0.95
0.95 0.95 0.96 0.95
0.95 0.94
0.92 0.9 0.9
0.9 0.9 0.9 0.9

1 1

0.95
0.95
0.9

0.9 0.85
1 1
0.98 0.95 0.98
0.95 0.96 0.96
0.94 0.9 0.94
0.92 0.92
0.9 0.9 0.85 0.9

Fig. 2. Scatter plot of first three components of fTDD feature vector for limb positions (1–5). Each colour pattern represent one of the 8 class of hand movement. (For
interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Table 2
Analysis 1: Inter-position classification accuracy (in %), averaged across 5 subjects
and 8 classes of hand movements.

Testing positions

POS1 POS2 POS3 POS4 POS5

POS1 98.85 70.29 62.44 71.95 61.69

POS2 76.62 95.27 76.81 74.99 67.13
Training positions POS3 71.66 73.62 96.38 69.03 61.59
POS4 70.63 66.70 66.75 97.46 86.10
POS5 68.45 71.11 66.01 88.93 97.29

was trained separately to see the effect of each position on the test
set as shown in Table 2. Understandably, the maximum accuracy
will be in the cases when the train/test data are taken from the
same position. From this matrix, we can get the idea of the corre-
Fig. 3. The schematic of the DNN classifier receiving the feature set as an input and lation between different positions. For instance, P1 and P5, which
predicting the result at the output. As it is a multi-class classification problem a
are the two extreme arm positions in which data is collected from
softmax function is used at the output layer. The activation function in the hidden
layer can be relu, tanh or sigmoid. the subjects, will have a poor correlation due to the fact that the
angle of deviation between the two arm positions is the maximum
(≈135◦ ). Also, P1 is in a relatively relaxed position as compared to
to the first layer, followed by the hidden layers and output layer to
P5 which will have a strong impact in the variation of the EMG
give the classification result.
signals generated.
Initially, similar analysis as mentioned in [33] were performed
on the considered features derived from the dataset comprising of
upper limb EMG signal, using DNN for validation purposes. The 6.2. Analysis 2: Training in multiple limb position
results obtained were following similar trends when compared
with their results obtained using SVM classifier [33]. Next, the In this analysis, testing of data is done on individual limb posi-
dataset was divided appropriately based on analysis 1 and 2 to con- tions separately, in which the training dataset consists of data from
tinue the experiment process. Afterwards, a comparative analysis all the arm positions except the position which is being tested. The
between the DNN classifier with other classifiers, viz., SVM, kNN, test positions which show low accuracy do not have a good likeness
random forest and decision tree was performed in terms of classi- with the other positions and hence must be considered in training
fication accuracy. The different analysis undertaken are mentioned set for better accuracy. We can confirm that positions 2 (P2), 4 (P4)
henceforth, out of which analysis 1 and 2 are done for validation and 5 (P5) are having better test accuracy as compared to other
purpose. The accuracy mentioned in our analysis is the average positions from the analysis as depicted in Table 3.
accuracy considering the five subjects.
6.3. Analysis 3: Dividing the train/test set appropriately
6.1. Analysis 1: Training in single limb position
As collecting the data at all the different positions is a tiring task
The train/test data were divided in equal proportion, i.e., for each for the subject, it is preferable to train at lesser limb positions while
subject at a particular limb position, there are (4704 + 4704) sam- keeping in mind its effect on the accuracy of classification. The posi-
ples of the train and test data consisting of eight classes of hand tions which should be considered in the train data are P1, P3, and
movements in a balanced manner. Here, data from each position P5 which can be understood from the combined result of analysis 1
6 A.K. Mukhopadhyay and S. Samui / Biomedical Signal Processing and Control 55 (2020) 101669

Table 3
Classification accuracy when training on EMG data from all positions, except the
one being tested on. These results are the average across all 8 movements and all 5
subjects.

Testing positions

POS1 POS2 POS3 POS4 POS5

69.37 76.61 73.53 74.73 74.87

and 2. From now onwards, further analysis will be done considering

the train data from the mentioned limb positions (P1, P3, and P5).
To get the best possible classification accuracy of sEMG using DNN
classifier with the optimal set of design parameters, the dataset is
divided into three sets, viz., train set, validation set and test set.
The total number of data samples for a particular limb position is
divided in (7:3) ratio i.e. (6592 + 2816 = 9408) corresponding to the
train and (validation + test) set respectively. Considering positions Fig. 4. Validation loss for different DNN models.
P1, P3 and P5 for the train set and all positions for validation and
test set, the data distribution in the train, validation and test set are
19,776 (≈58%), 7040 (≈21%) and 7040 (≈21%) respectively. A point a fully connected feed-forward architecture comprising three hid-
to be noted is that the data samples are chosen randomly from the den layers. The number of units in the input layer is equal to the
5s duration and allocated to train/ validation/ test set such that all feature dimension, and the output layer has a dimension equal to
the eight possible classes are distributed in a balanced fashion. the number of classes (i.e., 8). The choice of activation functions
Please note that EMG classification problem is absolutely sub- in deep networks has a significant effect on the training dynamics
ject specific. Henceforth, the parameters of each DNN has been and task performance. We have used the rectified linear unit (ReLU)
optimized for a specific subject. In other words, the subject is not as non-linearity in the hidden layers as ReLU yields much better
considered as a dimension of generalization; rather we want to optimized deep network with faster learning [42]. To prevent over-
achieve trained generalized model regarding various limb posi- fitting, we employed both L2 regularization and dropout separately
tions. In our experiment, we have used DNN classifier which follows as shown in Fig. 4. As the performance of DNN model with dropout

Fig. 5. Effect of various optimizer is illustrated in terms of loss on (a) train data and (b) validation data.

Fig. 6. DNN model accuracy and loss.

 A.K. Mukhopadhyay and S. Samui / Biomedical Signal Processing and Control 55 (2020) 101669 7

Fig. 9. Average test accuracy using the classifiers: NN, SVM, Random Forest, Decision
Tree, kNN.

7. Discussion

The dataset considered in analysis 3 (described in Section 6.3) is

evaluated with four other classifiers, viz., (1) SVM, (2) kNN, (3) Ran-
Fig. 7. Normalized confusion matrix.
dom Forest and (4) Decision Tree to compare with the performance
of DNN. Due to the subject specificity of the problem the classi-
fiers are fine-tuned to get the best possible results for each subject
considered. A Python based machine learning toolbox ‘Scikit-learn’
has been used to evaluate the performance based on analysis 3. The
complexity parameter C of the SVM classifier (LIBSVM library), as
well as the gamma parameter in the kernel function (‘linear’), were
optimized for each subject. In addition, we have used the LIBSVM
classifier with one-versus-rest approach as this method is provid-
ing better performance than one-versus-one approach. Similarly,
we have used kNN with ‘Brute-force’ algorithm and ‘minkowski’
distance metric. The average accuracy obtained among the five sub-
jects for SVM, kNN, Random Forest and Decision Tree are 98.66%,
90.64%, 91.78%, and 88.36% respectively as evident from Fig. 9. It
can be inferred that the accuracy obtained using DNN and SVM are
comparable and relatively higher than the other classifiers used.
One important point to be noted is that DNN can reduce the classi-
fication error for any subject by a significant amount compared to
SVM by tweaking the parameters, which will be different for each
subject. The other existing deep learning models such as DTGNN,
Fig. 8. ROC curve for multi-class classification. have the ability to extract rich contextual information from CTS sig-
nals. However, the use of DTGNN would increase the computational
overhead in the online classifier and hence is not explored in the
(with probability = 0.5) perform significantly better than L2 regular- current work. However, the performance of DTGNN could be stud-
ized model for most of the subjects under consideration, we used ied in the future on EMG signal classification performance for clini-
dropout for almost all experiments. Fig. 5 illustrates the effect of cal diagnosis of a patient in which the inference is performed offline.
different types of optimization algorithms in terms of loss on train
and validation data. It is quite evident from Fig. 5 that Adam opti- 8. Conclusions
mizer performs the best followed by SGD with momentum method
and the rest. Hence, we choose the Adam optimizer for evalua- This study mainly focused on the classification of the upper limb
tion. In addition, batch-normalization technique [43] has also been invariant myoelectric signal. The position of the limb effects the sig-
applied during fine-tuning which helps to make DNN more robust nal generated through the muscle surface and hence is a vital point
to the changes in the hyperparameters [44]. For implementing the to consider during the data collection process. In this article, we
DNN model, we have used Keras (with Tensorflow backend) [45], a delved into the detailed empirical exploration of DNN based EMG
popular deep learning programming framework. classification system. Our approach has avoided refining the EMG
An instance of the result of DNN training regarding loss function signal to obtain the MUAPs using methods such as ICA and con-
and accuracy have been illustrated in Fig. 6. The normalized con- sidered only the relevant features which can be directly obtained
fusion matrix for a particular subject (S2 considered) is plotted in from the time domain signal. The primary reason is that DNN can
Fig. 7 which depicts that the DNN model can perform significantly itself help in feature engineering aspects, hence discriminative fea-
well regarding accuracy. As the process of training is subject spe- tures requiring complex procedures is not necessary. Experimental
cific, similar confusion matrix can be generated for other subjects results in various analysis frameworks have demonstrated that
once the network parameters are optimized. It can be seen that DNN based system can outperform other classifiers such as kNN,
class 4 (wrist supination) has the lowest accuracy or in other words Random Forest and Decision Tree. Moreover, it can achieve com-
is having less number of true positives which is confirmed from petitive performance with the state-of-the-art SVM based model,
Fig. 8 which depicts the receiver operating characteristics (ROC) to even though the proposed DNN model requires minimal computa-
a multi-class problem. tion on feature engineering. The future work will be devoted to the
8 A.K. Mukhopadhyay and S. Samui / Biomedical Signal Processing and Control 55 (2020) 101669

exploration of other complex DNN models such as RNN (LSTM, GRU, [19] H. Nakamura, M. Yoshida, M. Kotani, K. Akazawa, T. Moritani, The application
etc.) to improve the performance further. Also, the performance of of independent component analysis to the multi-channel surface
electromyographic signals for separation of motor unit action potential
complex deep learning models such as DTGNN could be studied on trains: Part I – Measuring techniques, J. Electromyogr. Kinesiol. 14 (4) (2004)
different bio-medical applications having signals with CTS prop- 423–432.
erties. Although the dataset analyzed in this work has advantages [20] D. Staudenmann, A. Daffertshofer, I. Kingma, D.F. Stegeman, J.H. van Dieen,
Independent component analysis of high-density electromyography in
because of the non-invasive nature of recording the sEMG signals muscle force estimation, IEEE Trans. Biomed. Eng. 54 (4) (2007) 751–754.
from the circumference of the forearm of the subjects at differ- [21] J. Taelman, S. Van Huffel, A. Spaepen, Wavelet-independent component
ent upper limb positions, it has considered only healthy subjects. analysis to remove electrocardiography contamination in surface
electromyography, in: Engineering in Medicine and Biology Society, 2007,
Therefore, in future work there is scope for exploring other datasets
EMBS 2007. 29th Annual International Conference of the IEEE, IEEE, 2007, pp.
[46] consisting of amputees along with different DNN models. 682–685.
[22] B. Mijovic, M. De Vos, I. Gligorijevic, J. Taelman, S. Van Huffel, Source
separation from single-channel recordings by combining empirical-mode
Acknowledgments decomposition and independent component analysis, IEEE Trans. Biomed.
Eng. 57 (9) (2010) 2188–2196.
This work has been supported by the Ministry of Human [23] A. Gharehbaghi, M. Lindén, A deep machine learning method for classifying
cyclic time series of biological signals using time-growing neural network,
Resource Development (MHRD), Govt. of India. The authors are
IEEE Trans. Neural Netw. Learn. Syst. 29 (9) (2018) 4102–4115.
thankful to the editor and the anonymous reviewers for their [24] A. Gharehbaghi, T. Dutoit, P. Ask, L. Sörnmo, Detection of systolic ejection click
insightful suggestions and constructive feedback throughout the using time growing neural network, Med. Eng. Phys. 36 (4) (2014) 477–483.
[25] A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, K.J. Lang, Phoneme recognition
review process, which have considerably helped to improve the
using time-delay neural networks, in: Readings in Speech Recognition,
content of the article. Elsevier, 1990, pp. 393–404.
[26] R.N. Khushaba, A. Al-Ani, A. Al-Timemy, A. Al-Jumaily, A fusion of
time-domain descriptors for improved myoelectric hand control, in: 2016
Declaration of Competing Interest
IEEE Symposium Series on Computational Intelligence (SSCI), IEEE, 2016, pp.
1–6.
None declared. [27] M. Atzori, M. Cognolato, H. Müller, Deep learning with convolutional neural
networks applied to electromyography data: a resource for the classification
of movements for prosthetic hands, Front. Neurorobot. 10 (2016) 9.
References [28] C.-C. Chiu, T.N. Sainath, Y. Wu, R. Prabhavalkar, P. Nguyen, Z. Chen, A. Kannan,
R.J. Weiss, K. Rao, E. Gonina, et al., State-of-the-art speech recognition with
[1] M.B.I. Reaz, M. Hussain, F. Mohd-Yasin, Techniques of EMG signal analysis: sequence-to-sequence models, in: 2018 IEEE International Conference on
detection, processing, classification and applications, Biol. Proced. Online 8 (1) Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2018, pp. 4774–4778.
(2006) 11. [29] I. Sutskever, J. Martens, G.E. Hinton, Generating text with recurrent neural
[2] C.I. Christodoulou, C.S. Pattichis, Unsupervised pattern recognition for the networks, Proceedings of the 28th International Conference on Machine
classification of EMG signals, IEEE Trans. Biomed. Eng. 46 (2) (1999) 169–178. Learning (ICML-11) (2011) 1017–1024.
[3] K. Englehart, B. Hudgins, A robust, real-time control scheme for multifunction [30] S. Samui, I. Chakrabarti, S.K. Ghosh, Time-frequency masking based
myoelectric control, IEEE Trans. Biomed. Eng. 50 (7) (2003) 848–854. supervised speech enhancement framework using fuzzy deep belief network,
[4] T.R. Farrell, R.F. Weir, The optimal controller delay for myoelectric prostheses, Appl. Soft Comput. 74 (2019) 583–602.
IEEE Trans. Neural Syst. Rehabil. Eng. 15 (1) (2007) 111–118. [31] D. Ravì, C. Wong, F. Deligianni, M. Berthelot, J. Andreu-Perez, B. Lo, G.-Z. Yang,
[5] L.H. Smith, L.J. Hargrove, B.A. Lock, T.A. Kuiken, Determining the optimal Deep learning for health informatics, IEEE J. Biomed. Health Inform. 21 (1)
window length for pattern recognition-based myoelectric control: balancing (2017) 4–21.
the competing effects of classification error and controller delay, IEEE Trans. [32] D. Yang, W. Yang, Q. Huang, H. Liu, Classification of multiple finger motions
Neural Syst. Rehabil. Eng. 19 (2) (2011) 186–192. during dynamic upper limb movements, IEEE J. Biomed. Health Inform. 21 (1)
[6] D. Farina, N. Jiang, H. Rehbaum, A. Holobar, B. Graimann, H. Dietl, O.C. (2017) 134–141.
Aszmann, The extraction of neural information from the surface EMG for the [33] R.N. Khushaba, M. Takruri, J.V. Miro, S. Kodagoda, Towards limb position
control of upper-limb prostheses: emerging avenues and challenges, IEEE invariant myoelectric pattern recognition using time-dependent spectral
Trans. Neural Syst. Rehabil. Eng. 22 (4) (2014) 797–809. features, Neural Netw. 55 (2014) 42–58.
[7] K. Englehart, B. Hudgins, P.A. Parker, M. Stevenson, Classification of the [34] A. Phinyomark, P. Phukpattaranont, C. Limsakul, Feature reduction and
myoelectric signal using time-frequency based representations, Med. Eng. selection for EMG signal classification, Expert Syst. Appl. 39 (8) (2012)
Phys. 21 (6) (1999) 431–438. 7420–7431.
[8] T.A. Kuiken, G. Li, B.A. Lock, R.D. Lipschutz, L.A. Miller, K.A. Stubblefield, K.B. [35] D. Hofmann, N. Jiang, I. Vujaklija, D. Farina, Bayesian filtering of surface EMG
Englehart, Targeted muscle reinnervation for real-time myoelectric control of for accurate simultaneous and proportional prosthetic control, IEEE Trans.
multifunction artificial arms, JAMA 301 (6) (2009) 619–628. Neural Syst. Rehabil. Eng. 24 (12) (2016) 1333–1341.
[9] Y. Zhang, W. Herzog, M. Liu, A mathematical model of myoelectric signals [36] O.W. Samuel, X. Li, Y. Geng, M.G. Asogbon, P. Fang, Z. Huang, G. Li, Resolving
obtained during locomotion, in: Engineering in Medicine and Biology Society, the adverse impact of mobility on myoelectric pattern recognition in
1995, IEEE 17th Annual Conference, vol. 2, IEEE, 1995, pp. 1403–1404. upper-limb multifunctional prostheses, Comput. Biol. Med. 90 (2017) 76–87.
[10] M.A. Oskoei, H. Hu, Ga-based feature subset selection for myoelectric [37] M. Jochumsen, A. Waris, E.N. Kamavuako, The effect of arm position on
classification, in: IEEE International Conference on Robotics and Biomimetics, classification of hand gestures with intramuscular EMG, Biomed. Signal
2006, ROBIO’06, IEEE, 2006, pp. 1465–1470. Process. Control (2018), http://dx.doi.org/10.1016/j.bspc.2018.02.013.
[11] P. Erik Scheme, P. Kevin Englehart, Electromyogram pattern recognition for [38] A. Phinyomark, F. Quaine, S. Charbonnier, C. Serviere, F. Tarpin-Bernard, Y.
control of powered upper-limb prostheses: state of the art and challenges for Laurillau, Feature extraction of the first difference of EMG time series for EMG
clinical use, J. Rehabil. Res. Dev. 48 (6) (2011) 643. pattern recognition, Comput. Methods Programs Biomed. 117 (2) (2014)
[12] B. Hudgins, P. Parker, R.N. Scott, A new strategy for multifunction myoelectric 247–256.
control, IEEE Trans. Biomed. Eng. 40 (1) (1993) 82–94. [39] A.H. Al-Timemy, R.N. Khushaba, G. Bugmann, J. Escudero, Improving the
[13] M.A. Oskoei, H. Hu, Support vector machine-based classification scheme for performance against force variation of EMG controlled multifunctional
myoelectric control applied to upper limb, IEEE Trans. Biomed. Eng. 55 (8) upper-limb prostheses for transradial amputees, IEEE Trans. Neural Syst.
(2008) 1956–1965. Rehabil. Eng. 24 (6) (2016) 650–661.
[14] A.D. Chan, K.B. Englehart, Continuous myoelectric control for powered [40] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015)
prostheses using hidden Markov models, IEEE Trans. Biomed. Eng. 52 (1) 436–444.
(2005) 121–124. [41] D.E. Rumelhart, G.E. Hinton, R.J. Williams, Learning representations by
[15] B. Karlik, M.O. Tokhi, M. Alci, A fuzzy clustering neural network architecture back-propagating errors, Nature 323 (6088) (1986) 533.
for multifunction upper-limb prosthesis, IEEE Trans. Biomed. Eng. 50 (11) [42] V. Nair, G.E. Hinton, Rectified linear units improve restricted Boltzmann
(2003) 1255–1261. machines, Proceedings of the 27th International Conference on Machine
[16] J. Yousefi, A. Hamilton-Wright, Characterizing EMG data using Learning (ICML-10) (2010) 807–814.
machine-learning tools, Comput. Biol. Med. 51 (2014) 1–13. [43] S. Ioffe, C. Szegedy, Batch normalization: accelerating deep network training
[17] G.R. Naik, D.K. Kumar, V.P. Singh, M. Palaniswami, Hand gestures for HCI using by reducing internal covariate shift, International Conference on Machine
ICA of EMG Proceedings of the HCSNet Workshop on Use of Vision in Learning (2015) 448–456.
Human-Computer Interaction, vol. 56, Australian Computer Society, Inc., [44] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT Press, 2016.
2006, pp. 67–72. [45] F. Chollet, keras, https://github.com/fchollet/keras, 2015.
[18] M.R. Ahsan, M.I. Ibrahimy, O.O. Khalifa, Electromyography (EMG) signal based [46] M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A.-G.M. Hager, S. Elsig, G.
hand gesture recognition using artificial neural network (ANN), in: 2011 4th Giatsidis, F. Bassetto, H. Müller, Electromyography data for non-invasive
International Conference on Mechatronics (ICOM), IEEE, 2011, pp. 1–6. naturally-controlled robotic hand prostheses, Sci. Data 1 (2014), 140053.