Received 14 March 2023, accepted 9 April 2023, date of publication 17 April 2023, date of current version 24 April 2023.

Digital Object Identifier 10.1109/ACCESS.2023.3267674

Artificial Intelligence for sEMG-Based Muscular

Movement Recognition for Hand Prosthesis
NAFE MUHTASIM HYE 1 , UMMA HANY 1 , SUMIT CHAKRAVARTY 2, (Member, IEEE),
LUTFA AKTER3 , AND IMTIAZ AHMED 4 , (Member, IEEE)
1 Department of Electrical and Electronic Engineering, Ahsanullah University of Science and Technology, Dhaka 1208, Bangladesh
2 Department of Electrical and Computer Engineering, Kennesaw State University, Marietta, GA 30060, USA
3 Department of Electrical and Electronic Engineering, Bangladesh University of Engineering and Technology, Dhaka 1205, Bangladesh
4 Department of Electrical Engineering and Computer Science, Howard University, Washington, DC 20059, USA

Corresponding author: Umma Hany ([email protected])

This work was supported by the Start-Up Research Grant of the Department of Electrical Engineering and Computer Science, Howard
University, Washington, DC, USA.

ABSTRACT The muscular activities gathered by real-time myoelectric interfaces of surface electromyogra-
phy (sEMG) can be used to develop myoelectric prosthetic hands for physically disabled people. However,
the acquired myoelectric signals must be accurately classified in real time to properly control the operation
of the external devices. In this study, we propose methods for detecting and classifying muscular activi-
ties using sEMG signals. These methods include outlier removal, data manipulation, data preprocessing,
dimensionality reduction, and classification. We use the Ninapro database 1 (DB1) containing sEMG signals
from 27 intact subjects while performing 53 hand movements repeatedly. We apply the Principle Component
Analysis (PCA), Independent Component Analysis (ICA), and t-distributed Stochastic Neighbor Embedding
(t-SNE) feature extraction methods for dimensionality reduction. Five machine learning (ML) algorithms and
deep learning artificial neural networks (ANN) are applied for the classification of muscular movements. It is
observed that for the recognition of 53 muscular movements of 27 subjects with preprocessed raw data, ANN
obtains the highest accuracy of 93.92% for inter-subject and 97.73% for intra-subject movement recognition.
Among the ML algorithms, K-Nearest Neighbors (KNN) performs the best with both t-SNE features and the
preprocessed raw data in least computational time. With the preprocessed raw data, KNN obtains 93.174%
and 97.458% for inter-subject and intra-subject movement classification, respectively while with the t-SNE
features, KNN obtains 89.844% accuracy for inter-subject and 95.04% accuracy for intra-subject in reduced
computational time.

INDEX TERMS Gesture recognition, computational and artificial intelligence, biomedical signal
processing.

I. INTRODUCTION wearable depending on the requirements of the disable peo-

The myoelectric interfaces of the surface electromyogra- ple. To ensure the quality and proper operation of the external
phy gather muscular activity information for different move- devices, the muscular activities for different movements are
ments. The sEMG signals of different muscular activities can required to be classified accurately in real-time to control the
be used to control the operation of external prosthetic devices operation of the external devices.
that are used to make the life of physically disable people easy There are many works in the literature on muscular move-
and comfortable. The external devices may be wearable or not ment recognition based on the EMG signals [1]. In [2], the
authors consider feature extraction using the mean absolute
The associate editor coordinating the review of this manuscript and value (MAV), the variance (VAR), the waveform length
approving it for publication was Md. Kafiul Islam . (WL), sEMG histogram (HIST), cepstral coefficients (CC),

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

38850 For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ VOLUME 11, 2023
N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

short-time Fourier transform, marginal discrete wavelet trans- using transfer learning. The authors in [12] propose Trans-
form (mDWT) and classification using linear discriminant fer learning based multi-scale kernel convolutional neural
analysis (LDA), k-nearest neighbors (KNN), support vector network (TL-MKCNN) and achieve 97.22% within-session,
machine (SVM), multi-layer perceptron (MLP), SVM with 74.48% cross-subject, and 90.30% cross-day accuracy that
a radial basis function (RBF) kernel on the Ninapro DB1 are higher than the MKCNN model by 4.31%, 11.58%, and
and achieve a classification accuracy of 76% by several of 5.51%, respectively. The authors in [13] 86.3% accuracy
the above-mentioned methods. In [3] and [4], the authors use using their proposed Few-shot Learning for Hand Gesture
Ninapro DB1 of 27 subjects and classify the movements using (FS-HGR) framework. The authors in [14] use tensor-based
RMS features and a convolutional neural network (CNN) multilinear singular value decomposition (MLSVD) for hand
classifier. The authors in [3] report inter-session gesture gesture recognition with multiple channels for training and
recognition accuracy of 94% for 6 movements while the single channel for recognition. The authors in [14] apply
authors in [4] report 70.5% intra-session gesture recogni- the proposed methods on Ninapro, CapgMyo (DB-a, DB-
tion accuracy for 53 movements. The authors in [5] propose b and DB-c), and CSL-HDEG databases for intra-session,
a system that depends on the extraction of multi-channel inter-session and inter-subject evaluations and obtained high-
EMG activation trajectories underlying hand movements, and est accuracy of 75.2%, 75.4%, 68.3% and 67.7%, respec-
classifying the extracted trajectories using a metric based tively for inter-subject classification. The authors in [15]
on multi-dimensional dynamic time warping (MD-DTW). develop a real-time system for prosthetic hands control
The authors in [5] apply their proposed system on Ninapro and use a database with EMG data from local volunteers
database 2 (DB2) for 40 different hand movements of 40 sub- and Ninapro 2 and 3 databases. They implement a MLP
jects and obtain a classification accuracy of 90%. However, classifier on a platform for rapid prototyping (Raspberry
the main limitation of the proposed work is that a classi- Pi 3 Model B+) and generate responses in real-time (11ms)
fication of the movement cannot be obtained until the end with an average accuracy of 96.30% for 11 hand and wrist
of the trajectory is detected. This restricts the usability of gestures/movements.
the proposed system to applications that can be controlled Though there have been extensive research works on
using discrete commands. The authors in [6] use the raw sEMG-based movement recognition, still there is scope
sEMG of Ninapro DB1 and obtain 75.45% intra-session accu- for improvement considering big database of a higher
racy in recognition of 52 movements of 27 subjects using number of users and movements. For real-time hand
Long short-term memory (LSTM) classifier. The authors prosthesis, the muscular movements must be accurately
in [7] apply transfer learning (TL) and deep learning algo- detected and classified in real-time with low computational
rithms to learn features from the big data collected from complexity and time.
multiple users. Three different deep learning networks with In this paper, we propose the methods of sEMG sig-
raw EMG, spectrograms and continuous wavelet transform nals based movement detection and classification for hand
(CWT) as input are trained and tested on three datasets - prosthesis using artificial intelligence techniques. To con-
two datasets from Evaluation database comprised of 19 and trol the operation of the external prosthetic devices prop-
17 intact subjects and one dataset of 10 participants from erly, the muscular activities due to different movements are
Ninapro database 5 (DB5). The maximum achieved offline required to be detected and classified accurately. Therefore,
accuracy on test data from Evaluation database is 98.31% the sEMG signals for different muscular activities are col-
for 7 gestures of 17 participants using CWT-based Con- lected, preprocessed, and classified using different machine
vNet and 68.98% using Ninapro database of 18 gestures of learning and deep learning methods. We collect the sEMG
10 participants using raw EMG-based ConvNet. The authors data from the publicly available Ninapro DB1 [16]. Ninapro
in [8] use high density EMG signal to decode the motion DB1 includes 10 repetitions of 53 movements (including
intention based on the transient EMG signals and achieve rest position) of 27 intact subjects. We apply dimensionality
decoding performance of 94.21% with CapgMyo and below reduction techniques on the preprocessed dataset to reduce
70% using Ninapro database 4 (DB4) for 8 finger movements the computational complexity and processing time of the
using SVM. In [9], the authors use RMS features and apply machine learning methods. We apply different Linear feature
temporal convolutional network (TCN) classifier for gesture extraction techniques as PCA, ICA and Non-linear feature
recognition and report 89.76% intra-session accuracy in clas- extraction technique as t-SNE to reduce the dimensional-
sifying 52 movements of 27 subjects using Ninapro DB1. ity of the dataset. Among those, t-SNE performs the best
Transfer learning based several approaches are proposed in in terms of accuracy. t-SNE is a non-linear dimensionality
[10], [11], [12], and [13] and applied on Ninapro database 2, reduction technique suitable for visualizing high-dimensional
3, 5 and 6, respectively. The reported accuracy in [10] is data in a low-dimensional space of two or three dimen-
49.76% for self-decoding and 52.52% for subject transfer sions. After applying the feature extraction, we apply five
using CNN. The reported average accuracy in [11] is 67.98% different machine learning methods and compare the move-
with Gaussian kernel function SVM classifier (GKF-SVM) ment classification accuracy. As we can observe that for
using Mean value of square root (MSR) features and 70.40% 53 movements, the KNN classifier shows the best accuracy

VOLUME 11, 2023 38851

 N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

FIGURE 1. System block diagram.

of 93.174% for inter-subject and 97.458% for intra-subject

classification with the preprocessed raw data in the least
processing time among the ML methods. By extracting the
3-dimensional t-SNE features, the KNN obtains 89.844%
inter-subject accuracy and 95.04% intra-subject accuracy
with reduced processing time and computational complexity.
We also apply deep learning ANN for classification which
shows the best accuracy of 93.92% for inter-subject and
97.73% for intra-subject movement recognition.

II. SYSTEM MODEL

We propose sEMG based movement detection and classifica-
tion using artificial intelligence to control the operation of the FIGURE 2. Placement of the sEMG electrodes; Position 1: Eight equally
spaced electrodes around the radio humeral joint of the forearm; Position
prosthetic hand. The system block diagram of the proposed 2: Two additional electrodes placed on finger extensor and flexor
system is shown in Figure 1. Here, the system uses the muscles; Position 3 and 4: Cyberglove sensors. [17].
sEMG signals collected for different muscular activities of
the hand using real-time myoelectric interfaces or electrodes.
The sEMG signals of both training and test phases are prepro- as shown in Fig. 2. DB1 includes labeled data of 10 repeti-
cessed and classified to detect the intended hand movements tions of 52 hand movements and 1 rest position of 27 intact
using different machine learning and deep learning methods. subjects. During the experiment, the subjects were asked to
The feature extraction method is applied before machine repeat the 52 movements of Exercise A, B, and C as shown
learning-based classification to reduce the dimensionality of in Fig. 3 with the right hand for 5 seconds followed by
the sEMG dataset. It also reduces the computational complex- 3 seconds of rest. The three exercises include the following
ity and processing time of the classifier. Deep learning-based movements.
classifiers are applied to classify the movements using pre-
1) Exercise A: 12 basic movements of the fingers (flexions
processed raw data. The predicted movements may control
and extensions),
the operation of the prosthetic hand.
2) Exercise B: 8 isometric and isotonic hand config-
urations and 9 basic movements of the wrist, total
III. DATA SOURCE AND PARTICIPANTS 17 movements, and
We use the Ninapro DB1 available in the official Ninapro 3) Exercise C: It is done with 23 grasping and functional
repository [16]. The database contains muscular activity data movements.
acquired using OttoBock sEMG electrodes. The datasets Several signal processing steps including synchronization,
were collected in [17] using 10 OttoBock MyoBock 13E200- relabelling, and filtering was performed before making the
50 electrodes (www.ottobock.com), providing an amplified, data available on the repositories [16], [17]. The sEMG sig-
bandpass-filtered, and Root-Mean-Square (RMS) rectified nals are sampled to a high sampling frequency of 2 kHz to
version of the raw sEMG signals. Among the 10 electrodes, synchronize the data streams and then relabeled [17]. The
eight are equally spaced around the radio humeral joint of Ninapro DB1 contains one matlab file with synchronized
the forearm at position 1, and two are placed on the finger variables of each exercise for each subject. A single file
extensor digitorum and finger flexor digitorum at position 2 includes the variables as shown in Table 1.

38852 VOLUME 11, 2023

N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

FIGURE 3. Rest position and 52 hand movements of Exercise A: 12 Basic movements of the fingers (flexions and
extensions), Exercise B: 17 Isometric, isotonic hand configurations and Basic movements of the wrist and Exercise
C: 23 Grasping and functional movements. [17].

VOLUME 11, 2023 38853

 N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

FIGURE 4. Implementation Methodologies.

TABLE 1. Variables in a single matlab file of Ninapro DB1 of each processing time. Therefore, we apply the outliers removal,
exercises for each subject.
data manipulation, preprocessing, and dimensionality reduc-
tion methods to reduce the size of the data. The implemen-
tation methodologies with the system flow are shown in
Figure 4 and explained in the following.

A. OUTLIERS DETECTION AND REMOVAL

The required variables for our proposed movement recogni-
tion system are the emg and the stimulus. The emg variable
includes the sEMG signals of 10 electrodes and the stimulus
variable includes the labels of 53 movements (including rest).
To keep the required variables and to remove the outliers,
we apply the following methods:
1) First, the variables except emg, stimulus and restimulus
are removed.
2) Then, to extract the required data, we apply outliers
detection and removal method. To detect the outliers,
we apply a matching algorithm to find the matches
between the stimulus and restimulus. If any value of
the stimulus matches with the restimulus values, then
IV. METHODOLOGIES the corresponding sEMG signals (of 10 electrodes)
We propose the methodologies to develop the sEMG-based and the stimulus values are kept unchanged. Else if
muscular movement recognition system for hand prostheses. the stimulus doesn’t match the restimulus, then the
The methodologies include outliers detection and removal, corresponding sEMG signals and the stimulus values
data manipulation, preprocessing, feature selection, feature are considered as outliers and dropped. In this way, the
extraction, and classification. The proposed methodologies unnecessary variables and outliers are removed and the
are developed in Python programming language using a vari- required values of the emg and stimulus are extracted
ety of popular data analysis libraries and tools, including for further processing.
pandas, scipy.io, loadmat, numpy, matplotlib, seaborn, and
the Keras, TensorFlow and Scikit-Learn tools. Since Ninapro B. DATA MANIPULATION
dataset belongs to the big data category, it is not practical to We apply the following three steps for data manipulation
apply the entire dataset for classification due to very high before the preprocessing of data.

38854 VOLUME 11, 2023

N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

C. DATA PREPROCESSING
The following preprocessing methods are applied to the train-
ing and test data.

1) STANDARD SCALING
After data manipulation, we apply standard scaling on the
sEMG signals of input variable to remove the mean and to
scale the data to unit variance. The standard score is calcu-
lated as
(x − µ)
z= , (1)
s
where, µ is the mean and s is the standard deviation of the
samples. Standard scaling is applied to the input variable of
both machine learning and deep learning models.

2) ONE HOT ENCODING

One hot encoding is applied to encode the movement labels
of the output variable stimulus prior applying to the deep
FIGURE 5. Outliers removal and data manipulation. learning model for classification.

D. DIMENSIONALITY REDUCTION
1) DATA SPLITTING To reduce the dimensionality of the dataset, feature selection
First, we split data into training and test phases where 90% and feature extraction methods are applied. Feature selection
of the data are used for training and 10% data are used for discards less important features of the data whereas feature
testing. All the variables of the dataset are not required for extraction creates new features which summarize the con-
the sEMG based movement recognition. tents of the original features. Feature extraction improves the
accuracy, reduces overfitting risk, speeds up in training and
2) RELABELING improves data visualization.
There are 13 movements of Exercise A, 17 movements of
Exercise B and 23 movements of Exercise C. The 13 distinct 1) FEATURE SELECTION
movements of Exercise A are labeled as 0 to 12, 17 move- To reduce the dimensionality of the dataset, we apply
ments of Exercise B as 0 to 17 and 23 movements of Exercise the Recursive Feature Elimination with Cross Validation
C as 0 to 23, respectively in the corresponding stimulus (RFECV) which is a wrapper-based method of feature selec-
variables. However, 53 movements of Exercises A, B, and tion. RFECV ranks the features among the 10 sEMG signals
C should have 53 distinct labels. Therefore, the movement as per importance and then selects the optimal features by
labels of Exercise A (1 to 12), Exercise B (1 to 17), Exer- cross validation [18].
cise C (1 to 23) are relabeled as 1 to 12, 13 to 29 and
30 to 52, respectively to get distinct movement labels for the 2) FEATURE EXTRACTION
53 movements. We apply and compare the following feature extraction tech-
niques to reduce the dimensionality of the dataset.
3) CONCATENATION • Principle Component Analysis (PCA): PCA is a com-
Finally, the sEMG data of all 53 movements (including rest mon linear feature extraction method used to reduce
position) of 3 exercises and the corresponding stimulus labels the dimensionality of data by finding the best combi-
are combined in a single data file by concatenation process. nation of input features and capturing maximum infor-
Thus, the concatenated training and test data files include all mation about the dataset. However, if the components
the sEMG signals of 53 distinct movements. are not selected with care, it may miss some information
The emg variable including 10 sEMG signals of each [19], [20].
movement is used as the input in both training and test • Independent Component Analysis (ICA): ICA is a linear
phase. The stimulus including the movement labels is used method which takes the mixed independent components
as the output in the training phase. In the test phase, the and correctly identify them removing the unnecessary
stimulus is used to evaluate the accuracy of the predicted noises. However, ICA algorithms are sensitive to mea-
movements. surement noise [21].
The system flow diagram of the outliers removal and data • t-distributed Stochastic Neighbor Embedding (t-SNE):
manipulation is shown in Fig. 5. t-SNE is a very effective non-linear dimensionality

VOLUME 11, 2023 38855

 N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

reduction algorithm. t-SNE tries to minimize the diver-

gence between the different probability distribution of
the input features in the original high dimensional space
and in the reduced low dimensional space. The dissim-
ilarity of two different distributions is measured using
Kullback-Leiber (KL) divergence and it is minimized
using gradient descent. The original dimensional space
is modelled using a Gaussian Distribution, while the
lower-dimensional space is modelled using a student’s
t-distribution. t-SNE handles non-linear data efficiently
and it preserves local and global structure of the data.
However, t-SNE is slow and computationally complex
as it has a quadratic time and space complexity in the
number of data points. It also requires a lot of calcula-
tions for computing the pairwise conditional probabil-
ities for each data point and to minimize the sum of
the difference of the probabilities in higher and lower
dimensions. It involves hyper-parameter tuning which
may cause unwanted results due to incorrect tuning
[19], [22].

E. CLASSIFICATION AND MOVEMENT RECOGNITION

In the training phase, the sEMG signals are trained to the cor-
responding stimulus labels of movements. In the test phase,
the sEMG signals are applied to the classifier to detect the
corresponding movements. In the machine learning model,
we compare the accuracy by applying both preprocessed
data and the features extracted. As the deep learning model
performs better with raw data, only the preprocessed data are
applied for classification.

1) MACHINE LEARNING METHODS

For machine learning based classification, we employ the
Decision Tree (DT), KNN, Random Forest (RF), Extra Trees
(ET), and Extreme Gradient Boosting (XGB) classifiers [23],
[24], [25], [26], [27]. To fine-tune the parameters, we apply
GridSearchCV with three levels of cross validation. The
default settings of the DT classifier, RF classifier, and ET
classifier, perform the best. For the XGB, we use the ‘‘gpu
predictor’’ to speed up the training process. For the KNN
classifier, we use the ‘‘ball tree’’ algorithm as the optimum
parameter. For KNN, we considered 2 nearest neighbors (K
= 2), manhattan distance as a measure of distance and the
wights measured by distance. However, there is space for
improvement by boosting the estimators for all tree-based
methods which could not be implemented due to a lack of
computing configurations.

2) PROPOSED DEEP LEARNING ARCHITECTURE

We apply Artificial Neural Networks for deep learning-based
classification. The architecture of the ANN is shown in Fig. 6.
FIGURE 6. Deep learning architecture.
We start with 10 emg signals as inputs, then add a dense layer
of 3000 neurons with a relu activation function, followed by
another dense layer of 1500 neurons with a relu activation with activation functions of 750, 375, and 48 correspondingly
function. Next, a 0.2 dropout layer is added. 3 dense layers are then added. A dense layer as 53 class with a SoftMax

38856 VOLUME 11, 2023

N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

activation function is added last. Then, with batch size set TABLE 2. Accuracy (intra-subject) after applying different improvement
steps of outliers removal and preprocessing.
to 9000 and 300 epochs, we compile the model using Adam
as the optimizer and categorical cross entropy as the loss
function.

V. PERFORMANCE EVALUATION
We apply the following performance metrics to evaluate the
performance of the classification models [28].

A. CONFUSION MATRIX
Confusion Matrix is a visualization of ground-truth labels
the loss tends to 0. Cross-entropy loss function is defined as
versus model predictions. Each cell in the confusion matrix
n
represents any of the following evaluation factors: X
LCE = − ti log(pi ), for n classes (6)
1) True Positive (TP) signifies how many positive class i=1
samples are predicted correctly.
where, n is the number of classes, ti is the true label and pi is
2) True Negative (TN) signifies how many negative class
the Softmax probability for the ith class [29].
samples are predicted correctly.
3) False Positive (FP) signifies how many negative class
VI. SIMULATION AND RESULTS
samples are predicted incorrectly.
The proposed movement detection and classification sys-
4) False Negative (FN) signifies how many positive class
tem is developed in Python programming language using
samples are predicted incorrectly.
a variety of popular data analysis tools and libraries. The
libraries include pandas, scipy.io, loadmat, numpy, mat-
B. PRECISION
plotlib, seaborn. The classifiers are developed using Keras,
Precision is the ability of a classifier not to label an instance TensorFlow, and Scikit-Learn tools. The computational envi-
positive that is actually negative. It is defined as ronment of simulation, training and testing is based on a
TP 64-bit Windows operating system with an x64-based Intel(R)
Precision = . (2) Core(TM) i7-5500U CPU with 2.40 GHz processor and
TP + FP
8.00 GB of installed RAM. In addition, we leverage the
C. RECALL NVIDIA GeForce 840M GPU for efficient computation.
Recall is the ability of a classifier to find all positive instances. To verify the accuracy of the proposed movement detection
For each class, it is defined as and classification, we simulate the proposed methods using
Ninapro DB1 which contains sEMG signals of 10 electrodes
TP of 27 intact subjects for 53 movements. To analyze the effect
Recall = . (3)
TP + FN of outliers removal, initially we apply the ML algorithms on
the data of subject 1 without applying the outliers removal.
D. F1 SCORE
As we can see that the accuracy is very low as the data
The F1 score is a weighted harmonic mean of precision and contains outliers. Next, we apply the outliers removal, data
recall such that the best score is 1.0 and the worst is 0.0. F1 manipulation and preprocessing on the data of subject 1. It is
score is expressed as found that the accuracy significantly improves after apply-
2 × Recall × Precision ing the outliers removal. The accuracy further improves by
F1 score = . (4) applying the standard scaling as preprocessing. The findings
(Recall + Precision)
are shown in Table 2.
E. ACCURACY Then, we apply RFECV for feature selection on the data of
Classification accuracy is defined as the number of correct subject 1 only. We configure the Random Forest Classifier as
predictions divided by the total number of predictions, mul- estimator, step = 1, and cross validation = 5. Figure 7 shows
tiplied by 100. The classification accuracy is calculated as the five cross-validation results for an increasing number of
selected features, demonstrating how the accuracy improves
(TP + TN ) with each emg value. The best number of features is then
Accuracy = × 100. (5)
(TP + FP + FN + TN ) discovered to be 10, indicating that all the EMG signals of
the 10 electrodes contains important features.
F. CROSS ENTROPY LOSS FUNCTION Next, to verify the accuracy of movement recognition,
In classification model, a loss function is calculated based on we apply the machine learning algorithms on all the EMG
the probability of how far the predicted class is from the actual data of 27 subjects after outliers removal, data manipula-
expected value. Cross-entropy loss is used to optimize the tion and preprocessing steps without applying any feature
model by adjusting the weights during training to minimize extraction. Table 3 summarizes the performance in terms of

VOLUME 11, 2023 38857

 N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

FIGURE 9. Loss curve of ANN-based deep neural network.

FIGURE 7. Optimal feature selection using cross validation score.

TABLE 3. Accuracy scores (intra/inter-subject) of different machine

learning algorithms using raw sEMG of 27 subjects.

FIGURE 10. Accuracy curve of ANN-based deep neural network.

TABLE 4. Performance scores of deep learning ANN classifier with all

27 subjects.

FIGURE 8. Accuracy comparison of different machine learning algorithms

using raw sEMG of 27 subjects (Inter-subject).

number of epochs. Table 4 summarizes the results of both

Accuracy, Precision, Recall, F1-score for both intra-subject intra-subject and inter-subject movement classification per-
and inter-subject movement classification. Figure 8 com- formance of the deep learning ANN. We can see the ANN
pares the accuracy of inter-subject movement detection and can achieve highest intra-subject accuracy of 97.73% and
classification using different machine learning algorithms. inter-subject accuracy of 93.92% with preprocessed raw data.
We observe that the KNN performs the best for intra-subject Next, we apply three different feature extraction techniques
movement classification with 97.458% accuracy and the ET PCA, ICA and t-SNE to sequentially reduce the dimen-
classifier performs the best for inter-subject with 93.376% sionality of the input dataset emg which contains 10 EMG
accuracy using preprocessed sEMG data. features (E1 to E10) of different movements. Figure 11
Next, we apply the ANN based deep learning classification shows the impact of dimensionality reduction on the accu-
method on the preprocessed sEMG data of all 27 subjects racy of different classification methods. As we can see in
to detect the 53 movements. Figures 9 and 10 show how Fig. 11 that the performance of PCA and ICA significantly
the loss decreases and the accuracy improves with increasing drop if the dimensionality of the features are reduced below

38858 VOLUME 11, 2023

N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

FIGURE 11. Accuracy using different dimensional features; (a) PCA, (b) ICA, (c) t-SNE.

TABLE 5. Classification results (intra or inter-subject) with 8 dimensional TABLE 7. Classification results (intra or inter-subject) with 3 dimensional
features extracted using PCA. features extracted using t-SNE.

TABLE 6. Classification results (intra or inter-subject) with 9 dimensional TABLE 8. Training time comparison (in seconds) of different machine
features extracted using ICA. learning algorithms.

8 and 7, respectively whereas the performance of t-SNE and ICA to extract 9 dimensional features. We also apply
drops if the dimension is below 2. Therefore, to extract the non-linear dimensionality reduction method t-SNE to extract
data features of all subjects, we apply linear dimensionality 3-dimensional features of all subjects. Tables 5, 6 and 7 show
reduction methods PCA to extract 8 dimensional features the intra-subject and inter-subject classification results using

VOLUME 11, 2023 38859

 N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

FIGURE 12. Accuracy comparison (inter-subject) using different dimensionality reduction

techniques and by using raw sEMG.

TABLE 9. Accuracy and Training time comparison for different Exercises with t-SNE features and raw data.

the three feature extraction methods PCA, ICA and t-SNE, It is also observed that the training time of the ET classi-
respectively. As we can see that the best accuracy using fier is significantly reduced by using t-SNE features which
lowest dimensional features (3 dimension) can be achieved can speed up the processing time and can make the system
using t-SNE feature extraction method. Here we can see that computationally more intelligent. We can also see that KNN
using the 3 dimensional t-SNE features, highest intra-subject performs the best in terms of time accuracy and computa-
accuracy of 95.272% and inter-subject accuracy of 90.429% tional intelligence. KNN requires only 2.492 seconds training
can be achieved using ET classifier. Figure 12 compares time to classify preprocessed raw data of 53 movements
the inter-subject accuracy of the machine learning methods of all 27 subjects with inter-subject accuracy of 93.174%
using the preprocessed raw data and by applying different and 0.079 seconds to classify the extracted t-SNE features
dimensionality reduction techniques. It can be seen in Fig. 12 with inter-subject accuracy of 89.844%. Thus, in terms of
that the best accuracy can be obtained using the highest accuracy and time efficiency, KNN performs the best among
dimensional features of the preprocessed raw data whereas all the machine learning algorithms for classification of the
the best accuracy with lowest dimensional features can be 53 movements.
obtained using the t-SNE features. Thus, it may be concluded From the above classification results and the time com-
from the results of Tables 5, 6 and 7 and Fig. 12 that the KNN, parison in Table 8, it is observed that among the machine
RF and ET classifiers perform with significantly high accu- learning algorithms KNN performs the best with lowest com-
racy for both intra and inter-subject movement classification putational time. It is also observed in Table 8 that using
with preprocessed raw data as well as with 3-dimensional t- t-SNE feature extraction, the computational time of KNN
SNE features reducing the computational complexity. and ET is reduced further. Thus, to compare the performance
Table 8 compares the training time of the extracted low of machine learning based KNN, ET and the deep learning
dimensional features and the preprocessed raw data. It is ANN, the accuracy of KNN, ET and ANN are further ana-
observed that the training time of DT and RF classifier lyzed for different exercises using both t-SNE features and
increases with the extracted feature which is not expected. preprocessed raw sEMG data. The results are included in

38860 VOLUME 11, 2023

N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

TABLE 10. Accuracy and Training time for increasing number of movements with raw sEMG data.

TABLE 11. Accuracy comparison of the proposed methods to the state of the art works.

Table 9 which shows that for different exercises, the accuracy number of movements. Table 10 shows the accuracy and
is different and the accuracy above 94% and 97% is obtained the training time for increasing number of movements using
for Exercise 3 using t-SNE features and raw data, respec- KNN, ET and ANN. As we can see that the KNN performs the
tively. As the accuracy for Exercise 3 is comparatively higher, best with 97.266% accuracy for 23 movements of Exercise C
we further analyze the accuracy of Exercise 3 for increasing with the lowest training time of 1.335s using raw data. It is

VOLUME 11, 2023 38861

 N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

also observed that ET requires comparatively more time to results should also be verified using datasets transformed into
obtain 97.164% accuracy and ANN requires extensively high the time-frequency domain.
processing time of 22565.775s to get the accuracy of 97.899%
for 23 movements. Table 11 compares the accuracy of the REFERENCES
proposed methods to the state of the art works in the liter- [1] W. Li, P. Shi, and H. Yu, ‘‘Gesture recognition using surface electromyog-
ature. As we can see that our proposed methods can obtain raphy and deep learning for prostheses hand: State-of-the-art, challenges,
and future,’’ Frontiers Neurosci., vol. 15, Jan. 2021, Art. no. 621885.
significantly high accuracy for a large number of inter-subject [2] M. Atzori, A. Gijsberts, I. Kuzborskij, S. Elsig, A.-G. M. Hager, O. Deriaz,
and intra-subject movements classification compared to other C. Castellini, H. Müller, and B. Caputo, ‘‘Characterization of a benchmark
works in the literature. database for myoelectric movement classification,’’ IEEE Trans. Neural
Syst. Rehabil. Eng., vol. 23, no. 1, pp. 73–83, Jan. 2015.
[3] K.-H. Park and S.-W. Lee, ‘‘Movement intention decoding based on deep
learning for multiuser myoelectric interfaces,’’ in Proc. 4th Int. Winter
VII. CONCLUSION Conf. Brain–Comput. Interface (BCI), Feb. 2016, pp. 1–2.
[4] P. Tsinganos, B. Cornelis, J. Cornelis, B. Jansen, and A. Skodras, ‘‘Deep
In this paper, we have proposed sEMG-based movement learning in EMG-based gesture recognition,’’ in Proc. PhyCS, 2018,
detection and classification for hand prostheses using the data pp. 107–114.
of Ninapro DB1 of 53 hand movements of 27 intact subjects. [5] M. AbdelMaseeh, T. W. Chen, and D. W. Stashuk, ‘‘Extraction and classi-
First, we have proposed the outliers detection and removal to fication of multichannel electromyographic activation trajectories for hand
movement recognition,’’ IEEE Trans. Neural Syst. Rehabil. Eng., vol. 24,
remove the unnecessary data points. Then, we have applied no. 6, pp. 662–673, Jun. 2016.
data manipulation methods like data splitting, relabeling, [6] Y. Hu, Y. Wong, W. Wei, Y. Du, M. Kankanhalli, and W. Geng, ‘‘A novel
and concatenation to rearrange the data for easy processing. attention-based hybrid CNN-RNN architecture for sEMG-based gesture
recognition,’’ PLoS ONE, vol. 13, no. 10, 2018, Art. no. e0206049.
Next, we have applied standard scaling on the input variables [7] U. Côté-Allard, C. L. Fall, A. Drouin, A. Campeau-Lecours, C. Gosselin,
and one hot encoding (for ANN) on the output variables. K. Glette, F. Laviolette, and B. Gosselin, ‘‘Deep learning for electromyo-
Then, to reduce the computational complexity and time, graphic hand gesture signal classification using transfer learning,’’ IEEE
Trans. Neural Syst. Rehabil. Eng., vol. 27, no. 4, pp. 760–771, Jan. 2019.
dimensionality reduction methods PCA, ICA and t-SNE have [8] Y. Li, Q. Zhang, N. Zeng, J. Chen, and Q. Zhang, ‘‘Discrete hand motion
been applied to extract 8 dimensional, 9 dimensional and intention decoding based on transient myoelectric signals,’’ IEEE Access,
3 dimensional features, respectively. Then, we have applied vol. 7, pp. 81630–81639, 2019.
[9] P. Tsinganos, B. Cornelis, J. Cornelis, B. Jansen, and A. Skodras,
five machine learning algorithms and the deep learning ANN ‘‘Improved gesture recognition based on sEMG signals and TCN,’’ in Proc.
on the preprocessed raw data and the extracted features for IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), May 2019,
movement classification. Among the ML algorithms, KNN pp. 1169–1173.
[10] K.-T. Kim, C. Guan, and S.-W. Lee, ‘‘A subject-transfer framework based
performs the best in terms of accuracy, dimensionality reduc- on single-trial EMG analysis using convolutional neural networks,’’ IEEE
tion and least processing time. Using the extracted t-SNE Trans. Neural Syst. Rehabil. Eng., vol. 28, no. 1, pp. 94–103, Oct. 2020.
features, KNN shows 89.844% inter-subject and 95.04% [11] Y. Li, W. Zhang, Q. Zhang, and N. Zheng, ‘‘Transfer learning-based muscle
activity decoding scheme by low-frequency sEMG for wearable low-cost
intra-subject accuracy while using the preprocessed sEMG
application,’’ IEEE Access, vol. 9, pp. 22804–22815, 2021.
data, KNN shows 93.174% inter-subject and 97.458% intra- [12] Y. Zou and L. Cheng, ‘‘A transfer learning model for gesture recognition
subject accuracy in classifying 53 movements. Using the based on the deep features extracted by CNN,’’ IEEE Trans. Artif. Intell.,
preprocessed raw sEMG of 53 movements, ANN can obtain vol. 2, no. 5, pp. 447–458, Oct. 2021.
[13] E. Rahimian, S. Zabihi, A. Asif, D. Farina, S. F. Atashzar, and
the highest accuracy of 93.92% for inter-subject and 97.73% A. Mohammadi, ‘‘FS-HGR: Few-shot learning for hand gesture recog-
for intra-subject classification. It has also been observed nition via electromyography,’’ IEEE Trans. Neural Syst. Rehabil. Eng.,
that using KNN on preprocessed raw data, it is possible to vol. 29, pp. 1004–1015, 2021.
[14] S. Padhy, ‘‘A tensor-based approach using multilinear SVD for hand
obtain 98.749% and 97.266% inter-subject accuracy for 7 and gesture recognition from sEMG signals,’’ IEEE Sensors J., vol. 21, no. 5,
23 movements, respectively while using ANN, it is possible pp. 6634–6642, Dec. 2021.
to obtain 99.367% and 97.899% inter-subject accuracy for [15] J. O. de Oliveira de Souza, M. D. Bloedow, F. C. Rubo,
R. M. de Figueiredo, G. Pessin, and S. J. Rigo, ‘‘Investigation of different
7 and 23 movements, respectively. However, ANN requires approaches to real-time control of prosthetic hands with electromyography
an extensively high computational time of about 43394.59s to signals,’’ IEEE Sensors J., vol. 21, no. 18, pp. 20674–20684, Sep. 2021.
train the preprocessed raw data of 53 movements while KNN [16] M. Atzori. (2014). Ninapro Repository. [Online]. Available: http://ninapro.
hevs.ch
requires only 2.492s training time. Again, KNN can perform [17] M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A.-G. M. Hager, S. Elsig,
with high accuracy in a reduced computational time of 0.079s G. Giatsidis, F. Bassetto, and H. Müller, ‘‘Electromyography data for non-
using the t-SNE feature extraction. Thus, it may be concluded invasive naturally-controlled robotic hand prostheses,’’ Sci. Data, vol. 1,
no. 1, pp. 1–13, 2014.
that among the proposed methods, KNN performs the best
[18] (Jan. 22, 2021). Feature Selection Techniques in Machine
with high accuracy in the least computational time. There- Learning. Accessed: Dec. 2, 2022. [Online]. Available: https://www.
fore, the proposed methods recommend choosing machine geeksforgeeks.org/feature-selection-techniques-in-machinelearning/
learning-based algorithms for real-time applications such as [19] K. Vipin. (Dec. 30, 2020). Dimensionality Reduction—PCA vs
LDA vs T-SNE. Accessed: Dec. 2, 2022. [Online]. Available:
controlling prosthetic hands whereas deep learning-based https://medium.com/analytics-vidhya/dimensionality-reduction-pca-
algorithms can be chosen for offline applications due to their vs-lda-vs-t-sne-681636bc686
high accuracy. To improve the quality and impact of the work [20] S. Rajvi. (Sep. 13, 2020). Dimensionality Reduction using Principal
Component Analysis (PCA). Accessed: Dec. 2, 2022. [Online]. Available:
in the future, it is recommended to collect adequate datasets https://medium.com/analytics-vidhya/dimensionality-reduction-using-
for different types of disabilities of the amputee subjects. The principal-component-analysis-pca-41e364615766

38862 VOLUME 11, 2023

N. M. Hye et al.: Artificial Intelligence for sEMG-Based Muscular Movement Recognition for Hand Prosthesis

[21] K. P. Ashwini. Diving Deeper Into Dimension Reduction With Independent SUMIT CHAKRAVARTY (Member, IEEE)
Components Analysis (ICA). Accessed: Dec. 2, 2022. [Online]. Available: received the bachelor’s, master’s, and Ph.D.
https://blog.paperspace.com/dimension-reduction-with-independent- degrees in electrical and computer engineering.
components-analysis/ He is currently an associate professor of electri-
[22] L. Van der Maaten and G. Hinton, ‘‘Visualizing data using t-SNE,’’ cal and computer engineering. He also served as
J. Mach. Learn. Res., vol. 9, no. 11, pp. 1–12, 2008. a Postdoctoral Researcher for UPENN. He has
[23] L. Rokach and O. Maimon, ‘‘Decision trees,’’ in Data Mining and used his expertise in signal processing machine
Knowledge Discovery Handbook. Cham, Switzerland: Springer, 2005, learning and communications in his role as a prin-
pp. 165–192.
cipal scientist in industry and a faculty member
[24] G. Guo, H. Wang, D. Bell, Y. Bi, and K. Greer, ‘‘KNN model-based
in academia. His academic experience includes
approach in classification,’’ in Proc. OTM Confederated Int. Confer-
ences Move Meaningful Internet Syst. Cham, Switzerland: Springer, 2003, working at NASA-Goddard, the University of Maryland, and NYIT. His
pp. 986–996. industry experience in engineering and research includes working in various
[25] L. Breiman, ‘‘Random forests,’’ Mach. Learn., vol. 45, no. 1, pp. 5–32, roles such as an instrumentation engineer, a research intern, Siemens CAD
2001. and Apex Eclipse Communications, a Principal Scientist of SGT Inc., the
[26] P. Geurts, D. Ernst, and L. Wehenkel, ‘‘Extremely randomized trees,’’ Honeywell Research of the Automatic Control Solutions-Advanced Tech-
Mach. Learn., vol. 63, no. 1, pp. 3–42, Mar. 2006. nology Laboratories. He has published multiple journals and peer-reviewed
[27] T. Chen and C. Guestrin, ‘‘XGBoost: A scalable tree boosting system,’’ conference papers in venues like Pattern Recognition, IEEE SENSORS, IEEE
in Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2016, TRANSACTIONS ON GREEN COMMUNICATIONS AND NETWORKING, and Journal of
pp. 785–794. Medical Imaging. He has served as a Guest Editor for the Special Issues
[28] B. Aayush. Performance Metrics in Machine Learning [Complete on Signal Processing in Wireless Communications and Towards Reliable
Guide]. Accessed: Sep. 21, 2022. [Online]. Available: https://neptune. and Scalable Smart Cities: Internet of Things Meets Big Data and AI of
ai/blog/performance-metrics-in-machine-learning-complete-guide Electronics journal.
[29] W. Rose. Cross-Entropy Loss and Its Applications in Deep Learning.
Accessed Sep. 2, 2022. [Online]. Available: https://neptune.ai/blog/cross-
entropy-loss-and-its-applications-in-deep-learning

NAFE MUHTASIM HYE received the B.Sc.

degree in electrical and electronic engineering
(EEE) from the Ahsanullah University of Science
and Technology (AUST).
He is currently a Graduate Research Assistant LUTFA AKTER received the Ph.D. degree in elec-
with the Department of EEE, AUST. His research trical engineering from Kansas State University,
interests include artificial intelligence, machine USA, in 2010. She is currently a Professor with the
learning, deep learning, biomedical signals, and Department of Electrical and Electronic Engineer-
image processing. He was one of the founding ing (EEE), Bangladesh University of Engineer-
members of the university’s Mars Rover team, ing and Technology (BUET). Her current research
which finished 26th out of more than 65 competitors globally and fourth interests include biomedical signal processing,
in Asia, the European Rover Challenge, and Poland, supported by NASA cell-free massive MIMO, wireless sensor deploy-
and the Mars Society. He also completed an internship in corporate banking ment, the IoT, and the localization of wireless
and banking information technology from one of the most reputed banks in video capsule endoscopes.
Bangladesh.

UMMA HANY received the B.Sc. degree in

electrical and electronic engineering from the
Ahsanullah University of Science and Technology,
in 2004, the M.Sc. degree in electrical and elec-
tronic engineering from the Bangladesh University
of Engineering and Technology (BUET), in 2009,
and the Ph.D. degree in electrical and electronic IMTIAZ AHMED (Member, IEEE) received the
engineering (EEE) from the Bangladesh Univer- Ph.D. degree in electrical and computer engineer-
sity of Engineering and Technology, in 2017. ing from The University of British Columbia,
After her graduation, she served with IICT, Vancouver, BC, Canada. He is currently an Assis-
BUET, as a Hardware Engineer, and Huawei Technologies Bangladesh tant Professor with the Department of Electri-
Ltd., as a Core Network Engineer. She was a Ph.D. Visiting Scholar cal Engineering and Computer Science, Howard
with the Canadian Commonwealth Scholarship Program, University of University, Washington, DC, USA. He works in
Saskatchewan, Canada. She was with the Department of EEE, University of the areas of wireless communications, signal pro-
Asia Pacific, and the Bangladesh Army International University of Science cessing, and computer networks. After finishing
and Technology, as a Lecture, an Assistant Professor, and an Associate his Ph.D. degree, he was with Intel Corporation,
Professor. Currently, she is an Associate Professor with the Department San Diego, CA, USA, as a Wireless Systems Engineer, and Marshall Univer-
of EEE, Ahsanullah University of Science and Technology. She has sev- sity, Huntington, WV, USA, as an Assistant Professor. His research interests
eral publications in peer-reviewed journals and conferences. Her research include artificial intelligence-aided physical layer designs, integration of
interests include artificial intelligence, wireless sensor networks, wireless aerial and terrestrial communication networks, communication with THz
communications, localization, pattern recognition, biomedical signals, and bands, and intelligent reflecting surfaces.
image processing.

VOLUME 11, 2023 38863