Sensing and Bio-Sensing Research 29 (2020) 100353

Contents lists available at ScienceDirect

Sensing and Bio-Sensing Research

journal homepage: www.elsevier.com/locate/sbsr

Percentage estimation of muscular activity of the forearm by means of EMG T

signals based on the gesture recognized using CNN
Javier O. Pinzón-Arenas , Robinson Jiménez-Moreno, Astrid Rubiano
⁎

Faculty of Engineering, Universidad Militar Nueva Granada, Bogotá D.C. 110111, Colombia

ARTICLE INFO ABSTRACT

Keywords: Within muscle activity based on surface electromyographic (EMG) signals, the percentage estimate of muscle
Muscle activity activation, which is the level of intensity with which muscles have been activated, has not been exploited. This
Electromyographic signal work presents a system that allows the estimation of different movements, for this case five hand gestures, where
Convolutional neural network the EMG signals are obtained by means of a Myo armband. For the estimation, discrimination of sensors to be
Moving RMS
used is carried out according to the intensity of activity presented and the gesture made, recognized by a multi-
Butterworth filter
Hand gesture recognition
channel convolutional neural network. Likewise, the EMG envelope is obtained to perform the estimation,
evaluating two methods, root-mean-square (RMS) and signal filtering using a Butterworth filter. According to the
results, the system managed to recognize all the gestures made in real-time, as well as the effective percentage
estimation of muscle activity, with a minimum of 50% for stable force and 22% for incremental force, that is
when the signal has been reduced so much that it moves to a neutral gesture. In general, the implemented system
can be used in any type of gesture recognition application to improve its recognition, or even in rehabilitation
exercises that require showing the progress in muscle activity in different movements.

1. Introduction movements made by the hand [11–14], where they can recognize from
4 to 40 gestures. The gestures are applied to perform control of robotic
Nowadays, biomedical signals, or signals acquired from the human agents [13], recognition of the alphabet by means of sign language
body, have increased their popularity and use in the development of [15], among others.
applications in various fields of study, such as medicine and even en- In general, the applicability of EMG signals is mainly based on
gineering. One of the most commonly used signal types is electro- muscle activity, that is when the muscle is exercising a work and the
myographic (EMG) signals, which are generated by the electrical ac- intensity of activation that is generated. This topic has several points of
tivity caused by the contraction or relaxation of the fibers of a muscle focus of use, such as onset detection, i.e. when the activation of the
[1]. One of the great advantages of this type of signal is that it can be muscle begins when performing a force [16–18], which is described as
acquired superficially, i.e. it does not require an invasive device. This one of the challenges in the analysis of electromyographic signals, due
advantage has allowed for the development of different portable de- to the difficulty of estimating the real start of muscle activity, taking
vices specialized in specific parts of the body, for example, for reading into account the noise that may exist in the signal. The study of muscle
the surface EMG signals of the lower limbs that can detect muscle fa- activity has also been applied in health care, e.g. to analyze body
tigue [2] or to monitor muscle activity [3]. Likewise, there are bracelets postures and how different muscles interact in these, in order to see
that can be used in different parts of the body, mainly in the upper how it affects the health of people [19]. Another example allows per-
limbs, among which a well-known device is the Myo Armband [4], forming movement analysis to make comparisons between healthy
developed by Thalmic Labs™. This last device has been used in different subjects and subjects with some disease or muscular ailment [20], de-
areas, such as in health care [5], and in human-machine interaction pending on how the muscle is activated. Muscle activity has also served
interfaces [6,7]. to interpret the movements and postures of the arm for robotic arm
The versatility of the use of non-invasive EMG signal acquisition control [21], robotic hand control [22], and for robot-human colla-
devices has allowed the implementation of numerous developments and boration [23] by means of pattern recognition techniques. This has
investigations focused on the analysis of gait [8] and balance [9,10], caused a rising in the application of EMG signals in different technol-
and to a large extent, with regard to the recognition of gestures or ogies, mainly in prosthesis control [24]. In the first approaches, EMG

⁎
Corresponding author.
E-mail address: [email protected] (J.O. Pinzón-Arenas).

https://doi.org/10.1016/j.sbsr.2020.100353
Received 5 February 2020; Received in revised form 15 May 2020; Accepted 18 May 2020
2214-1804/ © 2020 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license
(http://creativecommons.org/licenses/BY/4.0/).
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

signals were combined with classical methods such as artificial neural making in order to establish the reference signal against which the
networks or fuzzy systems in order to control a multifunctional pros- comparison of muscle activity will be made. According to the literature,
thesis [25]. In recent studies, neural networks are still widely used for there are various techniques used to recognize hand gestures by means
prosthesis control [26,27], but more robust methods such as support of EMG signals, especially using artificial intelligence models, such as
vector machines are being used [28], thanks to its capability to dis- the artificial neural network [38], support vector machines [39], hybrid
criminate features of the signals [29]. Moreover, EMG signals have also frameworks using hidden Markov models [40], and k-nearest neighbors
used to control the force exerted by a prosthesis in a grasping task by [12,13]. However, with the development of more robust models, such
means of predictive control [30]. as those covered in a branch of artificial intelligence, called Deep
Another application based on muscle activation could be the per- Learning, where their specialty is the recognition of patterns [41], these
centage estimate of this, that is, how much the muscle has actually been have begun to be used for classification systems given their high per-
activated. This could be used to improve the analysis of movements or formance. An example can be seen in [42], where a deep neural net-
muscle fatigue, depending on the performance of the muscle. work is used to perform silent speech recognition by means of the
Additionally, having a support system via percentage estimation of movement of muscle at the moment of speaking, allowing the classifi-
muscle activity that helps to increase the performance of gesture re- cation of the phonemes performed. One of the best-known techniques of
cognition models would be useful. It would help to estimate in a better deep learning is the convolutional neural network (CNN) [43,44],
way the gesture done, by not only taking into account the features which has also been applied in this field to classify up to 50 types of
obtained from the signal but the percentage estimation of muscle ac- movement, using 22 EMG signals [45]. Additionally, CNN has the
tivity, giving an additional parameter that offers to a model a way to ability to have multiple channels as input, which allows having the
know when a movement o gesture has really changed and not being possibility of using the 8 sensors to perform the recognition of gestures.
confused when activity is very low and the gesture is still performed; For this reason, it was proposed to use a multi-channel CNN, which
this is due to the fact that pattern recognition systems that use EMG classifies the gesture according to a group of surface EMG signals de-
signals may degrade their classification accuracy because EMG activity rived from the muscle. This phase is carried out in such a way that the
starts decreasing when repetitive movements are done by a user [31], network is user-independent, that is, the gesture is recognized in-
affecting mainly to technologies that are focused on prostheses control. dependently of who wears the EMG signal acquisition device. In [46],
Furthermore, all over the globe, there are muscular problems derived the architecture used can be evidenced, where in this work the im-
from daily work activity, post-operatory situations or illnesses that re- plementation process is emphasized and explained in more detail.
quire physiological therapies of muscle rehabilitation [32–34], where a The second phase refers to the algorithm used to estimate muscular
system that allows measuring the degree of percentage estimation of activity by comparing the selected EMG signals against a reference of
muscle activity can give the patient autonomy to perform a specific the same user, making a comparison between two methods of filtering
therapy or exercise without supervision, thus having a means of feed- the signal in order to obtain the envelope, to then calculate the per-
back on how the activity was performed and which in turn serves as a centage estimation. This phase is user-dependent because the estimate
support for analysis and assessment of an expert in the area [35,36]. depends on a user's reference signal.
However, in the literature, muscle activity estimation classes found are The two phases are discussed in detail below.
based on the movement or exercise performed and how the activity
should be [37], but not on the level of activation that is occurring. 2.1. Hand gesture recognition system
Taking this into account, the objective of this work is to make a
contribution to the state-of-the-art focused on the development of a To determine the muscular activation, a group of gestures to be
system that allows estimating in a percentage way the muscular activity classified is established, delimiting the great variety of combinations
of a movement, which is validated with 5 hand gestures, performing the that allows the forearm to 5 movements called “Right”, “Left”, “Up”,
comparison of activations of the muscles to implement the estimation “Down” and “Stop”, the latter would correspond to a fist. The afore-
based on a reference signal from the user. The estimated percentage of mentioned gestures are shown in Fig. 1, where each must be done
muscle activity is made with respect to all the muscles activated in the keeping the index finger stretched and the rest of the hand closed,
movement, based on the EMG signals acquired from the forearm by additionally, the arm should be bent at approximately 90° in the frontal
means of a bracelet-type device, Myo armband. direction, to avoid additional stresses on the muscles by varying posi-
To improve the estimation, depending on the gesture or movement tions of how the movement can be performed, since this can cause
made with the hand, sensors that are not active or that do not have a confusion in the recognition system [47], especially by how the person
high degree of activation are eliminated. For this, the gesture is re- can perform the movement with respect to the position of the arm. To
cognized by means of a deep learning technique using 8 surface EMG avoid recognition of erroneous movements when muscle activity has
signals acquired by the Myo, which are processed to extract their fea- been reduced, an additional gesture, called “None”, is added, that refers
tures that will be entered into the neural network, and based on its to the arm rested or in a neutral position, which is established when the
result, the sensors that are arranged on the muscles that have activation user is not exerting a force or muscle tension in the arm.
are selected.
This work is organized as follows: Section 2 presents a detailed 2.1.1. Database
description of the system, which is divided into two phases, one con- For the implementation of the convolutional neural network, it is
cerning the recognition of the gesture, and another referring to the necessary to establish a database with defined parameters, for example,
percentage estimate of muscle activity. In Section 3, the tests and re- the size of the data to be processed by the network. The purpose is to
sults obtained from the coupling of the two phases in a single system use EMG signals for both gesture recognition and muscle activity, using
and the discussion of these are presented. Finally, Section 4 gives the a non-invasive device. The use of Myo Armband, shown in Fig. 2, is
conclusions reached from work. proposed. This consists of 8 individual segments where bio-sensors are
located in their internal part and are responsible for acquiring the
2. Materials and methods electromyographic signals generated by the electrical activity of the
forearm muscles in a non-invasive superficial way [4]. These segments
The development of the present work was executed in 2 phases that are joined by an elastic part, giving greater comfort to the user who uses
allowed the estimation of muscle activity in the forearm automatically. it, since it adjusts to the thickness of the arm. Each bio-sensor consists of
The first phase refers to the method of recognition of hand gestures to 3 main parts, a central one, which is responsible for analyzing and
be evaluated, since it is required to know what movement the user is detecting the activity of the muscles, and two laterals, that act as filters

2
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 1. Gestures to be recognized: (a) Left; (b) Right; (c) Up; (d) Down; (e) Stop; (f) None.

of 1 s and a rest of the same time; this acquisition procedure is repeated

every day for 2 days. To enter the EMG signals acquired by the bracelet,
an acquisition toolbox [50] is used, which allows not only to visualize
the signals of each of the sensors and save them according to a set
sampling time but also gives the option to use a predetermined function
to perform the processing of the acquired signal.
Commonly, a signal or set of unprocessed signals lack immediate
characteristic patterns, i.e. it tends to have a seemingly random beha-
vior, which means that for a CNN that is specialized in learning patterns
or high-level features, it is complex to learn to discriminate these sig-
nals, making the computational cost rise and to require a longer
learning time, or simply stagnate trying to find patterns. For this reason,
in various research, it has been chosen to use different methods of ex-
Fig. 2. Myo Armband device [4]. tracting characteristics from EMG signals in order to reduce the com-
putational cost of applications that use machine learning or deep
learning techniques [51]. For instance, in [42], the authors use the
on the same muscle analyzed, giving a property of immunity to noise logarithmic power of the signal to improve the learning of the neural
and improving the signal acquired by the central sensor. These sensors network, instead of using a raw signal.
are called active EMG electrodes [48]. Additionally, the bracelet con- On the other hand, to improve the learning of CNN, in [52], a map
tains an inertial measurement unit (IMU), which has 9 axes distributed called surface EMG image is used, which consists of a matrix containing
equally between a gyroscope, an accelerometer, and a magnetometer. the normalized EMG signals, such that each data represents a “pixel” of
For the acquisition of data of the gestures, the user must wear the the image, obtaining 89.3% accuracy in the recognition of 8 hand
Myo in the manner shown in Fig. 3.a, placing the sensor 4 on an in- gestures. For this reason, it is proposed to perform a signal processing
visible line between the ring finger and index finger, and approximately that allows extracting the characteristics of the EMG signal. It is decided
on the extensor digitorum muscle. The Myo device is located in the to use the Welch method [53] in order to obtain a map of features of the
upper part of the forearm, capturing the signals of the muscles that can signal, which is an image of energies of the signal displayed by means of
be seen in Fig. 3.b, where sensors 2 to 5 are located in the posterior the power spectral density (PSD), where each energy coefficient would
muscle group (the upper extensor muscles), and 6 to 8 and 1 overlap the represent, for the network, a pixel of said image. For this, the signal xi
anterior muscle group (the upper flexor muscles) [49]. (m) is divided into K overlapped segments that have a size N, as shown
The data of the movements are acquired from 20 healthy users, who in Eq. (1), where D is the overlap of the window on values already
perform each gesture 12 times in a row. Each repetition has a duration analyzed, taking into account that D = N means that there is no

3
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 3. (a) Position where the Myo armband should be worn; (b) Location of the armband over the forearm muscles [49].

overlapping. Following this, it is proceeded to perform the modified of only 5 layers of convolution and 2 dense layers (or fully connected)
periodogram proposed by Welch, using the Fast Fourier Transform of with a disconnection factor of 0.5. Each layer is made up of filters of
the modified signal together with a windowing function w(m), to be size 3 × 3 so that the network learns the behavior of the maps in
divided by the square of the windowing function, obtained by Eq. (3). temporary terms combined with the coefficients, except for the last
Finally, to calculate the PDS (P(f)), an average of the obtained peri- layer, which has filters of size 1 × 1, in order to reduce the computa-
odograms is performed, as shown in Eq. (2). tional cost and improve the learning of the features of the output of the
previous layer. In addition, each layer contains the addition of zeros to
x i (m) = x i (m + iD), where m = 0, …, N 1 i = 0, …, K 1 (1)
the edges, in order to improve the learning of the information of the
K 1 N 1 2 borders of the map and resize the input to avoid odd-sized value out-
1 1 j2 fm puts, to avoid information loss when performing the subsampling. In
P (f ) = w (m ) x i (m ) e
K i=0
NU m=0 (2) the last convolution layer, padding is not added, due to the size of the
filter, since it would learn complete sections of values equal to 0. The
N 1
1 value P, which refers to the number of rows or columns, is ordered as
U= (w (m)) 2
N m=0 (3) [top, bottom, left, right], and in the case that only one value is indicated,
it means that this value of rows or columns is added in all the edges. In
The use of the windowing function w(m) mainly helps to mitigate each convolution and fully connected layer, the last number that is
the discontinuities presented in the signal [54]. This process, based on displayed refers to the number of filters used in these layers. On the
the sampling frequency of the Myo that is 200 Hz, is performed with a other hand, the network does not use the ReLU activation layer, since
sliding window of analysis of 50 ms long, that is, in such a way that 10 the values of the coefficients of the feature map are negative and the
samples (K) are analyzed, from which its PSD is obtained, forming a function of this layer is to filter the negative values, causing all of them
feature map of 20 frames. In the same way, 101 coefficients are ob- to be eliminated.
tained per analysis window, since the length of the Fourier transform is The architecture is trained with the database set up for this purpose,
200 points (N), obtaining the energy of the frequencies between F = 0, at the same time that the validation is carried out to verify that, during
…, 100, where F ∈ ℕ. In total, a matrix of feature maps of size 101x20x8 training, it does not enter into overfitting, i.e. that the network learns
(# coefficients x # frames x # sensors) is obtained, since this process is patterns in a general way, what is verified by means of the accuracy of
also performed with each of the sensors. the validation set, which should increase as the network trains, since in
Each of the movements made by users is saved and analyzed, gen- the case that it begins to descend steadily or remains at a low value of
erating a database of 2880 multi-channel feature maps. Having this, it is accuracy, it would mean that the network is only memorizing the
decided to segment the database in 3 sets equally distributed in a training data and is not able to recognize external data. The training is
random way, i.e. from the whole database, 33.3% of the samples ob- parameterized at 200 epochs, using a learning rate of 10−4, so that it is
tained are randomly taken to build the training dataset, maintaining the reduced by 50% every 150 epochs, in order that in the last 50 training
same amount of data in each category; the other 33.3% is taken for the epochs, the network makes a fine adjustment of their weights. With
validation dataset, and the rest for the test dataset, obtaining sets these parameters, the network presents a behavior as shown in Fig. 6,
composed of 960 different feature maps each. These are used for where the network reaches an accuracy of approximately 90% before
training and validation of the CNN, and subsequent tests. An example of the 10th epoch, with a slight drop from the 30th to the 90th epoch,
the acquired maps is shown in Fig. 4, where the signals from each of the where it achieves a 95% accuracy. Finally, it achieves 99% accuracy of
sensors are plotted, and in front of these, their respective feature maps. training. As for the validation, it remains in constant growth throughout
the training, reaching 98.4% in the last epoch.
2.1.2. Convolutional neural network implementation In order to verify that the network has achieved a high level of
In order to recognize each of the established gestures, the archi- recognition, as shown by the validation, tests with the respective da-
tecture of a CNN is built. One of the main characteristics proposed taset are carried out, and subsequently, to obtain the confusion matrix
about the neural network is that it must be shallow, in order to obtain a of the classification results of all the maps entered. In this way, the
very small processing time in recognition of the gesture, i.e. below results shown in Fig. 7 are achieved. As can be seen, the network
10 ms, so that it does not interfere with the acquisition of the signal manages to classify the data with 99% accuracy, generating only 10
from the Myo, and in turn, the recognition is made in real-time. With erroneous classifications over 960 samples. The misclassifications were
this, the architecture shown in Fig. 5 is proposed. The network consists concentrated in the “Right” gesture, with 6 errors of recognition, being

4
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 4. Example of the movement “Right”, with its EMG signal read by each sensor and its respective feature map.

confused mainly with the “Up” gesture, possibly by the signal generated tests done with the network were performed in a GPU NVIDIA GTX
in the sensors when the movement does not become complete. The 1070.
gesture “None” got two erroneous recognitions, since, this gesture apart
from containing signals without generation of force made by the user,
2.2. Muscle activity system
some people made slight movements of the fingers, making the muscles,
although without much intensity, were activated, where possibly one of
This system is developed in order to estimate, once the gesture is
these movements was similar to the “Left” and “Right” gestures. Fur-
recognized, the muscular activity that is being generated at the moment
thermore, the processing time of the neural network was 4 ms, a value
of doing it. For its implementation, it is necessary to obtain a form or
much lower than estimated, making it suitable for the application. The
curve of the signal that can be compared against another, since it is not

5
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 5. Proposed shallow CNN architecture.

2.2.1. Envelope of the EMG signal

To obtain the linear envelope of the signal, it is necessary to carry
out the rectification of the original signal (Fig. 8.a), where first, the
mean of the signal is set to 0, then the absolute value of the signal is
obtained, in order to obtain the signal in positive values. This is done by
means of Eq. (4), where x is the original signal, x is the mean of the
signal, and xr is the final signal adjusted, obtaining a signal as shown in
Fig. 8.b.
x r = |x x| (4)

After carrying out the rectification of the signal, the envelope of the
signal is calculated. For this stage, multiple ways to obtain the EMG
envelope have been implemented, such as moving average [28], spline
interpolation, integrated EMG, and a wide variation of low pass filters,
among others [56]. Between them, there are three of the most used
methods [57] that are average rectified value (AVR), the root mean
square (RMS) [58] and low pass filtering by means of a Butterworth
Fig. 6. Training behavior of the shallow CNN. filter [59]. However, as mentioned in [60], the difference between AVR
and RMS is minimal for calculating the envelope but with a slight ad-
vantage for the RMS. For that reason, only RMS and the Butterworth
filer are used in this work in order to get the envelope of the EMG
signal, also because of their versatility and fast implementation and
computation. These two methods are shown below.
For the method using the RMS [58], Eq. (5) is used, where, for a
vector of n data, it is obtained the quadratic mean of this vector, to later
obtain its square root. Variable xi represents each of the values of the
array and n, the length of the array. This operation is done through the
entire rectified signal by means of an analysis sliding window of size n,
which can be displaced with an overlap, in such a way that the window
uses values previously analyzed, to make the envelope smoother.
n
1
xRMS = xi2
n i=1 (5)

For the filtering method, a Butterworth filter [59] is used as a low

pass filter, which is applied forward and backward in the signal. The
filter transfer function is described in Eq. (6), where ω is the angular
frequency and ωc is the angular cutoff frequency (2π * cutoff frequency),
and H(jω) is the transfer function at the frequency ω. To perform the
filtering, the order of the filter is set, which is defined by n and is ap-
plied to the rectified signal.
1
|H (j )| =
( )
2n
1+ c (6)

In order to select the most appropriate method for estimating

Fig. 7. Confusion matrix obtained from the test dataset. muscle activity, a comparison of the envelope generated by each of the
methods is done. For the filtering method by means of Butterworth,
which will act as a high pass filter, two cutoff frequencies are set, one at
advisable to make a comparison of the raw signals due to their random 10 Hz and the other at 5 Hz, since it is recommended for surface EMG
nature, and therefore, its impossibility to produce a signal similar to the signals [61], with a 4th order filter; while for the method using RMS, a
original one. Therefore, processing of the raw EMG signal must be sliding window with a size of 20 ms (4 samples) and 40 ms (8 samples)
performed in such a way that its variability and signal peaks are re- is established, the latter with an overlap of 35 ms in its advance. In
duced by means of smoothing using a linear envelope since this is de- Fig. 9.a, the rectified signal and the applied methods are displayed,
scribed as muscle activation [55]. where it is observed how the envelope acts on an intense signal and on a

6
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 8. (a) Raw EMG signal; (b) Rectified EMG signal.

much smoother signal with some activation peaks. To visualize in a a late termination at the moment of generating the envelope of the
better way how the envelope acts, a close view is taken over a time- peak, as it is possible to see in Fig. 9.b in the RMS curves, mainly in the
lapse (Fig. 9.b). The smoothest envelope is filtered at 5 Hz, however, second example, which makes non-existent information in the behavior
this makes many activation features disappear. On the other hand, the of the signal to be added. For this reason, the filtering method using a
RMS method with a 20 ms window maintains a large part of the signal 4th order Butterworth filter, with a cutoff frequency of 10 Hz, is se-
peaks and is the one with the lowest smoothing index, that is, the slopes lected.
between points are still very pronounced. The two envelopes that best
describe the signal, with a higher smoothing, are the Butterworth filter
with 10 Hz cutoff frequency and the RMS with a 40 ms window and 2.2.2. Selection of the sensors based on sensor activation
35 ms overlap. Nevertheless, when the rectified EMG signal has high To estimate the muscle activity in each gesture, it is required to
dispersed activations, the RMS envelope tends to have an early start or verify which sensors act at the moment of performing it, therefore, EMG
signal strength tests are performed to identify them. This is done

Fig. 9. (a) EMG envelope of 2 signals; (b) Zoomed EMG envelope.

7
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

because some muscles are not activated, which means that the mea- depending on the gesture recognized, the most relevant signals are
surement, taking into account all the muscles, might not be completely selected, as described in Section 2.2.2. for both, the reference signal and
correct, since the strength of the signals would be diminished by a poor the read EMG signal. Then, the selected signals are processed in order to
muscular activity. obtain their envelope, in order to calculate the percentage estimation of
To carry out these tests, the toolbox used is modified in the section the muscular activity of the whole movement or gesture.
of the signal display, where the curve of the envelope of the signal is Verification tests of operation and functioning of the coupled al-
now plotted instead of the raw signal. Additionally, the information gorithm are done by means of the graphic interface. The sample time of
visualization of the IMU is eliminated, and the percentage estimate of 1-s length is established, so that the signal processed by the PSD is
muscle activity and the recognized gesture are added. consistent with the CNN input size, setting this time also for the esti-
To select the sensors in each gesture, an activation threshold is es- mation of muscle activity. Additionally, apart from showing the muscle
tablished, which is the activation average greater than 0.1 of a signal of activity estimation using the Butterworth filtering, the analysis done
length 2 s, since a lower value represents a non-activation of the muscle, with the RMS with a 40 ms window is added, to see the percentage
according to the values acquired with the category “None”. If the ac- differences between the two best methods evaluated.
tivation of different users exceeds this threshold, the sensor is selected. The tests are carried out with 4 healthy users, i.e. they do not have
Taking into account this, the chosen sensors are listed below: any type of muscular problem or disease in their upper limbs. Each one
performs each gesture as best as possible so that it is saved as the re-
• “Right” gesture: Sensors 3, 4 and 5. An example of the test can be ference. Then, the user again performs each gesture, trying to copy the
seen in Fig. A.1. way they did it in the beginning, keeping it for a time of ± 20 s. Five
• “Left” gesture: Sensors 1, 7 and 8. An example of the test can be seen repetitions are made per user per gesture.
in Fig. A.2. In Fig. 11, two examples of “Right” gesture repetition done by two
• “Up” gesture: Sensors 4 and 5. An example of the test can be seen in different users are shown. In Fig. 11.a, from the second 165, the user
Fig. A.3. keeps the gesture almost constant for more than 15 s, but their muscle
• “Down” gesture: Sensors 1, 2, 3, 7 and 8. An example of the test can activity, although it has a slight reduction, does not exceed 70%, since
be seen in Fig. A.4. it is their fourth repetition, and although they maintain it, fatigue
• “Stop” gesture: Sensors 1, 2, 3, 4, 5, 6, 7 and 8. An example of the makes the muscle activity not very strong. On the other hand, in
test can be seen in Fig. A.5. Fig. 11.b, the user who performs the gesture makes the muscle activity
stay close to their reference, even having a greater activity. It can also
2.2.3. Percentage estimate of muscle activity be observed that, when the user does the movement (second 132), in
Due to the measurement of multiple sensors, it is necessary to de- the first second, the level of muscular activity is greater than when it is
termine in an arithmetic way a correlation of the information acquired already stabilized, having a type of electric peak, especially because the
in each of the sensors within a value that allows to be contrasted against movement was executed from rest instantaneously. In the same way, it
another from the same source, i.e. a reference of the muscular activity is possible to observe in the two users, how the sensors that are dis-
parameterized as “ideal” that is when the gesture is executed in a firm criminated for the gesture (number 1, 2, 6, 7 and 8) have a low or
and complete way, in order to calculate the percentage of a person's almost no activity when making the gesture, which indicates that in
muscle activity. For that reason, the comparison is based on the en- those positions, the muscles are not working. With respect to the esti-
velope of the signals, in such a way that the mean of the envelope of mation, in these two examples, both the percentage estimate using the
each sensor is acquired, depending on the gesture made. Then, the Butterworth filter and the one found using RMS, had very close results,
mean percentage of activation of each sensor is obtained, which is the with gaps of less than 0.5 percentage points.
relation between the mean of the envelope in the current measurement For the “Left” gesture, Fig. 12 is presented, where the signals ac-
and that of the reference signal. With this, the percentage of activity of quired by a single user at different times are shown. When the user is
each required sensor is obtained, to finally perform a weighting of the rested, he is able to maintain the intensity of activity for a long time,
percentages, obtaining a percentage estimate of muscle activity of the also having the overshoot of execution at the beginning of the move-
movement made. This procedure is defined by Eq. (7), where x e r
is ment, managing to maintain muscle activity over 100% (Fig. 12.a).
an array that contains the mean of the signal envelope of the sensors However, when other repetitions have already been made, the user does
defined for the gesture, x er r
is an array composed of the envelope not achieve a constancy in the execution of the gesture, having peaks of
mean of the user reference signal of said sensors, r refers to the number activation to return to the gesture position, finally decreasing to a lower
of sensors used for the gesture, and p is the resulting percentage esti- activation under 65%, over 13 s, due to muscular fatigue (Fig. 12.b).
mate of muscle activity. It should be highlighted that the size of x e and The estimation calculated by the two methods for this gesture has a
x er , and the value of r depends on the number of sensors used for the percentage gap of between 1.5% and 3%, which, although it is greater
gesture recognized or that is being performed by the user. than that of the “Right” gesture, is not considerable.
To perform the gestures “Up” and “Down”, there is not a wide

p=
r
i=1 ( x ei
x eri ) displacement in the trajectory of the gesture from its initial position,
r (7) compared with the previous two, hence users were able to maintain
almost constant signals during the different tests performed. For ex-
ample, in the “Down” gesture, estimates tended to be higher than 80%,
3. Results and discussions thanks to the ease of movement (Fig. 13.a), while in the “Up” gesture,
the estimation of muscle activity tended to decrease rapidly from the
With the two systems implemented, they are coupled into one. Its first repetition, because this movement is the most complex to perform,
operation is described in the flowchart shown in Fig. 10, where in the since it tends to generate greater fatigue and discomfort if the gesture is
first place, the EMG signal is read by the Myo armband, to then pass it held firm, having mostly estimates below 60%, although this was al-
through a processing stage, where by means of the PSD algorithm, a ready maintained throughout the entire test, as shown in the envelopes
feature map is obtained for each of the 8 signals. All the feature maps of Fig. 13.b, where the selected sensors (4 and 5) for this gesture did not
are sent to the CNN, which classify the gesture. If the gesture re- have a great variation, even, the signals of the sensors 6 and 7 show that
cognized is labeled as “None” the process starts again, but if not, it is possibly the user was not doing the gesture in a correct way in that
passed to the muscle activity system, which based on the gesture re- repetition. With respect to the methods used for the estimation, in these
cognized, a reference signal from the user is selected. In the same way, two gestures, the percentage gap was less than 0.5%.

8
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 10. Flowchart of the system.

As for the “Stop” gesture, in this, the variation of the muscular ac- almost instantly in a stable level of activation when the force is started,
tivity of the subjects can be appreciated in a better way. In this gesture, but others begin to increase constantly although it is the same gesture,
all sensors are used to estimate muscle activity, so it is possible to see as seen in Supplementary Video S2 and S4 (“Down”). On the other
when some sensors maintain the signal reading without variation, while hand, in other gestures, all signals selected for each gesture increase
others have increases or decreases, e.g. in Fig. 14.a, sensors 3 and 8 gradually, as for “Stop” (Supplementary Video S1), “Right”
have a fall between the seconds 310 and 315, while the others maintain (Supplementary Video S3), and “Up” (Supplementary Video S5) ges-
intensity with small variations or about the same average. Likewise, it is tures.
possible to see the fall of the envelope signal of the EMG through time With the results obtained, it is evident that the use of RMS shows in
due to fatigue, as shown in Fig. 14.b, where the user performs a pro- all cases an estimate with a lower value, due in part to the fact that by
longed repetition for 40 s, having a strong activity at the beginning, continuing to closely follow the curve, it tends to generate peaks close
which was read over 100%, but this was decreasing until reaching a to the original signal, which causes the signals with greater downwards
reading of 69.8%. For this gesture, the percentage gap of the estimation variation to influence the generation of the envelope to a greater extent,
between the two methods was less than 1%. as it happens with the signals of “Left”, while using the Butterworth
A special feature about most of the tests carried out is that the filter, it helps to avoid these peaks, due to its higher rate of smoothing
calculated minimum percentage estimate should be higher than 50%, of the signal. However, the two methods of obtaining the envelope can
since when it dropped from this value, the gesture that was being done be used, as long as it does not require high precision in muscle activity.
changed to “None”, since the level of confidence of the CNN decreases For instance, these can be used to support the recognition of robotic
when recognizing the feature map of the signals due to the low intensity commands using EMG signals, to indicate an agent to do any action
of these, which makes the network recognize them as “None”. Only in when to recognize the gesture and if the activity level is above a
one case, the calculated muscular activity managed to fall below 50%, threshold. Another case of application of the total system can be fo-
as shown in Fig. 15. This was mainly due to the fact that the EMG signal cused for rehabilitation exercises, where the user makes certain
of the “Up” gesture, which was the one being done, has a low intensity, movements, in such a way that the system recognizes that the required
so that CNN is able to recognize it despite having a low muscle activity, exercise is actually being done and, based on a reference of when the
without changing to the category “None”, causing it to reach a muscle user was in good health, estimate the degree of activation of the mus-
activity estimate of 40.93%, before it passed to the other category. cles and their progress in rehabilitation, without requiring an invasive
In order to see how the coupled system behaves, a test for each device, such as in the proposed exercises for rehabilitation of the carpal
gesture was performed, where it was asked to a healthy user to perform tunnel syndrome [62].
the movement, increasing the force exerted until reaching a high level If it is intended to use the system implemented by adding an un-
of activity, as the reference signal acquired from the user. As can be known or new gesture, e.g. a firm extended palm gesture, there should
seen in the videos presented in the Supplementary data (Supplementary be needed to re-train the network using the method of transfer learning
Video S1–S5), when the user starts with a small force, the system is able [63]. It should be done since the network already learned features of
to recognize the gesture, even with a small percentage of the muscle the PSD maps so, it will be only necessary to teach the network the new
activity, compared to that obtained when the gesture is performed gesture, taking into account that all the other gestures must be within
immediately and maintained with the same force, as happened with the the training, and the new gesture should have a similar number of
test done for the 4 users. Here, for “Stop” (Supplementary Video S1), samples with different forces exerted, in order to avoid the network to
“Left” (Supplementary Video S2) and “Right” (Supplementary Video keep learning the features older gestures instead of the new one and
S3) gestures, the network was able to correctly recognize the movement that the network stop recognizing the gesture if the force is not enough.
done with an estimated activity lower than 30%, although in some Furthermore, for the calculation of the envelope of the new gesture, it
gestures with very small activity tended to recognize other gesture. should follow the analysis performed, where the sensors that are acti-
Additionally, it is possible to observe how some sensors are activated vated must be discriminated from the non-active ones, to maintain the

9
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 11. Tests performed for the “Right” gesture. (a) User 1, after four repetitions. (b) User 4, in its first repetition.

10
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 12. Tests performed for the “Left” gesture. (a) User 2, in its first repetition. (b) User 2, after the third repetition.

11
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 13. Test performed for (a) “Down” gesture and (b) “Up” gesture.

12
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 14. Tests performed for “Stop” gesture by the User 3 (a) in the second and (b) fifth repetition.

13
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. 15. Lowest muscle activity obtained in the test performed.

precision of the percentage estimation of the muscle activity. The ad- identified which sensors were going to be used to perform the percen-
vantage of the calculation and comparison of the envelopes is that it is tage estimation of the muscular activity, since, depending on the ges-
insensitive to the number of sensors used, as long as they are correctly ture, not all sensors were required.
chosen.
In the case a user wants to change all the gestures, such as the ones 4. Conclusions
presented in the rehabilitation exercises in [62], or use a different type
of feature map instead of PSD, it can be followed two recommendations: In this paper, a new system of percentage estimation of muscle ac-
the first one is to use a shallow pre-trained model with learned features tivity was developed, which discriminates the sensors to be used in the
based on the selected feature map –since it may change the number of calculation of the estimate, based on the gesture that is being made and
channels used as network input–, and perform transfer learning. The its level of activation.
second one is to build their own model, either using a CNN or other The subsystem that uses deep learning, which focused on using a
classification technique, in order to recognize the gestures, as it was multi-channel convolutional neural network that, from the 8 signals
done here. Additionally, the sensors' discrimination based on the ges- acquired by the Myo and processed using PSD, managed to identify
tures must be done, taking into account an established threshold. The with a high percentage of accuracy each of the 5 gestures proposed plus
muscle activity system process keeps the same. With these re- a state of inactivity, with an average of 99%. For the subsystem of
commendations, the system will work correctly. percentage estimation of muscle activity, the envelope of the EMG
It should be kept in mind that the estimation of muscle activity signal was used, for which a comparison was made between two
using surface sensors does not estimate the position of the hand or methods, showing that by means of the Butterworth filter, the gener-
degree of the position of the gesture, because it was found that, when ated envelope presents a better smoothing of the raw signal, main-
the user did not make force or tension in the movement, the signals of taining the characteristics of the original curves. The results of the
the sensors dropped to an intensity similar to that of the category experimental tests showed the capacity of the coupled system to es-
“None” or without gesture, as it is possible to see in the peaks of the tablish the percentage of muscular activity exercised by a user, mana-
signals, where these were generated once the hand reached the final ging to estimate from a minimum of 50% of muscular activity when the
position of the gesture, or even in the “Stop” gesture, which despite gesture is performed instantly, i.e. exerting a fast and stable force
keeping the hand closed or in a fist position, the signal decreased de- throughout the gesture, before the gesture was recognized as “None”.
pending on the force exerted by the user. On the other hand, when the user does a force that gradually increases
On the other hand, CNN satisfactorily recognized all the gestures while performing the gesture, the system is able to recognize the
made by the users, allowing the system to be assertive when it was movement that is being performed with a percentage estimation of

14
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

approximately 22%. The developed system is adapted to implementa- Granada, grant number IMP-ING-2935.
tions that do not require a high degree of precision in muscle activity,
since it is developed with superficial non-invasive devices that limit the Authorship
uptake of the intensity of the signals, having the advantage of being
easily used by any person. All authors attest that they meet the current ICMJE criteria for
The system has three great features that are seen as advantages. The Authorship.
network has the capacity to use 8 channels or signals to improve its
recognition capacity without needing a high degree of depth, so it can
Declaration of Competing Interest
be retrained to identify any type of gesture depending on the applica-
tion, or even the system allows to replace it to use an own model; the
estimation of muscle activity is subject to the type of gesture that is The authors declare that they have no known competing financial
performed, since it is calculated depending on the number of sensors interests or personal relationships that could have appeared to influ-
used, avoiding the use of irrelevant signals, improving the estimation; ence the work reported in this paper.
the system can be used in any application that uses EMG signals,
without the need to use invasive devices. Acknowledgments
Supplementary data to this article can be found online at https://
doi.org/10.1016/j.sbsr.2020.100353. The authors are grateful to the Universidad Militar Nueva Granada,
which, through its Vice-rectory for research, finances the present pro-
Funding ject with code IMP-ING-2935 (being in force 2019–2020) and titled
“Prototipo robótico flexible para asistencia alimentaria”, from which
This research was supported by the Universidad Militar Nueva the present work is derived.

Appendix A. EMG sensor activations

Fig. A.1. EMG Sensor activation of the “Right” gesture.

15
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. A.2. EMG Sensor activation of the “Left” gesture.

Fig. A.3. EMG Sensor activation of the “Up” gesture.

16
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

Fig. A.4. EMG Sensor activation of the “Down” gesture.

Fig. A.5. EMG Sensor activation of the “Stop” gesture.

17
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

References [24] J. Ribeiro, F. Mota, T. Cavalcante, I. Nogueira, V. Gondim, V. Albuquerque,

A. Alexandria, Analysis of man-machine interfaces in upper-limb prosthesis: a re-
view, Robotics. 8 (1) (2019) 16, https://doi.org/10.3390/s18072366.
[1] D. Farina, D.F. Stegeman, R. Merletti, Biophysics of the generation of EMG signals, [25] F.H. Chan, Y.S. Yang, F.K. Lam, Y.T. Zhang, P.A. Parker, Fuzzy EMG classification
in: R. Merletti, D. Farina (Eds.), Surface Electromyography: Physiology, for prosthesis control, IEEE Trans. Rehabil. Eng. 8 (3) (2000) 305–311, https://doi.
Engineering, and Applications, Wiley-IEEE Press, Hoboken, 2016, pp. 1–24, , org/10.1109/86.867872.
https://doi.org/10.1002/9781119082934.ch02. [26] A.A. Adewuyi, L.J. Hargrove, T.A. Kuiken, Evaluating EMG feature and classifier
[2] S. De la Peña, A. Polo, C. Robles-Algarín, Implementation of a portable electro- selection for application to partial-hand prosthesis control, Front. Neurorobot. 10
myographic prototype for the detection of muscle fatigue, Electronics. 8 (6) (2019) (2016) 15, https://doi.org/10.3389/fnbot.2016.00015.
619, https://doi.org/10.3390/electronics8060619. [27] C. Calderon-Cordova, C. Ramírez, V. Barros, P.A. Quezada-Sarmiento, L. Barba-
[3] T. Isezaki, H. Kadone, A. Niijima, R. Aoki, T. Watanabe, T. Kimura, et al., Sock-type Guamán, EMG signal patterns recognition based on feedforward artificial neural
wearable sensor for estimating lower leg muscle activity using distal EMG signals, network applied to robotic prosthesis myoelectric control, Proceedings of the 2016
Sensors. 19 (8) (2019) 1954, https://doi.org/10.3390/s19081954. Future Technologies Conference (FTC), IEEE, 2016, December, pp. 868–875, ,
[4] North, Myo Gesture Control Armband tech specs, https://support.getmyo.com/hc/ https://doi.org/10.1109/FTC.2016.7821705.
en-us/articles/202648103-Myo-Gesture-Control-Armband-tech-specs, (2019) , [28] E. Noce, A.D. Bellingegni, A.L. Ciancio, R. Sacchetti, A. Davalli, E. Guglielmelli,
Accessed date: 15 May 2020. et al., EMG and ENG-envelope pattern recognition for prosthetic hand control, J.
[5] M. Sathiyanarayanan, S. Rajan, MYO Armband for physiotherapy healthcare: A case Neurosci. Methods 311 (2019) 38–46, https://doi.org/10.1016/j.jneumeth.2018.
study using gesture recognition application, Proceedings of the 2016 8th 10.004.
International Conference on Communication Systems and Networks (COMSNETS), [29] A.D. Bellingegni, E. Gruppioni, G. Colazzo, A. Davalli, R. Sacchetti, E. Guglielmelli,
IEEE, 2016, January, pp. 1–6, , https://doi.org/10.1109/COMSNETS.2016. L. Zollo, NLR, MLP, SVM, and LDA: a comparative analysis on EMG data from
7439933. people with trans-radial amputation, J. Neuroeng. Rehabil. 14 (1) (2017) 82,
[6] I.J. Ding, R. Lin, Z.Y. Lin, Service robot system with integration of wearable Myo https://doi.org/10.1186/s12984-017-0290-6.
armband for specialized hand gesture human–computer interfaces for people with [30] S. Dosen, M. Markovic, K. Somer, B. Graimann, D. Farina, EMG biofeedback for
disabilities with mobility problems, Comput. Electr. Eng. 69 (2018) 815–827, online predictive control of grasping force in a myoelectric prosthesis, J. Neuroeng.
https://doi.org/10.1016/j.compeleceng.2018.02.041. Rehabil. 12 (1) (2015) 55, https://doi.org/10.1186/s12984-015-0047-z.
[7] S. Masson, F. Fortuna, F. Moura, D. Soriano, Integrating Myo armband for the [31] J. He, D. Zhang, N. Jiang, X. Sheng, D. Farina, X. Zhu, User adaptation in long-term,
control of myoelectric upper limb prosthesis, Proceedings of the XXV Congresso open-loop myoelectric training: implications for EMG pattern recognition in pros-
Brasileiro de Engenharia Biomédica, Brasil, 2016, October. thesis control, J. Neural Eng. 12 (4) (2015) 046005, https://doi.org/10.1088/1741-
[8] N. Kunju, N. Kumar, D. Pankaj, A. Dhawan, A. Kumar, EMG signal analysis for 2560/12/4/046005.
identifying walking patterns of normal healthy individuals, Indian J. Biomech. 118 [32] F. Khan, B. Amatya, Rehabilitation in multiple sclerosis: a systematic review of
(2009) 118–122. systematic reviews, Arch. Phys. Med. Rehabil. 98 (2) (2017) 353–367, https://doi.
[9] J. Cui, et al., Wearable Gait Lab System providing quantitative statistical support for org/10.1016/j.apmr.2016.04.016.
human balance tests, Smart Health 3 (2017) 27–38, https://doi.org/10.1016/j. [33] R.E. Tanner, L.B. Brunker, J. Agergaard, K.M. Barrows, R.A. Briggs, O.S. Kwon,
smhl.2017.05.001. et al., Age-related differences in lean mass, protein synthesis and skeletal muscle
[10] S. Díaz, J.B. Stephenson, M.A. Labrador, Use of wearable sensor technology in gait, markers of proteolysis after bed rest and exercise rehabilitation, J. Physiol. 593 (18)
balance, and range of motion analysis, Appl. Sci. 10 (1) (2020) 234, https://doi. (2015) 4259–4273, https://doi.org/10.1113/JP270699.
org/10.3390/app10010234. [34] P. Edwards, J. Ebert, B. Joss, G. Bhabra, T. Ackland, A. Wang, Exercise re-
[11] F. Kerber, M. Puhl, A. Krüger, User-independent real-time hand gesture recognition habilitation in the non-operative management of rotator cuff tears: a review of the
based on surface electromyography, Proceedings of the 19th International literature, Int. J. Sports Phys. Ther. 11 (2) (2016) 279–301.
Conference on Human-Computer Interaction with Mobile Devices and Services, [35] R. Di Giminiani, S. Lancia, M. Ferrari, V. Quaresima, H.T. Vistisen, A. Kliltgaard,
ACM, 2017, September, p. 36, , https://doi.org/10.1145/3098279.3098553. et al., A wearable integrated textile EMG and muscle oximetry system for mon-
[12] M.E. Benalcázar, A.G. Jaramillo, A. Zea, A. Páez, V.H. Andaluz, Hand gesture re- itoring exercise-induced effects: a feasibility study, Proceedings of the 2018 IEEE
cognition using machine learning and the Myo armband, Proceedings of the 2017 International Symposium on Medical Measurements and Applications (MeMeA),
25th European Signal Processing Conference (EUSIPCO), IEEE, 2017, August, pp. IEEE, 2018, June, pp. 1–5, , https://doi.org/10.1109/MeMeA.2018.8438785.
1040–1044, , https://doi.org/10.23919/EUSIPCO.2017.8081366. [36] R.J. Cotton, J. Rogers, Wearable monitoring of joint angle and muscle activity, in:
[13] W.T. Shi, Z.J. Lyu, S.T. Tang, T.L. Chia, C.Y. Yang, A bionic hand controlled by hand IEEE (Ed.), Proceedings of the 2019 IEEE 16th International Conference on
gesture recognition based on surface EMG signals: a preliminary study, Biocybern. Rehabilitation Robotics (ICORR), 2019, June, pp. 258–263, , https://doi.org/10.
Biomed. Eng. 38 (1) (2018) 126–135, https://doi.org/10.1016/j.bbe.2017.11.001. 1109/ICORR.2019.8779538.
[14] M. Georgi, C. Amma, T. Schultz, Recognizing hand and finger gestures with IMU [37] M. Sekiya, S. Sakaino, T. Toshiaki, Linear logistic regression for estimation of lower
based motion and EMG based muscle activity sensing, Proceedings of the limb muscle activations, IEEE Trans. Neural Syst. Rehabil. Eng. 27 (3) (2019)
International Conference on Bio-inspired Systems and Signal Processing 523–532, https://doi.org/10.1109/TNSRE.2019.2898207.
(BIOSIGNALS-2015), Scitepress, 2015, January, pp. 99–108, , https://doi.org/10. [38] R.J. Oweis, R. Rihani, A. Alkhawaja, ANN-based EMG classification for myoelectric
5220/0005276900990108. control, Int. J. Med. Eng. Inform. 6 (4) (2014) 365–380, https://doi.org/10.1504/
[15] J.G. Abreu, J.M. Teixeira, L.S. Figueiredo, V. Teichrieb, Evaluating sign language IJMEI.2014.065442.
recognition using the myo armband, Proceedings of the 2016 XVIII Symposium on [39] H. Huang, T. Li, C. Bruschini, C. Enz, V.M. Koch, J. Justiz, et al., EMG pattern
Virtual and Augmented Reality (SVR), IEEE, 2016, June, pp. 64–70, , https://doi. recognition using decomposition techniques for constructing multiclass classifiers,
org/10.1109/SVR.2016.21. Proceedings of the 2016 6th IEEE International Conference on Biomedical Robotics
[16] J. Liu, D. Ying, W.Z. Rymer, EMG burst presence probability: a joint time–frequency and Biomechatronics (BioRob), IEEE, 2016, June, pp. 1296–1301, , https://doi.org/
representation of muscle activity and its application to onset detection, J. Biomech. 10.1109/BIOROB.2016.7523810.
48 (6) (2015) 1193–1197, https://doi.org/10.1016/j.jbiomech.2015.02.017. [40] M. Rossi, S. Benatti, E. Farella, L. Benini, Hybrid EMG classifier based on HMM and
[17] J. Liu, D. Ying, W.Z. Rymer, P. Zhou, Robust muscle activity onset detection using SVM for hand gesture recognition in prosthetics, Proceedings of the 2015 IEEE
an unsupervised electromyogram learning framework, PLoS One 10 (6) (2015) International Conference on Industrial Technology (ICIT), IEEE, 2015, pp.
e0127990, , https://doi.org/10.1371/journal.pone.0127990. 1700–1705, , https://doi.org/10.1109/ICIT.2015.7125342.
[18] C. D'Anna, T. Varrecchia, M. Schmid, S. Conforto, Using the frequency signature to [41] Y. Lecun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) 436–444,
detect muscular activity in weak and noisy myoelectric signals, Biomed. Signal https://doi.org/10.1038/nature14539.
Process. Control 52 (2019) 69–76, https://doi.org/10.1016/j.bspc.2019.02.026. [42] M. Wand, T. Schultz, Pattern learning with deep neural networks in EMG-based
[19] F. Khanam, S. Islam, M.A. Rahman, M. Ahmad, Muscle activity estimation through speech recognition, Proceedings of the 2014 36th Annual International Conference
surface EMG analysis during salat, Proceedings of the 2015 International of the IEEE Engineering in Medicine and Biology Society, IEEE, 2014, August, pp.
Conference on Electrical Engineering and Information Communication Technology 4200–4203, , https://doi.org/10.1109/EMBC.2014.6944550.
(ICEEICT), IEEE, 2015, May, pp. 1–6, , https://doi.org/10.1109/ICEEICT.2015. [43] M.D. Zeiler, R. Fergus, Visualizing and understanding convolutional networks,
7307402. Proceedings of the European Conference on Computer Vision, Springer, 2014,
[20] R.B. Souza, C.M. Powers, Differences in hip kinematics, muscle strength, and muscle September, pp. 818–833, , https://doi.org/10.1007/978-3-319-10590-1_53.
activation between subjects with and without patellofemoral pain, J. Orthop. [44] W. Rawat, Z. Wang, Deep convolutional neural networks for image classification: a
Sports. Phys. Ther. 39 (1) (2009) 12–19, https://doi.org/10.2519/jospt.2009.2885. comprehensive review, Neural Comput. 29 (9) (2017) 2352–2449, https://doi.org/
[21] F. Nougarou, A. Campeau-Lecours, R. Islam, D. Massicotte, B. Gosselin, Muscle 10.1162/neco_a_00990.
activity distribution features extracted from HD sEMG to perform forearm pattern [45] M. Atzori, M. Cognolato, H. Müller, Deep learning with convolutional neural net-
recognition, Proceedings of the 2018 IEEE Life Sciences Conference (LSC), IEEE, works applied to electromyography data: a resource for the classification of
2018, October, pp. 275–278, , https://doi.org/10.1109/LSC.2018.8572204. movements for prosthetic hands, Front. Neurorobot. 10 (2016) 9, https://doi.org/
[22] A. Rubiano, Smart control of a soft robotic hand prosthesis, Ph.D. Thesis disserta- 10.3389/fnbot.2016.00009.
tion, Paris Nanterre University, France, 2016. [46] J.O. Pinzón-Arenas, R. Jiménez-Moreno, J.E. Herrera-Benavides, Convolutional
[23] L. Chen, H. Wu, S. Duan, Y. Guan, J. Rojas, Learning human-robot collaboration neural network for hand gesture recognition using 8 different EMG signals,
insights through the integration of muscle activity in interaction motion models, Proceedings of the 2019 XXII Symposium on Image, Signal Processing and Artificial
Proceedings of the 2017 IEEE-RAS 17th International Conference on Humanoid Vision (STSIVA), IEEE, 2019, April, pp. 1–5, , https://doi.org/10.1109/STSIVA.
Robotics (Humanoids), IEEE, 2017, November, pp. 491–496, , https://doi.org/10. 2019.8730272.
1109/HUMANOIDS.2017.8246917. [47] R.N. Khushaba, A. Al-Timemy, S. Kodagoda, K. Nazarpour, Combined influence of

18
J.O. Pinzón-Arenas, et al. Sensing and Bio-Sensing Research 29 (2020) 100353

forearm orientation and muscular contraction on EMG pattern recognition, Expert wheelchair propulsion: influence of wheelchair configuration, Clin. Biomech. 25 (9)
Syst. Appl. 61 (2016) 154–161, https://doi.org/10.1016/j.eswa.2016.05.031. (2010) 879–885, https://doi.org/10.1016/j.clinbiomech.2010.07.002.
[48] P. Visconti, F. Gaetani, G.A. Zappatore, P. Primiceri, Technical features and func- [56] R.E. Singh, K. Iqbal, G. White, J.K. Holtz, A review of EMG techniques for detection
tionalities of Myo armband: an overview on related literature and advanced ap- of gait disorders, in: M.A. Aceves-Fernandez (Ed.), Artificial Intelligence:
plications of myoelectric armbands mainly focused on arm prostheses, Int. J. Smart Applications in Medicine and Biology, IntechOpen, London, 2019, , https://doi.org/
Sens. Intell. Syst. 11 (1) (2018) 1–25, https://doi.org/10.21307/ijssis-2018-005. 10.5772/intechopen.84403.
[49] A. Ganiev, H.S. Shin, K.H. Lee, Study on virtual control of a robotic arm via a myo [57] J.A. Guerrero, M.A. Castillo-Galván, J.E. Macías-Díaz, Novel electromyography
armband for the selfmanipulation of a hand amputee, Int. J. Appl. Eng. Res. 11 (2) signal envelopes based on binary segmentation, Biomed. Signal Process. Control 45
(2016) 775–782. (2018) 225–236, https://doi.org/10.1016/j.bspc.2018.05.026.
[50] J. Pinzón Arenas, R. Jiménez Moreno, R.D. Hernández Beleño, EMG signal acqui- [58] J.F. Kenney, E.S. Keeping, Root mean square, in: J.F. Kenney, E.S. Keeping (Eds.),
sition and processing application with CNN testing for MATLAB, Int. Rev. Autom. Mathematics of Statistics, 3rd ed., Van Nostrand, Princeton, NJ, 1962, pp. 59–60.
Control 11 (1) (2018) 44–51, https://doi.org/10.15866/ireaco.v11i1.13379. [59] N.A. Pashtoon, IIR digital filters, in: D.F. Elliott (Ed.), Handbook of Digital Signal
[51] Z. Arief, I.A. Sulistijono, R.A. Ardiansyah, Comparison of five time series EMG Processing, Academic Press, Cambridge, MA, 1987, pp. 289–357, , https://doi.org/
features extractions using Myo Armband, Proceedings of the 2015 International 10.1016/C2009-0-21739-9.
Electronics Symposium (IES), IEEE, 2015, September, pp. 11–14, , https://doi.org/ [60] D. Farina, R. Merletti, Comparison of algorithms for estimation of EMG variables
10.1109/ELECSYM.2015.7380805. during voluntary isometric contractions, J. Electromyogr. Kinesiol. 10 (5) (2000)
[52] W. Geng, Y. Du, W. Jin, W. Wei, Y. Hu, J. Li, Gesture recognition by instantaneous 337–349, https://doi.org/10.1016/S1050-6411(00)00025-0.
surface EMG images, Sci. Rep. 6 (2016) 36571, https://doi.org/10.1038/ [61] R. Merletti, P. Di Torino, Standards for reporting EMG data, J. Electromyogr.
srep36571. Kinesiol. 9 (1) (1999) 3–4.
[53] P. Welch, The use of fast Fourier transform for the estimation of power spectra: a [62] M.W. Coppieters, A.M. Alshami, Longitudinal excursion and strain in the median
method based on time averaging over short, modified periodograms, IEEE Trans. nerve during novel nerve gliding exercises for carpal tunnel syndrome, J. Orthop.
Audio Electroacoustics 15 (2) (1967) 70–73. Res. 25 (7) (2007) 972–980, https://doi.org/10.1002/jor.20310.
[54] A. Khazaee, A. Ebrahimzadeh, Classification of electrocardiogram signals with [63] C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, C. Liu, A survey on deep transfer
support vector machines and genetic algorithms using power spectral features, learning, Proceedings of the International conference on artificial neural networks,
Biomed. Signal Process. Control 5 (4) (2010) 252–263, https://doi.org/10.1016/j. Springer, 2018, October, pp. 270–279, , https://doi.org/10.1007/978-3-030-
bspc.2010.07.006. 01424-7_27.
[55] N. Louis, P. Gorce, Surface electromyography activity of upper limb muscle during

19