0% found this document useful (0 votes)
27 views9 pages

IEEE Access (Using CNN)

This document describes a study that uses convolutional neural networks for real-time gait analysis based on human pose estimation. The study presents an approach where skeletal images generated from human pose estimation are classified by a CNN to detect normal and abnormal gaits. This allows for gait analysis to be performed without specialized sensors or devices by analyzing video frames to extract skeletal poses and feeding them to a trained CNN classifier. The proposed method aims to provide a gait analysis mechanism that minimizes constraints like camera angle, walking speed, clothing, and other factors by relying on extracted skeletal poses rather than full video frames.

Uploaded by

Nikita
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
27 views9 pages

IEEE Access (Using CNN)

This document describes a study that uses convolutional neural networks for real-time gait analysis based on human pose estimation. The study presents an approach where skeletal images generated from human pose estimation are classified by a CNN to detect normal and abnormal gaits. This allows for gait analysis to be performed without specialized sensors or devices by analyzing video frames to extract skeletal poses and feeding them to a trained CNN classifier. The proposed method aims to provide a gait analysis mechanism that minimizes constraints like camera angle, walking speed, clothing, and other factors by relying on extracted skeletal poses rather than full video frames.

Uploaded by

Nikita
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

Received September 1, 2020, accepted October 5, 2020, date of publication October 12, 2020, date of current version October

30, 2020.
Digital Object Identifier 10.1109/ACCESS.2020.3030086

Human Pose Estimation-Based Real-Time Gait


Analysis Using Convolutional Neural Network
ALI ROHAN 1 , (Associate Member, IEEE), MOHAMMED RABAH2 ,
TAREK HOSNY3 , AND SUNG-HO KIM 4
1 Department of Mechanical, Robotics and Energy Engineering, Dongguk University-Seoul, Seoul 04620, South Korea
2 Computational Engineering and Analysis Research Group, Turku University of Applied Sciences, 20520 Turku, Finland
3 Communication Engineering Department, Al-Safwa High Institute of Engineering, High Ministry of Education, Cairo 11837, Egypt
4 Department of Control and Robotics Engineering, Kunsan National University, Gunsan 54150, South Korea

Corresponding author: Ali Rohan ([email protected])

ABSTRACT Gait analysis is widely used in clinical practice to help in understanding the gait abnormalities
and its association with a certain underlying medical condition for better diagnosis and prognosis. Several
technologies embedded in the specialized devices such as computer-interfaced video cameras to measure
patient motion, electrodes placed on the surface of the skin to appreciate muscle activity, force platforms
embedded in a walkway to monitor the forces and torques produced between the ambulatory patient and the
ground, Inertial Measurement Unit (IMU) sensors, and wearable devices are being used for this purpose.
All of these technologies require an expert to translate the data recorded by the said embedded specialized
devices, which is typically done by a medical expert but with the recent improvements in the field of Artificial
Intelligence (AI), especially in deep learning, it is possible now to create a mechanism where the translation of
the data can be performed by a deep learning tool such as Convolutional Neural Network (CNN). Therefore,
this work presents an approach where human pose estimation is combined with a CNN for classification
between normal and abnormal gait of a human with an ability to provide information about the detected
abnormalities form an extracted skeletal image in real-time.

INDEX TERMS Convolutional neural network, deep learning, gait analysis, pose estimation.

I. INTRODUCTION misaligned pelvis or sacrum [5], [6], comparative biomechan-


Almost every person on this planet excluding the disabled ics; to understand about the mechanics of locomotion [7] and
persons walks approximately 1 to 3 Km on a daily basis and it biometrics; to recognize people by the way they walk [8].
is considered a very important factor in defining our physical Aside from clinical applications, gait analysis is also used in
and mental health. Most people do not consider this impor- professional sports training to optimize and improve athletic
tance of walking but it would not be inevitable to say that performance [9].
an inability to walk or to be mobile can dramatically change There has been extensive research in the recent past to
a person’s life. It can cause significant health problems for develop a methodology using different techniques for gait
the short and long term. Some people can move with abnor- analysis. Previously, authors [10] introduced an average sil-
mal gait for many years without any prominent symptoms, houette representation for a sequence. The Gait Energy Image
whereas for others, abnormal gait can cause injuries, pain (GEI) was proposed in [11]. The average silhouette represen-
that can lead towards a bigger health problem such as Mus- tation is widely used than other techniques due to its simplic-
culoskeletal problems [1], Cardiovascular health issues [2], ity and it is found to be effective in saving the computation
and Mental health issues [3]. In medical diagnostics, gait time. In [12], authors presented another gait representation
analysis is widely used to study underlying pathological gait scheme called as Gait Entropy Image, and in [13] Gait using
conditions [4], for diagnosis in chiropractic and osteopathic Pal and Pal Entropy (GPPE). A recent work conducted by
professions; as hindrances in gait may be indicative of a authors [14], evaluated that the performance of different gait
representation templates falls behind the performance level
The associate editor coordinating the review of this manuscript and of GEI using large gait datasets. Principal component anal-
approving it for publication was Jianqing Zhu . ysis (PCA) was used in gait recognition [15] to extract the

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
191542 VOLUME 8, 2020
A. Rohan et al.: Human Pose Estimation-Based Real-Time Gait Analysis Using Convolutional Neural Network

FIGURE 1. Basic architecture of the proposed approach for gait analysis.

unique features from gait representation. Linear discriminant the unavailability of a proper gait analysis mechanism for
analysis (LDA) was used in [16] to reduce the dimensionality the subjects where problematic constraints such as the view
of the gait representation data into lower dimensional space- angle, walking speed, clothing, surface, carrying status, shoe,
separated classes but the problem with these techniques is and elapsed time are considered and minimalized in an effi-
that PCA and LDA do not take advantage of 2-dimensional cacious way. Therefore, for further progress, this research
data. In PCA and LDA, an image or 2-D data has to be work proposes an approach where human pose estimation
converted to a 1-dimension vector and this sometimes results is combined with a deep neural network such as CNN to
in poor recognition performance. Some subspace learning classify the normal and abnormal gait of a person. The reason
approaches have also been proposed [17]–[20], which started to use human pose estimation for gait analysis is that in pose
to consider learning the features from an object by con- estimation, deep learning-based CNN is used to detect the
sidering the representation of higher-order tensors. Sub- body points of a person without the worry of any problematic
space learning approaches are widely used approaches for constraint. This gives us skeletal images where body points
gait recognition. In [21], the authors proposed matrix-based are joined to form a skeleton of a person. Only these obtained
sparse bilinear discriminant analysis (SBDA) as a sparse skeletal images are further used for classification and gait
learning method effective for gait recognition. Also, Locality analysis. Hence, giving the freedom from using any wearable
Preserving Projections (LPP) [22] and Local Fisher Discrim- sensor or device on the subject’s body with minimal effect of
inant Analysis (LEFDA) [23] were employed to generate the any problematic constraint.
gait features. This paper is divided into the following sections:
Recently, Convolutional Neural Networks (CNNs) have Section 1 provides an introduction, Section 2 describes the
achieved great results in different fields of pattern recog- basic architecture of the system; Section 3 contains the results
nition, detection, and classification, especially in computer and discussion, and Section 4 concludes the study.
vision. CNN is a class of deep neural networks and is mostly
applied to analyze visual imagery. In [24], an efficient deep II. BASIC ARCHITECTURE OF THE SYSTEM
neural network architecture for visual recognition is pro- The basic architecture of the proposed approach is shown
posed and it is named as GoogLeNet. It is a big network in the Fig.1. The first step involves the recording of a
that comprises of around 27 layers, and it uses max and live video of a person’s gait movement. Each frame in
average pooling, dropout method, and a softmax classifier. the video is processed through the human pose estima-
In [25], the authors proposed a method to recognize periodic tion algorithm to obtain a skeleton image comprising of
human actions using CNN. In [26], a large CNN trained using 25 body points. The obtained image is given as an input
1.2 million high-resolution images from the ImageNet dataset to a CNN trained to classify multiple classes. In this
was introduced. A CNN architecture to speed up the training work, we trained the CNN for five types of classes. These
time for largescale video classification was proposed in [27]. classes are Normal, Abnormal Left Toe, Abnormal Left Foot,
One million YouTube videos belonging to 487 classes of Abnormal Right Toe, and Abnormal Right Foot. Once the
sports were trained using this architecture. trained CNN receives the input skeletal image of a person,
All these above-mentioned studies have shown some the output is given as the image with a label of predicted
advancement in the process of gait analysis but there is class.

VOLUME 8, 2020 191543


A. Rohan et al.: Human Pose Estimation-Based Real-Time Gait Analysis Using Convolutional Neural Network

FIGURE 2. Overall pipeline. (a) The method takes the entire image as the input for a CNN to jointly predict (b) confidence maps for body part detection
and (c) PAFs for part association. (d) The parsing step performs a set of bipartite matchings to associate body part candidates. (e) Finally assemble
them into full body poses for all people in the image.

A. HUMAN POSE ESTIMATION


Human pose estimation is the process of predicting the struc-
ture of the body pose from a single, typically monocular
image and is considered as one of the key problems in the field
of computer vision. The importance of the pose estimation
is its applicability in serval applications such as higher-level
reasoning in the context of human-computer interaction and
activity recognition; it is also one of the basic building blocks
for marker-less motion capture (MoCap) technology. MoCap
technology is useful for applications ranging from character
animation to clinical analysis of the gait. FIGURE 3. Part association strategies. (a) The body part detection
The human pose estimation faces several challenges candidates (red and blue dots) for two body part types and allconnection
candidates (grey lines). (b) The connection results using the midpoint
among which the most significant are: (1) variability of (yellow dots) representation: correct connection (black lines) and
human visual appearance in images, (2) variability in light- incorrect connections (green lines) that also satisfy the incidence
constraint. (c) The results using PAFs (yellow arrows). By encoding
ing conditions, (3) variability in human physique, (4) partial position and orientation over the support of the limb, PAFs eliminate
occlusions due to self-articulation and layering of objects in false associations.
the scene, (5) complexity of human skeletal structure, (6) high
dimensionality of the pose, and (7) the loss of 3d information the authors presented a novel feature representation called
that results from observing the pose from 2d planar image part affinity fields that preserves both location and orientation
projections. In the past, there was no approach that could information across the region of support of the limb (as shown
produce satisfactory results in general while dealing with in Fig. 3c). The part affinity is a 2D vector field for each
all of the aforementioned challenges but recently researchers limb, for each pixel in the area belonging to a particular limb,
have shown very promising results using deep learning tools. a 2D vector encodes the direction that points from one part of
In [28], [29], the authors proposed a method for multi- the limb to the other. Each type of limb has a corresponding
person 2D pose estimation using Part Affinity Fields (PAF). affinity field joining its two associated body parts. The overall
PAF is a set of 2D vector fields that encode the location pipeline of the algorithm is shown in Fig.2 and Fig. 4 presents
and orientation of limbs over the image domain. Given a set the flow chart. The demonstrated results have significantly
of detected body parts (shown as the red and blue points improved the issues related to the human pose estimation
in Fig. 3a), how do we assemble them to form the full- and it has made possible to utilize human pose estimation in
body poses of an unknown number of people? We need a application like gait analysis and this work is a step towards
confidence measure of the association for each pair of body that. Fig. 2 shows the pipeline of the human pose estimation
part detections, i.e., that they belong to the same person. proposed by the authors and used in this work.
One possible way to measure the association is to detect an
additional midpoint between each pair of parts on a limb B. CNN-BASED CLASSIFIER
and check for its incidence between candidate part detec- A CNN-based classifier is used in this work to classify the
tions, as shown in Fig. 3b. However, when people crowd normal and abnormal gait of a person. The architecture of
together—as they are prone to do—these midpoints are likely the CNN used in this work is based on Visual Geometry
to support false associations (shown as green lines in Fig. 3b). Group-19 (VGG-19) architecture. Providing more depth in
Such false associations arise due to two limitations in the the network, and size of the convolution filters, VGG net-
representation: (1) it encodes only the position, and not the works have a clear advantage over other network architec-
orientation, of each limb; (2) it reduces the region of support ture such as AlexNet. The detail of the architecture is given
of a limb to a single point. To address these limitations, in Table 1. The CNN is composed of 16 convolutional layers

191544 VOLUME 8, 2020


A. Rohan et al.: Human Pose Estimation-Based Real-Time Gait Analysis Using Convolutional Neural Network

TABLE 1. The architecture of the CNN.

FIGURE 4. Flow chart for Human Pose Estimation.

and 3 fully connected layers. The size of the convolutional


filter or respective fields is 3 × 3. Two 3 × 3 convolutional
layers cover an effective receptive field of 5 × 5 and three
3 × 3 such layers have a 7 × 7 effective receptive field to
incorporate forward flow with three non-linear rectifications
instead of a single. The additional non-linearity introduction
makes the decision function more discriminative and reduces
the number of parameters significantly.

1) CNN TRAINING
The training of the CNN was performed offline by collecting
and labeling a dataset comprising of multiple videos of a
person walking under different scenarios. The data collection
is one of the vital parts in achieving a good classification
accuracy. We collected the data by segregating the classes into
five categories; Normal, Abnormal Left Toe, Abnormal Left
Foot, Abnormal Right Toe, and Abnormal Right Foot. For
each category, the walking style of the person was made to
change to mimic some injury or problem in feet. For each
category we collected 50 videos, each video was recorded
using a camera at 60 Frame Per Second (FPS) with a res-
olution of 1280 × 720 for a duration of 50 seconds. These
videos were processed through the human pose estimation TABLE 2. PC Specifications.
algorithm to obtain skeletal images which were further used
for training and testing the CNN. 80% of these 50 videos
for each category was used for training the CNN while 20%
were used for testing the CNN. The network was trained for
45,000 iterations on a PC with specifications mentioned in
Table 2. The accuracy achieved for classification is 97.3%.
To create a problematic gait, we mimicked the walking style
by using some object tied with the toe of the person under
experiment, this gave us the mimicked data for the categories
of Abnormal Left Toe and Abnormal Right Toe. On the other category label. It can be seen the Fig. 5-9 that all of this
hand, to mimic some problems in feet we made the person data was collected inside our laboratory and the person was
walk in slippers with a foot cast and it gave us the data for walking on a treadmill.
the categories of Abnormal Left foot and Abnormal Right
Foot. For category Normal, the person walked without any III. RESULTS AND DISCUSSION
foot cast or object attached to the toe. Fig. 5-9 shows some The implementation of the proposed approach presented
example images extracted from the recorded videos with their in this work was done in real-time by distributing the

VOLUME 8, 2020 191545


A. Rohan et al.: Human Pose Estimation-Based Real-Time Gait Analysis Using Convolutional Neural Network

FIGURE 5. Example of the recorded data for category Normal.

FIGURE 8. Example of the recorded data for category Abnormal Right


Foot.

FIGURE 6. Example of the recorded data for category Abnormal Left Foot.

FIGURE 9. Example of the recorded data for category Abnormal Right Toe.

was 47ms. The FPS achieved during this process was almost
20 FPS. We compared the computational time and FPS of
our network with a CNN of ResNet50 architecture. For
ResNet50, the time it took to process a single frame was
almost 12ms higher than proposed CNN with FPS of 15FPS.
The results obtained are shown in Fig. 10-13. The
results shown in these figures are respective to the
Fig. 6-9. Fig. 10-13 shows the 25 body points skeletal images
extracted using human pose estimation. These extracted body
points are conjoined to form a respective skeletal image
which is further processed through the CNN-based classifier
for classification of the categories. It should be noticed that in
this work 25 body points are used to classify a gait abnormal-
FIGURE 7. Example of the recorded data for category Abnormal Left Toe.
ity problem related to just human feet. The reason to extract
and use 25 body points rather than just footpoints is due to
computational load on CPU and GPU, working in parallel the fact that the trained CNN achieved more accurate results
to reduce the computational time of the CNN. The recorded when more body points information was provided with a very
time to process a single frame and classifying the category low confusion rate.

191546 VOLUME 8, 2020


A. Rohan et al.: Human Pose Estimation-Based Real-Time Gait Analysis Using Convolutional Neural Network

FIGURE 12. Extracted skeletal image using pose estimation for Fig. 8.

FIGURE 10. Extracted skeletal image using pose estimation for Fig. 6.

FIGURE 13. Extracted skeletal image using pose estimation for Fig. 9.

FIGURE 11. Extracted skeletal image using pose estimation for Fig. 7.
(3) Frames-Per-Second (FPS): FPS is the rate at which a
classifier is capable of processing incoming camera frames.
In order to compute the overall performance of the clas-
To evaluate the performance of the classifier we applied the sifier, we define a composite score metric [30]. The rea-
following three metrics: son to create such matric is to measure overall performance
(1) Sensitivity: This metric is defined as the proportion including the FPS which is a very important parameter to
of true positives images that are correctly classified by a analyze computer vision and image processing applications.
classifier. This metric is calculated by taking into account Whereas, the only accuracy was used to determine the perfor-
the True Positives (Tpos) and False Negatives (Fneg) of the mance of the CNN-based classifier. This metric consists of a
detected class as given by (1). linear combination of Sensitivity and Precision together with
Tpos the achieved FPS. We parametrize the score with respect to
Sensitivity = (1) a vector of weights w ∈ [0, 1]3 as given by (3). We priori-
Tpos + Fneg
tized FPS with a weight of 0.4 over the other two accuracy-
(2) Precision: This metric is a widely used metric and is related metrics because FPS is a more prominent factor in
defined as the proportion of True Positives among all the performance evaluation of the overall system, whereas other
detected classes of the system and is given by (2). parameters were equally weighted with 0.2.
Tpos Score (w) = w1 × FPS + w2 × Sensitivity + w3 × Precision
Precision = (2)
Tpos + Fpos (3)

VOLUME 8, 2020 191547


A. Rohan et al.: Human Pose Estimation-Based Real-Time Gait Analysis Using Convolutional Neural Network

FIGURE 16. Output result of the CNN-based classifier with respect to


FIGURE 14. Output result of the CNN-based classifier with respect to Fig.8, and Fig. 12 with detected category as Abnormal Right Foot.
Fig.6, and Fig. 10 with detected category as Abnormal Left Foot.

FIGURE 17. Output result of the CNN-based classifier with respect to


FIGURE 15. Output result of the CNN-based classifier with respect to Fig.9, and Fig. 13 with detected category as Abnormal Right Toe.
Fig.7, and Fig. 11 with detected category as Abnormal Left Toe.

problematic constraints such as clothing, surface, carrying


With Tpos = 143, 000, Fneg = 3000 and Fpos = 4000 status and shoes as it solely considers the human pose based
out of 150,000 images database obtained from 50 videos. The on body point skeletal images.
calculated score with a sensitivity of 0.9795 and a precision As mentioned before, the main idea is to propose an
of 0.9728 for the classifier and average FPS of 20 was 8.39. approach using CNN, it can be noticed from the results
We compared our classifier with a ResNet50 architecture presented that by just putting a minute difference in a person’s
with the same database and the score calculated for the toe and feet. CNN was able to distinguish between normal and
ResNet50 was 6.24. More importantly, the accuracy achieved abnormal gait. That proves that if CNN is trained properly
for the classifier was 97.3% higher than ResNet50’s 96.5%. with proper constraints, not it can help performing other
Fig. 14-17 shows the results achieved at the output of the tasks such as posture analysis and also can easily distinguish
classifier. These results are related to Fig. 6-9 used as an input between gaits of different people. In the future, we are work-
for human pose estimation and Fig. 10-13 given as an input ing on more precise whole-body points posture analysis and
to the CNN-based classifier. The classifier returns the output this work is one of the successful steps towards that.
with the category name. in this work, the classification thresh-
old was kept at 90%. The CNN only returned the images of IV. CONCLUSION
the respective class, if the said classification threshold was In this work, an approach to develop an efficient gait analysis
achieved. Furthermore, the proposed approach was tested by mechanism using deep learning tools such as Convolutional
changing the walking speed with different view angles of Neural Network (CNN)-based classifier is presented. The
the person and the results found were equally promising. proposed approach uses the human pose estimation method
It is necessary to provide data with different view angles to classify the abnormalities found in a specific person’s
and walking speed while training the classifier in order to gait. By introducing a proper data collection and training
create a proper gait analysis mechanism. The advantage of scheme for CNN, the proposed approach addresses the prob-
the proposed approach is that it eliminates the impact of other lems related to gait analysis methods used previously. The

191548 VOLUME 8, 2020


A. Rohan et al.: Human Pose Estimation-Based Real-Time Gait Analysis Using Convolutional Neural Network

experimental results are promising in solving some of the typ- [19] D. Xu, S. Yan, D. Tao, L. Zhang, X. Li, and H.-J. Zhang, ‘‘Human gait
ical problem constraints involved in the gait analysis system. recognition with matrix representation,’’ IEEE Trans. Circuits Syst. Video
Technol., vol. 16, no. 7, pp. 896–903, Jul. 2006.
The accuracy achieved for classifying normal and abnormal [20] D. Tao, X. Li, X. Wu, and S. J. Maybank, ‘‘General tensor discriminant
gait is 97.3% that proves the applicability of the proposed analysis and Gabor features for gait recognition,’’ IEEE Trans. Pattern
mechanism. Furthermore, in the future, the system can be Anal. Mach. Intell., 29, no. 10, pp. 1700–1715, Oct. 2007.
[21] Z. Lai, Y. Xu, Z. Jin, and D. Zhang, ‘‘Human gait recognition via sparse
used to distinguish different people’s gait by adding more data discriminant projection learning,’’ IEEE Trans. Circuits Syst. Video Tech-
in the training process of the classifier. nol., vol. 24, no. 10, pp. 1651–1662, Oct. 2014.
[22] X. He and P. Niyogi, ‘‘Locality preserving projections,’’ in Proc. NIPS,
vol. 16, 2003, pp. 153–160.
REFERENCES [23] M. Sugiyama, ‘‘Dimensionality reduction of multimodal labeled data
[1] W. Pirker and R. Katzenschlager, ‘‘Gait disorders in adults and the elderly: by local Fisher discriminant analysis,’’ J. Mach. Learn. Res., vol. 8,
A clinical guide,’’ Wiener klinische Wochenschrift, vol. 129, nos. 3–4, pp. 1027–1061, May 2007.
pp. 81–95, Feb. 2017, doi: 10.1007/s00508-016-1096-4. [24] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,
[2] J. R. Jørgensen, D. T. Bech-Pedersen, P. Zeeman, J. Sørensen, V. Vanhoucke, and A. Rabinovich, ‘‘Going deeper with convolutions,’’
L. L. Andersen, and M. Schönberger, ‘‘Effect of intensive outpatient in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015,
physical training on gait performance and cardiovascular health in people pp. 1–9.
with hemiparesis after stroke,’’ Phys. Therapy, vol. 90, no. 4, pp. 527–537, [25] E. P. Ijjina and C. K. Mohan, ‘‘One-shot periodic activity recognition using
2010. convolutional neural networks,’’ in Proc. 13th Int. Conf. Mach. Learn.
[3] R. D. Sanders and P. M. Gillig, ‘‘Gait and its assessment in psychiatry,’’ Appl., Dec. 2014, pp. 388–391.
Psychiatry (Edgmont), vol. 7, no. 7, pp. 38–43, 2010. [26] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ‘‘Imagenet classification
[4] W. Khan and A. Badii, ‘‘Pathological gait classification and segmen- with deep convolutional neural networks,’’ in Proc. Adv. Neural Inf. Pro-
tation by processing the hip joints motion data to support mobile cess. Syst., 2012, pp. 1097–1105.
gait rehabilitation,’’ Res. C Med. Eng. Sci., vol. 7, no. 3, 2019, [27] A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and
doi: 10.31031/RMES.2019.07.000662. L. Fei-Fei, ‘‘Large-scale video classification with convolutional neural
networks,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014,
[5] K. K. Haussler, ‘‘Equine manual therapies in sport horse practice,’’ Veteri-
pp. 1725–1732.
nary Clinics North America: Equine Pract., vol. 34, no. 2, pp. 375–389,
[28] Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh, ‘‘Realtime multi-
Aug. 2018.
person 2D pose estimation using part affinity fields,’’ CoRR 2016,
[6] P. Gottipati, S. Fatone, T. Koski, P. A. Sugrue, and A. Ganju,
arXiv:1611.08050. [Online]. Available: https://arxiv.org/abs/1611.08050
‘‘Crouch gait in persons with positive sagittal spine alignment resolves
[29] Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh, ‘‘OpenPose:
with surgery,’’ Gait Posture, vol. 39, no. 1, pp. 372–377, 2014,
Realtime multi-person 2D pose estimation using part affinity fields,’’ 2018,
doi: 10.1016/j.gaitpost.2013.08.012.
arXiv:1812.08008. [Online]. Available: https://arxiv.org/abs/1812.08008
[7] M. J. Kennedy, M. Lamontagne, and P. E. Beaulé, ‘‘Femoroacetabular [30] A. Rohan, M. Rabah, and S.-H. Kim, ‘‘Convolutional neural
impingement alters hip and pelvic biomechanics during gait: walking network-based real-time object detection and tracking for parrot
biomechanics of FAI,’’ Gait Posture, vol. 30, no. 1, pp. 41–44, Jul. 2009, AR drone 2,’’ IEEE Access, vol. 7, pp. 69575–69584, 2019,
doi: 10.1016/j.gaitpost.2009.02.008. doi: 10.1109/ACCESS.2019.2919332.
[8] M. J. Marin-Jimenez, F. M. Castro, N. Guil, F. de la Torre, and
R. Medina-Carnicer, ‘‘Deep multi-task learning for gait-based biometrics,’’
in Proc. IEEE Int. Conf. Image Process. (ICIP), Beijing, China, Sep. 2017, ALI ROHAN (Associate Member, IEEE) received
pp. 106–110, doi: 10.1109/ICIP.2017.8296252. the B.S. degree in electrical engineering from
[9] T. M. Parker, L. R. Osternig, P. van Donkelaar, and L.-S. Chou, The University of Faisalabad, Pakistan, in 2012,
‘‘Balance control during gait in athletes and non-athletes following and the M.S. and Ph.D. degrees in electrical,
concussion,’’ Med. Eng. Phys., vol. 30, no. 8, pp. 959–967, 2008, electronics, and control engineering from Kunsan
doi: 10.1016/j.medengphy.2007.12.006. National University, South Korea, in 2018 and
[10] Z. Liu and S. Sarkar, ‘‘Simplest representation yet for gait recognition: 2020, respectively. From 2012 to 2013, he worked
Averaged silhouette,’’ in Proc. 17th Int. Conf. Pattern Recognit., vol. 4, as a Development Engineer at the Niagara Group
Aug. 2004, pp. 211–214. of Industries, Pakistan. From 2013 to 2015, he
[11] J. Han and B. Bhanu, ‘‘Individual recognition using gait energy image,’’ worked as a Project Engineer for Circle Club,
IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 2, pp. 316–322, Pakistan. From 2015 to 2016, he worked as a Project Manager for Steam
Feb. 2006. Masters, Pakistan, and also as a Lecturer at the Department of Electri-
[12] K. Bashir, T. Xiang, and S. Gong, ‘‘Gait recognition using gait entropy cal and Telecommunication Engineering, Government College University
image,’’ in Proc. 3rd Int. Conf. Imag. Crime Detection Prevention (ICDP), Faisalabad, Pakistan. From 2016 to 2020, he worked as a Research Associate
2009, pp. 1–6. with the Factory Automation and Intelligent Control Lab., Kunsan National
[13] M. Jeevan, N. Jain, M. Hanmandlu, and G. Chetty, ‘‘Gait recognition University. He is currently working as an Assistant Professor with the
based on gait pal and pal entropy image,’’ in Proc. IEEE Int. Conf. Image Department of Mechanical, Robotics, and Energy Engineering, Dongguk
Process., Sep. 2013, pp. 4195–4199. University, South Korea. His research interests includes machine learning,
[14] H. Iwama, M. Okumura, Y. Makihara, and Y. Yagi, ‘‘The ou-isir gait AI, UAV’s, power electronics, fuzzy logic, EV systems, image processing,
database comprising the large population dataset and performance evalua-
computer vision, and prognostics and health management (PHM).
tion of gait recognition,’’ IEEE Trans. Inf. Forensics Security, vol. 7, no. 5,
pp. 1511–1521, Oct. 2012.
MOHAMMED RABAH received the B.A. degree
[15] M. Turk and A. Pentland, ‘‘Eigenfaces for recognition,’’ J. Cogn. Neurosci.,
in electronics and communication engineering
vol. 3, no. 1, pp. 71–86, 1991.
from the AL-SAFWA High Institute of Engi-
[16] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, ‘‘Eigenfaces vs.
Fisherfaces: Recognition using class specific linear projection,’’ IEEE neering, Egypt, in 2015, and the M.S. degree in
Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711–720, Jul. 1997. electronics and information engineering and the
[17] D. Xu, S. Yan, L. Zhang, S. Lin, H.-J. Zhang, and T. S. Huang, ‘‘Recon- Ph.D. degree from Kunsan National University,
struction and recognition of tensor-based objects with concurrent sub- South Korea, in December 2017 and August 2020,
spaces analysis,’’ IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 1, respectively. He is currently working as a Research
pp. 36–47, Jan. 2008. Engineer with the Turku University of Applied
[18] S. Yan, D. Xu, Q. Yang, L. Zhang, X. Tang, and H.-J. Zhang, ‘‘Discriminant Sciences, Turku, Finland. His research interests
analysis with tensor representation,’’ in Proc. IEEE Comput. Soc. Conf. includes automation, control and intelligent systems. Furthermore, he is also
Comput. Vis. Pattern Recognit. (CVPR), vol. 1, Jun. 2005, pp. 526–532. interested in UAV’s applications, fuzzy systems, and deep learning.

VOLUME 8, 2020 191549


A. Rohan et al.: Human Pose Estimation-Based Real-Time Gait Analysis Using Convolutional Neural Network

TAREK HOSNY received the B.S., M.S., and SUNG-HO KIM received the B.S. degree in elec-
Ph.D. degrees in communication engineering from trical engineering from Korea University, in 1984,
the Shoubra Faculty of Engineering, Cairo, Egypt, the M.S. and Ph.D. degrees in electrical engi-
in 2006, 2014, and 2019, respectively. He is cur- neering from Korea University, in 1986 and
rently a Lecturer with the Al-Safwa High Insti- 1991, respectively, and the post-doc from Japan
tute of Engineering, Communication Engineering Hiroshima University, in 1996. He is currently a
Department, Cairo, Egypt. His research interests Professor with Kunsan National University. His
include LI-FI, visible light communication sys- research interests includes fuzzy logic, sensor net-
tems, machine learning, and optics. works, neural networks, intelligent control sys-
tems, renewable energy systems, fault diagnosis
systems.

191550 VOLUME 8, 2020

You might also like