Human Activities Recognition and Monitoring

System Using Machine Learning Techniques

2022 Trends in Electrical, Electronics, Computer Engineering Conference (TEECCON) | 978-1-6654-8366-7/22/$31.00 ©2022 IEEE | DOI: 10.1109/TEECCON54414.2022.9854829

RK Pinky Dr. Sapam Jitu Singh

Department of Computer Science and Engineering (CSE) Department of Computer Science and Engineering (CSE)
Manipur Institute of Technology, Manipur University Manipur Institute of Technology, Manipur University
Imphal, INDIA Imphal, INDIA
[email protected] [email protected]

Chongtham Pankaj
Department of Information Science and Engineering (ISE)
M S Ramaiah Institute of Technology
Bangalore, INDIA
[email protected]

Abstract—Human activity recognition is the wide range of field algorithm to predict human actions. The required number of
of research and challenging task to identify the actions of the nodes are limited and most efficient way to have only one
human in period of time based on received signal strength data node which makes comfortable of the person. The machine
in wireless sensor network. It is important to monitor activity
of a person for numerous reasons. Recently, Machine Learning learning algorithms results are found to predict less accuracy
approach shows capable of classifying the actions of the human with less data from single node over multiple nodes with multi-
by automatically using the raw sensor data. In this work, the dimensional data. Several issues are to be addressed when
dataset consists of received signal strength of seven activities using collecting the data from sensor data. If data from sensor nodes
three sensor nodes that are trained by using supervised machine are corrupted due to non-functional of the nodes which lead to
learning algorithms to recognize the actions and random activities
are monitored to identify the strange action of the person using loss of predicting coordination of the human movement. The
unsupervised machine learning. The proposed machine learning maintaining of the sensor nodes that is required over period
based human activity recognition model are evaluated and predict of time. The selection of machine learning methods which is
the seven human activities by achieving 90% of accuracy. The achieved efficient accuracy of prediction of the coordination
model is later improved to recognize the random actions of the of the nodes.
human.
Index Terms—Human Activity Recognition, Robotics, Machine The proposed work has the list of problem statements: To
Learning, Neural Network gather the data of human actions with number of sensor nodes
coordination. To pre-process and clean the dataset by finding
I. I NTRODUCTION missing values. To implement various supervised machine
learning algorithms that will suitable for the selected dataset of
In times of covid-19 pandemic, human movement are re- the human movement. To train the selected supervised machine
quired to monitor when it is necessary for quarantine to stop learning algorithm that predict and classify the different hu-
the spread of coronavirus. This requires the effective and man activities. To develop the unsupervised machine learning
economic system or device that is capable of detect the object methods that is able to forecast the unexpected movement of
movement within the defined parameter. Machine Learning is the object. To deploy machine learning model to predict the
the robust method that can solve complex problems. Various anomaly behavior and classify the human actions.
machine learning methods can learn pattern of the human Objectives of our work includes To select the benchmark
movement from wearable sensor data that can also predict dataset for the human movement To pre-process the selected
the movement. The proposed work can monitor the movement dataset for machine learning methods To implement supervised
of the human including walking, sitting, standing with the machine learning algorithms such as Support Vector Machines,
help of wearable sensor data. The supervised machine learning Decision Tree, Random Forest for classification of human
methods are used to predict the movement of the human activities. To implement the unsupervised machine learning
with the defined parameter horizontally but limited to detect algorithm that is kohonen’s Self-Organizing Maps to find the
movement vertically. anomaly movement of the object. To deploy machine learning
Challenges of our work is that the placement of wearable based monitoring of human actions.
sensor which is not much interfere with the normal activities
of the person is the challenges deal. Without the proper II. R ELATED W ORK
placement of the iris node [1][2], the data receiving from the In recent years, the analysis of human activities is necessary
node may not accurate that may affect the machine learning in numerous sectors such as medical sector, robotic indus-

978-1-6654-8366-7/22/$31.00 ©2022 IEEE 62

Authorized licensed use limited to: Khulna Univ of Engineering & Technology - KUET. Downloaded on November 14,2024 at 02:30:17 UTC from IEEE Xplore. Restrictions apply.
 trial sector. With the advancement of Artificial Intelligence – implemented and used to evaluate the human activity recog-
Machine Learning (AIML) methodologies, the recognition of nition classification using accelerometer and gyroscope data
human activities with the help of sensor data are considered and found 98 percent of accuracy rate. The authors [16] [17]
in recent research. In [6], the analysis of various machine [18] uses various machine learning methods to solve diverted
learning including Random Forest, Support Vector Machine, problems in anomaly detection, healthcare and wireless sensor
K-Nearest Neighbor that all achieve more than 85 percent of network which are achieved efficiency solutions.
accuracy of classification of human activities with the wearable
equipment The recognition of continuous human activities is III. M ETHODOLOGY
identified in [3] apart from pre-defined label set of activities A. Artificial Intelligence
using machine learning methods. The authors applied the one- Artificial Intelligence (AI) is the important area of the
class support vector machine which is unsupervised machine computer science in present that helps the problem solving to
learning that classified the human activities dynamically. The mimic the human behavior without actual human intervention.
monitoring of patient in medical sectors was considered in But AI is not the new technology in the computer science, it
[4] without pre-defined human activities continuously with was discussed in 1950s with deploying many mathematical
the help of semi-supervised machine learning. Online semi- model and statistics to enhance the technology of the machine
supervised learning which is trained continuous in real-time of functioning. It has many methods to enable machines to mimic
sensor’s data such as data from smartphone and other wearable the system like human thinking. Machine learning is the one
devices are capable of identifying human activities. The 10 of the subsets of the AI, it uses various algorithms to process
percent of labelled sensor data are examined in [5] that feed to data to act like human intelligence.
deep learning for learning the pattern. The authors use of semi-
supervised deep learning methods with less training datasets B. Machine Learning
and show the improvement of recognizing human activities. Machine Learning (ML) is the subset of the Artificial Intelli-
There are challenges to recognize multiple human activities gence that enables the system to predict, classify and clustering
by using single machine learning method [7]. Waist-mounted different types of data by implementing various algorithms.
accelerometer and gyroscope sensor are used and initially, Machine Learning has mainly two types : Supervised Machine
Random Forest is used to learn the activity patterns. Support Learning and Unsupervised Machine Learning. ML has the
Vector Machine and convolutional neural network are used capability of analyzing the high-dimensional data that leads
to identify each human activity. It is observed that support to explore unseen data points within the data. This can easily
vector machine plays big role in static human activity but not detect the data which is deviated from the normal data points.
in moving activity. Overall, it is achieved to reach over 95
percent using UCI-HAR dataset. The 3D skeleton data is also C. Supervised Machine Learning
used in [8] which is directly fed to the neural network with Supervised Machine Learning is one of the most popular
convolutional and dense layers. The result of neural network type of machine learning. As the name itself described, it
is passed to the support vector machine to classify using PKU learns the dataset from the labelled to classify to predicted
– MMD dataset. The authors claim by over 90 percent of results. It discovers rare information from the dataset with
accuracy is achieved using UTD – MHAD dataset. Even in help of robust algorithms that helps to predict and classify
[9], the long-short term memory network is used to extract the desired output. Some of the supervised machine learning
features from high dimensional sensor data. Using Human algorithms are: o Naı̈ve Bayes Classifier o Decision Tree
physical activities dataset for evaluation by LSTM with the o Random Forest o Support Vector Machine o K-Nearest
support vector machine, random forest and k-nearest neighbor Neighbor, etc.
to achieve over 90 percent of accuracy for HUGABD and HPA
D. Unsupervised Machine Learning
dataset. Recognition of human activity also helps in health
monitoring for individual [10]. The authors used the recurrent Unsupervised Machine Learning is one of the power-
neural network using ARAS multi-resident real-world dataset. ful machine learning algorithms. Unlike supervised machine
The authors [11] analyzed the total of nine different machine learning, it clusters the dataset into different categories with
learning methods using single dataset to point out the robust unlabeled dataset. It finds the rare data points which is deviated
methods. Long short term memory and deep learning methods from the similar data points to help the system detection.
are found to achieve the highest accuracy not greater than Some of the unsupervised machine learning algorithms are:
92 percent while among machine learning algorithms such o Kohonen’s Self-Organizing Maps o K-means o Means shift.
as logistic regression, gradient boosting achieving 90 to 92
IV. P ROPOSED M ETHODOLOGY
percent and other algorithm achieved less than 90 percent. In
[12], the light weight model using hybrid architecture with A. Dataset
convolutional neural network and long short term memory The dataset consists of real-life benchmark of human move-
are used using benchmark dataset by achieving nearly 95 ment. The data are collected from the devices that measures the
percent of accuracy. Also in [15], the convolutional neural RSS of the beacon packets they exchange among themselves
network as well as long short term memory neural network are in the wireless sensor [1]. For each activity 15 temporal

63
Authorized licensed use limited to: Khulna Univ of Engineering & Technology - KUET. Downloaded on November 14,2024 at 02:30:17 UTC from IEEE Xplore. Restrictions apply.
 machine learning algorithm such as random forest, k-Nearest
Neighbour, Support Vector Machine, Gaussian Naive Bayes
are implemented using python script. The pandas package is
used to analyze the dataset for the machine learning algo-
rithms. The dataset is divided into train and test dataset using
sklearn train test split function. The accuracy of the machine
learning algorithm with the dataset are obtained with the help
of confusion matrix from sklearn metrics function.
V. R ESULT AND D ISCUSSION
The RSS between the sensors are analyzed for human
actions such as sitting, standing, cycling, lying, walking,
bending and the signal data patterns are non-uniform as shown
in Fig.3, Fig.4 and Fig.5 for Signal Strength between Sensor
1 and 2, Sensor 1 and 3 and Sensor 2 and 3 respectively.
Fig. 1. Placement of three sensors on human body

sequences of input RSS data are present. The dataset consists

of 480 sequences, for the total number of 42240 data.

TABLE I
PARAMETER OF SENSOR DATA

Parameters values
Avg rss12 numerical value
Var rss12 numerical value
Avg rss13 numerical value
Var rss13 numerical value
Avg rss23 numerical value
Var rss23 numerical value
Fig. 3. Signal Strength between Sensor 1 and Sensor 2

B. Preprocessing data The signal data pattern for these human actions are observed
The dataset is clean by deleting null value and any other that would be very complex to classify each action by simple
missing data in the records. We have used the python scripting mathematical methods.
using mostly numpy and pandas packages to pre-process the
dataset. After cleaning process is done, each activity are
labelled with the respectively actions. The final dataset is made
by combining all the activity defined in raw dataset. The final
dataset are further divided into fine proportional of test and
train dataset for applying machine learning methods.
C. Experiment

Fig. 4. Signal Strength between Sensor 1 and Sensor 3

Machine learning algorithms are used to classify each action

and the accuracy of each classification are observed as follows.
A. Random Forest
Fig. 2. Sensor placement of Human body and Machine Learning modelling The accuracy of 91% is achieved using random forest
algorithm. In confusion matrix, the activities bending, lying,
The appropriate machine learning algorithms are selected sitting, standing are well classified compared to classification
for the dataset. With the help of sci-kit learn packages, the between cycling and walking.

64
Authorized licensed use limited to: Khulna Univ of Engineering & Technology - KUET. Downloaded on November 14,2024 at 02:30:17 UTC from IEEE Xplore. Restrictions apply.
 Fig. 9. Confusion Matrix - Gaussian Naive Bayes

VI. C ONCLUSION
Most of the industry increases use of robotics (Human
model) and healthcare system needs to monitor remotely the
Fig. 5. Signal Strength between Sensor 2 and Sensor 3
patient’s activities continuously for treatment. The Human
Activity Recognition system plays a big role with advancement
of machine learning techniques. In our work, using simple
sensor nodes to be attached to the human or can be worn
easily in three places that is in both knee and one in upper
part of the body. The measuring distance between these three
sensor nodes would be transmitted wirelessly. The machine
learning algorithms learn the sensor data pattern for each
pre-defined human activities and further the trained machine
Fig. 6. Confusion Matrix - RandomForest learning model is able to classify the pre-defined human
activities with un-labelled incoming sensor data. The random
B. K-Nearest Neighbour forest and K-nearest neighbour can achieve 90% of accuracy
K-Nearest Neighbour can achieve almost same as random of classification. Support Vector Machine and Gaussian naive
forest that is 89% of accuracy. In K-nearest Neighbour con- bayes can achieve 85% of accuracy.
fusion matrix results, cycling and walking are misclassified In continuation of this work, the unsupervised machine
more compared to other activities. learning algorithm is used to find the anomaly activity of
human movement apart from pre-defined human activities. The
neural network based unsupervised machine learning method
is developed that is capable of classification and predict
anomaly behaviour of the human movement.

R EFERENCES
[1] F.Palumbo, C.Gallicchio, R.Pucci and A.Micheli, ”‘Human Activity
Recognition using multisensor data fusion based on Reservior Com-
puting,“, Journal of Ambient Intelligence and Smart Environments, p.p.
Fig. 7. Confusion Matrix - K-Nearest Neighbour 87-107, 2016
[2] F.Palumbo, P.Barsocchi, C.Gallicchio, S.Chessa and A.Micheli, ”‘ Mul-
tisensor data fusion for activity recognition based on reservoir comput-
C. Support Vector Machine ing,“, Evaluating AAL Systems Through Competitive Benchmarking,
Communications in Computer and Information Science, vol. 386, p.p.
Support Vector Machine can achieve 86% of accuracy. 24, Springer, 2013.
[3] B. Haixia, P. N. Miquel, S. R. Raul and F. Peter, ”‘Human Activity
Recognition Based on Dynamic Active Learning,“, IEEE Journal of
Biomedical and Health Informatics, vol. 25, p.p. 9220934, 2021.
[4] L. C. Hugo and M. M. Jaoa, ”‘Human Activity Recognition by Means
of Online Semi-supervised Learning,“, IEEE International Conference
on Mobile Data Management, p.p. 75-77, 2016
[5] C. Kaixuan, Y. Lina, Z. Dalin, W Xianzhi, C. Xiaojun and N. Feip-
ing, ”‘A Semisupervised Recurrent Convolutional Attention Model for
Human Activity Recognition,“, IEEE Transactions on Neural Networks
and Learning Systems, vol. 31, p.p. 1747-1756, 2020.
[6] G. Negar and M. Mahta, ”‘A Comparison of Machine Learning Clas-
Fig. 8. Confusion Matrix - Support Vector Machine sifiers for Human Activity Recognition using Magnetic Induction-
based Motion signals,“, 14th European Conference on Antennas and
Propagation, p.p. 1-3, 2020.
D. Gaussion Naive Bayes [7] M. H. S. Md, A. Nafis, N. Koundinya and P. Kannappan, ”‘A Hybrid
Approach for Human Activity Recognition with Support Vector Machine
Gaussian Naive Bayes can achieve 85% of accuracy similar and 1D Convolutional Neural Network,“, IEEE Applied Imagery Pattern
to Support Vector Machine Recognition Workshop, p.p. 1-5, 2020.

65
Authorized licensed use limited to: Khulna Univ of Engineering & Technology - KUET. Downloaded on November 14,2024 at 02:30:17 UTC from IEEE Xplore. Restrictions apply.
 [8] H. Plamen, M. Agata and B. Ognian, ”‘Deep Learning and SVM-Based
Method for Human Activity Recognition with Skeleton Data,“, 28th
National Conference with International Participation, p.p. 49-52, 2020.
[9] K. Girja, C. Jayeeta, N. Anup, ”‘Effect of Reduced Dimensionality
on Deep learning for Human Activity Recognition,“, 11th International
Conference on Computing, Communication and Networking Technolo-
gies, p.p. 1-7, 2020.
[10] N. Anubhav, S. Abhishek, P. Thinagaran and S. Suman, ”‘Deep Learning
for Multi-Resident Activity Recognition in Ambient Sensing Smart
Homes,“, p.p. 340-341, 2019.
[11] P. D. Ashish and S. H. Jigarkumar, ”‘Performance Analysis of Su-
pervised Machine Learning Algorithms to Recognize Human Activity
in Ambient Assisted Living Environment,“, IEEE 16th India Council
International Conference, p.p. 1-4, 2019.
[12] G. P. Sonia, S. Qingquan and Z. Yan, ”‘Improved Sensor Based Human
Activity Recognition via Hybrid Convolutional and Recurrent Neural
Networks,“, IEEE International Symposium on Inertial Sensors and
Systems, p.p. 1-4, 2021.
[13] Q. Jin, W. Zhangjing, L. Xiancheng and L. Chunmin, ”‘Learning
Complex Spatio-Temporal Configurations of Body Joints for Online
Activity Recognition,“, IEEE Transactions on Human-Machine Systems,
vol. 48, p.p. 637-647, 2018.
[14] S. Sujittra and T. Kreangsak, ”‘An Individual model for Human Ac-
tivity Recognition Using Transfer Deep Learning,“, Joint International
Conference on Digital Arts, Media and Technology with ECTI Northern
Section Conference on Electrical, Electronics, Computer and Telecom-
munications Engineering, p.p. 149-152, 2020.
[15] T. Nilay adn O. Ozen, ”‘A Comparative Research on Human Activity
Recognition Using Deep Learning,“, 27th Signal Processing and Com-
munications Applications Conference, p.p. 1-4, 2019.
[16] B. P. Vijaya Kumar, D. V. Sujana, P. Chongtham, S. Shruthi and R.
Priyanka, ”‘Temporal based Network Packet Anomaly Detection using
Machine Learning,“, IEEE International Conference on Electronics,
Computing and Communication Technologies, p.p. 1-6, 2021.
[17] P. Chongtham, S. V. Konjengbam, S. R. Khoirom, ”‘Artificial Intelli-
gence enabled Web-Based Prediction of Diabetes using Machine Learn-
ing Approach,“, International Conference on Disruptive Technologies
for Multi-Disciplinary Research and Applications, p.p. 60-64, 2021.
[18] P. Chongtham, S. N. Gurumayum and S. R. Khoirom, ”‘Improved
Energy Lifetime of Integrated LEACH Protocol for Wireless Sensor
Network,“, International Conference on Disruptive Technologies for
Multi-Disciplinary Research and Applications, p.p. 265-269, 2021.

66
Authorized licensed use limited to: Khulna Univ of Engineering & Technology - KUET. Downloaded on November 14,2024 at 02:30:17 UTC from IEEE Xplore. Restrictions apply.