Khulna University of Engineering & Technology, Khulna

Department of Electrical and Electronic Engineering

Course No: EE 4000

Course Title: Project and Thesis

Report Title: Human Activity Recognition Using Machine Learning

techniques with smartphone sensor data.

Submitted by:
Ninad Kumar Sarker
Roll: 1903032
4th Year 2nd Term

Supervised by:
Dr. Md. Salah Uddin Yusuf
Professor
Department of Electrical and Electronic Engineering,KUET
 TABLE OF CONTENTS

SL. No. Contents Page no.

1. Introduction 3

2. Literature review 4-8

3. Motivation 8 - 10

4. Objectives 10

5. Progress 11 - 13

6. Future work 13

7. Conclusion 13 - 14
1.Introduction:
Human Activity Recognition (HAR) has emerged as a vital area of research, especially in the
context of ubiquitous computing and the widespread use of mobile devices. The growing
prevalence of smartphones equipped with advanced sensors, such as accelerometers,
gyroscopes, and magnetometers, has paved the way for continuous, non-intrusive
monitoring of human motion. These capabilities have unlocked a range of practical
applications, including healthcare monitoring, personalized fitness coaching, smart home
automation, elderly care, and real-time safety alerts.

The objective of HAR is to automatically detect and classify physical activities—such as

walking, running, sitting, standing, and lying down—by analyzing data collected from
embedded sensors. Accurate activity recognition enables proactive and personalized
services, making HAR systems indispensable in modern technological landscapes.
However, developing robust and efficient HAR models remains a significant challenge.
Traditional machine learning approaches, including k-Nearest Neighbors (k-NN), Support
Vector Machines (SVM), and Decision Trees, have been widely used in the past. These
methods often require extensive manual feature extraction and fine-tuning based on
domain knowledge, which can limit their adaptability and performance when applied to
complex or unseen activities.

Moreover, the variability in human motion, sensor placement, and the presence of noise in
sensor data introduce additional complexities. Human activities are inherently dynamic and
subject to numerous factors, such as speed, orientation, and environmental conditions. As
a result, there is a pressing need to explore more advanced machine learning techniques
that can effectively model and generalize across diverse patterns of human behavior.

This paper aims to address these challenges by investigating the use of machine learning
techniques for HAR using smartphone sensor data. We evaluate the effectiveness of various
machine learning models, exploring their strengths and limitations in accurately classifying
activities. The study emphasizes key aspects such as data preprocessing, feature
engineering, model optimization, and performance evaluation using real-world datasets. By
leveraging the power of machine learning, we seek to create a more accurate, adaptable,
and efficient HAR system that can enhance user experiences and improve the functionality
of smart healthcare and lifestyle applications. Our findings contribute valuable insights into
the design of future HAR systems, underscoring the transformative potential of machine
learning in human-centric applications.
2.Literature review:
1. "Accuracy vs Complexity: A Small Scale Dynamic Neural Networks Case" by M.
Dumpis and D. Navakauskas

• Summary: The paper investigates balancing accuracy and computational efficiency

in HAR using dynamically adjustable neural networks. It emphasizes reducing
energy consumption while maintaining acceptable accuracy, which is crucial for
real-time, resource-constrained applications like smartphone-based monitoring.

• Key Contributions: The dynamic approach optimizes model complexity based on the
input data context, ensuring scalability for practical use in everyday applications.

• Limitations & Shortcomings:

o The approach is tested only on a small, controlled dataset, raising concerns

about generalizability to diverse, real-world scenarios.

o It does not address the impact of different sensor placements or variations

in user behavior.

2. "ML-based Realtime User Activity and Transport Mode Detection using Smartphone
Sensors" by N. Liu

• Summary: This research focuses on real-time classification of user activities and

transport modes using machine learning, optimizing the performance of
smartphone sensors to detect behaviors efficiently.

• Key Contributions: The integration of deep learning with pedometer functions

enhances detection speed and reliability, especially in distinguishing between
various transport modes.

• Limitations & Shortcomings:

o The model's effectiveness may decrease in environments with inconsistent

or noisy sensor signals, such as urban areas with high traffic.

o The energy impact of continuous real-time monitoring on smartphone

battery life is not discussed in detail.
3. "Codebook-Based Feature Engineering for Human Activity Recognition Using
Multimodal Sensory Data" by S. Fatima et al.

• Summary: This paper introduces a codebook-based method for feature engineering,

which structures sensor data to enhance the performance of HAR systems. The
technique simplifies the representation of complex activity patterns for improved
classification.

• Key Contributions: The method shows improved accuracy by utilizing structured

features, particularly for multimodal datasets.

• Limitations & Shortcomings:

o The approach may not scale well to real-time applications due to the
computational overhead of feature encoding.

o There is limited exploration of how the codebook method performs under

variable sensor conditions or with incomplete data.

4. "Optimizing Human Activity Recognition with Ensemble Deep Learning on Wearable

Sensor Data" by N. Ashfaq et al.

• Summary: The authors propose an ensemble model combining CNNs, LSTMs, and
attention mechanisms to maximize HAR performance. The approach leverages the
strengths of each model, making it robust against noise and user variability.

• Key Contributions: The ensemble model achieves superior accuracy, adapting well
to various activities and user habits.

• Limitations & Shortcomings:

o The ensemble method is resource-intensive, potentially making it unsuitable

for low-power devices or real-time applications.

The paper does not discuss the challenges of synchronizing multiple deep learning models
or managing memory usage efficiently.

5. "Efficient Human Activity Recognition on Wearable Devices Using Knowledge

Distillation Techniques" by E. Souto et al.
 • Summary: The paper explores the use of knowledge distillation to compress large
HAR models for use on wearable devices, maintaining high accuracy while reducing
computational demands.

• Key Contributions: Distilled models operate efficiently on low-power hardware,

making them suitable for continuous monitoring applications.

• Limitations & Shortcomings:

o The method may not fully retain the original model's performance for
complex or overlapping activities.

o The research does not address the potential impact of real-world

environmental factors, such as extreme temperatures, on sensor
performance and model accuracy.

6. "An Optimal Feature Selection Method for Human Activity Recognition Using
Multimodal Sensory Data" by T. Haider, M.H. Khan, M.S. Farid

• Summary: This research proposes an optimal feature selection technique to

improve HAR performance while minimizing computational overhead. The method
combines statistical measures with machine learning to identify the most relevant
features.

• Key Contributions: The approach enhances classification accuracy and efficiency,

particularly in resource-constrained environments.

• Limitations & Shortcomings:

o The feature selection method may not adapt well to rapidly changing or
unpredictable activities.

o There is insufficient testing on diverse datasets, limiting the model's

generalizability across different populations.

7. "A Novel Hybrid Strategy Based on Swarm and Heterogeneous Federated Learning
for Activity Recognition" by H. Jamil, M.A. Khan, F. Jamil

• Summary: The authors present a hybrid framework combining swarm optimization

and federated learning to enhance HAR model performance while preserving data
 privacy. The approach allows for efficient, decentralized model training across
multiple devices.

• Key Contributions: The use of swarm optimization refines model parameters

efficiently, and federated learning ensures that user data remains secure.

• Limitations & Shortcomings:

o Federated learning requires significant communication overhead, which may

be problematic in areas with poor network connectivity.

o The paper does not address the potential risks of model drift over time as
user behavior changes.

8. "Real-life Boxing Activity Recognition with Smartphones Using Attention-Assisted

Deep Learning Models" by B. Jayakumar, N. Govindarajan

• Summary: This study applies attention-enhanced deep learning to classify complex

activities like boxing, using smartphone sensors. The model uses attention layers to
focus on relevant sensor features, improving recognition accuracy for high-intensity
activities.

• Key Contributions: The approach successfully distinguishes between nuanced, fast-

paced movements, making it ideal for sports analytics and fitness tracking.

• Limitations & Shortcomings:

o The model's performance may degrade when applied to activities with more
subtle or slower movements.

o The study focuses on a specific sport (boxing), limiting the findings'

applicability to broader HAR contexts.

9. "Temporal Convolutional Recurrent Neural Network for Elderly Activity

Recognition" by J.H. Ng, Y.H. Pang, S.R. Sekaran

• Summary: The authors develop a hybrid model that integrates Temporal

Convolutional Networks (TCNs) with RNNs to monitor elderly activities. The model
captures both short-term and long-term activity patterns, which is essential for
continuous monitoring.
 • Key Contributions: The combination of TCNs and RNNs ensures accurate
recognition of subtle, age-related movements, enhancing safety in assisted living
environments.

• Limitations & Shortcomings:

o The model may require frequent retraining to account for changes in an

individual's activity patterns over time.

o It lacks a thorough examination of the impact of sensor placement variability

on recognition performance.

10. "A Benchmark for Domain Adaptation and Generalization in Smartphone-Based

Human Activity Recognition" by O. Napoli, D. Duarte

• Summary: The paper provides a benchmark for evaluating HAR models' adaptability
and generalization across various domains. The authors compare several models,
focusing on their performance when tested in different environments and with
diverse populations.

• Key Contributions: The benchmark highlights the challenges of domain adaptation,

offering valuable insights and datasets for developing more robust HAR systems.

• Limitations & Shortcomings:

o The models evaluated show a significant drop in performance when applied

to new domains, emphasizing the need for more effective domain adaptation
strategies.

o The paper does not propose a concrete solution to address these adaptation
challenges, instead serving as a resource for future research.

3.Motivation:
The motivation behind studying Human Activity Recognition (HAR) with smartphone
sensors using machine learning lies in the growing demand for intelligent systems that can
monitor, interpret, and respond to human behavior in real time. As smartphones have
become ubiquitous, their embedded sensors, such as accelerometers, gyroscopes, and
magnetometers, provide a rich and accessible source of motion data. This abundance of
data opens up new possibilities for developing applications that can significantly impact
healthcare, fitness tracking, personal safety, and smart home automation.

1. Healthcare and Elderly Monitoring: One of the most compelling motivations is the
application of HAR in healthcare. For elderly individuals or patients with chronic
conditions, continuous monitoring of activities can provide critical information for
early detection of health issues or potential falls. This capability can enhance
independent living for the elderly, reduce healthcare costs, and provide peace of
mind to caregivers and family members.

2. Fitness and Lifestyle Applications: In the fitness industry, HAR can be used to
monitor exercise routines, track progress, and provide personalized coaching.
Accurate activity recognition allows for automated logging of workouts and better
insights into physical activity patterns, promoting healthier lifestyles and more
effective training programs.

3. Safety and Emergency Response: HAR can be leveraged for safety applications, such
as detecting abnormal behavior or emergencies. For example, automatic fall
detection using smartphone sensors can alert emergency contacts, potentially
saving lives in critical situations.

4. Smart Homes and Ambient Assisted Living: The integration of HAR with smart home
technologies enables the development of ambient assisted living systems. These
systems can automate household tasks based on the user's activity, enhancing
convenience and energy efficiency. For instance, lights could turn off when a user
lies down to sleep, or alarms could sound if unusual inactivity is detected, signaling
a potential medical issue.

5. Challenges with Traditional Methods: Traditional HAR approaches often rely on

handcrafted features and rule-based systems, which require domain expertise and
are limited in their ability to adapt to diverse and complex human activities.
Machine learning offers a more flexible and scalable solution, capable of learning
and generalizing patterns from raw sensor data, thus improving the accuracy and
robustness of activity recognition.

6. Advancements in Machine Learning: The rise of powerful machine learning

algorithms, especially in the domains of deep learning and ensemble learning, has
revolutionized how we approach HAR. These advancements provide the opportunity
to develop models that automatically extract features and handle the variability in
human motion, sensor noise, and changing environments, making HAR systems
more effective and reliable.
In summary, the motivation for this research is driven by the potential of machine learning-
enhanced HAR systems to improve quality of life, enhance safety, and create innovative
applications across various domains, all while leveraging the widespread availability of
smartphone technology. The goal is to harness the power of machine learning to build
systems that are accurate, efficient, and adaptable to the complexities of real-world
human behavior.

4.Objectives:
1.Feature extraction from raw sensor data and optimal feature selection.

2.Developing a robust machine learning model with higher accuracy.

3.Comparing the model with other existing models.

4.Building a compact system for real-time activity recognition.

5.Progress:
Smart phones have become a most useful tool in our daily life for communication with
advanced technology provided intelligent assistance to the user in their everyday activities.
The portable working framework with computing ability and interconnectivity, application
programming interfaces for executing outsiders’ tools and applications, mobile phones
have highlights such as cameras, GPS, web browsers so on., and implanted sensors such
as accelerometers and gyroscope which permits the improvement of applications in view of
client’s specific area, movement and context.

HAR Dataset from UCI dataset storehouse is utilized. This dataset is collected from 30
persons (referred as subjects in this dataset), performing different activities with a
smartphone to their waists. The data is recorded with the help of sensors (accelerometer
and Gyroscope) in that smartphone. This experiment was video recorded to label the data
manually.

This project is to build a model that predicts the human activities such as Walking,
Walking_Upstairs, Walking_Downstairs, Sitting, Standing and Laying.

Train and Test dataset was collected from HAR dataset from UCI dataset storehouse.There
were 561 features in datasets.

In the dataset, Y_labels are represented as numbers from 1 to 6 as their identifiers.

• WALKING as 1

• WALKING_UPSTAIRS as 2

• WALKING_DOWNSTAIRS as 3

• SITTING as 4

• STANDING as 5

• LAYING as

Train and Test ratio

30 subjects(volunteers) data is randomly split to 70% of the volunteers were taken
as training data and remaining 30% subjects’ recordings were taken for test data. e.g. 21
subjects for train and nine for test.
 • Data is almost balanced.

• Although there are fluctuations in the label counts, the labels are quite equally
distributed.

• Assuming the participants had to walk the same number of stairs upwards as well
as downwards and knowing the smartphones had a constant sampling rate, there
should be the same amount of datapoints for walking upstairs and downstairs.

• Disregarding the possibility of flawed data, the participants seem to walk roughly
10% faster downwards.

Machine Learning Models:

I used the 561 expert engineered features and we will apply classical Machine Learning
Model on top of it.

1.Logistic Regression

• Logistic regression is a linear model for classification. In this model, the

probabilities describing the possible outcomes of a single trial are modeled using a
logistic function. The logistic function is a sigmoid function, which takes any real
input and outputs a value between 0 and 1, and hence is ideal for classification.

When a model learns the training data too closely, it fails to fit new data or predict unseen
observations reliably. This condition is called overfitting and is countered, in one of many
ways, with ridge (L2) regularization. Ridge regularization penalizes model predictors if they
are too big, thus enforcing them to be small. This reduces model variance and avoids
overfitting

2.Decision Tree

• Decision trees is a hierarchical model also known as classification and regression trees.
They have the property of predicting response from data. The attributes of the decision
trees are mapped into nodes. The edges of the tree represent the possible output values.
Each branch of the tree represents a classification rule, from the root to the leaf node.

• This method has been used for several tasks in the field of pattern recognition and machine
learning as a predictive model. The main goal is to predict the next value given several input
variable.

Results:

ML model Accuracy Precision Recall F1 score

Logistic 0.9633525619273838 .97 0.98 0.99

regression

Decision tree 0.9514760773668137 0.92 0.99 0.96

6.Future Work:
1.feature extraction from raw sensor data and optimal feature selection.

2.Build more accurate model.

3.Build compact system and deploy in real-life application.

7.Conclusion:
In conclusion, the study of Human Activity Recognition (HAR) using smartphone sensors
and machine learning has significant implications for advancing technologies in
healthcare, fitness, safety, and smart environments. By leveraging the widespread
availability of smartphones and the power of machine learning algorithms, we can achieve
accurate and efficient recognition of human activities. This enables the development of
real-time, automated systems that improve user experiences and provide crucial
assistance in everyday life. However, challenges remain, such as handling sensor noise,
optimizing models for low-power devices, and ensuring robust performance across diverse
populations and environments. Continued research in this field will focus on enhancing
model efficiency, improving generalization, and integrating HAR systems seamlessly into
practical applications.