Design and Development of ML based Fitness

Trainer using Human Pose Estimation

Project Team

Sl. No. Reg. No. Student Name Department

1. 20ETEC004054 Siddhartha Hadimani ECE

2. 20ETEC004078 Yogesh B ECE

3. 19ETEC004016 Eshan Rao ECE

Supervisor: Dr. Punithavathi D

MAY – 2024

B-TECH 2020
FACULTY OF ENGINEERING AND TECHNOLOGY
M. S. RAMAIAH UNIVERSITY OF APPLIED SCIENCES
Bengaluru -560 054

3 ML based Fitness Trainer using Human Pose Estimation

1
 FACULTY OF ENGINEERING AND TECHNOLOGY

Certificate

This is to certify that the Group Project-2 titled “Design and Development of ML based
Fitness Trainer using Human Pose Estimation” is a bonafide record of the project work
carriedout by Mr. Siddhartha Hadimani, Yogesh B and Eshan Rao bearing Reg. No.
20ETEC004054, 20ETEC004078 and 19ETEC004016 Department of Electronic and
Communication Engineering – 2024 batch in partial fulfilment of requirements for
the award of B. Tech. Degree of M. S. Ramaiah University of Applied Sciences.
MAY - 2024

Supervisor: Dr. Punithavathi D

Associate Professor, Department of ECE

Dr. Punithavathi D Dr. J.V Desai

Head, Department of ECE Dean

3 ML based Fitness Trainer using Human Pose Estimation

1
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Declaration

Design and Development of ML based Fitness

Trainer using Human Pose Estimation

The project work is submitted in partial fulfilment of academic requirements for the award
of B. Tech. Degree in the Department of Electronics and communications engineering of
the Faculty of Engineering and Technology of Ramaiah University of Applied Sciences.
The project report submitted here with is a result of our own work and in conformance to
the guidelines on plagiarism as laid out in the University Student Handbook. All sections
of the text and results which have been obtained from other sources are fully referenced.
We understand that cheating and plagiarism constitute a breach of University regulations,
hence this project report has been passed through plagiarism check and the report has been
submitted to the supervisor.

Sl. No. Reg. No. Student Name Signature

1. 20ETEC004054 Siddhartha
Hadimani
2. 20ETEC004078 Yogesh B
3. 19ETEC004016 Eshan Rao

Date: 16th May 2024

ML based Fitness Trainer using Human Pose Estimation

3
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Acknowledgements

We take this opportunity to express our sincere gratitude and appreciation to Ramaiah
University of Applied Sciences, Bengaluru for providing us an opportunity to carry out
our group project work.

We would like to express our thanks and gratitude to Dr. J.V Desai, Dean
Ramaiah University of Applied Sciences for giving us theopportunity to study this
course and for his continuous support.

Most prominently, we would like to thank Dr. Punithavathi D, H.O.D., Electronics

and Communication Department, Faculty of Engineering and Technology, RUAS
for hisencouragement and support throughout the group project work.

With profound sense of appreciation, we acknowledge the guidance and support

extended by, Dr. Punithavathi D Assistant Professor, Electronics and
Communication Department, Faculty of Engineering & Technology, RUAS. Their
encouragement and invaluable technical support have helped us to complete our
group project successfully within the stipulated time.

We also thank all the faculty and supporting staff of the Electronics and Communication
Department, Faculty of Engineering & Technology who have helped me towards
successful completion of this dissertation work. Finally, we thank to our friends and
family for being with me in many issues during the project and their blessing have been a
constant source of inspiration for me.

ML based Fitness Trainer using Human Pose Estimation 6

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Summary

Human pose estimation, a cornerstone of modern technology, intersect with

fitness training and human-computer interaction, driving this project's innovation.
Leveraging real-time webcam feeds, our endeavor meticulously detects and
interprets human poses, supported by a comparative analysis of two primary
implementations: the classical approach and Google's MediaPipe framework.
The classical method integrates advanced computer vision algorithms, including
keypoint detection and pose estimation algorithms, meticulously applied to
webcam image frames. In contrast, MediaPipe, a Google-developed framework,
offers a sophisticated platform tailored for machine learning pipelines optimized
specifically for pose estimation tasks.

Our project meticulously integrates an array of tools and technologies. Blazepose,

a popular library, facilitates keypoint detection and skeletal pose estimation
from webcam feeds. TensorFlow, a versatile machine learning framework,
forms the backbone of our implementation, enabling seamless integration
with MediaPipe for advanced model training and inference. Furthermore,
we leverage Python programming language for its versatility and extensive
library support, enabling rapid development and prototyping. Additionally,
we utilize OpenCV for image processing tasks, ensuring robust preprocessing
of webcam feeds before pose estimation. Through exhaustive experimentation
and meticulous evaluation, our project showcases MediaPipe's superiority
in accuracy and computational efficiency. Its seamless integration of
advanced machine learning models and optimization strategies outpaces
the classical approach, setting a new standard for real-time human pose estimation.
This report presents a comprehensive exposition of human pose
estimation techniques, detailing the tools and technologies leveraged to
advance human-computer interaction and reshape fitness training paradigms.

ML based Fitness Trainer using Human Pose Estimation 6

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Contents

Contents
FACULTY OF ENGINEERING AND TECHNOLOGY……………….…...……......1
Declaration……………………………………………………………………………….3
Acknowledgements……………………………………………………………………………….6
Summary………………………………………………………………………………………….6
List of Figures……………………………………………………………………………………6
List of Tables……………………………………………………………………………………..6
Abbreviation and Acronyms………………………………………………………………….....6
CHAPTER 1:- Introduction and Motivation……………………………………………………8
1.1 Introduction……………………………………………………………………………8
1.2 Literature Survey………………………………………………………………………9
1.3 Motivation…………………………………………………………………………….9
CHAPTER 2:-Background Theory……………………………………………………………..11
2.1 Introduction…………………………………………………………………………...11
2.2 Frameworks and Libraries…………………………………………………………….13
2.3 Pose Detection Model (Dataset)………………………………………………….…...13
2.4 Program/Code and Explanation……………………………………………………….14
CHAPTER3:- Problem Definition………………………………………………………………30
3.1 Objectives………………………………………………………………………………………………………………..30
3.2 Methods and Methodologies…………………………………………………………………………………..31
3.3 Performance Metrics……………………………………………………………………………………………….32
CHAPTER4:-Problem Solving………………………………………………………………….33
4.1 Introduction…………………………………………………………………………...33
4.2 Block Diagram of Posture Correction System………………………………………..33
CHAPTER 5:- Results…………………………………………………………………………..35
5.1 Pose Detection Module………………………………………………………………..35
5.2 Bicep Curl Counter………………………………………………………………………………………….36
5.3 Squat Counter………………………………………………………………………...36
5.4 Pushup Counter………………………………………………………………....37
5.5 Hand Raise Counter…………………………………………………………….38

ML based Fitness Trainer using Human Pose Estimation 6

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

CHAPTER 6:- Conclusions and Suggestions for Future Work……………………………..39

6.1 Introduction………………………………………………………………………….39
6.2 Summary…………………………………………………………………………….39
6.3 Conclusion…………………………………………………………………………..39
6.4 Future Scope…………………………………………………………….…….39
Team Experience………………………………………………………………………...40
References……………………………………………………………………………….42

ML based Fitness Trainer using Human Pose Estimation 6

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

List of Figures

Figure 1.1: Pose Detection by Mediapipe

Figure 1.2: Pose Detection using Blazepose

Figure 1.3: OpenCV CNN process outlined

Figure 4.3: Block diagram of Poster Correction System

Figure 5.1: Result of Pose Detection Module

Figure 5.2: Result of Bicep Curl Counter

Figure 5.3: Result of Squat Counter

Figure 5.4: Result of Pushup Counter

Figure 5.5: Result of Hand Raise Counter

ML based Fitness Trainer using Human Pose Estimation 6

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

List of Tables

Table 3.1 Methods and Methodologies

Table 3.3 : Performance Metrics

ML based Fitness Trainer using Human Pose Estimation 6

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Abbreviation and Acronyms

CV: Computer Vision

ML: Machine Learning

DL: Deep Learning

HPE: Human Pose Estimation

2D-HPE: 2D Human Pose Estimation

3D-HPE: 3D Human Pose Estimation

RGB: Red Green Blue (color channels)

ROI: Region of Interest

CNN: Convolutional Neural Network

DNN: Deep Neural Network

API: Application Programming Interface

FPS: Frames Per Second

UI: User Interface

GUI: Graphical User Interface

IDE: Integrated Development Environment

GPU: Graphics Processing Unit

CPU: Central Processing Unit

IoT: Internet of Things

AR: Augmented Reality

MP: MediaPipe

ML based Fitness Trainer using Human Pose Estimation 6

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

CHAPTER 1:- Introduction and Motivation

1.1 Introduction
Human pose estimation emerges as a transformative force in the realm of fitness
journeys, revolutionizing the way individuals interact with technology during exercise
routines. As technology continues to advance, bridging the gap between humans and
machines, there is a growing reliance on computer-based devices to support various
aspects of our daily lives.

Human pose estimation holds immense promise for enhancing fitness experiences
by providing intuitive and personalized guidance. In fitness journeys, poses play a
vital role in guiding exercise movements and ensuring correct form and posture.
With the advent of human pose estimation technology, individuals can now receive
real-time feedback and assistance, akin to having a virtual fitness trainer by their side.
This technology not only enhances the effectiveness of workout routines but also
fosters a deeper connection between individuals and their fitness goals.

Moreover, human pose estimation technology caters to the diverse needs of fitness
Enthusiasts, including those wit varying levels of experience and physical abilities.
Whether performing static poses to maintain balance and stability or dynamic
movements to challenge agility and strength, the versatility of pose estimation
algorithms ensures adaptive and tailored guidance throughout the fitness journey.

In summary, human pose estimation represents a paradigm shift in the way

Individuals approach fitness exercises, offering unparalleled support and assistance
along their journey towards better health and well-being. By integrating pose
estimation technology into fitness routines, we pave the way for a future where
technology serves as a trusted ally in achieving fitness goals and promoting
overall wellness.

ML based Fitness Trainer using Human Pose Estimation 8

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

1.2 Literature Survey

Our project on the Design and Development of an ML-based Fitness Trainer using
Human Pose Estimation, we have meticulously reviewed a wide array of research
materials to build a foundation for our innovative approach. The survey encompasses
a thorough examination of various methodologies, algorithms, and technologies that
have been pivotal in the evolution of human pose estimation and it’s application in
fitness training.

Our literature review revealed several key studies that have significantly contributed
to the field:

• Detecting Gym Pose using Human Posture Recognition: This study

introduced a System that detects a user’s live exercise pose and provides
feedback on how to improve their form. It utilized TensorFlow, CNN, and
LSTM for exercise detection and OpenPose for body pose tracking.

• Pose Estimation for Fitness Tracker Using Machine Learning: This paper
discussed the challenges in human pose estimation and provided an overview of
considerable research work in pose estimation, including a machine learning
approach with OpenCV and MediaPipe.

• A Systematic Review of Recent Deep Learning Approaches for 3D

Human Pose Estimation: This comprehensive review focused on recent 3D
human pose estimation methods, highlighting the importance of monocular
images, videos and multi-view cameras.

1.3 Motivation

The development of an ML-based Fitness Trainer using Human Pose Estimation

stems rom a desire to revolutionize personal fitness training, making it more
accessible and effective.With a growing focus on health and the increasing demand
for home workout solutions, especially in light of global events such as the
pandemic, there's a clear need for a reliable and advanced fitness guide that can adapt
to individuals' schedules.

Traditional fitness approaches often involve expensive gym memberships, personal

trainers, and the necessity to exercise in specific locations. However, our project seeks
to democratize fitness training by bringing the expertise of a personal trainer directly
into users' homes.

ML based Fitness Trainer using Human Pose Estimation 9

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

By leveraging the advanced capabilities of MediaPipe’s BlazePose model, our system

offers precise, real-time feedback on users' exercise form, akin to having a personal
trainer available 24/7. Furthermore, our motivation extends to addressing the limitations
of existing fitness technologies. While wearable devices like fitness bands provide
some insights into workout routines, they lack the ability to offer detailed feedback
on exercise form.
Our ML-based Fitness Trainer bridges this gap by employing Human Pose Estimation
to analyze users' movements and provide corrective feedback, ensuring exercises
are performed correctly and safely.

ML based Fitness Trainer using Human Pose Estimation 10

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

CHAPTER 2:-Background Theory

2.1 Introduction

Human pose estimation for fitness aims to eliminate the need for trainers by
automating the process of identifying and tracking body positions and movements in
digital content.
Leveraging image processing and machine learning technologies, this approach
allows individuals to receive real-time feedback on their form and technique without
the presence of a human trainer. By providing accurate assessments and personalized
guidance, automated systems empower users to optimize their workout routines
independently, reducing reliance on traditional trainer-led sessions.

2.2 Frameworks and libraries

MediaPipe and OpenCV are two open-source frameworks that can be used
synergistically for pose estimation and control applications, introducing a seamless
fusion of real-time media processing and robust computer vision capabilities.
MediaPipe provides a versatile pipeline for processing live and streaming media,
encompassing both video and audio, and leverages machine learning models to unlock
intricate insights. On the other hand, OpenCV, a comprehensive library for computer
vision and image processing, empowers developers with tools for detection,
segmentation, tracking, and various image manipulations.
BlazePose, a module within MediaPipe's suite, is designed to detect and track multiple
body landmarks, providing insights into human poses. Specifically, BlazePose utilizes
a convolutional neural network (CNN) to process input images and generate a vector
of landmarks representing key body joints and limbs. This model's training draws from
a diverse dataset containing real-world and synthetic images, each meticulously
annotated with body landmarks. BlazePose typically detects and tracks around 33
body landmarks, allowing for comprehensive analysis of various poses and
movements.

The background theory of pose detection , image processing, and can be explained
as follows:

1. Pose Detection with MediaPipe:

ML based Fitness Trainer using Human Pose Estimation 11

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Figure 1.1 Pose detection with MediaPipe

MediaPipe is a versatile open-source framework that facilitates the development of

applications leveraging computer vision and machine learning. The pose detection
model within MediaPipe is designed to identify and track human body landmarks in
images or videos. This model can detect landmarks of human bodies, allowing for the
analysis of posture and categorization of movements.

Figure 1.2: Pose detection using Blazepose

ML based Fitness Trainer using Human Pose Estimation 12

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

2. OpenCV for Image Processing:

OpenCV is a popular open-source library for computer vision and image processing.
OpenCV is utilized for image processing tasks, such as capturing video frames from
a camera feed, converting images to grayscale, and applying filters to enhance the
quality of the image. It is also used for basic operations like thresholding and contour
detection. OpenCV provides various functions and methods to perform image
processing tasks, such as cv2.VideoCapture, cv2.cvtColor,
cv2.GaussianBlur, cv2.threshold, and cv2.findContours.

Figure 1.3 OpenCV CNN process outlined

2.3 Pose Detection Model (Dataset)

BlazePose is a popular pose detection model developed by Google. While there isn't
a specific dataset called "BlazePose dataset," Google has trained BlazePose using
various datasets for human pose estimation tasks. These datasets typically contain
images or videos with annotated human keypoints (e.g., keypoints for joints like
shoulders, elbows, wrists, hips, knees, and ankles).

ML based Fitness Trainer using Human Pose Estimation 13

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

2.4 Program/Code and Explanation

Pose Detection Module

ML based Fitness Trainer using Human Pose Estimation 14

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

ML based Fitness Trainer using Human Pose Estimation 15

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

This python code utilizes OpenCV and Mediapipe to perform pose detection on a
video.

Let’s break down the different components and functionalities of this script:

1. Import Libraries: The script starts by importing the necessary libraries:

• ‘cv2’: OpenCV, a popular library for computer vision tasks.

• ‘mediapipe’: A library by Google for building machine learning pipelines.
• ‘time’: Provides functions for working with time.
• ‘math’: Provides mathematical functions.

2. ‘poseDetector’ Class: This class encapsulates the functionality related to pose

detection.

• ‘init’ method: Initializes the pose detector object with various parameters
such as mode, model complexity, smoothness, enabling different segments,
detection confidence, and tracking confidence, and tracking confidence.
It also initializes MediaPipe’s pose detection module.
• ‘findPose’ method: Take an image as input, converts it to RGB format, and
processes it using Mediapipe’s pose detection module. It draws landmarks
and connections on the image if ‘draw’ parameter is set to ‘True’.
• ‘findPosition’ method: Find the position of landmarks on the detected pose.
It returns a list of landmark positions.
• ‘findAngle’ method: Calculates the angle between three specified landmarks.
It returns the calculated angle.

3. ‘main’ Function: The main function of the script.

• It initializes a video capture object from a video file(‘PoseVids/3.mp4’).

• It initializes the ‘poseDetector’ object.
• Inside the main loop:

▪ It reads frames from the video.

▪ It detects poses in the frame using the ‘findPose’ method.
▪ It finds landmark positions using the ‘findPosition’ method.
▪ If landmarks are detected, it prints the landmark list and draws a circle
around the 14th landmark( indexing starts from 0).
▪ It calculates the frames per second(fps) and displays it on the frame.

ML based Fitness Trainer using Human Pose Estimation 16

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

▪ It displays the frame with detected poses.

▪ It waits for a key press and exit the loop if a key is pressed.

4. Execution: The script runs the ‘main’ function if it’s directly executed as the main
Script.

Bicep Curl Counter:

ML based Fitness Trainer using Human Pose Estimation 17

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

This python script uses the OpenCV library to detect and track human poses, especially
focusing on counting bicep curls.

1. Importing Libraries: The code starts by importing necessary libraries such as ‘cv2’
(OpenCV), ‘numpy’ (for numerical operations), and ‘time’.

2. Importing PoseModule: It imports a custom module named ‘PoseModule’,

presumably containing functions related to pose detection and tracking.

3. Initialization Variables:

‘cap’: This variable is used to capture video frames. It can either capture from a video
file (‘PoseVids/g.mp4’) from a webcam(index ‘0’).
‘detector’: An instance of the ‘poseDetector’ class from the ‘PoseModule’.
‘count’: Initialized to keep track of the number of curls.
‘dir’: Used to track the direction of movement(up down) to count each curl correctly.
ML based Fitness Trainer using Human Pose Estimation 18
Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)
‘pTime’: Variable to store the previous time to calculate FPS(Frames Per Second).

4. Main Loop: The core of the program is in this infinite loop(‘while True’).

Inside this loop:

• It reads a frame from the video capture(‘cap.read()’).

• Resizes the frame to a standard size(1280x720).
• Detects and tracks poses in the frame using the ‘detector’ object.
• Calculates the angle of the right and left arms.
• Interpolates the angle to a percentage value (‘per’).
• Checks if the arm is fully extended(100% angle) or fully contracted
(0% angle) to count each curl.
• Updates the count and direction accordingly.
• Draws a visual representation of the curl count and angle percentage on the
frame.
• Calculates and displays the FPS.
• Displays the modified frame(‘img’) in a window named ‘Image’.
• Exits the loop if a key is pressed for a duration of 1 millisecond
(‘cv2.waitKey(1)’).

ML based Fitness Trainer using Human Pose Estimation 19

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Squat Counter

ML based Fitness Trainer using Human Pose Estimation 20

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

ML based Fitness Trainer using Human Pose Estimation 21

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

This Python script is for counting squats using a webcam feed and OpenCV library.
Let's break down the main components and functionalities:

1) Imports: The code imports necessary libraries such as cv2 for computer vision tasks
and PoseDetector from cvzone.PoseModule for detecting human poses.

2) Initialization: The code initializes a video capture object (cap) to capture video
from the default webcam. It also initializes a PoseDetector object with specified
detection and tracking confidence thresholds.

3) Angle Finder Class: This class is defined to calculate the angle between specific
body parts. It takes as input a list of detected landmarks (lmlist), and indices of six
points (p1 to p6). The angle() method calculates the angle between these points using
trigonometry.

4) Main Loop: Inside the main loop, the code continuously captures frames from the
webcam. It resizes the frame to a standard size.

5) Pose Detection: It uses the PoseDetector object to detect poses in the resized
frame. Landmark positions and bounding box information are obtained.

ML based Fitness Trainer using Human Pose Estimation 22

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)
6) Angle Calculation: The code calculates the angle of the left hand using the
angleFinder class. This angle is then used to determine the motion of the squat.

7) Squat Counting: Based on the angle of the left hand, the code counts the number
of squats performed. It detects when the hand angle crosses certain thresholds
(90 and 70 degrees) to increment the counter.

8) Visualization: It visualizes the counter on the screen by drawing rectangles and

putting text. It also draws rectangles indicating the position of the left hand.

9) Display: It displays the processed frame with annotations using cv2.imshow().

Pushup Counter

ML based Fitness Trainer using Human Pose Estimation 23

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

ML based Fitness Trainer using Human Pose Estimation 24

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

This code is a Python script that uses OpenCV and the cvzone library to count push-ups
from a video feed. Let's break down the main components:

1) Importing Libraries:

• cv2: OpenCV library for image and video processing.

• numpy: Library for numerical operations.
• cvzone: A library built on top of OpenCV for simplifying computer vision
tasks.
• PoseDetector from cvzone.PoseModule: A class for detecting and tracking
human poses in images or video frames.
• math: Python's built-in math library for mathematical operations.

2) Global Variables:

• counter: A variable to count the number of push-ups.

• direction: A variable to track the direction of movement (up or down) during
push-ups.

3) Video Capture:

• cap = cv2.VideoCapture('Pushups.mp4'): Initializes a video capture object to

read frames from the specified video file.

ML based Fitness Trainer using Human Pose Estimation 25

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

4) Pose Detection:

• pd = PoseDetector(trackCon=0.70, detectionCon=0.70): Initializes a

PoseDetector object with certain confidence thresholds for pose detection and
tracking.

5) Angle Calculation (Function angles()):

• This function calculates the angles between specific body landmarks to

determine the movement during push-ups.
• It takes a list of detected landmarks (lmlist) and indices of key points
representing the left and right arms during push-ups.
• Calculates the angles of the left and right arms relative to the torso.
• Interpolates these angles to map them to a range from 0 to 100 for better
visualization.
• Updates the push-up count (counter) based on the direction of movement.

6) Visualization:

• Draws circles at key points of the detected pose for visualization.

• Draws lines connecting the key points to represent body segments.
• Draws rectangles and text to visualize the push-up count and the angle of arm
movement.
• Changes the color of rectangles based on arm movement direction to indicate
correct or incorrect form.

7) Main Loop:

• Reads frames from the video feed (cap.read()).

• Resizes the frame for better visualization (cv2.resize()).
• Finds and detects poses in the frame (pd.findPose()).
• Extracts landmark points and bounding box information from the detected
poses (pd.findPosition()).
• Calls the angles() function to calculate angles and update the push-up count.
• Displays the annotated frame with push-up count and visual indicators
(cv2.imshow()).

ML based Fitness Trainer using Human Pose Estimation 26

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)
8) Exiting the Loop:

• Waits for a key press to exit the program (cv2.waitKey(1)).

Hand Raise

ML based Fitness Trainer using Human Pose Estimation 27

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

ML based Fitness Trainer using Human Pose Estimation 28

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

This code is for detecting and counting hand raises using computer vision with
OpenCV and the cvzone library. Let me break it down for you:

1) Importing Libraries: The code imports necessary libraries such as OpenCV and
cvzone, a library built on top of OpenCV for computer vision tasks.

2) Initializing Variables and Objects: It initializes the webcam (cap), a pose detector
(detector), and some variables for counting and tracking hand raises.

3) Angle Finder Class: This class is defined to calculate angles between specific
Points on the detected pose. It takes the pose landmarks (lmlist) and indices of points
(p1 to p6) as inputs along with a boolean drawPoints to determine whether to draw
points and lines on the image.

4) Angle Calculation: Inside the angle() method of the class, angles are calculated
using trigonometric functions (atan2) based on the landmark points provided. These
angles represent the orientation of the hands.

5) Main Loop: The main loop captures frames from the webcam, detects poses, and
calculates angles using the angleFinder class.

6) Counting Hand Raises: It counts the number of hand raises by detecting changes
in the angles of the hands. If both hands are raised (angles above a threshold), it
increments the counter.

7) Drawing Visualizations: Rectangles are drawn on the screen to visualize the

position of each hand. The color of the rectangles changes based on the angle of the
corresponding hand.

8) Displaying Results: The current count of hand raises is displayed on the screen.

9) Exiting the Program: The program exits when a key is pressed.

ML based Fitness Trainer using Human Pose Estimation 29

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

CHAPTER3:- Problem Definition

Title:

ML based Fitness Trainer using Human Pose Estimation.

Aim:

To Design and Develop an ML based Fitness Trainer using human pose estimation.

. 3.1 Objectives:

The project aims to achieve the following objectives:

• To conduct a literature survey on emerging fitness trends, the implementation of

ML in fitness technology.

• To develop a robust media acquisition system, integrated OpenCV for versatile

support.

• To implement pose estimation using Mediapipe, leverage BlazePose for

accurate 3D coordinates.

• To design and implement the pose estimation algorithm that compares the
detected develop with the ideal pose for a specific workout.

• To enable the system to provide real-time feedback to users based on the pose
comparison results.

• To optimize speed-accuracy balance, fine-tune pose estimation algorithms and

implement mechanism.

ML based Fitness Trainer using Human Pose Estimation 30

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

3.2 Methods and Methodologies:

Methods and Methodologies used to attain each objective are as follows:

Figure 3.2 Methods and Methodologies

ML based Fitness Trainer using Human Pose Estimation 31

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

3.3 Performance Metrics

Figure 3.3 Performance Metrics

ML based Fitness Trainer using Human Pose Estimation 32

Recognition based Control System for AR/VR Devicesm
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

CHAPTER4:-Problem Solving

4.1 Introduction

In the realm of fitness training, the fusion of machine learning and human pose
Estimation technologies has introduced in a new era of personalized guidance and
Feedback. Our exploration encompasses challenges such as accuracy in pose estimation,
robustness to environmental factors, personalization for diverse user needs, privacy
considerations, and optimizing user interfaces for seamless interaction.

4.2 Block Diagram of Poster Correction System

Figure 4.3 Block Diagram of Poster Correction System

ML based Fitness Trainer using Human Pose Estimation 33

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Here’s a breakdown of the process as depicted in the diagram:

a) Install and Setup: This is the initial where the necessary software and hardware
are installed and setup. This would be:

▪ Development Platform: Pycharm IDE, which is a popular

integrated development environment used for Python programming.
▪ Programming Language: Python, which is a widely used
high-level programming language, know for it’s easy Syntax
and broad library support.

b) Trained Model(mediapipe): This refers to a pre-trained machine learning

model specifically MediaPipe, which is a framework for building multimodal
applied machine learning pipelines.

c) ML Algorithms: These are the machine learning algorithms that the

system uses to process the data.

d) Camera(Image): This is the input to the system, which is an image or

video feed from a camera.

e) Dataset: The system likely uses a dataset to train the machine learning
algorithms. This dataset would contain examples of good and bad posture.

f) Make Detection: This step involves detecting the person and their posture
in the image or video feed.

g) Determine Joints: After detection, the system determines the positions

of various body joints.

h) Finding Angle: The system calculates the angles of various body parts to
assess the posture.

i) Posture Correction Display and Repetition: Finally, the system provides

feedback on the the user’s posture and posture corrections. The process
repeats as long as the system is running.

ML based Fitness Trainer using Human Pose Estimation 34

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

CHAPTER 5:- Results

5.1 POSE DETECTION MODULE

The pose detection module had successfully identified and marked the major joints
of the user’s body with colored dots, demonstrating its ability to recognize
key anatomical landmarks. The lines connecting these dots outlined the user’s
limbs, indicating that the module had effectively tracked the position and
movement of individual body parts. The clear overlay of dots and lines on the
user’s image suggested that the pose detection module had accurately and
precisely detected the position of each joint and limb.

Figure 5.1: Result of Pose Detection Module

ML based Fitness Trainer using Human Pose Estimation 35

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

5.2 BICEP CURL COUNTER

The bicep curl counter application successfully tracked and counted the user’s
bicep curl repetitions. The large green number ‘2’ indicates that two complete
bicep curls have been performed. The application provided real-time feedback
on the progress of the current repetition. The ‘87%’ displayed suggests that the
user was almost done with their third repetition.

Figure 5.2: Result of Bicep Curl Counter

5.3 SQUAT COUNTER

The squat counter application successfully identified and tracked the user’s
squatting motion. It marked the major joints of the user’s body with colored
dots and lines, demonstrating its ability to recognize key anatomical landmarks.
Each time the user moved from a standing position to a squatting position and
back, the application recognized this as a complete squat and incremented the
count. The updated count was then displayed to the user, providing real-time
tracking of the number of squats performed.

ML based Fitness Trainer using Human Pose Estimation 36

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Figure 5.3: Result of Squat Counter

5.4 PUSHUP COUNTER

Accurately counts push-ups by detecting full push-up cycles (arms fully extended to bent
and back to extended). Tracks and visualizes key points (shoulders, elbows, wrists) in
real-time on the video. Calculates and normalizes elbow angles to determine when a push-up
is completed. Displays the current push-up count, progress bars, and key point visualizations
on the video.

Figure 5.4 Result of Pushup Counter

ML based Fitness Trainer using Human Pose Estimation 37

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

5.5 HAND RAISE COUNTER

The program successfully detected hand movements in real-time using the PoseDetector
class from the cvzone module. It identified and tracked keypoints on the user's body. It
calculated the angles of the user's left and right hands using the angleFinder class to determine
if hands were raised or lowered. The program counted hand raises, incrementing a counter
each time the hands were raised above and then lowered below a certain angle. It provided
real-time visual feedback by displaying the hand raise count and hand angles on the screen,
using colored rectangles to represent the angles.

Figure 5.5 Result of Pushup Counter

ML based Fitness Trainer using Human Pose Estimation 38

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

CHAPTER 6:- Conclusions and Suggestions for Future Work

6.1 Introduction

This chapter presents a comprehensive approach to developing a machine

learning- based Fitness trainer, leveraging human pose estimation technology. It
delineates Strategies for addressing key components such as pose detection, bicep
curl counting, squat counting, pushup counting and Hand Raise. Each section
offers detailed solutions employing a range of techniques, from pose estimation
algorithm to pose-to-action mapping.

6.2 Summary

In summary, the problem-solving strategies for the ML-based Fitness Trainer

revolve around utilizing pose estimation for implementing adaptive algorithms
for precise exercise counting, and optimizing performance for real-time feedback.
For bicep curl, squat, and pushup counting, the focus is on developing
algorithms capable of recognizing and tracking specific movements with
high accuracy.

6.3 Conclusion

The outlined problem-solving approaches aim to create a fitness trainer system

that seamlessly interprets human movements and provides tailored feedback to
users. By amalgamating various machine learning techniques and algorithms,
this system endeavours to offer users and effective and intuitive means of
monitoring their exercise routines and ensuring proper form.

6.4 Future Scope

The future scope of this system involves further refinement and enhancement
of its individual components. Techniques such as reinforcement learning
could be explored to improve the adaptability and accuracy of exercise
counting algorithms. Additionally, integrating user-specific feedback
mechanisms and personalized workout recommendations could enhance the
overall user experience and effectiveness of the fitness trainer.

ML based Fitness Trainer using Human Pose Estimation 39

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

Team Experience

Collaborative Development:

In our ML-based fitness trainer project, collaboration was the cornerstone of

our success. We seamlessly integrated machine learning models with fitness
tracking technology to create a personalized workout experience. OpenCV
helped us analyze movement, while MediaPipe accurately tracked body poses.
AutoPy, NumPy, and PyCaw were instrumental in processing data and
providing real-time feedback. Through regular sync-ups, we ensured that each
team member's progress aligned, enabling us to merge code smoothly using
version control. By leveraging everyone's unique skills, we created a cohesive
system that revolutionized fitness training.

Problem-solving Approach:

Facing challenges head-on, our team embraced a collaborative problem-

solving approach. Brainstorming sessions became routine, fostering a
proactive mindset to tackle technical hurdles. By pooling our collective
intelligence, we swiftly addressed issues and refined our ML algorithms for
precise pose recognition. This approach not only enhanced the accuracy of our
fitness trainer but also pushed the boundaries of what was possible in
personalized workout experiences.

Code Review and Quality Assurance:

Maintaining code quality was essential to the success of our ML-based fitness
trainer. We conducted regular code reviews, focusing on consistency,
ML based Fitness Trainer using Human Pose Estimation 40
 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

readability, and adherence to best practices. This meticulous approach ensured

that our system operated flawlessly, providing users with a seamless fitness
training experience. Additionally, thorough quality assurance testing
guaranteed the reliability and accuracy of our ML models, reinforcing our
commitment to excellence.

Agile Methodology:

Adopting an agile methodology was crucial in navigating the dynamic

landscape of fitness technology. We embraced flexibility and adaptability,
responding to feedback and evolving requirements iteratively. Regular sprint
reviews allowed us to assess progress and make adjustments, ensuring that our
fitness trainer remained cutting-edge in the rapidly evolving AR/VR fitness
market.

Documentation:

Recognizing the importance of sustainability, we prioritized comprehensive

documentation throughout the project lifecycle. From codebase documentation
to system architecture, every aspect of our ML-based fitness trainer was
meticulously documented. This not only facilitated knowledge transfer within
the team but also ensured that future maintenance and updates could be
executed seamlessly, guaranteeing the longevity of our innovative solution.

ML based Fitness Trainer using Human Pose Estimation 41

 M.S.Ramaiah University of Applied Sciences – Faculty of Engineering and Technology (FET)

References

• Flores, Asia, et al. "Verum fitness: An AI powered mobile fitness safety and improvement
application." 2021 IEEE 33rd international conference on tools with Artificial Intelligence
(ICTAI). IEEE, 2021.

• Akpan, Aniebiet, and Ahmed Aldabbagh. "Remote Body Fitness Monitoring System with
Inter-User/Multi-user tracking Software Applications and Social Distancing Warning Sensor."
2020 2nd International Conference on Electrical, Control and Instrumentation Engineering
(ICICLE). IEEE, 2020.

• Taware, Gourangi, et al. "AI-based Workout Assistant and Fitness guide." 2022 6th
International Conference On Computing, Communication, Control And Automation
(ICCUBEA).IEEE,2022.

• Dsouza, Grandel, Deepak Maurya, and Anoop Patel. "Smart gym trainer using Human pose
estimation." 2020 IEEE International Conference for Innovation in Technology
(INOCON). IEEE, 2020.

• Kanase, Rahul Ravikant, et al. "Pose Estimation and Correcting Exercise Posture." ITM
Web of Conferences. Vol. 40. EDP Sciences, 2021.

• Bazarevsky, Valentin, et al. "Blazepose: On-device real-time body pose tracking." arXiv
preprint \\arXiv:2006.10204 (2020).

• Chen, Lvcai, Chunyan Yu, and Li Chen. "A multi- person pose estimation with
LSTM for video stream." 2019 3rd international conference on electronic
information technology and computer engineering (EITCE). IEEE, 2019.

• “Composite fields for human pose estimation” by S Kreiss,L Bertoni, and A Alah, IEEE
Conference on Computer Vision and Pattern Recognition pages 11977–11986, 2019. 1.

• M. Zolfaghari, G. L. Oliveira, N. Sedaghat, and T. Brox, “Chained multi-stream networks

exploiting pose, motion, and appearance for action classification and detection,”in
Proceedings of the IEEE International Conference on Computer Vision, 2017,
pp. 2904– 2913.

• M. Zago, M. Luzzago, T. Marangoni, M. De Cecco, M. Tarabini, and M. Galli, “3d tracking

of human motion using visual skeletonization and stereoscopic vision,” Frontiers in
bioengineering and biotechnology,vol. 8, p. 181, 2020.

ML based Fitness Trainer using Human Pose Estimation 42