Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas,
March 11, 2024
Content 1
Introduction Mediapipe open CV
METHODOLOGY Dataset Architecture
RESULT
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Introduction 2
▶ Sometimes, it’s hard to know if we’re doing exercises the right
way, especially if we don’t have someone to teach us, like a coach or a teacher. But guess what? There’s a cool thing called an AI Gym Trainer that can help us! It’s like having a smart friend who watches us while we exercise and tells us if we’re doing it right. This smart friend, or AI Gym Trainer, uses special technology called Robust Intelligent Posture Estimation. It’s like having a superpower that can see how we’re moving our body. It uses computer vision (like how a camera sees things) and machine learning (like how our brain learns new things) to understand how our body is moving during exercises. Then, it gives us feedback in real-time, which means right away, so we can adjust our posture and do the exercises correctly.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] introduction 3
▶ here are two special tools we use to help the program see your movements. ▶ One is called Mediapipe, and the other is called Open CV.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Introduction open CV 4
▶ Open CV is a popular computer vision library that provides a
range of functions for image and video processing. The library includes algorithms for feature detection, image filtering, and geometric transformations, which can be used for posture estimation(Open CV is like a magic tool that cleans up the pictures so the game can see your body even better.).
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] introduction Open CV 5
———————————– ▶ With the help of Mediapipe and OpenCV, the program can watch you move and figure out if you’re doing exercises right. It looks at the angles between different parts of your body, like your arms and legs, to see if you’re standing up straight or bending your knees correctly. ▶ Mediapipe framework can detect up to 33 human body landmarks with high accuracy. ▶ Open CV can calculate the angles between the detected landmarks and analyze the posture. It can also be used to remove background clutter and improve the accuracy of the landmark detection
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] introduction 6
▶ The system captures the user’s video feed
▶ and passes it through MediaPipe to detect the human body
landmarks. The detected landmarks are then passed to OpenCV to calculate the angles and estimate the posture.
▶ The system provides real-time feedback to the user on their
posture and suggests corrective measures.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] introduction 7
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Dataset 8
The proposed methodology is divided into four main
stages: 1. data collection 2. data preprocessing 3. pipeline training 4. and pipeline evaluation
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Dataset 9
The first and second stages of the methodology is: data
collection and preprocessing. ▶ In this stage: data of human postures is collected. The data is collected using a camera, such as a webcam or a smartphone camera, and includes different postures that are commonly used in fitness exercises. The data should include variations in lighting, camera angles, and postures to ensure that the pipeline is robust and can work in different conditions. The collected data is preprocessed to prepare it for pipeline training. The preprocessing steps include resizing the images to a fixed size, converting the images to grayscale, and applying data augmentation techniques, such as rotation, scaling, and flipping
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Dataset 10
The third stage of the methodology is: pipeline training.
▶ in This stage we taught the system to recognize how people move. We used "pipeline" which is (like a set of instructions for the system). We trained this pipeline using pictures that we prepared earlier with the help of Open CV. We wanted the system to learn from these pictures, so we showed it lots of examples. The system learned by looking at the differences between what it thought the posture was and what it actually was. The training process involves minimizing a loss function, which measures the difference between the predicted and actual human postures.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Dataset 11
The fourth stage of the methodology is: pipeline
evaluation ▶ in this stage: the trained pipeline is evaluated on a separate test data to measure its accuracy and robustness. The evaluation metrics used in this study include mean average precision (mAP), which measures the accuracy of the pipeline in detecting the human joints, and mean per-joint position error (MPJPE), which measures the accuracy of the pipeline in estimating the 3D pose of the human body.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] introduction 12
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Architecture 13
A. Installing and Importing the Dependencies
▶ Dependencies can be considered as the auxiliary code that a person wishes to call. Adding dependencies enables us to circumvent the tedious task of duplicating the piece of code that has already been designed and maintained. We’re using three main dependencies: MediaPipe, OpenCV, and NumPy. MediaPipe helps us analyze video data, OpenCV helps with the camera input, and NumPy helps with math calculations.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Architecture 14
B. Making Detections ▶ To detect body parts using Mediapipe, the first step is to capture or input an image or video to the framework. The framework then applies a deep learning model to detect the body parts in the image or video. The model uses machine learning algorithms to identify key points or landmarks on the human body, such as the eyes, nose, mouth, hands, and feet.Once the landmarks are detected, the framework can output their positions as a set of coordinates which we can use for different things, like measuring angles or tracking movement.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Architecture 15
▶ The process involved setting up a MediaPipe
▶ instance, recoloring the image into RGB, ▶ making detections, and then recoloring the image into BGR before rendering the detections. ▶ Landmarks were then drawn over the established major body parts to be used in the next module where the joints will be determined
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Architecture 16
C. Determining Joints ▶ First, we us Mediapipe to help us find important points on our bones "landmarks." For example, to find where our elbow joint is, we look for landmarks on our upper arm bone (humerus) and lower arm bones (ulna and radius). Once we find these landmarks, we can figure out where the elbow joint is. Mediapipe is really smart and can find lots of landmarks all over our body in real-time, like shoulders, elbows. Once we find these landmarks, we can use their positions to do more things, like tracking how they move or calculating angles between them. After we find all the important landmarks, we make sure they are correctly identified and labeled. Then, we check how well we can see the left shoulder, left elbow, and left wrist, because these are really important for us.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Architecture 17
D. Trigonometric Calculations ▶ Trigonometry helps us understand the relationships between angles and sides of triangles. That’s really handy for AI stuff like recognizing images, understanding language, and making robots move! To calculate these angles, we first need to know where the shoulder, elbow, and wrist are in a picture or video. We already found these points in an earlier step, so now we can use the data from there to calculate the angles. It’s like connecting the dots to understand how things are positioned!
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Architecture 18
E. Curl Counter ▶ we’re talking about "Curl Counter," which is a tool for tracking how many times someone does a specific exercise, like bicep curls, during their workout. how it works: First, the system needs to know where your arms, elbows, and shoulders are. Once it figures that out, it can start counting your curls. It does this by noticing when your elbow moves in a certain way during the curl, and it counts each time it sees that movement. The Curl Counter then shows you the number of curls you’ve done in real-time, usually on the screen while you’re working out. This helps you keep track of your progress and adjust your workout if needed.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] Methodology Architecture 19
▶ The Curl Counter can also collect data over time to show you how you’re improving. This can help you figure out what’s working and what you need to focus on during your workouts. the Curl Counter can be set up to show on the screen while you’re working out, and it’s programmed to count based on the angles it calculates from your movements. So, you can see your progress right there while you’re exercising!
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] RESULT 20
1. Comparison with YOLO v7:MediaPipe is more accurate than
YOLO v7 for single object detection. MediaPipe also performs better in terms of frames per second (FPS), especially on GPU, and can detect more key points (33 vs. 17) compared to YOLO v7.sty 2. Comparison with PoseNet and MoveNet:While PoseNet and MoveNet are faster, MediaPipe can detect more key points (33 vs. 17) and offers additional features like BlazePalm and BlazeHead detection. MediaPipe also has a higher FPS on GPU.sty 3. Comparison with OpenPose:MediaPipe outperforms OpenPose, especially in handling tracking, even though both can detect 17 key points.sty
In the final module, we demonstrated our completed project,
confirming that the framework works well in making detections, with an average accuracy (visibility) of 90%. AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, | Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1] References 21
[1] V. S. P. Bhamidipati, I. Saxena, D. Saisanthiya, and M. Retnadhas,
“Robust intelligent posture estimation for an ai gym trainer using mediapipe and opencv,” in 2023 International Conference on Networking and Communications (ICNWC). IEEE, 2023, pp. 1–7.
AVenkata Sai P Bhamidipati,Ishi Saxena, Mrs. D. Saisanthiya, Dr. Mervin Retnadhas, |
Robust Intelligent Posture Estimation for an AI Gym Trainer using Mediapipe and OpenCV [1]
