hello in this video I will teach you how

you can build this tennis analysis

project from scratch the project detects

and tracks the players as well as the

tennis balls across the whole video we

also detect the Court's key points to

know the player's position relative to

the court measure distances and even

determine if the ball is in or out we

will use the output of those models to

know how many times a player has shot

the ball and how fast he shot it we will

also be measuring the player's speed and

how many meters the player covered in

addition we'll be training two

convolutional neural networks in this

project the first one will be YOLO and

object detection models to detect the

fast moving tennis balls to enhance the

detection accuracy of the outof the-box

models the other one is a CNN that is

trained to estimate the Court's key

points and we'll be using pyto for that

so whether you're a beginner or an

experienced machine learning engineer

this project will totally make your

resume shine let's start the project I

have a project set up with one folder

called input videos this folder contains

one image and one video the image is a

screenshot of a tennis match and the

video is just a small bit of the same

tennis match we are going to use those

to develop and test our code our first

task is to detect the players and the

tennis ball with an object detector we

are going to use a popular object

detector model called

YOLO so the way that we're going to do

that is we're going to use a library

called Ultra litics so um we can just

pip install it first we can pip

install Ultra

litics and then run it it run very fast

on my machine because I have it already

uh but it might take a little bit more

time on your

end and then we're going to create a uh

a a file that is going to be called YOLO

inference this file we're going to be

playing around with yolo a little bit

we're going to see how it works and what

the output is and uh yeah see how ultral

litics work so to import YOLO we're just

going to do from ultral litics

import

YOLO and uh to import a YOLO model we

just do a model equal

YOLO and then provide what more what

model we want so here we want YOLO V8

which is the um the latest one that we

can use right

now so we have YOLO V 8X which is extra

extra large and to run it on um um on um

on an image we just do model Dot

model.

predict and give it the image path which

is going going to be called

input videos then slash

image.png which is going to be this

one um yeah we can also save it so we

can have save equal true and yeah that

is all for it so we can clear the output

here and just run it

it's going to take a little bit uh time

um yeah it's it's going to also download

the YOLO V8 model because it's not

readily available on my machine but in

the next run it will find it uh in my

local directory so it won't download it

again it will just download it the first

time um so it's just going to take a

little bit time and then uh run it on

the image okay now we're done so you're

going to find uh a new folder that was

created it is called runs and then

detect and then you have

predict um if you open predict you're

going to find the image with the

detections there are multiple detections

that you can find here which is multiple

people or players and then you're going

to have something like a tennis racket a

sports ball which is right behind the

tennis racket so you can uh actually see

it um you have the clock and um you have

uh you have also the uh the the name of

the class which is a person a sports

ball whatever and then followed by the

confidence score the confidence score is

a a number between zero and one tells

you how confident the model is that this

is um a person for example so the lower

the number is the the less the less

confident the the model is that this is

going to be actually what it is and it's

not going to be a false prediction

but yeah this is the output so far it

looks decent and U yeah the output like

the output for one object consists of

multiple things but let's focus on the

three things that we have right now

which is the bounding box the class name

and then the confidence scoree the

bounding box here is consistent of

mainly going to be four values um and it

can be in multiple uh ways to represent

it so we have for example um we can

represent it by the X Center and Y

Center which is the X position and the Y

position um of the um of the object

followed by the width and the height of

the uh of the uh of the

rectangle and we can also represented by

the um the far ends of the rectangle

like the it's called X minimum y minimum

then X maximum y maximum those two

points uh we're going to stick by the X

minimum y minimum then X maximum y

maximum format um just to stay

consistent then we have the uh class

name here right here and then we have

the

confidence um yeah we can also see those

uh outputs in a list format in the

output format just by creating um a

variable that stores the output we just

didn't store it before um and we can

just print it print the

result and we can also print the

bounding boxes because there are

bounding boxes inside the results so we

can have it we can have a look at it

also so we can have something like this

which is for box

in

um results of of 0 do

boxes then print

box uh this is going to print the boxes

um then we're going to run it

again it's going to take a little bit

time now it's done uh first you're going

to find that the runs have another

folder now which is called predict two

uh predict two has the latest prediction

that we did and it's going to be this

exactly the same thing so we won't have

to look at

it so looking at the result we have

first the boxes which is a boxes object

that we're going to look at

later um then we have the key points in

The Mask which is not important right

now those are not the for detection uh

those are for pose models and for

segmentation uh we're not going to use

it here so don't worry about them um we

also have the uh names which is um the

class names for example the ID zero

stands for person the id1 stands for

bicycle and so on and so

forth then we have the uh original image

and this is in pixel so we don't have to

worry about it um and yeah this is all

the main important stuff here then

followed by the boxes themselves the Box

themselves contains uh first of all the

class um here it's zero zero you can

scroll up and see that zero means person

so that's the class name uh but here's

the class ID so you can map it to the

class name easily followed by the

confidence

score and then you have the is track

which is going to be

tracking um tracking uh across frames in

a video uh we're going to tackle that um

when we predict on a video so I can show

you and and then we have the bounding

box themselves you can see different

types of formats here um but the one

that we're going to use is the X yxy

which is X minimum y minimum X maximum y

maximum and here's the X position uh of

the minimum one then the Y position then

the maximum X and then the maximum

Y and yeah you're going to find that we

have multiple boxes and and that is it

this is the uh results that we have so

in order for us to uh run it on a video

we can just replace the uh image path

with the video path like this input

video.

MP4 then we're going to run

it and it's going to take a little bit

more time because a video consists of

multiple images uh and in that case it

consists of 214 images so it's going to

run off those those images one at a time

um so it's going to take a while so I'll

be back when it

finishes okay so now we're done and we

have a predict three folder that we can

go check out so you can press detect you

can have the uh open the predict 3 and

then you have the output video so let's

open it

and you're going to find a very similar

output to the one that we have before

and yeah it is just running it again and

again over each uh frame in the video

but the one thing to note is that if you

look at the sports ball it's being

detected very low like the number of

times it's being detected in the whole

video is just maybe 10 um and this is

going to be very hard for us to analyze

the

the the motion of the ball and how it

goes and its speed and things like that

so we're going to need to uh fine-tune a

detector model to detect the ball a

little bit better so that for us to um

utilize this output a little bit

better so we can start by training and

fine-tuning um aolo model on the balls

on the tennis balls that are moving very

past so that we can detect it a little

bit better so the way that we're going

to do it is that maybe we're going to uh

do another uh folder called training and

inside this folder we can create a

notebook so we can call it um

tennis ball

detector

detector um training.

ipynb and then then you're going to have

a notebook set up first before like

jumping in we're going to need to find a

data set so that we can utilize and I

already found a data set on roof flow

and roof flow has a data set of very

similar images to the ones that we are

using in our um in our use case where

the there is an aial um photo of the

tennis court

and then the tennis ball is moving right

here and you also have a bounding box on

it uh by the way Robo flow also can give

you the ability to create your own data

set and it's done with very easily um

but yeah we're not going to annotate it

for us because it's already there and

you have 428 images in the training set

and 578 total images so this is is not a

big data set but it's going to be good

good enough for us uh for our use

case um so if you haven't already you

can just go create yourself a rof flow

account and then um I am going to use

YOLO V5 because it has the it got me the

best results in detecting the balls so

you can just click on

it make sure that you have the show

download code and um

yeah you can press

continue and it it gives you the code

that you're going to use to download the

uh data set so you just copy it and can

come back here

and

basically um like get data

set and you can paste it right

here so here the API key is is going to

be your key I just replaced it with a

string so that not to show you my key

and you can also like it also installs

the robo flow so we can install it right

here and I'm not going to train on this

machine I'm going to train in Google

collab so I'm just going to add also

install ultral

litics so you can install both and you

can download your data set right here

um so I want to show you the data set um

that is being downloaded I'm going to

run this uh now it's now it's being grun

uh and it's downloading the uh the data

set for us and yeah we're just going to

wait a little bit for it to finish so

now it's done and you're going to find

that a new folder has been created which

is called tennis ball detections -6 if

you open it you're going to have have uh

three folders which is training testing

and

validation and inside each one you're

going to find images and labels and if

you open the images you're going to find

the images that were shown before

and if you are going to open the

labels you're going to find the class

name which is uh going to be sorry the

class ID which is going to be zero

followed by the bounding boxes that we

have

so yeah to start training we're going to

need to Wrangle the data set a little

bit uh it's not going to be much uh but

the training code requires us to have

another folder inside this folder that

is called the the same name tenis ball

detection D6 and inside of it we're

going to

have training testing and

validation so in order to do this we're

just going to we can just do it manually

right here uh but I'm I do do want to

make the code reproducible so I'm going

to do it with code and I'm going to

import something called shuttle that is

going to help us move copy and paste

things around uh using

python so the way that we do it is that

we're going to write uh shuttle Das move

then give it the tennis B

detection then

train and we want to move

it here we want to move it again into

another folder like

this um so that way the the code for the

training code won't crash it it just uh

expects it to have it uh to have it in

this type of format so I'm just uh

abiding it uh so we can do the same for

the

test and for the

validation

you can run

it and it's going to finish anytime

soon okay so if you if you see the

results again so you're going to find

the tennis ball detections -6 then

tennis ball detections -6 and then

you're going to find the training and

testing and validation uh

folders uh to train the model it's going

to be simple with alter litics so the

way that we're going to do that is that

um we're just going to have um write

YOLO then uh mention that the task is

going to be detect which is going to be

uh detection because it has multiple

other things like key Point extraction

and uh segmentation so we're going to

specify that it's U only

detection and we can uh specify the mode

of it which is going to be training

we're not going to be detecting and

we're going to specify the model itself

which is going to be YOLO V5 and I'm

going to choose a big one which is going

to be uh this one.

PT and I'm going to also specify the

data set path which is going to be a

location uh Slash data

yaml and data set. location is from Robo

flow you're going to find it

here um uh so it's a data set and the

data set has a location attribute so

that's how you do it and you can also

specify the

epox to be as much as you want I'm going

to keep it 100 and you can also specify

the image size

um I'm going to specify to be a

640 and you can run it right here um and

uh yeah you will have the output uh but

I'm going to run it on Google cab

because it has uh free GPU so um I just

moved it just move the code right here

uh the same thing copy pasted it and uh

run the same thing it run the model it

trained it and then it produced another

folder called runs you open the runs you

have the tect you open the tect you have

the train and then you have

multiple um multiple metrics that you

can open up and see but the ones that

we're going to use is the weights uh

you're going to have the best weights

and the last weights download them both

like by pressing this download button I

already did um and yeah you just

download them then copy them and then uh

paste uh like paste them right here we

can create a new folder called

models and you're going to paste it

right here so let me uh let me go paste

them and come

back so I pasted them right here and

renamed them to YOLO V5 best and YOLO V5

last uh we're just going to use one of

them uh so for me I think the last was

doing a little bit better than the best

so uh but it might not be the case for

you so you can play around with both and

see which one works best for you um and

now we can use this model again for the

inference and see how it

goes so for the inference we can uh we

can use this model by specifying its

path so we can just do models SL

YOLO s YOLO 5-

last last.

PT and we can also specify the

confidence to be uh 0.2 I think it's uh

something very close to 0.2 right now

but yeah we can uh run it again on the

video and we can see the

results again this is going to take a

little bit time to run on all the video

so um I'll cut the video and come back

when it's

done so it's done now and we can go back

to our um folder to the output folder

that we have which is going to be runs

detect and then you have a predict for

and you can now see that the ball is

being detected way more than we had in

the

past um but you're going to also notice

that only the ball is being detected and

not the players not anything else that

is because we only trained on tennis

balls because that was the data set so

unfortunately right here we're going to

have two passes one with yolo V8 to

detect the players and one with yolo V

five the trained one to detect the

tennis balls and yeah but it's going to

be doable it's just going to take a

little bit more

time if we look at the previous YOLO V8

output video we will find that for each

frame we get a list of bounding boxes

and those bounding boxes are not the

same between frames as objects move from

one frame to another we want to have the

ability to match the player's bounding

boxes to the bounding boxes in the next

frame because right now we just have the

X and Y positions of each we have

nothing that explicitly states that this

is the same object between

frames this bounding box matching is

called object tracking where we say that

this bounding box is the same object as

this bounding box in the next frame this

way we can analyze the movement of the

same players across the video ultral

litics also gives us the ability to

track instead of predict so we we can do

track right

here and we can return it back to yov

V uh 8 and it was Yol V

8X run it it's going to take a while so

again I'm going to cut the video and

come back

later okay so now we're done uh we can

open it up again we can open the runs

detect and this time you're going to

find a track it's not going to be

predict it's going to be a track so if

you open it you're going to find that

each player has a different ID so this

is ID 1 and this is id2 and no matter

the frame that we are in we can always

say that this one is the player one and

this one is player two and this one are

persons with other IDs as

well um this is going to be very useful

for us and very helpful for us to all

always say or to always detect which is

which which person is which person with

ease um and yeah we can also track the

balls but since there's only one ball in

the field in uh in in most of the times

then we don't need to track the ball we

can just detect it and yeah we only know

that there's going to be only one so uh

no need for a Tracker right here and

before we proceed while we're training

stuff um in the in the final output

video I showed you that we also are

detecting the key points multiple key

points in the court uh those red key

points that you see right here uh those

multiple key points are going to be

helpful for us to detect how much a user

has um has covered in kilometers and how

much um and meters and how muches the

ball have traveled and it can also

detect whether the ball is in or out and

yeah it is very useful for us to detect

those key points um so the way that

we're going to do it is that we're going

to also find uh a data

set and I have found a repo that does

exactly the same thing which is uh

detecting multiple key points uh in the

same are view of the tennis court and

they also provided us with a data set

and they also have a model for it but

we're going to train our own so the uh

the data set right here they have it so

you can click on it and uh you can

download it easily from um from the uh

from the drive or we can just uh W get

it um

so the way that I'm did it is that I

used WG to download this so um you can

just copy this uh download WG function

uh WG command and we can write it on our

own

so to train this we're going to have

another um notebook that is called

tennis

court key

points

training.

ipynb and we can download our data

set

and this is going to download it in a

zip format so we will need to unzip it

so we can write unzip

tennis court de data set.

zip and then we can start uh with our

code so we can do a markdown right here

and say start

code uh for this code we're going to use

pyour so we're going to import

pytorch and we're we're going to also

import um the data set from pytor so

data set torch. utils do data

import

import data set and a data

loader and we can also

import um

the model itself which is

from
torch

Vision import models and we can also

import

Transformers

transforms uh which is going to help us

transform our data set a little bit so

you can run

this and

yeah uh first we can specify the device

that we're going to use so we have

device

equal um and you can see that it

autofilled so you can see Tor device

we're going to set it to Cuda if Cuda is

available which is uh GPU else we're

going to put

CPU and let's run it and afterwards

let's create our own uh torch data

set so the T the data set is where most

of our code is going to be here um and

the other things is going to be a little

bit easier so we can create a class for

this data set which is called key

points data set we can call it whatever

you want it's going to it's going to

inherit from the data set that we uh

imported in the

torch so it's going to inherit a lot of

the functions and uh uh things like that

and we can also plug it in into our um

training uh so the first thing is going

to make an initialized function that is

going to take the um image

directory as well as the data

file so before we proceed I wanted to

show you what data that we got by

downloading uh our data set so let me

move again to collab and you're going to

find that I here also use coolab for the

training so um you when when you

download the data set and you extract it

as well uh you're going to find that you

have

images it's going to take uh bit time a

bit of time to download to open those

images because we have a lot a lot more

than 500 the time um so you're going to

find the images are pretty similar to

what we have and what we saw before so

nothing interesting is here but you're

going to find also a Json file and this

Json file has the ground truth positions

of the uh

keypoints so first you're going to find

the

ID uh give it a minute it's just acting

up because it's a lot of data so you're

going to find an ID and um let an ID

which is the image ID and then the key

points KPS is the key points and you're

going to find 14 of them those 14 are

going to be X and Y positions so

actually in reality the model is

predicting 28 um 28 numbers it's not 14

numbers so you're going to find the X

and Y positions of each one and yeah

that is

it so given that we can jump back to the

code and create our own uh continue

creating our own data set so we can um

save the image directory here in the uh

self we can also open up the uh data

file which is open data file and we're

going to read the

data so uh it's predicted this wrong but

we can import

Json because it should it's in Json

format so we can just import it as a

dictionary do load and give it

f and yeah that is it so in the data set

you're going to find the dictionary that

I just showed you on the Google

collab and then we're going to uh write

um a

transformation um a transformation

function uh this one is going to

standardize the image

into uh the same size and it's going to

normalize it as well so you can write it

as

transforms

equal uh

transforms which is what we imported

from torch and we can

compose then we can give it a list of

Transformations that we want so the

first one is to transform it into um a

pil image do

to P image

because then we're going to resize it to

make it um 2 224 by

224 and then we we are going to uh make

this into a

tensor and lastly we're going to

normalize it so that the values can um

can be easier to train the values of the

pixels can be easier to

train the next function we want is to

define the length of the data set so you

can have the uh length of it it's just

the maybe the length of the data that we

have the the length of the dictionary

that we read

um and yeah afterwards we're going to

create a function that is going to get

um an item so in the training we are

going

to uh get items one by one or batch by

batch uh and then train it on this batch

and then um wait for the other one to

come so the way that we're going to do

this is we're going to do a function

called get item that's is going to to

take an

index and is going to return back the

image and the key

points so let's start by um getting the

annotations of it so we can say items.

index and

then um we want to get the image so we

can write

image uh

CV2 we're going to read the image and

read and then um we are going to specify

the

path so the path is going to be

self. image

directory then

slash then it's going to be

item do ID because if you remember we

have the the ID is going to be the same

name name as the

image and it's followed by

PNG now this will read in the image but

we don't have the CV2 imported so we can

import it using here Port

CV2 and then we want to get the height

and the width of the image so we can do

that easily by that

and we just can call the shape

function um and CB2 reads the image in a

BGR format but we want it to be in an

RGB format so we can have it like

this

CV2 do CVT color from RGB to

BGR

um afterwards we want to transform the

image by the Transformations that we

defined above so we can have this

self dot transforms and given the

image um then we want to have the key

points so the key points are going to be

item and we saw that it is in the

KPS um

key and yeah uh this one is going to

be um a list but uh for in order for us

to to to train it we need to make it

into a numpy

array um so we can import numpy at the

beginning then go back here and write

np. array and

and have it like this and also we

noticed that it was a 2d array so we

want it to have it only 1D so the way

that we do it is we make it um flatten

and flatten is just going to convert it

from 1D to
2D and at the end we just want it to be

float um so it's an array of floats

so we can

just specify the

type as np. FL

32 um now now the key points we

regarding that the image has the

original um size and it's not 224 by

224 so we would need to map the key

points to actually show this like to

actually be m map to the same um to the

same um positions after the resize so

the way that we're going to do that is

we're going to do like very simple cross

multiplication where I say if the where

the width where the original width was

for example the position

240 so if I made it uh 224 the width

what is going to be the

um what is going to be the new

location so the way that I'm going to do

it is that I'm going to multiply

224 by 240 divided by the width and this

cross multiplication will give us a the

end uh or end

position so you can do that by

specifying key

points and you can take the second uh

like

um yeah so you can take the first

element then the third element then The

Fifth Element so on and so forth so you

can have the width this this is how you

can do

it and the way that you're going to do

it is you can multiply it by

224

[Music]

um and then you divide by

width and this is to

adjust the x coordinates

um of the key points we will need to

also adjust the Y coordinates so we can

start off with one not zero and um and

divide by the height and not the

width and yeah and now we're done so you

can just return the

image and the key

points and your uh data set is ready so

we can

afterwards um we can specify the um we

can initialize this uh data set we can

specify the uh training and uh

validation uh data sets so you can do

like this uh

data set

equal key points let's run

this um so you can have the key points

data set you can specify the image

directory which is

images and then you can specify the Json

file which is going to be data

slash um and this data and this Json

file of course has the annotations so

it's um

train. Json and if you have any doubts

about those locations you can just go

back here and see it uh you can see that

the data has images and we have a

training Json and a validation J

Json and this is the training data set

so you can also have the validation data

set by doing this and specifying here

the uh Val

Json and yeah now you have your own data

sets just to use them in the um in the

training we need to put them into loader

so you're going to do that by writing it

like this and you can specify that data

loader give it the training data set

specify the batch size we can specify it

as eight here and we can also Shuffle

the same thing that we uh that we did

for training we can do for

validation and yeah and now we're

ready so now we want to uh now our data

set is ready we want to create our model

so to create our model we just need

um um so

yeah so to create our model um we're

going to initialize it first so we're

going to say that it's going to be

models and we're going to choose

reset

50 uh and we're going to choose the

pre-trained equal true now this is going

to uh use and download a pre-trained

convolutional neural network that was

trained on other

images and um this is this is going to

uh download the model and have it in the

U model uh um

variable uh but this reset 50 was not

trained for keypoint extraction so uh we

will need to replace the last layer to

reflect our own uh needs so this is

called like fine-tuning where you get

the reset 50 which is a a trained model

and you utilize most of its layers again

uh that was trained on another data set

uh on this

problem so the way that we're going to

do it is that we're going to specify the

last layer which is the fully connected

layer the last one and we're going to

say that this layer is going to be a

linear
layer which is just going to be a flat

layer

with

um just the features uh the input

features the the features that is going

to be inputed here and the output is

going to be 14 * 2 we will have 14 key

points and each key Point has an X and Y

which is going to be uh 28 at the end so

this just replaces the last layer of the

of the neural

network um so now our model is done uh

the next thing is that we want

to move the model to the uh correct

device so whether we have uh this device

is in Cuda or CPU we want to move this

to Cuda or

CPU and the last step is actually

training the model so we

can so we can start training the

model so first thing is that we want to

define a loss function and since this is

going to be a regression uh function

then we're going to use uh mean squared

error loss so we're going to have the

Criterion set to uh torch Ms

eloss uh then we're going to specify

also the the

optimizer and you can specify the

optimizer to be whatever you want um I I

chose Adam and I am going to also um

choose learning rate of 1

E4 now we can have as much EPO as we

want so I'm going to choose only

20 and we're going to Loop over each

Epoch in range

EPO and I am going to get the image

and key

points um again I'm I'm not running it

locally because I'm going to run it on

the um uh on the Google collab but the

rest of the tutorial I'm going to be

running locally so that uh you can see

things step by

step you can just say enumerate which is

going to return the index and we can

have the train

loader

and yeah the train loader is going to

return back the image and the key points

so we are going to have

the it's images it's multiple images so

you can have it the images to device we

can move it to the device we can also

move the key points to the device that

we want we then want to flush out the

optimizer so that it has um like to

flush show the uh

gradients um and then we want to predict

uh the

um we want to predict the model uh the

output uh like we want to use the model

to predict the key points on the images

that is

provided and after we predict it we want

to um U like uh calculate the loss which

is going to be like this we give it the

outputs and we give it the key points

and it's going to use the mean squared

error loss to calculate our

loss and at the end we want to do a

backward

propagation and we want to take a step

an Optimizer step just to do a learning

step and yeah and just for logging

purposes we can log every uh 10 steps

which is going to be like this which is

if I is divisible by 10 then we're going

to uh print out the epoch we're going to

print out the I and we're also going to

print out the

loss so you can go ahead in your uh

Google collab or if you want to train it

locally train it locally um and run it

so you can you can see that it ran and

the loss was very high now it's

decreasing a little by little and at the

end of the day you're going to need to

save it so after you run it uh you'll

need to uh save it in and download it so

so to save it we just going to run torch

Dove then give it the

model dot State dict

and you can uh specify the output path

of it so you can have the key

points

model.

pth and you run it and save it so I'm

running it in collab so you can save it

and then you can download it like we

downloaded the um YOLO model and then

you can come paste it

again um in the models directory and

paste

it uh now we have the other model

ready so now that we're done and our

models ready to be utilized we can close

down all this we can save it you can

download you can close all this

up and now we can start uh coding our

tennis analyzer

so we can just specify the main and

write it right

here and we can have the main

function and we can do a print word

right now and then we can call the

main so after we set up the main. pi uh

we just want to start with the uh

basically reading the video because we

won't be able to we we don't want to run

run it like that um we basically want to

run it

um um like frame by

frame uh so yeah we're going to need to

read in the video as frames and then

have the ability to also save it as

frames

uh so let's start with that uh we can

create a new folder called

utils and inside the utils uh we're

going to need to create a a folder a

file called video

TS okay

uh inside the video Tils we're going to

import

CV2 because this is the way that we're

going to read and save our videos we're

going to create a function called read

video it takes a video path um and then

it um opens um like it it opens a

connection to the video and right now we

have an empty uh list with frames

and we Loop uh and we're going to Loop

over this uh till the frames are done so

what we do is that we use c. read to

read in the frame and then we have the

return this return is going to be false

when the when the there is no more

frames to read so if there is so if it

is actually false we're going to break

this Loop otherwise we're going to

append the frame to the list of frames

and then at the end of the day we're

going to release it and return back the

frames um we are also going to want to

have a way to save the video so that we

can see what we do so we can Define save

uh

video that takes an

output

video

frames and it takes in an output video

path and it's going to save this as um

um it's going to yeah it's going to save

this as a video so yeah I'm going to

utilize what uh GitHub co-pilot has

suggested but I'm going to edit it a

little bit so first what the format that

we want is going to

be

MJ PG

and we want to Output it as a 24 frames

per

second and yeah that is it I'm going I'm

just going to remove that as

print and yeah this is it um in order to

expose those functions to the outside uh

to to outside of this folder um we need

to create a new file called init.py

and whips I just need to create replace

this with a DOT and in the init.py we

just like um mention all the functions

that we want to have um available

outside um outside the util so what

functions we want to expose outside the

utils and we write it here so we can

write from video utils we can import or

read video and save

video um and after we read and save

video we can go right here write

from

utils import read video and save video

we can also put it in Brackets and

because we're going to have a lot more

functions in the utils so make it neat

and organize from

now and yeah we can uh then uh read in

the data set read in the the video sorry

so we can specify the input video path

input

video path is equal to input videos.

input video.

MP4 then we can readin the video frames

by using the function that we have from

the

utils and afterwards

um we want to save the video so we can

have actually we can create a folder

called output videos that is going to

have our output um inside of it so we

can have the save video uh function that

we also have in the um

yours uh you can uh give it the video

frames and you can also create the

give it the output path so uh yeah let's

run this and see if it

works so there is an issue with the

output format so let me go and check it

real

quick so yeah this issue was because I

wrote MP4 at the end and it should

beavi so if we run it again you

shouldn't get this

error and yeah so you can open it up

here in the output videos and you're

going to see the same video

again uh outputed to

us okay uh that's set up that is all set

now so we can start defining our uh

trackers basically so you can create a

new folder called

trackers inside of it you can create a

file called called player tracker.

py and inside this player tracker we're

going to create the uh tracking for the

players so uh we're going to import YOLO

so from ultral litics import

YOLO and we're going to specify the

class which is player uh

tracker

we're going to specify the uh init

function and it's going to take in the

model path which is what model are we

going to use

and yeah and we can just do self. model

equal YOLO model

path and to detect on a frame uh we can

use the same code that we did earlier so

we can have Define

detect frame and we can just do the same

code that we have earlier so we're going

to take a frame which is basically an

image we have the uh results from it and

then we run the model on it so model.

track because we want the players to

retract here and we give it the frame

and we also tell it to P like the

persist function we're going to give it

true now persist here it tells the model

that the tracks that this is not going

to be um just one individual frame I'm

going to give you another frame

afterwards and you should be able to

persist the tracks within those

frames um because we're not giving the

whole video at one time so we need to uh

set this up so that it can remember the

tracks that were done

before um then uh we're just going to uh

need to have this um map of lists um

like this map of IDs to names so um it

is in the results so you can have the

results do names and it will give you

the results we're going to have the

output in a dictionary where the key is

going to be the player ID and the output

is going to be the bounding box and

we're going to Loop over each bounding

box in the results and we want to only

choose the bounding box that are uh

people that have a person inside of it

we want to exclude um clocks we want to

exclude rackets we want to exclude

everything except people um and we're

going to have another tracker for the

balls so we will exclude that also here

um so yeah we are going to take in the

zero with option because it's only one

image and we going to run

results of

boxes and we are going to get the track

ID and this is the number that you saw

on top of the uh people uh in in the

track example that I gave you before

we're going to make it as an integer

then we going to take box uh. id. to

list and this is the Tracker ID and we

want to also have the bounding box so we

can also do a result which is one result

um we can say that box and we want it to

have it in the xyxy format uh which is X

minimum y minimum X y maximum y maximum

and we're going to keep that consistent

throughout this

video and the uh we also want to have

the object class

name so the class ID so we can say

object CLS ID as box do

CLS and. to

list um if I can write list

correctly and we can also have the zero

one um and we want to map this ID to the

name so that we know what we're looking

at so we can have the

object class name as ID to name of

object

ID and we can simply say if the object

uh name is equal to person then we are

going to uh take it we're going to have

the track ID and then put uh the value

like the key as the track ID and the

value as the bounding box and if it's

not a person then we're not going to do

anything we're not going to take this so

we are going at the end to return this

um and have uh and and get the results

of one

frame we want to also create a function

that detects multiple frames so we can

have detect frames and this is going to

have the self and it's going to take

frames it's also going to have um it's

it's going to utilize the detect frame

function so what we do is we have the

player

detections and for frame in frames

and we want to do this we want to detect

each frame and then put it right here

and append it to the output to the

output

list and now we are done so you can have

this player tracker uh ready for use and

it will return back the list of like the

list of detections for each frame

and to also expose it outside this

folder we need to have the

init the init

one uh init file and you can write

from

player tracker import player

tracker you can put a dot right here

so um

afterwards we can detect from we can

import it so from trackers import player

tracker so we can we can initialize the

player tracker first so we can have the

uh

player

tracker as

player tracker and we give it

the YOLO V8 X1 as the model

path and yeah now we initialized it so

let's use it to get the detections so

player detections is equal

to player. detect frames and we give it

the video

frames and that should be it so here we

are reading the video Read

video and here we

are

detecting

players okay so now we're ready um but

the video won't have the uh detections

on top of it so we will need to create a

function that uh draws the bounding

boxes on top of the video frames so the

way that we're going to do that is that

we're going to add a new function into

the players uh tracker that is going to

add those um drawings on top of the

frames so you can go back again to the

player tracker then go back here

whoops and uh you can have a function

called Define draw Bo boxes BB boxes

bounding

boxes so it takes in the video

frames um and it takes in the player

detections um so uh we are also going to

have the output frames as an empty

list let's call it output video

frames and I am going to Loop over the

frame and the uh output dictionary so we

can have the frame then player dict IN

Zip and give it the video

frames and the player

detections zip allows us to Loop over

two lists at the same time so that we

don't have to keep an index of them um

and then we just want to draw the the

bounding boxes so here we are going to

draw the bounding

boxes so we are going to Loop over each

track

ID and BB box in the player dect

items and let's import CB2

and see how it goes so the first thing

we extract the X1 y1 X2 Y2 from the

bounding

box and then we draw a

rectangle that has the like the corners

of it which is going to be X1 y1 and the

other corner is going to be X2

Y2 um and we are going to specify a

color um and we can make this

color uh right here um so this is an an

RGB

format uh so uh yeah and yeah the two

here means that it's not going to be

filled it's just going to be the outside

uh borders of

it um we also want to put in some text

thatth tells us this is the player and

what id is it so we can

just uh do something like this

CB2 and then uh put

text we give it the

frame and then we give it

the string that we want to print out so

we have the player

ID and then we want to put in the track

ID uh we also want to have the position

of it so the position is going to be int

of the BB box of zero which is going to

be the minimum and it's also going to be

the end of the BB box of one which is

going to be

minimum y so this is minimum X and this

is minimum

Y and we want to also have a little bit

of a buffer so let's do it like uh minus

10 and yeah uh we can also specify

the um the font of it and the the font

size and the color of it as well so we

can have this and let's increase the

size a little bit so to

0.9 and this is and this should be it so

um and yeah you can also assign it right

here like this one you can assign it

frame equals frame but it actually

writes on top of the frame so we don't

have to do it so we can do this like we

can do it like this and afterwards we

just out put this again as a frame like

append it to the output and then at the

end of the function we return the output

frames um so yeah let's

also uh return back to the

main um use the player tracker right

here so we are going

to uh be drawing some things so uh draw

output and the first thing that we are

are going to draw is

draw player bounding

boxes and we are going

to

draw uh the bounding boxes give it the

video frames and give it the player

detections uh the video frames right

here we are going

to uh yeah we can call it here the

output uh video

frames and then save it again so yeah

let's run it h there is an error uh

trackers do no modules called it's

called like that so trackers do player

tracker

um

ah whoops I misspelled it right here so

it's

tracker um you can clear the output and

run it

again so it is going to take a while to

run this all um so yeah I am going next

to create um a stub where we can save

the output of this tracker uh into a

file and then read it instead of

predicting it again and again and it

would help us in development it will

help speed up the development a little

bit but uh yeah this is going to be the

next step so uh yeah I'm going to cut

the video for now and come back when the

run is

over

so I've come into an error right here

and it seems to be that I have missed

the bracket so if you did the same

mistake uh just come here at

the add the bracket for the int and make

sure that your brackets are

correct and
yeah I think I'm missing one so yeah

right here

and so everything is good

now um so yeah you can uh run it

again and yeah I'll come back when it's

uh when it's done

running okay so now it's done so let's

go back to our output videos and open

this and you're going to find the output

very similar to what we had uh in our uh

in our ultral litics output which is

going to be the player and then the ID I

didn't put the confidence because I

don't care about it right now I just

care about the IDS and the bounding

boxes so it all looks good now I just

want to add another thing in the player

trackers which is going to be uh basic

basically uh saving the output so that

we don't have when each time when we run

we don't have to run the detector again

we can just uh take it take the output

if there is any so we can create um a

folder called tracker

stops

tracker stops like this uh

whips

and we are going to be saving the output

of this uh detect frames um in a pickle

format so let's import

pickle and right here in the detect

frames uh

function uh we can

take two more inputs which is going to

be

read from stop it's going to be either

true or false um if it's false then

we're going to run it if it's true then

we're going to uh put uh put it into a

stop then we are going to take a stop

path and uh this stop path is going to

be none as

default um so yeah if

then if we have

the

um if we have the stop

path then if we have the if we have the

stop path which is going to

be if stop path uh is not

none uh then we are going to save it

basically so we are going to open it and

then uh pickle dump the player detection

uh inside this

path um we are also going to check if

the read from stop is actually true and

if it is um we are going to read the

Stop and not uh basically detect each

frame so we are going to check if read

from
stop is true um and the stop path is not

equal to none um then we are going to

open it then load the pickle and then

return the player

detections and yeah this is it right now

we want to add those two in so in the

detect frames we are going to give it

two more things uh we are going to read

from stub is equal to false right now

we're going to make it false because we

don't have a St and we're going to

generate it after this and the other one

is going to be stop

path and we are going

to

tracker specify the output one so

specify the output path which is going

to be tracker

stops

slash

then player

detections do pkl which is the pickle

format pickle extension so right now

we're going to run it again and it is

going to detect the frames but

afterwards it's going to save it in the

stops so that we don't have to run it

again uh so let's run it and we're going

to have to wait a while so I'm going to

cut the video again but hopefully this

is the uh last time that we're going to

detect it um with the players we're

going to do another one with the balls

but we're also going to run it one time

and then the Run should be a little bit

faster afterwards so yeah see you

then okay so now it's done and you you

can find them in the tracker stobs you

can find the player detection. pickle so

all what so all we do right now is just

change this to true and if we run it

again then you are going to find that it

takes way less time so um give it just a

couple of seconds and it's done so you

can go also to the output folder and

every time it it overwrites the file and

you can see that we have the same out

it uh now we want to do the same thing

uh the same like we want to track the

ball so you can copy it then uh copy the

player tracker and uh then you can paste

it right

in after pasting it you can uh rename it

to a ball

tracker like

that and after you name the ball tracker

you can uh rename also the

uh the class

name and we're going to change the

detect frame a little bit so the detect

frame right here uh

basically um is checking for the if

class name is equal to person and since

that we have only one class then we can

delete this if function so we we can

delete this uh if

statement and we can also delete this uh

checking for the names and we can also

delete this because it's not used

anymore and yeah we can rename the

player dict to Bal

dict and let's also have it right here

and H it here and let's not track it

since we only have one ball so we can

just predict it

we can set the confidence to uh 0 uh uh

like 15 uh I think that is um that was

good enough uh and 0.2 is also working

fine

um so uh we're going to we're not going

to have any tracks so we're going to

have only one ball but we want to have

the output consistent so we're going to

have um just one which is track one and

put at it the result

and yeah we also want to have um this

detect frames to uh to stay the same but

we don't want to have it to call to be

called player detections we can call it

ball

detections and rename each instance of

this and now we're done so the last

thing that we are going to need to

change is the draw

boxes um so the draw boxes right here is

going to be ball

ID and we can also call it here ball

dict I don't think that we're going to

change anything else than the

naming um but yeah that is

it uh yeah if I didn't miss anything I

think this is

it for yeah this is it for this class so

we can go here we can also import it in

the player tracker we can import ball

tracker we can go back here and we can

import ball tracker as

well let's um um so yeah let's detect

now the uh balls so we can call detect

players and ball

we can also have a

ball

tracker which is going to be ball

tracker and we are going to give it this

time the model path to be

models and YOLO 5

Dash um slash

last do
pt and we are going to detect also the

frames so like we did in the players

we're going to do with the uh ball as

well so we're going to call it ball

detections we are going to read from the

stop if it exists if it doesn't then we

read it um then we have the uh we can

call it the ball detections and save it

as

well and yeah afterwards in the drawing

the outputs we can also draw the uh ball

output like this and give it the uh ball

detections and yeah that should be it uh

we can now run it we'll have to wait a

little bit for the ball tracker to run

the first time um but yeah I'll come

back when it's

finished yeah before we I run into an

error that tells me that this doesn't

exist so you can change it to read stop

from false till we have the first

initial uh stop so I'm running into

another error again so let me uh figure

it out and come back to you

afterwards uh so I'm back and I have

this prectical true this is uh for only

tracking so make sure to remove it and

uh save it again

and run it and hopefully this time it

won't crash
again um just give it a minute make sure

that it's running and then I'm going to

cut the video if it is yeah it is

running so I'm going to come back when

it's finished um so yeah see you

then so now it's finished and we can go

back again to the output video now see

it and you can see that the ball is

being uh tracked as well uh so let's

give the ball another color than red so

that we can differentiate it quite

easily and we can uh do this by going to

the ball trackers then go to draw boxes

and give it here in the RGB section you

can just give it another color and I am

going to choose a yellowish color like

this and this time I'm not going to uh

I'm going to use the uh stubs so that I

don't have to run it again so like

this and you can just run it again and

yeah this time it should be a very fast

run so I'm not going to cut the video um

yeah it's done

already so you can now see the results

the ball is in uh yellow you can also

change the writing to

yellow uh so like here you can also

change it to be yellow and

yeah that is it let's now detect the uh

Court key points um so in order to do

that let's also create another folder

that is going to be called

Court line

detector and we are going to have um a

file inside of it that is called

Court line

detector. py so this is going to read in

the model that we have trained and then

use it for

prediction so in order to do that we are

going to import uh

torch because the model is written in

torch and we are also going to import

torch

uh

vision and uh we are going to

specifically import from it the

transforms uh to apply the same

Transformations that we did earlier on

this uh uh on the images uh we are going

to also import CV2 to read in the images

if we want and to do any uh thing uh any

uh uh any manipulations if we want and

yeah let's start uh writing the uh

class so we can have the uh whs we can

write the

cour line

detector and we can start off with the

init
function and we are going to take the

model path as an input

also

um so let me just fix that

and you can have the self model

equal models do uh we're going to do the

same one so we're going to have the uh

models. reset and I did not import it so

I need to import it first um and we are

also going to replace the last fully

connected layer with uh the one that um

that is going to have the uh

28 uh key points

um so FC is equal

to um

torch do

NN do linear and then it's going to be

self. model. fully

connected uh in features and then have

the 14 *

2 we can just write 28 if we want but I

just write this to remind myself that

it's 14 key points and with uh with X

and Y um then to load in the model I'm

just going to do the self. model. load

State dict uh we are going to load in

the model and I am going to map it to uh

look uh the CPU because because right

now I have a CPU machine and I don't

have a GPU inside of it so I'm just

going to map it to a

CPU so now we have created the model and

we have loaded the uh weights of the

model inside this model so that we can

have the same uh output that we uh like

we can get the predictions of the train

model that we have um the last thing

that we are going to do in the init is

going to specify the transform function

which is going to be the same thing that

we did so you can actually go copy and

paste it so you can have it right

here if you have it ready you can just

copy paste it if you want uh like this

and it should be uh fine um then we are

going to create the predict function

it's going to take in the image which is

a frame um we are going to basically

predict the first image and not predict

it again because the camera is not

moving so the key points won't change

positions so we are just going to uh run

it on the first frame and that's

it

um so what we do is that we have the

image uh we just make sure that it's in

RGB so we can just convert it from BGR

to

RGB and then we um apply the

Transformations on

the uh on the image so we just do the

self. transform and then give it the

image

RGB and we want it to have uh like we

want to UNS squeeze uh UNS squeeze it

and yeah we we then want to run the

model on it and the way that we do that

is that we do it like

this

outputs equals self do model and then

give it the image

um and you have the key points right now

in the outputs so you can have the

outputs

dot

squeeze uh then uh two list by the way

UNS squeezed right here just means that

if you have the image like

this um you just put another list on top

of it so it's it's going to be like this

uh this is uh on squeeze uh this is

essential because um when a model tries

to predict things it it usually takes in

a list of things to predict on and if

you are just predicting on one thing

like this one image uh you need to put

it into a list and the output is also

going to be a list so uh we also want to

have the zero um like you want to have

the zero position of it you can just

write dot do zero or you just can

squeeze it and by squeezing it it just

removes the list

so back to the key points thing where we

squeezed

it um we can uh move it to

CPU and then we can make it into a

numpy like

this and um we can also have the um like

like the key points right now are um are

the positions for the 224 by

224 so we need to map it again to the um

to the original image size and image

width so the way that we're going to do

that is that we're going to have the

original height and

original

width and we can leave it like this or

we can just take the first two

and uh do it like

that um so the way that we did it when

training is that we did it with cross

multiplication and this is how we are

going to do it again right here so we

are going to get the key

points then get all the width of the key

point so every uh every other element uh

starting from zero um then we are going

to multiply it by original. height over

2440 this is going to map it to the

original width and the original

height um we are also going to do the

same thing but for the

heights this was for the width and this

one was for the heights uh this one

starts from zero and this one is going

to start from one every second element

starting from

one and yeah this should be it we can

have the we can just return the key

points and there you go that is how you

um uh that is how you uh get the results

uh from the model before returning back

to the main we also want to draw those

draw those predictions on top of frames

so let's do that while we're here so we

can just have the

Define and uh called uh draw key

points and we can take an image and the

key points that we have just

returned um we can uh loop over each uh

one each

um uh each key point that we have and we

want to Loop over 2 by two and this is

because it is going to be XY then XY

then XY and one XY is one position so we

are going to Loop over it from zero and

to the length of key points but the next

step is going to jump two

so instead of I being one it's going to

be

two so we're going to skip one in the

middle so we're going to make x equal

int we just make sure that it's an INT

because like when you specify a pixel

position um it must be an INT and it

won't be fractions it won't accept any

fractions and then it's going to be the

I position and the Y is going to be the

I +1

position um afterwards we just want to

display two things we just want to put

the dot in the place and then we want to

have the um key Point number on top of

it so you can have the

CV2 dop put

text and you can write it on top of the

image then you can write um the um like

you can write the Str Str and then I /

by two and this is integr division so

it's going to be an integer at the end

um and X um

where minus 10 so this is going to take

the X position of it and the Y position

and we're going to make a buffer at 10

so we can put it on top of the um uh on

top of the

point so we are also going to specify

the font and the U uh font uh the font

sides as well so let's do that here so

font we also going to uh choose the

simplex one we are going to choose uh

font

size then we are also going

to choose the color this is in a BGR

format so this is or which is red um and

then we are going to

specify this and at the end we want to

put a

circle on the image in the XY

position and its uh radius is going to

be 5

pixels and um we are also going to make

it

red and then uh we are going to specify

that it's going to be filled so1 means

that it's going to be filled and at the

end of the day we are going to return

this

image now we are done with drawing on

one image but uh we also want to have a

convenient function where we can take a

list of images which is all the video

frames and run all uh the um and draw in

all of them and return back the results

so we can have the draw key points again

but on

video and this takes in the video

frames then the key

points and it has the output video

frames and yeah it Loops over the video

frames and the key points uh actually

uh yeah actually it did it uh like the

auto to complete um the theong so you

can remove this because there is only

one set of key points and we're going to

draw it again and again so the key

points right here and we are uh going to

put it right here in the key

points so again always check what um

what GitHub copet is uh is suggesting

because it might might suggest things

that are wrong and you need to fix it so

uh draw key points right here is I think

is working fine the draw key points it

takes key points and then the frame I'm

just checking it again uh so yeah it

looks like it's working

fine so

yeah uh so before moving forward we just

want to create an init

file and we want to

expose

the C line detector again outside of

this folder so we can

um use it in the main function in the

main. pi so we can write

from Court line detector import Cod line

detector we also want to have uh we want

want to like we want to do the same that

we did with the trackers we want to

initialize it then uh detect on the

frame so we can do that by writing

here um

cour line detection detector model we

can qu model path we can specify its

path first which is models Das line

model. PT not sure that it's the correct

name so I'm going to yeah it's not so

just copy the name and paste it right

here then um write the initialize the

call TR

detector like

this and at the end of the day get the

court key

points

and we are going to run it like this dot

predict and give it the first video

frame so give it only one video

frame and it should return back the key

points um and after that is done we can

also draw it so to draw it um

to draw it we can just write it right

here so you can have it uh right here

output video

frames and you are going to do Cod line

detector

dot draw

key

points on

video and then H give it the

output uh the output frames right here

give it the output

frames and then give it the quot key

points so we have the quot key points

right here just give it right here and

you should have the output ready I

think this might

not yeah I misspelled it um I misspelled

it at the beginning so I'm going to um

fix it uh I also noticed that here I'm

detecting both on video frames I should

be detecting on output video

frames and yeah that should be it uh I'm

making sure that here we're reading from

stubs uh and this prediction shouldn't

take long since we are predicting on

only one frame so yeah let's try it

out okay so now it's done so we can go

back to the output video right here and

we can see that the key points are being

extracted correctly uh there is a little

bit of error if you notice like for

three it's a little bit shifted to the

right um things like that but it is

close enough for our use case uh so

we'll proceed from

here um yeah so let's return back to the

code and let's um close down all those

unnecessary

tabs let's return back to the output and

focus more on the ball detections so you

can see that the ball is being detected

way better than what we had originally

but for example you can see right here

the ball is not being detected at all

and it's being detected like the one

frame before and the one frame after so

like this and this like you can see like

um we can easily um we can easily

estimate the position given the the the

position before it was lost and the

position after it was lost like this

one we can estimate it to be for example

the middle position um and if there were

more than one position you can divide it

by two divide the the the position by

two and to estimate it uh luckily we can

just uh use the interpolate uh function

in the pandas uh

library and it would uh fill out

those uh those bolts uh for us so let's

uh let's create that now so we can go

back again to the uh tracker which is

the

ball ball

tracker and uh we can write an

interpolation function so let's uh let's

start here so

um we can just write here

Define

interpolate ball

positions and it's going to take in the

uh ball positions or ball uh

yeah we have it called ball detections

in the main so but we don't care about

that right now um so what we do is that

we only have one track so we don't

actually need to have the track number I

just need the list of uh positions that

we uh that uh the bounding boxes that we

get so I can just have the ball

positions

equal and then dot

get one and if there is none and if

there's uh no one uh that means that

there was no detections I'm going I'm

just going to return back an empty

list and I'm just going to do a on line

for Loop that is going to Loop over the

ball

position so now we just have a list of

bounding boxes that is going to be empty

when there is no detections the list

inside is not is going to be empty when

there's no

detections uh we can convert this list

into a data frame so that we can do the

interpolation quite easily so we can

import

pandas as

PD then we can just do

pd. uh data

frame and then give it the ball

positions which is this

list and also give it the colums uh

which is going to be the column names

it's going to be um X

then y1 then X2

Y2 and I'm going to put it in a DF ball

uh

positions uh data frame this is what

it's called and then uh this just

converts it convert the list into a

panda data frame

um then we just interpolate the missing

values and the way that we do that is

that we

use the interpolate function inside of

pandas so we can just do this and run

interpolate and this

interpolates um if something is

missing uh between two frames or between

mult multiple uh frames but um it's it's

not going to interpolate positions at

the beginning so at the beginning we

want to make sure that there is no

missing values uh not to crash our code

so we can just duplicate the the the the

the earliest uh detection we can

duplicate it to the start of the frames

so the way that we do it is just we

backfill it so uh we do the bill

function to do it um again this is just

to handle the edge cases of um of not

handling of not getting any detections

the first frame but otherwise it should

be

fine um and then at the end we just want

to convert this back into the same um

format that we got the ball detections

from so um we have

B positions equal and then we can start

it again we had one as in the track

id1 um and then uh we can put X which is

going to be from the for Loop the on

line for Loop that we going to have and

DF

dot uh DF uh ball

positions dot 2

nump I'm going to make it a

uh an npire array and then I'm going to

make it into a

list and this is going to return back a

list of dictionary where one is going to

be the track ID and X is going to be the

bounding box it is the same format that

was uh handed to this function so I'm

just going to return it back uh to the

ball positions and the the reason reason

that we Chang it to Panda data frame

then back is that we just wanted to use

those interpolate function in the pandas

data frame so that we don't have to do

it by

hand um yeah so now we can just here we

can just do it like this and we can call

the interpolate function so we can have

the whips ball uh detections

we can call the ball

detections and whoops it's it's called

the ball

tracker so ball tracker dot

interpolate ball positions and give it

the bulb

detections and yeah we can run it again

right now and uh see how it

goes

okay so let's open this again and you're

going to find the ball is being detected

way more so um so you like the last

example I showed you here the ball was

not being detected but now it is uh

being detected although you can find

here that the bounding box is not

completely on top of the ball

but it is close enough so it is uh close

enough and if I run the whole video

you're going to find that the ball is

being detected in almost every

frame uh this is going to help us way

more in the analysis and makes it more

smoother so uh yeah um so we're going to

keep that and after we like before we

continue I want wanted

to I wanted to add the frame numbers on

top of the frame so that we can store it

and um Trace back uh how it how it

functions so we can just write draw

frame

number on top uh on the top left corner

so we just want to have the frame number

on the top left corner of the video um

and we can do it like this

um I am going to explain the code that

was written by um the uh GitHub cellet

which is going to be uh looping over

each one looping over each frame then

putting um the frame and then the number

of the frame and uh then it's specifying

an exact position of it which is

um X position of 10 and a y position of

30 and uh it's going to specify also the

U uh the font uh type and the um font

size and the color

um so yeah we can also make the

color uh like

this uh but yeah we should have the

frame number on the top uh on the top

left corner of the uh video right now so

let's run it

again okay now it's done so let's open

it and you have the frames on the uh top

left corner like you see here I'm just

going to minimize this a little bit so

you can see the frame number um and it's

in renting we can now like utilize it we

we'll be able to utilize it um further

on when we're uh tracing back anything

that is buggy that is happening um but

yeah it's going to be super useful

here so let's see the output

again and when you see the output we

have a lot of people being detected and

we just want to detect the two players

in the field uh we can do that by

selecting the players closest to the

court and we can use the key points to

to determine which players or people are

closest to the court so let's do that

right

now so we can uh return back to the U

main.py and we can see that like we have

the

quot key points and we have the player

detections right here so we can uh

basically add a function that can uh

determine which IDs or person IDs that

are closest to the court key

points so we can uh so we can go to the

player tracker and add a function that

is called chosen filter uh players so

we're going to choose the players and uh

filter them based on on their uh

proximity with the court so let's go

back again to The Trackers and the

players and let's create a function

called

Define

choose and filter

players this takes in the

um the chord key points

and then it takes in the player the

player

detections

and yeah um we are just going to choose

the players based on the first frame and

not all the videoos so we can do that by

just getting the player detections for

the first

frame and we can just make it like this

we take the zero with elements of the

player

detections and we can also

um do another functions called like

chosen um chosen players equals self do

choose choose players that takes in

court key points and the player

detections for the Press frame

now let's create this function so we can

have the Define choose players self then

C uh key points and then the player

detection

frames um we can start by calculating

the distances so the distances between

each um uh each uh player and the

court and we can do it by uh looping

over over each track so track ID and we

have the bounding box um in

player let's call that

player

dict so because it's a dictionary so we

can have the

items um and you have the track ID and

the bounding

box um and then we basically want to get

the middle of the box so we can do that

by um like um getting the X minimum y

minimum and dividing it by two to get

the middle of to get the middle position

and do the same for y and rather than

doing it here we can also add it in the

utils because um uh we are going to use

that also elsewhere so in the utils you

can go go ahead and create another uh

file called BB box which is bounding box

utils.py

and uh basically I want to get the

center of the box so we can have the

Define um

get um get

Center of BB

box and then I'm going to give it the

box and I'm going to extract the XY

positions add them both divide them by

two and add them both divide them by two

and return at the end the center X and

the center y the only thing that was

missed here is that they need to be int

so that they can be integ uh like pixel

positions so we can have it like this

and it should be good to go um we also

while we while we're here we also want

to measure distance between any two

given points so that uh we can measure

the distance between the

um uh between the point and the like the

key uh the bounding boxes and the key

points uh so the way that we do that is

we're going to create another function

that is called measure distance it's

going to take point .1 and

2 um and 01 is XY and point 2 is going

to be XY as well and we are going to

return um this equation so this equation

basically measures the distance between

any two points and it does that by uh

treating them as a right angle rectangle

and getting the hypotenuse of it so just

to visualize it you see here that um

like um you're measuring the distance

between a and c and the way that we do

that is that we get the a squ which is

the difference in X's uh sorry the

difference in y plus the difference in

um in x^ 2 and then you get the square

root of it and this is going to be the

distance between a and C uh the same

thing that we are doing right here and

multiply z z like the power 0.5 is just

the

root um and now we just want to export

those functions and in it so you can

have from

dot BB box

utils import get Center and measure

distance and in the players we can also

have like we can also import the this

from the utils and we can import system

as first and then system. pad. append um

like one folder before that just because

we want to go one folder back so that we

can see the utils and then we say from

utils

import uh we want measure distance and

get Center of bounding box now we are

ready to use it so we can say get Center

of bounding box and give it the bounding

box and now we have the uh center of the

bounding

box and now we can assign it to the uh

another variable called player Center

and now we have it now we just want to

calculate the distance between the court

and the player and we're going to

calculate the distance between the

player and each key point in the court

and see which one is the

closest so we're going to have uh

basically

minimum

distance um is equal to

float and then

Infinity then we are going to Loop over

each key

point and uh qu key points and we're

going to Loop over two at a time because

each one is going to be a one point um

then we are going to get the uh Court

key

Point um which is going to be the X and

Y positions then we are going to measure

the distance between the player Center

and the chord key point

and then put it into distance and if the

distance is less than the minimum

distance we take the minimum distance

and put it here um and yeah that is uh

going to be it so at the end we have the

distances do

append and then uh we append two things

we append the track ID

don't know if that was written correctly

track ID and then we app also the

minimum

distance so this way we have a tup we

have a list of tups with a track ID and

a minimum

distance

um so now we just want to choose the the

track IDs that have the lowest distances

between the key points so we can just uh

sort them

so we can sort the distances in

ascending

order okay and we can do that by this so

distances do sort and then we sort by

what value zero which is this one track

ID and one which is this one so we sort

the distances based on the minimum

distance and then we choose the first

two basically we get uh actually let's

call it choose the first two

tracks and you can have the uh like the

chosen uh chosen players is equal to

distance of zero and Zer which is going

to be track ID and then we are going to

get the next one which is one and zero

which is going to be also track ID so

this is going to be two players and we

can just return back at the end of

chosen

players and now let's also filter out

the chosen

players so now we are going to Loop over

the frames and we are only we are only

going to give the detections related to

those two chosen

players so I can have like

filtered

player

detections and I'm going to keep it as

an empty uh list then I'm going to Loop

over um the player

detections and I am going to uh choose

uh to Loop over each uh track ID in the

player dict and I am only going to take

the ones that are in the chosen players

so this is a oneline for Loop that can

be a little bit complicated but all it's

doing is that it is looping over the

player uh dick uh item by item and if

the track ID is in the chosen players

like this it is going to put the track

ID and the bounding box and return back

the dictionary afterwards after we have

the dictionary we append it to the

player D detections and then we just

return back the uh

output so now we have the output ready

and we can go back to the main uh right

here and we can uh now filter out the

required uh players so we can have uh

choose

players and we can write player

detections equal player

trackers do shoes and filter players we

can give it the court key points and can

give it the player detections and now we

have the last player detections we

should now have only two players and not

all the other people so let's run it and

also see the

output okay so we can open it again and

like you see like other people are not

being drone because we filtered the

bounding boxes out out and only two

players are being detected and this will

also simplify our lives a little bit uh

uh U simplifier life better

here so in the uh so in the the final

output video that I've showed you I've

also showed you uh something like a mini

cord that is being drawn um here so uh

you can see that uh this is uh this is

is um the court that is being done here

and here's the position of the player

one here's the position of player two

and here's the position of the ball

itself and this is what we call Mini

cour I call it mini cour right here um

we are going to be drawing on it and

also we are going to make it um

uh very similar to the original uh like

um proportions of the um um of of the

court so that we can measure distances

and we can measure uh like the distance

between the player in the ball and how

much uh did the ball cover and how much

did the player cover quite um uh

accurately and yeah so we are going it's

it's good for visualization but it's

also going to help us a little bit when

we are trying to determine um the

distance between uh any two

things uh so we can start

by uh like uh drawing this and then we

can move on from

there so you can uh write uh tennis

court uh

dimensions

and you have here the tennis core

Dimensions that you uh see so this is

called like the the the high one is

called the double alley which you can

see it's a 10 uh meter you also have the

uh Center Line the service line the

center

line and uh you also have this part is

called No Man's

Land um so yeah we're going to be

putting uh like some of those dimensions

in constants and it's going to help us

um understand what is the actual

distance that is being uh covered by

players in the

ball so let's create a folder called

constants

constants and let's create inside it um

an init

file and we can start by just writing

down a couple of things so we can write

the single line

width uh which is going to

be

8.23 this is what is called the single

line width I I just have the uh 8.23

which is this one and this is the double

line width double ali um so it's

10.97 so we can also write it here

double line width which is going to be

10.97 um there is a half quart line

width

um actually it's

height um and it's going to be

11.88 and let me tell you which one that

I'm referring to which is this one this

is the half court uh height this is the

U uh like the meter in

meters uh you also have the service line

width and it is 6.4 you also have the

double Aly

difference and it is

1.37 and again uh this is the double Al

difference right there so between the um

like the single allei and the double Ali

it's

1.37 uh um those are just a couple of

contents so uh so like constants that

are going to help us uh map between uh

pixel difference and meter difference so

yeah you can just uh take them from here

5.48 and while we're here we also want

to uh like determine the heights of the

players themselves and you can actually

actually uh like if you opened it you

can see that uh you can see the names of

those players and you can Google their

heights quite easily and I have Googled

them and I've know known them so I'm

going to just write them here uh so

player one

height in uh

meters and it's going to be

1.88 and the player uh two height in

meters

um it's going to be

1.91 those are also going to help us a

little bit in uh mapping out uh uh like

pixels to

met okay so now we're done with the

constants and now we want to draw the

mini qu so we can make another uh folder

called

Mini

and uh we can uh write the mini

uh

class mini cord

dopy and

yeah we can import

CV2 and we can also import

sis and we

can uh
from um we can sis. pad. append and we

can also return back one folder then

from

utils actually let's not do that right

now but we can import

constants

we can import the

constants okay sounds

good um we can def Define the class

start the class which is the

mini

Court uh then we can Define the init

function

and we can give it frame this Frame is

just going to be the initial frame of

the video uh just to get the the size uh

of it so

um we can start by defining how uh like

um we can start by defining how wide and

how tall this this should be like the

drawing style should it be uh large

should it be uh small uh I'm going to

set it like the ones that you see it

right here uh but yeah feel free to play

around with it a little bit so I'm going

to set the self do

drawing

rectangle

width uh which is going to be 250 it's

going to be uh 250
pixels so uh this white rectangle is

going to be 250 in width and it's going

to be 450 in height and the difference

between like the this padding

difference um uh this uh this padding uh

difference is going to be uh 20 uh I'm

going to make it 20 pixels uh around so

it's um there's going to be a little bit

of padding between the qut itself and

the white rectangle

and there is also a little bit of a

buffer between it and the top and the

right this is 50 pixels so I'm just

going to Define it right here so let's

define the drawing

rectangle uh height

now again this is the uh White rectangle

that you

saw and it's going to be 450

um let's also Define the buffer between

uh which is going to be this empty one

uh this part and this part uh the buffer

between the edges is going to be 50 and

the padding itself uh which is inside of

the white uh

rectangle um so padding of the quart is

going to be uh 20

pixels um

so yeah so now that this is done uh we

will just want to determine uh where

should we put the um uh this rectangle

like now we should uh find out the the

exact X and Y coordinates we should put

this rectangle on so let's create this

function here so we can have uh Define

set canvas

background box

position and it's it takes self and it

takes a

frame which is going to be the first

frame then we do a frame. frame. copy uh

just to not override the original frame

um we don't need to actually here but

let's keep it um we can uh Define the

end of the frame which

is this bit the this end bit to be uh

the frame do shape 1 which is going to

be the width of it minus self dot

drawing like self dot sorry

buffer which is going to be 50 so this

is going to start uh 50 but from the

end this is the end and we can also do

it self Dot

and Y and it's going to be self. buffer

which is going to be 50 from the top and

then 450 which is going to be this one

so self the buffer then self do drawing

rectangle height we also want to get the

start and the end so the start is going

to be end xus width minus the width and

the uh start is going to be also the end

y minus the

height and now we have the positions in

the uh self so don't we don't need to

return it but we do need to call it so

we can call set canvas background

position and we can give it the

frame awesome um now we just want to uh

see the position of the court itself

this this mini Court we we made the

position of the rectangle the white

rectangle now we want the position of

the

cord so let's also do a function for

that so Define set mini

Court

position and uh it takes in self and

actually we don't need anything else

so what we do is that we're going to

just add the paddings in the start and

end and um like the the start X and YX

so we can have the self. qut start x

equal self do start X Plus

self do padding

C okay then we can do the same for the Y

um and we can do the same for the index

but instead of adding it we subtract it

so that it can have uh a buffer right

here and um we can

also

have and we can also have the endex like

the end y uh so the n y minus padding

qu um now the what we also want to do is

that we want to define the width of the

qut so the cour width um and let's call

it Court drawing because it's just the

drawing uh it's going to be Court end x

minus uh the uh start

X now we also need to call

it like that in the init

okay uh now we just want to uh specify

those uh Court key points um the ones

that you see here in

Red so those Court key points are going

to be a little bit manual so bear with

me a little bit so we are going to write

Define uh

set put

drawing

key points okay I have misspelled it

right

here it takes in um nothing it just

takes in the

self and we can start by defining the

key points so we can have the key points

all at

drawing key

points we can have it all as zero and we

can have it 28 times so this is going to

create a list of zeros and it's going to

have 28 zeros um so let's define

01 or 0 Z to be

specific um so 0 Z is it's going to be

just this this point right here which is

going to be the start of the court so

start X and start

y so to do that we're just going to

create

drawing key

points of zero and drawing key points of

of one and it is going to be this start

X and start Y and let's also put ins on

top of it so that we can make sure that

those can fit into pixel

positions next we just want to do

01 and yeah end point1 is going to be

this one which is going to be this one

this is the end of X but still the the

start of Y so you can have the drawing

key points of two two two and three like

0o and one is for position zero 1 2 and

three is for position um 1 which is

going to be the end of X and the start

of Y this is correct so let's move

forward and let's also do it in the

point

two point 2 is going to be this point

which is right here um and this is two2

quarts

uh like with each other this is a half

qu and this is a half

qut and we want to be able to make it as

like as proportional as possible to the

actual uh meters that we have in the

constants so we're going to be adding

some conversion logic right

here so this conversion logic is just

going to be cross

multiplication uh but since we're going

to use it over and over uh let me put it

in the the

utils so in the utils you can have um a

new file called

conversions conversions.

py and you can open it right here um and

Define uh two functions which is convert

pixels to distance and convert meters uh

to uh pixel

so we're going to have

convert

pixel to a

distance to

MERS so convert a pixel distance to

meters so for example if the user moves

from

here to here how many meters did he move

we have the amount of pixels he moved it

he moved from uh like X and Y position

position and then he went to X and Y

position but we don't have the exact

meters that he moved and we can do that

uh by doing some cross multiplications

and seeing okay so from this point to

this point it is x m so from this point

or from this pixel to this pixel it is

it should be y meters we can do it with

simple cross

multiplication um so the way that we're

going to do it is but we're going to

take the pixel

distance and then we are going to take

the

reference

height in

meters then

reference

height in

pixels then we going to return it back

here so let me explain it first so now

we have the width of this um in pixels

and we also have the width uh of this in

the constants if you open it and yeah

the double Ali line the this one it's in

meters so you have this in pixels and

meters so you can do any cross

multiplications to find anything so for

examp example the height in pixels like

the width in pixels is let's say 20 and

it's in meters it's

10.97 and uh now I want I want to give

an arbitrary pixel distance and to

change it to meters I'm going to take

this multiply it by meters then divide

it by 20 uh this is simple cross

multiplication and this is um and we can

we can actually uh do it in in just one

line uh like this so pixel distance

times meters over the pixels and Heights

we also want to have the Define

convert

meters to pixel

distance so we have also the reference

height in meters and the reference

height in pixels and we're also going to

do some cross multiplication uh but but

this time we're going to multiply by the

pixel distance over the height

distance and yeah so let's now expose it

right here so from dot conversions

import convert distance pixels and

convert uh pixel distance and let's go

back here um think this

is should be constants

constants why is it adding it

SRE don't mind um then uh yeah we can

add from
utils um we should import

convert this

one and we should also

convert uh pixels pixel distance to

meter distance

so

yeah so let's continue down with the

points um we have the uh drawing

Point drawing key

point of

four and it is going to be uh the same x

value as the as the start so the X

didn't change so we can just

um write it as int self.

Court um start

x uh but the Y value changed so we can

have this

as um so position two is going to be um

so the start y plus uh two half quarts

so the way that we're going to do that

is we're going to uh take this start and

then y then add it to

convert pixels sorry we want to convert

meters to

pixels and then we have the Constance

dot um we want to get the half quart uh

height I just want to remember its name

yeah so half quart line height um

so this one and since it's going to be

the full qut like from here to to the

nut is the half qut so we want to do uh

uh like two times this so we're going to

multiply by two and let's not forget to

add the constants do half

cour and we want to give it the two

references that we have right now so the

reference in that that we are going to

use is that we already have the cord

width so we can uh simply write it right

here and uh this cord twist is going to

be constants do

double um like uh let's also go remember

the name it is

the double uh double line

width so we're going to give it the

double line width and then we are going

to give it the

court uh chord drawing

WID okay so now we have the point two

ready um and we are going to basically

converting a lot lot um of this um like

meters to pixels a lot so I'm going to

make it a separate helper function right

here um it's going to be

convert

meters um

to

pixels it's going to take the

meters and we are going to utilize this

so you have this and you run it like

that and instead of meters instead of

having it a constant we can take it here

and now we get the uh the distance in

pixels when we have the meters so we can

just do like this self and we can close

down the bracket here and remove this

um now it's a little bit easier for us

uh moving forward to 3 um so point3 is

going to be uh again this point which is

going to be 2 plus the width of it um so

let's uh let's do

it uh it's going to be um drawing six so

we're going to use the same x value as

the first one

like that and then we

can um actually I forgot a t right here

let's add it let's add it right here we

can add the uh core drawing

width uh since we already have it and we

can keep the same

um y value as the previous one so that

we

uh so we can keep the same y value as

this one so uh we had two and we're

going to keep the Y value to have it for

three um again we're going to Loop over

it again for

point4 and uh
4.4 we have other uh another uh another

uh 14 point that we want to do so what

I'm going to do to not bore you I'm

going to fill it out and then uh come

back

later so I've filled in the uh the all

all the points right here uh the other

the points and at the end I just assign

the drawing key points into a a self

variable so that we can have it uh here

so right now we're just going to do the

self and um going to call it again and

right now we just want to define the

lines so right here you have um apart

from the points you have the lines so

you need to understand which lines you

want to connect with like which two

points you want to con to connect a line

with and for this we're going to also

manually select those and let's start

with it um so we can Define set cour

lines and then give it the self and

self.

lines

equal and if you look at this you have a

line between zero and two and zero and

one so we can put them uh right here so

0 and two and you can also have uh Z and

one uh you also have a couple more uh

but I don't want to go like I don't want

to go through them all to not bore you

you can just uh it's pretty

straightforward uh so I'm just going to

copy paste

it and I pasted it and and I pasted the

lines right here so again from 0 to two

there's a line line from four to five

there's a line and you can find that

those are uh what we see so you can set

CT lines also right like this and that

is it and we are set with the uh like I

think there is no more manual work right

here so I think we are ready to maybe

draw it um so let's draw it so we have

the

draw

background rectangle which is the white

rectangle that you saw so we can take in

the

frame and we can just uh first

initialize the um um like uh we want to

draw a white rectangle so we're going to

do shapes like

NP Zer let me import n p

okay so now we have um the same like the

the frame that we had um but we like the

image that we have but with all zeros

and this is going to help us um do this

a little bit of transparency you can see

that it's not an actual like filled

color it has a little bit of

transparency in it and this is the way

that you do it um you do a dummy image

uh which is going to be all zeros and

then we going to add it to the image

that is going to be the frame but with a

little bit of opacity difference

um so on top of this we can draw the

rectangle uh which is going to be uh CV2

rectangle and then we draw on top of the

shapes one and then we give it the start

X start Y and then the end X and Y and

we also want to make it uh uh like uh

white so this is the white

BGR and negative one is filled and if

the numbers are confusing you you can

just write

CV2 do fi and it's going to be the same

thing if you hover over it uh you're

going to see that it's an INT which is

going to be negative

1 um so we're going to have an output

frame which is going to be the frame.

copy and we have going to set an alpha

which is going to be 0.5 so it's going

to be 50% transparent um then we are

going to set a mask which is going to be

shapes as type

bull so anything that is zero uh so it

doesn't have a rectangle uh so it's

going to have only the like the mask is

only going to be two for the rectangle

bits and we are going to get the output

then the mask only the pixels that we

want to have and we want to add it with

weighted so that we can um add it a

little bit with a transparency Factor we

want to add the frame and then the alpha

which is 0.5 we want to add the shapes

which has the rectangle we just want to

have the um the other transparency of

the shapes and yeah zero is

fulfilled and uh yeah so and then we get

the

mask um then we write output equal

CV2 and then we just want to convert it

so CVT

color

uh BGR to RGB you just want to convert

it to an RGB format and then we return

back the output of course this one draws

on one frame

only so let's draw it on all frames like

Define

draw mini

cord then

self and give it the

frames we then
uh have the

output frames which is going to be

for frame in

frames and we just want to have frame

equal self. draw

rectangle and give it the

frame and at the end we just want to

append it so output frames do

pen

frame and return it

back and this is it um we should now

have the white box and afterwards we

should be able to draw the key

points uh so let's uh see the results

till now and then we can

continue so in the mini in the mini cord

we just want to add an

init py and this is also going to help

us a little bit

with exposing the function or the class

outside of this

folder and we are going to go here

from mini Court import mini

cour we also want to

have um like here we want to initialize

the mini court so now we initialize the

mini

court

and and now we want to draw the mini qu

um on top of the the frames uh so let me

do

that so draw

mini

qut and I will have it the output frames

equal mini cord through mini cord output

frames and yeah that should be it let's

run it and hope that it works

okay so we have an

error um convert meters to pixels is not

defined um I think I've misf felt

something so let me

check yeah I've misspelled convert

meters to pixels I didn't I forgot to

put the two so uh let me uh rerun it

again and now I have another error which

is uh I Mis also wrote the uint 8 so I'm

going to fix it and uh come back to you

I think is going to be

here um and it's going to be

you int

8 so let me run

again

so now it's working fine we can open

this up again right here you can see

that the image is a little bit weird

because it's in BGR not in RGB uh but

you see also the uh rectangle that is

transparent uh right here um but now we

need to fix the uh BGR one so I see here

that we also convert it to BGR to

RGB

and yeah maybe we don't need to do that

give me a

minute yeah we didn't need to convert it

again from BGR to RGB because it was RGB

from the first place and now we have the

um like the mini mini qut background

setup okay so so now that we have the

background rectangle ready we can just

uh start drawing the court so to do that

we can just write a new function right

here uh Define draw Court

uh which is going to take a self and a

frame and we are going to Loop over each

point so in

range and uh starts from zero till the

self do uh drawing key points and we

Loop two at a time

and

uh X is going to be int um the self.

drawing key points of

I and Y is going to be the same but +

one because it's going to be the next

valuable uh

variable and then we just add a

circle on the

frame

with X and Y we are also going to set

the radius to five we are also going to

set the

color and we are going

to um make it filled which

is1 now we should have the

um now we should have the uh key points

ready so let me return the frame

and uh see the results but before seeing

the results we need to call it so I'm

going to call it right here self Dot and

give it the

frame and yeah that should be it so

let's call

it um there is something

missing

H yeah so I I wrote uh here uh like the

two here in the length it should be

outside the length so let's run

again okay uh let's uh see the output

right now so you can see the output of

this is um um having the key points

right here right here right here and it

looks fine but the width or the the

height of

the um the height of the white box is

not big enough um I think we can expand

it a little bit it should be fine but

yeah

um we can also make it in red like that

we can make it in
red um but we can also expand the

drawing rectangle height a little bit to

be for example uh

500 so that it

yeah

it's running right now so let's see the

output now the output looks fine we have

the court we have everything that is

ready we just want to connect the lines

and this is going to be uh uh

simpler so in the draw cord function uh

we just want to uh now draw the lines so

let's call draw

lines and so for line

in self.

lines uh what we want to

do yeah sorry four line in lines

um so

start

point

and it's going to be

um this one it's going to be

the uh drawing key points of line zero

and this is also going to be uh the Y of

it and uh we're going to have an end

point also and it's going to take from a

one so uh line

one uh so each line is going to have

like like we saw like two points zero

and one and we want to just connect them

both and we're going to connect them

both with the CV2 line and we're going

to give it the start position the end

point and we are going to give it also a

color let's say it's black and yeah that

should be it let's uh run it and see the

results so I made a small mistake

earlier where I didn't multiply the

position by two and this is because the

the zeroth position is starts from 0 to

1 the first position uh starts from 2

and three and the second position starts

from four and five so you can see that

you take the position multiply it by two

to get the X1 and then add one to get

the Y so you need to do this the same

here and if you uh run it here and and

you're going to get the uh correct

output uh wait for a while and you

should be done any anytime

soon okay uh now it's done if you look

at it now you have the lines ready we

just don't have the uh net line uh this

is going to be an easy one it's just

going to be the middle point of the cord

so we can just calculate it real quick

and and yeah to uh to draw the net we

just get draw

net so we need to get net start uh

point and it's going to be

self.

drawing key

points of zero because it's like the

zero with position and um it's also

going to be int which is like the start

key point is going to be the zeroth

position which is X which is going to be

um like this

x this X and it's going to be this x at

the same time but it's going to be in

between this one and this one so let me

uh calculate it like get the positions

of them both and then divided by two so

we can have um

self actually let's um let's use their

equation but edit this to

five so now we can have the start

position so let's do the same for the

end position let's copy paste it uh and

it right here put the end position and

make it start from position two instead

of zero and keep the Y the same and

lastly we just add the line and let's

make it this color and make it start

from net start and end at net end and

yeah we can now run it and see the

results

okay now it's finished and you can see

the chord is being drawn quite correctly

and yeah that looks

good

um so let's continue with that before

closing down this class uh we basically

need to add a couple more

functions

and yeah so we set a couple of variables

um so let's uh let's make functions to

get them so let's get for example the

start point of the

Min

um self and then we're going to return

back the self

uh self do

Court start

X self.

court

and

um

uh start whoops uh

Court start

Y and we can also get the width of the

mini qu so we can Define

get uh width

of mini cord and we can return back the

drawing width um we can also return back

the key points if we want so let's

get cour key

points actually let's call it cour

drawing key

points and let's uh return the uh

drawing key points and yeah I think this

should be it so now we're done with

drawing the mini cord and I think uh we

are ready to go back here

and uh

continue

um so right now we have

the

um this

output and everything looks good but

let's now detect the the ball hits so at

this Frame exactly let's minimize this a

little

bit at uh around frame between 10 to 12

the ball was hit and we want to like get

this ball hit frame and know which

player hit the ball and the same here um

so we can continue to detect the ball

frames

so now we just want to detect the uh bow

um we we just want to detect the ball

shots so we can create a temporary uh

folder um that is

called um

basically maybe called

analysis and let's make it a folder

sorry about that let's delete it

then let's create a folder called

analysis and inside of it let's create

um

ball

analysis.

ipynb um now let's

import pickle because we're going to

read the pickle uh St that we have for

the tracker right there and we are going

to also

import uh pandas we are going to import

M plot

lib pip plot and yeah I think that

should be it um so the first thing is

that we want to uh read uh the pickle

file um so we are going to read it like

that and let's call it uh ball

positions and let's also refer to the

correct path which is called

tracker uh

stops and ball

detections do

pkl let's run it and let's make sure

that it threaded correctly so it it

threaded

correctly um we can also copy paste the

The Filling of of the U the

interpolation that we had so let me open

up the tracker again and the ball

tracker and let's copy this

code and yeah let's copy this code and

put it right

here

um this is going to be for uh

aspd so yeah you just run it

again and now you just have the um uh a

pandas data frame that you can uh plot

out um so yeah so the first thing is

that uh we basically want to have the

position of the ball and we can can um

make um like the like um calculate the

center of the ball and the way that we

do that is that we can have um we can

also create the mid y only like the

middle y uh and it's going to

be uh this one like the y1 + Y 2/

2 and this is going to be get us the mid

middle of the Y afterwards uh you can

see that we have pretty smooth um pretty

smooth uh detections but in some

scenarios we need to remove any

outliers and we can do that by having

the rolling mean so we can uh add it

right here we actually don't need it

here but let's add it um for other use

cases maybe other videos have this J

action but we can have the rolling

mean and this rolling mean is going to

need a window so we are going to have a

window of
five and uh we are also going to set a

minimum

period of one um and we are not going to

center it

so Center equal

false um and we are going to get the

mean um

afterwards uh we

can basically plot

it so we can plot this

out

so when we plot it out like this you can

clearly see where the ball was hit so

the ball was moving like this then it

changed white positions here and it then

change white positions here it changed

white positions here it changed white

positions here and this is like the

normal behavior of the ball like if if

you see that um the ball is moving in

the y direction like this and after it

was hit it changed Direction so the ball

instead of uh being more and more like

the Y coordinates is being increased uh

it suddenly changed directions and now

it's it's

decreasing till the other player starts

hitting it again and instead of

decreasing the Y it starts increasing

again so we want to

detect when exactly in the plot that the

um that this is being um like the Y

coordinates is being

changed um we can uh we can do that by

like analyzing the difference between

each point and the other point is it

increasing is it decreasing and and how

much so we can define a Delta

y so we can have the uh ball

positions Delta

Y and then we have

the uh define B positions rolling mean

and we get the diff this difference is

just going to sub subract

the uh two consecutive um like rows from

each

other and yeah we can also try and plot

it out and it should give us

approximately a very similar graph so

there you go it's a very similar graph

and you can also understand where the

ball was hit here here here here and

here and yeah you can continue on with

that um so yeah to understand where the

ball was hit um we are going to Loop

over this and we are going to determine

whether the ball is increasing or

decreasing the Delta Y is increasing or

decreasing and we are going to get the

position of the the frame number

where the ball started to uh decrease

when it was increasing like this change

and it kept on changing for at least

like 25 frames so we will check this and

when it's at the

bottom we will check if it's increasing

again um it's going it's it's going to

be a very simple code so we can have the

ball

positions and we can have the ball hit

is all set to

zero so we can have it all set to zero

at the beginning and then I'm going to

Loop over this data frame

and like

this

DF um ball

positions

minus int and I'm also going to have the

um minimum uh minimum change

frames for hit and I'm going to send it

to 25 so at least it's moving uh like

increasing and then at least for 25

frames it's it kept decreasing so the

change here kept decreasing 25 frames at

a row or like maybe not in a row but um

within a certain

buffer uh so I can just uh have this uh

and uh loop over uh like the length of

the ball positions till minus minimum

changes and let's put a buffer of

20% so let's multiply it by

1.2

and continue um we we want to detect if

there is a negative change so there is a

negative change that is happening where

where the ball is here and then it

decreased in value so we can have a

negative position

change uh where is it's going to be um

uh like uh both positions Delta y

um iock uh of of I iock is just getting

the like the uh row number

I and it's going to be if if it's a

negative change then it's going to be

greater than

zero but the the preceding one is going

to be less than zero so it was

increasing and then it was decreasing so

now it's going to be less than zero we

also want to detect the uh positive

change which is it was uh decreasing and

then it was uh increasing and then it

became

increasing and we can call it here

positive

change positive position

change um and if there is a negative

change or a positive change um we can uh

begin to count how many frames the

change uh kept on going for so for

change frame in uh

range I +

1 till I

plus int and then

minimum uh let's call it like the uh oh

yeah so we need to multiply this here

and let's copy this put it right

here and we uh yeah and let's add one to

it and for for this we will need to copy

paste the same thing again we will need

to uh do it and detect if there's a

negative ch change or a positive change

uh but let's call it change in the

following

frame okay so now we have the change in

the following

frame and

um yeah but instead of checking I + one

we need to check the change

frame and if

and if there's and if there's a negative

uh and if there's a negative

change and there is a negative change in

the following

position um we just change

count and make it change count plus one

I change count plus equals 1 we want to

do the same thing for the positive

change so it's going to be the same

thing and at the end we want to have

if um and yeah here we just want to have

if the change count is greater than the

minimum

change um hit minus one and if if that's

the IDE if that's the scenario then we

need to change the location uh like the

rule the the ball hit in the row number

I to one so that uh we now have the uh

basically um uh the the ball hit

position so let's run it you're going to

find a couple of uh wearning messages

don't worry about it and yeah you can

run it again and you can see the ball

hit a column that is uh already there we

can filter it to only produce a or to

only give us the ball hits that are one

so let's do it like

this so ball hit equal equals to one and

let's turn it so you can find that

there's um a hit in uh position uh frame

11 frame 58 frame 95 frame 131 and frame

182 and let's check if that's

correct so here the frame number will

come in handy so let's um let's move

this and see so approximately frame 11

starts the hit and if we go back again

here so we have a hit in frame

11 and if we continue down the line we

can see that around uh frame 57 which is

here

58 um another hit was being done another

shot thought was being done and again

right here uh around frame 95 which is

another um another hit we can also see

here which is around

131 another hit is being done and the

last hit is at frames

182 so the logic is being um uh so the

logic is is quite good and we can

continue on from

this so we can just um uh get those

positions by just copy pasting this then

saying that we want the index of

it okay then we want it to make it to

list uh you need to yeah you need to

remove those packets from the index and

now you have it as a list we can now

just uh frame nums you can have that

frame

nums

with ball

hits okay so now it's all good you can

uh paste it in right

here and you can also try and

uh finalize your code um just just going

to copy all this put it all right here

so that I can copy the code and paste it

in the ball tracker so we can uh from

here we can start by uh writing the

Define

get ball

shot

frames it's going to take in the ball

positions

and it is going to run this

basically so again uh we need to uh

convert this into a data frame so we can

copy this from here uh like

that and yeah I think we're done so it's

it's uh it's um having it but we're it's

not returning it yet

so we can return it and now the uh

function is

done so let's uh do it right

here and we can

detect ball

shots and we can have the

ball

detector ball

tracker Dot

what's the get ball shot

frames and we give it the ball

detections we can write here ball

shot

frames now let's print it out and uh see

the results if it's the same or

not

so uh I think I forgot to uh copy paste

something which is initialization of

the uh ball hit so it's going to be ball

hit and it's going to be all zeros so

let me run it

again so yeah right now we have the 1158

and 95 all the frames that we detected

before are now being detected so all now

is

good we can keep printing it but uh yeah

we we removing we'll remove it uh quite

shortly up next is just we want to uh

convert the player positions into the

mini cord positions and this is going to

help us draw the mini cord that's first

and the second thing it's going to help

us uh determine the amount of distance

that is being covered by any object and

uh by tracking it on the mini cord and

not the actual pixels right

here um so let's uh start uh by doing

that uh so you can go back to the mini

cord that we had and we are going to add

one more function which is convert

bounding boxes to actual mini mini cord

coordinates uh so let me explain the

logic first we are going to measure the

distance between any key Point let's say

key Point 2 and the player itself and

now we have the distance in

pixels but I want to measure how much

like this distance in meters so what we

have for reference is that we have the

actual height of the player so we have

the height in meters and we have the

height in um in pixels so we can do

cross multiplications to find the

distance between the second key point or

basically the closest keyo and the

player

position and then I can come here and

say okay I can I have the distance in

meters can can we convert it back into

pixels and we already have this function

so we can just call this function with

meters and it will draw it in the uh

position that is going to be relative to

the

court and yeah that is going to be the

whole logic of

this um so let

me uh begin the code so we can have the

convert

convert

bounding boxes

to mini

cour uh
coordinates and we are going to take the

um

player uh

boxes and we are going to also take the

ball

boxes we're going to convert them here

into one function um and for

convenience and then we are going to

take the

original uh cour key

points um so yeah let's start first

by let's start first

by uh defining the player

Heights so we can say that um player one

has the constants dot uh

player

one uh height in

meters we can have the same thing done

for player

two like

this

and then um we can Define the output

boxes output player

boxes and we can Define the output ball

boxes and the first thing that we want

to do is we want to Loop over the

frames so we're going to have the uh

frame number

player uh bounding box and enumerate

which is going to uh return back the

index that we are are looping over and

let's Loop over the player bounding

boxes right

now

so uh the I so in the player bounding

boxes there are multiple tracks so we

are going to Loop over each track so I

can have the play player

ID and uh I can also do the bounding

box in player. BB box. items and now I

have it uh

ready um the first thing that we want to

do is that we want to uh get the foot

position of it so the foot position is

basically going to be like we are going

to get the middle of the uh of the Box

in terms of X and then get the maximum y

so it's going to be right here um we

don't want to have it to in the middle

because here is actually where the uh

the player is standing it's not here

it's uh it's actually

here so to do that we can go back to the

utils and add another uh function in the

uh BB box utils uh it's going to to be

called a get foot position so we can

have the

get

foot
position and it takes in a bounding box

and it returns in the foot position so

uh we can do it like

that so X1 uh Plus X2 which is the ex uh

like the x and x and we can divide it by

two and we return also the uh maximum

key point we also want to

uh expose it uh in the

init and we want

to uh import it so we can get the foot

position

of the BB

box and uh this is the foot

position

okay uh now I want to get the closest

key point so I want to

measure uh like uh like we we want to

get the

closest uh key Point uh um

in pixels

basically and we don't have to search

all the key points uh we can limit it to

a couple so let's choose them right now

um so you have this zero which is going

to represent the end and we have the two

is going to represent the other end and

let's also take the middle of the course

which is going to be 12 and 13 uh this

is going to be fair enough let's not uh

overdo it and search all key points so

let's search only zero and two which is

the extremes and two middle points which

is 12 and 13 this one and this

one um yeah so we will need to have um

function get get

closest key

Point

index uh that basically takes in the

foot position

and the uh original key

points and the uh filtered like the the

the key points that we want to search we

said zero two and 12 and

13 and uh we return back to the uh

closest key Point uh

index and this closest key point we will

be uh using to measure the distance

between it and the player and uh yeah

it's going to be the point of reference

uh for

us so let's also add this uh get closest

key Point uh index uh to the uh BB box

utils so we can have it right here get

close

closest key

Point index and we can have the point

that we want to uh measure the distance

from we can also get the key points and

the keyo indices that we want to have

and yeah we can uh loop over them let's

remove the suggested

code and we can get the

closest uh

distance which is going to be float of

infinity right now and we can get the

key

Point index um which is uh going to be

key

Point uh candidates of zero we're just

going to take the candidate zero right

now and overwrite it when we Loop then

we are going to Loop over the key

Point

candidates uh

indices so let's call it index in

keypoint

indices

and let's remove all to the suggested

code I don't think that it's going to

work for us so we can uh write that the

key point is equal to um key

points

of key Point

index times

2 and then we want to also have it for

the Y so it's going to be the same but

plus one at the end then we are going to

measure the
distance which is going to be uh only in

y distance so let's only measure y

distance so

point1 minus uh key Point uh

1 and if distance is smaller than the

closest key Point um then what we do is

that we assign the closest like the we

assign the distance to the closest key

point and then we we have the new index

uh that we want and at the end we can

just return

it let's now expose this

function um to the inits again so we can

access it let's go back to the bounding

box um uh to the mini cord and we add

also this

function and let's also uh yeah so it's

already being utilized

here then we want to get the player

height in pixels so let's uh get the

player height in

pixels um I want to also show you

something before we get the player

height is that the player might be

launching and is not uh straight up like

like this one and this one so uh the

idea is that I'm going to get the

maximum height within the fif within a

50 frame to ensure that the pixels truly

represent uh the total height of the

player uh so let me uh try and get the

uh frames that we are going to search

for um so let's uh get the

frame uh index uh minim

and it's going to be the maximum of zero

and frame number minus 20 so we're going

to uh do um uh we're going to go back 20

frames and we want to also have the

maximum number the maximum index which

is either going to be the length of the

frames or frame number uh plus 50 so we

can go up to uh up to 50 frames

um then we want to detect the height of

the Box in pixels so we can go back to

the

utils and add another function um to

that so we can have Define get

height of uh BB box and it basically

takes a bounding box

and let's see the

um

return and it returns back the height

which is the maximum y minus the minimum

y we can also expose it right here and

we can import it here and we can simply

uh get the heights of this so uh we can

say BB boxes

Heights in

pixels and we can simply have like uh

four I in

range and it's going to be frame

minimum till frame

maximum and we are going to get

the um get the height of the bounding

box and and this is going to be player

bounding box of

I now we have a list of heights of

bounding boxes in the frames of in the

in the list frames that we

want um we want to get the maximum one

so we can say max player height in

pixels and we can get the uh maximum

bounding box uh

height um afterwards we want to get the

um the court position so now we have the

enough information to um convert this

into the court position that we want uh

so let's create another function that is

called get mini qu

position so we are going to Define

another function so get mini

cord coordinates it's going to take in

the object

position uh

object

position it's going to take in the

closest uh key

point and it's going to also take the

closest keypoint
index and it's also going to take the

player height in

pixels and let me just uh organize it

into different lines so that it can be

easily uh seen

and

so uh player height in pixels and then

uh player height in meters so

player we can copy this and say

meters um yeah so right now we have the

closest key Point index um let's also

Define the um like let's get the actual

point

so we can say

closest

um key

Point

equal yep so it's going to be the

original key point to the closest key

point and then multiply it by two and we

are also going to get the Y position of

it okay so sounds good uh now we want to

continue with the mini

uh with the mini cord

coordinates um we want to first uh get

the distance uh between like the

distance X and the distance y uh between

uh the closest key point and the

player so we can get it so we can say

distance um

from

keyo

X pixel

pixs then we can say distance from Key

Point y y

pixels and then we can say it's equal to

measure

XY

distance and you give it two points

which is object position and closest key

Point object position and closest key

points now we don't have the function

yet so we can go back to the BB to the

bounding box utils we can um basically

Define it and see it right here like now

we get the X position the X difference

and then the Y different difference uh

separately and when we return them both

we also want to expose it in the init

and we also want to import it right

here and after we get this um we want to

convert this distance into meters so

let's uh

convert

pixel

distance to

M and what we do is that we write

distance from Key point x in

met
equal um convert

pixel distance to meters and then we

give it the pixels and we also give it

the player height in meters and then the

uh player height in pixels so um player

height in meters right

here and player height in

pixels we want to do the same thing but

for the Y position so let's copy paste

it and get it also for the Y

position like

that

and and now we have the meters ready so

this is the amount of meters that they

that the player has uh really uh like um

moved and now we like uh this is the

amount of meters that is between the key

point and the player and now we just

want to convert this into play like

pixel coordinates again but for the mini

qut so it's going to be for the mini

size

qut so we are going to say the convert

to mini Court coordinates

and what we do is that we write

mini cord X

distance in

pixels self do we have a already made

function for that which is convert

meters to pixels and we just give it the

meters so like like that we do the same

thing for the Y coordinates like this uh

we change it here to the Y and we change

here also Y and yeah we also have we

also can get the closest mini position

the closest uh Mini U uh mini Point Key

Point uh it's going to have the same

order as the original key points so we

don't have to do any calculations here

we just uh need to get the position of

it so we can say

closest

mini

qu key

point is equal

to um

self.

drawing

uh drawing uh key points

of

closest uh keyo index time 2 and we also

want to have this for the Y but we are

going to add one because it's a y

position and yeah that is

it let me just format the bracket

correctly and at the end we just want

to um add the distance in pixels to the

mini cord position to have the last

position so it's going to be mini

cord

player

position is equal

to um mini cord player position of zero

times the distance in X and we want to

have the same thing for the Y and yeah

that is

it we can return back the results like

that and yeah we can now call it here so

so we can call Mini uh

Court player

position

position which is going to be calling

get mini court court and we give it the

foot position and then we give it the

closest key point then we give it the

closest key Point index then we give it

the height and pixels of the player and

we also give it the player height in

met uh we then uh want to uh assign this

so what we want to do is we want to have

an output dictionary for each frame so

when output

player uh BB boxes uh dick and it's

going to have the play

ID as the key as usual and it's going to

have the position as the

value now we want to do the same thing

but for the ball and since the ball

doesn't have an exact height or the

height of it is going to be very small

um we are going to use the closest uh

player to the ball and we are going to

do the same calculation for it um so

yeah so let's calculate the uh balls

first so um let's get the ball

uh box which is equal to ball boxes of

frame

number and then we have the uh ball

position uh and we are going to get the

uh center of the Box um let's not get

the uh foot of it right now but let's

get the center of it and we can get the

center of the Box um and we can call it

off uh BB

box uh we can go here we can Define get

center of the BB box and we can return

and write it right here so this is going

to get the center of the Box uh which is

going to uh get both x's and divide them

by two get both y and divide them by two

and this is going to get the center of

the Box um then we also add it in the

init we also import it here in the um

mini uh position and now we can U we can

utilize it here so now we have the ball

position um we can get the uh closest

player ID to the

ball closest player

ID uh to ball and this is going to be

the minimum of

the

player uh BB box do

keys

where

um actually it it it's uh it did it

quite good so this is what we do measure

distance I think we already have the

measure distance in the utils right now

so we don't need to rewrite it so let me

explain what the suggested code was all

about um so right now it's going to

measure the distance between the ball

position and the uh get foot position

let's let's also get the center of it

let's get the center

position um and the center position of

the player so it's going to measure the

distance between this and this and it's

Al Al it's going to return the key of

the minimum player bounding

box and yeah

so that is it so we can go here

and we can yeah we can go here and we

can write the the code for the ball

which is if the

closest

uh if the closest player ID to the ball

is equal equal to the player ID uh what

we want to do is the to get the closest

key point also so

closest let's copy paste it so we need

to get the closest key Point

again like that and instead of uh using

the foot position we're going to use the

ball position

and now we can just run the mini cord

function again like

this and instead also of the of the foot

position we write the ball position and

now we have those two in place and the

maximum height and the player height are

also being

put um now the last last thing is that

we want to have an output dictionary so

let's call it

um output ball BB

box um so yeah um

let's so let's just add the output to

the uh to our list which is going to be

output ball boxes do append and it's

going to be one because it's only one

one uh one uh ball and we're going to

get the mini Court position right

here um and at the end we want to return

both

the so return the output player

dictionary and
the output

balls um actually we don't want to

return the uh we don't want to return

that we don't want to return the

dictionary we want to return the output

player boxes for the whole frames so

let's add it right

here and uh and let's replace it

here so now we are ready so this is the

uh coordinates of the um mini cord for

the players and the balls so we can go

back to the main and we can call it from

there right here we are going

to uh

convert the

positions uh to mini Court uh positions

so uh we are going to write player uh

mini cord

detections

and we are also going to write ball mini

core

detections and those are going to be

equal to

mini do convert to mini Court positions

and we give it the court points and uh

uh yeah so we give it the uh player

detections and the ball detections and

the court key points so we need to also

give it the uh Court key points right

here like

that and yeah uh now we want

to uh basically display this on the mini

cord so we can have uh like actually

let's run it first make sure that

nothing crashes and then we can create

another function that draws uh those

positions onto to the mini cour

itself okay so we have a small

error uh I'm going to trace it back and

come back to

you so I uh so the problem was that I

renamed the function here it was convert

to Min cord positions it should be

convert bounding box to mini cour

coordinates um so let me run it again

and see if we have any more

errors okay we have one more error which

is in line 32 of get center of the box

and uh I think the box has a key zero

error so let me trace it back and come

back to

you so I had two problems in the code

one is that I was treating this as a

bounding box B bounding box but it was

actually a dictionary so I needed to get

the one track the track number one and

this will be now a bounding box and the

other one is going to be the

um uh this one um I was uh giving it a

dictionary I needed to filter out the

current player ID and this is how I did

it um

and yeah that is now a bounding box also

so if you can rerun it

again it should be working fine right

now so now it's working fine um but we

want to also see the uh

results so we're going to make another

function that is going to add the uh uh

the this drawings to

it and we are almost uh uh finalized

with this mini cord so bear with me a

little bit so to draw any points on the

uh mini Court we can just call

draw uh

points on Mini

qut

and we can write the self we can get the

frames and we can get the positions that

we want to uh write it on and we can

also specify a color we can give it

um um a default

color and we can Loop over the frames

frame

number and

frame and numerate frames

um we can also Loop over the positions

that we want to uh uh that we want to

Output so uh right now we have the

position of this Frame number we have

the player ID we don't need it so we can

uh remove this we have the position and

we can draw a circle with the frame and

the

position and uh then uh the radius the

color

and it's

filled uh just to make sure that the

position is in we can just uh write it

like this

position and we

can make it as ins like

that and like that just to make sure

that those are ins actually um and yeah

X and

Y so so we can call this on our

code so right here uh we can have the

output frames equal mini cour Dot and

then

draw

points on Mini

qut and then we uh give it the output

frames we also give it the uh

positions uh that we want to uh put

which is the player mini Cod

detections player mini cord

detections and we can do the same for

the
balls uh so we can call the ball uh mini

detections and we can give it another

color just to differentiate both of them

and uh color is equal to uh 0 250 5 and

then

255 that's another color um we can then

uh run

it okay we have another error so let me

trace it back and come back to you

guys so yeah we didn't return back the

frames so we can just return it back

here

return frames and it should work

fine so let's open the

output and we don't see the key points

yet we don't see those uh the players

yet on the

here let me trace it back and come back

to

you so if you were following along I

made a small mistake when the brackets

um right here in the closest mini uh key

points I did uh the plus one outside the

bracket it should be inside so this was

the problem so you can save it and you

can rerun it again again and you should

find that the output has been uh on the

on your output

video just wait for it a little bit and

yeah that is it um so you can open this

here and you can see how the player is

relative to the uh position here and the

other player is here and the ball is

also very close to to its position and

if you run it you're going to see that

the ball is moving quite correctly and

the it's the ball is quite um that the P

pill is moving correctly and the ball is

a little bit jumpy because it's fast um

but otherwise everything looks good and

we can proceed from

here so the last thing that we are going

to work on is going to be um is going to

be the output of the kilometers per hour

and I'm going to show you the output the

final output again and this is

it that is the last thing that we are

going to work on it's going to have the

uh shot speed and the player speed and

it's going to

also um like count how many uh how many

shots that the that the player has uh

shot each player has shot and I'm not

dis exping it here but we'll be

calculating

also um so let's uh Dive Right

In this is the last bit but afterwards

you're going to have a super cool um

portfolio project to add to your CV so

it's going to be totally worth

it um yeah so let's jump into the main

and I think we are not going to need any

other classes maybe I'm wrong but uh I

think uh we're done with

those and the way that we are going to

do it is that um we are going to uh loop

over each

shot in those frames and let me go

through the logic first

so uh we are going to go each shot in

the frame so we have this uh this Frame

number which is frame number 11 for this

shot and we have the other shot which is

this

one um which is frame 55 uh we are going

to measure the distance that the ball

have covered um we are going to measure

it in uh U measure it um in the in

pixels first maybe in the uh mini mini

qut and uh we can then switch it to

meters

and uh we also have the uh frame

difference so we know that this Frame is

um frame number 11 and it took it around

um till frame 55 which is around maybe

uh 40 something frames to uh go there

and we know that per second we have 24

frames so we can so we can measure the

time in seconds and we have the meters

so we can measure the um the speed which

is the kilometers per hour uh we also

have like this is the player who shot

the ball we also have the position of

the opponent player and the amount that

he covered the amount of um meters that

he covered and we also have the time

that he covered it in so we can also

measure the player speed so we are going

to Loop over the shot and take the next

chot into

consideration and uh measure the

distance covered by the opponent player

and the ball and basically um calculate

the uh the kilometers per hour from

here so let's uh so let's start

um so for the for the shots we are going

to

um uh we are going to Loop over the shot

which is going to be shot index in range

length and we are going to have it uh

shot frames which we got

earlier and um we are not going to

include the last one because the last

one doesn't have another one uh that is

going to be uh like another uh frame so

we can't measure the the exact distance

covered or the uh the amount of time

covered uh but we are going to measure

it for the uh for all other ball

shots and yeah we can uh get the start

frame

basically which is going to be the ball

shot and uh we are going to get the ball

shot in Index this is the uh start frame

and the end frame of the shot is going

to be the next um the next bullshot so

it's going to be bullshot index +

one uh we can get the time difference in

seconds of the bullshot which is going

to be uh

bullshot

time in

seconds which is just going to be the uh

end end frame minus start frame over 24

because we have 24 frames per

second um then we have the uh get

distance uh covered by the

ball and first we want to measure the

distance in pixels so what we do is that

we get the distance uh

covered

by ball in

pixels and we are going to

measure the

distance we have this function in the

utils I don't know if we imported it

here or not so I don't think that we did

so let's do that here and yeah so let's

continue on and so we have the measure

distance function

and it will take the uh

ball mini cord detections of start frame

and it's going to take the ball mini

cord detections of the end

frame and because we only have one uh

track so then we are going to just get

the uh one yeah so we can we can use the

convert pixels to me meters um that we

have in the mini and it should return

back the

meters uh now to uh measure the uh speed

so speed of the

ball

shot in kilm per

hour um we have the speed of the ball

which is going to be the distance

covered by ball in meters over the time

in seconds time 3.6 and this time 3.6 is

just going to convert it to kilm per

hour um now this is uh going to be the

speed of the ball uh now we want to get

the speed of the opponent player so we

can have the

opponent uh

player

uh uh we can have the opponent player

speed and before that we can also have

the

um we can we can uh but before that we

need to understand which uh which player

shot the ball so we can do that by

simply measuring the uh distance between

the ball and the the two players and

getting uh the player that is closest

closest to the ball when the ball was

shot so we can do that by simply doing

that player who shot the

ball we want to do that so player uh we

want to get the player positions uh and

the player mini cord uh detections of

the start frame let's get the player

that is closest to the ball so we can

get the player who

shot uh the ball which is going to be

mini uh minimum of uh player uh

positions do

keys and we are going to have the

Lambda uh player

ID and we are going to measure the

distance and the distance we are going

to measure it is between the player

position and the ball position so we

have the player positions of uh player I

and we also have the ball detections of

start frame of one and this is going to

return back the player ID with the

minimum

distance uh with that in hand we can

also uh def know which uh which which

player ID is the opponent which is going

to be players

um

opponent opponent player

ID is going to be opponent player ID one

is going to be one if the player shot

the ball is equal to zero and else it's

going to be

um actually it's it's going to be one if

it's going if it's two and else is going

to be two so let's say that the the

player who shot the ball is player uh

two I'm going to return one uh as in the

opponent player I'm just going to return

the inverse of it and if it's not one if

it's not two then I'm going to return

two that's the opponent

player um I want to uh measure the

distance covered by the opponent in

pixels so

distance just like what we did in the

ball

uh but we are going to do it for the

covered by the opponent player by uh

opponent in

pixels and it's going to measure the

distance between the uh player um it's

going to player like the start frame of

the of the opponent ID and the end frame

of it

so that is good now we want to convert

it into meters so that is also going to

be the same as the mini qu uh one and

then we want to measure the speed so

speed

of

opponent opponent which is equal to the

distance covered by opponent over B shot

time frames in second which is going to

be the same uh frame time 3.6 to make it

in kilm per

hour uh yeah so now we have the speeds

and we have uh the speed of the ball and

the speed of the opponent uh we just

need to put it in a type of a uh

structure data structure so that we can

um display it in a better way so I am

going to choose to display it in a list

of dictionary uh because it's going to

be easier for me to change it to a

pandas data frame and to um expand it to

the whole um other frames as well

so let's create this um data structure

which is going to be just a list of

dictionary uh stats uh

data it's going to have a list and this

list is going to be a dictionary uh the

first one is going to be a dictionary

it's going to have a frame

number which is going to be zero at the

beginning then it's going to have the

player

one number of shots which is going to be

zero then uh it's going to have the

player one uh

total uh shot

speeds now this uh now the number of

shots is the number of times that the

user that the player have shot the ball

the total speed is the summation of all

the speeds that the user have

done and you can calculate the average

by just um having this over this like

this number over this number you can get

the

average so I'm not going to have the

average right here but I'm going to

calculate it

later and we can have the player one

last uh shot speed so this is the latest

shot that he did what's the speed of it

and we can also do it with zero and we

can have the player one uh total

distance uh total player speed um so

total

player speed which is going to be zero

and we can also have the actually

let's last player speed we can also have

the last player speed on and we can do

the same thing for the um player two so

number of shots and the total speed of

the shot the last shot uh the player

speed uh the last uh player speed as

well

um so that should be it um so let us uh

import a copy uh mechanism which is from

copy import deep copy because in order

to copy a dictionary you need to um uh

copy it and not copy it reference so

deep copy helps us uh to copy the

values um and the way way that we're

going to do that is that at the end of

this uh we are going to uh create a a

status uh like the stats the

statistics so we have the current player

stats which is going to be a deep copy

of the the the previous player stats and

we are going to overwrite some things so

for examp example the frame number is

going to be the sort of the frame

number and then we want to have the uh

player number of shots which is going to

be player number of shots plus one we

want to also have the total speed to be

added uh we also want to have the um

last shot to be added here and we want

to add the opponent player so the

current opponent player which is uh

right here we want to add to the total

speed of it and we want to add the last

speed and afterwards we want to have the

player stats data. append current player

stats so now we have for each start

frame we have this dictionary again but

with the new

stats and for that we will need um like

um now we just have the uh like the

short frames and we want to expand it to

the whole list of frames so that we just

can Loop over without too much logic and

um basically from frame zero to the

first start frame we just want to

duplicate this again and again so that

we have um like uh

so that for each frame we have its

statistics um we can also like Loop over

it and um do some logic so that we can

uh know which uh position to to choose

from here but I'm going to uh choose to

duplicate it and to make it into a

pandas data frame so that it can be

easier to um

display so let me change it into a

um first the data frame so data. DF uh I

don't know if we have pandas already

here so I think we do maybe we

don't um and then we have it so let's

import

pandas now we have it as a data frame um

let's also have a frames number data

frame so frames DF which is going to be

PD data frame which is equals to uh

frame

number and it's going to have a list um

of range of length of the video uh this

is going to be a data frame with frame

number and it's going to have zero till

the uh length of the frame

number

um I think there's a bracket missing or

something like that let me figure it

out yep there was a bracket

missing and I just added it right here

and now we can just do

this and merge it so now we are merging

the uh frames data frame with the player

stats and I'm going to have it uh left

join on the frame number and that is

going to result in multiple rows going

to have n because only the start frames

of the B shots have value so other

things will have n and I want to fill it

so the way that I'm going to do it is

that I'm going to do a front fill and uh

basically like this which is equal to FF

fill it's going to replace a nan with

its previous value that is not a Nan and

do this so on and so forth so that every

frame we have the um we have the output

of it um now I want to uh basically

calculate the average of it so let's

calculate also the player

tats and data DF of

player average

uh shot

speed and we are going whips we are

going

to um divide the total over the number

of

shots um I think I misspelled it right

here so let's um do this

and player

one uh short

speed over the number of shots this is

the correct

one um it's not it's it's it should be

player one average shot speed and let's

me check this player one to to shot

speed and the player one number of

shots that looks correct let's do the

same thing for player

two and let let's do the same for the

player speed so this is the player speed

and it's also getting the total over the

it's not um it's not doing the over

correctly so um it should get the

other so it should have the total speed

over the opponent's number of shots

because the player only should only like

we are only calculating the run when a

user shoots and uh how the uh opponent

reacts to it so this is the player one

over the number of shots that were done

over player two and we're going to do

the same for player two and we're going

to get the total of player two over the

number of shots done by player

one so now all all the statistics are

done we are just missing the last

drawing of this so we need

to um draw this uh draw those tats and

yeah uh and we are done we are good to

go

um so in the

utils uh we can uh do another um another

function

uh another

file

called

player

stats

dra utils.py

and we are going to put the uh to put

the uh the the drawing logic here it's

not going to be logic just going to be

drawing um it's uh it's going to be easy

to do but uh it's going to consume a lot

of lines so I thought to put it in the

utils and not in the main just to keep

the main

clean uh so we have the numpy as NP and

we also want to import

CB2 and let's define the

function so the function is going to be

draw player

stats and we are going to have the

output uh video

frames and we are going to have the

player

stats and we are going to Loop over it

so Loop over the uh player stats so in

player stats dot it

RS and uh we have the player

one shot speed as in row of player one

last shot

speed we have also Player Two Shot speed

that is going to be last player two shot

speed um we want to have the player one

speed which is going to

be um player one last player uh speed

and we want to have it also for player

two um we want to also get the average

player speeds so average one player shot

speed and then the uh player two we want

to do it for the speed also

um and average player

speed um now we want to get the frame so

frame is equal to the output frame of uh

index should be index

and now we want to have uh

basically it's not that one um we want

to have a black transparent box so we're

going to add also the position of the

box and then make it

transparent and and then we're going to

add those um

um

statistics

so adding those statistics are going to

be uh we are going to need the shapes

again because we're adding a transparent

uh uh black box which is going to be

also zeros like the one that we did

above above it

and um we want zero like frame and we

want to Al to have it

u in 8 let's write it correctly this

time and then we have the width is equal

to

350 the height is equal to

230 we also want to um Define a start

and an end X so start X is going to

be um frame.

shape

of uh 1 minus uh

400 and we also want to have the start y

um which is going to

be of zero and I think I wrote shape

wrong so let me fix

it and uh

550 and let's let's also Define the X's

so we have the end X is going to be

start X Plus width and then end Y is

going to be start y plus uh the height

um then we are going to have the overlay

which is going to be this one it's going

to be the

overlay start X start y and x and y and

we are going to to uh put a black one

this time a black box this time and it's

going to be filled and we are going to

Define an alpha of also

0.5 so we are going to add it weighted

again and we're going to add it overlay

the alpha the frame and then we have uh

the rest of

it um then the output video frames

um then we can just have the output

video frame of

index is equal to the

overlay is equal to the frame

sorry okay now it's time to put in the

text uh now the text is going to be

um like the text is going to be just

putting the text like the the times that

we put the text here before um it's just

going to be a matter of choosing the

positions so I'm just going to copy

paste it and walk you through

it so I copy pasted it right here and uh

right now we have the player one player

two and then I put the text I chose the

start position and the end and start

start x and x uh I also chose the colors

I did the same thing I put the shot

speed I then wrote The Shot speed

themselves and then I did the same for

the player speeds and then I did the

same for the average speeds uh So

currently right now what I want to do is

export this is expose this

again so uh from

dot

uh uh player stats import uh draw player

stats we want to also import it in the

main right

here like this and we want to now draw

it

so uh let's draw player

stats

and and we can have this like this so we

can have it uh like this draw the like

call the draw player stat function and

now we

just run

it so there's a so there's an error here

that is called uh convert pixels to

meters um because the mini cord function

does not have convert pixels to meters

it has convert meters to pixels so we

will need to um so we will need to

import that from the

utils convert meters to

pixels and we also want to import the uh

constants so what we do here is that we

just want

to uh convert it and use the double line

width like the width of the

uh the the field in meters and we also

have the width of the mini cord so we

are going to use those to do some to

convert the um pixels the pixel like the

pixels uh done into uh

meters so what we're going to do is that

we are going to do like

this we're going to choose the meters

right here then we are going to give it

the

reference uh uh the reference um meters

which is going to be uh constant.

double uh whoops it's not that was

filled

incorrectly so you have the

width and then you have the mini quart

and then you can get width of the

miniart

um get mini height I think I need the

width of

it um let me go and check if there's the

width

ready this function get mini

cord get width of mini chord I think we

have this

function so yeah so we have the get

width of mini chord

and we will do it again right

here and instead of that we can use this

and just delete

that let's run it

again so uh player one last shot speed I

think this one is in in

the uh drawer because I added another S

I think there shouldn't be an S like

here uh but let me check the

line line 139 which is yeah so it's

going to be the

uh drawer so yeah the speed is going to

be speed not

speeds um so like this so let me run it

again
and now it's done so you can go back to

the output and see it so um there is a

little bit of overlap so you can

actually push it down a little bit so

let's do that um let's uh move move this

here uh so let's move it down like 50

pixels so let's make it start at 50 uh

500 not

550 um let's also specify the width to

be a little bit

more uh I think it's going um

above no it's not

um oh yeah because it's not um so you're

going to find it to

have uh oh there's some weird numbers

going

on so we have some weird numbers like

here like 1,000 km/ hour and here is

also 1,000 km/ hour uh let me trace it

back and come back to you but uh let's

first before tracing it back let's see

if uh if this would push it down a

little bit more like 500 the make it 500

and instead of

550 and let's see if there's something

else that breaks but it

shouldn't Let's cross our

fingers and we should be able to see the

box but we should still be uh seeing

some weird kilm per hour that we need to

fix so yeah so right now it's uh it's

it's actually good but the kilometers

per hour is way too much and I'm going

to trace back Y and come back to you uh

so give me a minute so I'm back and I

know the error um I've used the convert

pixels to this I've used convert uh

meters to pixels uh and I should have

used the convert pixels uh to meters so

I I have the distance covered in pixels

and I want to convert it to meters but I

used the wrong function and uh you

should be using this one convert pixel

distance to meters

and if you run it again uh you should be

able to get the correct

output so you can now see um so you can

now see as as soon as the player hit the

ball you can see the shot speed of the

play player like this is 33 km/ hour and

player two is going to move by uh 1.7

km/ hour and when the other player shot

you can see that it was a harder shot by

a 43 km per hour and the other player uh

moved with 4.5 km per hour and if you

see that this player printed actually so

you can see that it has a 9 km per hour

and and

um and yeah you can also see the

averages right here the average player

speed the average player um uh shot

speed and the actual player speed um you

can do all sorts of fancy stuff like the

number of shots and you can analyze how

fast a player is running when he's

losing the pole and you can analyze how

fast um the ball like the how fast the

ball is shot to make um a winning shot

um you can also do a lot of other stuff

like check whether the ball is inside or

outside the court when it's hits the

ground and the project can be very big I

chose to make it simple right now uh but

yeah um you can we didn't also do

anything with the net so you can also

add a little bit of net logic if

something uh goes wrong or hits uh

before the net um so yeah you can you

can build on this but currently you have

a very cool portfolio project right in

your belt and hopefully it's going to

help you u landar a job in computer

vision and

AI so that's it and uh have a wonderful

day