3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

Automatic Addison
Build the Future

How to Make an Object Tracking Robot Using

Raspberry Pi

In this tutorial, I will show you how to give your wheeled robot the ability to follow a colored ball. You will get your rst
taste of computer vision and image processing.

Special shout out to Matt Timmons-Brown for this project idea. He is the author of a really good book on Raspberry Pi
robotics: (Learn Robotics with Raspberry Pi). Go check it out!

https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 1/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

Video
Here is a video of what we will build in this tutorial.

Computer Vision-Enabled Robot | Raspberry Pi | Automatic Addison

Requirements
Here are the requirements:

Build a wheeled robot powered by Raspberry Pi that must identify and follow a yellow rubber ball using OpenCV, a
library of programming functions for real-time computer vision and image processing.

You Will Need

The following components are used in this project. You will need:

Wheeled Robot
Yellow Rubber Ball (available from eBay)
Raspberry Pi Camera Module V2-8 (Standard)
2×2 Lego Brick (available from eBay)
VELCRO Brand Thin Clear Fasteners

Directions

Connecting the Raspberry Pi Camera Module

Make sure the Raspberry Pi is turned OFF.

Open the Camera Serial Interface on the Raspberry Pi. It is located next to the 3.5mm audio jack. Pull it upwards
delicately from either side.
https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 2/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

Insert the ribbon of the camera module into the Camera Serial Interface. Make sure the silver contacts face away from
the 3.5mm audio jack.

Hold the ribbon in place while pushing down on the Camera Serial Interface port. Make sure it is closed.

Mount the camera to the front of the robot.

https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 3/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

Power up Raspberry Pi.

Open up a con guration window:

sudo raspi-config

Interfacing Options –> ENTER –> Camera –> ENTER –> Yes

The camera is enabled.

Now, we need to set the resolution.

Advanced Options –> Resolution –> DMT Mode 82 1920×1080 60Hz 16: 9 –> ENTER –> Finish

Restart the Raspberry Pi by typing the following in a terminal window.

sudo reboot

Testing the Raspberry Pi Camera Module.

We need to take a test photo with our newly installed camera module.

Open a terminal window. Type the following command:

raspistill -o test_photo.jpg

Go to your home directory to see if the test photo is there. Here is the photo that mine took (back of my head).

https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 4/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

Setting Up Object Tracking

Now, we need to con gure our system so the robot can track a yellow rubber ball.

Download the dependencies for OpenCV, a library of programming functions for real-time computer vision and image
processing.

Type the following command into a terminal window:

sudo apt-get update

sudo apt-get install libblas-dev libatlas-base-dev libjasper-dev libqtgui4 libqt4-test

Y –> ENTER.

Wait a few moments while everything installs.

Install OpenCV using pip.

sudo pip3 install opencv-python

https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 5/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

Install the PiCamera library.

sudo apt-get install python3-picamera

Determining the HSV Value of the Yellow Ball

We need to select an appropriate HSV (hue, saturation, value) value for the yellow ball. HSV is an alternative color
representation that is frequently used instead of the RGB (Red Green Blue) color model I covered in my light and sound
wheeled robot post.

Here is the HSV table.

Since the ball is yellow, I’ll choose 60 as my starting number.

Open IDLE in your Raspberry Pi, and create a new le in your robot directory. Name it color_tester.py.

Here is the code for the program:

1 # Code source (Matt-Timmons Brown): https://github.com/the-raspberry-pi-guy/raspirobots

2 # import the necessary packages
3 from picamera.array import PiRGBArray
4 from picamera import PiCamera
5 import time
6 import cv2
7 import numpy as np
8
9
10 # initialize the camera and grab a reference to the raw camera capture
11 camera = PiCamera()
12 camera.resolution = (640, 480)
13 camera.framerate = 32
14 rawCapture = PiRGBArray(camera, size=(640, 480))
15
16 while True:
17 while True:
18 try:
19 hue_value = int(input("Hue value between 10 and 245: "))
20 if (hue_value < 10) or (hue_value > 245):
21 raise ValueError
22 except ValueError:
23 print("That isn't an integer between 10 and 245, try again")
24 else:
25 break
26
27 lower_red = np.array([hue_value-10,100,100])
28 upper_red = np.array([hue_value+10, 255, 255])
29
30 for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
31 image = frame.array
https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 6/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

32
33 hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
34
35 color_mask = cv2.inRange(hsv, lower_red, upper_red)
36
37 result = cv2.bitwise_and(image, image, mask= color_mask)
38
39 cv2.imshow("Camera Output", image)
40 cv2.imshow("HSV", hsv)
41 cv2.imshow("Color Mask", color_mask)
42 cv2.imshow("Final Result", result)
43
44 rawCapture.truncate(0)
45
46 k = cv2.waitKey(5) #& 0xFF
47 if "q" == chr(k & 255):
48 break

Place your ball about a yard in front of the camera.

Run the newly created program.

python3 color_tester.py

Choose 60.

You will see four windows.

Window 1. RGB representation

Window 2: HSV representation
Window 3: Show the portions of the frame that match a hue value of 60.
Window 4: Entire frame minus all portions that do NOT have a 60 hue value.

https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 7/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

To try a di erent hue value, select any of the four windows above. Press Q to halt the output of the video.

Go to the terminal window, and try a new hue valve. I’ll try 29 this time. It worked!

You keep trying di erent numbers until Window 4 shows mostly your ball and nothing else. Be patient and try LOTS of
numbers.

Write down the hue value you ended up with on a sheet of paper.

Press CTRL-C in the terminal window to stop running color_tester.py.

Coding the Ball-Following Program

Open IDLE. Create a new le in your robot directory named:

ball_following_yellow. py

Here is the code (Credit to Matt Timmons-Brown, the author of a really good book on Raspberry Pi robotics: (Learn
Robotics with Raspberry Pi):

1 # Code source (Matt-Timmons Brown): https://github.com/the-raspberry-pi-guy/raspirobots

2 from picamera.array import PiRGBArray
3 from picamera import PiCamera
4 import cv2
5 import numpy as np
6 import gpiozero
7
8 camera = PiCamera()
https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 8/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

9 image_width = 640
10 image_height = 480
11 camera.resolution = (image_width, image_height)
12 camera.framerate = 32
13 rawCapture = PiRGBArray(camera, size=(image_width, image_height))
14 center_image_x = image_width / 2
15 center_image_y = image_height / 2
16 minimum_area = 250
17 maximum_area = 100000
18
19 robot = gpiozero.Robot(left=(22,27), right=(17,18))
20 forward_speed = 1.0
21 turn_speed = 0.8
22
23 HUE_VAL = 29
24
25 lower_color = np.array([HUE_VAL-10,100,100])
26 upper_color = np.array([HUE_VAL+10, 255, 255])
27
28 for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):
29 image = frame.array
30
31 hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
32
33 color_mask = cv2.inRange(hsv, lower_color, upper_color)
34
35 image2, countours, hierarchy = cv2.findContours(color_mask, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMP
36
37 object_area = 0
38 object_x = 0
39 object_y = 0
40
41 for contour in countours:
42 x, y, width, height = cv2.boundingRect(contour)
43 found_area = width * height
44 center_x = x + (width / 2)
45 center_y = y + (height / 2)
46 if object_area < found_area:
47 object_area = found_area
48 object_x = center_x
49 object_y = center_y
50 if object_area > 0:
51 ball_location = [object_area, object_x, object_y]
52 else:
53 ball_location = None
54
55 if ball_location:
56 if (ball_location[0] > minimum_area) and (ball_location[0] < maximum_area):
57 if ball_location[1] > (center_image_x + (image_width/3)):
58 robot.right(turn_speed)
59 print("Turning right")
60 elif ball_location[1] < (center_image_x - (image_width/3)):
61 robot.left(turn_speed)
62 print("Turning left")
63 else:
64 robot.forward(forward_speed)
65 print("Forward")
66 elif (ball_location[0] < minimum_area):
67 robot.left(turn_speed)
68 print("Target isn't large enough, searching")
69 else:
70 robot.stop()
71 print("Target large enough, stopping")
72 else:
73 robot.left(turn_speed)
74 print("Target not found, searching")
75
76 rawCapture.truncate(0)

Running the Ball – Following Program

Place your robot in an open space on the oor with the yellow rubber ball.

Run the program.

python3 ball_following_yellow.py

Whenever you want to stop the program, type CTRL-C.

https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 9/10
3/26/2021 How to Make an Object Tracking Robot Using Raspberry Pi – Automatic Addison

automaticaddison / June 5, 2019 / Computer Vision, Robotics / education, ground, raspberry pi

Automatic Addison / Proudly powered by WordPress

https://automaticaddison.com/how-to-make-an-object-tracking-robot-using-raspberry-pi/ 10/10