Color Tracking

In this section, miniAuto will use the ESP32-S3 vision module to track a red
ball. It will simultaneously follow the red ball’s movement.

1. Program Flowchart

2. ESP32-S3 Vision Module

ESP32-S3 development board integrates an ESP32 chip and a camera

module. After being inserted into the expansion board, it uses the I2C
communication interface to read color and face data through I2C
communication.
ESP32-S3 is equipped with an OV2640 camera module, which transmits the
captured image data to the ESP32 chip via a serial port. The ESP32 chip can
process the image and transmit it to other devices through I2C communication.

3. Program Download

1
3.1 ESP32-S3
1) According to “5.3 Live Camera Feed/01 Live Camera Feed”, flash the
firmware located in “02 Program File/01 ESP32-S3 Color Tracking
Program/color_detection.bin” in the same directory as this section to the
ESP32-S3 vision module.

2) After flashing, disconnect the Type-C cable. Turn on the miniAuto to run the
program.

3.2 Arduino UNO

Note:

 The Bluetooth module must be removed before downloading the program, otherwise

the program download will fail due to serial interface conflict.

 Please turn the switch of the battery box to “OFF” when connecting the Type-B

download cable in case the download cable touches the power pin of the expansion board

by mistake, which may cause a short circuit.

1)Locate and open “02 Program File/02 UNO Color Tracking/color_tracking”

program file in the same dictionary as this section.

2)Connect the Arduino to the computer with the UNO cable (Type-B).
3)Click the “Select Board”, and the software will automatically detect current
Arduino serial interface. Then click to connect.
2
4)Click to download the program to the Arduino, and then wait for the
download to complete.

4. Program Outcome
After powering miniAuto on, the buzzer will beep once and the RGB light on
the expansion board will light up white. When you move the red ball in front of
the ESP32-S3, the robot will follow the movement of the ball.

5. Program Analysis
The name of this program is “color_tracking.ino”. The implementation logic of
the program can be referred to the following flowchart:

3
5.1 Import Library File
Import the necessary timer, tracking action, motor, and ESP-S3 vision module
communication header files for the game.

5.2 Define Pin and Create Objects

Define two timers (currently only Timer1 is used), an object of the vision
module, as well as two variables for controlling the movement speed and angle
of the robot. An array for storing the registers of the vision module’s line
tracking function is defined.

4
5.3 Initial Settings
1)Initialize related hardware equipment in “setup()” function. The first is serial
interface, which sets the baud rate of communication to 115200.

2)Initialize each module. Timer1 starts counting with a single time unit of 20ms.
1. “ticksFuncSet(ticksGetFunc)”: Binds the system's preset function
“ticksGetFunc” which is for obtaining the current system time to the
“platFormTicksFunction” function. The “platFormTicksFunction” function is
called to obtain current system time.
2. “Motors_Initialize()”: Initializes the motors.
3. “hw_cam.Init()”: Initializes the ESP32-S3 vision module.
4. “Controller_Init()”: Initializes the computing unit for the robot’s motion control
parameters.
5. “timerStart(&timer1, 20, timerPIDCalculateCallBack, "20ms cycle PID
calculate")”: Timer1 starts working. The callback function
“timerPIDCalculateCallBack” is triggered and called 20ms later. It passes
"20ms cycle PID calculate" as a parameter to the callback function.

5
5.4 Loop Call Sub-function

After initialization is complete, the program enters the “loop” function. The
"timersTaskRunning" function is called in a loop. It traverses and processes the
timers in the timer list that have reached their trigger time. The interrupt
functions are executed.
Then, the “Velocity_Controller(set_angle, set_speed, 0,
SIMULATE_PWM_CONTROL)” function is called to control the robot's
movement speed.

5.5 Timer Callback Function “timerPIDCalculateCallBack”

The function is used to determine if the robot has detected the target color
position. This is achieved by reading the vision module's line following function
register data via the “Linepatrol_Data_Receive” function. Then, control the
robot's movement accordingly.
If the target color is detected, the robot adjusts its movement angle, based on
the deviation between the color center position and the set target tracking point
position. This allows the target center position falls on the set target tracking
point position.
The timer starts counting for the next cycle. It works only once by default.
Call “timerStart” again can set the timer to loop work.

6
5.6 Timer Monitoring Function “timersTaskRunning”
The timer list is designed as a linked list by default. You can traverse and
add/remove members of the timer list via two pointers “pTimerList” and “entry”.
When a timer reaches its trigger time, the timer list considers it invalid and
removes it from the list. Therefore, if you want a timer to work repeatedly, call
the “timerStart” function again or multiple times in the callback function.

5.7 Motor Initialization Function

This function is used to initialize the motor pins. Call the “pinMode” function in
a “for” loop to set the motor IO port to output mode. Then, the
“Velocity_Controller” function is called to set the robot to stop.

7
5.8 Robot Velocity Control Function

The function sets the speed of each wheel of the robot. By analyzing the
kinematics of the robot's Mecanum wheels, the speed of each wheel can be
calculated. The calculated speed variables can be used as input parameters
for the “Motors_Set” function to adjust the motor speed and control the overall
movement of the robot. Please refer to relevant tutorials to learn about
kinematic analysis.

8
5.9 Motor Setting Function

This function adjusts the speed and direction of each wheel of the robot based
on the input parameter values. In the “for” loop, the robot's steering is adjusted
based on the calculated results. The “map” function maps the calculated
values from “0 to 100” to the range of “pwm_min to 255”. Next, the “digitalWrite”
and “analogWrite” functions sets the motor direction and speed.

6. Function Extension
How to modify the recognized color of ESP32-S3 from red to another color ?
Take blue as an example.
Locate the “Linepatrol_Data_Receive()” function for obtaining the information
of the vision module.

According to the macro definition register, modify the first parameter to the
register corresponding to the blue.

9
After modifying is completed, refer to “3.2 Arduino UNO” to download the
program again. Then, the robot can track blue.

7. FAQ
Q:Since the code was upload, the color can not be recognized.
A:Please check that you have connected the 4-pin cable to the I2C
interface.
Q:Why is the color recognized by the camera inaccurate or even incorrect?
A:Please try to minimize the complexity of the background. You can use a
single color or a simple environment as the background.

10