Air Drawing App Using Computer Vision Documentation
Air Drawing App Using Computer Vision Documentation
Computer Vision
Subtitle: Statement Code: CV-06
Table of Contents
Introduction
Problem Statement
Objective
Required Libraries & Tools
Methodology
Code
Code Explanation
Output
Output Screenshots
Challenges Faced
Future Enhancements
Conclusion
References
Introduction
This project is an innovative and interactive computer vision-based
application that enables users to draw on a digital canvas using a colored object
as a virtual pen. Built using Python and the OpenCV library, the application
captures real-time video from the user's webcam and detects a specific color
range through HSV (Hue, Saturation, Value) filtering. Trackbars are provided
for users to adjust HSV values dynamically, making it easier to isolate and track
the colored object accurately under varying lighting conditions.
Once the object is detected, contour detection is applied to find its
position on the screen. These coordinates are then continuously recorded and
used to draw lines on a virtual canvas, creating an experience similar to
freehand drawing. The application interface includes virtual buttons at the top of
the screen for changing brush colors (Blue, Green, Red, Yellow) and clearing
the canvas. When the user moves the object over these buttons, the program
responds accordingly by changing the drawing color or resetting the canvas.
This project not only demonstrates real-time object tracking and drawing
but also highlights gesture-based interaction without any physical contact. It’s a
fun and creative way to explore computer vision and has potential use in art
applications, education, and touchless UI systems.
Problem Statement: Air Drawing App – CV-06
Objective:
The aim of this project is to develop an interactive Air Drawing application that
allows users to draw digitally on a screen using the movement of a colored
object, such as a marker or cap, held in front of a webcam. The application
should be able to detect the object based on its color and track its motion in real-
time to translate that movement into drawing actions on a virtual canvas.
Detailed Description:
This application leverages computer vision techniques to recognize and track a
user-defined colored object using HSV (Hue, Saturation, Value) color filtering.
Once detected, the position of the object is continuously monitored frame-by-
frame through the webcam feed. The motion trail of the object is used to draw
lines on a blank virtual canvas, mimicking the effect of drawing with a pen in
mid-air. The application also includes on-screen buttons to change drawing
colors and clear the canvas, providing users with simple gesture-based controls.
The interface and drawing process should be responsive and visually smooth to
enhance the user experience.
Constraints:
The application must ensure accurate and consistent tracking of the
object.
It should provide real-time drawing feedback with minimal lag.
Color detection should be adaptable under different lighting conditions.
Objective
The primary objective of this project is to develop a real-time virtual
drawing application that enables users to draw on a digital canvas using a
colored object, such as a marker or cap, as a pen. This is accomplished by
utilizing a webcam to continuously capture the live video feed and applying
computer vision techniques to detect and track the movement of the colored
object. By identifying the object based on its unique color range in the HSV
(Hue, Saturation, Value) color space, the system can determine its position on
the screen.
As the object moves, its coordinates are recorded and used to simulate
freehand drawing on a virtual canvas. Additionally, the application incorporates
an intuitive user interface that allows users to switch between multiple colors—
Blue, Green, Red, and Yellow—by hovering the object over designated on-
screen buttons. A "Clear All" button is also available to reset the canvas with a
simple gesture.
This project combines the power of image processing and gesture
recognition to deliver a touchless, engaging drawing experience. It encourages
creativity, offers an interactive way to learn computer vision, and demonstrates
how technology can be used to develop contact-free human-computer
interaction systems for various applications like education, art, and accessibility
tools.
Required Libraries & Tools
To develop the Air Drawing application, several essential tools and libraries are
used to implement real-time computer vision functionalities, object tracking,
and drawing mechanisms. Below is a detailed description of each component:
1. Python 3.x:
Python serves as the core programming language for this project.
Its simplicity, readability, and vast ecosystem of libraries make it
ideal for rapid development of computer vision applications.
2. OpenCV (Open Source Computer Vision Library):
OpenCV is the primary library used in this project for image
processing and real-time computer vision tasks. It provides
functions to capture video from the webcam, convert images to
HSV color space, create masks for color detection, find contours,
and draw on both the live frame and virtual canvas.
3. NumPy:
NumPy is used for handling numerical operations and matrix
manipulations. It helps in defining kernels for image morphological
transformations like erosion and dilation, and is essential for
efficient pixel-level operations.
4. Webcam (Built-in or External):
A webcam is necessary to capture live video input. The application
utilizes the webcam feed to detect and track the colored object in
real time.
5. Optional Tools – Jupyter Notebook / IDE:
Development can be carried out in any Python-supporting IDE
such as PyCharm, VS Code, or Jupyter Notebook for better
visualization, debugging, and iterative testing during development.
Methodology
The Air Drawing application is built using real-time computer vision
techniques to allow users to draw on a virtual canvas by moving a colored
object in front of a webcam. This project leverages OpenCV and NumPy
libraries in Python to perform color detection, object tracking, and user
interaction through gesture-based inputs. The following steps describe the
working of the system in a detailed and sequential manner:
colors = [(255, 0, 0), (0, 255, 0), (0, 0, 255), (0, 255, 255)]
colorIndex = 0
# Keep looping
while True:
# Reading the frame from the camera
ret, frame = cap.read()
#Flipping the frame to see same side of yours
frame = cv2.flip(frame, 1)
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
# Adding the colour buttons to the live frame for colour access
frame = cv2.rectangle(frame, (40,1), (140,65), (122,122,122), -
1)
frame = cv2.rectangle(frame, (160,1), (255,65), colors[0], -1)
frame = cv2.rectangle(frame, (275,1), (370,65), colors[1], -1)
frame = cv2.rectangle(frame, (390,1), (485,65), colors[2], -1)
frame = cv2.rectangle(frame, (505,1), (600,65), colors[3], -1)
cv2.putText(frame, "CLEAR ALL", (49, 33),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)
cv2.putText(frame, "BLUE", (185, 33), cv2.FONT_HERSHEY_SIMPLEX,
0.5, (255, 255, 255), 2, cv2.LINE_AA)
cv2.putText(frame, "GREEN", (298, 33), cv2.FONT_HERSHEY_SIMPLEX,
0.5, (255, 255, 255), 2, cv2.LINE_AA)
cv2.putText(frame, "RED", (420, 33), cv2.FONT_HERSHEY_SIMPLEX,
0.5, (255, 255, 255), 2, cv2.LINE_AA)
cv2.putText(frame, "YELLOW", (520, 33),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (150,150,150), 2, cv2.LINE_AA)
blue_index = 0
green_index = 0
red_index = 0
yellow_index = 0
paintWindow[67:,:,:] = 255
elif 160 <= center[0] <= 255:
colorIndex = 0 # Blue
elif 275 <= center[0] <= 370:
colorIndex = 1 # Green
elif 390 <= center[0] <= 485:
colorIndex = 2 # Red
elif 505 <= center[0] <= 600:
colorIndex = 3 # Yellow
else :
if colorIndex == 0:
bpoints[blue_index].appendleft(center)
elif colorIndex == 1:
gpoints[green_index].appendleft(center)
elif colorIndex == 2:
rpoints[red_index].appendleft(center)
elif colorIndex == 3:
ypoints[yellow_index].appendleft(center)
# Append the next deques when nothing is detected to avois
messing up
else:
bpoints.append(deque(maxlen=512))
blue_index += 1
gpoints.append(deque(maxlen=512))
green_index += 1
rpoints.append(deque(maxlen=512))
red_index += 1
ypoints.append(deque(maxlen=512))
yellow_index += 1