Literature survey , code, methodology and

algorithm
Literature survey
This paper utilizes existing features for detecting facial landmarks. The step-by-step process of the
system is depicted in Figure 1. The method involves using a set of 68 predefined facial landmarks to
predict the shape of the face, allowing for the identification of different facial regions, such as the
eyebrows, eyes, and mouth, as illustrated in Figure 2.The facial landmark detection process is as
follows:
An image is input, and the face is identified using the Machine Learning Algorithm.
1. The 68 points are accurately detected to determine the (x, y) coordinates of various facial regions.
2. These landmarks are then localized to identify regions such as the eyes and mouth.
3. Any changes in the shape of these regions are used to indicate expressions related to drowsiness and
fatigue.
High-resolution cameras are integrated to monitor, capture, and extract frames from the video feed.
These frames are analyzed individually to study the patterns of facial features. This is done using Haar
Cascade Classifiers to calculate the Eye Aspect Ratio (EAR) and Mouth Aspect Ratio (MAR) for each
frame. If the EAR and MAR values exceed their respective thresholds, indicating a blink or a yawn, the
system considers this a sign of drowsiness. An alarm is triggered if consecutive frames show signs of
frequent blinking or yawning, alerting the driver by sounding continuously until they are fully awake.

Figure 1

Figure 2
Methodology
The process begins by identifying and locating the face within the image using a technique called facial
landmark detection. Once the face is pinpointed, shape prediction methods are employed to identify crucial
features on the face, such as the eyes, nose, and mouth. The detection of the face itself is performed using a set
of pre-built classifiers provided by OpenCV, specifically the HAAR cascades, which are well-known for their
accuracy in detecting objects like faces.In the next step, the system estimates the positions of 68 specific points
on the face, which are referred to as landmarks. These landmarks correspond to various facial structures, such as
the corners of the eyes, the tip of the nose, and the edges of the mouth. To achieve this, a pre-trained facial
landmark detector from the dlib library is used.

For detecting drowsiness, the Eye Aspect Ratio (EAR) is calculated by measuring the distances between certain
key points around the eyes. This ratio helps determine if the eyes are closing, which is a common sign of
drowsiness. Specifically, the ratio compares the width of the eye (the distance between the corners) with the
height (the distance between the upper and lower eyelids). When the eyes start to close, the EAR decreases,
indicating potential drowsiness.

For detecting yawning, the system calculates a value based on the distance between the upper and lower lips.
This distance is compared to a threshold value to determine if a yawn is occurring. Yawning, similar to blinking,
is another sign of drowsiness and can be used to assess the alertness of a person, particularly in situations where
staying awake is crucial, like driving.

To ensure the driver is alerted in case of drowsiness or frequent yawning, the system uses the espeak module.
This module converts text into spoken words, allowing the system to deliver voice alerts. These alerts are
designed to grab the driver's attention and help prevent accidents by encouraging them to take a break or stay
more focused on the road.

Step 1: Data Collection

The process starts by taking a picture using a camera.

Step 2: Data Preprocessing

After taking the picture, the system recognizes the face in it. Once the face is found, the system focuses on
detecting the eyes.

Step 3: Decision Making

The system then examines the images to determine whether the driver's eyes are open or closed. Based on this
evaluation, the system takes specific actions, such as sounding an alert or spraying water to wake up the driver.

Step 4: Performance Measurement

In this step, we check how well the system is working by measuring its accuracy and reliability based on the
results.

Step 5: Deployment
If the system meets expectations and delivers accurate results, it will be implemented for regular use.

Block Diagram for Drowsiness Detection.

 Video Frame Acquisition:

The process starts by capturing live video from a camera inside the vehicle, aimed at the driver’s face.

 Face Detection:

A method is used to find and isolate the driver’s face in each video frame for further analysis.
  Eye Tracking:

The system locates the eyes within the face and tracks them in each frame to monitor how long the eyes stay closed, how
often the driver blinks, and where the driver is looking to detect drowsiness.

 Mouth Tracking:

Similarly, the mouth is detected and tracked over time to check for yawning or other movements that might indicate
drowsiness.

 Drowsiness Detection:

The data from eye and mouth tracking, along with head position, is combined to assess the driver's level of drowsiness. If
certain signs are detected, the system triggers alerts.

ALGORITHM
1. Import Libraries: The program starts by loading the tools needed to handle video, process images, and detect
facial features.

2. Define Functions:

o alarm(msg): Sounds an alert if the driver seems drowsy or is yawning.

o eye_aspect_ratio(eye): Measures how open or closed the eyes are by checking the distance between key
points around the eyes.

o final_ear(shape): Combines the measurements from both eyes to get an overall view of how open the
driver’s eyes are.

o lip_distance(shape): Measures the distance between the upper and lower lips to check for yawning.

3. Set Up Webcam: Prepares the webcam to capture video of the driver’s face.

4. Load Detection Models: Loads tools to identify the face and specific facial features like eyes and mouth in the
video.

5. Start Video Stream: Begins capturing real-time video of the driver.

6. Main Loop:

o Continuously grabs frames (images) from the video.

o Each frame is resized and converted to black and white to make processing easier.

o Detects the face in the frame and then finds the eyes and mouth.

o Measures how open the eyes are and the distance between the lips.

o If the eyes are closed for too long or if yawning is detected, triggers an alarm.

o Shows information like blink rate, drowsiness count, and yawn count on the video feed.

7. Exit the Program: The program keeps running until the 'q' key is pressed, which stops everything and closes the
video feed.

CODE
 sys, time, os, subprocess: These are built-in Python libraries. They help with system tasks like interacting with the
operating system, handling time-related functions, and running external commands.

 scipy.spatial.distance, numpy: These libraries are used for mathematical tasks. For instance, they help measure
how far apart points are.

 cv2 (OpenCV): This library is for working with images and videos, including tasks like detecting faces in a video
feed.
  dlib: This library helps find specific facial features, like eyes and mouth, on a face.

 imutils: This library simplifies image processing tasks, such as resizing images.

 threading.Thread: This allows you to run tasks in the background, so the main program can keep running
smoothly.

Global Variables

 blink_count, DrowsyCount, YawnCount, alarm_status: These variables track various things like the number of
blinks, signs of drowsiness, and whether an alarm should be triggered.

Functions

 alarm(msg): This function handles alarm notifications. It triggers an alert if the system detects signs of drowsiness
or yawning.

 eye_aspect_ratio(eye): This function calculates a ratio to determine if the driver’s eyes are closed or open.

 final_ear(shape): Uses the eye aspect ratio to assess if the eyes are open or closed.

 lip_distance(shape): Measures the distance between the upper and lower lips to detect if the driver is yawning.

 update_ear_threshold(): Adjusts how sensitive the eye aspect ratio calculation is based on conditions like light,
though it uses a fixed value here.

 handle_drowsiness(): Checks if the driver is drowsy by monitoring how long their eyes stay closed.

Video Capture and Processing

 The program captures video from the webcam using OpenCV (cv2) and imutils.

 For each frame, it detects the driver’s face and then the eyes and mouth within that face.

 It calculates the openness of the eyes and whether the driver is yawning.

 Based on these calculations, it counts blinks, monitors for drowsiness, and checks for yawning.

Drowsiness and Yawn Detection

 If the driver’s eyes are closed for too long or if they are yawning a lot, the program triggers an alarm.

 If drowsiness or yawning is detected frequently, it can run an external script to send an email notification.

User Interface

 The program displays a video feed with information like blink rate, eye openness, drowsy count, and yawn count.

 The program continues running until the user presses the 'q' key to quit.

Cleanup

 When the program exits, it stops the video stream and closes all windows.

This program helps monitor a driver’s alertness by analyzing their eye and mouth movements through video, providing alerts
if they show signs of drowsiness or frequent yawning.