Computer Vision for Real-time Drowsiness

Detection System. Case Study: Image

Analyses of the Impact of Educational Games as
Analysis
Learning Strategy

by Mirlinda Ebibi
& Artan Ebibi
 Introduction
 Overall system Algorithm
 Methodology
 System implementation
 Face recognition
 Road analyses
 Conclusions, limitations, and future research
 References

Mirlinda Ebibi, PhD 2

Introduction

 Drowsy driving continues to be a major factor in road accidents globally,

posing a serious threat to public safety. Drowsy driving leads to numerous
deaths and injuries each year, underscoring the pressing need for
effective preventive measures.
 While traditional methods like driver education and rest
recommendations have been used to address this issue, they often fall
short in delivering immediate interventions to reduce the risks associated
with driver fatigue.

Mirlinda Ebibi, PhD 3

Introduction
 Recent advancements in computer vision technology have introduced new opportunities for
enhancing driver safety through automated drowsiness detection systems.
 By developing machine learning algorithms that analyze facial expressions, eye movements
and other physiological indicators, these systems can identify signs of drowsiness in real time
and issue timely alerts to drivers, potentially averting accidents proactively.
 Despite the promise of computer vision-based solutions, there remains a gap in
research regarding the optimization of image capture techniques for drowsiness
detection in vehicles.
 The effectiveness of these systems relies on the quality and reliability of the
captured images, which can be caused by factors like lighting conditions, camera
positioning, and vehicle vibrations.
 This paper aims to address this gap by exploring computer vision techniques for
optimizing image capture in real-time drowsiness detection systems.
4
OVERALL SYSTEM ALGORITHM
The drowsiness detection system operates as follows:
• The driver initiates the system by pressing the "Start Assist" button.
• Once activated, the system continuously monitors the driver’s condition
by capturing images every 0.25 seconds using a camera mounted in
front of the driver.
• These images are then sent to a database for processing by an
algorithm
• The camera captures rapid images of the driver’s eyes, face, and the
road. The algorithm analyses these images to assess the driver’s state.
• It specifically monitors the driver’s head movements and checks if the
eyes remain within a predefined threshold to determine alertness.
• A the backward-facing camera captures images of the road to ensure
the driver is not swerving and driving safe.

The dataset used by the algorithm includes images of eyes, faces, and
road conditions. By evaluating both the driver’s physical state and driving
behaviour, the system can detect signs of drowsiness and ensure the
driver is operating the vehicle safely
Mirlinda Ebibi, PhD 5
METHODOLOGY

The techniques used in this paper were helped by the OpenCV library of python by
using the methods below:

• Image Processing: Functions for manipulating and enhancing images, including

filtering, edge detection, and morphological transformations. Utilizing functions such as
cv2.filter2D() for filtering, cv2.Canny() for edge detection, and cv2.morphologyEx() for
morphological transformations
• Feature Detection and Matching: Methods to detect and match features, such as
facial landmarks, crucial for identifying signs of drowsiness. Employing methods like
cv2.detectMultiScale() for detecting faces and cv2.matchTemplate() for matching
features, crucial for identifying signs of drowsiness.
• Object Detection: Algorithms for detecting objects within images, such as the Haar
Cascade classifier for face and eye detection.

Mirlinda Ebibi, PhD 6

SYSTEM IMPLEMENTATION

IV.1 Image caputuring

The capuring of the images would be done by a camera no lower than 8-megapixels. Access to
the camera inside the vehicle is made possible by cv2.VideoCapture(0).

IV.2 Image processing

Haar Cascade Algorithm:

The technique of this system for face detection and recognition will be the Haar Cascade. The
reason why this algorithm is one of the best choices is that it can run in real time.

Haar cascade uses the cascading window, and it tries to compute features in every window
and classify whether it could be an object. This method will use the light and shadow to detect
objects in images.

Mirlinda Ebibi, PhD 7

Features
The object detection procedure relies on the extraction of simple features from images.
Unlike pixel-based methods, features provide a more efficient representation of
visual patterns and encode domain knowledge critical for effective classification.

• Two-Rectangle Feature: This feature computes the difference between the sum of
pixels within two horizontally or vertically adjacent rectangular regions.
• Three-Rectangle Feature: Here, the sum within two outside rectangles is subtracted
from the sum within a center rectangle.
• Four-Rectangle Feature: This feature calculates the difference between diagonal
pairs of rectangles.

Mirlinda Ebibi, PhD 8

Mirlinda Ebibi, PhD 9
EAR (Eye Aspect Ratio) evaluation and yawn detection:

• The EAR is a metric that determines eye openness or closeness.

• The system uses the Eye Aspect Ratio to determine the states which the driver will
go through, thus alarming him if he closes his eyes.
• The EAR is computed using the Euclidean distance in a vertical and horizontal
direction between the upper and lower eyelids.

• Another technique the system will use is PERCLOS.

• The PERCELOS (percentage of eye closure) metric is a fatigue detection metric that
calculates eye closure rate over the pupil in a given time frame.
• It is established on gradual and slower eyelid closures or eyelid sliding rather than
regular eye blinks.

Mirlinda Ebibi, PhD 10

We need to train the model by posing as drowsy and taking the pictures for the training
set. And after calculating the threshold, we can use it to determine the drowsiness for
other images. Mirlinda Ebibi, PhD 11
Yawn detection
• Yawning is also one of the main causes of drowsiness.
• Detecting it in a precise and correct manner is essential for the system.
• The calculation of the yawn will happen by getting the distance of the landmarks on the drivers lips
before and after the yawn.

Mirlinda Ebibi, PhD 12

ROAD ANALYSIS
• In addition to face recognition the importance of road analysis is essential. This
component focuses on detecting lane markings and whether the driver is
maintaining a proper lane discipline.
• The process for lane detection and tracking is done in several steps: capturing
images, enhancing the images for a more precise analysis and detection,
identifying lane markings using edge detection and Hough Transform, and
monitoring the vehicles position relative to the marked lanes.

Image preprocessing:

For a more simple processing we can turn the images into greyscale. Smoothing it out
and reducing the noise is also essential. With the help of OpenCV functions an example
of the converted image will be as follows:

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

blur = cv2.GaussianBlur(gray, (5, 5), 0)

Mirlinda Ebibi, PhD 13

Methods of simplifying the image – Canny edge detector:

Non-maximum suppressison:
Non-maximum suppression is used to thin the detected edges.
Double threasholding:
The threasholding is used to classify the strength of the pixels.
Edge tracking by Hysteresis:
Hysteresis is a technique used in edge detection, notably in algorithms like Canny
edge detection. It employs two thresholds: a high threshold and a low threshold.

Mirlinda Ebibi, PhD 14

Classification of edges
1. Strong Edges: Pixels with gradient values above the high threshold are classified as
strong edges, representing definitive boundaries in the image.

2. Weak Edges: Pixels with gradient values between the low and high thresholds are
labeled as weak edges. These are only considered edges if they are connected to strong
edges.

3. Noise Reduction: This method effectively minimizes noise by filtering out weak edges
that aren't linked to strong ones, preserving important features while eliminating less
relevant details.

Hysteresis -enhances edge representation in images, making it a valuable tool in

computer vision and image processing.
• Hysteresis is a technique, where it filters the pixels in a way that it keeps only
the strong edges, while reducing noise. It keeps the weak edges if and only if
they are directly connected to strong edges.
Mirlinda Ebibi, PhD 15
CONCLUSION
By integrating road analysis with drowsiness detection, this system presents a brilliant approach to enhancing
driving safety. Through the monitoring of both driver condition and lane swerving, it offers a solution for
completely avoiding the risks associated with drowsy driving. This approach not only alerts drivers to their own
fatigue but also alerts drivers to maintain lane discipline.
Integrating road analysis with drowsiness detection can greatly enhance driving safety through several important
features:
1. Real-Time Monitoring: The system continuously evaluates the driver’s alertness by analyzing metrics such as
eye movement, head position, and reaction times. This enables early identification of drowsiness before it leads
to dangerous situations.
2. Lane Detection: By assessing lane markings and vehicle positioning, the system can detect swerving or
unintentional lane departures. If the vehicle drifts out of its lane, the system will issue alerts to the driver.
3. Alerts and Interventions: When signs of drowsiness are detected, the system can provide visual or auditory
alerts to prompt the driver to take a break
4. Long-Term Benefits: Over time, the system can gather data on driving habits and fatigue patterns, offering
personalized feedback and recommendations. This can help drivers improve their habits and lower the risk of
drowsy driving incidents.

Mirlinda Ebibi, PhD 16

Conclusion

5. Potential Integration with Autonomous Systems: As autonomous driving technology

evolves, this drowsiness detection system could be integrated into semi-autonomous
vehicles, allowing for smoother transitions between manual and automated driving, further
enhancing safety.

By addressing both driver alertness and road awareness, this integrated approach aims to
significantly reduce accidents caused by drowsy driving, ultimately making roads safer for
everyone.

Mirlinda Ebibi, PhD 17

 References

[1] National Highway Traffic Safety Administration (NHTSA). "Drowsy Driving and Automobile Crashes." [Online]. Available: NHTSA.
[2] OpenCV Documentation. "Open Source Computer Vision Library." [Online]. Available: OpenCV.
[3] P. Viola and M. Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features," Proceedings of the 2001 IEEE Comp uter Society
Conference on Computer Vision and Pattern Recognition. CVPR 2001.
[4] F. Bergasa et al., "Real-time system for monitoring driver vigilance," IEEE Transactions on Intelligent Transportation Systems, vol. 7, no. 1, pp. 63 -77,
March 2006
[5] G. Eason, B. Noble, and I. N. Sneddon, “On certain integrals of Lipschitz-Hankel type involving products of Bessel functions,” Phil. Trans. Roy. Soc.
London, vol. A247, pp. 529–551, April 1955. (references)
[6] . Fleuret and D. Geman. Coarse-to-fine face detection. Int. J. Computer Vision, 2001.
[7] William T. Freeman and Edward H. Adelson. The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence
[8] Review on Drowsiness Detection, D. Khasim Vali

Mirlinda Ebibi, PhD 18

 Mirlinda Ebibi, PhD

Any questions?
Mirlinda Ebibi, PhD 19