Advanced Driver Assist Systems

Rajiv Ranjan Gupta

Lecture 08 – Lane Detection & YOLO
 Quick Recap !
Previous Module Contents
• Convolutional Neural Network
– What are Convolutional Neural Networks ?
– Convolutional Operation
– ReLU Layer – Rectifier Linear Network
– Pooling
– Flattening
– Full Connection
• What is TensorFlow ?
• Introduction to Tensors
• Introduction Numpy – Array Operations
 Scope
Module Contents
• Lane Detection on Video using OpenCV only
• YOLO Framework – Object Detection
• How does the YOLO Framework Function?
 Lane Detections using OpenCV
Using OpenCV and Python

Source - towardsdatascience
 Lane Detections using OpenCV
Applying Canny Detector
• We will feed in our sample video for lane detection as a series of continuous frames (images)

• The Canny Detector is a multi-stage algorithm optimized for fast real-time edge detection. The
fundamental goal of the algorithm is to detect sharp changes in luminosity (large gradients),
such as a shift from white to black, and defines them as edges, given a set of thresholds. The
Canny algorithm has four main stages:

1. Noise Reduction
2. Intensity gradient
3. Non-maximum suppression
4. Hysteresis thresholding

Source - towardsdatascience
 Lane Detections using OpenCV
Applying Canny Detector
1. Noise reduction -
As with all edge detection algorithms, noise is a crucial issue that often leads to false
detection. A 5x5 Gaussian filter is applied to convolve (smooth) the image to lower the
detector’s sensitivity to noise. This is done by using a kernel (in this case, a 5x5 kernel) of
normally distributed numbers to run across the entire image, setting each pixel value
equal to the weighted average of its neighbouring pixels.

Source - towardsdatascience
 Lane Detections using OpenCV
Applying Canny Detector
2. Intensity gradient -
 Lane Detections using OpenCV
Applying Canny Detector
2. Intensity gradient -

To be more graphical, let's assume we have a 1D-image. An edge is shown by the

"jump" in intensity in the plot below:

Source - docs.opencv.org
 Lane Detections using OpenCV
Applying Canny Detector
2. Intensity gradient -

The edge "jump" can be seen more easily if we take the first derivative (actually, here
appears as a maximum)

So, from the explanation above, we can deduce that a method to detect edges in an
image can be performed by locating pixel locations where the gradient is higher than
its neighbors (or to generalize, higher than a threshold) Source - docs.opencv.org
 Lane Detections using OpenCV
Applying Canny Detector
2. Intensity gradient -

We calculate two derivatives:

Horizontal changes: This is computed by convolving I with a kernel Gx with odd

size. For example for a kernel size of 3, Gx would be computed as:

Vertical changes: This is computed by convolving I with a kernel Gy with odd size.
For example for a kernel size of 3, Gy would be computed as:

Note - Assuming that the image to be operated is I Source - docs.opencv.org

 Lane Detections using OpenCV
Applying Canny Detector
2. Intensity gradient -
The smoothened image is then applied with a Sobel, Roberts, or Prewitt kernel (Sobel is
used in OpenCV) along the x-axis and y-axis to detect whether the edges are horizontal,
vertical, or diagonal.

Source - towardsdatascience
 Lane Detections using OpenCV
Applying Canny Detector
3. Non Maximum Supression -
Non-maximum suppression is applied to “thin” and effectively sharpen the edges. For
each pixel, the value is checked if it is a local maximum in the direction of the gradient
calculated previously.

A is on the edge with a vertical direction. As gradient is normal to the edge direction,
pixel values of B and C are compared with pixel values of A to determine if A is a local
maximum. If A is local maximum, non-maximum suppression is tested for the next
point. Otherwise, the pixel value of A is set to zero and A is suppressed.

Source - towardsdatascience
 Lane Detections using OpenCV
Applying Canny Detector
4. Hysterisis-
After non-maximum suppression, strong pixels are confirmed to be in the final map of
edges. However, weak pixels should be further analyzed to determine whether it
constitutes as edge or noise. Applying two pre-defined minVal and maxVal threshold
values, we set that any pixel with intensity gradient higher than maxVal are edges and
any pixel with intensity gradient lower than minVal are not edges and discarded. Pixels
with intensity gradient in between minVal and maxVal are only considered edges if they
are connected to a pixel with intensity gradient above maxVal.

Edge A is above maxVal so is considered an edge. Edge B is in

between maxVal and minVal but is not connected to any edge
above maxVal so is discarded. Edge C is in between maxVal and
minVal and is connected to edge A, an edge above maxVal, so is
considered an edge.
 Lane Detections using OpenCV
Segmenting lane area
• We will handcraft a triangular mask to segment the lane area and discard the irrelevant areas
in the frame to increase the effectiveness of our later stages.

Source - towardsdatascience
 Lane Detections using OpenCV
Hough Transform
• In the Cartesian coordinate system, we can represent a straight line as y = mx + b by plotting y
against x. However, we can also represent this line as a single point in Hough space by plotting
b against m. For example, a line with the equation y = 2x + 1 may be represented as (2, 1) in
Hough space.

Source - towardsdatascience
 Lane Detections using OpenCV
Hough Transform
• For example, a point at (2, 12) can be passed by y = 2x + 8, y = 3x + 6, y = 4x + 4, y = 5x + 2, y
= 6x, and so on. These possible lines can be plotted in Hough space as (2, 8), (3, 6), (4, 4), (5,
2), (6, 0). Notice that this produces a line of m against b coordinates in Hough space.

Source - towardsdatascience
𝜌 = 𝑥 cos 𝜃 + 𝑦 sin 𝜃
Automatic Detection

Lane Detection OpenCV Algorithm and plan of work.

Capturing and decoding Conversion of the Image to Applying filters to reduce

video file frame by frame Grayscale noise in video frames

Finding the region of

Detecting lanes using Hough Edge Detection Using Canny
interest and working on that
line transform Edge detection method
part
● Capturing and decoding video file: We will capture the video using Video Capture object and after the capturing has been
initialized every video frame is decoded (i.e. converting into a sequence of images).

● Grayscale conversion of image: The video frames are in RGB format, RGB is converted to grayscale because
processing a single channel image is faster than processing a three-channel colored image.

● Reduce noise: Noise can create false edges, therefore before going further, it’s imperative to perform image smoothening.
Gaussian filter is used to perform this process.
● Canny Edge Detector: It computes gradient in all directions of our blurred image and traces the edges with large changes
in intensity.

5 5 5
4 4 4
3 3 3
● Region of Interest: This step is to take into account only the region covered by the road lane. A mask is created here,
which is of the same dimension as our road image. Furthermore, bitwise AND operation is performed between each
pixel of our canny image and this mask. It ultimately masks the canny image and shows the region of interest traced by
the polygonal contour of the mask.
• Hough Line Transform: The Hough Line Transform is a transform used to detect straight lines. The Probabilistic
Hough Line Transform is used here, which gives output as the extremes of the detected lines
Lane Detections using OpenCV

Demonstration !