Real-Time Automotive Night Vision

System
An Image Processing Approach

Presented by

Gouranga Mandal
Department of Computer Science and Engineering
National Institute of Technology Agartala
Outlines
• Introduction & Problem Statement
• State-of-the-art and their Drawbacks
• Proposed Approach and Projected Framework
• Experimental Results and Discussion
• Processing time analysis
• Quantitative performance
• Method-wise Comparison
• Conclusion
• Publication
Introduction & Problem Statement
• Over 1.35 million people in the world die per year due to road accidents.
• Maximum accidents occurs at night/evening
• Main cause: The high-beam glare of today’s powerful headlight blinds the eyes of a
driver
• It is the most dangerous situation for a driver during the night
• Drivers are unable to see the road condition, road direction, objects on roads, and
surrounding areas
• Important reasons:
• High beam headlight glare from opposite direction vehicle
• very low light intensity and unclear view of the road due to darkness.
Sample frame with high-beam headlight

• High-beam headlight Glare of

oncoming vehicles at night

• Driver feels temporary

blindness

• High Chance of accidents

Drawbacks of existing night vision systems:

• People generally try to solve the problem by using:

• color windshield glass,
• color spectacle or
• night vision glass

• As a result:
• The color glass in a vehicle or spectacle sometimes reduces the high beam
headlight glare a little bit, but the view of the road gets darker
 Drawbacks of existing night vision systems:

• Few car companies are recently using a dash

camera (Dash-cam) at the front of the car to
acquire live video for image enhancement (such as BMW 5, 6 & 7 Series, Audi A8,
to solve the night vision problem faced by a
driver. Toyota Crown Hybrid & Landcruiser,
Mercedes-Benz E, S & CL-Class, Lexus LS &
X470, Cadillas DeVille, Honda Legend, etc)

• However, it tries to suppress the intensity of

the headlight beam. But, it also decreases
the light of the road view.
Drawbacks of existing night vision systems:

• Unable to dim the high beam headlights of oncoming vehicles

without affecting the road view.

• Used high cost sensors.

• Unable to provide bright road views inhibiting high-intensity

headlight beam at the same time.
As result:

• Drivers feel temporary blindness

• Many Accidents due to high-beam headlight at night

• Almost 50% of traffic deaths happen at night.

• Many people lost their valuable lives.

Proposed Approach:

Real-time automotive night vision system for drivers to inhibit headlight-glare of the
oncoming vehicles and enhance road visibility

• Drivers can get a bright and a prominent view of the road with dim headlights of oncoming
vehicles in real-time.

• Drivers can drive comfortably in presence of oncoming vehicle with high headlight beam at
night.

• Can also be used in autonomous vehicles.

• Minimum processing time.

Proposed Approach:

The primary contributions of this approach:

• Region-based distinct local image enhancement techniques are applied

• A dynamic-patch based novel low-light vision enhancement method is introduced for
low-light region. A smaller patch size is provided to darker pixels
• To reduce the huge processing time:
• The value of atmospheric light (A) is recalculated only when a new state starts.
• The adaptive Gaussian filter (a post-processing step) is not applied to the entire frame, it is applied only on
two common boundary areas of three regions. It smooths the border of regions.
Projected Framework
Region segmentation

Comparison of visual representation of output frames by few popular segmentation techniques

(a) Original frame, Segmentation using (b) Prewitt (c) Sobel (d) Log Gaussian (e) Canny (f) Proposed Combined
Method (Modified Canny Edge-Detector with huge threshold sigma (σ) + Seeded Region-Growing)
Region segmentation

Process of segmentation of a frame with high beam & glare.

a) The original frame b) Core Headlight portion (Region 1) is extracted c) Glare portion (region 2)
is extracted d) Three regions (1, 2, and 3) are segmented.
Masking of high-intensity beam for region 1

• Region 1: Core headlight region

• Target: to reduce high-intensity light

• A masking function is defined to reduce the

brightness of each pixel (intensity > P1) by
subtracting a positive threshold value (here 80
is used) to make the pixels darker

• Besides, it reduces the area of the core

Masking function to reduce the brightness of region 1
headlight region by 50 percent
Gamma correction for region 2

step 1. Input image

step 2. Determine the maximum pixel intensity
step 3. Select the value of Gamma
step 4. Look-Up table formation
step 5. Mapping of input pixel in the Look-Up table
Step 6. Output Image Variation of light intensity for different gamma (γ) value

Where (V) is voltage, (I) is intensity,

and (γ) is gamma

Mapping of image pixels to the values in look-up table

Low-light vision enhancement for region 3

Region 3: Darker view of the road The atmospheric light (A): avg. intensity (RGB)
of brightest 0.1% pixels (K)
The atmospheric light scattering model

The transmission t(p)

local patch

The ultimate scene radiance J(p)

Highest value of intensity
Adaptive gaussian filtering: to smooth the boundary of regions

• Additive Gaussian Filter gives the best output for most of the frames with respect
to smoothness and clearing of border spots in between the two regions.

• It is applied only on the boundary and surrounding pixels (not to the whole
frame) for smoothing the final output.

• Eight adjacent pixels in all directions of all boundary pixels are considered for
filtering.
Experimental Results: presence on oncoming vehicle

Experimental result of a frame with the high-intensity headlight of an oncoming bike in the evening by the proposed system.

(a) Original Frame (b) After enhancement of region 1 & 2 (c) Obtained the highest intensity by dynamic-patch (d) Transmission
map (e) Final output after low light enhancement of region 3 and adaptive Gaussian filter.
Experimental Results: without any oncoming vehicle

Low-light enhancement of frames (real-

time frames and frames from “BDD
Dataset (BDD100K)” [64] ) with no high
beam headlight of any oncoming vehicle
(a) Original frame
(b) Obtained the highest intensity by
dynamic-patch
(c) Transmission map
(d) Output frame.
Change of luminance before and after enhancement
Real-time test result with accuracy
Result of the proposed method on BDD dataset
Failure Cases

Frames, which are not enhanced

properly
(a) Original frame
(b) Obtained highest intensity by
dynamic-patch
(c) Transmission map
(d) Output frame.
Processing time analysis

Main strategies behind such a less processing time are as follows:

• In region 3, the atmospheric light (A) is not calculated in every

frame, it is calculated only when a new state starts.
• The adaptive Gaussian filter is not applied to all the pixels of
frames. It is applied only on two boundary areas.
• Smoothing is applied only if a frame consists of any high beam
headlight of an oncoming vehicle.
Quantitative performance

Quantitative performance assessment for the enhancement methods. (a) NIQMC (No-reference free
energy-based robust-metric for contrast-distortion) (b) BIQME (blind image quality measure of enhanced
images)
Method-wise Comparison
Conclusion
• Provide real-time clear real-time vision for driver at night

• Drivers can get a bright and a prominent view of the road with dim headlights of oncoming
vehicles in real-time.

• Drivers can drive comfortably in presence of oncoming vehicle with high headlight beam at
night.

• Maximum visibility and Minimum computation time

• Reduce risk, Passengers will be safe, Pedestrians can walk safely

• Road accidents, death and injuries, and delay in reaching the destination will also reduce.
Publication:
• Mandal, G., Bhattacharya, D. & De, P. Real-time automotive night-vision system for drivers to inhibit headlight glare of
the oncoming vehicles and enhance road visibility. Journal of Real-Time Image Processing (SCIE Indexed, IF: 1.968),
(2021). https://doi.org/10.1007/s11554-021-01104-z
Thank You