SSRG International Journal of VLSI and Signal Processing Volume 10 Issue 2, 22-26, May-Aug 2023

ISSN: 2394–2584 / https://doi.org/10.14445/23942584/IJVSP-V10I2P104 © 2023 Seventh Sense Research Group®

Original Article

Advanced Driver Assistance System (ADAS) on FPGA

Mayank Kumar1, A. Niharika2, Kethireddy Anjali Reddy3, Harsh Gupta4, K. N. Pushpalatha5
1,2,3,4,5
Department of Electronics and Communication Engineering, Dayananda Sagar College of Engineering,
Karnataka, India

Received: 30 April 2023 Revised: 07 June 2023 Accepted: 22 June 2023 Published: 06 July 2023

Abstract - Advanced Driver-Assistance Systems (ADAS) can help drivers in the driving process and increase driving safety
by automatically detecting objects, doing basic classification, implementing safeguards, etc. ADAS integrates multiple
subsystems, including object detection, scene segmentation, lane detection, and so on. In this paper, we establish a
framework for computer vision features, i.e., lane detection, object detection, object distance estimation and traffic sign
recognition of ADAS. Modern machine learning algorithms like Canny edge detection for lane detection and a CNN-based
approach are used for object detection. The system deployed aims to achieve higher (Frames Per Second) FPS for one
channel of 55 FPS. The performance of FPGA is optimized by software and hardware co-design. Realization on the DE-10
Nano board with Cyclone V FPGA and a dual-core ARM Cortex A9, which meets real-time processing requirements. An
increasing amount of automotive electronic hardware and software involves significant changes in the modern automobile
design process to address the convergence of conflicting goals - increased reliability, reduced costs, and shorter development
cycles. The prospectus to tackle car accident occurrences is making ADAS even more critical. This paper proposes an
efficient solution for ADAS on FPGA.

Keywords - ADAS, CNN, FPGA, FPS, Machine Learning.

1. Introduction An FPGA board is required for a level 5 autonomous vehicle,

Safety improvements for automobiles in the past such as particularly one employing complex vision tasks.[3]
shatter-resistant glass, three-point seatbelts, and airbags -
were safety measures adapted to mitigate accidents. Now, This paper proposes an efficient solution for ADAS on
ADAS systems actively improve safety with the help FPGA. Specifically, the objectives of this work are listed as
of modern technology like embedded vision. [1] follows. (1) We propose a framework for ADAS which
integrates the tasks of lane detection, object detection,
Automobiles are the foundation of the next generation of distance estimation, and traffic light recognition and achieves
mobile-connected devices, with rapid advancements in comparable precision with task-specific models. (2) We
autonomous vehicles. Autonomous applications are divided optimize the system performance through the joint
into different chips, called systems on a chip (SoCs). These optimization of software and hardware. The system will be
chips connect sensors to actuators through interfaces and deployed on Altera Cyclone V FPGA to achieve 55 FPS for
high-performance electronic controller units (ECUs). one channel.

Self-driving cars use various applications and

2. Reference Study
technologies to gain 360-degree vision, both near and far. All existing car automation techniques use
That means hardware designs are using more advanced microprocessors. The microprocessors used are typically
process nodes to meet ever-higher performance targets while embedded systems designed for automotive applications.
reducing power and footprint demands. Conventionally, These microprocessors had to meet the stringent
ADAS had been implemented on application-specific requirements of real-time processing, reliability, and safety
complex impromptu solutions built especially for criticality. With the advent of FPGAs in the 1980s, they
automotives since they suppose a large market.[2]. This gained popularity because of their reprogramming ability and
research aims to deploy an FPGA for implementing ADAS. parallel processing capabilities.

Lane detection is done using edge detection algorithms.

Owing to the restrictions of software for allowing a real-
Canny Edge Detection and Sobel Edge Detection are the
time embedded system to be implemented
most used edge detection algorithms.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)

 Mayank Kumar et al./ IJVSP, 10(2), 22-26, 2023

The former is often considered superior to the latter due road. This paper uses the Canny edge detection [4] algorithm
to several reasons like Accurate Edge Localization and for detecting lanes, as described in Figure 2.
Automatic Thresholding. This paper uses the Canny edge
detection algorithm. Camera
Input

Traffic light recognition is part of an autonomous system

that is supposed to take appropriate decisions on-road—
traffic light detection in complex urban environments. A ADAS PuTTY
Model/Code/
combination of colour-based segmentation, pattern matching, VNC Server
and machine learning techniques like the YOLOv5 algorithm Processing

was employed to detect traffic lights. LXDE

HPS
3. Materials and Methods
One way to implement ADAS is to use an FPGA as the
primary hardware platform. An FPGA is a type of Autonomous Car
Output VNC Viewer
Simulator/ Prototype
programmable hardware that can be configured to perform a
wide range of digital functions. It is often used in Fig. 1 Block Diagram
applications where flexibility and fast response times are
important. The Hough transform [5] is an image-processing
technique used in edge detection to improve the accuracy and
The processing unit in an ADAS system is responsible for robustness of lane detection. It is used in a variety of other
analysing the information from the sensors and making applications, such as detecting circles, ellipses, and other
decisions based on that. Sometimes, this may involve running geometric shapes in images.
complex algorithms or machine learning models to interpret
the data and determine the best course of action. The The breakthrough of neural networks is that object
processing unit may be implemented using a microcontroller, detection no longer must be a hand-crafted coding exercise.
a processor, or an FPGA, depending on the application's Neural networks allow features to be learned automatically
specific requirements. The FPGA used in this project is the from training examples.[6]
Intel DE10 Nano. The DE10-Nano is a compact, low-power
development board featuring an Intel Cyclone V FPGA. The The pipeline for object detection and distance estimation
DE10-Nano is equipped with a dual-core Arm Cortex-A9 is seen in Figure 3. This research uses the YOLOv5 [7]
processor, which can be used to run software applications. architecture for object detection. Object detection, a use case
for which YOLOv5 is designed, involves creating features
from input images. These are passed through a prediction
Lane detection is a technology used in self-driving cars system to draw boxes called bounding boxes around objects
ADAS to identify the location and boundaries of lanes on the and predict their classes.

Gaussian Canny Edge

Gray Scale Smoothing Detection

Hough Region
Draw Lines Transform Masking

Fig. 2 Lane detection pipeline

23
 Mayank Kumar et al./ IJVSP, 10(2), 22-26, 2023

Input video Pre-Processing Object Detection Co-ordinates of

Bounding Boxes

Distance
Visualize Final Draw predictions corresponding to Distance
Results on image Bounding Boxes Estimation

Fig. 3 Object detection pipeline

An older version of YOLO introduced the idea of

learning anchor boxes based on the distribution of these
boxes in the custom dataset with K-means and genetic
learning algorithms. This is very important for custom tasks
because the set of bounding box sizes and locations may be
dramatically different from the pre-set bounding box anchors
in another dataset. The YOLOv5 network predicts bounding
boxes as deviations from a given set of anchor box
dimensions to make box predictions. This model is trained Fig. 4 Input to line detection pipeline
on the FLIR dataset.

The distance between the camera and a target object in

the surrounding environment is determined using size-based
distance approximation.[9] This model is trained on the
KITTI dataset.

4. Results and Discussion

4.1. Lane Detection
Figure 4 is the snapshot of an input fed to the lane Fig. 5 Lane detected output
detection pipeline. Figure 5 is the final output. The lane
detection feature is used to detect the lane where the car is CARLA[11][13] was used to visualize these results in a
currently present and the ones that may exist on either side of real-time environment.
the lane. The green overlay on the input gives a clear
understanding of the path ahead of a driver. This feature has
increased importance when it is integrated into an
autonomous driving system [10] wherein the car is itself
expected to take decisions. The lane departure warning
system provides a physical alarm to the driver about the
departure from the existing lane.

Fig 6 Visualization of CARLA

23
 Mayank Kumar et al./ IJVSP, 10(2), 22-26, 2023

4.2. Object Detection, Object Distance Estimation, Traffic The improvement in the performance of the lane
Light Detection detection system due to pre-processing on FPGA can be seen
Figure 7 describes the features of object detection, object in Table 1.
distance estimation and traffic light detection. The bounding
Table 1. Comparison Results
boxes have different colours for different objects – orange
for cars, red for people, etc. At the header of these bounding Factor FPGA Processor
boxes are the identified objects' labels and a number that lies Frames per Second 81.3 27.01
between 0 to 1 called objectness. Objectness indicates the Time for one image frame 36.99ms 77.84ms
accuracy with which the algorithm has identified the object.
The programming language for pre-processing is
Further, a green bounding box enclosing a traffic signal
Verilog hardware description language. The function works
signaling ‘green for go’ is also seen. In the case of traffic
in 12.3 ms. This means that the performance is 81.3 fps. This
light detection, the bounding box labels the signal identified.
result is 3.01 times better than the computer solution (2.5
GHz Intel Core i5) and 18.4 times better than the ARM
Cortex A9 solution (667 MHz).

According to the August 2016 Traffic Safety Facts

Research Note by the National Highway Traffic Safety
Administration (NHTSA)[15], “The Nation lost 35,092 people
in crashes on U.S. roadways during 2015.” This 7.2%
increase was “the largest percentage increase in nearly 50
years.” An analysis revealed that about 94% of those
accidents were caused by human error. The application of
Fig. 7 Object detection, distance estimation, traffic light detection this project is a stepping stone towards bringing a
transformation in the automobile industry towards safer
5. Conclusion roads and safer pedestrians. The work carried out in this
The implementation of ADAS on FPGA comes with paper has brought to light, promising areas of further
advantages like parallel processing capabilities of FPGA[14], research [16] in the ADAS system optimization field. Further
low power consumption and better throughput, along with a improvements by deep model compression and software-
great deal of cost reduction. hardware collaborative optimization[16] can be obtained.

References
[1] The Synopsys website, 2023. [Online]. Available: https://www.synopsys.com/designware-ip/technical-bulletin/deep-learning-dwtb-
q217.html
[2] Juan Borrego-Carazo et al., “Resource-Constrained Machine Learning for ADAS: A Systematic Review,” IEEE Access, vol. 8, pp.
40573-40598, 2020. [CrossRef] [Google Scholar] [Publisher Link]
[3] Hamish Simmonds et al., “Autonomous Vehicle Development Using FPGA for Image Processing,” 2019 International Conference on
Field-Programmable Technology, 2019. [CrossRef] [Google Scholar] [Publisher Link]
[4] Ghassan Mahmoud Husien Amer, and Ahmed Mohamed Abushaala, “Edge Detection Methods,” 2015 2nd World Symposium on Web
Applications and Networking, 2015. [CrossRef] [Google Scholar] [Publisher Link]
[5] A Complete Guide On Hough Transform, 2022. [Online]. Available: https://www.analyticsvidhya.com/blog/2022/06/a-complete-guide-
on-hough-transform/
[6] YOLO: Algorithm for Object Detection Explained [+Examples], 2023. [Online]. Available: https://www.v7labs.com/blog/yolo-object-
detection
[7] M. Karthi et al., “Evolution of YOLO-V5 Algorithm for Object Detection: Automated Detection of Library Books and Performace
validation of Dataset,” 2021 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical
Systems, 2021. [CrossRef] [Google Scholar] [Publisher Link]
[8] M.R. Ezilarasan, and J. Brittopari, “An Efficient FPGA-Based Adaptive Filter for ICA Implementation in Adaptive Noise Cancellation,”
SSRG International Journal of Electrical and Electronics Engineering, vol. 10, no. 1, pp. 117-127, 2023. [CrossRef] [Publisher Link]
[9] Shi-Huang Chen, and Ruie-Shen Chen, “Vision-Based Distance Estimation for Multiple Vehicles Using Single Optical Camera,” 2011
Second International Conference on Innovations in Bio-inspired Computing and Applications, 2011. [CrossRef] [Google Scholar]
[Publisher Link]
[10] Abdulhakam AM. Assidiq et al., “Real time Lane Detection for Autonomous Vehicles,” 2008 International Conference on Computer
and Communication Engineering, 2008. [CrossRef] [Google Scholar] [Publisher Link]

24
 Mayank Kumar et al./ IJVSP, 10(2), 22-26, 2023

[11] CARLA Simulator. [Online]. Available: https://carla.readthedocs.io/en/latest/download/

[12] E. Ganesan, and V. Sakthivel, “A Novel FPGA Design with Hybrid LUT / MUX Architecture,” SSRG International Journal of
Electronics and Communication Engineering, vol. 3, no. 11, pp. 6-8, 2016. [CrossRef] [Google Scholar] [Publisher Link]
[13] Hyeonjae Jeon et al., “CARLA Simulator-Based Evaluation Framework Development of Lane Detection Accuracy Performance Under
Sensor Blockage Caused by Heavy Rain for Autonomous Vehicle,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 9977-9984,
2022. [CrossRef] [Google Scholar] [Publisher Link]
[14] Alexander Stumpf et al., “Benefits of an FPGA based SRM Controller,” 2014 9th IEEE Conference on Industrial Electronics and
Applications, 2014. [CrossRef] [Google Scholar] [Publisher Link]
[15] Traffic Safety Facts. [Online]. Available: https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812318
[16] Issam W. Damaj, Jibran K. Yousafzai, and Hussein T. Mouftah, “Future Trends in Connected and Autonomous Vehicles: Enabling
Communications and Processing Technologies,” IEEE Access, vol. 10, pp. 42334-42345, 2022. [CrossRef] [Google Scholar] [Publisher
Link]
[17] Ahmad Shawahna, Sadiq M. Sait, and Aiman El-Maleh, “FPGA-Based Accelerators of Deep Learning Networks for Learning and
Classification: A Review,” IEEE Access, vol. 7, pp. 7823-7859, 2019. [CrossRef] [Google Scholar] [Publisher Link]

25