Embedded Night-Vision System For Pedestrian Detection - Doc
Embedded Night-Vision System For Pedestrian Detection - Doc
ABSTRACT
The paper describes the use of thermal camera and IR night vision system
for the detection of Pedestrians and objects that may cause accident at night
time. As per the survey most of the accidents cause is due to low vision ability
of human at night time, which leads to most dangerous and higher number of
accidents at night with respect to day time. This system include the IR night
vision camera which detects the object with the help of IR LED and photodiode
pair, this camera have capability to detect the object up to 100m. The thermal
camera detects the heat generated by any of the object like cars, Human animals
etc. which gives us the facility to detect the object for higher range and with low
reflective surface where IR night vision may fails. With the use of these two
cameras mounted on car which helps the driver to drive safely. In this system,
HOG (Histogram of orientated gradients) algorithm and support vector machine
(SVM) is performed with the help of OpenCV in Matlab and EmguCV in Visual
Basic 2012. The system is tested on the video recorded using these cameras, and
got good and efficient result. And this system is cost efficient and easy to
implement.
INTRODUCTION
The paper describes the use of thermal camera and IR night vision system
for the detection of Pedestrians and objects that may cause accident at night
time. As per the survey most of the accidents cause is due to low vision ability
of human at night time, which leads to most dangerous and higher number of
accidents at night with respect to day time. This system include the IR night
vision camera which detects the object with the help of IR LED and photodiode
pair, this camera have capability to detect the object up to 100m. The thermal
camera detects the heat generated by any of the object like cars, Human animals
etc. which gives us the facility to detect the object for higher range and with low
reflective surface where IR night vision may fails. With the use of these two
cameras mounted on car which helps the driver to drive safely. In this system,
HOG (Histogram of orientated gradients) algorithm and support vector machine
(SVM) is performed with the help of OpenCV in Matlab and EmguCV in Visual
Basic 2012. The system is tested on the video recorded using these cameras, and
got good and efficient result. And this system is cost efficient and easy to
implement.
LITERATURE SURVEY
The Local Binary Patterns (LBP) [4] operator is one of the most
successful texture descriptors and has been widely used in various applications.
The idea of this operator is to assign each pixel a code depending on the gray
levels of its neighborhoods. The gray level of the central pixel (ic) of
coordinates (xc, yc) is compared to that of its neighbors (in). This descriptor is
robust against monotonous gray scale changes caused, and against lighting
variations. Another important property that is its simplicity of calculation, which
allows analyzing images in difficult settings in real time.
Another issue with these systems is their inability to generalize across different
thermal imaging devices or different ambient temperature ranges. The
performance of traditional pedestrian detection systems can degrade
significantly when the surrounding temperature matches that of the human body,
reducing the contrast between the pedestrian and the environment. Additionally,
in many cases, the real-time processing capability of traditional algorithms is
limited due to the high computational load required for feature extraction and
classification in thermal images, making them less suitable for fast-moving
applications like autonomous driving.
Despite these efforts, high false positives and missed detections remain key
challenges, limiting the overall reliability of traditional night-vision pedestrian
detection systems.
DISADVANTAGES
1. Limited Detection Range: Night-vision systems typically have a shorter
detection range compared to daylight systems, making it harder to detect
distant pedestrians.
2. Poor Performance in Adverse Weather: Systems often struggle in rain,
fog, or snow, as these conditions can obscure thermal imaging or infrared
signals.
3. False Positives and False Negatives: High rates of misclassifications,
such as detecting non-pedestrian objects or missing actual pedestrians,
reduce system reliability.
4. High Power Consumption: Embedded night-vision systems, especially
those using active infrared, can drain significant power, limiting their use
in battery-operated devices.
5. Expensive Hardware: Thermal cameras and infrared sensors are costly,
making the system expensive to implement and maintain.
6. Low Resolution: Many night-vision systems provide lower image
resolution, leading to less detailed images and making pedestrian
identification less accurate.
7. Latency in Real-time Processing: Real-time pedestrian detection can be
slow due to the high computational load of image processing in low light,
resulting in delayed responses.
8. Limited Field of View: Narrow field of view in some systems restricts
the area covered, increasing the risk of missing pedestrians outside the
system’s range.
9. Vulnerability to Ambient Light Interference: Sudden exposure to
headlights or streetlights can cause momentary blindness in infrared or
thermal-based systems.
10.Sensor Calibration Issues: Night-vision systems often require frequent
calibration to maintain accuracy, which can be difficult and time-
consuming in embedded systems.
PROPOSED SYSTEM
Pedestrian detection in low-light or nighttime conditions presents significant
challenges for intelligent transportation systems and autonomous driving. This
project proposes an embedded night-vision system for pedestrian detection
that leverages the capabilities of deep learning, specifically using the YOLOv2
(You Only Look Once, version 2) Convolutional Neural Network (CNN)
model. The system is designed to be integrated into vehicles or surveillance
platforms, where it can operate effectively in poor lighting environments to
identify pedestrians in real-time.
Now a days with advance technologies drivers may get lots of information from
sensors such as upcoming traffic signals, diversions, traffic conditions and many
more information but this sensors may not provide accurate information about
pedestrian or any other objects at night due to darkness or low quality cameras.
To overcome from this problem author is evaluating performance of YOLOV2
CNN (convolution neural networks) object detection model but this model also
unable to detect objects from NIGHT VISION.
In propose work to detect objects from NIGHT VISION author using HAAR
HOG descriptor with ADABOOST algorithm and this algorithm providing
better detection compare to YOLOV2 and its false detection rate is also less.
This algorithm will clear the image using OPENCV and ADABOOST and then
apply HAAR HOG features to detect pedestrian in that image.
In propose work I am using 6 night vision images and YOLOV2 able to detect
pedestrian from 4 images and ADABOOST able to detect pedestrian from all 6
images but it is detecting some false images also as pedestrian due to this reason
ADABOOST detection accuracy will be 80% and YOLOV2 detection accuracy
will be 4/6 = 0.66.
ADABOOST detection rate = 6/6 * 100 = 100% - 20 (for false detection rate) =
80%
➢ RAM - 4 GB (min)
➢ Hard Disk - 20 GB
SOFTWARE REQUIREMENTS:
ECONOMICAL FEASIBILITY
TECHNICAL FEASIBILITY
SOCIAL FEASIBILITY
ECONOMICAL FEASIBILITY
This study is carried out to check the economic impact that the system
will have on the organization. The amount of fund that the company can pour
into the research and development of the system is limited. The expenditures
must be justified. Thus the developed system as well within the budget and this
was achieved because most of the technologies used are freely available. Only
the customized products had to be purchased.
TECHNICAL FEASIBILITY
This study is carried out to check the technical feasibility, that is, the
technical requirements of the system. Any system developed must not have a
high demand on the available technical resources. This will lead to high
demands on the available technical resources. This will lead to high demands
being placed on the client. The developed system must have a modest
requirement, as only minimal or null changes are required for implementing this
system.
SOCIAL FEASIBILITY
The goal is for UML to become a common language for creating models of
object oriented computer software. In its current form UML is comprised of two
major components: a Meta-model and a notation. In the future, some form of
method or process may also be added to; or associated with, UML.
The UML represents a collection of best engineering practices that have proven
successful in the modeling of large and complex systems.
The UML is a very important part of developing objects oriented software and
the software development process. The UML uses mostly graphical notations to
express the design of software projects.
GOALS:
User
Night Vision Pedestrain Detection
using HAAR + AdaBoost
Exit
CLASS DIAGRAM:
user.
upload Nigh t Vision image
night vision pedestrain detection using YOLOV2
night vision pedestrain detection using HAAR+Adaboost
Exit
SEQUENCE DIAGRAM:
User System
Application
Exit
ACTIVITY DIAGRAM:
Collaboration diagram:
The biggest strength of Python is huge collection of standard library which can
be used for the following –
Machine Learning
Test frameworks
Multimedia
Advantages of Python :-
1. Extensive Libraries
Python downloads with an extensive library and it contain code for various
purposes like regular expressions, documentation-generation, unit-testing, web
browsers, threading, databases, CGI, email, image manipulation, and more. So,
we don’t have to write the complete code for that manually.
2. Extensible
As we have seen earlier, Python can be extended to other languages. You can
write some of your code in languages like C++ or C. This comes in handy,
especially in projects.
3. Embeddable
4. Improved Productivity
5. IOT Opportunities
Since Python forms the basis of new platforms like Raspberry Pi, it finds the
future bright for the Internet Of Things. This is a way to connect the language
with the real world.
6. Simple and Easy
When working with Java, you may have to create a class to print ‘Hello World’.
But in Python, just a print statement will do. It is also quite easy to
learn, understand, and code. This is why when people pick up Python, they have
a hard time adjusting to other more verbose languages like Java.
7. Readable
Because it is not such a verbose language, reading Python is much like reading
English. This is the reason why it is so easy to learn, understand, and code. It
also does not need curly braces to define blocks, and indentation is mandatory.
This further aids the readability of the code.
8. Object-Oriented
Like we said earlier, Python is freely available. But not only can you download
Python for free, but you can also download its source code, make changes to it,
and even distribute it. It downloads with an extensive collection of libraries to
help you with your tasks.
10. Portable
When you code your project in a language like C++, you may need to make
some changes to it if you want to run it on another platform. But it isn’t the
same with Python. Here, you need to code only once, and you can run it
anywhere. This is called Write Once Run Anywhere (WORA). However, you
need to be careful enough not to include any system-dependent features.
11. Interpreted
Any doubts till now in the advantages of Python? Mention in the comment
section.
Advantages of Python Over Other Languages
1. Less Coding
Almost all of the tasks done in Python requires less coding when the same task
is done in other languages. Python also has an awesome standard library
support, so you don’t have to search for any third-party libraries to get your job
done. This is the reason that many people suggest learning Python to beginners.
2. Affordable
The 2019 Github annual survey showed us that Python has overtaken Java in
the most popular programming language category.
Python code can run on any machine whether it is Linux, Mac or Windows.
Programmers need to learn different languages for different jobs but with
Python, you can professionally build web apps, perform data analysis
and machine learning, automate things,do web scraping and also build games
and powerful visualizations. It is an all-rounder programming language.
Disadvantages of Python
So far, we’ve seen why Python is a great choice for your project. But if you
choose it, you should be aware of its consequences as well. Let’s now see the
downsides of choosing Python over another language.
1. Speed Limitations
We have seen that Python code is executed line by line. But since Python is
interpreted, it often results in slow execution. This, however, isn’t a problem
unless speed is a focal point for the project. In other words, unless high speed is
a requirement, the benefits offered by Python are enough to distract us from its
speed limitations.
The reason it is not so famous despite the existence of Brython is that it isn’t
that secure.
3. Design Restrictions
As you know, Python is dynamically-typed. This means that you don’t need to
declare the type of variable while writing the code. It uses duck-typing. But
wait, what’s that? Well, it just means that if it looks like a duck, it must be a
duck. While this is easy on the programmers during coding, it can raise run-time
errors.
4. Underdeveloped Database Access Layers
5. Simple
No, we’re not kidding. Python’s simplicity can indeed be a problem. Take my
example. I don’t do Java, I’m more of a Python person. To me, its syntax is so
simple that the verbosity of Java code seems unnecessary.
This was all about the Advantages and Disadvantages of Python Programming
Language.
History of Python : -
What do the alphabet and the programming language Python have in common?
Right, both start with ABC. If we are talking about ABC in the Python context,
it's clear that the programming language ABC is meant. ABC is a general-
purpose programming language and programming environment, which had been
developed in the Netherlands, Amsterdam, at the CWI (Centrum Wiskunde
&Informatica). The greatest achievement of ABC was to influence the design of
Python.Python was conceptualized in the late 1980s. Guido van Rossum worked
that time in a project at the CWI, called Amoeba, a distributed operating system.
In an interview with Bill Venners1, Guido van Rossum said: "In the early 1980s,
I worked as an implementer on a team building a language called ABC at
Centrum voor Wiskunde en Informatica (CWI). I don't know how well people
know ABC's influence on Python. I try to mention ABC's influence because I'm
indebted to everything I learned during that project and to the people who
worked on it."Later on in the same Interview, Guido van Rossum continued: "I
remembered all my experience and some of my frustration with ABC. I decided
to try to design a simple scripting language that possessed some of ABC's better
properties, but without its problems. So I started typing. I created a simple
virtual machine, a simple parser, and a simple runtime. I made my own version
of the various ABC parts that I liked. I created a basic syntax, used indentation
for statement grouping instead of curly braces or begin-end blocks, and
developed a small number of powerful data types: a hash table (or dictionary, as
we call it), a list, strings, and numbers."
What is Machine Learning : -
Before we take a look at the details of various machine learning methods, let's
start by looking at what machine learning is, and what it isn't. Machine learning
is often categorized as a subfield of artificial intelligence, but I find that
categorization can often be misleading at first brush. The study of machine
learning certainly arose from research in this context, but in the data science
application of machine learning methods, it's more helpful to think of machine
learning as a means of building models of data.
At the most fundamental level, machine learning can be categorized into two
main types: supervised learning and unsupervised learning.
Human beings, at this moment, are the most intelligent and advanced species on
earth because they can think, evaluate and solve complex problems. On the
other side, AI is still in its initial stage and haven’t surpassed human intelligence
in many aspects. Then the question is that what is the need to make machine
learn? The most suitable reason for doing this is, “to make decisions, based on
data, with efficiency and scale”.
Emotion analysis
Sentiment analysis
Speech synthesis
Speech recognition
Customer segmentation
Object recognition
Fraud detection
Fraud prevention
Arthur Samuel coined the term “Machine Learning” in 1959 and defined it as
a “Field of study that gives computers the capability to learn without being
explicitly programmed”.
And that was the beginning of Machine Learning! In modern times, Machine
Learning is one of the most popular (if not the most!) career choices. According
to Indeed, Machine Learning Engineer Is The Best Job of 2019 with
a 344% growth and an average base salary of $146,085 per year.
But there is still a lot of doubt about what exactly is Machine Learning and how
to start learning it? So this article deals with the Basics of Machine Learning
and also the path you can follow to eventually become a full-fledged Machine
Learning Engineer. Now let’s get started!!!
This is a rough roadmap you can follow on your way to becoming an insanely
talented Machine Learning Engineer. Of course, you can always modify the
steps according to your needs to reach your desired end-goal!
In case you are a genius, you could start ML directly but normally, there are
some prerequisites that you need to know which include Linear Algebra,
Multivariate Calculus, Statistics, and Python. And if you don’t know these,
never fear! You don’t need a Ph.D. degree in these topics to get started but you
do need a basic understanding.
Data plays a huge role in Machine Learning. In fact, around 80% of your time
as an ML expert will be spent collecting and cleaning data. And statistics is a
field that handles the collection, analysis, and presentation of data. So it is no
surprise that you need to learn it!!!
Some of the key concepts in statistics that are important are Statistical
Significance, Probability Distributions, Hypothesis Testing, Regression, etc.
Also, Bayesian Thinking is also a very important part of ML which deals with
various concepts like Conditional Probability, Priors, and Posteriors, Maximum
Likelihood, etc.
Some people prefer to skip Linear Algebra, Multivariate Calculus and Statistics
and learn them as they go along with trial and error. But the one thing that you
absolutely cannot skip is Python! While there are other languages you can use
for Machine Learning like R, Scala, etc. Python is currently the most popular
language for ML. In fact, there are many Python libraries that are specifically
useful for Artificial Intelligence and Machine Learning such
as Keras, TensorFlow, Scikit-learn, etc.
So if you want to learn ML, it’s best if you learn Python! You can do that using
various online resources and courses such as Fork Python available Free on
GeeksforGeeks.
Now that you are done with the prerequisites, you can move on to actually
learning ML (Which is the fun part!!!) It’s best to start with the basics and then
move on to the more complicated stuff. Some of the basic concepts in ML are:
Machine Learning can review large volumes of data and discover specific trends
and patterns that would not be apparent to humans. For instance, for an e-
commerce website like Amazon, it serves to understand the browsing behaviors
and purchase histories of its users to help cater to the right products, deals, and
reminders relevant to them. It uses the results to reveal relevant advertisements
to them.
With ML, you don’t need to babysit your project every step of the way. Since it
means giving machines the ability to learn, it lets them make predictions and
also improve the algorithms on their own. A common example of this is anti-
virus softwares; they learn to filter new threats as they are recognized. ML is
also good at recognizing spam.
3. Continuous Improvement
5. Wide Applications
You could be an e-tailer or a healthcare provider and make ML work for you.
Where it does apply, it holds the capability to help deliver a much more
personal experience to customers while also targeting the right customers.
Disadvantages of Machine Learning :-
1. Data Acquisition
Machine Learning requires massive data sets to train on, and these should be
inclusive/unbiased, and of good quality. There can also be times where they
must wait for new data to be generated.
ML needs enough time to let the algorithms learn and develop enough to fulfill
their purpose with a considerable amount of accuracy and relevancy. It also
needs massive resources to function. This can mean additional requirements of
computer power for you.
3. Interpretation of Results
4. High error-susceptibility
SYSTEM TEST
TYPES OF TESTS
Unit testing
Unit testing involves the design of test cases that validate that the internal
program logic is functioning properly, and that program inputs produce valid
outputs. All decision branches and internal code flow should be validated. It is
the testing of individual software units of the application .it is done after the
completion of an individual unit before integration. This is a structural testing,
that relies on knowledge of its construction and is invasive. Unit tests perform
basic tests at component level and test a specific business process, application,
and/or system configuration. Unit tests ensure that each unique path of a
business process performs accurately to the documented specifications and
contains clearly defined inputs and expected results.
Integration testing
Functional test
System Test
System testing ensures that the entire integrated software system meets
requirements. It tests a configuration to ensure known and predictable results.
An example of system testing is the configuration oriented system integration
test. System testing is based on process descriptions and flows, emphasizing
pre-driven process links and integration points.
White Box Testing is a testing in which in which the software tester has
knowledge of the inner workings, structure and language of the software, or at
least its purpose. It is purpose. It is used to test areas that cannot be reached
from a black box level.
Black Box Testing is testing the software without any knowledge of the inner
workings, structure or language of the module being tested. Black box tests, as
most other kinds of tests, must be written from a definitive source document,
such as specification or requirements document, such as specification or
requirements document. It is a testing in which the software under test is
treated, as a black box .you cannot “see” into it. The test provides inputs and
responds to outputs without considering how the software works.
Unit Testing
Unit testing is usually conducted as part of a combined code and unit test phase
of the software lifecycle, although it is not uncommon for coding and unit
testing to be conducted as two distinct phases.
Field testing will be performed manually and functional tests will be written in
detail.
Test objectives
Features to be tested
Test Results: All the test cases mentioned above passed successfully. No
defects encountered.
Acceptance Testing
Test Results: All the test cases mentioned above passed successfully. No
defects encountered.
IMPLIMENTATION
To test above 2 Algorithms I am using below NIGHT VISION images
In last image we can see we are unable to see any pedestrian but ADABOOST can detect it.
In above screen click on ‘Upload Night Vision Image’ button and upload image
In above screen I am selecting ‘test.png’ image and then click on ‘Open’ button to load image
and to get below screen
In above screen showing uploaded original image and hardly we can see the pedestrian and
now try to detect that pedestrian using YOLOV2 algorithm by clicking on ‘Night Vision
Pedestrian Detection using YOLOV2’ button
In above screen first image is the original image and second image is the YOLOV2 resultant
image and in second image we did not find any bounding box across pedestrian so YOLOV2
unable to detect that pedestrian and now click on ‘Night Vision Pedestrian Detection using
HAAR + AdaBoost’ button to get below result
In above screen first image is the original image and second image is the resultant image
from HAAR + ADABOOST algorithm and this algorithm able to detect pedestrian
successfully and putting bounding box across detected pedestrian.
Now test with other image
In above screen uploading 3.png and then below is the YOLOV2 result
In above image YOLOV2 able to detect the persons and now test with ADABOOST
In above screen we can see ADABOOST detecting both persons accurately. Similarly you
can upload other images and test the
CONCLUSION
In conclusion, the development of an embedded night-vision system for
pedestrian detection using the YOLOv2 CNN architecture has demonstrated
significant potential for improving safety in low-visibility conditions.
YOLOv2's real-time object detection capabilities, coupled with its ability to
identify pedestrians even in challenging lighting environments, make it a robust
solution for night-time applications. By leveraging infrared imaging and
optimizing the model for embedded systems, this approach addresses both
hardware constraints and performance requirements, ensuring effective
detection while maintaining low power consumption and real-time processing
capabilities.
However, there are challenges and areas for improvement, such as further
refining the system’s ability to cope with adverse weather conditions and
enhancing detection accuracy in highly cluttered environments. Future work
may focus on integrating more advanced sensors or hybrid models that combine
multiple detection methods to improve robustness and performance across
different night-time scenarios.
Finally, the future scope of this project can extend to enhancing user interaction
through the development of mobile applications that provide alerts to
pedestrians and drivers in real time. Such applications could integrate with
smart city infrastructure to communicate detection data to nearby vehicles,
thereby improving overall road safety. As cities continue to integrate technology
into their infrastructure, the embedded night-vision system can play a vital role
in promoting pedestrian safety, particularly during nighttime or low-visibility
conditions, ultimately contributing to the development of smarter, safer urban
environments.
REFERENCES
1. Zhang, L., & Zhang, L. (2017). A review of pedestrian detection
methods. International Journal of Advanced Computer Science and
Applications, 8(8), 100-107.
2. Zheng, S., & Wang, Y. (2018). Pedestrian detection via deep learning: A
survey. Journal of Computer Science and Technology, 33(1), 1-20.
4. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only
look once: Unified real-time object detection. In Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 779-
788.
6. Liu, W., Anguelov, D., & Goh, Y. (2016). SSD: Single shot multibox
detector. In European Conference on Computer Vision (ECCV), 21-37.
11.Lin, T., & Chen, Y. (2019). Nighttime pedestrian detection using fusion
of visible and infrared images. IEEE Transactions on Image Processing,
28(12), 5965-5976.
13.Xiao, Y., & Wang, Y. (2020). Efficient pedestrian detection for intelligent
transportation systems using YOLOv2 on Raspberry Pi. IEEE Access, 8,
76359-76366.
14.Song, J., & Wu, Z. (2019). A real-time pedestrian detection system for
smart vehicles based on YOLOv2. Journal of Intelligent Transportation
Systems, 23(3), 224-232.
17.Zhang, D., & Yang, X. (2019). Combining thermal and visible images
for pedestrian detection using YOLOv2. IEEE Transactions on Image
Processing, 28(5), 2236-2248.