MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

CHAPTER 1
INTRODUCTION
Now a day’s traffic density on the streets increasing around the world tremendously.
It causes several problems on the day to day life of people. As we know that it is the era of
speed, so that nobody wants to wait for a long time at any cost. Everybody prefers to low
traffic density streets. Traffic congestion causes chaos on the road and it makes it difficult for
the commuters to travel. Many big cities are facing the problem. A country’s economy can be
affected immensely due to traffic jam. There are many reasons behind this problem among
which the common reasons are poor traffic management, cars changing lane, unplanned
stoppage etc. It has been observed in many cities that traffic lights are not working or are not
being followed by the drivers. To fight such situations traffic polices are placed and the
traffic congestions are handled by them manually. In congested cities, it is very tough to
handle huge traffic by a traffic police.

Considering traffic jam of some under developed cities, there is no doubt that the
traffic rules and infrastructures are lacking the standard. Even in the era of technology there is
hardly any technology implemented to reduce traffic congestion. Most traffic lights at the
signals are either not working or are not being followed by the drivers. Traffic lights are
changed based on the static time that has been set, not considering which lane/road is more
congested and needs to be cleared. Implementation of computer vision technology can help
mitigate the problem. By using computer vision we can detect vehicles, count vehicles and
monitor any unwanted situation on roads. One way to detect a real-time moving vehicle is
background subtraction method i.e. the difference between the current frame and background
model. The static background is identified at first. Then it is removed from the current frame
that is being detected to get the moving object. Thus it will help to calculate the density at
each lane.

This proposed system introduced a vehicle density-based traffic control system to

avoid above issues. This problem can be resolved by controlling the traffic density on the
roads. This system introduces a new method to control vehicle density by controlling the
traffic lights using Image processing. Vehicle density is measured using predefined classifiers
available in image processing. If the measured density is above the normal density (threshold

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 1

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

value) it passes an indication to the microcontroller which controls the projector and thereby
we can give appropriate traffic signal to display.

This system is a fully automated system that can replace the conventional pre-
determined fixed-time based traffic system with a dynamically managed traffic system. It can
also detect vehicle condition on road and auto-adjust the system according to the changing
road conditions which makes the system intelligent. The designed system can help solving
traffic problems in busy cities to a great extent by saving a significant amount of man-hours
that get lost waiting on jammed roads. This research focuses on factors, low-cost image
processing and traffic load balancing. Moreover we are also replacing the conventional traffic
light system with a more efficient system using LCD projector. This computer vision
technology can be used to reduce the traffic congestion and also helps to detect people who
are not wearing helmets to an extent.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 2

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

CHAPTER 2

COMPUTER VISION

2.1 What is computer vision?

Computer vision is an interdisciplinary field that deals with how computers can be

made to gain high-level understanding from digital images or videos. From the perspective
of engineering, it seeks to automate tasks that the human visual system can do. "Computer
vision is concerned with the automatic extraction, analysis and understanding of useful
information from a single image or a sequence of images. It involves the development of a
theoretical and algorithmic basis to achieve automatic visual understanding." As a scientific
discipline, computer vision is concerned with the theory behind artificial systems that extract
information from images. The image data can take many forms, such as video sequences,
views from multiple cameras, or multi-dimensional data from a medical scanner. As a
technological discipline, computer vision seeks to apply its theories and models for the
construction of computer vision systems.

Computer vision tasks include methods for acquiring, processing, analyzing and

understanding digital images, and extraction of high-dimensional data from the real world in
order to produce numerical or symbolic information, e.g. in the forms of decisions.
Understanding in this context means the transformation of visual images (the input of the
retina) into descriptions of the world that make sense to thought processes and can elicit
appropriate action. This image understanding can be seen as the disentangling of symbolic
information from image data using models constructed with the aid of geometry, physics,
statistics, and learning theory.

The scientific discipline of computer vision is concerned with the theory behind

artificial systems that extract information from images. The image data can take many forms,
such as video sequences, views from multiple cameras, multi-dimensional data from a 3D
scanner or medical scanning device. The technological discipline of computer vision seeks to
apply its theories and models to the construction of computer vision systems.

Sub-domains of computer vision include scene reconstruction, event detection, video

tracking, object recognition, 3D pose estimation, learning, indexing, motion
estimation, visual serving, 3D scene modeling, and image restoration.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 3

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

2.2 HISTORY OF COMPUTER VISION

In the late 1960s, computer vision began at universities which were

pioneering artificial intelligence. It was meant to mimic the human visual system, as a
stepping stone to endowing robots with intelligent behavior. In 1966, it was believed that this
could be achieved through a summer project, by attaching a camera to a computer and having
it "describe what it saw".

What distinguished computer vision from the prevalent field of digital image
processing at that time was a desire to extract three-dimensional structure from images with
the goal of achieving full scene understanding. Studies in the 1970s formed the early
foundations for many of the computer vision algorithms that exist today, including extraction
of edges from images, labeling of lines, non-polyhedral and polyhedral modeling,
representation of objects as interconnections of smaller structures, optical flow, and motion
estimation.

The next decade saw studies based on more rigorous mathematical analysis and
quantitative aspects of computer vision. These include the concept of scale-space, the
inference of shape from various cues such as shading, texture and focus, and contour models
known as snakes. Researchers also realized that many of these mathematical concepts could
be treated within the same optimization framework as regularization and Markov random
fields. By the 1990s, some of the previous research topics became more active than the
others. Research in projective 3-D reconstructions led to better understanding of camera
calibration. With the advent of optimization methods for camera calibration, it was realized
that a lot of the ideas were already explored in bundle adjustment theory from the field
of photogrammetric. This led to methods for sparse 3-D reconstructions of scenes from
multiple images. Progress was made on the dense stereo correspondence problem and further
multi-view stereo techniques. At the same time, variations of graph cut were used to
solve image segmentation. This decade also marked the first time statistical learning
techniques were used in practice to recognize faces in images. Toward the end of the 1990s, a
significant change came about with the increased interaction between the fields of computer
graphics and computer vision. This included image-based rendering, image morphing, view
interpolation, panoramic image stitching and early light-field rendering.

Recent work has seen the resurgence of feature-based methods, used in conjunction
with machine learning techniques and complex optimization frameworks. The advancement

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 4

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

of deep Learning techniques has brought further life to the field of computer vision. The
accuracy of deep learning algorithms on several benchmark computer vision data sets for
tasks ranging from classification, segmentation and optical flow has surpassed prior methods.

2.2 Why study Computer Vision?

 The most obvious answer is that there’s a fast-growing collection of useful

applications derived from this field of study. Here are just a handful of them:

 Image retrieval: Google Images uses content-based queries to search relevant images.
The algorithms analyze the content in the query image and return results based on best-
matched content.

 Gaming and controls: A great commercial product in gaming that uses stereo vision is
Microsoft Kindest.
 Surveillance: Surveillance cameras are ubiquitous at public locations and are used to
detect suspicious behaviors.
 Biometrics: Fingerprint, iris and face matching remains some common methods in
biometric identification.
 Smart cars: Vision remains the main source of information to detect traffic signs and
lights and other visual features.

I recently finished Stanford’s wonderful CS231n course on using Convolution for visual

recognition. Visual recognition tasks such as image classification, localization,
and detection are key components of Computer vision.

Recent developments in neural networks and deep learning approaches have greatly
advanced the performance of these state-of-the-art visual recognition systems. The course is a
phenomenal resource that taught me the details of deep learning architectures being used in
cutting-edge computer vision research. In this article, I want to share the 5 major computer
vision techniques I’ve learned as well as major deep learning models and applications using
each of them.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 5

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

2.2 RELATION BETWEEN COMPUTER VISION AND OTHER FIELDS

FIGURE 1:-RELATION BTWEEN COMPUTER VISION AND OTHER FIELDS

Artificial intelligence: Areas of artificial intelligence deal with autonomous planning

or deliberation for robotic systems to navigate through an environment. A detailed
understanding of these environments is required to navigate through them. Information about
the environment could be provided by a computer vision system, acting as a vision sensor and
providing high-level information about the environment and the robot.

Artificial intelligence and computer vision share other topics such as pattern
recognition and learning techniques. Consequently, computer vision is sometimes seen as a
part of the artificial intelligence field or the computer science field in general.

Physics: Solid-state physics is another field that is closely related to computer vision.
Most computer vision systems rely on image sensors, which detect electromagnetic radiation,
which is typically in the form of either visible or infra-red light. The sensors are designed
using quantum physics. The process by which light interacts with surfaces is explained using
physics. Physics explains the behavior of optics which are a core part of most imaging
systems. Sophisticated image sensors even require quantum mechanics to provide a complete
understanding of the image formation process. Also, various measurement problems in
physics can be addressed using computer vision, for example motion in fluids.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 6

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

Signal processing: Yet another field related to computer vision is signal processing.
Many methods for processing of one-variable signals, typically temporal signals, can be
extended in a natural way to processing of two-variable signals or multi-variable signals in
computer vision. However, because of the specific nature of images there are many methods
developed within computer vision which have no counterpart in processing of one-variable
signals. Together with the multi-dimensionality of the signal, this defines a subfield in signal
processing as a part of computer vision.

Neurobiology: A third field which plays an important role is neurobiology,

specifically the study of the biological vision system. Over the last century, there has been an
extensive study of eyes, neurons, and the brain structures devoted to processing of visual
stimuli in both humans and various animals. This has led to a coarse, yet complicated,
description of how "real" vision systems operate in order to solve certain vision-related tasks.
These results have led to a subfield within computer vision where artificial systems are
designed to mimic the processing and behavior of biological systems, at different levels of
complexity. Also, some of the learning-based methods developed within computer vision
(e.g. neural net and deep learning based image and feature analysis and classification) have
their background in biology.

Some strands of computer vision research are closely related to the study of biological
vision indeed, just as many strands of AI research are closely tied with research into human
consciousness, and the use of stored knowledge to interpret, integrate and utilize visual
information. The field of biological vision studies and models the physiological processes
behind visual perception in humans and other animals. Computer vision, on the other hand,
studies and describes the processes implemented in software and hardware behind artificial
vision systems. Interdisciplinary exchange between biological and computer vision has
proven fruitful for both fields.

Mathematics: Beside the above-mentioned views on computer vision, many of the

related research topics can also be studied from a purely mathematical point of view. For
example, many methods in computer vision are based on statistics, optimization or geometry.
Other fields:
Finally, a significant part of the field is devoted to the implementation aspect of
computer vision; how existing methods can be realized in various combinations of software
and hardware, or how these methods can be modified in order to gain processing speed.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 7

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

CHAPTER 3
COMPUTER VISION SYSTEM METHOD
3.1 COMPUTER VISION SYSTEM METHODLOGY

FIGURE 2:-BLOCK DIAGRAM OF COMPUTER VISION SYSTEM METHOD

Image acquisition: image acquisition is the first step in any of the visual processes.
Image acquisition is the creation of a digitally encoded representation of the visual
characteristics of an object, such as a physical scene or the interior structure of an object.
Preprocessing: preprocessing is a common name for operations with images at the lowest
level of abstraction both input and output are intensity images. The aim of preprocessing is an
improvement of the image data that suppresses unwanted distortions or enhances some image
features important for future processing.

Segmentation: In digital image processing and computer vision, image

segmentation is the process of partitioning a digital image into multiple segments
(sets of pixels, also known as image objects). The goal of segmentation is to simplify and/or
change the representation of an image into something that is more meaningful and easier to
analyze.[1][2] Image segmentation is typically used to locate objects and boundaries (lines,
curves, etc.) in images. More precisely, image segmentation is the process of assigning a
label to every pixel in an image such that pixels with the same label share certain
characteristics.

The result of image segmentation is a set of segments that collectively cover the entire
image, or a set of contours extracted from the image (see edge detection). Each of the pixels
in a region are similar with respect to some characteristic or computed property, such

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 8

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

as color, intensity, or texture. Adjacent regions are significantly different with respect to the
same characteristic(s).[1] When applied to a stack of images, typical in medical imaging, the
resulting contours after image segmentation can be used to create 3D reconstructions with the
help of interpolation algorithms like marching cubes.

Representation and description Representation and description almost always follow

the output of a segmentation stage, which usually is raw pixel data, constituting either the
boundary of a region or all the points in the region itself. Choosing a representation is only
part of the solution for transforming raw data into a form suitable for subsequent computer
processing. Description deals with extracting attributes that result in some quantitative
information of interest or are basic for differentiating one class of objects from another.

Recognition and interpretation: recognition means is a process that assigns a label to

an object based on the information provided by its descriptors. Interpretation means assigning
meaning to a recognized object.

Knowledge base: Knowledge may be as simple as detailing regions of an image

where the information of interest is known to be located, thus limiting the search that has to
be conducted in seeking that information. The knowledge base also can be quite complex,
such as an interrelated list of all major possible defects in a materials inspection problem or
an image database containing high-resolution satellite images of a region in connection with
change-detection applications.

3.2 COMPUTER VISION V/S HUMAN VISION

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 9

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

FIGURE 3:-HUMAN VISION AND COMPUTER VISION

Vision is an amazing feat of natural intelligence. More human brain devoted to vision
than anything else. First off, the objectives of the two systems are the same: to convert light
into useful signals from which to construct accurate models of the physical world. Similarly,
when considered at a high level, the structures of human and computer vision are somewhat
similar: both have light sensors which convert photons into a signal, a transfer mechanism to
carry the signal, and an “understanding place” where the signal is interpreted.But when you
trace the path of a light signal through a computer vision technology vs. a human vision
system, many differences emerge. Well in relative field of view, Human field of view is 220
degrees, ie everything that is in front of us and a little bit more. But computer vision systems
can have 360 degree field of view, and there is no “front” and “back”.

Computer vision technology is mostly uniform across all parts of the field of view.
Compare this with human vision, where what we are best at seeing varies across the field of
view. For instance, humans tend to see color better at the center of the visual field.
Conversely we can detect low light objects better at the periphery. There are two types of
light sensor in the human visual system whereas computer vision sensors don’t have this
specialization. Sensor cells on the human retina brake down into rods and cones.

Rods are way more numerous (outnumbering cones about 20:1) and are more
sensitive to the overall light level meaning they are the ones we use to see in low light
conditions. They are also more concentrated outside the center of the field of view allowing
us to see in the dark but delivering lower visual resolution. Cones can see different colors and
deliver stronger resolution, but don’t work in the dark and are more found in the center. So
while there is a similarity between computer vision technologies and human vision in that
both have millions of tiny light sensors, beyond this basic point, the sensors for each system
are pretty different.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 10

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

FIGURE 4:-COMPUTER VISION V/S HUMAN VISION

Hence humans can see the object the signal is sent from eyes to brain. Hence brain
can recognize and detect the object on its knowledge whereas, in computer vision camera
captures the image and it sends to computer, computer compare it with the knowledge base
and gives the result according to the comparison done.

3.3 Difference between computer vision and image processing

Sl. no Computer vision Image processing

1 Goal is to understand the image and its The goal is to enhance an image for later
contents. use, then this may be called image.

2 Research area within computer science, Research area within electrical

artificial intelligence. engineering, signal processing.

3 Focus on semantic, symbolic and Focus on syntax and low-level

geometric descriptions. architectures.

4 Concerned with how images reflect the Mostly concerned with image to image
3D world. transformations.

5 Input is image or video and output is Input is image and output is image.
object.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 11

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

3.3 TYPICAL TASKS

Each of the application areas described above employ a range of computer vision
tasks; more or less well-defined measurement problems or processing problems, which can
be solved using a variety of methods. Some examples of typical computer vision tasks are
presented below.

Computer vision tasks include methods for acquiring, processing, analyzing and

understanding digital images, and extraction of high-dimensional data from the real world in
order to produce numerical or symbolic information, e.g., in the forms of
decisions. Understanding in this context means the transformation of visual images (the input
of the retina) into descriptions of the world that can interface with other thought processes
and elicit appropriate action. This image understanding can be seen as the disentangling of
symbolic information from image data using models constructed with the aid of geometry,
physics, statistics, and learning theory.

RECOGNITION

The classical problem in computer vision, image processing, and machine vision is that
of determining whether or not the image data contains some specific object, feature, or
activity. Different varieties of the recognition problem are described in the literature.

 Object recognition (also called object classification) – one or several pre-specified

or learned objects or object classes can be recognized, usually together with their 2D
positions in the image or 3D poses in the scene. Blippar, Google Goggles and LikeThat
provide stand-alone programs that illustrate this functionality.
 Identification – an individual instance of an object is recognized. Examples include
identification of a specific person's face or fingerprint, identification of handwritten
digits, or identification of a specific vehicle.
 Detection – the image data are scanned for a specific condition. Examples include
detection of possible abnormal cells or tissues in medical images or detection of a vehicle
in an automatic road toll system. Detection based on relatively simple and fast
computations is sometimes used for finding smaller regions of interesting image data
which can be further analyzed by more computationally demanding techniques to
produce a correct interpretation.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 12

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

Currently, the best algorithms for such tasks are based on convolution neural networks.
An illustration of their capabilities is given by the Image Net Large Scale Visual Recognition
Challenge; this is a benchmark in object classification and detection, with millions of images
and hundreds of object classes. Performance of convolution neural networks, on the Image
Net tests, is now close to that of humans. [28] The best algorithms still struggle with objects
that are small or thin, such as a small ant on a stem of a flower or a person holding a quill in
their hand. They also have trouble with images that have been distorted with filters (an
increasingly common phenomenon with modern digital cameras). By contrast, those kinds of
images rarely trouble humans. Humans, however, tend to have trouble with other issues. For
example, they are not good at classifying objects into fine-grained classes, such as the
particular breed of dog or species of bird, whereas convolution neural networks handle this
with ease.

Several specialized tasks based on recognition exist, such as:

 Content-based image retrieval – finding all images in a larger set of images which
have a specific content. The content can be specified in different ways, for example in
terms of similarity relative a target image (give me all images similar to image X), or in
terms of high-level search criteria given as text input (give me all images which contain
many houses, are taken during winter, and have no cars in them).

 Pose estimation – estimating the position or orientation of a specific object relative to

the camera. An example application for this technique would be assisting a robot arm in
retrieving objects from a conveyor belt in an assembly line situation or picking parts from
a bin.
 Optical character recognition (OCR) – identifying characters in images of printed
or handwritten text, usually with a view to encoding the text in a format more amenable
to editing or indexing (e.g. ASCII).
 2D code reading – reading of 2D codes such as data matrix and QR codes.
 Shape Recognition Technology (SRT) in people counter systems differentiating
human beings (head and shoulder patterns) from objects

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 13

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

Motion analysis

Several tasks relate to motion estimation where an image sequence is processed to

produce an estimate of the velocity either at each points in the image or in the 3D scene, or
even of the camera that produces the images. Examples of such tasks are:

 Ego motion – determining the 3D rigid motion (rotation and translation) of the
camera from an image sequence produced by the camera.
 Tracking – following the movements of a (usually) smaller set of interest points or
objects (e.g., vehicles, humans or other organisms) in the image sequence.
 Optical flow – to determine, for each point in the image, how that point is moving
relative to the image plane, i.e., its apparent motion. This motion is a result both of how
the corresponding 3D point is moving in the scene and how the camera is moving relative
to the scene.

SCENE RECONSTRUCTION

Given one or (typically) more images of a scene, or a video, scene reconstruction aims
at computing a 3D model of the scene. In the simplest case the model can be a set of 3D
points. More sophisticated methods produce a complete 3D surface model. The advent of 3D
imaging not requiring motion or scanning, and related processing algorithms is enabling rapid
advances in this field. Grid-based 3D sensing can be used to acquire 3D images from multiple
angles. Algorithms are now available to stitch multiple 3D images together into point clouds
and 3D models.

IMAGE RESTORATION

The aim of image restoration is the removal of noise (sensor noise, motion blur, etc.)
from images. The simplest possible approach for noise removal is various types of filters
such as low-pass filters or median filters. More sophisticated methods assume a model of
how the local image structures look, to distinguish them from noise. By first analyzing the
image data in terms of the local image structures, such as lines or edges, and then controlling
the filtering based on local information from the analysis step, a better level of noise removal
is usually obtained compared to the simpler approaches.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 14

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

3.5 HARDWARE

There are many kinds of computer vision systems; however, all of them contain these
basic elements: a power source, at least one image acquisition device (camera, ccd, etc.), a
processor, and control and communication cables or some kind of wireless interconnection
mechanism. In addition, a practical vision system contains software, as well as a display in
order to monitor the system. Vision systems for inner spaces, as most industrial ones, contain
an illumination system and may be placed in a controlled environment. Furthermore, a
completed system includes many accessories such as camera supports, cables and connectors.

Most computer vision systems use visible-light cameras passively viewing a scene at
frame rates of at most 60 frames per second (usually far slower).

A few computer vision systems use image-acquisition hardware with active

illumination or something other than visible light or both, such as structured-light 3D
scanners, thermo graphic cameras, hyper spectral, radar imaging, LIDAR scanners, magnetic
resonance images, side-scan sonar, synthetic aperture sonar, etc. Such hardware captures
"images" that are then processed often using the same computer vision algorithms used to
process visible-light images.

While traditional broadcast and consumer video systems operate at a rate of 30 frames
per second, advances in digital signal processing and consumer graphics hardware has made
high-speed image acquisition, processing, and display possible for real-time systems on the
order of hundreds to thousands of frames per second. For applications in robotics, fast, real-
time video systems are critically important and often can simplify the processing needed for
certain algorithms. When combined with a high-speed projector, fast image acquisition
allows 3D measurement and feature tracking to be realized.

3.5 APPLICATIONS

Applications range from tasks such as industrial machine vision systems which, say, inspect
bottles speeding by on a production line, to research into artificial intelligence and computers
or robots that can comprehend the world around them. The computer vision and machine
vision fields have significant overlap. Computer vision covers the core technology of
automated image analysis which is used in many fields. Machine vision usually refers to a
process of combining automated image analysis with other methods and technologies to
provide automated inspection and robot guidance in industrial applications. In many
DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 15
 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

computer-vision applications, the computers are pre-programmed to solve a particular task,

but methods based on learning are now becoming increasingly common.

MEDICINE

One of the most prominent application fields is medical computer vision, or medical
image processing, characterized by the extraction of information from image data to diagnose
a patient. An example of this is detection of tumors, arteriosclerosis or other malign changes;
measurements of organ dimensions, blood flow, etc. are another example. It also supports
medical research by providing new information: e.g., about the structure of the brain, or about
the quality of medical treatments. Applications of computer vision in the medical area also
includes enhancement of images interpreted by humans ultrasonic images or X-ray images
for example to reduce the influence of noise.

MACHINE VISION

A second application area in computer vision is in industry, sometimes

called machine vision, where information is extracted for the purpose of supporting a
manufacturing process. One example is quality control where details or final products are
being automatically inspected in order to find defects. Another example is measurement of
position and orientation of details to be picked up by a robot arm. Machine vision is also
heavily used in agricultural process to remove undesirable food stuff from bulk material, a
process called optical sorting.

MILITARY

Military applications are probably one of the largest areas for computer vision. The
obvious examples are detection of enemy soldiers or vehicles and missile guidance. More
advanced systems for missile guidance send the missile to an area rather than a specific
target, and target selection is made when the missile reaches the area based on locally
acquired image data. Modern military concepts, such as "battlefield awareness", imply that
various sensors, including image sensors, provide a rich set of information about a combat
scene which can be used to support strategic decisions. In this case, automatic processing of
the data is used to reduce complexity and to fuse information from multiple sensors to
increase reliability.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 16

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

AUTONOMOUS VEHICLES

One of the newer application areas is autonomous vehicles, which

include submersibles, land-based vehicles (small robots with wheels, cars or trucks), aerial
vehicles, and unmanned aerial vehicles (UAV). The level of autonomy ranges from fully
autonomous (unmanned) vehicles to vehicles where computer-vision-based systems support a
driver or a pilot in various situations. Fully autonomous vehicles typically use computer
vision for navigation, e.g. for knowing where it is, or for producing a map of its environment
(SLAM) and for detecting obstacles. It can also be used for detecting certain task specific
events, e.g., a UAV looking for forest fires. Examples of supporting systems are obstacle
warning systems in cars, and systems for autonomous landing of aircraft. Several car
manufacturers have demonstrated systems for autonomous driving of cars, but this
technology has still not reached a level where it can be put on the market. There are ample
examples of military autonomous vehicles ranging from advanced missiles to UAVs for
recon missions or missile guidance. Space exploration is already being made with
autonomous vehicles using computer vision.

TACTILE FEEDBACK

Materials such as rubber and silicon are being used to create sensors that allow for
applications such as detecting micro undulations and calibrating robotic hands. Rubber can be
used in order to create a mold that can be placed over a finger, inside of this mold would be
multiple strain gauges. The finger mold and sensors could then be placed on top of a small
sheet of rubber containing an array of rubber pins. A user can then wear the finger mold and
trace a surface. A computer can then read the data from the strain gauges and measure if one
or more of the pins is being pushed upward. If a pin is being pushed upward then the
computer can recognize this as an imperfection in the surface. This sort of technology is
useful in order to receive accurate data of the imperfections on a very large surface.
[25]
 Another variation of this finger mold sensor are sensors that contain a camera suspended
in silicon. The silicon forms a dome around the outside of the camera and embedded in the
silicon are point markers that are equally spaced. These cameras can then be placed on
devices such as robotic hands in order to allow the computer to receive highly accurate tactile
data.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 17

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

CHAPTER 4
TRAFFIC CONTROL USING COMPUTER VISION
Now a day’s traffic density on the streets increasing around the world tremendously.
It causes several problems on the day to day life of people. As we know that it is the era of
speed, so that nobody wants to wait for a long time at any cost. Everybody prefers to low
traffic density streets. This proposed system introduced a vehicle density-based traffic control
system to avoid above issues. This problem can be resolved by controlling the traffic density
on the roads. This system introduces a new method to control vehicle density by controlling
the traffic lights using Image processing. Vehicle density is measured using predefined
classifiers available in image processing. If the measured density is above the normal density
(threshold value) it passes an indication to the microcontroller which controls the projector
and thereby we can give appropriate traffic signal to display.

One important application of computer vision is traffic monitoring and control. Here
we are presenting a system for detection of moving vehicles approaching an intersection or in
a highway by camera in the context of traffic light control systems. As the system is
dedicated to outdoor applications, efficient and robust vehicle detection under various
weather and illumination conditions is examined. To deal with these ever changing
conditions, vehicle detection relies on motion segmentation and color mapping to achieve
feature space segmentation. Experimental results using real outdoor sequences of images
demonstrate the system's robustness under various environmental conditions. It detects the
number of vehicles on each road and depending on the vehicles load on each road, this
system assigns optimized amount of waiting time (red signal light) and running time (green
signal light). This system is a fully automated system that can replace the conventional pre-
determined fixed-time based traffic system with a dynamically managed traffic system. It can
also detect vehicle condition on road and auto-adjust the system according to the changing
road conditions which makes the system intelligent. The designed system can help solving
traffic problems in busy cities to a great extent by saving a significant amount of man-hours
that get lost waiting on jammed roads. This research focuses on factors, low-cost image
processing and traffic load balancing. Moreover we are also replacing the conventional traffic
light system with a more efficient system using LCD projector. This computer vision
technology can be used to reduce the traffic congestion and also helps to detect people who
are not wearing helmets to an extent.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 18

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

Open CV is written in C++ and its primary interface is in C++, but it still retains a less
comprehensive though extensive older C interface. There are bindings in Python, Java and
MATLAB/OCTAVE. The API for these interfaces can be found in the online documentation.
Wrappers in other languages such as C#, Perl, Ch, Haskell and Ruby have been developed to
encourage adoption by a wider audience. All of the new developments and algorithms in
Open CV are now developed in the C++ interface. If the library finds Intel's Integrated
Performance Primitives on the system, it will use these proprietary optimized routines to
accelerate it. A CUDA based GPU interface has been in progress since September 2010. An
Openly-based GPU interface has been in progress since October 2012, documentation for
version 2.4.13.3 can be found at docs.opencv.org.

FIGURE 4:-Block diagram of vehicle detection

This system detects the vehicle moving through the road. The below given block
diagram describes the process of vehicle detection. First the primary process is the feeding of
input video. This input video is then given to the next stage which loads the trained model.
This trained model then helps in determining the count of the vehicle. Now we get a count of
the vehicle. As a next stage we are going to find out the traffic dental region. After getting the
most dental region, now as the final stage we are allowing time slots.

In the case of helmet detection, as a first stage, i.e., the training stage, we should
collect all the image database available to us. Then it should undergo the preprocessing stage
where it processes all the input data given to it. Next stage is the segmentation. In this stage
the already preprocessed image will now undergo segmentation where it is divided into
different segments for processing. After the segmentation phase it now undergo feature

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 19

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

extraction, which itself is a selection process in which only the needed parts are selected.
Now we have the image after the feature extraction. This input is then feed into ML training
for training purposes. After certain ML processes it is given to the trained model and as a
result we will obtain the final trained model.

Figure 5:-Block diagram of testing system

In the testing phase, as shown in the above diagram, new data is fed as the input to the
preprocessing stage in which it undergoes certain cleaning and processing techniques. After
the preprocessing stage it will as usual undergo segmentation. After segmentation it
undergoes the feature extraction process and the output of it is loaded to training model. From
the below trained model, it will determine whether the helmet should be detected or not
detected. This is the training/ validation phase of helmet detection. From the loaded trained
model we will know that the helmet can be detected or not detected.

This system successfully implemented the vehicle detection and helmet detection
using image processing and machine learning methods. This system uses open CV as a
software, and uses the concept of image processing. It is used to detecting the density of
traffic on road and according to the density we are controlling the traffic. The language used
here is C++ and python. Since we are using Open CV the entire cost of the project is
minimized, easy setup, good accuracy and speed. There for the traffic signaling system
become automatic using image processing through Open CV. Machine learning is used to
teach system. Input (picture of vehicles) is given to training models. System will identify
vehicles through machine learning. After identifying the vehicles, it counts the number and
according to the density a dynamic timing is set.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 20

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

The below given is the final output image of helmet detection and vehicle detection detected
in the road.

Helmet detection Vehicle detection

CHAPTER 5
DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 21
 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

FUTURE SCOPE
The current state of computer vision technology is powered by deep learning
algorithms that utilize a special kind of neural networks, Convolution Neural Networks
(CNNs), to make sense of images. These neural networks are trained to use thousands of
sample images to assist the algorithm to comprehend and break down everything that is
comprised of an image.
CNN scan images pixel by pixel which recognize patterns and memorizes them.
These networks also memorize the ideal output that they should offer for each input image or
categorizes components of images by scanning characteristics like contours and colors.
Afterward, this memory is practiced by the systems as the reference while scanning more
images, and with every iteration, the AI system becomes better at providing the precise
output.In several sectors computer vision technology is being used or tested, including
healthcare – diagnosing diseases by assessing images received from CT scans and other
medical procedures; Manufacturing by leveraging this, manufacturers ensure manufacturing
processes and finished products for non-conformance and imperfections; Security using
biometric analysis like retinal, facial recognition and fingerprint scanning to recognize
individuals for security purposes. Computer vision also is being leveraged in transportation to
guide autonomous vehicles by detecting hurdles, people, and road signs along the way. As
per the market research and insights, the automotive industry is one of the early adopters of
computer vision systems and continues to lead the largest share of the computer vision
market among other industries in the industrial vertical.Since computer vision swells its
impact in the human world, there are numerous things to contemplate on how it will alter the
way people see their lives and how they live it. With additional research on and fine-tuning of
the technology, the future of computer vision will see it perform a wide array of functions.
The technology will not only be easier to train but also be able to discern more from images
than they perform now.

Computer vision technologies will also be utilized alongside other technologies or

other subsets of AI to create more compelling applications. Image captioning applications, for
instance, where it can be integrated with natural language generation (NLG) to interpret the
objects in the surroundings for visually challenged people. In the development of artificial
general intelligence (AGI) and artificial superintelligence (ASI), computer vision also can
portray a significant role by providing them the ability to process information or even better
than the human visual system.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 22

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

As a rising market, computer vision technology is firmly connected to visual and augmented
reality (VR and AR). Indeed, the market players in recent time have a keen interest in the
amalgamation of VR and AR. And such attention is growing at a rapid pace which is being
reflected in the introduction of many state-of-the-art technological products.

CHAPTER 6

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 23

 MANAGEMENT OF IOT SENSOR DATA USING A FOG COMPUTING NODE

CONCLUSION
Technology of the vision of the computer, input after the computer by signal
number value, change several signal, and via treated to form figures or images of vision,
this include static image or and level three-dimensional cartoon, several image to deal
with by several course create work , vision of picture mainly. Computer vision to utilize
several science and technology of computer is it deal with figure or image wait for
vision behave to come mainly. Deal with and include computer graphics, the image is
duplicated or the especially good effect is dealt with, three-dimensional space picture
simulation, fictitious visual effect appear, the international image of the multimedia
appears and computer animation, etc.

It is very extensive that the vision of the computer is applied to the research on
automation, and as perform algorithms under gradual progress and constant
development, the use of the computer vision is more extensive. The ones that are used
most extensively now are combining type neural network and fuzzy theory, make the
accuracy of the computer vision higher, have intelligent functions even more, after
training, can reach cleverer request. For example: Utilize the fuzzy theory to transfer the
photo to a perceptivity figure of 3D, enable users to utilize and does not watch the
picture with the angle, can cut down and increase the picture too, already there are
systems of practical application abroad at present. And this research direction is applied
to the printed circuit board, the measuring of the agricultural products, the medical
products are discerned, build the design of the space. Waiting a moment, there are a
large number of relevant research theses constantly at present.

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING, BCE, SHRAVANABELAGOLA 24