2014

Pi Cars Raspberry

CONTAL Rebecca /MESTRE

Coralie/RICHTER Cannelle/ SAMOUN
Lirone

22/01/2014
 Pi cars Raspberry 20/01/2014

Acknowledgements

Before presenting this report, we would like to thank those who have helped us to carry
out our objectives:

 First of all, Mr. Precioso, our teacher tutor, that submitted our project topic. But
also for his valuable help, his availability and his advices. He also provided us the
electronic components that we needed and the RC cars.
 Mme Lingrand , for her help in image processing
 Mr Nedelec, for providing us computer screens.
 We would also like to thank the MAM department, for having offered us the
opportunity to work on this project.

CONTAL/MESTRE/RICHTER/SAMOUN Page 1
 Pi cars Raspberry 20/01/2014

Contents
Acknowledgements ................................................................................................................................. 1
I. Introduction ..................................................................................................................................... 4
II. First part .......................................................................................................................................... 5
A. Hardware ..................................................................................................................................... 5
1. The Raspberry Pi hardware ..................................................................................................... 5
2. Servo-motors assembly ........................................................................................................... 6
3. Linking the Raspberry Pi to the RC car controller.................................................................... 7
4. The cameras and the lights ..................................................................................................... 7
B. Software ...................................................................................................................................... 8
1. The OS...................................................................................................................................... 8
2. Wireless ................................................................................................................................... 9
3. Librairies installation ............................................................................................................... 9
4. Remote-controlling the Raspberry .......................................................................................... 9
III. Behavior of the car .................................................................................................................... 11
A. General overview ...................................................................................................................... 11
B. Detection Algorithms ................................................................................................................ 11
1. Global target point ................................................................................................................ 11
2. Avoiding walls and obstacles ................................................................................................. 15
3. CamShift ................................................................................................................................ 21
C. Guideline ................................................................................................................................... 25
IV. Result ......................................................................................................................................... 26
A. Achievement.............................................................................................................................. 26
B. Analyse ...................................................................................................................................... 27
C. Difficulties .................................................................................................................................. 29
D. To go further.............................................................................................................................. 29
1. Ultrasound sensor ................................................................................................................. 29
2. Correlation between two successive pictures....................................................................... 30
3. Measurement of the distance ............................................................................................... 30
4. Object tracking ...................................................................................................................... 30
5. Face Detect ............................................................................................................................ 32
6. Classifier ................................................................................................................................ 32
V. Conclusion ..................................................................................................................................... 33

CONTAL/MESTRE/RICHTER/SAMOUN Page 2
 Pi cars Raspberry 20/01/2014

Table of illustrations

Figure 1: Description raspberry pi ........................................................................................................... 6

Figure 2: Assembly of the servo-motors ................................................................................................. 6
Figure 3: Breadboard ............................................................................................................................... 7
Figure 4: Camera ..................................................................................................................................... 8
Figure 5: Pi Light ...................................................................................................................................... 8
Figure 6: SSH connexion to the raspberry pi ......................................................................................... 10
Figure 7: Server connexion + stream from Camera pi........................................................................... 10
Figure 8: First method ........................................................................................................................... 13
Figure 9: Second method ...................................................................................................................... 14
Figure 10: Problem of noise .................................................................................................................. 14
Figure 11: Delete noise.......................................................................................................................... 15
Figure 12 : 90 cm ................................................................................................................................... 16
Figure 13 : 80 cm ................................................................................................................................... 16
Figure 14 : 70 cm ................................................................................................................................... 16
Figure 15 : 60 cm ................................................................................................................................... 16
Figure 16: Obstacles .............................................................................................................................. 17
Figure 17: Detection of the wall ............................................................................................................ 17
Figure 18: Case where bad lines appear ............................................................................................... 18
Figure 19: Case where the obstacle wasn't detected ........................................................................... 18
Figure 20: After modification (figure 10)............................................................................................... 19
Figure 21: After modification (figure 11)............................................................................................... 20
Figure 22 : Without expansion neither smoothing ............................................................................... 20
Figure 23 : Corresponding edges ........................................................................................................... 20
Figure 24 : Expansion and smoothing ................................................................................................... 21
Figure 25 : Corresponding edges ........................................................................................................... 21
Figure 26: Centroid method .................................................................................................................. 21
Figure 27: Comparison of two close pictures. Result: 0.12099 ............................................................. 23
Figure 28: Another comparison. Result: 0.15313.................................................................................. 23
Figure 29: Comparison of two frames (They are very similar) .............................................................. 24
Figure 30: Camera + Raspberry Pi ......................................................................................................... 26
Figure 31: Pi Car..................................................................................................................................... 26
Figure 32: Pi Car..................................................................................................................................... 27
Figure 33 : a tracked object ................................................................................................................... 31
Figure 34 : Mask of the tracked object.................................................................................................. 31
Figure 35: a tracked door ...................................................................................................................... 31
Figure 36: After moving is still detected ............................................................................................... 32
Figure 37: Facedetect Pi camera ........................................................................................................... 32

CONTAL/MESTRE/RICHTER/SAMOUN Page 3
 Pi cars Raspberry 20/01/2014

I. Introduction

In order to make use of the different concepts we learned during our courses
in Applied Mathematics and Modelling in Polytech’ Nice Sophia, we must carry out a
project for 3 weeks. This project will allow us to apply the acquired knowledge and
discover new things. Moreover, this project will teach us the teamwork because we
are a group of 4.

The project that we must realize is an autonomous car controlled by computer

vision algorithms computed by a Raspberry Pi. It must be able, when completed, to
travel alone without hitting walls or obstacles, thanks to an on-board camera.

For this project, we are supervised by Mr Precioso, who gave us a Raspberry Pi

and various components essential to the project. Indeed, we will use a Raspberry Pi
which is a tiny computer.

This Raspberry Pi will allow us to operate the car with a Python program.
That’s why, we’ll explain, in a first part, the installation of Raspberry OS on the SD
card.

CONTAL/MESTRE/RICHTER/SAMOUN Page 4
 Pi cars Raspberry 20/01/2014

II. First part

A. Hardware

Needed hardware provided by Mr. Precioso for the development of the project:

 Two RC cars (one big-sized)

 Two Raspberry Pi
 Two cameras
 Two servo-motors
 A Raspberry case
 2 empty 64 GB cards and 1 4 GB cards with a Raspbian OS image preloaded
 Keyboards with touchpad, screens and Wi-Fi keys for each Raspberry
 a high capacity battery
 Ethernet wires

1. The Raspberry Pi hardware

We had a Raspberry Pi at our disposal. It is a fully featured and very compact

computer, which was the perfect device to work with, as we had the requirement that the
computing unit would have to fit on the RC car.

The Raspberry Pi has a 700 MHz processor, which is pretty limiting regarding the
complexity of the computer vision algorithms.

CONTAL/MESTRE/RICHTER/SAMOUN Page 5
 Pi cars Raspberry 20/01/2014

Figure 1: Description raspberry pi

The Raspberry Pi offers two USB connectors, a SD card slot, some GPIO headers, an
Ethernet connector, a CSI Connector for a camera, and an HDMI output. The whole is
powered by a micro-USB connector.

2. Servo-motors assembly

We had some servo-motors which were bought from the OpenElectrons.com

website. They were specially designed for us. We used them to move the camera on the car
by connecting them to the GPIO headers.

We had to do the assembly of the servo-motors:

Figure 2: Assembly of the servo-motors

We followed the instruction provided by OpenElectrons.com to perform the

assembly.

CONTAL/MESTRE/RICHTER/SAMOUN Page 6
 Pi cars Raspberry 20/01/2014

3. Linking the Raspberry Pi to the RC car controller

In order to link the Pi to the RC car controller, we wired both of them to a breadboard
to be able to send commands from the Raspberry, via the GPIO headers, to the remote
controller.

Figure 3: Breadboard

4. The cameras and the lights

We mounted one of the two small cameras on the servo-motors to be able to look
around. We connected its bus to the CSI connector.

There was also a LED light circle which was mounted around the camera which has to
mount on the servo-motors.

CONTAL/MESTRE/RICHTER/SAMOUN Page 7
 Pi cars Raspberry 20/01/2014

Figure 4: Camera

Figure 5: Pi Light

B. Software

1. The OS

We chose to use the Raspbian OS, that we had to install on a 64 GB SD card. It

involved numerous steps, including formatting the SD card to the FAT32 file system and
writing the Raspbian OS image on it, using the Win32DiskImager software.

We had some issues with the formatting step: we didn’t succeed using Linux because
of mounting and partitioning problems. So we had to use Windows and install a special
software to format to FAT32 (Fat32 Format).

CONTAL/MESTRE/RICHTER/SAMOUN Page 8
 Pi cars Raspberry 20/01/2014

2. Wireless

After installing the OS, on the Raspberry Pi SD card, we had to install some specific
drivers to get the Wi-Fi card to work.

We encountered some issues with some of the Wi-Fi cards that we had at hand:
while the WIFI USB “Wi-Pi” worked seamlessly without any driver installation, we had a hard
time to get the WIFI USB “TL-WN725N” to work because the drivers were incompatible with
our kernel version. We tried to downgrade our kernel, but this solution wasn’t viable
because it was breaking the compatibility with some critical libraries that we needed for the
project.

3. Librairies installation

Once the OS was ready to use, we had to install OpenCV for Python on it (with the
libopencv-dev and the python-opencv packages). We had to update the previously installed
packages to do so.

To control the servo-motors and the LED lights, we had to respectively install
additional packages, pi-pan and pi-light, provided by OpenElectrons.com. We had some issue
with this part of the installation, because after installing these packages, the GUI of the OS
didn’t display after rebooting the Raspberry. So, we sent a mail to OpenElectron to get this
sorted out. They answered quickly and advised us to reinstall a custom image of the OS that
they provided us.

4. Remote-controlling the Raspberry

We use SSH to have a remote terminal on the Raspberry and remotely execute our
scripts on it. We also configured a VNC Server to have control on the Raspberry GUI over Wi-
Fi and potentially visualize the video stream, which proved to be unfeasible.

We had a lot of troubles with the school hotspot while configuring both of these, so
we had to create an access point with one of our smartphone.

Then, we wanted to be able to remotely display the video stream from the camera in
real time. We couldn’t use SSH nor VNC for this, so we first tried JPG-Stream, a web server
allowing the visualization of the video on a web page hosted by the Raspberry. But this was
very limited because this software was designed for security cameras, which update every 1
or 2 seconds. We needed a faster update time so we looked for something else.

CONTAL/MESTRE/RICHTER/SAMOUN Page 9
 Pi cars Raspberry 20/01/2014

So we searched online and found a Python module named PiCamera (under BSD
license) that allowed us to have an interface to the Raspberry Pi’s camera module. Among its
features, there were some video streaming methods that were useful.

Figure 6: SSH connexion to the raspberry pi

Figure 7: Server connexion + stream from Camera pi

CONTAL/MESTRE/RICHTER/SAMOUN Page 10
 Pi cars Raspberry 20/01/2014

III. Behavior of the car

The car has to be able to move on its own by analyzing the snapshots taken by the
camera.

A. General overview

Our goal is to make the car move the more distance possible, and, if possible, goes all
around the floor. We also want the car to be able to get out of a dead-end by its own and
avoid obstacles. So we studied the environment in which we’ll have to operate the car.

We took a lot of picture of the building interior with the Pi-Camera to see the color
differences present in the camera point of view. We also got notice of some potential
problems, such as reflections, shadows or lights. We measured the corridor width to be sure
that the car would have enough space to move.

We also made some measurements about the car physic behavior: the maximum
angle of turning, its speed and acceleration, and how much it skidded while accelerating or
braking. That’s why, we made function which allow the car to make different way of turning
or moving back (20, 45, 90 degrees).

We also noticed, while testing, that we should have taken into account the fact that
the car was heavier with the things loaded on it (Raspberry, battery, servo-motors…), which
changed a lot of results. Moreover, the car skidded too so we weighed the rear.

The car is moving by taking a snapshot from the video stream from the camera,
analyzing it, and then taking a decision about which move to do.

B. Detection Algorithms

To achieve the automated move of the car, we separated the analysis in two steps:
first defining a global target point, then avoiding obstacles.

As the project specifications require the car to operate inside a building, most of our
algorithms make extensive use of the simple geometric shapes present in this kind of place.

1. Global target point

CONTAL/MESTRE/RICHTER/SAMOUN Page 11
 Pi cars Raspberry 20/01/2014

The global target point is a point toward which the car will head and go forward
before the next analysis. To achieve this, we chose to implement a method using vanishing
point.

The vanishing point is the point at which parallel lines receding from an observer
seem to converge. The Pi-car takes pictures and studies them to move.

To find the vanishing point, we changed the picture in grayscale, thanks to cvColor
from OpenCV:

cv2.cvtColor(src , code [, dst[, dstCn]]) → dst¶

– src represents the input image.

– Dst the output image.

– Code: cv2.COLOR_BGR2GRAY to convert the picture in gray levels.

Then, we use the canny filter that highlights the edges depending on values between
0 and 255. The display of contours will be more or less sensitive:

cv2.Canny(img, threshold1, threshold2[, edges[, apertureSize[, L2gradient]]]) → edges

– Img: represents the input image.

– threshold1, threshold2: two thresholds between 0 and 255.

– apertureSize=3

Next, the Hough Lines Transform allows us to extract the lines coordinates that
define the edges:

cv2.HoughLinesP (edges, rho,theta,threshold,minLineLength,maxLineGap)→ lines

– Edges: represents the edges picture got with Canny.

– theta: angle

– MinLineLenght: minimal length of the line to be reserved.

– MaxLineGap: two lines are linked up when they are in the same direction and
separate by a maximum of MaxLineGap pixels.

CONTAL/MESTRE/RICHTER/SAMOUN Page 12
 Pi cars Raspberry 20/01/2014

Afterwards, we took out the lines that are not vanishing lines. In other words, we
deleted vertical lines and horizontal lines.

Given the lines coordinates, we can now calculate the intersection of all straight lines.
The average of these points gives us the vanishing point. But this first method wasn’t
accurate.

Result:

Figure 8: First method

That’s why, we improved the first method:

In first, we noticed that sometimes some lines are superimposed approximately

which falsify the vanishing point. So, we don't take into account the intersection of lines
which have almost the same angle.

Then, in order to be more accurate, we calculate the average of the intersections and
remove the intersections that are too far from this average. And we calculate a new average
which defines the vanishing point.

Moreover, we improved the method which takes out vertical lines and horizontal lines. Now,
it also deletes lines which are close to vertical and horizontal lines.

Result:

CONTAL/MESTRE/RICHTER/SAMOUN Page 13
 Pi cars Raspberry 20/01/2014

Figure 9: Second method

Difficulties:

When we have this picture:

Figure 10: Problem of noise

The picture is falsified by lots of lines in the same area and in different directions. So,
we create a function which removes the lines in this area. In this function, we had to run
through the matrix of the picture with the lines in blue. For each cols, when the number of

CONTAL/MESTRE/RICHTER/SAMOUN Page 14
 Pi cars Raspberry 20/01/2014

blue pixel are upper than a threshold, all the lines present on this cols are deleted. The lines
take into account are in red.

Result:

Figure 11: Delete noise

But, when we test this algorithm on the raspberry, we noticed that the raspberry
takes too much time to run through the picture (around 2 minutes). Consequently, we took
out this algorithm and found another method.

So as to resolve this difficulty, we changed the thresholds of the function canny. We

applied three times the canny function with different values of threshold and we took the
set of all lines returned by these three values.

2. Avoiding walls and obstacles

The goal of this step of the analysis is to avoid the walls and any obstacle that would
come in the path of the car.

The first approach was to define a trapeze on the picture taken by the camera in
which we consider that any objects, inside the trapeze, are considered on the path of the
car. Therefore, they are considered as an obstacle to avoid.

CONTAL/MESTRE/RICHTER/SAMOUN Page 15
 Pi cars Raspberry 20/01/2014

We had to evaluate the minimal distance threshold between the car and an object in
front of it. To do this accurately, we also have to consider the speed of the car to modulate
the threshold. This trapeze is a safety area in which the car can move if it’s clear. If it’s not,
the car has to turn to avoid the collision.

Figure 12 : 90 cm Figure 13 : 80 cm

Figure 14 : 70 cm Figure 15 : 60 cm

First step algorithm:

The first thing we do is to take the snapshot from the camera and apply the Canny
filter and the Hough Line Transform to extract the lines inside.

First, we check if an extremity of a line is inside the trapeze. If there was any of this
one, a forward move is considered critical and an avoidance signal is sent to the car control
module.

Second step algorithm:

In a first version, we only considered only horizontal lines that are crossing the
trapeze. All these were considered as obstacles.

But we wanted to improve this method to be able to detect any kind of object. We
also thought that the robot might not be perfectly facing a wall, which would result in
skipping the wall edges lines because they are not perfectly horizontal.

CONTAL/MESTRE/RICHTER/SAMOUN Page 16
 Pi cars Raspberry 20/01/2014

In a second version of this second step, we are looking for the intersection of any lines with
the edges of the trapeze. If any, there’s an obstacle.

Figure 16: Obstacles

Figure 17: Detection of the wall

Difficulties:

This first aboard wasn’t effective in some cases. Indeed, although the canny function
returned a binary picture where the edges lines were clearly present, these lines weren't
always returned by the function HoughLinesP. Quite the reverse, in some pictures, lines
appear inside the trapeze even though there isn’t any impediment. This kind of mistake can
be caused by seal, a reflexion of the light or a mark on the ground.

To illustrate this problem, these images below:

CONTAL/MESTRE/RICHTER/SAMOUN Page 17
 Pi cars Raspberry 20/01/2014

Figure 18: Case where bad lines appear

Figure 19: Case where the obstacle wasn't detected

So, to resolve these two problems, we have in a first time applied a Gaussian filter to
minimise the noise.

CONTAL/MESTRE/RICHTER/SAMOUN Page 18
 Pi cars Raspberry 20/01/2014

Many methods already exist in OpenCv library. We choose the method cv2.GaussianBlur
which applies to the distribution of pixels of the initial picture.

We can choose the kernel size of this function, more the size are important, more the
smoothing will be strong.

In secondly, to increase the probability that the lines which are detected by the function
Canny are really take into account, we increase the thickness of the edges.

With this aim in mind, we used the function of OpenCv cv2.dilate(img,kernel) which fatten
the lines according to the size and the shape (RECTANGULAR, in CROSS or in CIRCLE) of the
kernel. As for us, after many tests we decided to use a kernel with a cross shape and with a
size of 3.

So for the precedent examples we obtain these results:

Figure 20: After modification (figure 10)

CONTAL/MESTRE/RICHTER/SAMOUN Page 19
 Pi cars Raspberry 20/01/2014

Figure 21: After modification (figure 11)

Moreover, the correction that these two methods make, they also led to more precision in
the general case.

A general instance with the binary edges pictures corresponding:

Figure 22 : Without expansion neither smoothing Figure 23 : Corresponding edges

CONTAL/MESTRE/RICHTER/SAMOUN Page 20
 Pi cars Raspberry 20/01/2014

Figure 24 : Expansion and smoothing Figure 25 : Corresponding edges

3. CamShift

a) Principle

When we argue about how the car could be moved autonomous way, there were
several possibilities and camShiftt was one of them.

First at all, we wanted the camera fixed on an area on the floor and stay on this area. The
idea was that the car fixed an area and could follow the path if the car couldn’t find a
vanishing point for instance.

Therefore, the idea of Meanshift came out. Let’s introduce what is CamShift. CamShift
means continuously Adaptive Mean Shift. MeanShift is an algorithm which allows especially
track an object.

In fact, this is an iterative algorithm which consists to converge on a point to the closest
maximum local.

First at all, it chooses a starting point named P, then it seeks the set of points named E which
are close to P. After P is moved to the centroid E and we all the steps from step 2 is repeated
until convergence. Here an example in 2D.

Figure 26: Centroid method

CONTAL/MESTRE/RICHTER/SAMOUN Page 21
 Pi cars Raspberry 20/01/2014

To use meanshift he has to fix a target, calculate its histogram and thank to the MeanShift
algorithm, the target could be backprojected on each frame for calculation. So for each
frame, the both histogram were compared.

First, for the implementation we have to do some stuff. First we have to define an area to
track. After define it, we had to calculate the mask of this region and apply to it and use the
backpropagation technique on it.

b) Difficulties

So, we have programmed this method and we realized that the algorithm worked
well that is to say when it finds an object, this one was well tracked. However the problem
was, once the object tracked couldn't be found on the area, it searched another area of
object to track and so on. Besides, this algorithm never wanted to fix the floor as an object,
so each time, it tried to look for an object on the window in order to track it. So we can’t
allow basing the move of the car by this first version of algorithm because if during the ride
the algorithm track any object, the car couldn’t be moved correctly.

We have another idea which consist at the first frame to define an area in order to
calculate its histogram and during all the ride, the camera seeks on the next window where
the histogram of the first area is close to it thanks to camShift, if the two histogram were the
car keep on moving but if on the area, the both wasn’t identical, the algorithm stop and used
another method until it found the same area.

That is what we did, but there was another problem which came out. That was the fact that
the color on the floor was not accurate. For us, we see the floor with a blue color but with
the camera the color was insipid and not blue. Besides as the color was dull, there were lots
of area on the windows except floor which have the same « insipid color » and consequently
the same histogram so the algorithm like the first time find each time others things to track.

We had to find another way to proceed. We have forgotten the CamShift but we have the
idea to use a method of openCv which calculate the distance between the two histograms
from each frame in order to compare them.

For instance, one histogram is calculated on a fix area and the next frame another histogram
is calculated. Then the method « CompareHist(hist1,hist2,method) » return to us the
distance between the two histograms. If the distance is lower to a certain value, the car
forward.

However, we have tested the method and we saw that it not accurate at all.

CONTAL/MESTRE/RICHTER/SAMOUN Page 22
 Pi cars Raspberry 20/01/2014

On the first picture, the two frames are practically both identical but the comparison the
histogram indicate to us the value of about 0.60 and after waiting a while we don’t know
why but the value change to 0.30 which is better because it show that the two histograms
are almost equals.

Therefore, we didn’t implement this method.

Just before implement this method for a continuous stream, we tested it on very similar
pictures, in order to know if the comparison of the both histogram works well. More the
result is close to one, more the picture is different and vice versa.

Figure 27: Comparison of two close pictures. Result: 0.12099

Figure 28: Another comparison. Result: 0.15313

CONTAL/MESTRE/RICHTER/SAMOUN Page 23
 Pi cars Raspberry 20/01/2014

Figure 29: Comparison of two frames (They are very similar)

However, although the both frames are very similar the histogram method
(comparison of two histograms) returns this result above which show that they aren’t
similar. Therefore, we can’t rest on this method.

c) Bonus – tracking an object

We wanted the car tracks an object and looks for it. If the object is not on the area,
the motorized camera searches around is the object is somewhere and forward until him.

There are two main steps: use a color filter and find contour of the tracked object
Step one: convert image from BGR to HSV. (Blue,green,red) (Hue,saturation,value) . Then
we have to filter the colour of interest between a Min and a Max threshold .This will leave
only the object we aim to track.

The final step consists in finding the contour of the object. We have a binary image as an
input and for output we have a vector of contours. Then we use the moment method which
consists in having the coordinate of the largest contour of the object by putting in input the
vector of contours.
http://www.davidhampgonsalves.com/opencv-python-color-tracking

CONTAL/MESTRE/RICHTER/SAMOUN Page 24
 Pi cars Raspberry 20/01/2014

C. Guideline

Our algorithm runs in a specific order:

 Finding an obstacle
Obstacle: Launch of the motorized camera
Motorized camera looks at 45 degrees to the right and 90
degrees to the right and at the same to the left.

When it finds a vanishing point, it goes in this direction,

otherwise it goes in the direction where there aren’t obstacles
or if not it goes back.

 No obstacle: Search vanishing point

o If there is a vanishing point towards the middle, it moves straight
o If the vanishing point is more to the right or left, it moves accordingly

 No barriers and no vanishing point: it moves straight

CONTAL/MESTRE/RICHTER/SAMOUN Page 25
 Pi cars Raspberry 20/01/2014

IV. Result

A. Achievement

Figure 30: Camera + Raspberry Pi

Figure 31: Pi Car

CONTAL/MESTRE/RICHTER/SAMOUN Page 26
 Pi cars Raspberry 20/01/2014

Figure 32: Pi Car

B. Analyse

Searching Vanishing Point Searching obstacles with trapeze method

CONTAL/MESTRE/RICHTER/SAMOUN Page 27
 Pi cars Raspberry 20/01/2014

CONTAL/MESTRE/RICHTER/SAMOUN Page 28
 Pi cars Raspberry 20/01/2014

C. Difficulties

Despite of the approaches we found to get round some difficulties, it also exists
problems we couldn’t resolve.

Depending on the time of the day, the lighting is different and the ground being very bright,
this colour changes a lot.

That why, the smoothing is sometimes not enough to remove the reflection.

Moreover, the camera's definition sometimes change and the pictures appear with a dark
yellow curtain and our methods are a little less efficient of this kind of images.

And finally, when the autonomous car is in front of a wall, he can't detect anything
and it will going on to move forward.

We hadn't enough time to resolve this problem, whereas we searched some solution like the
histogram's comparison between two successive pictures.

D. To go further

1. Ultrasound sensor

Three week are obviously too short to succeed very well a project like that. Although
we achieved our goal we have a lot of stuff that we wanted to realise and improve. Let’s take
a look of possible features that we thought about it.

CONTAL/MESTRE/RICHTER/SAMOUN Page 29
 Pi cars Raspberry 20/01/2014

We had seen that sometimes the car could be in front of the wall after have moving forward
too quickly, and therefore the car has had difficulties in moving. The first thing we though in
order to solve this problem was to take two pictures on two time interval and compare
them, with their histogram but although it worked well with some picture we have taken,
this method doesn’t work well with the car that is to say that the both histograms were most
of the time too different. The idea was to see if both pictures were similar. Indeed, this
means that the car was in front of the wall. In order to solve this problem correctly, for
instance, we could put on the car an ultrasonic sensor.

2. Correlation between two successive pictures

Sometimes we have encountered lots of difficulties about the floor. In fact the floor
was changing so many times either because of the light on it (reflection, light due to the
ceiling) or scratches on it. Therefore when we have looked for lines on the frame, lots of
scratches appeared like lines and for the trapeze method, it appear like an obstacle.

A possibility to solve this problem is to take two pictures with two time intervals, and make a
correlation between the two pictures in order to know if it’s almost the same picture or not.
Thus, the scratches won't bother us and this method could be better for the progress of the
car.

3. Measurement of the distance

In our actual project, the car is moving with the same speed during his ride. We have
the idea to measure the distance between the camera and the vanishing point in order to
know if the car is far from it or not. If it’s enough far, the car will move forward more quickly
than if the car was not far enough.

We have begun to read on how we could do that and we have seen that we should calibrate
the camera. It could be a great feature.

4. Object tracking

At the beginning, we wanted to use the camShift to make the car autonomous that is
to say; the car will track an area on the floor and will stay on it. Unfortunately, due to lots of
parameters it didn’t worked well. However the tracking of an object was done correctly and

CONTAL/MESTRE/RICHTER/SAMOUN Page 30
 Pi cars Raspberry 20/01/2014

we can find any object and track it. The next step and another idea are to look for the object
until the car finds it.

For instance, we could put a ball on the floor and the goal of the car is follow the ball and if
the ball doesn't appear on the area, the car with its motorized camera will look for it.

Figure 33 : a tracked object

Figure 34 : Mask of the tracked object

Another way to use Camshift is to detect doors in order to avoid them.

Figure 35: a tracked door

CONTAL/MESTRE/RICHTER/SAMOUN Page 31
 Pi cars Raspberry 20/01/2014

Figure 36: After moving is still detected

Once the door is detected, we should calculate the distance between the two cross
(Between the pi car and the door). (cf. measurement of the distance above)

5. Face Detect

We had done the face detect algorithm but due to a lack of time we didn’t implement
it on the Raspberry. Besides, the processor was not enough powerful to support in addition
the face Detect algorithm. It could be worked well but with more time to process.

Figure 37: Facedetect Pi camera

6. Classifier
To go further, we could train a classifier (Adaboost, Artificial neural network…) so that the car knows
if this is a door or not, or which person is it, among, a set of data for instance.

CONTAL/MESTRE/RICHTER/SAMOUN Page 32
 Pi cars Raspberry 20/01/2014

V. Conclusion

In the context of our project, we achieve an autonomous car. As a first step, we had
to document us on the pi raspberry as well as the pi car. Then, we work on how to treat the
images which are captured by the camera.

Through the various requirements imposed by the specification, we obtain the final result
which is as follows:

 Our pi because advances in detecting vanishing points, and detects the

different obstacles.

Certainly, some imperfections are still present on the car, but they do not affect its overall
operation.

CONTAL/MESTRE/RICHTER/SAMOUN Page 33