2018 15th International Multi-Conference on Systems, Signals & Devices (SSD)

Pedestrian Detection Based on Background

Compensation with Block-Matching Algorithm
Khouloud CHEBLI Anouar BEN KHALIFA
LATIS- Laboratory of Advanced Technology LATIS- Laboratory of Advanced Technology
and Intelligent Systems and Intelligent Systems
ENISo, Sousse University ENISo, Sousse University
Sousse, Tunisia Sousse, Tunisia
[email protected] [email protected]

Abstract— The development of autonomous vehicle is an • Separate the object motion from the camera motion by
important and active research area. In the last few years, graph-cut optimization [3] or by solving the Markov
pedestrian detection methods for a moving camera have been random field problem with nonparametric belief
severely developed. This field presents many challenges in order propagation [4].
to avoid the camera motion and recognize the dynamic objects.
This paper proposes a background compensation method for • Compensate the camera movement to fit the previous
pedestrian detection with a moving Camera. This method relies image to the current image [5].
on motion compensation to transfer the background model from
the current frame to the previous frame in order to detect In this paper, we propose a new algorithm for pedestrian
dynamic obstacles. This motion compensation is carried out using detection applied to both infrared and visible database. The
different block matching algorithms and the gradient algorithm is based on motion compensation method in order to
information of the images to establish the background's model reduce the false detection and maintaining a real time
motion. The proposed method was evaluated on a public performance. Our main idea is to establish a background
benchmark system: the CVC14 and achieved promising results as motion model between adjacent frames with the use of block
shown in this article. matching algorithm then make the background compensation
to distinguish dynamic objects in dynamic environment.
Keywords— Pedestrian detection, motion estimation, ego-
motion compensation, Block matching algorithm. The remaining part of our work is organized as follows,
Section 2 describes the most contemporary techniques in
pedestrian detection and motion compensation. Section 3
I. INTRODUCTION explains our proposed algorithm; Section 4 reports some
When it comes to the development of autonomous, or self- experimental results; finally, Section 5 presents the conclusion
driving, vehicles, a warehouse full of technology -Software of the work with few prospect for further study.
and hardware- is taken to make it happen. In this context much
research focuses on technology that integrates the vehicle with II. LITERATURE REVIEW
its environment. Detecting pedestrian is one of the essential
issues for understanding environment and to achieve the The previous works on pedestrian detection fields are
automatic operation of autonomous vehicle. based on color images. However, color-based pedestrian
detection is not efficient in challenging environments
Although, several pedestrian detection algorithms have especially at night-time and poor weather. Owing to the
been proposed so far, but still there is a need for an automatic development of sensor technology, infrared (IR) and far
system that can detect the human from a moving camera that infrared (FIR) camera sensors, were used to fix the problems
are more challenging as compared to a fixed camera. of color based pedestrian detection. Thanks to their
In this problem, it is important to achieve a computational independence of light conditions, the FIR cameras can be
efficiency as well as detection accuracy. Background used in both day-time and night-time with no or little
subtraction algorithms have been proposed and shown good difference, reaching vision beyond the usual limitations of
performances in fixed cameras. However, in mobile camera, day-light cameras. Moreover, the absence of colors or textures
the existing methods became no more robust because eases the processing towards interpretation. In consequence,
background is also changed by the camera movement [1]. the IR field seems to be promising and explains deep
investigation.
Depending on the literature, there are three based
approaches of the computer vision for a mobile camera: Based on the infrared properties, T. Kim et al. [6] proposed
a novel method to detect the remote pedestrian by generating
• Construct a panorama image using image registration regions of interest using transfer learning with convolutional
then applying a background subtraction algorithm neural network (CNN) feature, then following non-maximal
without any modification yet, many false detection are
remained present [2].

978-1-5386-5305-0/18/$31.00 ©2018 IEEE 497

Authorized licensed use limited to: UNIVERSITE DE SOUSSE. Downloaded on January 06,2021 at 10:02:58 UTC from IEEE Xplore. Restrictions apply.
 2018 15th International Multi-Conference on Systems, Signals & Devices (SSD)

suppression (NMS) with strong aspect ratio limitation to I. Jegham et al. [13] proposed a new approach for the ego-
improve the detection performance. motion compensation based on the interest point detector
Recent works have shown that efficient and robust shape- SURF, dividing the images into multiple moving objects
based cues can be obtained from histogram of oriented which are then characterized using the Local Binary Pattern
gradient (HOG) in images [7]. In this case P. Area [8] (LBP) descriptor and classified via the Support Vector
developed a complete method for pedestrian detection applied Machine (SVM) into two categories: pedestrian and undefined
to infrared images by associating an image descriptor based on object.
histograms of oriented gradients (HOG), with a Support
Vector Machine (SVM) classifier. III. PROPOSED METHOD
Nevertheless, in a dynamic situation, the presence of After studying several works concerning pedestrian
background and camera motion add difficulties to pedestrian detection [12][13][14][15], non-static camera [5][16][17][18]
detection. Hence, an extra process is required to handle the [19] and the properties of infrared image [8][20][18], we opt
camera motion in order to build the background model and for combining different approaches to provide our proposed
detect moving objects. There are three fundamental techniques method. This latter as indicated in the schematic diagram
to detect moving object under moving background: the below in Figure.1 is composed of the following sequences of
background model method [9], inter-frame difference method steps:
[10] and the optical flow field method [10].
T. Ni et al. [11] synthesize the three target detection ways A. Data acquisition
above by extracting moving velocity of feature points in The camera position for the pedestrian detection system is
grayscale images then based on block matching algorithm. entirely important to obtain a good view of the road. Thus, the
They calculated the optical flow to establish a background camera must be fixed on the top of the windshield of a vehicle
model and compensate the movement to finish with detecting so that the captured images contains a significant part of the
the moving obstacle. background and the dynamic objects. This view insure a good
region of interest, then a good camera motion estimation.
J. Hariyono et al. [12] presented a method which consist
on segmenting the region that present the same optical flows
after compensating the ego-motion of the camera. Therefore, B. Extraction of region of interest
the conformed optical flows are extracted. At the end, the After observing many captured images, we have
detected regions will be recognized. noticed that the 40% of the top of the image is sufficient to
estimate the background movement. This part contains most of
the information that we need to build our background model.
The 60% rest of the frame doesn't add useful information as it
shown in the Figure.2.

Fig.1. General process of proposed approach

498

Authorized licensed use limited to: UNIVERSITE DE SOUSSE. Downloaded on January 06,2021 at 10:02:58 UTC from IEEE Xplore. Restrictions apply.
 2018 15th International Multi-Conference on Systems, Signals & Devices (SSD)

the reference frame to find a matching, based on matching

C. Image processing criterion and gradient information. We obtain in result a
To enhance the contrast of images, we applied the motion vector for each block. One of our main object in this
histogram equalization, specially for the FIR database in the work is to evaluate different block matching algorithms to
case of dark background and foreground. define the most proper for our studies.
After obtaining the enhanced image, we need to obtain the
gradient information of our frame. In fact, the main thought of E. Image compensation
motion estimation based block matching algorithm is to divide In this Stage we realized a global translation for the frame
the reference image into a number of macro-blocks , and then k(t+1) with dx and dy to be at the same level as k(t).
looking for the matching sub-block in the target frame
according to a specific metrics. In the traditional algorithms, F. Background subtraction and object detection
all matching criterion are based on grayscale information. This After compensate the movement, a simple inter-frame
information is very sensitive to the noise and the change of difference between k(t) and k(t-1) conserves the dynamic
light and shade for the outside. To solve this matter, we objects which admit a different motion than the camera.
decided to concatenate the standard of block matching with
the gradient information on the image. Therefore, the
matching criterion does no longer consider the grayscale G. object recognition
conservation between different blocks in two successive Finally, we have to apply a feature extraction technique to
frames, but asks for the gradient conservation between get a feature vector for each object then use it latter in the
matching block and the original one. This new criterion makes classification process
the algorithm more accurate to deal with the different
condition of environment. IV. EXPERIMENTAL RESULTS AND ANALYSIS
The proposed approach was evaluated by using four
sequences of images from CVC14 pedestrian dataset.
It's recorded by both: visible and FIR cameras for day and
night activities. This dataset was acquired at 10 FPS. Table.1
shows the number of frames and annotated pedestrian for each
sequence in the dataset.
TABLE. 1. CVC-14 DATASET SUMMARY OF IMAGES AND ANNOTATED
PEDESTRIANS [21]

FIR Visible
Set Variable
Day Night Day Night
Fig. 2. Region of interest
Positives
2232 1386 2232 1586
frames
Negatives
D. Motion estimation Training 1463 2004 1463 2004
frames
We could classify the motion in a video frame with a Annoted
2769 2222 2672 2007
pedestrians
moving camera into: global motion and local motion. The one Frames 706 727 706 727
due to camera motion is called global motion, whereas the Testing Annoted
2433 1895 2302 1589
dynamic object in the scene result in local motion [16]. Pedestrians

Without compensation, the inter-frame difference

represents all motions caused by camera and dynamic objects We started with using full image for testing and we have
in the scene. To detect the pedestrian we need first to segment had quite satisfactory results. But after analysis, and to reduce
out only dynamic objects. the computational time we decided to select a region of
interest to estimate the global motion. This modification
Presuming that the image k at time t is the frame k(t) and improves the accuracy of the pedestrian classification as it
the image at time t+1 noted k(t+1). we think that, due to the shown in the TABLE. 2.
camera motion, each pixel moves from the position of k(t) to
the position of k(t+1). This relation could be presented by the We studied multiple algorithms for motion estimation [22],
equation(1): and one of the objectives of this work is that we want to
evaluate their performance and to decide the most proper for
our dataset.
After fixing the region of interest we passed to the motion
Where dx and dy are global motion distance in x-axis and estimation based on the gradient criterion of the image. The
y-axis respectively. results shows that this criterion, boost the exactness of
background compensation, which subsquently improves the
Each frame of size M x N is divided into square blocks segmentation of dynamic objects.
B(i, j) of size (b x b) with i = 1….,M/b and j = 1…..N/b. For
each block Bm in the current frame, a search is performed on

499

Authorized licensed use limited to: UNIVERSITE DE SOUSSE. Downloaded on January 06,2021 at 10:02:58 UTC from IEEE Xplore. Restrictions apply.
 2018 15th International Multi-Conference on Systems, Signals & Devices (SSD)

TABLE. 2. STUDY OF THE REGION OF INTEREST PERFOMANCES ON THE

DETECTION PROCESS

Day Night
Results
FIR Visible FIR Visible
Detection
72.76% 94% 75.75% 90.22%
Rate
Average
time per
0.21 0.27 0.19 0.19
image per
second
Detection
Rate with 84.68% 93.61% 72.86% 90%
ROI
Average Fig. 4. Some results of segmentation process
run time
per image
0.11 0.13 0.12 0.12 After all, by referring to [13], The LBP features of
per
second the segmented images are extracted and then classified via the
with ROI SVM with RBF kernel.
TABLE. 3. presents the results of our proposed approach
Figure.3 explains the effect of the motion compensation on in terms of detection rate (DR) and Accuracy using LBP
the subtraction between two consecutive frames (Figure.3(a)); descriptor and linear SVM classifier.
the result without compensation Figure.3(b), the result with
compensation based on grayscale conservation Figure.3(c) and Comparing to the results obtained in the literature [13] and
the result with compensation based on gradient conservation [21], our approach yields significant performance
Figure.3(d). improvement. Citing the example of ARPS algorithm; for the
day activity with a FIR camera, we have reached a detection
rate of 84.8% with 52% accuracy versus detection rate
76.77% and 41% accuracy according to[13] . Moreover, in the
case of Night_FIR sequence, we have achieved a detection
rate of 91% and 60% accuracy, when in literature, they
obtained 84.57% as detection rate and 59% accuracy. More
detailed results of literature are displayed in TABLE. 4.

V. CONCLUSION
In this paper, we proposed a method for pedestrian
detection with a mobile camera. Our approach aims to
compensate the camera motion and get the candidate region of
pedestrian. The background model was generated by applying
Fig. 3. Results of background subtration
different search algorithms. The results shows a great
performance in term of speed and detection rate. In the future
For the four sequences, we fixed the parameters of seven work we suggest to concatinate two block matching
algorithms. Next step consists to segment dynamic objects. algorithms to abtain an hybrid one which works well for both
First, we have to give labels to the related components of the infrared and visible camera.
image. Second, we detect the different crops. Figure. 4. shows
an example of results.

TABLE. 3. RUSLTS OF REGONITION (%) PROCESS BASED ON OUR APPROACH

Day Night
Algorithm Fir visible FIR Visible
DR Accuracy DR Accuracy DR Accuracy DR Accuracy
Exhaustive Search (ES) 71.5 67 93 43 95.73 52 90 51
Three Step Search (TSS) 73.47 63.9 94 50 90.9 52.8 90.08 52
New Three Step Search
88.79 57 93.75 48 90 50 89.39 50
(NTSS)
Simple and Efficient TSS
95.31 52 93.9 47 90 50 90.08 47
(SES)
Four Step Search (4SS) 73.9 62.7 93.75 47 83 56 89.66 46
Diamond Search (DS) 73.47 61 93.61 46 96 54 83.3 51
Adaptive Rood Pattern
84.88 52 93 40 91 60 89.66 49
Search (ARPS)

500

Authorized licensed use limited to: UNIVERSITE DE SOUSSE. Downloaded on January 06,2021 at 10:02:58 UTC from IEEE Xplore. Restrictions apply.
 2018 15th International Multi-Conference on Systems, Signals & Devices (SSD)

TABLE. 4. RESULTS OF RECOGNITION (%) PROCESS OF LITERATURE

Day Night [10] T. Bouwmans, “Traditional and recent approaches in background
Results
FIR Visible FIR Visible modeling for foreground detection: An overview,” Comput. Sci. Rev.,
Detection vol. 11–12, May 2014, pp. 31–66.
Rate of 76.77 91.7 84.57 70.48 [11] S. S. Sengar and S. Mukhopadhyay, “A novel method for moving object
[13] detection based on block based frame differencing,”
Accuracy International Conference on Recent Advances in Information
41 50 59 52
of [13] Technology, 2016, pp. 467–472.
Detection [12] J. Hariyono, V.-D. Hoang, and K.-H. Jo, “Moving Object Localization
Rate of 79.4 59.4 67.9 12.4 Using Optical Flow for Pedestrian Detection from a Moving Vehicle,”
[21] Sci.World J., vol. 2014, pp. 1–8.
[13] I. Jegham and A. Ben Khalifa, “Pedestrian detection in poor weather
conditions using moving camera,” International Conference on
REFERENCES Computer Systems and Applications, 2017, pp. 1–6.
[14] A. Halidou, X. You, M. Hamidine, R. A. Etoundi, L. H. Diakite, and
Souleimanou, “Fast pedestrian detection based on region of interest and
multiblock local binary pattern descriptors,” Comput. Electr. Eng., vol.
[1] K. Choi and J. Yun, “ Robust and fast moving object detection in a non- 40, no. 8, 2014, pp. 375–389.
stationary camera via foreground probability based sampling,” IEEE
International Conference on Image Processing, pp. 4897 - 4901 [15] H. Ramzan, B. Fatima, A. R. Shahid, S. Ziauddin, and A. A. Safi,
“Intelligent Pedestrian Detection using Optical Flow and HOG,”
[2] J. He, K. Ji and Z. Lv, “ A Fast and Robust Algorithm of Detection and International Journal of Advanced Computer Science and Applications
Segmentation for Moving Object,” International Conference on vol. 7, no. 9, 2016, pp. 408–417.
Intelligent Information Hiding and Multimedia Signal Processing, 2009,
pp. 718 - 721. [16] Y. S. Erdem, F. Galip, and I. F. Ince, “Estimation of Camera Ego-
Motion for Real-Time Computer Vision Applications, ” International
[3] O. Barnich and M. Van Droogenbroeck, “ViBe : A Universal Journal of Scientific Research in Information Systems and Engineering,
Background Subtraction Algorithm for Video Sequences,” IEEE vol. 1, no. 2, 2015, pp.1-6.
Transactions on Image Processing, 2011, Volume: 20, Issue: 6, pp. 1709
- 1724. [17] C. Lyu, Y. Lei, S. Hu, and S. Hong, “Background subtraction with
moving cameras via Bayesian lowrank estimation,” Int. Conf. Signal
[4] T. Schoenemann and D. Cremers, “High resolution motion layer Process. Proceedings, ICSP, 2017, pp. 133–137.
decomposition using dual-space graph cuts,” IEEE Conference on
Computer Vision and Pattern Recognition, 2008, pp. 1–7. [18] Fernández-Caballero, A., Castillo, J. C., Martínez-Cantos, J., &
Martínez-Tomás, R., “Optical flow or image subtraction in human
[5] S. Kwak, T. Lim, Woonhyun Nam, Bohyung Han, and Joon Hee Han, detection from infrared camera on mobile robot,” Rob. Auton. Syst., vol.
“Generalized background subtraction based on hybrid inference 58, no. 12, 2010, pp. 1273–1281.
by belief propagation and Bayesian filtering,” International Conference
on Computer Vision, 2011, pp. 2174–2181. [19] L. Kurnianggoro, W. Yang, Y. C. Hernandez and Kang-Hyun Jo
“Online Background-Subtraction with Motion Compensation for Freely
[6] Ni, T., Li, Q., Sun, L., & Huang, L, "Dynamic Obstacle Detection Based Moving Camera,” Intelligent Computing Theories and Application vol.
on Background Compensation in Robot’s Movement 10362, 2017, pp.569-578.
Space." International Conference on Mechatronics and Mechanical
Engineering, 2017, pp. 1–16. [20] M. Bertozzi, A. Broggi, P. Grisleri, T. Graf, and M. Meinecke,
“Pedestrian detection in infrared images,” IEEE IV2003 Intelligent
[7] T. Kim and S. Kim, “Remote Pedestrians Detection at Night-time in FIR Vehicles Symposium, 2003, pp. 662–667.
Image using Contrast Filtering and Locally Projected Region-based
CNN,” Infrared Technology and Applications XLIII,vol. 10177, 2017, [21] A. González et al., “Pedestrian detection at day/night time with visible
pp. 1–16. and FIR cameras: A comparison,” Sensors, vol. 16, no. 6, 2016,
pp. 1–11.
[8] D. G. Lowe, “Distinctive Image Features from Scale-Invariant
Keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, Nov. 2004, pp. 91–110. [22] M. R. Khammar, “Evaluation of different block matching algorithms to
motion estimation,” International Journal of VLSI and Embedded
[9] P. Area, “Pedestrian Detection using Infrared images Histograms of Systems-IJVES, Vol 03, 2012, pp. 148-153.
Oriented Gradients,” IEEE Intelligent Vehicles Symposium, 2006, pp.
206–212.

501

Authorized licensed use limited to: UNIVERSITE DE SOUSSE. Downloaded on January 06,2021 at 10:02:58 UTC from IEEE Xplore. Restrictions apply.