Proceedings of the 2010 IEEE

International Conference on Information and Automation

June 20 - 23, Harbin, China

A Multiple Object Tracking Method Using Kalman

Filter
Xin Li Kejun Wang,Wei Wang and Yang Li
Engineering Training Center of HarBin Engineering
Automation College of Harbin Engneering University
University
Harbin, Heilongjiang Province, 150001, China
[email protected]

Abstract –It is important to maintain the identity of multiple performance of the approach is dependent on the proposed
targets while tracking them in some applications such as detection and recognition algorithms. In another work [6],
behavior understanding. However, unsatisfying tracking results vector Kalman is proposed for tracking objects. In this paper,
may be produced due to different real-time conditions. These separate methods for occlusion and merge are applied to
conditions include: inter-object occlusion, occlusion of the ocjects
handle the confusing situations. Further states of the
by background obstacles, splits and merges, which are observed
when objects are being tracked in real-time. In this paper, an corresponding moving objects are searched using spiral
algorithm of feature-based using Kalman filter motion to handle searching prior to tracking. Medioni et al. [7] proposed an
multiple objects tracking is proposed. The system is fully approach based on graph theory for tracking multiple targets.
automatic and requires no manual input of any kind for Their algorithm considered splits only, and they used gray
initialization of tracking. Through establishing Kalman filter level correlation between objects and segmented blobs to
motion model with the features centroid and area of moving detect and handle splits. Recently Czyzewski and Dalka [8]
objects in a single fixed camera monitoring scene, using used Kalman filter with RGB color-based approach to
information obtained by detection to judge whether merge or measure the similarity between moving objects. A threshold is
split occurred, the calculation of the cost function can be used to
applied to measure the similarity between the detected regions
solve the problems of correspondence after split happened. The
algorithm proposed is validated on human and vehicle image which fails in fully occluded scenarios.
sequence 7KH UHVXOWV VKRZV WKDW WKH algorithm proposed In this paper, we use Kalman filter to establish object
achieveV efficient tracking of multiple moving objects under the motion model, using the current object’s information to
confusing situations. predict object's position, so that we can reduce the search
scope and search time of moving object to achieve fast
Index Terms - Kalam filter, motion model, multi-object tracking. Establishing corresponding relationship through
tracking , Occlusion moving object features matching to deal with separation after
objects merged.
I. INTRODUCTION Experimental results show the proposed method is able to
Moving object tracking of video image sequences is one ensure an efficient and robust tracking with merge and split of
of the most important subjects in computer vision. It has multi-object.
already been applied in many computer vision field, such as II. OBJECT TRACKING USING KALMAN FILTER
video surveillance, artificial intelligence, military guidance,
safety detection, and robot navigation, medical and biological A. Typical Kalman filter
application etc..[1]. In recent years, a number of successful Mathematically, Kalman filer is an estimator that predicts
single-object tracking system appeared, but In the presence of and corrects the states of wide range of linear processes[9]. It
several objects, the problem is one of multiple object tracking is not only efficient practically but attractive theoretically as
where targets and observations need to be matched from well Precisely, the optimal state is found with smallest
frame to frame in a video sequence. 7he multiple object possible variance error, recursively. However, an accurate
tracking is still a challenging, and it will be harder especially model is an essential requirement.
in the case of that the objects have a similar appearance [2]. In Kalman filer, we consider a tracking system where
To deal with these problems, researchers did a lot of xk is the state vector which represents the dynamic behaviour
works.and gained many good achievements. Nguyen et al. [3] of the object, where subscript k indicate the discrete time. The
used Kalman filter in distributed tracking system for tracking objective is to estimate xk from the measurement zk .
multiple moving people in a room using multiple cameras. Following is the mathematical description of Kalman filer,
Whereas Chang et al. [4] use both Bayesian network and which for understanding we have sectioned into four phases.
Kalman filtering to solve the correspondence problem 1) Process equation
between multiple objects. In [5], a video surveillance system xk Axk 1  wk 1 (1)
is proposed where detection, recognition and tracking of
object is carried out. However, multiple objects are tracked by
using the c-constant velocity Kalman algorithm. The

978-1-4244-5704-5/10/$26.00 ©2010 IEEE 1862

Authorized licensed use limited to: Pukyong National Univ. Downloaded on November 01,2024 at 11:46:10 UTC from IEEE Xplore. Restrictions apply.
 Where A represents the transition matrix and xk the state window size slightly larger than the object image size, so that
at time k  1 to k . Vector wk 1 is the Gassian process noise it not only can reduce the noise interference, but also can
reduce image processing time and increase speed of operation.
N (.) With following normal probability distribution p( w) . Kalman filter tracking model can be divided into three sub-
p( w) ~ N (0, Q) (2) modules which are :motion model, feature matching, model
2) Measurement equation update[11].
zk Hxk  vk (3) 1) Motion Estimation Model
Where H is the measurement matrix and zk is the Kalman filter used for tracking is defined in terms of its
measurement observed at time k  1 to k respectively. vk is states, motion model, and measurement equations matrix X k
the Gaussian measurement noise N (.) with normal is an eight-dimensional system state vector, which can be
probability distribution p(v) . expressed as:
p(v) ~ N (0, R ) (4) xk [ x0, k , y0, k , lk , hk , vx , k , v yˈk , vl , k vh , k ]T (10)
3) Time update equations Where, x0,k , y0,k represent horizontal and vertical centroid
Equation (1) and (3) describes a linear model at time k . coordinate, lk , hk represent half-width and half-height of the
As xk is not measured directly, therefore the information tracking window, vx , k , v y , k , vl , k , vh, k represents their speed
provided by measured zk is used to update the unknown respectively.
states xk .Apriori estimate of state xˆk and covariance error xˆk The measurement vector of the system adopts the
estimate is obtained for the next time step k . ollowing from:
xˆk Axˆk 1  wk (5) zk [ x0, k , y0, k , lk , hk ]T (11)
Pk APk 1 AT  Q (6) In the following, A
is the transition matrix and H
is the
measurement matrix of our tracking system along with the
4) Measurement update equations
Gaussian process wk
and measurement vk . These noise values
These equations are associated with the feedback of the
system. The objective is to estimate aposteriori estimating xˆk are entirely dependent on the system that is being tracked and
which is a linear combination of the apriori estimate and the adjusted empirically.
new measurement zk .These equations are given below: ª1 0 0 0 't 0 0 0º
«0 't 0 0 »»
K k Pk H T ( HPk H T  R ) 1 (7) « 1 0 0 0
  «0 0 1 0 0 0 't 0»
xˆ k xˆ  K k ( zk  Hxˆ )
k k (8) « »
 «0 0 0 1 0 0 0 't »
Pk (1  K k H ) P (9) A
k «0 0 0 0 1 0 0 0»
K k is the Kalman gain which is computed by above the « »
«0 0 0 0 0 1 0 0»
measurement update equations. After that aposterior state «0 0 0 0 0 0 1 0»
estimate xˆk and aposterior error estimate Pk is computed by « »
the measurement zk .The time and measurement equations are ¬« 0 0 0 0 0 0 0 1 ¼»

calculated recursively with previous aposterior estimates to Observation matrix H can be described as:
predict new aprior estimate. This recursive behaviour of ª1 0 0 0 0 0 0 0º
«0 1 0 0 0 0 0 0 »»
estimating the states is one of the highlights of the Kalman H «
filter. «0 0 1 0 0 0 0 0»
« »
B. Kalman Filter for Multi-object Tracking ¬0 0 0 1 0 0 0 0¼
Describing the object’s geometric features can include After the state equation and measurement equation of
location, shape and center of mass (centroid)[10], etc.. The motion model are defined, in the next frame, Kalman filter can
camera we used in this paper sampling time is 30fps, so there be used to estimate the object’s location and size in a small
is relatively little changed of moving object in two adjacent range, and to gain trajectories of moving objects.
frames, so we can consider that the size of tracking window 2) Feature matching
and cetroid position of moving target in the adjacent two Each moving object is described by its centroid and
frames are little changed, that is said, moving target in the tracking window, the horizontal and vertical centroid
adjacent two frames of the movement is continuously. coordinates and the area of the i th object in the k th frame were
Therefore, we choose centroid and size of tracking window as respectively described as xki , yki and S ki .
the feature value to describe moving object. After the moving First, centroid distance function between the ith object in
objects have been segmented, some process preparations for the kth frame and the jth object in the k+1th frame is defined as:
the subsequent moving object tracking is needed. First to
assign a tracking window for each moving object in the scene, ( xki  xkj1 ) 2  ( yki  ykj1 ) 2
D(i, j ) (12)
in order to prevent a lot of noise easy to come in, tracking
Maxn ( xki  xkn1 ) 2  ( yki  ykn1 ) 2
window should not be too large. We make the tracking

1863
Authorized licensed use limited to: Pukyong National Univ. Downloaded on November 01,2024 at 11:46:10 UTC from IEEE Xplore. Restrictions apply.
 Scend, the area difference between the ith object in the k k1 k2 k3 k4
frame and the jth object in the k+1th frame is defined as:
S ki  S ki 1
A(i, j ) (13) TA TAc
Maxn S kn  S kn1
T Tc
Where, S ki 4lki hki , which represents the area of tracking TB TBc
window. It reflects the degree of deformation of the window,
Figure 1 Illustration for merge and split
the smaller the value is, the closer description of the two
and TB also will be updated as well while updating object T.
objects’s shape is.
With these definitions we define cost function is: When the update has finished, object T turns to object T ' .
V (i, j ) D D(i, j )  E A(i, j ) (14) Object T ' starts to split in time k3 , during time k3 to time
Where, D  E 1 , we can make D 0.8 , E 0.2 here. k4 .object T ' splits into two objects, TA' and TB' . Then we
The smaller the cost function’s value is, the two objects are make a judgement whether the object is T or not, if it is, we
more likely have correspondence. believe that the moving object is the one which has merged
3) Model Update before, then we matching TA' and TB' with TA and TB in a
When the minimum value of cost function is found, we certain range of objects’ locations, establishing and updating
use the k  1th frame features to update parameters of kalman the correspondence, then delete object T. If it is not, the
filter motion model, and use them as the input in the next splitted out object is a new, to establish a new Kalman filter
frame. Repeatedly doing this to finish the model update until motion model for the objects respectively.
the moving objects disappeared. The algorithm steps are as follows:
III. OCCLUSION PROBLEM 1 If the current image is the first frame, establishing
motion model and assigning tracking window for each
In this paper, we use background subtraction method to moving object in the scene. If the current image is the
detect and extract moving objects, through detection results to k th frame, and the moving object do not fall into any of
determine whether there has a occlusion or split between those eatablished tracking windows, we consider it is a
multiple objects. new object, establishing a new Kalman filter motion
When we found by detection that multiple objects’s model, initializing the model for tracking.
region connected together, we believe that the objects’ region 2. Searching features for each object near the tracking
have merged, to make the multiple objects as a whole object window in the scene, calculate the value of the cost
to track, and to establish a new eigenvalue for object function, the minimum value is the best match.
matching. When a object contains more than one moving 3. To judge whether there is a occlusion happened, if it
object split into several independent moving objects, first to happened, go to the merge or split treatment. If not,
judge whether merge occurred before, if it happend, matching keep tracking the object until it disappeared.
the split objects’ with objects’ feature before splitting. If not, 4 Turning to the handling of the next frame until the
we consider the objects are news by splitting, new eigenvalues object disappeared, the tracking is complete.
will be established and new tracking windows will be
assigned for moving objects tracking. IV. EXPERIMENTAL RESULTS AND ANALYSIS
The Figure. 1 gives a two moving objects example for A. Indoor moving human body tracking
illustration. Figure. 2 shows the tracking results for a video of moving
As shown in Figure 1, two moving objects TA and TB human bodies in an indoor scene, the two persons are
occlusion occurred in time k1 , they merged into to one object exchanging positions and both of them have been tracked
T in time k2 . Starting from time k2 , objcet T will be tracked properly in spite of the merge that occurs due to occlusion. In
as a new moving object. During time k2 to time k3 , object TA Figure 2(a), these two persons are initialized and assigned a
tracking window and ID reapectively when they come into
this

(a) (b) (c)

1864
Authorized licensed use limited to: Pukyong National Univ. Downloaded on November 01,2024 at 11:46:10 UTC from IEEE Xplore. Restrictions apply.
 Figure 2 Tracking results for two persons in an indoor scene

(a) (b) (c)

(d) (e) (f)

Figure 3 Tracking results for three persons in an outdoor scene A

(a) (b) (c)

(d) (e) (f)

Figure 4 Tracking results for three persons in an outdoor scene B

field of view, and the shape of the two person is filled with correctly after multi-object splitted, see figure. 3(f).In figure 4,
green so that we can easy to see the changes during the Figure the situation for tracking is more complex, each of the person
2(b), occlusion occurs of the two persons, so we merge them is a indepent object, not only the Kalman motion models are
into one object, seeing it as a new object for tracking, need to be established for every object, but also the judgement
simultaneously assigning a new ID for it. In figure 2(c), the is needed whether any of these objects merged together or
moving object occurs separation, the splitted objects maintain split from a muti-person object. The information is sent to
their IDs correctly by using feature matching of Kalman filter their associated trackers to maintain correct ID for tracking
motion. It is difficult to track moving human bodies because after multi-person object separated. The tracking results show
their shape change is difficult to track and their motion is that the Kalman filter based tracker does a good job of
difficult to learn and predict. However, here also we have tracking the human bodies when they get partially or
been able to track multiple persons.correctly after they merge completely occluded, even in the difficult case of
and separate. simultaneous merge and split.
B. Outdoor moving human body tracking C. Vehicle Tracking
Figure. 3 and Figure. 4 show the tracking results for a We use traffic sequences to verify the tracking method
video of moving human bodies in an outdoor scene. The based on Kalman filter, with the tracking results as shown in
situation in fiure 3 is similar with figure 2, there are two Figure4, we can see that the method has a good tracking speed
moving objects in the scene, and both occured occlusion, but and it can track fast moving object such as vehichles as well.
the diffirence of the two experiences is that the second object When a car come into the scene, it is be seen as a new
with two persons and they been seen as one whole identity, tracking object, to distribute a new nember and to initialize
the tracking window for that car. The tracking results of vehicles

1865
Authorized licensed use limited to: Pukyong National Univ. Downloaded on November 01,2024 at 11:46:10 UTC from IEEE Xplore. Restrictions apply.
show the tracking method is able to correctly handle the new
entry and exit moving objects in the traffic monitoring scenes.

(a) (b) (c)

(d) (e) (f)

Figure 5 Tracking results for vehicles

[6] S. A. Vigus, D. R. Bull, and C. N. Canagarajah, “Video object tracking

V. CONCLUSION using region split and merge and a Kalman filter tracking algorithm,”: In
proceedings of ICIP, 2001, pp. 650-653.
In this paper, we researched multi-target tracking [7] G. Medioni, I. Cohen, F. Bremond, S. Hongeng, and R. Nevatia, “Event
algorithm based on Kalman filter. First to establish Kalman detection and analysis from video streams,” IEEE Trans. Pattern Anal.
filter motion model, to choose centroid and tracking window Mach. Intell., vol. 23, no. 8, pp. 873–889, Aug. 2001.
[8] A. Czyzewski and P. Dalka, “Examining Kalman Filters Applied to
as the features. Selecting the center of mass moving targets
Tracking Objects in Motion” 9th International Workshop on Image
and tracking of the window as the feature. Through feature Analysis for Multimedia Interactive Services, pp. 175-178,2008.
matching to establish information links. The use of matching [9] S. Pathan, A. Al-Hamadi, and B. Michaelis, “Intelligent feature-guided
results update the model of moving objects. using the updated multi-object tracking using Kalman filteU” 2009 2nd International
model as the next frame of the input parameters. So that can Conference on Computer, Control and Communication, vol. 6, pp17-18
Feb, 2009
be achieved continuous tracking of objects. combining with [10]Shiuh-Ku,Weng C.M Kuo,and S.K Tu,” Video object tracking using
obtained detection information to judge whether there is a adaptive Kalman filter,” Journal of Visual Communication and Image
block between objects. the establishment of motion model to Representation,vol.17,no.6,pp1190-208,Dec.2006
the emerged object. establishing motion model to the [11]M. Han, Sethi.A.,W.Hua and Y.H. Gong, “A detection-based multiple
object tracking method,” 2004 International Conference on Image
separation of blocked moving objects. And using the Processing,vol.5,pp. 3065-8 ,Oct.2004
prediction of Kalman filter matching and tracking.
Experimental selected different scenes and objects, and
effectiveness and robustness of the algorithm have been
proved
REFERENCES
[1] W. Zeng, G.B. Zhu, and Y. Li, “Point matching estimation for moving
object tracking based on Kalman filter,” 8th IEEE/ACIS International
Conference on Computer and Information Science (ICIS), pp. 1115-
19,June 2009.
[2] Pathan, S.S, Al-Hamadi. A, and Michaelis.B,” Intelligent feature-guided
multi-object tracking using Kalman filter,” 2nd International Conference
on Computer, Control and Communication, pp.1-6, Feb. 2009.
[3] N. Nguyen, H. H. Bui, S. Venkatesh, and G. West, “Multiple
cameracoordination in a surveillance system.” ACTA Automatica Sinica,
vol.29, pp. 408-422, 2003.
[4] T. H. Chang, S. Gong., Tracking Multiple People with a Multi-Camera
System: IEEE Workshop on Multi-Object Tracking, 2001, pp. 19–26.
[5] H. YU, Y. Wang., F. Kuang, Q. Wan, “Multi-moving Targets Detecting
and Tracking in a Surveillance System,”the 5th World Congress on
Intelligent Control and Automation, China, June, 2004.

1866
Authorized licensed use limited to: Pukyong National Univ. Downloaded on November 01,2024 at 11:46:10 UTC from IEEE Xplore. Restrictions apply.