2017 - Extended Object Tracking Using IMM Approach For A Real-World Vehicle
2017 - Extended Object Tracking Using IMM Approach For A Real-World Vehicle
Abstract— Autonomous driving poses unique challenges for the consideration of a trade-off between system performance
vehicle environment perception due to the complicated driving and implementation complexity.
environment where the autonomous vehicle connects itself with In vehicle applications, it is necessary for vehicle trackers
surrounding objects. Precise tracking of the relevant dynamic
traffic participants (e.g., vehicle/byciclist/pedestrian) becomes a to treat the object as extended (rigid) target. The extended
key component for the task of comprehensive environmental object tracking is very different from traditional point target
perception and reliable scene understanding. It is necessary tracking approaches (say, in aerospace applications) in many
for vehicle trackers to treat the objects as extended (rigid) important aspects [9][3]. A shape model is first required to
target, as opposed to traditional point target tracking (say, present spatial extent of the object. A good shape model
in aerospace applications). The extended object tracking is
an extremely challenging problem in real world, due to high can largely alleviate many complicated issues, such as mea-
requirements of the object estimation on accuracy of kine- surement origin uncertainty, target (representative) behavior
matic/shape information, association robustness, model match description and measurement-to-track association. In this
on various target motion behaviors, and statistical property study, we focus on on-road vehicle object and present a
amicability (e.g., estimation consistency/covariance reliability). rectangular model with a specified tracking reference point
We present an extended object tracker — based on an in-
teracting multiple model with unbiased mixing estimator for (i.e., the most observable corner over time).
kinematic information at a specified tracking reference point, a In complicated vehicle driving scenarios, a set of repre-
truncated Gaussian scheme for shape (width/length/orientation) sentative dynamic models that follow kinematic physics of
estimation, and a hierarchical association method according the traffic objects is required to comprehensively describe
to both kinematic and shape information — to tackle all of various behaviors of the sounding objects. The powerful
the major challenges. Our special effort is put on handling
an intriguing conflict between theory and practice: the so- interacting multiple model (IMM) approach is a natural
called likelihood credibility issue. That is, the likelihood is choice for such situations [2]. In this study, we choose a
expected to credibly reflect the data statistical probability but is model set consisting of a white noise stop model, a nearly
actually distorted/drifting in real world systems, due to mainly constant velocity model, a nearly constant acceleration model
artificial physics introduced in multiple-stage data processing. and a coordinate turn model. The adaptive switching in these
In this study, from systematic point of view, we design an
interacting multiple model based extended object tracker with selected models (with a proper process noise design) can
proper likelihood compensation in the statistically-distorted real reasonably handle the common, say, vehicle cut-in and stop-
world. It can be shown that the presented tracker can deliver go scenarios.
an effective estimation performance in real road traffic of the Our ultimate goal is to design a practically-effective track-
imperfect world. ing system. In order to achieve this, we must be able to
handle unavoidable conflicts between theory and practice.
I. I NTRODUCTION
One of the most concerning issues is the so-called likelihood
Autonomous driving poses unique challenges for vehicle credibility issue; that is, the data likelihood cannot credibly
environment perception due to the complex driving envi- reflect the corresponding statistical probability, due to mainly
ronment where the autonomous vehicle finds itself in and artificial physics introduced in earlier stage data processing1 .
interacts with surrounding objects. Precise tracking of the rel- This could lead to a likelihood drift — an input measurement
evant traffic participants (e.g., vehicles/byciclists/pedestrians) of the seemingly maximum likelihood is not necessarily
becomes a key component for the task of comprehensive en- the most probable one. Failure to compensate for the drift
vironmental perception and scene understanding. The desire would cause big problems in correct data association and
for preciseness requires a good shape model for the object mode switching of an IMM estimator. A proper strategy is
spatial extent, a set of representative kinematic models that required to deal with the intriguing data statistical likelihood
can comprehensively describe the object dynamic behaviors,
1 Some other aspects, say, data drop-outs and data asynchronization, could
and a proper practical strategy to handle data statistical
add up to the issue but will not be discussed in the study; the corresponding
“distortion” (between theory and practice) in real world. We impact is expected to be reasonably handled to an acceptable level in earlier
aim to deal with all the issues in our tracker design with stage.
639
i.e., the most observable corner of an extended target.
B. Shape State Estimation
There is a shape-visibility issue, due to limited sensor
detection/illuminating capability. The cluster pattern of point
clouds belonging to a same object could change dramatically
over time (especially for a partially occluded object). We
propose the following truncated Gaussian procedure for
shape estimation, to overcome the pattern sudden change
issue and yield a “stability”,
Without loss of generality, assume at time tk−1 we have
a shape estimate
s(k − 1) ∼ p(s(k − 1)|zs1:k−1 ) (3)
= N (ŝ(k − 1), Ps (k − 1)) (4)
Fig. 2: Lidar point clouds of clean corner observations of
surrounding vehicle objects With a selected shape propagation model, say, a Wiener
process model with process noise vs (k),
s(k) = s(k − 1) + vs (k) (5)
III. DYNAMIC AND S HAPE S TATE E STIMATION WITH
S TATISTICAL L IKELIHOOD “D ISTORTION ” we could have a predicted shape state estimate as
The dynamic state x(k) and shape state s(k) estimation s(k|k − 1) ∼ p(s(k)|zs1:k−1 )
procedures are carried out in a decoupled manner; an IMM = N (ŝ(k|k − 1), Ps (k|k − 1))
with unbiased mixing is used for the reference point dynamic Δ
state estimation and a truncated Gaussian model is employed = N s̄(k), P̄s (k) (6)
for the shape estimation. A hierarchical association using It is desirable that the shape estimate could be reasonably
both kinematic and shape information is carried out. Our insensitive to pattern variations of observation clusters under
special effort is put on handling the likelihood credibility the rigid shape assumption. We modify the predicted shape
issue; the statistical “distortion” between the theory and estimate in (6) as a truncated Gaussian with support [as , bs ]
practice is compensated by a linear strategy with empirical
compensation factors learned from preliminary studies. s̄(k) ∼ pt (s(k)|z1:k−1 )
Δ
A. Dynamic State Estimation = N s̄(k), P̄s (k), as , bs
p(s(k)|z1:k−1 )
There are two reasons making the IMM estimator a natural for as ≤ s(k) ≤ bs
= C(as ,bs ) (7)
choice for the reference point dynamic state estimation: 0 for others
i)the reference point dynamic state is the representative
kinematic information of the extend object, whose motion where C (as , bs ) is a normalization constant.
behaviors subject to change over time from one to another; Then, we can have the following update
ii) the reference point of the extended object is only a s(k) ∼ p(s(k)|z1:k ) (8)
partial representation of the extended object’s motions — for
= p(s(k)|z1:k−1 )p(z(k)|s(k)) (9)
example, when a vehicle makes a smooth turn, the nearest
corner might seem static. This could introduce significant ≈ p (s(k)|z1:k−1 )p(z(k)|s(k))
t
(10)
difficulty in data association and the IMM could alleviate Δ
= p (s(k)|z1:k )
t
(11)
the issues to avoid tracking divergence.
The IMM estimator for the reference point state estimation for any given new shape measurement, which is a bounding
consists of three major steps [2]: 1) mixing/interacting of box generated from Lidar clusters [12] based on the popular
the mode-conditioned estimates; 2) mode-conditioned base- Density-based spatial clustering of applications with noise
state estimation (prediction and update) with selected modes; (DBSCAN) [7].
3) mode probability prediction, update and retrodiction. For It can be seen that starting from a truncated Gaussian
typical vehicle driving scenarios, an IMM estimator should prior, we can also arrive a truncated Gaussian posterior6 .
have at least a stop mode (to handle stop-go situation) The shape estimation stability (i.e., overcoming possible
and a constant velocity mode — if needed, nearly constant sudden changes in measurement pattern) can be achieved
acceleration mode and nearly coordinate turn mode can be by carefully selecting the truncated support, which shall
added. The challenge lies in that the “distorted” likelihood consider noise level and point spread of the observation
information could lead to inaccurate mode interacting. clusters.
The measurement for the dynamic state estimation is the 6 The conjugate relationship can be straightforwardly reached by consid-
position of the nearest corner of each moving remote vehicle, ering truncated Gaussian as a scaled Gaussian.
640
C. Hierarchical Association
50
X Position (m)
hierarchical data association scheme based on first the object
dynamic information and then the shape information. 20
0 5 10 15
The measurement (zj ) to track (x̂i ) association highly relies Fig. 3: Highly non-stationary data (in velocity component)
on the covariance information S ij , which could be contami-
nated. An integrated (joint) probability data association [11]
or global nearest neighbour approach can be carried out the vehicle is accelerating. The following linear compensa-
based on the likelihood information. tion strategy is used as a temporal empirical solution in our
Additionally, the object shape information can be used for scenario study
data association in a deterministic way. A simple geometric
intersection of the prediction shape and observed shape is λ̄ = aλ + b (13)
used. The extended sharp intersection is evaluated by two where the modified likelihood λ̄ is compensated from a scale
criteria: i) absolute area overlapping (say, 2 m2 ) and ii) factor a and a possible shifting factor b. The factors can be
area overlapping ratio (say, 40% of already observed area learned from the scenario data [1].
— calculated from estimated length and width). As we
will show later in the scenario study, the shape intersection IV. S CENARIO S TUDY
association is an effective add-on in some situations where
In the scenario study, a multiple stage estimation architec-
the kinematic statistical association fails.
ture is used (as shown in Fig. 4). The multiple-stage Bayesian
A more general association approach based on sequential
estimation procedure could easily cause likelihood drifting
multiple hypothesis test (particularly effective for two vehi-
issue.
cles driving in parallel, where association ambiguity could
happen) is introduced in [15]. DOGMA Inference Object Tracking
(Hidden Markov Clustering (Hidden Markov
D. Likelihood Drifting Raw Model) “Super- Model)
Measurements Measurements”
In many practical situations, multiple-stage data process-
ing/estimation procedure are generally employed to achieve Fig. 4: The environment perception architecture
desired performance with a tradeoff between system per-
formance and implementation complexity. This introduces
issues like artificially inflating covariance matrices [1], auto-
correlated measurement sequence [14][17], and in particular
for the current in-vehicle grid-based sensor fusion system
a quantization effect in the measurement grid generation/a
heuristic mass evidence being put on occupancy estima-
tion [13]. All the issues could cause the likelihood distor-
tion/drifting issue.
Here is an example that always happens in practical ap-
plications. Fig. 3 shows our non-stationary input data, which
is known to be unreliable for tracking. For such data, a fixed
covariance is generally used, due to a black-box agreement
among different function modules. Apparently, this is wrong,
leading to a likelihood that cannot credibly reflect the correct
data statistical property.
The likelihood drifting issue could lead to a disaster result
in data association and IMM mode switching. Wrong data Fig. 5: Tracker input data from an occupancy grid system
could be associated to an irrelevant object, especially for (rainbow ring indicates estimated target motion direction),
nearest neighbour based association; IMM could select a from which a measurement box is generated based on a
nearly constant velocity model as a best match but, in truth, clustering approach.
641
We focus on tracking on-road dynamic objects. We carry
out our performance evaluation with best available informa-
tion we could have in different recordings from different
aspects: position estimation accuracy, shape estimation ac-
curacy and data association correct rate.
Fig. 5 illustrates the tracker input from earlier stage dy-
namic grid system. There are two known issues for current
implementation [12][16] that can affect our tracker perfor-
mance: i) it provides occupancy estimates with spatial extent
larger than an object should be (i.e., the non-white area is
persistently larger than truth); ii) it cannot handle high-speed Fig. 7: Shape estimation scenario (visible/partially visible
object (due to a failure on the dynamic estimation in the objects)
occupancy grid system) — e.g., as shown in the figure, the
Shape estimation
high-speed target (labelled 1) has inconsistent colors for its 5.5
642
ground truth can be obtained) and real traffic on California
roads. It is shown the presented extended object tracker is an
effective real-world solution for the current in-vehicle sensor
fusion system.
R EFERENCES
[1] P. Abbeel, et al., “Discriminative Training of Kalman Filters”, In
Robotics: Science and systems, vol. 2(1), 2005.
[2] Y. Bar-Shalom, X. R. Li and T. Kirubarajan, Estimation with Ap-
plications to Tracking and Navigation: Algorithms and Software for
Information Extraction, Wiley, 2001.
(a) Frame at t = 894 s [3] M. Baum and U. D. Hanebeck, “Extended Object Tracking with
Random Hypersurface Models”, IEEE Trans. on Aerosp. Electronic
Systems, vol. 50(1): 149-159, 2014.
[4] D. Comaniciu, et al., “Kernel-Based Object Tracking”, IEEE Trans. on
Pattern Analysis and Machine Intelligence, vol. 25(5): 564-577, 2003.
[5] C. Coue, C. Pradalier , et al., “Bayesian Occupancy Filtering for Mul-
titarget Tracking: An Automotive Application”, International Journal
of Robotics Research, vol. 25(1): 19-30, 2006.
[6] R. Danescu, et al., “Modeling and Tracking the Driving Environment
with a Particle-Based Occupancy Grid”, IEEE Trans. on Intelligent
Transportation Systems, vol. 12(4), December 2011.
[7] M. Ester, H. P. Kriegel, J. Sander, and X. Xu, “A Density-based
Algorithm for Discovering Clusters in Large Spatial Databases with
Noise”, Kdd, vol. 96(34): 226-231, 1996.
[8] D. Kellner, et al.“Tracking of Extended Objects with High-Resolution
(b) Frame at t = 967 s Doppler Radar”, IEEE Trans. on Intelligent Transportation Systems,
vol. 17(5): 1341-1353, 2016.
Fig. 9: Object tracking on road [9] W. Koch and R. Saul, “A Bayesian Approach to Extended Object
Tracking and Tracking of Loosely Structured Target Groups”, 8th
International Conference on Information Fusion (FUSION 2005), 2005.
[10] C. Laugier, Igor E. Paromtchik, et al., “Probabilistic Analysis of
consists of a drive along a straight road, stopping at a traffic Dynamic Scenes and Collision Risk Assessment to Improve Driving
signal with multiple surrounding vehicles and a left turn at an Safety, IEEE Intelligent Transportation Systems Magazine, pp: 4-19,
October 2011.
intersection. The number of remote vehicles in the scenario [11] D. Musicki, E. Robin and S. Stankovic, “Integrated Probabilistic Data
was changing from 1 to 8. Fig. 9 shows some typical scenes. Association”, IEEE Trans. on Automatic Control, vol. 39(6): 1237-1241,
The data association correct rate is evaluated in terms of 1994.
[12] J. Peukert, “Object Generation from a Dynamic Occupancy Gridmap
track loss. It is observed that the tracks were lost about 10 Considering Autocorrelation”, Master Thesis, Institute of Systems Op-
(out of 1800, for the section we selected) instances during the timization, KIT, Karlsruhe, Germany, Feb. 2016
test. Compared to the number of correct data associations, the [13] S. Thrun, et al.,, Probabilistic Robotics, MIT Press Campridge, 2005.
[14] T. Yuan, B. Duraisamy, J, Peukert, et al., “De-autocorrelation Tracking
track loss is insignificant. It is worth noting that, if the data Scheme for Multiple-Stage Bayesian Estimation in Vehicle Sensor
association is only carried out on kinematic information only, Fusion Systems”, in revision, IEEE Trans. on Industrial Informatics
track loss would increase when the object vehicle makes Special Section, Oct. 2017.
[15] T. Yuan, K. Krishnan, Q. Chen, et al., “Object Matching for Inter-
turns; the shape intersection helps to largely alleviate such Vehicle Communication Systems — an IMM-based track association
track loss (however, wrong association occasionally happens approach with sequential multiple hypothesis test”, IEEE Trans. on
for nearby objects driving in parallel; this can be improved Intelligent Transportation Systems, vol. 99: 1-12, Sep. 2017.
[16] T. Yuan, D. Nuss, et al., “Fundamental Properties of Dynamic Occu-
by a sequential hypothesis test approach [15]). pancy Grid Systems for Vehicle Environment Perception”, 2015 IEEE
International Workshop on CAMSAP, Cancun, Mexico, Dec. 2015
V. C ONCLUSIONS [17] T. Yuan, J, Peukert, et al., “Object Tracking with De-Autocorrelation
In the paper, we present an extended object tracking mod- Scheme for a Dynamic Occupancy Gridmap System”, 2016 IEEE
International Conf. on MFI, Baden Baden, Germany, Sep. 2016.
ule for a grid-based vehicle sensor fusion system. We design
the tracker from a systematic point of view, in combination
with other modules (e.g., occupancy inference/clustering) to
achieve a desired overall system performance. We resort our
solution to a balance between theory and practice, as opposed
to pursuit of more advanced statistical approaches (this is the
next step). An IMM estimator with a hierarchical association
using both kinematic and shape information is presented.
A special effort is put on handling a likelihood credibility
issue caused by mainly artificial physics in earlier stage
data processing. A linear likelihood compensation strategy
is used to account for the observed statistical “distortion”. A
comprehensive scenario study is carried out on the recordings
from the SimCity (a Daimler facility where well-surveyed
643