Performance valuation of racking lgorithm

Incorporating Attribute Data Processing via DSmT 1

J. Dezert
ONERA
29 Av. De la Division Leclerc,
92320 Chatillon,
France
[email protected]

A. Tchamova, L. Bojilov, P. Konstantinova

Institute for Parallel Processing,
Bulgarian Academy of Sciences,
Acad. G. Bonchev Str., bl. 25-A, 1113 Sofia
Bulgaria
[email protected], [email protected], [email protected]

Abstract The main objective of this paper is to

representing either position, distance, angle, velocity,

accelerations etc.) with correct targets; track filtering to
estimates and predict the state of targets once data
association has been performed. The first step is very
important for the quality of tracking performance since its
goal is to associate correctly observations to existing
tracks. The data association problem is very difficult to
solve in dense multitarget and cluttered environment. To
eliminate unlikely (kinematic-based) observation-to-track
pairings, the classical validation test [3,7] is carried on the
Mahalanobis distance

investigate the impact of the quality of attribute data

source on the performance of a target tracking
algorithm. An array of dense scenarios arranged
according to the distance between closely spaced targets
is studied by different confusion matrices. The used
algorithm is Generalized Data Association (GDA-MTT)
algorithm for multiple target tracking processing
kinematic as well as attribute data. The fusion rule for
attribute data is based on Dezert-Smarandache Theory
(DSmT). Besides the main goal a comparison is made
between the cited above algorithm and an algorithm
with Kinematic based only Data Association (KDAMTT). The measures of performance are evaluated
using intensive Monte Carlo simulation.
Keywords: Tracking, data association, estimation,
Dezert-Smarandache Theory (DSmT), fusion rules.

d 2j (k ) = 'j ( k ) S 1 j ( k ) ,
where

is

the

difference

between the predicted position z ( k ) and the j th

validated measurement z j (k ) , S is the innovation
covariance matrix, is a threshold constant defined from
the table of the chi-square distribution [3]. Once all the
validated measurements have been defined for the
surveillance region, a clustering procedure defines the
clusters of the tracks with shared observations. Further the
decision about observation-to-track associations within the
given cluster with n existed tracks and m received
measurements is considered. The Converted Measurement
Kalman Filter (CMKF) [5] coupled with a classical
Interacting Multiple Model (IMM) for maneuvering target
tracking is used to update the targets state vectors. In the
CMKF algorithm the classical linearized conversion is
used as the value of validation indicator for unbiased

1 Introduction
Target tracking of closely spaced targets is a challenging
problem. The kinematic information is often insufficient
to make correct decision which observation to be
associated to some existing track. A new approach
presented in [16] describes Generalized Data Association
(GDA) algorithm incorporating attribute information. The
presented results are encouraging, but it is important to
study the algorithm performance for more complex
scenarios with more maneuvering targets and different
levels of quality of attribute data source. It is important to
know the level of quality of the used attribute detection to
assure robust target tracking in critical, highly conflicting
situations. The goal of this paper is by using Monte Carlo
simulation to determine the sufficient level of quality of
attribute measurements that for given standard deviations
of the kinematic measurements (in our case azimuth and
distance) to assure allowable miscorrelations.

filtering, proposed in [11]

2 Problem formulation
Classical target tracking algorithms consist mainly of two
basic steps: data association to associate proper
measurements (usually kinematic measurement z ( k ) )
1

j ( k ) = z ( k ) z j (k )

(1)

< 0.4 is less than 0.01

r

in our scenario. The GDA-MTT improves data association

process by adding attribute measurements (like amplitude
information or RCS (radar cross section) [#16-7] ), or
eventually as in [6], target type decision coupled with
confusion matrix to classical kinematic measurements to
increase the performance of the MTT system. When
attribute data is available, the generalized (kinematic and
attribute) likelihood ratios are used to improve the
assignment. The GNN approach is used in order to make a
decision for data association on integral criterion base.
The used GDA approach consists in choosing a set of

This work is partially supported by MON grant MI-1506/05 and by Center of Excellence BIS21++ grant (FP6-2004-ACC-SSA-2).

15

{ }

assignments

ij

i = 1, , n and j = 1,, m , volume chosen as in [6] as Vc =

for

n
i= 1

with

a ij ,
a ij = log( LRgen ( i, j ) )

2.2.Attribute Likelihood Ratios for GDA

LRgen ( i, j ) = LRk ( i, j )LRa ( i, j ) ,

The major difficulty to implement GDA-MTT depends on

where LRk ( i, j ) and LRa ( i, j ) are kinematic and attribute

likelihood ratios respectively, and

1 if measurement j is assigned to track i

ij =
0 otherwise

aij take the following values [12]:

aij =
log ( LRk ( i , j ) LRa ( i , j ) )

if d ij2 >
if d ij2

The solution of the assignment matrix is the one that

minimizes the sum of the chosen elements. We solve the
assignment problem by realizing the extension of Munkres
algorithm, given in [9]. As a result one obtains the optimal
measurements to tracks association. Once the optimal
assignment is found, i.e. the correct association is
available, then standard tracking filter is used depending
on the dynamics of the tracking targets.

H j (j=1
for Fighter, j=2 for Cargo), included in the frame 2
probabilities [15] for the target type hypotheses,

conditioned by the correct assignment. These pignistic

probabilities are derived after the fusion between the
generalized basic belief assignments of the tracks old
attribute state history and the new attribute/ID observation,
obtained within the particular fusion rule. It is proven that
this approach outperforms most of the well known ones
for attribute data association. It is defined as :

2.1 Kinematic Likelihood Ratios for GDA

The kinematic likelihood ratios LR k ( i, j ) involved into

aij

are easy to obtain because they are based on the classical

statistical models for spatial distribution of false alarms
and for correct measurements [5]. LRk ( i, j ) is evaluated
LRk ( i, j ) = LFtrue ( i, j ) / LF false ,
as:
where LFtrue is the likelihood function that the
measurement j originates from a target (track) i and
LF false is the likelihood function that the measurement j
originates from a false alarm. At any given time k , LFtrue
is defined as
r
LFtrue = l = 1 l ( k ) LFl ( k ) ,
where r is the number of the models (in our case of two
nested models r = 2 are used for CMKF-IMM, l ( k ) is
the probability (weight) of the model l for the scan k ,
LFl ( k ) is the likelihood function that the measurement j
originates from target (track) i according to the model l ,
i.e.

LFl ( k ) =

1
2

S (k)
i
l

d l2

aij , and more

specifically the attribute likelihood ratios LRa ( i, j ) for
correct association between measurement j and target i
the correct derivation of coefficients

based only on attribute information. When attribute data

are available and their quality is sufficient, the attribute
likelihood ratio helps a lot to improve MTT performance.
In our case, the target type information is utilized from
RCS attribute measurement through fuzzification
interface. A particular confusion matrix is constructed to
model the sensors classification capability.
The approach for deriving LRa ( i, j ) within DSmT
[10,14,15] is based on relative variations of pignistic

Or, when the assignment matrix A = [ aij ] is constructed its

elements

12 Rii . In our

case,

j = 1 ij

where

nz
i= 1

n z = 2 is the measurement vector size and Rii are

sensor error standard deviations for azimuth and
distance D measurements.

that assures maximum of the total generalized likelihood

ratio sum by solving the classical assignment problem

min

( i, j ) / 2

( )

i P* =

= T
i P |Z
i
*

)
(3)

PT* Z ( H j ) PT*i ( H j )
2
i P * | Z = j= 1 i
PT*i ( H j )

PT*i Z= Ti ( H j ) PT*i ( H j )
2

*
i P | Z = Ti = j= 1
PT*i ( H j )

i.e. i P * | Z = Ti is obtained by forcing the attribute

observation mass vector to be the same as the attribute
mass vector of the considered real target, i.e.
mZ ( .) = mTi ( .) . The decision for the right association relies
on the minimum of expression (3). Because the
generalized likelihood ratio

RLgen is looking for the

maximum value, the final form of the attribute likelihood

ratio is defined to be inverse proportional to the

LF false is defined as LF false = Pfa / Vc , where Pfa is the

false alarm probability and

where

with

= T
i P* | Z i P* | Z
i

Vc is the resolution cell

16

i ( P* )

i defining the number of the track, i.e.

( )

LR a ( i, j) = 1 / i P * .

3 Numerical experiments frame and

results

9500
9000
8500

3.1 Experiments frame

8000

x 10

4
25
2 12 22 43
21 09
11 87
11 65
6 7 8 9 1 01 1 211 34

2
1 .5

5
1 2 34

Y [m ]

0 .5
0
-0 .5
-1

F
F F FF
FFF
FF
F
F F F F F F F FF

-1 .5
F
F F FF

-2
-2 .5

-2

-1 .5

-1

-0 .5
X [m ]

0 .5

1 .5
4

x 10

Figure 1 : Multitarget scenario with 25 targets

The typical tracking performances for KDA-MTT and
GDA-MTT algorithms are shown on figures 2 and 3
respectively.

Y [m ]

7500

For the experiments we use an extension of the program

packet TTLab [13], written in MatLab. This extension
takes into account the attribute information. A programhuman interface facilitates the changing of the design
parameters of the algorithms.
The simulation scenario consists of twenty five air targets
(Fighter and Cargo) moving in three groups from NorthWest to South-East with constant velocity of 170[m/sec].
The stationary sensor is at the origin with Tscan = 5 [sec],
measurement standard deviations 0.3[deg] and 100[m] for
azimuth and range respectively. The headings of the
central group are 135[deg] from North and for the left and
right groups are 150[deg] and 120[deg] respectively.
During the scans from 15th to 17th and from 48th to 50th
the targets of the left and right groups perform maneuvers
with transversal acceleration 4.4[ m / sec 2 ]. The targets
are closely spaced especially in the middle part of their
trajectories. The scenario is shown on figure 1.

7000
6500
6000
5500
5000
4500
-2

-1 .5

-1
X [m ]

-0 .5
x 10

Figure 3 : Typical performance with GDA-MTT

The Track Purity performance metrics is used to examine
the ratio of the correct associations. Track purity is
considered as a ratio of the number of correct observationto-track associations (in case of detected target) over the
total number of all possible associations during the
scenario tracking.
Our aim in these experiments is to investigate what level
of classifier accuracy we need in a particular scenario with
the given separation between closely spaced targets.
Recalling that the confusion matrix (CM) specifies the
prior accuracy of the classifier we perform consecutive
experiments starting with CM corresponding to the highest
accuracy and ending with a matrix close to real life.
Beforehand, we have implemented a series of experiments
with highest accuracy CM and different separations of the
targets starting with prohibited close separation
(approximately d = 1.5 res ; here res is residual
standard deviation, ranging from 260[m] at the beginning
of the trajectory to 155[m])[2]. With these experiments we
try to find out the particular targets separation which
insures good results in term of tracks purity metrics.
Besides the algorithm processing attribute data on the base
of Proportional Conflict Redistribution Rule number 5
(PCR5) from DSm theory simultaneously the same
tracking algorithm is run with the kinematic data
processing only.
3.2 Numerical results
We started our experiments with series of runs with
different target separation and confusion matrix

9500

0.995 0.005
CM =
.
0.005 0.995

9000
8500
8000

Y [m ]

7500
7000
6500
6000
5500
5000
4500
-2

-1 .8

-1 .6

-1 .4

-1 .2
X [m ]

-1

-0 .8

-0 .6
x 10

Figure 2 : Typical performance with KDA-MTT

Hereafter, because of symmetry we will show the first row

of the matrix only. All the values in the tables below are
averaged over the 50 Monte Carlo runs. At a distance of
300[m] between targets the results are extremely
discouraging for both the kinematic only and kinematic
and attribute data used (the first row of table 1). There is
no surprise because this separation corresponds to less
than 1.5 res . This row stands out with remarkable ratio
of attribute to kinematic percents of tracks purity. In

17

the kinematic case less than one tenth of tracks are

processed properly while with using the attribute data
almost two thirds of targets are not lost. Nevertheless, the
results are poor and unacceptable from the practical point
of view. In the next rows we increase gradually the
distance between targets reaching separation of 600[m].
This distance corresponds to 2.5 res and the results are
good enough especially for the DSmT based algorithm.

Distance [m]
300
350
400
450
500
550
600

CM(0.995, 0.005)

Table 3 : Distance =450[m] , PCR5 algorithm

Track purity [%]

GDA (PCR5)
KDA
57.99
8.65
74.47
12.43
87.45
21.17
93.24
35.47
95.94
56.12
96.74
74.74
97.76
86.40

Distance [m] 450

Confusion Matrix
0.995 0.005
0.99
0.01
0.98
0.02
0.97
0.03
0.96
0.04
0.95
0.05
0.94
0.06
0.93
0.07
0.92
0.08
0.91
0.09
0.90
0.10

Track Purity
93.24
91.51
89.53
86.83
85.26
82.48
79.41
75.38
75.25
74.27
70.69

Track Purity
93.24
82.48
70.69
52.04
46.90
43.01
42.20

After correct association is made the classical IMM

Kalman filtering algorithm is used to diminish position
errors. The figures 4 and 5 shows the errors along axes X
and Y with and without filtering. It can be seen the effect
of significant reduction of the sensor errors after filtering.
On figure 4 is presented the result of more precise model
1, and on figure 2 is the result of model 2 with bigger
values for errors.
140
X
Y
R
R

120

e r r o r -f ilt e r e d
e r r o r - f ilt e r e d
e r ro r a lo n g X
e r ro r a lo n g Y

100
80

60
40
20
0

10

20

30
scans

40

50

60

Figure 4 : Monte Carlo estimation of errors

along axes x and y for model 1
140
X
Y
R
R

120

e r r o r s fo r m o d e l 2

The next step is to choose this medium separation size

which ensures highly acceptable results. We take the
distance of 450[m] because it is in the middle of the table
and its results are very close to that of larger distances.
Now we start our runs with confusion matrix
(0.995;0.005) corresponding to highest accuracy and
gradually change its elements to more realistic values
(table 2). In this table the tracks purity data for
kinematic only algorithm are omitted because they do
not depend on confusion matrix values. Now we choose
the threshold of 85% for tracks purity value above which
could be said that the results are satisfying enough.
Actually, the choice of threshold is a matter of an expert
assessment and strongly depends on the particular
implementation.It can be seen from the table that the last
row stepping from the top with tracks purity value above
the chosen threshold is the row with CM with elements
(0.96;0.04). So that, if our task is to track targets separated
at normalized distance approximately 1.5 res to 3 res
we have to ensure classifier with mentioned above
confusion matrix. As a comparison could be remained the
value of tracks purity ratio for the kinematic algorithm
for this separation 35.47%.
Table 2 Track purity results with different confusion
matrices for scenario with distance 450[m]

Distance [m] 450

Confusion Matrix
0.995
0.005
0.95
0.05
0.90
0.10
0.80
0.20
0.70
0.30
0.60
0.40
0.55
0.45

E rro rs fo r m o d e l 1

Table 1: Pd=0.995,

Some additional experiments have been performed with

continuing change of the elements of CM worsening the
classifier accuracy and trying to answer the question how
looks the CM which do not influence the value of tracks
purity ratio, i.e. when the attribute algorithm gives the
same results as kinematic one for the chosen separation.
The results can be seen in table 3. Even for the values of
elements of CM close to the natural limit values of
(0.5;0.5) the investigated ratio remains slightly better (the
last row of table 3) than that of kinematic algorithm.

e r r o r -filte r e d
e r r o r - f ilt e r e d
e r r o r a lo n g X
e r r o r a lo n g Y

100
80
60
40
20
0

10

20

30

40

50

60

scans

Figure 5 : Monte Carlo estimation of errors

along axes x and y for model 2
On figure 6 the result for distance errors for the two
models is presented. It can be seen that the errors for the
more precise first model the errors are lower.

18

d is ta n c e e r r o r s fo r 1 -s t a n d 2 -n d m o d e ls

110
m o d e l1
m o d e l2

100
90
80
70
60
50
40

10

20

30

40

50

60

scans

Figure 6 : Monte Carlo estimation of distance

errors for first and second models

Conclusions
In this paper a series of experiments have been performed
aiming to investigate the influence of some circumstances
and values of some particular parameters on performance
capability of multiple target tracking algorithm processing
both kinematic and attribute data. The algorithm is based
on Global Nearest Neighbour-like approach and uses
Munkres algorithm to resolve the generalized association
matrix. The principles of Dezert-Smarandache theory of
plausible and paradoxical reasoning to utilize attribute
data are applied. The results show that even in dense
target scenarios and realistic accuracy of attribute data
classifier the algorithm performance meets requirements
concerning its practical implementation. Beside this
inference, the results once more underline the advantage
of used algorithm utilizing both kinematic and attribute
data over that one working with kinematic data only.

References
[1] Angelova D., B. Vassileva and Tz. Semerdjiev, Monte
Carlo-based Filter for Target Tracking with Feature
Measurement, Proceedings of the Fifth International
Conference on Information Fusion, July 8-11, Annapolis,
Maryland, USA, pp. 1499-1505, 2002
[2] Bar-Shalom
Y.,
Ed.,
Multitarget-Multisensor
Tracking: Advanced Applications, Norwood, Artech
House, 1990.
[3] Bar-Shalom Y., Fortmann T., Tracking and Data
Association, Academic Press, 1988
[4] Bar-Shalom, X.-R. Li, Estimation and Tracking:
Principles, Techniques and Software, Artech House, 1993.

[5] Bar-Shalom, X.-R. Li, Multitarget-multisensor

tracking:Principles and techniques, YBS, 1995.
[6] Bar-Shalom, Kirubarajan T., Gokberk C., Tracking
with classification-Aided Multiframe data Association,
Proc. of SPIE Conf. on Signal and Data Processing of
Small Targets, Vol. 5204, San Diego, Aug. 2003.
[7] Blackman S. Multitarget tracking with Radar
Apllications , Artech House, 1986
[8] Blackman S. and R.Popoli, Design and Analysis of
Modern Tracking Systems, Norwood, MA, Artech House,
1999
[9] Bourgeois F., J.C. Lassalle, An Extension of the
Munkres Algorithm for the Assignment Problem to
Rectangular Matrices, Communications of the ACM,
Vol.14, Dec.1971, pp.802-806.
[10] Dezert J., Foundations for a new theory of plausible
and paradoxical reasoning, Information & Security, An
international Journal, edited by Prof. Tzv. Semerdjiev,
CLPP, Bulgarian Academy of Sciences, Vol.9, 2002,
pp.13-57.
[11] Lerro Don, Y. Bar-Shalom, Tracking With Debiased
Consistent Converted Measurements Versus EKF, Trans.
On Aerospace and Electronic Systems, vol. 29, No 3, July
1993, pp1015-1022.
[12] Popp R. L., K. R. Pattipati and Y. Bar-Shalom,
Dynamically Adaptable m-Best 2-D Assignment
Algorithm and Multilevel Parallelization, IEEE Trans.on
Aerospace and Electronic Systems, Vol. 35 No.4, Oct.
1999.
[13] Semerdjiev Tzv., Djerassi Em., Bojilov L.,
Konstantinova P., Sensor Data Processing for Target
Tracking. Algorithms and Applications, [in Bulgarian]
SoftTrade, (2006)
[14] Smarandache F., Dezert J., (Editors), Advances and
Applications of DSmT for Information Fusion (Collected
Works), American Research Press, Rehobooth, 2004.
[15] Smarandache F., Dezert J., (Editors), Advances and
Applications of DSmT for Information Fusion (Collected
Works), vol. 2, American Research Press Rehobooth,
2006.
[16] Watson G., Blair W.D., Benchmark Problem for
Radar Resource Allocation and Tracking Maneuvering
Targets in the presence of ECM, Technical Report
NSWCDD/TR-96/10, 1996.

All in-text references underlined in blue are linked to publications on 19

ResearchGate, letting you access and read them immediately.