M.Tech Credit seminar report, Electronic Systems Group, EE Dept.

IIT Bombay, Nov 2004

1
Detection and Tracking Algorithms for IRST
Sushil Kumar (04307406)
Supervisor: Prof. U.B. Desai
Abstract
Infrared search and track system is an integral part of modern weaponry. The
detection and tracking algorithm forms the heart of an IRST system and their
effectiveness plays an important role in determining performance of the system.
This report studies various detection and tracking algorithms for multiple point
targets in noisy environment resulting in very low signal to noise ratio. Target
detection is carried out using spatial-temporal techniques needing multiple frames,
since targets are assumed to be irresolvable in a single image frame. The tracking
algorithms are classified in the basis of different approaches for data selection and
model selection. Data selection and model selection is used for tracking multiple
targets in dense clutter environment. An overview of the Interacting Multiple Model
Expectation Maximization algorithm, and brief description of Multiple Hypothesis
Tracking and Joint Probabilistic Data Association Filter algorithm is also presented.
Keywords: IRST, IMM-EM, JPDAF, MHT.
1. Introduction
The ever-increasing effectiveness of electronic warfare and the advent of anti-radiation
missiles create the need for covert, radar silent operations. Therefore systems are required
which are virtually immune to jamming, undetectable and yet capable of detecting targets
at reasonable ranges. The infrared search and track system (IRST) is the most suitable
choice for such scenario. IRST is a system that surveys environment by analyzing the
infrared radiation, emitted by the targets compared to background. The major work of the
IRST is detection and tracking of target. In a practical scenario target are not in ideal
conditions, they are generally surrounded by clutter. Hence so detection and tracking is
much more complication in practical scenario.
The developments of efficient clutter rejection algorithms are important to detect the
targets in the modern IRST systems. In low signal to noise (SNR) situation, the target
could not possibly be localized on the single frame so the successive frames are used to
detect the target. If the alignment is done properly the signals of the various images
would add up and a signal with suffiently large SNR are achieved, while the noise will be
cancel out. This approach of detection of dim target is usually referred as target before
detection (TBD) and generally used for detection of dim targets.
Tracking is the estimation of the state of the moving object based on the remote
measurement. It uses models of the real environment to estimate the past and present and
even predict the future. The use of a tracking system is to extract information from a
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
2
dynamic system. A track is a symbolic representation of a target moving through an area
of interest and represented by a filter state which gets updated on each new measurement.
2. IRST system
It is becoming common for passive electro-optical sensors to be included in the design of
airborne and terrestrial military platforms. Passive electro optical sensors are valued for
their lack of sensor emissions, electronics countermeasures immunity. IRST system is
wide field of view passive electro optical surveillance system designed for autonomous
target detection and track acquisition. An IRST system uses thermal sensor to detect
targets. A typical IRST system as shown in fig.1 have thermal detector which receives the
thermal signatures of the target. The thermal signature is then processed by font end
electronics and given to the tracker. The function of the tracker is to detect and tracking
the target and give coordinate information to the servo control unit for controlling the
sensor head. The control and display unit is a user interface with the system, where the
target information is displayed.
3. Target Detection
To introduce the concepts involved in target detection, the simplest case consists of a
sub-pixel target in uniform Gaussian noise. Because the target does not take up the whole
pixel, the signal from the target pixel consists of the target signal and Gaussian noise
from the background. The output for the pixel at (i,j) with the target present is given by
Y(i, j) t N(0, )
and with target absent
Y(i, j) N(0, )
Fig 1. Infrared search and track system
Front end electronics
TRACKER
Clutter
suppression
And detection
Tracking
Sensors
Servo
control unit
OPTICS
USER INTERFACE
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
3
Where Y(i,j), is the pixel signal, t is the target signal and N(0, ) is a Gaussian distribution
with zero mean and standard deviation = . A detection threshold, T, is usually set to be a
multiple, k, of the background standard deviation, i.e.
T=k
Fig.2 Target and noise probability distribution [2]
Representative distributions of the target and noise signal, and an assumed threshold, are
given in Fig.2. The noise distribution has zero mean, by definition, and the target
distribution is the noise distribution shifted by the value of the target signal. The
probability of detection, PD, is the integral of the target distribution above the threshold,
and the probability of false alarm, PA is the integral of the noise distribution above the
threshold. The integral of the target distribution below the threshold is the probability of
missed detection.
The next simplest case is where the standard deviation of the background varies over the
image. If a constant threshold was used the false alarm rate would vary over the image. A
constant false alarm rate (CFAR) can be obtained by setting the threshold based on the
local standard deviation evaluated over a sub-window of the image.
The most complex case is where the background consists of rapidly varying clutter which
varies in both mean and standard deviation. It is then necessary to use a filter to suppress
the background clutter and enhance the response of the target. The aim is to generate an
output from the filter which is a field of Gaussian white noise containing target peaks.
The previous methods can then be used to detect the targets.
One approach to target detection is to use a matched filter. The filter weights are chosen
to give an increased response from the filter when the target is present and a decreased
response when clutter is present. When the target is single pixel in size the clutter
suppression becomes more important. Another approach is to carry out a smoothing
operation which removes the salt and pepper noise from the image. This smoothed image
is subtracted from the original image the result is a collection of point-like objects which
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
4
may contain a target. Basically there are two classification of filter in image processing
and the others are derivates of them.
3.1 Classifications of detection system
There are two models for a target detection system. The track-after-detection system
(TAD) and Track before detect (TBD).
The TAD involves a target detection process to produce position estimates of possible
targets in a single frame, which are passed to a tracker to produce likely target trajectories.
The TAD method can work well when the target is bright, but not suitable for dim targets.
Track-before-detect (TBD) method is used, when the target is dim there is not enough
information in a single frame to allow detection of a target. In this case, simplified
tracking procedures are combined with detection procedures to increase the
discrimination between targets and backgrounds. In this method a number of frames are
used to accumulate gain enough information of the targets for proper detection.
3.2 Issues involve in IRST algorithm (TBD)
1. Effective Adaptive Spatial-Temporal Technique for Clutter Rejection
2. Optimal nonlinear filtering for track-before-detect in IR image sequences.
3.2.1 Effective Adaptive Spatial-Temporal Technique for Clutter Rejection
The adaptive spatial temporal clutter rejection technique is based on a multi-parametric
approximation of clutter which, after estimation of parameters, leads to an adaptive
spatial- temporal filter. The coefficients of the filter are calculated adaptively to
guarantee a minimum of empirical mean-square values of the filtering residual noise for
every time moment. The adaptive spatial-temporal filtering algorithm allows one to
suppress any background, regardless of its spatial variation. Simultaneously, the
algorithm estimates LOS drift and allows for jitter compensation [3].
This detection scheme consists of two algorithms, clutter rejection and jitter
compensation. The first one is based on the multi-parametric approximation of clutter by
decomposing it as a sum of products of unknown parameters and given spatial basis
functions. Parametric methods involve the assumptions about the structure of the clutter
statistics, which are represented by the models with adjustable parameters. The Fourier
basis or wavelet basis may used as basis function. In the algorithm, these parameters are
estimated along with the jitter according to the maximum likelihood principle. Since the
support of the basis and the number of terms in the sum are chosen in advance, so this is a
Fixed Support algorithm (FS-Algorithm). The second algorithm is based on the spline
approximation where not only the parameters of the approximation, but also the support
of the approximation, are unknown. A simple example of such approximation is a stepwise
constant approximation in unknown domains. In this case, not only the amplitudes,
but also the domains, should be estimated. This algorithm is known as an Adaptive
Support algorithm (AS-Algorithm).
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
5
3.2.2 Optimal nonlinear filtering for track-before-detect in IR image
sequences
The 3D matched filter proposed by Reed et al.18 and its generalizations (e.g. banks of
assumed velocity filters (BAVF)) provide a powerful processing technique for detecting
moving low observable targets. This technique is a centerpiece of various track-beforedetect
(TBD) systems. However, the 3D matched filter was designed for constant velocity
targets and its applicability to more complicated patterns of target dynamics is not
obvious. The 3D matched filter and BAVF are extended to the case of switching multiple
models of target dynamics. The 3D matched filtering can be cast into a general
framework of optimal spatial temporal nonlinear filtering for hidden Markov models.
This is a robust and computationally efficient Bayesian algorithm for detection and
tracking of low observable agile targets in IRST system. The structure of TBD algorithm
consists of two parts tracking and detection. Tracking is performed by utilization of the
optimal (nonlinear) Bayesian filter (ONBF), for detection a modified Generalized
Likelihood Ratio (GLR) is used.[4]
3.3 Comparison of spatial-temporal and spatial filter
The performance of the spatial temporal filter is shown in the Fig.3. The picture on the
left hand side shows a typical input (clutter and noisy) frame. The picture on the right
side shows the output of spatial temporal filtering, with temporal window T=20. Fig. 4
shows the results of spatial filter for the same input. The statistics for the both the filters
are shown in the table 1.
Fig. 3 clutter rejection spatial- temporal filter [3]
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
6
Fig. 4 clutter rejection: spatial nonparametric filter [3]
Minimum Maximum Mean Variance Gain
Input 3.04 105.25 56.10 482.86
Output(spatio-temporal),
T=20
-10.24 9.44 -0.011 10.84 16.5(db)
Output (spatial) -36.34 34.60 -0.004 114.99 6.2 (db)
Table 1 comparison of spatial-temporal and spatial filter [3]
4.1 Multiple targets tracking system
A typical multiple target tracking system (MTT) is shown if Fig 5. Signal processing unit
converts the signals from the sensor to measurements, which become the input data of the
MTT system. The incoming measurements are first considered for the update of existing
tracks. Gating tests evaluate which possible measurements to track pairings are
reasonable and a more detailed association technique is used to determine final pairings.
Measurements not associated to existing tracks might be potential candidates for
initiating new tentative tracks. When the quality of the measurements is high enough, a
tentative track becomes confirmed. Similarly the low quality measurements become
deleted. The tracks are predicted ahead for the next iteration and the processing cycle
repeats.
A gate based upon the maximum acceptable measurement plus tracking predicted
magnitude is placed around the predicted track. Only those observations are within the
gate are considered for update of the track. When closely spaced targets produce closely
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
7
spaced observations there will be conflicts such that there may multiple observations
within a track s gate and an observation may be within the gates of the multiple tracks.
This is handled by observation-to-track association and track maintenance functions.
Recursive processing is typically being used and tracks are formed on previous tracks.
Fig. 5 Typical multiple target tracking system
4.1.1 Estimator
Estimation is process of selection of a point from a sample space. The sample space may
be continuous and discrete. More precisely estimation is a process of defining a
probability distribution over a sample space. Decision is a process of selection of one
particular value from sample space based on estimated probability density function. This
value is optimal in some sense. The optimality criterion is defined with a cost function.
Generally the process of estimation and decision are dealt as a single process is referred
as an estimation process.
A measurement at time instant k is denoted as z(k) and the measurement set containing
all measurements unto time k is denoted by Z k .
Measurement z(k) is function of state x as
z(k) = h(k,x,w(k))
Where w(k) is measurement disturbance at time k.
An estimator is a function from measurement Z k to some value x that estimate the true
state x in some sense
x x(k, zk )
An estimate is particular value of the estimator function induced by the received
measurements.
Sensor data
processing and
measurement
formation
Observation-totrack
Association
Track maintenance
(Initialization,
confirmation and
detection)
Filtering and
prediction
Gating
computations
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
8
An likelihood function (x) describes how likely the measurements z(k) gives x
(x) p[zk / x]
A maximum likelihood is the estimate the value that maximize the above likelihood
function and given by
argmax ( ) argmax [ / ]
ML
k
xx
xxpzx
Maximum a posterior estimate takes into account the prior distribution for the random
variable x
This is the Baye s theorem and given by
argmax [ / ] argmax [ / ] [ ]
MAP
kk
xx
xpxzpzxpx
4.1.2 Data association
The formation of target tracks in a mulitarget environment is confounded by a significant
degree of uncertainty in the origin of target measurements. The procedure used to handle
the measurement origin uncertainties is called the data association. The data association
task can into two parts i.e observation-to-observation (track formation ) and observationto-
track association or track maintenance.
4.2 Tracking
Tracking of targets with less-than-unity probability of detection in the presence of false
alarms, data association is crucial. Tracking uses models of the real environment to
estimate the past and present and even predict the future. The use of a tracking system is
to extract information from a dynamic system A track is a symbolic representation of a
target moving through an area of interest. Internally, in the tracking system, a track is
represented by a filter state which gets updated on each new measurement.
Tracking in a multi-target, clutter is characterized by uncertainty in the origin of the
measurements. The tracking function consists of
1. Estimation of the current state of the target (i.e. filtering) based on uncertain
measurements selected according to certain rules [1]
2. Calculation of accuracy (usually covariance) and credibility associated with the state
estimation.
4.2 Tracking algorithm
4.2.1 IMM-EM algorithm
IMM-EM (interacting multiple model expectation maximization) algorithm involves EM
based tracking algorithm using interacting multiple models.
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
9
This algorithm is divided into two major parts. In the first part, the state is filtered using
EM algorithm, and the likelihood model is calculated for each model. . It is followed by
an IMM step, which updates the combined state estimate, the model probability, and
predicts the state for the next time instant. The centroid of the measurement is used for
evaluation the model probability [5].
4.2.1.1 Interacting multiple model algorithm
In the Interacting multiple model (IMM), uses multiple models that interacts through
state mixing to track a maneuvering target. The IMM consists of filter for each model. It
evaluates model probability for each filter and it is used for mixing state estimate of the
target. In IMM, for each target many filters based on different model are initialized.
4.2.1.2 EM algorithm
EM is an iterative optimization method to evaluate target state estimate using the
previous states estimate. This algorithm has two steps E-step and M-step. E-step can be
interpreted as constructing a local lower-bound to the posterior distribution, whereas the
M-step optimizes the bound, thereby improving the estimate for the unknowns.
4.2.1.3 Algorithm steps
In IMM algorithm to find the model probability of each model, it is required to find the
likelihood of an observation.
t N = targets present during the complete sequence of image frames
t ,k = combined state estimate at time instant k for target t
m( )
k t = state at time instant k by model m for target t
k Y = observation process at time instant k
k N = number of measurements obtained at time k
k Z = association process at time instant k
M = total number of models used in the IMM algorithm.
First the centroid t ( )
k y t of the observation for target t at time instant k is calculated. This
step is executed only once before starting the EM iteration. As soon as the observation set
is available, the assignment weight ( )
t
zk t is calculated and it is used to ( ) t
k y t given by
(0)
,,
1
(0)
,
1
()
()
()
k
k
N
kiki
ci
kN
ki
i
zty
yt
zt
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
10
Fig. 6 IMM-EM algorithm for two models
Based on this centroid of observations, the model likelihood is calculated and it is used to
evaluate the model probability. The centroid of observation t ( )
k y t is not used for state
estimation of the target. In Fig.6 shown the IMM-EM algorithms for two models, in
which k L and k P represents the likelihood and covariance calculated at time instant k.
The IMM-EM algorithm operates as follows
As the current set of observation k y become available, following two steps are performed
at time instant k:
(i) The observation set k y is validated using combined state prediction k / k 1 for a given
target.
(ii) EM step is evaluated sequentially for each target and for each model of a given target,
which are interlinked with each other through assignment weights. After completion of
EM step, for each target IMM step is executed.
4.2.2 Multiple Hypothesis algorithm (MHT)
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
11
MHT is a deferred decision logic in which alternative data as association hypotheses are
formed whenever the observation-to-track situations occurs, Rather than choosing the
best hypotheses, or combining the hypotheses as in the JPDA, the hypotheses are
propagated in a future in anticipation that subsequent data will resolve the uncertainty [6].
In the MHT, the hypotheses are carried over from previous scan. Then on the receipt
from the new data, each hypothesis is expanded into a set of new hypotheses by
considering all observation to track assignments for the tracks within the hypotheses. Fig
7. illustrate the logical overview of MHT algorithm. The family structure (node)
provides convenient mechanism for implementing MHT.
Fig 7. MHT logic overview
4.2.3 JPDA algorithm
The joint probabilistic data association (JPDA) algorithm is used to track multiple targets
by evaluating the measurement-to-track association probabilities and combining them to
find the state estimate
Two simple solutions for track update are strongest-neighbor filter (SNF) and the
nearestneighbor
filter (NNF). In the SNF, the signal with the highest intensity among the
validate measurements (in a gate) is used for track update and the others are discarded. In
the NNF, the measurement closest to the predicted measurement is used. While these
simple techniques work reasonably well with benign targets in sparse scenarios, they
begin to fail as the FA rate increases or with low observable (low probability of target
detection) maneuvering targets. Instead of using only one measurement among the
received ones and discarding the others, an alternative approach is to use all of the
validated measurements with different weights (probabilities), known as probabilistic
data association (PDA) [7].
The PDA algorithm calculates in real-time the probability that each validated
measurement is attributable to the target of interest. This probabilistic (Bayesian)
Gate Form new Track/
Hypothesis
Hypothesis/Track
Evaluation/deletion
Track
Prediction
Input
observation
Most likely
Hypotheses
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
12
information is used in a tracking filter, the PDAF, which accounts for the measurement
origin uncertainty.
4.4 Comparison
MHT performance is comparable to the conventional single hypothesis (GNN) method at
10 to 100 times the false alarm density of the GNN, but it is computationally complex
algorithm and not suitable for the real time systems.
JPDAF algorithm is less complex and faster the MHT algorithm. IMM-EM can
successfully tack maneuvering and non-maneuvering targets simultaneously. It takes less
time than JPDAF and it is suitable for real time applications.
5 Conclusion
The detection algorithm should have jitter compensation technique for detecting the
target with low false alarm so the adaptive spatial temporal algorithm is preferred over
the other. IMM-EM is faster algorithm which is suitable for implementing in real time
application. So the combination of these two algorithms will result a robust and reliable
system.
M.Tech Credit seminar report, Electronic Systems Group, EE Dept. IIT Bombay, Nov 2004
13
References
[1] Yaakov Bar-Shalom, Multitarget-Multisensor Tracking Advance Applications ,
Artech House Inc 1990.
[2] Robert Charles Warren, Detection of Distance Airborne Targets in Clutter
Backgrounds in Infrared Image Sequences .
[3] Alexander Tartakovsky and Rudolf Blazek, Effectve Adaptive Spatial-temporal
Technique for Clutter Rejection in IRST SPIE proceedings: signal and data
processing of small target,Vol.4048, Orlando,FL,2000.
[4] Boris Rozovskii and Anton Petrov, Optimal Nonlinear Filtering for Track-
Before-Detect in IR sequences ,SPIE Proceedings, Vol. 3809, Denver,1999.
[5] Mukesh A Zaveri, S N Merchant and Uday B Desai, Multiple Model Based
Tracking of Multiple Point Targets using Expectation Maximization Algorithm in
Infrared Image Sequence IIT Bombay,Mumbai, India.
[6] Samuel S.Blackman, Multiple Hypothesis Tracking for Multiple Target Tracking
IEEE,Vol 19,No.1,2004.
[7] Thiagalingam Kirubarajan and Yaakov Bar-Shalom, Probabilistic Data
Association Techniques for Target Tracking in Clutter , IEEE,Vol. 92, No.3,
2004.
This document was created with Win2PDF available at http://www.daneprairie.com.
The unregistered version of Win2PDF is for evaluation or non-commercial use only.