Color-Invariant Motion Detection

under Fast Illumination Changes

Paper by:Ming Xu and Tim Ellis
CIS 750
Presented by: Xiangdong Wen
Advisor: Prof. Latecki
 Agenda

>Introduction
>Color Fundamentals
>Color-Invariant Motion Detection
>Experimental Results
>Discussion
>Conclusion
 Introduction
>Motion detection algorithms: based on the differencing operation of
image intensities between a frame and a background image.
>Background image: reflects the static elements in a scene.
>Background image needs to be updated because
>Lack of a target-free training period;
>Gradual illumination variations;
>Background objects which then move.
>Updating scheme
Linear interpolation between the previous bg value and new observation
>Gaussian mixture model: based on gray-level or RGB color intensities
>could detect a large proportion of changes.
>cannot follow fast illumination changes:
>moving clouds,
>long shadows,
>switching of artificial lighting
 Identifying a particular object surface under
varying illumination
>Marchant and Onyango.(2000) proposed a physics-based method for
shadow compensation in scenes illuminated by daylight.
>Represented the daylight as a black body,
>Assumed the color filters to be of infinitely narrow bandwidth.
Results: as illumination changes, the ratio (R/B)/(G/B)^A depends on
surface reflection only. (A can be calculated from daylight model and
camera.)
>Finlayson et al.(2000) Using same scheme.
Results: Log-Chromaticity Differences (LCDs) ln(R/G) and ln(B/g) are
independent of light intensity and there exists a weighted combination
of LCDs which is independent of light intensity and light color
Adaptive schemes in color-invariant detection
of motion under varying illumination

>Wren et al (1997) used the normalised components, U/Y and V/Y of

YUV color space to remove shadows in a indoor scene
>A single adaptive Gaussian represents the probability density of
pixel belonging to the background.
>The scene without person has to be learned before to locate people.
>Raja et al (1998) used hue(H) and saturation(S) of an HSI color space
to decouple the influence of illumination changes in a indoor scene,
>A Gaussian mixture model was used to estimate the probabilities of
each pixel belonging to a multi-colored foreground object
>Each Gaussian models one color in the foreground object and was
learned in a training stage.
 Motion detection in outdoor environments
illuminated by daylight
>A refection model influenced by ambient objects is used.
>large-scale illumination changed mainly arises from
varying cloud cover.
>The dominant illumination comes from either direct
sunlight or reflection from clouds
>The normalised rgb color space is used to eliminate the
influence of varying illumination
>A Gaussian mixture model is used to model each pixel of
the background , provides multi-background modelling
capabilities for complex out scenes
 Colour Fundamentals
An image taken with a color camera is composed of sensor
responses as:

->illumination, ->reflectance of an object

surface
->Camera sensitivity
-> wavelength,
The image intensity:

>The appearance of objects is a result of interaction between

illumination and reflectance.
>To track the object surface, it is desirable to separate the
variation of the illumination from that of the surface reflection.
 Shadow model(1)
In an out door environment, fast illumination changes occur at the
regions where shadows emerge or disappear.
>large-scale (arising from moving cloud)
>small-scale (from objects themselves)
Shadow model (Gershon et al 1986)
>There is only one illuminant in the scene
>Some of the light does not reach the object because blocking
objects.
>create a shadow region and a directly lit region on the object.
>The shadow region is illuminated by each reflection objects j:
 Shadow model cont.
>The reflected light from the object surface
>For the directly lit region

>For the shadow region:

>Assume the chromatic average of the ambient objects is gray

i.e. it is relatively balanced in all visible wavelengths and

Where c is independent of and may varies over space

 Shadow model cont.
The assumption is realistic for the fast-moving cloud case, in which the
only illuminant is the sunlight and both the blocking and ambient
objects are gray(or white) clouds.
Under the assumption, the reflected light from directly lit and shadow
regions will stay in proportion for a given object surface. Thus the
image intensities at all color channels being in proportion no matter lit
or shadowed
The proportionality between RGB color channels can be represented
using the normalised color components :

Where each component of will keep constant for a given object

surface under varying illumination.
 Color-Invariant Motion Detection
A single Gausssion is sufficient to model a pixel value for one channel of
the RGB components resulting from a particular surface under
particular lighting and account for acquisition noise.
A single adaptive Gaussian is sufficient to model each RGB channel if
lighting changes gradually over time. The estimated background value
is interpolated between the previous estimation and the new
observation. It cannot follow an RGB component under fast
illumination changes. A normalized color component (rgb) for a given
object surface tends to be constant under lighting changes and is
appropriate to model using an adaptive Gaussion.
Multiple adaptive Gaussians (a mixture of Gaussions) are used to model a
pixel at which multiple object surfaces may appear as the
backgrounds. E.g. swaying trees
 Color-Invariant Motion Detection cont.

Let the pixel value at time t be and modeled by a

mixture of N Gaussian distributions. The probability of observing the
pixel value is:

Where G is the Gaussian probability density function of the I-th background

Bi, P(Xi | Bi)
P(Bi) reflecting the likelihood that the distribution accounts for the observed
data.
 Scheme
Every new observation, Xt, is checked against the N Gaussian
distributions, A match is defined as an observation within about 3
standard deviations of a distribution. If none of the N distributions
match the current pixel value, the least probable distribution is
replaced by the new observation.
For the matched distribution, i, the parameters are updated as:

For the unmatched distribution

The distribution(s) with greatest weight is(are)considered as the

background model.
 Experimental Results

To assess the significance of the color-invariant motion detection

>Evaluated the model at both pixel and frame levels using a set of
image sequences.
>The image sequence was captured at a frame rate of 2 hz
>Each frame was compressed in JPEG format
>frame size: 384x288 pixels.
This sequence well represents the abundant contexts of a day lit outdoor
environment:
>Fast illumination changes, waving trees, shading of tree canopies,
>Highlights of specula reflection, as well as pedestrians.
the absolute (RGB) and normalised (rgb) color components at selected pixels
through time.
(a) No foreground object is present.
(b) Foreground object are present.
The absolute color components (RGB) change greatly with the illumination.
The normalized color components (rgb) for a background pixel have flat
profiles under illumination.
For each foreground pixel, at least one rgb component appears as an apparent
spike.
The parameter updating procedure of the Gaussian background model
for one color component
(a) A lit region with foreground objects
(b) A shadowed region without foreground objects
The thin lines represent and (upper and lower) profiles,
respectively.
Comparing the RGB and rgb results under little illumination change
>The results are coherent.
>Because of the different emphasis of image contexts, the “blobs”
appear as different shapes.
The RGB and rgb results under a major illumination change.
(a) A large area of the background is detected as a huge
foreground object.
(c) Ground truth targets are clearly visible under fast
illumination changes
 Discusion(1)
>Appropriate selection of the initial deviation
>An underestimate of the initial deviation
prohibits many “ground truth” background pixels from being
adapted into background models
>An overestimate of the initial deviation
needs a longer learning period at the start of an image sequence
>Currently
it is manually selected and globally uniform according to the
noise level in shaded regions where the absolute noise level in
rgb components is high.
>In future
it may be automatically selected according to the local
spatial variation in the rgb components at the start time
 Discussion(2)
>The rgbl color space
>combined the intensity I, with the rgb color space
>is an invertible transformation from RGB space
>avoids the loss of the intensity information.
>robustly determinates the shadowed region:
>the rgb components are stable
>the I component is significantly lower.
>Two kinds of pixels which may be excluded from consideration:
>RGB components saturated can make the corresponding rgb
components unconstrained
>The rgb components in over-dark regions are very noisy.
>To alleviate this problem:
> Using cameras with auto iris control
> Gamma correction
 Conclutions

>A Gaussion mixture model based on the rgb color

space has been presented for maintaining a
background image for motion detection.
>The scheme is especially successful when applied
to outdoor scenes illuminated by daylight and is
robust to fast illumination changes arising from
moving cloud and self-shadows.
>The success results from a realistic reflection model
in which shadows are present.
Thank you!