Grouping tracked features using

common motion cue and graphs.

A Real-time Computer Vision System for Measuring Traffic

Parameters *
David Beymer, Philip McLauchlan, Benn Coifman, and Jitendra Malik
Dept. of Electrical Engineering and Computer Sciences
University of California
Berkeley, California, 94720-1776
E-mail : {beymer pm zephyr malik}@cs .berkeley .edu

Abstract Processing the track data to compute local traffic pa-

For the problem of tracking vehicles on freeways using ma- rameters including vehicle counts per lane, average
speeds, lane change frequencies, etc. These param-
chine vision, existing systems work well in free-flowing
trafic. Trafic engineers, however, are more interested in eters, together with track information (time stamp,
monitoring freeways when there is congestion, and cur- vehicle type, color, shape, position), are communi-
cated to the TMC at regular intervals.
rent systems break down for congested trafic due to the
problem of partial occlusion. We are developing a feature- At the TMC, local traffic parameters from each site
based tracking approach for the task of tracking vehicles are collated and displayed as desired, and/or used
under congestion. Instead of tracking entire vehicles, ve- in controlling signals, message displays, and other
hicle sub-features are tracked to make the system robust to traffic control devices. Computers at the TMC also
partial occlusion. In order to group together sub-features process the track information from neighboring cam-
that come from the same vehicle, the constraint of com- era sites to compute long-distance parameters such
mon motion is used. In this paper we describe the system, as link times and origin-destination counts.
0 real-time implementation using a network of DSP chips,
and experiments of the system on approximately 44 lane In this paper, we focus on the first two stages, the vehicle
hours of video data. segmentation and tracking stage and the computation of
traffic parameters from the tracking data.
1 Introduction 2 Tracking Approach
Traffic management and information systems must rely Tracking moving objects in video streams has been a pop-
on a system of sensors for estimating traffic parameters in ular topic in the field of computer vision in the last few
real-time. Currently, the dominant technology for this years; earlier contributions to the areas of multi-target
purpose is that of magnetic loop detectors, which are tracking and data association were made by control and
buried underneath highways to count vehicles passing over aerospace engineers. Our application entails several strin-
them. Video monitoring systems promise a number of gent requirements for a proposed scheme:
advantages. First, a much larger set of traffic parame-
ters can be estimated in addition to vehicle counts and 1. Automatic segmentation of a vehicle from the back-
speeds. These include vehicle classifications, link travel ground and other vehicles so that there can be a
times, lane changes, rapid accelerations or decelerations, unique track associated with each vehicle.
queue lengths at urban intersections, etc. Second, cam-
eras are less disruptive and less costly to install than loop 2. Deal with variety of vehicles - motorcycles, passenger
detectors, which require digging up the road surface. cars, buses, construction equipment, trucks, etc.
For some years, our group has been developing a proto- 3. Deal with range of traffic conditions - light midday
type vision-based traffic surveillance system [ll,121. The traffic, rush-hour congestion, varying speeds in dif-
core idea is to have video cameras mounted on poles or ferent lanes.
other tall structures looking down at the traffic scene.
Video is captured] digitized, and processed by onsite com- 4. Deal with variety of lighting conditions - day,
puters] and then transmitted in summary form to a Trans- evening, night, sunny, overcast, rainy days.
portation Management Center (TMC) for collation and 5. Real-time operation of the system.
computation of multi-site statistics such as link travel
times. Processing occurs in three stages: Even though a number of commercial systems for traffic
monitoring have been introduced recently, many of these
1. Segmentation of the scene into individual vehicles
criteria still cannot be met. In a recent evaluation of
and tracking each individual vehicle to refine and
a group of these commercial systems [4],problems were
update its position and velocity in 3D world coor- reported with congestion, long shadows linking together
dinates, until it leaves the tracking zone.
vehicles, and the transition between night and day.
'This research was supported by FHWA through a contract moni- In the computer vision literature, the different tracking
tored by J P L , and by Caltrans/PATH (MOU 152 and MOU 214). approaches for video data can be classified as follows.

1063-6919/97 $10.00 0 1997 IEEE 495

 ~

2.1 3D Model based tracking

c
I - x
Three-dimensional model-based vehicle tracking systems
have previously been investigated by several research
Y
groups, the most prominent being the groups at Karl-
sruhe [lo] and at the University of Reading[l, 151. The
emphasis is on recovering trajectories and models with
high accuracy for a small number of vehicles. The most
serious weakness of this approach is the reliance on de-
tailed geometric object models. It is unrealistic to expect
to be able to have detailed models for all vehicles that
could be found on the roadway.
2.2 Region based tracking
The idea here is to identify a connected region in the image Figure 1: A projective transform H , or homography, is used
- a “blob” - associated with each vehicle and then track
to map from image coordinates (z,y)to world coordinates
it over time using a cross-correlation measure. Initializa- (X,
Y).
tion of the process is most easily done by the background
subtraction technique. A Kalman filter-based adaptive psychologists as common fate. Point features that are seen
background model[8, 91 allows the background estimate
as moving rigidly together will be grouped together into a
to evolve as the weather and time of day affect lighting
single vehicle. But since there are many vehicles in traf-
conditions. Foreground objects (vehicles) are detected by
fic scenes, there is also an important segmentation aspect
subtracting the incoming image from the current back-
to the problem. One does not want to link together sub-
ground estimate, looking for pixels where this difference
features from neighboring vehicles. The grouping process
image is above some threshold and then finding connected
must be sensitive enough to pick up a motion that distin-
components.
guishes a vehicle from its neighbors, a motion such as a
This approach works fairly well in free-flowing traffic. slight acceleration or lane drift.
However, under congested traffic conditions, vehicles par-
To make the grouping system robust enough to seg-
tially occlude one another instead of being spatially iso- ment different vehicles, the spatial information guiding
lated, which makes the task of segmenting individual vehi-
the grouper will be integrated over a period of time, uti-
cles difficult. Such vehicles will become grouped together
lizing as many image frames as possible. Only the sub-
as one large blob in the foreground image.
features that are tracked from a detection region at the
2.3 Active contour based tracking bottom of the image to an exit region near the top will
A dual to the region based approach is tracking based on be allowed to participate in the final grouping. Thus, in
active contour models, or snakes. The idea is to have a order to fool the grouper, two vehicles would have to have
representation of the bounding contour of the object and identical motions during the entire time they were be-
keep dynamically updating it. The previous system for ve- ing tracked. In congested traffic, vehicles are constantly
hicle tracking developed in our group [ll,121 was based on changing their velocity to adjust to nearby traffic, thus
this approach. The advantage of having a contour based giving the grouper the information it needs to perform
representation instead of a region based representation is the segmentation. In free-flowing traffic, vehicles may be
reduced computational complexity. more likely to maintain constant spatial headways over
However, the inability to segment vehicles that are par- time, thus making the grouping constraint less useful.
tially occluded remains. If one could initialize a separate But in this scenario, there is more space between vehi-
contour for each vehicle, then one could track even in the cles, so a spatial proximity cue is added to aid the group-
presence of partial occlusion[ll]. However, initialization ing/segmentation process.
is the difficult part of the problem! Since most road surfaces are flat, the grouper exploits
an assumption that vehicle motion will be parallel to the
2.4 Feature based tracking road plane. To describe the road plane, the user simply
Finally, yet another approach to tracking abandons the specifies four or more line or point correspondences be-
idea of tracking objects as a whole but instead tracks tween the image road and a separate “world” road plane,
sub-feat,ures such as distinguishable points or lines on the as shown in Fig. 1. Based on this off-line step, the system
object. The advantage of this approach is that even in can compute a projective transform, or homography, be-
the presence of partial occlusion, some of the sub-features tween the image coordinates ( 2 ,y) and world coordinates
of the moving object remain visible. The technology of (XIY).By writing points in homogeneous coordinates,
tracking points and line features in a Kalman filtering this is a simple linear transform
formalism is well developed in the computer vision com-
munity. Since a vehicle could have multiple sub-features,
the new problem then is that of grouping - what set of
features belong to the same object.
1 -.[rl.
The scaling of H is arbitrary, so H ( 3 , 3 ) is often chosen
3 Motion-Based Grouping
to be 1.
The grouping of vehicle sub-features will be based on a The grouper considers sub-feature points in pairs. That
common motion constraint, a concept known to Gestalt is, the basic grouper computation is whether or not to

496
 ~

vehicles
-.....____-
traffic
J
vehicle flow, speed, etc Figure 3: Example corner features located by the system.
classes b to TMC

Figure 2: Block diagram of our vehicle tracking and grouping spatial headway are computed from the vehicle tracks. In
system. the future, we intend to add a vehicle classification mod-
ule that will identify vehicles as automobiles, motorcycles,
trucks, buses, etc. In this section, we describe the detec-
group together the 2D point features p a ( t )and p b ( t ) . The tion, tracking, and grouping modules.
dependence on time t is written to emphasize that the
4.2.1 Feature Detection and Tracking
grouper is working with sub-feature tracks, and hence has
access to the time history of points. The 3D coordinates Vehicle sub-features are detected and tracked in order
of these points in the real world will be written in u m e r
case ~,(t) and P b ( t ) .
__ to be insensitive to partial occlusion. Even if part of the
vehicle is obscured due to congested traffic, some of the
Consider the simple case where P, and Pb are at the vehicle’s sub-features should still remain visible.
same distance to the camera (e.g. both on the back face of Corner features are the chosen sub-features since they
a truck). In this scenario, the grouper only needs to look can be reliably tracked. Our corner detector computes the
at a simple function of the displacement vector p a ( t ) - windowed second moment matrix by averaging in a spatial
P b ( t ) . Since P, and Pb are both at the same distance window the 2x2 matrix, V I V P , where V I is the image
from the camera d , p,(t) and P b ( t ) are both scaled by gradient [6]. Corners are declared where the numerical
the same scale factor l / d . Thus, for points on the same rank of this matrix is 2 (smaller eigenvalue above thresh-
vehicle, p a @ )- P b ( t ) will be constant over time if we can old). Fig. 3 shows some example corner features detected
simply compensate for the l / d scaling. Fortunately, the by the system. When a corner sub-feature is detected, a
homography can be used for this compensation. Given a small 9x9 template of the grey level image is extracted
point (2,y) in the image, we can estimate the scale factor and used for correlation in the tracking module. Also,
s that transforms the region around that point to world while there are some undesirable corners present near the
coordinates. The difference vector p a ( t )- pb(t) can then vehicle boundaries and background, these corners will be
simply be scaled by s. pruned away by the feature tests employed by the tracker.
We have also considered the more general case where The tracking module tracks corner sub-features from
P , and Pb are not at the same distance from the cam- the detection region at the bottom of the image to the
era. Space considerations in these proceedings prevent a exit region near the top. To address the problem of noisy
discussion of this case; please see [3] for the details. measurements, we employ Kalman filtering[7] to provide
most likely estimates of the state of a vehicle sub-feature
4 Algorithm based on accumulated observations. In our system, the
4.1 Off-line camera definition state vector contains sub-feature positions and velocities
Before running the tracking and grouping system, the user ( X ,U,X , Y ) in the world coordinate system; vehicle accel-
specifies some camera-specific parameters off-line. These eration is captured in the system dynamics noise process.
parameters include: The measurement process in the Kalman filter is based
on normalized correlation. At each time frame, the
1. line correspondences for the homography (Fig. l),
Kalman filter predicts where to search for each corner
2. a detection region near the image bottom and an exit feature. This prediction is mapped back to the image
region near the image top, and plane, and then the template extracted when the corner
3 . a fiducial point for camera stabilization. was originally detected is correlated in a window around
the prediction. The template is scaled down over time to
4.2 On-line tracking and grouping reflect the fact that vehicles are getting smaller as they
A block diagram for our vehicle tracking and grouping move down the road surface. We can use the position in
system is shown in Fig. 2. First, the raw camera video world coordinates to predict the proper scale of the tem-
is stabilized by tracking a manually chosen fiducial point plate. Once we have located the correlation peak, this
to subpixel accuracy. Next, the stabilized video is sent measurement is mapped back onto the road plane. Fi-
to a detection module, which locates corner features in nally, the standard Kalman filter equations for updating
a detection zone near the bottom of the image. These the state and error variance are employed.
corner features are then tracked over time in the tracking T w o tests are used to eliminate bad sub-feature tracks:
module. Next, sub-feature tracks are grouped into vehi- 1. Kalman filter innovations. The distance between the
cle hypotheses in the grouping module. Finally, traffic Kalman filter prediction and the current measure-
parameters such as flow rate, average speed, and average ment is computed and the track is rejected if the

497
 Figure 4: Example tracks of corner features. Figure 5: Example groups of corner features.

distance is above a threshold. motion such as lane drift or an acceleration. When the
2. Imprecise measurement test. If the correlation values last track of a connected component enters the exit region,
form a broad, undefined peak around the correlation a new vehicle hypothesis is generated and the component
maximum, then the measurement process is probably is removed from the grouping graph.
not localizing the sub-feature within the needed pre- Fig. 5 shows the final groups computed for the vehicles
in the tracking region (which is the middle part of the
cision. To measure the peak’s curvature, we compute
image). Corner features are indicated by circles, and there
the number of pixels in the correlation peak that are
is an edge drawn between grouped corners.
within a certain fraction of the peak. The track is
rejected if the count is over a threshold.
How are the grouping thresholds in equation (1) deter-
mined? Consider how the median vehicle size changes as
Fig. 4 shows the time evolution of some example tracks, a function of the grouping threshold (Fig. 6). Here, we
plotted as position over time. The image shown is the assume that the same threshold is used for z and y, and
frame when the corners were originally detected. vehicle size is measured as the maximum distance between
4.2.2 Grouping any two points in the group. Empirically, one notices that
the plot of median vehicle size versus threshold exhibits
The purpose of the grouping module is to group to- two linear regimes:
gether sub-features that come from the same vehicle. The
central cue used by the grouper - common motion - was 1. Oversegmentation. Below optimum threshold.
described already in section 3. In this section, we dis- Vehicle size increases rapidly as one raises the thresh-
cuss the details of how the common motion constraint is old, as correct groups are still being constructed out
applied to the sub-feature tracks. of vehicle fragments.
The grouper organizes its task by constructing a graph 2. Overgrouping. Above optimum threshold. This
over time. The vertices are sub-feature tracks, edges part of the graph has a lower slope, as it is harder to
are grouping relationships between tracks, and connected group together different vehicles than it is to group
components correspond to vehicle hypotheses. When a a single vehicle’s sub-features.
new sub-feature is detected and is added to the g r o u p Given this relationship, our goal is to detect the break-
ing graph, it is initially connected to all neighboring point between the two regimes. In an off-line step, we
tracks within a certain radius in the image plane. The sample the graph by running the grouper at different
attitude of the grouper is that nearby tracks are com- thresholds and computing the median vehicle size. Next,
patible until they prove otherwise through relative mo- two line segments are fit to the graph by minimizing the
tion. For all pairs of tracks p,(t) and p b ( t ) joined by an sum of squared error, which locates the breakpoint. We
edge, the grouper keeps track of the relative displacement performed this procedure for all 7 video sequences in sec-
d(t) = pa(t)- p b ( t ) as scaled by the depth-compensating tion 6.2. The thresholds computed led to vehicle recog-
factor computed from the homography. Upon each time nition rates that were very close to the optimum thresh-
frame, another d value is computed for each edge, and the olds (optimum thresholds were computed via exhaustive
edge is broken if either search). In the worst case, the computed thresholds led
maxd,(t) - mind,(t) > z threshold, or (1) to a decline of only 3.6% in the recognition rate.
t t
5 Real-time System
mFxdy(t) - mindy(t)
t
> y threshold.
We have implemented the tracker on a network of 13 Texas
This breaks the link between two tracks if there is enough Instruments C40 DSPs, connected together as shown in
relative motion between the two. Fig. 7. The computationally heavy operations in the
In the normal evolution of the graph, vehicles are over- tracking algorithm - convolution in the feature detector
grouped near the detection region since the graph is lib- and correlation in feature tracker - are placed on the C40
erally connected at first. But as vehicles move down the network, while the grouper is run on the host PC. Run-
road, they are segmented as they perform a distinguishing ning the grouper on the P C is necessitated by memory

498
 aequence Uescription Length N G
Highway 55 heavy congestion 2:46 238 203
Florin Rd 1 free flow & congestion 2:46 244 260
Mac Rd 1 free flow & congestion 1:OO 69 68
Florin Rd 2 night 120 58 64
San Jose urban intersection 1:45 34 34
Mac Rd 2 free flow & congestion 3:36 249 265
Florin Rd 3 free flow & congestion 3:36 317 342
Figure 6 : The plot of vehicle size versus grouping threshold
exhibits two regimes: an oversegmentation regime and over- Table 1: Video sequences for laboratory testing. Length is
grouping regime. in min:sec, N is the number of actual vehicles (counted by a
human) and G is the number of reported vehicle groups.

m- urk
%d-=
rate is naturally reduced.
6 Results
Our tracking and grouping system has gone through two
major phases of testing. First, we tested a software-only,
off-line version of the system in terms of its ability to de-
tect vehicles. This testing gave us a “microscopic” view
of the system, allowing us to analyze errors such as false
detections, false negatives, and overgroupings. Second,
the real-time system was tested on a substantial amount
Figure 7: The C40 network used for feature tracking, consist- of data - 44 lane hours worth - to see if the system could
ing of an NCSC frame-grabber C44 module ((344s are cut-down
C40s with only four communication links); a quad C44 module accurately measure the aggregate parameters of flow, ve-
(four processors) for corner feature detection; a large memory locity, vehicle density, and average spacing.
module (8M RAM) for maintaining the state of current feature 6.1 Off-line testing of vehicle detection
tracks (track controller); six fast SRAM modules for feature
tracking; and a VGA graphics module for display. In order to analyze the behavior of the system at the
vehicle level, we tested the system’s vehicle detection rate
for a set of videotapes covering a range of scene condi-
requirements. The grouper needs to store track trajec- tions: congestion, free-flow, night, and an urban intersec-
tories, which would quickly exhaust the limited memory tion (see Table 1). Since we wanted to measure errors
available on the C40 modules. But keeping the grouper such as vehicle oversegmentation and overgrouping, vehi-
on the PC is also beneficial from a load balancing perspec- cle ground truth was manually defined for each sequence.
tive, as the PC is a 150MHz Pentium and thus equivalent For a particular vehicle, ground truth is a binary mask
to 3 to 4 C40s. outlining the vehicle in one or two frames. The number of
The processors are arranged in two loops, each of which ground truths is denoted as N in Table 1, and the number
is operated as a pipeline feeding back to its source. These of reported groups is G.
two pipelines are controlled by the frame grabber and the Table 2 shows the performance of our system using au-
track controller; they compute the corner features and tomatically computed grouping thresholds, as well as the
the track updates respectively. Four C44 processors are distribution of errors. A separate automatic evaluation
assigned to corner detection, each processing one quarter program compares the vehicle ground truths against the
of the user-defined detection region. The corners are fed groups reported by the tracker/grouper and tallies the
back to the frame-grabber, which passes them along with following events:
the original image to the track controller. A simple effi- 1. True match. A one-to-one matching between a
ciency gain is achieved by sending the image first, since ground truth and a group.
the track controller can then update the existing tracks
while the corners are computed. 2. False negative. An unmatched ground truth.
The job of the track controller is to maintain the state 3. Oversegmentation. A ground truth that matches
of the complete list of current tracks. It does this by more than one group.
receiving updates for existing tracks from its pipeline of 4. False positive. An unmatched group.
six C40s, and creating new tracks a t positions indicated
by the corner detector. The tracker C40s each update 5. Overgrouping. A group that matches more than one
one sixth of the tracks. Since track updates are fairly ground truth.
homogeneous tasks, this achieves good load balancing. In analyzing the results, it should be said that the High-
The performance of the tracker is 7.5Hz in uncongested way 55 sequence is a difficult one because of a poor cam-
traffic, dropping to 2Hz in congested traffic, where many era position and a number of large trucks that sometimes
more tracks are in progress at any given time. This reduc- completely occlude automobiles. In terms of trading off
tion in speed does not of itself lead to a reduction in per- the different error conditions, we have noticed that over-
formance of the tracker, since vehicle speeds in congested segmentation and overgrouping can be traded off by ad-
traffic are reduced, and so the requirement for tracking justing the grouping thresholds.

499
1 - true false over- false over- I
Sequence match neg. seg pos. group 2500 . . . . . . . . . . . . . . . . . . .
.. ._
, .
. ;",
....
. ,
.'
Highway 55 73.9% 18.5% 6.7% 4.9% 0.4%
. ;.... ,x- ,
Florin Rd 1 88.5% 1.6% 6.9% 1.9% 1.5% . i.; .,?
Mac Rd 1 94.2% 1.5% 1.5% 2.8% 1.4% ...... ... ..;. . . . . . . . . 1 . . . . . . .a.' . . . . . . . . . .
* ._
1..

Florin Rd 2 89.6% 6.9% 3.4% 20.0% 0.0% ,< .,::::. .

t . :

San Jose 85.3% 2.9% 5.9% 0.0% 2.9%

Mac Rd 2 80.3% 6.0% 10.4% 2.3% 1.5%
Florin Rd 3 84.5% 2.2% 10.1% 0.3% 1.8%

Table 2: Performance of the tracking/grouping system on the

off-line test sequences. When computing rates, the first three
columns divide the number of true matches, false negatives,
etc., by N; the final two columns divide by G.

As the first three sequences have long shadows, the ex- . .

perimental results show that the system can handle shad- 70- . . .:......... :. . . . . .. . .
ows - shadow sub-features tend to be unstable over time,
especially in congestion. eo-. . . . . . . ....... ....

6.2 On-line testing of traffic parameters 50 - . . . . . . . . . . . . . . . . . . . . . . . .

2.

-
P
Our second phase of testing evaluated the on-line system's ,.:>a.
5
>.U- , .. .: ,. . . . . . . . . . .
ability to estimate aggregate traffic parameters. The pa- :,z-
rameters typically used by traffic engineers to monitor the f
$0- . . . . . p'. , .. , . .
freeways include: '

,
.
4
,

20. . . . . . . .... ........ ..........

1. Flow. Number of vehicles per hour.
2. Velocity. Average vehicle velocity. . . . . . . . . . . . . . . . . . . . .
, .
3. Density. Number of vehicles per unit distance. 0'
4. Headway. Average spacing between vehicles.
These parameters are computed separately for each lane
of traffic and are averaged over a period of time (taken Figure 8: Scatter plots comparing flow q and velocity U to
to be 5 minutes in our experiments). Also, it should be ground truth for the 44 lane hours of data used to test the
apparent that these are not independent variables; we use real-time system.
the methodology from Edie[5] to compute these parame-
ters from the vehicle track data.
Ground truth is provided from inductive loop data that still move at the prevailing speed. The errors in flow,
was collected concurrently with the video data. Each lane density and spacing are due to missed or over counted
of traffic has two loops separated by 20 feet, giving us an vehicles. Often, an error of two or three vehicles in one
effective speed trap for measuring velocity. sample can be very significant. For example, one missed
Our system was tested on approximately 44 lane hours vehicle in a five minute sample at 1,000 veh/hr results in
of video from the Florin Road interchange along Highway a 2% error. At the mean flow for the data, 910 veh/hr,
99 in Sacramento, Calif. (see Fig. 9 for an example shot). the error per missed vehicle is slightly higher, at 2.2%.
The data includes all observed operating conditions: day, Another way to examine estimated traffic parameters
night, twilight, long shadows and rain; congestion and is as a time series. To demonstrate the performance of
free flow. Lane 1, on the left, is carpool (HOV) lane and our system during a dramatic change in lighting condi-
exhibited little if any congestion. Lane 3, on the right, ex- tions from night to day, in Fig. 10 we show flow q and
hibited some degree of congestion for approximately 20% velocity v for a two hour stretch of continuous video. The
of the time. Finally, the loops in lane 2 were bad so it video starts at night (5:30AM, see Fig. 9, left), progresses
was excluded from the final analysis. The video data was
divided into 5 minute aggregation periods, yielding 514
samples for the traffic parameters. Overall, there were %error % vel %flow % dens %headway
roughly 40,000 vehicles in the final video data set. less than samples samples samples samples
The vehicle track data from the real-time system can 2.5% 86% 1870 19Yo 19%
then be compared with the loop data over the 20 foot re- 5% 95% 31% 33% 34%
gion of overlap between the tracks and loop data. Fig. 8 10% 100% 60% 59% 60%
15% 100% 79% 79% 81%
shows scatter plots of the flow and velocity estimates pro- 100% 91% 90% 89%
20%
vided by the loop and vision data, and Table 3 summarizes 25% 100% 96% 96% 94%
the error distribution for velocity, flow, density, and head-
way, As one would expect from a feature based tracker,
the measured velocity is very accurate. Even if the tracker Table 3: Error distribution for velocity, flow, density, and
overgroups or oversegments vehicles, the erroneous blobs headway.

500
Figure 9: Two images from the start and end of a two hour
run of the real-time system.

20 -
____
tradcer,lane3
ground truth
5.5
0 6 6.5 7 7.5 Figure 11: Vehicle tracks for the real-time system during a
lime (hr)
-
flo 17r.Ul0195. Sminufellov
shockwave. When the slope is zero, the vehicle is completely
2500 slopped.
2000

zlsOO
rrlooo References
K.D. Baker and G.D. Sullivan. Performance assessment of model-
500
based tracking. In Proc. of the I E E E Workshop o n Applications
5.5
0 6 6.5 7 7.5 of Computer Vision, pp. 28-35, Palm Springs, CA, 1992.
lime (hr)
Yaakov Bar-Shalom and Thomas E. Fortmann. Tracking and Data
Association. Academic Press, New York, 1988.
Figure 10: Flow and velocity as a function of time. The David Beymer and Jitendra Malik. Tracking vehicles in congested
traffic. In S P I E Vol. 2908, Transportation Sensors and Controls:
sequence begins at 5:30 AM (night lighting conditions) and Colliszon Avoidance, T r a f i c Management, and I T S , pages 8-18,
finishes at daytime. Boston, MA, November 1996.
A. Chatziioanou, S. Hockaday, L . Ponce, S. Kaighn and C. Staley.
Vzdeo Image Processing S y s t e m s Applications an Transportation,
through sunrise and long shadows, and ends with day Phase II. Final Report, Calif. Poly. State Univ., San Luis Obispo,
Calif., Dec. 30, 1994.
(7:30 AM, see Fig. 9, right). In the plot of flow and veloc-
L.C. Edie. Discussion of traffic stream measurements and defini-
ity, there are 48 samples of 5 minute periods and roughly tions. In Proc. Second Internatzonal Symposium on the Theory
4,600 vehicles. Note that the morning rush hour peak of T r a f i c Flow, O E C D , pages 139-154, Paris, France, 1963.
starts during the sequence and approximately 30 minutes W. Forstner and E. Gulch. A fast operatorfor detection and precise
of data from lane three are under light congestion, and location of distinct points, corners, and centers of circular features.
In Proc. of the Intercommisszon Conf. on Fast Processrng of Pho-
thus, frequent occlusions. togrammetric Data, pages 281-305, 1987.
In addition, since the primary design goal in develop- Arthur Gelb, editor. Applzed Optimal Estimation. T h e MIT Press,
ing our system was to deal with congestion, we close the Cambridge, MA., 1974.
results section with an example of a “shockwave”. Fig. 11 Klaus-Peter Karmann and Achim von Brandt. Moving object recog-
plots vehicle tracks as the distance along the lane as a nition using an adaptive background memory. In V Cappellini, ed-
itor, Time- Varying Image Processing and Moving Object Recog-
function of time. In this case, ground truth was entered nition, 2 . Elsevier, Amsterdam, T h e Netherlands, 1990.
manually at a number of points along each vehicle’s tra-
Michael Kilger. A shadow handler in a video-based real-time traffic
jectory. In the regions of the graph where the slope goes monitoring system. In I E E E W o r k s h o p on Applications of C o m -
to zero, one notices that vehicles continue to be tracked p u t e r Vzsion, pages 1060-1066, Palm Springs, CA, 1992.
even when traffic has come to a complete stop. D. Koller, K. Daniilidis, and H.-H. Nagel. Model-based Object
Tracking in Monocular Image Sequences of Road Traffic Scenes.
7 Summary International Journal of C o m p u t e r Vision, 10: 257-281,1993.
D. Koller, J . Weber, and J . Malik. Robust multiple car tracking
We have presented a vehicle detection and tracking sys- with occlusion reasoning. In E C C V , pp. 189-196, Stockholm, Swe-
tem that is designed to operate in congested traffic. In- den, May 2-6, 1994.
stead of tracking entire vehicles, vehicle sub-features are D. Koller, J . Weber, T. Huang, J . Malik, G. Ogasawara, B. Rao,
tracked, which makes the system less sensitive to the prob- and S. Russell. Towards robust automatic traffic scene analysis in
real-time. In I C P R , Israel, November 1994.
lem of partial occlusion. In order to group sub-features
H.A. Mallot, H.H. Bulthoff, J.J. Little, and S. Bohrer. Inverse per-
that come from the same vehicle, the constraint of com- spective mapping simplifies optical flow computation and obstacle
mon motion over trajectory lifetimes is used. A real-time detection. Biological Cybernetics, 64(3):177-185, 1991.
version of the system has been implemented using a net- P. Michalopoulos. Vehicle detection video through image process-
work of C40 DSP chips connected to a host PC. The sys- ing: the Autoscope system. I E E E Trans. o n Vehicular Technology,
40:21-29, 1991.
tem has been tested on approximately 44 lane hours of
data and has demonstrated good performance not only in G.D.Sullivan. Visual interpretation of known objects in constrained
scenes. In Phzl. Trans. R o y . SOC( B ) , 337: 361-370, 1992.
congested traffic, but also on free-flowing, nighttime, and
urban intersection traffic.

501