See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/37988009

Lane Detection and Tracking Using a New Lane Model and a Distance Transform

Article  in  Machine Vision and Applications · July 2011

DOI: 10.1007/s00138-010-0307-7 · Source: OAI

CITATIONS READS
75 849

4 authors, including:

Ruyi Jiang Reinhard Klette

Shanghai Jiao Tong University Shandong Academy of Sciences
14 PUBLICATIONS   243 CITATIONS    790 PUBLICATIONS   6,761 CITATIONS   

SEE PROFILE SEE PROFILE

Tobi Vaudrey
Auckland University of Technology
51 PUBLICATIONS   963 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Learning from Big Retinal Data View project

One class classification View project

All content following this page was uploaded by Tobi Vaudrey on 30 May 2014.

The user has requested enhancement of the downloaded file.

 New Lane Model and Distance Transform
for Lane Detection and Tracking

Ruyi Jiang1 , Reinhard Klette2 , Tobi Vaudrey2 , and Shigang Wang1

1
Shanghai Jiao Tong University, Shanghai, China
2
The University of Auckland, Auckland, New Zealand

Abstract. Particle filtering of boundary points is a robust way to estimate lanes.

This paper introduces a new lane model in correspondence to this particle filter-
based approach, which is flexible to detect all kinds of lanes. A modified version
of an Euclidean distance transform is applied to an edge map of a road image
from a birds-eye view to provide information for boundary point detection. An
efficient lane tracking method is also discussed. The use of this distance transform
exploits useful information in lane detection situations, and greatly facilitates the
initialization of the particle filter, as well as lane tracking. Finally, the paper vali-
dates the algorithm with experimental evidence for lane detection and tracking.

1 Introduction
Lane detection plays a significant role in driver assistance systems (DAS), as it can help
estimate the geometry of the road ahead, as well as the lateral position of the ego-vehicle
on the road. Lane detection and tracking have been widely studied for driving on a free-
way or an urban road, for single or multiple lanes, with or without marks, based on
region (texture or color) or edge features. Various models have been applied to describe
the borders of a lane, such as piecewise linear segments, clothoids, parabola, hyperbola,
splines, or snakes. For a complete review of lane detection algorithms, please refer to
[4]. There are even some commercial lane detection systems available, working mainly
on highways. Loose et al (Daimler AG) state in [3]: “Despite the availability of lane de-
parture and lane keeping systems for highway assistance, unmarked and winding rural
roads still pose challenges to lane recognition systems.” Generally, it is a challenging
task to robustly detect lanes in varying situations, especially in complex urban environ-
ment.
This paper introduces a new weak (i.e., with no assumption about the global shape
of a lane) road model for lane detection and tracking. Instead of modeling global road
geometry, this new model only constrains relations between points on the left and right
lane boundaries. Tracking based on these points in the birds-eye image (using a particle
filter) provides lane detection results. Furthermore, a modified version of a standard
Euclidean Distance Transform (EDT) is applied on the edge map of the birds-eye image.
Utilizing some beneficial properties of this distance transform for lane detection and
tracking, this paper also specifies an innovative initialization method for the particle
filter.
This paper is organized as follows: Section 2 describes a new lane model. Section 3
explains a modified version of a standard Euclidean distance transform, and its useful-
ness for lane detection. Lane detection and tracking methods are introduced in Section 4
2 R. Jiang, R. Klette, T. Vaudrey, and S. Wang

PZ
aa a

H
b1 b2
y b1 b2

PL PR
Pc
L2 PL Pc(xc,yc) PR
L1
Ground manifold PL Pc(xc,yc) PR

x
(a) (b) (c)

Fig. 1. Lane model as used in this paper. (a) 3D lane view; boundaries are drawn in bold. (b)
Perspective 2D lane view in the input image. (c) Birds-eye image of the lane. Slope angles β1
and β2 are shown in the 3D view and the birds-eye images; the zenith angle α in the 3D and the
projective view. See text for further explanations.

and Section 5. Experimental results are given in Section 6. Finally, conclusions are pro-
vided in Section 7.

2 A New Lane Model

The new lane model is illustrated in Figure 1. Five parameters xc , yc , α, β1 , and β2 are
used to model opposite points PL and PR , located on the left or right lane boundary,
respectively. PC = (xc , yc ) is the (virtual) centerline point of a lane in the ground plane.
α is the zenith angle above PC , defined by an upward straight line segment between PC
and the zenith PZ of fixed length H, and a line incident with PZ and either PL or PR .
β1 and β2 are the slope angles between short line segments L1 and L2 and a vertical
line in the ground plane; the two short line segments L1 and L2 are defined by a fixed
length and local approximations to edges at lane boundaries (e.g., calculated during
point tracking). Ideally, L1 and L2 should coincide with tangents on lane boundaries
at points PL and PR ; in such an ideal case, β1 and β2 would be the angles between
tangential directions of lane boundaries at those points and a vertical line.
By applying this model, a lane is identified by two lane boundaries, and points are
tracked along those boundaries in the birds-eye image. This model does not use any
assumption about lane geometry, and applies to all kinds of lanes.

3 Distance transform
The distance transform is applied to the binary edge map, which labels each pixel with
the distance to the nearest edge pixel. The Euclidean distance transform (EDT) is, in
general, the preferred option; using the Euclidean metric for measuring the distance be-
tween pixels. A modified EDT was proposed in [7], called orientation distance trans-
form (ODT). This divides the Euclidean distance into a contributing component in the
row and column direction. A complex number is assigned to each pixel by the ODT,
with the distance component in the row direction as the real part, and the distance com-
ponent in the column direction as the imaginary part. Note that distance component in
row direction is signed, with a positive value indicating that the nearest edge point lies
 New Lane Model and Distance Transform 3

E1

d
i

r row direction
P2 P1

E2

d: Euclidean distance of P1 to the nearest edge point E1

r: Real part of ODT of P1 to the nearest edge point E1
i: Imaginary part of ODT of P1 to the nearest edge point E1

Fig. 2. Euclidean distance and Orientation distance. P1 and P2 are two neighboring non-edge
pixels. E1 is the nearest edge point to P1 , and E2 is the nearest edge point to P2 .

(a) (b) (c)

(d) (e)
Fig. 3. EDT and ODT on a birds-eye road image. (a) Birds-eye road image. (b) Binary edge map.
(c) Imaginary part of ODT. (d) Real part of ODT (absolute value). (e) EDT. (c)(d)(e) have been
contrast adjusted for better visibility. The square root of distance value in (c) and (d) will be the
Euclidean distance value as (e). The rectangle in (b) indicates a small window for comparisons
of dashed lane mark with EDT and RODT(see below).
to the right, and a negative value if it is to the left. See Figure 2 for the relationship
between ODT and EDT.
This paper uses only the Euclidean distance in the row direction, and we call this
the real orientation distance transform (RODT). An example of EDT and ODT in our
application is shown in Figure 3. The RODT of our edge map offers various benefits.
First, the initialization of lane detection becomes much easier (to be discussed in Sec-
tion 4.1). Second, dashed lane marks will make no difference with continuous ones in
the RODT, as illustrated in the comparison of the rectangle area in Figure 3. Third, more
information about the centerline is provided by the distance transform compared to the
edge map. Generally, a (non-edge) pixel on the centerline of a lane will have a local
maximum in distance to the lane boundaries. Thus, combined with the lane model in-
troduced in Section 2, a point with a high distance value is likely to be a centerline point
PC . The usefulness of these properties of the RODT will be discussed in the following
sections.
4 R. Jiang, R. Klette, T. Vaudrey, and S. Wang

(a) (b) (c)

(d) (e) (f)

Fig. 4. The overall work flow of lane detection. (a) Input image. (b) Birds-eye image. (c) Edge
map. (d) Distance transform. (e) Lane detection results, shown in the birds-eye image (left and
right lane boundaries, with lane center in the middle). (f) Lane detection results, shown in the
input image.

4 Lane Detection Using a Particle Filter

For lane detection in a single image, a particle filter is used to track points along lane
boundaries as in [5]. Furthermore, a novel initialization method is adopted based on
a distance transform applied to the birds-eye edge map. The whole procedure of lane
detection is illustrated in Figure 4 by an example.
The algorithm starts with mapping the perspective input image into a birds-eye view.
An edge detection method, as introduced in [1], is then adopted to detect lane-mark-like
edges. After binarization of the resulting edge map, a RODT is applied. The resulting
distance map allows us to design a novel initialization method for finding the initial
boundary points. These points are used to initialize the parameters of the particle filter,
for tracking further boundary points through the whole image; a lane is finally detected.

4.1 Initialization
The aim of the initialization step is to find an initial value (e.g., the x-coordinate of a
point PL and point PR in a selected image row) for the specified model. In [5], a clus-
tered particle filter is used in order to find a start point on a lane boundary. In distinction
to this, we fully utilize the distance map to find the first left and right boundary points.
In a pre-defined start row (near to the bottom) of the birds-eye image, a search is con-
ducted, starting at the middle of the row, for a pixel which has a positive distance value
but a negative distance value at its left neighbor (see Figure 5). When such a pixel is
found, the left and right boundary points in the start row are instantly known using the
distance value of the found pixel and of its left neighbor.
For the initial state X0 (xc0 , α0 , β10 , β20 ) of the particle filter, xc0 and α0 are ini-
tialized by using the detected left and right start points, while β10 and β20 are simply
set to be zero.

4.2 Particle filter for lane detection

Particle filters are widely used for lane detection and tracking, such as in [5, 6]. This
section discusses particle filtering for our new lane model. The state vector X =
(xc , α, β1 , β2 )T to be tracked is defined by the parameters of the lane model, with-
out yc , as yc will be calculated incrementally by applying a fixed step ∆, starting at
 New Lane Model and Distance Transform 5

row yc0 in the birds-eye image. For the application of a particle filter, two models are
discussed in the following.

The dynamic model. The dynamic model A is used to define the motion of particles in
the image. The prediction value X̂n is generated from Xn−1 by using X̂n = A · Xn−1 .
We simply take A as being the identity matrix, because of the assumed smoothness of
the lane boundary.

The observation model. The observation model determines each particle’s importance
factor for re-sampling. Based on the RODT information combining new model, it is
reasonable to assume points (xcn , ycn ) will have large distance values, and L1 and L2
coincide with short lines of pixels which all only have small distance values.
Tracking step n is identified by ycn = (yc0 + n · ∆). We calculate the lateral
position of the left boundary point of the lane from the predicted state vectors, with
X̂ni (x̂icn , α̂ni , β̂1i n , β̂2i n ) for the ith particle.
From now on, PL and PR only represent the lateral position of boundary points, for
simplicity. The left position is calculated as follows:

PLi = x̂icn − H · tan α̂ni

Next, the sum of the distance values along line segment L1 is as follows:
L1 /2   
X
SLi 1 = d PLi + j · sin β̂1i n , ycn + j · cos β̂2i n 
 
j=−L1 /2

Here, d(·, ·) is the distance value of the RODT. Calculating SLi 2 in the analogous way,
we obtain the ith importance factor
!
i 1 (SLi 1 − µ1 )2 (SLi 2 − µ2 )2
ωdist = exp − −
2πσ1 σ2 2σ1 2σ2

For the centerline point (xicn , ycn ), the importance factor is equal to
   2 
1
− µ
 
1  i
d(xcn ,ycn )  3
i
ωcenter = √ exp −
 
2σ3

σ3 2π

where µk and σk are constants, for k = 1, 2, 3 . The final observation model is given by
the factors
i i
ωi = ωdist · ωcenter

Left boundary Right boundary

edge pixel Search result eldge point

0 -1 -d d 1 0

Fig. 5. Illustration of the search procedure in the start row of the distance map. Note that the
distance values are signed, as described in Section 3.
6 R. Jiang, R. Klette, T. Vaudrey, and S. Wang

t=t+1 Adjustment of
Lane detection Detection of
in frame t {PLi, PRi, i=0,...,N-k} {PLi, PRi, i=N-k+1,...,N}

Birds-eye view Offset of

Frame t+1 mapping and {PLi, PRi, i=0,...,N-k} by
distance transform value of distance transform

Fig. 6. Efficient lane tracking scheme.

5 Efficient lane tracking

Lane tracking uses information defined by previous results to facilitate the current de-
tection. This section introduces an efficient lane tracking method. Note that when a
lane is detected (as in Section 4), it is reasonably represented just by two sequences
{PLn : n = 0, 1, . . . , N } and {PRn : n = 0, 1, . . . , N } of points on its left and right
lane boundaries in the birds-eye image. Here, N is determined by the forward-looking
distance. Tracking of a lane through an image sequence is then simplified as being
tracking of these two point sequences.
Efficient lane tracking simply uses previously detected lane boundary points, adjusts
them according to the ego-vehicle’s motion model, and then offsets them according to
values of the RODT on the current birds-eye edge map. The lane tracking scheme is
summarized in Figure 6.
(t) (t)
Sequences {PLn } and {PRn }, detected in frame t, are already partially driven
(t+1) (t+1)
through by the ego-vehicle at time t+1. The detection process of {PLn } and {PRn }
at time t + 1 is composed of three steps: adjustment caused by the driven distance and
the variation in yaw angle, new point detection, and offset specification according to the
values of the RODT in the birds-eye edge map.
Because of the driven distance between frames t and t + 1, it holds (in principle)
that
(t+1) (t) (t+1) (t)
PL n = PL(n+k) , PRn = PR(n+k) , n = 0, 1, . . . , N − k

Here, k is determined by the driven distance between time t and t + 1, and is usually a
small number. Furthermore, points
(t+1) (t+1)
{PLn : n = 0, 1, . . . , N − k} and {PRn : n = 0, 1, . . . , N − k}
are obtained by adding some translation (according to n) caused by the variation in
driving direction between t and t + 1.
(t+1) (t+1)
For the detection of {PLn : n = N − k + 1, . . . , N } and {PRn : n = N −
k + 1, . . . , N }, note that k is small and we also assume smoothness of lane boundaries.
Thus, we simply start as follows:
(t+1) (t+1) (t+1) (t+1)
PL n = PLn−1 , PRn = PRn−1 , n = N − k + 1, . . . , N
(t+1) (t+1)
For further refinement, those predictions {PLn } and {PRn } from the previous
result at frame t are likely to be already located near the true points on the bound-
aries, as the variation of a lane is usually minor between two subsequent frames. – The
 New Lane Model and Distance Transform 7

(a) (b) (c)

Fig. 7. Experimental results for lane detection. (a) Input images. (b) Lanes detected in the birds-
eye image. Note that the centerline of a lane is also marked. (c) Lanes detected in input images.

Fig. 8. Lane detection on roads with unmarked or blocked lane marks. (a) Input images. (b) Edge
map. (c) RODT. (d) Lane detected. Note that RODT contains more information than edge map
for lane detection as discussed in Sec. 3.

adjustment
(t+1) (t+1) (t+1)
PLn = PLn + d(PLn , ycn ), n = 0, 1, . . . , N
(t+1) (t+1) (t+1)
PRn = PRn + d(PRn , ycn ), n = 0, 1, . . . , N

of all N +1 points is finally achieved by information available from values of the RODT
of the current birds-eye edge map.

6 Experiments
Experiments were conducted on images and sequences recorded with the test vehicle
“HAKA1” of the .enpeda.. project [8].
Experimental results for lane detection are shown in Figure 7 and Figure 8. Differ-
ent scenarios are considered. Note that detected lane boundaries are sometimes locally
slightly curved. This is due to the fact that the distance transform of dashed lane marks
is slightly unaligned in column direction in the birds-eye image (see Figure 3).
Experimental results while using the efficient lane tracking method are illustrated in
Figure 9. The results are really acceptable except for some outliers for lane boundaries.
With an off-the-shelf computer, the computation time for the adjustment steps of effi-
cient lane tracking is negligible. The only measurable time needed is that for birds-eye
 8 R. Jiang, R. Klette, T. Vaudrey, and S. Wang

SEQ1

frame 0 frame 10 frame 20 frame 50

SEQ2

frame 0 frame 10 frame 20 frame 50

Fig. 9. Experimental results on two sequences using efficient lane tracking.

view mapping, edge detection and distance transform, and all three sub-processes can
be computed highly efficiently.

7 Conclusions
This paper introduced a new weak model of a lane, and a possible lane detection scheme
using a particle filter based on a monocular camera. Furthermore, an efficient lane track-
ing method was proposed and discussed.
A (simple and easy to calculate) distance transform was used in this paper for lane
detection and tracking. It shows that the distance transform is a powerful method to
exploit information in lane detection situations. The distance transform can deal with
dashed lane marks, provides information for the detection of the centerline of a lane,
finds initial values for the particle filter, and adjusts the tracking results conveniently.

Acknowledgments: This work is supported by the National Natural Science Founda-

tion of China under Grant 50875169.

References
1. M. Bertozzi, A. Broggi: GOLD - A parallel real-time stereo vision system for generic obsta-
cle and lane detection. IEEE Trans. Image Processing, 7:62–81, 1998.
2. Z. Kim: Robust lane detection and tracking in challenging scenarios. IEEE Trans. Intelligent
Transportation System, 9:16-26, 2008
3. H. Loose, U. Franke, and C. Stiller: Kalman particle filter for lane recognition on rural roads.
In Proc. IEEE Intelligent Vehicle Symp., to appear, 2009.
4. J. C. McCall and M. M. Trivedi: Video-based lane estimation and tracking for driver assis-
tance: survey, system, and evaluation. IEEE Trans. Intelligent Transportation System, 7:20–
37, 2006.
5. S. Sehestedt, S. Kodagoda, A. Alempijevic, and G. Dissanayake: Efficient lane detection and
tracking in urban environments. In Proc. European Conf. Mobile Robots, pages 126–131,
2007.
6. Y. Wang, L. Bai, M. Fairhurst: Robust road modeling and tracking using condensation. IEEE
Trans. Intelligent Transportation Systems, 9:570–579, 2008.
7. T. Wu, X. Q. Ding, S. J. Wang, and K. Q. Wang: Video object tracking using improved
chamfer matching and condensation particle filter. In Proc. SPIE-IS & T Electronic Imaging.
volume 6813, pages 04.1-04.10, 2008
8. http://www.mi.auckland.ac.nz

View publication stats