Determining the Focus of Expansion by Means of Flow eld Projections

Christof Born
Institut fur Neuroinformatik Ruhr{Universitat Bochum W-44780 Bochum, FRG

Abstract. A fast algorithm is proposed that allows determining the position of the central point of a radial vector eld. A typical application is nding the focus of expansion in a eld of velocityvectors and thus give information about the direction of heading of a moving observer or camera. The algorithm uses projections of the vector eld to enhance robustness against disturbances and to reduce the amount of data to be processed.

1 Introduction
The analysis of timedependent patterns plays a vital role as well in human visual perception as in technical applications Many approaches have been reported that model principles of analyzing motion in images (e.g. 11, 1]). The rst task is always to determine the place where in the sequence of perceived images motion can be found, in what direction and with what velocity. This information is contained in the optical ow eld, the vector eld of perceived velocity. Di erent approaches exist to compute this eld (e.g. 4, 3, 2, 10]). Once having found the ow eld, the second task is to interpret it to gain information such as egomotion or segmentation of moving objects. This is usually done by analyzing certain properties of the vector eld such as divergence or curl 7]. An interesting problem that this kind of analysis can be used for is nding the focus of expansion 5, 6, 8, 9]. The calculation of the focus of expansion usually requires a large number of calculations to be performed. In this paper, an algorithm is shown that reduces the amount of data by means of statistical operations and therefore increases the performance.

2 The Projection of Points

Before dealing with the question of how to nd the central point of a given vector eld it is necessary to look how the eld originates. A point P 2 IR3 is mapped onto an imagepoint P 0 2 IR3 on the projection plane E . The projection plane E is described by
E=f

x j x = Q + u + vg

E y P v O

u g P x

QF

Fig. 1. Geometry of the Projection

with Q 2 IR3 being the basepoint, u and v the direction vectors of E and , real factors. Without loss of generality u and v can be regarded as orthonormal: and juj = jvj = 1 : This choice allows to easily describe the projected image in Cartesian coordinates ( ; ) in the image plane. The focus F of the mapping is
F =Q+f (

u?v

with f being the focal distance of the system and n the normal of the plane. The line of projection g is (F ? P )g : The projected point P 0 is the intersection of g with E :
P0 = f g=f

)=Q+f |u {z v}
=

xjx = P +

(F ? P ); x = Q + u + vg : This describes an inhomogeneous system of linear equations of variables , and : u + v + (P ? Q ? f (u v)) = P ? Q : or

xjx = P +

M
with the matrix

0 1 @ A=h

M = (u j v j P ? Q ? f (u v))

and the inhomogeneity

Considering the basepoint Q moving with velocity r and the normal of the plane pointing in x-direction 001 0r 1 1 Q = @ 0 A + t @ r2 A 0 r3 and u = ey ; v = ez ) u v = ex the system of equations to be solved is now 00 0 P ? r t ? f 1 0 1 0P ? r t1 1 @ 1 0 1P2 ?1 r2 t A @ A = @ P1 2 ? r2 t A : 0 1 P3 ? r3 t P3 ? r3 t Inverting the matrix yields the coordinates of the projected moving point P : and = ?f (P3 + r3 t) : (1) = ?f (P2 + r2 t) The velocity eld is determined by the temporal derivatives of the coordinates:
V = P r + P 2 r1 ? r 2 f d = ? f 12 dt (P1 ? r1 t ? f )2 P1 ? r 1 t ? f P1 ? r1 t ? f

h =P ?Q

(2)

and
V =

for all points

d r3 + P3 r1 ? r3 f (3) = ?f P1 dt (P1 ? r1 t ? f )2 : The velocity eld vanishes at the focus of expansion: V = V = 0. This is ful lled

0 P1 1 @r r (f ? P1 ) A
r1 3 r1 (f
2

V (P ) V (P ) r1 ? foe = P1 ? f = ? foe (without loss of generality it was supposed t = 0) so the vector eld has only radial components around the point ( foe ; foe ): V (P ) ? foe V (P ) k ? foe :

It holds

According to equation (1) these points are mapped to r3 r2 and foe = f r : foe = f r
1 1

? P1 )

Let V(x; y) = (vx (x; y); vy (x; y)), f0 x < N g, f0 y < N g be the velocityeld obtained from a sequence of two consecutive images of size N N . These images can be thought of as perceived by an observer approaching a wall that is densely enough textured that there is a displacementvector computable at every position of the image. As mentioned above, the vector eld contains only radial components around the focus of expansion C = (cx ; cy ) = ( foe ; foe ), so

3 The Algorithm

V(x; y) =

The complete image can be divided into four sectors around C : { sector 1, \up right": for all vectors v it holds v > 0 and v > 0 { sector 2, \up left": for all vectors v2 1it holds v2x1x< 0 and v2y1y> 0 and so on.

x ? cx y ? cy

cx

Fig. 2. Partition of the vector eld in four sectors

Let Phy (y) be the normalized horizontal projection ( rst index \h") of the ycomponents (second index \y") of V , and similarly Phx(y), Pvy (x) und Pvx (x) the other three projections. It follows
Phy (y) =
?1 X 1 N (y ? c ) = y ? c 1N vy (x0 ; y) = y y N N
x0 =0

(4)

N ?1 X 1 N (N ? 1) ? N c = N ? 1 ? c = const 1 v (x0 ; y) = Phx (y) = x x x N x =0 x N 2 2 (5)

0

Pvy (x) =

?1 X 1N 1 N (N ? 1) ? N c = N ? 1 ? c = const v (x; y0 ) = y y y N y =0 y N 2 2 (6)

0

Pvx (x) =

?1 X 1 N (x ? c ) = x ? c x 1N vx (x; y0 ) = x x N N
y0 =0

(7)

vy

vy vx

cx

Vertical Projection

vx

Fig. 3. Horizontal and vertical projections, see equations (7) und (4)
Expressions (7) und (4) describe straight lines with values cy resp. cx at the origin having roots at y0 = cy resp. x0 = cx . Expressions (5) und (6) are constant, depending only on imagesize N and coordinates of C . This gives three possibilities to determine the position of C . 1. from the roots x0 and y0 in (7),(4): Phy (y0 ) = 0 , cy = y0 and Pvx (x0 ) = 0 , cx = x0 2. from the values at the origin in (7),(4): cy = ?Phy (0) and cx = ?Pvx (0) ?1 ? consty and cx = N ?1 ? constx 3. from the constants in (5),(6): cy = N2 2

Horizontal Projection

4 Problems and Solutions

As mentioned above, the important demands for the correctness of the results given by the algorithm are the density of the vector eld, the precision of the values, the uniform distance the object and the assumption that the vector eld ? ?to cx without any scaling. Considering a vector eld has the form V(x; y) = x y?cy ?cx with 6= 1 the rst method is still correct whereas the other V(x; y) = ?xy? cy two using the constant expressions and the values at the origin aren't applicable any more. They also may fail to give right results if the vector eld is only sparsely lled and vectors are found in special positions. Figure 4 shows the e ect: the four vectors may well result from analyzing images perceived when approaching a large target. Even there's no disturbance, the projections Phx and Pvy don't give valuable information whereas with Phy and Pvx still the position of C can be determined.
P hx P hy

P vy P vx

Fig. 4. Projections yield di erent results if the vector eld is not completely lled
Unlike synthetic testimages velocity elds generated by algorithms working on real images usually show \wrong" vectors (due to noise or mismatches) and contain much less vectors than desired. Also there are typically objects in various distances. Fortunately \wrong" vectors don't carry much weight due to the averaging e ect of the projections. This has been tested by adding noise vectors to the vector eld before applying the algorithm. The case of various objects in di erent distances has been tested by synthetic elds of vectors that had the usual radial orientation but random lengths. This corresponds to an approaching cloud of point-sized objects.

5 Results
In the case of a nondisturbed eld originating from an imagesequence describing the approaching to a single large object, the algorithm yields the correct result without any error. To cope with noise due to the algorithm supplying the vector eld, the projections of the vector-components were tted using linear regression. The position of the focus was then calculated based on the parameters of the tted straight lines. The algorithm was applied to vector elds of varying size, number of vectors and noise. Figure 5 shows the projections Phy and Pvx of elds with 50% noise i.e., to each vector a vector of half the original amplitude and arbitrary direction is added. Even with a noise level of about this magnitude it is possible to nd almost the correct result: the absolute di erence from correct position to computed position is 1.85 (mean value from 10 runs). Figure 6 shows the e ect of random vector length: although the distribution of the points is much more wide-spread the center can be detected. It should be mentioned that with increasing noise the error increases much faster than in the case without randomized length. This is easily understandable because if both orientation and length are strongly disturbed there's only a statistical distribution of vectors left.

Fig. 5. Projections of vector eld: length of vectors as computed from the image sequence, noise=50% ) abs. error: 1.85

1.84

Fig. 6. Projections of vector eld: vector length randomized, noise=10% ) abs. error:

Figure 7 shows the main advantage of the algorithm: the computing time varies linearly with the width N of the vector eld if common imagesizes are considered. Conventional algorithms require times in the order of N 2 because they analyze the complete eld, solve systems of linear equations or perform transformations on each vector.

Fig. 7. Calculationtime varies with O(imagesize)

The algorithm described so far is designed to nd the center of a purely radial vector eld. Imagescenes taken by a camera moving with all degrees of freedom will contain not only this radial components but also components due to rotation or torsion. These components have to be removed before applying the algorithm. The analysis of the vector eld in order to decompose the di erent components is subject of current work.

6 Summary
A fast and robust algorithm is presented that allows nding the central point of a given vector eld. The main advantage lies in the reduction of required computing time in comparison to conventional algorithms since for a vector eld of size N N the time varies with O(N ). Due to the use of projections the algorithm is easily parallelizable.

References
1. E. H. Adelson, J. R. Bergen: Spatiotemporal energy models for the perception of motion, Journal of the Optical Society of America A 2,2 (1985) 284{299 2. H. Bultho , J. Little, T. Poggio: A parallel algorithm for real-time computation of optical ow, Nature 337, 9 (1989) 549{553 3. D.J. Heeger, A. Jepson: Visual Perception of Three-Dimensional Motion, Neural Computation 2, 2 (1990) 129{137

4. B. K. P. Horn, B. G. Schunck: Determining Optical Flow, Arti cial Intelligence 17 (1981) 185{203 5. R. Jain: Direct Computation of the Focus of Expansion, IEEE Trans. Pattern Analysis and Machine Intell. 5, 1 (1983) 58{64 6. S. Negahdaripour, B.K.P. Horn: A Direct Method for Locating the Focus of Expansion, Computer Vision, Graphics and Image Processing: Image Understanding 46 (1989) 303{326 7. T. Poggio, A. Verri, V. Torre: Green Theorems and Qualitative Properties of the Optical Flow, Technical Report, Massachusetts Institute of Technology, Arti cial Intelligence Laboratory (1991) 8. K. Rangarajan, M. Shah: Interpretation of Motion Trajectories Using Focus of Expansion, IEEE Trans. Pattern Analysis and Machine Intell. 14, 12 (1992) 1205{ 1210 9. K. Tanaka, H.-S. Saito: Analysis of Motion of the Visual Field by Direction, Expansion/Contraction, and Rotation Cells Clustered in the Dorsal Part of the Medial Superior Temporal Area of the Macaque Monkey, Journal of Neurophysiology 62,3 (1989) 626{641 10. M. Tistarelli, G. Sandini: Estimation of depth from motion using an anthropomorphic visual sensor, Image Vision Computat. 8,4 (1990) 11. A. B. Watson, A. J. Ahumada Jr.: Model of human visual-motion sensing, Journal of the Optical Society of America A 2, 2 (1985) 322{341

aTEX macro package with LLNCS style This article was processed using the L