Scribd
Upload
140 views2 pages

Lucas-Kanade Method For Optical Flow

The document summarizes the Lucas-Kanade optical flow method. It begins by introducing optical flow and the brightness constancy assumption that pixel intensity remains constant between frames. This leads to the optical flow equation relating pixel intensity derivatives to flow vectors. The Lucas-Kanade method adds the assumption of spatial coherence to solve the aperture problem, expressing optical flow equations for a neighborhood as a least squares problem to solve for flow vectors. Harris corners make good features as they have well-defined intensity gradients in multiple directions.

Uploaded by

李金良
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
140 views2 pages

Lucas-Kanade Method For Optical Flow

The document summarizes the Lucas-Kanade optical flow method. It begins by introducing optical flow and the brightness constancy assumption that pixel intensity remains constant between frames. This leads to the optical flow equation relating pixel intensity derivatives to flow vectors. The Lucas-Kanade method adds the assumption of spatial coherence to solve the aperture problem, expressing optical flow equations for a neighborhood as a least squares problem to solve for flow vectors. Harris corners make good features as they have well-defined intensity gradients in multiple directions.

Uploaded by

李金良
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

1.

Lucas-Kanade Method for Optical Flow


1.1 Deriving optical flow equation
Optical flow methods are used to estimate motion of objects between two consecutive image frames. For example, in
the video above, the can of tea seems to be moving to the left. For our system to be able to understand that the can
is moving to the left, it would be useful to find a way to add vectors to the can (known as flow vectors) which
point to the left, thus describing its motion.
Given two consecutive frames, how can we find the flow vectors for the first frame which describe how objects move
between frames? To start, we make a reasonable assumption called the brightness constancy assumption: the
pixel intensity of a moving point stays the same between two consecutive frames with small time differences. In
other words, picking any pixel of the moving can, its brightness stays approximately the same between frames — its
movement should not affect its brightness after all.
Consider pixel intensity (a.k.a. brightness) I(x, y, t) of a point (x, y) in the first frame t. Suppose that the point has
moved to (x + ∆x, y + ∆y) after ∆t. According to the brightness constancy assumption, we can relate intensities of
the point in the two frames using the following equation:

I(x, y, t) = I(x + ∆x, y + ∆y, t + ∆t)

Coming back to the example of the moving can, this equation simply states that the point that we picked will have
the same intensity even after it moves in space (∆x and ∆y) and between frames (∆t). From this simple assumption,
we can derive what is known as the optical flow equation. For a given point for any frame, the optical flow
equation is given by:

Ix (p)vx + Iy (p)vy + It (p) = 0

Here, Ix , Iy and It are partial derivatives of pixel intensity I. Meanwhile,


vx and vy are flow vectors in the x− and y− direction, respectively. These are the vectors we care about! If we
can solve for these two values, we will be able to describe the motion of any object between frames.
You might be wondering how we went from the brightness constantcy assumption to the optical flow equation. Try
to derive it yourself!
• a. Using a first-order Taylor approximation, derive the optical flow equation from the brightness constancy
equation.
• b. Can the optical flow equation be solved given two consecutive frames without further assumption? Of
Ix , Iy , It , vx , and vy , which values can be computed directly given two consecutive frames? Which values
cannot be computed without additional information?
Your answer here: Write your answer in this markdown cell

Answer:
According to Taylor expansion, we have:

∂I ∂x ∂I ∂y ∂I
I(x + ∆x, y + ∆y, t + ∆t) = I(x, y, t) + + +
∂x ∂t ∂y ∂t ∂t
= I(x, y, t) + Ix (p)vx + Iy (p)vy + It (p)

According to the brightness constancy assumption, we have:

I(x, y, t) = I(x + ∆x, y + ∆y, t + ∆t)

so, we got:

1
Ix (p)vx + Iy (p)vy + It (p) = 0

1.2 Overview of Lucas-Kanade method


One issue with the optical flow equation is that there are two unknowns that we want to solve for (vx and vy ). This
problem is known as the aperture problem. In other words, just looking an "aperture" at one pixel at a time, it is
impossible to discern the true direction of motion of the object in question.
The Lucas–Kanade method solves this problem by adding another assumption: spatial coherence. That is, that
the motion of the image contents between two frames is approximately constant within a neighborhood of the point
p under consideration.
Consider a neighborhood of p, N (p) = {p1 , ..., pn } (e.g. 3x3 window around p). Adding the spatial coherence
assumption to the optical flow equation, we see that the following should be satisfied:
For every pi ∈ N (p),

Ix (pi )vx + Iy (pi )vy = −It (pi )

These equations can be written in matrix form Av = b, where

Ix (p1 ) Iy (p1 ) −It (p1 )


 
 Ix (p2 ) Iy (p2 )    −It (p2 )
v
A= . ..  v= x b= ..
 
 .. .  vy .
Ix (pn ) Iy (pn ) −It (pn )

We can now solve for the flow vectors (now represented as v) by solving the following least-squares problem:
AT Av = AT b.
• a. What is the condition for this equation to be solvable?
• b. Reason about why Harris corners might be good features to track using Lucas-Kanade method.

You might also like