Scribd
Upload
89 views

Lecture18 PDF

The document discusses epipolar geometry and the essential matrix. It begins by explaining that epipolar geometry describes the geometric relationship between two views of a scene taken from different positions. It then defines key concepts like epipoles, epipolar lines, and the essential matrix. The essential matrix encodes the epipolar geometry and allows mapping a point in one image to its corresponding epipolar line in the other image. The document derives the essential matrix from the camera transformation between views and shows it satisfies the epipolar constraint. It compares the essential matrix to a homography and discusses properties like epipoles and generalized versions like the fundamental matrix for uncalibrated cameras.

Uploaded by

yacobaschalew
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
89 views

Lecture18 PDF

The document discusses epipolar geometry and the essential matrix. It begins by explaining that epipolar geometry describes the geometric relationship between two views of a scene taken from different positions. It then defines key concepts like epipoles, epipolar lines, and the essential matrix. The essential matrix encodes the epipolar geometry and allows mapping a point in one image to its corresponding epipolar line in the other image. The document derives the essential matrix from the camera transformation between views and shows it satisfies the epipolar constraint. It compares the essential matrix to a homography and discusses properties like epipoles and generalized versions like the fundamental matrix for uncalibrated cameras.

Uploaded by

yacobaschalew
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 70

Epipolar Geometry

16-385 Computer Vision


Carnegie Mellon University (Kris Kitani)

Tie tiny threads on HERB and pin them to your eyeball

What would it look like?

You see points on HERB

What does the second observer see?

You see points on HERB

Second person sees lines

This is Epipolar Geometry

Epipole

Epipolar lines

Epipolar geometry
p

o
Image plane

Epipolar geometry
p

Baseline
Image plane

Epipolar geometry
p

e
Image plane

Baseline

Epipole
(projection of o on the image plane)

Epipolar geometry
p
Epipolar plane

e
Image plane

Baseline

Epipole
(projection of o on the image plane)

Epipolar geometry
Epipolar line
(intersection of Epipolar
plane and image plane)

p
Epipolar plane

l
o

l
e

Image plane

Baseline

Epipole
(projection of o on the image plane)

Epipolar constraint
p

x
o

x0

l
e

Potential matches for

x lie on the epipolar line l

x
o

x0

l
e

o0

The point x (left image) maps to a ___________ in the right image


The baseline connects the ___________ and ____________
An epipolar line (left image) maps to a __________ in the right image
An epipole e is a projection of the ______________ on the image plane
All epipolar lines in an image intersect at the ______________

Converging cameras

Where is the epipole in this image?

Converging cameras

Where is the epipole in this image?

Its not always in the image

Parallel cameras

Where is the epipole?

Parallel cameras

epipole at infinity

Forward moving camera

Forward moving camera

Where is the epipole?


What do the epipolar lines look like?

Epipole has same coordinates in both images.


Points move along lines radiating from Focus of expansion

The epipolar constraint is an important concept for stereo vision

Task: Match point in left image to point in right image

Left image

Right image

How would you do it?

The epipolar constraint is an important concept for stereo vision

Task: Match point in left image to point in right image

Left image

Right image

Want to avoid search over entire image


(if the images have been rectified)
Epipolar constrain reduces search to a single line

E
Essential Matrix
16-385 Computer Vision
Carnegie Mellon University (Kris Kitani)

Recall:Epipolar constraint
p

x
o

x0

l
e

Potential matches for

x lie on the epipolar line l

The epipolar geometry is an important concept for stereo vision

Task: Match point in left image to point in right image

Left image

Right image

How would you do it?

The epipolar constraint is an important concept for stereo vision

Task: Match point in left image to point in right image

Left image

Right image

Epipolar constrain reduces search to a single line

How do you compute the epipolar line?

Essential Matrix

The Essential Matrix is a 3 x 3 matrix that


encodes epipolar geometry

Given a point in one image,


multiplying by the essential matrix will tell us
the epipolar line in the second view.

Ex = l

X
0

x
o

o0

Representing the

Epipolar Line
ax + by + c = 0

in vector form

If the point

a
l=4 b 5
c

x is on the epipolar line l then

e
x

>

x l= ?

Epipolar Line
ax + by + c = 0

in vector form

If the point

a
l=4 b 5
c

x is on the epipolar line l then

e
x

>

x l=0

Recall:
Dot
Product
c=ab
b
a
ca=0

cb=0

So if

>

x l=0

and

Ex = l

then

0>

x Ex = ?
X
0

x
o

o0

So if

>

x l=0

and

Ex = l

then

0>

x Ex = 0
X
0

x
o

o0

Motivation
The Essential Matrix is a 3 x 3 matrix that
encodes epipolar geometry

Given a point in one image,


multiplying by the essential matrix will tell us
the epipolar line in the second view.

Essential Matrix vs Homography


Whats the difference between the essential matrix and a homography?

Essential Matrix vs Homography


Whats the difference between the essential matrix and a homography?
They are both 3 x 3 matrices but

Essential Matrix vs Homography


Whats the difference between the essential matrix and a homography?
They are both 3 x 3 matrices but

l = Ex

x = Hx

Essential matrix maps a


point to a line

Homography maps a
point to a point

Where does the Essential matrix come from?

X
x0

x
o

R, t

x = R(x

t)

X
x0

x
o

R, t

x = R(x

t)

Does this look familiar?

X
x0

x
o

R, t

x = R(x

t)

Camera-camera transform just like world-camera transform

X
x0

x
o

These three vectors are coplanar

x, t, x

X
x0

x
o

If these three vectors are coplanar

>

x, t, x

x (t x) = ?

then

X
x0

x
o

If these three vectors are coplanar

>

x, t, x

x (t x) = 0

then

Recall: Cross Product


Vector (cross) product
takes two vectors and returns a vector perpendicular to both

c=ab
b
a
ca=0

cb=0

X
x0

x
o

If these three vectors are coplanar

(x

>

x, t, x

t) (t x) = ?

then

X
x0

x
o

If these three vectors are coplanar

(x

>

x, t, x

t) (t x) = 0

then

putting it together
rigid motion
0

x = R(x

coplanarity

t)
0>

(x

>

t) (t x) = 0

(x R)(t x) = 0

putting it together
rigid motion
0

x = R(x

coplanarity

t)
0>

(x

>

t) (t x) = 0

(x R)(t x) = 0
0>
(x R)([t ]x) = 0

Cross product

a 2 b3
a b = 4 a 3 b1
a 1 b2

a 3 b2
a 1 b3 5
a 2 b1

Can also be written as a matrix multiplication

0
a b = [a] b = 4 a3
a2

a3
0
a1

32

a2
b1
a 1 5 4 b2 5
0
b3

Skew symmetric

putting it together
rigid motion
0

x = R(x

coplanarity

t)
0>

(x

>

t) (t x) = 0

(x R)(t x) = 0
0>
(x R)([t ]x) = 0
0>
x (R[t ])x = 0

putting it together
rigid motion
0

x = R(x

coplanarity

t)

(x

0>

>

t) (t x) = 0

(x R)(t x) = 0
0>
(x R)([t ]x) = 0
0>
x (R[t ])x = 0
0>

x Ex = 0

putting it together
rigid motion
0

x = R(x

coplanarity

t)

(x

>

t) (t x) = 0

0>

(x R)(t x) = 0
0>
(x R)([t ]x) = 0
0>
x (R[t ])x = 0
0>

x Ex = 0

Essential Matrix"
[Longuet-Higgins 1981]

properties of the E matrix


Longuet-Higgins equation

0>

x Ex = 0

(points in normalized coordinates)

properties of the E matrix


Longuet-Higgins equation

Epipolar lines

>

x l=0
0

l = Ex

0>

x Ex = 0
0> 0

x l =0
T

l=E x

(points in normalized coordinates)

properties of the E matrix


Longuet-Higgins equation

Epipolar lines

>

x l=0

x Ex = 0
0> 0

x l =0

l=E x

0>

Ee = 0

l = Ex
Epipoles

0>

e E=0

(points in normalized coordinates)

How do you generalize to


uncalibrated cameras?

F
Fundamental Matrix
16-385 Computer Vision

Recall:Epipolar constraint
p

x
o

x0

l
e

Potential matches for

x lie on the epipolar line l

Given a point in one image,


multiplying by the essential matrix will tell us
the epipolar line in the second view.

Ex = l

X
0

x
o

o0

Assumption: "
points aligned to camera coordinate axis (calibrated camera)

The
Fundamental matrix "
is a
generalization
of the
Essential matrix, "
where the assumption of
calibrated cameras
is removed

0>

Ex
=0
x
The Essential matrix operates on image points expressed in
normalized coordinates
(points have been aligned (normalized) to camera coordinates)

=K
x

camera
point

x0 = K

image
point

Writing out the epipolar constraint in terms of image coordinates

0>

0 >

0>

0 >

x K

x (K

0>

EK

EK

x=0

)x = 0

x Fx = 0

Same equation works in image coordinates!

0>

x Fx = 0
it maps pixels to epipolar lines

properties of the E matrix


Longuet-Higgins equation

Epipolar lines

>

x l=0

x Ex = 0
0> 0

x l =0

l=E x

0>

Ee = 0

l = Ex
Epipoles

0>

e E=0

(points in image coordinates)

Breaking down the fundamental matrix

F=K
F=K

0 >

0 >

EK

[t ]RK

Depends on both intrinsic and extrinsic parameters

Breaking down the fundamental matrix

F=K
F=K

0 >

0 >

EK

[t ]RK

Depends on both intrinsic and extrinsic parameters


How would you solve for F?
0>
xm Fxm

=0

You might also like