Robert Collins

CSE486, Penn State

Lecture 19:
Essential and Fundamental Matrices
Robert Collins
CSE486, Penn State
Epipolar Geometry
image1 image 2

Epipole : location of cam2 Epipole : location of cam1

as seen by cam1. as seen by cam2.
Robert Collins
CSE486, Penn State
Epipolar Geometry
image1 image 2

Corresponding points
lie on conjugate epipolar lines
Robert Collins
CSE486, Penn State
This Lecture…
image1 image 2

Given a point in one image, how do we determine

the corresponding epipolar line to search along in
the second image?
Robert Collins
CSE486, Penn State
Essential Matrix

The essential and fundamental matrices are 3x3 matrices

that “encode” the epipolar geometry of two views.

Motivation: Given a point in one image, multiplying

by the essential/fundamental matrix will tell us which
epipolar line to search along in the second view.
Robert Collins
CSE486, Penn State
Essential Matrix
P
Pl Pr

pl pr
Ol el er Or
T
R,T = rotation,
and translation

S=

E=RS is “essential matrix”

Robert Collins
CSE486, Penn State
Essential Matrix Properties

• has rank 2
 has both a left and right nullspace (important!!!!)

• depends only on the EXTRINSIC Parameters (R & T)

Robert Collins
CSE486, Penn State
Longuet-Higgins equation
Robert Collins
CSE486, Penn State
Longuet-Higgins equation

This relates
viewing rays
Importance of Longuet-Higgens ...
This relates
2D film points
Robert Collins
CSE486, Penn State
Longuet-Higgins Makes Sense

• Note, there is nothing magic about Longuet-

Higgins equation.
• A film point can also be thought of as a
viewing ray. They are equivalent.
• (u,v) 2D film point
• (u,v,f) 3D point on film plane
• k(u,v,f) viewing ray into the scene
• k(X, Y, Z) ray through point P in the scene
[hint: k=f/Z, and we have u=fX/Z, v=fY/Z].
Robert Collins
CSE486, Penn State
Epipolar Lines

• Let l be a line in the image:

• Using homogeneous coordinates:

Robert Collins
CSE486, Penn State
Epipolar Lines

• Remember:

pr belongs to epipolar line in the right image defined by

Robert Collins
CSE486, Penn State
Epipolar Lines

• Remember:

pl belongs to epipolar line in the left image defined by

Robert Collins
CSE486, Penn State
Epipoles
• Remember: epipoles belong to the epipolar lines

• And they belong to all the epipolar lines

We can use this to compute the location of the epipoles.

There will be an example, shortly...
Robert Collins
CSE486, Penn State
Essential Matrix Summary

Longuet-Higgins equation

Epipolar lines:

Epipoles:
Robert Collins
CSE486, Penn State
Fundamental Matrix

The essential matrix uses CAMERA coordinates

To use image coordinates we must consider the
INTRINSIC camera parameters:
Affine transform matrix
Pixel coord
Camera (film) coord
(row,col)
Robert Collins
CSE486, Penn State
Fundamental Matrix

short version: The same equation works in

pixel coordinates too!
Robert Collins
CSE486, Penn State
Fundamental Matrix Properties

• has rank 2
• depends on the INTRINSIC and EXTRINSIC Parameters (f, etc ;
R & T)
Analogous to essential matrix. The fundamental
matrix also tells how pixels (points) in each image
are related to epipolar lines in the other image.
Robert Collins
CSE486, Penn State
Example
Robert Collins
CSE486, Penn State
Example
Robert Collins
CSE486, Penn State
Example
-0.00310695 -0.0025646 2.96584
F= -0.028094 -0.00771621 56.3813
13.1905 -29.2007 -9999.79
Robert Collins
CSE486, Penn State
Example
-0.00310695 -0.0025646 2.96584 343.53
F= -0.028094 -0.00771621 56.3813 221.70
13.1905 -29.2007 -9999.79 1.0

left
0.0001 0.0295
0.0045  0.9996
-1.1942 -265.1531
normalize so sum of squares
of first two terms is 1 (optional)
x = 343.5300 y = 221.7005
Robert Collins
CSE486, Penn State
Example
-0.00310695 -0.0025646 2.96584 343.53
F= -0.028094 -0.00771621 56.3813 221.70
13.1905 -29.2007 -9999.79 1.0

left right
0.0295
0.9996
-265.1531

x = 343.5300 y = 221.7005
Robert Collins
CSE486, Penn State
Example
( 205.5526 80.5 1.0) -0.00310695 -0.0025646 2.96584
-0.028094 -0.00771621 56.3813
13.1905 -29.2007 -9999.79

L = (0.0010 -0.0030 -0.4851) right

 (0.3211 -0.9470 -151.39)

x = 205.5526 y = 80.5000
Robert Collins
CSE486, Penn State
Example
( 205.5526 80.5 1.0) -0.00310695 -0.0025646 2.96584
-0.028094 -0.00771621 56.3813
13.1905 -29.2007 -9999.79

L= (0.3211 -0.9470 -151.39)

right
left

x = 205.5526 y = 80.5000
Robert Collins
CSE486, Penn State
Example
where is the epipole?
F * eL = 0

vector in the right

nullspace of matrix F

However, due to noise,

F may not be singular.
So instead, next best
thing is eigenvector
associated with smallest
eigenvalue of F
Robert Collins
CSE486, Penn State
Example
>> [u,d] = eigs(F’ * F)
u= d = 1.0e8*
-0.0013 0.2586 -0.9660 -1.0000 0 0
0.0029 -0.9660 -0.2586 0 -0.0000 0
1.0000 0.0032 -0.0005 0 0 -0.0000

eigenvector associated with smallest eigenvalue

>> uu = u(:,3)
uu = ( -0.9660 -0.2586 -0.0005)

>> uu / uu(3) : to get pixel coords

(1861.02 498.21 1.0)
Robert Collins
CSE486, Penn State
Example
where is the epipole? e’r * F = 0
 F’ * er = 0
vector in the right
nullspace of matrix F’

However, due to noise,

F’ may not be singular.
So instead, next best
thing is eigenvector
associated with smallest
eigenvalue of F’
Robert Collins
CSE486, Penn State
Example
>> [u,d] = eigs(F * F’)
u= d = 1.0e8*
-0.0003 -0.0618 -0.9981 -1.0000 0 0
-0.0056 -0.9981 0.0618 0 -0.0000 0
1.0000 -0.0056 0.0001 0 0 -0.0000

eigenvector associated with smallest eigenvalue

>> uu = u(:,3)
uu = (-0.9981 0.0618 0.0001)

>> uu / uu(3) : to get pixel coords

(-19021.8 1177.97 1.0)