Computer Vision

Dr. Syed Faisal Bukhari

Associate Professor
Department of Data Science
Faculty of Computing and Information Technology
University of the Punjab
Textbook

Multiple View Geometry in Computer Vision,

Hartley, R., and Zisserman

Richard Szeliski, Computer Vision: Algorithms and

Applications, 1st edition, 2010

Dr. Faisal Bukhari, DDS, PU

Reference books
Readings for these lecture notes:
Hartley, R., and Zisserman, A. Multiple View Geometry in
Computer Vision, Cambridge University Press, 2004, Chapters 1-3.

Forsyth, D., and Ponce, J. Computer Vision: A Modern

Approach, Prentice-Hall, 2003, Chapter 2.

Linear Algebra and its application

by David C Lay
These notes contain material c Hartley and Zisserman (2004),
Forsyth and Ponce (2003), an Linear Algebra and its application
by David C Lay

Dr. Faisal Bukhari, DDS, PU

References
These notes are based
Dr. Matthew N. Dailey's course: AT70.20: Machine Vision
for Robotics and HCI

Dr. Sohaib Ahmad Khan CS436 / CS5310 Computer Vision

Fundamentals at LUMS

Dr. Faisal Bukhari, DDS, PU

2D Transformation
• Definition: A mapping from one 2D coordinate
system to another
• Also called
o spatial transformation,
o geometric transformation,
o warp

Dr. Faisal Bukhari, DDS, PU

Image Registration
oImage Registration: Process of transforming two images so
that same features overlap

oImage registration aims to geometrically align one image

with another and is a prerequisite for all brain imaging
applications that compare images across subjects, across
imaging modalities, or across time (Toga & Thompson,
2001).

Dr. Faisal Bukhari, DDS, PU

What is Image Registration?
o Process of aligning two or more images of the same scene.

oUsed when images are captured at different times,

viewpoints, or sensors.

oTransforms one image to match another using spatial

transformation models.

Dr. Faisal Bukhari, DDS, PU

Steps in Image Registration
1. Feature Detection: Identify key points (e.g., edges,
corners).

2. Feature Matching: Pair corresponding features between

images.

3. Transformation Estimation: Compute a mathematical

model to align images.

4. Resampling & Interpolation: Transform and interpolate the

image to match.
Dr. Faisal Bukhari, DDS, PU
Applications of Image Registration
oMedical Imaging: Aligning MRI, CT, and PET scans for
diagnosis.
oRemote Sensing: Change detection, disaster assessment,
and multi-sensor fusion.
oComputer Vision & AR: Object tracking, video stabilization,
and augmented reality.
oRobotics & Autonomous Systems: SLAM for robotics, self-
driving car navigation.
oForensics & Security: Fingerprint recognition, facial
authentication.
oAstronomy & Space: Aligning celestial images for deep
space observations.
Dr. Faisal Bukhari, DDS, PU
Example: Medical Imaging
oImage registration helps align MRI and CT scans for accurate
diagnosis.

oDetects tumor growth by comparing past and current scans.

oUsed in PET-CT fusion to combine anatomical and functional

imaging.

Dr. Faisal Bukhari, DDS, PU

Example: Remote Sensing
oSatellite images are aligned for change detection.

oUsed for tracking deforestation, urban expansion, and

disaster damage.

o Helps in merging optical and radar images for better

analysis.

Dr. Faisal Bukhari, DDS, PU

Conclusion
oImage registration is crucial for various fields like healthcare,
AI, and space.

o Aligns images from different sources for improved analysis.

oEnables better decision-making through accurate visual

integration.

Dr. Faisal Bukhari, DDS, PU

 Example Application: Image Registration

Reference
Image

Dr. Faisal Bukhari, DDS, PU

Mission
Images

Dr. Faisal Bukhari, DDS, PU

Registration = Computing Transformation

Dr. Faisal Bukhari, DDS, PU

Dr. Faisal Bukhari, DDS, PU
Dr. Faisal Bukhari, DDS, PU
Dr. Faisal Bukhari, DDS, PU
Dr. Faisal Bukhari, DDS, PU
Dr. Faisal Bukhari, DDS, PU
Dr. Faisal Bukhari, DDS, PU
 Applications of 2D
Panoramas Image Registration

Multiple Images Stitched Together

Dr. Faisal Bukhari, DDS, PU

 Applications of 2D
Panoramas Image Registration
Multiple Images Stitched Together

Image by Sergey Semenov (http://www.sergesemenov.com/) - Winner of Epson International Photographic Pano Award 2013
Dr. Faisal Bukhari, DDS, PU
http://www.dailymail.co.uk/sciencetech/article-2260276/New-York-youve-seen-Incredible-interactive-panorama-lets-zoom.html
 Applications of 2D
Panoramas Image Registration
Multiple Images Stitched Together

Image by Sergey Semenov (http://www.sergesemenov.com/) - Winner of Epson International Photographic Pano Award 2013
Dr. Faisal Bukhari, DDS, PU
http://www.dailymail.co.uk/sciencetech/article-2260276/New-York-youve-seen-Incredible-interactive-panorama-lets-zoom.html
 Applications of 2D
Image Registration
Spherical 360o Imaging

Lady Bug Camera, by Point Grey

6 0.8MP cameras image 75% of a full sphere
http://www.ptgrey.com/products/ladybug2/ladybug2_360_video_camera.asp
Dr. Faisal Bukhari, DDS, PU
 Applications of 2D
Image Registration
Spherical 360o Imaging

Lady Bug Camera, by Point Grey

6 0.8MP cameras image 75% of a full sphere
http://www.ptgrey.com/products/ladybug2/ladybug2_360_video_camera.asp
Dr. Faisal Bukhari, DDS, PU
2D Transformations
Basic operation of all 2D transformations is matrix
multiplication

oPoint to be transformed: (x, y)Τ

oPoint after transformation: (𝑥 ′ , 𝑦 ′ ) Τ

𝑎1 𝑎2 𝑥 𝑎1 𝑥 + 𝑎2 𝑦 𝑥′
𝑎3 𝑎4 𝑦 = 𝑎 𝑥+𝑎 𝑦 = ′
2×2 2×1 3 4 2×1 𝑦 2×1

Transformation Position before Position after

Matrix transformation transformation

Dr. Faisal Bukhari, DDS, PU

 Example
(0,0) (0,a) (0,0) (0,a)
Apply

0.5 0 
 0 0.5
  (a,0) (a,a)
(a,0) (a,a)
Original Transformed

0.5 0 0 0 0.5 0 0 0
= =
0 0.5 0 0 0 0.5 𝑎 0.5𝑎

0.5 0 𝑎 0.5𝑎 0.5 0 𝑎 0.5𝑎

= =
0 0.5 0 0 0 0.5 𝑎 0.5𝑎

Dr. Faisal Bukhari, DDS, PU

2D Transformations
1.5 0
=?
0 0.5

In general, scaling (zoom / unzoom) transformation is given by

𝑠𝑥 0
0 𝑠𝑦

Dr. Faisal Bukhari, DDS, PU

 2D Transformations
1 1
=?
0 1
1 1 0 0 1 1 0 1
= =
0 1 0 0 0 1 1 1

1 1 1 1 1 1 1 2
= =
0 1 0 0 0 1 1 1
(0,0) (0,1)
(0,0) (0,1)

(1,0) (1,1)
(1,0) (1,1)
Original
Transformed

Dr. Faisal Bukhari, DDS, PU (2,1)

Shear in x-direction

1 𝑒 𝑥 𝑥 + 𝑒𝑦
𝑦 = 𝑦
0 1

x-coordinate moves with an amount proportional

to the y-coordinate
For example:
1 1 1 1+1×0 1
= =
0 1 0 0 0

1 1 1 1+1×1 2
= =
0 1 1 1 1

Dr. Faisal Bukhari, DDS, PU

Shear in y-direction

1 0 𝑥 𝑥
=
𝑒 1 𝑦 𝑒𝑥 + 𝑦

y-coordinate moves with an amount proportional

to the x-coordinate

Dr. Faisal Bukhari, DDS, PU

Rotation
y
(x2 y2)

(x1 y1)


 Task: Relate (x2, y2) to (x1, y1)

Dr. Faisal Bukhari, DDS, PU

 x2  R cos(   )
Rotation y2  R sin(    )
x2  R cos cos   R sin  sin 
y
y2  R sin  cos   R cos sin 
(x2 y2)

R
x2  x1 cos  y1 sin 
(x1 y1)
y2  x1 sin   y1 cos
R

y1  R sin 

 x2  cos  sin    x1 
x  y    sin   
cos   y1  
 2 
x1  R cos 
R
R is rotation by  counterclockwise about origin
Dr. Faisal Bukhari, DDS, PU
Alternative Method for Derivation
Example Let T: ℝ𝟐 → ℝ𝟐 be the transformation that
rotates each point in ℝ𝟐 about the origin through an
angle 𝝋 with counterclockwise rotation for a
positive angle.

We could show geometrically that such a

transformation is linear. Find the standard matrix A
of this transformation.

Dr. Faisal Bukhari, DDS, PU

Derivation of Rotation matrix:
𝑩𝒂𝒔𝒆
∵ 𝒄𝒐𝒔𝝋 =
𝑯𝒚𝒑𝒐𝒕𝒆𝒏𝒐𝒖𝒔

𝑷𝒆𝒓𝒑𝒆𝒏𝒊𝒅𝒖𝒍𝒂𝒓
𝒔𝒊𝒏𝝋 =
𝑯𝒚𝒑𝒐𝒕𝒆𝒏𝒐𝒖𝒔

x = r 𝒄𝒐𝒔𝝋 -----------(1)
y = r 𝒔𝒊𝒏𝝋 ------------(2)
Squaring (1) and (2) and then
adding, we get
r = 𝒙𝟐 +𝒚𝟐
𝒙′ = r 𝒄𝒐𝒔(𝝋 + 𝜽) -------(3)
∵ 𝒄𝒐𝒔(𝝋 + 𝜽) = cos𝝋cos 𝜽 − sin𝝋𝐬𝐢𝐧 𝜽
Substitute the value of cos(𝜑 + 𝜃) in (3), we get
𝒙′ = r cos𝝋cos 𝜽 - r sin𝝋𝒔𝒊𝒏 𝜽 -----(4)
𝒚′ = r 𝒔𝒊𝒏 𝝋 + 𝜽 -------(5)
Dr. Faisal Bukhari, DDS, PU
∵ 𝒔𝒊𝒏 𝑨 + 𝑩 = sinAcos B − cos𝐀𝐬𝐢𝐧 B
Substitute the value of sin(𝜑 + 𝜃) in (5), we get
𝒚′ = r s𝐢𝐧𝝋cos 𝜽 + r c𝐨𝒔𝝋𝐬𝐢𝐧 𝜽 ------(6)
Substitute the values of x and y from (1) and (2) in
(4), we get
𝒙′ = xcos𝜽 - y𝐬𝐢𝐧 𝜽 ---------------------(7)
Substitute the values of x and y from (1) and (2) in
(6), we get
𝒚′ = ycos 𝜽 + x𝐬𝐢𝐧 𝜽
𝒚′ = x𝐬𝐢𝐧 𝜽 + ycos𝜽 -------------------(8)
Using (7) and (8), we get
𝒙′ 𝒄𝒐𝒔𝜽 −𝒔𝒊𝒏𝜽 𝒙
′ =
𝒚 𝒔𝒊𝒏𝜽 𝒄𝒐𝒔𝜽 𝒚
Dr. Faisal Bukhari, DDS, PU
Rotation
𝒄𝒐𝒔𝜽 −𝒔𝒊𝒏𝜽
oR(𝜽) =
𝒔𝒊𝒏𝜽 𝒄𝒐𝒔𝜽

oRotation Matrix has some special properties

oEach row/column has norm of 1 [prove]

oEach row/column is orthogonal to the other [prove]

oSo Rotation matrix is an orthonormal matrix

oInverse of an orthonormal matrix is its transpose [prove]

Dr. Faisal Bukhari, DDS, PU

To prove that each row (or column) is orthogonal to the other,
we compute their dot product.
Step 1: Define Rows and Compute Dot Product
The first row of R(𝜽) is
𝑟1 ​ = (cosθ,−sinθ)

The second row of R(𝜽) is

𝑟2 ​ = (sinθ,−cosθ)

The dot product of these two row vectors is:

𝑟1 ∙ 𝑟2 = (cosθ)(sinθ) + (−sinθ)(cosθ)
= cosθsinθ − sinθcosθ
=0
Since their dot product is zero, the rows are orthogonal.

Dr. Faisal Bukhari, DDS, PU

Step 2: Define Columns and Compute Dot Product
The first col of R(𝜽) is
𝑐𝑜𝑠𝜃
𝑐1 ​ =
𝑠𝑖𝑛𝜃

The second col of R(𝜽) is

−𝑠𝑖𝑛𝜃
𝑐2 ​ =
𝑐𝑜𝑠𝜃

The dot product of these two row vectors is:

𝑐1 ∙ 𝑐2 = (cosθ)(-sinθ) + (sinθ)(cosθ)
=-cosθsinθ +sinθcosθ
=0
Since their dot product is zero, the columns are orthogonal.

Dr. Faisal Bukhari, DDS, PU

Orthonormal Matrix
An orthonormal matrix is a square matrix whose columns
and rows are both orthogonal and unit vectors (having a
magnitude of 1).

oEach row/column of R(𝜽) has norm of 1

oEach row/column of R(𝜽) is orthogonal to the other

oSo Rotation matrix R(𝜽) is an orthonormal matrix

oInverse of an orthonormal matrix is its transpose

Dr. Faisal Bukhari, DDS, PU

Inverse of an orthonormal matrix is
its transpose
𝒄𝒐𝒔𝜽 −𝒔𝒊𝒏𝜽
R(𝜽) =
𝒔𝒊𝒏𝜽 𝒄𝒐𝒔𝜽

Inverse of R(𝜽), an orthonormal matrix is its transpose.

Transpose of R(𝜽) is

𝒄𝒐𝒔𝜽 𝒔𝒊𝒏𝜽
R(𝜽)−1 =
−𝒔𝒊𝒏𝜽 𝒄𝒐𝒔𝜽

Dr. Faisal Bukhari, DDS, PU

 𝒄𝒐𝒔𝜽 −𝒔𝒊𝒏𝜽
R(𝜽) =
𝒔𝒊𝒏𝜽 𝒄𝒐𝒔𝜽

det(R) = (𝑐𝑜𝑠𝜃 )(𝑐𝑜𝑠𝜃) – (−𝑠𝑖𝑛𝜃)(𝑠𝑖𝑛𝜃)

= 𝑐𝑜𝑠 2 𝜃 + 𝑠𝑖𝑛2 𝜃
=1
𝒄𝒐𝒔𝜽 𝒔𝒊𝒏𝜽
adj(R) =
−𝒔𝒊𝒏𝜽 𝒄𝒐𝒔𝜽
1
R−1 = adj(R)
det(R)

1 𝒄𝒐𝒔𝜽 𝒔𝒊𝒏𝜽
=
1 −𝒔𝒊𝒏𝜽 𝒄𝒐𝒔𝜽

𝒄𝒐𝒔𝜽 𝒔𝒊𝒏𝜽
R(𝜽)−1 =
−𝒔𝒊𝒏𝜽 𝒄𝒐𝒔𝜽
Dr. Faisal Bukhari, DDS, PU
Identifying Rotation and Reflection
Matrices
−1 0
= (-1)(-1) – 0 =1
0 −1
It is a rotation matrix

1 0
= -1
0 −1
It is a reflection matrix.

−1 0
= -1
0 1
It is a reflection matrix.

Dr. Faisal Bukhari, DDS, PU

Homogeneous system
In general, a matrix multiplication lets us linearly combine
components of a vector

𝑎 𝑏 𝑥 𝑎𝑥 + 𝑏𝑦
𝑦 =
𝑐 𝑑 𝑐𝑥 + 𝑑𝑦

oThis is sufficient for scale, rotate, skew transformations.

oBut notice, we can’t add a constant, within the same format.

Dr. Faisal Bukhari, DDS, PU

Homogeneous system
oSolution is to use homogeneous coordinates for vectors

𝑎 𝑏 𝑐 𝑥 𝑎𝑥 + 𝑏𝑦 + 𝑐
𝑑 𝑒 𝑓 𝑦 = 𝑑𝑥 + 𝑒𝑦 + 𝑓
0 0 1 1 1

oNow we can rotate, scale, and skew like before, AND

translate (note how the multiplication works out, above)

Dr. Faisal Bukhari, DDS, PU

Translation
In matrix form
𝑥2 1 0 𝑡𝑥 𝑥1
𝑦2 = 0 1 𝑡𝑦 𝑦1
1 0 0 1 1
T
oWe could not have written T multiplicatively without using
homogeneous coordinates

oCompact way to write

𝐈𝟐×𝟐 𝐭
o𝒙′ = 𝒙
𝟎𝑻 1
Dr. Faisal Bukhari, DDS, PU
Basic 2D Transformations
cosθ −sinθ 0 Sx 0 0
sinθ cosθ 0 0 Sy 0
0 0 1 0 0 1

1 0 tx 1 ex 0 1 0 0
0 1 ty 0 1 0 ey 1 0
0 0 1 0 0 1 0 0 1

Dr. Faisal Bukhari, DDS, PU

Concatenation or Composition of
Transformations
oSuppose we first want to scale, then rotate
o 𝒙′ = S𝒙 ------------(1)
o𝒙′′ = R𝒙′ ------------(2)
Substitute value of 𝑥 ′ from (1) into (2), we get
= R(S𝒙)
= (RS) 𝒙 (Using associate property of matrices)
oSo two transformations can be represented by a single
transformation matrix
oM = RS
oImportant: read from right-side to get order of application
of transformations

Dr. Faisal Bukhari, DDS, PU

Order of Transformations
Example 1:
Scaling of x and y coordinates by 0.5
Sx 0 0 0.5 0 0
0 Sy 0 = 0 0.5 0
0 0 1 0 0 1
Rotation by 450 in counter clock direction
cosθ −sinθ 0 cos450 −sin450 0
sinθ cosθ 0 = sin450 cos450 0
0 0 1 0 0 1

Dr. Faisal Bukhari, DDS, PU

Order of Transformations

Scaled Rotated

Original

Rotated Scaled
Original
Dr. Faisal Bukhari, DDS, PU
Order of Transformations
Example 2: Scaling of x coordinates by 0.5
Sx 0 0 0.5 0 0
0 Sy 0= 0 1 0
0 0 1 0 0 1

Rotation by 450 in counter clock direction

cosθ −sinθ 0 cos450 −sin450 0

sinθ cosθ 0 = sin450 cos450 0
0 0 1 0 0 1

Dr. Faisal Bukhari, DDS, PU

Order of Transformations

Scaled Rotated

Original

Rotated Scaled
Original
Dr. Faisal Bukhari, DDS, PU