6/10/2020

Using Edge Representation…

• Stereo matching problem
• Input:
– Two images with disparity
Line Detection – Camera Calibration information
• Computation
– Find corresponding features in two images
• Output
– Disparity in corresponding features is related to depth
• Edges and corners help in finding correspondences

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 1 LUMS Slides, Material from Ohio State 2
University ,USA University ,USA

Finding Shapes from Edges Finding Shapes from Edges

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 3 LUMS Slides, Material from Ohio State 4
University ,USA University ,USA

1
 6/10/2020

Edge Representation for Shape Analysis Problems in Finding Lines

• What about noisy edges?
Missing Data Noisy Data Multiple Lines

• Problem Def.: Find straight lines…

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
Images from: http://www.cogs.susx.ac.uk/users/davidy/teachvision/vision4.html
LUMS Slides, Material from Ohio State 5 LUMS Slides, Material from Ohio State 6
University ,USA University ,USA

Parameter Optimization: Line Fitting: Least Squared Error Solution

Least Squared Error Solutions • Step 1: Identify the model
• Fitting a line to a set of data points… – Equation of line: y = mx + c
• Step 2: Set up an error term which will give the
x y
1.3 5.7 goodness of every point with respect to the
2.4 7.3 (unknown) model
3.4 10.5
4.6 11.8 – Error induced by ith point:
5.3 13.9
6.6 16.3 – ei = mxi + c - yi
6.4 15.3 – Error for whole data: E = i ei2
8.0 17.9
8.9 20.8 – E = i (mxi + c – yi)2
9.2 20.9
• Step 3: Differentiate Error w.r.t. parameters, put
• Equation of best fit line ? equal to zero and solve for minimum point
Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 7 LUMS Slides, Material from Ohio State 8
University ,USA University ,USA

2
 6/10/2020

Line Fitting: Least Squared Error Line Fitting: Least Squared Error
Solution x y Solution
1.3 5.7
E   mxi  c  yi 
2

i
2.4 7.3
3.4 10.5
E 4.6 11.8
  mxi  c  yi xi  0 5.3 13.9
m i 6.6 16.3
E 6.4 15.3
  mxi  c  yi   0 8.0 17.9
c i
8.9 20.8
  xi2  x  m  x y  9.2 20.9
i i i
 i i
 i
  xi
 i
 1   c    y 
i i
i
380.63 56.1 m 914.68
= 140.4
56.1 10 c

Solution: m = 1.9274 c = 3.227

Solution: y = mx + c,
y=1.9274 x + 3.227 Syed Farooq Ali, Fall 2012, Reference:
Syed Farooq Ali, Fall 2012, Reference: LUMS Slides, Material from Ohio State University ,USA 9 LUMS Slides, Material from Ohio State 10
University ,USA

Least Squared Error Solution Finding Lines

• Disadvantages? • Problem Definition:
– Multiple Lines… • Given a binary image, find all significant lines
– Not robust to noise • Line: y = mx + c
• Estimate m,c parameters of all significant lines
• Example in presence of noise

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 11 LUMS Slides, Material from Ohio State 12
University ,USA University ,USA

3
 6/10/2020

Hough Transform Hough Transform for Lines

• Method to find any type of shape that can be • General Idea:

– Search for the best possible m and c
represented in parametric form parameters given the data
• E.g. lines, circles, parabolas, ellipses… • Consider all possible lines in the image
• Generalized Hough Transform • What is the most significant line?
– For arbitrary shapes • A line passing through most of the points
• A line that passes through 1 point gets one vote
• Find the line that gets most votes

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 13 LUMS Slides, Material from Ohio State 14
University ,USA University ,USA

Hough Transform for Lines Hough Transform for Lines

• Aim: Create a mechanism for voting
– A line should get as many votes as the points it passes x=2
y
through y=4
y=x+2
• Equation of line is y=mx+c
y=-x+6
– m is slope, c is intercept
(2,4)
y=2x
• Consider only one point (x,y) y=-2x+8
– For example (2,4)
And so on… (infinite lines)
• How many lines can pass through this point?

x
Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 15 LUMS Slides, Material from Ohio State 16
University ,USA University ,USA

4
 6/10/2020

Hough Transform for Lines Hough Transform for Lines

• Can we write the general expression for all the
lines passing through (2,4)? y y=4 m=0, c = 4
• All those lines will have a specific relationship y=x+2 m=1, c = 2
between m and c y=-x+6 m=-1, c = 6
• Any arbitrary combination of m and c will not (2,4) y=2x m=2, c = 0
y=-2x+8 m=-2, c = 8
pass through the given point; only certain
combinations will work Plot the m, c points in m-c
space
x
Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 17 LUMS Slides, Material from Ohio State 18
University ,USA University ,USA

Hough Transform for Lines Hough Transform for Lines

• Equation of line is y=mx+c
c
• We are given (x,y) [e.g. (2,4)]
y
• (m,c) are the unknowns
• Can be rewritten as c = (-x)m + y
4 • Consider (x,y) space: y=mx+c represents a line
(2,4)
• Consider transformed space (m,c), then
c=(-x)m + y is a line in this space
• (-x) is gradient, y is the intercept

x 2 m
Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 19 LUMS Slides, Material from Ohio State 20
University ,USA University ,USA

5
 6/10/2020

Interpretation • Aim: In x-y space, find a line passing through

most of the points
• Line in (m,c) space represents all possible lines
• i.e., What is the equal statement in (m-c)
that could pass through a single point (x,y)
space?
• Point in (x,y) space is a line in (m,c) space
• In m-c space, find a point from which most of
• Point in (m,c) space is a … the lines are passing through? (i.e., the point
• Line in (x,y) space in which most of the lines interest)

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 21 LUMS Slides, Material from Ohio State 22
University ,USA University ,USA

Finding Lines using Hough Transform – CONT.

Finding Lines using Hough Transform

y c

Solution: Where all

lines intersect…
4
(2,4)

m = 2, c = 0

x 2 m
Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 23 Syed Farooq Ali, Fall 2012, Reference: LUMS Slides, Material from Ohio State University ,USA 24
University ,USA

6
 6/10/2020

Hough Transform for Lines Algorithm

• Initialize Accumulator array, A, of two 1. Quantize parameter space
dimensions A[cmin, … , cmax, mmin, … , mmax]
(m, c) 2. For each edge point (x,y)
• For each point (x,y) in image, increment cells For (m = mmin, m  mmax, m++)
c = (-x)m + y
along line
A [c,m] = A [c,m] + 1;
c = -xm+y by 1
3. Find local maxima in A
• Find maximum point in accumulator array for
solution
Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 25 LUMS Slides, Material from Ohio State 26
University ,USA University ,USA

Hough Transform for Lines Alternate Line parameterization

• Problems with this procedure?
• What about the range of slope? • p = x cos + y sin y

• m spans - to  • p is the perpendicular

to the line
• Solution?
•  is the angle p makes
• Use alternate parameterization of line
with the x-axis
p

x
Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 27 LUMS Slides, Material from Ohio State 28
University ,USA University ,USA

7
 6/10/2020

Algorithm (polar form) HT for Lines (polar form)

1. Quantize parameter space • Point is (x,y) space represents ______ in the
A [min, … ,  max, pmin, … , pmax] parameter space (p, )?
2. For each edge point (x,y) • Answer: Sinusoid curve
For ( = min,   max, ++)
p = x cos + y sin
A [, p] = A [, p] + 1;

3. Find local maxima in A

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 29 LUMS Slides, Material from Ohio State 30
University ,USA University ,USA

HT for Lines (polar form) HT for Lines (polar form)

• Line in (x,y) space represents _______ in (p, )
space? p
Each line
appears as
two peaks

 (0-359)

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 31 LUMS Slides, Material from Ohio State 32
University ,USA University ,USA

8
 6/10/2020

Additional advantage of Polar Form Algorithm (polar form/improved)

• Line which passes through (x, y) was assumed 1. Quantize parameter space
to have all possible values of  A [min, … ,  max, pmin, … , pmax]
• Gradient direction? 2. For each edge point (x,y)
•  can be computed from gradient direction Compute  from gradient direction
p = x cos + y sin
A [, p] = A [, p] + 1;
3. Find local maxima in A

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 33 LUMS Slides, Material from Ohio State 34
University ,USA University ,USA

HT for Lines Finding Lines

• What about multiple lines • Detect peaks in the accumulator array
in an image? • Threshold – or more complicated peak
finding function

T = 120

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 35 LUMS Slides, Material from Ohio State 36
University ,USA University ,USA

9
 6/10/2020

Finding Lines Hough Transform for Circles

• Equation…
• Centered at (x0, y0) with radius r

( x  x0 ) 2  ( y  y0 ) 2  r 2
• Three unknowns… x0 y0 r
• Three dimensional parameter space
• Conceptually… ?

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 37 LUMS Slides, Material from Ohio State 38
University ,USA University ,USA

How to simplify this algorithm Hough transform

• Use of gradient direction,  • Can be applied to any parametric representation
f (x, a)=0
x0  x  r cos  • Initialize accumulator array, A to zeros…
• A is |a| dimensional
y0  y  r sin  • For each pixel x, and each a such that f(x,a)=0, A[a] =
A[a]+1
• Algorithm?... • Local maxima of A corresponds to curves f in image.

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 39 LUMS Slides, Material from Ohio State 40
University ,USA University ,USA

10
 6/10/2020

Finding more than one curve Hough Transform

• Given parametric representation of a curve
• Parameter space will have multiple maxima – LINE: p = x cos + y sin
• Threshold – CIRCLE: x0 = x – r cos
• Or use better methods to find maximum y0 = y – r sin
points – ELLIPSE: x0 = x – a cos
y0 = y – b sin
– GENERAL: f(x, a) = 0

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 41 LUMS Slides, Material from Ohio State 42
University ,USA University ,USA

Hough Transform
• Initialize A (accumulator array) to all zeros
• A is |a| dimensional
• For each pixel x, and each a such that f(x,a)=0, Generalized Hough Transform
A[a] = A[a]+1
• Local maxima of A corresponds to curves f in To find arbitrary shapes in images
image. Shapes which do not have an easy
parametric representation

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 43 LUMS Slides, Material from Ohio State 44
University ,USA University ,USA

11
 6/10/2020

Centroid and Area Example

• The average location of all 1 1 2 3 4 5 6 7 8 9 10 11 12 A = 16

pixels in a region R
r r
A ( r ,c)R
1 0 0 0 0 0 0 0 0 0 0 0 0
1 r = 1/16 * (2+2+2+2+3+4+5+6
c c
A ( r ,c)R
2 0 0 0 0 1 1 1 1 0 0 0 0
+7+7+7+7+6+5+4+3)
3 0 0 0 1 0 0 0 0 1 0 0 0
4 0 0 1 0 0 0 0 0 0 1 0 0 = 4.5
5 0 0 1 0 0 0 0 0 0 1 0 0
c = 1/16 * (5+6+7+8+9+10+10+
6 0 0 0 1 0 0 0 0 1 0 0 0 9+8+7+6+5+4+3+3+4)
• where A is the area of
7 0 0 0 0 1 1 1 1 0 0 0 0
region R A 1
( r , c )R
8 0 0 0 0 0 0 0 0 0 0 0 0
= 6.5

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 45 LUMS Slides, Material from Ohio State 46
University ,USA University ,USA

Generalized Hough Transform GHT - Training

• Training • Given the shape of interest
– A representation of shape of interest is built in the
form of an R-Table
• Detection
– Using R-Table, a given shape is matched to the
shape of interest
• Find Centroid (xc , yc) of shape

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 47 LUMS Slides, Material from Ohio State 48
University ,USA University ,USA

12
 6/10/2020

GHT - Training
GHT - Training
• Find r = (x’, y’) for each • R-Table is indexed by 
edge point
• xc = x + x’
• yc = y + y’
(xc, yc)

r y’

(x, y) x’

•  is the angle tangent at 

(x,y) makes with x-axis

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 49 LUMS Slides, Material from Ohio State 50
University ,USA University ,USA

Example - Training Detection

• Go to each (x,y) in image
=0 • Find φ
=45 • For corresponding entry in R Table
=90
• Find all possible locations of centriods
=135

=180
• xc = x + x’
=225 • yc = y + y’
=270 • Increment centroid accumulator by 1
=315

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 51 LUMS Slides, Material from Ohio State 52
University ,USA University ,USA

13
 6/10/2020

Detection GHT - Algorithm

• Quantize parameter space
=0 P [xcmin, … , xcmax, ycmin, … , ycmax]
=45 • For each edge point (x,y)
=90
– Compute φ from gradient direction
=135
– For each table entry in row φ
=180
xc = x + x’
=225 yc = y + y’
=270 – P [xc, yc] = P [xc, yc] + 1;
=315
• Find local maxima in P
Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 53 LUMS Slides, Material from Ohio State 54
University ,USA University ,USA

Rotation and Scaling Invariance

GHT - Questions
(Not in current course)
• Rotation invariance…
• Uniqueness of R-Table? x’’ = x’ cos + y’ sin
y’’ = -x’ sin + y’ cos
• Invariance to translation? • Rotation + Scaling invariance
• Invariance to rotation? x’’ = sx (x’ cos + y’ sin)
y’’ = sy (-x’ sin + y’ cos)
• Invariance to scaling? • Substitute in
xc = x + x’
yc = y + y’
• To get
xc = x + sx (x’ cos + y’ sin)
yc = y + sy (-x’ sin + y’ cos)
• Substitute these values of (xc, yc) in GHT algorithm

Syed Farooq Ali, Fall 2012, Reference: Syed Farooq Ali, Fall 2012, Reference:
LUMS Slides, Material from Ohio State 55 LUMS Slides, Material from Ohio State 56
University ,USA University ,USA

14