Lecture:

Pixels and Filters

Juan Carlos Niebles and Ranjay Krishna
Stanford Vision Lab

Stanford University Lecture 4- 1 26-Jan-18

 Announcements
• HW1 due Monday
• HW2 is out
• Class notes – Make sure to find the source and
cite the images you use.

Stanford University Lecture 4- 2 6-Oct-16

 What we will learn today?
• Image sampling and quantization
• Image histograms
• Images as functions
• Linear systems (filters)
• Convolution and correlation

Some background reading:

Forsyth and Ponce, Computer Vision, Chapter 7

Stanford University Lecture 4- 3 6-Oct-16

 What we will learn today?
• Image sampling and quantization
• Image histograms
• Images as functions
• Linear systems (filters)
• Convolution and correlation

Some background reading:

Forsyth and Ponce, Computer Vision, Chapter 7

Stanford University Lecture 4- 4 6-Oct-16

 Types of Images

Stanford University Lecture 4- 5 6-Oct-16

 Types of Images

Stanford University Lecture 4- 6 6-Oct-16

 Types of Images

Stanford University Lecture 4- 7 6-Oct-16

 Binary image representation

Slide credit: Ulas Bagci

Stanford University Lecture 4- 8 6-Oct-16
 Grayscale image representation

Slide credit: Ulas Bagci

Stanford University Lecture 4- 9 6-Oct-16
 Color Image - one channel

Slide credit: Ulas Bagci

Stanford University Lecture 4- 10 6-Oct-16
 Color image representation

Slide credit: Ulas Bagci

Stanford University Lecture 4- 11 6-Oct-16
 Images are sampled
What happens when we zoom into the images
we capture?

Stanford University Lecture 4- 12 6-Oct-16

 Errors due Sampling

Slide credit: Ulas Bagci

Stanford University Lecture 4- 13 6-Oct-16
 Resolution
is a sampling parameter, defined in dots per
inch (DPI) or equivalent measures of spatial
pixel density, and its standard value for recent
screen technologies is 72 dpi

Slide credit: Ulas Bagci

Stanford University Lecture 4- 14 6-Oct-16
 Images are Sampled and Quantized
• An image contains discrete number of pixels
– A simple example
75
– Pixel value:
• “grayscale”
(or “intensity”): [0,255]

231

148

Stanford University Lecture 4- 15 6-Oct-16

 Images are Sampled and Quantized
• An image contains discrete number of pixels
– A simple example
[90, 0, 53]
– Pixel value:
• “grayscale”
(or “intensity”): [0,255]
• “color”
– RGB: [R, G, B]
[249, 215, 203]
– Lab: [L, a, b]
– HSV: [H, S, V]
[213, 60, 67]

Stanford University Lecture 4- 16 6-Oct-16

With this loss of information (from
sampling and quantization),

Can we still use images for useful

tasks?

Stanford University Lecture 4- 17 6-Oct-16

 What we will learn today?
• Image sampling and quantization
• Image histograms
• Images as functions
• Linear systems (filters)
• Convolution and correlation

Some background reading:

Forsyth and Ponce, Computer Vision, Chapter 7

Stanford University Lecture 4- 18 6-Oct-16

 Histogram
• Histogram of an image provides the frequency
of the brightness (intensity) value in the
image.
def histogram(im):
h = np.zeros(255)
for row in im.shape[0]:
for col in im.shape[1]:
val = im[row, col]
h[val] += 1

Stanford University Lecture 4- 19 6-Oct-16

 Histogram
• Histogram captures the distribution of gray
levels in the image.
• How frequently each gray level occurs in the
image

Stanford University Lecture 4- 20 6-Oct-16

 Histogram

Slide credit: Dr. Mubarak Shah

Stanford University Lecture 4- 21 6-Oct-16
 Histogram – use case

Slide credit: Dr. Mubarak Shah

Stanford University Lecture 4- 22 6-Oct-16
 Histogram – another use case

Slide credit: Dr. Mubarak Shah

Stanford University Lecture 4- 23 6-Oct-16
 What we will learn today?
• Image sampling and quantization
• Image histograms
• Images as functions
• Linear systems (filters)
• Convolution and correlation

Some background reading:

Forsyth and Ponce, Computer Vision, Chapter 7

Stanford University Lecture 4- 24 6-Oct-16

 Images as discrete functions

• Images are usually digital (discrete):

– Sample the 2D space on a regular grid

• Represented as a matrix of integer values

pixel

Stanford University Lecture 4- 25 6-Oct-16

 Images as coordinates

Cartesian coordinates

Notation for
discrete
functions

Stanford University Lecture 4- 26 6-Oct-16

 Images as functions
• An Image as a function f from R2 to RM:
• f( x, y ) gives the intensity at position ( x, y )
• Defined over a rectangle, with a finite range:
f: [a,b] x [c,d ] à [0,255]
Domain range
support

Stanford University Lecture 4- 27 6-Oct-16

 Images as functions
• An Image as a function f from R2 to RM:
• f( x, y ) gives the intensity at position ( x, y )
• Defined over a rectangle, with a finite range:
f: [a,b] x [c,d ] à [0,255]
Domain range
support

é r ( x, y ) ù
• A color image: f ( x, y ) = êê g ( x, y ) úú
êë b( x, y ) úû

Stanford University Lecture 4- 28 6-Oct-16

 Histograms are a type of image
function

Stanford University Lecture 4- 29 6-Oct-16

 What we will learn today?
• Image sampling and quantization
• Image histograms
• Images as functions
• Linear systems (filters)
• Convolution and correlation

Some background reading:

Forsyth and Ponce, Computer Vision, Chapter 7

Stanford University Lecture 4- 30 6-Oct-16

 Systems and Filters
Filtering:
– Forming a new image whose pixel values are
transformed from original pixel values
Goals:
• Goal is to extract useful information from
images, or transform images into another
domain where we can modify/enhance image
properties
• Features (edges, corners, blobs…)
• super-resolution; in-painting; de-noising
Stanford University Lecture 4- 31 6-Oct-16
 System and Filters
• we define a system as a unit that converts an
input function f[n,m] into an output (or
response) function g[n,m], where (n,m) are
the independent variables.
– In the case for images, (n,m) represents the
spatial position in the image.

Stanford University Lecture 4- 32 6-Oct-16

 Super-resolution
De-noising

In-painting

Bertamio et al
Stanford University Lecture 4- 33 6-Oct-16
 Images as coordinates

Cartesian coordinates

Notation for
discrete
functions

Stanford University Lecture 4- 34 6-Oct-16

 2D discrete-space systems (filters)
S is the system operator, defined as a mapping or
assignment of a member of the set of possible
outputs g[n,m] to each member of the set of
possible inputs f[n,m].

Stanford University Lecture 4- 35 6-Oct-16

 Filter example #1: Moving Average
2D DS moving average over a 3 × 3 window of
neighborhood
h
1 1 1

1 1 1

1 1 1

Stanford University Lecture 4- 36 6-Oct-16

 Filter example #1: Moving Average

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0
0 0
0 0
0 90
90 0
0 90
90 90
90 90
90 0
0 0
0

0
0 0
0 0
0 90
90 90
90 90
90 90
90 90
90 0
0 0
0

0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0

Courtesy of S. Seitz
0
0 0
0 90
90 0
0 0
0 0
0 0
0 0
0 0
0 0
0

0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0

( f ∗ h)[m, n] = ∑ f [k, l] h[m − k, n − l]

k,l
Stanford University Lecture 4- 37 6-Oct-16
 Filter example #1: Moving Average

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

( f ∗ h)[m, n] = ∑ f [k, l] h[m − k, n − l]

k,l 38
Stanford University Lecture 4- 38 6-Oct-16
 Filter example #1: Moving Average

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

( f ∗ h)[m, n] = ∑ f [k, l] h[m − k, n − l]

k,l
Stanford University Lecture 4- 39 6-Oct-16
 Filter example #1: Moving Average

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

( f ∗ h)[m, n] = ∑ f [k, l] h[m − k, n − l]

k,l
Stanford University Lecture 4- 40 6-Oct-16
 Filter example #1: Moving Average

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

( f ∗ h)[m, n] = ∑ f [k, l] h[m − k, n − l]

k,l
Stanford University Lecture 4- 41 6-Oct-16
 Filter example #1: Moving Average

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10

0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

( f ∗ h)[m, n] = ∑ f [k, l] h[m − k, n − l]

k,l Source: S. Seitz

Stanford University Lecture 4- 42 6-Oct-16

 Filter example #1: Moving Average

In summary:
h[⋅ ,⋅ ]
• This filter “Replaces” each
pixel with an average of its 1 1 1
neighborhood. 1 1 1

1 1 1
• Achieve smoothing effect
(remove sharp features)

Stanford University Lecture 4- 43 6-Oct-16

 Filter example #1: Moving Average

Stanford University Lecture 4- 44 6-Oct-16

Filter example #2: Image Segmentation

• Image segmentation based on a simple

threshold:
255,

Stanford University Lecture 4- 45 6-Oct-16

 Properties of systems
• Amplitude properties:
– Additivity

Stanford University Lecture 4- 46 6-Oct-16

 Properties of systems
• Amplitude properties:
– Additivity

– Homogeneity

Stanford University Lecture 4- 47 6-Oct-16

 Properties of systems
• Amplitude properties:
– Additivity

– Homogeneity

– Superposition

Stanford University Lecture 4- 48 6-Oct-16

 Properties of systems
• Amplitude properties:
– Additivity

– Homogeneity

– Superposition

– Stability

Stanford University Lecture 4- 49 6-Oct-16

 Properties of systems
• Amplitude properties:
– Additivity

– Homogeneity

– Superposition

– Stability

– Invertibility

Stanford University Lecture 4- 50 6-Oct-16

 Properties of systems
• Spatial properties
– Causality

– Shift invariance:

Stanford University Lecture 4- 51 6-Oct-16

 Is the moving average system is shift invariant?

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10

0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Stanford University Lecture 4- 52 6-Oct-16

 Is the moving average system is shift invariant?

Yes!

Stanford University Lecture 4- 53 6-Oct-16

 Is the moving average system is casual?

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10

0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Stanford University Lecture 4- 54 6-Oct-16

 Linear Systems (filters)

• Linear filtering:
– Form a new image whose pixels are a weighted sum of
original pixel values
– Use the same set of weights at each point

• S is a linear system (function) iff it S satisfies

superposition property

Stanford University Lecture 4- 55 6-Oct-16

 Linear Systems (filters)

• Is the moving average a linear system?

• Is thresholding a linear system?

– f1[n,m] + f2[n,m] > T
– f1[n,m] < T No!
– f2[n,m]<T

Stanford University Lecture 4- 56 6-Oct-16

 2D impulse function
• 1 at [0,0].
• 0 everywhere else

Stanford University Lecture 4- 57 6-Oct-16

 LSI (linear shift invariant) systems
Impulse response

Stanford University Lecture 4- 58 6-Oct-16

 LSI (linear shift invariant) systems
Example: impulse response of the 3 by 3 moving
average filter:

h
1 1 1

1 1 1

1 1 1

Stanford University Lecture 4- 59 6-Oct-16

 Filter example #1: Moving Average
• 2D DS moving average over a 3 × 3 window of
neighborhood
h
1 1 1

1 1 1

1 1 1

1
( f * h)[m, n] = å f [k , l ] h[m - k , n - l ]
9 k ,l
Stanford University Lecture 4- 60 6-Oct-16
 LSI (linear shift invariant) systems
A simple LSI is one that shifts the pixels of an
image:
shifting property of the delta function

Stanford University Lecture 4- 61 6-Oct-16

 LSI (linear shift invariant) systems
A simple LSI is one that shifts the pixels of an
image:
shifting property of the delta function

Remember the superposition property:

superposition property

Stanford University Lecture 4- 62 6-Oct-16

 LSI (linear shift invariant) systems
With the superposition property, any LSI system
can be represented as a weighted sum of such
shifting systems:

Stanford University Lecture 4- 63 6-Oct-16

 LSI (linear shift invariant) systems
Rewriting the above summation:

Stanford University Lecture 4- 64 6-Oct-16

 LSI (linear shift invariant) systems
We define the filter of a LSI as:

f [n, m] ⇤ h[n, m]

Stanford University Lecture 4- 65 6-Oct-16

 What we will learn today?
• Images as functions
• Linear systems (filters)
• Convolution and correlation

Some background reading:

Forsyth and Ponce, Computer Vision, Chapter 7

Stanford University Lecture 4- 66 6-Oct-16

 1D Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.

h[k,l] f[k,l]

Stanford University Lecture 4- 67 6-Oct-16

 1D Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.

We first need to calculate h[n-k, m-l]

h[-k]

h[k] h[n-k]

Stanford University Lecture 4- 68 6-Oct-16

 Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.

h[-2-k]

f[k] g[-2]

Stanford University Lecture 4- 69 6-Oct-16

 Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.

h[-1-k]

f[k] g[-1]

Stanford University Lecture 4- 70 6-Oct-16

 Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.

h[-k]

f[k] g[0]

Stanford University Lecture 4- 71 6-Oct-16

 Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.

h[1-k]

f[k] g[1]

Stanford University Lecture 4- 72 6-Oct-16

 Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.

h[2-k]

f[k] g[2]

Stanford University Lecture 4- 73 6-Oct-16

 Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.

h[n-k]

f[k] g[n]

Stanford University Lecture 4- 74 6-Oct-16

 Discrete convolution (symbol: )
In summary, the steps for discrete convolution are:
• Fold h[k,l] about origin to form h[−k]
• Shift the folded results by n to form h[n − k]
• Multiply h[n − k] by f[k]
• Sum over all k
• Repeat for every n

n
Stanford University Lecture 4- 75 6-Oct-16
 2D convolution
2D convolution is very similar to 1D.
• The main difference is that we now have to iterate over 2 axis instead of 1.

f [n, m] ⇤ h[n, m]

Assume we have a filter(h[,]) that

is 3x3. and an image (f[,]) that is
7x7.

n
Stanford University Lecture 4- 76 6-Oct-16
 2D convolution
2D convolution is very similar to 1D.
• The main difference is that we now have to iterate over 2 axis instead of 1.

f [n, m] ⇤ h[n, m]

Assume we have a filter(h[,]) that

is 3x3. and an image (f[,]) that is
7x7.

n
Stanford University Lecture 4- 77 6-Oct-16
 2D convolution
2D convolution is very similar to 1D.
• The main difference is that we now have to iterate over 2 axis instead of 1.

f [n, m] ⇤ h[n, m]

Assume we have a filter(h[,]) that

is 3x3. and an image (f[,]) that is
7x7.

n
Stanford University Lecture 4- 78 6-Oct-16
 2D convolution
2D convolution is very similar to 1D.
• The main difference is that we now have to iterate over 2 axis instead of 1.

f [n, m] ⇤ h[n, m]

Assume we have a filter(h[,]) that

is 3x3. and an image (f[,]) that is
7x7.

n
Stanford University Lecture 4- 79 6-Oct-16
 2D convolution
2D convolution is very similar to 1D.
• The main difference is that we now have to iterate over 2 axis instead of 1.

f [n, m] ⇤ h[n, m]

Assume we have a filter(h[,]) that

is 3x3. and an image (f[,]) that is
7x7.

n
Stanford University Lecture 4- 80 6-Oct-16
 2D convolution
2D convolution is very similar to 1D.
• The main difference is that we now have to iterate over 2 axis instead of 1.

f [n, m] ⇤ h[n, m]

Assume we have a filter(h[,]) that

is 3x3. and an image (f[,]) that is
7x7.

Stanford University Lecture 4- 81 6-Oct-16

 LSI (linear shift invariant) systems
An LSI system is completely specified by its
impulse response.
shifting property of the delta function

superposition

Discrete convolution

f [n, m] ⇤ h[n, m]
Stanford University Lecture 4- 82 6-Oct-16
 2D convolution example

Slide credit: Song Ho Ahn

Stanford University Lecture 4- 83 6-Oct-16

 2D convolution example

Slide credit: Song Ho Ahn

Stanford University Lecture 4- 84 6-Oct-16

 2D convolution example

Slide credit: Song Ho Ahn

Stanford University Lecture 4- 85 6-Oct-16

 2D convolution example

Slide credit: Song Ho Ahn

Stanford University Lecture 4- 86 6-Oct-16

 2D convolution example

Slide credit: Song Ho Ahn

Stanford University Lecture 4- 87 6-Oct-16

 2D convolution example

Slide credit: Song Ho Ahn

Stanford University Lecture 4- 88 6-Oct-16

 2D convolution example

Slide credit: Song Ho Ahn

Stanford University Lecture 4- 89 6-Oct-16

 Convolution in 2D - examples

•0 •0 •0

* •0 •1 •0
= ?
•0 •0 •0

Original

Courtesy of D Lowe
Stanford University Lecture 4- 90 6-Oct-16
 Convolution in 2D - examples

•0 •0 •0

* •0 •1 •0
•0 •0 •0
=
Original Filtered

Courtesy of D Lowe
(no change)

Stanford University Lecture 4- 91 6-Oct-16

 Convolution in 2D - examples

•0 •0 •0

* •0 •0 •1
= ?
•0 •0 •0

Original

Courtesy of D Lowe
Stanford University Lecture 4- 92 6-Oct-16
 Convolution in 2D - examples

•0 •0 •0

* •0 •0 •1
•0 •0 •0
=
Original Shifted right

Courtesy of D Lowe
By 1 pixel

Stanford University Lecture 4- 93 6-Oct-16

 Convolution in 2D - examples

•1 •1 •1

* •1 •1 •1
•1 •1 •1
= ?
Original

Courtesy of D Lowe
Stanford University Lecture 4- 94 6-Oct-16
 Convolution in 2D - examples

•1 •1 •1

* •1 •1 •1
•1 •1 •1
=
Original Blur (with a

Courtesy of D Lowe
box filter)

Stanford University Lecture 4- 95 6-Oct-16

 Convolution in 2D - examples

•0 •0 •0 •1 •1 •1
•0 •2 •0
•0 •0 •0
- •1 •1 •1
•1 •1 •1
=?
(Note that filter sums to 1)
Original
“details of the image”

•0 •0 •0 •0 •0 •0 •1 •1 •1

Courtesy of D Lowe
•0 •1 •0
•0 •0 •0
+ •0 •1 •0
•0 •0 •0
- •1 •1 •1
•1 •1 •1

Stanford University Lecture 4- 96 6-Oct-16

• What does blurring take away?

– =

original smoothed (5x5) detail

• Let’s add it back:

+a =

original detail sharpened

Stanford University Lecture 4- 97 6-Oct-16

 Convolution in 2D –
Sharpening filter

•0 •0 •0 •1 •1 •1
•0 •2 •0
•0 •0 •0
- •1 •1 •1
•1 •1 •1 =
Original

Courtesy of D Lowe
Sharpening filter: Accentuates differences with local average

Stanford University Lecture 4- 98 6-Oct-16

 Image support and edge effect
•A computer will only convolve finite support
signals.
• That is: images that are zero for n,m outside some
rectangular region
• numpy’s convolution performs 2D DS convolution of
finite-support signals.

* = (N1 + N2 − 1) × (M1 +M2 − 1)

N2 ×M2

N1 ×M1

Stanford University Lecture 4- 99 6-Oct-16

 Image support and edge effect
•A computer will only convolve finite support
signals.
• What happens at the edge?

• zero “padding”
f • edge value replication
h • mirror extension
• more (beyond the scope of this class)

-> Matlab conv2 uses

zero-padding
Stanford University Lecture 4- 100 6-Oct-16
Slide credit: Wolfram Alpha

Stanford University Lecture 4- 101 6-Oct-16

 What we will learn today?
• Image sampling and quantization
• Image histograms
• Images as functions
• Linear systems (filters)
• Convolution and correlation

Some background reading:

Forsyth and Ponce, Computer Vision, Chapter 7

Stanford University Lecture 4- 102 6-Oct-16

 (Cross) correlation (symbol: )
Cross correlation of two 2D signals f[n,m] and g[n,m]

(k, l) is called the lag

• Equivalent to a convolution without the flip

⇤
rf g [n, m] = f [n, m] ⇤ g [ n, m]
(g* is defined as the complex conjugate of g. In this class, g(n,m) are real numbers, hence g*=g.)

Stanford University Lecture 4- 103 6-Oct-16

 (Cross) correlation – example

Courtesy of J. Fessler
Stanford University Lecture 4- 104 6-Oct-16
 (Cross) correlation – example

Courtesy of J. Fessler
Stanford University Lecture 4- 105 6-Oct-16
 (Cross) correlation – example

Courtesy of J. Fessler
numpy’s
correlate
Stanford University Lecture 4- 106 6-Oct-16
 (Cross) correlation – example
Left Right

scanline

Norm. cross corr. score

Stanford University Lecture 4- 107 6-Oct-16

Convolution vs. (Cross) Correlation

f*h f**h

Stanford University Lecture 4- 108 6-Oct-16

 Cross Correlation Application:
Vision system for TV remote
control
- uses template matching

Figure from “Computer Vision for Interactive Computer Graphics,” W.Freeman et al, IEEE Computer Graphics and Applications,
1998 copyright 1998, IEEE

Stanford University Lecture 4- 109 6-Oct-16

 properties
• Associative property:

• Distributive property:

The order doesn’t matter!

Stanford University Lecture 4- 110 6-Oct-16
 properties

• Shift property:

• Shift-invariance:

Stanford University Lecture 4- 111 6-Oct-16

Convolution vs. (Cross) Correlation
• A convolution is an integral that expresses the amount
of overlap of one function as it is shifted over another
function.
– convolution is a filtering operation

• Correlation compares the similarity of two sets of

data. Correlation computes a measure of similarity of
two input signals as they are shifted by one another.
The correlation result reaches a maximum at the time
when the two signals match best .
– correlation is a measure of relatedness of two signals

Stanford University Lecture 4- 112 6-Oct-16

 What we have learned today?
• Image sampling and quantization
• Image histograms
• Images as functions
• Linear systems (filters)
• Convolution and correlation

Stanford University Lecture 4- 113 6-Oct-16