Taking derivative by convolution

Partial derivatives with convolution

For 2D function f(x,y), the partial derivative is:

∂f ( x, y ) f ( x + ε , y ) − f ( x, y )
= lim
∂x ε →0 ε
For discrete data, we can approximate using finite
differences:
∂f ( x, y ) f ( x + 1, y ) − f ( x, y )
≈
∂x 1
To implement above as convolution, what would be the
associated filter?

Source: K. Grauman
 Partial derivatives of an image

∂f ( x, y ) ∂f ( x, y )
∂x ∂y

-1 1
-1 1 or
1 -1
Which shows changes with respect to x?
Image gradient

The gradient of an image:

The gradient points in the direction of most rapid increase

in intensity
• How does this direction relate to the direction of the edge?

The gradient direction is given by

The edge strength is given by the gradient magnitude

Source: Steve Seitz

Image Gradient

∂f ( x, y ) ∂f ( x, y )
∂x ∂y
Effects of noise
Consider a single row or column of the image
• Plotting intensity as a function of position gives a signal

Where is the edge?

Source: S. Seitz
 Solution: smooth first

f*g

d
( f ∗ g)
dx

d
• To find edges, look for peaks in ( f ∗ g)
dx Source: S. Seitz
Derivative theorem of convolution

This saves us one operation:

Derivative of Gaussian filter

* [1 -1] =
Derivative of Gaussian filter

x-direction y-direction

Which one finds horizontal/vertical edges?

Example

input image (“Lena”)

 Compute Gradients (DoG)

X-Derivative of Y-Derivative of Gradient Magnitude

Gaussian Gaussian
Get Orientation at Each Pixel
Threshold at minimum level
Get orientation

theta = atan2(-gy, gx)

MATLAB demo

im = im2double(imread(filemane));
g = fspecial('gaussian',15,2);
imagesc(g);
surfl(g);
gim = conv2(im,g,'same');
imagesc(conv2(im,[-1 1],'same'));
imagesc(conv2(gim,[-1 1],'same'));
dx = conv2(g,[-1 1],'same');
Surfl(dx);
imagesc(conv2(im,dx,'same'));
Finite difference filters
Other approximations of derivative filters exist:

Source: K. Grauman
 Practical matters
What is the size of the output?
MATLAB: filter2(g, f, shape) or conv2(g,f,shape)
• shape = ‘full’: output size is sum of sizes of f and g
• shape = ‘same’: output size is same as f
• shape = ‘valid’: output size is difference of sizes of f and g

full same valid

g g
g g
g g

f f f

g g
g g
g g

Source: S. Lazebnik
 Practical matters
What about near the edge?
• the filter window falls off the edge of the image
• need to extrapolate
• methods:
– clip filter (black)
– wrap around
– copy edge
– reflect across edge

Source: S. Marschner
 Q?
Practical matters

• methods (MATLAB):
– clip filter (black): imfilter(f, g, 0)
– wrap around: imfilter(f, g, ‘circular’)
– copy edge: imfilter(f, g, ‘replicate’)
– reflect across edge: imfilter(f, g, ‘symmetric’)

Source: S. Marschner
 Review: Smoothing vs. derivative filters
Smoothing filters
• Gaussian: remove “high-frequency” components;
“low-pass” filter
• Can the values of a smoothing filter be negative?
• What should the values sum to?
– One: constant regions are not affected by the filter

Derivative filters
• Derivatives of Gaussian
• Can the values of a derivative filter be negative?
• What should the values sum to?
– Zero: no response in constant regions
• High absolute value at points of high contrast
Template matching
Goal: find in image

Main challenge: What is a

good similarity or
distance measure
between two patches?
• Correlation
• Zero-mean correlation
• Sum Square Difference
• Normalized Cross Correlation

Side by Derek Hoiem

Matching with filters
Goal: find in image
Method 0: filter the image with eye patch
h[ m, n] = ∑ g[ k , l ] f [ m + k , n + l ]
k ,l
f = image
g = filter

What went wrong?

Input Filtered Image Side by Derek Hoiem

Matching with filters
Goal: find in image
Method 1: filter the image with zero-mean eye
h[ m, n] = ∑ ( f [ k , l ] − f ) ( g[ m + k , n + l ] )
k ,l mean of f

True detections

False
detections

Input Filtered Image (scaled) Thresholded Image

Matching with filters
Goal: find in image
Method 2: SSD
h[ m, n] = ∑ ( g[ k , l ] − f [ m + k , n + l ] )2
k ,l

True detections

Input 1- sqrt(SSD) Thresholded Image

 Matching with filters

Can SSD be implemented with linear filters?

h[ m, n] = ∑ ( g[ k , l ] − f [ m + k , n + l ] )2
k ,l

Side by Derek Hoiem

Matching with filters
What’s the potential
Goal: find in image downside of SSD?

Method 2: SSD
h[ m, n] = ∑ ( g[ k , l ] − f [ m + k , n + l ] )2
k ,l

Input 1- sqrt(SSD)
Side by Derek Hoiem
 Matching with filters
Goal: find in image
Method 3: Normalized cross-correlation
mean template mean image patch

∑ ( g[k , l ] − g )( f [m + k , n + l ] − f m ,n )
h[ m, n] = k ,l
0.5
 2
 ∑ ( g[ k , l ] − g ) ∑ ( f [ m + k , n + l ] − f m,n ) 
2

 k ,l k ,l 

Side by Derek Hoiem

Matching with filters
Goal: find in image
Method 3: Normalized cross-correlation

True detections

Input Normalized X-Correlation Thresholded Image

Matching with filters
Goal: find in image
Method 3: Normalized cross-correlation

True detections

Input Normalized X-Correlation Thresholded Image

 Q: What is the best method to use?

A: Depends
Zero-mean filter: fastest but not a great
matcher
SSD: next fastest, sensitive to overall
intensity
Normalized cross-correlation: slowest,
invariant to local average intensity and
contrast

Side by Derek Hoiem

Denoising

Gaussian
Filter

Additive Gaussian Noise

 Reducing Gaussian noise

Smoothing with larger standard deviations suppresses noise,

but also blurs the image
Source: S. Lazebnik
Reducing salt-and-pepper noise by
Gaussian smoothing

3x3 5x5 7x7

Alternative idea: Median filtering
A median filter operates over a window by
selecting the median intensity in the window

• Is median filtering linear?

Source: K. Grauman
Median filter
What advantage does median filtering
have over Gaussian filtering?
• Robustness to outliers

Source: K. Grauman
Median filter
Salt-and-pepper Median filtered
noise

MATLAB: medfilt2(image, [h w])

Source: M. Hebert
 Median vs. Gaussian filtering
3x3 5x5 7x7

Gaussian

Median
EXTRA SLIDES
A Gentle Introduction
to Bilateral Filtering
and its Applications

“Fixing the Gaussian Blur”:

the Bilateral Filter

Sylvain Paris – MIT CSAIL

 Blur Comes from
Averaging across Edges

input
* output

*
*
Same Gaussian kernel everywhere.
 Bilateral Filter [Aurich 95, Smith 97, Tomasi 98]
No Averaging across Edges

input
* output

*
*
The kernel shape depends on the image content.
Bilateral Filter Definition:
an Additional Edge Term
Same idea: weighted average of pixels.
new
not new new

∑ Gσ (|| p − q ||) Gσ (| I − I q |) I q
1
BF [ I ]p = p
Wp q∈S
s r

normalization space weight range weight

factor
I
 Illustration a 1D Image

• 1D image = line of pixels

• Better visualized as a plot

pixel
intensity

pixel position
Gaussian Blur and Bilateral Filter
Gaussian blur
p

q GB [ I ]p = ∑ Gσ (|| p − q ||) I q
q∈S
space
space

Bilateral filter
[Aurich 95, Smith 97, Tomasi 98]
p

∑ Gσ (|| p − q ||) Gσ (| I − I q |) I q
1
range

BF [ I ]p = p
q Wp q∈S
s r

space range
normalization
space
q
Space and Range Parameters

∑ Gσ (|| p − q ||) Gσ (| I − I q |) I q
1
BF [ I ]p = p
Wp q∈S
s r

• space σs : spatial extent of the kernel, size of

the considered neighborhood.

• range σr : “minimum” amplitude of an edge

 Influence of Pixels
Only pixels close in space and in range are considered.
space

range

p
 Exploring the Parameter Space
σr = ∞
σr = 0.1 σr = 0.25 (Gaussian blur)

input
σs = 2

σs = 6

σs = 18
 Varying the Range Parameter
σr = ∞
σr = 0.1 σr = 0.25 (Gaussian blur)

input
σs = 2

σs = 6

σs = 18
input
σr = 0.1
σr = 0.25
 σr = ∞
(Gaussian blur)
 Varying the Space Parameter
σr = ∞
σr = 0.1 σr = 0.25 (Gaussian blur)

input
σs = 2

σs = 6

σs = 18
input
σs = 2
σs = 6
σs = 18