Image Filtering

CS485/685 Computer Vision

Prof. George Bebis
 What is image filtering?
f(x,y) g(x,y)

filtering

filtering
 Image Filtering Methods

• Spatial Domain

• Frequency Domain (i.e., uses Fourier Transform)

 Spatial Domain Methods
f(x,y) g(x,y)

f(x,y) g(x,y)
 Point Processing Methods
• Convert a given pixel value to a new pixel value based on some
predefined function.
Point Processing Methods - Examples
Negative Contrast stretching

Thresholding

Histogram
Equalization
 Area Processing Methods
• Need to define:
(1) Area shape and size
(2) Operation

output image
 Area Shape and Size

•
•Area shape is typically defined using a
rectangular mask.

• Area size is determined by mask

size.
e.g., 3x3 or 5x5

• Mask size is an important parameter!

 Operation
• Typically linear combinations of pixel
values.
– e.g., weight pixel values and add them
together.
• Different results can be obtained using
different weights.
– e.g., smoothing, sharpening, edge
detection).

mask
 Example

10 5 3 0 0 0
4 6 1 0 0.5 0 8
1 1 8 0 1 0.5
Local image mask Modified image data
neighborhood
 Common Linear Operations

• Correlation

• Convolution
 Correlation
• A filtered image is generated as the center of the mask
visits every pixel in the input image.
n x n mask
h(i,j)

g(i,j)

f(i,j)

filtered
image
Handling Pixels Close to Boundaries

pad with zeroes wrap around

0 0 0 ……………………….0
0 0 0 ……………………….0

or
Correlation – Example
Geometric Interpretation of Correlation
• Suppose x and y are two n-dimensional vectors:

x  ( x1 , x2 ,..., xn ) y  ( y1 , y2 ,..., yn )
• The dot product of x with y is defined as:

x. y  x1 y1  x2 y2  ...  xn yn x

using vector
notation: x . y | x || y |cos( ) θ
y
• Correlation generalizes the notion of dot product
Geometric Interpretation of Correlation (cont’d)
cos(θ) measures the similarity between x and y

xy
x . y | x || y |cos( ) or cos( ) 
| x || y |

Normalized correlation (i.e., divide by lengths)

n n
2 2

  h(k , l ) f (i  k , j  l )
n n
k  l 
N (i, j )  n n
2 2
n n
2 2 2 2
[  h 2 ( k , l )]1/ 2 [   f 2 (i  k , j  l )]1/ 2
n n n n
k  l  k  l 
2 2 2 2
 Normalized Correlation
• Measure the similarity between images or parts of images.

mask

=
Application: TV Remote Control
Application : TV Remote Control (cont’d)
Application : TV Remote Control (cont’d)
Application : TV Remote Control (cont’d)
Application : TV Remote Control (cont’d)
Application : TV Remote Control (cont’d)
 Normalized Correlation (cont’d)
• Traditional correlation cannot handle changes due to:

• size
• orientation
• shape (e.g., deformable objects).

?
 Convolution
• Same as correlation except that the mask is flipped, both
horizontally and vertically.

1 2 3 7 8 9 V 9 8 7
H
4 5 6 4 5 6 6 5 4
7 8 9 1 2 3 3 2 1

n n n n
2 2 2 2
g (i, j )    h(k , l ) f (i  k , j  l )    h(i  k , j  l ) f (k , l )
n n n n
k  l  k  l 
2 2 2 2

For symmetric masks (i.e., h(i,j)=h(-i,-j)), Notation:

convolution is equivalent to correlation! h*f=f*h
 Correlation/Convolution Examples

Correlation:

Convolution:
 How do we choose the mask weights?
• Depends on the application.
• Usually by sampling certain functions and their derivatives.

1st derivative 2nd derivative

Gaussian of Gaussian of Gaussian

Good for Good for

image smoothing image sharpening
 Normalization of Mask Weights

• Sum of weights affects overall intensity of output image.

• Positive weights
– Normalize them such that they sum to one.
one
• Both positive and negative weights
– Should sum to zero (but not always)

1/9 1/16
 Smoothing Using Averaging
• Idea: replace each pixel by the average of its neighbors.

• Useful for reducing noise and unimportant details.

•The size of the mask controls the amount of smoothing.

 Smoothing Using Averaging (cont’d)
• Trade-off: noise vs blurring and loss of detail.
original 3x3 5x5 7x7

15x15 25x25
 Gaussian Smoothing
• Idea: replace each pixel by a weighted average of its
neighbors
• Mask weights are computed by sampling a Gaussian
function

Note: weight
values decrease
with distance
from mask
center!
 Gaussian Smoothing (cont’d)
mask size depends on σ :
• σ determines the degree of smoothing!
σ=3
 Gaussian Smoothing (cont’d)
Gaussian(sigma, hSize, h)
( x )2
float sigma, *h; 1 
int hSize; p( x)  e 2 2
{  2
int i;
float cst, tssq, x, sum;
halfSize=(int)(2.5*sigma);
cst = 1./(sigma*sqrt(2.0*PI)) ; hSize=2*halfSize;
tssq = 1./(2*sigma*sigma) ; if (hSize % 2 == 0) ++hSize; // odd size

for(i=0; i<hSize; i++) {

x=(float)(i-hSize/2);
h[i]=(cst*exp(-(x*x*tssq))) ;
}
// normalize
sum=0.0;
for(i=0;i<hSize;i++)
sum += h[i];
for(i=0;i<hSize;i++)
h[i] /= sum;
}
Gaussian Smoothing - Example

= 1 pixel = 5 pixels = 10 pixels = 30 pixels

Averaging vs Gaussian Smoothing

Averaging

Gaussian
 Properties of Gaussian

• Convolution with self is another Gaussian

• Special case: convolving two times with Gaussian kernel

of width is equivalent to convolving once with kernel of
width

* =
 Properties of Gaussian (cont’d)
• Separable kernel: a 2D Gaussian can be expressed as the
product of two 1D Gaussians.
 Properties of Gaussian (cont’d)

• 2D Gaussian convolution can be implemented more

efficiently using 1D convolutions:
 Properties of Gaussian (cont’d)

row get a new image Ir

Convolve each column of Ir with g
 Example
2D convolution
(center location only) O(n2)

The filter factors

into a product of 1D
filters:

Perform convolution
along rows: * =
O(2n)=O(n)
Followed by convolution
along the remaining column: * =
 Image Sharpening
• Idea: compute intensity differences in local image regions.
• Useful for emphasizing transitions in intensity (e.g., in
edge detection).

1st derivative
of Gaussian
Example