Image restoration

Adaptive local noise reduction filter

 Filter operation is not uniform at all pixel locations but depends on

the local characteristics (local mean, local variance) of the observed
image.

 The adaptive filter is given by:

2
fˆ(m, n)  g(m, n)  g(m, n)  m L (m, n)
 L (m, n)
2

 Usually, we need to be careful about the possibility of 2 (m, n)  2 ,

L 
in which case, we could potentially get a negative output gray value.
 This filter does the following:
o If 2  0 (or is small), the filter simply returns the value of
g(m,n).
o If the local variance 2L (m, n) is high relative to the noise
variance 2 , the filter returns a value close to g(m,n). This
usually corresponds to a location associated with edges in the
image.
o If the two variances are roughly equal, the filter does a simple
averaging over window Sab .
 Example

Adaptive median filtering

Periodic Interference/Noise

 Periodic noise or interference occurs in images due to electrical or

electromechanical interference during image acquisition.
 It is an example of spatially dependent noise.
 This type of noise can be very effectively removed using frequency
domain filtering. Recall that the spectrum of a pure sinusoid would be
a simple impulse at the appropriate frequency location.
 Bandreject filters
 Bandreject filters remove (or attenuate) a band of frequencies, around
some frequency, say D0 .
 An ideal bandreject filter is given by:
1 if D(u, v)  D0  W2
H (u, v)  0 if D0  W2  D(u, v)  D0  W2
1 D(u, v)  D0  W2
where D(u, v)  u 2  v2 .

 W is usually referred to the width of the (stop) band and D0 as the

center frequency.

 A Butterworth bandreject filter of order n is given by

1
H (u, v)  2n
.
D(u, v)W
1 2
D (u, v)  D20

 A Gaussian bandreject filter is given by 2

D 2 (u,v)  D 2
 12 0

H (u, v)  1  e
D(u,v)W
 Example

 Bandreject filters are ideally suited for filtering out periodic

interference.
 Recall that the Fourier transform of a pure sine or cosine function is
just a pair of impulses.
 Therefore the interference is “localized” in the spectral domain and
one can easily identify this region and filter it out.
 Bandpass filters
 Bandpass filters are the exact opposite of bandreject filters. They pass
a band of frequencies, around some frequency, say D0 (rejecting the
rest).
 One can write:
Hbp (u, v)  1  Hbr (u, v)
 Bandpass filter is usually used to isolate components of an image that
correspond to a band of frequencies.
 It can also be used to isolate noise interference, so that more detailed
analysis of the interference can be performed, independent of the
image.
 Notch filter

 It is a kind of bandreject/bandpass filter that rejects/passes a very

narrow set of frequencies, around a center frequency.
 Due to symmetry considerations, the notches must occur in
symmetric pairs about the origin of the frequency plane.
 The transfer function of an ideal notch-reject filter of radius D0 with
center frequency u0 , v0  is given by
0 if D (u, v)  D or D (u, v)  D
H (u, v)  1 0 2 0

1 otherwise
u  M / 2  u   v  N / 2  v
where
D1 (u, v) 

0
2
0.5
0 2

u  M / 2  u   v  N / 2  v
and
D2 (u, v) 

0
2
0.5
0 2

 The transfer function of a Butterworth notch reject filter of order n is

given by
1
H (u, v)  n
.
2
D0
1
D1(u, v)D2 (u, v)

 A Gaussian notch reject filter is given by

2
D1 (u ,v) D2 (u ,v)
 12 2

H (u, v)  1  e D0

 A notch pass filter can be obtained from a notch reject filter using:
Hnp (u, v)  1  Hnr (u, v)
Illustration of transfer function of notch filters
 Example
 Image corrupted by periodic horizontal scan lines.
 Optimum Notch Filtering

 When interference patterns are more complicated, the preceding

filters tend to reject more image information in an attempt to filter out
the noise.

 In this case, we first filter out the noise interference using a notch
pass filter:
N (u, v)  H (u, v)G(u, v)
(m, n)  F 1N (u, v)

 The image (m, n) yields a rough estimate of the interference pattern.

 We can then subtract off a weighted portion of (m, n) from the

image g(m,n) to obtain our restored image:
fˆ(m, n)  g (m, n)  w(m, n)(m, n)

 It is possible to design the weighting function or modulation function

w(m, n) in an optimal fashion.
 Linear, position-invariant degradation

 We will now consider the general degradation equation:

g (m, n)  h(m, n) * f (m, n)  (m, n)

G(u, v)  H (u, v)F (u, v)  N (u, v)

 This consists of a “blurring” function h(m,n), in addition the random

noise component (m, n).

 The blurring function h(m,n) is usually referred to as a point-spread

function (PSF) and represents the observed image corresponding to
imaging an impulse or point source of light.

 In this case, we need to have a good knowledge of the PSF h(m,n), in

addition to knowledge of the noise statistics. This can be done in
practice using one of the following methods:

Using Image observation

o Identify portions of the observed image (subimage) that are
relatively noise-free and which corresponds to some simple
structures.
Gs (u, v)
o We can then obtain s H (u, v)  , where G (u, v) is the
s
F̂s (u,v)
spectrum of the observed subimage, Fˆs (u, v) is our estimate of
the spectrum of the original image (based on the simple
structure that the subimage represents).
o Based on the characteristic of the function H s (u,v) , once can
rescale to obtain the overall PSF H (u,v) .
 Experimentation
 If feasible, image a known object, usually a point source of
light, using the given imaging equipment and setup.
 If A is the intensity of light source and G(u,v) is the observed
spectrum, we have
G(u, v)
H (u, v)  .
A

Modeling

 A physical model is often used to obtain the PSF.

 Blurring due to atmospheric turbulence can be modeled by the
transfer function:

H (u, v)  e k(u  v )
2 2 5/6

where k is a constant that depends on the nature of the turbulence.

 Note that this is similar to a Gaussian lowpass filter.
 Gaussian lowpass filter is also often used to model mild uniform
blurring.

 Precise mathematical modeling of the blurring process is sometime

used. For example, blurring due to uniform motion is modeled as:

sin(ua  vb)e j(uavb)

T
H (u, v) 
(ua  vb)
where T is the duration of exposure and a and b are the displacements in
the x- and y-directions, respectively, during this time T.
 Inverse Filter
 The simplest approach to restoration is direct inverse filtering. This is
obtained as follows:
G(u, v)
F̂ (u,v) 
H (u, v)
F̂ (u, v)  R(u, v)G(u, v), u, v  0,1, , N  1,

where

R(u, v)  1
.
H (u, v)

 We can rewrite this in the spatial domain as follows:

G(u, v)
fˆ(m, n)  g(m, n)* r(m, n)  IDFT .
H (u, v)

 In practice, we actually use a slightly modified filter:

1
, H (u, v)  
R(u, v)  H (u, v)
0, otherwise

where  is a small value. This avoids numerical problems when

H (u, v) is small.
 The inverse filter works fine provided there is no noise. This is
illustrated in the following example.
 Let us now analyze the performance of the inverse filter in the
presence of noise. Indeed, in this case:
G(u, v)  H (u, v)F (u, v)  N (u, v) ,
 which gives
G(u, v) N (u, v)
F̂ (u, v)   F (u, v) 
H (u, v) H (u, v)
 Hence noise actually gets amplified at frequencies where H (u, v) is
zero or very small. In fact, the contribution from the noise term
dominates at these frequencies.
 As illustrated by an example, the inverse filter fails miserably in the
presence of noise. It is therefore, seldom used in practice, in the
presence of noise.
 Inverse Filtering example (no noise)

1 1 1
1 1 1 1
h 2
N f (m, n)
1 1 1 N N

  0.001

N 7 N  11 N  15

g(m, n)

MSE  0.014 MSE  0.03 MSE  0.05

fˆ (m, n)

MSE  3.6 10-5 MSE  2.310-4 MSE  0.0029

 Inverse Filtering example (no noise)

1
H (u, v)  2

1 u v
2 2
f (m, n)
r0
  0.001
g(m, n) fˆ (m, n)

r0  11

MSE  0.02 MSE  0.008

r0  15

MSE  0.017 MSE  0.005

r0  23

MSE  0.013 MSE  0.0016

 Inverse Filtering example (with noise)

1 1 1
1 1 1 1
h
25 f (m, n)
1 1 1 55
Zero-mean Gaussian
  0.01 noise with variance  2

  0.03   0.01   0.02

g (m, n)

MSE  0.008 MSE  0.007 MSE  0.0075

fˆ (m, n)

MSE  0.9 MSE  0.09 MSE  0.47