EENG 860 Special Topics: Digital Image Processing

Lecture 8: Image Restoration In Spatial

Domain

Dr. Ahmadreza Baghaie

Department of Electrical and Computer Engineering
New York Institute of Technology

Spring 2020

Readings: Chapter 5 (sections 5.1-5.3)

1 / 40
How to read: read and experiment!
Table of Content

● Image Degradation/Restoration Model

● Noise Models
● Noise Parameter Estimation
● Restoration for Noise-Only Degradation – Spatial Filtering
● Spatial Filtering: Mean Filters
● Spatial Filtering: Order-Statistic Filters
● Adaptive Filters

2 / 40
What We Learned So Far ...

● Image Acquisition: sampling and quantization

● Intensity Transformations:
– Inversion
– Log transform
– Gamma correction
– Histogram equalization
– Histogram matching
● Spatial Filtering
● Frequency Filtering

3 / 40
What We Learned So Far ...

● Image Acquisition: sampling and quantization

● Intensity Transformations: inversion, log transform, Gamma correction,
histogram equalization, histogram matching
● Spatial Filtering:
– Correlation vs. convolution
– Smoothing spatial filters (box, Gaussian, median filters)
– Sharpening spatial filters (Laplacian, gradient, unsharp masking)
● Frequency Filtering

4 / 40
What We Learned So Far ...

● Image Acquisition: sampling and quantization

● Intensity Transformations: inversion, log transform, Gamma correction,
histogram equalization, histogram matching
● Spatial Filtering: correlation vs. convolution, smoothing spatial filters (box,
Gaussian, median filters), sharpening spatial filters (Laplacian, gradient,
unsharp masking)
● Frequency Filtering:
– Fourier series and Fourier transform
– Sampling and aliasing
– Discrete Fourier Transform (DFT)
– Lowpass filters (ideal, Gaussian, Butterworth)
– Highpass filters (ideal, Gaussian, Butterworth, Laplacian, unsharp
masking)
– Selective filtering
– Fast Fourier Transform (FFT) 5 / 40
Image Degradation/Restoration Model

●
Image degradation is modeled as an operator ℋ together with an additive
noise term for an input image f(x,y), to generate the degraded image g(x,y).
●
Given g(x,y), some knowledge about ℋ and some knowledge about the
additive noise term, the objective of restoration is to obtain an estimate of the
original image.

6 / 40
Image Degradation/Restoration Model

●
We will show later that if the operator ℋ is linear and position-invariant,
then the degraded image is given in the spatial domain by:

g ( x , y)=h( x , y)⊗f ( x , y)+n( x , y)

with h(x,y) as the spatial representation of the degradation function and
n(x,y) as the additive noise term.
● Given the convolution theorem, the degraded image in the frequency domain
is:

G(u , v)=H (u , v) F (u , v )+ N (u , v)

with the terms in capital letters as the Fourier transform of the corresponding
terms.
● Let’s focus on noise-only degradation for now!

7 / 40
Noise Models

● Sources of noise in digital images arise mainly during image acquisition and/
or transmission.
● For example in CCD cameras, light levels and sensor temperature are major
factors in the amount of noise.
● During transmission, images are affected by interference in the transmission
channel.
● For example in wireless transmission, lightning or other atmospheric
disturbances cause noise.

8 / 40
Spatial and Frequency Properties of Noise

● What parameters define the spatial characteristics of noise?

● Is the noise correlated with the image?
● When the Fourier spectrum of noise is constant, the noise is called white
noise.
● Why is it called white noise? The naming is because of the physical property
of white light, which contains all frequencies in the visible spectrum in equal
proportions.
● With the exception of spatially periodic noise, here we assume that noise is
independent of spatial coordinates, and that it is uncorrelated with respect to
the image itself.
● This is for simplification, as the complexities of spatially dependent and
correlated noise are beyond the scope of the course.

9 / 40
Noise Probability Distribution Functions (PDF)

● Statistical behavior of the intensity values in the noise component is

characterized by a Probability Distribution Function (PDF).
● The noise component n(x,y) is an image with the same size as the input
image.
● We simulate noise images by generating an array whose intensity values are
random numbers with a specified PDF.
● Important Noise PDFs:
– Gaussian noise
+∞
– Rayleigh noise
Mean: z̄= E( z)= ∫ zp (z)dz
– Erlang (Gamma) noise −∞
+∞
Exponential noise 2 2
–
Variance: σ =V ( z)= ∫ ( z− z̄) p ( z) dz
– Uniform noise −∞

– Salt-and-pepper noise
● Salt-and-pepper noise is applied differently.
10 / 40
Important Noise PDFs: Gaussian Noise

● Most frequently used, because of its mathematical tractability in both spatial

and frequency domains.
● The PDF of Gaussian noise, is given by:
2
( z− z̄)
1 −
2σ
2

p( z)= e for −∞< z<∞

√2 π σ
z : intensity
z̄ : mean (average) value of z
σ : standard deviation of z

11 / 40
Important Noise PDFs: Rayleigh Noise

● The PDF of Rayleigh noise is given by:

2
{
2
−(z−a) / b
( z−a)e z≥a
p ( z)= b
0 z<a

● The mean and variance of z are:

z̄=a+ √ π b /4
b(4− π )
2
σ=
4
● Note the displacement from the origin, and the right-skewed shape of the
PDF.

12 / 40
Important Noise PDFs: Erlang (Gamma) Noise

● The PDF of Erlang noise is given by, with a>b, b is a positive integer, and
“!” indicating factorial:

b b−1
a z

{
p ( z)= (b−1)!
0
e
−az
z≥0
z<0
● The mean and variance of z are:

b
z̄=
a
2 b
σ= 2
a

13 / 40
Important Noise PDFs: Exponential Noise

● The PDF of exponential noise is given by:

ae−az z≥0
p ( z)= {
0 z<0

where a>0.
● The mean and variance of z are:
1
z̄=
a
1
σ 2= 2
a
● This is a special case of Erlang PDF with b=1.

14 / 40
Important Noise PDFs: Uniform Noise

● The PDF of uniform noise is given by:

1
{
p ( z)= b−a
0
a≤ z≤b
otherwise

● The mean and variance of z are:

a+b
z̄=
2
2
(b−a)
σ 2=
12

15 / 40
Important Noise PDFs: Salt-and-Pepper Noise

● The PDF of salt-and-pepper noise is given by:

k
Ps for z=2 −1

{
p ( z)= P p for z=0
1−( P s + P p ) for z=V

where k is the number of bits used to represent the intensity values and V is
any integer value in the range 0<V<2k-1.

● Given an image f(x,y) of the same size as n(x,y), we:

– assign 0 to all locations in f where 0 occurs in n;

– assign a value of 2k-1 to all locations in f where value 2k-1 occurs in n;
– leave unchanged all locations in f where V occurs in n.
16 / 40
Important Noise PDFs: Salt-and-Pepper Noise

● Salt-and-pepper noise is also known as bipolar impulse noise (unipolar if

either Ps or Pp is 0), data-drop-out noise and spike noise.
● The noise density P is defined as the probability of any pixel being
corrupted by salt or pepper noise: P=Ps+Pp.

● For example if Ps=0.02 and Pp=0.01, then P=0.03 meaning that 2% of pixels
are corrupted by salt noise, 1% of pixels are corrupted by pepper noise, in
total, 3% of pixels are corrupted by salt-and-pepper noise.
● The mean and variance are defined as:

z̄=(0) P p +V (1−( P s + P p ))+(2k −1) P s

σ 2 =(0− z̄)2 P p +(V − z̄)2 (1−( P s + P p ))+(2k −1− z̄)2 P s

17 / 40
Important Noise PDFs: Examples

18 / 40
Important Noise PDFs: Examples

19 / 40
Periodic Noise

● Periodic noise in images arises from electrical or electro-mechanical

interference during image acquisition.
● Periodic noise is a spatially dependent noise.
● As seen before, it can be reduced significantly by frequency domain filtering.

20 / 40
Noise Parameter Estimation

● Periodic noise’s parameters can be estimated by inspection of the Fourier

spectrum; they usually produce frequency spikes in the Fourier domain.
● The PDFs’ noise parameters can be known from sensor specifications, or
estimated from images acquired from the systems.
● If the imaging system is available, we can capture a set of flat images, for
example by imaging a solid gray board with uniform illumination.
● When only images are available, and not the imaging system, we can
estimate the PDFs’ noise parameters from small patches of reasonably
constant background intensity.

21 / 40
Noise Parameter Estimation

● Consider the strip S, and let pS(zi), for i=0,1,2,…,L-1, denote the probability
estimates (normalized histogram values) of the intensities of the pixels in S,
and L the number of possible intensities (for an 8-bit image, L=256).
● The mean and variance of pixel values in S can be computed by:
L−1
z̄= ∑ zi pS ( zi )
i=0
L−1
σ 2= ∑ ( zi − z̄)2 p S ( zi )
i=0

● If the shape of histogram is Gaussian, mean and variance are all we need.
● For other cases, except salt-and-pepper, we use mean and variance to
calculate parameters a and b.
● For salt-and-pepper, we need to calculate the actual probabilities of salt and
pepper occurrence, by counting them and dividing the numbers by the total
number of pixels in the strip.
22 / 40
Restoration for Noise-Only Degradation – Spatial
Filtering
● When the image is only degraded with additive noise, we have the following
in the spatial and frequency domains:

g ( x , y )=f ( x , y )+n( x , y )

G(u , v )=F (u , v )+ N (u , v )
● If the noise component is known, you can simply subtract it from the
degraded image.
● For periodic noise, you can estimate the N(u,v) from the spectrum of G(u,v).
● In general, we don’t know the noise component.
● One way to estimate f(x,y) from its degraded version g(x,y), or in another
word denoising, is spatial filtering.
● We have seen several spatial filters before in Chapter 3.

23 / 40
Restoration for Noise-Only Degradation – Spatial
Filtering
● Mean Filters:
– Arithmetic Mean Filter
– Geometric Mean Filter
– Harmonic Mean Filter
– Contra-Harmonic Mean Filter
● Order-Statistic Filters:
– Median Filter
– Min/Max Filter
– Midpoint Filter
– Alpha-Trimmed Mean Filter
● Adaptive Filters:
– Adaptive, Local Noise Reduction Filter
– Adaptive Median Filter
24 / 40
Spatial Filtering: Mean Filters

● Assume Sxy as a rectangular neighborhood of size mxn centered at pixel

(x,y).
ˆf ( x, y )  1 g ( s, t )
● Arithmetic Mean Filter 
mn ( s ,t )S xy
– Smooths local variations, and noise
1
● Geometric Mean Filter   mn
ˆf ( x, y )  g ( s, t ) 
– Loses less image detail 
 ( s ,t )S xy 
● Harmonic Mean Filter
ˆf ( x, y )  mn
– Works well for salt noise
1
● Contra-Harmonic Mean Filter 
( s ,t )S xy g ( s, t )
– Q is the order of the filter
Q 1
– Positive Q, eliminates pepper noise  g ( s , t )
– Negative Q, eliminates salt noise ˆf ( x, y )  ( s ,t )S xy
Q
 g ( s , t )
– If Q=0, equivalent to arithmetic mean filter ( s ,t )S xy
25 / 40
– If Q=-1, equivalent to harmonic mean filter
Spatial Filtering: Mean Filters - Examples

● 8-bit X-ray image, with additive Gaussian noise of zero mean and variance of
400.

26 / 40
Spatial Filtering: Mean Filters - Examples

● 8-bit X-ray image, with salt and pepper noise with probability of 0.1.

27 / 40
Spatial Filtering: Mean Filters - Examples

● In general, arithmetic and geometric mean filters are useful for random noise
like Gaussian or uniform noise.
● The contra-harmonic filter is useful for impulse noise, but it must be known
whether the noise is dark or light, in order to select the proper sign for Q.

28 / 40
Spatial Filtering: Order-Statistic Filters

● Order-statistic filters are spatial filters whose response is based on ordering

(ranking) the intensity values in the neighborhood encompassed by the filter.
● Median Filter fˆ ( x, y ) median{g ( s, t )}
( s ,t )S xy
–Most useful for impulse noise
● Max/Min Filter fˆ ( x, y )  max {g ( s, t )} fˆ ( x, y )  min {g ( s, t )}
( s ,t )S xy ( s ,t )S xy

– Good for pepper/salt-style noise

ˆ ( x, y )  1  max {g ( s, t )}  min {g ( s, t )}
f
● Midpoint Filter 2  ( s ,t )S xy ( s ,t )S xy 
– Good for Gaussian or uniform noise
ˆf ( x, y )  1 g r ( s, t )
● Alpha-Trimmed Mean Filter 
mn  d ( s ,t )S xy
– Delete the d/2 highest and d/2 lowest intensity values in the
neighborhood, then take the average of the remaining pixels.
– If d=0, equivalent to arithmetic mean filter
– If d=mn-1, equivalent to median filter 29 / 40
Spatial Filtering: Order-Statistic Filters - Examples

● 8-bit X-ray image with salt-and-pepper noise with probabilities Ps=Pp=0.1.

30 / 40
Spatial Filtering: Order-Statistic Filters - Examples

● 8-bit X-ray image, with salt and pepper noise with probability of 0.1.

31 / 40
Spatial Filtering: Order-Statistic Filters - Examples

● 8-bit X-ray image, with additive uniform noise of zero mean and variance
800, plus salt-and-pepper noise with probabilities of Ps=Pp=0.1.

32 / 40
Spatial Filtering: Adaptive Filters

● The discussed filters are applied to an image without consideration of how

image characteristics vary from one pixel to another.
● Changing the filter parameters in an adaptive and local manner can improve
the performance of the filters greatly.
● This comes at a cost: increased filter complexity.
● Here, two adaptive spatial filters will be discussed:
– Adaptive, Local Noise Reduction Filter
– Adaptive Median Filter

33 / 40
Adaptive Filters: Adaptive, Local Noise Reduction
Filter

● The filter works based on the following quantities:

g ( x , y ) : value of noisy image at location ( x , y )
σ 2n : variance of the noise
z̄ S : local average intensities of the pixels in S xy
xy

σ 2S xy
: local variance of intentensities of the pixels in S xy
● The filter behaves as follows:
– If noise variance is zero (no noise), then f(x,y)=g(x,y);
– If local variance is high relative to noise variance (near edges), the filter
should return a value close to g(x,y);
– If the two variances are equal (local area has the same property as the
overall image), perform averaging to reduce noise.
2
σ
f^ ( x , y )=g( x , y)− 2n [ g( x , y)− z̄ S ] xy
σS xy
34 / 40
Adaptive Filters: Adaptive, Local Noise Reduction
Filter - Example

● 8-bit X-ray image, corrupted by additive Gaussian noise of zero mean and
variance of 1000.

35 / 40
Adaptive Filters: Adaptive Median Filter

● The adaptive median filter can handle higher probabilities of salt-and-pepper

noise, in addition to smoothing non-impulse noise.
● The method works based on the following notation:
z min =minimum intensity value in S xy z max =maximum intensity value in S xy
z med =median intensity value in S xy z xy =intensity at location ( x , y )
S max =maximum allowed size of S xy
● The adaptive median filter algorithm is as follows:
– Level A: If z min < zmed < z max , go to Level B
Else, increase the size of S xy
If S xy ≤S max , repeat Level A
Else, output z med .

– Level B: If z min < z xy < z max , output z xy .

Else output zmed .
36 / 40
Adaptive Filters: Adaptive Median Filter

● The adaptive median filter algorithm has three objectives:

– To remove salt-and-pepper (impulse) noise;
– To provide smoothing of other noise that may not be impulsive;
– To reduce distortion, such as thinning or thickening of object boundaries.
● Level A checks whether the median of the neighborhood is impulse-like. If
not, it goes to Level B, where it checks whether the center pixel is impulse-
like. If it is not, it return the pixel value, and if it is, it returns the median value.
● If the median is impulse-like in Level A, then it increases the size of the
neighborhood, given the condition that the new size cannot be larger that the
maximum allowed, and repeats Level A. If the new size is larger than the
allowed size, it outputs the median.
● Note that the neighborhood size, and the maximum allowed size are odd,
positive integers greater than 1.

37 / 40
Adaptive Filters: Adaptive Median Filter - Example

● 8-bit X-ray image, corrupted by salt-and-pepper noise with probabilities

Ps=Pp=0.25.

38 / 40
What is Next?

● Periodic Noise Reduction Using Frequency Domain Filtering

● Linear, Position Invariant Degradations
● Estimating the Degradation Function
● Inverse Filtering
● Minimum Mean Square Error (Wiener) Filtering
● Constrained Least Squares Filtering
● Geometric Mean Filter

39 / 40
Questions?
[email protected]

40 / 40