COMPUTER VISION (BAI515A)

MODULE-3
IMAGE RESTORATION AND RECONSTRUCTION
Introduction
The principle goal of restoration techniques is to improve an image in some predefined sense.
Restoration is an objective process and it attempts to recover an image that has been degraded
by using a prior knowledge of the degradation phenomenon. Thus, restoration techniques are
oriented toward the modeling the degradation and applying the inverse process in order to
recover the original image.
Image restoration vs. image enhancement:
• Enhancement:
• Largely a subjective process.
• Prior knowledge about the degradation is not a must (sometimes no degradation is
involved).
• Procedures are heuristic and take advantage of the psychophysical aspects of human
visual system.
• Restoration:
• More an objective process.
• Images are degraded.
• Tries to recover the images by using the knowledge about the degradation.
• Image restoration: recover an image that has been degraded by using a prior knowledge
of the degradation phenomenon.
• Model the degradation and applying the inverse process in order to recover the original
image.
• Image enhancement: “improve” an image subjectively.
• Image restoration: remove distortion from image, to go back to the “original”--objective
process.
Why Image Restoration?
Image restoration is to recover the original image by removing noise and blur from image.
Image blur is difficult to avoid in many situations like photography, to remove motion blur
caused by camera shake, radar imaging to remove the effect of image system response, etc.
Image noise is unwanted signal which comes in image from sensor such as thermal or electrical
signal and Environmental condition such as rain, snow etc.

DEPT. OF AIML, KNSIT 1

 COMPUTER VISION (BAI515A)

A Model of Image Degradation / Restoration Process:

As fig. shows, degradation process is modeled as degradation function that, together with the
additive noise term, operates on an input image f(x, y) to produce a degraded image g(x, y).
Given g(x, y), some knowledge about the degradation function H, and additive noise term, the
objective of restoration is to obtain an estimate of the original image.
Estimation of the output image should be as close as possible to the original input image.

If H is a linear, position- invariant process, then the degraded image is given in the spatial
domain by
g(x, y) = h(x, y)*f(x, y) + η(x, y)
Where h(x, y) is the spatial representation of the degradation function and symbol * indicates
convolution. Convolution in the spatial domain is analogous to multiplication in the frequency
domain.
The model of the degraded image is given in the frequency domain by
G(u, v) = H(u,v)F(u, v) + N(u, v)
Where the terms in capital letters are the Fourier transforms of the Corresponding terms of
previous spatial domain expression.
NOISE MODELS
Noise Sources:
The principle sources of noise in digital images arise during image acquisition and/or
transmission.
Image acquisition:
e.g., light levels, sensor temperature, etc.
Sensor performance is affected by environmental conditions during image acquisition, and by
the quality of sensing elements.
Transmission:
Interference in the channel used for transmission.

DEPT. OF AIML, KNSIT 2

 COMPUTER VISION (BAI515A)

e.g., lightning or other atmospheric disturbance in wireless network.

Spatial and frequency properties of Noise:
When the Fourier spectrum of noise is constant, the noise is usually called as white noise.
With the exception of spatially periodic noise, we assume that the Noise is independent of
spatial coordinates and that it is uncorrelated with respect to the image itself
Important Noise probability Density Functions:
Statistical behavior of the intensity values in the noise component is a matter of concern in the
analysis of image restoration model. These may be considered random variables, characterized
by a probability density function(PDF). The following are the most common PDFs found in
image processing applications.
Gaussian Noise:
The PDF of a Gaussian random variable, z, is given by

where z represents intensity, z is the mean (average) value of z, and σ is its standard deviation. The
standard deviation squared, σ2, is called the variance of z.
70% of its values will be in the range [(z - σ), (z + σ)], and about 95% will be in the range [(z
- 2σ), (z + 2σ)].
Rayleigh noise:
The PDF of Rayleigh noise is given by

The mean and variance of this density are given by

And

DEPT. OF AIML, KNSIT 3

 COMPUTER VISION (BAI515A)

The Rayleigh density can be quite useful for approximating skewed histograms.

Erlang (gamma) noise:

The PDF of Erlang noise is given by

where the parameters are such that a>0, b is a positive integer, and “!” indicates factorial. The
mean and variance of this density are given by

and

Exponential noise:

DEPT. OF AIML, KNSIT 4

 COMPUTER VISION (BAI515A)

The PDF of exponential noise is given by

where a > 0. The mean and variance of this density function are

and

Uniform noise:
The PDF of uniform noise is given by

The mean of this density function is given by

and its variance by

Impulse (salt-and-pepper) noise:

The PDF of (bipolar) impulse noise is given by

if b > a, gray-level b will appear as a light dot, while level a will appear like a dark dot. If
either Pa unipolar or Pb is zero, the impulse noise is called unipolar.

DEPT. OF AIML, KNSIT 5

 COMPUTER VISION (BAI515A)

Above PDFs provide useful tools for modeling a broad range of noise corruption situations
given below.

 Gaussian noise arises in an image due to factors such as Electronic circuit noise, sensor
noise due to poor illumination and/or high temperature
 The Rayleigh density is helpful in characterizing noise phenomenon in Range imaging.
 The exponential and gamma densities find application in laser imaging.
 Impulse noise is found in situations where quick transients, such as faulty switching
take place during imaging.
 Uniform density is useful as the basis for numerous random number generators that are
used in simulations.

Periodic noise:

 Periodic noise in an image arises typically from electrical or electromechanical

interference during image acquisition.
 It can be observed by visual inspection both in the spatial domain and frequency
domain.
 It is a type of spatially dependent noise
 Periodic noise can be reduced significantly via frequency domain filtering
Estimation of Noise Parameters
Periodic noise:
 Parameters can be estimated by inspection of the Fourier spectrum of the image.
 Periodic noise tends to produce frequency spikes that can be detected by visual analysis.
Noise PDFs:
 From sensor specifications
 If imaging sensors are available, capture a set of images of plain environments
 If only noisy images are available, parameters of the PDF involved can be estimated
from small patches of constant regions of the noisy images
The simplest use of the data from the image strips is for calculating the mean and variance of
intensity levels. Consider a strip (subimage) denoted by S, and let Ps(zi), i = 0, 1, 2,.…, L - 1,
denote the probability estimates (normalized histogram values) of the intensities of the pixels
in S, where L is the number of possible intensities in the entire image (e.g., 256 for an 8-bit
image).

DEPT. OF AIML, KNSIT 6

 COMPUTER VISION (BAI515A)

The mean and variance of the pixels in S:

and

The shape of the histogram identifies the closest PDF match. If the shape is approximately
Gaussian, then the mean and variance are all we need because the Gaussian PDF is completely
specified by these two parameters.

Restoration in the Presence of Noise Only ̶ Spatial Filtering

Degraded image in the spatial domain and frequency domain are as follows
g(x, y) = h(x, y)*f(x, y) + η(x, y)
and
G(u, v) = H(u,v)F(u, v) + N(u, v)
When the only degradation present in the image is noise, then the above equations become
g(x, y) = f(x, y) + η(x, y)
and
G(u, v) = F(u, v) + N(u, v)
Mean Filters
In this this topic noise reduction capabilities of the various spatial filters are discussed.
Arithmetic mean filter:
This is the simplest of the mean filters. Let Sxy represent the set of coordinates in a rectangular
subimage window (neighborhood) of size m×n centered at point (x, y). The arithmetic mean
filter computes the average value of the corrupted g(x, y) image in the area defined by Sxy. The
value of the restored image f̂ at point (x, y) is simply the arithmetic mean computed using the
pixels in the region defined by Sxy In other words,

This operation can be implemented using a spatial filter of size m×n in which all coefficients
have value 1/mn. A mean filter smooths local variations in an image, and noise is reduced as a
result of blurring.

DEPT. OF AIML, KNSIT 7

 COMPUTER VISION (BAI515A)

Geometric mean filters:

An image is restored using this filter is given by the expression

Generally, a geometric mean filter achieves smoothing comparable to the arithmetic mean
filter, but it tends to lose less image detail in the process.
Harmonic mean filter:
The Harmonic mean filter operation is given by the expression

The harmonic mean filter works well for salt noise, but fails for pepper noise. It does well also
with other types of noise like Gaussian noise.

Contraharmonic mean filter:

This filter yields a restored image based on the expression

where Q is called the order of the filter. This filter is well suited for reducing or virtually
eliminating the effects of salt-and-pepper noise. For positive values of Q, the filter eliminates
pepper noise. For negative values of Q it eliminates salt noise. It cannot do both simultaneously.
Note that the contraharmonic filter reduces to the arithmetic mean filter if Q = 0, and to the
harmonic mean filter if Q = -1.

Order Statistic Filters

Here some additional order statistics filters are discussed. Order statistics filters are spatial
filters whose response is based on ordering (ranking) the values of the pixels contained in the
image area encompassed by the filter. The ranking result determines the response of the filter.
Median filter:

DEPT. OF AIML, KNSIT 8

 COMPUTER VISION (BAI515A)

In this method value of a pixel is replaced by the median of the intensity levels in the
neighborhood of that pixel.
 Median represents the 50th percentile of a ranked set of numbers

 It has excellent noise reduction capabilities for certain types of random noise.
 Less blurring than linear smoothing filters of similar size
 It is effective in the presence of both bipolar and unipolar noise.
Max and min filter:
Max filter uses the 100th percentile of a ranked set of numbers

This filter is useful for finding the brightest points in an image. Also, because pepper noise has
very low values, it is reduced by this filter as a result of the max selection process in the
subimage area Sxy.

Min filter is the 0th percentile of a ranked set of numbers.

This filter is useful for finding the darkest points in an image. Also, it reduces salt noise as a
result of the min operation.
Midpoint filter:
The midpoint filter computes between the maximum and minimum values in the area
encompassed by the filter

This filter combines order statistics and averaging. It works best for randomly distributed noise,
like Gaussian or uniform noise.
Alpha-Trimmed Mean Filter:

DEPT. OF AIML, KNSIT 9

 COMPUTER VISION (BAI515A)

Suppose that we delete the d/2 lowest and the d/2 highest intensity values of g(s,t) in the
neighborhood Sxy. Let gr(s,t) represent the remaining mn-d pixels. A filter formed by averaging
these remaining pixels is called an alpha trimmed mean filter:

Where the value of d can range from 0 to mn-1. When d=0, the alpha trimmed filter reduces
to the arithmetic mean filter. If we choose d = mn-1, the filter becomes a median filter. For
other values of d, the alpha-trimmed filter is useful in situations involving multiple types of
noise, such as a combination of salt-and-pepper and Gaussian noise.
Adaptive filters
 The behavior of adaptive filter changes based on statistical characteristics of the image
inside the filter region defined by the mхn rectangular window.
 The performance is superior to that of the mean filters or order statistics filter.
 Filter complexity is increased when the filter is designed for improved filtering power.
Adaptive, Local Noise Reduction Filters:
 The simplest statistical measures of random are its mean and variance.
 The mean gives a measure of average intensity in the region over which the mean is
computed.
 The variance gives a measure of contrast in that region.
 Filter is to operate on local region Sxy
The response of the filter at any point (x, y) on which the region is centered is to be based on
four quantities:
(a) g(x, y), the value of the noisy image at (x, y);
(b) σ2ղ, the variance of the noise corrupting f(x, y) to form g(x, y);
(c) mL, the local mean of the pixels in Sxy and
(d) σ2L, the local variance of the pixels in Sxy
The behavior of the filter:
(a) if σ2ղ is zero, the filter should return simply the value of g ( x, y).
(b) if the local variance is high relative to σ2ղ, the filter should return a value close to g(x,y);
(c) if the two variances are equal, the filter returns the arithmetic mean value of the pixels in
Sxy .
An adaptive expression for obtaining f̂(x, y) based on these assumptions may be written as

DEPT. OF AIML, KNSIT 10

 COMPUTER VISION (BAI515A)

Adaptive Median Filters:

Median filter is effective for removing salt-and-pepper noise. The density of the impulse noise
cannot be too large. This filter preserves image details while smoothing non-impulse noise.
The notation:
Zmin = minimum intensity value in Sxy
Zmax = maximum intensity value in Sxy
Zmed = median intensity value in Sxy
Zxy = intensity value at coordinates ( x, y)
Smax = maximum allowed size of Sxy
The adaptive median-filtering works in two stages:
Stage A:
A1 = Zmed - Zmin
A2 = Zmed - Zmax
If A1>0 and A2<0, go to stage B
Else increase the window size
If window size ≤ Smax repeat stage A
Else output Zmed
Stage B:
B1 = Zxy - Zmin
B2 = Zxy - Zmax

if B1>0 and B2<0, output Zxy ; Else output Zmed

This filter has three main purposes:
 To remove salt and pepper (impulse) noise.
 To provide smoothing of other noise that may not impulsive, and
 To reduce distortion such as excessive thinning or thickening of object boundaries.
The purpose of stage A is to determine if the median filter output, Zmed , is an impulse (black
or white) or not. If the condition Zmin < Zmed < Zmax holds, then Zmed cannot be an impulse. In
this case go to stage B and test to see if the point in the center of the window, Zxy, is itself an
impulse. If the condition B1>0 AND B2<0 is true, then Zmin < Zxy < Zmax , and Zxy cannot be
an impulse.
DEPT. OF AIML, KNSIT 11
 COMPUTER VISION (BAI515A)

If the condition B1 > 0 AND B2 < 0 is false, then either Zxy = Zmin or Zxy = Zmax. In either case
the value of the pixel is an extreme value and the algorithm outputs the median value Zmed
which is not a noise impulse.
Periodic Noise Reduction by Frequency Domain Filtering
The basic idea
Periodic noise can be analyzed and filtered effectively using frequency domain techniques. The
basic idea is that the Periodic noise appears as concentrated bursts of energy in the Fourier
transform, at locations corresponding to the frequencies of the periodic interference.
Approach
 A selective filter is used to isolate the noise.
 Band reject, band pass, and notch filters are used as tools for periodic noise reduction.
 Band reject filters remove or attenuate a band of frequencies about the origin of the
Fourier transform.
 Band reject filtering is for noise removal in applications where the general locations of
noise components in frequency domain is approximately known.
Example is an image corrupted by additive periodic noise that can be approximated as 2 D
sinusoidal functions. Fourier transform of sine consists of two impulses that are mirror images
of each other about the origin of the transform.
The sinusoidal noise components appear as symmetric pairs of bright dots can be observed in
its Fourier Transform.
Band reject Filters
The transfer functions of ideal, butter-worth and Gaussian filters are as shown below. Its
perspective plots are shown in Figure 3.3

Figure 3.3: From left to right, perspective plots of ideal, Butterworth (of order 1), and Gaussian
band reject filters

DEPT. OF AIML, KNSIT 12

 COMPUTER VISION (BAI515A)

Ideal band reject filter:

Band reject filters remove or attenuate a band of frequencies about the origin of the Fourier
Transform.
Band-pass Filters
Band-pass filter performs the opposite of a band-pass filter.
The transfer function of Band-pass filter is obtained from corresponding transfer function of
Band – reject filter.
HBP (u, v) = 1 – HBR (u, v)
Generally, It removes too much image details. It is quite useful in isolating the effects on an
image caused by selected frequency bands.
Notch Filters
Notch filter rejects (or passes) frequencies in predefined neighborhoods about a center
frequency.
It appears in symmetric pairs about the origin because the Fourier transform of a real valued
image is symmetric.
The transfer function of Notch pass filter is
HNP(u,v) = 1 – HNR(u,v)
Where HNP(u,v) is the transfer function of the notch pass filter Corresponding to the notch
reject filter with transfer function HNR(u,v). This filter reduces the noise in the image without
introducing appreciable blurring.

Figure 3.4: Perspective plots of (a) ideal, (b) Butterworth (of order 2), and (c) Gaussian
notch (reject) filters.

DEPT. OF AIML, KNSIT 13

 COMPUTER VISION (BAI515A)

Optimum Notch Filter

Several interference components are present in the methods discussed in the preceding sections
are not always acceptable because they remove much image information.
Interference components are generally are not single frequency bursts, instead the components
tend to have broad skirts that carry information about the interference pattern and the skirts are
not always easily detectable.
This filtering method reduce the effect of degradations of the image.
The method discussed here is optimum, in the sense that it minimizes local variances of the
restored estimated image.
Procedure for restoration tasks in multiple periodic interference
Isolate the principal contributions of the interference pattern.
Subtract a variable, weighted portion of the pattern from the corrupted image.
As a first step extract the principal frequency components of the interference pattern Place a
notch pass filter at the location of each spike.
N(u, v) = HNP(u, v)G(u, v)
Where G(u,v) is the Fourier transform of the corrupted image. After a filter has been selected,
the corresponding pattern in the spatial domain is obtained from the expression.

In this section, we try to improve the restored image by introducing a modulation function.

---------------- (1)

Here the modulation or weighting function w(x, y) is a constant within a neighborhood of

size (2a+1) by (2b+1) about a point (x,y). Eqn. (1) is an estimate of f̂(x, y).
We optimize its performance by minimizing the local variance of the restored image at the
position (x,y).
Consider a neighborhood of size (2a + 1) by (2b + 1) about a point (x, y). The “local” variance
of f̂(x, y) at coordinates (x, y) can be estimated from the samples, as follows:

----------------- (2)

Where f̂(x, y) is the average value of f̂ in the neighborhood; that is,

DEPT. OF AIML, KNSIT 14

 COMPUTER VISION (BAI515A)

……………………….. (3)

Points on or near the edge of the image can be treated by considering partial neighborhoods or
by padding the border with 0s.
Substituting Eq. (1) into Eq. (2) yields

……………………. (4)

Assuming that w(x, y) remains essentially constant over the neighborhood gives the
approximation

For –a ≤ s ≤ a and –b ≤ t ≤ b. This assumption also results in the expression

in the neighborhood. With these approximations, Eq. (4) becomes

To minimize σ2(x, y) we solve

For w(x, y) .The result is

DEPT. OF AIML, KNSIT 15