Image Restoration

Digital Image Processing

 Image Restoration
• Image restoration concerns the removal or
reduction of degradations which have occurred
during the acquisition of the image.
– Some restoration can be performed successfully using
neighborhood operations, while others require the use
of frequency domain processes.

• Image degradations
– Noise : errors in the pixel values.
– Optical effects : out of focus blurring, blurring due to
camera motion...
 Image Degradation/Restoration Process
• If the degradation function H is linear and position-
invariant, then the degraded image is

g ( x, y )  h( x, y ) * f ( x, y )   ( x, y )  fˆ ( x, y )
Degraded Degradation Image Noise Estimate
Image Function
 Image Degradation/Restoration Process
• Frequency domain representation

G (u , v)  H (u , v) F (u , v)  N (u , v)  Fˆ (u , v)
Degraded Degradation Image Noise Estimate
Image Function

• If we knew the values of H and N, we could

recover F by inverse filtering :

ˆ G (u , v)  N (u , v)
F (u , v) 
H (u , v)
Is this approach practical?
 Noise
• Noise is any degradation in the image signal
caused by external disturbance.

• The principle sources of noise in digital images

arise during image acquisition (digitization) and
transmission.

• We would like to know what type of errors to

expect and the type of noise on the image so we can
choose the most appropriate method for reducing
the effects.
 Salt and Pepper Noise
• Also called impulse noise,
noise shot noise, or binary
noise.
• Caused by sharp, sudden disturbances in the image
signal.
• Its appearance is randomly scattered white or black
(or both) pixels over the image.
 Salt and Pepper Noise

from skimage import data, color, io

import skimage.util.noise as noise
image = color.rgb2gray(data.astronaut())

gn=noise.random_noise(image,mode='s&p', amount=0.2)

io.imshow(gn)
 Gaussian Noise
• Gaussian noise is white noise that is normally
distributed.
• Caused by random fluctuations in the signal.

gn=noise.random_noise(image,'gaussian’)
# m=0, v=0.1
 Speckle Noise
• Speckle noise (or speckle) can be modeled
by random values multiplied by pixel values.
• Also called multiplicative noise.
• Speckle noise is a major problem in some
radar applications.

gn=noise.random_noise(image,’speckle')

# v=0.04
 Periodic Noise
• Periodic noise is a periodic, spatially dependent
noise.
• Caused by electrical or electromechanical
interference.
 Periodic Noise
from skimage import data, color, io
import skimage.util.noise as noise
import skimage
import numpy as np

image = color.rgb2gray(data.astronaut())

r,c=image.shape
x,y=np.mgrid[0:r,0:c].astype('float32')
p=np.sin(x/3+y/3)+1.0

gp=(2*skimage.util.img_as_float(image)+p/2)/3

io.imshow(gp)
 Noise Cleaning
• Salt and pepper noise, Gaussian noise and speckle
noise can all be cleaned by using spatial filtering
techniques.

• Periodic noise, however, requires the use of

frequency domain filtering. This is because
whereas the other forms of noise can be modeled
as local degradations, periodic noise is a global
effect.
 Cleaning Salt and Pepper Noise

• Lowpass filtering = smoothing

• Median filtering
• Rank-order filtering
• Outlier method
 Lowpass Filtering
import matplotlib.pyplot as plt
from scipy import ndimage
from skimage import data, color, io
import skimage.util.noise as noise

image = color.rgb2gray(data.astronaut())

gn=noise.random_noise(image,mode='s&p')
result = ndimage.uniform_filter(gn, 7)

fig, axes = plt.subplots(nrows=1, ncols=2)

ax = axes.ravel()

ax[0].imshow(gn, cmap='gray')
ax[0].set_title("Salt Pepper")

ax[1].imshow(result, cmap='gray')
ax[1].set_title("Low pass filter")

ax[0].set_xlim(0, 512)
ax[0].set_ylim(512, 0)
plt.tight_layout()
plt.show()
Lowpass Filtering
Lowpass Filtering
Lowpass Filtering
Lowpass Filtering
Median Filtering
 Median Filtering
result = ndimage.median_filter(gn, 3)
 Median Filtering
result = ndimage.median_filter(gn, 5)
 Median Filtering
result = ndimage.median_filter(gn, 9)
Median Filtering
Median Filtering
Median Filtering
Median Filtering
Median Filtering
 Rank-Order Filtering
• Median filter is a special case of a more general
process called rank-order filter.
• The reason for using rank-order filter is when
dealing with nonrectangular masks.
•

0 1 0
1 1 1
0 1 0
 Rank-Order Filtering

image = color.rgb2gray(data.astronaut())

cross=array([[0,1,0],[1,1,1],[0,1,0]])

gn=noise.random_noise(image,mode='s&p')
result = ndimage.median_filter(gn, footprint=cross)
Rank-Order Filtering
 An Outlier Method
• Applying the median filter can in general be a slow
operation.

• Outlier Method
1. Choose a threshold value D
2. For a given pixel, compare its value p with the mean m of the
values of its eight neighbors
3. If |p − m| > D, then classify the pixel as noisy, otherwise not
4. If the pixel is noisy, replace its value with m; otherwise leave
its value unchanged
 Outlier Method
image = color.rgb2gray(data.astronaut())
av=array([[0,1,0],[1,1,1],[0,1,0]])/8.0
image_sp=noise.random_noise(image,mode='s&p')
image_sp_av=ndimage.convolve(image_sp,av)
D=0.2
r=(abs(image_sp-image_sp_av)>D)*1.0

imshow(r*image_sp_av+(1-r)*image_sp)

Ch8-p.200
 An Outlier Method
• Outlier method is not automatic, choosing appropriate D
value is critical.
 An Outlier Method
• Outlier method is not automatic, choosing appropriate D
value is critical.
 Cleaning Gaussian Noise

• Lowpass filtering
• Image averaging
• Adaptive filtering
 Image Averaging
• A simple approach to cleaning Gaussian noise is
to take the average of all the images taken at the
same scene.
• Examples:
– Satellite imaging
– Microscopy
• Because Gaussian noise has mean 0, averaging
will reduce the noise to 0
– noise / k (k = number of images)
 Image Averaging

image = color.rgb2gray(data.astronaut())
image_gaus=noise.random_noise(image, 'gaussian')
x,y=image_gaus.shape
t=zeros((x,y,10))

for i in range(10):
t[:,:,i] = noise.random_noise(image,'gaussian')

ta=mean(t,3)
Image Averaging
 Adaptive Filtering
• Adaptive filters are a class of filters that change
their characteristics according to the values
(statistics) of the grayscales under the mask.

• Performance is usually superior than global

filters, at the cost of increasing complexity.
 Adaptive Noise Reduction Filter
• Filter response is based on four quantities
– g(x,y) the value at (x,y)
– 2 variance of the noise (estimate)
–  L2 local variance
– mL local mean
• Filtering result is given as
 2
fˆ ( x, y )  g ( x, y )  [ g ( x , y )  mL ]
 L2

Assumption: 2   L2
 Adaptive Filtering
from scipy.signal import wiener
image = color.rgb2gray(data.astronaut())
image_gaus=noise.random_noise(image, 'gaussian')
image_wiener= wiener(image_gaus,[3,3])

Gaussian Noise Adaptive Filter

 Removal of Periodic Noise

• Frequency domain filtering

– Band reject filtering
– Notch filtering
 Removal of Periodic Noise
• Create periodic noise by overlaying image with a trigonometric
function:
image = color.rgb2gray(data.astronaut())

r,c=image.shape
x,y=np.mgrid[0:r,0:c].astype('float32')
p=np.sin(x/3+y/3)+1.0

gp=(2*skimage.util.img_as_float(image)+p/2)/3

io.imshow(gp)
Band Reject Filters
 Band Reject Filters
• Ideal bandreject filter
1 if D(u, v)  D0  W / 2

H (u , v)   0 if D0  W / 2  D(u , v)  D0  W / 2
1 if D(u, v)  D  W / 2
 0

• Butterworth Bandreject filter D0 = radial center

1 W = width of the band
H (u , v)  D (u , v )W 2n
1 [ ]
D 2 (u , v )  D02

• Gaussian bandreject filter

1 D 2 ( u ,v )  D02 2
 2 [ D ( u ,v )W ]
H (u , v)  1  e
Band Reject Filtering
Band Reject Filters
 Notch Filters

Ideal

Butterworth

Gaussian
 Notch Filtering
>> tf(156,:) = 0;
>> tf(102,:) = 0;
>> tf(:,170) = 0;
>> tf(:,88) = 0;
>> figure, fftshow(tf, 'log');
>> tfi = ifft2(tf);
>> figure, fftshow(tfi, 'abs')
 Image Degradation/Restoration Process
• Frequency domain representation

G (u , v)  H (u , v) F (u , v)
Degraded Degradation Image
Image Function

• Inverse Filtering

G (u , v)
F (u , v) 
H (u , v)
Inverse Filtering
 Inverse Filtering Problem
• Dividing very small values produces very large
values
• Solution:
– Apply lowpass filter L to the result
G (u , v)
F (u , v)  L(u , v)
H (u , v)

– Use constrained division

 G (u , v)
 if H (u , v)  d
F (u , v)   H (u , v)
 G (u , v) if H (u , v)  d

40 60

80 100
d=0.01 d=0.005

d=0.002 d=0.001
Motion Deblurring
Motion Deblurring
Motion Deblurring
 Estimating Degradation Function
• There are three principal ways to estimate the
degradation function for image restoration
– Estimation by image observation
– Estimation by experimentation
– Mathematical modeling

• This process sometimes is called blind

deconvolution because the true degradation
function is seldom known completely
 Estimation by Image Observation
• Find a small section of the image containing simple
structures and strong signal content
• Construct an undegraded image of the same size and
characteristics as the observed subimage
• Estimate the degradation function (assuming noise is
negligible due to strong signal)

Gs (u , v)
H s (u , v) 
Fˆs (u , v)
Gs (u , v)  FT of observed subimage
Fˆs (u , v)  FT of constructed subimage
 Estimation by Experimentation
• Obtain the impulse response of the degradation by
imaging an impulse (small dot of light) using the same or
similar imaging system with the system settings
• Estimate the degradation function

G (u , v)
H (u , v) 
A
G (u , v)  FT of the observed image
A  constant describing the strenght of the impulse