IMAGE RESTORATION

Le Thanh Ha, Ph.D

Assoc. Prof. at University of Engineering and Technology,
Vietnam National University
[email protected]; [email protected]; 0983 692 592
 Image Restoration
• Recovering the an degraded image using priori knowledge of
degradation phenomenon.
• Modeling the degradation and applying the inverse process in
order to recover the original image
• Applying in both spatial and frequency domains

10/25/2022 Le Thanh Ha 2
 -Image degradation/restoration process
-Noise models
-Periodic Noise
-Estimation of Noise Parameters

DEGRADATION MODELING

10/25/2022 Le Thanh Ha 3
 Image degradation/restoration process

g ( x, y) = h( x, y)  f ( x, y) +  ( x, y)

G(u, v) = H (u, v) F (u, v) + N (u, v)

10/25/2022 Le Thanh Ha 4
 Noise models
• The performance of imaging sensors is affected by a variety of
factors:
– Environmental conditions: background light levels
– Sensor temperature
– Electricity noise
–…
• Need a noise model capable for a specific type of imaging
devices

10/25/2022 Le Thanh Ha 5
 Gaussian noise
• Gaussian noise arises in an image due to
factors such as electronic circuit and sensor
noise due to low illumination or high
temperature

1 −( z − z ) 2
p( z ) = e
2 2 2

10/25/2022 Le Thanh Ha 6
 Rayleigh noise
• Reyleigh density is helpful in characterizing
noise in range imaging (depth sensing)

2 − ( z − a )2 / b
 ( z − a )e for z  a
p( z ) =  b
0 for z  a

10/25/2022 Le Thanh Ha 7
 Impulse noise
• Known as Salt-and-pepper noise and found in
situations where quick transients take place
when imaging (Faulty switching)

 Pa for z = a

p( z ) =  Pb for z = b
0
 otherwise

10/25/2022 Le Thanh Ha 8
 Exponential and Gamma noise
➢Finds their applications in laser imaging

Gamma noise Exponential noise

 ab z b −1 − az
 e for z  0 e − az for z  0
p( z ) =  (b − 1)! p( z ) = 
0 0 for z  0
 for z  0

10/25/2022 Le Thanh Ha 9
 Uniform noise
• Used for computer simulation noise

 1
 if a  z  b
p( z ) =  b − a
0 otherwise

10/25/2022 Le Thanh Ha 10
10/25/2022 Le Thanh Ha 11
 Test pattern

10/25/2022 Le Thanh Ha 12
10/25/2022 Le Thanh Ha 13
10/25/2022 Le Thanh Ha 14
 Periodic noise
• Arises typically from electrical or electromechanical
interference during image acquisition

10/25/2022 Le Thanh Ha 15
 Estimation of Noise Parameters
• Simple periodic noise parameters can be
visually estimated in Fourier domain.
• Other types of noise can be estimated in
spatial domain at smooth regions.

10/25/2022 Le Thanh Ha 16
 -Mean filters
-Order statistic filters
-Adaptive filters

IMAGE RESTORATION

10/25/2022 Le Thanh Ha 17
 Mean filters
• Arithmetic mean filter
• Geometric mean filter
• Harmonic mean filter
• Contrahamonic mean filter

10/25/2022 Le Thanh Ha 18
 Arithmetic mean filter
• Smooths local variations in an images and
noise is reduced as a result of blurring

ˆf ( x, y ) = 1

mn ( s ,t )S xy
g ( s, t )

10/25/2022 Le Thanh Ha 19
 Geometric mean filter
• Achieve smoothing comparable to the
arithmetic mean filter but lose less image
detail

1
  mn
fˆ ( x, y ) =   g ( s, t ) 
 ( s ,t )S xy 

10/25/2022 Le Thanh Ha 20
 Harmonic mean filter
• Works well for salt noise but fails for pepper
noise. It also does well with Gaussian noise.

mn
fˆ ( x, y ) =
1

( s ,t )S xy g ( s, t )

10/25/2022 Le Thanh Ha 21
 Contraharmonic mean filter

( s ,t )S xy
g ( s, t )Q +1
fˆ ( x, y ) =

( s ,t )S xy
g ( s, t )Q

• Q=0: Arithmetic mean filter

• Q=-1: Harmonic mean filter
• Q>0: the filter eliminates pepper noise
• Q<0: the filter eliminates salt noise

10/25/2022 Le Thanh Ha 22
10/25/2022 Le Thanh Ha 23
10/25/2022 Le Thanh Ha 24
 Order-Statistic Filters
• Median filter
• Max and min filters
• Midpoint filter
• Alpha-trimmed mean filter

10/25/2022 Le Thanh Ha 25
 Median filter
• Excellent noise-reduction capability with
considerably less blurring

fˆ ( x, y ) = median{g ( s, t )}
( s ,t )S xy

10/25/2022 Le Thanh Ha 26
10/25/2022 Le Thanh Ha 27
 Max and min filters
Max filter is useful for finding brightest points in an image
therefore for removing low values like pepper noise

fˆ ( x, y ) = max{g ( s, t )}
( s ,t )S xy

• Min filter is useful for finding darkest points in an image

therefore for removing low values like salt noise

fˆ ( x, y ) = min{g ( s, t )}
( s ,t )S xy

10/25/2022 Le Thanh Ha 28
10/25/2022 Le Thanh Ha 29
 Midpoint filter
• Combines order statistics and averaging and
works best for Gaussian or uniform noise

ˆf ( x, y ) = 1  min{g ( s, t )}+ max{g ( s, t )}

 
2  ( s ,t )S xy ( s ,t )S xy 

10/25/2022 Le Thanh Ha 30
 Alpha-trimmed mean filter
1
fˆ ( x, y ) =
mn − d

( s ,t )S xy
g r ( s, t )

• gr(s,t) is obtained by trimming left and right

lowest and highest values of g(s,t) in Sxy
• d=0: becomes arithmetic mean filter
• d=mn-1: becomes median filter
• Other value of d: useful for multiple types of
noise: noise-and-pepper and Gaussian noises.
10/25/2022 Le Thanh Ha 31
 Adaptive Filters
• Filter behaviors change based on statistical characteristics of
the image inside the filter region.
• Many types of adaptive filters:
– Adaptive local noise reduction filter
– Adaptive median filter
– Minimum Mean Square Error (Wiener) Filter
– Constrained Least Squares Filter
–…

10/25/2022 Le Thanh Ha 32
 Adaptive local noise reduction filter
 2 is the variance of the noise
 2

fˆ ( x, y ) = g ( x, y ) − 2  g ( x, y ) − mL 
L  L2 is the variance of pixels in Sxy
mL is the local mean of pixels in Sxy


• If  = 0 : no noise, filter returns g(x,y)
2

• If 𝜎𝐿2 ≫ 𝜎𝜂2 : edge regions, filter returns value close to g(x,y)


• if L
2
  2
 : local area has the same properties as the overall

image, filter returns average value

10/25/2022 Le Thanh Ha 33
10/25/2022 Le Thanh Ha 34
 Adaptive median filter
• zmin = minimum intensity value in Sxy
• zmax = maximum intensity value in Sxy
• zmed = median intensity value in Sxy
• zxy = intensity value at (x,y)
• Smax = maximum allowed size of Sxy

State A: State B:
A1 = zmed - zmin B1 = zxy – zmin
A2 = zmed – zmax B2 = zxy – zmax
if A1>0 and A2<0, goto state B If B1>0 and B2<0, output zxy
Else increase the window size Else output zmed
if window size  Smax repeat state A
Else output zmed

10/25/2022 Le Thanh Ha 35
 Adaptive median filter
• Statistically consider zmin and zmax are “impulse-like”
components.
• State A determines if zmed is an impulse or not
– If YES is an impulse increase neighbor window Sxy and repeat.
– If NO, go to state B.
• State B determines if center pixel zxy is an impulse or not
– If YES, return zmed
– If NO, return zxy

10/25/2022 Le Thanh Ha 36
10/25/2022 Le Thanh Ha 37
 -Bandreject filters
-Notch filters

PERIODIC NOISE REDUCTION IN FREQUENCY

DOMAIN

10/25/2022 Le Thanh Ha 38
 Bandreject filters
 W
1 if D (u , v )  D −
• Ideal bandreject filter 

 W
0
2
W
H (u, v) = 0 if D0 −  D(u, v)  D0 −
 2 2
 W
1 if D (u , v )  D0 +
 2

• Butterworth bandreject filter

1
H (u, v) = 2n
 D(u, v)W 
1+  2 2
 D (u, v) − D0 

• Gaussian bandreject filter 1  D 2 ( u , v ) − D02 

2
−  
2  D ( u , v )W 
H (u, v) = 1 − e
10/25/2022 Le Thanh Ha 39
 Band Reject filters

10/25/2022 Le Thanh Ha 40
10/25/2022 Le Thanh Ha 41
 Notch filter
• A notch filter reject frequencies in predefined neighborhoods
about a center frequency.

10/25/2022 Le Thanh Ha 42
 - Linear, Position-invariant Degradations
- Estimating the degradation function
- Inverse Filtering
- Different approaches

DEGRADATION ESTIMATION

10/25/2022 Le Thanh Ha 43
 Linear, position-invariant Degradation
• The linear, position-invariant Degradation can be written in
convolution term:
g ( x, y ) = h( x, y )* f ( x, y ) +  ( x, y )
G (u, v) = H (u, v) F (u, v) + N (u, v)

where (x,y) is position-invariant noise, h(x,y) is a degradation model.

Many types of degradations can be approximated by this process.

Restoration seeks to find filters that apply the process in reverse

(Deconvolution filters)
10/25/2022 Le Thanh Ha 44
 Estimation by image observation

• Estimation by image observation

– Degradation system H is completely characterized by
its impulse response
– Select a small section from the degraded image
g s ( x, y )
– Reconstruct an unblurred image of the same size
fˆ ( x, y )
s
– The degradation function can be estimated by
Gs (u, v )
H s ( u, v ) =
Fˆs (u, v ) 45
 Estimation by experimentation
Point spread function (PSF)
– Used in optics
– The impulse becomes a point of light
– The impulse response is commonly referred to as
the PSF
G ( u, v )
H ( u, v ) =
A

46
 Estimation by modeling
• Atmospheric turbulence

H (u , v) = e − k ( u + v 2 )5 / 6
2

− k ( u 2 + v 2 )5 / 6
H (u , v) = e

47
 Estimation by modeling
• Linear motion blurring

 sin  (ua + vb)e − j ( ua +vb)

T
H ( u, v ) =
 (ua + vb)

 sin  (ua + vb)e − j ( ua +vb)

T
H ( u, v ) =
 (ua + vb)
48
 Inverse filtering

• Degradation model
g ( x, y ) = f ( x, y )  h ( x, y ) +  ( x, y )
G (u, v) = F (u, v) H (u, v) + N (u, v)

• Inverse filter

ˆ G (u , v) N (u , v)
F (u , v) = +
H (u , v) H (u, v)
N (u , v)
= F (u, v) +
H (u , v)
49