0% found this document useful (0 votes)
43 views47 pages

Chapter5 Image Restoration: I Fil I Inverse Filtering Wiener Filtering

Chapter 5 discusses image restoration techniques. It introduces the goals of image restoration as compensating for defects that degrade an image. Common causes of image degradation include atmospheric turbulence, motion blur, and noise. Image restoration methods include unconstrained restoration using inverse filtering and constrained restoration using Wiener filtering. The chapter models the image degradation process using a linear shift-invariant system and expresses it in matrix form. It describes how the degradation matrix can be diagonalized through eigendecomposition to simplify the restoration process.

Uploaded by

Umme Sana
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
43 views47 pages

Chapter5 Image Restoration: I Fil I Inverse Filtering Wiener Filtering

Chapter 5 discusses image restoration techniques. It introduces the goals of image restoration as compensating for defects that degrade an image. Common causes of image degradation include atmospheric turbulence, motion blur, and noise. Image restoration methods include unconstrained restoration using inverse filtering and constrained restoration using Wiener filtering. The chapter models the image degradation process using a linear shift-invariant system and expresses it in matrix form. It describes how the degradation matrix can be diagonalized through eigendecomposition to simplify the restoration process.

Uploaded by

Umme Sana
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 47

Chapter5 Image Restoration

• Preview
• 5 1 Introduction
5.1 I d i
• 5.2 Diagonalization
• 5.3 Unconstrained
i d Restoration(
i ( inverse
i filtering)
fil i )
• 5.4 Constrained Restoration( wiener filtering)
• 5.5 Estimating the Degradation Function
• 5.6 Geometric Distortion Correction
• 5.7 image inpaiting

2003 Digital Image Processing 1


Prof.Zhengkai Liu Dr.Rong Zhang
5.1 Introduction
5.1.1 Purpose
co pe s e for"
"compensate o oro "undo"
u do defects
de ec s which
w c degrade
deg de an image.
ge.
5.1.2 Degrade Causes
((1)) atmospheric
h i turbulence
b l
(2) sampling, quantization p254
(3) motion blur
(4) camera misfocus
(5) noise

2003 Digital Image Processing 2


Prof.Zhengkai Liu Dr.Rong Zhang
5.1 Introduction
5.1.3 degradation model
n(x,y)

f( )
f(x,y) H + g(x y)
g(x,y)

Assume it is a linear, position- invariant system, We can model


a blurred image by

g ( x, y ) = f ( x, y ) * h ( x, y ) + n ( x, y )
h h(x,y)
Where h( ) is
i called
ll d as Point
i Spreadd Function
i (PSF)
( )
M −1 N −1
g ( x, y ) = ∑∑ f (m, n)h( x − m, y − n) + n( x, y)
m =0 n =0
2003 Digital Image Processing 3
Prof.Zhengkai Liu Dr.Rong Zhang
5.1 Introduction
5.1.4 Methods

Unconstrained Restoration: inverse filtering


Constrained Restoration: wiener filtering

5.1.5 problem expression

Estimate a true image f(x,y) from a degraded image g(x,y)


b d on prior
based l d off PSF h(x,y)and
i knowledge
k h( ) d the h statistical
i i l
properties of noise n(x,y)

2003 Digital Image Processing 4


Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.1 Matrix expression of degradation model: 1-D

g ( x ) = f ( x ) * h( x )

⎧ f ( x) 0 ≤ x ≤ A − 1
f e ( x) = ⎨
⎩ 0 else

⎧ h( x ) 0 ≤ x ≤ B − 1
he ( x) = ⎨
⎩ 0 else

2003 Digital Image Processing 5


Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.1 Matrix expression of degradation model: 1-D

M −1 M=A+B-1
g e ( x) = ∑ f e (m)he ( x − m) + ne ( x)
m= 0 xx=00,1,…,M
1 M-11

⎡ g e (0) ⎤ ⎡ he (0) he (−1) " he (− M + 1) ⎤ ⎡ f e (0) ⎤ ⎡ ne (0) ⎤


⎢ g (1) ⎥ ⎢ h (1) he (0) " he (− M + 2) ⎥⎥ ⎢⎢ f e (1) ⎥⎥ ⎢⎢ ne (1) ⎥⎥
g = Hf + n = ⎢ e ⎥=⎢ e +
⎢ # ⎥ ⎢ # # % # ⎥⎢ # ⎥ ⎢ # ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥
⎣⎢ g e ( M -1) ⎦⎥ ⎣⎢ he ( M − 1) he ( M − 2) " he (0) ⎦⎥ ⎣⎢ f e ( M -1) ⎦⎥ ⎣⎢ ne ( M − 1) ⎦⎥

2003 Digital Image Processing 6


Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.1 Matrix expression of degradation model: 1-D

he ( x) = he ( x + M )
⎡ he (0) he ( M − 1) " he (1) ⎤
⎢ h (1) he (0) " he (2) ⎥⎥
H= ⎢ e

⎢ # # % # ⎥
⎢ ⎥
⎢⎣ he ( M − 1) he ( M − 2) " he (0) ⎥⎦

H is circulant

2003 Digital Image Processing 7


Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.1 Matrix expression of degradation model: 2-D

⎧ f ( x, y ) 0 ≤ x ≤ A − 1 and 0 ≤ y ≤ B − 1
f e ( x) = ⎨
⎩ 0 A ≤ x ≤ M − 1 or B ≤ y ≤ N − 1

⎧ h ( x, y ) 0 ≤ x ≤ C − 1 and 0 ≤ y ≤ D − 1
he ( x) = ⎨
⎩ 0 A ≤ x ≤ M − 1 or B ≤ y ≤ N − 1

2003 Digital Image Processing 8


Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.1 Matrix expression of degradation model: 2-D

M −1 N −1
x=0,1,…,M-1
g e ( x, y ) = ∑ ∑ f e (m, n)he ( x − m, y − n)
m =0 n =0
yy=00,1,…,N
1 N-11

⎡ H0 H M −1 " H1 ⎤ ⎡ f e (0) ⎤ ⎡ ne (0) ⎤


⎢ H H 0 " H 2 ⎥⎥ ⎢⎢ f e (1) ⎥⎥ ⎢⎢ ne (1) ⎥⎥
g = Hff + n = ⎢ 1 +
⎢ # # % # ⎥⎢ # ⎥ ⎢ # ⎥
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢⎣ H M −1 H M − 2 " H 0 ⎥⎦ ⎢⎣ f e ( MN -1) ⎥⎦ ⎢⎣ ne ( MN − 1) ⎥⎦

H is block-circulant
2003 Digital Image Processing 9
Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.1 Matrix expression of degradation model: 2-D

where
⎡ he (i, 0) he (i, N − 1) 1) ⎤
" he (i,1)
⎢ h (i,1) he (i, 0) " he (i, 2) ⎥⎥
Hi = ⎢ e
⎢ # # % # ⎥
⎢ ⎥
⎢⎣ he (i, N − 1) he (i, N − 2) " he (i, 0) ⎥⎦

2003 Digital Image Processing 10


Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.2 Diagonliztion: 1-D
The eigenvector
g and eigenvalue
g of a circulant matrix H are
T
⎡ ⎛ 2π ⎞ ⎛ 2π ⎞⎤
w(k ) = ⎢1 exp ⎜ j k ⎟ " exp ⎜ j ( M − 1)k ⎟ ⎥
⎣ ⎝ M ⎠ ⎝ M ⎠⎦

⎛ 2π j ⎞ ⎛ 2π j ⎞
λ (k ) = he (0) + he ( M − 1) exp ⎜ − k ⎟ + " + he (1) exp ⎜ − ( M − 1)k ⎟
⎝ M ⎠ ⎝ M ⎠
Combine the M eigenvectors
g to a matrix
W = [ w(0) w(1) " w( M − 1)]
then the H can be expressed as
H = WDW −1 where D(k , k ) = λ (k )

2003 Digital Image Processing 11


Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.2 Diagonliztion: 1-D

for g = Hf + n

W −1 g = W −1 Hf + W −1n
= W −1WDW −1 f + W −1n
= DW −1 f + W −1n

G (u ) = H (u ) F (u ) + N (u )

2003 Digital Image Processing 12


Prof.Zhengkai Liu Dr.Rong Zhang
5.2 Diagonalization
5.2.2 Diagonliztion: 2-D

⎛ 2π ⎞
W (i, m) = exp ⎜ j im ⎟ WN
⎝ M ⎠

⎛ 2π ⎞
WN (k , n) = exp ⎜ j kn ⎟
⎝ N ⎠

H = WDW −1 G (u , v) = H (u , v) F (u , v) + N (u, v)

g(x,y) = f(x,y)*h(x,y) +n(x,y) in Spatial Coordinates


G(u,v) = F(u,v)H(u,v) +N(u,v) in frequency domain
g = Hf + n in vector form
2003 Digital Image Processing 13
Prof.Zhengkai Liu Dr.Rong Zhang
5.3 Inverse Filtering
5.3.1 assumption
H is
i given,
i andd the
th noise
i isi negligible
li ibl

5.3.2 degradation model

F(u,v) H G( )
G(u,v)

G (u , v) = H (u, v) F (u , v)

2003 Digital Image Processing 14


Prof.Zhengkai Liu Dr.Rong Zhang
5.3 Inverse Filtering
5.3.3 restoration
G (u, v)
F (u , v) =
ˆ = G (u , v) H I (u, v)
H (u, v)

1 deconvolution
H I (u, v) =
H (u , v)

5.3.4 properties G (u , v)
Fˆ (u , v) =
H (u , v)
H (u , v) F (u , v) + N (u , v)
=
H (u, v)
N (u , v)
= F (u, v) +
H (u, v)

2003 Digital Image Processing 15


Prof.Zhengkai Liu Dr.Rong Zhang
5.3 Inverse Filtering
5.3.4 properties

sensitive to additive noise: if H(u,v) has zero or very small value,


the N(u,v)/H(u,v) could easily dominate the estimate

5.3.5 improvements
p

H(u,v) ⎧1/ H (u, v) if u 2 + v 2 < w02


M (u , v) = ⎨
⎩ 1 else

or
⎧k if H (u, v) < d
u,v M (u , v) = ⎨
⎩1/ H (u , v) else

2003 Digital Image Processing 16


Prof.Zhengkai Liu Dr.Rong Zhang
5.3 Inverse Filtering
5.3.4 examples: without noise

Original image Blurred image Restored image

2003 Digital Image Processing 17


Prof.Zhengkai Liu Dr.Rong Zhang
5.3 Inverse Filtering
5.3.4 examples: with noise

Blurred and noised image Restored image

2003 Digital Image Processing 18


Prof.Zhengkai Liu Dr.Rong Zhang
5.3 Inverse Filtering
5.3.4 examples: deferent cutoff

2003 Digital Image Processing 19


Prof.Zhengkai Liu Dr.Rong Zhang
5 4 Wiener Filtering
5.4
5.4.1 assumption

H is given, and consider the image and noise as random processes


55.4.2
4 2 request
The mean square error between uncorrupted image and estimated
image is minimized.
minimized This error measure is given by

e =E (f − f)
2 ˆ {2
}

2003 Digital Image Processing 20


Prof.Zhengkai Liu Dr.Rong Zhang
5 4 Wiener Filtering
5.4
5.4.3 restoration

Fˆ (u , v) = G (u , v) HW (u , v)

1 H (u , v)
2 Wi
Wiener filter,
filt
HW (u , v) = × 1942
H (u, v) H (u , v) 2 + S n (u , v) / S f (u, v)

H (u , v)*
=
H (u , v) + S n (u , v) / S f (u , v)
2

where
Sn (u , v) = N (u, v)
2
Power spectrum of the noise

S f (u, v) = F (u , v)
2
Power spectrum of the undegraded image

2003 Digital Image Processing 21


Prof.Zhengkai Liu Dr.Rong Zhang
5 4 Wiener Filtering
5.4
5 4 5 estimate the power spectrum
5.4.5 Sf(u,v)=
(u v)= Sg(u,v)
(u v) / |H(u
|H(u,v)|
v)|

Sn(u,v)

S g (u , v) = H (u , v) S f (u , v) + S n (u, v)
2

u,v
Sg(u,v)
5.4.4 properties
•optimal in terms of the mean square error

Wh H(u,v)=0
•When H( ) 0, Hw(u,v)=0
( ) 0

2003 Digital Image Processing 22


Prof.Zhengkai Liu Dr.Rong Zhang
5.4 Wiener Filtering
5.4.5 experimental results:

2003 Digital Image Processing 23


Prof.Zhengkai Liu Dr.Rong Zhang
5.4 Wiener Filtering
5 4 5 experimental results:
5.4.5
The first raw:
Image corrupted by
motion blur and
additive noise

The second raw:


R l off inverse
Results i filtering
fil i

The third raw:


Results of Wiener filtering

2003 Digital Image Processing 24


Prof.Zhengkai Liu Dr.Rong Zhang
5 5Estimating the Degradation Function
5.5Estimating
(blind deconvolution)

5.5.1 Estimation by image observation


sub image g s ( x, y )
(1) Choose observed sub-image

(2) Denote the constructed sub-image as fˆs ( x, y )

(3) Assume noise is negligible


then
Gs (u , v)
H s (u , v) =
Fˆs (u , v)

2003 Digital Image Processing 25


Prof.Zhengkai Liu Dr.Rong Zhang
5 5Estimating the Degradation Function
5.5Estimating
5.5.2 Estimation by experimentation

If f(x,y)=δ(x,y)
δ(x y)
δ(x,y) h(x y)
h(x,y) g(x y)
g(x,y) Impulse
response
then
g ( x, y ) = δ ( x, y ) * h ( x, y ) = h ( x, y )

2003 Digital Image Processing 26


Prof.Zhengkai Liu Dr.Rong Zhang
5 5Estimating the Degradation Function
5.5Estimating
5.5.3 Estimation by modeling
An atmospheric turbulence model based on the
physical characteristics
Hufnagel and
− k ( u 2 + v 2 )5 / 6
Stanley, 1964
H (u , v) = e
where k is a constant

it has the same form as the Gaussian lowpass filter

2003 Digital Image Processing 27


Prof.Zhengkai Liu Dr.Rong Zhang
5.5Estimating the Degradation Function
5.5.3 Estimation by modeling

2003 Digital Image Processing 28


Prof.Zhengkai Liu Dr.Rong Zhang
5 5Estimating the Degradation Function
5.5Estimating
5.5.3 Estimation by modeling :restoration of uniform linear motion

If T is the duration of the exposure, the effect of image motion


follows that

g ( x, y ) = ∫ f [ x − x0 (t ), y − y0 (t ) ] dt
T

where x0(t) and y0(t) are the time varying components of


motion
i in h x-direction
i the di i andd y-direction.
di i Its Fourieri transform
f is
i
∞ ∞
G (u , v) = ∫ ∫ g ( x, y )e −2π j (ux + vy ) dxdy
−∞
∞ −∞

= ∫ ∫ ∫ f [ x − x0 (t ), y − y0 (t )]dt ⎤ e −2π j (ux + vy ) dxdy



∞ T ∞

−∞ −∞ ⎢
⎣ 0 ⎥⎦
2003 Digital Image Processing 29
Prof.Zhengkai Liu Dr.Rong Zhang
5 5Estimating the Degradation Function
5.5Estimating
5.5.3 Estimation by modeling :restoration of uniform linear motion

Reversing the order of integration:

G (u, v) = ∫
T
⎡ ∞ ∞ f [ x − x (t ), y − y (t )]e −2π j (ux + vy ) dxdy ⎤dt
0 ⎢⎣ ∫−∞ ∫−∞ 0 0 ⎥⎦

Using the translation Properties of Fourier transformation, then

T
G (u , v) = ∫ F (u , v)e−2π j[ux0 (t ) + vy0 (t )]dt
0

T
= F (u , v) ∫ e−2π j[ux0 (t ) + vy0 (t )]dt
0

2003 Digital Image Processing 30


Prof.Zhengkai Liu Dr.Rong Zhang
5 5Estimating the Degradation Function
5.5Estimating
5.5.3 Estimation by modeling :restoration of uniform linear motion
T
By defining H (u, v) = ∫ e −2π j[ux0 (t ) + vy0 (t )]dt
0

then G (u , v) = H (u, v) F (u , v)
Suppose
pp that the image
g in qquestion undergoes
g uniform linear
motion in the x-direction only, at a rate given by x0 (t ) = at / T
T
H (u , v) = ∫ e −2π j[ux0 (t )+ vy0 (t )]dt
0
T
= ∫ e −2π juat / T dt
0

T
= sin(π ua)e− jπ ua
π ua
2003 Digital Image Processing 31
Prof.Zhengkai Liu Dr.Rong Zhang
5 5Estimating the Degradation Function
5.5Estimating
5.5.3 Estimation by modeling :restoration of uniform linear motion
If we allow the y-component to wary as well with the motion
given by y0 (t ) = bt / T then the degradation function becomes
T
H (u, v) = sin[π (ua + vb)]e− jπ (ua + vb )
π (ua + vb)

2003 Digital Image Processing 32


Prof.Zhengkai Liu Dr.Rong Zhang
2003 Digital Image Processing 33
Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Transformation
5.6
5.6.introduction

f(x,y) T g(x,y)

g ( x, y ) = T [( f ( x, y )] = f ( x ', y ')
where
x ' = r ( x, y ) y ' = s ( x, y )
for example: zoom out x' = x/ 2 y' = y / 2
zoom in x ' = 2x y ' = 2 y

2003 Digital Image Processing 34


Prof.Zhengkai Liu Dr.Rong Zhang
2003 Digital Image Processing 35
Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Distortion Correction
5.6
5.6.1 introduction

A geometric transformation consists of two basic operations:


(1) Spatial transformation
. . . .
(2) Gray-level interpolation . . . .

.
(x,y)
空间变换

灰度赋值
.. ( x, , y, ) Direct transform

最近邻 Inverse transform


, ,
f(x,y) g (x , y )

2003 Digital Image Processing 36


Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Distortion Correction
5.6
5.6.2 spatial transformations

Linear correction

r ( x, y ) = a1 x + a2 y + a3
s ( x, y ) = b1 x + b2 y + b3

Quadratic correction

r ( x, y ) = a1 x 2 + a2 y 2 + a3 xy + a4 x + a5 y + a6
s ( x, y ) = b1 x 2 + b2 y 2 + b3 xy + b4 x + b5 y + b6

2003 Digital Image Processing 37


Prof.Zhengkai Liu Dr.Rong Zhang
2003 Digital Image Processing 38
Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Distortion Correction
5.6
5.6.3 gray-level interpolation
•Nearest neighbor interpolation
f(i,j)
f(i j+1)
f(i,j+1)
g ( x, y ) = f ( x ', y ')
= f (i + u , j + v)
(i j+1)
(i,j+1)
u, v ∈ (0,1) i, j ∈ Z v f(i+1,j+1)
(i,j)
u
f(i+1,j)
(i+1,j+1)
x ' = round ( x ') y ' = round ( y ')
(i 1 j)
(i+1,j)

2003 Digital Image Processing 39


Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Distortion Correction
5.6
5.6.3 gray-level interpolation f(x’,y’)

•Bi-linear interpolation
f(i,j)
A
f ( x ',' y ') = f (i + u , j + v) (x’,y’)
f(i j+1)
f(i,j+1)

u, v ∈ (0,1) i, j ∈ Z
(i j+1)
(i,j+1)
= (1 − u )(1 − v) f (i, j ) + (1 − u )vf (i, j + 1) f(i+1,j+1)
(i,j) B
+ u (1 − v) f (i + 1,
1 j ) + uvf (i + 1,
1 j + 1) f(i+1,j)
(i+1,j+1)
It can be operated by a mask
(i 1 j)
(i+1,j)
(1 − u )(1 − v) (1 − u )v
u (1 − v) uv

2003 Digital Image Processing 40


Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Distortion Correction
5.6
5.6.3 gray-level interpolation Y

Rotation in 2D plane (x',y')


(x,y)
x ' = r cos(α + θ ) = r cos α cosθ − r sin α sin θ θ
α
y ' = r sin(α + θ ) = r cos α sin θ + r sin α cos θ X

The original coordinated of the point in polar coordinates are:

x = r cos α y = r sin α

2003 Digital Image Processing 41


Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Distortion Correction
5.6
5.6.3 gray-level interpolation
Notes:
Rotation in 2D plane 1. The origin is in
the center of
x ' = x cos θ − y sin θ
image and the
y ' = x sin θ + y cos θ direction of x-
axis is opposite
2. The result
x image’s size is
depended on the
rotate angle

y θ = 45D 3. Interpolation
p is
2003 Digital Image Processing
needed 42
Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Distortion Correction
5.6
5.6.4 experimental results

grid Changed grid

spatial transform result


2003 Digital Image Processing 43
Prof.Zhengkai Liu Dr.Rong Zhang
5 6 Geometric Distortion Correction
5.6
5.6.4 experimental results

Spatial transform Nearest neighborhood Bi- linear


i
interpolation
l i interpolation

2003 Digital Image Processing 44


Prof.Zhengkai Liu Dr.Rong Zhang
Summary
• Inverse filtering is a very easy and accurate way to
restore an image provided that we know what the
bl i filter
blurring filt isi andd that
th t we have
h no noise
i
•Wiener filtering is the optimal tradeoff between the inverse
filtering and noise smoothing

• It is
i possible
ibl to
t restore
t an image
i without
ith t having
h i specific ifi
knowledge of degradation filter and additive noise. However, not
knowing the degradation filter h imposes the strictest limitations
on our restoration capabilities.

2003 Digital Image Processing 45


Prof.Zhengkai Liu Dr.Rong Zhang
homework
• 画出图象变质及图象恢复模型的方框图,
数学公式表示及说明其物理意义。
• 列出逆滤波和维纳滤波图象恢复的具体
步骤。
• 推导水平匀速直线运动模糊的点扩展函
数的数学公式并画出曲线。
• 写出图像任意角度旋转算法

2003 Digital Image Processing 46


Prof.Zhengkai Liu Dr.Rong Zhang
The End

2003 Digital Image Processing 47


Prof.Zhengkai Liu Dr.Rong Zhang

You might also like