Spatial operations

Spatial operations

𝑔𝑔(𝑥𝑥, 𝑦𝑦) = 𝑇𝑇(𝑓𝑓(𝑥𝑥, 𝑦𝑦))

 Fundamentals of spatial filtering

Filtering:
1) Pixel neighborhood, e.g. a small rectangle
2) Operation, e.g., linear or nonlinear

𝑎𝑎 𝑏𝑏

𝑔𝑔 𝑥𝑥, 𝑦𝑦 = � � 𝑤𝑤 𝑠𝑠, 𝑡𝑡 𝑓𝑓(𝑥𝑥 + 𝑠𝑠, 𝑦𝑦 + 𝑡𝑡)

𝑠𝑠=−𝑎𝑎 𝑡𝑡=−𝑏𝑏

Spatial filters, masks, kernels, templates, windows

Averaging
Convolution with impulse
Correlation with impulse
Correlation
with impulse

Convolution
with impulse
 Spatial correlation and convolution

Correlation:
𝑎𝑎 𝑏𝑏

𝑤𝑤 𝑥𝑥, 𝑦𝑦 𝑓𝑓(𝑥𝑥, 𝑦𝑦) = � � 𝑤𝑤 𝑠𝑠, 𝑡𝑡 𝑓𝑓(𝑥𝑥 + 𝑠𝑠, 𝑦𝑦 + 𝑡𝑡)

𝑠𝑠=−𝑎𝑎 𝑡𝑡=−𝑏𝑏

Convolution:
𝑎𝑎 𝑏𝑏

𝑤𝑤 𝑥𝑥, 𝑦𝑦 𝑓𝑓(𝑥𝑥, 𝑦𝑦) = � � 𝑤𝑤 𝑠𝑠, 𝑡𝑡 𝑓𝑓(𝑥𝑥 − 𝑠𝑠, 𝑦𝑦 − 𝑡𝑡)

𝑠𝑠=−𝑎𝑎 𝑡𝑡=−𝑏𝑏
 Vector representation of linear filtering

𝑅𝑅 = 𝑤𝑤1 𝑧𝑧1 + 𝑤𝑤2 𝑧𝑧2 +…+ 𝑤𝑤𝑚𝑚𝑚𝑚 𝑧𝑧𝑚𝑚𝑚𝑚

𝑚𝑚𝑚𝑚

= � 𝑤𝑤𝑘𝑘 𝑧𝑧𝑘𝑘
𝑘𝑘=1
= 𝑤𝑤 𝑇𝑇 𝑧𝑧

𝑅𝑅 = 𝑤𝑤1 𝑧𝑧1 + 𝑤𝑤2 𝑧𝑧2 +…+ 𝑤𝑤9 𝑧𝑧9

9

= � 𝑤𝑤𝑘𝑘 𝑧𝑧𝑘𝑘
𝑘𝑘=1
= 𝑤𝑤 𝑇𝑇 𝑧𝑧
 Generating spatial filter masks
9
1
Averaging: 𝑅𝑅 = � 𝑧𝑧𝑖𝑖
9
𝑖𝑖=1

𝑥𝑥 2 +𝑦𝑦 2
2D Gaussian: ℎ(𝑥𝑥, 𝑦𝑦) =
−
𝑒𝑒 2𝜎𝜎2
3 × 3 mask:
𝑤𝑤1 = ℎ(−1, −1)
𝑤𝑤2 = ℎ(−1,0)
⋮
𝑤𝑤9 = ℎ(1,1)
 Smoothing filters: Averaging masks

Averaging Weighted
(box filter) averaging

∑𝑎𝑎𝑠𝑠=−𝑎𝑎 ∑𝑏𝑏𝑡𝑡=−𝑏𝑏 𝑤𝑤 𝑠𝑠, 𝑡𝑡 𝑓𝑓(𝑥𝑥 + 𝑠𝑠, 𝑦𝑦 + 𝑡𝑡)

𝑔𝑔 𝑥𝑥, 𝑦𝑦 =
∑𝑎𝑎𝑠𝑠=−𝑎𝑎 ∑𝑏𝑏𝑡𝑡=−𝑏𝑏 𝑤𝑤 𝑠𝑠, 𝑡𝑡
 Choosing the size of the mask

Original n=3

n=5 n=9

n=15 n=35
 Choosing the size of the mask

Original: Filtered by: Thresholding

Hubble telescope 15x15 aver. mask
image
 Nonlinear filters: Order statistics filters

0th percentile: minimum

the darkest pixels

50th percentile: median

eliminate impulse noise

100th percentile: maximum

the brightest pixels
Nonlinear filters: Max filter
 Nonlinear filters: Median filter

In this example:
On the image borders, the median is found after mirroring the
section under the window with respect to the central element
 Comparison of averaging and median filters

1 1

Image row
0
 Comparison of averaging and median filters

1 1

Image row
0
1
2/3
Averaging 1/3 1/3
 Comparison of averaging and median filters

1 1

Image row
0
1
2/3
Averaging 1/3 1/3
1

Median
 Comparison of averaging and median filters

Original: Filtered by: Filtered by:

X-ray image 3x3 aver. mask 3x3 median filter
of a circuit board
+
Salt and pepper
noise
 Sharpening filters

Smoothing: averaging  integration

Sharpening  differentiation
-1st derivative:
𝜕𝜕𝑓𝑓
= 𝑓𝑓 𝑥𝑥 + 1 − 𝑓𝑓(𝑥𝑥)
𝜕𝜕𝜕𝜕

-2nd derivative:

𝜕𝜕 2 𝑓𝑓
2
= 𝑓𝑓 𝑥𝑥 + 1 + 𝑓𝑓 𝑥𝑥 − 1 − 2𝑓𝑓(𝑥𝑥)
𝜕𝜕𝑥𝑥
Sharpening filters
 Implementation of second-order derivative

Laplacian: Simplest isotropic filter

𝑛𝑛
𝜕𝜕 2 𝑓𝑓
Δ𝑓𝑓 = 𝛻𝛻 2 𝑓𝑓 = 𝛻𝛻 ⋅ 𝛻𝛻𝛻𝛻 = � 2
𝜕𝜕𝑥𝑥𝑖𝑖
𝑖𝑖=1
 Implementation of second-order derivative

𝜕𝜕 2 𝑓𝑓
2
= 𝑓𝑓 𝑥𝑥 + 1, 𝑦𝑦 + 𝑓𝑓 𝑥𝑥 − 1, 𝑦𝑦 − 2𝑓𝑓(𝑥𝑥, 𝑦𝑦)
𝜕𝜕𝑥𝑥

𝜕𝜕 2 𝑓𝑓
2
= 𝑓𝑓 𝑥𝑥, 𝑦𝑦 + 1 + 𝑓𝑓 𝑥𝑥, 𝑦𝑦 − 1 − 2𝑓𝑓(𝑥𝑥, 𝑦𝑦)
𝜕𝜕𝑦𝑦

𝛻𝛻 2 𝑓𝑓 𝑥𝑥, 𝑦𝑦 = 𝑓𝑓 𝑥𝑥 + 1, 𝑦𝑦 + 𝑓𝑓 𝑥𝑥 − 1, 𝑦𝑦 + 𝑓𝑓 𝑥𝑥, 𝑦𝑦 + 1 + 𝑓𝑓 𝑥𝑥, 𝑦𝑦 − 1 − 4𝑓𝑓(𝑥𝑥, 𝑦𝑦)

 Laplacian masks

Negative difference
equations used

Isotropic for Isotropic for

90-degree increments 45-degree increments

Sharpening: 𝑔𝑔(𝑥𝑥, 𝑦𝑦) = 𝑓𝑓(𝑥𝑥, 𝑦𝑦) + 𝑐𝑐[𝛻𝛻 2 𝑓𝑓(𝑥𝑥, 𝑦𝑦)]

 Image sharpening

Blurred  Laplacian

Scaled Sharpened
Laplacian Image
 (c=-1)

 Simplification
𝑔𝑔(𝑥𝑥, 𝑦𝑦) = 𝑓𝑓(𝑥𝑥, 𝑦𝑦) − 𝛻𝛻 2 𝑓𝑓(𝑥𝑥, 𝑦𝑦)
= 𝑓𝑓(𝑥𝑥, 𝑦𝑦) − [𝑓𝑓(𝑥𝑥 + 1, 𝑦𝑦) + 𝑓𝑓(𝑥𝑥 − 1, 𝑦𝑦) + 𝑓𝑓(𝑥𝑥, 𝑦𝑦 + 1) + 𝑓𝑓(𝑥𝑥, 𝑦𝑦 − 1)] + 4𝑓𝑓(𝑥𝑥, 𝑦𝑦)
= 5𝑓𝑓(𝑥𝑥, 𝑦𝑦) − [𝑓𝑓(𝑥𝑥 + 1, 𝑦𝑦) + 𝑓𝑓(𝑥𝑥 − 1, 𝑦𝑦) + 𝑓𝑓(𝑥𝑥, 𝑦𝑦 + 1) + 𝑓𝑓(𝑥𝑥, 𝑦𝑦 − 1)]

Scanning
Electron
Image


Without With
Diagonal Diagonal
Elements Elements
 
 Unsharp masking and high-boost filtering

̄ 𝑦𝑦)
𝑔𝑔𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 (𝑥𝑥, 𝑦𝑦) = 𝑓𝑓(𝑥𝑥, 𝑦𝑦) − 𝑓𝑓(𝑥𝑥,
Blurred

Sharpened
 𝑔𝑔(𝑥𝑥, 𝑦𝑦) = 𝑓𝑓(𝑥𝑥, 𝑦𝑦) + 𝑘𝑘 ∗ 𝑔𝑔𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 (𝑥𝑥, 𝑦𝑦)
𝑘𝑘 > 0
𝑘𝑘 = 1  Unsharp masking
𝑘𝑘 > 1  High-boost filtering
𝑘𝑘 < 1  Deemphasize mask
Unsharp masking
 First-order derivatives

𝜕𝜕𝜕𝜕
𝑔𝑔𝑥𝑥
Gradient: 𝛻𝛻𝛻𝛻 = 𝑔𝑔
𝜕𝜕𝜕𝜕
= 𝜕𝜕𝜕𝜕
𝑦𝑦
𝜕𝜕𝜕𝜕

Gradient image: 𝑀𝑀(𝑥𝑥, 𝑦𝑦) = 𝑚𝑚𝑚𝑚𝑚𝑚(𝛻𝛻𝛻𝛻) = 𝑔𝑔𝑥𝑥2 + 𝑔𝑔𝑦𝑦2

𝑀𝑀(𝑥𝑥, 𝑦𝑦) ≈ 𝑔𝑔𝑥𝑥 + 𝑔𝑔𝑦𝑦

 Gradient masks
Roberts cross-gradient operators:
𝑔𝑔𝑥𝑥 = 𝑧𝑧9 − 𝑧𝑧5
𝑔𝑔𝑦𝑦 = 𝑧𝑧8 − 𝑧𝑧6

2 2 1/2
𝑀𝑀(𝑥𝑥, 𝑦𝑦) = 𝑧𝑧9 − 𝑧𝑧5 + 𝑧𝑧8 − 𝑧𝑧6

𝑀𝑀(𝑥𝑥, 𝑦𝑦) ≈ 𝑧𝑧9 − 𝑧𝑧5 + 𝑧𝑧8 − 𝑧𝑧6

 Gradient masks
Sobel operators:
𝜕𝜕𝜕𝜕
𝑔𝑔𝑥𝑥 = = (𝑧𝑧7 + 2𝑧𝑧8 + 𝑧𝑧9 )- (𝑧𝑧1 + 2𝑧𝑧2 + 𝑧𝑧3 )
𝜕𝜕𝜕𝜕
𝜕𝜕𝜕𝜕
𝑔𝑔𝑦𝑦 = = (𝑧𝑧3 + 2𝑧𝑧6 + 𝑧𝑧9 )- (𝑧𝑧1 + 2𝑧𝑧4 + 𝑧𝑧7 )
𝜕𝜕𝜕𝜕

𝑀𝑀(𝑥𝑥, 𝑦𝑦) ≈ | 𝑧𝑧7 + 2𝑧𝑧8 + 𝑧𝑧9 − (𝑧𝑧1 + 2𝑧𝑧2 + 𝑧𝑧3 ) |

+ | 𝑧𝑧3 + 2𝑧𝑧6 + 𝑧𝑧9 − (𝑧𝑧1 + 2𝑧𝑧4 + 𝑧𝑧7 ) |
 Gradient application
Industrial inspection:

Contact lens Sobel operators used to

highlight defects
(a) Bone scan (b) Laplacian (c)Sharpened (d) Sobel of (a)
(e) Smoothing (f) Product of (g) Sum of (g) power-law
of (d) by (c) and (e) (a) and (f) transformation
5x5 averaging of (g)