15/03/2023

Image Processing
INT3404 1/ INT3404E 21
Giảng viên: TS. Nguyễn Thị Ngọc Diệp
Email: [email protected]

Schedule
Tuần Nội dung Yêu cầu đối với sinh viên (ngoài việc đọc tài liệu tham khảo)
1
Giới thiệu môn học Cài đặt môi trường: Python 3, OpenCV 3, Numpy, Jupyter Notebook

2 Ảnh số (Digital image) – Phép toán điểm (Point operations)

Làm quen với OpenCV + Python
3 Làm bài tập 1: điều chỉnh gamma tìm contrast hợp lý
Điều chỉnh độ tương phản (Contrast adjust)– Ghép ảnh (Combining images)
4 Histogram - Histogram equalization Thực hành ở nhà
5 Phép lọc trong không gian điểm ảnh (linear processing filtering) Thực hành ở nhà
6 Phép lọc trong không gian điểm ảnh cont. (linear processing filtering)
Bài tập mid-term
Thực hành: Ứng dụng của histogram; Tìm ảnh mẫu (Template matching)
7 Trích rút đặc trưng của ảnh
Thực hành ở nhà
Cạnh (Edge) và đường (Line) và texture
8 Các phép biến đổi hình thái (Morphological operations) Làm bài tập 2: tìm barcode
9 Chuyển đổi không gian – Miền tần số – Phép lọc trên miền tần số
Đăng ký thực hiện đồ án môn học
Thông báo liên quan đồ án môn học
10 Xử lý ảnh màu (Color digital image) Làm bài tập 3: Chuyển đổi mô hình màu và thực hiện phân vùng
11 Thực hành ở nhà
Các phép biến đổi hình học (Geometric transformations)
12 Nhiễu – Mô hình nhiễu – Khôi phục ảnh (Noise and restoration) Thực hành ở nhà
13 Nén ảnh (Compression) Thực hành ở nhà
14 Hướng dẫn thực hiện đồ án môn học Trình bày đồ án môn học
15 Hướng dẫn thực hiện đồ án môn học
Trình bày đồ án môn học
Tổng kết cuối kỳ 2

1
 15/03/2023

Week 7: Feature extraction

Edge, Line, Texture

Edge detection

2
 15/03/2023

What is an edge? 10.2 Point, Line, and Edge Detection 705

a b
c
FIGURE 10.2
(a) Image.
(b) Horizontal
- An edge = a significant local change
intensity profile
that includes the
in the image intensity isolated point
indicated by the
arrow.
(c) Subsampled
profile; the dashes
were added
for clarity. The
numbers in the
boxes are the
intensity values
of the dots shown
in the profile. The
derivatives were 7 Isolated point
obtained using 6
Intensity

Eqs. (10-4) for the 5

Ramp Step
first derivative 4 Line
and Eq. (10-7) for 3 Flat segment
the second. 2
1
0
Intensity values 5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7

Image credit: Gonzalez et.al., Fig. 10.2 First derivative !1!1!1!1!1 0 0 6 !6 0 0 0 1 2 !2!1 0 0 0 7 0 0 0

Second derivative !1 0 0 0 0 1 0 6 !12 6 0 0 1 1 !4 1 1 0 0 7 !7 0 0

5
in the ramp spans four pixels, the noise point is a single pixel, the line is three pixels
thick, and the transition of the step edge takes place between adjacent pixels. The
number of intensity levels was limited to eight for simplicity.
Consider the properties of the first and second derivatives as we traverse the
profile from left to right. Initially, the first-order derivative is nonzero at the onset
and along the entire intensity ramp, while the second-order derivative is nonzero
only at the onset and end of the ramp. Because the edges of digital images resemble
this type of transition, we conclude that first-order derivatives produce “thick” edges,
and second-order derivatives much thinner ones. Next we encounter the isolated

Image gradient
noise point. Here, the magnitude of the response at the point is much stronger for
the second- than for the first-order derivative. This is not unexpected, because a
second-order derivative is much more aggressive than a first-order derivative in
enhancing sharp changes. Thus, we can expect second-order derivatives to enhance
fine detail (including noise) much more than first-order derivatives. The line in this
example is rather thin, so it too is fine detail, and we see again that the second deriva-
tive has a larger magnitude. Finally, note in both the ramp and step edges that the

DIP4E_GLOBAL_Print_Ready.indb 705 6/16/2017 2:12:41 PM

Two types of gradients, with blue arrows to indicate the

direction of the gradient. Dark areas indicate higher values

3
 15/03/2023

Gradient
• Gradient of a function indicates how strong the function
increases.
• For 1-dimension function: f ( x) = x 2
¶f ( x)
Grad ( x) = = 2x
¶ ( x)
• Grad(2)=4 indicates the the increasing direction of the function is
to the right.
• Grad(-1)=-2 indicates the increasing direction of the function is to
the left.

First-order and second-order derivatives

Image credit: Gonzalez, Fig. 3.44

4
 15/03/2023

Edge detection using derivatives

(1) Detecting the local maxima or
minima of the first derivative

(2) Detecting the zero-crossings

of the second derivative

Image credit: MIPAV

Gradient of 2D discrete function

• Gradient of a 2-dimension function is calculated as
follows:
¶f ( x, y) ® ¶f ( x, y) ®
Grad ( x, y) = i+ j
¶x ¶y
• The gradient is approximated as follows (first-order
derivative) :

¶f ( x, y) ¶f ( x, y)
= f ( x + 1, y) - f ( x, y), = f ( x, y + 1) - f ( x, y)
¶x ¶y

10

5
 15/03/2023

Gradient of 2D image

convolution

Approximated by finite differences

Avoid shifting of image

11

Gradient
• The magnitude of gradient indicates the strong of edges:

2 2
æ ¶f ( x, y ) ö æ ¶f ( x, y ) ö
Grad ( x, y ) = ç ÷ +ç ÷
è ¶y ø è ¶x ø
• Gradient computation procedure:
• Calculate column gradient
• Calculate row gradient
• Calculate final gradient by the above function

12

6
 15/03/2023

10.2 Point, Line, and Edge Detection 719

a
b c z1 z2 z3

Various kernels used to compute the gradient

d e
f g z4 z5 z6
FIGURE 10.14
A 3 × 3 region
z7 z8 z9
of an image (the Line, and Edge Detection
10.2 Point, 719
z’s are intensity
values), and
various kernels !1 0 0 !1
z1 usedz2 to compute
z3
the gradient at the 0 1 1 0
point labeled z5 .
z4 z5 z6
Roberts

z7 z8 z9 !1 !1 !1 !1 0 1

A 3x3
!1 region
0 0of an
!1image
0 0 0 !1 0 1

1 1 1 !1 0 1
0 1 1 0
Prewitt
Roberts
!1 !2 !1 !1 0 1
!1 !1 !1 !1 0 1
0 0 0 !2 0 2
0 0 0 !1 0 1
1 2 1 !1 0 1

1 1 1 !1 0 1 Sobel

Prewitt

!1 !2 !1 !1 0 1 ∂f
gx = = (z7 + 2z8 + z9 ) − (z1 + 2z2 + z3 ) (10-24)
∂x
0 0 0 !2 0 2
and
∂f
1 2 1 !1 0 1 gy = = (z3 + 2z6 + z9 ) − (z1 + 2z4 + z7 ) (10-25)
13 ∂y
Sobel It can be demonstrated (see Problem 10.12) that using a 2 in the center location pro-
vides image smoothing. Figures 10.14(f) and (g) show the kernels used to implement
Eqs. (10-24) and (10-25). These kernels are called the Sobel operators (Sobel [1970]).
The Prewitt kernels are simpler to implement than the Sobel kernels, but the
∂f
gx = = (z7 + 2z8 + z9 ) − (z1 + 2z2slight
+ z3 ) computational(10-24)
difference between them typically is not an issue. The fact
∂x that the Sobel kernels have better noise-suppression (smoothing) characteristics
makes them preferable because, as mentioned earlier in the discussion of Fig. 10.11,
nd
noise suppression is an important issue when dealing with derivatives. Note that the
∂f
gy = = (z3 + 2z6 + z9 ) − (z1 + 2z4 + z7 ) (10-25)
∂y

Pixel Difference masks

t can be demonstrated (see Problem 10.12) that using a 2 in the center location pro-
ides image smoothing. Figures 10.14(f) and (g) show the kernels used to implement
Eqs. (10-24) and (10-25). These kernels are called the Sobel operators (Sobel [1970]).
The Prewitt kernels are simpler to implement than the Sobel kernels, but the
DIP4E_GLOBAL_Print_Ready.indb 719 6/16/2017 2:12:51 PM
light computational difference between them typically is not an issue. The fact
hat the Sobel kernels have better noise-suppression (smoothing) characteristics
makes them preferable because, as mentioned earlier in the discussion of Fig. 10.11,
noise suppression is an important issue when dealing with derivatives. Note that the
Column mask Row mask

é0 0 0 ù é0 -1 0 ù
ê0 1 -1ú ê0 1 0 ú
ê ú
6/16/2017 2:12:51 PM ê ú
êë0 0 0 úû êë0 0 0úû

14

7
 15/03/2023

Pixel difference example

Final edges
Column
Original Image
edges

Row
edges

15

Robert mask
• Roberts masks calculate gradient from two diagonals

Column Row

é0 0 -1ù é -1 0ù
ê0 1 0 ú ê 0 1 0ú
ê ú ê ú
êë0 0 0 úû êë 0 0 0 úû

16

8
 15/03/2023

Robert mask example

Column
edges
Original Image Final edges

Row
edges

17

Prewitt mask

Column Row

é1 0 -1ù é -1 -1 -1ù
1ê 1ê
ê 1 0 -1úú ê 0 0 0 úú
3 3
êë1 0 -1úû êë 1 1 1 úû

18

9
 15/03/2023

Prewitt mask
Row
edges
Original Image Final edges

Column
edges

19

Sobel mask
Column Row

é1 0 -1ù é -1 -2 -1ù
1ê 1ê
ê 2 0 -2 úú ê 0 0 0 úú
4 4
êë1 0 -1úû êë 1 2 1 úû

20

10
 15/03/2023

Sobel filter example

Original Image Combination
Column

Row

21

Laplace gradient
• Laplace edge in a continuous domain
æ ¶ 2 f ( x, y ) ¶ 2 f ( x, y ) ö
G ( x, y ) = - ç + ÷
è ¶x ¶y 2 ø
2

• In a discrete domain, Laplace edge is approximated by

G ( x, y ) = [ f ( x, y ) - f ( x, y - 1] - [ f ( x, y + 1) - f ( x, y )]
+[ f ( x, y ) - f ( x + 1, y ] - [ f ( x - 1, y ) - f ( x, y )]

= f ( x, y ) * H ( x, y )

22

11
 15/03/2023

Laplace mask

é0 0 0 ù é0 -1 0 ù
H = êê -1 2 -1úú + êê0 2 0 úú
êë 0 0 0 úû êë0 -1 0 úû
é0 -1 0ù
= êê -1 4 -1úú
êë 0 -1 0 úû

23

Laplace filter example

Original image I |I*H|

24

12
 15/03/2023

Highboost filtering with Laplace

• Overall
I highboost = c × I orginal + I highpass
æ é0 0 0 ù ö
ç ê ú ÷
= ç c × ê0 1 0 ú + H ÷ * I original
ç ê0 0 0 ú ÷
è ë û ø
• Using Laplace mask
æ é0 0 0ù é 0 -1 0 ù ö é0 -1 0 ù
ç ê ÷
I highboost = ç c × ê0 1 0ú + ê -1 4 -1ú ÷ * I original = ê -1 c + 4 -1úú * I original
ú ê ú ê
ç ê0 0 0ú ê 0 -1 0 ú ÷ êë 0 -1 0 úû
è ë û ë ûø

25

Highboost filter example

Original Image c=0.5 c=1

26

13
 15/03/2023

Gradient comparison
Pixel difference Robert Prewitt

Sobel Laplace

27

More advanced edge detection

28

14
 15/03/2023

LoG edge detection

31

LoG edge detection example

Original image LoG filtered LoG edge

32

15
 15/03/2023

Canny edge detection

1. Smooth with 5x5 Gaussian kernel

2. Gradient with Sobel kernels

3. Non-maximum suppression

4. Thresholding

https://towardsdatascience.com/canny-edge-detection-step-by-step-in-python-computer-vision-b49c3a2d8123

34

Canny edge detection example

Original image Canny

35

16
 15/03/2023

Line detection
Hough transform

36

Example: Line fitting

• Many objects characterized by presence of straight lines

37

17
 15/03/2023

Difficulty of line fitting

• Extra edge points (clutter),
multiple models
• Which points go with which line, if
any?
• Only some parts of each line
detected, and some parts are
missing:
• How to find a line that bridges
missing evidence?
• Noise in measured edge points,
orientations:
• How to detect true underlying
parameters

38

Fitting lines with Hough transform

• Given points that belong to a line, what is
the line?
• How many lines are there?
• Which points belong to which lines?
• Hough transform is a voting technique that
can be used to answer all of these questions
• Main idea:
1. Record vote for each possible line on which
each edge point lies
2. Look for lines that get many votes

39

18
 15/03/2023

Line planes

ab-plane or parameter space

y = a’x + b’

What if the line approaches the vertical or horizontal direction?

(i.e., infinity slope)

41

Polar representation for lines

rho: perpendicular distance from line to origin

theta: angle the perpendicular makes with the x-axis

Normal presentation of a line:

à Point in image space is now sinusoid segment in Hough space

42

19
 15/03/2023

Finding lines in an image: Hough algorithm

1. Using the polar 2. Create a Hough
parameterization Accumulator Array (keeps
the votes)

43

Example of Hough transform

44

20
 15/03/2023

Basic Hough transform algorithm

45

Line detection example

46

21
 15/03/2023

Line detection example

Canny edges
original

Vote space and Longest segments

top peaks found

47

Impact of noise on Hough

What difficulty does this present for an implementation?

48

22
 15/03/2023

Impact of noise on Hough

Everything appears to be “noise”, or random edge points, but we still see some peaks in the vote space

49

Texture analysis

50

23
 15/03/2023

What is texture?

• An image obeying some statistical properties

• Similar structures repeated over and over again
• Often has some degree of randomness

51

Aspects of texture
• Size/granularity (sand versus pebbles
versus boulders)
• Directionality/Orientation
• Random or regular (stucco versus bricks)

52

24
 15/03/2023

Statistical approach to texture

• Characterize texture using statistical measures computed from
grayscale intensities (or colors) alone
• Less intuitive, but applicable to all images and computationally
efficient
• Can be used for both classification of a given input texture and
segmentation of an image into different regions

53

Some (simple) statistical texture measures

• Edge density and direction

• Use an edge detector as the first step in texture analysis

• The number of edge pixels in a fixed-size region tells us how busy that
region is
• The directions of the edges also help characterize the texture

54

25
 15/03/2023

Two edge-based texture measures

1. edgeness per unit area
Fedgeness = |{ p | gradient_magnitude(p) ³ threshold}| / N

where N is the size of the unit area

2. edge magnitude and direction histograms
Fmagdir = ( Hmagnitude, Hdirection )

where these are the normalized histograms of gradient

magnitudes and gradient directions, respectively.

55

Example
Original Image Frei-Chen Thresholded
Edge Image Edge Image

Different F_{edgeness} for different regions

56

26
 15/03/2023

Local binary pattern measure

Image credit: https://www.pyimagesearch.com/2015/12/07/local-binary-patterns-with-python-opencv/

57

LBP example

Color Image -> Grayscale Image -> LBP Mask -> Normalized LBP Histogram

Image credit: http://hanzratech.in/2015/05/30/local-binary-patterns.html

58

27
 15/03/2023

Gray Level Co-occurence Matrix (GLCM)

• Distribution of co-occurring pixel values (grayscale values, or colors)

at a given offset
• A distance d, and an angle 𝜃

Haralick features -à contrast, correlation texture features

59

GLCM example

60

28