4/29/2009

Morphological
Image Processing:
Basic Algorithms

by Gleb V. Tcheslavski: [email protected]

http://ee.lamar.edu/gleb/dip/index.htm
Spring 2009 ELEN 4304/5365 DIP 1

Preliminaries
When dealing with binary images, one of the principal
applications of morphology is extracting image components
that are useful in the representation and description of
shape.
We consider morphological algorithms for extracting
boundaries, connected components, the convex hull, and the
skeleton of a region. We also develop methods (region
filling, thinning, thickening, and pruning) that are
frequently used in conjunction with these algorithms as pre-
or post-processing steps.

Spring 2009 ELEN 4304/5365 DIP 2

1
 4/29/2009

Boundary Extraction
The boundary of a set A, denoted as β(A), can be obtained by first
eroding A by B and then performing the set difference between A and
its erosion as follows:
β ( A) = A − ( A  B )
where B is a suitable structuring element.
Set A Structuring
element B

Boundary: set
Erosion of A difference
by B between A and
its erosion
Spring 2009 ELEN 4304/5365 DIP 3

Boundary Extraction

A binary image Boundary extracted using a 3x3

structuring element of ones

Size of structuring element defines the boundary being 1 pixel thick.

Spring 2009 ELEN 4304/5365 DIP 4

2
 4/29/2009

Hole Filling
A hole may be defined as a background region surrounded by a
connected border of foreground pixels.
Let denote byy A a set whose elements are 8-connected boundaries,,
each boundary enclosing a background region (a hole). Given a point
in each hole, the objective is to fill all the holes with ones (for binary
images).
We start from forming an array X0 of zeros (the same size as the array
containing A), except at the locations in X0 corresponding to the given
point in each hole, which is set to one. Then, the following procedure
fills all the holes with ones:
X k = ( X k −1 ⊕ B ) ∩ Ac k = 1, 2,3,...
where B is the symmetric structuring element.
Spring 2009 ELEN 4304/5365 DIP 5

Hole Filling
The algorithm terminates at the iteration step k if Xk = Xk-1.
The set Xk then contains all the filled holes; the union of Xk
and A contains all the filled holes and their boundaries.
The dilation would fill the entire area if left unchecked.
However, the intersection at each step with the complement
of A limits the result to inside the region of interest. This is
an example of how a morphological process can be
conditioned to meet a desired property. In the current
application, it can be called a conditional dilation.

Spring 2009 ELEN 4304/5365 DIP 6

3
 4/29/2009

Hole Filling
Complement of A
Set A

S
Structuring
i element
l

Set X0; a grey dot

indicates location
of a hole

A set with a filled

hole

Spring 2009 ELEN 4304/5365 DIP 7

Hole Filling
An image that could result from thresholding to 2 levels a scene
containing polished spheres (ball bearings). Dark spots could be
results of reflections. The objective is to eliminate reflections by hole
filling…
filli
A (white) point selected Result of filling that Result of filling all
inside one sphere component the spheres

Spring 2009 ELEN 4304/5365 DIP 8

4
 4/29/2009

Extraction of connected components

We discussed concepts of connectivity and connected components
earlier. Extraction of connected components from a binary image is
important for many automated image analysis applications.
Let A be a set containing one or more connected components. We
form an array X0 (of the same size as the array containing A), whose
elements are zeros (background values), except at each location
known to correspond to a point in each connected component in A,
which we set to one (foreground value). The objective is to start with
X0 and find all the connected components by the following iterative
procedure:
d
X k = ( X k −1 ⊕ B ) ∩ A k = 1, 2,3,...

where B is a suitable structuring element. The procedure terminates

when Xk = Xk-1 with Xk containing all the connected components of A.
Spring 2009 ELEN 4304/5365 DIP 9

Extraction of connected components

Structuring element
based on 8-connectivity

Set A

Spring 2009 ELEN 4304/5365 DIP 10

5
 4/29/2009

Extraction of connected components

Connected components are often
used for automated inspection.
X-rayy image
g of chicken filet with
bone fragments

Thresholded image

Image eroded with a 5x5 structuring

element

Spring 2009 ELEN 4304/5365 DIP 11

Extraction of connected components

It is of interest to be able to detect foreign fragments in processed
food before packaging or shipping.
In this case, the density of the bones is such that their intensity values
are different from background. After thresholding, we observe that the
points that remain are clustered into objects (bones). Therefore, we
can make sure that only objects of “significant” size remain by
eroding the thresholded image. for erosion, a 5x5 structuring element
was selected.
y the size of objects
Next, we analyze j that remain. We identifyy them byy
extracting the connected components in the image. As a result, 15
connected components were found with 3 of them being dominant in
size (133, 674, and 743 pixels). This is good enough to determine that
significant undesirable objects are contained in the image.

Spring 2009 ELEN 4304/5365 DIP 12

6
 4/29/2009

Convex Hull
A set A is said to be convex if the straight line segment joining any
two points in A lies entirely within A. The convex hull H of an
arbitrary set S is the smallest convex set containing S.
The difference H – S is called the convex deficiency of S. The convex
hull and convex deficiency are useful for object description.
Let Bi, i=1,2,3,4 represent the 4 structuring elements. The procedure
consists of implementing the equation:
X ki = ( X k −1 ⊗ B i ) ∪ A i = 1, 2,3, 4 k = 1, 2,3,...
with X 0i = A
When the procedure converges (Xik = Xik-1), we let Di = Xik. The
convex hull of A is then 4
C ( A ) = ∪ Di
i =1
Spring 2009 ELEN 4304/5365 DIP 13

Convex Hull
Therefore, the method consists of iteratively applying the
hit-or-miss transform to A with B1; when no further
changes
g occur,, we perform
p the union with A and call the
result D . The procedure is repeated with B2( applied to A)
1

until no further changes occur, and so on… the union of

the four resulting Ds is the convex hull of A.

Spring 2009 ELEN 4304/5365 DIP 14

7
 4/29/2009

Convex Hull
Structuring elements: “x” indicates
“do not care” conditions

A set A

Results of convergence with

structural elements

Convex hull showing the contributions

of each structuring element

Spring 2009 ELEN 4304/5365 DIP 15

Convex Hull
One shortcoming of the procedure is that the convex hull can grow
beyond the minimum dimensions required to guarantee convexity.
One simple approach to reduce this effect is to limit growth such that
it does not extend past the vertical and horizontal dimensions of the
original set.

The result of this limitation on the

previous image: a convex hull limited to
the dimensions of the original set.

Spring 2009 ELEN 4304/5365 DIP 16

8
 4/29/2009

Thinning
The thinning of a set A by a structuring element B is defined in terms
of the hit-or-miss transform:
A B = A − ( A ⊗ B) = A ∩ ( A ⊗ B)
ⓩ c

So far, we were interested only in pattern matching with the

structuring elements, so no background operation is required in the
hit-or-miss transform. A more useful expression for thinning A
symmetrically is based on a sequence of structuring elements:

{B} = {B1 , B 2 ,,...B n }

where Bi are rotated versions of Bi-1. The thinning by a sequence of

{B} = (... ( (ⓩ ) )
SEs:
ⓩ
A A ) B 2 ...ⓩ B n
B1ⓩ

Spring 2009 ELEN 4304/5365 DIP 17

Thinning
The process is to thin A by
one pass with B1, then thin
the result with one pass of Set Sequence of rotated SEs
B2, andd so on, until
til A is
i
thinned with one pass of Bn.
The entire process is repeated
until no further changes
occur.
Results of thinning with
eachh SE one after
ft another
th

Using 4 first SEs again

Result after convergence
Conversion to m-connectivity
Spring 2009 ELEN 4304/5365 DIP 18

9
 4/29/2009

Thickening
Thickening is a morphological dual of a thinning and is defined as

B = A ∪ ( A ⊗ B)
c
A
where B is a structuring element suitable for thickening. As in
thinning, thickening can be defined as a sequentional operation:

A {B} = (... ( ( A B1 ) ) )
B 2 ... Bn
The structuring element used for thickening has the same form as one
used for thinning but with all ones and zeros interchanged.
However, the usual procedure is to thin the background of the set to
be processed and then complement the result. Therefore, to thicken a
set A, we form its complement, thin it, and then complement the
result.
Spring 2009 ELEN 4304/5365 DIP 19

Thickening
Depending on the nature of A, the thickening procedure may result in
disconnected points. Therefore, this method is usually followed by
post-processing to remove disconnected points.

Set A Its
complement

Thinned Complement
complement
p of thinningg
of A complement...

Result of thickening with

no disconnected points
Spring 2009 ELEN 4304/5365 DIP 20

10
 4/29/2009

Skeletons
A skeleton S(A) of a set A can be viewed as:
a) If z is a point of S(A) and (D)z is the largest disk centered at z and
contained in A,, one cannot find a larger
g disk ((not necessarilyy
centered at z) containing (D)z and included in A. The disk (D)z is
called a maximum disk.
b) The disk (D)z touches the boundary of A at two or more different
places.
The skeleton of A can be expressed in terms of erosions and openings:
K
S ( A) = ∪ Sk ( A)
k =0 Structuring element
Sk ( A ) = ( A  kB ) − ( A  kB ) B
k successive erosions of A
Spring 2009 ELEN 4304/5365 DIP 21

Skeletons
( A  kB ) = (... ( ( A  B )  B )  ...)  B
K is the last iterative stepp before A erodes to an empty
p y set:

K = max {k | ( A  kB ) ≠ ∅}
S(A) can be obtained as the union of the skeleton subsets Sk(A). Also,
we can show that A can be reconstructed from these subsets by:
K
A = ∪ ( S k ( A ) ⊕ kB )
k =0
k successive dilations of A

( A ⊕ kB ) = (... ( ( A ⊕ B ) ⊕ B ) ⊕ ...) ⊕ B
Spring 2009 ELEN 4304/5365 DIP 22

11
 4/29/2009

Skeletons
Various positions
of maximum
Set A disks with
centers on the
skeleton of A

Another
maximum disk Complete
on a different skeleton of A
segment of the
skeleton of A

Spring 2009 ELEN 4304/5365 DIP 23

Openings by B
Set differences
Skeletons 2 partial skeletons and final

between 1st
and 2nd
columns

Set A

Erosion 1
of A
Skeleton of
Erosion 2 A (not connected)
of A: next Reconstruc-
erosion ted A
will be ∅ SE
Spring 2009 ELEN 4304/5365 DIP 24

12
 4/29/2009

Pruning
Pruning methods are an essential component to thinning and
skeletonizing algorithms since these procedures tend to leave parasitic
components that need to be “cleaned up” by post-processing. We start
with
ith a pruning
i problem
bl andd then
th develop
d l a morphological
h l i l solution.
l ti
A common approach in the automated recognition of printed characters
is to analyze the shape of the skeleton of each character. These
skeletons often are characterized by “spurs” (parasitic components).
Spurs are created during the erosion by non uniformities in the strokes
composing the characters. We develop a morphological technique for
h dli this
handling thi problem,
bl starting
t ti withith the
th assumption
ti that
th t the
th length
l th off a
parasitic component does not exceed a specific number of pixels.

Spring 2009 ELEN 4304/5365 DIP 25

Pruning
A parasitic component

Original image Structuring

A: a skeleton of elements: “x”
x
a printed “a” – don’t care

3 runs of End points: X2

thinning: X1

Dilations of end Pruned image:

points X4
conditioned on
A: X3
Spring 2009 ELEN 4304/5365 DIP 26

13
 4/29/2009

Pruning
The solution is based on suppressing a parasitic branch by successively
eliminating its end point. This also shortens (or eliminates) other
branches in the character. The assumption is that, in the absence of
other
th structural
t t l information,
i f ti any branch
b h with
ith 3 or less
l pixels
i l should
h ld be
b
eliminated. Thinning of an input set A with a sequence of structuring
elements designed to detect only end points achieves the desired result.
Let X1 = ⓩ
A { B}
where {B} is the sequence of structuring elements. This sequence
consists of two different structures, each of which is rotated 900 for a
total of 8 elements.
Applying the equation 3 times yields the set X1 and the next step is to
“restore” the character to its original form but with the parasitic
branches removed.

Spring 2009 ELEN 4304/5365 DIP 27

Pruning
To do this, we first form a set X2 containing all end points in X1:

X 2 = ∪ ( X1 ⊗ Bk )
8

k =1

Bk
where are the same structuring elements. Next step is dilation of the
end points 3 times using set A as a delimiter:
X3 = ( X2 ⊕ H ) ∩ A
where H is a 3x3 structuring element of ones and the intersection with
A as applied after each step. This type of conditional dilation prevents
appearance off non-zero elements
l outside
id the
h region
i off interest.
i
Finally, the union of X3 and X1 yields the desired result:
X 4 = X1 ∪ X 3

Spring 2009 ELEN 4304/5365 DIP 28

14
 4/29/2009

Pruning
In more complex situations, X3 sometimes includes the “tips” of some
parasitic branches. This can occur when the end points of these
branches are near the skeleton.
Although, they may be eliminated in X1, these elements may show up
during the dilation since they are valid points in A. The entire parasitic
elements are rarely picked up again since they are usually short
compared to the valid strokes. Therefore, their detection and
elimination is easy since they are disconnected regions.

Spring 2009 ELEN 4304/5365 DIP 29

Morphological reconstruction (MR)

Morphological reconstruction is a morphological
transform involving 2 images and a structuring element.
O iimage, the
One th marker,
k contains
t i the
th starting
t ti points
i t for
f the
th
transformation. The other image, the mask, constrains the
transformation. The structuring element is used to define
connectivity.

Spring 2009 ELEN 4304/5365 DIP 30

15
 4/29/2009

MR: geodesic dilation and erosion

Central to morphological reconstruction are the concepts of geodesic
dilation and geodesic erosion. Let F denote the marker image and G
the mask image (assuming that both are binary images and that F ⊆
G) Th
G). The geodesic
d i dilation
dil ti off size
i 1 off the
th marker
k image
i with
ith respectt
to the mask is
DG(1) ( F ) = ( F ⊕ B ) ∩ G
Here ∩ denotes the set intersection and may be interpreted as logical
AND since they are the same for binary sets.
The ggeodesic dilation off size n of F with respect
p to G is

(
DG( n ) ( F ) = DG(1) DG( n −1) ( F ) )
with
DG( 0) ( F ) = F

Spring 2009 ELEN 4304/5365 DIP 31

MR: geodesic dilation and erosion

In the last expression, the set intersection is performed at each step.
The intersection operator guarantees that mask G will limit the growth
(dilation) of marker F.

Geodesic erosion
Spring 2009 ELEN 4304/5365 DIP 32

16
 4/29/2009

MR: geodesic dilation and erosion

Similarly, the geodesic erosion of size 1 of the marker image F with
respect to the mask G is
EG(1) ( F ) = ( F  B ) ∪ G
where ∪ denotes the union (OR operation). The geodesic erosion of
size n of F with respect to G is defined as
EG( ) ( F ) = EG( ) EG(
n 1
( n −1)
( F ))
EG( 0) ( F ) = F
with

The union operation is performed at each iterative step and guarantees

that geodesic erosion of an image remains greater then or equal to its
mask image. Geodesic dilation and erosion are duals with respect to
set complementation.
Spring 2009 ELEN 4304/5365 DIP 33

MR: geodesic dilation and erosion

Geodesic dilation
Spring 2009 ELEN 4304/5365 DIP 34

17
 4/29/2009

MR by dilation and erosion

Morphological reconstruction by dilation of a mask image G from a
marker image F is defined as the geodesic dilation of F with respect
to G, iterated until stability is achieved:
RGD ( F ) = DG( ) ( F )
k

with k such that DG(k)(F) = DG(k+1)(F).

Considering these
marker, mask, and
structuring
element…

Spring 2009 ELEN 4304/5365 DIP 35

MR by dilation and erosion

Dilation dilated
with SE Result AND G

Spring 2009 ELEN 4304/5365 DIP 36

18
 4/29/2009

MR by dilation and erosion

The morphological reconstructed image is DG(5)(F), which is identical
to the mask F since F contained a single 1-valued pixel (this is
analogous to convolution of an image with an impulse, which simply
copies an image at the location of the impulse).

The morphological reconstruction by erosion of a mask G from a

marker image F is defined as the geodesic erosion of F with respect to
G, iterated until stability:
RGE ( F ) = EG( k ) ( F )
with k such that EG(k)(F) = EG(k+1)(F).

Spring 2009 ELEN 4304/5365 DIP 37

MR: sample applications

Morphological reconstruction has a broad spectrum of practical
applications, each determined by the selection of the marker and mask
images, by the structuring elements used, and by combinations of the
primitive operations as defined above.

Opening by reconstruction:
In a morphological opening, erosion removes small objects and the
subsequent dilation attempts to restore the shape of objects that
remain. However,, the accuracy y of this restoration is highly
g y dependent
p
on the similarity of the shapes of the objects and the structuring
element used.
Opening by reconstruction restores exactly the shapes of the objects
that remain after erosion.
Spring 2009 ELEN 4304/5365 DIP 38

19
 4/29/2009

MR: sample applications

The opening by reconstruction of size n of an image F is defined as
the reconstruction by dilation of F from the erosion of size n of F:
OR( ) ( F ) = RFD ⎡⎣( F  nB
B ) ⎤⎦
n

n erosions of F by B

We observe that F is used as the mask in this application.

A similar expression can be written for closing by reconstruction:

CR( ) ( F ) = RFE ⎡⎣( F ⊕ nB ) ⎤⎦

n

n dilations of F by B

Spring 2009 ELEN 4304/5365 DIP 39

MR: sample applications

Text image of size 918x2018; Erosion with a SE of size 51x1 pixels
average height of tall characters is 50

Opening of the image with the same Opening by reconstruction: interest

SE (for reference) to extract characters with long
Spring 2009 ELEN 4304/5365 DIP
vertical strokes 40

20
 4/29/2009

MR: sample applications

Filling holes:
We develop next a fully automatic procedure for hole filling based on
morphological reconstruction. Let I(x,y) be a binary image, assume
that we form a marker image F that is 0 everywhere , except at the
image border, where it is set to 1 – I:
⎧⎪1 − I ( x, y ) if ( x, y ) ison the border of I
F ( x, y ) = ⎨
⎪⎩0 otherwise
Then
H = ⎡⎣ RIDc ( F ) ⎤⎦
c

Is a binary image equal to I with all holes filled.

Spring 2009 ELEN 4304/5365 DIP 41

MR: sample applications

Its Its comple-

Original complement F dilated Intersec- ment
Intersec-
image by a 3x3 tion of
tion with
with a SE of dilation
comple-
comple
h l
hole ones with
ih
ment
comple-
ment

Spring 2009 ELEN 4304/5365 DIP 42

21
 4/29/2009

MR: sample applications

Text image of size 918x2018; Its complement
average height of tall characters is 50

Marker image F Result of hole filling

Spring 2009 ELEN 4304/5365 DIP 43

MR: sample applications

Border cleaning:
The extraction of objects from an image for subsequent shape analysis
is a fundamental task in automated image processing. An algorithm
for removing objects that touch (connected to) the border is useful
since:
1) it can be used to screen the image such that only complete objects
remain;
2) It can be used to detect partial objects that are present in the field
of view.
view
We develop a border-cleaning procedure based on morphological
reconstruction.

Spring 2009 ELEN 4304/5365 DIP 44

22
 4/29/2009

MR: sample applications

We use the original image as the mask and the following marker
image:
⎧⎪ I ( x, y ) if ( x, y ) is on the border of I
F ( x, y ) = ⎨
⎪⎩0 otherwise
The border-cleaning algorithm first computes the morphological
reconstruction RID(F), which extracts the objects touching the border,
and then computes the difference

X = I − RID ( F )
To obtain an image X with no objects touching the border.

Spring 2009 ELEN 4304/5365 DIP 45

MR: sample applications

For the same text image, we need to eliminate the incomplete
characters (ones touching the border). This may be used before
automatic character recognition.
g

Marker image F obtained using a The image with no objects touching

3x3 SE of ones the border

Spring 2009 ELEN 4304/5365 DIP 46

23
 4/29/2009

Structuring elements used

Basic types of structuring elements used in binary morphology:

Spring 2009 ELEN 4304/5365 DIP 47

24