PATTERN SPECTRA ALGORITHMS FOR PATTERN RECOGNITION

by

TCHANGOU TOUDJEU IGNACE

Submitted in fulfilment of the requirements for the degree

MAGISTER TECHNOLOGIAE: ELECTRONIC ENGINEERING

In the

Graduate School of Electrical and Electronic Engineering

FACULTY OF ENGINEERING

TSHWANE UNIVERSITY OF TECHNOLOGY

Supervisor: Prof Barend J. van Wyk

Co-Supervisor: Prof. Michael A. van Wyk

November 2006
____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 1
PATTERN SPECTRA FOR PATTERN RECOGNITION

DECLARATION

“I hereby declare that the dissertation submitted for the degree M Tech: Electronic

Engineering, at Tshwane University of Technology, is my own original work and has not

previously been submitted to any other institution of higher education. I further declare that all

sources cited or quoted are indicated and acknowledged by means of a comprehensive list of

references”.

I. Tchangou Toudjeu

Copyright© Tshwane University of Technology 2006

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY i
PATTERN SPECTRA FOR PATTERN RECOGNITION

To my dear parents

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY ii
PATTERN SPECTRA FOR PATTERN RECOGNITION

ACKNOWLEDGMENTS

I would like to express my sincere gratitude and appreciation to: my supervisors, Prof. Barend

J. van Wyk and Prof. M.A. van Wyk for their positive attitude and guidance toward the

successful completion of this project.

Furthermore, I would like to thank F’SATIE for all the facilities that I received to complete

this project. I thank the following people:

Mr Pierre Abeille, Director of F’SATIE.

Mr Damien Chatelain for consultation.

My colleague students and all who assisted me when I needed help.

Special thanks go to Tshwane University of Technology, F’SATIE and NRF [Grant number

TRD2005070100036] for their financial support.

Thanks to my dear mother for her support and encouragements.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY iii
PATTERN SPECTRA FOR PATTERN RECOGNITION

ABSTRACT

This dissertation deals with the application of pattern spectra algorithms to images of materials

of different types, for the purpose of pattern classification. As materials are often best

characterized by their texture, pattern spectra constitute a very important tool for texture

analysis. Two granulometric techniques and their resultant pattern spectra are discussed,

namely morphological granulometries based on structural openings and linear granulometries

based on the linear openings. Both are used to extract global image information. A novel

algorithm, not based on mathematical morphology called slope pattern spectra is also

proposed. The resulting pattern spectra from both the granulometric techniques and the

proposed algorithm are used in conjunction with a neural network to solve two pattern

recognition problems, namely classification and characterization (regression). Experiments are

conducted to compare the discussed pattern spectra algorithms in terms of speed and accuracy.

From the results it is evident that the slope pattern spectra algorithm is a fast and robust

alternative to granulometric-based techniques.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY iv
PATTERN SPECTRA FOR PATTERN RECOGNITION

CONTENTS

PAGE

DECLARATION..........................................................................................................................i

ACKNOWLEDGMENTS ......................................................................................................... iii

ABSTRACT ...............................................................................................................................iv

CONTENTS ................................................................................................................................v

LIST OF FIGURES ................................................................................................................. viii

LIST OF TABLES ......................................................................................................................x

GLOSSARY ...............................................................................................................................xi

CHAPTER 1................................................................................................................................1

INTRODUCTION .......................................................................................................................1
1.1 Background..................................................................................................................1
1.2 Problem statement .......................................................................................................2
1.2.1 Sub-problem 1 .....................................................................................................2
1.2.2 Sub-problem 2 .....................................................................................................3
1.2.3 Sub-problem 3 .....................................................................................................3
1.3 Methodology................................................................................................................3
1.4 Dissertation outline......................................................................................................4

CHAPTER 2................................................................................................................................6

LITERATURE REVIEW ............................................................................................................6

2.1 Texture.........................................................................................................................6
2.2 Texture analysis...........................................................................................................7
2.3 Mathematical morphology...........................................................................................8
2.4 Granulometries ..........................................................................................................10
2.5 Some applications of morphological techniques .......................................................11
2.6 Texture classification.................................................................................................11
2.7 Summary....................................................................................................................12

CHAPTER 3..............................................................................................................................13

MATHEMATICAL MORPHOLOGY .....................................................................................13

3.1 Basic definitions ........................................................................................................13
3.1.1 Binary image .....................................................................................................14

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY v
PATTERN SPECTRA FOR PATTERN RECOGNITION

3.1.2 Grayscale image ................................................................................................14

3.1.3 Structuring element ...........................................................................................15
3.2 Binary morphological operations ..............................................................................16
3.2.1 Binary dilation ...................................................................................................18
3.2.2 Binary erosion ...................................................................................................19
3.2.3 Binary opening ..................................................................................................20
3.2.4 Binary closing....................................................................................................21
3.3 Grayscale morphological operations .........................................................................22
3.3.1 Grayscale dilation ..............................................................................................23
3.3.2 Grayscale erosion ..............................................................................................24
3.3.3 Grayscale opening .............................................................................................25
3.3.4 Grayscale closing...............................................................................................26
3.4 Summary....................................................................................................................27

CHAPTER 4..............................................................................................................................28

GRANULOMETRIES ..............................................................................................................28
4.1 Basic concept.............................................................................................................28
4.2 Morphological granulometries and pattern spectrum................................................31
4.2.1 Size distribution.................................................................................................32
4.2.2 Pattern spectrum ................................................................................................35
4.3 Linear grayscale granulometries and pattern spectrum .............................................37
4.3.1 Linear grayscale granulometries........................................................................37
4.3.2 Horizontal pattern spectrum ..............................................................................40
4.4 Opening trees and grayscale granulometries .............................................................44
4.5 Summary....................................................................................................................46

CHAPTER 5..............................................................................................................................47

SLOPE PATTERN SPECTRA .................................................................................................47

5.1 Integral image ............................................................................................................47
5.2 Proposed algorithm....................................................................................................53
5.3 Summary....................................................................................................................56

CHAPTER 6..............................................................................................................................57

EXPERIMENTAL DESIGN.....................................................................................................57
6.1 The proposed supervised system ...............................................................................57
6.2 Feature extraction ......................................................................................................58
6.3 Neural networks.........................................................................................................59
6.3.1 Classification (Seed images) .............................................................................59
6.3.2 Regression (HSLA steel images).......................................................................61
6.3.3 Network training................................................................................................62
6.4 Implementation details of the proposed system ........................................................63
6.4.1 Sample images for classification (Seed images) ...............................................63
6.4.2 Sample image for regression (HSLA steel images)...........................................64

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY vi
PATTERN SPECTRA FOR PATTERN RECOGNITION

6.4.3 Pattern spectra implementation .........................................................................66

6.4.4 Classification implementation for seed images .................................................66
6.4.5 Regression implementation for steel images .....................................................67
6.5 Summary....................................................................................................................67

CHAPTER 7..............................................................................................................................69

EXPERIMENTAL RESULTS AND DISCUSSION ................................................................69

7.1 Experimental results ..................................................................................................69
7.1.1 Data acquisition results......................................................................................69
7.1.2 Pattern spectra results ........................................................................................70
7.1.3 Classification results for seed images................................................................71
7.1.4 Regression results for steel images....................................................................72
7.2 Discussion..................................................................................................................74
7.3 Summary....................................................................................................................76

CHAPTER 8..............................................................................................................................77

CONCLUSION AND FUTURE WORK..................................................................................77

8.1 Conclusions ...............................................................................................................77
8.2 Future work ...............................................................................................................78

BIBLIOGRAPHY .....................................................................................................................80

APPENDIX A...........................................................................................................................89

Mixture sample images and their pattern spectra ......................................................................89

APPENDIX B...........................................................................................................................95

HSLA sample images and their pattern spectra.........................................................................95

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY vii
PATTERN SPECTRA FOR PATTERN RECOGNITION

LIST OF FIGURES

PAGE
Figure 3.1: A binary image........................................................................................................14

Figure 3.2: Grayscale image......................................................................................................15

Figure 3.3: (a) Cross SE, (b), Diamond SE and (c) Horizontal Line SE. The cross mark is.....15

the reference pixel or center pixel. ............................................................................................15

Figure 3.4: Example of complement, union and intersection....................................................17

Figure 3.5: Translation operation on image A by t . ..................................................................17

Figure 3.6: Dilation of image A by the structuring element B . ...............................................19

Figure 3.7: Erosion of image A by the structuring element B . .................................................20

Figure 3.8: Opening of image A by the structuring element B ................................................21

Figure 3.9: Closing of image A by the structuring element B . ................................................22

Figure 3.10: Grayscale image representation. ...........................................................................23

Figure 3.11: Grayscale dilation. ................................................................................................24

Figure 3.12: Grayscale erosion..................................................................................................25

Figure 3.13: Grayscale opening.................................................................................................26

Figure 3.14: Grayscale closing. .................................................................................................27

Figure 4.1: A sequence of increasing structuring elements for λ = 2 , λ = 4 , λ = 6 and λ = 8 .

...................................................................................................................................................33

Figure 4.2: A decreasing family of openings of an image of seeds...........................................34

Figure 4.3: An image of mixed seeds (left), its size distribution (middle) and its cumulative

normalized size distribution (right). ..........................................................................................35

Figure 4.4: Pattern spectrum of the image of seeds..................................................................36

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY viii
PATTERN SPECTRA FOR PATTERN RECOGNITION

Figure 4.5: Illustration of a line segment...................................................................................37

Figure 4.7: Cross-section of I with a maximum M ..................................................................41

Figure 4.8: Illustration of linear grayscale algorithm for a line with two maxima....................42

Figure 4.9: Illustration of the horizontal pattern spectrum of the mixed seeds image shown in

the left of figure 4.2. ..................................................................................................................43

Figure 4.10: Opening tree representation of the cross-section shown in figure 4.8. The leaves

of the tree represent the image pixels. .......................................................................................44

Figure 5.1: The value at the point ( x, y ) corresponds to the sum of all pixels in the shaded

area.............................................................................................................................................48

Figure 5.2: A line segment and its integral representation........................................................49

Figure 5.3: Illustration of a cross section of a line from an image, its integral representation

and a curve corresponding to the integral line segment indicating two increasing slope

segments: the 1st one going from the 1st pixel to the 4th pixel and the 2nd one from the 6th pixel

till the 8th pixel...........................................................................................................................51

Figure 5.4: Slope pattern spectrum of a mixed seed image.......................................................55

Figure 6.1: Flowchart of the proposed classification system. ...................................................58

Figure 6.2: A feed-forward network architecture. ...............................................................61

Figure 6.3: Examples of seed sample images...........................................................................64

Figure 6.4: Examples of HSLA sample images. .......................................................................65

Figure 7.1: Regression results obtained when using morphological scheme. ...........................72

Figure 7.2: Regression results obtained when using linear scheme. .........................................73

Figure 7.3: Regression results obtained when using slope scheme...........................................73

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY ix
PATTERN SPECTRA FOR PATTERN RECOGNITION

LIST OF TABLES

PAGE

Table 4.1: Algorithm of a horizontal granulometry for a line of image I (Vincent, 2000:126-

127)............................................................................................................................................42

Table 4.2: Algorithm of a linear granulometry of the grayscale image I from its opening tree

representation.............................................................................................................................45

Table 5.1: Proposed slope pattern spectra algorithm.................................................................54

Table 6.1: Mixture of seeds. ......................................................................................................63

Table 7.1: Execution time of different feature extraction techniques for four sample images

shown in Appendix A and B. ....................................................................................................71

Table 7.2: Classification results.................................................................................................71

Table 7.3: Performance measures of the regression. .................................................................73

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY x
PATTERN SPECTRA FOR PATTERN RECOGNITION

GLOSSARY

HSLA: High Strength Low Allow.

K-NN: K-Nearest Neighbour.

LPS: Linear Pattern Spectra.

LUT: Look Up Table.

MPS: Morphological Pattern Spectra.

MSE: Mean Square Error.

SE: Structuring Element.

SPS: Slope Pattern Spectra.

SS: Slope Segment.

ISS: Increasing Slope Segment.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY xi
PATTERN SPECTRA FOR PATTERN RECOGNITION

CHAPTER 1

If we knew what it was we were doing, it would not be called research, would it?
(Albert Einstein)
Research is what I’m doing when I don’t know what I’m doing
(Werner von Braun)

INTRODUCTION

1.1 Background

Since texture is an important surface feature, many industrial materials such as wood, steel, or

mixed materials such as seeds can be characterized by their texture. The detection of defects or

classification for quality control can be done using texture analysis.

The relationship between physical properties and texture can be understood by investigating

the diversity of morphologies and their characteristics. The development of computational

vision techniques to address these issues should therefore focus on the determination of grain

size of particles, morphological characteristics and micro-morphology. A pattern spectrum is a

useful tool for texture analysis since it extracts the size distribution of grains.

Some previous work addressing these issues include mathematical morphology techniques for

the analysis of the civil engineering materials (Coster and Chermant, 2001), image analysis for

macro-segregation in a high-carbon continuously cast steel (Straffelini and Molinari, 1997),

the examination of hydrogen interaction in carbon steel by means of quantitative micro-

structural and feature descriptions (Sozanska et al. 2001), the influence of mixed grain size

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 1
PATTERN SPECTRA FOR PATTERN RECOGNITION

distributions on the toughness in high strength steels (From and Sanderstrom, 1999) and many

similar contributions focusing on mathematical morphology techniques.

The focus of this project will be to characterize the size or shape of different grains present in

material samples. It is envisaged that the determination of the grain size of particles,

morphological characteristics and the dispersion and orientation of grains can be captured by

pattern spectra. A pattern spectrum quantifies the morphological and statistical characteristics

of different phases observed. By representing images of different materials samples by their

pattern spectra, the problems such as classification or characterization can be reduced to a

pattern recognition problem. Recent contributions to morphological and fast granulometric

methods are investigated and a novel algorithm for pattern spectra, not based on mathematical

morphology, is presented.

1.2 Problem statement

The purpose of this work is to develop a fast and robust pattern spectrum algorithm for the

classification and characterization (regression) of materials as an alternative to morphology

based pattern spectra methods.

1.2.1 Sub-problem 1

Investigate and implement morphology based pattern spectra methods.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 2
PATTERN SPECTRA FOR PATTERN RECOGNITION

1.2.2 Sub-problem 2

Develop and implement the novel slope pattern spectra algorithm as a fast and accurate

alternative to morphology based pattern spectra methods.

1.2.3 Sub-problem 3

Conduct experiments to determine the efficiency and robustness of the proposed slope pattern

spectra algorithm. The experiments will include classification using seed images and

characterization (regression) using High Strength Low Allow (HSLA) steel images.

1.3 Methodology

Experiments are conducted in this dissertation using two diverse types of materials: a mixture

of seeds and HSLA steel samples. The sample images of these materials are captured and

regrouped with respect to some pre-defined criteria.

Granulometric methods are implemented by means of simulations written in MATLAB. The

proposed slope pattern spectra algorithm is also implemented and comparisons are made in

terms of speed and classification accuracy.

Pattern spectra that result from both granulometric methods and the proposed slope pattern

spectra algorithm are used as features. These pattern spectra are fed to a neural network for

the classification and characterization (regression) of the materials.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 3
PATTERN SPECTRA FOR PATTERN RECOGNITION

1.4 Dissertation outline

The main purposes of this dissertation are:

1. to provide an overview of mathematical morphology for image processing.

2. to investigate granulometric methods such as morphological granulometries

and linear granulometries.

3. to propose a new algorithm and a supervised system for classification and

characterization (regression).

Hence, in Chapter 2, a review of texture and texture analysis is presented. Mathematical

Morphology and granulometry as image analysis tools are introduced, as well as classification

using neural networks.

In Chapter 3, a review of the basic geometric characteristics of primitive morphology

operators are given. Some illustrative examples are also presented.

Chapter 4 is devoted to two granulometric methods: one based on morphological operations,

which sometimes is referred to as morphological granulometries and the other one based on

work done by Vincent (2000:119-133).

The proposed slope pattern spectra algorithm is described in Chapter 5.

The experimental design is presented in Chapter 6 by means of a proposed supervised system.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 4
PATTERN SPECTRA FOR PATTERN RECOGNITION

Experimental results are presented in Chapter 7 followed by a discussion of the simulation

results.

Chapter 8 concludes the dissertation and outlines future work.

In the Appendices, some experimental results are given. Sample images of seeds and their

respective pattern spectra are presented in Appendix A. Appendix B consists of the sample

images of HSLA steel and their respective pattern spectrum.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 5
PATTERN SPECTRA FOR PATTERN RECOGNITION

CHAPTER 2

The greatest challenge to any thinker is stating the problem in a way that will allow a solution.
(Bertrand Russell)

LITERATURE REVIEW

Materials, in general when captured as sample images, are rich in visual information. These

sample images can be characterized by structure and texture. Structure and texture are the

same phenomena except at different scales: individual objects, when repeated in either a

random or a predictable manner, form a structure and when the repeating objects are not

distinguishable but the repeating pattern still is, a texture is formed. In this chapter, a short

review on texture analysis is presented. In addition, some applications based on pattern spectra

such as texture classification are also discussed.

2.1 Texture

Texture can be defined as an image that is organized by a repeating pattern (Rosenfeld, 1982)

or un-repeating texture primitives, i.e. small particles such as grains. These primitive patterns

are sometimes referred to as “textons” (Brodatz, 1966). In Asano, Miyagawa and Fujio (2000),

textures in natural scenes contain particles of shape at various sizes since the shapes of these

particles depend on the materials of which the texture entities are made. Additionally, image

texture is generally a particular spatial arrangement of gray levels, with the property that the

gray level variations have to be of a rather high frequency, and that it presents a pseudo-

periodical character (Huet and Mattioli, 1996). Accordingly to these definitions, issues such as

feature extraction, and texture classification can be solved using texture analysis techniques.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 6
PATTERN SPECTRA FOR PATTERN RECOGNITION

2.2 Texture analysis

Since feature extraction computes a characteristic of a digital image that is able to numerically

describe its texture properties (Meterka and Strzelecki, 1998), this is considered as the first

stage of texture analysis. Pattern spectra belongs to this stage and have been used for a variety

of image analysis tasks, including texture segmentation (Dougherty, Newell and Pelz,

1992:1181-1198 and Dougherty et al, 1992:40-60), texture discrimination (Yamamoto and

Kotani, 1998:57-64) and texture classification (Chen and Dougherty, 1992:931-942).

There are two main groups of texture analysis methods, namely statistical methods and

structural methods (Haralick, 1979:786-809). The former is more suitable for disordered

textures, where the spatial distribution of gray levels is more random than structured and the

latter is more suitable for ordered texture (Huet and Mattioli, 1996: 297).

Statistical methods do not attempt to explicitly understand the hierarchical structure of the

texture, but they indirectly extract image features by the non-deterministic properties that

govern the distributions and relationship between the gray levels of the image. An example of

a statistical method is the Fourier transform (Pratt as quoted by Alessandro et al, 2003: 400)

which, by means of an energy spectrum, reflects the grayscale periodicity (spatial frequency

spectrum) in the image. This is a fast technique but performs poorly in practice because of its

lacks of spatial localization (Alessandro et al, 2003:401-405; Meterka and Strzelecki, 1998:3).

These methods will not be discussed further since the emphasis of this work is on pattern

spectra.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 7
PATTERN SPECTRA FOR PATTERN RECOGNITION

Structural methods, on the other hand, describe texture by first defining primitives and

placement rules. These methods have an advantage over statistical ones since they can provide

a symbolic description of the image. Mathematical morphology provides a powerful tool for

structural texture analysis (Meterka and Strzelecki, 1998).

2.3 Mathematical morphology

Mathematical morphology was born in 1964 from the investigation of the relationship between

the geometry of porous materials and their permeability done by Matheron and Serra at the

Ecole des Mines de Paris in Fontainebleau. It was first applied to binary images and then later

extended to grayscale images (Serra, 1988).

Soille (2005:2) simply defined mathematical morphology as a theory for the analysis of spatial

structures. This theory is thus a method of developing a quantitative description of the

geometrical structure of a signal or an image (Maragos and Scharfer, 1987). Hence

quantification consists of a transformation followed by a measurement (Serra, 1988).

Transformation in this framework refers to image processing and measurement refers to image

analysis.

Image processing constitutes any operation that with an image at the input produces an image

at the output. Mathematical morphology transformations operate as follows: sub-images, also

named structuring elements, interact with the original image to modify and extract

information. These structuring elements are related to “textons” in texture images. These

operations were initially only applied to binary images (image with two gray levels). When

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 8
PATTERN SPECTRA FOR PATTERN RECOGNITION

dealing with other images such as a grayscale image (image with more than two gray levels),

it was required to first threshold or convert this image to binary before performing a binary

morphological operation. This was found to be inconvenient and resulted in a loss of

information. In 1978, this problem was remedied by Nakagawa and Rosenfeld (1978) which

linked the two basic operation of mathematical morphology, namely binary dilation and binary

erosion, to maximum and minimum filters when dealing with grayscale images. A

combination of these operations led to advanced operations such as opening, closing, hit-or-

miss transform, top-hat, etc. (Serra, 1982).

An important aspect of a morphological transformation is the choice of a proper structuring

element. In Kotani (1998:57-64) the structuring element is optimized to obtain the best

discrimination of textures in order to categorise them. Similar optimization was performed by

Asano, Miyagawa and Fujio (2000:479-482) for texture characterization. In Chapter 3, the

associated mathematical concepts with illustrative examples are presented.

Image analysis refers to any operation that with an image at the input produces numbers

(measurements) at the output. In Aubert, Jeulin and Hashimoto (2000:253-262),

morphological measurements such as granulometry and anti-granulometry distributions were

analysed for surface texture classification. Two of the important morphological functions used

for image measurements are granulometric and anti-granulometric distribution functions

which characterize the size of objects of an image. The former uses morphological opening,

sometimes referred to as structural opening. The latter uses morphological closing which is

also referred to as structural closing.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 9
PATTERN SPECTRA FOR PATTERN RECOGNITION

2.4 Granulometries

The concept of granulometries has been introduced by Matheron (1967, 1975) in the late

sixties for analyzing objects and structure sizes in the images. Granulometries originally were

formulated for binary images (Dougherty, 1992:72-77) and were referred to as morphological

granulometries involving sequences of openings or closings with structuring elements of

increasing sizes. Other types of granulometries (anti-granulometries) named algebraic

granulometries were introduced by Serra (1988) based on algebraic opening (closing).

Although the granulometries during this period gave adequate descriptions of sizes and shapes

of objects in the images, they remained prohibitively costly on non-specialized hardware

because of their computation time. For this reason, Vincent (2000:119-133) proposed fast

granulometric methods for the extraction of global image information from grayscale images.

Moreover, granulometries as morphological image analysis tools were found particularly

useful for the estimation object sizes in binary and grayscale images. They were also useful for

characterizing textures based on their granulometric curves or pattern spectra (Vincent,

1996:273).

In Matheron (1967), granulometric analysis is often compared to a sifting process, where an

image is sifted through a series of sieves with increasing mesh size. Each mesh size removes

more than the previous one until the image finally becomes blank. The morphology-based

pattern spectrum can be seen as a signature provided by the rate at which an image is sifted.

Maragos (1989:709) has introduced the concept of an oriented pattern spectrum which enables

the extraction of 1D line structures of an image that live in a 2D space. The structuring

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 10
PATTERN SPECTRA FOR PATTERN RECOGNITION

element is a line segment forming an angle with the horizontal. In Werman and Peleg as

quoted by Maragos (1989:709), oriented openings are used for texture analysis.

2.5 Some applications of morphological techniques

Di Ruberto et al (2000:397 ) used morphological operators to detect and classify malaria

parasites in stained blood slides for the purpose of evaluating the parasitaemia of the blood.

Granulometries based on disk-shaped elements were used to capture information on cells and

parasite nuclei. The resulting pattern spectra characterized two predominant particle sizes in

the image, namely the nuclei of the trophozoites (3-7 pixels) and the red blood cells (15-25

pixels). Moreover, Colome-Serra et al (1992:1934-1935) employed greyscale granulometries

as a quantification technique to measure chronic renal damage. This method was found time

consuming.

2.6 Texture classification

Texture classification is the grouping of test samples into classes accordingly to some

criterion. There are two types of classification namely unsupervised classification and

supervised classification. The former is when the classes are not defined a priori and is not

often used for texture applications. The latter corresponds to the case where the classes are

defined a priori and is usually referred to as classification. There are many types of classifiers.

Among them the most used are statistical k-nearest neighbour (k-NN) and neural networks.

Though the k-NN classifier is simple and efficient, a large amount of memory is required,

resulting in slow performance. Since the focus of this work is not on neural networks, readers

are encouraged to consult Bishop (1995) for more detail.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 11
PATTERN SPECTRA FOR PATTERN RECOGNITION

2.7 Summary

Some materials are best characterized by their texture. Texture was defined in Section 2.1. The

concept of texture analysis was discussed in Section 2.2. Mathematical morphology and

granulometries were briefly introduced in Section 2.3 and Section 2.4 respectively. Some

applications using morphological techniques were presented in Section 2.5. Lastly, texture

classification was described in Section 2.6.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 12
PATTERN SPECTRA FOR PATTERN RECOGNITION

CHAPTER 3

I don't see how an epigram, being a bolt from the blue, with no introduction or cue, ever gets itself writ
(William James)

MATHEMATICAL MORPHOLOGY

Mathematical morphology is a general method for processing images based on set theory,

where images are presented as a set of points or pixels on which operations such as union and

intersection are performed (Bleau, Guise, & Leblanc 1992:1). Mathematical morphology was

developed by Matheron (1975) and Serra (1988) at the Ecole des Mines de Paris in

Fontainebleau. This theory has first been applied to binary images and later extended to

grayscale images.

3.1 Basic definitions

An image, in general, consists of a set or collection of pixels belonging to objects in the image.

A pixel is defined as an image unit. Alternatively an image can also be defined as a function of

two real variables, i.e. I ( x, y ) representing an amplitude or a pixel value. For the rest of this

review, we will only consider binary and grayscale images. We will therefore restrict to the

domain to Z 2 .

In Vincent and Soille (1991), a two-dimensional grayscale image is defined as follows:

Definition 3.1 (Two-dimensional grayscale image) Let I be a two-dimensional grayscale

image whose domain is denoted DI ⊂ Z 2 . I takes discrete gray values in a given range [0, N ] ,

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 13
PATTERN SPECTRA FOR PATTERN RECOGNITION

where N is an arbitrary positive integer. Letting p be any arbitrary pixel of I , we can define

a two-dimensional grayscale image as

 DI ⊂ Z 2 → {0,1,..., N }

I:  . (3.1)
 p
I ( p)

3.1.1 Binary image

Based on Definition 3.1, a binary image corresponds to a two-dimensional grayscale image

with two gray levels {0,1} , the range, being 1. A binary image is also defined as a black and

white image for it only has pixels of 1 or 0, corresponding to white and black respectively.

Figure 3.1 shows an example of a binary image.

Figure 3.1: A binary image.

3.1.2 Grayscale image

In this chapter, we defined a grayscale image as a two-dimensional grayscale image with a

limited range. Referring to Definition 2.1, N equals 255 implies 256 gray levels {0,1,…,255}.

Gray levels 0 and 255 correspond to black and white respectively. Figure 3.2 shows an

example of a grayscale image.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 14
PATTERN SPECTRA FOR PATTERN RECOGNITION

Figure 3.2: Grayscale image.

3.1.3 Structuring element

The morphology of an image relies on the analysis of images using elementary patterns or

structuring elements which can be considered as templates. (Awcock & Thomas, 1995:167). A

structuring element, denoted by SE , is as an image subset of Z 2 used to analyse the

topography of the image. For each structuring element we need to define its shape, size and its

center. These three characteristics are subject to the information needed to be extracted from

the image. Figure 3.3 illustrates different type of structuring elements.

(a) Cross (b) Diamond (c) Line SE

Figure 3.3: (a) Cross SE, (b), Diamond SE and (c) Horizontal Line SE. The cross mark is

the reference pixel or center pixel.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 15
PATTERN SPECTRA FOR PATTERN RECOGNITION

Additionally, a structuring element can also be characterized as a flat or non-flat structuring

element. When the intensity values are identical, constituting a uniform platform, this is

referred to as a flat structuring element. In most cases the flat structuring element is a binary

sub-image. On the contrary, the structuring element can be composed of different intensity

values such a grayscale sub-image. This is called non-flat structuring element. The structuring

elements used in this dissertation are flat structuring elements.

3.2 Binary morphological operations

Binary image processing operations have been collectively described as morphological

operations (Serra, 1982; Coster and Chermant, 1985; Dougherty and Astola, 1994, 1999;

Soille, 1999). The most popular operations are dilation and erosion, and their combinations

lead to more advanced morphological operations such as opening and closing.

As known from Awcock and Thomas (1995:165), binary mathematical morphology owes its

origin to set theory and deals with form and structure. In addition to the standard set

operations such as union, intersection, inclusion and complement {∪,∩, ⊂, c }, morphology

depends extensively on the translation operation. Therefore, from the Minkowski set

operations, the fundamental morphological operations are defined.

In this section, a square pixel representation is used to illustrate the effect of morphological

operations, starting with the standard set operations. The foreground is the filled squares or

pixels and the rest, which can be expressed as the complement of the foreground, are called

background. Figure 3.4 demonstrates the intersection, union and the complement operations.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 16
PATTERN SPECTRA FOR PATTERN RECOGNITION

A B Ac

A∪ B A∩ B

Figure 3.4: Example of complement, union and intersection.

Definition 3.2 (Translation) Let A be an image. The translation of A by the point t denoted

by At is defined by

At = A + t = {a + t | a ∈ A}. (3.2)

A t At

Denotes the origin and the coordinates of the translation vector.

Figure 3.5: Translation operation on image A by t .

Figure 3.5 shows how the foreground pixels are shifted with respect to the translation vector t .
____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 17
PATTERN SPECTRA FOR PATTERN RECOGNITION

Definition 3.3 (Minkowski’s operations) Given two sets or vectors A and B ,

Minkowski addition is defined as

A ⊕ B = ∪ ( A + b) (3.3)
b∈B

and Minkowski subtraction as

A B = ∩ ( A + b) (3.4)
b∈B

where b ∈ B .

Using Minkowski’s formulism and the translation notation, morphological operations are

defined in the next section.

3.2.1 Binary dilation

Definition 3.4 (Dilation) The dilation of a binary image A by structure element B , denoted by

A ⊕ B , is defined as

A ⊕ B = ∪ Ab . (3.5)
b∈B

By stepping the reference point or center of the structuring element over each pixel of the

foreground (object in the image to be eroded) until all the foreground pixels of the structuring

element fit over the foreground of the image, and then considering the union of foreground

pixels, the dilated image is produced. Figure 3.6 illustrates an example of dilation which

modifies an image A with respect to a cross structuring element B . Dilation, in general,

enlarges features in the image by adding pixels and fills small holes in the image.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 18
PATTERN SPECTRA FOR PATTERN RECOGNITION

A B A⊕ B

Figure 3.6: Dilation of image A by the structuring element B .

3.2.2 Binary erosion

Definition 3.5 (Erosion) The erosion of a binary image A by structuring element B , denoted

by A B, is defined as

A B = ∩ A−b , (3.6)
b∈B

or

A (− B ) = ∩ A−b , (3.7)
b∈B

where − B = {−b | b ∈ B} is the transposed form of the structuring element set.

Erosion of a binary image by a structuring element can be described intuitively by template

translation. Contrarily to dilation, after stepping the structuring element until all its foreground

pixels fit over the foreground of the image, only the intersection of foreground pixels of the

image and the structuring element are considered to produce an eroded image. Erosion, in

general, shrinks the image by removing foreground pixels in the image and eliminates objects

smaller than the structuring element. This operation is illustrated in Figure 3.7.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 19
PATTERN SPECTRA FOR PATTERN RECOGNITION

A B A B

Figure 3.7: Erosion of image A by the structuring element B .

3.2.3 Binary opening

Erosion is often used to remove noisy pixels or unsuitable small objects from an image.

Regrettably, this morphological operation also shrinks objects in the image. To overcome this

inconveniency, a morphological operation named opening is introduced. Opening is created by

erosion followed by dilation.

Definition 3.6 (Opening) The opening of a binary image A by a structuring element B ,

denoted by A  B , is defined as

A B = ( A B)⊕ B . (3.8)

The opening operation separates connected objects and smoothes object contours. Figure 3.8

shows how the object contour in the image A is smoothed from both the inside and the

outside.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 20
PATTERN SPECTRA FOR PATTERN RECOGNITION

A B A B

Figure 3.8: Opening of image A by the structuring element B .

A geographical metaphor can be employed to appropriately describe the effect of opening:

“opening smoothes object ‘coastlines’, eliminates small ‘islands’ and cuts narrow ‘isthmuses’”

(Awcock & Thomas, 1995:171). This operation, by means of its effect on an image, was found

to be a suitable precursor to studies of size distributions (Kraus et al, 1993:2).

3.2.4 Binary closing

Inversely, closing is created by performing dilation followed by erosion.

Definition 3.7 (Closing) The closing of a binary image A by a structuring element B ,

denoted by A • B , is defined as

A • B = ( A ⊕ B) B. (3.9)

The closing operation smoothes object contours in the image, especially from the outside and

can also fill small holes as illustrated in Figure 3.9. The smoothing effect of the object contour

highly depends on the characteristics of the structuring element.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 21
PATTERN SPECTRA FOR PATTERN RECOGNITION

A B A• B

Figure 3.9: Closing of image A by the structuring element B .

Due to the duality existing between opening and closing, closing can also be used for studies

of distributions of particles.

3.3 Grayscale morphological operations

Real world images are grayscale images. Binary images can be obtained by thresholding

grayscale images. This threshold operation often causes loss of information and introduces

significant errors in segmenting objects from the background that leads to poor results when

performing morphological operations on binary images (Hussain, 1991). For this reason, the

theory of mathematical morphology has been extended to grayscale images and signals (Serra,

1982). This extension can be realized in various ways (Bangham and Marshall, 1998:117-

128). Moreover binary morphological operations can be applied to grayscale images by

considering this kind of image equivalent to a stack of binary images as seen in Figure 3.9.

This method requires the use of a threshold or Look Up Table (LUT) technique for the

decomposition of grayscale image into a stack. Therefore binary operations as described

earlier in Section 3.3 are applied to each stack to give a new set of stacks. Hence the new set

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 22
PATTERN SPECTRA FOR PATTERN RECOGNITION

of stacks corresponds to the decomposition of the resulting grayscale image. Consequently, for

256 gray level images, 256 thresholds, 256 binary morphological operations and 255

summations are performed. Practically, this method as an extension to grayscale images is

time consuming.

<== Binary Image Level 1

<== Binary Image Level 0

Grayscale image Grayscale as a stack of binary

image

Figure 3.10: Grayscale image representation.

In Talbot (1994:211-221), an alternative method is described using grayscale extremum

operations over a neighborhood of pixels. The expression ‘extremum operation’ stands for

maximum (maxn) or minimum (minn) operations (Nakagawa and Rosenfeld, 1978:632-635).

In Section 3.3.1, grayscale operations are defined. Illustrative examples for each grayscale

image operation are included.

3.3.1 Grayscale dilation

Definition 3.8 (Grayscale dilation) The dilation of a grayscale image f by a structuring

element B , denoted by δ B ( f ) , is defined as

δ B ( f ) = ∨b∈B f −b . (3.10)

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 23
PATTERN SPECTRA FOR PATTERN RECOGNITION

Using the extremum operation, grayscale dilation can be express as

δ B ( f )( x ) = max b∈B f ( x + b) , (3.11)

where b denotes the position of point inside the structuring element set B relative to the

center or origin of the structuring element and x denotes the position of point relative to the

origin of the grayscale image f .

To obtain grayscale dilation, a structuring element is scanned over the image and at each

position only the maximum value lying within the structuring element at that position is taken.

Grayscale Image Grayscale Dilation

Figure 3.11: Grayscale dilation.

This operation grows the white regions of the original image and the dilated image looks

brighter.

3.3.2 Grayscale erosion

Definition 3.9 (Grayscale erosion) The erosion of a grayscale image f by a structuring

element B , denoted by ε B ( f ) , is defined as

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 24
PATTERN SPECTRA FOR PATTERN RECOGNITION

ε B ( f ) = ∧b∈B f −b . (3.12)

Using the extremum operation, grayscale erosion can be express as

ε B ( f )( x ) = min b∈B f ( x + b) . (3.13)

Grayscale erosion is obtained by proceeding the same way as in grayscale dilation, but only

the minimum value laying in the structuring element is taken.

Grayscale Image Grayscale Erosion

Figure 3.12: Grayscale erosion.

Contrarily to grayscale dilation, this operation shrinks white regions. Thus the eroded image

looks darker. Figure3.12 shows the effect of grayscale erosion.

3.3.3 Grayscale opening

Definition 3.10 (Grayscale opening) The opening of a grayscale image f by a structuring

element B , denoted by γ B ( f ) , is defined as

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 25
PATTERN SPECTRA FOR PATTERN RECOGNITION

γ B ( f ) = δ B [ε B ( f )] . (3.14)

Using the extremum operation, grayscale opening can be express as

γ B ( f )( x ) = max b∈B (min b∈B f ( x + b) ) . (3.15)

Grayscale Image Grayscale Opening

Figure 3.13: Grayscale opening.

Figure 3.13 shows how an opening of a grayscale image by a disk-shaped structuring, removes

high intensity points.

3.3.4 Grayscale closing

Definition 3.11 (Grayscale closing) The closing of a grayscale image f by a structuring

element B , denoted by φB ( f ) , is defined as

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 26
PATTERN SPECTRA FOR PATTERN RECOGNITION

φ B ( f ) = ε B [δ B ( f )] . (3.16)

Using the extremum operation, grayscale closing can be express as

φ B ( f )( x ) = min b∈B (max b∈B f ( x + b) ) . (3.17)

Grayscale Image Grayscale Closing

Figure 3.14: Grayscale closing.

Figure 3.14 illustrates the effect of closing on a grayscale image by a disk- shaped structuring

element with a three-pixel diameter. This operation fills holes in the image by removing low

valued points.

3.4 Summary

Section 3.1 provided basic definitions dealing with mathematical morphology: binary and

grayscale images and structuring elements as sub-images were defined. Binary morphological

operations were elaborated on Section 3.2. Some illustrative examples were also presented.

The last section was concerned with grayscale morphological operations.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 27
PATTERN SPECTRA FOR PATTERN RECOGNITION

CHAPTER 4

The important thing is never to stop questioning

(Albert Einstein)

GRANULOMETRIES

In Chapter 3, we reviewed primitive morphological operators and their basic geometric

characteristics. More advanced operations are constructed by combining these operators.

Granulometries are considered as morphological filters involving sequences of openings or

closings in order to extract global information from the image (Serra, 1982, 1988).

Granulometries were conceived by Matheron (1975) and first applied to binary images and

then to grayscale images to infer particle size distributions (Tscheschel, Stoyan & Hilfer,

2000:57) and characterize or classify textures (Soille, 1999; Vanrell & Vitria, 1993:152-161;

Chen & Dougherty, 1992) or shapes (Maragos, 1989:701-716). Granulometries in general are

used as a precursor in the classification of features in images of materials. In this chapter,

morphological granulometries and linear granulometries are discussed in detail.

4.1 Basic concept

Granulometries are comparable to a sieving process (Matheron, 1967; Jones & Soille, 1996).

Considering a heap of mixed seeds (or granules), to analyze how many seeds in the heap fit

into several classes, certain sieves with increasing hole sizes are used. The seeds that fall

through a given sieve size in the mesh are then removed. Hence, each set corresponding to a

mesh of a specific sieve size gives information on the seeds in the heap. The result of this

process leads to a discrete function expressing the amount of seeds for each specific sieve size.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 28
PATTERN SPECTRA FOR PATTERN RECOGNITION

From a morphological point of view, these sieves with increasing hole sizes are replaced by a

sequence of openings or closings with structuring elements of increasing size. As quoted by

Vincent (2000:119), these morphological operations have led Matheron (1975) to define a

granulometry as follows:

Definition 4.1 (Granulometry) Let ψ = (ψ λ )λ ≥0 be a family of image transformations

depending on a parameter λ . This family constitutes a granulometry if and only if the

following properties are satisfied:

∀λ ≥ 0 , ψ λ is increasing, (4.1)

∀λ ≥ 0 , ψ λ is anti-extensive, (4.2)

∀λ ≥ 0 , µ ≥ 0 , ψ λψ µ = ψ µψ λ = ψ max( λ , µ ) . (4.3)

Relatively to the analogy mentioned above, Definition 4.1 implies that a granulometry is a

transformation, depending on a size parameter λ , that satisfies the three properties

enumerated as:

1. Increasing: if we divide the initial seeds into two subsets A and B such that B contains

A, the filtering is said to be increasing if the B filtered contains A filtered. (see

Equation 4.7).

2. Anti-extensive: seeds non-filtered are a subset of the initial seeds in the heap. (see

Equation 4.8).

3. Idempotent or absorption: if we filter at two different sizes, we obtain the same result

no matter the order. (see Equation 4.9).

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 29
PATTERN SPECTRA FOR PATTERN RECOGNITION

An opening is an operation which is increasing, anti-extensive and idempotent. An equivalent

definition to Definition 4.1 is proposed by Vincent (2000:119):

Definition 4.2 (Granulometry as proposed by Vincent) Let ψ = (ψ λ )λ ≥0 be a family of

image transformations depending on a unique parameter λ . Then ψ = (ψ λ )λ ≥0 is a

granulometry if and only if it forms a decreasing family of openings, that is

∀λ ≥ 0 , ψ λ is an opening, (4.4)

∀λ ≥ 0 , µ ≥ 0 , λ ≥ µ ⇒ψ λ ≤ψ µ . (4.5)

This definition implies that the opening can be either a morphological opening or an algebraic

opening leading to either a morphological granulometry or algebraic granulometry (Serra,

1988) respectively.

Furthermore, a closing which is a morphological operation that is increasing, extensive and

idempotent (Serra, 1982; Vincent, 1997:119-120) can be used to define a granulometry by

closings as follows:

Definition 4.3 (Granulometry by closings) An increasing family φλ of closings, that is such

that

∀λ ≥ 0 , µ ≥ 0 , λ ≥ µ ⇒ φλ ≥ φ µ , (4.6)

is a granulometry by closings also named anti-granulometry.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 30
PATTERN SPECTRA FOR PATTERN RECOGNITION

The type of object size distribution differs for opening and closing operations. Granulometry

by openings can infer a size distribution of lighter objects as opposed to a granulometry by

closings inferring a size distribution of darker objects (refer to Figure 3.12 and Figure 3.13).

In the next section, most of the definitions are based on previous work of Dougherty

(1992:161, 1992:7-21), Maragos (1989:701-716) and Vincent (1994:43-102). Due to duality,

the morphological opening operation is considered to describe different types of size

distributions.

4.2 Morphological granulometries and pattern spectrum

Since a binary image is also defined as a grayscale image with only two gray levels, this

section is devoted to grayscales images. Grayscale morphological granulometries are then

considered as granulometries using structural operations (i.e. operations based on a structuring

element). These filters as proposed by Matheron (1975) must satisfy Definition 4.1 which

means they must be openings. Not all types of openings satisfy the absorption property

(Equation 4.3), therefore they can not be used as granulometries (Nacken, 1994). For this

reason, the choice of the structuring element is important. However, Matheron has

characterized granulometries based on morphological openings as follows (Matheron as

quoted by Vincent (2000:119)):

Definition 4.4 (Characterization) Let B be a compact set of  n . The family ψ = (ψ λ )λ ≥ 0 of

openings by the homothetic λB = {λb | b ∈ B}, of B is a granulometry if and only if B is

convex.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 31
PATTERN SPECTRA FOR PATTERN RECOGNITION

The structuring element can be a rectangle, a hexagon, or a line, which are then referred to as a

rectangular granulometry, disk granulometry, or linear granulometry, respectively. A disc

structuring element is used here for demonstration. In the next sections, the following

assumptions are made: f ( p ) is the grayscale image, λB the parametric disc structuring

element with r , the radius, and F (r ) the granulometry function, with r ∈ (0, ∞ ) . A sum

projection of F (r ) is referred to as a size distribution, and its first derivative as a pattern

spectrum (Maragos, 1989: 701–716 & Soille, 1999:25-58).

4.2.1 Size distribution

A size distribution is a set of openings ψ r with r from some ordered set Λ that satisfies the

following properties:

Increasingness: f ≤ g ⇒ ψ r ( f ) ≤ψ r ( g ) , (4.7)

Anti-extensivity: ψr(f )≤ f , (4.8)

Absorption: ψ r (ψ s ( f )) = ψ max (r , s ) ( f ) . (4.9)

As shown in Chapter 3, an opening is a morphological operation obtained by producing a

erosion followed by a dilation. Therefore a set of openings with r ≥ 0 , the scaling parameter,

is defined as

ψ r ( f ) = γ λ B ( f ) = max λb∈B ( min λb∈B f ( x + λb) ) , (4.10)

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 32
PATTERN SPECTRA FOR PATTERN RECOGNITION

which, when using the function-set notation, can be written as

ψ r ( f ) = f  λB . (4.11)

The homothetic λB of B with B being a disk structuring element, are illustrated in Figure

4.1.

Figure 4.1: A sequence of increasing structuring elements for λ = 2 , λ = 4 , λ = 6 and

λ = 8.

Furthermore, a size distribution function can be obtained by performing a series of

morphological openings with a sequence of structuring elements of increasing size (Figure

4.1). This function maps each structuring element to the number of objects or image pixels

removed during the opening operation with the corresponding structuring element. Figure 4.2

illustrates how, from an increasing family of structuring element, a decreasing family of

openings is produced.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 33
PATTERN SPECTRA FOR PATTERN RECOGNITION

Figure 4.2: A decreasing family of openings of an image of seeds.

As seen in Figure 4.2, objects in the image are removed or the intensity values of pixels

reduced progressively. By making a sum projection (sum of pixels) of opened images (images

obtained from an opening operation) on a scale-axis, a decreasing function known as a size

distribution, is defined as:

Definition 4.5 (Size distribution)

Let F (r ) be a measure of the image f ( p ) , by assuming that f ( p ) is bounded, at r = 0 ,

F (0 ) = ∑ f ( p ) and at a larger value of r F (r ) = 0 , a size distribution denoted by F (r ) is

defined by

F (r ) = ∑ f  rB( p ) . (4.12)

Equation 4.12 is further used to derive a normalized size distribution defined as

Definition 4.6 (Normalized size distribution)

The normalized size distribution denoted by Ν (r ) and defined as

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 34
PATTERN SPECTRA FOR PATTERN RECOGNITION

F (r )
Ν (r ) = 1 − , (4.13)
F (0)

is a cumulative distribution known as the granulometric size distribution of f ( p ) with respect

to the structuring element B with F (r ) being the volume of f  rB and F (0) the volume of

original image f ( p ) .

An illustration of the size distribution and its normalized size distribution are shown in Figure

4.3.

6
Mixed seeds x 10 Size Distribution Normalized Size Distribution
10 1

8 0.8

6 0.6
V olum e

V olum e

4 0.4

2 0.2

0 0
0 5 10 15 20 0 5 10 15 20
radius radius

Figure 4.3: An image of mixed seeds (left), its size distribution (middle) and its

cumulative normalized size distribution (right).

4.2.2 Pattern spectrum

Size distributions can be used to generate morphological pattern spectra, which resume the

action of a size distribution on a specific image (Urbach & Wilkinson, 2002:305). A pattern

spectrum is defined as the first derivative of the normalized size distribution.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 35
PATTERN SPECTRA FOR PATTERN RECOGNITION

Definition 4.7 (Pattern spectrum)

Let N ' (r ) be the first discrete derivative of the cumulative distribution function Ν (r ) , the

pattern spectrum denoted by P(r ) is the discrete density function, which is defined by

∀r > 0 , P(r ) = N ' (r ) = N (r + 1) − N (r ) . (4.14)

This function is also called a granulometric curve. An example of a pattern spectrum is shown

in Figure 4.4. This pattern spectrum corresponds to the image of seeds in Figure 4.3.

Pattern Spectrum
0.16

0.14

0.12

0.1
Volume

0.08

0.06

0.04

0.02

0
0 5 10 15 20
radius

Figure 4.4: Pattern spectrum of the image of seeds.

The pattern spectrum is considered as a result of the quantification of the rate at which the

grayscale image f ( p ) is being sieved. This method is useful for size and shape analysis of

granular images (Dougherty, 1992 and Maragos, 1989). In Chapter 6, this method has been

implemented for the classification seed images and for the characterization of steel images.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 36
PATTERN SPECTRA FOR PATTERN RECOGNITION

4.3 Linear grayscale granulometries and pattern spectrum

Haralick et al (1991:560-565) has proposed an algorithm that allows the computation of

granulometry functions with respect to any family of homothetic elements. This explicitly

means that the base element or structuring element does not have to be convex. In this section,

a line structuring element is used to preserve some characteristics in the images. This

structuring element can be used in any direction (horizontal, vertical or at a specified angle) to

generate a pattern spectrum in order to simplify size or shape analysis. This method is also

computational efficiency compared to the use of a set of morphological structuring elements

which is computationally costly and intensive; therefore relatively slow (Vincent, 2000:122).

The method described here is the linear grayscale granulometry proposed by Vincent

(2000:126-128), based on openings.

4.3.1 Linear grayscale granulometries

In this context, we consider a grayscale image I as defined in Chapter 3 and the structuring

element to be line segment Ln as illustrated in Figure 4.5.

Ln = • • •  • •


n +1 pixels

Figure 4.5: Illustration of a line segment.

The left and right neighbours of a pixel p belonging to the line segment considered in the

domain DI are denoted by N l ( p ) and N r ( p ) respectively.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 37
PATTERN SPECTRA FOR PATTERN RECOGNITION

The effect of an opening by Ln , n ≥ 0 on a grayscale image I is analyzed in this section. The

structuring element Ln can be in any direction. In order to facilitate the description of this

approach, we merely deal with a horizontal line segment as a structuring element and the

horizontal maximum defined as follows:

Definition 4.8 (Horizontal line segment)

A horizontal line segment S , of length l (S ) is defined as a set of pixels {p0 , p1 ,… , pn −1} such

that for 0 < i < n , pi = N r ( pi −1 ) .

An illustrative example of horizontal line segment S is presented in Figure 4.6. For example

the right neighbour of p0 is p1 and the one of p1 is p2 .

p0 p1 p2 p3 p4 p5 p6 p7

Figure 4.6: Illustration of a horizontal line segment of length l ( S ) = 8 .

Definition 4.9 (Horizontal maximum)

A horizontal maximum segment M of length l (M ) = n in a grayscale image I is the horizontal

line segment {p0 , p1 ,… , pn −1} such that

∀i , 0 < i < n , I ( pi ) = I ( p0 ) , (4.15)

and

I ( N l ( p0 )) < I ( p0 ) , I ( N r ( pn −1 )) < I ( p0 ) . (4.16)

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 38
PATTERN SPECTRA FOR PATTERN RECOGNITION

Therefore, by means of Definitions 4.8 and 4.9, an opening equivalent to the standard

morphological opening of an image I by a structuring element B denoted by I  B is

proposed by Vincent (2000:126) as:

Definition 4.10 (Linear opening)

Consider a horizontal maximum M = {p0 , p1 ,…, pn −1} , its length l (M ) = n and p ∈ M

∀k < n , ( I  Lk )( p ) = I ( p ) , (4.17)

for k = n , ( I  Ln )( p ) = max {I ( Nl ( p0 ) ) , I ( N r ( pn−1 ) )} , (4.18)

∀k > n , ( I  Lk )( p ) < I ( p ) . (4.19)

Literally, any opening of I by the line segment Lk such that k < n leaves this maximum

unchanged and on the other hand for any k ≥ n , any opening I  Lk of p , with p ∈ M , is lower

than in I .

Considering the cross-section of I in Figure 4.8, the opening of I with respect to the

Maximum 2 by a horizontal line segment Ly with ( y > 0) is calculated as

o ∀y < m , the intensity values I ( p ) and p ∈ Maximum 2 implies that the

opened region around the Maximum 2 remains unchanged

(( I  L ) ( p ) = I ( p )) .
y

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 39
PATTERN SPECTRA FOR PATTERN RECOGNITION

o For y = m , the intensity values at the location of the maximum 2 changes to the

maximum of its neighbours (right and left, also see Equation 4.21). Hence the

m-th bin is removed and the resulting cross-section is less than the original

cross-section.

o ∀y > m , the intensity values at the location of the Maximum 2 changes to lower

values than the maximum of the right neighbour and the left neighbour of

Maximum 2.

Equations 4.18 and 4.19 can be reduced to one equation: ∀k ≥ n , ( I  Lk )( p ) < I ( p ) .

4.3.2 Horizontal pattern spectrum

A pattern spectrum can be generated by quantifying the local effect of an opening of size

l (M ) = n on the pixels of the maximum M . The value of each pixel p belonging to M is

decreased from I ( p) to the maximum value of the neighbourhood pixels

{ }
max I ( N l ( p0 ) ) , I ( N r ( pn −1 ) ) . Hence, in granulometric terms, the contribution of

maximum M to n -th bin of the horizontal pattern spectrum denoted PS h (I ) is

n ×  I ( p ) − max I ( N l ( p0 ) ) , I ( N r ( pn −1 ) )  .
{ } (4.20)
 

The computation of the n-th bin horizontal pattern spectrum is illustrated in Figure 4.5 where

{ }
h1 and h2 correspond to I ( p ) and max I ( N l ( p0 ) ) , I ( N r ( pn −1 ) ) respectively.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 40
PATTERN SPECTRA FOR PATTERN RECOGNITION

Maximum M
h1

h2
l (M )

Figure 4.7: Cross-section of I with a maximum M .

Figure 4.7 illustrates how the horizontal pattern spectrum is derived. By multiplying the height

(h1 − h2 ) by l (M ) , we get the volume of the shaded area which corresponds to the local

contribution of this maximum to the l (M ) -th bin of the horizontal pattern spectrum (Vincent,

2000:126).

As described in Section 4.3.1, the horizontal opening of size l (M ) on the maximum M creates

{ }
a new plateau P at height equal to max I ( N l ( p0 ) ) , I ( N r ( pn −1 ) ) . This plateau P contains

the maximum M and may correspond itself to a maximum of I  Ln . In the case when it is a

maximum, its contribution to the l ( P ) -th bin of the pattern spectrum can be computed.

Consequently, the pattern spectrum of a grayscale image I is realized as follows: each line of

I is scanned from the left to the right. Each horizontal maximum of current line is then

identified, and its contribution to PS0 ( I ) is determined. If the new plateau containing the

previous maximum becomes a maximum, the contribution of this maximum to pattern

spectrum is computed as well. This process is iterated until the plateau formed is no longer a
____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 41
PATTERN SPECTRA FOR PATTERN RECOGNITION

maximum or until the horizontal line considered is covered. The next maximum of current line

is then considered, etc. until the line considered is covered. This process is illustrated by

Figure 4.8.

Figure 4.8: Illustration of linear grayscale algorithm for a line with two maxima.

Therefore the pattern spectrum is expressed as

( )
PS[n] = PS[n] + n × I ( P ) − max ( I ( pl ) , I ( pr ) ) , (4.21)

and the algorithm to generate the pattern spectrum is shown in Table 4.1.

Table 4.1: Algorithm of a horizontal granulometry for a line of image I (Vincent,

2000:126-127).

 Initialize pattern spectrum: for each n > 0 , PS [n] ← 0

 For each maximum M of this line (in any order) do:

- Let P ← M be the current maximum considered

- While P is a horizontal maximum, do:

pl ← N l ( P ) , neighbour of P to the left;

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 42
PATTERN SPECTRA FOR PATTERN RECOGNITION

pr ← N r ( P ) , neighbour of P to the right;

n ← l ( P ) , length of horizontal maximum P ;

Add contribution of maximum P to n -th bin of pattern spectrum:

( )
PS[n] = PS[n] + n × I ( P ) − max ( I ( pl ) , I ( pr ) ) ;

For any pixel p in P :

I ( P ) ← max ( I ( pl ) , I ( pr ) ) ;

P ← new plateau of pixels formed after opening of size n of P ;

- Put special marker on the left and right of the current plateau P so that

while processing subsequent maxima, we already know that this region

is a plateau and can skip over it;

The horizontal pattern spectrum corresponding to a sample image shown in Figure 4.3 is

illustrated in Figure 4.9.

5
x 10 Horizontal Pattern Spectrum
4.5

3.5

3
Volume

2.5

1.5

0.5

0
0 50 100 150
Length(n)

Figure 4.9: Illustration of the horizontal pattern spectrum of the mixed seeds image

shown in the left of figure 4.2.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 43
PATTERN SPECTRA FOR PATTERN RECOGNITION

This algorithm (Table 4.1) offers a very efficient, useful and accurate way to characterize a

grayscale image by extracting global size information directly from it. Additionally, the

execution time of this algorithm has eased the use of grayscale granulometries more

systematically. The results of implementing this algorithm on images of seed mixtures and

steel materials are presented in Chapter 6.

4.4 Opening trees and grayscale granulometries

The concept of an opening tree is generally proposed as a gray level extension of the opening

transform or a grayscale generalization of the concept of granulometry functions. As known

from the grayscale granulometry when the size of the opening increases, the values of each

pixel decreases monotonically. In Vincent (1996:273-280), an opening tree was used to

compactly represent the successive values of each pixel of a cross-section when performing

linear openings of increasing size in any direction. Figure 4.8 illustrates how the opening tree

can be used to capture the entire granulometric information for the cross-section shown in

Figure 4.6.

Figure 4.10: Opening tree representation of the cross-section shown in figure 4.8. The

leaves of the tree represent the image pixels.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 44
PATTERN SPECTRA FOR PATTERN RECOGNITION

Each node denoted by ( h, n ) represents the value h for the linear opening of size n . This

representation facilitates the computation of the pattern spectrum. This is mostly applied to

image of big size. An algorithm using the opening tree representation is illustrated in Table

4.2.

Table 4.2: Algorithm of a linear granulometry of the grayscale image I from its opening

tree representation.

 Initialize each bin of the pattern spectrum: for each n > 0 , PS [n] ← 0

 For each pixel p of I do:

- v ← I ( p) ;

- ( h, n ) ← node pointed at by p ;

- While ( h, n ) exists, do:

PS [ n] ← PS [ n] + ( v − h ) ;

v ← h;

( h, n ) ← next node down to the tree;

Though the algorithm in Table 4.2 is less efficient than the algorithm described in Table 4.1, it

easily generalizes the computation of granulometries using maxima of linear opening in

several orientations.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 45
PATTERN SPECTRA FOR PATTERN RECOGNITION

4.5 Summary

Two granulometric techniques were described in this chapter. The morphological

granulometry which uses convex structuring elements such as square, disk etc. was elaborated

on Section 4.3. The linear granulometry which used a horizontal line segment as a simple

structuring element was also discussed. Pattern spectra obtained from the morphological

granulometry were more descriptive than those obtained from the linear granulometry. In

terms of execution time the linear granulometry performed faster than the morphological

granulometry. An extension of the linear greyscale granulometry which is based on opening

trees was discussed in Section 4.4.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 46
PATTERN SPECTRA FOR PATTERN RECOGNITION

CHAPTER 5

To copy others is necessary but to copy oneself is pathetic

(Pablo Picasso)

SLOPE PATTERN SPECTRA

A novel algorithm using integral images, to derive slope pattern spectra is proposed in this

chapter. Although many pattern spectra algorithms have their roots in mathematical

morphology the proposed algorithm does not have its roots in mathematical morphology.

Granulometries by means of their resulting pattern spectra constitute a useful tool for texture

or image analysis since they are used to characterize size distributions. The slope pattern

spectra algorithm similar to morphological or linear granulometries extracts increasing slope

segments as pattern spectra. This slope pattern spectra algorithm is proposed as a fast and

robust alternative to granulometric methods.

5.1 Integral image

Image features, called integral image features, can be computed very rapidly by means of an

intermediate representation of an image called the integral image by Viola and Jones (2001).

These features are also referred to as “summed area tables” by Crow (1984:207-212) and

Lienhard & Maydt (2002:155-162). Moreover, the value of the integral image at location

(x, y ) is the sum of all the grayscale pixel values above and to the left. In Viola and Jones

(2001) this is illustrated as in Figure 5.1 with reference to Definition 5.1.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 47
PATTERN SPECTRA FOR PATTERN RECOGNITION

Figure 5.1: The value at the point ( x, y ) corresponds to the sum of all pixels in the

shaded area.

Definition 5.1 (Integral image)

Let i ( x, y ) be a grayscale image. The corresponding integral image ii ( x, y ) is defined as

ii ( x, y ) = ∑ i ( x ' , y ') ,
x '≤ x , y '≤ y
(5.1)

and it can be computed in one pass over the original image using

s ( x, y ) = s ( x, y − 1) + i ( x, y )
 , (5.2)
ii ( x , y ) = ii ( x − 1, y ) + s ( x , y )
where s (x, y ) is the cumulative row and s (x,−1) = 0 and ii (− 1, y ) = 0 .

According to Mitri et al (2005), the above definition can be reduced to

x y
ii ( x, y ) = ∑ ∑ i ( x' , y ') . (5.3)
x '= 0 y '= 0

In the next section, we will apply the integral image technique on a horizontal line segment

(see Definition 4.7).

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 48
PATTERN SPECTRA FOR PATTERN RECOGNITION

By taking a horizontal line segment S of length l (S ) , and applying the integral image

transformation on it, a horizontal line segment of the same length is obtained. Therefore we

propose a definition for this representation:

Definition 5.2 (Integral horizontal line segment)

An integral horizontal line segment IS , of length l ( IS ) = n is a horizontal line segment

{p0 , p1 ,…, pn−1} in the integral image F ( p ) of a grayscale image f ( p ) such that
i
for 0 ≤ i < n , F ( pi ) = ∑ f ( pk ) , (5.4)
k =0

and

for 0 ≤ i < n − 1 , F ( pi ) ≤ F ( N r ( pi ) ) . (5.5)

An integral horizontal line segment is illustrated by Figure 5.2.

Figure 5.2: A line segment and its integral representation.

The right neighbour pixel is the sum of all left neighbour pixels. A growth in pixel value is

observed as we move toward the right. Hence the resulting integral image is monotonically

increasing. It is important to mention that the pixel intensity value may exceed 255, but as
____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 49
PATTERN SPECTRA FOR PATTERN RECOGNITION

underlined in Definition 5.2, this is only an intermediate representation. Therefore it does not

have a definite meaning in image processing, but is nevertheless importance for consequent

image analysis. Important results were derived from the integral representation of the

horizontal line segment.

By observing the integral horizontal line segment, a slope segment is defined as the variation

in terms of intensity value when moving from one pixel location to another. This variation can

also be referred to as a discrete derivative function.

Let ∆F be the representation of the variation in terms of the intensity values from one pixel

location to another on the integral line segment IS . Hence ∆F is defined as

∀i ∈ , ∆F ( i ) = F ( N r ( pi ) ) − F ( pi ) . (5.6)

This representation of the variation of intensity values on the integral horizontal line segment

leads to a definition of a slope segment:

Definition 5.3 (Slope segment)

A slope segment denoted by SS , of length l ( SS ) = n is an integral horizontal line segment

{ p0 , p1 ,… , pn−1} such that

∀0 ≤ i < n − 1 , ∆F ( i ) ≤ ∆F ( i + 1) , (5.7)

or

∀0 ≤ i < n − 1 , ∆F ( i ) ≥ ∆F ( i + 1) . (5.8)

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 50
PATTERN SPECTRA FOR PATTERN RECOGNITION

Definition 5.3 corresponds to two types of slope segments. Equation 5.8 is referred to as a

decreasing slope segment and Equation 5.7 as an increasing slope segment.

Definition 5.4 (Increasing slope segment)

A increasing slope segment denoted by ISS , of length l ( ISS ) = n is a slope line segment

{ p0 , p1 ,… , pn−1} when ∀0 ≤ i < n − 1 , ∆F ( i ) ≤ ∆F ( i + 1) .

These definitions are illustrated in Figure 5.3.

45
Integral horizontal line segment
Curve representation of slope segment
40
Cross section of an image

35

30
Grayscale

25

20

15

10

0
1 2 3 4 5 6 7 8 9 10
n-th pixels

Figure 5.3: Illustration of a cross section of a line from an image, its integral

representation and a curve corresponding to the integral line segment indicating two

increasing slope segments: the 1st one going from the 1st pixel to the 4th pixel and the 2nd

one from the 6th pixel till the 8th pixel.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 51
PATTERN SPECTRA FOR PATTERN RECOGNITION

Further, by comparing different slope segments, an increasing slope segment is therefore

identified as the slope segment which represents an increasing variation in terms of intensity

values between two consecutive pixel locations.

Quantifying an increasing slope segment in each horizontal line of the image can be done

using Definitions 5.5 and 5.6:

Definition 5.5 (Measure of the increasing slope segment)

Let m ( ISS ) be a measure of this increasing slope segment. This measure can be taken as an

area covered by the slope segment. In addition, we know that ISS = { p0 ,..., pn −1} , and

that ∀0 ≤ i < n − 1 ∆F ( i ) = F ( N r ( pi ) ) − F ( pi ) , therefore the measure of the increasing slope

segment ISS is defined as

n−2
m ( ISS ) = ∑ ∆F ( i ) . (5.9)
i =0

This can be explicitly expressed as

m ( ISS ) = F ( N r ( p0 ) ) − F ( p0 ) + F ( N r ( p1 ) ) − F ( p1 ) + ... + F ( N r ( pn− 2 ) ) − F ( pn− 2 ) , (5.10)

which can be simplified to

m ( ISS ) = F ( N r ( pn − 2 ) ) − F ( p0 )

= F ( pn −1 ) − F ( p0 ) . (5.11)

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 52
PATTERN SPECTRA FOR PATTERN RECOGNITION

Definition 5.6 (Contribution of an increasing slope segment)

The contribution of the increasing slope segment to the n -th bin of the increasing slope

pattern spectrum PS ISS ( f ) is

m ( ISS )
, (5.12)
n

where n = l ( ISS ) is the length of the increasing slope segment.

5.2 Proposed algorithm

Based on the above definitions, the principle of the proposed algorithm is as follows: consider

a grayscale image f where horizontal lines of f are considered one after the other, and

scanned from left to right. In each horizontal line, possible slope segments SS are determined.

If the slope segment is an increasing slope segment ( ISS ), the pattern spectrum of size the

length of the increasing slope segment l ( ISS ) = n is then computed as the measure of the

increasing slope segment divided by its length l ( ISS ) = n . In addition, if the lengths of the

increasing slope segment determined at different horizontal lines are equal, the same pattern

m ( ISS )
spectrum bins are respectively incremented by . These operations are repeated until
n

the last horizontal line of f ( p ) is processed.

This proposed algorithm for a line of the grayscale image f is presented in Table 5.1.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 53
PATTERN SPECTRA FOR PATTERN RECOGNITION

Table 5.1: Proposed slope pattern spectra algorithm.

Initialization:

Pattern spectrum: for each n > 0 , PS [n] ← 0

Initial integral pixel value: F ( p) ← 0

Length of ISS : l ( ISS ) ← 0

for each pixel pi in the integral horizontal line and i ≥ 0 do:

F ( pi ) ← F ( p ) + f ( pi ) , the integral image;

∆F ( i ) ← F ( pi ) − F ( p ) , the slope segment;

if ∆F ( p ) > 0 and l ( ISS ) = 0

l ( ISS ) ← l ( ISS ) + 1 , length of the ISS increases;

∆F (i ) reference ← ∆F (i ) , ∆F ( i ) is stored;

else if ∆F ( i ) ≥ ∆F (i )reference

l ( ISS ) ← l ( ISS ) + 1 , length of the ISS increases;

∆F (i ) reference ← ∆F (i ) , the variation ∆F ( i ) is stored;

( )
m ( ISS ) ← ∆F ( pi ) − ∆F pi −l ( ISS ) ; measure of ISS

Specialmark = 0 , an ISS was found and measured;

else if Specialmark = 0

n ← l ( ISS ) , length of the measure ISS;

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 54
PATTERN SPECTRA FOR PATTERN RECOGNITION

m ( ISS )
PS [ n] ← PS [ n ] + , contribution of ISS to n -th bin
n

of PS ISS ;

∆F (i ) reference ← ∆F (i ) , ∆F ( i ) is stored;

l ( ISS ) = 1 ;

Specialmark = 1 , move to the next ISS;

end if

F ( p ) = F ( pi ) , the pixel pi is stored then we go to next one;

end for

An example of an image and its pattern spectrum obtained by the proposed algorithm is shown

in Figure 5.4.

Figure 5.4: Slope pattern spectrum of a mixed seed image.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 55
PATTERN SPECTRA FOR PATTERN RECOGNITION

5.3 Summary

An algorithm that derives pattern spectra based on the increasing slope segments was

proposed. The capability of discriminating texture with different patterns will be demonstrated

through two experiments described in Chapter 6 and the corresponding results will be

presented in Chapter 7.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 56
PATTERN SPECTRA FOR PATTERN RECOGNITION

CHAPTER 6

To steal ideas from one person is plagiarism; to steal from many is research
(Anonymous)

EXPERIMENTAL DESIGN

6.1 The proposed supervised system

In this chapter, a system is proposed to classify materials comprising the following three

stages as shown in Figure 6.1:

o Firstly, at the feature extraction stage, the information of features in the image is

extracted as a pattern spectrum. This can be done by the granulometric methods

reviewed in Chapter 4, or the new algorithm proposed in Section 5.2.

o Secondly, these pattern spectra are then fed to a suitable neural network depending on

the tasks to be performed.

o Finally, the classification or regression (depending on the application) is done at the

neural network stage.

In the following sections, the functionalities of each stage are clarified.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 57
PATTERN SPECTRA FOR PATTERN RECOGNITION

6.2 Feature extraction

This stage can be implemented by using one of three different methods for the extraction of

pattern spectra feature. The first method is the traditional granulometric method reviewed in

Section 4.2 which, according to Vincent (2000:119), is prohibitively costly, and has prevented

granulometries reaching a high level of popularity. The second method is the linear grayscale

granulometry also reviewed in Section 4.3, which is very efficient and faster than traditional

granulometric algorithms. The last one is the proposed slope pattern spectra algorithm not

based on mathematical morphology. This proposed method is based on the intermediate

representation of an image named integral image (Viola and Jones 2001) as was explained in

Chapter 5.

Input image

Feature extraction

Neural networks

Classification Characterization

Figure 6.1: Flowchart of the proposed supervised system.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 58
PATTERN SPECTRA FOR PATTERN RECOGNITION

6.3 Neural networks

As mentioned earlier, the nearest neighbour rule is a powerful classification technique (Duda

and Hart, 1973). Although this technique is simple to implement, it requires a large memory

space and has a long computing time. Consequently it is not figuring widely in near real-time

pattern recognition applications. Moreover, for some similar applications the recognition

performance of the nearest neighbour rule is comparable to Radial Basis Function (RBF) and

neural network classifiers (Yan, 1993: 317-324).

In the next sections, two applications based on neural networks are presented, namely

classification and regression using respectively the feed-forward neural network classifier and

the radial basis function neural network.

6.3.1 Classification (Seed images)

The classification of materials can be done by means of the pattern spectrum derived from

sample images. According to Bishop (1995:5), classification problems are overcome by

assigning an image via its pattern spectrum to one of two (or more) classes, C1 or C2 . In

addition, the outcome of the classification can be represented in terms of y which is 1 if the

pattern spectrum belongs to C1 , and 0 if it belongs to C2 . Thus pattern classification can be

seen as a mapping from a set of input variables x1 ,..., xn , to an output variable y , representing

the class label, often referred to as the target. This mapping is modeled in terms of some

mathematical function (Bishop, 1995:5) which contains a number of adjustable parameters and

is expressed as

yk = yk ( x; w) , (6.1)

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 59
PATTERN SPECTRA FOR PATTERN RECOGNITION

where x = x1 ,..., xn denotes the feature vector, w denotes the vector of parameters which is

often called weights, and k = 1,..., c denotes the k classes.

Practically, neural network models are determined by the choice of the set of functions y ( x; w) .

The values of the weights are determined based on the data set. This process is referred to as

learning or training. The data set is generally called a training set. The learning process is

called supervised learning if the process of determination of the weights involves the target

values, and called unsupervised learning if it does not involve the use of the target values

(Bishop, 1995:10). For the concerned classification application, a supervised approach based

on the back-propagation algorithm is sufficient since the characteristics of the samples images

are known by means of their respective pattern spectra. The classification network for our seed

images was chosen as a two-layer feed-forward neural network. This is illustrated by Figure

6.2.

As any other neural network, the feed-forward network constitutes a set of neurons or nodes.

These neurons can be grouped to constitute a layer. Figure 6.2 presents three layers which are

the input layer, the hidden layer and the output layer. Furthermore, a general artificial neuron

has five components which are:

o A set of inputs, xi.

o A set of weights, wi.

o A bias, bi.

o An activation function, f.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 60
PATTERN SPECTRA FOR PATTERN RECOGNITION

o A neuron output, yi.

Therefore the output of the network is generally expressed as

m
yi = f (∑wijxj +bi ) . (6.2)
j=1

6.3.2 Regression (HSLA steel images)

Regression implies that the outputs of the network are values of continuous variables, unlike

in the classification problems where new inputs are assigned to one of the discrete classes. The

regression problem is addressed in this chapter by employing an RBF function approximation

networks. RBF neural networks are good for implementing and modelling any continuous

input-out mapping and are mainly used in supervised applications (Bors, 2001:1,3).

Inputs Hidden Layer Output

x1
1
x2
1
x3 . . yi
1
. .
. .

xi
1

Figure 6.2: A feed-forward network architecture.

The typical topology of RBF networks is presented in Figure 6.2. Only one hidden layer is

used in this case (unlike for the back propagation topology where more than one hidden layer
____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 61
PATTERN SPECTRA FOR PATTERN RECOGNITION

can be used to achieve good performance). Each neuron in the hidden layer implements a

radial basis function, which in most cases is the Gaussian basis function (Moody, 1989:281-

294, Bishop, 1995:168 and Bors & Pitas, 1994:335-356). The Gaussian basis function is

expressed as

[
φ j ( X ) = exp − (X − µ j )T ∑ j (X − µ j ) ,
−1
] 6.3)

for j = 1,..., L , where X is the input feature vector, L is the number of hidden neurons, µ j and

∑ j
are the mean and the covariance matrix of the j-th Gaussian function. Equation 6.2

implies that the output of the network in Figure 6.2 can be expressed as

 m 
yi ( X ) = f  ∑ wijφ j ( X ) + wi 0  , 6.4)
 j =1 

or

M 
yi ( X ) = f  ∑ wijφ j ( X ) , (6.5)
 j =1 

when wi 0 , the biases, are absorbed into the weights.

6.3.3 Network training

One of the important aspects of neural networks is the training process. This process can be

either supervised or unsupervised, but we only deal with the supervised training process where

the inputs and outputs, respectively called training set and testing set, are provided. In this case

the network processes the inputs and compares its resulting outputs with the known outputs.

Furthermore, the weights, also called network parameters, are obtained by minimizing the

error function also known as the cost function.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 62
PATTERN SPECTRA FOR PATTERN RECOGNITION

6.4 Implementation details of the proposed system

Two types of materials are considered for the implementation of the proposed system. The

first is a mixture of the seeds and the second is High Strength Low Alloy (HSLA) steels.

6.4.1 Sample images for classification (Seed images)

The seed mixture is constituted of sunflower and millet seed. A glass meter of maximum

volume of 200 mm was used to measure the contribution of each type of seed to the mixture.

Table 6.1 shows each mixture and the volume of each seed contribution.

Table 6.1: Mixture of seeds.

Training set Testing set Sunflower Millet Class

Mixture 1 Mixture 15 25 ml 175 ml 0
Mixture 2 Mixture 16 25 ml 175 ml 0
Mixture 3 Mixture 17 50 ml 150 ml 0
Mixture 4 Mixture 18 50 ml 150 ml 0
Mixture 5 Mixture 19 75 ml 125 ml 0
Mixture 6 Mixture 20 75 ml 125 ml 0
Mixture 7 Mixture 21 125 ml 75 ml 1
Mixture 8 Mixture 22 125 ml 75 ml 1
Mixture 9 Mixture 23 125 ml 75 ml 1
Mixture 10 Mixture 24 125 ml 75 ml 1
Mixture 11 Mixture 25 150 ml 50 ml 1
Mixture 12 Mixture 26 150 ml 50 ml 1
Mixture 13 Mixture 27 175 ml 25 ml 1
Mixture 14 Mixture 28 175 ml 25 ml 1

The fourteen samples numbered 1 to 14 constitute the training set and the rest were used as the

testing set. The samples constituting the training set were pre-defined according to the ratio of

sunflower and millet volumes. Hence two classes were defined: class ‘1’ corresponding to the

volume of sunflower greater than the volume of the millet, and class ‘0’ in the contrary case.
____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 63
PATTERN SPECTRA FOR PATTERN RECOGNITION

This is illustrated by the last column in Table 6.1. Some of the sample images are shown in

Figure 6.3.

Figure 6.3: Examples of seed sample images.

6.4.2 Sample image for regression (HSLA steel images)

The steel samples were prepared as part of a study by the Department of Chemical and

Metallurgical Engineering at TUT to determine the effect on the microstructure of steel, when

laser instead of mechanical forming is used. Flat pieces of high strength steel micro-alloyed

with 0.03wt% Nb (Niobium), having a thickness of 3.5 mm, were laser treated using a 1kW

CO2 laser with an 8 mm diameter beam. Five laser scans were applied per cycle and each

sample was treated with a total of 13 cycles (i.e. a laser treated regions).

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 64
PATTERN SPECTRA FOR PATTERN RECOGNITION

The samples were sectioned using a cut-off machine, mounted onto a resin, and then ground

and polished using standard metallographic methods. Lastly the samples were etched in 2%

Nital, viewed under a microscope with a magnification factor of 200% and recorded using a

digital camera.

Twenty-four 256x256 images spaced evenly along the thickness of the sectioned sample in the

centre of a cycle (i.e. a laser treated region) were recorded for training a neural network. Two

samples were recorded at depths of 0.2917mm, 0.5834mm, 0.8751mm, 1.1668mm, 1.4585mm,

1.7502mm, 2.0419mm, 2.3336mm, 2.6253mm, 2.9170mm, 3.2087mm and 3.5004mm, giving a

total of 24 samples. In addition, another set of twenty-four 256x256 images, having the same

micro structural properties as the training set, were collected as the testing set. Some of the

sample images are shown in Figure 6.4.

Figure 6.4: Examples of HSLA sample images.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 65
PATTERN SPECTRA FOR PATTERN RECOGNITION

6.4.3 Pattern spectra implementation

Three types of pattern spectra were generated. The first corresponds to the morphological

pattern spectra obtained from the granulometric method using a disk as a structuring element.

The second is the linear pattern spectra that are generated by the linear granulometric

technique explained in Chapter 4. The last and third one is the slope pattern spectra discussed

in Chapter 5. The amplitudes of the linear and slope pattern spectra were scaled for improved

training and testing. Each pattern spectrum has been divided by the maximum peak of the

training set. The pattern spectra serve as inputs to the feed-forward or RBF networks.

6.4.4 Classification implementation for seed images

This application is realized by creating a two-layer feed-forward network using MATLAB the

following parameters:

o One hidden layer having ten neurons and an output layer constituting only one neuron.

o Each neuron in the network is activated with the logsig function. This activation

function is defined as

1
log sig ( n ) = , (6.6)
1 + e− n

and it corresponds to the log-sigmoid transfer function.

o The network is trained using the Levenberg-Marquardt backpropagation algorithm to

determine or update the weights and bias values. The training is stopped when the

target is reached (the output of the nework is equal to the target (class) when the

network is simulated with the training set (Mixture 1 until mixture 14) as input). The

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 66
PATTERN SPECTRA FOR PATTERN RECOGNITION

network is tested with the testing set (Mixture 15 until mixture 28) and the results

reported in the next chapter.

6.4.5 Regression implementation for steel images

A regression neural network is created to approximate the depth of an HSLA sample.

MATLAB offers a function newgrnn that creates a two-layer RBF network. The first layer has

radbas neurons. The function radbas implements Equation 6.6. The second layer, which is the

output layer, has purelin neurons. The purelin function is a linear function that is defined as

purelin ( n ) = n . (6.7)

Both the training set and the target values (classes) are used for determining the weight values.

The result obtained from this network, when simulating with a test set, is an approximation to

the target i.e. the values 0.2917, 0.5834 etc. given in section 6.4.2.

6.5 Summary

A supervised system was proposed in Section 6.1. The different stages namely feature

extraction and neural processing (classification or characterization) of the proposed system

were briefly described in this section. Feature extraction was discussed in Chapters 4 and 5:

the resulting pattern spectra from the traditional granulometries, the linear greyscale

granulometries and the slope pattern spectra were considered as feature vectors. In Section

6.3, details on neural networks with respect to classification and characterization (regression)

were provided. The feed-forward network for classification purposes and the radial basis

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 67
PATTERN SPECTRA FOR PATTERN RECOGNITION

function network for approximation purposes were also discussed. The implementation of

each stage of the proposed system was discussed in the final section.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 68
PATTERN SPECTRA FOR PATTERN RECOGNITION

CHAPTER 7

You may never know what results come of your action, but if you do nothing there will be no result
(Mahatma Gandhi)

EXPERIMENTAL RESULTS AND DISCUSSION

7.1 Experimental results

The seed samples are characterized by individual objects repeated in a random manner. The

steel texture shows a repeating pattern, but the represented objects are not as distinguishable as

for the seed samples. The steel samples are characterized by their microstructures. Pattern

spectra were extracted from the seed and steel images which were fed to the neural network

for classification and characterization. In this section, experimental results are presented.

These results are with respect to each stage implemented for the system proposed in Chapter 5,

and are discussed in Section 7.2. These experimental results are obtained using MATLAB

code running on a personal computer Intel Pentium 4 with a CPU speed of 2.66 GHz and a

memory size of 256MB RAM.

7.1.1 Data acquisition results

The acquired data is shown in Appendices A and B. In the first column of Appendix A forty-

eight sample images of the mixture of seeds are represented. These sample images are

separated into two equal sets. The first set constitutes sample images named “Mixture 1”,

“Mixture 2” up to “Mixture 14” and the rest of sample images (Mixture 25 up to Mixture 48)

constitute the second set.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 69
PATTERN SPECTRA FOR PATTERN RECOGNITION

In Appendix B, forty-eight sample images representing the texture of the steel are shown.

These sample images are also grouped into two equal sets in which the first twenty-four form

the training set and the remaining twenty-four the testing set.

7.1.2 Pattern spectra results

The feature extraction results are shown in Appendices A and B. The morphological pattern

spectra are shown in second column, the linear pattern spectra in the third column and the

slope pattern spectra in last and fourth column. Each pattern spectrum corresponds

respectively to each sample image represented in the first column.

The execution times of the traditional, linear granulometry and the proposed algorithm for

computation of a greyscale pattern spectrum is given in Table 7.1. The calculations of the

execution times were made using the MATLAB commands ‘tic’ and ‘toc’ for computing a

pattern spectrum. The execution time for each sample was measured and the average time was

calculated as the total execution time to compute the pattern spectra divided by the number of

samples. For any sample image, whether it is a mixture (seed sample) or a texture (HSLA

sample), the ratio (linear granulometry/ proposed algorithm) is about two orders of magnitude,

and the ratio (traditional granulometry/ proposed algorithm), is about five orders of magnitude.

For illustration, ten samples were used: five seed images and five steel images.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 70
PATTERN SPECTRA FOR PATTERN RECOGNITION

Table 7.1: Execution time of different feature extraction techniques for some sample

images shown in Appendix A and B.

7.1.3 Classification results for seed images

The classification results for seed images are shown in Table 6.2. The neural network when

tested using the training set (Mixture 1 to Mixture 14) as the inputs to the network gives a

100% of classification success, meaning each input to the network belongs to its correct class

(see second row in Table 7.2). The next row corresponds to the results when the network is

tested using the testing set (Mixture 15 to Mixture 28).

Table 7.2: Classification results.

Features Morphological pattern spectra Linear pattern spectra Slope pattern spectra
Training set 100% 100% 100%
Testing set 93% 93% 93%

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 71
PATTERN SPECTRA FOR PATTERN RECOGNITION

7.1.4 Regression results for steel images

Depending on the type of pattern spectra, the following figures show different results for the

characterization of HSLA steel samples. The dotted blue graph in each figure below is the

output of the RBF network which corresponds to the target vector representing the expected

depth for each pattern spectrum in the training set (Spectrum 1 up to Spectrum 24). The

inferred depths are represented by the red circled graph in each figure below and each

approximated depth is the output of RBF network for each pattern spectrum of the test set

(Spectrum 25 up to Spectrum 48) as an input to the RBF network.

Figure 7.1: Regression results obtained when using morphological scheme.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 72
PATTERN SPECTRA FOR PATTERN RECOGNITION

3.5

Target (Depth in millimeter)

3

2.5

1.5

0.5 Expected depths

Approximated depths
0
0 5 10 15 20 25
Number of HSLA sample

Figure 7.2: Regression results obtained when using linear scheme.

Figure 7.3: Regression results obtained when using slope scheme.

The performance measures for different types of pattern spectra are shown in Table 7.3.

Table 7.3: Performance measures of the regression.

Feature vectors MPS LPS SPS

MSE 0.1563 0.6802 0.1359

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 73
PATTERN SPECTRA FOR PATTERN RECOGNITION

7.2 Discussion

o Different pattern spectra obtained from different methods have a great importance in

these applications. For the neural network, the input data must fulfil certain

requirements in order for the neural network to perform well. The linear and slope

pattern spectra had to be normalized. These pattern spectra were processed in such a

way that their mean is zero and their variance one.

o The morphological pattern spectra gave good results for both applications. Even

though these morphological pattern spectra are suitable for classification and

characterization, their execution time is excessive. In Table 7.1 the execution time of

each method is shown. The proposed algorithm is two orders of magnitude faster than

the linear granulometry and five orders of magnitude faster than the morphological

granulometry, making it an ideal candidate for real-time implementation. The speed of

the proposed algorithm can therefore be justified by the intermediate representation of

an image called the integral image by Viola and Jones (2001) and the fact that the

algorithm has no nested for loops.

o Table 7.2 shows the classification results for different types of pattern spectra. These

results are calculated as the amount of correctly classified samples divided by the

number of the samples. The results of the morphological pattern spectra, the linear

pattern spectra and the slope pattern spectra are all similar and demonstrate that all

types of computed pattern spectra are suitable as data input to the chosen classifier.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 74
PATTERN SPECTRA FOR PATTERN RECOGNITION

The proposed method is not just fast but it is also efficient. This is shown in Table 7.2

where 93% correct classification was obtained.

o In Figure 7.1, Figure 7.2 and Figure 7.3, the outcome of the RBF network are

illustrated. The performance of the RBF network for different feature extraction

techniques is measured in terms of the mean square error. Table 7.3 shows that the

RBF network performance is not desirable for the linear pattern spectra as inputs to the

RBF network. The mean square error of the RBF network for both the morphological

pattern spectra and the slope pattern spectra, are low and are almost the same. In this

application where HSLA samples are characterized accordingly to their microstructural

deviation when they are laser treated, the proposed method once again is efficient. A

slight improvement of 0.0104 when compared to the morphological algorithm and a

larger improvement of 0.5443 when compared to the linear algorithm are observed.

o Although the linear granulometry and the proposed algorithm are faster than the

morphological granulometry (see Table 7.1), they both do not recognize the shape of

granules and therefore lack the notion of shape definition and the characteristic

definition of similar objects of various sizes. Despite these shortcomings, the linear

granulometry and the proposed algorithm are useful for deriving pattern spectra as

images signatures.

o As shown in Table 7.3, contrary to the classification results, the performance of linear

pattern spectra for characterization is poor. This can be explained by the fact that some

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 75
PATTERN SPECTRA FOR PATTERN RECOGNITION

linear patterns are very close to each other though they represent different

microstructural deviation. In other words, it is difficult for linear granulometries to

distinguish between HSLA samples.

o In this dissertation, the feature extraction techniques proved to be crucial since the

overall performance of a recognition system depends on the quality of the feature

extracted. Hence, the slope pattern spectra are suited to both the classification and

characterization cases.

7.3 Summary

The results and discussion presented in this chapter focused on the performance of the feature

extraction techniques presented earlier. The proposed supervised system performed better with

the proposed pattern spectrum algorithm. Results in Table 7.2 and Table 7.3, and Figures 7.1,

7.2 and 7.3 demonstrated the robustness of the proposed pattern spectra algorithm. Table 7.1

illustrated the improvement in execution time compared to the other methods.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 76
PATTERN SPECTRA FOR PATTERN RECOGNITION

CHAPTER 8

A conclusion is the place where we got tired of thinking

(Arthur Bloch)

CONCLUSION AND FUTURE WORK

8.1 Conclusions

Throughout this dissertation, three feature extraction techniques were presented. Feature

extraction in general is performed in order to increase the classification or regression speed,

accuracy and reduce the costs of analysis.

Firstly, mathematical morphological, which is a powerful tool for texture analysis was

discussed. This tool facilitates a good understanding of how shapes and sizes can be

characterized in an image. Some very important operations such as dilation, erosion, opening

and closing were also presented.

Secondly, two granulometric techniques for feature extraction techniques were discussed.

These techniques, by means of opening operations, extract spatial information or size

distributions from a given image. The first one, known as the traditional granulometry, makes

use of convex structuring elements (sub-images) to characterize images using pattern

spectrum. In this case the pattern spectrum maps the size (shape) of the structuring element at

different scales to the number of objects or image pixels removed during the opening

operation. These pattern spectra can distinguish between certain images.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 77
PATTERN SPECTRA FOR PATTERN RECOGNITION

Thirdly, an algorithm that generates a slope pattern spectrum is proposed in Chapter 5. By

applying an integral image transformation on each line of the original image, increasing slope

segments are observed. The distributions of such slope segments in terms of their sizes

correspond to the slope pattern spectrum. In Appendix A and Appendix B, the reader can find

examples of images distinguished by their slope pattern spectrum.

Lastly, a supervised system was proposed in order to evaluate the performance of the

discussed feature extraction techniques. Two applications were implemented using the

proposed supervised system. Furthermore, two types of materials were used for these

experiments. Some performance measures such as accuracy, speed and Mean Square Error

(MSE) were compared quantitatively. The results obtained from the supervised system were

very poor in terms of accuracy and MSE for the linear pattern spectra as inputs to the neural

networks, but the speed was acceptable. When using the traditional morphological pattern

spectra, the total time of execution of the supervised system was estimated to be three to four

days of MATLAB simulations. This result has limited the practical use of the traditional

granulometries (Vincent 2000:119,127). The performance measures when using the proposed

algorithm was significantly better than the others, showcasing the potential of the slope

pattern spectra algorithm.

8.2 Future work

Although the use of decreasing slope distributions was not addressed in this dissertation,

future developments will explore their ability to extract information from a given image in the

form of a pattern spectrum. The proposed algorithm will be more valuable if it could be

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 78
PATTERN SPECTRA FOR PATTERN RECOGNITION

defined as an Euclidean opening so that it satisfies the properties of granulometries. A larger

database could aid to more accurately quantify the robustness of the proposed supervised

system. This proposed technique can in future also be extended to colour images.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 79
PATTERN SPECTRA FOR PATTERN RECOGNITION

BIBLIOGRAPHY

ASANO, A., MIYAGAWA, M. & FUJIO, M. 2000. Texture Modelling by optimal gray scale

structuring elements using morphological pattern spectrum. In: Proceedings. 15th international

Conference on Pattern Recognition (ICPR 2000), (3):479-482.

AUBERT, A., JEULIN., D, HASHIMOTO, R. F. 2000. Surface Texture Classification from

Morphological Transformations. In: International Symposium on Mathematical Morphology,

2000, Palo Alto. Mathematical Morphology and its Applications to Image and Signal

Processing. Kluwer Academic Publlishers, 253-262.

AWCOCK, G. J., THOMAS, R.1995. Applied Image Processing. London: Macmillan Press

LTD.

BANGHAM, J. A. & MARSHALL, S. 1998. Image and signal processing with mathematical

morphology. Electronics & communication engineering journal, 10(3):117-128.

BISHOP, M. C. 1995. Neural Networks for pattern Recognition. New York: Oxford.

BLEAU, A., DE GUISE, J. & ROBERT LEBLANC, A. 1992. A new set of fast algorithms for

mathematical morphology. : 1. Idempotent geodesic transforms. VGIP: Image Understanding,

56(2):178-209.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 80
PATTERN SPECTRA FOR PATTERN RECOGNITION

BORS, A. G. 2001. Introduction of the radial basis function (RBF) networks. Online

Symposium for Electronics Engineers, (l):1:1-7. Available from: http://www.osee.net/.

[Accessed: 05/10/2006].

BORS, A. G., PITAS I. 1994. Minimal topology for a radial function neural network for

pattern classification. Digital Signal Processing: a review journal, (35):335-356.

BRODATZ, P. 1966. Textures. Dover Publications, New York.

CHEN, Y. & DOUGHERTY E. R. 1992. Texture classification by grayscale morphological

granulometries. In: SPIE, Visual Communications and Image Processing, (1818), Nov.:931-

942, Boston MA.

COLOME-SERRA, M.F., SERON, D., MORESO, F., CARRERA, M., GRINO, G.M. &

SERRA, J. 1992. Image analysis: Utility of grey level granulometry to measure renal

interstitial chronic damage. In: IEEE Proc. Of the annual International Conference, 5,

Nov.:1934-1935.

COSTER, M. & CHERMANT, J. L. 1985. Précis D’Analse D’Images Éditions du Centre

National de la Recherche Scientifique, Paris.

COSTER, M. & CHERMANT, J-L. 2001. Image analysis and morphology for civil

engineering materials. In: Cement and Concrete Composites, 23(2), Apr.:133-151.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 81
PATTERN SPECTRA FOR PATTERN RECOGNITION

CROW, F. 1984. Summed-area tables for texture mapping. In: Proceedings of SIGGRAPH,

18(3):207–212.

DI RUBERTO, C., DEMPSTER A., KHAN, S. & JARRA, B. 2002. Analysis of infected

blood cell images using morphological operators. Image and Vision Computing, (20)2,

Feb.:133-146.

DOUGHERTY, E. R. & ASTOLA, J.1994. An Introduction to Nonlinear Image processing.

SPIE, Bellingham, WA.

DOUGHERTY, E. R. & ASTOLA, J. 1999. Nonlinear Filters for Image Processing. SPIE,

Bellingham, WA.

DOUGHERTY, E. R., NEWELL, J. T. & PELZ, J. B. 1992. Morphological Texture based

Maximum-Likelihood Pixel Classification Based on Local Granulometric Moments. Pattern

Recognition, 25(10):1181-1198.

DOUGHERTY, E. R., PELZ, J. B., SAND, F. & LENT, A. 1992. Morphological image

segmentation by local granulometric size distributions. Journal of Electronic Imaging, 1(1):

Jan.:46-60.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 82
PATTERN SPECTRA FOR PATTERN RECOGNITION

DOUGHERTY, E. R. 1992. A Morphological Image Processing. SPIE Optical Engineering

Press- Tutorial Text in Optical Engineering, TT9, Belingham.

DOUGHERTY, E. R. 1992. Euclidean grayscale granulometries: Representation and

inducement. Journal of Mathematical Imaging and Vision, 17(1):7-21.

DUDA, R.O., HART, P.E. 1973. Pattern Classification and Scene Analysis. New York: John

Wiley & Sons,

FROM, A. & SANDSTTROM, R. 1999. Influence of mixed grain size distributions on the

toughness in high and extra high strength steels. In: Materials Characterization, (42)3,

Feb.:111-122.

HARALICK, R. 1979. Statistical and Structural Approaches to Texture. Proc. IEEE,

(67)5:786-804.

HARALICK, R. M., CHEN, S. & KANUNGO, T. 1991. Recursive opening transform. IEEE

International Vision and Pattern recognition Conference, Champagn Il, Jun.: 560-565.

HUET, F. & MATTIOLI, J. 1996. A textural analysis by mathematical morphology. In: P.

Maragos, R. Schafer, and M. Butt, editors, ISMM’ 96, International Symposium on

mathematical morphology, 273-280, Atlanta, GA, May 1996. Kluwer Academic Publishers.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 83
PATTERN SPECTRA FOR PATTERN RECOGNITION

JONES, R. & SOILLE, P. 1996. Periodical lines and their application to granulometries. In:

ISMM’96, International Symposium on Mathematical Morphology. Atlanta GA: Kluwer

Academic Publishers.

KRAUS, J.; HEIJMANS, H. M. & DOUGHERTY, E. R. 1993. Gray-scale granulometries

compatible with spatial scaling. Signal Processing, 34:1-17.

LIENHARD, R. & MAYDT, J. 2002. An Extended Set of Haar-Like Feature for Rapid Object

Detection. In: Proceedings of the IEEE Conference on Image processing (ICIP 02), Sept.:155-

162, New York, USA.

MATHERON, G. 1967. Elements pour une Theorie des Milieux Poreux. Paris: Masson

MATHERON, G. 1975. Random Sets and Integral Geometry. New York: John Wiley & Sons.

MARAGOS, P. 1989. Pattern Spectrum and Multiscale Shape Representation. IEEE

Transactions on Pattern Analysis and Machine Intelligence, 11(7): 701–716.

MARAGOS, P. & SCHARFER, R. W. 1987. Morphological Filters — Part I: Their set-

theoretic analysis and relations to linear shift-invariant filters. IEEE Transactions on

Acoustics, Speech and Signal Processing, ASSP-35:1153–1169.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 84
PATTERN SPECTRA FOR PATTERN RECOGNITION

MATERKA, A. & STRZELECKI, I. 1998. Texture Analysis Methods- Review, Technical

University of lotz, Institute of Electronics, COST B11 report, Brussels [Online]. Available

from: www.eletel.p.lodz.pl/cost/pdf_1.pdf [Accessed: 02/04/2006].

MITRI, S., PERVOLZ, K., SURMANN, H., and NUCHTER, A. 2005. Robust object

detection at regions of interest with an application in ball recognition. In: ProceedingsIEEE

2005 International Conference Robotics and Automation (ICRA '05) [Online], April:126-131.

Available from: http://kos.informatik.uni-osnabrueck.de/download/icra2005/newpaper.html

[Accessed: 31/07/2006].

MOODY, J. 1989. Fast learning in networks of locally-tuned processing units. Neural

Computation, (1): 281-294.

NAKAGAWA, Y. & ROSENFELD, A. 1978. A note on the use of local min and max

operations in digital picture processing. IEEE Trans. Systems. Man. And cybernetics, SMC-

8:632-635.

NACKEN, P. F. M. 1994. Chamfer metrics, the medial axis and mathematical morphology.

Journal Mathematical Imaging and Vision, 4:233-253.

ROSENFELD, A. & KAK, A. 1982. Digital Picture Processing. Vol. 1, Academic Press.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 85
PATTERN SPECTRA FOR PATTERN RECOGNITION

SERRA, J. 1988. Image Analysis and Mathematical Morphology’, vol. 2:Theoretical

Advances. London: Academic Press.

SERRA, J. 1982. Image Analysis and Mathematical Morphology. London: Academic Press.

SOILLE, P. 1999. Morphological Image Analysis. Berlin: Springer-Verlag.

SOILLE, P. 2005. Morphological Image Analysis: Principles and Applications. Springer. New

York: Springer-Verlag.

SOZANSKA, M.; SOJKA, L; BETAKOVA, C.; DAGBERT, C.; HYSPECKA, L.;

GALLANT, J. & TVRDY, M. 2001. Examination of hydrogen interaction in carbon steel by

means of quantitative microstructure and fracture description. In: Materials Characterization,

3(46):239-243.

STRAFFELINI, G, & MOLINARI, A. 1997. Image analysis Study of Macrosegregation in a

High carbon continuously Cast Steel. In: Materials Characterization, 38(4), Apr.:203-210.

TAIBOT, H. 1994. Le pre-traitement en morphologie mathematique. La revue de metallurgie-

CIT/Science et genie des materiaux, Feb: 211-225.

TSCHESCHEL, A., STOYAN, D., & HILFER, R. 2000. Erosion-Dilation Analysis for

Experimental and Synthetic Microstructures of Sedimentary Rock. Physica A, 284:46-8.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 86
PATTERN SPECTRA FOR PATTERN RECOGNITION

URBACH, E. R. & WILKINSON M. H. F. 2002. Shape-only granulometries and grey-scale

shape filters. Proceedings of In Proc. Int. Symp. Math. Morphology (ISMM) 2002.

VANRELL, R. & VITRIA, J. M. 1993. Mathematical morphology, granulometries, and

texture perception. Image Algebra and Morphological Image Processing IV, (2030). San

Diego CA.

VINCENT, L. 1996. Local Grayscale Granulometries Based on Opening Trees. Proc.

SMM'96, International Symposium on Mathematical Morphology Atlanta GA, May: 273-280.

VINCENT, L. & SOILLE, P. 1991. Watersheds in digital spaces: An efficient algorithm based

on immersion simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence,

13(6):583–598.

VINCENT, L. 2000. Fast Granulometric methods for the Extraction of Global Image

Information. In: Proceedings of PRASA. Broederstroom:119-140.

VINCENT, L. 1997. Current topics in applied morphological image analysis. In: Current

Trends in Stochastic Geometry and its Applications. London: Chapman & Hall.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 87
PATTERN SPECTRA FOR PATTERN RECOGNITION

VINCENT, L. & DOUGHERTY, E. R. 1994. Morphological segmentation for textures and

particles. In: Dougherty E. R. (ed). Digital image processing methods, 43-102., New York:

marcel Dekker.

VIOLA, P. & JONES, M. J. 2001. Robust real-time face detection. Internal Journal of

Computer Vision, 57(2):137-154.

YAMAMOTO, S., KOTANI, K. 1998. Optimization of structuring element on mathematical

morphology by GA and its application to Texture Analysis. IEICE Technical Report of

Computer Vision and Image Media, 109:57-64.

YAN, H.1993. Prototype optimisation of a Nearest Neighbour Classifier using a multi-layer

Neural Network. Pattern Recognition, 26:317-324.

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 88
PATTERN SPECTRA FOR PATTERN RECOGNITION

APPENDIX A

MIXTURE SAMPLE IMAGES AND THEIR PATTERN SPECTRA

Seed samples MPS LPS SPS

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 89
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 90
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 91
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 92
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 93
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 94
PATTERN SPECTRA FOR PATTERN RECOGNITION

APPENDIX B

HSLA SAMPLE IMAGES AND THEIR PATTERN SPECTRA

Seed samples MPS LPS SPS

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 95
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 96
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 97
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 98
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY 99
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY
100
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY
101
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY
102
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY
103
PATTERN SPECTRA FOR PATTERN RECOGNITION

____________________________________________________________________________
F’SATIE - TSHWANE UNIVERSITY OF TECHNOLOGY
104