See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/220571625

Automated defect inspection and classification of leather fabric

Article in Intelligent Data Analysis · November 2001

DOI: 10.3233/IDA-2001-5406 · Source: DBLP

CITATIONS READS

47 6,066

3 authors, including:

Jose A. Ventura
Pennsylvania State University
156 PUBLICATIONS 2,957 CITATIONS

SEE PROFILE

All content following this page was uploaded by Jose A. Ventura on 05 May 2014.

The user has requested enhancement of the downloaded file.

Intelligent Data Analysis 5 (2001) 355–370 355
IOS Press

Automated defect inspection and

classification of leather fabric
Choonjong Kwaka , José A. Venturab,∗ and Karim Tofang-Sazic
a Schoolof Industrial Engineering, Purdue University, West Lafayette, IN 47907-1287, USA
b Department of Industrial and Manufacturing Engineering, Pennsylvania State University, University
Park, PA 16802, USA
c Westwood Industries, 597 Glasgow Lane, Tupeco, MS 38803, USA

Received 18 October 2000

Revised 24 December 2000
Accepted 24 January 2001

Abstract. This paper describes an automated vision system for detecting and classifying surface defects on leather fabric.
In the defect inspection process, visual defects are located and reported through a two-step segmentation procedure based on
thresholding and morphological processing. In the defect classification process, the system utilizes both geometric and statistical
features as its feature sets; that is, a new normalized compactness measure, and first- and second-order statistical features. In an
effort to maximize the classification efficiency, a three-stage sequential decision-tree classifier is adopted for the classification
of five types of defects: lines, holes, stains, wears, and knots. If line defects are identified as a result of classification, they are
checked by a line combination algorithm to determine if they are parts of larger line defects and, in such a case, are reported as
combined line defects. Satisfactory results were achieved in the classification test with an overall accuracy of 91.25%.

Keywords: Machine vision, leather defects, leather inspection, defect inspection, defect classification

1. Introduction

Visual inspection and classification of leather surface defects are very important in the manufacturing
of leather products that require unusually high quality. These operations are currently performed by
human inspectors who tend to miss considerable numbers of defects because human beings are basically
inconsistent and inappropriate for such simple and repetitive tasks. Furthermore, since manual inspection
and classification are slow and labor-intensive tasks, they can become a critical bottleneck in the entire
production process. Automated inspection and classification can reduce human workloads and labor
costs while increasing throughput. More importantly, higher accuracy can be achieved by eliminating
human error due to fatigue.
In fact, much research has been carried out on automated inspection of metal surfaces, wood, and
textile fabrics, while relatively little work has been done in automated defect classification, mainly
because of the difficult nature of the problem. It is practically impossible to construct exact models of
defects for classification because their appearance and size greatly vary. It is almost impossible to find

∗
Corresponding author. Tel.: +1 814 865 3841; Fax: +1 814 863 4745; E-mail: [email protected].

1088-467X/01/$8.00  2001 – IOS Press. All rights reserved

356 C. Kwak et al. / Automated defect inspection and classification of leather fabric

two defects with the same shape and size, even if they belong to the same defect class. Nevertheless,
this classification process is necessary because it plays an important role in providing the information
for defect prevention. Defects should be classified into appropriate classes according to their cause and
origin in order to locate the source responsible for those defects and take corrective actions.
Zhang and Bresee [11] detected defective images in textile fabrics by individually applying two
different approaches. A somewhat simple classification was performed to discriminate knots from slubs
according to the ratio of length to width. If the length is more than twice the width, it is classified as a
slub. Otherwise, it is considered as a knot.
Brzakovic et al. [1] employed a pyramid-linking scheme to locate defects in wood and a hierarchical
defect classification scheme to classify four types of wood defects: cracks, mineral streaks, wormholes
and knots. At the first level of classification, the measure of compactness is used to first determine
whether a defect belongs to a linear class (cracks and mineral streaks) or a circular class (wormholes and
knots) and then, more specifically, assign it to one of four defect classes if the value is so obvious that
no further classification is necessary. Otherwise, the classifier uses, at the second level of classification,
the defect’s area or width according to whether the defect is linear or circular, respectively. If the above
measures are not enough for classification, the intensity variations within the defect are used at the third
level. But, this approach has two problems in terms of classification because it relies too much on shape
and geometric characteristics of defects. First, geometric and shape characteristics are very sensitive to
the result of segmentation, which adversely affects the classification performance, although some degree
of segmentation error is inevitable in real problems. Secondly, it was observed in our research that the
measure of compactness causes a critical problem for the classification of linear defects.
This paper describes an automated defect inspection and classification system for leather fabric.
That uses a two-step segmentation procedure for inspection based on thresholding and morphological
processing. This research proposes using both geometric and statistical features as its feature sets for the
classification of leather defects. Usually, domain-specific knowledge is used in the classification problem
to extract good features from raw data. Since the classification problem for these kinds of leather defects
is unique to the authors’ knowledge, the selection of appropriate features becomes very important in this
research. Furthermore, good feature selection makes classification much easier.
In this study, a three-stage classification scheme is used to maximize the classification efficiency.
In the first stage, line defects such as scratches, wrinkles, cuts and tears are classified by applying a
new normalized compactness measure which has certain advantages over the conventional compactness
measure. With the use of this normalized compactness measure, the two problems described by Brzakovic
et al. [1] can be avoided. The line defects identified in this stage go through a line combination test
to check if they are originally parts of larger defects because line defects often appear fragmented after
segmentation. If this is the case, they are reported together as larger line defects after the line combination
test. The first- and second-order statistical features are extracted from the regions with defects in the
second and third stages, respectively, and used for the corresponding classification stage only. Hole
defects can easily be discriminated in the second stage by having leather fabric pass over a white light
table in the system so that the gray-levels in the hole defect are those of the white baseboard of the table.
First-order statistical features are sequentially used to classify stain defects as well as hole defects in the
second stage. Knot and wear defects are classified by using the second-order statistical features in the
third stage.
This paper is organized as follows. The defect inspection process, which consists of image acquisition,
segmentation and reporting, is described in the next section. In Section 3, the list of defects covered in
this study is first introduced. This section also describes the feature sets and the classification scheme
along with a line combination algorithm. Finally, experimental results and conclusions are provided in
Sections 4 and 5, respectively.
 C. Kwak et al. / Automated defect inspection and classification of leather fabric 357

Table 1
The classification of leather types. Elegence, fargo,
regency and poloma classes require higher quality than
other classes
Finished Unfinished
Distressed look Non-distressed look Natural look
Captiva Elegence
Poloma Fargo
Cimarron Regency

2. Defect inspection

Song et al. [9] divided defect detection techniques into two categories according to whether they
involve a training phase. A defect detection system without a training phase locates defects by detecting
the pixels deviated from their background. However, this kind of method has a disadvantage in that it
may report defects although there is actually no defect. On the other hand, a defect detection system with
a training phase is trained on normal and abnormal patterns or, sometimes, on the normal pattern only to
learn the difference between acceptable and defective textures. Sometimes, when expert knowledge is
available, this training stage can be replaced.
In this study, we have observed that the results of image acquisition and segmentation are sensitive to
the leather type under inspection, mainly due to its reflectance properties. In fact, unlike other materials,
such as a metallic surface, the reflectance properties of leather greatly vary according to the leather type.
For this reason, it is very inefficient and even impossible in terms of segmentation performance to apply
the same segmentation method to all leather types. In practice, it is advisable to search for the most
appropriate specific segmentation method each time the production line adopts a new type of leather,
and add it into the database of expert knowledge, even though the overall segmentation steps for the new
leather type follow the basic routine of this study.
The current classification of leather types being used by Westwood Industries (Tupeco, MS) is presented
in Table 1. There are also several sub-styles in each class. Threshold values and various parameters
of the defect detection process are a function of the leather type being inspected and experimentally
determined. Note that another important consideration is quality standards of each production line
because different quality standards are usually set by different companies depending upon the quality
levels of their products in the market.
Considering the characteristics of leather products with various leather types and quality standards,
three leather types supplied by Westwood Industries were selected for testing in this research: the
regency, poloma and captiva classes. The regency and poloma classes, in particular, require very high
quality standards, while captiva is a common class of leather. The leather texture patterns used in this
research were restricted to plain patterns.
The defect inspection process includes image acquisition, segmentation, and reporting. A gray-level
image is first captured. The acquired gray-level image is processed for segmentation and a copy of it
is directly saved for the next defect classification process. After the gray level image is thresholded,
the resulting binary image is processed by a combination of binary morphological erosion and dilation
operations along with median filters to remove noise and fill the holes in detected defects. Binary
connected component analysis is then applied to the processed binary image. Finally, the information
obtained, such as the number of defects, their locations and sizes, is reported.
358 C. Kwak et al. / Automated defect inspection and classification of leather fabric

2.1. Image acquisition

The vision system utilized in this research is composed of a Pulnix TMC-74 high resolution CCD color
camera (768 × 493 image pixels), Sun VideoPix with a frame of data captured in real-time at 1/30 second
NTSC as a frame grabber, and Bencher VP-400 Copystand with a 64 cm × 64 cm table. The copystand
has four 300 watt upper lamps fixed at a height of 40 cm at the corners of the table and a 600 watt quartz
halogen illuminator built in a 40 cm × 40 cm white baseboard for back lighting. The four upper lamps
were adjusted to look at the entire light table in order to achieve uniform illumination across the field of
view. The base illuminator was not used because it did not make a significant difference. The image
of the leather fabric being inspected was partitioned into approximately 8 cm × 12 cm windows. All
the algorithms written in C were implemented on a SUN SPARC station 2 under SunOS OpenWindows
Version 3.
Illumination and the distance between camera and leather are two important factors in the experiment.
Clearly, significant variations in illumination lead to false results. The distance between camera and
leather was adequately adjusted by trial and error. It has been observed that too far a distance fails to
extract satisfactory statistical features of defects for good classification results. On the other hand, too
close a distance increases the processing burden due to more amounts of data to be processed over the
same area of leather surface. In this study, a distance of 40 cm was maintained from the camera lens to
the light table during the entire experiment.

2.2. Segmentation

Although the basic approach of this stage assumes that leather defects can be extracted from the
background (normal leather fabric) by their gray level values, it has been observed in the experiment
that the gray level distributions of defects and noise often overlapped, which complicates the separation
of defects from noise by just using conventional histogram-based thresholding approaches, such as fixed
thresholding or adaptive thresholding. The only two apparent differences between noise and defects are
their density and size.
A characteristic of leather is that it is generally used for high-quality products. While this fact puts an
emphasis on quality control of leather, from a segmentation point of view, it plays the role of facilitating
the segmentation of leather surface defects under a well-controlled inspection environment. That is
because leather should maintain uniform texture over the entire surface and a flaw could be critical in
high-quality leather products.
Since individual histogram-based thresholding techniques are not satisfactory for leather defect in-
spection, a two-step procedure is proposed for this stage motivated by the above observations. First,
a fixed thresholding scheme is used to separate defects from normal leather texture. Defects appear
abnormally brighter or darker than their background so that two threshold values are required. The
thresholded image, FT (i, j), is defined as


0 if F (i, j)
T1 or F (i, j)  T2
Ft (i, j) = (1)
255 otherwise

where F (i, j) is the original 256 gray level image of size 200 × 300 (pixels). Second, morphological
operations are adopted to utilize two apparent differences between noise and defects. Along with a
median filter, the opening operation is used to remove noise. Median filters are often combined with
the opening operation to improve the effect of noise removal. The segmentation processes for several
 C. Kwak et al. / Automated defect inspection and classification of leather fabric 359

Table 2
Segmentation processes for several types of leather. The number in the noise removal and hole filling processes represents the
number of operations to be performed
Class Style Color Tested area Thresholds Noise removal Hole filling
(cm2 ) (T1 , T2 ) Median filter Erosion Dilation Dilation Hole-filling Erosion
Regency 1 Light gray 4500 (157,188) 2 2 2 3 1 3
2 Gray 3000 (178,189) 2 2 2
Poloma 1 Purple 3000 (132,158) 1 1 1
Captiva 1 Light gray 3000 (120,152) 2 2 2
2 Yellow 7200 (130,180) 2 1 1

different leather types are summarized in Table 2. The sixth column in Table 2 shows a combination of
median filters, erosions and dilations enough to remove the noise that can occur after thresholding even
under normal conditions without defects for each leather type. These noise removal steps are closely
related to the choice of thresholding values for each leather type. Since the shapes of noise and defects
are unknown at this point, a 3 × 3 1 matrix was used as a structuring element in all cases.
Even after noise is removed, undesirable white spots in the defects (particularly, stains and knots)
often remain. Note that hole filling by closing is defect-dependent, unlike noise removal. The best
way to proceed in this case is to perform the closing operations with the same structuring element as
many times as necessary until all holes in all defects are filled. But, too much use of morphological
operations distorts the shape of the defects. As an alternative, in this study, three dilation operations are
first performed, then a hole-filling operation follows to fill all possible remaining holes, and finally three
erosion operations are done as shown in the last column of Table 2. The second hole filling operation
is performed by first finding 4-connected white background pixels in the image and then changing all
the remaining white pixels (hole pixels) to black. These steps were established through training defect
samples in the training phase and satisfactory segmentation results were obtained while distorting the
defects as little as possible. Figure 1 shows an example of the segmentation process with a natural stain
on the style 2 of the captiva class (see Fig. 2(d)).

2.3. Reporting

Real-time reporting is very important in the defect inspection process, while an emphasis is put on
accuracy rather than processing time in the defect classification process. For this reason, the result of
defect inspection is reported before classification.
Binary connected component analysis is applied to the segmented binary image. Then a size filter
is used to remove all components with less than A T = 25 pixels in size. The number of defects, their
locations and sizes as obtained by the analysis are immediately reported and saved as geometric features
for the following defect classification process. Also, the segmented binary image is used to obtain
gray-level masks of the defects when statistical features in the classification process are computed.

3. Defect classification

3.1. Defects on leather

In spite of an increasing need for automated defect classification systems, relatively little work has
been done in this area. This is attributed primarily to the irregular characteristics of defects themselves.
360 C. Kwak et al. / Automated defect inspection and classification of leather fabric

Fig. 1. Noise filtering process. (a) After thresholding. (b) After 2 median filters. (c) After 1 opening operation. (d) After
applying 3 dilations, hole filling and then 3 erosions.

Unpredictable variations in the appearance and size of defects make it impossible to construct exact
models of the defects. Much effort has been placed in the classification of defects using geometric
features, although this approach has definite limitations due to the above characteristics of defects.
A geometric feature, named normalized compactness measure, is used in this research. However,
statistical texture analysis is also adopted to overcome the limitations of the geometric approach, by
applying Rao’s texture definition and grouping [8] to the leather defects considered in this research.
Rao defines texture as ‘the surface markings or 2D appearance of a surface’, which includes wood,
semiconductor wafers and textiles. Texture is characterized by tonal primitive properties and spatial
relationships between them [5]. Rao has grouped texture into three major classes: strongly ordered
textures, disordered textures, and weakly ordered textures. While strongly ordered textures can be
characterized by their repetitive pattern, disordered textures show neither repetitiveness nor orientation
and may be described on the basis of their roughness. Weakly ordered textures are those that exhibit
some degree of orientation specificity at each point of the texture.
Typical examples of the five defects covered in this study are presented in Fig. 2. Line defects such as
wrinkles, scratches, and tears on leather fabric are often caused by damage on cow skin or occur during
the production and material handling stages. After segmentation, these line defects appear visually as
long and narrow lines. Stains covered in this research are a kind of natural variation in leather texture.
A natural stain has apparently many white spots on its region, and the boundary of a stain is not clear
but becomes gradually blurred towards the normal leather texture. Wears caused by friction and attrition
can be considered as one of disordered textures in Rao’s grouping. Knots, on the other hand, are an
example of weakly ordered textures. The extent of a knot includes a somewhat broad area including
 C. Kwak et al. / Automated defect inspection and classification of leather fabric 361

Fig. 2. Defects on leather. (a) Line. (b) Hole. (c) Knot. (d) Stain. (e) Wear.

its boundary part as well as its core part that has a visually undesirable grain. While knots are purely
biological defects, wears arise from both natural causes and manufacturing errors as line defects.

3.2. Normalized compactness measure

The conventional measure of compactness, C = P 2 /A, where P and A are an object’s perimeter and
area, respectively, has found many applications as a geometric feature. Brzakovic et al. [1] have used this
compactness measure for the classification of line defects in wood, because it usually represents well the
degree of the eccentricity or circularity of an object. However, this compactness measure has revealed
a critical problem in distinguishing line defects from other defects in this research. Figure 3 shows a
typical example of the problem with two binary images of real defects, of which one is the image of a
stain defect and the other is that of three line defects. In this example, it is impossible to differentiate
the line defects from the stain defect with the traditional compactness measure because this stain defect
shows a rather higher eccentricity than the line defects by its larger value in the measure of compactness,
unlike the common expectation.
To overcome this problem, a new normalized compactness measure is proposed in this study. The
normalized compactness measure is defined as N C = C/A. The rationale behind this measure is
that comparisons can be made under the same conditions by normalizing their compactness measures
with respect to the area. For the example in Fig. 3, the normalized compactness measure accurately
classifies line defects with N C threshold . In fact, after applying this normalized compactness measure, it
was observed through the entire experiment that line defects could be clearly discriminated from other
defects.
Furthermore, the normalized compactness measure has one more advantage over the conventional
compactness measure. Figure 4 shows two different binary images of the same defect due to the results
of good and poor segmentation. Since (a) is an exact binary mask of the wear defect used, it is known that
(b) produces a much higher value than a real value in the measure of compactness. However, by using
362 C. Kwak et al. / Automated defect inspection and classification of leather fabric

Fig. 3. Comparisons of the normalized compactness measure and the compactness measure. (a) C = 92.11, N C = 0.0067.
(b) (From the left) C = 45.75, 13.79, and 20.48, N C = 0.4279, 0.4756, and 0.6207.

Fig. 4. An example of another advantage of the normalized compactness measure. (a) The result of good segmentation.
C = 36.1308, N C = 0.0164. (b) The result of poor segmentation. C = 79.3163, N C = 0.0459.

the normalized compactness measure, possible misclassification into a line defect can be prevented. In
other words, the normalized compactness measure is much less sensitive to the result of segmentation.

3.3. First-order statistical measures

Another useful class of features that can be employed for classification purposes comprises the first-
order statistical measures, such as mean and variance. The first-order statistical measures employed are
defined as follows:
– Mean:
1 
µ= p(i, j), (2)
N
i j

– Variance:
1 
σ2 = [p(i, j) − µ]2 , (3)
N
i j

where p(i, j) is the gray-level of the (i, j)th element of a n × m image and N is the total number of
pixels in an image, i.e., N = n × m.
 C. Kwak et al. / Automated defect inspection and classification of leather fabric 363

Fig. 5. (a) A 4 × 4 image with four gray levels 0, 1, 2, and 3. (b) The corresponding 4 × 4 co-occurrence matrix for d = (1, 1).

3.4. Second-order statistical measures

The final class of features to be used include statistical measures based on the gray-level co-occurrence
matrix [4], under the assumption that texture analysis can explain leather defects. It is known that
these statistical measures capture well the spatial dependence of gray level values that contributes to the
perception of texture.
Let d = (∆x, ∆y) be a vector in the (x, y) plane. The gray-level co-occurrence matrix M d is defined
by Md (i, j) which is the number of pairs of gray-levels (i, j) occurring at separation d in a discrete
picture f (x, y). If an image has m gray levels, the size of its co-occurrence matrix will be m × m.
As a simple example, if the picture is given as Fig. 5(a) and the displacement vector d is chosen as
d = (∆x, ∆y) = (1, 1), then its co-occurrence matrix would be Fig. 5(b). Since the given image has
only four gray levels, its co-occurrence matrix is a 4 × 4 matrix.
The elements of Md are then normalized by dividing each element by the total number of pixel pairs.
In Fig. 5(b), each element is divided by 9. This normalized M d can be considered as a probability mass
function because the elements add up to 1. In other words, the normalized M d (i, j) is the estimated
probability of going from gray-level i to gray-level j given the displacement vector d = (∆x, ∆y).
Haralick et al. [4] defined a set of measures based on the co-occurrence matrix, and Conners et al. [5]
found that five of the measures are truly useful for application. In this research, pilot experiments have
shown that the following four of Conners’ five measures are effective in this specific problem domain:
– Energy:

E= n(i, j)2 , (4)
i i
– Entropy:

EN T = − n(i, j) log n(i, j), (5)
i j

– Inertia:

I= (i − j)2 n(i, j), (6)
i j
364 C. Kwak et al. / Automated defect inspection and classification of leather fabric

– Homogeneity:
 1
H= n(i, j), (7)
1 + (i − j)2
i j

where n(i, j) is the (i, j)th element of the given m × m matrix divided by the sum of all the matrix
elements.
The displacement vector d is certainly a very important factor in the definition of the gray-level
co-occurrence matrix. The most appropriate displacement vector d has been determined through pilot
experiments so that the discriminatory power is maximized. In this study, the displacement vectors
d = (1, 1), (3, 3), (5, 5), (7, 7), and (9, 9) have been used in the training phase.

3.5. Classification scheme

This system adopts a sequential decision-tree classification scheme in order to maximize the classifica-
tion efficiency. The overall scheme is designed according to the two general principles of decision trees:
(i) the computational complexity increases from the root of the tree towards the leaves, and (ii) simpler
trees are preferred because they are more likely to capture the structure inherent in the problem [7]. A
brief diagram of the sequential decision-tree classifier is shown in Fig. 6.
Once a defect is given to the system, the classification process starts at the root of the tree and a
decision is made at each node. The process continues until a leaf is encountered, at which time the defect
is assigned to the corresponding defect class and the process is completed. All the threshold values used
for each decision in the experiment are presented in Table 3.
When a defect is identified, the mask of the defect is obtained as a binary picture in which the object
points are black and the other background points are white. Geometric features, such as area, perimeter,
and location, are then calculated from the mask and saved for the classification process. At the first
stage of classification, it is checked if the given defect is a line defect using the normalized compactness
measure. If its normalized compactness measure indicates with N C < N C threshold that it is not a line
defect, the classification process continues. But, before entering the second stage, the gray level mask
of the defect is first obtained by using the binary mask and the original gray-level image, and then the
first-order statistical measures are extracted from the gray level mask. The second-order statistics are
calculated only when the third stage is required for computational efficiency.
Since the leather fabric being inspected is supposed to pass over a white light table in a black-box-type
inspection system, a hole defect has the same gray levels in its region as the white background. As a
result, the gray level distribution of a hole defect has a very high mean value and, at the same time, a
very small variance value. Usually, the mean value of a hole defect is not exactly the same as that of the
gray-level distribution of the white background due to shading around the boundary of the hole defect
and/or segmentation error. On the other hand, a stain defect turns out to have very high variance, which
seems to be reasonable considering its visual characteristics.
At the third stage, wears and knots are classified by using the second-order statistical features to capture
the difference of the spatial pattern in both defects. In fact, it has been observed that wears have smaller
variance values than knots but their difference is not sufficiently large for use in classification. This
observation shows well how the selection of inappropriate features makes classification difficult. Knots
have shown higher entropy/ inertia values but lower energy/ homogeneity values than wears, which can
be explained by the higher texture randomness of knots compared to wears.
 C. Kwak et al. / Automated defect inspection and classification of leather fabric 365

Fig. 6. A diagram of the overall classification scheme.

3.6. Line combination test

As a result of the sequential classification process, line defects are identified. Then, they need to be
examined to determine if they originally belonged to larger line defects. Certain kinds of line defects,
such as scratches, wrinkles, cuts, and tears, frequently tend to appear fragmented after segmentation due
to uneven contrast, noise or the resolution of the camera [1].
Suresh et al. [10] have used a three-step process to cope with this kind of problem in the inspection of
hot steel slabs. In the first step, for each component identified on a steel slab, the minimum bounding
rectangle is defined and then augmented by a certain amount. The Proximity Search Algorithm is
applied to find sets of proximate components whose rectangles touch. In the second step, a semantic
rule and a syntactic rule are applied to check spatial relationships and compatibility of component types,
366 C. Kwak et al. / Automated defect inspection and classification of leather fabric

Table 3
Threshold values used
Stage Node Condition (If) Conclusion (Then)
1 1 N C  N Cthreshold = 0.09 Line
2 2 M EAN  M EANmin = 200 Hole
3 V ARIAN CE  V ARIAN CEmin = 191 Stain
3 4 IN ERT IA  IN ERT IAmin = 177 Knot
IN ERT IA
IN ERT IAmax = 160 Wear
5 EN T ROP Y  EN T ROP Ymin = 5.0 Knot
EN T ROP Y
EN T ROP Ymax = 5.0 Wear

respectively. A semantic check is made to discriminate between the ‘end-to-end’ condition and the
‘overlapping and parallel’ condition, and a syntactic look-up table is used for checking the type of
compatibility. Finally, all the corresponding components are grouped together based on the information
and features obtained.
Since the above method is computationally complex, a more efficient line combination algorithm
is proposed in this study by using the orientations of line defects to decide whether they should be
combined. If the orientations of two line defects under examination are similar, they will be considered
to come from a single larger defect. Note that the defects being considered in this test have already been
classified in the previous stage as linear ones so that they have a unique orientation.
The orientation of a line defect can be represented by the orientation of the axis of elongation. Since
the axis of the least second moment is commonly used as the axis of elongation, the objective here is to
find the line by which the sum of the squares of the distances to the points in the defect is minimized.
The sum can be represented as

χ2 = 2
rij B(i, j), (8)
i j

where B(i, j) is a given binary image and r ij is the perpendicular

 distance from a defect point (i, j)
to the line [6]. The area A of the defect is given by A = B(i, j) and the position of the
i j

center of the area (center of mass), denoted as (x, y), is obtained by x = 1
A xij B(i, j) and
 i j
y= 1
A yij B(i, j). The line sought after can be represented in polar coordinates (see Fig. 7) as
i j
follows:
x sin θ − y cos θ + ρ = 0. (9)
By solving the minimization problem,

M inχ2 = 2
rij B(i, j), (10)
i j

ρ and θ can be obtained by the following two equations:

ρ = −x sin θ + y cos θ, (11)

b
tan 2θ = , (12)
a−c
 C. Kwak et al. / Automated defect inspection and classification of leather fabric 367

Fig. 7. Polar representation of a line. θ is an angle between the line and the x-axis and ρ is the distance line from the origine.

where

a= (xij )2 B(i, j), (13)
i j


b=2 xij yij

B(i, j), (14)
i j


 2
c= (yij ) B(i, j), (15)
i j

x = x − x, (16)

y  = y − y. (17)

Parameters ρ and θ are calculated for each line defect considered in this test and used to judge whether
two defects have to be grouped together; that is, if the differences in ρ’s and θ ’s are less than predetermined
values, ρthreshold = 20 (pixels) and θthreshold = 0.1 (radians), respectively, the defects are combined.
The input list in this algorithm includes all line defects within a given image.
The line combination algorithm is described as follows:
Step 1: Calculate and for each line defect by Eqs (11) and (12), respectively, and save them with their
line defects.
Step 2: Let the smallest-indexed line defect be L a . Mark La . If the index of La is the largest in the
input list, move La from the input list to the output list, and terminate. Otherwise, continue.
368 C. Kwak et al. / Automated defect inspection and classification of leather fabric

Table 4
The classification results obtained by the test on 80 defect samples
Input Output
Line Hole Stain Knot Wear Accuracy
Line 10 0 0 0 0 100%
Hole 0 10 0 0 0 100%
Stain 0 0 18 2 0 90%
Knot 0 0 0 18 2 90%
Wear 0 0 0 3 17 85%
Total accuracy 91.25%

Step 3: Let the smallest-indexed line defect among unmarked ones be L b . Mark Lb . If there are no
unmarked line defects in the input list, move L a from the input list to the output list. Clear all
marks and go back to Step 2. Otherwise, continue.
Step 4: If |ρa − ρb | < ρthreshold and |θa − θb | < θthreshold , continue. Otherwise, go back to Step 3.
Step 5: Calculate the new ρ and θ values by Eqs (11) and (12) by considering L a and Lb together.
Re-label Lb with La ’s original index, and save the new ρ and θ values for L a . Remove Lb
from the input list, and go to Step 3.
Since this line combination algorithm only uses two parameters, ρ and θ , it is computationally very
efficient. Obviously, this is an attractive characteristic, because the processing time is a very important
factor for meeting production requirements in real situations.

4. Computational results

The proposed automated defect classification system has been implemented to check its performance
through two phases: training and testing. A total of 140 defect samples obtained from the regency and
captiva classes were classified into five defect classes by human review in advance. The example set
used for training was made up of 60 defect samples: 12 defect samples for each class. After the training
phase, 80 defect samples were run through the system for the classification test: 10 defect samples each
for the line and hole defects, plus 20 defect samples each for the other three defect classes. Various
threshold values of the features used for classification were determined in the training phase and are given
in Table 3. The training phase showed that the displacement vector d for the second-order statistical
measures did not have satisfactory discriminatory power for small values such as d = (∆x, ∆y) = (1, 1)
and (3, 3). In the test phase, the displacement vector d was chosen as (∆x, ∆y) = (9, 9).
Classification results are tabulated in the form of a confusion matrix in Table 4. The classification of
line defects by using the normalized compactness measure turned out to be very effective. Only two or
three stain defects showed relatively high values (0.06–0.07) of the normalized compactness measure,
which is still far from the minimum threshold value (0.09) for line defects. Also, hole defects could
easily be classified because of their high mean and low variance.
A classification accuracy of 87.5% for wears and knots was achieved by using the second-order
statistical features. This can be accepted as a satisfactory result considering the difficult nature of
classification problems. Although the subject of the experiment is different, Dyer et al. [3] asked a
geologist and a naı̈ve subject to classify 180 terrain samples into only three classes and reported about
20 percent errors for both. This is not surprising because human beings are basically vulnerable to these
kinds of simple and repetitive tasks. The error rate of human reviewers is most likely to increase due to
fatigue as the experiment continues.
 C. Kwak et al. / Automated defect inspection and classification of leather fabric 369

It was observed that the second-order statistical features were highly correlated. Entropy and energy
showed negative correlation, and inertia and homogeneity also. Thus, through the entire training phase,
only inertia and entropy were chosen for classification instead of all four features because these two
features represent well the characteristics of the other two.
Also, it was found that not all four features have the same discriminatory power. In the training phase,
inertia and entropy showed better performance than homogeneity and energy in terms of classification
ability. In particular, inertia turned out to have the best discrimination power. In fact, the final analysis
of test results indicated that the classification results might have been similar even if only inertia had
been used in the third stage of classification.

5. Conclusions

This paper describes the development of an automated vision system to identify and classify visual
defects on leather fabric. While much work has been done on visual defect inspection in textile, wood
and steel slabs, little attention has been paid to the defect classification problem despite its importance,
mainly due to the difficulty of the problem itself.
In this study, defect inspection has been performed by a two-step segmentation procedure with thresh-
olding and morphological operations to improve segmentation performance. When individual histogram-
based thresholding approaches are not satisfactory, the addition of morphological operations is beneficial
for segmentation in that they have the effect of clustering as well as noise removal. Clustering effects
are reasonable because, even though a defect is discontinuous, good leather parts surrounded by the
discontinuous defective parts are useless anyway.
The normalized compactness measure, and the first- and second-order statistical features have been
used for the classification of five different defect classes on leather fabric. It has been observed that the
statistical features as well as the normalized compactness measure capture well the essential differences
among the defects. Also, it has been suggested that inertia by itself might represent the characteristics of
the four second-order statistical features for classification purposes. A decision-tree sequential classifier
has been used in an effort to improve the classification efficiency. This system has shown satisfactory
performance. The overall accuracy of the classification on the testing set was 91.25%.
Automated defect inspection and classification are very important because increased throughput and
greater classification accuracy can be achieved. Also, human inspectors can be removed from this kind
of tedious task in the long run.

References

[1] D. Brzakovic, H. Beck and N. Sufi, An approach to defect detection in materials characterized by complex textures,
Pattern Recognition 23(1/2) (1990), 99–107.
[2] R.W. Conners and C.A. Harlow, Toward a structural textural analyzer based on statistical methods, Computer Graphics
and Image Processing 12(3) (1980), 224–256.
[3] C.R. Dyer, J.S. Weszka and A. Rosenfeld, Experiments in terrain classification on LANDSAT imagery by texture analysis.
Technical Report TR-383, Computer Science Center, Univ. Maryland, College Park, MD, 1975.
[4] R.M. Haralick, K. Shanmugam and I. Dinstein, Textural features for image classification, IEEE Transactions on Systems,
Man and Cybernetics SMC-3(6) (1973), 610–621.
[5] R.M. Haralick, Statistical and structural approaches to texture, Proc. of the IEEE Conference 67 (1979), 786–804.
[6] B.K.P. Horn, Robot vision, McGraw-Hill, Inc., New York, 1986.
[7] J.R. Quinlan, Induction of decision trees, Machine Learning 1 (1986), 81–106.
 370 C. Kwak et al. / Automated defect inspection and classification of leather fabric

[8]
A.R. Rao, A taxonomy for texture description and identification, Springer-Verlag, Inc., New York, 1990.
[9]
K.Y. Song, M. Petrou and J. Kittler, Texture defect detection: A review. Applications of Artificial Intelligence X: Machine
Vision and Robotics, SPIE 1708 (1992), 99–106.
[10] B.R. Suresh, R.A. Fundakowski, T.S. Levitt and J.E. Overland, A real-time automated visual inspection system for hot
steel slabs, IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-5(6) (1983), 563–572.
[11] Y.F. Zhang and R.R. Bresee, Fabric defect detection and classification using image analysis, Textile Research Journal
65(1) (1995), 1–9.

View publication stats