Defect detection in textured surfaces using color ring-projection correlation

D. M. Tsai and Y. H. Tsai Machine Vision Lab. Department of Industrial Engineering and Management Yuan-Ze University, Chung-Li, Taiwan, R.O.C. E-mail: [email protected]

1. INTRODUCTION

Image analysis techniques are being increasingly used to automate industrial inspection. For defect inspection in complicated material surfaces, color and texture are two of the most important properties. Detecting an entire class of defects in colored texture images would be impossible with typical gray- level processing techniques. In this paper, we introduce a color ring-projection scheme to tackle the problem of defect detection in colored texture surfaces. The proposed color ring-projection representation reduces computational complexity by transforming 2-D images to 1-D patterns, and is rotation- invariant with respect to oriented structures of textures.

In automatic surface inspection, the task is to detect small surface defects that appear as local anomalies embedded in a homogeneous texture. The class of homogeneous image texture has a repetitive, self-similarity property that distinguishes it from any other class of images. Textures are generally classified into two major types, structural and statistical (Pikaz and Averbuch 1997). Structural textures are those that are composed of repetitions of some basis texture primitives, such as lines, with a deterministic rule of displacement. Statistical textures cannot be described with

primitives and displacement rules. The spatial distribution of color in such textured images is rather stochastic. Generally, structural textures such as textile fabrics and machined surfaces are highly oriented. Textural features of such images are orientation-dependent. Statistical textures such as leather and cast surfaces are isotropic, and are rotation- invariant. In this paper, we aim at the surface defect inspection for both structural and statistical textures without alignment requirement or prior knowledge of texture orientation.

For complicated textured-surfaces in gray- level images, spatial gray- level co-occurrence matrix methods (Sum and Wee 1982) in the spatial domain and Fourier transform methods (Liu and Jernigan 1990) in the frequency domain have been commonly used to describe texture features. A survey of co-occurrence matrix methods can be found in Siew and Hogdson (1988). Co-occurrence matrix approaches have been applied for the inspection of wood (Conners et al. 1983; Ojala et al. 1992), carpet wear (Siew and Hogdson 1988) and machined-surface roughness (Ramana and Ramamoorthy 1996).

Fourier-based methods characterize the spatial- frequency distribution of textured images, but they do not consider the information in the spatial domain and may overlook local deviations. In the recent past, Gabor filters (Daugman 1985) are well recognized as a joint spatial/spatial- frequency representation for analyzing textured images containing highly specific frequency and orientation characteristics. Gabor filter-based methods have been successfully applied for texture classification and segmentation (Teuner et al. 1995; Randen and Husoy 1999; Weldon and Higgins 1999) object detection (Jain et al. 1997), and inspection (Escofet et al. 1996). For unknown

orientation of structural textures, matched filtering approaches using Gabor filters require significant amount of computation, and are of limited use because each view of the oriented structures may require a unique filter. In terms of colored texture representation, classical Gabor filters do not exploit chromatic properties of textures and may fail in detecting color flaws.

Traditional texture analysis methods are inappropriate for colored texture images because they ignore chromatic information. In the analysis of color images, the description of image region has been performed mainly based on color histograms (Swain and Ballard 1991; Boukouvalas et al. 1999). However, color histograms lose the spatial information of a texture, and are not sufficient to detect local variation of small defects. More sophisticated color imaging methods have been developed for color texture classification and segmentation. Tan and Kittler (1992) used eight texture features derived from the Discrete Cosine Transform of each of the tree color bands for color texture classification. Liu and Yang (1994), Panjwani and Healey (1995), and Suen and Healey (1999) presented Markov random field models for unsupervised segmentation of colored texture images. The color features are defined by the parameters of Markov models, which are estimated with sophisticated maximum likelihood scheme or relaxation process. Healey and Slater (1994) used illumination invariant descriptors of the 3-D color histogram for colored texture recognition. Healey and Slater (1997) used a Gaussian low-pass filter and a

difference of Gaussian high-pass filter to derive a set of moment invariants of distributions in color images for illumination- invariant texture recognition. Thai and Healey (2000) further described a method for selecting texture discrimination filters based on the spatial content of the texture. Color image regions are represented by a

set of illumination- invariant color covariance features. The optimal filter is designed in the sense that it maximizes the distance between vectors of these features. This method is generally developed for classifying a set of known texture classes.

In industrial color texture inspection, Song et al. (1996) studied surface inspection on random macro color textures. They proposed a two-stage chromatic-structural approach. The first stage uses a histogram-based color clustering scheme to segment a color image into different color classes. The second stage then extracts structural features by blob analysis from each chromatic class separated in the first stage. Finally, the Bayes minimum error rule is applied to identify defects. This approach is developed specifically, and works well for highly irregular textures such as granite. It is relatively complicated to implement for highly regular textures found in industry. Boukouvalas et al. (1999) studied the problem of color shade grading for industrial inspection of ceramic tiles. Their grading method is based on the comparison of color histograms. Boukouvalas and Petruo (2000) also presented a method for perceptual color shade grading of randomly textured surfaces. The proposed method first removes the spatial blurring introduced by the input sensor, and then converts the data to a perceptual color space which emulates the spatial blurring of the human visual system. The data are finally converted to a perceptually uniform color space for color grading.

The traditional color imaging methods are more concerned with the problem of image segmentation than by the problem arising in inspecting colored texture surfaces, where local defects exhibit no distinct textural properties. Our work has been motivated by a need to develop an efficient and effective technique to detect and

locate defects in colored texture surfaces. In this study, we are considering an automatic surface inspection problem, where defect- free samples of the textures of interest must be given a priori. The proposed method aims at detecting defects, rather than classifying types of defects, embedded in homogeneous textures. For solving the problem of arbitrary orientations of structural textures and reducing computational complexity, we propose a rotation- invariant ring-projection representation for colored texture patterns. Color ring-projection transforms a 2-D color image
f ( x , y ) with

two variables x and y into a 1-D color pattern f (r ) as a function of radius r . The similarity of a sensed image with respective to the reference pattern is measured by the simple and straightforward normalized correlation. Since the proposed color ring-projection representation is rotation invariant, and reduces computational complexity from O(W 2 ) in a 2-D circular image of radius W to O (W ) in the ring-projection space, the easily- implemented correlation measure is insensitive to rotation changes, and is computationally fast.

This paper is organized as follows: Section 2 first describes the proposed color ring-projection representation of colored texture images, and then discusses the regularity measure of normalized correlation between two color ring-projection patterns. Section 3 presents the experimental results on a number of color textures found in industry. The paper is concluded in Section 4.

2. COLOR RING-PROJECTION AND CORRELATION MEASURE

In this study, a defect-free pattern defined in a circular window will slide over the entire sensed image on a pixel-by-pixel basis so that the degree of regularity of every pixel in the image can be determined. The normalized correlation is used as the regularity measure between the defect- free reference pattern and each sensed subimage. In order to reduce computational burden in the inspection process, the color ring-projection representation is proposed. It transforms a 2-D color image into a rotation- invariant pattern in the 1-D ring-projection space. The proposed transformation scheme for colored texture patterns is inspired by the ring-projection algorithm (Tang et al. 1991; Tang et al. 1998), which is originally developed for character recognition in binary images.

Color provides powerful information for texture analysis. The color of a pixel in the image is typically represented with the RGB tristimulus values, each corresponding to the red (R), green (G) and blue (B) frequency bands of the visible light spectrum. Let R( x, y) , G ( x, y) and B( x, y) denote the R, G and B stimulus values at pixel coordinates
( x, y), respectively.

The

color

ring-projection

transformation is carried out separately in each of the three primary-color planes. The pattern of a color texture is contained in a circular window of radius W . The reference pattern can be arbitrarily selected from a non-defective region of colored texture surfaces. The radius W is selected so that the representation of self- similarity of a homogeneous texture pattern is sufficient. Self-similarity means that all

sufficiently large subimages of a colored texture image are considered similar independently of their position.

Let C ( x , y ) represent a color feature at pixel ( x, y) in the Cartesian coordinates. The polar coordinates of C ( x , y ) is given by C (r cos , r sin ) , where x = r cos and y = r sin . The origin of polar coordinates is at the center of the circular window. The color ring-projection of the 2-D image C ( x , y ) at radius r , denoted by p (r ) , is defined as the mean value of C (r cos , r sin ) at the specific ring of radius r . Hence,
p (r) = 1 2r

2 0

C ( r cos , r sin ) d

(1)

The discrete color ring-projection of three primary-color planes R( x, y) , G ( x, y) and

B( x, y) are given, respectively, by

p R (r ) = p G (r) = p B (r ) =

1 nr 1 nr 1 nr

R (r cos
k

, r sin k ) , r sin k ) , r sin k )

(2) (3) (4)

G(r cos
k

B (r cos
k

where n r is the total number of pixels falling on the ring of radius r ,

r = 0,1,2,...,W . Therefore, a 2-D image with two independent variables x and y in

each color plane is now represented by the 1-D ring-projection pattern with one single variable r . Since the projection is constructed along circular rings of increasing radii, the derived 1-D color ring-projection pattern is invariant to rotation changes of its original 2-D image. Figure 1 demonstrates the representation of 1-D ring-projection patterns. Figures 1(a) and 1(b) show the images of an oriented textile fabric in two

distinct orientations. Figures 1(c) and 1(d) present the plots of ring projections as a function of radius r in color plane R(x, y). It can be seen from the figures that the plots of ring projections are approximately identical, regardless of orientation changes.

An original 2-D RGB image can now be represented by a sequence of RGB ring-projection vectors ( r ) = ( p R (r ), pG ( r ), p B ( r )) , for r = 0,1,2,...,W . Given a circular window of radius W, data dimensionality of the original 2-D RGB image is reduced from 3(W 2 ) to 3 W in the 1-D color ring-projection space.

In the inspection process, the regularity measure of a sensed subimage with respect to the defect-free reference pattern defined in the circular window of radius W is given by the normalize correlation. Let M ( r ) = ( p R ( r ), p G ( r ), p B (r )) S ( r ) = ( p R ( r ), pG ( r ), p B ( r ))

M (r ) and S (r ) represent the RGB ring-projection vectors of the defect- free

reference pattern and the sensed subimage, respectively, at the ring of radius r . The elements of both vectors M (r ) and S (r ) are obtained from eqs.(2)-(4). The normalized correlation between two color ring-projection patterns

{M (r ), r = 0,1,2,..., W } and {S ( r ), r = 0,1, 2,..., W } is defined by

P = MS MM SS

(5)

where
MS = [M ( r ) S ( r ) ] 3(W + 1) M S
r =0 W W

MM = M ( r ) 3(W + 1) M
2 2 r =0

 SS = S ( r ) 3(W + 1) S
2 r =0

and
M
W 1 = [p R (r) + p G (r) + p B (r)] (3W + 1) r =0 W 1 [ p R (r ) + p G (r ) + p B (r )] (3W + 1) r= 0

S =

and above denote the inner product and the norm, respectively. The normalized correlation P obtained from eq.(5) is between 1 and 1. In this study, we take only positive values of P such that P [0,1] , i.e.,

P = max { P ,0}

In the inspection process, the circular window will slide over the entire sensed image on a pixel-by-pixel basis, and then the correlation value of each pixel in the image can be determined. A homogeneous region in the sensed image will have correlation value approximate to unity, and any irregular region with respect to the defect- free reference pattern will have correlation value close to zero. Therefore, we employ the simple statistical process control principle to set up the control limit for distinguishing defects from homogeneous textures in the correlation image. Since the desired correlation value is unity between the reference pattern and each sensed subimage, only the lower control limit is required to detect the deviation of correlation values. The lower control limit (threshold) is given by K

(6)

where and are the mean and standard deviation of all correlation values obtained in the image of size M N , i.e.,
9

 = 2 =

1 M N 1 M N

M 1 N 1 y = 0 x =0

y=0 x=0

(x , y ) ( x, y ) ]
2

M 1 N 1

P ( x, y ) is the correlation value of the circular window with the center at ( x, y) . K

in eq.(6) is a control constant. A 3-sigma standard has been used normally in the statistical process control for detecting assignable causes. In this study, control constant K =3 is also used to set up the correlation threshold. If the correlation value of a pixel in the sensed image is larger than the control limit (threshold) 3 , the pixel is classified as a homogeneous element. Otherwise, it is classified as a defective element.

3. EXPERIMENTAL RESULTS

In this section, we present the experimental results on a number of colored texture surfaces for evaluating the validity of the proposed ring-projection correlation method. All experiments are implemented on a Pentium 300MHz personal computer using the C language. The RGB images are 256 256 pixels wide with eight bits of intensity per band. The radius of the circular window is 25 pixels for all test samples. In the training process, a circular subimage of radius 25 pixels for each texture class is arbitrarily selected from the defect- free textured surface to construct the color ring-projection pattern. Computation time of an image of size 256 256 and a circular window of radius 25 is approximately 6 seconds with the PC mentioned above.
10

Without loss of visibility, all test samples of color textures are displayed with gray- level images in this paper. Figure 2(a) shows a stripe-textured paper surface with water stains. Figure 2(b) presents the plot of the correlation function (1 P ) 255 in 3-D perspective, where the peaks indicate the locations of the stain defects. It can be seen from Figure 2(b) that all pixels in the homogeneous texture region have small

(1 P ) 255 values (i.e. P close to 1), and pixels in the defective regions have
relatively large (1 P ) 255 values (i.e. P approximate to 0). Figure 2(c) depicts the defect detection result of Figure 2(a) as a binary image, where pixels with correlation values below the control limit (threshold) 3 are represented by white intensity, and the ones above the control limit are represented by black intensity. It shows that the water stains in the original image are correctly detected and located in the resulting binary image. Figure 2(d) presents the correlation values as an intensity function for the gray-level version of the paper surface, where brightness is proportional to the magnitude of P . The texture information in the gray- level image is not as sufficient as its counterpart in the color image, and the resulting P values cannot detect the water stains.

Figure 3(a) shows a textile fabric with a shedding defect on the surface. Figure 3(b) illustrates the corresponding detection result as a binary image. It shows that the defect is reliably detected. In this study, we have used the 3-sigma standard (3 ) to set up the threshold for discriminating between defects and homogeneous regions. Noisy points with P values below -3 may be generated, as seen in Figure 3(b). The noisy points in the resulting binary images are generally very small in size and distributed scatteringly, whereas the true defects are clustered into large areas. We

11

may carry out a simple postprocessing such as blob analysis to prevent false alarm. The selection of constant K is not crucial at all for the control limit K . Figure 3(c) shows the binarized results of the textile fabric in Figure 3(a) using the 6-sigma (6 ) limit. It can be seen from Figures 3(b) and 3(c) that a looser control limit such as 6 yields less number of noisy points and makes the detected defect smaller in size in the binarized images. The shedding defect is reliably detected either using the 3-sigma (Figure 3(b)) or 6-sigma (Figure 3(c)) limit. In order to show the effect of changes in lighting, Figures 4(a) and 4(b) show respectively the overexposed and underexposed versions of the fabric in Figure 3(a). The corresponding detection results, as shown in Figures 4(c) and 4(d), reveal that the proposed ring-projection correlation scheme is insensitive to the change of uniform illumination.

Figure 5(a) and 5(b) show a milled surface with a small crack in two different orientations. The feedmarks on the milled surface show oriented structures of the texture. The defect- free reference pattern is obtained from the milled image with horizontal feedmarks as the one shown in the Figure 5(a). Figure 5(b) is a 90o -rotation version of Figure 5(a). From the resulting binary images in Figures 5(c) and 5(d), it can be seen that the crack in the milled surface has been reliably detected, regardless of the surface orientation.

12

4. CONCLUSIONS

In automatic surface inspection, small surface defects that appear as local anomalies embedded in a homogeneous texture must be detected. As local defects exhibit no distinct textural properties, this inspection task is clearly different from segmentation by texture. In this paper, we have presented a straightforward and efficient method for detecting unanticipated defects on colored texture surfaces. The core techniques of the proposed method are based on the rotation- invariant representation of color ring-projection and the easily- implemented correlation measure. The proposed method can detect any unanticipated defects as long as they break the homogeneity of textures, but it cannot further classify types of the detected defects.

The proposed color ring-projection transforms a 2-D color image in the Cartesian coordinates into a 1-D color pattern as a function of radius. The color features of each ring with a specific radius are represented by the mean RGB tristimulus values of all pixels falling on the ring. The color ring-projection representation is rotation- invariant and, therefore, is suitable for both statistical (isotropic) textures and structural (oriented) textures with unknown orientation.

Compared with matched filtering approaches such as Gabor filters, the proposed color ring-projection correlation method reduces computational complexity from O(W 2 ) to O (W ) , where W defines the width of the neighborhood window. Since matched filtering approaches are sensitive to orientation changes, they become

13

impractical when the orientation of structural textures cannot be predetermined. The proposed color ring-projection representation alleviates the burden of exhaustive search of orientation, and eliminates the requirement of physical alignment of orientation.

Based on the experimental results, the proposed color ring-projection correlation method performs reliably for highly regular textures such as man-made textile fabrics and machined surfaces. Its performance degrades gracefully for less regular textures such as natural wood and granite. A more robust and complex classifier other than the straightforward correlation measure is worth further investigation, which may extend the proposed color ring-projection representation for inspecting defects in less regular textures.

14

REFERENCES

Boukouvalas C, Kittler J, Marik R, Petrou M (1999) Color grading of randomly textured ceramic tiles using color histograms. IEEE Trans. Industrial Electronics 46: 219-226. Boukouvalas C, Petrou M (2000) Perceptual correction for color grading of random textures. Machine Vision and Applications 12: 129-136. Conners RW, McMillin CW, Lin K, and Vasquez-Espinosa RE (1983) Identifying and locating surface defects in wood. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-5: 573-583. Daugman JG (1985) Uncertainty relation for resolution in space, spatial- frequency, and orientation optimized by two-dimensional visual cortical filters. J. Opt. Soc. Amer. 2: 1160-1169. Escofet J, Navarro, RB, Millan MS, Pladellorens JM (1996) Detection of local defects in textile webs using Gabor filter. Proceedings of SPIE-The International Society for Optical Engineering, Bellingham, WA, 163-170. Healey G, Slater D (1994) Global color constancy: recognition of objects by use of illumination invariant properties of color distributions. J. Optical Society of America 11: 3003-3010. Healey G, Slater D (1997) Computing illuminaitona- invariant descriptors of spatially filtered color image regions. IEEE Trans. Image Processing 6: 1002-1013. Jain AK, Ratha NK, Lakshmanan S (1997) Object detection using Gabor filters. Pattern Recognition 30: 295-309. Liu J, Yang YH (1994) Multiresolution color image segmentation. IEEE Trans. Pattern Anal Mach. Intell. 16: 689-700. Liu SS, Jernigan ME (1990) Texture analysis and discrimination in additive noise.

15

Computer Vision, Graphics, Image Process. 49: 52-67.

&& Ojala T, Pietikainen M, Silven O (1992) Edge-based texture measures for surface inspection. Proceedings of the 11 th International Conference on Pattern Recognition, The Hague, The Netherlands, B-594~B-598.
Panjwani DK, Healey G (1995) Markov random fields for unsupervised segmentation of texture color images. IEEE Trans. Pattern Anal. Mach. Intell. 17: 939-954. Pikaz A, Averbuch A (1997) An efficient topological characterization of gray- levels textures using a multiresolution representation. Graphical Models and Image Processing 59:1-17. Ramana KV, Ramamoorthy B (1996) Statistical methods to compare the texture features of machined surfaces. Pattern Recognition 29: 1447-1459. Randen T, Husoy JH (1999) Filtering for texture classification: a comparative study. IEEE Trans. Pattern Anal. Mach. Intell. 21: 291-310. Siew LH, Hogdson RM (1988) Texture measures for carpet wear assessment. IEEE Trans. Pattern Anal. Mach. Intell. 10: 92-105. Song KY, Kittler J, Petrou M (1996) Defect detection in random colour textures. Image and Vision Computing 14: 667-683. Suen PH, Healey G (1999) Modeling and classifying color textures using random fields in a random environment. Pattern Recognition 32: 1009-1017. Sum C, Wee WG (1982) Neighboring gray level dependence matrix. Computer Vision Graphics Image Processing 23: 341-352. Swain MJ, Ballard DH (1991) Color indexing. Int. J. Computer Vision 7: 11-32. Tan SC, Kittler J (1992) On colour texture representation and classification. Proceedings of 2nd Int. Conf. on Image Processing, 390-395. Tang YY, Cheng HD, Suen CY (1991) Transformation-ring-projection (TRP) algorithm and its VLSI implementation. Int. J. Pattern Recog. Artif. Intell. 5: 25-56.
16

Tang YY, Li BF, Ma H, Liu J (1998) Ring-projection-wavelet- fractal signatures: a novel approach to feature extraction. IEEE Trans. Circuits and Systems : Analog and Digital Signal Processing 45: 1130-1134. Teuner A, Pichler O, Hosticka BJ (1995) Unsupervised texture segmentation of images using tuned matched Gabor filters. IEEE Trans. Image Processing 4: 863-870. Thai B, Healey G (2000) Optimal spatial filter selection for illumination- invariant color texture discrimination. IEEE Trans. Systems, Man, Cybernetics- Part B: Cybernetics 30: 7610-616. Weldon T, Higgins WE (1999) Designing multiple Gabor filters for multitexture image segmentation. Opt. Eng. 38: 1478-1489.

17

(a)

(b)

p R (r )
255 240 225 210 195 180 165 150 135 120 105 90 75 60 45 30 15 0

p R (r )
255 240 225 210 195 180 165 150 135 120 105 90 75 60 45 30 15 0

(c)

(d)

Figure 1. 1-D color ring-projection representation of a 2-D oriented texture in two distinct orientations:(a),(b) the textured images with concentric rings ; (c), (d) the ring projections in color plane R( x, y) .

18

(a)

(b)

(c)
Figure 2. (a) A stripe-structured paper surface with water stains. (b) The correlation function the paper surface.

(d)

(1 P ) 255 in 3-D perspective for the color version of

(c) The binarized image of (b), which shows the defects as a white intensity. (d) The resulting correlation values as an intensity function for the gray-level version of the paper surface.

19

(a)

(b)

(c)

Figure 3. (a) A textile fabric with a shedding defect. (b) The binarized image from the 3-sigma limit, which shows the shedding defect as a white intensity. (c) The binarized image from the 6-sigma limit.

20

(a)

(b)

(c)

(d)

Figure 4. (a)(b) The overexposed and underexposed images of the textile fabric shown in Figure 3(a); (c)(d) the resulting binary images of (a) and (b), respectively.

21

(a)

(b)

(c)

(d)

Figure 5. A milled surface in two different orientations: (a), (b) the original images; (c), (d) the corresponding binary images that show the crack as a white intensity.

22