FAULT DETECTION AND LOCALIZATION IN EMPTY WATER BOTTLES

THROUGH MACHINE VISION

Faisal Shafait, Syed Muhammad Imran, Sven Klette-Matzat
Hamburg University of Technology, Hamburg, Germany.
[ faisalshafait, syed.imranl @tu-harburg.de
Machine Vision Research Group,
Filtec-Europe Gmbh,
Hamburg, Germany.
Ernail: [email protected]

Abstract Bottle inspection systems can be divided into two

Th'ispaper presents a vision-based approach f o r fault categorizes [21: online inspection systems and offline
-detection for the empty water bottles. The goal of our inspection systems. The online inspection systems inspect
approach is to find all defects in the empty water bottles. the bottles while they are moving on the conveyer belt.
The main constraint is the real time operation as the The offline inspection systems take samples of the bottles,
bottles move continuously on the conveyer belt. Using and evaluate them off-line.
images obtained from the camera, we first find the Currently, state of the art in bottle inspection includes
position of the bottle in the image by generalized Hough both vision-based inspection and manual inspection. The
transform. Then we check the bottles for any defects o r ultimate goal of any bottle inspection system is to reject
presence of alien objects using rectangular and circular all defected bottles (termed as true negatives), while not
areas intensity inspection. The bottles with any defects rejecting any good or perfect bottle (termed as false
are rejected and so separated from the main stream. In negatives). Manual systems reduce the percentage of false
the statistics obtained, our inspection system successfully negatives down to 0%, but fail to detect all the defected
rejected all the faulty or defected bottles. The good bottle bottles. A recent study of unsuitable factors for bottle
rejection rate was also lower than 1 % of the total number inspection operators [3] shows that continuous stress on
of bottles inspected. This system can be used for bottle inspectors affects their alertness level, resulting in
inspecting any type of empty bottles for defects or
'
an increase in fault bottles missed rate. Methods based on
presence of alien objects. machine vision, on the other hand, successfully detect all
the defected bottles but have a relatively high number of
Keywords: Bottle Inspection: Surface Inspection; false negatives. The additional advantage of vision-based
Machine Vision; Quality Control. systems is their on-line operation resulting in more
stringent quality control.
1. INTRODUCTION The first step required to visually inspect a bottle is to
detect the position of the bottle in the camera image.
Quality control is an essential part of ,any bottle Hough transform [4] based image segmentation is used
production system. When the bottles are produced in a for this purpose. After obtaining the position of bottle in
manufacturing plant, they are inspected for manufacturing the camera image, the image area containing the bottle is
faults. However when empty bottles returned by the inspected for the presence of any faults or defects. The
consumers are to be reused, the focus of quality control inspection is done on the basis of pixel intensity
changes. Reusable bottles are becoming increasingly information. We introduce a new approach for fault
popular due to their cost effectiveness. Inspection of the localization by inspecting small segments simila.in shape
returned bottles, for re-using them, is. required to ensure to that of the area under inspection. This' information.
quality. The quality control at this stage refers to the about fault position is used to investigate the status of the
detection of any defects (damaged finish, cracks, shape fault under inspection.
deformation) or the presence of alien objects (dirt Section 2 starts with an overview of camera setup
particles, sticky objects) in the bottle. There are many used to inspect the bottles. Then bottle detection using
potential causes for the presence of such problems Hough transform is explained. Our approach of fault
including equipment faults, personnel errors, non- detection and localization is discussed in the last part of
standard procedures, or environment [I]. The focus of this section 2. Section 3 provides some statistical results based
paper is to eliminate the personnel errors by replacing on our vision inspection system. Section 4 concludes the
manual inspection with vision-based inspectiqn.

0-7803-8655-8/04/$20.00 0 2004 IEEE 30

paper by giving a bird’s eye view of the techniques used cameras is to find the position of the bottle in each image.
and results achieved. Real time object recognition using Hough transform is
2. METHOD used for this purpose. Fig 2 shows the image of the
bottom of the bottle passing through conveyer belt.
2.1 Camera Setup Shadow from Shadow from
bar code on handle ofthe
We need to have images of a bottle from different the bottle
positions, to inspect the surfaces of the bottles. To take
different views of the bottle under inspection, we have
used 3 cameras. The cameras were set into progressive
scan mode and their focus was manually adjusted for hest
picture quality. The exposure time for the cameras was
carefully adjusted to eliminate motion blur. This made the
presence of external light necessary. Hence two 60 watts
flicker-free lamps were used. In order to inspect different
surfaces (base, inner sidewall, outer sidewall) of the
bottle, the camera positions we have used are as explained
below.
Figure 2: Bottom of the bottle
2.1.1 Base View: We used a high-resolution camera to
inspect the bottom of the bottles for the presence of
2.2.1 Generalized Hough Transform: A prominent
foreign objects, mold and chipped heels. The camera
property of the conventional Hough transform is that its
position in Fig. la is used for this purpose.
applicability is restricted to detect analytic curves.
Therefore, Ballad [5] generalized the Hough transform to
detect arbitrary shapes. He also took the edge orientation
into account, which made the algorithm faster and also
greatly improved its accuracy by reducing the number of
false positives.

2.2.2 R-Table Construction: To perform the offline

phase of the GHT, the so-called R-table is constructed
using information about the position and orientation of the
edges in the reference image. The edges are extracted
using Sobel operator [6] as shown in Fig 3. We prefer to
(a) (b) (C)
use the Sobel filter because it represents a good
Figure 1:Different camera viewing positions. (a) Top compromise between computation time and accuracy. Its
view (b) Outer Sidewall view from left (c) Outer anisotropic response and its worst accuracy can be
Sidewall view from right. balanced by choosing an adequate quantization of the
gradient directions.
2.1.2 Inner Sidewall View: Image of inner sidewall of
the bottle is helpful to find any dirty material inside the
bottle. The camera position in Fig. l a provides this view
when the camera is placed close to the bottle’s top.

2.1.3 Outer Sidewall View: The positions of the camera

in Fig. Ib and Fig. IC are helpful to take outer sidewall
images of the bottle. These images are helpful to detect
the defected or broken bottles. These images are also
helpful in sorting of the bottles with different shapes and
Figure 3: Edge detection of the base of the bottle by
sizes.
Sobel operator.
2.2 Bottle Recognition The first step in generating the R-table is to choose a
reference point o = (xo. yo), e.g., the centroid of all edge
The first task after obtaining the images from the __
points p: (i = 1 . N;) in the reference image. Then r; = o .
- p ; is calculated for all points and r is stored as a function

31
of the corresponding gradient direction @. Assuming the approaches have a reduced computational complexity in
case of rigid motion, in the online phase a three- common but on the other hand have serious limitations.
dimensional accumulator m a y A is set up over the The use of relative gradient angles supposes the object not
domain of parameters, where the parameter space is to be occluded, whereas considering the slope and the
quantized and range restricted. Each finite cell of that curvature fails when dealing with shapes that are
array corresponds to's certain range of positions and composed mainly of straight lines.
orientations of the reference image in the search image,
which can be described by the three variables x, y , and 8.
Here, x and y describe the translated position of o in the
search image and 8 the orientation of the object in the
search image relative to the object in the reference image.
For each edge pixel p: in the search image and each R-
table corresponding to one orientation 4, all cells r, + p:
in A receive a vote, i.e., they are incremented by 1, within
the corresponding two dimensional hyper plane defined
by 6' = 6 under the condition that @si = @'i. Maxima in A
correspond to possible instances of the object in the
search image.
As we know the shape of the bottles varies from each
other so we used generalized Hough transform to detect
different shape of the bottles. In this particular case we Figure 5: Bottle detection from top view of a typical
have to detect the base of the bottle, and the base of the empty water gallon. The green line represents the
bottle is in circular shape. So, here we used circular detected bottle, while the purple arcs at the four
Hough transform, to detect circular area (base) of the corners represent the points making the best-fit circle.
bottle as shown in Fig 4.

We use the approach by Ulrich [IO] in which the

optimization is achieved in four steps. In the first step, a
hierarchical strategy i s adapted. In this strategy an image
pyramid of both images, the reference image and the
search image, is generated. This helps in making a rough
estimate of object position in the image at a vely low
computational cost. Then this search is refined in the .
following steps until the lowest level image in the
pyramid is reached. Morphological filtering and tiling is
Figure 4: Bottle detection by Hough Transform used to reduce the search region while moving from one
pyramid level to the other. Using this technique the GHT
@ @
2.2.3 Optimizing the Generalized Hough lkausform: is calculated in about 15 msec at an Intel Pentium 4,
The reduction of the high computational complexity of 2.6 GHz workstation.
both, the conventional Hough Transform (HT) and the
GHT, has been the subject of several publications.
Yacoub [7],for example, propose an HT algorithm based
2.3 Fault Detection
on a hierarchical processing for line detection. It performs
Once the position of the bottle in the camera images is
a classical HT on small suh-images and merges the
known, the next step is to inspect the bottle image for any
extracted lines with similar parameters by successively
joining four neighboring suh-images until the original defects. This step requires the examination of each pixel
on the bottle. Since we know the pixel intensities of
image size is reached. In object recognition this approach
perfect bottles, we inspect individual pixels on that basis.
is not reasonable because the object may he spread over
If the intensity of an individual pixel does not lie within
several sub-images, which results in a high sensitivity to
the tolerance limit, the pixel is designated as ourlier. The
clutter and noise. Other approaches reduce the dimension
of the parameter space by introducing additional intensities of an individual pixel, however, may change
due to noise, reflection, or change in external lighting
information: SER & SIU [8] use relative gradient angles
in the R-table, whereas MA & CHEN [9] consider the condition. Hence if decisions for presence of defects are
slope and the curvature as local properties. These
made on the basis of individual pixels, these decisions are
not very accurate.
Therefore we divide the bottle image into several
segments. The exact size of these segments depends on No. of Fault Bottle Fault Bottle Good
the required accuracy of inspection, since it defmes the Inspected Rejection Missed Bottle
smallest size of fault that can be detected. All the pixels in Bottles Rate Rate Rejection
a segment are inspected. If the number of outliers in a (B) (Bfi) (Eh) Rate (Em)
segment exceeds the pre-defined threshold, the segment is 32261 2.78% 0% 0.52%
marked as defected.
35188 2.76% 0% 0.81%
The shape of the segment is chosen on the basis of
shape of the bottle image itself. If the surface under 45217 1.72% 0% 0.35%
inspection is circular in shape, like the base of water
bottles, the image is also segmented into circular regions. 60496 I 2.05% ,I . 0% I 0.47%
These regions are further divided into arcs of equal
I
58979 I
I
2.72% 1, 0% I
I
0.88% I
lengths. Since water bottles usually have circular designs
at the base, the segmentation of the base image into
circles allows to set different tolerance and threshold Table 11: Statistirs based on manual inspection.
values for each segment.

(a) (b)
Figure 6Fault detection after image segmentation. (a)
Handle and bar code shadows are clearly detected. (b)
Foreign object inside the bottle is detected.
It was our main target to minimize B e in our inspection
Since we know the shape of the bottle under system. If we again observe the tables I1 & I we can see
inspection, we can separate the false negatives from true that B&s slightly increased in our inspection system as
negatives using information about the size and relative compared to manual inspection. There are many factors,
position of the detected regions. For example for the which are the cause to increase the Bp (good bottle
botde in Fig. 2, the bottle handle and the bar code have a rejection rate),B p mainly depends on environment,
fixed size.and relative distance, so these can be easily proper placement of vision inspection system and minor
separated from the other negatively detected regions. production faults (e.g. prominent part line) of the bottles.
These above mentioned factors are external force to
3. RESULTS AND DISCUSSION . increase the Bv values independent of the performance of
our vision-based inspection system.
The statistical results obtained using our visual inspection
tool are given in Table I. In order to compare the state of 4. CONCLUSION
the art in the field of bottle inspection with our visual
inspection tool, the result obtained from manual In this paper we have presented a vision-based
inspection are listed in Table II.Since manual inspection approach for empty water bottle's inspection. State of the
was not very precise, only approximate values are used in art machine vision tools are being used in bottle
Table II. inspection systems. The main problem in previous manual
If we compare both of the statistical results in tables 11 based inspection system was high missed rate of faulty
and I, we can see that (fault bottle miss rate) is very bottles. Our objective is to minimize fault bottle missed
high io manual inspection of bottles where as Bv is rate.
minimum down to 0%. In our vision based inspection We have used circular Hough transform for bottle
system Bm,is low down to minimum. detection, since the bottles under inspection had circular
base. Generalized Hough transform c q be used to detect

'33
 bottles of~anyshape. After bottle detection.’ we inspect the Photogrammetrie und . Fernerkundung
bottle for any defects or faults using image segmentation. (DGPF), pages 571-578, Berlin, 2001
A special technique of shape dependent segmentation was
used to obtain more precise information. The decision for
a defect was based on the intensity variations of the image ’ .
within each segment. This helped to remove the problem
of global illumination changes.
Using this vision based approach; we successfully
reduced the fault bottle missed rate’(&,) down to 0% in
real time processing. We got slightly increased good
bottles rejection rate (lower than I%), as compared to
.manual inspection, but the gain in fault bottle rejection
rate ensured more stringent quality control as compared to
manual inspection. Methods to reduce good bottle
rejection rate are open for future investigations.

References
[I] Kristin Safran and Barbara L. Marteney.
‘Troubleshooting Bottled Water Complaints,” Water
Quality Producfs Magazine, October 200 I .
[2] Mikell Knights, “Bottle Inspection Close-Up,”
Plastics Technology . Online Magazine.
htt~://www.~lasticstechnoloev.com/aRiclesll99908c
u4.html
[3] Takeo Kondo, “A Study of Unsuitable Factors for
Bottle Inspection’ Operator,’’ Foods and Food
Ingredients Journal of Japan, No.166, 1995.
[4] Hough, P.V. C., “Methods and means for recognizing
complex patterns,” U. S. Patenf 3,069,654. 1962.-
[5] D. H. Ballard, “Generalizing the Hough Transform to
Detect . Arbitrary Shapes,” Pattern Recognition,
13:lll-122, 1981.
[6] N. Kanopoulos, N. Vasanthavada, and R. Baker,
“Design of an edge detection filter using the Sobel
operator,” IEEE J. Solid-State Circuits 23 (1988).
171 Yacoub, S.B., “Detecting subjective contours with the
hierarchical Hough transform,” Proc. Fiflh
International Conference on Image Processing and
its Applications,~pages365-369, 1995.
(81 ‘ P . K. Ser and W. C.. Siu, “Non-analytic object
recognition using the Hough transform with matching
technique,” IEE Pwcessing - Computers and Digital
Techniques, 141( 1): 11-16, 1994.
[9] D. Ma and Xing Chen, “Hough transfo,rm using slope
.. and curvature as local properties to.detect arbitrary
2D shapes Proceedings of the Intemtional
Conference on Pattern Recognifion, pages 51 1-5 13,
1988.
[IO] Markus Ulrich, Carsten Steger, Albert Baumgartner,
’ ”
and Heinrich Ebner, “Real-time object recognition
using a modified generalized Hough transform,”
Deutschen Gesellschaj? fiir

34