170 IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.

10, October 2008

Webcam Based System for Press Part Industrial Inspection

Habibullah Akbar and Anton Satria Prabuwono

Faculty of Information and Communication Technology, Technical University of Malaysia Melaka

Locked Bag 1200, Hang Tuah Jaya, Ayer Keroh, 75450 Melaka, Malaysia

Summary In contrast, machine inspection performs the measurement

Industrial inspection is one of the crucial factors to ensure while the part being produced on line production. The
quality of product before reach the market. The system integrates lighting system, acquisition devices or
inspection tasks can be done by using visual system such camera, computer, image processing software, and
as human vision or machine vision or combination of both. handling equipment. The camera will capture images
In this paper, we describe a system that capable to control under suitable lighting condition [1]. Then, the image
quality of press part product. The scheme is based on enters the software to be analyzed and processed.
CMOS web-camera (webcam) in a production line. The Although most of typical camera is sensitive to the
main objectives are to develop image processing environment then it would be necessary to control the
algorithm which can measure the dimension of parts and lighting condition. The lighting condition can be
decide whether it is accepted or rejected. In this research, controlled using dimmer control based on the requirement
we divide the system into hardware and software of product being observed.
components. The result shows its possibility to be used as
automated visual inspection system for press part quality The result of image processing stages then enters the
control. The dimensions of the part can be obtained by classification module to determine whether an actual part
calculating the pixel value. matches the requirement and its standard or not. If the part
Key words: does not fit the quality, the output from this module will
Visual inspection, press part, webcam trigger the handling equipment to dissociate the part from
production line simulation.
In this paper, a scheme of machine inspection system in
1. Introduction presented. The method described allowed the dimension
Quality is one of the factors which determine the customer measurement by calculating the pixel value multiply the
satisfactory and its interchangeability to replace other part. geometrical constrain. The method has advantages that the
The process is done by comparing actual part and a set of system is low cost and does not need of traditional
requirement and its standard. The method for determine measurement equipment such as calipers, ruler, and
quality is divided into human inspection and machine checking fixture.
inspection. The paper is organized as follow: section 2 gives the
Human inspection usually executes measurement after part importance of Automated Visual Inspection (AVI) on
has been produced. In the traditional approach human or industrial area. Section 3 reviews the basic theory of
experts perform measurement using conventional image and related image processing method for the
instruments such as rules, calipers, and micrometers. In application. The detail of methodology used for the system
addition, current inspection can also be using other tools described in section 4, followed by result and discussion
e.g. probe-based Coordinate Measurement Machine in section 5. Finally, conclusion is given in section 6.
(CMM). CMM measures the dimension by contact the
probe to the surface of part. The probe position will be
2. Importance of AVI
measured by computer software and the result display at
the monitor. CMM is not automatically move the probe to Human inspection can perform complete quality
touch the point of measurement but needs of human action inspection but it is a time consuming process and causes
to manually move the probe into desired position. Another inconsistent result between operators [2]. Although it is
CMM type is using laser scanner as acquisition devices to the most common and simple inspection method but there
replace probe-based type and categorized as real time are several disadvantages because human can suffer of
inspection. illness, human error, and slow. On the other side, real time
inspection offers many advantages such as increasing

Manuscript received October 5, 2008

Manuscript revised October 20, 2008
IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.10, October 2008 171

speed, accuracy and reliability. In addition, vision based specular effect of metal surface. Fortunately, the light
systems have been implemented in the industrial sector all intensity is controllable since we have indoor application.
over the world [3].There are hundreds and thousands of Since we are dealing with real time application, the
different applications and many more are being developed computer hardware to be used should be fast enough to
or improved day by day [4]. calculate the routine task. The time consumes of accepted
or rejected decision depends on the computer speed.
As described above However, automated inspection Generally, the faster the computer, the less time required
system covers the full range of technical difficulty in to process the data.
computer vision [5]. Real time visual inspection is not a
simply image processing problem but the system combines Finally, the part status is used to trigger the handling
of many aspects as stated in earlier. mechanism to separate accepted part from defected part
over production line or vice versa. This scenario is
The other problem of applying automated inspection commonly used for developing AVI system in
system is the cost. The commercial one starts from USD manufacturing environment.
30,000 up to USD 200,000 or even more [6]. In addition,
companies who have installed them are unwilling to In addition, if there is a defective part, then the report can
release details of their system or the cost savings because be transmitted to DSS to be analyzed. The visual
they want to maintain a competitive advantage over their inspection system should be integrated within other
rival [7]. manufacturing process to perform total quality control and
reduce cost of producing defective product. Figure 1
The aim of this design is how to apply machine vision- shows the typical networking system which is using
based for press part inspection system. The design wireless networking for Computer Integrated
includes image acquisition, image processing software, Manufacturing (CIM).
handling equipment and simulation of line production in
the manufacturing environment. The requirements of
visual inspection for press part sorting is can measure the
dimension of press part.
As stated earlier, the purpose of visual inspection system
is to perform quality assurance by separating accepted part
from rejected product. To do this, a careful planning
should be taken in the design stage. The orientation is
based of the object to observe. The system should cover
the following aspects to perform total quality control [8].
1. Adaptive lighting control. Fig. 1 Visual inspection networking system.
2. Flexible software development approach to
accommodate changes in the future. The DSS is aim to decide proper action if there is
3. Networking system for communication between defective product detected. DSS will analyze the
Decision Support System (DSS), Computer Aided manufacturing process which causes the fail product. The
Design (CAD), Computer Aided Manufacturing wrong process could be from CAD, CAM or the CAPP.
(CAM), and Computer Aided Process Planning The CAD is aim to perform design analysis, test and
(CAPP) in manufacturing environment. evaluate the prototype, and production drawings. The
CAM and CAPP is aim to select the proper material,
However, our main focus in this paper is the requirement process and equipment for fabricate the product. All over
for the computer vision. The system consists of digital the system is integrated into a CIM which is aim to control
image acquisition, computer, image processing routines, the entire production process. CIM provides the data
and handling mechanism. storage, sensing state and modifying the manufacturing
Firstly, the choice of image acquisition devices is based on process.
minimum requirement of pixel accuracy. For instead, low We also have been developing the production line
resolution web camera can be using because it is very simulation for testing the inspection system. The system is
cheap. The disadvantage is the high noise but can be divided into hardware and software subsystem. The
smoothed in image processing routine. The result of image hardware subsystem as shown below consists of conveyor,
captured from the camera is significantly relies on lighting lighting system, web camera, fuzzy logic controller, and
condition. We must avoid direct illumination to reduce sorter. The lighting system is using dark field illumination
172 IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.10, October 2008

which is controlled by dimmer. The camera has resolution 1. Acquisition – This step is aim capture and
480 x 640 pixels. The conveyor has maximum speed up to transmits the image from the scene to computer
10 meters per minute. The hardware controlled is host.
AT89S52 microcontroller based. The handling equipment 2. Noise reduction – This step suppress noise
consists of plate metal mounted into stepper motor. introduce by acquisition process
3. Edge detection – This step detects sharply changes
in image.
3. Theoretical Review 4. Feature analysis – This step is aim to extract
feature descriptors about dimension of part
Camera records the particular scene and converts it into specification and compares feature descriptor from
2D image. The feature information of the image is previous step and its standard requirement. Then,
convenient to be expressed in pixel coordinates. In this the part is claimed whether accepted or rejected.
coordinate system, the image is treated as a grid of discrete
elements, ordered from top to bottom and left to right, as 3.1 Acquisition
illustrated by figure 2.
Notice that the image quality is significantly influence by
the illumination. Fortunately, the lighting can be set fixed
for industrial inspection. Under illumination, part surface
will reflect some amount of intensity from the light source.
Then some part of reflectance light will be recorded by
Complementary Metal Oxide Semiconductor (CMOS)
sensor cell and converted into electric signal.
Unfortunately, each optical sensor will introduce statistical
nature which attaches noise in image and creates
discrepancies of interpretation. CMOS-based camera has
10 times of pattern noise than CCD [9]. However, we can
Fig. 2 The pixel coordinate system. suppress the noise in by using filtering technique.
For pixel coordinates, the first component r (the row) 3.2 Noise Reduction
increases downward, while the second component c (the
column) increases to the right. Pixel coordinates are After image acquisition process, the image is converted
integer values and range between 1 and the length of the into grayscale from RGB mode. This is necessary since we
row or column. do not need the color information. We assume that the
noise can be a modeled as additive and random. In this
The main aspect in the computer vision for inspection is step, we use Wiener filter to estimates the local mean (µ)
the image processing processes. The routines is aim to and variance (σ2) around each pixel and described as
extract the feature descriptors which characterize the part. follow [10].
There are two features to be extracted of our part sample,
1
which are diameter and length. The following flow chart
will show the algorithm of the visual inspection, which are
μ=
NM
∑
n1 , n2 ∈ η
a ( n1 , n2 ) (1)

using in the system.

1
σ2 =
NM
∑
n1 , n2 ∈ η
a 2 ( n1 , n2 ) − μ 2 (2)

3.3 Edge Detection

Edge is a sharply changes of intensity in image. The
techniques can be classified based on first order derivative
and second order derivative. First technique is using
gradient vector and estimate the gradient direction e.g.
Roberts, Sobel, and Prewitt. The second technique is using
zero-crossing of the Laplacian or non-linear differential
expression.
The significantly differential change at the image gradient
Fig. 3 Image processing stages. can be used to detect any edge of each region. A well
IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.10, October 2008 173

known algorithm is the Sobel operator which detects the 4. Methodology

vertical and horizontal changes in image. The operator is
using two 3x3 kernels which are convolved with grayscale In this research, the system has following boundaries.
image. Firstly, image processing consists of following techniques
such as image filtering, image enhancement, edge
⎡ + 1 0 − 1⎤ ⎡+ 1 + 2 + 1⎤ detection, feature extraction, feature analysis, and
G x = ⎢⎢+ 2 0 − 2⎥⎥ ∗ I and G y = ⎢ 0 0 0 ⎥⎥ ∗ I (3) recognition. The software developed using visual C++
⎢
⎣⎢ + 1 0 − 1⎥⎦ ⎣⎢− 1 − 2 − 1⎥⎦ 2008 Integrated Development Environment. The computer
is HP Pavilion a6260d Home PC which has windows vista
The letter I denotes for image source, Gx and Gy denotes operating system. The lighting system is consists of 150
for images which contain the horizontal and vertical watt halogen bulb and dimmer control. The parts being
derivative approximation where * stands for 2-D inspected are shown in the figure 4, 5, and 6. The first part
dimensional convolution operation. Other edge detectors has rectangular shape and consists of four holes. The
such as Roberts operator is similar to the Sobel operator
second part has circle shape with one hole at the centre.
which shown below.
The third part has six edges and one inner diameter.
⎡+ 1 0 ⎤ ⎡ 0 + 1⎤
Gx = ⎢ ⎥∗I and Gy = ⎢ ⎥∗I (4) The hardware controller is using DT-51 petrafuzz module
⎣ 0 − 1⎦ ⎣− 1 0 ⎦
from innovativeelectronic based on AT89S52
However, one the most commonly used method for edge microcontroller. The camera is using Proview 5132
detection is Canny edge detector. It consists of the sensonic CMOS web-camera. The conveyor has maximum
following steps: speed at 10 meters per minute. The handling equipment
consists of pinset mounted into stepper motor and
• Smooth an image with a filter. controller from KHG electronics version 1.5.
• Calculate the gradient (magnitude and
orientation) of the smoothed image.
• Apply non-maximal suppression.
• Perform hysteresis thresholding.

3.4 Feature Analysis

The edge detection algorithm showed above is should be

performed for every pixel of the image. This increases the
amount of time to perform the calculation. There is another
technique that is simpler and faster based on the idea that Fig. 4 Rectangular.
different object level or regions have significantly different
grey levels. One method that is relatively simple, does not The feature description of the part is shown in the table 1.
require much specific knowledge of the image, and is
Table 1: Feature description of rectangular part parameter
robust against image noise, is the following iterative
Rectangular Part
In classification, the feature vector is treated as input. The
information represented by the vector represented is No Name Shape Parameter
determined. There are many techniques of classifiers 1 Feature A Circle Diameter A
include Bayesian, rule-based, and neural network. Feature B Circle Diameter B
2
However, in this paper, we used the rule-based method
only. Rule-based systems classify data to if-then rules. 3 Feature C Circle Diameter C
This method use causality to perform classification of 4 Feature D Circle Diameter D
object. For example shown in the following rule: Feature E Left _Edge Line E
5
IF feature 1 > reference + tolerance THEN reject 6 Feature F Bottom _Edge Line F
IF feature 1 < reference - tolerance THEN reject Feature G Right _Edge Line G
7
Otherwise THEN accept
8 Feature H Upper _Edge Line H
This rule would reject an object which is out of its
tolerance. The feature of the part denoted as letter.
174 IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.10, October 2008

Fig. 5 Circle.

Table 2: Feature description of circle part parameter

Circle Part
No Name Shape Parameter
1 Feature A Circle Diameter A
2 Feature B Circle Diameter B
Fig. 7 Waterfall model of the software lifecycle.

The waterfall has stages as follow:

Feasibility study – Explore whether a solution is attainable
within the given set of constrain.
Requirement – Description of behavior of the system to be
develop.
Analysis – Evaluate the requirements create model of how
Fig. 6 Hexagonal. the system works under given condition.
Architectural design – concerned with the overall form of
Table 3: Feature description of hexagonal part parameter solution.
Detailed design – Develop description of the element from
Hexagonal Part
previous phase.
No Name Shape Parameter
Implementation (coding) – Create realization of the
1 Feature A Edge Line A abstract design into software system.
2 Feature B Edge Line B Testing – Validation the implemented software against its
original requirements.
3 Feature C Edge Line C
Maintenance – Repeat all the task of previous phase to
4 Feature D Edge Line D check consistency with its design plan.
5 Feature E Edge Line E
6 Feature F Edge Line F
7 Feature G Circle Diameter G

4.1 Software Development Approach

The methodology is described as follow. Firstly, the
software development uses waterfall model which has
configuration as follow [11], [12]:

Fig. 8 GUI for inspection system.

The Graphical User Interface (GUI) shown in figure 8 is

developed under C++.NET 2008. After the software
finished, the testing will extended using black box model.
This testing is aim to the check the expected output by
give certain input.
IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.10, October 2008 175

4.2 Experimental Setup The results of image processing algorithm described in the
following section.
The complete design of the intelligent real time inspection
system is shown in figure 9. The system is divided into Filtering
hardware and software frameworks [13]. The hardware
framework is the production line simulation which The results of images part after applied wiener filtering are
consists of conveyor, press parts, CMOS web-camera, shown in figure 10.
lighting system, fuzzy logic controller, and handling
equipment or sorter. Assume that conveyor has direction
from the left to the right. While part is moving over
conveyor, camera will capture image part and transfer it
into computer via USB port. The image then processed
and analyzed by the software. If the inspection result
reports that the quality doesn’t fits its requirement and
standard, then the fuzzy logic controller will receive (a)
warning signal from computer and trigger the handling
equipment will dissociate the part from the production line.

(b) (c)
Fig. 10 Wiener filtering result.

Edge Detection
Now, we have better image from noise reduction step.
This step is aim to detect edge or sharply changes of
brightness in image.
Sobel Operator
The edge detected by Sobel operator is shown in figure 11.

Fig. 9 AVI system for press part sorting.

The software framework is using three-tiered model. It

consists of three tiers which are data, logic, and user
interface which can be implemented on a single machine
or client/server based machine. This approach is important (a)
since the quality control, product prototyping, and
maintenance management system is integrated in a
computer-based system. In the real manufacturing
environment, it is necessary to transfer data or information
from one computer into others at different location. For
example, the inspection report should be transfer to the (b) (c)
engineers and machinists that perform the prototyping
Fig. 11 Threshold 0.11, 0.15, 0.12 respectively for (a), (b), and (c).
product components for feedback and to the maintenance
management system to give actual information of The rectangular image is edge detection with threshold
manufacturing process. 0.11. In the circle image, both inner and outer diameters of
the circle are detected by set the threshold value equal as
0.15. In the hexagonal image, the features are detected by
5. Result and Discussion set the threshold value equal to 0.12.
The system has been tested with environment that
simulates the manufacturing process as closely as possible.
176 IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.10, October 2008

Roberts Operator diameter, then the part should be dissociated before

entering the assembly process. For instead, we use if-then
The edge detected by Roberts operator is shown in figure rule to classify accepted and rejected part. Rule-based
12. systems classify data to if-then rules. This method use
causality to perform classification of object. For example
shown in the following rule:
IF feature diameter < 95 and > 85 THEN part is rejected
Otherwise, part is accepted
This rule would reject hexagonal part if the diameter is not
(a) in the range of 85-95 mm. The software algorithm
developed and simulated using MATLAB.

(b) (c)

Fig. 12 Threshold 0.11, 0.10, 0.10 respectively for (a), (b), and (c).

The rectangular image is edge detection with threshold

0.11. In the circle image, both inner and outer diameters of
the circle are detected by set the threshold value equal as Fig. 14 Feature extraction of rectangular part.
0.10. The same goes for hexagonal image. The image of rectangular type is decomposed into region
Canny Operator by applying Canny operator. The line inside region shows
the longest path of area interest. The value is the number
Canny operation shows better result which is showed in of the pixel that builds the line. Figure 14 shows the
figure 13. Canny is more precise than others such as Sobel values of feature parameters are 389.83 pixels for length
and Roberts operators. The rectangular image is edge and 211.00 pixels for width. The diameter of four holes is
detection with threshold 0.35. In the circle image, both 40.25 pixels for right hole, 19.75 pixels for centre hole,
inner and outer diameters of the circle are detected by set 32.25 pixels for upper-left hole and 33.00 for lower-left
the threshold value equal as 0.90. In the hexagonal image, hole. The dimension of length, width and circle of the part
the features are detected by set the threshold value equal to can be obtained by multiplication between the value and
0.60. constant that related to the position of camera and part.
The same thing can be applied for the circle part.

(a)

Fig. 15 Feature extraction of circle part.

(b) (c) Figure 15 shows the value of feature parameters are

174.00 pixels for outer diameter and 67.50 pixels for inner
Fig. 13 Threshold 0.35, 0.90, 0.60 respectively for (a), (b), and (c). diameter.
Feature Analysis
In classification, the feature descriptor is compared with
its requirement and standard. The comparison is based on
the tolerance of part design. If the result shows that
significant or intolerable error such as out of length and
IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.10, October 2008 177

References
[1] Anton Satria Prabuwono, Riza Sulaiman, Abdul Razak
Hamdan and Hasniaty A., Development of Intelligent
Visual Inspection System (IVIS) for Bottling Machine, Proc.
of IEEE Tencon, 2006, ISBN 1-4244-0549-1.
[2] Khan, U.S., Iqbal, J. and Khan, M.A., Automated Inspection
System using Machine Vision, IEEE Proc. of the 34 AIPR
Fig. 16 Feature extraction of hexagonal part. Workshop, 2005.
[3] http://www.precarn.ca/intelligentsystems/details
Figure 16 shows the value of feature parameters are 53.75 [4] http://www.spie.org/web/meetings/calls/pw01/confs/EI11.ht
pixels for inner diameter and 56.15, 62.19, 58.05, 54.99, m1.
57.03, 59.60 pixels respectively for feature A, B, C, D, E, [5] Bowskill J., Katz T. and Downie J., Solder Inspection using
an Object-Oriented Approach to Machine Vision, SPIE
and F.
Proc. of Machine Vision Application in Industrial
The dimensions obtained from pixel measurement then Inspection III, 1995, pp. 34-35.
converted into length of mm. The result then compared [6] Newman, T.S. and Jain, A.K., A Survey of Automated
Visual Inspection, Computer Vision and Image
with the actual value to get the error of measurement. If
Understanding 61 (2), 1995, pp. 231-262.
the error is tolerable under the standard and part
[7] Pham, D.T. and Alcock, R.J., Smart Inspection System,
requirements then the part will be accepted. Otherwise, Academic Press, California, 2003, pp. 6.
handling equipment will dissociate part from production [8] Polling, C., Designing a Machine Vision System, spie’s
line simulation. oemagazine, May 2002, pp. 34.
[9] http://www.mintron.com
[10] Chen, J., Benesty, J., Huang, Y. and Doclo, S., New
6. Conclusion and Future Work Insights Into the Noise Reduction Wiener Filter, IEEE
Trans. on Audio, Speech, and Language Processing, Vol.
We had compared both of inspection method in industrial 14, No. 4, 2006, pp. 1218-1234.
application. The real time inspection method can be [11] Budgen, D., Software Design, Pearson Education Limited,
expected to improve quality control in manufacturing Edinburgh, 2003, pp. 47-48.
environment. However, the implementation of computer [12] McMahon, D., Rapid Application Development with Visual
C++, McGraw-Hill, USA, 2000, pp. 55-58.
vision in manufacturing is not simply image processing
[13] Akbar, H. and Prabuwono, A. S., The Design and
problem. The real time visual inspection is an integration Development of AVI System for Press Part Sorting, Proc. of
system of lighting system, image acquisition, computer, IEEE ICCSIT, 2008, pp. 683-686.
controller and handling equipment. The results show that
the dimension can be calculated using image processing Habibullah Akbar received the
algorithm which measures the length, width, edge, and B.Sc. in Physics from Institut
diameter of press part. Teknologi Bandung in 2006. He is a
master student and a research
The results show the potential of the system to be assistant at Faculty of ICT,
implemented in metal-based industry. It is shown that the Technical University of Malaysia
above algorithm can be used to perform real time visual Melaka. His research interests
inspection system. The routine consists of digital image include computer vision, robotics,
acquisition, noise reduction, edge detection, and feature instrumentation, electronics, and
extraction. However, future work is necessarily needed to material physics.
inspect another parameter of press part such as
Anton Satria Prabuwono received
straightness, flatness, roundness, angle, profile, and weight the B.Eng. in Electrical Engineering
for part with more complex shape. from National Institute of Technology
Yogyakarta in 1995 and B.Sc. in
Computer Science from Padjadjaran
University Bandung in 2000. He
Acknowledgments
earned his Ph.D. in Industrial
Computing from Universiti
The authors would like to thank The Ministry of Higher Kebangsaan Malaysia in 2006. He is
Education Malaysia for the financial support under a senior lecturer at Faculty of ICT,
Fundamental Research Grant Scheme Technical University of Malaysia Melaka. He has over 6 years
No. FRGS/2007/FTMK (5) – F0052. industrial experiences in Bumi Kaya Steel, Samsung Electronics,
and Coca-Cola Bottling. His research interests include industrial
automation, robotics, and machine vision.