International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 08 Issue: 09 | Sep 2021 www.irjet.net p-ISSN: 2395-0072

A Comprehensive Survey of Defect Detection in Manufacturing

Products using Deep Learning Techniques
Ayushman Durgapal1
1B.Tech, Mechanical Engineering, Sardar Vallabhbhai National Institute of Technology, Surat, Gujarat, India
---------------------------------------------------------------------***----------------------------------------------------------------------
Abstract - To improve manufacturing of quality products costs, enhance efficiency, and enhance product quality,
and decrease production cost, image processing and while also laying a strong foundation for the industry's
machine learning based techniques are widely used now-a- intelligent transition.
days. Advances in Artificial Intelligence technologies which As a result, numerous academics have evaluated defect-
provides advanced tools to inspect products, analyzed the detection-related technologies and applications in order to
quality of products, helps us to improved manufacturing give references for defect-detection technology application
quality and quantity. However, to improve quality of and study. In the industrial manufacturing line, quality
products, proper inspection of manufacturing defects is control is critical. To measure the quality of a product or
crucial. Therefore, in this study, we discuss the existing the output of a process, several techniques are being
image processing, computer vision and machine learning employed. Quality control techniques can be classed as
techniques that are often used to detect defects in the destructive or non-destructive, depending on the approach
products and help to inspects them effectively. Deep used to discover a fault on a surface/volume, as illustrated
learning based state-of-the-art studies are presented and in Figure1.
compared. We also highlighted the benefits and limitation
of existing algorithms which may help researchers to find a
better solution such challenges. Finally, paper summarize
the study by discussing most efficient approach for defect
detection in manufacturing process.

Key Words: Quality control, defect detection, Object

Recognition, Deep Learning, Image Processing and
Computer Vision.

1.INTRODUCTION

Defect detection and classification are two issues that

must be approached as separate challenges within the
artificial vision area. The majority of digital image
processing issues arise from unique situations in which
researchers are attempting to imitate or replace human
eyesight and decision-making processes with artificial
techniques. Internal holes [1], scratches [2] etc., and occur Figure 1: strategies of quality control
in the creation of mechanical goods in complicated
industrial processes owing to design and machine One of the most popular processes in business is the
production equipment failures, as well as adverse working visual-based approach for fault identification. Traditional
circumstances. Because of their frequent usage, products visual examination, on the other hand, is a non-measurable
can quickly corrode [3] and get fatigued [4]. These flaws procedure with varied and subjective results. Because of
raise business expenses, reduce the useful life of produced the complexity and uniqueness of each problem to
items, and waste a lot of resources, all while endangering address, researchers have been forced to design new
people's health and safety. As a result, identifying flaws is a automatic flaw detection systems with stringent
critical skill that businesses should have in order to requirements. Such a system, however, is dependent on
enhance the quality of manufactured goods without the material characteristics of the surfaces to be monitored
impacting productivity. The advantages of automatic flaw as well as the surrounding circumstances. Indeed, due to
identification over manual detection are clear. It not only dusty or resonant working locations, the environment
adjusts to an inappropriate environment, but also operates hampers any implementation in industrial applications.
with great accuracy and efficiency in the long term. Fault The process of describing a fault and categorizing it
detection technology research may lower production entails a succession of subjective choices. Because the size

© 2021, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 509
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 08 Issue: 09 | Sep 2021 www.irjet.net p-ISSN: 2395-0072

of flaws varies between industrial applications, the major which systems detect a defects and classification of defects
features of a defect are determined by the detection are directly influenced by the quality of the image
procedure's desired accuracy and resolution. Before processing method [7, 8].
building and implementing an automated system in any
Traditional defect-detection approaches, as well as
industrial quality control application, it is strongly
well-known fault or defect detection using deep learning
recommended to create a product quality standard.
algorithms, clearly have benefits. Classical defect detection
2. EXISTING TECHNIQUES FOR DEFECT methods are quite specific. For example, Osmosis testing
DETECTION technique is only appropriate for identifying flaws in
highly permeable and non-porous materials, although it
Defect detection technology is primarily used to detect has certain benefits over other approaches. However, most
fault or defect on the product surface and interior flaws. On classic detection techniques still require manual help to
the product surface, defect-detection technology relates to complete, especially when instrument debugging is
the identification of spots, pits, scratches, color variations, necessary prior to testing, and equipment development
and defects. An example of defects in the products is shown costs are significant, making it difficult to adapt and
in Figure 2. restricted by equipment life and manufacturing precision.
Due to their adaptability and lack of reliance on human
help, innovative defect-detection approaches, notably
machine vision and deep-learning methods have become
the most popular in recent years and are one of the main
technologies for automating defect identification. The new
technologies offer better inspection results and cheaper
costs than classic defect detection approaches, but they
still rely on huge volumes of learning data to drive model
updates and increase inspection accuracy.
3. DETECT DETECTION USING DEEP LEARNING
Deep-learning technology has advanced quickly and
achieved tremendous success in a variety of fields of study.
A deep neural network structure with many convolutions
layers is used in deep learning. The data may be better
reached in abstract ways such as edge and shape to
increase the efficacy of the deep-learning algorithm [9], by
combining low-level characteristics to produce a more
Figure 2: Defects in different areas of products abstract high-level representation of attribute categories
or features. As a result, several researchers are attempting
The magnetic powder is mixed with water, oil, or other to employ deep-learning technologies to detect product
liquid media in wet magnetic particle detection. Magnetic defects and enhance product quality [10, 11].
powder can recognize the exact site of defect with the help
of liquid pressure and external magnetic field [5]. 3.1 Defect detection using CNN [12]
The angle between the fault surface and the ultrasonic CNN is a feedforward neural network that stands for
propagation direction affects the ultrasonic testing impact "convolutional neural network." A CNN is made up of one
[6]. The signal returned is strong if the angle is vertical, or more convolutional and fully connected layers, as well
and the fault is quickly recognized. While, the signal as weights and pooling layers. Literature [13] is a popular
returned is faint if the angle is horizontal, making it simple LeNet convolution neural network topology. There are two
to identify a leak. As a result, in order to decrease leakage ways to use the LeNet network structure to detect defects
detection, the suitable detection sensitivity and associated in images: one is to design a complex multi-layer CNN
probe must be used. structure, use different network structures to add image
Image capture, defect detection, and classification are content features, and complete end-to-end training to
the primary components of machine vision detection. detect defects in images; the other is to combine CNN with
Machine vision is frequently utilized due to its quick, CRF model, train CNN with CRF energy function as
accurate, non-destructive, and low-cost features. The color, constraint, or optimize network prediction results with
texture, and geometric characteristics of things are used by CRF. Detection of product flaws is also a goal.
machine vision to identify them. The difficulty of image
processing is determined by the quality of picture capture.
In turn, the performance such as accuracy the rate by

© 2021, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 510
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 08 Issue: 09 | Sep 2021 www.irjet.net p-ISSN: 2395-0072

3.2 Neural Network based Defect-detection [14] and may increase accuracy by increasing network depth.
With the depth of the network increasing, the extraction
The most crucial steps of an autoencoder network are feature rises, but it is simple to cause the activation
coding and decoding. In the coding step, the input signal is function not to converge. CNN, Generative Adversarial
transformed into a coding signal for feature extraction; in Networks, etc. The objective of the deep residual network
the decoding stage, the feature information is transformed is to minimize the loss by optimizing the number of
into a reconstruction signal, and the reconstruction error network layers while increasing the network structure,
is reduced by modifying the weight and bias to accomplish such that the output and input element dimensions of the
defect detection [15]. The distinction between convolution layer in the residual unit are the same. The
autoencoder networks and other machine learning residual unit of ResNet is shown in Figure 4.
methods is that the autoencoder network's learning aim is
feature learning rather than classification. It also has a
significant ability to learn on its own and is capable of
extremely nonlinear mapping. It can learn nonlinear
metric functions to tackle the problem of complicated
background and foreground region segmentation. In the
subject of computer vision, particularly in picture
segmentation, substantial progress has been made. The
development of deep CNNs has resulted in significant gains
in a variety of image processing jobs. A basic overview of
CNNs is provided in this section.
In CNN, the input passes through a series of processing
processes, referred to as layers, in order to be processed.
Each layer I may be thought of as an arbitrary
transformation xi+1 = f(xi; θi) with inputs xi, outputs xi+1, Figure 4: Residual unit of deep model
and parameters I with inputs xi, outputs xi+1, and
parameters θi. The output is referred as feature map and ResNet is trained to detect defect based on regions
can be presented as shown below. after training it with COCO dataset and Microsoft Common
Objects in Context dataset. For a given input image, top 50
defect regions is found and shown in Figure 5.
Figure 3 schematically depicts the convolution of a H x
W x 1 kernel with an image, where H and W denoted the
height and width of kernel. It is feasible to produce
meaningful outputs, such as picture gradients, by
convolving specific types of kernels with the input image.
The first few convolutional layers in most contemporary
CNN designs extract characteristics like edges and
textures. Deeper convolutional layers in the network may
extract information like object forms that cover a larger
spatial region of the picture.

Figure 3: Features map generation using CNN. Figure 5: Detected top 50 defect regions

From the results, it is found that Residual Deep Neural

3.3 ResNet based Defect-detection [16]
network a segment image to find defect with and accuracy
of 93 %.
The architecture of residual model is similar to the
convolutional neural network except it add few residual
modules on the top of stacked convolutional operations.
The residual network has a simple optimization process
© 2021, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 511
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 08 Issue: 09 | Sep 2021 www.irjet.net p-ISSN: 2395-0072

3.4 Full Convolution Neural Network

A link between any two nodes between two adjacent

levels is referred to as a completely connected layer. There
will be more weight values in a fully connected neural
network since it uses a fully linked operation, indicating
Experiments with several hyperparameters revealed
that the network will demand more memory and
that the optimum hyperparameters have an accuracy of
processing. During the calculation of the fully connected
82.58 %.
neural network, the feature map generated by the
convolution layer is mapped into a fixed-length feature
vector. The complete convolution neural network can take
any size input picture, and by sampling the feature map of
the last convolution layer with the deconvolution layer, it
can recover to the same size as the original image. such
that a forecast may be made for each pixel while keeping
the spatial information in the original input picture, and
then categorize the upper sampling feature map pixel by
pixel.
3.5 Recurrent Neural Network [17]

The recurrent neural network originates from the

evolution direction of sequence data, with all cyclic units Figure 7: Proposed framework of deep model
connected in a chain form, as the input. Convolution and
pooling procedures are used by the CNN model to extract 3.6 Inverted ResNet and Hough transform based
feature information from input layer test samples. The defect-detection [18]
recurrent neural network replaces the CNN's convolution
process with the recurrent convolution operation. An effective product inspection approach is suggested
in this work in order to achieve a compromise between
To find a crack in Prestressed concrete (PSC) box-
recognition accuracy and overall model complexity. Figure
girder, in [17], a Recurrent Neural Network is trained. A
4 depicts the complete picture processing and
RNN is a type of artificial neural network in which hidden
identification pipeline. Product images could be recorded
nodes are linked together by a directed edge to form a
and sent to the server as the conveyer belt went along,
cyclic structure. This paradigm is well-known for its ability
allowing for additional analysis. Defective goods may be
to handle sequential data like text, sounds, and signals. The
recognized using the suggested technique in three stages:
RNN's most important property, as seen in Figure 6.
image preprocessing, ROI extraction, and picture
identification.
Pre-processing: Proposed method pre-process input image
first by smoothing it using Gaussian blur. The filter size
used to remove noise using Gaussian function is 3x3. Once
images are pre-processed, region of interest (ROI) are
Figure 6: Architecture of RNN to detect defects. extracted from it.
ROI Extraction: Since extraneous background content may
The SoftMax function is used to categorize the class of comprise a significant portion of the image, the collected
input data in artificial neural networks. The SoftMax images may not ensure that all included information is
function classifies a given sample by using the output layer task-related, resulting in needless computations during the
to normalize it. This function, for example, computes the next identification stage. In practice, a lighting source is
likelihood of being positive by inputting data from the used to highlight defective items and guarantee that the
hidden layer while evaluating if a cancer is benign (1) or collected photos are consistent. Fortunately, such a
malignant (0). The SoftMax function used here is shown in lighting source is generally rectangular or circular in form.
equation below: As a consequence, the probabilistic Hough transform is
used to swiftly determine the line or circle of the
illumination source edge, which can subsequently be used
to extract ROI from the filtered image.

© 2021, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 512
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 08 Issue: 09 | Sep 2021 www.irjet.net p-ISSN: 2395-0072

Identification of Images: The goal of this stage is to identify 5. CONCLUSIONS

the extracted ROI's detailed class. The picture identification
module is built by cascading inverted residual blocks to Industrial product quality is a key aspect of
achieve excellent classification accuracy and speed. manufacturing, and research into defect-detection
Performance of model is evaluated by calculating accuracy technology is crucial for ensuring product quality. This
and recall as shown in following equation. article gives a detailed review of product defect detection
technologies in complicated industrial processes, as well
as the current state of research. The experimental findings
of fault detection strategies were extensively presented,
and traditional defect-detection methods and deep-
learning defect-detection approaches were contrasted and
assessed. Meanwhile, defect-detection equipment was
Where Ti and Fi presents truly classified and false studied and assessed in conjunction with actual
classified samples. From the results, it is found that application requirements and the development of artificial
proposed method can detect defect with 99.60% accuracy. intelligence technology.
According to the findings, all of the tested techniques We hope that the study will aid industrial
employ well-preprocessed pictures. As a result, most deep businesses and researchers in understanding product
learning algorithms, except than MLP, require greater defect-detection technology research development in the
computer resources to achieve higher inspection accuracy. fields of deep learning and defect detection.
MLP is the quickest, but it's also the least accurate, as it
misses spatial information in the picture. Meanwhile, 6. REFERENCES
because MLP is parameter-intensive, it requires more
storage capacity to cache and store the learned MLP. [1] Peng, Y.; Liu, G.; Quan, Y.; Zeng, Q. The depth
measurement of internal defect based on laser
4. DISCUSSION speckle shearing interference. Opt. Laser Technol.
In the past few years numerous image processing and 2017, 92, 69–73.
machine learning based algorithms has been proposed to [2] Chen, J.; Li, C. Prediction and Control of Thermal
detect defects in the products manufacturing. Few Scratch Defect on Surface of Strip in Tandem Cold
approaches work well and produced acceptable accuracy. Rolling. J. Iron Steel Res. Int. 2015, 22, 106–114
Texture based approach, fusion of texture with other [3] Rodionova, I.G.; Zaitsev, A.I.; Baklanova, O.N.;
features. Local binary pattern features and GLCM features Kazankov, A.Y.; Naumenko, V.V.; Semernin, G.V. Effect
based defect detection approaches are one of them. of carbon steel structural inhomogeneity on
However, these approaches extract manual features which corrosion resistance in chlorine-containing media.
may not include unique pattern. Therefore, deep learning Metallurgist 2016, 59, 774–783.
takes advantages over traditional algorithms for defect [4] Amaya, J.; Lelah, A.; Zwolinski, P. Design for
detection. intensified use in product–service systems using life-
cycle analysis. J. Eng. Des. 2014, 25, 280–302.
Due to automatic feature learning, NN is widely used [5] Elrefai, A.L.; Sasada, I. Magnetic particle detection
for many detection and recognition problems. For detect system using fluxgate gradiometer on a permalloy
defection, a number of studies have been done using deep shielding disk. IEEE Magn. Lett. 2016, 7, 1–4.
learning and discussed in this paper. CNN performs well [6] Kusano, M.; Hatano, H.; Watanabe, M.; Takekawa, S.;
but it requires a lot of training times and processor. Yamawaki, H.; Oguchi, K.; Enoki, M. Mid-infrared
Moreover, as the depth of convolutional layer increases, pulsed laser ultrasonic testing for carbon fiber
training time also decreases. To overcome this limitation, a reinforced plastics. Ultrasonics 2018, 84, 310–318.
residual block based deep model which is also named as [7] Aytekin, Ç.; Rezaeitabar, Y.; Dogru, S.; Ulusoy, I.
ResNet, is proposed. ResNet works well and produced Railway fastener inspection by real-time machine
significant improvement in the performance. It also vision. IEEE Trans. Syst. Man Cybern. Syst. 2015, 45,
required less training time even with large depth of 1101–1107.
convolutional layers. However, it may not be suitable for [8] Lin, H.; Chen, H. Automated visual fault inspection of
time series data. Therefore, recurrent neural network optical elements using machine vision technologies.
based deep model architecture is proposed. Recently, a J. Appl. Eng. Sci. 2018, 16, 447–453.
inverted residual block based deep architectures along [9] Deng, L.; Yu, D. Deep learning: Methods and
with Hough transform features is proposed to detect applications. Found. Trends Signal Process. 2014, 7,
defects in the manufacturing products. From the results it 197–387.
can be concluded that the proposed system works well [10] Tao, X.; Wang, Z.; Zhang, Z.; Zhang, D.; Xu, D.; Gong, X.;
with real-time data and produces excellent results. Zhang, L. Wire Defect Recognition of Spring-Wire
Socket Using Multitask Convolutional Neural

© 2021, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 513
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 08 Issue: 09 | Sep 2021 www.irjet.net p-ISSN: 2395-0072

Networks. IEEE Trans. Compon. Packag. Manuf.

Technol. 2018, 8, 689–698.
[11] Jiang, J.; Chen, Z.; He, K. A feature-based method of
rapidly detecting global exact symmetries in CAD
models. Comput. Aided. Des. 2013, 45, 1081–1094.
[12] Yang, J.; Yang, G. Modified Convolutional Neural
Network Based on Dropout and the Stochastic
Gradient Descent Optimizer. Algorithms 2018, 11,
28.
[13] LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-
based learning applied to document recognition.
Proc. IEEE 1998, 86, 2278–2324.
[14] Bergmann, P.; Löwe, S.; Fauser, M.; Sattlegger, D.;
Steger, C. Improving Unsupervised Defect
Segmentation by Applying Structural Similarity to
Autoencoders. arXiv 2018, arXiv:1807.02011.
[15] Ozkan, S.; Kaya, B.; Akar, G.B. Endnet Sparse
autoencoder network for endmember extraction and
hyperspectral unmixing. IEEE Trans. Geosci. Remote
Sens. 2018, 57, 482–496.
[16] Yu, L.; Chen, H.; Dou, Q.; Qin, J.; Heng, P.A. Automated
melanoma recognition in dermoscopy images via
very deep residual networks. IEEE Trans. Med.
Imaging 2016, 36, 994–1004.
[17] Lei, J.; Gao, X.; Feng, Z.; Qiu, H.; Song, M. Scale
insensitive and focus driven mobile screen defect
detection in industry. Neurocomputing 2018, 294,
72–81.
[18] Wang, J., Fu, P., & Gao, R. X. (2019). Machine vision
intelligence for product defect inspection based on
deep learning and Hough transform. Journal of
Manufacturing Systems, 51, 52-60.

© 2021, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 514