Vol. 01, No. 04 (2023) 161-166, doi: 10.61552/JIBI.2023.04.001 - http://jibi.aspur.

rs

QUALITY CONTROL IN THE MANUFACTURING INDUSTRY

BASED ON THE APPLICATION OF COMPUTER VISION

Đorđe Mijailović Received 19.05.2023.

Aleksandar Đorđević 1 Accepted 28.08.2023.
Miladin Stefanović
Milan Erić

ABSTRACT
Keywords:
Application of technologies based on information communication
Industry 4.0., Quality 4.0.,Cloud, systems, internet networks, embedded systems, data storage in the
OpenCV, Computer Vision, CNN, cloud, computer vision, machine learning within manufacturing
SSIM, IOT industry leads to a number of advantages in the quality control
process. The goal of the work is the development of the system for
quality control within production industry with the application of
computer vision and algorithms for comparison and analysis. The
secondary goal of the work refers to the storage of data on product
quality and the control of production machines based on the analyzed
data. The result of the application of the mentioned system is reflected
in the identification of non-compliant products, which will lead to a
reduction of the company's losses.
© 2023 Journal of Innovations in Business and Industry

1. INTRODUCTION conformity, algorithmic systems (Rožanec et al., 2022,

Louw et al. , 2019, Yang et al., 2020).
Product quality control within the manufacturing This technology also has limitations, so it can be used to
industry is one of the most important indicators of detect text, color, dimensions, and scratches. The
success companies. The very development of accuracy of the obtained data depends on several
technology and its use within the industry leads to a factors, the most important of which relates to the
greater number of detection methods non-conformities algorithm itself used to detect inconsistencies, external
and quality control. As the area of the manufacturing factors such as lighting, vibrations, equipment quality,
industry itself differs from company to company, as while the speed of the system is directly related to the
well as the non-conformity that can be presented in hardware. The advantage of applying computer vision
several ways, several methods of quality control can be compared to manual methods of quality control are
applied. One of the ways of quality control is computer certainly based on speed, timely information in real
vision, which has a huge advantage over manual ways time, smaller losses, reduced costs of finishing products
of quality control in terms of the adaptability of the or semi-products, reduced production of scrap products,
system to a specific purpose, the application of deep which directly affects company profits (Yang et al.,
learning in the process of making a decision on non- 2020).

1
Corresponding author: Aleksandar Djordjevic
Email: [email protected] 161
 Quality Control in The Manufacturing Industry Based on The Application of Computer Vision

In addition to the large number of systems based on the 2.2. Computer vision
application of computer vision for quality control, this
area is still under-researched, it is necessary to strive for Computer vision as the basis of the system presented in
the development of less complex systems that will this work is based on the acquisition, processing and
require less software and hardware power and that will creation of data. The technological pillar of computer
achieve results at the same speed. Accordingly, there is vision is based on the use of a camera that is applied for
a need for the development of this system that will the data acquisition process (Stavropoulos et al., 2020;),
contribute to the efficiency and speed of product non- and it can be applied industrial dedicated cameras as well
conformity detection. The main goal of the system as general purpose cameras. When it comes to data
developed and presented in this work is the processing itself, it can be performed in several ways, the
development of a quality control system based on most common used by industrial computers while they
computer vision, where the focus is on external non- are all more commonly used general purpose computers
conformities of products within the manufacturing viewed from the aspect of finance (Stavropoulos et al.,
industry, where the amount of funds needed for its 2020;). The basis computer vision consists of algorithms
implementation should be taken into account. The applied for object tracking, dimensional control, object
advantages of using this system can be seen in the selection, object recognition, etc. (Stavropoulos et al.,
training of the system where convolutional neural 2020;). The complexity of the algorithm directly affects
networks were used, the use of which significantly the speed of the system, which is of high importance in
accelerated the training, which makes the system much some application cases. In order to increase the speed,
more adaptable to needs. they are often used and neural networks (Almazán-
In the continuation of the work, a review of the Lázaro et al., 2022;). In order to achieve satisfactory
literature will be presented, which will include the system performance, which is reflected in accuracy,
current technology that has been applied in the field of speed, and the necessary financial resources, it is
product quality with the use of computer vision. necessary to strive to reduce
Application of Industry 4.0. as well as the very complexity of the algorithm and at the same time
technologies it includes for solving problems in the maintain or increase its accuracy (Almazán-Lázaro et al.,
quality control process. 2022;). In order to achieve satisfactory system
performance, which is reflected in accuracy, speed, and
the necessary financial resources, it is necessary to strive
2. LITERATURE REVIEW to reduce the complexity of the algorithm and at the
same time maintain or increase its accuracy (Almazán-
2.1 Quality control in manufacturing industry Lázaro et al., 2022;). Also, the very functionality of the
computer vision system can be affected by external
Quality control within the framework production factors such as lighting, image resolution, vibrations,
industry can be seen as a process that consists of two distance, etc., in order to increase accuracy of these
sub-processes that make up the identification of quality systems, it is necessary to strive to reduce the influence
and quality comparison (Powell et al., 2021;). The of the mentioned factors. Today, computer vision finds
quality identification process can be seen as the process application in many industrial fields, and its
of determining the current quality of the product by contributions in the application of control quality can be
various measurement methods, the comparison process observed through the reduction of scrap, reduction of the
involves comparing the identified quality with the number of finished products, controlled production,
results that need to be achieved (Powell et al., 2021;). implementation on industrial robots where it increases
Quality of the product is closely related to the user of the precision of the robot's gait as well as the download
the product or service, which is actually a determinant position, recognition of different situation, product
of the level of required and achieved product quality palletization, control of industrial lines, reduction of the
(Rodrigues et al., 2020; Powell et al., 2021;). Seen from required number of employees in quality control
the side of the customer or user of the product on alone positions (Vukicevic et al., 2019).
Product quality is affected by a number of factors of the
same and which are often definedstandard or user 2.3. Industry 4.0.
requirements (Carvalho et al., 2021; Rodrigues et al.,
2020 ;). Also, the time of use of a product can define the Application of Industry 4.0. in the quality control
necessary quality when it is used in industry or when it process, it enables the implementation of modern
is used for general use. Quality can be expressed in a technologies that directly affect the quality control
number of ways, the basic way of expressing quality is process itself (Culot et al., 2020). The main goal of the
numerically, that is, when talking about some numerical application of Industry 4.0. se based on easier and faster
values, it can also be expressed in nominal scales, execution industrial processes, obtaining more accurate
descriptively, yes or no. data, reducing the possibility of errors, all with a
combination of the application of already implemented
technology and new ones (Bigliardi et al., 2020; Horváth
et al., 2019). One of the indicators of success

162
 Journal of Innovations in Business and Industry, Vol. 01, No. 04 (2023) 161-166, doi: 10.61552/JIBI.2023.04.001

applications of industry 4.0. is also reflected in process detection algorithms, the system's reaction is the process
control, adaptability system, availability (Albers et al., of stopping further production if the defined number of
2016). The application of internet networks, sensor non-conformities is exceeded products, data storage is
systems that can independently communicate and collect done in real time on the cloud system.
data is called IIOT (Industrial Internet of things), also
this technology includes a database as well as control 3.1. System software
industrial machines (Bal et al., 2019; Muller 2019; Tupa
et al., 2017, Singh et al., 2019). Seen from the side of The work process of the presented system can be
software that is applied within Industry 4.0. and observed in the following way, when starting the
depending on herself there are a number of industrial system, it is necessary to initially train the system, that
areas software systems, tools that lead to cost reduction, is, to define the non-conformity detection classes. The
faster work, increase efficiency and effectiveness as well user of the system performs the training by placing
as loss reduction (Mijailović et al., 2020). From the images of non-compliant products in the database from
hardware perspective of Industry 4.0. implies the where the system will use these images as reference
application of microcontroller platforms, programmable images, that is, images with which it will compare new
logic controller, which together implemented in the images downloaded from the production machine. After
quality control process and within one machine represent training, the system can be put into operation, training
embedded systems (Pasika et al., 2020; Izaddoost et al., must be performed every time the product is changed.
2020; Kullig et al., 2020). Big data provides the ability to Based on training and neural networks as well as a
store large amounts of data that can be analyzed further comparison algorithm, the system recognizes non-
to help predict future system failure. With the use of compliance, prints the result of non-compliance in
industrial cameras, it is possible to obtain quality data, percentages and stores the data on the cloud. Within the
which further, with the use of deep learning, can lead to aforementioned system, the SSIM (Structural Similarity
a reduction in the possibility of errors. (Maurizio et al, Index) algorithm was used, which compares pixel by
2021). pixel the reference image and the image currently
collected by this system, calculates the squared error
that is created and caused by different pixels. On the
3. CASE STUDY very result of algorithm and the accuracy of the system
can be affected more factor, the position of the camera
Within this chapter it will be presented a system that must always be in the same location in relation to the
was developed for visual quality control in the product, lighting can affect the quality itself photos and
production industry of small and medium-sized thus the result algorithm, if it is necessary to add the
companies with the use of opencv library, computer source of illumination must be addressed attention to the
vision, cloud system, CNN (Convolutional neural product itself, must not come to glare or shadows,
network). Since quality control is performed manually vibration is essential prevent them from affecting the
in most small and medium-sized companies, the goal of movement cameras. In Figure 1, you can see an
this paper is the development of an adaptable system for example of an image of a compliant product that was
application in the field of visual quality control. Current used to test the operation of the algorithm, all tests of
quality control process: an employee from the quality the algorithm were performed within the premises
department supervises the production as well as the university, laboratory center for quality.
products themselves, controlling the essential
characteristics of the product for its functionality.
Which means that an employee periodically monitors a
production machine and the semi-finished products it
produces, as he needs a certain amount of time to check
all machines. According to the above, if after the
departure of the employee who will perform the quality
control process on a certain machine, it produces a non-
compliant product, it will continue to do so until he
reaches the same machine again. The presented problem
can be solved with the application of computer vision if
it is a surface one non-compliance. The working
principle of the presented system is as follows: based on
data acquisition, their processing is performed, after
which the system's reaction and data storage take place.
Acquisition process data processing is done with the
help of system hardware using a camera, data
processing in the specific case of photos is done with
the use of opencv libraries, TensorFlow library, CNN
(convolutional neural network) and non-conformity Figure 1. Example of a compliant product
Source: Author

163
 Quality Control in The Manufacturing Industry Based on The Application of Computer Vision

It can be seen in Figure 2 example of a compliant

product that does not have a non-conformity. After
placing one or more reference images, the user of the
system, as well as placing the hardware at the physical
location where the acquisition will be performed, can
start the system. The system can also be used to check
products for non-compliance or if already collected
images are used. In Figure 3, you can see the non-
conforming semi-product that was used in the
comparison process.
As can be seen in Figure 2. the product has damage
caused during the production process. When comparing
picture 2 with picture 1, the result of the algorithm is
0.9258, i.e 92.5% similarity. Figure 3 shows the result Figure 4. Data stored in the cloud
of the algorithm. Source: Author
As can be seen from the results of the algorithm, the
similarity of image 2 and image 3 is 92.5%, i.e. 7.5% We can additionally process and display data stored in
are non-compliant pixels in the specific case of non- the cloud in several ways, the advantage of cloud data is
compliance. After the collected and processed data is its availability.
stored on the cloud system in thingspeak cloud was used
within this system, an example of stored data collected
by this system can be seen in Figure 4. 4. DISCUSSION

After analyzing the work of the sector for quality in

small and medium-sized companies in Serbia, it was
concluded that the quality control process is still carried
out manually today mainly due to the amount of funds
needed to implement modern solutions. When it comes
to the funds that will be invested in the system, it is
necessary to reduce the error of the system by reducing
the price, to adjust system performance available
hardware, enable simple user interaction. An advantage
this system can be viewed from the aspect of
adaptability in terms of changing the image acquisition
location itself, even though it is embedded-implemented
in an already existing system or machine, it can be
moved to another location. On the other hand, the
disadvantages of the system are its sensitivity to
external factors that directly affect the quality system
operation. The implemented system is implemented on
one production machine, which means that it is
necessary on every machine implement the same system
where they would get a wide network of data and
control over the entire production of a company, which
is actually another goal. By reducing the occurrence of
non-compliant products or even their complete
elimination, the company's profit is directly affected, as
well as the efficiency and speed of production response
to market demand, quality increases production.

Figure 2. Non-compliant product

Source: Author 5. CONCLUSION

The initial goal of this study which refers to the

development of the control system product quality
Figure 3. Comparison result within production industry was successfully
Source: Author implemented, the results of the system are presented.
The presented system is a universal solution that can be
applied both on the production machines themselves

164
 Journal of Innovations in Business and Industry, Vol. 01, No. 04 (2023) 161-166, doi: 10.61552/JIBI.2023.04.001

and can be set up within the quality center for quality secondary goal starts from the fact that there are
control of several different products. Depending on the currently not enough experts in the field of quality
location of the system application, it is necessary to pay control, also the application of such systems reduces the
attention to the external factors that may affect accuracy need for experts and increases the availability data. The
the result of the system. Collected and presented data is limitations of this study are related to external factors
stored on cloud system in real time, as far as possible that can affect the accuracy of the system, one of the
allows viewing of data in each moment. If viewed from main factors is related to the position of the product that
the aspect networking of the entire production of one the camera will photograph, the product must always be
companies where on every production if one such photographed in the same position so that the data is as
system were implemented in each machine, they would accurate as possible. Given that the system is based on
have an overview of the current state of production, the application of the ssim algorithm, which is in
where they would have a defined number of non- conjunction with CNN, and functions in the manner
compliant products, good products and products that are presented in this study, it represents a unique solution
going to be finished. Also, by networking the machines, based on the goals of this study. Future directions of
we can obtain data on the total efficiency for each of the research will be based on the implementation of several
machines in production. Quality control represents one different algorithms in the process of detecting non-
of the most important processes in every production compliant products, which will further reduce the
company which is from of essential importance for the possibility of system errors, as well as reducing the
development of the company as well as the ultimate influence of external factors on the operation of the
profit of the company, hence the goal for the system.
development of the quality control system, the

References:

Albers, A., Gladysz, B., Pinner, T., Butenko, V., &Stürmlinge, T. (2016). Procedure for defining the system
of objectives in the initial phase of an industry 4.0 project focusing on intelligent quality control
systems. Procedia CIRP, 52, 262–267. DOI: 10.1016/j.procir.2016.07.067.
Almazán-Lázaro, J.-A., López-Alba, E., &Díaz-Garrido, F.-A. (2022). Applied computer vision for
composite material manufacturing by optimizing the impregnation velocity: An experimental
approach. Journal of Manufacturing Processes, 74, 52-62. DOI: 10.1016/j.jmapro.2021.11.063.
Bal, H. Ç., &Çisil, E. (2019). Industry 4.0 and competitiveness. Procedia Computer Science, 158, 625-631.
DOI: 10.1016/j.procs.2019.09.096.
Bigliardi, B., Bottani, E., & Casella, G. (2020). Enabling technologies, application areas and impact of
industry 4.0: a bibliographic analysis. Procedia Manufacturing, 42, 322-326. DOI:
10.1016/j.promfg.2020.02.086.
Carvalho, A. V., Enriquea, D. V., Chouchene, A., & Santos, F. C. (2021). Quality 4.0: An overview.
Procedia Computer Science, 181, 341-346. DOI: 10.1016/j.procs.2021.01.176.
Culot, G., Orzes, G., Sartor, M., &Nassimbeni, G. (2020). The future of manufacturing: A Delphi based
scenario analysis on Industry 4.0. Technological Forecasting and Social Change, 157, 120092. DOI:
10.1016/j.techfore.2020.120092.
Horváth, D., &Szabó, R. Z. (2019). Driving forces and barriers of Industry 4.0: Do multinational and small
and medium-sized companies have equal opportunities? Technological Forecasting and Social
Change, 146, 119-132. DOI: 10.1016/j.techfore.2019.05.021.
Izaddoost, A., &Siewierski, M. (2020). Energy Efficient Data Transmission in IoT Platforms. Procedia
Computer Science, 175, 387-394. DOI: 10.1016/j.procs.2020.07.055.
Kullig, N., Lämmel, P., &Tcholtchev, N. (2020). Prototype implementation and evaluation of a blockchain
component on IoT devices. Procedia Computer Science, 175, 379-386.
https://doi.org/10.1016/j.procs.2020.07.054
Mauricio-Andrés, Z.-H., Castro-Vargas, J. A., Azorin-Lopez, J., & Garcia-Rodriguez, J. (2021). Deep
learning-based visual control assistant for assembly in Industry 4.0. Computers in Industry, 131,
103485. https://doi.org/10.1016/j.compind.2021.103485
Louw, L., &Droomer, M. (2019). Development of a low cost machine vision based quality control system
for a learning factory. Procedia Manufacturing, 31, 264-269.
https://doi.org/10.1016/j.promfg.2019.03.042
Müller, M. J. (2019). Contributions of Industry 4.0 to quality management - A SCOR perspective. IFAC-
PapersOnLine, 52(13), 1236-1241. https://doi.org/10.1016/j.ifacol.2019.11.367

165
 Quality Control in The Manufacturing Industry Based on The Application of Computer Vision

Mijailović, Đ., Karabašević, D., &Stanujkić, D. (2018). Development of the system for monitoring ambiental
factors by applying arduino platform. Trendovi u poslovanju, 12, 37-45.
https://doi.org/10.5937/TrendPos1802037M
Mijailović, Đ., Đorđević, A., Stefanović, M., Vidojević, D., Gayiyulina, A., &Projović, D. (2021). A cloud-
based with microcontroller platforms system designed to educate students within digitalization and
the Industry 4.0 paradigm. Sustainability, 13(22), 12396. https://doi.org/10.3390/su132212396
Pasika, S., &Gandla, S. T. (2020). Smart water quality monitoring system with cost-effective using IoT.
Heliyon, 6(7), e04096. https://doi.org/10.1016/j.heliyon.2020.e04096
Powell, D., Eleftheriadis, R., &Myklebust, O. (2021). Digitally enhanced quality management for zero defect
manufacturing. Procedia CIRP, 104, 1351-1354. https://doi.org/10.1016/j.procir.2021.11.227
Rožеnac, J. M., Zajec, P., Trajkova, E., Šircelj, B., Brecelj, B., Novalija, I., Dam, P., Fortuna, B.,
&Mladenić, D. (2022). Towards a comprehensive visual quality inspection for Industry 4.0. IFAC-
PapersOnLine, 55(10), 690-695. https://doi.org/10.1016/j.ifacol.2022.09.486
Rodrigues, H., Silva, F. J. G., Morgado, L. G., Sá, J. C., Ferreira, L. P., &Campilho, R. D. S. G. (2020). A
novel computer application for scrap reporting and data management in the manufacturing of
components for the automotive industry. Procedia Manufacturing, 51, 1319-1326. doi:
10.1016/j.promfg.2020.10.184.
Singh, I., Centea, D., &Elbestawi, M. (2019). IoT, IIoT and Cyber-Physical Systems Integration in the SEPT
Learning Factory. Procedia Manufacturing, 31, 116-122. doi: 10.1016/j.promfg.2019.03.019.
Stavropoulos, P., Papacharalampopoulos, A., & Petridis, D. (2020). A vision-based system for real-time
defect detection: a rubber compound part case study. Procedia CIRP, 93, 1230-1235. doi:
10.1016/j.procir.2020.04.159.
Tupa, J., Simota, J., & Steiner, F. (2017). Aspects of Risk Management Implementation for Industry 4.0.
Procedia Manufacturing, 11, 1223-1230. doi: 10.1016/j.promfg.2017.07.248.
Vukicevic, A. M., Djapan, M., Todorovic, P., Erić, M., Stefanovic, M., &Macuzic, I. (2019). Decision
support system for dimensional inspection of extruded rubber profiles. IEEE Access, 7, 112605-
112616. doi: 10.1109/ACCESS.2019.2934561.
Yang, J., Li, S., Wang, Z., Dong, H., Wang, J., & Tang, S. (2020). Using deep learning to detect defects in
manufacturing: a comprehensive survey and current challenges. Materials, 13(24), 5755. doi:
10.3390/ma13245755.

Đorđe Mijailović Aleksandar Đorđević Miladin Stefanović

Faculty of Engineering, Faculty of Engineering, Faculty of Engineering,
Kragujevac, Kragujevac, Kragujevac,
Serbia, Serbia Serbia
[email protected] [email protected] [email protected]
ORCID: ORCID: 0000-0003-2856-6578 ORCID: 0000-0002-2681-0875

Milan Erić
Faculty of Engineering,
Kragujevac,
Serbia
[email protected]
ORCID: 0000-0001-8691-4863

166