BACHELOR OF ENGINEERING TECHNOLOGY (AUTOMATION AND ROBOTICS) WITH HONOURS

IA–B223–30
AUTOMATION DEFECT DETECTION SYSTEM

NUR SYAMIMI BINTI ABDUL RAIF

50222120091
2023

UNIVERSITI KUALA LUMPUR

OCTOBER 2023
 TITLE PAGE

AUTOMATION DEFECT DETECTION SYSTEM

NUR SYAMIMI BINTI ABDUL RAIF

50222120091

Report Submitted to Fulfill the Partial Requirements

For the Bachelor of Engineering Technology (Automation and Robotics)
with Honours
Universiti Kuala Lumpur

OCTOBER 2023

i
 DECLARATION

I declare that this report is my original work and all references have been
cited adequately as required by the University.

Date: DD/MM/2023 Signature : ………………………………

Full Name : NUR SYAMIMI BINTI
ABDUL RAIF
ID Number : 50222120091

ii
 APPROVAL

We have supervised and examined this report and verify that it meets the
program and University’s requirements for the Bachelor of Engineering
Technology (Automation and Robotics) with Honours.

Date: 88/88/2023 Signature : ……………………………

Supervisor : Dr. Yusman Yusoff
Official Stamp :

Date: 88/88/2023 Signature : ……………………………

Supervisor : Co-Supervisor name
Official Stamp :

iii
 ACKNOWLEDGEMENT

I am deeply grateful to ALLAH "S.W.T" for providing me with the wisdom,

strength, patience, and support I needed to accomplish my research. The
conclusion of this research would not have been possible without His will and
favour.

I'd like to take this occasion to thank my supervisor, Dr. Yusman bin Yusof, for
his patience, advice, comments, stimulating suggestions, and encouragement
during my research, drafting of this report, and assistance throughout my
project work. This endeavour would not have been possible without his moral
support and drive to see it through.

My work is dedicated to my family. A special feeling of gratitude goes to my

beloved parents, who always encourage me and push me to be tenacious, and
to my darling siblings, who are always there for me and my biggest supporters
during my studies.

This endeavour, too, cannot be completed without the help of many people.
Thank you especially to my lecturer and others who assisted and advised me,
including the critical input for my planning and discoveries. Next, I'd like to
express my gratitude and appreciation to the University of Kuala Lumpur
Malaysia France Institute (UniKL MFI) for providing the facilities.

iv
 CONTENTS

TITLE PAGE....................................................................................................i
DECLARATION ..............................................................................................ii
APPROVAL ...................................................................................................iii
ACKNOWLEDGEMENT ................................................................................iv
CONTENTS ....................................................................................................v
LIST OF TABLES .........................................................................................vii
LIST OF FIGURES ......................................................................................viii
LIST OF ABBREVIATION .............................................................................ix
ABSTRACT ....................................................................................................x
ABSTRAK .....................................................................................................xi
CHAPTER 1: INTRODUCTION ......................................................................1
1.1 Overview ............................................................................................1
1.2 Problem statement .............................................................................3
1.3 Objectives ..........................................................................................3
1.4 Scope of project .................................................................................3
1.5 Significant of project ...........................................................................4
1.6 Report organization ............................................................................5
CHAPTER 2: LITERATURE REVIEW............................................................6
2.1 Introduction ........................................................................................6
2.2 Type of Steel Defect ..........................................................................7
2.2.1 Crack .....................................................................................7
2.2.2 Wrinkle ...................................................................................7
2.2.3 Scratch...................................................................................8
2.4 Deep Learning ...................................................................................9
2.5 Firebase ...........................................................................................10
2.5.1 Firebase Authentication .......................................................11
2.5.2 The Real-time Database ......................................................12
2.5.3 Cloud Storage for Firebase ..................................................12
2.6 Node-Red.........................................................................................13

v
 2.7 WhatsApp Twilio ..............................................................................14
CHAPTER 3: METHODOLOGY ...................................................................15
3.1 Introduction ......................................................................................15
3.2 Project Process Flow .......................................................................15
3.3 Project Management ........................................................................16
3.4 Summary..........................................................................................17
CHAPTER 4: RESULT AND DISCUSSION .................................................19
4.1 Introduction ......................................................................................19
4.2 Hardware Testing.............................................................................19
4.2.1 Result...................................................................................19
4.2.2 Discussion ...........................................................................19
4.3 Software Testing ..............................................................................19
4.3.1 Result...................................................................................19
4.3.2 Discussion ...........................................................................19
4.4 Functionality Testing ........................................................................19
4.4.1 Result...................................................................................19
4.4.2 Discussion ...........................................................................19
4.5 Chapter summary ............................................................................19
CHAPTER 5: CONCLUSION .......................................................................20
5.1 Introduction..............................................................................20
5.2 Limitation .................................................................................20
5.3 Future work .............................................................................20
5.4 Summary .................................................................................20
REFERENCES .............................................................................................21
APPENDIX A ................................................................................................23
Program code ............................................................................................23
APPENDIX B ................................................................................................24
Title ............................................................................................................24
APPENDIX C ................................................................................................25
Title ............................................................................................................25

vi
 LIST OF TABLES

Page No
Table 1 Project milestones............................................................................16
Table 2 Gantt chart .......................................................................................17

vii
 LIST OF FIGURES

Page No
Figure 1 Crack defect......................................................................................2
Figure 2 Stamping machine ............................................................................7
Figure 3 Typical surface of wrinkle .................................................................8
Figure 4 Scratch on steel surface ...................................................................9
Figure 5 Firebase console ............................................................................10
Figure 6 Firebase authentication diagram.....................................................11
Figure 7 Firebase Real-Time database.........................................................12
Figure 8 Cloud storage for firebase ..............................................................13
Figure 9 Node-Red .......................................................................................14
Figure 10 WhatsApp twilio ............................................................................14
Figure 11 Project process flow ......................................................................15

viii
 LIST OF ABBREVIATION

CNN Convolutional Neural Network

ADDS Automation Defect Detection System
CPU Central Processing Unit
GPU Graphic Processing Unit
MFI Malaysia France Institute
AI Artificial Intelligence
UniKL University Kuala Lumpur

ix
 ABSTRACT

In steel production and quality control, prompt detection of cracks and defects
is imperative. Variations in manufacturing processes and the structural
integrity of steel products emphasize the need for swift issue identification.
Defects like surface cracks, inclusions, and delamination can imperil product
quality and safety, potentially risking entire structures and incurring substantial
financial losses. Conventional defect detection methods, reliant on labour-
intensive manual inspection, are error-prone and demand extensive expertise
for precise defect identification. Hence, the adoption of autonomous systems
for automated detection of steel defects is indispensable. In this project, an
advanced deep learning model, employing Convolutional Neural Networks
(CNNs), was developed for detecting steel defects and cracks. The model was
trained on a dataset comprising images of defect-free steel from online
sources. The model's performance is pivotal in significantly reducing
inspection time, enhancing defect detection accuracy, and elevating overall
quality control in steel manufacturing. After the model was trained there will be
a system that give alert message through WhatsApp apps and defect image,
time and which station will be display on the dashboard in node-red.
Implementing this model in actual steel manufacturing facilities can notably
reduce the time and effort required for defect and crack identification, leading
to cost savings and heightened product quality. Furthermore, it ensures that
steel structures and products conform to the highest safety and performance
standards, promising a bright future for the application of deep learning in the
steel industry.

x
 ABSTRAK

Dalam pengeluaran keluli dan kawalan kualiti, pengesanan segera retak dan
cacat adalah penting. Perbezaan dalam proses pengeluaran dan integriti
struktur produk keluli menekankan keperluan untuk pengenalan masalah yang
cepat. Kesilapan seperti retakan permukaan, penyimpangan, dan delaminasi
boleh merosakkan kualiti produk dan keselamatan, berpotensi
membahayakan struktur keseluruhan dan menyebabkan kerugian kewangan
yang besar. Kaedah pengesanan cacat konvensional, bergantung kepada
pemeriksaan manual yang berat kerja, terdedah kepada kesilapan dan
memerlukan kepakaran yang luas untuk pengenalan cacat yang tepat. Oleh
itu, pengambilan sistem autonomi untuk pengesanan automatik cacat keluli
adalah perlu. Dalam projek ini, model pembelajaran mendalam lanjutan, yang
menggunakan Rangkaian Neural Convolutional (CNNs), telah dibangunkan
untuk mengesan cacat dan retak keluli. Model ini dilatih pada set data yang
terdiri daripada imej keluli bebas cacat dari sumber dalam talian. Teknik
augmentation telah digunakan untuk meningkatkan kepelbagaian dan saiz
dataset. Prestasi model ini adalah penting dalam mengurangkan masa
pemeriksaan secara ketara, meningkatkan ketepatan pengesanan cacat, dan
meningkatkan kawalan kualiti keseluruhan dalam pengeluaran keluli. Selepas
model dilatih akan ada sistem yang akan memberikan mesej amaran melalui
aplikasi WhatsApp dan imej yang rosak, masa dan stesen yang akan
dipaparkan pada papan panduan dalam node-red. Implementasi model ini
dalam kemudahan pengeluaran keluli sebenar boleh mengurangkan masa
dan usaha yang diperlukan untuk pengesanan cacat dan retak, yang
membawa kepada penjimatan kos dan kualiti produk yang lebih tinggi. Selain
itu, ia memastikan bahawa struktur keluli dan produk mematuhi piawaian
keselamatan dan prestasi tertinggi, menjanjikan masa depan yang cerah untuk
aplikasi pembelajaran mendalam dalam industri keluli.

xi
 CHAPTER 1: INTRODUCTION

1.1 Overview

Steel crack defect detection in industry is the critical practice of recognizing

cracks and defects in steel materials used in a variety of industrial sectors, and
it is critical in providing the safety and quality of steel-based goods. Steel is
widely used in industry; therefore, defect identification is an essential part of
preserving structural integrity and product reliability. Because of the potential
consequences of missed errors, such as structural collapses, equipment
breakdowns, and safety dangers, it's a high-risk operation. The impact of steel
cracks is significant, both economically and in terms of safety. While certain
businesses continue to use outdated crack detection methods, there is a rising
demand for technological innovations to improve the accuracy and efficiency
of detection operations.

In this project the student has considered the major types of defects
which is crack. In the context of the student's project, the focus has been on
examining the major type of defect, which is "crack". Cracks are a critical
concern in various fields, including materials science, engineering, and
construction. Understanding cracks and their implications is essential for
ensuring the reliability and safety of structures and materials. Cracks can
manifest for a multitude of reasons, often due to factors like stress,
temperature fluctuations, environmental conditions, or material defects.
Identifying the root cause of a crack is fundamental in determining the
appropriate course of action. Furthermore, it's essential to classify the type of
crack accurately, as surface cracks, subsurface cracks, and internal cracks
may require distinct approaches for detection and mitigation.

1
 The impacts of cracks cannot be underestimated. In many situations,
cracks can lead to a decrease in structural integrity, compromised safety, and
even catastrophic failures. Managing and addressing cracks in a timely and
effective manner is imperative. Research and innovation in materials science
and engineering continue to advance our understanding of crack formation and
management. This involves exploring materials with enhanced crack
resistance and developing innovative repair methods to ensure the longevity
and safety of structures and materials. In summary, cracks are a significant
concern in various industries, and their proper understanding, detection, and
management are essential for maintaining the integrity and safety of structures
and materials. The student's project on this topic represents a valuable
contribution to this field of study. Figure 1.1 shows the crack defect of product.

Figure 1 Crack defect

There are two types of detectives which are crack and non-crack for
steel, but nowadays there are limitations to diagnosis the disease which
requires time and expertise. Thus, having a model with artificial intelligence
(AI) algorithm to help operator identify the defect of steel will be able to reduce
related defects. Using deep learning is one of the AI algorithms that is used to
identify cracks of steel. In this case, it required a trained model to developed
deep learning using AI algorithms.

Deep learning algorithm is an advanced technique in AI. It is used in

speech recognition, object detection and object classification. Deep learning is
a supervised learning method that uses training and testing data to extract the
features from dataset and then, it will be used to predict, classify, and identify
any object or data afterward.

2
1.2 Problem statement

The existing method for defect detection is simply by using naked eye
observation on expert through which identification and detection of defect is
done. The common defect of the product is crack. With current focus on
increasing product of steel in Malaysia, there is need for operator to make
themselves with more knowledge on how to detect the defect. The limitation
so far has shown to diagnosis the defect which is requires time and expertise.

Nowadays, all these things will be alleviated by using automated

systems to process the samples in a fast and precise using algorithm deep
learning model and a complex system to determine which station, when it is
happened, and the defect image detect by the algorithm. Furthermore, a deep
learning model using CNN to identify the defect was an automated system
required. Also, this system will be developed by using collected data from
online.

1.3 Objectives

The aim of this project is to develop a CNN model for automation defect
detection system using deep learning. To fulfill this aim, several objectives
need to be achieved. The objectives are:

i) To evaluate the CNN model performance for defect detection.

ii) To develop a system for storage, alert message, and node-red
dashboard to display the information of the defect images.

1.4 Scope of project

The project scope of the “Automation Defect Detection System” was set up in
line with the aim of this project. This project is to detect a defect of steel which
is crack. By using deep learning and CNN architecture to develop a model with
high accuracy for defect crack detection. Thus, for this project required
hardware and software tools. The hardware used in this project is my own
machine, which is a laptop. The machine is used to store data, to code a model
and train the model. The model will be trained using Acer Swift 3 with AMD

3
Ryzen 5 3500U central processing unit (CPU). The memory of PC is 8GB
DDR4-2666 MHz and the graphic processing unit (GPU) uses by the PC is
Radeon Vega Mobile Gfx 2.10 GHz.

The software utilized is Jupiyter Notebook. The Jupyter Notebook is an

open-source software for engineers. It supports interactive data research and
scientific computing. Thus, it is the most significant component required for the
project because it uses Python language coding for training, testing and
assessment of the model.

Another software utilized is Visual Studio Code. The Visual Studio Code
is also free and open-source software. Its clear user interface is a bonus, and
the editor's nicely sectioned layout makes coding and troubleshooting much
easier. Thus, it is a perfect integrated development environment for Python.

1.5 Significant of project

The significant for this project is to successfully build a system for defect
detection with data security and privacy, ADDS frequently entail the usage of
sensitive data, it is vital to ensure the security and privacy of this information.
Compliance with applicable data protection standards, as well as the use of
secure data storage and communication techniques, are required.

Integration with manufacturing processes, the successful adoption of

ADDS requires seamless integration with existing production processes. The
system should be able to work alongside other technologies and procedures
without interfering with the entire production workflow.

Overall, this project focused on developing a system for defect detection

system that human specialists may find difficult to detect. The ADDS's capacity
to scale with the changing needs of a production operation is critical. The
system's scalability ensures that it stays effective as production quantities
increase.

4
1.6 Report organization

This report is made up of five main chapters to show the progress of this report.
Chapter 1 of this report is used to describe the application system of defect
detection. This chapter helps the readers to get an overview of the project after
reading the title.

Chapter 2 consists of literature review of the previous research done by

others in the past and uses the previous studies method to conduct the project.
This chapter summarized the related study and most current findings to the
title given.

Chapter 3 proposes the methodology that needed to be done in this

project process. In this chapter will be included all the processes needed to
complete this proposal.

Chapter 4 presents the results and discussion of the project. It

emphasizes and discusses the outcome of the project.

Chapter 5 states the conclusion of the project. It summarizes the

materials, the results, and the future works to improve the performance of this
presented project.

5
 CHAPTER 2: LITERATURE REVIEW

2.1 Introduction

This chapter will explain approximately the research and literature of preceding
project regarding crack detection using deep learning. Research from articles,
journals, books, and the internet has been overviewed and becoming a
reference to this project.

The automotive industry is constantly evolving to meet the demands of

consumers for safer, more efficient, and longer-lasting vehicles [1]. In this
pursuit of excellence, engineers and manufacturers are exploring new
materials and technologies to improve the performance and durability of
automobiles. One such breakthrough is the integration of crack surface steel
in car products, a pioneering approach that promises to revolutionize the way
we design and produce vehicles.

Crack surface steel, or CSS, is a cutting-edge material that leverages

controlled cracks or fractures within the steel structure to enhance its
performance characteristics. Unlike traditional steel, which is typically
associated with toughness and rigidity, CSS introduces a controlled level of
flexibility and energy absorption. This controlled cracking allows for improved
impact resistance, reduced weight, and better energy management, making it
an ideal choice for various automotive applications [2].

The integration of CSS in car products opens a real of possibilities for

enhancing vehicle safety, fuel efficiency, and sustainability. Whether applied
to critical structural components like chassis, bumpers, or side impact beams,
or in innovative ways such as lightweight body panels, CSS has the potential
to redefine the automotive landscape.

6
2.2 Type of Steel Defect

Defects have serious negative effects on the production industry, and the
correct diagnosis of defects is the key to avoiding these effects. However, the
existing disease diagnosis methods for defect detection are not accurate,
consumes time and expertise.

2.2.1 Crack

"Crack" refers to unwanted material cracks in stamping procedures.

Material characteristics, high stress, sharp corners in dies, insufficient
lubrication, work hardening, and temperature changes are some of the
causes[2]. Examining material qualities, die design, press settings,
lubrication, and temperature management are all important aspects of
prevention.

For stamped parts to have early crack detection, strict quality control
and inspection are essential. The stamping procedure involves inserting a
flat sheet of metal into a stamping press machine where a die surface
modifies the shape of the metal in stamping machine refer to Figure 2 [3].

Figure 2 Stamping machine

2.2.2 Wrinkle

A wrinkle is an unwanted fault in stamping processes where the material

being stamped creates folds or creases due to incorrect flow into the die or
excessive distortion. Material qualities, die design, lubrication, material
feed, and punch speed and force are all factors that contribute to wrinkles.
Wrinkle prevention entails choosing acceptable materials, optimizing die
7
design, applying proper lubrication, managing stamping settings, and
guaranteeing proper material temperature.
Preventing creases in stamped parts is critical for preserving product
quality and minimizing scrap. Surface wrinkling is a difficult challenge in
terms of surface imperfections that may have occurred during the panel
forming or edge-cutting processes. Refer to Figure 3.

Figure 3 Typical surface of wrinkle

2.2.3 Scratch

Scratch during the stamping process most likely refers to undesired surface
scratches or imperfections that might arise during the metal stamping
process. Metal stamping is a manufacturing process that involves shaping
and cutting metal sheets with the help of a press, die, and punch [1].
Scratches on the surface of stamped metal are undesirable because they
can degrade the part's quality and appearance. Process control, tool
maintenance, and material handling are all critical in reducing such faults
during the stamping process. Scratches on the surface of stamping parts
detract from the car's appearance and quality. Intense scratches will cause
rust, oxidation, and corrosion on the car's surface, reducing the vehicle's
service life [1]. Refer to Figure 4 that scratches on steel surface.

8
 Figure 4 Scratch on steel surface

2.4 Deep Learning

Deep learning algorithms are a subset of machine learning algorithms that

unlike machine learning algorithms, are capable of automatically extracting
features and function of the brain, known as artificial neural networks. Andrew
who is American Computer Scientist described deep learning in the context of
traditional artificial neural networks. In the 2013 talk titled “Deep Learning, Self-
Taught Learning and Unsupervised Feature Learning” ((IPAM)) he described
the idea of deep learning as:

1. Make learning algorithms much better and easier to use.

2. Make revolutionary advances in machine learning and AI.

Deep learning is just very big neural networks on a lot more data, the most
popular technique are Multilayer Perceptron Networks, Convolutional Neural
Networks and Long Short-Term Memory Recurrent Neural Networks. The
concept of Deep Learning (DL) originated in a 2006 article published in
Science by Hinton et al [4]. The basic idea of Deep Learning is a data analysis
and feature learning are performed using a neural network, where data
features are extracted through multiple hidden layers. Each hidden layer can
be considered as a perceptron, the perceptron is used to extract low-level
features and then combines the low-level features to obtain abstract high-level
features. Deep Learning overcomes the disadvantage of conventional
algorithms relying on artificially generated features and has attracted the
attention of an increasing number of researchers. It is now successfully applied

9
in computer vision, pattern recognition, speech recognition, natural language
processing, and recommender systems [5].

Each Neural Network architecture has advantages and disadvantages.

Feed-Forward Neural Networks, for example, are often employed to solve
classification and regression problems with simple structured input[6].
Recurrent Neural Networks are far more powerful in terms of long-term
memory and are used in sequential data such as text, audio, video, and
image[7]. ConvNets can be used in classification tasks for complicated data
such as photos while Generative Adversarial Networks perform better for
image production or style transfer tasks.

2.5 Firebase

Google officially announced Firebase in the summer of 2016 with the goals of
earning cash for developers, producing a successful application, and
promoting business growth. Firebase is an API update or upgrade that enables
additional functions like messaging and service authentication. As a result, the
cost and time required to construct these services are reduced. Firebase offers
new technology to help businesses advance through linking or
recommendations. The system is then linked by analysing the platform and is
simple to integrate into the application.

Figure 5 Firebase console

10
2.5.1 Firebase Authentication

Firebase Auth takes the least amount of time to integrate into

programming. The project that uses Firebase authentication takes more
time to code than FirebaseUI Auth. It is an online library that allows
clients to customise common Firebase scenarios to give user
authentication implementations in the form of user interface screens.
FirebaseUI Auth covers numerous practices for web and mobile phone
authentication and is simple to use [8]. There are three major steps to
integrating FirebaseUI Auth user authentication into apps: Configure
sign-in methods, personalise the sign-in UI, and use FirebaseUI to
complete the sign-in flow.

Figure 6 Firebase authentication diagram

The Firebase Authentication SDK takes longer to install than the old
one. It also enables verification for the authentication process over full
control of the perspective. Eventually, this option keeps a full version of
authentication with numerous stages to identify the user's account. It is
used to do a variety of functions like signing in and out, logging in
automatically, and updating the new user's info in the app [8].
Furthermore, the Firebase Authentication SDK federates identity
provider integration and anonymous authentication. The Firebase
Authentication SDK is primarily used to approach Firebase Auth. There
are various measures that need be taken: Configure the Sign-in
method, create UI flows for your sign-in methods, and transmit the
user's credentials to the Firebase Authentication SDK [9].

11
2.5.2 The Real-time Database

The traditional real-time database exemplifies a database system that

overcomes the real-time constraint to build a dependable database
system. This is Firebase's main product because it is a basic
functionality that makes use of API [9]. The Realtime Firebase defines
data structure and is protected by Firebase Database Security Rules
that describe who can access specific information. Real-time Database
may support up to 100 000 concurrent connections and 1000
connections per second for each database [8].

Figure 7 Firebase Real-Time database

Because of the heterogeneous database, the data set is altered and

upgraded, and real-time database integration with Cocoa Pods or
Gradle is used to improve client access time [9]. This approach also
allows for writing when offline while maintaining client security. These
are the basic steps for creating a Firebase Real-time Database:
Integrate the Firebase Realtime Database SDKs, build Realtime
Database References, enable Offline Persistence, and safeguard data
[8].

2.5.3 Cloud Storage for Firebase

Videos, photos, and documents are kept online with Firebase Cloud
Storage, which is backed up with Google Cloud Storage. To construct
and manage an infrastructure for dealing with huge files, a simple API
is used. As a result, papers are shared with specific people via the

12
 Realtime Database Law. Because it relates to Firebase Authentication,
this storage is cached. It is simple to upload and distribute files on
mobile devices that support both Google Cloud and Firebase using
Firebase SDKs [8].

Figure 8 Cloud storage for firebase

2.6 Node-Red

The Node-RED programming environment aims to empower system

developers and engineers by making programming as simple as possible by
connecting graphical components known as nodes. Node-RED IDE is a
powerful tool for developing IoT-based apps [10]. These applications could
include a variety of data collecting, transportation, monitoring, and analytics
scenarios. A group of students, professional developers, or engineers may opt
to create programmed (apps) that allow them to monitor their home or
workplace remotely [10]. Node-RED can be used by programmers who are
just starting out. However, like many other comparable tools that claim to be
beginner-friendly, it quickly requires users to gain some additional
programming abilities.

13
 Figure 9 Node-Red

The IoT platform's APIs are implemented using Node-RED, a flow-based

open-source visual programming tool [11] This enables Petrification to evolve
more quickly, give a more user-friendly user interface, and capitalise on the
rich ecosystem of resources provided by its users.

2.7 WhatsApp Twilio

Twilio is a communication platform that will be saved in the cloud using a

storage service. Twilio is a platform as a service (CpaaS) for cloud
communications. Twilio programming code functions to allow a system to
integrate or improve communication capabilities such as WhatsApp, phone,
and SMS [12]. WhatsApp messenger is an instant messaging platform or chat
programme that allows users to interact via the internet. Furthermore, users
can share files in the form of text, voice, video, or location. Customers must
provide a cellphone number to access the WhatsApp messenger application's
services [12].

Figure 10 WhatsApp twilio

14
 CHAPTER 3: METHODOLOGY

3.1 Introduction

Based on the report, the model that will be developed in the python is to detect
a defect from an existing dataset defect which is crack and non-defect. The
model will be used to detect the crack and non-crack on steel surfaces. After
the model detects the defect, the image will be displayed in dashboard in node-
red and there will be an alert message sent via WhatsApp. The image will be
stored in firebase storage.

3.2 Project Process Flow

The project is divided into two parts which are obtaining and preparing the
validation of the dataset and the develop a system that is for defect detection
which is include alert message, information display on node-red dashboard
and keep the image in firebase storage. We will take an existing dataset of
crack and use it in this project to investigate if it should be done faithfully to the
previous project.

Figure 11 Project process flow

15
3.3 Project Management

The project is carried out for two semesters which is around one year. The
project does not involve any financial proceedings. Table 1 shows the project
milestones for the one semester.

Table 1 Project milestones

Cumulative
Description Date Week
Completion (%)
Analyze the existing
23/10/2023 –
storage and 1 20
29/10/2023
hardware system.
Install and configure
the firebase 30/10/2023 –
2 until 4 30
authentication and 19/11/2023
storage.
Develop WhatsApp 20/11/2023 –
5 40
Twilio. 26/11/2023
Create Node-red
27/11/2023 –
dashboard and try 6 until 8 80
24/12/2023
the to get data.
Making the hardware 25/12/2023 –
9 until 11 90
for the project. 14/01/2023
Result analysis and 15/01/2023 –
12 95
conclusion 21/01/2023
22/01/2023 –
Report Writing 13 until 14 100
04/02/2023

Table 2 shows the Gantt chart for the project. The project will be carried out
from October 2023 to February 2024. The project is planned to be conducted
within 3 months 1 week.

16
 Table 2 Gantt chart

Month
Activity Start Date End Date
10 11 12 1 2

Analyze the existing

storage and 23/10/2023 29/10/2023
hardware system.

Install and configure

the firebase
30/10/2023 19/11/2023
authentication and
storage.

Develop WhatsApp
20/11/2023 26/11/2023
Twilio.

Create Node-red
dashboard and try 27/11/2023 24/12/2023
the to get data.

Making the hardware

25/12/2023 14/01/2023
for the project.

Result analysis and

15/01/2023 21/01/2023
conclusion

Report Writing 22/012023 04/02/2023

3.4 Summary

In this project, implement a deep learning detection solution for defect

detection system. Using computer vision, the system checks products for
irregularities and generates alerts when they are found. This model is used to
mimic the human brain that can know the crack on steel surface. The objective
is to evaluate the CNN model performance for defect and non-defect detection

17
and to develop a system for storage, alert message, and node-red dashboard
to display the information of the defect images.

The system discovers irregularities in automotive components such as

cracks, wrinkles, and scratches. When a fault is detected, this system reacts
quickly, sending out rapid WhatsApp alerts. These alerts notify key staff in a
timely manner, allowing for quick response and minimising production delays.
Each alert comes with a timestamped snapshot of the reported issue, which is
conveniently displayed on a Node-Red dashboard. Every discovered flaw is
safely kept in the cloud using Firebase storage, along with its related image
and timestamp.

In conclusion, this real-time visual representation enables engineers

and quality control workers to swiftly assess the problem, determine the
underlying cause, and make informed judgements about corrective steps. This
automated approach improves quality control, lowers human error, and
streamlines communication, ultimately contributing to a more efficient and
dependable automobile manufacturing process.

18
 CHAPTER 4: RESULT AND DISCUSSION

4.1 Introduction

4.2 Hardware Testing

4.2.1 Result

4.2.2 Discussion

4.3 Software Testing

4.3.1 Result

4.3.2 Discussion

4.4 Functionality Testing

4.4.1 Result

4.4.2 Discussion

4.5 Chapter summary

19
 CHAPTER 5: CONCLUSION

5.1 Introduction

5.2 Limitation

5.3 Future work

5.4 Summary

20
 REFERENCES

[1] H. I. Lin and F. S. Wibowo, “Image Data Assessment Approach for Deep Learning-
Based Metal Surface Defect-Detection Systems,” IEEE Access, vol. 9, pp. 47621–
47638, 2021, doi: 10.1109/ACCESS.2021.3068256.

[2] J. Yang, S. Li, Z. Wang, H. Dong, J. Wang, and S. Tang, “Using deep learning to detect
defects in manufacturing: A comprehensive survey and current challenges,”
Materials, vol. 13, no. 24. MDPI AG, pp. 1–23, Dec. 02, 2020. doi:
10.3390/ma13245755.

[3] X. Yin, Y. Chen, A. Bouferguene, H. Zaman, M. Al-Hussein, and L. Kurach, “A deep

learning-based framework for an automated defect detection system for sewer
pipes,” Autom Constr, vol. 109, Jan. 2020, doi: 10.1016/j.autcon.2019.102967.

[4] G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with

neural networks,” Science (1979), vol. 313, no. 5786, pp. 504–507, Jul. 2006, doi:
10.1126/science.1127647.

[5] J. Liu, L. Xu, X. Cao, K. Zhang, Q. Zhang, and Y. Cai, “Review on the Architectures and
Applications of Deep Learning in Agriculture,” in Proceedings - 2020 7th
International Conference on Information Science and Control Engineering, ICISCE
2020, Institute of Electrical and Electronics Engineers Inc., Dec. 2020, pp. 1234–
1240. doi: 10.1109/ICISCE50968.2020.00250.

[6] S. Chaturvedi, R. N. Titre, and N. Sondhiya, “Review of handwritten pattern

recognition of digits and special characters using feed forward neural network and
izhikevich neural model,” in Proceedings - International Conference on Electronic
Systems, Signal Processing, and Computing Technologies, ICESC 2014, 2014, pp. 425–
428. doi: 10.1109/ICESC.2014.83.

[7] B. Chandra and R. K. Sharma, “On improving recurrent neural network for image
classification,” in Proceedings of the International Joint Conference on Neural
Networks, Institute of Electrical and Electronics Engineers Inc., Jun. 2017, pp. 1904–
1907. doi: 10.1109/IJCNN.2017.7966083.

[8] “firebase”.

[9] U. A. Madaminov and M. R. Allaberganova, “Firebase Database Usage and

Application Technology in Modern Mobile Applications,” in 2023 IEEE XVI
International Scientific and Technical Conference Actual Problems of Electronic

21
 Instrument Engineering (APEIE), IEEE, Nov. 2023, pp. 1690–1694. doi:
10.1109/APEIE59731.2023.10347828.

[10] IEEE Education Society, IEEE Industrial Electronics Society, European Association for
Education in Electrical and Information Engineering, and Institute of Electrical and
Electronics Engineers, ITHET 2017 : 2017 16th International Conference on
Information Technology Based Higher Education and Training (ITHET) : June 10-12,
2017, Ohrid, Macedonia.

[11] H. Kim et al., “Petification: Node-RED Based Pet Care IoT Solution Using MQTT
Broker,” in International Conference on ICT Convergence, IEEE Computer Society,
2022, pp. 25–29. doi: 10.1109/ICTC55196.2022.9952423.

[12] R. Munadi, “LPG Gas Leakage System with Instant Messaging Whatsapp
Communication Media Based on Internet of Things,” 2020.

22
 APPENDIX A

Program code

Anything…

23
 APPENDIX B

Title

Anything…

24
 APPENDIX C

Title

Anything…

25