Signal and Image Processing

Home Assignment

Topic – An Industrial Case study on Deep learning image classification

An Industrial Case study on Deep learning

image classification

This case study explain a end-to-end use case of deep learning image
classification in order to automate the process of classifying
defective and non-defective castings in foundry.

Image of Cast product (Submersible pump impeller),

Casting Process: Casting is one of the major manufacturing
process in which molten metal is poured in to a cavity called mould
and allowed to cool till it gets solidified into product.

Casting defects: These are the defects in the cast product occurred
during the casting process and they are undesirable.There are many
types of defect in casting like blow hole, pin hole, burr, shrinkage
defects, mould material defects, pouring metal defects, metallurgical
defects etc.

Casting defects are undesirable and cause loss to the manufacturer,

therefore the quality department have to do visual inspection of the
products and separate the defective one from the good castings. The
visual inspection is labour intensive and time consuming, therefore
Convolution Neural Networks (CNN) could be used to automate this
process by image classification. The figure 1. shows the Casting
Inspector app developed in this project. The remaining sections of
this post explains step by step approach used in this case study.
Figure 1. Screen shot of Casting Inspector App developed in this work.

Step 1: Train the model using CNN.

1.1 Download the data set from the

URL : https://www.kaggle.com/ravirajsinh45/real-life-
industrial-dataset-of-casting-product and unzip the folder. There
are two categories in this image classification problem viz. defective
castings and non-defective (ok) castings. The dataset is already
bifurcated into train and test dataset. Train dataset consists of 3758
images of defective and 2875 images of non-defective castings and
test dataset consists of 453 images of defective and 262 images of
non-defective castings. Each image in the dataset is gray scaled
image of the shape 300X300 pixels.
1.2 Train the model: I have used somewhat modified VGG16
algorithm for training theses images. Prior to this, I tried with
Resnet 50 and Lenet algorithms but strangely, results were not
satisfactory. Below is the code I used to train the model, I trained the
model for 100 epochs each with 1000 steps on google colaboratory
GPU. It took more than 12 hours to complete the training process.
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense# Initialising the CNN
classifier = Sequential()# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (300, 300, 3),
activation = 'relu'))# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))# Step 3 -
Flattening
classifier.add(Flatten())# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))#
Compiling the CNN
classifier.compile(optimizer = 'adam', loss =
'binary_crossentropy', metrics = ['accuracy'])# Part 2 - Fitting
the CNN to the images
from keras.preprocessing.image import
ImageDataGeneratortrain_datagen = ImageDataGenerator(rescale =
1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip =
True)test_datagen = ImageDataGenerator(rescale =
1./255)training_set =
train_datagen.flow_from_directory('/content/drive/My
Drive/casting/casting_512x512/CASTING/train', target_size =
(300, 300), batch_size = 32, class_mode = 'binary')test_set =
test_datagen.flow_from_directory('/content/drive/My
Drive/casting/casting_512x512/CASTING/test', target_size = (300,
300),batch_size = 32,class_mode = 'binary')model =
classifier.fit_generator(training_set,
steps_per_epoch = 1000,
epochs = 100,
validation_data = test_set,
validation_steps = 1000)classifier.save("vggmodel.h5")
print("Saved model to disk")
Step 2: Use the trained model to classify the
images.

After training the model, the file containing the weights is generated
in this project it is named “vggmodel.h5”. This file was found to be
as heavy as 1.01 GB which was beyond my anticipation, at the end of
this tutorial I have provided the link to download this file. We can
use this file to predict or classify any other images of the castings.
Below is the code used to predict the image.
from keras.models import load_model
model = load_model('vggmodel.h5')import numpy as np
from keras.preprocessing import imagetest_image =
image.load_img('/home/murali/ds/data/casting
problem/casting_data/casting_data/train/ok_front/cast_ok_0_46.jp
eg', target_size = (300, 300))test_image =
image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = model.predict(test_image)
print(result)if result[0][0] == 1:
prediction = 'CASTING IS OK'
print(prediction)
else:
prediction = 'CASTING IS DEFECTIVE'
print(prediction)

However, this code can’t be used conveniently every time to predict

for every image. Therefore, I decided to write code for GUI based
app through which user can conveniently load casting image and can
classify just by click of a button. The next paragraph explains the
building of such GUI.

Step 3: Build a GUI using tkinter library of python.

GUI is built by using tkinter python library which is most easiest

and effective GUI building tool. It has rich collection of widgets
which makes the process much easier. The following are the
different parts of the code used to build GUI and their functions.

3.1 Importing the libraries

from tkinter import *
from keras.preprocessing import image
import numpy as np
from keras.models import load_model
# loading Python Imaging Library
from PIL import ImageTk, Image
# To get the dialog box to open when required
from tkinter import filedialog

First, we will import tkinter library, followed by other libraries as

shown above. These are used for GUI related functions, image
handling purpose, numpy is used for computation associated with
image. ImageTk for handling images within the GUI , filedialog for
browsing images by user etc.

3.2 Loading the model (weight file)

model = load_model('vggmodel.h5')
print('Model Loaded Sucessfully')

The above code will load the pre-trained model , I have added print
statement, so that user comes to know the status of process.

3.3 Function open_img() and openfilename()

openfilename() is a function which allows user to select a file from a

dialog box and returns the name of the file. open_img() loads the
image with file name provided by openfilename() function and saves
it as “casting.jpeg” for future use and finally, display it as a lable at
specified location on GUI window using place() function.
def open_img():
# Select the Imagename from a folder
x = openfilename()
# opens the image
img = Image.open(x)
im1 = img.save("casting.jpeg")
img = ImageTk.PhotoImage(img)
# create a label
panel = Label(root, image = img)
# set the image as img
panel.image = img
panel.place(bordermode=OUTSIDE, x=50, y=50)

def openfilename():
# open file dialog box to select image
# The dialogue box has a title "Open"
filename = filedialog.askopenfilename(title ='Select Image')

return filename

3.4 The prediction() function.

This function is used to predict the classification of the image using

model, find below the code for the same.
def prediction():
img ="casting.jpeg"
test_image = image.load_img(img, target_size = (300, 300))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = model.predict(test_image)

if result[0][0] == 1:
prediction = 'CASTING IS OK' '

else:
prediction = 'CASTING IS DEFECTIVE'

result = Label(root, text = prediction)

# set the image as img
result.place(bordermode=OUTSIDE, x=400, y=120)

Firstly, the “casting.jpeg” saved earlier by open_img() will be loaded

and suitably converted as test_image which can be used for
prediction purpose. The outcome of prediction is printed on the GUI
window as a label.

3.4 The GUI window

The GUI window is generated by following code:

# Create a window
root = Tk()
# Set Title as Image Loader
root.title("CASTING INSPECTOR")
# Set the resolution of window
root.geometry("700x450")
# Do't Allow Window to be resizable
root.resizable(width = False, height = False)

# Create a button and place it into the window using place

layout
btn_open_image = Button(root, text ='Open image', command =
open_img).place(x = 10, y= 400) btn_predict = Button(root, text
='Predict', command = prediction).place(x = 200, y= 400)
result_hd = Label(root, text = "RESULT")
result_hd.place(bordermode=OUTSIDE, x=400, y=90)
root.mainloop()

The root is the name of our window, root.title will set the title of our
window as “CASTING INSPECTOR”, root.geometry will set the size
of the GUI window. The root.resizable helps to restrict the resize
option. Then two buttons are created, one for browsing and opening
an image file and the second one for prediction using the model. The
button btn_open_image calls open_image() function discussed
earlier, where as button btn_predict will call the
function prediction() which will predict the class, the image belongs
to. Finally, the working of casting inspector is as show below
Conclusion

The post illustrated the implementation of deep learning image

classification as use case in industrial environment. The following
are the future scope for this project

1. The app can be integrated with a camera to capture real

time image of the cast product.

2. Model can be trained for more accuracy by increasing

number of epochs and by use of other deep learning
algorithms.

3. The app can be integrated with some actuator to activate

the mechanism to separate the defective casting as soon as
it is detected.

The code used in this project can be found

on https://github.com/muralimambekar/casting_inspector
The model (weight) file can be downloaded from

https://drive.google.com/file/d/1r0FbT0vd-YfFaWzPLx84txsKnLF-
G3Dc/view?usp=sharing

Step 3: Build a GUI using tkinter library of python.

https://github.com/muralimambekar/casting_inspector