Scribd
Upload
14 views16 pages

Using Transfer Learning and TensorFlow 2.0 To Classify Different Dog Breeds

This document outlines a project using TensorFlow 2.0 and transfer learning to classify different dog breeds from a dataset of over 10,000 labeled images. The workflow includes data preparation, model training, evaluation, and improvement, leveraging a pretrained model from TensorFlow Hub. The project emphasizes the importance of efficient data handling and experimentation to optimize the machine learning process.

Uploaded by

2021ecpulkitpathakb
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
14 views16 pages

Using Transfer Learning and TensorFlow 2.0 To Classify Different Dog Breeds

This document outlines a project using TensorFlow 2.0 and transfer learning to classify different dog breeds from a dataset of over 10,000 labeled images. The workflow includes data preparation, model training, evaluation, and improvement, leveraging a pretrained model from TensorFlow Hub. The project emphasizes the importance of efficient data handling and experimentation to optimize the machine learning process.

Uploaded by

2021ecpulkitpathakb
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 16

🐶 Using Transfer Learning and TensorFlow 2.

0 to Classify Different Dog Breeds

In this project we're going to be using machine learning to help us identify different breeds of dogs.

To do this, we'll be using data from the Kaggle dog breed identification competition. It consists of a
collection of 10,000+ labelled images of 120 different dog breeds.

This kind of problem is called multi-class image classification. It's multi-class because we're trying to
classify mutliple different breeds of dog. If we were only trying to classify dogs versus cats, it would
be called binary classification (one thing versus another).

Multi-class image classification is an important problem because it's the same kind of technology
Tesla uses in their self-driving cars or Airbnb uses in atuomatically adding information to their listings.

Since the most important step in a deep learng problem is getting the data ready (turning it into
numbers), that's what we're going to start with.

We're going to go through the following TensorFlow/Deep Learning workflow:

1.Get data ready (download from Kaggle, store, import).

2.Prepare the data (preprocessing, the 3 sets, X & y).

3.Choose and fit/train a model (TensorFlow Hub, tf.keras.applications, TensorBoard, EarlyStopping).

4.Evaluating a model (making predictions, comparing them with the ground truth labels).

5.Improve the model through experimentation (start with 1000 images, make sure it works, increase
the number of images).

6.Save, sharing and reloading your model (once you're happy with the results).

For preprocessing our data, we're going to use TensorFlow 2.x. The whole premise here is to get our
data into Tensors (arrays of numbers which can be run on GPUs) and then allow a machine learning
model to find patterns between them.

For our machine learning model, we're going to be using a pretrained deep learning model from
TensorFlow Hub.

The process of using a pretrained model and adapting it to your own problem is called transfer
learning. We do this because rather than train our own model from scratch (could be timely and
expensive), we leverage the patterns of another model which has been trained to classify images.

Get our workspace ready

#Import Tensorflow into Colab

import tensorflow as tf

import tensorflow_hub as hub

print("TF version:",tf.__version__)

print("TF Hub Version :",hub.__version__)

# Check for GPU


print("GPU", "available (YESS!!!!)" if tf.config.list_physical_devices("GPU") else "not available :(")

TF version: 2.15.0

TF Hub Version : 0.15.0

GPU available (YESS!!!!)

#!unzip /content/drive/MyDrive/Dog_Breed/dog-breed-identification.zip

Getting our data ready (turning images into tensors)

# Checkout the labels of our data

import pandas as pd

labels_csv = pd.read_csv("/content/drive/MyDrive/Dog_Breed/labels.csv")

print(labels_csv.describe())

print(labels_csv.head())

id breed

count 10222 10222

unique 10222 120

top 000bec180eb18c7604dcecc8fe0dba07 scottish_deerhound

freq 1 126

id breed

0 000bec180eb18c7604dcecc8fe0dba07 boston_bull

1 001513dfcb2ffafc82cccf4d8bbaba97 dingo

2 001cdf01b096e06d78e9e5112d419397 pekinese

3 00214f311d5d2247d5dfe4fe24b2303d bluetick

4 0021f9ceb3235effd7fcde7f7538ed62 golden_retriever
labels_csv.head()

id breed

0 000bec180eb18c7604dcecc8fe0dba07 boston_bull

1 001513dfcb2ffafc82cccf4d8bbaba97 dingo

2 001cdf01b096e06d78e9e5112d419397 pekinese

3 00214f311d5d2247d5dfe4fe24b2303d bluetick

4 0021f9ceb3235effd7fcde7f7538ed62 golden_retriever

# How many images are there of each breed?

labels_csv["breed"].value_counts().plot.bar(figsize=(20,10))

labels_csv["breed"].value_counts().median()

82.0
#Let's view an image

from IPython.display import display, Image

Image("/content/drive/MyDrive/Dog_Breed/train/000bec180eb18c7604dcecc8fe0dba07.jpg")

Getting images and their labels

Since we've got the image ID's and their labels in a DataFrame (labels_csv), we'll use it to create:

# Create pathnames from image ID's

filenames = ["/content/drive/My Drive/Dog_Breed/train/" + fname + ".jpg" for fname in


labels_csv["id"]]

# Check the first 10 filenames

filenames[:10]

['/content/drive/My Drive/Dog_Breed/train/000bec180eb18c7604dcecc8fe0dba07.jpg',

'/content/drive/My Drive/Dog_Breed/train/001513dfcb2ffafc82cccf4d8bbaba97.jpg',

'/content/drive/My Drive/Dog_Breed/train/001cdf01b096e06d78e9e5112d419397.jpg',

'/content/drive/My Drive/Dog_Breed/train/00214f311d5d2247d5dfe4fe24b2303d.jpg',

'/content/drive/My Drive/Dog_Breed/train/0021f9ceb3235effd7fcde7f7538ed62.jpg',

'/content/drive/My Drive/Dog_Breed/train/002211c81b498ef88e1b40b9abf84e1d.jpg',
'/content/drive/My Drive/Dog_Breed/train/00290d3e1fdd27226ba27a8ce248ce85.jpg',

'/content/drive/My Drive/Dog_Breed/train/002a283a315af96eaea0e28e7163b21b.jpg',

'/content/drive/My Drive/Dog_Breed/train/003df8b8a8b05244b1d920bb6cf451f9.jpg',

'/content/drive/My Drive/Dog_Breed/train/0042188c895a2f14ef64a918ed9c7b64.jpg']

# Check whether number of filenames matches number of actual image files

import os

if len(os.listdir("/content/drive/My Drive/Dog_Breed/train/")) == len(filenames):

print("Filenames match actual amount of files!")

else:

print("Filenames do not match actual amount of files, check the target directory.")

Filenames match actual amount of files!

# Check an image directly from a filepath

Image(filenames[8000])
# Check an image directly from a filepath

Image(filenames[5000])

# Check an image directly from a filepath

Image(filenames[9000])

labels_csv["breed"][9000]

tibetan_mastiff
import numpy as np

labels = labels_csv["breed"].to_numpy()

labels

array(['boston_bull', 'dingo', 'pekinese', ..., 'airedale',

'miniature_pinscher', 'chesapeake_bay_retriever'], dtype=object)

len(labels)

10222

#See if number of labels matches the number of filenames

if len(labels) == len(filenames):

print("Number of labels matches number of filenames!")

else:

print("Number of labels does not match number of filenames, check data directories!")

Number of labels matches number of filenames!

#Find the unique label values

unique_breeds = np.unique(labels)

len(unique_breeds)

120
# Turn a single label into an array of booleans

print(labels[0])

labels[0] == unique_breeds

boston_bull

array([False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, True, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False])

# Turn every label into a boolean array

boolean_labels = [label == unique_breeds for label in labels]

boolean_labels[:2]

[array([False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, True, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,


False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False]),

array([False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, True, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False])]

len(boolean_labels)

10222
# Example : Turning boolean array into integers

print(labels[0]) #original label

print(np.where(unique_breeds == labels[0])) #index where label occurs

print(boolean_labels[0].argmax()) #index where label occurs in boolean array

print(boolean_labels[0].astype(int)) #there will be a 1 where the sample label occurs

boston_bull

(array([19]),)

19

[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0000000000000000000000000000000000000

0000000000000000000000000000000000000

0 0 0 0 0 0 0 0 0]

print(labels[2])

print(boolean_labels[2].astype(int))

pekinese

[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0000000000000000000000000000000000000

0000000000010000000000000000000000000

0 0 0 0 0 0 0 0 0]

filenames[:10]

['/content/drive/My Drive/Dog_Breed/train/000bec180eb18c7604dcecc8fe0dba07.jpg',

'/content/drive/My Drive/Dog_Breed/train/001513dfcb2ffafc82cccf4d8bbaba97.jpg',

'/content/drive/My Drive/Dog_Breed/train/001cdf01b096e06d78e9e5112d419397.jpg',

'/content/drive/My Drive/Dog_Breed/train/00214f311d5d2247d5dfe4fe24b2303d.jpg',

'/content/drive/My Drive/Dog_Breed/train/0021f9ceb3235effd7fcde7f7538ed62.jpg',

'/content/drive/My Drive/Dog_Breed/train/002211c81b498ef88e1b40b9abf84e1d.jpg',

'/content/drive/My Drive/Dog_Breed/train/00290d3e1fdd27226ba27a8ce248ce85.jpg',

'/content/drive/My Drive/Dog_Breed/train/002a283a315af96eaea0e28e7163b21b.jpg',

'/content/drive/My Drive/Dog_Breed/train/003df8b8a8b05244b1d920bb6cf451f9.jpg',
'/content/drive/My Drive/Dog_Breed/train/0042188c895a2f14ef64a918ed9c7b64.jpg']

Creating our own validation set

Since the data from Kaggle doesn't come with validation set

# Setup X and y variable

X = filenames

y = boolean_labels

Since we're working with 10,000+ images, it's a good idea to work with a portion of them to make
sure things are working before training on them all.

This is because computing with 10,000+ images could take a fairly long time. And our goal when
working through machine learning projects is to reduce the time between experiments.

Let's start experimenting with 1000 and increase it as we need.

#Set number of images to use for experimenting

NUM_IMAGES = 1000 #@param {type:"slider", min:1000, max:10000, step:1000}

NUM_IMAGES

1000

# Let's split our data into train and validation sets

from sklearn.model_selection import train_test_split

#Split them into training and validation of total size NUM_IMAGES

X_train,X_val,y_train,y_val = train_test_split(X[:NUM_IMAGES],y[:NUM_IMAGES],test_size =
0.2,random_state = 42)

len(X_train),len(y_train),len(X_val),len(y_val)

(800, 800, 200, 200)


# Let's have a look at training data

X_train[:5],y_train[:5]

(['/content/drive/My Drive/Dog_Breed/train/00bee065dcec471f26394855c5c2f3de.jpg',

'/content/drive/My Drive/Dog_Breed/train/0d2f9e12a2611d911d91a339074c8154.jpg',

'/content/drive/My Drive/Dog_Breed/train/1108e48ce3e2d7d7fb527ae6e40ab486.jpg',

'/content/drive/My Drive/Dog_Breed/train/0dc3196b4213a2733d7f4bdcd41699d3.jpg',

'/content/drive/My Drive/Dog_Breed/train/146fbfac6b5b1f0de83a5d0c1b473377.jpg'],

[array([False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, True,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False]),

array([False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, True, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,


False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False]),

array([False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, True, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False]),

array([False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, True, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,


False, False, False]),

array([False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, True, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False, False, False, False, False, False, False,

False, False, False])])

Preprocessing images(turning images into Tensors)

preprocess our images into Tensors we're going to write a function which does a few things:

1) Takes an image filename as input.

2) Uses TensorFlow to read the file and save it to a variable, image.

3) Turn our image (a jpeg file) into Tensors.

4)Resize the image to be of shape (224, 224).

5)Return the modified image.

A good place to read about this type of function is the TensorFlow documentation on loading images.

You might be wondering why (224, 224), which is (heigh, width). It's because this is the size of input
our model (we'll see this soon) takes, an image which is (224, 224, 3).

What? Where's the 3 from? We're getting ahead of ourselves but that's the number of colour
channels per pixel, red, green and blue.

Let's make this a little more concrete.


# Convert image to NumPy array

from matplotlib.pyplot import imread

image = imread(filenames[42]) # read in an image

image.shape

(257, 350, 3)

# turn image into a tensor

tf.constant(image)[:2]

<tf.Tensor: shape=(2, 350, 3), dtype=uint8, numpy=

array([[[ 89, 137, 89],

[ 76, 124, 76],

[ 63, 111, 61],

...,

[ 77, 133, 86],

[ 76, 134, 86],

[ 76, 134, 86]],

[[ 72, 119, 75],

[ 67, 114, 68],

[ 63, 110, 64],

...,

[ 75, 131, 84],

[ 74, 132, 84],

[ 74, 132, 84]]], dtype=uint8)>


# Define image size

IMG_SIZE = 224

# Create a function for preprocessing images

def process_image(image_path):

"""

Takes an image file path and turns the image into a Tensor

"""

#Read in an image file

image = tf.io.read_file(image_path)

#Turn the jpeg image into numerical Tensor with 3 colours channels(RGB)

image = tf.image.decode_jpeg(image,channels = 3)

#Convert the colour channel values from 0-255 to 0-1 values

image = tf.image.convert_image_dtype(image,tf.float32)

#Resize the image to our desired value(224,224)

image = tf.image.resize(image,size = [IMG_SIZE,IMG_SIZE])

return image

You might also like