12/16/2019 Classifying images using Keras MobileNet and TensorFlow.

js in Google Chrome – Gogul Ilango

Contents X

Basic MobileNet in Python

Convert Keras model into Tf
JS layers format
Keras model into TensorFlow
JS
1. Load Keras model into
Classifying images using Keras
TF.js

MobileNet and TensorFlow.js in2.3.Google

Upload image from disk
Predict using MobileNet

Chrome model
References
Deep Learning | 27 July 2018

10 Comments

Limited time offer: Get 10

free Adobe Stock images.

ADS VIA CARBON

Follow @Gogul09 246 Fork 7 Star 7

In this blog post, we will understand how to perform image classification using
Keras MobileNet, deploy it in Google Chrome using TensorFlow.js and use it to make
live predictions in the browser.

Being a Python developer with knowledge on Web Development is surely going to

help you in the field of Artificial Intelligence in the long run. Because we now have

https://gogul.dev/software/mobile-net-tensorflow-js 1/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

the awesome capabilities of Keras and TensorFlow in a web browser using

TensorFlow.js.

Contents X
The interactive demo of what we will be making at the end of the tutorial is shown
Basic MobileNet in Python
below. You can play around with the buttons provided to make predictions on an
Convert Keras model into Tf
image. JS layers format
Keras model into TensorFlow
JS
1. Click Load Model to load MobileNet model in your browser.
1. Load Keras model into
2. Loading image -
TF.js
Click Demo Image to import a random image that belongs to an ImageNet
2. Upload image from disk
category. 3. Predict using MobileNet

Click Upload Image if you want to import an image from your model
disk.
References
3. Click Predict to make predictions on the image loaded in the browser.

Keras MobileNet + TensorFlow.js Demo

Load Model Demo Image Upload Image Predict

Note: The above demo uses state-of-the-art Keras MobileNet that's trained
on ImageNet with 1000 categories. If you upload an image that doesn't
belong to any of the 1000 ImageNet categories, then the prediction might
not be accurate!

Downloads

HTML CSS JavaScript Python

Basic MobileNet in Python

In Keras, MobileNet resides in the applications module. Keras offers out of the box
image classification using MobileNet if the category you want to predict is available
in the ImageNet categories. If the category doesn’t exist in ImageNet categories,

https://gogul.dev/software/mobile-net-tensorflow-js 2/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

there is a method called fine-tuning that tunes MobileNet for your dataset and
classes which we will discuss in another tutorial.

Contents X
MobileNet offers tons of advantages than other state-of-the-art convolutional neural
Basic MobileNet in Python
networks such as VGG16, VGG19, ResNet50, InceptionV3 and Xception.
Convert Keras model into Tf
JS layers format
MobileNets are light weight deep neural networks best suited forKeras
mobile and
model into TensorFlow
JS
embedded vision applications.
1. Load Keras model into
MobileNets are based on a streamlined architecture that uses depthwise separable
TF.js
convolutions. 2. Upload image from disk
MobileNet uses two simple global hyperparameters that e ciently tradesusing
3. Predict off MobileNet
between accuracy and latency. model
References
MobileNet could be used in object detection, negrain classi cation, face recognition,
large-scale geo localization etc.

Following are the advantages of using MobileNet over other state-of-the-art deep
learning models.

Reduced network size - 17MB.

Reduced number of parameters - 4.2 million.
Faster in performance and are useful for mobile applications.
Small, low-latency convolutional neural network.

Advantages always come up with some disadvantages and with MobileNet, it’s the
accuracy. Yes! Eventhough MobileNet has reduced size, reduced parameters and
performs faster, it is less accurate than other state-of-the-art networks as
discussed in this paper. But don’t worry. There is only a slight reduction in accuracy
when compared to other networks.

In this tutorial, we will follow the steps shown in Figure 1 to make Keras MobileNet
available in a web browser using TensorFlow.js.

https://gogul.dev/software/mobile-net-tensorflow-js 3/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

Contents X

Basic MobileNet in Python

Convert Keras model into Tf
JS layers format
Keras model into TensorFlow
JS
1. Load Keras model into
TF.js
2. Upload image from disk
3. Predict using MobileNet
model
References

Figure 1. Keras MobileNet in Google Chrome using TensorFlow.js

First, we will write a simple python script to make predictions on a test image using
Keras MobileNet.

Before sending an image into MobileNet, we need to process the image using 4
simple steps. And to do that, you don’t need OpenCV. Keras provides all the
necessary functions under keras.preprocessing module, and with some basic
numpy functions, you are ready to go!

1. Load the image and convert it to MobileNet’s input size (224, 224) using load_img()

function.
2. Convert the image into a numpy array using img_to_array() .
3. Expand the dimensions of the numpy array using np.expand_dims() .

https://gogul.dev/software/mobile-net-tensorflow-js 4/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

4. Preprocess the image by rescaling all the values to the range [-1, 1] using
mobilenet.preprocess_input() .

Contents X
basic-mobile-net.py code
Basic MobileNet in Python
Convert Keras model into Tf
JS layers format
1 # organize imports
Keras model into TensorFlow
2 import numpy as np
3 from keras.models import Model JS
4 from keras.preprocessing import image 1. Load Keras model into
5 from keras.applications import imagenet_utils, mobilenetTF.js
6 import tensorflowjs as tfjs 2. Upload image from disk
7
3. Predict using MobileNet
8 # process an image to be mobilenet friendly
model
9 def process_image(img_path):
10 img = image.load_img(img_path, target_size=(224, 224)) References
11 img_array = image.img_to_array(img)
12 img_array = np.expand_dims(img_array, axis=0)
13 pImg = mobilenet.preprocess_input(img_array)
14 return pImg
15
16 # main function
17 if __name__ == '__main__':
18
19 # path to test image
20 test_img_path = "G:\\git-repos\\mobile-net-projects\\dataset\\test\\test_im
21
22 # process the test image
23 pImg = process_image(test_img_path)
24
25 # define the mobilenet model
26 mobilenet = mobilenet.MobileNet()
27
28 # make predictions on test image using mobilenet
29 prediction = mobilenet.predict(pImg)
30
31 # obtain the top-5 predictions
32 results = imagenet_utils.decode_predictions(prediction)
33 print(results)
34
35 # convert the mobilenet model into tf.js model
36 save_path = "output\\mobilenet"
37 tfjs.converters.save_keras_model(mobilenet, save_path)
38 print("[INFO] saved tf.js mobilenet model to disk..")

[[('n01806143', 'peacock', 0.9998889),

https://gogul.dev/software/mobile-net-tensorflow-js 5/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

('n01806567', 'quail', 3.463593e-05),

('n02018795', 'bustard', 2.7573227e-05),
('n01847000', 'drake', 1.1352683e-05),
('n01795545', 'black_grouse', 1.0532762e-05)]]
Contents X

Basic MobileNet in Python

Convert Keras model into Tf
JS layers format
Lines 2-5 imports all the necessary functions to work with.
Keras model into TensorFlow
Lines 8-13 is the special de nition we use to process an image so
JS that it becomes
MobileNet friendly. 1. Load Keras model into
TF.js
Line 19 de nes the test image path.
2. Upload image from disk
Line 22 preprocesses the test image.
3. Predict using MobileNet
Line 25 instantiates the MobileNet model. model
Line 28 makes predictions on the test image using MobileNet model.
References

Line 31 gives the top-5 predictions of the test image.

Line 32 prints out the top-5 predictions of the test image.
Lines 36-38 converts keras mobilenet model into tf.js layers format at save_path .

Please make sure you change the test_img_path in line 19 to test an image from
your disk. Figure 2 (shown below) is the test image that I have chosen and the
MobileNet model accurately predicted it as a peacock with a probability of 99.99%.
Pretty cool! 😍

https://gogul.dev/software/mobile-net-tensorflow-js 6/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

Contents X

Basic MobileNet in Python

Convert Keras model into Tf
JS layers format
Keras model into TensorFlow
JS
1. Load Keras model into
TF.js
2. Upload image from disk
3. Predict using MobileNet
model
References

Figure 2. Input test image for MobileNet

Cool! Everything works perfectly in our Python environment. Now, we will use this
pretrained mobile net model in a web browser.

Convert Keras model into Tf.js layers format

Before deploying a keras model in web, we need to convert the Keras mobilenet
python model into tf.js layers format (which we already did in lines 36-38).

To deploy a Keras model in web, we need a package called tensorflowjs . Run the
below command to get it.

install tensorflowjs cmd

https://gogul.dev/software/mobile-net-tensorflow-js 7/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

1 pip install tensorflowjs

Contents X
After installing it, you can either run the command as a standalone one or you can
Basic MobileNet in Python
integrate it in your python script as shown below (which I prefer).
Convert Keras model into Tf
JS layers format
Keras model into TensorFlow
1. keras to tf.js layers format cmd
JS
1. Load Keras model into
TF.js
1 tensorflowjs_converter --input_format keras \
2 path_to_keras_model.h5 \ 2. Upload image from disk
3 path/to/tfjs_target_dir 3. Predict using MobileNet
model
References

2. inside python script code

1 import tensorflowjs as tfjs

2
3 def train(...):
4 model = keras.models.Sequential() # for example
5 ...
6 model.compile(...)
7 model.fit(...)
8 tfjs.converters.save_keras_model(model, tfjs_target_path)

The tfjs_target_path or save_path (in our case) is a folder that contains

model.json and a set of sharded weight binary files. If you take a look into
model.json file, you will see the model architecture or graph (a description of layers
and their connections) plus a manifest of the weight files.

Keras model into TensorFlow JS

For this tutorial, I used my GitHub pages repo to hold the keras mobilenet model
files. I copied the entire folder under save_path here.

This is crucial for our application to work because if you host these model files in a
different server, you might face CORS issue in your web app. Storing your model

https://gogul.dev/software/mobile-net-tensorflow-js 8/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

files in the same domain as your web app is safer and preferred way.

Let’s get started with the sweet TensorFlow.js 😘

Contents X

Basic MobileNet in Python

You need these three javascript libraries loaded in your website.
Convert Keras model into Tf
JS layers format
1. IMAGENET_CLASSES Keras model
variable that has all the ImageNet categories indexed into could
which TensorFlow
JS
easily be loaded from here.
1. Load Keras model into
2. TensorFlow.js latest source. TF.js
3. jQuery to make JavaScript easier. 2. Upload image from disk
3. Predict using MobileNet
model
index.html code
References

1 <script type="text/javascript" src="/js/imagenet_classes.js"></script>

2 <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@latest"></script>
3 <script type="text/javascript" src="https://code.jquery.com/jquery-2.1.1.min.

Once you load all the above three scripts, you can open up a new file named mobile-

net.js that will have all the functionality needed to make Keras MobileNet model
work in a web browser.

The user interface that I made at the start of the tutorial has HTML, CSS and
JavaScript code combined. We will look into model specific part instead of looking
into every single line of code.

1. Load Keras model into TF.js

Firstly, you need to load the Keras pretrained model json that you have stored in
your web server. To do this, you can use the below code snippet.

The below code snippet is an async function that loads a keras model json using
tf.loadModel() . In line 17, await means without disturbing the UI, you are asking
JavaScript to load model behind the scenes. To view the status of model loading,
we use a progress bar as well as console.log() .

https://gogul.dev/software/mobile-net-tensorflow-js 9/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

mobile-net.js code

1 let model; Contents X

2 async function loadModel() {
Basic MobileNet in Python
3 console.log("model loading..");
4 Convert Keras model into Tf
5 // display model loading progress box JS layers format
6 loader = document.getElementById("progress-box"); Keras model into TensorFlow
7 load_button = document.getElementById("load-button"); JS
8 loader.style.display = "block"; 1. Load Keras model into
9
TF.js
10 // model name is "mobilenet"
11 modelName = "mobilenet"; 2. Upload image from disk
12 3. Predict using MobileNet
13 // clear the model variable model
14 model = undefined; References
15
16 // load the model using a HTTPS request (where you have stored your model f
17 model = await tf.loadLayersModel('https://gogul09.github.io/models/mobilene
18
19 // hide model loading progress box
20 loader.style.display = "none";
21 load_button.disabled = true;
22 load_button.innerHTML = "Loaded Model";
23 console.log("model loaded..");
24 }

2. Upload image from disk

To upload an image from disk, you can use the below code snippet which makes
use of HTML5 File API. I have used a button Upload Image which has a change

handler associated with it.

index.html code

1 <!-- used to get image from disk -->

2 <input id="select-file-image" type="file">

mobile-net.js code

https://gogul.dev/software/mobile-net-tensorflow-js 10/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

1
2 // if there is a change to "Upload Image" button,
3 // load and render the image
4 $("#select-file-image").change(function() {
5 renderImage(this.files[0]); Contents X
6 }
Basic MobileNet in Python
7
8 // renders the image which is loaded from disk to the imgConvert
tag Keras model into Tf
9 function renderImage(file) { JS layers format
10 var reader = new FileReader(); Keras model into TensorFlow
11 reader.onload = function(event) { JS
12 img_url = event.target.result; 1. Load Keras model into
13 document.getElementById("test-image").src = img_url;
TF.js
14 }
reader.readAsDataURL(file); 2. Upload image from disk
15
} 3. Predict using MobileNet
model
References

3. Predict using MobileNet model

To make predictions using mobilenet that’s now loaded into Tf.js environment, we
need to perform two steps.

1. Preprocess the input image to be mobilenet friendly.

2. Make predicitons on the input image.

3.1 Preprocess the input image

As I have already mentioned, input image size to mobilenet is [224, 224] as well as
the features are scaled between [-1, 1]. You need to perform these two steps before
making predictions using the model. To do this, we use preprocessImage() function
that takes in two arguments image and modelName .

The input image can easily be loaded using tf.fromPixels() , resized using
resizeNearestNeighbor() and converting all the values in the image to float using
toFloat() .

After that, we feature scale the values in the image tensor using a scalar value of
127.5 which is the center value of image pixel range [0, 255]. For each pixel value in
the image, we subtract this offset value and divide by this offset value to scale
between [-1, 1]. We then expand the dimensions using expandDims() .

https://gogul.dev/software/mobile-net-tensorflow-js 11/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

mobile-net.js code

1 // preprocess the image to be mobilenet friendly Contents X

2 function preprocessImage(image, modelName) {
Basic MobileNet in Python
3
4 // resize the input image to mobilenet's target size ofConvert
(224, Keras
224) model into Tf
5 let tensor = tf.browser.fromPixels(image) JS layers format
6 .resizeNearestNeighbor([224, 224]) Keras model into TensorFlow
7 .toFloat(); JS
8 1. Load Keras model into
9 // if model is not available, send the tensor with expanded dimensions
TF.js
10 if (modelName === undefined) {
11 return tensor.expandDims(); 2. Upload image from disk
12 } 3. Predict using MobileNet
13 model
14 // if model is mobilenet, feature scale tensor image toReferences
range [-1, 1]
15 else if (modelName === "mobilenet") {
16 let offset = tf.scalar(127.5);
17 return tensor.sub(offset)
18 .div(offset)
19 .expandDims();
20 }
21
22 // else throw an error
23 else {
24 alert("Unknown model name..")
25 }
26 }

3.2 Predict using Tf.js model

After preprocessing the image, I have made a handler for Predict button. Again, this
is also an async function that uses await till the model make successfull
predictions.

Prediction using a Tf.js model is straightforward as Keras which uses

model.predict(tensor) . To get the predictions, we pass it data() to the former.

Results from the predictions are mapped to an array named results using
IMAGENET_CLASSES that we loaded at the beginning of this tutorial. We also sort this
array based on the probability that is highest using sort() and take only the top-5
probabilities using slice() .

https://gogul.dev/software/mobile-net-tensorflow-js 12/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

mobile-net.js code

1 // If "Predict Button" is clicked, preprocess the image Contents

and X
2 // make predictions using mobilenet
Basic MobileNet in Python
3 $("#predict-button").click(async function () {
4 // check if model loaded Convert Keras model into Tf
5 if (model == undefined) { JS layers format
6 alert("Please load the model first..") Keras model into TensorFlow
7 } JS
8 1. Load Keras model into
9 // check if image loaded
TF.js
10 if (document.getElementById("predict-box").style.display == "none") {
11 2. Upload
alert("Please load an image using 'Demo Image' or 'Upload imagebutton.
Image' from disk
12 } 3. Predict using MobileNet
13 model
14 // html-image element can be given to tf.fromPixels References
15 let image = document.getElementById("test-image");
16 let tensor = preprocessImage(image, modelName);
17
18 // make predictions on the preprocessed image tensor
19 let predictions = await model.predict(tensor).data();
20
21 // get the model's prediction results
22 let results = Array.from(predictions)
23 .map(function (p, i) {
24 return {
25 probability: p,
26 className: IMAGENET_CLASSES[i]
27 };
28 }).sort(function (a, b) {
29 return b.probability - a.probability;
30 }).slice(0, 5);
31
32 // display the top-1 prediction of the model
33 document.getElementById("results-box").style.display = "block";
34 document.getElementById("prediction").innerHTML = "MobileNet prediction - <
35
36 // display top-5 predictions of the model
37 var ul = document.getElementById("predict-list");
38 ul.innerHTML = "";
39 results.forEach(function (p) {
40 console.log(p.className + " " + p.probability.toFixed(6));
41 var li = document.createElement("LI");
42 li.innerHTML = p.className + " " + p.probability.toFixed(6);
43 ul.appendChild(li);
44 });
45 });

https://gogul.dev/software/mobile-net-tensorflow-js 13/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

There you go! We now have the power of state-of-the-art Keras pretrained model
MobileNet in a client browser that is able to make predictions on images that
belong to ImageNet category.
Contents X

Basic MobileNet in Python

Notice that the mobilenet model loads very quickly in the browser and makes
Convert Keras model into Tf
predictions very fast 😎 JS layers format
Keras model into TensorFlow
JS
References 1. Load Keras model into
TF.js
2. Upload image from disk
1. TensorFlow.js - O cial Documentation
3. Predict using MobileNet
2. Keras - O cial Documentation
model
3. Importing a Keras model into TensorFlow.js References
4. Introduction to TensorFlow.js - Intelligence and Learning
5. TensorFlow.js: Tensors - Intelligence and Learning
6. TensorFlow.js Quick Start
7. Session 6 - TensorFlow.js - Intelligence and Learning
8. Session 7 - TensorFlow.js Color Classi er - Intelligence and Learning
9. Tensor ow.js Explained
10. Webcam Tracking with Tensor ow.js
11. Try TensorFlow.js in your browser

In case if you found something useful to add to this article or you found a
bug in the code or would like to improve some points mentioned, feel free
to write it down in the comments. Hope you found something useful here.

10 Comments

© 2019 - gogul ilango | opinions are my own

https://gogul.dev/software/mobile-net-tensorflow-js 14/15
12/16/2019 Classifying images using Keras MobileNet and TensorFlow.js in Google Chrome – Gogul Ilango

https://gogul.dev/software/mobile-net-tensorflow-js 15/15