MLDD 1
MLDD 1
MACHINE LEARNING
Notice of Rights
All rights reserved. No part of this book may be reproduced, stored in a retrieval
system or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embodied in
critical articles or reviews.
Notice of Liability
The author and publisher have made every effort to ensure the accuracy of the
information herein. However, the information contained in this book is sold
without warranty, either express or implied. Neither the authors and SitePoint
Pty. Ltd., nor its dealers or distributors will be held liable for any damages to be
caused either directly or indirectly by the instructions contained in this book, or
by the software or hardware products described herein.
Trademark Notice
Rather than indicating every occurrence of a trademarked name as such, this
book uses the names only in an editorial fashion and to the benefit of the
trademark owner with no intention of infringement of the trademark.
A Deep Dive into Machine Learning iii
About SitePoint
SitePoint specializes in publishing fun, practical, and easy-to-understand content
for web professionals. Visit sitepoint.com to access our blogs, books,
newsletters, articles, and community forums. You’ll find a stack of information on
JavaScript, PHP, Ruby, mobile development, design, and more.
iv A Deep Dive into Machine Learning
Table of Contents
Preface ......................................................................................................................... vii
Machine ......................................................................................................................... 8
Big Picture .............................................................................................................................. 9
Code ........................................................................................................................................ 14
Python ........................................................................................................................... 19
Pytorch ................................................................................................................................... 21
Chapter 8: Caffe2..........................................................................................26
IBM Watson..........................................................................................................................38
ML Kit ......................................................................................................................................44
Preface vii
Preface
This little ebook began its life as an email course published by SitePoint. It
provides a beginner-friendly introduction to machine learning, shining a torch
across the landscape of machine learning and presenting lists of tools and
resources for delving into this cutting-edge field.
viii A Deep Dive into Machine Learning
Chapter
Become a
Machine
Learning
Learning
1
Machine
Become a Machine Learning Learning Machine ix
There are thousands and thousands of start-ups and services which claim to
harness machine learning (or neural networks or deep learning—pick your
buzzword). Sometimes, their ML tech is nothing more than a glorified chat bot
that’ll take a narrow range of inputs and come back with pre-chosen responses.
Other times, it’s genuinely impressive, or scary, what this stuff can do. It’s
sometimes hard, given the hype, to work out which is which.
But if you have any interest at all in the bleeding edge of technology, you should
learn what machine learning is, and how to spot the legit ML tech. In this book, I’ll
do my best to help.
Big Picture
Wait But Why’s post on the coming artificial intelligence revolution
[waitbutwhy] is a typically excellent, good first step when thinking about AI
and its potential impact on the world.
One that wasn’t included: AlphaGo Zero [deepmind], the network that learned
to be the best Go player in the world by playing itself.
A Reddit thread of answers to the question: If you had to show one paper to
someone to show that machine learning is beautiful, what would you choose?
[reddit].
x A Deep Dive into Machine Learning
Building Blocks
Some resources to help you frame everything else you see:
Real-world Inspiration
I found this piece from Robin Sloan [robinsloan], where he used machine
learning to turn his text editor into a science fiction expert, able to
autocomplete appropriately sci-fi stuff, hugely inspiring. It’s not AlphaGo-level
abilities, but it does make the subject much more accessible to my mind.
Chapter
Algorithm and
Blues
2
xii A Deep Dive into Machine Learning
If you a) have some more time, and b) enjoy math, the book The Mathematics
of Machine Learning [github/mml-book] will be right up your alley, offering the
mathematical foundations of ML, plus algorithms that use these.
Present Tense: TensorFlow xiii
Chapter
Present Tense:
TensorFlow
3
xiv A Deep Dive into Machine Learning
Learning
Here’s a TensorFlow tutorial [github/aymericdamien] full of examples to get
you started.
Code
The Ruby implementation of TensorFlow [github/somaticio].
Chapter
TensorFlow.js
and Machine
Learning with
JavaScript
4
TensorFlow.js and Machine Learning with JavaScript xvii
First up, this article [hackernoon] does a good job of busting some myths, and
providing a gentle intro to ML with JS.
TensorFlow.js
TensorFlow.js is an open-source machine learning library, allowing devs to use
the high-level API to train and run machine learning models in browser (and also
Node, as of a recent update). These models use the client’s GPU, and can be
easily added to a project with a script tag. If you like, you can also convert a pre-
trained model from TensorFlow and Keras to the TensorFlow.js format.
And a video guide to using Brain.js to distinguish between light or dark colors
[youtube].
Solid Snake: Machine Learning with Python xix
Chapter
Solid Snake:
Machine
Learning with
Python
5
xx A Deep Dive into Machine Learning
From what I’ve learned in my research for this book, when it comes to machine
learning code, Python is just the most popular choice. It’s relatively readable and
simple to understand, which means users can spend less of their energy
worrying about syntax errors. It also has a bunch of useful libraries available out
of the box, and its popularity creates a virtuous circle, so a bunch of open-source
ML libraries have been created by the community. Here’s a good summary
[hub.packtpub] of these and other arguments.
Now, let’s take a look at what’s out there for the budding Python machine
learning developer.
100 Days of ML Code [github/avik-jain] uses Python code for the exercises, so
is a good way to go deep on both ML and Python’s place in it.
If you’re enjoying practicing machine learning, and you’re thinking this may
become an actual pursuit/career for you, here are five useful books for
learning machine learning in Python [quantstart].
Solid Snake: Machine Learning with Python xxi
Pytorch
PyTorch is a Python library for machine learning, built by Facebook and based on
Lua-based machine learning library Torch (which we’ll get into later on in the
Deep Dive). I thought it made sense to look into this while we have a Python
focus.
Chapter
Lastly for this chapter, TorchCraft [github/torchcraft] lets you use a machine
learning network to play StarCraft. A complete overview of the project [arxiv].
xxiv A Deep Dive into Machine Learning
Chapter
Keras
7
Keras xxv
Chapter
Caffe2
8
Caffe2 xxvii
Caffe Model Zoo [github/caffe2] is a repo full of pre-trained networks that can
be used immediately. An essential visit for the budding Caffe-enthusiast. A
guide to using them [caffe2].
Chapter
CNTK, OK?
9
CNTK, OK? xxix
The official model gallery [microsoft], full of examples and tutorials for using
CNTK.
Chapter
Apache Spark
MLlib
10
Apache Spark MLlib xxxi
To close out the Machine Learning Deep Dive, I’ll be taking a look at two areas:
cloud-based machine learning integrations and moving machine learning to
mobile.
Cloud-based ML lets you take the machine learning principles and libraries
you’ve grown accustomed to and vastly increase your resources. There are a few
competitors in this space, all established names. Here’s a comparison of the
options I’ll cover—although it doesn’t include this chapter’s entry. Speaking of
which …
It doesn’t have the catchiest name, but Spark MLlib is a simple, scalable machine
library, built on top of the Spark cluster computing framework. MLlib plays nice
with popular ML languages/frameworks and runs very, very fast. Let’s take a
look.
But why choose just one? Using MLlib and TensorFlow together [adaltas].
Chapter
Azure Machine
Learning Service
11
Azure Machine Learning Service xxxiii
Azure Machine Learning Service is Microsoft’s entry into the cloud machine
learning game. It has a Python API that can plug into existing machine learning
libraries like TensorFlow and PyTorch, and you can develop locally and then scale
up. This is in contrast to its Machine Learning Studio product, which offers a
more abstract, drag-and-drop interface for working with machine learning
models, no coding required.
Use deep learning in Azure ML Service to analyze the human footprint in the
Amazon rainforest [medium/@weshoffman].
xxxiv A Deep Dive into Machine Learning
Chapter
Google Cloud
ML
12
Google Cloud ML xxxv
Cloud ML Engine
An introduction to Cloud ML Engine [bmc].
Cloud TPUs
Google also offers TPUs, a class of application-specific circuit custom built to
accelerate machine learning. You can use these in your own work, provided
you’re a Google Cloud customer, via its Cloud TPU [cloud.google] service.
An intro [bmc] to exactly what TPUs are, and why they’re handy for machine
learning.
Reference models and tools for Cloud TPUs [github/tensorflow]. For example,
an implementation of the RetinaNet object detection model [github/
tensorflow].
Speaking of, how to train and serve a realtime mobile object detector in 30
minutes [medium/tensorflow].
Chapter
Amazon
SageMaker and
IBM Watson 13
xxxviii A Deep Dive into Machine Learning
Amazon SageMaker
SageMaker is Amazon’s low-level machine learning platform that allows
developers to use familiar frameworks to build and run machine learning
projects on AWS. Amazon has more user-friendly, plug-and-play, no-code-
required tools like image and video analyzer Rekognition, but SageMaker is for
those who want to get their hands dirty.
The full “how it works” docs are worth reading [docs.aws.amazon]. I found the
“batch transform” tool [docs.aws.amazon] particularly interesting.
Is it right for you? [hackernoon]. Takeaways for me: it’s pretty easy to get
started, but potential dealbreakers include the expense and the (possibly)
time-consuming data uploads you’ll need to plan for.
IBM Watson
You might remember Watson as the system that beat humans at Jeopardy. IBM
has taken that system, or at least its brand name, and built it out into tools for
businesses, analysts and data scientists. The landscape here is a bit confusing,
with many similarly named services, and some just aren’t aimed at those wanting
to build their own models from scratch. Here are a few links that may help you
though.
Thanks to a partnership with Apple, Watson has also been integrated with the
iOS CoreML framework [medium/flawless-app-stories]. Funnily enough, we’re
about to look at mobile machine learning, so consider that an excellent segue.
xl A Deep Dive into Machine Learning
Chapter
Core ML 2
14
Core ML 2 xli
How to build Silicon Valley’s Not Hotdog app with Core ML in an afternoon
[heartbeat.fritz].
A big list of Core ML models [github/likedan]. That page also has a list of
models that could be converted to Core ML, so get cracking. Existing models I
liked the look of:
An interesting piece with some implications for the security of your models:
xlii A Deep Dive into Machine Learning
Chapter
Android:
TensorFlow Lite
and ML Kit 15
xliv A Deep Dive into Machine Learning
Ah, Android. When compared to the polish of the iOS ecosystem, things always
tend to get a little more diffuse, and machine learning tooling is no exception.
Let’s take a look at the two options you have for Android machine learning:
TensorFlow Lite and ML Kit. Here’s a good intro [medium/xebia-france] to the
two sides of the coin, complete with their respective limitations.
TensorFlow Lite
To take this Deep Dive full-circle, the official solution for running machine
learning models on Android is TensorFlow Lite tensorflow].
And a guide to using a TensorFlow Lite model for inference with ML Kit
[firebase.google].
ML Kit
Google’s ML Kit from Firebase is a machine learning SDK for Android and iOS. It’s
in beta, and was released at 2018’s Google I/O. It’s more user-friendly than TF
Lite, so might be a good place to start.
An intro [auto0].
Finally, a guide to using TensorFlow Lite and ML Kit to build a Pokidex [github/
the-dagger].