Introduction to Deep Learning and Neural Networks with

Python A Practical Guide

Visit the link below to download the full version of this book:

https://medidownload.com/product/introduction-to-deep-learning-and-neural-networ
ks-with-pythontm-a-practical-guide/

Click Download Now

Preface

Deep learning is the state-of-art technique for solving data-oriented real-world

problems. It proves its effectiveness in challenging applications like sentiment
analysis, text summarization, text translation, object detection and recognition,
image classification, speech synthesis, and many more.
To design a deep learning model to solve a problem, it is critical to have knowl-
edge about how things work behind the scenes. It is not just about using the deep
learning model as a black box. Both the theory and practical sides are important.
Introduction to Deep Learning and Neural Networks with Python™: A
Practical Guide is an intensive step-by-step guide for absolute beginners to ex-
plore how artificial neural networks (ANNs) work which are the basis of the
deep learning models. The book moves from the lowest possible level that starts
with the linear model Y = X until building a complete and generic neural network
that works regardless of the number of inputs, outputs, samples, hidden layers,
and hidden neurons.
The book discusses both the math behind each step in training the ANN and
then shows how it is implemented in Python. So, some little knowledge about
Python (and NumPy) is preferred.
Because building an implementation of the ANN from zero to hero is not
an easy task, the book works in a hierarchical approach that starts with some
specific and simple networks and then moves to generalize the network to work
with any architecture. Throughout the journey, many concepts are discussed
including train and test data, forward pass, backward pass, hidden layer, hidden
neurons, activation function, learning rate, gradient descent, sum of products,
and more.
Here is a description of the 11 chapters:
Chapter 1 prepares the development environment used throughout the book
where Python and the required libraries are installed. A virtual machine is cre-
ated to run Ubuntu for building Android apps out of the Python apps.
Chapter 2 discusses the basics of neural networks with the nonparametric
linear model Y = X. The weight and bias are introduced in the parametric linear
model Y = wX + b. The chapter works through some math examples manually to
explain how the training process works from applying the inputs until calculat-
ing the error. Out of the linear model, the graphical representation of a single
neuron is derived. The chapter also discusses how the sum of products at a neu-
ron is calculated in addition to the importance of activation functions.

xi
xii  Preface

Chapters 3–10 work closely on how neural networks work in addition to

their Python implementation. Each chapter starts where the previous one ends.
All of these chapters discuss the following:
1. Network architecture
2. Weights initialization
3. Forward pass
4. Backward pass
In Chapter 3, a network with a single input and single out is created without
using any hidden layers. The chapter explores how to make a prediction by
generating the output from the input, calculate the loss of the network, derive
the derivative of the sigmoid function, and calculate the gradient of the weights
using the chain of derivatives.
Rather than using just a single input, Chapter 4 uses 10 inputs. The chain of
derivatives of all weights are prepared to calculate the weights' gradients. The
chapter ends by working with any number of inputs.
Chapter 5 adds a single hidden layer with two neurons to the network archi-
tecture. The math calculations of the hidden neurons in both the forward and
backward passes are covered.
Chapter 6 starts by extending the number of hidden neurons to five and ends
by building a hidden layer that uses any number of neurons.
Chapter 7 adds a new hidden layer to the network so that it works on two
hidden layers. This chapter discusses two examples with a fixed number of hid-
den neurons and ends by working with any number of neurons in two hidden
layers. It builds a generalized solution to weights initialization.
Chapter 8 adds an additional third hidden layer where any number of neu-
rons can be used in the three hidden layers. It also generalizes the Python imple-
mentation of the forward pass so that it works with any number of hidden layers.
Chapter 9 reached a milestone by editing the latest code to work with any
number of hidden layers in both the forward and backward passes. The process
of calculating the gradients of all layers is automated without being restricted to
any number of hidden layers or neurons. This chapter reformats the project so
that a Python module named MLP.py is created to hold the backend operations
while giving the user a simple interface to build and use the neural network.
Chapter 10 adds additional features to the project so that it works with any
number of outputs and multiple training samples. It also supports the rectified
linear unit activation function in both the forward and backward passes. The
bias is introduced as an additional parameter to the weight. Finally, the gradient
descent can work with either the stochastic or batch modes.
Chapter 11 uses a framework called Kivy and KivyMD to build a desk-
top application that builds and trains a neural network using custom data. An
Android application is generated from Kivy using the Buildozer tool. The reader
is expected to have no knowledge of Kivy.
The source code of the book is available at this GitHub project: https://github.
com/ahmedfgad/IntroDLPython. Each chapter has its code within a separate folder.
Acknowledgments

Thanks, Allah, for inspiring us to write this book. It was a dream that comes into
action. We should clearly admit that our success is not only about being skillful
in something but the willingness of Allah [And my success is not but through
Allah. Upon him I have relied, and to Him I return. (Hud 88)].
We are keeping thanking Allah as David and Solomon previously did in
Surat An Naml 15 [We gave (in the past) knowledge to David and Solomon:
And they both said: “Praise be to Allah, Who has favoured us above many of his
servants who believe!” (An Naml 15)].
Thanks to our parents who did everything possible to help us in our life with-
out expecting a reward. While you surround us at home, we are safe and feel that
nothing can beat us. We hope to be grateful to you as said in Surat Luqman 14
[Be grateful to Me and to your parents; to Me is the [final] destination].
It is a wonderful experience to have our book published by Elsevier. Natalie
Farra, an acquisition editor, is a friendly person who cares about the subtle de-
tails. I never had a question that was not answered accurately in adequate time.
Sara Pianavilla, editorial project manager, supported us from submitting the
content to preparing the cover in a smooth process. Thanks to Punitha Radjane
for the information about the proofing system.
Motivated by this Hadith [AbuHurayrah reported that the Prophet Mohamed
(‫ )ﷺ‬said, “He who does not thank people does not thank Allah.”], we sepa-
rately thank the people who motivated and helped us to complete this book.

Ahmed Fawzy Gad

From time to time, new people arise in our life to help us proceed and follow the
right path. The first person who helped me to prepare this book is my coauthor
Fatima Ezzahra Jarmouni. You are an intelligent scientist with lots of inspiration
to stir the stagnant water. A few words cannot express my appreciation for your
kindness and willingness to help other people just for the sake of help.
Thanks, Dr. Rasha Atwa, a staff member at King Abdulaziz University, for
planting hope again in my heart. You are a kind student, teacher, mother, and hu-
man with a lot of experience. I am sure you are strong enough to help yourself,
your children, family, and even other people like me. Whatever good you do will
be rewarded by Allah [And whatever good you put forward for yourselves – you
will find it with Allah. (Al Muzzammil 20)].

xiii
xiv Acknowledgments

Life has many kind people and Prof. Amiya Nayak, a lecturer at the
University of Ottawa, is one of them. You are an initiator who saves no effort in
pushing and assisting me to continue my career.
Thanks to my childhood friend Ibrahim Elhagali for the continued support
and enthusiasm to push me emotionally. Keep up the good work you are doing
and I am confident that you will go beyond our expectations.
Thanks to Paperspace represented by its CEO Dillon Erb, COO Daniel
Kobran, and community manager Rachel Rapp for hosting my works. The same
goes for the Fritz AI team lead by CEO Dan Abdinoor, CTO Jameson Toole,
and head of Heartbeat community Austin Kodra. Feeling that you appreciate my
work is a key driver to write this book.
Thanks to my mother, father (may Allah have mercy on him), brother, sis-
ters, and all family members for standing with me. You relieved much load in
the last years and I hope I can do so in the future.

Fatima Ezzahra Jarmouni

It is a wonderful experience to work on my first book and will remain an achieve-
ment in my life. I thank my coauthor Ahmed Gad for giving me the chance to
participate in this book. You shared your experience and gave me enough sup-
port and advice. I liked working with you and I hope it is not the last time.
This book was not possible without the support I received from my friend
Batoul Badawi. You are much close to my heart like how our homes are. Every
meter in the long-distance to Mona Elmorshedy carries emotions and feelings.
Despite the distance, you are so close to my heart and I am looking for the mo-
ment to meet you and your daughter Deeda.
The time I enjoy with my colleagues Fatima Walid, Kaoutar Abgar, and
Fatima Ezzahra Elmortadi made a noticeable difference in adjusting my mood.
The crazy moments we had together refresh me to renew my energy to study,
work, and live.
The existence of my mother, father, brothers, and sisters gives me more con-
fidence and power. You always support me in all aspects of life, and I hope I can
return a little bit of your assistance one day.
Chapter 1

Preparing the development

environment
Chapter outline
Downloading and installing Python™ 3 1 Preparing Ubuntu® virtual
Installing required libraries 5 machine for PyPy 13
Preparing Ubuntu® virtual Conclusion 14
machine for Kivy 6

ABSTRACT
This chapter prepares the development environment that is used in the following chap-
ters. The book uses Python 3; thus, the chapter discusses how to download and install it.
On top of Python, a number of libraries are necessary to be installed. These libraries
include NumPy, Matplotlib, Cython, and Kivy. For building mobile applications out of
Kivy, a Linux system must exist. For this purpose, an Ubuntu virtual machine is prepared.
The chapter also gets the reader ready to use the recent Python implementation called
“PyPy” to see how it could boost the performance of Python scripts.

Downloading and installing Python™ 3

Python is one of the easiest languages to learn as it takes less overhead to make
a program that runs. This helps the beginners get started quickly without caring
about the details as in other languages, such as C or Java.
Python is a cross-platform language by which the same program is writ-
ten once and used by different operating systems. Python is a general-purpose
language that works on different platforms such as desktop, mobile, web, and
embedded systems. Thus, different types of applications can be built for differ-
ent platforms without having to use different languages.
Python has two main releases. The first one is Python 2 with its latest ver-
sion 2.7.18 released on April 2020, which just fixes some bugs in Python 2.7.17
released on October 19, 2019. Starting from January 1, 2020, Python 2 is no
longer being developed further as the Python team found serious issues that
cannot be solved. Note that Python 2.7.18 was developed in the period from
October 2019 to January 2020 but just released after that.

Introduction to Deep Learning and Neural Networks with Python™

https://doi.org/10.1016/B978-0-323-90933-4.00002-4 1
© 2021 Elsevier Inc. All rights reserved.
2 Introduction to deep learning and neural networks with Python™

After the Python 2 sunset date (January 1, 2020), Python 3 is the only sup-
ported version of Python. Its development started in 2008. There might still be
some Python 2 users but by time, Python 2 will die and only Python 3 will exist.
For that reason, this book uses Python 3.
The book source code is developed using Python 3.7. The latest Python 3
version is 3.8, which can still be used to run the book’s code without changes.
At the time of writing this chapter, the latest release of Python 3 (3.8.5) is pub-
lished on July 2, 2020.
The official Python website is python.org from which Python 3 can be
downloaded from the downloads page. To download Python 3.8.5, use this link:
python.org/downloads, which forwards to a page like that in Fig. 1.1. To down-
load Python 3.8.5 for Windows, use this link: https://www.python.org/ftp/py-
thon/3.8.5/python-3.8.5.exe. Its size is about 25 MB.
For Linux, Python 3 comes already installed. To upgrade to a specific ver-
sion, use this terminal command:
sudo apt install python3.8
For Mac, use this command:
brew install python3
After Python 3.8 is installed, one way of accessing it is through the terminal/
command prompt. For Windows and Mac, issue the python command as in
Fig. 1.2. For Linux, issue the python3 command.

FIG. 1.1 Download Python 3.8 from python.org

FIG. 1.2 Activate Python.

 Preparing the development environment Chapter | 1 3

The installed Python from python.org is a native Python distribution, which

only supports the built-in libraries. But there is a wide range of libraries that
support useful features that are regarded as essential to many Python develop-
ers. One of these libraries is Numerical Python (NumPy).
Python supports a tool called “pip” to install new libraries. The next com-
mand is used to install new libraries where < library-name > is replaced by
the library named (e.g. numpy). Remember to use pip3 for Linux and Mac.
pip install < library-name >
For each new library to be installed, a pip command is issued. Some errors
may occur while installing some libraries and it takes time to solve them. To ease
the support of the Python libraries, there are Python distributions that come with
many already installed Python libraries. One of these distributions is Anaconda
anaconda.com. It supports dozens of already installed Python libraries.
Anaconda supports four editions, which are individual, team, enterprise, and
professional. The individual edition is enough for this book. Fig. 1.3 shows that
Anaconda supports Windows, Mac, and Linux. Visit anaconda.com/products/
individual and choose the right installer according to your platform. For ex-
ample, the Python 3.8 download link for 64-bit Windows is repo.anaconda.com/
archive/Anaconda3-2020.07-Windows-x86_64.exe.
Compared to the size of the installer of the native Python distribution, the
Anaconda installer size is hundreds of megabytes as some libraries are already
packaged.
By installing Anaconda, not only Python will only be installed but also many
libraries will be available for use without having to install them.
Anaconda comes with an integrated development environment (IDE) called
“Spyder.” It is the best IDE for the MATLAB users moving to Python. Another

FIG. 1.3 Anaconda installers.

4 Introduction to deep learning and neural networks with Python™

popular IDE is PyCharm, which can be downloaded from this page jetbrains.
com/pycharm/download.
The Jupyter project is also a favorable choice for data scientists as it offers
notebooks in which not only code but also text and images can be embedded.
The Jupyter Notebook comes already installed in Anaconda. To activate it, issue
the next command from the terminal. It activates a server that handles a web-
based Jupyter Notebooks.
jupyter notebook
After the server runs, a new web page opens, as in Fig. 1.4, from which
Jupyter Notebooks can be created.
From the New button, select the version of Python used in the notebook and
it opens in a new browser tab as in Fig. 1.5.
Similar to the pip installer in the native Python distribution, Anaconda sup-
ports a new installer called “conda.” One of the differences between the two
installers is the cloud-based repository from which the installer searches for the
libraries. pip searches for the libraries in pypi.org where conda searches in
repo.anaconda.com. Note that pip can still be used to install libraries in the
Anaconda environment.
If conda is used to install a nonconda library (i.e., a library that is not available
at the Anaconda package repository), an error occurs similar to that in Fig. 1.6.

FIG. 1.4 Activating Jupyter Notebooks.

FIG. 1.5 Creating a New Jupyter Notebook.

 Preparing the development environment Chapter | 1 5

FIG. 1.6 Error installing a nonconda library using conda.

After Python is installed, the next section discusses the libraries that will be
used in the book.

Installing required libraries

Besides the native Python, some more libraries are required in this book. Here
is a list of these libraries:
● NumPy
● Matplotlib
● Cython
● Virtualenv
● Kivy
● KivyMD
● Buildozer
If Anaconda is used, then NumPy, Matplotlib, Cython, and Virtualenv will
be already installed. For the native Python distribution, then all of these libraries
must be installed using pip. For the complete list of the packages supported by
default in different Anaconda distributions, check this link docs.anaconda.com/
anaconda/packages/pkg-docs. For Python 3.8, the list is accessible from this
page docs.anaconda.com/anaconda/packages/py3.8_win-64.
NumPy is a library that supports the ndarray data type to process numeri-
cal arrays in Python. It is a very popular library in data science. Matplotlib is
used to create visualizations. Cython is a superset of Python to support the use
of C code within Python, which speeds-up the Python execution. Virtualenv
creates virtual environments where Python and its libraries can be installed.
Such virtual environments are isolated from the main Python installation. Kivy
is a cross-platform library for building native user interfaces. KivyMD offers
6 Introduction to deep learning and neural networks with Python™

a number of widgets that are compatible with Kivy and approximate Google’s
Material Design spec. Buildozer is a tool for building the Kivy applications for
mobile devices.
Kivy applications can be developed in Linux, Mac, and Windows. Kivy can
be installed using pip. Use pip3 rather than pip for Linux and Mac.
pip install kivy
Similarly, install KivyMD:
pip install kivymd
For deploying the Kivy applications to Android, which is the focus of this
book, only the Linux and OSX platforms support Buildozer. For iOS, only
OSX can be used. The next section prepares the Kivy environment in Linux for
building Android applications.

Preparing Ubuntu® virtual machine for Kivy

This section assumes that the user is running a different operating system (OS)
than Linux and thus prepares a virtual machine in which Ubuntu is installed.
If Linux is available on your machine (either as the main operating system or
as a virtual machine (VM)), then skip the section. Otherwise, please keep read-
ing the entire section. In our case, the main OS is Windows and Linux Ubuntu
is installed in a virtual machine.
The first thing to do is to download a Linux distribution (e.g., Ubuntu) to run
on a desktop PC. Visit this page ubuntu.com/download/desktop as in Fig. 1.7
and download Ubuntu. The download link for Ubuntu 20.04, the latest version
at the time of writing this chapter, is releases.ubuntu.com/20.04/ubuntu-20.04-
desktop-amd64.iso. The downloaded file is an ISO image of the OS ready for
being installed on virtualization software. Remember the location in which the
image is downloaded to use it later.

FIG. 1.7 Download Ubuntu for desktop.

 Preparing the development environment Chapter | 1 7

FIG. 1.8 Download VMware Workstation Pro.

Use the virtualization software of your choice. Examples include VMware

and VirtualBox. VMware is paid and available for trial for 30 days, but VirtualBox
is free. VMware Workstation Pro 15.5.1 is used in the book. Download the soft-
ware from this page vmware.com/mena/products/workstation-pro/workstation-
pro-evaluation.html as shown in Fig. 1.8.
Install VMware and open it. Fig. 1.9 shows how its home screen looks like.
Click on the Create a New Virtual Machine button to create a VM. The
same thing is achieved from the New Virtual Machine option in the File menu.
A new window appears as in Fig. 1.10. The steps to create the VM are not
complicated but it does not hurt to mention it. There is no need to customize the
configuration; thus, the Typical option is selected.

FIG. 1.9 VMware Workstation Pro Main Screen.

8 Introduction to deep learning and neural networks with Python™

FIG. 1.10 Creating a New VM.

Click Next to move to the window in Fig. 1.11 where the directory of the
downloaded Ubuntu ISO image is specified.
Click Next to open a new window as in Fig. 1.12 to create a new user by
entering the username and password. Remember them to log in to the OS after
being installed.
By clicking Next, the window in Fig. 1.13 appears to enter the VM name and
its location where all VM files are saved. Later, this location may be copied to
create a backup of the VM.
Click Next to move to the window in Fig. 1.14 to specify the disk size of the
VM. It defaults to 20 GB but you can increase it as much as you can to make
a room for the files and libraries downloaded later. If possible, set the size to
40 GB.
After clicking Next, a final Window appears summarizing the settings of the
VM as in Fig. 1.15. By default, the machine uses 2 GB of RAM, one processor,
and one core per processor. These settings are not the best.
Click the Customize Hardware button to customize these settings as in
Fig. 1.16. According to your physical RAM size, increase the RAM assigned
to the VM. In our case, 6 out of 16 GBs are assigned to the VM. From the
Processors tab, increase the number of cores assigned to the processor to
nearly half.
 Preparing the development environment Chapter | 1 9

FIG. 1.11 Specify the Ubuntu ISO Image Directory.

FIG. 1.12 VM username and password.

10 Introduction to deep learning and neural networks with Python™

FIG. 1.13 VM name and location.

FIG. 1.14 VM disk size.

 Preparing the development environment Chapter | 1 11

FIG. 1.15 Create the VM.

FIG. 1.16 Change the VM hardware settings.