5.

NEURAL NETWORKS AND DEEP LEARNING

INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS WITH KERAS

The term "Artificial Neural Network" is derived from Biological neural networks that develop the
structure of a human brain. Similar to the human brain that has neurons interconnected to one another,
artificial neural networks also have neurons that are interconnected to one another in various layers of the
networks. These neurons are known as nodes. Below illustrates the typical diagram of Biological Neural
Network.

The typical Artificial Neural Network is as follows

Dendrites from Biological Neural Network represent inputs in Artificial Neural Networks, cell
nucleus represents Nodes, synapse represents Weights, and Axon represents Output. Relationship
between Biological neural network and artificial neural network is as follows:

Biological Neural Network Artificial Neural Network

Dendrites Inputs
Cell nucleus Nodes
Synapse Weights
Axon Output

1
Architecture of an Artificial Neural Network:
Artificial Neural Network primarily consists of three layers:

1. Input Layer: It accepts inputs in several different formats

2. Hidden Layer: The layer present in-between input layer and output layer is the hidden layer. It
performs all the calculations to find hidden features and patterns.
3. Output Layer: The input goes through a series of transformations using the hidden layer, which finally
results in output that is conveyed using this layer.

The Artificial Neural Network takes input and computes the weighted sum of the inputs and includes
a bias. This computation is represented in the form of a transfer function.

Here, Wi is Weight, Xi is input value & b is bias.

Types of Artificial Neural Network (ANN):

1. Feedforward Neural Network: The feedforward neural network is one of the most basic artificial
neural networks. In this ANN, the data or the input provided travels in a single direction. It enters
into the ANN through the input layer and exits through the output layer while hidden layers may
or may not exist. So the feedforward neural network has a front propagated wave only and usually
does not have backpropagation.
2. Recurrent Neural Network: The Recurrent Neural Network saves the output of a layer and feeds
this output back to the input to better predict the outcome of the layer. The first layer in the RNN
is quite similar to the feed-forward neural network and the recurrent neural network starts once the
output of the first layer is computed. After this layer, each unit will remember some information
from the previous step so that it can act as a memory cell in performing computations.

2
 3. Convolutional Neural Network: A Convolutional neural network has some similarities to the feed-
forward neural network, where the connections between units have weights that determine the
influence of one unit on another unit. But a CNN has one or more than one convolutional layers
that use a convolution operation on the input and then pass the result obtained in the form of
output to the next layer. CNN has applications in speech and image processing which is
particularly useful in computer vision.
4. Modular Neural Network: A Modular Neural Network contains a collection of different neural
networks that work independently towards obtaining the output with no interaction between them.
Each of the different neural networks performs a different sub-task by obtaining unique inputs
compared to other networks. The advantage of this modular neural network is that it breaks down
a large and complex computational process into smaller components, thus decreasing its
complexity while still obtaining the required output.
5. Radial basis function Neural Network: Radial basis functions are those functions that consider the
distance of a point concerning the center. RBF functions have two layers. In the first layer, the
input is mapped into all the Radial basis functions in the hidden layer and then the output layer
computes the output in the next step. Radial basis function nets are normally used to model the
data that represents any underlying trend or function.

Applications of Artificial Neural Networks (ANN):

 Social Media
 Marketing and Sales
 Healthcare
 Personal Assistants

Advantages of Artificial Neural Network (ANN):

 Parallel processing capability
 Storing data on the entire network
 Capability to work with incomplete knowledge
 Having a memory distribution

Disadvantages of Artificial Neural Network (ANN):

 Assurance of proper network structure
 Unrecognized behaviour of the network
 Hardware dependence

3
  Difficulty of showing the issue to the network

Working of Artificial Neural Networks (ANN):

Artificial Neural Network can be best represented as a weighted directed graph, where the artificial
neurons form the nodes. The association between the neurons outputs and neuron inputs can be viewed as
the directed edges with weights. The Artificial Neural Network receives the input signal from the external
source in the form of a pattern and image in the form of a vector. These inputs are then mathematically
assigned by the notations x(n) for every n number of inputs.

Afterward, each of the input is multiplied by its corresponding weights (these weights are the details
utilized by the artificial neural networks to solve a specific problem). In general terms, these weights
normally represent the strength of the interconnection between neurons inside the artificial neural
network. All the weighted inputs are summarized inside the computing unit.
If the weighted sum is equal to zero, then bias is added to make the output non-zero or something
else to scale up to the system's response. Bias has the same input, and weight equals to 1. Here the total of
weighted inputs can be in the range of 0 to positive infinity. Here, to keep the response in the limits of the
desired value, a certain maximum value is benchmarked, and the total of weighted inputs is passed
through the activation function.
The activation function refers to the set of transfer functions used to achieve the desired output.
There is a different kind of the activation function, but primarily either linear or non-linear sets of

4
functions. Some of the commonly used sets of activation functions are the Binary, linear, and Tan
hyperbolic sigmoidal activation functions.

Keras:
Keras is one of the most powerful and easy to use python library, which is built on top of popular
deep learning libraries like TensorFlow, Theano, or Cognitive Toolkit (CNTK) etc., for creating deep
learning models.

Features:
 Consistent, simple and extensible API
 Minimal structure - easy to achieve the result without any frills
 It supports multiple platforms and backend
 It is user friendly framework which runs on both CPU and GPU
 Highly scalability of computation

Benefits:
 Larger community support
 Easy to test
 Keras neural networks are written in Python which makes things simpler
 Keras supports both convolution and recurrent networks
 Deep learning models are discrete components, so that, we can combine into many ways

IMPLEMENTING MLPS WITH KERAS

Multi-Layer Perceptrons (MLPs) are artificial neural networks that are widely used for
classification and regression tasks. Keras is a high-level neural network API written in Python, which
makes it easy to build and train deep learning models, including MLPs. Here are the steps to build and
train an MLP using Keras:

1. Import the required libraries:

import tensorflow as tf
from tensorflow import keras

2. Load the dataset:

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

5
 3. Preprocess the data:
x_train = x_train.reshape((60000, 28 * 28))
x_train = x_train.astype('float32') / 255
x_test = x_test.reshape((10000, 28 * 28))
x_test = x_test.astype('float32') / 255

y_train = keras.utils.to_categorical(y_train)
y_test = keras.utils.to_categorical(y_test)

4. Define the model architecture:

model = keras.Sequential([
keras.layers.Dense(512, activation='relu', input_shape=(28 * 28,)),
keras.layers.Dropout(0.5),
keras.layers.Dense(10, activation='softmax')
])

5. Compile the model:

model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

6. Train the model:

history = model.fit(x_train, y_train, epochs=10, batch_size=128,
validation_data=(x_test, y_test))

7. Evaluate the model on the test set:

test_loss, test_acc = model.evaluate(x_test, y_test)
print('Test accuracy:', test_acc)

The above creates a simple MLP with two dense layers and a dropout layer. The first dense layer
has 512 units and uses the ReLU activation function. The output layer has 10 units (one for each digit)
and uses the softmax activation function. The model is compiled using the RMSprop optimizer and the
categorical cross-entropy loss function. Finally, the model is trained for 10 epochs using a batch size of
128 and is evaluated on the test set.

6
INSTALLING TENSORFLOW 2
TensorFlow 2:
 TensorFlow 2 is a popular open-source machine learning library developed by Google. It is a
major update to the original TensorFlow library, and it introduces many new features and
improvements.
 TensorFlow 2 is designed to be more user-friendly and accessible than its predecessor. It
simplifies the process of building, training, and deploying machine learning models. It also
includes many high-level APIs that make it easier to use machine learning techniques, such as
deep neural networks.
 One of the key features of TensorFlow 2 is its support for eager execution, which allows us to run
operations immediately as they are called, rather than building up a graph of operations to be
executed later. This makes it easier to debug and experiment with models, and it also enables a
more natural coding style.
 TensorFlow 2 also includes support for Keras, a high-level API for building and training neural
networks. This integration allows us to take advantage of Keras's simplicity and ease of use, while
still benefiting from TensorFlow's powerful low-level capabilities.

The following are the steps for installing TensorFlow 2:

 First, check if the system has the necessary hardware and software requirements to install
TensorFlow 2. We need a 64-bit operating system, Python 3.6-3.9, and a compatible graphics card
if we plan to use GPU(Graphics Processing Unit) acceleration.

 Create a new virtual environment to install TensorFlow 2. This step is optional, but it's a good
practice to keep the TensorFlow installation isolated from other Python packages on thes ystem.
We can create a virtual environment using the following command in terminal or command
prompt:
python3 -m venv myenv
This creates a new virtual environment named myenv.

 Activate the virtual environment using the following command:

source myenv/bin/activate
If we're using Windows, we can activate the virtual environment using the following command
instead:
myenv\Scripts\activate

7
  Install TensorFlow 2 using pip by running the following command:
pip install tensorflow
If we have a compatible GPU and want to use GPU acceleration, we can install the GPU version
of TensorFlow by running the following command instead:
pip install tensorflow-gpu

 Verify that TensorFlow 2 has been installed correctly by importing it in a Python script or in a
Python shell:
import tensorflow as tf
print(tf.__version__)

This should print the version number of TensorFlow 2 that we installed.

Program to check whether TensorFlow 2 has been installed (or) not:

import tensorflow as tf
try:
tf_version = tf.__version__
print("TensorFlow version is:", tf_version)
except ImportError:
print("TensorFlow is not installed.")

LOADING AND PREPROCESSING DATA WITH TENSORFLOW 2

Loading and preprocessing data is an important step in building machine learning models using
TensorFlow 2.

Loading Data:
TensorFlow 2 supports various data sources, including NumPy arrays, pandas dataframes, and
TFRecords files. The most commonly used data sources are NumPy arrays and pandas dataframes.

NumPy arrays: NumPy arrays are a popular data structure in Python for numerical computations. They
can be used to represent tabular data, image data, and other types of data. To load data from a NumPy
array, we can use the tf.data.Dataset.from_tensor_slices() method. Here's an example of how to load data
from a NumPy array:

import tensorflow as tf

8
import numpy as np

# create a NumPy array with random data

data = np.random.random((100, 10))

# create a TensorFlow dataset from the NumPy array

dataset = tf.data.Dataset.from_tensor_slices(data)

Pandas dataframes: Pandas dataframes are another popular data structure in Python for data analysis.
They can be used to represent tabular data with labeled rows and columns. To load data from a pandas
dataframe, we can use the tf.data.Dataset.from_tensor_slices() method as well. Here's an example of how
to load data from a pandas dataframe:
import tensorflow as tf
import pandas as pd

# create a pandas dataframe with random data

data = pd.DataFrame(np.random.random((100, 10)))

# create a TensorFlow dataset from the pandas dataframe

dataset = tf.data.Dataset.from_tensor_slices(data.values)

TFRecords files: TFRecords files are a binary format used to store large amounts of data efficiently. They
are often used to store image data, audio data, and other types of data that can be represented as a
sequence of bytes. To load data from a TFRecords file, we can use the tf.data.TFRecordDataset() method.
Here's an example of how to load data from a TFRecords file:

import tensorflow as tf

# create a TensorFlow dataset from a TFRecords file

dataset = tf.data.TFRecordDataset('path/to/tfrecords/file')

Preprocessing Data:
After loading data, the next step is to preprocess it to prepare it for training. Preprocessing typically
involves transforming the data into a format that can be easily fed into a machine learning model.

9
Mapping: The map() method can be used to apply a function to each element of a dataset. This is often
used to transform the input data into a format that is suitable for a machine learning model. Here's an
example of how to use the map() method to apply a function to each element of a dataset:

import tensorflow as tf

# create a TensorFlow dataset

dataset = tf.data.Dataset.from_tensor_slices([1, 2, 3, 4, 5])

# define a function to square each element

def square_fn(x):
return x ** 2

# apply the function to each element of the dataset

dataset = dataset.map(square_fn)

Batch: The batch() method can be used to group the elements of a dataset into batches. This is often used
to feed multiple inputs to a machine learning model at once. Here's an example of how to use the batch()
method to group the elements of a dataset into batches:

import tensorflow as tf

# Create a dataset
dataset = tf.data.Dataset.range(10)

# Batch the dataset into groups of 3 elements

batched_dataset = dataset.batch(3)

# Print the batches

for batch in batched_dataset:
print(batch)

10