Unit V

Neural Networks and Deep Learning

Introduction to Artificial Neural Networks with Keras, Implementing MLPs with Keras, Installing
TensorFlow 2,Loading and Preprocessing Data with TensorFlo

Introduction to Artificial Neural Networks

• The development of ANN was the result of an attempt to replicate the workings of the human
brain.
• The workings of ANN are extremely similar to those of biological neural networks, although they
are not identical.
• ANN algorithm accepts only numeric and structured data.
• The term "Artificial Neural Network" is derived from Biological neural networks that develop
the structure of a human brain.
• Similar to the human brain that has neurons interconnected to one another, artificial neural
networks also have neurons that are interconnected to one another in various layers of the
networks. These neurons are known as nodes.
The typical Artificial Neural Network looks something like the given figure.

Biological Neural Network Artificial Neural Network

Dendrites Inputs

Cell nucleus Nodes

Synapse Weights

Axon Output

The architecture of an artificial neural network:

To understand the concept of the architecture of an artificial neural network, we have to
understand what a neural network consists of. In order to define a neural network that consists of
a large number of artificial neurons, which are termed units arranged in a sequence of layers.
Lets us look at various types of layers available in an artificial neural network.
Artificial Neural Network primarily consists of three layers:

Input Layer:

As the name suggests, it accepts inputs in several different formats provided by the programmer.

Hidden Layer:

The hidden layer presents in-between input and output layers. It performs all the calculations to
find hidden features and patterns.

Output Layer:

The input goes through a series of transformations using the hidden layer, which finally results in
output that is conveyed using this layer.

The artificial neural network takes input and computes the weighted sum of the inputs and
includes a bias. This computation is represented in the form of a transfer function.

It determines weighted total is passed as an input to an activation function to produce the output.
Activation functions choose whether a node should fire or not.
Artificial Neuron

Artificial Neuron are also called as perceptrons. Perceptron is a building block of an Artificial Neural
Network. Initially, in the mid of 19th century, Mr. Frank Rosenblatt invented the Perceptron for
performing certain calculations to detect input data capabilities or business intelligence. This consist of
the following basic terms:

• Input

• Weight

• Bias

• Activation Function

• Output
 • Input Nodes or Input Layer:

This is the primary component of Perceptron which accepts the initial data into the system for
further processing. Each input node contains a real numerical value.

• Wight and Bias:

Weight parameter represents the strength of the connection between units. This is another most
important parameter of Perceptron components. Weight is directly proportional to the strength of
the associated input neuron in deciding the output. Further, Bias can be considered as the line of
intercept in a linear equation.

• Activation Function:

These are the final and important components that help to determine whether the neuron will fire
or not. Activation Function can be considered primarily as a step function
Types of Activation functions:

• Sign function
• Step function, and
• Sigmoid function

How does Perceptron work?

• The perceptron model begins with the multiplication of all input values and their weights,
then adds these values together to create the weighted sum.
• Then this weighted sum is applied to the activation function 'f' to obtain the desired
output.
• This activation function is also known as the step function and is represented by 'f'.

Perceptron model works in two important steps as follows:

Step-1

In the first step first, multiply all input values with corresponding weight values and then add
them to determine the weighted sum. Mathematically, we can calculate the weighted sum as
follows:

∑wi*xi = x1*w1 + x2*w2 +…wn*xn

Add a special term called bias 'b' to this weighted sum to improve the model's performance.

∑wi*xi + b

Step-2

In the second step, an activation function is applied with the above-mentioned weighted sum,
which gives us output either in binary form or a continuous value as follows:

Y = f(∑wi*xi + b)

Types of Perceptron

• Single-Layer Perceptron: This type of perceptron is limited to learning linearly separable

patterns. effective for tasks where the data can be divided into distinct categories through a
straight line.

• Multilayer Perceptron: Multilayer perceptrons possess enhanced processing capabilities as they

consist of two or more layers, adept at handling more complex patterns and relationships within
the data.

Single Layer Perceptron Model:

• This is one of the easiest Artificial neural networks (ANN) types. The main objective of
the single-layer perceptron model is to analyze the linearly separable objects with binary
outcomes.
• In a single layer perceptron model, its algorithms do not contain recorded data, so it
begins with inconstantly allocated input for weight parameters. Further, it sums up all
inputs (weight). After adding all inputs, if the total sum of all inputs is more than a pre-
determined value, the model gets activated and shows the output value as +1.

• If the outcome is same as pre-determined or threshold value, then the performance of this
model is stated as satisfied, and weight demand does not change. "Single-layer
perceptron can learn only linearly separable patterns."

Multi-Layered Perceptron Model:

Like a single-layer perceptron model, a multi-layer perceptron model also has the same model
structure but has a greater number of hidden layers. The multi-layer perceptron model is also
known as the Backpropagation algorithm, which executes in two stages as follows:

• Forward Stage: Activation functions start from the input layer in the forward stage and
terminate on the output layer.
 • Backward Stage: In the backward stage, weight and bias values are modified as per the
model's requirement. In this stage, the error between actual output and demanded
originated backward on the output layer and ended on the input layer.

A multi-layer perceptron model has greater processing power and can process linear and non-
linear patterns. Further, it can also implement logic gates such as AND, OR, XOR, NAND,
NOT, XNOR, NOR.

Advantages of Multi-Layer Perceptron:

• A multi-layered perceptron model can be used to solve complex non-linear problems.

• It works well with both small and large input data.
• It helps us to obtain quick predictions after the training.
• It helps to obtain the same accuracy ratio with large as well as small data.

Disadvantages of Multi-Layer Perceptron:

• In Multi-layer perceptron, computations are difficult and time-consuming.

• In multi-layer Perceptron, it is difficult to predict how much the dependent variable
affects each independent variable.
• The model functioning depends on the quality of the training.

IMPLEMENTING MLP WITH KERAS

Multilayer Perceptrons (MLPs) can be implemented in Keras in several ways. Here are three
common approaches:

1. Sequential API: The Sequential API is the simplest way to build a model in Keras,
especially for MLPs where layers are stacked sequentially. As the name suggest, this API is
to build the model in sequential style, i.e. output of one layer will be fed to the later layer.

1.Import Libraries: Begin by importing the necessary libraries. For an MLP, you'll need
TensorFlow (or another backend like Theano or CNTK) and Keras.

import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

2.Define the Model: Initialize a Sequential model. This is the container where you can add
layers.
 model = Sequential()

3. Add Layers: Next, add layers to your model. For an MLP, you typically have at least one
input layer, one or more hidden layers, and an output layer.

# Adding the input layer

model.add(Dense(units=64, activation='relu', input_dim=10))

# Adding hidden layers

model.add(Dense(units=64, activation='relu'))
model.add(Dense(units=64, activation='relu'))

# Adding the output layer

model.add(Dense(units=1, activation='sigmoid'))

In this example, Dense layers are used which are fully connected layers. The units parameter
denotes the number of neurons in each layer. The activation parameter specifies the activation
function for the neurons. ReLU (Rectified Linear Unit) and sigmoid are commonly used
activation functions, but you can choose others based on your problem.

4. Compile the Model: After adding layers, compile the model. This involves specifying the loss
function, optimizer, and metrics to monitor during training.

model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])

Here, we use binary cross entropy as the loss function since it's a binary classification problem.
For multi-class classification, you might use categorical crossentropy. The optimizer 'adam' is a
popular choice, but other optimizers like SGD (Stochastic Gradient Descent) or RMSprop can
also be used.

• Train the Model: Train the model on your training data.

model.fit(X_train, y_train, epochs=10, batch_size=32)

Here, X_train is your input data and y_train is your target labels. epochs specify the number
of iterations over the entire dataset during training, and batch_size denotes the number of
samples per gradient update.

• Evaluate the Model: After training, evaluate the model's performance on unseen data

loss, accuracy = model.evaluate(X_test, y_test)

X_test and y_test represent your test data and corresponding labels. The evaluate method
returns the loss and accuracy of the model on the test data.

2.The Functional API in Keras provides a more flexible way to define neural networks
compared to the Sequential API. It allows you to create models with complex architectures,
including models with multiple inputs and outputs or models with shared layers. Let's break
down the steps involved in creating a Multilayer Perceptron (MLP) using the Functional API:

1. Import Libraries: Begin by importing the necessary libraries. You'll need TensorFlow
(or another backend like Theano or CNTK) and Keras.

python
• import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Dense

• Define Input: Define the input layer(s) using the Input function.

python
• inputs = Input(shape=(input_shape,))

Here, input_shape represents the shape of your input data. For example, if each sample has 10
features, input_shape would be (10,).

• Add Layers: Define the layers of your model and connect them to the input(s).

python
• x = Dense(units=64, activation='relu')(inputs)
x = Dense(units=64, activation='relu')(x)

Similar to the Sequential API, you use the Dense function to define fully connected layers. Each
layer takes the output of the previous layer as input.

• Define Output: Define the output layer and connect it to the last hidden layer.

python
• outputs = Dense(units=num_classes, activation='softmax')(x)

Here, num_classes represents the number of classes in your classification problem. If it's a
binary classification problem, num_classes would be 1 with a sigmoid activation function.

• Create Model: Create a Keras Model by specifying the input(s) and output(s).

python
• model = Model(inputs=inputs, outputs=outputs)

• Compile Model: Compile the model by specifying the loss function, optimizer, and metrics.
python
• model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])

Adjust the loss function, optimizer, and metrics based on your specific problem.

• Train Model: Train the model on your training data.

python
• model.fit(X_train, y_train, epochs=10, batch_size=32)

• Evaluate Model: Evaluate the model's performance on test data.

python
loss, accuracy = model.evaluate(X_test, y_test)

X_test and y_test represent your test data and corresponding labels.

Subclassing API

• Keras also provides a Subclassing API for building neural networks. The Subclassing
API allows for even greater
• flexibility and customization than the Sequential and Functional APIs. In this tutorial, we
will learn how to implement MLPs with the Keras Subclassing API.

Let's start by importing the necessary modules:

from keras.layers import Layer, Dense

from keras.models import Model

Next, we define a custom class for our MLP by subclassing the Layer class:

class MLP(Layer):
def __init__(self):
super(MLP, self).__init__()
self.dense1 = Dense(64, activation='relu')
self.dense2 = Dense(10, activation='softmax')
def call(self, inputs):
x = self.dense1(inputs)
x = self.dense2(x)
return x

In the above code, we define a custom class called MLP, which subclasses the Layer class. In the
constructor (init),
we create two Dense layers, one with 64 neurons and a ReLU activation function, and another
with 10 neuronsand a softmax activation function.

We store these layers as member variables of the class using the self keyword.

In the call method, we define the forward pass of our MLP. We first pass the input tensor
(inputs) through the

first Dense layer (dense1), and then pass the output through the second Dense layer (dense2). We
return the output tensor (x).

We can create an instance of our MLP class and use it to build a Keras model:

inputs = Input(shape=(100,))
mlp = MLP()
outputs = mlp(inputs)
model = Model(inputs=inputs, outputs=outputs)
In the above code, we first define the input shape of our MLP by creating an Input object with
the shape (100,).

Then, we create an instance of our MLP class called mlp, and pass the input object to it. This
returns the output tensor of our MLP, which we assign to the variable outputs.

Finally, we create a Model object by specifying the input and output layers. We pass the input
object to the inputs parameter, and the output tensor to the outputs parameter.

We can compile and train the model in the same way as we did with the other APIs:

model.compile(loss='categorical_crossentropy',
optimizer='sgd',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5, batch_size=32)
In this example, we use categorical cross-entropy as the loss function, stochastic gradient descent
(SGD) as the optimizer, and accuracy as the evaluation metric.

Installing TensorFlow 2

• TensorFlow is an open-source machine learning framework developed by Google. It provides a

comprehensive ecosystem of tools, libraries, and community resources for building and
deploying machine learning models.
• TensorFlow is designed to be flexible, scalable, and efficient, making it suitable for a wide range
of applications, from simple regression tasks to complex deep learning projects.
here's a step-by-step guide to install TensorFlow 2:
 1. First, you need to install Python on your computer. You can download the latest version
of Python from the official website: https://www.python.org/downloads/

2. Once you have installed Python, you can use pip (the package installer for Python) to install
TensorFlow.Open a terminal or command prompt and type the following command:
pip install tensorflow
This will install the latest version of TensorFlow available on the PyPI package index.

3. Alternatively, you can install a specific version of TensorFlow by specifying the version
number in the pip
command. For example, to install TensorFlow version 2.7.0, you would use the following
command:
pip install tensorflow==2.7.0
4.If you want to take advantage of your GPU for faster computation, you can install the GPU
version of TensorFlow.
First, make sure you have the necessary CUDA and cuDNN libraries installed on your system.
Then, use the
following command to install the GPU version of TensorFlow
pip install tensorflow-gpu
5.Once TensorFlow is installed, you can test it by opening a Python shell and running the
following command:
import tensorflow as tf
print(tf.__version__)
This should output the version number of TensorFlow that you have installed.
That's it! You should now have TensorFlow 2 installed on your system.
Loading and Preprocessing Data with TensorFlow
Loading and preprocessing data is a crucial step in machine learning and deep learning pipelines.
TensorFlow provides several tools and APIs for loading and preprocessing data efficiently. Here
are some common methods:
1. Loading Data: TensorFlow provides a tf.data API that makes it easy to load and process data.
This API provides several classes for reading data from different sources, such as files, numpy
arrays, or tensors. For example, to read data from a CSV file, you can use
tf.data.experimental.CsvDataset.
2. Data Augmentation: Data augmentation is a technique used to increase the diversity of the
training set by applying various transformations to the data. TensorFlow provides several APIs
for data augmentation, such as tf.image, which provides methods for image processing, such as
flipping, rotating, and resizing.
3. Normalization: Normalizing the data is an essential preprocessing step that helps to ensure
that the input features are in the same scale. You can use the tf.keras.layers.Normalization API
to normalize the data.
4. Batching: Batching is a technique used to split the data into small batches and process them
in parallel.TensorFlow provides several APIs for batching, such as tf.data.Dataset.batch.
5. Shuffling: Shuffling the data helps to prevent the model from overfitting to the order of the
examples in the training set. You can use the tf.data.Dataset.shuffle API to shuffle the data.
6. Caching: Caching the data in memory can help to speed up the training process. You can use
the tf.data.Dataset.cache API to cache the data.

Here is an example of how to load and preprocess data using TensorFlow:

import tensorflow as tf
# Load the data
dataset = tf.data.experimental.CsvDataset('data.csv', [tf.float32, tf.float32, tf.int32],
header=True)
# Data augmentation
dataset = dataset.map(lambda x, y, z: (tf.image.random_flip_left_right(x), y, z))
# Normalization
normalization_layer = tf.keras.layers.Normalization()
normalization_layer.adapt(dataset.map(lambda x, y, z: x))
dataset = dataset.map(lambda x, y, z: (normalization_layer(x), y, z))
# Batching
dataset = dataset.batch(32)
# Shuffling
dataset = dataset.shuffle(buffer_size=1000)
# Caching
dataset = dataset.cache()
# Train the model
model.fit(dataset, epochs=10