UNIT III DEEP LEARNING ALGORITHMS FOR AI 9

Artificial Neural Networks - Linear Associative Networks - Perceptrons -The

Backpropagation Algorithm - Hopfield Nets - Boltzmann Machines - Deep RBMs -
Variational Autoencoders - Deep Backprop Networks- Autoencoders
=====================================================================================

1. ARTIFICIAL NEURAL NETWORK

 The term "Artificial neural network" refers to a biologically inspired sub-field of

artificial intelligence modeled after the brain.

 An Artificial neural network is usually a computational network based on biological neural

networks that construct the structure of the human brain.

 Artificial neural networks also have neurons that are linked to each other in various layers of
the networks. These neurons are known as nodes.

What is Artificial Neural Network?

 The term "Artificial Neural Network" is derived from Biological neural networks that
develop the structure of a human brain. Similar to the human brain that has neurons
interconnected to one another, artificial neural networks also have neurons that are
interconnected to one another in various layers of the networks. These neurons are known as
nodes.
 Typical diagram of Biological Neural Network.

The typical Artificial Neural Network looks something like the given figure.

Relationship between Biological neural network and artificial neural network:

Biological Neural Network Artificial Neural Network

Dendrites Inputs

Cell nucleus Nodes

Synapse Weights

Axon Output
An Artificial Neural Network in the field of Artificial intelligence where it attempts to mimic the
network of neurons.

The artificial neural network is designed by programming computers to behave simply like
interconnected brain cells.

The architecture of an artificial neural network:

 In order to define a neural network that consists of a large number of artificial neurons,
which are termed units arranged in a sequence of layers.

 Artificial Neural Network primarily consists of three layers:

Input Layer:

 As the name suggests, it accepts inputs in several different formats provided by the
programmer.

Hidden Layer:

 The hidden layer presents in-between input and output layers. It performs all the
calculations to find hidden features and patterns.

Output Layer:

 The input goes through a series of transformations using the hidden layer, which finally
results in output that is conveyed using this layer.

 The artificial neural network takes input and computes the

weighted sum of the inputs and includes a bias. This
 computation is represented in the form of a transfer function.

 It determines weighted total is passed as an input to an activation function to produce the

output.

 Activation functions choose whether a node should fire or not. Only those who are fired
make it to the output layer. There are distinctive activation functions available that can be
applied upon the sort of task we are performing.

Advantages of Artificial Neural Network (ANN)

o Parallel processing capability:Artificial neural networks have a numerical value that can
perform more than one task simultaneously.

o Storing data on the entire network:Data that is used in traditional programming is stored
on the whole network, not on a database. The disappearance of a couple of pieces of data
in one place doesn't prevent the network from working.

o Capability to work with incomplete knowledge:After ANN training, the information

may produce output even with inadequate data. The loss of performance here relies upon
the significance of missing data.

o Having a memory distribution:For ANN is to be able to adapt, it is important to

determine the examples and to encourage the network according to the desired output by
demonstrating these examples to the network. The succession of the network is directly
proportional to the chosen instances, and if the event can't appear to the network in all its
aspects, it can produce false output.

o Having fault tolerance:Extortion of one or more cells of ANN does not prohibit it from
generating output, and this feature makes the network fault-tolerance.
Disadvantages of Artificial Neural Network:

o Assurance of proper network structure:There is no particular guideline for determining

the structure of artificial neural networks. The appropriate network structure is
accomplished through experience, trial, and error.

o Unrecognized behavior of the network:It is the most significant issue of ANN. When
ANN produces a testing solution, it does not provide insight concerning why and how. It
decreases trust in the network.

o Hardware dependence:Artificial neural networks need processors with parallel

processing power, as per their structure. Therefore, the realization of the equipment is
dependent.

o Difficulty of showing the issue to the network:ANNs can work with numerical data.
Problems must be converted into numerical values before being introduced to ANN. The
presentation mechanism to be resolved here will directly impact the performance of the
network. It relies on the user's abilities.

o The duration of the network is unknown:The network is reduced to a specific value of

the error, and this value does not give us optimum results.

How do artificial neural networks work?

 Artificial Neural Network can be best represented as a weighted directed graph, where the
artificial neurons form the nodes.

 The association between the neurons outputs and neuron inputs can be viewed as the
directed edges with weights.

 The Artificial Neural Network receives the input signal from the external source in the
form of a pattern and image in the form of a vector.

 These inputs are then mathematically assigned by the notations x(n) for every n number of
inputs.

 Afterward, each of the input is multiplied by its corresponding weights

 In general terms, these weights normally represent the strength of the interconnection
between neurons inside the artificial neural network.

 All the weighted inputs are summarized inside the computing unit.
 If the weighted sum is equal to zero, then bias is added to make the output non-zero or
something else to scale up to the system's response.

 Bias has the same input, and weight equals to 1. Here the total of weighted inputs can be in
the range of 0 to positive infinity.

 Here, to keep the response in the limits of the desired value, a certain maximum value is
benchmarked, and the total of weighted inputs is passed through the activation function.

 The activation function refers to the set of transfer functions used to achieve the desired
output.

 There is a different kind of the activation function, but primarily either linear or non-linear
sets of functions.

 Some of the commonly used sets of activation functions are the Binary, linear, and Tan
hyperbolic sigmoidal activation functions. Let us take a look at each of them in details: