Subset of Machine learning, Algorithms

inspired by the structure and function of

Human Brain

Definition

Structure

Structure
Neuron
Neuron takes input and do some
function to give the output

Function going to be the

Mathematical function

Basic Neural Network Structure

Structure

Layer is Made up of Nodes of

Neuron & Weights

Input Layer

Structure Hidden Layer

Output Layer

Layer

Fully connected layers connect

1 every neuron in one layer to
every neuron in the next layer

Fully Connected Layer

Extracting features in images,

Types which uses a filter to scan an
2 image, a few pixels at a time, and
outputs a feature map that
classifies each feature.

Convolution Layer

Pooling layers are used to reduce

the dimensions of the feature
maps. Thus, it reduces the
3
number of parameters to learn
and the amount of computation
Pooling Layer
performed in the network.

Numeric Values from Neural

Networks, machine learns by
itself. They self-adjust depending
Weights
on the difference between
predicted outputs vs training
inputs.

It is a mathematical formula
which helps the neuron to switch
ON/OFF

Elements

Graph

If value of X is greater than or

Step Function equal to 0, then output is 1, If
value of X is less than 0, then
output is 0

Since step Function is non

differentiable to zero, it can’t do
the gradient descent method, so
Activation Function it can’t update weights.

Graph

If value of X is infinity, then

output is 1, If value of X is
negative infinity, then output is 0
Sigmoid Function
It captures non-linearity in the
data
Types
It can use Gradient descent &
Back propagation method to
calculate weights.

Output range [0,1]

Formula

Graph
Softmax Function
It is used to map the non-normalized output
of a network to a probability distribution
over predicted output class.
Deep Learning
That is it converts output of last layer into a
essential probability distribution.

Tanh Function

ReLU Function

variables which determines the

network structure, how the
network is trained(Eg: Learning
Rate, No. of Units).

Dropout
Hyper Parameter
Activation function

Learning Rate
Types
Number of epochs

Batch size

Number of neurons

It is the simplest form of a neural

1 Perceptron
network

Structure

MLP's decision function is a step

2 Multilayer Perceptron
function & Output is Binary

Structure

Simplest form of neural networks

where input data travels in one
3 Feed Forward Neural Networks
direction only, passing through
artificial neural nodes

Structure

3 It contains a three-dimensional
arrangement of neuron
4 Convolutional Neural Network
Types Each neuron processes the
Structure information from a small part of
the visual field

Example

In Single Perceptron / Multi-layer AND, OR functions are linearly-

Perceptron(MLP), we only have separable & XOR function is not
linear separability linearly separable.

Radial Basis Function Neural It transforms the input signal into

5 another form, which can be then
Networks
feed into the network to get
linear separability
Structure It has exactly one hidden layer

It will remember the things from

6 Recurrent Neural Networks
Previous input to generate output

Structure

Numerical computation using data flow

graphs | Backend for Keras | Distributed
1 computing on multiple GPU

Tensor Flow

Framework for Deep learning | Same code

for CPU & GPU | Uses Theano/TF as
2 Backend | CNN | not support multi GPU
Keras

Old ML & DL library | Supports CUDA -

Compute Unified Device Architecture for
3 parallel computation | Supervised image
problem with CNN
4 Torch

Caffe is a deep learning framework made

Libraries with expression, speed, and modularity in
4 mind.Very high performance | Tuning Hyper
parameters | CNN & RNN(Facebook
extends) | | CPU & GPU
Caffe

Apache MXNet is an open-source deep

learning software framework, used to train,
5 and deploy deep neural networks|More
number of Language Bindings | Distributed
Mxnet
computing

Basic operation such as Sorting, Reshaping,

indexing | Scientific computing | Most new
6 feature belong in SciPy rather than Numpy

Numpy & Scipy

1 Fruit & vegetable classification

Segregator

2 AI in Cooking

Molecular Structure Analysis

5 3 AI in Autonomous Vehicle

Applications How Autonomous Car See

4 AI Doctor

Hospital Receptionist with Basic Doctor

Functions

Prediction of Disease from

5
Medical Image

Medical Image Analysis