Scribd
Upload
34 views28 pages

f8194544 Microsoft PowerPoint DeepLearning

Uploaded by

karunakar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
34 views28 pages

f8194544 Microsoft PowerPoint DeepLearning

Uploaded by

karunakar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

Implementation of Deep Learning Models

Colaboratory : Tesla GPU based Free Cloud

Cost : INR 3.5 Lacs

https://colab.research.google.com
Cloud Configurations

# GPU count and name (SMI: System Management Interface)


!nvidia-smi -L

!nvidia-smi

!lscpu |grep 'Model name'

# no.of sockets i.e available slots for physical processors


!lscpu | grep 'Socket(s):'

# no.of cores each processor is having


!lscpu | grep 'Core(s) per socket:'
Configuration: GPU Based Remote System
GPU: 1 x Tesla K80 , compute 3.7, having 2496 CUDA cores , 12GB
GDDR5 VRAM

CPU: 1 x single core hyper threaded i.e(1 core, 2 threads) Xeon


Processors @2.3Ghz (No Turbo Boost) , 45MB Cache

RAM: ~12.6 GB Available

Disk: ~320 GB Available (OverlayFS)

Idle Time: 90 minutes

Every 12 Hours: Disk, RAM, VRAM, CPU Cache etc. data on


allotted Virtual Machine get erased
https://colab.research.google.com/drive/151805XTDg--dgHb3-AXJCpnWaqRhop_2#scrollTo=vEWe-FHNDY3E
Extraction of Data Files
!apt-get install p7zip-full
!p7zip -d file_name.tar.7z
!tar -xvf file_name.tar
from google.colab import files
Sync Google Drive
Deep Learning and Transfer Functions in Keras
1. Activation Function or Transfer Function is used to
determine the output of node

2. Determine the output of neural network like Yes or No

3. It maps the resulting values in between 0 to 1 or -1 to


1 etc. (depending upon the function).
Categories of Activation / Transfer Functions
• Linear Activation Function
• Non-linear Activation Functions

If Activation function not applied, then the output signal will be a


simple linear function as a polynomial of one degree. Deep
Networks are complex
Equation : f(x) = x
Range : (-infinity to infinity)

Not fit for complexity or various parameters of usual data (Real Time)
that is fed to the neural networks.

Images have encoding in Spatial Domain rather than


Frequency Domain
• Most used Activation
Functions
• Makes easy to adapt /
generalize variety of data
and differentiate between
output

Key Terminologies to understand for nonlinear functions


• Derivative or Differential: Change in y-axis w.r.t.
change in x-axis (Slope)
• Monotonic function: A function which is either
entirely non-increasing or non-decreasing.

The Nonlinear Activation Functions are mainly divided on


the basis of their Range or Curves
Activation Function: Sigmoid / Logistic
Activation Function: Tanh - Hyperbolic Tangent
• Mathematical formula is f(x) = 1 - exp(-2x) / 1 + exp(-2x).

• It’s output is zero centered because its range in between -1 to 1


i.e -1 < output < 1 .

• Optimization is easier in this method hence in practice it is


always preferred over Sigmoid function

The sigmoid and hyperbolic


tangent activation functions
cannot be used in networks
with many layers due to the
Vanishing Gradient Problem
Need of Rectified Linear Unit (ReLU)
• Overcomes vanishing gradient problem, allow models learn faster and
perform better

• ReLU is default activation when developing MLP and CNN. The model
takes less time to train or run

• As ReLU is 0 for all negative inputs for any given unit not to activate at all
(For Missing Data or Data Sparsity)

• The downside for being zero for all negative values is a problem called
dying ReLU, Neurons die for all inputs and remain inactive no matter what
input is supplied, here no gradient flows

• The leak helps to increase the range of the ReLU function. Usually, the
value of a is 0.01 or so. When a is not 0.01 then it is called Randomized
ReLU. Therefore the range of the Leaky ReLU is (-infinity to infinity)
Avoidance of Vanishing Gradient in ReLU

In Back-propagation, while calculating gradients of loss


(Error) with respect to the weights, the gradients tends to
get smaller and smaller as we keep on moving backward in
the Network. This means that the neurons in
the Earlier layers learn very slowly as compared to the
neurons in the later layers in the Hierarchy. The Earlier
layers in the network are slowest to train.
Using TensorFlow APIs in Keras
# Uploading Dynamic Files Dense implements the operation:

from google.colab import files • A dense layer is just a regular layer of


uploaded = files.upload() neurons in a neural network.
• Each neuron receives input from all
the neurons in the previous layer,
thus densely connected.
# Create MLP in Keras
from keras.models import Sequential output = activation(dot(input, kernel) +
bias)
from keras.layers import Dense
import numpy Activation: Element-wise activation
function passed as the activation
argument
# fix random seed for reproducibility Kernel: Weights Matrix created by layer
numpy.random.seed(7)
bias: Bias vector created by the layer
(only applicable if use_bias is True).
# load dataset of scores
dataset = numpy.loadtxt("scores.csv", delimiter=",")

# split into input (X) and output (Y) variables


X = dataset[:,0:8]
Y = dataset[:,8]

# create model
model = Sequential()
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(4, activation='relu'))
model.add(Dense(2, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
https://keras.io/activations/
# Fit the model
model.fit(X, Y, epochs=100, batch_size=10)

# Evaluate the model


scores = model.evaluate(X, Y)
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

# Test Data
testdata = files.upload()
testdataset = numpy.loadtxt("testdata.csv", delimiter=",")
X2 = testdataset[:,0:8]
predictions = model.predict(X2)

# Round predictions
rounded = [round(x[0]) for x in predictions]
print(rounded)
Loss Functions
A loss function (or objective function, or optimization
score function) is one of the two parameters required
to compile a model:

from keras import losses


model.compile(loss='mean_squared_error',
optimizer='sgd')
• for binary_crossentropy: sigmoid activation, scalar
target

• for categorical_crossentropy: softmax activation, one-


hot encoded target

• If it is a multiclass problem, use categorical_crossentropy


Interpretation of Output

• Loss: A scalar value that we attempt to minimize during our training of


the model. The lower the loss, the closer our predictions are to the true
labels.
• Both loss and val_loss should be decreasing and Accuracy (acc and
val_acc) should be increasing.
• acc is the accuracy of training set. val_acc is the measure of how good
the predictions of your model are.
• Training loss is the average of the losses over each batch of training data
A function that transforms the values or
states the conditions for the decision of
the output neuron is known as
an activation function
• Sigmoid (In-Between Solution like
MAY-BE, Intermediate Prediction)
• Tanh
• Softmax
• and many others
Classification and Regression in Prediction

Classification -> Task of predicting a discrete class label


(If forecasting Target Class)

Regression -> Task of predicting a continuous quantity


(If forecasting a Value)
Metrics
A metric is a function that is used to judge the performance of your
model. Metric functions are to be supplied in the metrics parameter
when a model is compiled.

model.compile(loss='mean_squared_error',
optimizer='sgd',
metrics=['mae', 'acc'])

from keras import metrics

model.compile(loss='mean_squared_error',
optimizer='sgd',
metrics=[metrics.mae, metrics.categorical_accuracy])

A metric function is similar to a loss function, except that the results


from evaluating a metric are not used when training the model.
GAURAV KUMAR

Magma Research and Consultancy Services


Ambala Cantt., Haryana, India

Mobile Numbers : +91-9416366178, +91-9034001978

E-mail : [email protected]

http://www.gauravkumarindia.com

You might also like