Introduction To Using TensorFlow & Deep Learning

Presentation by:
In the name of God
Professor :
1
Introduction To Using TensorFlow

2Introduction To Using TensorFlow
Overview
■ TensorFlow
– What is TensorFlow
– TensorFlow Code Basics
– TensorFlow UseCase
■ Deep Learning
– CNN & RNN
– Exmple of Mnist Data Set Classification

What is TensorFlow
What are Tensors?
As shown in the image above, tensors are just multidimensional arrays, that allows you to represent data
having higher dimensions.
9 …
0

What is TensorFlow
■ VGG Network
■ Plain Network
■ Residual Network
■ Experiments
■ Conclusion Network
What are Tensors & Flow?
In fact, the name “TensorFlow” has been derived from the operations which neural networks
perform on tensors.
TensorFlow is a library based on Python that provides different types of functionality for implementing
Deep Learning Models. the term TensorFlow is made up of two terms – Tensor & Flow:

What is TensorFlow
TensorFlow (data flow) graph

TensorFlow Code Basics
■ Basically, the overall process of writing a TensorFlow program
involves two steps:
■ Building a Computational Graph
■ Running a Computational Graph
Let me explain you the above two steps one by one:

■ Building & Running The Computational Graph
■ Example: Tensor & Flow OR Data & Flow
import tensorflow as tf
# Build a graph
a = tf.constant(8.0)
b = tf.constant(9.0)
c = a * b
# Create the session object
sess = tf.Session()
output_c = sess.run(c)
print(output_c)
sess.close()

What is TensorFlow
 Main Components of Tensorflow:
A. Variables: Retain values between sessions, use for
weights/bias
B. Nodes: The operations
C. Tensors: Signals that pass from/to nodes
D. Placeholders: Used to send data between your
program and the tensorflow graph
E. Session: Place when graph is executed.
Points to Remember about placeholders:
•Placeholders are not initialized and contains no data.
•One must provides inputs or feeds to the placeholder which are considered during runtime.
•Executing a placeholder without input generates an error.

■ Building & Running The Computational Graph
■ Constants, Placeholder and Variables
import tensorflow as tf
# Creating placeholders
a = tf. placeholder(tf.float32)
b = tf. placeholder(tf.float32)
# Assigning multiplication operation w.r.t. a &amp; b to node mul
mul = a*b
# Create session object
sess = tf.Session()
# Executing mul by passing the values [1, 3] [2, 4] for a and b respectively
output = sess.run(mul, {a: [1,3], b: [2, 4]})
print('Multiplying a b:', output)
Output: [2. 12.]

■ Example : Linear Regression on tensorflow

Deep Learning
CNN & RNN

Artificial Intelligence
Deep Learning & Machine Learning

Deep Learning
■ Deep Learning vs Machine Learning

Deep Learning
■ Deep Learning with Neural Network

Deep Learning

■ Example of Neural Network:

Deep Learning
■ Neural Network (NN)
Forward Pass Backward Pass

Deep Learning
■ Convolutional Neural Network (CNN)
The three main processing stages in a CNN

Deep Learning

Deep Learning
Example filters learned by Krizhevsky et al. Each of the 96 filters shown here is of size [11x11x3], and each
one is shared by the 55*55 neurons in one depth slice.
1 2 3

Deep Learning

Deep Learning
■ Convolutional Neural Network (CNN) Denoising

Deep Learning

Deep Learning
http://scs.ryerson.ca/~aharley/vis/conv/The three main processing stages in a CNN

Deep Learning
■ Example :

■ Example :
Multi Layer Perceptron MNIST on tensorflow
The MNIST database (Modified National Institute of Standards and Technology database) is a large database of
handwritten digits that is commonly used for training various image processing systems.

■ Example : Multi Layer Perceptron MNIST on tensorflow
1. Load tensorflow library and MNIST data
2. Neural network parameters
3. Build graph
4. Initialize weights and construct the model
5. Define Loss function, and Optimizer
6. Launch graph

# Parameters
learning_rate = 0.001 training_epochs = 15 batch_size = 100
# Network Parameters
n_hidden_1 = 256 n_hidden_2 = 256 n_input = 784 n_classes = 10
# On this case we choose the AdamOptimizer
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
# Cross entropy loss function
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))
■ Example :

1. Load tensorflow library and MNIST data
import tensorflow as tf # Import MNIST data
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)
print('Test shape:',mnist.test.images.shape)
print('Train shape:',mnist.train.images.shape)
Test shape: (10000, 784)
Train shape: (55000, 784)

# Parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100
display_step = 1
# Network Parameters
n_hidden_1 = 256 # 1st layer number of features
n_hidden_2 = 256 # 2nd layer number of features
n_input = 784 # MNIST data input (img shape: 28*28)
n_classes = 10 # MNIST total classes (0-9 digits)
2. Neural network parameters

x = tf.placeholder("float", [None, n_input])
y = tf.placeholder("float", [None, n_classes])
# Create model
def multilayer_perceptron(x, weights, biases):
print('x:',x.get_shape(),'W1:',weights['h1'].get_shape(),'b1:',biases['b1'].get_shape())
layer_1 = tf.add(tf.matmul(x, weights['h1']), biases['b1'])
layer_1 = tf.nn.relu(layer_1)
print( 'layer_1:', layer_1.get_shape(), 'W2:', weights['h2'].get_shape(), 'b2:',
biases['b2'].get_shape())
layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
layer_2 = tf.nn.relu(layer_2)
print( 'layer_2:', layer_2.get_shape(), 'W3:', weights['out'].get_shape(), 'b3:',
biases['out'].get_shape())
out_layer = tf.matmul(layer_2, weights['out']) + biases['out']
print('out_layer:',out_layer.get_shape()) return out_layer
3. Build graph

# Store layers weight & bias
weights = { 'h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])), #784x256
'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])), #256x256
'out': tf.Variable(tf.random_normal([n_hidden_2, n_classes])) #256x10
}
biases = { 'b1': tf.Variable(tf.random_normal([n_hidden_1])), #256x1
'b2': tf.Variable(tf.random_normal([n_hidden_2])), #256x1
'out': tf.Variable(tf.random_normal([n_classes])) #10x1
}
# Construct model
pred = multilayer_perceptron(x, weights, biases)
4. Initialize weights and construct the model

5. Define Loss function, and Optimizer
# Cross entropy loss function
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))
# On this case we choose the AdamOptimizer
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

6.1 Launch graph
# Initializing the variables
init = tf.initialize_all_variables()
# Launch the graph
with tf.Session() as sess:
sess.run(init)
# Training cycle
for epoch in range(training_epochs):
avg_cost = 0.
total_batch = int(mnist.train.num_examples/batch_size)
# Loop over all batches
for i in range(total_batch):
batch_x, batch_y = mnist.train.next_batch(batch_size)
# Run optimization op (backprop) and cost op (to get loss value)
_, c = sess.run([optimizer, cost], feed_dict={x: batch_x, y: batch_y})
# Compute average loss
avg_cost += c / total_batch

6.2 Launch graph
# Display logs per epoch step
if epoch % display_step == 0:
print ("Epoch:", '%04d' % (epoch+1), "cost=",
"{:.9f}".format(avg_cost))
print("Optimization Finished!")
# Test model
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
# Calculate accuracy
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
# To keep sizes compatible with model
print ("Accuracy:", accuracy.eval({x: mnist.test.images, y: mnist.test.labels}))

6.2 Output of Execute graph with CNN
Epoch: 0001 cost= 152.289635962
Epoch: 0002 cost= 39.134648348
...
Epoch: 0015 cost= 0.850344581
Optimization Finished!
Accuracy: 0.9464

Deep Learning Layer
W
Increasing Depth Layer ?Increasing Parameter ?
Become Overfit

GoogleNet

ResNet

Results :

CapsNet

Parallel Processing

Reference
https://www.edureka.co/blog/tensorflow-tutorial/
http://howsam.org/1396/08/11/%D8%A2%D9%85%D9%88%D8%B2%D8%B4-
%D8%AA%D9%86%D8%B3%D9%88%D8%B1%D9%81%D9%84%D9%88/
http://www.7khatcode.com/7677/%D8%AA%D9%86%D8%B3%D9%88%D8%B1%D9%81%D9
%84%D9%88-tensorflow-%DA%86%DB%8C%D8%B3%D8%AA%D8%9F
https://blog.faradars.org/cnn-convolution-perceptron-neural-network-2/
https://leonardoaraujosantos.gitbooks.io/artificial-inteligence/content/loss-function.html
https://medium.com/machine-learning-in-practice/over-150-of-the-best-machine-learning-nlp-and-
python-tutorials-ive-found-ffce2939bd78
https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/
http://howsam.org/1396/08/12/%D9%86%D8%B5%D8%A8-
%D8%AA%D9%86%D8%B3%D9%88%D8%B1%D9%81%D9%84%D9%88/

Reference
http://howsam.org/1396/08/16/%D8%B4%D8%B1%D9%88%D8%B9-%DA%A9%D8%A7%D8%B1-
%D8%A8%D8%A7-%D8%AA%D9%86%D8%B3%D9%88%D8%B1%D9%81%D9%84%D9%88/
https://stanford.edu/~shervine/l/fa/teaching/cs-229/cheatsheet-supervised-learning
https://stanford.edu/~shervine/l/fa/teaching/cs-229/cheatsheet-deep-learning
https://stanford.edu/~shervine/l/fa/teaching/cs-229/cheatsheet-machine-learning-tips-and-tricks
https://chistio.ir/%D9%BE%D8%B3-%D8%A7%D9%86%D8%AA%D8%B4%D8%A7%D8%B1-
%D8%AE%D8%B7%D8%A7-back-propagation-%D8%B4%D8%A8%DA%A9%D9%87-
%D8%B9%D8%B5%D8%A8%DB%8C/
http://deeplearning.ir/%D9%BE%DB%8C%D8%B4%DB%8C%D9%86%D9%87-%D9%88-
%D9%85%D8%B1%D9%88%D8%B1%DB%8C-%D8%A8%D8%B1-
%D8%B1%D9%88%D8%B4%D9%87%D8%A7%DB%8C-
%D9%85%D8%AE%D8%AA%D9%84%D9%81-
%DB%8C%D8%A7%D8%AF%DA%AF%DB%8C%D8%B1%DB%8C/
https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/

Reference
https://www.youtube.com/watch?v=FmpDIaiMIeA
https://www.youtube.com/watch?v=2-Ol7ZB0MmU
https://brohrer.github.io/how_convolutional_neural_networks_work.html
http://www.rtc.us.es/nullhop-a-flexible-convolutional-neural-network-accelerator-
based-on-sparse-representations-of-feature-maps/

Introduction To Using TensorFlow & Deep Learning

More Related Content

What's hot (19)

Similar to Introduction To Using TensorFlow & Deep Learning (20)

Recently uploaded (20)

Introduction To Using TensorFlow & Deep Learning