Scribd
Upload
34 views20 pages

Tensorflow Placeholders and Optimizers

The document discusses placeholders and optimizers in TensorFlow. Placeholders are structures that hold input values which will be fed into the graph later during execution. They can have an optional shape argument. Optimizers extend the TensorFlow class and include additional information to train specific models. Examples of optimizers mentioned include stochastic gradient descent, momentum, AdaGrad, RMSProp, and Adam. Gradient descent optimization works by calculating the error between actual and predicted values, iterating to find the best weights, passing inputs through the network, and initializing random weights/biases which are then updated to reduce error based on neuron contributions to error.

Uploaded by

Devyansh Gupta
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
34 views20 pages

Tensorflow Placeholders and Optimizers

The document discusses placeholders and optimizers in TensorFlow. Placeholders are structures that hold input values which will be fed into the graph later during execution. They can have an optional shape argument. Optimizers extend the TensorFlow class and include additional information to train specific models. Examples of optimizers mentioned include stochastic gradient descent, momentum, AdaGrad, RMSProp, and Adam. Gradient descent optimization works by calculating the error between actual and predicted values, iterating to find the best weights, passing inputs through the network, and initializing random weights/biases which are then updated to reduce error based on neuron contributions to error.

Uploaded by

Devyansh Gupta
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Tensorflow Placeholders and

Optimizers
Placeholders
• TensorFlow, however, has designated built-in
structures for feeding input values. These
structures are called placeholders.
• Placeholders can be thought of as empty Variables
that will be filled with data later on.
• We use them by first constructing our graph and
only when it is executed feeding them with the
input data.
• Placeholders have an optional shape argument.
• If a shape is not fed or is passed as None, then the
placeholder can be fed with data of any size.
• It is common to use None for the dimension of
a matrix that corresponds to the number of
samples (usually rows), while having the length
of the features (usually columns) fixed:
• ph = tf.placeholder(tf.float32,shape=(None,10))
• Whenever we define a placeholder, we must
feed it with some input values or else an
exception will be thrown. The input data is
passed to the session.run() method as a
dictionary, where each key corresponds to a
placeholder variable name, and the matching
values are the data values given in the form of
a list or a NumPy array:
• sess.run(s,feed_dict={x: X_data,w: w_data})
import tensorflow.compat.v1 as tf
from tensorflow import *
import numpy as np
x_data = np.random.randn(5,10)
w_data = np.random.randn(10,1)
with tf.Graph().as_default():
x = tf.placeholder(tf.float32,shape=(5,10))
w = tf.placeholder(tf.float32,shape=(10,1))
b = tf.fill((5,1),-1.)
xw = tf.matmul(x,w)
xwb = xw + b
s = tf.reduce_max(xwb)
with tf.Session() as sess:

outs = sess.run(s,feed_dict={x: x_data,w: w_data})


print("outs = {}".format(outs))
These inputs are matrix-multiplied to create a fiveunit vector xw and added with a
constant vector b filled with the value -1. Finally, the variable s takes the maximum
value of that vector by using the tf.reduce_max() operation. The word reduce is used
because we are reducing a five-unit vector to a single scalar:
Optimizer
• Optimizers are the extended class, which
include added information to train a specific
model. The optimizer class is initialized with
given parameters but it is important to
remember that no Tensor is needed. The
optimizers are used for improving speed and
performance for training a specific model.
Examples
• Stochastic Gradient descent
• Stochastic Gradient descent with gradient clipping
• Momentum
• Nesterov momentum
• Adagrad
• Adadelta
• RMSProp
• Adam
• Adamax
Gradient Descent Optimizer
• Gradient descent optimization is considered
to be an important concept in data science.
• Consider the steps shown below to
understand the implementation of gradient
descent optimization −
import numpy as np
import tensorflow as tf

# === Create data and simulate results =====


tf.compat.v1.disable_eager_execution()
x_data = np.random.randn(2000,3)
w_real = [0.3,0.5,0.1]
b_real = -0.2
noise = np.random.randn(1,2000)*0.1
y_data = np.matmul(w_real,x_data.T) + b_real + noise
After only 10 iterations, the estimated weights and bias
are w = [0.301, 0.498, 0.098] and b = –0.198. The
original parameter values were w = [0.3,0.5,0.1]
and b = –0.2.
Almost a perfect match!
Visualization using Tensorboard
Introduction
• TensorFlow includes a visualization tool, which
is called the TensorBoard. It is used for
analyzing Data Flow Graph and also used to
understand machine-learning models. The
important feature of TensorBoard includes a
view of different types of statistics about the
parameters and details of any graph in vertical
alignment.
• Load Tensorboard:
%load_ext tensorboard

• Start Tensorboard:
%tensorboard --logdir logs
• # Control TensorBoard display. If no port is pro
vided, the most recently launched TensorBoar
d is used
notebook.display(port=6006, height=1000)
QUIZ
• Let us assume we implement an AND function to a single neuron. Below is a
tabular representation of an AND function. What would be the weights and bias?

What would be the weights and bias?


A. Bias = -1.5, w1 = 1, w2 = 1
B. Bias = 1.5, w1 = 2, w2 = 2
C. Bias = 1, w1 = 1.5, w2 = 1.5
D. None of these
A
• What are the steps for using a gradient descent
algorithm?
1. Calculate error between the actual value and the
predicted value
2. Reiterate until you find the best weights of network
3. Pass an input through the network and get values
from output layer
4. Initialize random weight and bias
5. Go to each neurons which contributes to the error
and change its respective values to reduce the error
A. 1, 2, 3, 4, 5
B. 5, 4, 3, 2, 1
C. 4, 3, 1, 5, 2
D. 3, 2, 1, 5, 4
C

You might also like