TensorFlow

0.3 beta

Xihan Li

2018 08 28
 Contents

1 2

2 TensorFlow 4
2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 TensorFlow 8
3.1 TensorFlow 1+1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 TensorFlow 14
4.1 Model Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.2 MLP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.3 CNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.4 RNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.5 DRL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.6 * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.7 Graph Execution *. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

5 TensorFlow 29
5.1 Checkpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.2 TensorBoard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.3 GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

6 TensorFlow 36
6.1 TensorFlow 1+1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

i
Bibliography 42

ii
 TensorFlow, 0.3 beta

Eager Execution | Based on Eager Execution

TensorFlow TensorFlow Eager Execution

Python TensorFlow

- TensorFlow TensorFlow https://www.tensorflowers.cn/b/48

TensorFlow

PDF https://www.tensorflowers.cn/t/6230

GitHub: https://github.com/snowkylin/TensorFlow-cn

This handbook is a concise introduction to TensorFlow based on TensorFlow s Eager Execution mode,
trying to help developers get started with TensorFlow quickly with some basic machine learning and Python
knowledge.

Friendly reminder: If you find something difficult to understand in reading, please check if you have a clear
understanding of the Prerequisites part of each chapter.

Q&A area - TensorFlow Chinese community A Concise Handbook of TensorFlow forum: https://www.
tensorflowers.cn/b/48 (If you have any questions about this tutorial, please ask in this forum of the Tensor-
Flow Chinese community)

PDF download: https://www.tensorflowers.cn/t/6230

GitHub: https://github.com/snowkylin/TensorFlow-cn

Contents 1
 CHAPTER 1

2018 3 30 Google TensorFlow Dev Summit

TensorFlow 1.8 Google
TensorFlow 2017
Eager Execution TensorFlow

The easiest way to get started with TensorFlow is using Eager Execution.

— https://www.tensorflow.org/get_started/

TensorFlow Graph Execution

Python PyTorch

PyTorch
TensorFlow TensorFlow
Graph Execution
TensorFlow Eager Execution
TensorFlow

TensorFlow
TensorFlow
TensorFlow
TensorFlow https://tensorflow.google.cn/tutorials

TensorFlow
/ TensorFlow

2
 TensorFlow, 0.3 beta

• TensorFlow Eager Execution

Graph Execution

• TensorFlow TensorFlow

• TensorFlow
tf.keras.Model tf.keras.layers.Layer

TensorFlow Chris Wu LATEX https:

//github.com/wklchris/Note-by-LaTeX LATEX
Ji-An Li

Zida Jin 1-4 Ming 5-6 Ji-An Li

Ji-An Li

Google TensorFlow
Luke Cheng Rui
Li, Pryce Mu TensorFlow TensorFlow
Tiezhen Wang

https://github.com/snowkylin/TensorFlow-cn/issues

Xihan Li

2018 8

3
 CHAPTER 2

TensorFlow

TensorFlow https://tensorflow.google.cn/install TensorFlow

Python Java Go C Windows OSX Linux
Python

2.1

GPU
Windows

• Python Anaconda Python 3.6

• Python IDE PyCharm Community Professional

• Anaconda Prompt pip install tensorflow

2.2

conda GPU TensorFlow

4
 TensorFlow, 0.3 beta

2.2.1

TensorFlow

1. NVIDIA GPU TensorFlow GPU

1
CPU CUDA Compute Capability
3.0 NVIDIA CUDA Compute Capability

2. Python Anaconda Python

NumPy SciPy Python
TensorFlow Windows Python 3.X

• Anaconda Anaconda PATH

Anaconda
Anaconda Prompt Anaconda

3. GPU NVIDIA CUDA Toolkit cuDNN

• NVIDIA CUDA
cuDNN CUDA

• Ubuntu System Setting

Software & Updates Additional Drivers Using NVIDIA binary driver
Apply Changes NVIDIA NVIDIA
Linux
Nouveau Secure Boot

• CUDA Toolkit cuDNN TensorFlow

• cuDNN CUDA

2.2.2

Anaconda Windows

1. tensorflow conda

conda create -n tensorflow python=X.X # X.X Python 3.6

2.
1 GPU GPU CPU GPU
1-2 GPU NVIDIA GeForce GTX 1080 Ti NVIDIA GeForce TITAN
GPU
TensorFlow CPU TensorFlow TensorFlow
GPU

2.2. 5
 TensorFlow, 0.3 beta

activate tensorflow

3. pip TensorFlow

CPU

pip install tensorflow

GPU

pip install tensorflow-gpu

TensorFlow Nightly
TensorFlow 1.8 Eager Execution Nightly
pip install tf-nightly CPU pip install
tf-nightly-gpu GPU GPU
CUDA cuDNN CUDA cuDNN

pip TensorFlow

2.3

activate tensorflow TensorFlow conda python

Python

import tensorflow as tf
tf.enable_eager_execution()

A = tf.constant([[1, 2], [3, 4]])

B = tf.constant([[5, 6], [7, 8]])
C = tf.matmul(A, B)

print(C)

tf.Tensor(
[[19 22]
[43 50]], shape=(2, 2), dtype=int32)

TensorFlow TensorFlow

Python Python http://www.runoob.com/python3/

python3-tutorial.html https://www.liaoxuefeng.com Python

2.3. 6
 TensorFlow, 0.3 beta

Python TensorFlow Python

Python IDE PyCharm .edu
PyCharm

2.3. 7
 CHAPTER 3

TensorFlow

TensorFlow

• Python import

• Python With

• NumPy Python TensorFlow

[ ]
1 2
• ×
3 4
[ ]
5 6
=?
7 8

• f (x, y) = x2 + xy + y 2 , ∂f ∂f
∂x =?, ∂y =?

3.1 TensorFlow 1+1

[ ] [ ] TensorFlow Python NumPy 1+1

1 2 5 6
× Hello World
3 4 7 8

8
 TensorFlow, 0.3 beta

import tensorflow as tf
tf.enable_eager_execution()

a = tf.constant(1)
b = tf.constant(1)
c = tf.add(a, b) # c = a + b

print(c)

A = tf.constant([[1, 2], [3, 4]])

B = tf.constant([[5, 6], [7, 8]])
C = tf.matmul(A, B)

print(C)

tf.Tensor(2, shape=(), dtype=int32)

tf.Tensor(
[[19 22]
[43 50]], shape=(2, 2), dtype=int32)

a b A B Tensor tf.add() tf.matmul() Operation

c C shape
dtype a b c int32 A B C 2×2 (2, 2)
int32

TensorFlow
tf.GradientTape() y(x) = x2 x=3

import tensorflow as tf
tf.enable_eager_execution()

x = tf.get_variable('x', shape=[1], initializer=tf.constant_initializer(3.))

with tf.GradientTape() as tape: # tf.GradientTape()

y = tf.square(x)
y_grad = tape.gradient(y, x) # y x
print([y.numpy(), y_grad.numpy()])

[array([9.], dtype=float32), array([6.], dtype=float32)]

x 3 Variable tf.get_variable()

3.1. TensorFlow 1+1 9

 TensorFlow, 0.3 beta

shape dtype tf.get_variable()

initializer tf.constant_initializer(3.) x
1
float32 3. TensorFlow
tf.GradientTape()
x y = tf.square(x)
y_grad = tape.gradient(y, x) y x

TensorFlow
tf.GradientTape()
[ ] [ ] L(w, b) = ∥Xw +b−y∥ 2
w = (1, 2)T , b = 1
1 2 1
w, b X= ,y =
3 4 2

X = tf.constant([[1., 2.], [3., 4.]])

y = tf.constant([[1.], [2.]])
w = tf.get_variable('w', shape=[2, 1], initializer=tf.constant_initializer([[1.], [2.]]))
b = tf.get_variable('b', shape=[1], initializer=tf.constant_initializer([1.]))
with tf.GradientTape() as tape:
L = 0.5 * tf.reduce_sum(tf.square(tf.matmul(X, w) + b - y))
w_grad, b_grad = tape.gradient(L, [w, b]) # L(w, b) w, b
print([L.numpy(), w_grad.numpy(), b_grad.numpy()])

[62.5, array([[35.],
[50.]], dtype=float32), array([15.], dtype=float32)]

tf.square() tf.reduce_sum()
axis
TensorFlow API
2
tf.reshape() tf.concat() TensorFlow API

TensorFlow

L((1, 2)T , 1) = 62.5

[ ]
∂L(w, b) 35
|w=(1,2)T ,b=1 =
∂w 50
∂L(w, b)
|w=(1,2)T ,b=1 = 15
∂b
1 Python 3. 3.0
2 Tensor Transformations Math TensorFlow API Python
NumPy

3.1. TensorFlow 1+1 10

 TensorFlow, 0.3 beta

3.2

2013 -2017

2013 2014 2015 2016 2017

12000 14000 15000 16500 17500

y = ax + b a b

import numpy as np

X_raw = np.array([2013, 2014, 2015, 2016, 2017])

y_raw = np.array([12000, 14000, 15000, 16500, 17500])

X = (X_raw - X_raw.min()) / (X_raw.max() - X_raw.min())

y = (y_raw - y_raw.min()) / (y_raw.max() - y_raw.min())

3
a b

f (x)

• x0 k=0

– f (x) ∇f (xk )

– xk+1 = xk − γ∇f (xk ) γ

– k ←k+1
∑n 2
mina,b L(a, b) = i=1 (axi +b−yi )

3.2.1 NumPy

TensorFlow
NumPy NumPy

np.dot() np.sum() NumPy MATLAB

3 TensorFlow

3.2. 11
 TensorFlow, 0.3 beta

4
a b a
b

a, b = 0, 0

num_epoch = 10000
learning_rate = 1e-3
for e in range(num_epoch):
#
y_pred = a * X + b
grad_a, grad_b = (y_pred - y).dot(X), (y_pred - y).sum()

#
a, b = a - learning_rate * grad_a, b - learning_rate * grad_b

print(a, b)

•
Adam Adagrad

TensorFlow

3.2.2 TensorFlow

5
TensorFlow Eager Execution NumPy
GPU
TensorFlow NumPy TensorFlow

• tape.gradient(ys, xs)

• optimizer.apply_gradients(grads_and_vars)

X = tf.constant(X)
y = tf.constant(y)

4
∑5 ∑5
L(x) = 1
i=1 (axi + b − yi )2 a b ∂L
= i=1 (axi + b − y)xi
∑5 2 ∂a
i=1 (axi + b − y)
∂L
∂b
=
5 Eager Execution Graph Execution TensorFlow 2018 3 1.8

3.2. 12
 TensorFlow, 0.3 beta

a = tf.get_variable('a', dtype=tf.float32, shape=[], initializer=tf.zeros_initializer)

b = tf.get_variable('b', dtype=tf.float32, shape=[], initializer=tf.zeros_initializer)
variables = [a, b]

num_epoch = 10000
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-3)
for e in range(num_epoch):
# tf.GradientTape()
with tf.GradientTape() as tape:
y_pred = a * X + b
loss = 0.5 * tf.reduce_sum(tf.square(y_pred - y))
# TensorFlow
grads = tape.gradient(loss, variables)
# TensorFlow
optimizer.apply_gradients(grads_and_vars=zip(grads, variables))

tf.train.
GradientDescentOptimizer(learning_rate=1e-3) Optimizer
1e-3
apply_gradients()

optimizer.apply_gradients() grads_and_vars
variables grads
Python List
[(grad_w, w), (grad_b, b)] grads = tape.gradient(loss, variables)
tape loss variables = [w, b] grads = [grad_w, grad_b]
Python zip() grads = [grad_w, grad_b] vars = [w, b]

y_pred = tf.matmul(X, w) + b
y_pred = model(X)

3.2. 13
 CHAPTER 4

TensorFlow

TensorFlow

• Python Python super()

__call__()

4.1 Model Layer

y_pred
= model(X) __init__()
1
call(input)

class MyModel(tf.keras.Model):
def __init__(self):
super().__init__() # Python 2 super(MyModel, self).__init__()
# call

def call(self, inputs):

1 Python myClass myClass() myClass.__call__()

tf.keras.Model __call__() call() keras
tf.keras.Model call() keras
__call__()

14
 TensorFlow, 0.3 beta

#
return output

tf.keras.Model Keras Python API

TensorFlow tf.keras.Model
model.variables

Layer

y_pred = tf.matmul(X, w) + b

import tensorflow as tf
tf.enable_eager_execution()

X = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])

y = tf.constant([[10.0], [20.0]])

class Linear(tf.keras.Model):
def __init__(self):
super().__init__()
self.dense = tf.keras.layers.Dense(units=1, kernel_initializer=tf.zeros_initializer(),
bias_initializer=tf.zeros_initializer())

def call(self, input):

output = self.dense(input)
return output

#
model = Linear()
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
for i in range(100):
with tf.GradientTape() as tape:
y_pred = model(X) #
loss = tf.reduce_mean(tf.square(y_pred - y))
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))
print(model.variables)

w b y_pred = tf.matmul(X, w) + b
tf.keras.layers.Dense call
output = activation(tf.matmul(input, kernel) + bias) +

4.1. Model Layer 15

 TensorFlow, 0.3 beta

kernel bias activation(x) = x

4.2 MLP

Multilayer Perceptron, MLP TensorFlow

MNIST [LeCun1998]

4.1: MNIST

DataLoader MNIST

class DataLoader():
def __init__(self):
mnist = tf.contrib.learn.datasets.load_dataset("mnist")
self.train_data = mnist.train.images # np.array [55000,␣
,→ 784]
self.train_labels = np.asarray(mnist.train.labels, dtype=np.int32) # np.array [55000] of␣
,→ int32
self.eval_data = mnist.test.images # np.array [10000,␣
,→ 784]
self.eval_labels = np.asarray(mnist.test.labels, dtype=np.int32) # np.array [10000] of␣
,→ int32

def get_batch(self, batch_size):

index = np.random.randint(0, np.shape(self.train_data)[0], batch_size)
return self.train_data[index, :], self.train_labels[index]

ReLU activation=tf.nn.relu
1×784 10 0
9 predict

class MLP(tf.keras.Model):
def __init__(self):
super().__init__()

4.2. MLP 16
 TensorFlow, 0.3 beta

self.dense1 = tf.keras.layers.Dense(units=100, activation=tf.nn.relu)

self.dense2 = tf.keras.layers.Dense(units=10)

def call(self, inputs):

x = self.dense1(inputs)
x = self.dense2(x)
return x

def predict(self, inputs):

logits = self(inputs)
return tf.argmax(logits, axis=-1)

num_batches = 1000
batch_size = 50
learning_rate = 0.001

model = MLP()
data_loader = DataLoader()
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)

• DataLoader

• loss

for batch_index in range(num_batches):

X, y = data_loader.get_batch(batch_size)
with tf.GradientTape() as tape:
y_logit_pred = model(tf.convert_to_tensor(X))
loss = tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_logit_pred)
print("batch %d: loss %f" % (batch_index, loss.numpy()))
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))

4.2. MLP 17
 TensorFlow, 0.3 beta

num_eval_samples = np.shape(data_loader.eval_labels)[0]
y_pred = model.predict(data_loader.eval_data).numpy()
print("test accuracy: %f" % (sum(y_pred == data_loader.eval_labels) / num_eval_samples))

test accuracy: 0.947900

95%

4.3 CNN

Convolutional Neural Network, CNN

Convolutional Layer Pooling Layer Dense Layer
Convolutional Neural Network

MLP

4.2: CNN

class CNN(tf.keras.Model):
def __init__(self):
super().__init__()
self.conv1 = tf.keras.layers.Conv2D(
filters=32, #
kernel_size=[5, 5], #
padding="same", # padding
activation=tf.nn.relu #
)
self.pool1 = tf.keras.layers.MaxPool2D(pool_size=[2, 2], strides=2)
self.conv2 = tf.keras.layers.Conv2D(
filters=64,

4.3. CNN 18
 TensorFlow, 0.3 beta

kernel_size=[5, 5],
padding="same",
activation=tf.nn.relu
)
self.pool2 = tf.keras.layers.MaxPool2D(pool_size=[2, 2], strides=2)
self.flatten = tf.keras.layers.Reshape(target_shape=(7 * 7 * 64,))
self.dense1 = tf.keras.layers.Dense(units=1024, activation=tf.nn.relu)
self.dense2 = tf.keras.layers.Dense(units=10)

def call(self, inputs):

inputs = tf.reshape(inputs, [-1, 28, 28, 1])
x = self.conv1(inputs) # [batch_size, 28, 28, 32]
x = self.pool1(x) # [batch_size, 14, 14, 32]
x = self.conv2(x) # [batch_size, 14, 14, 64]
x = self.pool2(x) # [batch_size, 7, 7, 64]
x = self.flatten(x) # [batch_size, 7 * 7 * 64]
x = self.dense1(x) # [batch_size, 1024]
x = self.dense2(x) # [batch_size, 10]
return x

def predict(self, inputs):

logits = self(inputs)

model = MLP() model = CNN() :

test accuracy: 0.988100

Dropout

4.4 RNN

Recurrent Neural Network, RNN

RNN

• Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs

• Recurrent Neural Network (part 1) Recurrent Neural

Network (part 2)

• LSTM Understanding LSTM Networks

• RNN [Graves2013]

4.4. RNN 19
 TensorFlow, 0.3 beta

2
RNN

I am a studen

13 13
t num_batch
seq_length [num_batch, seq_length]
num_chars [num_batch,
num_chars]

DataLoader

class DataLoader():
def __init__(self):
path = tf.keras.utils.get_file('nietzsche.txt',
origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')
with open(path, encoding='utf-8') as f:
self.raw_text = f.read().lower()
self.chars = sorted(list(set(self.raw_text)))
self.char_indices = dict((c, i) for i, c in enumerate(self.chars))
self.indices_char = dict((i, c) for i, c in enumerate(self.chars))
self.text = [self.char_indices[c] for c in self.raw_text]

def get_batch(self, seq_length, batch_size):

seq = []
next_char = []
for i in range(batch_size):
index = np.random.randint(0, len(self.text) - seq_length)
seq.append(self.text[index:index+seq_length])
next_char.append(self.text[index+seq_length])
return np.array(seq), np.array(next_char) # [num_batch, seq_length], [num_batch]

__init__ BasicLSTMCell
One Hot i n i 1
0 n num_char [num_batch, seq_length, num_chars]
RNN t RNN state t
inputs[:, t, :] RNN output t+1 RNN
RNN num_chars

2 https://github.com/keras-team/keras/blob/master/examples/lstm_text_generation.py

4.4. RNN 20
 TensorFlow, 0.3 beta

4.3: output, state = self.cell(inputs[:, t, :], state)

4.4: RNN

4.4. RNN 21
 TensorFlow, 0.3 beta

class RNN(tf.keras.Model):
def __init__(self, num_chars):
super().__init__()
self.num_chars = num_chars
self.cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=256)
self.dense = tf.keras.layers.Dense(units=self.num_chars)

def call(self, inputs):

batch_size, seq_length = tf.shape(inputs)
inputs = tf.one_hot(inputs, depth=self.num_chars) # [batch_size, seq_length, num_
,→ chars]
state = self.cell.zero_state(batch_size=batch_size, dtype=tf.float32)
for t in range(seq_length.numpy()):
output, state = self.cell(inputs[:, t, :], state)
output = self.dense(output)
return output

• DataLoader

• loss

data_loader = DataLoader()
model = RNN(len(data_loader.chars))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
for batch_index in range(num_batches):
X, y = data_loader.get_batch(seq_length, batch_size)
with tf.GradientTape() as tape:
y_logit_pred = model(X)
loss = tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_logit_pred)
print("batch %d: loss %f" % (batch_index, loss.numpy()))
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))

tf.argmax()

np.random.choice()
temperature

4.4. RNN 22
 TensorFlow, 0.3 beta

def predict(self, inputs, temperature=1.):

batch_size, _ = tf.shape(inputs)
logits = self(inputs)
prob = tf.nn.softmax(logits / temperature).numpy()
return np.array([np.random.choice(self.num_chars, p=prob[i, :])
for i in range(batch_size.numpy())])

X_, _ = data_loader.get_batch(seq_length, 1)
for diversity in [0.2, 0.5, 1.0, 1.2]:
X = X_
print("diversity %f:" % diversity)
for t in range(400):
y_pred = model.predict(X, diversity)
print(data_loader.indices_char[y_pred[0]], end='', flush=True)
X = np.concatenate([X[:, 1:], np.expand_dims(y_pred, axis=1)], axis=-1)

diversity 0.200000:
conserted and conseive to the conterned to it is a self--and seast and the selfes as a seast the␣
,→ expecience and and and the self--and the sered is a the enderself and the sersed and as a the␣
,→ concertion of the series of the self in the self--and the serse and and the seried enes and␣
,→ seast and the sense and the eadure to the self and the present and as a to the self--and the␣
,→ seligious and the enders

diversity 0.500000:
can is reast to as a seligut and the complesed
has fool which the self as it is a the beasing and us immery and seese for entoured underself of␣
,→ the seless and the sired a mears and everyther to out every sone thes and reapres and seralise␣
,→ as a streed liees of the serse to pease the cersess of the selung the elie one of the were as we␣
,→ and man one were perser has persines and conceity of all self-el

diversity 1.000000:
entoles by
their lisevers de weltaale, arh pesylmered, and so jejurted count have foursies as is
descinty iamo; to semplization refold, we dancey or theicks-welf--atolitious on his
such which
here
oth idey of pire master, ie gerw their endwit in ids, is an trees constenved mase commars is leed␣
,→ mad decemshime to the mor the elige. the fedies (byun their ope wopperfitious--antile and the it␣
,→ as the f

diversity 1.200000:

4.4. RNN 23
 TensorFlow, 0.3 beta

cain, elvotidue, madehoublesily

inselfy!--ie the rads incults of to prusely le]enfes patuateded:.--a coud--theiritibaior
,→ "nrallysengleswout peessparify oonsgoscess teemind thenry ansken suprerial mus, cigitioum: 4reas.
,→ whouph: who
eved
arn inneves to sya" natorne. hag open reals whicame oderedte,[fingo is
zisternethta simalfule dereeg hesls lang-lyes thas quiin turjentimy; periaspedey tomm--whach

4.5 DRL

Reinforcement learning RL
AlphaGo

• Demystifying Deep Reinforcement Learning

• [Mnih2013]

CartPole

4.5: CartPole

OpenAI Gym CartPole

Gym

import gym

4.5. DRL 24
 TensorFlow, 0.3 beta

env = gym.make('CartPole-v1') #
state = env.reset() #
while True:
env.render() #
action = model.predict(state) #

next_state, reward, done, info = env.step(action) #

if done: #
break

Deep Q-Learning

import tensorflow as tf
import numpy as np
import gym
import random
from collections import deque

tf.enable_eager_execution()
num_episodes = 500
num_exploration_episodes = 100
max_len_episode = 1000
batch_size = 32
learning_rate = 1e-3
gamma = 1.
initial_epsilon = 1.
final_epsilon = 0.01

# Q-network Q state action Q-value CartPole

2
class QNetwork(tf.keras.Model):
def __init__(self):
super().__init__()
self.dense1 = tf.keras.layers.Dense(units=24, activation=tf.nn.relu)
self.dense2 = tf.keras.layers.Dense(units=24, activation=tf.nn.relu)
self.dense3 = tf.keras.layers.Dense(units=2)

def call(self, inputs):

x = self.dense1(inputs)
x = self.dense2(x)

4.5. DRL 25
 TensorFlow, 0.3 beta

x = self.dense3(x)
return x

def predict(self, inputs):

q_values = self(inputs)
return tf.argmax(q_values, axis=-1)

env = gym.make('CartPole-v1') #
model = QNetwork()
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
replay_buffer = deque(maxlen=10000)
epsilon = initial_epsilon
for episode_id in range(num_episodes):
state = env.reset() #
epsilon = max(
initial_epsilon * (num_exploration_episodes - episode_id) / num_exploration_episodes,
final_epsilon)
for t in range(max_len_episode):
env.render() #
if random.random() < epsilon: # epsilon-greedy
action = env.action_space.sample() # epsilon
else:
action = model.predict(
tf.constant(np.expand_dims(state, axis=0), dtype=tf.float32)).numpy()
action = action[0]
next_state, reward, done, info = env.step(action) #

reward = -10. if done else reward # Game Over

replay_buffer.append((state, action, reward, next_state, done)) # (state, action,␣

,→ reward, next_state) done
state = next_state

if done: #
episode
print("episode %d, epsilon %f, score %d" % (episode_id, epsilon, t))
break

if len(replay_buffer) >= batch_size:

batch_state, batch_action, batch_reward, batch_next_state, batch_done = \
[np.array(a, dtype=np.float32) for a in zip(*random.sample(replay_buffer, batch_
,→ size))] # batch
q_value = model(tf.constant(batch_next_state, dtype=tf.float32))
y = batch_reward + (gamma * tf.reduce_max(q_value, axis=1)) * (1 - batch_done) #
y

4.5. DRL 26
 TensorFlow, 0.3 beta

with tf.GradientTape() as tape:

loss = tf.losses.mean_squared_error( # y Q-value
labels=y,
predictions=tf.reduce_sum(model(tf.constant(batch_state)) *
tf.one_hot(batch_action, depth=2), axis=1)
)
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables)) #

4.6 *

tf.keras.Model tf.keras.layers.Layer

class MyLayer(tf.keras.layers.Layer):
def __init__(self):
super().__init__()
#

def build(self, input_shape): # input_shape TensorShape

#
#
# __init__
self.variable_0 = self.add_variable(...)
self.variable_1 = self.add_variable(...)

def call(self, input):

#
return output

1
build call

class LinearLayer(tf.keras.layers.Layer):
def __init__(self):
super().__init__()

def build(self, input_shape): # here input_shape is a TensorShape

self.w = self.add_variable(name='w',
shape=[input_shape[-1], 1], initializer=tf.zeros_initializer())

4.6. * 27
 TensorFlow, 0.3 beta

self.b = self.add_variable(name='b',
shape=[1], initializer=tf.zeros_initializer())

def call(self, X):

y_pred = tf.matmul(X, self.w) + self.b
return y_pred

LinearLayer

class Linear(tf.keras.Model):
def __init__(self):
super().__init__()
self.layer = LinearLayer()

def call(self, input):

output = self.layer(input)
return output

4.7 Graph Execution *

Eager Execution Graph

3
Execution Graph Execution model(input_tensor)

model = Linear()
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
X_placeholder = tf.placeholder(name='X', shape=[None, 3], dtype=tf.float32)
y_placeholder = tf.placeholder(name='y', shape=[None, 1], dtype=tf.float32)
y_pred = model(X_placeholder)
loss = tf.reduce_mean(tf.square(y_pred - y_placeholder))
train_op = optimizer.minimize(loss)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(100):
sess.run(train_op, feed_dict={X_placeholder: X, y_placeholder: y})
print(sess.run(model.variables))

3 RNN RNN Eager Execution seq_length

RNN Graph Execution seq_length
tf.nn.dynamic_rnn

4.7. Graph Execution * 28

 CHAPTER 5

TensorFlow

TensorFlow

• Python Pickle

• Python **kwargs

5.1 Checkpoint

Python pickle
model.variables TensorFlow ResourceVariable

TensorFlow tf.train.Checkpoint save()

restore() TensorFlow Checkpointable State tf.
train.Optimizer , tf.Variable, tf.keras.Layer tf.keras.Model
Checkpoint

checkpoint = tf.train.Checkpoint(model=model)

tf.train.Checkpoint() **kwargs
tf.keras.Model
model tf.train.Optimizer optimizer

29
 TensorFlow, 0.3 beta

checkpoint = tf.train.Checkpoint(myAwesomeModel=model, myAwesomeOptimizer=optimizer)

myAwesomeModel model

checkpoint.save(save_path_with_prefix)

save_path_with_prefix + save
checkpoint.save('./save/model.ckpt') save
checkpoint model.ckpt-1.index model.ckpt-1.data-00000-of-00001
checkpoint.save() .index .data

checkpoint
checkpoint restore

model_to_be_restored = MyModel() #
checkpoint = tf.train.Checkpoint(myAwesomeModel=model_to_be_restored) #
myAwesomeModel
checkpoint.restore(save_path_with_prefix_and_index)

save_path_with_prefix_and_index + +
checkpoint.restore('./save/model.ckpt-1') model.ckpt 1

tf.train.latest_checkpoint(save_path)
checkpoint save model.ckpt-1.index
model.ckpt-10.index 10 tf.train.latest_checkpoint('./save') ./save/
model.ckpt-10

# train.py

model = MyModel()
checkpoint = tf.train.Checkpoint(myModel=model) # Checkpoint model
Optimizer
#
checkpoint.save('./save/model.ckpt') #

# test.py

model = MyModel()

5.1. Checkpoint 30
 TensorFlow, 0.3 beta

checkpoint = tf.train.Checkpoint(myModel=model) # Checkpoint model

checkpoint.restore(tf.train.latest_checkpoint('./save')) #
#

tf.train.Checkpoint tf.train.Saver Eager

Execution checkpoint.restore()
Checkpoint Eager Execution

train.py tf.keras.Model
save_weight() model test.py model load_weight()
model load_weight() tf.train.
Checkpoint tf.train.Checkpoint Graph
Execution

import tensorflow as tf
import numpy as np
from zh.model.mlp.mlp import MLP
from zh.model.mlp.utils import DataLoader

tf.enable_eager_execution()
mode = 'test'
num_batches = 1000
batch_size = 50
learning_rate = 0.001
data_loader = DataLoader()

def train():
model = MLP()
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
checkpoint = tf.train.Checkpoint(myAwesomeModel=model) # Checkpoint
model
for batch_index in range(num_batches):
X, y = data_loader.get_batch(batch_size)
with tf.GradientTape() as tape:
y_logit_pred = model(tf.convert_to_tensor(X))
loss = tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_logit_pred)
print("batch %d: loss %f" % (batch_index, loss.numpy()))
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))
if (batch_index + 1) % 100 == 0: # 100 Batch

5.1. Checkpoint 31
 TensorFlow, 0.3 beta

checkpoint.save('./save/model.ckpt') #

def test():
model_to_be_restored = MLP()
checkpoint = tf.train.Checkpoint(myAwesomeModel=model_to_be_restored) # Checkpoint
model_to_be_restored
checkpoint.restore(tf.train.latest_checkpoint('./save')) #
num_eval_samples = np.shape(data_loader.eval_labels)[0]
y_pred = model_to_be_restored.predict(tf.constant(data_loader.eval_data)).numpy()
print("test accuracy: %f" % (sum(y_pred == data_loader.eval_labels) / num_eval_samples))

if __name__ == '__main__':
if mode == 'train':
train()
if mode == 'test':
test()

save save 100 batch

7 model = 'test'
95%

5.2 TensorBoard

loss
TensorBoard

Eager Execution TensorBoard tf.contrib.summary

./tensorboard TensorBoard

summary_writer = tf.contrib.summary.create_file_writer('./tensorboard')

with summary_writer.as_default() tf.contrib.summary.

always_record_summaries() scalar tf.contrib.summary.
scalar(name, tensor, step=batch_index) step
batch

summary_writer = tf.contrib.summary.create_file_writer('./tensorboard')
with summary_writer.as_default(), tf.contrib.summary.always_record_summaries():
#
for batch_index in range(num_batches):

5.2. TensorBoard 32
 TensorFlow, 0.3 beta

# batch loss
tf.contrib.summary.scalar("loss", loss, step=batch_index)
tf.contrib.summary.scalar("MyScalar", my_scalar, step=batch_index) #

tf.contrib.summary.scalar()
scalar TensorBoard API

TensorFlow conda :

tensorboard --logdir=./tensorboard

http:// :6006 TensorBoard

TensorBoard 30

TensorBoard

• TensorBoard
TensorBoard --logdir

TensorBoard

import tensorflow as tf
import numpy as np
from zh.model.mlp.mlp import MLP
from zh.model.mlp.utils import DataLoader

tf.enable_eager_execution()

5.2. TensorBoard 33
 TensorFlow, 0.3 beta

num_batches = 10000
batch_size = 50
learning_rate = 0.001
model = MLP()
data_loader = DataLoader()
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
summary_writer = tf.contrib.summary.create_file_writer('./tensorboard') #
with summary_writer.as_default(), tf.contrib.summary.always_record_summaries():
for batch_index in range(num_batches):
X, y = data_loader.get_batch(batch_size)
with tf.GradientTape() as tape:
y_logit_pred = model(tf.convert_to_tensor(X))
loss = tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_logit_pred)
print("batch %d: loss %f" % (batch_index, loss.numpy()))
tf.contrib.summary.scalar("loss", loss, step=batch_index) # loss
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))

5.3 GPU

/ / GPU

nvidia-smi GPU Windows C:\Program Files\NVIDIA

Corporation\NVSMI Path Windows 10

CUDA_VISIBLE_DEVICES GPU 0,1

2,3 Linux :

export CUDA_VISIBLE_DEVICES=2,3

import os
os.environ['CUDA_VISIBLE_DEVICES'] = "2,3"

2,3

TensorFlow tf.
ConfigProto TensorFlow tf.ConfigProto
tf.enable_eager_execution() Config allow_growth
TensorFlow

5.3. GPU 34
 TensorFlow, 0.3 beta

config = tf.ConfigProto()
config.gpu_options.allow_growth = True
tf.enable_eager_execution(config=config)

per_process_gpu_memory_fraction TensorFlow 40% GPU

config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.4
tf.enable_eager_execution(config=config)

Graph Execution session tf.ConfigPhoto

5.3. GPU 35
 CHAPTER 6

TensorFlow

6.1 TensorFlow 1+1

TensorFlow 1+1 Hello World

import tensorflow as tf

#
a = tf.constant(1) # Tensor
b = tf.constant(1)
c = a + b # c = tf.add(a, b) c a b Add Operation

sess = tf.Session() # Session

c_ = sess.run(c) # Session run()
print(c_)

1+1 tf.placeholder() sess.run() feed_dict=

TensorFlow

import tensorflow as tf

36
 TensorFlow, 0.3 beta

a = tf.placeholder(dtype=tf.int32) # Tensor
b = tf.placeholder(dtype=tf.int32)
c = a + b

a_ = input("a = ") # a_
b_ = input("b = ")

sess = tf.Session()
c_ = sess.run(c, feed_dict={a: a_, b: b_}) # feed_dict c
print("a + b = %d" % c_)

>>> a = 2
>>> b = 3
a + b = 5

** ** Variable tf.get_variable()

0 1

import tensorflow as tf

a = tf.get_variable(name='a', shape=[])
initializer = tf.assign(a, 0) # tf.assign(x, y) y x
a_plus_1 = a + 1 # a + tf.constant(1)
plus_one_op = tf.assign(a, a_plus_1)

sess = tf.Session()
sess.run(initializer)
for i in range(5):
sess.run(plus_one_op) # a
a_ = sess.run(a) # a a_
print(a_)

1.0
2.0
3.0
4.0
5.0

tf.global_variables_initializer()

6.1. TensorFlow 1+1 37

 TensorFlow, 0.3 beta

import tensorflow as tf

a = tf.get_variable(name='a', shape=[], initializer=tf.zeros_initializer) # 0

a_plus_1 = a + 1
plus_one_op = tf.assign(a, a_plus_1)

sess = tf.Session()
sess.run(tf.global_variables_initializer()) #
for i in range(5):
sess.run(plus_one_op)
a_ = sess.run(a)
print(a_)
[ ]
1 1 1
1 1 1
 
1 1
 
1 1
 
1 1

import tensorflow as tf

A = tf.ones(shape=[2, 3]) # tf.ones(shape) shape 1

B = tf.ones(shape=[3, 2])
C = tf.matmul(A, B)

sess = tf.Session()
C_ = sess.run(C)
print(C_)

[[3. 3.]
[3. 3.]]

Placeholder Variable

6.2

NumPy Eager Execution TensorFlow Graph Execution

TensorFlow

6.2. 38
 TensorFlow, 0.3 beta

tf.placeholder() tf.get_variable() Vari-

able tf.assign() sess.run(output_node, feed_dict={input_node:
data})

import tensorflow as tf

#
learning_rate_ = tf.placeholder(dtype=tf.float32)
X_ = tf.placeholder(dtype=tf.float32, shape=[5])
y_ = tf.placeholder(dtype=tf.float32, shape=[5])
a = tf.get_variable('a', dtype=tf.float32, shape=[], initializer=tf.zeros_initializer)
b = tf.get_variable('b', dtype=tf.float32, shape=[], initializer=tf.zeros_initializer)

y_pred = a * X_ + b
loss = tf.constant(0.5) * tf.reduce_sum(tf.square(y_pred - y_))

#
grad_a = tf.reduce_sum((y_pred - y_) * X_)
grad_b = tf.reduce_sum(y_pred - y_)

#
new_a = a - learning_rate_ * grad_a
new_b = b - learning_rate_ * grad_b
update_a = tf.assign(a, new_a)
update_b = tf.assign(b, new_b)

train_op = [update_a, update_b]

#
#

num_epoch = 10000
learning_rate = 1e-3
with tf.Session() as sess:
# a b
tf.global_variables_initializer().run()
#
for e in range(num_epoch):
sess.run(train_op, feed_dict={X_: X, y_: y, learning_rate_: learning_rate})
print(sess.run([a, b]))

TensorFlow
TensorFlow tf.gradients(ys, xs) loss
a b

6.2. 39
 TensorFlow, 0.3 beta

#
grad_a = tf.reduce_sum((y_pred - y_) * X_)
grad_b = tf.reduce_sum(y_pred - y_)

grad_a, grad_b = tf.gradients(loss, [a, b])

TensorFlow optimizer

#
grad_a = tf.reduce_sum((y_pred - y_) * X_)
grad_b = tf.reduce_sum(y_pred - y_)

#
new_a = a - learning_rate_ * grad_a
new_b = b - learning_rate_ * grad_b
update_a = tf.assign(a, new_a)
update_b = tf.assign(b, new_b)

train_op = [update_a, update_b]

optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate_)
grad = optimizer.compute_gradients(loss)
train_op = optimizer.apply_gradients(grad)

TensorFlow tf.train.GradientDescentOptimizer()
compute_gradients(loss) loss
apply_gradients(grad)

train_op = tf.train.GradientDescentOptimizer(learning_rate=learning_rate_).minimize(loss)

import tensorflow as tf

learning_rate_ = tf.placeholder(dtype=tf.float32)
X_ = tf.placeholder(dtype=tf.float32, shape=[5])
y_ = tf.placeholder(dtype=tf.float32, shape=[5])

6.2. 40
 TensorFlow, 0.3 beta

a = tf.get_variable('a', dtype=tf.float32, shape=[], initializer=tf.zeros_initializer)

b = tf.get_variable('b', dtype=tf.float32, shape=[], initializer=tf.zeros_initializer)

y_pred = a * X_ + b
loss = tf.constant(0.5) * tf.reduce_sum(tf.square(y_pred - y_))

# TensorFlow
train_op = tf.train.GradientDescentOptimizer(learning_rate=learning_rate_).minimize(loss)

num_epoch = 10000
learning_rate = 1e-3
with tf.Session() as sess:
tf.global_variables_initializer().run()
for e in range(num_epoch):
sess.run(train_op, feed_dict={X_: X, y_: y, learning_rate_: learning_rate})
print(sess.run([a, b]))

6.2. 41
 Bibliography

[LeCun1998] 25. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to docu-
ment recognition. Proceedings of the IEEE, 86(11):2278-2324, November 1998. http://yann.lecun.
com/exdb/mnist/

[Graves2013] Graves, Alex. Generating Sequences With Recurrent Neural Networks. ArXiv:1308.0850
[Cs], August 4, 2013. http://arxiv.org/abs/1308.0850.

[Mnih2013] Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan
Wierstra, and Martin Riedmiller. Playing Atari with Deep Reinforcement Learning. ArXiv:1312.5602
[Cs], December 19, 2013. http://arxiv.org/abs/1312.5602.

42