Basic of Python

Nguyễn Thanh Chuyên

Khoa Công nghệ Thông tin – ĐH Công Nghiệp
[email protected]
0938487684
 Why Python?
• Python: Rapidly becoming the
default platform for practical
machine learning/data science.
• This was not always the case:
10 years ago everyone was
using Matlab.
• However, due to licensing
issues and heavy development
of Python, scientific Python
started to gain its user base.
• Python’s strength is in its
variability and huge community
 Essential Modules
• numpy: The matrix / numerical analysis layer at the bottom
• scipy: Scientific computing utilities (linalg, FFT, signal/image
processing...)
• scikit-learn: Machine learning
• matplotlib: Plotting and visualization
• opencv: Computer vision
• pandas: Data analysis
• statsmodels: Statistics in Python
• Tensorflow, keras: Deep learning
• PyCharm: Editor
• spyder: Scientific PYthon Development EnviRonment (another
editor)
Where to get Python?
 Language
• Python was designed to be a highly readable
language.
• Python uses whitespace to delimit program blocks.
• All used modules are imported using an import
declaration.
ˆThe members of a module are referred using the
dot: np.cos([1,2,3])
• Interpreted language. Also interactive with
IPython
extensions.
 Editors
• We can use the Spyder editor, Visual Studio Code,
PyCharm.
• The window contains panes: editor and console.
• F5 : Run code; F9 : Run selected region.
• Alternatively, you can use whatever editor you like, and
run everything on the
• Python code can be executed either from a script file
(*.py) or in the interactive mode (just like Matlab).
• For the interactive mode; just execute python from the
command line.
 Help
• For each command, help is there to refresh your
memory
 Using Modules
• Python libraries are called modules.
• Each module needs to be imported
before use.
• Three common alternatives:
• Import the full module:
import numpy
• Import selected functions from the
module:
from numpy import array, sin,
cos
• Import all functions from the module:
from numpy import *
 Using Modules
• All methods support shortcuts; e.g.,
import numpy as np
• Sometimes import <module> fails, if the module is in fact a
collection of modules.
• For example, import scipy. Instead, use import scipy.signal

• Importing all functions from the module is not recommended,

because different modules may contain functions with the same
name.
 Using Modules
• Practically all scientific
computing in Python is based on
numpy and scipy modules.
• NumPy provides a numerical
array as an alternative to Python
list.
• The list type is very generic and
accepts any mixture of data types.
• Although practical for generic
manipulation, it is becomes
inefficient in computing.
• Instead, the NumPy array is more
limited and more focused on
numerical computing.
More on Vectors
Matrices
Matrix Product
Indexing
Indexing
 Indexing
• Also N-dimensional arrays (e.g., matrices) can be
indexed similarly. This operation is called slicing, and
the result is a slice of the matrix.
• In the example on the below, we extract items on the
rows 2:4 = [2,3] and columns 1,2,4 (shown in red).
• Note: the first index is the row; not "x-coordinate".
 Indexing
• To specify only column or row indices, use ":"
alone.
• Now we wish to extract two bottom rows.
 N-dimensional array
• Higher-dimensional arrays
are frequently encountered
in machine learning.
• For example, a set of 1000
color images of size w × h
= 128 × 96 is represented
as a 1000 × 96 × 128 × 3
array.
• Here, dimensions are:
image index, y-coordinate,
x-coordinate, color
channel.
 Functions
• Functions are defined using the def keyword.
• Function definition can appear anywhere in the
code.
• Functions can be imported to other files using
import.
• Function arguments can be positional or named (see
code).
• Named arguments improve readability and are handy
for setting the last argument in a long list.
Functions
 Loops and Stuff
• Loops and other usual programming constructs are easy to remember.
• for can loop over anything iterable, such as a list or a file.
• In Matlab, appending values to a vector in a loop is not recommended.
Python lists are actual lists, so appending is fine.
 Reading in a Data file
• Suppose we need to read a csv file (text file with Comma Separated
Values) into Python.
• The file consists of 216 rows (samples) with 4000 measurements
each.
• We will write file reading code from scratch.
• Alternatively, many modules contain csv-reading functions
• numpy.loadtxt
• numpy.genfromtxt
• csv.reader
• pandas.read_csv
Reading in a Data file
 Visualization
• The matplotlib module is our plotting library.
• Function names are often similar to Matlab.
• Usually you want to "import matplotlib.pyplot".
• Alternatively, "from matplotlib.pylab import *"
makes the environment very similar to Matlab.
 Visualization
• Lec1_ex2.py
 Visualization
• Even rather complicated graphics are easy to generate using Matplotlib.
• Lec1_ex3.py
• TENSORFLOW!
 What’s TensorFlow?
• Open source software library for numerical
computation using data flow graphs
• Originally developed by Google Brain Team to conduct
machine learning and deep neural networks research
• General enough to be applicable in a wide variety of
other domains as well
• TensorFlow provides an extensive suite of functions
and classes that allow users to build various models
from scratch.
 Why TensorFlow?
• Python API
• Portability: deploy computation to one or more CPUs or
GPUs in a desktop, server, or mobile device with a single API
• Flexibility: from Raspberry Pi, Android, Windows, iOS,
Linux to server farms
• Visualization (TensorBoard is da bomb)
• Checkpoints (for managing experiments)
• Large community (> 10,000 commits and > 3000 TF-related
repos in 1 year)
• Awesome projects already using TensorFlow
• Get started
import tensorflow as tf
• Simplified TensorFlow?
• TF Learn (tf.contrib.learn): simplified interface that helps users
transition from the the world of one-liner such as scikit-learn
• TF Slim (tf.contrib.slim): lightweight library for defining, training
and evaluating complex models in TensorFlow.
• TensorFlow provides an extensive suite of functions and classes
that allow users to define models from scratch.
• High level API: Keras, TFLearn, Pretty Tensor
 Data Flow Graphs
• TensorFlow separates definition of computations from
their execution

• Phase 1: assemble a graph

• Phase 2: use a session to execute operations in the
graph.
 What’s a tensor
• An n-dimensional array
• 0-d tensor: scalar (number)
• 1-d tensor: vector
• 2-d tensor: matrix
• and so on
 Data Flow Graphs
import tensorflow as tf
a=tf.add(3,5)

TF automatically names the nodes when you don’t

explicitly name them (x = 3, y = 5)

Nodes: operators, variables, and constants

Edges: tensors

Tensors are data.

Data Flow -> Tensor Flow
 Data Flow Graphs
• import tensorflow as tf
a=tf.add(3,5)
print a
>> Tensor("Add:0", shape=(), dtype=int32)
(Not 8)

How to get the value

of a?
 How to get the value of a?
• Create a session, assign it to variable sess so we can
call it later
• Within the session, evaluate the graph to fetch the
value of a
import tensorflow as tf
a = tf.add(3, 5)
sess = tf.Session()
print sess.run(a)
sess.close()
The session will look at the graph,
trying to think: hmm, how can I get
the value of a, then it computes all
the nodes that leads to a.
 More graphs

x=2
y=3
op1 = tf.add(x, y)
op2 = tf. multiply(x, y)
op3 = tf.pow(op2, op1)
with tf.Session() as sess:
op3 = sess.run(op3)
 Placeholders (inputs)

Placeholders are nodes whose value is fed

in at execution time
(inputs, labels, …)

node1 = tf.placeholder(tf.float32)
node2 = tf.placeholder(tf.float32)
node3 = tf.add(node1,node2)
tf.Session().run(node3, {node1 : 3,
node2 : 4})
You need to specify all placeholders on which the “subgraph you
are using for your computation” depends
 Variables (mutable state)
Variables are stateful nodes which output their current
value. State is retained across multiple executions of a graph

(mostly parameters)

W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W * x + b #Operator Overloading!
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
sess.run(linear_model)
Neural network (Programming
model)

W b
x

a tensorflow graph
 Neural network (Programming
model)

Mathematical operations:
MatMul: Multiply two matrices
Add: Add elementwise
ReLU: Activate with elementwise
rectified linear function

0, x <= 0
ReLu(x) =
x, x > 0
 Neural network (Programming
model)

import tensorflow as tf

b = tf.Variable(tf.zeros((100,)))
W = tf.Variable(tf.random_uniform((784, 100), -1, 1))

x = tf.placeholder(tf.float32, (1, 784))

h = tf.nn.relu(tf.matmul(x, W) + b)
• So far:
• Built a graph using variables and placeholders
• Deploy the graph onto a session, i.e., execution
environment

• Next: train model

• Define loss function
• Compute gradients
 Defining loss
• Use placeholder for labels
• Build loss node using labels and prediction

prediction = tf.nn.softmax(...) #Output of neural network

label = tf.placeholder(tf.float32, [100, 10])

cross_entropy = -tf.reduce_sum(label * tf.log(prediction), axis=1)

 Gradient computation:
Backpropagation
train_step =
tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

• tf.train.GradientDescentOptimizer is an Optimizer object

• tf.train.GradientDescentOptimizer(lr).minimize(cross_entropy) adds
optimization operation to computation graph

• TensorFlow graph nodes have attached gradient

operations Gradient with respect to parameters
computed with backpropagation …
automatically
 Classification of handwritten
digits
• Lec1_ex5.py
• https://github.com/easy-tensorflow/easy-tensorflow/blob/master/3_
Neural_Network/Tutorials/1_Neural_Network.ipynb