HCIA-AI V1.0 Lab Guide
HCIA-AI V1.0 Lab Guide
HCIA-AI V1.0
HUAWEI Cloud Modelarts Guide
Issue: 1.0
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and
the customer. All or part of the products, services and features described in this document may not be
within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
ii
Introduction to Huawei Certification System
Based on cutting-edge technologies and professional training systems, Huawei certification meets the
diverse AI technology demands of various clients. Huawei is committed to providing practical and
professional technical certification for our clients.
HCIA-AI V1.0 certification is intended to popularize AI and help understand deep learning and Huawei
Cloud EI, and learn the basic capabilities of programming based on the TensorFlow framework, as a
motive to promote talent training in the AI industry.
Content of HCIA-AI V1.0 includes but is not limited to: AI overview, Python programming and
experiments, mathematics basics and experiments, TensorFlow introduction and experiments, deep
learning pre-knowledge, deep learning overview, Huawei cloud EI overview, and application experiments
for image recognition, voice recognition and man-machine dialogue.
HCIA-AI certification will prove that you systematically understand and grasp Python programming,
essential mathematics knowledge in AI, basic programming methods of machine learning and deep
learning platform TensorFlow, pre-knowledge and overview of deep learning, overview of Huawei cloud
EI, basic programming for image recognition, voice recognition, and man-machine dialogue. With this
certification, you have required knowledge and techniques for AI pre-sales basic support, AI after-sales
technical support, AI products sales, AI project management, and are qualified for positions such as
natural language processing (NLP) engineers, image processing engineers, voice processing engineers
and machine learning algorithm engineers.
Enterprises with HCIA-AI-certified engineers have the basic understanding of AI technology, framework,
and programming, and capable of leveraging AI, machine learning, and deep learning technologies, as
well as the open-source TensorFlow framework to design and develop AI products and solutions like
machine learning, image recognition, voice recognition, and man-machine dialogue.
Huawei certification will help you open the industry window and the door to changes, standing in the
forefront of the AI world!
iii
2020-04-15 Copyright © Huawei Technologies Co., Ltd. iv
iv
Content
Introduction to Huawei Certification System.................................................................. iii
1 HUAWEI Cloud Modelarts Guide ................................................................................... 6
1.1 Access HUAWEI Cloud .......................................................................................................................... 6
1.2 Log in HUAWEI Cloud ............................................................................................................................ 6
1.3 Enter the “Modelarts” ............................................................................................................................. 7
v
HCIA-AI Python Programming Basics Experiment Guide
Please use Google browser and visit the Huawei cloud:https://www.huaweicloud.com/en-us/, and then
choose English version of China station.
Choose Beijing4.
We have created the notebooks, so you don’t need to add access key. Click “Cancel” and find the
notebook with your account name.
Wait a minute until your notebook is running, and then open your notebook.
When you enter the Jupyter interface, you have entered the experimental environment.
In this course, we use the TensorFlow-1.8 version, so you can create the TensorFlow-1.8 and complete
one experiment.
HCIA-AI V1.0
Python Programming Basics
Experiment Guide
Issue: 1.0
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and
the customer. All or part of the products, services and features described in this document may not be
within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
ii
HCIA-AI V1.0 Python Programming Basics Experiment Guide
Based on cutting-edge technologies and professional training systems, Huawei certification meets the
diverse AI technology demands of various clients. Huawei is committed to providing practical and
professional technical certification for our clients.
HCIA-AI V1.0 certification is intended to popularize AI and help understand deep learning and Huawei
Cloud EI, and learn the basic capabilities of programming based on the TensorFlow framework, as a
motive to promote talent training in the AI industry.
Content of HCIA-AI V1.0 includes but is not limited to: AI overview, Python programming and
experiments, mathematics basics and experiments, TensorFlow introduction and experiments, deep
learning pre-knowledge, deep learning overview, Huawei cloud EI overview, and application experiments
for image recognition, voice recognition and man-machine dialogue.
HCIA-AI certification will prove that you systematically understand and grasp Python programming,
essential mathematics knowledge in AI, basic programming methods of machine learning and deep
learning platform TensorFlow, pre-knowledge and overview of deep learning, overview of Huawei cloud
EI, basic programming for image recognition, voice recognition, and man-machine dialogue. With this
certification, you have required knowledge and techniques for AI pre-sales basic support, AI after-sales
technical support, AI products sales, AI project management, and are qualified for positions such as
natural language processing (NLP) engineers, image processing engineers, voice processing engineers
and machine learning algorithm engineers.
Enterprises with HCIA-AI-certified engineers have the basic understanding of AI technology, framework,
and programming, and capable of leveraging AI, machine learning, and deep learning technologies, as
well as the open-source TensorFlow framework to design and develop AI products and solutions like
machine learning, image recognition, voice recognition, and man-machine dialogue.
Huawei certification will help you open the industry window and the door to changes, standing in the
forefront of the AI world!
Preface
Brief Introduction
This document is an HCIA-AI certification training course, intended to trainees who
are preparing for HCIA-AI tests or readers who want to know about AI basics. After
understanding this document, you can perform simple Python basic programming,
laying a solid foundation for later AI development.
Contents
This experimental guide covers Python programming basics, and is intended to help
trainees and readers easily develop capabilities of developing AI.
Experimental Environment
This experimental environment is compiled on Python 3.6.
Content
Introduction to Huawei Certification System.................................................................. iii
1 Python Programming Basics ......................................................................................... 7
1.1 Experiment Overview ............................................................................................................................. 7
1.1.1 Experiment Introduction....................................................................................................................... 7
1.1.2 Objective............................................................................................................................................. 7
1.2 Experiment Code ................................................................................................................................... 7
1.2.1 Hello World ......................................................................................................................................... 7
1.2.2 Data Type: Number ............................................................................................................................. 7
1.2.3 Data Type: Character String ................................................................................................................ 8
1.2.4 Data Type: List .................................................................................................................................... 9
1.2.5 Data Type: Tuple ................................................................................................................................10
1.2.6 Data Type: Dictionary .........................................................................................................................10
1.2.7 Data Type: Set ...................................................................................................................................10
1.2.8 Deep Copy and Shallow Copy ............................................................................................................ 11
1.2.9 if Statement........................................................................................................................................ 11
1.2.10 Loop Statement ................................................................................................................................12
1.2.11 Customizing a Function ....................................................................................................................13
1.2.12 Object-oriented Programming...........................................................................................................14
1.2.13 Standard Library Usage....................................................................................................................16
1.2.14 I/O Operations..................................................................................................................................20
1.2.15 Multiple Tasks...................................................................................................................................21
1.2.16 Regular Expressions ........................................................................................................................23
1.1.2 Objective
The following experiments help you master the Python programming language and
lay a foundation for the AI experiment in the future.
----End
Input:
animals = ['cat', 'dog', 'monkey']
# list.append(obj): Add a new object to the end of a list.
animals.append ('fish') # Append an element.
print(animals)# Output: ['cat', 'dog', 'monkey', ‘fish’]
# list.remove(obj): Remove the first match for a value in the list.
animals.remove ('fish') # Delete element fish.
print(animals)# Output: ['cat', 'dog', 'monkey']
# list.insert(index, obj): Insert a specified object to a specified position in the list. The index indicates
the position.
animals.insert (1, 'fish') # Insert element fish at subscript 1.
print(animals)# Output: ['cat', ‘fish’, 'dog', 'monkey']
# list.pop([index=-1]): Remove the element (the last element by default) corresponding to the
subscript in the list. The index indicates the subscript.
animals.pop(1)# Delete the element whose subscript is 1.
print(animals)# Output: ['cat', 'dog', 'monkey']
# Traverse and obtain the elements and indexes.
# enumerate(sequence): Return an index sequence consisting of a data object that can be traversed
and list the data and subscripts. This function is usually used in the for loop.
for i in enumerate(animals):
print(i)# Index consisting of the element subscript and element
Output:
(0, cat)
(1, dog)
(2, monkey)
# List derivation.
squares = [x*2 for x in animals]# Generate a list of elements that comply with rules in batches.
print(squares) #['catcat ', 'dogdog ', 'monkeymonkey ']
list1 = [12,45,32,55]
# list.sort(cmp=None, key=None, reverse=False): The cmp parameter is an optional parameter. If
this parameter is specified, the method specified by this parameter is used for sorting. The key
parameter is an element used for comparison, reverse indicates the sorting rule, and False
indicates the ascending order.
list1.sort()# Sort the list.
print(list1)# Output: [12,32,45,55]
# list.reverse(): Elements in the reverse list.
list1.reverse()# Reverse the list.
set.
# set.add(obj): Add an element to a set. If the element to be added already exists in the set, no
operation is performed.
sample_set.add('Data') # Add element Data to the set.
print(sample_set)# Output: {'Prince', 'Techs', 'Data'}
print(len(sample_set)) # Output: 3
# set.remove(obj): Remove a specified element from a set.
sample_set.remove('Data') # Delete element Data.
print(sample_set) # {'Prince', 'Techs'}
list2 = [1,3,1,5,3]
print(list(set(list2)))# The output is [1,3,5]. The uniqueness of the set elements is used to deduplicate
the list.
sample_set = frozenset(sample_set)# Unchangeable set.
Output:
Dict1:{‘name’:’lee’, ‘age’:89, ‘num’:[1,6,8]}
Dict_copy :{'name':'lee', 'age':89, 'num':[1,6,8]}# The shallow copy data is modified.
Dict_dcopy :{'name':'lee', 'age':89, 'num':[1,2,8]}# The deep copy data is not modified.
1.2.9 if Statement
You can use the if statement in Python to determine the level of a score input by a
user.
# Determine the entered score.
# input(): Receive input data.
score = input("Please enter your score.") # The input function receives input, which is a character
string.
# try:… except Exception:… is a Python statement used to capture exceptions. If an error occurs in
the statement in the try statement, the except statement will be executed.
try:
score = float(score)# Convert the score to a number.
if 100>=score>=90: # Check whether the entered value
is greater than the score of a level.
print("Excellent") # Generate the level when conditions are met.
Output:
1*1= 1
2*1= 2 2*2= 4
3*1= 3 3*2= 6 3*3= 9
4*1= 4 4*2= 8 4*3=12 4*4=16
5*1= 5 5*2=10 5*3=15 5*4=20 5*5=25
6*1= 6 6*2=12 6*3=18 6*4=24 6*5=30 6*6=36
7*1= 7 7*2=14 7*3=21 7*4=28 7*5=35 7*6=42 7*7=49
8*1= 8 8*2=16 8*3=24 8*4=32 8*5=40 8*6=48 8*7=56 8*8=64
9*1= 9 9*2=18 9*3=27 9*4=36 9*5=45 9*6=54 9*7=63 9*8=72 9*9=81
Output:
1
2
Exit the current loop.
4
Exit the current big loop.
----End
Output:
[0, 1, 1, 2, 3]
Output:
hello, world!
Greetings, world!
Greetings, universe!
hello, Gumby!
----End
Instantiate a class.
dog = Dog ("Husky",2)
dog.sit()
dog.roll_over()
Output:
Husky is now sitting
Husky rolled over!
Output:
Name : Zara ,Salary: 2000
Name : Manni ,Salary: 5000
Total Employee 2
Output:
Method constructed by invoking a child class
The method of a child class is invoked.
The method of a parent class is invoked.
Parent attribute: 200
class JustCounter:
__secretCount = 0 # Private variable.
publicCount = 0 # Public variable.
def count(self):
self.__secretCount += 1
self.publicCount += 1
print(self.__secretCount)
counter = JustCounter()
counter.count()
counter.count()
print(counter.publicCount)
print(counter.__secretCount) # An error is reported, indicating that the instance cannot access
private variables.
Output:
1
---------------------------------------------------------------------------
<ipython-input-4-ee25477dd398> in <module>
10 counter.count()
11 print(counter.publicCount)
----End
Output:
0
1
2
3
4
5
An exception has occurred, use %tb to see the full traceback.
Output:
['D:\\python3.6\\python36.zip',
'D:\\python3.6\\DLLs',
'D:\\python3.6\\lib',
'D:\\python3.6',
'',
'D:\\python3.6\\lib\\site-packages',
'D:\\python3.6\\lib\\site-packages\\IPython\\extensions',
'C:\\Users\\xxx\\.ipython']
Output:
'win32'
sys.argv: Transfer parameters from outside of the program to the program. The
parameters are transferred in list format. The first parameter is the current file name.
Create the .py file test.py (in the current folder or on the desktop) and write the
following code:
print(sys.argv[1])
Switch to the file path in the command line and run the program.
python test.py hello
Output:
hello
Step 2 os
import os
# os.getpid() Obtain the current process ID.
Output:
ID of the current process: 7580
D:\TensorFlow\IA1.0\.ipynb_checkpoints\Untitled-checkpoint.ipynb
D:\TensorFlow\IA1.0\test.py
D:\TensorFlow\IA1.0\Untitled.ipynb
D:\TensorFlow\IA1.0\.ipynb_checkpoints
# os.path.exists(path): If the file exists, True is returned; if the file does not exist, False is returned.
print("Whether the text.txt file exists: ",os.path.exists("text.txt"))
# os.path.getsize(path): Return the file size. If the file does not exist, an error is returned.
print("Size of the text.txt file: ",os.path.getsize("text.txt"))
Output:
Absolute path of text.txt is D:\python project\text.txt
Whether text.txt exists: True
Size of the text.txt file: 0
Whether text.txt is a file: True
Whether text.txt is a folder: False
Step 3 time
import time
# time.time(): Obtain the current timestamp.
time_now = time.time()
print("Timestamp:",time_now)
#time.strftime(format[, t]): Receive the time tuple and return the local time expressed in a readable
string, in the format specified by the format parameter.
print(time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()))
Output:
Timestamp: 1555950340.4777014
Local time: time.struct_time(tm_year=2019, tm_mon=4, tm_mday=23, tm_hour=0, tm_min=25,
tm_sec=40, tm_wday=1, tm_yday=113, tm_isdst=0)
Local time: Tue Apr 23 00:25:40 2019
2019-04-23 00:25:40
----End
Output:
Output:
python
file operation
print(f.read())
Output:
python file operation
----End
def work2():
for i in range(3):
print("work2 is being executed...%d"%i)
sleep(1)
if __name__ == '__main__':
print('---Start---:%s'%ctime())
t1 = threading.Thread(target=work1)# Thread 1
t2 = threading.Thread(target=work2)# Thread 2
# Start the thread.
t1.start()
t2.start()
sleep(5)
print('---End---:%s' %ctime())
Output:
---Start---:Mon Apr 15 10:55:16 2019
work1 is being executed...0
Thread synchronization:
import threading
import time
g_num = 0
def test1(num):
global g_num # Use the global variable g_num.
for i in range(num):
mutex.acquire() # Lock
g_num += 1
mutex.release() # Unlock
print("---test1---g_num=%d"%g_num)
def test2(num):
global g_num
for i in range(num):
mutex.acquire() # Lock
g_num += 1
mutex.release() # Unlock
print("---test2---g_num=%d"%g_num)
p2 = threading.Thread(target=test2, args=(1000000,))
p2.start()
print("The final result after operations are performed on the same global variable by two
threads:%s" % g_num)
Output:
---test2---g_num=1971982---test1---g_num=2000000
After two threads operate the same global variable, the final result is as follows: 2000000
Example:
import re
print(re.match('www', 'www.huawei.com').span()) # Match at the start position.
print(re.match('com', 'www.huawei.com')) # Not match at the start position.
Output:
(0, 3)
None
Example:
import re
line = "Cats are smarter than dogs"
searchObj = re.search( r'(.*) are (.*?) .*', line, re.M|re.I)
if searchObj:
print("searchObj.group() : ", searchObj.group())
print("searchObj.group(1) : ", searchObj.group(1))
print("searchObj.group(2) : ", searchObj.group(2))
else:
print("Nothing found!!" )
Output:
searchObj.group() : Cats are smarter than dogs
searchObj.group(1) : Cats
searchObj.group(2) : smarter
Example:
import re
phone = "2019-0101-000 # This is a phone number."
# Delete the Python comment in the character string.
num = re.sub(r'#.*$', "", phone)
print("The phone number is: ", num)
# Delete the hyphens from the phone number.
num = re.sub(r'\D', "", phone)
print("The phone number is: ", num)
Output:
The phone number is: 2019-0101-000
The phone number is: 20190101000
Example:
import re
pattern = re.compile(r'\d+') # At least one digit is matched.
n = pattern.match ('one12twothree34four') # The information is not found from the header.
print(n)
Output:
None
<_sre.SRE_Match object; span=(3, 5), match='12'>
12
Step 5 re.split()
The split method splits a character string based on the matched substring and returns
a list. The usage of the method is as follows:
re.split(pattern, string[, maxsplit=0, flags=0])
Example:
import re
s = re.split('\W+', 'www.huawei.com')
print(s)
Output:
['www', 'huawei', 'com']
----End
HCIA-AI V1.0
AI Mathematics
Experiment Guide
Issue: 1.0
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and
the customer. All or part of the products, services and features described in this document may not be
within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees
or representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
Website: https://www.huawei.com/
Email: [email protected]
Contents
1 Experiment Overview ..................................................................................................... 7
1.1 Experiment Introduction ......................................................................................................................... 7
1.2 Description............................................................................................................................................. 7
1.3 Skill Requirements ................................................................................................................................. 7
1.4 Experiment Environment Overview......................................................................................................... 8
1 Experiment Overview
1.2 Description
This document describes five experiments:
Experiment 1: basic mathematics experiment
Experiment 2: implementation of operators related to linear algebra
Experiment 3: implementation of statistical distribution of the probability theory
Experiment 4: implementation of the least squares method
Experiment 5: implementation of gradient descent
2.1 Introduction
2.1.1 Content
The basic mathematics knowledge is widely used in data mining, especially in
algorithm design and numerical processing. The main purpose of this section is to
implement some basic mathematical algorithms based on the Python language and
basic mathematics modules, laying a foundation for learning data mining.
2.1.2 Frameworks
This document mainly uses the math library, NumPy library, and SciPy library. The
math library is a standard library of Python and provides some common mathematical
functions. The NumPy library is an extended library of Python, used for numerical
calculation. It can solve problems about linear algebra, random number generation,
and Fourier transform. The SciPy library is used to handle problems related to
statistics, optimization, interpolation, and integration.
2.2 Implementation
Import libraries:
import math
import numpy as np
Output:
5
Input:
math.ceil(4.99)
Output:
5
Output:
4
Input:
math.floor(4.999)
Output:
4
Output:
45.0
Input:
math.degrees(math.pi)
Output:
180.0
Output:
2.718281828459045
Output:
0.003
Output:
6
Output:
10
Output:
2.0
math.log(10)
Output:
2.302585092994046
Output:
10.0
2.2.11 pi Implementation
pi is a numerical constant, indicating the circular constant.
Input:
math.pi
Output:
3.141592653589793
Output:
81.0
3.1 Introduction
3.1.1 Linear Algebra
Linear algebra is a discipline widely used in various engineering fields. The concepts
and conclusions of linear algebra can greatly simplify the derivations and expressions
of data mining formulas. Linear algebra can simplify complex problems so that we
can perform efficient mathematical operations.
Linear algebra is a mathematical tool. It not only provides the technology for array
operations, but also provides data structures such as vectors and matrices to store
numbers and rules for addition, subtraction, multiplication, and division.
be transformed from 1x100 to 10x10 through the reshape operation. The following is
an example:
Input:
Generate a vector that contains integers from 0 to 11.
x = np.arange(12)
print(x)
Output:
[ 0 1 2 3 4 5 6 7 8 9 10 11]
Output:
(12,)
Convert x into a two-dimensional matrix, where the first dimension of the matrix is 1.
x = x.reshape(1,12)
print(x)
Output:
[[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]]
Output:
(1, 12)
Output:
[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]]
Output:
[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]]
Input:
A.T
Output:
array([[ 0, 4, 8],
[ 1, 5, 9],
[ 2, 6, 10],
[ 3, 7, 11]])
Output:
[[0 1]
[2 3]
[4 5]]
Input:
print(B)
Output:
[[0, 1, 2],
[3, 4, 5]]
Matrix multiplication:
np.matmul(A,B)
Output:
array([[ 3, 4, 5],
[ 9, 14, 19],
[15, 24, 33]])
Matrix multiplication:
print(A*A)
Output:
array([[ 0, 1],
[ 4, 9],
[16, 25]])
Matrix addition:
print(A + A)
Output:
array([[ 0, 2],
[ 4, 6],
[ 8, 10]])
Output:
array([[0, 1],
[2, 3]])
Inverse matrix:
np.linalg.inv(A)
Output:
array([[-1.5, 0.5],
[ 1. , 0. ]])
#The plt.subplots() function returns a figure instance named fig and an AxesSubplot
instance named ax. The fig parameter indicates the entire figure, and ax indicates
the coordinate axis. Plotting:
fig, ax = plt.subplots()
#Draw a grid:
ax.grid(alpha=0.4)
Illustration: The vectors with the blue arrow are eigenvectors, and the space formed
by the two red lines is the eigenspace.
3.2.7 Determinant
This section describes how to obtain the determinant of a matrix.
Input:
E = [[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]
print(np.linalg.det(E))
Output:
0.0
users. So the question is, what method can be used to automatically classify the
articles? The following describes a simple approach. Generally, the title of an article
consists of a few core concept words, which represent the subject of the article.
Therefore, we can determine whether two articles belong to the same type based on
the similarity between titles. The similarity between titles depends on the similarity
between words. Then how do we check the similarity between words in the titles? We
can use singular value decomposition to solve this problem. Assume that there are
eight titles. The keywords of the titles are as follows:
title_1 = ["dad","dad","stock"]
title_2 = ["books","books","value","estate"]
title_3 = ["books","decomposition"]
title_4 = ["stock"]
title_5 = ["dad"]
title_6 = ["value","singular","decomposition"]
title_7 = ["dad","singular"]
title_8 = ["singular","estate","decomposition"]
Input:
#Import modules:
import numpy as np
import matplotlib.pyplot as plt
#Enter keywords:
words = ["books","dad","stock","value","singular","estate","decomposition"]
#Assume that there are eight titles and seven keywords. Record the number of times
each keyword appears in each title to obtain matrix X. In matrix X, each row indicates
a title, each column indicates a keyword, and each element in the matrix indicates
the number of times a keyword appears in a title.
X=np.array([[0,2,1,0,0,0,0],[2,0,0,1,0,1,0],[1,0,0,0,0,0,1],[0,0,1,0,0,0,0],[0,1,0,0,0,0,0],[0,0,0,1,1,0,1],
[0,1,0,0,1,0,0],[0,0,0,0,1,1,1]])
Output:
U= [[-1.87135757e-01 -7.93624528e-01 2.45011855e-01 -2.05404352e-01
-3.88578059e-16 5.75779114e-16 -2.57394431e-01 -4.08248290e-01]
[-6.92896814e-01 2.88368077e-01 5.67788037e-01 2.22142537e-01
2.54000254e-01 -6.37019839e-16 -2.21623012e-02 2.05865892e-17]
[-3.53233681e-01 1.22606651e-01 3.49203461e-02 -4.51735990e-01
-7.62000762e-01 1.27403968e-15 2.72513448e-01 3.80488702e-17]
Output:
the results in descending order, where each singular value corresponds to a left
singular vector and a right singular vector:
s= [2.85653844 2.63792139 2.06449303 1.14829917 1. 1.
0.54848559]
s.shape (7,)
Output:
Vh [[-6.08788345e-01 -2.29949618e-01 -7.46612474e-02 -3.80854846e-01
Vh.shape (7, 7)
information, but the singular values usually attenuate quickly. Therefore, the first K
singular values and corresponding singular vectors include most information in the
image. As a result, an image formed by the first K singular values and their singular
vectors can achieve basically the same definition as the original image, but the data
amount is greatly reduced. In this way, image data compression can be implemented.
Input:
import numpy as np
from pylab import *
import matplotlib.pyplot as plt
Output:
Figure 3-3
(512,)
Figure 3-4
import numpy as np
from scipy.linalg import solve
a = np.array([[10, 2, 5], [4, 4, 2], [2, 2, 2]])
b = np.array([10,8,5])
x = solve(a, b)
print(x)
Output of print(x):
array([0.25 1.25 1. ])
4.1 Introduction
4.1.1 Probability Theory
Probability theory is a branch of mathematics concerned with the quantitative
regularity of random phenomena. A random phenomenon is a situation in which we
know what outcomes could happen, but we do not know which particular outcome did
or will happen, while a decisive phenomenon is a situation in which a result inevitably
occurs under certain conditions.
The probability theory is a mathematical tool used to describe uncertainties. A large
number of data mining algorithms build models based on the sample probabilistic
information or through inference.
Output:
4.5
Input:
np.mean(ll,0) # Calculate the mean value by column. The value 0 indicates the column vector.
Output:
array([2., 3., 4., 5., 6., 7.])
Input:
np.mean(ll,1) # Calculate the mean value by row. The value 1 indicates the row vector.
Output:
array([3.5, 5.5])
Output:
3.6875
Input:
np.var(ll,1) # The value of the second parameter is 1, indicating that the variance is calculated
by row.
Output:
[2.91666667 2.91666667]
Output:
1.9790570145063195
Output:
4.916666666666666
#Function-based implementation:
np.corrcoef(vc,vb)
Output:
array([[1. , 0.99998623],
[0.99998623, 1. ]])
# The n and p parameters indicate the success times and probability in the binomial formula,
respectively, and size indicates the number of sampling times.
binom_sim = binom.rvs(n=10, p=0.3, size=10000)
print('Data:',binom_sim)
print('Mean: %g' % np.mean(binom_sim))
print('SD: %g' % np.std(binom_sim, ddof=1))
# Generate a histogram. The x parameter indicates the data distribution of each bin, corresponding
to the x axis. The bins parameter indicates the number of bars in total. The normed parameter
indicates whether to perform normalization. By default, the sum of the percentages of all bars is 1.
Output:
Data: [2 4 3 ... 3 4 1]# 10,000 numbers that comply with the binomial distribution.
Mean: 2.9821
SD: 1.43478
Figure 4-1
# Generate 10,000 numbers that comply with the Poisson distribution where the value of lambda is
2.
X= np.random.poisson(lam=2, size=10000)
# Generate grids.
plt.grid()
plt.show()
Figure 4-2
mu = 0
sigma = 1
# Distribution sampling points.
x = np.arange(-5, 5, 0.1)
# Generate normal distribution that complies with mu and sigma.
y = norm.pdf(x, mu, sigma)
plt.plot(x, y)
plt.xlabel('x')
plt.ylabel('density')
plt.show()
Figure 4-3
5 Optimization Experiment
#Define a residual function. The value of the function is the difference between the
polynomial fitting result and the actual value.
def residuals_func(p, y, x):
ret = fit_func(p, x) - y
return ret
pl.show()
Output:
Fitting Parameters: [-1.22007936e+03 5.79215138e+03 -1.10709926e+04 1.08840736e+04
-5.81549888e+03 1.65346694e+03 -2.42724147e+02 1.96199338e+01
-2.14013567e-02]
determined, we can estimate the price of any house according to the formula. In
order to obtain 𝜃0 , 𝜃1 , and 𝜃2 , we have found a copy of the city's house prices and
data about the house areas, floor area ratios, and afforested areas. How do we use
the data to help us obtain 𝜃0 , 𝜃1 , and 𝜃2 ? To evaluate the quality of 𝜃0 , 𝜃1 , and 𝜃2 ,
we can compare the value of 𝑦𝑝𝑟𝑒 (obtained through the formula 𝜃0 *s + 𝜃1 *r + 𝜃2 *d
based on most of the actual data) with that of the real house price y. If the values are
similar, the 𝜃0 , 𝜃1 , and 𝜃2 are high-quality data. Similar to the case in 5.1.2 Case
Introduction where we have obtained a large amount of known data, a good
polynomial should be optimal in the overall data rather than just at some data points.
1 1 2
Therefore, ∑𝑚 𝑖=1 (𝑦𝑖 − 𝑦(𝑝𝑟𝑒,𝑖) ) is used to show the overall quality, and m
𝑚 2
indicates the number of known data points. Then the problem is simplified into an
optimization problem, that is, to obtain 𝜃0 , 𝜃1 , and 𝜃2 that can minimize the value of
1 𝑚 1 2
∑ (𝑦𝑖 − 𝑦(𝑝𝑟𝑒,𝑖) ) . The following is an example. We have five copies of the
𝑚 𝑖=1 2
actual data.
The following describes how to use the gradient descent algorithm to obtain the
1 1 2
optimal 𝜃0 , 𝜃1 , and 𝜃2 to minimize the value of 𝑚 ∑𝑚
𝑖=1 2 (𝑦𝑖 − 𝑦(𝑝𝑟𝑒,𝑖) ) .
theta0 = 0
theta1 = 0
theta2 = 0
while True:
cnt += 1
print(' theta0 : %f, theta1 : %f, theta2 : %f, error1 : %f' % (theta0, theta1, theta2, error1) )
print('Done: theta0 : %f, theta1 : %f, theta2 : %f' % (theta0, theta1, theta2) )
print ('Number of iterations: %d' % cnt )
HCIA-AI
TensorFlow Programming Basics
Experiment Guide
Version: 1.0
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and
the customer. All or part of the products, services and features described in this document may not be
within the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
Email: [email protected]
Contents
2 Session ............................................................................................................................ 3
2.1 Introduction to the Experiment ................................................................................................................ 3
2.1.1 About the Experiment .......................................................................................................................... 3
2.1.2 Objectives of the Experiment ............................................................................................................... 3
2.1.3 Experiment Content............................................................................................................................. 3
2.2 Experimental Process ............................................................................................................................ 3
2.2.1 Setting Encoding Declarations ............................................................................................................. 3
2.2.2 Importing a Module ............................................................................................................................. 3
2.2.3 Defining Constants .............................................................................................................................. 3
2.2.4 Creating a Session .............................................................................................................................. 3
2.2.5 Experimental Results .......................................................................................................................... 4
2.3 Instance Description............................................................................................................................... 4
5 Visualization of TensorBoard....................................................................................... 10
5.1 Introduction to the Experiment ...............................................................................................................10
5.1.1 About the Experiment .........................................................................................................................10
5.1.2 Objectives of the Experiment ..............................................................................................................10
5.1.3 Experiment Content............................................................................................................................10
5.2 Experimental Process ...........................................................................................................................10
5.2.1 Setting Encoding Declarations ............................................................................................................10
5.2.2 Importing a Module ............................................................................................................................10
5.2.3 Generating Analog Data ..................................................................................................................... 11
5.2.4 Resetting the Computation Graph....................................................................................................... 11
5.2.5 Creating a Model ................................................................................................................................ 11
5.2.6 Creating a Forward Structure.............................................................................................................. 11
5.2.7 Reverse Optimization ......................................................................................................................... 11
5.2.8 Initializing Variables ............................................................................................................................12
5.2.9 Starting a Session ..............................................................................................................................12
5.2.10 Writing Data to the Model ....................................................................................... 错误!未定义书签。
5.2.11 Graphical Display ................................................................................................... 错误!未定义书签。
5.2.12 Experimental Results........................................................................................................................13
5.3 Instance Description.................................................................................................... 错误!未定义书签。
1 Hello, TensorFlow!
2 Session
3 Matrix Multiplication
#tf.random_normal(shape,
# mean=0.0,
# stddev=1.0,
# dtype=dtypes.float32,
# seed=None,
# name=None)
#A random normal distribution is generated.
#shape stands for the matrix dimension. For example,
#tf.random_normal([2,3],mean=1.0, stddev=1.0) is a matrix of two rows and three
columns.
#mean stands for an average. Its default value is 0.0. stddev stands for standard
deviation. Its default value is 1.0.
#seed stands for random seed. Its default value is None.
w1 = tf.Variable(tf.random_normal([2,3],mean=1.0, stddev=1.0))
w2 = tf.Variable(tf.random_normal([3,1],mean=1.0, stddev=1.0))
[[7.3619137]]
4 Definition of Variables
var3 = tf.Variable(12.0 )
var4 = tf.Variable(13.0 )
with tf.variable_scope("test2"):
var6 = tf.get_variable("varname",shape=[2],dtype=tf.float32)
5 Visualization of TensorBoard
tf.reset_default_graph()
#Model parameters
W = tf.Variable(tf.random_normal([1]), name="weight")
b = tf.Variable(tf.zeros([1]), name="bias")
#Set parameters.
training_epochs = 20
display_step = 2
#Generate a summary.
summary_str = sess.run(merged_summary_op,feed_dict={X: x, Y: y});
summary_writer.add_summary(summary_str, epoch);#Write summary
to files.
plotdata["avgloss"] = moving_average(plotdata["loss"])
plt.figure(1)
plt.subplot(211)
plt.plot(plotdata["batchsize"], plotdata["avgloss"], 'b--')
plt.xlabel('Minibatch number')
plt.ylabel('Loss')
plt.title('Minibatch run vs. Training loss')
plt.show()
6 Graphic Operation
g = tf.Graph()
with g.as_default():
c1 = tf.constant(0.0)
print(c1.graph)
print(g)
print(c.graph)
g2 = tf.get_default_graph()
print(g2)
tf.reset_default_graph()
g3 = tf.get_default_graph()
print(g3)
print(tensor1.op.name)
testop = g3.get_operation_by_name("exampleop")
print(testop)
}
attr {
key: "transpose_b"
value {
b: false
}
}
[[7.]]
Tensor("exampleop:0", shape=(1, 1), dtype=float32)
[<tf.Operation 'Const' type=Const>]
Tensor("Const:0", shape=(), dtype=float32)
________________________
7 Linear Regression:
Housing Price Forecast
import numpy as np
import matplotlib.pyplot as plt
...
HCIA-AI
Image Recognition Programming
Experiment Guide
Version: 1.0
1
Copyright © Huawei Technologies Co., Ltd. 2018. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior written
consent of Huawei Technologies Co., Ltd.
and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.
Notice
The purchased products, services and features are stipulated by the contract made between Huawei and
the customer. All or part of the products, services and features described in this document may not be within
the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.
The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.
Brief Introduction
This document is an HCIA-AI certification training course. It is intended for
trainees who are preparing for HCIA-AI tests or readers who want to know about AI
basics. After understanding this document, you will be able to perform basic AI image
recognition programming.
Contents
This document contains one experiment (TensorFlow image recognition) to help
trainees or readers get started and build the basic AI programming capability.
The experiment includes image data preparation, library and module installation, and
finally image recognition realization.
Experimental Guide
Contents
Introduction to Huawei Certification System................................................................... ii
About This Document ....................................................................................................... iii
1 Image Recognition Experiment Overview .................................................................... 1
1.1 Experimental Environment ..................................................................................................................... 1
1.1.1 Experimental Environment .................................................................................................................. 1
1.1.2 Main Framework ................................................................................................................................. 1
1.1.3 Application Framework List .................................................................................................................. 1
1.2 Data Set Introduction ............................................................................................................................. 2
1.2.1 Data Set .............................................................................................................................................. 2
1.2.2 Data Set Obtaining .............................................................................................................................. 2
1.3 CNN....................................................................................................................................................... 2
1.3.1 Structure ............................................................................................................................................. 2
1.3.2 Overview............................................................................................................................................. 3
1.4 Experiment............................................................................................................................................. 3
1.4.1 Content ............................................................................................................................................... 3
1.4.2 Step .................................................................................................................................................... 3
1.5 Summary ............................................................................................................................................... 3
Experimental Guide
os
glob
time
Vim
Git
1.3 CNN
1.3.1 Structure
The CNN, one of the mainstream deep learning and neural network algorithms, is
mainly used for image recognition. The following figure shows the CNN structure.
1.3.2 Overview
The CNN is an in-depth Forward Feedback Artificial Neural Network (FFANN), which
has been successfully applied in the field of image recognition. The network structure
mainly includes the input layer, convolutional layer, pooling layer, full connection
layer, and output layer.
Generally, the basic structure of the CNN includes two layers. The first layer is the
feature extraction layer: The input of each neuron is connected to the local
acceptance field of the previous layer, and the features of the local acceptance field
are extracted. Once a local feature is extracted, the location relationship between it
and other features is determined. The second layer is the feature mapping layer.
Each computing layer of the network consists of multiple feature mappings. Each
feature mapping is a plane and the weights of all neurons on the plane are the same.
The feature mapping structure uses the sigmoid function that has the smallest
influence function kernel as the activation function of the convolutional network, so
that the feature mapping has the rotation invariance.
1.4 Experiment
1.4.1 Content
This experiment focuses on image recognition. Based on TensorFlow depth learning
framework and CNN, it aims to identify flower types using an open-source data set.
1.4.2 Step
The main steps of the experiment are as follows:
1. Importing frameworks
2. Preparing paths
3. Processing images
4. Defining placeholders
5. Defining the network structure
6. Defining the loss function and optimizer.
7. Model training and prediction
8. Model prediction
1.5 Summary
This chapter mainly introduces the environment and data set of the image recognition
experiment, briefly explains the CNN, and addresses the preparations based on the
TensorFlow framework.
----End
----End
The preceding jpg files are the sample pictures in the flower_photos data set.
----End
2.3 Summary
This chapter describes how to obtain the experiment data, download the experiment
data, and view related data sets to facilitate subsequent data operations.
time: module for processing time data. In this experiment, the time module is used to
return the current time stamp and calculate the time required for running each epoch
script.
Input:
# -*- coding:uft-8
from skimage import io,transform
import glob
import os
import tensorflow as tf
import numpy as np
import time
c = 3 # channels
----End
label=label[arr]
----End
Step 3 Define the model operation on the training set and verification set, and print the
results.
Train a total of five epoches. Each epoch needs to run on the training set and
verification set. Print the corresponding loss value and accuracy of two data sets
as well as the duration of each epoch operation.
Input:
for epoch in range(n_epoch):
start_time = time.time()
#training
train_loss, train_acc, n_batch = 0, 0, 0
for x_train_a, y_train_a in minibatches(x_train, y_train, batch_size,
shuffle=True):
_,err,ac=sess.run([train_op,loss,acc], feed_dict={x: x_train_a, y_: y_train_a})
train_loss += err; train_acc += ac; n_batch += 1
if n_batch % display_step == 1:
print(" batch acc: %f batch loss: %f" %
(np.sum(ac),np.sum(err)))
print(" train loss: %f" % (np.sum(train_loss)/ n_batch))
print(" train acc: %f" % (np.sum(train_acc)/ n_batch))
#validation
val_loss, val_acc, n_batch = 0, 0, 0
for x_val_a, y_val_a in minibatches(x_val, y_val, batch_size, shuffle=False):
err, ac = sess.run([loss,acc], feed_dict={x: x_val_a, y_: y_val_a})
val_loss += err; val_acc += ac; n_batch += 1
print(" validation loss: %f" % (np.sum(val_loss)/ n_batch))
print(" validation acc: %f" % (np.sum(val_acc)/ n_batch))
print('-------------------------------------------------------')
----End
sess.close()
model_file=tf.train.latest_checkpoint(model_path)
print(model_file)
path1 = path+path1
path2 = path+path2
path3 = path+path3
path4 = path+path4
path5 = path+path5
----End
3.3 Summary
This chapter describes how to construct a CNN, define the model, set related
parameters, and test sample data based on the TensorFlow.