Module 1

Introduction to Machine Learning

Outline

 What is Machine Learning (ML)? And why ML?

 What is learning?
 What is a well-designed learning problem?
 Learn to play checkers
 What types ML tasks to study?
 Course outline
What is Machine Learning (ML)?

 Machine Learning (ML) is a sub-field of Artificial Intelligence (AI)

 It is concerned with:
 Developing computational theories/algorithms for learning ( the
theoretical aspect of ML)
 Building learning systems (the application aspect of ML)
 Tom Mitchell: ML is concerned with the question of how to construct
computer programs that automatically improve with experience
 Many, many successful applications in solving real-world problems
 Link to some videos showing ML applications (AlphaGo)
What is Machine Learning (ML)?

 Other authors definition of ML:

 Alan Turing: Teaching a “child machine” by “reward and
punishment” to build an “adult machine”
 Sergios Theoredordis, author of the book “Machine Learning:
A Bayesian and Optimization Perspective”: ML is a study of
“learning from data”, “then make predictions”.
 Trevor Hastie, Robert Tibshirani, and Jerome Friedman,
authors of the book “The Elements of Statistical Learning”:
“This book is about learning from data”.
 Typical ML problem - learn to predict a pregnant patient’s
chance to have an emergency C-section:
Data:

Given:
• 9714 patient records, each describing a patient’s pregnancy and birth
• Each patient’s record is described by 215 features
Learn to predict:
• Classes of future patients with high risks of emergency Cesarean section
Learned results:

One of the 18 learned rules:

If: No previous vaginal delivery, and Abnormal 2nd trimester ultrasound,
and Malpresentation at admission:
Then: The probability of emergency C-section is 0.6
Over training data: 26/41 = 0.63
Over test data: 12/20 = 0.6
Other ML examples

 Given: videos of various persons and their

facial expression emotions (happy, sad, etc.)
on each video frame
Learn to classify the facial expression
emotions in videos of a new person
 Given a training database of handwritten digits
Learn to classify future handwritten digits
 Given a collection of customer review
texts/ratings on movies and box office sales
Learn to predict new movie ratings and sales
Why Machine Learning?
 Recent progress in algorithms and theory
 Growing flood of online data
 Computational power and storage capacity is available
 Industry-scale applications
Using Tom Mitchell’s wording: “time is right”
Niches of ML:
 Data mining : using historical data to improve decisions
medical records  medical knowledge (Electronic Health Records Mining)
 Software applications we can't program by hand (we do NOT have a good understanding of the
solution method, or we can NOT anticipate all possible situations the program would be in)
 autonomous driving
 speech recognition
 Self customizing programs
 Recommender system that learns about the customer’s interests and recommends products
Where Is this Headed
Today: traditional ML + Deep learning algorithms
 Applied to structured (tabular) data, texts, images, videos, Web, social media data
 Large-scale commercial applications
 Lots of interests in ML

Opportunity for tomorrow: enormous impact

 Learn across full mixed-media data
 Learn from multiple data sources, IoTs, Smart cities with learning embedded
 Learn by active experimentation
 Learn decisions rather than predictions
 Cumulative, lifelong learning
 Programming languages with learning embedded?
Relevant Disciplines

 Artificial intelligence
 Computer science and engineering
 Statistics and mathematics
 Information theory
 Philosophy
 Psychology and neurobiology
 Control theory
 Linguistics
 Big data
 ……
What is Learning?

 Learn: Dictionary definition:

 Acquire/gain knowledge of or skill in (something) by study,
experience, or instruction.
 Modify behavioral tendency through experience

 Can you write down 15 to 20 words (nouns, adjectives)

closely related to the word “learning”?
 Key point: learning can NOT take place in isolation! A well-
designed learning problem needs several important
components!
Learning: Involves P, T, E, K, A

 Learning: Learning = Improving performance P at task T via

experience E by acquiring knowledge K using self-changing
algorithm A
 T: Task to be performed by the learning agent (e.g., recognize facial
expression emotion)
 P: Performance measure (e.g., prediction accuracy)
 E: Experience of the learning system (e.g., going over a set of
training data)
 K: Knowledge acquired by the learner (e.g., a set of classification
rules, neural network weights)
 A: Algorithms used the learner to achieve learning (e.g., decision-tree
learning algorithm)
Learning to play the game of checkers
 T Play checkers

• Arthur Samuel (1952, 1959,

1967) was the pioneer in
developing a ML system for
game playing (for the checkers
game). The program was
demonstrated on TV in 1956,
creating a strong impression.
• Samuel’s program is a
reinforcement learning system.
Learning to play the game of checkers
 P: % of games won in world tournament
 E: what experience? (play against human player, against self?
…)
 K: what is the knowledge to be learned? And how to represent
it?
 A: What is the algorithm to learn the knowledge?
Type of Training Experience

 Direct or indirect? Tell the learner feedback at each step of

action vs. only tell the learner at the end of the entire game
move sequence
 Teacher or not?
 Use known book moves or not?

 A problem: is training experience representative of

performance goal?
 Choose the target function

 ChooseMove: Board  Move?

 V: Board  ? (value function evaluating the merit of the
board)
 Note that if we know the rule of the game, learning V is
sufficient to play the game well
s

𝑠1 S3
𝑆2
V()
V()
V(
Possible Definition for Target Function V

 If b is a final board state that is won, then V(b) =+100

 if b is a final board state that is lost, then V(b) = -100
 if b is a final board state that is draw, then V(b) = 0
 if b is a not a final state in the game, then V(b) = V(b’)
where b’ is the best final board state that can be achieved
starting from b and playing optimally until the end of the game.

 This gives correct values, but is not operational - we may not

be able to explore the game tree all the way to the end to find
the b’
Representation of target function V

 A set of rules?
 A neural network?
 A polynomial function of board features?
A linear function representation of the target V

 V(b) =

• bp(b): the number of black pieces on board b

• rp(b): the number of red pieces on board b
• bp(b): the number of black kings on board
• rp(b): the number of red kings on board b
• bt(b): the number of black pieces on board b that are being threatened
• rt(b): the number of red pieces on board b that are being threatened
Obtaining training target values

 Idea: Use successor state value (successor(b)) to

approximate V(b) and iterate
 V(b): the true target function
 (b): the current approximation of V
 the estimated training target function
One way to estimate training value:
b
 (b)  (successor(b))
c

 Board “d” is successor of “b” d

Choose learning algorithm (weight update method)

 LMS (Least Mean Square) weight learning algorithm:

Initialize weight vector W = (By random real values
 Do repeatedly:
 Select a training example <b, (b) > at random
 Compute error(b):
 error(b) = (b) - (b)
 For each update weight
  + c * error(b) * (b)

 Here c is some small positive constant, say 0.1, called learning rate
 (b): the ith feature value for board b. Note (b) = 1
The complete design

Experiment
Generator
New problem
^
𝑉

Performanc Generalize
e r

Game trace Training

examples
{<, () >, …}
Critic
Some important issues in ML

 What algorithms can approximate functions well (and when)?

 How does number of training examples influence accuracy?
 How does complexity of hypothesis representation impact it?
 How does noisy data influence accuracy?
 What are the theoretical limits of learnability?
 How can prior knowledge of learner help?
 What clues can we get from biological learning systems?
 How can systems alter their own representations?
Main types of ML tasks
 Supervised learning - training data is of the form
D = {<>, <>, …, <>} : desired output for input
 Unsupervised learning - training data is of the form
D = {<>, <>, …, <>}. NO desired output y is given
 Reinforcement learning
There are some feedback (reinforcement) information sporadically,
maybe after a long sequence of observation-action cycles. The
checkers learning program belongs to this category.
 There are some hybrid types of learning tasks such as semi-
supervised learning, but typically the first 3 types are the most
common ones.
Course Outline: 7 Modules
 M1: Intro to ML and tree-based classifiers
 M2: Linear regression and logistic regression
 M3: Neural networks and deep learning
 M4: Evaluating hypothesis and support vector machines (SVM)
 M5: Bayesian learning
 M6: Unsupervised learning: clustering and dimensionality
reduction
 M7: Reinforcement learning