Scribd
Upload
16 views30 pages

3 - Machine Learning Overview

Uploaded by

ironpro224
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
16 views30 pages

3 - Machine Learning Overview

Uploaded by

ironpro224
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 30

Trường Đại học Công nghiệp Thành phố Hồ Chí Minh

Machine Learning Overview

Giảng viên: Tiến sĩ Bùi Thanh Hùng


Bộ môn Khoa học dữ liệu
Khoa Công nghệ thông tin
Đại học Công nghiệp TP HCM
Email: [email protected]
Website: https://sites.google.com/site/hungthanhbui1980/
Outline & Content
• What is machine learning?
• Learning system model
• Training and testing
• Performance
• Algorithms
• Machine learning structure
• What are we seeking?
• Learning techniques
• Applications
• Conclusion
Machine Learning

https://techswizard.com/3
Machine Learning

https://www.zeolearn.com/
Machine Learning

https://www.zeolearn.com/
5
What is machine learning?
• A branch of artificial intelligence,
concerned with the design and
development of algorithms that allow
computers to evolve behaviors based on
empirical data.

• As intelligence requires knowledge, it is


necessary for the computers to acquire
knowledge.
Machine Learning

A computer program is said to learn from


experience E with respect to some class of tasks
T and performance measure P, if its performance
at tasks in T, as measured by P, improves with
experience E.

Tom Mitchell

7
Machine Learning

playing checkers

8
Machine Learning

recognizing cats

9
Learning system model

Testing

Input Learning
Samples Method

System

Training
Training and testing

Data acquisition Practical usage

Universal set
(unobserved)

Training set Testing set


(observed) (unobserved)
Training and testing
• Training is the process of making the system able to
learn.

• No free lunch rule:


– Training set and testing set come from the same distribution
– Need to make some assumptions or bias
Performance
• There are several factors affecting the
performance:
– Types of training provided
– The form and extent of any initial background
knowledge
– The type of feedback provided
– The learning algorithms used

• Two important factors:


– Modeling
– Optimization
Algorithms
• The success of machine learning system also depends
on the algorithms.

• The algorithms control the search to find and build the


knowledge structures.

• The learning algorithms should extract useful information


from training examples.
Algorithms
Algorithms
• Supervised learning ( )
– Prediction
– Classification (discrete labels), Regression (real values)
• Unsupervised learning ( )
– Clustering
– Probability distribution estimation
– Finding association (in features)
– Dimension reduction
• Semi-supervised learning
• Reinforcement learning
– Decision making (robot, chess machine)
Algorithms

Supervised Unsupervised
learning learning

Semi-supervised 17
Machine learning structure
• Supervised learning
Machine learning structure
• Unsupervised learning
What are we seeking?
• Supervised: Low E-out or maximize probabilistic terms

E-in: for training set


E-out: for testing set

• Unsupervised: Minimum quantization error, Minimum


distance, MAP, MLE(maximum likelihood estimation)
What are we seeking?
Under-fitting VS. Over-fitting (fixed N)

error

(model = hypothesis + loss


functions)
Learning techniques
• Supervised learning categories and techniques
– Linear classifier (numerical functions)
– Parametric (Probabilistic functions)
• Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden
Markov models (HMM), Probabilistic graphical models
– Non-parametric (Instance-based functions)
• K-nearest neighbors, Kernel regression, Kernel density
estimation, Local regression
– Non-metric (Symbolic functions)
• Classification and regression tree (CART), decision tree

– Aggregation
• Bagging (bootstrap + aggregation), Adaboost, Random forest
Learning techniques
• Linear classifier

, where w is an d-dim vector (learned)

• Techniques:
– Perceptron
– Logistic regression
– Support vector machine (SVM)
– Ada-line
– Multi-layer perceptron (MLP)
Learning techniques
Using perceptron learning algorithm(PLA)

Traini Testin
Error
ngrate: Errorg rate:
0.10 0.156
Learning techniques
Using logistic regression

Traini Testin
ngrate:
Error Errorg rate:
0.11 0.145
Learning techniques
• Non-linear case

• Support vector machine (SVM):


– Linear to nonlinear: Feature transform and kernel function
Learning techniques
• Unsupervised learning categories and techniques
– Clustering
• K-means clustering
• Spectral clustering
– Density Estimation
• Gaussian mixture model (GMM)
• Graphical models
– Dimensionality reduction
• Principal component analysis (PCA)
• Factor analysis
Applications
• Face detection
• Object detection and recognition
• Image segmentation
• Multimedia event detection
• Economical and commercial usage
Conclusion
We have a simple overview of some
techniques and algorithms in machine
learning. Furthermore, there are more and
more techniques apply machine learning as
a solution. In the future, machine learning
will play an important role in our daily life.

You might also like