Machine Learning Foundations

(機器學習基石)

Lecture 3: Types of Learning

Hsuan-Tien Lin (林軒田)
[email protected]

Department of Computer Science

& Information Engineering
National Taiwan University
(國立台灣大學資訊工程系)

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 0/40

Types of Learning

Roadmap
1 When Can Machines Learn?

Lecture 2: Learning to Answer Yes/No

PLA A takes linear separable D and perceptrons H
to get hypothesis g

Lecture 3: Types of Learning

Learning with Different Output Space Y
Learning with Different Data Label yn
Learning with Different Protocol f ⇒ (xn , yn )
Learning with Different Input Space X

2 Why Can Machines Learn?

3 How Can Machines Learn?
4 How Can Machines Learn Better?

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 1/40

Types of Learning Learning with Different Output Space Y

Credit Approval Problem Revisited

age 23 years
gender female
annual salary NTD 1,000,000
year in residence 1 year
unknown target function year in job 0.5 year
f: X →Y current debt 200,000
(ideal credit approval formula) credit? {no(−1), yes(+1)}

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A
(historical records in bank) (‘learned’ formula to be used)

hypothesis set
H

(set of candidate formula)

Y = {−1, +1}: binary classification

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 2/40
Types of Learning Learning with Different Output Space Y

More Binary Classification Problems

• credit approve/disapprove
• email spam/non-spam
• patient sick/not sick
• ad profitable/not profitable
• answer correct/incorrect (KDDCup 2010)

core and important problem with

many tools as building block of other tools

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 3/40

Types of Learning Learning with Different Output Space Y

Multiclass Classification: Coin Recognition Problem

• classify US coins (1c, 5c, 10c, 25c)
25 by (size, mass)
Mass

5
• Y = {1c, 5c, 10c, 25c}, or
1 Y = {1, 2, · · · , K } (abstractly)
• binary classification: special case
10
with K = 2
Size

Other Multiclass Classification Problems

• written digits ⇒ 0, 1, · · · , 9
• pictures ⇒ apple, orange, strawberry
• emails ⇒ spam, primary, social, promotion, update (Google)

many applications in practice,

especially for ‘recognition’
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 4/40
Types of Learning Learning with Different Output Space Y

Multiclass Classification: Which Fruit?

?
(image by Robert-Owen-Wahl from Pixabay)

apple orange strawberry kiwi

(images by Pexels, PublicDomainPictures, 192635, Rob van der Meijden from Pixabay)

Y = {apple, orange, strawberry, kiwi}

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 5/40

Types of Learning Learning with Different Output Space Y

Multilabel Classification: Which Fruits?

?: {apple, orange, kiwi}

(image by Michal Jarmoluk from Pixabay)

apple orange strawberry kiwi

(images by Pexels, PublicDomainPictures, 192635, Rob van der Meijden from Pixabay)

multilabel classification:
classify input to multiple (or no) categories
Y = 2{apple,orange,strawberry,kiwi}

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 6/40

Types of Learning Learning with Different Output Space Y

What Tags?

?: {machine learning, data structure, data mining, object

oriented programming, artificial intelligence, compiler,
architecture, chemistry, textbook, children book, . . . etc. }
another multilabel classification problem:
tagging input to multiple categories

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 7/40

Types of Learning Learning with Different Output Space Y

Binary Relevance: Multilabel Classification via Yes/No

binary multilabel w/ L classes: L yes/no questions
classification machine learning (Y), data structure (N), data
{yes, no} mining (Y), OOP (N), AI (Y), compiler (N),
architecture (N), chemistry (N), textbook (Y),
children book (N), etc.

• Binary Relevance (BR): reduction (transformation) to multiple

isolated binary classification
• disadvantages (addressed by more sophisticated models):
• isolation—hidden relations not exploited
(e.g. ML and DM highly correlated, ML subset of AI, textbook &
children book disjoint)
• imbalanced—few yes, many no

BR for multilabel classification:

uses binary classification as a core tool
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 8/40
Types of Learning Learning with Different Output Space Y

Regression: Patient Recovery Prediction Problem

• binary classification: patient features ⇒ sick or not
• multiclass classification: patient features ⇒ which type of cancer
• regression: patient features ⇒ how many days before recovery
• Y = R or Y = [lower, upper] ⊂ R (bounded regression)
—deeply studied in statistics

Other Regression Problems

• company data ⇒ stock price
• climate data ⇒ temperature

also core and important with many ‘statistical’

tools as building block of other tools

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 9/40

Types of Learning Learning with Different Output Space Y

Sophisticated Output: Image Generation Problems

Style Transfer

+ ⇒
(Leonardo da Vinci, (Van Gogh, (Pjfinlay,
in Public Domain) in Public Domain) with CC0)
all images are downloaded from Wikipedia

Other Image Generation Problems

• noisy image ⇒ clean image
• low-resolution image ⇒ high-resolution image

Y: a ‘manifold’ ⊂ Rw×h×c ,
arguably not just multi-pixel regression
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 10/40
Types of Learning Learning with Different Output Space Y

Mini Summary
Learning with Different Output Space Y
• binary classification: Y = {−1, +1}
• multiclass classification: Y = {1, 2, · · · , K }
• multilabel classification: Y = 2{1,2,··· ,K }
• regression: Y = R
unknown target function
f: X →Y
• image generation: Y ⊂ Rw×h×c
• . . . and a lot more!!

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A

hypothesis set
H

core tools: binary classification and regression

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 11/40
Types of Learning Learning with Different Output Space Y

Fun Time
What is this learning problem?
The entrance system of the school gym, which does automatic face
recognition based on machine learning, is built to charge four different
groups of users differently: Staff, Student, Professor, Other. What type
of learning problem best fits the need of the system?
1 binary classification
2 multiclass classification
3 regression
4 structured learning

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 12/40

Types of Learning Learning with Different Output Space Y

Fun Time
What is this learning problem?
The entrance system of the school gym, which does automatic face
recognition based on machine learning, is built to charge four different
groups of users differently: Staff, Student, Professor, Other. What type
of learning problem best fits the need of the system?
1 binary classification
2 multiclass classification
3 regression
4 structured learning

Reference Answer: 2
There is an ‘explicit’ Y that contains four
classes.

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 12/40

Types of Learning Learning with Different Data Label yn

Supervised: Coin Recognition Revisited

25

Mass
5
unknown target function 1

f: X →Y
10

Size

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A

hypothesis set
H

supervised learning:
every xn comes with corresponding yn

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 13/40

Types of Learning Learning with Different Data Label yn

Unsupervised: Coin Recognition without yn

25
Mass

Mass
5
1

10

Size Size

supervised multiclass classification unsupervised multiclass classification

⇐⇒ ‘clustering’

Other Clustering Problems

• articles ⇒ topics
• consumer profiles ⇒ consumer groups

clustering: a challenging but useful problem

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 14/40

Types of Learning Learning with Different Data Label yn

Unsupervised: Coin Recognition without yn

25
Mass

Mass
5
1

10

Size Size

supervised multiclass classification unsupervised multiclass classification

⇐⇒ ‘clustering’

Other Clustering Problems

• articles ⇒ topics
• consumer profiles ⇒ consumer groups

clustering: a challenging but useful problem

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 14/40

Types of Learning Learning with Different Data Label yn

Unsupervised: Learning without yn

Other Unsupervised Learning Problems
• clustering: {xn } ⇒ cluster(x)
(≈ ‘unsupervised multiclass classification’)
—i.e. articles ⇒ topics
• density estimation: {xn } ⇒ density(x)
(≈ ‘unsupervised bounded regression’)
—i.e. traffic reports with location ⇒ dangerous areas
• outlier detection: {xn } ⇒ unusual(x)
(≈ extreme ‘unsupervised binary classification’)
—i.e. Internet logs ⇒ intrusion alert
• . . . and a lot more!!

unsupervised learning: diverse, with possibly

very different performance goals

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 15/40

Types of Learning Learning with Different Data Label yn

Semi-supervised: Coin Recognition with Some yn

25 25
Mass

Mass

Mass
5 5
1 1

10 10

Size Size Size

supervised semi-supervised unsupervised (clustering)

Other Semi-supervised Learning Problems

• face images with a few labeled ⇒ face identifier (Facebook)
• medicine data with a few labeled ⇒ medicine effect predictor

semi-supervised learning: leverage unlabeled

data to avoid ‘expensive’ labeling

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 16/40

Types of Learning Learning with Different Data Label yn

Self-supervised: Unsupervised + Self-defined Goal(s)

jigsaw puzzle: pieces → full picture

(Figure 1 of Noroozi and Favaro,

Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles. ECCV 2016)

Other Popular Goals

• colorization: grayscale image → colored image
• center word prediction: chunk of text → center word
• next sentence prediction: sentence A → is sentence B next?

self-supervised learning: recipe to learn

‘physical knowledge’ before actual task
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 17/40
Types of Learning Learning with Different Data Label yn

Weakly-supervised: Learning without True yn

complementary label: ȳn (‘not’ label) instead of yn

(Figure 1 of Yu et al., Learning with Biased Complementary Labels, ECCV 2018)

Other Weak Supervisions

• partial label: a set Yn that contains true yn
• noisy label: yn′ , a noisy version of true yn
• proportion label: aggregated statistics of a set of yn

weakly-supervised learning: another

realistic (?) family to reduce labeling burden

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 18/40

Types of Learning Learning with Different Data Label yn

Reinforcement Learning
a ‘very different’ but natural way of learning

Teach Your Dog: Say ‘Sit Down’

The dog pees on the ground.
BAD DOG. THAT’S A VERY WRONG ACTION.
• cannot easily show the dog that yn = sit
when xn = ‘sit down’
• but can ‘punish’ to say ỹn = pee is wrong

Other Reinforcement Learning Problems Using (x, ỹ , goodness)

• (customer, ad choice, ad click earning) ⇒ ad system
• (cards, strategy, winning amount) ⇒ black jack agent

reinforcement: learn with ‘partial/implicit

information’ (often sequentially)

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 19/40

Types of Learning Learning with Different Data Label yn

Reinforcement Learning
a ‘very different’ but natural way of learning

Teach Your Dog: Say ‘Sit Down’

The dog sits down.
Good Dog. Let me give you some cookies.
• still cannot show yn = sit
when xn = ‘sit down’
• but can ‘reward’ to say ỹn = sit is good

Other Reinforcement Learning Problems Using (x, ỹ , goodness)

• (customer, ad choice, ad click earning) ⇒ ad system
• (cards, strategy, winning amount) ⇒ black jack agent

reinforcement: learn with ‘partial/implicit

information’ (often sequentially)

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 19/40

Types of Learning Learning with Different Data Label yn

THE Most Well-known Reinforcement Learning Agent

(Public Domain, from Wikipedia; used here for education purpose; all other rights still belong to Google DeepMind)

Non-ML Techniques ML Techniques

Monte C. Tree Search Deep Learning Reinforcement Learn.
≈ move simulation in ≈ board analysis in ≈ (self)-practice in
brain human brain human training

(CC-BY-SA 3.0 by Stannered on (CC-BY-SA 2.0 by Frej Bjon on (Public Domain, from Wikipedia)
Wikipedia) Wikipedia)

good AI: important to use the right

techniques—ML & others, including human

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 20/40

Types of Learning Learning with Different Data Label yn

The LATEST Well-known RL Agent

(Public Domain, from Wikipedia; used here for education purpose; all other rights still belong to OpenAI)

GPT-3 chatGPT
Self-Supervised Supervised (Few-Shot) + Supervised (Ranking) + Reinforcement

• mainly next-token
prediction from 2048
tokens
• 175 billion parameters
trained with 500 billion
tokens

staged-ML important for

building huge ML systems
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 21/40
Types of Learning Learning with Different Data Label yn
Mini Summary
Learning with Different Data Label yn
• supervised: all yn
• unsupervised: no yn
• self-supervised: self-defined yn′ from xn
• semi-supervised: some yn
• weakly-supervised: no true yn
unknown target function
f: X →Y
• reinforcement: implicit yn by goodness(ỹn )
• . . . and more!!

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A

hypothesis set
H

core tool: supervised learning

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 22/40
Types of Learning Learning with Different Data Label yn

Fun Time
What is this learning problem?
To build a tree recognition system, a company decides to gather one
million of pictures on the Internet. Then, it asks each of the 10
company members to view 100 pictures and record whether each
picture contains a tree. The pictures and records are then fed to a
learning algorithm to build the system. What type of learning problem
does the algorithm need to solve?
1 supervised
2 unsupervised
3 semi-supervised
4 reinforcement

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 23/40

Types of Learning Learning with Different Data Label yn

Fun Time
What is this learning problem?
To build a tree recognition system, a company decides to gather one
million of pictures on the Internet. Then, it asks each of the 10
company members to view 100 pictures and record whether each
picture contains a tree. The pictures and records are then fed to a
learning algorithm to build the system. What type of learning problem
does the algorithm need to solve?
1 supervised
2 unsupervised
3 semi-supervised
4 reinforcement

Reference Answer: 3
The 1, 000 records are the labeled (xn , yn ); the
other 999, 000 pictures are the unlabeled xn .
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 23/40
Types of Learning Learning with Different Protocol f ⇒ (xn , yn )

Batch Learning: Coin Recognition Revisited

25

Mass
5
unknown target function 1

f: X →Y
10

Size

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A

hypothesis set
H

batch supervised multiclass classification:

learn from all known data

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 24/40

Types of Learning Learning with Different Protocol f ⇒ (xn , yn )

More Batch Learning Problems

25

Mass

Mass
5
1

10

Size Size

• batch of (email, spam?) ⇒ spam filter

• batch of (patient, cancer) ⇒ cancer
classifier
• batch of patient data ⇒ group of patients

batch learning: a very common protocol

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 25/40

Types of Learning Learning with Different Protocol f ⇒ (xn , yn )

Online: Spam Filter that ‘Improves’

• batch spam filter:
learn with known (email, spam?) pairs, and predict with fixed g
• online spam filter, which sequentially:
1 observe an email xt
2 predict spam status with current gt (xt )
3 receive ‘desired label’ yt from user, and then update gt with (xt , yt )

Connection to What We Have Learned

• PLA can be easily adapted to online protocol (how?)
• reinforcement learning is often done online (why?)

online: hypothesis ‘improves’ through receiving

data instances sequentially

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 26/40

Types of Learning Learning with Different Protocol f ⇒ (xn , yn )

Online + Batch for Real-World Applications

online prediction
model re-trained with for each request
historical daily batch
data, incrementally or
completely batch labelled data
collected daily

purely online purely batch

• incremental update costly • cannot capture drifts/trends
online well
• delayed labels hard to • complete re-training
handle properly possibly costly

real-world ML system
different from textbook settings
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 27/40
Types of Learning Learning with Different Protocol f ⇒ (xn , yn )

Active Learning: Learning by ‘Asking’

Protocol ⇔ Learning Philosophy
• batch: ‘duck feeding’
• online: ‘passive sequential’
unknown target function
f: X →Y • active: ‘question asking’ (sequentially)
—query the yn of the chosen xn

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A

hypothesis set
H

active: improve hypothesis with fewer labels

(hopefully) by asking questions strategically

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 28/40

Types of Learning Learning with Different Protocol f ⇒ (xn , yn )

Making Active Learning More Realistic

open-source tool libact developed by NTU CLLab (Yang, 2017)

https://github.com/ntucllab/libact
• including many popular strategies
• received > 500 stars and continuous issues

“libact is a Python package designed to make ac-

tive learning easier for real-world users”

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 29/40

Types of Learning Learning with Different Protocol f ⇒ (xn , yn )

Mini Summary
Learning with Different Protocol f ⇒ (xn , yn )
• batch: all known data
• online: sequential (passive) data
• online + batch: best of both worlds
unknown target function
f: X →Y • active: strategically-observed data
• . . . and more!!

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A

hypothesis set
H

core protocol: batch

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 30/40
Types of Learning Learning with Different Protocol f ⇒ (xn , yn )
Fun Time
What is this learning problem?
A photographer has 100, 000 pictures, each containing one baseball
player. He wants to automatically categorize the pictures by its player
inside. He starts by categorizing 1, 000 pictures by himself, and then
writes an algorithm that tries to categorize the other pictures if it is
‘confident’ on the category while pausing for (& learning from) human
input if not. What protocol best describes the nature of the algorithm?
1 batch
2 online
3 active
4 random

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 31/40

Types of Learning Learning with Different Protocol f ⇒ (xn , yn )
Fun Time
What is this learning problem?
A photographer has 100, 000 pictures, each containing one baseball
player. He wants to automatically categorize the pictures by its player
inside. He starts by categorizing 1, 000 pictures by himself, and then
writes an algorithm that tries to categorize the other pictures if it is
‘confident’ on the category while pausing for (& learning from) human
input if not. What protocol best describes the nature of the algorithm?
1 batch
2 online
3 active
4 random

Reference Answer: 3
The algorithm takes a active but naïve
strategy: ask when ‘confused’. You should
probably do the same when taking a class. :-)
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 31/40
Types of Learning Learning with Different Input Space X

Credit Approval Problem Revisited

age 23 years
gender female
annual salary NTD 1,000,000
unknown target function
year in residence 1 year
f: X →Y
year in job 0.5 year
(ideal credit approval formula) current debt 200,000

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A
(historical records in bank) (‘learned’ formula to be used)

hypothesis set
H

(set of candidate formula)

concrete features: each dimension of X ⊆ Rd

represents ‘sophisticated physical meaning’
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 32/40
Types of Learning Learning with Different Input Space X

More on Concrete Features

• (size, mass) for coin classification

• customer info for credit approval 25

Mass
• patient info for cancer diagnosis 5
1
• often including ‘human intelligence’
on the learning task 10

Size

concrete features: the ‘easy’ ones for ML

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 33/40

Types of Learning Learning with Different Input Space X

Raw Features: Digit Recognition Problem (1/2)

• digit recognition problem: features ⇒ meaning of digit

• a typical supervised multiclass classification problem

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 34/40

Types of Learning Learning with Different Input Space X

Raw Features: Digit Recognition Problem (2/2)

by Concrete Features by Raw Features

• 16 by 16 gray image x ≡
(0, 0, 0.9, 0.6, · · · ) ∈ R256
• ‘simple physical meaning’;
thus more difficult for ML
than concrete features

x =(symmetry, density)

Other Problems with Raw Features

• image pixels, speech signal, etc.

raw features: often need human or machines

to convert to concrete ones
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 35/40
Types of Learning Learning with Different Input Space X

Abstract Features: Rating Prediction Problem

Rating Prediction Problem (KDDCup 2011)
• given previous (userid, itemid, rating) tuples, predict the rating that
some userid would give to itemid?
• a regression problem with Y ⊆ R as rating and X ⊆ N × N as
(userid, itemid)
• ‘no physical meaning’; thus even more difficult for ML

Other Problems with Abstract Features

• student ID in online tutoring system (KDDCup 2010)
• advertisement ID in online ad system

abstract: again need ‘feature

conversion/extraction/construction’

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 36/40

Types of Learning Learning with Different Input Space X

Deep Learning: ‘Automatic’ Conversion

from Raw (or Abstract) to Concrete
positive weight
✻
negative weight

is it a ‘1’ ? ✲ z1 z5 ✛ is it a ‘5’ ?

φ2 φ3 φ4 φ5
φ1 φ6

,
• layered extraction: simple to complex features
• natural for difficult learning task with raw features, like vision

deep learning: currently popular in

vision/speech/. . .

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 37/40

Types of Learning Learning with Different Input Space X

Mini Summary
Learning with Different Input Space X
• concrete: sophisticated (and related)
physical meaning
• raw: simple physical meaning
unknown target function
f: X →Y • abstract: no (or little) physical meaning
• . . . and more!!

training examples learning final hypothesis

D : (x1 , y1 ), · · · , (xN , yN ) algorithm g≈f
A

hypothesis set
H

‘easy’ input: concrete

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 38/40
Types of Learning Learning with Different Input Space X

Fun Time
What features can be used?
Consider a problem of building an online image advertisement system
that shows the users the most relevant images. What features can you
choose to use?
1 concrete
2 concrete, raw
3 concrete, abstract
4 concrete, raw, abstract

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 39/40

Types of Learning Learning with Different Input Space X

Fun Time
What features can be used?
Consider a problem of building an online image advertisement system
that shows the users the most relevant images. What features can you
choose to use?
1 concrete
2 concrete, raw
3 concrete, abstract
4 concrete, raw, abstract

Reference Answer: 4
concrete user features, raw image features,
and maybe abstract user/image IDs

Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 39/40

Types of Learning Learning with Different Input Space X
Summary
1 When Can Machines Learn?

Lecture 2: Learning to Answer Yes/No

Lecture 3: Types of Learning
Learning with Different Output Space Y
[classification], [regression], sophisticated
Learning with Different Data Label yn
[supervised], un/semi/self-s., reinforcement
Learning with Different Protocol f ⇒ (xn , yn )
[batch], online, active
Learning with Different Input Space X
[concrete], raw, abstract
• next: learning is impossible?!
2 Why Can Machines Learn?
3 How Can Machines Learn?
4 How Can Machines Learn Better?
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 40/40