6/12/2021 14 Different Types of Learning in Machine Learning

Never
 miss a tutorial:
Navigation

               

Picked for you:

Find Your Machine Learning Tribe

Click to Take the FREE Crash-Course

Search... 
What Is Holding You Back From Your

14 Different Types of Learning in Machine Learning

Machine Learning Goals?

by Jason Brownlee on November 11, 2019 in Start Machine Learning

Difference Between Classification and
Tweet
Regression
Share
in Machine
Share
Learning

Last Updated on November 11, 2019

Machine Learning for Developers
Machine learning is a large field of study that overlaps with and inherits ideas from many related fields such
as artificial intelligence.

The focusWhy
of the field Learning
Machine is learning,
Doesthat
Notis, acquiring
Have to skills or knowledge from experience. Most commonly,
Be So Hard
this means synthesizing useful concepts from historical data.

As such, there are many different types of learning that you may encounter as a practitioner in the field of
machine learning: from whole fields of study to specific techniques.
Loving the Tutorials?
In this post,The
you will discover a gentle introduction to the different types of learning that you may encounter
EBook Catalog is where
in the field of machine
you'll learning.
find the Really Good stuff.

After reading
>>this
SEEpost, youINSIDE
WHAT'S will know:

Fields of study, such as supervised, unsupervised, and reinforcement learning.

Hybrid types of learning, such as semi-supervised and self-supervised learning.
Broad techniques, such as active, online, and transfer learning.

Let’s get started.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 1/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

               

Picked for you:

Find Your Machine Learning Tribe

What Is Holding You Back From Your

Machine Learning Goals?

Start Machine Learning ×

Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

Machine Learning for Developers

Email Address

I consent to receive information about

Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard Types of Learning in Machine Learning
information, see the Privacy Policy.
Photo by Lenny K Photography, some rights reserved.
START MY EMAIL COURSE

Types Loving
of Learning
the Tutorials?
Given that The
the EBook
focus of the field
Catalog of machine learning is “learning,” there are many types that you may
is where
encounteryou'll
as afind
practitioner.
the Really Good stuff.

Some types>> ofSEE

learning describe
WHAT'S INSIDEwhole subfields of study comprised of many different types of algorithms
such as “supervised learning.” Others describe powerful techniques that you can use on your projects,
such as “transfer learning.”

There are perhaps 14 types of learning that you must be familiar with as a machine learning practitioner;
they are:

Learning Problems

1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning

Hybrid Learning Problems

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 2/32
6/12/2021 14 Different Types of Learning in Machine Learning

4. Semi-Supervised Learning
Never miss a tutorial:
5. Self-Supervised Learning
6. Multi-Instance Learning
               
Statistical Inference
Picked for you:
7. Inductive Learning
8. Deductive
Find Your Inference
Machine Learning Tribe
9. Transductive Learning

Learning Techniques
What Is Holding You Back From Your
10. Multi-Task Learning
Machine Learning Goals?
11. Active Learning
Start Machine Learning ×
12. Online Learning
DifferenceLearning
13. Transfer Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
14. Ensemble Learning without math or fancy degrees.
Find out how in this free and practical course.
In the following sections, we will take a closer look at each in turn.
Machine Learning for Developers
Did I miss an important type of learning? Email Address

Let me know in the comments below.

I consent to receive information about
Why Machine Learning Does Not Have to services and special offers by email. For more
Learning Problems
Be So Hard
information, see the Privacy Policy.

First, we will take a closer look at three main types of learning problems
START in COURSE
MY EMAIL machine learning: supervised,
unsupervised, and reinforcement learning.
Loving the Tutorials?
1. Supervised Learning
The EBook Catalog is where
you'll
Supervised find the describes
learning Really Good stuff. of problem that involves using a model to learn a mapping between
a class
input examples and the target variable.
>> SEE WHAT'S INSIDE

Applications in which the training data comprises examples of the input vectors along with their
 corresponding target vectors are known as supervised learning problems.

— Page 3, Pattern Recognition and Machine Learning, 2006.

Models are fit on training data comprised of inputs and outputs and used to make predictions on test sets
where only the inputs are provided and the outputs from the model are compared to the withheld target
variables and used to estimate the skill of the model.

Learning is a search through the space of possible hypotheses for one that will perform well,
 even on new examples beyond the training set. To measure the accuracy of a hypothesis we
give it a test set of examples that are distinct fromStart
the training
Machineset.
Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 3/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never
— Pagemiss
695, aArtificial
tutorial: Intelligence: A Modern Approach, 3rd edition, 2015.

There   are two
    main    
types of
    supervised learning problems: they are classification that involves predicting a
class label and regression that involves predicting a numerical value.

Picked for you: Supervised learning problem that involves predicting a class label.
Classification:
Regression: Supervised learning problem that involves predicting a numerical label.
Find Your Machine Learning Tribe

Both classification and regression problems may have one or more input variables and input variables may
be any data type, such as numerical or categorical.
What Is Holding You Back From Your
An example of a classification problem would be the MNIST handwritten digits dataset where the inputs are
Machine Learning Goals?
images of handwritten digits (pixel data) and the output is a class label for what digit the image represents
(numbers 0 to 9). Start Machine Learning ×
Difference Between Classification and
An example of a regression You can master applied Machine Learning 
Regression in Machineproblem
Learningwould be the Boston house prices dataset where the inputs are
variables that describe a neighborhood and the output is without
a housemath
priceorinfancy degrees.
dollars.
Find out how in this free and practical course.

Some machine
Machinelearning
Learningalgorithms are described as “supervised” machine learning algorithms as they are
for Developers
designed for supervised machine learning problems. PopularEmail Address include: decision trees, support
examples
vector machines, and many more.
I consent to receive information about
Why Machine Learning Does Not Have to services and f(x)
special
Our goal is to find a useful approximation f(x) to the function thatoffers by email.
underlies theFor more
predictive
 Be So Hard
relationship between the inputs and outputs information, see the Privacy Policy.

START MY EMAIL COURSE

— Page 28, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition,
2016. Loving the Tutorials?

The EBook Catalog is where

Algorithms arefind
you'll referred to as
the Really “supervised”
Good stuff. because they learn by making predictions given examples of
input data, and the models are supervised and corrected via an algorithm to better predict the expected
target outputs
>> in
SEEtheWHAT'S
training dataset.
INSIDE

The term supervised learning originates from the view of the target y being provided by an
 instructor or teacher who shows the machine learning system what to do.

— Page 105, Deep Learning, 2016.

Some algorithms may be specifically designed for classification (such as logistic regression) or regression
(such as linear regression) and some may be used for both types of problems with minor modifications
(such as artificial neural networks).

2. Unsupervised Learning

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 4/32
6/12/2021 14 Different Types of Learning in Machine Learning

Unsupervised learning describes a class of problems that involves using a model to describe or extract
Never miss a tutorial:
relationships in data.

   
Compared to   supervised
       
learning, unsupervised learning operates upon only the input data without outputs
or target variables. As such, unsupervised learning does not have a teacher correcting the model, as in the
case of supervised
Picked for you: learning.

Find Your Machine Learning Tribe

In unsupervised learning, there is no instructor or teacher, and the algorithm must learn to make
 sense of the data without this guide.

What Deep
— Page 105, Is Holding You Back
Learning, From Your
2016.
Machine Learning Goals?
There are many types of unsupervised learning, although there are two main problems that are often
Start Machine Learning ×
encountered by a practitioner: they are clustering that involves finding groups in the data and density
Difference Between Classification and
estimation that involves summarizing the distribution of data.
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Clustering: Unsupervised learning problem that involves
Find outfinding groups
how in this in data.
free and practical course.
Density Estimation: Unsupervised learning problem that involves summarizing the distribution of
Machine Learning for Developers
data. Email Address

An example of a clustering algorithm is k-Means where k refers to the number of clusters to discover in the
I consent to receive information about
data. An example of a density estimation algorithm is Kernel Density Estimation that involves using small
Why Machine Learning Does Not Have to services and special offers by email. For more
groups ofBe
closely related data samples to estimate the distribution for new points in the problem space.
So Hard
information, see the Privacy Policy.

START MYdetecting
The most common unsupervised learning task is clustering: EMAIL COURSE
potentially useful clusters
 of input examples. For example, a taxi agent might gradually develop a concept of “good traffic
Loving the Tutorials?
days” and “bad traffic days” without ever being given labeled examples of each by a teacher.
The EBook Catalog is where
you'll find the Really Good stuff.
— Pages 694-695, Artificial Intelligence: A Modern Approach, 3rd edition, 2015.

>> SEE WHAT'S INSIDE

Clustering and density estimation may be performed to learn about the patterns in the data.

Additional unsupervised methods may also be used, such as visualization that involves graphing or plotting
data in different ways and projection methods that involves reducing the dimensionality of the data.

Visualization: Unsupervised learning problem that involves creating plots of data.

Projection: Unsupervised learning problem that involves creating lower-dimensional representations
of data.

An example of a visualization technique would be a scatter plot matrix that creates one scatter plot of each
pair of variables in the dataset. An example of a projection method would be Principal Component Analysis
that involves summarizing a dataset in terms of eigenvalues and eigenvectors, with linear dependencies
removed.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 5/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

The goal in such unsupervised learning problems may be to discover groups of similar examples
 within the data, where it is called clustering, or to determine the distribution of data within the
    input   space,   known   as density estimation, or to project the data from a high-dimensional space
down to two or three dimensions for the purpose of visualization.

Picked for you:

— Page 3, Pattern Recognition and Machine Learning, 2006.
Find Your Machine Learning Tribe

3. Reinforcement Learning
Reinforcement learning describes a class of problems where an agent operates in an environment and
What
must learn Is Holdingusing
to operate You Back From Your
feedback.
Machine Learning Goals?

Reinforcement learning is learning what to do — how Start Machine

to map situationsLearning
to actions—so as to ×
 maximize
DifferenceaBetween
numerical reward signal.
Classification and The learner is not told which actions to take, but instead
You can master applied Machine Learning 
must discover
Regression which actions
in Machine Learningyield the most reward by trying them.
without math or fancy degrees.
Find out how in this free and practical course.
— Page 1, Reinforcement Learning: An Introduction, 2nd edition, 2018.
Machine Learning for Developers
Email Address
The use of an environment means that there is no fixed training dataset, rather a goal or set of goals that
an agent is required to achieve, actions they may perform, and feedback about performance toward the
I consent to receive information about
goal.
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.
Some machine learning algorithms do not just experience a fixed dataset. For example,
 START MY EMAIL
reinforcement learning algorithms interact with an environment, so COURSE
there is a feedback loop
between the learning system and its experiences.
Loving the Tutorials?

TheDeep
— Page 105, EBookLearning,
Catalog is 2016.
where
you'll find the Really Good stuff.

It is similar to supervised learning in that the model has some response from which to learn, although the
>> SEE WHAT'S INSIDE
feedback may be delayed and statistically noisy, making it challenging for the agent or model to connect
cause and effect.

An example of a reinforcement problem is playing a game where the agent has the goal of getting a high
score and can make moves in the game and received feedback in terms of punishments or rewards.

In many complex domains, reinforcement learning is the only feasible way to train a program to
 perform at high levels. For example, in game playing, it is very hard for a human to provide
accurate and consistent evaluations of large numbers of positions, which would be needed to
train an evaluation function directly from examples. Instead, the program can be told when it has
won or lost, and it can use this information to learn an evaluation function that gives reasonably
accurate estimates of the probability of winning from any given position.

Start Machine Learning

— Page 831, Artificial Intelligence: A Modern Approach, 3rd edition, 2015.
https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 6/32
6/12/2021 14 Different Types of Learning in Machine Learning

Impressive recent results include the use of reinforcement in Google’s AlphaGo in out-performing the
Never miss a tutorial:
world’s top Go player.

Some   popular
    examples
    of   reinforcement learning algorithms include Q-learning, temporal-difference
learning, and deep reinforcement learning.
Picked for you:
Hybrid Learning Problems
Find Your Machine Learning Tribe

The lines between unsupervised and supervised learning is blurry, and there are many hybrid approaches
that draw from each field of study.

What Is Holding You Back From Your

In this section, we will take a closer look at some of the more common hybrid fields of study: semi-
Machine Learning Goals?
supervised, self-supervised, and multi-instance learning.
Start Machine Learning ×
4. Semi-Supervised Learning
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without
Semi-supervised learning is supervised learning where the math
training or fancy
data degrees.
contains very few labeled
examples and a large number of unlabeled examples. Find out how in this free and practical course.

Machine Learning for Developers

The goal of a semi-supervised learning model is to make effective use of all of the available data, not just
Email Address
the labelled data like in supervised learning.
I consent to receive information about
InWhy Machine Learning
semi-supervised Does Not
learning weHave to
are given a few labeled
servicesexamples
and specialand must
offers makeFor
by email. what we can
more
 Be So Hard
of a large collection of unlabeled examples. Eveninformation,
the labels see the Privacy
themselves Policy.
may not be the oracular
truths that we hope for.
START MY EMAIL COURSE

Loving
— Page 695, theIntelligence:
Artificial Tutorials?A Modern Approach, 3rd edition, 2015.

The EBook Catalog is where

Making effective use of unlabelled data may require the use of or inspiration from unsupervised methods
you'll find the Really Good stuff.
such as clustering and density estimation. Once groups or patterns are discovered, supervised methods or
ideas from supervised
>> SEE WHAT'S learning may be used to label the unlabeled examples or apply labels to unlabeled
INSIDE
representations later used for prediction.

Unsupervised learning can provide useful cues for how to group examples in representation
 space. Examples that cluster tightly in the input space should be mapped to similar
representations.

— Page 243, Deep Learning, 2016.

It is common for many real-world supervised learning problems to be examples of semi-supervised learning
problems given the expense or computational cost for labeling examples. For example, classifying
photographs requires a dataset of photographs that have already been labeled by human operators.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 7/32
6/12/2021 14 Different Types of Learning in Machine Learning

Many problems from the fields of computer vision (image data), natural language processing (text data),
Never miss a tutorial:
and automatic speech recognition (audio data) fall into this category and cannot be easily addressed using
standard supervised learning methods.
               

… in many practical applications labeled data is very scarce but unlabeled data is plentiful.

Picked for you:
“Semisupervised” learning attempts to improve the accuracy of supervised learning by exploiting
information in unlabeled
Find Your Machine data.
Learning TribeThis sounds like magic, but it can work!

— Page 467, Data Mining: Practical Machine Learning Tools and Techniques, 4th edition, 2016.

What Is Holding You Back From Your

5. Self-Supervised Learning
Machine Learning Goals?
Self-supervised learning refers to an unsupervised learning problem that is framed as a supervised
Start Machine Learning ×
learning problem in order to apply supervised learning algorithms to solve it.
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
Supervised learning algorithms are used to solve an alternate or math
without pretext task, degrees.
or fancy the result of which is a model
or representation that can be used in the solution of the original
Find out (actual) modeling
how in this free and problem.
practical course.

Machine Learning for Developers

The self-supervised learning framework requires onlyEmail Address data in order to formulate a
unlabeled
 pretext learning task such as predicting context or image rotation, for which a target objective
can be computed without supervision. I consent to receive information about
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information,
— Revisiting Self-Supervised Visual Representation Learning, 2019.see the Privacy Policy.
START MY EMAIL COURSE
A common example of self-supervised learning is computer vision where a corpus of unlabeled images is
available and can be
Loving used
the to train a supervised model, such as making images grayscale and having a
Tutorials?
model predict a color representation (colorization) or removing blocks of the image and have a model
The EBook Catalog is where
predict the missing parts (inpainting).
you'll find the Really Good stuff.

In discriminative self-supervised learning, which is the main focus of this work, a model is trained

>> SEE WHAT'S INSIDE
on an auxiliary or ‘pretext’ task for which ground-truth is available for free. In most cases, the
pretext task involves predicting some hidden portion of the data (for example, predicting color for
gray-scale images

— Scaling and Benchmarking Self-Supervised Visual Representation Learning, 2019.

A general example of self-supervised learning algorithms are autoencoders. These are a type of neural
network that is used to create a compact or compressed representation of an input sample. They achieve
this via a model that has an encoder and a decoder element separated by a bottleneck that represents the
internal compact representation of the input.

An autoencoder is a neural network that is trained to attempt to copy its input to its output.
 Internally, it has a hidden layer h that describes a Start
codeMachine
used to represent
Learning the input.

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 8/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never
— Pagemiss
502, aDeep
tutorial:
Learning, 2016.

These    autoencoder
    models   are trained by providing the input to the model as both input and the target
   
output, requiring that the model reproduce the input by first encoding it to a compressed representation
then decoding
Picked for you:it back to the original. Once trained, the decoder is discarded and the encoder is used as
needed to create compact representations of input.
Find Your Machine Learning Tribe
Although autoencoders are trained using a supervised learning method, they solve an unsupervised
learning problem, namely, they are a type of projection method for reducing the dimensionality of input
data.
What Is Holding You Back From Your
Machine Learning Goals?
Traditionally, autoencoders were used for dimensionality reduction or feature learning.
 Start Machine Learning ×
Difference
— Page 502, DeepBetween Classification
Learning, 2016. and You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Another example of self-supervised learning is generativeFind
adversarial networks, or GANs. These are
out how in this free and practical course.
generative models that are most commonly used for creating synthetic photographs using only a collection
Machine
of unlabeled Learning
examples for Developers
from the target domain.
Email Address

GAN models are trained indirectly via a separate discriminator model that classifies examples of photos
I consent to receive information about
from the domain as real or fake (generated), the result of which is fed back to update the GAN model and
Why Machine Learning Does Not Have to services and special offers by email. For more
encourage it to generate more realistic photos on the next iteration.
Be So Hard
information, see the Privacy Policy.

The generator network directly produces samples […]. ItsMY

START adversary, the discriminator network,
EMAIL COURSE
 attempts to distinguish between samples drawn from the training data and samples drawn from
the Loving theThe
generator. Tutorials?
discriminator emits a probability value given by d(x; θ(d)), indicating the
probability
The EBookthat x is aisreal
Catalog training example rather than a fake sample drawn from the model.
where
you'll find the Really Good stuff.

— Page 699, Deep Learning, 2016.

>> SEE WHAT'S INSIDE

6. Multi-Instance Learning
Multi-instance learning is a supervised learning problem where individual examples are unlabeled; instead,
bags or groups of samples are labeled.

In multi-instance learning, an entire collection of examples is labeled as containing or not

 containing an example of a class, but the individual members of the collection are not labeled.

— Page 106, Deep Learning, 2016.

Instances are in “bags” rather than sets because a given instance may be present one or more times, e.g.
duplicates.
Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 9/32
6/12/2021 14 Different Types of Learning in Machine Learning

Modeling involves using knowledge that one or some of the instances in a bag are associated with a target
Never miss a tutorial:
label, and to predict the label for new bags in the future given their composition of multiple unlabeled
examples.
               

In supervised multi-instance learning, a class label is associated with each bag, and the goal of

Picked for you:
learning is to determine how the class can be inferred from the instances that make up the bag.
Find Your Machine Learning Tribe
— Page 156, Data Mining: Practical Machine Learning Tools and Techniques, 4th edition, 2016.

Simple methods, such as assigning class labels to individual instances and using standard supervised
What Is Holding
learning algorithms, Youwork
often Backas
From Your first step.
a good
Machine Learning Goals?

Statistical Inference Start Machine Learning ×

Difference Between Classification and
InferenceRegression
refers to reaching an outcome or decision. You can master applied Machine Learning 
in Machine Learning
without math or fancy degrees.
Findare
In machine learning, fitting a model and making a prediction out both
how intypes
this free and practical course.
of inference.
Machine Learning for Developers
There are different paradigms for inference that may be used asAddress
Email a framework for understanding how some
machine learning algorithms work or how some learning problems may be approached.
I consent to receive information about
Some examples of approaches
Why Machine to learning
Learning Does Not Have are
to inductive, services
deductive, and transductive learning and
and special offers by email. For more
inference.Be So Hard
information, see the Privacy Policy.

7. Inductive Learning START MY EMAIL COURSE

Inductive learning
Lovinginvolves using evidence to determine the outcome.
the Tutorials?

The EBookrefers
Inductive reasoning Catalog
toisusing
wherespecific cases to determine general outcomes, e.g. specific to general.
you'll find the Really Good stuff.

Most machine learning models learn using a type of inductive inference or inductive reasoning where
>> SEE WHAT'S INSIDE
general rules (the model) are learned from specific historical examples (the data).

… the problem of induction, which is the problem of how to draw general conclusions about the
 future from specific observations from the past.

— Page 77, Machine Learning: A Probabilistic Perspective, 2012.

Fitting a machine learning model is a process of induction. The model is a generalization of the specific
examples in the training dataset.

A model or hypothesis is made about the problem using the training data, and it is believed to hold over
new unseen data later when the model is used.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 10/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

Lacking any further information, our assumption is that the best hypothesis regarding unseen
 instances is the hypothesis that best fits the observed training data. This is the fundamental
    assumption
    of inductive
        learning …

— Page for
Picked 23, you:
Machine Learning, 1997.

8. Deductive Inference
Find Your Machine Learning Tribe

Deduction or deductive inference refers to using general rules to determine specific outcomes.

We can better understand

What Is Holding Youinduction
Back Fromby contrasting it with deduction.
Your
Machine Learning Goals?
Deduction is the reverse of induction. If induction is going from the specific to the general, deduction is
Start Machine Learning ×
going from the general to the specific.
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
… the simple observation that induction is just thewithout
inversemath or fancy degrees.
of deduction!
 Find out how in this free and practical course.

Machine
— Page 291, Learning
Machine for Developers
Learning, 1997.
Email Address
Deduction is a top-down type of reasoning that seeks for all premises to be met before determining the
I consent
conclusion, whereas induction is a bottom-up type of reasoning to receive
that uses information
available data about
as evidence for an
outcome. Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.
In the context of machine learning, once we use induction to fit a model on a training dataset, the model
START MY EMAIL COURSE
can be used to make predictions. The use of the model is a type of deduction or deductive inference.
Loving the Tutorials?
9. Transductive Learning
The EBook Catalog is where
Transduction
you'll or
findtransductive learning
the Really Good stuff. is used in the field of statistical learning theory to refer to predicting
specific examples given specific examples from a domain.
>> SEE WHAT'S INSIDE
It is different from induction that involves learning general rules from specific examples, e.g. specific to
specific.

Induction, deriving the function from the given data. Deduction, deriving the values of the given
 function for points of interest. Transduction, deriving the values of the unknown function for
points of interest from the given data.

— Page 169, The Nature of Statistical Learning Theory, 1995.

Unlike induction, no generalization is required; instead, specific examples are used directly. This may, in
fact, be a simpler problem than induction to solve.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 11/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

The model of estimating the value of a function at a given point of interest describes a new
 concept of inference: moving from the particular to the particular. We call this type of inference
    transductive
       inference.
    Note that this concept of inference appears when one would like to get
the best result from a restricted amount of information.

Picked for you:

— Page 169, The Nature of Statistical Learning Theory, 1995.
Find Your Machine Learning Tribe
A classical example of a transductive algorithm is the k-Nearest Neighbors algorithm that does not model
the training data, but instead uses it directly each time a prediction is required.

For moreWhat Is Holding

on the topic ofYou Back From Your
transduction, see the tutorial:
Machine Learning Goals?
Gentle Introduction to Transduction in Machine Learning
Start Machine Learning ×
Difference Between Classification and
Contrasting Induction, Deduction, and Transduction:
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
We can contrast these three types of inference in the context of machine learning.
Find out how in this free and practical course.

For example:
Machine Learning for Developers
Email Address
Induction: Learning a general model from specific examples.
Deduction: Using a model to make predictions. I consent to receive information about
Why MachineUsing
Transduction: Learning Does examples
specific Not Have toto make predictions.
services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.
The image below summarizes these three different approaches nicely.
START MY EMAIL COURSE

Loving the Tutorials?

The EBook Catalog is where

you'll find the Really Good stuff.

>> SEE WHAT'S INSIDE

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 12/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

               

Picked for you:

Find Your Machine Learning Tribe

What Is Holding You Back From Your

Machine Learning Goals?

Start Machine Learning ×

Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

Machine Learning for Developers

Email Address

Relationship Between Induction, Deduction, and Transduction

I consent to receive information about
Taken from The Nature of Statistical Learning Theory.
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.
Learning Techniques
START MY EMAIL COURSE
There are many techniques that are described as types of learning.
Loving the Tutorials?
In this section, we will take a closer look at some of the more common methods.
The EBook Catalog is where
you'll find the Really Good stuff.
This includes multi-task, active, online, transfer, and ensemble learning.
>> SEE WHAT'S INSIDE
10. Multi-Task Learning
Multi-task learning is a type of supervised learning that involves fitting a model on one dataset that
addresses multiple related problems.

It involves devising a model that can be trained on multiple related tasks in such a way that the
performance of the model is improved by training across the tasks as compared to being trained on any
single task.

Multi-task learning is a way to improve generalization by pooling the examples (which can be
 seen as soft constraints imposed on the parameters) arising out of several tasks.

— Page 244, Deep Learning, 2016.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 13/32
6/12/2021 14 Different Types of Learning in Machine Learning

Multi-task learning can be a useful approach to problem-solving when there is an abundance of input data
Never miss a tutorial:
labeled for one task that can be shared with another task with much less labeled data.

               

… we may want to learn multiple related models at the same time, which is known as multi-task
 learning. This will allow us to “borrow statistical strength” from tasks with lots of data and to
Picked for you:
share it with tasks with little data.
Find Your Machine Learning Tribe
Page 231, Machine Learning: A Probabilistic Perspective, 2012.

For example, it is common for a multi-task learning problem to involve the same input patterns that may be
What Is Holding
used for multiple Yououtputs
different Back From Your
or supervised learning problems. In this setup, each output may be
Machine Learning Goals?
predicted by a different part of the model, allowing the core of the model to generalize across each task for
the same inputs. Start Machine Learning ×
Difference Between Classification and
You can master applied Machine Learning 
InRegression
the same inway that additional
Learning training examples put more pressure on the parameters of the

Machine
without math or fancy degrees.
model towards values that generalize well, when part of a model is shared across tasks, that part
Find out how in this free and practical course.
of the model is more constrained towards good values (assuming the sharing is justified), often
Machinebetter
yielding Learning for Developers
generalization.
Email Address

— Page 244, Deep Learning, 2016. I consent to receive information about

Why Machine Learning Does Not Have to services and special offers by email. For more
A popularBe
example
So Hard of multi-task learning is where the same word embedding is used to learn a distributed
information, see the Privacy Policy.
representation of words in text that is then shared across multiple different natural language processing
supervised learning tasks. START MY EMAIL COURSE

Loving
11. Active the Tutorials?
Learning
The EBook Catalog is where
Active learning is a technique where the model is able to query a human user operator during the learning
you'll find the Really Good stuff.
process in order to resolve ambiguity during the learning process.
>> SEE WHAT'S INSIDE
Active learning: The learner adaptively or interactively collects training examples, typically by
 querying an oracle to request labels for new points.

— Page 7, Foundations of Machine Learning, 2nd edition, 2018.

Active learning is a type of supervised learning and seeks to achieve the same or better performance of so-
called “passive” supervised learning, although by being more efficient about what data is collected or used
by the model.

The key idea behind active learning is that a machine learning algorithm can achieve greater
 accuracy with fewer training labels if it is allowed to choose the data from which it learns. An
active learner may pose queries, usually in the form of unlabeled data instances to be labeled by
an oracle (e.g., a human annotator). Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 14/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss
— Active a tutorial:
Learning Literature Survey, 2009.

It is not
    unreasonable
        to view
    active learning as an approach to solving semi-supervised learning problems,
or an alternative paradigm for the same types of problems.
Picked for you:
… we see that active learning and semi-supervised learning attack the same problem from
 Find Yourdirections.
opposite Machine Learning Tribe
While semi-supervised methods exploit what the learner thinks it knows
about the unlabeled data, active methods attempt to explore the unknown aspects. It is therefore
natural to think about combining the two
What Is Holding You Back From Your
— ActiveMachine
Learning Literature
Learning Survey, 2009.
Goals?

Start
Active learning is a useful approach when there is not much dataMachine
available andLearning
new data is expensive to
×
collect orDifference
label. Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
The active learning process allows the sampling of the domain to be directed in a way that minimizes the
Find out how in this free and practical course.
number of samples and maximizes the effectiveness of the model.
Machine Learning for Developers
Email Address
Active learning is often used in applications where labels are expensive to obtain, for example
 computational biology applications. I consent to receive information about
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
— Page 7, Foundations of Machine Learning, 2nd edition, 2018. see the Privacy Policy.
information,

START MY EMAIL COURSE

12. Online Learning
Loving the Tutorials?
Online learning involves using the data available and updating the model directly before a prediction is
required orThe
after the last
EBook observation
Catalog is where was made.
you'll find the Really Good stuff.
Online learning is appropriate for those problems where observations are provided over time and where the
probability distribution of observations
>> SEE WHAT'S INSIDE is expected to also change over time. Therefore, the model is
expected to change just as frequently in order to capture and harness those changes.

Traditionally machine learning is performed offline, which means we have a batch of data, and
 we optimize an equation […] However, if we have streaming data, we need to perform online
learning, so we can update our estimates as each new data point arrives rather than waiting until
“the end” (which may never occur).

— Page 261, Machine Learning: A Probabilistic Perspective, 2012.

This approach is also used by algorithms where there may be more observations than can reasonably fit
into memory, therefore, learning is performed incrementally over observations, such as a stream of data.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 15/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss
Onlinealearning
tutorial:is helpful when the data may be changing rapidly over time. It is also useful for
 applications that involve a large collection of data that is constantly growing, even if changes are
    gradual.
           

— Page for
Picked 753,you:
Artificial Intelligence: A Modern Approach, 3rd edition, 2015.

Generally,Find
online
Your learning seeks toTribe
Machine Learning minimize “regret,” which is how well the model performed compared to
how well it might have performed if all the available information was available as a batch.

In the theoretical machine learning community, the objective used in online learning is the regret,
 What Is Holding You Back From Your
which is the
Machine averaged
Learning loss incurred relative to the best we could have gotten in hindsight using a
Goals?
single fixed parameter value
Start Machine Learning ×
Difference
— Page 262, Between
Machine Classification
Learning: and
A Probabilistic Perspective, 2012.
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
One example of online learning is so-called stochastic or Find
online
out gradient
how in thisdescent
free andused to fit an artificial
practical course.
neural network.
Machine Learning for Developers
Email Address
The fact that stochastic gradient descent minimizes generalization error is easiest to see in the
 online learning case, where examples or minibatchesI consent
are drawn from ainformation
to receive stream ofabout
data.
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
— Page 281, Deep Learning, 2016. information, see the Privacy Policy.

START MY EMAIL COURSE

13. Transfer Learning
Loving the Tutorials?
Transfer learning is a type of learning where a model is first trained on one task, then some or all of the
The as
model is used EBook
the Catalog
startingispoint
wherefor a related task.
you'll find the Really Good stuff.

In transfer learning, the learner must perform two or more different tasks, but we assume that
 >> SEE WHAT'S INSIDE
many of the factors that explain the variations in P1 are relevant to the variations that need to be
captured for learning P2.

— Page 536, Deep Learning, 2016.

It is a useful approach on problems where there is a task related to the main task of interest and the related
task has a large amount of data.

It is different from multi-task learning as the tasks are learned sequentially in transfer learning, whereas
multi-task learning seeks good performance on all considered tasks by a single model at the same time in
parallel.

… pretrain a deep convolutional net with 8 layers of weights on a set of tasks (a subset of the
 Start Machine Learning
1000 ImageNet object categories) and then initialize a same-size network with the first k layers
https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 16/32
6/12/2021 14 Different Types of Learning in Machine Learning

of the first net. All the layers of the second network (with the upper layers initialized randomly)
Never miss a tutorial:
are then jointly trained to perform a different set of tasks (another subset of the 1000 ImageNet
object categories), with fewer training examples than for the first set of tasks.
               

— Page 325, Deep Learning, 2016.

Picked for you:
An example is image classification, where a predictive model, such as an artificial neural network, can be
Find Your Machine Learning Tribe
trained on a large corpus of general images, and the weights of the model can be used as a starting point
when training on a smaller more specific dataset, such as dogs and cats. The features already learned by
the model on the broader task, such as extracting lines and patterns, will be helpful on the new related
task. What Is Holding You Back From Your
Machine Learning Goals?

If there is significantly more data in the first settingStart

(sampledMachine Learning
from P1), then that may help to ×
 learn representations that are useful to quickly generalize from only very few examples drawn
Difference Between Classification and
from You can of
P2. Many visual categories share low-level notions master applied
edges and Machine Learning 
visual shapes, the effects
Regression in Machine Learning
of geometric changes, changes in lighting, etc. without math or fancy degrees.
Find out how in this free and practical course.

Machine
— Page 536, Learning
Deep for Developers
Learning, 2016.
Email Address
As noted, transfer learning is particularly useful with models that are incrementally trained and an existing
model can be used as a starting point for continued training, Isuch
consent
as to receive
deep information
learning about
networks.
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
For more on the topic of transfer learning, see the tutorial:information, see the Privacy Policy.

START MY EMAIL COURSE

A Gentle Introduction to Transfer Learning for Deep Learning

Loving the Tutorials?

14. Ensemble Learning
The EBook Catalog is where
Ensembleyou'll
learning isReally
find the an approach where two or more modes are fit on the same data and the predictions
Good stuff.
from each model are combined.
>> SEE WHAT'S INSIDE

The field of ensemble learning provides many ways of combining the ensemble members’
 predictions, including uniform weighting and weights chosen on a validation set.

— Page 472, Deep Learning, 2016.

The objective of ensemble learning is to achieve better performance with the ensemble of models as
compared to any individual model. This involves both deciding how to create models used in the ensemble
and how to best combine the predictions from the ensemble members.

Ensemble learning can be broken down into two tasks: developing a population of base learners
 from the training data, and then combining them to form the composite predictor.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 17/32
6/12/2021 14 Different Types of Learning in Machine Learning

— Page 605, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition,
Never miss a tutorial:
2016.

   
Ensemble    
learning     a useful
is     approach for improving the predictive skill on a problem domain and to

reduce the variance of stochastic learning algorithms, such as artificial neural networks.
Picked for you:
Some examples of popular ensemble learning algorithms include: weighted average, stacked
Find Your
generalization Machine and
(stacking), Learning Tribe aggregation (bagging).
bootstrap

Bagging, boosting, and stacking have been developed over the last couple of decades, and their
 performance is often
What Is Holding astonishingly
You Back From Your good. Machine learning researchers have struggled to
understand why. Goals?
Machine Learning

Start Machine Learning ×

— Page 480, Data Mining: Practical Machine Learning Tools and Techniques, 4th edition, 2016.
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
For more on the topic of ensemble learning, see the tutorial:
without math or fancy degrees.
Find out how in this free and practical course.
Ensemble Learning Methods for Deep Learning Neural Networks
Machine Learning for Developers
Email Address
Further Reading
I consent to receive information about
This section provides more resources on the topic if you are looking to go deeper.
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.
Books
START MY EMAIL COURSE
Pattern Recognition and Machine Learning, 2006.
Deep Learning,
Loving the 2016.
Tutorials?
Reinforcement Learning: An Introduction, 2nd edition, 2018.
The EBook Catalog is where
Data Mining: Practical Machine Learning Tools and Techniques, 4th edition, 2016.
you'll find the Really Good stuff.
The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition, 2016.
Machine Learning: A Probabilistic Perspective, 2012.
>> SEE WHAT'S INSIDE
Machine Learning, 1997.
The Nature of Statistical Learning Theory, 1995.
Foundations of Machine Learning, 2nd edition, 2018.
Artificial Intelligence: A Modern Approach, 3rd edition, 2015.

Papers
Revisiting Self-Supervised Visual Representation Learning, 2019.
Active Learning Literature Survey, 2009.

Tutorials
Supervised and Unsupervised Machine Learning Algorithms
Why Do Machine Learning Algorithms Work on New Data?
How Machine Learning Algorithms Work Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 18/32
6/12/2021 14 Different Types of Learning in Machine Learning

Gentle Introduction to Transduction in Machine Learning

Never miss a tutorial:
A Gentle Introduction to Transfer Learning for Deep Learning
Ensemble Learning Methods for Deep Learning Neural Networks
               

Videos
Picked for you:
Yann LeCun @EPFL – “Self-supervised learning: could machines learn like humans?” 2018
Find Your Machine Learning Tribe
Articles
Supervised learning, Wikipedia.
Unsupervised learning,
What Is Holding Wikipedia.
You Back From Your
Reinforcement learning,
Machine Learning Wikipedia.
Goals?
Semi-supervised learning, Wikipedia.
Start Machine Learning ×
Multi-task learning, Wikipedia.
Difference
Multiple Between
instance Classification
learning, Wikipedia.and
You can master applied Machine Learning 
Regression in Machine Learning
Inductive reasoning, Wikipedia. without math or fancy degrees.
Deductive reasoning, Wikipedia. Find out how in this free and practical course.
Transduction (machine learning), Wikipedia.
Machine Learning for Developers
Active learning (machine learning), Wikipedia. Email Address
Online machine learning, Wikipedia.
Transfer learning, Wikipedia. I consent to receive information about
Ensemble learning,
Why Machine Wikipedia.
Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.
Summary
START MY EMAIL COURSE

In this post, you discovered a gentle introduction to the different types of learning that you may encounter in
Loving the Tutorials?
the field of machine learning.
The EBook Catalog is where
Specifically, you
you'll findlearned:
the Really Good stuff.

Fields of
>>study, such as
SEE WHAT'S supervised, unsupervised, and reinforcement learning.
INSIDE
Hybrid types of learning, such as semi-supervised and self-supervised learning.
Broad techniques, such as active, online, and transfer learning.

Do you have any questions?

Ask your questions in the comments below and I will do my best to answer.

Tweet Share Share

About Jason Brownlee

Jason Brownlee, PhD is a machine learning specialist who teaches developers how to get results with
modern machine learning methods via hands-on tutorials.
View all posts by Jason Brownlee →

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 19/32
6/12/2021 14 Different Types of Learning in Machine Learning

 A Gentle Introduction to Maximum a Posteriori (MAP) for Machine Learning

Never miss a tutorial:
How to Save a NumPy Array to File for Machine Learning 

               

Picked for you:

68 Responses to 14 Different Types of Learning in Machine Learning
Find Your Machine Learning Tribe

REPLY 
Avram November 11, 2019 at 7:23 am #

What Is Holding You Back From Your

Hi Jason,
Machine Learning Goals?
Great post again.Thanks for it.
Are you planning to post about reinforcement learning examples and issues?
Start Machine Learning ×
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Jason Brownlee November 11, 2019 at 1:32 pm #Find out how in this free and practical course. REPLY 

Machine Learning for Developers

Thanks.
Email Address
Perhaps in the future. Why are you intersted in the topic?
I consent to receive information about
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy. REPLY 
Gareth November 11, 2019 at 8:47 pm #
START MY EMAIL COURSE
More practical texts on reinforcement learning would be a good thing. The material in this
Loving
space the Tutorials?
is either too specific on the theory or too much repetition on pre-existing open ai gym
environments.
The EBook Catalog is where
you'll find the Really Good stuff.

>> SEE WHAT'S INSIDE REPLY 

Jason Brownlee November 12, 2019 at 6:38 am #

Thanks.

The tech doesn’t look useful for anything except games, e.g. the gym, and robotics. That’s
probably why there are no useful examples.

Avram November 12, 2019 at 9:34 am #

I don’t agree with you.You can use RL in travel salesman problem,job

scheduling,vehicle routing problems etc.that has not discovered yet.So,I can see the topic
will be promising in lots of industrial areas in the future.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 20/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

Jason Brownlee November 12, 2019 at 2:02 pm #

            You

    can, but classical combinatorial optimization methods do these problems better
and faster. E.g. you would not use it “at work”, at least from what I can see.

Picked for you:

Find Your Machine Learning Tribe

Pierre September 2, 2020 at 11:52 am #

Concerning combinatorial optimization methods, what python libraries would you

recommend for typical engineering optimization problems? Do you have book or articles to
What Is Holding You Back From Your
recommend
Machine Learning on the subject?
Goals?
Thanks!
Start Machine Learning ×
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Jason Brownlee September 2, 2020 at 1:30 pm #
Find out how in this free and practical course.
Sorry,
Machine Learning for I don’t have tutorials on optimization, I hope to write about the topic in the
Developers
future. Email Address

This is a great book on function optimization:

I consent to receive information about
https://amzn.to/31J3I8l
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.

START MY EMAIL COURSE REPLY 

Prof Ravi Vadlamani November 16, 2019 at 1:57 am #
Loving the Tutorials?
Very good post!
The EBook Catalog is where
you'll find the Really Good stuff.

REPLY 
>> SEE WHAT'S
JasonINSIDE
Brownlee November 16, 2019 at 7:27 am #

Thanks!

REPLY 
roberto November 20, 2019 at 10:36 pm #

I’m also interested in it, specifically for the time series.

REPLY 
Souleymane THIAM January 4, 2020 at 11:30 pm #

Very important post.

thank you a lot dear teacher Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 21/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

REPLY 
        Jason
    Brownlee
    January 5, 2020 at 7:05 am #

Thanks.
Picked for you:

Find Your Machine Learning Tribe

REPLY 
Avram November 11, 2019 at 7:19 pm #

I think,in the near future,this topic and supervised and unsupervised models will be gathered to
What Is Holding You Back From Your
form strong AI.Knowing it may be well for anybody who is interested in AI.
Machine Learning Goals?

Start Machine Learning ×

Difference Between Classification and REPLY 
Jason Brownlee November 12, 2019 at 6:36 am #You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Thanks for sharing. Find out how in this free and practical course.

Machine Learning for Developers

Email Address
REPLY 
marco November 12, 2019 at 6:53 pm #
I consent to receive information about
Why Machine Learning Does Not Have to services and special offers by email. For more
Hello Jason,
Be So Hard
question about GAN. information, see the Privacy Policy.
Is possibile to implement a GAN only with KERAS? Is possibile with Sklearn as well?
START MY EMAIL COURSE
Do you have an easy example to better undestand and try GAN?
What areLoving
possiblethe
fields of application of GAN?
Tutorials?
Thanks,
Marco The EBook Catalog is where
you'll find the Really Good stuff.

>> SEE WHAT'S INSIDE

REPLY 
Jason Brownlee November 13, 2019 at 5:38 am #

Yes, you can implement GANs with Keras.

I have many examples, start here:

https://machinelearningmastery.com/start-here/#gans

REPLY 
Savan Vachhani November 13, 2019 at 12:53 am #

Hi Jason,

I am loving your website and all the work you have done for a beginner. A quick non-technical question
though – why there isn’t much stuff that uses Google’s Colab?
StartAny particular
Machine reason?
Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 22/32
6/12/2021 14 Different Types of Learning in Machine Learning

Thanks
Never miss a tutorial:

               

REPLY 
Jason Brownlee November 13, 2019 at 5:47 am #
Picked for you:
Thanks!
Find
I try Your agnostic
to stay Machine Learning Tribe like google/ms/amazon. Run the code wherever you like.
from platforms

What Is Holding You Back From Your REPLY 

Savan Vachhani November 22, 2019 at 9:32 pm #
Machine Learning Goals?

Hi Jason, thank you for your reply. I get what you mean now that I have dabbled in the field
Start Machine Learning ×
for some time.
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee November 23, 2019 at 6:50 am #
Machine Learning for Developers
Email Address
Thanks.

I consent to receive information about

Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard REPLY 
Patryk Golabek November 15, 2019 at 6:08 am # information, see the Privacy Policy.

START MY EMAIL COURSE

This is a really nice post. I think the best one so far and I do read all of them.
I’m stealing this one for our internal wiki.
Loving the Tutorials?

The EBook Catalog is where

you'll find the Really Good stuff.
REPLY 
Jason Brownlee November 15, 2019 at 7:57 am #
>> SEE WHAT'S INSIDE
Thanks!

REPLY 
Hammid Olanrewaju Adeleke December 1, 2019 at 3:34 am #

Dear Author,

Great post from you as usual.

Please I need you to shed more light on other techniques of adding attributes to data sets apart from
Genetic Algorithm (And perhaps the better or the best among them based on literatures).
Also how is Hybridization of algorithms achieved or implemented, for example in WEKA.

Thank you whilst awaiting your response.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 23/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial: REPLY 

Jason Brownlee December 1, 2019 at 5:43 am #

            Thanks!

   

I believe you are describing feature engineering. Perhaps this will give you ideas:
Picked for you:
http://machinelearningmastery.com/discover-feature-engineering-how-to-engineer-features-and-
how-to-get-good-at-it/
Find Your Machine Learning Tribe

REPLY 
Nordin November 15, 2019 at 9:56 am #
What Is Holding You Back From Your
Machine Learning Goals?
Great post!
Start Machine Learning ×
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning REPLY 
Jason Brownlee November 16, 2019 at 7:15 am #without math or fancy degrees.
Find out how in this free and practical course.
Thanks.
Machine Learning for Developers
Email Address

I consent to receive information about REPLY 

Leonardo November 17, 2019 at 9:55 am #
Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
Hi Jason, Nice post. information, see the Privacy Policy.
I have one question about another kind of learning, http://ai.stanford.edu/~hllee/icml07-
START MY EMAIL COURSE
selftaughtlearning.pdf, self-taught learning created by Andrew Ng. Maybe you didn’t know it or it wasn’t
relevant?
Loving the Tutorials?
Whats your opinion?
The EBook Catalog is where
Thanksyou'll find the Really Good stuff.

>> SEE WHAT'S INSIDE

REPLY 
Jason Brownlee November 18, 2019 at 6:41 am #

Thanks!

Yes, I believe this is “self-supervised learning” in the above post.

REPLY 
Tsang November 18, 2019 at 1:45 pm #

Hi nice post, what about these other ones:

Federated Learning https://ai.googleblog.com/2017/04/federated-learning-collaborative.html

Curriculum Learning https://ronan.collobert.com/pub/matos/2009_curriculum_icml.pdf

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 24/32
6/12/2021 14 Different Types of Learning in Machine Learning

Confident Learning https://l7.curtisnorthcutt.com/confident-learning

Never miss a tutorial:
Weakly supervised Learning
https://pdfs.semanticscholar.org/3adc/fd254b271bcc2fb7e2a62d750db17e6c2c08.pdf
               

Picked for you:

REPLY 
Jason Brownlee November 18, 2019 at 1:48 pm #
Find Your Machine Learning Tribe

Very nice, thanks for sharing!

What Is Holding You Back From Your

Machine Learning Goals? REPLY 
Jewel Ahmed November 28, 2019 at 5:53 pm #
Start Machine Learning ×
Thank you for sharing.
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jason Brownlee November 29, 2019 at 6:44 am #
Machine Learning for Developers
Email Address
You’re welcome.

I consent to receive information about

Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard REPLY 
EDDISON HAYDEN LEWIS December 29, 2019 at information,
3:26 am # see the Privacy Policy.

START MY EMAIL COURSE

The article is very insightful as it indicates the all relevant paradigms of machine learning in
practice. But, “What about the prospect of Auto Machine Learning?”
Loving the Tutorials?

The EBook Catalog is where

you'll find the Really Good stuff.
REPLY 
Jason Brownlee December 29, 2019 at 6:08 am #
>> SEE WHAT'S INSIDE
Thanks.

Good suggestion!

REPLY 
Rajesh Chukka December 31, 2019 at 1:24 am #

Hi nice post ,Very insightful

also add the meta learning to the stack ..’Learning to Learn’ is currently hottest research areas in deep
learning

REPLY 
Jason Brownlee December 31, 2019 at 7:33 am #Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 25/32
6/12/2021 14 Different Types of Learning in Machine Learning

Great suggestion! Thanks.

Never miss a tutorial:

               

REPLY 
Jack Rief February 18, 2020 at 7:50 pm #
Picked for you:
Hi Jason.
You posted greatMachine
Find Your concepts on Artificial
Learning Tribe Intelligence. I am happy to know that there is someone who is
posting blogs regarding Artificial Intelligence. You properly described about the problems and techniques
occur in machine work of Artificial Intelligence. You should post some more blogs related to Artificial
Intelligence.
What Is Holding You Back From Your
Machine Learning Goals?

Start Machine Learning ×

REPLY 
Jason Brownlee February 19, 2020 at 8:00 am #
Difference Between Classification and
You can master applied Machine Learning 
Thanks,
Regression but it isLearning
in Machine machine learning, not AI. E.g. a subfield of AI.
without math or fancy degrees.
Find out
AI is not really useful to the average developer. Predictive how in with
modeling this free and
machinepractical course.
learning is.

Machine Learning for Developers

Email Address
REPLY 
Maryam April 3, 2020 at 8:54 am #
I consent to receive information about
Why Machine Learning Does Not Have to services and special offers by email. For more
Dear Jason,
Be So Hard
thanks for your explanation of “Multi-Task learning”. information, see the Privacy Policy.
But I just want you to give me your tutorial about implementing “auxiliary loss” in that.
START MY EMAIL COURSE
Waiting for your reply.

Best Loving the Tutorials?

MaryamThe EBook Catalog is where
you'll find the Really Good stuff.

>> SEE WHAT'S INSIDE

REPLY 
Jason Brownlee April 3, 2020 at 9:41 am #

Thanks for your suggestion.

REPLY 
Ravish Qureshi May 6, 2020 at 4:34 pm #

This post just saved me hours of googling and time! Awsome post Jason.

REPLY 
Jason Brownlee May 7, 2020 at 6:41 am #

Thanks, I’m happy to hear that! Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 26/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

REPLY 
   
A   Rai May
   
16, 2020 at 3:38 am #
   

Hi Jason,
Picked for you:
your website helps me alot in my PhD. Thank you very much for sharing.

Find Your Machine Learning Tribe

REPLY 
Jason Brownlee May 16, 2020 at 6:18 am #

What IsYou’re
Holding You BackI’m
welcome, From Yourthat it helps!
happy
Machine Learning Goals?

Start Machine Learning ×

Difference Between Classification and REPLY 
Star June 5, 2020 at 1:58 pm # You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Hi Jason; Find out how in this free and practical course.
Thanks for this interesting post, I was just wondering if you show us an example of reinforcement learning?
What isMachine
RL usedLearning for than
for other Developers
games? How do we know if it is a good idea to apply it for our set of data?
Email Address

I consent to receive information about

Why Machine Learning Does Not Have to services and special offers by email. For moreREPLY 
Jason Brownlee June 6, 2020 at 7:41 am #
Be So Hard
information, see the Privacy Policy.
Good question, I answer it here:
START MY EMAIL COURSE
https://machinelearningmastery.com/faq/single-faq/do-you-have-tutorials-on-deep-reinforcement-
learning
Loving the Tutorials?

The EBook Catalog is where

you'll find the Really Good stuff.
REPLY 
Gemma Almendros June 14, 2020 at 7:01 pm #
>> SEE WHAT'S INSIDE
Hi Jason, im the girl that asked you about a work from University on other post.
I’d like to know if Multitask learning could be linear or if it has some types within and if it’s used just for
supervised problems or other problems?

Gemma.

REPLY 
Jason Brownlee June 15, 2020 at 6:04 am #

Multi-task learning could use linear or nonlinear methods.

Yes, multi-task learning could be further divided into sub-types of problems, no doubt.

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 27/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial: REPLY 

Karl Johan Haarberg June 24, 2020 at 11:21 pm #

    Great
    post,
    Jason.
   

I would recommend making a distinction between shallow and deep learning. Given your relatively
Picked for you: list of different types of learning in ML, you might consider introducing extended analytics
comprehensive
(i.e. utilising a combination of several different AI, ML, and DL techniques = augmented/virtual/mixed
Find Your Machine Learning Tribe
analytics) wrt. learning. I believe that e.g. linear/non-linear, stochastic/ deterministic, regular/shrinkage, etc.
fall more into the art and technique of feature engineering. Finally, given the tremendous work in the
research frontier on XAI (and commercial AI), I would also add explainable learning.

Best regards,
What Is Karl Johan
Holding You Back From Your
Machine Learning Goals?

Start Machine Learning ×

Difference Between Classification REPLY 
Jason Brownlee June 25,and
2020 at 6:20 am #
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Great suggestions, thanks Karl!
Find out how in this free and practical course.

Machine Learning for Developers

Email Address
REPLY 
Daniel Kapitan July 4, 2020 at 5:00 pm #
I consent to receive information about
WhyExcellent
Machinepost, Jason!
Learning I am
Does NotinHave
the process
to of creating a tree
services andofspecial
machine learning
offers algorithms
by email. For morefor
Be So Hard
educational purposes (see https://kumu.io/dkapitan/tree-of-machine-learning-algorithms/ , it’s work in
information, see the Privacy Policy.
progress) and I am trying to figure out how to map types of learning.
START MY EMAIL COURSE
One specific question I have is where to position feature / representational learning in all of this. It seems a
major concept
Lovingfor the
e.g. Tutorials?
textmining and image classification. Problem is find is that as you go down the tree, it
becomes a bit convoluted since many different types of learning are combined.
The EBook Catalog is where
Any thoughts orthe
you'll find suggestions arestuff.
Really Good most welcome!

>> SEE WHAT'S INSIDE

REPLY 
Jason Brownlee July 5, 2020 at 6:59 am #

That sounds like a fun project!

Feature selection and representation learning like autoencoders are a type of data preparation.

REPLY 
Maliha August 24, 2020 at 2:25 am #

Hi jason, Is there any blog post or tutorial related to Adversaries Attack in DNN?

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 28/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial: REPLY 

Jason Brownlee August 24, 2020 at 6:29 am #

            Not at

    this stage, thanks for the suggestion.

Picked for you:

REPLY 
Richard
Find Griffin
Your Machine August 19,
Learning 2020 at 1:09 am #
Tribe

This is a super précis on the overall field! Thank you!!

What Is Holding You Back From Your

Machine Learning Goals?
REPLY 
Jason Brownlee August 19, 2020 at 6:03 am #
Start Machine Learning ×
Thanks!
Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.
REPLY 
Jaideep
Machine Adhvaryu
Learning September 1, 2020 at 6:09 am #
for Developers
Email Address
Hi,

This post is a great read but has completely confused me. I consent to receive information about
Why Machine Learning Does Not Have to services and special offers by email. For more
1. are all
Be these 14 techniques mutually exclusive
So Hard
information, see the Privacy Policy.
2. especially why only 3 (transduction, active and online learning) are called machine learning
3. How to understand learning problems and learning techniques START MY EMAIL COURSE

Jason, i think this is a very important post but would request you to simplify for the newbies like me.
Loving the Tutorials?
Thanks!
The EBook Catalog is where
you'll find the Really Good stuff.

>> SEE WHAT'S INSIDE REPLY 

Jason Brownlee September 1, 2020 at 6:39 am #

No, not at all.

I recommend starting off by focusing on supervised learning and this process:

https://machinelearningmastery.com/start-here/#getstarted

REPLY 
titou September 5, 2020 at 1:40 am #

life-long learning

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 29/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial: REPLY 

Jason Brownlee September 5, 2020 at 6:51 am #

        Thanks!

       

Picked for you:

REPLY 
Subhash
Find Achutha
Your Machine September
Learning Tribe 20, 2020 at 11:30 am #

sir can give feedback to this

https://github.com/achuthasubhash/Complete-Life-Cycle-of-a-Data-Science-Project
What Is Holding You Back From Your
if possible mail Learning
Machine to me regarding
Goals? any [email protected]

Start Machine Learning ×

Difference Between Classification and
You can master applied Machine Learning  REPLY 
Jason
Regression Brownlee
in Machine Learning
September 20, 2020 at 1:34 pm #
without math or fancy degrees.

I’m happy to answer specific questions aboutFind out how

machine in this free and
learning, practical course.
but I don’t have the capacity to
review/debug code/data/projects:
Machine Learning for Developers
Email Address
https://machinelearningmastery.com/faq/single-faq/can-you-read-review-or-debug-my-code

I consent to receive information about

Why Machine Learning Does Not Have to services and special offers by email. For more
REPLY 
BeSubhash
So Hard Achutha September 20, 2020 at 2:23 pm #
information, see the Privacy Policy.

ok thank you START MY EMAIL COURSE

Loving the Tutorials?

The EBook Catalog is where REPLY 

Srilalitha Veerubhotla October 4, 2020 at 11:31 am #
you'll find the Really Good stuff.

Hello Sir,
>> SEE WHAT'S INSIDE
This article has very much covered. You can also add federated, curriculum, and confident learning
techniques.

Thank you,

REPLY 
Jason Brownlee October 4, 2020 at 2:58 pm #

Thanks!

REPLY 
Shiva Shrestha April 20, 2021 at 2:48 am #
Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 30/32
6/12/2021 14 Different Types of Learning in Machine Learning

What about association problems in unsupervised learning?

Never miss a tutorial:

               

REPLY 
Jason Brownlee April 20, 2021 at 6:01 am #
Picked for you:
Good question.
Find Your Machine Learning Tribe

Leave a Reply
What Is Holding You Back From Your
Machine Learning Goals?

Start Machine Learning ×

Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

Machine Learning for Developers

Email Address

Name (required) I consent to receive information about

Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.
Email (will not be published) (required)
START MY EMAIL COURSE

Loving the Tutorials?

Website

The EBook Catalog is where

you'll find the Really Good stuff.
SUBMIT COMMENT
>> SEE WHAT'S INSIDE

Welcome!
I'm Jason Brownlee PhD
and I help developers get results with machine learning.
Read more

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 31/32
6/12/2021 14 Different Types of Learning in Machine Learning

Never miss a tutorial:

               

Picked for you:

Find Your Machine Learning Tribe

What Is Holding You Back From Your

Machine Learning Goals?

Start Machine Learning ×

Difference Between Classification and
You can master applied Machine Learning 
Regression in Machine Learning
without math or fancy degrees.
Find out how in this free and practical course.

Machine Learning for Developers

Email Address

I consent to receive information about

Why Machine Learning Does Not Have to services and special offers by email. For more
Be So Hard
information, see the Privacy Policy.

START MY EMAIL COURSE

Loving the Tutorials?

The EBook Catalog is where

you'll find the Really Good stuff.
© 2021 Machine Learning Mastery Pty. Ltd. All Rights Reserved.
LinkedIn | Twitter | Facebook | Newsletter | RSS
>> SEE WHAT'S INSIDE
Privacy | Disclaimer | Terms | Contact | Sitemap | Search

Start Machine Learning

https://machinelearningmastery.com/types-of-learning-in-machine-learning/ 32/32