Basics of Machine Learning and

Classifications

Dr. Helal Uddin Ahmed

PhD in Computer Engineering (South Korea), MBA (UK), MSc (DU), BSc.(DU)
Professor
Department of MIS
University of Dhaka
Mob. 01783368720
[email protected]
 What is Machine Learning?

• Machine Learning is a system that can learn from example through

self-improvement and without being explicitly coded by
programmer.
• a machine can singularly learn from the data (i.e., example) to
produce accurate results.
• Machine learning combines data with statistical tools to predict an
output
• Machine learning is closely related to data mining and Bayesian
predictive modeling.
• The machine receives data as input, use an algorithm to formulate
answers.
• A typical machine learning tasks are to provide a recommendation.
– For those who have a Netflix account, all recommendations of movies or
series are based on the user's historical data.
 How does Machine learning work?

• The way the machine learns is similar to the

human being. Humans learn from experience.
The more we know, the more easily we can
predict.
• By analogy, when we face an unknown
situation, the likelihood of success is lower
than the known situation.
• Machines are trained the same. To make an
accurate prediction, the machine sees many
examples.
 How does Machine learning work?

• The core objective of machine learning is

the learning and inference.
• First of all, the machine learns through the
discovery of patterns.
• This discovery is made from the data.
• The list of attributes used to solve a problem is
called a feature vector.
•  You can think of a feature vector as a subset
of data that is used to tackle a problem.
 Learning Phases

• The machine uses some algorithms and real

situation transform into model
 Process (1): Model Construction

Classification
Algorithms
Training
Data

NAME RANK YEARS TENURED Classifier

Mike Assistant Prof 3 no (Model)
Mary Assistant Prof 7 yes
Bill Professor 2 yes
Jim Associate Prof 7 yes IF rank = ‘professor’
Dave Assistant Prof 6 no
OR years > 6
Anne Associate Prof 3 no
THEN tenured = ‘yes’
6
Process (2): Using the Model in Prediction

Classifier

Testing
Data Unseen Data

(Jeff, Professor, 4)
NAME RANK YEARS TENURED
Tom Assistant Prof 2 no Tenured?
Merlisa Associate Prof 7 no
George Professor 5 yes
Joseph Assistant Prof 7 yes
7
 Types of Machine learning

• Association
• Supervised Learning
– Classification
– Regression
• Unsupervised Learning
• Reinforcement Learning
 Association

• A typical example of association rule mining is

market basket analysis.
• This process analyzes customer buying habits
– by finding associations between the different items
that customers place in their shopping baskets
• The discovery of such associations can help
retailers develop marketing strategies
– by gaining insight into which items are frequently
purchased together by customers.
 Association
• For instance, if customers are buying milk, how
likely are they to also buy bread (and what kind
of bread) on the same trip to the supermarket?
• Such information can lead to increased sales by
helping retailers to do selective marketing and
plan their shelf space.
• For instance, placing milk and bread within
close proximity may further encourage the sale
of these items together within single visits to
the store.
 Classification
Supervised learning (classification)

• Supervision: The training data (observations, measurements, etc.) are

accompanied by labels indicating the class of the observations
• New data is classified based on the training set
Classification
• predicts categorical class labels
• classifies data (constructs a model) based on the training set and the
values (class labels) in a classifying attribute and uses it in classifying new
data

Typical applications
• Credit/loan approval:
• Medical diagnosis: if a tumor is cancerous or benign
• Fraud detection: if a transaction is fraudulent
• Web page categorization: which category it is
 Classification

Supervised learning (classification)

• Model construction: describing a set of predetermined classes
– Each tuple/sample is assumed to belong to a predefined class, as
determined by the class label attribute
– The set of tuples used for model construction is training set
– The model is represented as classification rules, decision trees, or
mathematical formulae
• Model usage: for classifying future or unknown objects
– Estimate accuracy of the model
• The known label of test sample is compared with the classified result
from the model
• Accuracy rate is the percentage of test set samples that are correctly
classified by the model
• Test set is independent of training set
– If the accuracy is acceptable, use the model to classify new data
• Note: If the test set is used to select models, it is called validation (test) set
 Decision Tree Induction: An Example
age income student credit_rating buys_computer
<=30 high no fair no
 Training data set: Buys_computer <=30 high no excellent no
 The data set follows an example of 31…40 high no fair yes
>40 medium no fair yes
Quinlan’s ID3 (Playing Tennis) >40 low yes fair yes
 Resulting tree: >40 low yes excellent no
31…40 low yes excellent yes
age? <=30 medium no fair no
<=30 low yes fair yes
>40 medium yes fair yes
<=30 medium yes excellent yes
<=30 overcast
31..40 >40 31…40 medium no excellent yes
31…40 high yes fair yes
>40 medium no excellent no

student? yes credit rating?

no yes excellent fair

no yes no yes
14
 Clustering

• Unsupervised learning (clustering)

– The class labels of training data is unknown
– Given a set of measurements, observations, etc. with the
aim of establishing the existence of classes or clusters in
the data
• Clustering: Grouping similar instances
• Example applications
– Customer segmentation in CRM
– Image compression: Color quantization
– Bioinformatics: Learning motifs
 Reinforcement learning
• Reinforcement learning trains an actor or agent to respond
to an environment in a way that maximizes some value,
usually by trial and error. That’s different from supervised
and unsupervised learning
• Imagine you are supposed to cross an unknown field in the
middle of a pitch black night without a torch. There can be
pits and stones in the field, the position of those are
unfamiliar to you. There's a simple rule - if you fall into a
hole or hit a rock, you must start again from your initial
point.
• You start walking forward blindly, only counting the
number of steps you take. After x steps, you fall into a pit.
Your reward was x points since you walked that many steps.
 Reinforcement learning
• You start again from your initial position, but after x steps,
you take a detour either left/right and again move
forward. You hit a stone after y steps. This time your
reward was y which is greater than x. You decide to take
this path again but with more caution.
• When you start again, you make a detour after x steps,
another after y steps and manage to fall into another pit
after z steps. This time the reward was z points which was
greater than y, and you decide that this is a good path to
take again.
• You restart again, make the detours fter x, y and z steps to
reach the other side of the field. Thus, you've learned to
cross the field without the need of light
 Reinforcement learning

• Applications:
– Game playing
– Robot in a maze