Chapter 2

Machine Learning(ML)
Part I
Introduction : What is Machine Learning?
❖ There is no universally accepted definition for machine learning.
❖ Was first coined by Arthur Lee Samuel a pioneer in the AI field, in 1959

❖ It is:-
• Study of algorithms that
• improve their performance
• at some task
• with experience
❖ Enable computers to learn without being explicitly
programmed(Arthur Samuel, in 1959 at IBM)
Introduction : What is Machine Learning?

❖ ML is concerned with the question of how to construct

computer programs that automatically improve with experience
using Model(We will discuss this later)

❖ A computer program which learns from experience is called a

machine learning program or simply a learning program. Such a
program is sometimes also referred to as a learner.
Introduction : Why “Learn” ?
What We Talk About When We Talk About“Learning”
❖ Learning is gaining Knowledge from experience.

❖ A computer program is said to learn from

experience E with respect to some class of tasks
T and performance measure P, if its performance
at tasks T, as measured by P, improves with
experience E.
Introduction : Why “Learn” ?
❖ Examples: i) Handwriting recognition learning problem
• Task T: Recognizing and classifying handwritten words within
images
• Performance P: Percent of words correctly classified
• Training experience E: A dataset of handwritten words with
given classifications
ii) A robot driving learning problem
• Task T: Driving on highways using vision sensors
• Performance measure P: Average distance traveled before an
error
• Training Experience: A sequence of images and steering
commands recorded while observing a human driver
Introduction : Why “Learn” ?
❖ The reason why we develop computer is to
automate various kinds of processes.

❖ The goals behind machine learning was to replace

the need for developing computer programs
“manually.

❖ We can think of machine learning as the process of

“automating automation.
Introduction : Why “Learn” ?
Learning is used when:
❖Human expertise does not exist (navigating
on Mars),
❖Humans are unable to explain their expertise
(speech recognition)
❖Solution changes in time (routing on a
computer network)
❖Solution needs to be adapted to particular
cases (user biometrics)
 Introduction
Machine learning vs. “classic(Traditional)” programming.
How machines learn?
Basic components of learning process
The learning process, whether by a human or a machine, can be divided
into four components, namely:
1. Data storage: for storing and retrieving huge amounts of data
2. Abstraction: process of extracting knowledge about stored data.
creating general concepts about the data as a whole, creation of
new models.
3. Generalization: the process of turning the knowledge about stored
data into a form that can be utilized for future action. The goal is to
discover the most relevant properties of data used for future tasks.
4. Evaluation: the process of giving feedback to the user to measure
the utility of the learned knowledge.
How machines learn?
Basic components of learning process
Growth of Machine Learning
❖ Machine learning is preferred approach to
• Speech recognition, Natural language processing
• Computer vision
• Medical outcomes analysis
• Robot control
• Computational biology
❖ This trend is accelerating
• Improved machine learning algorithms
• Improved data capture, networking, faster computers
• Software too complex to write by hand
• New sensors / IO devices
• Demand for self-customization to user, environment
Types of machines learn?
In general, ML algorithms can be classified into 3 types.

1. Supervised Learning
• Classification
• Regression/Prediction
2. Unsupervised Learning
• Clustering
• Dimensionality Reduction
3. Reinforcement Learning
Supervised learning
❖ It is the process of algorithm learning from the training dataset.
❖ It can be thought of as a teacher supervising the learning process.
❖ The algorithm iteratively makes predictions on the training data and is
corrected(there is correct outputs) by the teacher.
❖ It is in which a machine learns from known datasets (set of training
examples), and then predicts the output.

❖ It is learning from labeled training data (i.e., inputs that also contain
the desired outputs or targets; basically, “examples” of what we want
to predict).
❖ Learning stops when the algorithm achieves an acceptable level of
performance.
Supervised learning
Supervised learning
Supervised learning: Classification
❖Classification deals with categorizing several instances
in one of the predefined categories/classes.
❖It specifies the class to which data elements belong to.

❖It predicts a class for an input variable as well.

▪ For example, classifying the weather as sunny, cloudy, or rainy.

Supervised learning: Classification
Example1: Here the class label is categorical.
Supervised learning: Classification
Description:
❖ Each training input xi is a D-dimensional vector of
numbers, representing, say, the age, ed, employ, address,
income, debtinc, creddebt and othdebt of a person.
❖ These are called features, attributes, dimensions, predictor
variable, independent variables, columns or covariates.
❖ In general, however, xi could be a complex structured
object, such as an image, a sentence, an email message, a
time series, a molecular shape, a graph, etc
Supervised learning: Classification
Example 2: Consider the following data for a classification problem

❖ Data in Table is the training set of data.

❖ There are two attributes “Score1” and “Score2”.
❖ The class label is called “Result”.
❖ The class label has two possible values “Pass” and “Fail”.
❖ The data can be divided into two categories or classes: The set
of data for which the class label is “Pass” and the set of data for
which the class label is“Fail”
Problem ?
Let us assume that we have no knowledge
about the data other than what is given in the
table. Now, the problem can be posed as
follows: If we have some new data, say “Score1
= 25” and “Score2 = 36”, what value should be
assigned to “Result” corresponding to the new
data; in other words, to which of the two
categories or classes the new observation
should be assigned? See this Figure for a
graphical representation of the problem
From Graphical representation of data in Table Solid dots represent data in “Pass” class and hollow dots data
in “Fail” class. The class label of the square dot is to be determined. To answer this question, using the given
data alone we need to find the rule, or the formula, or the method that has been used in assigning the
values to the class label “Result”. The problem of finding this rule or formula or the method is the
classification problem. In general, even the general form of the rule or function or method will not be
known. So several different rules, etc. may have to be tested to obtain the correct rule or function or
method
 ❖ Real life examples ❖ Algorithm
There are several machine learning algorithms
• Face recognition for classification.
• Speech recognition The following are some of the well-known
• Medical diagnosis algorithms.
• Logistic regression
• Knowledge extraction
• Naive Bayes algorithm
• Compression • k-NN algorithm
• Decision tree algorithm
• Support vector machine algorithm
• Random forest algorithm
Supervised learning: Regression
❖ Regression is just like classification except the response variable is continuous.
❖ problem is the problem of predicting the value of a numeric variable based on
observed values of the variable.
❖ The value of the output variable may be a number, such as an integer or a
floating point value.
❖ These are often quantities, such as amounts and sizes.
❖ The input variables may be discrete or real-valued.
❖ Regression analysis consists of a set of machine learning methods that allow us
to predict a continuous outcome variable (y) based on the value of one or
multiple predictor variables (x).
Supervised learning: Regression
Regression can be Linear or Non Linear
❖ In linear regression, the data are modeled to fit a straight line as:
❖ y = wx+b, variable, y (called a response variable), can be modeled as a linear function
of another random variable, x (called a predictor variable) where the variance of y is
assumed to be constant. x and y are numeric database attributes.
❖ The coefficients, w and b (called regression coefficients), specify the slope of the line
and the y-intercept, respectively.
Supervised learning: Regression
In non-linear regression, the data can‘t be modelled to fit a straight line i.e.
modelled to fit a curve as shown in below figure.
Supervised learning: Regression
+
Linear regression models the relationship between a dependent variable and one or
more explanatory variables using a linear function.
Can be formulated as below: y = bo + b1*x1
Eg. Prediction of height based on age.
Feature(X): age
Class(y): height
Regression equation will be: height = w*age + b
If two or more explanatory variables have a linear relationship with the dependent
variable, the regression is called a multiple linear regression, which can be formulated
as below: y = bo + b1*x1 + b2*x2 .. + bn*xn
Reading. Differentiate Simple linear regression and Multi linear regression.
Unsupervised Learning

❖Unsupervised machine learning is the process of inferring

underlying hidden patterns from historical data.
❖ Within such an approach, a machine learning model tries to
find any similarities, differences, patterns, and structure in
data by itself.
❖used when the information used to train is neither classified
nor labeled.
❖There is no complete and clean labelled dataset.
❖ No prior human intervention is needed.
Unsupervised Learning
+
The system doesn‘t figure out the right output, but it
explores the data and can draw inferences from
datasets to describe hidden structures from unlabeled
data.
.
Unsupervised Learning: Clustering

❖ Clustering automatically
categorizes data into groups
according to similarity criteria
❖ It evaluates the similarity
based on a metric like
Euclidean distance, Cosine
similarity, Manhattan distance
Unsupervised Learning: Clustering

Example Algorithms developed to implement

Clustering
K-mean Clustering
2. Hierarchical Clustering

Read these algorithm how the work.

Unsupervised Learning: Dimensionality reduction

❖ In many learning problems, the datasets have large number of

variables. Sometimes, the number of variables is more than the
number of observations.
For example, such situations have arisen in many scientific
fields such as image processing, mass spectrometry, time
series analysis, internet search engines, and automatic text
analysis among others.

❖ Statistical and machine learning methods have some difficulty when

dealing with such high-dimensional data.
❖ Normally the number of input variables is reduced before the
machine learning algorithms can be successfully applied.
Unsupervised Learning: Dimensionality reduction

❖ In statistical and machine learning, dimension reduction is the

process of reducing the number of variables under
consideration by obtaining a smaller set of principal variables
❖ It addresses the number of attributes of the dataset by
transforming it from its original representation to one with a
reduced set of features.
❖ goal is to obtain a new dataset that preserves, up to a level,
the original structure of the data, so its analysis will result in
the same or equivalent patterns present in the original data.
Unsupervised Learning: Dimensionality reduction

❖ Two kinds of methods for reducing the attributes in a dataset:

feature selection and feature extraction.
1. Feature selection
❖ Is interested in finding k of the total of n features that give us the most
information and we discard the other (n−k) dimensions.
❖ By only keeping the most relevant variables from the original dataset
2. Feature Extraction
Transforming the space containing many dimensions into space with fewer
dimensions.
Is used when we keep the whole information but use fewer resources while
processing the information.
Unsupervised Learning: Dimensionality reduction

Curse of dimensionality
Refers to the challenges that arise when working with high-dimensional data.
So
❖ High-dimensional data is challenging to handle.
❖ More features increase model complexity and risk of
overfitting.
Therefore, Overfitting leads to poor performance on
new data.
Unsupervised Learning: Dimensionality reduction

The main drawbacks of high-dimensional datasets are

•Increased data requirements: More records are needed to

represent all feature combinations.
•Overfitting risk: More features can lead to overly complex
models that fit to outliers.
•Longer training times: Higher dimensionality increases
computational complexity, slowing training.
•Higher storage needs: Larger datasets consume more storage
space.
Question
1. We are given information on user’s credit card transactions. We would like to detect
whether some of the transactions are fraudulent by finding some transactions that are
different from the other transactions. We have no information on whether any particular
transaction is fraudulent or not. Is this a supervised or unsupervised learning problem?
(A) Supervised learning
(B) Unsupervised learning

2. Is the following problem classification or regression? You are given historical data on the
price of a house at several points in time. You want to predict the price of this house next
month.
(A) Classification
(B) Regression
C) This is not supervised learning