Virtual Environment

Hope_Artificial_Intelligence:HAI 1
Purpose

Hope_Artificial_Intelligence:HAI 2
Ramisha Rani K, AI Trainer

Key Terms
Mrs. Ramisha Rani K, AI Trainer.

www.hopelearning.net Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer

KEY TERMS
Key Dataset
Term
01 The information about an event
Key Variable
Term
02 Input Features | Attributes

Population
Key
Term Entire Values of a Variable
03
Sample
Key
Term Sample of data taken from population dataset which should
04 represent the properties of population

Key Statistic
Term Finding the value for sample dataset.(E.g., Mean)
05

www.hopelearning.net Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
KEY TERMS

Key Parameter
Term
06 Statistics which gives value for population
Key Data
Term
07 Values of variable

www.hopelearning.net Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
Key Terms
Dataset The information about an event

www.hopelearning.net Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
Key Terms
Variable Input Features

www.hopelearning.net Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
Key Terms

Data Value of the dataset

www.hopelearning.net Hope Artificial Intelligence:HAI @hope_artificial_intelligence

Ramisha Rani K, AI Trainer
Key Terms
Entire values of a Variable or Entire
Population Dataset

www.hopelearning.net Hope Artificial Intelligence:HAI @hope_artificial_intelligence

#learnaiwithramisha

Do you think Artificial Intelligence is only

for Computer Science Department?

l EEE Wire
l
M echanica Netw ess
ork
Ph
CE ysi
E cs

Ch
vil

em
Ci

ist
ry
tics

His
ma

tor
the

y.
Ma

www.hopelearning.net @hope_artificial_intelligence
#learnaiwithramisha
Artificial Intelligence

Past data Algorithm Output

Heart SVM
Disease KNN Final
Forest Fire … Model
…

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 11
#learnaiwithramisha

Types of Problem Statement in in Artificial Intelligence

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 12
#learnaiwithramisha

Supervised
Learning
Past Data/input Data

Input/Variables/Feature

Output/label

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 13
#learnaiwithramisha

Take way for Supervised Learning

Requirement should be clear.

Input and Output are well defined.

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 14
 SCREENING-1
#learnaiwithramisha

Types of Problem Statement in Artificial Intelligence

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 15
#learnaiwithramisha
Unsupervised
Learning
Past Data/input Data

Input/Variables/Feature

@hope_artificial_intelligence
www.hopelearning.net Machine Deep Learning(TM)
Hope_Artificial_Intelligence:HAI 16
#learnaiwithramisha
Unsupervised
Learning

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 17
#learnaiwithramisha

Take way for Unsupervised Learning

We don’t know what we need.

Only Input data, So we can do

Clustering.

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 18
#learnaiwithramisha

Types of Problem Statement in Artificial Intelligence

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 19
#learnaiwithramisha

Semi Supervised

Past Data/input Data

Input/Variables/Feature

Output/label

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 20
#learnaiwithramisha

Take way for Semi Supervised Learning

We know requirements

But Half of the output are not

labelled.

@hope_artificial_intelligence
www.hopelearning.net Machine Deep Learning(TM)
Hope_Artificial_Intelligence:HAI 21
#learnaiwithramisha

SCREENING - 2

Artificial Intelligence
Problem Identification

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 22
#learnaiwithramisha
Problem Identification on Supervised Learning
Classification
Classifying the output based on the input parameter.

Yes/No
Dog/ Cat
Categorical Value House/not house

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 23
#learnaiwithramisha
Problem Identification on Supervised Learning
Regression

Numerical Value

Hope_Artificial_Intelligence:HAI 24

@hope_artificial_intelligence
www.hopelearning.net
#learnaiwithramisha

See the Picture Tell a Story

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 25
#learnaiwithramisha

Linear Graph

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 26
#learnaiwithramisha

Multiple Linear

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 27
#learnaiwithramisha

Artificial Intelligence

Machine Learning
Deep Learning

Supervised Learning Unsupervised Learning Semi Supervised Learning

Regression Classification Clustering

 Simple Linear Regression  Logistic Classification  K-Means Clustering
 Multiple Linear Regression  K-Nearest Neighbor(K-NN)  Hierarchical Clustering
 Logistic Regression  Support Vector Machine (SVM)
 Polynomial Regression  Kernel SVM
 Support Vector Machine Regression  Naive Bayes
 Decision Tree Regression  Decision Tree Classification
 Random Forest Regression  Random Forest Classification

@hope_artificial_intelligence
Hope_Artificial_Intelligence:HAI 28
www.hopelearning.net
#learnaiwithramisha
Algorithms

Algorithm for both Regression Algorithm Classification Algorithm

Regression and Classification

Support Vector Machine Linear Algorithm

Logistic Algorithm

Multiple Linear Navies Bayes

Decision Tree Regression KNN

Random Forest Polynomial Regression

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 29
+91 95979 40880

Hope_Artificial_Intelligence:HAI 30
#learnaiwithramisha
Polynomial Graph

www.hopelearning.net Hope_Artificial_Intelligence:HAI 31
Image Source: http://www.egwald.ca/linearalgebra/polynomials.php @hope_artificial_intelligence
#learnaiwithramisha

 Take Away Concepts In Simple Linear Regression

 Why it is Simple Linear Regression?

 How this regression helps for future prediction?

 Validating Parameter:
 Sum of Square Error(SSE) or Residual Sum of Square(RSS)
 Sum of Square Regression(SSR) or Explained Sum of Square(ESS)
 Sum of Square Total(SST)
 R Squared(R2)
 Adjusted R Squared

 When to use simple linear regression?

www.hopelearning.net Hope_Artificial_Intelligence:HAI 32
@hope_artificial_intelligence
#learnaiwithramisha

 Why it is Simple Linear Regression?

It has one input(X/Independent) and one output (y/Dependent Variable/Response variable).

It uses the straight line equation y= mX+c

Where,
y= output(which forms straight line by adding all data points)

m= slope= dy/dx= weight (which says about constant distance between two data points)

X= Input( if input changes, then output will also change accordingly)

c= bias= intercept= starting of the straight line= initial value= minimum value.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 33
@hope_artificial_intelligence
#learnaiwithramisha How this regression helps for future prediction?
SIMPLE LINEAR REGRESSION

Dataset

Predicted Value X Y
10
Y=2X+0 1 2
Y Dependant Variable 8 2 4
6 3 6
4 8
4
w 5 10

2 Initial Value(b)/Minimum value/ Origin

0 3 5
1 2 4
Y =wX +b X Independent Variable
y)( n (∑ 𝑥𝑦 ¿−(∑ x ) ( ∑ 𝑦)
W= Slope= ___________________ B= Bias=Initial Value=Minimum Value= ___________________
( (

www.hopelearning.net Hope_Artificial_Intelligence:HAI 34
@hope_artificial_intelligence
#learnaiwithramisha
Types of scattered data with Linear Regression line

D.V D.V

Actual Value I.V

I.V
Predicted Value

D.V
D.V

I.V
I.V

www.hopelearning.net Hope_Artificial_Intelligence:HAI
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: 1 .Sum of Square Error(SSE) or Residual Sum of Square(RSS)

Formula:
yi

10 Error
−
−Y Error= Actual Value(yi) – Predicted Value(yi)
Y Dependant Variable

8
Where, i = Observation point
6
n= number of Observation point
_
4 = Actual Value

2 =Predicted Value

0
1 2 3 4 5
X Independent Variable
www.hopelearning.net Hope_Artificial_Intelligence:HAI 36
@hope_artificial_intelligence
#learnaiwithramisha

Error
Input Actual Output Predicted Output Error=(Actual -Predicted)2
1 3.8 3.5 0.09
3 4.5 4.7 0.02
4 5.6 5.3 0.09
5 4.6 1.4 10.24
6 2.3 3.4 1.44
9 7.6 7.1 0.25
10 3.4 2.3 0.87

www.hopelearning.net Hope_Artificial_Intelligence:HAI 37
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: 1.Sum of Square Error(SSE) or Residual Sum of Square(RSS)

Formula:
𝑛
𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒 𝐸𝑟𝑟𝑜𝑟 (𝑆𝑆𝐸)=∑ ( 𝑦𝑖−−
𝑦𝑖 ) 2
r 𝑖=0
r ro Where, i = Observation point
reE yi

10 u a
f Sq −Y n= number of Observation point
o
m _
Su
Y Dependant Variable

8 = Actual Value

6 =Predicted Value

Take away:
2 If,
Higher the SSE, then predicted value is poor
0 Smaller the SSE, then predicted value is good
1 2 3 4 5
X Independent Variable

www.hopelearning.net Hope_Artificial_Intelligence:HAI 38
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: 2.Sum of Square Regression(SSR) or Explained Sum of Square(ESS)

Formula:
𝑛
𝑆𝑢𝑚 𝑜𝑓 𝑆𝑞𝑢𝑎𝑟𝑒 𝑅𝑒𝑔𝑟𝑒𝑠𝑠𝑖𝑜𝑛(𝑆𝑆𝑅)=∑−
( 𝑦 𝑖− 𝑦𝑚𝑒𝑎𝑛 ) 2
𝑖=0

10 Where, i = Observation point

−Y
Y Dependant Variable

8 n= number of Observation point

SSR _
6 =Predicted Value
ymean
= Mean of Dependant
4 Variable(Response variable)

2
Take away:
 Higher the SSR(or)ESS, better the model performance
0
1 2 3 4 5

X Independent Variable
www.hopelearning.net Hope_Artificial_Intelligence:HAI 39
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: 3.Sum of Square Total(SST)

Formula: 𝑛
𝑆𝑢𝑚 𝑆𝑞𝑢𝑎𝑟𝑒 𝑇𝑜𝑡𝑎𝑙(𝑆𝑆𝑇 )=∑ ( 𝑦 𝑖 − 𝑦𝑚𝑒𝑎𝑛 ) 2
𝑖=0
SST= SSR+SSE
Where, i = Observation point
Y
10
n= number of Observation point
Y Dependant Variable

8 SST = Actual Value

6 = Mean of Dependant
ymean
Variable(Response variable)
4

2
Take away:
0
1 2 3 4 5 If,
Smaller the SST, better the model
X Independent Variable
www.hopelearning.net Hope_Artificial_Intelligence:HAI
@hope_artificial_intelligence
#learnaiwithramisha Validating parameter: 4. R Squared(R2)

𝑛
_
SSR
∑_________________
( 𝑦 𝑖− 𝑦𝑚𝑒𝑎𝑛 ) 2 Purpose of R2 :
To know , how well the model is fitted.
______ = 𝑖= 0
R 2 =
SST 𝑛

∑ ( 𝑦 𝑖− 𝑦𝑚𝑒𝑎𝑛 ) 2 How R2 differs from other parameters like SSE,SSR and SST ?
𝑖= 0
SSE, SSR and SST range varies with dataset to dataset.
But,
Where, i = Observation point R2 exists between 0 and 1
If,
n= number of Observation point R2 = nearly to 1, then built model has better performance.
_ R2 = nearly to 0, then built model has poor performance.
= Actual Value
The only drawback of R2 is that if new predictors (X) are added
=Predicted Value to our model, R2 only increases or remains constant but it never
decreases. We can not judge that by increasing complexity of
= Mean of Dependant our model, are we making it more accurate
Variable(Response variable)
www.hopelearning.net Hope_Artificial_Intelligence:HAI
@hope_artificial_intelligence
#learnaiwithramisha

Graphs with Equation

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 42
#learnaiwithramisha
Validating parameter: 5.Adjusted R2

n 1
R2  1 (1  R2 );
nk
n  number of observations,
k  number of independent variables.

R2 Adjusted R2

 R2 shows, how the model is fitted with  Adjusted R2 helps to find most
the actual data points. significant independent variable.

 R2 gets increase when new independent  Adjusted R2 only gets increase when
variable is added to the existing model most significant independent variable
irrespective of poor significant or most is added to the model otherwise
significant independent variable. stays constant.

 It has only positive value  It may have negative value.

between o to 1.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 43
@hope_artificial_intelligence
#learnaiwithramisha
The Purpose of Training and Test Set

Original Data Training

Test Set: 20
set : 100 Dataset: 80

Hope_Artificial_Intelligence:HAI 44
www.hopelearning.net
#learnaiwithramisha

Training Dataset: 80

Weight
,Bais
Y= 0.3x1+0.4x2+0
Using Training
Data Set

Model

www.hopelearning.net Hope_Artificial_Intelligence:HAI 45
@hope_artificial_intelligence
#learnaiwithramisha

Test Set: 20

x1 x2 Y Y= 0.3x1+0.4x2+0 Found using Training Set

+0 =27.95
35.5

If(y>30):
print(“Unfit”)
Else
print(“fit”)

www.hopelearning.net Hope_Artificial_Intelligence:HAI 46
@hope_artificial_intelligence
Types of fitting

Hope_Artificial_Intelligence:HAI 47
 Simple Linear Regression- Take Away points
#learnaiwithramisha Model = 0.3X + 5 (for understanding purpose 0.3 = weight or slope, 5= Intercept or Initial value )

Concept Name Formula Inference

Simple Linear Y=wX+b One Dependent variable and One Independent
Variable
y)( Spread of data should be in linear
W= ___________________
(
B= (∑ 𝑥𝑦 ¿−(∑ x ) ( ∑ 𝑦)
n ___________________
(
Sum of Square Error(SSE) SSE= SSE maximum, poor the model
(or) (or) SSE minimum, better the model
Residual Sum of Square (RSS) SSE= summation(Actual value – Predicted Value) 2
(or) (Note: SSE value varies dataset to dataset)
Unexplained Sum of Square
Sum of Square Regression(SSR) SSR= SSR maximum, better the model
(or) (or) SSR minimum, poor the model
Explained Sum of Square(ESS) SSR=summation(Predicted Value – Dependent Value
mean)2 (Note: SSR value varies dataset to dataset)

Sum of Square Total(SST) SST= SST maximum, poor the Model

(or) SST minimum, better the model
SST = summation(Actual value -Predicted Value) 2
(Note: SST value varies dataset to dataset)

R-Squared (R2) R2 = SSR/SST If, R2 = nearly to 1, then built model has

better performance.
R2 = nearly to 0, then built model has poor
performance.

(Note: R2 exists between 0 and 1)

www.hopelearning.net Hope_Artificial_Intelligence:HAI 48
@hope_artificial_intelligence
#learnaiwithramisha

Assumption for Linear Regression

BEFORE THE MODEL

The Quantitative Data Condition Should be numbers

The Straight Enough Condition (or “linearity”)  should be in Linear Pattern

The Outlier Condition Should not have outlier

AFTER THE MODEL

Normality of Error

Homoscedasticity  Variance should be equal in overall spread

Log Transform if slightly curve can use Log Transform to convert perfect linear.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 49
@hope_artificial_intelligence
 Steps
Problem Identification

Reg/classification

Check Pattern

Split our dataset into training set and Test set

Model(Training Set)

Validating parameter(Test set)

Assumption

Before the model and After the model(Linear problem)

Hope_Artificial_Intelligence:HAI 50
#learnaiwithramisha

Multiple Linear Regression

Multiple Linear
Regression

www.hopelearning.net Hope_Artificial_Intelligence:HAI 51
@hope_artificial_intelligence
#learnaiwithramisha

Multiple Linear Regression

Multiple Linear
Regression

Simple Linear
Regression

www.hopelearning.net Hope_Artificial_Intelligence:HAI 52
@hope_artificial_intelligence
#learnaiwithramisha

Assumption for Multiple Linear Regression

BEFORE THE MODEL

The Quantitative Data Condition Should be numbers

The Straight Enough Condition (or “linearity”)  should be in Linear Pattern

The Outlier Condition Should not have outlier

AFTER THE MODEL

Homoscedasticity  Variance should be equal in overall spread

No Multicollinearity  Should not have Multicollinearity

www.hopelearning.net Hope_Artificial_Intelligence:HAI 53
@hope_artificial_intelligence
#learnaiwithramisha

Algorithm

NON-LINEAR ALGORITHM

Polynomial Algorithm
LINEAR ALGORITHM
Support Vector Machine
Simple Linear Algorithm
Decision Tree
Multiple Linear Algorithm
Random Forest

KNN

Naive's Bayes

www.hopelearning.net Hope_Artificial_Intelligence:HAI 54
@hope_artificial_intelligence
#learnaiwithramisha

Problem Statement of Non-Linear Algorithm

Finding Truth

www.hopelearning.net Hope_Artificial_Intelligence:HAI 55
@hope_artificial_intelligence
#learnaiwithramisha
Types of Fitting| Over fitting, Under fitting, well fitting

www.hopelearning.net Hope_Artificial_Intelligence:HAI 56
@hope_artificial_intelligence
#learnaiwithramisha

Polynomial Regression

Polynomial
Regression

www.hopelearning.net Hope_Artificial_Intelligence:HAI 57
@hope_artificial_intelligence
#learnaiwithramisha

Ccomparison

Multiple
Linear
Regression

Simple
Linear
Regression

Polyno
mial
Regressi
on

www.hopelearning.net Hope_Artificial_Intelligence:HAI 58
@hope_artificial_intelligence
#learnaiwithramisha

Assumption for Polynomial Regression

The Quantitative Data

The Outlier

Data Spread should be in Curve

www.hopelearning.net Hope_Artificial_Intelligence:HAI 59
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93

www.hopelearning.net Hope_Artificial_Intelligence:HAI 60
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

www.hopelearning.net Hope_Artificial_Intelligence:HAI 61
Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93 @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93

www.hopelearning.net Hope_Artificial_Intelligence:HAI 62
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

What if closer data point Exists?

Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93

www.hopelearning.net Hope_Artificial_Intelligence:HAI 63
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

What if closer data point Exists?

www.hopelearning.net Hope_Artificial_Intelligence:HAI 64
Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93 @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset

www.hopelearning.net Hope_Artificial_Intelligence:HAI 65
Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93 @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset- Three Dimensional

www.hopelearning.net Hope_Artificial_Intelligence:HAI 66
Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93 @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset- Three Dimensional- One Plane

www.hopelearning.net Hope_Artificial_Intelligence:HAI 67
Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93 @hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset- Three Dimensional -3 Plane

Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93

www.hopelearning.net Hope_Artificial_Intelligence:HAI 68
@hope_artificial_intelligence
#learnaiwithramisha

Support Vector Machine

Non- Separable Dataset- Three Dimensional-3 Plane

Image source: https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93

www.hopelearning.net Hope_Artificial_Intelligence:HAI 69
@hope_artificial_intelligence
#learnaiwithramisha

Assumptions for Support Vector Machine

The data spread should be Non-Linear

Pattern

No Outliers

www.hopelearning.net Hope_Artificial_Intelligence:HAI 70
@hope_artificial_intelligence
#learnaiwithramisha

Decision Tree

Image Source: https://towardsdatascience.com/random-forest-a-powerful-ensemble-learning-algorithm-2bf132ba639d

www.hopelearning.net Hope_Artificial_Intelligence:HAI 71
@hope_artificial_intelligence
#learnaiwithramisha

www.hopelearning.net Hope_Artificial_Intelligence:HAI 72
@hope_artificial_intelligence
#learnaiwithramisha

Decision Tree

Image Source: https://towardsdatascience.com/random-forest-a-powerful-ensemble-learning-algorithm-2bf132ba639d

www.hopelearning.net Hope_Artificial_Intelligence:HAI 73
@hope_artificial_intelligence
#learnaiwithramisha

Important Terminology related to Decision Trees

1.Root Node: It represents the entire population or sample and this further gets divided into two or more
homogeneous sets.
2.Splitting: It is a process of dividing a node into two or more sub-nodes.
3.Decision Node: When a sub-node splits into further sub-nodes, then it is called the decision node.
4.Leaf / Terminal Node: Nodes do not split is called Leaf or Terminal node.
5.Pruning: When we remove sub-nodes of a decision node, this process is called pruning. You can say
the opposite process of splitting.
6.Branch / Sub-Tree: A subsection of the entire tree is called branch or sub-tree.
7.Parent and Child Node: A node, which is divided into sub-nodes is called a parent node of sub-nodes
whereas sub-nodes are the child of a parent node.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 74
@hope_artificial_intelligence
#learnaiwithramisha

How to select the best variable from the dataset for Root Node

Entropy, Information Gini index, Gain Ratio,

gain,

Reduction in
Variance

www.hopelearning.net Hope_Artificial_Intelligence:HAI 75
@hope_artificial_intelligence
How to select the best variable from the dataset for Root Node

Entropy

If ,
Entropy is larger Randomness is high  perfectly will not able to
predict and Vice versa

Hope_Artificial_Intelligence:HAI 76
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Entropy

ID3 follows the rule — A branch with an entropy of zero is a

leaf node and A branch with entropy more than zero needs
further splitting.
www.hopelearning.net Hope_Artificial_Intelligence:HAI 77
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Information
gain

Constructing a decision tree is all about finding an attribute that returns the highest
information gain and the smallest entropy.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 78
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Information
gain

Where “before” is the dataset before the split, K is the number of subsets generated by the split, and
(j, after) is subset j after the split.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 79
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Gini index,

You can understand the Gini index as a cost function used to evaluate splits in the dataset.
It is calculated by subtracting the sum of the squared probabilities of each class from one.
It favors larger partitions and easy to implement whereas information gain favors smaller partitions with
distinct values.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 80
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Gain Ratio,

Gain ratio overcomes the problem with information gain by taking into account the number
of branches that would result before making the split.
It corrects information gain by taking the intrinsic information of a split into account.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 81
@hope_artificial_intelligence
#learnaiwithramisha
How to select the best variable from the dataset for Root Node

Reduction in
Variance

This algorithm uses the standard formula of variance to choose the best split.
The split with lower variance is selected as the criteria to split the population:

www.hopelearning.net Hope_Artificial_Intelligence:HAI 82
@hope_artificial_intelligence
#learnaiwithramisha
How to avoid/counter Overfitting in Decision Trees?

Pruning Decision Tree

www.hopelearning.net Hope_Artificial_Intelligence:HAI 83
@hope_artificial_intelligence
#learnaiwithramisha
How to avoid/counter Overfitting in Decision Trees?

Pruning Decision Tree

www.hopelearning.net Hope_Artificial_Intelligence:HAI 84
@hope_artificial_intelligence
#learnaiwithramisha

Decision Tree

Image Source: https://towardsdatascience.com/random-forest-a-powerful-ensemble-learning-algorithm-2bf132ba639d

www.hopelearning.net Hope_Artificial_Intelligence:HAI 85
@hope_artificial_intelligence
#learnaiwithramisha

Important points for Decision Tree

There is Possible of overfitting, because of huge

decision split.
If we change the data in training, there is huge
difference in model

www.hopelearning.net Hope_Artificial_Intelligence:HAI 86
@hope_artificial_intelligence
Random Forest

Hope_Artificial_Intelligence:HAI 87
#learnaiwithramisha

Random Forest

Ensemble Learning

Bagging or Bootstrap Aggregation

Random feature selection

www.hopelearning.net Hope_Artificial_Intelligence:HAI 88
@hope_artificial_intelligence
#learnaiwithramisha Random Forest
Ensemble Learning

Image Source: https://towardsdatascience.com/random-forest-a-powerful-ensemble-learning-algorithm-2bf132ba639d

www.hopelearning.net Hope_Artificial_Intelligence:HAI 89
@hope_artificial_intelligence
#learnaiwithramisha Random Forest
Ensemble Learning

Image Source: https://towardsdatascience.com/random-forest-a-powerful-ensemble-learning-algorithm-2bf132ba639d

www.hopelearning.net 90
Hope_Artificial_Intelligence:HAI @hope_artificial_intelligence
#learnaiwithramisha
Random Forest
Bagging or Bootstrap Aggregation

www.hopelearning.net Hope_Artificial_Intelligence:HAI 91
@hope_artificial_intelligence
#learnaiwithramisha
Random Forest
Ensemble Learning

www.hopelearning.net Hope_Artificial_Intelligence:HAI 92
@hope_artificial_intelligence
#learnaiwithramisha
Random Forest

Image Source: https://towardsdatascience.com/random-forest-a-powerful-ensemble-learning-algorithm-2bf132ba639d

www.hopelearning.net Hope_Artificial_Intelligence:HAI 93
@hope_artificial_intelligence
#learnaiwithramisha

Pure Classification Algorithm

K-Nearest Neighbor

Navies' Bayes

www.hopelearning.net Hope_Artificial_Intelligence:HAI 94
@hope_artificial_intelligence
#learnaiwithramisha

K- Nearest Neighbour

www.hopelearning.net Hope_Artificial_Intelligence:HAI 95
@hope_artificial_intelligence
#learnaiwithramisha

K- Nearest Neighbour

www.hopelearning.net Hope_Artificial_Intelligence:HAI 96
@hope_artificial_intelligence
#learnaiwithramisha

K- Nearest Neighbour

www.hopelearning.net Hope_Artificial_Intelligence:HAI 97
@hope_artificial_intelligence
#learnaiwithramisha

Low Bias, Low Variance- Good Model

Low Bias, High Variance- Over Fitting

Model

High Bias, Low Variance- Under Fitting

Model

High Bias, High Variance- Poor Model

www.hopelearning.net Hope_Artificial_Intelligence:HAI 98
Image Source: https://medium.com/30-days-of-machine-learning/day-3-k-nearest-neighbors-and-bias-variance-tradeoff-75f84d515bdb @hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

Naïve Bayes is a probabilistic machine learning

algorithm based on the Bayes Theorem.

Conditional probability is a measure of the

probability of an event occurring given that another
event has (by assumption, presumption, assertion, or
evidence) occurred.
www.hopelearning.net Hope_Artificial_Intelligence:HAI 99
@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

In simpler terms, Bayes’ Theorem is a way of finding a

probability when we know certain other probabilities.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 100

@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

www.hopelearning.net Hope_Artificial_Intelligence:HAI 101

@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes
Assumptions

Independent  Each variable

should not have any connection

Equal  All the variables are

equally important

www.hopelearning.net Hope_Artificial_Intelligence:HAI 102

@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

The variable y is the class variable(stolen?), which represents if the car is stolen or not given the conditions.
Variable X represents the parameters/features.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 103

@hope_artificial_intelligence
#learnaiwithramisha
Navies' Bayes

www.hopelearning.net Hope_Artificial_Intelligence:HAI 104

@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes

Since 0.144 > 0.048, Which means given the features RED
SUV and Domestic, our example gets classified as ’NO’ the car
is not stolen.
www.hopelearning.net Hope_Artificial_Intelligence:HAI 105
@hope_artificial_intelligence
#learnaiwithramisha

Navies' Bayes
The zero-frequency problem

www.hopelearning.net Hope_Artificial_Intelligence:HAI 106

@hope_artificial_intelligence
#learnaiwithramisha

Types of Algorithm based on spread on data

Linear Algorithm
• Linear
• Multiple Linear

Non-Linear Algorithm(One Problem

Statement-4 models)
• Polynomial
• Support Vector Machine
• Decision Tree
• Random Forest

www.hopelearning.net Hope_Artificial_Intelligence:HAI 107

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Regression Minimum Error

www.hopelearning.net Hope_Artificial_Intelligence:HAI 108

@hope_artificial_intelligence
#learnaiwithramisha

www.hopelearning.net Hope_Artificial_Intelligence:HAI 109

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class

N= Positive Negative

Positive True Positive False Negative

(TP) (FN)
Actual Class

Negative False Positive True Negative

(FP) (TN)

www.hopelearning.net Hope_Artificial_Intelligence:HAI 110

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)

Negative False Positive True Negative

(FP) (TN)

www.hopelearning.net Hope_Artificial_Intelligence:HAI 111

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
Actual Class

(TP) (FN)

Negative False Positive True Negative

(FP) (TN)

www.hopelearning.net Hope_Artificial_Intelligence:HAI 112

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
to be positive.

Negative False Positive True Negative

(FP) (TN)

www.hopelearning.net Hope_Artificial_Intelligence:HAI 113

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 114

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
False Positive (FP) : Observation is negative, but is predicted
positive.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 115

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
False Positive (FP) : Observation is negative, but is predicted
positive.
True Negative (TN) : Observation is negative, and is predicted
to be negative.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 116

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
Type I Error to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
True Negative (TN) : Observation is negative, and is predicted
to be negative.

False Positive (FP) : Observation is negative, but is predicted

positive.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 117

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
Type I Error to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
Type II Error
True Negative (TN) : Observation is negative, and is predicted
to be negative.

False Positive (FP) : Observation is negative, but is predicted

positive.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 118

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class Positive (P) : Observation is positive (for example: is an apple).

N= Positive Negative Negative (N) : Observation is not positive (for example: is

not an apple).
Positive True Positive False Negative
True Positive (TP) : Observation is positive, and is predicted
Actual Class

(TP) (FN)
Type I Error to be positive.

Negative False Positive True Negative False Negative (FN) : Observation is positive, but is predicted
(FP) (TN) negative.
Type II Error
True Negative (TN) : Observation is negative, and is predicted
to be negative.

False Positive (FP) : Observation is negative, but is predicted

positive.

www.hopelearning.net Hope_Artificial_Intelligence:HAI 119

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class
N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)
Error I
Negative False Positive True Negative
(FP) (TN)
Error II

www.hopelearning.net Hope_Artificial_Intelligence:HAI 120

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Recall (Sensitivity) - Recall is the ratio of

correctly predicted positive observations to
the all observations in actual class - yes.
Predicted Class
N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)
Error I
Negative False Positive True Negative
(FP) (TN)
Error II

www.hopelearning.net Hope_Artificial_Intelligence:HAI 121

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Precision(Specificity) - Precision is the

ratio of correctly predicted positive
observations to the total predicted positive
Predicted Class observations.

N= Positive Negative

Positive True Positive False Negative Recall (Sensitivity) - Recall is the ratio of
Actual Class

(TP) (FN) correctly predicted positive observations to

Error I the all observations in actual class - yes.

Negative False Positive True Negative

(FP) (TN)
Error II

www.hopelearning.net Hope_Artificial_Intelligence:HAI 122

@hope_artificial_intelligence
#learnaiwithramisha

F1 score - F1 Score is the weighted average of Precision and Recall.

Therefore, this score takes both false positives and false negatives into account.
Intuitively it is not as easy to understand as accuracy, but F1 is usually more useful
than accuracy, especially if you have an uneven class distribution.
Accuracy works best if false positives and false negatives have similar cost. If the cost
of false positives and false negatives are very different, it’s better to look at both
Precision and Recall. In our case, F1 score is 0.701.

F1 Score = 2*(Recall * Precision) / (Recall + Precision)

www.hopelearning.net Hope_Artificial_Intelligence:HAI 123

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class
N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)
Error I
Negative False Positive True Negative
(FP) (TN)
Error II

www.hopelearning.net Hope_Artificial_Intelligence:HAI 124

@hope_artificial_intelligence
 Validating Parameter For Supervised Learning
#learnaiwithramisha

Classification Confusion Matrix

Predicted Class

N= Positive Negative

Positive True Positive False Negative

Actual Class

(TP) (FN)
Error I
Negative False Positive True Negative
(FP) (TN)
Error II

www.hopelearning.net Hope_Artificial_Intelligence:HAI 125

@hope_artificial_intelligence
#learnaiwithramisha
Algorithms

Algorithm for both Regression Algorithm Classification Algorithm

Regression and Classification

Support Vector Machine Linear Algorithm

Logistic Algorithm

Multiple Linear Navies Bayes

Decision Tree Regression KNN

Random Forest Polynomial Regression

@hope_artificial_intelligence
www.hopelearning.net Hope_Artificial_Intelligence:HAI 126
#learnaiwithramisha

Logistic Algorithm

www.hopelearning.net Hope_Artificial_Intelligence:HAI 127

@hope_artificial_intelligence
#learnaiwithramisha

www.hopelearning.net Hope_Artificial_Intelligence:HAI 128

@hope_artificial_intelligence
#learnaiwithramisha

Unsupervised : Clustering Algorithm

K-Means

Hierarchical

www.hopelearning.net Hope_Artificial_Intelligence:HAI 129

@hope_artificial_intelligence
#learnaiwithramisha

K means

www.hopelearning.net Hope_Artificial_Intelligence:HAI 130

@hope_artificial_intelligence
#learnaiwithramisha

K means

www.hopelearning.net Hope_Artificial_Intelligence:HAI 131

@hope_artificial_intelligence
#learnaiwithramisha

Hierarchical

Agglomerative
• Compute the proximity matrix
• Let each data point be a cluster
• Repeat: Merge the two closest clusters
and update the proximity matrix
• Until only a single cluster remains

Divisive
• Opposite of Agglomerative

www.hopelearning.net Hope_Artificial_Intelligence:HAI 132

@hope_artificial_intelligence
#learnaiwithramisha
Hierarchical
Agglomerative

www.hopelearning.net Hope_Artificial_Intelligence:HAI 133

@hope_artificial_intelligence
 sad

https://towardsdatascience.com/the-complete-guide-to-decisio
n-trees-28a4e3c7be14

https://www.listendata.com/2018/01/linear-regression-in-pyth
on.html

https://www.statsmodels.org/dev/examples/notebooks/genera
ted/regression_diagnostics.html

Hope_Artificial_Intelligence:HAI 134