Peoples Empowerment Group

ISBM COLLEGE OF ENGINEERING, PUNE

DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22

Assignment No. 2

2.1 Title

2.2Problem Definition:

A dataset collected in a cosmetics shop showing details of customers and whether or not
they responded to a special offer to buy a new lip-stick is shown in table below. Use this
dataset to build a decision tree, with Buys as the target variable, to help in buying lip-sticks
in the future. Find the root node of decision tree. According to the decision tree you have
made from previous training data set, what is the decision for the test data: [Age < 21,
Income = Low, Gender = Female, Marital Status = Married]?

2.3Prerequisite:

Basic of Python, Data Mining Algorithm, Concept of Decision Tree Classifier

2.4Software Requirements:

Anaconda with Python 3.7

2.5Hardware Requirement:

PIV, 2GB RAM, 500 GB HDD, Lenovo A13-4089Model.

2.6Learning Objectives:

Learn How to Apply Decision Tree Classifier to find the root node of decision tree.
According to the decision tree you have made from previous training data set.

2.7Outcomes:

After completion of this assignment students are able Implement code for Create Decision
tree for given dataset and find the root node for same based on the given condition.

2.8Theory Concepts:

2.8.1 Motivation
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22

Suppose we have following plot for two classes represented by black circle and blue
squares. Is it possible to draw a single separation line? Perhaps no.

Can you draw single division line for these classes?

We will need more than one line, to divide into classes. Something similar to following
image:

We need two lines one for threshold of x and

threshold for y.

We need two lines here one separating according to threshold value of x and other for
threshold value of y. Decision Tree Classifier, repetitively divides the working area(plot)
into sub part by identifying lines. (Repetitively because there may be two distant
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22

regions of same class divided by other as shown in image below).

So, when does it terminate?
1. Either it has divided into classes that are pure (only containing members of single
class)
2. Some criteria of classifier attributes are met.

1. Impurity-
In above division, we had clear separation of classes. But what if we had following case?
Impurity is when we have a trace of one class division into other. This can arise due to
following reason
2. We run out of available features to divide the class upon.
We tolerate some percentage of impurity (we stop further division) for faster
performance. (There is always tradeoff between accuracy and performance).
For example, in second case we may stop our division when we have x number of
fewer number of elements left. This is also known as gini impurity.

2. Entropy
Entropy is degree of randomness of elements or in other words it is measure of
impurity. Mathematically, it can be calculated with the help of probability of the items as:

It is negative summation of probability times the log of probability of item x.

For example,
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22

if we have items as number of dice face occurrence in a throw

event as 1123, the entropy is
p(1) = 0.5
p(2) = 0.25
p(3) = 0.25
entropy = - (0.5 * log(0.5)) - (0.25 * log(0.25)) -(0.25 * log(0.25)
= 0.45

3. Information Gain

Suppose we have multiple features to divide the current working set. What feature should
we select for division? Perhaps one that gives us less impurity.
Suppose we divide the classes into multiple branches as follows, the information gain
at any node is defined as
Information Gain (n) = Entropy(x) — ([weighted average] * entropy(children
for feature)) This need a bit explanation!
Suppose we have following class to work with initially

112234445

Suppose we divide them based on property: divisible by 2

Entropy at root level: 0.66

Entropy of left child: 0.45, weighted value = (4/9) * .45 = 0.2
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22

Entropy of right child: 0.29, weighted value = (5/9) * 0.29 = 0.16

Information Gain = 0.66 - [0.2 + 0.16] = 0.3

Check what information gain we get if we take decision as prime number instead of
divide by 2. Which one is better for this case?
Decision tree at every stage selects the one that gives best information gain.
When information gain is 0 means the feature does not divide the working set at all.

2.9 Given Data set in Our Definition

What is the decision for the test data: [Age < 21, Income = Low, Gender = Female,
Marital Status = Married]?

Answer is Whether Yes, or No??

1. Which of the attributes would be the root node?
A. Age
B. Income
C. Gender
D. Marital Status
Solution: C
2. What is the decision for the test data [Age < 21, Income = Low, Gender = Female, Marital
Status = Married]?
A. Yes
B. No
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22

Solution: A
[Hints: construct the decision tree to answer these questions]

2.10 Algorithm

1. Import the Required Packages

2. Read Given Dataset
3. Perform the label Encoding Mean Convert String value into Numerical values
4. Import and Apply Decision Tree Classifier
5. Predict value for the given Expression like [Age < 21, Income = Low, Gender
= Female, Marital Status = Married]? In encoding Values [1,1,0,0]
6. Import the packages for Create Decision Tree.
7. Check the Decision Tree Created based on Expression.

Code:

#import packages

import numpy as np
import pandas as pd
import graphviz

#reading Dataset
dataset=pd.read_csv('E:/Soft Computing Practicals/Lab Practicals  Assignment2

/dataset.csv')

X=dataset.iloc[:,1:-1]
Y=dataset.iloc[:,5].values

#Performing label encoding

from sklearn.preprocessing import LabelEncoder
labelencoder_X = LabelEncoder()
X = X.apply(LabelEncoder().fit_transform)
print(X)

#Find the label with encoded value

for i in range(4):
    col=X.iloc[:,i:i+1]
   if (i==0):
     print("age= 21-35 as 0, age= <21 as 1 & age= >35 as 2")
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22

   if (i==1):
     print("income= High as 0, income= Medium as 2 & income= Low as 1")
   if (i==2):
     print("gender= Male as 1, gender= Female as 0")
   if (i==3):
     print("Marital Status= Single as 1, Marital Status= Married as 0")

#Creating Decision Tree Classifier

from sklearn.tree import DecisionTreeClassifier
regressor=DecisionTreeClassifier()
regressor.fit(X.iloc[:,:5],Y)

#Predict the value for given test data

X_in=np.array([1,1,0,0])
Y_pred=regressor.predict([X_in])
print("Predicted Value for Given Test Data is : ",Y_pred)

#Creating Graph
from sklearn.datasets import load_iris
from sklearn import tree
iris = load_iris()
dot_data = tree.export_graphviz(regressor, out_file=None, filled=True, rounded=True,
special_characters=True)  
graph = graphviz.Source(dot_data)  
graph.render("tree")

2.11 Output
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22
 Peoples Empowerment Group
ISBM COLLEGE OF ENGINEERING, PUNE
DEPARTMENT OF COMPUTER ENGINEERING
Academic Year 2021-22

2.12 Conclusion
We learn that how to create a Decision Tree based on given decision, Find the Root Node
of the tree using Decision tree Classifier.