Scribd
Upload
18 views40 pages

AI - Mod 5. Part 2

PPt-2

Uploaded by

rohanpics717
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
18 views40 pages

AI - Mod 5. Part 2

PPt-2

Uploaded by

rohanpics717
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 40

LEARNING DECISION TREES

• A decision tree is a type of supervised learning


algorithm that is commonly used in machine learning
to model and predict outcomes based on input data.
• It is a tree-like structure where each internal node
tests on attribute, each branch corresponds to
attribute value and each leaf node represents the
final decision or prediction.
• The decision tree algorithm falls under the category
of supervised learning. They can be used to solve
both regression and classification problems.
Decision Tree Terminologies

• Root Node: Root node is from the decision tree starts.


It represents the entire dataset, which further divided into
two or more homogeneous sets.
• Internal Nodes (Decision Nodes): Nodes in the tree whose
choices are determined by the values of particular attributes.
There are branches on these nodes that go to other nodes.
• Leaf Nodes (Terminal Nodes): Final output node.
• Branches (Edges): Links between nodes that show how
decisions are made in response to particular circumstances.
• Splitting: The process of dividing a node into two or more
sub-nodes based on a decision criterion. It involves selecting
a feature and a threshold to create subsets of data.
• Decision Criterion: The rule or condition used
to determine how the data should be split at a
decision node. It involves comparing feature
values against a threshold.
• Pruning: The process of removing branches or
nodes from a decision tree to improve its
generalisation and prevent over fitting.
Why Decision Tree?
• It mimic human thinking ability while making a
decision. The logic behind thinking is easily
understood because it shows tree like structure
• The family of decision tree learning algorithms includes
algorithms like ID3, CART, ASSISTANT, etc.
• They are supervised learning algorithms used for both
classification and regression tasks.
• Each node in the decision tree represents a test of an
attribute of the instance, and a branch descending from
that node indicates one of the possible values for that
attribute.
• So, classification of an instance starts at a root node of
the decision tree, tests an attribute at this node, then
moves down the tree branch corresponding to the value
of the attribute. This process is then repeated for the
subtree and splitting process continues till leaf nodes
until final decision obtained.
• In the Decision Tree, the major challenge is
the identification of the attribute for the root
node at each level. This process is known as
attribute selection. There are two popular
attribute selection measures:
• Information Gain
• Gini Index
What is entropy in decision tree?
• In decision trees, entropy is a measure of
impurity or disorder within a dataset. It
quantifies the uncertainty associated with
classifying instances, guiding the algorithm to
make informative splits for effective decision-
making.
• It's used to evaluate the quality of a model
and its ability to make accurate predictions.
• Information Gain
• To find the best feature that serves as a root
node in terms of information gain, first use
each defining feature, split the dataset along
the values of these descriptive features, and
then calculate the entropy of the dataset.
• Then, subtract this value from the initially
calculated entropy of the dataset to see how
much this feature splitting reduces the original
entropy, which gives the information gain of a
feature and is calculated as:
• Gini Index
• The split made in decision tree is pure if all the
data points are separated into different classes.
• It is calculated by subtracting the sum of
squared probabilities of each class from one.
• A feature with a lower Gini index is chosen for a
split.
Example-1
From (1) and (2), A1 give more information gain. So make A1 as root node.
Example 2
Example-3

You might also like