Scribd
Upload
19 views

Decision Tree Learning

The document discusses decision tree learning and classification. It describes how decision trees work by using internal nodes to test attributes, branches to represent test outcomes, and leaf nodes to hold class labels. It also discusses how decision trees are used for classification by tracing a path from the root to a leaf node to predict a class label. Some advantages of decision trees are that they do not require parameter setting, can handle multidimensional data, and have an intuitive tree structure that is easy for humans to understand. Decision trees are commonly used for applications such as medicine, manufacturing, financial analysis, and more.

Uploaded by

discodancerhasan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
19 views

Decision Tree Learning

The document discusses decision tree learning and classification. It describes how decision trees work by using internal nodes to test attributes, branches to represent test outcomes, and leaf nodes to hold class labels. It also discusses how decision trees are used for classification by tracing a path from the root to a leaf node to predict a class label. Some advantages of decision trees are that they do not require parameter setting, can handle multidimensional data, and have an intuitive tree structure that is easy for humans to understand. Decision trees are commonly used for applications such as medicine, manufacturing, financial analysis, and more.

Uploaded by

discodancerhasan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

CSE-463

Machine Learning

Decision Tree Learning


Md. Rashadur Rahman
Department of CSE
CUET
Decision Tree

• A Decision Tree is a flowchart-like tree structure, where each


▪ Internal Node (Non leaf node)
denotes a test on an attribute
▪ Branch (Edge)
represents an outcome of the test
▪ Leaf Node (or terminal node
holds a class label

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 2


A Sample Training and Testing Dataset

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 3


A Sample Decision Tree

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 4


How are decision trees used for classification?

Given a tuple, X, for which the associated class label is unknown,


▪ The attribute values of the tuple are tested against the decision tree.

▪ A path is traced from the root to a leaf node, which holds the class
prediction for that tuple.

▪ Decision trees can easily be converted to classification rules.

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 5


Classification using Decision Tree

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 6


Advantages of Decision Tree (Why are decision tree classifiers so popular?)

▪ The construction of decision tree classifiers does not require any domain
knowledge or parameter setting, and therefore is appropriate for
exploratory knowledge discovery.
▪ Decision trees can handle multidimensional data.
▪ Their representation of acquired knowledge in tree form is intuitive and
generally easy to assimilate by humans.
▪ The learning and classification steps of decision tree induction are simple
and fast.
In general, decision tree classifiers have good accuracy. However, successful
use may depend on the data at hand.

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 7


Application of Decision Tree
Decision tree induction algorithms have been used for classification in
many application areas such as:

✓medicine,
✓ manufacturing and production,
✓ financial analysis,
✓ astronomy,
✓molecular biology

Decision trees are the basis of several commercial rule induction systems.

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 8


A different Decision Tree

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 9


Decision Tree

• Which?

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 10


A criterion for attribute selection
• Which is the best attribute?
– The one which will result in the smallest tree
– Heuristic: choose the attribute that produces the “purest” nodes

• Popular impurity criterion: information gain


– Information gain increases with the average purity of the subsets that an
attribute produces
• Strategy: choose attribute that results in greatest information gain

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 11


25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 12
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 13
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 14
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 15
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 16
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 17
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 18
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 19
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 20
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 21
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 22
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 23
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 24
25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 25
Example of same class

RID age income student? credit_rating buys_computer


1 youth high no fair no
2 youth high no excellent no
3 senior low yes excellent no
4 youth medium no fair no
5 senior medium no excellent no

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 26


Thank You

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 27


References

[1] Han, J., Kamber, M., & Pei, J. (2012). Data mining concepts and techniques third
edition. University of Illinois at Urbana-Champaign Micheline Kamber Jian Pei Simon
Fraser University

25/07/2023 Department of CSE, Chittagong University of Engineering & Technology 28

You might also like