Scribd
Upload
77 views

Data Mining - Decision Tree

This document provides an overview of decision trees for data mining. It discusses how decision trees are used for classification and prediction problems through a binary recursive partitioning process. The strengths include generating understandable rules with low computational requirements. Weaknesses include being weak for estimating continuous variables, time series problems, and handling missing data. Key algorithms like CART are described along with issues like choosing inputs, splits, pruning, and stopping criteria.

Uploaded by

Amitav Pattnaik
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
77 views

Data Mining - Decision Tree

This document provides an overview of decision trees for data mining. It discusses how decision trees are used for classification and prediction problems through a binary recursive partitioning process. The strengths include generating understandable rules with low computational requirements. Weaknesses include being weak for estimating continuous variables, time series problems, and handling missing data. Key algorithms like CART are described along with issues like choosing inputs, splits, pruning, and stopping criteria.

Uploaded by

Amitav Pattnaik
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 13

Data Mining – Decision Tree

Table of Content
• Application Situation
• Overview
• CART (Classification And Regression Tree)
• Strengths and Weaknesses
• Case using XL Miner

Data Mining - Decision Tree 2


Application Situation
Case Imagine Object
Case Issue of Credit
Age
Decision Worth
Income Giving Credit
Tree
(Y / N)
Married
Status Own
House?

Data Mining - Decision Tree 3


Overview
Directed Knowledge Discovery (Supervised
Learning) technique
Used for Classification (Classification Tree) as well
as prediction (Regression Tree)
Two Step Process
• Build Decision Tree using training set by
repeatedly splitting tree based on best splitting
attribute (that minimizes diversity)
• Apply Decision Tree to classify unknown data

Data Mining - Decision Tree 4


Overview
Gives rules that are easy to understand, code
in SQL, easy to implement
Decision tree represents series of questions.
Answer determines what to ask next
Questions divide search space into rectangular
region. Tuple is classified based on the region it
falls
Many paths result in same class. (Different Reason)
Good questions lead to result in a few questions

Data Mining - Decision Tree 5


Overview
Statistical classification methods (e.g. linear
regression) partition data into classes by drawing a
line.
Decision Tree allows several different ways for a
record to become part of the target class. (instead
of only one way)
e.g. in credit card industry, there are many ways in
which customer are profitable

Data Mining - Decision Tree 6


Overview
Important Issues
• Choosing Input Attributes
• Splits (for Gender, Split is obvious.. M or F. For
attributes with more than two values, it is not
trivial. For Continuous value, even more difficult.
One option: convert into discrete categories)
• Ordering of splitting is important
• Balances tree with fewer levels preferred

Data Mining - Decision Tree 7


Overview
Important Issues (Cont)
• Stopping Criteria: All value at a node
having one class, or most values at a node
having one class, or minimum levels
• Training Data (Too small does not work
well, Too much might lead to over-fitting)
• Pruning: Improve performance, avoid over-
fitting)
Data Mining - Decision Tree 8
RT Theoretical Foundation - 1
7.091

• Regression tree is built through a RM


5

process known as binary recursive 48 12

partitioning. This is an iterative 39.82667

process of splitting the data into 3.818


15

partitions, and then splitting it up DIS

36
further on each of the branches. 12

• Initially all of the records in 29.54074 6.388


5

training set (the pre-classified RM

records that are used to determine 13


23

the structure of the tree) are 22.11333 27.07037

together in one big box.

Data Mining - Decision Tree 9


RT Theoretical Foundation - 2
7.091

• The algorithm then tries breaking RM


5

up the data, using every possible 48 12

binary split on every field. The 39.82667

algorithm chooses the split that 3.818


15

partitions the data into two parts DIS

36
such that it minimizes the sum of 12

the squared deviations from the 29.54074 6.388


5
mean in the separate parts.  RM

13
23

22.11333 27.07037

Data Mining - Decision Tree 10


Strengths and Weaknesses
Generates understandable rules
Not much computation required to perform
classification (Not training part)
Can handle continuous as well as categorical
variables
Gives important fields for classification

Data Mining - Decision Tree 11


Strengths and Weaknesses
Weak in estimating continuous variable
Weak in handling continuous data for classification
(need to convert into categories, e.g. income, Age)
Weak for time series problem
Some algorithms Error-prone with too many classes
Computationally expensive to train
Difficulty in handling missing data

Data Mining - Decision Tree 12


References
1. Margaret Dunham, “Data Mining – Introductory
and Advanced Topics”, Pearson Edition (Ch 4.4:
Decision Tree Based Algorithms)
2. Michael Berry, Gordon Linoff, “Data Mining
Techniques ”, Wiley Publications (Ch 12:
Decision Trees)

Data Mining - Decision Tree 13

You might also like