Scribd
Upload
10 views5 pages

Classification and Regression Tree

CART (Classification and Regression Tree) is a decision tree algorithm used for predictive modeling, capable of handling both classification and regression tasks. It splits data into subsets based on input features, aiming to maximize purity for classification and minimize variance for regression. Key characteristics include non-parametric nature, automatic feature selection, and ease of interpretability, though it can be prone to overfitting and instability.

Uploaded by

zoopashqadeer
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
10 views5 pages

Classification and Regression Tree

CART (Classification and Regression Tree) is a decision tree algorithm used for predictive modeling, capable of handling both classification and regression tasks. It splits data into subsets based on input features, aiming to maximize purity for classification and minimize variance for regression. Key characteristics include non-parametric nature, automatic feature selection, and ease of interpretability, though it can be prone to overfitting and instability.

Uploaded by

zoopashqadeer
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

Classification and Regression Tree (CART) in Statistics

CART is a type of decision tree algorithm used for predictive modeling in statistics and machine learning.
It splits data into subsets based on input features, forming a tree structure. CART is widely used in two
types of problems:

1. Classification Trees:

 Used when the target variable is categorical (e.g., "Yes/No", "High/Low").

 Goal: Predict the class label for a given set of input features.

 At each split, the algorithm tries to maximize the purity of the resulting subsets (using measures
like Gini Index or Entropy).

 Example:

o Predicting whether a customer will buy a product based on their age and income.

2. Regression Trees:

 Used when the target variable is continuous (e.g., height, weight, price).

 Goal: Predict a numerical value for the target variable.

 At each split, the algorithm minimizes the variance (or other error measures like Mean Squared
Error) within the resulting subsets.

 Example:

o Predicting house prices based on location, size, and number of bedrooms.

How CART Works:

1. Root Node: The tree starts with the entire dataset.

2. Splitting Criteria:

o For classification: Split data to maximize class purity.

o For regression: Split data to minimize variance.

3. Branches: Each split creates branches representing subsets of data.

4. Leaf Nodes: End nodes where predictions are made (a single class for classification or an average
value for regression).
Key Characteristics:

 Non-parametric: Does not assume a specific distribution for the data.

 Feature Selection: Automatically selects the most relevant features for splitting.

 Interpretability: Easy to understand and visualize.

Advantages:

 Simple to interpret and explain.

 Handles both categorical and numerical data.

 Can model complex, non-linear relationships.

Disadvantages:

 Prone to overfitting (can learn noise in the data).

 Can be unstable (small changes in data may lead to a completely different tree).

 Requires pruning or ensemble methods (like Random Forest) for better generalization.

Key Differences:

Aspect Classification Tree Regression Tree


Target Categorical (e.g., "Yes/No", "Class Continuous (e.g., price, weight,
Variable A/B"). salary).
Goal Predict the class label of data points. Predict a numerical value.
Splitting Gini Index, Entropy, or Variance reduction or Mean
Criteria Misclassification Rate. Squared Error.
Leaf Node Average or predicted numerical
Majority class label of the subset.
Value value.
LEAF NODE:

The leaf node value in a decision tree is the final prediction or output made at the end of a branch. It
represents the result of the splitting process for a particular path through the tree.

The leaf node value depends on the type of decision tree being used:

1. Leaf Node Value in Classification Trees

 The leaf node value represents the class label or probability distribution of classes for the data
points in that node.

 How It’s Determined:

o Majority Class Rule: The most frequent class in the leaf node is assigned as the
prediction for that path.

o Class Probability: Optionally, the proportions of classes in the leaf can also be used as
probabilities.

Example:

If a leaf node contains the following class distribution:

 Class A: 70%

 Class B: 30% Then:

 The predicted value is Class A (majority class).

 The probabilities are:

o P(A)=0.7P(A) = 0.7P(A)=0.7

o P(B)=0.3P(B) = 0.3P(B)=0.3

2. Leaf Node Value in Regression Trees

 The leaf node value represents the numerical prediction for the data points in that node.

 How It’s Determined:

o It is typically the mean (average) of the target variable values of all data points in the
leaf.

Example:

If a leaf node contains the following target values:

 50,60,7050, 60, 7050,60,70 Then:


 The predicted value for that leaf is: Leaf Node Value=50+60+703=60\text{Leaf Node Value} = \
frac{50 + 60 + 70}{3} = 60Leaf Node Value=350+60+70=60

Why Is the Leaf Node Value Important?

 It is the final output of the tree for any given input data point.

 For a given set of features, a path through the tree leads to a specific leaf node, and the value of
that leaf determines the tree's prediction.

You might also like