Scribd
Upload
7 views5 pages

Write A Program To Demonstrate Decision Tree Algorithm For A Classification Problem and Perform Parameter Tuning For Better Results

The document outlines a program to demonstrate the Decision Tree Algorithm for both classification and regression problems using the Iris dataset. It details the steps for implementing the Decision Tree Classifier, including parameter tuning through Grid Search with Cross-Validation, and the Decision Tree Regressor, including performance evaluation using Mean Squared Error. Additionally, it includes a section with viva questions related to the concepts discussed.

Uploaded by

yaswanthbusireddy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views5 pages

Write A Program To Demonstrate Decision Tree Algorithm For A Classification Problem and Perform Parameter Tuning For Better Results

The document outlines a program to demonstrate the Decision Tree Algorithm for both classification and regression problems using the Iris dataset. It details the steps for implementing the Decision Tree Classifier, including parameter tuning through Grid Search with Cross-Validation, and the Decision Tree Regressor, including performance evaluation using Mean Squared Error. Additionally, it includes a section with viva questions related to the concepts discussed.

Uploaded by

yaswanthbusireddy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

4.

Demonstrate Decision Tree Algorithm for a classification problem


and perform parameter tuning for better results
AIM:
Write a program to demonstrate Decision Tree Algorithm for a
classification problem and perform parameter tuning for better results.

Description:
Decision Trees in Machine Learning

Decision Trees are a popular supervised learning algorithm used for both
classification and regression tasks. They mimic the human decision-making
process by creating a tree-like model of decisions and their possible
consequences.

Parameter Tuning

The Process: Parameter tuning, also known as hyperparameter optimization, is


the crucial step of finding the best set of hyperparameters for a given machine
learning model.
Hyperparameters: These are settings within a machine learning algorithm that
aren't learned from the training data itself. They are set before the training
process begins.

Algorithm:
Step 1: Start the Program.
Step 2: Import all necessary libraries for the problem.
Step 3: Loads the Iris dataset and splits it into training and testing sets.
Step 4: Create a Decision Tree Classifier and Train the initial decision tree model on the
training data using clf.fit()
Step 5: Use the trained model to make predictions on the test data using clf.predict().
Step 6: Calculate the accuracy of the initial model using accuracy_score()

Step 7: Create a dictionary param_grid to define the hyperparameters to tune and their possible
values

Step 8: Perform Grid Search with Cross-Validation and Print best parameters ,score.

Step 9:Create a new DecisionTreeClassifier instance with the best parameters found by grid
search and Train this new model on the training data.

Step 10. Make predictions on the test set using the model trained with the best parameters.

Step 11.Calculate and print the accuracy of the model with the best parameters.
Step 12. Visualize the decision tree

Step 13. Stop the program


PROGRAM:
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.metrics import accuracy_score
import matplotlib.pyplot as plt
from sklearn.tree import plot_tree

# Load the Iris dataset


iris = load_iris()
X = iris.data
y = iris.target

# Split the data


X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3,
random_state=42)

# Create a Decision Tree Classifier


clf = DecisionTreeClassifier(random_state=42)

# Train the model


clf.fit(X_train, y_train)

# Make predictions
y_pred = clf.predict(X_test)

# Evaluate accuracy
accuracy = accuracy_score(y_test, y_pred)
print("Initial Accuracy:", accuracy)

# Define parameter grid for tuning


param_grid = {
'criterion': ['gini', 'entropy'],
'max_depth': [None, 5, 10, 15],
'min_samples_split': [2, 5, 10],
'min_samples_leaf': [1, 2, 4]
}
# Perform Grid Search with Cross-Validation
grid_search = GridSearchCV(estimator=clf, param_grid=param_grid, cv=5,
scoring='accuracy')
grid_search.fit(X_train, y_train)

# Print best parameters and score


print("Best Parameters:", grid_search.best_params_)
print("Best Score:", grid_search.best_score_)

# Train with best parameters


best_clf = DecisionTreeClassifier(**grid_search.best_params_)
best_clf.fit(X_train, y_train)

# Evaluate with best parameters


best_y_pred = best_clf.predict(X_test)
best_accuracy = accuracy_score(y_test, best_y_pred)
print("Accuracy with Best Parameters:", best_accuracy)

# Visualize the decision tree (optional)


plt.figure(figsize=(15, 10))
plot_tree(best_clf, feature_names=iris.feature_names,
class_names=iris.target_names, filled=True)
plt.show()

5.DEMOSTRATE DECISION TREE ALGORITHM


FOR A REGRESSION PROBLEM
AIM :
Write a program to demonstrate decision tree algorithm for a regression
problem.
Description:
Decision Tree Regression is a powerful tool for predicting continuous values
effectively capturing non-linear patterns in data.
Decision Tree Regression is a non-linear regression method that can handle
complex datasets with intricate patterns.
It uses a tree-like model to make predictions, making it both flexible and easy
to interpret.
ALGORITHM:
Step 1: Import necessary libraries
Step 2: Load and prepare the Iris dataset
Step 3: Use train_test_split() to divide the data into training and testing sets.
Step 4: Create a Decision Tree Regressor
Step 5: Train the decision tree regressor on the training data using regressor.fit()
Step 6: Use the trained model to make predictions on the test data using
regressor.predict()
Step 7: Calculate the Mean Squared Error (MSE) and Root Mean Squared Error
(RMSE) to evaluate the model's performance.
Step 8: Use plot_tree() to visualize the trained decision tree, making it easier to
understand the decision-making process
PROGRAM:
from sklearn.tree import DecisionTreeRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
from sklearn.datasets import load_iris
from sklearn.tree import plot_tree
import matplotlib.pyplot as plt
import numpy as np

# Load the Iris dataset


iris = load_iris()
# Select features and target
X = iris.data[:, 2].reshape(-1, 1) # Petal length as the feature
y = iris.data[:, 0] # Sepal length as the target

# Split data into training and testing sets


X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
# Create a Decision Tree Regressor
regressor = DecisionTreeRegressor(random_state=42)

# Train the model


regressor.fit(X_train, y_train)

# Make predictions
y_pred = regressor.predict(X_test)

# Evaluate the model


mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
print("Root Mean Squared Error:", rmse)

# Visualize the decision tree


plt.figure(figsize=(10, 6))
plot_tree(regressor, feature_names=["Petal Length"], filled=True)
plt.show()

VIVA QUESTIONS:
1. What is DecisionTreeClassifier and DecisionTreeRegressor in python.
2.What is the purpose of matplotlib library?
3. Describe Grid Search with Cross-Validation?
4.How to Visualize the decision tree?
5.Define Root Mean Squared Error.

You might also like