Scribd
Upload
5 views5 pages

DWM Exp4

The document outlines an experiment focused on implementing classification algorithms, specifically Decision Tree and Naïve Bayes, using Python. It details the procedure for loading and preprocessing data, training classifiers, and evaluating their performance. The conclusion highlights the strengths of each algorithm, noting that the choice depends on dataset characteristics.

Uploaded by

Mayank vora
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
5 views5 pages

DWM Exp4

The document outlines an experiment focused on implementing classification algorithms, specifically Decision Tree and Naïve Bayes, using Python. It details the procedure for loading and preprocessing data, training classifiers, and evaluating their performance. The conclusion highlights the strengths of each algorithm, noting that the choice depends on dataset characteristics.

Uploaded by

Mayank vora
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

DWM Exp.

Name - Reshab Singh


Class - TE-03 / 57
Batch - C

Aim: Implementation of Classification Algorithm (Decision Tree / Naïve


Bayes) using Python

Introduction:

Classification algorithms are used in machine learning to categorize data


into predefined classes. Decision Tree and Naïve Bayes are two popular
classification techniques. Decision Trees use a hierarchical model of
decisions based on feature values, while Naïve Bayes applies probabilistic
principles based on Bayes' theorem assuming feature independence.

Procedure:

1. Load the dataset.


2. Preprocess and split the data into training and testing sets.
3. Standardize the features (optional for better performance).
4. Train a Decision Tree classifier and a Naïve Bayes classifier.
5. Evaluate the models using accuracy scores and classification reports.
6. Compare the results and analyze the findings.

Program Codes:
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.tree import DecisionTreeClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import accuracy_score, classification_report

# Load dataset
data = load_iris()
X, y = data.data, data.target
# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)

# Standardize features (optional but can help some algorithms)


scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# Decision Tree Classifier


dt_model = DecisionTreeClassifier()
dt_model.fit(X_train, y_train)
dt_predictions = dt_model.predict(X_test)

# Naïve Bayes Classifier


nb_model = GaussianNB()
nb_model.fit(X_train, y_train)
nb_predictions = nb_model.predict(X_test)

# Evaluate models
print("Decision Tree Accuracy:", accuracy_score(y_test, dt_predictions))
print("Decision Tree Classification Report:\n",
classification_report(y_test, dt_predictions))

print("Naïve Bayes Accuracy:", accuracy_score(y_test, nb_predictions))


print("Naïve Bayes Classification Report:\n",
classification_report(y_test, nb_predictions))
Conclusion: Both Decision Tree and Naïve Bayes classifiers are effective
for classification tasks. Decision Trees provide an interpretable structure,
whereas Naïve Bayes is computationally efficient and works well with small
datasets. The choice between these depends on the dataset characteristics
and requirements.
Review Questions:

1. What is a Decision Tree classifier, and how does it work?

Ans. A Decision Tree classifier is a supervised learning algorithm used


for classification and regression tasks. It works by splitting the dataset into
smaller subsets based on feature values, forming a tree-like structure. The
decision tree consists of:

● Root Node: Represents the entire dataset and the first feature split.
● Internal Nodes: Represent decision rules applied to split the data.
● Leaf Nodes: Represent the final class labels.

The tree uses splitting criteria such as Gini impurity or Entropy


(Information Gain) to determine the best feature at each step. The
algorithm continues splitting until a stopping condition is met, such as
reaching a maximum depth or when further splits do not improve accuracy.

2.Explain the Naïve Bayes algorithm and its underlying assumptions.

Ans. The Naïve Bayes classifier is a probabilistic machine learning model


based on Bayes' theorem, which calculates the probability of a class given
a set of features. The algorithm is called “naïve” because it assumes that
all features are independent, which is often not true in real-world data.

Types of Naïve Bayes classifiers:


● Gaussian Naïve Bayes (for continuous data, assumes normal
distribution).
● Multinomial Naïve Bayes (for text classification and discrete data).
● Bernoulli Naïve Bayes (for binary/boolean features).

3. Compare the working principles of Decision Tree and Naïve Bayes


classifiers.
Ans.

4. What are the different types of Decision Tree splitting criteria?

Ans.

You might also like