1.

Given the data below:

Weather S O R S S O R R S R S O O R

Play No Yes Yes Yes Yes Yes No No Yes Yes No Yes Yes No

S: Sunny O: Overcast R: Rainy

i) Calculate the probability of playing when the weather is overcast
ii) Calculate the probability of playing when the weather is sunny.
Ans) Given Data:
Weather: S, O, R, S, S, O, R, R, S, R, S, O, O, R
Play: No, Yes , Yes, Yes, Yes, Yes, No, No, Yes, Yes, No, Yes, Yes, No
Legend:S = Sunny , O = Overcast , R = Rainy
Step 1: Organize the data by weather type
Let’s count how many times each weather occurs and how many times "Play = Yes" for each
type.
Sunny (S):
Positions: 1, 4, 5, 9, 11
Play: No, Yes, Yes, Yes, No
• Total Sunny = 5
• Play = Yes when Sunny = 3
Overcast (O):
Positions: 2, 6, 12, 13
Play: Yes, Yes, Yes, Yes
• Total Overcast = 4
• Play = Yes when Overcast = 4
Rainy (R):Positions: 3, 7, 8, 10, 14
Play: Yes, No, No, Yes, No
Total Rainy = 5
• Play = Yes when Rainy = 2

i) Probability of Playing when Weather is Overcast

ii) Probability of Playing when Weather is Sunny

2. Discuss Support Vector Machine (SVM)? Explain the concept of Large
Margin Classifiers and how Kernel Functions help in handling nonlinear
data.? Support Vector Machine (SVM) SVM is a supervised machine
learning algorithm mainly used for classification (and sometimes regression).
It works by finding the best boundary (called a hyperplane) that separates
different classes in the data. Key Concepts:Hyperplane: A decision
boundary that separates data points of different classes. In 2D, it’s a line; in 3D,
it’s a plane.Support Vectors: The data points closest to the hyperplane. These
points are most critical in defining the position and orientation of the
hyperplane.Margin: The distance between the hyperplane and the nearest data
points from both classes.Large Margin Classifier SVM is a large margin
classifier, which means:It chooses the hyperplane that maximizes the margin
between the two classes.A larger margin leads to better generalization on unseen
data, reducing overfitting.Why is the margin important?A small change in
input should not drastically affect the output. A large margin ensures that the
classifier is robust and stable to slight variations. Handling Nonlinear Data:
The Need for Kernel Functions Sometimes data is not linearly separable,
meaning we can't draw a straight line (or plane) to separate the classes. Enter:
Kernel Functions Kernel Functions allow SVM to map the data into a higher-
dimensional space where it becomes linearly separable.
3.Analyze Principal Component Analysis (PCA) with mathematical derivation. Discuss
the steps of PCA algorithm with its applications and why is it useful in machine
learning?Ans Principal Component Analysis (PCA) is a powerful statistical technique used in
machine learning and data science for dimensionality reduction. It transforms a high-
dimensional dataset into a lower-dimensional space while preserving as much of the original
variability (information) as possible. This is especially useful when dealing with large datasets
where many features are correlated or redundant, as it helps in reducing noise, improving model
performance, and visualizing complex data in 2D or 3D.Steps of the PCA Algorithm
Standardize the Data: Subtract the mean and divide by the standard deviation for each feature
to ensure all features are on the same scale.Compute the Covariance Matrix: This matrix
represents how features vary with respect to each other.Calculate Eigenvalues and
Eigenvectors: These are derived from the covariance matrix to identify principal
components.Sort and Select Top k Components: Choose the eigenvectors with the largest
eigenvalues (those with the most variance).Project the Data: Transform the original data to
the new subspace formed by the top k eigenvectors.Applications of PCA Image
Compression: Reduces image size while retaining key features.Face Recognition:
Identifies patterns in facial features using fewer components.Bioinformatics:
Analyzes gene expression data efficiently. Preprocessing: Reduces features
before applying ML models to enhance performance.Why PCA is Useful in
Machine Learning Reduces Overfitting by eliminating noise and redundant
features.
4. Discuss the working of the Naïve Bayes Classifier with an example. What
are its advantages and limitations?
Ans) The Naïve Bayes Classifier is a probabilistic machine learning algorithm
based on Bayes' Theorem. It is called "naïve" because it assumes that the features
are independent of each other — which is rarely
true in real-
world data, but the algorithm still
performs surprisingly well in
many applications.
The algorithm works by first
calculating the prior probabilities of each class from the training data. ->Then,
for a given input, it calculates the likelihood of that input belonging to each class
by multiplying the conditional probabilities of the features given that class. Using
Bayes' Theorem, it computes the posterior probability for each class and selects
the one with the highest probability as the predicted class. Advantages of Naïve
Bayes Easy to implement and computationally efficient Performs well with large
datasets and high-dimensional data Works particularly well for text
classification problems Limitations of Naïve Bayes Assumes independence
among features, which is rarely true May perform poorly when features are
correlated
5. Discuss different Kernel Functions used in SVM. Explain their role in
handling non-linearly separable data.Ans) In Support Vector Machines
(SVM), Kernel Functions play a critical role in enabling the model to classify
data that is not linearly separable in its original feature space. When data cannot
be separated using a straight line or hyperplane, SVM uses a technique called the
"kernel trick" to implicitly map the data into a higher-dimensional space
where a linear separator can be found. This transformation is done without
explicitly calculating the new coordinates, making the process computationally
efficient.Commonly Used Kernel Functions in SVM
Linear Kernel

Used when the data is linearly separable.

o Fast and requires fewer computational resources.
2. Polynomial Kernel

o Maps input features into a polynomial feature space.

o Useful when the data has curved boundaries.
o Parameters: c (coefficient), d (degree of the polynomial).
Role of Kernel Functions in Nonlinear Data Handling
• Enable nonlinear classification: By mapping input features to a higher-
dimensional space where linear separation becomes possible.
• Avoid explicit transformation: The kernel trick lets us operate in high
dimensions without the computational cost.
• Model complex decision boundaries: Especially useful for datasets with
overlapping or circular patterns.
1. Compare Agglomerative and Divisive Hierarchical Clustering. How
do they work, and which one is more efficient?
Ans)

The steps of Agglomerative Clustering are:

1. Treat each data point as a cluster.
2. Find the two closest clusters based on a similarity measure (e.g.,
single linkage, complete linkage, average linkage).
3. Merge the two closest clusters.
4. Repeat steps 2-3 until only one cluster remains or the desired number
of clusters is achieved.
The steps of Divisive Clustering are:
1. Start with all data points in one cluster.
2. Divide the cluster into two subclusters based on some splitting
criterion (often based on maximizing the separation between them).
3. Repeat step 2 recursively for each new subcluster until each data point
is its own cluster or the desired number of clusters is achieved.
7. Discuss evaluation metrics with its type and importance.?Ans)Evaluation
metrics are crucial for assessing the performance of machine learning models.
These metrics help determine how well the model is performing, allowing data
scientists and engineers to make informed decisions regarding model
improvement or selection.Types of Evaluation Metrics:Classification Metrics:
Used for models predicting categorical outcomes.Accuracy: Measures the
overall correctness of the model. It’s the ratio of correct predictions to the total
number of predictions.Precision: Focuses on the positive predictions. It measures
the proportion of correct positive predictions out of all predicted positives.Recall
(Sensitivity): Measures the proportion of actual positives correctly identified by
the model.F1 Score: The harmonic mean of precision and recall, useful for
imbalanced data.Regression Metrics: Used for models predicting continuous
outcomes.Mean Absolute Error (MAE): Average of the absolute differences
between predicted and actual values.Mean Squared Error (MSE): Average of
squared differences between predicted and actual values. Penalizes larger errors
more.Root Mean Squared Error (RMSE): Square root of MSE, providing a
measure of error in the same unit as the output.R-squared (R²): Measures the
proportion of variance in the target variable that is explained by the
model.Importance:Evaluation metrics help in:Model Selection: Choosing the
best-performing model for a given task.Model Improvement: Identifying areas
where the model can be optimized or improved.Handling Imbalanced Data:
Metrics like F1 score and AUC-ROC are useful for imbalanced datasets where
accuracy may not provide a true reflection of performance.
7).What is Bayesian Concept Learning? Explain its advantages,
disadvantages and common applications.Ans) Bayesian Concept
Learning:Bayesian Concept Learning is a probabilistic approach to machine
learning that uses Bayes' Theorem to make predictions based on prior knowledge
and observed data. It learns a concept or classification by updating its belief about
the probability of a class, given the observed evidence. The core idea is to
estimate the posterior probability P(C∣X)P(C|X)P(C∣X) (the probability of a class
given some features), using the prior probability P(C)P(C)P(C) and the likelihood
P(X∣C)P(X|C)P(X∣C) of observing the features given the
class.Advantages:Incorporates Prior Knowledge Handles Uncertainty
AdaptableDisadvantages:Computationally Expensive Requires Prior
KnowledgeAssumes Independence Common Applications:Spam Filtering
Medical Diagnosis Robotics
8. Explain the concept of Large Margin Classifiers in
SVM.?Ans)1.Maximizing the Margin: The concept of Large Margin
Classifiers in Support Vector Machines (SVM) is centered around finding the
hyperplane that maximizes the margin between different classes. The margin is
the distance between the closest data points (support vectors) from either class to
the hyperplane.2.Support Vectors: The data points that are closest to the
hyperplane are called support vectors. These support vectors are critical in
defining the decision boundary of the classifier. Only the support vectors
contribute to the final model.3.Optimal Hyperplane: The goal of an SVM is to
find the optimal hyperplane that separates the data points of different classes
while ensuring the margin (distance between the hyperplane and the support
vectors) is as large as possible. This separation leads to better generalization and
less risk of overfitting.4.Hard vs. Soft Margin: In real-world scenarios, data may
not be perfectly separable. Hard Margin SVM works when the data is linearly
separable, while Soft Margin SVM allows for some misclassifications, adding a
penalty for each misclassification to balance between margin size and
classification error.5.Better Generalization: By maximizing the margin, Large
Margin Classifiers reduce the model’s complexity and improve generalization.
A larger margin leads to a lower risk of overfitting, as the classifier is less
sensitive to small fluctuations in the training data.
9. Describe the K-Nearest Neighbors (KNN) algorithm and its working
mechanism.Ans) K-Nearest Neighbors (KNN) is a simple, non-parametric, and
instance-based machine learning algorithm used for classification and regression
tasks. The algorithm works by finding the K closest training samples to a given
test point based on a distance metric (typically Euclidean distance) and then
assigning the most frequent class (in classification) or the average value (in
regression) of those K neighbors to the test point.Working Mechanism:Choose
the number of neighbors (K): Decide on the number of neighbors to consider
for making predictions.Calculate the distance: For each test point, calculate the
distance from all training data points.Identify K closest points: Identify the K
data points that are closest to the test point.Classify or Predict:For classification:
Assign the test point to the most common class among the K neighbors.For
regression: Compute the average (or weighted average) of the target values of the
K neighbors.Return the result: Output the predicted class (classification) or
value (regression) for the test point.KNN is easy to understand and implement
but can become computationally expensive with large datasets, as it requires
storing all training data and calculating distances during prediction.
10. Explain the steps involved in Principal Component Analysis (PCA).Ans)
Steps in Principal Component Analysis (PCA):Standardize the Data: Scale
the features to have zero mean and unit variance.Calculate the Covariance
Matrix: Find the covariance between each pair of features to understand their
relationships.Compute Eigenvalues and Eigenvectors: Solve for eigenvalues
and eigenvectors of the covariance matrix.Sort Eigenvalues and Eigenvectors:
Arrange the eigenvectors in descending order based on their corresponding
eigenvalues.Select Top k Eigenvectors: Choose the top k eigenvectors with the
highest eigenvalues to form a new feature space.Project the Data: Project the
original data onto the selected eigenvectors, reducing the
dimensionality.Interpret Results: Use the transformed data for analysis or
machine learning tasks.
11. Explain k-Fold Cross-Validation with an example.Ans) k-Fold Cross-
Validation is a model evaluation technique used to assess the performance of a
machine learning model by partitioning the data into k subsets (folds). The model
is trained and tested k times, with each fold used as the test set exactly once, and
the remaining k-1 folds are used as the training set. This provides a better estimate
of the model’s performance by reducing variability.Steps Involved:Split the
Dataset: Divide the dataset into k equally sized subsets (or folds).Iterate k
Times: For each fold, use the remaining k-1 folds for training and the current fold
for testing.Evaluate Performance: For each iteration, calculate the model’s
performance metrics (like accuracy, F1 score, etc.).Average the Results: After
all k iterations, average the performance metrics to get an overall evaluation of
the model.Example:Suppose we have a dataset with 100 data points, and we
choose k = 5 (5-fold cross-validation).Split the data into 5 folds, each with 20
data points.First iteration: Train the model on folds 2, 3, 4, and 5, then test it on
fold 1.Second iteration: Train on folds 1, 3, 4, and 5, then test on fold 2.Repeat
this process for all 5 iterations.After all iterations, compute the average
performance across all folds.
12. Define and differentiate Precision, Recall, and F1-score in classification.
Ans) Precision, Recall, and F1-Score (Short Explanation)
• Precision:
o Formula:
o Focus: The proportion of predicted positives that are actually
correct.
o Use: Important when false positives are costly (e.g., in spam
detection).
• Recall:
o Formula:
o Focus: The proportion of actual positives that are correctly
identified.
o Use: Important when false negatives are costly (e.g., in medical
diagnoses).
• F1-Score:
o Formula:
o Focus: The harmonic mean of precision and recall.
o Use: Useful when both false positives and false negatives are
important and need balancing.
13. What are Model Evaluation Metrics, and why are they important in
machine learning?Ans) Model Evaluation Metrics are quantitative measures
used to assess the performance of a machine learning model. These metrics help
to determine how well the model is making predictions and whether it can
generalize well to unseen data.Importance of Model Evaluation
Metrics:Performance Measurement: They provide a concrete way to evaluate
how well a model is performing, such as its accuracy, precision, recall, or error
rate.Comparison of Models: Evaluation metrics allow comparison between
different models or algorithms to select the best-performing one for a given
problem.Identifying Overfitting or Underfitting: They help detect overfitting
(where the model performs well on training data but poorly on test data) or
underfitting (where the model performs poorly on both training and test
data).Model Improvement: By analyzing these metrics, data scientists can
identify areas for improvement, such as tuning model hyperparameters or
adjusting the training data.Choosing the Right Metric: Different tasks require
different evaluation metrics. For example, precision and recall are critical in
classification problems with imbalanced classes, while mean squared error
(MSE) is more useful for regression tasks.
14. What is the formula for calculating probability in Naïve Bayes
Classifier?
Ans)

15. What is the role of kernel functions in SVM?

Ans) Kernel functions in SVM transform data into a higher-dimensional space
where it is easier to find a separating hyperplane. They allow SVM to handle
non-linearly separable data without explicitly calculating the transformation.
Common kernels include linear, polynomial, and radial basis function (RBF).
16. Define K-Nearest Neighbors (KNN) Algorithm.
Ans) The K-Nearest Neighbors (KNN) algorithm is a simple, instance-based
classification method that predicts the class of a data point based on the
majority class of its K closest neighbors. The distance between points is
typically measured using Euclidean distance. KNN is non-parametric and
doesn't require training, making it easy to implement but computationally
expensive during prediction.
17. Write two key points of generative models for discrete data.
Ans) 1. Modeling the Joint Probability
2. Conditional Probability
18. Centroid in K-Means Clustering:A centroid in K-Means clustering is the
center of a cluster, calculated as the mean of all data points assigned to that cluster.
It represents the "average" point in a cluster and is used to update cluster
assignments in each iteration.
19.Difference Between Agglomerative and Divisive Clustering:Agglomerative
clustering is a bottom-up approach where each data point starts as its own cluster
and merges iteratively. Divisive clustering is a top-down approach, starting with
one large cluster and recursively splitting it into smaller ones.
20.Dendrogram in Hierarchical Clustering:A dendrogram is a tree-like
diagram used to visualize the hierarchical clustering process, showing how
clusters are merged or split at each step, and it helps determine the optimal
number of clusters.
21.Main Advantage of PCA:The main advantage of PCA is dimensionality
reduction, which helps to reduce the number of features while retaining most of
the variance in the data, making the model simpler and faster.
22.Gaussian Mixture Model (GMM):A Gaussian Mixture Model is a
probabilistic model that assumes all data points are generated from a mixture of
several Gaussian distributions, each representing a cluster. GMM is flexible and
can model elliptical clusters.
23.Expectation-Maximization (EM) Algorithm:The EM algorithm is an
iterative method used to find maximum likelihood estimates of parameters in
models with latent variables. It alternates between an expectation step (E-step)
and a maximization step (M-step) to optimize the model.
24.Cross-Validation:Cross-validation is a model validation technique where the
dataset is split into multiple subsets, and the model is trained and tested on
different folds to ensure its performance generalizes well to unseen data.

25.Hyperparameter Tuning:Hyperparameter tuning is the process of finding the

optimal set of hyperparameters for a machine learning model to improve its
performance. It is often done using techniques like grid search or random search.
26.Confusion Matrix:A confusion matrix is a table used to evaluate the
performance of a classification model, showing the counts of true positives, true
negatives, false positives, and false negatives. It helps calculate metrics like
accuracy, precision, recall, and F1-score.
27.F1-Score and Its Importance: The F1-score is the harmonic mean of
precision and recall, providing a balance between the two. It is especially
important when dealing with imbalanced datasets where both false positives and
false negatives matter.
28.Laplace Smoothing: Laplace smoothing is a technique used to smooth
probability estimates in probabilistic models, particularly in text classification. It
adds a small constant (usually 1) to the frequency count to avoid zero probabilities
for unseen events.