The k-Nearest Neighbors algorithm is a sim-

supervised
ple learning technique that classifies
data points based on the labels of their k
nearest neighbors in the training dataset.
Decision trees are a popular machine learning
technique used for classification and regres-
It operates on the principle of similarity, us- Definition and Mecha… tasks.
distance metrics like Euclidean or Manhattan sion They create a model that predicts the
ing value of a target variable based on several
distance to determine the nearest neighbors. input features.

Step 1: Select the number K of neighbors to

consider. Definition and Characteristics The model is represented as a tree structure,
where each internal node represents a fea-
each
ture, branch represents a decision rule, and
Step 2: Calculate the distance between the each leaf node represents an outcome.
target data point and all training data
points. k-Nearest Neighbors (k-NN)
Working of k-NN Overfitting: Decision trees can create overly
Step 3: Identify the k nearest neighbors Decision Trees complex models that do not generalize well to
on the calculated distances.
based unseen data.

Step 4: For classification, assign the class label Instability: Small changes in the dataset can
based on majority voting among the neigh- lead to a completely different tree structure,
bors. Limitations of Decision Trees making them sensitive to data variations.
Advantages: k-NN is easy to implement, adap-
easily
ts to new data, and requires few
hyperparameters. Bias in Imbalanced Datasets: Decision trees
favor
may the majority class if the dataset is
imbalanced, leading to poor performance on
Disadvantages: It does not scale well with large minority classes.
Advantages and Disadvantages of k-NN
datasets, is affected by the curse of
dimensionality, and is prone to overfitting due
to noise in the data.
Logistic regression is a statistical method used
for modeling the probability of a binary

Support Vector Machines are supervised Machine Learning outcome. It predicts the likelihood of an event
occurring by fitting data to a logistic function,
learning algorithms used for classification
regression.
and They find a hyperplane that Techniques: Overview which maps inputs to probabilities between 0
and 1.
separates data points of different classes with
the maximum margin. of Decision Trees,
Logistic Regression, Definition and Purpose It is primarily used in classification problems
where the goal is to predict categories, such as
The support vectors are the data points closest
to the hyperplane and are critical for defining
Overview and Functionality SVM, and k-NN spam detection or disease diagnosis.

the optimal decision boundary.

Binary Classification: It predicts one of two
possible outcomes, such as churn prediction or
When data is not linearly separable, SVM can fraud detection.
transform the input data into a higher-
dimensional space using nonlinear mapping.
This allows for finding a linear separating Multiclass Classification: Logistic regression
hyperplane in the new space. can be extended to multiclass problems using
Key Uses of Logistic Regression methods like One-vs-All or Softmax Regres-
Logistic Regression sion.
Kernel functions facilitate this transformation, Handling Nonlinear… Support Vector Machines (SVM) Interpretability: The coefficients in logistic
enabling SVM to handle complex relation- regression provide insights into how input
without
ships explicitly mapping data to higher variables influence the target variable.
dimensions.

Confusion Matrix: A table summarizing the

Advantages: SVM performs well in high- model's predictions against true labels,
dimensional spaces, is resilient to outliers, consisting of true positives, false positives,
supports
and both binary and multiclass negatives, and false negatives.
classification. true

Advantages and Disadvantages of SVM Accuracy: The proportion of correctly

Disadvantages: SVM can be slow for large classified instances out of the total instances,
datasets, requires careful parameter tun- suitable for balanced datasets.
and Metrics for Model Performance
ing, is sensitive to noise in the data.

Precision and Recall: Precision measures the

accuracy of positive predictions, while recall
measures the ability to identify all positive
instances.