Assessing a Single Classification Algorithm and Two Classification Algorithms
Assessing a Single Classification Algorithm and Two Classification Algorithms
CLASSIFICATION ALGORITHMS
Unsupervised Reinforcement
Feature Supervised Learning
Learning Learning
Learns by interacting
Learns from unlabeled
Definition Learns from labeled data with environment
data
through rewards
Learn optimal actions
Predict outcomes Discover hidden
Goal to maximize reward
(classification/regression) patterns or structure
over time
States, actions, and
Input Data Labeled (input-output pairs) Unlabeled (only inputs) rewards from
environment
Groupings, associations, Policy or value
Output Predicted label or value
or structure function
Q-learning, Deep Q-
Linear regression, SVM, K-means, PCA,
Networks (DQN),
Examples decision trees, neural hierarchical clustering,
policy gradient
networks autoencoders
methods
1
Unsupervised Reinforcement
Feature Supervised Learning
Learning Learning
Silhouette score,
Accuracy, precision, recall, Cumulative reward,
Evaluation cohesion/separation
F1-score convergence rate
metrics
Robotics, game
Customer segmentation,
Application Spam detection, medical playing,
anomaly detection, data
Domains diagnosis, stock prediction recommendation
compression
systems
Moderate — no need High — needs many
Dependency High — needs large labeled
for labels, but quality of interactions with
on Data datasets
insights varies environment
Complex, involves
Usually faster, but
Training Generally straightforward, exploration vs.
interpretation can be
Complexity but depends on algorithm exploitation trade-
tricky
offs
A supervised learning algorithm is a type of machine learning algorithm that learns from
labeled training data to make predictions or decisions.
2
LINEAR REGRESSION
y=mx+b
Where:
y= predicted output
x= input feature
m = slope (coefficient)
b= intercept
y=b0+b1x1+b2x2+⋯+bnxn
Types of Linear Regression:
Advantages:
Disadvantages:
Sensitive to outliers
Can underfit if the true relationship is non-linear
Performance drops if features are not scaled or normalized
3
Logistic Regression:
Logistic Regression is a supervised learning algorithm used for classification tasks. It predicts
the probability that a given input belongs to a particular class (typically binary), using a logistic
(sigmoid) function to map linear combinations of input features to a probability between 0 and
1.
Logistic regression estimates the probability that a data point belongs to a class using the
formula:
1
𝑝( 𝑦 = 1 | 𝑥 ) =
1+ 𝑒 −(𝑏0+𝑏1𝑥1+𝑏2𝑥2….+𝑏𝑛𝑥𝑛)
Where:
Advantages:
Disadvantages:
4
SUPPORT VECTOR MACHINE (SVM)
A Support Vector Machine (SVM) is a supervised machine learning algorithm used for
classification and sometimes regression tasks. SVM aims to find the optimal hyperplane that
maximally separates data points of different classes in the feature space.
Key Concept:
SVM finds a decision boundary (hyperplane) that has the maximum margin between
the two classes.
The support vectors are the data points closest to the hyperplane—they define the
margin.
Aspect SVM
Task Classification, Regression
Linear? Works for both linear and non-linear data
Key Feature Maximizing margin between classes
Sensitive To Parameter tuning (C, gamma), kernel choice
Strengths High accuracy, handles high-dimensional data well
Weaknesses Computationally expensive, less interpretable
Types
1. Linear SVM
o Used when data is linearly separable
o Finds a straight-line hyperplane in 2D, or a flat hyperplane in higher
dimensions
2. Non-linear SVM (using Kernel Trick)
o Used when data is not linearly separable
o Transforms data into a higher-dimensional space using kernels to make it
separable
3. Support Vector Regression (SVR)
o Applies the SVM principles to regression problems rather than classification
Advantages:
Disadvantages:
5
Not suitable for very large datasets
Difficult to interpret and tune (e.g., kernel choice, parameters like C and gamma)
NAIVE BAYES
Naive Bayes is a supervised learning algorithm based on Bayes’ Theorem with a naive
assumption that features are independent given the class label
Advantages:
Disadvantages:
6
DECISION TREE
A Decision Tree is a supervised learning algorithm used for both classification and
regression tasks. It works by splitting the dataset into subsets based on the feature values to
form a tree-like structure where each node represents a feature, each branch a decision rule,
and each leaf node a final output.
1. Classification Trees
o Used when the target variable is categorical
o Example: Classifying an email as spam or not
2. Regression Trees
o Used when the target variable is continuous
o Example: Predicting house price
Advantages:
Disadvantages:
7
RANDOM FOREST
Advantages:
Disadvantages:
8
K-NEAREST NEIGHBORS (KNN)
Advantages:
Disadvantages:
9
UNSUPERVISED LEARNING
Key Characteristics:
1. Clustering
o Groups similar data points together
o Example: K-Means, Hierarchical Clustering, DBSCAN
2. Dimensionality Reduction
o Reduces the number of input variables
o Example: PCA (Principal Component Analysis), t-SNE
3. Association Rule Learning
o Finds relationships between variables
o Example: Apriori, Eclat (used in market basket analysis)
K-MEANS CLUSTERING
10
4. Recalculate the centroids as the mean of assigned points
5. Repeat steps 3–4 until centroids don’t change significantly (convergence)
Advantages:
Disadvantages:
11
The model is typically trained using the Expectation-Maximization (EM) algorithm.
1. Spherical GMM
o Each component has the same variance in all directions.
2. Diagonal GMM
o Each component has a diagonal covariance matrix (features are uncorrelated).
3. Full GMM
o Each component has a full covariance matrix (features can be correlated).
4. Tied GMM
o All components share the same covariance matrix.
Advantages:
Flexible clustering – can model elliptical clusters (unlike K-Means' circular ones)
Probabilistic approach – gives soft assignments (probabilities of belonging to each
cluster)
Works well when clusters overlap
More powerful than K-Means for complex distributions
Disadvantages:
12