ARTIFICIAL INTELLIGENCE
WEEK 06
LECTURE 12
TOPICS TO COVER IN THIS LECTURE
• Unsupervised machine learning
• Challenges
• Advantages
• Disadvantages
• Un-supervised machine learning techniques
• Clustering
• Associative Rule Learning
• Dimensionality reduction
• Clustering techniques
• K means alogrithm
SUPERVISED MACHINE LEARNING
• Supervised machine learning algorithms are techniques
in which models are trained using labeled data under the
supervision of training data. But there may be many
cases in which we do not have labeled data and need to
find the hidden patterns from the given dataset.
• So, to solve such types of cases in machine learning, we
need unsupervised learning techniques.
U N S U P E RV I S E D M AC H I N E L E A R N I N G
• Unsupervised learning is a branch of machine learning
that deals with unlabeled data. Unlike supervised
learning, where the data is labeled with a specific
category or outcome, unsupervised learning algorithms
are tasked with finding patterns and relationships within
the data without any prior knowledge of the data’s
meaning. This makes unsupervised learning a powerful
tool for exploratory data analysis, where the goal is to
understand the underlying structure of the data.
• In artificial intelligence, machine learning that takes place
in the absence of human supervision is known as
unsupervised machine learning. Unsupervised machine
learning models, in contrast to supervised learning, are
given unlabeled data and allow discover patterns and
insights on their own—without explicit direction or
instruction.
• Unsupervised machine learning analyzes and clusters
unlabeled datasets using machine learning algorithms.
These algorithms find hidden patterns and data without
any human intervention, i.e., we don’t give output to our
model. The training model has only input parameter
values and discovers the groups or patterns on its own.
How does unsupervised learning work?
• Unsupervised learning works by analyzing unlabeled data to
identify patterns and relationships.
• The data is not labeled with any predefined categories or
outcomes, so the algorithm must find these patterns and
relationships on its own.
• This can be a challenging task, but it can also be very
rewarding, as it can reveal insights into the data that would not
be apparent from a labeled dataset.
• Data-set in Figure A is Mall data that contains information about
its clients that subscribe to them.
• Once subscribed they are provided a membership card and the
mall has complete information about the customer and his/her
every purchase.
• Now using this data and unsupervised learning techniques, the
The input to the unsupervised learning models is as
follows:
•Unstructured data: May contain
noisy(meaningless) data, missing values, or
unknown data
•Unlabeled data: Data only contains a value for
input parameters, there is no targeted
value(output). It is easy to collect as compared to
the labeled one in the Supervised approach.
Unsupervised Learning Algorithms
There are mainly 3 types of Algorithms
which are used for Unsupervised dataset.
•Clustering
•Association Rule Learning
•Dimensionality Reduction
CLUSTERING
• Clustering in unsupervised machine learning is the process of grouping unlabeled
data into clusters based on their similarities. The goal of clustering is to identify
patterns and relationships in the data without any prior knowledge of the data’s
meaning.
• Broadly this technique is applied to group data based on different patterns, such
as similarities or differences, our machine model finds. These algorithms are used
to process raw, unclassified data objects into groups. For example, in the above
figure, we have not given output parameter values, so this technique will be used
to group clients based on the input parameters provided by our data.
• Some common clustering algorithms
• K-means Clustering: Partitioning Data into K Clusters
• Hierarchical Clustering: Building a Hierarchical Structure of Clusters
• Density-Based Clustering (DBSCAN): Identifying Clusters Based on Density
• Mean-Shift Clustering: Finding Clusters Based on Mode Seeking
• Spectral Clustering: Utilizing Spectral Graph Theory for Clustering
A SS O C I AT I O N R U L E L E A R N I N G
• Association rule learning is also known as association rule mining is a
common technique used to discover associations in unsupervised machine
learning. This technique is a rule-based ML technique that finds out some very
useful relations between parameters of a large data set. This technique is
basically used for market basket analysis that helps to better understand the
relationship between different products. For e.g. shopping stores use
algorithms based on this technique to find out the relationship between the
sale of one product w.r.t to another’s sales based on customer behavior. Like if
a customer buys milk, then he may also buy bread, eggs, or butter. Once
trained well, such models can be used to increase their sales by planning
different offers.
• Apriori Algorithm: A Classic Method for Rule Induction
• FP-Growth Algorithm: An Efficient Alternative to Apriori
• Eclat Algorithm: Exploiting Closed Itemsets for Efficient Rule Mining
DIMENSIONALITY REDUCTION
• Dimensionality reduction is the process of reducing the number of features in a
dataset while preserving as much information as possible. This technique is useful
for improving the performance of machine learning algorithms and for data
visualization.
• Examples of dimensionality reduction algorithms include Dimensionality reduction
is the process of reducing the number of features in a dataset while preserving as
much information as possible.
• Principal Component Analysis (PCA): Linear Transformation for Reduced
Dimensions
• Linear Discriminant Analysis (LDA): Dimensionality Reduction for Discrimination
• Non-negative Matrix Factorization (NMF): Decomposing Data into Non-negative
Components
• Locally Linear Embedding (LLE): Preserving Local Geometry in Reduced
Dimensions
CHALLENGES OF UNSUPERVISED LEARNING
Here are the key challenges of unsupervised learning
• Evaluation: Assessing the performance of unsupervised learning algorithms is
difficult without predefined labels or categories.
• Interpretability: Understanding the decision-making process of unsupervised
learning models is often challenging.
• Overfitting: Unsupervised learning algorithms can overfit to the specific dataset
used for training, limiting their ability to generalize to new data.
• Data quality: Unsupervised learning algorithms are sensitive to the quality of
the input data. Noisy or incomplete data can lead to misleading or inaccurate
results.
• Computational complexity: Some unsupervised learning
algorithms, particularly those dealing with high-dimensional data or large
datasets, can be computationally expensive.
ADVANTAGES OF UNSUPERVISED LEARNING
• No labeled data required: Unlike supervised
learning, unsupervised learning does not require labeled
data, which can be expensive and time-consuming to collect.
• Can uncover hidden patterns: Unsupervised learning
algorithms can identify patterns and relationships in data that
may not be obvious to humans.
• Can be used for a variety of tasks: Unsupervised learning can
be used for a variety of tasks, such as clustering, dimensionality
reduction, and anomaly detection.
• Can be used to explore new data: Unsupervised learning can
be used to explore new data and gain insights that may not be
possible with other methods.
D I S A D VA N TA G E S O F U N S U P E R V I S E D L E A R N I N G
• Difficult to evaluate: It can be difficult to evaluate the performance of
unsupervised learning algorithms, as there are no predefined labels or
categories against which to compare results.
• Can be difficult to interpret: It can be difficult to understand the decision-
making process of unsupervised learning models.
• Can be sensitive to the quality of the data: Unsupervised learning
algorithms can be sensitive to the quality of the input data. Noisy or
incomplete data can lead to misleading or inaccurate results.
• Can be computationally expensive: Some unsupervised learning
algorithms, particularly those dealing with high-dimensional data or large
datasets, can be computationally expensive
APPLICATIONS
• Customer segmentation: Unsupervised learning can be used to segment customers
into groups based on their demographics, behavior, or preferences. This can help
businesses to better understand their customers and target them with more relevant
marketing campaigns.
• Fraud detection: Unsupervised learning can be used to detect fraud in financial data
by identifying transactions that deviate from the expected patterns. This can help to
prevent fraud by flagging these transactions for further investigation.
• Recommendation systems: Unsupervised learning can be used to recommend items
to users based on their past behavior or preferences. For example, a recommendation
system might use unsupervised learning to identify users who have similar taste in
movies, and then recommend movies that those users have enjoyed.
• Natural language processing (NLP): Unsupervised learning is used in a variety of
NLP tasks, including topic modeling, document clustering, and part-of-speech tagging.
• Image analysis: Unsupervised learning is used in a variety of image analysis
tasks, including image segmentation, object detection, and image pattern recognition.
K-MEANS CLUSTERING ALGORITHM
• K means clustering, assigns data points to one of the K clusters depending on their
distance from the center of the clusters.
• It starts by randomly assigning the clusters centroid in the space.
• Centroid is a data point that represents the center of the cluster.
• Then each data point assign to one of the cluster based on its distance from
centroid of the cluster.
• After assigning each point to one of the cluster, new cluster centroids are
assigned.
• This process runs iteratively until it finds good cluster.
• In the analysis we assume that number of cluster is given in advanced and we
have to put points in one of the group.
• In some cases, K is not clearly defined, and we have to think about the optimal
number of K. K Means clustering performs best data is well separated.
• When data points overlapped this clustering is not suitable. K Means is faster as
• It provides strong coupling between the data points. K Means
cluster do not provide clear information regarding the quality of
clusters.
• Different initial assignment of cluster centroid may lead to
different clusters.
• Also, K Means algorithm is sensitive to noise. It may have stuck
in local minima.
WHAT IS THE OBJECTIVE OF K-MEANS CLUSTERING?
• The goal of clustering
is to divide the
population or set of
data points into a
number of groups so
that the data points
within each group are
more comparable to
one another and
different from the
data points within the
How do we solve this optimization problem? We can’t just
use gradient descent because there are discrete variables
(assignment variables zi). We can’t really use dynamic
programming because there are continuous variables (the
centroids µk). • To motivate the solution, consider a
simple example with four points. As always, let’s try to
break up the problem into subproblems. • What if we
knew the optimal centroids? Then computing the
assignment vectors is trivial (for each point, choose the
closest center). • What if we knew the optimal
assignments? Then computing the centroids is also trivial
(one can check that this is just averaging the points
assigned to that center). • The only problem is that we
NUMERICAL EXAMPLE
Point Coordinates
Input A1 (2,10)
Dataset A2 (2,6)
A3
we need to make 3 clusters (11,11)
A4 (6,9)
A5 (6,4)
A6 (1,2)
A7 (5,10)
A8 (4,9)
A9 (10,12)
A10 (7,5)
A11 (9,11)
A12 (4,6)
A13 (3,10)
A14 (3,8)
A15 (6,11)
First, we will randomly choose 3 centroids from the given
data. A2 (2,6), A7 (5,10), and A15 (6,11) as the centroids
of the initial clusters.
•Centroid 1=(2,6) is associated with cluster 1.
•Centroid 2=(5,10) is associated with cluster 2.
•Now
Centroid
we will3=(6,11)
find the is associated with cluster 3.
euclidean distance between
each point and the centroids.
Based on the minimum
distance of each point from
the centroids, we will assign
the points to a cluster.
Distance from Distance from
Distance from Assigned
Point Centroid 2 Centroid 3
Centroid 1 (2,6) Cluster
(5,10) (6,11)
A1 (2,10) 4 3 4.123106 Cluster 2
A2 (2,6) 0 5 6.403124 Cluster 1
A3 (11,11) 10.29563 6.082763 5 Cluster 3
A4 (6,9) 5 1.414214 2 Cluster 2
A5 (6,4) 4.472136 6.082763 7 Cluster 1
A6 (1,2) 4.123106 8.944272 10.29563 Cluster 1
A7 (5,10) 5 0 1.414214 Cluster 2
A8 (4,9) 3.605551 1.414214 2.828427 Cluster 2
A9 (10,12) 10 5.385165 4.123106 Cluster 3
A10 (7,5) 5.09902 5.385165 6.082763 Cluster 1
A11 (9,11) 8.602325 4.123106 3 Cluster 3
A12 (4,6) 2 4.123106 5.385165 Cluster 1
A13 (3,10) 4.123106 2 3.162278 Cluster 2
A14 (3,8) 2.236068 2.828427 4.242641 Cluster 1
A15 (6,11) 6.403124 1.414214 0 Cluster 3
Results from 1st iteration of K means clustering
The formula to calculate the centroid of a cluster is the mean of all the
points in that cluster. For a cluster containing n points in a 2D space,
the centroid C can be calculated as follows:
Given the points in the cluster:
clustering algorithm and assigned each point into a cluster.
In the above table, the point that is closest to the centroid of a
given cluster is assigned to the cluster.
Now, we will calculate the new centroid for each cluster.
•In cluster 1, we have 6 points i.e. A2 (2,6), A5 (6,4), A6 (1,2),
A10 (7,5), A12 (4,6), A14 (3,8). To calculate the new centroid for
cluster 1, we will find the mean of the x and y coordinates of each
point in the cluster. Hence, the new centroid for cluster 1 is (3.833,
5.167).
•In cluster 2, we have 5 points i.e. A1 (2,10), A4 (6,9), A7 (5,10) ,
A8 (4,9), and A13 (3,10). Hence, the new centroid for cluster 2 is
(4, 9.6)
•In cluster 3, we have 4 points i.e. A3 (11,11), A9 (10,12), A11
(9,11), and A15 (6,11). Hence, the new centroid for cluster 3 is (9,
Now that we have calculated new centroids for each
cluster, we will calculate the distance of each data point
from the new centroids.
Then, we will assign the points to clusters based on their
distance from the centroids.
Distance from Distance from Distance from
Assigned
Point Centroid 1 centroid 2 (4, centroid 3 (9,
Cluster
(3.833, 5.167) 9.6) 11.25)
A1 (2,10) 5.169 2.040 7.111 Cluster 2
A2 (2,6) 2.013 4.118 8.750 Cluster 1
A3 (11,11) 9.241 7.139 2.016 Cluster 3
A4 (6,9) 4.403 2.088 3.750 Cluster 2
A5 (6,4) 2.461 5.946 7.846 Cluster 1
A6 (1,2) 4.249 8.171 12.230 Cluster 1
A7 (5,10) 4.972 1.077 4.191 Cluster 2
A8 (4,9) 3.837 0.600 5.483 Cluster 2
A9 (10,12) 9.204 6.462 1.250 Cluster 3
A10 (7,5) 3.171 5.492 6.562 Cluster 1
A11 (9,11) 7.792 5.192 0.250 Cluster 3
A12 (4,6) 0.850 3.600 7.250 Cluster 1
A13 (3,10) 4.904 1.077 6.129 Cluster 2
A14 (3,8) 2.953 1.887 6.824 Cluster 2
A15 (6,11) 6.223 2.441 3.010 Cluster 2
Results from 2nd iteration of K means clustering
Now, we have completed the second iteration of the k-means
clustering algorithm and assigned each point into an updated
cluster.
Now, we will calculate the new centroid for each cluster for the
third iteration.
•In cluster 1, we have 5 points i.e. A2 (2,6), A5 (6,4), A6 (1,2), A10
(7,5), and A12 (4,6). To calculate the new centroid for cluster 1, we
will find the mean of the x and y coordinates of each point in the
cluster. Hence, the new centroid for cluster 1 is (4, 4.6).
•In cluster 2, we have 7 points i.e. A1 (2,10), A4 (6,9), A7 (5,10) ,
A8 (4,9), A13 (3,10), A14 (3,8), and A15 (6,11). Hence, the new
centroid for cluster 2 is (4.143, 9.571)
•In cluster 3, we have 3 points i.e. A3 (11,11), A9 (10,12), and A11
(9,11). Hence, the new centroid for cluster 3 is (10, 11.333).
At this point, we have calculated new centroids for
each cluster. Now, we will calculate the distance of
each data point from the new centroids. Then, we
will assign the points to clusters based on their
distance from the centroids.
Distance from Distance from Distance from
Assigned
Point Centroid 1 (4, centroid 2 centroid 3 (10,
Cluster
4.6) (4.143, 9.571) 11.333)
A1 (2,10) 5.758 2.186 8.110 Cluster 2
A2 (2,6) 2.441 4.165 9.615 Cluster 1
A3 (11,11) 9.485 7.004 1.054 Cluster 3
A4 (6,9) 4.833 1.943 4.631 Cluster 2
A5 (6,4) 2.088 5.872 8.353 Cluster 1
A6 (1,2) 3.970 8.197 12.966 Cluster 1
A7 (5,10) 5.492 0.958 5.175 Cluster 2
A8 (4,9) 4.400 0.589 6.438 Cluster 2
A9 (10,12) 9.527 6.341 0.667 Cluster 3
A10 (7,5) 3.027 5.390 7.008 Cluster 1
A11 (9,11) 8.122 5.063 1.054 Cluster 3
A12 (4,6) 1.400 3.574 8.028 Cluster 1
A13 (3,10) 5.492 1.221 7.126 Cluster 2
A14 (3,8) 3.544 1.943 7.753 Cluster 2
A15 (6,11) 6.705 2.343 4.014 Cluster 2
Results from 3rd iteration of K means clustering
Now, we have completed the third iteration of the k-means
clustering algorithm and assigned each point into an updated
cluster
Now, we will calculate the new centroid for each cluster
for the third iteration.
•In cluster 1, we have 5 points i.e. A2 (2,6), A5 (6,4), A6
(1,2), A10 (7,5), and A12 (4,6). To calculate the new
centroid for cluster 1, we will find the mean of the x and y
coordinates of each point in the cluster. Hence, the new
centroid for cluster 1 is (4, 4.6).
•In cluster 2, we have 7 points i.e. A1 (2,10), A4 (6,9), A7
(5,10) , A8 (4,9), A13 (3,10), A14 (3,8), and A15 (6,11).
Hence, the new centroid for cluster 2 is (4.143, 9.571)
•In cluster 3, we have 3 points i.e. A3 (11,11), A9 (10,12),
Here, you can observe that no point has changed its
cluster compared to the previous iteration. Due to this,
the centroid also remains constant. Therefore, we will say
that the clusters have been stabilized. Hence, the clusters
obtained after the third iteration are the final clusters
made from the given dataset. If we plot the clusters on a
graph, the graph looks like as follows.
EVALUATION METRICS FOR UNSUPERVISED
• In unsupervised learning, accuracy is typically not measured in
the same way it is in supervised learning, because unsupervised
models don’t rely on labeled data. Instead of comparing
predictions against ground truth labels, we evaluate
unsupervised models based on how well they reveal structure or
•Silhouette Score: Measures how similar each point is to its own cluster
patterns in the data.
compared to other clusters. A higher score indicates that the points are well-
clustered. It ranges from -1 (poor clustering) to +1 (good clustering), with a
score of 0 indicating overlapping clusters.
•Davies-Bouldin Index: Measures the average similarity ratio of each cluster
with the one that is most similar to it. Lower values indicate better clustering.
•Inertia (within-cluster sum of squares): In algorithms like k-means, inertia
measures how tightly the data points are clustered around centroids. Lower
inertia generally means better clustering, but this metric can also increase with
the number of clusters, so it’s not always the most reliable on its own.
QUESTIONS?
