Learning Objectives

• Introduction to Clustering
• Understand K-Means clustering
• Learn DBSCAN a Density-based clustering
• Compare K-Means clustering with DBSCAN

2
Let’s Look At…

We will take a look at an

Unsupervised unsupervised learning
algorithm to understand how
Learning data without labels can also be
useful.

4
And Now Clustering…

Task of dividing the population

or data points into a number of
groups such that data points in
What is the same groups are more
similar to other data points in
Clustering? the same group and dissimilar
to the data points in other
groups.

5
Clustering

7
Clustering

8
Clustering

9
Clustering

10
Clustering

11
Clustering: K Means

13
Clustering: K Means

14
Clustering: K Means

15
Clustering: K Means

16
Clustering: K Means

17
Clustering: K Means

18
Clustering: K Means

19
Clustering: K Means

20
Clustering: K Means

21
Clustering: K Means

22
Clustering: K Means

24
Clustering: K Means

25
Limitations: K Means

26
Limitations: K Means

27
Limitations: K Means

28
Limitations: K Means

29
Clustering: K Means
Hard clustering and soft clustering
In hard clustering, one data
point can belong to one cluster
only.

But in soft clustering, the output

provided is a probability
likelihood of a data point
belonging to each of the
predefined numbers of clusters.

30
DBSCAN
Density-based spatial clustering of applications with noise (DBSCAN)
● DBSCAN is a density-based spatial clustering algorithm proposed in 1996
by Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu.
● It is a clustering method that identifies groups of data points that are
densely packed together while flagging outliers that are isolated in low-
density areas.
● DBSCAN is widely recognized and frequently cited as one of the most
common clustering algorithms in data analysis and machine learning!

32
Density Based Clustering
Basic idea
● Clusters are dense regions in
the data space, separated by
regions of lower object density
● Discovers clusters of arbitrary
shape
Where are the clusters?

33
Density Definition
● ε-Neighborhood: All points q within a radius of ε from a given point p.

Any distance function can be used based on the application

34
Density Definition
● ε-Neighborhood: All points q within a radius of ε from a given point p.

Any distance function can be used based on the application

36
Density Definition
● High Density: ε-Neighborhood of a point contains at least MinPts
number of points.

Example:
Density of p is “high” (MinPts = 4)
Density of q is “low” (MinPts = 4)

37
Type of Points
● A point is a core point if it has at least MinPts number of points
within ε. These are points that are at the interior of a cluster.
● A border point has fewer than MinPts within ε, but is in the
neighborhood of a core point.
● An outlier point is any point that is not a core point nor a border
point.

38
Type of Points

MinPts = 5

39
Type of Points

40
Density Reachability

Directly density-reachable

42
Density Reachability

Directly density-reachable

43
Density Reachability

Directly density-reachable

Is q is directly density-reachable from p?

Is p is directly density-reachable from q?
Is Density-reachability is symmetric?

44
Density Reachability

Directly density-reachable

Is q is directly density-reachable from p? ✅

Is p is directly density-reachable from q? ❌
Is Density-reachability is symmetric? ❌

45
Density Reachability

Density-reachable (Indirectly)

● A point p is directly density-reachable from p2

● p2 is directly density-reachable from p1
● p1 is directly density-reachable from q
p ← p2 ← p1 ← q form a chain

46
Density Reachability

Density-reachable (Indirectly)

● p is (indirectly) density-reachable from q

● q is not density-reachable from p

p ← p2 ← p1 ← q form a chain

47
Density Connectivity

Density-connected

https://www.geeksforgeeks.org/

48
DBSCAN Algorithm ● ε = 2 cm
● MinPts = 3

50
DBSCAN Algorithm

51
DBSCAN Algorithm

52
DBSCAN Algorithm

for each p ∈ D do
if p is not yet classified then

if p is a core-object then
collect all objects density reachable
from p and assign them to a new
cluster
else
assign p to outlier

53
DBSCAN Algorithm

for each p ∈ D do
if p is not yet classified then
What is the time complexity?

if p is a core-object then
collect all objects density reachable What is the space complexity?
from p and assign them to a new
cluster
else
assign p to outlier

54
DBSCAN is Sensitive to Parameters

56
DBSCAN vs. K Means

57
DBSCAN vs. K Means

DBSCAN K Means
● In DBSCAN two parameters are ● In K-Means only one parameter is required
required for training the Model is for training the model

● Clusters formed in DBSCAN can be ● Clusters formed in K-Means are spherical

of any arbitrary shape or convex in shape

● DBSCAN can work well with ● K-Means is very sensitive to the number of
datasets having noise and outliers clusters so it need to specified
● K-Means does not work well with outliers
data

58