Scribd
Upload
19 views4 pages

Spectral Clustering

Spectral clustering is a graph theory-based method that identifies clusters using eigenvectors of a matrix derived from the data, making it effective for complex cluster shapes. It constructs a similarity graph, computes the Laplacian matrix, solves for eigenvectors, reduces dimensionality, and then applies a clustering algorithm. This approach is advantageous for non-linear clusters and structured data, leveraging relationships between data points rather than absolute distances.

Uploaded by

Rana Ben Fraj
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
19 views4 pages

Spectral Clustering

Spectral clustering is a graph theory-based method that identifies clusters using eigenvectors of a matrix derived from the data, making it effective for complex cluster shapes. It constructs a similarity graph, computes the Laplacian matrix, solves for eigenvectors, reduces dimensionality, and then applies a clustering algorithm. This approach is advantageous for non-linear clusters and structured data, leveraging relationships between data points rather than absolute distances.

Uploaded by

Rana Ben Fraj
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Spectral Clustering

Definition :
Spectral clustering is a powerful clustering method based on graph theory. It
uses the properties of the eigenvectors of a matrix (derived from the data) to
identify clusters. It is particularly effective for datasets where clusters have
complex or irregular shapes, which traditional clustering algorithms (like K-
Means) might struggle to detect.

Why Use Spectral Clustering?


1. Non-linear Clusters: Unlike K-Means, which assumes clusters are
spherical, spectral clustering can detect clusters of arbitrary shapes.

2. Graph-Based: It leverages the relationship (or similarity) between data


points rather than their absolute distances, making it suitable for structured
data.

3. Dimensionality Reduction: By focusing on a few key eigenvectors, it


reduces the complexity of the data, highlighting the essential clustering
structure.

Key Concepts :
1. Similarity Graph:
A graph is constructed where:

Nodes represent data points.

Edges represent the similarity between pairs of points.

2. Adjacency Matrix (AAA) :


Represents the connections (edges) between data points (nodes).

Spectral Clustering 1
Types of adjacency matrices:

1. Binary Adjacency Matrix: Aij​=1 if nodes i and j are connected; 0


otherwise.

2. Weighted Adjacency Matrix: Aij reflects the similarity between nodes i


and j using measures like the Gaussian Kernel

3. Degree Matrix (DDD):


A diagonal matrix where each diagonal element represents the degree of a
node (total weight of edges connected to it)

For n nodes, D is an n×n matrix

4. Laplacian Matrix (LLL)


Captures the graph structure by combining the adjacency and degree
matrices.

Types:

1. Unnormalized Laplacian:
L=D−A

Spectral Clustering 2
2. Normalized Laplacian:Normalized versions are used to handle graphs
with varying node degrees.

Symmetric: Lsym=D^(−1/2) LD^(−1/2)

Random Walk: Lrw=D^(−1) L

Steps of Spectral Clustering


Step 1: Construct the Similarity Graph
Represent the data as a graph where:

Nodes = data points.

Edges = similarity between points.

Build the Adjacency Matrix (AAA):

Use a similarity measure (e.g., Gaussian kernel or k-nearest neighbors).

Choose the graph type:

Fully connected: Every node is connected to all others.

k-Nearest Neighbors: Connect each node to its k nearest neighbors.

Epsilon graph: Connect nodes within a distance ϵ.

Step 2: Compute the Laplacian Matrix


Calculate L=D−A (or a normalized version).
The Laplacian encodes the structure of the graph and will help identify
clusters.

Step 3: Solve for Eigenvectors


Solve the eigenvalue problem: Lv=λv

Spectral Clustering 3
λ = eigenvalues / v = eigenvectors.
Select the smallest k eigenvectors (excluding the zero eigenvector for
normalized Laplacians).

Step 4: Dimensionality Reduction


Use the selected eigenvectors to represent the data in a lower-dimensional
space:

Each row of the eigenvector matrix is treated as a new data point.

These rows capture cluster relationships.

Step 5: Cluster the Transformed Data


Apply a clustering algorithm (e.g., K-Means) to the transformed data to
assign clusters.

Spectral Clustering 4

You might also like