Clustering

Computer Vision
CSE M164
Today’s class
• Fitting and alignment
– One more algorithm: ICP
– Review of all the algorithms

• Clustering algorithms
– K-means
– Hierarchical clustering
– Spectral clustering
What if you want to align but have no prior
matched pairs?

• Hough transform and RANSAC not applicable

• Important applications

Medical imaging: match brain Robotics: match point clouds

scans or contours
Iterative Closest Points (ICP) Algorithm

Goal: estimate transform between two dense

sets of points

1. Assign each point in {Set 1} to its nearest neighbor

in {Set 2}
2. Estimate transformation parameters
– e.g., least squares or robust least squares
3. Transform the points in {Set 1} using estimated
parameters
4. Repeat steps 2-4 until change is very small
Example: solving for translation

A1

A2 A3 B1

B2 B3

Given matched points in {A} and {B}, estimate the translation of the object
 xiB   xiA   t x 
 B   A    t 
 yi   yi   y 
Example: solving for translation

A1

A2 A3 (tx, ty) B1

B2 B3

Least squares solution

1. Write down objective function
 xiB   xiA   t x 
 B   A    t 
2. Derived solution
a) Compute derivative
 yi   yi   y 
b) Compute solution 1 0  x1B  x1A 
0  B A
3. Computational solution 1 y  y
   tx   1 1 
a) Write in form Ax=b       
  t  
0    x nB  x nA 
y
b) Solve using pseudo-inverse or 1
eigenvalue decomposition  0 1  y nB  y nA 
Example: solving for translation

A1
A5
B4
A2 A3 (tx, ty) B1

A4
B2 B3
B5

Problem: outliers

RANSAC solution  xiB   xiA   t x 

1. Sample a set of matching points (1 pair)  B   A    t 
2. Solve for transformation parameters  yi   yi   y 
3. Score parameters with number of inliers
4. Repeat steps 1-3 N times
Example: solving for translation
B4
A1 B5 B6

A2 A3 (tx, ty) B1

A4 B2 B3
A5 A6

Problem: outliers, multiple objects, and/or many-to-one matches

Hough transform solution  xiB   xiA   t x 

1. Initialize a grid of parameter values  B   A    t 
2. Each matched pair casts a vote for  yi   yi   y 
consistent values
3. Find the parameters with the most votes
4. Solve using least squares with inliers
Example: solving for translation

(tx, ty)

Problem: no initial guesses for correspondence

ICP solution  xiB   xiA   t x 

1. Find nearest neighbors for each point  B   A    t 
2. Compute transform using matches  yi   yi   y 
3. Move points using transform
4. Repeat steps 1-3 until convergence
 Clustering
Group together similar points and represent
them with a single token

Key Challenges:
1) What makes two points/images/patches similar?
2) How do we compute an overall grouping from
pairwise similarities?
Why do we cluster?
• Summarizing data
– Look at large amounts of data
– Patch-based compression or denoising
– Represent a large continuous vector with the cluster number

• Counting
– Histograms of texture, color, SIFT vectors

• Segmentation
– Separate the image into different regions

• Prediction
– Images in the same cluster may have the same labels
How do we cluster?

• K-means
– Iteratively re-assign points to the nearest cluster center

• Agglomerative clustering
– Start with each point as its own cluster and iteratively
merge the closest clusters

• Spectral clustering
– Split the nodes in a graph based on assigned links with
similarity weights
Clustering for Summarization
Goal: cluster to minimize variance in data
given clusters
– Preserve information

Cluster center Data

N K
c * , δ * argmin N1   ij c i  x j 
2

c ,δ j i

Whether xj is assigned to ci
 K-means

0. Initialize 1. Assign Points to 2. Re-compute Repeat (1) and (2)

Cluster Centers Clusters Means

Illustration Source: wikipedia

K-means
1. Initialize cluster centers: c0 ; t=0

2. Assign each point to the closest center

N K

  c  xj
t t 1 2
δ argmin 1
N ij i
δ j i

3. Update cluster centers

N K
as the mean of the points
c t argmin N1   ijt c i  x j 
2

c j i

4. Repeat 2-3 until no points are re-assigned (t=t+1)

K-means: design choices
• Initialization
– Randomly select K points as initial cluster center
– Or greedily choose K points to minimize residual

• Distance measures
– Traditionally Euclidean, could be others

• Optimization
– Will converge to a local minimum
– May want to perform multiple restarts

Illustration Source: wikipedia

K-means clustering using intensity or color

Image Clusters on intensity Clusters on color

How to choose the number of clusters?
• Minimum Description Length (MDL) principle for
model comparison

• Minimize Schwarz Criterion

– also called Bayes Information Criteria (BIC)
How to evaluate clusters?

• Generative
– How well are points reconstructed from the clusters?
– Example: Predict the next word in a sequence

• Discriminative
– How well do the clusters correspond to labels?
• Purity
– Example: Spectral clustering
– Note: unsupervised clustering does not aim to be
discriminative
How to choose the number of clusters?
• Validation set
– Try different numbers of clusters and look at
performance
• When building dictionaries (discussed later), more
clusters typically work better
Conclusions: K-means
Good
• Finds cluster centers that minimize conditional variance (good
representation of data)
• Simple to implement, widespread application

Bad
• Prone to local minima
• Need to choose K
• All clusters have the same parameters (e.g., distance measure
is non-adaptive)
• Can be slow: each iteration is O(KNd) for N d-dimensional
points
Building Visual Dictionaries
1. Sample patches from
a database
– E.g., 128 dimensional
SIFT vectors

2. Cluster the patches

– Cluster centers are
the dictionary

3. Assign a codeword
(number) to each
new patch, according
to the nearest cluster
 Examples of learned codewords

Most likely codewords for 4 learned “topics”

EM with multinomial (problem 3) to get topics

http://www.robots.ox.ac.uk/~vgg/publications/papers/sivic05b.pdf Sivic et al. ICCV 2005

Common similarity/distance measures
• P-norms
– City Block (L1)
– Euclidean (L2)
Here xi is the
– L-infinity
distance
between two
points

• Mahalanobis
– Scaled Euclidean

• Cosine distance
K-medoids
• Just like K-means except
– Represent the cluster with one of its members,
rather than the mean of its members
– Choose the member (data point) that minimizes
cluster dissimilarity

• Applicable when a mean is not meaningful

– E.g., clustering values of hue or using L-infinity
similarity
Agglomerative clustering
Agglomerative clustering
Agglomerative clustering
Agglomerative clustering
Agglomerative clustering
Agglomerative clustering
How to define cluster similarity?
- Average distance between points, maximum
distance, minimum distance
- Distance between means or medoids

How many clusters?

- Clustering creates a dendrogram (a tree)
- Threshold based on max number of clusters
or based on distance between merges

distance
Conclusions: Agglomerative Clustering
Good
• Simple to implement, widespread application
• Clusters have adaptive shapes
• Provides a hierarchy of clusters

Bad
• May have imbalanced clusters
• Still have to choose number of clusters or threshold
• Need to use an “ultrametric” to get a meaningful
hierarchy
Spectral clustering
Group points based on links in a graph

B
A
Cuts in a graph

B
A

Normalized Cut
• a cut penalizes large segments
• fix by normalizing for size of segments

• volume(A) = sum of costs of all edges that touch A

Source: Seitz
Normalized cuts for segmentation
Visual PageRank
• Determining importance by random walk
– What’s the probability that you will randomly walk
to a given node?
• Create adjacency matrix based on visual similarity
• Edge weights determine probability of transition

Jing Baluja 2008

Which algorithm to use?
• Quantization/Summarization: K-means
– Aims to preserve variance of original data
– Can easily assign new point to a cluster

Summary of 20,000 photos of Rome using

Quantization for
“greedy k-means”
computing histograms
http://grail.cs.washington.edu/projects/canonview/
Which algorithm to use?
• Image segmentation: agglomerative clustering
– More flexible with distance measures (e.g., can be
based on boundary prediction)
– Adapts better to specific data
– Hierarchy can be useful

http://www.cs.berkeley.edu/~arbelaez/UCM.html
Which algorithm to use?
• Image segmentation: spectral clustering
– Can provide more regular regions
– Spectral methods also used to propagate global
cues
Things to remember
• K-means useful for summarization,
building dictionaries of patches,
general clustering

• Agglomerative clustering useful for

segmentation, general clustering

• Spectral clustering useful for

determining relevance,
summarization, segmentation