Simple Linear Iterative Clustering (SLIC)

SLIC (Achanta et al. PAMI 2012)

1. Initialize cluster centers on pixel
grid in steps S
- Features: Lab color, x-y position
2. Move centers to position in 3x3
window with smallest gradient
3. Compare each pixel to cluster
center within 2S pixel distance
and assign to nearest
4. Recompute cluster centers as + Fast 0.36s for 320x240
mean color/position of pixels + Regular superpixels
+ Superpixels fit boundaries
belonging to each cluster - May miss thin objects
- Large number of superpixels
5. Stop when residual error is
11/22/2021 small CAP5415 - Lecture 16 Image Segmentation - II 42
Mean-Shift Segmentation
• Mean-shift is a variant of an iterative steepest-ascent method to seek stationary
points (i.e., peaks) in a density function, which is applicable in many areas of
multi-dimensional data analysis.

http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html
11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 44
Mean-Shift Segmentation
• Mean-shift is a variant of an iterative steepest-ascent method to seek stationary
points (i.e., peaks) in a density function, which is applicable in many areas of
multi-dimensional data analysis.

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 45

Mean-Shift Segmentation
• Mean-shift is a variant of an iterative steepest-ascent method to seek stationary
points (i.e., peaks) in a density function, which is applicable in many areas of
multi-dimensional data analysis.
• Attempts to find all possible cluster centers in feature space (unlike k-means,
where there is a requirement to know the number of different clusters).

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 46

Mean-Shift Segmentation
• Mean-shift is a variant of an iterative steepest-ascent method to seek stationary
points (i.e., peaks) in a density function, which is applicable in many areas of
multi-dimensional data analysis.
• The mean shift algorithm seeks modes or local maxima of density in the
feature space

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 47

Mean-Shift Basics
• Mean-Shift is a procedure for locating maxima of a density function
given discrete data samples from that function.

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 48

Mean-Shift Basics
• Mean-Shift is a procedure for locating maxima of a density function
given discrete data samples from that function.
• This is an iterative method, and we start with an initial estimation of
x.

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 49

Mean-Shift Basics
• Mean-Shift is a procedure for locating maxima of a density function
given discrete data samples from that function.
• This is an iterative method, and we start with an initial estimation of
x.
• Let a kernel function K(xi-x) be given, determining the weight of
nearby points for re-estimation of the mean.

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 50

Mean-Shift Basics
• Mean-Shift is a procedure for locating maxima of a density function
given discrete data samples from that function.
• This is an iterative method, and we start with an initial estimation of
x.
• Let a kernel function K(xi-x) be given, determining the weight of
nearby points for re-estimation of the mean.
• The weighted mean of the density in the window determined by K

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 51

Mean-Shift Basics
• Mean-Shift is a procedure for locating maxima of a density function
given discrete data samples from that function.
• This is an iterative method, and we start with an initial estimation of
x.
• Let a kernel function K(xi-x) be given, determining the weight of
nearby points for re-estimation of the mean.
• The weighted mean of the density in the window determined by K

N(x): neighborhood of x
m(x) – x: mean shift
11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 52
Mean-Shift Search
window

Center of
mass

Mean Shift
vector

11/22/2021 FIND
CAP5415 DENSEST
- Lecture REGION! - II
16 Image Segmentation 53
Slide by Y. Ukrainitz & B. Sarel
Mean-Shift Search
window

Center of
mass

Mean Shift
vector

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 54

Slide by Y. Ukrainitz & B. Sarel
Mean-Shift Search
window

Center of
mass

Mean Shift
vector

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 55

Slide by Y. Ukrainitz & B. Sarel
Mean-Shift Search
window

Center of
mass

Mean Shift
vector

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 56

Slide by Y. Ukrainitz & B. Sarel
Mean-Shift Search
window

Center of
mass

Mean Shift
vector

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 57

Slide by Y. Ukrainitz & B. Sarel
Mean-Shift Search
window

Center of
mass

Mean Shift
vector

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 58

Slide by Y. Ukrainitz & B. Sarel
Mean-Shift Search
window

Center of
mass

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 59

Slide by Y. Ukrainitz & B. Sarel
Practical Knowledge

Tessellate the space Run the procedure in parallel

11/22/2021 with windows CAP5415 - Lecture 16 Image Segmentation - II 60
Mean shift clustering
The mean shift algorithm seeks modes of the given set of points
1. Choose kernel and bandwidth
2. For each point:
a) Center a window on that point
b) Compute the mean of the data in the search window
c) Center the search window at the new mean location
d) Repeat (b,c) until convergence
3. Assign points that lead to nearby modes to the same cluster

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 61

Mean-Shift Clustering
• Gradient ascent requires computing a gradient of the density function.
• Density function is only approximated by a distribution of samples, the
gradient can only be approximated as well !!!
• This is done by a kernel window estimator.

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 62

Slide by Y. Ukrainitz & B. Sarel
Trajectory: x, m(x), (m(m(x)), …..

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 63

Mean-Shift Segmentation
• Find features (color, gradients,
texture, etc)
• Initialize windows at individual
feature points
• Perform mean shift for each window
until convergence
• Merge windows that end up near
the same “peak” or mode

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 64

Ex: Mean-Shift Segmentation

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 65

Ex: Mean-Shift Segmentation

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 66

Mean-Shift Clustering

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 67

Demo

https://spin.atomicobject.com/2015/05/26/mean-shift-clustering/
11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 68
Demo

https://spin.atomicobject.com/2015/05/26/mean-shift-clustering/
11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 69
Demo

https://spin.atomicobject.com/2015/05/26/mean-shift-clustering/
11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 70
Mean-shift pros and cons
• Pros
• Does not assume spherical clusters
• Just a single parameter (window size)
• Finds variable number of modes
• Robust to outliers
• Cons
• Output depends on window size
• Computationally expensive
• Does not scale well with dimension of feature space

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 71

Slide Credits and References
• Ulas Bagci
• David Kauchak (k-means)
• R.Klette, Concise Computer Vision, 2014.
• Y. Ukrainitz & B. Sarel, Weizmann
• Thu Huong Nhuyen, TU Dresden
• Jens N. Kaftan and Andr`e A. Bell and Til Aach . Mean Shift Segmentation Evaluation of Optimization Techniques
• Cheng, Y. (1995). Mean Shift, Mode Seeking, and Clustering. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 17(8):790–799.
• KEINOSUKE FUKUNAGA, AND LARRY D. HOSTETLER. The Estimation of the Gradient of a Density Function, with Applications
in Pattern – Recognition. IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-21, NO. 1, JANUARY 1975
• Comaniciu, D. and Meer, P. (1997). Robust Analysis of Feature Spaces: Color Image Segmentation.
• In IEEE Conference on Computer Vision and Pattern Recognition. CVPR 1997, pages 750–755.
• Comaniciu, D. and Meer, P. (1999). Mean Shift Analysis an Applications. In International Conference on Computer Vision.
ICCV 1999, volume 2, pages 1197–1203.
• Comaniciu, D. and Meer, P. (2002). Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 24(5):603–619.
• K. Fukunaga, L.D. Hostetler, The estimation of the gradient of a density function, with applications in pattern recognition,
IEEE Trans. Inf. Theory 21 (1975) 3240.
• Yiping Hong , Jianqiang Yi and Dongbin Zhao . Improved mean shift segmentation approach for natural images
11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 72
 Questions?

11/22/2021 CAP5415 - Lecture 16 Image Segmentation - II 73