Pattern Recognition Letters: René Vidal, Paolo Favaro

Pattern Recognition Letters xxx (2013) xxx–xxx
Contents lists available at ScienceDirect
Pattern Recognition Letters

journal homepage: www.elsevier.com/locate/patrec
Low rank subspace clustering (LRSC)

René Vidal a,⇑, Paolo Favaro b
a
Center for Imaging Science, Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
b
Institute of Informatics and Applied Mathematics, University of Bern, 3012, Switzerland
a r t i c l e i n f o a b s t r a c t
Article history: We consider the problem of fitting a union of subspaces to a collection of data points drawn from one or
Available online xxxx more subspaces and corrupted by noise and/or gross errors. We pose this problem as a non-convex opti-
mization problem, where the goal is to decompose the corrupted data matrix as the sum of a clean and
Communicated by Kim Boyer
self-expressive dictionary plus a matrix of noise and/or gross errors. By self-expressive we mean a dictio-
nary whose atoms can be expressed as linear combinations of themselves with low-rank coefficients. In
Keywords: the case of noisy data, our key contribution is to show that this non-convex matrix decomposition prob-
Subspace clustering
lem can be solved in closed form from the SVD of the noisy data matrix. The solution involves a novel
Low-rank and sparse methods
Principal component analysis
polynomial thresholding operator on the singular values of the data matrix, which requires minimal
Motion segmentation shrinkage. For one subspace, a particular case of our framework leads to classical PCA, which requires
Face clustering no shrinkage. For multiple subspaces, the low-rank coefficients obtained by our framework can be used
to construct a data affinity matrix from which the clustering of the data according to the subspaces can be
obtained by spectral clustering. In the case of data corrupted by gross errors, we solve the problem using
an alternating minimization approach, which combines our polynomial thresholding operator with the
more traditional shrinkage-thresholding operator. Experiments on motion segmentation and face clus-
tering show that our framework performs on par with state-of-the-art techniques at a reduced compu-
tational cost.
! 2013 Elsevier B.V. All rights reserved.
1. Introduction as subspace clustering, finds numerous applications in computer vi-

sion, e.g., image segmentation (Yang et al., 2008), motion segmen-
The past few decades have seen an explosion in the availability tation (Vidal et al., 2008) and face clustering (Ho et al., 2003),
of datasets from multiple modalities. While such datasets are usu- image processing, e.g., image representation and compression
ally very high-dimensional, their intrinsic dimension is often much (Hong et al., 2006), and systems theory, e.g., hybrid system identi-
smaller than the dimension of the ambient space. For instance, the fication (Vidal et al., 2003b).
number of pixels in an image can be huge, yet most computer
vision models use a few parameters to describe the appearance,
geometry and dynamics of a scene. This has motivated the devel-
opment of a number of techniques for finding low-dimensional 1.1. Prior work on subspace clustering
representations of high-dimensional data.
One of the most commonly used methods is Principal Compo- Over the past decade, a number of subspace clustering methods
nent Analysis (PCA), which models the data with a single low- have been developed. This includes algebraic methods (Boult and
dimensional subspace. In practice, however, the data points could Brown, 1991; Costeira and Kanade, 1998; Gear, 1998; Vidal et al.,
be drawn from multiple subspaces and the membership of the data 2003a; Vidal et al., 2004; Vidal et al., 2005), iterative methods
points to the subspaces could be unknown. For instance, a video se- (Bradley and Mangasarian, 2000; Tseng, 2000; Agarwal and
quence could contain several moving objects and different sub- Mustafa, 2004; Lu and Vidal, 2006; Zhang et al., 2009), statistical
spaces might be needed to describe the motion of different methods (Tipping and Bishop, 1999,Sugaya and Kanatani, 2004,
objects in the scene. Therefore, there is a need to simultaneously Gruber and Weiss, 2004,Yang et al., 2006, Ma et al., 2007,Rao
cluster the data into multiple subspaces and find a low-dimen- et al., 2008, Rao et al., 2010), and spectral clustering-based meth-
sional subspace fitting each group of points. This problem, known ods (Boult and Brown, 1991; Yan and Pollefeys, 2006; Zhang
et al., 2010; Goh and Vidal, 2007; Elhamifar and Vidal, 2009;
Elhamifar and Vidal, 2010; Elhamifar and Vidal, 2013; Liu et al.,
⇑ Corresponding author. Tel.: +1 410 516 7306; fax: +1 410 516 7775. 2010; Chen and Lerman, 2009). Among them, methods based on
E-mail address: [email protected] (R. Vidal). spectral clustering have been shown to perform very well for
0167-8655/$ - see front matter ! 2013 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.patrec.2013.08.006
Please cite this article in press as: Vidal, R., Favaro, P. Low rank subspace clustering (LRSC). Pattern Recognition Lett. (2013), http://dx.doi.org/10.1016/
j.patrec.2013.08.006
2 R. Vidal, P. Favaro / Pattern Recognition Letters xxx (2013) xxx–xxx
several applications in computer vision (see Vidal (2011) for a re- N

X
view and comparison of existing methods). aj ¼ ai cij or A ¼ AC; ð1Þ
i¼1
Spectral clustering-based methods (see von Luxburg, 2007 for a
review) decompose the subspace clustering problem in two steps. where C ¼ ½cij # is the matrix of coefficients. This constraint aims to
In the first step, a symmetric affinity matrix C ¼ ½cij # is constructed, capture the fact that a point in a linear subspace can be expressed
where cij ¼ cji P 0 measures whether points i and j belong to the as a linear combination of other points in the same subspace. There-
same subspace. Ideally cij $ 1 if points i and j are in the same sub- fore, we expect cij to be zero if points i and j are in different
space and cij $ 0 otherwise. In the second step, a weighted undi- subspaces.
rected graph is constructed where the data points are the nodes Notice that the constraint A ¼ AC is non-convex, because both A
and the affinities cij are the weights. The segmentation of the data and C are unknown. This is an important difference with respect to
is then found by clustering the eigenvectors of the graph Laplacian existing methods, which enforce D ¼ DC where D is the dictionary
using central clustering techniques, such as k-means. Arguably, the of corrupted data points. Another important difference is that we
most difficult step is to build a good affinity matrix. This is because directly enforce C to be symmetric, while existing methods sym-
two points could be very close to each other, but lie in different metrize C as a post-processing step.
subspaces (e.g., near the intersection of two subspaces). Con- The proposed framework, which we call Low Rank Subspace
versely, two points could be far from each other, but lie in the same Clustering (LRSC), is based on solving the following non-convex
subspace. optimization problem:
Earlier methods for building an affinity matrix (Boult and
Brown, 1991; Costeira and Kanade, 1998) compute the singular min kCk) þ 2s kA + ACk2F þ a2 kGk2F þ ckEk1
ðPÞ A;C;E;G
value decomposition (SVD) of the data matrix D ¼ U RV > and let
s:t: D ¼ A þ G þ E and C ¼ C > ;
C ¼ V 1V > 1 , where the columns of V 1 are the top r ¼ rankðDÞ singular
vectors of D. The rationale behind this choice is that cij ¼ 0 when P P P
where kXk) ¼ i ri ðXÞ, kXk2F ¼ ij X 2ij and kXk1 ¼ ij jX ij j are, respec-
points i and j are in different independent subspaces and the data tively, the nuclear, Frobenius and ‘1 norms of X. The above formula-
are uncorrupted, as shown in Vidal et al. (2005). In practice, how- tion encourages:
ever, the data are often contaminated by noise and gross errors.
In such cases, the equation cij ¼ 0 does not hold, even if the rank , C to be low-rank (by minimizing kCk) ),
of the noiseless D was given. Moreover, selecting a good value , A to be self-expressive (by minimizing kA + ACk2F ),
for r becomes very difficult, because D is full rank. Furthermore, , G to be small (by minimizing kGk2F ), and
the equation cij ¼ 0 is derived under the assumption that the sub- , E to be sparse (by minimizing kEk1 ).
spaces are linear. In practice, many datasets are better modeled by
affine subspaces. The main contribution of our work is to show that important
More recent methods for building an affinity matrix address particular cases of P (see Table 1) can be solved in closed form from
these issues by using techniques from sparse and low-rank repre- the SVD of the data matrix. In particular, we show that in the
sentation. For instance, it is shown in Elhamifar and Vidal (2009, absence of gross errors (i.e., c ¼ 1), A and C can be obtained by
2010, 2013) that a point in a union of multiple subspaces admits thresholding the singular values of D and A, respectively. The thres-
a sparse representation with respect to the dictionary formed by holding is done using a novel polynomial thresholding operator,
all other data points, i.e., D ¼ DC, where C is sparse. It is also shown which reduces the amount of shrinkage with respect to existing
in Elhamifar and Vidal (2009, 2010, 2013) that, if the subspaces are methods. Indeed, when the self-similarity constraint A ¼ AC is en-
independent, the nonzero coefficients in the sparse representation forced exactly (i.e., a ¼ 1), the optimal solution for A reduces to
of a point correspond to other points in the same subspace, i.e., if classical PCA, which does not perform any shrinkage. Moreover,
cij –0, then points i and j belong to the same subspace. Moreover, the optimal solution for C reduces to the affinity matrix for sub-
the nonzero coefficients can be obtained by ‘1 minimization. These space clustering proposed by Costeira and Kanade (1998). In the
coefficients are then converted into symmetric and nonnegative case of data corrupted by gross errors, a closed-form solution ap-
affinities, from which the segmentation is found using spectral pears elusive. We thus use an augmented Lagrange multipliers
clustering. A very similar approach is presented in Liu et al. method. Each iteration of our method involves a polynomial thres-
(2010). The major difference is that a low-rank representation is holding of the singular values to reduce the rank and a regular
used in lieu of the sparsest representation. While the same shrinkage-thresholding to reduce gross errors.
principle of representing a point as a linear combination of other
points has been successfully used when the data are corrupted
by noise and gross errors, from a theoretical viewpoint it is not Table 1
Particular cases of P solved in this paper.
clear that the above methods are effective when using a corrupted
dictionary. Relaxed Exact
Uncorrupted
1.2. Paper contributions P 1 : Section 3.1 P 2 : Section 3.2
0<s<1 s¼1
In this paper, we propose a general optimization framework for a¼1 a¼1
c¼1 c¼1
solving the subspace clustering problem in the case of data cor-
rupted by noise and/or gross errors. Given a corrupted data matrix Noise
P 3 : Section 4.1 P 4 : Section 4.2
D 2 RM'N , we wish to decompose it as the sum of a self-expressive, 0<s<1 s¼1
noise-free and outlier-free (clean) data matrix A 2 RM'N , a noise 0<a<1 0<a<1
matrix G 2 RM'N , and a matrix of sparse gross errors E 2 RM'N . c¼1 c¼1
We assume that the columns of the matrix A ¼ ½ a1 ; a2 ; . . . ; aN # are Gross errors
points in RM drawn from a union of n P 1 low-dimensional linear P 5 : Section 5.1 P 6 : Section 5.2
n 0<s<1 s¼1
subspaces of unknown dimensions fdi gi¼1 , where di ( M. We also
0<a<1 0<a<1
assume that A is self-expressive, which means that the clean data
0<c<1 0<c<1
points can be expressed as linear combinations of themselves, i.e.,
j.patrec.2013.08.006
R. Vidal, P. Favaro / Pattern Recognition Letters xxx (2013) xxx–xxx 3
1.3. Paper outline Notice that the latter solution does not coincide with the one gi-
ven by PCA, which performs hard-thresholding of the singular val-
The remainder of the paper is organized as follows (see Table 1): ues of D without shrinking them by 1=a.
Section 2 reviews existing results on sparse representation and
rank minimization for subspace estimation and clustering as well
2.1.2. Principal component pursuit
as some background material needed for our derivations. Section 3
While the above methods work well for data corrupted by
formulates the low rank subspace clustering problem for linear
Gaussian noise, they break down for data corrupted by gross errors.
subspaces in the absence of noise or gross errors and derives a
In Candès et al. (2011) this issue is addressed by assuming sparse
closed form solution for A and C. Section 4 extends the results of
gross errors, i.e., only a small percentage of the entries of D are cor-
Section 3 to data contaminated by noise and derives a closed form
rupted. Hence, the goal is to decompose the data matrix D as the
solution for A and C based on the polynomial thresholding opera-
sum of a low-rank matrix A and a sparse matrix E, i.e.,
tor. Section 5 extends the results to data contaminated by both
noise and gross errors and shows that A and C can be found using min rankðAÞ þ ckEk0 s:t: D ¼ A þ E; ð8Þ
A;E
alternating minimization. Section 6 presents experiments that
evaluate our method on synthetic and real data. Section 7 gives
where c > 0 is a parameter and kEk0 is the number of nonzero en-
the conclusions.
tries in E. Since this problem is in general NP hard, a common prac-
tice is to replace the rank of A by its nuclear norm and the ‘0 semi-
2. Background norm by the ‘1 norm. It is shown in Candès et al. (2011) that, under
broad conditions, the optimal solution to the problem in (8) is iden-
In this section we review existing results on sparse representa- tical to that of the convex problem
tion and rank minimization for subspace estimation (Section 2.1)
and and subspace clustering (Section 2.2). min kAk) þ ckEk1 s:t: D ¼ A þ E: ð9Þ
A;E
2.1. Subspace estimation by sparse representation and rank While a closed form solution to this problem is not known, con-
minimization vex optimization techniques can be used to find the minimizer. We
refer the reader to Lin et al. (2011) for a review of numerous ap-
2.1.1. Low rank minimization proaches. One such approach is the Augmented Lagrange Multi-
Given a data matrix corrupted by Gaussian noise D ¼ A þ G, plier (ALM) method, an iterative approach for solving the
where A is an unknown low-rank matrix and G represents the following optimization problem
noise, the problem of finding a low-rank approximation of D can
a
be formulated as max minkAk) þ ckEk1 þ hY; D + A + Ei þ kD + A + Ek2F ; ð10Þ
Y A;E 2
min kD + Ak2F subject to rankðAÞ 6 r: ð2Þ
A
where the third term enforces the equality constraint via the matrix
The optimal solution to this (PCA) problem is given by of Lagrange multipliers Y, while the fourth term (which is zero at
A ¼ UHrrþ1 ðRÞV > , where D ¼ U RV > is the SVD of D, rk is the kth the optimum) makes the cost function strictly convex. The optimi-
singular value of D, and H! ðxÞ is the hard thresholding operator: zation problem in (10) is solved using the inexact ALM method in
! (13). This method is motivated by the fact that the minimization
x jxj > !
H! ðxÞ ¼ ð3Þ over A and E for a fixed Y can be re-written as
0 else:
a
When r is unknown, the problem in (2) can be formulated as minkAk) þ ckEk1 þ kD + A + E þ a+1 Yk2F : ð11Þ
A;E 2
a
min rankðAÞ þ kD + Ak2F ; ð4Þ Given E and Y, it follows from the solution of (6) that the opti-
A 2
mal A is A ¼ US a+1 ðRÞV > , where U RV > is the SVD of D + E þ a+1 Y.
where a > 0 is a parameter. Since the optimal solution of (2) for a Given A and Y, the optimal E satisfies
fixed rank r ¼ rankðAÞ is A ¼ UHrrþ1 ðRÞV > , the problem in (4) is
equivalent to +aðD + A + E þ a+1 YÞ þ csignðEÞ ¼ 0: ð12Þ
aX It is shown in Lin et al. (2011) that this equation can be solved in
min r þ r2k : ð5Þ
r 2 k>r closed form using the shrinkage-thresholding operator as
pffiffiffiffiffiffiffiffi E ¼ S ca+1 ðD + A þ a+1 YÞ. Therefore, the inexact ALM method iter-
The optimal r is the smallest r such that rrþ1 6 2=a. Therefore, ates the following steps till convergence
the optimal A is given by A ¼ UHpffi2 ðRÞV > .
a
Since rank minimization problems are in general NP hard, a ðU; R; VÞ ¼ svdðD + Ek þ a+1
k Y kÞ
common practice (see Recht et al., 2010) is to replace the rank of
Akþ1 ¼ US a+1 ðRÞV >
A by its nuclear norm kAk) , i.e., the sum of its singular values, k
which leads to the following convex problem Ekþ1 ¼ S ca+1 ðD + Akþ1 þ a+1
k YkÞ
ð13Þ
k
a Y kþ1 ¼ Y k þ ak ðD + Akþ1 + Ekþ1 Þ

min kAk) þ kD + Ak2F ; ð6Þ
A 2 akþ1 ¼ qak :
where a > 0 is a user-defined parameter. It is shown in Cai et al.
This ALM method is essentially an iterated thresholding algo-
(2008) that the optimal solution to the problem in (6) is given by
rithm, which alternates between thresholding the SVD of
A ¼ US a1 ðRÞV > , where S ! ðxÞ is the shrinkage-thresholding operator
8 D + E þ Y=a to get A and thresholding D + A þ Y=a to get E. The up-
<x+ ! x > !
> date for Y is simply a gradient ascent step. Also, to guarantee the
S ! ðxÞ ¼ x þ ! x < +! ð7Þ convergence of the algorithm, the parameter a is updated by
>
: choosing the parameter q such that q > 1 so as to generate a se-
0 else:
quence ak that goes to infinity.
j.patrec.2013.08.006
2.2. Subspace clustering by sparse representation and rank 2.2.2. Low rank representation (LRR)
minimization This algorithm (Liu et al., 2010) is very similar to SSC, except
that it aims to find a low-rank representation instead of a sparse
Consider now the more challenging problem of clustering data representation. LRR is motivated by the fact that for uncorrupted
n
drawn from multiple subspaces. In what follows, we discuss two data drawn from n independent subspaces of dimensions fdi gi¼1 ,
methods based on sparse and low-rank representation for address- Pn
the data matrix D is low rank because r ¼ rankðDÞ ¼ i¼1 di , which
ing this problem. is assumed to be much smaller than minfM; Ng. Therefore, the
equation D ¼ DC has a solution C that is low-rank. The LRR
algorithm finds C by solving the following convex optimization
2.2.1. Sparse subspace clustering (SSC)
problem
The work of Elhamifar and Vidal (2009) shows that, in the case
of uncorrupted data, an affinity matrix for solving the subspace min kCk) s:t: D ¼ DC: ð18Þ
C
clustering problem can be constructed by expressing each data
point as a linear combination of all other data points. That is, we It is shown in Liu et al. (2011) that the optimal solution to (18)
wish to find a matrix C such that D ¼ DC and diagðCÞ ¼ 0. In prin- is given by C ¼ V 1 V >
1 , where D ¼ U 1 R1 V 1 is the rank r SVD of D.
>
ciple, this leads to an ill-posed problem with many possible solu- Notice that this solution for C coincides with the affinity matrix
tions. To resolve this issue, the principle of sparsity is invoked. proposed by Costeira and Kanade (1998), as described in the intro-
Specifically, every point is written as a sparse linear combination duction. It is shown in Vidal et al. (2008) that this matrix is such
of all other data points by minimizing the number of nonzero coef- that C ij ¼ 0 when points i and j are in different subspaces, hence
ficients. That is it can be used to build an affinity matrix.
X In the case of data contaminated by noise or gross errors, the
min kC i k0 s:t: D ¼ DC and diagðCÞ ¼ 0; ð14Þ LRR algorithm solves the convex optimization problem
C
i
min kCk) þ ckEk2;1 s:t: D ¼ DC þ E; ð19Þ
C
where C i is the ith column of C. Since this problem is combinatorial, P qffiP ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
2
a simpler ‘1 optimization problem is solved where kEk2;1 ¼ Nk¼1 N
j¼1 jEjk j is the ‘2;1 norm of the matrix of er-
rors E. Notice that the problem in (19) is analogous to that in (17),
min kCk1 s:t: D ¼ DC and diagðCÞ ¼ 0: ð15Þ except that the ‘1 norms of C and E are replaced by the nuclear norm
C
of C and the ‘2;1 norm of E, respectively. The nuclear norm is a con-
It is shown in Elhamifar and Vidal (2009, 2010, 2013) that, un- vex relaxation of the rank of C, while the ‘2;1 norm is a convex relax-
der some conditions on the subspaces and the data, the solutions to ation of the number of nonzero columns of E. Therefore, the LRR
the optimization problems in (14) and (15) are such that C ij ¼ 0 algorithm aims to minimize the rank of the matrix of coefficients
when points i and j are in different subspaces. In other words, and the number of corrupted data points, while the SSC algorithm
the nonzero coefficients of the ith column of C correspond to points aims to minimize the number of nonzero coefficients and the num-
in the same subspace as point i. Therefore, one can use C to define ber of corrupted entries. As before, the optimal C is used to define an
an affinity matrix as jCj þ jC > j. The segmentation of the data is then affinity matrix jCj þ jC > j and the segmentation of the data is ob-
obtained by applying spectral clustering (von Luxburg, 2007) to tained by applying spectral clustering to this affinity.
this affinity.
In the case of data contaminated by noise G, the SSC algorithm 3. Low rank subspace clustering with uncorrupted data
assumes that each data point can be written as a linear combina-
tion of other data points up to an error G, i.e., D ¼ DC þ G, and In this section, we consider the low rank subspace clustering
solves the following convex problem problem P in the case of uncorrupted data, i.e., a ¼ 1 and c ¼ 1
so that G ¼ E ¼ 0 and D ¼ A. In Section 3.1, we show that the opti-
a mal solution for C can be obtained in closed form from the SVD of A
min kCk1 þ kGk2F s:t: D ¼ DC þ G and diagðCÞ ¼ 0: ð16Þ
C;G 2 by applying a nonlinear thresholding to its singular values. In
In the case of data contaminated also by gross errors E, the SSC Section 3.2, we assume that the self-expressiveness constraint is
algorithm assumes that D ¼ DC þ G þ E, where E is sparse. Since satisfied exactly, i.e., s ¼ 1 so that A ¼ AC. As shown in Liu et al.
both C and E are sparse, the equation D ¼ DC þ G þ E ¼ (2011), the optimal solution to this problem can be obtained by
>
½DI#½C > E> # þ G means that each point is written as a sparse linear hard thresholding the singular values of A. Here, we provide a sim-
combination of a dictionary composed of all other data points plus pler derivation of this result.
the columns of the identity matrix I. Thus, one can find C by solving
the following convex optimization problem 3.1. Uncorrupted data and relaxed constraints
a Consider the optimization problem P with uncorrupted data

min kCk1 þ kGk2F þ ckEk1
C;G;E 2 ð17Þ and relaxed self-expressiveness constraint, i.e., s < 1 so that
s:t: D ¼ DC þ G þ E and diagðCÞ ¼ 0: A $ AC. In this case, the problem P reduces to
s
While SSC works well in practice, until recently there was no ðP1 Þ min kCk) þ kA + ACk2F s:t: C ¼ C > :
theoretical guarantee that, in the case of corrupted data, the
C 2
nonzero coefficients correspond to points in the same subspace. Notice that P1 is convex on C, but not strictly convex. Therefore,
We refer the reader to Soltanolkotabi et al. (2013) for very recent we do not know a priori if the minimizer of P 1 is unique. The fol-
results in this direction. Moreover, notice that the model is not lowing theorem shows that the minimizer of P1 is unique and
really a subspace plus error model, because a contaminated data can be computed in closed form from the SVD of A.
point is written as a linear combination of other contaminated
points plus an error. To the best of our knowledge, there is no Theorem 1. Let A ¼ U KV > be the SVD of A, where the diagonal
method that tries to simultaneously recover a clean dictionary entries of K ¼ diagðfki gÞ are the singular values of A in decreasing
and cluster the data within this framework. order. The optimal solution to P1 is given by
j.patrec.2013.08.006
# $
1 be equal to one. Now, if dj ¼ 0, then Kwj ¼ K1 V > 1 U C ej ¼ 0, where ej is
C ¼ VP s ðKÞV > ¼ V 1 I + K+2
1 V >1 ; ð20Þ
s the jth column of the identity. This means that the columns of U C
associated to dj ¼ 0 must be orthogonal to the columns of V 1 , and
where the operator P s acts on the diagonal entries of K as
( pffiffiffi hence the columns of U C associated with dj ¼ 1 must be in the range
1
: 1 + sx 2 x > 1= s of V 1 . Thus, U C ¼ ½ V 1 R1 U 2 R2 #P for some rotation matrices R1 and
P s ðxÞ¼ pffiffiffi ; ð21Þ
0 x 6 1= s R2 , and permutation matrix P, and so the optimal C is
% &
I 0
and U ¼ ½U 1 U 2 #; K ¼ diagðK1 ; K2 Þ and V ¼ ½V 1 V 2 # are partitioned C ¼ ½ V 1 R1 V 2 R2 # ½ V 1 R1 V 2 R2 #> ¼ V 1 V >1 ; ð26Þ
pffiffiffi pffiffiffi 0 0
according to the sets I1 ¼ fi : ki > 1= sg and I2 ¼ fi : ki 6 1= sg.
Moreover, the optimal value of P1 is as claimed. h
# $
:X 1 sX 2
Us ðAÞ¼ 1 + k+2
i þ ki : ð22Þ
i2I 1
2s 2 i2I 2 4. Low rank subspace clustering with noisy data
In this section, we consider the low rank subspace clustering

Proof. See Appendix B. h
problem P in the case of noisy data, i.e., k ¼ 1, so that E ¼ 0 and
Notice that the optimal solution for C is obtained by applying a D ¼ A þ G. While in principle the resulting problem appears to be
nonlinear thresholding P s to the singular values of A. Singular very similar to those in eqs. (16) and (19), there are a number of
values smaller than p1ffisffi are mapped to zero, while larger singular differences. First, notice that instead of expressing the noisy data
values are mapped closer to one. Notice also that the optimal value as a linear combination of itself plus noise, i.e., D ¼ DC þ G, we
of P 1 ; Us ðAÞ, is a decomposable function of the singular values of A, search for a clean dictionary, A, which is self-expressive, i.e.,
P 2
as are the Frobenius and nuclear norms of A; kAk2F ¼ ki and A ¼ AC. We then assume that the data are obtained by adding noise
P
kAk) ¼ ki , respectively. However, unlike kAkF or kAk) ; Us ðAÞ is to the clean dictionary, i.e., D ¼ A þ G. As a consequence, our meth-
not a convex function of A because Us ðkÞ is quadratic near zero od searches simultaneously for a clean dictionary A, the coeffi-
and saturates as k increases, as illustrated in Fig. 1(a). Interestingly, cients C and the noise G. Second, the main difference with (16) is
as s goes to infinity, Us ðAÞ approaches rankðAÞ, as we shall see. that the ‘1 norm of the matrix of the coefficients is replaced by
Therefore, we may view Us ðAÞ as a non-convex relaxation of the nuclear norm. Third, the main difference with (19) is that the
rankðAÞ. ‘2;1 norm of the matrix of the noise is replaced by the Frobenius
norm. Fourth, our method enforces the symmetry of the affinity
3.2. Uncorrupted data and exact constraints matrix as part of the optimization problem, rather than as a
post-processing step.
Consider now the optimization problem P with uncorrupted As we will show in this section, these modifications result in a
data and exact self-expressiveness constraint, i.e., s ¼ 1 so that key difference between our method and the state of the art: while
A $ AC. This leads to the following optimization problem the solution to (16) requires ‘1 minimization and the solution to
(19) requires an ALM method, the solution to P with noisy data
ðP2 Þ min kCk) s:t: A ¼ AC and C ¼ C > :
C can be computed in closed form from the SVD of the data matrix
D. For the relaxed problem, P 3 , the closed-form solution for A is
The following Theorem shows that the Costeira and Kanade
found by applying a polynomial thresholding to the singular values
affinity matrix C ¼ V 1 V >
1 is the optimal solution to P 2 . The theorem
of D, as we will see in Section 4.1. For the exact problem, P 4 , the
follows as a corollary of Theorem 1 by letting s ! 1. An alterna-
closed-form solution for A is given by classical PCA, except that
tive proof can be found in Liu et al. (2011). Here, we provide a sim-
the number of principal components can be automatically deter-
pler and more direct proof.
mined, as we will see in Section 4.2.
Theorem 2. Let A ¼ U KV > be the SVD of A, where the diagonal
4.1. Noisy data and relaxed constraints
entries of K ¼ diagðfki gÞ are the singular values of A in decreasing
order. The optimal solution to P2 is given by
Consider the optimization problem P with noisy data and re-
laxed self-expressiveness constraint, i.e.
C ¼ V 1 V >1 ; ð23Þ
s a
where V ¼ ½V 1 V 2 # is partitioned according to the sets I1 ¼ fi : ki > 0g ðP 3 Þ min kCk) þ kA + ACk2F þ kD + Ak2F s:t: C ¼ C > :
A;C 2 2
and I2 ¼ fi : ki ¼ 0g. Moreover, the optimal value is
X The key difference between P 3 and the problem P1 considered in
U1 ðAÞ ¼ 1 ¼ rankðAÞ: ð24Þ Section 3.1 is that A is now unknown. Hence the cost function in P 3
i2I1
is not convex in ðA; CÞ because of the product AC. Nonetheless, we
will show in this subsection that the optimal solution is still un-
Proof. Let C ¼ U C DU > ique, unless one of the singular values of D satisfies a constraint
C be the eigenvalue decomposition (EVD) of C.
Then A ¼ AC can be rewritten as U KV > ¼ U KV > U C DU > that depends on a and s. In such a degenerate case, the problem
C , which
reduces to has two optimal solutions. Moreover, the optimal solutions for
both A and C can be computed in closed form from the SVD of D,
KV > U C ¼ KV > U C D ð25Þ as stated in the following theorem.
since U U ¼ I and
>
¼ I. Let W ¼ V U C ¼ ½ w1 ; . . . ; wN #. Then,
U>
C UC
>
Theorem 3. Let D ¼ U RV > be the SVD of the data matrix D. The
Kwj ¼ Kwj dj for all j ¼ 1; . . . ; N. This means that dj ¼ 1 if Kwj –0 optimal solutions to P 3 are of the form
and dj is arbitrary otherwise. Since our goal is to minimize
P A ¼ U KV > and C ¼ VP s ðKÞV > ; ð27Þ
kCk) ¼ kDk) ¼ Nj¼1 jdj j, we need to set as many dj to zero as possible.
Since A ¼ AC implies that rankðAÞ 6 rankðCÞ, we can set at most where each entry of K ¼ diagðk1 ; . . . ; kn Þ is obtained from each entry of
N + rankðAÞ of the dj to zero and the remaining rankðAÞ of the dj must R ¼ diagðr1 ; . . . ; rn Þ as the solutions to
j.patrec.2013.08.006
Fig. 1. Plots of Us , w and P a;s .

( pffiffiffi
k þ a1s k+3 if k > 1= s; taking the derivative of /ðk; rÞ w.r.t. k and setting it to zero, we
r ¼ wðkÞ¼: pffiffiffi ð28Þ obtain r ¼ wðkÞ. Therefore, ki is the solution of ri ¼ wðkÞ that min-
k þ as k if k 6 1= s;
imizes /ðk; ri Þ, as claimed. For a more detailed proof, we refer the
that minimize reader to Appendix B. h
( pffiffiffi
1 + 21s k+2 k > 1= s; It follows from Theorem 3 that the singular values of A can be
: a
/ðk; rÞ¼ ðr + kÞ2 þ pffiffiffi ð29Þ obtained from those of D by solving the equation r ¼ wðkÞ. When
2 sk 2
k 6 1= s:
2 3s 6 a, the solution for k is unique (see Fig. 1(b)). However, when
3s > a there are three possible solutions (see Fig. 1(c)). The follow-
Proof. Notice that when A is fixed, P 3 reduces to P1 . Therefore, it ing theorem shows that, in general, only one of these solutions is
follows from Theorem 1 that, given A, the optimal solution for C the global minimizer. Moreover, this solution can be obtained by
is C ¼ VP s ðKÞV > , where A ¼ U KV > is the SVD of A. Moreover, it fol- applying a polynomial thresholding operator k ¼ P a;s ðrÞ to the sin-
lows from (22) that if we replace the optimal C into the cost of P 3 , gular values of D.
then P3 is reduces to
a Theorem 4. There exists a r) > 0 such that the solutions to (28) that
min Us ðAÞ þ kD + Ak2F : ð30Þ minimize (29) can be computed as
A 2
!
It is easy to see that, because Us ðAÞ is a decomposable function : b1 ðrÞ if r 6 r)
k ¼ P a;s ðrÞ¼ ð31Þ
of the singular values of A, the optimal solution for A in (30) must b3 ðrÞ if r > r) ;
be of the form A ¼ U KV > for some K ¼ diagðfki gni¼1 Þ. After :
where b1 ðrÞ ¼ aþs r
a and b3 ðrÞ is the real root of the polynomial
substituting this expression for A into (30), we obtain that K must
P
minimize Us ðKÞ þ a2 kR + Kk2F ¼ ni¼1 /ðki ; ri Þ. Therefore, each ki can 1
pðkÞ ¼ k4 + rk3 þ ¼0 ð32Þ
be obtained independently as the minimizer of /ðk; ri Þ. After as
j.patrec.2013.08.006
that minimizes (29). If 3s 6 a, the solution for k is unique and thresholding operator Hpffi2 in (3). Therefore, the optimal A can be
a
# $
1 aþs obtained from the SVD of D ¼ U RV > as A ¼ UHpffi2 ðRÞV > , while
r) ¼ w pffiffiffi ¼ pffiffiffi : ð33Þ a
s a s the optimal C is given by Theorem 2.
If 3s > a, the solution for k is unique, except when r satisfies
Theorem 5. Let D ¼ U RV > be the SVD of the data matrix D. The
/ðb1 ðrÞ; rÞ ¼ /ðb3 ðrÞ; rÞ; ð34Þ optimal solution to P4 is given by
and r) must lie in the range
rffiffiffiffiffiffi A ¼ U 1 R1 V >1 and C ¼ V 1 V >1 ; ð41Þ
4 4 3 aþs qffiffi
< r) < pffiffiffi : ð35Þ where R1 contains the singular values of D that are larger than 2
3 as a s a, and
U 1 and V 1 contain the corresponding singular vectors.
Proof. See Appendix B. h

5. Low rank subspace clustering with corrupted data
It follows from Theorems 3 and 4 that the optimal solutions to
P 3 can be obtained from the SVD of D ¼ U RV > as
In this section, we consider the low-rank subspace clustering
A ¼ UP a;s ðRÞV > and C ¼ VP s ðP a;s ðRÞÞV > : ð36Þ problem in the case of data corrupted by both noise and gross er-
rors, i.e., problem P. Similar to the case of noisy data discussed in
However, to compute P a;s , we need to compute the threshold r) .
Section 4, the major difference between P and the problems in
Unfortunately, finding a formula for r) is not straightforward, be-
(17) and (19) is that, rather than using a corrupted dictionary,
cause it requires solving (34) (see B for details). While this equation
we search simultaneously for the clean dictionary A, the low-rank
can be solved numerically for each a and s, a simple closed-form
coefficients C and the sparse errors E. Also, notice that the ‘1 norm
formula can be obtained when a1s ’ 0 (relative to r). In this case, of the matrix of coefficients is replaced by the nuclear norm, the
the quartic becomes pðkÞ ¼ k4 + rk3 ¼ 0, which can be immediately ‘2;1 norm of the matrix of errors is replaced by the ‘1 norm, and
solved and yields three solutions that are equal to 0 and are hence we enforce the symmetry of the affinity matrix as part of the opti-
pffiffiffi
out of the range k > 1= s. The only valid solution to the quartic is mization problem rather than as a post-processing. A closed form
pffiffiffi solution to the low-rank subspace clustering problem in the case
k¼r 8r : r > 1= s: ð37Þ
of data corrupted by noise and gross errors appears elusive at this
Thus, a simpler thresholding procedure can be obtained by point. Therefore, we propose to solve P using an alternating mini-
approximating the thresholding function with two piecewise lin- mization approach.
pffiffiffi
ear functions. One is exact (when k 6 1= s) and the other one is
pffiffiffi
approximate (when k > 1= s). The approximation, however, is 5.1. Corrupted data and relaxed constraints
quite accurate for a wide range of values for a and s. Since we have
two linear functions, we can easily find a threshold for r as the va- 5.1.1. Iterative polynomial thresholding (IPT)
lue r
e ) at which the discontinuity happens. To do so, we can plug in We begin by considering the relaxed problem P, which is equiv-
the given solutions in (34). We obtain alent to
as e 2 1 ðP 5 Þ min kCk) þ 2s kA + ACk2F þ a2 kD + A + Ek2F þ ckEk1
r ¼ 1 + e2 : ð38Þ A;C;E
2ða þ sÞ ) 2s r )
s:t: C ¼ C > :
This gives 4 solutions, out of which the only suitable one is
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffir
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi When E is fixed, this problem reduces to P 3 , except that D is re-
a þs aþs placed by D + E. Therefore, it follows from (36) that A and C can be
re ) ¼ þ : ð39Þ computed from the SVD of D + E ¼ U RV > as
as 2 as
Finally, the approximate polynomial thresholding operator can A ¼ UP a;s ðRÞV > and C ¼ VP s ðP a;s ðRÞÞV > ; ð42Þ
be written as where P s is the operator in (21) and P a;s is the polynomial thres-
(
r if r > r
e) holding operator in (31). When A and C are fixed, the optimal solu-
e a;s ðrÞ ¼
k¼P ð40Þ tion for E satisfies
aþs raif r 6 r
e ):
+aðD + A + EÞ þ csignðEÞ ¼ 0: ð43Þ
Notice that as s increases, the largest singular values of D are
preserved, rather than shrunk by the operator S a+1 in (7). Notice This equation can be solved in closed form by using the shrink-
also that the smallest singular values of D are shrunk by scaling age-thresholding operator in (7) and the solution is
them down, as opposed to subtracting a threshold. E ¼ S ac ðD + AÞ: ð44Þ
4.2. Noisy data and exact constraints This suggest an iterative thresholding algorithm that, starting
from A0 ¼ D and E0 ¼ 0, alternates between applying polynomial
Assume now that the data are generated from the exact self- thresholding to D + Ek to obtain Akþ1 and applying shrinkage-thres-
expressive model, A ¼ AC, and contaminated by noise, i.e., holding to D + Akþ1 to obtain Ekþ1 , i.e.,
D ¼ A þ G. This leads to the optimization problem ðU k ; Rk ; V k Þ ¼ svdðD + Ek Þ
a Akþ1 ¼ U k P a;s ðRk ÞV >k
ðP4 Þ min kCk) þ kD + Ak2F s:t: A ¼ AC and C ¼ C : > ð45Þ
A;C 2
Ekþ1 ¼ S ca+1 ðD + Akþ1 Þ:
This problem can be seen as the limiting case of P3 with s ! 1.
qffiffi Notice that we do not need to compute C at each iteration be-
In this case, b1 ðrÞ ¼ r, b3 ðrÞ ¼ 0 and r) ¼ 2a, hence the cause the updates for A and E do not depend on C. Therefore, we
polynomial thresholding operator P a;s in (31) reduces to the hard can obtain C from A upon convergence. Although the optimization
j.patrec.2013.08.006
problem in (42) is non-convex, the algorithm in (45) is guaranteed 5.2. Corrupted data and exact constraints
to converge, as shown in Tseng (2001). Specifically, it follows from
Theorem 1 that the optimization problem in (42) is equivalent to Let us now consider the low rank subspace clustering problem
the minimization of the cost function P, where the constraint A ¼ AC is enforced exactly, i.e.,
a min kCk) þ a2 kGk2F þ ckEk1

f ðA; EÞ ¼ Us ðAÞ þ kD + A + Ek2F þ ckEk1 : ð46Þ ðP6 Þ A;C;E;G
2
s:t: D ¼ A þ G þ E; A ¼ AC and C ¼ C > :
It is easy to see that the algorithm in (45) is a coordinate des-
cent method applied to the minimization of f. This function is con- This problem can be seen as the limiting case of P 5 with s ! 1.
tinuous, has a compact level set fðA; EÞ : f ðA; EÞ 6 f ðA0 ; E0 Þg, and has In this case, the polynomial thresholding operator P a;s becomes the
at most one minimum in E as per (44). Therefore, it follows from hard thresholding operator Hpffi2 . Therefore, we can solve P6 using
a
Theorem 4.1 part (c) in Tseng (2001) that the algorithm in (45) the IPT and ADMM algorithms described in Section 5.1 with P a;s re-
converges to a coordinate-wise minimum of f. placed by Hp2ffi .
Notice, however, that this minimum is not guaranteed to be a a
global minimum. Moreover, in practice its convergence can be

slow as observed in Lin et al. (2011) for similar problems. 6. Experiments
In this section we evaluate the performance of LRSC on two

5.1.2. Alternating direction method of multipliers (ADMM) computer vision tasks: motion segmentation and face clustering.
We now propose an alternative solution to P 5 in which we en- Using the subspace clustering error,
force the constraint D ¼ A þ E exactly. This means that we tolerate
# of misclassified points
outliers, but we do not tolerate noise. Using the method of multi- Subspace clustering error ¼ ; ð51Þ
total # of points
pliers, this problem can be formulated as
as a measure of performance, we compare LRSC to state-of-the-art
s l
max min > kCk) þ kA + ACk2F þ kD + A + Ek2F subspace clustering algorithms based on spectral clustering, such
Y A;E;C:C¼C 2 2 as LSA (Yan and Pollefeys, 2006), SCC (Chen and Lerman, 2009),
þ hY; D + A + Ei þ ckEk1 : ð47Þ LRR (Liu et al., 2010), and SSC (Elhamifar and Vidal, 2013). We
choose these methods as a baseline, because they have been shown
In this formulation, the term with l does not play the role of
to perform very well on the above tasks, as reported in Vidal (2011).
penalizing the noise G ¼ D + A + E, as before. Instead, it augments
For the state-of-the-art algorithms, we use the implementations
the Lagrangian with the squared norm of the constraint.
provided by their authors. The parameters of the different methods
To solve the minimization problem over ðA; E; CÞ, notice that
are set as shown in Table 2.
when E is fixed the optimization over A and C is equivalent to
Notice that the SSC and LRR algorithms in Elhamifar and Vidal
s l (2013) and Liu et al. (2010), respectively, apply spectral clustering
min kCk) þ kA + ACk2F þ kD + A + E þ l+1 Yk2F s:t: C ¼ C > : to a similarity graph built from the solution of their proposed opti-
A;C 2 2
mization programs. Specifically, SSC uses the affinity jCj þ jCj> ,
It follows from (36) that the optimal solutions for A and C can be while LRR uses the affinity jCj. However, the implementation of
computed from the SVD of D + E þ l+1 Y ¼ U RV > as the SSC algorithm normalizes the columns of C to be of unit ‘1
norm. To investigate the effect of this post-processing step, we re-
A ¼ UP l;s ðRÞV > and C ¼ VP s ðP l;s ðRÞÞV > ; ð48Þ port the results for both cases of without (SSC) and with (SSC-N)
the column normalization step. Also, the code of the LRR algorithm
Conversely, when A and C are fixed, the optimization problem
in Liu et al. (2012) applies a heuristic post-processing step to the
over E reduces to
low-rank solution prior to building the similarity graph, similar
l to Lauer and Schnörr (2009). Thus, we report the results for both
min kD + A + E þ l+1 Yk2F þ ckEk1 : ð49Þ without (LRR) and with (LRR-H) the heuristic post-processing step.
E 2
Notice also that the original published code of LRR contains the
As discussed in Section 2.1, the optimal solution for E is given as function ‘‘compacc.m’’ for computing the misclassification rate,
E ¼ S c=l ðD + A þ l+1 YÞ. which is erroneous, as noted in Elhamifar and Vidal (2013). Here,
Given A and E, the ADMM algorithm updates Y using gradient we use the correct code for computing the misclassification rate
ascent with step size l, which gives Y Y þ lðD + A + EÞ. There- and as a result, the reported performance for LRR-H is different
fore, starting from A0 ¼ D, E0 ¼ 0 and Y 0 ¼ 0, we obtain the follow- from the published results in Liu et al. (2010, 2012). Likewise,
ing ADMM for solving the low-rank subspace clustering problem in our results for LRSC are different from those in Favaro et al.
the presence of gross corruptions, (2011), which also used the erroneous function ‘‘compacc.m’’.
Finally, to have a fair comparison, since LSA and SCC need to
ðU k ; Rk ; V k Þ ¼ svdðD + Ek þ l+1
k YkÞ know the number of subspaces a priori and the estimation of the
Akþ1 ¼ U k P lk ;s ðRk ÞV >k number of subspaces from the eigenspectrum of the graph Lapla-
Ekþ1 ¼ S cl+1 ðD + Akþ1 þ l+1 cian in the noisy setting is often unreliable, we provide the number
k Y kÞ ð50Þ
k
of subspaces as an input to all the algorithms.
Y kþ1 ¼ Y k þ lk ðD + Akþ1 + Ekþ1 Þ
lkþ1 ¼ qlk ; 6.1. Experiments on motion segmentation
where q > 1 is a parameter. As in the case of the IPT method, C is Motion segmentation refers to the problem of clustering a set of
obtained from A upon convergence. Experimentally, we have ob- 2D point trajectories extracted from a video sequence into groups
served that our method always converges. However, while the con- corresponding to different rigid-body motions. Here, the data ma-
vergence of the ADMM is well studied for convex problems, we are trix D is of dimension 2F ' N, where N is the number of 2D trajec-
not aware of any extensions to the nonconvex case. tories and F is the number of frames in the video. Under the affine
j.patrec.2013.08.006
Table 2 6.2. Experiments on face clustering

Parameter setup of different algorithms. K is the number of nearest neighbors used by
LSA to fit a local subspace around each data point, d is the dimension of each subspace
assumed by LSA and SCC, s is a parameter weighting the self-expressiveness error, a is
Face clustering refers to the problem of clustering a set of face
a parameter weighting noise, and c is a parameter weighting gross errors by the ‘1 images from multiple individuals according to the identity of each
(SSC, LRSC) or ‘2;1 (LRR) norms. individual. Here, the data matrix D is of dimension P ' N, where P
Parameter LSA SCC LRR SSC LRSC
is the number of pixels, and N is the number of images. For a
Lambertian object, the set of all images taken under all lighting
Motion segmentation
K 8
conditions, but the same viewpoint and expression, forms a cone
d 4 3 in the image space, which can be well approximated by a low-
s 420, 4:5'10 4
pffiffiffiffiffi , 6'10
4
pffiffiffiffiffi dimensional subspace (Basri and Jacobs, 2003). In practice, a few
MN MN
a 800
>
3000, 5000, 1 pixels deviate from the Lambertian model due to cast shadows
mini maxj–i jdi dj j
and specularities, which can be modeled as sparse outlying entries.
c 4 1 5, 1
g 0.03 Therefore, the face clustering problem reduces to clustering a set of
l0 100 images according to multiple subspaces and corrupted by sparse
q 1.0 1.1 gross errors.
Face clustering We use the Extended Yale B database (Lee et al., 2005) to eval-
K 7 uate the performance of LRSC against that of state-of-the-art meth-
d 5 9 ods. The database includes 64 frontal face images of 38 individuals
s 0.01, 0.03, 0.045, 0.07, 0.1
a 1 0.03, 0.045, 0.07, 0.1, 1
acquired under 64 different lighting conditions. Each image is
c 0.18 20
10+6 , 0.01, 1
cropped to 192 ' 168 pixels. Fig. 3 shows sample images from
mini maxj–i kdj k1
g 0.03, 0.04 the database. To reduce the computational cost and the memory
l0 0.03 requirements of all algorithms, we downsample the images to
q 1.0 1.5 48 ' 42 pixels and treat each 2; 016-dimensional vectorized image
as a data point.
Following the experimental setup of Elhamifar and Vidal
(2013), we divide the 38 subjects into 4 groups, where the first
projection model, the 2D trajectories associated with a single rigid- three groups correspond to subjects 1 to 10, 11 to 20, 21 to 30,
body motion live in an affine subspace of R2F of dimension d ¼ 1; 2 and the fourth group corresponds to subjects 31 to 38. For each
or 3 (Tomasi and Kanade, 1992). Therefore, the trajectories associ- of the first three groups we consider all choices of
ated with n different moving objects lie in a union of n affine sub- n 2 f2; 3; 5; 8; 10g subjects and for the last group we consider all
spaces in R2F , and the motion segmentation problem reduces to choices of n 2 f2; 3; 5; 8g. Finally, we apply all subspace clustering
clustering a collection of point trajectories according to multiple algorithms for each trial, i.e., each set of n subjects.
affine subspaces. Since LRSC is designed to cluster linear subspaces, In Table 5 we report Table 3 of Elhamifar and Vidal (2013). This
we apply LRSC to the trajectories in homogeneous coordinates, i.e., table shows the average and median subspace clustering errors of
we append a constant g ¼ 0:1 and work with 2F þ 1 dimensional different algorithms. The results are obtained by first applying the
vectors. Robust Principal Component Analysis (RPCA) algorithm of Candès
We use the Hopkins155 motion segmentation database (Tron et al. (2011) to the face images of each subject and then applying
and Vidal, 2007) to evaluate the performance of LRSC against that different subspace clustering algorithms to the low-rank compo-
of other algorithms. The database, which is available online at nent of the data obtained by RPCA. The purpose of this experiment
http://www.vision.jhu.edu/data/hopkins155, consists of 155 is to show that when there are no outliers in the data, our solution
sequences of two and three motions. For each sequence, the 2D correctly identifies the subspaces. As described in Elhamifar and
trajectories are extracted automatically with a tracker and outliers Vidal, 2013, notice that LSA and SCC do not perform well, even with
are manually removed. Fig. 2 shows some sample images with the de-corrupted data. Notice also that LRR-H does not perform well
feature points superimposed. for more than 8 subjects, showing that the post processing step
Table 3(a) and (b) give the average subspace clustering error ob- on the obtained low-rank coefficient matrix not always improves
tained by different variants of LRSC on the Hopkins 155 motion the result of LRR. SSC and LRSC, on the other hand, perform very
segmentation database. We can see that most variants of LRSC well, with LRSC achieving perfect performance. In this test LRSC
have a similar performance. This is expected, because the trajecto- corresponds to P 5 + ADMM with an additional term hY 1 ; A + ACi,
ries are corrupted by noise, but do not have gross errors. Therefore, where Y 1 is the Lagrange multiplier for the equation A ¼ AC, with
the Frobenius norm on the errors performs almost as well as the ‘1 parameters l0 ¼ 3s0 ¼ 0:5ð1:25=r1 ðDÞÞ2 , c ¼ 0:008, q ¼ 1:8. The
norm. However, the performance depends on the choice of the update step for the additional Lagrange multiplier and the aug-
parameters. In particular, notice that choosing s that depends on mented coefficient sk is Y 1;kþ1 ¼ Y 1;k þ sk ðAkþ1 + Akþ1 C kþ1 Þ and
the number of motions and size of each sequence gives better skþ1 ¼ qsk . We also have repeated the test with our current imple-
results than using a fixed s. mentations and also found the error to be consistently 0 across all
Table 4(a) and (b) compare the best results of LRSC against the subjects. In our test we preprocessed each subject with
state-of-the-art results. Overall, LRSC compares favorably against P5 + ADMM with parameters g ¼ 0, s ¼ 0:01, c ¼ 10+6 and run
LSA, SCC and LRR without post-processing of the affinity matrix. for 10 iterations. Then we ran P 3 with parameters g ¼ 0:04,
Relative to LRR with post-processing, LRSC performs worse when s ¼ 0:1 and a ¼ 0:1.
the data is not projected, and better when the data is projected. Table 6 shows the results of applying different clustering algo-
However, LRSC does not perform as well as either version of SSC rithms to the original data, without first applying RPCA to each
(with or without post-processing). group. Notice that the performance of LSA and SCC deteriorates
Overall, we can see that even the simplest version of LRSC (P1 ), dramatically, showing that these methods are very sensitive to
whose solution can be computed in closed form, performs on par gross errors. The performance of LRR is better, but the errors are
with state-of-the-art motion segmentation methods, which re- still very high, especially as the number of subjects increases. In
quire solving a convex optimization problem. We also notice that this case, the post processing step of LRR-H does help to signifi-
LRSC has almost the same performance with or without projection. cantly reduce the clustering error. SSC-N, on the other hand,
j.patrec.2013.08.006
Fig. 2. Motion segmentation: given feature points on multiple rigidly moving objects tracked in multiple frames of a video (top), the goal is to separate the feature trajectories
according to the moving objects (bottom).
Table 3
Clustering error (%) of different variants of the LRSC algorithm on the Hopkins 155 database. The parameters in the first four columns are set as s ¼ 420, a ¼ 3000 for 2 motions,
4 4
a ¼ 5000 for 3 motions and c ¼ 5. The parameters in the last four columns are set as s ¼ 4:5'10
pffiffiffiffiffi and a ¼ 3000 for two motions, s ¼ 6'10
MN
pffiffiffiffiffi and a ¼ 5000 for 3 motions, and c ¼ 5. For
MN
P5 -ADMM, we set l0 ¼ 100 and q ¼ 1:1. In all cases, we use homogeneous coordinates with g ¼ 0:1. Boldface indicates the top performing algorithm in each experiment.
Method P1 P3 P 5 -ADMM P 5 -IPT P1 P3 P 5 -ADMM P 5 -IPT

(a) 2F-dimensional data points
2 Motions
Mean 3.39 3.27 3.13 3.27 2.58 2.57 2.62 2.57
Median 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
3 Motions
Mean 7.28 7.29 7.31 7.29 6.68 6.64 6.76 6.67
Median 2.53 2.53 2.53 2.53 1.76 1.76 1.76 1.76
All
Mean 4.25 4.16 4.05 4.16 3.49 3.47 3.53 3.48
Median 0.00 0.19 0.00 0.19 0.09 0.09 0.00 0.09
(b) 4n-dimensional data points obtained by applying PCA to original data
2 Motions
Mean 3.19 3.28 3.93 3.28 2.59 2.57 3.43 2.57
Median 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
3 Motions
Mean 7.27 7.28 8.07 7.28 6.64 6.67 8.13 6.62
Median 2.54 2.54 3.76 2.54 1.76 1.76 2.30 1.76
All
Mean 4.09 4.16 4.85 4.17 3.49 3.48 4.48 3.47
Median 0.19 0.19 0.21 0.19 0.19 0.09 0.19 0.00
performs very well, achieving a clustering error of about 10% for 10 7. Discussion and conclusion
subjects. Table 7 shows the performance of different variants of
LRSC on the same data. P1 has the best performance for 2 and 3 We have proposed a new algorithm for clustering data drawn
subjects, and the worst performance for 5, 8 and 10 subjects. All from a union of subspaces and corrupted by noise/gross errors.
other variants of LRSC have similar performance, perhaps with Our approach was based on solving a non-convex optimization
P5 -IPT being the best. Overall, LRSC performs better than LSA, problem whose solution provides an affinity matrix for spectral
SCC and LRR, and worse than LRR-H and SSC-N. clustering. Our key contribution was to show that important par-
Finally, Fig. 4 shows the average computational time of each ticular cases of our formulation can be solved in closed form by
algorithm as a function of the number of subjects (or equivalently applying a polynomial thresholding operator to the SVD of the
the number of data points). Note that the computational time of data. A drawback of our approach to be addressed in the future
SCC is drastically higher than other algorithms. This comes from is the need to tune the parameters of our cost function. Further re-
the fact that the complexity of SCC increases exponentially in the search is also needed to understand the correctness of the resulting
dimension of the subspaces, which in this case is d ¼ 9. On the other affinity matrix in the presence of noise and corruptions. Finally, all
hand, SSC, LRR and LRSC use fast and efficient convex optimization existing methods decouple the learning of the affinity from the
techniques which keep their computational time lower than that of segmentation of the data. Further research is needed to integrate
other algorithms. Overall, LRR and LRSC are the fastest methods. these two steps into a single objective.
j.patrec.2013.08.006
Table 4
Clustering error (%) on the Hopkins 155 database. Boldface indicates the top performing algorithm in each experiment.
Method LSA SCC LRR LRR-H SSC SSC-N LRSC

(a) 2F-dimensional data points
2 Motions
Mean 4.23 2.89 4.10 2.13 2.07 1.52 2.57
Median 0.56 0.00 0.22 0.00 0.00 0.00 0.00
3 Motions
Mean 7.02 8.25 9.89 4.03 5.27 4.40 6.64
Median 1.45 0.24 6.22 1.43 0.40 0.56 1.76
All
Mean 4.86 4.10 5.41 2.56 2.79 2.18 3.47
Median 0.89 0.00 0.53 0.00 0.00 0.00 0.09
(b) 4n-dimensional data points obtained by applying PCA to original data

2 Motions
Mean 3:61 3:04 4:83 3:41 2.14 1:83 2:57
Median 0:51 0:00 0:26 0.00 0:00 0:00 0:00
3 Motions
Mean 7:65 7:91 9:89 4:86 5.29 4:40 6:62
Median 1:27 1:14 6:22 1:47 0.40 0:56 1:76
All
Mean 4:52 4:14 5:98 3:74 2.85 2:41 3:47
Median 0:57 0.00 0:59 0.00 0:00 0:00 0.00
Fig. 3. Face clustering: given face images of multiple subjects (top), the goal is to find images that belong to the same subject (bottom).
Appendix A. The Von Neumann trace inequality where r1 ðXÞ P r2 ðXÞ P - - - P 0 and r1 ðYÞ P r2 ðYÞ P - - - P 0 are
the descending singular values of X and Y respectively. The case of
This appendix reviews two matrix product inequalities, which equality occurs if and only if it is possible to find unitary matrices U X
we will use later in our derivations. and V X that simultaneously singular value-decompose X and Y in the
sense that
Lemma 1 (Von Neumann’s inequality). For any m ' n real valued
matrices X and Y, X ¼ U X RX V >X and Y ¼ U X RY V >X ; ðA:2Þ
n
X where RX and RY denote the m ' n diagonal matrices with the singular
traceðX > YÞ 6 ri ðXÞri ðYÞ; ðA:1Þ
values of X and Y, respectively, down in the diagonal.
i¼1
Table 5 Table 6
Clustering error (%) of different algorithms on the Extended Yale B database after Clustering error (%) of different algorithms on the Extended Yale B database without
applying RPCA separately to the images from each subject. Boldface indicates the top pre-processing the data. Boldface indicates the top performing algorithm in each
performing algorithm in each experiment. experiment.
Method LSA SCC LRR LRR-H SSC-N LRSC Method LSA SCC LRR LRR-H SSC-N
2 Subjects 2 Subjects
Mean 6:15 1:29 0:09 0:05 0:06 0.00 Mean 32.80 16.62 9.52 2.54 1.86
Median 0.00 0.00 0.00 0.00 0.00 0.00 Median 47.66 7.82 5.47 0.78 0.00
Mean 11:67 19:33 0:12 0:10 0:08 0.00 Mean 52:29 38:16 19:52 4:21 3.10
Median 2:60 8:59 0.00 0.00 0.00 0.00 Median 50.00 39.06 14.58 2.60 1.04
Mean 21:08 47:53 0:16 0:15 0:07 0.00 Mean 58:02 58:90 34:16 6:90 4.31
Median 19:21 47:19 0.00 0.00 0.00 0.00 Median 56.87 59.38 35.00 5.63 2.50
Mean 30:04 64:20 4:50 11:57 0:06 0.00 Mean 59:19 66:11 41:19 14:34 5.85
Median 29:00 63:77 0:20 15:43 0.00 0.00 Median 58.59 64.65 43.75 10.06 4.49
Mean 35:31 63:80 0:15 13:02 0:89 0.00 Mean 60:42 73:02 38:85 22:92 10.94
Median 30:16 64:84 0.00 13:13 0:31 0.00 Median 57.50 75.78 41.09 23.59 5.63
j.patrec.2013.08.006
Table 7 Proof. Let Y ¼ kI + Z, where k P kZk2 . Then

Clustering error (%) of our algorithms on the Extended Yale B database without pre- traceðXYÞ ¼ traceðXðkI + ZÞÞ ¼ ktraceðXÞ + traceðXZÞ. Also,
processing the data. The parameters of LRSC P1 and P3 are set as s ¼ 0:045, and
a ¼ 0:045, while the coefficient for homogeneous coordinates is set as g ¼ 0:03. P5 - n
X Xn
ADMM was run for 20 iterations, with parameters s ¼ 0:03, c ¼ 0:01, g ¼ 0:03, traceðXYÞ 6 ri ðXÞri ðYÞ ¼ i¼1
ri ðXÞri ðkI + ZÞ
l0 ¼ 0:03 and q ¼ 1:5. P5 -IPT was run for 10 iterations, with parameters a ¼ 0:07, i¼1
s ¼ 0:07, c ¼ 0:01, and g ¼ 0:03. n
X
P1 P3 P 5 -ADMM P 5 -IPT
¼ ri ðXÞðk + rn+iþ1 ðZÞÞ
i¼1
2 Subjects n
X
3.15 4.28 3.27 4.52 ¼ ktraceðXÞ + ri ðXÞrn+iþ1 ðZÞ: ðA:5Þ
2.34 3.91 2.34 3.91
i¼1
3 Subjects P
4.71 6.23 5.46 5.80 It follows from Lemma 1 that traceðXZÞ P ni¼1 ri ðXÞrn+iþ1 ðZÞ,
4.17 5.73 4.17 5.73 as claimed. Moreover, the equality is achieved if an only if there
5 Subjects exists a matrix U X (recall that X and Z are symmetric) such that
X and Y ¼ U X RY U X . Therefore,
X ¼ U X RX U > >
13.06 14.31 14.31 8.97
8.44 7.81 8.44 7.81
Z ¼ kI + Y ¼ kI + U X RY U >X ¼ kI + U X RkI+Z U >X
8 Subjects
26.83 23.73 29.91 22.61 ¼ kI + U X ðkI + PRZ P> ÞU >X ¼ U X PRZ P> U >X ðA:6Þ
28.71 27.83 30.27 26.46
10 Subjects
as claimed. h
35.89 31.25 36.88 29.01
34.84 27.65 36.25 26.88
Appendix B. Proofs of the main theorems
Theorem 6. Let A ¼ U KV > be the SVD of A, where the diagonal

entries of K ¼ diagðfki gÞ are the singular values of A in decreasing
order. The optimal solution to P1 is
# $
1
C ¼ VP s ðKÞV > ¼ V 1 I + K+2
1 V >1 ; ðB:1Þ
s
where the operator P s acts on the diagonal entries of K as
( pffiffiffi
1 + s1x2 x > 1= s;
P s ðxÞ ¼ pffiffiffi ðB:2Þ
0 x 6 1= s;
and U ¼ ½U 1 U 2 #, K ¼ diagðK1 ; K2 Þ and V ¼ ½V 1 V 2 # are partitioned

pffiffiffi pffiffiffi
according to the sets I1 ¼ fi : ki > 1= sg and I2 ¼ fi : ki 6 1= sg.
Moreover, the optimal value is
# $
:X 1 sX 2
Us ðAÞ¼ 1 + k+2i þ ki : ðB:3Þ
i2I1
2 s 2 i2I 2
Fig. 4. Average computational time (s) of the algorithms on the Extended Yale B
database as a function of the number of subjects. Proof. Let A ¼ U KV > be the SVD of A and C ¼ U C DU >
C be the eigen-
value decomposition (EVD) of C. The cost function of P1 reduces to
Proof. See Mirsky (1975). h s

kU C DU >C k) þ kU KV > ðI + U C DU >C Þk2F
2
s
Lemma 2. For any n ' n real valued, symmetric positive definite ¼ kDk) þ kKV > U C ðI + DÞU >C k2F
2
matrices X and Z, s
¼ kDk) þ kKWðI + DÞk2F ; ðB:4Þ
n
2
X
traceðXZÞ P ri ðXÞrn+iþ1 ðZÞ; ðA:3Þ where W ¼ V U C . To minimize this cost with respect to W, we only
>
i¼1 need to consider the last term of the cost function, i.e.,
' (
where r1 ðXÞ P r2 ðXÞ P - - - P 0 and r1 ðZÞ P r2 ðZÞ P - - - P 0 are kKWðI + DÞk2F ¼ trace ðI + DÞ2 W > K2 W : ðB:5Þ
the descending singular values of X and Z, respectively. The case of
Applying Lemma 2 to X ¼ WðI + DÞ2 W > and Z ¼ K2 , we obtain
equality occurs if and only if it is possible to find a unitary matrix U X
that for all unitary matrices W
that simultaneously singular value-decomposes X and Z in the sense
that ' ( XN ' (
min trace ðI + DÞ2 W > K2 W ¼ ri ðI + DÞ2 rn+iþ1 ðK2 Þ; ðB:6Þ
W
i¼1
X ¼ UX R >
X UX and Z ¼ U X PRZ P >
U >X ; ðA:4Þ
where the minimum is achieved by a permutation matrix W ¼ P>
where RX and RZ denote the n ' n diagonal matrices with the singular that sorts the diagonal entries of K2 in ascending order, i.e., the
values of X and Z, respectively, down in the diagonal in descending diagonal entries of PK2 P> are in ascending order.
2 2
order, and P is a permutation matrix such that PRZ P> contains the Let the ith' largest( entry of ðI + DÞ and K be, respectively,
2 2 2 2 2
singular values of Z in the diagonal in ascending order. ð1 + di Þ ¼ ri ðI + DÞ and mn+iþ1 ¼ ki ¼ ri ðK Þ. Then
j.patrec.2013.08.006
s N
X N
sX Proof. The proof of this result will be done in two steps. First, we
min kDk) þ kKWðI + DÞk2F ¼ jdi j þ m2i ð1 + di Þ2 : ðB:7Þ will show that the optimal C can be computed in closed form from
W 2 i¼1
2 i¼1
the SVD of A. Second, we will show that the optimal A can be
To find the optimal D, we can solve for each di independently as obtained in closed form from the SVD of D.
di ¼ argmind jdj þ 2s m2i ð1 + dÞ2 . As shown, e.g., by Parikh and Boyd
A closed-form solution for C. Notice that when A is fixed, P3
(2013) , the solution to this problem can be found in closed form
reduces to P1 . Therefore, it follows from Theorem 1 that the
using the shrinkage-thresholding operator in (7), which gives
( pffiffiffi optimal solution for C is C ¼ VP s ðKÞV > , where A ¼ U KV > is the
1 + s1m2 mi > 1= s SVD of A. Moreover, it follows from (22) that if we replace the
di ¼ S 1 ð1Þ ¼ i
pffiffiffi : ðB:8Þ optimal C into the cost of P3 , then P3 is equivalent to
sm2
i 0 mi 6 1= s
a
Then, di ¼ P s ðkn+iþ1 Þ, which can be compactly written as min Us ðAÞ þ kD + Ak2F : ðB:15Þ
A 2
D ¼ PP s ðKÞP> . Therefore,
" # A closed-form solution for A. Let D ¼ U RV > and A ¼ U A KV >
A be the
> I + 1s K+2
1 0 SVDs of D and A, respectively. Then,
P DP ¼ P s ð KÞ ¼ ; ðB:9Þ
0 0
kD + Ak2F ¼ kU RV > + U A KV >A k2F ;
where K ¼ diagðK1 ; K2 Þ is partitioned according to the sets ¼ kRk2F + 2traceðV RU > U A KV >A Þ þ kKk2F ; ðB:16Þ
pffiffiffi pffiffiffi
I1 ¼ fi : ki > 1= sg and I2 ¼ fi : ki 6 1= sg.
To find the optimal W, notice from Lemma 2 that the equality ¼ kR k2F + 2traceðRW 1 K W >2 Þ þ kK k2F ;
' ( P
trace ðI + DÞ2 W > K2 W ¼ N 2 2
where W 1 ¼ U U A and W 2 ¼ V > V A . Therefore, the minimization
>
i¼1 ð1 + di Þ kn+iþ1 is achieved if and
only if there exists a unitary matrix U X such that over A in (B.15) can be carried out by minimizing first with respect
to W 1 and W 2 and then with respect to K.
ðI + DÞ2 ¼ U X ðI + DÞ2 U >X and W > K2 W ¼ U X PK2 P> U >X : ðB:10Þ The minimization over W 1 and W 2 is equivalent to
Since the SVD of a matrix is unique up to the sign of the singular max traceðRW 1 KW >2 Þ: ðB:17Þ
vectors associated with different singular values and up to a W 1 ;W 2
rotation and sign of the singular vectors associated with repeated By letting X ¼ R and Y ¼ W 1 KW >
2 in Lemma 1, we obtain
singular values, we conclude that U X ¼ I up to the aforementioned
n
X n
X
ambiguities of the SVD of ðI + DÞ2 . Likewise, we have that
max traceðRW 1 KW >2 Þ ¼ ri ðRÞri ðKÞ ¼ ri ki : ðB:18Þ
W > ¼ U X P up to the aforementioned ambiguities of the SVD of W 1 ;W 2
i¼1 i¼1
K2 . Now, if K2 has repeated singular values, then ðI + DÞ2 has
repeated eigenvalues at the same locations. Therefore, Moreover, the maximum is achieved if and only if there exist
W > ¼ U X P ¼ P up to a block-diagonal transformation, where each orthogonal matrices U W and V W such that
block is an orthonormal matrix that corresponds to a repeated R ¼ U W RV >W and W 1 KW >2 ¼ U W KV >W : ðB:19Þ
singular value of D. Nonetheless, even though W may not be
unique, the matrix C is always unique and equal to Hence, the optimal solutions are W 1 ¼ U W ¼ I and W 2 ¼ V W ¼ I
up to a unitary transformation that accounts for the sign and rota-
C¼ U C DU >C¼ VW DW V ¼ V P DPV > > > >
tional ambiguities of the singular vectors of R. This means that A
" #
I + 1s K+2 0 > 1 and D have the same singular vectors, i.e., U A ¼ U and V A ¼ V,
1
¼ ½ V1 V2 # ½ V 1 V 2 # ¼ V 1 ðI + K+2 ÞV > ; ðB:11Þ
0 0 s 1 1 and that kD + Ak2F ¼ kUðR + KÞV > k2F ¼ kR + Kk2F . By substituting
this expression for kD + Ak2F into (B.15), we obtain
as claimed.
X 1 sX 2 aX
Finally, the optimal C is such that AC ¼ U 1 ðK1 + 1s K+1 >
1 ÞV 1 and min ð1 + k+2 k þ ðri + ki Þ2 ;
i Þþ ðB:20Þ
A + AC ¼ U 2 K2 V >
2 þ 1
U 1 K 1 V >
1 . This shows (22), because K
i2I
2s 2 i2I i 2 i
s 1 2
!
s X# 1 +2
$
s X k+2 X 2 pffiffiffi pffiffiffi
where I1 ¼ fi : ki > 1= sg and I2 ¼ fi : ki 6 1= sg.
kCk) þ kA + ACk2F ¼ 1 + ki þ i
þ ki ;
2 s 2 i2I s2 It follows from the above equation that the optimal ki can be
i2I 1 i2I 1 2
obtained independently for each ri by minimizing the ith term of
as claimed. h the above summation, which is of the form /ðk; rÞ in (29). The first
order derivative of / is given by
Theorem 3. Let D ¼ U RV > be the SVD of the data matrix D. The opti-
( pffiffiffi
1
@/ sk
+3
k > 1= s
mal solutions to P3 are of the form ¼ aðk + rÞ þ pffiffiffi : ðB:21Þ
@k sk k 6 1= s
>
A ¼ U KV and C ¼ VP s ðKÞV ; >
ðB:12Þ
Therefore, the optimal k’s can be obtained as the solutions of the
where each entry of K ¼ diagðk1 ; . . . ; kn Þ is obtained from each entry of nonlinear equation r ¼ wðkÞ in (28) that minimize (29). This com-
R ¼ diagðr1 ; . . . ; rn Þ as the solutions to pletes the proof of Theorem 3. h
( pffiffiffi
1 +3
: k þ as k if k > 1= s
r ¼ wðkÞ¼ pffiffiffi ; ðB:13Þ
k þ as k if k 6 1= s Theorem 4. There exists a r) > 0 such that the solutions to (28) that
minimize (29) can be computed as
that minimize !
( pffiffiffi : b1 ðrÞ if r 6 r)
1+ 1 +2
k > 1= s k ¼ P a;s ðrÞ¼ ðB:22Þ
: a k b3 ðrÞ if r > r) ;
/ðk; rÞ¼ ðr + kÞ2 þ 2s
pffiffiffi : ðB:14Þ
2 s k2 k 6 1= s :
2 where b1 ðrÞ ¼ aþs r
a and b3 ðrÞ is the real root of the polynomial
j.patrec.2013.08.006
1 Now, notice from Fig. 1(c) that when r < h1 the optimal solution is
pðkÞ ¼ k4 + rk3 þ ¼0 ðB:23Þ qffiffiffiffi
as k ¼ b1 . When r ¼ h1 , b1 ¼ 3ða4þa sÞ 4 a3s is a minimum and
that minimizes (29). If 3s 6 a, the solution for k is unique and qffiffiffiffi
# $ b2 ¼ b3 ¼ 4 a3s is an inflection point, thus the optimal solution is
1 aþs
r) ¼ w pffiffiffi ¼ pffiffiffi : ðB:24Þ k ¼ b1 .1 When r > h3 , the optimal solution is b3 . Finally, when
s a s r ¼ h3 ; b1 ¼ b2 ¼ p1ffisffi is a maximum and b3 is a minimum, thus the
If 3s > a, the solution for k is unique, except when r satisfies optimal solution is k ¼ b3 . Therefore, the threshold for r must lie
/ðb1 ðrÞ; rÞ ¼ /ðb3 ðrÞ; rÞ; ðB:25Þ in the range
rffiffiffiffiffi
and r) must lie in the range 4 4 3 aþs
rffiffiffiffiffi < r) < pffiffiffi : ! ðB:32Þ
4 4 3 aþs 3 as a s
< r) < pffiffiffi : ðB:26Þ
3 as a s
References
Proof. When 3s 6 a, the solution to r ¼ wðkÞ that minimizes
/ðk; rÞ is unique, as illustrated in Fig. 1(b). This is because Agarwal, P., Mustafa, N., 2004. k-Means projective clustering. In: ACM Symposium
on Principles of database systems.
( pffiffiffi Basri, R., Jacobs, D., 2003. Lambertian reflection and linear subspaces. IEEE
@2/ a + 3s k+4 k > 1= s Transactions on Pattern Analysis and Machine Intelligence 25 (3), 218–233.
¼ pffiffiffi ðB:27Þ
@k2 aþs k 6 1= s Boult, T., Brown, L., 1991. Factorization-based segmentation of motions. In: IEEE
Workshop on Motion Understanding, pp. 179–186.
Bradley, P.S., Mangasarian, O.L., 2000. k-plane clustering. Journal of Global
is strictly positive, hence / is a strictly convex function of k. When
Optimization 16 (1), 23–32.
k 6 p1ffisffi, the unique solution to r ¼ wðkÞ is given by Cai, J.-F., Candés, E.J., Shen, Z., 2008. A singular value thresholding algorithm for
matrix completion. SIAM Journal of Optimization 20 (4), 1956–1982.
: a
k ¼ b1 ðrÞ¼ r: ðB:28Þ Candès, E., Li, X., Ma, Y., Wright, J., 2011. Robust principal component analysis?
aþs Journal of the ACM 58 (3).
þs Chen, G., Lerman, G., 2009. Spectral curvature clustering (SCC). International Journal
From this equation we immediately conclude that r) ¼ aap ffiffi.
s of Computer Vision 81 (3), 317–330.
Now, if k > p1ffisffi, then k must be one of the real solutions of the poly- Costeira, J., Kanade, T., 1998. A multibody factorization method for independently
moving objects. International Journal of Computer Vision 29 (3), 159–179.
nomial in (B.23). Since 3s 6 a, this polynomial has a unique real Elhamifar, E., Vidal, R., 2009. Sparse subspace clustering. In: IEEE Conference on
root with multiplicity 2, which we denote as b3 ðq rÞ.
ffiffiffiffi Computer Vision and Pattern Recognition.
:
When 3s > a, we have k ¼ b1 ðrÞ if r < h1 ¼ 43 4 a3s and k ¼ b3 ðrÞ Elhamifar, E., Vidal, R., 2010. Clustering disjoint subspaces via sparse
: þsffiffi representation. In: IEEE International Conference on Acoustics, Speech, and
if r > h3 ¼ aaps, as illustrated in Fig. 1(c). However, if h1 6 r 6 h3 Signal Processing.
there could be up to three solutions for k. The first candidate Elhamifar, E., Vidal, R., 2013. Sparse subspace clustering: Algorithm, theory, and
applications. IEEE Transactions on Pattern Analysis and Machine Intelligence.
solution is b1 ðrÞ. The remaining two candidate solutions b2 ðrÞ and
Favaro, P., Vidal, R., Ravichandran, A., 2011. A closed form solution to robust
b3 ðrÞ are the two real roots of the polynomial in (B.23), with b2 ðrÞ subspace estimation and clustering. In: IEEE Conference on Computer Vision
being the smallest and b3 ðrÞ being the largest root. The other two and Pattern Recognition.
Gear, C.W., 1998. Multibody grouping from motion images. Int. Journal of Computer
roots of p are complex. Out of the three candidate solutions, b1 ðrÞ
Vision 29 (2), 133–150.
and b3 ðrÞ correspond to a minimum and b2 ðrÞ corresponds to a Goh, A., Vidal, R., 2007. Segmenting motions of different types by unsupervised
maximum, because manifold clustering. In: IEEE Conference on Computer Vision and Pattern
rffiffiffiffiffiffi rffiffiffiffiffi Recognition.
pffiffiffi 4 3 4 3 Gruber, A., Weiss, Y., 2004. Multibody factorization with uncertainty and missing
b1 ðrÞ 6 1= s; b2 ðrÞ < and b3 ðrÞ > ; ðB:29Þ data using the EM algorithm. In: IEEE Conf. on Computer Vision and Pattern
as as Recognition, vol. I, pp. 707–714.
2
Ho, J., Yang, M.H., Lim, J., Lee, K., Kriegman, D., 2003. Clustering appearances of
and so @@k/2 is positive for b1 , negative for b2 and positive for b3 .
objects under varying illumination conditions. In: IEEE Conference on Computer
Therefore, except when r is such that (B.25) holds true, the solution Vision and Pattern Recognition.
to r ¼ wðkÞ that minimizes (29) is unique and equal to Hong, W., Wright, J., Huang, K., Ma, Y., 2006. Multi-scale hybrid linear models for lossy
! image representation. IEEE Trans. on Image Processing 15 (12), 3655–3671.
b1 ðrÞ if /ðb1 ðrÞ; rÞ < /ðb3 ðrÞ; rÞ Lauer, F., Schnörr, C., 2009. Spectral clustering of linear subspaces for motion
k¼ ðB:30Þ segmentation. In: IEEE International Conference on Computer Vision.
b3 ðrÞ if /ðb1 ðrÞ; rÞ > /ðb3 ðrÞ; rÞ: Lee, K.-C., Ho, J., Kriegman, D., 2005. Acquiring linear subspaces for face recognition
under variable lighting. IEEE Transactions on Pattern Analysis and Machine
To show that k can be obtained as in (B.22), we need show that Intelligence 27 (5), 684–698.
there exists a r) > 0 such that /ðb1 ðrÞ; rÞ < /ðb3 ðrÞ; rÞ for r < r) Lin, Z., Chen, M., Wu, L., Ma, Y., 2011. The augmented Lagrange multiplier method
and /ðb1 ðrÞ; rÞ > /ðb3 ðrÞ; rÞ for r > r) . Because of the intermedi- for exact recovery of corrupted low-rank matrices. arXiv:1009.5055v2.
Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y., 2011. Robust recovery of subspace
ate value theorem, it is sufficient to show that
structures by low-rank representation. In: http://arxiv.org/pdf/1010.2955v1.
f ðrÞ ¼ /ðb1 ðrÞ; rÞ + /ðb3 ðrÞ; rÞ ðB:31Þ

1
One can also show that /ðb1 ðr1 Þ; r1 Þ < /ðb3 ðr1 Þ; r1 Þ as follows:
is continuous and increasing for r 2 ½h1 ; h3 #, negative at h1 and posi- rffiffiffiffiffi rffiffiffiffiffi
tive at h3 , so that there is a r) 2 ðh1 ; h3 Þ such that f ðr) Þ ¼ 0. The 1 16 3 as 8s a
/ðb1 ðr1 Þ; r1 Þ ¼ ¼
function f is continuous in ½h1 ; h3 #, because (a) / is a continuous 2 a þ s 9 as 3ða þ sÞ 3s
1 a 1 a 2
function of ðk; rÞ, (b) the roots of a polynomial (b1 and b2 ) vary con- /ðb3 ðr1 Þ; r1 Þ ¼ 1 + b+2 þ ðr + b3 Þ2 ¼ 1 + b+2 þ b
2s 3 2 2s 3 18 3
tinuously as a function of the coefficients (r) and (c) the composi- 9 + asb3 4
1
r
1 a
ffi ffi ffi ffi ffi
tion of two continuous functions is continuous. Also, f is ¼1+ ¼1+ ¼1+ :
18sb23 3sb23 3 3s
increasing in ½h1 ; h3 #, because
) ) ) ) Therefore, /ðb1 ðr1 Þ; r1 Þ < /ðb3 ðr1 Þ; r1 Þ because
df @/) db1 @/)) @/) db3 @/)) # $rffiffiffiffiffi # $2
¼ )) þ ) + )) + ) 1 8s a 9s þ a a 3
dr @k ðb1 ;rÞ dr @ r ðb1 ;rÞ @k ðb3 ;rÞ dr @ r ðb3 ;rÞ 3 aþs
þ1
3s
< 1 ()
aþs 27s
< 1 () ð3s + aÞ > 0;
¼ 0 þ aðr + b1 Þ + 0 + aðr + b3 Þ ¼ aðb3 + b1 Þ > 0: which follows from the fact that 3s > a.
j.patrec.2013.08.006
Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y., 2012. Robust recovery of subspace Tseng, P., 2000. Nearest q-flat to m points. Journal of Optimization Theory and
structures by low-rank representation. IEEE Transactions on Pattern Analysis Applications 105 (1), 249–252.
and Machine Intelligence. Tseng, P., 2001. Convergence of a block coordinate descent method for
Liu, G., Lin, Z., Yu, Y., 2010. Robust subspace segmentation by low-rank nondifferentiable minimization. Journal of Optimization Theory and
representation. In: International Conference on Machine Learning. Applications 109 (3), 475–494.
Lu, L., Vidal, R., 2006. Combined central and subspace clustering on computer vision Vidal, R., 2011. Subspace clustering. IEEE Signal Processing Magazine 28 (3), 52–68.
applications. In: International Conference on Machine Learning, pp. 593–600. Vidal, R., Ma, Y., Piazzi, J., 2004. A new GPCA algorithm for clustering subspaces by
Ma, Y., Derksen, H., Hong, W., Wright, J., 2007. Segmentation of multivariate mixed fitting, differentiating and dividing polynomials. In: IEEE Conference on
data via lossy coding and compression. IEEE Transactions on Pattern Analysis Computer Vision and Pattern Recognition, vol. I, pp. 510–517.
and Machine Intelligence 29 (9), 1546–1562. Vidal, R., Ma, Y., Sastry, S., 2003a. Generalized Principal Component Analysis
Mirsky, L., 1975. A trace inequality of John von Neumann. Monatshefte für (GPCA). In: IEEE Conference on Computer Vision and Pattern Recognition, vol. I,
Mathematic 79, 303–306. pp. 621–628.
Parikh, N., Boyd, S., 2013. Proximal Algorithms. Now. Vidal, R., Ma, Y., Sastry, S., 2005. Generalized Principal Component Analysis (GPCA).
Rao, S., Tron, R., Ma, Y., Vidal, R., 2008. Motion segmentation via robust subspace IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (12), 1–15.
separation in the presence of outlying, incomplete, or corrupted trajectories. In: Vidal, R., Soatto, S., Ma, Y., Sastry, S., 2003b. An algebraic geometric approach to the
IEEE Conference on Computer Vision and Pattern Recognition. identification of a class of linear hybrid systems. In: Conference on Decision and
Rao, S., Tron, R., Vidal, R., Ma, Y., 2010. Motion segmentation in the presence of Control, pp. 167–172.
outlying, incomplete, or corrupted trajectories. IEEE Transactions on Pattern Vidal, R., Tron, R., Hartley, R., 2008. Multiframe motion segmentation with missing
Analysis and Machine Intelligence 32 (10), 1832–1845. data using PowerFactorization and GPCA. International Journal of Computer
Recht, B., Fazel, M., Parrilo, P., 2010. Guaranteed minimum-rank solutions of linear Vision 79 (1), 85–105.
matrix equations via nuclear norm minimization. SIAM Review 52 (3), 471–501. von Luxburg, U., 2007. A tutorial on spectral clustering. Statistics and Computing 17.
Soltanolkotabi, M., Elhamifar, E., Candes, E., 2013. Robust subspace clustering. Yan, J., Pollefeys, M., 2006. A general framework for motion segmentation:
http://arxiv.org/abs/1301.2603. Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In:
Sugaya, Y., Kanatani, K., 2004. Geometric structure of degeneracy for multi-body European Conf. on Computer Vision, pp. 94–106.
motion segmentation. In: Workshop on Statistical Methods in Video Processing. Yang, A., Wright, J., Ma, Y., Sastry, S., 2008. Unsupervised segmentation of natural
Tipping, M., Bishop, C., 1999. Mixtures of probabilistic principal component images via lossy data compression. Computer Vision and Image Understanding
analyzers. Neural Computation 11 (2), 443–482. 110 (2), 212–225.
Tomasi, C., Kanade, T., 1992. Shape and motion from image streams under Yang, A.Y., Rao, S., Ma, Y., 2006. Robust statistical estimation and segmentation of
orthography: A factorization method. International Journal of Computer multiple subspaces. In: Workshop on 25 years of RANSAC.
Vision 9, 137–154. Zhang, T., Szlam, A., Lerman, G., 2009. Median k-flats for hybrid linear modeling
Tron, R., Vidal, R., 2007. A benchmark for the comparison of 3-D motion with many outliers. In: Workshop on Subspace Methods.
segmentation algorithms. In: IEEE Conference on Computer Vision and Zhang, T., Szlam, A., Wang, Y., Lerman, G., 2010. Hybrid linear modeling via local
Pattern Recognition. best-fit flats. In: IEEE Conference on Computer Vision and Pattern Recognition,
pp. 1927–1934.
j.patrec.2013.08.006

Pattern Recognition Letters: René Vidal, Paolo Favaro

Uploaded by

Document Informationclick to expand document information

Copyright:

Available Formats

Pattern Recognition Letters: René Vidal, Paolo Favaro

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Pattern Recognition Letters: René Vidal, Paolo Favaro

Uploaded by

Copyright:

Available Formats

Pattern Recognition Letters xxx (2013) xxx–xxx

Contents lists available at ScienceDirect

Pattern Recognition Letters

Low rank subspace clustering (LRSC)

1. Introduction as subspace clustering, finds numerous applications in computer vi-

several applications in computer vision (see Vidal (2011) for a re- N

a Y kþ1 ¼ Y k þ ak ðD + Akþ1 + Ekþ1 Þ

a Consider the optimization problem P with uncorrupted data

In this section, we consider the low rank subspace clustering

Fig. 1. Plots of Us , w and P a;s .

Proof. See Appendix B. h

a min kCk) þ a2 kGk2F þ ckEk1

global minimum. Moreover, in practice its convergence can be

In this section we evaluate the performance of LRSC on two

Table 2 6.2. Experiments on face clustering

Method P1 P3 P 5 -ADMM P 5 -IPT P1 P3 P 5 -ADMM P 5 -IPT

Method LSA SCC LRR LRR-H SSC SSC-N LRSC

(b) 4n-dimensional data points obtained by applying PCA to original data

Table 7 Proof. Let Y ¼ kI + Z, where k P kZk2 . Then

Theorem 6. Let A ¼ U KV > be the SVD of A, where the diagonal

and U ¼ ½U 1 U 2 #, K ¼ diagðK1 ; K2 Þ and V ¼ ½V 1 V 2 # are partitioned

Proof. See Mirsky (1975). h s

f ðrÞ ¼ /ðb1 ðrÞ; rÞ + /ðb3 ðrÞ; rÞ ðB:31Þ

You might also like