An Improved Algorithm for Approximating
the Radii of Point Sets ∗
† ‡
Yinyu Ye Jiawei Zhang
April 3, 2003
Abstract
We consider the problem of computing the outer-radii of point sets. In this problem, we
are given integers n, d, k where k ≤ d, and a set P of n points in Rd . The goal is to compute
the outer k-radius of P , denoted by Rk (P ), which is the minimum, over all (d−k)-dimensional
flats F , of maxp∈P d(p, F ), where d(p, F ) is the Euclidean distance between the point p
and flat F . Computing the radii of point sets is a fundamental problem in computational
convexity with significantly many applications. The problem admits a polynomial time
algorithm when the dimension d is constant [9]. Here we are interested in the general case
when the dimension d is not fixed and can be as large as n, where the problem becomes
NP-hard even for k = 1.
It has been known that Rk (P ) can be approximated in polynomial time √ by a factor of
(1 + ε), for any ε > 0, when d − k is a fixed constant [15, 2]. A factor of O( log n) approx-
imation for R1 (P ), the width of the point set P , is implied from the results of Nemirovski
[19] and Nesterov [18]. The first approximation algorithm for general k has been proposed by
Varadarajan, Venkatesh and Zhang [20]. Their algorithm is based on semidefinite program-
ming √ relaxation and the Johnson-Lindenstrauss lemma, and it has a performance guarantee
of O( log n · log d). √
In this paper, we show that Rk (P ) can be approximated by a factor√ of O( log n) for
any 1 ≤ k ≤ d and √ thereby improve the ratio of [20] by a factor of O( log d) that could
be as large as O( log n). This ratio also matches the previously best known ratio for
approximating the special case R1 (P ), the width of point set P . Our algorithm is based
on semidefinite programming relaxation with a new mixed deterministic and randomized
rounding procedure.
1 Introduction
Computing the outer k-radius of a point set is a fundamental problem in computational convexity
with applications in global optimization, data mining, statistics, and clustering and has received
considerable attention in the computational geometry literature [12, 13, 15]. In this problem,
∗
Research supported in part by NSF grant DMI-0231600.
†
Management Science and Engineering and, by courtesy, Electrical Engineering, Stanford University, Stanford,
CA 94305. E-mail: [email protected].
‡
Management Science and Engineering, Stanford University, Stanford, CA 94305. E-mail:
[email protected].
1
we are given integers n, d, k where k ≤ d, and a set P of n points in Rd . The goal is to compute
the outer k-radius of P , denoted by Rk (P ), which is the minimum, over all (d − k)-dimensional
flats F , of maxp∈P d(p, F ), where d(p, F ) is the Euclidean distance between the point p and flat
F . A (d − k)-flat is simply an affine subspace of Rn of dimension k. (See Section 2 for a more
precise definition of Rk (P )). Roughly speaking, the outer k-radius Rk (P ) measures how well
the point set P can be approximated by an affine subspace of dimension d − k. A few special
cases of Rk (P ) which received particular attentions includes: R1 (P ), the width of P ; Rd (P ), the
radius of the minimum enclosing ball of P ; and Rd−1 (P ), the radius of the minimum enclosing
cylinder of P .
When the dimension d is a fixed constant, Rk (P ) can be computed exactly in polynomial
time [9]. It is also known that Rk (P ) can be approximated by a factor of (1 + ε), for any ε > 0,
in O(n + ( 1ε )O(dk) ) time [1, 14]. In this paper, we are interested in the general scenario when the
dimension d is not fixed and can be as large as n,
When the dimension d is a part of the input, the complexity of computing Rk (P ) depends
on whether d − k is constant or not. It is well-known that the problem is polynomial time
solvable when d − k = 0, i.e., the minimum enclosing ball of a set of points can be computed in
polynomial time (Gritzmann and Klee [12]). To the best of our knowledge, whether the problem
is NP-hard or not is still open when d − k = 1. However, Badoiu et al. [2] show that Rd−1 (P )
can be approximated in polynomial time by a factor of (1 + ε), for any ε > 0. Har-Peled and
Varadarajan [15, 16] generalize this result and show that Rk (P ) can be approximated by a factor
of (1 + ε) for any ε > 0 when d − k is constant.
More hardness results are known when d − k becomes large or k becomes small. Bodlaender
et al. [4] show that the problem is NP-hard when k = 1. This is true even for the case n = 2d
( [12]). Gritzmann and Klee [12] also show that it is NP-hard to compute Rk (P ) if k ≤ c · d,
for any fixed 0 < c < 1. These negative results are further improved by Brieden et al. [5] and
Brieden [8], latter of which has shown that it is NP-hard to approximate R1 (P ), the width of a
point set, to within any constant factor. Furthermore, Varadarajan, Venkatesh and Zhang [20]
show that there exists some constant δ > 0 such that for any 0 < ε < 1, there is no quasi-
polynomial time algorithm that approximates Rk (P ) within (log n)δ for k ≤ d − dε unless NP
O(1)
⊆ DTIME [2(log m) ].
On the positive side, the algorithms of Nesterov [19] and Nemirovski et al. √ [18] imply that
R1 (P ), the width of point set P , can be approximated within a factor of O( log n). Another
algorithm for approximating the width of a p point set is given by Brieden et al. [6, 7] and
their algorithm has a performance guarantee d/ log d that is measured in the dimension d.
The first approximation algorithm for general k have been proposed by Varadarajan et al [20].
Their algorithm is based on semidefinite programming
√ relaxation and the Johnson-Lindenstrauss
lemma, and has a performance factor of O( log n · log d).
Above mentioned results on computing Rk (P ) would give us a projection that the problem is
harder when d−k becomes larger or k becomes smaller. However, the result of Varadarajan et al
[20] does not confirm
√ this trend, since we have already known that R1 (P ) can √ be approximated
by a factor of O(√ log n) while, for general k, the ratio proved in [20] is O( log n√· log d), which
is larger than O( log n). Therefore, they have conjectured that the factor of O( log n) applies
to general k as well.
The main result of the present paper is to show that Rk (P ) can indeed be approximated
2
√
by the factor of O( log n) for all 1 ≤ k ≤ d, and therefore√proves their conjecture. Note
that the
√ new approximation ratio is reduced by a factor of O( log d), which could be as large
as O( log n). Our algorithm is based on semidefinite programming relaxation with a mixed
deterministic and randomized rounding procedure, in contrast to all other purely randomized
rounding procedures used for semidefinite programming approximation.
2 Preliminaries and SDP relaxation
Generally speaking, the problem of computing Rk (P ) can be formulated as a quadratic minimiza-
tion problem. Semidfinite programming (SDP) problems, where the unknowns are represented
by positive semidefinite matrices, have recently been developed for approximating such prob-
lems; see, for example, Goemans and Williamson [10]. In the case of k = 1, computing R1 (P )
corresponds to a SDP problem plus an additional requirement that the rank of the unknown
matrix equals 1. Removing the rank requirement, the SDP problem becomes a relaxation of the
original problem and polynomially solvable for any given accuracy. Once obtaining an optimal
solution, say X, of the SDP relaxation, one would like to generate a rank-1 matrix, say X̂ = yy T ,
from X, where y is a column vector and serves as an solution to the original problem. Such
rank reduction is called “rounding”, and many procedures are proposed and almost all of them
are randomized, see, for example, [3].
One particular procedure has been proposed by Nemirovski et al. [18] which can be used for
approximating R1 (P ). Their procedure is a simple randomized rounding that can described as
follows: an optimal solution X of the SDP relaxation, whose rank could be as large as d, can be
represented as (e.g., by eigenvector decomposition)
X = λ1 v1 v1T + λ2 v2 v2T + · · · λd vd vdT .
Then one can generate a single vector y by take a random combination of the vectors v1 , v2 , · · · , vd
where the coefficients of the combination takes values of −1 or 1 uniformly and independently.
When k ≥ 2, one needs to generate k rank-1 matrices from X, the optimal solution of the
corresponding SDP relaxation, such that
k
X
X̂ = yi yiT
i=1
where yi s are orthogonal to each other. The method of Varadarajan et al [20] first applies the
Johnson-Lindenstrauss randomized dimension reduction technique [17] to reduce the rank of
solution X to k + O(log n · log d) without losing much in the quality of the solution in terms
of the objective value. Then they show that among the k + O(log n · log d) vectors, which are
orthogonal
√ to each other, k vectors can be randomly chosen as the solution with an approximate
ratio O( log n · log d).
Our algorithm is based on the same SDP relaxation developed in [20]. However, our rounding
procedure is different. Our procedure works as follows. Once obtaining an optimal solution for
the SDP relaxation with
X = λ1 v1 v1T + λ2 v2 v2T + · · · λd vd vdT ,
3
we deterministically partition the vectors v1 , v2 , · · · , vd into k groups where group j may contain
nj vectors and each group can be seen as a single semidefinite matrix with rank nj . We can then
generate one vector from each group using the randomized rounding procedure similar to that
of Nemirovski et al. [18]. The k vectors generated by this rounding procedure will automatically
satisfy the condition that any pair of them must be orthogonal √ to each other, and the quality
of these vectors have an approximation ratio no more than O( log n).
We now present the quadratic program formulation of the k-radii problem and its semidefinite
programming relaxation. It will be helpful to first introduce some notations that will be used
later. The trace of a given matrix A, denoted by Tr(A), is the sum of the entries on the
main diagonal of A. We use I to denote the identity matrix whose dimension will be clear
in the context. The inner product of two vector p p and q is denoted by hp, qi. The 2-norm
of a vector x, denoted by kxk, is defined by hx, xi. A positive semidefinite matrix X are
represented by X º 0. For simplicity, we assume that the P is symmetric in the sense that if
p ∈ P then −p ∈ P . Denote the set {−p|p ∈ P } by −P . Let Q = P ∪ −P . It is clear that
Rk (P ) ≤ Rk (Q) ≤ 2Rk (P ). Therefore, if we found a good approximation for Rk (Q) then it
must also be a good approximation for Rk (P ).
Thus, the square of Rk (P ) can be defined by the optimal value of the following quadratic
minimization problem:
Rk (P )2 := Minimize α
k
X
Subject to hp, xi i2 ≤ α, ∀p ∈ P,
i=1
kxi k2 = 1, i = 1, ..., k,
hxi , xj i = 0, ∀i 6= j.
Assume that x1 , x2 , · · · , xk ∈ Rd is the optimal solution of (1). Then one can easily verify that
the matrix X = x1 xT1 + x2 xT2 + · · · xk xTk is a feasible solution for the following semidefinite
program:
αk∗ := Minimize α
Subject to Tr(ppT X) (= pT Xp) ≤ α, ∀p ∈ P,
Tr(X) = k,
I − X º 0, X º 0.
It follows that αk∗ ≤ Rk (P )2 . The following Lemma, which is proved in Varadarajan et al [20],
follows from the above observations. We reproduce the proof below for completeness.
Lemma 1. There exists an integer r ≥ k such that we can compute, in polynomial time, r
nonnegative reals λ1 , λ2 , · · · , λr and r orthorgonal unit vectors v1 , v2 , · · · , vr such that
Pr
(i). i=1 λi = k.
(ii). max1≤i≤r λi ≤ 1.
Pr 2 2
(iii). i=1 λi hp, vi i ≤ Rk (P ) , for any p ∈ P .
4
Proof. We solve the semidefinte program (1), and let X ∗ be an optimal solution of (1). We
claim that the rank of X ∗ , say r, is at least k. This follows from the fact that Tr(X ∗ ) = k and
I −X ∗ º 0. In other words, Tr(X ∗ ) = k implies that the sum of the eigenvalues of X ∗ is equal to
k, and I − X ∗ º 0 implies that the all eigenvalues are less than or equal to 1. Therefore, X ∗ has
at least k non-zero eigenvalues, which implies that the rank of X ∗ is at least k. Let λ1 , λ2 , · · · , λr
be thePr nonnegative eigenvalues and v1 , v2 , · · · , vr be the corresponding eigenvectors. Then we
have ri=1 λi = k and max1≤i≤r λi ≤ 1. Furthermore, for any p ∈ P ,
r
X r
X r
X
2
λi hp, vi i = λi Tr(pp T
vi viT ) = Tr(pp T
λi vi viT ) = Tr(ppT X ∗ ) ≤ αk∗ ≤ Rk (P )2 .
i=1 i=1 i=1
3 Deterministic First Rounding
In this section, we prove a lemma concerning how to deterministically group the eigenvalues and
their eigenvectors. This lemma is simple but play a key role for proving our main result.
Lemma 2. The index set {1, 2, · · · , r} can be partitioned into k sets I1 , I2 , · · · , Ik such that
(i). ∪ki=1 Ii = {1, 2, · · · , r}, and for any i 6= j, Ii ∩ Ij = ∅.
(ii). For any i : 1 ≤ i ≤ k, j∈Ii λj ≥ 12 .
P
Proof. Recall that rj=1 λj = 1 and λj ≤ 1 for all j. Without loss of generality, we can assume
P
that λ1 ≥ λ2 ≥ · · · ≥ λr . Our partitioning algorithm is the same as the Longest-Processing-Time
heuristic algorithm for parallel machine scheduling problem. The algorithm works as follows:
STEP 1. For i = 1, 2, · · · , k, set Ii = ∅ and let Li = 0. Let I = {1, 2, · · · , r}.
STPE 2. While I 6= ∅
choose j form I with the smallest index;
choose set i with the samllest value Li ;
Let Ii := Ii ∪ {j}, Li := Li + λj and I := I − {j}.
It is clear that when the algorithm stops, the sets I1 , I2 , · · · , Ik satisfy condition (i). Now
we prove condition (2) by contradiction. Assume that there exists some t such that
X 1
λj < .
2
j∈It
We now claim that, for all i, X
λj ≤ 1.
j∈Ii
Otherwise, suppose j∈I ′ λj > 1 for some t′ . Note that λj ≤ 1 for every j and thus there are
P
t
at least two eigenvalues are assigned to It′ . Denote the last eigenvalues by λs′ . It follows that
X X X
λj − λs′ = λj ≤ λj
j∈It′ j∈It′ \{s′ } j∈It
5
since, otherwise, we would have not assigned λs′ to It′ in the algorithm. However, since
P 1
j∈It λj < 2 , we must have
X X 1
λj − λs′ = λj < .
2
j∈It′ j∈It′ \{s′ }
Thus,
X 1 1
λs′ > λj − > .
2 2
j∈It′
This is impossible since λs′ is the last eigenvalues assigned to It′ , which implies λs′ ≤ λj for
every j ∈ It′ , and we have already proved that there must exist an l such that s′ 6= l ∈ It′ and
X 1
λl ≤ λj < .
2
j∈It′ \{s′ }
λj < 12 . It follows that
P P
Therefore, j∈Ii λj ≤ 1 for all i, and in particular j∈Is
k X
X
λj < k.
i=1 j∈Ij
However, we know that, by condition (i),
k X
X r
X
λj = λj = k.
i=1 j∈Ij j=1
This results a contradiction. Therefore, such t does not exists and we have proved condition
(ii).
Notice that the running time of the partitioning algorithm is bounded by O(r · k).
4 Randomized Second Rounding
Assume now that we have found I1 , I2 , · · · , Ik . Then our next randomized rounding procedure
works as follows.
STEP 1. Generate a r dimensional random vector φ such that each entry of φ takes value,
independently, −1 or 1 with probability 21 each way.
STEP 2. For i = 1, 2, · · · , k, let
P p
j∈Ii φj λj · vj
xi = qP .
λ
j∈Ii j
The following Lemmas show that x1 , x2 , · · · , xk form a feasible solution for the original
problem. In other words, they are k orthogonal unit vectors.
6
Lemma 3. For i = 1, 2, · · · , k, kxi k = 1 with probability 1.
Proof. Recall that hvl , vj i = 0 for any l 6= j and kvj k = 1. By definition,
kxi k2 = hxi , xi i
*P p P p +
j∈Ii φj λj vj j∈Ii φj λj vj
= qP , qP
j∈Ii λ j j∈Ii λj
* +
1 X p X p
= P φj λj vj , φj λj vj
j∈Ii λj j∈Ii j∈Ii
1 X p p
= P hφj λj vj , φj λj vj i
j∈Ii λj
j∈Ii
1 X
λj vj k2
p
= P kφj
j∈Ii λj
j∈Ii
1 X
= P (φj )2 λj kvj k2
j∈Ii λj j∈Ii
1 X
= P λj
j∈Ii λj
j∈Ii
= 1
Lemma 4. If s 6= t then hxs , xt i = 0 with probability 1.
Proof. The proof is similar as that of Lemma 3.
hxs , xt i
*P p P p +
φj
j∈Isλj vj j∈It φj λj vj
= qP , qP
j∈Is λj j∈It λj
* +
1 X p X p
= qP P φj λj vj , φj λj vj
λ
j∈Is j · λ
j∈It j j∈Is j∈It
= 0.
The last equality holds since for any j ∈ Is and l ∈ It , hvj , vl i = 0.
Now we establish a bound on the performance of our algorithm. First, let us introduce
Bernstein’s Theorem (see, e.g., [18]).
7
Lemma 5. Let φ be a random vector whose entries are independent and either 1 or −1 with
probability .5 each way. Then, for any vector e and β > 0,
β
prob{hφ, ei2 > βkek2 } < 2exp(− ).
2
λj · hp, vj i2 . Then we have
P
Let Cip = j∈Ii
Lemma 6. For each i = 1, 2, · · · , k and each p ∈ P , we have
2
prob{hp, xi i2 > 12 ln(n) · Cip } < .
n3
p
Proof. Given i and p, define a |Ii | dimensional vector e such that its entries are λj hp, vj i, j ∈ Ii ,
respectively. Furthermore, we define the vector φ|Ii whose entries are those of φ with indices in
Ii . First notice that
X p X
kek2 = ( λj hp, vj i)2 = λj · hp, vj i2 = Cip .
j∈Ii j∈Ii
On the other hand,
hp, xi i2
* P p +2
λj vj φj
j∈Ii
= p, qP
j∈Ii λj
* +2
1 Xp
= P p, λj vj φj
j∈Ii λj j∈Ii
* +2
Xp X 1
≤ 2 p, λj vj φj (since λj ≥ )
2
j∈Ii j∈Ii
2
Xp
= 2 λj φj hp, vj i
j∈Ii
= 2 hφ|Ii , e)2
Thus
prob{hp, xi i2 > 12 ln(n)Cip } ≤ prob{hφ|Ii , ei2 > 6 ln(n)kek2 }.
Therefore, the conclusion of the lemma follows by using Lemma 5 and by letting β = 6 ln(n).
Theorem 1. We can computed in polynomial time, a (d − k)-flat such p that, with probability at
least 1 − n2 , the distance between any point p ∈ P and F is at most 12 ln(n) · Rk (P ).
8
Proof. For given i = 1, 2, · · · , k and p ∈ P , consider the event
Bip = {φ|hp, xi i2 > 12 ln(n) · Cip }
and B = ∪i,p Bip . The probability that the event B happens is bounded by
X 2kn 2
prob{hp, xi i2 > 12 ln(n) · Cip } < 3
≤ .
n n
i,p
If B does not happen, then for any i and p,
hp, xi i2 ≤ 12 ln(n) · Cip .
Therefore, for each p ∈ P ,
k
X k
X
hp, xi i2 ≤ 12 ln(n) ·Cip ≤ 12 ln(n) · Rk (P )2 .
i=1 i=1
The last inequality follows from Lemma 1. This completes the proof by taking F as the flat
which is orthogonal to the vectors x1 , x2 , · · · , xk .
5 Final Remark
Finding efficient rounding methods for semidefinite programming relaxation plays a key role
in constructing better approximation algorithms for various hard optimization problems. All
of them developed to date are randomized in nature. Therefore, the mixed deterministic and
randomized rounding procedure developed in this paper may have its own independent value.
We expect its further applications in approximating various computational geometry and space
embedding problems.
References
[1] G. Barequet and S. Har-Peled, “Efficiently Approximating the Minimum-Volume Bound-
ing Box of a Point Set in Three Dimensions,” J. Algorithms, 38:91-109, 2001.
[2] M. Badoiu, S. Har-Peled and P. Indyk, “Approximate Clustering via Core-sets,” In Proc.
ACM Symp. Theory of Computing, 2002.
[3] D. Bertsimas and Y. Ye, “Semidefinite relaxations, multivariate normal distributions, and
order statistics,” Handbook of Combinatorial Optimization (Vol. 3), D.-Z. Du and P.M.
Pardalos (Eds.) pp. 1-19, (1998 Kluwer Academic Publishers).
[4] H.L. Bodlaender, P. Gritzmann, V. Klee and J. Van Leeuwen, “The Computational Com-
plexity of Norm Maximization”, Combinatorica, 10: 203-225, 1990.
9
[5] A. Brieden, P. Gritzmann, and V. Klee, “Inapproximability of Some Geometric and
Quadratic Optimization Problems,” In P.M. Pardalos, editor, Approximation and Com-
plexity in Numerical Optimization: Continuous and Discrete Problems, 96-115, Kluwer,
2000.
[6] A. Brieden, P. Gritzmann, R. Kannan, V. Klee, L. Lovasz and M. Simonovits, “Determinis-
tic and Randomized Polynomial-time Approximation of Radii,” To appear in Mathematika.
[7] A. Brieden, P. Gritzmann, R. Kannan, V. Klee, L. Lovasz and M. Simonovits, “Approx-
imation of Diameters: Randomization Doesn’t Help,” In Proc. IEEE Symp. Foundations
of Comp. Sci., 244-251, 1998.
[8] A. Brieden, “Geometric Optimization Problems Likely Not Contained in APX,” Discrete
Comput. Geom., 28:201-209, 2002.
[9] U. Faigle, W. Kern, and M. Streng, “Note on the Computaional Complexity of j-Radii of
Polytopes in Rn ,” Mathematical Programming, 73:1-5, 1996.
[10] M. X. Goemans and D.P. Williamson, “Improved Approximation Algorithms for Maximum
Cut and Satisfiability Problems using Semi-definite Programming,” Journal of the ACM,
42:1115-1145, 1995.
[11] P. Gritzmann and V. Klee, “Inner and Outer j-Radii of Convex Bodies in Finite-
Dimensional Normed Spaces,” Discrete Comput. Geom., 7:255-280, 1992.
[12] P. Gritzmann and V. Klee, “Computational Complexity of Inner and Outer j-Radii of
Polytopes in Finite-Dimenional Normed Spaces,” Math. Program., 59:162-213, 1993.
[13] P. Gritzmann and V. Klee, “On the Complexity of Some basic Problems in Computational
Convexity: I. Containment Problems,” Discrete Math., 136:129-174, 1994.
[14] S. Har-Peled and K.R. Varadarajan, “Approximate Shape Fitting via Linearization,” In
Proc. 42nd Annu. IEEE Sympos. Found. Comput. Sci., 66-73, 2001.
[15] S. Har-Peled and K. Varadarajan, “Projective Clustering in High Dimensions Using Core-
sets,” In Proc. ACM Symp. Comput. Geom., 2002.
[16] S. Har-Peled and K. Varadarajan, “High-Dimensional Shap Fitting in Linear Time,” In
Proc. ACM Symp. Comput. Geom., 2003.
[17] W.B. Johnson and J. Lindenstrauss, “Extensions of Lipshitz Mapping into Hilbert Space,”
Comtemporary Mathematics, 26:189-206, 1984.
[18] A. Nemirovski, C. Roos and T. Terlaky, “On Maximization of Quadratic Forms Over
Intersection of Ellipsoids with Common Center,” Math. Program., 86:463-473, 1999.
[19] Yu. Nesterov, “Global Quadratic Optimization via Conic Relaxation,” Working Paper,
CORE, Catholic University of Louvaine, Belgium, 1998.
[20] K. Varadarajan, S. Venkatesh and J. Zhang, “On Approximating the Radii of Point Sets
in High Dimensions,” In Proc. 43rd Annu. IEEE Sympos. Found. Comput. Sci., 2002.
10