Luo Tutorial JSAC
Luo Tutorial JSAC
8, AUGUST 2006
Tutorial Paper
Abstract—Convex optimization methods are widely used in the There have been significant advances in the research in
design and analysis of communication systems and signal pro- convex optimization (e.g., interior-point method [1] and conic
cessing algorithms. This tutorial surveys some of recent progress optimization [2]) over the last two decades. The first part of
in this area. The tutorial contains two parts. The first part
gives a survey of basic concepts and main techniques in convex
this tutorial provides an overview of these developments and
optimization. Special emphasis is placed on a class of conic op- describes the basic optimization concepts, models and tools
timization problems, including second-order cone programming that are most relevant to signal processing and communication
and semidefinite programming. The second half of the survey gives applications. The second half of this paper includes examples
several examples of the application of conic programming to com- illustrating Lagrangian duality, the method of relaxation, and
munication problems. We give an interpretation of Lagrangian robust optimization techniques, which are useful in many
duality in a multiuser multi-antenna communication problem; we
illustrate the role of semidefinite relaxation in multiuser detection engineering applications. Open problems and future research
problems; we review methods to formulate robust optimization directions are also discussed.
problems via second-order cone programming techniques.
II. CONVEX OPTIMIZATION
Index Terms—Convex optimization, digital communications,
duality, second-order cone programming (SOCP), semidefinite In order to recognize convex optimization problems in
programming (SDP), signal processing. engineering applications, one must first be familiar with the
basic concepts of convexity and the commonly used convex
optimization models. This section provides a concise review
I. INTRODUCTION of these optimization concepts and models including linear
programming, second-order cone programming, and semidef-
Manuscript received May 3, 2006; revised May 8, 2006. The work of A. Basic Optimization Concepts
Z.-Q. Luo was supported in part by the National Science Foundation under Convex Sets: A set is said to be convex if for any
Grant DMS-0312416. The work of W. Yu was supported in part by the
Natural Sciences and Engineering Research Council (NSERC) of Canada two points , , the line segment joining and also lies
and in part by the Canada Research Chairs Program. in . Mathematically, it is defined by the following property:
Z.-Q. Luo is with the Department of Electrical and Computer Engineering,
University of Minnesota, Minneapolis, MN 55455 USA (e-mail: luozq@ece.
umn.edu).
W. Yu is with The Edward S. Rogers Sr. Department Electrical and Computer
Engineering, University of Toronto, Toronto, ON M5S 3G4, Canada (e-mail:
[email protected]). Many well-known sets are convex, for example, the unit ball
Digital Object Identifier 10.1109/JSAC.2006.879347 . However, the unit sphere
0733-8716/$20.00 © 2006 IEEE
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
LUO AND YU: AN INTRODUCTION TO CONVEX OPTIMIZATION FOR COMMUNICATIONS AND SIGNAL PROCESSING 1427
is not convex since the line segment joining any two distinct and always dominates the function . There are many
points is no longer on the unit sphere. In general, a convex set examples of convex functions, including the commonly seen
must be a solid body, containing no holes, and always curve out- univariate functions , , , as well as multivariate func-
ward. Other examples of convex sets include ellipsoids, hyper- tions , , where , , and are given data ma-
cubes, polyhedral sets, and so on. In the real line , convex sets trix/vector/constant. We say is concave if is convex. The
correspond to intervals (open or closed). The most important entropy function is a concave function over .
property about convex set is the fact that the intersection of any If is continuously differentiable, then the convexity of is
number (possibly uncountable) of convex sets remains convex. equivalent to
For example, the set is the intersec-
tion of the unit ball with the nonnegative orthant , both of
which are convex. Thus, their intersection is also convex. The
union of two convex sets is typically nonconvex. In other words, the first-order Taylor series expansion serves as
Convex Cones: A convex cone is a special type of convex a global underestimator of . Furthermore, if is twice con-
set which is closed under positive scaling: for each and tinuously differentiable, then the convexity of is equivalent
each , . Convex cones arise in various forms in to the positive semidefiniteness of its Hessian: ,
engineering applications. The most common convex cones are . Thus, a linear function is always convex, while a
the following. quadratic function is convex if and only if
1) Nonnegative orthant . . Notice that the linear plus the constant term
2) Second-order cone (also known as ice-cream cone) does not have any bearing on the convexity (or the lack of) of .
One can think of numerous examples of functions which are nei-
ther convex nor concave. For instance, the function is convex
over and concave over the region , but is neither
convex nor concave over .
In practice, sometimes it is more convenient to The most important property about convex functions is the
work with the so-called rotated second-order cone: fact that they are closed under summation, positive scaling, and
. This cone is the pointwise maximum operations. In particular, if the ’s
equivalent to the standard via a simple linear are convex, then so is (even though it is typically
transformation. nondifferentiable). A notable connection between convex set
3) Positive semidefinite matrix cone and convex function is the fact that the level sets of any convex
function are always convex, i.e., is convex
for any . The converse is not true, however. For example,
the function is nonconvex, but its level sets are
convex.
For any convex cone , we can define its dual cone Convex Optimization Problems: Consider a generic opti-
mization problem (in the minimization form)
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
1428 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 24, NO. 8, AUGUST 2006
For example, the following entropy maximization problem is B. Lagrangian Duality and Karush–Kuhn–Tucker Condition
convex: Consider the following (not necessarily convex) optimization
problem:
(3)
Let denote the global minimum value of (3). For symmetry
reason, we will call (3) the primal optimization problem, and
where the linear equalities may represent the usual call the primal vector. Introducing dual variables and
moment matching constraints. , we can form the Lagrangian function
Let us now put in perspective the role of convexity in opti-
mization. It is well known that, for the problem of solving a
system of equations, linearity is the dividing line between the
“easy” and “difficult” problems.1 Once a problem is formulated The so-called dual function associated with (3) is de-
as a solution to a system of linear equations, the problem is con- fined as
sidered done since we can simply solve it either analytically or
using existing numerical softwares. In fact, there are many ef-
ficient and reliable softwares available for solving systems of
linear equations, but none for nonlinear equations. The lack of Notice that, as a pointwise minimum of a family of linear func-
high-quality softwares for solving nonlinear equations is merely tions (in ), the dual function is always concave.
a reflection of the fact that they are intrinsically difficult to solve. We will say is dual feasible if and is finite.
In contrast, the dividing line between the “easy” and “diffi- The well-known weak duality result says the following.
cult” problems in optimization is no longer linearity, but rather Proposition 1: For any primal feasible vector and any dual
convexity. Convex optimization problems are the largest sub- feasible vector , there holds
class of optimization problems which are efficiently solvable,
whereas nonconvex optimization problems are generally diffi-
cult. The theory, algorithms and software tools for convex opti-
mization problems have advanced significantly over the last 50 In other words, for any dual feasible vector , the dual func-
years. There are now (freely downloadable) high-quality soft-
tion value always serves as a lower bound on the primal
wares which can deliver accurate solutions efficiently and reli-
objective value . Notice that and are chosen in-
ably without the usual headaches of initialization, step-size se-
dependent from each other (so long as they are both feasible).
lection or the risk of getting trapped in a local minimum. Once
Thus, for all dual feasible vector . The
an engineering problem is formulated in a convex manner, it
largest lower bound for can be found by solving the following
is reasonable to consider it “solved” (or “game over”), at least
from a numerical perspective. dual optimization problem:
For any convex optimization problem, the set of global op-
timal solutions is always convex. Moreover, every local optimal
(4)
solution is also a global optimal solution, so there is no danger
of being stuck at a local solution. There are other benefits Notice that the dual problem (4) is always convex regardless of
associated with a convex optimization formulation. For one the convexity of the primal problem (3), since is con-
thing, there exist highly efficient interior-point optimization cave. Let us denote the maximum value of (4) by . Then, we
algorithms whose worst-case complexity (i.e., the total number have . Interestingly, for most convex optimization prob-
of arithmetic operations required to find an -optimal solution) lems (satisfying some mild constraint qualification conditions,
grows gracefully as a polynomial function of the problem data such as the existence of a strict interior point), we actually have
length and . In addition, there exists an extensive duality . This is called strong duality.
theory for convex optimization problems, a consequence of In general, the dual function is difficult to compute.
which is the existence of a computable mathematical certifi- However, for special classes of convex optimization problems
cate for infeasible convex optimization problems. As a result, (see Section II-C), we can derive their duals explicitly. Below
well-designed softwares for solving convex optimization prob- is a simple example illustrating the concept of duality for linear
lems typically return either an optimal solution, or a certificate programming.
(in the form of a dual vector) that establishes the infeasibility Example: Let us consider the following linear programming
of the problem. The latter property is extremely valuable in problem:
engineering design since it enables us to identify constraints
which are too restrictive.
1These notions can be made precise using the computational complexity
theory; e.g., NP-hardness results. (5)
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
LUO AND YU: AN INTRODUCTION TO CONVEX OPTIMIZATION FOR COMMUNICATIONS AND SIGNAL PROCESSING 1429
The primal optimal solution is unique and equal to it is attained at a KKT point , which, together with some dual
, with . The Lagrangian function is given feasible vector , satisfies the KKT condition (6)–(10).
by , and the dual function Detecting Infeasibility: Efficient detection of infeasibility
is given by is essential in engineering design applications. However, the
problem of detecting and removing the incompatible con-
straints is NP-hard in general, especially if the constraints are
nonconvex. However, for convex constraints, we can make use
of duality theory to prove inconsistency. Let us consider the
following example.
if Example: Determine if the following linear system is
otherwise feasible:
Thus, the dual linear program can be written as
(6)
(12)
(7)
(8) Clearly, the existence of a such serves as a certificate for the
(9) incompatibility of the linear inequalities in (11). What is inter-
and esting (and nontrivial) is the fact that the converse is also true.
That is, if the system (11) is infeasible, then there always exists
a mathematical certificate satisfying (12). Results of this kind
(10) are called the theorems of alternatives, and are related to the
well-known Farkas’ lemma for the linear feasibility problem.
Collectively, the conditions (6)–(10) are called the Karush– The above result can also be extended to the nonlinear con-
Kuhn–Tucker (KKT) condition for optimality. Notice that the text. For instance, consider a system of convex (possibly non-
first two conditions (6) and (7) represent primal feasibility linear) inequality system
of , condition (8) represents dual feasibility, condition (9)
signifies the complementary slackness for the primal and dual
inequality constraint pairs: and , while the (13)
last condition (10) is equivalent to .
For the above linear programming example, we can easily Then, either (13) is feasible or there exists some nonzero
check that the vector and the Lagrangian mul- satisfying
tipliers satisfy the above KKT con-
dition. Moreover, they are the unique solution of (6)–(10). Thus, (14)
is the unique primal optimal solution for (5).
In general, the KKT condition is necessary but not sufficient Exactly one of the above two conditions holds true. The exis-
for optimality. However, for convex optimization problems (and tence of a nonzero satisfying (14) proves the infeasibility
under mild constraint qualification conditions), the KKT condi- of (13). Such a serves as a certificate of infeasibility. Modern
tion is also sufficient. If the constraints in (3) are absent, the cor- softwares (e.g., SeDuMi [3]) for solving convex optimization
responding KKT condition simply reduces to the well-known problems either generate an optimal solution or a certificate
stationarity condition for unconstrained optimization problem: showing infeasibility. In contrast, softwares for nonconvex op-
. That is, an unconstrained local minimum must timization problems cannot detect infeasibility. They typically
be attained at a stationary point (at which the gradient of van- fail to converge when the underlying problem is infeasible, ei-
ishes). However, in the presence of constraints, local optimal so- ther due to data overflow or because the maximum number of
lution of (3) is no longer attained at a stationary point; instead, iterations is exceeded.
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
1430 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 24, NO. 8, AUGUST 2006
C. Linear Conic Optimization Models In practice, linear matrix inequalities of the form
can be easily mapped to the form in (23). The
We now review several commonly used convex optimization
optimality condition is given by
models in engineering design applications. Consider a primal-
dual pair of optimization problems
(15) (24)
and
(17)
and its dual becomes (25)
where is a positive parameter. By driving and under
(18) mild assumptions, the central path converges to an optimal
The optimality condition is given by primal-dual solution pair for (15) and (16). Notice that the cen-
tral path condition (25) is exactly the necessary and sufficient
optimality condition for the following convex problem:
Second-Order Cone Programming :
Let with
(namely, ). Similarly, we denote
with . (26)
The data vector with , and the In other words, the points on the central path corresponds to the
data matrix . In this case, the linear conic optimal solution of (26) and the associated optimal dual solu-
optimization problem (15) reduces to tion. Here, the function is called the barrier func-
tion for the positive semidefinite matrix cone .
Many interior-point algorithms follow (approximately) the
(19) central path to achieve optimality. As a result, the iterates are
and its dual becomes required to remain in a neighborhood of the central path which
can be defined as
(20)
In practice, the second-order cone programming (SOCP) con-
straints usually take the form of , which can be (27)
easily mapped to the form in (19). The optimality condition for
(19) is given by With this definition, a generic interior-point path-following al-
gorithm can be stated as follows.
(21)
Generic Path-Following Algorithm
Semidefinite Programming (SDP) or : In this
case, the linear conic optimization problem reduces to
1) Given a strictly feasible primal-dual pair
with . Set
(22) 2) Let , , and .
and its dual becomes 3) Compute a search direction and
the largest step such that
. Set ,
, .
(23) 4) Set and return to 2) until convergence.
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
LUO AND YU: AN INTRODUCTION TO CONVEX OPTIMIZATION FOR COMMUNICATIONS AND SIGNAL PROCESSING 1431
Fig. 1. Uplink-downlink beamforming duality can be interpreted as a Lagrangian duality in convex optimization.
There are many choices for the search direction illustrate the concepts covered so far. Communication prob-
. For example, we can take it as the solution of lems often involve the optimization of some design objective,
the following linear system of equations: such as transmit power or the detection probability of error,
subject to resource constraints, such as bandwidth and power.
Traditional optimization methods in communications focus on
analytic or adaptive solutions to the problem—as in minimum
mean-squared error (MMSE) equalizer design. The MMSE
(28)
problem is essentially a convex quadratic program (QP). This
section focuses on the application of modern techniques such
where is a nonsingular matrix and
as linear conic programming.
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
1432 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 24, NO. 8, AUGUST 2006
An important class of optimal downlink beamforming problem Because can be assumed to be real, we may take the
involves finding a set of ’s that minimizes the total transmit square root of the above equation. The constraint becomes a
power, while satisfying a given set of SINR constraints second-order cone constraint, which is convex. The optimiza-
tion problem now becomes
(31)
(32)
(35)
For simplicity, we make the assumption that the set of is
feasible.
As stated in (32), the SINR constraint is not convex. How- which is in the form of a second-order cone program (19).
ever, this seemingly nonconvex problem has many of the fea-
tures of convex problems. For example, as shown in [6]–[9],
this downlink problem can be solved in an iterative fashion via B. Uplink-Downlink Duality via Lagrangian Duality
a so-called uplink-downlink duality. Further, it turns out that In engineering design, one is often interested in not only the
the above problem can be relaxed [10] or transformed [11] into numerical solution to the problem, but also the structure of the
a convex optimization problem. This section gives an overview optimal solution. When a problem is formulated as a convex op-
of the approaches in [10] and [11] to illustrate the application of timization problem, exploring its dual often reveals such struc-
convex optimization. In the next section, we take advantage of tures. In this section, we illustrate this point by showing that the
the convex formulation and illustrate an engineering interpreta- Lagrangian dual of the SOCP problem [11] has an engineering
tion of Lagrangian duality in this setting. interpretation, which is known as uplink-downlink [6], [7], [9],
One approach to solving the beamforming problem is based [12], [13].
on a reformulation of (31) in terms of new matrix variables Several different versions of uplink-downlink duality have
[10]. Clearly, is a positive semidefinite matrix. Define been developed in the literature. In the beamforming context,
. The optimization problem (31), in terms of , uplink-downlink duality refers to the fact that the minimum
then becomes the following SDP power needed to achieve a certain set of SINR targets in a
downlink multiple-input–multiple-output (MIMO) channel is
the same as the minimum power needed to achieve the same set
of SINR targets in the uplink channel, where the uplink channel
is derived by reversing the input and the output of the downlink
[6]–[9], [14]. A very similar uplink-downlink duality can be
(33) derived from an information theory point of view. As shown
in [12], [13], [15], and [16], the capacity region of a downlink
However, the original problem (31) requires the optimal solu- broadcast channel is identical to the capacity region of an
tion to be rank-1, which is not guaranteed a priori in the uplink channel with the same sum power constraint. These two
above formulation. In effect, the above formulation relaxes the duality results can be unified via Lagrangian duality in convex
rank-1 constraint. For this reason, (33) is referred to as a SDP optimization, first shown in [17]. The following outlines the
relaxation. main steps.
Surprisingly, as shown by Bengtsson and Ottersten [10], the We begin the development by writing down the Lagrangian
above SDP relaxation is guaranteed to have at least one op- of the downlink beamforming problem (31)2:
timal solution which is rank one. This unexpected property lends
tremendous power of convex optimization to this apparently
nonconvex problem. In Section IV-B, we will return to SDP re-
laxation and consider a closely related multicast problem for
which this property does not hold.
The fundamental reason that the SDP relaxation is optimal
for the above problem is that this problem can be reformulated
as a convex problem. This is shown in [11] using SOCP. Ob-
serve that an arbitrary phase rotation can be added to the beam-
forming vectors without affecting the SINR. Thus, can
be chosen to be real without the loss of generality, Now, let
. The SINR constraints become (36)
(34) in genera,l be written in different forms, each of which may lead to a different
dual. In this case, the duals of (31) and (35) turn out to be the same.
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
LUO AND YU: AN INTRODUCTION TO CONVEX OPTIMIZATION FOR COMMUNICATIONS AND SIGNAL PROCESSING 1433
The dual objective is further enhanced if we also take different base-station antenna
constraints into consideration. This is explored in the recent
work [17]. Duality is useful because the uplink problem is
(37) easier to solve. For example, an iterative optimization of and
leads to the optimal solution for the downlink [6], [7], [9].
It is easy to see that if is not a Algorithms based on convex optimization have been considered
positive semidefinite matrix, then there exists a set of which in [11] and [17].
would make . As should be chosen to maximize
C. Capacity Region Duality
, the second term in (36) leads to a positive semidefinite
constraint in the dual problem. In particular, the Lagrangian dual Instead of considering a power minimization problem subject
problem is to SINR constraints, in many applications, it is useful to con-
sider the reverse problem of maximizing a rate region, subject
to a power constraint. Under many practical coding and modula-
tion schemes, the SINR can be directly related to the achievable
rate using an SNR-gap approximation: ,
where is the gap to capacity, which is always greater than 1.
(38) The SINR duality stated in the previous section directly leads
to a rate-region duality as well. The argument follows from the
Interestingly, the above dual problem can be shown to corre- fact that since the minimum transmit power for each fixed set
spond to an uplink problem with as the (scaled) uplink power, of SINRs is the same for both uplink and downlink, the set of
as the uplink channel, and as the SINR constraint. achievable SINR regions under a fixed power must also be the
The uplink channel is depicted in Fig. 1. The sum power min- same in both uplink and downlink. Therefore, the rate regions
imization problem for the uplink channel can be formulated as must be the same.
This argument can be further generalized to a downlink
channel with multiple receive antennas for each user, and
it directly leads to the information theoretical duality of the
multiple-access channel and the broadcast channel [12], [13] by
letting and by using interference subtraction techniques.
(39)
Interference can be directly subtracted in the multiple-access
channel at the receiver, and can be presubtracted at the trans-
where the optimization is over the uplink power and the re- mitter in the broadcast channel. The previous proof of this
ceiver beamforming vectors . Clearly, the optimal is just capacity region uplink-downlink duality relies on an intricate
the MMSE filter . Substituting transmit covariance matrix transformation. The Lagrangian
the MMSE filter into the constraints of (39) and after a matrix approach shows an alternative way of deriving the same thing.
manipulation, it is possible to show that the uplink problem is For completeness, the most general capacity region duality
equivalent to [12], [13] is stated as follows. Consider a MIMO downlink
channel where each user is equipped with multiple antennas
(41)
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
1434 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 24, NO. 8, AUGUST 2006
The capacity region for the multiple-access channel may be ex- A. Multiuser Detection
pressed as Consider a detection and estimation problem in the receiver
processing of a MIMO channel
(46)
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
LUO AND YU: AN INTRODUCTION TO CONVEX OPTIMIZATION FOR COMMUNICATIONS AND SIGNAL PROCESSING 1435
models the requirement that the total received signal power at re-
ceiver must be above a given threshold (normalized to 1). This
is equivalent to the SNR constraint considered in Section III-A
(but without interference). To minimize the total transmit power
Fig. 2. BER as a function of SNR for different detectors. Simulation parame- subject to individual SNR requirements (one per each receiver),
ters: BPSK modulation n = 10.
we are led to the following nonconvex QP
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
1436 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 24, NO. 8, AUGUST 2006
V. ROBUST OPTIMIZATION
Robust optimization models in mathematical programming
have received much attention recently; see, e.g., [31]–[33]. In
this section, we briefly review some of these models and their
extensions.
Fig. 3. Upper bound on = for m = 8, n = 4, over 300 realizations of Consider a convex optimization of the form
complex Gaussian i.i.d. channel vector entries.
(54)
required to be real-valued, then the above approximation ratio
becomes quadratic in [30]. where each is convex. In many engineering design applica-
tions, the data defining the constraint and the objective func-
C. Future Research tions may be inexact, corrupted by noise, or may fluctuate with
An interesting future direction is to generalize the above for- time around a nominal value. In such cases, the traditional op-
mulation to a situation of multicast groups, timization approach simply solves (54) by using the nominal
, where is the index set for receivers partic- value of the data. However, an optimal solution for the nom-
ipating in multicast group , and . Common inal formulation (54) may yield poor performance or become
information is transmitted to members within each group; in- infeasible when each is perturbed in the actual design. A
dependent information is transmitted to different groups. As- more appropriate design approach is to seek a high-quality so-
sume that , , . Denote lution which can remain feasible and deliver high-quality per-
, . formance in all possible realizations of unknown perturbations.
Let denote the beamforming weight vector ap- This principle was formulated rigorously in [31]–[33]. Specifi-
plied to the transmitting antenna elements to transmit mul- cally, we consider a family of perturbed functions parameterized
ticast stream . The signal transmitted by the antenna array by , with taken from an uncertainty set . Then, a
is equal to , where is the information-bearing robustly feasible solution is the one that satisfies
signal directed to multicast group . This setup includes the
case of broadcasting (51) and the case of individual
transmissions (31) as special cases. If ’s are i.i.d. Thus, a robustly feasible solution is, in a sense, strongly fea-
with zero-mean and unit variance, then the total transmit power sible, since it is required to satisfy all slightly perturbed ver-
is equal to . The joint design of transmit beam- sion of the nominal constraint. The robust optimal solution can
formers subject to received SINR constraints can be posed as now be defined as a robust feasible solution which minimizes
follows: the worst-case objective value . This gives rise to
the following formulation:
(55)
Let us assume the perturbation vector enters the objective
(53) and the constraint functions in such a way that preserves con-
It can be seen that this problem is exactly the kind of nonconvex vexity, i.e., each remains a convex function for each
QPs considered in (51) except that here is no longer positive . In this case, the robust counterpart (55) of the original
semidefinite. Simulation results show that the corresponding (nominal case) convex problem (54) remains convex since its
SDP relaxation (52) still provides an excellent solution for this constraints are convex (for each and ) and the objective func-
multigroup multicast beamforming problem. tion is also convex.
The beamforming problem (53) is NP-hard in general (even 3A QP is homogeneous if it does not have linear terms. A homogeneous QP
the single group case (51) is NP-hard); see [29]. However, always has an SDP relaxation.
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
LUO AND YU: AN INTRODUCTION TO CONVEX OPTIMIZATION FOR COMMUNICATIONS AND SIGNAL PROCESSING 1437
REFERENCES
[1] Y. Nesterov and A. Nemirovskii, Interior Point Polynomial Methods
in Convex Programming. Philadelphia, PA: SIAM, 1994, vol. 13,
(56) Studies in Applied Mathematics.
[2] H. Wolkowicz, R. Saigal, and L. Vandenberghe, Handbook of Semidef-
where each is a prespecified scalar. In the above inite Programming: Theory, Algorithms and Applications. Norwell,
MA: Kluwer, 1999.
formulation, we have and [3] J. F. Sturm, “Using SeDuMi 1.02, a Matlab toolbox for optimiza-
. The main observation tion over symmetric cones,” Optim. Methods Softw. vol. 11–12, pp.
is that the robust constraint 625–653, 1999. [Online]. Available: http://www.fewcal.kub.nl/sturm/
software/sedumi.html
[4] M. Chiang, “Geometric programming for communication systems,”
Foundations and Trends Commun. Inf. Theory, vol. 2, no. 1–2, pp.
1–156, Aug. 2005.
is equivalent to the following second-order cone constraint [5] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge,
U.K.: Cambridge Univ. Press, 2004.
[6] F. Rashid-Farrokhi, K. J. R. Liu, and L. Tassiulas, “Transmit beam-
forming and power control for cellular wireless systems,” IEEE J. Sel.
Areas Commun., vol. 16, no. 8, pp. 1437–1450, Oct. 1998.
[7] E. Visotsky and U. Madhow, “Optimum beamforming using transmit
In this way, the robust linear program (56) is reformulated as an antenna arrays,” in Proc. IEEE Veh. Technol. Conf., May 1999, vol. 1,
pp. 851–856.
equivalent SOCP [8] F. Rashid-Farrokhi, L. Tassiulas, and K. J. R. Liu, “Joint optimal power
control and beamforming in wireless networks using antenna arrays,”
IEEE J. Sel. Areas Commun., vol. 46, no. 10, pp. 1313–1324, Oct. 1998.
[9] M. Schubert and H. Boche, “Solution of the multiuser downlink beam-
forming problem with individual SINR constraints,” IEEE Trans. Veh.
Technol., vol. 53, no. 1, pp. 18–28, Jan. 2004.
[10] M. Bengtsson and B. Ottersten, “Optimal and suboptimal transmit
beamforming,” in Handbook of Antennas in Wireless Communications,
References [31]–[33] have shown that the robust counterpart L. C. Godara, Ed. Boca Raton, FL: CRC, 2002.
of some other well-known convex optimization problems can [11] A. Wiesel, Y. C. Eldar, and S. Shamai, “Linear precoding via conic
also be reformulated in a finite way as a conic optimization optimization for fixed MIMO receivers,” IEEE Trans. Signal Process.,
vol. 54, no. 3, pp. 161–176, Jan. 2006.
problem, often as an SOCP or SDP. [12] S. Vishwanath, N. Jindal, and A. Goldsmith, “Duality, achievable rates,
As an application example, we consider the robust beam- and sum-rate capacity of Gaussian MIMO broadcast channels,” IEEE
forming problem of finding a such that Trans. Inf. Theory, vol. 49, no. 10, pp. 2658–2668, Oct. 2003.
[13] P. Viswanath and D. Tse, “Sum capacity of the multiple antenna
Gaussian broadcast channel and uplink-downlink duality,” IEEE
Trans. Inf. Theory, vol. 49, no. 8, pp. 1912–1921, Aug. 2003.
(57) [14] M. Schubert and H. Boche, “Iterative multiuser uplink and downlink
beamforming under SINR constraints,” IEEE Trans. Signal Process.,
vol. 53, no. 7, pp. 2324–2334, Jul. 2005.
where is the nominal steering vector, is the sample correla- [15] G. Caire and S. Shamai, “On the achievable throughput of a multi-
tion matrix, and represents the error size in the estimation of antenna Gaussian broadcast channel,” IEEE Trans. Inf. Theory, vol. 49,
steering vector . As it stands, the above robust beamforming pp. 1691–1706, Jul. 2003.
[16] H. Weingarten, Y. Steinberg, and S. Shamai, “Capacity region of the
formulation has an infinitely many nonconvex quadratic con- Gaussian MIMO broadcast channel,” IEEE Trans. Inf. Theory, 2006,
straints. Interestingly, by exploiting the phase ambiguity in to be published.
and using the same transformation outlined above, we can show [17] W. Yu and T. Lan, “Transmitter optimization for the multi-antenna
downlink with per-antenna power constraints,” IEEE Trans. Signal
[34] that (57) is equivalent to the following convex optimization Process., 2005, submitted for publication.
problem (SOCP): [18] N. Jindal, W. Rhee, S. Vishwanath, S. A. Jafar, and A. Goldsmith,
“Sum power iterative water-filling for multi-antenna Gaussian broad-
cast channels,” IEEE Trans. Inf. Theory, vol. 51, no. 4, pp. 1570–1580,
Apr. 2005.
[19] W. Yu, “Sum-capacity computation for the Gaussian vector broadcast
channel via dual decomposition,” IEEE Trans. Inf. Theory, vol. 52, no.
which can be solved efficiently with complexity. 2, pp. 754–759, Feb. 2006.
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.
1438 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 24, NO. 8, AUGUST 2006
[20] ——, “Uplink-downlink duality via minimax duality,” IEEE Trans. Inf. Zhi-Quan (Tom) Luo (SM’03) received the B.Sc.
Theory, vol. 52, no. 2, pp. 361–374, Feb. 2006. degree in mathematics from Peking University,
[21] B. Song, R. L. Cruz, and B. D. Rao, “Network duality and its applica- Peking, China, in 1984. From 1985 to 1989, he
tion to multi-user MIMO wireless networks with SINR constraints,” in studied at the Operations Research Center and the
Proc. IEEE Int. Conf. Commun., May 2005, vol. 4, pp. 2684–2689. Department of Electrical Engineering and Computer
[22] R. Cendrillon, W. Yu, M. Moonen, J. Verliden, and T. Bostoen, Science, Massachusetts Institute of Technology,
“Optimal spectrum balancing for digital subscriber lines,” IEEE Trans. Cambridge, where he received the Ph.D. degree in
Commun., vol. 54, no. 5, pp. 922–933, May 2006. operations research.
[23] W. Yu and R. Lui, “Dual methods for nonconvex spectrum optimization During the academic year of 1984 to 1985, he
of multicarrier systems,” IEEE Trans. Commun., vol. 54, no. 7, Jul. was with the Nankai Institute of Mathematics,
2006. Tianjin, China. In 1989, he joined the Department of
[24] J. Papandriopoulos and J. Evans, “Low-complexity distributed algo- Electrical and Computer Engineering, McMaster University, Hamilton, ON,
rithms for spectrum balancing in multi-user DSL networks,” in Proc. Canada, where he became a Professor in 1998 and held the Canada Research
IEEE Int. Conf. Commun., Jun. 2006. Chair in Information Processing since 2001. Since April 2003, he has been a
[25] M. Chiang, C. W. Tan, D. Palomar, D. O’Neil, and D. Julian, “Power Professor with the Department of Electrical and Computer Engineering and
control by geometric programming,” IEEE Trans. Wireless Commun., holds an endowed ADC Research Chair in Wireless Telecommunications with
submitted for publication, 2005, Appeared in part at IEEE INFOCOM, the Digital Technology Center, University of Minnesota. His research interests
New York, Jun. 2002. lie in the union of large-scale optimization, information theory and coding, data
[26] U. Fincke and M. Pohst, “Improved methods for calculating vectors communications, and signal processing.
of short length in a lattice, including a complexity analysis,” Math. Prof. Luo received an IEEE Signal Processing Society Best Paper Award in
Comput., vol. 44, no. 1, pp. 463–471, Apr. 1985. 2004. He is a member of the Society for Industrial and Applied Mathematics
[27] J. Jaldén and B. Ottersten, “On the complexity of sphere decoding in (SIAM) and MPS. He is also a member of the Signal Processing for Com-
digital communications,” IEEE Trans. Signal Process., vol. 53, no. 4, munications (SPCOM) and Signal Processing Theory and Methods (SPTM)
pp. 1474–1484, Apr. 2005. Technical Committees of the IEEE Signal Processing Society. From 2000 to
[28] M. Kisialiou and Z.-Q. Luo, “Performance analysis of quasi-max- 2004, he served as an Associate Editor for the IEEE TRANSACTIONS ON SIGNAL
imum-likelihood detector based on semi-definite programming,” in PROCESSING and Mathematics of Computation. He is presently serving as an
Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2005, vol. 3, Associate Editor for several international journals including SIAM Journal on
pp. 433–436. Optimization and Mathematics of Operations Research.
[29] N. D. Sidiropoulos, T. N. Davidson, and Z.-Q. Luo, “Transmit beam-
forming for physical layer multicasting,” IEEE Trans. Signal Process.,
vol. 54, no. 6, pp. 2239–2251, Jun. 2006.
[30] Z.-Q. Luo, N. D. Sidiropoulos, P. Tseng, and S. Zhang, “Approxima- Wei Yu (S’97–M’02) received the B.A.Sc. degree in
tion bounds for quadratic optimization with homogeneous quadratic computer engineering and mathematics from the Uni-
constraints,” SIAM J. Optimization, 2006, accepted for publication. versity of Waterloo, Waterloo, ON, Canada, in 1997,
[31] A. Ben-Tal, L. El Ghaoui, and A. Nemirovskii, “Robust semidefi- and the M.S. and Ph.D. degrees in electrical engi-
nite programming,” in Handbook of Semidefinite Programming, H. neering from Stanford University, Stanford, CA, in
Wolkowicz, R. Saigal, and L. Vandenberghe, Eds. Norwell, MA: 1998 and 2002, respectively.
Kluwer, Mar. 2000. Since 2002, he has been an Assistant Professor
[32] A. Ben-Tal and A. Nemirovskii, “Robust convex optimization,” Math. with the Electrical and Computer Engineering
Oper. Res., vol. 23, pp. 769–805, 1998. Department, University of Toronto, Toronto, ON,
[33] L. El Ghaoui, F. Oustry, and H. Lebret, “Robust solutions to uncertain Canada, where he also holds a Canada Research
semidefinite programs,” SIAM J. Optimization, vol. 9, no. 1, pp. 33–52, Chair. His main research interests are multiuser
1998. information theory, optimization, wireless communications, and broadband
[34] S. Vorobyov, A. Gershman, and Z.-Q. Luo, “Robust adaptive beam- access networks.
forming using worst-case performance optimization: a solution to the Prof. Yu is an Editor of the IEEE TRANSACTIONS ON WIRELESS
signal mismatch problem,” IEEE Trans. Signal Process., vol. 51, pp. COMMUNICATIONS and a Guest Editor of the EURASIP Journal on Ap-
313–323, 2003. plied Signal Processing (Special Issue on Advanced Signal Processing for
Digital Subscriber Lines).
Authorized licensed use limited to: SIMON FRASER UNIVERSITY. Downloaded on March 13, 2009 at 20:52 from IEEE Xplore. Restrictions apply.