C&O367: Nonlinear Optimization

(Winter 2013)
Assignment 2
H. Wolkowicz
Posted ???

Due: Thursday, Jan. 31, 10:00AM (before class),

1 Convex Sets
1.1 Intersection of Convex Sets
Show that the intersection of convex sets is a convex set. Is the union of convex sets convex?
(If not provide a counterexample.)
Solution: Let C1 , · · · , Ck be convex sets, and define C := ∩Ck . Assume x, y ∈ C
and λ ∈ [0, 1]. By definition of C we have x, y ∈ Ci , i ∈ {1, · · · , k}, and by
convexity we have λx+(1−λ)y ∈ Ci , i ∈ {1, · · · , k}. This means λx+(1−λ)y ∈ C,
so C is convex. (A similar argument holds for an infinite number of sets.)
This is not true for the union of convex sets. Let C1 := {0} and C2 := {1} as
subsets of R. Both sets are clearly convex, but the union of them is not convex.

1.2 Midpoint Convex

A set is C ⊆ Rn is midpoint convex if whenever x, y ∈ C we have 21 (x + y) ∈ C. It is clear
that C convex implies that C is midpoint convex.
1. Suppose that C is a closed midpoint convex set. Show that C is a convex set.

Solution: Let x, y ∈ C and assume that z is a point on the line segment

between x and y. We construct a sequence {zi } := xi +y
2
i
, where x0 = x,
y0 = y, and
 
zi zi ≤ z zi zi > z
xi+1 = yi+1 =
xi zi > z yi zi ≥ z

By this construction, z is always on the line segment between xi and yi , so

we have kzi − zk ≤ kx − yk2−i which means {zi } converges to z. By midpoint
convexity of C we have zi ∈ C for all i, and because C is closed we have
z ∈ C. This is true for all z on the line segment between x and y, so C is
convex.

2. Find an example of a set C that is midpoint convex but is not a convex set.

1
 Solution: Q, the set of all rational numbers is midpoint convex, but clearly
is not convex.

1.3 Halfspaces
1. Give conditions that guarantee that one halfspace contains another, i.e.,
{x ∈ Rn : aT x ≤ b} ⊆ {x ∈ Rn : āT x ≤ b̄},
where both a, ā are not zero.
Solution: Let’s denote the first halfspace {x ∈ Rn : aT x ≤ b} by H1 and the
other one by H2 . First I prove that there exists α > 0 such that a = αā,
i.e., a and ā are parallel. If not, there exists a vector w such that aT w = 0
and āT w > 0. Now for any point x ∈ H1 , we have aT (x + λw) = aT x ≤ b.
This means for all values λ > 0, aT (x + λw) ∈ H1 . However, we have
āT (x + λw) → ∞ when λ goes to +∞, and āT (x + λw) is not in H2 for all
values of λ which is a contradiction. Hence, a = αā. It’s easy to show that
for H1 ⊆ H2 , we must have b ≤ αb̄.
2. When are the halfspaces equal?
Solution: In this case we have H1 ⊆ H2 and H2 ⊆ H1 . By the above
explanation, there exists α > 0 such that a = αā and b = αb̄.

2 Convex Functions
2.1 Three slope rule
Suppose that f : R → R is convex and that x < y < z. Show that
f(y) − f(x) f(z) − f(x) f(z) − f(y)
≤ ≤ .
y−x z−x z−y
Solution: By simple calculations we can write y as the convex combination of
x and z as follows:
z−y y−x
y= x+ z,
z−x z−x
z−y y−x
+ =1
z−x z−x
convexity of f implies that
z−y y−x
f(y) ≤ f(x) + f(z).
z−x z−x
Both of the inequalities are derived by simple reformulations.

2
2.2 Convex and Strictly Convex Functions
Are the following functions convex or strictly convex on the specified sets:

1. f(x, y) = 5x2 + 2xy + y2 − x + 2y + 3 on D = R2 .

Solution: The Hessian of the function is

 
2 10 2
∇ f(x, y) = ,
2 2

which is positive definite, so the function is strictly convex.

2. f(x, y) = c1 x1 +c2 /x1 +c3 x2 +c4 /x2 on D = R2++ , where the ci are all positive numbers.

Solution: The Hessian of the function is

2c2 x−3
 
2 0
∇ f(x, y) = ,
0 2c4 y−3

which is positive definite for (x, y) ∈ R2++ and ci > 0, so the function is
strictly convex on the domain.

2.3 Quadratic Convex Functions

A quadratic function is one that can be expressed as

q(x) = x · Ax + b · x + α,

where A ∈ S n , the space of real symmetric n × n matrices, b ∈ Rn , and α ∈ R.

1. Show that the function

q(x) = (x1 − x2 )2 + (x1 + 2x2 + 1)2 − 8x1 x2

is a quadratic function by finding the appropriate A, b, α.

Solution: If you expand q(x), you can see that it’s a quadratic function
with the following parameters:
   
2 −3 2
A= , b= , α = 1.
−3 5 4

2. Compute the gradient and Hessian of q(x) and express them using the A, b, α computed
above in Item 1.

Solution: It is easy to check that ∇q(x) = 2Ax + b and ∇2 q(x) = 2A

which is true for any quadratic function.

3
3. Show that a quadratic function q(x) is convex if, and only if, A is positive semidefinite
and it is strictly convex if A is positive definite.
Solution: By the fact that ∇2 q(x) = 2A, the result is clear from the theorem
proved for a general convex function. But we use the following proof which
also solves the next question. For any x, y ∈ Rn and any λ ∈ (0, 1), let’s
define Q(x, y, λ) as

Q(x, y, λ) := λq(x) + (1 − λ)q(y) − q(λx + (1 − λ)y) = λ(1 − λ)(x − y)T A(x − y)

q(x) is convex if and only if Q(x, y, λ) ≥ 0 for all x, y ∈ Rn and any λ ∈ (0, 1).
As λ(1 − λ) > 0, this is true if and only if A is positive semidefinite. q(x)
is strictly convex if and only if Q(x, y, λ) > 0 for all x, y ∈ Rn and any
λ ∈ (0, 1), which is true if and only if A is positive definite.
4. Suppose that the quadratic function q(x) is strictly convex. Is the matrix A positive
definite?
Solution: Yes, Proved in the previous question.
5. Suppose that q(x) is a quadratic function of n variables such that the corresponding
matrix A is positive definite. Show that 0 = 2Ax + b has a unique solution and that
this solution is the strict global minimizer of q(x).
Solution: A is positive definite, so is invertible. Then 2Ax + b = 0 has
the unique solution x∗ = − 12 A−1 b. x∗ is a critical point and q(x) is strictly
convex, so x∗ is the unique global minimizer.
6. Suppose that q(x) is a quadratic function of n variables. Find necessary and sufficient
conditions for q(x) to be bounded below.
Solution: I claim that q(x) is bounded below if and only if A is positive
semidefinite and b ∈ R(A) (range of A). First note that for a convex
function, point x∗ is a global minimizer if and only if it’s gradient is equal
to zero (this is true only for convex functions). Assume that A is positive
semidefinite and b ∈ R(A), then by the above results, q(x) is convex. b ∈
R(A) means that 2Ax + b = 0 has a solution which is a global minimizer;
q(x) is bounded below.
Now, assume that q(x) is bounded below. If A is not positive semidefinite,
then there exists y such that yT Ay < 0. We may assume that bT y ≤ 0,
otherwise take −y instead. Then q(λy) goes to −∞ when λ goes to +∞
which is a contradiction. Hence, A is positive semidefinite. If b ∈ / R(A),
T
then b is not orthogonal to the null space of A = A. Hence, there exists y
such that yT Ay = 0 and bT y < 0. Again, q(λy) goes to −∞ when λ goes
to +∞ which is a contradiction; we must have b ∈ R(A).