Solutions to Mid-semester Examination

Indian Institute of Technology (BHU) Varanasi

Optimization Techniques: MA 526
Odd-semester 2020–2021

Problem 1. Identify with justification if the given point (x̄1 , x̄2 ) is a saddle point or a local minimum or a
local maximum of the following functions:
(a) f (x1 , x2 ) = 4x41 + x1 x22 − 2x21 , (x̄1 , x̄2 ) = (0, 0)
x1 +x2
(b) g(x1 , x2 ) = x21 +x22 +1
, (x̄1 , x̄2 ) = (− √12 , − √12 )

Solution. (a) As fx1 (0, 0) = 0 = fx2 (0, 0), (0, 0) is a stationary point.
 
−4 0
The Hessian matrix of f at (0, 0) is , which is negative semidefinite. So, no conclusion can be
0 0
made from the Hessian matrix.
p
Denote Nδ (0, 0) = {(x1 , x2 ) ∈ R2 : x21 + x22 < δ}.
 4   
Notice that for any 0 < δ < 1, we have δ4 , √δ2 ∈ Nδ (0, 0), √δ2 , 0 ∈ Nδ (0, 0) and
   
δ 4 √δ δ 16
f 4 , 2 − f (0, 0) = 64 > 0 and f √δ , 0
2
− f (0, 0) = δ 2 (δ 2 − 1) < 0.

Hence, (0, 0) is a saddle point of f .

(b) Stationary points are given by ∇g(x1 , x2 ) = 0, i.e.,
( 2
−x1 − 2x1 x2 + x22 = −1
x21 − 2x1 x2 − x22 = −1
(
x21 = x22
or,
x1 x2 = 21
(
x1 = x2 since x1 x2 > 0
or, 1
x1 x2 = 2
n    
or, (x1 , x2 ) = √12 , √12 or − √12 , − √12

p p p
Notice that for any (x1 , x2 ) in R2 , x1 + x2 ≥ − (x1 + x2 )2 = − x21 + x22 + 2x1 x2 ≥ − 2x21 + 2x22 ,
and hence we have

√ √
p
x1 + x2 x21 + x22 ξ 1  
f (x1 , x2 ) = 2 ≥ − 2 ≥ − 2 max = − √ = f − √1 , − √1 .
x1 + x22 + 1 x21 + x22 + 1 ξ≥0 ξ 2 + 1 2 2 2
 
Thus, − √12 , − √12 is a point of (global) minimum of g.

x21 x32
Problem 2. (a) Find the directions in which the function f (x1 , x2 ) = 2 + 4

(i) increases most rapidly at (1, 0),

1
 (ii) decreases most rapidly at (0, 1).
(iii) What are the directions of zero changes of f at (1,1)?
(b) Consider the following function and check if it is differentiable:
( p
x1
|x1 | x21 + x22 if x1 =
6 0
f (x1 , x2 ) =
0 if x1 = 0.
Does this function have directional derivative at (0, 0) along every direction? Justify your answer.
Solution. (a) Given that f (x1 , x2 ) = 21 x21 + 14 x32 . The gradient of the function f is given by:
" # 
∂f 
∂x1 x1
∇f (x1 , x2 ) = ∂f = 3 2 .
∂x 4 x2
2

(i) The unit direction in which f increases most rapidly at (1, 0) is

[1, 0]>
 
∇f (1, 0) 1
= = .
k∇f (1, 0)k k[1, 0]> k 0

(ii) The unit direction in which f decreases most rapidly at (0, 1) is

−[0, 43 ]>
 
∇f (0, 1) 0
− = = .
k∇f (0, 1)k k[0, 34 ]k −1

(iii) Let u
b = (u1 , u2 ) be a unit direction. Then, the directional derivative of f along the direction u
b is
given by
Dub f (x1 , x2 ) = ∇f (x1 , x2 )> u
b.
Direction of zero changes of f at point (1, 1) is calculated by solving the following equation:
∇f (1, 1).b
u=0
=⇒ (3bi + 4b
j).(u1bi + u2b
j) = 0
3
=⇒ 3u1 + 4u2 = 0 =⇒ u1 = − u2 .
4
Also, u
b is unit direction. Therefore,
u21 + u22 = 1
9
=⇒ u21 + u21 = 1
16
4
=⇒ u1 = ± .
5
Hence, the directions of zero changes of f at (1, 1) are ( 54 , − 35 ) and (− 45 , 35 ).
(b) Given that (
x1
p
|x1 | x21 + x22 if x1 6= 0
f (x1 , x2 ) =
0 if x1 = 0.
For the unit vector u
b = (u1 , u2 ), the direction derivative at (0, 0) is given by
f (0 + tu1 , 0 + tu2 ) − f (0, 0)
Dub f (0, 0) = lim
t→0 t
f (tu1 , tu2 )
= lim
t→0
( t
0, if u1 = 0
= u1
|u1 | , if u1 6= 0.

2
 Therefore, the directional derivatives in all directions exist.

h
√
|h|
h2 +k2 −h
Note that fx1 (0, 0) = 1 and fx2 (0, 0) = 0. Note that (h, k) = √
h2 +k2
6→ 0 as (h, k) → (0, 0). For
h √1 )
example, h = k gives ( |h| − 2
6→ 0 as h → 0. Therefore, the function f is not differentiable at (0, 0).

Problem 3. Suppose g : Rn → R is twice continuously differentiable and there exists x̄ and d in Rn such
that g(x̄) = g(x̄ + d). If the matrix ∇2 g(x̄ + αd) is positive definite for all α ∈ (0, 1), then prove that there
exists γ ∈ (0, 1) which is a local minimum point of θ(α) = g(x̄ + αd). Is x̄ + γd a local minimum point of g?
Justify your answer.

Solution. By assumption, ∃ x̄ and d in Rn such that g(x̄) = g(x̄ + d) and ∇2 g(x̄ + αd) is PD for all α ∈ (0, 1).
Also, given that θ(α) = g(x̄ + αd) and g : Rn → R is twice continuously differentiable. By mean value
theorem,

θ0 (γ) = θ(1) − θ(0) for some γ ∈ (0, 1)

equivalently,

θ0 (γ) = g(x̄ + d) − g(x̄) for some γ ∈ (0, 1)

=⇒ θ0 (γ) = 0, (g(x̄ + d) = g(x̄))

Hence, γ is stationary point of θ. As

θ00 (γ) = d> ∇2 g(x̄ + γd)d > 0, for γ ∈ (0, 1),

γ is local minimizer of θ.

Since θ(α) = g(x̄ + αd), can we say x̄ + γd is local minimizer of g?

Since d is a specific direction, the answer is NO. (Note: If g(x̄) = g(x̄ + d) for all d ∈ Rn is given, then we
can say x̄ + αd is local minimum of g.)
For instance, take g(x1 , x2 ) = x21 + x22 − 1. Also, x̄ = (1, 2)> and d = (1, −1)> are such that g(x̄) = g(x̄ + d).
The matrix  
2 0
∇2 g(x̄ + αd) =
0 2
is positive definite for all α ∈ (0, 1). The local minimum point of θ(α) = g(x̄ + αd) = 2α2 − 2α + 4 is
γ = 21 ∈ (0, 1). Note that the minimum point of the function g is (0, 0)> , but x̄ + γd = (3/2, 3/2)> is not a
local minimum of the function g. Hence, x̄ + γd is not necessarily a local minimum of the function g.

Problem 4. Examine if the following sets are convex:

√
(a) S1 = {(a1 , a2 , a3 ) : |φ(t)| ≤ 1 + 2}, where φ(t) = 1 + a1 t + a2 t2 + a3 t3 , t ∈ R
n
!
X
n > 2 −xi
(b) S2 = {x ∈ R : (x Qx + 1) + log e ≤ 5}, where Q is an n × n positive semidefinite symmetric
i=1
matrix.

Solution. (a) Given

√
S1 = {(a1 , a2 , a3 ) : |φ(t)| ≤ 1 + 2}, where φ(t) = 1 + a1 t + a2 t2 + a3 t3 .

Let (a11 , a12 , a13 ), (a21 , a22 , a23 ) be any two arbitrary points from the set S1 . The set S1 is convex if for
any λ ∈ [0, 1],

λ(a11 , a12 , a13 ) + (1 − λ)(a21 , a22 , a23 ) ∈ S1 .

3
 Since (a11 , a12 , a13 ) and (a21 , a22 , a23 ) ∈ S1 ,
√
|φ1 (t)| ≤ 1 + 2, where φ1 (t) = 1 + a11 t + a12 t2 + a13 t3
√
and |φ2 (t)| ≤ 1 + 2, where φ2 (t) = 1 + a21 t + a22 t2 + a23 t3 .

Denote

x = λ(a11 , a12 , a13 ) + (1 − λ)(a21 , a22 , a23 )

= (λa11 + (1 − λ)a21 , λa12 + (1 − λ)a22 , λa13 + (1 − λ)a23 ).

Corresponding to x, φx (t) = 1 + (λa11 + (1 − λ)a21 )t + (λa12 + (1 − λ)a22 )t2 +(λa13 + (1 − λ)a23 )t3 .
Also,

|φx (t)| = |1 + λ(a11 t + a12 t2 + a13 t3 ) + (1 − λ)(a21 t + a22 t2 + a23 t3 )|

≤ 1 + λ|a11 t + a12 t2 + a13 t3 | + (1 − λ)|a21 t + a22 t2 + a23 t3 | (by triangle inequality)
√ √
≤ 1 + λ( 2) + (1 − λ)( 2)
√
≤ 1 + 2.

Hence, set S1 is convex.

n
(b) Note that the functions (x> Qx + 1)2 and log( e−xi ) are convex. Also, the sum of two convex functions
P
i=1
is a convex function. Therefore, the function
n
X
(x> Qx + 1)2 + log( e−xi ) is convex.
i=1

then its level set {x ∈ C : f (x) ≤ γ}, for any γ ∈ R, is convex. Here
If f is a convex function, P
n
f (x) = (x> Qx + 1)2 + log( i=1 e−xi ) and γ = 5. Hence, set S2 is convex.

Problem 5. Examine if the following functions are coercive:

(a) f (x, y) = log(x2 y 2 ) − x − y on R2 \ {(x, y) : xy = 0}
(b) g(x, y) = x2 + y 2 − sin(xy) on R2

Solution. (a) This function is not coercive since it does not possess the limit property of a coercive function.
Let φ(t) = −t + 2 log |t|, for t 6= 0. Then, f (x, y) = φ(x) + φ(y). Here, φ(1) = −1 and

t2
lim φ(t) = lim log = −∞.
t→∞ t→∞ et
So, lim f (x, 1) = lim (φ(x) − 1) = −∞ =
6 ∞.
x→∞ x→∞

(b) Since sin(xy) ≤ 1, we have

x2 + y 2 − 1 ≤ x2 + y 2 − sin(xy).
√ p
Hence, for every M > 0, there exists L = M + 1 > 0 such that if x2 + y 2 > L, then f (x, y) > M.
Thus, √ lim g(x, y) = ∞, and hence g is coercive.
x2 +y 2 →∞

Problem 6. Check if the following functions are Lipschitz:

(
x3/2 sin x1 if x 6= 0
(a) f (x) = on [0, 1].
0 if x = 0

4
 1
1 
(b) f (x) = x2 on 2, 2 .

Solution. (a) Given

(
x3/2 sin x1 if x 6= 0
f (x) = on [0, 1]
0 if x = 0

Here,
3√
(
0 − √1x cos x1 + 1
2 x sin x , if x 6= 0
f (x) =
0 if x = 0
|f (x1 )−f (x2 )|
and hence sup |x1 −x2 | = sup |f 0 (ξ)| = ∞. Therefore, f is not Lipschitz on [0, 1].
x1 ,x2 ∈[0,1] ξ∈[0,1]

1
(b) f (x) = x2 .

Here, f 0 (x) = − x23 .

|f (x1 )−f (x2 )|
and sup x1 −x2 = sup |f 0 (ξ)| = sup 2
ξ3 = 16.
x1 ,x2 ∈[ 12 ,2] ξ∈[ 12 ,2] ξ∈[ 21 ,2]
Hence, f is Lipschitz on [ 21 , 2].

Problem 7. Let A be a matrix of order m × n, b ∈ Rm . Prove or give a counter example for the following
statement.
‘Exactly one of the following systems has a solution:
System 1: ∃x ∈ Rn that satisfies Ax ≤ b
System 2: ∃z ∈ Rm that satisfies z ≥ 0, AT z = 0, bT z < 0’.

Solution. Recall the Farkas’ Lemma: exactly one of the following two systems has a solution—

System I: ∃x ∈ Rn such that Ax ≤ 0 and cT x > 0

System II: ∃y ≥ 0 such that AT y = c.

Notice that given System 1 and System 2 cannot be both true. Otherwise, we have the following impos-
sibility:
0 = xT (AT z) = (Ax)T z ≤ bT z < 0.
Next, we prove that if System 2 does not have a solution, then System 1 must have a solution. This will
prove, contrapositively, that if System 1 has no solution, then System 2 must have a solution.
T T
If System 2 does not hold, then for all z ≥ 0 with
 TA z = 0, we have b z ≥ 0.
A z 0
Denote Ā = [A b]. Then, for all z ≥ 0, ĀT z = T = n×1 .
b z ≥0
 
T 0n×1
Therefore, @z ≥ 0 such that Ā z = = c̄, say. Which shows that System II of Farkas’ lemma does
−1
not have a solution. Thus, System I must have a solution, i.e.,
 
x
∃ x̄ = ∈ Rn × R such that Āx̄ ≤ 0 and c̄T x > 0
α
i.e., Ax + bα ≤ 0 and − α > 0
 
x
i.e., A −α ≤ b,

i.e., System I has a solution. Hence, the result follows.

Problem 8. Let function x1 e−(x1 +3x2 ) be convex on the set {(x1 , x2 ) ∈ R2 : ax1 + bx2 + c ≥ 0} for some
constants a, b, c ∈ R. Identify the value of a − 9b + c.

5
Solution. Given

f (x1 , x2 ) = x1 exp−(x1 +3x2 ) .

We compute partial derivatives of first and second order:

fx1 (x1 , x2 ) = (1 − x1 ) exp−(x1 +3x2 )

fx1 x1 (x1 , x2 ) = (x1 − 2) exp−(x1 +3x2 )
fx2 (x1 , x2 ) = −3x1 exp−(x1 +3x2 )
fx2 x2 (x1 , x2 ) = 9x1 exp−(x1 +3x2 )
fx1 x2 (x1 , x2 ) = 3(x1 − 1) exp−(x1 +3x2 ) .

The Hession matrix is given by

 
f fx1 x2
∇ f (x) = x1 x1
2
fx2 x1 fx2 x2
 
−(x1 +3x2 ) x1 − 2 3(x1 − 1)
= exp .
3(x1 − 1) 9x1

Now,

|∇2 f (x)| = exp−2(x1 +3x2 ) (9x21 − 18x1 − 9(x1 − 1)2 )

= exp−2(x1 +3x2 ) (9x21 − 18x1 − 9x21 − 9 + 18x1 )
= −9 exp−2(x1 +3x2 ) ≤ 0 ∀ x1 , x2 ∈ R.

Hence, f is not convex function on the set

{(x1 , x2 ) ∈ R2 : ax1 + bx2 + c ≥ 0}.

Also, the nonconvexity of f does not depend on the choice a, b and c. Therefore, we can not identify the
value of a − 9b + c.

∗ ∗ ∗ ∗ ∗∗