Optimization and Algorithms

November 10, 2021

Write your name:

Write your student number:

Quiz
1. A basic question. (3 points) Task 1 of the project is about solving the problem
PT PT −1 2
minimize
x,u t=1 kEx(t) − q(t)k + λ t=1 ku(t)k (1)
subject to x(1) = xinitial
x(t + 1) = Ax(t) + Bu(t), for 1 ≤ t ≤ T − 1.

What does Ex(t) represent?

(A) The velocity of the target at time t

(B) The position of our vehicle at time t
(C) The control signal at time t
(D) The velocity of our vehicle at time t
(E) The position of the target at time t
(F) The state of our vehicle at time t

Write your answer (A, B, C, D, E, or F) here: B

2. Levenberg-Marquardt. (4 points) Suppose we use the Levenberg-Marquardt (LM)

method to address the optimization problem

minimize
n
f (x), (2)
x∈R
PP
where f : Rn → R is given by f (x) = 2
p=1 (fp (x)) . Assume that the function
n
fp : R → R is differentiable for 1 ≤ p ≤ P .
Suppose the current iterate is xk . To get the next iterate, the LM method starts by
solving one of the following problems:

(A) PP
minimize
n p=1 fp (xk ) + ∇fp (xk )T (x − xk ) + λk kx − xk k2
x∈R

(B)
minimize
n
f (xk ) + ∇f (xk )T (x − xk ) + λk kx − xk k2
x∈R

(C)
PP
2
minimize
n p=1 fp (xk ) + ∇fp (xk )T (x − xk ) + λk kx − xk k2
x∈R

1
 (D)
PP
2
minimize
n p=1 fp (xk ) + ∇fp (xk )T (x − xk ) + λk kx − xk k
x∈R

(E)

2
minimize
n
f (xk ) + ∇f (xk )T (x − xk ) + λk kx − xk k2
x∈R

(F)
P 2
P
minimize
n p=1 fp (xk ) + ∇fp (xk )T (x − xk ) + λk kx − xk k2
x∈R

Which problem?
Write your answer (A, B, C, D, E, or F) here: C

3. Gradient. (3 points) Consider the function f : R3 → R,

   2
x2
f (x1 , x2 , x3 ) =  x3  + 1 .
x1

The gradient ∇f (4, 0, 3) is one of the following vectors:

(A)  
0
36/5
48/5
(B)  
0
9
8
(C)  
48
0
36
(D)  
0
36
48
(E)  
9
0
8
(F)  
48/5
 0 
36/5

2
 Which one?
Write your answer (A, B, C, D, E, or F) here: F

4. Random target. (3 points) Task 5 of the project is about analyzing an optimization

problem of the form
K T T −1
!
X X X 2
minimize pk kExk (t) − qk (t)k + λ kuk (t)k (3)
x1 ,u1 ,x2 ,u2
t=1 t=1
|k=1 {z }
f (x1 ,u1 ,x2 ,u2 )
subject to x1 (1) = xinitial
x1 (t + 1) = Ax1 (t) + Bu1 (t) for 1 ≤ t ≤ T − 1
x2 (1) = xinitial
x2 (t + 1) = Ax2 (t) + Bu2 (t) for 1 ≤ t ≤ T − 1.

Recall that A, B, and xinitial are given constants. Also, λ > 0, p1 > 0, p2 > 0, and
p1 + p2 = 1.
One of the following statements about problem (3) is true:

(A) The function f is strongly convex

(B) The function f is a quadratic
(C) The solution of the optimization problem (3) when p1 = 0.7, p2 = 0.3, and
λ = 1 is the same as the solution of the optimization problem (3) when p1 = 0.7,
p2 = 0.3, and λ = 10
(D) The solution of the optimization problem (3) when p1 = 0.7, p2 = 0.3, and
λ = 1 is the same as the solution of the optimization problem (3) when p1 = 0.2,
p2 = 0.8, and λ = 1
(E) We have x?1 (t) = x?2 (t) and u?1 (t) = u?2 (t) for t = 1, 2, . . . , 24, where the symbol
(x?1 , u?1 , x?2 , u?2 ) denotes the solution of optimization problem (3)
(F) The function f is not convex

Which one?
Write your answer (A, B, C, D, E, or F) here: D

5. Convexity. (4 points) Consider the optimization problem

minimize
n
kAx − bk + f (x), (4)
x∈R

where A ∈ Rm×n and b ∈ Rn . Consider the following choices for the form of
function f :

(A) f (x) = kx − ck
(B) f (x) = (cT x)2
Tx
(C) f (x) = ec + kck2
(D) f (x) = cT x
(E) f (x) = (x − c)T x

3
 (F) f (x) = kxk + cT x.

For one of the six forms of f above, the optimization problem (4) is guaranteed
to have a unique global minimizer, regardless of how the constants A, b, and c are
chosen.
For which form?
Write your answer (A, B, C, D, E, or F) here: E

6. Trade-off. (3 points) Consider the optimization problem

minimize
n
f (x) + ρ g(x), (5)
x∈R

where f and g are nonnegative functions Rn → R (that is, f (x) ≥ 0 and g(x) ≥ 0
for all x ∈ Rn ), and ρ is a positive number.
Suppose that x1 is a global minimizer for (5) when ρ = ρ1 , and suppose that x2 is a
global minimizer for (5) when ρ = ρ2 . Consider that ρ2 > ρ1 > 0.
One of the following inequalities is guaranteed to be true:

(A) f (x1 ) + ρ1 g(x1 ) > f (x2 ) + ρ2 g(x2 )

(B) f (x2 ) > f (x1 )
(C) g(x2 ) ≥ g(x1 )
(D) g(x2 ) > g(x1 )
(E) f (x1 ) + ρ1 g(x1 ) < f (x2 ) + ρ2 g(x2 )
(F) f (x2 ) ≥ f (x1 )

Which inequality?
Write your answer (A, B, C, D, E, or F) here: F