Scribd
Upload
80 views2 pages

Practice questions-EE5180

This document contains practice questions related to convex sets, convex functions, and convex optimization. It begins with 5 questions about proving whether sets are convex or not. The next section includes 3 questions about showing functions are convex. Section 2.1 has 8 questions related to convex optimization, including defining convex sets/functions, properties of gradient descent, and analyzing convergence rates. Recommended reading materials on convex optimization are provided at the end.

Uploaded by

collinj1402
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
80 views2 pages

Practice questions-EE5180

This document contains practice questions related to convex sets, convex functions, and convex optimization. It begins with 5 questions about proving whether sets are convex or not. The next section includes 3 questions about showing functions are convex. Section 2.1 has 8 questions related to convex optimization, including defining convex sets/functions, properties of gradient descent, and analyzing convergence rates. Recommended reading materials on convex optimization are provided at the end.

Uploaded by

collinj1402
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Practice Questions 1: EE-5180

Rahul Meshram
August 22, 2023

1 Convex sets
1. Show that the hyperbolic set {x ∈ R2+ | x1 x2 ≥ 1} is convex.
2. Let C = {x ∈ Rn | α ≤ aT x ≤ β}. Is C convex? Provide proof.
3. Let C = {x ∈ Rn | αi ≤ xi ≤ βi , i = 1, 2, · · · , n}. Is C convex? Provide proof.
4. The set C = {x| x + S2 ⊆ S1 } where S1 , S2 ⊆ Rn with S1 is convex. Is C convex? Provide proof.
5. Let C ⊆ Rn be the solution set of quadratic inequality, i.e.,

C = {x ∈ Rn | xT Ax + bT x + c ≤ 0}

with A ∈ Sn , b ∈ Rn and c ∈ Rn . Show that C is convex if A ⪰ 0.

2 Convex functions
1. Suppose f : R → R is convex, with R ⊆ domf. Show that F (x) is convex, where
1 x
Z
F (x) = f (t)dt
x 0
domF = R++ .
2. If f1 and f2 are convex functions, then show that f (x) = max{f1 (x), f2 (x)} is also convex.
3. Show that f (x) = h(g(x)) is convex function, if h is convex, non-decreasing and g is convex function.

2.1 Example of convex optimization


1. Write definition of convex set and convex function
2. If function f is convex and differentiable, then is following true? For all y ∈ Rd

f (y) ≥ f (x) + ∇f (x)T (y − x) (1)

3. Write definition of Lipschitzness. Is function f (x) = x2 β Lipschtiz?


4. Suppose function f (x) = g1 (g2 (x)), where g1 is β1 Lipschitz and g2 is β2 Lipschitz. Then what can we
say about Lipschitz property of function f ?
5. Write a definition of smoothness of function f.
6. Suppose the gradient descent algorithm is

wt+1 = wt − η∇f (wt ) (2)

Write a first order approximation for this. Here η is learning rate.

1
7. Show that the gradient descent algorithm is same as solution of following optimization problem.
1
wt+1 = arg min ||w − wt ||2 + η f (wt )+ < w − wt , ∇f (wt ) >

(3)
2
Here < x, y >= xT y is the inner product of two vector.
PT
8. Suppose that w = T1 t=1 wt and f is convex function.
• Show that
T
1X
f (w) − f (w∗ ) ≤ (f (wt ) − f (w∗ )) (4)
T t=1

Here, w∗ is the minimizer of f (w).


• Further, show that
T
1X
f (w) − f (w∗ ) ≤ < wt − w∗ , ∇f (wt ) >
T t=1

• Let v1 , · · · , vT be an arbitrary sequence of vectors. Any algorithm with an initialization w1 = 0


and update rule of the form
wt+1 = wt − ηvt (5)
Show that
T T
X
t ∗||w∗ ||2 ηX
< w − w , vt >≤ + ||vt ||2 . (6)
t=1
2η 2 t=1
q
• Show that for ||vt || ≤ β and η = B2
β2 T , ||w∗ || ≤ B

T
1X Bβ
< wt − w∗ , vt >≤ √ .
T t=1 T

• Show that
Bβ
f (w) − f (w∗ ) ≤ √ . (7)
T
For any ϵ > 0, in order to achieve f (w) − f (w∗ ) ≤ ϵ, for how long we have to run Gradient descent
algorithm.

3 Recommended reading material


• Book: Convex optimization by Stephen Boyd (Chapter 2 and Chapter 3)

• Book: Understanding Machine Learning Algorithms by Shai Shalev-Shwartz and Shai Ben-David
(Chapter 12, 13 and 14 )

You might also like