Problem Sheet 1

Optimization Techniques: MA 526

September 12, 2021 D EBDAS G HOSH

1. For a given norm k · k in Rn , the sphere S(0; 1) = {x ∈ Rn : k · k = 1} in Rn is called unit

sphere w.r.t. the norm k · k. Draw the unit sphere in R2 w.r.t. the following norms:

(a) kxk1 = |x1 | + |x2 |

1/2
(b) kxk2 = (x21 + x22 )
(c) kxk∞ = max {|x1 |, |x2 |}
1/4
(d) kxk4 = (x41 + x42 ) .
1
2. In R2 , draw the neighbourhoods of radius 2
about the point (1, 1) w.r.t. the norms given in
Problem 1.

3. By Matlab, draw the α-level curves for α = 0.7, 7, 70, 200 and 700 for the following function
(known as Rosenbrock’s function or banana function)

f (x1 , x2 ) = 100(x2 − x1 )2 + (1 − x1 )2 .

4. Show that the function f : Rn → R is continuous at x̄ if and only if for every sequence {xn }
in Rn with xn → x̄, f (x̄n ) → f (x̄).

5. Let f : R2 → R be a continuous function. Prove the following:

(a) {(x1 , x2 ) ∈ R2 : f (x1 , x2 ) < 1} is an open set. (Hint: The set is f −1 (−∞, 1) ).
(b) {(x1 , x2 ) ∈ R2 : 1 < f (x1 , x2 ) < 2} is an open set.
(c) {(x1 , x2 ) ∈ R2 : f (x1 , x2 ) = 1} is a closed set.
(d) α-sublevel sets of f are closed sets for all α ∈ R.
(e) {(x1 , x2 ) ∈ R2 : f (x1 , x2 ) ≥ 2} is a closed set.

6. If G is a compact subset of Rn and f : Rn → R is a continuous function, prove that f (G) is

compact. (Hint: Use Bolzano-Weierstrass Theorem.)

7. Give an example of a set in R2 that is

(a) both open and closed

(b) neither open nor closed.

8. Is the set (x1 , x2 ) ∈ R2 : 1 ≤ esin x1 + (1 + |x2 |)cos x2 ≤ 2 compact? Explain why.

1
 
9. Is the set (x1 , x2 ) ∈ R2 : 0 ≤ x ≤ 4, x2 = esin x1 + (1 + |x1 |)cos x1 compact? Explain why.

10. A function f is called locally lipschitz on S ⊆ Rn if every x̄ ∈ S, there exists a neighbourhood

Nr (x̄) of x̄ such that f is Lipschitz on Nr (x̄) ∩ S. Construct a locally Lipschitz function that
is not Lipschitz continuous.

11. Prove that if f : R → R has bounded derivative, then f is Lipschitz on R.

12. Is every differentiable function Lipschitz?

13. Check if the following functions are Lipschitz:

(
x2 sin x1 if x 6= 0
(a) f (x) = on [0, 1].
0 if x = 0
(
x3/2 sin x1 if x 6= 0
(b) f (x) = on [0, 1].
0 if x = 0
(c) f (x) = x12 on 12 , 2 .
 

14. Let f1 be Lipschitz on S1 and f2 be Lipschitz on S2 . Suppose f1 (S1 ) ⊂ S2 . Show that f2 ◦ f1

(= f2 (f1 )) is Lipschitz on S1 .

15. Let f : Rn → R be a smooth function. Prove that the vector ∇f (x0 ) is orthogonal to the
tangent vector at x0 on the level set determined by {x ∈ Rn : f (x) = f (x0 )}.

16. Let {xn } be a sequence in Rn and f : Rn → R be a function such that the sequence {f (xn )}
is monotonic decreasing and bounded below. Prove that lim (f (xn+1 ) − f (xn )) = 0.
n→∞

17. Let {xn } be a convergent sequence in Rn and {xn } does not converges to the null vector.
Show that there exists a real constant c > 0 and a subsequence {xnk } of {xn } such that
kxnk k ≥ c for all k ∈ N.

18. A function f : Rn → R is said to be uniformly continuous on S ⊆ Rn if for any  > 0, there

exists δ > 0 such that |f (x) − f (y)| <  for all x, y ∈ S satisfying kx − yk < δ. Give an
example of function that is uniformly continuous but not Lipschitz continuous on a set. Is
every Lipschitz continuous function uniformly continuous? Explain.

19. Let {gk } and {sk } be two sequences in Rn so that gk> sk → 0. If {gk } has no subsequence
converging to zero, prove that there is a subsequence {gnk } of {gk } and a real number c > 0
such that kgnk k ≥ c for all k ∈ N and ksnk k → 0.

20. If the functions f : Rn → R and g : Rn → R satisfy f (x) = −g(x) + o(g(x)) and g(x) > 0
for all x 6= 0, then show that there exists a neighborhood Bδ (0) such that f (x) < 0 for all
x ∈ Bδ (0) \ {0}.

21. If A is real matrix of order n × n and y is a column vector in Rn , then show that

(a) ∇ y > Ax = A> y = y > A

(b) ∇ x> Ax = A + A> x

2
 (c) ∇ y > x = y

(d) ∇ x> x = 2x,

where x is the generic variable in Rn and ∇f (x) denotes the gradient of function f at x.
22. Find the Taylor’s quadratic approximation of the following functions about the given point x0 .
(a) f (x) = x1 e−x2 + x2 + 1, x0 = (1, 0)> .
(b) f (x) = x41 + 2x21 x22 + x42 , x0 = (1, 1)> .
(c) f (x) = ex1 −x2 + ex1 +x2 + x1 + x2 + 1, x0 = (1, 0)> .
(d) f (x) = 2x1 + 6x2 − 2x21 − 3x22 + 4x1 x2 , x0 = (0, 0)> .
x2 x2
23. Define the functions f : R2 → R and g : R → R2 by f (x) = 61 + 42 and g(t) =
(3t + 5, 2t − 6)> . Let F : R → R be given by F (t) = f (g(t)). Evaluate dF
dt
using the chain
rule.
24. Consider f (x) = 12 x1 x2 , g(s, t) = (4s + 3t, 2s + t)> . Evaluate ∂
∂s
f (g(s, t)) and ∂
∂t
f (g(s, t))
by the chain rule.
25. Let f : Rn → R be differentiable. From the formula of directional derivative f 0 (x; d) =
∇f (x)> d, ||d|| = 1, find the directions of the maximum growth and minimum growth of f .
x21 x22
26. Find the directions in which the function f (x1 , x2 ) = 2
+ 2

(i) increases most rapidly at (1, 1),

(ii) decreases most rapidly at (1, 1).
(iii) What are the directions of zero changes of f at (1,1)?
x21
27. With the help of the gradient of f (x) = 4
+ x22 at (−2, 1), find the equation of the tangent to
x21
the ellipse 4
+ x22 = 2 at (−2, 1).
28. Consider f (x1 , x2 ) = x21 − x1 x2 + x22 − x2 . Find a direction d for which
(i) f 0 ((1, −1); d) = 4
(ii) f 0 ((1, −1); d) < 0.
29. If f : R2 → R is defined in some neighbourhood of (a1 , a2 ), fx1 (a1 , a2 ) exists and fx2 is
continuous at (a1 , a2 ), then show that f is differentiable at (a1 , a2 ).
30. For an f : R2 → R, show by an example that the condition of continuity of one of the
first order partial derivative of f at (a1 , a2 ) is a sufficient condition but not necessary for
differentiability at (a1 , a2 ).
31. If a function f : Rn → R is differentiable at x̄, show that f is continous at x̄.
( x x2
1 2
x21 +x42
if (x1 , x2 ) 6= (0, 0)
32. Find the directional derivative of f (x1 , x2 ) = at (0, 0) along
0 if (x1 , x2 ) = (0, 0)
 
√1 , √1
2 2
. Why this directional derivative is not equal to ∇f (x̄)> d?

3
33. Let H be a n × n symmetric matrix. Using the eigenvalue characterizations of definiteness,
show that H is positive definite if and only if H is positive semidefinite and nonsingular.
 
2 2 3
34. Consider the function f (x) = x> Ax, x ∈ R3 , where A = 1 3 1. For what values of θ,
1 2 θ
the Hessian is positive definite?

35. Find the Hessian of the following function at (0, 0). Why the Hessian is not a symmetric
matrix?
( x3 x
2
1 2
2, if x21 + x22 6= 0
(a) f (x1 , x2 ) = x1 +x2
0, if x21 + x22 = 0
(
x1 x2 , if |x1 | ≥ |x2 |
(b) f (x1 , x2 ) =
−x1 x2 , if |x1 | < |x2 |.

36. Check if the following functions are differentiable at (0, 0) :

( 6 4
x −2y
2 2 , if x2 + y 2 6= 0
(a) f (x, y) = x +y
0 if x2 + y 2 = 0
( 3 3
x −y
x2 +y 2
, if x2 + y 2 6= 0
(b) f (x, y) =
0 if x2 + y 2 = 0.

37. Let f : Rn → R be continuously differentiable on Rn . For a given x̄ ∈ Rn and a direction d

in Rn , define a function φ : R → R by φ(t) = f (x̄ + td). Show that φ is differentiable and
φ0 (t) = d> ∇f (x̄ + td).

38. Let f : Rn → R be twice continuously differentiable on Rn . For a given x̄ ∈ Rn and a

direction d in Rn , define a function φ : R → R by φ(t) = f (x̄ + td). Show that φ is twice
differentiable function and φ00 = d> ∇2 f (x̄ + td)d.

39. Consider Problem 37 with d 6= 0. If φ has a minimum at t = t∗ and let x∗ = x̄ + t∗ d, prove

that ∇f (x∗ ) ⊥ d.

40. Let f : Rn → R be a continuously differentiable function, and g : R → Rn be given

by g(t) = (g1 (t), g2 (t), . . . , gn (t)) . Let φ(t) = f (g(t)). Show that φ is differentiable and
φ0 (t) = ∇f (g(t))> g 0 (t).

41. (Mean value theorem in Rn ). Let f : Rn → R be a continuously differentiable function.

Show that for any given x̄ and d ∈ Rn ,
R1
(a) f (x̄ + d) = f (x̄) + 0 ∇f (x̄ + td)> d dt and

(b) f (x̄ + d) − f (x̄) = ∇f (ξ)> d for some ξ in the open line segment joining x̄ and x̄ + d.

42. If f is twice continuously differentiable function, then for any x̄ and d ∈ Rn show that there
exists ξ in the open line segment joining x̄ and x̄ + d such that

f (x̄ + d) = f (x̄) + ∇f (x̄)> d + 12 d> ∇2 f (ξ)d.

4
43. Is norm uniformly continuous?

44. Show that the eigen values of a symmetric matrix are real.

45. Let A be a symmetric n × n matrix with eigen values λ1 , λ2 , . . . , λn . If λ1 ≤ λ2 ≤ · · · ≤ λn ,

then show that
λ1 kxk2 ≤ x> Ax ≤ λn kxk2 for all x ∈ Rn .