Faculté des Sciences & Techniques

MASTER 1 MATH ACSYON

Practical Optimization - Lesson 3 :

Unconstrained Optimization & Least-Squares Problems
Homeworks
Vinh Thanh Ho, Paul Armand

E XERCISE 1 (Generalized Rosenbrock)

1. Minimize the generalized Rosenbrock function

n−1
(1 − x k )2 + 100(x k+1 − x k2 )2
X¡ ¢
g (x) =
k=1

for n = 4 by means of the solver SNOPT and with the starting point (−1.1,1,1,1).
2. Calculate ∇g (x) by hand and then use MATLAB to verify that the convergence point returned by SNOPT is a stationary
point.
3. Calculate ∇2 g (x) by hand and then use MATLAB to compute the smallest eigenvalue at the convergence point
(indicate the method used). Conclude the optimality of this point for the optimization problem.

E XERCISE 2 (Unconstrained Optimization) · 10 points

1. Minimize the function

f (x) = (x 1 (1 − (x 2 )3 ) − 2.625)2 + (2.25 − x 1 (1 − (x 2 )2 ))2 + (1.5 − x 1 (1 − x 2 ))2 , x = (x 1 , x 2 ) ∈ R2

by means of the solver KNITRO with the starting point (0,0).

Report the found optimal solution x ∗ and its objective function value.
2. Calculate ∇ f (x) by hand and then use MATLAB/OCTAVE to verify that the convergence point x ∗ is a stationary point.
3. Calculate ∇2 f (x) by hand and then use MATLAB/OCTAVE to compute the smallest eigenvalue at x ∗ . Is x ∗ a strict
minimum ? Explain.
4. Is there another stationary point ? Which one if any ? Minimum ? Maximum ? Saddle point ?

E XERCISE 3 (Bird wings)

A biologist is studying the length of birds wings and would like to compute the parameters of his theoretical model. His
measures are in the file wing.dat below. The data are denoted by (t i ,y i ) for i = 1, . . . , n where n = 71 and are plotted on
Figure 1. The mathematical model is based on the function
x2
f x (t ) = ,
1 + x 2 x 3 e −x1 t

where t is the age of a bird, f x (t ) is the wing length and x i > 0, i = 1,2,3, are the parameters to be determined. This kind of
function is called a sigmoid curve (or logistic curve) and is commonly used to modelize biological phenomenons such as
population growth.
1. We first want to use a polynomial model of the form
p
qk t k .
X
q(t ) =
k=0

a. Determine polynomials of degree p = 3,4,5 that approximate the data by means of the solution of linear
least-squares problems. To compute the coefficients of the polynomials, apply the four different methods, that is
Cholesky, QR, solution of an augmented system (do not use the structure of sparse matrices) and SVD. Use a
Matlab or Octave script.
b. For each value of p, compute the value of the norm of the residuals and plot a graphic to compare the theoretical
model and the data.
c. Are the results better when the degree p increases ? Do you think that a polynomial a good model ?
2. We get now approximate values of the three parameters x 1 , x 2 and x 3 for the sigmoid model.
a. Knowing that x 1 > 0, calculate limt →∞ f x (t ) and deduce an approximation of x 2 .
b. Calculate f x (0) and deduce an approximation of x 3 .
 c. Determine the inflection point of the curve (the value of t such that f x′′ vanishes and changes sign) and deduce
an approximation of x 1 .
d. Plot the curve of f x with the values of the parameters determined previously and draw the data (t i ,y i ) on the
same graphic window.
3. Now we would like to compute optimal parameters for the model. To do that we solve the following nonlinear least
squares problem,
n
minimizex∈R3 φ(x) := ( f x (t i ) − y i )2 .
X
i =1

a. Modelize the problem with AMPL and solve it. Use the values calculated at the previous question as starting
values of the minimization procedure.
b. Plot the curve of f x with the value of the optimal parameters and compare it with the model obtained in
question 2.
c. Calculate ∇φ(x) at the solution and check the first order optimality conditions.
d. Calculate the Hessian matrix ∇2 φ(x) at the solution and check the second order optimality conditions.
e. Try to solve the problem with the starting point (0,0,0) and comment the results returned by the solver.

# wing.dat
data;
param n := 71;
param t :=
1 0 9 2 17 3 25 8 33 16 41 22 49 27 57 29 65 33
2 0 10 2 18 3 26 9 34 17 42 22 50 27 58 30 66 34
3 1 11 2 19 4 27 9 35 18 43 23 51 28 59 30 67 34
4 1 12 2 20 4 28 11 36 18 44 23 52 28 60 31 68 37
5 1 13 2 21 4 29 13 37 19 45 24 53 28 61 31 69 37
6 1 14 2 22 5 30 13 38 20 46 25 54 28 62 31 70 39
7 1 15 2 23 6 31 15 39 20 47 25 55 28 63 32 71 41
8 1 16 2 24 7 32 15 40 22 48 26 56 29 64 32;
param y :=
1 19 9 24 17 25 25 41 33 100 41 145 49 200 57 220 65 261
2 20 10 25 18 24 26 50 34 110 42 157 50 210 58 248 66 270
3 19 11 24 19 29 27 56 35 123 43 185 51 225 59 230 67 250
4 19 12 22 20 30 28 70 36 120 44 142 52 210 60 240 68 275
5 20 13 22 21 30 29 75 37 126 45 190 53 205 61 235 69 280
6 20 14 23 22 29 30 54 38 145 46 180 54 215 62 218 70 270
7 21 15 21 23 29 31 90 39 130 47 177 55 210 63 215 71 300
8 21 16 27 24 30 32 95 40 160 48 175 56 210 64 247;

71 measures Logistic curve

300 300

250 250

200 200
Wing length

fx(t)

150 150

100 100

50 50

0 0
0 5 10 15 20 25 30 35 40 45 0 5 10 15 20 25 30 35 40 45
Bird age t

F IGURE 1 – The set of measures and a sigmoid curve