Optimization

P. Gourdel1
April 29, 2010

1 [email protected],

University of Paris 1

University Paris 1
M1QEM1

2008/2009
Mr. Gourdel

Chapter 1

Presentation of Optimization
minimum of f on C). The function f is the objective function and the
set C is the set of feasible points (admissible points), it is often described
by a finite list of constraints.
We will note

Notations
n
On the
p product will be denoted by
Pneuclidean space R , the inner
hu, vi = i=1 ui vi , the norm is kuk = hu, ui. We denote by B(x, r)
and B(x, r) the closed ball and the open ball of center x and radius r.
If A is a subset of Rn , int A is the interior of A and cl A the closure of
A. We denote x y (respectively x > y, x  y) if for all h = 1, . . . , n
xh yh (respectively for all h = 1, . . . , n xh yh and there exists at least
one indice for which the inequality is strict, respectively for all h = 1, . . . , n
xh > yh ).
If f is a linear form from Rn to R, there exists a unique vector u Rn
such that f (v) = hu, vi for all v Rn .
We denote

(P) max f (x)

xC

1.1.1

xC

Definition 1.1 The point x is solution of (P) (respectively of (Q)) if

x C and if for all x in C, f (x) f (x) (respectively f (x) f (x)).
Definition 1.2 The point x is a local solution of (P) (respectively of
(Q)) if x C and if there exists > 0 such that for all x in C B(x, ),
f (x) f (x) (respectively f (x) f (x)).
Definition 1.3 We define the value of Problem (P) (respectively of (Q))
the supremum (respectively the infimum ) of the set {f (x) | x C}. This
value is either finite or infinite.

Rn+ = {x Rn | x1 0, . . . , xn 0} = {x Rn | x 0}
Rn+ \ {0} = {x Rn | x > 0}
Rn++ = {x Rn | x1 > 0, . . . , xn > 0} = {x Rn | x  0}

1.1

resp. (Q) min f (x)

If the domain C is empty, we will let by convention val(P ) = and

val(Q) = +. We will denote Sol(P ) for the set of solutions of (P).
One should distinguish between c a solution of (P) which is a vector of Rn and v = f (c) the corresponding value which is an element of
[, +]. There might exist several solutions c while the value v is
unique.

Mathematical presentation
Definitions

Example 1.1

Let us consider f : A R (where A R is the domain of function f ) and

C A. The problem consists in finding the maximum (respectively the

(P) max sin x

xR

val(P) = 1 while Sol(P) = {/2 + 2k | k Z}

1. Prove that the three following problems are equivalents.

Example 1.2

2. Prove with a counter-exemple that (P1 ) and (P2 ) are not necessarily
equivalent if is only non decreasing.

(P) max
xR

x
1 + x2

Exercise 1.3 (**) Let f be a function defined on C. Let us suppose that

: X R is an non-decreasing function, where X contains cl(f (C)). We
assume in addition that (P1 ) has a finite value.


max f (x)
max (f (x))
(P1 )
(P2 )
xC
xC

val(P) = 1 while Sol(P) =

Exercise 1.1 (*) Determine the set of solutions, the set of local solutions, and the value of the following problems:


max 1 x2
max 1 x2
1. (PY )
(PX )
xR
x ]2, 3[

min x2 + x + 1
2. (PZ )
xR

min x2 + x + 1
3. (PT )
xN

1. Prove that val(P2 ) (val(P1 )).

2. Prove that if is continuous then val(P2 ) = (val(P1 )).
3. Show that if there exists a solution, then val(P2 ) = (val(P1 )).
4. Let us consider f (x) = x, C = ]0, 2[ and : R R defined by
(x) = int(x) + x where int(x) denotes the ceiling function (the
ceiling fonction of x is min{n Z | x n}). Show that is
increasing. Compute val(P1 ), val(P2 ) and (val(P1 )). Prove that
val(P2 ) < (val(P1 ))

Proposition 1.1 Let f : A R. Let us consider the optimization problem (P), maxxC f (x) whose value is , we have
Sol(P) = f 1 ({}) C and Sol(P) = {x A | f (x) } C.

Exercise 1.4 (*) Let X Y be two subsets of Rn and f be a function

from X to R. Let us consider two optimization problems:


max f (x)
max f (x)
(PX )
(PY )
xX
xY

In particular, this set is closed in A if f is continuous and C is a closed

set.
In particular, if val(P)
/ R, then Sol(P) = .

1. Show that val(PX ) val(PY ).

Definition 1.4 If (P) and (Q) are two optimization problems, we say
that they are equivalent if their sets of solution are equal (but in general,
their values are not equal).

2. If y Sol(PY ), and if y X, show that y Sol(PX ). This means :

Sol(PY ) X Sol(PX )

Exercise 1.2 (*) Let f be a function defined on C. Let us suppose that

: X R is an increasing function, where X contains f (C).



max f (x)
max (f (x))
min (f (x))
(P1 )
(P2 )
(P3 )
xC
xC
xC

3. If x Sol(PX ), and if val(PX ) = val(PY ), then prove that x

Sol(PY ). Deduce that
val(PX ) = val(PY ) Sol(PX ) Sol(PY )
2

4. We assume that for all y in the set Y , there exists x X such that
f (x) f (y), show that val(PX ) = val(PY ).

Definition 1.5 Let us consider the optimization problem (P),

maxxC f (x) (respectively minxC f (x). The sequence (xk )k is said to
be a maximizing (respectively minimizing) sequence for (P) if for all k,
xk C and if the limit of f (xk ) exists (either finite or infinite) and is
equal to the value of the problem (P).

5. In the following example, show that Sol(PY ) 6 Sol(PX ).


(PX )

max 1 x2
x ]2, 3[


(PY )

max 1 x2
xR

Exercise 1.7 (*) Let us consider the following optimization problem

where is a given parameter:

6. Same question for Sol(PX ) 6 Sol(PY ).



(PX )

max ex (sin x + 2)
x [0, ]


(PY )

(PX )

max ex (sin x + 2)
xR

min x2
xR

1) If > 0, determine a minimizing sequence.

2) If = 0, determine a minimizing sequence.
3) If < 0, does there a minimizing sequence.

Exercise 1.5 (*) Let X be a nonempty subset of Rn and f , g be continuous functions from X to R. Let us consider two optimization problems:


max f (x)
max f (x)
xX
(PX )
(PY )
xX

g(x) < 0

Exercise 1.8 (**) Let X be a nonempty subset of Rn , and f be a function from X to R. Let us consider the following optimization problem:

Let us assume that x is a local solution of PY that satisfies g(x) < 0.

Prove that x is a local solution of PX .


(PX )

Exercise 1.6 (**) Let (P) be the optimization problem maxxC f (x).
Let us define the set

max f (x)
xX

1. Prove that there exists a maximizing sequence (xk )k , i.e. a sequence

of elements in X such that f (xk ) maxxX f (x).

D = {(x, y) Rn R | x C and y f (x)}

2. Prove that there exists a maximizing sequence (yk )k such that f (yk )
(f (yk+1 ).

Let us define g on R R par g(x, y) = y and the optimization problem

(Q), max(x,y)D g(x, y).

3. Let us suppose moreover that Sol(P) = , prove that there exists a

maximizing sequence (zk )k such that f (zk ) < (f (zk+1 ).

1. Show that (P) and (Q) have the same value.

2. Show that if x Sol(P), then (x, f (x)) Sol(Q).

1.1.2

3. Prove that if (x, y) Sol(Q), then x Sol(P), and y = f (x).

Geometric interpretation

Depending wether the optimal point belongs or not to the boundary of

the set of feasible points, we get using the level sets two kinds of pictures.

4. Are the two problems equivalent ?

3

consumers demand is the set of solutions of

max u(x)
p1 x1 + . . . + p` x` w
(PX )

x1 0, . . . , x` 0

1.2.2

Producer Theory

In microeconomics, we consider a firm which produces the good `, using

goods (1, . . . , ` 1) as inputs. We describe the production set with f ,
to R. Let us give a price vector p =
production function from R`1
+
(p1 , . . . , p`1 ) of the inputs and a level of production y` 0, The cost
function c((p1 , . . . , p`1 ), y` ) of the firm is the value fo the problem

min p1 y1 + . . . + p`1 y`1

y` = f (y1 , . . . , y`1 )
(P)

y1 0, . . . , y`1 0

Figure 1.1: The optimal point belongs to the boundary

The firms demand of inputs corresponds to the set of solutions of (P). In

addition, if we consider the price p` of the unique output, the total offer
(with usual signs convention) of the firm is the set of solutions of

(Q)

1.2.3

1.2.1

Finance theory

In finance, there are S possible states of the world tomorrow with the
corresponding probabilities 1 , . . . S . Today, we can buy or sell J assets
with price q1 , . . . , qJ . If we own one unit of asset j, we will receive if
state s occurs, the amount (possibly negative) ajs . The investor will try
to maximize the expected value of his stochastic income with respect to
his initial capital w. He will buy a portfolio (z1 , . . . , zJ ), solution of

Figure 1.2: The optimal point belongs to the interior

1.2

max p` y` c((p1 , . . . , p`1 ), y` )

y` 0

Examples of economic problems

Consumer theory

(
R`+

In microeconomics, we suppose that u is a utility function from

to
R. let us give a price vector p = (p1 , . . . , p` ) and a wealth w 0. The
4

P
s Jj=1 ajs zj
j=1 qj zj w

max
PJ

PS

s=1

1.2.4

Statistics

A second class of problem consists in maximizing the income. We assume that the planer owns an initial stock 1 , . . . , ` of inputs and that the
marginal income of process j is rj . The problem consists in determining
all activity levels xj solutions of

Pm
rx
max
Pm jj=1 j j
i=1 h xj h for all h = 1, . . . , `,

xj 0, for all j

In statistics, we determine an estimator using the maximum of the likehood, and we determine the regressions lines by minimizing the sum of
the squares of the distance to the line among all possible lines.

1.2.5

Transportation problems

Let us consider a firm with m units of production P1 , . . . , Pm , which

produce quantities q1 , . . . , qm of a certain good. There are n markets
M1 , . . . , Mn to provide whose respective demands are 1 , . . . , n . In order
to transport one unit of good from the the unity i to the market j, there is
a cost ij . We try to provide all the markets at the lowest transportation
cost. We have to determine all the flows xij (quantity moved from Pi to
Mj ) solution of

P Pn
min m

i=1
j=1 ij xij

m
Pni=1 xij j for all j,

j=1 xij qi for all i,

xij 0, for all i and all j

1.2.6

Constant returns to scale

Let us consider a firm using m processes P1 , . . . , Pm of production. The

process Pj is characterized by a vector j R` . For a single level of
activity, there are hj units of good h produced by the firm if hj 0 and
hj units of good h used by the firm in the process if hj 0. The total
amount of activity of Process Pj will be denoted by xj 0.
A first class of problem consists in furnishing the demand at minimal
cost. There are ` markets (one for each good) with respective demands
1 , . . . , n and the marginal cost of Process Pj is j . The problem consists
in determining all activity levels xj solutions of

Pm
x
max
Pm jj=1 j j
i=1 h xj i for all i,

xj 0, for all j
5

University Paris 1
M1QEM1

2008/2009
Mr. Gourdel

Chapter 2

Convexity of sets
2.1
2.1.1

Definition

Exercise 2.1 (*) We recall that a set J included in R is an intervall if

it satisfies the following property :

Definition of a convex set

x J, y J, z R,

Let E be a vector space, for all couple of elements (x, y) of E, we will

denote by [x, y], the segment which is the subset of E defined by

Show that every compact interval of R is a line segment of R and that

the only nonempty convex subsets of R are the intervals.

[x, y] = {tx + (1 t)y | t [0, 1]}

x z y z J.

Exercise 2.2 (*) Use Definition 2.1 in order to show that the following
sets are convex:

a) {(x, y) R2 | x2 + y 2 < 4},

[x, y] = [y, x]

b) {(x, y) R2 | |x| + |y| 2},

by

c) {(x, y) R2 | max{|x|, |y|} 2},

Figure 2.1: segment

d) the set of the (n n)-matrices with elements 0.

Definition 2.1 A subset C of E is convex if for all (x, y) C C, the
set [x, y] is contained in C.

Note that there is no notion of concave set. It is important to

emphasize that it not a topologic concept, it can be defined even if the
vector space is not embedded with a topology. Among convex sets, some
of them are open, closed or even neither closed nor open.

Examples: For all couple of elements (x, y) in E, [x, y] is a convex subset

of E. All vector subspace is affine (see the next part), all affine subspace
is convex. All open (respectively closed) balls are convex. All set of
solutions of linear system (involving equalities, large or strict inequalities)
is convex, in particular any affine half-space. If E = R, we can characterize
the convex subsets which are the intervals.

Exercise 2.3 (**) Let A be a subset of Rn . We will introduce the following property
Forall x A, y A, 12 (x + y) A
(P)
6

a) Prove that J is a closed set containing both 0 and 1.

C

b) Prove that when t and t0 are in J, then (t + t0 )/2 is in J.

c) Let us define for each p N, the set Jp


k
p
Jp =
| k N, k 2
2p

Figure 2.2: A convex and closed half space

Prove by induction that for each p N, Jp J. (Hint: note that

(1/2)(Jp + Jp ) = Jp+1 .

d) Prove that J = [0, 1] and that A is convex.

4. Summarize the exercise.

Exercise 2.4 (**) In R2 , consider the triangle with vertices x0 , x1 , x2
(non-collinear points).
Figure 2.3: A convex half space which is neither closed nor open
1. Prove that for all u R2 , there exists a unique (0 , 1 , 2 ) R3+ such
that u = 0 x0 + 1 x1 + 2 x2 and 0 + 1 + 2 = 1. The coefficients
0 , 1 , 2 are called the barycentric coordinates of u with respect to
x0 , x1 , x2 .

1. Prove that this property is satisfied when A is convex.

2. Prove that Q satisfies this property though it is not a convex set.

2. Let y be the midpoint of the side opposite x0 and let z be the intersection of the three medians. What are the barycentric coordinates
(with respect to {x0 , x1 , x2 }) of respectively: x0 , x1 , x2 , y, z?

3. We assume that A is a closed set that satisfies Property (P). We

want to prove that it is a convex set. Let us fix x and y in A, and let
us introduce the set
J = {t [0, 1], tx + (1 t)y A}.

2.1.2

Definition of an affine subspace

Definition 2.2 A subset A of E is affine if for all couple of distinct points

of A, the line defined by those two points is still in A. Formally, if for all
(x, y) A A, the set = {tx + (1 t)y | t R} is included in A.

Note that the empty set is affine. It is easy to check that any translation of
a vector subspace is an affine set. Exercise 2.8 will show that the converse
is true when the set is nonempty.
Let us introduce as a complement the definition of a strictly convex
set (the definition given here is not the most general).

Figure 2.4: A convex and open half space

7

Definition 2.3 Let C be a subset of E with a nonempty interior, C is

strictly convex if for all (x, y) cl C cl C, such that x 6= y, and for all
]0, 1[, x + (1 )y int C.

Definition 2.5 Let (xi )ni=1 , be n points of Rn . An affine combination

n
of (xP
x of Rn such that there exists Rn satisfying
i )i=1 is an element
P
x = ni=1 i xi and ni=1 i = 1.

Exercise 2.5 (**) Let C be a subset contained dans E.

Proposition 2.1 Let C be a subset of E. The set C is convex if and only

if C contains all the convex combinations of finite families of elements of
C.

1. Prove that C is convex if and only if for all (x, y) C C, such that
x 6= y, and for all ]0, 1[, x + (1 )y C.

Proof of Proposition 2.1.

It is obvious that if C contains all the
convex combinations of finite families of elements of C then C is a convex
subset of C.
Reciprocally, we prove the result by induction on the cardinal of the
family. if the family has one or two elements, the definition of a convex
subset show that all convex combination of this family remain in C. Let us
assume that this is true for all the families that have at most n elements.
Let (xP
a family of elements of C. Let S n and let
1 , . . . , xn , xn+1 ) beP
n+1
x := i=1 i xi . Since n+1
i=1 i = 1, there exists at least one i which
is not equal to 0. Let us assume with no loss of generality that 1 6= 0.
Then

2. Prove that if C is strictly convex, then C is convex.

3. If C is strictly convex and D be a subset of E such that int(C)
D cl(C), prove that D is convex.
4. Give an example of C and D, subsets of R2 , such that C is convex,
int(C) D cl(C) but D is not convex.

2.1.3

Definition of the unit simplex of Rn

Let us now introduce a very important example of convex set in Rn , the

unit simplex (or usual simplex) denoted by S n1 which is defined by:
S n1 = { = (1 , . . . , n ) Rn+ |

n
X

x=

i = 1}

n
X
i=1

i=1

! n

X i

+ n+1 xn+1
i
x
i
n

X
i=1

i
i=1

We can remark that S n1 is convex, closed and bounded, consequently it

is a compact set.

using the induction hypothesis, if we define

0

x =

2.2

First properties

n
X
i=1

X
i
xi =
i x i ,
n
X
i=1
i
i=1

Definition 2.4 Let (xi )ki=1 , be k points of Rn . A convex combination (of

k
n
k1
length k) of (xP
i )i=1 is an element x of R such that there exists S
k
satisfying x = i=1 i xi .

i , we can remark that x0 is an element of C since it is a

where i = X
n
i

Exercise 2.6 (*) If (x, y) is a couple of elements in Rn , show that the

set of convex combinations of x and y is [x, y].

convex combination
Pn of a family of n elements of C. In order to conclude,
if we let =
i=1 i , [0, 1], this allows us to see x as a convex

i=1

combination of x0 and xn+1 since x = x0 + (1 )xn+1 . Consequently x

is in C, which ends the proof.


Exercise 2.10 (**) Let (Ci )iI be a family of convex subsets of some
vector space E.
Let us now assume that I is any set of indices and that the family of
convex subsets satisfies that for all (i, j) I I, there exists k I such
that Ci Cj Ck . then iI Ci is convex.

Proposition 2.2 Let A be a subset of E. The set A is affine if and only

if A contains all the affine combinations of finite families of elements of
A.

2.2.2

Exercise 2.7 (*) Show Proposition 2.2.

Proposition 2.4 Let E be a vector space. Let (Ci )iI , be a finite family
of convex subsets of E. Then
Y
X
X
Ci }
ci | (ci )
Ci := {

Exercise 2.8 (*) Let A be a nonempty affine subset of E (vector space)

and a A.
For any a in A, let us define the translated set Ba := A {a} =
A + {a}. This can reformulated as, x Ba if and only if x + a A.
Check that 0 Ba .

iI

iI

iI

is a convex subset of E.

Deduce from Proposition 2.2 that Ba is a vector subspace of E.

The proof is left to the reader, it is worth to notice that for a convex
set 2C = C + C and that this is not true in general.

Deduce from Proposition 2.2 for if a and a are in A, then Ba = Ba ,

which means that the set B is independent of the particular choice
of a.

Exercise 2.11 (**) We recall that if X is a subset of some vector space

E, 2X = {z E | x X, z = 2x}.

Consequently, the set B is called the direction of A and we

call affine the dimension of A the dimension of B as a vector
space. We do not care about the dimension of the emptyset.

2.2.1

Stability by the sum operation

1. Let C be a convex subset of E. Prove that 2C = C + C.

2. Let us consider in R2 , the set A = {(x, y) R2 | xy = 0}, draw A,
prove that 2A = A, A + A = R2 , and deduce that 2A =
6 A + A.

Stability by intersection

2.3

Note that the union of two convex sets is not convex.

Proposition 2.3 Let E be a vector space and let (Ci )iI be a family of
convex subsets of E. Then iI Ci is convex.

Stability with respect to affine functions

Definition 2.6 Let A and B be two affine subspace, and f : A B.

The mapping f is affine if for all couple of points (x, y) of A, and for all
t R,
f (tx + (1 t)y) = tf (x) + (1 t)f (y).

Exercise 2.9 (**) Let (Ci )iN (respectively (Di )i ) be a family of convex
subsets of some vector space E.
1) If for all integer i, Ci Ci+1 , then iN Ci is convex.
S
T
j
2) Show that
k=0
j=k D is a convex set.

Exercise 2.12 (**) Let A and B be two affine spaces with respective
directions E and F . Let f A B and a A.
9

1. Prove that f is an affine mapping if and only if there exists a linear

mapping : E F such that for all x of A, f (x) = f (a) + (x a).

(i). The convex hull of A, denoted by co(A), is the intersection of all

convex subsets of E containing A.

2. Show moreover that does not depend on the choice of a.

(ii). The affine hull of A, denoted by Aff(A), is the intersection of all affine
subspaces of E containing A.

Consequently, is called the linear mapping associated to the

affine mapping f .

Remark 2.1 Note that it follows from the definition that co() = and
that Aff() = . Since E is convex (and affine), if A is nonempty, co(A)
(respectively Aff A) is well defined and nonempty. Since the set of convex subsets (respectively affine subsets) is stable by intersection, co(A)
(respectively Aff(A)) is the smallest (in the sense of inclusion) convex (respectively affine) subset containing A. this implies for example that if C
is a convex subset containing A, then co(A) C.

Proposition 2.5 Let E be a vector space.

(i). Let C be a convex subset of E and let R. Then C := {c | c
C} is convex in E.
(ii). Let f , be an affine mapping from E to some affine subspace A (contained in some vector space F ), and let C be a convex subset of E.
Then f (C) is a convex subset of F .

Proposition 2.8 Let A be a subset of some vector space E.

(i). The set co(A) is the set of all convex combinations of elements
from A.

The proof is left to the reader.

Proposition 2.6 Let f be an affine mapping from A to B. Let us assume
that the affine subspace A is contained in the vector space E while the
affine subspace B is contained in the vector space F . Let C be a convex
subset convex of F . Then f 1 (C) is a convex subset de E.

(ii). The set Aff(A) is the set of all affine combinations of elements
from A.
Proof of Proposition 2.8. (i) Let us denote by B (respectively D), the
set of all convex combinations of elements from A (respectively co(A)).
One has B D since A (co(A)). Moreover, we can apply Proposition
2.1 together with the convexity of co(A) (cf. the previous remark) in order
to get D = co(A). Consequently B co(A).
Let us now show that co(A) B. It is clear that A B. Hence, in
order to prove this inclusion, il suffices to prove that B is convex. Let x
and y, be two elements of B and t [0, 1]. By definition of B, there exists
two families (x1 . . . , xn ) and
(y1 , . . . , yp ) of elements
from A, S n1 and
P
P
S p1 such that x = ni=1 i xi and y = pj=1 j yj . One has

The proof is left to the reader.

Proposition 2.7 Let (Ci )iI be a finite familyQ
of sets such that each Ci
i
is aQconvex subset of the vector space E . Then iI Ci is a convex subset
of iI E i .
The proof is left to the reader.

2.4
2.4.1

Convex hull, affine hull and conic hull

Convex hull and affine hull

tx + (1 t)y =

Definition 2.7 Let A be a subset of some vector space E.

n
X
i=1

10

ti xi +

p
X
j=1

(1 t)j yj .

Then tx + (1 t)y is a convex combination of (x1 , . . . , xn , y1 . . . , yp ) since

n
X

ti +

i=1

p
X

Definition 2.9 Let E be a vector space which has a finite dimension, and
A be a nonempty subset, either finite or infinite. We call the dimension
of A the dimension of the affine subspace generated by A.

(1 t)j = t + (1 t) = 1

With this definition, it is easy to show that the unit simplex S n1 is

(n 1)-dimensional.

j=1

and therefore, (t1 , . . . , tn , (1 t)1 . . . , (1 t)p ) belongs to S n+p1 .

(ii) Similar to first part.

2.4.2

Cone and Conic hull

Definition 2.10 A subset K of E is a blunt cone of vertex 0 if for all

x K and for all t > 0, tx belongs to K. If in addition, it contains 0, the
set is a pointed cone of vertex 0.
By translation, we may define the notion of cone of vertex z.

Definition 2.8 Let us call (convex) polytope, the convex hull of a finite
set.
Example 2.1 Let us define the simplex (denoted by S n1 ) as the subset
of Rn which is the polytope generated by the elements of the canonic
basis. The affine subspace generated by Aff(S n1 ), is the hyperplane
{x Rn | x1 + . . . + xn = 1}.

Here, a cone will mean ici pointed cone of vertex 0.

Example, any vector subspace is a cone of vertex 0. If A is an affine
subspace and if a A, then A is a cone of vertex a. A cone can be
non-convex, K = {(x, y) R2 | xy = 0}. A bounded nonempty cone is
reduced to its vertex. A nonempty cone K (blunt or pointed) of vertex 0
satisfies 0 K.

Exercise 2.13 (**) Let U and V be two subsets of Rn . We will denote

by co U the closure of the convex set U .
Show that: co U + co V co(co U + V ); co U + V co(U + V ).
Prove then that co(U + V ) = co U + co V .
If co U is compact, show that co(U + V ) = co U + co V .

Proposition 2.9 Let K be a subset of E.

(i). K is a pointed cone of vertex 0 if and only if for all x K and for
all t 0, tx belongs to K.

Exercise 2.14 (*) Let A and B be convex subsetsof Rn , Let us define

D = co(A B) and E = {z = x + (1 )y | x A, y B, 0 1}.

(ii). A one K (pointed or blunt) is convex if and only if it is stable by

addition.

Show that E D.
Check that E is convex and contains A. Deduce that

Proof of Proposition 2.9. Assertion (i) is trivial.

(ii) Let K be a convex cone. Let x an y be two elements of K. Then
1
(x
+ y) belongs to K since it is convex and x + y can be written as
2
1
2( 2 (x + y)) which belongs to K since it is a cone. Consequently K is
stable by addition.
Let K be a cone stable by addition and let x and y be two elements
of K. One has to show that for all t ]0, 1[, tx + (1 t)y belongs to K.

co(A B) = {z = x + (1 )y | x A, y B, 0 1}.
More generally, prove that for a finite collection (Ai )pi=1 of convex
n
subsets
Pp
Sp of i R ,
i
)
=
{z
=
| i 0, xi Ai , i =
co( i=1 A
i=1 i x
Pp
1, . . . p,
i=1 i = 1}.
11

The vectors tx and (1 t)y are elements of K since it is a cone. Since K

is stable by addition, tx + (1 t)y belongs to K.

Examples: All space subspace of E is a convex cone of vertex 0. All
set of solutions of a linear system of homogeneous linear equations and
homogeneous linear inequations is a convex cone. The image of a convex
cone of vertex z by an affine mapping f is a convex cone which vertex is
equal to f (z). The inverse image of a convex cone of vertex 0 by a linear
mapping f is a convex cone which vertex is equal to 0.
Definition 2.11 Let A be a nonempty subset of E. The convex conic
hull of A is the intersection of all the convex pointed cone of vertex 0
containing A. It is denoted by K(A).
Since E is a convex cone, the set K(A) is well defined. It is easy to prove
that all intersection of convex cones is also a convex cone. Consequently,
K(A) is the smallest (in the sense of inclusion) convex cone containing
A. It is also obvious that K(A) is the set of all linear combinations with
non-negative coefficients of elements from A.

12

University Paris 1
M1QEM1

2008/2009
Mr. Gourdel

Chapter 3

The one dimensional results

3.1

Existence results

1. when x ]a, b[, one has f 0 (x) = 0,

2. when x = a, one has f 0 (x) 0,

The main starting point of existence results is Weierstrasss Theorem and

its corollaries.

3. when x = b, one has f 0 (x) 0.

Theorem 3.1 (Weierstrass) If the domain is compact, and if f is continuous then f is bounded and the extremum are reached.

Exercise 3.2 (*) Let us use the notations of Proposition 3.1.

The proof is based on the notion of maximizing sequence (respectively

minimizing). When the domain is not bounded, numerous problems can
be solved using the coercivity condition.

1. Prove Proposition 3.1.

Definition 3.1 The function f : R R is said coercive if

2. Let us fix a = . Prove that is the solution (even it is a global

solution) of maxx[,] ex . What is the value of the derivative f 0 (a) ?

lim f (x) = lim f (x) = +.

x+

3.2.2

Exercise 3.1 (*) Let us assume that f : R R is coercive and continuous, show that f has a lower bound and that there exists at least a
minimum.

3.2
3.2.1

Alternative formulation in terms of multipliers

Let us focus on the previous problem when the domain is bounded (a and
b finite). The problem maxx[a,b] f (x) can be formulated as

max f (x)
max f (x)
x b 0 i.e (P1 )
g1 (x) 0
(P)

ax0
g2 (x) 0

First order condition

Necessary condition

with the notations g1 (x) = x b and g2 (x) = a x. We can state

Proposition 3.1 under the following version.

Proposition 3.1 Let a < b , we let I = [a, b] R and we

consider f : I R differentiable on I, if we assume that the problem
maxxI f (x) has a local solution x R, then
13

Proposition 3.2 Let < a < b < , we let I = [a, b] and we consider
f : I R differentiable on I, we assume that the problem (P1 ) has a local
solution x, then there exist multipliers 1 0 and 2 0 such that

We say that the Slater condition is satisfied for this list of constraints if
there exists a point x such that x O, and for each i = 1, . . . , p,

x) 0
either gi is an affine function and gi (
or

gi is a convex function and gi (

x) < 0

f 0 (x) = 1 g10 (x) + 2 g20 (x)

and 1 g1 (x) = 2 g2 (x) = 0.

Note that in the previous definition, the point x is feasible.

Exercise 3.3 (*) Deduce Proposition 3.2 from Proposition 3.1.

Be careful, if we consider the optimization problem,

max x
(P)
g1 (x) = x2 0

Exercise 3.4 (*) Let f : [a, b] R, let us consider the optimization

problems

max f (x) R
max f (x)
xb0
(x b)3 0
(P1 )
(P2 )

ax0
ax0

(3.1)

It is easy to prove that x = 0 is the unique solution of (P) but nevertherless

the system
0
f (x) = 1 g10 (x)
1 0

1 g1 (x) = 0

1) Compare the two problems.

2) Show that the Slater condition is satisfied for (P1 ).
3) Show that the Slater condition is NOT satisfied for (P2 ).
Exercise 3.5 (*) Let us consider again the problem

max x
(P)
g1 (x) = x2 0

has no solution.
Even with a single variable, the optimality does not imply
the existence of multipliers, we need an additional condition
(qualification condition). Solving the previous system dealing
with the existence of multipliers at point x makes sense only if
this point is qualified.
We will now introduce a first version of the notion of qualification.
This is not the better one, though it allow to give an intuition on the
notion.

Show that the Slater qualification condition does not hold.

Proposition 3.3 Let x be a local

(P)

Definition 3.2 Let us consider the following set of feasible points, the
set O is open and gi are continuous on O.

solution of the optimization problem:

max f (x)
xO
g1 (x) 0
...
gp (x) 0

Let us assume that the Slater condition is satisfied and that each function
is differentiable on a neighborhood of x, then, there exists (i )pi=1 Rp+
such that
 0
f (x) = 1 g10 (x) + . . . + p gp0 (x)
i gi (x) = 0 for all i = 1, . . . , p

xO

g1 (x) 0
...

gp (x) 0
14

Proof. Let us define the (possibly empty) sets, K = {i {1, . . . , p} |

gi (x) = 0}, K+ = {i K | gi0 (x) > 0}, K0 = {i K | gi0 (x) = 0}and
K = {i K | gi0 (x) < 0}. Note that we only need to prove the existence
of (i )iK since when x
/ K, the complementary condition i gi (x) = 0
implies that i = 0. let us prove the existence of multipliers with respect
to the value of f 0 (x).

It is obvious that (i ) satisfies all the conditions.

3.3
3.3.1

If f 0 (x) = 0, it suffices to let in addition i = 0, for each i K.

Convex and concave functions

Definitions

In this part, f is a function defined on U convex subset (interval) of R to

R.

Let us now consider for example the case where f 0 (x) > 0, (a symmetric argument allows to treat the case where f 0 (x) < 0). We can introduce
the alternative problem

max f (x)
(P1 )
gi (x) 0, for all i K

Definition 3.3 The function f is convex (resp. concave) if for all (x, y)
U U and for all t [0, 1],
f (tx + (1 t)y) tf (x) + (1 t)f (y).

Using Exercise 1.4, we know that x is also a local solution of (P1 ).

(respectively) f (tx + (1 t)y) tf (x) + (1 t)f (y).

Step 1 Let us prove that K+ K0 is nonempty. First, one can notice that
there exists some > 0, such that for each x ]x, x + ], gk (x) < 0,
for each k K , and f (x) > f (x). The value of can be chosen such
that that x is a global solution of

max f (x)
(P2 ) gi (x) 0, for all i K

x [x , x + ]

The function f is strictly convex if for all (x, y) U U such that

x 6= y and for all t ]0, 1[, f (tx + (1 t)y) < tf (x) + (1 t)f (y).
A function f is convex if and only if f is concave. Consequently,
the results obtained for convex functions can be translated in terms of
concave functions. These notions can be defined locally.
Definition 3.4 The epigraph and the hypograph (denoted by epi(f ) and
hypo(f )) of a function f are the sets (see figure 3.1) defined by

Since f (x+) > f (x), there exists some k K such that gk (x+) > 0.
In view of the choice of , we already know that k
/ K .
Step 2 Let us know prove that K+ is nonenmpty. If K+ is empty, then k
K0 , and the convexity of gk implies that x is a global minimum for gk .
The Slater condition assumes that either gk is affine or gk (
x) < 0 =
gk (x). The second case is excluded and therefore gk is affine and more
precisely constant. This is not possible since gk (x + ) > 0 = gk (x).

epi(f ) = {(x, t) U R | t f (x)}.

hypo(f ) = {(x, t) U R | t f (x)}.
Theorem 3.2 The three following properties are equivalent:
(i) f is convex (resp. concave);
(ii) for all k 2, (xi ) P
U k and S k1 ,
P
f ( ki=1 i xi ) (resp. ) ki=1 i f (xi );
(iii) The epigraph (resp. the hypograph) of f is a convex subset of R2 .

Step 3 Since K is nonempty, we can choose some k K, and let

i = 0
if i 6= k
0
f (x)
k = 0
gk (x)
15

E(f )

3. If C is a nonempty convex convex subset of R, the distance to C

defined by dC (x) = inf{|x c| such that c C} is convex.

Cf

Proposition 3.4 (i) A finite sum of convex functions (resp. concave)

defined on U is convex (resp. concave);
(ii) if f is convex (resp. concave) and > 0, f is convex (resp.
concave);
(iii) The supremum (resp. infimum) of a family of convex functions
(resp. concave) defined on U is convex (resp. concave) on its domain
(when the supremum is finite);
(iv) If f is a convex function (resp. concave) from I to J, intervals of
R, and if is a convex function (resp. concave) non-decreasing from I to
R then f is convex (resp. concave).
(v) if g is an affine function from R to R and f a convex function on
U R, then f g is a convex function on g 1 (U ).

H(f )

Figure 3.1: graph, epigraph and hypograph

Proof of Theorem 3.2. We will only prove the result in the convex
case. It is obvious that (ii) implies (i). Let us now show that (i) implies
(iii). Let (x, ) and (y, ) two elements of epi(f ) and let t [0, 1]. It
means that f (x) and f (y) . Since f is convex, f (tx + (1 t)y)
tf (x) + (1 t)f (y). Then, f (tx + (1 t)y) t + (1 t). This is
equivalent to t(x, ) + (1 t)(y, ) = (tx + (1 t)y, t + (1 t)) belongs
to the epigraph of f . Therefore, this set is convex.
We will end the proof by showing that (iii) implies (ii). Let k 2,
(xi ) (Rn )k and S k . then, (xi , f (xi )) is an element of the epigraph
of f and since this set is convex,
k
X

The proof of this proposition is left to the reader.

Exercise 3.6 (*) Let f be the function defined by f (x) = |x| on R,
Show that f is convex on [0, +[ and on ], 0] but not on R.

k
k
X
X
i (xi , f (xi )) = (
i x i ,
i f (xi ))

i=1

i=1

i=1

3.3.2

is P
an element P
of epi(f ).
Then, by definition of the epigraph,
k
k
f ( i=1 i xi ) i=1 i f (xi ). 

quasi-convex functions

Definition 3.5 Let f be a real-valued function defined on an interval I,

we say that f is quasi-concave if for all R, the set {x I | f (x) }
is convex. We say that f is quasi-convex if the sets {x I | f (x) }
are convex.

Examples:
1. Let us recall a function R R, is an affine function if there exists
and such that for all x in R,

Exercise 3.7 (*) Let f be a real-valued function defined on an interval

U,
Show that f is quasi-convex if and only if for all x, y of U and all
[0, 1],
f (x + (1 )y) max(f (x), f (y))

f (x) = x + .
Any affine function is both convex and concave, but not strictly. It
can be easily proved that if f is convex and concave on U , then it is
the restriction on U of an affine function.

Proposition 3.5 Let f be a real-valued function defined on an interval,

1) if f is convex, then f is quasi-convex.
2) The function f is quasi-convex if and only if (f ) is quasi-concave.

2. |.| is a convex function.

16

3) if f is weakly monotone, then f is both quasi-concave and quasiconvex.

1. Show that if x is a local solution of minxU f (x), then it is also a

global solution.

For example, the exponential function is convex and quasi-concave.

2. If there exists a solution to the minimization problem, then it is

unique.

Exercise 3.8 (*) Show Proposition 3.5.

Exercise 3.13 (**) Let U be an open interval of R and f a function C 2

from U R. We assume that for all x U , f 0 (x) = 0 f 00 (x) > 0.

Exercise 3.9 (*) Let U be an open interval of R and f be a function C 1

from U to R. Show that f is quasi-convex if and only if

1. The goal of this question is to show by contradiction that f is strictly

quasi-convex :

x, U, y U, f (y) f (x) f 0 (x)(x y) 0.

Exercise 3.10 (*) Let f be the function defined by f (x) = |x| on R,
Show that f is quasi-convex on [0, +[ and on ], 0] but not on R.

a) Let x0 6= x1 be two elements of U , such that there exists x

]x0 , x1 [ satisfying f (x ) max(f (x0 ), f (x1 )). Show that the problem
maxu[x0 ,x1 ] f (u) has at least one solution z satisfying z ]x0 , x1 [.

There exists several notions of strict quasi-convexity, and one should

be cautious and check the definition used. Here we will use the following
definition.

b) Show that f 0 (z) = 0, and that f 00 (z) > 0. Deduce the contradiction.

Definition 3.6 Let f be a real-valued function defined on an interval U ,

We say that f is strictly quasi-convex if f is quasi-convex and if for all
x, y from U satisfying x 6= y and for all ]0, 1[,

2. Let x U , such that f 0 (x) = 0, show that x is the unique global

minimum (necessary and sufficient condition). (Hint, one may use
exercise 3.12).

f (x + (1 )y) < max(f (x), f (y))

3. Let g be a function U R C 1 , such that g 0 (x) 6= 0 for all x U ,

show that g is both strictly quasi-convex and strictly quasi-concave.

When the function is continuous, we can propose another characterization of strict convexity (see exercise 3.11).

Exercise 3.14 1) Show with a counter-example that the sum of two

quasi-convex functions is not necessarily quasi-convex.
2) Show with a counter-example that the sum of a quasi-convex function
and a convex function is not necessarily quasi-convex.
3)Show with a counter-example that the sum of a strictly quasi-convex
function and a strictly convex function is not necessarily quasi-convex.

Exercise 3.11 (**) Let f be a continuous real-valued function defined

on an interval U , we assume that for all x, y from U satisfying x 6= y,
f (x) = f (y) and for all ]0, 1[,
f (x + (1 )y) < f (x)
Show that f is strictly quasi-convex.

3.3.3

Exercise 3.12 (**) Let U be an open interval of R and f be a strictly

quasi-convex function:

We will give in this part important results on the continuity of convex

functions.
17

Regularity of convex functions

2) Let us recall that f is Lipschitz on X if for all (x, x0 ) X X,

|f (x) f (x0 )| k|x x0 |. It is easy to prove that a Lipschitz function is
continuous.
If f is bounded from below by m and from above by M , we can write
for all x < y in [a, b],

Proposition 3.6 if f is convex (resp. concave) and if x < x0 < x00 are
elements of I, then (figure 3.2),
f (x0 ) f (x)
f (x00 ) f (x)
f (x00 ) f (x0 )

x0 x
x00 x
x00 x0

f (x ) f (x)
f (x) f (y)
f (y) f (y + )

(x ) x
xy
y (y + )

.
b

Then

M m
f (x) f (y)
M m

xy



m |x y|.
This allows us to deduce that |f (x) f (y)| M

3) Easy deduction of 1) and of 2). 

Corollary 3.2 Let f be a convex function on some interval U ,

1) if x1 < x2 < x3 < x4 < x5 are elements of U , then

Figure 3.2: Monotonicity of the slope of a convex function

We can deduce from this that f is Lipschitz and even give an explicit
bound to the Lipschitz constant.

f (x1 ) f (x2 )
f (x2 ) f (x3 )
f (x3 ) f (x4 )
f (x4 ) f (x5 )

x1 x2
x2 x3
x3 x4
x4 x5

Corollary 3.1 1) Let , f a convex function on I = [, ], then f

is bounded on I.
2) Let a b, > 0 f be a convex function on I = [a , b + ], we
assume that on I, f is bounded from below by m and from above by M .
then f is k-Lipschitz on [a, b], for k = (M m)/ and therefore continuous
on [a, b].
3) Let f be a convex function on U , then f is locally lipschitz and
continuous on the interior of U .

2) The right and left derivatives exist on the interior of U and moreover
if x1 < x2 are interior points, then
fr0 (x1 )

f (x2 ) f (x1 )
f`0 (x2 ).
x2 x1

3) The right and left derivatives are non decreasing on the interior of
U , and more over if x is an interior point, f`0 (x1 ) fr0 (x1 ).

Proof : 1) Since f is convex, it is also quasi-convex. For all x [, ],

f (x) M = max (f (), f ()).
If we denote = ( + )/2 and m = 2f () M , we want to prove that
for all x [, ], f (x) m. Let us denote by y = 2 x, by definition
is the middle of {x, y}. The convexity of f allows us to deduce that
f () (f (x) + f (y))/2. Since f (x) 2f () f (y), then m is a lower
bound.

Remark 3.1 Be careful, f may be discontinuous on the boundary of the

domain. For example,

1 if x = 0
f (x) =
0 if x > 0
is convex on R+ .
18

Exercise 3.15 (***) Let f be a function defined on some interval I, and

let x be a point of I. Let us define the set

Conversely, we assume that the property on the derivative of f is

satisfied. Let (x, y) U U and let t ]0, 1[. We have

f (x) = { R | f (x) f (x) + (x x), for all x I}.

f (x) f (x + t(y x)) f 0 (x + t(y x))(t(y x))

and

1. Show that if x is an interior point of I, and if the right and the lef
derivatives exists at point x, then for all f (x), we have fr0 (x)
and fl0 (x).

f (y) f (x + t(y x)) f 0 (x + t(y x))((1 t)(y x)).

If we multiply the first inequality by (1 t), the second inequality by t,
and if we sum, then we get that (1 t)(f (x) f (x + t(y x))) + t(f (y)
f (x + t(y x))) (t(1 t) + t(1 t))f 0 (x + t(y x))(y x) = 0. Then
f (x + t(y x)) = f ((1 t)x + ty) (1 t)f (x) + tf (y), i.e. f is convex.

The next exercise propose an alternative proof of the converse implication.

2. We assume that f is convex. Deduce from Proposition 3.2 that if x

is an interior point of I, f (x) = [fl0 (x), fr0 (x)], in particular, that the
set is nonempty.
3. Let f : R R defined by f (x) = min(0, x), determine for all x in
R, the set f (x)
4. We assume that f is convex and that f (x) = {a}, show that f is
differentiable at point x and that f 0 (x) = a.

3.3.4

Exercise 3.16 Let f be a differentiable mapping defined on the convex

set U , we assume that for all (x, x0 ) U U , f (x) f (x0 ) f 0 (x0 )(x x0 ).
1) Show that
\
epi(f ) =
{(x, y) U R | y f (x0 ) + f 0 (x0 )(x x0 )}.

Characterization with derivatives

Let us end this part by studying the possible characterizations when the
first (respectively the second) derivative exists.

x0 U

2) Using the characterization given by Theorem 3.2, deduce that f is

convex.

Proposition 3.7 Let f be a differentiable function defined on some convex set U R. The function f is convex if and only if for all (x, y)
U U , f (y) f (x) f 0 (x)(y x).

The monotony of f 0 characterize entirely the convexity, this can be

formalized in the next proposition.

Proof of Proposition 3.7. Let us first consider the implication .

Let (x, y) U U . If x = y, there is nothing to prove. Let us assume
now that x 6= y, using mean value Theorem, there exists some c in ]x, y[
if x < y and in ]y, x[ if y < x such that

Proposition 3.8 Let f be a differentiable function on the interval U .

The function f is convex if and only if for all (x, y) U U , (f 0 (y)
f 0 (x))(y x) 0.
Proof of Proposition 3.8. The implication is a consequence of the
weak monotony of f 0 (see Corollary 3.2).

f (y) f (x)
= f 0 (c)
yx

For the converse implication, let us consider x and y two elements of

U . Using the proposition 3.7, it suffices to show that f (y) f (x) (x

The weak monotony of f (See Corollary 3.2) allows us to conclude.

19

y)f 0 (x) 0 in order to prove that f is convex. If x = y, there is nothing

to prove.
We will first consider the case x > y. Since y x < 0, we have
by assumption that for all t of ]y, x[, (f 0 (t) f 0 (x))(t x) 0, and
consequently
f 0 (t) f 0 (x) 0. If we integrate on [y, x] this inequality, we
Rx 0
get y (f (t) f 0 (x))dt 0, this means f (y) f (x) (x y)f 0 (x) 0,
The proof of the case x < y is similar. 

When the problem is convex, we can state:

Proposition 3.10 Let us consider the optimization problem :

max f (x)

xO
g1 (x) 0

...

gp (x) 0

Let us now consider the case where f is twice differentiable on U .

We assume that x is a feasible point for this system. If the function f

is globally concave, (respectively locally concave at point x), and if the
functions gi are globally convex, (respectively locally convex at point x)
and if there exists (i )pi=1 Rp+ such that
 0
f (x) = 1 g10 (x) + . . . + p gp0 (x)
i gi (x) = 0 for all i = 1, . . . , p.

Corollary 3.3 Let f be a twice differentiable function on the interval U ,

f is convex if and only if for all x U , f 00 (x) 0.
Proof of Proposition 3.3. Indeed, f is convex, if and only if f 0 is
weakly increasing, therefore if and only if f 00 0 on U . 
In order to get a result involving strict convexity, we may use the results
of the next exercise.

then, the condition is sufficient, x is a global solution (respectively local)

of the maximization problem.

Exercise 3.17 1) Show that the function f defined by f (x) = x4 is

strictly convex on R but nevertheless the second derivative is not always
positive on R since f 00 (0) = 0.
2) Let f be a twice differentiable function on the interval U . Show that if
for all x U , f 00 (x) > 0, then f is strictly convex on U .
3) Let f be a C 2 function on U a neighborhood of x, and assume that
f 00 (x) > 0. Show that there exists V a (possibly smaller) neighborhood of
x such that f is strictly convex on V .
4) Let f be a C 1 function on some interval U such that f 0 is increasing
(strictly). Prove that f is strictly convex on V .

3.4

Proof of the proposition

P 3.10. Let us call the Lagrangian of the
problem L(x) = f (x) pi=1 i gi (x). Since this function is globally concave, (respectively locally concave at point x) and L0 (x) = 0, we can write
that for all point x feasible (respectively feasible and in a neighborhood of
x), L(x) L(x). we can conclude if we note that for each feasible point
x, L(x) f (x) and that L(x) = f (x) (in view of the conditions on the
multipliers). .
Exercise 3.18 Let us consider the problem:

min x3
(P)
1 x 0

Sufficient conditions for optimality

1) Check that the Slater condition is satisfied, that the objective function
is strictly quasi-concave, that the constraint function is convexe.
2) Write the conditions for the existence of a multiplier at some feasible
point x and show that there exists two solutions: x = 0 associated to
the multiplier = 0 and x = 1 associated to the multiplier = 3.
3) Show that the point x = 0 is not a solution (even a local solution) of
our optimization problem.

When the problem is not convex, the only case where we can conclude is
the following:
Proposition 3.9 Let a < b , we let I = [a, b] R and we
consider f : I R differentiable at point a. If f 0 (a) > 0, a is a local
solution of minimization problem.
20

3.5

Introduction to sensibility analysis

0
f (x) 0
0
(S ) f 0 (x)(g(x) ) = 0

g(x) 0

Let us consider in this part an optimization problem with a single inequality constraint involving one real parameter

max f (x)
(P )
g(x)

One should note that the associated multiplier is unique. In order to

study the regularity of the value function, we will distinguish several case
depending on the sign of the multiplier.

We assume that the functions f and g are C 1 on R and that for all ,
there exists a unique solution x . We denote by () the value of (P ),
we want to study the behavior of the function . Note that it is obvious
that when increases, the set of feasible points is bigger or equal, which
implies that the function will be non-decreasing.

First case: g(h()) < .

In this case, the associated multiplier denoted by is equal to 0. We
have,
 0
f (x ) = 0
0.(g(x ) ) = 0
For all 0 close enough to , g(h()) < 0 and the couple (h(), 0) is
solution of (P 0 ), i.e. that h( 0 ) = h(). The solution is here constant.
Since ( 0 ) = f (x 0 ), we can deduce that 0 () = 0 = .

Theorem 3.3 Let us assume that the function f is strictly concave, g

convex, that for all x of R, g 0 (x) > 0, and that the limit of f in is
, then for all ,

Second case :g(x ) = and > 0.

f 0 (h()) > 0
We have, (S 0 )
g(h()) = 0
In view of the implicit function theorem, there exists > 0, and a
function : [ , + ] [h() , h() + ], such that for all 0
[ , + ], g(( 0 )) = 0 .
It is obvious that locally ( 0 ) is solution of (S0 0 ), this means that
locally h = , in particular h is C 1 in a neighborhood of . Since ( 0 ) =
f (h( 0 )), we can deduce that

1. there exists a unique solution denoted by h().

2. the function h is continuous at point and has a right and left derivative at each point.
3. the function is C 1 at point and 0 () = where is the unique
multiplier associated to the optimal solution h().
Proof of Theorem 3.3. According to the question of existence, since
g is increasing, there exists an inverse and the problem can be equivalently
stated as maxx],g1 ()] f (x). Since limx f (x) = , and since f
is continuous (in view of its concavity on R), this problems has a solution
which is unique since f is strictly concave.
Let us first remark that the assumption on g 0 implies that each point
of the domain will be qualified. Therefore x is solution of (P ) if and only
if there exists a multiplier . Then, (x, ) is solution of (S )

0 0
= f 0 (x)/g 0 (x) 0

0
f (x) = g (x)
(f 0 (x)/g 0 (x))(g(x) ) = 0
(S )

(g(x)

)
=
0

g(x) 0

g(x) 0

0 () = 0 ()f 0 (()) =

1
f 0 (h()) =
g (h())
0

Third case : g(x ) = and = 0.

We have,
 0
f (x ) = 0
g(x ) = 0
As in the first case, for all 0 > and close enough to , the couple
(h(), 0) is solution of (P 0 ), i.e. that h( 0 ) = h(). We can deduce
21

that h is continuous on the right, has a right derivative, at point ,

moreover the right derivative 0r () = 0 = .
As in the second case, for all 0 < close enough to , in view of
the implicit function theorem, there exists > 0, and a function
: [ , + ] [x , x + ], such that for all 0 [ , + ],
g(( 0 )) = 0 .
It remains to study the sign of the multiplier. Since f 0 (()) = 0,
is increasing, we can deduce that the strict concavity of f implies
that f 0 (( 0 )) 0 for all 0 < close enough to . Then ( 0 ) is
solution of (S0 0 ), i.e. h = . We can deduce that h is continuous on
the left, has a left derivative at point , moreover the left derivative
0` () = 0 ()f 0 (()) =

1
f 0 (x ) =
g (x )
0

In this theorem, h is not necessarily C 1 , while = f h is C 1 . .

Exercise 3.19 Let be a real parameter, we consider the problem

max x2
(P )
x
Show that
Sol(P ) = {0} if 0 and {} if 0
Deduce that in particular the solution is not differentiable at point 0 with
respect to .
Exercise 3.20 Let h defined by h(x) = g(x).
1) Compare the sets of solutions and the values of the problems


max f (x)
max f (x)
(P )
(Q )
g(x)
h(x)
2) Deduce, that in Theorem 3.3, the conclusion remains true if we replace
the condition for all x, g 0 (x) > 0 by for all x, g 0 (x) < 0.

22

University Paris 1
M1QEM1

2008/2009
Mr. Gourdel

Chapter 4

Finite dimensional optimization

4.1

Existence results

As in dimension 1, the main starting point of existence results is Weierstrasss Theorem and its corollaries.

(P)

Theorem 4.1 (Weierstrass) Let f be a function whose domain is compact, and if f is continuous then f is bounded and the extrema are reached.

min x + y

s.t. x 0
y0

(1 + x)(1 + y) 2

1. Draw the set of feasible points compact.

2. Prove that we can apply 4.1 to this problem.

The proof uses the notion of maximizing sequence (cf. Definition 1.5)
and is based on the following lemma.

Exercise 4.2 (*) Let us consider

max

s.t.
(P)

Lemma 4.1 Let us consider a maximizing sequence (xk )k . If the domain

is closed, the objective function f is continuous, then any cluster point of
(xk )k is a solution to the maximization problem.
When the domain is not bounded, numerous problems can be solved
using the coercivity condition.

the following optimization problem.

3x1 x2 x32
x1 0
x2 0
x1 2x2 = 5
2x1 + 5x2 20

1. Is the set of feasible points compact ?

Definition 4.1 The function f : A R is said coercive if

Proposition 4.1 Let us assume that f : A R is coercive and continuous, and that A is closed, then f has a lower bound and there exists at
least a minimum.

2. Prove that we can apply Proposition 4.1 to the modified problem

min 3x1 x2 + x32

s.t. x1 0
x2 0
(Q)

x1 2x2 = 5

2x1 + 5x2 20

Exercise 4.1 (*) Let us consider the following optimization problem.

3. conclude

lim

kxk+

f (x) = +.

23

4.2
4.2.1

Convex and quasi-convex functions

Definition 4.4 Let f be a real-valued function defined on a convex set

U , We say that f is strictly quasi-convex if f is quasi-convex and if for all
x, y from U satisfying x 6= y and for all ]0, 1[,

Definitions

In this part, f is a function defined on U convex subset of Rn with values

in R.

f (x + (1 )y) < max(f (x), f (y))

Exercise 4.4 (**)

Definition 4.2 The function f is convex (resp. concave) if for all (x, y)
U U and for all t [0, 1],

1. Let U be an open convex set on which the function f is assumed to

be quasi-convex, We assume moreover that f is continuous on the
closure of U . Show that f is quasi-convex on U .

f (tx + (1 t)y) tf (x) + (1 t)f (y).

(respectively) f (tx + (1 t)y) tf (x) + (1 t)f (y).

2. Let U = R2++ = {x R2 | xp
1 > 0 and x2 > 0}, let us consider f
4
defined on U by f (x1 , x2 ) = x1 x2 ). Is f strictly quasi-concave on
U ? on U ?

The function f is strictly convex if for all (x, y) U U such that

x 6= y and for all t ]0, 1[, f (tx + (1 t)y) < tf (x) + (1 t)f (y).

Remark 4.1 Let U be a convex subset of Rn . One has the equivalence

Exercise 4.3 (**)

The function f is convex (respectively strictly convex, quasi-convex,

strictly quasi convex) on U ,

1. Let U be an open convex set on which the function f is assumed to

be convex, We assume moreover that f is continuous on the closure
of U . Show that f is convex on U .
R2++

for all x 6= y in U , the function x,y defined from [0, 1] to R by,

2. Let U =
= {x R | p
x1 > 0 and x2 > 0}, let us consider f
defined on U by f (x1 , x2 ) = 4 x1 x2 ). Is f strictly concave on U ? on
U?

x,y (t) = f (ty + (1 t)x) = f (x + t(y x))

is convex (respectively strictly convex, quasi-convex, strictly quasi
convex) on [0, 1].

A function f is convex if and only if f is concave. Consequently,

the results obtained for convex functions can be translated in terms of
concave functions. These notions can be defined locally.

Note that x,y is differentiable (resp. twice differentiable) when f is differentiable (resp. twice differentiable) then, 0x,y (t) = hf (x+t(yx)), yxi
and 00x,y (t) = hHf (x + t(y x)))(y x), y xi.

Definition 4.3 Let f be a real-valued function defined on a convex set

U , we say that f is quasi-concave if for all R, the set {x U | f (x)
} is convex. We say that f is quasi-convex if for all R, the set
{x U | f (x) } is convex.

Definition 4.5 The epigraph and the hypograph (denoted by epi(f ) and
hypo(f )) of a real-valued function f are the sets (see figure 4.1) defined
by
epi(f ) = {(x, t) U R | t f (x)}.

There exist several notions of strict quasi-convexity, and one should

be cautious and check the definition used. Here we will use the following
definition.

hypo(f ) = {(x, t) U R | t f (x)}.

24



4. If f is a convex function (resp. concave) from U (convex set of Rn )

to I (interval of R), and if is a convex function (resp. concave)
non-decreasing from I to R then f is convex (resp. concave) on
U.

 

5. if g is an affine function from Rn to Rp and f a convex function on

U Rp , then f g is a convex function on g 1 (U ).

Figure 4.1: graph and epigraph

Theorem 4.2 The three following properties are equivalent:

The proof of this proposition is left to the reader.

1. f is convex (resp. concave);

k

k1

Exercise 4.5 Let U be a convex set of Rn and f be a convex function:

Show that if x is a local solution of minxU f (x), then it is also a global
solution.

2. forPall k 2, (xi ) U andP S , (unit simplex of R )

f ( ki=1 i xi ) (resp. ) ki=1 i f (xi );
3. The epigraph (resp. the hypograph) of f is a convex subset of Rn+1 .

Exercise 4.6 (*) Let f be a real-valued function defined on a convex

set U , show that f is quasi-convex if and only if for all x, y of U and all
[0, 1],
f (x + (1 )y) max(f (x), f (y))

Proof of Theorem 4.2. Simple adaptation of the proof of Theorem 3.2

Examples:
1. Any affine real-valued function is both convex and concave, but not
strictly. It can be easily proved that if f is convex and concave on U ,
then it is the restriction to U of an affine function, (still called affine
function by an abuse of language).

Exercise 4.7 (*) Let U be an open convex set of Rn and f be a strictly

quasi-convex function:

2. Each norm is a convex function.

1. Show that if x is a local solution of minxU f (x), then it is also a

global solution.

3. If C is a nonempty convex subset of Rn , the distance to C defined by

dC (x) = inf{kx ck such that c C} is convex.

2. If there exists a solution to the minimization problem, then it is

unique.

Proposition 4.2 1. A finite sum of convex functions (resp. concave)

defined on U is convex (resp. concave);

When the function is regular, we can propose characterizations of

(strict) quasi-convexity (see exercises 4.8 and 4.9).

2. if f is convex (resp. concave) and > 0, f is convex (resp. concave);

Exercise 4.8 (*) Let U be an open set of Rn and f be a function C 1 on

U . Show that f is quasi-convex if and only if

3. The supremum (resp. infimum) of a family of convex functions (resp.

concave) defined on U is convex (resp. concave) on its domain (when
the supremum is finite);

x, U, y U, f (y) f (x) hf (x), x yi 0.

25

Note that this property is false in an infinite dimension setting. For

example, if we consider E = R[X] (polynomial functions), embedded with
the norm
deg(P )
deg(P )
X
X
kP k =
|ak |, when P =
ak X k .

Exercise 4.9 (*) Let f be a continuous real-valued function defined on a

convex set U , we assume that for all x, y of U satisfying x 6= y, f (x) = f (y)
and for all ]0, 1[,
f (x + (1 )y) < f (x)

k=0

Show that f is strictly quasi-convex.

k=0

It is easy to see that the linear (and consequently convex) is NOT continuous, where (P ) = P 0 (0).

Exercise 4.10 (*) Let U be a convex subset of R and f be a quasiconvex function. We want to study minxU f (x).

4.2.3

1. Show that the set of solution to the minimization problem is convex.

Characterization of convexity with derivatives

Let us end this part by studying the possible characterizations when the
first (respectively the second) derivative exists.

2. Prove with a counter-example that a local solution is not necessarily

a global solution. Hint: consider f : R R, defined by

x 1 if x 1,
0
if x [1, 1],
f (x) =

x1
if x 1.

Proposition 4.4 Let f be a differentiable function defined on some convex set U Rn . The function f is convex if and only if for all
(x, y) U U ,
f (y) f (x) + hf (x), y xi.

Exercise 4.11 (**) Let U be an open convex set of Rn and f a function

C 2 from U R. We assume that for all x U , and all v Rn \ {0},
hf (x), vi = 0 v t Hf (x)v > 0. Show that f is strictly quasi-convex.
Hint: use Exercise 3.13 and Remark 4.1.

We will give in this part important results on the continuity of convex

functions.

Proof of Proposition 4.4.

Let us first consider the implication . Let (x, y) U U and x 6= y.
In view of Remark 4.1, we know that the function x,y is convex. Since this
function has a derivative, we know that x,y (1) x,y (0) 0x,y (0)(1 0)
which leads to the conclusion.
Conversely, we assume that the property on the gradient of f is satisfied. Let x0 6= x1 in U , let us consider the function x0 ,x1 . In view of
Proposition 3.7, it suffices to prove that for all t 6= t0 (in [0, 1]),

Proposition 4.3 Let U be a convex set, > 0 and f be a convex function

on V = U + B(0, ),

x0 ,x1 (t0 ) x0 ,x1 (t) 0x0 ,x1 (t)(t0 t)

4.2.2

Regularity of convex functions

in order to prove that x0 ,x1 is convex. If we denote by x = tx1 + (1 t)x0

and y = t0 x1 + (1 t0 )x0 , the left side of the previous inequality can be
interpreted as f (y) f (x), while the right side is equal to hf (x), y xi.
Since x0 ,x1 is convex, it follows from Remark 4.1 that f is convex. 
The next exercise propose an alternative proof of the converse implication.

1. We assume that on V , f is bounded from below by m and from above

by M . then f is k-Lipschitz on U , for k = (M m)/ and therefore
continuous on U .
2. Let f be a convex function on U , then f is locally lipschitz and continuous on the interior of U .
26

Exercise 4.12 (**) Let f be a differentiable mapping defined on the

convex set U , we assume that for all (x, x0 ) U U , f (x) f (x0 )
hf (x0 ), x x0 i.

For the converse implication, let us consider x and y two elements of

U . We know that for all t of ]0, 1[,
hf (ty + (1 t)x) f (x), (ty + (1 t)x) xi 0,

1. Show that
epi(f ) =

and consequently hf (ty + (1 t)x), y xi hf (x), y xi. If we

integrate on [0, 1] this inequality, we get
Z 1
(hf (ty + (1 t)x), y xidt hf (x), y xi,

{(x, y) U R | y f (x0 ) + hf (x0 ), x x0 i}.

x0 U

2. Using the characterization given by Theorem 4.2, deduce that f is

convex.

this means f (y) f (x) hf (x), x yi 0. Using Proposition 4.4, we

can deduce that f is convex. 

Exercise 4.13 (***) Let f be a function defined on some convex set U ,

and let x be an interior point. Let us define the set (cf. Exercise 3.15)

Let us now consider the case where f is twice differentiable on U . We

will denote by Hf (x) the hessian matrix at point x, we recall that this
matrix is symmetric when f is C 2 .

f (x) = { Rn | f (x) f (x) + h, x xi, for all x U }

In order to simplify the notations, it may be easier to consider the case
x = 0.

Definition 4.6 Let M be a symmetric matrix (n, n).

We say that M is positive semidefinite (respectively negative semidefinite) if for all v Rn , hv, M vi 0 (respectively hv, M vi 0).

1. Apply Proposition 4.4 in order to show that if f is convex and differentiable at point x, then f (x) f (x).

We say that M is positive definite (respectively negative definite) if

for all v Rn \ {0}, hv, M vi > 0 (respectively hv, M vi < 0).

2. If f is differentiable at point x, write the Taylor expansion of degree

1 at a neighborhood of x, and deduce that f (x) = f (x).

Proposition 4.6 Let M be a symmetric matrix (n, n).

3. We assume that f is convex and that f (x) = {}, show that f is

differentiable at point x and that f (x) = .

the matrix M has n eigenvalues (possibly equal),

the matrix M is positive semidefinite (respectively negative semidefinite) if and only if all eigenvalues are non-negative, (respectively
non-positive),

Proposition 4.5 Let f be a differentiable function on some convex set

U . The function f is convex if and only if for all (x, y) U U ,
hf (y) f (x), y xi 0.

the matrix M is positive definite (respectively negative definite) if and

only if all eigenvalues are positive, (respectively negative).

Proof of Proposition 4.5.

The implication is a consequence of
Remark 4.1. Indeed, we know that the function x,y is convex, which
implies 0x,y (1) 0x,y (0) 0. This leads to the conclusion.

Exercise 4.14 (*) Let M be a symmetric (n, n)matrix, then

if M is positive semidefinite, then det M 0 and T r(M ) 0.
27

 if M is positive definite, then det M > 0 and T r(M ) > 0.

2) Let M be the following matrix,

if M is negative semidefinite, then T r(M ) 0 and det M 0 when

n is odd, det M 0 when n is even.

3 0 0
M = 0 0 0
0 0 1

if M is negative definite, then T r(M ) < 0 and det M < 0 when n is

odd, det M > 0 when n is even.

Prove that for all k = 1, . . . , 3, k 0, but that M is not positive

semidefinite.

Exercise 4.15 (*) When the dimension is equal

we can reenforce
 to 2, 
r s
the conclusion of the previous exercise. Let M =
be a symmetric
s t
(2, 2)matrix, then

Proposition 4.8 Let f be a twice continuously differentiable function on

U , which is a convex and open subset of Rn . Then, f is convex if and
only if for all x U , Hf (x) is positive semidefinite.

the matrix M is positive semidefinite if and only if det M 0 and

T r(M ) 0.

Proof of Proposition 4.8.

Indeed, if f is convex, then for any x
n
in U and v in R , there exists > 0, such that y = x + v U . We
can consider the function x,y which is convex. We can deduce that
hv, Hf (x)vi00x,y (0) 0. This implies that Hf (x) is positive semidefinite.
Conversely, let us assume that at each point of U , Hf (x) is positive
semidefinite. Then the computation of 00x,y (cf. Remark 4.1) allows us to
deduce that x,y is convex. 
A simple adaptation of the previous proof allows us to deduce the
following proposition:

the matrix M is positive definite if and only if det M > 0 and

T r(M ) > 0.
Exercise 4.16 (*) Let M be the following matrix,

1 0 0
M = 0 1 0
0
0 3
Show that det M > 0 and T r(M ) > 0, and that M is not positive semidefinite.

Proposition 4.9 Let f be a twice continuously differentiable function on

some convex set U ,
1) If for all x U , Hf (x) is positive definite, then f is strictly convex.
2) If at point x, the hessian matrix is positive definite, then f is strictly
convex in some neighborhood of x.

Proposition 4.7 Let M be a symmetric matrix, then M is positive definite if and only if for all k = 1, . . . , n, k > 0, where


 a11 . . . a1k 


 .. . .

.
.
k =  .
. . 


 ak1 . . . akk 

4.3

Exercise 4.17 (*) 1) Let M be a symmetric matrix, such that M is

positive semidefinite, then for all k = 1, . . . , n, k 0, where


 a11 . . . a1k 


 .. . .

.
.
k =  .
. . 


 ak1 . . . akk 

4.3.1

Unconstrained optimization
Classical case

The problem maxxU f (x) is said unconstrained if the set U is an open

subset of Rn .
28

Remark 4.2 Let us consider the unconstrained problem minxU f (x) and
x U . The point x is a local solution of this problem if and only if it is
an interior point of the set {x U | f (x) f (x)}.

If f is twice differentiable at point x, then has a second derivative and

00 (0) 0. In view of the chain rule theorem, 00 (t) = hu, Hf (x + tu)(u)i.
This leads to hu, Hf (x)(u)i = 0. Since this is true for any u Rn , we can
conclude that Hf (x) is positive semidefinite.

Remark 4.3 The previous remark can only be used for unconstrained
optmization: for example, x = (0, 0) is a local solution of the problem
minxU f (x) (where U = R2+ NOT open and f (x) = x21 + x22 ), but the
corresponding set {x U | f (x) f (x)} is reduced to {x} and the
interiory property does not hold.
Proposition 4.10 Let
minxU f (x).

us

consider

the

unconstrained

Note that some results of unconstrained optimization can be used even

if the domain is not open (cf. Exercise 1.5).
Remark 4.4 Let x is a local solution of the problem minxX f (x). When
X is a neighborhood of x, there exists some V open subset of C containing
x such that x is a local solution of the problem minxV f (x). This allows
to apply the first and second order necessary conditions.

problem

If x is a local solution, and if f is differentiable at point x, then

f (x) = 0, (critical point)

Exercise 4.18 We can reenforce the third result of the previous proposition. For all k strictly smaller than the smallest eigenvalue of the Hessian
matrix Hf (x) (in particular, k can be chosen positive), there exists some
> 0 such that for all x B(
x, r), f (x) f (
x) + k2 kx xk2 .

If x is a local solution, and if f is twice continuously differentiable at

point x, then Hf (x) is positive semidefinite,
If x is a critical point, if f is twice continuously differentiable at
point x, and if Hf (x) is positive definite, then x is a local solution.
In addition, there exists some > 0, such that on B(x, ) U , x is
the unique solution.

4.3.2

Extension to affine constraints

Let us consider the following problem


min f (x)
x U, Ax = b

If x is a critical point and if f is convex, then x is a global solution.

Proof of Proposition 4.10
It suffices to remark that if x is a local solution, then for any direction
u Rn , there exists some positive real number > 0, such that the
function
: ], [ R
(t) = f (x + tu)

The set of feasible points C is {x U | Ax = b}, where A is a (n p)matrix and b Rp . This correspond to the case of p affine constraints
represented by the p rows of A and the vector b. If we denote by aj the
vector de Rn corresponding to the j-row of A, the set C is defined by:
x C if and only if haj , xi bj = 0 for all j = 1, . . . , p.

is defined, and that 0 is a (global) minimum.

If we denote by gj (x) = haj , xi bj (which are affine functions), and

J = {1, . . . , p}, the problem can be written as

min f (x)
gj (x) = 0, j J

xU

If f is differentiable at point x, then has a derivative and 0 (0) = 0.

In view of the chain rule theorem, 0 (t) = hf (x + tu), ui. This leads to
hf (x), ui 0. Since this is true for any u Rn , we can conclude that
f (x) = 0.
29

Note that C can be written as U 1 (b) where is the linear mapping

defined by x (x) = Ax. Let x be a local solution of the problem:

min f (x)
Ax = b

xU

Since aj = gj (x), this means that

f (
x) =

4.4

Since U is open et x U , there exists some > 0 such that for all
t ], [, x + tu U . Once again, we can consider

4.4.1

Karush-Kuhn-Tucker
Necessary condition

Let us consider the domain defined by the list of constraints

fi (x) = 0, i I
gj (x) 0, j J

xU

u : ], [ R
u (t) = f (x + tu)
Consequently, for all t ], [, u (t) := f (x+tu) u (0) = f (x), which
means that 0 is a minimum de u . This implies that 0u (0) = 0. We can
compute 0u (0) = f (
x) u = 0.
Let (u1 , . . . uk ) be a basis of the kernel
Note that when U is equal to Rn , we can entirely describe C which is
equal to {x} + Ker . This implies that 0 is a solution of

min f (x + 1 u1 + . . . + k uk )
1 R, . . . , k R

Among the inequality constraints, we will distinguish those that correspond to affine function. This can be formulated as J is equal to the
partition Ja Jna , where gj is affine when j Ja . The set U is open which
may hide strict inequalities.
Definition 4.7 We say that the Slater condition is satisfied if
all fi are affine,

In the general case, C is equal to U ({x} + Ker ). This implies that

there exists > 0, such that 0 is a solution of

min h()
1 R, . . . , k R

B(0, )

all gj are convex

there exists x be a feasible point of the previous system, satisfying
moreover gj (x) < 0, for all j Jna .
Remark 4.5 When all the constraints are affine, then the Slaters condition is equivalent to to the existence of a feasible point.

where h() = f (x + 1 u1 + . . . + k uk ).
This leads to h(0) = 0, and we can formulate this result as f (
x)
t
is orthogonal to Ker . It follows that f (x) is in the image of , the
transposed of de . We recall that the matrix of t is the matrix where
the rows are the columns of A, the vectors aj . We have the result, there
exists some vector in Rp (vector of Lagrange multipliers) such that:
f (
x) =

j gj (x)

j=1

For all u Ker , note that for all t R, A(

x + tu) = b.

p
X

p
X

Theorem 4.3 (Karush-Kuhn-Tucker) Let us consider the optimization problem:

min f (x)

fi (x) = 0, i I
gj (x) 0, j J

xU

j aj

j=1

30

2. Prove that for any p 1, for any Rp+ , Rp+ ,

where U is an open set. Let x be a local solution of this problem. We

assume that there exists V open neighborhood of x such that f , fi , gj are
continuously differentiable on V . If the Slater condition is satisfied then
there exists RI and RJ+ such that
f (x) +

i fi (x) +

iI

j gj (x) = 0

p
X

3. Prove that the Karush-Kuhn-Tucker system is equivalent to there

exists (x, , ) U RI RJ+ such that

fi (x) = 0, i I

g (x) 0, j J
j
P
(x) = 0,

jJ j gjP

f (x) + iI i fi (x) + jJ j gj (x) = 0

(4.1)

jJ

and j gj (x) = 0, for all j J.

Note that if we distinguish among the inequalities, the set of binding
constraints at point x by J(x) = {j J | gj (x)}, then the condition 4.1
can be rewritten as,
f (x) +

X
iI

i fi (x) +

j gj (x) = 0.

4.4.2

(4.2)

Definition 4.8 We will associate to the previous minimization problem,

the following system Karush-Kuhn-Tucker conditions

fi (x) = 0, i I

gj (x) 0, j J

i R, i I
j 0, j J

j gj (x) = 0, for all j J

P
P

f (x) + iI i fi (x) + jJ j gj (x) = 0

xU

is convex if the functions hj are affine, the functions gi are convex, and if
f is convex 1 .
When the problem is convex, we can state:
Proposition 4.11 Let
us consider the optimization problem:
min f (x)

g1 (x) 0
...

gp (x) 0

xU

1. Prove that for all z R2 ,

z1 0,
z2 0,

z1 + z2 = 0,

Sufficient conditions for optimality

Definition 4.9 We say that the problem

min f (x)

x U,

h1 (x) = 0

...
hq (x) = 0

g1 (x) 0

...

gp (x) 0

jJ(x)

Note that when j

/ J(x), in view of the complementary condition
(j gj (x) = 0), we can deduce that the multiplier j is equal to zero.

Exercise 4.19

i i = 0 i i = 0, for all i = 1, . . . , p

i=1

z1 = 0,
z2 = 0.

31

for a maximization problem, the objective function is concave.

Let us assume that x be a feasible point and that U is an open set.

If the function f is convex on U (globally convex), (respectively locally
convex at point x), and if the functions gi are convex on U , (respectively
locally convex at point x) and if there exists (i )pi=1 Rp+ such that

f (x) + 1 g1 (x) + . . . + p gp (x) = 0
i gi (x) = 0 for all i = 1, . . . , p.

then, the condition is sufficient, (respectively sufficient) x is a solution

(repectively local solution) of the minimization problem.
We can extend in the next exercise Corollary 4.1 by assuming that
the objective function is only quasi-concave but only if the point is not a
critical point (cf. Exercise 3.18).
Exercise 4.20 Let us consider (x, , ) solution of Karush-Kuhn-Tucker
system (Definition 4.8) associated to the minimization problem

min f (x)

fi (x) = 0, i I
gj (x) 0, j J

xU

then, the condition is sufficient, x is a global solution (respectively local)

of the minimization problem.
Proof of the proposition 4.11. In order to simplify the notation, we
will focus on the global case. Let us call the Lagrangian of the problem
L(x, ) = f (x) +

p
X

i gi (x).

We assume in addition that the objective function is quasi-concave on

U and continuous on U , the functions fi are affine, and for all j J,
the functions gj are convex on U , The goal of the exercise is to show by
contradiction that if f (x) 6= 0, then x is a solution of the minimization
problem. With no loss of generality, we may assume that x = 0.

i=1

We can consider the partial function H(x) = L(x, ), The function H

is convex on U , and x is a critical point of H. So for each x U (feasible
or not), H(x) H(x).

1. Let us assume that there exists some feasible point y such that f (y) <
f (0), prove the existence of some > 0 such that for all z B(y, ),
f (z) < f (x).

We can conclude, if in addition we notice that for each feasible point

x, gi (x) 0 i gi (x) 0 L(x, ) f (x) and that L(x, ) = f (x) (in
view of the complementary relations). 

2. Let us fix some z B(y, ). Show that the function := 0,z is

quasi-concave and consequently 0 (0) 0.
P
P
3. Deduce from f (x) + iI i fi (x) + jJ(x) jgj (x) = 0 that
0 (0) 0.

Corollary 4.1 Let us consider the optimization problem:

min f (x)

xU
fi (x) = 0, i I

gj (x) 0, j J

4. Compute 0 (0) and deduce that f (0) B(y, ). Conclude.

Let us assume that x be a feasible point and that U is an open set. If the
function f is (respectively locally at point x) convex, the functions fi are
affine, if for all j J(x) functions gj are convex on U (respectively locally
J(x)
at point x), and if there exists () RI and R+ such that
P
P

f (x) iI i fi (x) + jJ(x) jgj (x) = 0
j gj (x) = 0 for all j J.

4.5
4.5.1

Polarity and orthogonality

Separation theorems

Theorem 4.4 Let A and B be two nonempty disjoint convex subsets of

Rn . If A is compact and B is closed, then there exists y E, y 6= 0 such
32

3. A = A := (((A ) ) )

that:
sup{y a | a A} < inf{y b | b B}

Exercise 4.23 Let A be a nonempty subset of Rn

Theorem 4.5 Let A and B be two nonempty disjoint convex subsets of

Rn . Then there exists a hyperplane separating A and B.
y E, y 6= 0 such that

1. If A is a cone subspace, then y belongs to A if and only if a y a

is bounded from above on A.
2. If A is a linear subspace, then A = A ; y belongs to A if and only
if a y a is bounded from above on A. A vector y belongs to A
if and only if a y a is bounded from below on A.

sup{y a | a A} inf{y b | b B}

4.5.2

definitions

Theorem 4.6 (Bipolar Theorem) If A be a nonempty subset of Rn ,

then, (A ) = cl(K(A)) and (A ) = vect(A), where vect(A) is the linear
subspace spanned by A.

Definition 4.10 Let A be a subset of Rn , the polar cone of A, denoted

by A is defined by
A = {y E | for all a A, y a 0}

Corollary 4.2 Let A be a nonempty subset of Rn . The set A is a closed

convex cone if and only if A = (A ) . The set A is a linear subspace if
and only if A = (A ) .

The orthogonal of A, denoted by A is defined by

A = {y E | for all a A, y a = 0}

Corollary 4.3 Let M be a linear subspace of Rn , M = E if and only if

M = {0}.

Proposition 4.12 Let A be a subset of Rn

1. A is a closed convex cone (of vertex 0),

Exercise 4.24 Let M be a convex set of Rn . M is a neighborhood of 0

if and only if M = {0}.

2. A is a linear subspace,
3. A = A (A) ;

4.5.3

4. If A1 A2 , then A2 A1 and A
2 A1 ;

Farkas lemma

Theorem 4.7 (Farkas lemma) Let (ai )iI be a finite family of elements in Rn . Let
X
A={
i ai | RI+ }

5. If A 6= , A = (cl(K(A))) and A = (vect(A)) where vect(A) is

the linear subspace spanned by A.

iI

Exercise 4.21 (*) Prove Proposition 4.12.

and
Exercise 4.22 Let A be a nonempty subset of R

B = {y E | y ai 0, for all i I}
one has A = B and B = A.

1. A A := (A )
2. A = A := ((A ) )

The key argument is Exercise 4.26.

33

Exercise 4.27 Let (ai )i=1,...,p be a finite family of elements in Rn . Let

b Rn satisfying the following property:

Exercise 4.25 (**) Let (a1 , . . . , ap ) be a finite subset of E = Rn .

1) Show that co(a1 , . . . , ap ) is a compact set.
2) Extend this result if E is a normed vector space.

For all y Rn

Exercise 4.26 (***) Let (a1 , . . . , ap ) be a finite subset of E = Rn .

The goal of this exercise is
on p that the set
Pto show by induction
p
K(a1 , . . . , ap ) which denotes { pi=1 i ai | R+
} is closed.
A) 1) show that if ap+1 K(a1 , . . . , ap ) then

P
Prove (using the hint) that there exists RI+ such that b = iI i ai .
Hint: Prove that 0 is a solution of the following optimization problem
and write the Karush-Kuhn-Tucker conditions.

max y b y a1 0
...

y ap 0

p+1

K(a1 , . . . , ap+1 ) =

K({aj | j 6= i}).

i=1

2) show that if M and N are two closed convex cones of E, such that
M (N ) = {0} then M + N is closed.

4.6

B) 1) Show that the result is true when p = 1.

2) Write the proof (one will have to distinguish wether ap+1 belongs
or not to K(a1 , . . . , ap )).

Tangent and normal cones

Let us now introduce two definitions.

Definition 4.11 Let C be a convex set containing c. The tangent cone
to the set C at point c denoted by TC (c) is

Corollary 4.4 (Second version of Farkas lemma) Let (ai )iI and
(bj )jJ be finite families of elements in Rn . Let
X
X
A={
i ai +
j bj | RI+ , RJ }
iI

y a1 0
...
y b 0.

y ap 0

TC (c) = cl{t(c0 c) | t 0, c0 C}
The normal cone to the set C at point c denoted by NC (c) is:

jJ

NC (c) = {y E | y c y c0 c0 C}.

et
B = {y E | y ai 0, for all i I, y bj = 0, for all j J}

Exercise 4.28 (*) (i) Let C be a convex set containing c, and D denotes
is translation D := C + {c}. Show that TC (c) = TD (0) and NC (c) =
ND (0).
(ii) Let c A B, then TA (c) TB (c) and NB (c) NA (c).

one has A = B and B = A.

Corollary 4.5 Let (ai )iI be a finite family of elements in Rn . Let
X
A={
i ai | RI }

Proposition 4.13 (i) For all c C, TC (c) and NC (c) are nonempty,
closed and convex cones.
(ii) TC (c) = (NC (c)) and NC (c) = (TC (c)) .

iI

and
B = {y E | y ai = 0, pour tout i I}

It is very important to emphasize that these notions are local notions,

cf. next exercise.

One has, A = B and B = A.

34

Exercise 4.29 (*) Let C1 and C2 be convex sets containing c, if there

exists > 0 such that C1 B(c, ) = C2 B(c, ) then NC1 (c) = NC2 (c)
and TC1 (c) = TC2 (c).
Exercise 4.30 (*) Let us consider min x2 + y 2 under the constraint 2x +
y 4.
1) Is the Slater condition satisfied for this problem ?
2) Write the Karush-Kuhn-Tucker conditions of this problem.
3) Solve the Karush-Kuhn-Tucker system.
4) Discuss wether the conditions are necessary/sufficient in order to
solve the optimization problem.
Exercise 4.31 (**)
let us consider the following optimization problems:

max 3x1 x2 x32

s.c. x1 0
x2 0
(P)

x1 2x2 = 5

2x1 + 5x2 20

max 3x1 x2 x32

s.c. x1 > 0
x2 > 0
(Q)

x1 2x2 = 5

20 2x1 5x2 0

1) Draw the set of feasible points for (P) and comment.

2) Prove that x is feasible for (P) if and only if x is feasible for (Q).
Deduce that the two problems are equivalent.
3) Prove that Sol(P) is nonempty.
4) Write the Karush-Kuhn-Tucker conditions.
5) Solve the Karush-Kun-Tucker conditions.
6) Solve the optimization problem (P)

35

University Paris 1
M1QEM1

2008/2009
Mr. Gourdel

Chapter 5

Linear Programming
5.1

A linear programming problem consists in finding the maximum (resp.

minimum) value of a linear functional, subject to a finite number of linear
constraints. If c and ai , i = 1, . . . , m are elements of Rn , and if b =
(b1 , . . . , bm ) belongs to Rm , the most general form of a linear programming
problem is the following:

5.1.1

The main results

existence

Note that in general, we can not use Weierstrass theorem neither a coercivity property of the objective function.

Maximize (resp. minimize)

f (x) = c x

Proposition 5.1 Given a linear programming problem (P ) (maximization), one of the three following alternatives holds:
- either there is no feasible point for Problem (P ), the value is equal
to , Sol(P ) = .
- either the objective function is not bounded from above on the
nonempty set of feasible points, the value is equal to +, Sol(P ) = .
- either the objective function is not bounded from above on the
nonempty set of feasible points, the value is finite and the set of solutions
is nonempty.

subject to the conditions

ai x = bi , i = 1, . . . , m

x Rn .

The linear functional f (x) is called the objective function. The linear
equations and inequations are called constraints of the problem. The
set of points that satisfy these linear equations or inequations is called
feasible set. An element of this set is a feasible solution; it is an optimal
solution if it solves the maximization (resp. minimization) problem. Let
(P ) and (P 0 ) be two optimization problems, we recall that (P ) and (P 0 )
are equivalent if their sets of solutions are equal.
Note that in general, we can not use Weierstrass theorem neither a
coercivity property of the objective function.

5.1.2

Necessary and sufficient conditions of optimality

Let f k (resp. f 0k ) k = 1, . . . , q, hk (resp. h0k ) k = 1, . . . , r, g (resp.

g 0 ) be linear functional on Rn , k (resp. k0 ) k = 1, . . . , q, k (resp.
k0 ) k = 1, . . . , r be real numbers. We now consider linear programming
36

Theorem 5.1 (i) If (x, ) is a saddle-point of L, then x is a solution to

(P ) and () are the Karush-Kuhn-Tucker coefficients associated with x.
Moreover, L(x, ) = f (x).
(ii) If (x, ) satisfies Karush-Kuhn-Tucker Conditions for Problem (P ),
then (x, ) is a saddle-point of L.

problems expressed in the following form:

Minimize g(x) subject to
f k (x) = k , k = 1, . . . , q
hk (x) k , k = 1, . . . , r
x Rn .

(I)

In this case, the dual problem can be defined as

Proposition 5.2 For a feasible solution x to be an optimal solution of
Problem (I), it is both necessary and sufficient that there exist some KKT
multipliers 1 , . . . , q R and 1 , . . . , r R+ .

(D)

Note that the Slater condition is satisfied if and only if there exists a
feasible point.

where
G() = infn L(x, )

Corollary 5.1 For a feasible solution x to be an optimal solution of Problem (I), it is both necessary and sufficient
that there
Pq
Pr existk 1 , . . . , q R
k
and 1 , . . . , r R+ such that g = k=1 k f + k=1 k h , with k = 0 if
hk (x) > k .

xR

Note that G() = L(x, ) = f (x) = val(P )

5.3
5.2
(P )

Saddle point properties

5.3.1

min f (x) subject to

fi (x) 0, i I
x Rn .

L(x, ) = f (x) +

The duality theorem of linear programming

Canonical form of the duality theorem

Let A be a (m n)-matrix, b Rm , c Rn . Let us consider the following

linear programming problem (P ), here referred to as the primal problem,

The Lagrangian of minimization Problem (P ) is the function L : Rn

RI+ R defined by for all (x, ) = (x, (i )iI ),
X

max G() subject to

RI+ .

(P )

i fi (x).

Minimize c x subject to
Ax b
x0
x Rn .

iI

and define its dual problem (D)

We say that (x, ) is a saddle-point of L if for all RI+ , x Rn ,
(D)
L(x, ) L(x, ) L(x, ).
37

Maximize b p subject to
t
pA t c
p0
p Rm .

Writing down the dual of the dual leads to the given primal problem.
For this reason, the linear programming problems (P ) and (D) are referred
to as primal problem and dual problem in canonical form. Each one of both
problems is said to be the dual of the other one.

1) Write the Karush-Kuhn-Tucker conditions of (P), solve the primal.

2) Write the dual and solve it.

Proposition 5.3 Given the pair of dual linear programming problems

(P ) and (D), one of the two following alternatives holds:
- either (P ) and (D) have a couple (x, p) of optimal solutions satisfying: p b = c x (obviously, the same relation p b = c x holds then for
every couple (x, p) of optimal solutions since the primal and the dual have
the same value);
- or neither (P ) nor (D) has an optimal solution and one of both
feasible sets is empty.
In what follows, the duality theorem is used to establish an important
result on linear inequalities which extends Exercise4.27.

5.3.2

Application
lemma

to

nonhomogeneous

Farkas

Exercise 5.1 (***) Let for every i = 0, 1, . . . , m, ai Rn , i R be

such that the system ai x i , i = 1, . . . , m is consistent. Then in order
that the following implication is true
ai x i , i = 1, . . . , m a0 x 0

(5.1)

it is necessary and sufficient that there exist nonnegative real numbers

i 0 such that
a0 =

m
X
i=1

i ai and 0

m
X

i i .

i=1

Exercise 5.2 Let p1 > 0, p2 > 0 and the optimization problem

min x1 x2
s.c. x1 0, x2 0
(P)

p1 x1 + p2 x2 1
38

University Paris 1
M1QEM1

2008/2009
Mr. Gourdel

Chapter 6

Brief presentation of Scilab

6.1
6.1.1

The linear case

6.1.2

Graphic solving

In order to determine the set of feasible points, we will draw the line D1
whose equation is x + 4y = 10, D2 whose equation is 5x + 2y = 20 and
finally D3 whose equation is x + 2y = 6. We can draw the figure 6.1.
The graphical analysis (figure 6.2) allows us to determine the optimal
point of this problem. The optimum is unique (classical case), the
point is M3 defined by the intersection of D2 and D3 . We determine its
coordinates by solving the following system:

5x + 2y = 20
x + 2y = 6

Economical motivation

Exemple 6.1 Let us consider a firm which is able to produce two kinds
of output (we do not face indivisibility problems). Each output needs to
spend some time on three workshops.
Output A requires to spend 1 hour on the first workshop P 1, 5 hours
on P 2 and 1 hour on P 3.
Output B requires to spend 4 hour on the first workshop P 1, 2 hours
on P 2 and 2 hours on P 3.

We can solve and find the production (3,5, 1,25), which corresponds to a
profit of 10, 75e.
At this point, we can determine the multipliers R5+ which are
solutions of

1 (x) = 0

2 (y) = 0

3 (x + 4y 10) = 0
4 (5x + 2y 20) = 0

+ 2y 
6) = 0 
5 (x 



 
 
 


2
1
0
1
5
1

+ 2
+ 3
+ 4
+ 5
= 1

3
0
1
4
2
2

The actual renting contract for P 1 allows us to use it during 10 hours per
day, 20 hours for P 2 and 6 hours for P 3. The unit profit for output A is
(respectively B) is 2 e (respectively 3 e).
Let us denote by x and y the quantities of output A and output B.
The profit maximization can be modelled as
max 2x + 3y

s.t. x 0

y0
(P1 )
x + 4y 10

5x + 2y 20

x + 2y 6

Since the constraints 1, 2 and 3 are not binding, the associated multipliers
39









576
0123

./01

23*+







4

598


 

  
!#"%$&
')(

 -./, 5;:




*+,-


59<

Figure 6.2: Optimal point of Example 6.1






   

  
    
  
    

  

  



  

  














6.1.3

Scilab Solver

Let us use the scilab software (www.scilab.org). Scilab is a scientific

software package for numerical computations providing a powerful open
computing environment for engineering and scientific applications. Scilab
is an open source software. Since 1994 it has been distributed freely along
with the source code via the Internet. It is currently used in educational
and industrial environments around the world.
Let us transform our problem as

min 2x1 3x2

s.t. x 0
(P1 )

Ax b
where A is a matrix with three rows and 2 columns and b a column
vector

Figure 6.1: feasible set of Problem (P1 ).

are equal to zero.

1 = 0

2 = 0

0
 
 
3 = 

2
5
1

+ 5
= 4

3
2
2

1 4
A= 5 2
1 2

10
b = 20 .
6

Let us write the following text file exemple1.sci

One gets 4 = 0.125 and 5 = 1.375.

40

// My first Scilab program

P=-[2;3];
b=[10;20;6];
A=[1,4;5,2;1,2]
Z sup=[]; //max value for x, empty here which means that
there is not such constraints
Z inf=[0;0]; //min value for x

with the help (cf. figure 6.4).

[Zopt,lag,CA]=linpro(P,A,b,Z inf,Z sup)

Figure 6.4: help window for Instruction linpro.

Figure 6.3: Numerical solution

It is easy to understand the syntax of Scilab for defining matrices,
the coefficients are written by rows and separated by a semicolon. The
characters // explains that what follows has to be treated as comments.
At the end of a line, when there is a semicolon, the result will not be
printed on the screen. The syntax of the linpro instruction can be found

In the software, we use the menu File, in order to select the file exemple1.sci. The software does not only give us the point (x, y) solution
but also the value of the problem and a vector of multipliers (which are
not necessarily unique).
41

6.1.4

Sensibility analysis


Let us consider now the modified problem (P2 ), when the workshop P 1 is available during 11 hours. How will move the production (solution of the problem) and the profit (value of the problem)
max 2x + 3y
s.t. x 0
y0
(P2 )
x + 4y 11
5x + 2y+ 20
x + 2y 6
This problem is very similar to the original one (P1 ) (same objective
function, almost same set of feasible points). On the figure 6.5, one can
understand that the domain has been transform by a translation of one
of the lines defining the original set of feasible points. On the picture 6.6,

79: 8
2345

7 ;8
0123




45,


 

 
 
 "!$#&%'

)(+*

 /01. 7=<




,-./


7?>

Figure 6.6: Graphical solving of Problem (P2 )

6.1.5


%$
#"


     
       
      
       
       
          

     
 
     
!

 
      
 
     

















Second modification of the constraints

it is easy to remark that the change for the set of feasible points does
not imply a change on the coordinates of the optimal point which is still
M3 (3,5, 1,25). Consequently the profit is unchanged.

Let us consider the same problem as (P1 ) except that now, the third
workshop will be available during 6 hours and a half. How will move the
production (solution of the problem) and the profit (value of the problem)
This problem can be written as:
max 2x + 3y
s.t. x 0
y0
(P3 )
x + 4y 10
5x + 2y+ 20
x + 2y 6, 5
On the picture 6.7, we present the new set of feasible points.
On the picture 6.8, we can check that the new optimal point is the
point defined by

5x + 2y = 20
x + 2y = 6.5

Remark 6.1 For large values of the available time of Workshop 1, this
constraint is not binding, while it is binding for small enough values.

The solution is (3,375, 1,3125), and corresponds to a profit of 11, 4375.

The additional profit is equal to 0, 6875 e. If the renting of this additional
half hour costs less than 0, 6875 e, renting this additional half hour is a
profitable operation. The marginal price of an additonal hour needs




Figure 6.5: modification of the feasible corresponding to the translation

of Line D1

42


   
  
     

     



         
        

        

       

          
        

     

   


  
 











Each constraint has an implicit price, the shadow price, which is given
by the multiplier.

6.1.6

In order to propose several simulations without changing our scilab program, we can consider a new program
// A second program
labels=["Workshop 1";"2";"3"];
[ok,u,v,w]=getvalue("define parameters",labels,...
list("vec",1,"vec",1,"vec",1),["10";"20";"6"]);
P=-[2;3];
A=[1,4;5,2;1,2];
Z sup=[]; //max value for x, empty here which means that
there is not such constraints
Z inf=[0;0]; //min value for x
b=[u;v;w]
// -----------------------// Calcul de loptimum
// -----------------------[Zopt,lag,CA]=linpro(P,A,S,Z inf,Z sup)

Figure 6.7: feasible set corresponding to (P3 )

 
 

  

  "!$#&%'(*)

687
1234

 


6:9
6 9
/012   ; :
./0-  <



5

34+,




 

+,-.

Alternative program

The instruction getvalue allows to type in a window the values of

the parameters (See Figure 6.9). This allows us to solve both (P2) and

6>=


Figure 6.8: graphical solving of (P3 )

to be less than 2 0.6875 = 1.375. Note that this corresponds to the
value of the multiplier associated with the third workshop (it is the case
here because the multiplier is unique and because the relaxation of the
constraint is not too large).

Figure 6.9: the parameters window

43

(P3) (See Figure 6.10).

written
[f,xopt]=optim(cost,[1;2])
Let us start with initial point (1, 2).

Figure 6.10: Solving (P3)

Figure 6.11: Solving (P5)

6.2

The non linear case

6.3

Unconstrained optimization

Let us consider the problem (P5 ) min x2 + y 2 . We can use the following
scilab program :
function [f,g,ind]=cost(x,ind)
f=x(1)^
2+x(2)^
2, g=[2*x(1);2*x(2)]
endfunction;

6.3.1

Unconstrained Optimization

max 3x + 3y
Let us consider sous contraintes x [2, 10]
(P6 )
y [10, 10]
The scilab program will give us a 3-dimensional plotting and the solution.

// g is the gradient of f
// here, ind is an unused parameter but which has to be
44

// programme exemple3.sci
function [f,g,ind]=cost(x,ind)
f=x(1)^2+x(2)^2, g=[2*x(1);2*x(2)]
endfunction;
function [z]=C(x,y) , z=x^2+y^2, endfunction;
x=2:10;y=-10:10;
z=feval(x,y,C);
xbasc();
plot3d(x,y,z);
[f,xopt,gopt]=optim(cost,b,[2;-10],[10;10],[5;5])
// f nest pas nul
// The gradient at the opt point is normal
// to the boundary

200
180
160
140

120
100
80
60
40
20
0
10

10

!2

!4

!6

!8

!10

Figure 6.13: graph of objective function of (P6 )

Figure 6.12: Solving (P6)

We remark that at the optimal point, the gradient of the objective
function is not equal to zero, (it is (4, 0)).
45

Contents
1 Presentation of Optimization
1.1 Mathematical presentation . . . . . . . . . . . . . . . . . .
1.2 Examples of economic problems . . . . . . . . . . . . . . .
2 Convexity of sets
2.1 Definition . . . . . . . . . . . . . . . . .
2.2 First properties . . . . . . . . . . . . . .
2.3 Stability with respect to affine functions
2.4 Convex hull, affine hull and conic hull . .
3 The
3.1
3.2
3.3
3.4
3.5

one dimensional results

Existence results . . . . . . . . . .
First order condition . . . . . . . .
Convex and concave functions . . .
Sufficient conditions for optimality
Introduction to sensibility analysis

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

.
.
.
.

.
.
.
.
.

1
1
4

.
.
.
.

6
6
8
9
10

.
.
.
.
.

13
13
13
15
20
21

4 Finite dimensional optimization

4.1 Existence results . . . . . . . . . .
4.2 Convex and quasi-convex functions
4.3 Unconstrained optimization . . . .
4.4 Karush-Kuhn-Tucker . . . . . . . .
4.5 Polarity and orthogonality . . . . .
4.6 Tangent and normal cones . . . . .

46

.
.
.
.
.
.

23
23
24
28
30
32
34

5 Linear Programming
5.1 The main results . . . . . . . . . . . . . . . . . . . . . . .
5.2 Saddle point properties . . . . . . . . . . . . . . . . . . . .
5.3 The duality theorem of linear programming . . . . . . . .

36
36
37
37

6 Brief presentation of Scilab

6.1 The linear case . . . . . . . . . . . . . . . . . . . . . . . .
6.2 The non linear case . . . . . . . . . . . . . . . . . . . . . .
6.3 Unconstrained optimization . . . . . . . . . . . . . . . . .

39
39
44
44

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.