Introduction to Optimization

Introduction

Optimality
conditions

Pedro Gajardo1 and Eladio Ocana2

1 Universidad Tecnica Federico Santa Mara - Valparaso - Chile

2
IMCA-FC. Universidad Nacional de Ingeniera - Lima - Peru

Mathematics of Bio-Economics (MABIES) - IHP

January 29, 2013 - Paris

Contents
Introduction

Optimality
conditions

Introduction

Optimality conditions
Optimization problems
Introduction

Optimality
conditions

In order to formulate an optimization problem, the following

concepts must be very clear:

decision variables

restrictions

objective function
Optimization problems
Introduction

Optimality
conditions

In order to formulate an optimization problem, the following

concepts must be very clear:

decision variables

restrictions

objective function
Optimization problems
Introduction

Optimality
conditions

In order to formulate an optimization problem, the following

concepts must be very clear:

decision variables

restrictions

objective function
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting

rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the

decision variable is (x X)
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting

rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the

decision variable is (x X)
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting

rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the

decision variable is (x X)
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting

rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the

decision variable is (x X)
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting

rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the

decision variable is (x X)
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Firstly, the set X where the decision variable belongs, is a

(linear) vector space
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Firstly, the set X where the decision variable belongs, is a

(linear) vector space
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Firstly, the set X where the decision variable belongs, is a

(linear) vector space
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Firstly, the set X where the decision variable belongs, is a

(linear) vector space
Optimization problems
Restrictions
Introduction

Optimality
Constraints that the decision variable has to satisfy conditions

If for a certain value of the decision variable the

restrictions are satisfied, we say that it is a feasible
solution

In a general framework, we denote by C X the set of all

feasible solutions

If we have two decision variables, x1 and x2 and they have

to satisfy the following constraints: x1 0, x2 0,
2 x1 + 3 x2 5, we denote

C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction

Optimality
Constraints that the decision variable has to satisfy conditions

If for a certain value of the decision variable the

restrictions are satisfied, we say that it is a feasible
solution

In a general framework, we denote by C X the set of all

feasible solutions

If we have two decision variables, x1 and x2 and they have

to satisfy the following constraints: x1 0, x2 0,
2 x1 + 3 x2 5, we denote

C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction

Optimality
Constraints that the decision variable has to satisfy conditions

If for a certain value of the decision variable the

restrictions are satisfied, we say that it is a feasible
solution

In a general framework, we denote by C X the set of all

feasible solutions

If we have two decision variables, x1 and x2 and they have

to satisfy the following constraints: x1 0, x2 0,
2 x1 + 3 x2 5, we denote

C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction

Optimality
Constraints that the decision variable has to satisfy conditions

If for a certain value of the decision variable the

restrictions are satisfied, we say that it is a feasible
solution

In a general framework, we denote by C X the set of all

feasible solutions

If we have two decision variables, x1 and x2 and they have

to satisfy the following constraints: x1 0, x2 0,
2 x1 + 3 x2 5, we denote

C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Secondly, the set C of feasible solutions is not a discrete set (set

of isolated points)
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Secondly, the set C of feasible solutions is not a discrete set (set

of isolated points)
Optimization problems
Objective function
Introduction

Optimality
conditions

It is the mathematical representation for measuring the

goodness of values for the decision variable

This representation is done through a function

f : X R
Optimization problems
Objective function
Introduction

Optimality
conditions

It is the mathematical representation for measuring the

goodness of values for the decision variable

This representation is done through a function

f : X R
Optimization problems
Introduction

To find the best value for the decision variable from all feasible Optimality
conditions
solutions

To find x C X such that

f (x) f (x) (or f (x) f (x)) for all x C

 
min


f (x) or max f (x)
xX xX

subject to


xC

Optimization problems
Introduction

To find the best value for the decision variable from all feasible Optimality
conditions
solutions

To find x C X such that

f (x) f (x) (or f (x) f (x)) for all x C

 
min


f (x) or max f (x)
xX xX

subject to


xC

Optimization problems
Introduction

To find the best value for the decision variable from all feasible Optimality
conditions
solutions

To find x C X such that

f (x) f (x) (or f (x) f (x)) for all x C

 
min


f (x) or max f (x)
xX xX

subject to


xC

Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence

Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence

Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence

Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence

Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence

Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b

Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L

L
b= 2 a
L


New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b

Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L

L
b= 2 a
L


New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b

Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L

L
b= 2 a
L


New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b

Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L

L
b= 2 a
L


New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b

Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L

L
b= 2 a
L


New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b

Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L

L
b= 2 a
L


New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b

Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L

L
b= 2 a
L


New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b

Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L

L
b= 2 a
L


New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

L


max f (a) = a 2 a


aR

subject to


a>0


a < L2

Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments

Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments

Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments

Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments

Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments

Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments

Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:

a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:

a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:

a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:

a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:

a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:

a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

max (1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn

x1 ,x2 ,...,xn R



subject to





xj 0 j = 1, 2, . . . , n






x1 + x2 + . . . + xn B


aj xj bj j = 1, 2, . . . , n

Optimization problems
Example: Least square problem
Introduction

Optimality
conditions

Given points on the plane, to find the line of best fit through
these points

Objective: To minimize the (squared) error between a line

and the points

Decision variables: the parameters of a line

Optimization problems
Example: Least square problem
Introduction

Optimality
conditions

Given points on the plane, to find the line of best fit through
these points

Objective: To minimize the (squared) error between a line

and the points

Decision variables: the parameters of a line

Optimization problems
Example: Least square problem
Introduction

Optimality
Given points conditions

(x1 , y1 ); (x2 , y2 ); . . . ; (xn , yn )

to find the line of best fit through these points

Decision variables (the parameters of a line): m and n,
where the expression of a line in the plane is

y = mx + n
Objective: To minimize
n
X
(mxk + n yk )2
k=1
Optimization problems
Example: Least square problem
Introduction

Optimality
Given points conditions

(x1 , y1 ); (x2 , y2 ); . . . ; (xn , yn )

to find the line of best fit through these points

Decision variables (the parameters of a line): m and n,
where the expression of a line in the plane is

y = mx + n
Objective: To minimize
n
X
(mxk + n yk )2
k=1
Optimization problems
Example: Least square problem
Introduction

Optimality
conditions

n
(mxk + n yk )2
P
max
m, n R k=1
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each

period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each

period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each

period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each

period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each

period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each

period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each

period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1

x(t0 ) = x0 given (e.g. the current state of the resource)

where
x(t) 0 is the level of the resource at period t

h(t) 0 is the harvesting at period t

F : R R is the biological growth function of the

resource
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1

x(t0 ) = x0 given (e.g. the current state of the resource)

where
x(t) 0 is the level of the resource at period t

h(t) 0 is the harvesting at period t

F : R R is the biological growth function of the

resource
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1

x(t0 ) = x0 given (e.g. the current state of the resource)

where
x(t) 0 is the level of the resource at period t

h(t) 0 is the harvesting at period t

F : R R is the biological growth function of the

resource
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

The total benefits can be represented by

T1
X
B= (tt0 ) U(x(t), h(t))
t=t0

where
U(x, h) is the instantaneous profit if we have x and we
harvest h

0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

The total benefits can be represented by

T1
X
B= (tt0 ) U(x(t), h(t))
t=t0

where
U(x, h) is the instantaneous profit if we have x and we
harvest h

0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

The total benefits can be represented by

T1
X
B= (tt0 ) U(x(t), h(t))
t=t0

where
U(x, h) is the instantaneous profit if we have x and we
harvest h

0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

T1
P (tt )


max 0 U(x(t), h(t))

h(t0 ),h(t0 +1),...,h(T1) t=t0



subject to







x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1

x(t0 ) = x0







x(t) 0

t = t0 + 1, t0 + 2, . . . , T




0 h(t) F(x(t)) t = t0 , t0 + 1, . . . , T 1

Optimization problems
Important remark: min is equivalent to max
Introduction

Optimality
conditions
We can restrict our study only to minimization problems

To find x C X such that

f (x) f (x) for all x C

min f (x)
xX

subject to


xC

Optimization problems
Important remark: min is equivalent to max
Introduction

Optimality
conditions
We can restrict our study only to minimization problems

To find x C X such that

f (x) f (x) for all x C

min f (x)
xX

subject to


xC

Optimization problems
Important remark: min is equivalent to max
Introduction

Optimality
conditions

To find x C X such that

f (x) f (x) for all x C

max f (x)
xX

subject to


xC

Optimization problems
Important remark: min is equivalent to max
Introduction

Optimality
conditions

To find x C X such that

f (x) f (x) for all x C

max f (x)
xX

subject to


xC

Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
max f (x) conditions
xX

(Pmax ) subject to


xC

is equivalent to problem

min f (x)
xX

(Pmin ) subject to


xC

In the following sense:

x is solution of (Pmax ) if and only if it is solution of (Pmin )

The optimal value of (Pmax ) is the negative of the optimal

value of (Pmin )
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
max f (x) conditions
xX

(Pmax ) subject to


xC

is equivalent to problem

min f (x)
xX

(Pmin ) subject to


xC

In the following sense:

x is solution of (Pmax ) if and only if it is solution of (Pmin )

The optimal value of (Pmax ) is the negative of the optimal

value of (Pmin )
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
max f (x) conditions
xX

(Pmax ) subject to


xC

is equivalent to problem

min f (x)
xX

(Pmin ) subject to


xC

In the following sense:

x is solution of (Pmax ) if and only if it is solution of (Pmin )

The optimal value of (Pmax ) is the negative of the optimal

value of (Pmin )
Optimization problems
Existence of solutions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

When (P) has solution?

If [a, b] = C X = R

When X = Rn , there exists at least one solution if C is

closed and bounded
Optimization problems
Existence of solutions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

When (P) has solution?

If [a, b] = C X = R

When X = Rn , there exists at least one solution if C is

closed and bounded
Contents
Introduction

Optimality
conditions

Introduction

Optimality conditions
Optimality conditions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

Optimality conditions are mathematical expressions: equations

or system of equations, system of inequalities, ordinary
differential equations, partial differential equations

Solutions of the mathematical problems obtained from

optimality conditions are related with the solutions of the
optimization problem
Optimality conditions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

Optimality conditions are mathematical expressions: equations

or system of equations, system of inequalities, ordinary
differential equations, partial differential equations

Solutions of the mathematical problems obtained from

optimality conditions are related with the solutions of the
optimization problem
Optimality conditions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

Optimality conditions are mathematical expressions: equations

or system of equations, system of inequalities, ordinary
differential equations, partial differential equations

Solutions of the mathematical problems obtained from

optimality conditions are related with the solutions of the
optimization problem
Optimality conditions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

The idea is to obtain optimality conditions and then to solve the

associated mathematical problems (analytically or numerically)
Optimality conditions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

The idea is to obtain optimality conditions and then to solve the

associated mathematical problems (analytically or numerically)
Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are

represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of

(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of

candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)

or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)

Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are

represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of

(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of

candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)

or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)

Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are

represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of

(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of

candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)

or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)

Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are

represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of

(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of

candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)

or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)

Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are

represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of

(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of

candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)

or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)

Optimality conditions
Necessary condition: unrestricted case
Introduction

Optimality
conditions
n
(P) min f (x)
xR

where f : R R
A necessary condition is f (x) = 0 where

f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0

If we know the expression of f we can solve the equation

f (x) = 0 in order to have candidates of solutions

Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction

Optimality
conditions
n
(P) min f (x)
xR

where f : R R
A necessary condition is f (x) = 0 where

f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0

If we know the expression of f we can solve the equation

f (x) = 0 in order to have candidates of solutions

Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction

Optimality
conditions
n
(P) min f (x)
xR

where f : R R
A necessary condition is f (x) = 0 where

f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0

If we know the expression of f we can solve the equation

f (x) = 0 in order to have candidates of solutions

Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction

Optimality
conditions
n
(P) min f (x)
xR

where f : R R
A necessary condition is f (x) = 0 where

f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0

If we know the expression of f we can solve the equation

f (x) = 0 in order to have candidates of solutions

Warning!!!
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction

A necessary condition of the problem Optimality

conditions

min f (x)
xR
(P)
axb
is


Find x, , such that



f (x) + = 0







x [a, b]

(S)



, 0






(x a) = 0


(x b) = 0

Optimality conditions
Necessary conditions: restrictions
Introduction

A necessary condition of the problem Optimality

conditions

min f (x)
xR
(P)
axb
is


Find x, , such that



f (x) + = 0







x [a, b]

(S)



, 0






(x a) = 0


(x b) = 0

Optimality conditions
Global vs local minimum
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

We say that x C is a global minimum, if it is a solution

of (P)

We say that x C is a local minimum, if for every x C

close enough, one has f (x) f (x)

A global optimum is a local optimum

Optimality conditions
Global vs local minimum
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

We say that x C is a global minimum, if it is a solution

of (P)

We say that x C is a local minimum, if for every x C

close enough, one has f (x) f (x)

A global optimum is a local optimum

Optimality conditions
Global vs local minimum
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

We say that x C is a global minimum, if it is a solution

of (P)

We say that x C is a local minimum, if for every x C

close enough, one has f (x) f (x)

A global optimum is a local optimum

Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)

is a solution of (P)

The idea is to find solutions of (S) in order to have

solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local

solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and

f (x) > 0
Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)

is a solution of (P)

The idea is to find solutions of (S) in order to have

solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local

solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and

f (x) > 0
Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)

is a solution of (P)

The idea is to find solutions of (S) in order to have

solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local

solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and

f (x) > 0
Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)

is a solution of (P)

The idea is to find solutions of (S) in order to have

solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local

solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and

f (x) > 0
Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)

is a solution of (P)

The idea is to find solutions of (S) in order to have

solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local

solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and

f (x) > 0
Optimality conditions
Sufficient condition
Introduction

Optimality
conditions

If we find x R such that

f (x) = 0
(
(S)
f (x) > 0

Then x is a local minimum of the problem

n
(P) min f (x)
xR
Optimality conditions
Sufficient condition
Introduction

Optimality
conditions

If we find x R such that

f (x) = 0
(
(S)
f (x) > 0

Then x is a local minimum of the problem

n
(P) min f (x)
xR
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following

concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization

problem (deal with f (x) instead of f (x))
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following

concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization

problem (deal with f (x) instead of f (x))
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following

concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization

problem (deal with f (x) instead of f (x))
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following

concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization

problem (deal with f (x) instead of f (x))
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following

concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization

problem (deal with f (x) instead of f (x))
Conclusions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Optimality conditions are mathematical systems

The idea is to solve these systems in order to:

have candidates of solutions (if the system is a necessary

condition)

obtain a local solution (if the system is a sufficient

condition)
Conclusions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Optimality conditions are mathematical systems

The idea is to solve these systems in order to:

have candidates of solutions (if the system is a necessary

condition)

obtain a local solution (if the system is a sufficient

condition)
Conclusions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Optimality conditions are mathematical systems

The idea is to solve these systems in order to:

have candidates of solutions (if the system is a necessary

condition)

obtain a local solution (if the system is a sufficient

condition)
Conclusions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Optimality conditions are mathematical systems

The idea is to solve these systems in order to:

have candidates of solutions (if the system is a necessary

condition)

obtain a local solution (if the system is a sufficient

condition)
Next part
Introduction

Optimality

min f (x) conditions
xX

(P) subject to


xC

Several decision variables x1 , x2 , . . . , xn . That is

x1
x2
x = . Rn = X

..
xn

Necessary and sufficient conditions

Why the convexity (or concavity) is relevant?

Next part
Introduction

Optimality

min f (x) conditions
xX

(P) subject to


xC

Several decision variables x1 , x2 , . . . , xn . That is

x1
x2
x = . Rn = X

..
xn

Necessary and sufficient conditions

Why the convexity (or concavity) is relevant?

Next part
Introduction

Optimality

min f (x) conditions
xX

(P) subject to


xC

Several decision variables x1 , x2 , . . . , xn . That is

x1
x2
x = . Rn = X

..
xn

Necessary and sufficient conditions

Why the convexity (or concavity) is relevant?