0% found this document useful (0 votes)
77 views

Introduction To Optimization

This document provides an introduction to optimization problems. It discusses that an optimization problem requires three main concepts: decision variables, restrictions, and an objective function. Decision variables are the variables that can be chosen, and their set is denoted as X. Restrictions are the constraints that the decision variables must satisfy in order for a solution to be feasible, and their set is denoted as C, which is a subset of X. The objective is to find values for the decision variables that optimize the objective function while satisfying the restrictions.

Uploaded by

fwlwllw
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
77 views

Introduction To Optimization

This document provides an introduction to optimization problems. It discusses that an optimization problem requires three main concepts: decision variables, restrictions, and an objective function. Decision variables are the variables that can be chosen, and their set is denoted as X. Restrictions are the constraints that the decision variables must satisfy in order for a solution to be feasible, and their set is denoted as C, which is a subset of X. The objective is to find values for the decision variables that optimize the objective function while satisfying the restrictions.

Uploaded by

fwlwllw
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 125

Introduction to Optimization

Introduction

Optimality
conditions

Pedro Gajardo1 and Eladio Ocana2

1 Universidad Tecnica Federico Santa Mara - Valparaso - Chile

2
IMCA-FC. Universidad Nacional de Ingeniera - Lima - Peru

Mathematics of Bio-Economics (MABIES) - IHP

January 29, 2013 - Paris


Contents
Introduction

Optimality
conditions

Introduction

Optimality conditions
Optimization problems
Introduction

Optimality
conditions

In order to formulate an optimization problem, the following


concepts must be very clear:

decision variables

restrictions

objective function
Optimization problems
Introduction

Optimality
conditions

In order to formulate an optimization problem, the following


concepts must be very clear:

decision variables

restrictions

objective function
Optimization problems
Introduction

Optimality
conditions

In order to formulate an optimization problem, the following


concepts must be very clear:

decision variables

restrictions

objective function
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting


rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the


decision variable is (x X)
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting


rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the


decision variable is (x X)
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting


rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the


decision variable is (x X)
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting


rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the


decision variable is (x X)
Optimization problems
Decision variables
Introduction

Optimality
conditions

One or more variables on which we can decide (harvesting


rate or effort, level of investment, distribution of tasks,
parameters)
Objective: to find the best value for the decision variable

We denote by x the decision variable

x can be a number, a vector, a sequence, a function, ..........

In a general framework, we denote by X the set where the


decision variable is (x X)
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Firstly, the set X where the decision variable belongs, is a


(linear) vector space
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Firstly, the set X where the decision variable belongs, is a


(linear) vector space
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Firstly, the set X where the decision variable belongs, is a


(linear) vector space
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Firstly, the set X where the decision variable belongs, is a


(linear) vector space
Optimization problems
Restrictions
Introduction

Optimality
Constraints that the decision variable has to satisfy conditions

If for a certain value of the decision variable the


restrictions are satisfied, we say that it is a feasible
solution

In a general framework, we denote by C X the set of all


feasible solutions

If we have two decision variables, x1 and x2 and they have


to satisfy the following constraints: x1 0, x2 0,
2 x1 + 3 x2 5, we denote

C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction

Optimality
Constraints that the decision variable has to satisfy conditions

If for a certain value of the decision variable the


restrictions are satisfied, we say that it is a feasible
solution

In a general framework, we denote by C X the set of all


feasible solutions

If we have two decision variables, x1 and x2 and they have


to satisfy the following constraints: x1 0, x2 0,
2 x1 + 3 x2 5, we denote

C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction

Optimality
Constraints that the decision variable has to satisfy conditions

If for a certain value of the decision variable the


restrictions are satisfied, we say that it is a feasible
solution

In a general framework, we denote by C X the set of all


feasible solutions

If we have two decision variables, x1 and x2 and they have


to satisfy the following constraints: x1 0, x2 0,
2 x1 + 3 x2 5, we denote

C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction

Optimality
Constraints that the decision variable has to satisfy conditions

If for a certain value of the decision variable the


restrictions are satisfied, we say that it is a feasible
solution

In a general framework, we denote by C X the set of all


feasible solutions

If we have two decision variables, x1 and x2 and they have


to satisfy the following constraints: x1 0, x2 0,
2 x1 + 3 x2 5, we denote

C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Secondly, the set C of feasible solutions is not a discrete set (set


of isolated points)
Warning
Introduction

Optimality
conditions

This tutorial is on Continuous Optimization

That means that decision variables are continuous variables

What that means?

Secondly, the set C of feasible solutions is not a discrete set (set


of isolated points)
Optimization problems
Objective function
Introduction

Optimality
conditions

It is the mathematical representation for measuring the


goodness of values for the decision variable

This representation is done through a function

f : X R
Optimization problems
Objective function
Introduction

Optimality
conditions

It is the mathematical representation for measuring the


goodness of values for the decision variable

This representation is done through a function

f : X R
Optimization problems
Introduction

To find the best value for the decision variable from all feasible Optimality
conditions
solutions

To find x C X such that

f (x) f (x) (or f (x) f (x)) for all x C

 
min


f (x) or max f (x)
xX xX

subject to


xC

Optimization problems
Introduction

To find the best value for the decision variable from all feasible Optimality
conditions
solutions

To find x C X such that

f (x) f (x) (or f (x) f (x)) for all x C

 
min


f (x) or max f (x)
xX xX

subject to


xC

Optimization problems
Introduction

To find the best value for the decision variable from all feasible Optimality
conditions
solutions

To find x C X such that

f (x) f (x) (or f (x) f (x)) for all x C

 
min


f (x) or max f (x)
xX xX

subject to


xC

Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence


Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence


Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence


Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence


Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

With a given fence, to enclose the largest rectangular area

Objective: maximize the rectangular surface

Decision variables: lengths of the rectangular figure

Constraints:

positive lengths

length of the fence


Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b


Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L


L
b= 2 a
L

New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b


Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L


L
b= 2 a
L

New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b


Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L


L
b= 2 a
L

New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b


Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L


L
b= 2 a
L

New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b


Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L


L
b= 2 a
L

New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b


Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L


L
b= 2 a
L

New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b


Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L


L
b= 2 a
L

New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

Decision variables: a (height) and b (width)

Objective: maximize s(a, b) = a b


Constraints:
positive lengths: a > 0, b > 0

length of fence (L > 0): 2(a + b) = L


L
b= 2 a
L

New objective function f (a) = a 2 a
L
Rewrite restrictions: a > 0, a < 2
Optimization problems
Example: maximizing a rectangular surface
Introduction

Optimality
conditions

L

max f (a) = a 2 a


aR

subject to


a>0


a < L2

Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments


Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments


Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments


Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments


Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments


Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions

A firm wishes to maximize the utilities of its portafolio

Objective: maximize profits

Decision variables: amount to invest in each fund

Constraints:

nonnegative investments

budget restrictions

minimal or maximal bounds for investments


Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:


a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:


a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:


a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:


a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:


a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction
Decision variables: x1 , x2 , . . . , xn , where xj the quantity to Optimality
conditions
invest in the fund j
Objective: maximize
(1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
where rj is the rentability of the fund j
Restrictions:
Non-negative investments: xj 0 j = 1, 2, . . . , n
Budget restriction:
x1 + x2 + . . . + xn B

Minimal/maximal bounds for investments:


a j xj b j for j = 1, 2, . . . , n
Optimization problems
Example: Optimizing a portafolio
Introduction

Optimality
conditions


max (1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn

x1 ,x2 ,...,xn R



subject to





xj 0 j = 1, 2, . . . , n






x1 + x2 + . . . + xn B


aj xj bj j = 1, 2, . . . , n

Optimization problems
Example: Least square problem
Introduction

Optimality
conditions

Given points on the plane, to find the line of best fit through
these points

Objective: To minimize the (squared) error between a line


and the points

Decision variables: the parameters of a line


Optimization problems
Example: Least square problem
Introduction

Optimality
conditions

Given points on the plane, to find the line of best fit through
these points

Objective: To minimize the (squared) error between a line


and the points

Decision variables: the parameters of a line


Optimization problems
Example: Least square problem
Introduction

Optimality
Given points conditions

(x1 , y1 ); (x2 , y2 ); . . . ; (xn , yn )

to find the line of best fit through these points


Decision variables (the parameters of a line): m and n,
where the expression of a line in the plane is

y = mx + n
Objective: To minimize
n
X
(mxk + n yk )2
k=1
Optimization problems
Example: Least square problem
Introduction

Optimality
Given points conditions

(x1 , y1 ); (x2 , y2 ); . . . ; (xn , yn )

to find the line of best fit through these points


Decision variables (the parameters of a line): m and n,
where the expression of a line in the plane is

y = mx + n
Objective: To minimize
n
X
(mxk + n yk )2
k=1
Optimization problems
Example: Least square problem
Introduction

Optimality
conditions

n
(mxk + n yk )2
P
max
m, n R k=1
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each


period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each


period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each


period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each


period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each


period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each


period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource

Objective: maximize present utilities

Decision variables: harvesting levels (or effort) at each


period

Constraints:

nonnegative harvesting

biology

relation between harvesting and the amount of the resource

environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1

x(t0 ) = x0 given (e.g. the current state of the resource)

where
x(t) 0 is the level of the resource at period t

h(t) 0 is the harvesting at period t

F : R R is the biological growth function of the


resource
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1

x(t0 ) = x0 given (e.g. the current state of the resource)

where
x(t) 0 is the level of the resource at period t

h(t) 0 is the harvesting at period t

F : R R is the biological growth function of the


resource
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1

x(t0 ) = x0 given (e.g. the current state of the resource)

where
x(t) 0 is the level of the resource at period t

h(t) 0 is the harvesting at period t

F : R R is the biological growth function of the


resource
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

The total benefits can be represented by


T1
X
B= (tt0 ) U(x(t), h(t))
t=t0

where
U(x, h) is the instantaneous profit if we have x and we
harvest h

0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

The total benefits can be represented by


T1
X
B= (tt0 ) U(x(t), h(t))
t=t0

where
U(x, h) is the instantaneous profit if we have x and we
harvest h

0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions

The total benefits can be represented by


T1
X
B= (tt0 ) U(x(t), h(t))
t=t0

where
U(x, h) is the instantaneous profit if we have x and we
harvest h

0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction

Optimality
conditions


T1
P (tt )


max 0 U(x(t), h(t))

h(t0 ),h(t0 +1),...,h(T1) t=t0



subject to







x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1

x(t0 ) = x0







x(t) 0

t = t0 + 1, t0 + 2, . . . , T




0 h(t) F(x(t)) t = t0 , t0 + 1, . . . , T 1

Optimization problems
Important remark: min is equivalent to max
Introduction

Optimality
conditions
We can restrict our study only to minimization problems

To find x C X such that

f (x) f (x) for all x C


min f (x)
xX

subject to


xC

Optimization problems
Important remark: min is equivalent to max
Introduction

Optimality
conditions
We can restrict our study only to minimization problems

To find x C X such that

f (x) f (x) for all x C


min f (x)
xX

subject to


xC

Optimization problems
Important remark: min is equivalent to max
Introduction

Optimality
conditions

To find x C X such that

f (x) f (x) for all x C


max f (x)
xX

subject to


xC

Optimization problems
Important remark: min is equivalent to max
Introduction

Optimality
conditions

To find x C X such that

f (x) f (x) for all x C


max f (x)
xX

subject to


xC

Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
max f (x) conditions
xX

(Pmax ) subject to


xC

is equivalent to problem

min f (x)
xX

(Pmin ) subject to


xC

In the following sense:


x is solution of (Pmax ) if and only if it is solution of (Pmin )

The optimal value of (Pmax ) is the negative of the optimal


value of (Pmin )
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
max f (x) conditions
xX

(Pmax ) subject to


xC

is equivalent to problem

min f (x)
xX

(Pmin ) subject to


xC

In the following sense:


x is solution of (Pmax ) if and only if it is solution of (Pmin )

The optimal value of (Pmax ) is the negative of the optimal


value of (Pmin )
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
max f (x) conditions
xX

(Pmax ) subject to


xC

is equivalent to problem

min f (x)
xX

(Pmin ) subject to


xC

In the following sense:


x is solution of (Pmax ) if and only if it is solution of (Pmin )

The optimal value of (Pmax ) is the negative of the optimal


value of (Pmin )
Optimization problems
Existence of solutions
Introduction

Optimality
conditions


min f (x)
xX

(P) subject to


xC

When (P) has solution?


If [a, b] = C X = R

When X = Rn , there exists at least one solution if C is


closed and bounded
Optimization problems
Existence of solutions
Introduction

Optimality
conditions


min f (x)
xX

(P) subject to


xC

When (P) has solution?


If [a, b] = C X = R

When X = Rn , there exists at least one solution if C is


closed and bounded
Contents
Introduction

Optimality
conditions

Introduction

Optimality conditions
Optimality conditions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

Optimality conditions are mathematical expressions: equations


or system of equations, system of inequalities, ordinary
differential equations, partial differential equations

Solutions of the mathematical problems obtained from


optimality conditions are related with the solutions of the
optimization problem
Optimality conditions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

Optimality conditions are mathematical expressions: equations


or system of equations, system of inequalities, ordinary
differential equations, partial differential equations

Solutions of the mathematical problems obtained from


optimality conditions are related with the solutions of the
optimization problem
Optimality conditions
Introduction

Optimality
conditions

min f (x)
xX

(P) subject to


xC

Optimality conditions are mathematical expressions: equations


or system of equations, system of inequalities, ordinary
differential equations, partial differential equations

Solutions of the mathematical problems obtained from


optimality conditions are related with the solutions of the
optimization problem
Optimality conditions
Introduction

Optimality
conditions


min f (x)
xX

(P) subject to


xC

The idea is to obtain optimality conditions and then to solve the


associated mathematical problems (analytically or numerically)
Optimality conditions
Introduction

Optimality
conditions


min f (x)
xX

(P) subject to


xC

The idea is to obtain optimality conditions and then to solve the


associated mathematical problems (analytically or numerically)
Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are


represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of


(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of


candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)


or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)


Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are


represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of


(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of


candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)


or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)


Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are


represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of


(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of


candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)


or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)


Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are


represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of


(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of


candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)


or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)


Optimality conditions
Necessary conditions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Suppose that optimality conditions of this problem are


represented by a system of equations (S)

We say that (S) is a necessary condition if a solution of


(P) is a solution of (S)

The idea is to find solutions of (S) in order to have a list of


candidates for the solutions of (P)

If (S) is solved only for one point, then it is solution of (P)


or (P) does not have solution

Warning: A solution of (S) may be not a solution of (P)


Optimality conditions
Necessary condition: unrestricted case
Introduction

Optimality
conditions
n
(P) min f (x)
xR

where f : R R
A necessary condition is f (x) = 0 where

f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0

If we know the expression of f we can solve the equation


f (x) = 0 in order to have candidates of solutions

Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction

Optimality
conditions
n
(P) min f (x)
xR

where f : R R
A necessary condition is f (x) = 0 where

f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0

If we know the expression of f we can solve the equation


f (x) = 0 in order to have candidates of solutions

Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction

Optimality
conditions
n
(P) min f (x)
xR

where f : R R
A necessary condition is f (x) = 0 where

f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0

If we know the expression of f we can solve the equation


f (x) = 0 in order to have candidates of solutions

Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction

Optimality
conditions
n
(P) min f (x)
xR

where f : R R
A necessary condition is f (x) = 0 where

f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0

If we know the expression of f we can solve the equation


f (x) = 0 in order to have candidates of solutions

Warning!!!
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb

where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)

There are three posibilities:

an interior point a < x < b is a solution. Then f (x) = 0

x = a is a solution. Then f (x) = f (a) 0

x = b is a solution. Then f (x) = f (b) 0

Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction

A necessary condition of the problem Optimality


conditions

min f (x)
xR
(P)
axb
is


Find x, , such that



f (x) + = 0







x [a, b]

(S)



, 0






(x a) = 0


(x b) = 0

Optimality conditions
Necessary conditions: restrictions
Introduction

A necessary condition of the problem Optimality


conditions

min f (x)
xR
(P)
axb
is


Find x, , such that



f (x) + = 0







x [a, b]

(S)



, 0






(x a) = 0


(x b) = 0

Optimality conditions
Global vs local minimum
Introduction

Optimality
conditions


min f (x)
xX

(P) subject to


xC

We say that x C is a global minimum, if it is a solution


of (P)

We say that x C is a local minimum, if for every x C


close enough, one has f (x) f (x)

A global optimum is a local optimum


Optimality conditions
Global vs local minimum
Introduction

Optimality
conditions


min f (x)
xX

(P) subject to


xC

We say that x C is a global minimum, if it is a solution


of (P)

We say that x C is a local minimum, if for every x C


close enough, one has f (x) f (x)

A global optimum is a local optimum


Optimality conditions
Global vs local minimum
Introduction

Optimality
conditions


min f (x)
xX

(P) subject to


xC

We say that x C is a global minimum, if it is a solution


of (P)

We say that x C is a local minimum, if for every x C


close enough, one has f (x) f (x)

A global optimum is a local optimum


Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)


is a solution of (P)

The idea is to find solutions of (S) in order to have


solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local


solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and


f (x) > 0
Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)


is a solution of (P)

The idea is to find solutions of (S) in order to have


solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local


solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and


f (x) > 0
Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)


is a solution of (P)

The idea is to find solutions of (S) in order to have


solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local


solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and


f (x) > 0
Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)


is a solution of (P)

The idea is to find solutions of (S) in order to have


solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local


solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and


f (x) > 0
Optimality conditions
Sufficient condition: unrestricted case
Introduction

n Optimality
conditions
(P) min f (x)
xR

We say that (S) is a sufficient condition if a solution of (S)


is a solution of (P)

The idea is to find solutions of (S) in order to have


solutions of (P)

In the general case, this is very difficult

It is much realistic to have that solutions of (S) are local


solutions of (P)

A (local) sufficient condition to (P) is f (x) = 0 and


f (x) > 0
Optimality conditions
Sufficient condition
Introduction

Optimality
conditions

If we find x R such that

f (x) = 0
(
(S)
f (x) > 0

Then x is a local minimum of the problem


n
(P) min f (x)
xR
Optimality conditions
Sufficient condition
Introduction

Optimality
conditions

If we find x R such that

f (x) = 0
(
(S)
f (x) > 0

Then x is a local minimum of the problem


n
(P) min f (x)
xR
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following


concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization


problem (deal with f (x) instead of f (x))
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following


concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization


problem (deal with f (x) instead of f (x))
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following


concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization


problem (deal with f (x) instead of f (x))
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following


concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization


problem (deal with f (x) instead of f (x))
Conclusions
Introduction
min f (x) Optimality
xX
conditions
(P) subject to


xC

To formulate an optimization problem, the following


concepts must be clear:

Decision variable(s): x X

Objective function: f : X R

Restrictions: C X

A maximization problem is equivalent to a minimization


problem (deal with f (x) instead of f (x))
Conclusions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Optimality conditions are mathematical systems

The idea is to solve these systems in order to:

have candidates of solutions (if the system is a necessary


condition)

obtain a local solution (if the system is a sufficient


condition)
Conclusions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Optimality conditions are mathematical systems

The idea is to solve these systems in order to:

have candidates of solutions (if the system is a necessary


condition)

obtain a local solution (if the system is a sufficient


condition)
Conclusions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Optimality conditions are mathematical systems

The idea is to solve these systems in order to:

have candidates of solutions (if the system is a necessary


condition)

obtain a local solution (if the system is a sufficient


condition)
Conclusions
Introduction
Optimality
min f (x) conditions
xX

(P) subject to


xC

Optimality conditions are mathematical systems

The idea is to solve these systems in order to:

have candidates of solutions (if the system is a necessary


condition)

obtain a local solution (if the system is a sufficient


condition)
Next part
Introduction

Optimality

min f (x) conditions
xX

(P) subject to


xC

Several decision variables x1 , x2 , . . . , xn . That is



x1
x2
x = . Rn = X

..
xn

Necessary and sufficient conditions

Why the convexity (or concavity) is relevant?


Next part
Introduction

Optimality

min f (x) conditions
xX

(P) subject to


xC

Several decision variables x1 , x2 , . . . , xn . That is



x1
x2
x = . Rn = X

..
xn

Necessary and sufficient conditions

Why the convexity (or concavity) is relevant?


Next part
Introduction

Optimality

min f (x) conditions
xX

(P) subject to


xC

Several decision variables x1 , x2 , . . . , xn . That is



x1
x2
x = . Rn = X

..
xn

Necessary and sufficient conditions

Why the convexity (or concavity) is relevant?

You might also like