Introduction To Optimization
Introduction To Optimization
Introduction
Optimality
conditions
2
IMCA-FC. Universidad Nacional de Ingeniera - Lima - Peru
Optimality
conditions
Introduction
Optimality conditions
Optimization problems
Introduction
Optimality
conditions
decision variables
restrictions
objective function
Optimization problems
Introduction
Optimality
conditions
decision variables
restrictions
objective function
Optimization problems
Introduction
Optimality
conditions
decision variables
restrictions
objective function
Optimization problems
Decision variables
Introduction
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
Constraints that the decision variable has to satisfy conditions
C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction
Optimality
Constraints that the decision variable has to satisfy conditions
C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction
Optimality
Constraints that the decision variable has to satisfy conditions
C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Optimization problems
Restrictions
Introduction
Optimality
Constraints that the decision variable has to satisfy conditions
C = {(x1 , x2 ) R2 | x1 0, x2 0, 2 x1 + 3x2 5}
Warning
Introduction
Optimality
conditions
Optimality
conditions
Optimality
conditions
f : X R
Optimization problems
Objective function
Introduction
Optimality
conditions
f : X R
Optimization problems
Introduction
To find the best value for the decision variable from all feasible Optimality
conditions
solutions
min
f (x) or max f (x)
xX xX
subject to
xC
Optimization problems
Introduction
To find the best value for the decision variable from all feasible Optimality
conditions
solutions
min
f (x) or max f (x)
xX xX
subject to
xC
Optimization problems
Introduction
To find the best value for the decision variable from all feasible Optimality
conditions
solutions
min
f (x) or max f (x)
xX xX
subject to
xC
Optimization problems
Example: maximizing a rectangular surface
Introduction
Optimality
conditions
Constraints:
positive lengths
Optimality
conditions
Constraints:
positive lengths
Optimality
conditions
Constraints:
positive lengths
Optimality
conditions
Constraints:
positive lengths
Optimality
conditions
Constraints:
positive lengths
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
Optimality
conditions
L
max f (a) = a 2 a
aR
subject to
a>0
a < L2
Optimization problems
Example: Optimizing a portafolio
Introduction
Optimality
conditions
Constraints:
nonnegative investments
budget restrictions
Optimality
conditions
Constraints:
nonnegative investments
budget restrictions
Optimality
conditions
Constraints:
nonnegative investments
budget restrictions
Optimality
conditions
Constraints:
nonnegative investments
budget restrictions
Optimality
conditions
Constraints:
nonnegative investments
budget restrictions
Optimality
conditions
Constraints:
nonnegative investments
budget restrictions
Optimality
conditions
max (1 + r1 )x1 + (1 + r2 )x2 + . . . + (1 + rn )xn
x1 ,x2 ,...,xn R
subject to
xj 0 j = 1, 2, . . . , n
x1 + x2 + . . . + xn B
aj xj bj j = 1, 2, . . . , n
Optimization problems
Example: Least square problem
Introduction
Optimality
conditions
Given points on the plane, to find the line of best fit through
these points
Optimality
conditions
Given points on the plane, to find the line of best fit through
these points
Optimality
Given points conditions
y = mx + n
Objective: To minimize
n
X
(mxk + n yk )2
k=1
Optimization problems
Example: Least square problem
Introduction
Optimality
Given points conditions
y = mx + n
Objective: To minimize
n
X
(mxk + n yk )2
k=1
Optimization problems
Example: Least square problem
Introduction
Optimality
conditions
n
(mxk + n yk )2
P
max
m, n R k=1
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource
Constraints:
nonnegative harvesting
biology
environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource
Constraints:
nonnegative harvesting
biology
environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource
Constraints:
nonnegative harvesting
biology
environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource
Constraints:
nonnegative harvesting
biology
environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource
Constraints:
nonnegative harvesting
biology
environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource
Constraints:
nonnegative harvesting
biology
environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
To maximize the benefit from the harvesting of a natural
resource
Constraints:
nonnegative harvesting
biology
environmental constraints
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
where
x(t) 0 is the level of the resource at period t
Optimality
conditions
where
x(t) 0 is the level of the resource at period t
Optimality
conditions
where
x(t) 0 is the level of the resource at period t
Optimality
conditions
where
U(x, h) is the instantaneous profit if we have x and we
harvest h
0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
where
U(x, h) is the instantaneous profit if we have x and we
harvest h
0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
where
U(x, h) is the instantaneous profit if we have x and we
harvest h
0 1 is a descount factor
Optimization problems
Example: Harvesting of a renewable resource
Introduction
Optimality
conditions
T1
P (tt )
max 0 U(x(t), h(t))
h(t0 ),h(t0 +1),...,h(T1) t=t0
subject to
x(t + 1) = F(x(t)) h(t) t = t0 , t0 + 1, . . . , T 1
x(t0 ) = x0
x(t) 0
t = t0 + 1, t0 + 2, . . . , T
0 h(t) F(x(t)) t = t0 , t0 + 1, . . . , T 1
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
conditions
We can restrict our study only to minimization problems
min f (x)
xX
subject to
xC
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
conditions
We can restrict our study only to minimization problems
min f (x)
xX
subject to
xC
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
conditions
max f (x)
xX
subject to
xC
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
conditions
max f (x)
xX
subject to
xC
Optimization problems
Important remark: min is equivalent to max
Introduction
Optimality
max f (x) conditions
xX
(Pmax ) subject to
xC
is equivalent to problem
min f (x)
xX
(Pmin ) subject to
xC
is equivalent to problem
min f (x)
xX
(Pmin ) subject to
xC
is equivalent to problem
min f (x)
xX
(Pmin ) subject to
xC
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
Introduction
Optimality conditions
Optimality conditions
Introduction
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
n
(P) min f (x)
xR
where f : R R
A necessary condition is f (x) = 0 where
f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0
Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction
Optimality
conditions
n
(P) min f (x)
xR
where f : R R
A necessary condition is f (x) = 0 where
f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0
Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction
Optimality
conditions
n
(P) min f (x)
xR
where f : R R
A necessary condition is f (x) = 0 where
f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0
Warning!!!
Optimality conditions
Necessary condition: unrestricted case
Introduction
Optimality
conditions
n
(P) min f (x)
xR
where f : R R
A necessary condition is f (x) = 0 where
f (x + t) f (x)
f (x) = lim
t0 t
That means: If x is a solution of (P) then f (x) = 0
Warning!!!
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb
where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)
Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb
where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)
Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb
where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)
Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb
where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)
Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb
where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)
Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
min f (x) conditions
xR
(P)
axb
where f : R R
At a point x observe that f increase in the sense of (the
sign of) f (x)
Observe that if f (a) > 0 and f (b) < 0, then there exists
a < x < b such that f (x) = 0
Optimality conditions
Necessary conditions: restrictions
Introduction
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
min f (x)
xX
(P) subject to
xC
Optimality
conditions
min f (x)
xX
(P) subject to
xC
n Optimality
conditions
(P) min f (x)
xR
n Optimality
conditions
(P) min f (x)
xR
n Optimality
conditions
(P) min f (x)
xR
n Optimality
conditions
(P) min f (x)
xR
n Optimality
conditions
(P) min f (x)
xR
Optimality
conditions
f (x) = 0
(
(S)
f (x) > 0
Optimality
conditions
f (x) = 0
(
(S)
f (x) > 0
Decision variable(s): x X
Objective function: f : X R
Restrictions: C X
Decision variable(s): x X
Objective function: f : X R
Restrictions: C X
Decision variable(s): x X
Objective function: f : X R
Restrictions: C X
Decision variable(s): x X
Objective function: f : X R
Restrictions: C X
Decision variable(s): x X
Objective function: f : X R
Restrictions: C X
Optimality
min f (x) conditions
xX
(P) subject to
xC
Optimality
min f (x) conditions
xX
(P) subject to
xC
Optimality
min f (x) conditions
xX
(P) subject to
xC