Lecture 2 - Optimization With Equality Constraints
Lecture 2 - Optimization With Equality Constraints
Constrained optimization
3
In general cases we do not know whether a constraint will be
binding.
Sometime we can use our economic understanding to tell us if
a constraint is binding
– Example: a non-satiated consumer will always spend all
her income, so the budget constraint will be satisfied with
equality
4
Objectives and constraints - example
A firm chooses output x to maximize a profit function
π = -x2 + 10x - 6
Because of a staff shortage, it cannot produce an output higher than x = 4
What are the objective and constraint functions?
The objective function: π = -x2 + 10x-6
The constraint: x ≤ 4
x
4
5
Note that without the constraint the optimum is x = 5
So the constraint is binding (but a constraint of, say, x ≤ 6 would not be)
x
4 5
6
Sometime in the following (and in the textbook) we
denote:
𝑑𝑓 𝑥, 𝑦
= 𝑓1 𝑥, 𝑦
𝑑𝑥
𝑑𝑓 𝑥, 𝑦
= 𝑓2 𝑥, 𝑦
𝑑𝑦
more in general
𝑑𝑓 𝑥1 , 𝑥2 , … 𝑥𝑗 … 𝑥𝑛
= 𝑓𝑗 𝑥1 , 𝑥2 , … 𝑥𝑗 … 𝑥𝑛
𝑑𝑥𝑗
7
Constrained optimization with two variables and one
constraint
The problem is:
max 𝑓(𝑥, 𝑦)
𝑥,𝑦
𝑠. 𝑡 𝑔 𝑥, 𝑦 = 𝑐 𝑥, 𝑦 ∈ 𝑆
To get the solution we have to write the Lagrangean:
𝐿(𝑥, 𝑦, 𝜆) = 𝑓 (𝑥, 𝑦) − 𝜆(𝑔(𝑥, 𝑦) − 𝑐)
𝑓1 𝑥 ∗ , 𝑦 ∗ 𝑓2 𝑥 ∗ , 𝑦 ∗
∗ ∗
=
𝑔1 𝑥 , 𝑦 𝑔2 𝑥 ∗ , 𝑦 ∗
x 9
𝑓1 𝑥 ∗ , 𝑦 ∗ 𝑓2 𝑥 ∗ , 𝑦 ∗
∗ ∗
= ∗ ∗
=𝜆
𝑔1 𝑥 , 𝑦 𝑔2 𝑥 , 𝑦
𝑓2 𝑥 ∗ ,𝑦 ∗
Using = 𝜆 we get 𝑓1 𝑥 ∗ , 𝑦 ∗ − 𝜆𝑔1 𝑥 ∗ , 𝑦 ∗ = 0
𝑔2 𝑥 ∗ ,𝑦 ∗
𝑓1 𝑥 ∗ ,𝑦 ∗
Using = 𝜆 we get 𝑓2 𝑥 ∗ , 𝑦 ∗ − 𝜆𝑔2 𝑥 ∗ , 𝑦 ∗ = 0
𝑔1 𝑥 ∗ ,𝑦 ∗
Moreover the solution has to satisfy the constraint 𝑔 𝑥 ∗ , 𝑦 ∗ = 𝑐
Then the solution has to satisfy the following three equations:
𝑓1 𝑥 ∗ , 𝑦 ∗ − 𝜆𝑔1 𝑥 ∗ , 𝑦 ∗ = 0
𝑓2 𝑥 ∗ , 𝑦 ∗ − 𝜆𝑔2 𝑥 ∗ , 𝑦 ∗ = 0
𝑔 𝑥∗, 𝑦∗ = 𝑐
These equations are the derivatives of the Lagrangean
𝐿(𝑥, 𝑦, 𝜆) = 𝑓 (𝑥, 𝑦) − 𝜆(𝑔(𝑥, 𝑦) − 𝑐)
with respect to 𝑥, 𝑦 and 𝜆 to be zero
10
The first two equations are know as the first order conditions
Proposition (necessary conditions)
Let 𝑓 and 𝑔 be continuously differentiable functions of two
variables defined on the set S, 𝑐 be a number.
Suppose that:
• (𝑥 ∗ , 𝑦 ∗ ) is an interior point of S that solves the problem
max 𝑓(𝑥, 𝑦) 𝑠. 𝑡 𝑔 𝑥, 𝑦 = 𝑐 𝑥, 𝑦 ∈ 𝑆
𝑥,𝑦
∗ ∗
• 𝑒𝑖𝑡ℎ𝑒𝑟 𝑔1 𝑥 , 𝑦 ≠ 0 𝑜𝑟 𝑔2(𝑥 ∗ , 𝑦 ∗ ) ≠ 0.
Then there is a unique number 𝜆 such that (𝑥 ∗ , 𝑦 ∗ ) is a stationary
point of the Lagrangean
𝐿(𝑥, 𝑦) = 𝑓 (𝑥, 𝑦) − 𝜆(𝑔(𝑥, 𝑦) − 𝑐)
That is, (𝑥 ∗ , 𝑦 ∗ ) satisfies the first-order conditions.
𝐿1(𝑥 ∗ , 𝑦 ∗ ) = 𝑓1 (𝑥 ∗ , 𝑦 ∗ ) − 𝜆𝑔1(𝑥 ∗ , 𝑦 ∗ ) = 0
𝐿2(𝑥 ∗ , 𝑦 ∗ ) = 𝑓2 (𝑥 ∗ , 𝑦 ∗ ) − 𝜆𝑔2(𝑥 ∗ , 𝑦 ∗ ) = 0.
and 𝑔(𝑥 ∗ , 𝑦 ∗ ) = 𝑐. 11
Procedure for the solution
12
Example 1
max 𝑥 𝑎 𝑦 𝑏 𝑠. 𝑡 𝑥 + 𝑦 = 𝑐
𝑥,𝑦
10𝑎 10𝑏
4. Then the solution of the problem is 𝑥 = 𝑦=
𝑎+𝑏 𝑎+𝑏
14
Interpretation of λ
𝜕𝑓 ∗ (𝑐)
= 𝜆∗ (𝑐)
𝜕𝑐
the value of the Lagrange multiplier at the solution of the
problem is equal to the rate of change in the maximal value of
the objective function as the constraint is relaxed.
15
Example 2:
max 𝑥2
𝑥
𝑠. 𝑡. 𝑥=𝑐
solution is 𝑥 = 𝑐 so the maximized value of the objective
function is 𝑐2.
Its derivative respect to 𝑐 is 2𝑐
Now consider the Lagrangean
𝐿 𝑥 = 𝑥2 − 𝜆(𝑥 − 𝑐)
The FOC is 2𝑥 − 𝜆 = 0.
Then 𝑥 = 𝑐 and 𝜆 = 2𝑐 satisfy FOC and the constraint.
Note that 𝜆 is equal to the derivative of the maximized value of
the function with respect to 𝑐
16
From example 1:
𝑐𝑎 𝑐𝑏 𝑎𝑎 𝑏𝑏
𝑥= 𝑦= 𝜆= ∙ 𝑐 𝑎+𝑏−1
𝑎+𝑏 𝑎+𝑏 𝑎+𝑏 𝑎+𝑏−1
𝑎𝑎 𝑏𝑏
its derivative respect to 𝑐 is ∙ 𝑐 𝑎+𝑏−1 , i.e. 𝜆
𝑎+𝑏 𝑎+𝑏−1
17
Conditions under which a stationary point is a local optimum
max 𝑓(𝑥, 𝑦)
𝑥,𝑦
𝑠. 𝑡 𝑔 𝑥, 𝑦 = 𝑐
𝐿 𝑥, 𝑦 = 𝑓 𝑥, 𝑦 − 𝜆(𝑔 𝑥, 𝑦 − 𝑐)
0 𝑔1 (𝑥, 𝑦) 𝑔2 (𝑥, 𝑦)
= 𝑔1 (𝑥, 𝑦) 𝑓11 𝑥, 𝑦 − 𝜆𝑔11 (𝑥, 𝑦) 𝑓12 𝑥, 𝑦 − 𝜆𝑔12 (𝑥, 𝑦)
𝑔2 (𝑥, 𝑦) 𝑓21 𝑥, 𝑦 − 𝜆𝑔21 (𝑥, 𝑦) 𝑓22 𝑥, 𝑦 − 𝜆𝑔22 (𝑥, 𝑦)
18
Suppose that it exists a value 𝜆∗ such that 𝑥 ∗ , 𝑦 ∗ is a
stationary point of the Lagrangean.
To check if it is a local maximum
1) Compute the bordered Hessian at the values 𝑥 ∗ , 𝑦 ∗ , 𝜆∗ ,
i.e. 𝐻𝑏 𝑥 ∗ , 𝑦 ∗ , 𝜆∗
2) Compute its determinant, i.e |𝐻𝑏 𝑥 ∗ , 𝑦 ∗ , 𝜆∗ |
3) If |𝐻𝑏 𝑥 ∗ , 𝑦 ∗ , 𝜆∗ | > 0 𝑡ℎ𝑒𝑛 𝑥 ∗ , 𝑦 ∗ is a local maximizer
19
Example 3
max 𝑥 3 𝑦 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑥 + 𝑦 = 6 𝑥, 𝑦 > 0
𝑥,𝑦
𝐿 𝑥, 𝑦 = 3 ln 𝑥 + ln 𝑦 − 𝜆(𝑥 + 𝑦 − 6)
FOC are:
3 1
− 𝜆 = 0, −𝜆 =0 𝑥 + 𝑦 = 6
𝑥 𝑦
2
The solution is 𝑥 = 4.5, 𝑦 = 1.5, 𝜆 =
3
20
Borderd Hessian of the Lagrangean is
0 1 1 0 1 1
3 3
1 − 2 0
=1 − 0
2
𝐻𝑏 𝑥, 𝑦, 𝜆 = 𝑥 𝐻𝑏 4.5, 1.5, 4.52
1 3 1
1 0 − 2 1 0 −
𝑦 1.52
3 1
The determinant is + 2 > 0, then the solution is a local
4.52 1.5
maximizer
21
Conditions under which a stationary point is a
global optimum
22
Example 4
Consider example 3
max 𝑥 3 𝑦 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑥 + 𝑦 = 6 𝑥, 𝑦 > 0
𝑥,𝑦
2
We found that the solution of the FOC 𝑥 = 4.5, 𝑦 = 1.5, 𝜆 =
3
is a local maximizer. Is it a global maximizer?
For a global maximizer we need that Lagrangean is concave
𝐿 𝑥, 𝑦 = 3 ln 𝑥 + ln 𝑦 − 𝜆(𝑥 + 𝑦 − 6)
Given that constraint is linear we need to check the objective
function
The Hessian of the objective function is
3
− 2 0
𝑥
𝐻= 1
0 − 2
𝑦 23
3
− 2 0
𝑥
𝐻= 1
0 − 2
𝑦
The leading principal minors are:
3 3 1
𝐷1 = − 2 < 0 and 𝐷2 = > 0 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑥, 𝑦 > 0
𝑥 𝑥2 𝑦2
Then the Hessian is negative definite, so the objective
function is strictly concave, and the point 𝑥 = 4.5, 𝑦 = 1.5 is a
global maximum.
24
Optimization with equality constraints: n variables, m
constraints: necessary conditions
𝜕𝑔1 𝜕𝑔1
⋯
𝜕𝑥1 𝜕𝑥𝑛
𝐽= ⋮ ⋱ ⋮
𝜕𝑔𝑚 𝜕𝑔𝑚
⋯
𝜕𝑥1 𝜕𝑥𝑛
25
Then there are unique numbers 𝜆1, … , 𝜆𝑚 such that 𝑥 ∗ is a
stationary point of the Lagrangean function 𝐿 defined by
𝑚
𝐿(𝑥) = 𝑓(𝑥) − 𝜆𝑗 𝑔𝑗 𝑥 − 𝑐𝑗
𝑗=1
26
Conditions under which necessary conditions are sufficient
28
Interpretation of λ
29
Quasi-concave functions
30
It is quasiconvex if:
𝑓(𝑥″) = 𝑓(𝜆𝑥 + (1 − 𝜆)𝑥’ ) ≤ max(𝑓(𝑥), 𝑓(𝑥’) )
Note that a convex function is also quasi-convex
The bottom left picture shows that the opposite is not true
y
y
31
The importance of concavity and quasi-concavity
Consider the problem
max 𝑓(𝑥) 𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝑔𝑗(𝑥) = 𝑐𝑗 𝑓𝑜𝑟 𝑗 = 1, … , 𝑚
𝑥
and let 𝑥∗ be a stationary point of the lagrangean
If
1. 𝑓(𝑥) is explicitly quasi-concave
2. The constrained set is convex
3. then 𝑥 ∗ is a global maximum
If
1. 𝑓(𝑥) is strictly quasi-concave
2. The constrained set is convex
3. then 𝑥 ∗ is the unique global maximum 32
Convex sets.
A convex set, X, is such that for any two elements of the set, x
and x’ any convex combination of them is also a member of
the set.
x'
33
e.g. convex but not strictly convex
Convex sets.
If, for any two points in the set S, the line segment
connecting these two points lie entirely in S, then S is a
convex set.
x2 p1x1+p2x2 ≤ m x2 U(x)≥ U
x1 x
34 1
Non-Convex sets.
U
x2
U(x)≤ U
x1
35
A different definition of quasi concavity
Let 𝑓 be a multivariate function defined on the set 𝑆.
𝑓 is quasi concave if, for any number 𝑎, the set of points for
which 𝑓(𝑥) ≥ 𝑎 is convex.
0 𝑓1 𝑥 𝑓2 𝑥 𝑓𝑛 𝑥
𝑓1 𝑥 𝑓11 𝑥 𝑓12 𝑥 𝑓1𝑛 𝑥
𝐵 = 𝑓2 𝑥 𝑓21 𝑥 𝑓22 𝑥 𝑓2𝑛 𝑥
0 𝑓1 𝑥 𝑓2 𝑥
0 𝑓1 𝑥
|𝐵1 | = , |𝐵2 | = 𝑓1 𝑥 𝑓11 𝑥 𝑓12 𝑥 ,……………
𝑓1 𝑥 𝑓11 𝑥
𝑓2 𝑥 𝑓21 𝑥 𝑓22 𝑥
38
If 𝑓 is quasi concave then
|𝐵𝑥 | ≥ 0 if 𝑥 is even and
|𝐵𝑥 | ≤ 0 if 𝑥 is odd
39
Envelope theorem: unconstrained problem
Let 𝑓(𝑥, 𝑟) be a continuously differentiable function where 𝑥
is an n-vector of variables and 𝑟 is a k-vector of
parameters.
40
If the solution of the maximization problem is a continuously
differentiable function of r then:
𝑑𝑓 ∗ 𝑟 𝑑𝑓 𝑥, 𝑟
= evaluated in 𝑥 ∗ (𝑟),
𝑑𝑟𝑖 𝑑𝑟𝑖
the change in the maximal value of the function as a
parameter changes is the change caused by the direct
impact of the parameter on the function, holding the value
of x fixed at its optimal value;
the indirect effect, resulting from the change in the optimal
value of x caused by a change in the parameter, is zero
41
Example 6
max 𝑝 ln 𝑥 − 𝑐𝑥
𝑝
FOC is − 𝑐 = 0
𝑥
∗ 𝑝
then 𝑥 =
𝑐
𝑝
and 𝑓 ∗ 𝑝, 𝑐 = 𝑝 ln − 𝑝
𝑐
The effect of a change of parameter c on the maximized
value is:
𝑑𝑓 ∗ 𝑝, 𝑐 𝑝
=−
𝑑𝑐 𝑐
Consider the derivative of the objective function evaluated
at the solution 𝑥 ∗
𝑑𝑝 ln 𝑥 − 𝑐𝑥
= −𝑥
𝑑𝑐
𝑝 𝑝
Evaluating it in 𝑥 ∗ = we get −
𝑐 𝑐 42
Envelope theorem: constrained problems
43
Example 7 max 𝑥𝑦 𝑠. 𝑡 𝑥 + 𝑦 = 𝐵
𝑥,𝑦
𝐿 𝑥, 𝑦, 𝜆 = 𝑥𝑦 − 𝜆 𝑥 + 𝑦 − 𝐵
we solve: 𝑦−𝜆 =0
𝑥−𝜆 =0
𝑥+𝑦 =𝐵
𝐵 𝐵2
then 𝑥∗ = 𝑦∗ = 𝜆∗ = and 𝑓∗ 𝐵 =
2 4
The effect of a change of parameter c on the maximized
value is:
𝑑𝑓 ∗ 𝐵 𝐵
=
𝑑𝐵 2
Consider the derivative of the Lagrangean evaluated at the
solution 𝑥 ∗
𝑑 𝑥𝑦 − 𝜆 𝑥 + 𝑦 − 𝐵 𝐵
=
𝑑𝐵 2 44