Scribd Logo
Upload

Welcome to Scribd!

  • 39 views

    Week 4

    Uploaded by

    Mohmmad Break
    The document discusses necessary and sufficient conditions for unconstrained and equality constrained optimization problems. It provides: 1) Definitions of necessary conditions (NC), which must be satisfied at an optimum point but don't guarantee optimality, and sufficient conditions (SC), which guarantee optimality if satisfied. 2) The conditions are different for unconstrained versus constrained problems, where in constrained problems optima must satisfy constraints. 3) For unconstrained problems, first and second order NC involve finding stationary points where the first derivative is zero and the second derivative is positive. SC require the second derivative be positive. 4) For equality constrained problems, the Lagrange multiplier theorem provides NC involving setting derivatives of the cost function

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Week 4

    Uploaded by

    Mohmmad Break
    39 views15 pages
    The document discusses necessary and sufficient conditions for unconstrained and equality constrained optimization problems. It provides: 1) Definitions of necessary conditions (NC), which must be satisfied at an optimum point but don't guarantee optimality, and sufficient conditions (SC), which guarantee optimality if satisfied. 2) The conditions are different for unconstrained versus constrained problems, where in constrained problems optima must satisfy constraints. 3) For unconstrained problems, first and second order NC involve finding stationary points where the first derivative is zero and the second derivative is positive. SC require the second derivative be positive. 4) For equality constrained problems, the Lagrange multiplier theorem provides NC involving setting derivatives of the cost function

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses necessary and sufficient conditions for unconstrained and equality constrained optimization problems. It provides: 1) Definitions of necessary conditions (NC), which must be satisfied at an optimum point but don't guarantee optimality, and sufficient conditions (SC), which guarantee optimality if satisfied. 2) The conditions are different for unconstrained versus constrained problems, where in constrained problems optima must satisfy constraints. 3) For unconstrained problems, first and second order NC involve finding stationary points where the first derivative is zero and the second derivative is positive. SC require the second derivative be positive. 4) For equality constrained problems, the Lagrange multiplier theorem provides NC involving setting derivatives of the cost function

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    39 views15 pages

    Week 4

    Uploaded by

    Mohmmad Break
    The document discusses necessary and sufficient conditions for unconstrained and equality constrained optimization problems. It provides: 1) Definitions of necessary conditions (NC), which must be satisfied at an optimum point but don't guarantee optimality, and sufficient conditions (SC), which guarantee optimality if satisfied. 2) The conditions are different for unconstrained versus constrained problems, where in constrained problems optima must satisfy constraints. 3) For unconstrained problems, first and second order NC involve finding stationary points where the first derivative is zero and the second derivative is positive. SC require the second derivative be positive. 4) For equality constrained problems, the Lagrange multiplier theorem provides NC involving setting derivatives of the cost function

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 15

    Arora 4.3-4.

    5
    – Necessary and sufficient conditions
    – Unconstrained problems
    – Equality constrained problems

    [email protected]
    1
    4.3 Necessary and Sufficient
    Conditions
    • Necessary Conditions (NC)
    – Must be satisfied at optimum point
    – Do not guarantee optimality
    – Points that satisfy necessary conditions are called
    candidate points
    – If a point does not satisfy the necessary conditions (is not a
    candidate point), it cannot be the optimum
    • Sufficient Conditions (SC)
    – A candidate point that satisfies the sufficient conditions is
    optimum
    – If the sufficient conditions are not satisfied, we cannot
    always conclude that the candidate point is not optimum

    2
    Unconstrained vs. Constrained
    Optimum Design
    In unconstrained optimization, every point is
    feasible
    In constrained optimization, minima outside of
    the feasible range are not solutions

    3
    Unconstrained vs. Constrained Optimum Design
    Unconstrained Constrained
    Equality Constrained
    x2 f(x1,x2)=1
    x2
    f(x1,x 2)=2
    f(x1,x2)=3
    Feasible f(x1,x2)=4
    Region x1 x1

    h(x1,x2)=0

    Inequality constrained
    f(x1,x2)=2
    x2 Feasible
    f(x1,x 2)=3
    Region
    f(x1,x2)=4

    x1
    4
    4.4 Optimality Conditions for
    Unconstrained Problems
    Minimize f(x) without any constraints on x
    • We use optimality conditions to
    – Check whether a point is a local minimum
    – Solve for locally optimal points
    • To obtain local optimality conditions
    – Assume x* is a local minimum
    – Examine the function and its derivatives in a small
    neighborhood about x*
    – If we move a small distance away, f(x) will not
    decrease

    5
    Necessary Conditions
    First order necessary condition for a local minimum
    (maximum) at x*
    𝑓 ′ 𝒙∗ = 0
    Stationary points
    – Local minima
    – Local maxima
    – Inflection points

    Second order necessary condition for a local


    minimum at x*
    𝑓 ′′ 𝒙∗ ≥ 0
    6
    Sufficient Conditions
    Sufficient condition for a point that satisfies the
    first order necessary condition to be a local
    minimum
    𝑓 ′′ 𝒙∗ > 0

    Note, if 𝑓 ′′ 𝒙∗ = 0, we need to evaluate


    higher-order derivatives

    7
    NCs and SCs
    • The necessary conditions must be satisfied at
    a minimum point.
    • The necessary conditions may also be
    satisfied at points that are not minima. A point
    that satisfies the necessary conditions is
    simply a candidate local minimum.
    • If the sufficient condition is satisfied at a
    candidate point, then it is a minimum.

    8
    Optimality Conditions for Functions of Several Variables

    First order necessary condition


    ❑𝑓 𝒙∗ = 0
    or
    𝜕𝑓 𝒙∗
    = 0 for 𝑖 = 1 𝑡𝑜 𝑛
    𝜕𝑥 𝑖

    Second order necessary condition


    𝜕2𝑓
    𝐻 𝒙∗ = is positive semidefinite or positive definite at 𝒙∗
    𝜕𝑥 𝑖 𝜕𝑥 𝑗
    𝑛×

    Second order sufficient condition


    If 𝐻 𝒙∗ is positive definite at 𝒙∗, then 𝒙∗ is a local minimum

    Other
    If 𝐻 𝒙∗ is indefinite at 𝒙∗, then 𝒙∗ is an inflection point

    9
    4.5 Necessary Conditions for Equality-
    Constrained Problems
    A point 𝒙∗ satisfying the constraints 𝒉(𝒙∗) = 0 is a
    regular point of the feasible set if
    • 𝒇(𝒙∗) is differentiable and
    • gradient vectors of all active constraints at 𝒙∗ are
    linearly independent
    Note: If the problem includes inequality constraints,
    the gradients of all active constraints must be linearly
    independent

    10
    Lagrange Multiplier Theorem
    Consider the equality constrained optimization problem

    Minimize 𝑓 𝒙
    Subject to ℎ𝑖 𝒙 = 0; 𝑖 = 1 𝑡𝑜 𝑝

    Let 𝒙∗ be a regular point and a local minimum for the problem.


    Then there exist unique Lagrange multipliers 𝑣 ∗ ; 𝑗 = 1 𝑡𝑜 𝑝
    such that

    𝜕𝑓 𝒙∗ 𝑝 ∗ 𝜕ℎ 𝑗 𝒙∗
    + σ 𝑗=1 𝑣𝑗 = 0; 𝑖 = 1 𝑡𝑜 𝑛
    𝜕 𝜕
    ℎ𝑗 𝒙∗ = 0; 𝑗 = 1 𝑡𝑜 𝑝

    11
    Properties of Lagrange Function

    • Lagrange function is stationary with respect to 𝒙 and 𝒗, so it can be


    treated as an unconstrained function in 𝒙 and 𝒗 to determine
    stationary points
    • Points that satisfy the necessary conditions are candidate points
    (may be minimum, maximum, or inflection points)
    • We need 2nd order NCs and SCs to distinguish between them
    • In equality constrained problems, Lagrange multipliers are free in
    sign
    • The gradient of the cost function is a linear combination of the
    gradients of the constraints at the candidate point, with 𝑣∗ as the
    scalars of the linear combinations
    • Often, NCs of the Lagrange Multiplier Theorem lead to a set of
    nonlinear equations that can’t be solved analytically

    12
    4.4 Examples
    1. Find local minima for
    𝑓 𝑥 = cos(𝑥)

    2. Find local min and max points for


    𝑓 𝑥 = 𝑥 3 − 𝑥 2 − 4𝑥 + 4

    13
    4.5 Examples
    1. Maximize
    𝑓 𝑥1 , 𝑥2 , 𝑥3 = 𝑥1𝑥2 + 𝑥2𝑥3 + 𝑥1𝑥3
    Subject to
    ℎ 𝑥1, 𝑥2, 𝑥3 = 𝑥1 + 𝑥2 + 𝑥3 − 9 = 0

    2. Minimize
    𝑓 𝑥1, 𝑥2 = 𝑥12 − 2𝑥1𝑥2 + 3𝑥22 − 4
    Subject to
    ℎ 𝑥1, 𝑥2 = −𝑥1 + 𝑥2 − 2 = 0

    14
    Example: Minimum Cost Cylindrical
    Tank Design
    Design a minimum-cost cylindrical tank closed at both
    ends to contain a fixed volume V. The cost depends
    directly on the area of sheet metal used.
    Let
    C = $ per unit area of sheet metal
    R = tank radius
    H = tank height

    15

    You might also like