0% found this document useful (0 votes)
72 views28 pages

Optimization Using Calculus: Stationary Points: Functions of Single and Two Variables

This document discusses optimization methods for functions of single and two variables. It defines stationary points as points where the derivative of a function is equal to zero. For a function of a single variable, a necessary condition for a relative maximum is that the derivative is equal to zero at that point. A sufficient condition also requires that subsequent derivatives are equal to zero. Examples are provided to demonstrate finding maxima, minima, and inflection points for various functions. Optimization of functions with two variables is illustrated using contour maps.

Uploaded by

tejas m
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
72 views28 pages

Optimization Using Calculus: Stationary Points: Functions of Single and Two Variables

This document discusses optimization methods for functions of single and two variables. It defines stationary points as points where the derivative of a function is equal to zero. For a function of a single variable, a necessary condition for a relative maximum is that the derivative is equal to zero at that point. A sufficient condition also requires that subsequent derivatives are equal to zero. Examples are provided to demonstrate finding maxima, minima, and inflection points for various functions. Optimization of functions with two variables is illustrated using contour maps.

Uploaded by

tejas m
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

Optimization using Calculus

Stationary Points:
Functions of Single
and Two Variables

1 D Nagesh Kumar, IISc Optimization Methods: M2L1


Objectives

¾ To define stationary points


¾ Look into the necessary and sufficient conditions for the
relative maximum of a function of a single variable and
for a function of two variables.
¾ To define the global optimum in comparison to the
relative or local optimum

2 D Nagesh Kumar, IISc Optimization Methods: M2L1


Stationary points

¾ For a continuous and differentiable function f(x) a


stationary point x* is a point at which the function
vanishes, i.e. f ’(x) = 0 at x = x*. x* belongs to its domain
of definition.
¾ A stationary point may be a minimum, maximum or an
inflection point

3 D Nagesh Kumar, IISc Optimization Methods: M2L1


Stationary points

Figure showing the three types of stationary points (a) inflection point
(b) minimum (c) maximum

4 D Nagesh Kumar, IISc Optimization Methods: M2L1


Relative and Global Optimum

• A function is said to have a relative or local minimum at x = x* if


f ( x* ) ≤ f ( x + h) for all sufficiently small positive and negative
values of h, i.e. in the near vicinity of the point x.
• Similarly, a point x* is called a relative or local maximum if
f ( x* ) ≥ f ( x + h) for all values of h sufficiently close to zero.
• A function is said to have a global or absolute minimum at x = x* if
f ( x* ) ≤ f ( x ) for all x in the domain over which f(x) is defined.
• Similarly, a function is said to have a global or absolute maximum at
x = x* if f ( x* ) ≥ f ( x ) for all x in the domain over which f (x) is
defined.

5 D Nagesh Kumar, IISc Optimization Methods: M2L1


Relative and Global Optimum …contd.

A1, A2, A3 = Relative maxima


A2 = Global maximum
B1, B2 = Relative minima
B1 = Global minimum

.
A2
Relative minimum is
also global optimum

.
f(x) f(x)

.
A1
.
A3

.
B2
.
B1
x x
a b a b

Fig. 2

6 D Nagesh Kumar, IISc Optimization Methods: M2L1


Functions of a single variable

¾ Consider the function f(x) defined for a≤ x≤b


¾ To find the value of x* ∈ [ a, b] such that x* maximizes f(x) we need
to solve a single-variable optimization problem.
¾ We have the following theorems to understand the necessary and
sufficient conditions for the relative maximum of a function of a
single variable.

7 D Nagesh Kumar, IISc Optimization Methods: M2L1


Functions of a single variable …contd.

¾ Necessary condition : For a single variable function f(x) defined for x


∈ [ a, b] which has a relative maximum at x = x* , x* ∈ [ a, b] if the
derivative f ‘(x) = df(x)/dx exists as a finite number at x = x* then
f ‘(x*) = 0.
¾ We need to keep in mind that the above theorem holds good for
relative minimum as well.
¾ The theorem only considers a domain where the function is
continuous and derivative.
¾ It does not indicate the outcome if a maxima or minima exists at a
point where the derivative fails to exist. This scenario is shown in the
figure below, where the slopes m1 and m2 at the point of a maxima are
unequal, hence cannot be found as depicted by the theorem.
8 D Nagesh Kumar, IISc Optimization Methods: M2L1
Functions of a single variable …contd.

Some Notes:
z The theorem does not consider if
the maxima or minima occurs at
the end point of the interval of
definition.
z The theorem does not say that the
function will have a maximum or
minimum at every point where
f ’(x) = 0, since this condition
f ’(x) = 0 is for stationary points
which include inflection points
which do not mean a maxima or
a minima.
9 D Nagesh Kumar, IISc Optimization Methods: M2L1
Sufficient condition

z For the same function stated above let f ’(x*) = f ”(x*) = . . .


= f (n-1)(x*) = 0, but f (n)(x*) ≠ 0, then it can be said that
f (x*) is
– (a) a minimum value of f (x) if f (n)(x*) > 0 and n is even
– (b) a maximum value of f (x) if f (n)(x*) < 0 and n is even
– (c) neither a maximum or a minimum if n is odd

10 D Nagesh Kumar, IISc Optimization Methods: M2L1


Example 1

Find the optimum value of the function f ( x) = x 2 + 3x − 5


and also state if the function attains a maximum or a
minimum.
Solution
f '( x ) = 2 x + 3 = 0 for maxima or minima.
or x* = -3/2
f ''( x*) = 2 which is positive hence the point x* = -3/2 is a point of
minima and the function attains a minimum value of -29/4 at this point.

11 D Nagesh Kumar, IISc Optimization Methods: M2L1


Example 2

Find the optimum value of the function f ( x) = ( x − 2) 4 and


also state if the function attains a maximum or a minimum
Solution:
f '( x) = 4( x − 2)3 = 0 or x = x* = 2 for maxima or minima.

f ''( x*) = 12( x * −2) 2 = 0 at x* = 2

f '''( x*) = 24( x * −2) = 0 at x* = 2

( )
f ′′′′ x * = 24 at x* = 2

Hence fn(x) is positive and n is even hence the point x = x* = 2 is a point of


minimum and the function attains a minimum value of 0 at this point.
12 D Nagesh Kumar, IISc Optimization Methods: M2L1
Example 3

Analyze the function f ( x) = 12 x5 − 45 x 4 + 40 x3 + 5 and


classify the stationary points as maxima, minima and points
of inflection.
Solution: f '( x) = 60 x 4 − 180 x3 + 120 x 2 = 0
=> x 4 − 3x3 + 2 x 2 = 0
or x = 0,1, 2
Consider the point x = x* = 0
f '' ( x* ) = 240( x* )3 − 540( x* ) 2 + 240 x* = 0 at x * = 0
f ''' ( x* ) = 720( x* ) 2 − 1080 x* + 240 = 240 at x * = 0
13 D Nagesh Kumar, IISc Optimization Methods: M2L1
Example 3 …contd.
Since the third derivative is non-zero, x = x* = 0 is neither a point of
maximum or minimum but it is a point of inflection

Consider x = x* = 1
f '' ( x* ) = 240( x* )3 − 540( x* ) 2 + 240 x* = −60 at x* = 1
Since the second derivative is negative the point x = x* = 1 is a point of
local maxima with a maximum value of f(x) = 12 – 45 + 40 + 5 = 12

Consider x = x* = 2
f '' ( x* ) = 240( x* )3 − 540( x* ) 2 + 240 x* = 240 at x* = 2
Since the second derivative is positive, the point x = x* = 2 is a point of
local minima with a minimum value of f(x) = -11
14 D Nagesh Kumar, IISc Optimization Methods: M2L1
Example 4

The horse power generated by a Pelton wheel is proportional to u(v-u) where


u is the velocity of the wheel, which is variable and v is the velocity of the jet
which is fixed. Show that the efficiency of the Pelton wheel will be maximum
at u = v/2.
Solution: f = K.u (v − u )
∂f
= 0 => Kv − 2Ku = 0
∂u
v
or u =
2
where K is a proportionality constant (assumed positive).
∂2 f v
= −2K which is negative. Hence, f is maximum at u =
∂u 2 u=
v 2
2

15 D Nagesh Kumar, IISc Optimization Methods: M2L1


Functions of two variables

¾ The concept discussed for one variable functions


may be easily extended to functions of multiple
variables.
¾ Functions of two variables are best illustrated by
contour maps, analogous to geographical maps.
¾ A contour is a line representing a constant value of f(x)
as shown in the following figure. From this we can
identify maxima, minima and points of inflection.

16 D Nagesh Kumar, IISc Optimization Methods: M2L1


A contour plot

17 D Nagesh Kumar, IISc Optimization Methods: M2L1


Necessary conditions

¾ As can be seen in the above contour map, perturbations


from points of local minima in any direction result in an
increase in the response function f(x), i.e.
¾ the slope of the function is zero at this point of local
minima.
¾ Similarly, at maxima and points of inflection as the slope
is zero, the first derivative of the function with respect to
the variables are zero.

18 D Nagesh Kumar, IISc Optimization Methods: M2L1


Necessary conditions …contd.

¾ Which gives us ∂f = 0; ∂f = 0 at the stationary points. i.e. the


∂x1 ∂x2

gradient vector of f(X), Δ x f at X = X* = [x1 , x2] defined as follows,


must equal zero:
⎡ ∂f ⎤
⎢ ∂x ( Χ*) ⎥
Δx f = ⎢ ⎥=0
1

⎢ ∂f ⎥
⎢ ∂x ( Χ *) ⎥
⎣ 2 ⎦

This is the necessary condition.

19 D Nagesh Kumar, IISc Optimization Methods: M2L1


Sufficient conditions

¾ Consider the following second order derivatives:


∂2 f ∂2 f ∂2 f
; 2;
∂x1 ∂x2 ∂x1∂x2
2

¾ The Hessian matrix defined by H is made using the above second


order derivatives.
⎛ ∂ f 2 2
∂ f ⎞
⎜ ⎟
∂ x 2
∂x1∂x2 ⎟
H=⎜ 2 1
⎜ ∂ f ∂2 f ⎟
⎜⎜ 2 ⎟
⎝ ∂x1∂x2 ∂x2 ⎟⎠[ x , x ]
1 2

20 D Nagesh Kumar, IISc Optimization Methods: M2L1


Sufficient conditions …contd.

¾ The value of determinant of the H is calculated and


¾ if H is positive definite then the point X = [x1, x2] is a
point of local minima.
¾ if H is negative definite then the point X = [x1, x2] is a
point of local maxima.
¾ if H is neither then the point X = [x1, x2] is neither a
point of maxima nor minima.

21 D Nagesh Kumar, IISc Optimization Methods: M2L1


Example 5

Locate the stationary points of f(X) and classify them as relative


maxima, relative minima or neither based on the rules discussed in the
lecture.
f ( X) = 2 x13 / 3 − 2 x1 x2 − 5 x1 + 2 x22 + 4 x2 + 5

Solution

22 D Nagesh Kumar, IISc Optimization Methods: M2L1


Example 5 …contd.
∂f
From (X) = 0 ,
∂x1

8 x22 + 14 x2 + 3 = 0
(2 x2 + 3)(4 x2 + 1) = 0
x2 = −3 / 2 or x2 = −1/ 4

So the two stationary points are


X1 = [-1,-3/2] and X2 = [3/2,-1/4]

23 D Nagesh Kumar, IISc Optimization Methods: M2L1


Example 5 …contd.
∂2 f ∂2 f ∂2 f ∂2 f
The Hessian of f(X) is = 4 x1 ; 2 = 4; = = −2
∂x12
∂x2 ∂x1∂x2 ∂x2 ∂x1
⎡ 4 x1 −2 ⎤
H=⎢ ⎥
⎣ −2 4 ⎦
λ − 4 x1 2
λI - H =
2 λ −4
λ+4 2
At X1 = [-1,-3/2] , λ I - H = = (λ + 4)(λ − 4) − 4 = 0
2 λ −4
Since one eigen value is positive
λ − 16 − 4 = 0
2
and one negative, X1 is neither
λ1 = + 12 λ2 = − 12 a relative maximum nor a
relative minimum
24 D Nagesh Kumar, IISc Optimization Methods: M2L1
Example 5 …contd.

At X2 = [3/2,-1/4]
λ −6 2
λI - H = = (λ − 6)(λ − 4) − 4 = 0
2 λ −4

λ1 = 5 + 5 λ2 = 5 − 5

Since both the eigen values are positive, X-2 is a local minimum.
Minimum value of f(x) is -0.375

25 D Nagesh Kumar, IISc Optimization Methods: M2L1


Example 6

Maximize f(X) = 20 + 2 x1 − x1 + 6 x2 − 3 x2 / 2
2 2

⎡ ∂f ⎤
⎢ ∂x ( Χ*) ⎥ ⎡ 2 − 2 x ⎤ ⎡0⎤
Δx f = ⎢ ⎥=⎢ =⎢ ⎥
1 1

⎢ ∂ ⎥ ⎣ − ⎥ X* = [1,2]
f 6 3 x2⎦ ⎣ 0⎦
⎢ ∂x ( Χ*) ⎥
⎣ 2 ⎦

∂2 f ∂2 f ∂2 f ⎡ −2 0 ⎤
= −2; 2 = −3; =0; H=⎢ ⎥
∂x12
∂x2 ∂x1∂x2 ⎣ 0 −3 ⎦

26 D Nagesh Kumar, IISc Optimization Methods: M2L1


Example 6 …contd.

λ+2 0
λI - H = = (λ + 2)(λ + 3) = 0
0 λ +3
λ1 = −2 and λ2 = −3
Since both the eigen values are negative, f(X) is concave and
the required ratio x1:x2 = 1:2 with a global maximum strength
of f(X) = 27 MPa

27 D Nagesh Kumar, IISc Optimization Methods: M2L1


Thank you

28 D Nagesh Kumar, IISc Optimization Methods: M2L1

You might also like