Gradient Vectors

Extrema of Functions of Two Variables

Optimization of Functions of Several Variables

Math 301
Lecture 5

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Lecture Outline

1 Gradient Vectors

2 Extrema of Functions of Two Variables

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Recommended Reading

Stewart’s Calculus: the titles covered in sections 14.6 and 14.7

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Significance of the Gradient Vector

Recall Example
Find the direction in which f (x, y ) = x 2 /2 + y 2 /2
The gradient vector of f (x, y ) is
✓ ◆ (a) increases most rapidly at the point (1, 1), and
@f @f @f @f
rf = , = i+ j.
@x @y @x @y (b) decreases most rapidly at (1, 1)
The directional derivative of f (x, y ) in the direction of (c) What are the directions of zero change in f at (1, 1).
the unit vector u is Du f (P) = rf (P).u Solution
Remark (a) The function increases most rapidly in the direction of
If u is a unit vector,
rf (1, 1) = (x, y ) = (1, 1) = i + j. The direction is
(1,1)
Du f (P) = rf (P).u = |rf (P)| cos ✓ i+j 1 1
given by the unit vector u = = p i+ p j
|i + j| 2 2
where ✓ is the angle between rf (P) and u. (b) The function decreases most rapidly in the direction of
Since Du f (P) captures the rate of change in the direction of u, it rf at (1, 1) which is given by the unit vector u.
follows that (c) The directions of zero change at (1, 1) are the directions
(i) At any point P, f increases most rapidly in the direction orthogonal to rf , namely the directions orthogonal to u.
✓ = 0, that is, when u is in the direction of rf (P). The These would be
rate of change in this direction is n= p1 i + p1 j and n = p1 i p1 j.
2 2 2 2
Du f (P) = |rf (P)|.
(ii) Similarly, at any point P, f decreases most rapidly in the
direction ✓ = ⇡, that is, in the direction of rf (P).
The rate of change in this direction is
Du f (P) = |rf (P)|.
(iii) Any direction u orthogonal to rf 6= 0 is a direction of
zero change in f because in that case, ✓ = ⇡/2 and
consequently Du f = 0.

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Gradient Vectors
Remarks
Remarks
(i) A vector is normal to a curve at a point P if it is normal to
(i) Recall that for a function of three variables
the tangent to the curve at P.
f (x, y , z) the gradient of f is defined as
(ii) Gradient vectors are normal to level curves of z = f (x, y ). ✓ ◆
@f @f @f
Assume that f (x, y ) has continuous partial derivatives. At rf = , ,
every point (x0 , y0 ) in the domain of f , the gradient of f is @x @y @z
normal to the level curve through (x0 , y0 ). (ii) Since the directional derivative of f (x, y , z) at P in
the direction of a unit vector u is Du f = rf .u, it
follows all the same that the rf (x, y , z) is the
direction of maximum increase of f at P(x, y , z).
(iii) If f (x, y , z) is a function of three variables, rf is
normal to the level surfaces of f (which means that
rf (a, b, c) is normal to the tangent plane to the
level surface f (x, y , z) = const at P(a, b, c) on the
surface).

A Rough Proof
Let u be a unit vector tangent at (x0 , y0 ) to the level curve
C of f (x, y ) through this point. u defines the direction of
the level curve C at (x0 , y0 ).
Along the level curve C the function f (x, y ) does not
change its values. Thus, in the direction of this curve at
(x0 , y0 ), the rate of change of f (x, y ) is 0. In other words,
the directional of f (x, y ) in the direction u at (x0 , y0 ) is 0.
Since Du f (x0 , y0 ) = rf (x0 , y0 ).u,
Du f (x0 , y0 ) = 0 () rf (x0 , y0 ) and u are orthogonal.

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Gradient Vectors
Example
Find a unit normal vector n of the cone of equation
z 2 = 4x 2 + 4y 2 at the point P(1, 0, 2).
Solution
The equation of the cone z 2 = 4x 2 + 4y 2 can be rewritten
as

4x 2 + 4y 2 z 2 = 0.

Therefore, the cone can be thought of as the level surface

for k = 0 of the function

f (x, y , z) = 4x 2 + 4y 2 z2.

Thus, rf = (8x, 8y , 2z) is normal to the cone and a unit

vector normal to the cone at P is
✓ ◆
rf (1, 0, 2) (8, 0, 4) 2 1
n= = = p , 0, p .
|rf (1, 0, 2)| |(8, 0, 4)| 2 5

Remark
n points downwards since its z component is negative. A
unit normal vector to the cone at P that points upwards is
n.

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Recall: Extrema of Single Variable Functions

Remark
Local Extreme Values The condition f 0 (c) = 0 in the first derivative theorem is
Local Maximum only necessary but not sufficient! Meaning that, if we happen
f has a local maximum value at a point x0 in its domain D if to know that x0 is a critical point of f , that is f 0 (x0 ) = 0, it
f (x)  f (x0 ) for all x in D nearby x0 . does not follow automatically that f admits a local extreme
value at x0 . See the illustration in the below figure.
Local Minimum
f has a local minimum value at a point x0 in its domain D if
f (x) f (x0 ) for all x in D nearby x0 .

Second Derivative Test for Local Extrema

Suppose f 00 is continuous on an open interval that contains
x = c.
(i) If f 0 (c) = 0 and f 00 (c) < 0, then f has a local
maximum at x = c.
(ii) If f 0 (c) = 0 and f 00 (c) > 0, then f has a local
The function shown in the graph is defined in D = [a, b] and attains minimum at x = c.
local extreme values at x = a, x = c, x = e, x = d, and x = b.
(iii) If f 0 (c) = 0 and f 00 (c) = 0, then the test fails. The
The First Derivative Theorem for Local Extreme Values function f may have a local maximum, a local
If f has a local maximum or minimum value at an interior point c of its minimum, or neither.
domain, and if f 0 is defined at c, then f 0 (c) = 0.

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Extrema of Functions of Two Variables

Let f be a function of two variables and assume its graph to be shown in the figure. Local extreme values of f can
be defined in a similiar way to those of functions in one variable.

A function of two variables has a local maximum at (a, b) if f (x, y )  f (a, b) when (x, y ) is near (a, b).
The number f (a, b) is called a local maximum value.
A function of two variables has a local minimum at (a, b) if f (x, y ) f (a, b) when (x, y ) is near (a, b).
The number f (a, b) is called a local minimum value.
If the inequalities in the above definitions hold for all (x, y ) in the domain of f , then the extreme value (be
it local minimum or local maximum) is called absolute.
Question
How do we locate points where z = f (x, y ) attains its extreme values?

The answer is provided by the theorem on the following slide.

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Extrema of Functions of Two Variables

The First Derivative Theorem for Local Extreme Values
If f admits a local extreme value at a point (a, b) that is interior to
its domain and if the partial derivatives of f at (a, b) exist there, Remark
then: The first derivative theorem for local extreme
values refers to points interior to the domain.
8
> @f In a planar domain R, points interior to the
< @x (a, b) =0
domain are points that lie inside the domain
>
: @f (a, b) far from its boundary.
@y
=0

Proof
If f (x, y ) attains a local extreme value at (a, b), it follows that the
functions g (x) and h(y ) shown in the below figure attain local
extreme values at x = a and y = b, respectively.
Therefore, g 0 (a) = 0 and h0 (b) = 0 which imply the fact that the
partial derivatives at (a, b) are null.

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Extrema of Functions of Two Variables

critical points
A critical point (or stationary point) of a function of several variables f is one at which the partial
derivatives of f are zero or where one of the partial derivatives does not exist.

Remark
The First Derivative Theorem for Local Extreme Values implies that critical points are the only candidates in the
interior of the domain of f where f would attain its local extrema.
Example
Locate all critical points of f (x, y ) = x 2 y x2 2y 2 .
The partial derivatives are

fx = 2xy 2x = 2x(y 1) fy = x 2 4y .

Setting these equal to zero gives

( (
2x(y 1) = 0 x = 0 or y = 1
() .
x2 4y = 0 x2 = 4y

Substituiting the values in the first line into the equation in the second we obtain:
Case 1: x = 0.
Substituting into x 2 = 4y gives 0 = 4y and hence y = 0. Thus, the critical point for this case is
(x, y ) = (0, 0).
Case 2: y = 1.
Substituting into x 2 = 4y gives x 2 = 4 and hence x = ±2. Thus, we get two critical points for this case:
( 2, 1) and (2, 1).

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Extrema of Functions of Two Variables

Remark
Similar to functions of a single variable, a critical point of a function of Saddle points
several variables does not necessarily yield an extreme value of the Critical points that are solutions of
function, as shown in the adjacent figure. 8
> @f
< @x (a, b) =0

>
: @f (a, b)
@y
=0

but do not yield an extreme value are called

saddle points of the function.

Question
Is there a way to decide whether a critical point of a
function of two variables will yield an extreme value of
the function?
Yes there is, and it is a version for functions of two
Saddle point at (0, 0) Contour map of z = y 2 x2 variables of the second derivative test of functions of
a single variable.
The function z = y 2 x 2 has a critical point at (0, 0) which is interior to
it’s domain (this can be seen by simply observing that the tangents at
(0, 0) to the slices in the x direction and the y direction are horizontal).
However, (0, 0) is neither a highest nor a lowest point on the graph.
Therefore, (0, 0) does not yield a local extreme value.

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Second Derivative Test for Local Extreme Values

Second Derivative Test for Local Extreme Values

Suppose that second partial derivatives of f (x, y ) are continuous and assume that (a, b) is a critical point in
the interior of the domain of the function f (that is, fx (a, b) = 0 and fy (a, b) = 0). Let D be the quantity

D = D(a, b) = fxx (a, b)fyy (a, b) [fxy (a, b)]2 .

Then, if D > 0 is positive,

f has a local minimum at (a, b) if fxx (a, b) > 0.
f has a local maximum at (a, b) if fxx (a, b) < 0.
If D is negative
f has a saddle point at (a, b).
If D = 0, the test fails to inform.

Remark
The expression D = fxx fyy [fxy ]2 is called the discriminant or Hessian of f . It may be remembered as the
determinant,

fxx fxy
D =
fxy fyy

Rami Younes Optimization of Functions of Several Variables

 Gradient Vectors
Extrema of Functions of Two Variables

Second Derivative Test for Local Extreme Values

Example Example

A delivery company accepts only

Find the extreme values, if any, of
rectangular boxes the sum of whose
length and girth (perimeter of a
f (x, y ) = x 2 y x2 2y 2 . cross-section) does not exceed 108 in.
Find the dimensions of an acceptable
Solution box of largest volume.
The partial derivatives of f are
Solution
Let x, y and z represent the length, width and height of the largest
fx = 2xy 2x fy = x 2 4y
volume rectangular box. The girth is 2y + 2z.We wish to find the
maximum volume V = xyz such that x + 2y + 2z = 108.Therefore,
and its critical points have been found on a previous slide to the volume can be rewritten as
be (0, 0), ( 2, 1) and (2, 1).We need the second partial
derivatives in order to test for extreme values at the critical V (y , z) = (108 2y 2z)yz = 108yz 2y 2 z 2yz 2 .
points: fxx = 2y 2 fxy = 2x fyy = 4 Finding critical point of V :

2 2
Vy = 108z 4yz 2z 2 = (108 4y 2z)z = 0
D(x, y ) = fxx fyy fxy = 8y + 8 4x
Vz = 108y 2y 2 4yz = (108 2y 4z)y = 0
give the critical points (0, 0), (0, 54), (54, 0), and (18, 18).The
(i) At the point (0, 0): D = 8. Because D > 0 and volume is 0 at (0, 0), (0, 54), (54, 0), obviously not a maximum value.
fxx (0, 0) = 2 < 0, the second derivative test tells
us that f attains a local maximum at (0, 0). Applying the second derivative test at (18, 18):
2
D = Vyy Vzz Vyz = ( 4z)( 4y ) (108 4y 4z)2 which
(ii) At the point (2, 1):D = 16. Since D < 0, f has a
implies that D(18, 18) = 3888 > 0. Since Vyy (18, 18) = 72 < 0,
saddle point at (2, 1) (neither max nor min).
we conclude that the (18, 18) gives a maximum value.
(iii) At the point ( 2, 1): D = 16. Since D < 0, f
Therefore, the dimensions of the maximum volume acceptable box are
has a saddle point at ( 2, 1)(neither max nor min). x = 108 2(18) 2(18) = 36 in, y = 18 in and z = 18 in.

Rami Younes Optimization of Functions of Several Variables