VC.

03
Gradient Vectors, Level Curves,
Maximums/Minimums/Saddle Points
 Example 1: The Gradient Vector
df
Let f(x)  x 2 . Then  2x. This can be thought of as a vector that
dx
tells you the direction of greatest increase on the curve. The magnitude
of the vector tells you how steep the increase.
Let's try a few
df
x-values in :
dx

Created by Christopher Grattoni. All rights reserved.

 Example 1: The Gradient Vector
We can put the tails of our vectors on the curve itself
to get picture that's a little easier to work with:
What do you notice about the magnitude of the gradient vector at x=0?

Created by Christopher Grattoni. All rights reserved.

 Example 2: The Gradient Vector
We can try again with f(x)  sin(x) :
 3
What do you notice about the magnitude of the gradient vector at x  and x  ?
2 2

Created by Christopher Grattoni. All rights reserved.

 Definition: The Gradient Vector
Let f( x 1 , x 2 ,..., x n ) be a function of n variables. Then the
gradient vector is defined as follows:

 f f f 
f   , ,..., 
 x 1 x 2 x n 
The gradient vector is designed to point in the direction
of the greatest INITIAL increase on your curve/surface/etc.
Notice that the gradient vector always lives in one dimension
lower than function does. 3D surface? 2D gradient vector. 2D
curve? 1D gradient vector. 4D hypersurface? 3D gradient vector.
Created by Christopher Grattoni. All rights reserved.
 Definition: The Gradient Vector
Let f(x1 , x 2 ,..., x n ) be a function of n variables and let
 f  f f 
f   , ,..., 
 x 1  x 2 x n 

Ex. If f(x)  sin(x), Ex. If f(x, y)  x 3  y 3  3x  3y,

f   cos(x)  
f  3x 2  3,3y 2  3 
Theorem: If f( x 1 , x 2 ,..., x n ) is a function of n variables, then the
candidates for local maxima/minima are where f   0, 0,...,0  .
NOTE : These are just CANDIDATES, not necessarily extrema!!
Created by Christopher Grattoni. All rights reserved.
 Example 3: A Surface and Gradient Field

Let f(x, y)  (cos 2 (x)  cos 2 (y))2 . Then we can use Mathematica
 f f 
to find f(x, y)   , .
 x y 
This is the surface plotted together
with a projected gradient field below it.

The surface is like a mountain

and the gradient vector is like a
magic compass that, no matter where
you are standing on the mountain,
points in the immediate direction that
is the steepest uphill.


f(x, y)  4 cos3 (x)sin(x)  4 cos(x)sin(x)cos(y),2
Created by Christopher Grattoni. All rights reserved. cos (x)sin(y)  2cos(y)sin(y)
2

 Example 3: A Surface and Gradient Field
 f f 
Let f(x, y)  (cos2 (x)  cos 2 (y))2 and f(x, y)   , :
  x y 

This is the surface plotted with its

gradient vectors plotted directly on it.


f(x, y)  4 cos3 (x)sin(x)  4 cos(x)sin(x)cos(y),2
Created by Christopher Grattoni. All rights reserved. cos (x)sin(y)  2cos(y)sin(y)
2

 Example 3: A Surface and Gradient Field
 f f 
Let f(x, y)  (cos2 (x)  cos 2 (y))2 and f(x, y)   , :
  x y 
One of the main reasons we like to plot
the gradient vectors (called a gradient
vector field) is that we can figure out
quite a bit about a 3D surface without
the hard work of a 3D plot:

The vectors tell you the direction of

greatest initial increase on the surface
at a given point, and their magnitude
tells you how steep it is!


f(x, y)  4 cos3 (x)sin(x)  4 cos(x)sin(x)cos(y),2
Created by Christopher Grattoni. All rights reserved. cos (x)sin(y)  2cos(y)sin(y)
2

 Example 4: A Surface and Contour Plot
 f f 
Let f(x, y)  (cos2 (x)  cos 2 (y))2 and f(x, y)   , :
  x y 
We can also generate a "contour plot" of our
surface. This is also called a set of level curves.
These are merely slices of the curve projected
onto the xy-plane:

Created by Christopher Grattoni. All rights reserved.

 Example 4: Contour Plots in Real
Life

This is merely just a technique for representing 3D data in 2D.

Created by Christopher Grattoni. All rights reserved.

 Example 4: A Surface and Contour Plot
 f f 
Let f(x, y)  (cos2 (x)  cos 2 (y))2 and f(x, y)   , :
  x y 

We can put our contour plot

and gradient field together:
Recall that the gradient always
points in the direction of
greatest initial increase, so it
must get you from one level
curve to the next as efficiently
as possible. That is, the
gradient vector is
perpendicular to the level
curve passing through its tail.
Created by Christopher Grattoni. All rights reserved.
 Example 4: A Surface and Contour Plot
 f f 
Let f(x, y)  (cos2 (x)  cos 2 (y))2 and f(x, y)   , :
  x y 
Theorem: The gradient
is always perpendicular
to the level curve
through its tail.

Proof: Later in the notes.

Note : The gradient

is not necessarily
perpendicular to the
surface itself, just its 2D
level curves!
Created by Christopher Grattoni. All rights reserved.
 Example 5: The Gradient Points in the
Direction of Greatest Initial Increase
 f f 
Let f(x, y)  (cos 2 (x)  cos 2 (y))2 and f(x, y)   ,  . Show that if we are at point
  x y 
(1,1,f(1,1)) and move in the direction of the gradient f(1,1), we must go up before we
go down:

Here is a plot of f(x(t),y(t)) versus t

where (x(t),y(t))  (1,1)  tf(1,1)
for 0  t  1 :

Created by Christopher Grattoni. All rights reserved.

 Example 6: A Path Along our Surface

Let f(x, y)  (cos2 (x)  cos2 (y))2 and let (x(t), y(t))  cos2 (1  t), sin(2t ) .  
Finally, plot f(x(t), y(t)), a path on the surface.

The surface is a like a mountain

and the path is a hiking trail on
that mountain!

Created by Christopher Grattoni. All rights reserved.

 Example 6: A Path Along our Surface

Let f(x, y )  (cos2 (x)  cos2 (y))2 and let (x(t), y(t))  cos2 (1  t), sin(2t) .  
Finally, plot f(x(t), y(t)), a path on the surface. Add in the gradient vectors
on the path:

The surface is a like a mountain

and the path is a hiking trail on
that mountain, but the gradient
vectors are basically a magic compass
telling you the steepest direction
from where you are standing. You
can use this to figure out how steep
the road ahead is.
Created by Christopher Grattoni. All rights reserved.
 Example 6: Tangent Vectors and
Gradient Vectors on Our Path
Let f(x , y)  (cos2 (x)  cos2 (y))2 and let
 
( x(t), y(t))  cos2 (1  t), sin(2t) . The gradient
vectors along the path are blue and tangent vectors
are orange.
Let's go point by point and describe
what's happening on the hiking trail:

Created by Christopher Grattoni. All rights reserved.

 Example 6: Tangent Vectors and
Gradient Vectors on Our Path
Let f(x , y)  (cos2 (x)  cos2 (y))2 and let ( x(t), y(t))  cos2 (1  t),sin(2t) .  
The gradient vectors along the path are blue and tangent vectors are orange.
We saw the angle between the gradient,
 f f 
 ,  , and the tangent vector, (x'(t),y'(t)),
 x y 
is the key here! Recall that V  W  V W cos( ) :

 f f 
If the angle between  ,  and (x'(t),y'(t)) is acute,
  x y 
 f f 
 ,   (x'(t),y'(t))>0 and we are walking uphill.
  x y 

 f f 
If the angle between  ,  and (x'(t),y'(t)) is obtuse,
  x y 
 f f 
 ,   (x'(t),y'(t))<0 and we are walking downhill.
  x y  Created by Christopher Grattoni. All rights reserved.
 The Derivative of f(x(t),y(t)) With Respect to t

 f f   f f 
If the angle between  ,  and (x'(t),y'(t)) is acute, If the angle between  ,  and (x'(t),y'(t)) is obtuse,
 x y   x y 
 f f   f f 
 ,   (x'(t),y'(t))>0 and we are walking uphill.  ,   (x'(t),y'(t))<0 and we are walking downhill.
 x y   x y 

Walking uphill is like saying f(x(t),y(t)) is increasing at t.

Walking downhill is like saying f(x(t),y(t)) is decreasing at t.

The 2D Chain Rule:

df(x(t), y(t))  f f 
 ,   (x'(t),y'(t))
dt  x  y 

Created by Christopher Grattoni. All rights reserved.

 The Chain Rule in n-Dimensions

df(x(t), y(t))  f f 
The 2D Chain Rule:  ,   (x'(t),y'(t))
dt   x y 
Chain Rules in Other Dimensions:
df(x(t))
1D :  f '(x(t))x '(t)
dt
df(x(t), y(t), z(t))  f f f 
3D :  , ,   (x'(t),y'(t),z'(t))
dt   x y z 
df(x 1 (t),..., x n (t))  f f 
n-D:   ,...,   (x 1'(t),...,x n'(t))
dt  x 1 x n 
"The derivative of the outside times the derivative of the inside."
(Gradient) (Dot Product) (Tangent Vector)
Created by Christopher Grattoni. All rights reserved.
 Chain Rule Proves the Gradient is
Perpendicular to the Level Curve
Theorem: The gradient is always perpendicular to the level curve through its tail.
Proof: We will only show this for a surface z  f(x,y) whose
level curve c  f(x,y) can be parameterized by (x(t), y(t)).
Then a tangent vector on the level curve can be described
by (x'(t),y'(t)).
 f f 
Next, the gradient is f(x, y)   , .
  x y 
For them to be perpendicular, we want their dot product
to be 0:
 f f  df(x(t), y(t))
 ,   (x'(t), y'(t))  By Chain Rule...
 x y  dt
0 But c  f(x, y), and the
So the gradient is  to the level curve. derivative of a constant is 0...
Another way to think of this is a level curve is defined as a path
df(x(t), y(t))
on the surface where your height stays constant. So  0...
Created by Christopher Grattoni. All rights reserved.
dt
 Example 7: Using the Chain Rule

df
Let f(x, y)  x 2  xy and let (x(t), y(t))   cos(t), sin(t)  . Find :
dt
 f f 
 ,    2x  y, x    2cos(t)  sin(t), cos(t) 
 x y 
(x '(t), y '(t))  (  sin(t), cos(t))

df(x(t), y(t))  f f 
 ,   (x'(t),y'(t))
dt  x y 
  2 cos(t)  sin(t), cos( t )   (  sin(t), cos( t ))
 2 sin(t) cos(t)  sin2 (t)  cos 2 (t)
 cos(2t)  sin(2t)
Created by Christopher Grattoni. All rights reserved.
 Example 8: Identifying Local Extrema
from the Gradient
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 .  
Find all local maxima/minima on this surface.

Theorem: The candidates for

local extrema are where f   0, 0  .

 
f  3x 2  3,3y 2  3  (0, 0)
x  1, y  1

Critical Points:
 1,1  ,  1, 1  ,  1,1  ,  1, 1 
Created by Christopher Grattoni. All rights reserved.
 Example 8: Identifying Local Extrema
from the Gradient
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 .  
Find all local maxima/minima on this surface.

Critical Points:
 1,1  ,  1, 1  ,  1,1  ,  1, 1 
(1, 1) and ( 1,1) are called
saddle points:
They bring you up in one
direction down in the other.
These are not extrema.

Task: Look at the surface in

Mathematica and convince
yourself these are not extrema. Created by Christopher Grattoni. All rights reserved.
 Example 8: Identifying Local Extrema
from the Gradient
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 .  
Find all local maxima/minima on this surface.
Saddle point analogy:

A Saddle Point is to 3D as a Non-Max/Min Horizontal Tangent is to 2D.

Created by Christopher Grattoni. All rights reserved.
 Example 8: Identifying Local Extrema
from the Gradient
 
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 .
Find all local maxima/minima on this surface.
Remaining Candidates:
 1,1  and  1, 1 
The gradient vectors show
that (  1,  1) is a local max
(the surrounding vectors all
point to (  1,  1)).
The gradient vectors show
that (1,1) is a local min
(the surrounding vectors all
point away from (1,1)).
Saddle points have gradient
vectors both flowing in and out.
 Example 8: Identifying Local Extrema
from the Gradient
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 .  
Find all local maxima/minima on this surface.

Summary:
Local Maximum: (  1,  1)
Local Minimum: (1,1)
Saddle Points: (1, 1) and ( 1,1)

Created by Christopher Grattoni. All rights reserved.

Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums
You might wonder how to identify maxima, minima,
and saddle points without having to use Mathematica
to plot the gradient field. For this, we generalize the
second derivative test.
Note: This is not a part of the University of Illinois
Vector Calculus curriculum, but I have included it
because it is a part of most multivariable calculus
curricula, and numerous students have asked me
about this in the past. It will not be on the quiz,
test, or the literacy sheet. But you might see it
next year. Created by Christopher Grattoni. All rights reserved.
Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums
Recall: The second derivative states that if f '(a)  0, then:
If f ''(a)  0, then we have a local maximum at x = a
If f ''(a)  0, then we have a local minimum at x = a
If f ''(a)  0, then the test is inconclusive.
x5 4x3
Example: Let f(x)   :
5 3
f '( 2)  0 & f ''( 2)  0 : local maximum at x  2.
f '(2)  0 & f ''(2)  0 : local minimum at x  2.
f '(0)  0 & f ''(0)  0 : test inconclusive at x  0.

Created by Christopher Grattoni. All rights reserved.

Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums

Now for a maximum, we need to be concave down

in the x-direction AND y-direction. Likewise, for a minimum,
we need to be concave up in the x-direction AND y-direction.
If we mix between the two, we have a saddle point.

The tool that lets us analyze this is the Hessian Matrix:

  2f  2f 
 2 
 x xy 
Hf(x, y) 
  2f  2f 
 
 yx y 
2

Created by Christopher Grattoni. All rights reserved.

Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums

 2f  2f
x 2 xy
Let D(x, y)  Hf(x, y)  2 . Then:
 f  f 2

yx y 2
D(a,b) fxx (a,b) Conclusion
  (a,b, f(a,b)) is a local maximum.
  (a,b, f(a,b)) is a local minimum.
 n/a (a,b, f(a,b)) is a saddle point.
0 n/a The test is inconclusive.
Created by Christopher Grattoni. All rights reserved.
Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums
D(a,b) fxx (a,b) Conclusion
  (a,b, f(a,b)) is a local maximum.
  (a,b, f(a,b)) is a local minimum.
 n/a (a,b, f(a,b)) is a saddle point.
0 n/a The test is inconclusive.

How does this work? Well, if D(a,b) is positive, it is telling us that the "second
derivatives in the x-direction and y-direction" are pointing the same way.
So if D(a,b) is positive and fxx (a,b) is negative, then we are "concave down" in
both directions which is a local maximum. If D(a,b) is positive and fxx (a,b) is
positive, then we are "concave up" in both directions, which is a local minimum.

If D(a,b) is negative, then the x-direction and y-direction second derivatives

are "pointing in different directions," or a saddle point.

If D(a,b) is zero, we can't tell.

Created by Christopher Grattoni. All rights reserved.
Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 with critical  
points at (  1,  1), (1,1), (1, 1), and ( 1,1). Use the Hessian determinant to identify
maxima, minima, and saddle points:
 2f  2f
x 2 xy 6x 0
D(x, y)  Hf(x, y)    36xy
 2f  f 2
0 6y
yx y 2
Test (1,1) : D(a,b) fxx (a,b) Conclusion
  (a,b, f(a,b)) is a local maximum.
D(1,1)  36  0
  (a,b, f(a,b)) is a local minimum.
fxx (1,1)  6x (1,1)  6  0  n/a (a,b, f(a,b)) is a saddle point.
0 n/a The test is inconclusive.

Hence, (1,1,f(1,1)) is a local minimum!

Created by Christopher Grattoni. All rights reserved.
Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 with critical  
points at (  1,  1), (1,1), (1, 1), and ( 1,1). Use the Hessian determinant to identify
maxima, minima, and saddle points:
 2f  2f
x 2 xy 6x 0
D(x, y)  Hf(x, y)    36xy
 2f  f 2
0 6y
yx y 2
Test ( 1, 1) : D(a,b) fxx (a,b) Conclusion
D( 1, 1)  36  0   (a,b, f(a,b)) is a local maximum.
  (a,b, f(a,b)) is a local minimum.
fxx ( 1, 1)  6x ( 1, 1)  6  0
 n/a (a,b, f(a,b)) is a saddle point.
0 n/a The test is inconclusive.

Hence, (  1,  1,f(  1,  1)) is a local maximum!

Created by Christopher Grattoni. All rights reserved.
Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 with critical  
points at (  1,  1), (1,1), (1, 1), and ( 1,1). Use the Hessian determinant to identify
maxima, minima, and saddle points:
 2f  2f
x 2 xy 6x 0
D(x, y)  Hf(x, y)    36xy
 2f  f 2
0 6y
yx y 2

Test (1, 1) : D(a,b) fxx (a,b)

 
Conclusion
(a,b, f(a,b)) is a local maximum.
D(1, 1)  36  0   (a,b, f(a,b)) is a local minimum.
 n/a (a,b, f(a,b)) is a saddle point.
0 n/a The test is inconclusive.

Hence, (1,  1,f(1,  1)) is a saddle point!

Created by Christopher Grattoni. All rights reserved.
Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 with critical  
points at (  1,  1), (1,1), (1, 1), and ( 1,1). Use the Hessian determinant to identify
maxima, minima, and saddle points:
 2f  2f
x 2 xy 6x 0
D(x, y)  Hf(x, y)    36xy
 2f  f 2
0 6y
yx y 2

Test ( 1,1) : D(a,b) fxx (a,b)

 
Conclusion
(a,b, f(a,b)) is a local maximum.
D( 1,1)  36  0   (a,b, f(a,b)) is a local minimum.
 n/a (a,b, f(a,b)) is a saddle point.
0 n/a The test is inconclusive.

Hence, (  1,1,f(  1,1)) is a saddle point!

Created by Christopher Grattoni. All rights reserved.
Detour: The Second Partial Derivative Test
to Identify Maximums and Minimums
Let f(x, y)  x 3  y 3  3x  3y. We previously found f  3x 2  3,3y 2  3 .  
Find all local maxima/minima on this surface.

Summary:
Local Maximum: (  1,  1)
Local Minimum: (1,1)
Saddle Points: (1, 1) and ( 1,1)

Created by Christopher Grattoni. All rights reserved.

 Example 9: Another Visit to Our Surface

Let f(x, y)  x 3  y 3  3x  3y and suppose you are standing on the surface at the
point  2,3,20  . If you decide to walk in the direction of ( 7,2), do you go uphill
or downhill on f(x,y) when you take your first step?


f  3x 2  3,3y 2  3 
f(2,3)   9,24 
 9, 24    7, 2  15
Since this dot product is negative, we know
our first step will be downhill.

This negative tells us the angle between the

gradient at (2,3) and our direction vector is
obtuse. We are walking against the direction
advised by our Gradient Compass: downhill!
 Example 10: Level Surfaces and
Entering the Fourth Dimension

Let f(x, y)  x 3  y3  3x  3y. We previously found f  3x 2  3,3y 2  3 
with a local maximum at (  1,  1) and a local maximum at (1,1).
One idea: ifyou
Pretend your friend
had likes to
a 2-dimensional
swim,
friendyou could ask who
(Pac-Man?) him/her
had to
never
swim on the
visited the contour plot. The How
third dimension.
local
wouldmaxyou
would be where
explain the
it to him?
water is hottest and the minimum
would be the coldest. You've helped
your friend experience the third
dimension as temperature rather
than a spatial dimension.
Swimming  to the level curves towards
the max leads you to the greatest initial
temperature increase.
 Example 10: Level Surfaces and
Entering the Fourth Dimension
Let f(x, y, z)  (x  1)2  (y  2)2  z 2 . We can plot level surfaces of this
4D-surface by letting k  (x  1)2  (y  2)2  z 2 for various values of k.
Task: Look at the level surfaces in
Mathematica by clicking the slider
(don't drag it though)
We could experience 4D like
the swimmer experienced 3D.
We could swim in 3D space
and experience the fourth
dimension as temperature.
The gradient vectors tell
us the path to the greatest
initial temperature increase
from a given point in space. Created by Christopher Grattoni. All rights reserved.
 Example 11: A Different Analogy for the
Fourth Dimension?
Let T(x, y, z)  8(x 4  x 2  y 4  y 2 )  z  2. We can plot level surfaces of this
4D-surface by letting k  8(x 4  x 2  y 4  y 2 )  z  2 for various values of k.
This time, treat T(x,y,z) as a function that
takes a position in space (x,y,z) and outputs
the temperature at that point. So our
swimmer can experience an extra
dimension as the temperature at that
particular point. Level surfaces can be
found by graphing various T(x, y, z)  k.

Task: Look at this surface in Mathematica.

Click (don't drag) around the slider and
think about how a 3D swimmer would
experience this diving tank.
Created by Christopher Grattoni. All rights reserved.
 Example 11: A Different Analogy for the
Fourth Dimension?
Let T(x, y, z)  8(x 4  x 2  y 4  y 2 )  z  2. We can plot level surfaces of this
4D-surface by letting k  8(x 4  x 2  y 4  y 2 )  z  2 for various values of k.
Discussion Questions:
1) Where should the swimmer go
for the coolest locations?
2) The hottest?
3) What does a level surface mean
in the context of this scenario?
4) Given the particular level surface the
swimmer is on, in what direction does
he swim for the greatest initial
temperature increase? Created by Christopher Grattoni. All rights reserved.
 Final Thoughts: f(x,y) versus f(x,y,z)
z = f(x,y) w = f(x,y,z)

Graph 3-D surface 4-D (hyper)surface

Can’t truly graph it!

Level Sets Level Curve Level Surface

k = f(x,y) k = f(x,y,z)

Gradient Vectors 2D Vectors 3D Vectors

 f  f f 
  f f  f   , , 
f   ,   x y z 
  x y 
(2D vectors that are  to (3D vectors that are  to
the level curves, NOT the the level surface, NOT the
3D surface itself)
Created by Christopher Grattoni. All rights reserved.4D surface itself)
Lagrange Multipliers Powerpoint
• G.6 and the Literacy Sheet will require knowledge of
Lagrange multipliers
• I am going to try this a bit differently this year:
• Since you will all get to these problems at different
points, I am going to post the Powerpoint to
Schoology and a link to a YouTube video where you
can view this on your own timeframe
• The quiz will be difficult if you skip this…

Created by Christopher Grattoni. All rights reserved.