Jim Lambers

MAT 419/519
Summer Session 2011-12
Lecture 4 Notes

These notes correspond to Sections 1.4 and 1.5 in the text.

Coercive Functions and Global Minimizers

We now know how to prove that a critical point of a function f (x) is a global minimizer if the
Hessian of f (x) is positive semidefinite on all of Rn (or a strict global minimizer if the Hessian
is positive definite), but we need a way to establish global minimizers even if the Hessian is not
necessarily positive semidefinite on all of Rn . Fortunately, this is easily accomplished in certain
cases.
Previously, we defined a set D ⊆ Rn to be closed if its complement Dc in Rn is open. Intuitively,
a closed set includes its boundary, if it has one, whereas an open set does not. This sets the stage
for the following definitions.
Definition We then say that a set D ⊆ Rn is bounded if there exists a constant M > 0 such that
kxk < M for all x ∈ D. The set D is said to be compact if it is closed and bounded.
Example A closed interval [a, b] is bounded, and is therefore also compact. The circle and its
interior {(x, y) | x2 + y 2 ≤ 1} is a closed set, and is also bounded, and therefore it is compact. The
interval [0, ∞) is closed, as its complement (−∞, 0) is open, but it is not bounded, so it is not
compact either. 2
The following theorem is an essential result concerning the existence of global minimizers and
maximizers on compact sets.
Theorem Let D be a compact subset of Rn . If f (x) is a continuous function on D, then f (x) has
a global maximizer and a global minimizer on D.
We now describe functions for which global minimizers can be found even on sets that are not
bounded or not closed.
Definition A continuous function f (x) that is defined on all of Rn is coercive if

lim f (x) = +∞.

kxk→∞

That is, for any constant M > 0 there exists a constant RM > 0 such that kf (xk > M whenever
kxk > RM .

1
Example The function f (x, y) = x2 + y 2 is coercive, as

lim f (x, y) = lim kxk2 = +∞.

kxk→∞ kxk→∞

2
Example A linear function is never coercive. For instance, a linear function on R2 has the form

f (x, y) = ax + by + c,

for constants a, b and c, and is equal to c along the line defined by the equation ax + by = 0. Since
kxk → ∞ along this line, but f (x, y) = c along this line, f (x, y) is not coercive. 2
Example The function f (x, y) = x2 − 2xy + y 2 is not coercive, as f (x, y) = (x − y)2 , which means
that f (x, y) = 0 on the line y = x, along which kxk becomes infinite. 2
As these examples show, in order for a function to be coercive, it must approach +∞ along any
path within Rn on which kxk becomes infinite.
The following theorem indicates the usefulness of knowing whether a function is coercive.
Theorem Let f (x) be a continuous function defined on all of Rn . If f (x) is coercive, then f (x)
has a global minimizer. Furthermore, if the first partial derivatives of f (x) exist on all of Rn , then
any global minimizers of f (x) can be found among the critical points of f (x).
This theorem can be proved by using the fact that f (x) is coercive to find a compact subset of Rn
on which f (x) must have a global minimizer, by the preceding theorem.
To find the global minimizer of a coercive function f (x), it is sufficient to find the critical points
of f (x), and then evaluate f (x) at each of these points. The critical points for which f (x) assumes
the smallest values are then the global minimizers.
Example Let f (x, y) = x4 − 4xy + y 4 . Then we have

∇f (x, y) = (4x3 − 4y, −4x + 4y 3 ),

and therefore critical points satisfy y = x3 and x = y 3 . That is, x = x9 , which means x = 0, x = 1
or x = −1. This yields the critical points (0, 0), (1, 1) and (−1, −1). However, as the Hessian is

12x2
 
−4
Hf (x, y) = ,
−4 12y 2

which is indefinite at (0, 0), as its determinant is equal to −16.

We instead determine whether f (x, y) is coercive. We have
 
4 4 4xy
f (x, y) = (x + y ) 1 − 4 .
x + y4

2
As k(x, y)k → ∞, 4xy/(x4 + y 4 ) → 0, while x4 + y 4 → +∞. It follows that

lim f (x, y) = lim (x4 + y 4 )(1 − 0) = +∞.

k(x,y)k→∞ k(x,y)k→∞

Therefore, f (x, y) is coercive, and the three critical points are candidates for global minimizers.
Evaluating f (x, y) at (0, 0), (1, 1) and (−1, 1), we obtain

f (0, 0) = 0, f (1, 1) = −2, f (−1, −1) = −2,

and therefore (1, 1) and (−1, −1) are global minimizers. 2

Eigenvalues and Positive Definite Matrices

We now use concepts from linear algebra to obtain simpler, more intuitive criteria for determining
whether a symmetric matrix, such as the Hessian of a function at a point, is positive or negative
definite or semidefinite.
Definition Let A be an n × n symmetric matrix. A nonzero vector x ∈ Rn is an eigenvector of A
if there exists a scalar λ such that
Ax = λx.
The scalar λ is called an eigenvalue of A corresponding to x.
From the equation
Ax − λx = (A − λI)x = 0,
and the fact that x 6= 0, it follows that the matrix A − λI is not invertible (that is, it is singular).
Therefore, any eigenvalue λ of A satisfies

det(A − λI) = 0.

This determinant is a polynomial of degree n in λ, which is called the characteristic polynomial.

Therefore, the eigenvalues can be found by computing the characteristic polynomial, and then
computing its roots.
For a general matrix A, the eigenvalues may be real or complex, because a polynomial with
real coefficients can have complex roots, but the eigenvalues of a symmetric matrix A are real.
Furthermore, if A is symmetric, there exists an orthogonal matrix P , meaning that P T P = I, such
that
A = P DP T ,
where D is a diagonal matrix whose diagonal entries are the eigenvalues of A. The columns of P
are orthonormal vectors, meaning that they are orthogonal and are of magnitude 1. They are also
the eigenvectors of A.

3
 It follows that

QA (x) = x · Ax
= xT Ax
= xT P DP T x
= (P T x)T D(P T x)
= yT Dy
Xn
= λi yi2 ,
j=1

where y = P T x and  
λ1 0 ··· 0
. .
λ2 . . ..
 
 0 
D=
 .. .. ..
.

 . . . 0 
0 · · · 0 λn
This simple form of QA (x) yields the following conditions for classifying A.
Theorem Let A be a symmetric matrix. Then

1. A is positive definite if and only if all of its eigenvalues are positive;

2. A is negative definite if and only if all of its eigenvalues are negative;

3. A is positive semidefinite if and only if all of its eigenvalues are nonnegative;

4. A is negative semidefinite if and only if all of its eigenvalues are nonpositive;

5. A is indefinite if and only if at least one of its eigenvalues is positive and at least one of its
eigenvalues is negative.

We now illustrate the use of these conditions for optimization.

Example Let f (x, y, z) = x2 + y 2 + z 2 − 4xy. Then we have

∇f (x, y, z) = (2x − 4y, 2y − 4x, 2z),

which yields the critical point (0, 0, 0), and

 
2 −4 0
Hf (x, y, z) =  −4 2 0 .
0 0 2

4
This matrix has the characteristic polynomial
 
2 − λ −4 0
det(Hf (x, y, z) − λI) = det  −4 2 − λ 0 
0 0 2−λ
= (2 − λ)[(2 − λ)2 − 16]
= (2 − λ)(λ2 − 4λ − 12)
= (2 − λ)(λ + 2)(λ − 6).

Therefore, the eigenvalues are 2, −2 and 6, which means that the Hessian is indefinite. We conclude
that (0, 0, 0) is a saddle point, and there are no global maximizers or minimizers. 2

Exercises
1. Chapter 1, Exercise 8

2. Chapter 1, Exercise 12

3. Chapter 1, Exercise 13

4. Chapter 1, Exercise 14