Basic Linear Algebra, Metric Spaces, Di¤erential Calculus and

Nonlinear Programming1
Fall 2019

antonio villanacci

June 7, 2019

1
I would like to thank the following friends for helpful comments and discussions: Laura Carosi, Michele Gori,
Marina Pireddu and all the students and teaching assistants of the courses I used these notes for in the past several
years.
2
Contents

I Basic Linear Algebra 7

1 Systems of linear equations 9

1.1 Linear equations and solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2 Systems of linear equations, equivalent systems and elementary operations . . . . . . . . . . . 10
1.3 Systems in triangular and echelon form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4 Reduction algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.5 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.6 Systems of linear equations and matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2 The Euclidean Space Rn 21

2.1 Sum and scalar multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2 Scalar product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3 Norms and Distances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3 Matrices 27
3.1 Matrix operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2 Inverse matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3 Elementary matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4 Elementary column operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4 Vector spaces 45
4.1 De…nition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.3 Vector subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.4 Linear combinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.5 Row and column space of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.6 Linear dependence and independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.7 Basis and dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.8 Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.9 Row and column span . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5 Determinant and rank of a matrix 65

5.1 De…nition and properties of the determinant of a matrix . . . . . . . . . . . . . . . . . . . . . 65
5.2 Rank of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.3 Inverse matrices (continued) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.4 Span of a matrix, linearly independent rows and columns, rank . . . . . . . . . . . . . . . . . 73
5.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3
4 CONTENTS

6 Linear functions 75
6.1 De…nition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
6.2 Kernel and Image of a linear function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6.3 Nonsingular functions and isomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
6.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

7 Linear functions and matrices 83

7.1 From a linear function to the associated matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 83
7.2 From a matrix to the associated linear function . . . . . . . . . . . . . . . . . . . . . . . . . . 85
7.3 M (m; n) and L (V; U ) are isomorphic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7.4 Some related properties of a linear function and associated matrix . . . . . . . . . . . . . . . 88
7.5 Some facts on L (Rn ; Rm ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
U
7.6 Examples of computation of [l]V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

8 Solutions to systems of linear equations 95

8.1 Some preliminary basic facts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
8.2 A solution method: Rouchè-Capelli’s and Cramer’s theorems . . . . . . . . . . . . . . . . . . 96
8.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

II Some topology in metric spaces 107

9 Metric spaces 109
9.1 De…nitions and examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
9.2 Open and closed sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
9.2.1 Sets which are open or closed in metric subspaces. . . . . . . . . . . . . . . . . . . . . 117
9.3 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
9.4 Sequential characterization of closed sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
9.5 Compactness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
9.5.1 Compactness and bounded, closed sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
9.5.2 Sequential compactness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
9.6 Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
9.6.1 Cauchy sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
9.6.2 Complete metric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
9.6.3 Completeness and closedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
9.7 Fixed point theorem: contractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
9.8 Appendices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
9.8.1 Some characterizations of open and closed sets . . . . . . . . . . . . . . . . . . . . . . 136
9.8.2 Norms and metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
9.8.3 Distance between sets, diameters and “topological separation”. . . . . . . . . . . . . . 143
9.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

10 Functions 149
10.1 Limits of functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
10.2 Continuous Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
10.3 Continuous functions on compact sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
10.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

11 Correspondence, maximum theorem and a …xed point theorem 157

11.1 Continuous Correspondences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
11.2 The Maximum Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
11.3 Fixed point theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
11.4 Application of the maximum theorem to the consumer problem . . . . . . . . . . . . . . . . . 173
CONTENTS 5

III Di¤erential calculus in Euclidean spaces 177

12 Partial derivatives and directional derivatives 179
12.1 Partial Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
12.2 Directional Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

13 Di¤erentiability 187
13.1 Total Derivative and Di¤erentiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
13.2 Total Derivatives in terms of Partial Derivatives. . . . . . . . . . . . . . . . . . . . . . . . . . 189

14 Some Theorems 191

14.1 The chain rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
14.2 Mean value theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
14.3 A su¢ cient condition for di¤erentiability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
14.4 A su¢ cient condition for equality of mixed partial derivatives . . . . . . . . . . . . . . . . . . 202
14.5 Taylor’s theorem for real valued functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

15 Implicit function theorem 207

15.1 Some intuition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
15.2 Functions with full rank square Jacobian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
15.3 The inverse function theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
15.4 The implicit function theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
15.5 Some geometrical remarks on the gradient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
15.6 Extremum problems with equality constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . 220
15.7 Exercises on part III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

IV Nonlinear programming 223

16 Convex sets 225
16.1 De…nition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
16.2 Separation of convex sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
16.3 Farkas’Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

17 Concave functions 231

17.1 Di¤erent Kinds of Concave Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
17.1.1 Concave Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
17.1.2 Strictly Concave Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
17.1.3 Quasi-Concave Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
17.1.4 Strictly Quasi-concave Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
17.1.5 Pseudo-concave Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
17.2 Relationships among Di¤erent Kinds of Concavity . . . . . . . . . . . . . . . . . . . . . . . . 243
17.2.1 Hessians and Concavity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

18 Maximization Problems 249

18.1 The case of inequality constraints: Kuhn-Tucker theorems . . . . . . . . . . . . . . . . . . . . 249
18.1.1 On uniqueness of the solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
18.2 The Case of Equality Constraints: Lagrange Theorem. . . . . . . . . . . . . . . . . . . . . . 254
18.3 The Case of Both Equality and Inequality Constraints. . . . . . . . . . . . . . . . . . . . . . . 256
18.4 Main Steps to Solve a (Nice) Maximization Problem . . . . . . . . . . . . . . . . . . . . . . . 257
18.4.1 Some problems and some solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
18.5 The Implicit Function Theorem and Comparative Statics Analysis . . . . . . . . . . . . . . . 264
18.5.1 Maximization problem without constraint . . . . . . . . . . . . . . . . . . . . . . . . . 265
18.5.2 Maximization problem with equality constraints . . . . . . . . . . . . . . . . . . . . . 266
18.5.3 Maximization problem with Inequality Constraints . . . . . . . . . . . . . . . . . . . . 266
18.6 The Envelope Theorem and the meaning of multipliers . . . . . . . . . . . . . . . . . . . . . . 268
18.6.1 The Envelope Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
18.6.2 On the meaning of the multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
6 CONTENTS

19 Applications to Economics 271

19.1 A Walrasian Consumer Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
19.1.1 Zero consumption is allowed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
19.1.2 Zero consumption is not allowed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
19.2 Production . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
19.3 The demand for insurance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
19.4 Exercises on part IV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

V Problem Sets 279

20 Exercises 281
20.1 Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
20.2 Some topology in metric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
20.2.1 Basic topology in metric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
20.2.2 Correspondences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
20.3 Di¤erential Calculus in Euclidean Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
20.4 Nonlinear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

21 Solutions 295
21.1 Linear Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
21.2 Some topology in metric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
21.2.1 Basic topology in metric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
21.2.2 Correspondences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
21.3 Di¤erential Calculus in Euclidean Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
21.4 Nonlinear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
 Part I

Basic Linear Algebra

7
Chapter 1

Systems of linear equations

1.1 Linear equations and solutions

De…nition 1 A1 linear equation in the unknowns x1 ; x2 ; :::; xn is an equation of the form

a1 x1 + a2 x2 + :::an xn = b; (1.1)

where b 2 R and 8j 2 f1; :::; ng ; aj 2 R . The real number aj is called the coe¢ cient of xj and b is called
the constant of the equation. aj for j 2 f1; :::; ng and b are also called parameters of system (1:1).
n
De…nition 2 A solution to the linear equation (1:1) is an ordered n-tuple (x1 ; :::; xn ) := (xj )j=1 such2 that
the following statement (obtained by substituting xj in the place of xj for any j ) is true:

a1 x1 + a2 x2 + :::an xn = b;

The set of all such solutions is called the solution set or the general solution or, simply, the solution of
equation (1:1).

The following fact is well known.

Proposition 3 Let the linear equation

ax = b (1.2)
in the unknown (variable) x 2 R and parameters a; b 2 R be given. Then,
b
1. if a 6= 0, then x = a is the unique solution to (1:2);
2. if a = 0 and b 6= 0, then (1:2) has no solutions;
3. if a = 0 and b = 0, then any real number is a solution to (1:2).

De…nition 4 A linear equation (1:1) is said to be degenerate if 8j 2 f1; :::; ng, aj = 0, i.e., it has the form

0x1 + 0x2 + :::0xn = b; (1.3)

Clearly,

1. if b 6= 0, then equation (1:3) has no solution,

n
2. if b = 0; any n-tuple (xj )j=1 is a solution to (1:3).

De…nition 5 Let a non-degenerate equation of the form (1:1) be given. The leading unknown of the linear
equation (1:1) is the …rst unknown with a nonzero coe¢ cient, i.e., xp is the leading unknown if

8j 2 f1; :::; p 1g ; aj = 0 and ap 6= 0:

For any j 2 f1; :::; ng n fpg ; xj is called a free variable - consistently with the following obvious result.
1 In this part, I often follow Lipschutz (1991).
2 “:=” means “equal by de…nition”.

9
10 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

Proposition 6 Consider a non-degenerate linear equation a1 x1 + a2 x2 + :::an xn = b with leading unknown

xp . Then the set of solutions to that equation is
( P )
n
b j2f1;:::;ngnf;pg aj xj
(xk )k=1 : 8j 2 f1; :::; ng n fpg ; xj 2 R and xp =
ap

1.2 Systems of linear equations, equivalent systems and elemen-

tary operations
De…nition 7 A system of m linear equations in the n unknowns x1 ; x2 ; :::; xn is a system of the form
8
>
> a11 x1 + ::: + a1j xj + ::: + a1n xn = b1
>
>
< :::
ai1 x1 + ::: + aij xj + ::: + ain xn = bi (1.4)
>
>
>
> :::
:
am1 xi + ::: + amj xj + ::: + amn xn = bm

where 8i 2 f1; :::; mg and 8j 2 f1; :::; ng, aij 2 R and 8i 2 f1; :::; mg, bi 2 R. We call Li the i th linear
equation of system (1:4).
n
A solution to the above system is an ordered n-tuple (xj )j=1 which is a solution of each equation of the
system. The set of all such solutions is called the solution set of the system.

De…nition 8 Systems of linear equations are equivalent if their solutions set is the same.

The following fact is obvious.

Proposition 9 Assume that a system of linear equations contains the degenerate equation

L: 0x1 + 0x2 + :::0xn = b:

1. If b = 0, then L may be deleted from the system without changing the solution set;
2. if b 6= 0, then the system has no solutions.

A way to solve a system of linear equations is to transform it in an equivalent system whose solution set
is “easy” to be found. In what follows we make precise the above sentence.

De…nition 10 An elementary operation on a system of linear equations (1:4) is one of the following oper-
ations:

[E1 ] Interchange Li with Lj , an operation denoted by Li $ Lj (which we can read “put Li in the place of
Lj and Lj in the place of Li ”);
[E2 ] Multiply Li by k 2 Rn f0g, denoted by kLi ! Li ; k 6= 0 (which we can read “put kLi in the place of
Li , with k 6= 0”);
[E3 ] Replace Li by ( k times Lj plus Li ), denoted by (Li + kLj ) ! Li (which we can read “put Li + kLj in
the place of Li ”).

Sometimes we apply [E2 ] and [E3 ] in one step, i.e., we perform the following operation

[E] Replace Li by ( k 0 times Lj and k 2 Rn f0g times Li ), denoted by (k 0 Lj + kLi ) ! Li ; k 6= 0.

Elementary operations are important because of the following obvious result.

Proposition 11 If S1 is a system of linear equations obtained from a system S2 of linear equations using
a …nite number of elementary operations, then system S1 and S2 are equivalent.

In what follows, …rst we de…ne two types of “simple”systems (triangular and echelon form systems), and
we see why those systems are in fact “easy”to solve. Then, we show how to transform any system in one of
those “simple” systems.
1.3. SYSTEMS IN TRIANGULAR AND ECHELON FORM 11

1.3 Systems in triangular and echelon form

De…nition 12 A linear system (1:4) is in triangular form if the number n of equations is equal to the number
n of unknowns and 8i 2 f1; :::; ng, xi is the leading unknown of equation i, i.e., the system has the following
form: 8
>
> a11 x1 +a12 x2 +::: +a1;n 1 xn 1 +a1n xn = b1
>
>
< a22 x2 +::: +a2;n 1 xn 1 +a2n xn = b2
::: (1.5)
>
>
>
> an 1;n 1 xn 1 +a n 1n xn = b n 1
:
ann xn = bn
where 8i 2 f1; :::; ng, aii 6= 0.

Proposition 13 System (1:5) has a unique solution.

Proof. We can compute the solution of system (1:5) using the following procedure, known as back-
substitution.
First, since by assumption ann 6= 0, we solve the last equation with respect to the last unknown, i.e., we
get
bn
xn = :
ann
Second, we substitute that value of xn in the next-to-the-last equation and solve it for the next-to-the-last
unknown, i.e.,
bn 1 an 1;n abnn n

xn 1 =
an 1;n 1
and so on. The process ends when we have determined the …rst unknown, x1 .
Observe that the above procedure shows that the solution to a system in triangular form is unique since,
at each step of the algorithm, the value of each xi is uniquely determined, as a consequence of Proposition
3, conclusion 1.

De…nition 14 A linear system (1:4) is said to be in echelon form if

1. no equation is degenerate, and

2. the leading unknown in each equation is to the right of the leading unknown of the preceding equation.

In other words, the system is of the form

8
>
> a11 x1 +::: +a1j2 xj2 +::: +a1s xs =b1
>
>
< a2j2 xj2 +::: +a2;j3 xj3 +::: +a2s xs =b2
a3;j3 xj3 +::: +a3s xs =b3 (1.6)
>
>
>
> :::
:
ar;jr xjr +ar;jr +1 +::: +ars xs =br

with j1 := 1 < j2 < ::: < jr and a11 ; a2j2 ; :::; arjr 6= 0. Observe that the above system has r equations and
s variables and that s r. The leading unknown in equation i 2 f1; :::; rg is xji .

Remark 15 Systems with no degenerate equations are the “interesting” ones. If an equation is degenerate
and the right hand side term is zero, then you can erase it; if the right hand side term is not zero, then the
system has no solutions.

De…nition 16 An unknown xk in system (1:6) is called a free variable if xk is not the leading unknown in
any equation, i.e., 8i 2 f1; :::; rg ; xk 6= xji .

In system (1:6), there are r leading unknowns, r equations and s r 0 free variables.

Proposition 17 Let a system in echelon form with r equations and s variables be given. Then, the following
results hold true.
12 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

1. If s = r, i.e., the number of unknowns is equal to the number of equations, then the system has a
unique solution;

2. if s > r, i.e., the number of unknowns is greater than the number of equations, then we can arbitrarily
assign values to the n r > 0 free variables and obtain solutions of the system.

Proof. We prove the theorem by induction on the number r of equations of the system.
Step 1. r = 1.
In this case, we have a single, non-degenerate linear equation, to which Proposition 6 applies if s > r = 1,
and Proposition 3 applies if s = r = 1.
Step 2.
Assume that r > 1 and the desired conclusion is true for a system with r 1 equations. Consider the
given system in the form (1:6) and erase the …rst equation, so obtaining the following system:
8
>
> a2j2 xj2 +::: +a2;j3 xj3 +::: = b2
<
a3;j3 xj3 +:::
(1.7)
>
> ::: :::
:
ar;jr xjr +ar;jr +1 +ars xs = br

in the unknowns xj2 ; :::; xs . First of all observe that the above system is in echelon form and has r 1
equation; therefore we can apply the induction argument distinguishing the two case s > r and s = r:
If s > r, then we can assign arbitrary values to the free variables, whose number is (the “old” number
minus the erased ones)
s r (j2 j1 1) = s r j2 + 2
and obtain a solution of system (1:7). Consider the …rst equation of the original system

a11 x1 +a12 x2 +::: +a1;j2 1 xj2 1 +a1j2 xj2 +::: = b1 : (1.8)

We immediately see that the above found values together with arbitrary values for the additional

j2 2

free variable of equation (1:8) yield a solution of that equation, as desired. Observe also that the values
given to the variables x1 ; :::; xj2 1 from the …rst equation do satisfy the other equations simply because their
coe¢ cients are zero there.
If s = r, the system in echelon form, in fact, becomes a system in triangular form and then the solution
exists and it is unique.

Remark 18 From the proof of the previous Proposition, if the echelon system (1:6) contains more unknowns
than equations, i.e., s > r, then the system has an in…nite number of solutions since each of the s r 1
free variables may be assigned an arbitrary real number.

1.4 Reduction algorithm

The following algorithm (sometimes called row reduction) reduces system (1:4) of m equation and n unknowns
to either echelon form, or triangular form, or shows that the system has no solution. The algorithm then
gives a proof of the following result.

Proposition 19 Any system of linear equations has either

1. in…nite solutions, or

2. a unique solution, or

3. no solutions.
1.4. REDUCTION ALGORITHM 13

Reduction algorithm.
Consider a system of the form (1:4) such that
8j 2 f1; :::; ng ; 9i 2 f1; ::; mg such that aij 6= 0; (1.9)
i.e., a system in which each variable has a nonzero coe¢ cient in at least one equation. If that is not the
case, the remaining variables can renamed in order to have (1:9) satis…ed.
Step 1. Interchange equations so that the …rst unknown, x1 , appears with a nonzero coe¢ cient in the …rst
equation; i.e., rearrange the equations in the system in order to have a11 6= 0.
Step 2. Use a11 as a “pivot” to eliminate x1 from all equations but the …rst equation. That is, for each
i > 1, apply the elementary operation
ai1
[E3 ] : L1 + Li ! Li
a11
or
[E] : ai1 L1 + a11 Li ! Li :
Step 3. Examine each new equation L :

1. If L has the form

0x1 + 0x2 + :::: + 0xn = 0;
or if L is a multiple of another equation, then delete L from the system.3
2. If L has the form
0x1 + 0x2 + :::: + 0xn = b;
with b 6= 0, then exit the algorithm. The system has no solutions.

Step 4. Repeat Steps 1, 2 and 3 with the subsystem formed by all the equations, excluding the …rst equation.
Step 5. Continue the above process until the system is in echelon form or a degenerate equation is obtained
in Step 3.2.
Summarizing, our method for solving system (1:4) consists of two steps:
Step A. Use the above reduction algorithm to reduce system (1:4) to an equivalent simpler system (in
triangular form, system (1:5) or echelon form (1:6)).
Step B. If the system is in triangular form, use back-substitution to …nd the solution; if the system is
in echelon form, bring the free variables on the right hand side of each equation, give them arbitrary values
(say, the name of the free variable with an upper bar), and then use back-substitution.
Example 20 8
< x1 + 2x2 + ( 3)x3 = 1
3x1 + ( 1) x2 + 2x3 = 7
:
5x1 + 3x2 + ( 4) x3 = 2
Step A.
Step 1. Nothing to do.
Step 2. Apply the operations
3L1 + L2 ! L2
and
5L1 + L3 ! L3 ;
to get 8
< x1 + 2x2 + ( 3)x3 = 1
( 7) x2 + 11x3 = 10
:
( 7) x2 + 11x3 = 7
3 The justi…cation of Step 3 is Propositon 9 and the fact that if L = kL0 for some other equation L0 in the system, then the

operation kL0 + L ! L replace L by 0x1 + 0x2 + :::: + 0xn = 0; which again may be deleted by Propositon 9.
14 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

Step 3. Examine each new equations L2 and L3 :

1. L2 and L3 do not have the form

0x1 + 0x2 + :::: + 0xn = 0;

L2 is not a multiple L3 ;
2. L2 and L3 do not have the form

0x1 + 0x2 + :::: + 0xn = b;

Step 4.
Step 1.1 Nothing to do.
Step 2.1 Apply the operation
L2 + L3 ! L3

to get 8
< x1 + 2x2 + ( 3)x3 = 1
( 7) x2 + 11x3 = 10
:
0x1 + 0x2 + 0x3 = 3

Step 3.1 L3 has the form

0x1 + 0x2 + :::: + 0xn = b;

1. with b = 3 6= 0, then exit the algorithm. The system has no solutions.

1.5 Matrices
De…nition 21 Given m; n 2 N, a matrix (of real numbers) of order m n is a table of real numbers with
m rows and n columns as displayed below.
2 3
a11 a12 ::: a1j ::: a1n
6 a21 a22 ::: a2j ::: a2n 7
6 7
6 ::: 7
6 7
6 ai1 ai2 ::: aij ::: ain 7
6 7
4 ::: 5
am1 am2 ::: amj ::: amn

For any i 2 f1; :::; mg and any j 2 f1; :::; ng the real numbers aij are called entries of the matrix; the
…rst subscript i denotes the row the entries belongs to, the second subscript j denotes the column the entries
belongs to. We will usually denote matrices with capital letters and we will write Am n to denote a matrix
of order m n. Sometimes it is useful to denote a matrix by its “typical” element and we write[aij ] i2f1;:::;mg ,
j2f1;:::;ng

or simply [aij ] if no ambiguity arises about the number of rows and columns. For i 2 f1; :::; mg,

ai1 ai2 ::: aij ::: ain

is called the i th row of A and it denoted by Ri (A). For j 2 f1; :::; ng,
2 3
a1j
6 a2j 7
6 7
6 ::: 7
6 7
6 aij 7
6 7
4 ::: 5
amj

is called the j th column of A and it denoted by C j (A).

We denote the set of m n matrices by Mm;n , and we write, in an equivalent manner, Am n or
A 2 Mm;n .
1.5. MATRICES 15

De…nition 22 The matrix 2 3

a1
Am 1 = 4 ::: 5
am
is called column vector and the matrix

A1 n = a1 ; ::: an

is called row vector. We usually denote row or column vectors by small Latin letters.

De…nition 23 The …rst nonzero entry in a row R of a matrix Am n is called the leading nonzero entry of
R. If R has no leading nonzero entries, i.e., if every entry in R is zero, then R is called a zero row. If all the
rows of A are zero, i.e., each entry of A is zero, then A is called a zero matrix, denoted by 0m n or simply
0, if no confusion arises.

In the previous sections, we de…ned triangular and echelon systems of linear equations. Below, we de…ne
triangular, echelon matrices and a special kind of echelon matrices. In Section (1:6), we will see that there
is a simple relationship between systems and matrices.

De…nition 24 A matrix Am n is square if m = n. A square matrix A belonging to Mm;m is called square

matrix of order m.

De…nition 25 Given A = [aij ] 2 Mm;m , the main diagonal of A is made up by the entries aii with
i 2 f1; ::; mg.

De…nition 26 A square matrix A = [aij ] 2 Mm;m is an upper triangular matrix or simply a triangular
matrix if all entries below the main diagonal are equal to zero, i.e., 8i; j 2 f1; ::; mg ; if i > j; then aij = 0:

De…nition 27 A 2 Mmm is called diagonal matrix of order m if any element outside the principal diagonal
is equal to zero, i.e., 8i; j 2 f1; :::; mg such that i 6= j, aij = 0.

De…nition 28 A matrix A 2 Mm;n is called an echelon (form) matrix, or it is said to be in echelon form,
if the following two conditions hold:

1. All zero rows, if any, are on the bottom of the matrix.

2. The leading nonzero entry of each row is to the right of the leading nonzero entry in the preceding row.

De…nition 29 If a matrix A is in echelon form, then its leading nonzero entries are called pivot entries, or
simply, pivots

Remark 30 If a matrix A 2 Mm;n is in echelon form and r is the number of its pivot entries, then
r min fm; ng. In fact, r m, because the matrix may have zero rows and r n, because the leading
nonzero entries of the …rst row maybe not in the …rst column, and the other leading nonzero entries may be
“strictly to the right” of previous leading nonzero entry.

De…nition 31 A matrix A 2 Mm;n is called in row canonical form if

1. it is in echelon form,
2. each pivot is 1, and
3. each pivot is the only nonzero entry in its column.

Example 32 1. All the matrices below are echelon matrices; only the fourth one is in row canonical form.
2 3 2 3
0 7 0 0 1 2 2 3 2 0 1 2 4 2 3 2 3
6 0 0 0 1 7 6 7 1 2 3 0 1 3 0 0 4
6 3 3 7 6 0 0 1 1 3 3 0 7 4 0 0
4 0 0 0 0 ; ; 1 5; 4 0 0 0 1 0 3 5:
0 7 5 4 0 0 0 0 0 7 1 5
0 0 0 0 0 0 0 1 2
0 0 0 0 0 0 0 0 0 0 0 0 0

2. Any zero matrix is in row canonical form.

16 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

Remark 33 Let a matrix Am n in row canonical form be given. As a consequence of the de…nition, we
have what follows.
1. If some rows from A are erased, the resulting matrix is still in row canonical form.
2. If some columns of zeros are added, the resulting matrix is still in row canonical form.

De…nition 34 Denote by Ri the i th row of a matrix A. An elementary row operation is one of the
following operations on the rows of A:

[E1 ] (Row interchange) Interchange Ri with Rj , an operation denoted by Ri $ Rj (which we can read “put
Ri in the place of Rj and Rj in the place of Ri ”);;

[E2 ] (Row scaling) Multiply Ri by k 2 Rn f0g, denoted by kRi ! Ri ; k 6= 0 (which we can read “put kRi in
the place of Ri , with k 6= 0”);

[E3 ] (Row addition) Replace Ri by ( k times Rj plus Ri ), denoted by Ri + kRj ! Ri (which we can read
“put Ri + kRj in the place of Ri ”).

Sometimes we apply [E2 ] and [E3 ] in one step, i.e., we perform the following operation

[E] Replace Ri by ( k 0 times Rj and k 2 Rn f0g times Ri ), denoted by k 0 Rj + kRi ! Ri ; k 6= 0.

De…nition 35 A matrix A 2 Mm;n is said to be row equivalent to a matrix B 2 Mm;n if B can be obtained
from A by a …nite number of elementary row operations.

It is hard not to recognize the similarity of the above operations and those used in solving systems of
linear equations.
We use the expression “row reduce” as having the meaning of “transform a given matrix into another
matrix using row operations”. The following algorithm “row reduces” a matrix A into a matrix in echelon
form.
Row reduction algorithm to echelon form.
Consider a matrix A = [aij ] 2 Mm;n :

Step 1. Find the …rst column with a nonzero entry. Suppose it is column j1 .

Step 2. Interchange the rows so that a nonzero entry appears in the …rst row of column j1 , i.e., so that
a1j1 6= 0.

Step 3. Use a1j1 as a “pivot” to obtain zeros below a1j1 , i.e., for each i > 1, apply the row operation

aij1
[E3 ] : R1 + Ri ! Ri
a1j1

or
[E] : aij1 R1 + a11 Ri ! Ri :

Step 4. Repeat Steps 1, 2 and 3 with the submatrix formed by all the rows, excluding the …rst row.

Step 5. Continue the above process until the matrix is in echelon form.

Example 36 Let’s apply the above algorithm to the following matrix

2 3
1 2 3 1
4 3 1 2 7 5
5 3 4 2

Step 1. Find the …rst column with a nonzero entry: that is C 1 , and therefore j1 = 1.

Step 2. Interchange the rows so that a nonzero entry appears in the …rst row of column j1 , i.e., so that
a1j1 6= 0: a1j1 = a11 = 1 6= 0.
1.5. MATRICES 17

Step 3. Use a11 as a “pivot” to obtain zeros below a11 . Apply the row operations
3R1 + R2 ! R2

and
5R1 + R3 ! R3 ;
to get 2 3
1 2 3 1
4 0 7 11 10 5
0 7 11 7
Step 4. Apply the operation
R2 + R3 ! R3
to get 2 3
1 2 3 1
4 0 7 11 10 5
0 0 0 3
which is is in echelon form.
Row reduction algorithm from echelon form to row canonical form.
Consider a matrix A = [aij ] 2 Mm;n in echelon form, say with pivots
a1j1 ; a2j2 ; :::; arjr :
r 1
Step 1. Multiply the last nonzero row R by arjr ;so that the leading nonzero entry of that row becomes 1.

Step 2. Use arjr as a “pivot” to obtain zeros above the pivot, i.e., for each i 2 fr 1; r 2; :::; 1g, apply
the row operation
[E3 ] : ai;jr Rr + Ri ! Ri :
Step 3. Repeat Steps 1 and 2 for rows Rr 1
; Rr 2
; :::; R2 .
1
Step 4. Multiply R1 by a1j1 .

Example 37 Consider the matrix 2 3

1 2 3 1
4 0 7 11 10 5
0 0 0 3
in echelon form, with leading nonzero entries
a11 = 1; a22 = 7; a34 = 3:
Step 1. Multiply the last nonzero row R3 by 13 ;so that the leading nonzero entry becomes 1:
2 3
1 2 3 1
4 0 7 11 10 5
0 0 0 1

Step 2. Use arjr = a34 as a “pivot” to obtain zeros above the pivot, i.e., for each i 2 fr 1; r 2; :::; 1g =
f2; 1g, apply the row operation
[E3 ] : ai;jr Rr + Ri ! Ri ;
which in our case are
a2;4 R3 + R2 ! R2 i.e., 10R3 + R2 ! R2 ;
a1;4 R3 + R1 ! R1 i.e., R3 + R1 ! R1 :
Then, we get 2 3
1 2 3 0
4 0 7 11 0 5
0 0 0 1
18 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS

1
Step 3. Multiply R2 by 7, and get
2 3
1 2 3 0
4 0 1 11
0 5
7
0 0 0 1

Use a23 as a “pivot” to obtain zeros above the pivot, applying the operation:

2R2 + R1 ! R1 ,

to get 2 3
1
1 0 7 0
4 0 1 11
0 5
7
0 0 0 1

which is in row reduced form.

Proposition 38 Any matrix A 2 Mm;n is row equivalent to a matrix in row canonical form.

Proof. The two above algorithms show that any matrix is row equivalent to at least one matrix in row
canonical form.

Remark 39 In fact, in Proposition 157, we will show that: Any matrix A 2 Mm;n is row equivalent to a
unique matrix in row canonical form.

1.6 Systems of linear equations and matrices

De…nition 40 Given system (1:4), i.e., a system of m linear equation in the n unknowns x1 ; x2 ; :::; xn
8
>
> a11 x1 + ::: + a1j xj + ::: + a1n xn = b1
>
>
< :::
ai1 x1 + ::: + aij xj + ::: + ain xn = bi
>
>
> :::
>
:
am1 xi + ::: + amj xj + ::: + amn xn = bm ;

the matrix 2 3
a11 ::: a1j ::: a1n b1
6 ::: 7
6 7
6 ai1 ::: aij ::: ain bi 7
6 7
4 ::: 5
am1 ::: amj ::: amn bm
is called the augmented matrix M of system (1:4).

Each row of M corresponds to an equation of the system, and each column of M corresponds to the
coe¢ cients of an unknown, except the last column which corresponds to the constant of the system.
In an obvious way, given an arbitrary matrix M , we can …nd a unique system whose associated matrix
is M ; moreover, given a system of linear equations, there is only one matrix M associated with it. We can
therefore identify system of linear equations with (augmented) matrices.
The coe¢ cient matrix of the system is
2 3
a11 ::: a1j ::: a1n
6 ::: 7
6 7
A=6 6 ai1 ::: aij ::: ain
7
7
4 ::: 5
am1 ::: amj ::: amn

One way to solve a system of linear equations is as follows:

1. Reduce its augmented matrix M to echelon form, which tells if the system has solution; if M has a
row of the form (0; 0; :::; 0; b) with b 6= 0, then the system has no solution and you can stop. If the system
admits solutions go to the step below.
1.7. EXERCISES 19

2. Reduce the matrix in echelon form obtained in the above step to its row canonical form. Write
the corresponding system. In each equation, bring the free variables on the right hand side, obtaining a
triangular system. Solve by back-substitution.
The simple justi…cation of this process comes from the following facts:

1. Any elementary row operation of the augmented matrix M of the system is equivalent to applying the
corresponding operation on the system itself.
2. The system has a solution if and only if the echelon form of the augmented matrix M does not have
a row of the form (0; 0; :::; 0; b) with b 6= 0 - simply because that row corresponds to a degenerate
equation.
3. In the row canonical form of the augmented matrix M (excluding zero rows) the coe¢ cient of each
non-free variable is a leading nonzero entry which is equal to one and is the only nonzero entry in its
respective column; hence the free variable form of the solution is obtained by simply transferring the
free variable terms to the other side of each equation.

Example 41 Consider the system presented in Example 20:

8
< x1 + 2x2 + ( 3)x3 = 1
3x1 + ( 1) x2 + 2x3 = 7
:
5x1 + 3x2 + ( 4) x3 = 2

The associated augmented matrix is:

2 3
1 2 3 1
4 3 1 2 7 5
5 3 4 2

In example 36, we have see that the echelon form of the above matrix is
2 3
1 2 3 1
4 0 7 11 10 5
0 0 0 3

which has its last row of the form (0; 0; :::; 0; b) with b = 3 6= 0, and therefore the system has no solution.

1.7 Exercises
Chapter 1 in Lipschutz:
1,2,3,4,6,7,8,10, 11,12,14,15,16.
20 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS
Chapter 2

The Euclidean Space Rn

2.1 Sum and scalar multiplication

It is well known that the real line is a representation of the set R of real numbers. Similarly, a ordered pair
(x; y) of real numbers can be used to represent a point in the plane and a triple (x; y; z) or (x1 ; x2 ; x3 ) a
n
point in the space. In general, if n 2 N := f1; 2; :::; g, we can de…ne (x1 ; x2 ; :::; xn ) or (xi )i=1 as a point in
the n space:

De…nition 42 Rn := R ::: R.

In other words, Rn is the Cartesian product of R multiplied n times by itself.

De…nition 43 The elements of Rn are ordered n-tuple of real numbers, are usually called vectors and are
denoted by
x = (x1 ; x2 ; :::; xn ) or x = (xi )ni=1 :
xi is called i th component of x 2 Rn .

De…nition 44 x = (xi )ni=1 2 Rn and y = (yi )ni=1 are equal if

8i 2 f1; :::; ng ; xi = yi :

In that case we write x = y.

Let us introduce two operations on Rn and analyze some properties they satisfy.

De…nition 45 Given x 2 Rn ; y 2 Rn , we call addition or sum of x and y the element denoted by x + y 2 Rn

obtained as follows
x + y := (xi + yi )ni=1 :

De…nition 46 An element 2 R is called scalar.

De…nition 47 Given x 2 Rn and 2 R, we call scalar multiplication of x by the element x 2 Rn

obtained as follows
x := ( xi )ni=1 :

Geometrical interpretation of the two operations in the case n = 2:

21
22 CHAPTER 2. THE EUCLIDEAN SPACE RN

From the well known properties of the sum and product of real numbers it is possible to verify that the
following properties of the above operations do hold true.
Properties of addition.
A1. (Associative) 8x; y 2 Rn ; (x + y) + z = x + (y + z);
A2. (existence of null element) there exists an element e in Rn such that for any x 2 Rn , x + e = x; in
fact such element is unique and it is denoted by 0;
A3. (existence of inverse element) 8x 2 Rn 9y 2 Rn such that x + y = 0; in fact, that element is unique
and denoted by x;
A4. (Commutative) 8x; y 2 Rn , x + y = y + x.
Properties of multiplication.
M1. (distributive) 8 2 R; x 2 Rn ; y 2 Rn (x + y) = x + y;
M2. (distributive) 8 2 R; 2 R; x 2 Rn , ( + )x = x + x
M3. 8 2 R; 2 R; x 2 Rn , ( )x = ( x);
M4. 8x 2 Rn , 1x = x.

2.2 Scalar product

De…nition 48 Given x = (xi )ni=1 ; y = (yi )ni=1 2 Rn , we call dot, scalar or inner product of x and y, denoted
by xy or x y, the scalar
Xn
xi yi 2 R:
i=1

Remark 49 The scalar product of elements of Rn satis…es the following properties.

1. 8x; y 2 Rn x y = y x;
2. 8 ; 2 R; 8x; y; z 2 Rn ( x + y) z = (x z) + (y z);
3. 8x 2 Rn ; x x 0;
4. 8x 2 Rn ; x x = 0 () x = 0.

De…nition 50 The set Rn with above described three operations (addition, scalar multiplication and dot
product) is usually called Euclidean space of dimension n.

De…nition 51 Given x = (xi )ni=1 2 Rn , we denote the (Euclidean) norm or length of x by

v
u n
1 uX
kxk := (x x) = t
2
2
(xi ) :
i=1

Geometrical Interpretation of scalar products in R2 .

Given x = (x1 ; x2 ) 2 R2 n f0g, from elementary trigonometry we know that

x = (kxk cos ; kxk sin ) (2.1)

where is the measure of the angle between the positive part of the horizontal axes and x itself.
2.3. NORMS AND DISTANCES 23

Using the above observation we can verify that given x = (x1 ; x2 ) and y = (y1 ; y2 ) in R2 n f0g,

xy = kxk kyk cos ( )

where is an1 angle between x and y.
scan and insert picture (Marcellini-Sbordone page 179)
From the picture and (2:1), we have

x = (kxk cos ( 1 ) ; kxk sin ( 1 ))

and

y = (kyk cos ( 2 ) ; kyk sin ( 2 )) :

Then2
xy = kxk kyk (cos ( 1) cos ( 2) + sin ( 1) sin ( 2 )) = kxk kyk cos ( 2 1) :

Taken x and y not belonging to the same line, de…ne := (angle between x and y with minimum
measure): From the above equality, it follows that

= 2 , x y=0
< 2 , x y>0
> 2 , x y < 0:

De…nition 52 x; y 2 Rn n f0g are orthogonal if xy = 0.

2.3 Norms and Distances

Proposition 53 (Properties of the norm). Let 2 R and x; y 2 Rn :
1. kxk 0, and kxk = 0 , x = 0,
2. k xk = j j kxk,
3. kx + yk kxk + kyk (Triangle inequality),
4. jxyj kxk kyk (Cauchy-Schwarz inequality ).
qP Pn
n 2 2 2
Proof. 1. By de…nition kxk = i=1 (xi ) 0: Moreover, kxk = 0 , kxk = 0 , i=1 (xi ) = 0 ,
x = 0. qP qP
n 2 (x )2 = j j n 2
2. k xk = i=1 i i=1 (xi ) = j j kxk.
4. (3 is proved using 4)
2 2 2
We want to show that jxyj kxk kyk or jxyj kxk kyk , i.e.,

n
!2 n
! n
!
X X X
xi yi x2i yi2
i=1 i=1 i=1
Pn Pn Pn
De…ned X := i=1 x2i ; Y := i=1 yi2 and Z := i=1 xi yi , we have to prove that

Z2 XY: (2.2)
1 Recall that 8x 2 R, cos x = cos ( x) = cos (2 x).

2 Recall that for any x1 ; x2 2 R

cos (x1 x2 ) = cos (x1 ) cos (x2 ) sin (x1 ) sin (x2 ) ;

and
cos (x1 ) = cos ( x1 )
24 CHAPTER 2. THE EUCLIDEAN SPACE RN

Observe that
Pn 2
8a 2 R; 1: (axi + yi ) 0; and
Pi=1
n 2
2: i=1 (axi + yi ) = 0 , 8i 2 f1; :::; ng ; axi + yi = 0

Moreover,
n
X n
X n
X n
X
2 2
(axi + yi ) = a x2i + 2a xi yi + yi2 = a2 X + 2aZ + Y 0 (2.3)
i=1 i=1 i=1 i=1
Z
If X > 0, we can take a = X, and from (2:3), we get

Z2 Z2
0 X 2 +Y
X2 X
or
Z2 XY;
as desired.
If X = 0, then x = 0 and Z = 0, and (2:2) is true simply because 0 0.
2 2
3. It su¢ ces to show that kx + yk (kxk + kyk) .
n
X n
X
2 2 2 2
kx + yk = (xi + yi ) = (xi ) + 2xi yi + (yi ) =
i=1 i=1
(4 ab ove)
2 2 2 2 2 2 2
= kxk + 2xy + kyk kxk + 2 jxyj + kyk kxk + 2 kxk kyk + kyk = (kxk + kyk) :

Proposition 54 For any x; y 2 Rn and any ; 2 R, we have

1. jkxk kykj kx yk, and
2. k x + yk2 = 2 kxk2 + 2 x y + 2 kyk2 .

Proof. 1. Recall that 8a; b 2 R

b a b , jaj b:
From Proposition 53.3,identifying x with x y and y with y, we get kx y + yk kx yk + kyk, i.e.,

kxk kyk kx yk

From Proposition 53.3, identifying x with y x and y with x, we get ky x + xk ky xk + kxk, i.e.,

kyk kxk ky xk = kx yk

and
kx yk kxk kyk ;
as desired.
2.
Pn 2 Pn 2 2 2
k x + yk2 = i=1 ( i xi + yi ) = i=1 (xi ) + 2 (xi ) (yi ) + 2
(yi ) =

2 Pn 2 Pn 2
Pn 2 2
= i=1 (xi ) + 2 i=1 (xi ) (yi ) + i=1 (yi ) = kxk2 + 2 x y+ 2
kyk2 :

n
De…nition 55 For any n 2 Nn f0g and for any i 2 f1; :::; ng ; ein := eij;n j=1
2 Rn with
8
< 0 if i 6= j
ein;j =
:
1 if i=j

In other words, ein is an element of Rn whose components are all zero, but the i th component which is
equal to 1. The vector ein is called the i th canonical vector in Rn .
2.4. EXERCISES 25

Remark 56 8x 2 Rn ,
n
X
kxk jxi j ;
i=1

as veri…ed below.
n
X (1) n
X n
X n
X
(2)
kxk = xi ei xi ei = jxi j ei = jxi j ;
i=1 i=1 i=1 i=1

where (1) follows from the triangle inequality, i.e., Proposition 53.3, and (2) from Proposition 53.2.

De…nition 57 Given x; y 2 Rn ;we denote the (Euclidean) distance between x and y by

d (x; y) := kx yk

Proposition 58 (Properties of the distance). Let x; y; z 2 Rn :

1. d (x; y) 0, and d (x; y) = 0 , x = y,
2. d (x; y) = d (y; x),
3. d (x; z) d (x; y) + d (y; z) (Triangle inequality).

Proof. 1. It follows from property 1 of the norm.

2. It follows from the de…nition of the distance as a norm.
3. Identifying x with x y and y with y z in property 3 of the norm, we get
k(x y) + (y z)k kx yk + ky zk, i.e., the desired result.

2.4 Exercises
From Lipschutz (1991), starting from page 53: 2.1 ! 2.4, 2.12 ! 2.19, 2.26, 2.27.
26 CHAPTER 2. THE EUCLIDEAN SPACE RN
Chapter 3

Matrices

We presented the concept of matrix in De…nition 21. In this chapter, we study further properties of matrices.
De…nition 59 Given two matrices A = [aij ]i2f1;:::;mg ,A = [aij ]i2f1;:::;mg 2 Mm;n , we say that A = B if
j2f1;:::;ng j2f1;:::;ng

8i 2 f1; :::; mg ; j 2 f1; :::; ng ; aij = bij :

De…nition 60 The transpose of a matrix A 2 Mm;n , denoted by AT belongs to Mn;m and it is the matrix
obtained by writing the rows of A, in order, as columns:
2 3T 2 3
a11 ::: a1j ::: a1n a11 ::: ai1 ::: am1
6 ::: 7 6 ::: 7
6 7 6 7
AT = 6 6 ai1 ::: aij ::: ain 7 6
7 = 6 a1j ::: aij ::: amj 7:
7
4 ::: 5 4 ::: 5
am1 ::: amj ::: amn a1n ::: ain ::: amn

In other words, row 1 of the matrix A becomes column 1 of AT ; row 2 of A becomes column 2 of AT ,
and so on, up to row m which becomes column m of AT . Same results is obtained proceeding as follows:
column 1of A becomes row 1 of AT ; column 2 of A becomes row 2 of AT , and so on, up to column n which
becomes row n of AT . More formally, given A = [aij ]i2f1;:::;mg 2Mm;n , then
j2f1;:::;ng

AT = [aji ] j2f1;:::;ng 2 Mn;m :

i2f1;:::;mg

De…nition 61 A matrix A 2Mn;n is said to be symmetric if A = AT , i.e., 8i; j 2 f1; :::; ng, aij = aji .
Remark 62 We can write a matrix Am n = [aij ] as

2 3
R1 (A)
6 ::: 7
6 7
A=6
6 Ri (A) 7 = C 1 (A) ; :::; C j (A) ; :::; C n (A)
7
4 ::: 5
Rm (A)
where
R (A) = [ai1 ; :::; aij ; :::ain ] := Ri1 (A) ; :::; Rij (A) ; :::Rin (A) 2 Rn
i
for i 2 f1; :::; mg and
2 3 2 3
a1j C j1 (A)
6 7 6 7
6 7 6 7
C j (A) = 6
6 aij
7 := 6 C ji (A)
7 6
7 2 Rn
7 for j 2 f1; :::; ng :
4 5 4 5
amj C jm (A)

In other words, Ri (A) denotes row i of the matrix A and C j (A) denotes column j of matrix
A.

27
28 CHAPTER 3. MATRICES

3.1 Matrix operations

De…nition 63 Given the matrices Am n := [aij ] and Bm n := [bij ], the sum of A and B, denoted by A + B
is the matrix Cm n = [cij ] such that

8i 2 f1; :::; mg ; j 2 f1; :::; ng ; cij = aij + bij

De…nition 64 Given the matrices Am n := [aij ] and the scalar , the product of the matrix A by the scalar
, denoted by A or A, is the matrix obtained by multiplying each entry A by :

A := [ aij ]

Remark 65 It is easy to verify that the set of matrices Mm;n with the above de…ned sum and scalar mul-
tiplication satis…es all the properties listed for elements of Rn in Section 2.1.

De…nition 66 Given A = [aij ] 2 Mm;n , B = [bjk ] 2 Mn;p , the product A B is a matrix C = [cik ] 2 Mm;p
such that
n
X
8i 2 f1; :::mg ; 8k 2 f1; :::; pg ; cik := aij bjk = Ri (A) C k (B)
j=1

i.e., since 2 3
R1 (A)
6 ::: 7
6 7
A=6
6 Ri (A) 7 ; B = C 1 (B) ; :::; C k (B) ; :::; C p (B)
7 (3.1)
4 ::: 5
Rm (A)
2 3
R1 (A) C 1 (B) ::: R1 (A) C k (B) ::: R1 (A) C p (B)
6 ::: 7
6 7
AB = 6
6 Ri (A) C 1 (B) ::: Ri (A) C k (B) ::: Ri (A) C p (B) 7
7 (3.2)
4 ::: 5
Rm (A) C 1 (B) ::: Rm (A) C k (B) ::: Rm (A) C p (B)

Remark 67 If A 2 M1n , B 2 Mn;1 , the above de…nition coincides with the de…nition of scalar product
between elements of Rn . In what follows, we often identify an element of Rn with a row or a column vectors
( - see De…nition 22) consistently with we what write. In other words Am n x = y means that x and y are
column vector with n entries, and wAm n = z means that w and z are row vectors with m entries.

De…nition 68 If two matrices are such that a given operation between them is well de…ned, we say that they
are conformable with respect to that operation.

Remark 69 If A; B 2Mm;n , they are conformable with respect to matrix addition. If A 2Mm;n and
B 2Mn;p , they are conformable with respect to multiplying A on the left of B. We often say the two matrices
are conformable and let the context de…ne precisely the sense in which conformability is to be understood.

Remark 70 (For future use) 8k 2 f1; :::; pg,

2 1 3 2 3
R (A) R1 (A) C k (B)
6 ::: 7 6 ::: 7
6 i 7 6 7
A C k (B) = 66 R (A)
7 C k (B) = 6
7 6 Ri (A) C k (B) 7
7 (3.3)
4 ::: 5 4 ::: 5
Rm (A) Rm (A) C k (B)

Then, just comparing (3:2) and (3:3), we get

AB = A C 1 (B) ::: A C k (B) ::: A C p (B) ; (3.4)

i.e.,
C k (AB) = A C k (B) :
3.1. MATRIX OPERATIONS 29

Similarly, 8i 2 f1; :::; mg,

Ri (A) B = Ri (A) C 1 (B) ::: C k (B) ::: C p (B) =

(3.5)
= R (A) C (B) ::: Ri (A) C k (B) ::: Ri (A) C p (B) :
i 1

Then, just comparing (3:2) and (3:5), we get

2 3
R1 (A) B
6 ::: 7
6 7
AB = 6
6 Ri (A) B 7;
7 (3.6)
4 ::: 5
Rm (A) B

i.e.,
Ri (AB) = Ri (A) B:

De…nition 71 A submatrix of a matrix A 2Mm;n is a matrix obtained from A erasing some rows and
columns.

De…nition 72 A matrix A 2Mm;n is partitioned in blocks if it is written as submatrices using a system of

horizontal and vertical lines.

Example 73 The matrix 2 3

1 2 3 4 5
6 6 7 8 9 0 7
6 7
4 1 2 3 4 5 5
6 7 8 9 0
can be partitioned in block submatrices in several ways. For example as follows
2 3
1 2 j 3 4 5
6 6 7 j 8 9 0 7
6 7
6 j 7;
6 7
4 1 2 j 3 4 5 5
6 7 j 8 9 0

whose blocks are

1 2 3 4 5 1 2 3 4 5
; ; ; :
6 7 8 9 0 6 7 8 9 0

The reason of the partition into blocks is that the result of operations on block matrices can obtained
by carrying out the computation with blocks, just as if they were actual scalar entries of the matrices, as
described below.

Remark 74 We verify below that for matrix multiplication, we do not commit an error if, upon conformably
partitioning two matrices, we proceed to regard the partitioned blocks as real numbers and apply the usual
rules.
n1 n2 n1 n2
1. Take a := (ai )i=1 2 Rn1 ; b := (bj )j=1 2 Rn2 ; c := (ci )i=1 2 Rn1 ; d := (dj )j=1 2 Rn2 ;
2 3
c n1
X n2
X
a j b 4 5 = ai ci + bj dj = a c + b d: (3.7)
1 (n1 +n2 )
d (n1 +n2 ) 1
i=1 j=1

2.
Take A 2Mm;n1 ; B 2Mm;n2 ; C 2Mn1 ;p ; D 2Mn2 ;p , with
2 1 3 2 1 3
R (A) R (B)
A = 4 ::: 5 ; B = 4 ::: 5;
Rm (A) Rm (B)
30 CHAPTER 3. MATRICES

C = C 1 (C) ; :::; C p (C) ;

D = C 1 (D) ; :::; C p (D)

Then,
2 3
R1 (A) R1 (B)
C C 1 (C) ; :::; C p (C)
A B = 4 ::: ::: 5 =
m (n1 +n2 ) D C 1 (D) ; :::; C p (D)
(n1 +n2 ) p Rm (A) Rm (B)
2 3
R1 (A) C 1 (C) + R1 (B) C 1 (D) ::: R1 (A) +R1 (B) C p (D)
= 4 ::: 5=
m 1 m 1 m p m p
R (A) C (C) + R (B) C (D) R (A) C (C) + R (B) C (D)
2 1 3 2 3
R (A) C 1 (C) ::: R1 (A) C p (C) R1 (B) C 1 (D) ::: R1 (B) C p (D)
= 4 ::: 5 + 4 ::: 5=
m 1 m p m 1 m p
R (A) C (C) R (A) C (C) R (B) C (D) R (B) C (D)
= AC + BD:

De…nition 75 Let the matrices Ai 2 M (ni ; ni ) for i 2 f1; :::; Kg, then the matrix
2 3
A1
6 A2 7
6 7 !
6 . .. 7 XK XK
6 7
A=6 6
72M
7 ni ; ni
6 Ai 7 i=1 i=1
6 .. 7
4 . 5
AK

is called block diagonal matrix.

Very often having information on the matrices Ai gives information on A.

Remark 76 It is easy, but cumbersome, to verify the following properties.

1. (associative property) 8A 2 Mm;n , 8B 2 Mnp , 8C 2 Mpq , A(BC) = (AB)C;

2. (distributive property) 8A 2 Mm;n , 8B 2 Mm;n , 8C 2 Mnp , (A + B)C = AC + BC.
3. (linearity) 8x; y 2 Rn and 8 ; 2 R;

A ( x + y) = A ( x) + B ( y) = Ax + Ay

It is false that:
1. (commutative property) 8A 2 Mm;n , 8B 2 Mnp , AB = BA;
2. (cancellation law) 8A 2 Mm;n , 8B; C 2 Mnp ; hA 6= 0; AB = ACi =) hB = Ci;
3. 8A 2 Mm;n , 8B 2 Mnp , hA 6= 0; AB = 0i =) hB = 0i.
Let’s show why the above statements are false.
1. 2 3
1 0
1 2 1
A= B=4 2 1 5
1 1 3
0 1
2 3
1 0
1 2 1 4 5 3
AB = 2 1 5=
1 1 3 1 4
0 1
2 3 2 3
1 0 1 2 1
1 2 1
BA = 4 2 1 5 =4 1 5 5 5
1 1 3
0 1 1 1 3
1 2 1 0
C= D=
1 1 3 2
3.1. MATRIX OPERATIONS 31

1 2 1 0 7 4
CD = =
1 1 3 2 2 2

1 0 1 2 1 2
DC = =
3 2 1 1 1 8
Observe that since the commutative property does not hold true, we have to distinguish between “left
factor out” and “right factor out” and also between “left multiplication or pre-multiplication” and “right
multiplication or post-multiplication”:

AB + AC = A (B + C)
EF + GF = (E + G) F
AB + CA 6= A (B + C)
AB + CA 6= (B + C) A

2.
Given
3 1 4 1 1 2
A= B= C= ;
6 2 5 6 4 3
we have
3 1 4 1 7 9
AB = =
6 2 5 6 14 18
3 1 1 2 7 9
AC = =
6 2 4 3 14 18
3.
Observe that 3: ) 2: and therefore :2: ) :3: Otherwise, you can simply observe that 3: follows from 2:,
choosing A in 3: equal to A in 2:, and B in 3: equal to B C in 2::

3 1 4 1 1 2 3 1 3 1 0 0
A (B C) = = =
6 2 5 6 4 3 6 2 9 3 0 0
Since the associative property of the product between matrices does hold true we can give the following
de…nition.

De…nition 77 Given A 2 Mm;m ,

Ak := A A ::: A :
1 2 k times

Observe that if A 2 Mm;m and k; l 2 Nn f0g, then

Ak Al = Ak+l :

Remark 78 Properties of transpose matrices. For any m; n; p 2 N,

1. 8A 2 Mm;n (AT )T = A
2. 8A; B 2 Mm;n (A + B)T = AT + B T
3. 8 2 R, 8A 2 Mm;n ( A)T = AT
4. 8A 2 Mm;n , 8B 2 Mn;p (AB)T = B T AT
Matrices and linear systems.
In Section 1.6, we have seen that a system of m linear equation in the n unknowns x1 ; x2 ; :::; xn and
parameters aij , for i 2 f1; :::; mg, j 2 f1; :::; ng, (bi )ni=1 2 Rn is displayed below:
8
>
> a11 x1 + ::: + a1j xj + ::: + a1n xn = b1
>
>
< :::
ai1 x1 + ::: + aij xj + ::: + ain xn = bi (3.8)
>
>
>
> :::
:
am1 xi + ::: + amj xj + ::: + amn xn = bm
32 CHAPTER 3. MATRICES

Moreover, the matrix 2 3

a11 ::: a1j ::: a1n b1
6 ::: 7
6 7
6 ai1 ::: aij ::: ain bi 7
6 7
4 ::: 5
am1 ::: amj ::: amn bm
is called the augmented matrix M of system (1:4). The coe¢ cient matrix A of the system is
2 3
a11 ::: a1j ::: a1n
6 ::: 7
6 7
A=6 6 ai1 ::: aij ::: ain 7 7
4 ::: 5
am1 ::: amj ::: amn
Using the notations we described in the present section, we can rewrite linear equations and systems of
linear equations in a convenient and short manner, as described below.
The linear equation in the unknowns x1 ; :::; xn and parameters a1 ; :::; ai ; :::; an ; b 2 R
a1 x1 + ::: + ai xi + ::: + an xn = b
can be rewritten as
n
X
ai xi = b
i=1
or

2 3 a x=b
x1
where a = [a1 ; :::; an ] and x = 4 ::: 5.
xn
The linear system (3:8) can be rewritten as
8 n
> P
>
> a1j xj = b1
>
< j=1
:::
>
>
> P
>
n
amj xj = bm
:
j=1
or 8 1
< R (A) x = b1
:::
: m
R (A) x = bm
or
Ax = b
where A = [aij ].
De…nition 79 The trace of A 2 Mmm , written tr A, is the sum of the diagonal entries, i.e.,
m
X
tr A = aii :
i=1

De…nition 80 The identity matrix Im is a diagonal matrix of order m with each element on the principal
diagonal equal to 1. If no confusion arises, we simply write I in the place of Im .
n
Remark 81 1. 8n 2 Nn f0g ; (Im ) = Im ;
2. 8A 2 Mm;n ; Im A = AIn = A:
Proposition 82 Let A; B 2 M (m; m) and k 2 R. Then
1. tr (A + B) = tr A + tr B;
2. tr kA = k tr A;
3. tr AB = tr BA.
Proof. Exercise.
3.2. INVERSE MATRICES 33

3.2 Inverse matrices

De…nition 83 Given a matrix An n, , a matrix Bn n is called an inverse of A if
AB = BA = In :
We then say that A is invertible, or that A admits an inverse.
Proposition 84 If A admits an inverse, then the inverse is unique.
Proof. Let the inverse matrices B and C of A be given. Then
AB = BA = In (3.9)
and

AC = CA = In (3.10)
Left multiplying the …rst two terms in the equality (3:9) by C, we get
(CA) B = C (BA)
and from (3:10) and (3:9) we get B = C, as desired.
Thanks to the above Proposition, we can present the following de…nition.
1
De…nition 85 If the inverse of A does exist, then it is denoted by A .
Example 86 1. Assume that for i 2 f1; :::; ng ; i 6= 0:The diagonal matrix
2 3
1
6 .. 7
4 . 5
n

is invertible and its inverse is 2 3

1
1
6 .. 7
4 . 5:
1
n

2. It is easy to check that the following matrix is not invertible.

1 1
:
2 2
Remark 87 If a row or a column of A is zero, then A is not invertible, as veri…ed below.
Without loss of generality, assume the …rst row of A is equal to zero. Assume that B is the inverse of A.
But then, since I = AB, we wold have 1 = R1 (A) C 1 (B) = 0, a contradiction.
Proposition 88 If A 2 Mm;m and B 2 Mm;m are invertible matrices, then AB is invertible and
1 1 1
(AB) =B A :
Proof.
1 1 1 1 1 1
(AB) B A = A BB A = AIA = AA = I:
1 1 1 1 1 1
B A (AB) = B A A B=B IB = B B = I:

Remark 89 The existence of the inverse matrix gives an obvious way of solving systems of linear equations
with the same number of equations and unknowns.
Given the system
An n x = b;
1
if A exists, then
1
x=A b:
34 CHAPTER 3. MATRICES

Proposition 90 (Some other properties of the inverse matrix)

Let the invertible matrix A be given.
1
1. A 1 in invertible and A 1 = A;
1 1 T
2. AT is invertible and AT = A ;
1
Proof. 1. We want to verify that the inverse of A is A, i.e.,
1 1
A A = I and AA = I;
which is obvious.
2. Observe that
1 T T
AT A = A 1
A = I T = I;
and
1 T 1 T
A AT = A A = I:

3.3 Elementary matrices

Below, we recall the de…nition of elementary row operations on a matrix A 2Mm n presented in De…nition
34.
De…nition 91 An elementary row operation on a matrix A 2Mm n is one of the following operations on the
rows of A:
[E1 ] (Row interchange) Interchange Ri with Rj , denoted by Ri $ Rj ;
[E2 ] (Row scaling) Multiply Ri by k 2 Rn f0g, denoted by kRi ! Ri ; k 6= 0;
[E3 ] (Row addition) Replace Ri by ( k times Rj plus Ri ), denoted by Ri + kRj ! Ri .
Sometimes we apply [E2 ] and [E3 ] in one step, i.e., we perform the following operation
[E 0 ] Replace Ri by ( k 0 times Rj and k 2 Rn f0g times Ri ), denoted by k 0 Rj + kRi ! Ri ; k 6= 0.
De…nition 92 Let E be the set of functions E : Mm;n ! Mm;n which associate with any matrix A 2 Mm;n
a matrix E (A) obtained from A via an elementary row operation presented in De…nition 91. For i 2 f1; 2; 3g,
let Ei E be the set of elementary row operation functions of type i presented in De…nition 91.
De…nition 93 For any E 2 E, de…ne
EE = E (Im ) 2 Mm;m :
EE is called the elementary matrix corresponding to the elementary row operation function E.
With some abuse of terminology, we call any E 2 E an elementary row operation (omitting the word
“function”), and we sometimes omit the subscript E.
Proposition 94 Each elementary row operations E1 ; E2 and E3 has an inverse, and that inverse is of the
same type, i.e., for i 2 f1; 2; 3g, E 2 Ei , E 1 2 Ei .
Proof. 1. The inverse of Ri $ Rj is Rj $ Ri :
2. The inverse of kRi ! Ri ; k 6= 0 is k 1 Ri ! Ri :
3. The inverse of Ri + kRj ! Ri is kRj + Ri ! Ri :
Remark 95 Given the row, canonical1 vectors eim , for i 2 f1; :::; mg,
2 1 3
em
6 ::: 7
6 i 7
6 em 7
6 7
Im = 6 6 :::
7
7
6 ejm 7
6 7
4 ::: 5
enm
1 See De…nition 55.
3.3. ELEMENTARY MATRICES 35

The following Proposition shows that the result of applying an elementary row operation E to a matrix
A can be obtained by premultiplying A by the corresponding elementary matrix EE .

Proposition 96 For any A 2Mm;n and for any E 2 E,

E (A) = E (Im ) A := EE A: (3.11)

Proof. Recall that 2 3

R1 (A)
6 ::: 7
6 7
6 Ri (A) 7
6 7
A=6
6 ::: 7
7
6 Rj (A) 7
6 7
4 ::: 5
Rm (A)
We have to prove that (3:11) does hold true 8E 2 fE1 ; E2 ; E3 g.
1. E 2 E1 :
First of all observe that
2 1 3 2 3
em R1 (A)
6 ::: 7 6 ::: 7
6 j 7 6 7
6 em 7 6 Rj (A) 7
6 7 6 7
E (Im ) = 6
6 :::
7
7 and E (A) = 6
6 ::: 7:
7
6 eim 7 6 Ri (A) 7
6 7 6 7
4 ::: 5 4 ::: 5
emm Rm (A)

From (3:6), 2 3 2 3
e1m A R1 (A)
6 ::: 7 6 ::: 7
6 7 6 7
6 ejm A 7 6 Rj (A) 7
6 7 6 7
E (Im ) A = 6
6 ::: 7=6
7 6 ::: 7;
7
6 eim A 7 6 Ri (A) 7
6 7 6 7
4 ::: 5 4 ::: 5
emm A Rm (A)
as desired.
2. E 2 E2 :
Observe that
2 3 2 3
e1m R1 (A)
6 ::: 7 6 ::: 7
6 7 6 7
6 k eim 7 6 k Rj (A) 7
6 7 6 7
E (Im ) = 6
6 ::: 7
7 and E (A) = 6
6 ::: 7:
7
6 ejm 7 6 Ri (A) 7
6 7 6 7
4 ::: 5 4 ::: 5
emm Rm (A)
2 3 2 3
e1m A R1 (A)
6 ::: 7 6 ::: 7
6 7 6 7
6 k eim A 7 6 k Ri (A) 7
6 7 6 7
E (Im ) A = 6
6 ::: 7=6
7 6 ::: 7;
7
6 ejm A 7 6 Rj (A) 7
6 7 6 7
4 ::: 5 4 ::: 5
emm A Rm (A)
as desired.
3. E 2 E3 :
36 CHAPTER 3. MATRICES

Observe that
2 3 2 3
e1m R1 (A)
6 ::: 7 6 ::: 7
6 7 6 7
6 eim + k ejm 7 6 Ri (A) + k Rj (A) 7
6 7 6 7
E (Im ) = 6
6 ::: 7
7 and E (A) = 6
6 ::: 7:
7
6 ejm 7 6 Ri (A) 7
6 7 6 7
4 ::: 5 4 ::: 5
emm Rm (A)
2 3 2 3 2 3
e1m e1m A R1 (A)
6 ::: 7 6 ::: 7 6 ::: 7
6 7 6 7 6 7
6 eim + k ejm 7 6 eim + k ejm A 7 6 Ri (A) + k Rj (A) 7
6 7 6 7 6 7
E (Im ) A = 6
6 ::: 7A = 6
7 6 ::: 7=6
7 6 ::: 7;
7
6 ejm 7 6 ejm A 7 6 Rj (A) 7
6 7 6 7 6 7
4 ::: 5 4 ::: 5 4 ::: 5
emm emm A Rm (A)
as desired.

Corollary 97 If A is row equivalent to B, then there exist k 2 N and elementary matrices E1 ; :::; Ek such
that
B = E1 E2 ::: Ek A

Proof. It follows from the de…nition of row equivalence and Proposition 96.
1
Proposition 98 Every elementary matrix EE is invertible and (EE ) is an elementary matrix. In fact,
1
(EE ) = EE 1 :

Proof. Given an elementary matrix E, from De…nition 93, 9E 2 E such that

E = E (I) (3.12)

De…ne
E0 = E 1
(I) :

Then
def. inv. func. 1 (3:12) 1 Prop. (96). 1 def. E 0
I = E (E (I)) = E (E) = E (I) E = E0E

and
def. inv. 1 def. E 0 Prop. (96). (3:12)
I = E E (I) = E (E 0 ) = E (I) E 0 = EE 0 :

Corollary 99 If E1 ; :::; Ek are elementary matrices, then

P := E1 E2 ::: Ek

is an invertible matrix.

Proof. It follows from Proposition 88 and Proposition 98. In fact, Ek 1 :::E2 1 E1 1 is the inverse of
P:

Proposition 100 Let A 2Mm n be given. Then, there exist a matrix B 2Mm n in row canonical form,
k 2 N and elementary matrices E1 ; :::; Ek such that

B = E1 E2 ::: Ek A:
3.3. ELEMENTARY MATRICES 37

Proof. From Proposition 38, there exist k 2 N elementary operations E 1 ; :::; E k such that

E 1 E 2 ::: E k (A) = B:

From Proposition 96, 8j 2 f1; :::; kg ;

E j (M ) = E j (I) M := Ej M:

Then,

E 1 E 2 ::: E k (A) = E 1 E 2 ::: E k 1

E k (A) = E 1 E 2 ::: E k 1
(Ek A) =

= E 1 E 2 ::: E k 2
Ek 1
(Ek A) = E 1 E 2 ::: E k 2
(Ek 1 Ek A) =

= E 1 E 2 :::E k 3
Ek 2
(Ek 1 Ek A) = E 1 E 2 :::E k 3
(Ek 2 Ek 1 Ek A) =

::: = E1 E2 ::: Ek A;

as desired.

Remark 101 In fact, in Proposition 157, we will show that the matrix B of the above Corollary is unique.

Proposition 102 To be row equivalent is an equivalence relation.

Proof. Obvious.

Proposition 103 8n 2 Nn f0g , An n is in row canonical form and it is invertible , A = I.

Proof. [(]Obvious.
[)]
We proceed by induction on n:
Case 1. n = 1.
The case n = 1 is obvious. To try to better understand the logic of the proof, take n = 2, i.e., suppose
that
a11 a12
A=
a21 a22
is in row canonical form and invertible. Observe that A 6= 0.
1. a11 = 1. Suppose a11 = 0. Then, from 1. in the de…nition of matrix in echelon form - see De…nition
28 - a12 6= 0 (otherwise, you would have a zero row not on the bottom of the matrix). Then, from 2. in
that de…nition, we must have a21 = 0. But then the …rst column is zero, contradicting the fact that A is
invertible - see Remark 87. Since a11 6= 0, then from 2. in the De…nition of row canonical form matrix - see
De…nition 31 - we get a11 = 1.
2. a21 = 0. It follows from the fact that a11 = 1 and 3. in De…nition 31.
3. a22 = 1: Suppose a22 = 0, but then the last row would be zero, contradicting the fact that A is
invertible and a22 is the leading nonzero entry of the second row, i.e., a22 6= 0. Then from 2. in the
De…nition of row canonical form matrix, we get a22 = 1.
4. a12 = 0. It follows from the fact that a22 = 1 and 3. in De…nition 31.
Case 2. Assume that statement is true for n 1.
Suppose that 2 3
a11 a12 ::: a1j ::: a1n
6 a21 a22 ::: a2j ::: a2n 7
6 7
6 ::: 7
A=6 6 ai1 ai2 ::: aij ::: ain 7
7
6 7
4 ::: 5
an1 an2 ::: anj ::: ann
is in row canonical form and invertible.
1. a11 = 1. Suppose a11 = 0. Then, from 1. in the de…nition of matrix in echelon form - see De…nition
28 -
(a12 ; :::; a1n ) 6= 0:
38 CHAPTER 3. MATRICES

Then, from 2. in that de…nition, we must have

2 3
a21
6 ::: 7
6 7
6 ai1 7 = 0:
6 7
4 ::: 5
an1

But then the …rst column is zero, contradicting the fact that A is invertible - see Remark 87. Since a11 6= 0,
then from 2. in the De…nition of row canonical form matrix - see De…nition 31 - we get a11 = 1.
2. Therefore, we can rewrite the matrix as follows
2 3
1 a12 ::: a1j ::: a1n
6 0 a22 ::: a2j ::: a2n 7
6 7
6 ::: 7 1 a
A=6 7
6 0 ai2 ::: aij ::: ain 7 = 0 A22 (3.13)
6 7
4 ::: 5
0 an2 ::: anj ::: ann

with obvious de…nitions of a and A22 . Since, by assumption, A is invertible, there exists B which we can
partition in the same we partitioned A, i.e.,

b11 b
B= :
c B22

and such that B is invertible. Then,

b11 b 1 a b11 b11 + bB22 1 0

In = BA = = = ;
c B22 0 A22 c ca + B22 A22 0 In 1

then c = 0 and A22 B22 = In 1:

Moreover,

1 a b11 b b11 + ac b + aB22 1 0

In = AB = = = : (3.14)
0 A22 c B22 c A22 B22 0 In 1

Therefore, A22 is invertible. From 3.13, A22 can be obtained from A erasing the …rst row and then erasing
a column of zero, from Remark 33, A22 is a row reduced form matrix. Then, we can apply the assumption
of the induction argument to conclude that A22 = In 1 . Then, from 3.13,

1 a
A= :
0 I

Since, by assumption, An n is in row canonical form, from 3. in De…nition 31, a = 0, and, as desired
A = I.

Proposition 104 Let A belong to Mm;m . Then the following statements are equivalent.

1. A is invertible;
2. A is row equivalent to Im ;
3. A is the product of elementary matrices.

Proof. 1: ) 2:
From Proposition 100, there exist a matrix B 2Mm m in row canonical form, k 2 N and elementary
matrices E1 ; :::; Ek such that
B = E1 E2 ::: Ek A:
Since A is invertible and, from Corollary 99, E1 E2 ::: Ek is invertible as well, from Proposition 88, B
is invertible as well. Then, from Proposition 103, B = I.
2: ) 3:
3.3. ELEMENTARY MATRICES 39

By assumption and from Corollary 97, there exist k 2 N and elementary matrices E1 ; :::; Ek such that

A = E1 E2 ::: Ek I;

Since 8i 2 f1; :::; kg, Ei is an elementary matrix, the desired result follows.
3: ) 1:
By assumption, there exist k 2 N and elementary matrices E1 ; :::; Ek such that

A = E1 E2 ::: Ek :

Since, from Proposition 98, 8i 2 f1; :::; kg, Ei is invertible, A is invertible as well, from Proposition 88.

Proposition 105 Let Am n be given.

1. Bm n is row equivalent to Am n , there exists an invertible Pm m such that B = P A.
2. Pm m is an invertible matrix ) P A is row equivalent to A:

Proof. 1.
[)] From Corollaries 99 and 97, B = E1 ::: Ek A with (E1 ::: Ek ) invertible matrix. Then, it su¢ ces
to take P = E1 ::: Ek .
[(] From Proposition 104, P is row equivalent to I, i.e., there exist E1 ; :::; Ek such that P = E1 ::: Ek I.
Then by assumption B = E1 ::: Ek I A, i.e., B is row equivalent to A.
2.
From Proposition 104, P is the product of elementary matrices. Then, the desired result follows from
Proposition 96.

Proposition 106 If A is row equivalent to a matrix with a zero row, then A is not invertible.

Proof. Suppose otherwise, i.e., A is row equivalent to a matrix C with a zero row and A is invertible.
From Proposition 105, there exists an invertible P such that A = P C and then P 1 A = C. Since A
and P 1 are invertible, then, from Proposition 88, P 1 A is invertible, while C, from Remark 87, C is not
invertible, a contradiction.

Remark 107 From Proposition 104, we know that if Am m is invertible, then there exist E1 ; :::; Ek such
that
I = E1 ::: Ek A (3.15)
or
1
A = E1 ::: Ek I: (3.16)
Then, from (3:15)and (3:16), if A is invertible then A 1 is equal to the …nite product of those elementary
matrices which “transform”A in I, or, equivalently, can be obtained applying a …nite number of corresponding
elementary operations to the identity matrix I. That observation leads to the following (Gaussian elimination)
algorithm, which either show that an arbitrary matrix Am m is not invertible or …nds the inverse of A.

An algorithm to …nd the inverse of a matrix Am m or to show the matrix is not invertible.

Step 1. Construct the following matrix Mm (2m) :

A Im

Step 2. Row reduce M to echelon form. If the process generates a zero row in the part of M corresponding
to A, then stop: A is not invertible : A is row equivalent to a matrix with a zero row and therefore,
from Proposition 106 is not invertible. Otherwise, the part of M corresponding to A is a triangular
matrix.
Step 3. Row reduce M to the row canonical form

Im B

1
Then, from Remark 107, A = B.
40 CHAPTER 3. MATRICES

Example 108 We …nd the inverse of 2 3

1 0 2
A=4 2 1 3 5;
4 1 8
applying the above algorithm.
Step 1. 2 3
1 0 2 1 0 0
M =4 2 1 3 0 1 0 5:
4 1 8 0 0 1
Step 2. 2 3 2 3
1 0 2 j 1 0 0 1 0 2 j 1 0 0
4 0 1 1 j 2 1 0 5;4 0 1 1 j 2 1 0 5
0 1 0 j 4 0 1 0 0 1 j 6 1 1
The matrix is invertible.
Step 3. 2 3 2 3
1 0 2 j 1 0 0 1 0 2 j 1 0 0
4 0 1 1 j 2 1 0 5;4 0 1 1 j 2 1 0 5;
0 0 1 j 6 1 1 0 0 1 j 6 1 1
2 3 2 3
1 0 0 j 11 2 2 1 0 0 j 11 2 2
4 0 1 0 j 4 0 1 5;4 0 1 0 j 4 0 1 5
0 0 1 j 6 1 1 0 0 1 j 6 1 1
Then 2 3
11 2 2
A 1
=4 4 0 1 5:
6 1 1
Example 109
1 3 j 1 0
4 2 j 0 1
1 3 j 1 0
0 10 j 4 1
1 3 j 1 0
4 1
0 1 j 10 10
2 0
1 0 j 10 3
4 1
0 1 j 10 10

3.4 Elementary column operations

This section repeats some of the discussion of the previous section using column instead of rows of a matrix.
De…nition 110 An elementary column operation is one of the following operations on the columns of Am n:

[F1 ] (Column interchange) Interchange Ci with Cj , denoted by Ci $ Cj ;

[F2 ] (Column scaling) Multiply Ci by k 2 Rn f0g, denoted by kCi ! Ci ; k 6= 0;
[F3 ] (Column addition) Replace Ci by ( k times Cj plus Ci ), denoted by (Ci + kCj ) ! Ci ..
Each of the above column operation has an inverse operation of the same type just like the corresponding
row operations.
De…nition 111 Let F be an elementary column operation on a matrix Am n. We denote the resulting
matrix by F (A). We de…ne also
FF = F (In ) 2 Mn;n :
FF is then called an elementary matrix corresponding to the elementary column operation F. We sometimes
omit the subscript F.
3.4. ELEMENTARY COLUMN OPERATIONS 41

De…nition 112 Given an elementary row operation E, de…ne FE , if it exists2 , as the column operation
obtained by E substituting the word row with the word column. Similarly, given an elementary column
operation F de…ne EF , if it exists, as the row operation obtained by F substituting the word column with the
word row.

In what follows, F and E are such that F = F E and EF = E.

Proposition 113 Let a matrix Am n be given. Then

T
F (A) = E AT :
T
Proof. The above fact is equivalent to E AT = (F (A)) and it is a consequence of the fact that the
columns of A are the rows of AT and vice versa. As an exercise, carefully do the proof in the case of each of
the three elementary operation types.

Remark 114 The above Proposition says that applying the column operation F to a matrix A gives the
same result as applying the corresponding row operation EF to AT and then taking the transpose.

Proposition 115 Let a matrix Am n be given. Then

1.
T
F (A) = A (E (I)) = A F (I) ;
or, since E := E (I) and F := F (I),
F (A) = A E T = A F: (3.17)
2. F = E T and F is invertible.

Proof. 1.
Lemma 113 T Lemma 96 T T Lemma 113
F (A) = = E AT = = E (I) AT = A (E (I)) = A F (I) :

2. From (3:17), we then get

F := F (I) = I E T = E T :
From Proposition 90 and Proposition 98, it follows that F is invertible.

Remark 116 The above Proposition says that says that the result of applying an elementary column oper-
ation F on a matrix A can be obtained by postmultiplying A by the corresponding elementary matrix F .

De…nition 117 A matrix Bm n is said column equivalent to a matrix Am n if B can be obtained from A
using a …nite number of elementary column operations.

Remark 118 By de…nition of row equivalent, column equivalent and transpose of a matrix, we have that

A and B are row equivalent , AT and B T are column equivalent,

and
A and B are column equivalent , AT and B T are row equivalent.

Proposition 119 1. Bm n is column equivalent to Am n , there exists an invertible Qn n such that

Bm n = Am n Qn n .
2. Qn n in invertible matrix ) AQ is column equivalent to A.

Proof. It is very similar to the proof of Proposition 105.

De…nition 120 A matrix Bm n is said equivalent to a matrix Am n if B can be obtained from A using a
…nite number of elementary row and column operations.
2 Of course, if you exchange the …rst and the third row, and the matrix has only two columns, you cannot exchange the …rst

and the third column.

42 CHAPTER 3. MATRICES

Proposition 121 A matrix Bm nis equivalent to a matrix Am n , there exist invertible matrices Pm m
and Qn n such that Bm n = Pm m Am n Qn n .

Proof. [)]
By assumption B = E1 ::: Ek A F1 ::: Fh .
[(]
Similar to the proof of Proposition 105.

Proposition 122 For any matrix Am n there exists a number r 2 f0; 1; :::; min fm; ngg such that A is
equivalent to the block matrix of the form
Ir 0
: (3.18)
0 0

Proof. The proof is constructive in the form of an algorithm.

Step 1. Row reduce A to row canonical form, with leading nonzero entries a11 ; a2j2 ; :::; ajjr .
Step 2. Interchange C 2 and Cj2 , C 3 and Cj3 and so on up to Cr and Cjr :You then get a matrix of the form

Ir B
:
0 0

Step 3. Use column operations to replace entries in B with zeros.

Remark 123 From Proposition 157 the matrix in Step 2 is unique and therefore the resulting matrix in
Step 3, i.e., matrix (3:18) is unique.

Proposition 124 For any A 2Mm;n , there exists invertible matrices P 2Mm;m and Q 2Mn;n and r 2
f0; 1; :::; min fm; ngg such that
Ir 0
P AQ =
0 0

Proof. It follows immediately from Propositions 122 and 121.

Remark 125 From Proposition 157 the number r in the statement of the previous Proposition is unique.

Example 126 Take 2 3

1 2 3
A=4 4 5 6 5
5 7 9
Then 2 3 2 3
1 2 3 R3 (R1 +R2 )!R3 1 2 3
4 4 5 6 5 ! 4 4 5 6 5 !
5 7 9 0 0 0
2 3 2 3
2 1 2
1 2 3 1 2 2 1 2 3
R 4R !R 3 R !R
! 4 0 3 6 5 ! 4 0 1 2 5 !
0 0 0 0 0 0
2 3 2 3 2 3
1 0 3 1 0 0 1 0 0
C 2 2C 1 !C 2 C3 3C 1 !C 3 C3 2C 2 !C 3
! 4 0 1 2 5 ! 4 0 1 2 5 ! 4 0 1 0 5
0 0 0 0 0 0 0 0 0
We can …nd matrices P and Q as follows.
2 32 3 2 3
1 0 0 1 2 3 1 2 3
4 0 1 0 54 4 5 6 5=4 4 5 6 5
1 1 1 5 7 9 0 0 0
3.4. ELEMENTARY COLUMN OPERATIONS 43

2 32 3 2 3
1 0 0 1 2 3 1 2 3
4 4 1 0 54 4 5 6 5 = 4 0 3 6 5
0 0 1 0 0 0 0 0 0
2 32 3 2 3
1 0 0 1 2 3 1 2 3
4 0 1
0 54 0 3 6 5=4 0 1 2 5
3
0 0 1 0 0 0 0 0 0
Then 2 32 32 3 2 3
1 0 0 1 0 0 1 0 0 1 0 0
P =4 0 1
3 0 54 4 1 0 54 0 1 0 5 = 4 43 1
3 0 5;
0 0 1 0 0 1 1 1 1 1 1 1
indeed 2 32 3 2 3
1 0 0 1 2
3 1 2 3
4 4 1
0 54 4 6 5=4 0
5 1 2 5:
3 3
1 1 1 5 7
9 0 0 0
2 32 3 2 3
1 2 3 1 2 0 1 0 3
4 0 1 2 54 0 1 0 5=4 0 1 2 5
0 0 0 0 0 1 0 0 0
2 32 3 2 3
1 0 3 1 0 3 1 0 0
4 0 1 2 54 0 1 0 5=4 0 1 2 5
0 0 0 0 0 1 0 0 0
d
Then 2 32 32 3 2 3
1 2 0 1 0 3 1 0 0 1 2 1
Q=4 0 1 0 54 0 1 0 54 0 1 2 5=4 0 1 2 5;
0 0 1 0 0 1 0 0 1 0 0 1
indeed, 2 32 3 2 3
1 2 3 1 2 1 1 0 0
4 0 1 2 54 0 1 2 5=4 0 1 0 5:
0 0 0 0 0 1 0 0 0
Summarizing
2 32 32 3 2 3
1 0 0 1 2 3 1 2 1 1 0 0
P AQ = 4 4
3
1
3 0 54 4 5 6 54 0 1 2 5=4 0 1 0 5
1 1 1 5 7 9 0 0 1 0 0 0
1
Proposition 127 If Am m Bm m = I, then BA = I and therefore A is invertible and A = B.
Proof. Suppose A is not invertible, the from Proposition 104 is not row equivalent to Im and from
Proposition 122, A is equivalent to a block matrix of the form displayed in (3:18) with r < m. Then, from
Proposition 121, there exist invertible matrices Pm m and Qm m such that
Ir 0
P AQ = ;
0 0
and from AB = I, we get
1
P = P AQQ B
and
Ir 0 1
Q B =P
0 0
Therefore, P has some zero rows and columns, contradicting that P is invertible.
Remark 128 The previous Proposition says that to verify that A in invertible it is enough to check that
AB = I.
Remark 129 We will come back to the analysis of further properties of the inverse and on another way of
computing it in Section 5.3.
44 CHAPTER 3. MATRICES

3.5 Exercises
From Lipschutz (1991),
starting from page 81: 3.1 - 3.11, 3.14 - 3.16;
starting from page 111: 4.1, 4.4, 4.5, 4.7, 4.8.
Chapter 4

Vector spaces

4.1 De…nition
De…nition 130 Let a nonempty set F with the operations of
addition which assigns to any x; y 2 F an element denoted by x y 2 F; and
multiplication which assigns to any x; y 2 F an element denoted by x y 2 F
be given. (F; ; ) is called a …eld, if the following properties hold true.
1. (Commutative) 8x; y 2 F , x y=y x and x y=y x;
2. (Associative) 8x; y; z 2 F; (x y) z=x (y z) and (x y) z=x (y z);
3. (Distributive) 8x; y; z 2 F; x (y z) = (x y) (x z);
4. (Existence of null elements) 9f0 ; f1 2 F such that 8x 2 F , f0 x = x and f1 x = x;
5. (Existence of a negative element) 8x 2 F 9y 2 F such that x y = f0 ;
From the above properties, it follows that f0 and f1 are unique.1 We denote f0 and f1 by 0 and 1,
respectively.
6. (Existence of an inverse element) 8x 2 F n f0g, 9y 2 F such that x y = 1.
Elements of a …eld are called scalars.
Example 131 The set R of real numbers with the standard addition and multiplication is a …eld. From the
above properties all the rules of “elementary” algebra can be deduced.2 The set C of complex numbers is a
…eld .
The sets N := f1; 2; ; 3; :::; n; :::g and Z of positive integers and integers, respectively, with the standard
addition and multiplication are not …elds.
De…nition 132 Let (F; ; ) be a …eld and V be a nonempty set with the operations of
addition which assigns to any u; v 2 V an element denoted by u + v 2 V; and
scalar multiplication which assigns to any u 2 V and any 2 F an element u2V.
Then (V; +; ) is called a vector space on the …eld (F; ; ) and its elements are called vectors if the
following properties are satis…ed.
A1. (Associative) 8u; v; w 2 V; (u + v) + w = u + (v + w);
A2. (existence of zero element) there exists an element 0 in V such that 8u 2 V , u + 0 = u;
A3. (existence of inverse element) 8u 2 V 9v 2 V such that u + v = 0;
A4. (Commutative) 8u; v 2 V , u + v = v + u;
M1. (distributive) 8 2 F and 8u; v 2 V; (u + v) = u+ v;
M2. (distributive) 8 ; 2 F and 8u 2 V; ( ) u= u+ u;
M3. 8 ; 2 F and 8u 2 V; ( ) u= ( u);
M4. 8u 2 V; 1 u = u.
Elements of a vector space are called vectors.
1 The proof of that result is very similar to the proof of Proposition 134.1 and 2.
2 See, for example, Apostol (1967), Section 13.2, page 17.

45
46 CHAPTER 4. VECTOR SPACES

Remark 133 In the remainder of these notes, if no confusion arises, for ease of notation, we will denote a
…eld simply by F and a vector space by V . Moreover, we will write + in the place of , and we will omit
and , i.e., we will write xy instead of x y and v instead of v.

Proposition 134 If V is a vector space, then (as a consequence of the …rst four properties)
1. The zero vector is unique and it is denoted by 0.
2. 8u 2 V , the inverse element of u is unique and it is denoted by u.
3. (cancellation law) 8u; v; w 2 V;

u + w = v + w ) u = v:

Proof. 1. Assume that there exist 01 ; 02 2 V which are zero vectors. Then by de…nition of zero vector -
see (A2) - we have that
01 + 02 = 01 and 02 + 01 = 02 :
From (A:4) ;
01 + 02 = 02 + 01 ;
and therefore 01 = 02 .
2. Given u 2 V , assume there exist v 1 ; v 2 2 V such that

u + v 1 = 0 and u + v 2 = 0:

Then
v2 = v2 + 0 = v2 + u + v1 = v2 + u + v1 = u + v2 + v1 = 0 + v1 = v1 :
3.
(1) (2) (3)
u + w = v + w ) u + w + ( w) = v + w + ( w) ) u + 0 = v + 0 ) u = v;
where (1) follows from the de…nition of operation, (2) from the de…nition of w and (3) from the de…nition
of 0.

Remark 135 From A2. in De…nition 132, we have that for any vector space V , 0 2 V:

Proposition 136 If V is a vector space over a …eld F , then

1. For 0 2 F and 8u 2 V , 0u = 0:
2. For 0 2 V and 8 2 F , 0 = 0:
3. If 2 F , u 2 V and u = 0, then either = 0 or u = 0 or both.
4. 8 2 F and 8u 2 V , ( )u = ( u) = ( u) := u.

Proof. 1. From (M 1),

0u + 0u = (0 + 0) u = 0u:
Then, adding (0u) to both sides,

0u + 0u + ( (0u)) = 0u + ( (0u))

and, using (A3),

0u + 0 = 0
and, using (A2), we get the desired result.
2. From (A2),
0 + 0 = 0;
then multiplying both sides by and using (M 1),

0= (0 + 0) = 0 + 0;

and, using (A3),

0 + ( ( 0)) = 0 + 0 + ( ( 0))
4.2. EXAMPLES 47

and, using (A2), we get the desired result.

3. Assume that u = 0 and 6= 0. Then
1 1 1
u = 1u = u= ( u) = 0 = 0:

Taking the contropositive of the above result, we get hu 6= 0i ) h u 6= 0 _ = 0i. Therefore hu 6= 0 ^ u = 0i )

h = 0i.
4. From u + ( u) = 0, we get (u + ( u)) = 0, and then u + ( u) = 0, and therefore ( u) =
( u).
From +( ) = 0, we get ( + ( )) u = 0u, and then u +( ) u = 0, and therefore ( u) = ( ) u.

Remark 137 From Proposition 136.4, and (M 4)in De…nition 132, we have

( 1) u = 1 ( u) = (1u) = u:

We also de…ne subtraction as follows:

v u := v + ( u)

4.2 Examples
Euclidean spaces.3
The Euclidean space Rn with sum and scalar product de…ned in Chapter 2 is a vector space over the
…eld R with the standard addition and multiplication
Matrices on R.
For any m; n 2 N, the set Mm;n of matrices with elements belonging to the …eld R with the operation of
addition and scalar multiplication, as de…ned in Section 2.3 is a vector space on the …eld R and it is denoted
by
M (m; n) :
Matrices on a …eld F:
For any m; n 2 N, we can also consider the set of matrices whose entries are elements belonging to an
arbitrary …eld F . It is easy to check that set is a vector space on the …eld F; with the operation of addition
and scalar multiplication inherited by F; is a vector space and it is denoted by

MF (m; n) :

We do set
MR (m; n) = M (m; n) :
Polynomials
The set of all polynomials
a0 + a1 t + a2 t2 + ::: + an tn
with n 2 N [ f0g and a0 ; a1 ; a2 ; :::; an 2 R is a vector space on R with respect to the standard sum
between polynomials and scalar multiplication.
Function space F (X).
Given a nonempty set X, the set of all functions f : X ! R with obvious sum and scalar multiplication
is a a vector space on R. More generally we can consider the vector space of functions f : X ! F , where X
is a nonempty set and F is an arbitrary …eld, on the same …eld F .
Sets which are not vector spaces.
(0; +1) and [0; +1) are not a vector spaces in R.
For any n 2 N, the set of all polynomials of degree n is not a vector space on R.
On the role of the field: put Exercise 5.29 from Lipschutz 2nd edition.
3 A detailed proof of some of the statements below is contained, for example, in Ho¤man and Kunze (1971), starting from

page 28.
48 CHAPTER 4. VECTOR SPACES

4.3 Vector subspaces

In what follows, if no ambiguity may arise, we will say “vector space” instead of “vector space on a …eld”.

De…nition 138 Let W be a subset of a vector space V . W is called a vector subspace of V if W is a vector
space with respect to the operation of vector addition and scalar multiplication de…ned on V restricted to W .
In other words, given a vector space (V; +; ) on a …eld (F; ; ) and a subset W of V , we say that W is a
vector subspace of V if, de…ned

+jW W :W W ! W; (w1 ; w2 ) 7! w1 + w2

and
jF W :F W ! W; ( ; w2 ) 7! w2 ;
then W; +jW W ; jF W is a vector space on the …eld (F; ; ).

Proposition 139 Let W be a subset of a vector space V . The following three statements are equivalent.

1. W is a vector subspace of V:
2. a. W 6= ?;
b. 8u; v 2 W; u + v 2 W ;
c. 8u 2 W; 2 F; u 2 W.
3. a. W 6= ?;
b. 8u; v 2 W; 8 ; 2 F; u + v 2 W.

Proof. 2: ) 3:
From 2c., u 2 W and v 2 W . Then, from 2b., u + v 2 W , as desired.
2: ( 3:
From 3b., identifying ; u; ; v with 1; u; 1; v, we get u + v 2 W , i.e., 2b.
From 3b., identifying ; u; ; v with 21 ; u; 12 ; u, we get 12 u + 12 u 2 W . Moreover, since W V , then
u 2 V and from the distributive property (M 1) for V , we get 12 u + 12 u = 12 + 12 u = u.
1: ) 2:
All properties follow immediately from the de…nition of vector space. Let’s check the commutative
property. We want to show that given w1 ; w2 2 W , then w1 + w2 = w2 + w1 . Indeed,
W V
w1 ; w2 2 W ) w1 ; w2 2 V;

and then
def. restriction prop. in V
w1 +jW W w2 = w1 + w2 = w2 + w1 = w2 +jW W w1 :
1: ( 2:
2a., 2b. and 2c. insure that the “preliminary properties” of a vector space are satis…ed: W 6= ? and the
operations are well de…ned.
Properties A1; A4; M 1; M 2; M 3 and M 4 are satis…ed because W V . Let’s check the remaining two
properties.
(A2): We want to show that there exists 0W 2 W such that 8w 2 W , we have w + 0W = w.
Since W 6= ?, we can take w 2 W . Since W V , then w 2 V and from Proposition 136.1,

0F w = 0V : (4.1)

Moreover, from 2c., identifying ; u with 0F ; w, we get 0V 2 W . Then 8w 2 W V , w + 0V = w, i.e.,

0W = 0V := 0: (4.2)

(A3): We want to show that 8w 2 W 9w0 2 W such that w + w0 = 0.

From 2.c, we have ( 1) w 2 W and 1w 2 W . Then
(M 4) (M 1) (136:1)
( 1) w + w = ( 1) w + 1w = ( 1 + 1) w = 0F w = 0:

Taking w0 = ( 1) w, we have shown the desired result.

4.4. LINEAR COMBINATIONS 49

Remark 140 (“A Recipe to try to show that a set is a vector space”) An often successful way to
show that a set S is a vector space is the following one:
1. …nd a vector space V such that S V ; in Section 4.2 above, we provide a list of “commonly used”
vector spaces;
2. use Proposition 139.

Example 141 1. Given an arbitrary vector space V , f0g and V are vector subspaces of V .
2. Given R3 ,
W := (x1 ; x2 ; x3 ) 2 R3 : x3 = 0
is a vector subspace of R3 . Observe that w 2 W if and if only if there exist w1 ; w2 2 R such that
w = (w1 ; w2 ; 0).
3. Given the space V of polynomials, the set W of all polynomials of degree n is a vector subspace of V:
4. The set of all bounded or continuous or di¤ erentiable or integrable functions f : X ! R is a vector
subspace of F (X).
5. If V and W are vector spaces, then V \ W is a vector subspace of V and W .
6. [0; +1) is not a vector subspace of R.
7. Let V = f0g R R2 and W = R f0g R2 . Then V [ W is not a vector subspace of R2 .

4.4 Linear combinations

Notation convention. Unless otherwise stated, a greek (or Latin) letter with a subscript denotes a scalar;
a Latin letter with a superscript denotes a vector.

De…nition 142 Let a vector space V on a …eld F , m 2 N and vectors v 1 ; v 2 ; :::; v m 2 V be given. The linear
combination of the vectors v 1 ; v 2 ; :::; v m via scalars 1 ; 2 ; :::; m 2 F is the vector
m
X
i
iv :
i=1

The set of all such combinations

( m
)
m
X
m i
v 2 V : 9( i )i=1 2F such that v = iv
i=1

is called span of v 1 ; v 2 ; :::; v m and it is denoted by

span v 1 ; v 2 ; :::; v m :

add examples

De…nition 143 Let V be a vector space and S a subset of V . span (S) is the set of all linear combinations
of vectors in S , i.e.,
( m
)
m
X
1 2 m m i
span (S) = v 2 V : 9m 2 N; 9v ; v ; :::; v 2 S; 9 ( i )i=1 2 F such that v = iv :
i=1

span (?) := f0g.

Proposition 144 Let V be a vector space and S 6= ?; S V . Then, “span (S) is the smallest vector space
containing S”, i.e.,

1. a. S span (S) and b. span (S) is a vector subspace of V .

2. If W is a subspace of V and S W , then span (S) W.

50 CHAPTER 4. VECTOR SPACES

Proof. 1a. Given v 2 S, 1v = v 2 span (S) : 1b. Since S = 6 ?, then span (S) 6= ?. Given ; 2 F
1 n 1 m
and P
v; w 2span S: Then 9
Pm 1 ; :::; n 2 F , v ; :::; v 2 S and 1 ; :::; m 2 F , w ; :::; w 2 S; such that
n i j
v = i=1 i v and w = j=1 j w . Then
n
X m
X
i
v+ w= ( i) v + j wj 2 spanS:
i=1 j=1
Pn
2. Take v 2 span S. Then 9 1 ; :::; n 2 F , v 1 ; :::; v n 2 S W such that v = i=1 iv
i
2 W , as desired.

De…nition 145 Let V be a vector space and v 1 ; v 2 ; :::; v m 2 V . If V = span v 1 ; v 2 ; :::; v m , we say that V
is the vector space generated or spanned by the vectors v 1 ; v 2 ; :::; v m .
Example 146 1. R3 = span (f(1; 0; 0) ; (0; 1; 0) ; (0; 0; 1)g);
2. span (f(1; 1)g) = span (f(1; 1) ; (2; 2)g) = (x; y) 2 R2 : x = y ;
3. span (f(1; 1) ; (0; 1)g) = R2 .
2. span ftn gn2N is equal to the vector space of all polynomials.
Exercise 147 1. If A B, then span (A) span (B);
2. if W is a vector subspace of V , then span (W ) = W:

4.5 Row and column space of a matrix

De…nition 148 Given A 2 M (m; n),
row spanA := span R1 (A) ; ::; Ri (A) ; :::; Rm (A)
is called the row space of A or row span of A.
The column space of A or col spanA is
col spanA := span C 1 (A) ; ::; C j (A) ; :::; C n (A) :
Remark 149 Given A 2 M (m; n)
col spanA = row spanAT :
Remark 150 Linear combinations of columns and rows of a matrix.
Let A 2 M (m; n) ; x 2 Rn and y 2 Rm . Then, 8j 2 f1; ::; ng ; C j (A) 2 Rm and 8i 2 f1; :::; mg,
R (A) 2 Rn . Then,
i

Remark 151
n
X
Ax = xj C j (A) ;
j=1

as veri…ed below.
2 3 2 3 2 Pn 3
a11 a12 ::: a1j ::: a1n x1 a1j xj
Pj=1
n
6 a21 a22 ::: a2j ::: a2n 7 6 x2 7 6 j=1 a2j xj
7
6 7 6 7 6 7
6 ::: 7 6 ::: 7 6 7
Ax = 6 7 6 7 = 6 Pn ::: 7;
6 ai1 ai2 ::: aij ::: ain 7 6 xj 7 6 7
6 7 6 7 6 j=1 aij xj 7
4 ::: 5 4 ::: 5 4 5
Pn :::
am1 am2 ::: amj ::: amn xn j=1 amj xj

2 3 2 Pn 3
a1j xj a1j
Pj=1
n
6 a2j 7 6 7
j=1 xj a2j 7
n
X n
X 6 7 6
6 7 6 7
xj C j (A) = xj 6 7 6 Pn 7:
6 aij 7 = 6 x a 7
j=1 j=1 6 7 6 j=1 j ij 7
4 5 4 5
Pn
amj x a
j=1 j mj
4.5. ROW AND COLUMN SPACE OF A MATRIX 51

Therefore,

Ax is a linear combination of the columns of A via the components of the vector x.

Moreover,
m
X
yA = yi Ri (A) ;
i=1

as veri…ed below.
2 3
a11 a12 ::: a1j ::: a1n
6 a21 a22 ::: a2j ::: a2n 7
6 7
6 ::: 7
yA = [y1 ; :::; yi ; :::; ym ] 6
6
7=
6 ai1 ai2 ::: aij ::: ain 7
7
4 ::: 5
am1 am2 ::: amj ::: amn
Pm Pm Pm Pm
= i=1 yi ai1 i=1 yi ai2 ::: i=1 yi aij ::: i=1 yi ain ;

Pm Pm
i=1 yi Ri (A) = i=1 yi ai1 ai2 ::: aij ::: ain =
Pm Pm Pm Pm
= i=1 yi ai1 i=1 yi ai2 ::: i=1 yi aij ::: i=1 yi ain :
Therefore,

yA is a linear combination of the rows of A via the components of the vector y.

As a consequence of the above observation, we have what follow.

1.
row span A = fw 2 Rn : 9 y 2 Rm such that w = yAg :

2.
col span A = fz 2 Rm : 9 x 2 Rn such that z = Axg ;

Proposition 152 Given A; B 2 M (m; n),

1. if A is row equivalent to B, then row spanA = row spanB;
2. if A is column equivalent to B, then col spanA = col spanB.

Proof. 1. B is obtained by A via elementary row operations. Therefore, 8i 2 f1; :::; mg, either
i. Ri (B) = Ri (A), or
ii. Ri (B) is a linear combination of rows of A.
Therefore, row spanB row spanA. Since A is obtained by B via elementary row operations, row spanB
row spanA.
2. if A is column equivalent to B, then AT is row equivalent to B T and therefore, from i. above,
row spanAT = row spanB T . Then the result follows from Remark 149.

Remark 153 Let A 2 M (m; n) be given and assume that

m
X
n
b := (bj )j=1 = ci Ri (A) ;
i=1

i.e., b is a linear combination of the rows of A. Then,

m
X
8j 2 f1; :::; ng ; bj = ci Rij (A) ;
i=1

where 8i 2 f1; :::mg and 8j 2 f1; :::ng ; Rij (A) is the j th component of the i th row Ri (A) of A.
52 CHAPTER 4. VECTOR SPACES

Lemma 154 Assume that A; B 2 M (m; n) are in echelon form with pivots

a1j1 ; :::; aiji ; :::; arjr ;

and
b1k1 ; :::; biki ; :::; bsks ;
respectively, and4 r; s min fm; ng. Then

hrow spanA = row spanBi ) hs = r and for i 2 f1; :::; sg ; ji = ki i :

Proof. Preliminary remark 1. If A = 0, then A = B and s = r = 0:

Preliminary remark 2. Assume that A; B 6= 0 and then s; r 1. We want to verify that j1 = k1 . Suppose
j1 < k1 . Then, by de…nition of echelon matrix, Cj1 (B) = 0, otherwise you would contradict Property 2 of
the De…nition 28 of echelon matrix. Then, from the assumption that row spanA = row spanB;we have that
R1 (A) is a linear combination of the rows of B, via some coe¢ cients c1 ; :::; cm , and from Remark 153 and
the fact that C j1 (B) = 0, we have that a1j1 = c1 0 + :::cm 0 = 0, contradicting the fact that a1j1 is a pivot
for A. Therefore, j1 k1 . A perfectly symmetric argument shows that j1 k1 .
We can now prove the result by induction on the number m of rows.
Step 1. m = 1.
It is basically the proof of Preliminary Remark 2.
Step 2.
Given A; B 2 M (m; n), de…ne A0 ; B 0 2 M (m 1; n) as the matrices obtained erasing the …rst row in
matrix A and B respectively. From Remark 33, A0 and B 0 are still in echelon form. If we show that
row spanA0 = row spanB 0 , from the induction assumption, and using Preliminary Remark 2, we get the
desired result.
m
Let R = (a1 ; :::; an ) be any row of A0 . Since R 2 row spanB, 9 (di )i=1 such that
m
X
R= di Ri (B) :
i=1

Since A is in echelon form and we erased its …rst row, we have that if i j1 = k1 , then ai = 0, otherwise
you would contradict the de…nition of j1 . Since B is in echelon form, each entry in its k1 th column are
zero, but b1k1 which is di¤erent from zero. Then,
m
X
a1k1 = 0 = di bik1 = d1 bik1 ;
i=1
Pm
and therefore d1 = 0, i.e., R = i=2 di Ri (B), or R 2 row spanB 0 , as desired. Symmetric argument
shows the other inclusion.

Remark 155 Given

1 1 2 2
A= and B = ;
2 1 6 3
clearly A 6= B and row spanA = row spanB.

Proposition 156 Assume that A; B 2 M (m; n) are in row canonical form. Then,

hrow spanA = row spanBi , hA = Bi :

Proof. [(] Obvious.

[)] From Lemma 154, the number of pivots in A and B is the same. Therefore, A and B have the same
number s of nonzero rows, which in fact are the …rst s rows. Take i 2 f1; :::; sg. Since row spanA = row spanB,
s
there exists (ch )h=1 such that
Xs
Ri (A) = ch Rh (B) : (4.3)
h=1
4 See Remark 30.
4.5. ROW AND COLUMN SPACE OF A MATRIX 53

We want then to show that ci = 1 and 8l 2 f1; :::; sg n fig, cl = 0.

Let aiji be the pivot of Ri (A) , i.e., aiji is the nonzero ji th component of Ri (A). Then, from Remark
153,
Xs Xs
aiji = ch Rhji (B) = ch bhji : (4.4)
h=1 h=1

From Lemma 154, for i 2 f1; :::; sg ; ji = ki , and therefore biji is a pivot entry for B, and since B is in
row reduced form, biji is the only nonzero element in the ji column of B. Therefore, from (4:4),
s
X
aiji = ch Rhji (B) = ci biji :
h=1

Since A and B are in row canonical form aiji = biji = 1 and therefore

ci = 1:

Now take l 2 f1; :::; sg n fig and consider the pivot element bljl in Rl (B). From (4:3) and Remark 153,
s
X
aijl = ch bhjl = cl ; (4.5)
h=1

where the last equalities follow from the fact that B is in row reduced form and therefore bljl is the only
nonzero element in the jl th column of B, in fact, bljl = 1. From Lemma 154, since bljl is a pivot element
for B, aljl is a pivot element for A. Since A is in row reduced form, aljl is the only nonzero element in
column jl of A. Therefore, since l 6= i, aijl = 0, and from (4:5), the desired result,

8l 2 f1; :::; sg n fig ; cl = 0:;

does follow.

Proposition 157 For every A 2 M (m; n), there exists a unique B 2 M (m; n) which is in row canonical
form and row equivalent to A.

Proof. The existence of at least one matrix with the desired properties is the content of Proposition 38.
Suppose that there exists B1 and B2 with those properties. Then from Proposition 152, we get

row spanA = row spanB1 = row spanB2 :

From Proposition 156,

B1 = B2 :

Corollary 158 1. For any matrix A 2 M (m; n) there exists a unique number r 2 f0; 1; :::; min fm; ngg
such that A is equivalent to the block matrix of the form

Ir 0
:
0 0

2. For any A 2 M (m; n), there exist invertible matrices P 2 M (m; m) and Q 2 M (n; n) and a unique
number r 2 f0; 1; :::; min fm; ngg such that

Ir 0
P AQ =
0 0

Proof. 1.
From Step 1 in the proof of Proposition 122 and from Proposition 157, there exists a unique matrix A
which is row equivalent to A and it is in row canonical form.
54 CHAPTER 4. VECTOR SPACES

From Step 2 and 3 in the proof of Proposition 122 and from Proposition 157, there exist a unique matrix
T
Ir 0
0 0

which is row equivalent to A T and it is in row canonical form. Therefore the desired result follows.
2.
From Proposition 105.2, P A is row equivalent to A; from Proposition 119.2, P AQ is column equivalent
to P A. Therefore, P AQ is equivalent to A:From Proposition 124 ,

Ir0 0
0 0

is equivalent to A. From part 1 of the present Proposition, the desired result then follows.

2 2
Example 159 Let A = be given. Then,
1 1

1 1 1
2 2 2 R !R 1 1 R2 R1 !R2 1 1 C 2 C 1 !C 1 1 0
! ! ! :
1 1 1 1 0 0 0 0

Indeed,
1
1 0 2 0 2 2 1 1 1 0
= ;
1 1 0 1 1 1 0 1 0 0
and
1 1
1 0 2 0 2 0
P = = 1 ;
1 1 0 1 2 1

1 1
Q= :
0 1

4.6 Linear dependence and independence

De…nition 160 Let a vector space V on a …eld F and a number m 2 N be given. The vectors v 1 ; v 2 ; :::; v m 2
V are linearly dependent if
either m = 1 and v 1 = 0,
or m > 1 and 9 k 2 f1; :::mg and there exist (m 1) 1 coe¢ cients j 2 F with j 2 f1; :::mg n fkg such
that X
vk = jv
j

j2f1;:::mgnfkg

or, shortly, X
vk = jv
j

j6=k

i.e., there exists a vector equal to a linear combination of the other vectors.

Geometrical interpretation of linear (in)dependence in R2 :

Proposition 161 Let a vector space V on a …eld F and vectors v 1 ; v 2 ; :::; v m 2 V be given. The vectors
v 1 ; v 2 ; :::; v m 2 V are linearly dependent vectors if and only if
m
X
m
9( 1 ; :::; i ; :::; m ) 2 F n f0g such that i v i = 0; (4.6)
i=1

i.e., there exists a linear combination of the vectors equal to the null vector and with some nonzero
coe¢ cient.
4.6. LINEAR DEPENDENCE AND INDEPENDENCE 55

Proof. [)]
If m = 1, any 2 Rn f0g is such that 0 = 0. Assume then that m > 1. Take

i if i 6= j
i =
1 if i = j
[(]
If m = 1; 9 2 Rn f0g is such that v 1 = 0;then from Proposition 58.3 v 1 = 0. Assume then that
m > 1. Without loss of generality take 1 6= 0. Then,
X
1 i
1v + iv = 0
i6=1

and
X
i
v1 = vi :
i6=1 1

Proposition 162 Let m 2 and v 1 ; :::; v m be nonzero linearly dependent vectors. Then, one of the vectors
k 1 Pk 1
is a linear combination of the preceding vectors, i.e., 9 k > 1 and i i=1 such that v k = i=1 i v i .
m Pm
Proof. Since v 1 ; :::; v m are linearly dependent, 9 ( i )i=1 2 Rm n f0g such that i=1 i v i = 0. Let k be
the largest i such that i 6= 0, i.e.,
9k 2 f1; :::; mg such that k 6= 0 and 8i 2 fk + 1; :::; mg ; i = 0: (4.7)
Pm
Consider the case k = 1. Then we would have 1 6= 0 and 8i > 1, i = 0 and therefore 0 = i=1 i v i =
1 1 m
1 v , contradicting the assumption that v ; :::; v are nonzero vectors. Then, we must have k > 1, and from
(4:7) , we have
X m k
X
i i
0= i v = iv
i=1 i=1
and
k
X1
k i
kv = iv ;
i=1
or, as desired,
k
X1 i
vk = vi :
i=1 k
i
It is then enough to choose i = for any i 2 f1; :::; k 1g.
k

Example 163 Take the vectors x1 = (1; 2); x2 = ( 1; 2) and x3 = (0; 4). x1 ; x2 ; x3 are linearly dependent
vectors: x1 = 1 x2 + 0 x3 . Observe that there are no 1 ; 2 2 R such that x3 = 1 x1 + 2 x2 .
De…nition 164 Let a vector space V be given. The vectors v 1 ; v 2 ; :::; v m 2 V are linearly independent
vectors if they are not linearly dependent.
Remark 165 The vectors v 1 ; v 2 ; :::; v m are linearly independent if h : (4:6)i holds true, i.e., if
Pm
8 ( 1 ; :::; i ; :::; m ) 2 F m n f0g it is the case that i=1 i v i = 6 0,
or
m
X
m
( 1 ; :::; i ; :::; m ) 2 F n f0g ) i v i 6= 0
i=1
or
m
X
i vi = 0 ) ( 1 ; :::; i ; :::; m) =0
i=1
or the only linear combination of the vectors which is equal to the null vector has each coe¢ cient equal to
zero.
56 CHAPTER 4. VECTOR SPACES

Example 166 The vectors (1; 2) ; (1; 5) are linearly independent.

Remark 167 From Remark 150, we have what follows:
hAx = 0 ) x = 0i , hthe column vectors of A are linearly independenti (4.8)
hyA = 0 ) y = 0i , hthe row vectors of A are linearly independenti (4.9)
Example 168 Let V be the vector space of all functions f : R ! R. f; g; h de…ned below are linearly
independent:
f (x) = ex ; g (x) = x2 ; h (x) = x:
Assume that ( 1; 2; 3) 2 R3 are such that
1 f+ 2 g+ 3 h = 0V ;
which means that
8x 2 R; 1 f (x) + 2 g(x) + 3 h(x) = 0:
We want to show that ( 1 ; 2 ; 3 ) = (0; 0; 0): The trick is to …nd appropriate values of x to get the desired
value of ( 1 ; 2 ; 3 ). Choose x to take values 0; 1; 1. We obtain the following system of equations
8
< 1 = 0
e 1 + 2 + 3 = 0
:
e 1 1 + 2 + ( 1) 3 = 0
It then follows that ( 1; 2; 3) = (0; 0; 0), as desired.
Example 169 1. The vectors v 1 ; :::; v m are linearly dependent if 9k 2 f1; :::; mg such that v k = 0:
X
vk + 0 vi = 0
i6=k
0
2. The vectors v 1 ; :::; v m are linearly dependent if 9k; k 0 2 f1; :::; mg and 2 Rn f1g such that v k = v k :
0 X
vk vk + 0 vi = 0
i6=k;k0

Proposition 170 The nonzero rows of a matrix A in echelon form are linearly independent.
Proof. We will show that each row of A starting from the …rst one is not a linear combination of the
subsequent rows. Then, as a consequence of Proposition 162, the desired result will follow.
Since A is in echelon form, the …rst row has a pivot below which all the elements are zero. Then that
row cannot be a linear combination of the following rows. Similar argument applies to the other rows.
Remark 171 (Herstein (1975), page 178) “We point out that linear dependence is a function not only of
the vectors but also of the …eld. For instance, the …eld of complex numbers is a vector space over the …eld of
real numbers and it is also a vector space over the …eld of complex numbers. The elements v1 = 1; v2 = i in
it are linearly independent over the reals, but linearly dependent over the complexes, since iv1 + ( 1) v2 = 0.”
Proposition 172 Let V be a vector space and v 1 ; v 2 ; :::; v m 2 V . If v 1 ; :::; v m are linearly dependent vectors
and v m+1 ; :::; v m+k 2 V are arbitrary vectors, then
v 1 ; :::; v m ; v m+1 ; :::; v m+k
are linearly dependent vectors.
Proof. From the assumptions 9i 2 f1; :::; mg and (aj )j6=i such that
X
vj = jv
j

j6=i

But then X
vj = jv
j
+ 0 v m+1 + 0 v m+k ;
j6=i
as desired
Proposition 173 If v 1 ; :::; v m 2 V are linearly independent vectors, S := v 1 ; :::; v m and S 0 S; then
vectors in S 0 are linearly independent.
Proof. Suppose otherwise, but then you contradict the previous proposition.
4.7. BASIS AND DIMENSION 57

4.7 Basis and dimension

De…nition 174 Given a nonempty set X, a sequence on X is a function x : N ! X, where N N. A
subsequence of x is the restriction of x to a subset of N , i.e., xjN . If the cardinality of N is …nite, then the
sequence is called a …nite sequence. If the cardinality of N is in…nite, the the sequence is called an in…nite
sequence.

Remark 175 Very commonly, for any i 2 N , the value x (i) is denoted by xi ; the sequence is denoted by
(xi )i2N ; the subsequence of x equal to xjN with N f1; :::; ng is denoted by (xi )i2N :

Remark 176 Observe that given x; y 2 X, the two …nite sequences (x; y) and (y; x) are di¤ erent: in other
words, for (…nite) sequences, “order matters”.

Remark 177 Observe that Proposition 173 can be rephrased as follows: A …nite sequence of vectors is
linearly independent if and only if any subsequence is linearly independent. That observation motivates and
makes the following de…nition consistent with the de…nition of linear independent …nite sequences.

De…nition 178 Let N N be given. A sequence (vn )n2N of vectors in a vector space V is linearly inde-
pendent if any …nite subsequence of (vn )n2N is linearly independent. A sequence (vn )n2N of vectors in a
vector space V is linearly dependent if it not linearly independent, i.e., if there exists a …nite subsequence of
(vn )n2N which is linearly dependent.

Remark 179 We consider the following sentences as having the same meaning:
“the vectors v i , with i 2 N , in the vectors space V are linearly dependent (independent) vectors”;
“the sequence (vn )n2N in the vectors space V is linearly dependent (independent)”;
“ (vn )n2N in the vectors space V is linearly dependent (independent)”.

De…nition 180 Let N N be given. A sequence (vn )n2N of vectors in a vector space V on a …eld F is
called a basis of V if

1. (vn )n2N is linearly independent;

2. span (vn )n2N = V .

Lemma 181 Suppose that given a vector space V , span v 1 ; :::; v m = V .

1. If w 2 V , then w; v 1 ; :::; v m are linearly dependent and span w; v 1 ; :::; v m = V ;

2. If v i is a linear combination of v 1 ; :::; v i 1

, then span v 1 ; :::; v i 1
; v i+1 ; :::; v m = V .

Proof. Obvious.

Lemma 182 (Replacement Lemma) Let a vector space V and vectors

v 1 ; :::; v n 2 V

and
w1 ; :::; wm 2 V
be given. If

1. span v 1 ; :::; v n = V ,
2. w1 ; :::; wm are linearly independent,

then

1. n m,
2. a. If n = m, then span w1 ; :::; wm = V .
b. if n > m, there exists v i1 ; :::; v in m
v 1 ; :::; v n such that

span w1 ; :::; wm ; v i1 ; :::; v in m

= V:
58 CHAPTER 4. VECTOR SPACES

Proof. Observe preliminary that since w1 ; :::; wm are linearly independent, for any j 2 f1; :::; mg, wj 6= 0.
We now distinguish 2 cases: Case 1. For any i 2 f1; :::; ng, v i 6= 0, and Case 2. There exists i 2 f1; :::; ng
such that v i = 0.
Case 1.
m
Now consider the case n = 1. w1 ; :::; wm 2 V implies that there exists ( j )j=1 2 Rm such that 8j; j 6= 0
and wj = j v 1 ; then it has to be m = 1 (and conclusion 1 holds) and since w1 = 1 v1 , span w1 = V (and
conclusion 2 holds).
Consider now the case n 2.
First of all, observe that from Lemma 181.1, w1 ; v 1 ; :::; v n are linearly dependent and

span w1 ; v 1 ; :::; v n = V:

By Lemma 162, there exists k1 2 f1; :::; ng such that v k is a linear combination of the preceding vectors.
Then from Lemma 181.2, we have
span w1 ; v i i6=k = V:
1

1 2 i
Then again from Lemma 181.1, w ; w ; v i6=k1
are linearly dependent and

span w1 ; w2 ; v i i6=k1
= V:

By Lemma 162, there exists k2 2 f2; :::; ng n fk1 g such that v k2 or w2 is a linear combination of the preceding
vectors. That vector cannot be w2 because of assumption 2. Therefore,

span w1 ; w2 ; v i i6=k1 ;k2

= V:

We can now distinguish three cases: m < n; m = n and m > n.

Now if m < n; after m steps of the above procedure we get

span w1 ; :::; wm ; v i i6=k1 ;k2 ;::;km

= V;

which shows 2.a. If m = n;we have

span w1 ; :::; wm = V;

which shows 2.b.

Let’s now show that it cannot be m > n. Suppose that is the case. Then, after n of the above steps, we get
span w1 ; :::; wn = V and therefore wn+1 is a linear combination of w1 ; :::; wn , contradicting assumption
2.
Case 2.
In the present case, we assume that there exists a set I0 such that I0 6= ? and I0 f1; :::; ng. De…ne also
I1 = f1; :::; ng nI0 and n1 = #I1 . Clearly,

span v i : i 2 I1 = span v 1 ; :::; v n = V

and n1 < n. Then, repeating the “replacement” argument described in Case 1 applied to

v i such that i 2 I1

and
w1 ; :::; wm ;
we get
(i) n > n1 m, i.e., Conclusion 1 holds and Conclusion 2a does not hold true, and
(ii) there exists
v i1 ; :::; v i#I1 m v i : i 2 I1 v 1 ; :::; v n
such that
span w1 ; :::; wm ; v i1 ; :::; v i#I1 m
= V;
and therefore Conclusion 2b does hold true.
4.7. BASIS AND DIMENSION 59

Proposition 183 Assume that u1 ; u2 ; :::; un and v 1 ; v 2 ; :::; v m are bases of V . Then n = m.

Proof. By de…nition of basis we have that

span u1 ; u2 ; :::; un = V and v 1 ; v 2 ; :::; v m are linearly independent.

Then from Lemma 182, m n. Similarly,

span v 1 ; v 2 ; :::; v m = V and u1 ; u2 ; :::; un are linearly independent,

and from Lemma 182, n m.

The above Proposition allows to give the following De…nition.

De…nition 184 A vector space V has dimension n 2 N if there exists a basis of V with cardinality n. In
that case, we say that V has …nite dimension (equal to n) and we write dim V = n. If a vector space does
not have …nite dimension, it is said to be of in…nite dimension.

De…nition 185 The vector space f0g has dimension 0. A (indeed, the) basis of f0g is the empty set.

Example 186 1. A basis of Rn is e1 ; :::; ei ; :::; en , where ei is de…ned in De…nition 55. That basis is called
canonical basis. Then dim Rn = n.
2. Consider the vector space Pn (t) of polynomials of degree n. t0 ; t1 ; :::tn is a basis of Pn (t) and
therefore dim Pn (t) = n + 1.

Example 187 Put the example of infinite dimensional space from Hoffman and Kunze, page 43.

Proposition 188 Let V be a vector space of dimension n.

1. m > n vectors in V are linearly dependent;

2. If u1 ; :::; un 2 V are linearly independent, then u1 ; :::; un is a basis of V ;

3. If span u1 ; :::; un = V , then u1 ; :::; un is a basis of V:

Proof. Let w1 ; :::; wn be a basis of V .

1. We want to show that v 1 ; :::; v m arbitrary vectors in V are linearly dependent. Suppose otherwise,
then by Lemma 182, we would have m n, a contradiction.
2. It is the content of Lemma 182.2.a.
3. We have to show that u1 ; :::; un are linearly independent. Suppose otherwise. Then there exists
k 2 f1; :::; ng such that span ui i6=k = V , but since w1 ; :::; wn are linearly independent, from
Proposition 182 (the Replacement Lemma), we have n n 1, a contradiction.

Remark 189 The above Proposition 188 shows that in the case of …nite dimensional vector spaces, one of
the two conditions de…ning a basis is su¢ cient to obtain a basis.

Proposition 190 (Completion Lemma) Let V be a vector space of dimension n and w1 ; :::; wm 2 V be
linearly independent, with5 m n. If m < n, then, there exist u1 ; :::; un m such that

w1 ; :::; wm ; u1 ; :::; un m

is a basis of V .

Proof. Take a basis v 1 ; :::; v n of V . Then from Conclusion 2.b in Lemma 182,

span w1 ; :::; wm ; v i1 ; :::; v in m

= V:

Then from Proposition 188.3, we get the desired result.

5 The inequality m n follows from Proposition 188.1.
60 CHAPTER 4. VECTOR SPACES

Proposition 191 Let W be a subspace of an n dimensional vector space V . Then

1. dim W n;

2. If dim W = n, then W = V .

Proof. 1. From Proposition 188.1, m > n vectors in V are linearly dependent. Since the vectors in a
basis of W are linearly independent, then dim W n.
2. If w1 ; :::; wn is a basis of W , then span w1 ; :::; wn = W . Moreover, those vectors are n linearly
independent vectors in V . Therefore from Proposition 188.2., span w1 ; :::; wn = V .

Remark 192 As a trivial consequence of Proposition 190, V = span u1 ; :::; ur ) dim V r.

Example 193 Let W be a subspace of R3 , whose dimension is 3. Then from the previous Proposition,
dim W 2 f0; 1; 2; 3g. In fact,
1. If dim W = 0, then W = f0g, i.e., a point,
2. if dim W = 1, then W is a straight line trough the origin,
3. if dim W = 2, then W is a plane trough the origin,
4. if dim W = 3, then W = R3 .

4.8 Coordinates
Proposition 194 Let u1 ; uP2
; :::; un be a basis of V on a …eld F if and only if 8v 2 V there exists a unique
n n
( i )i=1 2 F such that v = i=1 i ui .
n

n n Pn Pn
Proof. [)] Suppose there exist ( i )i=1 ; ( i )i=1 2 Rn such that v = i=1 i ui = i=1 i ui . Then
n
X n
X n
X
i i i
0= iu iu = ( i i) u :
i=1 i=1 i=1

Since u1 ; u2 ; :::; un are linearly independent,

8i 2 f1; :::; ng ; i i = 0;

as desired.
[(]
PnClearly V = span (S); wePare left with showing that u1 ; u2 ; :::; un are linearly independent. Consider
i n i n n
Pni
i=1 u = 0. Moreover, i=1 0 u = 0. But since there exists a unique ( i )i=1 2 R such that
v = i=1 i ui , it must be the case that 8i 2 f1; :::; ng ; i = 0.
The above proposition allows to give the following de…nition.

De…nition 195 Given a vector space V on a …eld F with a basis V = v 1 ; :::; v n , the associated coordinate
function is
crV : V ! F n ; v 7! crV (v) := [v]V
where crV (v) is the unique vectors of coe¢ cients
Pn which give v as a linear combination of the element of
n
the basis, i.e., crV (v) := ( i )i=1 such that v = i=1 i v i .

4.9 Row and column span

We start our analysis with a needed lemma.

Lemma 196 Given a vector space V , if

1. the vectors v 1 ; :::; v k 2 V are linearly independent, and
2. v k+1 2 V and the vectors v 1 ; :::; v k ; v k+1 are linearly dependent,
then
v k+1 is a linear combination of the vectors v 1 ; :::; v k .
4.9. ROW AND COLUMN SPAN 61

Proof. Since v 1 ; :::; v k ; v k+1 are linearly dependent, then

X
9i 2 f1; :::; k + 1g ; j j2f1;:::;k+1gnfig such that v i = jv
j
:
j2f1;:::;k+1gnfig

If i = k + 1;we are done. If i 6= k + 1, without loss of generality, take i = 1. Then

k+1
X
9 j j2f1;:::;k+1gnf1g such that v 1 = jv
j
:
j=2

Pk
If k+1 = 0, we would have v 1 j=2 jv
j
= 0, contradicting Assumption 1: Then
0 1
k
X
1 @ v1 A:
v k+1 = jv
k+1 j=2

Remark 197 Let V be a vector space and S T V . Then,

1. span S span T ;
2. span (span S) = span S.

De…nition 198 For any A 2 M (m; n),

the row rank of A is the dim(row span of A);
the column rank of A is the dim(col span of A):

Proposition 199 For any A 2 M (m; n), row rank of A is equal to column rank of A.

Proof. Let A be an arbitrary m n matrix

2 3
a11 ::: a1j ::: a1n
6 ::: 7
6 7
6 ai1 ::: aij ::: ain 7
6 7
4 ::: 5
am1 ::: amj ::: amn

Suppose the row rank is r m and the following r vectors form a basis of the row space:
2 3
S1 = [b11 ::: b1j ::: b1n ]
6 ::: 7
6 7
6 Sk = [bk1 ::: bkj ::: bkn ] 7
6 7
4 ::: 5
Sr = [br1 ::: brj ::: brn ]

Then, each row vector of A is a linear combination of the above vectors, i.e., we have
r
X
8i 2 f1; :::; mg ; Ri = ik Sk ;
k=1

or
r
X
8i 2 f1; :::; mg ; ai1 ::: aij ::: ain = ik [bk1 ::: bkj ::: bkn ] ;
k=1

and setting the j component of each of the above vector equations equal to each other, we have
r
X
8j 2 f1; ::; ng and 8i 2 f1; :::; mg ; aij = ik bkj ;
k=1
62 CHAPTER 4. VECTOR SPACES

and 8 Pr
>
> a1j = k=1 1k bkj ;
>
>
< :::: Pr
8j 2 f1; ::; ng ; aij = k=1 ik bkj ;
>
>
>
> ::: Pr
:
amj = k=1 mk bkj ;
or 2 3 2 3
a1j 1k
6 ::: 7 Xr 6 ::: 7
6 7 6 7
8j 2 f1; ::; ng ; 6
6 aij 7 =
7 bkj 6
6 ik
7;
7
4 ::: 5 k=1 4 ::: 5
amj mk

i.e., each column of A is a linear combination of the r vectors

80 1 0 1 0 19
>
> 11 1k 1r >
>
>
> C>
<BB ::: C
C
B :::
B
C
C
B :::
B
>
C=
B i1 C ; :::; B ik C ; :::; B ir C :
> B C B C B C>
>
> @ ::: A @ ::: A @ ::: A>
>
>
: >
;
m1 mk mr

Then, from Remark 192,

dim col span A r = row rank A; (4.10)
i.e.,
col rank A row rank A:
From (4:10) ;which holds for arbitrary matrix A, we also get

dim col span AT row rank AT : (4.11)

Moreover,
Rmk: 149
dim col span AT = dim row span A := row rank A
and
Rmk: 149
row rank AT := dim row span AT = dim col span A:
Then, from the two above equalities and (4:11), we get

row rank A dim col span A; (4.12)

and (4:10) and (4:12) gives the desired result.

Proposition 200 For any A 2 M (m; n),

1. the row rank of A is equal to the maximum number of linearly independent rows of A;
2. the column rank of A is equal to the maximum number of linearly independent columns of A.

Proof. 1.
Let k1 := row rank of A and k2 :=maximum number of linearly independent rows of A. Assume our
claim is false and therefore, either a. k1 > k2 , or b: k1 < k2 :
a.
Let (v 1 ; :::; v k2 ) a …nite sequence of linearly independent rows of A:Since, by assumption, k2 is the
maximum number of linearly independent rows of A, then, from Lemma 196, the other rows of A are a linear
combination of v 1 ; ::; ; v k2 and from Lemma 181, span(v 1 ; :::; v k2 ) = row spanA. Then, (v 1 ; :::; v k2 ) is a basis
of span A and therefore k1 := row rankA := dim row spanA = k2 , contradicting the assumption of case a.
b.
In this case we would have k2 linearly independent vectors in a vector space of dimension k1 , with k2 > k1 ,
contradicting Proposition 188.1 .
2.
The proof is basically the same as the above one.
4.10. EXERCISES 63

Corollary 201 For every A 2 M (m; n),

row rank A = maximum number of linearly independent rows of A =

(4.13)
= maximum number of linearly independent columns of A = col rank A:

Proof. It follows from Propositions 199 and 200.

Exercise 202 Check the above result on the following matrix

2 3
1 2 3
4 5 1 6 5:
3 7 10

4.10 Exercises
Problem sets: 1,2,3,4,5,6,7.
From Lipschutz (1991), starting from page 162:
5.3, 5.7, 5.8, 5.9, 5.10, 5.12 ! 5.15, 5.17 ! 5.23, 5.24 ! 5.29, 5.31 ! 5.34, 5.46 ! 5.49.
64 CHAPTER 4. VECTOR SPACES
Chapter 5

Determinant and rank of a matrix

In this chapter we are going to introduce the de…nition of determinant, an useful tool to study linear
dependence, invertibility of matrices and solutions to systems of linear equations.

5.1 De…nition and properties of the determinant of a matrix

To motivate the de…nition of determinant, we present an informal discussion of a way to …nd solutions to
the linear system with two equations and two unknowns, shown below.

a11 x1 + a12 x2 = b1
(5.1)
a21 x1 + a22 x2 = b2

The system can be rewritten as follows

Ax = b
where
a11 a12 x1 b1
A= ; x= ; b= :
a21 a22 x2 b2
Let’s informally discuss how to …nd solutions to system (5:1). If a22 6= 0 and a12 6= 0, multiplying both
sides of the …rst equation by a22 , of the second equation by a12 and adding up, we get

a11 a22 x1 + a12 a22 x2 a12 a21 x1 a12 a22 x2 = a22 b1 a12 b2
Therefore, if
a11 a22 a12 a21 6= 0
we have
b1 a22 b2 a12
x1 = (5.2)
a11 a22 a12 a21
In a similar manner1 we have

b2 a11 b1 a21
x2 = (5.3)
a11 a22 a12 a21
We can then the following preliminary de…nition: given A 2 M22 , the determinant of A is

a11 a12
det A = det := a11 a22 a12 a21
a21 a22

Using the de…nition of determinant, we can rewrite (5:2) and (5:3)as follows.
1 Assuming a
21 6= 0 and a11 6= 0, multiply both sides of the …rst equation by a21 , of the second equation by a11 and then
add up.

65
66 CHAPTER 5. DETERMINANT AND RANK OF A MATRIX

b1 a12 a11 b1
det det
b2 a22 a21 b2
x1 = and x2 =
det A det A
We can now present the de…nition of the determinant of a square matrix An n for arbitrary n 2 N.

De…nition 203 Given n > 1 and A 2 M (n; n), 8i; j 2 f1; :::; ng, we call Aij 2 M (n 1; n 1) the matrix
obtained from A erasing the i th row and the j th column.

De…nition 204 Given A 2 M (1; 1), i.e., A = [a] with a 2 R. The determinant of A is denoted by det A
and we let det A := a. For Nn f1g, given A 2 M (n; n), we de…ne the determinant of A as
n
X
det A := ( 1)1+j a1j det A1j
j=1

n
Observe that [a1j ]j=1 is the …rst row of A, i.e.,
n
X
det A := ( 1)1+j R1j (A) det A1j
j=1

Example 205 For n = 2, we have

2
X
a11 a12
det = ( 1)1+j a1j det A1j = ( 1)1+1 a11 det A11 + ( 1)1+2 a12 det A12 =
a21 a22
j=1

= a11 a22 a12 a21

and we get the informal de…nition given above.

Example 206 For n = 3, we have

2 3
a11 a12 a13
det A = det 4 a21 a22 a23 5 = ( 1)1+1 a11 det A11 + ( 1)1+2 a12 det A12 + ( 1)1+3 a13 det A13
a31 a32 a33

De…nition 207 Given A = [aij ] 2 M (n; n), , 8i; j; det Aij is called minor of aij in A;

( 1)i+j det Aij

is called cofactor of aij in A.

Theorem 208 Given A 2 M (n; n), det A is equal to the sum of the products of the elements of any rows or
column for the corresponding cofactors, i.e.,
n
X
8i 2 f1; :::ng ; det A = ( 1)i+j Rij (A) det Aij (5.4)
j=1

and
n
X
8j 2 f1; :::ng ; det A = ( 1)i+j C ji (A) det Aij (5.5)
i=1

Proof. Omitted. We are going to omit several proofs about determinants. There are di¤erent ways of
introducing the concept of determinant of a square matrix. One of them uses the concept of permutations
- see, for example, Lipschutz (1991), Chapter 7. Another one is an axiomatic approach - see, for example,
Lang (1971) - he introduces (three) properties that a function f : M (n; n) ! R has to satisfy and then
shows that there exists a unique such function, called determinant. Following the …rst approach the proof
of the present theorem can be found on page 252, in Lipschutz (1991) Theorem 7.8, or following the second
approach, in Lang (1971), page 128, Theorem 4.
5.1. DEFINITION AND PROPERTIES OF THE DETERMINANT OF A MATRIX 67

De…nition 209 The expression used above for the computation of det A in (5.4) is called “(Laplace) expan-
sion” of the determinant by row i:
The expression used above for the computation of det A in (5.5) is called “(Laplace) expansion” of the
determinant by column j.

De…nition 210 Consider a matrix An n : Let 1 k n: A k th order principal submatrix (minor) of A

is the (determinant of the) square submatrix of A obtained deleting (n k) rows and (n k) columns in the
same position.

Theorem 211 (Properties of determinants)

Let the matrix A = [aij ] 2 M (n; n) be given. Properties presented below hold true even if words “column,
columns” are substituted by the words “row, rows”.

1. det A = det AT .

2. if two columns are interchanged, the determinant changes its sign,.

Pp
3. if there exists j 2 f1; ::; ng such that C j (A) = k=1 k bk , then
" p
# p
X X
1 k n 1 k n
det C (A) ; :::; k b ; :::; C (A) = k det C (A) ; :::; b ; :::; C (A) ;
k=1 k=1

i.e., the determinant of a matrix which has a column equal to the linear combination of some vectors is
equal to the linear combination of the determinants of the matrices in which the column under analysis
is each of the vector of the initial linear combination, and, therefore, 8 2 R and 8j 2 f1; :::; ng,

det C 1 (A) ; :::; C j (A) ; :::; C n (A) = det C 1 (A) ; :::; C j (A) ; :::; C n (A) = det A:

4. if 9j 2 f1; :::; ng such that C j (A) = 0, then det A = 0, i.e., if a matrix has column equal to zero, then
the determinant is zero.

5. if 9 j; k 2 f1; :::; ng and 2 Rsuch that C j (A) = C k (A), then det A = 0, i.e., the determinant of a
matrix with two columns proportional one to the other is zero.
P
6. If 9k 2 f1; :::; ng and 1 ; :::; k 1 ; k+1 ; :::; n 2 R such that C k (A) = j6=k j C j (A), then det A = 0,
i.e., if a column is equal to a linear combination of the other columns, then det A = 0.

7. 2 3
X
det 4C 1 (A) ; :::; C k (A) + j C j (A) ; :::; C n (A)5 = det A
j6=k

Pn i+j
8. 8j; j 2 f1; :::; ng ; i=1 aij ( 1) det Aij = 0, i.e., the sum of the products of the elements of a
column times the cofactor of the analogous elements of another column is equal to zero.

9. If A is triangular, det A = a11 ::: a22 :::ann , i.e., if A is triangular (for example, diagonal), the
determinant is the product of the elements on the diagonal.

Proof. 1.
Consider the expansion of the determinant by the …rst row for the matrix A and the expansion of the
determinant by the …rst column for the matrix AT .
2.
We proceed by induction. Let A be the starting matrix and A0 the matrix with the interchanged columns.
P (2) is obviously true.
P (n 1) ) P (n)
Expand det A and det A0 by a column which is not any of the interchanged ones:
n
X
det A = ( 1)i+j Cji (A) det Aij
i=1
68 CHAPTER 5. DETERMINANT AND RANK OF A MATRIX

n
X
det A0 = ( 1)i+k Cji (A) det A0ij
i=1

Since 8k 2 f1; :::; ng, Akj , A0kj 2 M (n 1; n 1), and they have interchanged column, by the induction
argument, det Akj = det A0kj , and the desired result follows.
3.
Observe that !n
Xp X p
k k
kb = k bi :
k=1 k=1 i=1
Then, Pp
det C 1 (A) ; :::; k=1 kb
k
; :::; C n (A) =
2 2 k
3 3
b1
6 Xp 6 ::: 7 7
6 1 6 k 7 7
= det 6
6C (A) ; :::; 6 7 n
k 6 bi 7 ; :::; C (A)7 =
7
4 k=1 4 ::: 5 5
k
bn
2 2 Pp k
3 3
k=1 k b1
6 6 7 7
6 1 6 Pp ::: k 7 7
= det 6
6C (A) ; :::; 6
6 k=1 k i 7 b 7 ; :::; C n
(A)7=
7
4 4 ::: 5 5
Pp k
k=1 k n b
n p
!
X i+j
X
k
= ( 1) k bi det Aij =
i=1 k=1
Pp Pn i+j Pp
= k=1 k i=1 ( 1) bki det Aij = k=1 k det C 1 (A) ; :::; bk ; :::; C n (A) :
4.
It is su¢ cient to expand the determinant by the column equal to zero.
5.
e := C 1 (A) ; C 1 (A) ; C 3 (A) ; :::; C n (A) be given.
Let A := C 1 (A) ; C 1 (A) ; C 3 (A) ; :::; C n (A) and A
Then det A = det C 1 (A) ; C 1 (A) ; C 3 (A) ; :::; C n (A) = det A.e Interchanging the …rst column with the
second column of the matrix A, e from property 2, we have that det A e = det A e and therefore det A e = 0, and
det A = det A e = 0.
6.
It follows from 3 and 5.
7.
It follows from 3 and 6.
8.
It follows from the fact that the obtained expression is the determinant of a matrix with two equal
columns.
9.
It can be shown by induction and expanding the determinant by the …rs row or column, choosing one
which has all the elements equal to zero excluding at most the …rst element. In other words, in the case of
an upper triangular matrix, we can say what follows.

a11 a12
det = a11 a22 :
0 a22
2 3
a11 a12 a13 ::: a1n 2 3
6 7 a22 a23 ::: a2n
6 0 a22 a23 ::: a2n 7 6 0
6 7 6 a33 a3n 7
7
det 6 0 0 a33 a3n 7 = a11 det 6 7 = a11 a22 a33 :::ann :
6 7 4 .. 5
4 ::: .. 5 .
.
::: 0 ::: ann
0 ::: 0 ::: ann
5.2. RANK OF A MATRIX 69

Theorem 212 For any A; B 2 M (n; n), det(AB) = det A det B:

Proof. Exercise.

De…nition 213 A 2 M (n; n) is called nonsingular if det A 6= 0.

5.2 Rank of a matrix

De…nition 214 Given A 2 M (m; n), a square submatrix of A of order k min fm; ng is a matrix obtained
considering the elements belonging to k rows and k columns of A.

De…nition 215 Given A 2 M (m; n), the rank of A is the greatest order of square nonsingular submatrices
of A.

Remark 216 rankA min fm; ng.

To compute rank A, with A 2 M (m; n), we can proceed as follows.

1. Consider k = min fm; ng, and the set of square submatrices of A of order k. If there exists a nonsingular
matrix among them, then rank A = k. If all the square submatrices of A of order k are singular, go to step
2 below.
2. Consider k 1, and then the set of the square submatrices of A of order k 1. If there exists a
nonsingular matrix among them, then rank A = k 1. If all square submatrices of order k 1are singular,
go to step 3.
3. Consider k 2 ...
and so on.

Remark 217 1. rank In = n.

2. The rank of a matrix with a zero row or column is equal to the rank of that matrix without that row
or column, i.e.,
A A 0
rank = rank A 0 = rank = rank A
0 0 0

That result follows from the fact that the determinant of any square submatrix of A involving that zero
row or columns is zero.
3. From the above results, we also have that

Ir 0
rank =r
0 0

We now describe an easier way to the compute the rank of A, which in fact involves elementary row and
column operations we studied in Chapter 1.

Proposition 218 Given A; A0 2 M (m; n),

h A is equivalent to A0 i , hrank A = rank A0 i

Proof. [)] Since A is equivalent to A0 , it is possible to go from A to A0 through a …nite number of

elementary row or column operations. In each step, in any square submatrix A of A which has been changed
accordingly to those operations, the elementary row or column operations 1, 2 and 3 (i.e., 1. row or column
interchange, 2. row or column scaling and 3. row or column addition) are such that the determinant of
A remains unchanged or changes its sign (Property 2, Theorem 211), it is multiplied by a nonzero constant
(Property 3), remains unchanged (Property 7), respectively.
Therefore, each submatrix A whose determinant is di¤erent from zero remains with determinant di¤erent
from zero and any submatrix A whose determinant is zero remains with zero determinant.
[(]
70 CHAPTER 5. DETERMINANT AND RANK OF A MATRIX

From Corollary 158.2 2 , we have that there exist unique rb and rb0 such that

Irb 0
A is equivalent to
0 0

and
Irb0 0
A0 is equivalent to :
0 0
Moreover, from [)] part of the present proposition, and Remark 217

Irb 0
rankA = rank = rb
0 0

and
Irb0 0
rankA0 = rank = rb0 :
0 0
Then, by assumption, rb = rb0 := r, and A and A0 are equivalent to

Ir 0
0 0

and therefore A is equivalent to A0 :

Example 219 Given 2 3

1 2 3
4 2 3 4 5
3 5 7
we can perform the following elementary rows and column operations, and cancellation of zero row and
columns on the matrix:
2 3 2 3
1 2 3 1 2 3
4 2 3 1 2 3 1 1 3
4 5;4 2 3 4 5; ; ;
2 3 4 2 1 4
3 5 7 0 0 0

1 1 0 1 1 0 0 1 0 1 0
; ; ; :
2 1 1 0 0 1 0 0 1 0 1

Therefore, the rank of the matrix is 2.

5.3 Inverse matrices (continued)

Using the notion of determinant, we can …nd another way of analyzing the problems of i. existence and ii.
computation of the inverse matrix. To do that, we introduce the concept of adjoint matrix.

De…nition 220 Given a matrix An n , we call adjoint matrix of A, and we denote it by Adj A, the matrix
whose elements are the cofactors3 of the corresponding elements of AT .
2 That results says what follows:

For any A 2 M (m; n), there exist invertible matrices P 2 M (m; m) and Q 2 M (n; n) and a unique number r 2
f0; 1; :::; min fm; ngg such that A is equivalent to

Ir 0
P AQ =
0 0

3 From De…nition 207, recall that given A = [aij ] 2 M (m; m), 8i; j;

( 1)i+j det Aij

is called cofacor of aij in A.

5.3. INVERSE MATRICES (CONTINUED) 71

Remark 221 In other words to construct Adj A;

1. construct AT ,
2. consider the cofactors of each element of AT .

Example 222
1 2 1 0 2 2
A= ; AT = ; Adj A =
0 2 2 2 0 1
Observe that
1 2 2 2 2 0
= = (det A) I:
0 2 0 1 0 2

Example 223
2 3 2 3 2 3
1 2 3 1 0 1 4 6 1
A=4 0 1 2 5 ; AT = 4 2 1 2 5 ; Adj A = 4 2 3 2 5
1 2 0 3 2 0 1 0 1

Proposition 224 Given An n, we have

A Adj A = Adj A A = det A I (5.6)

Proof. Making the product A Adj A := B, we have

1. 8i 2 f1; :::; ng, the i th element on the diagonal of B is the expansion of the determinant by the
i th row and therefore is equal to det A.
2. any element not on the diagonal of B is the product of the elements of a row times the corresponding
cofactor a parallel row and it is therefore equal to zero due to Property 8 of the determinants stated in
Theorem 211).

Example 225 2 3
1 2 3
det 4 0 1 2 5= 3
1 2 0
2 32 3 2 3
1 2 3 4 6 1 3 0 0
4 0 1 2 54 2 3 2 5=4 0 3 0 5= 3 I
1 2 0 1 0 1 0 0 3

Proposition 226 Given an n n matrix A, the following statements are equivalent:

1. det A 6= 0, i.e., A is nonsingular;

1
2. A exists, i.e., A is invertible;

3. rank A = n;

4. row rankA = n;

5. the column vectors of the matrix A are linearly independent;

6. col rankA = n;

7. the row vectors of the matrix A are linearly independent;

Proof. 1 ) 2
From (5:6) and from the fact that det A 6= 0, we have

Adj A Adj A
A = A=I
det A det A
and therefore
1 Adj A
A = (5.7)
det A
72 CHAPTER 5. DETERMINANT AND RANK OF A MATRIX

1(2
AA 1 = I ) det AA 1
= det I ) det A det A 1
= 1 ) det A 6= 0 (and det A 1
6= 0).

1,3
It follows from the de…nition of rank and the fact that A is n n matrix.
2)5
From (4:8), it su¢ ces to show that hAx = 0 ) x = 0i. Since A 1 exists, Ax = 0 ) A 1 Ax = A 1 0 )
x = 0.
2(5
From Proposition 188.2, the n linearly independent column vectors C 1 (A) ; :::; Ci (A) ; :::; C n (A) are
a basis of Rn . Therefore, each vector in Rn is equal to a linear combination of those vectors. Then 8k 2
f1; :::; ng 9bk 2 Rn such that the k th vector ek in the canonical basis is equal to C 1 (A) ; :::; Ci (A) ; :::; C n (A)
bk = Abk , i.e.,
e1 ::: ek ::: en = Ab1 ::: Abk ::: Abn

or, from (3:4) in Remark 70, de…ned

B := b1 ::: bk ::: bn

I = AB

i.e., A 1 exists (and it is equal to B).

The remaining equivalences follow from Corollary 201.

1
Remark 227 From the proof of the previous Proposition, we also have that, if det A 6= 0, then det A =
1
(det A) .

Remark 228 The previous theorem gives a way to compute the inverse matrix as explained in (5:7).

Example 229 1.
2 3 1 2 3
1 2 3 4 6 1
4 0 14
1 2 5 = 2 3 2 5
3
1 2 0 1 0 1

2.
2 3 1 2 3
0 1 1 1 0 1
4 1 1 0 5 =4 1 1 1 5
1 1 1 0 1 1

3.
2 3 1 2 3
0 1 2 0 1 2
4 3 4 5 5 does not exist because det 4 3 4 5 5=0
6 7 8 6 7 8
4.
2 3 1 2 3 1
3
0 1 0 1 4 2
4 2 0 2 5 =4 1 0 0 5
1 1
1 2 3 1 4 2

5.
2 3 1 2 1
3
a 0 0 a 0 0
4 0 b 0 5 =4 0 1
0 5
b
1
0 0 c 0 0 c

if a; b; c 6= 0.
5.4. SPAN OF A MATRIX, LINEARLY INDEPENDENT ROWS AND COLUMNS, RANK 73

5.4 Span of a matrix, linearly independent rows and columns,

rank
Proposition 230 Given A 2 M (m; n), then

row rank A = rank A:

Proof. First proof.

The following result which is the content of Corollary 158.5 plays a crucial role in the proof:
For any A 2 M (m; n), there exist invertible matrices P 2 M (m; m) and Q 2 M (n; n) and a unique
number r 2 f0; 1; :::; min fm; ngg such that

Ir 0
A is equivalent to P AQ = .
0 0

From the above result, Proposition 218 and Remark 217 , we have that

Ir 0
rank A = rank = r: (5.8)
0 0

From Propositions 105 and 152, we have

row span A = row span P A;

from Propositions 119 and 152, we have

Ir 0
col span P A = col span P AQ = col span :
0 0

From Corollary 201,

dim row span P A = dim col span P A:
Therefore
Ir 0
dim row span A = dim col span = r; (5.9)
0 0
where the last equality follows simply because

Ir 0 Ir
col span = col span ;
0 0 0

Ir
and the r column vectors of the matrix are linearly independent and therefore, from Proposition ?? ,
0
Ir
they are a basis of span .
0
From (5:8) and (5:9), the desired result follows.

Second proof.
We are going to show that row rank A = rank A.
Recall that
* r 2 N such that +
row rank A := i . r row vectors of A are linearly independent,
ii. if m > r, any …nite sequence of rows of A of cardinality > r is linearly dependent.

We ant to show that

1. if row rank A = r, then rank A = r, and
2. if rank A = r, then row rank A = r.
1.
Consider the r linearly independent row vectors of A. Since r is the maximal number of linearly indepen-
dent row vectors, from Lemma 196, each of the remaining (m r) row vectors is a linear combination of the
74 CHAPTER 5. DETERMINANT AND RANK OF A MATRIX

r linearly independent ones. Then, up to reordering of the rows of A, which do not change either row rank
A or rank A, there exist matrices A1 2 M (r; n) and A2 2 M (m r; n) such that

A1
rank A = rank = rank A1
A2

where the last equality follows from Proposition 218. Then r is the maximum number of linearly in-
dependent row vectors of A1 and therefore, from Proposition 199, the maximum number of linearly inde-
pendent column vectors of A1 : Then, again from Lemma 196, we have that there exist A11 2 M (r; r) and
A12 2 M (r; n r) such that

rank A1 = rank A11 A12 = rank A11

Then the square r r matrix A11 contains r linearly independent vectors. Then from Proposition 226,
the result follows.
2.
By Assumption, up to reordering of rows, which do not change either row rank A or rank A,
2 3
s r n r s
6 7
A=6 4 r
7
5
A11 A12 A13
m r A21 A22 A23

with
rank A11 = r:
Then from Proposition 226, row, and column, vectors of A12 are linearly independent. Then from
Corollary ??, the r row vectors of
A11 A12 A13
are linearly independent
Now suppose that the maximum number of linearly independent row vectors of A are r0 > r (and the
other m r0 row vectors of A are linear combinations of them). Then from part 1 of the present proof, rank
A = r0 > r, contradicting the assumption.

Remark 231 From Corollary 201 and the above Proposition, we have for any matrix Am n, the following
numbers are equal.
1. rank A :=greatest order of square nonsingular submatrices of A,
2. row rank A := dim row span A,
3. max number of linear independent rows of A,
4. col rank A := dim col span A,
5. max number of linear independent columns of A.

Corollary 232 For any matrix A 2 M(m; n), there exists r 2 f0; :::; min fm; ngg such that A is equivalent

Ir 0
;
0 0

where r = rank A.

5.5 Exercises
Problem sets: 9,10
From Lipschutz (1991), starting from page 115:
4.13, 4.14;
starting from page 258:
7.1 ! 7.10, 7.14 ! 7.16, 7.44, 7.48.
Chapter 6

Linear functions

6.1 De…nition
De…nition 233 Given the vector spaces V and U over the same …eld F , a function l : V ! U is linear if

1. 8v; w 2 V , l (v + w) = l (v) + l (w), and

2. 8 2 F; 8v 2 V , l ( v) = l (v).

Call L (V; U ) the set of all such functions. Any time we write L (V; U ), we implicitly assume that V and
U are vector spaces on the same …eld F .

In other words, l is linear if it “preserves” the two basic operations of vector spaces.

Remark 234 1. l 2 L (V; U ) i¤ 8v; w 2 V and 8 ; 2 F; l ( v + w) = l (v) + l (w);

2. If l 2 L (V; U ), then l (0) = 0: for arbitrary x 2 V , l (0) = l (0x) = 0l (x) = 0.

Example 235 Let V and U be vector spaces. The following functions are linear.
1. (identity function)
l1 : V ! V; l1 (v) = v:
2. (null function)
l2 : V ! U; l2 (v) = 0:
3.
8a 2 F; la : V ! V; la (v) = av:
4. (projection function)
8n 2 N; 8k 2 N;
n+k n
projn+k;n : Rn+k ! Rn ; projn+k;n : (xi )i=1 7! (xi )i=1 ;
5. (immersion function)
8n 2 N; 8k 2 N;
n n
in;n+k : Rn ! Rn+k ; in;n+k : (xi )i=1 7! ((xi )i=1 ; 0) with 0 2 Rk :

Example 236 Taken A 2 M (m; n), then

l : R n ! Rm ; l (x) = Ax

is a linear function, as shown in part 3 in Remark 76.

Remark 237 Let V and U be vector spaces. If l 2 L (V; U ), then

graph l := f(v; u) 2 V U : u = l (v)g

is a vector subspace of V U.

75
76 CHAPTER 6. LINEAR FUNCTIONS

Example 238 Let V be the vector space of polynomials in the variable t. The following functions are linear.
1. The derivative de…nes a function D : V ! V as

D : p 7! p0

where p0 is the derivative function of p.

2. The de…nite integral from 0 to 1 de…nes a function i : V ! R as
Z 1
i : p 7! p (t) dt:
0

1
Proposition 239 If l 2 L (V; U ) is invertible, then its inverse l is linear.

Proof. Take arbitrary u; u0 2 U and ; 2 F . Then, since l is onto, there exist v; v 0 2 V such that

l (v) = u and l (v 0 ) = u0

and by de…nition of inverse function

1 1
l (u) = v and l (u0 ) = v 0 :

Then
u + u0 = l (v) + l (v 0 ) = l ( v + v 0 )
where last equality comes from the linearity of l. Then again by de…nition of inverse,
1
l ( u + u0 ) = v + v 0 = l 1
(u) + l 1
(u0 ) :

6.2 Kernel and Image of a linear function

De…nition 240 Assume that l 2 L (V; U ). The kernel of l, denoted by ker l is the set
1
fv 2 V : l (v) = 0g = l (0) :

The Image of l, denoted by Im l is the set

fu 2 U : 9v 2 V such that f (v) = ug = l (V ) :

Proposition 241 Given l 2 L (V; U ), then

1. ker l is a vector subspace of V; and
2. Im l is a vector subspace of U .

Proof. 1.
Since l (0) = 0, then 0 2 ker l.
Take v 1 ; v 2 2 ker l and ; 2 F . Then

l v1 + v2 = l v1 + l v2 = 0

i.e., v 1 + v 2 2 ker l.
2.
Since 0 2 V and l (0) = 0, then 0 2 Im l.
Take w1 ; w2 2 Im l and ; 2 F . Then for i 2 f1; 2g , 9v i 2 V such that l v i = wi . Moreover,

w1 + w2 = l (vi ) + l v 2 = l v1 + v2

i.e., w1 + w2 2 Im l.

Proposition 242 If span v 1 ; :::; v n = V and l 2 L (V; U ), then span l v 1 ; :::; l (v n ) = Im l.

6.2. KERNEL AND IMAGE OF A LINEAR FUNCTION 77

n
Proof.
Pn Taken u 2 Im l, there exists v 2 V such that l (v) = u. Moreover, 9 ( i )i=1 2 Rn such that
v = i=1 i v i . Then
n
! n
X X
i i
u = l (v) = l iv = il v ;
i=1 i=1

as desired.

Remark 243 From the previous proposition, we have that if v 1 ; :::; v n is a basis of V , then

n dim span l v 1 ; :::; l (v n ) = dim Im l:

Example 244 Let V the vector space of polynomials and D3 : V ! V , p 7! p000 , i.e., the third derivative of
p. Then
ker D3 = set of polynomials of degree 2;
since D3 at2 + bt + c = 0 and D3 (tn ) 6= 0 for n > 2. Moreover,

Im D3 = V;

since every polynomial is the third derivative of some polynomial.

Proposition 245 (Dimension Theorem)If V is a …nite dimensional vector space and l 2 L (V; U ), then

dim V = dim ker l + dim Im l

Proof. (Idea of the proof.

1. Using a basis of ker l (with n1 elements) and a basis of Im l (with n2 elements), we construct n1 + n2
vectors which generate V .
2. We show those vectors are linearly independent (by contradiction), and therefore a basis of V , and
therefore dim V = n1 + n2 .)
Since ker l V and from Remark 243, ker l and Im l have …nite dimension. Therefore, we can de…ne
n1 = dim ker l and n2 = dim Im l.
Take an arbitrary v 2 V . Let
w1 ; ::; wn1 be a basis of ker l (6.1)
and
u1 ; ::; un2 be a basis of Im l (6.2)
Then,
8i 2 f1; ::; n2 g ; 9v i 2 V such that ui = l v i (6.3)
From (6:2),
n2
X
n
2
9c = (ci )i=1 such that l (v) = ci ui (6.4)
i=1

Then, from (6:4) and (6:3) ;we get

n2
X n2
X
l (v) = ci ui = ci l v i
i=1 i=1

and from linearity of l

n2 n2
! n2
!
X X X
i i i
0 = l (v) ci l v = l (v) l ci v =l v ci v
i=1 i=1 i=1

i.e.,
n2
X
v ci v i 2 ker l (6.5)
i=1
78 CHAPTER 6. LINEAR FUNCTIONS

From (6:1),
n2
X n1
X
n
9 (dj )j=1 such that v ci v i = dj w j
i=1 j=1

Summarizing, we have
n2
X n1
X
n n
2
8v 2 V; 9 (ci )i=1 1
and (dj )j=1 such that v = ci v i + dj w j
i=1 j=1

Therefore, we found n1 + n2 vectors which generate V ; if we show that they are linearly independent .i.,
then, by de…nition, they are a basis and therefore n = n1 + n2 as desired.
We want to show that
n2
X n1
X
i j 2 n n1
iv + jw = 0 ) ( i )i=1 , j j=1 =0 (6.6)
i=1 j=1

Then 0 1
Xn2 n1
X n2
X n1
X
0=l @ iv
i
+ jw
jA
= il v i
+ jl wj
i=1 j=1 i=1 j=1

From (6:1), i.e., w1 ; ::; wn1 is a basis of ker l, and from (6:3), we get
n2
X
i ui =0
i=1

From (6:2), i.e., u1 ; ::; un2 is a basis of Im l,

n
2
( i )i=1 =0 (6.7)

But from the assumption in (6:6) and (6:7) we have that

n1
X
j
jw =0
j=1

and since w1 ; ::; wn1 is a basis of ker l, we get also that

n1
j j=1 = 0;

as desired.

Example 246 Let V and U be vector spaces, with dim V = n.

In 1. and 2. below, we verify the statement of the Dimension Theorem: in 3. and 4., we use that
statement.
1. Identity function idV :
dim Im idV = n
dim ker l = 0:
2. Null function 0 2 L (V; U )
dim Im 0 = 0
dim ker 0 = n:
3. l 2 L R2 ; R ;
l ((x1 ; x2 )) = x1 :
Since ker l = (x1 ; x2 ) 2 R : x1 = 0 , f(0; 1)g is a basis1 of ker l and
2

dim ker l = 1;
dim Im l = 2 1 = 1:
1 In Remark 315 we will present an algorithm to compute a basis of ker l.
6.3. NONSINGULAR FUNCTIONS AND ISOMORPHISMS 79

4. l 2 L R3 ; R2 ; 2 3
x1
1 2 3 4 x2 5 :
l ((x1 ; x2 ; x3 )) =
0 1 0
x3
De…ned
1 2 3
A= ;
0 1 0
since
Im l = y 2 R2 : 9x 2 R3 such that Ax = y = span col A = rank A;
and since the …rst two column vectors of A are linearly independent, we have that

dim Im l = 2
dim ker l = 3 2 = 1:

Exercise 247 Let l 2 L(R3 ; R3 ) such that

0 1 0 1
x x cos y sin
l @ y A 7! @ x sin + y cos A
z z

be given. Find ker l and Im l (Answer: ker l = f0g; Im l = R3 ).

6.3 Nonsingular functions and isomorphisms

De…nition 248 l 2 L (V; U ) is singular if 9v 2 V n f0g such that l (v) = 0.

Remark 249 Thus l 2 L (V; U ) is nonsingular if 8v 2 V n f0g ; l (v) 6= 0 i.e., ker l = f0g. Brie‡y,

l 2 L (V; U ) is nonsingular , ker l = f0g.

Remark 250 In Remark 289, we will discuss the relationship between singular matrices and singular linear
functions.

Example 251 1. Let l 2 L(R3 ; R3 ) be the projection mapping into the xy plane, i.e.,

l : 0 R 3 1 ! 0 R3 1
x x
@ y A 7! @ y A
z 0

Then l is singular, since for any a 2 R, l(0; 0; a) = (0; 0; 0).

2. l 2 L(R3 ; R3 ) de…ned in Example 247 is nonsingular, since ker l = f0g.

Proposition 252 If l 2 L (V; U ) is nonsingular, then the images of linearly independent vectors are linearly
independent.

Proof. Suppose v 1 ; :::; v n are linearly independent. We want to show that l v 1 ; :::; l (v n ) are linearly
independent as well. Suppose
Xn
i
i l v = 0:
i=1

Then !
n
X
i
l i v = 0:
i=1
Pn
and therefore v i 2 ker l = f0g, where the last equality comes from the fact that l is non-
Pn i=1 i i n
singular. Then i=1 i v = 0 and, since v 1 ; :::; v n are linearly independent, ( i )i=1 = 0, as desired.
80 CHAPTER 6. LINEAR FUNCTIONS

De…nition 253 Let two vector spaces V and U be given. U is isomorphic to V if there exists a function
l 2 L (V; U ) which is one-to-one and onto. l is called an isomorphism from V to U .

Remark 254 By de…nition of isomorphism , if l is an isomorphism, the l is invertible and therefore, from
Proposition 239, l 1 is linear.

Remark 255 “To be isomorphic” is an equivalence relation.

Proposition 256 Any n-dimensional vector space V on a …eld F is isomorphic to F n .

Proof. Since V and F n are vector spaces, we are left with showing that there exists an isomorphism
between them. Let V = v 1 ; :::; v n be a basis of V . Recall that we de…ne

cr : V ! F n ; v 7! [v]V ;

where cr stands for “coordinates”.

1. cr is linear. Given v; w 2 V , suppose
n
X n
X
v= ai v i and w= bi v i
i=1 i=1

i.e.,
n n
[v]V = [ai ]i=1 and [w]V = [bi ]i=1 .
8 ; 2 F and 8v 1 ; v 2 2 V ,
n
X n
X n
X
i i
v+ w= ai v + bi v = ( ai + bi ) v i
i=1 i=1 i=1

i.e.,
n n
[ v + w]V = [ai ]i=1 + [bi ]i=1i = [v]V + [w]V :
n Pn n
2. cr is onto. 8 (a)i=1 2 Rn , cr i=1 ai v
i
= (a)i=1 . Pn
Pn3. cr is one-to-one. cr (v) = cr (w) implies that v = w, simply because v = i=1 cri (v) ui and w =
i
i=1 cri (w) u .

Remark 257 If two spaces S and C are isomorphic, then we can use the isomorphism between the two
spaces to infer properties about one the two, knowing properties of the other one. Indeed, sometimes it is
possible to show a “complicated space” C is isomorphic to a “simple space” S. Then, we can …rst show
properties about S, and then, using the isomorphism, infer properties of the complicated space C.

Proposition 258 Let V and U be …nite dimensional vectors spaces on the same …eld F such that S =
v 1 ; :::; v n is a basis of V and u1 ; :::; un are arbitrary vectors in U . Then there exists a unique linear
function l : V ! U such that 8i 2 f1; :::; ng, l v i = ui .

Proof. The proof goes the following three steps.

1. De…ne l;
2. Show that l is linear;
3. Show that l is unique.
1. Using the de…nition of coordinates, 8v 2 V , de…ne
n
X i
l : V ! U; v 7! [v]s ui ;
i=1

i
where [v]S denotes the i th component of the vector [v]S . Recall that ejn 2 Rn is the j th element in the
n
n
canonical basis of R and de…ned ejn := ejn;i , we have
i=1
8
< 1 if i=j
ejn;i =
:
0 if i 6= j:
6.3. NONSINGULAR FUNCTIONS AND ISOMORPHISMS 81

Then 8j 2 f1; :::; ng, v j S

= ejn and
n
X n
X
i
l vj = vj s
ui = ejn;i ui = uj :
i=1 i=1

2. Let v; w 2 V and ; 2 F . Then

n
X n
X n
X n
X
i i i i i
l ( v + w) = [ v + w]S ui = [v]S + [w]S ui = [v]S ui + [w]S ui ;
i=1 i=1 i=1 i=1

where the before the last equality follows from the linearity of [:] - see the proof of Proposition 256
3. Suppose g 2 L (V; U ) and 8i 2 f1; :::; ng, g v i = ui . Then 8v 2 V ,
n
! n n
X i
X i
X i
g (v) = g [v]S v i = [v]S g v i = [v]S ui = l (v)
i=1 i=1 i=1

where the last equality follows from the de…nition of l.

Remark 259 Observe that if V and U are …nite(nonzero) dimensional vector spaces, there is a multitude
of functions from V to U . The above Proposition says that linear functions are completely determined by
what “they do to the elements of a basis”of V .

Example 260 We want to …nd the unique linear mapping l : R2 ! R2 such that

l(1; 2) = (2; 3) and l(0; 1) = (1; 4):

Observe that B := f(1; 2); (0; 1)g is a basis of R2 . For any (a; b) 2 R2 , there exist x; y 2 R such that

(a; b) = x(1; 2) + y(0; 1) = (x; 2x + y);

i.e., a = x and b = 2x + y and therefore x = a and y = 2a + b. Then,

l linear
l(a; b) = l(x(1; 2) + y(0; 1)) = xl(1; 2) + yl(0; 1) = a(2; 3) + ( 2a + b)(1; 4) = (b; 5a + 4b);

i.e.,
l : R2 ! R2 ; (a; b) 7! (b; 5a + 4b):

Proposition 261 Assume that l 2 L (V; U ) :Then,

l is one-to-one , l is nonsingular

Proof. [)]
Take v 2 ker l. Then
l (v) = 0 = l (0)
where last equality follows from Remark 234. Since l is one-to-one, v = 0.
[(] If l (v) = l (w), then l (v w) = 0 and, since l is nonsingular, v w = 0.

Proposition 262 Assume that V and U are …nite dimensional vector spaces and l 2 L (V; U ) :Then,
1. l is one-to-one ) dim V dim U ;
2. l is onto ) dim V dim U ;
3. l is invertible ) dim V = dim U .

Proof. The main ingredient in the proof is Proposition 245 , i.e., the Dimension Theorem.
1. Since l is one-to-one, from the previous Proposition, dim ker l = 0. Then, from Proposition 245 (the
Dimension Theorem), dim V = dim Im l. Since Im l is a subspace of U , then dim Im l dim U .
2. Since l is onto i¤ Im l = U , from Proposition 245 (the Dimension Theorem), we get

dim V = dim ker l + dim U dim U:

3. l is invertible i¤ l is one-to-one and onto.

82 CHAPTER 6. LINEAR FUNCTIONS

Proposition 263 Let V and U be …nite dimensional vector space on the same …eld F . Then,

U and V are isomorphic , dim U = dim V:

Proof. [)]
It follows from the de…nition of isomorphism and part 3 in the previous Proposition.
[(]
Assume that V and U are vector spaces such that dim V = dim U = n. Then, from Proposition 256, V
and U are isomorphic to F n and from Remark 255, the result follows.

Proposition 264 Suppose V and U are vector spaces such that dim V = dim U = n and l 2 L (V; U ). Then
the following statements are equivalent.
1. l is nonsingular, i.e., ker l = f0g ;
2. l is one-to-one,
3. l is onto,
4. l is an isomorphism.

Proof. [1 , 2] :
It is the content of Proposition 261.
[1 ) 3]
Since l is nonsingular, then ker l = f0g and dim ker l = 0. Then, from Proposition 245 (the Dimension
Theorem), i.e., dim V = dim ker l + dim Im l, and the fact dim V = dim U , we get dim U = dim Im l. Since
Im l U and U is …nite dimensional, from Proposition 191, Im l = U , i.e., l is onto, as desired.
[3 ) 1]
Since l is onto, dim Im l = dim V and from Proposition 245 (the Dimension Theorem), dim V = dim ker t+
dim V; and therefore dim ker l = 0, i.e., l is nonsingular.
[1 ) 4]
It follows from the de…nition of isomorphism and the facts that [1 , 2] and [1 ) 3].
[4 ) 1]
It follows from the de…nition of isomorphism and the facts that [2 , 1].

De…nition 265 A vector space endomorphism is a linear function from a vector space V into itself.

6.4 Exercises
From Lipschutz (1991), starting from page 325:
9.3, 9.6, 9.9 ! 9.11, 9.16 ! 9.21, 9.26, 9.27, 9.31 ! 9.35, 9.42 ! 9.44; observe that Lipschutz denotes
L (V; U ) by Hom (V; U ).
Chapter 7

Linear functions and matrices

In Remark 65 we have seen that the set of m n matrices with the standard sum and scalar multiplication
is a vector space, called M (m; n). We are going to show that:
1. the set L (V; U ) with naturally de…ned sum and scalar multiplication is a vector space, called L (V; U );
2. If dim V = n and dim U = m, then L (V; U ) and M (m; n) are isomorphic.

7.1 From a linear function to the associated matrix

De…nition 266 Suppose V and U are vector spaces over a …eld F and l1 ; l2 2 L (V; U ) and 2 F.

l1 + l2 : V ! U; v 7! l1 (v) + l2 (v)

l1 : V ! U; v 7! l1 (v) :

Proposition 267 L (V; U ) with the above de…ned operations is a vector space on F , denoted by L (V; U ).

Proof. Exercise.1

Proposition 268 Compositions of linear functions are linear.

Proof. Suppose V , U , W are vector spaces over a …eld F; l1 2 L (V; U ) and l2 2 L (U; W ). We want to
show that l := l2 l1 2 L (V; W ). Indeed, for any 1 ; 2 2 F and for any v 1 ; v 2 2 V , we have that
1 2 1 2 1 2
l 1v + 2v := (l2 l1 ) 1v + 2v = l 2 l1 1v + 2v =

= l2 1 l1 v1 + 2 l1 v2 = 1 l2 l1 v 1 + 2 l2 l1 v 2 = 1l v1 + 2l v2 ;
as desired.

De…nition 269 Suppose l 2 L (V; U ) , V = v 1 ; :::; v n is a basis of V , U = u1 ; :::; um is a basis of U .

Then,
U
[l]V := l v 1 U ::: l v j U ::: [l (v n )]U 2 M (m; n) ; (7.1)
where for any j 2 f1; :::; ng, l v j U is a column vector, is called the matrix representation of l relative
U
to the basis V and U. In words, [l]V is the matrix whose columns are the coordinates relative to the basis of
the codomain of l of the images of each vector in the basis of the domain of l.

Remark 270 Observe that by de…nition of coordinates, there is a unique matrix representation of a linear
function relative to the basis V and U.

De…nition 271 Suppose l 2 L (V; U ) , V: = v 1 ; :::; v n is a basis of V , U := u1 ; :::; um is a basis of U .

U
'U
V : L (V; U ) ! M (m; n) ; l 7! [l]V de…ned in (7:1) :

If no confusion may arise, we will denote 'U

V simply by '.
1 For a detailed proof see Lipschutz (1989), page 270.

83
84 CHAPTER 7. LINEAR FUNCTIONS AND MATRICES

V
Example 272 Given idV 2 L (V; V ) and a basis V = v 1 ; :::; v n of V , then [idV ]V = In .

The proposition below shows that multiplying the coordinate vector of v relative to the basis V by the
U
matrix [l]V , we get the coordinate vector of l (v) relative to the basis U.

Proposition 273 8v 2 V ,
U
[l]V [v]V = [l (v)]U (7.2)

Proof. Assume v 2 V . First of all observe that

2 1 3
[v]V
6 ::: 7 Xn
6 7
U
[l]V [v]V = l v 1
::: l v j
::: [l (v )]U n 6 j
[v]V 7= j
[v]V l vj :
U U 6 7 U
4 ::: 5 j=1
n
[v]V

Moreover, from the linearity of the function crU := [:]U , and using the fact that the composition of linear
functions is a linear function, we get:
0 1
Xn n
X n
X
j j j
[l (v)]U = crU (l (v)) = (crU l) @ [v] V vj A = [v]V (crU l) v j = [v]V l vj U
:
j=1 j=1 j=1

Example 274 Let’s verify equality (7:2) in the case in which

a.
x1 + x2
l : R2 ! R2 ; (x1 ; x2 ) 7!
x1 x2

b. the basis V of the domain of l is

1 0
; ;
0 1

c. the basis U of the codomain of l is

1 2
; ;
1 1

d.
3
v= :
4

The main needed computations are presented below.

U 1 0 1 1 1 3
[l]V := l ; l = ; = ;
0 U
1 U
1 U
1 U
0 2

3
[v]V = ;
4

7 9
[l (v)]U = = ;
1 U
8

U 1 3 3 9
[l]V [v]V = = :
0 2 4 8
7.2. FROM A MATRIX TO THE ASSOCIATED LINEAR FUNCTION 85

7.2 From a matrix to the associated linear function

Given A 2 M (m; n) ; recall that 8i 2 f1; ::; mg, Ri (A) denotes the i th row vector of A, i.e.,
2 1 3
R (A)
6 ::: 7
6 i 7
A=6 6 R (A) 7
7
4 ::: 5
Rm (A) m n

De…nition 275 Consider vector spaces V and U with bases V = v 1 ; :::; v n and U = u1 ; :::; um , respec-
tively. Given A 2 M (m; n) ; de…ne
m
X
U
lA;V : V ! U; v 7! Ri (A) [v]V ui :
i=1

1 2 1 3
Example 276 Take U = V = R2 , V = E 2 , U = ; and A = . Then,
1 1 0 2

U
P2 x1 1 x1 2
lA;V (x1 ; x2 ) := i=1 Ri (A) [v]E2 ui = 1 3 + 0 2 =
x2 1 x2 1

1 2 x1 3x2 + 4x2 x1 + x2
= (x1 3x2 ) + 2x2 = =
1 1 x1 3x2 + 2x2 x1 x2
U U
Proposition 277 lA;V de…ned above is linear, i.e., lA;V 2 L (V; U ).

Proof. 8 ; 2 R and 8v 1 ; v 2 2 V ,
U
Pm Pm
lA;V v 1 + v 2 = i=1 Ri (A) v1 + v2 V
ui = i=1 Ri (A) v1 V
+ v2 V
ui =
Pm Pm
= i=1 Ri (A) v1 V
ui + i=1 Ri (A) v2 V
U
ui = lA;V U
v 1 + lA;V v2 :

where the second equality follows from the proof of Proposition 256.

De…nition 278 Given the vector spaces V and U with bases V = v 1 ; :::; v n and U = u1 ; :::; um , respec-
tively, de…ne
U U
V : M (m; n) ! L (V; U ) : A 7! lA;V :
U
If no confusion may arise, we will denote V simply by .

Proposition 279 de…ned above is linear.

Proof. We want to show that 8 ; 2 R and 8A; B 2 M (m; n),

( A + B) = (A) + (B)

i.e.,
lVA+ B;U
U
= lA;V V
+ lB;U
i.e., 8v 2 V ,
lVA+ B;U
U
(v) = lA;V V
(v) + lB;U (v) :
Now, Pm
lVA+ B;U (v) = i=1 Ri (A) + Ri (B) [v]V ui =
Pm Pm
= i=1 Ri (A) [v]V ui + i=1
U
Ri (B) [v]V ui = lA;V V
(v) + lB;U (v) ;
where the …rst equality come from De…nition 275.
86 CHAPTER 7. LINEAR FUNCTIONS AND MATRICES

7.3 M (m; n) and L (V; U ) are isomorphic

Proposition 280 Given the vector space V and U with dimension n and m, respectively,
M (m; n) and L (V; U ) are isomorphic,
and
dim L (V; U ) = mn:
Proof. Linearity of the two spaces was proved above. We want now to check that presented in
De…nition 278 is an isomorphism, i.e., is linear, one-to-one and onto. In fact, thanks to Proposition 279,
it is enough to show that is invertible.
First proof.
1. is one-to-one: see Theorem 2, page 105 in Lang (1971);
2. is onto: see bottom of page 107 in Lang (1970).
Second proof.
1. ' = idL(V;U ) .
Given l 2 L (V; U ), we want to show that 8v 2 V;
l (v) = (( ') (l)) (v)
i.e., from Proposition 256,
[l (v)]U = [(( ') (l)) (v)]U :
First of all, observe that from (7:2), we have
U
[l (v)]U = [l]V [v]V :
Moreover,
(1)
h i (2)
hP i (3)
U n U
[(( ') (l)) (v)]U = [l]V (v) = i=1 i th row of [l]V [v]V ui =
U U
h in (4)
U U
= i th row of [l]V [v]V = [l]V [v]V
i=1

where (1) comes from the de…nition of ', (2) from the de…nition of , (3) from the de…nition of [:]U , (4)
from the de…nition of product between matrices.
2. ' = idM(m;n) .
Given A 2 M (m; n), we want to show that (' ) (A) = A. By de…nition of ,
m
X
U U
(A) = lA;V such that 8v 2 V , lA;V (v) = Ri (A) [v]V ui : (7.3)
i=1

By de…nition of ',
U U
' ( (A)) = lA;V V
.
U U
Therefore, we want to show that lA;V V
= A. Observe that from 7.3,
Pm Pm
U
lA;V (v 1 ) = i=1 Ri (A) [v 1 ]V ui = i=1 [ai1 ;:::;aij ;:::;ain ] [1;:::;0;:::;0] ui = a11 u1 +:::+ai1 ui +::::+am1 um
::::
Pm Pm
U
lA;V (v j ) = i=1 Ri (A) [v j ]V ui = i=1 [ai1 ;:::;aij ;:::;ain ] [0;:::;1;:::;0] ui = a1j u1 +:::+aij ui +:::+amj um
:::
Pm Pm
U
lA;V (v n ) = i=1 Ri (A) [v n ]V ui = i=1 [ai1 ;:::;aij ;:::;ain ] [0;:::;0;:::;1] ui = a1n u1 +:::+ain ui +:::+amn um

(From the above, it is clear why in de…nition 269 we take the transpose.) Therefore,
2 3
a11 ::: a1j ::: a1n
6 ::: 7
U 6 7
U 6
lA;V V = 6 ai1 aij ain 77 = A;
4 ::: 5
am1 ::: amj ::: amn
as desired.
The fact that dim L (V; U ) follows from Proposition 263.
7.3. M (M; N ) AND L (V; U ) ARE ISOMORPHIC 87

Proposition 281 Let the following objects be given.

1. Vector spaces V with basis V = v 1 ; :::; v j ; :::; v n , U with basis U = u1 ; :::; ui ; :::; um and W with
basis W = w1 ; :::; wk ; :::; wp ;
2. l1 2 L (V; U ) and l2 2 L (U; W ).
Then
W W U
[l2 l1 ]V = [l2 ]U [l1 ]V ;
or
'W W U
V (l2 l1 ) = 'U (l2 ) 'V (l1 ) :

Proof. By de…nition2
U
[l1 ]V = l1 v 1 U
::: l1 v j U
::: [l1 (v n )]U m n
:=
2 3 2 3
l11 v 1 ::: l11 v j ::: l11 (v n ) l111 ::: l11j ::: l11n
6 ::: 7 6 ::: 7
6 7 6 7
:= 6
6 l1i v 1 l1i v j l1i (v n ) 7
7 := 6
6 l1i1 l1ij l1in 7
7 :=
4 ::: 5 4 ::: 5
l1m v 1 ::: l1m v j ::: l1m (v n ) l1m1 ::: l1mj ::: l1mn
h i
:= l1ij := A 2 M (m; n) ;
i2f1;:::;mg;j2f1;:::;ng

Pm ij
and therefore 8j 2 f1; :::; ng ; l1 v j = i=1 l1 ui .
Similarly,
W
[l2 ]U = l2 u1 W
::: l2 u i W
::: [l2 (um )]W p m
:=
2 3 2 3
l21 u1 ::: l21 ui ::: l21 (um ) l211 ::: l21i ::: l21m
6 ::: 7 6 ::: 7
6 7 6 7
:= 6
6 l2k u1 l2k ui l2k (um ) 7
7 := 6
6 l2k1 l2ki l2km 7
7 :=
4 ::: 5 4 ::: 5
l2p u1 ::: l2p ui ::: l2p (um ) l2p1 ::: l2pi ::: l2pm

:= l2ki k2f1;:::;pg;i2f1;:::;mg
:= B 2 M (p; m) ;
Pp
and therefore 8i 2 f1; :::; mg ; l2 ui = ki
k=1 l2 wk .
Moreover, de…ned l := (l2 l1 ), we get
W
[l2 l1 ]V = l v1 W
::: l vj W
::: [l (v n )]W p n
:=
2 3 2 3
l1 v 1 ::: l1 v j ::: l1 (v n ) l11 ::: l1j ::: l1n
6 ::: 7 6 ::: 7
6 7 6 7
:= 6
6 lk v 1 lk v j lk (v n ) 7
7 := 6
6 lk1 lkj lkn 7
7 :=
4 ::: 5 4 ::: 5
lp v 1 ::: lp v j ::: lp (v n ) lp1 ::: lpj ::: lpn

:= lkj k2f1;:::;pg;j2f1;:::;ng
:= C 2 M (p; n) ;
Pp
and therefore 8j 2 f1; :::; ng ; l v j = k=1 lkj wk .
Now, 8j 2 f1; :::; ng
Pm ij i
l v j = (l2 l1 ) v j = l2 l1 v j = l2 i=1 l1 u =
Pm ij i
Pm ij Pp ki
Pp Pm ki ij
i=1 l1 l2 u = i=1 l1 k=1 l2 w = k=1 i=1 l2 l1 wk :
k

2 l1 v 1 ; ::; [l1 (v n )]u :are column vectors.

u
88 CHAPTER 7. LINEAR FUNCTIONS AND MATRICES

The above says that 8j 2 f1; :::; ng, the j th column of C is

2 Pm 3
1i
i=1 l2 l1ij
6 7
6 Pm 7
6 ki
l1ij 7
6 i=1 l2 7
4 5
Pm pi ij
i=1 l2 l1

On the other hand, the j th column of B A is

2 2 3 3
l11j
6 6 7 7
6 6 ij 7 7
6 l211 ::: l21i ::: l21m 6 l 7 7
6 6 1 7 7
6 4 5 7
6 7
6 l1mj 7
6 7
6 ::: 7
2 3 6
6 2 3 7 2 P
7 m 3
[1st row of B] [j th column of A] 6 l11j 7 1i
i=1 l2 l1ij
6 7 6 6 7 7 6 7
6 ::: 7 6
6 6 ij 7 7 6 P
7 6 7
6 th column of A] 7 l2k1 l2ki l2km 6 l 7 m ki
l1ij 7
6 [k th row of B] [j 7=6 6 1 7 7=6 i=1 l2 7
4 ::: 5 6
6 4 5 7 4
7 5
6 7 Pm pi ij
[p th row of B] [j th column of A] 6 l1mj 7 i=1 l2 l1
6 7
6 ::: 7
6 2 3 7
6
6 l11j 7
7
6 6 7 7
6 6 ij 7 7
6 l2p1 ::: l2pi ::: l2pm 6 l 7 7
6 6 1 7 7
4 4 5 5
l1mj

as desired.

7.4 Some related properties of a linear function and associated

matrix
In this section, the following objects will be given: a vector space V with a basis V = v 1 ; :::; v n ; a vector
space U with a basis U = u1 ; :::; un ; l 2 L (V; U ) and A 2 M (m; n).
From Remark 150, recall that

col span A = fz 2 Rm : 9 x 2 Rn such that z = Axg ;

U
Lemma 282 crU (Im l) = col span [l]V .

Proof. [ ]
def cr def Im l
y 2 crU (Im l) ) 9u 2 Im l such that crU (u) = [u]U = y ) 9v 2 V such that l (v) = u ) 9v 2 V
Prop. 273 U U
such that [l (v)]U = y ) 9v 2 V such that [l]V [v]V = y ) y 2 col span [l]V .
[ ]
U
We want to show that y 2 col span [l]V ) y 2 crU (Im l), i.e., 9u 2 Im l such that y = [u]U .
U U def cr Pn U
y 2 col span [l]V ) 9xy 2 Rn such that [l]V xy = y ) 9v = j
j=1 xy;j v such that [l]V [v]V =
Prop. 273 u=l(v)
y ) 9v 2 V such that [l (v)]U = y ) 9u 2 Im l such that y = [u]U , as desired.
U U
Lemma 283 dim Im l = dim col span [l]V = rank [l]V .

Proof. It follows from the above Lemma, the proof of Proposition 256, which says that crU is an
isomorphism and Proposition 263, which says that isomorphic spaces have the same dimension.
7.4. SOME RELATED PROPERTIES OF A LINEAR FUNCTION AND ASSOCIATED MATRIX 89

Proposition 284 Given l 2 L (V; U ),

U
1. l onto , rank [l]V = dim U ;
U
2. l one-to-one , rank [l]V = dim V ;
h i 1
U 1 V U
3. l invertible , [l]V invertible, and in that case l U
= [l]V .

Proof. Recall that from Remark 249 and Proposition 261, l one-to-one , l nonsingular , ker l = f0g.
Lem m a 283 U
1. l onto , Im l = U , dim Im l = dim U , rank [l]V = dim U .
Prop osition 245 Lem m a 283 U
2. l one-to-one , dim Im l = dim V , rank [l]V = dim V .
3. The …rst part of the statement follows from 1. and 2. above. The second part is proven below. First
of all observe that for any vector space W with a basis W = w1 ; ::; wk , we have that idW 2 L (W; W )
and
W
[idW ]W = idW w1 W ; ::; idW wk W = Ik :

Moreover, if l is invertible
1
l l = idV

and
1 V V
l l V
= [idU ]V = Im :

Since
1 V 1 V U
l l V
= l U
[l]V ;

the desired result follows.

Remark 285 From the de…nitions of ' and , we have what follows:
1.
U U
l = (' (l)) = [l]V = l[l] U
;V
:
V

2.
U U U
A = ' ( (A)) = ' lA;V = lA;V V
:
U
Lemma 286 crU Im lA;V = col spanA.
U U
Proof. Recall that ' (l) = [l]V and (A) = lA;V . For any l 2 L (V; U ),

Lemma 282 U Def: 271

crU (Im l) = col span [l]V = col span ' (l) (7.4)
U
Take l = lA;V . Then from (7:4) ;we have
U U Rmk: 285:2
crU Im lA;V = col span ' lA;V = col span A:

U
Lemma 287 dim Im lA;V = dim col spanA = rankA.
U
Proof. Since Lemma 283 holds for any l 2 L (V; U ) and lA;V 2 L (V; U ), we have that

U U U Rmk: 285:2 Rmk: 231:

dim Im lA;V = dim col span lA;V V
= dim col span A = rank A:

Proposition 288 Let A 2 M (m; n) be given.

90 CHAPTER 7. LINEAR FUNCTIONS AND MATRICES

U
1. rankA = m , lA;V onto;
U
2. rankA = n , lA;V one-to-one;
U V U 1
3. A invertible , lA;V invertible, and in that case lA 1 ;U = lA;V :

Lem m a 287 U U
Proof. 1. rankA = m , dim Im lA;V = m , lA;V onto;
(1) U Prop osition 261
U
2. rankA = n , dim ker lA;V =0 , lA;V one-to-one,
U U
where (1) the …rst equivalence follows form the fact that n = dim ker lA;V + dim Im lA;V , and Lemma 287.
Prop. 226 1 and 2 ab ove U
3. First statement: A invertible , rankA = m = n , lA;V invertible.
U U 1
Second statement: Since lA;V invertible, there exists lA;V : U ! V such that

U 1 U
idV = lA;V lA;V :

Then
Prop. 281 1 Rm k. 285 1
I = 'V
V (idV ) = 'V
U
U
lA;V 'U U
V lA;V = 'V
U
U
lA;V A:

Then, by de…nition of inverse matrix,

1 1
A = 'V
U
U
lA;V

and
V 1 V 1 1 1
U A = U 'V
U
U
lA;V = idL(U;U ) U
lA;V U
= lA;V :
V
Finally, from the de…nition of U, we have
V 1 V
U A = lA 1 ;U ;

as desired.

Remark 289 Consider A 2 M (n; n). Then from Proposition 288,

U
A invertible , lA;V invertible;

from Proposition 226,

A invertible , A nonsingular;
from Proposition 264,
U U
lA;V invertible , lA;V nonsingular.
Therefore,
U
A nonsingular , lA;V nonsingular.
Symmetrically,
U U
[l]V invertible , [l]V nonsingular , l invertible, l nonsingular.

Proposition 290 Let l 2 L(V; U ) be given. Then there exists a basis V of V and a basis U of U such that

I 0
[l]U
V = ;
0 0

where I is the r-square identity matrix and r = rank[l]U

V.

Proof. Suppose dim V = m and dim U = n. Let W = ker l and U 0 = Iml. By assumption, rank[l]U
V = r.
Then, by Lemma 283,
dim Iml = rank[l]U
V = r:

Therefore, from the Dimension Theorem

dim ker = dim V dim Iml = m r:

7.5. SOME FACTS ON L RN ; RM 91

Let W = w1 ; :::; wm r be a basis of W . Then, from the Completion Lemma, there exist vectors v 1 ; :::; v r 2
V such that V := v 1 ; :::; v r ; w1 ; :::; wm r is a basis of V . For any i 2 f1; :::; rg, set ui = l(v i ). Then,
u1 ; :::; ur is a basis of U 0 . Then, again from the Completion Lemma, there exists ur+1 ; :::; un 2 U such
that U := u1 ; :::; ur ; ur+1 ; :::; un is a basis of U . Then,

l(v 1 ) = u1 = 1u1 + 0u2 + ::: + 0ur + 0ur+1 + ::: + 0un ;

l(v 2 ) = u2 = 0u1 + 1u2 + ::: + 0ur + 0ur+1 + ::: + 0un ;

l(v r ) = ur = 0u1 + 0u2 + ::: + 1ur + 0ur+1 + ::: + 0un ;

l(w1 ) = 0 = 0u1 + 0u2 + ::: + 0ur + 0ur+1 + ::: + 0un ;

l(wm r
) = 0 = 0u1 + 0u2 + ::: + 0ur + 0ur+1 + ::: + 0un ;

i.e.,
I 0
[l]U
V = :
0 0

7.5 Some facts on L (Rn ; Rm )

In this Section, we specialize (basically repeat) the content of the previous Section in the important case in
which n
V = Rn ; V = ejn j=1 := en

m
U = Rm U = eim i=1
:= em

v=x (7.5)
and therefore
l 2 L (Rn ; Rm ) :
From L (Rn ; Rm ) to M (m; n).
From De…nition 269, we have
h i
e
[l]emn
= l e1n em
::: l ejn em
::: [l (enn )]em =
(7.6)
= l e1n ::: l ejn ::: l (enn ) := [l] ;

from De…nition 271,

' := 'eem
n
: L (Rn ; Rm ) ! M (m; n) ; l 7! [l] ;
from Proposition 273,
[l] x = l (x) : (7.7)
From M (m; n) to L (Rn ; Rm ).
From De…nition 275,
em
lA := lA;e m
: Rn ! Rm ; (7.8)
Pm
lA (x) = i=1 Ri (A) [x]en eim =
2 3
R1 (A) x
6 ::: 7 (7.9)
6 7
=6
6 Ri (A) x 7
7 = Ax:
4 ::: 5
Rm (A) x
92 CHAPTER 7. LINEAR FUNCTIONS AND MATRICES

From De…nition 278,

em
:= en : M (m; n) ! L (Rn ; Rm ) : A 7! lA : :

From Proposition 280,

M (m; n) and L (Rn ; Rm ) are isomorphic.
From Proposition 281, if l1 2 L (Rn ; Rm ) and l2 2 L (Rm ; Rp ), then

[l2 l1 ] = [l2 ] [l1 ] : (7.10)

Some related properties.

From Proposition 284, given l 2 L (Rn ; Rm ),
1. l onto , rank [l] = m;
2. l one-to-one , rank [l] = n;
1
3. l invertible , [l] invertible, and in that case l 1 = [l] .
From Remark 285,
1.
l = (' (l)) = ([l]) = l[l] :
2.
A = ' ( (A)) = ' (lA ) = [lA ] :
From Proposition 288, given A 2 M (m; n),
1. rankA = m , lA onto;
2. rankA = n , lA one-to-one;
1
3. A invertible , lA invertible, and in that case lA 1 = (lA ) :

Remark 291 From (7:7) and Remark 150,

Im l := fy 2 Rm : 9x 2 Rn such that y = [l] xg = col span [l] : (7.11)

Then, from the above and Remark 231,

dim Im l = rank [l] = max # linearly independent columns of [l] : (7.12)

Similarly, from (7:9) and Remark 231, we get

Im lA := fy 2 Rm : 9x 2 Rn such that y = lA x = Axg = col span A;

and
dim Im lA = rank A = max # linearly independent columns of A:

Remark 292 Recipe to …nd a basis of Im l for l 2 L (Rn ; Rm ).

Assume that l 2 L (Rn ; Rm ). The above Remark gives a way of …nding a basis of Im l: it is enough to
consider a number equal to rank [l] of linearly independent vectors among the column vectors of [l]. In a
more detailed way, we have what follows.
1. Compute [l].
2. Compute dim Im l = rank [l] := r.
3. To …nd a basis of Im l, we have to …nd r vectors which are a. linearly independent, and b. elements of
Im l. Indeed, it is enough to take r linearly independent columns of [l]. Observe that for any i 2 f1; :::; mg,
C i ([l]) 2 Im l = col span [l].
To get a “simpler” basis, you can make elementary operations on those column. Recall that elementary
operations on linearly independent vectors lead to linearly independent vectors, and that elementary operations
on vectors lead to vector belonging to the span of the starting vectors.

Example 293 Given n 2 N with n 3, and the linear function

8 Pn
< i=1 xi
n
l : Rn ! R3 ; x := (xi )i=1 7! x2 :
:
x2 + x3
 U
7.6. EXAMPLES OF COMPUTATION OF [L]V 93

…nd a basis forIm l.

1. 2 3
1 1 1 1 ::: 1
[l] = 4 0 1 0 0 ::: 0 5
0 1 1 0 ::: 0
2. 2 3 2 3
1 1 1 1 0 1
rank 4 0 1 0 5 = rank 4 0 1 0 5=3
0 1 1 0 0 1
3. A basis of Im la is given by the column vectors of
2 3
1 0 1
4 0 1 0 5:
0 0 1
Remark 294 From (7:7), we have that
ker l = fx 2 Rn : [l] x = 0g ;
i.e., ker l is the set, in fact the vector space, of solution to the systems [l] x = 0. In Remark 315, we will
describe an algorithm to …nd a basis of the kernel of an arbitrary linear function.
Remark 295 From Remark 291 and Proposition 245 (the Dimension Theorem), given l 2 L (Rn ; Rm ),
dim Rn = dim ker l + rank [l] ;
and given A 2 M (m; n),
dim Rn = dim ker lA + rank A:

7.6 Examples of computation of [l]UV

1. id 2 L (V; V ) :
V
[id]V = id v 1 V
; :::; id v j V
; :::; [id (v n )]V =

= v1 V
; :::; v j V
; :::; [v n ]V = e1n ; :::; ejn ; :::; enn = In :
2. 0 2 L (V; U ) :
U
[0]V = [[0]V ; :::; [0]V ; :::; [0]V ] = 0 2 M (m; n) :
3. l 2 L (V; V ), with 2 F:
V
[l ]V = v1 V
; :::; vj V
; :::; [ v n ]V = v1 V
; :::; vj V
; :::; [v n ]V =

= e1n ; :::; ejn ; :::; enn = In :

n m
4. lA 2 L (R ; R ), with A 2 M (m; n).

[lA ] = A e1n ; :::; A ejn ; :::; A enn = A e1n ; :::; ejn ; :::; enn = A In = A:
n+k n
5. (projection function) projn+k;n 2 L Rn+k ; Rn ; projn+k;n : (xi )i=1 7! (xi )i=1 : De…ned
projn+k;n := p, we have

[p] = p e1n+k ; :::; p enn+k ; p en+1 n+k

n+k ; :::; p en+k = e1n ; :::; enn ; 0; :::; 0 = [In j0] ;
where 0 2 M (n; k).
n n
6. (immersion function) in;n+k 2 L Rn ; Rn+k ; in;n+k : (xi )i=1 7! ((xi )i=1 ; 0) with 0 2 Rk :De…ned
in;n+k := i, we have

e1n ::: enn In

[i] = i e1n ; :::; i (enn ) = = ;
0 ::: 0 0
where 0 2 M (k; n).
94 CHAPTER 7. LINEAR FUNCTIONS AND MATRICES

Remark 296 Point 4. above implies that if l : Rn ! Rm ; x 7! Ax, then [l] = A. In other words, to
compute [l] you do not have to take the image of each element in the canonical basis; the …rst line of [l] is
the vector of the coe¢ cient of the …rst component function of l and so on. For example, if l : R2 ! R3 ;
8
< a11 x1 + a12 x2
x 7! a21 x1 + a22 x2 ;
:
a31 x1 + a32 x2

then 2 3
a11 a12
[l] = 4 a21 a22 5
a31 a32

7.7 Exercises
Problem sets: 15,16,17,18,19,21,22.
From Lipschutz (1991), starting from page 352:
10.1 ! 10.9, 10.29 ! 10.33.
Chapter 8

Solutions to systems of linear

equations

8.1 Some preliminary basic facts

Let’s recall some basic de…nition from Section 1.6.

De…nition 297 Consider the following linear system with m equations and n unknowns
8
< a11 x1 +
> + a1n xn = b1
..
> .
:
am1 x1 + + amn xn = bm

which can be rewritten as

Ax = b
Am n is called matrix of the coe¢ cients (or coe¢ cient matrix) associated with the system and Mm (n+1) =
A j b is called augmented matrix associated with the system.

Recall the following de…nition.

De…nition 298 Two linear system are said to be equivalent if they have the same solutions.

Let’s recall some basic facts we discussed in previous chapters.

Remark 299 It is well known that the following operations applied to a system of linear equations lead to
an equivalent system:
I) interchange two equations;
II) multiply both sides of an equation by a nonzero real number;
III) add left and right hand side of an equation to the left and right hand side of another equation;
IV) change the place of the unknowns.
The transformations I), II), III) and IV) are said elementary transformations, and, as it is well known,
they do not change the solution set of the system they are applied to.
Those transformations correspond to elementary operations on rows of M or columns of A in the way
described below
I) interchange two rows of M ;
II) multiply a row of M by a nonzero real number;
III) sum a row of M to another row of M ;
IV) interchange two columns of A.
The above described operations do not change the rank of A and they do not change the rank of M - see
Proposition 218.

Homogenous linear system.

95
96 CHAPTER 8. SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS

De…nition 300 A linear system for which b = 0, i.e., of the type

Ax = 0

with A 2 M (m; n), is called homogenous system.

Remark 301 Obviously, 0 is a solution of the homogenous system. The set of solution of a homogeneous
system is ker lA . From Remark 295,
dim ker lA = n rank A:

8.2 A solution method: Rouchè-Capelli’s and Cramer’s theorems

The solution method presented in this section is based on two basic theorems.
1. Rouchè-Capelli’s Theorem, which gives necessary and su¢ cient condition for the existence of solutions;
2. Cramer’s Theorem, which gives a method to compute solutions - if they exist.

Theorem 302 (Rouche Capelli) A system with m equations and n unknowns

Am nx =b (8.1)

has solutions
,
rank A = rank A j b

Proof. [)]
Let x be a solution to 8.1. Then, b is a linear combination, via the solution x of the columns of A.
Then, from Proposition 218,

rank A j b = rank A j 0 = rank A :

[(]
1st proof.
We want to show that
n
X
n
9x 2 R such that Ax = b, i.e., b = xj C j (A) :
j=1

By assumption, rank A = rank A j b := r. Since rank A = r, there are r linearly independent

column vectors of A, say C j1 (A) ; :::;,C jr (A). Since rank A j b = r, C j1 (A) ; :::;,C jr (A) ; b are linearly
dependent and from Lemma 196, b is a linear combinations of the vectors C j1 (A) ; :::;,C jr (A), i.e., 9 (xj )j2R
P
such that b = j2R xj C j (A) and
X X
b= xj C j (A) + 0 Cj 0 (A) :
j2R j 0 2f1;:::;ngnR

n
Then, x = xj j=1
such that
8
< xj if j2R
xj =
:
0 if j 0 2 f1; :::; ng nR

is a solution to Ax = b.
Second proof.
By assumption , rank [A] = rank A j b := r. Then, from Proposition ..., there exist r linearly
independent row vectors of A. Perform row interchanging operations on A j b to get the r linearly
independent rows of A as the …rst r rows, i.e., the following matrix

A0 j b0
:
A00 j b00
8.2. A SOLUTION METHOD: ROUCHÈ-CAPELLI’S AND CRAMER’S THEOREMS 97

A0
Observe that rankA0 = rank A0 j b0 = r. Interchange columns of
to have r linearly
A00
independent columns as the …rst r columns. From Remark 299, reordering columns of A and rows of
A j b does not change the rank of A; does not change the rank of A j b and does not change
the set of solutions. Therefore, we constructed the following system
A A12 x1 b0
= ;
A21 A22 x2 b00
where A12 2 M (r; n r) ; A21 2 M (m r; r) ; A22 2 M (m r; n r) ; x1 2 Rr ; x2 2 Rn r ; b0 2 Rr ; b00 2
m r
R ;
x1 ; x2 has been obtained from x performing on it the same permutations performed on the columns of
A,
(b0 ; b00 ) has been obtained from b performing on it the same permutations performed on the rows of
A j b .
Since
rank A A12 b0 = rank A = r;
then the r rows of A A12 b0 are linearly independent and since
A A12 b0
rank = r;
A21 A22 b00
A A12 b0
then the r rows of A A12 b0 are a basis of the span of the rows of :Then the
A21 A22 b00
last m r rows are linear combinations of the …rst r rows. Therefore, using again Remark 299, we have that
Ax = b is equivalent to
x1
A A12 = b0
x2
or, using Remark 74.2,
A x1 + A12 x2 = b0
and
1
x1 = (A ) b0 A12 x2 2 Rr
while x2 can be chosen arbitrarily; more precisely
n o
1
x1 ; x2 2 Rn : x1 = (A ) b0 A12 x2 2 Rr and x2 2 Rn r

is the nonempty set of solution to the system Ax = b.

Exercise 303 Apply the procedure described in the proof of Roche - Capelli theorem to solve the system
Ax = b, where 2 3 2 3
1 1 1 2 4
A = 4 2 2 2 1 5 and b = 4 5 5 :
3 3 3 3 9
The main step in the exercise amount to rewrite A and b as follows.
2 3
1 2 1 1 4
A A12 b0
=4 2 1 2 2 5 5
A21 A22 b00
3 3 3 3 9
4
and therefore x1 = (x3 ; x4 ), x2 = (x1 ; x2 ), b0 = ; b00 = (9) :Then,
5
1
x1 = (A ) b0 A12 x2 2 Rr
specializes in
1
x3 1 2 4 1 1 x1
= =
x4 2 1 5 2 2 x2

1 2
3 3 4 x1 + x2 x1 + x2 + 2
= 2 1 = :
3 3 5 2x1 + 2x2 1
98 CHAPTER 8. SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS

Theorem 304 (Cramer) A system with n equations and n unknowns

An nx =b

with det A 6= 0, has a unique solution x = (x1 ; :::; xi ; :::; xn ) where for i 2 f1; :::; ng ;
det Ai
xi =
det A
and Ai is the matrix obtained from A substituting the column vector b in the place of the i th column.
1 1 1
Proof. since det A 6= 0, A exists and it is unique. Moreover, from Ax = b, we get A Ax = A b and
1
x=A b

Moreover
1 1
A b= Adj A b
det A
It is then enough to verify that 2 3
det A1
6 ::: 7
6 7
Adj A b = 6
6 det Ai 7
7
4 ::: 5
det An
which we omit (see Exercise 7.34, page 268, in Lipschutz (1991).
The combinations of Rouche-Capelli and Cramer’s Theorem allow to give a method to solve any linear
system - apart from computational di¢ culties.

Remark 305 Rouche’-Capelli and Cramer’s Theorem based method.

Let the following system with m equations and n unknowns be given:

Am nx = b:

0. Simplify the system using elementary row operations on A j b and elementary column operations
on A. Those operation do not change rank A, rank B; set of solution to system Ax = b:
1. Compute rank A and rank A j b .
i. If
rank A 6= rank A j b ;
then the system has no solution.
ii. If
rank A = rank A j b := r;
then the system has solutions which can be computed as follows.
2. Extract a square r-dimensional invertible submatrix Ar from A.
i. Discard the equations, if any, whose corresponding rows are not part of Ar :
ii. In the remaining equations, bring on the right hand side the terms containing unknowns whose
coe¢ cients are not part of the matrix Ar , if any.
iii. You then get a system to which Cramer’s Theorem can be applied, treating as constant the expressions
on the right hand side and which contain n r unknowns. Those unknowns can be chosen arbitrarily.
Sometimes it is said that then the system has “1n r ” solutions or that the system admits n r degrees of
freedom. More formally, we can say what follows.

De…nition 306 Given S; T Rn , we de…ne the sum of the sets S and T , denoted by S + T , as follows

fx 2 Rn : 9s 2 S; 9t 2 T such that x = s + tg :

Proposition 307 Assume that the set S of solutions to the system Ax = b is nonempty and let x 2 S.
Then
S = fx g + ker lA := x 2 Rn : 9x0 2 ker lA such that x = x + x0
8.2. A SOLUTION METHOD: ROUCHÈ-CAPELLI’S AND CRAMER’S THEOREMS 99

Proof. [ ]
Take x 2 S. We want to …nd x0 2 ker A such that x = x +x0 . Take x0 = x x . Clearly x = x +(x x ).
Moreover,
(1)
Ax0 = A (x x ) = b b=0 (8.2)
where (1) follows from the fact that x; x 2 S.
[ ]
Take x = x + x0 with x 2 S and x0 2 ker A. Then

Ax = Ax + Ax0 = b + 0 = b:

Remark 308 The above proposition implies that a linear system either has no solutions, or has a unique
solution, or has in…nite solutions.

De…nition 309 V is an a¢ ne subspace of Rn if there exists a vector subspace W of Rn and a vector x 2 Rn

such that
V = fxg + W
We say that the1 dimension of the a¢ ne subspace V is dim W:

Remark 310 Let a 2 R be given. First of all observe that

(x; y) 2 R2 : y = ax = ker l;

where l 2 L R2 ; R and l (x; y) = ax y: Let’s present a geometric description of Proposition 307. We want
to verify that the following two sets are equal.

S := f(x0 ; y0 )g + (x; y) 2 R2 : y = ax ;

T := (x; y) 2 R2 : y = a (x x0 ) + y0 :
In words, we want to verify that the a¢ ne space “f(x0 ; y0 )g plus ker l” is nothing but the set of points
belonging to the line with slope a and going through the point (x0 ; y0 ).
S T . Take (x0 ; y 0 ) 2 S; then 9x00 2 R such that x0 = x0 + x00 and y 0 = y0 + ax00 . We have tho check
that y 0 = a (x0 x0 ) + y0 :Indeed, a (x0 x0 ) + y0 = a (x0 + x00 x0 + y) = ax00 + y0 :
T S. Take (x0 ; y 0 ) such that y 0 = a (x0 x0 ) + y0 . Take x00 = x0 + x0 :Then x0 = x0 + x00 and
y = y0 + ax00 , as desired.
0

Remark 311 Since dim ker lA = n rank A, the above Proposition and De…nition say that if a nonhomo-
geneous systems has solutions, then the set of solutions is an a¢ ne space of dimension n rank A.

Example 312 Consider the system (in one equation and two unknowns):

x1 + x2 1 = 0: (8.3)

Then x := (1; 0) is a solution to the above system. Moreover, de…ned l : R2 ! R, l (x 1 ; x2 ) = x1 + x2

ker l := fl (x) = 0g = (x1 ; x2 ) 2 R2 : x2 = x1 :

1 1
The set of solutions to (8:3) is f(0; 1)g + ker l. Observe that f(0; 1)g + ker l = f(1; 0)g + ker l = 2; 2 +
ker l.
x
1 If W 0 and W 00 are vector subspaces of Rn , x 2 Rn and V := fxg + W 0 = fxg + W 00 , then W 0 = W 00 .

Take w 2 W 0 , then x + w 2 V = fxg + W 00 . Then there exists x 2 fxg and w b 2 W 00 such that x + w = x + w.
b Then
w=w b 2 W 00 , and W 0 W 00 . Similar proof applies for the opposite inclusion.
100 CHAPTER 8. SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS

2.5

0
-5 -2.5 0 2.5 5

-2.5

-5

Remark 313 Exercise 314 Apply the algorithm described in Remark 305 to solve the following linear
system. 8
< x1 + 2x2 + 3x3 = 1
4x1 + 5x2 + 6x3 = 2
:
5x1 + 7x2 + 9x3 = 3
The associated matrix A j b is
2 3
1 2 3 j 1
4 4 5 6 j 2 5
5 7 9 j 3

1. Since the third row of the matrix A j b is equal to the sum of the …rst two rows, and since

1 2
det =5 8= 3;
4 5

we have that
rank A = rank A j b = 2;
and the system has solutions.
2. De…ne
1 2
A2 =
4 5
i. Discarding the equations, whose corresponding rows are not part of A2 and, in the remaining equations,
bringing on the right hand side the terms containing unknowns whose coe¢ cients are not part of the matrix
A2 , we get 8
< x1 + 2x2 = 1 3x3
4x1 + 5x2 = 2 6x3
:
5x1 + 7x2 = 3 9x3
iii. Then, using Cramer’s Theorem, recalling that det A2 = 3;we get

1 3x3 2
det
2 6x3 5 1
x1 = = x3 ;
3 3
1 1 3x3
det
4 2 6x3 2
x2 = = 2x3 ;
3 3
8.2. A SOLUTION METHOD: ROUCHÈ-CAPELLI’S AND CRAMER’S THEOREMS 101

and the solution set is

1 2
(x1 ; x2 ; x3 ) 2 R3 : x1 = x3 ; x2 = 2x3 :
3 3

Remark 315 How to …nd a basis of ker :

Let A 2 M (m; n) be given and rankA = r min fm; ng. Then, from the second proof of Rouche’-
Capelli’s Theorem, we have that system
Ax = 0
admits the following set of solutions
n o
1
x1 ; x2 2 Rr Rn r
: x1 = (A ) A12 x2 (8.4)

Observe that dim ker lA = n r := p. Then, a basis of ker lA is

1 1
[A ] A12 e1p [A ] A12 epp
B= ; :::; :
e1p epp

To check the above statement, we check that 1. B ker lA , and 2. B is linearly independent2 .
1. It follows from (8:4);
2. It follows from the fact that det e1p ; :::; epp = det I = 1.

Example 316
x1 + x2 + x3 + 2x4 = 0
x1 x2 + x3 + 2x4 = 0

De…ned
1 1 1 2
A = ; A12 =
1 1 1 2
the starting system can be rewritten as
1
x1 1 1 1 2 x3
= :
x2 1 1 1 2 x4

Then a basis of ker is

82 1 3 2 1 39
>
> 1 1 1 2 1 1 1 1 2 0 >
>
<6 7 6 7=
1 1 1 2 0 1 1 1 2 1
B= 6 4
7;6
5 4
7 =
5>
>
> 1 0 >
: ;
0 1
82 3 2 39
>
> 1 2 >
>
<6 7 6 0 7=
0
= 6 7 6
4 1 5;4 0
7
5>
>
> >
: ;
0 1

Example 317 Discuss the following system (i.e., say if admits solutions).
8
< x1 + x2 + x3 = 4
x1 + x2 + 2x3 = 8
:
2x1 + 2x2 + 3x3 = 12

The augmented matrix [Ajb] is: 2 3

1 1 1 j 4
4 1 1 2 j 8 5
2 2 3 j 12
2 Observe that B is made up by p vectors and dim ker lA = p.
102 CHAPTER 8. SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS

2 3
1 1 1 j 4
rank [Ajb] = rank 4 1 1 2 j 8 5 = 2 = rank A
0 0 0 j 0
From Step 2 of Rouche’-Capelli and Cramer’s method, we can consider the system

x2 + x3 = 4 x1
x2 + 2x3 = 8 x1

1 1
Therefore, x1 can be chosen arbitrarily and since det = 1,
1 2

4 x1 1
x2 = det = x1
8 x1 2

1 4 x1
x3 = det =4
1 8 x1
Therefore, the set of solution is

(x1 ; x2 ; x3 ) 2 R3 : x2 = x1 ; x3 = 4

Example 318 Discuss the following system

x1 + x2 = 2
x1 x2 = 0

The augmented matrix [Ajb] is:

1 1 j 2
1 1 j 0
Since det A = 1 1= 2 6= 0,
rank [Ajb] = rankA = 2
and the system has a unique solution:

2 1
det
0 1 2
x1 = = =1
2 2
1 2
det
1 0 2
x2 = = =1
2 2
Therefore, the set of solution is
f(1; 1)g

Example 319 Discuss the following system

x1 + x2 = 2
x1 + x2 = 0

The augmented matrix [Ajb] is:

1 1 j 2
1 1 j 0
1 2
Since det A = 1 1 = 0, and det = 2 6= 0, we have that
1 0

rank [Ajb] = 2 6= 1 = rankA

and the system has no solutions. Therefore, the set of solution is ?.

8.2. A SOLUTION METHOD: ROUCHÈ-CAPELLI’S AND CRAMER’S THEOREMS 103

Example 320 Discuss the following system

x1 + x2 = 2
2x1 + 2x2 = 4

The augmented matrix [Ajb] is:

1 1 j 2
2 2 j 4
From rank properties,

1 1 1 1
rank = rank =1
2 2 0 0

1 1 2 1 1 2
rank = rank =1
2 2 4 0 0 0
Recall that elementary operations of rows on the augmented matrix do not change the rank of either the
augmented or coe¢ cient matrices.
Therefore
rank [Ajb] = 1 = rankA
and the system has in…nite solutions. More precisely, the set of solutions is

(x1 ; x2 ) 2 R2 : x1 = 2 x2

Example 321 Say for which value of the parameter k 2 R the following system has one, in…nite or no
solutions:
8
< (k 1) x + (k + 2) y = 1
x + ky = 1
:
x 2y = 1
2 3
k 1 k+2 j 1
[Ajb] = 4 1 k j 1 5
1 2 j 1
2 3
k 1 k+2 1
1 k k 1 k+2 k 1 k+2
det [Ajb] = det 4 1 k 1 5 = det det + det =
1 2 1 2 1 k
1 2 1

(2 k) ( 2k + 2 k 2) + k 2 k+k+2 =2 k + 2k + k + k 2 + 2 = 2k + k 2 + 4
= 1 17 < 0: Therefore, the determinant is never equal to zero and rank [Ajb] = 3: Since rank A3 2 2,
the solution set of the system is empty of each value of k.

Remark 322 To solve a parametric linear system Ax = b, where A 2 M (m; n) n f0g, it is convenient to
proceed as follows.

1. Perform “easy” row operations on [Ajb];

2. Compute min fm; n + 1g := k and consider the k k submatrices of the matrix [Ajb].
There are two possibilities.
Case 1. There exists a k k submatrix whose determinant is di¤ erent from zero for some values of
the parameters;
Case 2. All k k submatrices have zero determinant for each value of the parameters.
If Case 2 occurs, then at least one row of [Ajb] is a linear combinations of other rows; therefore you
can eliminate it. We can therefore assume that we are in Case 1. In that case, proceed as described
below.
104 CHAPTER 8. SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS

3. among those k k submatrices, choose a matrix A which is a submatrix of A, if possible; if you have
more than one matrix to choose among, choose “the easiest one” from a computational viewpoint, i.e.,
that one with highest number of zeros, the lowest number of times a parameters appear, ... ;

4. compute det A , a function of the parameter;

5. analyze the cases det A 6= 0 and possibly det A = 0:

Example 323 Say for which value of the parameter a 2 R the following system has one, in…nite or no
solutions:
ax1 + x2 + x3 = 2
x1 ax2 = 0

Example 324 Say for which value of the parameter a 2 R the following system has one, in…nite or no
solutions: 8
< ax1 + x2 + x3 = 2
x1 ax2 = 0
:
2ax1 + ax2 = 4
2 3
a 1 1 2
[Ajb] = 4 1 a 0 0 5
2a a 0 4
2 3
a 1 1
1
det 4 1 a 0 5 = a + 2a2 = 0 if a = 0; :
2
2a a 0
1
Therefore, if a 6= 0; 2,
rank [Ajb] = rangoA = 3
and the system has a unique solution.
If a = 0;
2 3
0 1 1 2
[Ajb] = 4 1 0 0 0 5
0 0 0 4
0 1
Since det = 1, rank A = 2. On the other hand,
1 0
2 3
0 1 2
1 2
det 4 1 0 0 5= 1 det = 4 6= 0
0 4
0 0 4

and therefore rank [Ajb] = 3; and

rank [Ajb] = 3 6= 2 = rangoA
and the system has no solutions.
If a = 21 ;
2 1
3
2 1 1 2
[Ajb] = 4 1 1
2 0 0 5
1
1 2 0 4
Since
1 1 1
det 1 = ;
2 0 2
rangoA = 2:
Since 2 3
1
2 1 2
1 2
det 4 1 0 0 5= det = 4
0 4
1 0 4
8.2. A SOLUTION METHOD: ROUCHÈ-CAPELLI’S AND CRAMER’S THEOREMS 105

and therefore
rank [Ajb] = 3;
rank [Ajb] = 3 6= 2 = rangoA
and the system has no solutions.

Example 325 Say for which value of the parameter a 2 R the following system has one, in…nite or no
solutions: 8
< (a + 1) x1 + ( 2) x2 + 2ax3 = a
ax1 + ( a) x2 + x3 = 2
:
2 + (2a 4) x2 + (4a 2) x3 = 2a + 4
Then, 2 3
a+1 2 2a j a
[Ajb] = 4 a a 1 j 2 5:
2 2a 4 4a 2 j 2a + 4
It is easy to see that we are in Case 2 described in Remark 322: all 3 3 submatrices of [Ajb] have determinant
equal to zero - indeed, the last row is equal to 2 times the …rst row plus ( 2) times the second row. We can
then erase the third equation/row to get the following system and matrix.

(a + 1) x1 + ( 2) x2 + 2ax3 = a
ax1 + ( a) x2 + x3 = 2

a+1 2 2a j a
[Ajb] =
a a 1 j 2

2 2a
det = 2a2 2 = 0;
a 1
whose solutions are 1; 1. Therefore, if a 2 Rn f 1; 1g,

rank [Ajb] = 2 = rangoA

and the system has in…nite solutions. Let’s study the system for a 2 f 1; 1g.
If a = 1, we get
0 2 2 j 1
[Ajb] =
1 1 1 j 2
and since
0 2
det = 2 6= 0
1 1
we have again
rank [Ajb] = 2 = rangoA
If a = 1,we have
2 2 2 j 1
[Ajb] =
1 1 1 j 2
and
rank [Ajb] = 2 > 1 = rangoA
and the system has no solution..

Example 326 Say for which value of the parameter a 2 R the following system has one, in…nite or no
solutions: 8
>
> ax1 + x2 = 1
<
x1 + x2 = a
> 2x1 + x2
> = 3a
:
3x1 + 2x2 = a
106 CHAPTER 8. SOLUTIONS TO SYSTEMS OF LINEAR EQUATIONS
2 3
a 1 j 1
6 1 1 j a 7
[Ajb] = 6
4 2
7
1 j 3a 5
3 2 j a
Observe that
rank A4 2 2:
2 3
1 1 a
det 4 2 1 3a 5 = 3a = 0 if a = 0:
3 2 a
Therefore, if a 2 Rn f0g,
rank [Ajb] = 3 > 2 rank A4 2

If a = 0, 2 3
0 1 j 1
6 1 1 j 0 7
6
[Ajb] = 4 7
2 1 j 0 5
3 2 j 0
and since 2 3
0 1 1
det 4 1 1 0 5= 1
2 1 0
the system has no solution for a = 0.
Summarizing, 8a 2 R, the system has no solutions.

8.3 Exercises
Problem sets: 20,23,24.
From Lipschutz (1991), page 179: 5.56; starting from page 263, 7.17 ! 7.20.
 Part II

Some topology in metric spaces

107
Chapter 9

Metric spaces

9.1 De…nitions and examples

De…nition 327 Let X be a nonempty set. A metric or distance on X is a function d : X X ! R such
that 8x; y; z 2 X
1. (a:) d (x; y) 0, and (b:) d (x; y) = 0 , x = y,
2. d (x; y) = d (y; x),
3. d (x; z) d (x; y) + d (y; z) (Triangle inequality).
(X; d) is called a metric space.

Remark 328 Observe that the de…nition requires that 8x; y 2 X, it must be the case that d (x; y) 2 R.

Example 329 n-dimensional Euclidean space with Euclidean metric.

Given n 2 N, take X = Rn , and

n
! 21
X 2
n n
d2;n : R R ! R; (x; y) 7! (xi yi ) :
i=1

(X; d2;n ) was shown to be a metric space in Proposition 58, Section 2.3. d2;n is called the Euclidean distance
in Rn . In what follows, unless needed, we write simply d2 in the place of d2;n .

Proposition 330 (Discrete metric space) Given a nonempty set X and the function
8
< 0 if x = y
d : X 2 ! R; d (x; y) =
:
1 if x = 6 y;

(X; d) is a metric space, called discrete metric space.

Proof. 1a. 0; 1 0:
1b. From the de…nition, d (x; y) = 0 , x = y.
2. It follows from the fact that x = y , y = x and x 6= y , y 6= x.
3. If x = z, the result follows. If x 6= z, then it cannot be x = y and y = z, and again the result follows.

Proposition 331 Given n 2 N; p 2 [1; +1) ; X = Rn ;

n
! p1
X p
n n
d:R R ! R; (x; y) 7! jxi yi j ;
i=1

(X; d) is a metric space.

109
110 CHAPTER 9. METRIC SPACES

Proof. 1a. It follows from the de…nition of absolute value.

1b. [(]Obvious.
Pn p 1 Pn p
[)] ( i=1 jxi yi j ) p = 0 , i=1 jxi yi j = 0 ) for any i , jxi yi j = 0 )for any i , xi yi = 0.
2. It follows from the fact jxi yi j = jyi xi j.
3. First of all observe that
n
! p1
X p
d (x; z) = j(xi yi ) + (yi zi )j :
i=1

Then, it is enough to show that

n
! p1 n
! p1 n
! p1
X p
X p
X p
j(xi yi ) + (yi zi )j jxi yi j + j(yi zi )j
i=1 i=1 i=1

which is a consequence of Proposition 332 below.

Proposition 332 Taken n 2 N; p 2 [1; +1) ; X = Rn ; a; b 2 Rn

n
! p1 n
! p1 n
! p1
X p
X p
X p
jai + bi j jai j + jbi j
i=1 i=1 i=1

Proof. It follows from the proof of the Proposition 335 below.

De…nition 333 Let R1 be the set of sequences in R.

De…nition 334 For any p 2 [1; +1), de…ne1

( +1
)
X p
p 1
l = (xn )n2N 2 R : jxn j < +1 ;
n=1

i.e., roughly speaking, lp is the set of sequences whose associated series are absolutely convergent.

Proposition 335 (Minkowski inequality). 8 (xn )n2N ; (yn )n2N 2 lp ; 8p 2 [1; +1),

+1
! p1 +1
! p1 +1
! p1
X p
X p
X p
jxn + yn j jxn j + jyn j : (9.1)
n=1 n=1 n=1

Proof. If either (xn )n2N or (yn )n2N are such that 8n 2 N; xn = 0 or 8n 2 N; yn = 0, i.e., if either
sequence is the constant sequence of zeros, then (9:1) is trivially true.
Then, we can consider the case in which
1 1
P+1 p p P+1 p p
9 ; 2 R++ such that n=1 jxn j = and n=1 jyn j = . (9.2)

De…ne p p
jxn j jyn j
8n 2 N; bn =
x and ybn = : (9.3)

Then
+1
X +1
X
bn =
x ybn = 1: (9.4)
n=1 n=1

For any n 2 N, from the triangle inequality for the absolute value, we have

jxn + yn j jxn j + jyn j ;

1 For basic results on series, see, for example, Section 10.5 in Apostol (1967).
9.1. DEFINITIONS AND EXAMPLES 111

since 8p 2 [1; +1), fp : R+ ! R; fp (t) = tp is an increasing function, we have

p p
jxn + yn j (jxn j + jyn j) : (9.5)

Moreover, from (9:3),

p p
1 1 1 1
p p
(jxn j + jyn j) = jb
xn j p + jb
yn j p =( + ) jb
xn j p + jb
yn j p : (9.6)
+ +

Since 8p 2 [1; +1), fp is convex (just observe that fp00 (t) = p (p 1) tp 2

0), we get
p
jb
xn j + jb
yn j jb
xn j + jb
yn j (9.7)
+ + + +

From (9:5) ; (9:6) and (9:7), we get

p p
jxn + yn j ( + ) jb
xn j + jb
yn j :
+ +

From the above inequalities and basic properties of the series, we then get
+1 +1 +1
!
X p p
X X (9:4) p p
jxn + yn j ( + ) jb
xn j + jb
yn j = ( + ) + =( + ) :
n=1
+ n=1
+ n=1
+ +

Therefore, using (9:2), we get

0 ! p1 ! p1 1p
+1
X +1
X +1
X
p @ p p A ;
jxn + yn j jxn j + jyn j
n=1 n=1 n=1

and therefore the desired result.

Proposition 336 (lp ; dp ) with

+1
! p1
X p
p p
dp : l l ! R; dp (xn )n2N ; (yn )n2N = jxn yn j :
n=1

is a metric space.
1
P+1 p p
Proof. We …rst of all have to check that dp (x; y) 2 R, i.e., that n=1 jxn yn j converges.

+1
! p1 +1
! p1 +1
! p1 +1
! p1
X p
X p
X p
X p
jxn yn j = jxn + ( yn )j jxn j + jyn j < +1;
n=1 n=1 n=1 n=1

where the …rst inequality follows from Minkowski inequality and the second inequality from the assump-
tion that we are considering sequences in lp .
Properties 1 and 2 of the distance follow easily from the de…nition. Property 3 is again a consequence of
Minkowski inequality:

+1
! p1 +1
! p1 +1
! p1
X p
X p
X p
dp (x; z) = j(xn yn ) + (yn zn )j jxn yn j + j(yn zn )j := dp (x; y)+dp (y; z) :
n=1 n=1 n=1
112 CHAPTER 9. METRIC SPACES

De…nition 337 Let T be a non empty set. B (T ) is the set of all bounded real functions de…ned on T , i.e.,

B (T ) := ff : T ! R : sup fjf (x)j : x 2 T g < +1g ;

and2
d1 : B (T ) B (T ) ! R; d1 (f; g) = sup fjf (x) g (x)j : x 2 T g

De…nition 339
l1 = (xn )n2N 2 R1 : sup fjxn j : n 2 Ng < +1
is called the set of bounded real sequences, and, still using the symbol of the previous de…nition,

d1 : l 1 l1 ! R; d1 (xn )n2N ; (yn )n2N = sup fjxn yn j : n 2 Ng

Proposition 340 (B (T ) ; d1 )and (l1 ; d1 ) are metric spaces, and d1 is called the sup metric.

Proof. We show that (B (T ) ; d1 ) is a metric space. As usual, the di¢ cult part is to show property 3 of
d1 , which is done below.
8f; g; h 2 B (T ) ; 8x 2 T;

jf (x) g (x)j jf (x) h (x)j + jh (x) g (x)j

sup fjf (x) h (x)j : x 2 T g + sup fjh (x) g (x)j : x 2 T g =

= d1 (f; h) + d1 (h; g) :

Then,8x 2 T;
d1 (f; g) := sup jf (x) g (x)j d1 (f; g) + d1 (h; g) :

Exercise 341 If (X; d) is a metric space, then

d
X;
1+d

is a metric space.

Proposition 342 Given a metric space (X; d) and a set Y such that ? 6= Y X, then Y; djY Y is a
metric space.

Proof. By de…nition.

De…nition 343 Given a metric space (X; d) and a set Y such that ? 6= Y X, then Y; djY Y , or simply,
(Y; d) is called a metric subspace of X.

Example 344 1. Given R with the (Euclidean) distance d2;1 , ([0; 1) ; d2;1 ) is a metric subspace of (R; d2;1 ).
2. Given R2 with the (Euclidean) distance d2;2 , (f0g R; d2;2 ) is a metric subspace of R2 ; d2;2 .

Exercise 345 Let C ([0; 1]) be the set of continuous functions from [0; 1] to R. Show that a metric on that
set is de…ned by Z 1
d (f; g) = jf (x) g (x)j dx;
0

where f; g 2 C ([0; 1]).

2

De…nition 338 Observe that d1 (f; g) 2 R :

d1 (f; g) := sup fjf (x) g (x)j : x 2 T g sup fjf (x)j : x 2 T g + sup fjg (x)j : x 2 T g < +1:
9.2. OPEN AND CLOSED SETS 113

Example 346 Let X be the set of continuous functions from R to R, and consider d (f; g) = supx2R jf (x) g (x)j.
(X; d) is not a metric space because d is not a function from X 2 to R: it can be supx2R jf (x) g (x)j = +1.

Example 347 Let X = fa; b; cg and d : X 2 ! R such that

d (a; b) = d (b; a) = 2

d (a; c) = d (c; a) = 0

d (b; c) = d (c; b) = 1:

Since d (a; b) = 2 > 0 + 1 = d (a; c) + d (b; c), then (X; d) is not a metric space.

Example 348 Given n 2 N; p 2 (0; 1) ; X = R2 ; de…ne

2
! p1
X p
2 2
d:R R ! R; (x; y) 7! jxi yi j ;
i=1

(X; d) is not a metric space, as shown below. Take x = (0; 1) ; y = (1; 0) and z = (0; 0). Then
1 1
d (x; y) = (1p + 1p ) p = 2 p ;
1
d (x; z) = (0p + 1p ) p = 1

d (z; y) = 1:
1
Then, d (x; y) (d (x; z) + d (z; y)) = 2 p 2 > 0:

9.2 Open and closed sets

De…nition 349 Let (X; d) be a metric space. 8x0 2 X and 8r 2 R++ , the open r-ball of x0 in (X; d) is the
set
B(X;d) (x0 ; r) = fx 2 X : d (x; x0 ) < rg :
If there is no ambiguity about the metric space (X; d) we are considering, we use the lighter notation
B (x0 ; r) in the place of B(X;d) (x0 ; r).

Example 350 1.
B(R;d2 ) (x0 ; r) = (x0 r; x0 + r)
is the open interval of radius r centered in x0 .
2. q
2 2
B(R2 ;d2 ) (x0 ; r) = (x1 ; x2 ) 2 R2 : (x1 x01 ) + (x2 x02 ) < r

is the open disk of radius r centered in x0 .

3. In R2 with the metric d given by

d ((x1 ; x2 ) ; (y1 ; y2 )) = max fjx1 y1 ; jx2 y2 jjg

the open ball B (0; 1) can be pictured as done below:

a square around zero.

De…nition 351 Let (X; d) be a metric space. x is an interior point of S X if

there exists an open ball centered in x and contained in S, i.e.,
9 r 2 R++ such that B (x; r) S.

De…nition 352 The set of all interior points of S is called the Interior of S and it is denoted by Int(X;d) S
or simply by Int S.
114 CHAPTER 9. METRIC SPACES

Remark 353 Int (S) S, simply because x 2 Int (S) ) x 2 B (x; r) S, where the …rst inclusion follows
from the de…nition of open ball and the second one from the de…nition of Interior. In other words, to …nd
interior points of S, we can limit our search to points belonging to S.
It is not true that 8S X; S Int (S), as shown below. We want to prove that

:(8S X; 8x 2 S; x 2 S ) x 2 IntS);

i.e.,
(9S X and x 2 S such that x 2
= Int S).
Take (X; d) = (R; d2 ), S = f1g and x = 1. Then, clearly 1 2 f1g, but 1 2
= Int S : 8r 2 R++ , (1 r; 1 + r) *
f1g.

Remark 354 To understand the following example, recall that 8a; b 2 R such that a < b, 9c 2 Q and
d 2 RnQ such that c; d 2 (a; b) - see, for example, Apostol (1967).

Example 355 Let (R; d2 ) be given.

1. Int N = Int Q =?.
2. 8a; b 2 R; a < b, Int [a; b] = Int [a; b) = Int (a; b] = Int (a; b) = (a; b).
3. Int R = R.
4. Int ? = ?.

De…nition 356 Let (X; d) be a metric space. A set S X is open in (X; d) ; or (X; d)-open, or open with
respect to the metric space (X; d), if S Int S, i.e., S = Int S, i.e.,

8x 2 S, 9r 2 R++ such that B(X;d) (x; r) := fy 2 X : d (y; x) < rg S:

Remark 357 Let (R; d2 ) be given. From Example 355, it follows that
N; Q; [a; b] ; [a; b) ; (a; b] are not open sets, and (a; b) ; R and ? are open sets. In particular, open interval
are open sets, but there are open sets which are not open interval. Take for example S = (0; 1) [ (2; 3).

Exercise 358 8n 2 N; 8i 2 f1; :::; ng ; 8ai ; bi 2 R with ai < bi ,

n
i=1 (ai ; bi )

is (Rn ; d2 ) open.

Proposition 359 Let (X; d) be a metric space. An open ball is an open set.

Proof. Take y 2 B (x0 ; r). De…ne

=r d (x0 ; y) : (9.8)
First of all, observe that, since y 2 B (x0 ; r), d (x0 ; y) < r and then 2 R++ . It is then enough to show
that B (y; ) B (x0 ; r), i.e., we assume that

d (z; y) < (9.9)

and we want to show that d (z; x0 ) < r. From the triangle inequality
(9:9);(9:8)
d (z; x0 ) d (z; y) + d (y; x0 ) < + (r ) = r;

as desired.

Example 360 In a discrete metric space (X; d), 8x 2 X; 8r 2 (0; 1] ; B (x; r) := fy 2 X : d (x; y) < rg =
fxg and 8r > 1; B (x; r) := fy 2 X : d (x; y) < rg = X. Then, it is easy to show that any subset of a discrete
metric space is open, as veri…ed below. Let (X; d) be a discrete metric space and S X. For any x 2 S,
take " = 21 ; then B x; 12 = fxg S.

De…nition 361 Let a metric space (X; d) be given. A set T X is (X; d) closed or closed in (X; d) if its
complement in X, i.e., XnT is open in (X; d).
9.2. OPEN AND CLOSED SETS 115

If no ambiguity arises, we simply say that T is closed in X, or even, T is closed; we also write T C in the
place of XnT .
C
Remark 362 S is open , S C is closed, simply because S C closed , S C = S is open.

Example 363 The following sets are closed in (R; d2 ): R; N; ?; 8a; b 2 R, a < b, fag and [a; b].

Remark 364 It is false that:

S is not open ) S is closed
(and therefore that S is not closed ) S is open), i.e., there exist sets which are not open and not closed,
for example (0; 1] in (R; d2 ). There are also two sets which are both open and closed: ? and Rn in (Rn ; d2 ).

Proposition 365 Let a metric space (X; d) be given.

1. ? and X are open sets.
2. The union of any (…nite or in…nite) collection of open sets is an open set.
3. The intersection of any …nite collection of open sets is an open set.

Proof. 1.
8x 2 X; 8r 2 R++ , B (x; r) X. ? is open because it contains no elements.
2.
Let I be a collection of open sets and S = [A2I A. Assume that x 2 S. Then there exists A 2 I such
that x 2 A. Then, for some r 2 R++

x 2 B (x; r) A S
where the …rst inclusion follows from fact that A is open and the second one from the de…nition of S.
3.
Let F be a collection of open sets, i.e., F = fAn gn2N , where N N, #N is …nite and 8n 2 N , An is an
open set. Take S = \n2N An . If S = ?, we are done. Assume that S 6= ? and that x 2 S. Then from the
fact that each set A is open and from the de…nition of S as the intersection of sets

8n 2 N; 9rn 2 R++ such that x 2 B (x; rn ) An

Since N is a …nite set, there exists a positive r = min frn : n 2 N g > 0. Then

8n 2 N; x 2 B (x; r ) B (x; rn ) An

and from the very de…nition of intersections

x 2 B (x; r ) \n2N B (x; rn ) \n2N An = S:

Remark 366 The assumption that #N is …nite cannot be dispensed with:

1 1 1
\+1
n=1 B 0; = \+1
n=1 ; = f0g
n n n

is not open.

Remark 367 A generalization of metric spaces is the concept of topological spaces. In fact, we have the
following de…nition which “ assumes the previous Proposition”.
Let X be a nonempty set. A collection T of subsets of X is said to be a topology on X if
1. ? and X belong to T ,
2. The union of any (…nite or in…nite) collection of sets in T belongs to T ,
3. The intersection of any …nite collection of sets in T belongs to T .
(X; T ) is called a topological space.
The members of T are said to be open set with respect to the topology T , or (X; T ) open.
116 CHAPTER 9. METRIC SPACES

Proposition 368 Let a metric space (X; d) be given.

1. ? and X are closed sets.
2. The intersection of any (…nite or in…nite) collection of closed sets is a closed set.
3. The union of any …nite collection of closed sets is a closed set.

Proof. 1
It follows from the de…nition of closed set, the fact that ?C = X; X C = ? and Proposition 365.
2.
Let I be a collection of closed sets and S = \B2I B. Then, from de Morgan’s laws,
C
S C = (\B2I B) = [B2I B C

Then from Remark 362, 8B 2 I, B C is open and from Proposition 365.1, [B2I B C is open as well.
2.
Let F be a collection of closed sets, i.e., F = fBn gn2N , where N N, #N is …nite and 8n 2 N , Bn is
an open set. Take S = [n2N Bn . Then, from de Morgan’s laws,
C
S C = ([n2N Bn ) = \n2N BnC

Then from Remark 362, 8n 2 N , B C is open and from Proposition 365.2, \B2I B C is open as well.

Remark 369 The assumption that #N is …nite cannot be dispensed with:

C C
1 1 1 1
\+1
n=1 B 0; = [+1
n=1 B 0; = [+1
n=1 1; ; [ ; +1 = Rn f0g :
n n n n

is not closed.

De…nition 370 If S is both closed and open in (X; d), S is called clopen in (X; d).

Remark 371 In any metric space (X; d), X and ? are clopen.

Proposition 372 In any metric space (X; d), fxg is closed.

Proof. We want to show that Xn fxg is open. If X = fxg, then Xn fxg = ?, and we are done. If
X 6= fxg, take y 2 X, where y 6= x. Taken
r = d (y; x) (9.10)
with r > 0, because x 6= y. We are left with showing that B (y; r) Xn fxg, which is true because of the
(9:10)
following argument. Suppose otherwise; then x 2 B (y; r), i.e., r = d (y; x) < r, a contradiction.

Remark 373 From Example 360, any set in any discrete metric space is open. Therefore, the complement
of each set is open, and therefore each set is then clopen.

De…nition 374 Let a metric space (X; d) and a set S X be given. x is an boundary point of S if
any open ball centered in x intersects both S and its complement in X, i.e.,

8r 2 R++ ; B (x; r) \ S 6= ? ^ B (x; r) \ S C 6= ?:

De…nition 375 The set of all boundary points of S is called the Boundary of S and it is denoted by F (S).

Exercise 376 F (S) = F S C .

Exercise 377 F (S) is a closed set.

De…nition 378 The closure of S, denoted by Cl (S) is the intersection of all closed sets containing S, i.e.,
Cl (S) = \S 0 2S S 0 where S := fS 0 X : S 0 is closed and S 0 Sg.

Proposition 379 1. Cl (S)is a closed set;

2. S is closed , S = Cl (S).
9.2. OPEN AND CLOSED SETS 117

Proof. 1.
It follows from the de…nition and Proposition 368.
2.
[(]
It follows from 1. above.
[)]
Since S is closed, then S 2 S. Therefore, Cl (S) = S \ (\S 0 2S S 0 ) = S.

De…nition 380 x 2 X is an accumulation point for S X if any open ball centered at x contains points
of S di¤ erent from x, i.e., if
8r 2 R++ ; (Sn fxg) \ B (x; r) 6= ?
The set of accumulation points of S is denoted by D (S) and it is called the Derived set of S.

De…nition 381 x 2 X is an isolated point for S X if x 2 S and it is not an accumulation point for S,
i.e.,
x 2 S and 9r 2 R++ such that (Sn fxg) \ B (x; r) = ?;
or
9r 2 R++ such that S \ B (x; r) = fxg :
The set of isolated points of S is denoted by Is (S).

Proposition 382 D (S) = fx 2 Rn : 8r 2 R++ ; S \ B (x; r) has an in…nite cardinalityg.

Proof. [ ]
Suppose otherwise, i.e., x is an accumulation point of S and 9r 2 R++ such that S \ B (x; r) =
fx1 ; :::; xn g.Then de…ned := min fd (x; xi ) : i 2 f1; :::; ngg, (Sn fxg) \ B x; 2 = ?, a contradiction.
[ ]
Since S \ B (x; r) has an in…nite cardinality, then (Sn fxg) \ B (x; r) 6= ?.

9.2.1 Sets which are open or closed in metric subspaces.

Remark 383 1. [0; 1) is ([0; 1) ; d2 ) open.
2. [0; 1) is not (R; d2 ) open. We want to show

: 8x0 2 [0; 1) ; 9r 2 R++ such that B(R;d2 ) (x0 ; r) = (x0 r; x0 + r) [0; 1) ; (9.11)

i.e.,

9x0 2 [0; 1) such that 8r 2 R++ ; 9x0 2 R such that x0 2 (x0 r; x0 + r) and x0 2
= [0; 1) :

It is enough to take x0 = 0 and x0 = 2r .

3. Let ([0; +1) ; d2 ) be given. [0; 1) is ([0; +1) ; d2 ) open, as shown below. By de…nition of open set,
- go back to De…nition 356 and read it again - we have that, given the metric space ((0; +1) ; d2 ), [0; 1) is
open if
n o
8x0 2 [0; 1), 9r 2 R++ such that B([0;+1);d2 ) (x0 ; r) := x 2 [0; +1) : d (x0 ; x) < r = [0; r) [0; 1) :

If x0 2 (0; 1), then take r = min fx0 ; 1 x0 g > 0.

If x0 = 0, then take r = 21 . Therefore, we have B(R+ ;d2 ) 0; 12 = x 2 R+ : jx 0j < 1
2 = 0; 12 [0; 1).

Remark 384 1. (0; 1) is ((0; 1) ; d2 ) closed.

2. (0; 1] is ((0; +1) ; d2 ) closed, simply because (1; +1) is open.

Proposition 385 Let a metric space (X; d), a metric subspace (Y; d) of (X; d) and a set S Y be given.

S is open in (Y; d) , there exists a set O open in (X; d) such that S = Y \ O:

118 CHAPTER 9. METRIC SPACES

Proof. Preliminary remark.

8x0 2 Y; 8r 2 R++ ;
(9.12)
B(Y;d) (x0 ; r) := fx 2 Y : d (x0 ; x) < rg = Y \ fx 2 X : d (x0 ; x) < rg = Y \ B(X;d) (x0 ; r) :

[)]
Taken x0 2 S, by assumption 9rx0 2 R++ such that B(Y;d) (x0 ; r) S Y . Then

(9:12) distributive laws

S = [x0 2S B(Y;d) (x0 ; r) = [x0 2S Y \ B(X;d) (x0 ; r) = Y \ [x0 2S B(X;d) (x0 ; r) ;

and the it is enough to take O = [x0 2S B(X;d) (x0 ; r) to get the desired result.
[(]
Take x0 2 S. then, x0 2 O, and, since, by assumption, O is open in (X; d) ; 9r 2 R++ such that
B(X;d) (x0 ; r) O. Then

(9:12)
B(Y;d) (x0 ; r) = Y \ B(X;d) (x0 ; r) O \ Y = S;

where the last equality follows from the assumption. Summarizing, 8x0 2 S; 9r 2 R++ such that B(Y;d) (x0 ; r)
S, as desired.

Corollary 386 Let a metric space (X; d), a metric subspace (Y; d) of (X; d) and a set S Y be given.
1.
hS closed in (Y; d)i , hthere exists a set C closed in (X; d) such that S = Y \ C:i :
2.
)
hS open (respectively, closed) in (X; d)i hS open (respectively, closed) in (Y; d)i :
:
3. If Y is open (respectively, closed) in X,

hS open (respectively, closed) in (X; d)i ( hS open (respectively, closed) in (Y; d)i :

i.e., “the implication ( in the above statement 2. does hold true”.

Proof. 1.
def. Prop. 385
hS closed in (Y; d)i , hY nS open in (Y; d)i ,
, hthere exists an open set S 00 in (X; d) such that Y nS = S 00 \ Y i ,
, hthere exists a closed set S 0 in (X; d) such that S = S 0 \ Y i ;
where the last equivalence is proved below;
[(]
Take S 00 = XnS 0 , open in (X; d) by de…nition. We want to show that
if S 00 = XnS, S = S 0 \ Y and Y X, then Y nS = S 00 \ Y :

x 2 Y nS if f x2Y ^ x2 =S
x2Y ^ (x 2= S0 \ Y )
x2Y ^ (: (x 2 S 0 \ Y ))
x2Y ^ (: (x 2 S 0 ^ x 2 Y ))
x2Y ^ ((x 2= S0 _ x 2= Y ))
(x 2 Y ^x2 = S 0 ) _ ((x 2 Y ^ x 2
= Y ))
x2Y ^x2 = S0
x 2 S 00 \ Y if f x 2 Y ^ x 2 S 00
= S0)
x 2 Y ^ (x 2 X ^ x 2
= S0
(x 2 Y ^ x 2 X) ^ x 2
0
x2Y ^x2 =S
[)]
Take S 0 = XnS:Then S 0 is closed in (X; d). We want to show that
if T 0 = XnS 00 ; Y nS = S 00 \ Y; Y X, then S = S 0 \ Y .
9.3. SEQUENCES 119

Observe that we want to show that Y nS = Y n (S 0 \ Y ) ;or from the assumptions, we want to show that

S 00 \ Y = Y n ((XnS 00 ) \ Y ) :

x 2 Y n ((XnS 00 ) \ Y ) if f x 2 Y ^ (: (x 2 XnS 00 ^ x 2 Y ))
x 2 Y ^ (x 2 = XnS 00 _ x 2= Y)
x 2 Y ^ (x 2 S 00 _ x 2= Y)
(x 2 Y ^ x 2 S 00 ) _ (x 2 Y ^ x 2
= Y)
x 2 Y ^ x 2 S 00
x 2 S 00 \ Y
2. and 3.
Exercises.

9.3 Sequences
Unless otherwise speci…ed, up to the end of the chapter, we assume that

X is a metric space with metric d,

and
Rn is the metric space with Euclidean metric.

De…nition 387 A sequence in X is a function x : N ! X.

Usually, for any n 2 N, the value x (n) is denoted by xn ;which is called the n-th term of the sequence;
the sequence is denoted by (xn )n2N .

De…nition 388 Given a nonempty set X, X 1 is the set of sequences (xn )n2N such that 8n 2 N, xn 2 X.

De…nition 389 A strictly increasing sequence of natural numbers is a sequence (kn )n2N in N such

1 k1 < k2 < ::: < kn < :::

De…nition 390 A subsequence of a sequence (xn )n2N is a sequence (yn )n2N such that there exists a strictly
increasing sequence (kn )n2N of natural numbers such that 8n 2 N, yn = xkn .

De…nition 391 A sequence (xn )n2N 2 X 1 is said to be (X; d) convergent to x0 2 X (or convergent to
x0 2 X with respect to the metric space (X; d) ) if

8" > 0; 9n0 2 N such that 8n > n0 ; d (xn ; x0 ) < " (9.13)

x0 is called the limit of the sequence (xn )n2N and we write

n
limn!+1 xn = x0 , or xn ! x0 : (9.14)

(xn )n2N in a metric space (X; d) is convergent if there exist x0 2 X such that (9:13) holds. In that case,
we say that the sequence converges to x0 and x0 is the limit of the sequence.3

Remark 392 A more precise, and heavy, notation for (9:14) would be
n
limn!+1 xn = x0 or xn ! x0
(X;d) (X;d)

1
Remark 393 Observe that n n2N+ converges with respect to (R; d2 ) and it does not converge with respect
to (R++ ; d2 ) .

Proposition 394 limn!+1 xn = x0 , limn!+1 d (xn ; x0 ) = 0.

3 For the last sentence in the De…nition, see, for example, Morris (2007), page 121.
120 CHAPTER 9. METRIC SPACES

Proof. Observe that we can de…ne the sequence (d (xn ; x0 ))n2N in R. Then from de…nition 391, we have
that limn!+1 d (xn ; x0 ) = 0 means that

8" > 0; 9n0 2 N such that 8n > n0 ; jd (xn ; x0 ) 0j < ":

Remark 395 Since (d (xn ; x0 ))n2N is a sequence in R, all well known results hold for that sequence. Some
of those results are listed below.

Proposition 396 (Some properties of sequences in R).

All the following statements concern sequences in R.
1. Every convergent sequence is bounded.
2. Every increasing (decreasing) sequence that is bounded above (below) converges to its sup (inf).
3. Every sequence has a monotone subsequence.
4. (Bolzano-Weierstrass 1) Every bounded sequence has a convergent subsequence.
5. (Bolzano-Weierstrass 2) Every sequence contained in a closed and bounded set has a convergent
subsequence in the set.
Moreover, suppose that (xn )n2N and (yn )n2N are sequences in R and limn!1 xn = x0 and limn!+1 yn =
y0 . Then
6. limn!+1 (xn + yn ) = x0 + y0 ;
7. limn!+1 xn yn = x0 y0 ;
8. if 8n 2 N, xn 6= 0 and x0 6= 0, limn!+1 x1n = x10 ;
9. if 8n 2 N, xn yn , then x0 y0 ;
10. Let (zn )n2N be a sequence such that 8n 2 N, xn zn yn , and assume that x0 = y0 . Then
limn!+1 zn = x0 .

Proof. See Villanacci, (in progress), Basic Facts on sequences, series and integrals in R, mimeo.

Proposition 397 If (xn )n2N converges to x0 and (yn )n2N is a subsequence of (xn )n2N , then (yn )n2N con-
verges to x0 .

Proof. By de…nition of subsequence, there exists a strictly increasing sequence (kn )n2N of natural
numbers, i.e., 1 < k1 < k2 < ::: < kn < :::, such that 8n 2 N, yn = xkn .
If n ! +1, then kn ! +1. Moreover, 8n, 9kn such that

d (x0 ; xkn ) = d (x0 ; yn )

Taking limits of both sides for n ! +1, we get the desired result.

Proposition 398 A sequence in (X; d) converges at most to one element in X.

n n
Proof. Assume that xn ! p and xn ! q; we want to show that p = q. From the Triangle inequality,

8n 2 N; 0 d (p; q) d (p; xn ) + d (xn ; q) (9.15)

Since d (p; xn ) ! 0 and d (xn ; q) ! 0, Proposition 396.10 and (9:15) imply that d (p; q) = 0 and therefore
p = q.

k
Proposition 399 Given a sequence (xn )n2N = xin i=1 n2N
in Rk ,
D E
(xn )n2N Rk converges to x , 8i 2 f1; :::; kg ; xin n2N
R converges to xi ;

and
n
lim xn = lim xin :
n!+1 n!+1
i=1
9.3. SEQUENCES 121

Proof. [)]
Observe that q
2
xin xi = (xin xi ) d (xn x) :
Then, the result follows.
[(]
By assumption, 8" > 0 and 8i 2 f1; :::; kg ; there exists n0 such that 8n > n0 , we have xin xi < p" .
k
Then 8n > n0 ,
k
! 21 k
!1 k
! 12
X 2 X "
2 2 X 1
d (xn x) = xin xi < p = "2 = ":
i=1 i=1
k i=1
k

Proposition 400 Suppose that (xn )n2N and (yn )n2N are sequences in Rk and limn!1 xn = x0 and limn!+1 yn =
y0 . Then

1. limn!+1 (xn + yn ) = x0 + y0 ;
2. 8c 2 R, limn!+1 c xn = c x0 ;
3. limn!+1 xn yn = x0 y0 .

Proof. It follows from Propositions 396 and 399.

Example 401 In Proposition 340 , we have seen that (B ([0; 1]) ; d1 ) is a metric space. Observe that de…ned
8n 2 N,
fn : [0; 1] ! R; t 7! tn ;
1
we have that (fn )n 2 B ([0; 1]) . Moreover, 8t 2 [0; 1], fn t n2N 2 R1 and it converges in (R; d2 ). In
fact, 8
< 0 if t 2 [0; 1)
n
lim t =
n!+1 :
1 if t = 1:
De…ne 8
< 0 if t 2 [0; 1)
f : [0; 1] ! R; t 7! :
:
1 if t = 1:

y 1

0.75

0.5

0.25

0
0 0.25 0.5 0.75 1

We want to check that it is false that

fm ! f;
(B([0;1]);d1 )

i.e., it is false that d1 (fm ; f ) ! 0. Then, we have to check

m

: 8" > 0 9N" 2 N such that 8n > N" , d1 (fn ; f ) < " ;
122 CHAPTER 9. METRIC SPACES

i.e.,
9" > 0 such that 8N" 2 N; 9n > N" such that d1 (fn ; f ) " :
Then, taken " = 14 , it su¢ ce to show that
1
8m 2 N; 9t 2 (0; 1) such that fm t f t 4 :
1 m
It is then enough to take t = 2 :

Exercise 402 For any metric space (X; d) and (xn )n2N 2 X 1 ,
* +
xn ! x , xn ! x :
(X;d) (X; 1+d
1
)

9.4 Sequential characterization of closed sets

Proposition 403 Let (X; d) be a metric space and S X.4

S is closed , any (X; d) convergent sequence (xn )n2N 2 S 1 converges to an element of S :

Proof. We want to show that

S is closed ,

(xn )n2N is such that 1. 8n 2 N; xn 2 S; and

, ) 3. x0 2 S :
2. xn ! x0

[)]
We are going to show the contrapositive of the desired statement. 5 Therefore, we assume that there
exists a sequence (xn )n2N 2 S 1 such that xn ! x0 and x0 2 = S. We want to show that S is not closed, i.e.,
X n S is not open, i.e., there exists x 2 X n S such that for any r > 0, we do have B (x; r) \ S 6= ?. Indeed,
take x = x0 . Since xn ! x0 , then for any r > 0 there exists Nr 2 N such that for any n > Nr , we have
xn 2 B (x0 ; r). Since for any n 2 N, xn 2 S, we have that for any n > Nr , xn 2 B (x0 ; r) \ S, as desired.
[(]
Suppose otherwise, i.e., S is not closed. Then, XnS is not open. Then, 9 x 2 XnS such that 8n 2 N,
9xn 2 X such that xn 2 B x; n1 \ S, i.e.,
i. x 2 XnS
ii. 8n 2 N, xn 2 S,
iii. d (xn ; x) < n1 , and therefore xn ! x,
and i., ii. and iii. contradict the assumption.

Remark 404 The Appendix to this chapter contains some other characterizations of closed sets and sum-
marizes all the presented characterizations of open and closed sets.

9.5 Compactness
De…nition 405 Let (X; d) be a metric space, S a subset of X, and be a set of arbitrary cardinality. A
family S = fS g 2 such that 8 2 , S is (X; d) open, is said to be an open cover of S if S [ 2 S .
A subfamily S 0 of S is called a subcover of S if S [S 0 2S 0 S 0 .

De…nition 406 A metric space (X; d) is compact if every open cover of X has a …nite subcover.
A set S X is compact in X if every open cover of S has a …nite subcover of S.

Example 407 Any …nite set in any metric space is compact.

n
Take S = fxi gi=1 in (X; d) and an open cover S of S. For any i 2 f1; :::; ng, take an open set in S which
contains xi ; call it Si . Then S 0 = fSi : i 2 f1; :::; ngg is the desired open subcover of S.
4 Proposition 467 in Appendix 9.8.1 presents a di¤erent proof of the result below.
5 Recall that the contrapositive of the statement “if p, then q” is “if not q, then not p.”
9.5. COMPACTNESS 123

Example 408 1. (0; 1) is not compact in (R; d2 ).

We want to show that the following statement is true:

: h8S such that [S2S S (0; 1) ; 9S 0 S such that #S 0 is …nite and [S2S 0 S (0; 1)i ;

i.e.,

9S such that [S2S S (0; 1) and 8S 0 S either #S 0 is in…nite or [S2S 0 S + (0; 1) :

1
Take S = n; 1 n2Nnf0;1g
and S 0 any …nite subcover of S:Then there exists a …nite set N such that
S 0 = n1 ; 1 n2N . Take n = max fn 2 N g. Then, [S2S 0 S = [n2N n1 ; 1 = n1 ; 1 and n1 ; 1 + (0; 1).
2. (0; 1] is not compact in ((0; +1) ; d2 ). Take S = n1 ; 2 n2N and S 0 any …nite subcover of S:Then there
exists a …nite set N such that S 0 = n1 ; 2 n2N . Take n = max fn 2 N g. Then, [S2S 0 S = [n2N n1 ; 2 =
1 1
n ; 2 and n ; 2 + (0; 1].

Proposition 409 Let (X; d) be a metric space.

X compact and C X closed ) hC compacti :

Proof. Take an open cover S of C. Then S [ (XnC) is an open cover of X. Since X is compact, then
there exists an open covers S 0 of S [ (XnC) which cover X. Then S 0 n fXnCg is a …nite subcover of S which
covers C.

9.5.1 Compactness and bounded, closed sets

De…nition 410 Let (X; d) be a metric space and a nonempty subset S of X. S is bounded in (X; d) if
9r 2 R++ such that 8x; y 2 S, d (x; y) < r.

Proposition 411 Given a metric space (X; d) and a nonempty subset S of X, then

S is bounded , 9r 2 R++ and 9z 2 X such that S B (z; r ).

Proof. [)] Take r = r and an arbitrary point z in S X.

[)] Take x; y 2 S. Then
d (x; y) d (x; z) + d (z; y) < 2r:
Then it is enough to take r = 2r.

Exercise 412 Show that

1.given S (X; d), then

S is bounded , 8z 2 X 9rz 2 R++ such that S B (z; rz );

2. given, S Rn , then

S is bounded , 9x; x 2 Rn such that for any x 2 S and for any i 2 f1; :::; ng, xi < xi < xi .

Proposition 413 The …nite union of bounded set is bounded.

n
Proof. Take n 2 N and fSi gi=1 such that 8i 2 f1; :::; ng, Si is bounded. Then, 8i 2 f1; :::; ng, 9r 2 R++
n
such that Si B (z; ri ). Take r = max fri gi=1 . Then [ni=1 Si [ni=1 B (z; ri ) B (z; r).

Proposition 414 Let (X; d) be a metric space and S a subset of X.

S compact ) S bounded.

Proof. If S = ?, we are done. Assume then that S 6= ?, and take x 2 S and B = fB (x; n)gn2N . B is
an open cover of X and therefore of S. Then, there exists B 0 B such that

B 0 = fB (x; ni )gi2N ;

where N is a …nite set and B 0 covers S.

Then takes n = maxi2N ni , we get S B (x; n ) as desired.
124 CHAPTER 9. METRIC SPACES

Proposition 415 Let (X; d) be a metric space and S a subset of X.

S compact ) S closed.

Proof. If S = X, we are done by Proposition 368. Assume that S 6= X: we want to show that XnS is
open. Take y 2 S and x 2 XnS. Then, taken ry 2 R such that

1
0 < ry < d (x; y) ;
2
we have
B (y; ry ) \ B (x; ry ) = ?:
Now, S = fB (y; ry ) : y 2 Sg is an open cover of S, and since S is compact, there exists a …nite subcover
S 0 of S which covers S, say
S 0 = fB (yn ; rn )gn2N ;
such that N is a …nite set. Take
r = min rn ;
n2N

and therefore r > 0. Then 8n 2 N ,

B (yn ; rn ) \ B (x; rn ) = ?;

B (yn ; rn ) \ B (x; r ) = ?;
and
([n2N B (yn ; rn )) \ B (x; r ) = ?:
Since fB (yn ; rn )gn2N covers S, we then have

S \ B (x; r ) = ?;

or
B (x; r ) XnS:
Therefore, we have shown that

8x 2 XnS; 9r 2 R++ such that B (x; r ) XnS;

i.e., XnS is open and S is closed.

Remark 416 Summarizing, we have seen that in any metric space

S compact ) S bounded and closed.

The opposite implication is false. In fact, the following sets are bounded, closed and not compact.
1. Let the metric space ((0; +1) ; d2 ). (0; 1] is closed from Remark 384, it is clearly bounded and it is
not compact from Example 408.2 .
2. (X; d) where X is an in…nite set and d is the discrete metric.
X is closed, from Remark 373 .
X is bounded: take x 2 X and r = 2 :
X is not compact. Take S = fB (x; 1)gx2X . Then 8x 2 X there exists a unique element Sx in S such
that x 2 Sx . 6

Remark 417 In next section we are going to show that if (X; d) is an Euclidean space with the Euclidean
distance and S X, then
S compact ( S bounded and closed.
6 For other examples, see among others, page 155, Ok (2007).
9.5. COMPACTNESS 125

9.5.2 Sequential compactness

De…nition 418 Let a metric space (X; d) be given. S X is sequentially compact if every sequence of
elements of S has a subsequence which converges to an element of S, i.e.,

(xn )n2N is a sequence in S ) 9 a subsequence (yn )n2N of (xn )n2N such that yn ! x 2 S :

In what follows, we want to prove that in metric spaces, compactness is equivalent to sequential com-
pactness. To do that requires some work and the introduction of some, useful in itself, concepts.

Proposition 419 (Nested intervals) For every n 2 N, de…ne In = [an ; bn ] R such that In+1 In . Then
\n2N In 6= ?:

Proof. By assumption,
a1 a2 ::: an ::: (9.16)
and
:::bn bn 1 ::: b1 (9.17)
Then,
8m; n 2 N; am < bn
simply because, if m > n, then am < bm bn ;where the …rst inequality follows from the de…nition of
interval Im and the second one from (9:17), and if m n, then am an bn ;where the …rst inequality
follows from (9:16) and the second one from the de…nition of interval In .
Then A := fan : n 2 Ng is nonempty and bounded above by bn for any n:Then sup A := s exists.
Since 8n 2 N, bn is an upper bound for A,

8n 2 N; s bn

and from the de…nition of sup;

8n 2 N; an s
Then
8n 2 N; an s bn
and
8n 2 N; In 6= ?:

Remark 420 The statement in the above Proposition is false if instead of taking closed bounded intervals
we take either open or unbounded intervals. To see that consider In = 0; n1 and In = [n; +1].

Proposition 421 (Bolzano- Weirstrass) If S Rn has in…nite cardinality and is bounded, then S admits
at least an accumulation point, i.e., D (S) 6= ?.

Proof. Step 1. n = 1:
Since S is bounded, 9a0 ; b0 2 R such that S [a0 ; b0 ] := B0 :Divide B0 in two subinterval of equal length:

a0 + b0 a0 + b0
a0 ; and ; b0
2 2
Choose an interval which contains an in…nite number of points in S. Call B1 = [a1 ; b1 ] that interval.
Proceed as above for B1 . We therefore obtain a family of intervals

B0 B1 :::: Bn :::

Observe that lenght B0 := b0 a0 and

b0 a0
8n 2 N; lenght Bn = :
2n
Therefore, 8" > 0; 9 N 2 N such that 8n > N; lenght Bn < ".
126 CHAPTER 9. METRIC SPACES

From Proposition 419, it follows that

9x 2 \+1
n=0 Bn

We are now left with showing that x is an accumulation point for S

8r 2 R++ ; B (x; r) contains an in…nite number of points.

By construction, 8n 2 N; Bn contains an in…nite number of points; it is therefore enough to show that

8r 2 R++ ; 9n 2 N such that B (x; r) Bn .

Observe that

B (x; r) Bn , (x r; x + r) [an ; bn ] , x r < an < bn < x + r , max fx an ; bn xg < r

Moreover, since x 2 [an ; bn ],

b0 a0
max fx an ; bn xg < bn an = lenght Bn =
2n
Therefore, it su¢ ces to show that

b0 a0
8r 2 R++ ; 9n 2 N such that <r
2n
i.e., n 2 N and n > log2 (b0 a0 ).
Step 2. Omitted (See Ok (2007)).

Remark 422 The above Proposition does not say that there exists an accumulation point which belongs to
S. To see that, consider S = n1 : n 2 N .

Proposition 423 Let a metric space (X; d) be given and consider the following statements.

1. S is compact set;

2. Every in…nite subset of S has an accumulation point which belongs to S, i.e.,

hT S ^ #T is in…nitei ) hD (T ) \ S 6= ?i ;

3. S is sequentially compact

4. S is closed and bounded.

Then
1: , 2: , 3 ) 4:
If X = Rn , d = d2 , then we also have that
3 ( 4:
More precisely, S is (Rn ; d2 ) compact , S is (Rn ; d2 ) closed and bounded.

Proof. (1) ) (2)7

Take an in…nite subset T S and suppose otherwise. Then, no point in S is an accumulation point of
T , i.e., 8x 2 S 9rx > 0 such that
B (x; rx ) \ T n fxg = ?:
Then8
B (x; rx ) \ T fxg : (9.19)
7 Proofs of 1 ) 2 and 2 ) 3 are taken from Aliprantis and Burkinshaw (1990), pages 38-39.
8 In general,
AnB = C ) A C [ B; (9.18)
as shown below.
9.5. COMPACTNESS 127

Since
S [x2S B (x; rx )
and S is compact, 9x1 ; :::; xn such that

S [ni=1 B (xi ; ri )

Then, since T S,

T =S\T ([ni=1 B (xi ; ri )) \ T = [ni=1 (B (xi ; ri ) \ T ) fx1 ; :::; xn g

where the last inclusion follows from (9:19). But then #T n, a contradiction.
(2) ) (3)
Take a sequence (xn )n2N of elements in S.
If # fxn : n 2 Ng is …nite, then 9xn such that xj = xn for j in an in…nite subset of N, and (xn ; :::; xn ; :::)
is the required convergent subsequence - converging to xn 2 S.
If # fxn : n 2 Ng is in…nite, then there exists a subsequence (yn )n2N of (xn )n2N with an in…nite amount
distinct values, i.e., such that 8n; m 2 N; n 6= m, we have yn 6= ym . To construct the subsequence (yn )n2N ,
proceed as follows.
y1 = x1 := xk1 ,
y2 = xk2 2= fxk1 g,
y3 = xk3 2= fxk1 ; xk2 g,
...
yn = xkn 2 = xk1 ; xk2 ; :::xkn 1 ,
...
Since T := fyn : n 2 Ng is an in…nite subset of S, by assumption it does have an accumulation point x in
S; moreover, we can rede…ne (yn )n2N in order to have 8n 2 N, yn 6= x 9 , as follows. If 9k such that yk = x,
take the (sub)sequence (yk+1 ; yk+2 ; :::) = (yk+n )n2N . With some abuse of notation, call still (yn )n2N the
sequence so obtained. Now take a further subsequence as follows, using the fact that x is an accumulation
point of fyn : n 2 Ng := T ,
ym1 2 T such that d (ym1 ; x) < 11 , n o
1
ym2 2 T such that d (ym2 ; x) < min ; (d (ym ; x))m m1 ;
n2 o
1
ym3 2 T such that d (ym3 ; x) < min 3 ; (d (ym ; x))m m2
;
... n o
ymn 2 T such that d (ymn ; x) < min n1 ; (d (ym ; x))m mn 1 ;
n o
Observe that since 8n; d (ymn ; x) < min (d (ym ; x))m mn 1 , we have that 8n; mn > mn 1 and therefore
(ymn )n2N is a subsequence of (yn )n2N and therefore of (xn )n2N . Finally, since

1
lim d (ymn ; x) < lim =0
n!+1 n!+1 n
we also have that
lim ymn = x
n!+1

Since AnB = A \ B C , by assumption, we have

A \ BC [ B = C [ B

Moreover,
A \ B C [ B = (A [ B) \ B C [ B = A [ B A
Observe that the inclusion in (9:18) can be strict, i.e., it can be
AnB = C ^ A C [ B;
just take A = f1g ; B = f2g and C = f1g :

AnB = f1g = C ^ A = f1g C [ B = f1; 2g :;

9 Below we need to have d (yn ; x) > 0.

128 CHAPTER 9. METRIC SPACES

as desired.
(3) ) (1)
It is the content of Proposition 432 below.
(1) ) (4)
It is the content of Remark 416.
If X = Rn , (4) ) (2)
Take an in…nite subset T S. Since S is bounded T is bounded as well. Then from Bolzano-Weirestrass
theorem, i.e., Proposition 421, D (T ) 6= ?. Since T S, from Proposition 458, D (T ) D (S) and since S is
closed, D (S) S. Then, summarizing ? 6= D (T ) S and therefore D (T ) \ S = D (T ) 6= ?, as desired.
To complete the proof of the above Theorem it su¢ ces to show sequential compactness implies compact-
ness, which is done below, and it requires some preliminary results.

De…nition 424 Let (X; d) be a metric space and S a subset of X. S is totally bounded if 8" > 0; 9 a …nite
set T S such that S [x2T B (x; ").

Proposition 425 Let (X; d) be a metric space and S a subset of X.

S totally bounded ) S bounded.

Proof. It follows from the de…nition of totally bounded sets and from Proposition 413.

Remark 426 In the previous Proposition, the opposite implication does not hold true.

Example 427 Take (X; d) where X is an in…nite set and d is the discrete metric. Then, if " = 21 , a ball
is needed to “take care of each element in X” . Similar situation arises in the following probably more
interesting example.

Example 428 Consider the metric space l2 ; d2 - see Proposition 336. Recall that
( +1
)
X 2
2 1
l = (xn )n2N 2 R : jxn j < +1
n=1

and
+1
! 12
X 2
2 2
d2 : l l ! R+ ; (xn )n2N ; (yn )n2N 7! jxn yn j :
n=1

De…ne em = (em;n )n2N such that

8
< 1 if n=m
em;n =
:
0 if n 6= m;

and S = fem : m 2 Ng. In other words, S =

f(1; 0; 0; :::; 0; :::) ; (0; 1; 0; :::; 0; :::) ; (0; 0; 1; :::; 0; :::) ; :::g :
P+1 2
Observe that 8m 2 N, n=1 jem;n j = 1 and therefore S l2 . We now want to check that S is bounded,
but not totally bounded. The main ingredient of the argument below is that
p
8m; p 2 N such that m 6= p; d (em ; ep ) = 2: (9.20)
1
P+1 2 2 1
1. S is bounded. For any m 2 N, d (e1 ; em ) = n=1 je1;n ; em;n j = 22 :
2. S is not totally bounded. We want to show that 9" > 0 such that for any …nite subset T of S there
exists x 2 S such that x 2= [x2T B (x; "). Take " = 1and let T = fek : k 2 N g with N arbitrary …nite
p subset
of N. Then, for k 0 2 NnN , ek0 2 S and from (9:20) ; for any k 0 2 NnN and k 2 N , d (ek ; ek0 ) = 2 > 1.
Therefore, for k 0 2 NnN , ek0 2
= [k2N B (ek ; 1) :

Remark 429 In (Rn ; d2 ), S bounded ) S totally bounded.

9.5. COMPACTNESS 129

Lemma 430 Let (X; d) be a metric space and S a subset of X.

S sequentially compact ) S totally bounded.

Proof. Suppose otherwise, i.e., 9" > 0 such that for any …nite set T S, S " [x2T B (x; "). We are
now going to construct a sequence in S which does not admit any convergent subsequence, contradicting
sequential compactness.
Take an arbitrary
x1 2 S:
Then, by assumption S " B (x1 ; "). Then take x2 2 SnB (x1 ; "), i.e.,

x2 2 S and d (x1 ; x2 ) > ":

By assumption, S " B (x1 ; ") [ B (x2 ; "). Then, take x3 2 Sn (B (x1 ; ") [ B (x2 ; ")), i.e.,

x3 2 S and for i 2 f1; 2g ; d (x3 ; xi ) > ":

By the axiom of choice, we get that

8n 2 N; xn 2 S and for i 2 f1; :::; n 1g ; d (xn ; xi ) > ":

Therefore, we have constructed a sequence (xn )n2N 2 S 1 such that

8i; j 2 N; if i 6= j, then d (xi ; xj ) > ": (9.21)

But, then it is easy to check that (xn )n2N does not have any convergent subsequence in S, as veri…ed
below. Suppose otherwise, then (xn )n2N would admit a subsequence (xm )m2N 2 S 1 such that xm ! x 2 S.
But, by de…nition of convergence, 9N 2 N such that 8m > N; d (xm ; x) < 2" , and therefore

d (xm ; xm+1 ) d (xm ; x) + d (xm+1 ; x) < ";

contradicting (9:21).

Lemma 431 Let (X; d) be a metric space and S a subset of X.

S sequentially compact
) 9" > 0 such that 8x 2 S; 9Ox 2 S such that B (x; ") Ox :
S is an open cover of S

Proof. Suppose otherwise; then

1
8n 2 N+ ; 9xn 2 S such that 8O 2 S; B xn ; * O: (9.22)
n

By sequential compactness, the sequence (xn )n2N 2 S 1 admits a subsequence, without loss of generality
the sequence itself, (xn )n2N 2 S 1 such that xn ! x 2 S. Since S is an open cover of S, 9 O 2 S such that
x 2 O and, since O is open, 9" > 0 such that

B (x; ") O: (9.23)

Since xn ! x, 9M 2 N such thatfxM +i ; i 2 Ng B x; 2" . Now, take n > max M; 2" . Then,

1
B xn ; B (x; ") : (9.24)
n
1
i.e., d (y; xn ) < n ) d (y; x) < ", as shown below.

1 "
d (y; x) d (y; xn ) + d (xn ; x) < + < ":
n 2
From (9:23) and (9:24), we get B xn ; n1 O 2 S, contradicting (9:22).
130 CHAPTER 9. METRIC SPACES

Proposition 432 Let (X; d) be a metric space and S a subset of X.

S sequentially compact ) S compact.
Proof. Take an open cover S of S. Since S is sequentially compact, from Lemma 431,
9" > 0 such that 8x 2 S 9Ox 2 S such that B (x; ") Ox :
Moreover, from Lemma 430 and the de…nition of total boundedness, there exists a …nite set T S such
that S [x2T B (x; ") [x2T Ox . But then fOx : x 2 T g is the required subcover of S which covers S.
We conclude our discussion on compactness with some results we hope will clarify the concept of com-
pactness in Rn :
Proposition 433 Let X be a proper subset of Rn , and C a subset of X.
C is bounded and (Rn ; d2 ) closed
(m 1)
C is (Rn ; d2 ) compact
(m 2)
C is (X; d2 ) compact
+ (not *) 3
C is bounded and (X; d2 ) closed
Proof. [1 m]
It is the content of (Propositions 414, 415 and last part of) Proposition 423.
To show the other result, observe preliminarily that
(X \ S ) [ (X \ S ) = X \ (S [ S )
[2 +]
Take T := fT g 2A such that 8 2 A; T is (X; d) open and C [ 2A T . From Proposition 385,
8 2 A; 9 S such that S is (Rn ; d2 ) open and T = X \ S :
Then
C [ 2A T [ 2A (X \ S ) = X \ ([ 2A S ):
We then have that
C [ 2A S ;
i.e., S := fS g 2A is a (R ; d2 ) open cover of C and since C is (Rn ; d2 ) compact, then there exists a …nite
n

subcover fSi gi2N of S such that

C [i2N Si :
Since C X , we then have
C ([i2N Si ) \ X = [i2N (Si \ X) = [i2N Ti ;
i.e., fTi gi2N is a (X; d) open subcover of fT g 2A which covers C, as required.
[2 *]
Take S := fS g 2A such that 8 2 A; S is (Rn ; d2 ) open and C [ 2A S . From Proposition 385,
8 2 A; T := X \ S is (X; d) open:
Since C X , we then have
C ([ 2A S )\X =[ 2A (S \ X) = [ 2A T :
Then, by assumption, there exists fTi gi2N is an open subcover of fT g 2A which covers C, and therefore
there exists a set N with …nite cardinality such that
C [i2N Ti = [i2N (Si \ X) = ([i2N Si ) \ X ([i2N Si ) ;
n
i.e., fSi gi2N is a (R ; d2 ) open subcover of fS g 2A which covers C, as required.
[3 +]
It is the content of Propositions 414, 415.
[3 not *]
See Remark 416.1.
9.6. COMPLETENESS 131

Remark 434 The proof of part [2 m] above can be used to show the following result.
Given a metric space (X; d), a metric subspace (Y; d) a set C Y , then

C is (Y; d) compact
m
C is (X; d) compact

In other words, (X 0 ; d) compactness of C X 0 X is an intrinsic property of C: it does not depend by

the subspace X 0 you are considering. On the other hand, as we have seen, closedness and openness are not
an intrinsic property of the set.

Remark 435 Observe also that to de…ne “anyway” compact sets as closed and bounded sets would not be
a good choice. The conclusion of the extreme value theorem (see Theorem 522) would not hold in that
case. That theorem basically says that a continuous real valued function on a compact set admits a global
maximum. It is not the case that a continuous real valued function on a closed and bounded set admits a
global maximum: consider the continuous function
1
f : (0; 1] ! R; f (x) = :
x
The set (0; 1] is bounded and closed (in ((0; +1) ; d2 ) and f has no maximum on (0; 1].

9.6 Completeness
9.6.1 Cauchy sequences
De…nition 436 Let (X; d) be a metric space. A sequence (xn )n2N 2 X 1 is a Cauchy sequence if

8" > 0; 9N 2 N such that 8l; m > N; d (xl ; xm ) < ":

Proposition 437 Let a metric space (X; d) and a sequence (xn )n2N 2 X 1 be given.

1. (xn )n2N is convergent ) (xn )n2N is Cauchy, but not vice-versa;

2. (xn )n2N is Cauchy ) (xn )n2N is bounded;
3. (xn )n2N is Cauchy and it has a subsequence converging to x 2 X ) (xn )n2N is convergent to x 2 X.

Proof. 1.
[)] Since (xn )n2N is convergent, by de…nition, 9x 2 X such that xn ! x; 9N 2 N such that 8l; m > N ,
d (x; xl ) < 2" and d (x; xm ) < 2" . But then d (xl ; xm ) d (xl ; x) + d (x; xm ) < 2" + 2" = ":
[:]
1
Take X = (0; 1) ; d = absolute value, (xn )n2N 2 (0; 1) such that 8n 2 N; xn = n1 .
(xn )n2N is Cauchy:

1 1 1 1 1 1 1 1 " "
8" > 0; d ; = < + = + < + = ";
l m l m l m l m 2 2

where the last inequality is true if 1l < 2" and m

1
< 2" , i.e., if l > 2
" and m > 2" . Then, it is enough to take
2
N > " and N 2 N, to get the desired result.
(xn )n2N is not convergent to any point in (0; 1):
take any x 2 (0; 1). We want to show that

9" > 0 such that 8N 2 N 9n > N such that d (xn ; x) > ":
x 1 1 x
Take " = 2 > 0 and 8N 2 N , take n 2 N such that n < min N; 2 . Then, n > N , and

1 1 x x
x =x >x = = ":
n n 2 2
2.
132 CHAPTER 9. METRIC SPACES

Take " = 1. Then 9N 2 N such that 8l; m > N; d (xl ; xm ) < 1. If N = 1, we are done. If N 1, de…ne

r = max f1; d (x1 ; xN ) ; :::; d (xN 1 ; xN )g :

Then
fxn : n 2 Ng B (xN ; r) :
3.
Let (xnk )k2N be a convergent subsequence to x 2 X. Then,

d (xn ; x) d (xn ; xnk ) + d (xnk ; x) :

Since d (xn ; xnk ) ! 0, because the sequence is Cauchy, and d (xnk n ; x) ! 0, because the subsequence is
convergent, the desired result follows.

9.6.2 Complete metric spaces

De…nition 438 A metric space (X; d) is complete if every Cauchy sequence is a convergent sequence.

Remark 439 If a metric space is complete, to show convergence you do not need to guess the limit of the
sequence: it is enough to show that the sequence is Cauchy.
1
Example 440 ((0; 1) ; absolute value) is not a complete metric space; it is enough to consider n n2N .

Example 441 Let (X; d) be a discrete metric space. Then, it is complete. Take a Cauchy sequence
(xn )n2N 2 X 1 . Then, we claim that 9N 2 N and x 2 X such that 8n > N; xn = x. Suppose other-
wise:
8N 2 N; 9m; m0 > N such that xm 6= xm0 ;
but then d (xm ; xm0 ) = 1, contradicting the fact that the sequence is Cauchy.

Example 442 (Q; d2 ) is not a complete metric space. Since RnQ is dense in R, 8x 2 RnQ, we can …nd
(xn )n2N 2 Q1 such that xn ! x.

Proposition 443 Rk ; d2 is complete.

Proof. 1. (R; d2 ) is complete.

Take a Cauchy sequence (xn )n2N 2 R1 . Then, from Proposition 437.2, it is bounded. Then from
Bolzano-Weierstrass Theorem (i.e., Proposition 396.4), (xn )n2N does have a convergent subsequence - i.e.,
9 (yn )n2N 2 R1 which is a subsequence of (xn )n2N and such that yn ! a 2 R. Then from Proposition 437.3.
2. For any k 2; Rk ; d2 is complete.
1
Take a Cauchy sequence (xn )n2N 2 Rk For i 2 f1; :::; kg, consider xin n2N 2 R1 . Then, for any
n; m 2 N,
xin xim < kxn xm k :
Then, 8i 2 f1; :::; kg, xin n2N 2 R1 is Cauchy and therefore from 1. above, 8i 2 f1; :::; kg, xin n2N
is
convergent. Finally, from Proposition 399, the desired result follows.

Example 444 For any nonempty set T , (B (T ) ; d1 ) is a complete metric space.

1
Let (fn )n2N 2 (B (T )) be a Cauchy sequence. For any x 2 T , (fn (x))n2N 2 R1 is a Cauchy se-
quence, and since R is complete, it has a convergent subsequence, without loss of generality, (fn (x))n2N
itself converging say to fx 2 R. De…ne

f : T ! R; : x 7! fx :

We are going to show that (i). f 2 B (T ), and (ii) fn ! f .

(i). Since (fn )n is Cauchy,

8" > 0; 9N 2 N such that 8l; m > N; d1 (fl ; fm ) := sup jfl (x) fm (x)j < ":
x2T
9.6. COMPLETENESS 133

Then,
8x 2 T; jfl (x) fm (x)j sup jfl (x) fm (x)j = d1 (fl ; fm ) < ": (9.25)
x2T

Taking limits of both sides of (9:25) for l ! +1, and using the continuity of the absolute value function,
we have that
8x 2 T; lim jfl (x) fm (x)j = jf (x) fm (x)j < ": (9.26)
l!+1

Since10
8x 2 T; j jf (x)j jfm (x)j j j f (x) fm (x) j < ";

and therefore,
8x 2 T; jf (x)j fm (x) + ":

Since fl 2 B (T ), f 2 B (T ) as well.
(ii) From (9:26), we also have that

8x 2 T; jf (x) fm (x)j < ";

and by de…nition of sup

d1 (fm ; f ) := sup jfm (x) f (x)j < ";
x2T

i.e., d1 (fm ; f ) ! 0.

For future use, we also show the following result.

Proposition 445
BC (X) := ff : X ! R : f is bounded and continuousg

endowed with the metric d (f; g) = supx2X jf (x) g (x)j is a complete metric space.

Proof. See Stokey and Lucas (1989), page 47.

9.6.3 Completeness and closedness

Proposition 446 Let a metric space (X; d) and a metric subspace (Y; d) of (X; d) be given.

1. Y complete ) Y closed;

2. Y complete ( Y closed and X complete.

Proof. 1.
Take (xn )n2N 2 Y 1 such that xn ! x. From Proposition 403, it is enough to show that x 2 Y . Since
(xn )n2N is convergent in X, then it is Cauchy. Since Y is complete, by de…nition, xn ! x 2 Y .
2.
Take a Cauchy sequence (xn )n2N 2 Y 1 . We want to show that xn ! x 2 Y . Since Y X, (xn )n2N is
Cauchy in X, and since X is complete, xn ! x 2 X. But since Y is closed, x 2 Y:

Remark 447 An example of a metric subspace (Y; d) of (X; d) which is closed and not complete is the
following one. (X; d) = (R++ ; d2 ), (Y; d) = ((0; 1] ; d2 ) and (xn )n2N = n1 n2N .

Corollary 448 Let a complete metric space (X; d) and a metric subspace (Y; d) of (X; d) be given. Then,

Y complete , Y closed.
1 0 See, for example, page 37 in Ok (2007).
134 CHAPTER 9. METRIC SPACES

9.7 Fixed point theorem: contractions

De…nition 449 Let (X; d) be a metric space. A function ' : X ! X is said to be a contraction if

9k 2 (0; 1) such that 8x; y 2 X; d ( (x) ; (y)) k d (x; y) :

The inf of the set of k satisfying the above condition is called contraction coe¢ cient of .

Example 450 1. Given (R; d2 ),

f : R ! R; x 7! x
is a contraction i¤ j j < 1; in that case j j is the contraction coe¢ cient of f .
2. Let S be a nonempty open subset of R and f : S ! S a di¤ erentiable function. If

sup jf 0 (x)j < 1;

x2S

then f is a contraction.

De…nition 451 For any f; g 2 X B (T ), we say that f g if 8x 2 T; f (x) g (x).

Proposition 452 (Blackwell) Let the following objects be given:

1. a nonempty set T ;
2. X is a nonempty subset of the set B (T ) such that 8f 2 X; 8 2 R+ ; f + 2 X;
3. : X ! X is increasing, i.e., f g ) (f ) (g);
4. 9 2 (0; 1) such that 8f 2 X; 8 2 R+ ; (f + ) (f ) + .
Then is a contraction with contraction coe¢ cient .

Proof. 8f; g 2 X; 8x 2 T

f (x) g (x) jf (x) g (x)j sup jf (x) g (x)j = d1 (f; g) :

x2T

Therefore, f g + d1 (f; g), and from Assumption 3,

(f ) (g + d1 (f; g)) :

Then, from Assumption 4,

9 2 (0; 1) such that (g + d1 (f; g)) (g) + d1 (f; g) ;

and therefore
(f ) (g) + d1 (f; g) : (9.27)
Since the argument above is symmetric with respect to f and g, we also have

(g) (f ) + d1 (f; g) : (9.28)

From (9:27) and (9:28) and the de…nition of absolute value, we have

j (f ) (g)j d1 (f; g) ;

as desired.

Proposition 453 (Banach …xed point theorem) Let (X; d) be a complete metric space. If : X ! X is a
contraction with coe¢ cient k, then

9! x 2 X such that x = (x ) : (9.29)

and
n
8x0 2 X and 8n 2 N; d( (x0 ) ; x ) k n d (x0 ; x ) ; (9.30)
1 2 n
where n := ( :::: ).
Proof. (9:29) holds true.
9.7. FIXED POINT THEOREM: CONTRACTIONS 135

Take any x0 2 X and de…ne the sequence

(xn )n2N 2 X 1 ; with 8n 2 N; xn+1 = (xn ) :

We want to show that 1. that (xn )n2N is Cauchy, 2. its limit is a …xed point for , and 3. that …xed
point is unique.
1. First of all observe that
8n 2 N; d (xn+1 ; xn ) k n d (x1 ; x0 ) ; (9.31)
where k is the contraction coe¢ cient of , as shown by induction below.
Step 1: P (1) is true:
d (x2 ; x1 ) = d ( (x1 ) ; (x0 )) kd (x1 ; x0 )
from the de…nition of the chosen sequence and the assumption that is a contraction.
Step 2. P (n 1) ) P (n) :

d (xn+1 ; xn ) = d ( (xn ) ; (xn 1 )) kd (xn ; xn 1) k n d (x1 ; x0 )

from the de…nition of the chosen sequence, the assumption that is a contraction and the assumption of the
induction step.
Now, for any m; l 2 N with m > l;

d (xm ; xl ) d (xm ; xm 1) + d (xm 1 ; xm 2 ) + ::: + d (xl+1 ; xl )

km l
km 1
+ km 2
+ ::: + k l d (x1 ; x0 ) kl 1 1 k d (x1 ; x0 ) ;

where the …rst inequality follows from the triangle inequality, the third one from the following computa-
tion11 :
1 km l
k m 1 + k m 2 + ::: + k l = k l 1 + k + ::: + k m l+1 = k l :
1 k
Finally, since k 2 (0; 1) ;we get
kl
d (xm ; xl ) d (x1 ; x0 ) : (9.32)
1 k
If x1 = x0 ; then for any m; l 2 N with m > l; d (xm ; xl ) = 0 and 8n 2 N; xn = x0 and the se-
quence is converging and therefore it is Cauchy. Therefore, consider the case x1 6= x0 :From (9:32) it follows
N "(1 k)
that (xn )n2N 2 X 1 is Cauchy: 8" > 0 choose N 2 N such that 1k k d (x1 ; x0 ) < ", i.e., k N < d(x 1 ;x0 )
and
"(1 k)
log
d(x1 ;x0 )
N> log .
2. Since (X; d) is a complete metric space, (xn )n2N 2 X 1 does converge say to x 2 X, and, in fact, we
want to show that (x ) = x . Then, 8" > 0; 9N 2 N such that 8n > N;

d ( (x ) ; x ) d ( (x ) ; xn+1 ) + d xn+1 ; x

d ( (x ) ; (xn )) + d xn+1 ; x kd (x ; xn ) + d xn+1 ; x

" "
2 + 2 = ";
1 1 We are also using the basic facat used to study geometrical series. De…ne
sn 1 + a + a2 + ::: + an ;
:
Multiply both sides of the above equality by(1 a):
(1 a) sn (1 a) 1 + a + a2 + ::: + an

(1 a) sn 1 + a + a2 + ::: + an a + a2 + ::: + an+1 = 1 an+1

Divide both sides by (1 a):
1 an+1 1 an+1
sn 1 + a + a2 + ::: + an a + a2 + ::: + an+1 = =
1 a 1 a 1 a
136 CHAPTER 9. METRIC SPACES

where the …rst equality comes from the triangle inequality, the second one from the construction of the
sequence (xn )n2N 2 X 1 , the third one from the assumption that is a contraction and the last one from
the fact that (xn )n2N converges to x . Since " is arbitrary, d ( (x ) ; x ) = 0, as desired.
3. Suppose that xb is another …xed point for - beside x . Then,

d (b
x; x ) = d ( (b
x) ; (x )) kd (b
x; x )

and assuming x b 6= x would imply 1 k, a contradiction of the fact that is a contraction with contraction
coe¢ cient k.
(9:30) hods true.
We show the claim by induction on n 2 N.
P (1) is true.
d ( (x0 ) ; x ) = d ( (x0 ) ; (x )) k d (x0 ; x ) ;
where the equality follows from the fact that x is a …xed point for , and the inequality by the fact that
is a contraction.
P (n 1) is true implies that P (n) is true.
n n n 1
d( (x0 ) ; x ) = d ( (x0 ) ; (x )) = d (x0 ) ; (x )

n 1
k d (x0 ) ; x k kn 1
d (x0 ; x ) = k n d (x0 ; x ) :

9.8 Appendices.
9.8.1 Some characterizations of open and closed sets
Remark 454 From basic set theory, we have AC \ B = ? , B A, as veri…ed below.
: 9x : x 2 AC ^ x 2 B = h8x : x 2 A _ : (x 2 B)i =

( )
= h8x : : (x 2 B) _ x 2 Ai = h8x : x 2 B ) x 2 Ai ;

where ( ) follows from the fact that hp ) qi = h(:p) _ qi.

Proposition 455 S is open , S \F (S) = ?.

Proof. [)]
Suppose otherwise, i.e., 9x 2 S \F (S). Since x 2 F (S), 8r 2 R++ ; B (x; r) \ S C = 6 ?. Then, from
Remark 454, 8r 2 R++ , it is false that B (x; r) S, contradicting the assumption that S is open.
[(]
Suppose otherwise, i.e., 9x 2 S such that

8r 2 R++ ; B (x; r) \ S C 6= ? (9.33)

Moreover
x 2 B (x; r) \ S 6= ? (9.34)
But (9:33) and (9:34) imply x 2 F (S). Since x 2 S, we would have S \F (S) 6= ?, contradicting the
assumption.

Proposition 456 S is closed , F (S) S.

Proof.
(1) (2) (3)
S closed , S C open , S C \F S C = ? , S C \F (S) = ? , F (S) S
where
(1) follows from Proposition 455;
(2) follows from Remark 376
(3) follows Remark 454.
9.8. APPENDICES. 137

Proposition 457 S is closed , D (S) S:

Proof. We are going to use Proposition 456, i.e., S is closed , F (S) S.

[)]
Suppose otherwise, i.e.,

9x 2
= S such that 8r 2 R++ ; (Sn fxg) \ B (x; r) 6= ?

and since x 2
= S, it is also true that

8r 2 R++ ; S \ B (x; r) 6= ? (9.35)

and
8r 2 R++ ; S C \ B (x; r) 6= ? (9.36)

From (9:35) and (9:36), it follows that x 2 F (S), while x 2

= S, which from Proposition 456 contradicts
the assumption that S is closed.
[(]
Suppose otherwise, i.e., using Proposition 456,

9x 2 F (S) such that x 2

=S

Then, by de…nition of F (S),

8r 2 R++ ; B (x; r) \ S 6= ?.

Since x 2
= S, we also have
8r 2 R++ ; B (x; r) \ (Sn fxg) 6= ?,

i.e., x 2 D (S) and x 2

= S, a contradiction.

Proposition 458 8S; T X, S T ) D (S) D (T ).

Proof. Take x 2 D (S). Then

8r 2 R++ ; (Sn fxg) \ B (x; r) 6= ?: (9.37)

Since S T , we also have

(T n fxg) \ B (x; r) (Sn fxg) \ B (x; r) : (9.38)

From (9:37) and (9:38), we get x 2 D (T ).

Proposition 459 S [ D (S) is a closed set.

C
Proof. Take x 2 (S [ D (S)) i.e., x 2
= S and x 2
= D (S). We want to show that

9r 2 R++ such that B (x; r) \ (S [ D (S)) = ?;

i.e.,
9r 2 R++ such that (B (x; r) \ S) [ (B (x; r) \ D (S)) = ?;

Since x 2
= D (S), 9r 2 R++ such that B (x; r) \ (Sn fxg) = ?. Since x 2
= S, we also have that

9r 2 R++ such that S \ B (x; r) = ?: (9.39)

We are then left with showing that B (x; r) \ D (S) = ?. If y 2 B (x; r), then from (9:39), y 2
= S and
B (x; r) \ S n fyg = ?; i.e., y 2
= D (S) ;i.e., B (x; r) \ D (S) = ?, as desired.

Proposition 460 Cl (S) = S [ D (S).

138 CHAPTER 9. METRIC SPACES

Proof. [ ]
Since
S Cl (S) (9.40)
from Proposition 458,
D (S) D (Cl (S)) : (9.41)
Since Cl (S) is closed, from Proposition 457,

D (Cl (S)) Cl (S) (9.42)

From (9:40) ; and (9:41) ; (9:42), we get

S [ D (S) Cl (S)

[ ]
Since, from Proposition 459, S [ D (S) is closed and contains S, then by de…nition of Cl (S),

Cl (S) S [ D (S) :

To proceed in our analysis, we need the following result.

Lemma 461 For any metric space (X; d) and any S X, we have that
1. X = Int S [ F (S) [ Int S C , and
C
2. (Int S [ F (S)) = Int S C :

Proof. If either S = ? or S = X, the results are trivial. Otherwise, observe that either x 2 S or
x 2 X n S:
1. If x 2 S;then
either 9r 2 R++ such that B (x; r) S and then x 2 Int S,
or 8r 2 R++ , B (x; r) \ S C 6= ? and then x 2 F (S) :
Similarly, if x 2 X n S;then
either 9r0 2 R++ such that B (x; r0 ) X n S and then x 2 Int (X n S),
or 8r0 2 R++ , B (x; r0 ) \ S 6= ? and then x 2 F (S) :
2. By de…nition of Interior and Boundary of a set, (Int S [ F (S)) \ Int S C = ?:
Now, for arbitrary sets A; B X such that A [ B = X and A \ B = ?, we have what follow:
C
A [ B = X , (A [ B) = X C , AC \ B C = ?, and from Remark 454, B C A;
C
A \ B = ? , A \ BC = ? ) A BC .
Therefore we can the desired result.

Proposition 462 Cl (S) = Int S [ F (S).

Proof. From Lemma 461, it is enough to show that

C
(Cl (S)) = Int S C :

[ ]
Take x 2 Int S C . Then, 9r 2 R++ such that B (X; r) S C and therefore B (x; r) \ S = ? and, since
x2
= S,
B (x; r) \ (Sn fxg) = ?:
Then x 2
= S and x 2
= D (S), i.e.,
x2
= S [ D (S) = Cl (S)
C
where last equality follows from Proposition 460. In other words, x 2 (Cl (S)) .
[ ]
C C
Take x 2 (Cl (S)) = (D (S) [ S) . Since x 2 = D (S),

9r 2 R++ such that (Sn fxg) \ B (x; r) = ? (9.43)

9.8. APPENDICES. 139

Since x 2
= S,
9r 2 R++ such that S \ B (x; r) = ? (9.44)
i.e.,
9r 2 R++ such that B (x; r) SC (9.45)
C
and x 2 Int S .

De…nition 463 x 2 X is an adherent point for S if 8r 2 R++ ; B (x; r) \ S 6= ? and

Ad (S) := fx 2 X : 8r 2 R++ ; B (x; r) \ S 6= ?g

Corollary 464 1. Cl (S) = Ad (S).

2. A set S is closed , Ad (S) = S.

Proof. 1.
[ ]
x 2 Cl (S) ) hx 2 IntS or F (S)i and in both cases the desired conclusion is insured.
[ ]
If x 2 S, then, by de…nition of closure, x 2 Cl (S). If x 2 = S;then S = Sn fxg and, from the assumption,
8r 2 R++ ; B (x; r) \ (Sn fxg) 6= ?, i.e., x 2 D (S) which is contained in Cl (S)from Proposition 460.
2. It follows from 1. above and Proposition 379.2.

Proposition 465 Let S Rn be given. Then, F (S) = Cl (S) n Int(S).

Proof. We want to show

h 8r 2 R++ ; B (x; r) \ S 6= ? and B (x; r) \ S c 6= ? i , h x 2 Cl (S) nIntS i :

From De…nition ?? and ??, x 2 Cl (S) nInt (S) i¤ 8r 2 R++ ; B (x; r)\S 6= ? and : (9 > 0 such that B (x; ) S),
i.e., 8 > 0,
B (x; ) \ (Rn nS) = B (x; ) \ Rn \ S C = B (x; ) \ S c 6= ?:
Therefore, x 2 Cl (S) nInt (S) i¤ 8r 2 R++ ; B (x; r) \ S 6= ? and B (x; ) \ S C 6= ?.

Proposition 466 x 2 Cl (S) , 9 (xn )n2N in S converging to x.

Proof. [)]
From Corollary 464, if x 2 Cl (S) then 8n 2 N, we can take xn 2 B x; n1 \ S. Then d (x; xn ) < 1
n and
limn!+1 d (x; xn ) = 0.
[(]
By de…nition of convergence,

8" > 0; 9n" 2 N such that 8n > n" ; d (xn ; x) < " or xn 2 B (x; ")

or
8" > 0; B (x; ") \ S fxn : n > n" g
and
8" > 0; B (x; ") \ S 6= ?
i.e., x 2 Ad (S) ; and from the Corollary 464.1, the desired result follows.

Proposition 467 S is closed , any convergent sequence (xn )n2N with elements in S converges to an
element of S.

Proof. We are going to show that S is closed using Proposition 457, i.e., S is closed , D (S) S. We
want to show that
(xn )n2N is such that 1. 8n 2 N; xn 2 S; and
hD (S) Si , ) x0 2 S ;
2. xn ! x0

[)]
140 CHAPTER 9. METRIC SPACES

Suppose otherwise, i.e., there exists (xn )n2N such that 1. 8n 2 N; xn 2 S. and 2:xn ! x0 , but x0 2
= S.
By de…nition of convergent sequence, we have

8" > 0; 9n0 2 N such that 8n > n0 ; d (xn ; x0 ) < "

and, since 8n 2 N; xn 2 S,
fxn : n > n0 g B (x0 ; ") \ (Sn fx0 g)
Then,
8" > 0; B (x0 ; ") \ (Sn fx0 g) 6= ?
and therefore x0 2 D (S) while x0 2
= S, contradicting the fact that S is closed.
[(]
Suppose otherwise, i.e., 9 x0 2 D (S) and x0 2 = S. We are going to construct a convergent sequence
(xn )n2N with elements in S which converges to x0 (a point not belonging to S).
From the de…nition of accumulation point,

1
8n 2 N; (Sn fx0 g) \ B x; 6= ?:
n

Then, we can take xn 2 (Sn fxg) \ B x; n1 , and since d (xn ; x0 ) < 1

n, we have that d (xn ; x0 ) ! 0.
Summarizing, the following statements are equivalent:

1. S is open (i.e., S Int S)

2. S C is closed,
3. S \F (S) = ?,

and the following statements are equivalent:

1. S is closed,
2. S C is open,
3. F (S) S;
4. S = Cl (S) :
5. D (S) S;
6. Ad (S) = S;
7. any convergent sequence (xn )n2N with elements in S converges to an element of S:

9.8.2 Norms and metrics

In these notes, a …eld K can be either R or C.

De…nition 468 A norm on a vector space E on a …eld K 2 fR; Cg is a function

jj jj : E ! R; x 7! jjxjj

which satis…es the following properties: 8x; y 2 E,8 2 K,

1. jjxjj 0 (non negativity),

2. jjx + yjj jjxjj + jjyjj (triangle inequality),
12
3. jj xjj = j j jjxjj (homogeneity),
4. jjxjj = 0 ) x = 0 (separation).
1 2 If
p
2 R, j j is the absolute value of ; if = (a; b) 2 C, then j j = a2 + b2 .
9.8. APPENDICES. 141

Proposition 469 Given a norm jj jj on E, 8x; y 2 E

1. x = 0 ) jjxjj = 0
2. jjx yjj = jjy xjj
3. j(jjxjj jjyjj)j jjx yjj.

Proof.
1. Since E is a vector space, 8x 2 E, 0x = 0. Then
(a) (b)
jj0jj = jj0xjj = j0j jjxjj = 0jjxjj = 0

where (a) follows from property 3 of norm and (b) from the de…nition of absolute value.
(c)
2. jjx yjj = jj y+xjj = 13 jj y ( x)jj = jj( 1)y+( 1)( x)jj = jj( 1)(y x)jj = j 1j jjy xjj = jjy xjj
where (c) follows from Proposition 136.
3. From the de…nition of absolute value, we want to show that

jjx yjj jjxjj jjyjj jjx yjj:

Indeed,
jjxjj = jjx y + yjj jjx yjj + jjyjj;
i.e., jjx yjj jjxjj jjyjj; and

jjyjj = jjy x + xjj jjy xjj + jjxjj = jjx yjj + jjxjj;

i.e., jjx yjj jjxjj jjyjj, as desired.

Proposition 470 If properties 2. and 3. in De…nition 468 hold true, then property 1. in the same de…nition
holds true.

Proof. We want to show that

hjjx + yjj jjxjj + jjyjj ^ jj xjj = j j jjxjji

) h8x 2 E; jjxjj 0i
Observe that if x = 0, from Proposition 469.2 (which uses only property 3 of De…nition 468) we have jjxjj = 0.
Then

jj0jj jjx xjj jjxjj + jj xjj and therefore

jjxjj jjxjj = j 1j jjxjj = jjxjj:

Now, if jjxjj < 0; we would have a negative number strictly larger than a positive number, which is a
contradiction.

De…nition 471 The pair (E; jj jj), where E is a vector space and jj jj is a norm, is called a normed vector
space.

Remark 472 Normed spaces are, by de…nition, vector space.

De…nition 473 A seminorm is a function satisfying properties 1, 2 and 3 in De…nition 468.

De…nition 474 Given a non-empty set X, a function d : X X ! R is called a metric or a distance on

X if 8x; y; z 2 X,
1 3 8v 2 V ( 1)v = v and ( v) = ( 1)(( 1)v) = (( 1)( 1))v = 1 v = v:
142 CHAPTER 9. METRIC SPACES

1. d(x; y) 0 (non negativity),

2. d(x; y) = 0 , x = y (coincidence),
3. d(x; y) = d(y; x) (symmetry),
4. d(x; z) d(x; y) + d(y; z) (triangle inequality),

(X; d) is called a metric space.

De…nition 475 Given a normed vector space (E; jj jj); the metric

d : E 2 ! R; (x; y) 7! jjx yjj

is called the metric induced by the norm jj jj:

Proposition 476 Given a normed vector space (E; jj jj),

d:E E ! R; (x; y) 7! jjx yjj

is a metric and (E; d) is a metric space.

Proof.
1. It follows from the fact that x; y 2 E ) x y 2 E and property 1 of the norm.
2. It follows from property 1 of the norm and Proposition 469.1.
3. It follows from Proposition 469.2.
4. d(x; z) = jjx zjj = jj(x y) + (y z)jj jjx yjj + jjy zjj = d(x; y) + d(y; z).

Proposition 477 If jj jj is a norm on a vector space E and

d:E E ! R; (x; y) 7! jjx yjj

then 8x; y; z 2 E; 8 2 K

a. d(x; 0) = jjxjj
b. d(x + z; y + z) = d(x; y) (translation invariance)
c. d( x; y) = j jd(x; y) (homogeneity).

Proof.

a. d(x; 0) = jjx 0jj = jjxjj

b. d(x + z; y + z) = jj(x + z) (y + z)jj = jjx yjj = d(x; y)
c. d( x; y) = jj x yjj = jj (x y)jj = j j jjx yjj = j jd(x; y).

Proposition 478 Let (E; d) be a metric space such that d satis…es translation invariance and homogeneity.
Then
n : E ! R; x 7! d(x; 0)
is a norm and 8x; y 2 E, n(y x) = d(x; y).

Proof.

1. n(x) = d(x; 0) 0, where the inequality follows from property 1 in De…nition 4,

9.8. APPENDICES. 143

2.
(a) (b)
n(x + y) = d(x + y; 0) = d(x + y y; 0 y) = d(x; y) d(x; 0) + d(0; y) =
(c) (d)
= d(x; 0) + d( y; 0) = d(0; y) + d(0; x) = n(y) + n(x);

where (a) follows from translation invariance, (b) from triangle inequality in De…nition 474, (c) from
symmetry in De…nition 474 and (d) from homogeneity.
3.
n( x) = d( x; 0) = j jd(x; 0) = j jn(x);

4.
n(x) = 0 ) d(x; 0) = 0 ) x = 0:

It follows that

n(y x) = d(y x; 0) = d(y x + x; 0 + x) = d(y; x) = d(x; y):

Remark 479 The above Proposition suggests that the following statement is false:
Given a metric space (E; d), then nd : E ! R; : x 7! d(x; 0) is a norm on E.
The fact that the above statement is false is veri…ed below. Take an arbitrary vector space E with the
discrete metric d, (
0 if x = y
d : E E ! R; d(x; y) =
1 if x 6= y
First of all, let’s verify that d does not satisfy (translation invariance and homogeneity), otherwise from
Proposition 478, we would contradict the desired result. Indeed homogeneity fails.
Take x 6= y and = 2 then
d(x; y) = 1 and d( x; y) = 1
j jd(x; y) = 2 6= 1:
Let’s now show that in the case of the discrete metric

n : E ! R; x 7! d(x; 0)

is not a norm. Take x 6= 0 and = 2 then

jj xjj = d( x; 0) = 1
j jd(x; 0) = 2:

9.8.3 Distance between sets, diameters and “topological separation”.

De…nition 480 Let (X; d) be a metric space. The distance between a point x 2 X and a non-empty subset
A of X is denoted and de…ned by
d(x; A) = inffd(x; a) : a 2 Ag

The distance between two non-empty subsets A and B of X is denoted and de…ned by

d(A; B) = inffd(a; b) : a 2 A; b 2 Bg

De…nition 481 The diameter of a non-empty subset A of X is denoted and de…ned by

d(A) = supfd(a1 ; a2 ) : a1 ; a2 2 Ag

If the diameter of A is …nite, then A is said to be bounded, otherwise A is said to be unbounded.

144 CHAPTER 9. METRIC SPACES

Proposition 482 Let A and B be non-empty subsets of a metric space (X; d) and let x 2 X. Then
1. d(x; A) and d(A; B) are non-negative real numbers;
2. If x 2 A, then d(x; A) = 0;
3. If A \ B 6= ?, then d(A; B) = 0;
4. If A is …nite, then d(A) < 1, i.e., A is bounded.

Proof. 1.It follows directly from De…nition 327.

2. Since x 2 A, we have that

Def 327 Def 480

0 = d(x; x) 2 fd(x; a) : a 2 Ag R+

3.
A \ B 6= ? ) 9 : x such that x 2 A and x 2 B
But, d(x; x) = 0, therefore we can proceed like we did in point 2 above.
4. Since A is …nite, d(A) = supfd(a1 ; a2 ) : a1 ; a2 2 Ag is the max of a …nite set and therefore it is …nite.

Remark 483 The converses of the statements 2, 3 and 4 in Proposition 482 do not hold true, as veri…ed
below.
1. d(x; A) = 0 ; x 2 A:
Consider the euclidean topology on R and take A = (0; 1). We have that 0 2 = A, while d(0; A) = 0:
2. d(A; B) = 0 ; A \ B 6= ?:
Consider the euclidean topology on R and take A = (0; 1) and B = [1; 2). We have that A \ B = ?, while
d(A; B) = 0:
3. d(A) < 1 ; A is …nite:
Consider the euclidean topology on R and take A = [0; 1]. A is not …nite, but we have that d(A) = 1.

Remark 484 For the empty set, the following conventions are adopted:

1. d(x; ?) = +1;

2. d(A; ?) = d(?; A) = +1;

3. d(?) = 1.

Proposition 485 Let x 2 X with (X; d) metric space. The closure of a subset A of X is equal to the set of
points whose distance from A is zero, i.e.,

Cl(A) = fx : d(x; A) = 0g

Proof. From Proposition 368 we know that Cl(A) = A [ D(A). We need to show that the points of A
and D(A) have distance zero from A. First, from Proposition 482 point 2, we have that 8x 2 A; d(x; A) = 0.
Now consider D(A)
Def accumulation p oint
x 2 D(A) , 8 G open, such that x 2 G; (G n fxg) \ A 6= ? )
(X; d) m etric space
) 8r > 0; (B(X;d) (x; r) n fxg) \ A 6= ?
Now assume that 9 x0 2 D(A) such that d(x0 ; A) = " 6= 0. But then,
"
B(X;d) x0 ; \ A = ?:
2
Therefore x0 is not an accumulation point for A. It follows that x 2 D(A) ) d(x; A) = 0. In conclusion,
since all the points of A and D(A) have distance zero from A, we have that the points of Cl(A) = A [ D(A)
have distance 0 from A.

Remark 486 From the previous Proposition, in a metric space (X; d), A X is closed if A = fx 2 X :
d(x; A) = 0g:
9.8. APPENDICES. 145

Corollary 487 In a metric space (X; d), all …nite sets are closed.

Proof. From the de…nition of distance - see De…nition 327 - since d(x; y) = 0 , x = y, we have that
the only point with zero distance from a singleton set fxg X is the point x itself. Hence by the above
proposition we have that, in a metric space, singleton sets are closed. But since by Corollary ?? any …nite
union of closed sets is closed, we have that …nite sets are closed as well.

Corollary 488 Let (X; d) be a metric space. Let A be an (X; d)-closed set, then x 2
= A ) d(x; a) > 0.

Proof.
Cl(A) = A
d(x; A) 6= 0 ) x 2
= Cl(A) ) x2
= A:

Proposition 489 (A topological separation property) Let (X; d) be a metric space and A; B closed disjoint
subsets of X. Then there exist open disjoint sets G and H such that A G and B H.

Proof. If either A or B is empty, we can take G and H equal to ? and X. Assume now that A and B
are non-empty. Take a 2 A and b 2 B. Since A and B are disjoint, i.e., A \ B = ?,

ha 2 A and a 2
= Bi and hb 2
= A and b 2 Bi:

From the Corollary 488, 9 a > 0 and b > 0 such that d(a; B) = a and d(b; A) = b. For any a 2 A and
any b 2 B, de…ne,
1 1
Sa = B a; a and Sb = B b; b :
3 3
We want to show that our desired sets G and H are indeed
[ [
G= Sa and H = Sb :
a2A b2B

1. G and H are clearly open

2. Clearly, A G and B H

3. We are left with showing that G \ H = ?. Suppose otherwise, i.e., 9x 2 G \ H. Then by de…nition of
G and H,
1 1
9ax 2 A and bx 2 B such that x 2 B ax ; a \ B bx ; b : (9.46)
3 3
Since ax 2 A and bx 2 B and A and B are disjoint we have that d(ax ; bx ) = " > 0. Then, by de…nition
of distance between a point and a set,

d(ax ; B) := a " and d(bx ; A) := b ": (9.47)

Moreover, from (9.46),

1 1
d(ax ; x) < a; d(bx ; x) < b (9.48)
3 3
Then
(9.48) 1 1 (9.47) 2
" := d(ax ; bx ) d(ax ; x) + d(bx ; x) < a + b ":
3 3 3
i.e., 0 < " < 32 ", a contradiction.

Example 490 Consider the metric space (R2 ; d) and the following two subsets of R2 :

A = f(x; y) : xy 1; x < 0g; B = f(x; y) : xy 1; x > 0g:

A and B are both closed and disjoint. However, d(A; B) = 0.

146 CHAPTER 9. METRIC SPACES

y 10

7.5

2.5

0
-4 -2 0 2 4

Lemma 491 Let (X; d) be a metric space and S X. Then

d(S) = d(Cl(S)):

Proof. By applying the de…nition of diameter, we want to prove that , de…ned

A := fd(s1 ; s2 ) : s1 ; s2 2 Sg and B := fd(s1 ; s2 ) : s1 ; s2 2 Cl(S)g := sup B;

sup A = sup B:
[ ] It follows from the fact that, since S Cl(S), A B.
[ ] From Proposition 485 we have that Cl(S) = fx : d(x; S) = 0g. Therefore 8 x 2 F(S), 9 s 2 S such that
d(x; s ) = 014 . Taken s1 ; s2 2Cl(S), since Cl(S) = F(S) [ (S), we have three possible cases.

Case 1: s1 ; s2 2 S. In this case d(s1 ; s2 ) 2 A.

Case 2: s1 2 F(S) and s2 2 S. From Proposition 485 mentioned above, it is possible to …nd s3 2 S such
that d(s1 ; s3 ) = 0. Hence, from the triangle inequality, d(s1 ; s2 ) d(s1 ; s3 ) + d(s3 ; s2 ) = d(s3 ; s2 ) 2 A.

Case 3: s1 ; s2 2 F(S). Just like we did for Case 2, we can …nd s3 ; s4 2 S such that d(s1 ; s3 ) = 0 and
d(s2 ; s4 ) = 0. Therefore d(s1 ; s2 ) d(s1 ; s3 ) + d(s3 ; s2 ) d(s3 ; s4 ) + d(s4 ; s2 ) = d(s3 ; s4 ) 2 A.

In all cases we have found that for every element in B it is possible to …nd an element in A which is
greater or equal, therefore sup A sup B, as desired.

De…nition 492 Let A be a nonempty set in Rn . Then, a function f : A ! Rm is said to satisfy a Lipschitz
condition on A, or simply to be Lipschitz continuous on A, if

9s 2 R such that 8x; y 2 A, kf (x) f (y)k skx yk:

Example 493 The norm function satis…es a Lipschitz condition. In fact,

(1)
kkxk kykk = jkxk kykj kx yk;

where (1) follows from Proposition ??.3: indeed, the norm function satis…es the Lipschitz condition with
s = 1.

Proposition 494 Let A be a nonempty set in Rn , and let the function f : A ! Rm satisfy a Lipschitz
condition on A. Then, f is continuous on A.
1 4 Remember that, from Proposition ??, Cl(S) = Int(S) [ F (S). Since Int(S) S Cl(S),

F (S) [ S = Cl(S):
9.9. EXERCISES 147

Proof. Let (an )n2N be a sequence of points of A convergent to a 2 A, i.e., such that

lim kan ak = 0: (9.49)

n!+1

Since 9s 2 R such that kf (an ) f (a)k skan ak, from (9:49),

lim kf (an ) f (a)k = 0;

n!+1

whence the sequence (f (an ))n2N converges to f (a). Since this is true 8a 2 A, f is continuous on A.

Proposition 495 The distance function is Lipschitz continuous and therefore it is continuous on Rn .

Proof. 1st proof. Let x; y 2 Rn and A be a nonempty set in Rn . Then, by de…nition of distance as inf,
8" > 0 9a 2 A such that
ky ak < dA (y) + ": (9.50)
Now,
Def. ?? (9:50)
dA (x) kx ak = kx y+y ak kx yk + ky ak < kx yk + dA (y) + ";
whence, since " > 0 is arbitrary,
dA (x) kx yk + dA (y);
or
dA (x) dA (y) kx yk:
Since the above argument is perfectly symmetric with respect to x and y, we can repeat it interchanging x
with y in order to get
dA (y) dA (x) ky xk;
and then, as desired,
jdA (x) dA (y)j kx yk:
2. (Nguyen (2016)).
Take x; y 2 A. Then, for any u 2 A, we have

d (x; A) kx uk kx yk + ky uk :

Taking inf of both sides of the above inequalities with respect to u, we get
Def. d(:::;A)
d (x; A) kx uk kx yk + inf fky uk : u 2 Ag = kx yk + d (y; A) ;

or
d (x; A) d (y; A) kx yk : (9.51)
Repeating the above argument interchanging x with y, we get

d (y; A) d (x; A) kx yk ;

and therefore
d (x; A) d (y; A) kx yk : (9.52)
(9:51) and (9:52) are the desired result.

9.9 Exercises
Problem sets: 3,4,5,6.
From Lipschutz (1965), starting from page 54: 1, 18, 19, 20, 23, 28 (observe that Lipschutz uses the word
“range” in the place of “image”);
starting from page 120: 1, 3, 6, 7, 25, 29.
148 CHAPTER 9. METRIC SPACES
Chapter 10

Functions

10.1 Limits of functions

In what follows we take for given metric spaces (X; d) and (X 0 ; d0 ) and sets S X and T X 0.

De…nition 496 Given x0 2 D (S) ;i.e., given an accumulation point x0 for S, and f : S ! T , we write

lim f (x) = l 2 X 0
x!x0

if
8" > 0; 9 > 0 such that x 2 B(X;d) (x0 ; ) \ S n fx0 g ) f (x) 2 B(X 0 ;d0 ) (l; ")
or
8" > 0; 9 > 0 such that h x 2 S ^ 0 < d (x; x0 ) < i ) d0 (f (x) ; l) < "

Proposition 497 Given x0 2 D (S) and f : S ! T ,

hlimx!x0 f (x) = li
,
* for any sequence (x ) in S such that 8n 2 N, xn 6= x0 and limn!+1 xn =x0 , +
n n2N

limn!+1 f (xn ) = l:

Proof. for the following proof see also Proposition 6.2.4, page 123 in Morris.
[)]
Take
a sequence (xn )n2N in S such that 8n 2 N; xn 6= x0 and lim xn = x0 :
n!+1

We want to show that limn!+1 f (xn ) = l, i.e.,

8" > 0; 9n0 2 N such that 8n > n0 ; d (f (xn ) ; l) < ":

Since limx!x0 f (x) = l,

8" > 0; 9 > 0 such that x 2 S ^ 0 < d (x; x0 ) < ) d (f (x) ; l) < ":

Since limn!+1 xn = x
( )
8 > 0; 9n0 2 N such that 8n > n0 ; 0 < d (xn ; x0 ) < ;

where ( ) follows from the fact that 8n 2 N, xn 6= x0 :

Therefore, combining the above results, we get

8" > 0; 9n0 2 N such that 8n > n0 ; d (f (xn ) ; l) < "

as desired.

149
150 CHAPTER 10. FUNCTIONS

[(]
Suppose otherwise, then
1
9" > 0 such that 8 n = n, i.e., 8 n 2 N, 9xn 2 S such that
(10.1)
1
xn 2 S ^ 0 < d (xn ; x0 ) < n and d (f (x) ; l) ":

Consider (xn )n2N ; then, from the above and from Proposition 394, xn ! x0 , and from the above
(speci…cally the fact that 0 < d (xn ; x0 )), we also have that 8n 2 N; xn 6= x0 . Then by assumption,
limn!+1 f (xn ) = l, i.e., by de…nition of limit,

8" > 0; 9N 2 N such that if n > N; then jf (xn l)j < ";

contradicting (10:1).

Proposition 498 (uniqueness) Given x0 2 D (S) and f : S ! T ,

lim f (x) = l1 and lim f (x) = l2 ) hl1 = l2 i

x!x0 x!x0

Proof. It follows from Proposition 497 and Proposition 398.

Proposition 499 Given S X, x0 2 D (S) and f; g : S ! R, , and

lim f (x) = l and lim g (x) = m

x!x0 x!x0

1. limx!x0 f (x) + g (x) = l + m;

2. limx!x0 f (x) g (x) = l m;
f (x) l
3. if m 6= 0 and 8x 2 S, g (x) 6= 0, limx!x0 g(x) = m.

Proof. It follows from Proposition 497 and Proposition 396.

Remark 500 Given f : R2 ! R, it is false that

if limx!x0 f (x; y0 ) = f (x0 ; y0 ) and limy!y0 f (x0 ; y) = f (x0 ; y0 ), then lim(x;y)!(x0 ;y0 ) f (x; y) = f (x0 ; y0 ).
A desired counterexample is presented below. Let the function f : R2 ! R,
8 xy
< x2 +y2 if (x; y) 6= 0;
f (x; y) =
:
0 if (x; y) = 0;

be given. Then,
1.
lim f (x; 0) = lim 0 = 0 = f (0; 0) ; lim f (0; y) = lim 0 = 0 = f (0; 0) ;
x!0 x!0 y!0 y!0

2. We want to show
: (8" > 0; 9 > 0 such that (x; y) 2 B (0; ) ) jf (x; y)j < "), i.e.,
9 " > 0 such that 8 > 0 9 (x; y) 2 R2 such that k(x; y)k < and f (x; y) ".
2
Take " = 41 and (x; y) = (x; x) 2 B (0; ) n f0g. Then f (x; x) = (x)(x)
2
+(x)2
= 12 > 14 , as desired.

Proposition 501 Let f : S X ! Rm , x0 2 D (S) and for any j 2 f1; :::; mg, fj : S ! R be such that
8x 2 S,
m
f (x) = (fj (x))j=1
Then, if either limx!x0 f (x) = l 2 Rm or for any j 2 f1; :::; mg, limx!x0 fj (x) = lj 2 R, then
m
lim f (x) = lim fj (x) :
x!x0 x!x0
j=1

Proof. It follows from Propositions 497 and 399.

10.2. CONTINUOUS FUNCTIONS 151

10.2 Continuous Functions

De…nition 502 Given a metric space (X; d) and a set V X, an open neighborhood of V is an open set
containing V .

Remark 503 Sometimes, an open neighborhood is simply called a neighborhood.

De…nition 504 Take S (X; d) ; T (Y; d0 ), x0 2 S and f : S ! T . Then, f is (X; d) (Y; d0 ) continuous
at x0 if
8" > 0; 9 > 0 such that x 2 B(X:d) (x0 ; ) \ S ) f (x) 2 B(X 0 ;d0 ) (f (x0 ) ; ") ;
i.e.,
8" > 0; 9 > 0 such that x 2 S ^ d (x; x0 ) < ) d0 (f (x) ; f (x0 )) < ";
i.e.,
8" > 0; 9 > 0; f B(X;d) (x0 ; ) \ S B(X 0 ;d0 ) (f (x0 ) ; ") ;
i.e.,
for any open neighborhood V of f (x0 ),
there exists an open neighborhood U of x0 such that f (U \ S) V:
If f is continuous at x0 for every x0 in S, f is continuous on S.

Remark 505 If x0 is an isolated point of S, f is continuos at x0 . If x0 is an accumulation point for S, f

is continuous at x0 if and only if limx!x0 f (x) = f (x0 ).

Proposition 506 Suppose that Z X 00 , where (X 000 ; d00 ) is a metric space and

f : S ! T; g : W f (S) ! Z

h : S ! Z; h (x) = g (f (x))
If f is continuous at x0 2 S and g is continuous at f (x0 ), then h is continuous at x0 .

Proof. Exercise (see Apostol (1974), page 79) or Ok, page 206.

Proposition 507 Take f; g : S Rn ! R. If f and g are continuous, then

1. f + g is continuous;

2. f g is continuous;
f
3. if 8x 2 S, g (x) 6= 0, g is continuous.

Proof. If x0 is an isolated point of S, from Remark 505, we are done. If x0 is an accumulation point for
S, the result follows from Remark 505 and Proposition 499.

Proposition 508 Let f : S X ! Rm , and for any j 2 f1; :::; mg fj : S ! R be such that 8x 2 S,
m
f (x) = (fj (x))j=1

Then,
hf is continuousi , h8j 2 f1; :::; mg ; fj is continuousi

Proof. The proof follows the strategy used in Proposition 399.

n
De…nition 509 Given for any i 2 f1; :::; ng ; Si R, f : i=1 Si ! R is continuous in each variable
separately if 8i 2 f1; :::; ng and 8x0i 2 Si ,

0
fx0i : k6=i Sk ! R; fx0i (xk )k6=i = f x1 ; ::; xi 1 ; xi ; xi+1 ; :::; xn

is continuous.
152 CHAPTER 10. FUNCTIONS

Proposition 510 Given for any i 2 f1; :::; ng ;

n
f: i=1 Si ! R is continuous ) f is continuous in each variable separately

Proof. Exercise.

Remark 511 It is false that

f is continuous in each variable separately ) f is continuous

To see that consider f : R2 ! R,

8 xy
< x2 +y 2 if (x; y) 6= 0
f (x; y) =
:
0 if (x; y) = 0

The following Proposition is useful to show continuity of functions using the results about continuity of
functions from R to R.
n
Proposition 512 For any k 2 f1; :::; ng ; take Sk X, and de…ne S := k=1 Sk X n . Moreover, take
i 2 f1; :::; ng and let
g : Si ! Y; : xi 7! g (xi )
be a continuous function and
n
f : S ! Y; (xk )k=1 7! g (xi ) :
Then f is continuous.

Example 513 An example of the objects described in the above Proposition is the following one.

g : [0; ] ! R; g (x) = sin x;

f : [0; ] [ ; 0] ! R; f (x; y) = sin x:

Proof. of Proposition 512. We want to show that

8x0 2 S; 8" > 0 9 > 0 such that d (x; x0 ) < ^ x 2 S ) d (f (x) ; f (x0 )) < "

We know that
0 0
8xi0 2 Si ; 8" > 0 9 > 0 such that d (xi ; xi0 ) < ^ xi 2 S ) d (g (xi ) ; g (xi0 )) < "

Take = 0 . Then d (x; x0 ) < ^ x 2 S ) d (xi ; xi0 ) < 0

^ xi 2 S and " > d (g (xi ) ; g (xi0 )) =
d (f (x) ; f (x0 )), as desired.

Exercise 514 Show that the following function is continuous.

0 x1 1
e + cos (x1 x2 )
f (x1 x2 ) = @ sinex2x1 A
2
f : R2 ! R3 ;
x1 + x2
From Proposition 508, it su¢ ces to show that each component function is continuous. We are going to
show that f1 : R2 ! R,
f1 (x1 x2 ) = ex1 + cos (x1 x2 )
is continuous, leaving the proof of the continuity of the other component functions to the reader.
1. f11 : R2 ! R; f11 (x1 ; x2 ) = ex1 is continuous from Proposition 512 and “Calculus 1”;
2. h1 : R2 ! R; h1 (x1 ; x2 ) = x1 is continuous from Proposition 512 and “Calculus 1”,
h2 : R2 ! R; h2 (x1 ; x2 ) = x2 is continuous from Proposition 512 and “Calculus 1”,
g : R2 ! R; g (x1 ; x2 ) = h1 (x1 ; x2 ) h2 (x1 ; x2 ) = x1 x2 is continuous from Proposition 507.2,
: R ! R, (x) = cos x is continuous from “Calculus 1”,
f12 : R2 ! R; f12 (x1 ; x2 ) = ( g) (x1 ; x2 ) = cos (x1 x2 ) is continuous from Proposition 506 (continu-
ity of composition).
3. f1 = f11 + f12 is continuous from Proposition 507.1.
10.2. CONTINUOUS FUNCTIONS 153

The following Proposition is useful in the proofs of several results.

n n
Proposition 515 Let S; T be arbitrary sets, f : S ! T , fAi gi=1 a family of subsets of S and fBi gi=1 a
family of subsets of T:Then

1. “ inverse image preserves inclusions, unions, intersections and set di¤erences”, i.e.,
1 1
a. B1 B2 ) f (B)1 f B2 ,
1
b. f ([ni=1 Bi ) = [ni=1 f 1 (Bi ) ;
1
c. f (\ni=1 Bi ) = \ni=1 f 1 (Bi ) ;
1
d. f (B1 nB2 ) = f 1 (B1 ) nf 1 (B2 ) ;

2. “ image preserves inclusions, unions, only”, i.e.,

e. A1 A2 ) f (A)1 f (A2 ),
f. f ([ni=1 Ai ) = [ni=1 f (Ai ) ;
g. f (\ni=1 Ai ) \ni=1 f (Ai ) ;
and
if f is one-to-one, then f (\ni=1 Ai ) = \ni=1 f (Ai ) ;
h. f (A1 nA2 ) f (A1 ) nf (A2 ) ;and
if f is one-to-one and onto, then f (A1 nA2 ) = f (A1 ) nf (A2 ) ;

3. “relationship between image and inverse image”

i. A1 f 1 (f (A1 )) ; and
1
if f is one-to-one, then A1 = f (f (Ai )) ;
1
l. B1 f f (B1 ) , and
1
if f is onto, then B1 = f f (B1 ) .

Proof.
...
g.
(i) : y 2 f (A1 \ A2 ) , 9x 2 A1 \ A2 such that f (x) = y;
(ii) : y 2 f (A1 )\f (A2 ) , y 2 f (A1 ) ^ y 2 f (A2 ) , (9x1 2 A1 such that f (x1 ) = y)^(9x2 2 A2 such that f (x2 ) = y)
To show that (i) ) (ii) it is enough to take x1 = x and x2 = x:
...

Proposition 516 f : X ! Y is continuous ,

1
V Y is open ) f (V ) X is open.

Proof. [)]
1
Take a point x0 2 f (V ). We want to show that
1
9r > 0 such that B (x0 ; r) f (V )

De…ne y0 = f (x0 ) 2 V . Since V Y is open,

9" > 0 such that B (y0 ; ") V (10.2)

Since f is continuous,

8" > 0; 9 > 0; f (B (x0 ; )) B (f (x0 ) ; ") = B (y0 ; ") (10.3)

Then, taken r = , we have

(1) (2) (3)
1 1 1
B (x0 ; r) = B (x0 ; ) f (f (B (x0 ; ))) f (B (y0 ; ")) f (V )
154 CHAPTER 10. FUNCTIONS

where (1) follows from 3.i in Proposition 515,

(2) follows from 1.a in Proposition 515 and (10:3)
(3) follows from 1.a in Proposition 515 and (10:2).
[(]
Take x0 2 X and de…ne y0 = f (x0 ); we want to show that f is continuous at x0 .
Take " > 0, then B (y0 ; ") is open and, by assumption,
1
f (B (y0 ; ")) X is open. (10.4)

Moreover, by de…nition of y0 ,
1
x0 2 f (B (y0 ; ")) (10.5)
(10:4) and (10:5) imply that
1
9 > 0 such that B (x0 ; ) f (B (y0 ; ")) (10.6)

Then
(1) (2)
1
f (B (x0 ; )) f f (B (y0 ; ")) (B (y0 ; "))
where
(1) follows from 2.e in Proposition 515 and (10:6) ;
(2) follows from 2.l in Proposition 515

Proposition 517 f : X ! Y is continuous ,

1
V Y closed ) f (V ) X closed.

Proof. [)]
V closed in Y ) Y nV open. Then
1 1 1 1
f (Y nV ) = f (Y ) n f (V ) = Xnf (V ) (10.7)

where the …rst equality follows from 1.d in Proposition 515.

Since f is continuous and Y nV open;then from (10:7) Xnf 1 (V ) X is open and therefore f 1
(V ) is
closed.
[(]
We want to show that for every open set V in Y , f 1 (V ) is open.

1 1 1
V open ) Y nV closed ) f (Y nV ) closed, Xnf (V ) closed, f (V ) open.

De…nition 518 A function f : X ! Y is open if

S X open ) f (S) open;

it is closed if
S X closed ) f (S) closed.

Exercise 519 Through simple examples show the relationship between open,closed and continuous functions.

We can summarize our discussion on continuous function in the following Proposition.

Proposition 520 Let f be a function between metric spaces (X; d) and (Y; d0 ). Then the following state-
ments are equivalent:

1. f is continuous;
1
2. V Y is open ) f (V ) X is open;
1
3. V Y closed ) f (V ) X closed;
4. 8x0 2 X; 8 (xn )n2N 2 X 1 such that limn!+1 xn = x0 , limn!+1 f (xn ) = f (x0 ) :
10.3. CONTINUOUS FUNCTIONS ON COMPACT SETS 155

10.3 Continuous functions on compact sets

Proposition 521 Given f : X ! Y; if S is a compact subset of X and f is continuous, then f (S) is a
compact subset (of Y ).

Proof. Let F be an open covering of f (S), so that

f (S) [A2F A: (10.8)

We want to show that F admits an open subcover which covers f (S). Since f is continuous,
1
8A 2 F; f (A) is open in X

Moreover,
(1) (2) (3)
1 1 1
S f (f (S)) f ([A2F A) = [A2F f (A)
where
(1) follows from 3.i in Proposition 515,
(2) follows from 1.a in Proposition 515 and (10:8),
(3) follows from 1.b in Proposition 515.
In other words f 1 (A) A2F is an open cover of S. Since S is compact there exists A1 ; ::; An 2 F such
that
S [ni=1 f 1
(Ai ) :
Then
(1) (2) (3)
f (S) f [ni=1 f 1
(A) = [ni=1 f f 1
(Ai ) [ni=1 Ai
where
(1) follows from 1.a in Proposition 515,
(2) follows from 2.f in Proposition 515,
(3) follows from 3.l in Proposition 515.

Proposition 522 (Extreme Value Theorem) If S a nonempty, compact subset of X and f : S ! R is

continuous, then f admits global maximum and minimum on S, i.e.,

9xmin ; xmax 2 S such that 8x 2 S; f (xmin ) f (x) f (xmax ) :

Proof. From the previous Proposition f (S) is closed and bounded. Therefore, since f (S) is bounded,
there exists M = sup f (S). By de…nition of sup;

8" > 0; B (M; ") \ f (S) 6= ?

Then1 , 8n 2 N;take
1
n 2 B M; \ f (S) :
n
Then, ( n )n2N is such that 8n 2 N, n 2 f (S) and 0 < d ( n ; M ) < n1 . Therefore, n ! M;and since
f (S) is closed, M 2 f (S). But M 2 f (S) means that 9xmax 2 S such that f (xmax ) = M and the fact that
M = sup f (S) implies that 8x 2 S; f (x) f (xmax ). Similar reasoning holds for xmin :
We conclude the section showing a result useful in itself and needed to show the inverse function theorem
- see Section 15.3.

Proposition 523 Let f : X ! Y be a function from a metric space (X; d) to another metric space (Y; d0 ).
Assume that f is one-to-one and onto. If X is compact and f is continuous, then the inverse function f 1
is continuous.
1 The fact that M 2 f (S) can be also proved as follows: from Proposition 464 , M 2 Cl f (S) = f (S), where the last equality

follows from the fact that f (S) is closed.

156 CHAPTER 10. FUNCTIONS

Proof. Exercise.
We are going to use the above result to show that a “ well behaved” consumer problem does have a
solution.
Let the following objects be given.
Price vector p 2 Rn++ ; consumption vector x 2 Rn , consumer’s wealth w 2 R++ , continuous utility
function u : Rn ! R, x 7! u (x). The consumer solves the following problem. For given, p 2 Rn++ ,w 2 R++ ,
…nd x which gives the maximum value to the utility function u under the constraint x 2 C (p; w) de…ned as
n
fx = (xi )i=1 2 Rn : 8i 2 f1; :::; ng ; xi 0 and px wg :

As an application of Propositions 522 and 423, we have to show that for any p 2 Rn++ ,w 2 R++ ;
1. C (p; w) 6= ?;
2. C (p; w) is bounded, and
3. C (p; w) is closed,
1. 0 2 C (p; w) :
2. Clearly if S Rn ;then S is bounded i¤
n n
S is bounded below, i.e., 9x = (xi )i=1 2 Rn such that 8x = (xi )i=1 2 S, we have that 8i 2 f1; :::; ng,
xi xi , and
n n
S is bounded above, i.e., 9x = (xi )i=1 2 Rn such that 8x = (xi )i=1 2 S, we have that 8i 2 f1; :::; ng,
xi xi .
C (p; w) is bounded below by zero, i.e., we can take x = 0. C (p; w) is bounded above because for every
i 2 f1; :::; ng ; P
w i0 6=i pi0 xi0 w
xi ;
pi pi
where the …rst inequality comes from the fact that px w; and the second n o inequality from the fact that
n
p 2 Rn++ and x 2 Rn+ :Then we can take x = (m; m; :::; m), where m = max pwi .
i=1
3. De…ne
n
for i 2 f1; :::; ng ; gi : Rn ! R; x = (xi )i=1 7! xi ;
and
n
h : Rn ! R; x = (xi )i=1 7! w px:
All the above functions are continuous and clearly,
n
C (p; w) = fx = (xi )i=1 2 Rn : 8i 2 f1; :::; ng ; gi (x) 0 and h (x) 0g :

Moreover,
n
C (p; w) = fx = (xi )i=1 2 Rn : 8i 2 f1; :::; ng ; gi (x) 2 [0; +1) and h (x) 2 [0; +1)g =

= \ni=1 gi 1 ([0; +1)) \ h 1

([0; +1)) :
[0; +1) is a closed set and since for any gi is continuous and h is continuous, from Proposition 520.3,the
following sets are closed
8i 2 f1; :::; ng ; gi 1 ([0; +1)) and h 1 ([0; +1)) :
Then the desired result follows from the fact that intersection of closed set is closed.

10.4 Exercises
From Lipschutz (1965), starting from page 61: 30, 32, 34; starting from page 106:19,20.
Chapter 11

Correspondence, maximum theorem

and a …xed point theorem

11.1 Continuous Correspondences

De…nition 524 Let1 two metric spaces (X; dX ) and (Y; dY ) be given. A correspondence, or set-valued
function, ' from X to Y is a rule which associates a subset of Y with each element of X, and it is described
by the notation
' : X !! Y; ' : x 7!7! ' (x) :

Remark 525 In other words, a correspondence ' : X !! Y can be identi…ed with a function from X to
2Y (the set of all subsets of Y ).
Moreover, if we identify x with fxg, a function from X to Y can be thought as a particular correspondence.

Remark 526 Some authors make part of the De…nition of correspondence the fact that ' is not empty
valued, i.e., that 8x 2 X, ' (x) 6= ?.

In what follows, unless otherwise stated, (X; dX ) and (Y; dY ) are assumed to be metric spaces and are
denoted by X and Y , respectively.

De…nition 527 Given U X, ' (U ) = [x2U ' (x) = fy 2 Y : 9x 2 U such that y 2 ' (x)g.

De…nition 528 The graph of ' : X !! Y is

graph ' := f(x; y) 2 X Y : y 2 ' (x)g :

De…nition 529 Consider ' : X !! Y: ' is Upper Hemi-Continuous (UHC) at x 2 X if

' (x) 6= ? and
for every open neighborhood V of ' (x) ; there exists an open neighborhood U of x such that for every
x0 2 U; ' (x0 ) V (or ' (U ) V ).
' is UHC if it is UHC at every x 2 X:

De…nition 530 Consider ' : X !! Y: ' is Lower Hemi-Continuous (LHC) at x 2 X if

' (x) 6= ? and
for any open set V in Y such that ' (x) \ V 6= ?; there exists an open neighborhood U of x such that for
every x0 2 U; ' (x0 ) \ V 6= ?:
' is LHC if it is LHC at every x 2 X:

Example 531 Consider X = R+ and Y = [0; 1], and

[0; 1] if x=0
'1 (x) =
f0g if x > 0:
1 This chapter is based mainly on McLean (1985), Hildebrand (1974), HIldebrand and Kirman (1974) and Ok (2007).

157
158CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

f0g if x=0
'2 (x) =
[0; 1] if x > 0:
'1 is UHC and not LHC; '2 is LHC and not UHC.

Some (partial) intuition about the above de…nitions can be given as follows.
Upper Hemi-Continuity does not allow ”explosions”. In other words, ' is not UHC at x if there exists
a small enough open neighborhood of x such that ' does “explode”, i.e., it becomes much bigger in that
neighborhood.
Lower Hemi-Continuity does not allow ”implosions”. In other words, ' is not LHC at x if there exists
a small enough open neighborhood of x such that ' does “implode”, i.e., it becomes much smaller in that
neighborhood.
In other words, “UHC ) no explosion” and “LHC ) no implosion”( or “explosion ) not UHC” and
“implosion ) not LHC)”. On the other hand, opposite implications are false, i.e.,
it is false that “explosion ( not UHC” and “implosion ( not LHC”, or, in an equivalent manner,
it is false that “no explosion ) UHC” and “no implosion ) LHC”.
An example of a correspondence which neither explodes nor explodes and which is not UHC and not
LHC is presented below.

[1; 2] if x 2 [0; 1)
' : R+ !! R; ' : x 7!7!
[3; 4] if x 2 [1; +1)
' does not implode or explode if you move away from 1 (in a small open neighborhood of 1): on the
right of 1, ' does not change; on the left, it changes completely. Clearly, ' is neither UHC nor LHC (in 1).
The following correspondence is both UHC and LHC:

' : R+ !! R; ' : x 7!7! [x; x + 1]

A, maybe disturbing, example is the following one

' : R+ !! R; ' : x 7!7! (x; x + 1) :

Observe that the graph of the correspondence under consideration “does not implode, does not explode,
does not jump”. In fact, the above correspondence is LHC, but it is not UHC in any x 2 R+ , as veri…ed
below. We want to show that
* +
for every neighborhood V of ' (x) ;
not
there exists a neighborhood U of x such that for every x0 2 U; ' (x0 ) V

i.e.,
* +
there exists a neighborhood V of ' (x) such that:

for every neighborhood U of x there exists x0 2 U such that ' (x0 ) * V

Just take V = ' (x) = (x; x + 1); then for any open neighborhood U of x and, in fact, 8x0 2 U n fxg,
' (x0 ) * V .

Example 532 The correspondence below

[1; 2] if x 2 [0; 1]
' : R+ !! R; ' : x 7!7!
[3; 4] if x 2 [1; +1)

is UHC, but not LHC.

De…nition 533 ' : X !! Y is a continuous correspondence if it is both UHC and LHC.

Remark 534 Summarizing the above results, we can maybe say that a correspondence which is both UHC
and LHC, in fact a continuous correspondence, is a correspondence which agrees with our intuition of a graph
without explosions, implosions or jumps.
11.1. CONTINUOUS CORRESPONDENCES 159

Proposition 535 1. If ' : X !! Y is either UHC or LHC and it is a function, then it is a continuous
function.
2. If ' : X !! Y is a continuous function, then it is a UHC and LHC correspondence.

Proof.
1.
Case 1. ' is UHC.
First proof. Use the fourth characterization of continuous function in De…nition 504.
Second proof. Recall that a function f : X ! Y is continuous i¤ [V open in Y ] ) f 1 (V ) open in X .
Take V open in Y . Consider x 2 f 1 (V ), i.e., x such that f (x) 2 V . By assumption f is UHC and therefore
9 an open neighborhood U of x such that f (U ) V . Then, U f 1 f (U ) f 1 (V ) : Then, for any
x 2 f (V ), we have found an open set U which contains x and is contained in f 1 (V ) ; i.e., f 1 (V ) is
1

open.
Case 2. ' is LHC.
See Remark 542 below.
2.
The results follows from the de…nitions and again from Remark 542 below.

Remark 536 Propositions below partially answer the following question: are Upper and Lower Hemi-
Continuity preserved under set operations?

Proposition 537 Let '1 ; '2 : X !! Y be given. If '1 and '2 are UHC, then

' : X !! Y , x 7! '1 (x) [ '2 (x)

is UHC.

Proof. By assumption, for k = 1; 2; 'k is UHC at x, for any x 2 X. i.e.,

i. 'k (x) 6= ?, and
ii. for any open neighborhood Vk of 'k (x) ; there exists an open neighborhood Uk of x such that
8 x0k 2 Uk ; ' (x0k ) Vk .
We want to show that conditions described in i) and ii) above do hold true for ' := '1 [ '1 .
i’. ' (x) := '1 (x) [ '2 (x) 6= ? simply because, by assumption, '1 (x) 6= ? and '2 (x) 6= ?.
ii’. We want to show that, taken an open neighborhood V of '1 (x) [ '2 (x), then there exists an open
neighborhood U of x such that 8x0 2 U; '1 (x0 ) [ '2 (x0 ) V:
Since V is open and V '1 (x) [ '2 (x), then

V is an open neighborhood of '1 (x) and of '2 (x) : (11.1)

Then, from (11:1) and assumption ii.,

there exists an open neighborhood U1 of x, such that, 8x01 2 U1 ; '1 (x01 ) V; (11.2)
and

there exists an open neighborhood U2 of x, such that, 8x02 2 U2 ; '2 (x02 ) V: (11.3)
Taken U = U1 \ U2 , we want to show that

8x0 2 U; '1 (x0 ) [ '2 (x0 ) V:

def. intersection (11:2) def. intersection (14:36)

Indeed, x0 2 U ) x0 2 U1 ) '1 (x0 ) V and x0 2 U ) x0 2 U2 ) '2 (x0 )
V . Then x 2 U ) ' (x ) := '1 (x0 ) [ '2 (x0 ) V , as desired.
0 0

Proposition 538 Let '1 ; '2 : X !! Y be given. If '1 and '2 are LHC, then

' : X !! Y , x 7! '1 (x) [ '2 (x)

is LHC:
160CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

Proof. By assumption, for k = 1; 2; 'k is LHC at x, for any x 2 X, i.e.,

i. 'k (x) 6= ? and
ii. for any open set Vk in Y such that 'k (x) \ Vk 6= ?; there exists an open neighborhood Uk of x such
that for every x0k 2 Uk ; 'k (x0k ) \ Vk 6= ?.
We want to show i) and ii) for '1 [ '2 .
i’. Obvious.
ii’. We want to show that taken an open set V in Y such that 'k (x) \ V 6= ?, i.e., such that

('1 (x) [ '2 (x)) \ V 6= ?; (11.4)

then there exists an open neighborhood U of x such that, for every x0 2 U; ('1 (x0 ) [ '2 (x0 )) \ V 6= ?.
From (11:4), we have that ('1 (x) \ V )[('2 (x) \ V ) 6= ?. Then, either '1 (x)\V 6= ? or '2 (x)\V 6= ?.
Without loss of generality, assume that '1 (x) \ V 6= ?. Then, from assumption ii., we have that there exists
an open neighborhood U1 of x such that for every x01 2 U1 ; '1 (x01 ) \ V 6= ?. Then,

('1 (x0 ) [ '2 (x0 )) \ V = ('1 (x0 ) \ V ) [ ('2 (x0 ) \ V ) 6= ?;

as desired.

)
Proposition 539 Let the correspondence ' : X !! Y be given. ' is UHC Cl' is UHC.
:

Proof. [)]
Let ' be UHC at x 2 X. Then, by assumption, ' (x) 6= ? and

for every open neighborhood V of ' (x),

(11.5)
there exists an open neighborhood U of x such that for every x0 2 U; ' (x0 ) V:

We want to show that Cl' (x) 6= ?, which follows from the de…nition of Closure, and

for every open neighborhood V1 of Cl' (x),

there exists an open neighborhood U1 of x such that for every x1 2 U1 ; Cl' (x1 ) V1 .

Since Cl (' (x)) V1 , then Cl (' (x)) and V1C are closed disjoint sets. Then from Proposition 489 there
exists open sets H1 and H2 such that

a. H1 \ H2 = ?, b. Cl (' (x)) H1 , and c. V1C H2 . (11.6)

It then su¢ ces to show that for any x 2 U , where U is de…ned in (11:5), we have that Cl' (x) V1 .
Since, from (11:6:b:), we have Cl (' (x)) H1 , it su¢ ces to show that H1 V1 . Suppose otherwise, i.e.,
(11:6:c:)
there exists y 2 H1 \ V1C H2 , contradicting (11:6:a:).
[:]
It is enough to consider ' : (0; 1) !! R, x 7!7! (0; x) (where (0; x) := fz 2 R : 0 < z < xg.
Very often, checking if a correspondence is UHC or LHC is not easy. We present some related concepts
which are more convenient to use.

De…nition 540 ' : X !! Y is “sequentially LHC” at x 2 X if ' (x) 6= ? and

for every sequence (xn )n2N 2 X 1 such that xn ! x, and for every y 2 ' (x),
there exists a sequence (yn )n2N 2 Y 1 such that 8 n 2 N, yn 2 ' (xn ) and yn ! y.
' is ”sequentially LHC” if it is ”sequentially LHC” at every x 2 X.

Proposition 541 2 Let ' : X !! Y be given. ' is LHC at x 2 X , ' is LHC in terms of sequences at
x 2 X.
2 See Proposition 4 page 229 in Ok (2007).
11.1. CONTINUOUS CORRESPONDENCES 161

Proof. [)]
Take
(xn )n2N 2 X 1 such that xn ! x and y 2 ' (x) : (11.7)
We want to …nd (yn ) 2 Y 1 such that 8n 2 N; yn 2 ' (xn ) and yn ! y.
Since, from (11:7), y 2 ' (x), then for any k 2 N, we have B y; k1 \ ' (x) 6= ?. Since, by assumption, '
is LHC , then

1
8k 2 N, 9 k > 0 such that 8xk 2 B (x; k) , we have B y; \ ' (xk ) 6= ?: (11.8)
k

We now construct a strictly increasing sequence (nk )k2N of natural numbers as follows. Since, from
(11:7), xn ! x, we have that
for k = 1, there exists n1 2 N such that 8n n1 1, we have xn 2 B (x; 1 ) and n1 1.
For k = 2, there exists n02 2 N such that 8n n02 , we have xn 2 B (x; 2 ); we can then choose
0
n2 2 fn1 + 1; n1 + 2; :::g, n2 n2 and n2 n1 + 1 2.
Then, it is easy to show by an induction argument that:
For arbitrary k 2 N, we then have that 9nk 2 fnk 1 + 1; nk+1 + 2; :::g N such that 8n nk , we have
xn 2 B (x; k ) and nk nk 1 + 1 k.
Then, we have constructed a strictly increasing sequence (nk )k2N of natural numbers such that

8k 2 N; 9nk 2 N such that 8n 2 fnk ; nk + 1; :::; nk+1 1g ; xn 2 B (x; k) : (11.9)

Then, from (11:9) and (11:8), we have that

1
8k 2 N 9nk 2 N such that 8n 2 fnk ; nk + 1; :::; nk+1 1g ; B y; \ ' (xn ) 6= ?:
k

Then
1
8k 2 N 9nk 2 N such that 8n 2 fnk ; nk + 1; :::; nk+1 1g ; we can take yn 2 B y; \ ' (xn ) : (11.10)
k

We have then constructed a sequence (yn )n2N 2 Y 1 such that 8n 2 N; yn 2 ' (xn ) - observe that, by
construction, [k2N fnk ; nk + 1; :::; nk+1 + 1g = N because, as we have shown above, for any k 2 N, we do
have nk k.
To conclude the proof we are left with showing that

yn ! y: (11.11)

i.e., that 8" > 0 9k 2 N such that 8n k ; yn 2 B (y; ").

Observe that from (11:10), we have that
if k = 1, then for any n 2 fn1 1; :::; n2 1g, we have that yn 2 B y; 11 ;
if k = 2, then for any n 2 fn2 1; :::; n3 1g, we have that yn 2 B y; 12 B y; 11 ,
and
for arbitrary k 2 N, for any n 2 fnk 1; :::; nk+1 1g, we have that yn 2 B y; k1 B y; k 1 1
B y; 11 .
Therefore,
1
8k 2 N 9nk 2 N such that 8n nk , we have yn 2 B y; :
k
Then, taken k 2 N and k > 1" , we have that there exists nk 2 N such that 8n nk , we have
yn 2 B y; k1 B (y; "), as desired.
[(]
Assume otherwise, i.e., there exists an open set V such that

' (x) \ V 6= ? (11.12)

and such that for any open neighborhood U of x, there exists xU 2 U such that ' (xU ) \ V = ?:
162CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

Consider the following family of open neighborhood of x

1
B x; :n2N :
n

Then 8n 2 N; 9 xn 2 B x; n1 , and therefore xn ! x; such that

' (xn ) \ V = ?: (11.13)

From (11:12), we can take y 2 ' (x) \ V: By assumption, we know that there exists a sequence (yn )n2N 2 Y 1
such that 8n 2 N, yn 2 ' (xn ) and yn ! y: Since V is open and y 2 V; 9 n such that n > n implies that
yn 2 V: Therefore, for any n > n,
y 2 ' (xn ) \ V: (11.14)
But (11:14) contradicts (11:13).
Thanks to the above Proposition from now on we talk simply of Lower Hemi-Continuous correspondences.

Remark 542 If ' : X !! Y is LHC and it is a function, then it is a continuous function. The result fol-
lows from the characterization of Lower Hemi-Continuity in terms of sequences and from the characterization
of continuous functions presented in Proposition 520.

Proposition 543 For any i 2 f1; ::; ng, let 'i : X !! Y be LHC at x 2 X. Then
Pn
1) the correspondence x 7!7! i=1 'i (x) : X !! Y is LHC at x.
n
2) the correspondence x 7!7! i=1 'i (x) : X !! Y1 ::: Yn is LHC at x.

Proof. See HIldebrand and Kirman (1976), pages 197-198.

De…nition 544 Given a set A Rn , the convex hull of A is denoted by convA and it is de…ned as follows.3

(m n
)
X X
i
ia 2 Rn : m 2 N and, for i 2 f1; :::; mg ; ai 2 A; i 0; i =1 :
i=1 i=1

Proposition 545 Let the correspondence ' : X !! Rm be LHC at x. Then the correspondence convex
hull of ', i.e., x 7!7! conv' (x) : X !! Rm , is LHC at x.

Proof. See Hildenbrand and Kirman (1976), pages 197-198.

De…nition 546 ' : X !! Y is closed, or "sequentially UHC", at x 2 X if ' (x) 6= ? and

for every sequence (xn )n2N 2 X 1 such that xn ! x, and for every sequence (yn )n2N 2 Y 1 such that
yn 2 ' (xn ) and yn ! y;
it is the case that y 2 ' (x) :
' is closed if it is closed at every x 2 X:

Proposition 547 Let 'i : X !! Y , x 7!7! 'i (x) for i 2 f1; :::; kg be given. If 'i is LHC at x 2 X, then
the following correspondences are LHC as well.
1.
: X !! Y k ; x 7!!7 k
i=1 'i (x) :

2. Assume that Y is a vector space.

k
X
: X !! Y; x 7!7! 'i (x) ;
i=1

3.
conv (') : X !! Rm ; x 7! conv (' (x)) :
3 For a discussion of the concept of convex hull and related concepts see Villanacci, A., (in progress), Basic Convex Analysis,

mimeo, Università degli Studi di Firenze, and the references listed there.
11.1. CONTINUOUS CORRESPONDENCES 163

Proof. See Proposition 8, page 27, Proposition 9, page 28 and Proposition 10, page 28, respectively, in
Hildenbrand (1974).
4
Proposition 548 ' is closed , graph ' is a closed set in X Y.

Proof. An equivalent way of stating the De…nition of closed correspondence is the following one: for
1
every sequence (xn ; yn )n2N 2 (X Y ) such that 8n 2 N, (xn ; yn ) 2 graph ' and (xn ; yn ) ! (x; y), it is
the case that (x; y) 2 graph '. Then, from the characterization of closed sets in terms of sequences, i.e.,
Proposition 403, the desired result follows.

Remark 549 Because of the above result, many author use the expression “' has closed graph” in the place
of “' is closed”.

Remark 550 The de…nition of closed correspondence does NOT reduce to continuity in the case of functions,
as the following example shows.

f0g if x=0
'3 : R+ !! R; '3 (x) = 1
x if x > 0:

' is a closed correspondence, but it is not a continuous function.

y 5

3.75

2.5

1.25

0
0 1.25 2.5 3.75 5

De…nition 551 A set C Rn is convex if 8x1 , x2 2 C and 8 2 [0; 1], (1 ) x1 + x2 2 C.

De…nition 552 ' : X !! Y is closed (non-empty, convex, compact ...) valued if for every x 2 X, ' (x)
is a closed (non-empty, convex, compact ...) set.

)
Proposition 553 Let ' : X !! Y be given. ' closed ' closed valued.
:

Proof. [)]
We want to show that given x 2 X, if (yn )n2N is a sequence of elements in ' (x) which converges to y,
then y 2 ' (x). Setting for any n 2 N; xn = x; we get xn ! x; yn 2 ' (xn ) ; yn ! y. Then, since ' is
closed, y 2 ' (x), as desired.
[:]
'2 in Example 531 is closed valued, but not closed.

;
Remark 554 Let ' : X !! Y be given. ' is UHC ' is closed.
:
[;]
'4 : R+ !! R; '4 (x) = [0; 1)
is UHC and not closed.
[:]
'3 in Remark 550 is closed and not UHC, simply because it is not a continuous “function”.
4 ((X Y ) ; d ) with d ((x; x0 ) ; (y; y 0 )) := max fd (x; x0 ) ; d0 (y; y 0 )g is a metric space.
164CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

Proposition 555 Let a closed correspondence ' : X !! Y and a compact set K X be given. Then
' (K) is closed.
1
Proof. Given (yn )n2N 2 (' (K)) such that yn ! y 2 Y , we want to show that y 2 ' (K).
Since for any n 2 N, we have yn 2 ' (K), then there exists xn 2 K such that yn 2 ' (xn ). Since K is
compact by assumption, the sequence (xn )n2N 2 K 1 admits a subsequence (xv )v2N such that xv ! x 2 K.
Then, we have that xv 2 K X, yv ! y, xv ! x. Then since ' is closed y 2 ' (x) K, as desired.

Proposition 556 Let ' : X !! Y be given. If ' is UHC (at x) and closed valued (at x), then ' is closed
(at x).

Proof. Take an arbitrary x 2 X: We want to show that ' is closed at x, i.e., given (xn )n2N 2 X 1 , if xn !
x; yn 2 ' (xn ) and yn ! y, then y 2 ' (x). Since ' (x) is a closed set, it su¢ ces to show that y 2 Cl ( (x)),
i.e.,5 8" > 0; B (y; ") \ ' (x) 6= ?.
Consider B z; 2" : z 2 ' (x) . Then, [z2'(x) B z; 2" := V is open and contains ' (x) : Since ' is U HC
at x, then there exists an open neighborhood U of x such that

' (U ) V: (11.15)

Since xn ! x 2 U; 9b b;
n 2 N such that 8n > n xn 2 U; and, from (11:15), ' (xn ) V: Since yn 2 ' (xn ),
"
b;
8n > n yn 2 V := [z2'(x) B z; : (11.16)
2
b; 9zn 2 ' (x) such that yn 2 B zn ; 2" and then
From (11:16) ; 8n > n
"
d (yn ; zn ) < : (11.17)
2
Since yn ! y; 9n such that 8n > n ;
"
d (yn ; y) <
: (11.18)
2
From (11:17) and (11:18) ; 8n > max fb n; n g ; zn 2 ' (x) and d (y; z ) d (y; yn ) + d (yn ; z ) < "; i.e.,
zn 2 B (y; ") \ ' (x) and then for any " > 0, B (y; ") \ ' (x) 6= ?, as desired.

Proposition 557 Let ' : X !! Y be given. If ' is closed and there exists a compact set K Y such that
' (X) K, then ' is UHC.
Therefore, in simpler terms, if ' is closed (at x)and Y is compact, then ' is UHC (at x).

Proof. Assume that there exists x 2 X such that ' is not UHC at x 2 X, i.e., there exist an open
neighborhood V of ' (x) such that for every open neighborhood Ux of x; ' (Ux ) \ V C 6= ?: In particular,
8n 2 N; ' B x; n1 \ V C 6= ?: Therefore, we can construct a sequence (xn )n2N 2 X 1 such that xn ! x
and for any n 2 N; ' (xn ) \ V C 6= ?. Now, take yn 2 ' (xn ) \ V C . Since yn 2 ' (X) K and K is compact,
and therefore sequentially compact, up to a subsequence, yn ! y 2 K. Moreover, since 8n 2 N; yn 2 V C
and V C is closed,
y 2 V C: (11.19)
Since ' is closed and xn ! x; yn 2 ' (xn ) ; yn ! y; we have that y 2 ' (x). Since, by assumption,
' (x) V; we have that
y 2 V: (11.20)
But (11:20) contradicts (11:19) :
None of the Assumptions of the above Proposition can be dispensed of. All the examples below show
correspondences which are not UHC.

Example 558 1.
1
2 if x 2 [0; 2]
' : R+ !! R; ' (x) =
f1g if x > 2:
Y = [0; 1] ; but ' is not closed.
5 See Corollary 464.
11.1. CONTINUOUS CORRESPONDENCES 165

2.
f0g if x=0
' : R+ !! R; ' (x) = 1
x if x > 0:
' is closed, but ' (X) = R+ ; which is closed, but not bounded.
3.
f0g if x 2 [0; 1)
' : [0; 1] !! [0; 1) ; ' (x) = 1
2 if x = 1:
' is closed (in Y ), but Y = [0; 1) is not compact. Observe that if you consider

#
f0g if x 2 [0; 1)
' : [0; 1] !! [0; 1 ]; ' (x) = 1 ;
2 if x = 1:

then ' is not closed.

Proposition 559 For any i 2 f1; :::; ng, let the correspondences 'i : X !! Y be compact valued and UHC
at x 2 X. Then
1. The correspondence : X !! Y n ; x 7!7! ni=1 'i (x) :s compact valued P
and UHC at x.
n
2. If Y is a vector space, then the correspondence : X !! Y; : x 7!7! i=1 'i (x) is compact valued
and UHC at x.

Proof. See Propositions 4 and 5, page 25 in Hildenbrand (1974).

Proposition 560 Let the correspondence ' : X !! Rm be compact valued and UHC at x 2 X. Then the
convex hull of ', x 7!7!conv' (x) : X !! Rm , is compact valued and UHC at x.

Proof. See Hildenbrand (1974), page 26.

Proposition 561 Let '1 ; '2 : X !! Y be given. Assume that '1 (x) \ '2 (x) 6= ?. If '1 is compact-valued
and UHC and '2 is closed then

' : X !! Y , such that, x 7! '1 (x) \ '2 (x) , is UHC and compact-valued.

Proof. See Hildenbrand and Kirman (1976), page 195.

In Proposition 570 below, we present a characterization of UHC correspondences in the case of compact
valued correspondences (in the proof of that result, Proposition 568 below is needed).

De…nition 562 Consider ' : X !! Y; V Y:

The strong inverse image of V via ' is

s 1
' (V ) := fx 2 X : ' (x) V g;
:
The weak inverse image of V via ' is

w 1
' (V ) := fx 2 X : ' (x) \ V 6= ?g :

Remark 563 1. 8 V Y; s ' 1 (V ) w

' 1 (V ) :
2. If ' is a function, the usual de…nition of inverse image coincides with both above de…nitions.

Proposition 564 Consider ' : X !! Y:

1.1. ' is UHC , for every open set V in Y; s ' 1 (V ) is open in X;
1.2. ' is UHC , for every closed set V in Y; w ' 1 (V ) is closed in X;
2.1. ' is LHC , for every open set V in Y; w ' 1 (V ) is open in X;
2.2. ' is LHC , for every closed set V in Y; s ' 1 (V ) is closed in X.6
6 Part 2.2 of the Proposition will be used in the proof of the Maximum Theorem.
166CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

Proof.
[1:1:; )] Consider V open in Y . Take x0 2 s ' 1 (V ); by de…nition of s ' 1 ; ' (x0 ) V: By de…nition
of UHC correspondence, 9 an open neighborhood U of x0 such that 8x 2 U; ' (x) 2 V: Then x0 2 U
s
' 1 (V ) :
[1:1:; (] Take an arbitrary x0 2 X and an open neighborhood V of ' (x0 ) : Then x0 2 s ' 1 (V ) and
s
' 1 (V ) is open by assumption. Therefore (just identifying U with s ' 1 (V )), we have proved that ' is
UHC.

To show 1:2, preliminarily, observe that

w 1 C
' (V ) = s' 1
VC : (11.21)

w 1 C
(To see that, simply observe that ' (V ) := fx 2 X : ' (x) \ V = ?g and s ' 1
V C := x 2 X : ' (x) VC )
Assum:; (1:1) s (11:21) C
[1:2:; )] V closed , V C open , ' 1 VC = w
' 1
(V ) open , w ' 1
(V ) closed.
[1:2:; (]
From (1:1:) ; it su¢ ces to show that 8 open set V in Y; s ' 1
(V ) is open in X: Then,
Assum:w s C (11:21)
V open , V C closed , ' 1 V C closed , w ' 1 V C = ' 1 (V ) open.
[2:1 )]
Let V be an open set in Y , we want to show that w ' 1 (V ) is open in X. If w ' 1 (V ) = ? we are done.
Assume then that w ' 1 (V ) 6= ? and take x0 2 w ' 1 (V ) :Then by de…nition of w ' 1 (V ) we have that
' (x0 ) \ V 6= ? and from the assumption that ' is LHC, we have that

there exists an open neighborhood Ux0 of x0 such that 8x 2 Ux0 ; ' (x) \ V 6= ?; (11.22)

Summarizing, we have shown that for any x0 2 w ' 1 (V ), we have there exists an open neighborhood
Ux0 of x0 such that Ux0 w ' 1 (V ), as desired.
[2:1 (]
We want to show that 8x 2 X and any open set V such that ' (x) \ V 6= ?, there exists an open
neighborhood Ux of x such that 8x0 2 Ux ; ' (x0 ) \ V 6= ?.
Since ' (x) \ V 6= ?, then x 2 w ' 1 (V ). By assumption, w ' 1 (V ) is open; then there exists an open
neighborhood Ux of x, such that Ux w ' 1 (V ) ; i.e., 8x0 2 Ux ; ' (x0 ) \ V 6= ? as desired.
[2:2 )]
assum:;(2:1) w C (11:21)
V closed , V C open , w
' 1 V C is open , ' 1 VC = s ' 1 (V ) is closed.
[2:2 (]
From 2.1 it su¢ ces to show that for any open set V in Y , w ' 1 (V ) is open in X.
assum: s s C (11:21) w
V open , V C closed ) ' 1
V C closed , ' 1
VC = ' 1
(V ) open.

;
Remark 565 Observe that ' is UHC for every closed set V in Y; s ' 1
(V ) is closed in X:
:
[;]
Consider
[0; 2] if x 2 [0; 1]
' : R+ !! R; ' (x) =
[0; 1] if x > 1:

' is UHC and [0; 1] is closed, but s ' 1

([0; 1]) := fx 2 R+ : ' (x) [0; 1]g = (1; +1) is not closed.
[:]
Consider
0; 21 [ f1g if x=0
' : R+ !! R+ ; ' (x) =
[0; 1] if x > 0:

For any closed set in Y := R+ , s ' 1 (V ) can be only one of the following set, and each of them is closed:
f0g ; R+ ; ?: On the other hand, ' is not UHC in 0:
11.1. CONTINUOUS CORRESPONDENCES 167

De…nition 566 Let the vector spaces (X; dX ) ; (Y; dY ) and (Z; dZ ) and the correspondences ' : X !!
Y; : Y !! Z be given. The composition of ' with is
' : X !! Z;

( ') (x) := [y2'(x) (y) = fz 2 Z : 9x 2 X such that z 2 (' (x))g

Proposition 567 Consider ' : X !! Y; : Y !! Z: If ' and are UHC, then ' is UHC.

Proof.
1 1
Step 1. s ( ') (V ) = s
' 1 s
(V ) :

s 1
( ') (V ) = fx 2 X : (' (x)) V g = fx 2 X : 8y 2 ' (x) ; (y) Vg=

s 1 s 1 1
= x 2 X : 8y 2 ' (x) ; y 2 (V ) = x 2 X : ' (x) (V ) = s ' 1 s
(V ) :

Step 2. Desired result.

1
Take V open in Z. From Theorem 564, we want to show that s ( ') (V ) is open in X: From
s 1 s 1 s 1 s 1
step 1, we have that (' ) (V ) = ' (V ) : Now, (V ) is open because is UHC, and
s 1 s 1
' (V ) is open because ' is UHC.

Proposition 568 Consider ' : X !! Y: If ' is UHC and compact valued, and A X is a compact set,
then ' (A) is compact.

Proof.
Consider an arbitrary open cover fC g 2I for ' (A) : Since ' (A) := [x2A ' (x) and ' is compact valued,
there exists a …nite set Nx I such that

' (x) [ 2Nx C := Gx : (11.23)

Since for every 2 Nx ; C is open, then Gx is open. Since ' is UHC, s ' 1 (Gx ) is open. Moreover,
x 2 s ' 1 (Gx ): this is the case because, by de…nition, x 2 s ' 1 (Gx ) i¤ ' (x) Gx ; which is just (11:23) :
Therefore, s ' 1 (Gx ) x2A is an open cover of A: Since, by assumption, A is compact, there exists a …nite
m
set fxi gi=1 A such that A [m i=1
s
' 1 (Gxi ) : Finally,

(1) (2)
' [m
' (A)
i=1
s
' 1 (Gxi ) [mi=1 '
s
' 1 (Gxi ) [m m
i=1 Gxi = [i=1 [ 2Nxi C ;
n om
and fC g 2Nx is a …nite subcover of fC g 2I . We are left with showing (1) and (2) above.
i i=1
(1) : In general, it is the case that ' ([m
i=1 Si ) [m
i=1 ' (Si ) :
y 2 ' ([i=1 Si ) , 9x 2 [i=1 Si such that y 2 ' (x) ) 9i such that y 2 ' (x) ' (Si ) ) y 2 [m
m m
i=1 ' (Si ) :
(2) : In general, it is the case that ' s ' 1 (A) A:
y 2 ' s ' 1 (A) ) 9x 2 s ' 1 (A) such that y 2 ' (x). But, by de…nition of s ' 1 (A) ; and since
x 2 s ' 1 (A) ; it follows that ' (x) A and therefore y 2 A:

Remark 569 Observe that the assumptions in the above Proposition cannot be dispensed of, as veri…ed
below.
Consider ' : R+ !! R ; ' (x) = [0; 1). Observe that ' is U HC and bounded valued but not closed valued
, and ' ([0; 1]) = [0; 1) is not compact.
Consider ' : R+ !! R ; ' (x) = R+ . Observe that ' is U HC and closed valued, but not bounded valued,
and ' ([0; 1]) = R+ is not compact.
fxg if x 6= 1
Consider ' : R+ !! R+ ; ' (x) = Observe that ' is not U HC and ' ([0; 1]) =
f0g if x = 1:
[0; 1) is not compact.
168CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

Proposition 570 Let a compact valued correspondence ' : X !! Y be given. Then

' it is UHC at x
,
i. ' (x) 6= ?, and
ii. for every (xn )n2N 2 X 1 such that xn ! x and for every (yn )n2N 2 Y 1 such that yn 2 ' (xn ), there
exists a subsequence (y ) 2N of (yn )n2N , such that y ! y 2 ' (x).

Proof. [)]
Since xn ! x, then K := fxn : n 2 Ng is a compact set (just use the de…nition of compact sets in terms
of open sets or the characterization of compactness in terms of sequential compactness). Since ' is UHC
and compact valued by assumption and since K is compact, then from Proposition 568, we have that ' (K)
is compact. Since for any n 2 N, yn 2 ' (xn ) ' (K), then fyn : n 2 Ng ' (K) and therefore it admits a
subsequence (y ) 2N such that y ! y 2 ' (K). We are left with showing that y 2 ' (x). Since ' is compact
valued, then it is closed valued and therefore since ' is UHC, then from Proposition 556, ' is closed at
x:Then since xv ! x, y ! y and for any v 2 N, y 2 ' (x ), we have that y 2 ' (x), as desired.
[(]
Suppose otherwise, i.e., ' is not UHC at x 2 X, i.e.,

there exists an open neighborhood V of ' (x) such that for any open neighborhood U of x,
(11.24)
there exists x0 2 U such that ' (x0 ) \ V C 6= ?.

Then, for any n 2 N, there exists x0n 2 B x; n1 such that ' (x0n ) \ V C 6= ?. Therefore, x0n ! x and

there exists a sequence (yn )n2N such that for any n 2 N, yn 2 ' (x0n ) \ V C : (11.25)

By assumption, there exists a subsequence (y ) 2N of (yn )n2N , such that

(11:24)
y ! y 2 ' (x) V: (11.26)

Then, from (11:24), for any 2 N, we have y 2 V C , a closed set and, therefore, since y ! y, then we do
have y 2 V C , contradicting (11:26).

Remark 571 Below, we summarize some facts we showed in the present Section, in a somehow informal
manner.

; ;
h(if ' is a fcn., it is cnt.i , h' is UHCi h' is sequentially UHC,i.e., closedi h(if ' is a fcn., it is cnt.i
: (

h(if ' is a fcn, it is continuousi , h' is LHCi , h' is sequentially LHCi

h' UHC and closed valued at xi ) h' is closed at xi

h' UHC at x i ( h' is closed at x and Im ' contained in a compact seti

11.2 The Maximum Theorem

Theorem 572 (Maximum Theorem) Let the metric spaces ( ; d ) ; (X; dX ), the correspondence : !!
X and a function u : X ! R be given.7 De…ne

: !! X;
( ) = fz 2 ( ) : 8x 2 ( ) ; u (z; ) u (x; )g = arg maxx2 ( ) u (x; ) ;

Assume that
is non-empty valued, compact valued and continuous,
7 Obviously, stands for “budget correspondence” and u for “utility function”.
11.2. THE MAXIMUM THEOREM 169

u is continuous.
Then
1. is non-empty valued, compact valued, UHC and closed,and
2.
v : ! R; v : 7! max u (x; ) :
x2 ( )

is continuous.

Proof.

is non-empty valued.

It is a consequence of the fact that is non-empty valued and compact valued and of the Extreme Value
Theorem - see Proposition 522.

is compact valued.

We are going to show that for any 2 ; ( ) is a sequentially compact set. Consider a sequence
(xn )n2N 2 X 1 such that fxn : n 2 Ng ( ) : Since ( ) ( ) and ( ) is compact by assumption,
without loss of generality, up to a subsequence, xn ! x0 2 ( ) : We are left with showing that x0 2 ( ) :
Take an arbitrary z 2 ( ). Since fxn : n 2 Ng ( ) ; we have that u (xn ; ) u (z; ) : By continuity of
u; taking limits with respect to n of both sides, we get u (x0 ; ) u (z; ) ; i.e., x0 2 ( ), as desired.

is UHC.

From Proposition 564, it su¢ ces to show that given an arbitrary closed set V in X; w 1 (V ) :=
f 2 : ( ) \ V 6= ?g is closed in : Consider an arbitrary sequence ( n )n2N such that f n : n 2 Ng
1
w
(V ) and such that n ! 0 . We have to show that 0 2 w 1 (V ).
Take a sequence (xn )n2N 2 X 1 such that for every n, xn 2 ( n ) \ V 6= ?. Since ( n ) ( n ) ; it
follows that xn 2 ( n ). We can now show the following
Claim. There exists a subsequence (xnk )nk 2N of (xn )n2N such that xnk ! x0 and x0 2 ( 0 ) :
Proof of the Claim.
Since f n : n 2 Ng [ f 0 g is a compact set (Show it), and since, by assumption, is UHC and compact
valued, from Proposition 568, (f n : n 2 Ng [ f 0 g) is compact. Since fxn gn (f n g [ f 0 g) ; there
exists a subsequence (xnk )nk 2N of (xn )n2N which converges to some x0 : Since is compact valued, it is
closed valued, too. Then, is UHC and closed valued and from Proposition 556, is closed. Since

nk ! 0; xnk 2 ( nk ) ; xnk ! x0 ;

the fact that is closed implies that x0 2 ( 0 ).

End of the Proof of the Claim.
Choose an arbitrary element z0 such that z0 2 ( 0 ). Since we assumed that n ! 0 and since is
LHC, there exists a sequence (zn )n2N 2 X 1 such that zn 2 ( n ) and zn ! z0 .
Summarizing, and taking the subsequences of ( n )n2N and (zn )n2N corresponding to (xnk )nk 2N , we have
for any nk ;
nk ! 0 ;
xnk ! x0 ; xnk 2 ( nk ) ; x0 2 ( 0 ) ;
znk ! z0 ; znk 2 ( nk ) ; z0 2 ( 0 ) :
Then for any nk ; we have that u (xnk ; nk ) u (znk ; nk ) : Since u is continuous, taking limits, we get
that u (x0 ; 0 ) u (z0 ; 0 ) :Since the choice of z0 in ( 0 ) was arbitrary, we have then x0 2 ( 0 ) :
Finally, since (xnk )nk 2N 2 V 1 ; xnk ! x0 and V is closed, x0 2 V: Then x0 2 ( 0 ) \ V and 0 2
f 2 : ( ) \ V 6= ?g := w 1 (V ) ; which was the desired result.

is closed.

is UHC and compact valued, and therefore closed valued. Then, from Proposition 556, it is closed, too.
170CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

v is a continuous function.
The basic idea of the proof is that v is a function and “it is equal to” the composition of UHC
correspondences; therefore, it is a continuous function. A precise argument goes as follows.
Let the following correspondences be given:

( ; id) : !! X ; 7! ( ) f g;

:X !! R; (x; ) 7! fu (x; )g :

Then, from De…nition 566,

( ( ; id)) ( ) = [(x; )2 ( ) f g fu (x; )g :

By de…nition of ,

8 2 ; 8x 2 ( ) ; [(x; )2 ( ) f g fu (x; )g = fu (x; )g ;

and
8 2 ; ( ( ; id)) ( ) = fu (x; )g = fv ( )g : (11.27)

Now, ( ; id) is UHC, and since u is a continuous function, is UHC as well. From Proposition 567,
( ; id) is UHC and, from 11.27, v is a continuous function.

A sometimes more useful version of the Maximum Theorem is one which does not use the fact that is
UHC.

Theorem 573 (Maximum Theorem) Consider the correspondence : !! X and the function u :
X ! R de…ned in Theorem 572 and ; X Euclidean spaces.
Assume that
is non-empty valued, compact valued, convex valued, closed and LHC.
u is continuous.
Then
1. is a non-empty valued, compact valued, closed and UHC correspondence;
2. v is a continuous function.

Proof.
The desired result follows from next Proposition.

Proposition 574 Assume that ' : X !! Y is non-empty valued, compact valued, convex valued, closed
and LHC. Then ' is UHC.

Proof.
Since8 by assumption, ' is compact valued, we can apply Proposition 570, and therefore it su¢ ces to
show that for every (xn )n2N 2 X 1 such that xn ! x and for every (yn )n2N 2 Y 1 such that yn 2 ' (xn ),
there exists a subsequence (y ) 2N of (yn )n2N , such that y ! y 2 ' (x).
Indeed, it su¢ ces to show that (y ) 2N is bounded: if that is the case, then it admits a convergent
subsequence, i.e., there exists a subsequence (y 0 ) 0 2N of (y ) 2N such that y 0 ! y. Moreover, for any
0
2 N, y 0 2 ' (x 0 ) and x 0 ! x. Therefore, since ' is closed by assumption, we do have that y 2 ' (x)
and our proof is complete. Therefore, below we prove that (y ) 2N is bounded.
Since, by assumption, ' is LHC, then, from Proposition 541, for every sequence (xn )n2N 2 X 1 such
that xn ! x, and for every z 2 ' (x),
there exists a sequence (zn )n2N 2 Y 1 such that 8 n 2 N, zn 2 ' (xn ) and zn ! z. Since for any n 2 N,
yn 2 ' (xn ) and since ' is convex valued by assumption, we have that for any n 2 N, [yn ; zn ] ' (xn ),
where [yn ; zn ] := f(1 ) yn + xn : 2 [0; 1]g is the segment from yn to zn .
8 The proof is taken from Hildenbrand (1974), Lemma 1, page 33. I added some details which should be carefully

checked.
11.2. THE MAXIMUM THEOREM 171

Now, suppose our claim is false, i.e., (yn )n2N is unbounded.

Claim 1. For any " > 0, there exists N" 2 N such that for any n > N" , there exists yn0 2 [yn ; zn ] such
that d (yn0 ; ' (x)) = ".
Proof of Claim 1.
De…ne B (' (x) ; ") = fw 2 Y : d (w; ' (x)) < "g which is a bounded and convex set containing the
(bounded, convex) set ' (x). Since, by assumption, (yn )n2N is unbounded and zn ! z 2 ' (x), then
9N" 2 N such that 8n > N" , yn 2 = B (' (x) ; ") and zn 2 B (' (x) ; "). Then, (exercise)9 there exists
0
yn 2 [yn ; zn ] \ F (B (' (x) ; ")), where F (S) is the boundary or frontier of the set S. We are left with
showing that 8n > N" ; d (yn0 ; ' (x)) = ";which is done below.
Suppose d (yn0 ; ' (x)) < ". Since d(; ' (x)) : X ! R is a continuous function - from Proposition 495 -
then B (' (x) ; ") is open and then yn0 2 B (' (x) ; ") = Int (B (' (x) ; ")), contradicting the fact that yn0 2
F (B (' (x) ; ")) :
Suppose that d (yn0 ; ' (x)) > ". Since d(; ' (x)) : X ! R is a continuous function, then there exists r > 0
C
such that 8w 2 B (yn0 ; r), d (w; ' (x)) > ", i.e., yn0 2 Int (fu 2 Y : d (u; ' (x)) > "g) = Int fu 2 Y : d (u; ' (x)) "g
C C
Int fu 2 Y : d (u; ' (x)) < "g = Int B (' (x) ; ") , again contradicting the fact that yn0 2 F (B (' (x) ; ")) :
End of the proof of Claim 1.
Claim 2. (yn0 )n2N is bounded.
Proof of Claim 2.
We want to show that there exists M 0 2 R++ such that 8n; m 2 N, d (yn0 ; ym
0
) < M 0.
From Proposition 351 in villanacci, Convex Analysis, mimeo, or Proposition 1.9.1 in Webster (1994) , we
have the following result.
Let A be a nonempty closed set in Rn and let x 2 Rn . Then 9a0 2 A such that d(x; A) = kx a0 k.
Then, since ' (x) is compact and therefore closed, we have that for any n; m 2 N, there exists zn ; zm 2
' (x) such that
Claim 1 Claim 1
" = d (yn0 ; ' (x)) = d (yn0 ; zn ) and " = 0
d (ym 0
; ' (x)) = d (ym ; zm ) :

Then,
d (yn0 ; ym
0
) d (yn0 ; zn ) + d (zn ; zm ) + d (zm ; ym
0
) = " + M + ";
as desired.
End of the proof of Claim 2.
We are now ready to get the desired contradiction. From Claim 2, up to a subsequence, we have that
yn0 ! ye, and since from Claim 1, for any n > N" , d (yn0 ; ' (x)) = ", and d(; ' (x)) : X ! R is a continuous
function, we do have d (e
y ; ' (x)) = " > 0 and therefore

ye 2
= ' (x) : (11.28)

Observe also that we have found

(xn )n2N 2 X 1 such that xn ! x and (yn0 )n2N 2 Y 1 such that yn0 2 [yn ; zn ] ' (xn ) and yn0 ! ye.
Since, by assumption, ' is closed, then
ye 2 ' (x) ;
contradicting (11:28), as desired.

The following result allows to substitute the requirement “ is LHC”with the easier to check requirement
“Cl is LHC”.

Proposition 575 Consider the correspondence ' : !! X: ' is LHC , Cl is LHC.

Proof.
Preliminary Claim.
V open set, ; Cl ( ) \ V 6= ? ) ' ( ) \ V 6= ?:
Proof of the Preliminary Claim.
Take z 2 Cl ( ) \ V 6= ?. Since V is open, 9" > 0 such that B (z; ") V: Since z 2 Cl ( ) ;
9 fzn g ' ( ) such that zn ! z. But then 9n" such that n > n" ) zn 2 B (z; ") V . But zn 2 V and
zn 2 ' ( ) implies that ' ( ) \ V 6= ?:
9 That result is used to show that a bounded set is Peano measurable i¤ its boundary is Peano measurable.
172CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

End of the Proof of the Preliminary Claim.

[)]
Take an open set V such that Cl ( ) \ V 6= ?: We want to show that there exists an open set U
such that 2 U and 8 2 U ; Cl ( ) \ V 6= ?: From the preliminary remark, it must be the case that
' ( )\V 6= ?: Then, since ' is LHC, there exists an open set U such that 2 U and 8 2 U ; ' ( )\V 6= ?:
Since Cl ( ) ' ( ) ; we also have Cl ( ) \ V 6= ?: Choosing U = U; we are done.
[(]
Since ' ( ) \ V 6= ?; then Cl ( ) \ V 6= ?; and, by assumption, 9 open set U 0 such that 2 U 0 and
8 2 U 0 ; Cl ( ) \ V 6= ?: Then, from the preliminary remark, it must be the case that
' ( ) \ V 6= ?:

Remark 576 In some economic models, a convenient strategy to show that a correspondence is LHC
is the following one. Introduce a correspondence b ; show that b is LHC; show that Cl b = . Then from
the above Proposition 575, the desired result follows - see, for example, point 5 the proof of Proposition 591
below.

We can summarize what said above in the following result.

Theorem 577 (Maximum Theorem: summary) Let the metric spaces ( ; d ) ; (X; dX ), the correspondence
: !! X and a function u : X ! R be given.10 De…ne

: !! X;
( ) = fz 2 ( ) : 8x 2 ( ) ; u (z; ) u (x; )g = arg maxx2 ( ) u (x; ) ;

Assume that
1. is non-empty valued, compact valued and either
a. continuous,
b. convex valued, closed and LHC, or
c. convex valued, closed and such that Cl (') is LHC,
2. u is continuous.
Then
1. is non-empty valued, compact valued, usc and closed,and
2.
v : ! R; v : 7! max u (x; ) :
x2 ( )

is continuous.

Proof. We list precise references for the proof of cases a., b. and c of Assumption 1.
Assumption 1a.
Theorem 572.
Assumption 1b.
Theorem 573.
Assumption 1c.
It follows from Proposition 573.

Remark 578 There are other version of the maximum theorems; in a footnote on page 306, Ok (2007)
points out the existence of two more general versions of the theorem by Walker (1979) and by Leininger
(1984).

11.3 Fixed point theorems

A thorough analysis of the many versions of …xed point theorems existing in the literature is outside the
scope of this notes. Below, we present a useful relatively general version of …xed point theorems both in the
case of functions and correspondences.
1 0 Obviously, stands for “budget correspondence” and u for “utility function”.
11.4. APPLICATION OF THE MAXIMUM THEOREM TO THE CONSUMER PROBLEM 173

Theorem 579 (The Brouwer Fixed Point Theorem)

For any n 2 N, let S be a nonempty, compact, convex subset of Rn . If f : S ! S is a continuous function,
then 9x 2 S such that f (x) = x.

Proof. For a (not self-contained) proof, see Ok (2007), page 279.

Just to try to avoid having a Section without a proof, let’s show the following extremely simple version
of that theorem.

Proposition 580 If f : [0; 1] ! [0; 1] is a continuous function, then 9x 2 [0; 1] such that f (x) = x.

Proof. If f (0) = 0 or f (1) = 1, the result is true. Then suppose otherwise, i.e., f (0) 6= 0 and f (1) 6= 1,
i.e., since the domain of f is [0; 1], suppose that f (0) > 0 and f (1) < 1. De…ne

g : [0; 1] ! R; : x 7! x f (x) :

Clearly, g is continuous, g (0) = f (0) < 0 and g (1) = 1 f (1) > 0. Then, from the intermediate value
for continuous functions, 9x 2 [0; 1] such that g (x) = x f (x) = 0, i.e., x = f (x), as desired.

Theorem 581 (Kakutani’s Fixed Point Theorem) For any n 2 N, if

1. X is a nonempty, compact, convex subset of Rn , and
2. if ' : X !! X is a nonempty and convex valued,and
a. closed correspondence, or
b. closed valued and usc, or
c. closed valued and lsc,
then 9x 2 X such that ' (x) 3 x.

Proof. We list precise references for the proof of cases a., b. and c of Assumption 2.
Assumption 2a.
See Ok (2007), page 331, or Hildebrand (1974) page 39)
Assumption 2b.
From Proposition 556, or Proposition 3b, page 295, Ok (2007),

h' usc and closed valued at xi ) h' is closed at xi

Assumption 2c.
See Ok (2007), Corollary 1, page 337.

11.4 Application of the maximum theorem to the consumer prob-

lem
De…nition 582 (Mas Colell (1996), page 17) Commodities are goods and services available for purchases
in the market.

We assume the number of commodities is …nite and equal to C. Commodities are indexed by superscript
c = 1; :::; C:

De…nition 583 A commodity vector is an element of the commodity space RC :

De…nition 584 (almost Mas Colell(1996), page 18) A consumption set is a subset of the commodity
space RC . It is denoted by X. Its elements are the vector of commodities the individual can conceivably
consume given the physical or institutional constraints imposed by the environment.

Example 585 See Mas colell pages 18, 19.

Common assumptions on X are that it is convex,bounded below and unbounded. Unless otherwise stated,
we make the following stronger

Assumption 1 X = RC C
+ := x 2 R : x 0 :
174CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

De…nition 586 p 2 RC is the vector of commodity prices.

Households’ choices are limited also by an economic constraint: they cannot buy goods whose value is
bigger than their wealth, i.e., it must be the case that px w; where w is household’s wealth.

Remark 587 w can take di¤ erent speci…cations. For example, we can have w = pe; where e 2 RC is the
vector of goods owned by the household; i.e., her endowments.

Assumption 2 All commodities are traded in markets at publicly observable prices, expressed in monetary
unit terms.
Assumption 3 All commodities are assumed to be strictly goods (and not ”bad”), i.e., p 2 RC
++ .

Assumption 4 Households behave as if they cannot in‡uence prices.

De…nition 588 The budget set is

(p; w) := x 2 RC
+ : px w :
With some abuse of notation we de…ne the budget correspondence as

: RC
++ R++ !! RC ; (p; w) = x 2 RC
+ : px w :

De…nition 589 The utility function is

u : X ! R; x 7! u (x)

De…nition 590 The Utility Maximization Problem (U M P ) is

maxx2RC+ u (x) s:t: px w; or x 2 (p; w) :

: RC
++ R++ !! RC ; (p; w) = arg max (U M P ) is the demand correspondence.

Theorem 591 is a non-empty valued, compact valued, closed and UHC correspondence and

v : RC
++ R++ ! R; v : (p; w) 7! max (U M P ) ;

i.e., the indirect utility function, is a continuous function.

Proof.
As an application of the (second version of) the Maximum Theorem, i.e., Theorem 573, we have to show
that is non-empty valued, compact valued, convex valued, closed and LHC.
1. is non-empty valued.
C
w
x = Cp c 2 (p; w) (or, simpler, 0 2 (p; w)).
c=1
2. is compact valued.
(p; w) is closed because is the intersection of the inverse image of closed sets via continuous functions.
(p; w) is bounded below by zero.
P 0 0
w 0 pc xc
w
(p; w) is bounded above because for every c; xc c 6=c
pc pc ; where the …rst inequality comes
from the fact that px w; and the second inequality from the fact that p 2 RC C
++ and x 2 R+ :
3. is convex valued.
To see that, simply, observe that (1 ) px0 + px00 (1 ) w + w = w:
4. is closed.
We want to show that for every sequence f(pn ; wn )gn RC ++ R++ such that
(pn ; wn ) ! (p; w) ; xn 2 (pn ; wn ) ; xn ! x;
it is the case that x 2 (p; w) :
Since xn 2 (pn ; wn ), we have that pn xn wn and xn 0: Taking limits of both sides of both
inequalities, we get px w and x 0;i.e., x 2 (p; w) :
5. is LHC.
We proceed as follows: a. Int is LHC; b. Cl Int = . Then, from Proposition 575 the result follows.
11.4. APPLICATION OF THE MAXIMUM THEOREM TO THE CONSUMER PROBLEM 175

a. Observe that Int (p; w) := x 2 RC

+ :x 0 and px < w and that Int (p; w) 6= ?; since x =
C
w
2Cpc 2 Int (p; w) : We want to show that the following is true.
c=1
1
For every sequence (pn ; wn )n 2 RC++ R++ such that (pn ; wn ) ! (p; w) and for any x 2 Int (p; w) ;
there exists a sequence fxn gn RC + such that 8n; xn 2 Int (pn ; wn ) and xn ! x:
pn x wn ! px w < 0 (where the strict inequality follows from the fact that x 2 Int (p; w) : Then,
9N such that n N ) pn x wn < 0:
For n N; choose an arbitrary xn 2 Int (pn ; wn ) 6= ?: Since pn x w < 0, for every n > N; there exists
"n > 0 such that z 2 B (x; "n ) ) pn z wn < 0:
For any n > N; choose xn = x + p1C min "2n ; n1 1. Then,

! 12
2
1 "n 1 "n 1
d (x; xn ) = C p min ; = min ; < "n ;
C 2 n 2 n

i.e., xn 2 B (x; "n ) and therefore

pn xn wn < 0 (1) :
Since xn x; we also have

xn 0 (2) :
(1) and (2) imply that xn 2 Int (pn ; wn ) : Moreover, since xn x; we have 0 5 limn!+1 (xn x) =
limn!1 p1C min "2n ; n1 1 5 limn!1 n1 p1C 1 = 0; i.e., limn!+1 xn = x:11
b.
It follows from the fact that the budget correspondence is the intersection of the inverse images of half
spaces via continuous functions.
2.
It follows from Proposition 592, part (4), and the Maximum Theorem.

Proposition 592 For every (p; w) 2 RC ++ R++ ;

(1) 8 2 R++ ; ( p; w) = (p; w) ;
(2) if u is LNS, 8x 2 RC + ; x 2 (p; w) ) px = w;
(3) if u is quasi-concave12 , is convex valued;
(4) if u is strictly quasi-concave,13 is single valued, i.e., it is a function.

Proof.
(1)
It simply follows from the fact that 8 2 R++; ( p; w) = (p; w) :
(2)
Suppose otherwise, then 9x0 2 RC + such that x 2
0
(p; w) and px0 < w: Therefore,9"0 > 0 such that
0 0 0 0
B (x ; " ) (p; w) (take " = d (x ; H (p; w))). Then, from the fact that u is LNS, there exists x such that
x 2 B (x0 ; "0 ) (p; w) and u (x ) > u (x0 ), i.e., x0 2 = (p; w) ; a contradiction.
(3)
1 1 Or simply
1
0 lim d (x; xn ) lim = 0:
n!1 n!1 n
12 A continuous function f is quasi-concave i¤ 8x0 ; x00 2 X; 8 2 [0; 1],
0 00
f ((1 )x + x ) min f x0 ; f (x00 ) :

13

De…nition 593 f is strictly quasi-concave

i¤ 8 x0 ; x00 2 X; such that x0 6= x00 ; and 8 2 (0; 1), we have that
f ((1 )x0 + x00 ) > min f x0 ; f (x00 ) :
176CHAPTER 11. CORRESPONDENCE, MAXIMUM THEOREM AND A FIXED POINT THEOREM

Assume there exist x0 ; x00 such that x0 ; x00 2 (p; w). We want to show that 8 2 [0; 1] ; x := (1 ) x0 +
00 0 00
x 2 (p; w) : Observe that u (x ) = u (x ) := u . From the quasi-concavity of u; we have u x u : We
are therefore left with showing that x 2 (p; w) ; i.e., is convex valued. To see that, simply, observe that
px = (1 ) px0 + px00 (1 ) w + w = w:
(4) Assume otherwise. Following exactly the same argument as above we have x0 ; x00 2 (p; w) ; and
px w: Since u is strictly quasi concave, we also have that u x > u (x0 ) = u (x00 ) := u ; which
0 00
contradicts the fact that x ; x 2 (p; w) :

Proposition 594 If u is a continuous LNS utility function, then the indirect utility function has the follow-
ing properties.
For every (p; w) 2 RC ++ R++ ;
(1) 8 2 R++ ; v ( p; w) = v (p; w) ;
(2) Strictly increasing in w and for every c; non increasing in pc ;
(3) for every v 2 R, f(p; w) : v (p; w) vg is convex.
(4) continuous.

Proof.
(1) It follows from Proposition 592 (2) :
(2)
If w increases, say by w, then, from Proposition 592 (2) ; px (p; w) < w + w: De…ne x (p; w) := x0 .
Then,9"0 > 0 such that B (x0 ; "0 ) (p; w + w) (take "0 = d (x0 ; H (p; w + w))). Then, from the fact
that u is LNS, there exists x such that x 2 B (x0 ; "0 ) (p; w + w) and u (x ) > u (x0 ). The result
follows observing that v (p; w + w) u (x ) :
0 C
Similar proof applies to the case of a decrease in p. Assume pc < 0: De…ne := ( c )c=1 2 RC with
0 0
c
= 0 i¤ c 6= c0 and c = pc . Then,
( 0)
0 0
px (p; w) = w ) (p + ) x (p; w) = px (p; w) + pc xc (p; w) =
0 0
= w + pc xc (p; w) w: The remaining part of the proof is the same as in the case of an increase of w.
(3) Take (p0 ; w0 ) ; (p00 ; w00 ) 2 f(p; w) : v (p; w) vg := S (v) : We want to show that 8 2 [0; 1] ; p ; w :=
(1 ) (p0 ; w0 ) + (p00 ; w00 ) 2 S (v) ; i.e., x 2 p ; w ) u (x) > v:
x 2 p ; w ) p x w , (1 ) p0 + p00 (1 ) w0 + w00 :
Then, either p x w or p x w : If p x w , then u (x) v (p0 ; w0 ) v: Similarly, if p00 x w00 .
0 0 00 00 0 0

(4)
It was proved in Theorem 591.
 Part III

Di¤erential calculus in Euclidean

spaces

177
Chapter 12

Partial derivatives and directional

derivatives

12.1 Partial Derivatives

The1 concept of partial derivative is not that di¤erent from the concept of “standard”derivative of a function
from R to R, in fact we are going to see that partial derivatives of a function f : Rn ! R are just standard
derivatives of a naturally associated function from R to R:
Recall that for any k 2 f1; :::; ng, ekn = (0; ::; 1; :::; 0) is the k th vector in the canonical basis of Rn .
n
De…nition 595 Let a set S Rn , a point x0 = (x0k )k=1 2 Int S and a function f : S ! R be given. If
the following limit exists and it is …nite

f x0 + hekn f (x0 ) (h:=xk x0k ) f x0 + (xk x0k ) ekn f (x0 )

lim = lim ; (12.1)
h!0 h xk !x0k xk x0k

then it is called the partial derivative of f with respect to the k th coordinate computed in x0 and it is
denoted by any of the following symbols
@f @f (x)
Dxk f (x0 ) ; Dk f (x0 ) ; @xk (x0 ) ; @xk jx=x0 :

Remark 596 As said above, partial derivatives are not really a new concept. We are just treating f as a
function of one variable at the time, keeping the other variables …xed. In other words, for simplicity taking
S = Rn and using the notation of the above de…nition, we can de…ne

gk : R ! R; gk (xk ) = f x0 + (xk x0k ) ekn

a function of only one variable, and

(1) gk (xk ) gk (x0k ) (2)

gk0 (x0k ) = limxk !x0k xk x0k =

f (x0 +(xk x0k )ek

n ) f (x0 +(x0k x0k )ek
n)
= limxk !x0k = (12.2)
xk x0k

f (x0 +(xk x0k )ek

n ) f (x0 ) (3)
= limxk !x0k xk x0k = Dxk f (x0 ) :

where
(1) follows from the de…nition of derivative of a function from R to R ,
(2) from the de…nition of the function gk ,
(3) from the de…nition of partial derivative.
1 In this Part, I follow closely Section 5.14 and chapters 12 and 13 in Apostol (1974).

179
180 CHAPTER 12. PARTIAL DERIVATIVES AND DIRECTIONAL DERIVATIVES

Example 597 Given f : R3 ! R;

f (x1 ; x2 ; x3 ) = exy cos x + sin yz

we have 0 1 0 1
Dx1 f (x) (sin x) exy + y (cos x) exy
@ Dx2 f (x) A = @ z cos yz + x (cos x) exy A
Dx2 f (x) y cos yz

Remark 598 Loosely speaking, we can give the following geometrical interpretation of partial derivatives.
Given f : R2 ! R admitting partial derivatives, @f@x (x0 )
1
is the slope of the graph of the function obtained
cutting the graph of f with a plane which is
orthogonal to the x1 x2 plane, and
going through the line parallel to the x1 axis and passing through the point x0 , line to which we have given
the same orientation as the x1 axis.

De…nition 599 Given an open subset S in Rn and a function f : S ! R; if 8k 2 f1; :::; ng, the limit in
(12:1) exists, we call the gradient of f in x0 the following vector
n
(Dk f (x0 ))k=1

and we denote it by
Df (x0 )

Remark 600 The existence of the gradient for f in x0 does not imply continuity of the function in x0 , as
the following example shows.
8
>
> either x1 = 0 or x2 = 0
< 0 if
2 i.e., (x1 ; x2 ) 2 (f0g R) [ (R f0g)
f : R ! R; f (x1 ; x2 ) =
>
>
:
1 otherwise

f (x1 ; 0) f (0; 0) 0
D1 f (0) = lim = lim =0
x1 !0 x1 0 x1 !0 x1 0
and similarly
D2 f (0) = 0:
f is not continuous in 0: we want to show that 9" > 0 such that 8 > 0 there exists (x1 ; x2 ) 2 R2 such
that (x1 ; x2 ) 2 B (0; ) and jf (x1 ; x2 ) f (0; 0)j ". Take " = 21 and any (x1 ; x2 ) 2 B (0; ) such that
q q
2 2 2
x1 6= 0 and x2 6= 0(for example, x1 = x2 = 2 ; then k(x1 ; x2 )k = + = p1
4 4 2 = 2
< ). Then
jf (x1 ; x2 ) f (0; 0)j = 1 > ".

12.2 Directional Derivatives

A …rst generalization of the concept of partial derivative of a function is presented in De…nition 602 below.

De…nition 601 Given

f :S Rn ! R m ; x 7! f (x) ;
8i 2 f1; ::; m; g, the function

fi : S Rn ! R; x 7! i th component of f (x) :

is called the i th component function of f .

Therefore,
m
8x 2 S; f (x) = (fi (x))i=1 : (12.3)
12.2. DIRECTIONAL DERIVATIVES 181

De…nition 602 Given m; n 2 N, a set S Rn , x0 2 Int S; u 2 Rn , h 2 R such that x0 + hu 2 S,

m
f : S ! R , we call the directional derivative of f at x0 in the direction u, denoted by the symbol

f 0 (x0 ; u) ;

the limit
f (x0 + hu) f (x0 )
lim (12.4)
h!0 h
if it exists and it is componentwise …nite.

Remark 603 Assume that the limit in (12:4) exists and it is …nite. Then, from (12:3) and using Proposition
501,
m
f (x0 + hu) f (x0 ) fi (x0 + hu) fi (x0 ) m
f 0 (x0 ; u) = lim = lim = (fi0 (x0 ; u))i=1 :
h!0 h h!0 h i=1

If u = ejn , the j th element of the canonical basis in Rn , we then have

!m
0 fi x0 + hejn fi (x0 ) m ( ) m
f x0 ; ejn = lim = fi0 x0 ; ejn i=1
= Dxj fi (x0 ) i=1
:= Dxj f (x0 ) (12.5)
h!0 h
i=1

where equality ( ) follows from (12:2).

We can then construct a matrix whose n columns are the above vectors, a matrix which involves all partial
derivative of all component functions of f . That matrix is formally de…ned below.
m
De…nition 604 Assume that f = (fi )i=1 : S Rn ! Rm admits all partial derivatives in x0 . The Jacobian
matrix of f at x0 is denoted by Df (x0 ) and is the following m n matrix:
2 3
Dx1 f1 (x0 ) ::: Dxj f1 (x0 ) ::: Dxn f1 (x0 )
6 ::: ::: ::: 7
6 7
6 Dx1 fi (x0 ) ::: Dxj fi (x0 ) ::: Dxn fi (x0 ) 7
6 7=
4 ::: ::: ::: 5
Dx1 fm (x0 ) ::: Dxj fm (x0 ) ::: Dxn fm (x0 )

= Dx1 f (x0 ) ::: Dxj f (x0 ) ::: Dxn f (x0 ) =

= f 0 x0 ; e1n ::: f 0 x0 ; ejn ::: f 0 (x0 ; enn ) :

Remark 605 (How to easily write the Jacobian matrix of a function.)

To compute the Jacobian of f is convenient to construct a table as follows.

1. In the …rst column, write the m vector component functions f1 ; :::; fi ; :::; fm of f .
2. In the …rst row, write the subvectors x1 ; :::; xj ; :::; xn of x.
3. For each i and j, write the partial Jacobian matrix Dxj fi (x) in the entry at the intersection of the
i th row and j th column.

We then obtain the following table,

x1 ::: xj ::: xn

f1 d Dx1 f1 (x) Dxj f1 (x) Dxn f1 (x) e

::: j j
fi j Dx1 fi (x) Dxj fi (x) Dxn fi (x) j
::: j j
fm b Dx1 fm (x) Dxj fm (x) Dxn fm (x) c
where the Jacobian matrix is the part of the table between square brackets.
182 CHAPTER 12. PARTIAL DERIVATIVES AND DIRECTIONAL DERIVATIVES

Example 606 Given f : R4 ! R5 ;

0 xy 1
x2 +1
B x+yz C
B ex C
f (x; y; z; t) = B
B
xyzt
et
C
C
@ x+y+z+t A
x2 + t2

its Jacobian matrix is

2 y y x 3
x2 +1 2x2 (x2 +1) 2 x2 +1 0 0
6 1 1
(x + yz) z y
0 7
6 ex ex ex ex 7
6 ty ezt tx ezt tx eyt xy ezt txy ezt 7
6 7
4 1 1 1 1 5
2x 0 0 2t 5 4

Remark 607 From Remark 603,

8u 2 Rn , f 0 (x0 ; u) exists ) Df (x0 ) exists (12.6)

On the other hand, the opposite implication does not hold true. Consider the example in Remark 600.
There, we have seen that
Dx f (0) = Dy f (0) = 0:
But if u = (u1 ; u2 ) with u1 6= 0 and u2 6= 0, we have

f (0 + hu) f (0) 1 0
lim = lim
h!0 h h!0 h
does not exist .

Remark 608 Again loosely speaking, we can give the following geometrical interpretation of directional
derivatives. Take f : R2 ! R admitting directional derivatives. f (x0 ; u) with kuk = 1 is the slope the graph
of the function obtained cutting the graph of f with a plane which is
orthogonal to the x1 x2 plane, and
going through the line going through the points x0 and x0 + u, line to which we have given the same
orientation as u.

Example 609 Take

2
f : Rn ! R; x 7! x x = kxk
Then, the existence of f 0 (x0 ; u) can be checked computing the following limit.
f (x0 +hu) f (x0 ) (x0 +hu)(x0 +hu) x0 x0
limh!0 h = limh!0 h =

x0 x0 +hx0 u+hux0 +h2 uu x0 x0

= limh!0 h =

2hx0 u+h2 uu
= limh!0 h = limh!0 2x0 u + huu = 2x0 u

Remark 610 Given S Rn , a function f : S ! Rm and a point x0 2 Int S

f 0 (x0 ; u) = f 0 (x0 ; u) :

Proof.
f (x0 +h( u)) f (x0 ) f (x0 hu) f (x0 )
f 0 (x0 ; u) = limh!0 h = limh!0 h =

f (x0 +( h)u) f (x0 ) k:= h f (x0 +ku) f (x0 )

= limk!0 h = limk!0 k = f 0 (x0 ; u) :
12.2. DIRECTIONAL DERIVATIVES 183

Remark 611 It is not the case that

8u 2 Rn , f 0 (x0 ; u) exists ) f is continuous in x0 (12.7)

as the following example shows. Consider

8 xy 2
< x2 +y 4 if x 6= 0
f : R2 ! R; f (x; y) =
:
0 if x = 0; i:e:; (x; y) 2 f0g R

Let’s compute f 0 (0; u). If u1 6= 0:

f (0 + hu) f (0) hu1 h2 u22 u1 u22 u22

lim = lim = lim =
h!0 h h!0 (h2 u2 4 4
1 + h u2 ) h h!0 u2 2 4
1 + h u2 u1

If u1 = 0;we have

f (0 + hu) f (0) f (0; hu2 ) f (0) 0

lim = lim = lim =0
h!0 h h!0 h h!0 h

On the other hand, if x = y 2 and x; y 6= 0, i.e., along the graph of the parabola x = y 2 except the origin,
we have
y4 1
f (x; y) = f y 2 ; y = 4 4
=
y +y 2
while
f (0; 0) = 0:
To prove that f is not continuous in 0 we have to show that 9" > 0 such that 8 > 0 there exists (x; y) 2
2
R2 such that (x; y) 2 B (0; ) and jf (x; y) f (0; 0)j = x2xy
+y 4 ". Take " = 1=4 and consistently with the
2
above argument yn = n and xn = n2 . Then

1 1
8n 2 N; jf (xn ; yn )j = f yn2 ; xn = >"= :
2 4
We are left with showing that 9n 2 N such that

(xn ; yn ) 2 B (0; ) ; i:e:; k (xn ; yn ) k < :

Indeed s s
4 2 4 2
k (xn ; yn ) k = + and lim + =0
n4 n2 n!1 n4 n2
Then, by de…nition of limit, 8 > 0; 9n 2 N such that k (xn ; yn ) k < , as desired.

Remark 612 Roughly speaking, the existence of partial derivatives in a given point in all directions implies
“continuity along straight lines” through that point; it does not imply “ continuity along all possible curves
through that point”, as in the case of the parabola in the picture above.

Remark 613 We are now left with two problems:

1. Is there a de…nition of derivative whose existence implies continuity?
2. Is there any “easy” way to compute the directional derivative?

Appendix (to be corrected)

There are other de…nitions of directional derivatives used in the literature.
Let the following objects be given: m; n 2 N, a set S Rn , x0 2 Int S; u 2 Rn , h 2 R such that
m
x0 + hu 2 S, f : S ! R ,
184 CHAPTER 12. PARTIAL DERIVATIVES AND DIRECTIONAL DERIVATIVES

De…nition 614 (our de…nition following Apostol (1974)) We call the directional derivative of f at x0 in
the direction u according to Apostol, denoted by the symbol

fA0 (x0 ; u) ;

the limit
f (x0 + hu) f (x0 )
lim (12.8)
h!0 h
if it exists and it is …nite.

De…nition 615 (Girsanov (1972)) We call the directional derivative of f at x0 in the direction u according
to Girsanov, denoted by the symbol
0
fG (x0 ; u) ;
the limit
f (x0 + hu) f (x0 )
lim (12.9)
h!0+ h
if it exists and it is …nite.

De…nition 616 (Wikipedia) Take u 2 Rn such that kuk = 1.We call the directional derivative of f at x0 in
the direction u according to Wikipedia, denoted by the symbol
0
fW (x0 ; u) ;

the limit
f (x0 + hu) f (x0 )
lim (12.10)
h!0+ h
if it exists and it is …nite.

Fact 1. For given x0 2 S; u 2 Rn

A ) G ) W;
while the opposite implications do not hold true. In particular, to see way A : G, just take f : R ! R;
8
< 0 if x < 0
f (x) = ;
:
1 if x 0

and observe that while the right derivative in 0 is

f (h) f (0) 1 1
lim = lim = 0;
h!0+ h h!0 h
while the left derivative is
f (h) f (0) 0 1
lim = lim = +1:
h!0 h h!0 h
Fact 2. For given x0 2 S;
0 0
fW (x; u) exists ) fG (x; v) exists for any v = u and 2 R++ .

Proof.

0 f (x0 +hv) f (x0 ) f (x0 +h u) f (x0 ) k= h>0

fG (x; v) = limh!0+ h = limh!0+ h =

f (x0 +ku) f (x0 ) 0

= limh!0+ k = fW (x; u) :
Fact 3. Assume that u 6= 0 and x0 2 Rn . Then the following implications are true:

8u 2 Rn , fA0 (x; u) exists , 0

8u 2 Rn , fG (x; u) exists , 0
8u 2 Rn such that kuk = 1, fW (x; u) exists.
12.2. DIRECTIONAL DERIVATIVES 185

Proof.
From Fact 1, we are left with showing just two implications.
G ) A.
We want to show that
f (x0 + hu) f (x0 ) f (x0 + hv) f (x0 )
8u 2 Rn ; lim+ 2 R )8v 2 Rn ; lim 2 R:
h!0 h h!0 h
f (x0 +hv) f (x0 )
Therefore, it su¢ ces to show that l := limh!0 h 2 R:Take u = v. Then,

f (x0 hu) f (x0 ) f (x0 hu) f (x0 ) k= h f (x0 + ku) f (x0 )

l = lim = lim = lim+ 2 R:
h!0 h h!0 h k!0 k
W ) G.
The proof of this implication is basically the proof of Fact 2.We want to show that

f (x0 + hu) f (x0 ) f (x0 + hv) f (x0 )

8u 2 Rn such that kuk = 1; lim+ 2 R )8v 2 Rn n f0g ; l := lim+ 2 R:
h!0 h h!0 h
In fact,
v
f x0 + h kvk kvk f (x0 )
l := lim+ 2 R;
h!0 h
v
simply because kvk = 1.

Remark 617 We can give the following geometrical interpretation of directional derivatives. First of all
observe that from Proposition 620,

f (x0 + hu) f (x0 )

f 0 (x0 ; u) := lim = dfx0 (u) :
h!0 h
If f : R ! R, we then have
f 0 (x0 ; u) = f 0 (x0 ) u:
Therefore, if u = 1; we have
f 0 (x0 ; u) = f 0 (x0 ) ;
and if u > 0, we have
sign f 0 (x0 ; u) = sign f 0 (x0 ) :
Take now f : R2 ! R admitting directional derivatives. Then,

f 0 (x0 ; u) = Df (x0 ) u with kuk = 1

is the slope the graph of the function obtained cutting the graph of f with a plane which is
orthogonal to the x1 x2 plane, and
along the line going through the points x0 and x0 + u, in the direction from x0 to x0 + u.
186 CHAPTER 12. PARTIAL DERIVATIVES AND DIRECTIONAL DERIVATIVES
Chapter 13

Di¤erentiability

13.1 Total Derivative and Di¤erentiability

If f : R ! R, we say that f is di¤erentiable in x0 , if the following limit exists and it is …nite

f (x0 + h) f (x0 )
lim
h!0 h

and we write
f (x0 + h) f (x0 )
f 0 (x0 ) = lim
h!0 h
or, in equivalent manner,
f (x0 + h) f (x0 ) f 0 (x0 ) h
lim =0
h!0 h
and
f (x0 + h) (f (x0 ) + f 0 (x0 ) h) = r (h)

where
r (h)
lim = 0;
h!0 h

or
f (x0 + h) = f (x0 ) + f 0 (x0 ) h + r (h) ;

or using what said in Section 6.1,

f (x0 + h) = f (x0 ) + lf 0 (x0 ) (h) + r (h)

where lf 0 (x0 ) 2 L (R; R)

r(h)
and limh!0 h =0

De…nition 618 Given a set S Rn , x0 2 Int S; f : S ! Rm ; we say that f is di¤ erentiable at x0 if

there exists
a linear function dfx0 : Rn ! Rm

such that for any u 2 Rn ,u 6= 0, such that x0 + u 2 S;

f (x0 + u) f (x0 ) dfx0 (u)

lim =0 (13.1)
u!0 kuk

In that case, the linear function dfx0 is called the total derivative or the di¤ erential or simply the derivative
of f at x0 .

187
188 CHAPTER 13. DIFFERENTIABILITY

Remark 619 Obviously, given the condition of the previous De…nition, we can say that f is di¤ erentiable
at x0 if there exists a linear function dfx0 : Rn ! Rm such that 8u 2 Rn such that x0 + u 2 S

r (u)
f (x0 + u) = f (x0 ) + dfx0 (u) + r (u) ; with lim =0 (13.2)
u!0 kuk

or

f (x0 + u) = f (x0 ) + dfx0 (u) + kuk Ex0 (u) ; with lim Ex0 (u) = 0 (13.3)
u!0

The above equations are called the …rst-order Taylor formula (of f at x0 in the direction u). Condition
(13:3) is the most convenient one to use in many instances.

Proposition 620 Assume that f : S ! Rm is di¤ erentiable at x0 , then

8u 2 Rn ; f 0 (x0 ; u) = dfx0 (u) :

Proof.
f (x0 + hu) f (x0 ) (1)
f 0 (x0 ; u) := lim =
h!0 h
f (x0 ) + dfx0 (hu) + khuk Ex0 (hu) f (x0 ) (2) hdfx0 (u) + jhj kuk Ex0 (hu) (3)
= lim = lim =
h!0 h h!0 h

(4) (5)
= lim dfx0 (u) + lim sign (h) kuk Ex0 (hu) = dfx0 (u) + kuk lim sign (h) Ex0 (hu) = dfx0 (u) ;
h!0 h!0 hu!0

where
(1) follows from (13:3) with hu in the place of u,
(2) from the fact that dfx0 is linear, and from a property of a norm,
(3) from the fact that jhj
h = sign (h),
1

(4) from the fact that h ! 0 implies that hu ! 0,

(5) from the assumption that f is di¤erentiable in x0 .

Remark 621 The above Proposition implies that if the di¤ erential exists, then it is unique - from the fact
that the limit is unique, if it exists.

Proposition 622 If f : S ! Rm is di¤ erentiable at x0 , then f is continuous at x0 .

Proof. We have to prove that

lim f (x0 + u) f (x0 ) = 0
u!0

i.e., from (13:3), it su¢ ces to show that

(1) (2)
lim dfx0 (u) + kuk Ex0 (u) = dfx0 (0) + lim kuk Ex0 (u) = 0
u!0 u!0

where
(1) follows from the fact that dfx0 is linear and therefore continuous, which is shown in Lemma 623 below.
(2) follows from the fact again that dfx0 is linear, and therefore dfx (0) = 0, and from (13:3) :
1 sign is the function de…ned as follows:
8
< 1 if x<0
sign : R ! f 1; 0 + 1g ; x 7! 0 if x=0
:
+1 if x > 0:
13.2. TOTAL DERIVATIVES IN TERMS OF PARTIAL DERIVATIVES. 189

Lemma 623 If dfx0 is linear, then it is continuous.

Proof. 1st proof.
Since dfx0 is linear, then there exist A 2 M (m; n) such that for any u 2 Rn , dfx0 (u) = A u. Then, the
desired result follows from Remark 514.
2nd proof.
We have to show that 8x0 2 Rn ; 8" > 0 9 > 0 such that kx x0 k < ) kl (x) l (x0 )k < ". De…ned
[l] = A, we have that

kl (x) l (x0 )k = kA x x0 k =

(1) Pm (2)
(13.4)
= R1 (A) (x x0 ) ; :::; Rm (A) (x x0 ) i=1 Ri (A) (x x0 )
Pm
i=1 Ri (A) kx x0 k m maxi2f1;:::;mg Ri (A) kx x0 k ;

where (1) follows from Remark 56, and (2) from Proposition 53.4, i.e., Cauchy-Schwarz inequality. Take
"
= :
m maxi2f1;:::;mg fRi (A)g

Then we have that kx x0 k < implies that m maxi2f1;:::;mg Ri (A) kx x0 k < ", and from (13:4) ;
kl (x) l (x0 )k < ", as desired.

Remark 624 The above Proposition is the answer to Question 1 in Remark 613. We still do not have a
answer to Question 2 and another question naturally arises at this point:
3. Is there an “easy” way of checking di¤ erentiability?

13.2 Total Derivatives in terms of Partial Derivatives.

In Remark 626 below, we will answer question 2 in Remark 613: Is there any “easy” way to compute the
directional derivative?
m
Proposition 625 Assume that f = (fj )j=1 : S Rn ! Rm is di¤ erentiable in x0 . The matrix associated
with dfx0 with respect to the canonical bases of Rn and Rm is the Jacobian matrix Df (x0 ), i.e.,

[dfx0 ] = Df (x0 ) ;

i.e.,
8x 2 Rn ; dfx0 (x) = Df (x0 ) x: (13.5)

Proof. From De…nitiion 269,

[dfx0 ] = dfx0 e1n ::: dfx0 ein ::: dfx0 (enn ) m n

From Proposition 620,

8i 2 f1; :::; ng ; dfx0 ei = f 0 x0 ; ei ;
and from (12:5)
m
f 0 x0 ; ei = (Dxi fj (x0 ))j=1 :
Then
m m m
[dfx0 ] = (Dx1 fj (x0 ))j=1 ::: (Dxi fj (x0 ))j=1 ::: (Dx1 fj (x0 ))j=1 ;
m n

as desired.

Remark 626 From Proposition 620, part 1, and the above Proposition 625, we have that if f is di¤ erentiable
in x0 , then 8u 2 Rm
8u 2 Rm ; f 0 (x0 ; u) = Df (x0 ) u:
190 CHAPTER 13. DIFFERENTIABILITY

Remark 627 From (13:5), we get

m
(1) X m
(2) X
kdfx0 (x)k = k[Df (x0 )]mn xk jDfj (x0 ) xj kDfj (x0 )k kxk
j=1 j=1

where
(1) follows from Remark 56, and
(2) from Cauchy-SchwarzPinequality in Proposition 53.
m
Therefore, de…ned := j=1 kDfj (x0 )k ;we have that

kdfx0 (x)k kxk

and
lim kdfx0 (x)k = 0
x!0

Remark 628 We have seen that

f di¤ erentiable in x0 ) f admits directional derivative in x0 ) Df (x0 ) exists
:
+ (not +) (not +)
f continuous in xo f continuous in xo f continuous in xo

Therefore
f di¤ erentiable in x0 : Df (x0 ) exists
and
f di¤ erentiable in x0 : f admits directional derivative in x0
We still do not have an answer to question 3 in Remark 624: Is there an easy way of checking di¤ eren-
tiability? We will provide an answer in Proposition 661.
Chapter 14

Some Theorems

We …rst introduce some needed de…nitions.

De…nition 629 Given an open S Rn ,

n m
f : S ! Rm ; x := (xj )j=1 7! f (x) = (fi (x))i=1

I f1; :::; mg and J f1; :::; ng ;

the partial Jacobian of (fi )i2I with respect to (xj )j2J in x0 2 S is the following (#I) (#J) submatrix
of Df (x0 )
@fi (x0 )
;
@xj i2I; j2J

and it is denoted by
D(xj )j2J (fj )i2I (x0 )

Example 630 Take:

n1
S an open subset of Rn1 , with generic element x0 = (xj )j=1 ,
n2 00 n2
T an open subset of R , with generic element x = (xk )k=1 and

f :S T ! Rm ; (x0 ; x00 ) 7! f (x0 ; x00 )

Then, de…ned n = n1 + n1 ; we have

2 3
Dx1 f1 (x0 ) ::: Dxn1 f1 (x0 )
6 ::: ::: 7
6 7
Dx0 f (x0 ) = 6
6 Dx1 fi (x0 ) ::: Dxn1 fi (x0 ) 7
7
4 ::: ::: 5
Dx1 fm (x0 ) ::: Dxn1 fm (x0 ) m n1

and, similarly,
2 3
Dxn1 +1 f1 (x0 ) ::: Dxn f1 (x0 )
6 ::: ::: 7
6 7
f (x0 ) := 6
6 Dxn1 +1 fi (x0 ) ::: Dxn fi (x0 ) 7
7
4 ::: ::: 5
Dxn1 +1 fm (x0 ) ::: Dxn fm (x0 )
m n2
and therefore
Df (x0 ) := Dx0 f (x0 ) Dx00 f (x0 ) m n
n
@f (x)
De…nition 631 Given an open set S Rn and f : S ! R, assume that 8x 2 S, Df (x) := @xj
j=1
exists. Then, 8j 2 f1; :::; ng, we de…ne the j th partial derivative function as
@f @f (x)
: S ! R; x 7!
@xj @xj

191
192 CHAPTER 14. SOME THEOREMS

Assuming that the above function has partial derivative with respect to xk for k 2 f1; :::; ng, we de…ne it
as the mixed second order partial derivative of f with respect to xj and xk and we write

@ 2 f (x) @ @f (x)
@xj
:=
@xj @xk @xk

De…nition 632 Given f : S Rn ! R; the Hessian matrix of f at x0 is the n n matrix

@2f
D2 f (x0 ) := (x0 )
@xj @xk j;k=1;:::;n

Remark 633 D2 f (x0 ) is the Jacobian matrix of the gradient function of f .

Example 634 Compute the Hessian function of f : R3++ ! R,

f (x; y; z) = ex cos y + z 2 + x2 log y + log x + log z + 2t log t

We …rst compute the gradient:

0 1 1
2x ln y + (cos y) ex + x
B (sin y) ex + x2 C
B y C
@ 2z + 1 A
z
2 ln t + 2
and then the Hessian matrix
2 1 3
2 ln y + (cos y) ex x2 (sin y) ex + 2 xy 0 0
6 (sin y) ex + 2 x (cos y) ex xy2
2
0 0 7
6 y 7
4 0 0 1
+2 0 5
z2
2
0 0 0 t

14.1 The chain rule

Proposition 635 (Chain Rule) Given S Rn , T Rm ;

f :S R n ! Rm ;

such that Im f T
g:T Rm ! R p ;
assume that f and g are di¤ erentiable in x0 and y0 = f (x0 ), respectively. Then

h:S Rn ! Rp ; h (x) = (g f ) (x)

is di¤ erentiable in x0 and

dhx0 = dgf (x0 ) dfx0 :

Proof. We want to show that there exists a linear function dhx0 : Rn ! Rp such that

h (x0 + u) = h (x0 ) + dhx0 (u) + kuk Ex0 (u) ; with lim Ex0 (u) = 0;
u!0

and dhx0 = dgf (x0 ) dfx0 .

Taking u su¢ ciently small (in order to have x0 + u 2 S), we have

h (x0 + u) h (x0 ) = g [f (x0 + u)] g [f (x0 )] = g [f (x0 + u)] g (y0 )

and de…ned
v = f (x0 + u) y0
14.1. THE CHAIN RULE 193

we get
h (x0 + u) h (x0 ) = g (y0 + v) g (y0 ) :
Since f is di¤erentiable in x0 , we get
v = dfx0 (u) + kuk Ex0 (u) ; with lim Ex0 (u) = 0: (14.1)
u!0

Since g is di¤erentiable in y0 = f (x0 ), we get

g (y0 + v) g (y0 ) = dgy0 (v) + kvk Ey0 (v) ; with lim Ey0 (v) = 0: (14.2)
v!0

Inserting (14:1) in (14:2), we get

g (y0 + v) g (y0 ) = dgy0 (dfx0 (u) + kuk Ex0 (u))+kvk Ey0 (v) = dgy0 (dgx0 (u))+kuk dgy0 (Ex0 (u))+kvk Ey0 (v)
De…ned 8
< 0 if u=0
Ex0 (u) := ;
: kvk
dfy0 (Ex0 (u)) + kuk Ey0 (v) if u 6= 0
we are left with showing that
lim Ex0 (u) = 0:
u!0
Observe that
lim dfy0 (Ex0 (u)) = 0
u!0
since linear functions are continuous and from (14:1). Moreover, since limu!0 v = limu!0 (f (x0 + u) y0 ) =
0, from (14:2), we get
lim Ey0 (v) = 0:
u!0
kvk
Finally, we have to show that limu!0 kuk is bounded. Now, from the de…nition of u and from (627),
Pm
de…ned := j=1 kDfj (x0 )k,
kvk = kdfx0 (u) + kuk Ex0 (u)k kdfx0 (u)k + kuk kEx0 (u)k ( + kEx0 (u)k) kuk
and
kvk
lim lim ( + kEx0 (u)k) = ;
u!0 kuk u!0

as desired.
Remark 636 From Proposition 625 and Proposition 281, or simply (7:10), we also have
Dh (x0 )p n = Dg (f (x0 ))p m Df (x0 )m n ;
or
Dx h (x0 ) = Dy g (y)jy=f (xo ) Dx f (x0 ) ;
Observe that Dg (f (x0 )) is obtained computing Dg (y) and then substituting f (x0 ) in the place of y. We
therefore also write Dg (f (x0 )) = Dg (y)jy=f (x0 ) .
Example 637 Take
f : R ! R2 ; x 7! 3x; 4x + x2 (14.3)
and
g : R2 ! R; (x1 ; x2 ) 7! x1 + x2 :
Then
h:R!R x 7! 3x + 4x + x2 = 7x + x2
and
h0 (x0 ) = 7 + 2x0 (14.4)
Therefore, from Remark 636,
3
Dh (x0 )1 1 = Dg (f (x0 ))1 2 Df (x0 )2 1 = [1 ; 1] = 3 + 4 + 2x0 = 7 + 2x0 :
4 + 2x0
194 CHAPTER 14. SOME THEOREMS

Example 638 Sometimes we have to solve (not very well formulated) problems as the following one.“Given
the function
u (g1 (l) ; :::; gn (l) ; l) ;
what is the e¤ ect of a change in l on the value of u?”
Below, 1. we formulate the problem in a rigorous way;
2. applying the Chain Rule Proposition, we answer the question.
1. Let the following functions be given

u : Rn+1 ! R (x1 ; :::xn ; l) 7! u ((x1 ; :::xn ; l))

gi : R ! R; l 7! gi (l) for any i 2 f1; ::; ng ;

' : R ! Rn+1 ; l 7! (g1 (l) ; :::; gn (l) ; l)

v : R ! R; l 7! (u ') (l) = u (g1 (l) ; :::; gn (l) ; l)

Assume that any function de…ned above is di¤ erentiable.

2. First of all, we identify the symbols in the Proposition with the symbols in the case we are analyzing
consistently with the following table.

Chain Rule Proposition x0 f g y h n m p

Example under analysis l ' u (x1 ; :::xn ; l) v 1 n+1 1

Then, we have that

Dh (x0 )p n = Dg (f (x0 ))p m Df (x0 )m n

or
Dx h (x0 ) = Dy g (y)jy=f (xo ) Dx f (x0 ) ;
becomes
Dv (l)1 1 = Du (' (l))1 (n+1) D' (l)(n+1) 1

or
Dv (l) = D(x1 ;:::xn ;l) u ((x1 ; :::xn ; l))j(x1 ;:::xn ;l)='(l) Dl ' (l) :
Since
Dl ' (l) = (g10 (l) ; :::; gn0 (l) ; 1)
and

D(x1 ;:::xn ;l) u (x1 ; :::xn ; l)j(x1 ;:::xn ;l)='(l) = (Dx1 u (:::) ; ::; Dxn u (:::) ; Dl u (:::))j(x1 ;:::xn ;l)='(l) ;

then,
Dv (l)1 1 =

= (Dx1 u (' (l)) ; ::; Dxn u(' (l) ; Dl u (' (l))) (g10 (l) ; :::; gn0 (l) ; 1) =
Pn
=( i=1 Dxi u(' (l) gi0 (l)) + Dl u(' (l) :
14.1. THE CHAIN RULE 195

De…nition 639 Given f : S Rn ! Rm1 , g : S Rn ! Rm2 ;

(f; g) : Rn ! Rm1 +m2 ; (f; g) (x) = (f (x) ; g (x))

Remark 640 Clearly,

Df (x0 )
D (f; g) (x0 ) =
Dg (x0 )

Example 641 Given

f : R ! R2 ; : x 7! (sin x; cos x)

g : R2 ! R2 ; : (y1 ; y2 ) 7! (y1 + y2 ; y1 y2 )

h = g f : R ! R2 ; : x 7! (sin x + cos x; sin x cos x)

verify the conclusion of the Chain Rule Proposition.

cos x
Df (x) =
sin x

1 1
Dg (y) =
y2 y1

1 1
Dg (f (x)) =
cos x sin x

1 1 cos x cos x sin x

Dg (f (x)) Df (x) = = = Dh (x)
cos x sin x sin x cos2 x sin2 x

Example 642 Take

g : Rk ! Rn ; : t 7! g (t)

f : Rn Rk ! R m ; : (x; t) 7! f (x; t)

h : Rk ! Rm ; : t 7! f (g (t) ; t)
Then
ge := (g; idRk ) : Rk ! Rn Rk ; t 7! (g (t) ; t)
and
h=f ge = f (g; idRk ; )
Therefore, assuming that f; g; h are di¤ erentiable,

Dg (t0 )
[Dh (t0 )]m k = [Df (g (t0 ) ; t0 )]m (n+k) =
I (n+k) k

[Dg (t0 )]n k

= [Dx f (g (t0 ) ; t0 )]m n j [Dt f (g (t0 ) ; t0 )]m k =
Ik k

[Dx f (g (t0 ) ; t0 )]m n [Dg (t0 )]n k + [Dt f (g (t0 ) ; t0 )]m k

In the case k = n = m = 1, the above expression

df (x = g (t) ; t) @f (g (t) ; t) dg (t) @f (g (t) ; t)

= +
dt @x dt @t
or
df (g (t) ; t) @f (x; t) dg (t) @f (x; t)
= +
dt @x jx=g(t) dt @t jx=g(t)
196 CHAPTER 14. SOME THEOREMS

14.2 Mean value theorem

Proposition 643 (Mean Value Theorem) Let S be an open subset of Rn and f : S ! Rm a di¤ erentiable
function. Let x; y 2 S be such that the line segment joining them is contained in S, i.e.,

L (x; y) := fz 2 Rn : 9 2 [0; 1] such that z = (1 ) x + yg S:

Then

8a 2 Rm ; 9 z 2 L (x; y) such that a [f (y) f (x)] = a [Df (z) (y x)]

Remark 644 Under the assumptions of the above theorem, the following conclusion is false:

9 z 2 L (x; y) such that f (y) f (x) = Df (z) (y x):

But if f : S ! Rm=1 , then setting a 2 Rm=1 equal to 1, we get that the above statement is indeed true.

Example 645 Let f : R ! R2 , t 7! (cos t; sin t). Then

Df (t) u = u ( sin t; cos t)

for every real u. The Mean-Value formula

f (y) f (x) = Df (z) (y x)

cannot hold when x = 0 and y = 2 , since the left member is zero and the right member is a vector of length
2 .

Proof. of Proposition 643

De…ne u = y x. Since S is open and L (x; y) S, 9 > 0 such that 8t 2 ( ; 1 + ) such that
x + tu = (1 t) x + ty 2 S. Taken a 2 Rm , de…ne
Pm
F :( ; 1 + ) ! R; : t 7! a f (x + tu) = j+1 aj fj (x + tu)

Then
m
X
F 0 (t) = aj [Dfj (x + tu)]1 n un 1 = a1 m [Df (x + tu)]m n un 1
j+1

and F is continuous on [0; 1] and di¤erentiable on (0; 1); then, we can apply “Calculus 1” Mean Value
Theorem and conclude that

9 2 (0; 1) such that F (1) F (0) = F 0 ( ) ;

and by de…nition of F and u,

9 2 (0; 1) such that f (y) f (x) = a Df (x + u) (y x)

which choosing z = x + u gives the desired result.

Remark 646 Using the results we have seen on directional derivatives, the conclusion of the above theorem
can be rewritten as follows.

9 z 2 L (x; y) such that f (y) f (x) = f 0 (z; y x)

As in the case of real functions of real variables, the Mean Value Theorem allows to give a simple
relationship between the sign of the derivative and monotonicity of the function.

De…nition 647 A set C Rn is convex if 8x1 , x2 2 C and 8 2 [0; 1], (1 ) x1 + x2 2 C.

Proposition 648 Let S be an open and convex subset of Rn and f : S ! Rm a di¤ erentiable function. If
8x 2 S, dfx = 0, then f is constant on S.
14.2. MEAN VALUE THEOREM 197

Proof. Take arbitrary x; y 2 S. Then since S is convex and f is di¤erential, from the Mean Value
Theorem, we have that

8a 2 Rm ; 9 z 2 L (x; y) such that a [f (y) f (x)] = a [Df (z) (y x)] = 0:

Taken a = f (y) f (x), we get that

kf (y) f (x)k = 0
and therefore
f (x) = f (y) ;
as desired.
n n
De…nition 649 Given x := (xi )i=1 ; y := (yi )i=1 2 Rn ;

x y means 8i 2 f1; :::; ng ; xi yi ;

x>y means x y ^ x 6= y;

x y means 8i 2 f1; :::; ng ; xi > yi :

De…nition 650 f : S Rn ! R is increasing if 8x; y 2 S, x > y ) f (x) f (y).

f is strictly increasing if 8x; y 2 S, x > y ) f (x) > f (y) :

Proposition 651 Take an open, convex subset S of Rn , and f : S ! R di¤ erentiable.

1. If 8x 2 S, Df (x) 0, then f is increasing;
2. If 8x 2 S, Df (x) >> 0, then f is strictly increasing.

Proof. 1. Take y x. Since S is convex, L (x; y) S. Then from the Mean Value Theorem,

9 z 2 L (x; y) such that f (y) f (x) = Df (z) (y x)

Since y x 0 and Df (z) 0, the result follows.

2. Take x > y. Since S is convex, L (x; y) S. Then from the Mean Value Theorem,

9 z 2 L (x; y) such that f (y) f (x) = Df (z) (y x) (14.5)

Since y x > 0 and Df (z) >> 0, the result follows.

Remark 652 The statement “If 8x 2 S, Df (x) > 0, then f is strictly increasing”is false as veri…ed below.
We want to …nd f : S Rn ! R such that 8x 2 S, Df (x) > 0 and f is not strictly increasing. Take

f : Rn ! R; (x1 ; x2 ) 7! x1 : (14.6)

Then
Df (x) = (1; 0) > (0; 0) : (14.7)
Now we want to to show that is false that f is strictly increasing, i.e., we want to show that we can have

x0 := (x01 ; x02 ) > (x001 ; x002 ) := x00 and f (x0 ) f (x00 ) (14.8)

Take
x0 = (0; 2) and x00 = (0; 1) (14.9)
Then
f (x0 ) = 0 = f (x00 ) (14.10)
as desired.

Corollary 653 Take an open, convex subset S of Rn , and f 2 C 1 (S; R). If 9x0 2 S and u 2 Rn n f0g such
that f 0 (x0 ; u) > 0, then 9 t 2 R++ such that 8t 2 0; t ,

f (x0 + tu) f (x0 ) :

198 CHAPTER 14. SOME THEOREMS

Proof. Since f is C 1 and f 0 (x0 ; u) = Df (x0 ) u > 0, 9r > 0 such that

8x 2 B (x0 ; r) ; f 0 (x; u) > 0:

1
Then 8t 2 ( r; r), x0 + kuk tu x0 = t < r, and therefore

t
f 0 x0 + u; u >0
kuk

Then, from the Mean Value Theorem, 8t 2 0; 2r ,

f (x0 + tu) f (x0 ) = f 0 (x0 + tu; u) 0:

De…nition 654 Given a function f : S Rn ! R, x0 2 S is a point of local maximum for f if

9 > 0 such that 8x 2 B (x0 ; ) ; f (x0 ) f (x) ;

x0 is a point of global maximum for f if

8x 2 S; f (x0 ) f (x) :

x0 2 S is a point of strict local maximum for f if

9 > 0 such that 8x 2 B (x0 ; ) ; f (x0 ) > f (x) ;

x0 is a point of strict global maximum for f if

8x 2 S; f (x0 ) > f (x) :

Local, global, strict minima are de…ned in obvious manner

Proposition 655 If S Rn , f : S ! R admits all partial derivatives in x0 2 Int S and x0 is a point of

local maximum or minimum, then Df (x0 ) = 0.

Proof. Since x0 is a point of local maximum, 9 > 0 such that 8x 2 B (x0 ; ), f (x0 ) f (x). As in
Remark 596, for any k 2 f1; :::; ng, de…ne

gk : R ! R; g (xk ) = f x0 + (xk x0k ) ekn :

Then gk has a local maximum point at x0k . Then from Calculus 1,

gk0 (x0k ) = 0
Since, again from Remark 596, we have

Dk f (x0 ) = gk0 (x0 ) :

the result follows.

14.3 A su¢ cient condition for di¤erentiability

Proofs in the present and next section have to be carefully read (see also my handwritten notes).
m
De…nition 656 f = (fi )i=1 : S Rn ! Rm is continuously di¤ erentiable on A S, or f is C 1 on S, or
f 2 C (A; Rm ) if 8i 2 f1; :::; mg ; j 2 f1; :::; ng ;

Dxj fi : A ! R; x 7! Dxj fi (x) is continuous.

14.3. A SUFFICIENT CONDITION FOR DIFFERENTIABILITY 199

De…nition 657 f : S Rn ! R is twice continuously di¤ erentiable on A S, or f is C 2 on S, or

2 m
f 2 C (A; R ) if 8j; k 2 f1; :::; ng ;
@2f @2f
: A ! R; x 7! (x) is continuous.
@xj @xk @xj @xk
The proof of the main result of the section, i.e., Proposition 661, requires some Lemmas.

Lemma 658 If L is a linear function from Rn to Rm , then L is di¤ erentiable.

Proof. We want to show that there exists a linear function Tx0 : Rn ! Rm such that if v 2 B (x0 ; ") S,
then
L (x0 + v) = L (x0 ) + Tx0 (v) + kvk Ex0 (v) with lim Ex0 (v) = 0 (14.11)
v!0
Take
Tx0 = L: (14.12)
Then by the de…nition of linear function

L (x0 + v) = L (x0 ) + L (v) = L (x0 ) + Tx0 (v) (14.13)

and taken
Ex0 = 0;
we get the desired result.

Lemma 659 The projection function

j
: Rm ! R; j
: (y1 ; :::; yj ; :::; yn ) 7! yj (14.14)

is di¤ erentiable.
j
Proof. The claim is true because of Lemma 658 and because is linear as shown below. Given x,y
2 Rm and 2 R, then
j j j j j
(x + y) = xj + yj = (x) + (y) and ( x) = xj = (x) (14.15)

Lemma 660 Given f : Rn ! Rm , f = (f1 ; :::; fj ; :::; fm ), then f is di¤ erentiable if and only if fj is
di¤ erentiable for any j 2 f1; :::; mg .
Proof. [)] Observe that fj = j f , where j is the projection function. Since j is di¤ erentiable
from Lemma 659, and f is di¤ erentiable by assumption, from the Chain Rule, i.e. Proposition 635, fj is
di¤ erentiable.
[(] By assumption
fj (x0 + v) = fj (x0 ) + Dfj (x0 ) v + kvk Exj 0 (v) (14.16)
with
lim Exj 0 (v) = 0 (14.17)
v!0
for any j. Therefore
0 1
Ex10 (v)
B : C
B C
f (x0 + v)m = f (x0 )m B : C
n 1 + Df (x0 )m n vn 1 + kvk B C (14.18)
@ : A
Exm0 (v) m n

with 0 1
Ex10 (v)
B : C
B C
lim B : C = 0; (14.19)
v!0 B C
@ : A
Exm0 (v) m n
as desired
200 CHAPTER 14. SOME THEOREMS

Proposition 661 If f : S Rn ! Rm is C 1 in an open neighborhood of x0 , then it is di¤ erentiable in x0 .

Proof. By assumption 9 > 0 such that f is C 1 in B (x0 ; ). Since, by de…nition,

f is C 1 , f or any j 2 f1; :::; mg ; fj is C 1

and since, from Lemma 660,

f is dif f erentiable , f or any j 2 f1; :::; mg ; fj is dif f erentiable;

then it is enough to show that for any j 2 f1; :::; mg, fj is di¤erentiable. For simplicity of notation- dropping
the subscript j- we have to show that f : S ! R is di¤erentiable at x0 .
Take an arbitrary v 2 Rn n f0g such that x0 + v 2 B x0 ; 2 and therefore

kvk < : (14.20)

2
Then recalling that ekn := (0; :::; 1; :::; 0) 2 Rn , de…ne
v
= kvk; y= kvk ; and then y = v; (14.21)

and
v0 = 0; v1 = y1 e1n = (y1 ; 0; :::; 0)

v2 = y1 e1n + y2 e2n = v1 + y2 e2n = (y1 ; y2 ; 0; :::; 0) ; :::

Pk
vk = h=1 yh ehn = vk 1 + yk ekn = (y1 ; y2 ; :::; yk ; 0; :::; 0) ; :::
Pn
vn = h=1 yn ehn = y

Therefore,
Pn
k=1 [f (x0 + vk ) f (x0 + vk 1 )] =

= f (x0 + v1 ) f (x0 + v0 ) + f (x0 + v2 ) f (x0 + v1 ) + ::: + f (x0 + vn ) f (x0 + vn 1) =

= f (x0 + vn ) f (x0 + v0 ) = f (x0 + y) f (x0 + 0) = f (x0 + v) f (x0 ) :

(14.22)
For any k 2 f1; :::; mg, de…ne bk = x0 + vk 1 , i.e.,

bk = x0 + vk 1 = (x01 + y1 ; :::; x0;k 1 + yk 1 ; x0;k ; :::; x0;n )

and
bk+1 = bk + yk ekn = (x01 + y1 ; :::; x0;k 1 + yk 1 ; x0;k + x0;k ; x0;k+1 ; :::; x0;n ) :
Therefore

f (x0 + vk ) f (x0 + vk 1) = f x0 + vk 1 + yk ekn f (x0 + vk 1) = f bk + yk ekn f (bk ) (14.23)

From (14:22) and (14:23) we get

n
X
f (x0 + v) f (x0 ) = f bk + yk ekn f (bk ) : (14.24)
k=1

Observe that the k th term in the sum in (14:24) is

f bk + yk ekn f (bk ) = f (bk + (0; :::; yk ; :::; 0)) f (bk ) :

Moreover, if
jxk j < 2 jyk j ; (14.25)
14.3. A SUFFICIENT CONDITION FOR DIFFERENTIABILITY 201

then
kvk
jxk j < 2 jyk j < 2 kvk = 2 kvk < ; (14.26)
kvk
and also
def. bk ;vk
bk + xk ekn = x0 + vk 1 + xk ekn = x0 + ( y1 ; :::; yk 1 ; xk ; 0; :::; 0)

is such that
1
P 2 xk 2 2 (14:25)
bk + xk ekn x0 = k y1 ; :::; yk 1 ; xk ; 0; :::; 0k i6=k (yi ) + <

1
P 2 2 yk
2 2
(14:21) (14:20)
< i6=k (yi ) + = 2 kyk = 2 kvk < :

Summarizing, if jxk j < 2 jyk j, then

bk + xk ekn 2 B (x0 ; ) : (14.27)

Now, de…ne for any k 2 f1; :::; mg,

gk : ( 2 jyk j ; +2 jyk j) ! R; xk 7! gk (xk ) := f bk + xk ekn :

Then using the same trick used in Remark 596 we have that for any xk 2 ( 2 jyk j ; +2 jyk j),

gk (xk + h) gk (xk ) f bk + xk ekn + hekn f bk + xk ekn

lim = lim = Dxk f bk + xk ekn ;
h!0 h h!0 h
i.e., gk is di¤erentiable. Then we can use the Intermediate Value Theorem for di¤erentiable functions
from subsets of R to R applying it to gk : [0; yk ] ( 2 jyk j ; +2 jyk j) ! R or to gk : [ yk ; 0]
( 2 jyk j ; +2 jyk j) ! R to conclude that there exists k 2 (0; yk ) such that gk ( yk ) gk (0) = g 0 ( k )
( yk ), or, there exists k 2 ( yk ; 0) such that gk (0) gk ( yk ) = g 0 ( k ) ( yk ), i.e., gk (0) + gk ( yk ) =
g 0 ( k ) ( yk ) :
Summarizing,

for any k 2 f1; :::; mg there exists k 2 (0; yk ) or k 2 ( yk ; 0) such that

gk ( yk ) gk (0) = g 0 ( k) ( yk ),

and
k
for any k 2 f1; :::; mg there exists ak := bk + k en 2 L bk ; bk + yk ekn such that
(14.28)
f bk + yk ekn f (bk ) = Dxk f (ak ) yk .
def. a def. b !0) !0
Now, lim !0 ak = k lim !0 bk + k ekn = k lim !0 x0 + vk 1 + k
k en =k x0 . Then, since
f is C 1 , we do have lim !0 Dxk f (ak ) = Dxk f (x0 ) and also

for any k 2 f1; :::; mg ; lim Dk f ( k) = lim Dk f ( k) = Dk f (x0 )

k !x0 !0

we have

for any k 2 f1; :::; mg ; Dxk f ( k) = Dk f (x0 ) + Ek ( ) with lim Ek ( ) = 0: (14.29)

!0

Inserting (14.29) in (14.28), we get

f bk + yk ekn f (bk ) = yk Dk f (x0 ) + yk Ek ( ) (14.30)

Inserting (14.30) in (20.3), we get

n
X n
X
f (x0 + v) f (x0 ) = yk Dk f (x0 ) + yk Ek ( ) = Df (x0 ) y + kvk Ex0 (v) (14.31)
k=1 k=1
202 CHAPTER 14. SOME THEOREMS

where
n
X
Ex0 (v) := y k Ek ( ) :
k=1
Then from (14.29), we have
n
X
lim Ex0 (v) = lim yk Ek ( ) = 0: (14.32)
v!0 !0
k=1
(14.31) and (14.32) are the de…nition of di¤erentiability.
Example 662 Consider f : R2 ! R2 ; f : (x; y) 7! (sin xy; cos xy). The Jacobian of f is
y cos xy x cos xy
y sin xy x sin xy
So it is clear that f is C 1 on R2 because each partial derivative exists and is continuous for any (x; y) 2 R2
and therefore f is di¤ erentiable and its derivative is its Jacobian.
Remark 663 The above result is the answer to Question 3 in Remark 624. To show that f : Rm ! Rn is
di¤ erentiable, it is enough to verify that all its partial derivatives, i.e., the entries of the Jacobian matrix,
are continuous functions.

14.4 A su¢ cient condition for equality of mixed partial derivatives

Remark 664 We may have
@2f @2f
(x0 ) 6= (x0 ) ;
@xi @xk @xk @xi
as shown below.
Consider 8
> xy (x2 y 2 )
< x2 +y 2 if (x; y) 6= 0
2
f : R ! R; f (x; y) =
>
:
0 if (x; y) = 0
We want to check that
@2f @2f
(0; 0) 6= (0; 0)
@x@y @y@x
Indeed,
@f 2
if (x; y) 6= 0; (x; y) = y 3x2 y2 x2 + y 2 2x x3 xy 2 y x2 + y 2 =
@x
2
= y x4 y 4 + 4x2 y 2 x2 + y 2 ;
and
h 0 h2
@f f (h; 0) f (0; 0) h2 0
(0; 0) = lim = lim = lim 0 = 0;
@x h!0 h h!0 h h!0
therefore
@f (x;y)
@ @x
@f
(0; h) @f
(0; 0) h5
@x @x
(0; 0) = lim = lim = 1:
@y h!0 h h!0 h5
Similarly, we have that
@f 2
if (x; y) 6= 0; (x; y) = x x2 y2 2y (xy) x2 + y 2 2y (xy) x2 y2 x2 + y 2 ;
@y
and
0 h h2
@f f (0; h) f (0; 0) h2 0
(0; 0) = lim = lim = lim 0 = 0;
@y h!0 h h!0 h h!0

@f (x;y) @f @f
@ @y (h; 0) (0; 0) h5
@y @y
(0; 0) = lim = lim = 1 6= 1:
@x h!0 h h!0 h5
14.4. A SUFFICIENT CONDITION FOR EQUALITY OF MIXED PARTIAL DERIVATIVES 203

The following Proposition gives a su¢ cient condition to avoid the above nuisance.

Proposition 665 Given f : S Rn ! R and x0 2 Int (S), if, for any i; k 2 1; :::; n,
@f @f
1. @x i
and @x k
exist on B (x0 ; ) S, and
2. they are di¤ erentiable in x0 , then

@2f @2f
(x0 ) = (x0 ) (14.33)
@xi @xk @xk @xi

Proof. Since we are considering only two variables (xi and xk ) with respect to which we di¤ erentiate,
and we are keeping the other variables …xed, without loss of generality, we can restrict ourselves to a function
f : R2 ! R; (x; y) 7! f (x; y). Finally, for computational simplicity, we consider x0 = (0; 0). Therefore we
have to prove that
@2f @2f
(0; 0) = (0; 0)
@x@y @y@x
Consider h su¢ ciently small so that (h; h) ; (h; 0) ; (0; h) 2 B (0; ). De…ne

(h) = f (h; h) f (h; 0) f (0; h) + f (0; 0)

and
:( ; ) ! R; (x) = f (x; h) f (x; 0)
Then
(h) = (h) (0)
@f
and since, by Assumption 1., @x exists on B (0; ) S

0 @f @f
(x) = (x; h) (x; 0) : (14.34)
@x @x
Since, by Assumption 2., is di¤ erentiable on ( ; ), we can apply the "One-dimensional Mean Value
Theorem" to on [0; h] therefore, 9^1 := 1 h with 1 2 (0; 1) such that

0 @f @f
(h) = (h) (0) = (h 0) ( 1 h) = h ( 1 h; h) ( 1 h; 0) : (14.35)
@x @x

De…ne
@f
g : ( h; h) ! R; g (y) = ( 1 h; y) ; (14.36)
@x
so that
@2f
g 0 (y) = ( 1 h; y) : (14.37)
@x@y
Then
(14:36) @f @f (14:35)
h [g (h) g (0)] = h ( 1 h; h) ( 1 h; 0) = (h) (14.38)
@x @x
Now we apply the "One dimensional Mean Value Theorem" to g (y) on [0; h], so that 9^2 = 2h with
2 2 (0; 1) such that
h g 0 ( 2 h) = g (h) g (0) (14.39)
and
(14:38);(14:39) (14:37) @2f
(h) = h2 g0 ( 2 h) = h2 ( 1 h; 2 h) : (14.40)
@x@y
We might have started by expressing (h) as follows

(h) = (h) (0)

with
(y) = f (h; y) f (0; y)
204 CHAPTER 14. SOME THEOREMS

Now we can follow the same procedure above to show a needed result similar to the one in (14.40). De…ne
(h) = f (h; h) f (h; 0) f (0; h) + f (0; 0)
and
:( ; ) ! R; (y) = f (h; y) f (0; y)
Then
(h) = (h) (0)
@f
and since, by assumption 1., @y exists on B (0; ) S

0 @f @f
(y) = (h; y) (0; y) : (14.41)
@y @y
Since, by assumption 2., is di¤ erentiable on ( ; ), we can apply the "One-dimensional Mean Value
Theorem" to on [0; h] so that 9^3 := 3 h with 3 2 (0; 1) such that

0 @f @f
(h) = (h) (0) = (h 0) ( 3 h) = h (h; 3 h) (0; 3 h) : (14.42)
@y @y
De…ne
@f
g : ( h; h) ! R; g (x) = (x; 3 h) ; (14.43)
@y
so that
@2f
g 0 (x) = (x; 3 h) : (14.44)
@y@x
Then
14:43 @f @f 14:42
h [g (h) g (0)] = h (h; 3 h) (0; 3 h) = (h) (14.45)
@y @y
Now we apply the "One dimensional Mean Value Theorem" to g (x) on [0; h], so that 9^4 = 4h with
4 2 (0; 1) such that
h g 0 ( 4 h) = g (h) g (0) (14.46)
and
(14:45);(14:46) (14:44) @2f
(h) = h2 g 0 ( 4 h) = h2 ( 4 h; 3 h) (14.47)
@y@x
From (14.40) and (14.47) we get
@2f @2f
( 1 h; 2 h) = ( 4 h; 3 h)
@x@y @y@x
@2f @2f
Taking the limits for h ! 0, we have ih ! 0 for i = 1; 2; 3; 4 and then by the continuity of @x@y and @y@x ,
@2f @2f
i.e., Assumption 2., we get @x@y (0; 0) = @y@x (0; 0), as desired

14.5 Taylor’s theorem for real valued functions

To get Taylor’s theorem for a function fP: Rm ! R, we introduce some notation in line with the de…nition
n
of directional derivative, i.e., f 0 (x; u) = i=1 Dxi f (x) ui :
De…nition 666 Assume S is an open subset of Rm , the function f : S ! R admits partial derivatives at
least up to order m, x 2 S; u 2 Rm . Then
n X
X n
f 00 (x; u) := Di;j f (x) ui uj ;
i=1 j=1

n X
X n X
n
f 000 (x; u) := Di;j;k f (x) ui uj uk
i=1 j=1 k=1

and similar de…nition applies to f (m) (x; u).

14.5. TAYLOR’S THEOREM FOR REAL VALUED FUNCTIONS 205

Proposition 667 (Taylor’s formula) Assume S is an open subset of Rm , the function f : S ! R admits
partial derivatives at least up to order m, x 2 S; u 2 Rm . Assume also that all its partial derivative of order
< m are di¤ erentiable. If y and x are such that L (y; x) S, then there exists z 2 L (y; x) such that
m
X1 1 (k) 1 (m)
f (y) = f (x) + f (x; y x) + f (z; y z) :
k! m!
k=1

Proof. . Since S is open and L (x; y) S, 9 > 0 such that 8t 2 ( ; 1 + ) we have x + t (y x) 2 S.

De…ne g : ( ; 1 + ) ! R
g (t) = f (x + t (y x)) :
From standard “Calculus 1” Taylor’s theorem, we have that 9 2 (0; 1) such that
m
X1 1 (k) 1 (m)
f (y) f (x) = g (1) g (0) = g (0) + g ( ):
k! m!
k=1

Moreover,
n
X
g 0 (t) = Df (x + t (y x)) (y x) = Dxi f (x + t (y x)) (yi xi ) = f 0 (x + t (y x) ; y x) ;
i=1

n X
X n
g 00 (t) = Dxi ;xj f (x + t (y x)) (yi xi ) (yj xj ) = f 00 (x + t (y x) ; y x)
i=1 j=1

and similarly
g (m) (t) = f 0(m) (x + t (y x) ; y x)
Then the desired result follow substituting 0 in the place of t where needed and choosing z = x+ (y x).
206 CHAPTER 14. SOME THEOREMS
Chapter 15

Implicit function theorem

15.1 Some intuition

Below, we present an informal discussion of the Implicit Function Theorem. Assume that

f : R2 ! R; (x; t) 7! f (x; t)

is at least C 1 . The basic goal is to study the nonlinear equation

f (x; t) = 0;

where x can be interpreted as an endogenous variable and t as a parameter (or an exogenous variable).
Assume that
9 x0 ; t0 2 R2 such that f x0 ; t0 = 0
and for some " > 0
9 a C 1 function g : t0 "; t0 + " ! R; t 7! g (t)
such that
f (g (t) ; t) = 0 (15.1)
We can then say that g describes the solution to the equation

f (x; t) = 0;

in the unknown variable x and parameter t, in an open neighborhood of t0 . Therefore, using the Chain
Rule - and in fact, Remark 642 - applied to both sides of (15:1), we get

@f (x; t) dg (t) @f (x; t)

+ =0
@x jx=g(t) dt @t jx=g(t)

and
@f (x; t)
assuming that 6= 0
@x jx=g(t)
we have
@f (x;t)
dg (t) @t jx=g(t)
= @f (x;t)
(15.2)
dt
@x jx=g(t)

The above expression is the derivative of the function implicitly de…ned by (15:1) close by to the value
t0 . In other words, it is the slope of the level curve f (x; t) = 0 at the point (t; g (t)).
For example, taken
f : R2 ! R, (x; t) 7! x2 + t2 1
f (x; t) = 0 describes the circle with center in the origin and radius equal to 1. Putting t on the horizontal
axis and x on the vertical axis, we have the following picture.

207
208 CHAPTER 15. IMPLICIT FUNCTION THEOREM

Clearly
f ((0; 1)) = 0
p
As long as t 2 ( 1; 1) ; g (t) = 1 t2 is such that

f (g (t) ; t) = 0 (15.3)

Observe that
p
d 1 t2 t
= p
dt 1 t2
and
@f (x;t)
@t jx=g(t) 2t t
@f (x;t)
= = p
2x jx=g(t) 1 t2
@x jx=g(t)

p1
For example for t = p1 , g 0 (t) = p 2
= 1:
2 1
1 2

Let’s try to present a more detailed geometrical interpretation1 . Consider the set (x; t) 2 R2 : f (x; t) = 0
presented in the following picture.
Insert picture a., page 80 in Sydsaeter (1981).
In this case, does equation
f (x; t) = 0 (15.4)
de…ne x as a function of t? Certainly, the curve presented in the picture is not the graph of a function
with x as dependent variable and t as an independent variable for all values of t in R. In fact,
1. if t 2 ( 1; t1 ], there is only one value of x which satis…es equation (15:4);
2. if t 2 (t1 ; t2 ), there are two values of x for which f (x; t) = 0;
3. if t 2 (t2 ; +1), there are no values satisfying the equation.
If we consider t belonging to an interval contained in (t1 ; t2 ), we have to to restrict the admissible range
of variation of x in order to conclude that equation (15:4) de…nes x as a function of t in that interval. For
example, we see that if the rectangle R is as indicated in the picture , the given equation de…nes x as a
function of t, for well chosen domain and codomain - naturally associated with R. The graph of that function
is indicated in the …gure below.
Insert picture b., page 80 in Sydsaeter (1981).
The size of R is limited by the fact that we need to de…ne a function and therefore one and only one value
has to be associated with t: Similar rectangles and associated solutions to the equation can be constructed
for all other points on the curve, except one: (t2 ; x2 ). Irrespectively of how small we choose the rectangle
around that point, there will be values of t close to t2 , say t0 , such that there are two values of x, say x0
and x00 , with the property that both (t0 ; x0 ) and (t0 ; x00 ) satisfy the equation and lie inside the rectangle.
Therefore, equation (15:4) does not de…ne x as a function of t in an open neighborhood of the point (t2 ; x2 ).
In fact, there the slope of the tangent to the curve is in…nite. If you try to use expression (15:2) to compute
the slope of the curve de…ned by x2 + t2 = 1 in the point (1; 0), you get an expression with zero in the
denominator.
On the basis of the above discussion, we see that it is crucial to require the condition

@f (x; t)
6= 0
@x jx=g(t)

to insure the possibility of locally writing x as a solution (to (15:4)) function of t.

We can informally, summarize what we said as follow.
If f is C 1 ; f (x0 ; t0 ) = 0 and @f@x
(x;t)
j(x;t)=(x0 ;t0 )
6= 0, then f (x; t) = 0 de…ne x as a C 1 function g of t in
@f (x;t)
an open neighborhood of t0 , and g 0 (t) = @t
@f (x;t) .
@x jx=g(t)
Next sections provide a formal statement and proof of the Implicit Function Theorem. Some work is
needed.
1 This discussion is taken from Sydsaeter (1981), page 80-81.
15.2. FUNCTIONS WITH FULL RANK SQUARE JACOBIAN 209

15.2 Functions with full rank square Jacobian

This section gives some properties of function with nonzero Jacobian determinant at certain points. These
results will be used later in the proof of the implicit function theorem.
Proposition 668 Taken a 2 Rn , r 2 R++ ; assume that
n
1. f := (fi )i=1 : Rn ! Rn is continuous on Cl (B (a; r)) ;
2. 8x 2 B (a; r) ; [Df (x)]n n exists and det Df (x) 6= 0;
3. 8x 2 F (B (a; r)) ; f (x) 6= f (a) :
Then, 9 2 R++ such that
f (B (a; r)) B (f (a) ; ) ;
i.e., f (a) 2 Intf (B (a; r)).
Proof. De…ne B := B (a; r) and
g : F (B) ! R; x 7! kf (x) f (a)k :
From Assumption 3, 8x 2 F (B), g (x) > 0. Moreover, since g is continuos and F (B) is compact, then g
attains a global minimum value m > 0 on F (B). Taken = m 2 ; we want to show that T := B (f (a) ; )
f (B), i.e., 8y 2 T , y 2 f (B) : De…ne
h : Cl (B) ! R; x 7! kf (x) yk :
Since h is continuous and Cl (B) is compact, h attains a global minimum in a point c 2 Cl (B).
Claim 669 c 2 Int (B).
Proof of the Claim Since Cl (S) = Int (S) [ F (S) it su¢ ces to show that c 62 F. Observe that, since
y 2 T = B f (a) ; = m 2 , then
m
h (a) = kf (a) yk < (15.5)
2
Therefore, since c is a global minimum point for h, it must be the case that h (c) h (a) < m
2 . Now take
x 2 F (B); then
(1)
h (x) = kf (x) yk = kf (x) f (a) (y f (a))k
m (3) m (15:5)
(2)
kf (x) f (a)k ky f (a)k > h (a) ;
g (x)
2 2
where (1) follows from Remark 54, (2) from (15:5) and (3) from the fact that g has minimum value
equal to m. Therefore, [8x 2 F (B) ; h (x) > h (a)] and h does not attain its minimum on F (B), therefore
c 62 F (B).
End of Proof the Claim
Then h and h2 get their minimum at c 2 B 2 . Since
n
X
2 2
H (x) := h2 (x) = kf (x) yk = (fi (x) yi )
i=1

from Proposition 655, DH (c) = 0, i.e.,

n
X
8k 2 f1; :::; ng ; 2 Dxk fi (c) (fi (c) yi ) = 0
i=1

i.e.,
[Df (c)] (f (c) y) = 0:
Then, from Assumption 2 and from the Claim above, Df (c) has full rank and therefore
f (c) = y;
i.e., since c 2 B; y 2 f (B), ad desired.
The following pictures show a case where the hypothesis and the conclusion of the above Proposition are
satis…ed and some cases in which some hypothesis are not satis…ed and the conclusion of the theorem is not
true. The numbers under each picture indicate the hypothesis which are not satis…ed.
2 For any x 2 Cl (B), h (x) 0, c 2 B and h (x) h (c) 0. Therefore, h2 (x) h2 (c).
210 CHAPTER 15. IMPLICIT FUNCTION THEOREM

De…nition 670 Given a set X Rn and a set Y Rm , a function f : X ! Y is an open function if 8

open subset U of X, f (U ) is open in Y .

Now we are going to present some conditions for openness of a function.

Proposition 671 (1st su¢ cient condition for openness of a function)

Let an open set A Rn and a function f : A ! Rn be given. If
1. f is continuous,
2. f is one-to-one,
3. 8x 2 A, Df (x) exists and det Df (x) 6= 0,
then f (A) is open.

Proof. Taken b 2 f (A), there exists a 2 A such that f (a) = b. Since A is open, there exists r 2 R++
such that B := B (a; r) A. Moreover, since f is one-to-one and since a 2
= F (B), 8x 2 F (B), f (x) 6=
f (a) :Then3 , for su¢ ciently small r, Cl (B (a; r)) A, and the assumptions of Proposition 668 are satis…ed
and there exists 2 R++ such that

f (A) f (Cl (B (a; r))) B (f (a) ; ) ;

as desired.

De…nition 672 Given f : S ! T , and A S, the function fjjA is de…ned as follows

fjjA : A ! f (A) ; fjjA (x) = f (x) :

4

Proposition 673 Let an open set A Rn and a function f : A ! Rn be given. If

1. f is C 1 ,
2. 9a 2 A such that det Df (a) 6= 0,
then 9r 2 R++ such that f is one-to-one on B (a; r), and, therefore, fjjB(a;r) is invertible.
n n
Proof. Consider (Rn ) with generic element z := z i i=1
, where 8i 2 f1; :::; ng ; z i 2 Rn , and de…ne
2 3
Df1 (z 1 )
6 ::: 7
n 6 7
h:R n2
! R; z i i=1 7 det 6
! 6 Dfi z i 7 :
7
4 ::: 5
Dfn (z n )

Observe that h is continuous because f is C 1 and the determinant function is continuous in its entries.
Moreover, from Assumption 2,
h (a; :::; a; :::; a) = det Df (a) 6= 0:
Therefore, 9r 2 R++ such that
n n
8 zi i=1
2 B ((a; :::; a; :::; a) ; nr) ; h zi i=1
6= 0:

De…ned n o
n 2
^ :=
B zi 2 Rn : 8i 2 f1; :::; ng ; z i = z 1 ;
i=1

^ 6= ?.
observe that B := B ((a; :::; a; :::; a) ; nr) \ B
De…ned also
n n
proj : (Rn ) ! Rn ; proj : z i i=1
7! z 1 ;
3 Simply observe that 8r 2 R++ , Cl (B) x; r2 B (x; r).
4 We distinguish between fjjA de…ned above and fjA de…ned as follows:

fjA : A ! T; fjA (x) = f (x) :

Then FjjA is by de…nition onto, while fjA in not necessarly so.
15.2. FUNCTIONS WITH FULL RANK SQUARE JACOBIAN 211

5
observe that proj (B ) = B (a; r) Rn and 8i 2 f1; ::; ng ; 8zi 2 B (a; r), z 1 ; :::; z i :::; z n 2 B and
therefore h z 1 ; :::; z i :::; z n 6= 0; or, summarizing,

9r 2 R++ such that 8i 2 f1; ::; ng ; 8zi 2 B (a; r) ; h z 1 ; :::; z i :::; z n 6= 0

We now want to show that f is one-to-one on B (a; r). Suppose otherwise, i.e., given x; y 2 B (a; r),
f (x) = f (y), but x 6= y. We can now apply the Mean Value Theorem (see Remark 644) to f i for any
i 2 f1; :::; ng on the segment L (x; y) B (a; r). Therefore,

8i 2 f1; ::; ng ; 9z i 2 L (x; y) such that 0 = fi (x) fi (y) = Df z i (y x)

i.e., 2 3
Df1 (z 1 )
6 ::: 7
6 7
6 Dfi z i 7 (y x) = 0
6 7
4 ::: 5
Dfn (z n )
n
Observe that z i 2 B (a; r) 8i, and therefore z i i=1 2 B ((a; :::; a; :::; a) ; nr)from (15.8) in the previous
footnote and therefore 2 3
Df1 (z 1 )
6 ::: 7
6 7
det 6 Df z i 7 = h zi n 6= 0
6 i 7 i=1
4 ::: 5
Dfn (z n )
and therefore y = x, a contradiction.

Corollary 674 Let an open subset A Rn and a function f : A ! Rn be given. If:

1. f is C 1 ,
2. there exists a 2 A such that det Df (a) 6= 0,
then 9r 2 R++ such that fjB(a;r ) is an open function.
Proof. From continuity of the determinant with respect to the entries of the matrix, 9 r1 > 0 such that
8x 2 B (a; r1 ), det Df (x) 6= 0. From Proposition 673, 9 r2 > 0 such that f is one-to-one on B (a; r2 ). Taken
r = min fr1 ; r2 g, all the assumption of Proposition 671 are satis…ed with respect to fB(a;r ) and therefore it
is open.

Corollary 675 Under the Assumption of Corollary 674, for any open neighborhood Na of a, there exists an
open neighborhood Nf (a) of f (a) such that f (Na ) Nf (a) .
Proof. Since Na is open, then there exists r1 such that

B (a; r1 ) Na : (15.9)

From Corollary 674, there exists r2 > 0 such that fjB(a;r2 ) is an open function.
5
(1) n
X (z1 2B(a;r))
n n
k zi i=1
(a; :::; a) k = k z i a i=1
k kz i ak < nr;
i=1
where (1) follows from what said below.
Given z 1 ; :::; z n 2 Rn ;
n X
X n n
X
n 2
jj z i i=1
jj2 := zji = jjz i jj2 : (15.6)
i=1 j=1 i=1
Moreover,
n
!2 n n X
n n
X X X X
i
jjz jj = jjz i jj2 + 2 jjz i jj jjz j jj jjz i jj2 ; (15.7)
i=1 i=1 i=1 j=1 i=1
and then v
n u n n
X n (15:6) uX (15:7) X
jj z i i=1 jj = t jjz i jj2 jjz i jj; (15.8)
i=1 i=1 i=1

as desidered.
212 CHAPTER 15. IMPLICIT FUNCTION THEOREM

De…ned r = min fr1 ; r2 g, then f (B (a; r)) is open and, obviously, f (a) 2 f (B (a; r)).
Then
r<r1 (15:9)
9" > 0 such that Nf (a) := B (f (a) ; ") f (B (a; r)) f (B (a; r1 )) f (Na ) ; (15.10)

as desired

Remark 676 The result contained in Proposition 72 is not a global result, i.e., it is false that if f is C 1
and its Jacobian has full rank everywhere in the domain, then f is one-to-one. Just take the function tan.
The next result gives a global property (in terms of openness of the function).

Proposition 677 (2nd su¢ cient condition for openness of a function) Let an open set A Rn and a
function f : A ! Rn be given. If
1. f is C 1 ,
2. 8x 2 A, det Df (x) 6= 0,
then f is an open function.

Proof. Take an open set S A. From Proposition 673,8x 2 S there exists rx 2 R++ such that f is
one-to-one on B (x; rx ). Moreover, for any x 2 S, 9rx0 > 0 such that B (x; rx0 ) S. Take rx := min frx ; rx0 g.
Then, from Proposition 671, f (B (x; rx )) is open in Rn . Since S is open, we can then write S = [x2S B (x; rx )
and
f (S) = f ([x2S B (x; rx )) = [x2S f (B (x; rx ))
where the second equality follows from Proposition 515 506.2..f ), and then f (S) is an open set.

15.3 The inverse function theorem

Proposition 673 shows that a C 1 function with full rank square Jacobian in a point a has a local inverse in
an open neighborhood of a. The inverse function theorem give local di¤erentiability properties of that local
inverse function.

Lemma 678 Let X and Y be subsets of Euclidean spaces. If g is the inverse function of f : X ! Y and
A X, then gjjf (A) is the inverse of fjjA , and
if g is the inverse function of f : X ! Y and B Y , then gjjB is the inverse of fjjg(B) .

Proof. Proof below to be reviewed carefully.

By assumption
f : X ! Y; x 7! f (x)
and, from De…nition 672
fjjA : A ! f (A) ; x 7! fjjA (x) = f (x) :
Since f is invertible by assumption, then fjjA is invertible: f is one-to-one and a fortiori fjjA is one-to-one;
furthermore fjjA is also onto by de…nition. So the inverse function of fjjA is

fjjA1 : f (A) ! A y = f (x) 7! f 1

(y) = f 1
(f (x)) = x: (15.11)

By assumption g is the inverse function of f and therefore it is de…ned as

g : Y ! X; y = f (x) 7! g (y) = g (f (x)) = x

and
gjjf (A) : f (A) ! g (f (A)) ; y = f (x) 7! gjjf (A) (y) = g (y) = g (f (x)) = x: (15.12)
Moreover,
g (f (A)) = A; (15.13)
and therefore (15:11), (15:12) and (15:13) prove the …rst statement.
Now let’s move on the second statement, whose proof is almost identical to the above one.
15.3. THE INVERSE FUNCTION THEOREM 213

By assumption
f : X ! Y; x 7! f (x)
and, from De…nition 672

fjjg(B) : g (B) ! f (g (B)) ; x 7! fjjg(B) (x) = f (x) :

Since f is invertible by assumption, then fjjg(B) is invertible: f is one-to-one and a fortiori fjjg(B) is one-to-
one; furthermore fjjg(B) is also onto by de…nition. So the inverse function of fjjg(B) is
1 1 1
fjjg(B) : f (g (B)) ! g (B) y = f (x) 7! f (y) = f (f (x)) = x: (15.14)

By assumption g is the inverse function of f and therefore it is de…ned as

g : Y ! X; y = f (x) 7! g (y) = g (f (x)) = x

and
gjjB : B ! g (B) ; y = f (x) 7! gjjB (y) = g (y) = g (f (x)) = x (15.15)
Then,
f (g (B)) = B (15.16)
and therefore (15:14), (15:15) and (15:16) prove the second statement.

Proposition 679 Let f : X ! Y be a function from a metric space (X; d) to another metric space (Y; d0 ).
Assume that f is one-to-one and onto. If X is compact and f is continuous, then the inverse function f 1
is continuous.
1
Proof. It is su¢ cient to show that for any closed set S in X, f 1 (S) = f (S) is closed in Y . Take
S closed set in X. Then, from Proposition 409, S is compact. But, being f continuous, from Proposition
521, f (S) is compact and therefore it is closed from Proposition 415.

Proposition 680 (Inverse function Theorem) Let an open set S Rn and a function f : S ! Rn be given.
If
1. f is C 1 , and
2. 9a 2 S, such that det Df (a) 6= 0,
then there exist two open sets X S and Y f (S) and a unique function g such that
1. a 2 X and f (a) 2 Y ,
2. Y = f (X),
3. f is one-to-one on X,
4. g : Y ! X is the inverse of fjjX (and X = g (Y ))
5. g is C 1 .

Proof. Since f is C 1 , 9r1 2 R++ such that 8x 2 B (a; r1 ) ; det Df (x) 6= 0. Then, from Proposition
673, f is one-to-one on B (a; r1 ). Then take r2 2 (0; r1 ), and de…ne B := B (a; r2 ) : Observe that Cl (B)
B (a; r1 ) :Using the fact that f is one-to-one on B (a; r1 ) and therefore on B (a; r2 ), we get that Assumption
3 in Proposition 668 is satis…ed - while the other two are trivially satis…ed. Then, 9 2 R++ such that

f (B) B (f (a) ; ) := Y:

De…ne also
1
X := f (Y ) \ B; (15.17)
an open set because Y and B are open sets and f is continuous. Since f is one-to-one and continuous
on the compact set Cl (B) , from Proposition 78, there exists a unique continuous inverse function gb :
f (Cl (B)) ! Cl (B) of fjCl (B) . From de…nition of Y ,

Y f (B) f (Cl (B)) : (15.18)

From de…nition of X,

f (X) = Y \ f (B) = Y: (15.19)

214 CHAPTER 15. IMPLICIT FUNCTION THEOREM

Then, from Lemma 678,

g := g^jjY is the inverse f unction of fjjX ; (15.20)
(15:20) (15:19)
Then, X = g (f (X)) = g (Y ).
The above results show conclusions 2-4 of the Proposition. About conclusion 1, observe that a 2
f 1 (B (f (a; ))) \ B (a; r2 ) = X and f (a) 2 f (X) = Y .
We are then left with proving condition 5.
Following what already said in the proof of Proposition 673, we can de…ne
2 3
Df1 (z 1 )
6 ::: 7
n 6 7
: z i i=1 7! det 6 7 :
2
i
h : Rn ! R; 6 Dfi z 7
4 ::: 5
Dfn (z n )

and get that, from Assumption 2,

h (a; :::; a; :::; a) = det Df (a) 6= 0;

and- see the proof of Proposition 673 for details-

9r 2 R++ such that 8i 2 f1; ::; ng ; 8zi 2 B (a; r) ; h z 1 ; :::; z i :::; z n 6= 0; (15.21)

and trivially also

9r 2 R++ such that 8z 2 B (a; r) ; h (z; :::; z:::; z) = det Df (z) 6= 0: (15.22)

Assuming, without loss of generality that r1 < r, we have that

Cl (B) := Cl ((B) (a; r2 )) B (a; r1 ) B (a; r) : (15.23)

n
Then,8z 1 ; :::; z n 2 Cl (B) ; h z 1 ; :::; z i :::; z n 6= 0: Writing g = g i i=1 , we want to prove that 8i 2
f1; ::; ng, g i is C 1 . We go through the following two steps: 1. 8y 2 Y; 8i; k 2 f1; ::; ng ; Dyk g i (y) exists, and
2: it is continuous.
Step 1.
We want to show that the following limit exists and it is …nite

g i y + hekn g i (y)
lim :
h!0 h
De…ne n
x = (xi )i=1 = g (y) 2 X Cl (B)
(15.24)
0 n
x = (x0i )i=1 =g y+ hekn 2X Cl (B)
Then
f (x0 ) f (x) = y + hekn y = hekn : (15.25)
We can now apply the Mean Value Theorem to f i for i 2 f1; ::; ng: 9z i 2 L (x; x0 ) Cl (B) , where the
inclusion follows from the fact that x; x0 2 Cl (B) a convex set, such that

f i (x0 ) f i (x) Df i z i (x0 x)

8i 2 f1; ::; ng ; = (15.26)
h h
and therefore, from (15.25) and (15.26)
2 3
Df1 (z 1 )
6 ::: 7
6 71 0
6 Dfi z i 7 (x x) = ekn : (15.27)
6 7h
4 ::: 5
Dfn (z n )
15.4. THE IMPLICIT FUNCTION THEOREM 215

De…ne 2 3
Df1 (z 1 )
6 ::: 7
6 7
A=6
6 Dfi z i 7
7
4 ::: 5
Dfn (z n ) m m

Then, from (15:21), (15.27) admits a unique solution,

1 0 (15:24) g y + hekn g (y) ' z 1 ; :::; z n

(x x) = =
h h h (z 1 ; :::; z n )
where , from the Cramer rules, ' takes values which are determinants of a matrix involving entries of A
(and therefore they are continuous). We are left with showing that

' z 1 ; :::; z n
lim
h!0 h (z 1 ; :::; z n )

exists and it is …nite, i.e., the limit of the numerator exists and its …nite and the limit of the denominator
exists is …nite and nonzero.
Then, if h ! 0, y + hekn ! y, and, being g continuous, x0 ! x and, since z i 2 L (x; x0 ), z i ! x for any
i. Then, h z 1 ; :::; z n ! h (x; :::; x) 6= 0; because, from 15.24, x 2 Cl (B) and from (15:22) and (15.23).
Moreover, ' z 1 ; :::; z n ! ' (x; :::; x).
Step 2.
Since
g i y + hekn g i (y) ' (x; :::; x)
lim =
h!0 h h (x; :::; x)
and ' and h are continuous functions, the desired result follows.

15.4 The implicit function theorem

Theorem 681 Given open sets S Rn and T Rk and a function

f :S T ! Rn ; : (x; t) 7! f (x; t) ;

assume that

1. f is C 1 , and
there exists (x0 ; t0 ) 2 S T such that
2. f (x0 ; t0 ) = 0,
3. Dx f (x0 ; t0 )n n is invertible.

Then there exists N (t0 ) T open neighborhood of t0 and a unique function

g : N (t0 ) ! Rn

such that

1. g is C 1 ,
2. g (t0 ) = x0
3. f(x; t) 2 Rn N (t0 ) : f (x; t) = 0g = f(x; t) 2 Rn N (t0 ) : x = g(t)g := graph g.

Proof. Proof below to be reviewed carefully.

The main idea to prove the theorem is as follows:

1. Apply the Inverse Function Theorem to F (x; t) = (f (x; t) ; t) around (x0 ; t0 );

216 CHAPTER 15. IMPLICIT FUNCTION THEOREM

1
2. De…ne G = F with F (A) = B and G (B) = A;
3. De…ne N (t0 ) = ft 2 T : (0; t) 2 Bg, g (t) = v (0; t).

We want to apply the Inverse Function Theorem to the following function

F :S T ! Rn+k F : (x; t) 7! (f (x; t) ; t) :

Observe that
D(x;t) f (x; t) Dx f (x; t) Dt f (x; t)
D(x;t) F (x; t) = =
(n+k) (n+k) D(x;t) t 0 I

Therefore det D(x;t) F (x; t) = det Dx f (x; t) and in particular,

det D(x;t) F (x0 ; t0 ) = detDx f (x0 ; t0 ) 6= 0: (15.28)

Moreover,
(1)
F (x0 ; t0 ) = (f (x0 ; t0 ) ; t0 ) = (0; t0 )
where
(1) follows from Assumption 2.
Because f is C 1 by Assumption 1 and the function id : Rk ! Rk ; t 7! t is C 1 as a function, because
it is linear, F is C 1 on S T . Therefore we can apply the Inverse Function Theorem to F around (x0 ; t0 )
where det D(x;t) F (x0 ; t0 ) 6= 0 from 15.28, i.e., there exist two open sets A S T and B F (S T ) and
a unique function G such that

1. (x0 ; t0 ) 2 A and F (x0 ; t0 ) = (0; t0 ) 2 B ,

2. B = F (A),
3. F is one-to one on A,
4. G : B ! A is the inverse function of FjjA and A = G (B),
5. G is C 1 on B.

Being G : B ! A and A; B Rn+k , we can de…ne v : B ! S Rn and w : B ! T Rk such that

G = (v; w). Therefore,
(x; t) = G (F (x; t)) = (v (F (x; t)) ; w (F (x; t)))
i.e.,
v (F (x; t)) = x (15.29)
and
w (F (x; t)) = t (15.30)
0 0 0 0
Being F restricted to A onto and one-to-one, for any (x; t) 2 B, 9! (x ; t ) 2 A such that [F (x ; t ) = (x; t)],
then,by de…nition of F , we have that t0 = t, i.e., shortly,

8 (x; t) 2 B 9! (x0 ; t) 2 A such that (x; t) = F (x0 ; t) : (15.31)

Therefore applying v to the both side of the equality in (15.31), we get

(15:29)
8 (x; t) 2 B; v (x; t) = v (F (x0 ; t)) = x0 : (15.32)

Now since from 1. above (0; t0 ) 2 B, v (0; t0 ) is well de…ned. Moreover,

1:above (15:31)
v (0; t0 ) = v (F (x0 ; t0 )) = x0 : (15.33)

We also get that

(15:32) (15:31)
F (v (x; t) ; t) = F (x0 ; t) = (x; t)
15.4. THE IMPLICIT FUNCTION THEOREM 217

Summarizing,
8 (x; t) 2 B; F (v (x; t) ; t) = (x; t) : (15.34)
Now we can de…ne N (t0 ), and g of the conclusion of the theorem as follows:

N (t0 ) = ft 2 T : (0; t) 2 Bg;

which is open in Rk because B is open in Rn+k ; see Lemma 682 below.

g : N (t0 ) ! Rn ; g : t 7! v (0; t)

is C 1 because v are components of G which is C 1 on Y f0g N (t0 ). Moreover

(15:33)
g (t0 ) = v (0; t0 ) = x0 ;

moreover,
(15:34)
8 (x; t) 2 B (f (v (x; t) ; t)) =F (v (x; t) ; t) = (x; t) ;
i.e.,

f (v (x; t) ; t) = x (15.35)
But then since N (t0 ) := ft 2 T : (0; t) 2 Bg we have that

8t 2 N (t0 ) ; f (v (0; t) ; t) = 0
and …nally by de…nition of g, f (g (t) ; t) = 0. The only thing left to prove is that g is unique. Take any
other h such that f (h (t) ; t) = f (g (t) ; t) 8t 2 N (t0 ). But being f one-to-one (because F is one-to-one),
this implies h (t) = g (t).

Lemma 682 De…ned

1 :X Y ! X; (x; y) 7! x
2 :X Y ! Y; (x; y) 7! y:
6
1. Both 1 and 2 are continuous and open function;
1
2. For any open set O X Y , and for any x 2 X, Ox := fy 2 Y : (x; y) 2 Og = 2 O\ 1 (fxg) ;
3. Ox is open.

Proof. 1. We prove the result only for 1, the proof for 2 being almost identical.
a. 1 is continuous.
We have to prove that

8 (x0 ; y0 ) 2 X Y; 8" > 0 9 > 0 such that k (x; y) (x0 ; y0 ) k < ) kx x0 k < "

Indeed take = ". Then

kx x0 k k (x; y) (x0 ; y0 ) k < = ":
b. 1 is open.
We have to prove that S open in X Y ) x (s) open in X. Take y 2 1 (S), we have to …nd " > 0
such that B (y; ") 1 (S), i.e., we want to …nd " > 0 such that

ky 0 yk ) y 0 2 1 (S) ; i:e:; (x; y 0 ) 2 S: (15.36)

Since y 2 1 (S), then, by de…nition of 1, (x; y) 2 S and since S is open then

9 > 0 such that B ((x; y) ; ) S; (15.37)

Take
"= > 0: (15.38)
6 Point 1. is not used to show the implicit function theorem.
218 CHAPTER 15. IMPLICIT FUNCTION THEOREM

Then
(??)
k (x; y 0 ) (x; y) k = ky 0 yk < ":
Then from (15.38) and (15.37), (x; y 0 ) 2 S as desired.
2. y 2 2 O \ 1 1 (fxg) , 9 (x0 ; y 0 ) 2 O \ 1 1 (x) such that 2 (x0 ; y 0 ) = y , 9 (x0 ; y 0 ) 2 O and
(x ; y ) 2 1 1 (x) := f(x00 ; y 00 ) 2 X Y : x00 = xg such that 2 (x0 ; y 0 ) = y , 9 (x0 ; y 0 ) 2 O and such that
0 0

x0 = x y 0 = y 2 Y , y 2 Y; (x; y) 2 O , y 2 Ox .
3. We want to show that if y^ 2 Ox , then

9 > 0 such that B (^

y; ) Ox : (15.39)

Since y^ 2 Ox , then by de…nition of Ox , (x; y^) 2 O, and since O is open, there exists r > 0 such that

B ((x; y^) ; r) O: (15.40)

Now take = r; to show (15.39) we are going to show that

ky y^k < r ) y 2 Ox :

Indeed,
ky y^k < r ) k (x; y) (x y^) < r )

1 1
) (x; y) 2 B ((x; y^) ; r) \ 1 (x) O\ 1 (x) )

1
)y= 2 (x; y) 2 2 O\ 1 (x) = Ox

Remark 683 Conclusion 2. in the statement of Theorem 681 can be rewritten as

8t 2 N (t0 ) ; f (g (t) ; t) = 0 (15.41)

Computing the Jacobian of both sides of (15:41), using Remark 642, we get

8t 2 N (t0 ) , 0 = [Dx f (g (t) ; t)]n n [Dg (t)]n k + [Dt f (g (t) ; t)]n k (15.42)

and using Assumption 3 of the Implicit Function Theorem, we get

1
8t 2 N (t0 ) , [Dg (t)]n k = [Dx f (g (t) ; t)]n n [Dt f (g (t) ; t)]n k

Observe that (15:42) can be rewritten as the following k systems of equations: 8i 2 f1; :::; kg,

[Dx f (g (t) ; t)]n n [Dti g (t)]n 1 = [Dti f (g (t) ; t)]n 1

Example 684 7 Discuss the application of the Implicit Function Theorem to f : R5 ! R2

2ex1 + x2 t1 4t2 + 3
f (x1 ; x2 ; t1 ; t2 ; t3 ) 7!
x2 cos x1 6x1 + 2t1 t3

at x0 ; t0 = (0; 1; 3; 2; 7) :
Let’s check that each assumption of the Theorem is veri…ed.

1. f (x0 ; t0 ) = 0 . Obvious.
7 The example is taken from Rudin (1976), pages 227-228.
15.5. SOME GEOMETRICAL REMARKS ON THE GRADIENT 219

2. f is C 1 :
We have to compute the Jacobian of the function and check that each entry is a continuous function.

x1 x2 t1 t2 t3

2ex1 + x2 t1 4t2 + 3 2ex1 t1 x2 4 0

x2 cos x1 6x1 + 2t1 t3 x2 sin x1 6 cos x1 2 0 1
3. [Dx f (x0 ; t0 )]n n is invertible.
2ex1 t1 2 3
[Dx f (x0 ; t0 )] = =
x2 sin x1 6 cos x1 j(0;1;3;2;7)
6 1

whose determinant is 20.

Therefore, we can apply the Implicit Function Theorem and compute the Jacobian of g : N (t0 ) R2 !
N (x0 ) R3 :
1
2ex1 t1 x2 4 0
Dg (t) = =
x2 sin x1 6 cos x1 2 0 1
1 2t1 x2 cos x1 4 cos x1 t1
=
6t1 + 2 (cos x1 ) ex1 + t1 x2 sin x1 6x2 4ex1 x22 sin x1 4x2 sin x1 + 24 2ex1
Exercise 685 Given the utility function u : R2++ ! R++ ; (x; y) 7! u (x; y) satisfying the following properties
i. u is C 2 , ii. 8 (x; y) 2 R2++ ; Du (x; y) >> 0;iii. 8 (x; y) 2 R2++ ; Dxx u (x; y) < 0; Dyy u (x; y) <
0; Dxy u (x; y) > 0;
compute the Marginal Rate of Substitution in (x0 ; y0 ) and say if the graph of each indi¤ erence curve is
concave.
y 5

3.75

2.5

1.25

0
0 1.25 2.5 3.75 5

15.5 Some geometrical remarks on the gradient

In what follows we make some geometrical, not rigorous remarks on the meaning of the gradient, using the
implicit function theorem. Consider an open subset X of R2 , a C 1 function
f : X ! R; : (x; y) 7! f (x; y)
where a 2 R. Assume that set
L (a) := f(x; y) 2 X : f (x; y) = ag
@f (x;y) @f (x;y)
is such that 8 (x; y) 2 X, @y 6= 0 and @x 6= 0 , then
220 CHAPTER 15. IMPLICIT FUNCTION THEOREM

1. L (a) is the graph of a C 1 function from a subset of R to R;

2. (x ; y ) 2 L (a) ) the line going through the origin and the point Df (x ; y ) is orthogonal to the line
going through the origin and parallel to the tangent line to L (a) at (x ; y ) ;or the line tangent to the
curve L (a) in (x ; y ) is orthogonal to the line to which the gradient belongs to.

3. (x ; y ) 2 L (a) ) the directional derivative of f at (x ; y ) in the the direction u such that kuk = 1
Df (x ;y )
is the largest one if u = kDf (x ;y )k .

1. It follows from the Implicit Function Theorem.

2. The slope of the line going through the origin and the vector Df (x ; y ) is
@f (x ;y )
@y
@f (x ;y )
(15.43)
@x

Again from the Implicit Function Theorem, the slope of the tangent line to L (a) in (x ; y ) is
@f (x ;y )
@x
@f (x ;y )
(15.44)
@y

The product between the expressions in (15:43) and (15:44) is equal to 1.

3. the directional derivative of f at (x ; y ) in the the direction u is

f 0 ( (x ; y ) ; u) = Df (x ; y ) u = kDf (x ; y )k kuk cos

where is an angle in between the two vectors. Then the above quantity is the greatest possible i¤ cos = 1,
Df (x ;y )
i.e., u is colinear with Df (x ; y ), i.e., u = kDf (x ;y )k .

15.6 Extremum problems with equality constraints.

Given the open set X Rn , consider the C 1 functions

f : X ! R; f : x 7! f (x) ;

m
g : X ! Rm ; g : x 7! g (x) := (gj (x))j=1
with m n: Consider also the following “maximization problem”:

(P ) maxx2X f (x) subject to g (x) = 0 (15.45)

The set
C := fx 2 X : g (x) = 0g
is called the constraint set associated with problem (15:45).

De…nition 686 The solution set to problem (15:45) is the set

fx 2 C : 8x 2 C; f (x ) f (x)g ;

and it is denoted by arg max (15:45).

The function
T
L:X Rm ! R; L : (x; ) 7! f (x) + g (x)
is called Lagrange function associated with problem (15:45).
15.6. EXTREMUM PROBLEMS WITH EQUALITY CONSTRAINTS. 221

Theorem 687 Given the open set X Rn and the C 1 functions

m
f : X ! R; f : x 7! f (x) ; g : X ! Rm ; g : x 7! g (x) := (gj (x))j=1 ;

assume that
1. f and g are C 1 functions,
2. x0 is a solution to problem (15:45),8 and
3. rank [Dg (x0 )]m n = m:
Then, there exists 0 2 Rm ; such that, DL (x0 ; 0) = 0, i.e.,

Df (x0 ) + 0 Dg (x0 ) =0
(15.46)
g (x0 ) = 0
m n m
Proof. De…ne x0 := (xi )i=1 2 Rm and t = (xm+k )k=1 2 Rn m
and therefore x = (x0 ; t). From
Assumption 3, without loss of generality,

det [Dx0 g (x0 )]m m 6= 0: (15.47)

We want to show that there exists 0 2 Rm which is a solution to the system

[Df (x0 )]1 n + 1 m [Dg (x0 )]m n = 0: (15.48)

We can rewrite (15:48) as follows

Dx0 f (x0 )1 m j Dt f (x0 )1 (n m) + 1 m Dx0 g (x0 )m m j Dt g (x0 )m (n m) =0

or 8
< Dx0 f (x0 )1 m + 1 m Dx0 g (x0 )m m =0 (1)
(15.49)
: Dt f (x0 )1 Dt g (x0 )m
(n m) + 1 m (n m) =0 (2)
From (15:47), there exists a unique solution 0 to subsystem (1) in (15:49). If n = m, we are done.
Assume now that n > m. We have now to verify that 0 is a solution to subsystem (2) in (15:49), as well.
To get the desired result, we are going to use the Implicit Function Theorem. Summarizing, observe that

1. g is C 1 , 2. g(x00 ; t0 ) = 0, 3. det [Dx0 g (x00 ; t0 )]m m 6= 0;

i.e., all the assumption of the Implicit Function Theorem are veri…ed. Then we can conclude that there
exist N (x0 ) Rm open neighborhood of x00 and a unique function ' : N (t0 ) ! Rn m such that

1. ' is C 1 , 2. ' (t0 ) = x00 , 3. 8t 2 N (t0 ) ; g (' (t) ; t) = 0: (15.50)

De…ne now
F : N (t0 ) Rn m
! R; : t 7! f (' (t) ; t) ;
and
G : N (t0 ) Rn m
! Rm ; : t 7! g (' (t) ; t) :
Then, from (15:50) and from Remark 683, we have that 8t 2 N (t0 ),

0 = [DG (t)]m (n m) = [Dx0 g (' (t) ; t)]m m [D' (t)]m (n m) + [Dt g (' (t) ; t)]m (n m) : (15.51)

Since9 , from (15:50), 8t 2 N (t0 ) ; g (' (t) ; t) = 0 and since

(15:50)
x0 := (x00 ; t0 ) = (' (t0 ) ; t0 ) (15.52)

is a solution to problem (15:45), we have that f (x0 ) = F (t0 ) F (t), i.e., brie‡y,

8t 2 N (t0 ) ; F (t0 ) F (t) :

8 The result does apply to the case in which x is a local maximum for Problem (15:45). Obviously the result apply to the
0
case of (local) minima, as well.
9 The only place where the proof has to be slightly changed to get the result for local maxima is here.
222 CHAPTER 15. IMPLICIT FUNCTION THEOREM

Then, from Proposition 655, DF (t0 ) = 0. Then, from the de…nition of F and the Chain Rule, we have

[Dx0 f (' (t0 ) ; t0 )]1 m [D' (t0 )]m (n m) + [Dt f (' (t0 ) ; t0 )]1 (n m) = 0: (15.53)

Premultiplying (15:51) by , we get

1 m [Dx0 g (' (t) ; t)]m m [D' (t)]m (n m) + 1 m [Dt g (' (t) ; t)]m (n m) = 0: (15.54)

Adding up (15:53) and (15:54) ,computed at t = t0 , get

([Dx0 f (' (t0 ) ; t0 )] + [Dx0 g (' (t0 ) ; t0 )]) [D' (t0 )] + [Dt f (' (t0 ) ; t0 )] + + [Dt g (' (t0 ) ; t0 )] = 0;

and from (15:52),

([Dx0 f (x0 )] + [Dx0 g (x0 )]) [D' (t0 )] + [Dt f (x0 )] + [Dt g (x0 )] = 0: (15.55)

Then, from the de…nition of 0 as the unique solution to (1) in (15:49) ;we have that [Dx0 f (x0 )] + 0
[Dx0 g (x0 )] = 0, and then from (15:55) computed at = 0 , we have

[Dt f (x0 )] + 0 [Dt g (x0 )] = 0;

i.e., (2) in (15:49), the desired result.

15.7 Exercises on part III

Problem sets: all of the problems on part III.
See Tito Pietra’s …le (available on line): Exercises 1 ! 14 (excluding exercises 3, 5, 15).
 Part IV

Nonlinear programming

223
Chapter 16

Convex sets

16.1 De…nition
De…nition 688 A set C Rn is convex if 8x1 , x2 2 C and 8 2 [0; 1], (1 ) x1 + x2 2 C.

De…nition 689 A set C Rn is strictly convex if 8x1 , x2 2 C such that x1 6= x2 , and 8 2 (0; 1),
(1 ) x1 + x2 2 Int C.

Remark 690 If C is strictly convex, then C is convex, but not vice-versa.

Proposition 691 The intersection of an arbitrary family of convex sets is convex.

Proof. We want to show that given a family fCi gi2I of convex sets, if x; y 2 C := \i2I Ci then (1 ) x+
y 2 C. x; y 2 C implies that x; y 2 Ci , 8i 2 I. Since Ci is convex, 8i 2 I, 8 2 [0; 1]; (1 ) x + y 2 Ci ,
and 8 2 [0; 1] (1 ) x + y 2 C.

Exercise 692 8n 2 N; 8i 2 f1; :::; ng ; Ii is an interval in R, then

n
i=1 Ii

is a convex set.

16.2 Separation of convex sets

De…nition 693 Let H be a set in Rn . Then,

h H is a hyperplane i ,
n n
, h 9c0 2 R; (ci )i=1 2 Rn n f0g such that H = f(xi )i=1 2 Rn : c0 + c1 x1 + ::: + cn xn = 0g i :

De…nition 694 Let A and B be subsets of Rn , and let H := fx 2 Rn : c0 + c x = 0g be a hyperplane in

Rn .1 Then, H is said to

1. separate A and B if

A H := fx 2 Rn : c0 + c x 0g and B H+ := fx 2 Rn : c0 + c x 0g ;

i.e.,
8a 2 A; 8b 2 B; c0 + c a 0 c0 + c b;
i.e.,
8a 2 A; 8b 2 B; c a c0 c b;

2. separate A and B properly if it separates them and A [ B * H;

1 When we introduce a hyperplane by equation c0 + cx = 0, consistently with the de…nition of hyperplane, we assume c 6= 0.

225
226 CHAPTER 16. CONVEX SETS

3. separate A and B strictly if

A fx 2 Rn : c0 + c x < 0g and B fx 2 Rn : c0 + c x > 0g :

Example 695 The convex sets (x; y) 2 R2 : x 0 and (x; y) 2 R2 : x > 0; y > x1 in R2 cannot be strictly
separated, but they are properly separated by the y-axis.
Clearly, it is not always possible to separate two convex sets by a hyperplane. For instance, there is no
line in R2 separating the set f0g and the closed unit disc (x; y) 2 R2 : x2 + y 2 1 .

Remark 696 Let A; B be two sets in Rn such that at least one of them is nonempty. If they can be strictly
separated, then they can also be properly separated.

Remark 697 Let H be a hyperplane in Rn , and let A and B be two subsets of H. Then, H separates A
and B, but does not separate them properly.

Proposition 698 Let a hyperplane H := fx 2 Rn : c0 + c x = 0g be given. Then,2.

h H separates properly A and B i , h H separates properly Cl (A) and Cl (B) i :

Proof. [(]
Obvious.
[)]
We …rst present two proofs of the fact H separates Cl (A) and Cl (B), and then we show that the
separation is proper.
1st proof.
h A H i ) h Cl (A) Cl (H ) = H i ;
where we used the fact that H is closed.
Similarly, B H+ ) Cl (H+ ) : H+
2nd proof.
Take (a ; b ) 2 Cl (A) Cl (B). Then there exists sequences (an )n2N 2 A1 and (bn )n2N 2 B 1 such that
an ! a and bn ! b . By assumption,

8n 2 N; c0 + c an 0 c0 + c bn :

Taking limits for n ! +1, we get

c0 + c a 0 c0 + c b ;

as desired.
We now show that the separation is proper:,
S Cl(S)
A [ B * H ) A [ B \ H C 6= ? ) Cl (A) [ Cl (B) \ H C 6= ?:

The following three Propositions are presented without proofs. Detailed, self-contained proofs of those
results are contained, for example, in Villanacci, A., (in progress), Basic Convex Analysis, mimeo, Università
degli Studi di Firenze.

Proposition 699 Let A be a closed nonempty convex set in Rn such that 0 2

= A. Then, there exists a
hyperplane in Rn that strictly separates A and f0g.

Corollary 700 Let B be a closed nonempty convex set in Rn such that w 2

= B. Then, there exists a
hyperplane in Rn that strictly separates B and fwg.

Proof. Exercise.

Proposition 701 Let A be a nonempty convex set in Rn such that 0 2

= A. Then, there exists a hyperplane
H in Rn such that A H and f0g H+ .
16.2. SEPARATION OF CONVEX SETS 227

Proposition 702 Let A be a nonempty convex set in Rn such that 0 2

= Int (A). Then, there exists a
hyperplane H in Rn such that A H and f0g H+ .

Proposition 703 Let A and B be nonempty convex sets in Rn . If one of the following conditions holds,
then there exists a hyperplane H such that A H and B H+ :

1. 0 2
=A B;

2. 0 2
= Int (A B);

3. IntB 6= ? and 0 2
=A Int (B).

Proof. 1.
Since A and B are convex, we show that A B is convex. Let x; y 2 A B where x = a1 b1 , y = a2 b2 ,
and a1 ; a2 2 A; b1 ; b2 2 B. Let 2 [0; 1]: Then,

(1 )x + y = (1 )(a1 b1 ) + (a2 b2 ) = [(1 )a1 + a2 ] [(1 )b1 + b2 ] 2 A B

by convexity of A and B:
Hence, from our assumption and Lemma ??, there exists a hyperplane H = fx 2 Rn : c x = c0 g that
separates f0g and A B. Without loss of generality, A B H+ and c0 = 0. Then, 8a 2 A; b 2 B,

c (a b) 0,c a c b:

Hence, there must exists H 0 = fx0 2 Rn : c x0 = c1 g such that

c a c1 c b;

i.e., such that B H 0 and A H+ 0

.
2.
From our assumption and 702, there exists a hyperplane that separates f0g and A B, whence, following
the Proof of point 1 above, we are done.
3.
From our assumption and point 1 above, there exists a hyperplane H that separates A and Int(B). From
Remark 698, H separates Cl (A) and Cl (Int (B)) = Cl (B), where last equality follows from the Assumption
that Int(B) 6= ? and Proposition 691.7. Then since the closure of a set contains the set, the desired result
follows.

Proposition 704 Let A and B be subsets of Rn . Then

02
=A B , A \ B = ?:

Proof.
02
=A B ,

: (0 2 A B) ,

: (9a 2 A; 9b 2 B such that a b = 0) ,

8a 2 A; 8b 2 B; a 6= b ,

A \ B = ?:

Proposition 705 Let A and B be nonempty convex sets in Rn . If one of the following conditions holds
true, then there exists a hyperplane H such that A H and B H+ .
1. A \ B = ?;
2. IntB 6= ? and A\Int(B) = ?.
228 CHAPTER 16. CONVEX SETS

16.3 Farkas’Lemma
Proposition 706 If2
1. v1 ; :::; vm ; w 2 Rn , and
2. v1 x 0; :::; vm x 0 ) wx 0,
then
m
X
m
9 := ( i )i=1 2 Rm
+ such that w= i vi :
i=1

Proof. De…ne ( )
m
X m
C= y 2 Rn : i vi with ( i )i=1 2 Rm
+ :
i=1

Observe that since 0 2 Rm

+ , then
0 2 C; (16.1)
and
8 2 R+ and 8y 2 C, we have y 2 C: (16.2)
We want to show that w 2 C.
Claim 1. C is a convex, nonempty and closed set.
Proof of Claim 1.
The fact that C is a convex and nonempty is obvious. Let’s check closedness. Take (xk )k2N 2 C 1 such
m
that xk ! x. We want to show that x 2 C. Indeed, xk 2 C means that there exists ( ik )i=1 such that
m
X
xk = ik vi ! x: (16.3)
i=1

Then
m
X
8" > 0 9N" 2 N such that 8k > N" ; ik vi x < ":
i=1

Now, in general, we have that kxk = kx y + yk kx yk + kyk, i.e.,

kxk kyk kx yk :

Therefore,
m
X m
X
ik vi kxk ik vi x < "; (16.4)
i=1 i=1

and
m
X
0 ik vi < kxk + ": (16.5)
i=1
m
Then, as veri…ed below, (( ik )i=1 )k2N is a bounded sequence. Suppose otherwise; then, since by assump-
tion, for any i 2 f1; :::; mg and any k 2 N, ik 0, we have that there exists i 2 f1; :::; mg such that
ik ! +1. Then,
Xm Xm
lim ik vi = lim ik kvi k = +1;
k!+1 k!+1
i=1 i=1

violating (16:5).
Then, up to a subsequence,
m
k := ( ik )i=1 ! e 2 Rm
+: (16.6)
Then,
m
X m
X
i (16:3)
ik v ! ei v i = x;
i=1 i=1
2 In this Section, I follow very closely Section 8.1.2 in Montrucchio (1998).
16.3. FARKAS’LEMMA 229

and, then, from (16:6), x 2 C, as desired.

End of the Proof of Claim 1.
Suppose now our Claim is false, i.e., w 2= C. We can then apply Corollary 699 to conclude that there
exist a 2 Rn n f0g and 2 R such that

8x 2 C; aw > > ax: (16.7)

From (16:1), 0 2 C and then from (16:7), we have

8x 2 C; aw > > 0: (16.8)

Claim 2.
8x 2 C; ax 0:
Proof of Claim 2.
Suppose otherwise, i.e., 9x 2 C such that

ax > 0:

From (16:2), and the fact that x 2 C, we have that for any 0, x 2 C. Then, from (16:7), we get

( 0) (>0)
8 0; >a x = (ax ) > 0;

which is clearly false: if = 2 ax > 0, we get > 2 ax ax , i.e., 1 > 2.

End of the Proof of Claim 2.
Summarizing, we have shown that

9a 2 Rn n f0g such that 8x 2 C; ax 0 and aw > 0:

Since v1 ; v2 ; :::; vm 2 C, we have that

av1 0; :::; avm 0 and aw > 0;

contradicting the assumption.

Proposition 707 (A version of Farkas Lemma) Given A 2 M(m; n), b 2 Rn ,

either 1. 9x 2 Rn such that Ax 0 and bx > 0,

or 2. 9y 2 Rm such that yA = b and y 0;
but not both.

Proof. We want to show that either

1. 8
< Ax 0
(16.9)
:
bx > 0
has a solution, or
2. 8
< yA = b
(16.10)
:
y 0
has a solution, but not both.
Claim. It su¢ ces to show
a. (:1) ) 2;
b. 2 ) (:1) :
Proof of the Claim.
Indeed, a. , (:2) ) 1 and b. , 1 ) (:2) : Therefore, showing a. and b. implies showing

(:1) , 2, and (:2) , 1: (16.11)

230 CHAPTER 16. CONVEX SETS

Observe also that we have that 1 _ 2. Suppose otherwise, i.e., (:1) ^ (:2). But (:1) and (16:11) imply
2, a contradiction.
End of the proof of the Claim.
We are now left with showing a. and b.
a. Suppose that (16:9) has a no solution. Then Ax 0 implies that bx 0. Then from Proposition 706,
identifying vi with Ri (A), the i th row of A, and w with b, we have that
m
9 := ( i )i=1 2 Rm
+ such that b = A;

i.e., is a solution to (16:10).

b. By assumption, there exists y 2 Rm n
+ such that yA = b. Then, taken x 2 R , we also have yAx = bx.
i
Now if bx > 0, since y 0, we have that 9i 2 f1; :::; mg such that R (A) x > 0, and therefore (16:9) has no
solution.
Chapter 17

Concave functions

Consider1 a set X Rn , a set Rk and the functions f : X ! R; g : X ! Rm ; h : X !

Rl : The goal of this Chapter is to study the problem:
for given f; g; h and for given 2 ,

max f (x; ) s:t: g (x; ) 0 and h (x; ) = 0;

x2X

under suitable assumptions. The role of concavity (and di¤erentiability) of the functions f ,g and h is
crucial.
In what follows, unless needed, we omit the depends on .

17.1 Di¤erent Kinds of Concave Functions

Maintained Assumptions in this Chapter. Unless otherwise stated,

X is an open and convex subset of Rn .

f is a function such that

f : X ! R; : x 7! f (x) :

For each type of concavity we study, we present

1. the de…nition in the case in which f is C 0 (i.e., continuous),
2. an attempt of a “partial characterization” of that de…nition in the case in which f is C 1 and C 2 ; by
partial characterization, we mean a statement which is either su¢ cient or necessary for the concept presented
in the case of continuous f ;
3. the relationship between the di¤erent partial characterizations;
4. the relationship between the type of concavity and critical points and local or global extrema of f .
Finally, we study the relationship between di¤erent kinds of concavities.
The following pictures are taken from David Cass’s Microeconomics Course I followed at the University
of Pennsylvania (in 1985) and summarize points 1., 2. and 3. above.

1 This part is based on Cass (1991).

231
232 CHAPTER 17. CONCAVE FUNCTIONS

17.1.1 Concave Functions.

De…nition 708 Consider a C 0 function f . f is concave i¤ 8 x0 ; x00 2 X; 8 2 [0; 1],

f ((1 )x0 + x00 ) (1 )f (x0 ) + f (x00 ):

Proposition 709 Consider a C 0 function f .

f is concave
,
M = f(x; y) 2 X R : y f (x) g is convex.

Proof.
[)]
Take (x0 ; y 0 ) ; (x00 ; y 00 ) 2 M: We want to show that

8 2 [0; 1] ; ((1 ) x0 + x00 ; (1 ) y 0 + y 00 ) 2 M:

But, from the de…nition of M; we get that

(1 )y 0 + y 00 (1 )f (x0 ) + f (x00 ) f ((1 )x0 + x00 ):

[(]
From the de…nition of M; 8 x0 ; x00 2 X ; (x0 ; f (x0 )) 2 M and (x00 ; f (x00 )) 2 M:
Since M is convex,
((1 ) x0 + x00 ; (1 ) f (x0 ) + f (x00 )) 2 M
and from the de…nition of M;

(1 ) f (x0 ) + f (x00 ) f ( x0 + (1 ) x00 )

as desired.

Proposition 710 (Some properties of concave functions).

1. If f; g : X ! R are concave functions and a; b 2 R+ , then the function af + bg : X ! R; af + bg :
x 7! af (x) + bg (x) is a concave function.
2. If f : X ! R is a concave function and F : A ! R , with A Im f , is nondecreasing and concave,
then F f is a concave function.

Proof.
1. This result follows by a direct application of the de…nition.
2. Let x0 ; x00 2 X and 2 [0; 1] : Then

(1) (2)
(F f ) ((1 ) x0 + x00 ) F ((1 ) f (x0 ) + f (x00 )) (1 ) (F f ) (x0 ) + (F f ) (x00 ) ;

where (1) comes from the fact that f is concave and F is non decreasing, and
(2) comes from the fact that F is concave.

Remark 711 (from Sydsæter (1981)). With the notation of part 2 of the above Proposition, the assumption p
that F is concave cannot be dropped, as the following example shows. Take f; F : R++ ! R++ , f (x) = x
3
and F (y) = y 3 : Then f is concave and F is strictly increasing, but F f (x) = x 2 and its second derivative
1
is 43 x 2 > 0. Then, from Calculus I, we know that F f is strictly convex and therefore it is not concave.
Of course, the monotonicity assumption cannot be dispensed either. Consider f (x) = x2 and F (y) =
y: Then, (F f ) (x) = x2 , which is not concave.

Proposition 712 Consider a di¤ erentiable function f .

f is concave
,
8x0 ; x00 2 X; f (x00 ) f (x0 ) Df (x0 )(x00 x0 ):
17.1. DIFFERENT KINDS OF CONCAVE FUNCTIONS 233

Proof.
[)]
From the de…nition of concavity, we have that for 2 (0; 1) ;

(1 ) f (x0 ) + f (x00 ) f (x0 + (x00 x0 )) )

(f (x00 ) f (x0 )) f (x0 + (x00 x0 )) f (x0 ) )

f (x0 + (x00 x0 )) f (x0 )

f (x00 ) f (x0 ) :

Taking limits of both sides of the lasts inequality for ! 0; we get the desired result.
[(]
Consider x0 ; x00 2 X and 2 (0; 1). For 2 f0; 1g ; the desired result is clearly true. Since X is
convex; x := (1 ) x0 + x00 2 X. By assumption,

f (x00 ) f (x ) Df (x )(x00 x ) and

0
f (x0 ) f (x ) Df (x )(x x )
Multiplying the …rst expression by , the second one by (1 ) and summing up, we get

(f (x00 ) f (x )) + (1 )(f (x0 ) f (x )) Df (x )( (x00 x ) + (1 )(x0 x ))

Since
(x00 x ) + (1 ) (x0 x )=x x = 0;
we get
f (x00 ) + (1 ) f (x0 ) f (x );
i.e., the desired result.

De…nition 713 Given a symmetric matrix An n, A is negative semide…nite if 8x 2 Rn , xAx 0. A is

negative de…nite if 8x 2 Rn n f0g, xAx < 0.

Proposition 714 Consider a C 2 function f .

f is concave
,
8x 2 X; D2 f (x) is negative semide…nite.

Proof.
[)]
We want to show that 8u 2 Rn ; 8 x0 2 X, it is the case that uT D2 f (x0 )u 0: Since X is open, 8 x0 2 X
9 a 2 R++ such that jhj < a ) (x0 + hu) 2 X . Taken I := ( a; a) R, de…ne

g : I ! R; g : h 7! f (x0 + hu) f (x0 ) Df (x0 )hu:

Observe that
g 0 (h) = Dx f (x0 + hu) u + Df (x0 ) u
and
g 00 (h) = u D2 f (x0 + hu) u
Since f is a concave function, from Proposition 712, we have that 8 h 2 I; g(h) 0: Since g(0) = 0; h = 0
0
is a maximum point. Then, g (0) = 0 and

g 00 (0) 0 (1) :
0
Moreover, 8h 2 I; g (h) = Df (x0 + hu)u Df (x0 )u and g 00 (h) = uT D2 f (x0 + hu)u: Then,

g 00 (0) = u D2 f (x0 ) u (2) :

234 CHAPTER 17. CONCAVE FUNCTIONS

(1) and (2) give the desired result.

[(]
Consider x; x0 2 X: From Taylor’s Theorem (see Proposition 667), we get
1 T
f (x) = f (x0 ) + Df (x0 )(x x0 ) + x x0 D2 f (x)(x x0 )
2
T
where x = x0 + (x x0 ); for some 2 (0; 1) : Since, by assumption, x x0 D2 f (x)(x x0 ) 0; we have
that
f (x) f (x0 ) Df (x0 )(x x0 );
the desired result.

Some Properties.
Proposition 715 Consider a concave function f . If x0 is a local maximum point, then it is a global
maximum point.

Proof.
By de…nition of local maximum point, we know that 9 > 0 such that 8x 2 B (x0 ; ) ; f (x0 ) f (x) :
Take y 2 X; we want to show that f x0 f (y) :
Since X is convex,
8 2 [0; 1] ; (1 ) x0 + y 2 X:
Take 0 > 0 and su¢ ciently small to have 1 0
x0 + 0
y 2 B(x0 ; ): To …nd such 0 ; just solve the in-
0
equality 1 x0 + 0 y x0 = 0 y x0 = 0 y x0 < ; where, without loss of generality,
0
y 6= x .
Then,
f concave
0 0 0 0
f x0 f 1 x0 + y 1 f (x0 ) + f (y);
0 0 0
or f (x0 ) f (y): Dividing both sides of the inequality by > 0; we get f (x0 ) f (y):

Proposition 716 Consider a di¤ erentiable and concave function f . If Df (x0 ) = 0, then x0 is a global
maximum point.

Proof.
From Proposition 712, if Df (x0 ) = 0; we get that 8 x 2 X; f x0 f (x), the desired result.

17.1.2 Strictly Concave Functions.

De…nition 717 Consider a C 0 function f . f is strictly concave i¤ 8 x0 ; x00 2 X such that x0 6= x00 ; 8 2
(0; 1),
f ((1 )x0 + x00 ) > (1 )f (x0 ) + f (x00 ):

Proposition 718 Consider a C 1 function f .

f is strictly concave
, 8x0 ; x00 2 X such that x0 6= x00 ;

f (x00 ) f (x0 ) < Df (x0 )(x00 x0 ):

Proof.
[)]
Since strict concavity implies concavity, it is the case that

8x0 ; x00 2 X ; f (x00 ) f (x0 ) Df (x0 )(x00 x0 ): (17.1)

By contradiction, suppose f is not strictly concave. Then, from 17.1, we have that
17.1. DIFFERENT KINDS OF CONCAVE FUNCTIONS 235

9 x0 ; x00 2 X; x0 6= x00 such that f (x00 ) = f (x0 ) + Df (x0 )(x00 x0 ): (17.2)

From the de…nition of strict concavity and 17.2, for 2 (0; 1) ;

f ((1 )x0 + x00 ) > (1 )f (x0 ) + f (x0 ) + Df (x0 )(x00 x0 )

or

f ((1 )x0 + x00 ) > f (x0 ) + Df (x0 )(x00 x0 ): (17.3)

Applying 17.1 to the points x ( ) := (1 )x0 + x00 and x0 ; we get that for 2 (0; 1) ;

f ((1 )x0 + x00 ) f (x0 ) + Df (x0 )((1 )x0 + x00 x0 )

or

f ((1 )x0 + x00 ) f (x0 ) + Df (x0 )(x00 x0 ): (17.4)

And 17.4 contradicts 17.3.
[(] The proof is very similar to that one in Proposition 709.

Proposition 719 Consider a C 2 function f . If

8x 2 X; D2 f (x) is negative de…nite,

then f is strictly concave.

Proof.
The proof is similar to that of Proposition 714.

Remark 720 In the above Proposition, the opposite implication does not hold. The standard counterexample
is f : R ! R; f : x 7! x4 :

Some Properties.
Proposition 721 Consider a strictly concave, C 0 function f: If x0 is a local maximum point, then it is a
strict global maximum point, i.e., the unique global maximum point.

Proof.
First, we show that a. it is a global maximum point, and then b. the desired result.
a. It follows from the fact that strict concavity is stronger than concavity and from Proposition 715.
b. Suppose otherwise, i.e., 9x0 ; x0 2 X such that x0 6= x0 and both of them are global maximum points.
Then, 8 2 (0; 1) ; (1 ) x0 + x0 2 X; since X is convex, and

f (1 ) x0 + x0 > (1 ) f (x0 ) + f x0 = f (x0 ) = f x0 ;

a contradiction.

Proposition 722 Consider a strictly concave, di¤ erentiable function f . If Df x0 = 0; then x0 is a strict
global maximum point.

Proof.
Take an arbitrary x 2 X such that x 6= x0 : Then from Proposition 718, we have that f (x) < f (x0 ) +
Df (x0 )(x x0 ) = f x0 ; the desired result.
236 CHAPTER 17. CONCAVE FUNCTIONS

17.1.3 Quasi-Concave Functions.

De…nitions.
De…nition 723 Consider a C 0 function f . f is quasi-concave i¤ 8x0 ; x00 2 X; 8 2 [0; 1],

f ((1 )x0 + x00 ) min ff (x0 ) ; f (x00 )g :

Proposition 724 If f : X ! R is a quasi-concave function and F : R ! R is non decreasing, then F f

is a quasi-concave function.

Proof.
Without loss of generality, assume

f (x00 ) f (x0 ) (1) :

Then, since f is quasi-concave, we have

f ((1 )x0 + x00 ) f (x0 ) (2) :

Then,
(a) (b)
F (f ((1 ) x0 + x00 )) F (f (x0 )) = min fF (f (x0 )) ; F (f (x00 ))g ;
where (a) comes from (2) and the fact that F is nondecreasing, and
(b) comes from (1) and the fact that F is nondecreasing.

Proposition 725 Consider a C 0 function f . f is quasi-concave ,

8 2 R; B ( ) := fx 2 X : f (x) g is convex.

Proof.
[)] [Strategy: write what you want to show].
We want to show that 8 2 R and 8 2 [0; 1], we have that

hx0 ; x00 2 B (a)i ) h(1 ) x0 + x00 2 B ( )i ;

i.e.,

hf (x0 ) and f (x00 ) i ) hf ((1 ) x0 + x00 ) i:

But by Assumption,
def x0 ;x00
f ((1 )x0 + x00 ) min ff (x0 ) ; f (x00 )g :

[(]
Consider arbitrary x0 ; x00 2 X. De…ne := min ff (x0 ) ; f (x00 )g. Then x0 ; x00 2 B ( ) : By assumption,
8 2 [0; 1], (1 ) x0 + x00 2 B ( ) ; i.e.,

f ((1 )x0 + x00 ) := min ff (x0 ) ; f (x00 )g :

Proposition 726 Consider a di¤ erentiable function f . f is quasi-concave , 8x0 ; x00 2 X;

f (x00 ) f (x0 ) 0 ) Df (x0 )(x00 x0 ) 0:

Proof.
[)] [Strategy: Use the de…nition of directional derivative.]
Take x0 ; x00 such that f (x00 ) f (x0 ) : By assumption,

f ((1 ) x0 + x00 ) min ff (x0 ) ; f (x00 )g = f (x0 )

17.1. DIFFERENT KINDS OF CONCAVE FUNCTIONS 237

and
f ((1 ) x0 + x00 ) f (x0 ) 0:
Dividing both sides of the above inequality by > 0, and taking limits for ! 0+ ; we get

f (x0 + (x00 x0 )) f (x0 )

lim+ = Df (x0 )(x00 x0 ) 0:
!0

[(]
Without loss of generality, take

f (x0 ) = min ff (x0 ) ; f (x00 )g (1) :

De…ne
' : [0; 1] ! R; ' : 7! f ((1 ) x0 + x00 ) :
We want to show that
8 2 [0; 1] ; ' ( ) ' (0) :
Suppose otherwise, i.e., 9 2 [0; 1] such that ' ( ) < ' (0). Observe that in fact it cannot be
2 f0; 1g: if = 0, we would have ' (0) < ' (0), and if = 1, we would have ' (1) < ' (0), i.e.,
f (x00 ) < f (x0 ), contradicting (1). Then, we have that

9 2 (0; 1) such that ' ( ) < ' (0) (2) :

Observe that from (1) ; we also have that

' (1) ' (0) (3) :

Therefore, see Lemma 727, 9 > such that

'0 ( )>0 (4) ; and

'( ) < ' (0) (5) :

From (4) ; and using the de…nition of '0 , and the Chain Rule,2 we get

0 < '0 ( ) = [Df ((1 ) x0 + x00 )] (x00 x0 ) (6) :

De…ne x := (1 ) x0 + x00 : From (5) ; and the assumption, we get that

f (x ) < f (x0 ) :
Therefore, by assumption,

0 Df (x ) (x0 x ) = Df (x ) ( ) (x00 x0 ) ;

i.e.,

T
[Df (x )] (x00 x0 ) 0 (7) :
But (7) contradicts (6) :

Lemma 727 Consider a function g : [a; b] ! R with the following properties:

1. g is di¤ erentiable on (a; b) ;
2. there exists c 2 (a; b) such that g (b) g (a) > g (c) :
Then, 9 t 2 (c; b) such that g 0 (t) > 0 and g (t) < g (a) :
2 De…ned v : [0; 1] ! X Rn ; 7! (1 ) x0 + x00 , we have that ' = f v. Therefore, '0 ( ) = Df (v ( )) Dv ( ).
238 CHAPTER 17. CONCAVE FUNCTIONS

Proof.
Without loss of generality and to simplify notation, assume g (a) = 0: De…ne A := fx 2 [c; b] : g (x) = 0g :
Observe that A = [c; b] \ g 1 (0) is closed; and it is non empty, because g is continuous and by assumption
g (c) < 0 and g (b) 0:
Therefore, A is compact, and we can de…ne := min A:
Claim. x 2 [c; ) ) g (x) < 0:
Suppose not, i.e., 9y 2 (c; ) such that g (y) 0: If g (y) = 0; could not be min A: If g (y) > 0; since
g (c) < 0 and g is continuous, there exists x0 2 (c; y) (c; ) ; again contradicting the de…nition of : End of
the proof of the Claim.
Finally, applying Lagrange Theorem to g on [c; ], we have that 9t 2 (c; ) such that g 0 (t) = g( ) g(c)c .
Since g ( ) = 0 and g (c) < 0, we have that g 0 (t) < 0: From the above Claim, the desired result then follows.

Proposition 728 Consider a C 2 function f . If f is quasi-concave then

8x 2 X; 8 2 Rn such that Df (x) = 0; T

D2 f (x) 0:

Proof.
for another proof- see Laura Carosi’ s file
Suppose otherwise, i.e., 9 x 2 X; and 9 2 Rn such that Df (x) = 0 and T D2 f x0 > 0:
T 2
Since the function h : X ! R, h : x 7! D f (x) is continuous and X is open, 8 2 [0; 1] ; 9" > 0 such
that if k x x0 k< " ; then

D2 f x + (1 ) x0 >0 (1) :
De…ne x := x0 + k k; with 0 < < " : Then,

kx x0 k = k k= <"
k k

and x satis…es (1). Observe that

k k
= (x x0 ):

Then, we can rewrite (1) as

T
x x0 D2 f x + (1 ) x0 x x0 > 0

From Taylor Theorem, 9 2 (0; 1) such that

1
f (x) = f (x0 ) + (x x0 )T Df x0 + (x x0 )T D2 f ( x + (1 )x0 )(x x0 ):
2
Since Df (x0 ) = 0 and from (1) ; we have

f (x) > f (x0 ) (2):

e = x0 + (
Letting x =k k); using the same procedure as above, we can conclude that

x) > f (x0 ) (3):

f (e
But, since x0 = 12 (x + x
e); (2) and (3) contradict the De…nition of quasi-concavity.

Remark 729 In the above Proposition, the opposite implication does not hold. Consider f : R ! R; f :
x 7! x4 :From Proposition 725,
p thatp function is clearly not quasi-concave. Take > 0. Then B ( ) =
x 2 R : x4 = ( 1; ) [ ( ; +1) which is not convex.
On the other hand observe the following. f 0 (x) = 4x3 and 4x3 = 0 if either x = 0 or = 0: In both
cases 12x2 = 0: (This is example is taken from Avriel M. and others (1988), page 91).
17.1. DIFFERENT KINDS OF CONCAVE FUNCTIONS 239

Some Properties.
Remark 730 Consider a quasi concave function f . It is NOT the case that
if x0 is a local maximum point , then it is a global maximum point. To see that, consider the following
function.
x2 + 1 if x<1
f : R ! R; f : x 7!
0 if x 1

Proposition 731 Consider a C 0 quasi-concave function f . If x0 is a strict local maximum point, then it is
a strict global maximum point.

Proof.
By assumption, 9 > 0 such that if x 2 B (x0 ; ) \ X and x0 6= x; then f (x0 ) > f (x) :
Suppose the conclusion of the Proposition is false; then 9x0 2 X such that f (x0 ) f (x0 ) :
Since f is quasi-concave,

8 2 [0; 1] ; f ((1 ) x0 + x0 ) f (x0 ) : (1)

For su¢ ciently small ; (1 ) x0 + x0 2 B (x0 ; ) and (1) above holds, contradicting the fact that x0
is the strict local maximum point.

Proposition 732 Consider f : (a; b) ! R: f monotone ) f quasi-concave.

Proof.
Without loss of generality, take x00 x0 .
Case 1. f is increasing. Then f (x00 ) f (x0 ) : If 2 [0; 1], then (1 ) x0 + x00 = x0 + (x00 x0 ) x0
0 00 0
and therefore f ((1 ) x + x ) f (x ).
Case 2. f is decreasing. Then f (x00 ) f (x0 ) : If 2 [0; 1], then (1 ) x0 + x00 = (1 ) x0 (1 ) x00 +
00 00 00 0 00 0 00 00
x =x (1 ) (x x ) x and therefore f ((1 ) x + x ) f (x ) :

Remark 733 The following statement is false: If f1 and f2 are quasi-concave and a; b 2 R+ , then af1 + bf2
is quasi-concave.
It is enough to consider f1 ; f2 : R ! R, f1 (x) = x3 + x, and f2 (x) = 4x. Since f10 > 0, then f1 and,
of course, f2 are monotone and then, from Proposition 732, they are quasi-concave. On the other hand,
g (x) = f1 (x) + f2 (x) = x3 x has a strict local maximum in x = 1which is not a strict global maximum,
and therefore, from Proposition 731, g is not quasi-concave.
x3 3x

y 20

10

0
-2.5 -1.25 0 1.25 2.5

-10

-20
240 CHAPTER 17. CONCAVE FUNCTIONS

Remark 734 Consider a di¤ erentiable quasi-concave function f . It is NOT the case that
if Df (x0 ) = 0, then x0 is a global maximum point.
Just consider f : R ! R; f : x 7! x3 and x0 = 0; and use Proposition 732.

17.1.4 Strictly Quasi-concave Functions.

De…nitions.
De…nition 735 Consider a C 0 function f . f is strictly quasi-concave
i¤ 8 x0 ; x00 2 X; such that x0 6= x00 ; and 8 2 (0; 1), we have that

f ((1 )x0 + x00 ) > min ff (x0 ) ; f (x00 )g :

Proposition 736 Consider a C 0 function f . f is strictly quasi-concave ) 8 2 R; B ( ) := fx 2 X : f (x) g

is strictly convex.

Proof.
Taken an arbitrary and x0 ; x00 2 B( ), with x0 6= x00 , we want to show that 8 2 (0; 1), we have that

x := (1 ) x0 + x00 2 Int B ( )
Since f is strictly quasi-concave,

f x > min ff (x0 ) ; f (x00 )g

Since f is C 0 , there exists > 0 such that 8x 2 B x ;

f (x) >

i.e., B x ; B ( ), as desired. Of course, we are using the fact that fx 2 X : f (x) > g B ( ).

Remark 737 Observe that in Proposition 736, the opposite implication does not hold true: just consider
f : R ! R; f : x 7! 1.
Observe that 8 1, B ( ) = R, and 8 > 1, B ( ) = ?: On the other hand, f is not strictly quasi-
concave.

De…nition 738 Consider a di¤ erentiable function f . f is di¤ erentiable-strictly-quasi-concave i¤

8 x0 ; x00 2 X; such that x0 6= x00 ; we have that

f (x00 ) f (x0 ) 0 ) Df (x0 )(x00 x0 ) > 0:

Proposition 739 Consider a di¤ erentiable function f .

If f is di¤ erentiable-strictly-quasi-concave, then f is strictly quasi-concave.

Proof.
The proof is analogous to the case of quasi concave functions.

Remark 740 Given a di¤ erentiable function, it is not the case that strict-quasi-concavity implies di¤ erentiable-
strict-quasi-concavity.
f : R ! R; f : x 7! x3 a. is di¤ erentiable and strictly quasi concave and b. it is not di¤ erentiable-
strictly-quasi-concave.
a. f is strictly increasing and therefore strictly quasi concave - see Fact below.
b. Take x0 = 0 and x00 = 1: Then f (1) = 1 > 0 = f (0) : But Df (x0 ) (x00 x0 ) = 0 1 = 0 0:

Remark 741 If we restrict the class of di¤ erentiable functions to whose with non-zero gradients everywhere
in the domain, then di¤ erentiable-strict-quasi-concavity and strict-quasi-concavity are equivalent (see Balasko
(1988), Math. 7.2.).

Fact. Consider f : (a; b) ! R. f strictly monotone ) f strictly quasi concave.

17.1. DIFFERENT KINDS OF CONCAVE FUNCTIONS 241

Proof.
By assumption, x0 6= x00 , say x0 < x00 implies that f (x0 ) < f (x00 ) (or f (x0 ) > f (x00 )). If 2 (0; 1), then
(1 ) x0 + x00 > x0 and therefore f ((1 ) x0 + x00 ) > min ff (x0 ) ; f (x00 )g :

Proposition 742 Consider a C 2 function f . If

8x 2 X; 8 2 Rn n f0g ; we have that Df (x) =0) T

D2 f (x) <0 ;

then f is di¤ erentiable-strictly-quasi-concave.

Proof.

Suppose otherwise, i.e., there exist x0 ; x00 2 X such that

x0 6= x00 ; f (x00 ) f (x0 ) and Df (x0 ) (x00 x0 ) 0:

Since X is an open set, 9a 2 R++ such the following function is well de…ned:

g : [ a; 1] ! R; g : h 7! f ((1 h) x0 + hx00 ) :

Since g is continuous, there exists hm 2 [0; 1] which is a global minimum. We now proceed as follows.
Step 1. hm 2= f0; 1g : Step 2. hm is a strict local maximum point, a contradiction.
Preliminary observe that

g 0 (h) = Df (x0 + h (x00 x0 )) (x00 x0 )

and
T
g 00 (h) = (x00 x0 ) D2 f (x0 + h (x00 x0 )) (x00 x0 ) :
Step 1. If Df (x0 ) (x00 x0 ) = 0; then, by assumption,
T
(x00 x0 ) D2 f (x0 + h (x00 x0 )) (x00 x0 ) < 0:

Therefore, zero is a strict local maximum (see, for example, Theorem 13.10, page 378, in Apostol (1974) ).
Therefore, there exists h 2 R such that g (h ) = f (x0 + h (x00 x0 )) < f (x0 ) = g (0) :
If
g 0 (0) = Df (x0 ) (x00 x0 ) < 0;
then there exists h 2 R such that

g (h ) = f (x0 + h (x00 x0 )) < f (x0 ) = g (0) :

Moreover, g (1) = f (x00 ) f (x0 ) : In conclusion, neither zero nor one can be global minimum points for
g on [0; 1] :
Step 2. Since the global minimum point hm 2 (0; 1) ; we have that

0 = g 0 (hm ) = Df (x0 + hm (x00 x0 )) (x00 x0 ) :

Then, by assumption,
T
g 00 (0) = (x00 x0 ) D2 f (x0 + hm (x00 x0 )) (x00 x0 ) < 0;
but then hm is a strict local maximum point, a contradiction.

Remark 743 Di¤ erentiable-strict-quasi-concavity does not imply the condition presented in Proposition 742.
f : R ! R; f : x 7! x4 is di¤ erentiable-strictly-quasi-concave (in next section we will show that strict-
concavity implies di¤ erentiable-strict-quasi-concavity). On the other hand, take x = 0: Then Df (x ) = 0:
Therefore, for any 2 Rn n f0g ; we have Df (x ) = 0; but T D2 f (x ) = 0 0:
242 CHAPTER 17. CONCAVE FUNCTIONS

Some Properties.
Proposition 744 Consider a di¤ erentiable-strictly-quasi-concave function f .
x is a strict global maximum point , Df (x ) = 0:
Proof.
[)] Obvious.
[(] From the contropositive of the de…nition of di¤erentiable-strictly-quasi-concave function, we have:
8 x ; x00 2 X; such that x 6= x00 ; it is the case that Df (x )(x00 x ) 0 ) f (x00 ) f (x ) < 0 or
00
f (x ) > f (x ) : Since Df (x ) = 0; then the desired result follows.

Remark 745 Obviously, we also have that if f is di¤ erentaible-strictly-quasi-concave, it is the case that:
x local maximum point ) x is a strict maximum point.
Remark 746 The above implication is true also for continuous strictly quasi concave functions. (Sup-
pose otherwise, i.e., 9 x0 2 X such that f (x0 ) f (x ). Since f is strictly quasi-concave, 8 2 (0; 1),
f ((1 ) x + x0 ) > f (x ), which for su¢ ciently small contradicts the fact that x is a local maximum
point.
Is there a de…nition of ? concavity weaker than concavity and such that:
If f is a ? concave function, then
x is a global maximum point i¤ Df (x ) = 0:
The answer is given in the next section.

17.1.5 Pseudo-concave Functions.

De…nition 747 Consider a di¤ erentiable function f . f is pseudo-concave i¤

8x0 ; x00 2 X; f (x00 ) > f (x0 ) ) Df (x0 ) (x00 x0 ) > 0;

or

8x0 ; x00 2 X; Df (x0 ) (x00 x0 ) 0 ) f (x00 ) f (x0 ) :

Proposition 748 If f is a pseudo-concave function, then
x is a global maximum point , Df (x ) = 0:
Proof.
[)] Obvious.
[(] Df (x ) = 0 ) 8x 2 X; Df (x ) (x x ) 0 ) f (x) f (x ) :

Remark 749 Observe that the following “de…nition of pseudo-concavity” will not be useful:
8x0 ; x00 2 X; Df (x0 ) (x00 x0 ) 0 ) f (x00 ) f (x0 ) (17.5)
For such a de…nition the above Proposition would still apply, but it is not weaker than concavity. Simply
consider the function f : R ! R; f : x 7! x2 . That function is concave, but it does not satisfy condition
(17.5). Take x0 = 2 and x00 = 1. Then, f 0 (x0 ) (x00 x0 ) = 4 ( 1 ( 2)) = 4 > 0, but f (x00 ) = 1 >
f (x0 ) = 4.
We summarize some of the results of this subsection in the following tables.
Class of function Fundamental properties
Uniqueness
C ) G max L max ) G max
of G. max
Strictly concave Yes Yes Yes
Concave Yes Yes No
Di¤.ble-str.-q.-conc. Yes Yes Yes
Pseudoconcave Yes Yes No
Quasiconcave No No No
17.2. RELATIONSHIPS AMONG DIFFERENT KINDS OF CONCAVITY 243

where C stands for property of being a critical point, and L and G stand for local and global, respectively.
Observe that the …rst, the second and the last row of the second column apply to the case of C 0 and not
necessarily di¤erentiable functions.

17.2 Relationships among Di¤erent Kinds of Concavity

The relationships among di¤erent de…nitions of concavity in the case of di¤ erentiable functions are summa-
rized in the following table.

strict concavity
+ &
linearity ) a¢ nity ) concavity
+
pseudo-concavity ( di¤erentiable-strict-quasi-concavity
+
quasi-concavity

All the implications which are not implied by those explicitly written do not hold true.
In what follows, we prove the truth of each implication described in the table and we explain why the
other implications do no hold.
Recall that
1. f : Rn ! Rm is a linear function i¤ 8x0 ; x00 2 Rn ; 8a; b 2 R f (ax0 + bx00 ) = af (x0 ) + bf (x00 );
2. g : Rn ! Rm is an a¢ ne function i¤ there exists a linear function f : Rn ! Rm and c 2 Rm such that
8x 2 Rn , g (x) = f (x) + c.

SC ) C
Obvious (“a > b ) a b”).

C ) PC
From the assumption and from Proposition 712, we have that f (x00 ) f (x0 ) Df (x0 ) (x00 x0 ) : Then
f (x00 ) f (x0 ) > 0 ) Df (x0 ) (x00 x0 ) > 0:

P C ) QC
Suppose otherwise, i.e., 9 x0 ; x00 2 X and 9 [0; 1] such that

f ((1 ) x0 + x00 ) < min ff (x0 ) ; f (x00 )g :

De…ne x ( ) := (1 ) x0 + x00 : Consider the segment L (x0 ; x00 ) joining x0 to x00 : Take 2 arg min f (x ( )) s:t: 2
[0; 1] : is well de…ned from the Extreme Value Theorem. Observe that 6= 0; 1; because f (x ( )) <
min ff (x (0)) = f (x0 ) ; f (x (1)) = f (x00 )g :
Therefore, 8 2 [0; 1] and 8 2 (0; 1) ;
f x f (1 )x + x( ) :

(1 )x + x( )
#

" " " "

x0 x( ) x x00
Then,

f (1 )x + x( ) f x
8 2 [0; 1] ; 0 lim+ = Df x x( ) x :
!0
244 CHAPTER 17. CONCAVE FUNCTIONS

Taking = 0; 1 in the above expression, we get:

Df x x0 x 0 (1)
and

Df x x00 x 0 (2) :
Since

x0 x = x0 1 x0 x00 = (x00 x0 ) (3) ;

and

x00 x = x00 1 x0 x00 = 1 (x00 x0 ) (4) ;

substituting (3) in (1) ; and (4) in (2) ; we get
( )
Df x (x00 x0 ) 0;
and
(+)
1 Df x (x00 x0 ) 0:
Therefore,
(4)
0 = Df x (x00 x0 ) = Df x 1 (x00 x0 ) =

= Df x x00 x :
Then, by pseudo-concavity,
f (x00 ) f x (5) :
By assumption,
f (x ( )) < f (x00 ) (6) :
(5) and (6) contradict the de…nition of :

DSQC ) P C
Obvious.

SC ) DSQC
Obvious.

L)C
Obvious.

C ; SC
f : R ! R; f : x 7! x:

QC ; P C
0 if x 0
f : R ! R; f : x 7! 1
e x2 if x>0
17.2. RELATIONSHIPS AMONG DIFFERENT KINDS OF CONCAVITY 245

f is clearly nondecreasing and therefore, from Lemma 732, quasi-concave.

f is not pseudo-concave: 0 < f ( 1) > f (1) = 0; but
f 0 (1) ( 1 1) = 0 ( 2) = 0:

P C ; C , DSQC ; C and DSQC ; SC

Take f : (1; +1) ! R; f : x 7! x3 :
(>0) (>0)
Take x0 < x00 : Then f (x00 ) > f (x0 ) : Moreover, Df (x0 )(x00 x0 ) > 0: Therefore, f is DSQC and therefore
P C. Since f 00 (x) > 0; f is strictly convex and therefore it is not concave and, a fortiori, it is not strictly
concave.

P C ; DSQC , C ; DSQC
Consider f : R ! R; f : x 7! 1. f is clearly concave and P C, as well ( 8x0 ; x00 2 R, Df (x0 ) (x00 x0 ) 0).
Moreover, any point in R is a critical point, but it is not the unique global maximum point. Therefore, from
Proposition 744, f is not di¤erentiable - strictly - quasi - concave.

QC ; DSQC
If so, we would have QC ) DSQC ) P C, contradicting the fact that QC ; P C:

C ; L and SC ; L
f : R ! R; f : x 7! x2 :

17.2.1 Hessians and Concavity.

In this subsection, we study the relation between submatrices of a matrix involving the Hessian matrix of a
C 2 function and the concavity of that function.

De…nition 750 Consider a matrix An n : Let 1 k n:

A k th order principal submatrix (minor) of A is the (determinant of the) square submatrix of A
obtained deleting (n k) rows and (n k) columns in the same position. Denote these matrices by D ei :
k
The k th order leading principal submatrix (minor) of A is the (determinant of the) square submatrix
of A obtained deleting the last (n k) rows and the last (n k) columns. Denote these matrices by Dk :

Example 751 Consider 2 3

a11 a12 a13
A = 4 a21 a22 a23 5 :
a31 a32 a33
Then

e 11 = a11 ; D
D e 12 = a22 ; D
e 13 = a33 ; e 11 = a11 ;
D1 = D

e 21 = a11 a12 e 22 = a11 a13 e 23 = a22 a23

D ; D ; D ;
a21 a22 a31 a33 a32 a33

e 21 = a11 a12
D2 = D ;
a21 a22

e 31 = A:
D3 = D

De…nition 752 Consider a C 2 function f : X Rn ! R. The bordered Hessian of f is the following

matrix

0 Df (x)
Bf (x) = T :
[Df (x)] D2 f (x) (n+1) (n+1)
246 CHAPTER 17. CONCAVE FUNCTIONS

Theorem 753 (Simon, (1985), Theorem 1.9.c, page 79 and Sydsaeter (1981), Theorem 5.17, page 259).
Consider a C 2 function f : X ! R.
1. If 8x 2 X; 8k 2 f1; :::; ng ;
k
sign k leading principal minor of D2 f (x) = sign ( 1) ;
then f is strictly concave.
2. 8x 2 X; 8k 2 f1; :::; ng ;
k
sign non zero k principal minor of D2 f (x) = sign ( 1) ;
i¤ f is concave.
3. If n 2 and 8x 2 X; 8k 2 f3; :::; n + 1g ;
k 1
sign (k leading principal minor of Bf (x)) = sign ( 1) ;
then f is pseudo concave and, therefore, quasi-concave.
4. If f is quasi-concave, then 8x 2 X; 8k 2 f2; :::; n + 1g ;
k 1
sign (non zero k leading principal minors of Bf (x)) = sign ( 1)

Remark 754 It can be proved that Conditions in part 1 and 2 of the above Theorem are su¢ cient for
D2 f (x) being negative de…nite and equivalent to D2 f (x) being negative semide…nite, respectively.

Remark 755 (From Sydsaetter (1981), page 239) It is tempting to conjecture that a function f is concave
i¤
k
8x 2 X; 8k 2 f1; :::; ng ; sign non zero k leading principal minor of D2 f (x) = sign ( 1) ; (17.6)

That conjecture is false. Consider

f : R3 ! R; f : (x1 ; x2 ; x3 ) 7! x22 + x23 :

Then Df (x) = (0; 2x2 ; 2x3 ) and 2 3

0 0 0
D2 f (x) = 4 0 2 0 5
0 0 2
All the leading principal minors of the above matrix are zero, and therefore Condition 17.6 is satis…ed, but
f is not a concave function. Take x0 = (0; 0; 0) and x00 (0; 0; 1). Then
2
8 2 (0; 1) ; f ((1 ) x0 + x00 ) = < (1 ) f (x0 ) + f (x00 ) =

Example 756 Consider f : R2++ ! R; f : (x; y) 7! x y ; with ; 2 R++ . Observe that 8 (x; y) 2
R2++ ; f (x; y) > 0. Verify that

1. if + < 1, then f is strictly concave;

2. 8 ; 2 R++ , f is quasi-concave;
3. + 1 if and only if f is concave.

1.
Dx f (x; y) = x 1 y = x f (x; y) ;
Dy f (x; y) = x y 1 = y f (x; y) ;
2 ( 1)
Dx;x f (x; y) = ( 1) x 2 y = x2 f (x; y) ;
2 ( 1)
Dy;y f (x; y) = ( 1) x y 2 = y2 f (x; y) ;
2 1 1
Dx;:y f (x; y) = x y = x yf (x; y) :
" ( 1)
#
x2 x y
D2 f (x; y) = f (x; y) ( 1) :
x y y2
( 1)
a. x2 <0, 2 (0; 1).
2 2
( 1) ( 1) 1 2 2
b. x2 y 2 = x2 y 2 ( + 1) =
; >0
= x21y2 (1 )>0 , + < 1:
In conclusion, if ; 2 (0; 1) and + < 1, then f is strictly concave.
17.2. RELATIONSHIPS AMONG DIFFERENT KINDS OF CONCAVITY 247

2.
Observe that
f (x; y) = g (h (x; y))
where
h : R2++ ! R; (x; y) 7! ln x + ln y
z
g : R ! R; z 7! e
Since h is strictly concave (why?) and therefore quasi-concave and g is strictly increasing, the desired
result follows from Proposition 724.
3.
Obvious from above results.
248 CHAPTER 17. CONCAVE FUNCTIONS
Chapter 18

Maximization Problems

Let the following objects be given:

1. an open convex set X Rn , n 2 N;

2. f : X ! R; g : X ! Rm , h : X ! Rk , m; k 2 N, with f; g; h at least di¤erentiable.

The goal of this Chapter is to study the problem.

maxx2X f (x)
s:t: g (x) 0 (1) (18.1)
h (x) = 0 (2)

f is called objective function; x choice variable vector; (1) and (2) in (18.1) constraints; g and h
constraint functions;

C := fx 2 X : g (x) 0 and h (x) = 0g

is the constraint set.
To solve the problem (18.1) means to describe the following set

fx 2 C : 8x 2 C; f (x ) f (x)g

which is called solution set to problem (18.1) and it is also denoted by arg max (18.1). We will proceed as
follows.
1. We will analyze in detail the problem with inequality constraints, i.e.,
maxx2X f (x)
s:t: g (x) 0 (1)
2. We will analyze in detail the problem with equality constraints, i.e.,
maxx2X f (x)
s:t: h (x) = 0 (2)
3. We will describe how to solve the problem with both equality and inequality constraints, i.e.,
maxx2X f (x)
s:t: g (x) 0 (1)
h (x) = 0 (2)

18.1 The case of inequality constraints: Kuhn-Tucker theorems

m
Consider the open and convex set X Rn and the di¤erentiable functions f : X ! R; g := g j j=1
:X!
Rm : The problem we want to study is

maxx2X f (x) s:t: g (x) 0: (18.2)

249
250 CHAPTER 18. MAXIMIZATION PROBLEMS

De…nition 757 The Kuhn-Tucker system (or conditions) associated with problem 18.2 is
8
>
> Df (x) + Dg (x) = 0 (1)
<
0 (2)
(18.3)
>
> g (x) 0 (3)
:
g (x) = 0 (4)

Equations (1) are called …rst order conditions; equations (2) ; (3) and (4) are called complementary slack-
ness conditions.

Remark 758 (x; ) 2 X Rm is a solution to Kuhn-Tucker system i¤ it is a solution to any of the following
systems:

1. 8 Pm
@f (x) @gj (x)
>
> + j=1 j = 0 f or i = 1; :::; n (1)
< @xi @xi
j 0 f or j = 1; :::; m (2)
>
> g (x) 0 f or j = 1; :::; m (3)
: j
j gj (x) = 0 f or j = 1; :::; m (4)

2.
Df (x) + Dg (x) = 0 (1)
min f j ; gj (x)g = 0 f or j = 1; :::; m (2)

Moreover, (x; ) 2 X Rm is a solution to Kuhn-Tucker system i¤ it is a solution to

Df (x) + Dg (x) = 0 (1)

and for each j = 1; :::; m, to one of the following conditions

either ( j > 0 and gj (x) = 0 )

or ( j =0 gj (x) > 0 )
or ( j =0 gj (x) = 0 )

De…nition 759 Given x 2 C; we say that j is a binding constraint at x if gj (x ) = 0: Let

J (x ) := fj 2 f1; :::; mg : gj (x ) = 0g ;

g := (gj )j2J (x ) ; gb := (gj )j 2J

= (x )

and
m = #J (x ) :

De…nition 760 x 2 Rn satis…es the constraint quali…cations associated with problem 18.2 if it is a solution
to
maxx2Rn Df (x ) x s:t: Dg (x ) (x x ) 0 (18.4)

The above problem is obtained from 18.2

1. replacing g with g ;

2. linearizing f and g around x , i.e., substituting f and g with f (x ) + Df (x ) (x x ) and g (x ) +

Dg (x ) (x x ), respectively;

3. dropping redundant terms, i.e., the term f (x ) in the objective function, and the term g (x ) = 0 in
the constraint.

Theorem 761 Suppose x is a solution to problem 18.2 and to problem 18.4, then there exists 2 Rm
such that (x ; ) satis…es Kuhn-Tucker conditions.

The proof of the above theorem requires the following lemma.

18.1. THE CASE OF INEQUALITY CONSTRAINTS: KUHN-TUCKER THEOREMS 251

Lemma 762 (Farkas) Given a matrix Am n and a vector a 2 Rn ,

either 1. there exists 2 Rm + such that a = A,
or 2. there exists y 2 Rn such that Ay 0 and ay < 0,
but not both.

Proof. It follows immediately from Proposition 707.

Proof. of Theorem 761
(main steps: 1. use the fact x is a solution to problem 18.4; 2. apply Farkas Lemma; 3. choose
= ( from Farkas ; 0)).
Since x is a solution to problem 18.4, for any x 2 Rn such that Dg (x ) (x x ) 0 it is the case that
Df (x ) x Df (x ) x or

Dg (x ) (x x ) 0 ) [ Df (x )] (x x ) 0: (18.5)

Applying Farkas Lemma identifying

a with Df (x )

and
A with Dg (x )
we have that either
1. there exists 2 Rm+ such that
Df (x ) = Dg (x ) (18.6)
n
or 2. there exists y 2 R such that

Dg (x ) y 0 and Df (x ) y < 0 (18.7)

but not both 1 and 2.

Choose x = y + x and therefore you have y = x x . Then, 18.7 contradicts 18.5. Therefore, 1. above
holds.
Now, choose := ( ; 0) 2 Rm Rm m , we have that

Dg (x )
Df (x ) + Dg (x ) = Df (x ) + ( ; 0) = Df (x ) + Dg (x ) = 0
Db
g (x )

where the last equality follows from 18.6;

0 by Farkas Lemma;
g (x ) 0 from the assumption that x solves problem 18.2;
g (x )
g (x ) = ( ; 0) = g (x) = 0, where the last equality follows from the de…nition of g .
gb (x )

Theorem 763 If x is a solution to problem (18:2) and

either for j = 1; ::; m, gj is pseudo-concave and 9x++ 2 X such that g (x++ ) 0,
or rank Dg (x ) = m := #J (x ),
then x solves problem (18:4) :

Proof. We prove the conclusion of the theorem under the …rst set of conditions.
e ... ; 2. use the two assumptions; 3. move from x in the direction
Main steps: 1. suppose otherwise: 9 x
x := (1 e + x++ .
)x
e 2 Rn such that
Suppose that the conclusion of the theorem is false. Then there exists x

Dg (x ) (e
x x ) 0 and Df (x ) (e
x x )>0 (18.8)

Moreover, from the de…nition of g and x++ , we have that

g x++ >> 0 = g (x )

Since for j = 1; ::; m, gj is pseudo-concave we have that

Dg (x ) x++ x 0 (18.9)
252 CHAPTER 18. MAXIMIZATION PROBLEMS

De…ne
x := (1 e + x++
)x
with 2 (0; 1). Observe that

x x = (1 e + x++
)x (1 )x x = (1 ) (e
x x )+ x++ x

Therefore,

Dg (x ) x x = (1 ) Dg (x ) (e
x x ) + Dg (x ) x++ x 0 (18.10)

where the last equality come from 18.8 and 18.9.

Moreover,

Df (x ) x x = (1 ) Df (x ) (e
x x ) + Df (x ) x++ x 0 (18.11)

where the last equality come from 18.8 and a choice of su¢ ciently small.1
Observe that from Remark 626, 18.10 and 18.11 we have that
0
(g ) x ; x 0

and
f0 x ; x >0
Therefore, using the fact that X is open, and that gb (x ) 0, there exists such that

x + x x 2X
g x + x x g (x ) = 0
(18.12)
f x + x x > f (x )
gb x + x x 0

But then 18.12 contradicts the fact that x solves problem (18:2).
From Theorems 761 and 763, we then get the following corollary.

Theorem 764 Suppose x is a solution to problem 18.2, and one of the following constraint quali…cations
hold:
a. for j = 1; :::; m; gj is pseudo-concave and there exists x++ 2 X such that g (x++ ) 0
b. rank Dg (x ) = #J ;
Then there exists 2 Rm such that (x ; ) solves the system 18.3.

Theorem 765 If f is pseudo-concave, and for j = 1; :::; m; gj is quasi-concave, and (x ; ) solves the
system 18.3, then x solves problem 18.2.

Proof. Main steps: 1. suppose otherwise and use the fact that f is pseudo-concave; 2. for j 2 J (x ),
use the quasi-concavity of gj ; 3. for j 2 J (x ), use (second part of) kuhn-Tucker conditions; 4. Observe
that 2. and 3. above contradict the …rst part of Kuhn-Tucker conditions.)
b 2 X such that
Suppose otherwise, i.e., there exists x

g (b
x) 0 and f (b
x) > f (x ) (18.13)

From 18.13 and the fact that f pseudo-concave, we get

Df (x ) (b
x x )>0 (18.14)
1 Assume that 2 (0; 1), 2 R++ and 2 R. We want to show that there exist 2 (0; 1) such that
(1 ) + >0
i.e.,

> ( )
If ( ) = 0, the claim is true.
If ( ) > 0, any < will work (observe that > 0).
If ( ) < 0, the claim is clearly true because 0 < and ( ) < 0.
18.1. THE CASE OF INEQUALITY CONSTRAINTS: KUHN-TUCKER THEOREMS 253

From 18.13, the fact that g (x ) = 0 and that gj is quasi-concave, we get that

for j 2 J (x ) ; Dg j (x ) (b
x x ) 0

and since 0,
j
for j 2 J (x ) ; j Dg (x ) (b
x x ) 0 (18.15)

For j 2 Jb (x ), from Kuhn-Tucker conditions, we have that gj (x ) > 0 and j = 0, and therefore

for j 2 Jb (x ) ; j Dg
j
(x ) (b
x x )=0 (18.16)
But then from 18.14, 18.15 and 18.16, we have

Df (x ) (b
x x )+ Dg (x ) (b
x x )>0

contradicting Kuhn-Tucker conditions.

We can summarize the above results as follows. Call (M ) the problem

maxx2X f (x) s:t: g (x) 0 (18.17)

and de…ne
M := arg max (M ) (18.18)

S := fx 2 X : 9 2 Rm such that (x; ) is a solution to Kuhn-Tucker system (18:3)g (18.19)

1. Assume that one of the following conditions hold:

(a) for j = 1; :::; m; gj is pseudo-concave and there exists x++ 2 X such that g (x++ ) 0
(b) rank Dg (x ) = #J .
Then
x 2M )x 2S

2. Assume that both the following conditions hold:

(a) f is pseudo-concave, and

(b) for j = 1; :::; m; gj is quasi-concave.
Then
x 2 S ) x 2 M:

18.1.1 On uniqueness of the solution

The following proposition is a useful tool to show uniqueness.

Proposition 766 The solution to problem

maxx2X f (x) s.t. g (x) 0 (P )

either does not exist or it is unique if one of the following conditions holds

1. f is strictly quasi-concave, and

for j 2 f1; :::; mg, gj is quasi-concave;

2. f is quasi-concave and locally non-satiated (i.e., 8x 2 X; 8" > 0; there exists x0 2 B (x; ") such that
f (x0 ) > f (x) ), and
for j 2 f1; :::; mg, gj is strictly quasi-concave.
254 CHAPTER 18. MAXIMIZATION PROBLEMS

Proof. 1.
Since gj is quasi concave V j := fx 2 X : gj (x) 0g is convex. Since the intersection of convex sets is
convex V = \m j
j=1 V is convex.
Suppose that both x0 and x00 are solutions to problem (P ) and x0 6= x00 . Then for any 2 (0; 1),

(1 ) x0 + x00 2 V (18.20)

because V is convex, and

f ((1 ) x0 + x00 ) > min ff (x0 ) ; f (x00 )g = f (x0 ) = f (x00 ) (18.21)

because f is strictly-quasi-concave.
But (18.20) and (18.21) contradict the fact that x0 and x00 are solutions to problem (P ).
2.
Observe that V is strictly convex because each V j is strictly convex. Suppose that both x0 and x00 are
solutions to problem (P ) and x0 6= x00 . Then for any 2 (0; 1),

x ( ) := (1 ) x0 + x00 2 Int V

i.e., 9" > 0 such that B (x ( ) ; ") V . Since f is locally non-satiated, there exists x0 2 B (x ( ) ; ") V
such that
f (b
x) > f (x ( )) (18.22)
Since f is quasi-concave,
f (x ( )) f (x0 ) = f (x00 ) (18.23)
(18.22) and (18.23) contradict the fact that x0 and x00 are solutions to problem (P ).

Remark 767 1. If f is strictly increasing (i.e., 8x0 ; x00 2 X such that x0 > x00 , we have that f (x0 ) > f (x00 )
) or strictly decreasing, then f is locally non-satiated.
2. If f is a¢ ne and not constant, then f is quasi-concave and Locally NonSatiated.
Proof of 2.
f : Rn ! R a¢ ne and not constant means that there exists a 2 R and b 2 Rn f0g such that f : x 7!
n
a + bT x. Take an arbitrary x and " > 0. For i 2 f1; :::; ng, de…ne i := k" (sign bi ) and x e := x + ( i )i=1 ,
with k 6= 0 and whichPwill be computed below. Then
n
x) = a + bx + i=1 k" jbi j > f (x);
f (e
qP
n n 2
x xk = k" ((sign bi ) bi )i=1 = k"
ke i=1 (bi )
1
= k" kbk < " if k > kbk :

Remark 768 In part 2 of the statement of the Proposition f has to be both quasi-concave and Locally
NonSatiated.
a. Example of f quasi-concave (and gj strictly-quasi-concave) with more than one solution:

max 1 s:t: x+1 0 1 x 0

x2R

The set of solution is [ 1; +1]

a. Example of f Locally NonSatiated (and gj strictly-quasi-concave) with more than one solution:

max x2 s:t: x+1 0 1 x 0

x2R

The set of solutions is f 1; +1g.

18.2 The Case of Equality Constraints: Lagrange Theorem.

Consider the C 1 functions

f : X ! R; f : x 7! f (x) ;

m
g : X ! Rm ; g : x 7! g (x) := (gj (x))j=1
18.2. THE CASE OF EQUALITY CONSTRAINTS: LAGRANGE THEOREM. 255

with m n: Consider also the following “”maximization problem:

(P ) maxx2X f (x) s:t: g (x) = 0 (18.24)

L:X Rm ! R; L : (x; ) 7! f (x) + g (x)

is called Lagrange function associated with problem (15.45).
We recall below the statement of Theorem 687.

Theorem 769 (Necessary Conditions)

Assume that rank [Dg (x )] = m:
Under the above condition, we have that
x is a local maximum for (P )
)
there exists 2 Rm ; such that
Df (x ) + Dg (x ) = 0
(18.25)
g (x ) = 0

Remark 770 The full rank condition in the above Theorem cannot be dispensed. The following example
shows a case in which x is a solution to maximization problem (18.24), Dg (x ) does not have full rank and
there exists no satisfying Condition 18.25. Consider

max(x;y)2R2 x s:t: x3 y = 0
x3 + y = 0

The constraint set is f(0; 0)g and therefore the solution is just (x ; y ) = (0; 0). The Jacobian matrix of
the constraint function is

3x2 1 0 1
=
3x2 1 j(x ;y )
0 1
which does have full rank.

(0; 0) = Df (x ; y ) + ( 1; 2 ) Dg (x ;y ) =
0 1
= (1; 0) + ( 1; 2) = (1; 1 + 2) ;
0 1
from which it follows that there exists no solving the above system.

Theorem 771 (Su¢ cient Conditions)

Assume that
1. f is pseudo-concave,
2. for j = 1; :::; m; gj is quasi concave.
Under the above conditions, we have what follows.
[there exist (x ; ) 2 X Rm such that
3. 0;
4. Df (x ) + Dg (x ) = 0; and
5. g (x ) = 0 ]
)
x solves (P ) :

Proof.
b 2 X such that
Suppose otherwise, i.e., there exists x

f or j = 1; :::; m; gj (b
x) = gj (x ) = 0 (1) ; and

f (b
x) > f (x ) (2) :
Quasi-concavity of gj and (1) imply that
256 CHAPTER 18. MAXIMIZATION PROBLEMS

Dg j (x ) (b
x x ) 0 (3) :
Pseudo concavity of f and (2) imply that

Df (x ) (b
x x )>0 (4) :
But then
(>0) ( 0) ( 0)
Assumption
0 = [Df (x ) + Dg (x )] (b
x x ) = Df (x ) (b
x x )+ Dg (x ) (b
x x ) > 0;
a contradiction.

18.3 The Case of Both Equality and Inequality Constraints.

Consider
the open and convex set X Rn and the di¤erentiable functions f : X ! R; g : X ! Rm ; h : X ! Rl :
Consider the problem

maxx2X f (x) s:t: g (x) 0

(18.26)
h (x) = 0:
Observe that

h (x) = 0 ,

, for k = 1; :::; l; hk (x) = 0 ,

, for k = 1; :::; l; hk1 (x) := hk (x) 0 and hk2 (x) := hk (x) 0:

l l
De…ned h:1 (x) := hk1 (x) k=1
and h:2 (x) := hk2 (x) k=1
; problem 18.26 with associated multipliers
can be rewritten as

maxx2X f (x) s:t: g (x) 0

h:1 (x) 0 1 (18.27)
h:2 (x) 0 2

The Lagrangian function of the above problem is

T T
L (x; ; 1; 2) = f (x) + g (x) + ( 1 2) h (x) =

m
X l
X
k k T
= f (x) + j gj (x) + 1 2 h (x) ;
j=1 k=1

and the Kuhn-Tucker Conditions are:

T
Df (x) + T Dg (x) + ( 1 2 ) Dh (x) = 0
gj (x) 0; j 0; j gj (x) = 0; for j = 1; :::; m; (18.28)
hk (x) = 0; k
1
k
2 := T 0; for k = 1; :::; l:

Theorem 772 Assume that f; g and h are C 2 functions and that

Dg (x )
either rank = m + l;
Dh (x )
or for j = 1; :::; m; gj is pseudoconcave, and 8k, hk and hk are pseudoconcave
Under the above conditions,
if x solves 18.26, then 9 (x ; ; ) 2 X Rm Rl which satis…es the associated Kuhn-Tucker conditions.
18.4. MAIN STEPS TO SOLVE A (NICE) MAXIMIZATION PROBLEM 257

Proof. The above conditions are called “Weak reverse convex constraint quali…cation” (Mangasarian
(1969)) or “Reverse constraint quali…cation” (Bazaraa and Shetty (1976)). The needed result is presented
and proved in
Mangasarian2 ,- see 4, page 172 and Theorem 6, page 173, and Bazaraa and Shetty (1976) - see 7 page
148, and theorems 6.2.3, page 148 and Theorem 6.2.4, page 150.
See also El-Hodiri (1991), Theorem 1, page 48 and Simon (1985), Theorem 4.4. (iii), page 104.

Remark 773 For other conditions, see Theorem 5.8, page 124, in Jahn (1996).

Theorem 774 Assume that

f is pseudo-concave, and
for j = 1; :::; m; gj is quasi-concave, and for k = 1; :::; l; hk is quasi-concave and hk is quasi-concave.
Under the above conditions,
if (x ; ; ) 2 X Rm Rl satis…es the Kuhn-Tucker conditions associated with 18.26, then x solves
18.26.

Proof.
This follows from Theorems proved in the case of inequality constraints.

Similarly, to what we have done in previous sections, we can summarize what said above as follows.
Call (M2 ) the problem
maxx2X f (x) s:t: g (x) 0
(18.29)
h (x) = 0:
and de…ne
M2 := arg max (M2 )

S2 := fx 2 X : 9 2 Rm such that (x; ) is a solution to Kuhn-Tucker system (18:28)g

1. Assume that one of the following conditions hold:

Dg (x )
(a) rank = m + l;or
Dh (x )
(b) for j = 1; :::; m; gj is linear, and h (x) is a¢ ne.
Then
M2 S2

2. Assume that both the following conditions hold:

(a) f is pseudo-concave, and

(b) for j = 1; :::; m; gj is quasi-concave, and for k = 1; :::; l; hk is quasi-concave and hk is
quasi-concave.
Then
M2 S2

18.4 Main Steps to Solve a (Nice) Maximization Problem

We have studied the problem

maxx2X f (x) s:t: g (x) 0 (M )

which we call a maximization problem in the “canonical form”, i.e., a maximization problem with con-
straints in the form of “ ”, and we have de…ned

M := arg max (M )
2 What Mangasarian calls a linear function is what we call an a¢ ne function.
258 CHAPTER 18. MAXIMIZATION PROBLEMS

C := fx 2 X : g (x) 0g

S := fx 2 X : 9 2 Rm such that (x; ) satis…es Kuhn-Tucker Conditions (18:3)g

m
Recall that X is an open, convex subset of Rn , f : X ! R, 8j 2 f1; :::; mg, gj : X ! R and g := (gj )j=1 :
X ! Rm .
In many cases, we have to study the following problem

max f (x) s:t: g (x) 0; (M 0 )

x

in which the set X is not speci…ed.

We list the main steps to try to solve (M 0 ).
1. Canonical form.
Write the problem in the (in fact, our de…nition of) canonical form. Sometimes the problem contains a
parameter 2 an open subset of Rk . Then we should write: for given 2

max f (x; ) s:t: g (x; ) 0:

x2X

2. The set X and the functions f and g.

a. De…ne the functions fe, ge naturally arising from the problem with domain equal to their de…nition set,
where the de…nition set of a function ' is the largest set D' which can be the domain of that function.
b. Determine X. A possible choice for X is the intersection of the “de…nition set”of each function, , i.e.,

X = Df \ Dg1 \ ::: \ Dgm

c. Check if X is open and convex.

d. To apply the analysis described in the previous sections, show, if possible, that f and g are of class
C 2 or at least C 1 .
3. Existence.
Try to apply the Extreme Value Theorem. If f is at least C 1 , then f is continuous and therefore we have
to check if the constraint set C is non-empty and compact. Recall that a set S in Rn is compact if and only
if S is Rn closed and Rn bounded.
Boundedness has to be shown “brute force”, i.e., using the speci…c form of the maximization problem.
If X = Rn , then
C := fx 2 X : g (x) 0g
1
is Rn closed, because of the following well-known argument: C = \m 2
j=1 gj ([0; +1)) ; since gj is C (or at
1
least C 1 ) and therefore continuous, and [0; +1) closed, gj ([0; +1)) is closed in X = Rn ; then C is closed
because intersection of closed sets.
A problem may arise if X is an open proper subset of Rn . In that case the above argument shows that
C is a X closed set and therefore it is not necessarily Rn closed. A possible way out is the following one.
Consider the set
e := fx 2 Rn : ge (x) 0g
C
which is Rn closed - see argument above. Then, we are left with showing that
e
C=C
e is compact, C is compact as well.3
If C
4. Number of solutions.
See subsection 18.1.1. In fact, summarizing what said there, we know that the solution to (M ), if any, is
unique if
1. f is strictly-quasi-concave, and for j 2 f1; :::; mg, gj is quasi-concave; or
2. for j 2 f1; :::; mg, gj is strictly-quasi-concave and
either a. f is quasi-concave and locally non-satiated,
or b. f is a¢ ne and non-costant,
3 See Example below for an application of the above presented strategy.
18.4. MAIN STEPS TO SOLVE A (NICE) MAXIMIZATION PROBLEM 259

or c. f is quasi-concave and strictly monotone,

or d. f is quasi-concave and 8x 2 X, Df (x) >> 0,
or e. f is quasi-concave and 8x 2 X, Df (x) << 0.
5. Necessity of K-T conditions.
Check if the conditions which insure that M S hold, i.e.,
either a. for j = 1; :::; m; gj is pseudo-concave and there exists x++ 2 X such that g (x++ ) 0,
or b. rank Dg (x ) = #J .
If those conditions holds, each property we show it holds for elements of S does hold a fortiori for elements
of M .
6. Su¢ ciency of K-T conditions.
Check if the conditions which insure that M S hold, i.e., that
f is pseudo-concave and for j = 1; :::; m; gj is quasi-concave.
If those conditions holds, each property we show it does not hold for elements of S does not hold a
fortiori for elements of M .
7. K-T conditions.
Write the Lagrangian function and then the Kuhn-Tucker conditions.
8. Solve the K-T conditions.
Try to solve the system of Kuhn-Tucker conditions in the unknown variables (x; ) : To do that;
either, analyze all cases,
or, try to get a “good conjecture” and check if the conjecture is correct.
Example 775 Discuss the problem
1 1
max(x1 ;x2 ) 2 log (1 + x1 ) + 3 log (1 + x2 ) s:t: x1 0
x2 0
x1 + x2 w
with w > 0:
1. Canonical form.
For given w 2 R++ ,
1 1
max(x1 ;x2 ) 2 log (1 + x1 ) + 3 log (1 + x2 ) s:t: x1 0
x2 0 (18.30)
w x1 x2 0
2. The set X and the functions f and g.
a.
fe : ( 1; +1) ! R;
2
(x1 ; x2 ) 7! 21 log (1 + x1 ) + 1
3 log (1 + x2 )
2
ge1 : R ! R (x1 ; x2 ) 7! x1
ge2 : R2 ! R (x1 ; x2 ) 7! x2
ge3 : R2 ! R (x1 ; x2 ) !7 w x1 x2
b.
2
X = ( 1; +1)
and therefore f and g are just fe and ge restricted to X.
c. X is open and convex because Cartesian product of open intervals which are open, convex sets.
d. Let’s try to compute the Hessian matrices of f; g1 ; g2 ; g3 . Gradients are
1 1
Df (x1 ; x2 ) ; = 2(x1 +1) ; 3(x2 +1)
De
g1 (x1 ; x2 ) = (1; 0)
De
g2 (x1 ; x2 ) = (0; 1)
De
g3 (x1 ; x2 ) = ( 1; 1)
Hessian matrices are " #
1
2 2(x1 +1)2
0
D f (x1 ; x2 ) ; = 1
0 3(x2 +1)2
D2 ge1 (x1 ; x2 ) = 0
D2 ge2 (x1 ; x2 ) = 0
D2 ge3 (x1 ; x2 ) = 0
260 CHAPTER 18. MAXIMIZATION PROBLEMS

In fact, g1 ; g2 ; g3 are a¢ ne functions. In conclusion, f and g1 ; g2 ; g3 are C 2 . In fact, g1 and g2 are linear
and g3 is a¢ ne.
3. Existence.
C is clearly bounded: 8x 2 C,
(0; 0) (x1 ; x2 ) (w; w)
In fact, the …rst two constraint simply say that (x1 ; x2 ) (0; 0). Moreover, from the third constraint x1
w x2 w, simply because x2 0; similar argument can be used to show that x2 w.
To show closedness, use the strategy proposed above.
e := fx 2 Rn : g (x)
C 0g

is obviously closed. Since C e e

R2+ , because of the …rst two constraints, C X := ( 1; +1) and
2

therefore C = C e\X =C e is closed.

We can then conclude that C is compact and therefore arg max (18:30) 6= ?.
4. Number of solutions.
From the analysis of the Hessian and using Theorem 753, parts 1 ad 2, we have that f is strictly concave:
1
2 <0
2 (x1 + 1)
" 1
#
2(x1 +1)2
0 1 1
det 1 = 2 2 >0
0 3(x2 +1)2 2 (x1 + 1) 3 (x2 + 1)
Moreover g1 ; g2 ; g3 are a¢ ne and therefore concave. From Proposition 766, part 1, the solution is unique.
5. Necessity of K-T conditions.
Since each gj is a¢ ne and therefore pseudo-concave, we are left with showing that there exists x++ 2 X
such that g (x++ ) >> 0. Just take x++ ++
1 ; x2 = w4 (1; 1) :
w
4 > 0
w
4 > 0
w w w
w 4 4 = 2 > 0

Therefore
M S
6. Su¢ ciency of K-T conditions.
f is strictly concave and therefore pseudo-concave, and each gj is linear and therefore quasi-concave.
Therefore
M S
7. K-T conditions.
1 1
L (x1 ; x2 ; 1; 2; ; w) = log (1 + x1 ) + log (1 + x2 ) + 1 x1 + 2 x2 + (w x1 x2 )
2 3
8 1
>
> 2(x1 +1) + 1 = 0
>
> 1
< 3(x2 +1) + 2 = 0
> min fx1 ; 1 g = 0
>
> min fx ; g = 0
>
: 2 2
min fw x1 x2 ; g = 0
8. Solve the K-T conditions.
Conjecture: x1 > 0 and therefore 1 = 0; x2 > 0 and therefore 2 = 0; w x1 x2 = 0:The Kuhn-Tucker
system becomes: 8 1
> = 0
>
> 2(x1 +1)
>
> 1
= 0
>
< 3(x2 +1)
w x1 x2 = 0
>
> 0
>
>
>
: x1 > 0; x2 > 0
>
1 = 0; 2 = 0
18.4. MAIN STEPS TO SOLVE A (NICE) MAXIMIZATION PROBLEM 261

Then,
8 1
> =
>
> 2(x1 +1)
>
> 1
=
>
< 3(x2 +1)
w x1 x2 = 0
>
> > 0
>
>
>
: x1 > 0; x2 > 0
>
1 = 0; 2 = 0

8 1
>
> x1 = 2 1
>
> 1
>
> x2 = 3 1
>
< 1 1
w 2 1 3 1 = 0
>
> > 0
>
>
>
> x > 0; x2 > 0
>
: 1
1 = 0; 2 = 0

1 1 5 5 1 6(w+2)
0=w 2 1 3 1 =w 6 + 2; and =
6(w+2) > 0. Then x1 = 2 1= 25 1=
3w+6 5 3w+1 1 6(w+2) 2w+4 5
5 = 5 and x2 = 3 1= 35 1= 5 = 2w5 1 .
Summarizing
8 3w+1
>
> x1 = 5 >0
< x = 2w 1
>0
2 5
5
>
> = 6(w+2) > 0
:
1 = 0; 2 =0

Observe that while x1 > 0 for any value of w, x2 > 0i¤ w > 12 . Therefore, for w 2 0; 12 , the above one
is not a solution, and we have to come up with another conjecture;
x1 = w and therefore 1 = 0; x2 = 0 and 2 0; w x1 x2 = 0 and 0:The Kuhn-Tucker conditions
become
8 1
>
> 2(w+1) = 0
>
> 1
>
> 3 + 2 = 0
>
>
< 1 = 0
x2 = 0
>
>
>
> 2 0
>
>
>
> x1 = w
:
0

and
8 1
> = 2(w+1) >0
>
>
>
> 1
= 2(w+1) 31 = 3 2w 2
= 1 2w
>
<
2 6(w+1) 6(w+1)
1 = 0
>
> x2 = 0
>
>
>
> 0
: 2
x1 = w

1 2w
2= 6(w+1) = 0 if w = 12 ;and . 2 = 6(w+1)
1 2w
> 0 if w 2 0; 12
Summarizing, the unique solution x to the maximization problem is

if w 2 0; 12 ; then x1 = w, 1 =0 and x2 = 0; 2 >0

if w = 21 ; then x1 = w, 1 =0 x2 = 0; 2 =0
if w 2 21 ; +1 ; then x1 = 3w+1
5 >0 1 =0 and x2 = 2w5 1
>0 2 =0

The graph of x1 as a function of w is presented below (please, complete the picture)

262 CHAPTER 18. MAXIMIZATION PROBLEMS

x1 1

0.75

0.5

0.25

0
0 0.5 1 1.5 2

The graph below shows constraint sets for di¤ erent “important” values of w and some signi…cant level
curve of the objective function.
w 2 0; 21 :

x2 1.5

0.5

0
-1 -0.5 0 0.5 1 1.5

x1

-0.5

-1

1
w= 2 :

x2 1.5

0.5

0
-1 -0.5 0 0.5 1 1.5

x1

-0.5

-1

1
w> 2 :
18.4. MAIN STEPS TO SOLVE A (NICE) MAXIMIZATION PROBLEM 263

x2 1.5

0.5

0
-1 -0.5 0 0.5 1 1.5

x1

-0.5

-1

Observe that in the example, we get that if 2 = 0, the associated constraint x2 0 is not signi…cant.
See Subsection 18.6.2, for a discussion of that statement.

Of course, several problems may arise in applying the above procedure. Below, we describe some com-
monly encountered problems and some possible (partial) solutions.

18.4.1 Some problems and some solutions

1. The set X:
X is not open.
Rewrite the problem in terms of an open set X 0 and some added constraints. A standard example is the
following one.

maxx2Rn+ f (x) s:t: g (x) 0

which can be rewritten as
maxx2Rn f (x) s:t: g (x) 0
x 0

2. Existence.
a. The constraint set is not compact. Consider again the problem.

maxx2Rn+ f (x) s:t: g (x) 0: (P )

If the constraint set is not compact, it is sometimes possible to …nd another maximization problem such
that
i. its constraint set is compact and nonempty, and
ii. whose solution set is contained in the solution set of the problem we are analyzing.
A way to try to achieve both i. and ii. above is to “restrict the constraint set (to make it compact)
without eliminating the solution of the original problem”. Sometimes, a problem with the above properties
is the following one.

maxx2X f (x) s:t: g (x) 0 (P 1)

f (x) f (b
x) 0

b is an element of X such that g (b

where x x) 0.
De…ne
M := arg max (P ) M 1 := arg max (P 1)
and V and V 1 the constraint sets of Problems (P ) and (P 1), respectively. Observe that

V1 V (18.31)

If V 1 is compact, then M 1 6= ? and the only thing left to show is that M 1 M , which is always insured
as proved below.
264 CHAPTER 18. MAXIMIZATION PROBLEMS

Proposition 776 1. M 1 M and 2. M M 1:

Proof. 1.
If M 1 = ?, we are done. Suppose that M 1 6= ?, and that the conclusion of the Proposition is false, i.e.,
there exists x1 2 M 1 such that
a. x1 2 M 1 , and b. x1 2
= M , or
a. 8x 2 X such that g (x) 0 and f (x) f (b x), we have f x1 f (x);
and
b. either i. x1 2
= V,
or ii. 9e
x 2 X such that
g (e
x) 0 (18.32)
and
x) > f x1
f (e (18.33)
Let’s show that i. and ii. cannot hold.
i.
It cannot hold simply because V 1 V , from 18.31.
ii.
Since x1 2 V 1 ,
f x1 f (b
x) (18.34)
From (18.33) and (18.34), it follows that

f (e
x) > f (b
x) (18.35)

But (18.32), (18.35) and (18.33) contradict the de…nition of x1 , i.e., a. above.
2.
If M = ?, then we are done. Suppose now that M 6= ? and take x 2 M . We want to show that a.
x 2 V 1 and b. for any x 2 V 1 , we have f (x ) f (x).
a. Since x 2 M V , if our claim is false, then we have x 2 V n V 1 , i.e., f (x ) < f (b b 2 V;
x), with x
contradicting the fact that x 2 M .
b. If x 2 V 1 , then x 2 V - simply because V 1 V . Therefore f (x ) > f (x) by the assumption that
x 2 M.

b. Existence without the Extreme Value Theorem If you are not able to show existence, but
i. su¢ cient conditions to apply Kuhn-Tucker conditions hold, and
ii. you are able to …nd a solution to the Kuhn-Tucker conditions,
then a solution exists.

18.5 The Implicit Function Theorem and Comparative Statics

Analysis
The Implicit Function Theorem can be used to study how solutions (x 2 X Rn ) to maximizations
m
problems and, if needed, associated Lagrange or Kuhn-Tucker multipliers ( 2 R ) change when parameters
( 2 Rk ) change. That analysis can be done if the solutions to the maximization problem (and the
multipliers) are solution to a system of equation of the form

F1 (x; ) = 0

with (# choice variables) = (# dimension of the codomain of F1 ), or

F2 ( ; ) = 0

where := (x; ), and (# choice variables and multipliers) = (# dimension of the codomain of F2 ),
To apply the Implicit Function Theorem, it must be the case that the following conditions do hold.

1. (# choice variables x) = (# dimension of the codomain of F1 ), or

(# choice variables and multipliers) = (# dimension of the codomain of F2 ).
18.5. THE IMPLICIT FUNCTION THEOREM AND COMPARATIVE STATICS ANALYSIS 265

2. Fi has to be at least C 1 :That condition is insured if the above systems are obtained from maximization
problems characterized by functions f; g which are at least C 2 : usually the above systems contain some
form of …rst order conditions, which are written using …rst derivatives of f and g.
3. F1 (x ; 0 ) = 0 or F2 ( ; 0 ) = 0: The existence of a solution to the system is usually the result of the
strategy to describe how to solve a maximization form - see above Section 18.4.
4. det [Dx F1 (x ; 0 )]n n 6= 0 or det [D F2 ( ; 0 )](n+m) (n+m) 6= 0: That condition has to be veri…ed
directly on the problem.
If the above conditions are veri…ed, the Implicit Function Theorem allow to conclude what follows (in
reference to F2 ).
There exist an open neighborhood N ( ) X of , an open neighborhood N ( 0 ) of 0 and a
unique C 1 function g : N ( 0 ) Rp ! N ( ) X Rn such that 8 2 N ( 0 ) ; F (g ( ) ; ) = 0 and
h i 1 h i
Dg ( ) = D F ( ; )j =g( ) D F ( ; )j =g( )

Therefore, using the above expression, we may be able to say if the increase in any value of any parameter
implies an increase in the value of any choice variable (or multiplier).
Three signi…cant cases of application of the above procedure are presented below. We are going to
consider C 2 functions de…ned on open subsets of Euclidean spaces.

18.5.1 Maximization problem without constraint

Assume that the problem to study is
max f (x; )
x2X

and that
1. f is concave;
2. There exists a solution x to the above problem associated with 0.
Then, from Proposition 716, we know that x is a solution to
Df (x; 0) =0
Therefore, we can try to apply the Implicit Function Theorem to
F1 (x; ) = Df (x; 0)

Example 777 Consider the maximization problem of a …rm, described as follows. Let the following objects
be given.
Price p 2 R++ of output; quantity x 2 R of output; index t 2 R of technological change; production
function f : R ! R; x 7! f (x) ; such that f 2 C 2 (R; R), f 0 > 0 and f 00 < 0; cost function c : R2 !
R; (x; t) 7! c (x; t) such that c 2 C 2 R2 ; R ; Dx c > 0, Dxx c > 0 > 0, Dt c < 0.
Then, the maximization problem we want to analyze is
max (x; t) := pf (x) c (x; t) :
x2R

We omit here the analysis of the existence problems. Observe that

Dx = pf 0 (x) Dx (x; t) ;

Dxx = pf 00 (x) Dxx (x; t) < 0:

Therefore, the objective function is strictly concave and solutions to the maximization problem are solution
to the equation
F (x; p; t) := pf 0 (x) Dx (x; t) = 0:
To study the e¤ ect of a change in t on the solution value of x, we can apply the Implicit Function Theorem
to the above equation, to get that
Dt F (x; p; t) Dxt (x; t)
Dt (p; t) = = :
Dx F (x; p; t) pf 00 (x) Dxx (x; t)
266 CHAPTER 18. MAXIMIZATION PROBLEMS

Since pf 00 (x) Dxx (x; t) < 0, then the sign of Dt (p; t) is equal to the sign of Dxt (x; t), where Dxt (x; t)
is the derivative of the marginal cost with respect to t.
Another example of application of the strategy illustrated above is presented in Section 19.3.

18.5.2 Maximization problem with equality constraints

Consider a maximization problem

maxx2X f (x; ) s:t: g (x; ) = 0;

Assume that necessary and su¢ cient conditions to apply Lagrange Theorem hold and that there exists
a vector (x ; ) which is a solution (not necessarily unique) associated with the parameter 0 . Therefore,
we can try to apply the Implicit Function Theorem to

Df (x ; 0 ) + Dg (x ; 0)
F2 ( ; ) = (18.36)
g (x ; 0 ) :

18.5.3 Maximization problem with Inequality Constraints

Consider the following maximization problems with inequality constraints. For given 2 ,

maxx2X f (x; ) s:t: g (x; ) 0 (18.37)

Moreover, assume that the set of solutions of that problem is nonempty and characterized by the set of
solutions of the associated Kuhn-Tucker system, i.e., using the notation of Subsection 18.1,
M = S 6= ?:
We have seen that we can write Kuhn-Tucker conditions in one of the two following ways, beside some other
ones, 8
>
> Df (x; ) + Dg (x; ) = 0 (1)
<
0 (2)
(18.38)
>
> g (x; ) 0 (3)
:
g (x; ) = 0 (4)
or

Df (x; ) + Dg (x; ) = 0 (1)

(18.39)
min f j ; gj (x; )g = 0 for j 2 f1; :::; mg (2)
The Implicit Function Theorem cannot be applied to either system (18:38) or system (18:39): system
(18:38) contains inequalities; system (18:39) involves functions which are not di¤ erentiable. We present below
conditions under which the Implicit Function Theorem can be anyway applied to allow to make comparative
statics analysis. Take a solution (x ; ; 0 ) to the above system(s). Assume that
for each j, either j > 0 or gj (x ; 0) > 0:

In other words, there is no j such that j = gj (x ; 0) = 0. Consider a partition J ; Jb of f1; ::; mg, and
the resulting Kuhn-Tucker conditions.
8
> Df (x ; 0 ) + Dg (x ; 0) = 0
>
>
>
< j >0 for j 2J
gj (x ; 0 ) = 0 for j 2J (18.40)
>
>
> j =0
> for j 2 Jb
:
gj (x ; 0 ) > 0 for j 2 Jb
De…ne
g (x ; 0 ) := (gj (x ; 0 ))j2J
gb (x ; 0 ) := (gj (x ; 0 ))j2Jb
:= j j2J
b :=
j j2Jb
18.5. THE IMPLICIT FUNCTION THEOREM AND COMPARATIVE STATICS ANALYSIS 267

Write the system of equations obtained from system (18.40) eliminating strict inequality constraints and
substituting in the zero variables:

Df (x ; 0 ) + Dg (x ; 0 ) = 0
(18.41)
g (x ; 0 ) = 0
Observe that the number of equations is equal to the number of “remaining” unknowns and they are
n + #J
i.e., Condition 1 presented at the beginning of the present Section 18.5 is satis…ed. Assume that the
needed rank condition does hold and we therefore can apply the Implicit Function Theorem to
Df (x ; 0 ) + Dg (x ; 0)
F2 ( ; ) = =0
g (x ; 0 )
Then. we can conclude that there exists a unique C 1 function ' de…ned in an open neighborhood N1 of 0
such that
8 2 N1 ; ' ( ) := (x ( ) ; ( ))
is a solution to system (18.41) at .
Therefore, by de…nition of ',
T
Df (x ( ) ; ) + ( ) Dg (x ( ) ; ) = 0
(18.42)
g (x ( ) ; ) = 0
Since ' is continuous and ( 0) > 0 and gb (x ( 0) ; 0) > 0, there exist an open neighborhood N2 N1
of 0 such that 8 2 N2
( ) > 0
(18.43)
gb (x ( ) ; ) > 0
Take also 8 2 N2 n
b ( ) = 0 (18.44)

Then, systems (18.42), (18.43) and (18.44) say that 8 2 N2 , x ( ) ; ( ) ; b ( ) satisfy Kuhn-Tucker
conditions for problem (18.37) and therefore, since C = M , they are solutions to the maximization problem.
The above conclusion does not hold true if Kuhn-Tucker conditions are of the following form
8
>
> Df (x; ) + T Dg (x; ) = 0
<
j = 0; gj (x; ) = 0 for j 2 J 0
(18.45)
>
> j > 0; gj (x; ) = 0 for j 2 J 00
:
j = 0; gj (x; ) > 0 for j 2 Jb
where J 0 6= ?, J 00 and Jb is a partition of J.
In that case, applying the same procedure described above, i.e., eliminating strict inequality constraints
and substituting in the zero variables, leads to the following systems in the unknowns x 2 Rn and ( j )j2J 00 2
00
R#J : 8
< Df (x; ) + ( j )j2J 00 D (gj )j2J 00 (x; ) = 0
g (x; ) = 0 for j 2 J 0
: j
gj (x; ) = 0 for j 2 J 00
and therefore the number of equation is n + #J 00 + #J 0 > n + #J 00 ;simply because we are considering
the case J 0 6= ?. Therefore the crucial condition
(# choice variables and multipliers) = (# dimension of the codomain of F2 )
is violated.
Even if the Implicit Function Theorem could be applied to the equations contained in (18.45), in an open
neighborhood of 0 we could have
j ( ) < 0 and/or gj (x ( ) ; ) < 0 for j 2 J 0
Then ' ( ) would be solutions to a set of equations and inequalities which are not Kuhn-Tucker condi-
tions of the maximization problem under analysis, and therefore x ( ) would not be a solution to the that
maximization problem.
An example of application of the strategy illustrated above is presented in Section 19.1.
268 CHAPTER 18. MAXIMIZATION PROBLEMS

18.6 The Envelope Theorem and the meaning of multipliers

18.6.1 The Envelope Theorem
Consider the problem (M ) : for given 2 ;

maxx2X f (x; ) s:t: g (x; ) = 0

Assume that for every , the above problem admits a unique solution characterized by Lagrange conditions
and that the Implicit function theorem can be applied. Then, there exists an open set O such that

x : O ! X; x : 7! arg max (P ) ;
v : O ! R; v : !
7 max (P ) and
: O ! Rm ; 7! unique Lagrange multiplier vector

are di¤erentiable functions.

Theorem 778 For any 2 O and for any pair of associated (x ; ) := (x ( ); ( )), we have

D v( ) = D L (x ; ; )

i.e.,
D v( ) = D f (x ; )+ D g (x ; )

Remark 779 Observe that the above analysis applies also to the case of inequality constraints, as long as
the set of binding constraints does not change.

Proof. of Theorem 778 By de…nition of v (:) and x (:) ; we have that

8 2 O; v ( ) = f (x ( ) ; ) : (1)

Consider an arbitrary value and the unique associate solution x = x ( ) of problem (P ) : Di¤erentiating
both sides of (1) with respect to and computing at ; we get
h i h i h i
[D v ( )]1 k = Dx f (x; )j(x ; ) D x ( )j = + D f (x; )j(x ; ) (2)
1 n n k 1 k

From Lagrange conditions

Dx f (x; )j(x ; ) = Dx g (x; )j(x ; ) (3) ;

where is the unique value of the Lagrange multiplier. Moreover

8 2 O; g (x ( ) ; ) = 0: (4)

Di¤erentiating both sides of (4) with respect to and computing at ; we get

h i h i h i
Dx g (x; )j(x ; ) [D x ( )]j = + D g (x; )j(x ; ) = 0 (5) :
m n n k m k

Finally,
(2);(3) (5)
[D v ( )]1 k = Dx g (x; )j(x ; ) D x ( )j = + D f (x; )j(x ; ) =

= D f (x; )j(x ; ) + D g (x; )j(x ; )

18.6. THE ENVELOPE THEOREM AND THE MEANING OF MULTIPLIERS 269

18.6.2 On the meaning of the multipliers

The main goal of this subsection is to try to formalize the following statements.
1. The fact that j = 0 indicates that the associated constraint gj (x) 0 is not signi…cant - see Propo-
sition 780 below.
2. The fact that j > 0 indicates that a way to increase the value of the objective function is to violate
the associated constraint gj (x) 0 - see Proposition 781 below.
For simplicity, consider the case m = 1. Let (CP ) be the problem
max f (x) s:t: g (x) 0
x2X

and (U P ) the problem

max f (x)
x2X
with f strictly quasi-concave, g is quasi-concave and solutions to both problem exist . De…ne x :=
arg max (CP ) with associated multiplier , and x := arg max (U P ).
Proposition 780 If = 0, then x = arg max (U P ) , x = arg max (CP ) :
Proof. By the assumptions of this section, the solution to (CP ) exists, is unique, it is equal to x and
there exists such that
Df (x ) + Dg (x ) = 0
min fg (x ) ; g = 0:
Moreover, the solution to (U P ) exists, is unique and it is the solution to
Df (x) = 0:
Since = 0, the desired result follows.
Take " > 0 and k 2 ( "; +1). Let (CP k) be the problem
max f (x) s:t: g (x) k
x2X

Let
b : ( "; +1) ! X; k 7! arg max (CP k)
x

vb : ( "; +1) ! R; k 7! max (CP k) := f (b

x (k))
Let b (k) be such that x
b (k) ; b (k) is the solution to the associated Kuhn-Tucker conditions.
Observe that
x =x b (0) ; = b (0) (18.46)
0
Proposition 781 If > 0, then vb (0) < 0.
Proof. From the envelope theorem,
@ (f (x) + (g (x) k)) b (k)
8k 2 ( "; +1) ; vb0 (k) = =
@k x(k);b(k)
jb

and from (18.46)

vb0 (0) = b (0) = < 0:

Remark 782 Consider the following problem. For given a 2 R,

maxx2X f (x) s:t: g (x) a 0 (18.47)
Assume that the above problem is “well-behaved” and that x (a) = arg max (18.47), v (a) = f (x (a)) and
(x (a) ; (a)) is the solution of the associated Kuhn-Tucker conditions. Then, applying the Envelope Theorem
we have
v 0 (a) = (a)
270 CHAPTER 18. MAXIMIZATION PROBLEMS
Chapter 19

Applications to Economics

19.1 A Walrasian Consumer Problems

In this section, we present two versions of the Walrasian consumer’s problem.

19.1.1 Zero consumption is allowed.

We consider again the problem presented in Section 10.3, whose basic characteristics we repeat below. Let the
following objects be given: price vector p 2 Rn++ ; consumption vector x 2 Rn , consumer’s wealth w 2 R++ ,
continuous utility function u : Rn ! R, x 7! u (x). The consumer solves the following problem. For given,
p 2 Rn++ ,w 2 R++ , solve

(P 0) maxx2Rn u (x) s:t: w px 0

x1 0 1

:::

xn 0: n

where we wrote multipliers next to each constraint. To be able to easily apply Kuhn-Tucker theorems,
we also assume that
1. u is C 2 ;
2. for any x 2 Rn , Du (x) >> 0, (and therefore u is strictly increasing), and
3. or any x 2 Rn , D2 u (x) is negative de…nite, (and therefore u is strictly concave).
It is not di¢ cult to show that all the steps in the “Recipe to solve a nice maximization problem” (see
Section 18.4) do go through. Let’s write and discuss the system of Kuhn-Tucker conditions, which is presented
below. 8
>
> Dx1 u (x) p1 + 1 = 0
>
>
>
> :::
>
>
>
> Dxn u (x) pn + n = 0
>
>
<
min f ; w pxg = 0 (19.1)
>
>
>
>
>
>
>
> min f 1 ; x1 g = 0
>
>
>
> :::
:
min f n ; xn g = 0
Let (x ; ; ) denote the solution to the above system associated to a given (p; w).
Claim1. > 0 and therefore w px = 0.
Proof of the Claim.
Assume otherwise; then, from system (19:1), we get Dx1 u (x) + 1 = 0; but, 1 0 and, by assumption
Du (x) >> 0,
we also have Dx1 u (x) + 1 > 0, a contradiction.
Claim1. Assume n = 2. Then,

271
272 CHAPTER 19. APPLICATIONS TO ECONOMICS

(a) if
w
Dx1 u p1 ; 0 p1
< ; (19.2)
Dx2 u w p2
p1 ; 0

then x2 > 0 and 2 = 0;

(b) if
p1 Dx1 u 0; pw2
< ;
p2 Dx2 u 0; pw2

then x1 > 0 and 1 = 0.

Proof of the Claim.
(a)
w
Assume otherwise, i.e., x2 = 0 and therefore 2 0. From Claim 1, we have that x1 = p1 and therefore
1 = 0. Then, Kuhn-Tucker system becomes, omitting stars,
8
>
> w
p1
> Dx1 u p1 ; 0
>
= 0
>
>
<
> Dx2 u pw1 ; 0 p2 + 2 = 0
>
>
>
>
>
:
2 0:

Then,
w
w w Dx1 u p1 ; 0
2
0 = Dx2 u ;0 p + 2 = Dx2 u ;0 p2 + 2;
p1 p1 p1
p1
and multiplying both sides by w
, we get
p2 Dx2 u p1 ;0

w
p1 Dx1 u p1 ; 0 p1
0= + 2 ;
p2 Dx2 u w
p2 Dx2 u w
p1 ; 0 p1 ; 0

and therefore
w
p1 Dx1 u p1 ; 0 p1
= 2 0;
p2 Dx2 u w
p2 Dx2 u w
p1 ; 0 p1 ; 0

contradicting assumption (19:2).

(b))
The proof is similar to the above one.

19.1.2 Zero consumption is not allowed.

In this case, we assume the domain of the utility function is Rn++ . We also make the following assumptions:
1. u is a C 2 function;
2. u is di¤erentiably strictly increasing, i.e., 8x 2 Rn++ ; Du (x) 0;
3. u is di¤erentiably strictly quasi-concave, i.e., 8x 2 Rn++ ; x 6= 0 and Du (x) x = 0 ) xT D2 u (x) x <
0; (this assumption is more general of the corresponding assumption used in the previous section);
4. for any u 2 R; x 2 Rn++ : u (x) u is closed in Rn .

Remark 783 Assumption 4 means that “indi¤ erence curves do not touch the axes”.

The maximization problem for household h is

(P 1) maxx2Rn++ u (x) s:t: px w 0:

19.1. A WALRASIAN CONSUMER PROBLEMS 273

The budget set of the above problem is clearly not compact. But below, we show that the solution set to
(P 1) is the same as the solution set to (P 2) and (P 3) below: indeed, the constraint set of (P 3) is compact.

(P 2) maxx2Rn++ u (x) s:t: px w = 0;

(P 3) maxx2Rn++ u (x) s:t: px w 0;

u (x) u (e ) ;
where e 2 x 2 Rn++ : px w .
Denote by C1 ; C2 and C3 the constraint sets of Problem (P 1) ; (P 2) and (P 3), respectively. De…ne
Mi = arg max (P i) for i 2 f1; 2; 3g.

Proposition 784 1. M3 6= ?;
2. M1 = M2 = M3 .

Proof. 1.
From the Extreme Value Theorem, it su¢ ces to show that C3 is compact. Indeed,

C31 := x 2 Rn++ : px w 0

is closed in Rn , because a¢ ne functions on Rn are continuous. Moreover,

C32 := x 2 Rn++ : u (x) u (e )

is closed in Rn , by assumption. Then C is closed in Rn because intersection of two closed sets in Rn .

n
C3 is bounded below by 0 2 Rn and above by pwi 2 Rn .
i=1
2.
a. M1 = M3 .
It follows directly from the analysis presented in Subsection 2 in Section 18.4.1.
b. M1 = M2 .
We want to show that
def:
x 2 M1 , x 2 C1 and for any x 2 C1 , we have u (x ) u (x)

def:
x 2 M2 , x 2 C2 and for any x 2 C2 , we have u (x ) u (x)

[)]
If x 2 M1 , then x 2 C2 : suppose not; then x 2 C1 n C2 , i.e., px < w. But then, by strict monotonicity
of u, x 2= M1 .
Moreover, since C2 C1 , then u (x ) u (x) for any x 2 C2 , as desired.
[(]
If x 2 M2 , then x 2 C2 C1 . Now, suppose that there exists x 2 C1 n C2 such that

u (x) > u (x ) : (19.3)

Since px < w, then p x + (w px) e1n = w, where e1n = (1; 0; :::; 0) is the …rst element in the canonical
basis of Rn : Therefore, x
e := x + (w px) e1n 2 C2 and, from strict monotonicity of u and from (19:3), we
have
u (e
x) > (x) > u (x ) ;
contradicting the fact that x 2 M2 .

Theorem 785 Under the above Assumptions,

h : RC
++ R++ !! RC
++ ; (p; w) 7!7! arg max (P )

is indeed a C 1 function.
274 CHAPTER 19. APPLICATIONS TO ECONOMICS

Proof.
Observe that, from it can be easily shown that, is a function.
We want to show that (P 2) satis…es necessary and su¢ cient conditions to Lagrange Theorem, and then
apply the Implicit Function Theorem to the First Order Conditions of that problem.
The necessary condition is satis…ed because Dx [px w] = p 6= 0;
De…ne also
: Rn++ Rn++1 ! Rn++ ;
: (p; w) 7! Lagrange multiplier for (P 2) :
The su¢ cient conditions are satis…ed because: from Assumptions (smooth 4), u is di¤erentiably strictly
quasi-concave; the constraint is linear; the Lagrange multiplier is strictly positive -see below.
The Lagrangian function for problem (P 2) and the associated First Order Conditions are described below.
L (x; ; p; w) = u (x) + ( px + w)
(F OC) (1) Du (x) p = 0
(2) px + w = 0
De…ne
F : Rn++ R++ Rn++ R++ ! Rn R;
Du (x) p
F : (x; ; p; w) 7! :
px + w
As an application of the Implicit Function Theorem, it is enough to show that D(x; ) F (x; ; p; w) has
full row rank (n + 1).
Suppose D(x; ) F does not have full rank; then there would exist
x 2 Rn and 2 R such that := ( x; ) 6= 0 and D(x; ) F ( x; ) = 0, or
D2 u (x) pT x
= 0;
p 0
or
(a) D2 u (x) x pT =0
(b) p x =0
:
The idea of the proof is to contradict Assumption u3.
Claim 1. x 6= 0:
By assumption it must be 6= 0 and therefore, if x = 0; 6= 0: Since p 2 Rn++ ; pT 6= 0: Moreover, if
T
x = 0; from (a), we would have p = 0; a contradiction. :
Claim 2. Du x = 0:
From (F OC1) ; we have Du x hp x = 0; using (b) the desired result follows .
Claim 3. xT D2 u x = 0:
Premultiplying (a) by xT ; we get xT D2 u (x) x xT pT = 0: Using (b) ; the result follows.
Claims 1, 2 and 3 contradict Assumption u3.

The above result gives also a way of computing D(p;w) x (p; w) ; as an application of the Implicit Function
Theorem .
Since

x p w

Du (x) p D2 u pT In 0
px + w p 0 x 1

Dp x Dw x
D(p;w) (x; ) (p; w) = =
(n+1) (n+1) Dp Dp
1
D2 u pT In 0
=
p 0 (n+1) (n+1)
x 1 (n+1) (n+1)

To compute the inverse of the above matrix, we can use the following fact about the inverse of partitioned
matrix (see for example, Goldberger, (1963), page 26)
19.2. PRODUCTION 275

Let A be an n n nonsingular matrix partitioned as

E F
A= ;
G H
1
where En1 n1 ; Fn1 n2 ; G n2 n1 ; Hn2 n2 and n1 + n2 = n: Suppose that E and D := H GE F are
non singular. Then
1
1 E I + F D 1 GE 1
E 1
FD 1
A = :
D 1 GE 1 D 1

If we assume that D2 u is negative de…nite and therefore invertible, we have

" 1 1 1
#
1 1 T 1
D2 u pT D2 I+ p p D2 D2 pT
=
p 0 1
p D2
1 1

1
where = p D2 pT 2 R++ :
And

h i
1 1 T 1 1 1 In
[Dp x (p; w)]n n = D2 I+ p p D2 D2 pT =
h x i
1 1 T 1 1 1 1 1 1
= D2 I+ p p D2 D2 pT x = D2 I + pT p D 2 + pT x

h i 0 1
1 1 T 1 1 1 1
[Dw x (p; w)]n 1 = D2 I+ p p D2 D2 pT = D2 pT
1
h i
1 In 1 1
[Dp (p; w)]1 n = 1
p D2 1
= p D2 +x :
x
h i 0
1 1 1 1
[Dw (p; w)]1 1 = p D2 1
= :
:
:
As a simple application of the Envelope Theorem, we also have that, de…ned the indirect utility function
as

v : Rn+1
++ ! R; v : (p; w) 7! u (x (p; w)) ;

we have that

D(p;w) v (p; w) = xT 1 :

19.2 Production
C
De…nition 786 A production vector (or input-output or netput vector) is a vector y := (y c )c=1 2 RC which
describes the net outputs of C commodities from a production process. Positive numbers denote outputs,
negative numbers denote inputs, zero numbers denote commodities neither used nor produced.

Observe that, given the above de…nition, py is the pro…t of the …rm.

De…nition 787 The set of all feasible production vectors is called the production set Y RC : If y 2 Y; then
y can be obtained as a result of the production process; if y 2
= Y;that is not the case.

De…nition 788 The Pro…t Maximization Problem (PMP) is

max py s:t: y 2 Y:
y
276 CHAPTER 19. APPLICATIONS TO ECONOMICS

It is convenient to describe the production set Y using a function F : RC ! R called the transformation
function. That is done as follows:
Y = y 2 RC : F (y) 0 :
We list below a smooth version of the assumptions made on Y , using the transformation function.
Some assumption on F (:) :
(1) 9y 2 RC such that F (y) 0:
(2) F is C 2 .
(3) (No Free Lunch) If y 0; then F (y) < 0:
(4) (Possibility of Inaction) F (0) = 0:
(5) (F is di¤erentiably strictly decreasing) 8y 2 RC ; DF (y) 0
(6) (Irreversibility) If y 6= 0 and F (y) 0; then F ( y) < 0:
(7) (F is di¤erentiably strictly concave) 8 2 RC n f0g ; T D2 F (y) < 0:

De…nition 789 Consider a function F (:) satisfying the above properties and a strictly positive real number
N . The Smooth Pro…t Maximization Problem (SPMP) is

max py s:t: F (y) 0 and kyk N: (19.4)

y

Remark 790 For any solution to the above problem it must be the case that F (y) = 0. Suppose there exists
a solution y 0 to (SPMP) such that F (y 0 ) > 0: Since F is continuous, in fact C 2 , there exists " > 0 such that
1 1 1
PC " 2 2 " 2 2 "2 2
z 2 B (y 0 ; ") ) F (z) > 0: Take z 0 = y 0 + "C1 : Then, d (y 0 ; z 0 ) := c=1 C = C C = C =
p" < ": Therefore z 0 2 B (y 0 ; ") and
C
F (z 0 ) > 0 (1) :
But,
" 1
pz 0 = py 0 + p > py 0 (2) :
C
(1) and (2) contradict the fact that y 0 solves (SPMP).

Proposition 791 If a solution with kyk < N to (SP M P ) exists : Then y : RC C

++ ! R , p 7! arg max (19:4)
is a well de…ned C 1 function.

Proof.
Let’s …rst show that y (p) is single valued.
Suppose there exist y; y 0 2 y (p) with y 6= y 0 : Consider y := (1 ) y + y 0 : Since F (:) is strictly
0
concave, it follows that F y > (1 ) F (y) + F (y ) 0; where the last inequality comes from the fact
that y; y 0 2 y (p) : But then F y > 0: Then following the same argument as in Remark 790, there exists
" > 0 such that z 0 = y + "C1 and F (z 0 ) > 0: But pz 0 > py = (1 ) py + py 0 = py; contradicting the fact
that y 2 y (p) :
Let’s now show that y is C 1
From Remark 790 and from the assumption that kyk < N; (SPMP) can be rewritten as maxy py s:t: F (y) =
0: We can then try to apply Lagrange Theorem.
Necessary conditions: DF (y) 0;
su¢ cient conditions: py is linear and therefore pseudo-concave; F (:) is di¤erentiably strictly concave and
therefore quasi-concave; the Lagrange multiplier is strictly positive -see below.
Therefore, the solutions to (SP M P ) are characterized by the following First Order Conditions, i.e., the
derivative of the Lagrangian function with respect to y and equated to zero:

y p
L (y; p) = py + F (y) : p + DF (y) = 0 F (y) = 0
:
Observe that = D 1p1 > 0:
y F (y)
As usual to show di¤erentiability of the choice function we take derivatives of the First Order Conditions.

y
T
p + DF (y) = 0 D2 F (y) [DF (y)]
F (y) = 0 DF (y) 0
19.3. THE DEMAND FOR INSURANCE 277

We want to show that the above matrix has full rank. By contradiction, assume that there exists
:= ( y; ) 2 RC R; = 6 0 such that
T
D2 F (y) [DF (y)] y
= 0;
DF (y) 0

i.e.,

T
D2 F (y) y + [DF (y)] = 0 (a) ;

DF (y) y=0 (b) :

T
Premultiplying (a) by y T ; we get y T D2 F (y) y + y T [DF (y)] = 0: From (b) ; it follows
T 2
that y D F (y) y = 0; contradicting the di¤erentiably strict concavity of F (:) :
(3)
From the Envelope Theorem, we know that if y; is the unique pair of solution-multiplier associated
with p; we have that
Dp (p)jp = Dp (py)j(p;y) + Dp F (y)j(p;y) :
Since Dp (py)j(p;y) = y, Dp F (y) = 0 and, by de…nition of y; y = y (p) ; we get Dp (p)jp = y (p) ; as
desired:
(4)
From (3) ; we have that Dp y (p) = Dp2 (p) : Since (:) is convex -see Proposition ?? (2)- the result
follows.
(5)
From Proposition ?? (4) and the fact that y (p) is single valued, we know that 8 2 R++ ; y (p) y ( p) =
0. Taking derivatives with respect to ; we have Dp y (p)j( p) p = 0: For = 1; the desired result follows.

19.3 The demand for insurance

Consider an individual whose wealth is

W d with probability , and

W with probability 1 ,
where W > 0 and d > 0:
Let the function
u : A ! R; u : c ! u (c)

be the individual’s Bernoulli function.

Assumption 1. 8c 2 R; u0 (c) > 0 and u00 (c) < 0:

Assumption 2. u is bounded above.

An insurance company o¤ers a contract with following features: the potentially insured individual pays
a premium p in each state and receives d if the accident occurs. The (potentially insured) individual can
buy a quantity a 2 R of the contract. In the case, she pays a premium (a p) in each state and receives a
reimbursement (a d) if the accident occurs. Therefore, if the individual buys a quantity a of the contract,
she get a wealth described as follows

W1 := W d ap + ad with probability , and

(19.5)
W2 := W ap with probability 1 :

Remark 792 It is reasonable to assume that p 2 (0; d) :

278 CHAPTER 19. APPLICATIONS TO ECONOMICS

De…ne
U : R ! R; U : a 7! u (W d ap + ad) + (1 ) u (W ap) :
Then the individual solves the following problem. For given, W 2 R++ ; d 2 R++ ; p 2 (0; d) ; 2 (0; 1)

maxa2R U (a) (M ) (19.6)

To show existence of a solution, we introduce the problem presented below. For given W 2 R++ ; d 2
R++ ; p 2 (0; d) ; 2 (0; 1)

maxa2R U (a) s:t: U (a) U (0) (M 0 )

De…nedA := arg max (M ) and A0 := arg max (M 0 ) ; the existence of solution to (M ) ; follows from the
Proposition below.

Proposition 793 1. A0 A : 2. A0 6= ?:

Proof.
Exercise

To show that the solution is unique, observe that

U 0 (a) = u0 (W d + a (d p)) (d p) + (1 ) u0 (W ap) ( p) (19.7)

and therefore

( ) (+) (+) ( ) (+)

(+) 00 2
U 00 (a) = u (W d + a (d p))(d p) + (1 )u00 (W ap) p2 < 0:
Summarizing, the unique solution of problem (M ) is the unique solution of the equation:

U 0 (a) = 0:

De…nition 794 a : R++ (0; 1) (0; d) R++ ! R;

a : (d; ; p; W ) 7! arg max (M ) :
U : ! R;
U : 7! u (W d + a ( ) (d p)) + (1 ) u (W a ( ) p)

Proposition 795 The signs of the derivatives of a and U with respect to are presented in the following
table1 :
d p W

a > 0 if a 2 [0; 1] >0 T0 0 if a 1

U 0 if a 2 [0; 1] 0 if a 2 [0; 1] 0 if a 0 >0

Proof. Exercise.

19.4 Exercises on part IV

See Tito Pietra’s …le (available on line): Exercises 15.1 ! 15.6.

1 Conditions on a ( ) contained in the table can be expressed in terms of exogenous variables.

 Part V

Problem Sets

279
Chapter 20

Exercises

20.1 Linear Algebra

1.
Show that the set of pair of real numbers is not a vector space with respect to the following operations:
(i). (a; b) + (c; d) = (a + c; b + d) and k (a; b) = (ka; b) ;
(ii) (a; b) + (c; d) = (a + c; b) and k (a; b) = (ka; kb) :

2.
Show that W is not a vector subspace of R3 on R if
(i) W = (x; y; z) 2 R3 : z 0 ;
(ii) W = x 2 R3 : kxk 1 ;
(iii) W = Q3 :

3.
Let V be the vector space of all functions f : R ! R. Show the W is a vector subspace of V if
(i) W = ff 2 V : f (1) = 0g ;
(ii) W = ff 2 V : f (1) = f (2)g :

4.

Show that
(i).

V = (x1 ; x2 ; x3 ) 2 R3 : x1 + x2 + x3 = 0
is a vector subspace of R3 ;
(ii).
S = f(1; 1; 0) ; (0; 1; 1)g
is a basis for V .

5.
Show the following fact.
Proposition. Let a matrix A 2 M (n; n), with n 2 N be given. The set

CA := fB 2 M (n; n) : BA = ABg

is a vector subspace of M (n; n) (with respect to the …eld R).

6.
Let U and V be vector subspaces of a vector space W . Show that

U + V := fw 2 W : 9u 2 U and v 2 V such that w = u + vg

281
282 CHAPTER 20. EXERCISES

is a vector subspace of W .

7.
Show that the following set of vectors is linearly independent:

f(1; 1; 1; ) ; (0; 1; 1) ; (0; 0; 1)g :

8. Using the de…nition, …nd the change-of-basis matrix from

S = fu1 = (1; 2); u2 = (3; 5)g

to
E = fe1 = (1; 0); e2 = (0; 1)g
and from E to S. Check the conclusion of Proposition ??, i.e., that one matrix is the inverse of the other
one.

9. Say if the determinant of the following matrix is di¤erent from zero.

2 3
6 2 1 0 5
62 1 1 2 17
6 7
C=6 6 1 1 2 2 377
43 0 2 3 15
1 1 3 4 2

10. Say for which values of k 2 R the following matrix has rank a. 4, b. 3:
2 3
k+1 1 k 2
A := 4 k 1 2 k k 5
1 0 1 1

11.
Show that
V = (x1 ; x2 ; x3 ) 2 R3 : x1 x2 = 0
is a vector subspace of R3 and …nd a basis for V .

12.
Given

l : R4 ! R4 ; l (x1 ; x2 ; x3 ; x4 ) = x1 ; x1 + x2 ; x1 + x2 + x3 ; x1 + x2 + x3 + x4

show it is linear, compute the associated matrix with respect to canonical bases, and compute ker l and Iml.

13.
Complete the text below.
Proposition. Assume that l 2 L (V; U ) and ker l = f0g. Then,

8u 2 Im l; there exists a unique v 2 V such that l (v) = u:

Proof.
Since ::::::::::::::::::::::::, by de…nition, there exists v 2 V such that

l (v) = u: (20.1)

Take v 0 2 V such that l (v 0 ) = u. We want to show that

:::::::::::::::::::::::: (20.2)
20.1. LINEAR ALGEBRA 283

Observe that
(a)
l (v) l (v 0 ) = ::::::::::::::::::::::::: (20.3)
where (a) follows from ::::::::::::::::::::::::.
Moreover,
(b)
l (v) l (v 0 ) = :::::::::::::::::::::::::; (20.4)
where (b) follows from ::::::::::::::::::::::::.
Therefore,
l (v v 0 ) = 0;
and, by de…nition of ker l,
::::::::::::::::::::::::: (20.5)
Since, ::::::::::::::::::::::::., from (20:5), it follows that

v v 0 = 0:

14.
Let the following sets be given:

V = (x1 ; x2 ; x3 ; x4 ) 2 R4 : x1 x2 + x3 x4 = 0

and
W = (x1 ; x2 ; x3 ; x4 ) 2 R4 : x1 + x2 + x3 + x4 = 0
If possible, …nd a basis of V \ W .

15.
Say if the following statement is true or false.
1
Let V and U be vector spaces on R, W a vector subspace of U and l 2 L (V; U ). Then l (W ) is a vector
subspace of V .

16.
Let the following full rank matrices

a11 a12 b11 b12

A= B=
a21 a22 b21 b22

be given. Say for which values of k 2 R; the following linear system has solutions.
2 3
2 3 x1 2 3
1 a11 a12 0 0 0 6 7 k
6 2 a21 a22 0 x
6 0 0 7 6 2 7 6
7 6 x3 7 6 1 7
7
6 3 5 6 b11 b12 0 7 6 7 6 7 =6 2 7
6
6 x4 7 7
4 4 7 8 b21 b22 0 4 5 6 7 4 3 5
x5 5
1 a11 a12 0 0 k k
x6

17.
Consider the following Proposition contained in Section 8.1 in the class Notes:
Proposition .8v 2 V;
U
[l]V [v]V = [l (v)]U (20.6)
Verify the above equality in the case in which
a.
x1 + x2
l : R2 ! R2 ; (x1 ; x2 ) 7!
x1 x2
b. the basis V of the domain of l is
1 0
; ;
0 1
284 CHAPTER 20. EXERCISES

c. the basis U of the codomain of l is

1 2
; ;
1 1

d.
3
v= :
4

18.
Complete the following proof.
Proposition. Let
n; m 2 N such that m > n; and
a vector subspace L of Rm such that dim L = n
be given. Then, there exists l 2 L (Rn ; Rm ) such that Im l = L.
n
Proof. Let v i i=1 be a basis of L Rm . Take l 2 L (Rn ; Rm ) such that

8i 2 f1; :::; ng ; l2 ein = v i ;

where ein is the i–th element in the canonical basis in Rn . Such function does exists and, in fact, it is
unique as a consequence of a Proposition in the Class Notes that we copy below:
..........................................................................................................................
Then, from the Dimension theorem

dim Iml = ::::::::::::::::::::::::::::::::::::::::::::::

Moreover,
n
L = ::::::::::: v i i=1
:::::::::::::::
Summarizing,
L Im l , dim L = n and dim Im l n;
and therefore
dim Iml = n:
Finally, from Proposition .................................in the class Notes since L Im l , dim L = n and
dim Iml = n; we have that Im l = L, as desired.
Proposition ............................. in the class Notes says what follows:
.......................................................................................

19.
Say for which value of the parameter a 2 R the following system has one, in…nite or no solutions
8
>
> ax1 + x2 = 1
<
x1 + x2 = a
> 2x1
> + x2 = 3a
:
3x1 + 2x2 = a

20.
Say for which values of k;the system below admits one, none or in…nite solutions.

A (k) x = b (k)
where k 2 R, and 2 3 2 3
1 0 k 1
6 1 k 2 k 7 6 k 7
A (k) 6 7; b (k) 6 7:
4 1 k 5 4 1 5
1 k 1 0
21.
20.1. LINEAR ALGEBRA 285

Let V = v 1 ; v 2 ; :::; v n be a set of vectors in Rn such that for any i; j 2 f1; :::; ng,
8
< 0 if i 6= j
vi vj = (20.7)
:
1 if i = j:

Show that V is a basis of Rn .

22.
Given a vector space V on a …eld F , sets A; B V and k 2 F , we de…ne

A + B := fv 2 V : there exist a 2 A and b 2 B such that v = a + bg ;

kA := fv 2 V : there exist a 2 A such that v = kag :

Given a vector space V on a …eld F; a linear function T 2 L(V; V ) and W vector subspace of V , W is
said to be T -invariant if
T (W ) W:
Let W be both S-invariant and T -invariant and let k 2 F . Show that
a. W is S + T -invariant;
b. W is S T -invariant;
c. W is kT -invariant.
23.
Show that the set of all 2 2 symmetric real matrices is a vector subspace of M (2; 2) and compute its
dimension.
24.
Let V be a vector space on a …eld F and W a vector subspace of V . Show that
a. W + W = W , and
b. for any 2 F n f0g, W = W .
25.
Let Pn (R) be the set polynomials of degree smaller or equal than n 2 N+ on the set of real numbers ,
i.e.,
( n
)
X
i
Pn (R) = f : R ! R such that 9a0 ; a1 ; :::; an 2 R such that for any t 2 R; f (t) = ai t :
i=0

Show that Pn (R) is isomorphic to Rn+1 .

286 CHAPTER 20. EXERCISES

20.2 Some topology in metric spaces

20.2.1 Basic topology in metric spaces
1.
Do Exercise 341: Let d be a metric on a non-empty set X. Show that

d (x; y)
d0 : X X ! R; d (x; y) =
1 + d (x; y)

is a metric on X:

2.
Let X be the set of continuous real valued functions with domain [0; 1] R and
Z 1
d (f; g) = f (x) g (x) dx;
0

where the integral is the Riemann Integral (that one you learned in Calculus 1). Show that (X; d) is not
a metric space.

3.
Do Exercise 358 for n = 2 : 8n 2 N; 8i 2 f1; :::; ng ; 8ai ; bi 2 R with ai < bi ,
n
i=1 (ai ; bi )

is (Rn ; d2 ) open.

4.
Show the second equality in Remark 366:

1 1
\+1
n=1 ; = f0g
n n

5.
Say if the following set is (R; d2 ) open or closed:

n 1
S := x 2 R : 9 n 2 N such that x = ( 1)
n

6.
Say if the following set is (R; d2 ) open or closed:

1 1
A := [+1
n=1 ; 10 :
n n
7.
Do Exercise 376: show that F (S) = F S C .

8.
Do Exercise 377: show that F (S) is a closed set.

9.
Let the metric space (R; d2 ) be given. Find Int S; Cl (S) ; F (S) ; D (S) ; Is (S) and say if S is open or
closed for S = Q, S = (0; 1) and S = x 2 R : 9n 2 N+ such that x = n1 .

10.
Show that the following statements are false:
20.2. SOME TOPOLOGY IN METRIC SPACES 287

a. Cl (Int S) = S,
b. Int Cl (S) = S:
11.
Given S R, say if the following statements are true or false.
a. S is an open bounded interval ) S is an open set;
b. S is an open set ) S is an open bounded interval;
c. x 2 F (S) ) x 2 D (S);
d. x 2 D (S) ) x 2 F (S) :

12.
Using the de…nition of convergent sequences, show that the following sequences do converge:
a. (xn )n2N 2 R1 such that 8n 2 N; xn = 1;
b. (xn )n2N 2 R1 such that 8n 2 N; xn = n1 .

13.
Using Proposition 403, show that [0; 1] is (R; d2 ) closed.

14.
Show the following result: A subset of a discrete space, i.e., a metric space with the discrete metric, is
compact if and only if it is …nite.

15.
Say if the following statement is true: An open set is not compact.

16.
Using the de…nition of compactness, show the following statement: Any open ball in R2 ; d2 is not
compact.

17.
Show that f (A [ B) = f (A) [ f (B) :

18.
Show that f (A \ B) 6= f (A) \ (B) :

19.
Using the characterization of continuous functions in terms of open sets, show that for any metric space
(X; d) the constant function is continuous.

20.
a. Say if the following sets are (Rn ; d2 ) compact:
i.

Rn+ ;
ii.

8x 2 Rn and8r 2 R++ ; ; Cl B (x; r) :

b. Say if the following set is (R; d2 ) compact:

1
x 2 R : 9n 2 N such that x = :
n

21.
Given the continuous functions
g : R n ! Rm
288 CHAPTER 20. EXERCISES

show that the following set is closed

fx 2 Rn : g (x) 0g

22.
Assume that f : Rm ! Rn is continuous. Say if

X = fx 2 Rm : f (x) = 0g

is (a) closed, (b) is compact.

23.
Using the characterization of continuous functions in terms of open sets, show that the following function
is not continuous 8
< x if x 6= 0
f : R ! R; f (x) =
:
1 if x = 0

24.
Using the Extreme Value Theorem, say if the following maximization problems have solutions (with k k
being the Euclidean norm).
n
X
maxn xi s:t: kxk 1
x2R
i=1
n
X
maxn xi s:t: kxk < 1
x2R
i=1
n
X
maxn xi s:t: kxk 1
x2R
i=1

25.
Let (E; kkE ), (F; kkF ) be normed vector spaces. A function f : (E; kkE ), (F; kkF ) is bounded if

9M 2 R++ such that 8x 2 E kf (x)kF M:

Show that given a linear function l : E ! F ,

l is bounded , l = 0:

26.
f : (X; d) ! R is upper semicontinuous at x0 2 X if

8" > 0; 9 > 0 such that d (x x0 ) < ) f (x) < f (x0 ) + ":

f is upper semicontinuous if is upper semicontinuous at any x0 2 X.

Show that the following statements are equivalent:
a. f is upper semicontinuous;
b. for any 2 R, fx 2 X : f (x) < g is (X; d) open;
c. for any 2 R, fx 2 X : f (x) g is (X; d) closed.
27.
Let A be a subset of (Rn ; d) where d is the Euclidean distance. Show that if A is (Rn ; d) open, then for
any x 2 X, fxg + A is (Rn ; d) open.
Hint: use the fact that for any x; y 2 Rn , d (x; y) = kx yk and therefore, for any a 2 Rn , d (a + x; a + y) =
ka + x a yk = kx yk = d (x; y).
28.
Let (X; d) be a metric space. Show that if K1 and K2 are compact subsets of X, then K1 +K2 is compact.
29.
20.2. SOME TOPOLOGY IN METRIC SPACES 289

Given two metric spaces (E; d1 ) and (F; d2 ), a function f : E ! F is an isometry with respect to d1 and
d2 if 8x1 ; x2 2 E,
d2 (f (x1 ); f (x2 )) = d1 (x1 ; x2 ):
Show that if f : E ! F is an isometry then
a.f is one-to-one;
b. f^ : E ! f (E) is invertible;
c. f is continuous.

20.2.2 Correspondences
To solve the following exercises on correspondences, we need some preliminary de…nitions.1
A set C Rn is convex if 8x1 , x2 2 C and 8 2 [0; 1], (1 ) x1 + x2 2 C.
A set C Rn is strictly convex if 8x1 , x2 2 C and 8 2 (0; 1), (1 ) x1 + x2 2 Int C.
Consider an open and convex set X Rn and a continuous function f : X ! R; f is quasi-concave i¤ 8
x0 ; x00 2 X; 8 2 [0; 1],
f ((1 )x0 + x00 ) min ff (x0 ) ; f (x00 )g :
f is strictly quasi-concave

De…nition 796 i¤ 8 x0 ; x00 2 X; such that x0 6= x00 ; and 8 2 (0; 1), we have that

f ((1 )x0 + x00 ) > min ff (x0 ) ; f (x00 )g :

We de…ne the budget correspondence as

De…nition 797
: RC
++ R++ !! RC ; (p; w) = x 2 RC
+ : px w :

The Utility Maximization Problem (U M P ) is

De…nition 798
maxx2RC+ u (x) s:t: px w; or x 2 (p; w)

: RC
++ R++ !! RC ; (p; w) = arg max (U M P ) is the demand correspondence.

The Pro…t Maximization Problem (PMP) is

max py s:t: y 2 Y:
y

De…nition 799 The supply correspondence is

y : RC C
++ !! R ; y (p) = arg max(P M P ):

We can now solve some exercises. (the numbering has to be changed)

1.
Show that is non-empty valued.
2.
Show that for every (p; w) 2 RC ++ R++ ;
(a) if u is quasiconcave, is convex valued;
(b) if u is strictly quasiconcave, is single valued, i.e., it is a function.
3.
Show that is closed.
4.
If a solution to (PMP) exists, show the following properties hold.
(a) If Y is convex, y (:) is convex valued;
(b) If Y is strictly convex (i.e., 8 2 (0; 1) ; y := (1 ) y 0 + y 00 2 Int Y ), y (:) is single valued.
5
1 The de…nition of quasi-concavity and strict quasi-concavity will be studied in detail in Chapter .
290 CHAPTER 20. EXERCISES

Consider 1; 2 : [0; 2] !! R;
8
< 1 + 0:25 x; x2 1 if x 2 [0; 1)
1 (x) = [ 1; 1] if x=1 ;
:
1 + 0:25 x; x2 1 if x 2 (1; 2]
and
8
< 1 + 0:25 x; x2 1 if x 2 [0; 1)
2 (x) = [ 0:75; 0:25] if x=1 :
:
1 + 0:25 x; x2 1 if x 2 (1; 2]
Say if 1 and 2 are LHC, UHC, closed, convex valued, compact valued.
6.
Consider : R+ !! R;

sin x1 if x>0
(x) = ;
[ 1; 1] if x=0
Say if is LHC, UHC, closed.
7.
Consider : [0; 1] !! [ 1; 1]

[0; 1] if x 2 Q \ [0; 1]
(x) = :
[ 1; 0] if x 2 [0; 1] nQ
Say if is LHC, UHC, closed.
8.
Consider 1 ; 2 : [0; 3] !! R;

1 (x) = x2 2; x2 ;
and

2 (x) = x2 3; x2 1 ;

3 (x) := ( 1 \ 2 ) (x) := 1 (x) \ 2 (x) :

Say if 1, 2 and 3 are LHC, UHC, closed.

20.3 Di¤erential Calculus in Euclidean Spaces

1.
Using the de…nition, compute the partial derivative of the following function in an arbitrary point (x0 ; y0 ) :

f : R2 ! R; f (x; y) = 2x2 xy + y 2 :

2.
If possible, compute partial derivatives of the following functions.
a. f (x; y) = x arctan xy ;
b. f (x; y) = xy ; p
x+y
c. f (x; y) = (sin (x + y)) in (0; 3)

3,
Given the function f : R2 ! R,
8 xy
< x2 +y 2 if x2 + y 2 6= 0
f (x; y) = ;
:
0 otherwise
20.3. DIFFERENTIAL CALCULUS IN EUCLIDEAN SPACES 291

show that it admits both partial derivatives in (0; 0) and it is not continuous in (0; 0).

4.
Using the de…nition, compute the directional derivative f 0 ((1; 1) ; ( 1; 2 )) with 1; 2 6= 0 for f : R2 !
R,
x+y
f (x; y) = :
x2 + y 2 + 1

5.
Let the following function be given.
8 x2 y 2
< x3 +y 3 if x 6= 0
f : R2 ! R; (x; y) 7!
:
y if x = 0:

Using the de…nition of a directional derivative, compute, if possible f 0 (0; u) for every u 2 R2 :

6.
Using the de…nition, show that the following functions are di¤erentiable.
a. l 2 L (Rn ; Rm );
n
b. the projection function f : Rn ! R, f : xi i=1 7! x1 .
7.
Show the following result which was used in the proof of Proposition 620. A linear function l : Rn ! Rm
is continuous.

8.
Compute the Jacobian matrix of f : R2 ! R3 ;

f (x; y) = (sin x cos y; sin x sin y; cos x cos y)

9.
Given di¤erentiable functions g; h : R ! R and y 2 Rn f0g, compute the Jacobian matrix of f : R3 ! R3 ;
g(h(x))
f (x; y; z) = g (x) h (z) ; y ; ex g(h(x))

10 .
Compute total derivative and directional derivative at x0 in the direction u.
a.
1 1 1
f : R3++ ! R; f (x1 ; x2 ; x3 ) = log x1 + log x2 + log x3
3 6 2
x0 = (1; 1; 2), u = p1 (1; 1; 1);
3
b.

f : R3 ! R; f (x1 ; x2 ; x3 ) = x21 + 2x22 x23 2x1 x2 6x2 x3

x0 = (1; 0; 1), u = p1 ; 0; p1 ;
2 2
c.

f : R2 ! R; f (x1 ; x2 ) = x1 ex1 x2
x0 = (0; 0), u = (2; 3).

11 .
Given 1
f (x; y; z) = x2 + y 2 + z 2 2
;
292 CHAPTER 20. EXERCISES

show that if (x; y; z) 6= 0, then

@ 2 f (x; y; z) @ 2 f (x; y; z) @ 2 f (x; y; z)

+ + =0
@x2 @y 2 @z 2

12 .
Given the C 2 functions g; h : R ! R++ , compute the Jacobian matrix of
g(x)
f : R3 ! R3 ; f (x; y; z) = h(z) ; g (h (x)) + xy; ln (g (x) + h (x))

13 .
Given the functions
ex + y
f : R2 ! R2 ; f (x; y) =
ey + x
g : R ! R; x 7! g (x)
h : R ! R2 ; h (x) = f (x; g (x))
2
Assume that g is C . a. compute the di¤erential of h in 0; b. check the conclusion of the Chain Rule.

14 .
Let the following di¤erentiable functions be given.

f : R3 ! R (x1 ; x2 ; x3 ) 7! f (x1 ; x2 ; x3 )

g : R3 ! R (x1 ; x2 ; x3 ) 7! g (x1 ; x2 ; x3 )
0 1
f (x1 ; x2 ; x3 )
a : R3 ! R3 ; (x1 ; x2 ; x3 ) 7! @ g (x1 ; x2 ; x3 ) A
x1

g (y1 ; y2 ; y3 )
b : R3 ! R2 ; (y1 ; y2 ; y3 ) 7!
f (y1 ; y2 ; y3 )

Compute the directional derivative of the function b a in the point (0; 0; 0) in the direction (1; 1; 1).

15 .
Using the theorems of Chapter 16, show that the function in (??) is di¤erentiable.

16 .
Given
f : R3 ! R1 ; (x; y; z) 7! z + x + y 3 + 2x2 y 2 + 3xyz + z 3 9;
say if you can apply the Implicit Function Theorem to the function in (x0 ; y0 ; z0 ) = (1; 1; 1) and, if
@y
possible, compute @x
@z and @z in (1; 1; 1).

17 .
Using the notation of the statement of the Implicit Function Theorem presented in the Class Notes, say
if that Theorem can be applied to the cases described below; if it can be applied, compute the Jacobian of
g. (Assume that a solution to the system f (x; t) = 0 does exist).
a. f : R4 ! R2 ;
x21 x22 + 2t1 + 3t2
f (x1 ; x2 ; t1 ; t2 ) 7!
x1 x2 + t1 t2
b. f : R4 ! R2 ;
2x1 x2 + t1 + t22
f (x1 ; x2 ; t1 ; t2 ) 7!
x21 + x22 + t21 2t1 t2 + t22
20.4. NONLINEAR PROGRAMMING 293

c. f : R4 ! R2 ;
t21 t22 + 2x1 + 3x2
f (x1 ; x2 ; t1 ; t2 ) 7!
t1 t2 + x1 x2

18.
Say under which conditions, if z 3 xz y = 0, then

@2z 3z 2 + x
= 3
@x@y (3z 2 x)

19. Do Exercise 685: Let the utility function u : R2++ ! R++ ; (x; y) 7! u (x; y) be given. Assume
that it satis…es the following properties i. u is C 2 , ii. 8 (x; y) 2 R2++ ; Du (x; y) >> 0;iii. 8 (x; y) 2 R2++ ;
Dxx u (x; y) < 0; Dyy u (x; y) < 0; Dxy u (x; y) > 0. Compute the Marginal Rate of Substitution in (x0 ; y0 )
and say if the graph of each indi¤erence curve is concave.
20.
Let the function f; g : Rn ! R be given and assume that

for every x0 2 Rn and every u 2 Rn , the directional derivatives f 0 (x0 ; u) and g 0 (x0 ; u) do exist.

De…ne h : Rn ! R; x 7! f (x) g (x). If possible, compute h0 (x0 ; u) for every x0 2 Rn and every u 2 Rn .
21.
A function f : R2 ! R; x 7! f (x) is homogenous of degree n 2 N+ if

for every x = (x1 ; x2 ) 2 R2 and every a 2 R+ , f (ax1 ; ax2 ) = an f (x1 ; x2 ) :

Show that if f is homogenous of degree n and f 2 C 1 R2 ; R , then

for every x = (x1 ; x2 ) 2 R2 ; x1 Dx1 f (x1 ; x2 ) + x2 Dx2 f (x1 ; x2 ) = nf (x1 ; x2 ) :

20.4 Nonlinear Programming

1. 2 Determine, if possible, the nonnegative parameter values for which the following functions f : X !
n
R; f : (xi )i=1 := x 7! f (x) are concave, pseudo-concave, quasi-concave, strictly concave.
(a) X = R++ ; f (x) = x ;
Pn
(b) X = Rn++ ; n 2; f (x) = i=1 i (xi ) i (for pseudo-concavity and quasi-concavity consider only
the case n = 2).
(c) X = R; f (x) = min f ; x g:

2.
a. Discuss the following problem. For given 2 (0; 1), a 2 (0; +1),
1
max(x1 ;x2 ) u (x) + (1 ) u (y) s.t. y a 2x
y 2a 2x
x 0
y 0

where u : R ! R is a C 2 function such that 8z 2 R, u0 (z) > 0 and u00 (z) < 0.
b. Say if there exist values of ( ; a) such that (x; y; 1 ; 2 ; 3 ; 4 ) = 32 a; 23 a; 1 ; 0; 0; 0 , with 1 > 0; is a
solution to Kuhn-Tucker conditions, where for j 2 f1; 2; 3; 4g, j the multiplier associated with constraint j.
c. “Assuming”that the …rst, third and fourth constraint hold with a strict inequality, and the multiplier
associated with the second constraint is strictly positive, describe in detail how to compute the e¤ect of a
change of a or on a solution of the problem.

3.
2 Exercise 1 is taken from David Cass’problem sets for his Microeconomics course at the University of Pennsylvania.
294 CHAPTER 20. EXERCISES

a. Discuss the following problem. For given 2 (0; 1) ; w1 ; w2 2 R++ ;

max(x;y;m)2R2++ R log x + (1 ) log y s:t

w1 m x 0
w2 + m y 0
b. Compute the e¤ect of a change of w1 on the component x of the solution.
c. Compute the e¤ect of a change of on the objective function computed at the solution of the problem.

4.
a. Discuss the following problem.
min(x;y)2R2 x2 + y 2 4x 6y s:t: x+y 6
y 2
x 0
y 0
Let (x ; y ) be a solution to the the problem.
b. Can it be x = 0 ?
c. Can it be (x ; y ) = (2; 2) ?.

5.
Characterize the solutions to the following problems.
(a) (consumption-investment)

max(c1 ;c2 ;k)2R3 u (c1 ) + u (c2 )

s:t:
c1 + k e
c2 f (k)
c1 ; c2 ; k 0;
where u : R ! R; u0 > 0; u00 < 0; f : R+ ! R; f 0 > 0; f 00 < 0 and such that f (0) = 0; 2 (0; 1) ; e 2
R++ : After having written Kuhn Tucker conditions, consider just the case in which c1 ; c2 ; k > 0:
(b) (labor-leisure)

max(x;l)2R2 u (x; l)
s:t:
px + wl wl
l l
x; l 0;
where u : R2 is C 2 ; 8 (x; l) Du (x; l) 0; u is di¤erentiably strictly quasi-concave, i.e.,8 (x; l) ; if 6= 0
and Du (x; l) = 0, then T D2 u < 0; p > 0; w > 0 and l > 0:
Describe solutions for which x > 0 and 0 < l < l,

6.
(a) Consider the model described in Exercise 6: (a) : What would be the e¤ect on consumption (c1 ; c2 ) of
an increase in initial endowment e?
What would be the e¤ect on (the value of the objective function computed at the solution of the problem)
of an increase in initial endowment e?
Assume that f (k) = ak ; with a 2 R++ and 2 (0; 1) : What would be the e¤ect on consumption
(c1 ; c2 ) of an increase in a?
(b) Consider the model described in Exercise 6: (b) :What would be the e¤ect on leisure l of an increase
in the wage rate w? in the price level p?
What would be the e¤ect on (the value of the objective function computed at the solution of the problem)
of an increase in the wage rate w? in the price level p?:
7.
Show that if f : R2 ! R; x 7! f (x) is homogenous of degree 1, then
f is concave , for any x; y 2 R2 ; f (x + y) f (x) + f (y) :
Chapter 21

Solutions

21.1 Linear Algebra

1.
We want to prove that the following sets are not vector spaces. Thus, it is enough to …nd a counter-
example violating one of the conditions de…ning vector spaces.
(i) The de…nition violates the so-called M2 distributive assumption since for k = 1 and a = b = 1,

(1 + 1) (1; 1) = 2 (1; 1) = (2; 1) : while : 1 (1; 1) + 1 (1; 1) = (2; 2)

(ii) The de…nition violates the so-called A4 commutative property since for a = b = c = 1 and d = 0,

(1; 1) + (1; 0) = (2; 1) 6= (1; 0) + (1; 1) = (2; 0)

2.
(i) Take any w := (x; y; z) 2 W with z > 0, and 2 R with < 0; then w = ( x; y; z) with z < 0
and therefore w 2 = W:
(ii) Take any nonzero w 2 W and de…ne = 2=jjwjj. Observe that jj wjj = 2 > 1 and therefore w 2 = W:
(iii) Multiplication of any nonzero element of Q3 by 2 R n Q will give an element of R3 n Q3 instead of
Q3 .
3.
We use Proposition 139. Therefore, we have to check that
a. 0 2 W ; b. 8u; v 2 W; 8 ; 2 F; u + v 2 W .
De…ne simply by 0 the function f : R ! R such that 8x 2 R, f (x) = 0.
(i) a. Since 0 (1) = 0, 0 2 W .
b.
u (1) + v (1) = 0+ 0 = 0;
where the …rst equality follows from the assumption that u; v 2 W . Then, indeed, u + v 2 W .
(ii) a. Since 0 (1) = 0 = 0 (2), we have that 0 2 W .
b.
u (1) + v (1) = u (2) + v (2) ;
where the equality follows from the assumption that u; v 2 W and therefore u (1) = u (2) and v (1) = v (2).
4.
Again we use Proposition 139 and we have to check that
a. 0 2 W ; b. 8u; v 2 W; 8 ; 2 F; u + v 2 W .
a. (0; 0; 0) 2 W simply because 0 + 0 + 0 = 0.
b. Given u = (u1 ; u2 ; u3 ) ; v = (v1 ; v2 ; v3 ) 2 V , i.e., such that

u1 + u2 + u3 = 0 and v1 + v2 + v3 = 0; (21.1)

295
296 CHAPTER 21. SOLUTIONS

we have
u + v = ( u 1 + v1 ; u 2 + v2 ; u 3 + v3 ) :
Then,
(21:1)
u 1 + v1 + u 2 + v2 + u 3 + v3 = (u1 + u2 + u3 ) + (v1 + v2 + v3 ) = 0:

(ii) We have to check that a. S is linearly independent and b. span S = V .

a.
(1; 1; 0) + (0; 1; 1) = + =0
implies that = = 0.
b. Taken (x1 ; x2 ; x3 ) 2 V , we want to …nd ; 2 R such that (x1 ; x2 ; x3 ) = (1; 1; 0) + (0; 1; 1) =
+ , i.e., we want to …nd ; 2 R such that
8
>
> x1 =
<
x2 = +
>
> x3 =
:
x1 + x2 + x3 = 0
8
< x2 x3 =
x2 = +
:
x3 =
Then, = x2 x3 ; = x3 is the (unique) solution to the above system.

5.
1. 0 2 M (n; n) : A0 = 0A = 0.
2. 8 ; 2 R and 8B; B 0 2 CA ,

( B + B 0 ) A = BA + B 0 A = AB + AB 0 = A B + A B 0 = A ( B + B 0 ) :

6.
i. 0 2 U + V , because 0 2 U and 0 2 V:
ii. Take ; 2 F and w1 ; w2 2 U +V . Then there exists u1 ; u2 2 U and v 1 ; v 2 2 V such that w1 = u1 +v 1
and w2 = u2 + v 2 . Therefore,

w1 + w2 = u1 + v 1 + u2 + v 2 = u1 + u2 + v 1 + v 2 2 U + V;

because U and V are vector spaces and therefore u1 + v 1 2 U and u2 + v 2 2 V .

7. P3 P3
We want to show that if i=1 i vi = 0; then i = 0 for all i. Note that i=1 i vi = ( 1 ; 1 + 2 ; 1 +
2 + 3 ) = 0, which implies the desired result.
8.
n
We want to apply De…nition ??: Consider a vector space V of dimension n and two bases V = v i i=1
n
and U = uk k=1 of V . Then,

P = u1 V
::: uk V
::: [un ]V 2 M (n; n) ;

is called the change-of-basis matrix from the basis V to the basis U. Then in our case, the change-of-basis
matrix from S to E is
P = e1 S e2 S 2 M (2; 2) :
Moreover,using also Proposition ??, the change-of-basis matrix from S to E is
1
Q= v1 E
v2 E
=P :

Computation of e1 S : We want to …nd and such that e1 = u1 + u2 , i.e., (1; 0) = (1; 2)+ (3; 5) =
+3 2 +5 ; i.e., 8
< +3 = 1
:
2 +5 = 0
21.1. LINEAR ALGEBRA 297

whose solution is = 5; = 2:
Computation of e2 S : We want to …nd and such that e2 = u1 + u2 , i.e., (0; 1) = (1; 2)+ (3; 5) =
+3 2 +5 ; i.e., 8
< +3 = 0
:
2 +5 = 1
+3 =0
2 +5 =1
whose solution is = 3; = 1: Therefore,

5 3
P = :
2 1

Since E is the canonical basis, we have

S = fu1 = (1; 2); u2 = (3; 5)g

1 3
Q= :
2 5
Finally
1
1 3 5 3
=
2 5 2 1
as desired.
9.
Easiest way: use row and column operations to change C to a triangular matrix.
2 3 2 3
6 2 1 0 5 1 1 2 2 3
62 1 1 2 1 7 62 1 1 2 17
6 7 6 7
det C = det 6
6 1 1 2 2 3 7 = R1 $ R3 = det 6 6
7 6 2 1 0 577
43 0 2 3 15 43 0 2 3 15
1 1 3 4 2 1 1 3 4 2
2 3 2 3
1 1 2 2 3 2 1 2 2
3 1 1 2 2 3
62 2R + R ! R
6 1 1 2 177 6
60 1 3 2 57
6R1 + R3 ! R3 7 6 7
= det 6
66 2 1 0 577=4
6 7= det 6
60 4 11 12 137
7
43 3R1 + R4 ! R4 5
0 2 3 15 40 3 4 9 105
R1 + R5 ! R5
1 1 3 4 2 0 0 1 2 5
2 3 2 3
1 1 2 2 3 1 1 2 2 3
60 1 3 2 57 60 1 3 2 57
6 7 2
4R + R ! R 3 3 6 7
= det 660 4 11 12 137
7= = det 6
60 0 1 4 777
40 3R2 + R4 ! R4
3 4 9 105 40 0 5 3 55
0 0 1 2 5 0 0 1 2 5
2 3 2 3
1 1 2 2 3 1 1 2 2 3
60 1 3 2 57 60 1 3 2 57
6 7 5R3 + R4 ! R4 6 7
= det 6
60 0 1 4 777= = det 6
60 0 1 4 7 77
40 R3 + R5 ! R5
0 5 3 55 40 0 0 17 305
0 0 1 2 5 0 0 0 6 12

2 3 2 3
1 1 2 2 3 1 1 2 2 3
60 1 3 2 57 60 1 3 2 57
6 7 6 4 6 7
= det 6
60 0 1 4 7 7 5 5
7 = 17 R + R ! R = det 6
60 0 1 4 7 7
7
40 0 0 17 305 40 0 0 17 305
24
0 0 0 6 12 0 0 0 0 17
298 CHAPTER 21. SOLUTIONS

2 3
1 1 2 2 3
60 1 3 2 57
6 7 24
= det 6
60 0 1 4 7 7
7= (1)( 1)(1)( 17)( ) =) det C = 24
40 17
0 0 17 305
24
0 0 0 0 17

10.
Observe that it cannot be 4 as rank(A) minf#rows; #collumsg. It’s easy to check that rank(A) = 3
by using elementary operations on rows and columns of A :
2R3 + R1 ! R1 , C 4 + C 1 ! C 1 , C 4 + C 3 ! C 3 , C 1 + C 3 ! C 3 , C 2 + C 3 ! C 3 ,
to get
2 3
k 1 1 0 0
4 0 0 1 k5
0 0 0 1
which has the last three columns independent for any k.
11.
De…ned
l : R3 ! R; (x1 ; x2 ; x3 ) 7! x1 x2 ;
it is easy to check that l is linear and V = ker l;a vector space. Moreover,

[l] = 1 1 0 :

Therefore, dim ker l = 3 rank [l] = 3 1 = 2: u1 = (1; 1; 0) and u2 = (0; 0; 1) are independent elements
of V . Therefore, from Remark 189, u1 ; u2 are a basis for V .
12.
Linearity is easy. By de…nition, l is linear if 8u; v 2 R4 and 8 ; 2 R; l( u + v) = l(u) + l(v): Then,

l(u) + l(v) =
= (u1 ;u1 +u2 ;u1 +u2 +u3 ;u1 +u2 +u3 +u4 )++ (v1 ;v1 +v2 ;v1 +v2 +v3 ;v1 +v2 +v3 +v4 ) =

=( u1 + v1 ; u1 + u2 + v1 + v2 ; u1 + u2 + u3 + v1 + v2 + v3 ; u1 + u2 + u3 + u4 + v1 + v2 + v3 + v4 ) =

= l( u + v)

and then l is linear.

2 3
1 0 0 0
61 1 0 07
[l] = 6
41
7:
1 1 05
1 1 1 1
Therefore, dim ker l = 4 rank [l] = 4 4 = 0: Moreover, dim Im l = 4 and the column vectors of [l] are
a basis of Im l.
13.
Proposition. Assume that l 2 L (V; U ) and ker l = f0g. Then,

8u 2 Iml; there exists a unique v 2 V such that l (v) = u:

Proof.
Since u 2 Iml, by de…nition, there exists v 2 V such that

l (v) = u: (21.2)

Take v 0 2 V such that l (v 0 ) = u. We want to show that

v = v0 : (21.3)
21.1. LINEAR ALGEBRA 299

Observe that
(a)
l (v) l (v 0 ) = u u = 0; (21.4)
where (a) follows from (21:2) and (21:3).
Moreover,
(b)
l (v) l (v 0 ) = l (v v0 ) ; (21.5)
where (b) follows from the assumption that l 2 L (V; U ).
Therefore,
l (v v 0 ) = 0;
and, by de…nition of ker l,
v v 0 2 ker l: (21.6)
Since, by assumption, ker l = f0g, from (21:6), it follows that

v v 0 = 0:

14.
Both V and W are ker of linear function; therefore V , W and V \W are vector subspaces of R4 . Moreover
8 8 9
< < x1 x2 + x3 x4 = 0 =
V \ W = (x1 ; x2 ; x3 ; x4 ) 2 R4 :
: : ;
x1 + x2 + x3 + x4 = 0

1 1 1 1
rank =2
1 1 1 1
Therefore, dim ker l = dim V \ W = 4 2 = 2.
Let’s compute a basis of V \ W :
8
< x1 x2 = x3 + x4
:
x1 + x2 = x3 x4
After taking sum and subtraction we get following expression
8
< x1 = x3
:
x2 = x4

A basis consists two linearly independent vectors. For example,

f( 1; 0; 1; 0) ; (0; 1; 0; 0)g :
15.
By proposition 139, we have to show that
1. 0 2 l 1 (W ) ;
2. 8 ; 2 R and v 1 ; v 2 2 l 1 (W ) we have that v 1 + v 2 2 l 1 (W ).
1.
l2L(V;U ) W vector space
l (0) = 0 2 W
,
2.Since v 1 ; v 2 2 l 1
(W ),
l v 1 ; l v 2 2 W: (21.7)
Then
l2L(V;U ) (a)
l v1 + v2 = l v1 + l v2 2 W
where (a) follows from (21:7) and the fact that W is a vector space.

16.
300 CHAPTER 21. SOLUTIONS

Observe that 2 3
a11 a12 0 0 0
6 a21 a22 0 0 0 7
6 7
det 6
6 5 6 b11 b12 0 7 = det A det B k:
7
4 7 8 b21 b22 0 5
a11 a12 0 0 k
Then, if k 6= 0;then the rank of both matrix of coe¢ cients and augmented matrix is 5 and the set of
solution to the system is an a¢ ne subspace of R6 of dimension 1: If k = 0;then the system is
2 3
2 3 x 2 3
1 a11 a12 0 0 0 6 1 7 0
6 2 a21 a22 0 0 0 7 6 x2 7 6 1 7
6 7 6 x3 7 6 7
6 3 5 6 b11 b12 0 7 6 7 6 7
6 7 6 x4 7 = 6 2 7 ;
4 4 7 8 b21 b22 0 5 6 4 x5 5
7 4 3 5
1 a11 a12 0 0 0 0
x6
which is equivalent to the system
2 3
2 3 x1 2 3
1 a11 a12 0 0 0 6 6 x2 7
7 0
6 2 a21 a22 0 0 0 7 6 x3 7 6 1 7
6 76 7=6 7;
4 3 5 6 b11 b12 0 6
5
6 x4 7 4
7 2 5
4 7 8 b21 b22 0 4 x5 5 3
x6

whose set of solution is an a¢ ne subspace of R6 of dimension 2:

17.

U 1 0 1 1 1 3
[l]V := l ; l = ; = ;
0 U
1 U
1 U
1 U
0 2

3
[v]V =
4

7 9
[l (v)]U = =
1 U
8

U 1 3 3 9
[l]V [v]V = =
0 2 4 8
18.
Let
n; m 2 N such that m > n; and
a vector subspace L of Rm such that dim L = n
be given. Then, there exists l 2 L (Rn ; Rm ) such that Im l = L.
n
Proof. Let v i i=1 be a basis of L Rm . Take l 2 L (Rn ; Rm ) such that

8i 2 f1; :::; ng ; l2 ein = v i ;

where ein is the i–th element in the canonical basis in Rn . Such function does exists and, in fact, it is
unique as a consequence of a Proposition in the Class Notes that we copy below:
Let V and U be …nite dimensional vectors spaces such that S = v 1 ; :::; v n is a basis of V and u1 ; :::; un
is a set of arbitrary vectors in U . Then there exists a unique linear function l : V ! U such that 8i 2 f1; :::; ng,
l v i = ui - see Proposition 273, page 82.
Then, from the Dimension theorem

dim Iml = n dim ker l n:

n
Moreover, L = span v i i=1
Iml. Summarizing,

L Im l , dim L = n and dim Iml n;

21.1. LINEAR ALGEBRA 301

and therefore
dim Iml = n:
Finally, from Proposition 179 in the class Notes since L Im l , dim L = n and dim Iml = n; we have
that Iml = L, as desired.
Proposition 179 in the class Notes says what follows: Proposition. Let W be a subspace of an n dimensional
vector space V . Then, 1. dim W n; 2. If dim W = n, then W = V .

19.
2 3 2 3
a 1 1 1 1 a
61 1 a7 6 7
[A j b] = 6 7 =) R1 $ R2 =) 6a 1 1 7
42 1 3a5 42 1 3a5
32 a 3 2 a
2 3 2 3
1 1 a 1 1 a
6a aR1 + R2 ! R2
6 1 177 =)
60 1 a 1 a2 7
42 2R1 + R3 ! R3 =) 6 7 := [A0 (a) j b0 (a)]
1 3a5 40 1 a 5
3R1 + R4 ! R4
3 2 a 0 1 2a
Since 2 3
1 1 a
det 40 1 a 5 = 3a;
0 1 2a
We have that if a 6= 0, then rank A 2 < 3 = rank [A0 (a) j b0 (a)], and the system has no solutions. If
a = 0, [A0 (a) j b0 (a)] becomes 2 3
1 1 0
60 1 17
6 7
40 1 05
0 1 0
whose rank is 3 and again the system has no solutions.

20.
2 3
1 0 k 1
6 1 k 2 k k 7
[A (k) jb (k)] 6 7
4 1 k 1 5
1 k 1 0
2 3
1 0 k 1
det 4 1 k 1 5=2 2k
1 k 1 0
If k 6= 1, the system has no solutions. If k = 1;
2 3
1 0 0
6 0 1 1 7
[A (1) jb (1)] 6 7
4 1 1 1 5
1 0 0
2 3
1 0 0
4 0 1 1 5
1 1 1
1 0 0
0 1 1
Then, if k = 1, there exists a unique solution.
21.
The following Proposition is contained in the class Notes.
302 CHAPTER 21. SOLUTIONS

Proposition 800 Let V be a vector space of dimension n.

1. If S = u1 ; :::; un V is a linearly independent set, then it is a basis of V ;

2. If span u1 ; :::; un = V , then u1 ; :::; un is a basis of V:

n
From that Proposition, it su¢ ces to show that V is linearly independent, i.e., given ( i )i=1 2 Rn ; if

n
X
i
iv = 0 (21.8)
i=1

then
n
( i )i=1 = 0:

Now, for any j 2 f1; :::; ng, we have

n
! n
(1) X (2) X
i i j (3)
0 = iv vj = iv v = j;
i=1 i=1

where (1) follows from (21:8) ;

(2) follows from properties of the scalar product;
(3) follows from (20:7) :
22.
a. Let w 2 W . By assumption, S(w) 2 W and T (w) 2 W and since W is a subspace, S(w) + T (w) 2 W 1 .
Therefore, (S + T )(w) = S(w) + T (w) 2 W , as desired.
b. Let w 2 W . By assumption, T (w) 2 W . Then (S T )(w) = S(T (w)) 2 W since W is S-invariant.
c. Let w 2 W . By assumption, (kT )(w) = kT (w) 2 W .
23.
The set of 2 2 symmetric real matrices is

S = A 2 M (2; 2) : A = AT :

We want to show that

i. 0 2 S and
ii. for any ; 2 R, for any A; B 2 S, A + B 2 S.
i. 0 = 0T
T
2.( A + B) = AT + B T = A + B:
We want to show that
1 0 0 0 0 1
B := ; ;
0 0 0 1 1 0
is a basis of S and therefore dim S = 3. B is clearly linearly independent. Moreover,

a b
S= A 2 M (2; 2) : 9a; b; c 2 R such that A = ;
b c
and
a b 1 0 0 0 0 1
=a +c +b ;
b c 0 0 0 1 1 0
i.e., span (B)) = S, as desired.
24.
a. [ ] Taken w 2 W;we want to …nd w1 ; w2 2 W such that w = w1 + w2 :take w1 = w and w2 = 0 2 W:
[ ] Take w1 ; w2 2 W . Then w1 + w2 2 W by de…nition of vector space.
b. [ ] Let w 2 W . Then 1 w 2 W and 1
w w 2 W.
[ ] It follows from the de…nition of vector space.
25.
1 see Proposition 138 in Villanacci(20 September, 2012)
21.1. LINEAR ALGEBRA 303

The isomorphism is ' : Pn (R) ! Rn+1

n
X n
f (t) = ai ti 7! (ai )i=0 :
i=0

Indeed , ' is linear

...
and de…ned : Rn+1 ! Pn (R),
n
X
n
(ai )i=0 ! f (t) = ai ti ;
i=0

we have that ' = idjPn (R) and ' = idjRn+1

....
304 CHAPTER 21. SOLUTIONS

21.2 Some topology in metric spaces

21.2.1 Basic topology in metric spaces
1.
To prove that d0 is a metric, we have to check the properties listed in De…nition 327.
a. d0 (x; y) 0; d0 (x; y) = 0 , x = y
By de…nition of d0 (x; y); it is always going to be positive as d(x; y) 0. Furthermore, d0 (x; y) = 0 ()
d(x; y) = 0 () x = y.
b. d0 (x; y) = d0 (y; x)
Applying the de…nition

d(x; y) d(y; x)
d0 (x; y) = d0 (y; x) () =
1 + d(x; y) 1 + d(y; x)
but d(x; y) = d(y; x) so we have

d(x; y) d(x; y)
=
1 + d(x; y) 1 + d(x; y)
c. d0 (x; z) d0 (x; y) + d0 (y; z)
Applying the de…nition

d(x; z) d(x; y) d(y; z)

d0 (x; z) d0 (x; y) + d0 (y; z) () +
1 + d(x; z) 1 + d(x; y) 1 + d(y; z)
Multiplying both sides by [1 + d(x; z)][1 + d(x; y)][1 + d(y; z)]

d(x; z)[1 + d(x; y)][1 + d(y; z)] d(x; y)[1 + d(x; z)][1 + d(y; z)] + d(y; z)[1 + d(x; z)][1 + d(x; y)]

Simplifying we obtain

d(x; z) d(x; y) + d(y; z) + [[1 + d(x; z)][1 + d(x; y)][1 + d(y; z)] + 2[1 + d(x; y)][1 + d(y; z)]]

which concludes the proof.

2.
It is enough to show that one of the properties de…ning a metric does not hold.
It can be d (f; g) = 0 and f 6= g. Take

f (x) = 0; 8x 2 [0; 1] ;

and
g (x) = 2x + 1
Then,
Z 1
( 2x + 1) dx = 0:
0

It can be d (f; g) < 0:Consider the null function and the function that take value 1 for all x in [0; 1]. Then
R1
d(0; 1) = 0
1 dx: by linearity of the Riemann integral, which is equal to 1. Then, d(0; 1) < 0.

3.
De…ne S = (a1 ; b1 ) (a2 ; b2 ) and take x0 := x01 ; x02 2 S. Then, for i 2 f1; 2g, there exist "i > 0 such
that x0i 2 B x0i ; "i (ai ; bi ). Take " = min f"1 ; "2 g. Then, for i 2 f1; 2g, x0i 2 B x0i ; " (ai ; bi ) and,
0
de…ned B =.B x1 ; " B x02 ; " , we have that x0 2 B S. It then su¢ ces to show that B x0 ; " B.
Observe that
21.2. SOME TOPOLOGY IN METRIC SPACES 305

x 2 B x0 ; " , d x; x0 < ";

q
2
d x01 ; 0 ; (x1 ; 0) = (x01 x1 ) = x01 x1 ;
and
q q
2 2
d x01 ; 0 ; (x1 ; 0) = (x01 x1 ) (x01 x1 ) + (x01 x1 ) = d x; x0 :

4.
Show the second equality in Remark 366:

1 1
\+1
n=1 ; = f0g
n n

5.

1 1 1 1 1
S= 1; + ; ; ; ; ; :::
2 3 4 5 6
The set is not open: it su¢ ces to …nd x 2 S and such that x 2
= Int S; take for example 1. We want to
show that it false that
9" > 0 such that ( 1 "; 1 + ") S:
In fact, 8" > 0; 1 2" 2 ( 1 "; 1 + "), but 1 2" 2 = S. The set is not closed. It su¢ ces to show
that F (S) is not contained in S, in fact that 0 2= S (obvious) and 0 2 F (S). We want to show that 8" > 0,
n n
B (0; ") \ S 6= ?:In fact, ( 1) n1 2 B (0; ") if n is even and ( 1) n1 = n1 < ". It is then enough to take n
even and n > 1" .

6.

A = (0; 10)
The set is (R; d2 ) open, as a union of in…nite collection of open sets. The set is not closed, because Ac is
not open. 10 or 0 do not belongs to Int(AC )

7.
The solution immediately follow from De…nition of boundary of a set: Let a metric space (X; d) and a
set S X be given. x is an boundary point of S if
any open ball centered in x intersects both S and its complement in X, i.e., 8r 2 R++ ; B (x; r) \ S 6= ?
^ B (x; r) \ S C 6= ?:
As you can see nothing changes in de…nition above if you replace the set with its complement.
8.

C
x 2 (F (S)) ,x2 = (F (S))
, : 8r 2 R++ ; B (x; r) \ S 6= ? ^ B (x; r) \ S C 6= ?
, 9r 2 R++ such that B (x; r) \ S = ? _ B (x; r) \ S C 6= ?
, 9r 2 R++ such that B (x; r) S C _ B (x; r) S (21.9)
, x 2 Int S C _ x 2 Int S
(1)
, 9rx 2 R++ such that either a. B (x; rx ) Int S C or b. B (x; rx ) Int S:

where (1) follows from the fact that the Interior of a set is an open set.
C
If case a. in (21:9) holds true, then, using Lemma 461, B (x; rx ) (F (S)) and similarly for case b., as
desired.

9.
306 CHAPTER 21. SOLUTIONS

Int S Cl (S) F (S) D (S) Is (S) open or closed

S=Q ? R R R ? neither open nor closed

S = (0; 1) (0; 1) [0; 1] f0; 1g [0; 1] ? open

1
S= n n2N+
? S [ f0g S [ f0g f0g S neither open nor closed

10.
a.
Take S = N. Then, Int S = ?, Cl (?) = ?, and Cl (Int S) = ? 6= N =S.
b.
Take S = N. Then, Cl (S) = N, Int N = ?, and Int Cl (S) = ? 6= N =S.

11.
a.
True. If S is an open bounded interval, then 9a; b 2 R, a < b such that S = (a; b). Take x 2 S and
= min fjx aj ; jx bjg. Then I (x; ) (a; b).
b.
False. (0; 1) [ (2; 3) is an open set, but it is not an open interval.
c.
False. Take S := f0; 1g. 0 2 F (S), but 0 2 = D (S)
d. :
False. Take S (0; 1) : 21 2 D (S) 0; but 12 2
= F (S) :
12.
Recall that: A sequence (xn )n2N 2 X 1 is said to be (X; d) convergent to x0 2 X (or convergent with
respect to the metric space (X; d) ) if 8" > 0; 9n0 2 N such that 8n > n0 ; d (xn ; x0 ) < ".
a.
(xn )n2N 2 R1 such that 8n 2 N; : xn = 1
Let > 0 then by de…nition of (xn )n2N , 8n > 0, d(xn ; 1) = 0 < . So that

lim xn = 1
n!1

b.
(xn )n2N 2 R1 such that 8n 2 N; : xn = n1
Let > 0. Because N is unbounded, 9n0 2 N , such that n0 > 1 . Then 8n > n0 , d(xn ; 0) = 1
n < 1
n0 < .
Then, by de…nition of a limit, we proved that

lim xn = 0
n!1

13.
1
Take (xn )n2N 2 [0; 1] such that xn ! x0 ; we want to show that x0 2 [0; 1]. Suppose otherwise, i.e.,
x0 2= [0; 1].
Case 1. x0 < 0: By de…nition of convergence, chosen " = x20 > 0, there exists n" 2 N such that 8n > n" ,
d (xn ; x0 ) < "; i.e., jxn x0 j < " = x20 , i.e., x0 + x20 < xn < x0 x20 = x20 < 0. Summarizing, 8n > n" ,
1
xn 2= [0; 1] ; contradicting the assumption that (xn )n2N 2 [0; 1] :
Case 2. x0 > 1. Similar to case 1.

14.
This is Example 7.15, page 150, Morris (2007):
1. In fact, we have the following result: Let (X; d) be a metric space and A = fx1 ; :::; xn g any …nite
subset of X:Then A is compact, as shown below.
Let Oi , i 2 I be any family of open sets such that A [i2I Oi . Then for each xj 2 A, there exists Oij
such that xj 2 Oij . Then A Oi1 [ Oi2 [ ::: [ Oin . Therefore A is compact.
21.2. SOME TOPOLOGY IN METRIC SPACES 307

2. Conversely, let A be compact. Then the family of singleton sets Ox = fxg, x 2 A is such that each Ox is
open and A [x2A Ox . Since A is compact, there exists Ox1 ; Ox2 ; :::; Oxn such that A Ox1 [Ox2 [:::[Oxn ,
that is, A fx1 ; :::; xn g. Hence, A is …nite.

15.
In general it is false. For example in a discrete metric space: see previous exercise.

16.
1
Take an open ball B (x; r). Consider S = B x; r 1 n n2Nnf0;1g
. Observe that S is an open cover
1
of B (x; r); in fact [n2Nnf0;1g B x; r 1 n = B (x; r) ;as shown below.
0 1 1
[ ] x 2 [n2Nnf0;1g B x; r 1 n , 9n Nn f0; 1g such x 2 B x; r 1
x0 nx0 B (x; r).
0 0 0 1 r
[ ] Take x 2 B (x; r). Then, d (x; x ) < r. Take n such that d (x ; x) < r 1 nx0 , i.e., n > r d(x0 ;x)
(and n > 1), then x0 2 B x; r 1 n1 .
Consider an arbitrary subcover of S, i.e.,
1
S0 = B x; r 1
n n2N

with #N = N 2 N. De…ne n = min fn 2 N g. Then [n2N B x; r 1 n1 = B x; r 1 n

1
, and if
= [n2N B x; r 1 n1 .
d (x0 ; x) 2 r 1 n1 ; r , then x0 2 B (x; r) and x0 2

17.
1st proof.
We have to show that f (A [ B) f (A) [ f (B) and f (A) [ f (B) f (A [ B).
To prove the …rst inclusion, take y 2 f (A [ B); then 9x 2 A [ B such that f (x) = y. Then either x 2 A
or x 2 B that implies f (x) = y 2 A or f (x) = y 2 B. In both case y 2 f (A) [ f (B)
We now show the opposite e inclusion. Let y 2 f (A) [ f (B), then y 2 f (A) or y 2 f (B), but y 2 f (A)
implies that 9x 2 A such that f (x) = y. The same implication for y 2 f (B). As results, y = f (x) in either
case with x 2 A [ B i.e. y 2 f (A [ B).
2nd proof.

y 2 f (A [ B) ,
, 9x 2 A [ B such that f (x) = y

, (9x 2 A such that f (x) = y) _ (9x 2 B such that f (x) = y)

, (y 2 f (A)) _ (y 2 f (B))

, y 2 f (A) [ f (B)

18.:
First proof. Take f = sin; A = [ 2 ; 0] ; B = [0; 2 ].
Second proof. Consider
f : f0; 1g ! R; x 7! 1
Then take A = f0g and B = f1g. Then A \ B = ?, so f (A \ B) = ?. But as f (A) = f (B) = f1g, we have
that f (A) \ f (B) = f1g =
6 ?.

19.
Take c 2 R and de…ne the following function

f : X ! Y; f (x) = c:

It su¢ ces to show that the preimage of every open subset of the domain is open in the codomain. The
inverse image of any open set K is either X (if c 2 K) or ? (if c 2
= K), which are both open sets.

20.
308 CHAPTER 21. SOLUTIONS

a.
i. Rn+ is not bounded, then by Proposition 423 it is not compact.
ii. Cl B(x; r) is compact.
From Proposition 423, it su¢ ces to show that the set is closed and bounded.
Cl B(x; r) is closed from Proposition 379.
Cl B(x; r) is bounded because Cl B (x; r) = fy 2 Rn : d (x; y) rg B (x; 2r).
Let’s show in detail the equality.
i. Cl B (x; r) C.
1
Pn 2 2
The function dx : Rn ! R; dx (y) = d (x; y) := i=1 (xi y i ) is continuous. Therefore, C =
dx 1 ([0; r]) is closed. Since B (x; r) C, by de…nition of closure, the desired result follows.
ii. Cl B (x; r) C.
From Corollary 464, it su¢ ces to show that Ad B (x; r) C. If d (y; x) < r, we are done. Suppose
that d (y; x) = r:We want to show that for every " > 0; we have that B (x; r) \ B (y; ") 6= ?:If " > r, then
x 2 B (x; r) \ B (y; "). Now take, " r. It is enough to take a point “very close to y inside B (x; r)”". For
"
example, we can verify that z 2 B (x; r) \ B (y; "), where z = x + 1 2r ) (y x) :Indeed,
" " "
d (x; z) = 1 )d (y; x) = (1 )r = r < r;
2r 2r 2
and
" " "
d (y; z) = d (y; x) = r = < ":
2r 2r 2
c.
See solution to Exercise 5, where it was shown that S is not closed and therefore using Proposition 423,
we can conclude S is not compact.

21.
m
Observe that given for any j 2 f1; :::; mg ; the continuous functions gj : Rn ! R and g = (gj )j=1 , we can
de…ne
C := fx 2 Rn : g (x) 0g :
Then C is closed, because of the following argument:
1 1
C = \m n
j=1 gj ([0; +1)) ; since gj is continuous, and [0; +1) is closed, then gj ([0; +1)) is closed in R ;
then C is closed because intersection of closed sets.

22.
The set is closed, because X = f 1 (f0g) :
The set is not compact: take f as the constant function.

23.
1
LetV = BY (1; "); be an open ball around the value 1 of the codomain, with " < 1. f (V ) = f0g[BX (1; ")
is the union of an open set and a closed set, so is neither open nor closed.

24.
To apply the ExtremePnValue Theorem, we …rst have to check if the function to be maximized is continuous.
Clearly, the function i=1 xi is continuous as is the sum of a¢ ne functions. Therefore, to check for the
existence of solutions for the problems we only have to check for the compactness of the restrictions.
The …rst set is closed, because it is the inverse image of the closed set [0; 1] via the continuous function kk.
The …rst set is bounded as well by de…nition. Therefore the set is compact and the function is continuous,
we can apply Extreme Value theorem. The second set is not closed, therefore it is not compact and Extreme
Value theorem can not be applied. The third set is unbounded, and therefore it is not compact and the
Extreme Value theorem can not be applied.
25.
[(]
Obvious.
[)]
We want to show that l 6= 0 ) l is not bounded, i.e., 8M 2 R++ ; 9x 2 E such that kl (x)kF > M .
21.2. SOME TOPOLOGY IN METRIC SPACES 309

2M
Since l 6= 0; 9y 2 Enf0g such that l(y) 6= 0. De…ne x = kl(x)kF y. Then

2M 2M
kl (y)kF = l x = kl (x)kF = 2M > M;
kl (x)kF F
kl (x)kF

as desired.
26.
a ) b:
Take an arbitrary 2 R; if fx 2 X : f (x) < g = f 1 (( 1; )) = ?, we are done. Otherwise, take
x0 2 f 1 (( 1; )), Then, f (x0 ) := " > 0 and by de…nition of upper semicontinuity , we have

9 > 0 such that d (x x0 ) < ) f (x) < f (x0 ) + " = f (x0 ) + f (x0 ) = ;

i.e., B (x0 ; ) f 1 (( 1; )), i.e., the desired result.

b , c:
fx 2 X : f (x) < g = X n fx 2 X : f (x) g.
b ) a:
We want to show that
1
8x0 2 X; 8" > 0; 9 > 0 such that B (x0 ; ) f (( 1; f (x0 ) + ")) :

But by assumption f 1 (( 1; f (x0 ) + ")) is an open set and contains x0 and therefore the desired result
follows.
27.
Take y 2 fxg + A: Then there exists a 2 A such that y = x + a and since A is open there exists " > 0
such that
a 2 B (a; ") A: (21.10)
We want to show that
i. fxg + B (a; ") is an open ball centered at y = x + a, i.e., fxg + B (a; ") = B (x + a; "), and
ii. B (x + a; ") fxg + A:
i.
Hint
[ ] y 2 fxg + B (a; ") , 9z 2 X such that d (z; a) < " and y = x + z ) d (y; x + a) = d (x + z; x + a) =
d (z; a) < " ) y 2 B (x + a; ").
[ ] y 2 B (x + a; ") , d (y; x + a) < ": Now since y = x + (y x) and d (y x; a) = ky x ak =
ky (x + a)k = d (y; x + a) < ", we get the desired conclusion.
ii.
y 2 B (x + a; ") , ky (x + a)k < ". Since y = x + (y x) and k(y x) ak < ", i.e., y x 2
(21:10)
B (a; ") A, we get the desired result.
28.
By assumption, for any i 2 f1; 2g and for any fxni gn Ki ; there exists xi 2 Ki such that, up to a
subsequence xni ! xi :Take fy n g K1 + K2 = K. Then 8n; y n = xn1 + xn2 with xni 2 Ki ; i = 1; 2:Thus
taking converging subsequences of (xni )n , i 2 f1; 2g ; we get y n ! x1 + x2 2 K as desired.
29.
a. We want to show that 8x1 ; x2 2 E, f (x1 ) = f (x2 ) ) x1 = x2 . Indeed

0 = d(f (x1 ); f (x2 )) = d(x1 ; x2 ) ) x1 = x2 :

b. It follows from a.
c. We want to show that 8x0 2 E; 8" > 0; 9 > 0 such that

d1 (x; x0 ) < ) d2 (f (x); f (x0 )) < ":

Take = ". Then,

d1 (x; x0 ) < " ) d2 (f (x); f (x0 )) < ":
310 CHAPTER 21. SOLUTIONS

21.2.2 Correspondences
1.
Since u is a continuous function, from the Extreme Value Theorem , we are left with showing that for
every (p; w) ; (p; w) is non empty and compact,i.e., is non empty valued and compact valued.
C
w
x= Cpc 2 (p; w) :
c=1
(p; w) is closed because is the intersection of the inverse image of two closed sets via continuous functions.
(p; w) is bounded below by zero.
P 0 0
w w 0 pc xc
(p; w) is bounded above because for every c; xc c 6=c
pc pc ; where the …rst inequality comes
from the fact that px w; and the second inequality from the fact that p 2 RC ++ and x 2 R+ :
C

2.
(a)Consider x0 ; x00 2 (p; w) : We want to show that 8 2 [0; 1] ; x := (1 ) x0 + x00 2 (p; w) :
0 00
Observe that u (x ) = u (x ) := u . From the quasiconcavity of u; we have u x u : We are therefore
left with showing that x 2 (p; w) ; i.e., is convex valued. To see that, simply, observe that px =
(1 ) px0 + px00 (1 ) w + w = w:
(b) Assume otherwise. Following exactly the same argument as above we have x0 ; x00 2 (p; w) ; and
px w: Since u is strictly quasi concave, we also have that u x > u (x0 ) = u (x00 ) := u ; which
0 00
contradicts the fact that x ; x 2 (p; w) :
3.
We want to show that for every (p; w) the following is true. For every sequence f(pn ; wn )gn RC ++ R++
such that
(pn ; wn ) ! (p; w) ; xn 2 (pn ; wn ) ; xn ! x;
it is the case that x 2 (p; w) :
Since xn 2 (pn ; wn ), we have that pn xn wn : Taking limits of both sides, we get px w;i.e.,
x 2 (p; w) :
4.
(a) We want to show that 8y 0 ; y 00 2 y (p) ; 8 2 [0; 1] ; it is the case that y := (1 ) y 0 + y 00 2 y (p) ;
i.e., y 2 Y and 8y 2 Y; py py:
y 2 Y simply because Y is convex.
y 0 ;y 00 2y(p)
py := (1 ) py 0 + py 00 (1 ) py + py = py:
(b)Suppose not; then 9y 0 ; y 00 2 Y such that y 0 6= y 00 and such that

8y 2 Y; py 0 = py 00 > py (1) :

Since Y is strictly convex,8 2 (0; 1) ; y := (1 ) y 0 + y 00 2 Int Y: Then, 9" > 0 such that B y ; "
q
"
PC " 2
Y: Consider y := y + 2C 1; where 1 := (1; :::; 1) 2 RC : d y ; y = c=1 2C = 2p"C : Then, y 2
B y ;" Y and, since p 0, we have that py > py = py 0 = py 00 ; contradicting (1) :
5.
This exercise is taken from Beavis and Dobbs (1990), pages 74-78.
y
3.75

2.5

1.25

0
0 0.5 1 1.5 2

-1.25

For every x 2 [0; 2], both 1 (x) and 2 (x) are closed, bounded intervals and therefore convex and
compact sets. Clearly 1 is closed and 2 is not closed.
21.2. SOME TOPOLOGY IN METRIC SPACES 311

1 and 2 are clearly UHC and LHC for x 6= 1: Using the de…nitions, it is easy to see that for x = 1; 1
is UHC, and not LHC and 2 is LHC and not UHC.
6.
y 1

0.5

0
0 1.25 2.5 3.75 5

-0.5

-1

For every x > 0, is a continuous function. Therefore, for those values of x; is both UHC and LHC.
is UHC in 0: For every neighborhood of [ 1; 1] and for any neighborhood of f0g in R+ ; (x) [ 1; 1] :
is not LHC in 0: Take the open set V = 21 ; 32 ;we want to show that 8" > 0 9z 2 (0; ") such that
(z ) 2 = 21 ; 32 : Take n 2 N such that n1 < " and z = n1 : Then 0 < z < " and sin z = sin n = 0 2 = 12 ; 32 :
Since is UHC and closed valued, from Proposition 16 is closed.
7. p
is not closed. Take xn = 2n2 2 [0; 1] for every n 2 N: Observe that xn ! 0: For every n 2 N; yn =
1 2 (xn ) and yn ! 1: But 1 2 (0) = [0; 1] :
1 3
is not UHC. Take x = 0 and a neighborhood V = 2 ; 2 of (0) = [0; 1] : Then 8" > 0; 9x 2 (0; ") nQ:
Therefore, (x ) = [ 1; 0] * V:
is not LHC. Take x = 0 and the open set V = 12 ; 32 :Then (0) \ 12 ; 32 = [0; 1] \ 12 ; 32 = 12 ; 1 6= ?:
But, as above, 8" > 0; 9x 2 (0; ") nQ: . Then (x ) \ V = [ 1; 0] \ 21 ; 32 = ?:
8.
(This exercise is taken from Klein, E. (1973), Mathematical Methods in Theoretical Economics, Academic
Press, New York, NY, page 119).
Observe that 3 (x) = x2 2; x2 1 :
y

7.5

2.5

0
0 0.5 1 1.5 2 2.5 3

-2.5

1 ([0; 3]) = (x; y) 2 R2 : x 0; 3; y x2 2; y x2 : 1 ([0; 3]) is de…ned in terms of weak in-

equalities and continuous functions and it is closed and therefore 1 is closed. Similar argument applies to
2 and 3 :
Since [ 10; 10] is a compact set such that 1 ([0; 3]) [ 10; 10] ; from Proposition 17, 1 is UHC. Similar
argument applies to 2 and 3 :
1 is LHC. Take an arbitrary x 2 [0; 3] and a open set V with non-empty intersection with 1 (x) =
2 2 2 2 p p 2
x 2; x : To …x ideas, take V = "; x + " ; with " 2 0; x : Then, take U = "; x + " : Then for
every x 2 U; x2 V \ 1 (x) :
Similar argument applies to 2 and 3:
312 CHAPTER 21. SOLUTIONS

21.3 Di¤erential Calculus in Euclidean Spaces

1.
The partial derivative of f with respect to the …rst coordinate at the point (x0 ; y0 ), is - if it exists and is
…nite -
f (x; y0 ) f (x0 ; y0 )
lim
x!x0 x x0

f (x; y0 ) f (x0 ; y0 ) 2x2 xy0 + y02 (2x20 x0 y0 + y02 )

=
x x0 x x0
2x2 2x20 (xy0 x0 y0 )
=
x x0
= 2(x + x0 ) y0

Then
D1 f (x0 ; y0 ) = 4x0 y0
The partial derivative of f with respect to the second coordinate at the point (x0 ; y0 ), is - if it exists and
is …nite -
f (x0 ; y) f (x0 ; y0 )
lim
y!y0 y y0

f (x0 ; y) f (x0 ; y0 ) 2x2 x0 y + y 2 (2x20 x0 y0 + y02 )

= 0
y y0 y y0
x0 (y y0 ) + y 2 y02
=
y y0
= x0 + (y + y0 )

D2 f (x0 ; y0 ) = x0 + 2y0 :
2.
a.
The domain of f is Rnf0g R. As arctan is di¤erentiable over the whole domain, we may compute the
partial derivative over the whole domain of f at the point (x; y) - we omit from now on the superscript 0

y y 1
D1 f (x; y) = arctan + x( 2 )
x x 1 + ( xy )2
y 1
= arctan y
x 1 + ( xy )2
1 1 1
D2 f (x; y) = x =
x 1 + ( xy )2 1 + ( xy )2

b.
The function is de…ned on R++ R. and

8(x; y) 2 R++ R; f (x; y) = ey ln x

Thus as exp and ln are di¤erentiable over their whole respective domain, we may compute the partial
derivatives :
y y ln x
D1 f (x; y) = e = yxy 1
x
D2 f (x; y) = ln(x)eylnx = ln(x)xy

c.
21.3. DIFFERENTIAL CALCULUS IN EUCLIDEAN SPACES 313
p p
f (x; y) = (sin(x + y)) x+y = e x+y ln[sin(x+y)] in (0; 3).
We check that sin(0+3) > 0 so that the point belongs to the domain of the function. Both partial derivatives
in (x; y) have the same expression since f is symmetric with respect to x and y.

1 p p
D1 f (x; y) = D2 f (x; y) = p ln[sin(x + y)] + x + y cot(x + y) (sin(x + y)) x+y ;
2 x+y
and
1 p p
D1 f (0; 3) = D2 f (0; 3) = p ln[sin(3)] + 3 tan(3) (sin(3)) 3
2 3
3.
a.
f (h; 0) f (0; 0) 0
Dx f (0; 0) = lim = lim = 0;
h!0 h h!0 h

b.
f (0; k) f (0; 0) 0
Dy f (0; 0) = lim = lim = 0;
k!0 h h!0 h
c.
1
Consider the sequences (xn )n2N and (yn )n2N such that 8n 2 N; : xn = yn = n. We have that
lim (xn ; yn ) = 0R2 , but
n!0
1
n2
f (xn ; yn ) = 1 1 :
n2 + n2
Then
1
lim f (xn ; yn ) = 6= f (0; 0) = 0
n!0 2
Thus, f is not continuous in (0; 0).

4.

f (1 + h 1; 1 +h 2) f (1; 1)
f 0 ((1; 1); ( 1; 2 )) = limh!0 =
h
1 2 + h( 1 + 2 ) 2
limh!0 =
h (1 + h 1 )2 + (1 + h 2 )2 + 1 3

1 + 2
=
9
5.
The directional derivative of f at the point x0 = (0; 0) in the direction u 2 R is given by

f (x0 + hu) f (x0 )

lim :
h!0 h
Take any u = (u1 ; u2 ) 2 R2 . Then

f (x0 + hu) f (x0 ) f ((hu1 ; hu2 )) f ((0; 0)) f ((hu1 ; hu2 ))

lim = lim = lim
h!0 h h!0 h h!0 h
If u1 6= 0, then
(hu1 )2 (hu2 )2 h4 (u21 u22 )
2
(hu1 ) + (hu2 ) 2 h (u21 + u22 )
2 hu2 u2
lim = lim = lim 2 1 22 = 0
h!0 h h!0 h h!0 u1 + u2

If u1 = 0:
f ((hu1 ; hu2 )) hu2
lim = lim = lim u2 = u2 :
h!0 h h!0 h h!0

6.
314 CHAPTER 21. SOLUTIONS

a) given x0 2 Rn we need to …nd Tx0 : Rn ! Rm linear and Ex0 with limv!0 Ex0 (v) = 0. Take Tx0 = l
and Ex0 0. Then, the desired result follows.

b) projection is linear, so by a) is di¤erentiable.

7.
From the de…nition of continuity, we want to show that 8x0 2 Rn ; 8" > 0 9 > 0 such that kx x0 k <
) kl (x) l (x0 )k < ". De…ned [l] = A, we have that
kl (x) l (x0 )k = kA x x0 k =

(1) Pm (2)
(21.11)
= R1 (A) (x x0 ) ; :::; Rm (A) (x x0 ) i=1 Ri (A) (x x0 )
Pm
i=1 Ri (A) kx x0 k m maxi2f1;:::;mg Ri (A) kx x0 k ;
where (1) follows from Remark 56 and (2) from Proposition 53.4, i.e., Cauchy-Schwarz inequality. Take
"
= :
m maxi2f1;:::;mg fRi (A)g

Then we have that kx x0 k < implies that kx x0 k m maxi2f1;:::;mg Ri (A) < ", and from
(21:11) ; kl (x) l (x0 )k < ", as desired.
8.
2 3
cos x cos y sin x sin y
Df (x; y) = 4 cos x sin y sin x cos y 5 :
sin x cos y cos x sin y

9.
2 3
g 0 (x)h(z) 0 g(x)h0 (z)
Df (x; y; z) = 4 g 0 (h(x))h0 (x)=y g(h(x))=y 2 0 5:
0 0
exp(xg(h(x))((g(h(x)) + g (h(x))h (x)x)) 0 0

10 .
a.
f is di¤erentiable over its domain since x 7! log x is di¤erentiable over R++ . Then from Proposition 625, we
know that
[dfx ] = Df (x) = 3x1 1 ; 6x1 2 ; 2x1 3
Then
1 1 1
[dfx0 ] = 3 6 4
By application of Remark 626, we have that
0 1
1 p
1 3
f 0 (x0 ; u) = Df (x0 ):u = p 1
3
1
6
1
4 : @1A =
3 4
1
b.
As a polynomial expression, f is di¤erentiable over its domain.
[dfx ] = Df (x) = 2x1 2x2 ; 4x2 2x1 6x3 ; 2x3 6x2
Then
[dfx0 ] = 2 4 2
and 0 1
p1
2
f 0 (x0 ; u) = Df (x0 ):u = 2 4 2 :@ 0 A = 0
p1
2
21.3. DIFFERENTIAL CALCULUS IN EUCLIDEAN SPACES 315

c.
[dfx ] = Df (x) = ex1 x2 + x2 x1 ex1 x2 ; x21 ex1 x2
Then
[dfx0 ] = 1 0
and
2
f 0 (x0 ; u) = Df (x0 ):u = 1; 0 : =2
3
11 .
Then since f is in C 1 (Rnf0g), we know that f admits partial derivative functions and that these
functions admit themselves partial derivatives in (x; y; z). Since, the function is symmetric in its arguments,
2
it is enough to compute explicitly @ f@x
(x;y;z)
2 .

@f 3
(x; y; z) = x(x2 + y 2 + z 2 ) 2
@x
Then
@2f 3 5
(x; y; z) = (x2 + y 2 + z 2 ) 2 + 3x2 (x2 + y 2 + z 2 ) 2
@x2
Then 8(x; y; z) 2 R3 nf0g,

@2f @2f @2f 3 5

2
(x; y; z) + 2 (x; y; z) + 2 (x; y; z) = 3(x2 + y 2 + z 2 ) 2 + (3x2 + 3y 2 + 3z 2 )(x2 + y 2 + z 2 ) 2
@x @y @z
3 3
= 3(x2 + y 2 + z 2 ) 2 + 3(x2 + y 2 + z 2 ) 2

=0

12 .
g; h 2 C 2 (R; R++ )2

g(x)
f : R3 ! R3 ; f (x; y; z) = h(z) ; g(h(x)) + xy; ln(g(x) + h(x))

g(x)
Since Im(g) R++ , (x; y; z) 7! h(z) is di¤erentiable as the ratio of di¤erentiable functions. And since
Im(g + h) R++ and that ln is di¤erentiable over R++ , x 7! ln(g(x) + h(x)) is di¤erentiable by proposition
619. Then 2 3
g 0 (x) g(x)
h(z) 0 h0 (z) h(z) 2
6 7
Df (x; y; z) = 4h0 (x)g 0 (h(x)) + y x 0 5
0 0
g (x)+h (x)
g(x)+h(x) 0 0
13 .
a.
Since

ex + g (x)
h (x) = ;
eg(x) + x
then
ex + g 0 (x)
[dhx ] = Dh (x) = ;
0
g (x) eg(x) + 1

1 + g 0 (0)
[dh0 ] = Dh (0) = ;
0
g (0) eg(0) + 1
b.
Let us de…ne l : R ! R2 ; l(x) = (x; g(x)). Then h = f l. As l is di¤erentiable on R and f is
di¤erentiable on R2 we may apply the “chain rule".

dh0 = dfl(0) dl0

316 CHAPTER 21. SOLUTIONS

1
[dlx ] = Dl(x) =
g 0 (x)
ex 1
[df(x;y) ] = Dl(x; y) =
1 ey
Then
e0 1 1 1 + g 0 (0)
[dh0 ] = g(0) 0 =
1 e g (0) 1 + g 0 (0)eg(0)
14 .

Dx (b a) (x) = Dy b (y)jy=a(x) Dx a (x) :

Dy1 g (y) Dy2 g (y) Dy3 g (y)

Dy b (y) =
Dy1 f (y) Dy2 f (y) Dy3 f (y) jy=a(x)
2 3
Dx1 f (x) Dx2 f (x) Dx3 f (x)
Dx a (x) = 4 Dx1 g (x) Dx2 g (x) Dx3 g (x) 5
1 0 0

Dx (b a) (x) =

2 3=
2 3
6 Dx1 f (x) Dx2 f (x) Dx3 f (x) 7
4 Dy1 g(f (x);g(x);x1 ) Dy2 g(f (x);g(x);x1 ) Dy3 g(f (x);g(x);x1 ) 56 7
6 Dx g(x) Dx2 g(x) Dx3 g(x) 7
4 1 5
Dy1 f (f (x);g(x);x1 ) Dy2 f (f (x);g(x);x1 ) Dy3 f (f (x);g(x);x1 )
1 0 0
2 3
Dx1 f Dx2 f Dx3 f
Dy1 g Dy 2 g Dy 3 g 4 Dx1 g
= Dx2 g Dx3 g 5 =
Dy1 f Dy 2 f Dy3 f
1 0 0

Dy1 g Dx1 f + Dy2 gDx1 g + Dy3 g Dy1 g Dx2 f + Dy2 g Dx2 g Dy1 g Dx3 f + Dy2 g Dx3 g
= :
Dy1 f Dx1 f + Dy2 f Dx1 g + Dy3 f Dy1 f Dx2 f + Dy2 f Dx2 g Dy1 f Dx3 f + Dy2 f Dx3 g
15 .
By the su¢ cient condition of di¤erentiability, it is enough to show that the function f 2 C 1 . Partial
@f @f
derivatives are = 2x + y and = 2y + x –both are indeed continuous, so f is di¤erentiable.
@x @y
16 .
i) f 2 C 1 as Df (x; y; z) = (1 + 4xy 2 + 3yz; 3y 2 + 4x2 y + 3z; 1 + 3xy + 3z 2 ) has continuous entries
(everywhere, in particular around (x0 ; y0 ; z0 ))).
ii) f (x0 ; y0 ; z0 ) = 0 by direct calculation.
iii) fz0 = @f 0 @f 0 @f
@z j(x0 ;y0 ;z0 ) = 7 6= 0, fy = @y j(x0 ;y0 ;z0 ) = 10 6= 0 and fx = @x j(x0 ;y0 ;z0 ) = 8 6= 0.
Therefore we can apply Implicit Function Theorem around (x0 ; y0 ; z0 ) = (1; 1; 1) to get
@x fz0
= = 7=8;
@z fx0
@y fz0
= = 7=10:
@z fy0
17 .
a)
2x1 2x2 2 3
Df =
x2 x1 1 1
and each entry of Df is continuous; then f is C 1 . det Dx f (x; t) = 2x21 + 2x22 =
6 0 except for x1 = x2 = 0.
Finally,
1
2x1 2x2 2 3 1 2x1 + 2x2 3x1 2x2
Dg(t) = = 2 2 :
x2 x1 1 1 2x1 + 2x2 2x2 + 2x1 2x1 3x2
21.3. DIFFERENTIAL CALCULUS IN EUCLIDEAN SPACES 317

b)
2x2 2x1 1 2t2
Df =
2x1 2x2 2t1 2t2 2t1 + 2t2
continuous, det Dx f (x; t) = 4x22 4x21 6= 0 except for jx1 j = jx2 j. Finally
1
2x2 2x1 1 2t2
Dg(t) = =
2x1 2x2 2t1 2t2 2t1 + 2t2
:
1 4x1 t1 + 4x1 t2 + 2x2 4x1 t1 4x1 t2 + 4x2 t2
= 4x22 4x21 2x1 + 4x2 t1 4x2 t2 4x1 t2 4x2 t1 + 4x2 t2

c)
2 3 2t1 2t2
Df =
1 1 t2 t1
continuous, det Dx f (x; t) = 5 6= 0 always. Finally
1
2 3 2t1 2t2 1 2t1 3t2 2t2 3t1
Dg(t) = = :
1 1 t2 t1 5 2t1 + 2t2 2t2 + 2t1
18.
As an application of the Implicit Function Theorem, we have that
@ (z 3 xz y )
@z @x z
= @(z 3 xz y)
=
@x 3z 2 x
@z

if 3z 2 x 6= 0. Then,
z(x;y) @z @z
@2z @ 3(z(x;y))2 x 3z 2 x 6 @y z2
@y
= = 2
@x@y @y (3z 2 x)
Since,
@ (z 3 xz y )
@z @y 1
= @(z 3 xz y)
= ;
@y 3z 2 x
@z
we get
1
@2z 3z 2 x 3z 2 x 6 3z21 x z2 3z 2 x
= 2 = 3
@x@y (3z 2 x) (3z 2 x)
19.
As an application of the Implicit Function Theorem, we have that the Marginal Rate of Substitution in
(x0 ; y0 ) is
@(u(x;y) k)
dy @x
= @(u(x;y) k)
<0
dx j(x;y)=(x0 ;y0 )
@y j(x;y)=(x0 ;y0 )
! !
( ) (+) ( ) (+) (+) ( ) ( ) (+)
dy dy
Dx u(x;y(x)) Dxx u + Dxy u dx Dy u Dxy u + Dyy u dx Dx u
d2 y @ Dy u(x;y(x))
= = 2 >0
dx2 @x (Dy u)
(+)

and therefore the function y (x) describing indi¤erence curves is convex.

20.
Adapt the proof for the case of the derivative of the product of functions from R to R.
21.
Di¤erentiate both sides of
f (ax1 ; ax2 ) = an f (x1 ; x2 )
with respect to a and then replace a with 1:
318 CHAPTER 21. SOLUTIONS

21.4 Nonlinear Programming

1.
(a)
If = 0; then f (x) = : The constant function is concave and therefore pseudo-concave, quasi-concave,
not strictly concave.
If > 0, f 0 (x) = x 1 ; f 00 (x) = ( 1) x 2 :
0; 0
f 00 (x) 0 , ( 1) 0 , 0 1 , f concave ) f quasi-concave.
f 00 (x) < 0 , ( > 0 and 2 (0; 1)) ) f strictly concave.
(b)
The Hessian matrix of f is
2 3
1 1( 1 1) x 1 2 0
6 .. 7
D2 f (x) = 4 . 5
2
0 n n( n 1) x n

:
D2 f (x) is negative semide…nite , (8i; i 2 [0; 1]) ) f is concave.
D2 f (x) is negative de…nitive , (8i; i > 0 and i 2 (0; 1)) ) f is strictly concave.
The border Hessian matrix is
2 1 1
3
0 1 1x 1 n nx n
6 1 1x 1 1 1 1( 1 1) x 1 2
0 7
6 7
B (f (x)) = 6 . 7
4 j .. 5
1 2
n nx 0 n n( n 1) x n
:
The determinant of the signi…cant leading principal minors are
1
0 1 1x 1
2 2 1 2
det 1 2 = 1 1 x 1 <0
1x ( 1) x
1 1
1 1 1 1

2 1 1
3
0 1 1x 1
2 2x 2

det 4 1 1x 1 1
1 1 ( 1 1) x 1 2
0 5=
1 2
2 2x 1 0 2 2 ( 2 1) x 2

1 1 2 1 1 2
= 1 1x 1
1 1x 1
2 2 ( 2 1) x 2
2 2x 1
2 2x 2
1 1 ( 1 1) x 1 =

= 1+ 2 4
1 1 2 2x 1 1x 2 ( 2 1) + 2 2x 1 ( 1 1) =

= 1+ 2 4
1 1 2 2x 1 1x 2 ( 2 1) + 2 2x 1 ( 1 1) > 0

i¤ for i = 1; 2; i > 0 and i 2 (0; 1) :

(c)
If = 0; then f (x) = min f ; g = 0:
If > 0; we have

The intersection of the two line has coordinates x : = + ; :

f is clearly not strictly concave, because it is constant in a subset of its domain. Let’s show it is concave
and therefore pseudo-concave and quasi-concave.
Given x0 ; x00 2 X; 3 cases are possible.
Case 1. x0 ; x00 x :
Case 2. x0 ; x00 x :
Case 3. x0 x and x00 x :
The most di¢ cult case is case 3: we want to show that (1 ) f (x0 ) + f (x00 ) f ((1 ) x0 + x00 ) :
Then, we have
21.4. NONLINEAR PROGRAMMING 319

(1 ) f (x0 ) + f (x00 ) = (1 ) min f ; x0 g + min f ; x00 g = (1 ) ( x0 )+ :

Since, by construction x0 ;

(1 ) ( x0 )+ ;

since, by construction x00 ;

(1 ) ( x0 )+ (1 ) ( x0 ) + (( x00 )= [(1 ) x0 + x00 ] :

Then

(1 ) f (x0 ) + f (x00 ) min f ; [(1 ) x0 + x00 ] g = f ((1 ) x0 + x00 ) ;

as desired.
2.
a.
i. Canonical form.
For given 2 (0; 1), a 2 (0; +1),

max(x;y)2R2 u (x) + (1 ) u (y) s.t. a 21 x y 0 1

2a 2x y 0 2
x 0 3
y 0 4

1 2 2
y=a 2x ; solution is x= a; y = a
y = 2a 2x 3 3

y 2

1.5

0.5

0
0 0.5 1 1.5 2

ii. The set X and the functions f and g ( X open and convex; f and g at least di¤erentiable).
a. The domain of all function is R2 . Take X = R2 which is open and convex.
b. Df (x; y) = ( u0 (x) ; (1 ) u0 (y)).The Hessian matrix is

u00 (x) 0
0 (1 ) u00 (y)

Therefore, f and g are C 2 functions and f is strictly concave and the functions g j are a¢ ne.
iii. Existence.
C is closed and bounded below by (0; 0) and above by (a; a) :
320 CHAPTER 21. SOLUTIONS

y a 21 x a
2x 2a y 2a.
iv. Number of solutions.
The solution is unique because f is strictly concave and the functions g j are a¢ ne and therefore concave.
v. Necessity of K-T conditions.
The functions g j are a¢ ne and therefore concave.
x++ = 21 a; 12 a

a 12 12 a 1 1
2a = 4a > 0
2a 2 12 a 1 1
2a = 2a > 0
1
2a > 0
1
2a > 0
vi. Su¢ ciency of K-T conditions.
The objective function is strictly concave and the functions g j are a¢ ne
vii. K-T conditions.

1
L (x; y; 1 ; :::; 4; ; a) = u (x) + (1 ) u (y) + 1 a x y + 2 (2a 2x y) + 3x + 4 y:
2
8
>
> u0 (x) 21 1 2 2 + 3 =0
>
>
>
> (1 ) u0 (y) 1 2+ 4 =0
<
min 1 ; a 12 x y =0
> min f 2 ; 2a 2x yg
> =0
>
>
>
> min f 3 ; xg =0
:
min f 4 ; yg =0
2 2
b. Inserting (x; y; 1; 2; 3; 4) = 3 a; 3 a; 1 ; 0; 0; 0 , with 1 > 0, in the Kuhn-Tucker conditions we
get: 8
>
> u0 23 a 1
2 1 =0
>
> 0 2
>
> (1 ) u 3a 1 =0
<
a 12 23 a 23 a =0
>
> min 0; 2a 2 23 a 2
3a =0
>
>
>
> min 0; 23 a =0
:
min 0; 23 a =0
and 8
>
> = 2 u0 23 a > 0
1
>
>
1 >
> = (1 ) u0 23 a 0
12 <
2
a 2 3a 3a = 0
>
> min f0; 0g =0
>
> 2
>
> min 0; 3 a = 0
:
min 0; 23 a =0
Therefore, the proposed vector is a solution if

2 2
2 u0 a = (1 ) u0 a > 0;
3 3
i.e.,
1
2 =1 or =
and for any a 2 R++ :
3
c. If the …rst, third and fourth constraint hold with a strict inequality, and the multiplier associated with
the second constraint is strictly positive, Kuhn-Tucker conditions become:
8
>
> u0 (x) 2 2 =0
>
> 0
>
> (1 ) u (y) 2 =0
<
a 21 x y >0
> 2a 2x y
> =0
>
>
>
> x > 0
:
y >0
21.4. NONLINEAR PROGRAMMING 321

8
< u0 (x) 2 2 =0
(1 ) u0 (y) 2 =0
:
2a 2x y =0

x y 2 a

u0 (x) 2 2 u00 (x) 0 2 u0 (x) 0

(1 ) u0 (y) 2 0 (1 ) u00 (y) 1 u0 (y) 0
2a 2x y 2 1 0 0 2
2 3
u00 (x) 0 2
det 4 0 (1 ) u00 (y) 1 5=
2 1 0

(1 ) u00 (y) 1 0 2
= u00 (x) det 2 det =
1 0 (1 ) u00 (y) 1

= u00 (x) 4 (1 ) u00 (y) > 0

2 3 1 2 3
u00 (x) 0 2 u0 (x) 0
D( ;a) (x; y; 2) = 4 0 (1 ) u00 (y) 1 5 4 u0 (y) 0 5
2 1 0 0 2

Using maple:
2 3 1
u00 (x) 0 2
4 0 (1 00
) u (y) 1 5 =
2 1 0
2 3
1 2 2u00 (y) 2 u00 (y)
= 1 4 2 4 u00 (x) 5
u00 (x) 4(1 )u00 (y)
2u00 (y) 2 u00 (y) u00 (x) 00 00
u (x) u (y) 2 00
u (x) u00 (y)

D( ;a) (x; y; 2) =
2 32 3
1 2 2u00 (y) (1 ) u0 (x) 0
= 1 4 2 4 u00 (x) 5 4 u0 (y) 0 5 =
u00 (x) 4(1 00
)u (y)
2u00 (y) (1 ) u00 (x) u00 (x) u00 (y) (1 ) 0 2
2 3
u0 (x) 2u0 (y) 4u00 (y) (1 )
= 1 4 2u0 (x) 4u0 (y) 2 u00 (x) 5
u00 (x)+4(1 )u00 (y)
00
2u (y) (1 ) u0 (x) + u00 (x) ( u0 (y)) 2 u00 (x) u00 (y) (1 )

3.
i. Canonical form.
For given 2 (0; 1) ; w1 ; w2 2 R++ ;

max(x;y;m)2R2++ R log x + (1 ) log y s:t

w1 m x 0 x
w2 + m y 0 y

where x and y are the multipliers associated with the …rst and the second constraint respectively.
ii. The set X and the functions f and g ( X open and convex; f and g at least di¤erentiable)
322 CHAPTER 21. SOLUTIONS

The set X = R2++ R is open and convex. The constraint functions are a¢ ne and therefore C 2 . The
gradient and the Hessian matrix of the objective function are computed below:

x y m

1
log x + (1 ) log y x y 0

x y m

x x2 0 0

1 1
y 0 y2 0

0 0 0 0

Therefore, the objective function is C 2 and concave, but not strictly concave.
iii. Existence.
The problem has the same solution set as the following problem:

max(x;y;m)2R2++ R log x + (1 ) log y s:t

w1 m x 0 x
w2 + m y 0 y
log x + (1 ) log y log w1 + (1 ) log w2

whose constraint set is compact (details left to the reader).

iv. Number of solutions.
The objective function is concave and the constraint functions are a¢ ne; uniqueness is not insured on
the basis of the su¢ cient conditions presented in the notes.
v. Necessity of K-T conditions.
++
Constraint functions are a¢ ne and therefore pseudo-concave. Choose (x; y; m) = w21 ; w22 ; 0 :
vi. Su¢ ciency of Kuhn-Tucker conditions.
f is concave and therefore pseudo-concave and constraint functions are a¢ ne and therefore quasi-concave..
vii. K-T conditions.

Dx L = 0 ) x =0 x
1
Dy L = 0 ) y y =0
Dm L = 0 ) x+ y =0
min fw1 m x; xg =0
min fw2 + m y; yg =0
viii. Solve the K-T conditions.
1
Constraints are binding: x = x > 0 and y = y > 0: Then, we get

x = and x = x
x
1 1
y =y and y = y

x =y :=
w1 m x = 0
w2 + m y = 0

x = x and x = x
1 1
y = y and y = y

x = y
w1 m =0
1
w2 + m =0
1 1
Then w1 = w2 + and = w1 +w2 :Therefore
21.4. NONLINEAR PROGRAMMING 323

1
x = w1 +w 2
1
y = w1 +w2
x = (w1 + w2 )
y = (1 ) (w1 + w2 )
b., c.
Computations of the desired derivatives are straightforward.
4.
i. Canonical form.
max(x;y)2R2 x2 y 2 + 4x + 6y s:t: x y+6 0 1
2 y 0 2
x 0 x
y 0: y

ii. The set X and the functions f and g ( X open and convex; f and g at least di¤erentiable)
X = R2 is open and convex. The constraint functions are a¢ ne and therefore C 2 . The gradient and
Hessian matrix of the objective function are computed below.

x y
x2 y 2 + 4x + 6y 2x + 4 2y + 6
;

x y

2x + 4 2 0
2y + 6 0 2
Therefore the objective function is C 2 and strictly concave.
iii. Existence.
The constraint set C is nonempty ( 0 belongs to it) and closed. It is bounded below by 0: y is bounded
y 0
above by 2. x is bounded above because of the …rst constraint:: x 6 y 6: Therefore C is compact.
iv. Number of solutions.
Since the objective function is strictly concave (and therefore strictly quasi-concave) and the constraint
function are a¢ ne and therefore quasi-concave, the solution is unique.
v. Necessity of K-T conditions.
Constraints are a¢ ne and therefore pseudo-concave. Take (x++ ; y ++ ) = (1; 1) :
vi. Su¢ ciency of K-T conditions.
The objective function is strictly concave and therefore pseudo-concave. Constraints are a¢ ne and
therefore quasi-concave.
vii. K-T conditions.
L x; y; 1 ; 2 ; x ; y = x2 y 2 + 4x + 6y + 1 ( x y + 6) + + 2 (2 y) + x x + y y:

2x + 4 1+ x =0
2y + 6 1 2+ y =0
min f x y + 6; 1 g = 0
min f2 y; 2 g = 0
min fx; x g = 0
min y; y = 0

b.

+4 1+ x =0
2y + 6 2+ y =0
min f y + 6; 1 g = 0
min f2 y; 2 g = 0
min f0; x g = 0
min y; y = 0
324 CHAPTER 21. SOLUTIONS

Since y 2; we get y + 6 > 0 and therefore 1 = 0. But then x = 4; which contradicts the
Kuhn-Tucker conditions above.
c.
4+4 1+ x =0
4+6 2+ y =0
min f 4 + 6; 1 g = 0
min f2 2; 2 g = 0
min f2; x g = 0
min 2; y = 0

1 + x=0
+2 2 + y=0
1 =0
min f0; 2g =0
x =0
y =0

x =0
+2 2 =0
1 =0
min f0; 2g =0
x =0
y =0

x =0
2 =2
1 =0
x =0
y =0

Therefore x ; y ; 1 ; 2 ; x ; y = (2; 2; 0; 2; 0) is a solution to the Kuhn-Tucker conditions

5 .2
5. a.
i. Canonical form.
For given 2 (0; 1) ; e 2 R++ ,

max(c1 ;c2 ;k)2R3 u (c1 ) + u (c2 )

s:t:
e c1 k 0 1
f (k) c2 0 2
c1 0 1
c2 0 2
k 0: 3

ii. The set X and the functions f and g ( X open and convex; f and g at least di¤erentiable).
X = R3 is open and convex. Let’s compute the gradient and the Hessian matrix of the second constraint:

c1 c2 k

f (k) c2 0 1 f 0 (k)

c1 c2 k

0 0 0 0

1 0 0 0

f 0 (k) 0 0 f 00 (k) < 0

2 Exercise 7 and 8 are taken from David Cass’problem sets for his Microeconomics course at the University of Pennsylvania.
21.4. NONLINEAR PROGRAMMING 325

Therefore the second constraint function is C 2 and concave; the other constraint functions are a¢ ne.
Let’s compute the gradient and the Hessian matrix of the objective functions:

c1 c2 k

u (c1 ) + u (c2 ) u0 (c1 ) u0 (c2 ) 0

c1 c2 k

u0 (c1 ) u00 (c1 ) 0 0

u0 (c2 ) 0 u00 (c2 ) 0

0 0 0 0

Therefore the objective function is C 2 and concave.

iii. Existence.
The objective function is continuous on R3 :
The constraint set is closed because inverse image of closed sets via continuous functions. It is bounded
below by 0. It is bounded above: suppose not then
if c1 ! +1; then from the …rst constraint it must be k ! 1; which is impossible;
if c2 ! +1; then from the second constraint and the fact that f 0 > 0; it must be k ! +1; violating the
…rst constraint;
if k ! +1; then the …rst constraint is violated.
Therefore, as an application of the Extreme Value Theorem, a solution exists.
iv. Number of solutions.
Since the objective function is concave and the constraint functions are either concave or a¢ ne, uniqueness
is not insured on the basis of the su¢ cient conditions presented in the notes.
v. Necessity of K-T conditions.
The constraints are a¢ ne or concave. Take c++ ++ ++
1 ; c2 ; k = 4e ; 12 f 4e ; 4e : Then the constraints are
veri…ed with strict inequality.
vi. Su¢ ciency of K-T conditions.
The objective function is concave and therefore pseudo-concave. The constraint functions are either
concave or a¢ ne and therefore quasi-concave.
vii. K-T conditions.
L (c1 ; c2 ; k; 1 ; 2 ; 1 ; 2 ; 3 ) := u (c1 ) + u (c2 ) + 1 (e c1 k) + 2 (f (k) c2 ) + 1 c1 + 2 c2 + 3 k:
8 0
>
> u (c1 ) 1+ 1 =0
>
> 0
>
> u (c2 ) 2+ 2 =0
>
> 0
>
> 1 + 2 f (k) + 3 = 0
<
min fe c1 k; 1 g = 0
>
> min ff (k) c2 ; 2 g = 0
>
>
>
> min fc1 ; 1 g = 0
>
>
>
> min fc2 ; 2 g = 0
:
min fk; 3 g = 0

viii. Solve the K-T conditions.

Since we are looking for positive solution we get
8 0
>
> u (c1 ) 1 =0
>
>
>
> u0 (c2 ) 2 =0
>
> 0
>
> 1 + 2 f (k) = 0
<
min fe c1 k; 1 g = 0
>
> min ff (k) c2 ; 2 g = 0
>
>
>
> 1 = 0
>
>
>
> = 0
: 2
3 = 0
326 CHAPTER 21. SOLUTIONS

8 0
>
> u (c1 ) 1 =0
>
>
< u0 (c2 ) 2 =0
0
1 + 2 f (k) = 0
>
>
>
> e c1 k = 0
:
f (k) c2 = 0
Observe that from the …rst two equations of the above system, 1; 2 > 0:
5. b.
i. Canonical form.
For given p > 0; w > 0 and l > 0;

max(x;l)2R2 u (x; l)
s:t:
px wl + wl 0
l l 0
x 0
l 0

ii. The set X and the functions f and g ( X open and convex; f and g at least di¤erentiable).
X = R2 is open and convex.
The constraint functions are a¢ ne and therefore C 2 . The objective function is C 2 and di¤erentiably
strictly quasi concave by assumption.
iii. Existence.
The objective function is continuous on R3 :
The constraint set is closed because inverse image of closed sets via continuous functions. It is bounded
below by 0. It is bounded above: suppose not then
if x ! +1; then from the …rst constraint (px + wl = wl);it must be l ! 1; which is impossible.
Similar case is obtained, if l ! +1:
Therefore, as an application of the Extreme Value Theorem, a solution exists.
iv. Number of solutions.
The budget set is convex. The function is di¤erentiably strictly quasi concave and therefore strictly
quasi-concave and the solution is unique.
v. Necessity of K-T conditions.
The constraints are pseudo-concave (x++ ; l++ ) = ( w l l
3p ; 3 ) satis…es the constraints with strict inequalities.
vi. Su¢ ciency of K-T conditions.
The objective function is di¤erentiably strictly quasi-concave and therefore pseudo-concave. The con-
straint functions are quasi-concave
vii. K-T conditions.
L c1 ; c2 ; k; 1 ; 2 ; 3 ; 4 ; p; w; l := u (x; l) + 1 px wl + wl + 2 l l + 3 x + 4 l:

Dx u 1p + 3 = 0
Dl u 1w 2+ 4 =0
min px wl + wl; 1 = 0
min l l; 2 = 0
min fx; 3 g = 0
min fl; 4 g = 0

viii. Solve the K-T conditions.

Since we are looking for solutions at which x > 0 and 0 < l < l, we get

Dx u 1p + 3 = 0
Dl u 1w 2+ 4 =0
min px wl + wl; 1 = 0
2 =0
3 =0
4 =0
21.4. NONLINEAR PROGRAMMING 327

Dx u 1p
=0
Dl u 1w
=0
min px wl + wl; 1 =0
and then 1 > 0 and

8
< Dx u 1p =0
Dl u 1 w =0 :
:
px wl + wl = 0
7.
a.
Let’s apply the Implicit Function Theorem (:= IFT) to the conditions found in Exercise 7.(a). Writing
them in the usual informal way we have:

c1 c2 k 1 2 e a

u0 (c1 ) 1 =0 u00 (c1 ) 1

u0 (c2 ) 2 =0 u00 (c2 ) 1
0 00
1 + 2 f (k) = 0 2f (k) 1 f 0 (k) 2 k 1

e c1 k = 0 1 1 1
f (k) c2 = 0 1 f 0 (k) k
To apply the IFT, we need to check that the following matrix has full rank
2 00 3
u (c1 ) 1
6 u00 (c2 ) 1 7
6 7
M := 66 2 f 00
(k) 1 f 0
(k) 7
7
4 1 1 5
1 f 0 (k)
:
Suppose not then there exists := ( c1 ; c2 ; k; 1; 2 ) 6= 0 such that M = 0; i.e.,
8 00
>
> u (c1 ) c1 + 1 =0
>
>
< u00 (c2 ) c2 + 2 =0
00 0
2 f (k) k+ 1 + f (k) 2 =0
>
>
>
> c1 + k =0
:
c2 + f 0 (k) k+ =0

Recall that
[M =0) = 0] if f M has full rank.
The idea of the proof is either you prove directly [M =0) = 0] ; or you 1. assume M = 0 and
6= 0 and you get a contradiction.
u00 (c1 ) c1
If we de…ne c := ( c1 ; c2 ), := ( 1; 2) ; D2 := ; the above
u00 (c2 ) c2
system can be rewritten as
8 2
>
> D c+ =0
< 00
2f (k) k+ [ 1; f 0 (k)] =0
:
>
> 1
: c+ k =0
f 0 (k)
8
>
> cT D2 c+ cT =0 (1)
< 00
k 2 f (k) k+ k [ 1; f 0 (k)] =0 (2)
>
> T T 1
: c+ k =0 (3)
f 0 (k)
:
(1) T (3) T 1 (2)
cT D2 c = cT = c = k = k [ 1; f 0 (k)] =
f 0 (k)
328 CHAPTER 21. SOLUTIONS

= k 2 f 00 (k) k > 0;
2 2
while cT D2 c = ( c1 ) u00 (c1 ) + ( c2 ) u00 (c1 ) < 0: since we got a contradiction, M has full rank.
Therefore, in a neighborhood of the solution we have

2 3 1 2 3
u00 (c1 ) 1
6 u00 (c2 ) 1 7 6 7
6 7 6 7
D(e;a) (c1 ; c2 ; k; 1; 2) = 6 2f (k) 00
1 f (k) 0 7 6 k 1 7:
6 7 6 2 7
4 1 1 5 4 1 5
1 f 0 (k) k

To compute the inverse of the above matrix, we can use the following fact about the inverse of partitioned
matrix (see Goldberger, (1964), page 27:
Let A be an n n nonsingular matrix partitioned as

E F
A= ;
G H
1
where En1 n1 ; Fn1 n2 ; G n2 n1 ; Hn2 n2 and n1 + n2 = n: Suppose that E and D := H GE F are
non singular. Then
1
1 E I + F D 1 GE 1
E 1
FD 1
A = :
D 1 GE 1 D 1

In fact, using Maple, with obviously simpli…ed notation, we get

2 3 1
u1 0 0 1 0
6 0 u2 0 0 1 7
6 7
6 0 0 2 f2 1 f1 7 =
6 7
4 1 0 1 0 0 5
0 1 f1 0 0
2 1 f1 1 u2 f12 + 2 f2 f1
3
u1 + u2 f12 + 2 f2 u1 + u2 f12 + 2 f2 u1 + u2 f12 + 2 f2 u1 + u2 f12 + 2 f2
u 2 u1 +
u2 f12 + 2 f2
6 f12 7
6 f1 f1
f1 u1 + u2uf12 + 2 f2 u1 + 2 f2 7
6 u1 + u2 f12 + 2 f2 u1 + u2 f12 + 2 f2 u1 + u2 f12 + 2 f2 u1 + u2 f12 + 2 f2 7
6 f1
1
7
6 1
u1 + u2 f12 + 2 f2 u1 + u2 f12 + 2 f2
1
u1 + u2 f12 + 2 f2
u1
u1 + u2 f12 + 2 f2
u2 u1 + u2ff12 + 2 f2 7
6 1 7
6 2
u2 f 1 + 2 f 2
f1 u1 + u2uf12 + 2 f2 u1 u f 2+ f
u1 u1 + 2u21f 2 +2 22f2 u2 f1 u1 + u2uf12 + 2 f2 7
4 u1 + u2 f12 + 2 f2 1 u1 + u2 f12 + 2 f2 1 1
5
u2 u1 + u2ff12 + 2 f2 u 1 + 2 f2
u1 + u2 f12 + 2 f2
u2 u1 + u2ff12 + 2 f2 u2 f1 u1 + u2uf12 + 2 f2 u2 u1 +u1u+ 2 f2
2
2 f1 + 2 f2
1 1 1

Therefore,
2 3
2 3 u2 f1 + 2 f2 2 k 1 + u2 f 1 k
u1 + u2 f1 + 2 f2 u 1 + u 2 f1 + 2 f2
De c1 Da c1 6 f1 2 k 1 +k u1 +k 2 f2
7
6 De c2 7 6 f1 u1 + u2uf11 + 2 f2 7
6 Da c2 7 6 u 1 + u 2 f1 + 2 f2 7
6 De k 7=6 u1 2 k
1
+ u 2 f1 k 7
6 Da k 7 6 7:
5 6 7
u1 + u2 f1 + 2 f2 u1 + u2 f1 + 2 f2
4 De 1 Da 1 6 1
7
4 u1 u1 +u2uf21f+1 +2 f22f2 u1 2u1 +k + u 2 f1 k
u2 f1 + 2 f2 5
De ; 2 Da ; 2 1
u2 f1 u1 + u2uf11 + 2 f2 u2 f1 2u1k+ u2+k u1 +k 2 f2
f1 + 2 f2

+ + +
u2 f1 + 2 f2 u00 (c2 )f 0 + 2 f 00
Then De c1 = u 1 + u 2 f1 + 2 f2 := u (c1 )+ u (c2 )f 0 + 2 f 00
00 00 "= ">0
+ + +
+
u1 f 0 u00 (c1 )
De c2 = f1 u1 + u2 f 1 + 2 f2
:= u00 (c1 )+ u00 (c2 )f 0 + 2f
00 "= ">0
+ + +
+ + + + +
1
k 1 + u2 f 1 k 2 k + u00 (c2 )f 0 k
Da k = 2
u 1 + u 2 f1 + 2 f2 := u (c1 )+ u (c2 )f + 2 f 00
00 00 0 ;
+ + +

+ + + + +
1
which has sign equal to sign 2 k + u00 (c2 )f 0 k :
b.
21.4. NONLINEAR PROGRAMMING 329

Let’s apply the Implicit Function Theorem to the conditions found in a previous exercise. Writing them
in the usual informal way we have:

x l 1 p w l

Dx u 1p=0 Dx2 2
Dxl p 1
2 2
Dl u 1w = 0 Dxl Dl w 1
px wl + wl = 0 p w 1 l l w
To apply the IFT, we need to check that the following matrix has full rank
2 2 2
3
Dx Dxl p
M := 4 Dxl 2
Dl2 w 5
p w
:
Dx2 Dxl 2
p D2 q
De…ned D2 := ; q := ; we have M := :
Dxl Dl2
2
w qT
Suppose not then there exists := ( y; ) 2 R2 R n f0g such that M = 0; i.e.,

D2 y q =0 (1)
qT y =0 (2)
We are going to show
Step 1. y 6= 0; Step 2. Du y = 0; Step 3. It is not the case that y T D2 y < 0:
These results contradict the assumption about u:
Step 1.
Suppose y = 0: Since q 0; from (1) ; we get = 0; and therefore = 0; a contradiction.
Step 2.
From the First Order Conditions, we have
Du 1 q = 0 (3) :
(3) (2)
Du y = 1q y = 0:
Step 3.
(1) (2)
y T D2 y = y T q = 0:
Therefore, in a neighborhood of the solution we have
2 2 2
3 12 3
Dx Dxl p 1
D(p;w;l) (x; l; 1 ) = 4 Dxl 2
Dl2 w 5 4 1 5:
p w 1 l l w
Unfortunately, here we cannot use the formula seen in the Exercise 4 (a) because the Hessian of the
utility function is not necessarily nonsingular. We can invert the matrix using the de…nition of inverse. (For
the inverse of a partitioned matrix with this characteristics see also Dhrymes, P. J., (1978), Mathematics for
Econometrics, 2nd edition, Springer-Verlag, New York, NY, Addendum pages 142-144.
With obvious notation and using Maple, we get
2 3 1 2 w2 w
p dx w2 2dpw+p dw+pdl 3
dx d p dx w2 2dpw+p2 dl 2d
l dx w2 2dpw+p2 dl
4 d 6 p2 7
dl w 5 = 4 p dx w2 2dpw+p w
2d
l dx w2 2dpw+p2 dl
dx w+dp
dx w2 2dpw+p2 dl 5
p w 0 dw+pdl dx w+dp dx dl +d2

dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl

Therefore,
D(p;w;l) (x; l; 1) =
2 w2 w dw+pdl 32 3
dx w2 2dpw+p2 dl p dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl 1 0 0
6 p2 74
4 w
p dx w2 2dpw+p 2d dx w2 2dpw+p2 dl
dx w+dp
dx w2 2dpw+p2 dl 5 0 1 0 5=
l
dw+pdl dx w+dp dx dl +d2 1 l l w
dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl
2 w2 1 dw+pdl pw 1 ldw+lpdl dw+pdl
3
dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl w
6 pw 1 dx w+dp p2 1 ldx w+ldp dx w+dp 7
=4 dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl w 5
2 2
1 dw+ 1 pdl +dx dl d 1 dx w+ 1 dp ldx dl +ld dx dl +d2
dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl dx w2 2dpw+p2 dl w
330 CHAPTER 21. SOLUTIONS

pw 1 dx w + dp
Dp l =
dx w 2 2dpw + p2 dl

p2 1 ldx w + ldp
Dw l = :
dx w 2 2dpw + p2 dl
The sign of these expressions is ambiguous, unless other assumptions are made.
7.
[)]
Since f is concave, then
1 1 1 1
f x+ y f (x) + f (y) :
2 2 2 2
Since f is homogenous of degree 1, then

1 1
f (x + y) = f (x + y) :
2 2

Therefore,
f (x + y) f (x) + f (y) :
[(]
Since f is homogenous of degree 1, then for any z 2 R2 and any a 2 R+ , we have

f (az) = af (z) : (21.12)

By assumption, we have that

for any x; y 2 R2 ; f (x + y) f (x) + f (y) : (21.13)

Then, for any 2 [0; 1],

(21:13) (21:12)
f ((1 ) x + y) f ((1 ) x) + f ( y) = (1 ) f (x) + f (y) ;
as desired.
21.4. NONLINEAR PROGRAMMING 331

References

Aliprantis, C. D.and O. Burkinshaw, (1990), Principles of Real Analysis, 2nd edition, Academic Press, St
Louis, Missouri.
Apostol, T. M., (1967), Calculus, Volume 1, 2nd edition, John Wiley & Sons, New York, NY.
Apostol, T. M., (1974), Mathematical Analysis, 2nd edition, Addison-Wesley Publishing Company, Reading,
MA.
Azariadis, C., (2000), Intertemporal Macroeconomics, Blackwell Publisher Ltd, Oxford, UK.
Balasko, Y., (1988), Foundations of the Theory of General Equilibrium,. Academic Press, Boston.
Bazaraa, M.S. and C. M. Shetty, (1976), Foundations of Optimization, Springer-Verlag, Berlin.
Bartle, R. G., (1964), The Elements of Real Analysis, John Wiley & Sons, New York, NY.
Beavis, B. and I. Dobbs, (1990), Optimization and stability theory for economic analysis, Cambridge Uni-
versity Press, Cambridge, UK.
Cass D., (1991), Nonlinear Programming for Economists, mimeo, University of Pennsylvania.
de la Fuente, A., (2000), Mathematical Methods and Models for Economists, Cambridge University Press,
Cambridge, UK.
Dhrymes, P. J., (1978), Mathematics for Econometrics, 2nd edition Springer-Verlag, New York, NY
El-Hodiri, M. A., (1991), Extrema of smooth functions, Springer-Verlag, Berlin.
Francini, E., (2004), Esercitazioni del corso di Analisi Superiore, mimeo, Università degli Studi di Firenze.
Girsanov, I.V., (1972), Lectures on Mathematical Theory of Extremum Problems, Springer-Verlag, Berlin
and New York.
Goldberger, A.S, (1964), Econometric Theory, John Wiley & Sons, New York, NY.
Gori, F., (2014), Notes for the course “Mathematical Economics”, mimeo, Universita’degli Studi di Firenze.
Hildebrand, W., (1974), Core and equilibria of a large economy, Princiton University Press, Princeton, NJ.
Hildebrand W., and Kirman A. (1976), Introduction to Equilibrium Analysis, North-Holland, Amsterdam.
Jahn, J., (1996), Introduction to the Theory of Nonlinear Optimization, 2nd revised edition, Springer-Verlag,
Berlin.
Herstein, I. N., (1975), Topics in Algebra, 2nd edition, John Wiley & Sons, New York, NY.
Hirsch, M. W. and S. Smale, (1974), Di¤ erential Equations, Dynamical Systems and Linear Algebra, Aca-
demic Press, New York, NY.
Ho¤man, K. and R. Kunze, (1971), Linear Algebra, 2nd edition, Prentice-Hall, Inc., Englewood Cli¤s, New
Jersey.
Lang S. (1971), Linear Algebra, second edition, Addison Wesley, Reading.
Lang, S., (1986), Introduction to Linear Algebra, 2nd edition, Springer-Verlag, New York, NY.
Leininger W. (1984), A generalisation of the ‘Maximum Theorem’, Economics Letters, 15, 309-313.
Lipschutz, S., (1989), 3000 Solved Problems in Linear Algebra, McGraw-Hill, New York, NY.
Lipschutz, S., (1991), Theory and Problems in Linear Algebra, second edition, Schaum’s Outline Series,
McGraw-Hill, New York, NY.
Lipschutz, S., (1965), General Topology, McGraw-Hill, New York, NY.
332 CHAPTER 21. SOLUTIONS

Mangasarian O. L. (1994), Nonlinear Programming, SIAM, Philadelphia.

McLean, R., (1985), Class notes for the course of Mathematical Economics (708), University of Pennsylva-
nia, Philadelphia, PA, mimeo.

Montrucchio, L., (1998), Introduzione alle teoria delle scelte, (in Italian), Carocci Editore, Rome, Italy.
Moore, J. C., (1999), Mathematical Methods for Economic Theory 1, Springer-Verlag, Berlin.
Moore, J. C., (1999), Mathematical Methods for Economic Theory 2, Springer-Verlag, Berlin.
Ok, E. A., (2007), Real Analysis with Economic Applications, Princeton University Press, Princeton NJ.
Ross, K. A., (1980), Elementary analysis: The theory of calculus, Springer-Verlag, New York, NY.
Rudin, W., (1976), Principles of Mathematical Analysis, 3rd edition, McGraw-Hill, New York, NY.
Simon, C. S. (1986), Scalar and vector maximization: calculus techniques with economic applications, in
Reiter S, Studies in mathematical economics, The Mathematical Association of America, p. 62-159

Simon, C. P. and L. Blume, (1994), Mathematics for Economists, Norton, New York
Simmons, G. F., (1963), Introduction to Topology and Modern Analysis, McGraw-Hill, New York.
Smith, L.,(1992), Linear Algebra, 2nd edition, Springer-Verlag, New York, NY.
Stokey, N. C., Lucas, E. C. (with Prescott, E. C.), (1989), Recursive methods in Economic Dynamics,
Harvard University Press, Cambridge, MA.
Sydsaeter, K., (1981), Topics in Mathematical Analysis for Economists, Academic Press, London, UK.
Taylor, A. E. and W. R. Mann, (1984), Advanced Calculus, 3rd ed., John Wiley & Sons, New York, NY.
Villanacci, A., (in progress), Basic Convex Analysis, mimeo, Università degli Studi di Firenze.
Villanacci, A., (in progress), Notes on Mathematics 3: Linear Algebra, Linear and Nonlinear Systems of
Di¤erential Equations, mimeo, Università degli Studi di Firenze.
Villanacci, A., (in progress), Notes on Mathematics 4, Topology, Banach Spaces, Di¤erential Calculus and
Measure, mimeo, Università degli Studi di Firenze.
Villanacci, A., (in progress), Notes on In…nite Dimensional Optimization, mimeo, Università degli Studi di
Firenze.
Villanacci, A., (in progress), An Introduction to Dynamic Programming, mimeo, Università degli Studi di
Firenze.
Villanacci, A., (in progress), Basic Facts on sequences, series and integrals in R, mimeo, Università degli
Studi di Firenze.
Walker M. (1979), A generalization of the maximum theorem, International Journal of Economics, 20,
260-272.

Webster, R., (1994), Convexity, Oxford University Press.

Wrede, R. C. and M. Spiegel (2002), Theory and Problems of Advanced Calculus, 2nd edition, Schaum’s
outlines, McGraw Hill, New York.