Qualifying Exams Study Sheet

Jesse Adams
August 10, 2014

Contents
1) Previous Material
1.a) Calculus . . . . . . . . . . . .
1.a.i) Fundamental Theorem
1.a.ii) Sums . . . . . . . . .
1.a.iii) Trig Substitutions . .
1.a.iv) Integrals . . . . . . . .
1.a.v) Sequences . . . . . . .
1.a.vi) Series . . . . . . . . .
1.a.vii) Vectors . . . . . . . .
1.a.viii)Coordinate Systems .
1.a.ix) Integration Elements .
1.a.x) Multiple Integrals . .
1.a.xi) Vector Fields . . . . .
1.a.xii) Trig Identities . . . . .
1.b) Differential Equations . . . .

. . . . . . .
of Calculus
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

3
3
3
3
3
3
3
4
4
4
4
5
5
5
6

2) Numerical Analysis
2.a) Linear Algebra . . . . . . . . . . . . .
2.a.i) Basics . . . . . . . . . . . . . .
2.a.ii) Norms . . . . . . . . . . . . . .
2.a.iii) Singular Value Decomposition .
2.a.iv) Projectors . . . . . . . . . . . .
2.a.v) QR Factorization . . . . . . . .
2.a.vi) Least Squares . . . . . . . . . .
2.a.vii) Conditioning . . . . . . . . . .
2.a.viii)Stability . . . . . . . . . . . . .
2.a.ix) Gaussian Elimination . . . . .
2.a.x) Cholesky Factorization . . . . .
2.a.xi) Eigenvalues . . . . . . . . . . .
2.a.xii) Iterative Methods . . . . . . .
2.b) Numerical Methods . . . . . . . . . . .
2.b.i) Functional Iteration . . . . . .
2.b.ii) Polynomial Interpolation . . .
2.b.iii) ODEs . . . . . . . . . . . . . .
2.b.iv) Basic Concepts . . . . . . . . .
2.b.v) Methods . . . . . . . . . . . . .
2.b.vi) BVPs . . . . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

6
6
6
6
7
7
7
8
8
9
9
9
10
10
11
11
11
12
12
12
13

3) Analysis
3.a) Metric Spaces . . . . . . . . .
3.a.i) `p Spaces . . . . . . .
3.a.ii) Lebesgue (Lp ) Spaces
3.a.iii) Normed Linear Spaces
3.b) Topology . . . . . . . . . . .

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

14
14
14
15
15
15

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

3.c)

3.d)

3.e)
3.f)

3.g)

3.b.i) Basic Definitions . . . . . . . . . . .

3.b.ii) Defining Topologies and Continuity .
Distributions . . . . . . . . . . . . . . . . .
3.c.i) Basics . . . . . . . . . . . . . . . . .
3.c.ii) Specific Distributions . . . . . . . .
3.c.iii) Function Spaces . . . . . . . . . . .
Measure Theory . . . . . . . . . . . . . . .
3.d.i) Basic Definitions . . . . . . . . . . .
3.d.ii) Measurable Spaces . . . . . . . . . .
3.d.iii) Probability Spaces . . . . . . . . . .
Convergence and Compactness . . . . . . .
3.e.i) Basic Definitions . . . . . . . . . . .
Uniformity . . . . . . . . . . . . . . . . . .
3.f.i) Basic Definitions . . . . . . . . . . .
3.f.ii) Interchanging Limits and Integrals .
3.f.iii) More on Integrals . . . . . . . . . . .
Convergence Theorems . . . . . . . . . . . .
3.g.i) Basic Definitions . . . . . . . . . . .
3.g.ii) Theorems . . . . . . . . . . . . . . .

4) Principals and Methods

4.a) Dynamics of Nonlinear Systems . . . . . .
4.a.i) Dimensional Analysis . . . . . . .
4.a.ii) Phase Plane . . . . . . . . . . . . .
4.a.iii) Dispersion Relations . . . . . . . .
4.b) Contour Integration . . . . . . . . . . . .
4.b.i) Complex Basics . . . . . . . . . . .
4.b.ii) Integration . . . . . . . . . . . . .
4.c) Fourier Series . . . . . . . . . . . . . . . .
4.c.i) Hilbert Space . . . . . . . . . . . .
4.c.ii) Theorems . . . . . . . . . . . . . .
4.d) Distributions . . . . . . . . . . . . . . . .
4.d.i) Basics . . . . . . . . . . . . . . . .
4.e) Fourier Transforms . . . . . . . . . . . . .
4.e.i) Common Functions/Distributions .
4.e.ii) Other stuff . . . . . . . . . . . . .
4.e.iii) Sampling . . . . . . . . . . . . . .
4.e.iv) Other Transforms . . . . . . . . .
4.f) Spectral Theory . . . . . . . . . . . . . . .
4.f.i) Basics . . . . . . . . . . . . . . . .
4.f.ii) Sturm-Liouville . . . . . . . . . . .
4.g) Greens Functions . . . . . . . . . . . . .
4.g.i) Variation of Parameters . . . . . .
4.g.ii) Greens Functions . . . . . . . . .

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

15
16
16
16
17
17
18
18
18
18
19
19
19
19
20
20
20
20
21

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

21
21
21
21
22
22
22
22
23
23
23
23
23
24
24
24
25
25
25
25
25
26
26
26

1)

Previous Material

1.a)

Calculus

1.a.i)

Fundamental Theorem of Calculus

Given f (x) continuous and Riemann integrable on [a, b],

Rx
1. F (x) = a f (t)dt is continuously differentiable on (a, b), with F 0 (x) = f (x).
Rb
b
2. a f (x)dx = [F (x)]a = F (b) F (a).
1.a.ii)
1.

n
X
k=1

2.

Sums
n
X

n(n + 1)
k=
;
2

k=1

n
X

n(n + 1)(2n + 1)
;
k =
6
2

k =

k=1

n(n + 1)
2

2

X
2
1
=
2
n
6
n=1

1.a.iii) Trig Substitutions

a
1. b2 x2 a2 x = sec ()
b

a
2. a2 b2 x2 x = sin ()
b

a
3. a2 + b2 x2 x = tan ()
b
4. x = tan (/2) sin() =
1.a.iv)

2x
1 x2
2 dx
,
cos()
=
, d =
2
2
1+x
1+x
1 + x2

Integrals
R

x=b

dy
dx

2

y=d

dx
dy

2

t=f

s

dx
dt

2

dy
dt

2

1. Arc Length: L = ds =
1+
dx =
1+
dy =
+
x=a
y=c
t=e
s
 2
Z =h
dr
2
r +
d
d
=g
R
R
2. Surface Area: A = 2y ds about x-axis, A = 2x ds about y-axis, with ds as defined in arc length.
1.a.v)

dt =

Sequences

R
1. Integral test: Given continuous, positive,
and decreasing f (x) on [k, ) with f (n) = an , then if k f (x) dx
P
is convergent/divergent, then so is n=k an .
P
P
P
2. P
Comparison test: Given two series with 0 an bn n, then
bn <
an < ;
an =
bn = .
3. Limit comparison test: c = lim

an
. If 0 < c < , then either both converge or both diverge.
bn

4. Alternating series test: Given an = (1)n bn or an = (1)n+1 bn with bn 0, if lim bn = 0 and {bn }
n
P
decreasing, then
an is convergent.


 an+1 

. L < 1 convergence, L > 1 divergence.
5. Ratio test: L = lim 
n
an 
p
6. Root test: L = lim n |an |. L < 1 absolute convergence, L > 1 divergence.
n

7. Absolute convergence:

|an | < . Implies convergence, otherwise the series is conditionally convergent.

3

1.a.vi)

Series

1. Power series:

n

1
1 X f (x)
provided |f (x)| < |a|, and a 6= 0.
=
a f (x)
a n=0
a

2. Taylor series: f (x) =

X
f (n) (x0 )
(x x0 )n
n!
n=0
n

3. Binomial series: (a + b) =

n  
X
n
k=0

ank bk

 
 
X
k n
k
k(k 1) (k n + 1)
k
4. (1 + z) =
.
z for |z| < 1, and
=
n!
n
n
n=0
5. ez =

1.a.vii)

X
zn
;
n!
n=0

sin(z) =

z 2n+1
(1)n ;
(2n
+
1)!
n=0

cos(z) =

X
z 2n
(1)n ;
(2n)!
n=0

ln(z) =

X
(z 1)n
(1)n+1
n
n=1

Vectors

Given a vector function r(t) with r0 (t) 6= 0,

1. Unit tangent vector: T(t) =

r0 (t)
.
||r0 (t)||

2. Unit normal vector: N(t) =

T0 (t)
.
||T0 (t)||

3. Binormal vector: B(t) = T(t) N(t).

Rb
Rt
4. Arc length: L = a ||r0 (t)|| dt = 0 ||r0 (u)|| du.
5. Curvature: =
1.a.viii)

||r0 (t) r00 (t)||

||T0 (t)||
=
.
3
||r0 (t)||
||r0 (t)||

Coordinate Systems

1. Cylindrical: r2 = x2 + y 2 , = tan1 (y/x), z = z;

2. Spherical: r = sin(), = , z = cos();
cos();
2 = x2 + y 2 + z 2
1.a.ix)

x = r cos(), y = r sin(), z = z.

2 = r 2 + z 2 ;

Integration Elements

x1


u1
 (x) 

 du = x2
1. In general, take the determinant: dx = 
u1
u 
x3
u1

2. Cylindrical:
(a)
(b)
(c)
(d)

Constant radius: dA = r d dz
Constant angle: dA = dr dz
Constant height: dA = r dr d.
Volume: dV = r dr d dz

3. Spherical:
(a)
(b)
(c)
(d)

x = sin() cos(), y = sin() sin(), z =

Constant radius: dA = 2 sin() d d

Constant : dA = sin() d dr
Constant : dA = d d
Volume: dV = 2 sin() d d d
4

x1
u2
x2
u2
x3
u2

x1
u3
x2
u3
x3
u3

du

1.a.x)

Multiple Integrals
ZZ

ZZ

1. Change of variables:

f (x, y) dA =
D



 (x, y) 

 dudv (similarly for triple integrals).
f (g(u, v), h(u, v)) 
(u, v) 
S

2. Cylindrical: dA = r drd, dV = r dzdrd, Spherical: dV = 2 sin() ddd.

1.a.xi)

Vector Fields

1. Gradient: f (x, y, z) = hfx , fy , fz i.

2. Conservative vector field: F such that F = f , where f is called the potential function.
3. Given vector field F = P i + Qj on open, simply connected D. If P, Q have continuous 1st order derivatives in
P
Q
D and
=
then F is conservative.
y
x
4. Greens Theorem: C Zpositively oriented,Z Zpiecewise
smooth
 closed curve enclosing D. P, Q have continuous

Q P
1st order partials, then

dA.
P dx + Q dy =
x
y
C
D

5. Curl: Let F = P i + Qj + Rk.


 i

curl(F) = F =  x
P

Then
j


k 

z 
R

(a) If f (x, y, z) has continuous 2nd order partials, then curl(f ) = 0.

(b) If F is a conservative vector field, then curl(F) = 0.
6. Divergence:
div(F) = F =

P
Q R
+
+
x
y
z

7. Stokes Theorem: Given smooth surface S bounded by simple, closed, smooth curve C and vector field F,
Z
ZZ
F dr =
curl(F) dS
C

8. Divergence Theorem: Given simple solid region E, boundary surface S, and vector field F with continuous
1st order partials,
ZZ
ZZZ
F dS =
div(F)dV
S

1.a.xii)

Trig Identities

1. cos(a b) = cos(a) cos(b) sin(a) sin(b)

2. sin(a b) = sin(a) cos(b) sin(b) cos(a)
3. sin(2) = 2 sin() cos()
4. cos(2) = 1 2 sin2 () = 2 cos2 () 1

1.b)

2)
2.a)
2.a.i)

Differential Equations

Numerical Analysis
Linear Algebra
Basics

For a matrix A Cmn ,

1. Range: Space spanned by the columns of A, i.e. im(A) = range(A) = {y : Ax = y}.
2. Nullspace: ker(A) = null(A) = {x : Ax = 0}.
3. In Rn , null(A) = (range(A> )) , and null(A> ) = (range(A)) .
4. Unitary: Q = Q1 .
5. Symmetric: A> = A.
(a) A real real eigenvalues, and is diagonalizable by real, orthogonal Q (i.e. D = Q> AQ).
(b) A1 symmetric iff A symmetric.
(c) If A, B both symmetric, then AB symmetric iff AB = BA (i.e. they commute).
6. Hermitian: A = A.
(a) Main diagonal is real.
(b) Has real eigenvalues.
(c) Normal, i.e. A A = AA .
7. Skew Hermitian: A = A. If A Rmn , then its skew symmetric.
Q
8. Determinant: det(A) = j j
P
P
9. Trace: trace(A) = j ajj = j j
10. Positive definite: x Ax > 0 x 6= 0, x Cm .
(a) Hermitian pos. def.: Positive definite with A = A.
(b) All eigenvalues real and positive
(c) For i 6= j , vi vj (eigenvectors are orthogonal).
(d) For HPD A, full rank X Cmn , then X AX is HPD
11. Spectral Radius: (A) = maxi (|i |), and (A) ||A||.
2.a.ii)

Norms

1. Must satisfy
(a) ||x|| 0, with ||x|| = 0 x = 0
(b) ||x|| = || ||x||
(c) ||x + y|| ||x|| + ||y||
2. All vector norms on Cn are equivalent, i.e. 0 < c1 < c2 < : c1 |||| |||| c2 ||||
3. Induced Matrix Norms: ||A||(m,n) = sup

xCn
x6=0

||Ax||(m)
||x||(n)

sup
||x||(n) =1

4. ||ABx|| ||A|| ||Bx|| ||A|| ||B|| ||x|| ||AB|| ||A|| ||B||

||Ax||(m)


1/2
X
2
5. Frobenius Norm: ||A||F =
|aij |
=
i,j

!1/2
X

2
||ai ||2

p
= trace(A A) =

sX

i2

P
6. Nuclear Norm: ||A|| = trace( A A) = i i
7. Unitary matrices Q satisfy ||QA||2 = ||A||2 , ||QA||F = ||A||F
2.a.iii)

Singular Value Decomposition

Given A Cmn , with m n

V , with U
Cmn ,
Rnn , and V Cnn ; U
, V unitary, and
diagonal with
1. Reduced SVD: A = U
0
decreasing elements.
2. Full SVD: A = U V with
U=

V =

V
(V )


,

0(mn)n

3. Finding the SVD:

(a) j s are the square roots of the eigenvalues of A A or AA .
(b) Find eigenvectors: (j I A A)vj = 0, or (j I AA )uj = 0.

1 .
=
1 AV , or V = A U
(c) Find the other matrix: U
4. Properties:
(a) r = rank(A) = #{j > 0}.
(b) range(A) = spanhu1 , . . . , ur i, null(A ) = spanhur+1 , . . . , um i.
(c) null(A) = spanhvr+1 , . . . , vn i, range(A ) = spanhv1 , . . . , vr i.
Pr
(d) A = j=1 j uj vj (for rank k < r approx, use the first k; gives min error in 2 and Frobenius norms).
2.a.iv)

Projectors

1. Projector: P 2 = P
2. Complimentary Projector: I P ; range(I P ) = null(P ) and null(I P ) = range(P )
3. Orthogonal Projector: P 2 = P and P = P .
aa
; Pa = I Pa .
a a
(b) Onto range of A (arbitrary basis): P = A(A A)1 A = AA+
(a) Rank 1 orthogonal projectors (a 6= 0): Pa =

2.a.v)

QR Factorization

For a matrix A Cmn , and m n

R,
with Q
Cmn unitary, R
Cnn upper triangular.
1. Reduced QR: A = Q
2. Full QR: A = QR with
Q=


,

R=

0(mn)n

3. Finding Q and R: Gram-Schmidt. By hand, use classical:



Pj1
j1


X
aj i=1 rij qi



qj =
rij = qi aj
rjj = aj
rij qi 


rjj
i=1


4. Modified Gram-Schmidt: A R1 R2 Rn = Q
|
{z
}
1
R


vv
I 0
where F = I 2 .
5. Householder Triangularization: Qn Qn1 Q1 A = R, with Qk =
0 F
{z
}
|
v v


6. Algorithms: Pseudocode.
Classical Gram-Schmidt:

Modified Gram-Schmidt:

for j = 1:n
Q[ : , j ] = A[ : , j ]
f o r i = 1 : j 1
R[ i , j ] = Q [ : ,
Q[ : , j ] = Q[ : ,
R[ j , j ] = norm (Q [ :
Q[ : , j ] = Q[ : , j ]

for i = 1:n
R[ i , i ] = norm (A [ :
Q[ : , i ] = A[ : , i ]
f o r j = i +1:n
R[ i , j ] = Q [ : ,
A[ : , j ] = A[ : ,

i ] A[ : , j ]
j ] R[ i , j ] Q [ : ,
, j ] , 2)
/ R[ j , j ]

i]

, i ] , 2)
/ R[ i , i ]
i ] A[ : , j ]
j ] R[ i , j ] Q [ : ,

i]

Householder QR:
% Note t h a t t h e o ut pu t i s R = A, and W m a t r i c e s
for i = 1:n
x = A[ i : , i ]
x [ 1 ] = s i g n ( x [ 1 ] ) norm ( x ) + x [ 1 ]
W[ i : , i ] = x / norm ( x , 2 )
A[ i : , i : ] = A[ i : , i : ] 2 W[ i : , i ] (W[ i : , i ] A[ i : , i : ] )
% To s o l v e , n o t e t h a t
% Rx = Qb ,
for i = 1:n
b [ i : ] = b [ i : ] 2 W[ i : , i ] (W[ i : , i ] b [ i : ] )
% Use back s u b s t i t u t i o n t o s o l v e f o r x
x = R\b

2.a.vi)

Least Squares

1. Normal Equations: A Ax = A b

= V
1 U
1 Q
2. Pseudoinverse: A+ = (A A)1 A = R
3. Solution methods:
(a) Cholesky: A A = R R, where R is upper triangular (requires full rank A). Best for speed, bad for error.
(b) QR: Reduces to Rx = Q b, use back substitution. Good method unless A is close to rank deficient.
V x = U
b, then solve. Stable even if A close to rank deficient, but more time/memory consuming.
(c) SVD:
2.a.vii)

Conditioning

||f ||
= ||J(x)||
||x||


||f || ||x||
||J(x)||
2. Relative condition number: = sup
/
=
||f
(x)||
||x||
||f
(x)||
/ ||x||
x

1. Absolute condition number:

= sup
x

3. Matrix-vector condition: =

||A||||x||
||Ax||



4. Matrix condition number: (A) = ||A|| A1  or ||A|| ||A+ || in rank deficient case.

2.a.viii)

Stability

Given problem f : X Y , and algorithm f : X Y , for all x X,





f (x) f (x)
1. Accuracy:
= O(m )
||f (x)||




f
(x)

f
(
x
)



||
x x||
= O(m ) and
= O(m )
2. Stability:
x:
||x||
||f (x)||
3. Backward stability: f(x) = f (
x) for some x
as above.
4. Both Householder triangularization is backward stable, and so is MGS provided Q b is formed implicitly.
2.a.ix)

Gaussian Elimination

1
1
1. No pivoting: Use for hand calcs, not stable. A = LU where L = L1
1 L2 Lm1 is lower triangular, U is
xjk
upper triangular. We have `jk = xkk for k < j m, and

Lk =

..

.
1
`k+1,k
..
.

1
..

`mk

`21
L=
.
..
`m1

0
1
..
.

..
.
..
.
`m,m1

0
..
.

0
1

2. Partial pivoting: Do row interchanges to maximize |xkk |.

U = Lm1 Pm1 L2 P2 L1 P1 A = (L0m1 L02 L01 )(Pm1 P2 P1 )A = L1 P A
1
1
where L0k = Pm1 Pk+1 Lk Pk+1
Pm1
. Solve to get P A = LU .

3. Full pivoting: (L0m1 L02 L01 )(Pm1 P2 P1 )A(Q1 Q2 Qm1 ) = L1 P AQ = U . Here, L and P are as
before, and Q = Q1 Q2 Qm is another permutation matrix. Solve for P AQ = LU .
4. Algorithms:
GE No Pivot:

GE Partial Pivot:

U = A; L = I
f o r k = 1 :m1
f o r j = k +1:m
L [ j , k ] = U[ j , k ] / U[ k , k ]
U[ j , k : ] = U[ j , k : ] L [ j , k ] U[ k , k : ]

U = A; L = I ; P = I
f o r k = 1 :m1
f o r j = k +1:m
S e l e c t i >= k : abs (U[ i , k ] ) i s maximized
U[ k , k : ] , U[ i , k : ] = U[ i , k : ] , U[ k , k : ]
P[ k , : ] , P[ i , : ] = P[ i , : ] , P[ k , : ]
f o r j = k +1:m
L [ j , k ] = U[ j , k ] / U[ k , k ]
U[ j , k : ] = U[ j , k : ] L [ j , k ] U[ k , k : ]

2.a.x)

Cholesky Factorization

1. Given Hermitian positive definite A, solve for A = R R, where R is upper triangular.




 

0
w /
a
w

0 1

= R1 Rn I Rn R1
A = 11
=
w K
w/ I
0
I
0 K ww
| {z } | {z }
a11
{z
}|
{z
}
|
R
{z
}|
R
R1

where =

R1

A1

a11 .

2. Algorithm:
Cholesky:
R = A
for k = 1:n
f o r j = k +1:n
R[ j , j : ] = R[ j , j : ] R[ k , j : ] R[ k , j : ] / R[ k , k ]
R[ k , k : ] = R[ k , k : ] / s q r t (R[ k , k ] )

2.a.xi)

Eigenvalues

1. Eigenvalue Decomposition: A = XX 1 , with X nonsingular, diagonal.

2. Characteristic Polynomial: PA (z) = det(zI A).
3. Similarity Transform: X Cmm nonsingular. A, B similar if A = X 1 BX.
4. Eigenvalue Multiplicity:
(a) Geometric: # of lin. indep. eigenvectors for .
(b) Algebraic: multiplicity of root of char. poly. for
(c) algebraic geometric
5. Defective/Degenerate: if algebraic mult. > geometric mult.
6. Nondefective: iff it has an eigenvalue decomposition.
7. Unitary Diagonalization: A = QQ , with Q unitary. A normal.
8. Schur Factorization: A = QT Q , with T upper triangular. Every square matrix has one.
2.a.xii)

Iterative Methods

1. Want to solve Ax = b starting with initial guess.

1
b}
2. Jacobi: (D L U )x = b Dx = (L + U )x + b x = D1 (L + U ) x + |D{z
{z
}
|
cJ

TJ

3. Gauss-Seidel: (D L U )x = b (D L)x = U x + b x = (D L)1 U x + (D L)1 b

|
{z
}
|
{z
}
TGS

4. Algorithms:
Jacobi:
D = d i a g ( d i a g (A) )
for i = 1: cIter
x = D \ ( (D A) x + b )

Gauss-Seidel:
D = d i a g ( d i a g (A ) ) ; L = t r i l (A ) ; U = t r i u (A)
for i = 1: cIter
x = (D L) \ ( b + Ux)

5. Converges if T converges, i.e. (T ) < 1.

10

cGS

2.b)

Numerical Methods

2.b.i)

Functional Iteration

Casting nonlinear system as a fixed point problem: x = g(x) for x Rn , and g(x) = (g1 (x), g2 (x), . . . , gn (x))> .
Iterating x(k+1) = g(x(k) ).




1. Contraction
Mapping:
If ||g(x) g(y)|| ||x y|| x, y : x x(0)  , y x(0)  with 0 <



1, and g(x(0) x(0)  (1 ), then limk x(k) = a where g(a) = a, and a unique (in this region).


 gi (x) 
st

 i, j = 1 : n and x B (a) = {x Rn : ||x a|| },
2. If gi (x) has continuous 1 order s: 

xj 
n
then x(0) B (a) x(k) a (unique).
3. Nonlinear Systems: g(x) = x A(x)f (x).
(a) Easy: A(x) = A.
(b) Newton: A(x) = J 1 (x) (inverse Jacobian of f ).
2.b.ii)

Polynomial Interpolation

1. Lagrange: Pn (x) =

n
X

f (xj )nj (x), where nj (x) =

j=0

(x xi )/

i6=j

Y
(xj xi ), where xj , j = 0 : n are known
i6=j

points.
def

2. Pointwise Error: Rn (x) = f (x) Pn (x) =

(xxj ) (n+1)
()
j=0 (n+1)! f

Qn

for some x0 < < xn .

3. Rolles Theorem: a point between 2 zeros where f 0 (z) = 0 for continuous f .

4. Numerical Differentiation/Integration: For f C([a, b]), xi [a, b], i = 0 : n then
f (x) = Pn (x) +
f 0 (xi ) =

n
X

n (x) (n+1)
f
()
(n + 1)!

f (xj )0nj (xi ) +

j=0

|
f (k) (xi )

n
X

{z

approx

n (x) =

n
Y

(x xj )

j=0

n0 (xi ) (n+1)
f
((xi ))
(n + 1)!
{z
}
|
err

(k)

f (xj )nj (xi )

j=0

f (x) dx
a

Z
n
X
j=0

nj (x) dx f (xj )

Rb
Pn
5. Weighted Least Squares: f L2 ([a, b]). Then Qn (x) = j=0 cj Pj (x) with cj = a f (x)Pj (x)w(x) dx miniRb
Rb
2
2
mizes ||f (x) Qn (x)||2 = a |f (x) Qn (x)| w(x) dx, with inner product hf (x), g(x)iw = a f (x)g(x)w(x) dx.
Rb
w(x) 0 on [a, b], and a w(x) dx > 0.
6. G-S Orthonormalization: Given {gi (x)}ni=0
d0 = 1/ ||g0 ||L2

f0 (x) = d0 g0 (x)
f1 (x) = d1 [g1 (x) c01 f0 (x)]

n1
X
fn (x) = dn gn (x)
cjn fj (x)

d1 = 1/ ||g1 c01 f0 ||L2

cij = hfi , gj i

j=0

11

7. Trig. Interpolation: On [, ] xk = kh, k = n : n, h = /n.

Un (x) =

n
X


1
cn einx + cn einx +
cj eijx
2
j=n

1
cj =
2n
2.b.iii)

n
X


1
f (xn )eijxn + f (xn eijxn +
f (xk )eijxk
2

!
j = n : n

k=n

ODEs

1. Initial Value Problems:

(
y0
= f (t, y)
y = [y1 , y2 , . . . , ym ]> , f = [f1 , f2 , . . . , fm ]> , c = [c1 , c2 , . . . , cm ]>
y(0) = c
autonomous: f (t, y) = g(y)
2. Given a system u(m) = g(t, u, u0 , . . . , u(m1) ), let y = (u, u0 , . . . , u(m1) ), and
(
yn0 = yn+1
n=1:m1
0
ym
= g(t, y1 , y2 , . . . , ym )
(t)| = |y(0) y
(0)| e<()t .
3. Stability: Test equation y0 = y, C. Solution: et y(0), t 0. Then |y(t) y
(a) Stable: <() 0.
(b) Assympytotically stable: <() < 0.
(c) Unstable: <() > 0.
2.b.iv)

Basic Concepts

1. Local Truncation Error: dn = Nh y(tn )

(a) Consistent (accurate) if dn 0 as hn 0 for all n
(b) dn = O(hpn ), p Z+ Nh is accurate order p
2. 0-Stability: If h0 , k > 0 : for all mesh fns xn , zn with h h0


|xn zn | k |x0 z0 | + max |Nh xn (tj ) Nh zn (tj )|
1jN

3. Absolute Stability: Test fn y0 = y, y(0) = c. Region in C satisfying |yn | |yn1 |

4. Stiffness: Require extremely small step size for explicit methods. Specifically, if b< << 1, where b is the
interval length.
5. A-Stable: Region of absolute stability includes {<(z) 0}.
6. Rough Problems: Cant bound derivatives by const. of moderate size. Need to break up solution at
discontinuities.
2.b.v)

Methods

1. Forward Euler: yn = yn1 + hn f (tn1 , yn1 )

2. Backward Euler: yn = yn1 + hn f (tn , yn )
3. Trapezoidal Method: yn = yn1 +

hn
(f (tn1 , yn1 ) + f (tn , yn ))
2

0
4. Taylor Series Method: y0 = f (t, y), yn = yn1 + hyn1
+

12

h2 00
2! yn1

5. Explicit Midpoint:
(
n1/2 = yn1 + h2 f (tn1 , yn1 )
y
n1/2 )
yn
= yn1 + hf (tn1/2 , y
6. RK Methods: For 1 i s
(

Yi
yn

Ps
= yn1 + h j=1 aij f (tn1 + cj , h, Yj )
Ps
= yn1 + h i=1 bi f (tn1 + ci h, Yi )

Ki
yn



Ps
= f tn1 + ci h, yn1 + h j=1 aij Kj
Ps
= yn1 + h i=1 bi Ki

(a) Tableau:
c1 a11
c2 a21
..
..
.
.
cs as1
b1
(b) Require: ci =

a12
a22
..
.
as2
b2

Ps

j=1

a1s
a2s
..
.

ass
bs

aij for i = 1 : s, and

Ps

j=1 bj

=1

(c) Explicit if aij = 0 for j i.

(` 1)!
, 1 ` + k p, where C = diag(c).
(` + k)!

(d) Order p if b> Ak C `1 1 =

(e) L-Stable: A-stable and |R(z)| 0 as |z| .

7. Linear Multistep Methods:
k
X

j ynj = h

j=0

k
X

j fnj

j=0

(a) Adams Family: 0 = 1, 1 = 1, j = 0 for j > 1.

(b) BDF: Derived from interpolating polynomial. With 0 = 1, this is

k
X

i yni = h0 f (tn , yn )

i=0

(c) Order: Order p if 0 = C0 = = Cp 6= Cp+1 , where

k
k
k
X
X
X
1
1
C0 =
j ,
Ci = (1)i
j i j +
j i1 j
i!
(i

1)!
j=0
j=0
j=0
(d) Char. Poly.s: () =

Pk

j=0

j kj , () =

Pk

j=0

j kj .

(e) 0-Stable: All roots of () satisfy |i | 1, and if |i | = 1 it is simple. Strongly stable if all |i | < 1
(except = 1), weakly stable if 0-stable but not strongly stable.
(f) Stability Region: z =

(ei )
(ei )

8. Predictor Corrector Methods: Use explicit multistep method to predict, then implicit method to correct.
2.b.vi)

BVPs

(
y0 = A(t)y + q(t)
0<t<b
B0 y(0) + Bb y(b) = b

13

1. Fundamental solution: Y 0 = A(t)Y , Y (0) = I. Gives general solution:



Z t
Z b
1
y(t) = Y (t) c +
Y (s)q(s) ds ,
Qc = b Bb Y (b)
Y 1 (s)q(s) ds,
0

Q = B0 + Bb Y (b)

unique solution iff Q nonsingular.

2. Greens Functions:
3. Shooting:
4. Finite Difference:

3)

Analysis

1. Pointwise Convergence:  > 0, x I, N (, x) : |fn (x) f (x)| < , n > N (x, ).
2. Uniform Convergence:  > 0, N () : |fn (x) f (x)| <  x and n > N (). ALT: limn ||fn f || = 0.

3.a)

Metric Spaces

1. A pair (M, d), M a set, d : M M [0, ) a function, satisfying

(i) d(x, y) = 0 x = y
(ii) d(x, y) = d(y, x)
(iii) d(x, z) d(x, y) + d(y, z)
2. Convergence: Given (M, d), {xn }, x. xn x if  > 0 N () : n > N () d(xn , x) < .
3. Equivalence of Metrics: Given (M, d), (M, d0 ), equivalent if  > 0, : B 0 (x, ) B(x, ), and 0 > 0,
0 : B(x, 0 ) B 0 (x, 0 ).
3.a.i)

`p Spaces

For 1 p the norm is given by

1/p

X
p
|xj | ,
||x|| = sup |xj |
||x||p =
0j<

j=0

`p = {x : ||x||p < }. If 1 p < q , then `p `q (strict).

1. H
olders Inequality: 1 p, q with 1/p + 1/q = 1. Then for x `p , y `q ,

|xj yj |

j=0

1/p
p
|xj |

j=0

1/q
q
|yj |

j=0

2. Jensens Inequality: f : [a, b] R convex (i.e. f (px

P1 + (1 p)x2 ) pf (x1 ) + (1 p)f (x2 ) for x1 < x2 ,
0 < p < 1), a x1 xn b, and pi (0, 1) with i pi = 1 then

n
n
X
X

pj f (xj )
f
pj xj
j=1

j=1

3. Minkowski Inequality: For x, y `p , ||x + y||p ||x||p + ||y||p .

P
4. Converse of H
older: If c > 0 :
ak xk c ||x||p x Rn , then ||a||q c.
5. Continuity: Metric spaces (M, d), (N, d0 ), function f : M N . f is continuous at x0 M if  > 0, > 0 :
d(x, x0 ) < d0 (f (x), f (x0 )) < .
14

Lebesgue (Lp ) Spaces

3.a.ii)

For 1 p and interval X, the norm is given by

Z
||f ||p =

1/p
,
|f (x)| dx
p

||f || = sup |f (x)|

xX

Lp (X) = {f : ||f ||p < }. If 1 p < q and X finite, then Lq Lp (strict).

1. If 1 r < s and X finite, {fn } Ls (X), then ||fn f ||s 0 ||fn f ||r 0.
2. If fn f pointwise and ||fn ||p ||f ||p , then fn f in Lp .
3.a.iii)

Normed Linear Spaces

1. Norms: See norms. A seminorm doesnt require x = 0 for (x) = 0.

2. Inner Product Space: A function F (x, y) : X X R is an inner product if
(a) F is linear in each argument
(b) F (x, x) 0 and F (x, x) = 0 iff x = 0
(c) F (x, y) = F (y, x)
2

A norm is derived from an inner product iff the parallelogram law holds, i.e. ||x + y|| + ||x y|| =
2
2
2 ||x|| + 2 ||y|| .

3.b)

Topology

Topological Space: Set X, collection of open subsets T that satisfy

(i) , X T
(ii) U, V T U V T
[
(iii) {U }I T
U T

3.b.i)

Basic Definitions

1. Open Ball: B(x, ) = {y : d(x, y) < }.

2. Open: If x U,  : B(x, ) U , then U open.
(a) Finite intersections of open sets are open
(b) All unions of open sets are open
3. Closed: If complement is open.
(a) Finite unions of closed sets are closed
(b) All intersections of closed sets are closed
4. Interior Point: x A X. x is an interior point if open Ux A.
5. Interior: A = { all interior points of A }.
6. Point of Closure: A X, x X is a point of closure of A if open Ux , Ux A 6= .
7. Accumulation Point: (or limit point) if Ux (A\{x}) 6= .
8. Closure: A X, and F(A) = {F : F closed, A F }. Then the closure is A = F F (A) F .
ALT: A = { all points of closure }
(a) A A
15

(b) A B A B
(c) A = A
(d) A B = A B
(e) =
9. G : Countable intersection of open sets.
10. F : Countable union of closed sets.
3.b.ii)

Defining Topologies and Continuity

1. Base: Collection B = {B(x, ) : x X,  > 0} with

(a) BB = X
(b) x B1 , B2 B3 B1 B2 : x B3


2. Sub-base: B0 X with BB0 B = X. Then B = nk=1 Bk0 B0 is a base for the topology.
3. Weak topology: X has two topologies T , S. S weaker than T means S T .
4. Local base:
5. Continuity: Two topological spaces (X, T ), (Y, S), function f : X Y . f is continuous at x0 if V Y
open, with f (x0 ) V , U X with x0 U , and x U f (x) V .
(a) A function is continuous iff xn x f (xn ) f (x).
(b) If f continuous, then f 1 (open) = open.
6. Convergence: xn x if U 3 x, N : n N xn U .
7. Connected: (X, T ) connected if no nonempty sets A, B X with A B = X, and A B = .
(X, T ) connected and f : X Y continuous (Y, S) connected.
8. Hausdorff Space: For every x, y X, x 6= y, open U, V , with x U , y V , U V = .
(a) Convergent sequences have unique limits
(b) Complement of {x} is open.

3.c)

Distributions

Topological linear space of functions D. Distributions are complex-valued continuous (with respect to D) linear
functions D0 . I.e. if n in D, then hT, n i hT, i.
3.c.i)

Basics

1. Taylors Formula: (N + 1)-times differentiable on [M, M ], N +1 continuous.

(x) =

Z
N
N
X
X
xN +1 1
1 (j)
1 (j)
(0)xj +
(1 u)N (N +1) (xu) du =
(0)xj + xN +1 (x)
j!
N
!
j!
0
j=0
j=0

where (x) is continuous, and has as many derivatives as .

R
2. Improper Integrals: f C(R). Then R f (x) dx converges if
Z
lim

lim

A B

Z
f (x) dx

or

lim

f (x) dx + lim
A

exist and are finite, with C arbitrary.

16

f (x) dx
C

3. Principal Value: Converges if

Z

lim

f (x) dx
R

exists and is finite.

4. Support: supp() = {x : (x) 6= 0}. Support is compact if M : supp() [M, M ].


1
5. Bump Function: f (x) = exp
on (a, b) and 0 else.
(x a)(x b)
6. Distribution: T : D C satisfies
(a) T is complex linear: T (1 + 2 ) = T (1 ) + T (2 ) , C and 1 , 2 D.
(b) T is continuous WRT the topology on D.
3.c.ii)

Specific Distributions
"Z
#
 
Z
Z
(x)
(0)
(x) (0)
1
def
dx = lim
dx +
dx
() = lim
1. Principal Value: P V
0 |x|>
0
x
x
x
|x|> x
|x|>

1
2.
= PV
x i0
3.c.iii)

 
1
i(x)
x

Function Spaces

1. C : infinitely differentiable functions (smooth).

2. C0 : subset of C with compact support.
k
3. CN
: k times continuously differentiable with support in [N, N ].
k

4. Schwartz space S : subset of C with limabsx |x| |D (x)| = 0 for all k, .

5. C0 Convergence: n if:
(a) supp(n ) [N, N ] independent of n.
(m) n

(b) Derivatives n

(m) uniformly, i.e. lim

sup

n x[N,N ]



 (m)

n (x) (m) (x) 0

k
ALT: T D0 iff N , B(N ), k(N ): |hT, i| B ||||N,k CN
.

6. Order: Smallest k independent of N : above convergence definition holds.

def

7. Norm: ||||N,k =

k
X





sup (j) (t)

j=0 [N,N ]
def

8. Topology: d(1 , 2 ) =

||1 2 ||N,k

k=0

1 + ||1 2 ||N,k

1
2k

9. Convergence: {Tj } D0 . If lim hTj , i = hT, i D, then Tj converges to T .

j


10. Derivatives: T (n) , = (1)n T, (n) .

17

3.d)

Measure Theory

3.d.i)

Basic Definitions

1. Dense: A B. A is dense in B if B A. A set is dense iff (Ac ) = .

2. Nowhere Dense: Interior of closure is empty: (A) = . A nowhere dense iff (Ac ) = X.
3. (A )c = Ac and (Ac ) = (A)c .
4. First Category: Countable union of nowhere dense sets (also meager).
5. Residual: Complement of first category (also comeager). Residual sets are dense.
6. Second Category: Not first category.
7. Measure Zero:  > 0, {Bn } countable collection of open balls Bn = B(xn , n ) such that
A

Bn

and

n=1

volume of Bn 

n=1

8. Baire Category Theorem in R:

(a) The complement of a 1st category set is dense.
(b) The intersection of countably many open dense sets is dense.
9. Cantor Intersection Theorem: Given nonempty closed and bounded {Cn } with C0 C1 Cn ,
then Ck 6= .
3.d.ii)

Measurable Spaces

1. (X, B), set X, collection of subsets B satisfying

(i) X B, B
(ii) A B Ac B
Sn
(iii) {Aj } B j=1 B
This defines an algebra of sets, and if n = a algebra.
2. Measurable: (X, B), (Y, C) measurable spaces. Then f : X Y is measurable if A C f 1 (A) B.
3. Borel algebra: Given (X, T ), the algebra generated by T .
4. Additive Measure: on (X, B). : B [0, ]
(a) () = 0
Sn
Pn
(b) {Ak } B mutually disjoint ( k=1 Ak ) = k=1 (Ak ) where n can by .
3.d.iii)

Probability Spaces

1. A measurable space + a measure, (X, B, P ), with P : B [0, 1].

2. Borel-Cantelli:
(a) Given (X, B, P ) (P (X) = 1). Let E1 , E2 , . . . be events ( B). Then if

P (Em ) < ,

P (Em i.o.) = P (lim sup Em ) = 0

m=1

P (En ) = , then P (En i.o.) = 1.

Qk
3. Independent: E1 , . . . , En are independent if P (Ei1 . . . Eik ) = j=1 P (Eij ).
(b) If E1 , E2 , . . . independent and

n=1

4. Borel Zero-One Law: En independently often (i.o.) can only have probability 0 or 1.
18

3.e)
3.e.i)

Convergence and Compactness

Basic Definitions

In a metric space (M, d).

1. Cauchy Sequence: If  > 0, N () : m, n > N d(xm , xn ) < , then {xn } is Cauchy. If xn x, then
{xn } is Cauchy.
2. Complete: Every Cauchy sequence converges to a limit x M .
(a) Rn is complete.
(b) `p is complete.
(c) Lp is complete.
(d) Closed subspace of complete space is complete.
3. Contraction Mapping Theorem: (M, d) complete metric space. f : M M continuous, and 0 < k < 1
such that d(f (x), f (y)) kd(x, y) x, y M . Then there exists a unique z M for which f (z) = z.
4. Sequential Compactness: In (X, T ). A X is compact if every sequence {xn } A has a convergent
subsequence with limit in A.
5. Compactness:
(a) In Rn , compact = closed and bounded.
(b) In (M, d), compact = complete and totally bounded or = sequentially compact.
(c) In (X, T ) compact if every open cover has a finite subcover.
(d) Compact sets are closed and bounded.
(e) Closed subsets of compact sets are compact.
(f) Compact sets are complete.
(g) f : M R continuous and M compact, then f assumes its max and min values.
6. Net:  > 0, finite collection x1 , . . . , xn such that nj=1 B(xj , ) M .
7. Totally Bounded: M is totally bounded if there is an net for every  > 0.
8. Open Cover: (X, T ), A X, O = {U }I (open sets), and A I U .
9. Finite Subcover: Finite subset of O that covers A.

3.f )
3.f.i)

Uniformity
Basic Definitions

1. Uniformly Continuous: f : (M, d) (M 0 , d0 ). If  > 0, > 0 : d(x, y) < d0 (f (x), f (y)) < .
If A M is compact and f : M M 0 is continuous, then f is uniformly continuous on A.
2. Uniform Convergence: fn : (M, d) (M 0 , d0 ).
(a) fn converges to f if , x, N (x, ) : n > N (x, ) d0 (f (x), fn (x)) < .
(b) fn converges uniformly to f if , N () : n > N () d0 (f (x), fn (x)) <  x.
(c) {fn } is Cauchy if , x, N () : n, m > N (x, ) d0 (fn (x), fm (x)) < .
(d) {fn } is uniformly Cauchy if , N () : n, m > N () d0 (fn (x), fm (x)) <  x.
3. fn : M M 0 , M 0 complete. fn converges Uniformly iff it is uniformly Cauchy.
P
4. A M , fn : A R. n=0 fn (x) converges uniformly on A if the sequence of partial sums converges uniformly.
P
5. Weirstrass
M-test: (for uniform convergence). If Mn 0 : |fn (x)| Mn x A, and n=0 Mn < ,
P
then n=0 fn converges uniformly.
19

3.f.ii)

Interchanging Limits and Integrals

Given fn f uniformly, fn continuous.

1. limh0 limn fn (x0 + h) = limn limh0 fn (x0 + h), i.e. f is continuous.
RP
PR
2.
fn =
fn .
3. Given fn f (uniform not required) on [a, b], fn0 continuous, and fn0 g uniformly. Then f 0 = g.
3.f.iii)

More on Integrals

1. Riemann integrable: Define Mi = max{f (t) : ti t ti+1 }, mi = min{f (t) : ti t ti+1 }, and U (f ; ) =
n
n
X
X
Mi (ti+1 ti ), L(f ; ) =
mi (ti+1 ti ) for some partition . If inf all U (f ; ) = supall L(f ; ) then
i=0

i=0

f is Riemann integrable.
(a) f on [a, b] is R-integrable if it is continuous.
(b) f on [a, b] is R-integrable iff the set of points of discontinuity of f has Lebesgue measure zero.
(c) Riemann integral exists Lebesgue integral exists, and they are equal (not converse).
2. Lebesgue integrable: Suppose f bounded, M, f (t) M and partition range: M 1 = y0 < y1 < <
n
X
yn < yn+1 = M + 1. Let i be the length of {t : yi f (t) < yi+1 }. Then the Lebesgue sum is:
yi i .
i=0

Alternatively, define
(
f (x)
f+ (x) =
0

if f (x) 0
else

f (x) =

Z
Then f (x) = f+ (x) f (x), and

Z
f+ d

f d =
X

f (x) textif f (x) 0

0
else

f d (provided at least 1 of the two integrals on

X

the right is finite).

3. Simple Function: A1 , . . . , An X pairwise disjoint measurable sets. (x) =

n
X

j cAj (x) with j 0 is a

j=1

Z
non-negative simple function, and has integral

d =
X

n
X

j (Aj ).

j=1

4. Measurable Function: (X, B) a measurable space. f : X [, ] is measurable if {t : f (t) < } B for

each R. Limit of measurable functions is measurable.

3.g)
3.g.i)

Convergence Theorems
Basic Definitions


def

1. Lim inf: lim inf xn = lim

n

mn

def

2. Lim sup: lim sup xn = lim

n


inf xm , alternatively the leftmost limit point (or ).


sup xm , alternatively the rightmost limit point (or ).
mn

3. Markovs Inequality: (X, B, ), f : X [, ] measurable.  > 0, ({x : |f (x)| > )

1


4. Chebyshevs Inequality: (X, B, ), f : X [, ] measurable.  > 0, ({x : |f (x)| > )

20

Z
|f | d.
X

1
2

|f | d.
X

3.g.ii)

Theorems
Z

1. Fatous Lemma: {fn } sequence of non-negative measurable functions. Then

lim inf fn d = lim inf

X

fn d.
X

2. Monotone Convergence: {fn } sequence of non-negative

measurable
functions,
Z
Z
Z and for almost every x,
{fn (x)} is nondecreasing with limit f (x). Then lim
fn d =
lim fn d =
f d.
n

3. Corollary: gn 0 measurable. Then

Z X

gn d =

X n=1

X n

X
Z
X
n=1

gn d.

4. Lebesgue Dominated Convergence:

(X, B, ),
and fn f a.e.. Suppose g L1 :
Z
Z {fn } measurable,
Z
|fn (x)| |g(x)| a.e., n. Then lim
fn d =
lim fn d =
f d.
n

4)
4.a)
4.a.i)

X n

Principals and Methods

Dynamics of Nonlinear Systems
Dimensional Analysis

d
x
=
1. Set x = L
x, t = T t, u = C u
, where is dimensionless. Then x = x dx

1
,
L x

t = t ddtt =

1
T

t.

2. Discrete Symmetries: Example: sign invariance, i.e. given solution u to differential equation, u is also a
solution.
3. Continuous Symmetries: Translation invariance, e.g. t t + , x x + .
4. Traveling Wave: Both time and spatially invariant. u(x, t) = u(x ct) = u(z).
5. Scaling Symmetry: u (x, t) = c u(a x, b t), with z(x, t) = z(a x, b t).
6. Buckingham Pi: Given f (x1 , . . . , xn ) = 0 of n physical variables in k physical units, can restate as F (1 , . . . , p ) =
n
Y
a
0, where i =
xj j , and p = n k.
j=0

7. Symmetries Reductions:
(a) Time translation (t t + ) steady solutions (u(x, t) = u(x))
(b) Space translation (x x + ) homogeneous solutions (u(x, t) = u(t))
(c) Rescaling symmetries (u u ) self similar solutions (u(x, t) = tc/a u(x/ta/b ))
4.a.ii)

Phase Plane

1. Potential Systems: uz = v, vz = f (u).

R
R
(a) Potential V (u) = f (u) du = uzz du. Get phase plane from graph of potential.
(b) Energy E(z) = V (u) + 21 u2z
 
 
u
d u
2. Linearization of FPs: dz
= A
where A is the linearized Jacobian evaluated at the fixed point.
v
v
Pn
General solution for x = Ax is x(t) = i=1 ci ei t vi where Avi = i vi .

21

(a) det(A) = 1 2 , tr(A) = 1 + 2 for 2 2 case.

(b) 0 < 1 < 2 < 0 unstable/stable node
(c) 1 < 0 < 2 saddle
(d) 0 < 1 = 2 < 0 unstable/stable improper node
(e) 1 = 2 , and 0 < <() < 0 unstable/stable spiral
(f) 1 = 2 , and <() = 0 elliptic FP/center
3. Heteroclinic Orbit: connects two fixed points
4. Homoclinic Orbit: connects a fixed point to itself
4.a.iii)

Dispersion Relations

4.b)

Contour Integration

4.b.i)

Complex Basics

1. Cauchy-Riemann: The function f (z) = u(x, y) + iv(x, y) is differentiable at z = x + iy iff ux = vy and

vx = uy , and all partials are continuous in a neighborhood of z.
2. Analytic: At a point if differentiable at that point; in a region if analytic at every point in the region.
3. Entire: Analytic at every point in C except .

4. Singular Point: Where f is not analytic.

4.b.ii)

Integration
I

f (z)
dz.
k+1
C (z z0 )
I

X
f (z)
1
n
2. Laurent Series: f (z) =
cn (z z0 ) where cn =
dz.
2i
(z

z0 )n+1
C
n=

1. Cauchys Integral Formula: f (k) (z0 ) =

k!
2i

(a) Strength: cn for pole of order n.

H
1
(b) Residue: c1 = 2
f (z) dz
C
(c) Essential singularity: number of cn terms.

dk1 
1
f (z)(z z0 )k for a pole of order k.
zz0 (k 1)! dz k1

  
1
1
4. Residue at : Res (f (z), ) = Res 2 f
,0
z
z
3. Residue: Res (f (z), z0 ) = lim

I
5. Cauchys Residue Theorem: For a simple closed contour C,

f (z) dz = 2i
C

N
X

rj , where rj are residues

j=1

of poles in the interior of C.



6. Jordans Z
Lemma: If f (z) 0 uniformly on Rei , 0 , and f (Rei  G(R), and limR G(R) = 0,
then lim
eikz f (z) dz = 0, for k > 0. Take lower arc for k < 0.
R

CR

7. Branch cuts: Let x = ze2i .

22

4.c)

Fourier Series

Given a 2L periodic function f

 n 
 n 
 n 
X
a0 X
an cos
x + bn sin
x
or
=
x
cn exp
2 n=1
L
L
L
n=
Z
Z
Z L
1 L
1
1 L
f (x) cos(nx) dx
bn =
f (x) sin(nx) dx
cn =
f (x)einx dx
an =
L L
L L
2L L

f (x) =

a0 = 2c0 ;
4.c.i)

an = (cn + cn );

bn = i(cn cn );

cn = (an ibn )/2

Hilbert Space

A complete, normed linear space whose norm comes from an inner product.
1. Inner Product: Satisfies hf, f i 0 with = iff f = 0, hf, gi = hf, gi, hf + g, hi = hf, hi + hg, hi,
hf, gi = hg, f i.
Rb
2. For L2 [a, b], hf, gi = a f (x)g(x) dx.
2

3. Norm: ||f || = hf, f i.

4. Has a dense orthonormal basis, for L2 [a, b] : n = e2inx/(ba) / b a

4.c.ii)

Theorems

1. Riemann-Lebesgue: Given f L1 [a, b], limn |cn | = 0. More specifically, for f C n [a, b] (actually only
c
need f (n) L1 [a, b]), |ck |
n for |k| 1, and constant c.
|k|
2

2. Parsevals Identity: For f L2 [a, b], ||f ||L2 = 2

|ck | .

k=
2

3. Carlsons Theorem: f L Sn (f ) f pointwise almost everywhere, where Sn is the partial sum of the
Fourier series.
4. For f (x) periodic, piecewise smooth on [a, b], and integrable, then in (a, b), limn Sn (f ) = f (x) where f is
continuous, and limn Sn (f ) = 12 (f (x+ ) + f (x )) where f is discontinuous (convergence may break down at
endpoints).
5. For f (x) periodic, continuous, and piecwise smooth on [a, b], then Sn (f ) f uniformly. The convergence is at
the rate ||Sn (f ) f || = O(n1p ), where f (p) L1 [a, b] (or O(np ) for f C p [a, b]).
6. Gibbs Phenomenon: Apporximately 10% error near discontinuities in function.
7. Can integrate term by term to get Fourier series of F (x) (but need to make it periodic), and differentiate term
by term to get Fourier series of f 0 (x) on the condition that f C, f 0 L1 .

4.d)

Distributions

Also see distributions.

4.d.i)

Basics

 n

 d

1. Null sequence: In C0 (R), {m (x)} such that lim
sup  n m (x) = 0 for all n Z+ . In S(R),
m x[K,K] dx



 k dn
{m (x)} such that lim sup x
m (x) = 0 for all k, n Z+ .
m xR
dxn




 dm 

dm 
2. Seminorms: Examples are pm,k () = sup xk m , and qm,k () = sup (1 + x2 )k m .
dx
dx
xR
xR
23

3. Continuity: Given {m } , then T continuous if T [m ] T [].

4. To prove T is a distribution, show that T [ + ] = T [] + T [] (linearity) and T [m ] 0 for all null
sequences (continuity). Break up integral and use seminorms to prove.

5. If f Lloc
1 , then Tf is a distribution in C0 and S.

6. For smooth, then T [] = T [].

Z
Z
T (z)(y 1 (z))
dy, where y(x) is the change of
7. Change of Variables: T[]
T (y(x))(x) dx
|y 0 (y 1 (z))|
variables.
8. Convolution: Tf ? Tg = Tf ?g . If Tfn Tf , Tgn Tg , then Tfn ? Tgn S where S depends only on Tf , Tg
(not n).

4.e)

Fourier Transforms
Z

Z
h
i
1
fb(k)eikx dk
F 1 fb(k) = f (x) =
2

f (x)eikx dx

F [f (x)] = fb(k) =

4.e.i)

Common Functions/Distributions



1. Sinc and box: F [sinc(x)] = [1,1] , and F [1,1] = 2 sinc(k)
h
i


2
2
2. Gaussian: F ex /a = a exp ak
4
3. Sin and cos: F [sin(ax)] = i [(k + a) (k a)] and F [cos(ax)] = [(k + a) + (k a)]
4. Delta: F [(x)] = 1, and F [1] = 2(k)
5. Heaviside: F [H(x)] = (k) iP V

6. Principal Value and sign: F P V
4.e.ii)

1
k
1
x



= isign(k), and F [sign(x)] = 2iP V

1
k

Other stuff

1. Poisson Sum:

(x 2n) =

n=

1 X ikx
e
2

|a|

hR
x

i
f (t) dt =

(nT ) =

n=

k=

2. fb(k) = fb(k), f R fb Hermitian, i.e. fb(k) = fb(k);

3. F [f (x b)] = eikb fb(k);
F [f (ax)] = 1 fb k


4. F f (n) = (ik)n fb(k);




1 X
2k

b
T
T
k=

b
fb(x) = 2f (x)

1 b
ik f (k)

+ fb(0)(k)

5. F [xn f (x)] = in fb(n) (k)

6. f L1 fb is bounded.
7. F : L1 (R) C(R), F : L2 (R) L2 (R)
8. Parsevals/Plancherels Theorem: If f, g L1 (R), then

D
E
2
fb, gb = 2 hf, gi, so ||f ||2 =

f L2 (R)).
h
i
9. Convolution: F [f ? g] = fb gb, and F 1 f ? g = 2f g.
Z
10. Multi dimensions: F [f (x)] = fb(k) =
f (x)eikx dx;
Rn

cf [] = T b[] = Tf []
11. FT of Distributions: T
b
f
24

h
i
F 1 fb(k) = f (x) =

1
(2)n

Z
Rn

1
2

 2
 b
f  (for
2

fb(k)eikx dk

4.e.iii)

Sampling

1. Nyquist Rate: 0 /, where fb() = 0 for || > 0

2. Sampling frequency: 1/ > Nyquist rate, i.e. < /0 .
3. Shannon sampling:
4.e.iv)

Other Transforms

? P V (1/x), so F [H[f ]] = 1 fb(k) F [P V (1/x)] = isign(k)fb(k).

Z
Z c+i
1
st
f (t)e
dt;
f (t) =
F (s)est ds
2. Laplace Transform: L [f ] = F (s) =
2i ci
0

1. Hilbert Transform: H[f ] =

4.f )

1
f

Spectral Theory

4.f.i)

Basics

1. Operator: L : X X. Defined on the domain D(L).

||Lu||
. This implies ||Lu|| ||L|| ||u||. Bounded continuous.
uX ||u||
2
2
Find bound by bounding ||Lu|| = hLu, Lui c ||u|| , or show unbounded via sequence of un s.

2. Bounded: If ||L|| , where ||L|| = sup

3. Adjoint: An operator L such that hLu, vi = hu, L vi. If L is bounded and linear, then L exists and is
bounded and linear. Also, L = L, and ||L|| = ||L ||.
4. Normal: L L = LL .
5. Self Adjoint: L = L. formally if they have different domains. All eigenvalues are real. No residual
spectrum.
6. Resolvent: Complete normed space X 6= {0} and linear operator L : D X, D X. The resolvent is
R (L) = (L I)1 .
7. Regular value: such that R (L) exists, is bounded, and is defined on a dense set in X.
8. Point Spectrum: p (L) = { : (L I)1 d.n.e.}. Eigenvalues of L, Lu = u. p (L) p (L ).
9. Continuous Spectrum: c (L) = { : (L I)1 exists and is unbounded}.
10. Residual Spectrum: c (L) = { : (LI)1 exists, possibly bounded, but not defined on a dense set of X}.
11. Rayleigh Quotient: =
4.f.ii)

hLu, ui
.
hu, ui

Sturm-Liouville

1. Lu =

1
((p(x)ux )x + q(x)u(x)) on some domain [a, b]
(x)

2. Always self adjoint.

3. Eigenvalues 1 < 2 < , and eigenfunctions are orthogonal, complete basis.

25

4.g)

Greens Functions

4.g.i)

Variation of Parameters


u u2 
 = u1 u02 u2 u01 , for solutions Lu1 = Lu2 = 0 of the S-L operator.
1. Wronskian: W (x) = det  10
u1 u02 

2. v10 (x) =

u2 (x)f (x)
;
p(x)W (x)

u1 (x)f (x)
.
p(x)W (x)

v20 (x) =
Z

u2 (t)f (t)
dt + u2 (x)
p(t)W (t)

3. up (x) = u1 v1 + u2 v2 = u1 (x)
0

Z
0

u1 (t)f (t)
dt
p(t)W (t)

4. u(x) = uh (x) + up (x).

4.g.ii)

Greens Functions

Assuming S-L operator L = (pu0 )0 + qu.

1. Satisfies LG(x, t) = (x t) as well as boundary conditions of L, and G is continuous at x = t.
2. Jump condition: Gx (t+ , t) Gx (t1 , t) =

1
p(t)

3. General solution: find 2 linearly indep. homogenous solutions u1 , u2 . Then

(
a1 (t)u1 (x) + a2 (t)u2 (x)
x<t
G(x, t) =
b1 (t)u2 (x) + b2 (t)u2 (x)
x>t
Use BCs, continuity, and jump to find a1 , a2 , b1 , b2 . Solution of Lu = f on [c, d] is
Z
u(x) =

Z
G(x, t)f (t) dt =

Gb (x, t)f (t) dt +

c

Ga (x, t)f (t) dt

x

26