Math preliminaries: definitions and results

Linear space
A linear (or vector) space H over field R is a set H with two
operations + : H H H and : R H(or H R) H with
the following properties:
a) Closure: x, y H, a R, x + y H, a x H.
b) Associative: x, y , z H, x + (y + z) = (x + y ) + z.
c) Commutative: x, y H, x + y = y + x.
d) Identity: 0 H, x Hx + 0 = x.
e) Inverse: x H x, x + (x) = 0.
f) Scalar multiplication compatibility:
x H, a, b R, a (b x) = a b (x).
g) Multiplicative identity: 1 R, x H, 1 x = x.
h) Distribution of over +:
x, y H, a R, a (x + y ) = a x + a y .
i) Distribution of scalar addition:
a, bR, x H, (a + b) x = a x + b x.

Norm
A norm is a function k k: X R (where X is a linear space)
with the following properties: below x, y X are any vectors and
c R is any scalar.
a) k x k 0.
b) k x k= 0 iff x = 0.
c) k c x k= |c| k x k.
d) k x + y kk x k + k y k.

Example 1
i) X = R, k x k= |x|.
q
ii) X = Rn , k x k= x12 + . . . + xn2 - Euclidean norm.
P
iii) X = Rn , k x k1 = ni=1 |xi | - L1 norm.
iv) X = Rn , k x k = max(x1 , . . . , xn ) - L norm.

Inner product
Any inner product is a bilinear function h, i : X X R (where
X is a linear space) with the following properties (x, y , z X are
any vectors in X and c R is any scalar):
i) hx, xi 0 .
ii) chx, y i = hcx, y i = hx, cy i.
iii) hx, y i = hy , xi.
iv) hx + y , zi = hx, zi + hy , zi. Also, hx, y + zi = hx, y i + hx, zi.
v) ? non-degeneracy only for strict inner product. hx, xi = 0 iff
x = 0.
The field R can be substituted by the field of complex numbers
(and using conjugates for symmetry).

Metric spaces - 1
Let M be a metric space with metric d. For x M, r > 0
Br (x) = {y M | d(x, y ) < r }
is called the Open ball/neighbourhood around x.
x M is a limit point of S M if Br (x), y S, y 6= x and
y Br (x).
If x S and x is not a limit point of S then x is an isolated point.
x M is a boundary point of S if every open ball Br (x) contain
a point y S and a point z
/ S.
If S contains all its boundary points S is said to be closed.
x S is an interior point if r > 0, Br (x) S.
S is open if every x S in an interior point.

Metric spaces - 2
A collection
of sets C = {C1 , C2 , . . .} is a cover of S if
S
S Ci C Ci . If each Ci is an open set then C is an open cover.
If C 0 C and C 0 is a cover then C 0 is called a sub-cover.
S is compact if every open cover of S contains a finite sub-cover.
S is sequentially compact if every sequence in S contains a
convergent sub-sequence.
S is dense in M if every point of M is a limit point of S.
Notation: Closure(S)=clo(S) or S c ; Interior(S)=int(S) or S o .
S is bounded if r R s.t. S Br (x) for each x S

Metric spaces - 3

Theorem 2
S M is closed iff S is open (S is complement of S).

Theorem 3
Let {Si }iN be a collection of open sets and {Cj }jN be a
collection of closed sets. Then:
S
a) i Si is open.
T
b) j Cj is closed.
T
c) If {Si } is a finite collection then i Si is open.
S
d) If {Cj } is a finite collection then j Cj is closed.

Metric spaces - 4

Theorem 4
If S is a compact subset of M (a metric space) then S is closed.

Theorem 5
A subset S M of a metric space is compact iff it is sequentially
compact.

Theorem 6 (Heine-Borel)
A subset S Rn is compact iff it is closed and bounded.

Metric spaces - 5
A function f : M1 M2 where (M1 , d1 ) and M2 , d2 ) are metric
spaces is uniformly continuous if for any  > 0, > 0 s.t.
x, y M1 , d1 (x, y ) < implies that d2 (f (x), f (y )) < .
f is said to be Lipschitz continuous if L R, L 0 and
d2 (f (x), f (y )) Ld1 (x, y ). L is called the Lipschitz constant.

Theorem 7
Every Lipschitz continuous function is uniformly continuous.
A set of functions S C [a, b] is equicontinuous at x [a, b] if
for any  > 0, > 0 s.t. for y [a, b] if |x y | < then
|f (x) f (y )| <  for every f S. If S is equicontinuous for every
x [a, b] then S is equicontinuous.

Metric spaces - 6
A sequence {xi }i N in a metric space M is a Cauchy sequence
if for any  > 0 there is an integer N > 0 such that d(xm , xn ) < 
whenever n, m > N. Any convergent sequence is a Cauchy
sequence. A set S M is complete if every Cauchy sequence in
S converges to a point in S. A complete normed linear space is
called a Banach space. A complete inner product space is called a
Hilbert space.
A metric space M is separable if it contains a countable dense
subset.

Theorem 8
A Hilbert space is separable iff it has a countable orthonormal
basis.
All infinite dimensional Hilbert spaces are isometrically isomorphic
to `2 .

Function spaces - 1
A linear space F that has functions as elements is called a
function space. Addition and scalar multiplication is defined
pointwise. A Sequence can be defined as a function on N. So
sequence spaces can also be thought of as function spaces.
Let [a, b] be a closed, bounded interval of R. Define
F = {f | f : [a, b] R and L > 0 s.t. |f (x)| L, x [a, b]}
F is a linear space. The sub-space C [a, b] F defined by:
C [a, b] = {f F | f is continuous.}
is the sub-space of all continuous functions on [a, b].

Function spaces - 2

A norm on F and C [a, b] is k F k = sup(|f (x)| | x [a, b]). With

Linear algebra - 1
For any linear space S (with field F ) the linear
Pk combination of
vectors s1 , s2 , . . . , sk S is the vector v = i=1 i si where
i F .
A set of vectorsPs1 , s2 , . . . , sn S is linearly dependent if
n
i F , s.t.
i=1 i si = 0 where not all i = 0. The set
{s1 , . . . , sn } is linearly independent if it is not linearly dependent.
Set of vectors {s1 , . . . , sn } span a linear space S if s S, s can
be written as a linear combination of vectors in {s1 , . . . , sn }.
If {si , . . . , sm } is a linearly independent set of vectors in S and it
spans S then it is called a basis.
A linear space S is finite dimensional if it has a finite basis. The
number of vectors in the basis is called the dimension of S.

Linear algebra - 2

Lemma 9
If {s1 , . . . , sn } span the linear space S and t1 , . . . , tm S is any
linearly independent set of vectors then m n

Theorem 10
All bases of a finite dimensional linear space S contain the same
number of vectors (equal to its dimension).