CHAPTER 2 BASIC MEASURE THEORY 1

CHAPTER 2: BASIC MEASURE THEORY

CHAPTER 2 BASIC MEASURE THEORY 2

Set Theory and Topology in Real Space

CHAPTER 2 BASIC MEASURE THEORY 3

• Basic concepts in set theory

– element, set, whole space (Ω)
– power set: 2Ω ; empty set: ∅
– set relationship: A ⊆ B, ∩α Aα , ∪α Aα , Ac , A − B

A − B = A ∩ Bc
CHAPTER 2 BASIC MEASURE THEORY 4

• Set operations
– properties: for any B, {Aα },
B ∩ {∪α Aα } = ∪α {B ∩ Aα } , B ∪ {∩α Aα } = ∩α {B ∪ Aα } ,
c c
{∪α Aα } = ∩α Acα , {∩α Aα } = ∪α Acα . ( de Morgan law)

– partition of a set

A1 ∪ A2 ∪ A3 ∪ ... = A1 ∪ (A2 − A1 ) ∪ (A3 − A1 ∪ A2 ) ∪ ...

– lim supn An = ∩∞ n=1 {∪ ∞

m=n Am }
lim inf n An = ∪∞
n=1 {∩ ∞
m=n Am } .
CHAPTER 2 BASIC MEASURE THEORY 5

• Topology in the Euclidean space

– open set, closed set, compact set
– properties: the union of any number of open sets is
open; A is closed if and only if for any sequence {xn }
in A such that xn → x, x must belong to A
– only ∅ and the whole real line are both open set and
closed
– any open-set covering of a compact set has finite
number of open sets covering the compact set
CHAPTER 2 BASIC MEASURE THEORY 6

Measure Space
CHAPTER 2 BASIC MEASURE THEORY 7

• Motivating example: counting measure

– Ω = {x1 , x2 , ...}
– a set function µ# (A) is the number of points in A.
– (a) µ# (∅) = 0;
(b) if A1 , A2 , ... are disjoint sets of Ω, then
∑
#
µ (∪n An ) = µ# (An ).
n
CHAPTER 2 BASIC MEASURE THEORY 8

• Motivating example: Lebesgue measure

– Ω = (−∞, ∞)
– how to measure the sizes of possibly any subsets in
R? a set function λ?
– (a) λ(∅) = 0;
(b) for any disjoint sets A1 , A2 , ...,
∑
λ(∪n An ) = λ(An )
n

– assign the length to any est of B0

∪ni=1 (ai , bi ] ∪ (−∞, b] ∪ (a, ∞), disjoint intervals

– What about non-intervals? how about in Rk ?

CHAPTER 2 BASIC MEASURE THEORY 9

• Three components in defining a measure space

– the whole space, Ω
– a collection of subsets whose sizes are measurable, A,
– a set function µ assigns negative values (sizes) to each
set of A and satisfies properties (a) and (b)
CHAPTER 2 BASIC MEASURE THEORY 10

Field, σ-filed
CHAPTER 2 BASIC MEASURE THEORY 11

• Some intuition
– A contains the sets whose sizes are measurable
– A should be closed under complement or union
CHAPTER 2 BASIC MEASURE THEORY 12

Definition 2.1 (fields, σ-fields) A non-void class A of

subsets of Ω is called a:
(i) field or algebra if A, B ∈ A implies that A ∪ B ∈ A
and Ac ∈ A; equivalently, A is closed under complements
and finite unions.
(ii) σ-field or σ-algebra if A is a field and A1 , A2 , ... ∈ A
implies ∪∞ i=1 Ai ∈ A; equivalently, A is closed under
complements and countable unions. †
CHAPTER 2 BASIC MEASURE THEORY 13

• Properties of σ-field
Proposition 2.1. (i) For a field A, ∅, Ω ∈ A and if
A1 , ..., An ∈ A, ∩ni=1 Ai ∈ A.
(ii) For a σ-field A, if A1 , A2 , ... ∈ A, then ∩∞
i=1 Ai ∈ A.
CHAPTER 2 BASIC MEASURE THEORY 14

Proof
(i) For any A ∈ A, Ω = A ∪ Ac ∈ A ⇒∅ = Ωc ∈ A.

A1 , ..., An ∈ A
⇒ ∩ni=1 Ai = (∪ni=1 Aci )c ∈ A.

(ii) (∩∞ ∞
i=1 Ai ) = ∪i=1 Ai .
c c
CHAPTER 2 BASIC MEASURE THEORY 15

• Examples of σ-field
– A = {∅, Ω} and 2Ω = {A : A ⊂ Ω}
– B0 is a field but not a σ-field

(a, b) = ∪∞
n=1 (a, b − 1/n] ∈
/ B0

– A = {A : A is in R and Ac is countable}
A is closed under countable union but not
complement
CHAPTER 2 BASIC MEASURE THEORY 16

• Measure defined on a σ-field

Definition 2.2 (measure, probability measure)
(i) A measure µ is a function from a σ-field A to [0, ∞)
∞ ∑∞
satisfying: µ(∅) = 0; µ(∪n=1 An ) = n=1 µ(An ) for any
countable (finite) disjoint sets A1 , A2 , ... ∈ A. The latter
is called the countable additivity.
(ii) Additionally, if µ(Ω) = 1, µ is a probability measure
and we usually use P instead of µ to indicate a
probability measure.
CHAPTER 2 BASIC MEASURE THEORY 17

• Properties of measure
Proposition 2.2
(i) If {An } ⊂ A and An ⊂ An+1 for all n, then
µ(∪∞ n=1 An ) = limn→∞ µ(An ).
(ii) If {An } ⊂ A, µ(A1 ) < ∞ and An ⊃ An+1 for all n,
then µ(∩∞ n=1 An ) = limn→∞ µ(An ).
∑
(iii) For any {An } ⊂ A, µ(∪n An ) ≤ n µ(An ) (countable
sub-additivity).
(iv) µ(lim inf n An ) = limn µ(∩∞
k=n An ) ≤ lim inf n µ(An )
CHAPTER 2 BASIC MEASURE THEORY 18

Proof
(i) µ(∪∞
n=1 An ) = µ(A1 ∪ (A2 − A1 ) ∪ ...) = µ(A1 ) + µ(A2 − A1 ) + ....
= limn {µ(A1 ) + µ(A2 − A1 ) + ... + µ(An − An−1 )} = limn µ(An ).

(ii)
µ(∩∞ ∞ ∞
n=1 An ) = µ(A1 )−µ(A1 −∩n=1 An ) = µ(A1 )−µ(∪n=1 (A1 ∩An )).
c

A1 ∩ Acn is increasing
⇒ the second term equals limn µ(A1 ∩ Acn ) = µ(A1 ) − limn µ(An ).

(iii)
∑n
µ(∪n An ) = limn µ(A1 ∪ ... ∪ An ) = limn { i=1 µ(Ai − ∪j<i Aj )}
∑n ∑
≤ limn i=1 µ(Ai ) = n µ(An ).

(iv) µ(lim inf n An ) = limn µ(∩∞

k=n An ) ≤ lim inf n µ(An ).
CHAPTER 2 BASIC MEASURE THEORY 19

• Measures space
a triplet (Ω, A, µ)
– set in A is called a measurable set
– If µ = P is a probability measure, (Ω, A, P ) is a
probability measure space: probability sample and
probability event
– a measure µ is called σ-finite if there exists a
countable sets {Fn } ⊂ A such that Ω = ∪n Fn and for
each Fn , µ(Fn ) < ∞.
CHAPTER 2 BASIC MEASURE THEORY 20

• Examples of measure space

– discrete measure:
∑
µ(A) = mi , A ∈ A.
ωi ∈A

– counting measure µ# in any space, say R: it is not

σ-finite.
CHAPTER 2 BASIC MEASURE THEORY 21

Measure Space Construction

CHAPTER 2 BASIC MEASURE THEORY 22

• Two basic questions

– Can we find a σ-field containing all the sets of C?
– Can we obtain a set function defined for any set of
this σ-field such that the set function agrees with µ in
C?
CHAPTER 2 BASIC MEASURE THEORY 23

• Answer to the first question

Proposition 2.3 (i) Arbitrary intersections of fields
(σ-fields) are fields (σ-fields).
(ii) For any class C of subsets of Ω, there exists a minimal
σ-field containing C and we denote it as σ(C).
CHAPTER 2 BASIC MEASURE THEORY 24

Proof
(i) is obvious.
For (ii),
σ(C) = ∩C⊂A,A is σ-field A,

i.e., the intersection of all the σ-fields containing C.

CHAPTER 2 BASIC MEASURE THEORY 25

• Answer to the second question

Theorem 2.1 (Caratheodory Extension Theorem)
A measure µ on a field C can be extended to a measure
on the minimal σ-field σ(C). If µ is σ-finite on C, then
the extension is unique and also σ-finite.
Construction
{∞ }
∑
µ∗ (A) = inf µ(Ai ) : Ai ∈ C, A ⊂ ∪∞
i=1 Ai .
i=1
CHAPTER 2 BASIC MEASURE THEORY 26

• Application to measure construction

– generate a σ-field containing B0 : Borel σ-field B
– extend λ to B: the Lebesgue measure
– (R, B, λ) is named the Borel measure space
– in Rk , we obtain (Rk , Bk , λk )
CHAPTER 2 BASIC MEASURE THEORY 27

• Other measure construction on B

– F is non-decreasing and right-continuous
– a set function in B0 : λF ((a, b]) = F (b) − F (a)
– measure extension λF in B: Lebesgue-Stieltjes
measure generated by F
– the Lebesuge measure is a special case with F (x) = x
– if F is a distribution function, this measure is a
probability measure in R
CHAPTER 2 BASIC MEASURE THEORY 28

• Completion after measure construction

– motivation: any subsets of a zero-measure set should
be given measure zero but may not be in A
– Completion: add these nuisance sets to A
CHAPTER 2 BASIC MEASURE THEORY 29

• Details of completion
– obtain another measure space (Ω, Ā, µ̄)

Ā = {A ∪ N : A ∈ A,

N ⊂ B for some B ∈ A such that µ(B) = 0}

and µ̄(A ∪ N ) = µ(A).
– the completion of the Borel measure space is the
Lebesgue measure space and the completed Borel
σ-field is the σ-field of Lebesgue sets
– we always assume that a measure space is completed
CHAPTER 2 BASIC MEASURE THEORY 30

Measurable Function
CHAPTER 2 BASIC MEASURE THEORY 31

• Definition
Definition 2.3 (measurable function) Let X : Ω 7→ R
be a function defined on Ω. X is measurable if for x ∈ R,
the set {ω ∈ Ω : X(ω) ≤ x} is measurable, equivalently,
belongs to A. Especially, if the measure space is a
probability measure space, X is called a random variable.
CHAPTER 2 BASIC MEASURE THEORY 32

• Property of measurable function

Proposition 2.4 If X is measurable, then for any
B ∈ B, X −1 (B) = {ω : X(ω) ∈ B} is measurable.
CHAPTER 2 BASIC MEASURE THEORY 33

Proof
∗
{ −1
}
B = B : B ⊂ R, X (B) is measurable in A

(−∞, x] ∈ B ∗ .

B ∈ B ∗ ⇒X −1 (B) ∈ A⇒X −1 (B c ) = Ω − X −1 (B) ∈ A

then B c ∈ B ∗ .

B1 , B2 , ... ∈ B ∗ ⇒X −1 (B1 ), X −1 (B2 ), ... ∈ A ⇒

X −1 (B1 ∪ B2 ∪ ...) = X −1 (B1 ) ∪ X −1 (B2 ) ∪ ... ∈ A.
⇒ B1 ∪ B2 ∪ ... ∈ B ∗ .

⇒ B ∗ is a σ-field containing all intervals of the type (−∞, x] ⇒

B ⊂ B∗ .
For any Borel set B, X −1 (B) is measurable in A.
CHAPTER 2 BASIC MEASURE THEORY 34

• Construction of measurable function

∑n
– simple function: i=1 xi IAi (ω), Ai ∈ A
– the finite summation and the maximum of simple
functions are still simple functions
– any elementary functions of measurable functions are
measurable
CHAPTER 2 BASIC MEASURE THEORY 35

Proposition 2.5 Suppose that {Xn } are measurable.

Then so are X1 + X2 , X1 X2 , X12 and supn Xn , inf n Xn ,
lim supn Xn and lim inf n Xn .
CHAPTER 2 BASIC MEASURE THEORY 36

Proof
{X1 + X2 ≤ x} = Ω − {X1 + X2 > x} =
Ω − ∪r∈Q {X1 > r} ∩ {X2 > x − r} , Q={ all rational numbers}.

{ 2 } √ √
X1 ≤ x = {X1 ≤ x} − {X1 < − x}.

{ }
X1 X2 = (X1 + X2 ) − X1 − X2 /2
2 2 2

{supn Xn ≤ x} = ∩n {Xn ≤ x} .
{inf n Xn ≤ x} = {supn (−Xn ) ≥ −x} .

{lim supn Xn ≤ x} = ∩r∈Q,r>0 ∪∞

n=1 ∩k≥n {Xk < x + r} .

lim inf n Xn = − lim supn (−Xn ).

CHAPTER 2 BASIC MEASURE THEORY 37

• Approximating measurable function with simple

functions
Proposition 2.6 For any measurable function X ≥ 0,
there exists an increasing sequence of simple functions
{Xn } such that Xn (ω) increases to X(ω) as n goes to
infinity.
CHAPTER 2 BASIC MEASURE THEORY 38

Proof

∑
n2 n
−1
k k k+1
Xn (ω) = I{ ≤ X(ω) < } + nI {X(ω) ≥ n}
2n 2n 2n
k=0

⇒ Xn is increasing over n.

⇒ if X(ω) < n, then |Xn (ω) − X(ω)| < 1

2n .

⇒ Xn (ω) converges to X(ω).

If X is bounded, supω |Xn (ω) − X(ω)| < 1

2n
CHAPTER 2 BASIC MEASURE THEORY 39

Integration
CHAPTER 2 BASIC MEASURE THEORY 40

Definition 2.4 (i) For any simple function

∑n ∑n
X(ω) = i=1 xi IAi (ω), we define i=1 xi µ(Ai ) as the
integral of X with respect to measure µ, denoted as
∫
Xdµ.
∫
(ii) For any X ≥ 0, we define Xdµ as
∫ ∫
Xdµ = sup Y dµ.
Y is simple function, 0 ≤ Y ≤ X
CHAPTER 2 BASIC MEASURE THEORY 41

(iii) For general X, let X + = max(X, 0) and

X − = max(−X, 0). Then X = X + − X−. If one of
∫ + ∫ −
X dµ, X dµ is finite, we define
∫ ∫ ∫
Xdµ = X + dµ − X − dµ.
CHAPTER 2 BASIC MEASURE THEORY 42

• Some notes
∫ ∫ ∫
– X is integrable if |X|dµ = X dµ + X − dµ is finite
+

– the definition (ii) is consistent with (i) when X itself

is a simple function
– for a probability measure space and X is a random
∫
variable, Xdµ ≡ E[X]
CHAPTER 2 BASIC MEASURE THEORY 43

• Fundamental properties of integration

Proposition 2.7 (i) For two measurable functions
∫ ∫
X1 ≥ 0 and X2 ≥ 0, if X1 ≤ X2 , then X1 dµ ≤ X2 dµ.
(ii) For X ≥ 0 and any sequence of simple functions Yn
∫ ∫
increasing to X, Yn dµ → Xdµ.
CHAPTER 2 BASIC MEASURE THEORY 44

Proof
(i) For any simple function 0 ≤ Y ≤ X1 , Y ≤ X2 .
∫ ∫
⇒ Y dµ ≤ X2 dµ .
Take the supreme over all the simple functions less than X1
∫ ∫
⇒ X1 dµ ≤ X2 dµ.

∫ ∫
(ii) From (i), Yn dµ is increasing and bounded by Xdµ.

∑m
It suffices to show that for any simple function Z = i=1 xi IAi (ω),
where {Ai , 1 ≤ i ≤ m} are disjoint measurable sets and xi > 0,
such that 0 ≤ Z ≤ X,
∫ ∑
m
lim Yn dµ ≥ xi µ(Ai ).
n
i=1
CHAPTER 2 BASIC MEASURE THEORY 45

We consider two cases.

∫ ∑m
Case 1. Zdµ = i=1 xi µ(Ai ) is finite thus both xi and µ(Ai ) are
finite.
Fix an ϵ > 0, let Ain = Ai ∩ {ω : Yn (ω) > xi − ϵ} . ⇒ Ain increases
to Ai ⇒ µ(Ain ) increases to µ(Ai ).

When n is large,
∫ ∑
m
Yn dµ ≥ (xi − ϵ)µ(Ai ).
i=1
∫ ∫ ∑m
⇒ limn Yn dµ ≥ Zdµ − ϵ i=1 µ(Ai ).
∫ ∫
⇒ limn Yn dµ ≥ Zdµ by letting ϵ approach 0.
CHAPTER 2 BASIC MEASURE THEORY 46

∫
Case 2 suppose Zdµ = ∞ then there exists some i from
{1, ..., m}, say 1, so that µ(A1 ) = ∞ or x1 = ∞.

Choose any 0 < x < x1 and 0 < y < µ(A1 ).

A1n = A1 ∩ {ω : Yn (ω) > x} increases to A1 . n large enough,

µ(A1n ) > y
∫
⇒ limn Yn dµ ≥ xy.

∫
⇒ Letting x → x1 and y → µ(A1 ), conclude limn Yn dµ = ∞.

∫ ∫
⇒ limn Yn dµ ≥ Zdµ.
CHAPTER 2 BASIC MEASURE THEORY 47

• Elementary properties
∫ ∫
Proposition 2.8 Suppose Xdµ, Y dµ and
∫ ∫
Xdµ + Y dµ exit. Then
∫ ∫ ∫ ∫ ∫
(i) (X + Y )dµ = Xdµ + Y dµ, cXdµ = c Xdµ;
∫
(ii) X ≥ 0 implies Xdµ ≥ 0; X ≥ Y implies
∫ ∫
Xdµ ≥ Y dµ; and X = Y a.e. implies that
∫ ∫
Xdµ = Y dµ;
(iii) |X| ≤ Y with Y integrable implies that X is
integrable; X and Y are integrable implies that X + Y is
integrable.
CHAPTER 2 BASIC MEASURE THEORY 48

• Calculation of integration by definition

∫ {n2n −1 }
∑ k k k+1
Xdµ = lim n
µ( n ≤ X < n
) + nµ(X ≥ n) .
n
k=1 2 2 2
CHAPTER 2 BASIC MEASURE THEORY 49

• Integration w.r.t counting measure or Lebesgue

measure
∫ # ∑
– gdµ = i g(xi ).
∫
– continuous function g(x), gdλ is equal to the usual
∫
Riemann integral g(x)dx
– (Ω, B, λF ), where F is differentiable except
discontinuous points {x1 , x2 , ...},
∫ ∑ ∫
gdλF = g(xi ) {F (xi ) − F (xi −)} + g(x)f (x)dx,
i

where f (x) is the derivative of F (x).

CHAPTER 2 BASIC MEASURE THEORY 50

Convergence Theorems
CHAPTER 2 BASIC MEASURE THEORY 51

• Monotone convergence theorem (MCT)

Theorem 2.2 If Xn ≥ 0 and Xn increases to X, then
∫ ∫
Xn dµ → Xdµ.
CHAPTER 2 BASIC MEASURE THEORY 52

Proof
Choose nonnegative simple function Xkm increasing to Xk as
m → ∞. Define Yn = maxk≤n Xkn .
⇒ {Yn } is an increasing series of simple functions
∫ ∫ ∫
Xkn ≤ Yn ≤ Xn , so Xkn dµ ≤ Yn dµ ≤ Xn dµ.

⇒ n → ∞ Xk ≤ limn Yn ≤ X and
∫ ∫ ∫ ∫
Xk dµ ≤ lim Yn dµ = lim Yn dµ ≤ lim Xn dµ
n n n

⇒ k → ∞, X ≤ limn Yn ≤ X and
∫ ∫ ∫
lim Xk dµ ≤ lim Yn dµ ≤ lim Xn dµ.
k n n

The result holds.

CHAPTER 2 BASIC MEASURE THEORY 53

• Counter example
Xn (x) = −I(x > n)/n in the Lebesgue measure space.
∫
Xn increases to zero but Xn dλ = −∞
CHAPTER 2 BASIC MEASURE THEORY 54

• Fatou’s Lemma
Theorem 2.3 If Xn ≥ 0 then
∫ ∫
lim inf Xn dµ ≤ lim inf Xn dµ.
n n
CHAPTER 2 BASIC MEASURE THEORY 55

Proof

∞
lim inf Xn = sup inf Xm .
n n=1 m≥n

⇒ {inf m≥n Xm } increases to lim inf n Xn .

By the MCT,
∫ ∫ ∫
lim inf Xn dµ = lim inf Xm dµ ≤ Xn dµ.
n n m≥n
CHAPTER 2 BASIC MEASURE THEORY 56

• Two definitions in convergence

Definition 2.4 A sequence Xn converges almost
everywhere (a.e.) to X, denoted Xn →a.e. X, if
Xn (ω) → X(ω) for all ω ∈ Ω − N where µ(N ) = 0. If µ is
a probability, we write a.e. as a.s. (almost surely). A
sequence Xn converges in measure to a measurable
function X, denoted Xn →µ X, if µ(|Xn − X| ≥ ϵ) → 0
for all ϵ > 0. If µ is a probability measure, we say Xn
converges in probability to X.
CHAPTER 2 BASIC MEASURE THEORY 57

• Properties of convergence
Proposition 2.9 Let {Xn }, X be finite measurable
functions. Then Xn →a.e. X if and only if for any ϵ > 0,

µ(∩∞
n=1 ∪m≥n {|Xm − X| ≥ ϵ}) = 0.

If µ(Ω) < ∞, then Xn →a.e. X if and only if for any ϵ > 0,

µ(∪m≥n {|Xm − X| ≥ ϵ}) → 0.

CHAPTER 2 BASIC MEASURE THEORY 58

Proof

{ }
c ∞ ∞ 1
{ω : Xn (Ω) → X(ω)} = ∪k=1 ∩n=1 ∪m≥n ω : |Xm (ω) − X(ω)| ≥ .
k

Xn →a.e X⇒ the measure of the left-hand side is zero.

⇒ ∩∞
n=1 ∪m≥n {|Xm − X| ≥ ϵ} has measure zero.

For the other direction, choose ϵ = 1/k for any k, then by countable
sub-additivity,
{ }
1
µ(∪∞ ∩ ∞
∪
k=1 n=1 m≥n ω : |X m (ω) − X(ω)| ≥ )
k
∑ { }
∞ 1
≤ µ(∩n=1 ∪m≥n ω : |Xm (ω) − X(ω)| ≥ ) = 0.
k
k
⇒ Xn →a.e. X.
When µ(Ω) < ∞, the latter holds by Proposition 2.2.
CHAPTER 2 BASIC MEASURE THEORY 59

• Relationship between two convergence modes

Proposition 2.10 Let Xn be finite a.e.
(i) If Xn →µ X, then there exists a subsequence
Xnk →a.e X.
(ii) If µ(Ω) < ∞ and Xn →a.e. X, then Xn →µ X.
CHAPTER 2 BASIC MEASURE THEORY 60

Proof
(i) Find nk
P (|Xnk − X| ≥ 2−k ) < 2−k .
{ −k
}
⇒µ(∪m≥k {|Xnm − X| ≥ ϵ}) ≤ µ(∪m≥k |Xnm − X| ≥ 2 )
∑
≤ m≥k 2−m → 0.

⇒Xnk →a.e X.

(ii) is direct from the second part of Proposition 2.9.

CHAPTER 2 BASIC MEASURE THEORY 61

• Examples of convergence
– Let X2n +k = I(x ∈ [k/2n , (k + 1)/2n )), 0 ≤ k < 2n in
the Lebesgue measure space. Then Xn →λ 0 but does
not converge to zero almost everywhere.
– Xn = nI(|x| > n) →a.e. 0 but λ(|Xn | > ϵ) → ∞.
CHAPTER 2 BASIC MEASURE THEORY 62

• Dominated Convergence Theorem (DCT)

Theorem 2.4 If |Xn | ≤ Y a.e. with Y integrable, and if
∫
Xn →µ X (or Xn →a.e. X), then |Xn − X|dµ → 0 and
∫ ∫
lim Xn dµ = Xdµ.
CHAPTER 2 BASIC MEASURE THEORY 63

Proof
Assume Xn →a.e X. Define Zn = 2Y − |Xn − X|. Zn ≥ 0 and
Zn → 2Y .

⇒ From the Fatou’s lemma,

∫ ∫
2Y dµ ≤ lim inf (2Y − |Xn − X|)dµ.
n
∫
⇒ lim supn |Xn − X|dµ ≤ 0.

If Xn →µ X and the result does not hold for some subsequence of

Xn , by Proposition 2.10, there exits a further sub-sequence
converging to X almost surely. However, the result holds for this
further subsequence. Contradiction!
CHAPTER 2 BASIC MEASURE THEORY 64

• Interchange of integral and limit or derivative

Theorem 2.5 Suppose that X(ω, t) is measurable for
each t ∈ (a, b).
(i) If X(ω, t) is a.e. continuous in t at t0 and
|X(ω, t)| ≤ Y (ω), a.e. for |t − t0 | < δ with Y integrable,
then ∫ ∫
lim X(ω, t)dµ = X(ω, t0 )dµ.
t→t0
CHAPTER 2 BASIC MEASURE THEORY 65

(ii) Suppose ∂t∂

X(ω, t) exists for a.e. ω, all t ∈ (a, b) and
| ∂t
∂
X(ω, t)| ≤ Y (ω), a.e. for all t ∈ (a, b) with Y
integrable. Then
∂ ∫ ∫
∂
X(ω, t)dµ = X(ω, t)dµ.
∂t ∂t
CHAPTER 2 BASIC MEASURE THEORY 66

Proof
(i) follows from the DCT and the subsequence argument.

(ii)
∫ ∫
∂ X(ω, t + h) − X(ω, t)
X(ω, t)dµ = lim dµ.
∂t h→0 h
Then from the conditions and (i), such a limit can be taken within
the integration.
CHAPTER 2 BASIC MEASURE THEORY 67

Product of Measures
CHAPTER 2 BASIC MEASURE THEORY 68

• Definition
– Ω1 × Ω2 = {(ω1 , ω2 ) : ω1 ∈ Ω1 , ω2 ∈ Ω2 }
– A1 × A2 = σ({A1 × A2 : A1 ∈ A1 , A2 ∈ A2 })
– (µ1 × µ2 )(A1 × A2 ) = µ1 (A1 )µ2 (A2 ). with its
extension to all sets in the A1 × A2
CHAPTER 2 BASIC MEASURE THEORY 69

• Examples
– (Rk = R × ... × R, B × ... × B, λ × ... × λ)

λ × ... × λ ≡ λk

– Ω = {1, 2, 3...}

(R × Ω, B × 2Ω , λ × µ# )
CHAPTER 2 BASIC MEASURE THEORY 70

• Integration on the product measure space

– In calculus,
∫ ∫ ∫ ∫ ∫
R2 f (x, y)dxdy = x y f (x, y)dydx = y x f (x, y)dxdy

– Do we have the same equality in the product measure

space?
CHAPTER 2 BASIC MEASURE THEORY 71

Theorem 2.6 (Fubini-Tonelli Theorem) Suppose

that X : Ω1 × Ω2 → R is A1 × A2 measurable and X ≥ 0.
Then ∫
X(ω1 , ω2 )dµ1 is A2 measurable,
Ω1
∫
X(ω1 , ω2 )dµ2 is A1 measurable,
Ω2
∫ ∫ {∫ }
X(ω1 , ω2 )d(µ1 × µ2 ) = X(ω1 , ω2 )dµ2 dµ1
Ω1 ×Ω2 Ω1 Ω2
∫ {∫ }
= X(ω1 , ω2 )dµ1 dµ2 .
Ω2 Ω1
CHAPTER 2 BASIC MEASURE THEORY 72

• Conclusion from Theorem 2.6

– in general, X = X + − X − . Then the above results
hold for X + and X − . Thus, if
∫
Ω1 ×Ω2 |X(ω1 , ω2 )|d(µ1 × µ2 ) is finite, then the above
results hold.
CHAPTER 2 BASIC MEASURE THEORY 73

• One example
– let (Ω, 2Ω , µ# ) be a counting measure space where
Ω = {1, 2, 3, ...} and (R, B, λ) be the Lebesgue
measure space
– define f (x, y) = I(0 ≤ x ≤ y) exp{−y}; then
∫ ∫ ∫
f (x, y)d{µ# × λ} = { f (x, y)dλ(y)}dµ# (x)
Ω×R Ω R
∫ ∞
∑
= exp{−x}dµ# (x) = exp{−n} = 1/(e − 1).
Ω n=1
CHAPTER 2 BASIC MEASURE THEORY 74

Derivative of Measures
CHAPTER 2 BASIC MEASURE THEORY 75

• Motivation
– let (Ω, A, µ) be a measurable space and let X be a
non-negative measurable function on Ω
∫ ∫
– a set function ν as ν(A) = A Xdµ = IA Xdµ for
each A ∈ A.
– ν is a measure on (Ω, A)
– observe X = dν/dµ
CHAPTER 2 BASIC MEASURE THEORY 76

• Absolute continuity
Definition 2.5 If for any A ∈ A, µ(A) = 0 implies that
ν(A) = 0, then ν is said to be absolutely continuous with
respect to µ, and we write ν ≺≺ µ. Sometimes it is also
said that ν is dominated by µ.
CHAPTER 2 BASIC MEASURE THEORY 77

• Equivalent conditions
Proposition 2.11 Suppose ν(Ω) < ∞. Then ν ≺≺ µ if
and only if for any ϵ > 0, there exists a δ such that
ν(A) < ϵ whenever µ(A) < δ.
CHAPTER 2 BASIC MEASURE THEORY 78

Proof
“ ⇐′′ is clear.

To prove “⇒′′ , suppose there exists ϵ and a set An such that

ν(An ) > ϵ and µ(An ) < n−2 .

∑
Since n µ(An ) < ∞,
∑
µ(lim sup An ) ≤ µ(An ) → 0.
n
m≥n

⇒ µ(lim supn An ) = 0.

However, ν(lim supn An ) = limn ν(∪m≥n Am ) ≥ lim supn ν(An ) ≥ ϵ.

Contradiction!
CHAPTER 2 BASIC MEASURE THEORY 79

• Existence and uniqueness of the derivative

Theorem 2.7 (Radon-Nikodym theorem) Let
(Ω, A, µ) be a σ-finite measure space, and let ν be a
measurable on (Ω, A) with ν ≺≺ µ. Then there exists a
∫
measurable function X ≥ 0 such that ν(A) = A Xdµ for
all A ∈ A. X is unique in the sense that if another
measurable function Y also satisfies the equation, then
X = Y , a.e.
CHAPTER 2 BASIC MEASURE THEORY 80

• Transformation of integration using derivative

Proposition 2.13 Suppose ν and µ are σ-finite measure
defined on a measure space (Ω, A) with ν ≺≺ µ, and
∫
suppose Z is a measurable function such that Zdν is
well defined. Then for any A ∈ A,
∫ ∫
dν
Zdν = Z dµ.
A A dµ
CHAPTER 2 BASIC MEASURE THEORY 81

Proof
(i) If Z = IB where B ∈ A, then
∫ ∫ ∫
dν dν
Zdν = ν(A ∩ B) = dµ = IB dµ.
A A∩B dµ A dµ

(ii) If Z ≥ 0, find a sequence of simple function Zn increasing to Z.

∫ ∫ dν
For Zn , A Zn dν = A Zn dµ dµ. Take limits on both sides and apply
the MCT.

(iii) For any Z, write Z = Z + − Z − .

∫ ∫ + ∫ − ∫ + dν ∫ − dν ∫ dν
Zdν = Z dν − Z dν = Z dµ dµ − Z dµ dµ = Z dµ dµ.
CHAPTER 2 BASIC MEASURE THEORY 82

Induced Measure
CHAPTER 2 BASIC MEASURE THEORY 83

• Definition
– let X be a measurable function defined on (Ω, A, µ).
– for any B ∈ B, define µX (B) = µ(X −1 (B))
– µX is called a measure induced by X: (R, B, µX ).
CHAPTER 2 BASIC MEASURE THEORY 84

• Density function of X
– (R, B, ν) is another measure space (often the counting
measure or the Lebesgue measure)
– suppose µX is dominated by ν with the derivative
– f ≡ dµX /dν is called the density of X with respect to
the dominating measure ν
CHAPTER 2 BASIC MEASURE THEORY 85

• Comparison with usual density function

– (Ω, A, µ) = (Ω, A, P ) is a probability space
– X is a random variable
– if ν is the counting measure, f (x) is in fact the
probability mass function of X
– if ν is the Lebesgue measure, f (x) is the probability
density function of X
CHAPTER 2 BASIC MEASURE THEORY 86

• Integration using density

∫ ∫ ∫
– Ω g(X(ω))dµ(ω) = R g(x)dµX (x) = R g(x)f (x)dν(x)
– the integration of g(X) on the original measure space
Ω can be transformed as the integration of g(x) on R
with respect to the induced-measure µX and can be
further transformed as the integration of g(x)f (x)
with respect to the dominating measure ν
CHAPTER 2 BASIC MEASURE THEORY 87

• Interpretation in probability space

∫
– in probability space, E[g(X)] = R g(x)f (x)dν(x)
– any expectations regarding random variable X can be
computed via its probability mass function (ν is
counting measure) or density function (ν is Lebesgue
measure)
– in statistical calculation, we do NOT need to specify
whatever probability measure space X is defined on,
while solely rely on f (x) and ν.
CHAPTER 2 BASIC MEASURE THEORY 88

Probability Measure
CHAPTER 2 BASIC MEASURE THEORY 89

• A few important reminders

– a probability measure space (Ω, A, P ) is a measure
space with P (Ω) = 1;
– random variable (or random vector in
multi-dimensional real space) X is any measurable
function;
– integration of X is equivalent to the expectation;
CHAPTER 2 BASIC MEASURE THEORY 90

– the density or the mass function of X is the

Radon-Nikydom derivative of the X-induced measure
with respect to the Lebesgue measure or the counting
measure in real space;
– using the mass function or density function,
statisticians unconsciously ignore the underlying
probability measure space (Ω, A, P ).
CHAPTER 2 BASIC MEASURE THEORY 91

• Cumulative distribution function revisited

– F (x) is a nondecreasing function with F (−∞) = 0
and F (∞) = 1;
– F (x) is right-continuous;
– λF , the Lebesgue-Stieljes measure generated by F is
exactly the same measure as the one induced by X,
i.e., PX .
CHAPTER 2 BASIC MEASURE THEORY 92

Conditional Probability
CHAPTER 2 BASIC MEASURE THEORY 93

• A simple motivation
– the conditional probability of an event A given
another event B has two possibilities:
P (A|B) = P (A ∩ B)/P (B)
P (A|B c ) = P (A ∩ B c )/P (B c );
– equivalently, A given the event B is a measurable
function assigned to the σ-field {∅, B, B c , Ω},

P (A|B)IB (ω) + P (A|B c )IB c (ω).

CHAPTER 2 BASIC MEASURE THEORY 94

• Definition of conditional probability

An event A given a sub-σ-field ℵ, P (A|ℵ)
– it is a measurable and integrable function on (Ω, ℵ);
– for any G ∈ ℵ,
∫
P (A|ℵ)dP = P (A ∩ G).
G
CHAPTER 2 BASIC MEASURE THEORY 95

• Existence and Uniqueness of Conditional Probability

Function
Theorem 2.8 The measurable function P (A|ℵ) exists
and is unique in the sense that any two functions
satisfying the definition are the same almost surely.
CHAPTER 2 BASIC MEASURE THEORY 96

Proof
In (Ω, ℵ, P ), define a set function ν on ℵ such that
ν(G) = P (A ∩ G) for any G ∈ ℵ.

⇒ ν is a measure and P (G) = 0 implies that ν(G) = 0 ⇒ν ≺≺ P .

⇒ By the Radon-Nikodym theorem, there exits a ℵ-measurable

∫
function X such that ν(G) = G XdP.
⇒ X satisfies the properties (i) and (ii).

∫ ∫
Suppose X and Y both are measurable in ℵ and G
XdP = G
Y dP
for any G ∈ ℵ. Choose choose G = {X − Y ≥ 0} and
∫
G = {X − Y < 0} ⇒ |X − Y |dP = 0 ⇒ X = Y , a.s.
CHAPTER 2 BASIC MEASURE THEORY 97

• Properties of conditional probability

Theorem 2.9 P (∅|ℵ) = 0, P (Ω|ℵ) = 1 a.e. and

0 ≤ P (A|ℵ) ≤ 1

for each A ∈ A. if A1 , A2 , ... is finite or countable

sequence of disjoint sets in A, then
∑
P (∪n An |ℵ) = P (An |ℵ).
n
CHAPTER 2 BASIC MEASURE THEORY 98

Conditional Expectation
CHAPTER 2 BASIC MEASURE THEORY 99

• Definition
X given ℵ, denoted E[X|ℵ]
– E[X|ℵ] is measurable in ℵ and integrable;
∫ ∫
– for any G ∈ ℵ, G E[X|ℵ]dP = G XdP, equivalently;
E [E[X|ℵ]IG ] = E[XIG ], a.e.
– The existence and the uniqueness of E[X|ℵ] can be
shown similar to Theorem 2.8.
CHAPTER 2 BASIC MEASURE THEORY 100

• Properties of conditional expectation

Theorem 2.10 Suppose X, Y, Xn are integrable.
(i) If X = a a.s., then E[X|ℵ] = a.
(ii) E[aX + bY |ℵ] = aE[X|ℵ] + b[Y |ℵ].
(iii) If X ≤ Y a.s., then E[X|ℵ] ≤ E[Y |ℵ].
(iv) |E[X|ℵ]| ≤ E[|X||ℵ].
(v) If limn Xn = X a.s., |Xn | ≤ Y and Y is integrable,
then limn E[Xn |ℵ] = E[X|ℵ].
(vi) If X is measurable in ℵ, E[XY |ℵ] = XE[Y |ℵ]. (vii)
For two sub-σ fields ℵ1 and ℵ2 such that ℵ1 ⊂ ℵ2 ,

E [E[X|ℵ2 ]|ℵ1 ] = E[X|ℵ1 ].

(viii) P (A|ℵ) = E[IA |ℵ].

CHAPTER 2 BASIC MEASURE THEORY 101

Proof
(i)-(iv) be shown directly using the definition.

To prove (v), consider Zn = supm≥n |Xm − X|. Zn decreases to 0.

⇒|E[Xn |ℵ] − E[X|ℵ]| ≤ E[Zn |ℵ]. E[Zn |ℵ] decreases to a limit
Z ≥ 0.
Remains to show Z = 0 a.s. Note E[Zn |ℵ] ≤ E[2Y |ℵ] ⇒ by the
∫ ∫
DCT, E[Z] = E[Z|ℵ]dP ≤ E[Zn |ℵ]dP → 0. ⇒ Z = 0 a.s.

For (vii), for any G ∈ ℵ1 ⊂ ℵ2 ,

∫ ∫ ∫
E[X|ℵ2 ]dP = XdP = E[X|ℵ1 ]dP.
G G G

(viii) is clear from the definition of the conditional probability.

CHAPTER 2 BASIC MEASURE THEORY 102

∑
To see (vi) holds, consider simple function first, X = i xi IBi
where Bi are disjoint set in ℵ. For any G ∈ ℵ,
∫ ∫ ∑ ∫
E[XY |ℵ]dP = XY dP = xi Y dP
G G i G∩Bi

∑ ∫ ∫
= xi E[Y |ℵ]dP = XE[Y |ℵ]d.
i G∩Bi G

⇒ E[XY |ℵ] = XE[Y |ℵ].

For any X, a sequence of simple functions Xn converges to X and
|Xn | ≤ |X|. Then
∫ ∫
Xn Y dP = Xn E[Y |ℵ]dP.
G G

Note that |Xn E[Y |ℵ]| = |E[Xn Y |ℵ]| ≤ E[|XY ||ℵ]. From the DCT,
∫ ∫
G
XY dP = G XE[Y |ℵ]dP.
CHAPTER 2 BASIC MEASURE THEORY 103

• Relation to classical conditional density

– ℵ = σ(Y ): the σ-field generated by the class
{{Y ≤ y} : y ∈ R} ⇒P (X ∈ B|ℵ) = g(B, Y )
∫ ∫
– P (X ∈ B|ℵ)dP =
Y ≤y0 I(y ≤ y0 )g(B, y)fY (y)dy =
P (X ∈ B, Y ≤ y0 )
∫ ∫
= I(y ≤ y0 ) f (x, y)dxdy.
B
∫
– g(B, y)fY (y) = B f (x, y)dx⇒P (X ∈ B|ℵ) =
∫
B f (x|y)dx.

– the conditional density of X|Y = y is the density

function of the conditional probability measure
P (X ∈ ·|ℵ) with respect to the Lebesgue measure.
CHAPTER 2 BASIC MEASURE THEORY 104

• Relation to classical conditional expectation

– E[X|ℵ] = g(Y ) for some measurable function g(·)
∫ ∫
– I(Y ≤ y0 )E[X|ℵ]dP = I(y ≤ y0 )g(y)fY (y)dy
∫
= E[XI(Y ≤ y0 )] = I(y ≤ y0 )xf (x, y)dxdy
∫
– g(y) = xf (x, y)dx/fY (y)
– E[X|ℵ] is the same as the classical conditional
expectation of X given Y = y
CHAPTER 2 BASIC MEASURE THEORY 105

READING MATERIALS : Lehmann and Casella,

Sections 1.2 and 1.3, Lehmann Testing Statistical
Hypotheses, Chapter 2 (Optional)