Lecture 2: Measures 1 of 17

Course: Theory of Probability I

Term: Fall 2013
Instructor: Gordan Zitkovic

Lecture 2
Measures

Measure spaces

Definition 2.1 (Measure). Let (S, S) be a measurable space. A map-

ping µ : S → [0, ∞] is called a (positive) measure if

1. µ(∅) = 0, and

2. µ(∪n An ) = ∑n∈N µ( An ), for all pairwise disjoint { An }n∈N in S .

A triple (S, S , µ) consisting of a non-empty set, a σ-algebra S on it and

a measure µ on S is called a measure space.

Remark 2.2.

1. A mapping whose domain is some nonempty set A of subsets of

some set S is sometimes called a set function.

2. If the requirement 2. in the definition of the measure is weakened

so that it is only required that µ( A1 ∪ · · · ∪ An ) = µ( A1 ) + · · · +
µ( An ), for n ∈ N, and pairwise disjoint A1 , . . . , An , we say that the
mapping µ is a finitely-additive measure. If we want to stress that
a mapping µ satisfies the original requirement 2. for sequences of
sets, we say that µ is σ-additive (countably additive).

Definition 2.3 (Terminology). A measure µ on the measurable space

(S, S) is called

1. a probability measure, if µ(S) = 1,

2. a finite measure, if µ(S) < ∞,

3. a σ-finite measure, if there exists a sequence { An }n∈N in S such

that ∪n An = S and µ( An ) < ∞,

4. diffuse or atom-free, if µ({ x }) = 0, whenever x ∈ S and { x } ∈ S .

A set N ∈ S is said to be null if µ( N ) = 0.

Last Updated: September 19, 2013

Lecture 2: Measures 2 of 17

Example 2.4 (Examples of measures). Let S be a non-empty set, and

let S be a σ-algebra on S.

1. Measures on countable sets. Suppose that S is a finite or countable

set. Then each measure µ on S = 2S is of the form

µ( A) = ∑ p ( x ),
x∈ A

for some function p : S → [0, ∞] (why?). In particular, for a finite set

S with N elements, if p( x ) = 1/N then µ is a probability measure
called the uniform measure1 on S. 1
In the finite case, it has the well-known
property that µ( A) = #A #S , where # de-
2. Dirac measure. For x ∈ S, we define the set function δx on S by notes the cardinality (number of ele-
 ments).
1, x ∈ A,
δx ( A) =
0, x 6∈ A.

It is easy to check that δx is indeed a measure on S . Alternatively, δx

is called the point mass at x (or an atom on x, or the Dirac function,
even though it is not really a function). Moreover, δx is a probability
measure and, therefore, a finite and a σ-finite measure. It is atom
free only if { x } 6∈ S .
3. Counting Measure. Define a set function µ : S → [0, ∞] by

#A, A is finite,
µ( A) =
∞, A is infinite,

where, as above, #A denotes the number of elements in the set A.

Again, it is not hard to check that µ is a measure - it is called the
counting measure. Clearly, µ is a finite measure if and only is S
is a finite set. µ could be σ-finite, though, even without S being
finite. Simply take S = N, S = 2N . In that case µ(S) = ∞, but for
An = {n}, n ∈ N, we have µ( An ) = 1, and S = ∪n An . Finally, µ is
never atom-free and it is a probability measure only if #S = 1.

Example 2.5 (A finitely-additive set function which is not a measure).

Let S = N, and S = 2S . For A ∈ S define µ( A) = 0 if A is finite and
µ( A) = ∞, otherwise. For A1 , . . . , An ⊆ S, either

1. all Ai is finite, for i = 1, . . . , n. Then ∪in=1 Ai is also finite and so

n
0 = µ(∪in=1 Ai ) = ∑ µ ( A i ), or
i =1

2. at least one Ai is infinite. Then ∪in=1 Ai is also infinite and so

n
∞ = µ(∪in=1 Ai ) = ∑ µ ( A i ).
i =1

Last Updated: September 19, 2013

Lecture 2: Measures 3 of 17

On the other hand, take Ai = {i }, for i ∈ N. Then µ( Ai ) = 0, for Note: It is possible to construct very
each i ∈ N, and, so, ∑i∈N µ( Ai ) = 0, but µ(∪i Ai ) = µ(N) = ∞. simple-looking finite-additive measures
which are not σ-additive. For example,
there exist {0, 1}-valued finitely-additive
Proposition 2.6 (First properties of measures). Let (S, S , µ) be a measure measures on all subsets of N, which are
not σ-additive. Such objects are called
space. ultrafilters and their existence is equiva-
lent to a certain version of the Axiom of
1. For A1 , . . . , An ∈ S with Ai ∩ A j = ∅, for i 6= j, we have Choice.
n
∑ µ( Ai ) = µ(∪in=1 Ai ) (Finite additivity)
i =1

2. If A, B ∈ S , A ⊆ B, then

µ( A) ≤ µ( B) (Monotonicity of measures)

3. If { An }n∈N in S is increasing, then

µ(∪n An ) = lim µ( An ) = sup µ( An ).

n n

(Continuity with respect to increasing sequences)

4. If { An }n∈N in S is decreasing and µ( A1 ) < ∞, then

µ(∩n An ) = lim µ( An ) = inf µ( An ).

n n

(Continuity with respect to decreasing sequences)

5. For a sequence { An }n∈N in S , we have

µ(∪n An ) ≤ ∑ µ ( A n ). (Subadditivity)
n ∈N

Proof.
1. Note that the sequence A1 , A2 , . . . , An , ∅, ∅, . . . is pairwise disjoint,
and so, by σ-additivity,

µ(∪in=1 Ai ) = µ(∪i∈N Ai ) = ∑ µ ( Ai )
i ∈N
n ∞ n
= ∑ µ ( Ai ) + ∑ µ(∅) = ∑ µ ( A i ).
i =1 i = n +1 i =1

2. Write B as a disjoint union A ∪ ( B \ A) of elements of S . By (1)

above, µ( B) = µ( A) + µ( B \ A) ≥ µ( A).

3. Define B1 = A1 , Bn = An \ An−1 for n > 1. Then { Bn }n∈N is a

pairwise disjoint sequence in S with ∪nk=1 Bk = An for each n ∈ N
(why?). By σ-additivity we have
n
µ(∪n An ) = µ(∪n Bn ) = ∑ µ( Bn ) = lim
n
∑ µ( Bk )
n ∈N k =1
= lim µ(∪nk=1 Bk ) = lim µ( An ).
n n

Last Updated: September 19, 2013

Lecture 2: Measures 4 of 17

4. Consider the increasing sequence { Bn }n∈N in S given by Bn = A1 \

An . By De Morgan laws, finiteness of µ( A1 ) and (3) above, we have

µ( A1 ) − µ(∩n An ) = µ( A1 \ (∩n An )) = µ(∪n Bn ) = lim µ( Bn )

n
= lim µ( A1 \ An ) = µ( A1 ) − lim µ( An ).
n n

Subtracting both sides from µ( A1 ) < ∞ produces the statement.

5. We start from the observation that for A1 , A1 ∈ S the set A1 ∪ A2

can be written as a disjoint union

A1 ∪ A2 = ( A1 \ A2 ) ∪ ( A2 \ A1 ) ∪ ( A1 ∩ A2 ),

so that

µ ( A1 ∪ A2 ) = µ ( A1 \ A2 ) + µ ( A2 \ A1 ) + µ ( A1 ∩ A2 ).

On the other hand,

µ( A1 ) + µ( A2 ) = (µ( A1 \ A2 ) + µ( A1 ∩ A2 ))
 
+ µ ( A2 \ A1 ) + µ ( A1 ∩ A2 )
= µ( A1 \ A2 ) + µ( A2 \ A1 ) + 2µ( A1 ∩ A2 ),

and so

µ( A1 ) + µ( A2 ) − µ( A1 ∪ A2 ) = µ( A1 ∩ A2 ) ≥ 0.

Induction can be used to show that

n
µ ( A1 ∪ · · · ∪ A n ) ≤ ∑ µ ( A k ).
k =1

Since all µ( An ) are nonnegative, we now have

µ( A1 ∪ · · · ∪ An ) ≤ α, for each n ∈ N, where α = ∑ µ ( A n ).

n ∈N

The sequence { Bn }n∈N given by Bn = ∪nk=1 Ak is increasing, so the

continuity of measure with respect to increasing sequences implies
that

µ(∪n An ) = µ(∪n Bn ) = lim µ( Bn ) = lim µ( A1 ∪ · · · ∪ An ) ≤ α.

n n

Remark 2.7. The condition µ( A1 ) < ∞ in the part (4) of Proposition

2.6 cannot be significantly relaxed. Indeed, let µ be the counting mea-
sure on N, and let An = {n, n + 1, . . . }. Then µ( An ) = ∞ and, so
limn µ( An ) = ∞. On the other hand, ∩ An = ∅, so µ(∩n An ) = 0.

Last Updated: September 19, 2013

Lecture 2: Measures 5 of 17

In addition to unions and intersections, one can produce other im-

portant new sets from sequences of old ones. More specifically, let
{ An }n∈N be a sequence of subsets of S. The subset lim infn An of S,
defined by

lim inf An = ∪n Bn , where Bn = ∩k≥n Ak ,

n

is called the limit inferior of the sequence An . It is also denoted by

limn An or { An , ev.} (ev. stands for eventually2 ). 2
the reason for the use of the word even-
Similarly, the subset lim supn An of S, defined by tually is the following: lim infn An is the
set of all x ∈ S which belong to An for all
but finitely many values of the index n, i.e.,
lim sup An = ∩n Bn , where Bn = ∪k≥n Ak , from some value of the index n onwards.
n

is called the limit superior of the sequence An . It is also denoted by

limn An or { An , i.o.} (i.o. stands for infinitely often3 ). Clearly, we have 3
in words, lim supn An is the set of all
x ∈ S which belong An for infinitely
lim inf An ⊆ lim sup An . many values of n.
n n

Problem 2.1. Let (S, S , µ) be a finite measure space. Show that Hint: For the second part, a measure
space with finite (and small) S will do.
µ(lim inf An ) ≤ lim inf µ( An ) ≤ lim sup µ( An ) ≤ µ(lim sup An ),

for any sequence { An }n∈N in S . Give an example of a (single) se-

quence { An }n∈N for which all inequalities above are strict.

Proposition 2.8 (Borel-Cantelli Lemma I). Let (S, S , µ) be a measure

space, and let { An }n∈N be a sequence of sets in S with the property that
∑n∈N µ( An ) < ∞. Then

µ(lim sup An ) = 0.
n

Proof. Set Bn = ∪k≥n Ak , so that { Bn }n∈N is a decreasing sequence of

sets in S with lim supn An = ∩n Bn , and so

µ(lim sup An ) ≤ µ( Bn ), for each n ∈ N.

Using the subadditivity of measures of Proposition 2.6, part 5., we get

∞
µ( Bn ) ≤ ∑ µ ( A n ). (2.1)
k=n

Since ∑n∈N µ( An ) converges, the right-hand side of (2.1) can be made

arbitrarily small by choosing large enough n ∈ N.

Extensions of measures and the coin-toss space

Example 1.19 has introduced the measurable space ({−1, 1}N , S), with
S = ⊗n 2{−1,1} being the product σ-algebra on {−1, 1}N . The purpose

Last Updated: September 19, 2013

Lecture 2: Measures 6 of 17

of the present section is to turn ({−1, 1}N , S) into a measure space,

i.e., to define a suitable measure on it. It is easy to construct just any
measure on {−1, 1}N , but the one we are after is the one which will
justify the name coin-toss space.
The intuition we have about tossing a fair coin infinitely many times
should help us start with the definition of the coin-toss measure - de-
noted by µC - on cylinders. Since the coordinate spaces {−1, 1} are
particularly simple, each product cylinder is of the form C = {−1, 1}N
or C = Cn1 ,...,nk ;b1 ,...,bk , where
n o
Cn1 ,...,nk ;b1 ,...,bk = s = (s1 , s2 , . . . ) ∈ {−1, 1}N : sn1 = b1 , . . . , snk = bk ,

for some k ∈ N, and a choice of 1 ≤ n1 < n2 < · · · < nk ∈ N of

coordinates and the corresponding values b1 , b2 , . . . , bk ∈ {−1, 1}.
In the language of elementary probability, each cylinder corresponds
to the event when the outcome of the ni -th coin is bi ∈ {−1, 1}, for
i = 1, . . . , n. The measure (probability) of this event can only be given
by
µC (Cn1 ,...,nk ;b1 ,...,bk ) = 1
× 12 × · · · × 21 = 2−k .
|2 {z } (2.2)
k times

The hard part is to extend this definition to all elements of S , and not
only cylinders. For example, in order to state the law of large numbers
later on, we will need to be able to compute the measure of the set
n n o
s ∈ {−1, 1}N : lim n1 ∑ sk = 0 ,
n
k =1

which is clearly not a cylinder.

Problem 1.9 states, however, that cylinders form an algebra and gen-
erate the σ-algebra S . Luckily, this puts us close to the conditions of
the following important theorem of Caratheodory.

Theorem 2.9 (Caratheodory’s Extension Theorem). Let S be a non- Note: In words, a σ-additive measure
empty set, let A be an algebra of its subsets and let µ : A → [0, ∞] be a on an algebra A can be extended to
a σ-additive measure on the σ-algebra
set-function with the following properties: generated by A. It is clear that the σ-
additivity requirement of Theorem 2.9 is
1. µ(∅) = 0, and necessary, but it is quite surprising that
it is actually sufficient.
2. µ( A) = ∑∞
n=1 µ ( An ), if { An }n∈N ⊆ A is a partition of A.

Then, there exists a measure µ̃ on σ (A) with the property that µ( A) = µ̃( A)
for A ∈ A.

Of Theorem 2.9. PART I. We start by defining a “measure-like object”, Note: Intuitively, we try all different
called an outer measure, µ∗ : 2S → [0, ∞] in the following way: countable covers of B with elements of
A and minimize the total µ.
n ∞ ∞ o
µ∗ ( B) = inf ∑ An , An ∈ A for all n ∈ N .
[
µ( An ) : B ⊆
n =1 n =1

Last Updated: September 19, 2013

Lecture 2: Measures 7 of 17

Even though we don’t expect the infimum in the definition of µ∗ to be

attained, µ∗ has the following properties:

1. µ∗ (∅) = 0 (nontriviality),

2. for B ⊆ C, µ∗ ( B) ≤ µ∗ (C ) (monotonicity), and

3. µ∗ (∪k Bk ) ≤ ∑∞ ∗
k =1 µ ( Bk ) (subadditivity)

Parts 1. and 2. are immediately clear, while, to show 3., we pick ε > 0
and k ∈ N and find a countable cover { Akn }n∈N with elements of A
such that
∞
µ∗ ( Bk ) ≥ ∑ µ( Akn ) + 2εk .
n =1

Using { Akn }k∈N,n∈N as a candidate cover for ∪k Bk , we conclude that

µ∗ (∪k Bk ) ≤ ∑∞ ∗
k =1 µ ( Bk ) + ε. This being true for each ε > 0 implies 3.
We remark at this point that µ∗ and µ coincide on A. By using
( A, ∅, ∅, . . . ) as a candidate countable cover of A ∈ A, we can con-
clude that µ∗ ( A) ≤ µ( A), for all A ∈ A. Conversely, suppose that
A ⊆ ∪n∈N An with An ∈ A. Given that the elements of A form an al-
gebra, we can assume that An ⊆ A, for all n ∈ N and that { An }n∈N are
pairwise disjoint, as any sequence covering A can be transformed into
such a sequence without increasing ∑n µ( An ). The assumed countable Note: It is, probably, interesting to note
additivity of µ on A now comes into play since, for partition { An }n∈N that this is the only place in the entire
proof where the countable additivity of
of A into elements in A, we necessarily have ∑n µ( An ) = µ( A), and, µ on A is used.
so µ∗ ( A) ≥ µ( A).
PART II. The set-function µ∗ is, in general, not a measure, but comes
with the advantage of being defined on all subsets of S. The central
idea of the proof is to recover countable additivity by restricting its do-
main a little. We say that a subset M ⊆ S is Caratheodory-measurable
or µ∗ -measurable if

µ∗ ( B) = µ∗ ( B ∩ M) + µ∗ ( B ∩ Mc ) for all B ⊆ S, (2.3)

with the family of all µ∗ -measurable subsets of S denoted by M∗ . We

note that, by subadditivity, the equality sign in the definition of the
measurability can be replaced by ≥; this will be used below.
The first thing we need to establish about M∗ is that it is an algebra
and that µ∗ is a finitely-additive measure on M∗ . Clearly ∅ ∈ A∗ and
the complement axiom follows directly from the symmetry in (2.3).
Only the closure under finite unions needs some discussion, and, by
induction, we only need to consider two-element unions; for that, we
pick M, N ∈ M∗ , and introduce the following notation

M00 = Mc ∩ N c , M01 = Mc ∩ N, M10 = M ∩ N c M11 = M ∩ N. (2.4)

Last Updated: September 19, 2013

Lecture 2: Measures 8 of 17

By the measurability of M and N, for any B ⊆ S, we have

µ∗ ( B) = µ∗ ( B ∩ N c ) + µ∗ ( B ∩ N )
= µ∗ ( M00 ∩ B) + µ∗ ( M10 ∩ B) + µ∗ ( M01 ∩ B) + µ∗ ( M11 ∩ B)
On the other hand, M01 ∪ M10 ∪ M11 = M ∪ N, so that, by subadditiv-
ity and (2.4), we have

µ∗ ( M ∪ N )c ∩ B + µ∗ ( M ∪ N ) ∩ B



 
= µ∗ ( M00 ∩ B) + µ∗ ( M01 ∩ B) ∪ ( M10 ∩ B) ∪ ( M11 ∩ B)
≤ µ∗ ( M00 ∩ B) + µ∗ ( M10 ∩ B) + µ∗ ( M01 ∩ B) + µ∗ ( M11 ∩ B)
= µ ∗ ( B ),
which implies that that M ∪ N ∈ M∗ . When M ∩ N = ∅ an applica-
tion of measurability of N to B = M ∪ N yields the finite additivity of
µ∗ on M∗ :

µ∗ ( M ∪ N ) = µ∗ (( M ∪ N ) ∩ N ) + µ∗ (( M ∪ N ) ∩ N c ) = µ∗ ( N ) + µ∗ ( M).

PART III. We now turn to the closure of M∗ under countable unions

and the σ-additive property of µ∗ . Since M∗ already known to be an
algebra, it will be enough to show that it is closed under pairwise-
disjoint unions, i.e., that M ∈ M∗ whenever { Mn }n∈N are pairwise
disjoint elements in M∗ with M = ∪n Mn .
For n ∈ N, we set Ln = ∪nk=1 Mk so that, for B ⊆ S, we have

µ∗ ( B) = µ∗ ( B ∩ Ln ) + µ∗ ( B ∩ Lcn )
≥ ∑nk=1 µ∗ ( B ∩ Mk ) + µ∗ ( B ∩ Mc ),
so that
µ ∗ ( B ) ≥ µ ∗ ( B ∩ M c ) + ∑ k ∈N µ ∗ ( B ∩ Mk )
≥ µ∗ ( B ∩ Mc ) + µ∗ (∪k ( B ∩ Mk )) + µ∗ ( B ∩ Mc )
= µ ∗ ( B ∩ M ) + µ ∗ ( B ∩ M c ).
Since all the inequalities above need to be equalities, we immediately
conclude that, with B = S,

µ∗ ( M ) = ∑ µ ∗ ( Mk ) ,
k

i.e., that µ∗
is a countably-additive measure on M∗ . Since A ⊆ M∗ , we
have M∗ ⊇ σ(A) and µ̃ = µ∗ |σ(A) is the required σ-additive extension
of µ.

Back to the coin-toss space. In order to apply Theorem 2.9 in our

situation, we need to check that µ is indeed a countably-additive mea-
sure on the algebra A of all cylinders. The following problem will help
pinpoint the hard part of the argument:

Last Updated: September 19, 2013

Lecture 2: Measures 9 of 17

Problem 2.2. Let A be an algebra on a non-empty set S, and let µ :

A → [0, ∞] be a finite (µ(S) < ∞) and finitely-additive set function on
S with the following, additional, property:

lim µ( An ) = 0, whenever An & ∅. (2.5)

n

Then µ satisfies the conditions of Theorem 2.9.

The part about finite additivity is easy (perhaps a bit messy) and we
leave it to the reader:

Problem 2.3. Show that the set-function µC , defined by (2.2) on the

product cyllinders and extended by additivity to the algebra A of
cylinders, is finitely additive.

Lemma 2.10 (Conditions of Caratheodory’s theorem). The set-function

µC , defined by (2.2), and extenede by additivity to the the algebra A of cylin-
ders, has the property (2.5).

Proof. By Problem 1.10, cylinders are closed sets, and so { An }n∈N is a

sequence of closed sets whose intersection is empty. The same prob-
lem states that {−1, 1}N is compact, so, by the finite-intersection prop-
erty4 , we have An1 ∩ . . . Ank = ∅, for some finite collection n1 , . . . , nk 4
The finite-intersection property refers to
of indices. Since { An }n∈N is decreasing, we must have An = ∅, for all the following fact, familiar from real
analysis: If a family of closed sets of a
n ≥ nk , and, consequently, limn µ( An ) = 0. compact topological space has empty in-
tersection, then it admits a finite subfam-
ily with an empty intersection.
Proposition 2.11 (Existence of the coin-toss measure). There exists a
measure µC on ({−1, 1}N , S) with the property that (2.2) holds for all cylin-
ders.

Proof. Thanks to Lemma 2.10, Theorem 2.9 can now be used.

In order to prove uniqueness, we will need the celebrated π-λ The-

orem of Eugene Dynkin:

Theorem 2.12 (Dynkin’s “π-λ” Theorem). Let P be a π-system on a non-

empty set S, and let Λ be a λ-system which contains P . Then Λ also contains
the σ-algebra σ (P ) generated by P .

Proof. Using the result of part 4. of Problem 1.1, we only need to prove
that λ(P ) (where λ(P ) denotes the λ-system generated by P ) is a π-
system. For A ⊆ S, let G A denote the family of all subsets of S whose
intersections with A are in λ(P ):

G A = {C ⊆ S : C ∩ A ∈ λ(P )}.

Last Updated: September 19, 2013

Lecture 2: Measures 10 of 17

Claim: G A is a λ-system for A ∈ λ(P ).

• Since A ∈ λ(P ), clearly S ∈ G A .

• For an increasing family {Cn }n∈N in G A we have (∪n Cn ) ∩

A = ∪n (Cn ∩ A). Each Cn ∩ A is in Λ, and the family {Cn ∩
A}n∈N is increasing, so (∪n Cn ) ∩ A ∈ Λ.

• Finally, for C1 , C2 ∈ G with C1 ⊆ C2 , we have

(C2 \ C1 ) ∩ A = (C2 ∩ A) \ (C1 ∩ A) ∈ Λ,

because C1 ∩ A ⊆ C2 ∩ A.

P is a π-system, so for any A ∈ P , we have P ⊆ G A . Therefore,

λ(P ) ⊆ G A , because G A is a λ-system. In other words, for A ∈ P and
B ∈ λ(P ), we have A ∩ B ∈ λ(P ).
That means, however, that P ⊆ G B , for any B ∈ λ(P ). Using the fact
that G B is a λ-system we must also have λ(P ) ⊆ G B , for any B ∈ λ(P ),
i.e., A ∩ B ∈ λ(P ), for all A, B ∈ λ(P ), which shows that λ(P ) is
π-system.

Proposition 2.13 (Measures which agree on a π-system). Let (S, S) be

a measurable space, and let P be a π-system which generates S . Suppose that
µ1 and µ2 are two measures on S with the property that µ1 (S) = µ2 (S) < ∞
and
µ1 ( A) = µ2 ( A), for all A ∈ P .
Then µ1 = µ2 , i.e., µ1 ( A) = µ2 ( A), for all A ∈ S .

Proof. Let L be the family of all subsets A of S for which µ1 ( A) =

µ2 ( A). Clearly P ⊆ L, but L is, potentially, bigger. In fact, it follows
easily from the elementary properties of measures (see Proposition 2.6)
and the fact that µ1 (S) = µ2 (S) < ∞ that it necessarily has the struc-
ture of a λ-system5 . By Theorem 2.12 (the π-λ Theorem), L contains 5
It seems that the structure of a λ-system
the σ-algebra generated by P , i.e., S ⊆ L. On the other hand, by is defined so that it would exactly de-
scribe the structure of the family of all
definition, L ⊆ S and so µ1 = µ2 . sets on which two measures (with the
same total mass) agree. The structure
of the π-system corresponds to the min-
Proposition 2.14 (Uniqueness of the coin-toss measure). The measure imal assumption that allows Proposition
µC is the unique measure on ({−1, 1}N , S) with the property that (2.2) 2.13 to hold.
holds for all cylinders.

Proof. The existence is the content of Proposition 2.11. To prove unique-

ness, it suffices to note that algebras are π-systems and use Proposition
2.13.

Problem 2.4. Define D1 , D2 ⊆ {−1, 1}N by

Last Updated: September 19, 2013

Lecture 2: Measures 11 of 17

1. D1 = {s ∈ {−1, 1}N : lim supn sn = 1},

2. D2 = {s ∈ {−1, 1}N : ∃ N ∈ N, s N = s N +1 = s N +2 }.

Show that D1 , D2 ∈ S and compute µ( D1 ), µ( D2 ).

Our next task is to probe the structure of the σ-algebra S on {−1, 1}N
N
a little bit more and show that S 6= 2{−1,1} . It is interesting that such
a result (which deals exclusively with the structure of S ) requires a
use of a measure in its proof.

Example 2.15 (A non-measurable subset of {−1, 1}N (*)). Since σ-

algebras are closed under countable set operations, and since the prod-
uct σ-algebra S for the coin-toss space {−1, 1}N is generated by sets
obtained by restricting finite collections of coordinates, one is tempted
to think that S contains all subsets of {−1, 1}N . That is not the case.
We will use the axiom of choice, together with the fact that a measure
µC can be defined on the whole of {−1, 1}N , to show to “construct”
an example of a non-measurable set.
Let us start by constructing a relation ∼ on {−1, 1}N in the fol-
lowing6 way: we set s1 ∼ s2 if and only if there exists n ∈ N such 6
In words, s1 and s2 are related if they
that s1k = s2k , for k ≥ n (here, as always, si = (s1i , s2i , . . . ), i = 1, 2). only differ in a finite number of coordi-
nates.
It is easy to check that ∼ is an equivalence relation and that it splits
{−1, 1}N into disjoint equivalence classes. One of the many equiva-
lent forms of the axiom of choice states that there exists a subset N of
{−1, 1}N which contains exactly one element from each of the equiv-
alence classes.
Let us suppose that N is an element in S and see if we can reach a
contradiction. For each nonempty n = {n1 , . . . , nk } ∈ 2N N
f in , where 2 f in
denotes the family of all finite subsets of N, let us define the mapping
Tn : {−1, 1}N → {−1, 1}N in the following7 manner: 7
Tn flips the signs of the elements of its
 argument on the positions correspond-
−s , l ∈ n, ing to n.
l
T∅ = Id and ( Tn (s))l = for n ∈ N.
sl , l 6∈ n,

Since n is finite, Tn preserves the ∼-equivalence class of each ele-

ment. Consequently (and using the fact that N contains exactly one
element from each equivalence class) the sets N and Tn ( N ) = { Tn (s) :
s ∈ N } are disjoint. Similarly and more generally, the sets Tn ( N ) and
Tn0 ( N ) are also disjoint whenever n 6= n0 . On the other hand, each
s ∈ {−1, 1}N is equivalent to some ŝ ∈ N, i.e., it can be obtained from
ŝ by flipping a finite number of coordinates. Therefore, the family

N = { Tn ( N ) : n ∈ 2N
f in }

forms a partition of {−1, 1}N .

Last Updated: September 19, 2013

Lecture 2: Measures 12 of 17

The mapping Tn has several other nice properties. First of all, it

is immediate that it is involutory, i.e., Tn ◦ Tn = Id. To show that
it is (S , S)-measurable, we need to prove that its composition with
each projection map πk : S → {−1, 1} is measurable. This follows
immediately from the fact that for k ∈ N

C ,
k;1 k 6∈ n,
(πk ◦ Tn )−1 ({1}) =
Ck;−1 , k ∈ n,

where, for b ∈ {−1, 1}, we recall that Ck;b = {s ∈ {−1, 1}N : sk = b}

is a product cylinder. If we combine the involutivity and measurability
of Tn , we immediately conclude that Tn ( A) ∈ S for each A ∈ S . In
particular, N ⊆ S .
In addition to preserving measurability, the map Tn also preserves
the measure8 the in µC , i.e., µC ( Tn ( A)) = µC ( A), for all A ∈ S . To 8
Actually, we say that a map f from a
prove that, let us pick n ∈ F and consider the set-function µn : S → measure space (S, S , µS ) to the measure
space ( T, T , µ T ) is measure preserving
[0, 1] given by if it is measurable and µS ( f −1 ( A)) =
µn ( A) = µC ( Tn ( A)). µ T ( A), for all A ∈ T . The involutivity
of the map Tn implies that this general
It is a simple matter to show that µn is, in fact, a measure on (S, S) definition agrees with our usage in this
with µn (S) = 1. Moreover, thanks to the simple form (2.2) of the action example.

of the measure µC on cylinders, it is clear that µn = µC on the algebra

of all cylinders. It suffices to invoke Proposition 2.13 to conclude that
µn = µC on the entire S , i.e., that Tn preserves µC .
The above properties of the maps Tn , n ∈ F can imply the following:
N is a partition of S into countably many measurable subsets of equal
measure. Such a partition { N1 , N2 , . . . } cannot exist, however. Indeed
if it did, one of the following two cases would occur:
1. µ( N1 ) = 0. In that case

µ(S) = µ(∪k Nk ) = ∑ µ( Nk ) = ∑ 0 = 0 6= 1 = µ(S).

n n

2. µ( N1 ) = α > 0. In that case

µ(S) = µ(∪k Nk ) = ∑ µ( Nk ) = ∑ α = ∞ 6= 1 = µ(S).

n n

Therefore, the set N cannot be measurable9 in S . 9

Somewhat heavier set-theoretic ma-
chinery can be used to prove that most of
the subsets of S are not in S , in the sense
The Lebesgue measure that the cardinality of the set S is strictly
smaller than the cardinality of the set 2S
of all subsets of S
As we shall see, the coin-toss space can be used as a sort of a universal
measure space in probability theory. We use it here to construct the
Lebesgue measure on [0, 1]. We start with the notion somewhat dual
to the already introduced notion of the pull-back in Definition 1.8. We
leave it as an exercise for the reader to show that the set function f ∗ µ
from Definition 2.16 is indeed a measure.

Last Updated: September 19, 2013

Lecture 2: Measures 13 of 17

Definition 2.16 (Push-forwards). Let (S, S , µ) be a measure space and

let ( T, T ) be a measurable space. The measure f ∗ µ on ( T, T ), defined
by
f ∗ µ( B) = µ( f −1 ( B)), for B ∈ T ,
is called the push-forward of the measure µ by f .

Let f : {−1, 1}N → [0, 1] be the mapping given by

∞  
f (s) = ∑ 1+ s k
2 2−k , s ∈ {−1, 1}N .
k =1

The idea is to use f to establish a correspondence between all real num-

bers in [0, 1] and their expansions in the binary system, with the coding
−1 7→ 0 and 1 7→ 1. It is interesting to note that f is not one-to-one10 10
The reason for this is, poetically speak-
as it, for example, maps s1 = (1, −1, −1, . . . ) and s2 = (−1, 1, 1, . . . ) ing, that [0, 1] is not the Cantor set.

into the same value - namely 21 . Let us show, first, that the map f is
continuous in the metric d defined by part (1.2) of Problem 1.9. Indeed,
we pick s1 and s2 in {−1, 1}N and remember that for d(s1 , s2 ) ≤ 2−n ,
the first n − 1 coordinates of s1 and s2 coincide. Therefore,
∞
| f (s1 ) − f (s2 )| ≤ ∑ 2−k = 2−n+1 = 2d(s1 , s2 ).
k=n

Hence, the map f is Lipschitz and, therefore, continuous.

The continuity of f (together with the fact that S is the Borel σ-
algebra for the topology induced by the metric d) implies that f :
({−1, 1}N , S) → ([0, 1], B([0, 1])) is a measurable mapping. There-
fore, the push-forward λ = f ∗ (µ) is well defined on ([0, 1], B([0, 1])),
and we call it the Lebesgue measure on [0, 1].

Proposition 2.17 (Intuitive properties of the Lebesgue measure). The

Lebesgue measure λ on ([0, 1], B([0, 1])) satisfies

λ([ a, b)) = b − a, λ({ a}) = 0 for 0 ≤ a < b ≤ 1. (2.6)

Proof.

1. Consider a, b of the form b = 2kn and b = k2+n1 , for n ∈ N and k < 2n .

For such a, b we have f −1 ([ a, b)) = C1,...,n;c1 ,c2 ,...,cn , where c1 c2 . . . cn
is the base-2 expansion of k (after the “recoding” −1 7→ 0, 1 7→ 1).
By the very definition of λ and the form (2.2) of the action of the
coin-toss measure µC on cylinders, we have
  

λ [ a, b) = µC f −1 [ a, b) = µC (C1,...,n;c1 ,c2 ,...,cn ) = 2−n = k2+n1 − 2kn .

Therefore, (2.6) holds for a, b of the form b = 2kn and b = 2ln , for
n ∈ N, k < 2n and l = k + 1. Using (finite) additivity of λ, we

Last Updated: September 19, 2013

Lecture 2: Measures 14 of 17

immediately conclude that (2.6) holds for all k, l, i.e., that it holds
for all dyadic rationals. A general a ∈ (0, 1] can be approximated
by an increasing sequence {qn }n∈N of dyadic rationals from the left,
and the continuity of measures with respect to decreasing sequences
implies that
     
λ [ a, p) = λ ∩n [qn , p) = lim λ [qn , p) = lim( p − qn ) = ( p − a),
n n

whenever a ∈ (0, 1] and p is a dyadic rational. In order to remove

the dyadicity requirement from the right limit, we approximate it
from the left by a sequence { pn }n∈N of dyadic rationals with pn > a,
and use the continuity with respect to increasing sequences to get,
for a < b ∈ (0, 1),
     
λ [ a, b) = λ ∪n [ a, pn ) = lim λ [ a, pn ) = lim( pn − a) = (b − a).
n n

The Lebesgue measure has another important property:

Problem 2.5. Show that the Lebesgue measure is translation invari- Hint: Use Proposition 2.13 for the second
ant. More precisely, for B ∈ B([0, 1]) and x ∈ [0, 1), we have part.

1. B +1 x = {b + x (mod 1) : b ∈ B} is in B([0, 1]) and

2. λ( B +1 x ) = λ( B),

where, for a ∈ [0, 2), we define Geometrically, the set x +1 B is obtained

 from B by translating it to the right by x
and then shifting the part that is “stick-
 a, a ≤ 1, ing out” by 1 to the left.
a (mod 1) =
 a − 1, a > 1.

Finally, the notion of the Lebesgue measure is just as useful on the

entire R, as on its compact subset [0, 1]. For a general B ∈ B(R), we
can define the Lebesgue measure of B by measuring its intersections
with all intervals of the form [n, n + 1), and adding them together, i.e.,
∞  
∑


λ( B) = λ B ∩ [n, n + 1) − n .
n=−∞

Note how we are overloading the notation and using the letter λ for
both the Lebesgue measure on [0, 1] and the Lebesgue measure on R.
It is a quite tedious, but does not require any new tools, to show
that many of the properties of λ on [0, 1] transfer to λ on R:

Problem 2.6. Let λ be the Lebesgue measure on (R, B(R)). Show that

1. λ([ a, b)) = b − a, λ({ a}) = 0 for a < b,

Last Updated: September 19, 2013

Lecture 2: Measures 15 of 17

2. λ is σ-finite but not finite,

3. λ( B + x ) = λ( B), for all B ∈ B(R) and x ∈ R, where B + x =

{ b + x : b ∈ B }.

Additional Problems

Problem 2.7 (Local separation by constants). Let (S, S , µ) be a measure

space and let f , g ∈ L0 (S, S , µ) satisfy µ { x ∈ S : f ( x ) < g( x )} > 0.

Prove or construct a counterexample for the following statement:

“There exist constants a, b ∈ R such that


µ { x ∈ S : f ( x ) ≤ a < b ≤ g( x )} > 0.”

Problem 2.8 (A pseudometric on sets). Let (S, S , µ) be a finite measure Note: Let X be a nonempty set. A func-
space. For A, B ∈ S define tion d : X × X → [0, ∞) is called a pseu-
dometric if
d( A, B) = µ( A M B), 1. d( x, y) + d(y, x ) ≥ d( x, z), for all
x, y, z ∈ X,
where M denotes the symmetric difference: A M B = ( A \ B) ∪ ( B \ A). 2. d( x, y) = d(y, x ), for all x, y ∈ X, and
Show that d is a pseudometric on S , and for A ∈ S describe the set of 3. d( x, x ) = 0, for all x ∈ X.
all B ∈ S with d( A, B) = 0. Note how the only difference between a
metric and a pseudometric is that for a
metric d( x, y) = 0 implies x = y, while
Problem 2.9 (Complete measure spaces). A measure space (S, S , µ) is no such requirement is imposed on a
called complete if all subsets of null sets are themselves in S . For a pseudometric.
(possibly incomplete) measure space (S, S , µ) we define the comple-
tion (S, S ∗ , µ∗ ) in the following way: Note: Unfortunately, the same notation
µ∗ is often used for the completion of
S ∗ = { A ∪ N ∗ : A ∈ S and N ∗ ⊆ N for some N ∈ S with µ( N ) = 0}. the measure µ and the outer measure as-
sociated with µ as in the proof of The-
orem 2.9. Fortunately, it can be shown
For B ∈ S ∗ with representation B = A ∪ N ∗ we set µ∗ ( B) = µ( A).
that these two object coincide on the do-
main of the completion.
1. Show that S ∗ is a σ-algebra.

2. Show that the definition µ∗ ( B) = µ( A) above does not depend on

the choice of the decomposition B = A ∪ N ∗ , i.e., that µ( Â) = µ( A)
if B =  ∪ N̂ ∗ is another decomposition of B into a set  in S and
a subset N̂ of a null set in S .

3. Show that µ∗ is a measure on (S, S ∗ ) and that (S, S ∗ , µ∗ ) is a com-

plete measure space with the property that µ∗ ( A) = µ( A), for
A ∈ S.

Problem 2.10 (The Cantor set). The Cantor set is defined as the collec-
tion of all real numbers x in [0, 1] with the representation
∞
x= ∑ c n 3− n , where cn ∈ {0, 2}.
n =1

Show that it is Borel-measurable and compute its Lebesgue measure.

Last Updated: September 19, 2013

Lecture 2: Measures 16 of 17

Problem 2.11 (The uniform measure on a circle). Let S1 be the unit

circle, and let f : [0, 1) → S1 be the “winding map”
 
f ( x ) = cos(2πx ), sin(2πx ) , x ∈ [0, 1).

1. Show that the map f is (B([0, 1)), S 1 )-measurable, where S 1 denotes

the Borel σ-algebra on S1 (with the topology inherited from R2 ).

2. For α ∈ (0, 2π ), let Rα denote the (counter-clockwise) rotation of R2

with center (0, 0) and angle α. Show that Rα ( A) = { Rα ( x ) : x ∈ A}
is in S 1 if and only if A ∈ S 1 .

3. Let µ1 be the push-forward of the Lebesgue measure λ by the map Note: The measure µ1 is called the uni-
f . Show that µ1 is rotation-invariant, i.e., that µ1 ( A) = µ1 Rα ( A) . form measure (or the uniform distribu-


tion) on S1 .

Problem 2.12 (Asymptotic densities). We say that the subset A of N

admits asymptotic density if the limit

#( A ∩ {1, 2, . . . , n})
d( A) = lim ,
n n
exists (remember that # denotes the number of elements of a set). Let
D be the collection of all subsets of N which admit asymptotic density.

1. Is D an algebra? A σ-algebra?

2. Is the map A 7→ d( A) finitely-additive on D ? A measure?

Problem 2.13 (A subset of the coin-toss space). An element in {−1, 1}N

(i.e., a sequence s = (s1 , s2 , . . . ) where sn ∈ {−1, 1} for all n ∈ N) is
said to be eventually periodic if there exists N0 , K ∈ N such that
sn = sn+K for all n ≥ N0 . Let P ⊆ {−1, 1}N be the collection of all
eventually-period sequences. Show that P is measurable in the product
σ-algebra S and compute µC ( P).

Problem 2.14 (Regular measures). The measure space (S, S , µ), where
(S, d) is a metric space and S is a σ-algebra on S which contains the
Borel σ-algebra B(d) on S is called regular if for each A ∈ S and each
ε > 0 there exist a closed set C and an open set O such that C ⊆ A ⊆ O
and µ(O \ C ) < ε.

1. Suppose that (S, S , µ) is a regular measure space, and that the mea-
sure space (S, B(d), µ|B(d) ) is obtained from (S, S , µ) by restrict-
ing the measure µ onto the σ-algebra of Borel sets. Show that
S ⊆ B(d)∗ , where S, B(d)∗ , (µ|B(d) )∗ is the completion (in the

sense of Problem 2.9) of (S, B(d), µ|B(d) )

2. Suppose that (S, d) is a metric space and that µ is a finite measure

on B(d). Show that (S, B(d), µ) is a regular measure space.

Last Updated: September 19, 2013

Lecture 2: Measures 17 of 17

Hint: Consider a collection A of subsets A of S such that for each ε > 0 there exists
a closed set C and an open set O with C ⊆ A ⊆ O and µ(O \ C ) < ε. Argue that A is
a σ-algebra. Then show that each closed set can be written as an intersection of open
sets; use (but prove, first) the fact that the map
x 7→ d( x, C ) = inf{d( x, y) : y ∈ C },

is continuous on S for any nonempty C ⊆ S.

3. Show that (S, B(d), µ) is regular if µ is not necessarily finite, but has
the property that µ( A) < ∞ whenever A ∈ B(d) is bounded, i.e.,
when sup{d( x, y) : x, y ∈ A} < ∞.
Hint: Pick a point x0 ∈ S and, for n ∈ N, define the family { Rn }n∈N of subsets of S
as follows:
R1 = { x ∈ S : d( x, x0 ) < 2}, and
Rn = { x ∈ S : n − 1 < d( x, x0 ) < n + 1}, for n > 1,

as well as a sequence {µn }n∈N of set functions on B(d), given by µn ( A) = µ( A ∩ Rn ),

for A ∈ B(d). Under the right circumstances, even countable unions of closed sets
are closed.

4. Conclude that the Lebesgue measure on Rn , B(Rn ) is regular.

Last Updated: September 19, 2013