Probabilityspace

courtesy of Francesca Biagiani
University of Munich
Foundations of Probability Theory: a Sketch
Probability Spaces
Kolmogorov set down a clear mathematical foundation for probability theory
in 1933. The basic ingredient is a triple (Ω, F, P), where
• Ω is the set of all possible outcomes ω.
• F is a σ-field (or σ-algebra): a collection of subsets (=events) A ⊂ Ω

such that
i) Ω ∈ F
ii) if A ∈ F then Ac = Ω \ A ∈ F
S
iii) if A1 , A2 , . . . ∈ F then n An ∈ F
• P is a probability measure on (Ω, F): a mapping P : F → [0, 1] such

that
i) P[Ω] = 1 (normalizing condition)

S P
ii) if A1 , A2 , . . . ∈ F are disjoint then P [ n An ] = n P[An ].
The pair (Ω, F) is a measurable space; (Ω, F, P) is a probability space.
Typical construction of a σ-field: let Z be a collection of subsets A ⊂ Ω.

The σ-field generated by Z is the smallest σ-field containing Z:
\
σ(Z) := G (this is a σ-field).
G σ-field
G⊃Z
Examples:
1
i) Z = {B1 , B2 , . . .} a countable partition of Ω. Then
( )
[
σ(Z) = Bi |I subset of N .
i∈I
The Bi s are called atoms.

ii) Ω = Rd . The Borel σ-field is
B(Rd ) := σ({open sets in Rd }).
Random Variables
Let (E, E) be a measurable space (the ”state space”, e.g. E = R, Rd , C[0, 1], . . .).
A map X : Ω → E is an (E-valued) random variable (or (F/E)-measurable)
if
X −1 (B) ∈ F for all B ∈ E.
The σ-field generated by X is the smallest σ-field on Ω that makes X mea-
surable:
\
σ(X) := G = X −1 (E).
G σ-field
X G/E-measurable
For (E, E) = (R, B(R)) we have σ(X) = σ({X ≤ x}, x ∈ R); and X : Ω → R
is a random variable if and only if {X ≤ x} ∈ F for all x ∈ R. The
distribution function of X is
F (x) = P [X ≤ x] , x ∈ R.
A map Y : Ω → R is σ(X)-measurable if and only if Y = f (X) for some
measurable function f : R → R (this also hold for more general state spaces).
Independence
A collection G1 , G2 , . . . ⊂ F of σ-fields is independent if for every choice
Ai ∈ Gi , i = 1, 2, . . . , the events A1 , A2 , . . . are independent:
P [A1 ∩ A2 ∩ . . . ∩ An ] = P[A1 ] . . . P[An ] for all n ∈ N.
The random variables X1 , X2 , . . . are independent if σ(X1 ), σ(X2 ), . . . are
independent.
2
Expectation
Let X be a non-negative random variable. Then there exists a sequence
(Xn ) of simple random variables (taken only finitely many values) such that
0 ≤ X1 ≤ X2 ≤ . . . ≤ Xn ↑ X a.s. Then 0 ≤ E[X1 ] ≤ E[X2 ] ≤ . . . (the
definition of E[Xn ] is clear) and the limit
E[X] := lim E[Xn ] (monotone convergence!)
n
exists (can be +∞). A real-valued random variable X = X+ − X− is inte-

grable (X ∈ L1 ) if E[|X|] < ∞, which holds if and only if E[X+ ] < ∞ and
E[X− ] < ∞, and then
Z
E[X] = XdP := E[X+ ] − E[X− ]
Ω
is the expectation of X. Moreover,

Z Z
E[h(X)] = h(x)dF (x) = h(x)f (x)dx
R R
For any measurable h : R → R such that h(X) ∈ L1 , where F (x) is the

distribution function and f (x) the density function (if it exists) of X.
Fundamental Result for Modelling

Given distribution functions F1 , F2 , . . . one can always construct a probability
space (Ω, F, P) and a sequence X1 , X2 , . . . of independent random variables
such that Xn has distribution Fn for all n ∈ N. (Simulation of i.i.d. random
variables!)
Idea: realize Xn on Ωn = ([0, 1], B[0, 1], Leb), take infinite product Ω =
Ω1 × Ω2 × . . . (see Chapter 8 in D. Williams, Probability with Martingales,
Cambridge University Press, 1995).
Conditional Expectation
Let G ⊂ F be a σ-field, and X an F-measurable, integrable random variable.
We consider G as the collection of events that can be ”observed” (the available
information). What is the best prediction, say Y , of X given G?
3
• If G = {Ω, ∅} then Y = E[X].
• If G has finitely many atoms A1 , . . . , An (each with P[Ai ] > 0) then

Z
1
Y = XdP on Ai .
P[Ai ] Ai
In both cases we clearly have E[Y 1A ] = E[X1A ] for all A ∈ G. This is in fact
the defining property: Y is called conditional expectation of X with respect
to G if
i) Y is G-measurable
ii) E[Y 1A ] = E[X1A ] for all A ∈ G.

We write Y = E[X|G], and this is defined up to a P-null set.
Theorem: The conditional expectation E[X|G] exists.
The idea of the proof is as follows: (1) If X ∈ L2 (Ω, F, P): L2 (Ω, G, P) is

a closed subspace of L2 (Ω, F, P) (these are Hilbert spaces with inner product
hU, V i = E[U V ]). Let Y be the ortogonal projection of X onto L2 (Ω, G, P).
Then E[Y 1A ] = hY, 1A i = hX, 1A i = E[X1A ] for all 1A ∈ L2 (Ω, G, P)(⇔
A ∈ G) and we are done. (2) If X ≥ 0: approximate X by simple random
variables Xn ↑ X a.s. (the Xn s are in L2 (Ω, G, P)!) and obtain E[X|G] as
monotone limit of E[Xn |G]. (3) For general X ∈ L1 write X = X+ − X− .
Here is a list of frequently used properties.

i) E[E[X|G]] = E[X].
ii) If X ∈ G then E[X|G] = X.
iii) (Monotonicity) If X ≤ Y then E[X|G] ≤ E[Y |G].
iv) (Linearity) E[aX + Y |G] = aE[X|G] + E[Y |G].
v) (cDCT) If |Xn | ≤ V , where V ∈ L1 , and P[Xn → X] = 1 then X ∈ L1

and
lim E[Xn |G] = E[lim Xn |G] = E[X|G].

n n
4
vi) (cMCT) If 0 ≤ X1 ≤ X2 ≤ . . . then
lim E[Xn |G] = E[lim Xn |G] (≤ ∞).

n n
vii) (cFatou) If Xn ≥ 0 then
E[lim inf Xn |G] ≤ lim inf E[Xn |G].

n n
viii) (Tower property) If H ⊂ G is a σ-field then
E[E[X|G]| H] = E[X|H].
ix) (”Taking out what is known”) If Y is G-measurable and |XY | and |X|
are integrable then
E[XY |G] = Y E[X|G].
x) (Independence) If G is independent of X then
E[X|G] = E[X].
Note: By convention all equalities between random variables hold a.s.
References
[1] W. Feller: An Introduction to Probability Theory and Its Applications.
Vol. I. Wiley, New York (1968).
[2] W. Feller. An Introduction to Probability Theory and Its Applica-

tions.Vol. II. Wiley, New York.
[3] J.Jacod, P.Protter. Probability Essentials, Springer.
[4] D. Williams, Probability with Martingales, Cambridge University Press,

1995.

Probabilityspace

Uploaded by

Copyright:

Available Formats

Probabilityspace

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Probabilityspace

Uploaded by

Copyright:

Available Formats

courtesy of Francesca Biagiani

Foundations of Probability Theory: a Sketch

• Ω is the set of all possible outcomes ω.

• F is a σ-field (or σ-algebra): a collection of subsets (=events) A ⊂ Ω

• P is a probability measure on (Ω, F): a mapping P : F → [0, 1] such

i) P[Ω] = 1 (normalizing condition)

The pair (Ω, F) is a measurable space; (Ω, F, P) is a probability space.

Typical construction of a σ-field: let Z be a collection of subsets A ⊂ Ω.

The Bi s are called atoms.

exists (can be +∞). A real-valued random variable X = X+ − X− is inte-

is the expectation of X. Moreover,

For any measurable h : R → R such that h(X) ∈ L1 , where F (x) is the

Fundamental Result for Modelling

• If G has finitely many atoms A1 , . . . , An (each with P[Ai ] > 0) then

ii) E[Y 1A ] = E[X1A ] for all A ∈ G.

Theorem: The conditional expectation E[X|G] exists.

The idea of the proof is as follows: (1) If X ∈ L2 (Ω, F, P): L2 (Ω, G, P) is

Here is a list of frequently used properties.

ii) If X ∈ G then E[X|G] = X.

iii) (Monotonicity) If X ≤ Y then E[X|G] ≤ E[Y |G].

iv) (Linearity) E[aX + Y |G] = aE[X|G] + E[Y |G].

v) (cDCT) If |Xn | ≤ V , where V ∈ L1 , and P[Xn → X] = 1 then X ∈ L1

lim E[Xn |G] = E[lim Xn |G] = E[X|G].

lim E[Xn |G] = E[lim Xn |G] (≤ ∞).

vii) (cFatou) If Xn ≥ 0 then

E[lim inf Xn |G] ≤ lim inf E[Xn |G].

viii) (Tower property) If H ⊂ G is a σ-field then

E[XY |G] = Y E[X|G].

x) (Independence) If G is independent of X then

Note: By convention all equalities between random variables hold a.s.

[2] W. Feller. An Introduction to Probability Theory and Its Applica-

[3] J.Jacod, P.Protter. Probability Essentials, Springer.

[4] D. Williams, Probability with Martingales, Cambridge University Press,

You might also like