Expected Value: The Starting Point

Two Important Theorems

Lecture 5

Damiano Rossello

DEB
University of Catania

Probability For Finance

Department of Economics and Business
Fall 2015 - Catania, Italy
 Expected Value: The Starting Point
Two Important Theorems

Outline

1 Expected Value: The Starting Point

A Motivating Example
Simple Rv’s
Nonnegative rv’s
General rv’s

2 Two Important Theorems

Monotone Convergence Theorem
Change of Variable Formula
Integrals of the Stieltjes Type
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Example

Let (Ω, F ) be an event space and take a fixed event A ∈ F . Define a Bernoulli rv by
X (ω) = 1 if ω ∈ A and X (ω) = 0 otherwise. The probability distribution PX of such a
rv is given by two masses p := P(X = 1) and 1 − p := P(X = 0), where 0 < p < 1.
Now, you have the following gamble: win € 1000 if the event {X = 1} occurs,
otherwise loose € 900 if event {X = 0} occurs. This may be modelled by a new rv
Y (ω) which takes on this two monetary values with probabilities p and 1 − p,
respectively. Note that X and Y share the same distribution.

To forecast the win/loss you may compute X := p · 1000 + (−900) · (1 − p). E.g.,
assuming p = 21 you expect to win X = 50 euro. A physical interpretation of X is that
it represents the centre of gravity of the numbers −900 and 1000 in R, on which you
put two probability weights, i.e. PX (B) = pδ1000 (B) + (1 − p)δ−900 (B) for every Borel
set B ⊂ R and δx (B) being the Dirac-delta measure. Observe that X is a number
lying between −900 and 1000, ∀p ∈ (0, 1).
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

0.35 900 0.65 1000

900 335 1000

X

Figure: The Bernoulli distribution PX is discrete with total mass 1 and X is its
centre of gravity, with p = 0.65
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Remark
The number X in the previous Example is based on the concept of a convex
combination of two real numbers. Let x, y ∈ R be such that x < y and define:
xλ := λ x + (1 − λ) y , for every 0 6 λ 6 1,
where xλ ∈ [x, y ]. The weights λ and (1 − λ) may be thought of as probability masses,
since their sum is 1. Now, choose P the numbers x1 , . . . P , xn ∈ [x, y ] and n positive
weights λ1 , . . . , λn such that ni=1 λi = 1. Then S = ni=1 λi xi ∈ [x,Py ], where S is
the average of the xi ’s and satisfies P x 6 S 6 y . Indeed, assume that ni=2 λi xi ∈ [x, y ]
for n − 1 numbers. From condition ni=1 λi = 1 deduce 1 − λ1 > λi . This implies
λi λi
< 1, each i = 2, 3, . . . , n. We can find z = ni=2 1−λ
P
0 < 1−λ xi within [x, y ], so
1 Pn 1
the convex combination i=1 λi xi = λ1 x1 + (1 − λ1 )z belongs to [x, y ]. Note that S
lies between min{x1 , . . . , xn } and max{x1 , . . . , xn }.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Indicator Functions

We can extend the calculation of X to more elaborated rv’s. The key

idea in constructing complicated rv’s X is that one can start from simpler
structures to approximate the X ’s. Let Ω be nonempty and take A ⊂ Ω.
The indicator function of A is the map defined on Ω by:
(
1 if ω ∈ A
IA (ω) =
0 if ω ∈ Ac .

The random function IA is a rv iff A ∈ F (exercise for the student).

 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

0 1
IA 1
A c

0 1
A IA 1
A
Ac
IA 1

0 1
IA 1

Borel set B 0 1
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Some Properties

Essentially, indicator functions establish a 1-to-1 correspondence between

measurable maps (rv’s) and measurable sets (events): set operations can
be translated into (algebraic) operations on functions. For any pair of
events A, B ∈ F it holds:
1 IA∪B = IA + IB − IA∩B = max{IA , IB };
2 IA∩B = IA − IA\B = IA · IB = min{IA , IB };
3 IA\B = max{0, IA − IB };
4 IAc = 1 − IA ;
5 IA 6 IB iff A ⊂ B;
6 IA∆B = |IA − IB |.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Exercise
Show Properties 1 to 6 for indicator functions. Observe that Properties 1
to 6 are considered pointwise, i.e. they are verified comparing IA (ω) and
IB (ω) for each ω ∈ Ω. Show that 1 becomes IA∪B = IA + IB if A and B
are disjoint. Show that 3 is simply IA − IB whenever B ⊂ A. Property 6
refers to symmetric difference between sets.

Note that the rv X in Example 1 may be defined as X = IA , with

P(IA ∈ B) = p if 1 ∈ B and B ⊂ R is a Borel set.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Linear Combinations of Indicators

Any constant random function induces the partition {∅, Ω} of Ω by

means of inverse images (see Lecture 3). More generally, by Exercise 1
any indicator function gives a partition of Ω through {∅, Ω, A, Ac }.

Recall that (see Lecture 3) a discrete rv may take on a finite number of

values {x1 , . . . , xn }. This latter set is the range X (Ω) of a random
function obtained as follows:

(1) X = x1 · IA1 + · · · + xn · IAn ,

where Ai = X −1 ({xi }) = {X = xi }, i = 1, . . . , n. Here the partition is

due to {∅, A1 , . . . P
, An , Ω}. In the reverse direction, define the linear
n
combination X = i=1 xi · IAi of indicators such that events A1 , . . . , An
form a partition of Ω.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Intermezzo 1: Preserving Measurability

Pn
We wish X = i=1 xi · IAi being a rv. The affirmative answer comes
from a series of properties concerning measurability of random functions
and composition of maps. We start with the following preliminary result:

Theorem (Measurability on a Generator)

Let (Ω, F) be an event space and let (R, B) be the real measurable
space. Define G collecting Borel sets B ⊂ R that generate B, i.e.
B = σ(G). A map X : Ω → R is a rv (a measurable map) iff
X −1 (B) ∈ F for every B ∈ G.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Proof.
(⇒) Trivial. (⇐) Let us assume X −1 (B) ∈ F for every B ∈ G. To show
that this implies measurability on the whole Borel σ-algebra B, we will
use a squeezing argument. By assumption G ⊂ B, so it contains some of
the Borel sets B ∈ B whose inverse images are in F. The collection
E = {B : X −1 (B) ∈ F} is of course greater, G ⊂ E, and is a σ-algebra
(see Lecture 3). Moreover, E ⊂ B and by hypotheses σ(G) = B. This
implies B = E and then every X −1 (B) belong to F for every B ∈ B,
which entails the required result.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Recall that (see Lecture 3) the collection of all half-rays (−∞, x],
together with ∅, generates the Borel σ-algebra B for every x ∈ R.

Thus by Theorem 1 and the above, to verify that a random function

X : Ω → R is actually a rv we need only to check that {X 6 x} is an
event in F, for every real number x.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Now do:
Exercise
Prove the following Theorem:
Let (Ω, F ) be an event space and let X , Y be two rv’s defined here. Then:
1 X + Y is a rv
2 α · X is a rv, for any α ∈ R
X
3
Y
is a rv, for any ω ∈ Ω such that Y (ω) 6= 0
4 the maximum and minimum between rv’s, X ∨ Y := max{X , Y } and
X ∧ Y := min{X , Y }, is again a rv.
Hint: for 1 show that {X + Y 6 x} is an event in F for every x ∈ R, and similarly for
2,3. For the maximum and minimum of rv’s, show that for every x ∈ R the set
{X ∨ Y 6 x} equals {X 6 x} ∩ {Y 6 x} and the set {X ∧ Y 6 x} equals
{X 6 x} ∪ {Y 6 x}.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Simple Rv’s

By Formula (1), combining n indicator functions over a partition

∪ni=1 Ai = Ω of the sample space and using coefficients xi ∈ R amounts to
iterate the sum of rv’s a finitePnumber of times. By the aforementioned
n
discussion we have that X = i=1 xi IAi is thus a rv called simple.

Observe that any simple rv is a bounded function, i.e. we can find C > 0
such that |X (ω)| 6 C for every ω ∈ Ω. A simple rv might be defined
through different choices of the partition, but the standard one is always
Ai = {X = xi }.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

X I A 1 I A 2 I A3 I A4
A1 A1 A41
A4 A43
IA41 IA42 IA43 A42
A2 A2
A3 A3

X
1 2 3 4

The ith event is further decomposed into Ai1 ∪ · · · ∪ Aim . Thus, in the original
representation of X the ith term must be replaced xi IAi = xi I∪m Aij = m
P
j=1 j=1 xi IAij .
The Figure above illustrates this with i = 4 and m = 3. It would be possible to replace
all the events Ai with others Bj that form a different partition, but leaving the
coefficients xi unaffected.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Expectation: Step 1

Pn
Let X = i=1 xi IAi be a simple rv. The expectation of X is:

(2) E(X ) := x1 P(A1 ) + · · · + xn P(An ) > 0.

This is well-defined and finite. As a special case we have

(3) E(IA ) := 1 · P(A) + 0 · P(Ac ) = P(A)

for X = IA , so the expectation coincides with the probability of A.

 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Exercise
Let S be the collection of all simple rv’s defined over (Ω, F, P). Show
that:
1 The constant unit function 1 := IΩ belongs to S; the constant zero
function 0 := I∅ is also in S.
2 The product c · X between a number c ∈ R and a simple rv X is an
element of S.
3 The sum X + Y of two simple rv’s is again a simple rv in S.
4 The product of any pair X , Y of simple rv’s is in S.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Exercise (The Set S is a Vector Space)

Show that:
1 0 + X = X , for simple rv’s 0, X ∈ S.
2 There exists a simple rv −X ∈ S such that X + (−X ) = 0.
3 1 · X = X , for simple rv’s 1, X ∈ S.
4 a · (b · X ) = (a · b) · X , for a simple rv X ∈ S and constants a, b ∈ R.
5 (a + b) · X = a · X + b · X , for a simple rv X ∈ S and constants
a, b ∈ R.
6 a · (X + Y ) = a · X + a · Y , for simple rv’s X , Y ∈ S and a constant
a ∈ R.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Exercise (Properties of E(X ))

Show that:
1 If a rv X is degenerate, i.e. X (ω) = c for every ω ∈ Ω, then
E(X ) = c.
2 Let X , Y ∈ S be simple rv’s. Then E(aX + bY ) = aE(X ) + bE(Y ),
for any a, b ∈ R. Expectation is linear.
3 Let X , Y ∈ S be simple rv’s such that X (ω) 6 Y (ω) for every
ω ∈ Ω, then E(X ) 6 E(Y ). Expectation preserves ordering.
Hint: by Exercise 4 the collection S of simple rv’s is a vector space, i.e. is closed
under summing simplePrv’s and underPthe product of scalars a ∈ R by simple rv’s.
Then, whenever X = i xi IAi , Y = j yj IBj and a, b ∈ R it holds
P P
Z = aX + bY = i j (axi + byj )IAi ∩Bj is a rv Z ∈ S, taking values axi + byj on
events Ai ∩ Bj ; these still form a partition of Ω.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Example
Pn
Let X = i=1 xi I{X =xi } be a nonnegative simple rv, i.e. xi > 0, and let
c > 0. By linearity we have E(c X ) = c E(X ). Namely, expectation is
positive homogeneous.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Exercise (Inclusion-Exclusion Formula Revisited)

Let A1 , A2 , . . . , An be events.
Qn
1 Prove I∪ni=1 Ai = 1 − i=1 (1 − IAi ).
2 Prove Inclusion-Exclusion Formula by applying Step 1 in the
definition of expectation to the identity in 1.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Expectation: Step 2

Let S+ ⊂ S be the subset of nonnegative simple rv’s. The expectation of

a nonnegative rv X > 0 is:

(4) E(X ) := sup E(Z ) : 0 6 Z 6 X , Z ∈ S+ .

Indeed, Step 1 gives each E(Z ) with respect to nonnegative simple rv’s
Z ∈ S+ that approximate X from
 below. The set
E(Z ) : 0 6 Z 6 X , Z simple is of the form [0, x) or [0, x], where
x = ∞ is allowed.
The supremum above always exists, since it is taken over a subset of
increasing real numbers. It may be infinite even if X is not.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

The correctness of formula (4) is based on the following result:

Lemma
Suppose X > 0 is a nonnegative rv. Then, there exists an increasing
sequence X1 6 X2 6 X3 6 · · · of simple rv’s which are nonnegative and
converge pointwise to X .

Whence, the definition of E(X ) for a nonnegative rv X can be restated as:

 
lim E(Xn ) = E lim Xn = E(X ),
n→∞ n→∞

Expectation and limits can be exchanged.

 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Remark
We can define a measurable map Xn : Ω → R for every n ∈ N, then obtaining a
sequence (Xn )n∈N of rv’s. For each ω ∈ Ω the set
{X1 (ω), X2 (ω), . . .} = {x1 , x2 , . . .}
is an infinite sequence of real numbers. If limn→∞ xn = x, then the number x
corresponds to the given ω. It might happen that for any other ω the above limit
exists (but in general it is not equal to the same number x). Thus, we have a function
that to each ω attaches a number given by a limit. In this case the sequence (Xn )n∈N
is said to converge pointwise to the limit function X : Ω → R. Namely, ∀ ω ∈ Ω and
∀  > 0, there exists a positive integer N such that n > N implies |Xn (ω) − X (ω)| < .
It can be shown that the random function X is a rv too (see later).
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Example
Let (Ω, F) be an event space with Ω = [0, 1] and F = B ∩ [0, 1], the
trace σ-algebra of Borel sets in the unit interval. Define the sequence of
functions Xn (ω) = ω n for every n ∈ N. Each Xn is measurable (since it is
a continuous function, see later), thus it is a rv. We have limn Xn (ω) = 0
if ω ∈ [0, 1) and limn Xn (ω) = 1 if ω = 1. Thus the sequence (Xn )n∈N
just defined converges pointwise to the limit function
(
0 if ω ∈ [0, 1)
X (ω) =
1 if ω = 1.

The convergence is very rapid to X (ω) = 0 for ω near zero, but very slow for ω near 1.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Xn X

1 1

1 2

0 1 0 1
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Exercise
Assume X ∈ S+ . Show that E(X ) given by formula (4) equals that
of formula (2).
Let X , Y > 0. Show that the expectation of nonnegative rv’s is
linear and order preserving as for rv’s in S.
Let X > 0 and assume E(X ) < ∞. Show that P(X < ∞) = 1.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Remark (Almost Sure Comparisons)

Let X , Y be any pair of rv’s. If X (ω) 6= Y (ω), except on those ω ∈ Ω for
which {ω : X (ω) = Y (ω)} has probability 1, then P(X 6= Y ) = 0. We
a.s.
write X = Y and say that X is almost surely, briefly a.s., equal to Y .
This can be extended to other cases than equality, such as X 6 Y a.s.,
a.s.
X > Y a.s., and so on. Observe that X = Y implies equality in
distribution, i.e. PX (B) = PY (B) for any Borel set B ∈ R, but the
converse is not true.
a.s.
A rv is said to be a.s. degenerate, written X = c, if for a constant
c ∈ R we have P(X = c) = 1 but X can differs from the number c on a
P-null event. Thus, P(X 6= c) = 0.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Exercise
a.s.
Let X be an a.s. degenerate rv, i.e. X = c. Show that E(X ) = c.
In general, if X = c then is distribution function its FX (x) = 1 for every x > c and
zero otherwise. We have:
FX (c−) = P(X < c) = lim P(X 6 c − h) = 0,
h↓0

while FX (c) = P(X 6 c) = 1. This implies P(X = c) = FX (c) − FX (c−) = 1 and X

acts like a constant, i.e. its distribution is actually the Dirac-delta δc (B) for any Borel
set B ⊂ R.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Expectation: Final Step

Let X , Y be any pair of rv’s. By 4 of Exercise 2 and by choosing Y = 0
as a degenerate rv, we have:

X + := max{0, X }, X − := max{0, −X } = − min{0, X }.

Observe that X + , X − > 0. We call X + the positive part of X and X −

its negative part. Clearly, X = X + − X − . We may define E(X + ) and
E(X − ) by formula (4) in Step 2. Consequently, if both expectation are
finite we can let:

(5) E(X ) = E(X + ) − E(X − ) < ∞.

All the properties of expectation studied for nonnegative rv’s hold true. When the
expectation of either X + or X − is infinite we have E(X ) = ±∞; when both equal
infinity we no longer can define E(X ).
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Remark
We can write X + = X · I{X >0} and X − = −X · I{X 60} . The absolute
value |X | = X + + X − is the sum of two nonnegative rv’s. Then, by 1 of
Exercise 2 it is a rv. Clearly, X + 6 |X | and X − 6 |X |, for every outcome
ω ∈ Ω.
 A Motivating Example
Expected Value: The Starting Point Simple Rv’s
Two Important Theorems Nonnegative rv’s
General rv’s

Remark
There is an equivalent condition for the existence of E(X ), obtained
through the decomposition |X | = X + + X − . Assume
E(|X |) = E(X + ) + E(X − ) is finite and take the expectation of both sides
in X + 6 |X | and in X − 6 |X |. It follows E(X + ) < ∞ and E(X − ) < ∞.
Thus, the finiteness condition in equation (5) is fulfilled. On the other
hand, if we assume that both E(X + ) and E(X − ) are finite, then by
summing and using linearity we get E(|X |) < ∞.

Exercise
Prove the important inequality:
(6) |E(X )| 6 E(|X |).
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Manipulating Expectation to the Limit

The following theorem is of key importance. Comparing with Lemma 1,

now we do not need simple rv’s.
Theorem
Let (Xn )n∈N be a sequence of nonnegative rv’s and let X be any rv such
that 0 6 X1 6 X2 6 · · · 6 X . If the sequence converges pointwise to X ,
then
lim E(Xn ) = E(X ).
n→∞
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Proof.
Assuming pointwise convergence Xn → X , we have X is a rv. Furthermore,
E(X1 ) 6 E(X2 ) 6 · · · 6 E(X ) by assumption. Hence, limn E(Xn ) 6 E(X ). It suffices to
show the converse inequality. The case E(X1 ) = ∞ is trivial, then we assume
E(X1 ) < ∞. By formula (4)P we can approximate E(X ) by the expectation of a
nonnegative simple rv Z = m i=1 zi IAi such that Z 6 X . The latter
P condition means
that zi 6 X (ω) for every ω ∈ Ai . Let us show that limn E(Xn ) > m i=1 zi P(Ai ). Pick
 > 0, define Ain = {ω P ∈ Ai : Xn (ω) > zi − } and observe that Ain ↑ Ai as n → ∞.
Furthermore, E(Xn ) > m i=1 (zi − )P(Ain ). By continuity from below we have

lim (zi − )P(Ain ) = (zi − )P(Ai ),

n→∞

which implies limn→∞ E(Xn ) > m (zi − )P(Ai ) = m

P P
Pi=1 i=1 zi P(Ai ) − . Since  > 0 is
m
arbitrary it follows limn E(Xn ) > i=1 zi P(Ai ) and we are done.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Remark
In the proof of Theorem 2 we use the following fact: Let a, b ∈ R such
that a 6 b + , and  > 0, then a 6 b. The proof runs by contradiction:
assume a 6 b +  and ¬(a 6 b) = a > b. But choosing  = a−b 2 > 0 we
arrive to a > b + , which contradicts the assumption.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Exercise
Let X , Y > 0. Give an alternative proof that the expectation is linear,
using the Monotone Convergence Theorem.

Hint
Find sequences (Xn )n∈N and (Yn )n∈N of simple rv’s such that Xn → X
and Yn → Y pointwise. Then take the sequence of sums (Xn + Yn )n∈N .
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Exercise
Let X and Y rv’s defined on the same probability space and such that
a.s.
X = Y . Show that E(X ) = E(Y ).

Hint
Use lemma 1 and the Monotone Convergence Theorem.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Exercise
Let X be a nonnegative discrete rv, i.e. X (Ω) = {x1 , x2 , . . .} with xi > 0
each i. Show that its expectation is:
∞
X
(7) xi P(X = xi ), where events {X = xi } decompose Ω.
i=1

Hint
Define the random variable
(
X if X ∈ {x1 , x2 , . . . , xn }
Yn =
0 if X ∈ {xn+1 , xn+2 , . . .},

and observe that Yn < X , for every ω ∈ Ω and some n ∈ N. Then, use formula (2) to
write E(Yn ). Eventually, use Lemma 1 and the Mon. Conv. Th.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

So far we seen that expectation is linear. But what happen if we try to

compute E(X1 + X2 + · · · ), for an infinite sequence of nonnegative rv’s?

Exercise
Show that
E(X1 + X2 + · · · ) = E(X1 ) + E(X2 ) + · · ·
Hint Pn
for n ∈ N. Then apply the Monotone
Write the partial sum Sn = i=1 Xi P
∞
Convergence Theorem to the series i=1 Xi = limn→∞ Sn .
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Remark
By Exercise 12, to guarantee that the expectation of a discrete rv X is
finite we must impose:
∞
X
(8) |xi |P(X = xi ) < ∞.
i=1

Indeed, X is not necessarily > 0 so we can decompose it by

X = X + − X − , where the positive and negative parts are nonnegative
discrete rv’s. Condition (8) resembles that of absolute convergence of
numerical series (note that |a b| = |a| |b|, with a = xi and
b = P(X = x) > 0).
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Example
Let X be a rv with X (Ω) = {1, 2, . . .}, and let PX be given by masses
1
P(Xi = i) = i(i+1)
∈ (0, 1), i = 1, 2, . . . .
Pn 1
Observe that sn = i=1 i(i+1) may be written as:
1 1 1
sn = 1·2
+ 2·3
+ ··· + n(n+1)
   
1 1 1 1 1 1 1



= 1− 2
+ 2
− 3
+ ··· + n−1
− n
+ n
− n+1
1
= 1− n+1
.
Thus the total mass is limn→∞ sn = 1. Unfortunately, the expectation of X is infinite
for the condition (8) is not fulfilled:
n
X ∞
X
1 1
lim |i| · i(i+1)
= i+1
= ∞.
n→∞
i=1 i=1

We drop the absolute signs and consider the second sum as the harmonic series, which
is known to be divergent.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Example (Expectation of Binomial Rv’s)

Let XP∼ Bin(n, p) be a Binomial rv. Then E(X ) equals np. Indeed,
X = ni=1 i I{X =i} and we have:
n n  n
X X n!
E(X ) = i p i (1 − p)n−i = i p i (1 − p)n−i
i=0 |
i i=0
i!(n − i)!
{z }
=P(X =i)
n
X (n − 1)!
= np p i−1 (1 − p)n−i
i=0
(i − 1)!(n − i)!
n−1
X n − 1 k
= np p (1 − p)n−1−k
k=0
k

= np(p + (1 − p))n−1 = np.

Exercise

Let X ∼ N(µ, σ 2 ) be a Gaussian rv, Y ∼ U(a, b) be a uniform rv and Z ∼ Esp(λ) be

an exponential rv. Show that µ = E(X ), E(Y ) = a+b
2
and E(Z ) = λ1 .
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Expectation as an Integral
R
We are going to show that E(X ) = Ω X dP, i.e. expectation is an
integral. More precisely, it is a Lebesgue type integral. A fundamental
argument is the following:
Theorem

There exists a unique measure ` : B → [0, ∞] called the Lebesgue

measure, which assigns to each interval (a, b] ∈ B its length
`((a, b]) = b − a.
Recall that there is a 1-to-1 correspondence between probability measures P and df’s
F . This framework can be made more general simply by dropping normalization
P(Ω) = 1, so that P is a nonnegative measure (a countable additive set functions),
and by requiring only càdlàg functions, i.e. giving up the asymptotic behavior of F
and then boundedness in [0, 1]. Thus, a proof of Theorem 3 might be based on the
choice F (x) = x for the corresponding càdlàg function with P = `.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

A Remark

A measure m on (R, B) such that m((a, b]) < ∞, for finite a, b ∈ R (viz.
Lebesgue-Stieltjes measure, see later in this Lecture), gives raise to a
càdlàg function as follows:

m((0, x]), if x > 0
F (x) =
−m((x, 0]), if x 6 0.

It can be shown (exercise for the student) that m((a, b]) = F (b) − F (a)
and F is indeed right-continuous and nondecreasing.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Exercise
Show the following.
1 The Lebesgue measure ` is not finite, i.e. `(R) = ∞, but it is
σ-finite.
2 ` is diffusive.
3 `([a, b)) = `((a, b)) = `([a, b]), for finite a < b.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Lebesgue Integral in R

Another important concept is that of a Borel-measurable function

f : R → R, i.e. f −1 ((−∞, x]) =: {f 6 x} ∈ B for every x ∈ R. Indeed,
such a function is defined between the measurable space (R, B) and itself.

We have to build a more general integration process starting from

nonnegative real-valued functions on [a, b]. Thus, we need a quick review
of Riemann integration.
In the sequel we will also need the analogous of an indicator rv. An indicator of Borel
sets is simply IB (x) = 1 iff x ∈ B, and zero otherwise. In particular, B may be an
interval.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Intermezzo 2: Uniform Probability Spaces

Let ((0, 1], B(0,1] ) be the event space for drawing at random a number
within the unit interval. Now, take the Lebesgue measure ` restricted to
B(0,1] .

`(B) is the probability that a number 0 < x 6 1 lies exactly in the Borel
set B ∈ B(0,1] . Clearly `((0, 1]) = 1.

If B = {q1 , q2 , . . .} is a countable subset of (0, 1] we then have `(B) = 0.

Of course, ((0, 1], B(0,1] , `) is a uniform continuous probability space, i.e.
a nonzero probability is put only on Borel sets greater than a singleton
{x} such as intervals [a, b] ⊂ (0, 1].
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Example
With the above in mind we are
in position to show that there exists a rv X < ∞ but
E(X ) = ∞. Let (0, 1], B(0,1] , ` be a continuous uniform probability space, where ` is
meant as the restricted Lebesgue measure. Define X = ∞ i
P
i=1 2 I(2−i ,2−(i−1) ] > 0,
−i −(i−1)

where the dyadic intervals 2 , 2 decompose (0, 1]. This is a discrete rv
becausePits range X ((0, 1]) is countable. Now, choose n ∈ N and consider the simple
n
rv Z = i=1 2i I(2−i ,2−(i−1) ] > 0. We have Z 6 X for every ω ∈ (0, 1] and by formula
(2):
n
X n
X
2i · ` 2−i , 2−(i−1) 2i · (2−(i−1) − 2−i )


E(Z ) = =
i=1 i=1
Xn
= 2i · 2−i = n.
i=1

Thus by formula (4) E(X ) > E(Z ) for every n ∈ N so that E(X ) = ∞.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Riemann vs Lebesgue Integration /1 *

Let f : [a, b] → R be a nonnegative bounded function. Recall that when
f is Riemann-integrable the lower sums and the upper sums, respectively
n
X 
LPn = L(Pn , f ) := mi (xi − xi−1 ), mi = inf f (x) : x ∈ [xi−1 , xi ]
i=1

n
X 
UPn = U(Pn , f ) := Mi (xi − xi−1 ), Mi = sup f (x) : x ∈ [xi−1 , xi ] ,
i=1

must coincide as partitions Pn = {a = x0 < x1 < . . . < xn = b} refine

each other, i.e. they get finer and finer, Pn ⊂ Pn+1 , and the mesh
kPn k = max{(xi − xi−1 ) : i = 1, . . . , n} goes to zero for large n ∈ N.
Thus: Z b
lim LPn = lim UPn = f (x)dx.
n→∞ n→∞ a
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

In fact, the set {LPn : Pn be a partition} is bounded above and the set
{UPn : Pn be a partition} is bounded below. It can be shown that f is
Riemann-integrable iff
(Z )
Z b b
f (x)dx = sup g (x)dx : 0 6 g 6 f
a a

where g is a step function, i.e. g (x) = ci for x ∈ (xi−1 , xi ) so it is

Rb Pn
constant over subintervals in [a, b], and a g (x)dx := i=1 ci (xi − xi−1 ).
Rb
E.g., choosing ci = mi we have that the integral a g (x)dx of a step
function equals the lower sum LPn .
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Riemann Integration suffers Pathologies

Example
Let f (x) = IQ∩[a,b] (x) be 1 if x is a rational number lying in [a, b] and zero otherwise.
This function is not Riemann-integrable. Indeed, any lower sum LPn is zero and any
upper sum UPn is 1. Thus, taking finer partitions and sending the mesh to zero as
n → ∞ we have LPn < UPn .

Example
Let (fn (x))n∈N be a sequence of functions all defined on [a, b]. Assume pointwise
convergence to the limit function f (x), i.e. fn (x) → f (x) as n → ∞ for every
x ∈ [a, b]. Then, it can be shown that
Z b Z b Z b
lim fn (x)dx = lim fn (x)dx = f (x)dx
n→∞ a a n→∞ a

does not always hold.

 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Riemann vs Lebesgue Integration /2 *

To intergate more general functions than the bounded ones, let f : [a, b] → R be a
nonnegative Borel-measurable map. Then fix a partition Pn = {y0 < y1 < . . . < yn }
of the codomain [0, ∞) of f > 0, such that Ji = [yi−1 , yi ) are pairwise disjoint
right-open intervals. Observe that every Ji contains some f (x), for x ∈ [a, b].

Next, take inverse images f −1 (Ji ) = {yi−1 6 f < yi } = Ai and define a nonnegative
step function g (x) = c1 IA1 + · · · + cn IAn , where clearly the Borel sets Ai ’s decompose
P x ∈ [a, b]. Furthermore,
[a, b] and 0 6 ci = inf Ji . This entailsR g (x) 6 f (x), each
define the Lebesgue integral of g by [a,b] g (x)d` := ni=1 ci `(Ji ). We say that f is
Lebesgue-integrable iff Z
lim LePn = f (x)d`,
n→∞ [a,b]
R
where LePn is nothing but the lower-Lebesgue sum equals to [a,b] g (x)d`.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

y = g(x)
c4

c3

c2
c1
c5

a x1 x2 x3 b

P5
Figure: Step function g (x) = i=1 ci IAi (x), with
A1 = [a, x1 ), A2 = {x1 }, A3 = (x1 , x2 ), A4 = [x2 , x3 ), A5 = [x3 , b]
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Riemann vs Lebesgue Integration /3 *

The limit above is understood as n → ∞ and the mesh

kPn k = max{(yi − yi−1 ) : i = 1, . . . , n} goes to zero. If each Ai is an interval we
clearly have LPn = LePn . Step functions are obviously Borel-measurable.

It is easily seen that LePn 6 LePn+1 . So to state the integrability condition we write:
Z b Z b 
f (x)d` = sup g (x)d` : 0 6 g 6 f , g step function.
a a

The supremum might be ∞.

 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

c5 y = f (x)

c4
c2
c3

c1
y = g (x)

a x1 x2 x3 x4 b

P5
Figure: Bounded nonnegative function f (x). The step function g (x) = i=1 ci IAi (x),
approximates f from below, with
A1 = [a,x1 ), A2 = [x1 ,
x2 ), A3 = [x2 , x3 ), A4 = [x3 , x4 ), A5 = [x4 , b] and
ci = inf f (x) : x ∈ Ai = inf Ji
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Riemann vs Lebesgue Integration /4 *

For more general domains than [a, b], such as Borel sets B ⊂ R, we always can define:
Z
f (x)d`, with f > 0.
B

For more general Borel measurable functions f we apply the integration process

separately to the positive part f + := max{0, f } and to the negative part

f − := max{0, −f } = − min{0, f } and then take the difference
Z Z Z
f (x)d` = f + (x)d` − f − (x)d`.
B B B
R
A function f is Lebesgue integrable iff B |f (x)|d` < ∞, where as usual
+ −
|f | = f + f . We often write dx instead of d`.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Example
R
The function f (x) = IQ∩[a,b] (x) has Lebesgue integral [a,b]
f (x)d` = 0.

Write f = 1 · IQ∩[a,b] + 0 · I(Q∩[a,b])c (x). Then, compute the integral as

1 · `(Q ∩ [a, b]) + 0 · `((Q ∩ [a, b])c ) which is equal to zero!

Indeed, Q ∩ [a, b] = {q1 , q2 , . . .} is the countable set of all rational points

within [a, b].
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Riemann vs Lebesgue Integration /5

Indeed, we have the following result:

Theorem

Let f : [a, b] → R with −∞ < a < b < ∞.

(i) If f is Riemann-integrable over [a, b], then it is Lebesgue-integrable and both the
integrals coincide.
(ii) If f is bounded in [a, b], then it is Riemann-integrable iff it is almost everywhere
(a.e.) continuous.

Recall that f is a.e. continuous over a Borel subset B ⊂ R whenever the set
{x ∈ B : f (x) is not continuous} =: {f is not continuous}
has zero Lebesgue measure. As a corollary we get: The Riemann integral of every
continuous f coincides with the Lebesgue integral.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Some Properties of the Lebesgue Integral

For Borel-measurable functions f , g : R → R the following properties hold:
R
(i) If f (x) = c for every x, then it is Lebesgue integrable and R f (x)d` = c.
(ii) If
R both f , g are Lebesgue integrable
R R b ∈ R, then
and a,
R {a f (x) + b g (x)}d` = a R f (x)d` + b R g (x)d`.
(iii) R
If f (x) 6 g (x)
R for every x, and both functions are Lebesgue integrable, then
R f (x)d` 6 R g (x)d`.
(iv) R f (x)d` = ∞
R P R
i=1 Bi f (x)d` provided that B1 , B2 , . . . form a partition of R, i.e.
Bi ∩ Bj = ∅ for i 6= j and ∪∞ i=1 Bi = R.
(v) For every sequence of nonnegative Borel-measurable functions f1 , f2 , . . . it holds
R P ∞ P∞ R
R n=1 fn d` = n=1 R fn d`.
R  R
(vi)  R f (x)d` 6 R |f (x)|d`
Properties (i) and (iii) can be stated in the a.e. sense, with the Lebesgue measure
being zero over the sets {f 6= c} and {f > g }.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

The Probabilistic Connection /1

The steps in the construction of E(X ) are the same as in the definition of a Lebesgue
integral.
Start with nonnegative simple rv’s P
(measurable random functions taking on a
finite number of values) and build ni=1 zi P(Ai ), where zi stands for ci ’s of step
functions; ` is replaced by P.
This is nothing but the ‘lower sum’ of simple rv’s Z , which are less than or equal
to a rv X > 0.
Take the supremum of such ni=1 zi P(Ai ) over all ‘partition’ of Ω, leading to
P
sequences of simple rv’s Zn that better approximate X (viz. converging
pointwise to X ).

It is apparent that the further advantage of Lebesgue integration is its applicability to

measurable functions defined over abstract domains (such as rv’s). Thus,
Z Z Z
E(X ) = X dP = X + dP − X − dP.
Ω Ω Ω
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

The Probabilistic Connection /2

With all the above in mind we can rephrase the definition of a continuous probability
distribution on (R, B) induced by a rv X . WeR need the density % : R → [0, ∞) to be a
nonnegative Borel-measurable function with R %(x)d` = 1.

The probability of the Borel set B ⊂ R translates to the probability of the event
{X ∈ B} and is written:
Z
PX (B) = P(X ∈ B) = %(x)d`.
B
R
In particular, for B = (a, b] we have (a,b] %(x)d`. Clearly, if % is in addition a
continuous function the Lebesgue integral used in defining these probabilities is equal
to the Riemann integral is in addition B is an interval.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Remark
R
To show that the set function B 7→ B %(x)d` defined over the Borel σ-algebra B is
actually a probability measure we only P need a pairwise disjoint sequences B1 , B2 , . . . of
Borel sets and the property I∪∞ n=1 Bn
= ∞ n=1 IBn of indicator functions, together with
property (v) of the Lebesgue integral:
Z
PX (∪∞n=1 B n ) = I∪∞
n=1 Bn
%(x)d`
R
∞
Z X
= IBn %(x)d`
R n=1
∞ Z
X ∞
X
= IBn %(x)d` = PX (Bn ).
n=1 R n=1
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Integrating over Subsets

In the previous Remark we used the following fact. For a rv X take its
R A ∈ F, then write its expected value
truncated version X IA for any event
E(X IA ) as the Lebesgue integral A X dP. Observe that X IA = X for
every ω ∈ A.
Exercise
R
Show that E(X IA ) is in fact equal to A X dP.

Hint
Take the restriction of X to the event A, then define the expectation above over the
restricted probability space (A, FA ), with FA the trace σ-algebra of subsets A ∩ B for
P(A∩B)
all B ∈ F . Use the probability measure PA (B) := P(B) for every B ∈ FA and all the
steps defining E( · ).
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Change of Variable in the Integral

Finally we arrive to:

Theorem
Let X be a rv with distribution PX and let g : R → R be a
Borel-measurable function. Then we have
Z
(9) E(g (X )) = g (x)dPX ,
R

provided that either side is well-defined.

Taking expectation
R is meant as
R∞the step by step procedure studied so far.
Note that R may be written −∞ .
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Before to prove the Theorem above let you do the following:

Exercise
Let X : Ω → R be a rv over the event space (Ω, F ). Let:
(
1 if x ∈ B
IB (x) = for every Borel set B ∈ B.
0 if x ∈ B c ,

Then, the composed map IB ◦ X = IB (X ) equals the indicator of the event {X ∈ B}.

The key idea in the above Exercise is:

Theorem

Let (Ω, F ) be an event space and let (R, B) be the real measurable space. Define the
rv X : Ω → R and the Borel-measurable function g : R → R. Then the composed map
Z = g ◦ X : Ω → R is a rv.

The proof is left as an exercise.

 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Proof.
First
R we check Rformula (9) for indicators g (X ) = IB (X ), where B ∈RB. We have
Ω IB (X )dP = Ω I{X ∈B} dP = E[I{X ∈B} ] = P(X ∈ B) = PX (B) = R IB (x)dPX . By
linearity, formula (9) holds for g (X ) ∈ S+ . If g (X ) is any rv, then we apply the
Monotone Convergence Theorem to both g (X )+ and g (X )− as follows:
Z Z
g (X )dP = lim(gn+ (X ) − gn+ (X ))dP
Ω Ω n
Z Z
= lim gn+ (X )dP − lim gn− (X )dP
n Ω n Ω
Z Z
= lim gn+ (x)dPX − lim gn− (x)dPX
n R n R
Z
= lim(gn+ (x) − gn+ (x))dPX
R n
Z Z
= (g + (x) − g − (x))dPX = g (x)dPX ,
R R

where gn+ (X ) ↑ g + (X ), gn− (X ) ↑ g − (X ) pointwise as n → ∞ and gn+ (X ), gn− (X ) are

taken from S+ .
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Consequences

Theorem 5 enable us to compute the expectation of a rv X by means of

a numerical Rintegral over the real line. Finding E(g (X )) should requires
to consider Ω g (X )dPg (X ) , since in general g (X ) is a new rv with
different distribution Pg (X ) . Formula (9) is instead based on the old PX .
Moreover, by the 1-to-1 correspondence between probability measures
and df’s we can safely replace dPX by dFX .
R∞
Taking the identity function g (x) = x we clearly get E(X ) = −∞ xdFX .
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

If the probability distribution PX is continuous we get the following result.

Corollary

Let X be a random variable with continuous distribution PX having

density %(x). Then, the expectation of X is the Lebesgue integral
Z
(10) E(g (X )) = g (x)%(x)d`
R

whenever g : R → R is Borel-measurable.

When both sides of equation (10) are finite, Corollary 1 says that the
Borel function g (x)%(x) is integrable.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Example

Let X ∼ N(0, 1) be a standard Gaussian rv. Transforming X by the

x
Borel-measurable (indeed continuous) function
R ∞ gx(x) = e , we get the
X
new rv Y = e . The expectation E(Y ) is −∞ e %(x)dx, where
2
%(x) = √1 e−x /2 is the density function of PX .
2π
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Exercise
Let X , Y be rv’s with finite expectation and let g : R → R be a
Borel-measurable function such that g (X ), g (Y ) have finite expectation
a.s.
too. Prove in an alternative way that if X = Y then
E(g (X )) = E(g (Y )).

Hint
Use the Change of Variable Formula.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Exercise
Let X be a rv overPa probability space (Ω, F, P). Define the nonnegative
n
simple rv g (X ) = i=1 ai I{X ∈Bi } , where ai > 0 and ∪ni=1 Bi = R is a
finite partition of R. The function g : R → R is clearly Borel-measurable.
R
Assume E(g (X )) < ∞ and show that the expectation equals R g (x)dPX .
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Exercise
Assume X = c for every ω ∈ Ω. Use the Change of Variable Formula to
compute E(g (X )) as:
Z
(11) g (X )dP = g (c).
Ω

Hint
Use the Dirac-delta measure PX (B) = δc (B), for Borel sets B ⊂ R and
c ∈ R.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Summary

To compute the expectation of a rv X we use the formula

Z ∞
E(X ) = x dFX .
−∞

Thus we have two practical cases.

FX is (absolutely) continuous and % is continuous ⇒
dFX (x) =R∞%(x)dx (by the Fundamental Theorem of Calculus) and
E(X ) = −∞ x%(x)dx, a Riemann integral.
P∞ ⇒ dFX (x) = FX (xi ) − FX (xi−1 ) and
FX is ‘discrete’
E(X ) = i=1 xi P(X = xi ).
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Not Only a Matter of Notation

Lebesgue measure and integration provides us with a powerful tool for manipulating
(symbolically or numerically) expectations. From now on we will write dx in the
Lebesgue integral, if no confusion arises on what type of integral is involved.

dP = P(dω) ⇐⇒ dPX = P(dx) ⇐⇒ dFX = FX (dx).

R
When a density arises, PX (B) = B %(x)dx, the probability distribution
R of X acts on
‘infinitesimal’ Borel set dx through PX (dx) = %(x)dx, i.e. PX (B) = B dPX .

From E(IA ) = P(A), let A = {X ∈ B}. Thus:

Z Z
I{X ∈B} dP = PX (B) = dFX .
Ω B

The probability of a Borel subset B over (R, B) can be computed by integrating dFX
here.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Lebesgue-Stieltjes Integral
R∞
The integral −∞
xdFX defining E(X ) is of a special kind. It originates
as follows.
1 Define a Lebesgue-Stieltjes measure on (R, B), i.e. a countably
additive set function m generated by a càdlàg function F as
m((a, b]) = F (b) − F (a) for finite reals a < b, and such that m(I ) is
finite for bounded intervals I ⊂ R.
2 Define a Borel-measurable function g : R → R.
RBuild the Lebesgue-type integral of g step by step to get
3
∞
−∞
g (x)dF .
4 This is called the Lebesgue-Stieltjes integral.
5 To get the expectation set g (x) = x and consider only càdlàg
functions F such that F (−∞) = 0 and F (∞) = 1, i.e. df’s.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Riemann-Stieltjes Integral

R∞
g (x)dF using the Riemann approach. If ab g (x)dx exists,
R
Now let us construct −∞
Pn
then it is approximately i=1 g (ci )(xi − xi−1 ) where ci ∈ (xi−1 , xi ] and [a, b] is
partitioned by points {a = x0 < x1 < · · · < xn = n}. Here the ‘representative value’
g (ci ) of g over each (xi−1 , xi ] is weighted by `((xi−1 , xi ]). Recall that ` is generated
by the càdlàg function F (x) = x.

Instead,
Pn we can use a different ‘weight function’ F (x) and form the sum
i=1 g (ci )(F (xi ) − F (xi−1 )). If for n → ∞ the sum converges to a limit, we call it
the Riemann-Stieltjes integral of g with respect to F . It is crucial that g (x) and F (x)
have not discontinuities at the same points x.
 Monotone Convergence Theorem
Expected Value: The Starting Point
Change of Variable Formula
Two Important Theorems
Integrals of the Stieltjes Type

Just to Remind You

R∞
The improper Riemann integral a g (x)dx exists iff
Rb
K (b) := a g (x)dx has finite limit limb→∞ K (b). Similarly, we can
Rb
define other improper Riemann integrals, such as −∞ g (x)dx and
R∞ R∞
−∞
g (x)dx. Thus, the Riemann-Stieltjes integral a g (x)dF (x)
can be defined over the whole R.
In writing Lebesgue-Stieltjes integrals
R over bounded intervals, the
end-points matter. For example, (a,b] g (x)dF differs from
R
[a,b]
g (x)dF , and similarly for [a, b) and (a, b). This is because the
underlying construction is based on càdlàg functions F that might
be step functions with F (x) − F (x−) 6= 0.
Sufficient conditions for the existence of the Riemann-Stieltjes
integral: g is continuous and F is increasing, or
F is expressed as the difference of two increasing functions.