MATH 275B&C notes 210

31. R EVERSIBILITY
Here we continue the discussion of stationary measures with attention focused on the
particular case of measures that are reversible.

31.1 Two-sided chain and its reversal.

We start with the observation that the mere existence of a stationary distribution allows
us to extend the chain to negative infinity:
Lemma 31.1 (Two-sided Markov chain) Given a standard Borel space (S, S), let p be a
transition probability and let µ P I1 . Then there exists a probability space (W, F , P) supporting
a sequence tXn unPZ of S-valued random variables such that
@n P Z : P( Xn P ¨) = µ(¨) (31.1)
and, denoting Fn := s( Xk : k § n),
@n P Z : P( Xn+1 P ¨ | Fn ) = p( Xn , ¨) a.s. (31.2)

Proof. We proceed as in the argument sketched at the beginning of the proof of The-
orem 29.3. Set (W, F ) := (SZ , SbZ ) and let Xn : SZ Ñ S be the projection on the n-th
coordinate. For n • 0 denote Gn := s( X´n , . . . , Xn ) and for A P Gn set
ª n´1
π
Pn ( A) := µ(dx´n ) p( x j , dx j+1 ) (31.3)
A j=´n

Then Pn is a probability measure on (W, Fn ). Using that µ P I1 and that p( xn , ¨) is a

probability we then verify Pn+1 ( A) = Pn ( A) and so tPn un•0 is a consistent family of
measures. The Kolmogorov Extension î Theorem ensures that these are restrictions of a
unique probability measure P on s ( n•0 Gn ) = SbZ .
The condition (31.1) is verified readily from (31.3) and the fact that µ P I1 . For (31.2)
the formula (31.3) gives
@m § n : P( Xn+1 P ¨ | Fm,n ) = p( Xn , ¨) a.s. (31.4)
with Fm,n := s( Xk : mî§ k § n). Taking m Ñ ´8 with the help of the Lévy Forward
Theorem and Fn = s( m§n Fm,n ) we then get (31.2). ⇤
With paths of the Markov chain extended to negative times, a natural question is then:
What is the law of the reversed path tX´n unPZ ? This is answered easily for countable-
stated chains:
Lemma 31.2 Let S be countable and, given a transition kernel P and an invariant distribu-
tion µ, let tXn unPZ be a two-sided Markov chain such that (31.1) holds. Then
$
& µ(y) P(y, x ), if µ( x ) ° 0
Q( x, y) := µ( x ) (31.5)
%
dxy , if µ( x ) = 0
Then tX´n unPZ is Markov chain with transition kernel Q.

Preliminary version (subject to change anytime!) Typeset: June 1, 2025

211 MATH 275B&C notes

Proof. For x P S such that µ( x ) ° 0, the fact that µ is stationary gives

ÿ 1 ÿ 1
Q( x, y) = µ(y)P(y, x ) = µ( x ) = 1 (31.6)
yPS
µ ( x ) yPS
µ ( x)

and so Q( x, ¨) is a probability mass function. For µ( x ) = 0 this follows directly and so Q

is indeed a transition kernel.
Given any integers m † n and xm , . . . , xn P S, observe that the stationarity of µ
forces µ( xi ) = 0 for all i † n once µ( xn ) = 0. Assuming µ( xn ) ° 0 we can thus write
n
π
µ( xm ) P( xi´1 , xi ) = µ( xm )P( xm , xm+1 ) . . . P( xn´1 , xn )
i = m +1
(31.7)
π n´1
µ( xm ) µ( xn´1
= P ( x m , x m +1 ) . . . P( xn´1 , xn )µ( xn ) = µ( xn ) Q ( x i +1 , x i )
µ ( x m +1 ) µ( xn )
i=m

Since the quantities on the extreme ends vanish when µ( xn ) = 0, the equality holds in
general. This means that the finite dimensional distributions of tX´n unPZ are indeed
distributed as a Markov chain with transition kernel Q. Standard extension arguments
then extend this to the full distribution on (SZ , SbZ ). ⇤
We call the chain with the transition kernel Q the reversed chain. Note that the defi-
nition of Q makes sense, and the reversed chain is thus well defined, even if µ is just
a stationary measure (i.e., if µ(S) = 8) although the connection to the reversal of the
Markov chain is less clear (we need to “start” the chain from the infinite measure).
In uncountable state spaces we proceed quite similarly. A key technical issue is the
definition of an analogue of Q for which we will need to assume that S has the structure
of a standard Borel space:
Lemma 31.3 Let (S, S) be a standard Borel space and let p be a transition probability on S.
Given µ P I there exists a transition probability q such that
ª ª
@A, B P S : µ(dx ) p( x, B) = µ(dy)q(y, A) (31.8)
A B
Moreover, if tXn unPZ is the two-sided Markov chain such that (31.1–31.2) hold, then tX´n unPZ
is a Markov chain with transition probability q.

Proof. Let n denote the joint distribution of ( X0 , X1 ) on (S ˆ S, S b S). Note that then
for each A, B P S, ª
n( A ˆ B) = µ(dx ) p( x, B) (31.9)
A
The fact that S is standard Borel implies that there exists a regular conditional distribu-
tion q of X0 given s( X1 ); i.e., a map q : S ˆ S Ñ [0, 1] which is S-measurable in the first
coordinate and a probability measure on (S, S) in the second coordinate such that, for
each A, B P S, ª ª
µ(dy)q(y, A) = 1B (y)q(y, A)dµ = n( A ˆ B) (31.10)
B

Preliminary version (subject to change anytime!) Typeset: June 1, 2025

MATH 275B&C notes 212

In conjunction with (31.9) this proves the equality (31.8).

Using (31.8) we now check that, for any m † n and any Am , . . . , An P S, we have
ª πn ª n´1
π
µ(dxm ) p( xi´1 , dxi ) = µ(dxn ) q( xi+1 , dxi ) (31.11)
Am ˆ¨¨¨ˆAn i = m +1 Am ˆ¨¨¨ˆAn i=m

which is the analogue of (31.7). The rest of the argument is the exactly as for countable S
and so we omit it. ⇤
Note that n( A ˆ B) § µ( B) which means that we can always try to define q as the
dn( Aˆ¨)
Radon-Nikodym derivative dµ . However, while this is automatically a measurable
function we cannot generally guarantee that this is a measure as a function of A. This is
where the technical assumption on the structure of S enters. As for the countable case,
one can make sense of q even when µ is an infinite s-finite measure. However, we will
not attempt to spell out the details.

31.2 Reversible measures.

With the law of the reversed chain identified as a Markov chain, a natural question is:
For what Markov chains does the reverse chain have the same law as the primal chain?
This leads us to the following concept:
Definition 31.4 (Reversible measures) Given a transition probability p on (S, S), a mea-
sure n on (S, S) is said to be reversible if
ª ª
@A, B P S : n(dx ) p( x, B) = n(dy) p(y, A) (31.12)
A B

As is checked by taking both A and B to be singletons, for countable S the above

condition reduces to
@x, y P S : n( x )P( x, y) = n(y)P(y, x ) (31.13)
where n denotes the probability mass function associated with n and P is the transition
kernel. Note that (31.13) holds automatically when x = y so we only need to check it
for x ‰ y. For the same reason, we only need to check this for x and y such that at least
one of P( x, y) ‰ 0 and P(y, x ) ‰ 0 hold. Another observation is:
Lemma 31.5 A reversible measure is automatically stationary.

Proof. Take A := S in (31.12) and use that p(y, S) = 1. ⇤

This suggests that, in order to identify stationary measures of a chain, we first search
for the reversible once. This gives us:
Another proof of Lemma 30.9. Observe that n defined in (30.26) obeys
ak P(k, k + 1)
n ( k + 1) = n ( k ) = n(k) (31.14)
b k +1 P(k + 1, k)
Since all transition happens between pairs of the form (k, k + 1), this shows that n is
reversible and, by Lemma 31.5, thus stationary. ⇤
Preliminary version (subject to change anytime!) Typeset: June 1, 2025
213 MATH 275B&C notes

Another example is the random walk on a weighted graph; see Example 29.13. Here
(using the notations there), p ( x )P( x, y) = a( x, y) and p is a reversible measure as soon
as a( x, y) = a(y, x ) for all pairs ( x, y). We will return to this example when we discuss
connection of Markov chains to electric networks.
While a passage through reversible measures is attractive, we should bear in mind
that reversibility is a special condition and that most Markov chains do not admit re-
versible measures, period. To show that the situation is even more interesting, recall
the example of a biased simple random walk on Z discussed in Lemma 30.11 where
we showed that every stationary measure is a linear combination an + bn1 (with non-
negative a and b) of the constant measure
@k P Z : n(k ) = 1 (31.15)
and the exponentially tilted measure
✓ ◆k
1´ p
@k P Z : n1 (k ) = (31.16)
p
We now readily check that n is not reversible unless p = 1/2 (when it coincides with n1 )
while n1 is reversible for all p P (0, 1).
As it turns out, for countable state spaces, the existence of reversible measures can be
linked to the geometry of the graph naturally associated with the Markov chain:
Theorem 31.6 (Kolmogorov’s cycle conditions) Let S be countable and let P be an irre-
ducible transition kernel. Then there exists a non-vanishing reversible measure if and only if
n
π n´1
π
@n • 1@x0 , . . . , xn P S : xn = x0 ñ P( xi´1 , xi ) = P ( x i +1 , x i ) (31.17)
i =1 i =0

Under this condition, the reversible measure is determined uniquely modulo normalization.

Proof. If n is a reversible measure then irreducibility implies n( x ) ° 0 for all x P S.

Multiplying the product on the left in (31.17) by n( x0 ) shows
n
π n
π n´1
π
n ( x0 ) P( xi´1 , xi ) = n( xn ) P( xi , xi´1 ) = n( xn ) P ( x i +1 , x i ) (31.18)
i =1 i =1 i =0

When xn = x0 we can cancel n( x0 ) = n( xn ) and get equality of the product.

For the converse, suppose that (31.17) hold and pick x0 , . . . , xn P S and x01 , . . . , xm
1 P S
1 1
with x0 = x0 and xm = xn and
n
π m
π
P( xi´1 , xi ) ° 0 ^ P( x1j´1 , x1j ) ° 0 (31.19)
i =1 j =1

The cycle condition (31.17) then implies

✓πn ◆✓π
m ◆ ✓π
m ◆✓π
n ◆
P( xi´1 , xi ) P( x1j , x1j´1 ) = P( x1j´1 , x1j ) P( xi , xi´1 ) (31.20)
i =1 j =1 j =1 i =1

Preliminary version (subject to change anytime!) Typeset: June 1, 2025

MATH 275B&C notes 214

and so all four products are strictly positive. This now rewrites as
n
π m P( x 1 , x 1 )
π
P( xi´1 , xi ) j´1 j
= (31.21)
i =1
P( xi´1 , xi ) j =1
P( x1j , x1j´1 )

implying that the product does not depend on the choice of the path from x0 to xn . We
now fix n( x0 ) ° 0 and define
n
π P( xi´1 , xi )
n ( x ) : = n ( x0 ) (31.22)
P( xi´1 , xi )
i =1
±
for any choice of the path±x0 , . . . , xn = x with in=1 P( xi´1 , xi ) ° 0, which by our pre-
vious argument ensures in=1 P( xi , xi´1 ) ° 0. Such a path exists by the assumed irre-
ducibility and so n( x ) ° 0 for all x P S.
For any distinct x, y P S with P(y, x ) ° 0 we then get
P( x, y)
n(y) = n( x ) (31.23)
P(y, x )
by concatenating y at the end of the path x0 , . . . , xn = x. This shows that P( x, y) ° 0
as well and that (31.13) holds and n is thus a reversible measure. To get the unique-
ness modulo normalization, observe that any reversible measure will obey (31.22) and
so n( x )/n( x0 ) does not depend on the choice of the measure. ⇤

31.3 Functional-analytic connection.

The existence of stationary measures opens up another line of approach to Markov

chains based on tools from functional analysis. We start with:
Lemma 31.7 Let p be the transition probability on (S, S) and let µ P I . Let a P [1, 8]. For
each f P La (µ) the integral defining function P f in (30.14) is well defined and finite µ-a.e. The
map f fiÑ P f defines a bounded linear operator P : La (µ) Ñ La (µ) with }P f }a § } f }a .

Proof. Note that for any measurable f , we have |P f ( x )| § (P| f |)( x ) with the integral
defining P f well-defined and convergent whenever (P| f |)( x ) † 8, so for existence it
suffices to show that P| f | † 8 µ-a.e. for all f P La . We will prove this along with the
bound on the La -norm. Indeed, a † 8 we invoke Jensen’s inequality to get
|P f |a § P | f |a (31.24)
Combining with stationarity of µ, this now yields
ª ª ª ª
ˇ ˇa
|P f | dµ § P | f | dµ = µ(dx ) p( x, dy)ˇ f (y)ˇ = µ(dy) f (y)a
a a
(31.25)

proving the desired inequality }P f }a § } f }a . For a = 8 we argue directly }P f }8 § } f }8 .

It follows that P is, on each La (µ), an everywhere-defined linear operator with operator
norm at most 1. ⇤
We now link to the topics discussed earlier via:

Preliminary version (subject to change anytime!) Typeset: June 1, 2025

215 MATH 275B&C notes

Lemma 31.8 Given a standard Borel space (S, S), a transition probability p and a stationary
distribution µ, let q be the transition probability of the reversed chain. Then the adjoint P+ of the
operator P from Lemma 31.7 on L2 (µ) takes the form
ª
+
P f ( x ) = q( x, dy) f (y), µ-a.e. (31.26)

In particular, µ is reversible if and only if P is self-adjoint.

≥
Proof. The space L2 (µ) is endowed with the canonical inner product x f , gy := f gdµ. A
calculation gives ª
x f , P gy = µ(dx ) p( x, dy) f ( x ) g(y)
ª (31.27)
= µ(dy)q(y, dx ) f ( x ) g(y) = xQ f , gy
where ª
Q f (y) := q(y, dx ) f ( x ) (31.28)
and where the middle equality follows by a routine extension from (31.8) via the Mono-
tone Class Theorem. (One first proves (31.27) for positive f and g using Tonelli’s theo-
rem and then argues the general case using Fubini’s theorem.) The functional-analytic
definition of the adjoint gives us Q = P+ which now applies the claim. ⇤
Clearly, the adjoint P+ exists in all circumstances (as P is bounded) and so the main
point of the representation (31.26) is that P+ is associated with a transition probability
of a Markov chain. The connection with self-adjointness is technically advantageous
because it enables arguments based on the spectral theorem. On the other hand, the
conclusions are then restricted to events of positive µ-measure which may not be always
advantageous. (This is why the theory of Markov processes is rather built over the
Banach space of continuous functions.)

Preliminary version (subject to change anytime!) Typeset: June 1, 2025