Solutions to Homework 4 - Markov Chains

1) Transforming a random process into a Markov chain. Consider the random process XN = X0 , X1 , . . . , Xn , . . .
with state space S = {0, 1, 2}. Suppose
 e
  Pij , n even
P Xn+1 = j Xn = i, Xn−1 = in−1 , . . . , X0 = i0 = (1)
Pijo , n odd
P2 P2
where j=0 Pije = j=0 Pijo = 1, i ∈ S. As defined, XN is not a Markov chain (MC) because the probabilities in
(1) depend on the time step n (through whether n is even or odd). This contradicts the basic defining property of
a MC, that requires sole dependence on i and j.
The state space of XN is S = {0, 1, 2} and we consider the augmented state space S = {0, 1, 2, 0, 1, 2} where
0, 1, 2 are the original states when n is even and 0, 1, 2 accounts for the same respective states when n is odd. In
this case we obtain a MC as desired, with transition probability matrix
e e e
 
0 0 0 P00 P01 P02
 0 e e e 
 0 0 P10 P11 P12 
 0 e e e 
0 0 P20 P21 P22 
P=  P00o o o .
 o P01 o
P02
o
0 0 0  
 P10 P11 P12 0 0 0 
o o o
P20 P21 P22 0 0 0

2) Coin flips. Suppose that coin 1 has probability 0.7 of coming up heads whereas coin 2 has probability 0.6 of
coming up heads. If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow, otherwise we
select coin 2 to flip tomorrow. The coin initially flipped is equally likely to be coin 1 or coin 2.
Let Xn be the coin flipped at day n. Given the previous definition of the problem, clearly XN = X0 , X1 , . . . , Xn , . . .
constitutes a MC with state space is S = {1, 2}. The transition matrix is
 
0.7 0.3
P= .
0.6 0.4
The initial distribution is P [X0 = 1] = 1/2 and P [X0 = 2] = 1/2 which we can arrange in a column vector
p(0) = [1/2, 1/2]T .
We are interested in computing the probability that the coin flipped on the third day after the initial flip is coin 1.
Given our notation, this can be translated as computing the unconditional probability P [X3 = 1]. To that end, we
will need to compute the three-step transition probability matrix
 
0.667 0.333
P(3) = P3 = P × P × P = .
0.666 0.334
Using total probability we have
3 3
   
P [X3 = 1] = P X3 = 1 X0 = 1 P [X0 = 1] + P X3 = 1 X0 = 2 P [X0 = 2] = P11 P [X0 = 1] + P21 P [X0 = 2]
which corresponds to the first entry of the vector
    

T 0.667 0.666 1/2 0.6665
p(3) = P3 p(0) = = .
0.333 0.334 1/2 0.3335
Accordingly, the answer is P [X3 = 1] = 0.6665.
Moving on to the next question, suppose that the coin flipped on Monday comes up heads. We want to compute
the probability that the coin flipped on Friday of the same week also comes up heads. Let n0 denote the Monday
when the coin was flipped. We know that Xn0 +1 = 1, because the flip that took place at time n0 came up heads.
The Friday of interest corresponds to time n0 + 4 and so we are asked to compute
4
 
P Xn0 +5 = 1 Xn0 +1 = 1 = P11 .
 Class Class

Fig. 1. State transition diagrams for the MCs with transition probility matrices (left) P1 and (right) P2 .

The four-step transition probability matrix is

 
0.6667 0.3333
P(4) = P4 = P × P3 =
0.6666 0.3334
 
and therefore the desired result is P Xn0 +5 = 1 Xn0 +1 = 1 = 0.6667.
3) Positive transition matrices remain positive. Let P be the transition probability matrix of a MC with s states,
say. Suppose that for some positive integer r, Pr has all positive entries. We want to show that so does Pn for
n ≥ r. By an induction argument, it is clear that it suffices to show that Pr+1 = P × Pr has all positive entries.
Let pTi denote the i-th row vector of the transition matrix P (with entries Pik ∈ [0, 1] for all k = 1, . . . , s), and
let prj denote the j-th column vector of the r-step probability transition matrix Pr (with entries Pkj r
∈ (0, 1] for
r+1 r
all k = 1, . . . , s). The i, j-th entry of the product P = P × P is given by the inner product
s
X X
Pijr+1 = pTi prj = r
Pik Pkj = r
Pik Pkj > 0.
k=1 k s.t. Pik >0

The probability Pijr+1 is positive because the last summation only involves Ps strictly positive summands. (Notice that
not all Pik , k = 1, . . . s, can be identically zero because we must have k=1 Pik = 1.) It is clear that the previous
result holds for all i, j with i = 1, . . . , s and j = 1, . . . , s, establishing that Pr+1 will have all positive entries as
desired.
4) Communication classes. A MC with state space S1 = {0, 1, 2} has the following transition matrix
 
0 1/2 1/2
P1 =  1/2 0 1/2  .
1/2 1/2 0
The information in P1 is also summarized in the state transition diagram shown in Fig. 1 (left). From the diagram
it is clear that all states communicate, and therefore they all belong to a single class C1 = {0, 1, 2}. The MC is
irreducible, and all the states in the class have to be recurrent because the state space is finite.
A second MC is considered with state space S2 = {0, 1, 2, 3}. The transition probability matrix is
 
0 0 0 1
 0 0 0 1 
P2 =  1/2 1/2 0 0 


0 0 1 0
and the corresponding state transition diagram is also shown in Fig. 1 (right). As in the first case we observe that
all states communicate, and therefore they all belong to a singles class C1 = {0, 1, 2, 3}. The MC is irreducible,
and all the states in the class have to be recurrent because the state space is finite.
 Class
Class Class

Class
Class

Class

Class

Fig. 2. State transition diagrams for the MCs with transition probility matrices (left) P3 and (right) P4 .

The third considered MC has state space S3 = {0, 1, 2, 3, 4} and transition probability matrix
 
1/2 0 1/2 0 0
 1/4 1/2 1/4 0 0 
 
P3 =  1/2 0 1/2 0 0  .
 0 0 0 1/2 1/2 
0 0 0 1/2 1/2
The state transition diagram is shown in Fig. 2 (left). Notice that states 0 and 2 communicate, whereas state 1 is not
accessible from any other state except from itself. Finally states 3 and 4 also communicate but are not accessible
from any other state. Accordingly, in this case we encounter three communication classes, C1 = {0, 2}, C2 = {1}
and C3 = {3, 4}. The states in the classes C1 and C3 are all recurrent because if the MC is initialized in either
class, it will stay there forever. On the other hand, state 1 (and hence class C2 ) is transient, because starting from
state 1 the MC will almost surely leave the state and never return.
For the last MC the state space is S4 = {0, 1, 2, 3, 4}. The transition probability matrix is given by
 
1/4 3/4 0 0 0
 1/2 1/2 0 0 0 
 
P4 =  0
 0 1 0 0  
 0 0 1/3 2/3 0 
1 0 0 0 0
while the corresponding state transition diagram is shown in Fig. 2 (right). From the state diagram we observe that
states 0 and 1 communicate and form the first class C1 = {0, 1}, whose states are recurrent. State 2 is an absorbing
state, therefore it is recurrent and it forms a second class C2 = {2}. Finally states 3 and 4 are not accesible from any
other state, and they separately form two more classes C3 = {3} and C4 = {4}. These states are clearly transient.
5) Multi-step, unconditional, and joint MC probabilities. Suppose that XN = X0 , X1 , . . . , Xn , . . . is a MC with
state space S = {1, 2}, transition probability matrix
 
1/5 4/5
P=
2/5 3/5
and initial distribution P [X0 = 1] = 3/4 and P [X0 = 2] = 1/4.
  2
We first want to compute the conditional probability P X3 = 1 X1 = 2 = P21 . To that end, the two-step
(2)
transition probability matrix P is
 
0.36 0.64
P(2) = P2 = P × P = .
0.32 0.68
  2
Hence, the desired result is P X3 = 1 X1 = 2 = P21 = 0.32.
Second, from the Markov property we have
    1
P X3 = 1 X2 = 1, X1 = 1, X0 = 2 = P X3 = 1 X2 = 1 = P11 = .
5
Next, the unconditional probability P [X2 = 2] can be calculated from the initial distribution p0 = [3/4 1/4]T and
the second row of the transposed two-step transition matrix (P2 )T . In fact, since
    2 3 2 1
P [X2 = 2] = P X2 = 2 X0 = 1 P [X0 = 1] + P X2 = 2 X0 = 2 P [X0 = 2] = P12 × + P22 ×
4 4
we finally obtain  
3/4
P [X2 = 2] = (0.64 0.68) = 0.65.
1/4
To evaluate the required joint probability, use the definition of conditional probability, i.e.,
2
 
P [X0 = 1, X2 = 1] = P X2 = 1 X0 = 1 P [X0 = 1] = P11 × P [X0 = 1] = 0.36 × 0.75 = 0.27.
Finally, to find the unconditional expectation E [X2 ] recall we already know P [X2 = 2] = 0.65, which implies
P [X2 = 1] = 0.35. Accordingly, the desired expectation is
E [X2 ] = 1 × 0.35 + 2 × 0.65 = 1.65.

6) Something the Markov property does not imply. Suppose that XN = X0 , X1 , . . . , Xn , . . . is a MC, j, in−1 , . . . , i1 , i0
are elements of the state space S, and A is a subset of the state space. In this problem we will find an example
for which
   
P Xn+1 = j Xn ∈ A, Xn−1 = in−1 , . . . , X1 = i1 , X0 = i0 6= P Xn+1 = j Xn ∈ A .
Such an example can be constructed as follows; suppose that a single state i that belongs to the subset A is
accessible from the state in−1 . So for this particular MC, the event of interest satisfies
{Xn+1 = j Xn ∈ A, Xn−1 = in−1 , . . . , X0 = i0 } = {Xn+1 = j Xn = i, Xn−1 = in−1 , . . . , X0 = i0 }
because we know that given Xn−1 = in−1 , the only accessible state from A at time instant n (and generally any
other time instant) is i. So from the Markov property we have
P [Xn+1 = j|Xn ∈ A, Xn−1 = in−1 , . . . , X0 = i0 ] = P [Xn+1 = j|Xn = in , Xn−1 = in−1 , . . . , X0 = i0 ] = Pij
Suppose that A = S, then because {Xn ∈ S} is the sure event we have
   
P Xn+1 = j Xn ∈ A = P Xn+1 = j Xn ∈ S = P [Xn+1 = j] = pj (n + 1)
which is an uncoditional probability that depends on the initial distribution p(0), and the time instant n + 1. It
should be clear now that the quantities Pij and pj (n + 1) will be different in general, and thus making the required
point.
For a concrete example, consider a MC with state space S = {1, 2}, transition matrix
 
1/2 1/2
P=
1 0
and initial distribution P [X0 = 1] = 1/4, P [X0 = 2] = 3/4. Pick A = {1, 2} and note that only state 1 ∈ A is
accessible from state 2. Therefore we can compute, for n = 1 say
    1
P X2 = 1 X1 ∈ A, X0 = 2 = P X2 = 1 X1 = 1, X0 = 2 = P11 = .
2
On the other hand,
  2 2 3 1 1 3 9
P X2 = 1 X1 ∈ A = P [X2 = 1] = P11 P [X0 = 1] + P21 P [X0 = 2] = × + × = .
4 4 2 4 16
7) A branching process.
A) Is the count of women a Markov chain? The process XN which indicates the total amount of women across
generations is clearly a MC with state space S = {0, 1, 2, . . .}, since it satisfies the Markov property. To see this,
note that considering a particular generation, the number of women in the next one depends exclusively on the
number of women in the present generation and is independent of the past history of women population. Formally,
we can write
XXn
Xn+1 = Di if Xn > 0
i=1

and Xn+1 = 0 if Xn = 0.
Exploring some of the transition probabilities of this MC it is immediate to realize that

0 if j 6= 0
P0j = P [Xn+1 = j|Xn = 0] = (2)
1 if j = 0
since once the total population of women vanishes it cannot increase in the future. Similarly, we can assert that
"X #
X n

P1j = P [Xn+1 = j|Xn = 1] = P Di = j|Xn = 1 = P [D1 = j] = pj , j ≥ 0

i=1

The above expression has a simple interpretation, the amount of women in the next generation depends only on
the number of daughters the only woman in the present generation has. Likewise, for i = 2 we can write
j
"X #
Xn X
P2j = P Di = j|Xn = 2 = P [D1 + D2 = j] = pk pj−k
i=1 k=0

which corresponds to the probability of having j women in the next generation if there are two in the present one.
This can be obtained as the sum of the probabilities that the first woman has no daughters and the second one has j,
plus the first woman having one daughter and the second one having j − 1, and so on. The assumed independence
between D1 and D2 allows us to write this as the product of both probabilities. Hence, we see that calculating P2j
requires solving a discrete convolution between the pmf of D1 and the pmf of D2 (which are the same function
since the Di are i.i.d.). In general, calculating Pij requires solving an i-term convolution whose analytical difficulty
depends on the distribution of Di . (See also exercise 7 in the practice midterm for the general expression.)
Nevertheless, we can still characterize even more transition probabilities. For example,
i
Y i
Y
Pi0 := P [Xn+1 = 0|Xn = i] = P [Dk = 0] = p0 = pi0
k=1 k=1

is the probability of the population becoming extinct in the next generation given that there are i individuals in the
current one. For this to happen, every woman must bear no daughters and, by independence, the above expression
follows.
We can also deduce that the probability of a state transitioning into itself is positive for all states i. We already
know this is true for i = 0 and, for i > 0 note that one possible way of maintaining the same amount of women
through a generation is that every woman has exactly one daughter. This motivates the following inequality
Pii = P Xn+1 = i Xn = i ≥ pi1 > 0, i ≥ 0.
 

Here we are assuming that the probability of bearing exactly one daughter is strictly positive.
While state 0 (extinction) is an absorbing state and hence clearly recurrent, for all other states i 6= 0 there is a
positive probability pi0 of going from i to 0 which is an absorbing state. Accordingly, every state different from 0
is a transient state and the MC is neither irreducible nor recurrent.
B) Is the count of women classified by mitochondrial DNA type a Markov chain? Consider now the amount of
women Xrn with mitochondrial DNA of type r, the process XrN is not a MC. To see this, focus on the state 0,
that is, the state that indicates that there are no women of type r in the current generation. This may happen as
a consequence of two very  It might be the case that the type r existed in the past and is now
 different situations.
extinct, in which case, P Xrm > 0 Xrn = 0 = 0, for all m > n. However, it might be the case that the mutation
type r has not been created yet. In this case, there is positive probability of having Xrm > 0 for some future
generation. Consequently, the transition probabilities out of the state 0 depend on the past. Since the process does
not satisfy the Markov property, it is not a MC. Note that this is the only state that presents this inconvenience.
Suppose now that we are given the information that at some time n, Xrn > 0. Define the random process
Xr,n:∞ := Xrn , Xr,n+1 , . . . that starts at time n with the information that Xrn > 0. This process is clearly a MC
since the state 0 has now only one interpretation, i.e. an absorbing state as the one described for the process XN
with P0j = 0 for any j 6= 0 and P00 = 1. Slightly abusing notation, here Pij denotes the transition probability
from state i to state j in the process Xr,n:∞ . The transition probabilities out of the state 1 are different from the
ones in XN . In particular,
P1j = (1 − q)pj , j > 0
meaning, the probability of having exactly j daughters without a mutation in their mitochondrial DNA. For
transitioning from 1 to 0 we have
P10 = p0 + (1 − p0 )q = q + (1 − q)p0
where both expressions are trivially equal but they arise from different arguments. The first one expresses that the
population of women of type r becomes extinct if one of two mutually exclusive events occur: the woman has no
daughters or she has daughters and they suffer a DNA mutation. The second expression indicates the probability
of the union of the events that the daughters present the mutation or that they do not present the mutation but
the woman bears no daughter. Note that P10 6= p0 + q since the events of having no daughters and presenting the
mutation in the DNA are not disjoint.
Appealing to independence, we readily obtain
i

i
Pi0 = P10 = p0 + (1 − p0 )q , i ≥ 0
since for the population of type r to become extinct every woman must have either no daughters or some daughters
and present the mutation.
By following a reasoning analogous to the one performed for XN in Part A, we can see that there is a strictly
positive probability of self transitioning for all states i, that is,
i
Pii ≥ ((1 − q)p1 ) > 0, i ≥ 0.
Similarly, recurrence can be analyzed to find that 0 is the only recurrent state, just as in the previous process. Again,
the MC is neither irreducible nor recurrent.
C) System simulation. Next, you can find a Matlab script to perform the system simulation, and generate the plots
requested for Parts D and E.

close all; clear;clc;

X_o = 100;
max_t = 50;
max_types = 1000; % a safely large number (matlab reallocates size if this
% is not enough)
mu = 1.05;
q = 10ˆ-2;

X=zeros(max_types, max_t); % preallocate vector for population size

number_of_types=zeros(1, max_t); % preallocate vector for number of types
X(1:X_o,1) = 1;
number_of_types(1)=X_o; %initialize first generation
number_of_extinct_types=zeros(1,max_t);

for n=2:max_t
disp([’n=’,num2str(n)])
number_of_types(n)=number_of_types(n-1);
for type = 1:number_of_types(n-1);
for i = 1:X(type,n-1)
daughters = poissrnd(mu,1,1); % daw number of daughters
mutation = binornd(1,q,1,1); % draw mutation indicator
if mutation
 number_of_types(n) = number_of_types(n)+1;
X(number_of_types(n),n) = daughters;
else
X(type,n) = X(type,n) + daughters;
end % if-else
end % i
if X(type,n)== 0
number_of_extinct_types(n)=number_of_extinct_types(n)+1;
end % if
end % type
end % n

figure
plot(1:max_t, X)
xlabel(’generation’,’FontSize’,14)
ylabel(’number of women of each type’,’FontSize’,14)
title(’$q=10ˆ{-2},$ $X_{0}=100,$ $n=50$’,’FontSize’,14,’Interpreter’,’latex’)

figure
stairs(1:max_t, X’)
xlabel(’generation’,’FontSize’,14)
ylabel(’number of women of each type’,’FontSize’,14)
title(’$q=10ˆ{-2},$ $X_{0}=100,$ $n=50$’,’FontSize’,14,’Interpreter’,’latex’)

figure
stairs(1:max_t, [number_of_types;number_of_extinct_types]’,’LineWidth’,2)
xlabel(’generation’,’FontSize’,14)
ylabel(’number of women of each type’,’FontSize’,14)
title(’$q=10ˆ{-2},$ $X_{0}=100,$ $n=50$’,’FontSize’,14,’Interpreter’,’latex’)
axis([0 50 0 number_of_types(end)])
legend(’number of types’,’number of extinct types’,’Location’,’Best’)

figure
bar(1:number_of_types(end), X(1:number_of_types(end),max_t),’r’)
xlabel(’types’,’FontSize’,14)
ylabel(’number of women of each type’,’FontSize’,14)
title(’Histogram of the final number of women of each type’,’FontSize’,14)

D) Simulation tests one. Here we use the script in Part C to run a simulation with q = 10−2 as the rate of mutation,
generation 0 initialized with X0 = 100 individuals all having different mitochondrial DNA types. The branching
process evolves for a horizon of nmax = 50 generations. The results are depicted in in Fig. 3.
E) Simulation tests two. We repeat Part D with rate of mutation q = 0 and X0 = 400 individuals of different types.
This tells you about the chances of your direct female line surviving into the next 10 centuries. The results are
depicted in in Fig. 4. Notice that most of the types go extinct, a few have a moderate number of individuals and 1
or 2 have a large number of individuals. This means that far into the future, most of your direct female lines will
be extinct, but one of you will have a very large number of survivors. But who among you is the one surviving
into the future is determined by chance.
F) Expected value of the number of direct line female descendants. As mentioned in Part A, the number of individuals
in the n + 1-st generation can be written as
Xn
X
Xn+1 = Di . (3)
i=1
 q = 10 − 2, X 0 = 100, n = 50 q = 10 − 2, X 0 = 100, n = 50
350 350

300 300
number of women of each type

number of women of each type

250 250

200 200

150 150

100 100

50 50

0 0
0 5 10 15 20 25 30 35 40 45 50 0 5 10 15 20 25 30 35 40 45 50
generation generation

q = 10 − 2, X 0 = 100, n = 50 Histogram of the final number of women of each type

300

350

250
300
number of women of each type

number of women of each type

250 200

200
150

150

100

100
number of types
number of extinct types 50
50

0 0
0 5 10 15 20 25 30 35 40 45 50 0 50 100 150 200 250 300 350 400
generation types

Fig. 3. Simulation results starting from 100 types and after 50 generations, where q = 0.01. (Part D). (top-left) Evolution of types (using
plot); (top-right) Evolution of types (using stairs); (bottom-left) Number of overall types and extinct types over generations; and (bottom-right)
Histogram of the final number of each type.

From (3) we can compute the expected value of the total population in the n-th generation, i.e. E [Xn ] := µn . Since
we are dealing with a summation, the linearity of the expected value operator will be a useful tool. However, it is
important to note that Xn−1 as well as Di are random variables. Hence, we are dealing with a summation of a
random number of RVs, also known as compound RV. The good news is that Xn−1 and Di are independent. This
is a direct consequence of the problem definition, since the child bearing probability of a given woman does not
depend on the total population of her generation. To determine µn , begin by conditioning on the value of Xn−1 to
arrive at
 
Xn−1
" k # " k # k k
X X X X X
E  Di Xn−1 = k = E
 Di Xn−1 = k = E Di = E [Di ] = ν = kν.
i=1 i=1 i=1 i=1 i=1

Now using iterated expectations we can determine E [Xn ] through the following recursive relationship
∞
X ∞
X
E [Xn ] = E [Xn |Xn−1 = k] P [Xn−1 = k] = kνP [Xn−1 = k] = νE [Xn−1 ] .
k=0 k=0

We can apply this result repeatedly, since its true for all n > 0,
E [Xn ] = E [Xn−1 ] ν = E [Xn−2 ] ν 2 = · · · = E [X0 ] ν n = X0 ν n ,
where E [X0 ] = X0 . Note that in this way we have written the expected value of the total population in any
future generation as a function of the generation, the initial population and the average fertility of every woman.
In particular, if the number of daughters of every woman is modeled as Poisson with parameter λ as we have done
so far, we have
E [Xn ] = X0 λn . (4)
 q = 0, X 0 = 400, n = 50 q = 0, X 0 = 400, n = 50
500 500

450 450

400 400
number of women of each type

number of women of each type

350 350

300 300

250 250

200 200

150 150

100 100

50 50

0 0
0 5 10 15 20 25 30 35 40 45 50 0 5 10 15 20 25 30 35 40 45 50
generation generation

q = 10 − 2, X 0 = 100, n = 50 Histogram of the final number of women of each type

500
400

450
350
400
number of women of each type

number of women of each type

300
number of types 350
number of extinct types
250 300

200 250

200
150

150
100
100

50
50

0 0
0 5 10 15 20 25 30 35 40 45 50 0 50 100 150 200 250 300 350 400 450
generation types

Fig. 4. Simulation results starting from 400 types and after 50 generations, where q = 0. (Part E). (top-left) Evolution of types (using plot); (top-
right) Evolution of types (using stairs); (bottom-left) Number of overall types and extinct types over generations; and (bottom-right) Histogram
of the final number of each type.

If we fix λ > 1 as in our previous simulations, we have an exponential increase in the expected value of the
population with time. To illustrate the validity of this result, Figure 5 depicts the total population and the expected
value as a function of time for the first of the simulations already analyzed. We see that the theoretical result is
validated by this particular realization of the process.
Following an analogous procedure, we arrive to the conclusion that
µrn = E [Xrn ] = E [Xr0 ] νrn = νrn = (1 − q)n ν n , (5)
where assumed that every DNA type is associated with an original population of one woman (this gives the expected
number of descendants of any given generation 0 individual).
G) Extinction in probability and almost sure extinction. From Markov’s inequality, since the RVs Xrn are nonneg-
ative we have that for arbitrary  > 0, and all n ≥ 0
µrn νn
P [Xrn ≥ ] ≤ = r.
 
If νr < 1, taking limits as n → ∞ yields
νrn
lim P [Xrn ≥ ] ≤ lim
= 0 ⇒ lim P [Xrn < ] = 1.
n→∞ n→∞  n→∞

This establishes that for νr < 1, type r goes extinct in probability.

H) Probability of eventual extinction (q = 0). Fix q = 0. Denote as Pe (j) the probability of eventual extinction of
mitochondrial type r, when the number of generation 0 type r individuals is Xr0 = j, i.e.,
   
Pe (j) := lim P Xrn = 0 Xr0 = j := P Xr∞ = 0 Xr0 = j .
n→∞
 1200

1000

800

total population
600

400

200

0
0 5 10 15 20 25 30 35 40 45 50
generation

Fig. 5. The expected total population is plotted in blue as described by (4). In green, the total population from the first simulation (q = 0.01,
λ = 1.05, X0 = 100) is plotted. The numerical results validate the theoretical analysis (Part F).

We already showed that for ν < 1, then Pe (j) = 1 independent of j. When ν > 1 is turns out that Pe (j) < 1, and
the equation determining Pe (1) can be obtained by conditioning on the number of daughters in the first generation,
and using the law of total probability. Upon conditioning on Xr1 = j, note that
     j j
P Xr∞ = 0 Xr1 = j, Xr0 = 1 = P Xr∞ = 0 Xr1 = j = P Xr∞ = 0 Xr0 = 1 = [Pe (1)] .
Given that at generation 1 there are j individuals, the key observation here is that from the Markov property the
probability of eventual extinction is the probability that each of these j individual lines goes extinct. Because these
j
events are independent, we have that P Xr∞ = 0 Xr1 = j = [Pe (1)] . Now we apply the law of total probability
to obtain ∞ ∞
X     X j
Pe (1) = P Xr∞ = 0 Xr1 = j, Xr0 = 1 P Xr1 = j Xr0 = 1 = [Pe (1)] pj
j=0 j=0

as desired. If in general one has Xr0 = j individuals in generation 0, by independence it follows that
j
Pe (j) = [Pe (1)]
since eventual extinction of the whole population necessitates the extinction of all j individual lines.