Scribd
Upload
5 views1 page

hw3 2024

Stochastic processes

Uploaded by

samrathalderextra2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
5 views1 page

hw3 2024

Stochastic processes

Uploaded by

samrathalderextra2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

Home Work 3 (2024)

MTH 212M/ MTH 412A


Applied Stochastic Process

1. Prove or give a counter example that any relation operation is reflexive, i.e. a is related
to a, for all a ∈ S.

2. Prove or give a counter example that any relation operation is symmetric, i.e. if a is
related to b, then b is related to a.

3. Prove or give a counter example that any relation operation is transitive, i.e. if a is
related to b and b is realted to c, then a is related to c.

4. Prove that if S is countable, then an equivalence relation divides S into disjoint equiv-
alent classes.

5. Find the transition probability matrix P of the following Markov Chain. Consider a
sequence of tosses of a coin with the probability of head p. At time n, after n tosses
of the coin, the state of the process is the number of heads in the n tosses minus the
number of tails. Find the number of equivalent classes of the Markov Chain.

6. Find the transition probability matrix P of the following Markov Chain. N black balls
and N white balls are placed in two urns so that each urn contains N balls. At each
step one ball is selected at random from each urn, and the two balls are interchanged.
The state of the system is the number of white balls in the first urn. Find the number
of equivalent classes of the Markov Chain. Is it an irreducible Markov Chain? Find
the periods of each state.

7. Determine the classes and the periodicity of the various states for a Markov Chain
with the following transition probability matrix.
0 0 1 0 0 1 0 0
   
1 0 0 0 0 0 0 1
(a)  1 (b) 
   
1
0 0 0 1 0 0
 
2 2
1 1 1 1 2
3 3 3
0 3
0 3
0

8. Prove or give a counter example that if P is a doubly stochastic matrix then Pn is also
a doubly stochastic matrix.

9. Let a Markov Chain has r states. Prove that if the state k can be reached from j, then
it can be reached in r − 1 steps or less.

10. Let
1−a a 1 b a (1 − a − b)n a −a
     
n
P= then P = +
b 1−b a+b b a a+b −b b

You might also like