Scribd
Upload
19 views

Assignment 1

This document contains the details of Assignment 1 for the course BSCS 32063 - Stochastic Processes. It includes 4 questions related to stochastic processes and Markov chains, covering topics like binomial distribution, transition probability matrices, recurrent states, and absorption probabilities. Students are asked to determine distributions, analyze Markov chain properties, calculate expectations and variances, and find transition and absorption probabilities. The assignment is due on Monday, May 7 at 5:30pm.

Uploaded by

sanka
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
19 views

Assignment 1

This document contains the details of Assignment 1 for the course BSCS 32063 - Stochastic Processes. It includes 4 questions related to stochastic processes and Markov chains, covering topics like binomial distribution, transition probability matrices, recurrent states, and absorption probabilities. Students are asked to determine distributions, analyze Markov chain properties, calculate expectations and variances, and find transition and absorption probabilities. The assignment is due on Monday, May 7 at 5:30pm.

Uploaded by

sanka
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

BSCS 32063 - Stochastic Processes

Assignment 1, due Monday, May. 7 at 5:30pm

1. A coin having probability p of resulting in heads is successively flipped until the rth
head appears. Let X be the total number of flips required. What is the distribution of
X? (Give the name and explain why you picked that distribution)
Argue that the probability mass function of X is given by
 
n−1 r
p(k) = p (1 − p)n−r for k=r, r+1, . . . .
r−1

Hint: How many successes must there be in the first n − 1 trials?

2. Consider a Markov chain on S = {1, 2, 3, 4, 5, 6} specified by the transition probability


matrix
1 3

4 4 0 0 0 0
1 1
0 0 0 0
2 2 
0 0 1 0 0 0
 
P = 1 2 .
0 0 3 3 0 0
 
1 0 0 0 0 0
0 0 0 0 0 1

(a) What are the (communicating) classes of this Markov chain? Is the Markov chain
irreducible ?

(b) Which states are transient and which are recurrent? Justify your answers.

(c) What is the period of each state of this Markov chain? Is the Markov chain ape-
riodic ?

(d) Let X0 be the initial state with distribution p(0) = (0, 41 , 34 , 0, 0, 0)> corresponding
to the probability of being in states 1, 2, 3, 4, 5, 6 respectively.
Let X0 , X1 , X2 , . . . be the Markov chain constructed using P above. What is
E(X1 )?
(e) What is Var(X1 )?
(f) What is E(X3 )?

3. The times that Tom wakes up on any given morning are 5:30 am, 6:00 am or 7:00 am
Assume that if Tom wakes up at 5:30 am on a given morning, then the probability that
he wakes up at 5:30 am the next morning is 80%, at 6:00 am is 10%, and at 7:00 am is
10%.
If Tom wakes up at at 6:00 am on a given morning, then the probability that he wakes
up at 5:30 am the next morning is 20%, at 6:00 am is 60%, and at 7:00 am is 20%.
If Tom wakes up at at 7:00 am on a given morning, then the probability that he wakes
up at 5:30 am the next morning is 25%, at 6:00 am is 25%, and at 7:00 am is 50%.

(a) Form a Markov chain by following Tom’s daily waking up time over a few days.
Define the corresponding random process and the state space clearly.

(b) Set up the matrix of transition probabilities.

(c) Find the probability of Tom waking up tomorrow morning at 5:30 am if he woke
up at 7:00 am yesterday morning.

4. (a) Explain what is meant by a ”recurrent state”.

(b) Consider the Markov chain given below.

There are two recurrent classes, R1 and R2 . Given that R1 = {1, 2},
i. Find R2 .

ii. Assuming X0 = 3, find the probability that the chain gets absorbed in R1 .

You might also like