Scribd
Upload
154 views2 pages

Markov Chain

A stochastic process is a collection of random variables indexed by time or some other parameter. It describes the evolution of a system through time. A stochastic process can be discrete-time if the index set is countable, or continuous-time if the index set is an interval of real numbers. The state space is the set of all possible values the random variables can take on.

Uploaded by

Dharmendra Kumar
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
154 views2 pages

Markov Chain

A stochastic process is a collection of random variables indexed by time or some other parameter. It describes the evolution of a system through time. A stochastic process can be discrete-time if the index set is countable, or continuous-time if the index set is an interval of real numbers. The state space is the set of all possible values the random variables can take on.

Uploaded by

Dharmendra Kumar
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 2

2.8. Stochastic Processes A stochastic process {X(t),t T } is a collection of random variables.

That is, for each

t T, X(t) is a random variable. The index t is often interpreted as time and, as a result, we refer to X(t) as the state of the process at time t . For example, X(t)might equal the total number of customers that have entered a supermarket by time t or the number of customers in the supermarket at time t ; or the total amount of sales that have been recorded in the market by time t; etc.
The set T is called the index set of the process. When T is a countable set the stochastic process is said to be a discrete-time process. If T is an interval of the real line, the stochastic process is said to be a continuous-time process. For instance, {Xn, n = 0, 1, . . .} is a discrete-time stochastic process indexed by the nonnegative integers; while {X(t), t > 0} is a continuous-time stochastic process indexed by the nonnegative real numbers. The state space of a stochastic process is defined as the set of all possible values that the random variables X(t) can assume. Thus, a stochastic process is a family of random variables that describes the evolution through time of some (physical) process. One can see much of stochastic processes in the Sec 2.8, Introduction to probability Models, Sheldon Ross.

1. Three white and three black balls are distributed in two urns in such a way that each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the first urn contains i white balls. At each step, we draw one ball from each urn and place the ball drawn from the first urn into the second, and conversely with the ball from the second urn. Let Xn denote the state of the system after the nth step. Explain why {Xn, n = 0, 1, 2,...} is a Markov chain and calculate its transition probability matrix. 2. In the Dark Ages, Harvard, Dartmouth, and Yale admitted only male students. Assume that, at
that time, 80 percent of the sons of Harvard men went to Harvard and the rest went to Yale, 40 percent of the sons of Yale men went to Yale, and the rest split evenly between Harvard and Dartmouth; and of the sons of Dartmouth men, 70 percent went to Dartmouth, 20 percent to Harvard, and 10 percent to Yale. We form a Markov chain with transition matrix

3. Modify Example 11.6 by assuming that the son of a Harvard man always went to Harvard. The transition matrix is now

4. (Ehrenfest Model) The following is a special case of a model, called the Ehrenfest model,that has been used to explain diffusion of gases. We have two urns that, between them, contain four balls. At each step, one of the four balls is chosen at random and moved from the urn that it is in into the other urn. We choose, as states, the number of balls in the First urn. The transition matrix is then

5. 6. 7. 8. 9. 10. 11. 1. 2. 3. 4. 5. 6. 7.

You might also like