Scribd
Upload
15 views11 pages

SP 2 Stochastic Process

Uploaded by

TANISHA SINHA
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
15 views11 pages

SP 2 Stochastic Process

Uploaded by

TANISHA SINHA
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

Introduction to

Stochastic Processes
Introduction and its classification
• Stochastic Process: Develops the theory for understanding
randomness in process.
• A process is a sequence of events where each step follows from the
last after a random choice.

EXAMPLE: (Gambler’s Ruin Problem) You start with $x and toss a fair
coin repeatedly. Every time you throw a Head, you win $1. Every time
you throw a Tail, you lose $1. You will stop when you reach $N or when
you lose everything. What is the probability that you lose everything?
• You start with $30 and toss a fair coin repeatedly. Every time you
throw a Head, you win $5. Every time you throw a Tail, you lose
$5. You will stop when you reach $100 or when you lose
everything. What is the probability that you lose everything?

• P30 = 1 – 30/100 = 0.7


Example: Winning at tennis
What is your probability of winning a game of tennis, starting from the even
score Deuce (40-40), if your probability of winning each point is p = 0.3 and
your opponent’s is q = 0.7?

0.3 vD = P(Wins | start at state D)


0.3 Ahead Wins vA = P(Wins | start at state A)
0.7 vB = P(Wins | start at state B)
Deuce 0.7
vD = pvA + qvB
0.7 vA = p × 1 + qvD
0.3 Behind Loses
vB = pvD + q × 0

𝑝2
vD =
1−2𝑝𝑞
EXAMPLE:
Single channel system with Poisson arrivals and exponential service times.
• Arrival rate (λ) is less than the service rate (µ)
• Let n be the number of customers in the system at any time t
• Pn be the steady state probability of exactly n customers in the system

State diagram of the system


Rate In = Rate Out
At node 0, µ P1 = λ P0
At node 1, λ P0 + µ P2 = λ P1 + µ P1
At node 2, λ P1 + µ P3 = λ P2 + µ P2
.
.
At node n-1, λ Pn-2 + µ Pn = λ Pn-1 + µ Pn-1
At node n, λ Pn-1 + µ Pn+1 = λ Pn + µ Pn

At node 0, P1 = (λ/µ) P0
At node 1, P2 = (λ/µ)2 P0
By Induction we get, Pn = (λ/µ)n P0
where P0 is the probability that there is no customers in the system, or it
indicates that the proportion of time the system remains idle.
Definition and Classification
• A stochastic process is a family of random variables, instead of single
random variable, that is indexed by a parameter such as time.
• {X(t), t ∈ T } is a collection of random variables.
• For each t ∈ T, X(t) is a random variable referred as the state of the
process at time t (State Space, I); T is called the index set of the
process, often interpreted as time.
• For example, X(t) might be equal to the total number of customers
that have entered a supermarket by time t ; or the total amount of
sales that have been recorded in the market by time t;
• When Index set, T is a countable set or discrete, the stochastic process
is said to be a discrete-parameter process. If T is an interval of the real
line, the stochastic process is said to be a continuous-parameter
process.
• For instance, {X(t) , t = 0, 1, . . .} is a discrete-time stochastic process
indexed by the nonnegative integers;
• While {X(t), t ≥ 0} is a continuous-time stochastic process indexed by
the nonnegative real numbers.
• State Space, I can be discrete or continuous.
• I = {0, 1, 2, 3, … } is a discrete-state process also called as “Chain”.
• If state space is continuous, the called as continuous-state process
Index Set, T
Discrete Continuous
Discrete Discrete-parameter Continuous-parameter
stochastic chain stochastic chain
State Space, I
Continuous Discrete-parameter Continuous-parameter
continuous-state process continuous-state process

N(k) = number of jobs in the queuing system at the time of departure of kth customer.
{N(k) , k = 1, 2,. . .} ; I = {0, 1, 2, 3, …..}; T = {1, 2, 3, … }
X(t) = number of jobs in the queuing system at the time t
{X(t) | t ε T} ; I = {0, 1, 2, 3, …..}; T = { t | 0 ≤ t < ∞}
W(k) = Time that the kth customer has to wait in the system
{W(t) | t ε T} ; I = { x | 0 ≤ x < ∞ }; T = {1, 2, 3, … }

You might also like