Scribd
Upload
8 views14 pages

L2 Queing Theory

The document provides an introduction to stochastic processes, defining key concepts such as parameter space, state space, and types of processes including discrete and continuous time processes. It discusses examples like random walks, Bernoulli processes, and Poisson processes, highlighting their properties and applications. The document serves as a foundational overview for understanding the behavior and characteristics of stochastic processes.

Uploaded by

Tej
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views14 pages

L2 Queing Theory

The document provides an introduction to stochastic processes, defining key concepts such as parameter space, state space, and types of processes including discrete and continuous time processes. It discusses examples like random walks, Bernoulli processes, and Poisson processes, highlighting their properties and applications. The document serves as a foundational overview for understanding the behavior and characteristics of stochastic processes.

Uploaded by

Tej
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

CS 3.

307: Intro to Stochastic Processes

Tejas Bodas

Assistant Professor, IIIT Hyderabad

1 / 23
Introduction to Stochastic processes

2 / 23
Introduction to Stochastic processes
▶ Stochastic process {X (t), t ∈ T } on a probability space
(Ω, F , P) is a collection of random variables defined such that
for every t ∈ T we have X (t) : Ω → S.

▶ T is the parameter space (often resembles time) and S is the


state space.

▶ Random variable X (t) is often denoted by X (ω, t).

▶ When t is fixed and ω is the only variable, we have a random


variable X (·, t). When ω is fixed and t is the variable, we
have a X (ω, ·) as a function of time. This is also called as a
realization or sampe path of a stochastic process.

3 / 23
Introduction to Stochastic processes

▶ When T is countable, we have a discrete time process.

▶ If T is a subset of real line, we have a continuous time


process.

▶ State space could be integers or real numbers

▶ State space could be Rn or Zn valued

4 / 23
Elementary Examples

▶ The process of rolling a dice 6 times.

▶ You bank balance over a week.

▶ Temperature fluctuations in a 1hr window.

▶ Number of customers in IKEA every day.

5 / 23
Introduction to Stochastic processes

A c.t.s.p. is called an independent increment process if for


any choice of parameters t0 < t1 < . . . < tn , the n increment
random variables X (t1 ) − X (t0 ), X (t2 ) − X (t1 ), . . . , X (tn ) −
X (tn−1 ) are independent.

The c.t.s.p. is said to have stationary increments if in addition


X (t2 +s)−X (t1 +s) has the same distribution as X (t2 )−X (t1 )
for all t1 , t2 ∈ T and any s > 0.

6 / 23
Examples
▶ Sequence of i.i.d random variables.

▶ General random walk: If X1 , X2 , . . . is a sequence i.i.d of


Pn
random variables, then Sn = i=1 Xi is a random walk.

▶ Weiner process: {X (t), t ≥ 0} is a Weiner process if


1. X (0) = 0
2. {X (t), t ≥ 0} has stationary and independent increments
3. for every t > 0, X (t) is normally distributed with mean 0 and
variance t.

▶ {X (t), t ≥ 0} is a Markov process if for t1 < t2 < . . . tn < t


we have
P(X (t) ≤ x |X (t1 ) = x1 , . . . , X (tn ) = xn ) = P(X (t) ≤ x |X (tn ) = xn )

▶ Random walk and Weiner process are examples of Markov


processes.
7 / 23
Bernoulli/Binomial process

▶ Bernoulli(p) random variable

▶ Bernoulli process is a sequence of independent r.v.’s


{Xi , i = 1, 2, . . .} where each Xi is a Bernoulli(p) random
variable.

▶ Binomial random variable Sn counts the sum of n independent


Bernoulli(p) variables

▶ let Xi denote the associated Bernoulli variable for toss i,


Pn
i = 1, . . . , n. Then Sn = i=1 Xi denotes the number of
heads/event and P(Sn = k) = kn p k (1 − p)n−k .

▶ E [Sn ] ? Var(Sn ) ?

8 / 23
Bernoulli/Binomial process

Pn
▶ {Sn = i=1 Xi , n = 1, 2, . . .} is called as a Binomial process.

▶ Let T := {smallest n : Sn > 0.}.

▶ T is a geometric random variable with parameter p, i.e.,


P(T = n1 ) = p(1 − p)(n1 −1) .

▶ Memoryless property: P(T > m + n/T > n) = P(T > m).

9 / 23
Counting process

Stochastic process {N(t), t ≥ 0} is a counting process if it


represents the total number of events upto time t.

It satisfies the following


▶ N(t) ≥ 0 and is integer valued

▶ For s ≤ t, we have N(s) ≤ N(t). N(t) − N(s) denotes the


number of events in the interval (t, s)

▶ N(t) can have independent increments

▶ N(t) can have stationary increments

10 / 23
Poisson process

A Poisson process with rate λ, λ ≥ 0 is a counting process


{N(t), t ≥ 0} with the following properties
▶ N(0) = 0

▶ N(t) has independent and stationary increments

▶ Number of events in an interval of length t is a Poisson


distribution with mean λt. (Hence stationary increments)

▶ E [N(t + s) − N(t)] = λs
Condition 3 is difficult to verify ! Hence ...

11 / 23
Poisson process – Alternative definition

f (h)
A function f is said to be o(h) if limh→0 h = 0.

A Poisson process with rate λ, λ ≥ 0 is a counting process


{N(t), t ≥ 0} with the following properties
▶ N(0) = 0

▶ N(t) has independent and stationary increments

▶ P{N(h) = 1} = λh + o(h)

▶ P{N(h) ≥ 2} = o(h)

12 / 23
Poisson process

Lemma
Definition 1 =⇒ Definition 2
Proof on board.
Lemma
Definition 2 =⇒ Definition 1
Self Study: Refer Sheldon Ross, Stochastic processes, Theorem
2.1.1

13 / 23
Poisson Processes Definition 3

A ctsp {N(t), t ≥ 0} is a Poisson process with rate λ > 0 if


▶ N(0) = 0

▶ N(t) is a counting process with stationary and independent


increments

▶ Xi , the time interval between i − 1th and ith event is


exponentially distributed with parameter λ.

14 / 23

You might also like