Probability Distributions: Random variables-discrete and

continuous, Cumulative Distribution Function, Introduction to

Bernoulli, Binomial, Geometric, Poisson, Triangular, Weibull,
Uniform, Normal, Gamma and Exponential distributions
 Random Variables

Random variable – a numerical description of the

outcome of an experiment. Random variables are
denoted by capital letters, X, Y, …; specific values by
lower case letters, x, y, …
– Discrete random variable – the number of possible
outcomes can be counted
– Continuous random variable – outcomes over one or
more continuous intervals of real numbers
 Discrete Random Variables
For example,
• the outcomes of rolling dice,
• the type of weather for the next day
• customer reactions to a product are discrete random
variables.
The number of outcomes may be finite or countably
infinite, such as the number of hits on a Web site link
during some period of time—we cannot place a
guaranteed upper limit on this number— nevertheless,
the outcomes can be counted.
 Discrete Random Variables
• X is a discrete random variable if the number of possible values
of X is finite, or countably infinite.
• Example: Consider jobs arriving at a job shop.
• Let X be the number of jobs arriving each week at a job shop.
• Rx = possible values of X (range space of X) = {0,1,2,…}
• p(xi) = probability the random variable is xi = P(X = xi)
p(xi), i = 1,2, … must satisfy:
1. p ( xi ) 0, for all i

2.  i 1
p ( xi ) 1

The collection of pairs [xi, p(xi)], i = 1,2,…, is called the probability distribution
of X, and p(xi) is called the probability mass function (pmf) of X.

4
 Continuous Random Variables [Probability Review]

• X is a continuous random variable if its range space Rx is an interval or a

collection of intervals. (Eg. the daily temperature, the time to complete a
task, the time between failures of a machine etc.
• The probability that X lies in the interval [a,b] is given by:
b
P(a  X b)  f ( x)dx
a

• f(x), denoted as the pdf of X, satisfies:

1. f ( x) 0 , for all x in R X
2. f ( x)dx 1
RX

3. f ( x) 0, if x is not in RX

• Properties
x0
1. P( X  x0 ) 0, because x0
f ( x)dx 0
2. P(a  X b) P (a  X b) P (a  X  b) P (a  X  b)
5
 Continuous Random Variables [Probability Review]

• Example: Life of an inspection device is given by X, a

continuous random variable with pdf:

1  x/2
 e , x 0
f ( x)  2
0, otherwise

– X has an exponential distribution with mean 2 years

– Probability that the device’s life is between 2 and 3 years is:
1 3  x/2
P (2  x 3)  e dx 0.14
2 2
6
 Cumulative Distribution Function [Probability Review]

• Cumulative Distribution Function (cdf) is denoted by F(x), where F(x) = P(X

<= x)
– If X is discrete, then F ( x)   p ( xi )
all
xi x
– If X is continuous, then x
F ( x)  f (t )dt

• Properties
1. F is nondecreasing function. If a b, then F (a ) F (b)
2. lim x  F ( x) 1
3. lim x   F ( x) 0

• All probability questions about X can be answered in terms of the cdf, e.g.:

P (a  X b) F (b)  F (a ), for all a  b

7
 Cumulative Distribution Function [Probability Review]

• Example: An inspection device has cdf:

1 x  t/2
F ( x)  e dt 1  e  x / 2
2 0

– The probability that the device lasts for less than 2 years:
P (0  X 2) F (2)  F (0) F (2) 1  e  1 0.632

– The probability that it lasts between 2 and 3 years:

P (2  X 3) F (3)  F (2) (1  e  ( 3 / 2 ) )  (1  e  1 ) 0.145

8
 Expectation [Probability Review]

• The expected value of X is denoted by E(X)

– If X is discrete E ( x)  xi p ( xi )
all i

– If X is continuous 
E ( x)  xf ( x)dx


– a.k.a the mean, m, or the 1st moment of X

– A measure of the central tendency
• The variance of X is denoted by V(X) or var(X) or s2
– Definition: V(X) = E[(X – E[X])2]
– Also, V(X) = E(X2) – [E(x)]2
– A measure of the spread or variation of the possible values of X around the
mean
• The standard deviation of X is denoted by s
– Definition: square root of V(X)
– Expressed in the same units as the mean

9
 Expectations [Probability Review]

• Example: The mean of life of the previous inspection device is:


1   x/2  x/2 
E ( X )   xe dx  xe   e  x / 2 dx 2
2 0 0
0
• To compute variance of X, we first compute E(X2):


1  2  x/2  x/2 
E ( X )   x e dx  x e
2 2  2  xe  x / 2 dx 8
2 0 0
• Hence, the variance and standard deviation0 of the device’s life are:

V ( X ) 8  2 2 4
  V ( X ) 2
10
 Useful Statistical Models
• In this section, statistical models appropriate
to some application areas are presented. The
areas include:
– Queueing systems
– Inventory and supply-chain systems
– Reliability and maintainability
– Limited data

11
 Queueing Systems [Useful
Models]
• In a queueing system, interarrival and service-time patterns
can be probablistic
– For more queueing examples, see Chapter 2; Chapter 6 is all queueing systems

• Sample statistical models for interarrival or service time

distribution:
– Exponential distribution: if service times are completely random
– Normal distribution: fairly constant but with some random variability
(either positive or negative)
– Truncated normal distribution: similar to normal distribution but with
restricted value.
– Gamma and Weibull distribution: more general than exponential
(involving location of the modes of pdf’s and the shapes of tails.)

12
 Inventory and supply chain [Useful
Models]
• In realistic inventory and supply-chain systems, there are at
least three random variables:
– The number of units demanded per order or per time period
– The time between demands
– The lead time

• Sample statistical models for lead time distribution:

– Gamma

• Sample statistical models for demand distribution:

– Poisson: simple and extensively tabulated.
– Negative binomial distribution: longer tail than Poisson (more large
demands).
– Geometric: special case of negative binomial given at least one
demand has occurred.

13
 Reliability and maintainability
[Useful Models]

• Time to failure (TTF)

– Exponential: failures are random
– Gamma: for standby redundancy where each
component has an exponential TTF
– Weibull: failure is due to the most serious of a
large number of defects in a system of
components
– Normal: failures are due to wear

14
 Other areas [Useful Models]

• For cases with limited data, some useful

distributions are:
– Uniform, triangular and beta
• Other distribution: Bernoulli, binomial and
hyperexponential.

15
 Discrete Distributions
• Discrete random variables are used to
describe random phenomena in which only
integer values can occur.
• In this section, we will learn about:
– Bernoulli trials and Bernoulli distribution
– Binomial distribution
– Geometric and negative binomial distribution
– Poisson distribution

16
 Bernoulli Trials
and Bernoulli Distribution [Discrete
• Bernoulli Trials: Dist’n]
– Consider an experiment consisting of n trials, each can be a success or
a failure.
• Let Xj = 1 if the jth experiment is a success
• and Xj = 0 if the jth experiment is a failure
– The Bernoulli distribution (one trial):
 p, x j 1, j 1,2,..., n

p j ( x j )  p ( x j ) 1  p q, x j 0 ,j 1,2 ,...,n
0, otherwise

– where E(Xj) = p and V(Xj) = p (1-p) = p q
• Bernoulli process:
– The n Bernoulli trials where trails are independent:
p(x1,x2,…, xn) = p1(x1) p2(x2) … pn(xn)

17
 Binomial Distribution [Discrete Dist’n]

• The number of successes in n Bernoulli trials, X, has a binomial

distribution.
 n  x n  x
  p q , x 0,1,2,..., n
p ( x)  x 
0, otherwise


The number of Probability that

outcomes having the there are
required number of x successes and (n-
successes and failures x) failures

– The mean, E(x) = p + p + … + p = n*p

– The variance, V(X) = pq + pq + … + pq = n*pq

18
 Geometric & Negative
Binomial Distribution [Discrete
Dist’n]
• Geometric distribution
– The number of Bernoulli trials, X, to achieve the 1st success:
 q x  1 p, x 0,1,2,..., n
p ( x) 
0, otherwise
– E(x) = 1/p, and V(X) = q/p2

• Negative binomial distribution

– The number of Bernoulli trials, X, until the kth success
– If Y is a negative binomial distribution with parameters p and k, then:
 y  1 y  k k
  q p , y k , k  1, k  2,...
p ( x)  k  1
– E(Y) = k/p, and V(X) =kq/p
0, 2 otherwise

19
 Poisson Distribution [Discrete
Dist’n]
• Poisson distribution describes many random processes quite
well and is mathematically quite simple.
– where a > 0, pdf and cdf are:
 e  x x
e  i

p ( x)  x! , x 0,1,... F ( x) 
i 0 i!
0, otherwise
– E(X) = a = V(X)

20
 Poisson Distribution [Discrete
Dist’n]

• Example: A computer repair person is “beeped” each time

there is a call for service. The number of beeps per hour ~
Poisson(a = 2 per hour).

– The probability of three beeps in the next hour:

p(3) = e-223/3! = 0.18
also, p(3) = F(3) – F(2) = 0.857-0.677=0.18

– The probability of two or more beeps in a 1-hour period:

p(2 or more) = 1 – p(0) – p(1)
= 1 – F(1)
= 0.594

21
 Continuous Distributions
• Continuous random variables can be used to
describe random phenomena in which the
variable can take on any value in some
interval.
• In this section, the distributions studied are:
– Uniform
– Exponential
– Normal
– Weibull
– Lognormal 22
 Uniform Distribution
[Continuous Dist’n]
• A random variable X is uniformly distributed on the interval
(a,b), U(a,b), if its pdf and cdf are:
0, x a
 1 x  a
 , a  x b
f ( x)  b  a F ( x)  , a  x b
0, otherwise b  a
1, x b

• Properties
– P(x1 < X < x2) is proportional to the length of the interval [F(x2) – F(x1) =
(x2-x1)/(b-a)]
– E(X) = (a+b)/2 V(X) = (b-a)2/12
• U(0,1) provides the means to generate random numbers,
from which random variates can be generated.

23
 Exponential Distribution [Continuous
Dist’n]
• A random variable X is exponentially distributed with
parameter l > 0 if its pdf and cdf are:
e  x , x 0 0, x 0
f ( x)  F ( x)  x  t
0, elsewhere 
0
e dt 1  e  x , x 0

 E(X) = 1/l V(X) = 1/l2

 Used to model interarrival times
when arrivals are completely random,
and to model service times that are
highly variable
 For several different exponential pdf’s
(see figure), the value of intercept on
the vertical axis is l, and all pdf’s
eventually intersect.
24
Exponential Distribution [Continuous
Dist’n]

• Memoryless property
– For all s and t greater or equal to 0:
P(X > s+t | X > s) = P(X > t)

– Example: A lamp ~ exp(l = 1/3 per hour), hence,

on average, 1 failure per 3 hours.
• The probability that the lamp lasts longer than its mean
life is: P(X > 3) = 1-(1-e-3/3) = e-1 = 0.368
• The probability that the lamp lasts between 2 to 3
hours is:
P(2 <= X <= 3) = F(3) – F(2) = 0.145
• The probability that it lasts for another hour given it is
operating for 2.5 hours:
P(X > 3.5 | X > 2.5) = P(X > 1) = e-1/3 = 0.717 25
 Normal Distribution [Continuous
Dist’n]
• A random variable X is normally distributed has the pdf:
1  1  x   2 
f ( x)  exp      ,   x 
 2  2    

– Mean:     
– Variance:  2 0
– Denoted as X ~ N(m,s2)
• Special properties:
– lim x   f ( x) 0, and lim x   f (. x) 0
– f(m-x)=f(m+x); the pdf is symmetric about m.
– The maximum value of the pdf occurs at x = m; the mean and mode are
equal.

26
 Normal Distribution [Continuous
Dist’n]
• Evaluating the distribution:
– Use numerical methods (no closed form)
– Independent of m and s, using the standard normal distribution:
Z ~ N(0,1)
– Transformation of variables: let Z = (X - m) / s,
 x  
F ( x) PX  x  P Z  
  
( x  ) / 1  z2 / 2
 e dz

2
( x  ) /
  ( z )dz  ( x  ) z
, where  ( z ) 
1  t2 / 2
e dt
 
2

27
 Normal Distribution [Continuous
Dist’n]
• Example: The time required to load an oceangoing vessel, X, is
distributed as N(12,4)
– The probability that the vessel is loaded in less than 10 hours:
 10  12 
F (10)    ( 1) 0.1587
 2 
• Using the symmetry property, F(1) is the complement of F (-1)

28
 Weibull Distribution [Continuous
Dist’n]
• A random variable X has a Weibull distribution if its pdf has the form:
  x   1  
 x  

f ( x)    
exp      , x 
    
0, otherwise

• 3 parameters:
– Location parameter: u, (  )
– Scale parameter: b , (b > 0)
– Shape parameter. a, (> 0)
• Example: u = 0 and a = 1:

When b = 1,
X ~ exp(l = 1/a)
29
 Lognormal Distribution [Continuous
Dist’n]
• A random variable X has a lognormal distribution if its pdf has
the form:
 1  ln x  μ 2  m=1, s2=0.5,1,2.
 exp   , x 0
f ( x)  2π σx  2σ 2

0, otherwise


– Mean E(X) = em+s2/2

– Variance V(X) = e2m+s2/2 (es2 - 1)

• Relationship with normal distribution

– When Y ~ N(m, s2), then X = eY ~ lognormal(m, s2)
– Parameters m and s2 are not the mean and variance of the lognormal

30
 Poisson Process
• Definition: N(t) is a counting function that represents the
number of events occurred in [0,t].
• A counting process {N(t), t>=0} is a Poisson process with
mean rate l if:
– Arrivals occur one at a time
– {N(t), t>=0} has stationary increments
– {N(t), t>=0} has independent increments
• Properties
e  t ( t ) n
P[ N (t ) n]  , for t 0 and n 0,1,2,...
n!
– Equal mean and variance: E[N(t)] = V[N(t)] = lt
– Stationary increment: The number of arrivals in time s to t is also
Poisson-distributed with mean l(t-s)

31
 Interarrival Times [Poisson
Process]
• Consider the interarrival times of a Possion process (A1, A2, …), where Ai is
the elapsed time between arrival i and arrival i+1

– The 1st arrival occurs after time t iff there are no arrivals in the interval [0,t],
hence:
P{A1 > t} = P{N(t) = 0} = e-lt
P{A1 <= t} = 1 – e-lt [cdf of exp(l)]
– Interarrival times, A1, A2, …, are exponentially distributed and independent with
mean 1/l

Arrival counts ~ Poi(l) Interarrival time ~ Exp(1/l)

Stationary & Independent Memoryless

32
 Splitting and Pooling [Poisson
Dist’n]

• Splitting:
– Suppose each event of a Poisson process can be classified as Type I,
with probability p and Type II, with probability 1-p.
– N(t) = N1(t) + N2(t), where N1(t) and N2(t) are both Poisson processes
with rates l p and l (1-p)
lp N1(t) ~ Poi[lp]

l
N(t) ~ Poi(l)

N2(t) ~ Poi[l(1-p)]
l(1-p)

• Pooling:
– Suppose two Poisson processes are pooled together
– N1(t) + N2(t) = N(t), where N(t) is a Poisson processes with rates l1 +
l
l2 N1(t) ~ Poi[l ]
1 1 l + l
1 2
N(t) ~ Poi(l + l )
1 2

N2(t) ~ Poi[l ] l
2 2
33
 Nonstationary Poisson
Process (NSPP) [Poisson Dist’n]
• Poisson Process without the stationary increments, characterized by l(t), the
arrival rate at time t.
• The expected number of arrivals by time t, L(t):
t

Λ(t)  λ(s)ds
0

• Relating stationary Poisson process n(t) with rate l=1 and NSPP N(t) with
rate l(t):
– Let arrival times of a stationary process with rate l = 1 be t1, t2, …, and
arrival times of a NSPP with rate l(t) be T1, T2, …, we know:
ti = L(Ti)
Ti = L-1(ti)

34
 Nonstationary Poisson
Process (NSPP) [Poisson Dist’n]
• Example: Suppose arrivals to a Post Office have rates 2 per minute from 8
am until 12 pm, and then 0.5 per minute until 4 pm.
• Let t = 0 correspond to 8 am, NSPP N(t) has rate function:
2, 0 t  4
 (t ) 
0.5, 4 t 8
Expected number of arrivals by time t:
2t , 0 t  4
(t )  4 t t

0
2 ds  4 0 .5 ds 
2
 6, 4 t 8

• Hence, the probability distribution of the number of arrivals between 11 am

and 2 pm.
P[N(6) – N(3) = k] = P[N(L(6)) – N(L(3)) = k]
= P[N(9) – N(6) = k]
= e(9-6)(9-6)k/k! = e3(3)k/k!

35
 Empirical Distributions [Poisson
Dist’n]
• A distribution whose parameters are the observed values in a
sample of data.
– May be used when it is impossible or unnecessary to establish that a
random variable has any particular parametric distribution.
– Advantage: no assumption beyond the observed values in the sample.
– Disadvantage: sample might not cover the entire range of possible
values.

36