Introduction to Probability

and Statistics
Slides 3 – Chapter 3

Ammar M. Sarhan,
[email protected]
Department of Mathematics and Statistics,
Dalhousie University
Fall Semester 2008
 Chapter 3

Discrete Random
Variables and
Probability Distributions

Dr. Ammar M. Sarhan 2

 Chapter Outlines
3.1 Random Variable
3.2 Probability Distribution for Discrete
Random Variables
3.3 Expected Value of Discrete Random
variables
3.4 Binomial Distribution
3.5 Hypergeometric & Negative Distributions
3.6 Poisson Distribution

Dr. Ammar M. Sarhan 3

 3.1 Random Variables
In a statistical experiment, it is often very important to allocate
numerical values to the outcomes.
Example:
• Experiment: testing two components
(D=defective, N=non-defective)
• Sample space: S={DD,DN,ND,NN}
• Let X = number of defective components.
• Assigned numerical values to the outcomes are:

Notice that, the set of all possible values of the variable X is {0, 1, 2}.
Dr. Ammar M. Sarhan 4
 Random Variable

For a given sample space S of some experiment, a random variable is

any rule (function) that associates a real number with each outcome in S.

Notation: " X " denotes the random variable .

" x " denotes a particular value of the random variable X.

Bernoulli Random Variable

Any random variable whose only possible values are 0 and 1 is called
a Bernoulli random variable.

Dr. Ammar M. Sarhan 5

 Types of Random Variables:

A random variable X is called a discrete random variable if its set of

possible values is countable, i.e., x ∈ {x1, x2,…}

A random variable X is called a continuous random variable if it can

take values on a continuous scale, i.e., x ∈ {x: a < x < b; a, b ∈ R}

In most practical problems:

A discrete random variable represents count data, such as the
number of defectives in a sample of k items.

A continuous random variable represents measured data, such as

height.

Dr. Ammar M. Sarhan 6

 3.2 Probability Distributions for Discrete Random Variables

Probability Distribution

The probability distribution or probability mass function (pmf) of a

discrete rv X is defined for every number x by

p ( x ) = P (all s ∈ S : X ( s ) = x ) = P ( X = x )
The probability mass function, p(x), of a discrete random variable X
satisfies the following:
1) p ( x ) = P ( X = x ) ≥ 0.

2) ∑ p( x) = 1.
all x

Note:
P( A) = ∑ p( x) = ∑ P( X = x).
all x∈ A all x∈ A

Dr. Ammar M. Sarhan 7

 Parameter of a Probability Distribution

¾Suppose that p(x) depends on a quantity that can be assigned any

one of a number of possible values, each with different value
determining a different probability distribution.
¾Such a quantity is called a parameter of the distribution.
¾The collection of all distributions for all different parameters is called
a family of distributions.

Example: For 0< a < 1,

⎧1 − a if x = 0
⎪
p ( x ; a ) = ⎨a if x = 1
⎪0
⎩ otherwise

Each choice of a yields a different pmf.

Dr. Ammar M. Sarhan 8

 Cumulative Distribution Function
The cumulative distribution function (cdf) F(x) of a discrete rv
variable X with pmf p(x) is defined by

F ( x) = P( X ≤ x ) = ∑ p( y).
y: y≤ x
For any number x, F(x) is the probability that the observed value of X
will be at most x.

Proposition

For any two numbers a and b with a ≤ b

P(a ≤ X ≤ b) = F(b) – F(a-)
“a–” represents the largest possible X value that is strictly less than a.

Note: For integers

P(a ≤ X ≤ b) = F(b) - F(a -1)
Dr. Ammar M. Sarhan 9
 Probability Distribution for the Random Variable X

A probability distribution for a random variable X:

x -3 -2 -1 0 1 4 6
P(X=x) 0.13 0.16 0.17 0.20 0.16 0.11 0.07

Find:
1)F(x)
2)P(X > 0) = 1- P(X ≤ 0) =1 - F(0) = 1 - 0.66 = 0.34
3)P(-2 ≤ X ≤ 1) = P(-2 ≤ X ≤ 1) = F(1) – F(-2-) = F(1) – F(-3) = 0.69
4)P(-2 ≤ X < 1) = P(-2 ≤ X < 1) = F(1-) – F(-2-) = F(0) – F(-3) = 0.53
5)P(-2 < X ≤ 1) = P(-2 < X ≤ 1) = F(1) – F(-2) = F(1) – F(-2) = 0.53
6)P(-3 < X < 1) = P(-3 < X < 1) = F(1-) – F(-2) = F(0) – F(-2) = 0. 37
1) F(x)
x -3 -2 -1 0 1 4 6
F(x) 0.13 0.29 0.46 0.66 0.82 0.93 1.00

Dr. Ammar M. Sarhan 10

 x -3 -2 -1 0 1 4 6
F(x) 0.13 0.29 0.46 0.66 0.82 0.93 1.00

1.0
F(x)

0.5

0.1

-3 -2 -1 0 1 2 3 4 5 6
x -3 -2 -1 0 1 4 6
P(X=x) 0.13 0.16 0.17 0.20 0.16 0.11 0.07

Dr. Ammar M. Sarhan 11

 x -3 -2 -1 0 1 4 6
pmf:
P(X=x) 0.13 0.16 0.17 0.20 0.16 0.11 0.07

CDF: x -3 -2 -1 0 1 4 6
F(x) 0.13 0.29 0.46 0.66 0.82 0.93 1.00

The CDF can be written as ⎧0 if x < −3

⎪0.13 if − 3 ≤ x < −2
⎪
⎪0.29 if − 2 ≤ x < −1
⎪
⎪0.46 if − 1 ≤ x < 0
F ( x) = ⎨
⎪0.66 if 0 ≤ x < 1
⎪0.82 if 1 ≤ x < 4
⎪
⎪0.93 if 4 ≤ x < 6
⎪1.00 if 6 ≤ x
⎩
Dr. Ammar M. Sarhan 12
 Example.
Tossing a non-balance coin 2 times independently.
Sample space: S={HH, HT, TH, TT}.
Suppose P(H) = ½ P(T) P(H) =1/3 and P(T)=2/3.
Let X= number of heads
Sample point Value of X (x) Probability
HH 2 P(HH) = P(H) P(H) = 1/3×1/3=1/9
HT 1 P(HT) =P(H) P(T) =1/3×2/3 = 2/9
TH 1 P(HT) = P(H) P(T) =1/3×2/3 = 2/9
TT 0 P(TT) = P(T) P(T) = 2/3×2/3 = 4/9

The possible values of X are: 0, 1, and 2.

X is a discrete random variable.
(X=x) p(x) = P(X=x) Probability distribution of the X
(X=0)={TT} 4/9 X 0 1 2 Total
(X=1)={HT, TH} 2/9 + 2/9 =4/9 P(X=x) 4/9 4/9 1/9 1.0
(X=2)={HH} 1/9
Dr. Ammar M. Sarhan 13
P(X<1) = P(X=0)=4/9
= F(1-) = F(0) =4/9
P(X≤1) = P(X=0) + P(X=1) = 4/9+4/9 = 8/9
= F(1) = 8/9
P(X≥0.5) = P(X=1) + P(X=2) = 4/9+1/9 = 5/9
= 1-F(0.5-) = 1-F(0) = 1-4/9 = 5/9
P(X>8) = P(φ) = 0
=1-F(8) = 1-1 = 0
P(X<10) = P(X=0) + P(X=1) + P(X=2) = P(S) = 1
= F(10) = 1
⎧0 if x < 0
⎪4 / 9 if 0 ≤ x < 1
F(x)
⎪
F ( x) = ⎨ X 0 1 2
⎪8 / 9 if 1 ≤ x < 2 F(x) 4/9 8/9 9/9
⎪⎩1 if 2 ≤ x

Dr. Ammar M. Sarhan 14

 Example :

A shipment of 8 similar microcomputers to a retail outlet contains 3

that are defective and 5 are non-defective. If a school makes a
random purchase of 2 of these computers, find the probability
distribution of the number of defectives .

We need to find the probability distribution of the random variable:

X = the number of defective computers purchased.
Experiment: selecting 2 computers at random out of 8

N(S) = equally likely outcomes

The possible values of X are: x = 0, 1, 2.

Dr. Ammar M. Sarhan 15

N(X=0)={0D and 2N} =

N(X=1)= {1D and 1N} =

N(X=2)={2D and 1N} =

P(X=0)= P(X=1)=

P(X=2)=

Dr. Ammar M. Sarhan 16

 The probability distribution of X

p(x) = P(X=x)

, x =0,1,2.

p(x) =

0, otherwise.

Hypergeometric Distribution

Dr. Ammar M. Sarhan 17

 3.3 Expected Values of Discrete Random Variables
The Expected Value of X
Let X be a discrete rv with set of possible values D and pmf p(x). The
expected value or mean value of X, denoted E(X) or μX or μ, is
E ( X ) = μ X = ∑ x ⋅ p( x).
x∈D

Example: Use the data below to find out the expected number of the
number of credit cards that a student will possess.
X = # credit cards x 0 1 2 3 4 5 6
p(x) 0.08 0.28 0.38 0.16 0.06 0.03 0.01

E(X) = x1 p1 + x2 p2 + ... + x1 p1
=0(0.08) +1(0.28)+2(0.38)+3(0.16)+4(0.06)+5( 0.03) + 6(0.01)
=1.97≈ 2 credit cards

Dr. Ammar M. Sarhan 18

 The Expected Value of a Function
If the rv X has the set of possible values D and pmf p(x), then the
expected value of any function h(x), denoted E[h(X)] or (μh(X)) is
E[h( X )] = μ h ( X ) = ∑ h( x) ⋅ p ( x).
x∈D

Rules of the Expected Value

E[aX + b] = a · E(X) + b

1) For any constant a, E[aX] = a · E(X)

2) For any constant b, E[X + b] = E(X) + b

Dr. Ammar M. Sarhan 19

 The Variance and Standard Deviation

Let X have pmf p(x), and expected value μ. Then the variance of X,
denoted V(X) (or σ X2 or σ2), is

V ( X ) = E[( X − μ ) 2 ] = ∑ ( x − μ ) 2 ⋅ p ( x).
x∈D

The standard deviation (SD) of X is

σ X = σ 2
X

20
Dr. Ammar M. Sarhan 20
Example: The quiz scores for a particular student are given below:
22, 25, 20, 18, 12, 20, 24, 20, 20, 25, 24, 25, 18
Compute the variance and standard deviation.
Solution: Let X be the quiz score.
Value (X) 12 18 20 22 24 25
Frequency 1 2 4 1 2 3
p(x) Probability 0.08 0.15 0.31 0.08 0.15 0.23

μ = ∑ x ⋅ p( x) = 21
x∈D

V ( X ) = ( x1 − μ ) 2 p1 + ( x1 − μ ) 2 p1 + ... + ( xn − μ ) 2 pn
= (12 − 21) 2 0.08 + (18 − 21) 2 0.15 + (20 − 21) 2 0.31 +
(22 − 21) 2 0.08 + (24 − 21) 2 0.15 + (25 − 21) 2 0.23 = 13.25

σ = V ( X ) = 13.25 ≈ 3.64
21
Dr. Ammar M. Sarhan 21
Shortcut Formula for Variance

Rules of Variance

22
Dr. Ammar M. Sarhan 22
 3.4 The Binomial Probability Distribution
Binomial Experiment
An experiment for which the following four conditions are satisfied is
called a binomial experiment.
1. The experiment consists of a sequence of n trials, where n is fixed
in advance of the experiment.
2. The trials are identical, and each trial can result in one of the same
two possible outcomes, which are denoted by success (S) or failure
(F).
3. The trials are independent
4. The probability of success is constant from trial to trial: denoted by p.

23
Dr. Ammar M. Sarhan 23
 Binomial Random Variable
Given a binomial experiment consisting of n trials, the binomial
random variable X associated with this experiment is defined as
X = the number of S’s among n trials.
Notation for the pmf of a Binomial rv
Because the pmf of a binomial rv X depends on the two parameters n
and p, we denote the pmf by b(x;n,p).
Computation of a Binomial pmf
⎧⎛ n ⎞ x
⎪⎜⎜ ⎟⎟ p (1 − p ) n − x x = 0,1, L, n
b( x; n, p) = ⎨⎝ p ⎠
⎪0
⎩ otherwise
The expected value and Variance of a Binomial distribution
If X ~ b(x;n,p) (1) E(X) = n p (2) V(X) = n p (1- p)
Dr. Ammar Sarhan 24
Dr. Ammar M. Sarhan 24
Example: If the probability of a student successfully passing this course
(C or better) is 0.82, find the probability that given 8 students
a. all 8 pass.
⎛8⎞
P( X = 8) = b(8;8,0.82) = ⎜⎜ ⎟⎟ (0.82)8 (1 − 0.82)8−8 ≈ 0.2044
⎝ 8⎠
b. none pass.
⎛8 ⎞
P( X = 0) = b(0;8,0.82) = ⎜⎜ ⎟⎟ (0.82) 0 (1 − 0.82)8−0 ≈ 0.0000011
⎝0⎠
c. at least 6 pass.
P ( X ≥ 6) = b(6;8,0.82) + b(7;8,0.82) + b(8;8,0.82)
⎛8 ⎞ ⎛8 ⎞
= ⎜⎜ ⎟⎟ (0.82) (1 − 0.82) + ⎜⎜ ⎟⎟ (0.82) 7 (1 − 0.82)8−7
6 8− 6

⎝ 6⎠ ⎝7⎠
⎛8⎞
⎜⎜ ⎟⎟ (0.82)8 (1 − 0.82)8−8 ≈ 0.8392
⎝8⎠ 25
Dr. Ammar M. Sarhan 25
d. The expected number of students passed the course.
E(X) = n p = 8 (0.82) = 6.56 ≈ 7 students
e. The variance.

V(X) = n p (1- p)
= 8 (0.82) (1- 0.82) = 8 (0.82) (0.18)
= 1.1808

26
Dr. Ammar M. Sarhan 26
 3.5 Hypergeometric and Negative Binomial Distributions
The Hypergeometric Distribution
The three assumptions that lead to a hypergeometric distribution:
1. The population or set to be sampled consists of N individuals,
objects, or elements (a finite population).
2. Each individual can be characterized as a success (S) or failure (F),
and there are M successes in the population.
3. A sample of n individuals is selected without replacement in such a
way that each subset of size n is equally likely to be chosen.
If X is the number of S’s in a completely random sample of size n
drawn from a population consisting of M S’s and (N – M) F’s, then
the probability distribution of X will be called the hypergeometric
distribution. [ h (n,M,N) ]
27
Dr. Ammar M. Sarhan 27
 Computation of a Hypergeometric distribution
N
⎛ M ⎞⎛ N − M ⎞
⎜⎜ ⎟⎟ ⎜⎜ ⎟⎟
⎝ x ⎠⎝ n − x ⎠ M N-M
P( X = x) = h( x; n, M , N ) = , ⎛M ⎞ ⎛N −M ⎞
⎛N⎞ ⎜⎜ ⎟⎟ ⎜⎜
−
⎟⎟
⎜⎜ ⎟⎟ ⎝ x⎠
x n-x ⎝ n x ⎠
⎝n⎠
n
max(0, n − N + M ) ≤ x ≤ min(n, M ) ⎛N⎞
⎜⎜ ⎟⎟
⎝n⎠

The expected value and Variance of a Hypergeometric distribution

If X ~ h (n,M,N)

(1) E ( X ) = n M
⎛ N −n⎞ M ⎛ M⎞
(2) V ( X ) = ⎜ ⎟⋅n⋅ ⋅ ⎜1 − ⎟
N ⎝ N −1 ⎠ N ⎝ N⎠
Dr. Ammar Sarhan 28
Dr. Ammar M. Sarhan 28
Example:
Lots of 40 components each are called acceptable if they contain no
more than 3 defectives. The procedure for sampling the lot is to select 5
components at random (without replacement) and to reject the lot if a
defective is found. What is the probability that exactly one defective is
found in the sample if there are 3 defectives in the entire lot.
Solution: 40
Let X= number of defectives in the sample.
3 37
X ~ h(n, M, N) ≡ h(5, 3, 40) ⎛ 3⎞
⎜⎜ ⎟⎟
⎛ 40 − 3 ⎞
⎜⎜ ⎟⎟
⎝ x⎠ − ⎠
5-x ⎝
5 x
N = 40, M = 3, n = 5. x
⎛ 3 ⎞ ⎛ 37 ⎞
⎜⎜ ⎟⎟ ⎜⎜ ⎟⎟ 5
⎝ x⎠⎝5 − x⎠ ⎛ 40 ⎞
P( X = x) = , x = 0, 1, 2, 3 ⎜⎜ ⎟⎟
⎛ 40 ⎞ ⎝5⎠
⎜⎜ ⎟⎟
⎝5⎠
max(0, 5 − 37) = 0 ≤ x ≤ min(5, 3) = 3
29
Dr. Ammar M. Sarhan 29
The probability that exactly one defective is found in the sample is

P(X= 1) =
Example:
In the previous example, find the expected value and the variance of the
number of defectives in the sample.
Solution:
M 3 3
(1) E ( X ) = n = 5 = = 0.375
N 40 8
⎛ N −n⎞ M ⎛ M⎞
(2) V ( X ) = ⎜ ⎟ ⋅ n ⋅ ⋅ ⎜1 − ⎟
⎝ N −1 ⎠ N ⎝ N⎠
⎛ 40 − 5 ⎞ 3 ⎛ 3 ⎞
=⎜ ⎟ ⋅ 5 ⋅ ⋅ ⎜1 − ⎟ = 0.311298
⎝ 40 − 1 ⎠ 40 ⎝ 40 ⎠
Dr. Ammar Sarhan 30
Dr. Ammar M. Sarhan 30
 The Negative Binomial Distribution
The negative binomial rv and distribution are based on an experiment
satisfying the following four conditions:
1. The experiment consists of a sequence of independent trials.
2. Each trial can result in a success (S) or a failure (F).
3. The probability of success is constant from trial to trial, so
P(S on trial i) = p for i = 1, 2, 3, …
4. The experiment continues until a total of r successes have been
observed, where r is a specified positive integer.
If X is the number of failures that precede the r-th success, then the
probability distribution of X will be called the negative binomial
distribution. [ nb (r, p) ]

The event (X = x) is equivalent to {r-1 S’s in the first (x+r-1) trials and
an S in the (x+r)th}.
31
Dr. Ammar M. Sarhan 31
 Computation of a Negative Binomial pmf

⎛ x + r − 1⎞ r
nb( x; r , p ) = ⎜⎜ ⎟⎟ p (1 − p) x , x = 0,1,2, L
⎝ r −1 ⎠

The expected value and Variance of a Negative Binomial

distribution
If X ~ nb(r, p) (1) E(X) = r (1-p)/p (2) V(X) = r (1-p)/p2

Geometric Distribution:
When r = 1 in nb(r, p) then the nb distribution reduces to geometric
distribution.

32
Dr. Ammar M. Sarhan 32
Example:
Suppose that p=P(male birth) = 0.5. A parent wishes to have exactly
two female children in their family. They will have children until this
condition is fulfilled.
1. What is the probability that the family has x male children?
Let X be the number of Ms precede the 2nd F. X ~ nb(2, 0.5)
The prob. that the family has x male children = P(X=x)
= nb(x; 2,0.5), x=0,1,2,...
2. What is the probability that the family has 4 children?
= P(X = 2) = nb(2; 2, 0.5) = 0.1875

3. What is the probability that the family has at most 4 children?

= P(X ≤ 2 ) = P(X = 0) + P(X = 1) + P(X = 2)
= nb(0; 2, 0.5) + nb(1; 2, 0.5) + nb(2; 2, 0.5)
= 0.6875
Dr. Ammar Sarhan 33
Dr. Ammar M. Sarhan 33
4. How many male children would you expect this family to have?
How many children would you expect this family to have?
E(X) = r (1 – p)/ p = 2 (1-0.5) / 0.5 = 2

2 + E(X) = 2 + 2 = 4

34
Dr. Ammar M. Sarhan 34
 3.6 The Poisson Probability Distribution
A random variable X is said to have a Poisson distribution with
parameter λ (λ > 0), if the pmf of X is

λx e − λ
p ( x; λ ) = , x = 0,1,2, L
x!
The Poisson Distribution as a Limit

Suppose that in the binomial pmf b(x;n, p), we let n → ∞ and p → 0

in such a way that np approaches a value λ (λ > 0). Then
b(x; n, p) → p(x; λ)

The expected value and Variance of a Poisson distribution

If X has a Poisson distribution with parameter λ , then
E(X) = V(X) = λ
35
Dr. Ammar M. Sarhan 35
 Poisson Process
Assumptions:
1. There exists a parameter α > 0 such that for any short time interval of
length Δt , the probability that exactly one event is received is
α Δt + o(Δt )
2. The probability of more than one event during Δt is o(Δt ).
3. The number of events during the time interval Δt is independent of
the number that occurred prior to this time interval.
Poisson Process
Pk (t) = e- αt (αt)k / k! , so that the number of pulses (events) during a
time interval of length t is a Poisson rv with parameter αt. The expected
number of pulses (events) during any such time interval is αt, so the
expected number during a unit time interval is α.

36
Dr. Ammar M. Sarhan 36
Example:
Suppose that the number of typing errors per page has a Poisson
distribution with average 6 typing errors.
(1) What is the probability that in a given page:
(i) The number of typing errors will be 7?
(ii) The number of typing errors will be at least 2?
Let X = number of typing errors per page. X ~ Poisson (6)
6 x e −6
P( X = x) = p ( x; 6) = , x = 0,1,2, L
x!
67 e −6
(i) P ( X = 7) = p (7; 6) = = 0.13768.
7!
(ii) P( X ≥ 2) = 1 − P( X < 2) = 1 − [P ( X = 0) + P ( X = 1)]

=1 - p(0; 6) – p(1; 6) = 0.982650

Dr. Ammar Sarhan 37
Dr. Ammar M. Sarhan 37
(2) What is the probability that in 2 pages there will be 10 typing errors?

Let X = number of typing errors in 2 pages. X ~ Poisson (12)

1210 e −12
P( X = 10) = p (10;12) = = 0.1084.
10!
(3) What is the probability that in a half page there will be no typing
errors?
Let X = number of typing errors in a half page. X ~ Poisson (3)

30 e −3
P( X = 0) = p(0; 3) = = 0.0497871 .
0!

38
Dr. Ammar M. Sarhan 38