Review of Probability

& Statistics

Dr. Hakeem–Ur–Rehman
IQTM-PU

1
 Outlines

▪ Review of Probability
o Fundamentals of Probability
o Definitions of Probability
o Mutually Exclusive & Not Mutually Exclusive Events
o Conditional Probability
o Random Variables
o Expectation, Variance, and Moments

2
 Fundamentals of Probability
▪ Experiment: outcome is unknown beforehand.
▪ Example: Rolling a die and observing the number that is rolled is a
probability experiment.

▪ Result of a single trial in a probability experiment is the outcome.

▪ Sample space ‘Ω’ : set of all possible outcomes ‘ω’ of experiment.

▪ Example: The sample space when rolling a die has six outcomes. {1, 2, 3,
4, 5, 6}

▪ Event: a subset of the sample space Ω

▪ Event A occurs if the outcome ω ∈ A.
Example: A die is rolled. Event A = {2, 4, 6} is rolling an even number.

3
 Definitions of Probability
▪ Classical (or theoretical) probability is used when each outcome in a sample
space is equally likely to occur. The classical probability for event ‘E’ is given by
Number of outcomes in event
P (E ) = .
Total number of outcomes in sample space
Example: A die is rolled. Find the probability of Event A: rolling a 5.
1
There is one outcome in Event A: {5} P(A) = 6  0.167
▪ Empirical (or statistical) probability is based on observations obtained from
probability experiments. The empirical frequency of an event E is the relative
frequency of event E.

P (E ) = Frequency of Event E
Total frequency
Example: A travel agent determines that in every 50 reservations she makes, 12
will be for a cruise. What is the probability that the next reservation she makes will
be for a cruise? P(cruise) = 12 = 0.24
50
▪ Subjective probability results from intuition, educated guesses, and estimates.
Example: A business analyst predicts that the probability of a certain union going
on strike is 0.15. 4
 Fundamentals of Probability…
▪ Mutually Exclusive Events:
o Two events, A and B, are mutually exclusive if they cannot occur
at the same time.
▪ Union: A ∪ B = {ω ∈ Ω : ω ∈ A or ω ∈ B}

▪ Intersection: AB ≡ A ∩ B = {ω ∈ Ω : ω ∈ A and ω ∈ B}

A, B mutually exclusive (or disjoint) if A ∩ B = ∅.

A and B

A
B A B

A and B are mutually A and B are not mutually

exclusive. exclusive. 5
 Fundamentals of Probability…
▪ The complement of Event ‘A’ is the set of all outcomes in the
sample space that are not included in event ‘A’. (Denoted A′ and read
“A prime.”)
o P(A) + P (A′ ) = 1
o P(A) = 1 – P (A′ )
o P (A′ ) = 1 – P(A)

▪ Probability ‘P’ assigns to each event (set) ‘A ⊆ Ω’ a number in [0, 1]

such that
1. 0 ≤ P(A) ≤ 1 for any event A ⊆ Ω.
2. P(Ω) = 1.
3. If A1,A2, . . . , An is (possibly infinite) sequence of mutually
exclusive events,
𝑛
𝑛
𝑃 𝑈𝑖=1 𝐴𝑖 = ෍ 𝑃(𝐴𝑖 )
𝑖=1
▪ Some properties of probability
for all A,B ⊆ Ω, P(A ∪ B) = P(A)+P(B)−P(A ∩ B) 6
 Conditional Probability
▪ A conditional probability is the probability of an event occurring,
given that another event has already occurred.
𝑷(𝑨 𝒂𝒏𝒅 𝑩) 𝑷(𝑨 ∩ 𝑩)
“Probability of B, given A” 𝑷 𝑩ൗ𝑨 = =
𝑷(𝑨) 𝑷(𝑨)

7
 Conditional Probability
Example-1: There are 5 red chip, 4 blue chips, and 6 white chips in a basket. Two
chips are randomly selected. Find the probability that the second chip is red given that
the first chip is blue. (Assume that the first chip is not replaced.)
Because the first chip is selected and not replaced, there are only 14 chips remaining.
5
P (selecting a red chip / first chip is blue) =  0.357
14

Example-2: 100 college students were surveyed and asked how many hours a week
they spent studying. The results are in the table below. Find the probability that a
student spends more than 10 hours studying given that the student is a male.
Less then 5 5 to 10 More than 10 Total
Male 11 22 16 49
Female 13 24 14 51
Total 24 46 30 100
The sample space consists of the 49 male students. Of these 49, 16 spend more than
10 hours a week studying.
P (more than 10 hours / male) = 16
49
 0.327
8
 Independent Events & Multiplication Rule
Two events are independent if the occurrence of one of the events does not affect the
probability of the other event. Two events A and B are independent
if P (B |A) = P (B) or if P (A |B) = P (A).
▪ Events that are not independent are dependent.

Multiplication Rule:
▪The probability that two events, A and B will occur in sequence is
P (A and B) = P (A) · P (B |A).
▪If event A and B are independent, then the rule can be simplified to
P (A and B) = P (A) · P (B).

Example-1: Two cards are selected, without replacement, from a deck. Find the probability
of selecting a diamond, and then selecting a spade.
Because the card is not replaced, the events are dependent.
13 13 169
P (diamond and spade) = P (diamond) · P (spade |diamond). = 52  51 = 2652  0.064

Example-2: A die is rolled and two coins are tossed. Find the probability of rolling a 5, and
flipping two tails. 1 1
P (rolling a 5) = 6 . Whether or not the roll is a 5, P (Tail ) = ,
so the events are independent. 2

P (5 and T and T ) = P (5)· P (T )· P (T ) = 1  1  1 = 1  0.042

6 2 2 24 9
 Practice Problems
1. The following frequency distribution represents the ages of 30 students in a
statistics class. What is the probability that a student is between 26 and 33
years old?
Ages Frequency ( f )
18 – 25 13
26 – 33 8
34 – 41 4 P (age 26 to 33) = 8/30
42 – 49 3
50 – 57 2
 f = 30
2. Out of 36 people applying for the job, 20 are men and 16 are women. Eight of the men
and 12 of the women have Ph.D.’s. If one person is selected at random for the first
interview, find the probability that the one chosen has a Ph. D. First organize the
information:
a) What is a probability that applicant have Ph.D.?
P(PhD) = 20/36
b) What is a probability that the one chosen is a woman
and has a Ph.D.?
P(W and PhD) = 12/36
c) What is a probability that the one chosen is a woman or has a Ph.D.?
P(W or PhD) = P(W) + P(PhD) – P(W and PhD)
= 16/36 + 20/36 – 12/36 = 24/36 = 2/3
10
 Practice Problems…
3. Of the 20 television programs to be aired this evening, Marc plans to watch one,
which he will pick at random by throwing a dart at TV schedule. If 8 of the
programs are educational, 9 are interesting, and 5 are both educational and
interesting, find the probability that the show he watches will have at least one of
these attributes.
If ‘E’ represent “educational” and ‘I’ represent “interesting”, then
P(E) = 8/20, P(I) = 9/20, and P(E and I) = 5/20
8 9 5 𝟏𝟐
𝑃 𝐸 𝑜𝑟 𝐼 = 𝑃 𝐸 ∪ 𝐼 = 𝑃 𝐸 + 𝑃 𝐼 − 𝑃 𝐸 ∩ 𝐼 = + − =
20 20 20 𝟐𝟎
4. The probability that a person selected at random did not graduate from high school
is 0.25. If three people are selected at random, find the probability that
a) all three do not have a high school diploma.
Since each person is independent
P(all three do not have a high school diploma) = (.25)3
b) all three have a high school diploma.
P(all three have a high school diploma) = (1 − .25)3 = (.75)3
c) at least one has high school diploma.
P(at least one has HS diploma) = 1 – P(none have HS diploma) = 1 – (.25)3 = .9844

In general: P(event happening at least once) = 1 – P(event does not happen) 11

 Practice Problems…

5. The World Wide Insurance Company found that 53% of the

residents of a city had homeowner’s insurance with the company.
Of these clients, 27% also had car insurance with the company. If
a resident is selected at random, find the probability that the
resident has both homeowners and car insurance with the World
Wide Insurance Company.

Given:
▪ P(homeowner’s insurance) = 53%
▪ P(car insurance / homeowner’s insurance) = 27%
P (A and B) = P (A) · P (B |A).
P(homeowners and car insurance) = (.53) (.27) = .1431
12
 Random Variables
A random variable ‘x’ represents a numerical value associated with
each outcome of a probability distribution.

▪ Random variable (RV) X is function from Ω to R, i.e., X(ω) ∈ R for

each outcome ω ∈ Ω

Example: Flip coin three times: Ω={HHH, HHT, . . . , TTT}

Define RV: X(ω) to be # H’s in outcome ‘ω’

If ω = HTH, then X(ω) = 2, and

P(X = 2) = P({ω ∈ Ω : X(ω) = 2}) = P({HHT, HTH, THH}) = 3/8

 Types of Random Variables
1. Discrete:
▪ Support is finite or countable set, e.g., N ≡ {0, 1, 2,.. .} or {0, 1}.
▪ Distribution described by a probability mass function (PMF) p
where p(x) = P(X = x) and
𝑃 𝑋 ∈ 𝐴 = ෍ 𝑃(𝑥)
𝑥∈𝐴
2. Continuous:
▪ Support is a continuous set, e.g., R or [0, 1].
▪ Distribution described by a probability density function (PDF) p
𝑑𝐹(𝑥) 𝑥
𝑓(𝑥) where 𝑓 𝑥 = , 𝑖. 𝑒. 𝐹 𝑥 = ‫׬‬−∞ 𝑓 𝑢 𝑑𝑢 and
𝑑𝑥

𝑃 𝑋 ∈ 𝐴 = න 𝑓 𝑥 𝑑𝑥
𝐴
 Probability Distributions
A random variable is a function that assigns a real number to each outcome in the
sample space of a random experiment.

The probability distribution of a random variable X gives the probability for each
value of X.

Probability Mass Function

For a discrete random variable X with possible values x1, x2, …, xn, a probability
mass function is a function such that:

15
 Cumulative Distribution Function (CDF)
▪ Probabilistic behavior described by distribution function
▪ RV defined by cumulative distribution function (CDF) F → F(x) = P(X ≤ x)
▪ Properties of CDF:
o 0 ≤ F(x) ≤ 1

o F(x) is non-decreasing function

o limx→−∞ F(x) = 0,

o limx→+∞ F(x) = 1

Example: Flip fair coin three times

Let X = number of heads, so P (X = 0)=1/8, P (X= 1)= 3/8,
P (X = 2)=3/8, P (X= 3)= 1/8
▪ Support: {0, 1, 2, 3}
▪ CDF F (x) = P (X ≤ x) for each −∞ < x < ∞:
Expectation, Variance, and Moments
Expectation, Variance, and Moments
21