IS141: Statistics and Probability

Probability Theory

Dr. Emmanuel

April 3, 2024
Outline

Probability
Introduction
Axiomatic definition of Probability
Conditional Probability
Independent events
Theorem of Total Probability
Baye’s Theorem
Counting Technique
Introduction

Probability theory is the study of random or unpredictable

experiment.

Although all possible outcomes of an experiment may be known in

advance, the outcomes of a particular performance of an
experiment cannot be predicted owing to a number of unknown
causes.

For instance, Although the number of telephone calls received in a

board in a 5 minutes interval is non-negative integer, we cannot
predict exactly the number of calls received in the next 5 minutes.

In such situations we talk of the chance or probability of

occurrence of a particular outcome, which is the quantitative
measure of the likelihood of occurrence of the outcome.
Events and the Sample Space
The term experiment to describe a method of collecting data, by
observing events in either controlled or uncontrolled situations.

An experiment is the process by which an observation (or

measurement) is obtained.

The observation or measurement generated by an experiment may

or may not be numerical. Examples of experiments are:
▶ Recording a test grade
▶ Measuring daily rainfall
▶ Recording a person’s opinion on the location of a new
recycling center
▶ Testing a printed circuit board to determine whether it is
defective or acceptable
▶ Tossing a coin and observing the face that appears
A simple event is the outcome observed on a single repetition of
an experiment.

Event is a collection of simple events.

Sample space is the collection of all simple events of an

experiment.
Example
Experiment: Record a person’s blood type. The four possible
outcomes are these simple events:
E 1: Blood type A
E 2: Blood type B
E 3: Blood type AB
E 4: Blood type O

The sample space is S = {E 1, E 2, E 3, E 4}, or

S = {A, B, AB, O}.
Example: A medical technician records a person’s blood type and
Rh factor. List these simple events in the experiment.

Solution: For each person, a two stage procedure is needed to

record the two variables of interest. The tree diagram for these
simple events is shown below.
The eight simple events in the tree diagram form the sample space,
S = {A+, A−, B+, B−, AB+, AB−, O+, O−}

Definition of Probability of an event

Let S be the sample space (outcomes assumed to be equally likely)
and A be an event (subset of S) associated with a random
experiment.

Then the probability of event A occurring denoted as P(A), is

defined by

n(A) Number of cases favourable toA

P(A) = = . (1)
n(S) Exhaustive number of cases inS

For example, the probability of getting an even number in the die

tossing experiment is 0.5, since S = {1, 2, 3, 4, 5, 6}, A = {2, 4, 6},
n(S) = 6 and n(A) = 3.
Statistical Definition of Probability
Let a random experiment be repeated n times and let an event A
occur nA times out of the n trials.
nA
The ratio n is called the relative frequency of the event A.

As n increases, nnA shows a tendency to stabilize and approach a

constant value, denoted by P(A) called the probability of an event
A, i.e.
nA
P(A) = lim . (2)
n→∞ n
For example, if we want to find the probability that a spare part
produced by a machine is defective, we study the record of
defective items produced by the machine over a considered period
of time. If out of 10000 items produced, 500 are defective it is
assumed that the probability of a defective item is 0.05.
Axiomatic Definition of Probability

Let S be the sample space and A be an event associated with a

random experiment. Then the probability of the event A, denoted
by P(A) satisfies the following axioms
(i) 0 ≤ P(A) ≤ 1
(ii) P(S) = 1
(iii) If A is an impossible event, P(A) = 0
(iv) If A and B are mutually exclusive events,
P(A ∪ B) = P(A) + P(B)
(v) If A1 , A2 , · · · , An , · · · are sets of mutually exclusive events,
then
P(A1 ∪A2 ∪· · ·∪An ∪· · · ) = P(A1 )+P(A2 )+· · ·+P(An )+· · ·
Note: A set of events are said to be mutually exclusive if the
occurrence of any one of them excludes the occurrence of the
others, i.e. P(A1 ∩ A2 ∩ · · · ) = 0.
Laws of Probability

In the development of the probability theory, all results are derived

directly/indirectly using the axioms of probability.

The following are some of the basic results.

(i) The probability of an impossible event is zero i.e. if ϕ is the
event containing no sample point, P(ϕ) = 0.
(ii) If Ā is the complement of event A, then P(Ā) = 1 − P(A) ≤ 1.
(iii) If A and B are any two events,
P(A ∪ B) = P(A) + P(B) − P(A ∩ B ≤ P(A) + P(B).
(iv) If B ⊂ A, P(B) ≤ P(A).
Conditional Probability

Conditional probability of an event B, assuming that the event A

has happened, is denoted by P(B/A) and defined as

P(A ∩ B)
P(B/A) = , P(A) ̸= 0. (3)
P(A)

For example, when a fair dice is tossed, the conditional probability

of getting 1, given that an odd number has been obtained, is given
as follows: S = {1, 2, 3, 3, 4, 5, 6}, A = {1, 3, 5}, B = {1}. Then
P(B/A) = P(A∩B) 1/6
P(A) = 1/2 = 3 .
1
Independent Events

A set of events is said to be independent if the occurrence any one

of them does not depend on the occurrence/non-occurrence of the
others.

If two events A and B are independent, then

(i) P(B/A) = P(B),
(ii) P(A ∩ B) = P(A) × P(B), this can be extended to any
number of independent events
(iii) events Ā and B (similarly A and B̄ ) are also independent,
(iv) events Ā and B̄ are also independent
Example
Two defective tubes get mixed up with two good ones. The tubes
are tested, one by one, until both defective are found. What is the
probability that the last defective tube is obtained on (i) the
second test, (ii) third test and (iii) the fourth test?
Answer
Let D represent defective and N represent non-defective tube.

(i)P(Second D in the second test)

= P(D in the first test and D in the second test)
= P(D1 ∩ D2 )
= P(D1 ) × P(D2 ), (by independence)
2 1 1
= × =
4 3 6
(ii)P(Second D in the third test) = P(D1 ∩ N2 ∩ D3 or N1 ∩ D2 ∩ D3 )
= P(D1 ∩ N2 ∩ T3 ) + P(N1 ∩ D2 ∩ T3 )
2 2 1 2 2 1
= × × + × ×
4 3 2 4 3 2
1
= .
3

(iii)P(Second D in the fourth test)

= P(D1 ∩ N2 ∩ N3 ∩ D4 or N1 ∩ D2 ∩ N3 ∩ D4 or N1 ∩ N2 ∩ D3 ∩ D4 )
= P(D1 ∩ N2 ∩ N3 ∩ D4 ) + P(N1 ∩ D2 ∩ N3 ∩ D4 ) + P(N1 ∩ N2 ∩ D3 ∩ D4 )
2 2 1 2 2 1 2 2 1 1
= × × ×1+ × × ×1+ × × ×1= .
4 3 2 4 3 2 4 3 2 2
Theorem of Total Probability

If B1 , B2 , · · · , Bn is a set of exhaustive and mutually exclusive

events, and A is another event associated with (caused by) Bi , then
n
X
P(A) = P(Bi )P(A/Bi ) (4)
i=1

Proof
The inner circle represent the event A, A can occur along with (due
to ) B1 , B2 , · · · , Bn that are exhaustive and mutually exclusive.
AB1 , AB2 , · · · , ABn are also mutually exclusive,
It follows that A  = AB1 +AB2 + · · · + ABn
P P n
P
Thus, P(A) = P ABi = P(ABi ) = P(Bi )P(A/Bi ).
i i i=1
Baye’s Theorem
If B1 , B2 , · · · , Bn is a set of exhaustive and mutually exclusive
events associated with a random experiment and A is another
event associated with (or caused by) Bi , then

P(Bi ) × P(A/Bi )
P(Bi /A) = n , i = 1, 2, . . . , n. (5)
P
P(Bi ) × P(A/Bi )
i=1

Proof

P(Bi ∩ A) = P(Bi ) × P(A/Bi ) = P(A) × P(Bi /A)

P(Bi ) × P(A/Bi )
∴ P(Bi /A) =
P(A)
P(Bi ) × P(A/Bi )
= n , i = 1, 2, . . . , n
P
P(Bi ) × P(A/Bi )
i=1
Example
One-half percent of the population has AIDS. There is a test to
detect AIDS. A positive test result is supposed to mean that you
have AIDS but the test is not perfect. For people with AIDS, the
test misses the diagnosis 2% of the times. And for the people
without AIDS, the test incorrectly tells 3% of them that they have
AIDS.
(a) What is the probability that a person picked at random will
test positive?
(b) What is the probability that you have AIDS given that your
test comes back positive?

Answer:
Let A denote the event of one who has AIDS and B denote the
event that the test comes out positive.
(a) The probability that a person picked at random will test
positive is given by

P(test positive) = (0.005)(0.98) + (0.995)(0.03)

= 0.0049 + 0.0298 = 0.035.

(b) The probability that you have AIDS given that your test
comes back positive is given by
favorable positive branches
P(A/B) =
total positive branches
(0.005)(0.98)
=
(0.005)(0.98) + (0.995)(0.03)
0.0049
= = 0.14.
0.035
Counting Technique

There are three basic counting techniques: multiplication rule,

permutation and combination.

Multiplication Rule
If E1 is an experiment with n1 outcomes and E2 is an experiment
with n2 possible outcomes,
then the experiment which consists of performing E1 first and then
E2 consists of n1 n2 possible outcomes.

Example
Find the possible number of outcomes in a sequence of two tosses
of a fair coin.
Solution
The number of possible outcomes is 2 × 2 = 4. This can easily be
shown by tree diagram.
Example
Find the number of possible outcomes of the rolling of a die and
then tossing a coin.

Example
How many different license plates are possible if the country
numbering system uses three letters followed by three digits.

Solution
Here we have letters A to Z for three positions in the plate number.
Also, we have digits 0 to 9 for three positions.
Then,

the number of different license plates = (26)3 (10)3

= (17576)(1000)
= 17, 576, 000.
Permutation

Permutation is an ordered arrangement of objects (letters or

numbers).

These n objects could be distinct or not distinct.

The number of permutations of n distinct objects taken r at a time

denoted by nPr is given by
n!
nPr = ,
(n − r )!

where n! = n × (n − 1) × (n − 2) × · · · × 2 × 1 and 0! = 1.
Example
How many numbers with three distinct digits are possible using the
digits 3, 4, 5, 6, 7, 8?

Solution
We need to find nPr where n = 6 and r = 3
n! 6!
nPr = = = 120.
(n − r )! (6 − 3)!

Then, Five of these numbers are 345, 356, 347, 378, 567
The arrangement of n objects such that q of them resemble is
n!
given by q! .
Example
In how many ways can the letters of the word ESSENTIAL be
arranged?
Combination

Unlike in permutation, in the case of combination, the order is not

important.

We can define a combination as a selection

  of r objects in a group
n
of n objects. It is denoted by nCr or , and defined by
r
 
n n!
= .
r (n − r )!r !

Example
How many committees of two chemists and one physicist can be
formed from 4 chemists and 3 physicists?
Assignment 1
1. Assume that not all airline passengers show up for their
reserved seat. An airline sells 125 tickets for a flight that
holds only 120 passengers. The probability that a passenger
does not show up is 0.10, and the passengers behave
independently.
(a) What is the probability that every passenger who shows up can
take the flight?
(b) What is the probability that the flight departs with no empty
seats?
2. A particular mix of concrete reaches a specified level of
hardness after X hours, where X ∼ N(25, 3.82). A building is
constructed in which concrete is laid in three stages. As soon
as the concrete is found to be hard in one stage, the concrete
for the next stage is laid. Calculate the probability that Stage
3 commences at least 45 hours after Stage 1 commences.

Submission deadline: Before 3rd May 2024