ST 1201: Probability Theory

SUFIANI, Michael
[email protected]
0694-482740,0685-718630

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 1

 UNIT I: INTRODUCTION
Basic Terminologies.
Some Kinds of Events.
Probability of an Event.
Axioms of Probability.
More on Probability of an event.
Condition Probability
Bayer’s Theorem.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 2

 1.1. Basic Terminologies
 An experiment
An experiment is the process that yields one of a given set of the
possible outcomes.

 Possible Outcomes of an Experiment

Possible outcomes of an experiment are the results which may occur
from any experiment.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 3

1.1. Basic Terminologies
 Sample space
The set of all possible outcomes of a statistical experiment is called
the sample space and is normally represented by the symbol S.

NOTE
A sample space may have a finite or infinite depending on number
of elements it comprises.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 4

1.1. Basic Terminologies
Finite Sample Space
Is when all elements that consist a sample space can be listed down.
Example 1(Tossing a coin)
Sample space: S = {Head or Tail} or we could write
S= {H, T}
Example 2 (Tossing of a Die)
If the interest is on the number shown on the face of the die then the
sample space is
S= {1, 2, 3, 4, 5, 6}
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 5
 1.1. Basic Terminologies
Infinite Sample Space
This is when a sample space consists of member or elements that can’t
all be listed down.
In such cases sample space is given as statement or rules.

Example
 S={x : x is a student with GPA below 3.5}

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 6

 1.1. Basic Terminologies
 An Element
Each outcome in sample space is called elements or member of the
sample space or simply sample point.

Example (Tossing a Coin)

Sample space is : S = {Head, Tail}
Here elements are: Head and Tail.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 7

 1.1. Basic Terminologies
 An Event
This is the subset of sample space which comprising of one or more sample
point or elements of a sample space.
Example 1 (Tossing a Coin)
Sample space is : S = {Head, Tail}
Some events are: E1={Head}, E2={Tail}
Example 2 (Tossing of a Die)
Sample space is : S= {1, 2, 3, 4, 5, 6}
Some events are: E1={4}, E2={1, 2}, E3={2, 4, 6}, E4={1, 3, 5} etc.
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 8
1.2. Some Kinds of Events
A Compliment of an Event
Consists of all elements that are in sample space but not in specified
event.
Example
Consider a sample space S={Head, Tail} of tossing a coin.
Let an event A being stated as A={Head}
A compliment of event A denoted as A’ or 𝐴𝑐 is given as 𝐴′ ={Tail}.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 9

 1.2. Some Kinds of Events
 Intersection of Events
Two or more events are said to have intersection if they have one or more
member(s) in common.
Example
 If S={1, 2, 3, 4, 5, 8, 9, 14, 17, 23, 36, 44, 56, 67, 78}
A={1, 5, 9, 14, 23, 56, 78}
B={5, 8, 14, 56, 67, 78}
The Intersection of event A and B is : A∩ B = *5, 14, 56, 78+

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 10

 1.2. Some Kinds of Events
Union of two or more Events
The union event of two or more events comprise of all element that are
found in all those events without repetition of elements.
Example
 If S={1, 2, 3, 4, 5, 8, 9, 14, 17, 23, 36, 44, 56, 67, 78}
A={1, 5, 14, 23, 56}
B={5, 8, 14, 56, 67, 78}
The Union of events A and B is :
A ∪ B = *1, 5, 8, 14, 23, 56, 67, 78+
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 11
 1.2. Some Kinds of Events
NOTE
It can be noted that 𝐴 ∪ 𝐴𝑐 =𝑆

 Mutual Exclusive Events or Disjoint Events

 Are the events which have no elements in common, thus there is no
element that is found in both of the events.
Or
 Mutually exclusive events are the events where by happening of any
one of them exclude the happening of all the others.
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 12
1.2. Some Kinds of Events
Example
 Consider a sample space comprise of age groups of respondents
in experiment of willingness to wait for the future income:
S={x : x is respondents’ age groups}
Some events:
A={x: x < 14}
B={x: 14≤x< 18+
C={x: 18 ≤x≤ 25}
D={x: x> 25+
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 13
1.2. Some Kinds of Events
NOTE
 None of above events do intersect thus simply means these are
disjoint sets or are mutually exclusive events.

That is:
A∩B ∩ C ∩ D=∅
A∩B = ∅
C∩D=∅

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 14

 1.2. Some Kinds of Events
 Independent Events
 Events are said to be independent if and only if the occurrence of one
event does not affect or limit the occurrence of other events.
Example
 Consider the results of students’ university exams for 3 courses :
ST 1101, LG 1101 and DS 1101.
 Does the results from ST 1101 affect the results of LG 1101 or DS
1101?
The answer here is definitely no that is results for one course does not
affect the other results and such events are called independent events.
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 15
 1.2. Some Kinds of Events
Exhaustive Events
 Events are said to be exhaustive if their union cover all the events within the
entire sample space.

For example events 𝐴 and 𝐵 are said to be collectively exhaustive if

𝐴∪𝐵 =𝑆

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 16

 1.3. Probability of an Event
The probability of a simple event is a number that measures the likelihood that
the event will occur when the experiment is performed.
The probability can be approximated by the proportion of times that the simple
event is observed when the experiment is repeated a very large number of times.
Mathematical Expression of Probability of an Event
Probability of an event 𝐸, denoted by 𝑃 (𝐸), is the ratio of the number of
outcome of 𝐸 to the total number of outcomes in the sample space.
That is
𝑛(𝐴)
𝑃(𝐴) =
𝑛(𝑆)
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 17
 1.4. Axioms of Probability
These are general properties that guard or provide the rule for probability to
operate.
Assuming the sample space S has the following events 𝐴1, 𝐴2, 𝐴3, … 𝐴𝑘 , then
these events are assumed to have the following properties:
All probability of events must lie between 0 and 1. That is:
0 ≤ 𝑃 𝐴𝑖 ≤ 1, where 𝑖 = 1, 2, 3, … , 𝑘.
The sum of probability of all these events must be equal to 1. That is:

𝑘
𝑖=1 𝑃 𝐴𝑖 = 1

Thus 𝑃 𝐴1 +𝑃 𝐴2 +…+𝑃 𝐴𝑘 =1
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 18
 1.5. More on Probability of an Event
Mutual Exclusive Events
 Note that the probability of a null event is equal to zero. That is:
P{∅ }=0
 Since we know that for the two mutually exclusive events 𝐴 and 𝐵 ,
𝐴 ∩ 𝐵 = ∅ which implies that 𝑃 𝐴 ∩ 𝐵 = 0.
It follows that 𝑃 𝐴 ∪ 𝐵 = 𝑃 𝐴 + 𝑃 𝐵 = 𝑃 𝐴 + 𝑃 𝐴′ = 𝑃 𝑆 = 1
Example 1
Consider the experiment of tossing two fair coins. Calculate the probability of at
least one head to appear by using the complementary relationship.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 19

 1.5. More on Probability of an Event
Example 2
Consider the following sample space of numbers:

S = {-4, -2, 1, 3, 5, 6, 7, 8, 9, 10}. Let the events A and B be defined as:

A= choosing a negative number from S and

B = choosing an odd number from S.

Find the probability of choosing a negative or an odd number from S.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 20

 1.5. More on Probability of an Event
Independent Events
Two events A and B are said to be independent if and only if
𝑃 𝐴∩𝐵 =𝑃 𝐴 𝑃 𝐵
Example
Consider the die-tossing experiment with equally likely simple events
(1,2,3,4,5,6). Define the events A and B as follows:
A:{observe an even number}={2,4,6}
B: {Observe a number less than or equal to 4}={1,2,3,4 }
Determine whether the two events are independent.
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 21
1.5. More on Probability of an Event
Exhaustive Events
If 𝐴 and 𝐵 are any two events then the probability that at least
one of them occurs is given by:
𝑃 𝐴∪𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴∩𝐵

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 22

 1.6. Conditional Probability
This is the probability of one event, given that another event has occurred.

 Let 𝐴 and 𝐵 be events, we denote the conditional probability as 𝑃 (𝐴/𝐵)

(read as the probability that event 𝐴 occurs given event 𝐵 has occurred).

Mathematically conditional probability of event 𝐴 given event 𝐵 has occurred

is written as:
𝑃(𝐴∩𝐵)
𝑃(𝐴/𝐵)=
𝑃(𝐵)
where 𝑃(𝐵) ≠ 0
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 23
 1.6. Conditional Probability
Example 1
Let event A denote that “student is female” and let event B denote that
“student is Tanzanian” a class of 100 students suppose 60 are Tanzanians, and
suppose that 10 of the Tanzanian students are females. Find the probability that
if I pick a Tanzanian student, she will be a girl?
Example 2
 If the probability that regularly scheduled flight depart on time is 0.83, the
probability that it arrives on time is 0.82 and the probability that it departs and
arrives on time is 0.78 then, find the probability that a plane
(a) Arrives on time given that it departed on time.
(b) Departed on time given that it has arrived on time.
Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 24
 1.7. Bayes' Theorem
 It describes the probability of an event based on prior knowledge of
conditions.
Let E1 and E2 be two mutually exclusive events forming a partition of the
sample space S , for any arbitrary event E of the sample space such that
P(E) ≠ 0.
That is:

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 25

1.7. Bayes' Theorem
It follows that:

𝑃 𝐸𝑖 ∩ 𝐸 𝑃 𝐸/𝐸𝑖 𝑃 𝐸𝑖
𝑃 𝐸𝑖 /𝐸 = =
𝑃 𝐸 𝑃 𝐸/𝐸1 𝑃 𝐸1 + 𝑃 𝐸/𝐸2 𝑃 𝐸2

Let us proof the theorem by first deriving the expression for the
law of total probability.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 26

Proof
P( Ei E )
P( Ei / E )   P( Ei E )  P( Ei / E ) P( E )......(i )
P( E )

Also
P( E Ei )
P( E / Ei )   P( E Ei )  P ( Ei E )  P ( E / Ei ) P ( Ei )......(ii )
P( Ei )

Since (i)=(ii) then

P( Ei / E ) P( E )  P( E / Ei ) P( Ei )
P( E / Ei ) P( Ei )
 P( Ei / E )  ........(iii )
P( E )
HABIEL LUVANDA
But
E  (E E1 ) ( E E2 )
 P( E )  P( E E1 )  P( E E2 )
 P( E / E1 ) P( E1 )  P( E / E2 ) P( E2 )........(iv)
This is called the law of total probability
Substitute (iv) in (iii) gives
P( E / Ei ) P( Ei )
P( Ei / E ) 
P( E / E1 ) P( E1 )  P( E / E2 ) P( E2 )

HABIEL LUVANDA
 1.7. Bayes' Theorem
NOTE
 Generally Suppose 𝐴1 , 𝐴2 , 𝐴3 , 𝐴4 , … , 𝐴𝑘 be a partition of sample space 𝑆
i.e the event 𝐴𝑖 are mutually exclusive and their union is S.
 Let B be any event of S with 𝑃(𝐵)>0 , we have
∀𝑖 = 1, 2, 3, … , 𝑘

𝑃 𝐴𝑖 ∩ 𝐵 𝑃 𝐵/𝐴𝑖 𝑃 𝐴𝑖
𝑃 𝐴𝑖 /𝐵 = = 𝑘
𝑃 𝐵 𝑖=1 𝑃 𝐵/𝐴𝑖 𝑃 𝐴𝑖

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 29

 1.7. Bayes' Theorem
Example 1
The sample space S is described as "the integers 1 to 15" and is partitioned into
the following events:

E1 = "the integers 1 to 8" and

E2 = "the integers 9 to 15".

If E is the event "even number" then use Bayes' theorem to find P(E1 | E) .

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 30

 1.7. Bayes' Theorem
Example 2
 In a factory line A, B and C produce respectively 30, 45 and 25 % of the
total production. The percent of the rejected items are 8, 3 and 5 respectively.
An item is selected at random.

(i) what is the probability that it will be a reject?

(ii) If a rejected item is selected, what is the probability that it come from line
C?

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 31

Exercise 1
1. The following table gives the probabilities of a randomly selected individual, the
selected individual is a smoker (A) and individual gets cancer (B).
Gets cancer (B) Does not get
Cancer (B’)

Smoker (A) 0.50 0.20

Nonsmoker (A’) 0.10 0.20

(a) Determine whether or not events A and B are independent?

(b) What is the probability that a selected individual gets cancer given the individual
is smoker.
(c) What is the probability that a selected individual gets cancer given the individual
is nonsmoker.

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 32

Exercise 1
2. The local area network (LAN) for the College of Informatics and Virtual Education
at The University of Dodoma is temporarily shut-down for repairs. Previous shutdowns
have been due to hardware failure, software failure, or power failure. Maintenance
engineers have determined that the probabilities of hardware, software, and power
problems are 0.01, 0.05 and 0.02 respectively. They have also determined that if
the system experience hardware problems, it shuts down 73% of the time. Similarly,
if software problems occur, the system shuts down 12% of the time; and if power
failure occurs, the system shuts down 88% of the time.
(a) What is the probability of the shut down.
(b) What is the probability that the current shutdown of the LAN is due to hardware
failure?

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 33

End

Tuesday, April 4, 2023 HISTORY & DEVELOPMENT OF STATISTICS 34