1.

Introduction to Probability Theory

JOMEL R. ALANZALON
STAT 3201: Statistical Theory
1st Semester A.Y. 2023-2024

CENTRAL LUZON STATE UNIVERSITY

1.2 Building Blocks of the
Probability Structure
JOMEL R. ALANZALON
STAT 3201: Statistical Theory
1st Semester A.Y. 2023-2024

CENTRAL LUZON STATE UNIVERSITY

 DEPARTMENT of
STATISTICS
1.2.1 Deterministic vs. Stochastic Models
An abstract model is a description of the essential properties of a phenomenon
that is formulated in mathematical terms. It is used as a theoretical
approximation of reality to help us understand the world around us.
Example: Volume of a rectangular aquarium.

𝑉 =𝑙×𝑤×ℎ

Abstract Model

Introduction to Probability Theory | 3

 DEPARTMENT of
STATISTICS
1.2.1 Deterministic vs. Stochastic Models
An abstract model is a description of the essential properties of a phenomenon
that is formulated in mathematical terms. It is used as a theoretical
approximation of reality to help us understand the world around us.
• Deterministic model describes a phenomenon whose outcome is fixed.
▪ Example: Area of a circle
• Stochastic model (Probabilistic model) describes the unpredictable
variation of the outcomes of a random experiment
▪ Example: Tossing a balanced coin and observing the upper face
• Probability theory is a stochastic modeling of the phenomenon of chance or
randomness.
Introduction to Probability Theory | 4
 DEPARTMENT of
STATISTICS 1.2.2 Random Experiment and Sample Space
• Random Experiment - experiment that can be repeated under similar
conditions, but whose outcome cannot be predicted with certainty, even
when the same experiment has been performed many times
The following are always associated with a random experiment:
o Sample space - the set of all possible outcomes, denoted by 𝛀
o Sample points (also referred as outcomes or realizations) – the
elements of the sample space, denoted by 𝝎
o Events - subsets of 𝛀 for which the probability is defined, denoted by
capital Latin letters, e.g., A, B, C

Introduction to Probability Theory | 5

 DEPARTMENT of
STATISTICS 1.2.2 Random Experiment and Sample Space

Example: Tossing a coin

Consider an experiment involving a single coin toss.

The possible outcomes are either head (H) or tail (T),

therefore the sample space is

Ω = 𝐻, 𝑇

Introduction to Probability Theory | 6

 DEPARTMENT of
STATISTICS 1.2.2 Random Experiment and Sample Space
Example: Tossing two coins Therefore sample space is
Consider an experiment Ω = 𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇 , 𝑛 Ω =4
involving 2 coin tosses.
Now, we can consider the following events:
𝐴 = event of observing exactly one head
𝐴 = {𝐻𝑇, 𝑇𝐻}
𝐵 = event of observing at least one head
𝐵 = {𝐻𝑇, 𝑇𝐻, 𝐻𝐻}
𝐶 = event of observing the same outcome
𝐶 = {𝐻𝐻, 𝑇𝑇}

Introduction to Probability Theory | 7

DEPARTMENT of
STATISTICS 1.2.2 Random Experiment and Sample Space
Discrete Sample Space
• Finite
• Example: Number of dots in a roll of a die
Ω = 1, 2, 3, 4, 5, 6 , n Ω = 6
• or countably infinite
• Example: Set of positive even numbers
Ω = 2, 4, … , n Ω = ∞

Continuous Sample Space

• neither finite nor countably infinite
• Example: Travel time from San Jose City to Quezon City

Introduction to Probability Theory | 8

 DEPARTMENT of
STATISTICS 1.2.3 The Event Space
• Class of all events associated with a given experiment
• denoted by a script Latin letter, usually A, B, F, …
Assumptions of the Event Space
We will always assume that the event space is a Boolean algebra (or
simply called algebra or field). That is, if A = event space then:
Propositions 1.19 – 1.21
1.19. Ω ∈ A. This implies that the sample space must be an event. Since it is an event then
its probability must be defined. Ω viewed as an event is called the sure event.
1.20. If 𝐴 ∈ A then 𝐴𝑐 ∈ A. This implies that if A is an event then its complement must
also be an event.
1.21. If 𝐴1 ∈ A and 𝐴2 ∈ A, then 𝐴1 ∪ 𝐴2 ∈ A. It implies that if 𝐴1 and 𝐴2 are events
then their union must also be an event.

Introduction to Probability Theory | 9

 DEPARTMENT of
STATISTICS 1.2.3 The Event Space
Example
Tossing a coin
Consider an experiment involving a single coin toss. Recall that the
sample space is
Ω = 𝐻, 𝑇

There are a total of 4 subsets of Ω and these comprise the power set.
𝒫 Ω = {Ω, ∅, 𝐻 , 𝑇 }

Always remember that the power set is a valid event space since it always
satisfies the 3 assumptions of an event space.
Introduction to Probability Theory | 10
 DEPARTMENT of
STATISTICS 1.2.3 The Event Space
Example
Identify which of the following is a valid event space?
A = Ω, ∅
B = Ω, 𝐻 , {𝑇}
C = 𝐻 ,Ω
Observe that A can be considered as a valid event space since it satisfies the 3
assumptions of an event space.
• For the first proposition is satisfied since Ω ∈ A
• Ω𝑐 = ∅ 𝑎𝑛𝑑 ∅𝑐 = Ω and are in the event space.
• The union of the two elements is in the event space, that is, (Ω ∪ ∅) = Ω
How about B and C?

Introduction to Probability Theory | 11

 DEPARTMENT of
STATISTICS 1.2.3 The Event Space
The theorems below follow from the assumption that the event space is
an algebra:
Theorem 1.22
∅∈A

Proof
We start with the given assumptions/propositions of event space.
Ω∈A by Proposition 1.19
Ω𝑐 ∈ A by Proposition 1.20
∅∈A by Complement Laws

Introduction to Probability Theory | 12

 DEPARTMENT of
STATISTICS 1.2.3 The Event Space
The theorems below follow from the assumption that the event space is an algebra:
Theorem 1.23
If 𝐴1 ∈ A and 𝐴2 ∈ A , then 𝐴1 ∩ 𝐴2 ∈ A
Proof
We start with the given assumptions/propositions of event space.
𝐴1 ∈ A, 𝐴2 ∈ A Given
𝐴1𝑐 ∈ A, 𝐴𝑐2 ∈ A by Proposition 1.20
(𝐴1𝑐 ∪ 𝐴𝑐2 ) ∈ A by Proposition 1.21
(𝐴1 ∩ 𝐴2 )𝑐 ∈ A by De Morgan’s Law
(𝐴1 ∩ 𝐴2 )𝑐 𝑐 ∈ A by Proposition 1.20
𝐴1 ∩ 𝐴2 ∈ A by Involution Law

Introduction to Probability Theory | 13

 DEPARTMENT of
STATISTICS 1.2.3 The Event Space

Theorem 1.24
If 𝐴1 ∈ A, 𝐴2 ∈ A, … , 𝐴𝑛 ∈ A, then
𝑛

ራ 𝐴𝑖 ∈ A and
𝑖=1
𝑛

ሩ 𝐴𝑖 ∈ A
𝑖=1

Introduction to Probability Theory | 14

 DEPARTMENT of
STATISTICS 1.2.3 The Event Space (Proof by Mathematical Induction)
Step 1. Prove that it is true for 𝑛 = 1
1

ራ 𝐴𝑖 = 𝐴1 ∈ A
𝑖=1
Step 2. Assume that it is true for 𝑛 = 𝑘. If 𝐴1 ∈ A, 𝐴2 ∈ A, … , 𝐴𝑘 ∈ A, then
𝑘

ራ 𝐴𝑖 ∈ A that is 𝐴1 ∪ 𝐴2 ∪ ⋯ ∪ 𝐴𝑘 ∈ A
𝑖=1
Step 3. Prove that it is true for 𝑛 = 𝑘 + 1, that is, if
𝐴1 ∈ A, 𝐴2 ∈ A, … , 𝐴𝑘 ∈ A, 𝐴𝑘+1 ∈ A then
𝑘+1

ራ 𝐴𝑖 ∈ A
𝑖=1
Introduction to Probability Theory | 15
 DEPARTMENT of
STATISTICS 1.2.3 The Event Space
𝑘+1 𝑘

ራ 𝐴𝑖 = 𝐴1 ∪ 𝐴2 ∪ ⋯ ∪ 𝐴𝑘 ∪ 𝐴𝑘+1 = ራ 𝐴𝑖 ∪ 𝐴𝑘+1
𝑖=1 𝑖=1

Since ‫=𝑖𝑘ڂ‬1 𝐴𝑖 ∈ A and 𝐴𝑘+1 ∈ A, then by Proposition 1.21,

𝑘 𝑘+1

ራ 𝐴𝑖 ∪ 𝐴𝑘+1 = ራ 𝐴𝑖 ∈ A
𝑖=1 𝑖=1

Therefore,
𝑛

ራ 𝐴𝑖 ∈ A
𝑖=1
Introduction to Probability Theory | 16
 DEPARTMENT of
STATISTICS 1.2.3 The Event Space (Proof Intersection using MI)
Step 1. Prove that it is true for 𝑛 = 1
1

ሩ 𝐴𝑖 = 𝐴1 ∈ A
𝑖=1
Step 2. Assume that it is true for 𝑛 = 𝑘. If 𝐴1 ∈ A, 𝐴2 ∈ A, … , 𝐴𝑘 ∈ A, then
𝑘

ሩ 𝐴𝑖 ∈ A that is 𝐴1 ∩ 𝐴2 ∩ ⋯ ∩ 𝐴𝑘 ∈ A
𝑖=1
Step 3. Prove that it is true for 𝑛 = 𝑘 + 1, that is, if
𝐴1 ∈ A, 𝐴2 ∈ A, … , 𝐴𝑘 ∈ A, 𝐴𝑘+1 ∈ A then
𝑘+1

ሩ 𝐴𝑖 ∈ A
𝑖=1
Introduction to Probability Theory | 17
 DEPARTMENT of
STATISTICS 1.2.3 The Event Space
𝑘+1 𝑘

ሩ 𝐴𝑖 = 𝐴1 ∩ 𝐴2 ∩ ⋯ ∩ 𝐴𝑘 ∩ 𝐴𝑘+1 = ሩ 𝐴𝑖 ∩ 𝐴𝑘+1
𝑖=1 𝑖=1

Since ‫=𝑖𝑘ځ‬1 𝐴𝑖 ∈ A and 𝐴𝑘+1 ∈ A, then by Theorem 1.23,

𝑘 𝑘+1

ሩ 𝐴𝑖 ∩ 𝐴𝑘+1 = ሩ 𝐴𝑖 ∈ A
𝑖=1 𝑖=1

Therefore,
𝑛

ሩ 𝐴𝑖 ∈ A
𝑖=1
Introduction to Probability Theory | 18
 DEPARTMENT of
STATISTICS 1.2.3 The Event Space

• Elementary event - event consisting of a single sample point

• Compound event - an event that can be expressed as the union of

distinct elementary events

• Mutually exclusive or Disjoint events - events that do not take place or

occur at the same time

Introduction to Probability Theory | 19

1.3 Methods of Assigning
Probabilities
JOMEL R. ALANZALON
STAT 3201: Statistical Theory
1st Semester A.Y. 2023-2024

CENTRAL LUZON STATE UNIVERSITY

 DEPARTMENT of
STATISTICS 1.3 Methods of Assigning Probabilities

Probability is expressed in different ways. Some of the terms people use

for probability are chance, likelihood, percentage and proportion.

Formally,

• Probability is a long-term chance that a certain outcome will occur from

some random process.
• Probability of an event is a real number, attached to the event, that tells
us how likely that event is.

Introduction to Probability Theory | 21

 DEPARTMENT of
STATISTICS 1.3.1 Classical Approach or A Priori Probability
If the sample space of a random experiment has 𝑛(Ω) equally likely outcomes
and if 𝑛(𝐴) of these outcomes are elements of A, then the probability of A can
be assigned by:
𝑛 𝐴
𝑃 𝐴 =
𝑛(Ω)
• This method requires that the sample space is finite and all the elementary events
(in this case, singletons) are assigned the same probabilities. If even just one of
these conditions is not satisfied then the formula given in the definition may not be
valid.
• The “equally likely” assumption requires the experiment to be carried out in such a
way that the assumption is realistic; such as, using a balanced coin, using a die
that is not loaded, using a well-shuffled deck of cards, using simple random
sampling, and so forth. This assumption also requires that the sample space is
appropriately defined.
Introduction to Probability Theory | 22
 DEPARTMENT of
STATISTICS 1.3.1 Classical Approach or A Priori Probability
Example: One urn contains 3 red balls, 2 white balls, and 1 blue ball. A
second urn contains 1 red ball, 2 white balls, and 3 blue balls. One ball is
selected at random from each urn.

Ω = 𝑥, 𝑦 𝑥𝜖{𝑅1 , 𝑅2 , 𝑅3 , 𝑊1 , 𝑊2 , 𝐵1 } and y𝜖{𝑅1 , 𝑊1 , 𝑊2 , 𝐵1 , 𝐵2 , 𝐵3 }}

𝑛 Ω = 6 ∗ 6 = 36

Let A = event of selecting 2 red balls 𝑛 𝐴 =3∗1=3

B = event of selecting 2 white balls 𝑛 𝐵 =2∗2=4

Find the probability of each event using the classical definition of probability
𝑛 𝐴 3 𝟏 𝑛 𝐵 4 𝟏
𝑃 𝐴 = 𝑛(Ω) = 36 = 𝟏𝟐 𝑃 𝐵 = = =
𝑛(Ω) 36 𝟗
Introduction to Probability Theory | 23
 DEPARTMENT of
STATISTICS 1.3.1 Classical Approach or A Priori Probability

Example: Consider the experiment of tossing two coins.

Ω = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇}

𝑛 Ω =4

Let 𝐶 = event that at least 1 head comes up

𝐶 = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻} 𝑛 𝐶 =3

Find the probability of event 𝐶 using the classical definition of probability

𝑛 𝐶 𝟑
𝑃 𝐶 = = = 𝟎. 𝟕𝟓 = 𝟕𝟓%
𝑛(Ω) 𝟒

Introduction to Probability Theory | 24

 DEPARTMENT of
STATISTICS 1.3.2 Relative Frequency Approach or
A Posteriori Probability
Suppose a random experiment is repeated 𝑛 times under uniform
conditions, and if event 𝐴 occurred 𝑛𝑎 times, then the relative frequency
𝑛𝑎
for which 𝐴 occur is 𝑓𝑛 𝐴 = 𝑛 . If the limit of 𝑓𝑛 𝐴 as n approaches
infinity exists then one can assign the probability of A by:

𝑃 𝐴 = 𝑙𝑖𝑚 𝑓𝑛 𝐴
n→∞

• This method requires the existence of the limit of the relative

frequencies. This property is known as statistical regularity. This
property will be satisfied if the trials are independent and are
performed under uniform conditions.

Introduction to Probability Theory | 25

 DEPARTMENT of
STATISTICS 1.3.2 Relative Frequency Approach or
A Posteriori Probability 𝑛𝑎
𝑃 𝐴 =
Example: Consider the experiment of tossing a coin. 𝑛
Let 𝐴 = event that a head comes up. The first few Trial Relative
outcomes of one possible sequence of trials is as No. Outcome Frequency
follows: 1 H 1/1=1
2 H 2/2=1
For the first few trials, the values of the relative 3 T 2/3=0.67
frequencies will fluctuate. However, the relative 4 H 3/4=0.75
frequencies will tend to stabilize to its limit as n 5 T 3/5=0.6
becomes large. 6 H 4/6=0.67
7 T 4/7=0.57
• A good estimate of the probability of an event 8 T 4/8=0.5
𝐴 is its relative frequency of occurrence, so 9 H 5/9=0.56
long as n is large enough. 10 H 6/10=0.6
Introduction to Probability Theory | 26
 DEPARTMENT of
STATISTICS 1.3.3 Subjective Approach

Let 𝐴 be an event. The subjective probability of event 𝐴 is the value 𝑝,

0 < 𝑝 < 1, determined by the observer and influenced by the event
itself only insofar as the observer is well informed about it. The observer
chooses p to be close to 1 if he feels that event A is very likely to occur,
while, he chooses p close to 0 if he feels that the event is unlikely to
occur.

• Two different people may assign different subjective probabilities

to the same event, even if they have access to the same
information.

Introduction to Probability Theory | 27

 1.4 Applications of
Probability Theory
JOMEL R. ALANZALON
STAT 3201: Statistical Theory
1st Semester A.Y. 2023-2024

CENTRAL LUZON STATE UNIVERSITY

 DEPARTMENT of
STATISTICS 1.4 Applications of Probability Theory

Probability Theory is used in diverse fields.

Examples of its application are enumerated below:
• A fisherman checks the chance of rain before going to the sea.
• A doctor tells a patient the chance of survival before performing a surgery.
• A politician wants to know the probability of winning before running in the
election.
• An investor looks at the stock patterns before joining the Stock market.

Introduction to Probability Theory | 29