Chapter 1: Introduction to probability

Phan Thi Khanh Van

E-mail: [email protected]

January 9, 2025

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 1 / 33

Table of Contents

1 Set Theory and Counting Rules

Set Theory
Counting rule

2 Sample space, Events and Probability

Sample Space and Events
Modern probability
Conditional Probability

3 Total probability formula and Bayes’ formula

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 2 / 33

Definition
Empty set: ∅
Subset: A ⊂ B
Proper Subset: A B
Disjoint Sets: A ∩ B = ∅
Venn Diagram: Represents the relationship between sets.
Cardinality: |A|

Complement: A (or Ac , A0 )
A = Ω\A
A = {x|x ∈ Ω và x ∈
/ A}
A=A

Example example
If Ω = {1, 2, 3, 4, 5, 6}, A = {1, 3, 5}⇒ Ac = {2, 4, 6}.
If Ω = R, A = [2, 4]⇒ Ac = (−∞, 2) ∪ (4, ∞).

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 3 / 33

Set operations

Union: A + B or A ∪ B
A + B = {x|x ∈ A or x ∈ B}

Intersection: AB or A ∩ B
AB = {x|x ∈ A and x ∈ B}

Example
1 {1, 3, 4, 7} ∩ {2, 3, 5}= {3}. 3 {1, 3, 4, 7} ∪ {2, 3, 5}= {1, 2, 3, 4, 5, 7}.
2 [1, 4) ∩ [2, 7]= [2, 4). 4 [1, 4) ∪ [2, 7]= [1, 7].

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 4 / 33

Set operations

Difference: B\A
B\A = {x|x ∈ B và x ∈
/ A}

Symmetric difference: A M B
A M B = {x|x ∈ A\B or x ∈ B\A}

Example
1 {1, 3, 4, 7}\{2, 3, 5}= {1, 4, 7}. 3 {1, 3, 4, 7} M {2, 3, 5}= {1, 2, 4, 5, 7}.
2 [1, 4)\[2, 7]= [1, 2). 4 [1, 4) M [2, 7]= [1, 2) ∪ [4, 7].

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 5 / 33

Algebra of sets

Distribution Law
1 A(B + C ) = AB + AC
2 A + (BC ) = (A + B)(A + C )

De Morgan’s Rule
1 A + B = A B.
2 AB = A + B

Difference laws
1 A\(B + C ) = (A\B)(A\C )
2 A\(BC ) = (A\B) + (A\C )

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 6 / 33

Counting rules

Counting rules
1 Addition rule: A and B are 2 disjoint sets. Choose 1 element from A and B:
|A| + |B| ways to choose.
2 Multiplication rule: Choose 1 element from A, and 1 element from B: |A|.|B| ways
to choose.
3 Shepherd’s rule: ] sheep = ] legs /4:When the counting is repeated by the same
factor, we must divide by this factor.
4 Mississippi Rule: ] different permutations of a string of characters = ]
permutations if all characters are different /] permutations obtained by reversing the
positions of identical characters.

Example of the addition rule

Selecting a person from a group of 3 men and 4 women, we have 7 ways to choose.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 7 / 33

Example of the multiplication rule
1 There are 3 shirts, 4 trousers and 2 pairs of shoes. The number of possible outfits is
3.4.2 = 24.
2 The number of results obtained when tossing a coin 10 times: 210 .
3 The number of results obtained when tossing a die 3 times: 63 .
4 When designing a website, we have 4 colors, 3 fonts and 3 image placements. The
number of possible designs is 4.3.3 = 36.

An example of the shepherd’s rule

The number of ways to form a committee of 3 professors from a group of 10 professors:
10.9.8
3.2.1
= 120.

An example of the Mississippi rule:

11!
1 The number of distinct permutations of the word MISSISSIPPI. 4!4!2!
= 34650.
2 Number of ways to arrange a row of 6 identical wooden bars, including 2 blue bars,
6!
2 yellow bars and 2 red bars: 2!2!2! = 90.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 8 / 33

Choose k elements from a set of n elements
1 with replacement, order counts: nk
2 without replacement, order counts: Akn = n!
(n−k)!

3 without replacement, regardless of order: Cnk = n!

k!(n−k)!

Example
1 Form a word of 4 letters from 24 letters. Number of different words: 244 .
2 A printed circuit board has 8 different locations in which 1 component can be
placed. If 4 different components are to be placed on the board, how many designs
are possible? A48 = (8−4)!
8!
.
3 In a hospital, the operating room needs to schedule 3 knee surgeries (k) and 2 hip
surgeries (h) in a day. How many sequences are there? C52 = 2!3!
5!
.
4 A bin of 50 parts contains 3 defectives and 47 non-defective parts. A sample of 6
parts is selected without replacement. How many samples of size 6 contain 2
defective parts? C32 C47
4
.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 9 / 33

Random Experiment, Sample Space, Event

Random Experiment
An experiment is a process of observing a certain phenomenon. An trial is a single
execution of an experiment. An experiment that yields many different results when
performed under the same conditions is a random experiment.

Sample Space Ω
The set of all possible outcomes is called the sample space. Notation Ω.
Each element of Ω is called an elementary event.

Event
A subset of the sample space is called an event.
certain event: Ω.
impossible event: ∅.
random event: An event that may or may not occur: A ⊂ Ω, A 6= Ω, A 6= ∅.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 10 / 33

Example: Consider a random experiment of rolling a die
Sample space : Ω = {1, 2, 3, 4, 5, 6}.
{2} is an elementary event.
A = {1, 3, 5} ⊂ Ω is an event.

Example of recycle time of a flash

Randomly select a camera and record the recycle time of a flash
⇒ Ω = R+ = {x|x > 0} - continuous.
Suppose it is known that all recycle times are between 1.5 and 5 seconds.
⇒ Ω = {x|1.5 < x < 5} - continuous.
It is known that the recycle time has only 3 values(low, medium or
high).⇒ Ω = {low , medium, high} - discrete.
Does the camera conform to minimum recycle time specifications? ⇒ Ω = {yes, no}
- discrete.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 11 / 33

Operations of Events
Intersection AB: A and B. If AB = ∅, then A and B are called
Union A + B: A or B or AB occurs. exclusive or disjoint.
Difference A \ B: A occurs and B If E1 + E2 + · · · + Ek = Ω, then
does not occur. E1 , E2 , · · · , Ek are called collectively
Complement Ā: A does not occur. exhaustive.

Classical definition of probability for finite sample spaces

|A|
If |Ω| is finite and each possible outcome in Ω is equally likely, then P(A) = |Ω|
∈ [0, 1].

Example
Roll a die.
Ω = {1, 2, 3, 4, 5, 6}.
P(1) = P(2) = ... = P(6) = 16 . P(Ω) = 1, P(∅) = 0.
A = {1}, P(A) = 16 .
B = {1, 3, 5}: The number of dots is odd, P(B) = 36 = 21 .
C = {3, 6}: The number of dots is divisible by 3, P(C ) = 31 .
C + B = {1, 3, 5, 6}, BC = {3}.
A = {2, 3, 4, 5, 6}, P(A) = 1 − P(A)= 56 .

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 12 / 33

Example
Suppose there are 9 identical socks in a drawer in a dark room, of which 6 are orange and
3 are blue. Randomly choose 2 socks, all choices are equally likely. Calculate the
probability that 2 socks are the same color. If instead of taking 2 socks, we take 3 socks,
calculate the probability that at least 2 of them are the same color.

Number of ways to choose 2 socks out of 9 socks |Ω| = C92 .

A1 : 2 socks are both orange, A2 : 2 socks are both blue, A : 2 socks are the same color.
Because A1 and A2 are mutually exclusive:
|A1 | + |A2 | C62 + C32
P(A) = P(A1 ) + P(A2 )= = = 0.5
|Ω| C92

Example
Toss 3 fair coins. Find the probability that at least 2 faces are different.

A: at least 2 faces are different. A = {HHT , HTH, THH...}

A: 3 similar faces A = {HHH, TTT }
|A| 2 1 3
P(A) = 1 − P(A) = 1 − =1− 3 =1− =
|Ω| 2 4 4

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 13 / 33

Example
In a box there are 45 non-defective products and 5 defective products. Randomly select 3
products. Calculate the probability that there are at least 2 defective products.

A: The event that in 3 products there are at least 2 defective products.

|A| C 2 C 1 + C53
P(A) = = 5 45 3 .
|Ω| C50

Geometric definition of probability

Suppose that in each trial the outcomes have the same probability of occurring and are
represented by geometric points on the domain in the probability space Ω. Then
measure of A
P(A) = , (measure is length, area, or volume).
measure of Ω

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 14 / 33

Example
Consider the quadratic equation x 2 + ax + b = 0 with a ∈ [0, 1], b ∈ [−1, 1].
a. Calculate the probability that the above equation has 2 distinct real roots.
b. Calculate the probability that the equation has 2 identical roots.
c. Calculate the probability that the equation has 2 complex roots.

a. Consider A as the event that the

equation has two distinct real roots. A
a2
occurs when ∆ = a2 − 4b > 0 ⇔ b < 4
.
R 1 a2
SA ( + 1)da
P(A) = = 0 4 = 13
24
.
SΩ 2
b. Consider B as the event that the
equation has two identical roots. B occurs
2
when b = a4 .
SB 0
P(B) = = = 0.
SΩ 2
c. P(A + B) = 1 − P(A + B)
11
= 1 − P(A) − P(B) = 24
.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 15 / 33

Modern Probability: Axioms for Probability Definition
Let A be the set of all events. Probability is a real function
P : A → [0, 1]
satisfying the following axioms
1 0 ≤ P(A) ≤ 1, ∀A ∈ A.
2 P(Ω) = 1.
3 For every 2 disjoint events A, B:
P(A + B) = P(A) + P(B).
For every event A ∈ A, P(A) is called the probability of A.

Properties
1 0 ≤ P(A) ≤ 1 5 With the set of pairwise mutually
2 P(Ω) = 1 and P(∅) = 0. exclusive events A1 , · · · , An :
3 P(A) = 1 − P(A) P(A1 + A2 + · · · + An )
4 If A ⊂ B then P(A) ≤ P(B) = P(A1 ) + · · · + P(An )

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 16 / 33

 P(A + B) = P(A \ B) + P(B).
P(A \ B) + P(AB) = P(A).

Addition formula
Given 2 event
P(A + B) = P(A) + P(B) − P(AB)
Given 3 event

P(A + B + C ) = P(A) + P(B) + P(C ) − P(AB) − P(AC ) − P(BC ) + P(ABC )

In general,
 
P P P P
P Ai = P(Ai ) − P(Ai Aj ) + P(Ai Aj Ak ) − · · ·
i i i,j i,j,k

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 17 / 33

Example
A patient arrives at a doctor’s office with a sore throat and low-grade fever. After an
exam, the doctor decides that the patient has either a bacterial infection or a viral
infection or both. The doctor decides that there is a probability of 0.7 that the patient
has a bacterial infection and a probability of 0.4 that the person has a viral infection.
What is the probability that the patient has both infections?

A: the event that the patient has a bacterial infection, P(A) = 0.7.
B: the event that the patient has a viral infection, P(B) = 0.4.
Because the patient has at least one infection: P(A + B) = 1.
The probability that the patient has both infections:
P(AB) = P(A) + P(B) − P(A + B)= 0.7 + 0.4 − 1 = 0.1.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 18 / 33

Conditional Probability

The probability of event B when event A occurs, denoted by P(B|A) (probability of B

given A), is:
P(BA)
P(B|A) = , for P(A) > 0.
P(A)

Properties
0 ≤ P(B|A) ≤ 1.
P(B|B) = 1.
If AC = ∅, then P(A + C |B) = P(A|B) + P(C |B).
P(Ā|B) = 1 − P(A|B).

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 19 / 33

Example
Tossing a die. Calculate the probability that we have 2 dots knowing that the number of
dots is even.

B = {2, 4, 6}: the number of dots is even.

1
A = {2}, P(A) = = P(AB).
6
P(AB) 1
P(A|B) = = .
P(B) 3
Remark: Here, because all outcomes are equally likely, we can use:
P(AB) |AB|/|Ω| |AB|
P(A|B) = = =
P(B) |B|/|Ω| |B|

Example
Roll the dice A = {2, 3, 4, 6}, B = {2, 5, 6}. Calculate P(A|B).

|AB| 2
AB = {2, 6}, P(A|B) = = .
|B| 3

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 20 / 33

Example
Given that in 2 babies, there is at least 1 girl. Calculate the probability that both are girls.

Ω = {TT , TG , GT , GG }
A = {GG }: Both are girls. B: At least 1 child is a girl, B = {TG , GT , GG }.
|AB| 1
P(A|B) = = .
|B| 3

Example
Knowing that of the two children, the older one is a girl. Calculate the probability that
both are girls.

Ω = {TT , TG , GT , GG }.
A: Both are girls
B = {GG , GT }: The elder child is a girl
|AB| 1
P(A|B) = = .
|B| 2

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 21 / 33

Multiplication Rule
P(AB) = P(B|A)P(A) = P(A|B)P(B)

Example
The probability that a part produced in stage 1 of a machining operation meets
specifications is 0.9. The probability that it meets specifications in the second stage,
given that it meets specifications in the first stage, is 0.95. What is the probability that
both stages meet specifications?

P(both 2) = P(stage 2|stage 1).P(stage 1) = 0.95 × 0.9 = 0.855

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 22 / 33

Example
Choose 1 card from a deck of 52 cards.
a) Calculate the probability that the card is an Ace.
b) Knowing that the card is red. Calculate the probability that it is an Ace.

4 1
a)A: the card is A, P(A) = = .
52 13
2 1
b) B: the card is red. P(A|B) = = = P(A). A and B are 2 independent events.
26 13
Independent Events
A is independent of B (B occurring does not affect whether A occurs) if one of the
following equivalent statements holds
1 P(A|B) = P(A)
2 P(B|A) = P(B)
3 P(AB) = P(A)P(B)
The events A1 , · · · , Ak are said to be pairwise independent if for every subset Ai1 , · · · , Aij
in j subsets of these events (j = 2, 3, · · · , k), we have
P(Ai1 Ai2 · · · Aij ) = P(Ai1 ) · · · P(Aij )

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 23 / 33

Example
A polycarbonate plastic plate from a supplier is analyzed for its scratch and impact
resistance. The results from 100 disks are summarized as follows:
Impact resistance
High Low
Scratch resistance High 70 9
Low 16 5
Let A be the event that the disk has high impact resistance, and let B be the event that
the disk has high scratch resistance. Are events A and B independent?

|A| 86
P(A) = |Ω
= 100
,
|B| 79
P(B) = |Ω = 100
,
P(AB) = |AB|
|Ω
= 70
100
6= P(A) ∗ P(B).
So A and B are not independent.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 24 / 33

Example

A circuit is connected as shown. The circuit operates when current flows from left to
right. The probability that the components operate is given in the figure and it is known
that the components operate independently of each other. Calculate the probability that
the circuit operates.

Let Ai , i = 1, 2..., 6 be the events that the ith component operates, respectively. O is the
event that the circuit operates.
P(A1 ) = 0.9, P(A1 ) = 0.95, P(A3 ) = 0.9, P(A4 ) = 0.95, P(A5 ) = 0.8, P(A6 ) = 0.9.
P(O) = P ((A1 + A2 )(A3 + A4 )(A5 + A6 ))
= P(A1 + A2 )P(A3 + A4 )P(A5 + A6 ) (since the dead links operate independently)
= [1 − P(A1 .A2 )][1 − P(A3 .A4 )][1 − P(A5 .A6 )]
= (1 − 0.1 ∗ 0.05)(1 − 0.1 ∗ 0.05)(1 − 0.2 ∗ 0.1) = 0.9702.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 25 / 33

We have: P(B) = P(BA) + P(B Ā)

Total probability formula (2 events)

P(B) = P(B|A)P(A) + P(B|Ā)P(Ā)

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 26 / 33

Semiconductor pollution
Information about product defects product based on contamination during chip
manufacturing is given below. Let F and H denote the events of defective product and
highly contaminated chip. Find the probability of the product being defective.

From the table we have P(F |H) = 0.1, P(H) = 0.2, P(F |H̄) = 0.005, P(H̄) = 0.8.
Using the total probability formula, we obtain
P(F ) = P(F |H).P(H) + P(F |H̄).P(H̄) = 0.02 + 0.004 = 0.024

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 27 / 33

Example
A batch of 50 containers for frozen orange juice contains 5 that are defective. Two are
selected, at random, without replacement, from the batch. Let A and B denote the
events that the first and second container selected is defective, respectively.
a. Are A and B independent events?
b. If the sample were done with replacement, would A and B be independent?

5
a. P(A) = = 0.01.
500
4 5
P(B) = P(B|A) ∗ P(A) + P(B|Ā) ∗ P(Ā) = 499
∗ 0.01 + 499
∗ 0.99 = 0.01 6= P(B|A).
So A and B are not independent.
b. If the sample were done with replacement:
P(B) = 0.01 = P(B|A): A and B are independent.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 28 / 33

Total probability formula (multiple events)
k
P k
P
P(B) = P(BEi ) = P(B|Ei )P(Ei ),
i=1 i=1

k
S
with Ei ∩ Ej = ∅ (mutually exclusive) for all i 6= j and Ω = Ei (collectively exhaustive).
i=1

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 29 / 33

Bayes formula (2 events)
From multiplication formula:
P(AB) = P(A|B)P(B) = P(B|A)P(A),
we have
P(B|A)P(A)
P(A|B) = P(B)
, for P(B) > 0: Bayesian Formula.

Bayes formula (multiple events)

If E1 , E2 , ..., Ek are k mutually exclusive (disjoint) and collectively exhaustive events (with
a sum of Ω), then from the total probability formula:
P(B) = P(B|E1 )P(E1 ) + P(B|E2 )P(E2 ) + · · · + P(B|Ek )P(Ek ),
we have
P(B|E1 )P(E1 ) P(B|E1 )P(E1 )
P(E1 |B) = P(B)
= P(B|E1 )P(E1 )+P(B|E2 )P(E2 )+···+P(B|Ek )P(Ek )
,

for P(B) > 0: Bayesian formula.

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 30 / 33

Example
Given 2 identical bottles. In the first bottle there are 4 blue candies and 2 red candies. In
the second bottle there are 3 blue candies and 3 red candies. Suppose that we randomly
select a bottle and take a candy from it, knowing that the candy taken out is blue. Find
the probability that this candy is taken from the first bottle.

B1 : The candy is taken from the first bottle, B1 : The candy is taken from the second
1
bottle. We have: P(B1 ) = P(B1 ) = .
2
A: The candy is blue.
Probability that the candy is blue, knowing that it is taken from the first and second
2
bottles P(A|B1 ) = , P(A|B1 ) = 12
3
Probability that the candy is taken from the first bottle:
2 1 1 1 7
P(A) = P(A|B1 ).P(B1 ) + P(A|B1 ).P(B1 ) = . + . = .
3 2 2 2 12
P(A|B1 )P(B1 ) 2/3.1/2 4
Using Bayes’s formula: P(B1 |A) = = = .
P(A) 7/12 7

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 31 / 33

Example
Customers are asked to evaluate pre-production designs. According to the data, 95% of
highly successful products receive good reviews, 60% of moderately successful products
receive good reviews, and 10% of poorly successful products receive good reviews. In
addition, there are 40% of highly successful products, 35% of moderately successful
products, and 25% of poorly successful products.
a) What is the probability that a product receives good reviews?
b) If a new design receives good reviews, what is the probability that it will be a highly
successful product?
c) If a product does not receive good reviews, what is the probability that it will be a
highly successful product?

A, B, C , T : the event that the product is highly successful, moderately successful, poorly
successful, and well-reviewed.
We have: P(T |A) = 0.95, P(T |B) = 0.6,P(T |C ) = 0.1,
P(A) = 0.4, P(B) = 0.35, P(C ) = 0.25. a)
P(T ) = P(T |A).P(A) + P(T |B).P(B) + P(T |C ).P(C ) = 0.615. b)
P(T |A).P(A) 0.95 ∗ 0.4
P(A|T ) = = = 0.6179.
P(T ) 0.615
P(T̄ |A).P(A) 0.05 ∗ 0.4
c) P(A|T̄ ) = = = 0.0519.
P(T̄ ) 1 − 0.615
(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 32 / 33
 Thank you for your attention!

(Phan Thi Khanh Van) Chapter 1: Introduction to probability January 9, 2025 33 / 33