Chapter 1

PROBABILITY

NGUYỄN THỊ THU THỦY(1)

SCHOOL OF APPLIED MATHEMATICS AND INFORMATICS

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

HANOI – 2022

(1)
Email: [email protected]
Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 1/96 HANOI – 2022 1 / 96
CHAPTER OUTLINE

Chapter outline
1 SAMPLE SPACES AND EVENTS
2 INTERPRETATIONS AND AXIOMS OF PROBABILITY
3 ADDITION RULES
4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES
5 TOTAL PROBABILITY RULES. BAYES’ THEOREM
6 BERNOULLI TRIAL FORMULARS

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 2/96 HANOI – 2022 2 / 96

LEARNING OBJECTIVES
Learning Objectives
After careful study of this chapter you should be able to do the following:
1 Understand and describe sample spaces and events for random experiments with graphs, tables, lists, or
tree diagrams
2 Interpret probabilities and use probabilities of outcomes to calculate probabilities of events in discrete
sample spaces
3 Use permutation and combinations to count the number of outcomes in both an event and the sample
space
4 Calculate the probabilities of joint events such as unions and intersections from the probabilities of
individual events
5 Interpret and calculate conditional probabilities of events
6 Determine the independence of events and use independence to calculate probabilities
7 Use Bayes’ theorem to calculate conditional probabilities
8 Interpret and calculate probabilities of Bernoulli trial

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 3/96 HANOI – 2022 3 / 96

 1.1 SAMPLE SPACES AND EVENTS

Content
1 1.1 SAMPLE SPACES AND EVENTS
1.1.1 Random Experiments
1.1.2. Sample Space
1.1.3. Event
1.1.4 Event Relations
1.1.5 Counting Techniques
2 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY
1.2.1 Probability of an Event. Equally Likely Outcomes
1.2.2 Theoretical Probability. Empirical Probability
1.2.3 Axioms of Probability
3 1.3 ADDITION RULES
4 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES
1.4.1 Conditional Probability
1.4.2 Multiplicative Rule
1.4.3 Independence
5 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM
1.5.1 Total Probability Rule
1.5.2 Bayes’ Theorem
6 1.6 BERNOULLI TRIAL FORMULARS
Nguyễn
1.6.1ThịBernoulli
Thu Thủy (SAMI-HUST) ProSta-CHAP1
trial/Independent Trials 4/96 HANOI – 2022 4 / 96
 1.1 SAMPLE SPACES AND EVENTS 1.1.1 Random Experiments

Random Experiments
Definition 1 (Random Experiments)
An experiment that can result in different outcomes, even though it is repeated in the same manner every
time, is called a random experiment.

Example 1 (Some examples of experiment)

1 Flip a coin. Did it land with heads or tails facing up?

2 Walk to a bus stop. How long do you wait for the arrival of a bus?
3 Give a lecture. How many students are seated in the fourth row?

Example 2 (An experiment consists of the following procedure, observation, and model)

1 Procedure: Flip a coin and let it land on a table.

2 Observation: Observe which side (head or tail) faces you after the coin lands.
3 Model: Heads and tails are equally likely. The result of each flip is unrelated to the results of previous
flips.
Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 5/96 HANOI – 2022 5 / 96
 1.1 SAMPLE SPACES AND EVENTS 1.1.1 Random Experiments

Random Experiments
Note
An experiment consists of both a procedure and observations.
It is important to understand that two experiments with the same procedure but with different
observations are different experiments.

Example 3

Flip a coin three times

(a) Observe the sequence of heads and tails.
(b) Observe the number of heads.

Note
These two experiments have the same procedure: flip a coin three times.
They are different experiments because they require different observations.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 6/96 HANOI – 2022 6 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.2. Sample Space

Sample Space

Definition 2 (Sample Space)

1 An outcome of an experiment is any possible observation of that experiment.
2 The set of all possible outcomes of a random experiment is called the sample space of the
experiment. The sample space is denoted as S.

Example 4

1 The sample space in Example 1(a) is S = {H, T } where H is the outcome “observe head,” and T is
the outcome “observe tail.”
2 The sample space in Example 3(a) is

S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }.

3 The sample space in Example 3(b) is S = {0, 1, 2, 3}.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 7/96 HANOI – 2022 7 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.2. Sample Space

Sample Space

Note
In some experiments, it is helpful to list the elements of the sample space systematically by means of a tree
diagram.

Example 5

An experiment consists of flipping a coin and then flipping it a second time if a head occurs. If a tail
occurs on the first flip, then a die is tossed once. To list the elements of the sample space providing the
most information, we construct the tree diagram of Figure 1. The sample space is

S = {HH, HT, T 1, T 2, T 3, T 4, T 5, T 6}.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 8/96 HANOI – 2022 8 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.2. Sample Space

1.1.2. Sample Space

Figure 1

Figure 1: Tree diagram for Example 5

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 9/96 HANOI – 2022 9 / 96
 1.1 SAMPLE SPACES AND EVENTS 1.1.2. Sample Space

Sample Space

Note
Sample spaces with a large or infinite number of sample points are best described by a statement or rule
method.

Example 6

1 If the possible outcomes of an experiment are the set of cities in the world with a population over 1
million, our sample space is written

S = {x | x is a city with a population over 1 million},

which reads “S is the set of all x such that x is a city with a population over 1 million”. The vertical
bar is read “such that.”
2 If S is the set of all points (x, y) on the boundary or the interior of a circle of radius 2 with center at
the origin, we write the rule
S = {(x, y) | x2 + y 2 ≤ 4}.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 10/96 HANOI – 2022 10 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.2. Sample Space

Sample Space

Note
A sample space is discrete if it consists of a finite or countable infinite set of outcomes. A sample space is
continuous if it contains an interval (either finite or infinite) of real numbers.

Example 7
In Example 6(1), the choice S ∈ R+ is an example of a continuous sample space, whereas S = {0, 1, 2, 3} is
a discrete sample space.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 11/96 HANOI – 2022 11 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.3. Event

Event

Definition 3 (Event)

An event is a set of outcomes of an experiment (or a subset of a sample space).

Example 8

Suppose we roll a six-sided die and observe the number of dots on the side facing upwards. We can label
these outcomes Ei , i = 1, . . . , 6 where Ei denotes the outcome that i dots appear on the up face. The
sample space is S = {E1 , E2 , . . . , E6 }. Each subset of S is an event. Examples of events are
1 The event A = {Roll 4 or higher } = {E4 , E5 , E6 }.
2 The event B = {The roll is even} = {E2 , E4 , E6 }.
3 The event C = {The roll is the square of an integer} = {E1 , E4 }.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 12/96 HANOI – 2022 12 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.3. Event

Event

Example 9

Wait for someone to make a phone call and observe the duration of the call in minutes. An outcome x is a
nonnegative real number. The sample space is S = {x | x ≥ 0}. The event A “the phone call lasts longer
than five minutes” is A = {x | x > 5}.

Definition 4 (Simple event)

A simple event is an event that consists of exactly one outcome.

Example 10

See Example 8. List the simple events in the experiment. Event E1 : Observe a 1. Event E2 : Observe a 2.
Event E3 : Observe a 3. Event E4 : Observe a 4. Event E5 : Observe a 5. Event E6 : Observe a 6.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 13/96 HANOI – 2022 13 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Union. Intersection. Complement

Definition 5 (Union. Intersection. Complement)
1 The union of two events is the event that consists of all outcomes that are contained in either of the
two events. We denote the union as A ∪ B.
2 The intersection of two events is the event that consists of all outcomes that are contained in both of
the two events. We denote the intersection as A ∩ B.
3 The complement of an event in a sample space is the set of outcomes in the sample space that are
not in the event. We denote the complement of the event A as AC . The notation A is also used in
other literature to denote the complement.

Note
We will use a shorthand for unions and intersections of n events:
n
[ n
\
Ai = A1 ∪ A2 ∪ · · · ∪ An and Ai = A1 ∩ A2 ∩ · · · ∩ An .
i=1 i=1

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 14/96 HANOI – 2022 14 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Union. Intersection. Complement

Figure 2: Venn diagram of A ∪ B and A ∩ B

The relationship between events and the corresponding sample space can be illustrated graphically by means of
Venn diagram.

Figure 2: Venn diagram of A ∪ B and A ∩ B

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 15/96 HANOI – 2022 15 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Union. Intersection. Complement

Figure 3: Venn diagram of complement of an event

Figure 3: The complement of an event

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 16/96 HANOI – 2022 16 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Mutually Exclusive

Definition 6 (Mutually Exclusive)

1 Two events, denoted as A and B, such that

A∩B =∅

are said to be mutually exclusive.

2 A collection of events A1 , A2 , . . . , An is mutually exclusive if and only if

Ai ∩ Aj = ∅, i 6= j.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 17/96 HANOI – 2022 17 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Mutually Exclusive

Figure 4: Venn diagram of mutually exclusive events

Figure 4: Mutually exclusive events

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 18/96 HANOI – 2022 18 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Mutually Exclusive

Example 11
1 In Example 8, events A and B are not mutually exclusive, because they have one outcome in common.
2 In Example 10 the six simple events E1 , E2 , . . . , E6 form a set of all mutually exclusive outcomes of
the experiment.
3 In Figure 5, we see that events A, B, and C are all subsets of the sample space S. It is also clear that
event B is a subset of event A, event B ∩ C has no elements and hence B and C are mutually
exclusive, event A ∩ C has at least one element, event A ∩ B = B, and event A ∪ B = A.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 19/96 HANOI – 2022 19 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Mutually Exclusive
Figure 5

Figure 5: Events of the sample space S

Events B and C are mutually exclusive; event A ∩ C has at least one element; event A ∩ B = B; and event
A ∪ B = A.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 20/96 HANOI – 2022 20 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Mutually Exclusive

Property 1
Several results that follow from the foregoing definitions, which may easily be verified by means of Venn
diagrams, are as follows:

1. A ∩ ∅ = ∅. 6. ∅C = S.
2. A ∪ ∅ = A. 7. (AC )C = A.
3. A ∩ AC = ∅. 8. (A ∩ B)C = AC ∪ B C .
4. A ∪ AC = S. 9. (A ∪ B)C = AC ∩ B C .
5. S C = ∅.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 21/96 HANOI – 2022 21 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Collectively Exhaustive. Event Space

Definition 7 (Collectively Exhaustive)

A collection of events A1 , A2 , . . . , An is collectively exhaustive if and only if

A1 ∪ A2 ∪ · · · ∪ An = S.

Definition 8 (Event Space)

An event space is a collectively exhaustive, mutually exclusive set of events.

Note
The set A = {A1 , A2 , . . . , An } is an event space if and only if

A1 ∪ A2 ∪ · · · ∪ An = S, and Ai ∩ Aj = ∅, i 6= j, i, j = 1, 2, . . . , n.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 22/96 HANOI – 2022 22 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Collectively Exhaustive. Event Space

Example 12

Flip four coins, a penny, a nickel, a dime, and a quarter. Examine the coins in order (penny, then nickel,
then dime, then quarter) and observe whether each coin shows a head (H) or a tail (T ). The sample space
consists of 16 four-letter words, with each letter either H or T . For example, the outcome T T HH refers to
the penny and the nickel showing tails and the dime and quarter showing heads. There are 16 members of
the sample space.

Example 13

Continuing Example 12, let Ai = {outcomes with i heads}. Each Ai is an event containing one or more
outcomes. For example, A1 = {T T T H, T T HT, T HT T, HT T T } contains four outcomes. The set
B = {A0 , A1 , A2 , A3 , A4 } is an event space. Its members are mutually exclusive and collectively exhaustive.
It is not a sample space because it lacks the finest-grain property. Learning that an experiment produces
an event A1 tells you that one coin came up heads, but it doesn’t tell you which coin it was.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 23/96 HANOI – 2022 23 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Collectively Exhaustive. Event Space

Theorem 1

For an event space B = {B1 , B2 , . . . , Bm } and any event A in the sample space, let Ci = A ∩ Bi ,
i = 1, 2, . . . , m. For i 6= j, the events Ci and Cj are mutually exclusive and

A = C1 ∪ C2 ∪ · · · ∪ Cm (1)

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 24/96 HANOI – 2022 24 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Collectively Exhaustive. Event Space

Figure 6

Figure 6: Figure for Theorem 1

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 25/96 HANOI – 2022 25 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.4 Event Relations

Collectively Exhaustive. Event Space

Example 14

In the coin-tossing experiment of Example 12, let A equal the set of outcomes with less than three heads:

A = {T T T T, HT T T, T HT T, T T HT, T T T H, HHT T, HT HT, HT T H, T T HH, T HT H, T HHT }.

From Example 13, let Bi = {outcomes with i heads}. Since {B0 , B1 , B2 , B3 , B4 } is an event space,
Theorem 1 states that

A = (A ∩ B0 ) ∪ (A ∩ B1 ) ∪ (A ∩ B2 ) ∪ (A ∩ B3 ) ∪ (A ∩ B4 ).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 26/96 HANOI – 2022 26 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Multiplication Rule (for counting techniques)

Rule 1 (Multiplication Rule (for counting techniques))

Assume an operation can be described as a sequence of k steps, and the number of ways of completing step
1 is n1 , and the number of ways of completing step 2 is n2 for each way of completing step 1, and the
number of ways of completing step 3 is n3 for each way of completing step 2, and so forth. The total
number of ways of completing the operation is

n1 × n2 × · · · × nk (2)

Example 15

In the design of a casing for a gear housing, we can use four different types of fasteners, three different bolt
lengths, and three different bolt locations. From the multiplication rule,

4 × 3 × 3 = 36 different designs are possible.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 27/96 HANOI – 2022 27 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Permutations
Definition 9 (Permutations of n different elements – ORDER, without REPEAT, from n objects taken n
at a time)
The number of permutations of n different elements is n! where

n! = 1 × 2 × 3 · · · × n (3)

Example 16

Shuffle a deck and observe each card starting from the top. The outcome of the experiment is an ordered
sequence of the 52 cards of the deck. How many possible outcomes are there?

Solution. The procedure consists of 52 subexperiments. In each one the observation is the identity of one
card. The first subexperiment has 52 possible outcomes corresponding to the 52 cards that could be drawn.
After the first card is drawn, the second subexperiment has 51 possible outcomes corresponding to the 51
remaining cards. The total number of outcomes is
52 × 51 × · · · × 1 = 52!

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 28/96 HANOI – 2022 28 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Permutations of Subsets
Definition 10 (Permutations of subsets – ORDER, without REPEAT, from n objects taken k at a time)
The number of permutations of subsets of k elements selected from a set of n different elements is

n!
Akn = n(n − 1)(n − 2) . . . (n − k + 1) = (4)
(n − k)!

Example 17

A printed circuit board has eight different locations in which a component can be placed. If four different
components are to be placed on the board, how many different designs are possible?

Solution. Each design consists of selecting a location from the eight locations for the first component, a
location from the remaining seven for the second component, a location from the remaining six for the
third component, and a location from the remaining five for the fourth component. Therefore,
8!
A48 = 8 × 7 × 6 × 5 = = 1680 different designs are possible
(8 − 4)!

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 29/96 HANOI – 2022 29 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Permutations of Similar Objects

Definition 11 (Permutations of similar objects)
The number of permutations of n = n1 + n2 + · · · + nk objects of which n1 are of one type, n2 are of a
second type,. . . , and nk are of an kth type is

n!
(5)
n1 !n2 ! . . . nk !

Example 18

A part is labeled by printing with four thick lines, three medium lines, and two thin lines. If each ordering
of the nine lines represents a different label, how many different labels can be generated by using this
scheme?

Solution. From Equation (5), the number of possible part labels is

9!
= 1260.
4!3!2!
Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 30/96 HANOI – 2022 30 / 96
 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Combination

Definition 12 (Combination – No ORDER, without REPEAT, from n objects taken k at a time)

The number of combinations, subsets of size k that can be selected from a set of n elements, is denoted as
k

n
or Cnk and

k
n!
= Cnk = (6)
n k!(n − k)!

Note
The number of combinations and the number of permutations are related:

Akn
Cnk = (7)
k!

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 31/96 HANOI – 2022 31 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Combination

Example 19
A printed circuit board may be purchased from five suppliers. In how many ways can three suppliers be
chosen from the five?

Solution. Since it is important to know only which three have been chosen, not the order of selection, the
number of ways is
5!
C53 = = 10.
3!2!

Note
In random experiments in which items are selected from a batch, an item may or may not be replaced before the
next one is selected. This is referred to as sampling with or without replacement, respectively.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 32/96 HANOI – 2022 32 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Combination
Example 20

A bin of 50 manufactured parts contains three defective parts and 47 nondefective parts. A sample of six
parts is selected from the 50 parts without replacement. That is, each part can only be selected once and
the sample is a subset of the 50 parts. How many different samples are there of size six that contain
exactly two defective parts?

Solution. A subset containing exactly two defective parts can be formed by first choosing the two defective
parts from the three defective parts. Using Equation (6), this step can be completed in
3!
C32 = = 3 different ways.
2!1!
Then, the second step is to select the remaining four parts from the 47 acceptable parts in the bin. The
second step can be completed in
4 47!
C47 = = 178365 different ways.
4!43!
Therefore, from the multiplication rule, the number of subsets of size six that contain exactly two defective
items is
3 × 178365 = 535095 different ways.
Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 33/96 HANOI – 2022 33 / 96
 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Sampling with Replacement

Definition 13 (Sampling with Replacement – ORDER, with REPEAT, from n objects taken k at a time)

Given n distinguishable objects, there are

k
An = nk (1.11)

ways to choose with replacement an ordered sample of k objects.

Example 21

The access code to a house’s security system consists of 3 digits. Each digit can be 0 through 9. How many
different codes are available if each digit can be repeated?

Solution. Because each digit can be repeated, there are 10 choices for each of the 3 digits

10 × 10 × 10 = 103 = 1000 codes.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 34/96 HANOI – 2022 34 / 96

 1.1 SAMPLE SPACES AND EVENTS 1.1.5 Counting Techniques

Sampling with Replacement

Practice Test 1
There are 10 competitors in the 100m final at the Olympics. How many ways can the gold, silver, and
bronze medals be awarded?

Practice Test 2
There are 6 SAMI professors and 7 SEM professors taking part in an escape room. If 4 SAMI professors
and 4 SEM professors are to be chosen and divided into 4 pairs (one SAMI professor with one SEM
professor per pair), how many pairings are possible?

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 35/96 HANOI – 2022 35 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY

Content
1 1.1 SAMPLE SPACES AND EVENTS
1.1.1 Random Experiments
1.1.2. Sample Space
1.1.3. Event
1.1.4 Event Relations
1.1.5 Counting Techniques
2 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY
1.2.1 Probability of an Event. Equally Likely Outcomes
1.2.2 Theoretical Probability. Empirical Probability
1.2.3 Axioms of Probability
3 1.3 ADDITION RULES
4 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES
1.4.1 Conditional Probability
1.4.2 Multiplicative Rule
1.4.3 Independence
5 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM
1.5.1 Total Probability Rule
1.5.2 Bayes’ Theorem
6 1.6 BERNOULLI TRIAL FORMULARS
Nguyễn
1.6.1ThịBernoulli
Thu Thủy (SAMI-HUST) ProSta-CHAP1
trial/Independent Trials 36/96 HANOI – 2022 36 / 96
 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.1 Probability of an Event. Equally Likely Outcomes

Probability of an Event

Probability
To find the probability of an event E, we sum all the probabilities assigned to the sample points in E. This sum
is called the probability of E and is denoted by P (E).

Definition 14 (Probability of an Event)

The probability P (E) of an event E is a measure of our belief that the event E will occur.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 37/96 HANOI – 2022 37 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.1 Probability of an Event. Equally Likely Outcomes

Equally Likely Outcomes

Equally Likely
Whenever a sample space consists of n possible outcomes that are equally likely, the probability of each
outcome is 1/n.

Theorem 2

For an experiment with sample space S = {E1 , . . . , En } in which each outcome Ei is equally likely,

1
P (Ei ) = , i = 1, . . . , n (8)
n

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 38/96 HANOI – 2022 38 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.2 Theoretical Probability. Empirical Probability

Theoretical Probability

Definition 15 (Theoretical Probability)

If an experiment can result in any one of n different equally likely outcomes, and if exactly m of these
outcomes correspond to event A, then the probability of event A is

m
P (A) = (9)
n

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 39/96 HANOI – 2022 39 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.2 Theoretical Probability. Empirical Probability

Theoretical Probability
Example 22
A statistics class for engineers consists of 25 industrial, 10 mechanical, 10 electrical, and 8 civil engineering
students. If a person is randomly selected by the instructor to answer a question, find the probability that
the student chosen is (a) an industrial engineering major and (b) a civil engineering or an electrical
engineering major.

Solution. Denote by I, M, E, and C the students majoring in industrial, mechanical, electrical, and civil
engineering, respectively. The total number of students in the class is 53, all of whom are equally likely to be
selected.
1 Since 25 of the 53 students are majoring in industrial engineering, the probability of event I, selecting an
industrial engineering major at random, is
25
P (I) = .
53
2 Since 18 of the 53 students are civil or electrical engineering majors, it follows that
18
P (C ∪ E) = .
53

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 40/96 HANOI – 2022 40 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.2 Theoretical Probability. Empirical Probability

Theoretical Probability

Theorem 3 (Properties of Probability)

Since P (A) behaves like a relative frequency,

(a) P (A) must be a proportion lying between 0 and 1.
(b) P (A) = 0 if the event A never occurs.
(c) P (A) = 1 if the event A always occurs.
(d) If A ⊂ B, then P (A) ≤ P (B).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 41/96 HANOI – 2022 41 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.2 Theoretical Probability. Empirical Probability

Empirical Probability

Example 23
Toss n times a coin we have

Experimenters n Number of heads m Relative frequency f = m/n

Buyp-phong 4040 2048 0.5069
Pearson 12000 6019 0.5016
Pearson 24000 12012 0.5005

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 42/96 HANOI – 2022 42 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.2 Theoretical Probability. Empirical Probability

Empirical Probability

Definition 16 (Empirical Probability)

If an experiment is performed n times, then the relative frequency of a particular occurrence say, A is

Frequency
Relative frequency =
n

where the frequency is the number of times the event A occurred. If you let n, the number of repetitions of
the experiment, become larger and larger (n → ∞), you will eventually generate the entire population. In
this population, the relative frequency of the event A is defined as the probability of event A, that is

Frequency
P (A) = lim (10)
n→∞ n

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 43/96 HANOI – 2022 43 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.3 Axioms of Probability

Axioms of Probability

Property 2 (Properties of Probability)

Probability is a number that is assigned to each member of a collection of events from a random experiment
that satisfies the following properties: If S is the sample space and E is any event in a random experiment,
1 P (S) = 1.
2 0 ≤ P (E) ≤ 1.
3 For two events E1 and E2 with E1 ∩ E2 6= ∅, P (E1 ∪ E2 ) = P (E1 ) + P (E2 ).

Note
These axioms imply the following results.
1 P (∅) = 0;
2 For any event E, P (E) = 1 − P (E C );
3 If the event E1 is contained in the event E2 , P (E1 ) ≤ P (E2 ).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 44/96 HANOI – 2022 44 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.3 Axioms of Probability

Axioms of Probability

Remark 1 (Requirements for simple event probabilities)

(a) Each probability must lie between 0 and 1.

(b) The sum of the probabilities for all simple events in S equals 1.

Example 24

A die is loaded in such a way that an even number is twice as likely to occur as an odd number.
1 If E is the event that a number less than 4 occurs on a single toss of the die, find P (E).
2 Let A be the event that an even number turns up and let B be the event that a number divisible by 3
occurs. Find P (A ∪ B) and P (A ∩ B).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 45/96 HANOI – 2022 45 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.3 Axioms of Probability

Axioms of Probability

Example 24 Solution
The sample space is S = {1, 2, 3, 4, 5, 6}. We assign a probability of w to each odd number and a probability of
2w to each even number. Since the sum of the probabilities must be 1, we have 9w = 1 or w = 1/9. Hence,
probabilities of 1/9 and 2/9 are assigned to each odd and even number, respectively.
1 Therefore,
1 2 1 4
E = {1, 2, 3} and P (E) = + + = .
9 9 9 9
2 For the events A = {2, 4, 6} and B = {3, 6}, we have

A ∪ B = {2, 3, 4, 6} and A ∩ B = {6}.

By assigning a probability of 1/9 to each odd number and 2/9 to each even number, we have
7 2
P (A ∪ B) = and P (A ∩ B) = .
9 9

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 46/96 HANOI – 2022 46 / 96

 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY 1.2.3 Axioms of Probability

Axioms of Probability

Practice Test 3
If there are 5 people named A, B, C, D, and E, and they are randomly arranged in a row (with each
ordering equally likely), what is the probability that A and B are placed next to each other?

Practice Test 4
A fair pair of dice is rolled until the sum of the two numbers occurs as 7. Compute the probability that an
even number of rolls are needed.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 47/96 HANOI – 2022 47 / 96

 1.3 ADDITION RULES

Content
1 1.1 SAMPLE SPACES AND EVENTS
1.1.1 Random Experiments
1.1.2. Sample Space
1.1.3. Event
1.1.4 Event Relations
1.1.5 Counting Techniques
2 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY
1.2.1 Probability of an Event. Equally Likely Outcomes
1.2.2 Theoretical Probability. Empirical Probability
1.2.3 Axioms of Probability
3 1.3 ADDITION RULES
4 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES
1.4.1 Conditional Probability
1.4.2 Multiplicative Rule
1.4.3 Independence
5 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM
1.5.1 Total Probability Rule
1.5.2 Bayes’ Theorem
6 1.6 BERNOULLI TRIAL FORMULARS
Nguyễn
1.6.1ThịBernoulli
Thu Thủy (SAMI-HUST) ProSta-CHAP1
trial/Independent Trials 48/96 HANOI – 2022 48 / 96
 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Theorem 4 (Probability of a Union)

If A and B are two events, then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B) (11)

Note
Notice in the Venn diagram in Figure 7 that the sum P (A) + P (B) double counts the simple events that are
common to both A and B. Subtracting P (A ∩ B) gives the correct result.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 49/96 HANOI – 2022 49 / 96

 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Figure 7

Figure 7: Additive rule of probability

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 50/96 HANOI – 2022 50 / 96

 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Example 25
John is going to graduate from an industrial engineering department in a university by the end of the
semester. After being interviewed at two companies he likes, he assesses that his probability of getting an
offer from company A is 0.8, and his probability of getting an offer from company B is 0.6. If he believes
that the probability that he will get offers from both companies is 0.5, what is the probability that he will
get at least one offer from these two companies?

Solution. Using the additive rule (11), we have

P (A ∪ B) = P (A) + P (B) − P (A ∩ B) = 0.8 + 0.6 − 0.5 = 0.9.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 51/96 HANOI – 2022 51 / 96

 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Corollary 1
If A and B are mutually exclusive, then

P (A ∪ B) = P (A) + P (B) (12)

Figure 8

Figure 8: Two disjoint events

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 52/96 HANOI – 2022 52 / 96
 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Corollary 2
If A and AC are complementary events, then

P (A) + P (AC ) = 1 (13)

Remark 2
You can see from the Venn diagram in Figure 3 that A and AC are mutually exclusive and that
A ∪ AC = S, the entire sample space. It follows that

P (A) + P (AC ) = 1 and P (A) = 1 − P (AC )

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 53/96 HANOI – 2022 53 / 96

 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Example 26
If the probabilities that an automobile mechanic will service 3, 4, 5, 6, 7, or 8 or more cars on any given
workday are, respectively, 0.12, 0.19, 0.28, 0.24, 0.10, and 0.07, what is the probability that he will service
at least 5 cars on his next day at work?

Solution. Let E be the event that at least 5 cars are serviced. Now, P (E) = 1 − P (E C ), where E C is the event
that fewer than 5 cars are serviced. Since

P (E C ) = 0.12 + 0.19 = 0.31,

it follows from (13) that

P (E) = 1 − 0.31 = 0.69.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 54/96 HANOI – 2022 54 / 96

 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Theorem 5 (Three Events)

For three events A, B, and C,

P (A ∪ B ∪ C)= P (A) + P (B) + P (C)

(14)
−P (A ∩ B) − P (A ∩ C) − P (B ∩ C) + P (A ∩ B ∩ C).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 55/96 HANOI – 2022 55 / 96

 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Example 27
In a certain city, three newspapers A, B, and C are published. Suppose that 65 percent of the families in
the city subscribe to newspaper A, 35 percent of the families subscribe to newspaper B, and 35 percent of
the families subscribe to newspaper C. Suppose also that 25 percent of the families subscribe to both A
and B, 12 percent subscribe to both A and C, 15 percent subscribe to both B and C, and 8 percent
subscribe to all three newspapers A, B, and C. What percentage of the families in the city subscribe to at
least one of the three newspapers?

Solution. We have, P (A) = 0.65, P (B) = 0.35, P (C) = 0.35, P (A ∩ B) = 0.25, P (A ∩ C) = 0.12,
P (B ∩ C) = 0.15, and P (A ∩ B ∩ C) = 0.08. It follows from Theorem 5 that

P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A ∩ B) − P (A ∩ C) − P (B ∩ C) + P (A ∩ B ∩ C)

= 0.65 + 0.35 + 0.35 − 0.25 − 0.12 − 0.15 + 0.08 = 0.91.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 56/96 HANOI – 2022 56 / 96

 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Theorem 6 (m Events)

For m events A1 , A2 , · · · , Am , then

m
X  Xm X X
P Ai = P (Ai ) − P (Ai ∩ Aj ) + P (Ai ∩ Aj ∩ Ak ) − . . .
i=1 i=1 i<j i<j<k (15)
m−1
+(−1) P (A1 ∩ A2 ∩ · · · ∩ Am ).

Corollary 3

If A1 , A2 , . . . , An are mutually exclusive, then

P (A1 ∪ A2 ∪ · · · ∪ An ) = P (A1 ) + P (A2 ) + · · · + P (An ) (16)

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 57/96 HANOI – 2022 57 / 96

 1.3 ADDITION RULES

Probability of a Union/Additive Rules

Example 28
Suppose that a school band contains 10 students from the freshman class, 20 students from the sophomore
class, 30 students from the junior class, and 40 students from the senior class. If 15 students are selected at
random from the band, what is the probability that at least one students from each of the four classes?

Solution. Let F = “No one be selected from freshman class,” O = “No one be selected from sophomore class,”
J = “No one be selected from junior class,” E = “No one be selected from senior class,” p = P (At least one
from each class be selected).
p = P (F ∩ O ∩ J ∩ E) = P (F ∪ O ∪ J ∪ E) = 1 − P (F ∪ O ∪ J ∪ E).
It follows from Theorem 6 that
P (F ∪ O ∪ J ∪ E) = P (F ) + P (O) + P (J) + P (E) − P (F ∩ O) − P (F ∩ J) − P (F ∩ E) − P (O ∩ J)
− P (O ∩ E) − P (J ∩ E) + P (F ∩ O ∩ J) + P (F ∩ O ∩ E) + P (F ∩ J ∩ E) + P (O ∩ J ∩ E)
1 h 15 15 15 15 15 15 15 15 15 15
− P (F ∩ O ∩ J ∩ E) = 15 C90 + C80 + C70 + C60 − C70 − C60 − C50 − C50 − C40 − C30
C100
i
15 15 15
+ C40 + C30 + C20 + 0 − 0 = 0.79305.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 58/96 HANOI – 2022 58 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES

Content
1 1.1 SAMPLE SPACES AND EVENTS
1.1.1 Random Experiments
1.1.2. Sample Space
1.1.3. Event
1.1.4 Event Relations
1.1.5 Counting Techniques
2 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY
1.2.1 Probability of an Event. Equally Likely Outcomes
1.2.2 Theoretical Probability. Empirical Probability
1.2.3 Axioms of Probability
3 1.3 ADDITION RULES
4 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES
1.4.1 Conditional Probability
1.4.2 Multiplicative Rule
1.4.3 Independence
5 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM
1.5.1 Total Probability Rule
1.5.2 Bayes’ Theorem
6 1.6 BERNOULLI TRIAL FORMULARS
Nguyễn
1.6.1ThịBernoulli
Thu Thủy (SAMI-HUST) ProSta-CHAP1
trial/Independent Trials 59/96 HANOI – 2022 59 / 96
 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.1 Conditional Probability

Conditional Probability
Conditional Probability
The probability of an event B under the knowledge that the outcome will be in event A is denoted as

P (B|A)

and this is called the conditional probability of B given A.

Definition 17 (Conditional probability)

The conditional probability of event A, given that event B has occurred is

P (A ∩ B)
P (A|B) = , provided P (B) > 0 (17)
P (B)

The conditional probability of event B, given that event A has occurred is

P (A ∩ B)
P (B|A) = , provided P (A) > 0 (18)
P (A)
Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 60/96 HANOI – 2022 60 / 96
 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.1 Conditional Probability

Conditional Probability

Example 29

The probability that a regularly scheduled flight departs on time is P (D) = 0.83; the probability that it
arrives on time is P (A) = 0.82; and the probability that it departs and arrives on time is P (D ∩ A) = 0.78.
Find the probability that a plane (a) arrives on time, given that it departed on time, and (b) departed on
time, given that it has arrived on time.

Solution.
(a) The probability that a plane arrives on time, given that it departed on time, is

P (A ∩ D) 0.78
P (A|D) = = = 0.94.
P (D) 0.83

(b) The probability that a plane departed on time, given that it has arrived on time, is
P (A ∩ D) 0.78
P (D|A) = = = 0.95.
P (A) 0.82

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 61/96 HANOI – 2022 61 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.1 Conditional Probability

Conditional Probability

Remark 3

A conditional probability measure P (A|B) has the following properties:

(a) P (A|B) ≥ 0.
(b) P (B|B) = 1.

Example 30

Consider an experiment that consists of testing two integrated circuits that come from the same silicon
wafer, and observing in each case whether a circuit is accepted (a) or rejected (r). Consider the a priori
probability model

P (rr) = 0.01, P (ra) = 0.01, P (ar) = 0.01, P (aa) = 0.97.

Find the probability of A = “second chip rejected” and B = “first chip rejected.”

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 62/96 HANOI – 2022 62 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.1 Conditional Probability

Conditional Probability
Example 30 Solution
The sample space of the experiment is S = {rr, ra, ar, aa}. A is the union of two disjoint events (outcomes) rr
and ar. Therefore, the a priori probability that the second chip is rejected is

P (A) = P (rr) + P (ar) = 0.02.

This is also the a priori probability that the first chip is rejected:

P (B) = P (rr) + P (ra) = 0.02.

The conditional probability of the second chip being rejected given that the first chip is rejected is, by definition,
the ratio of P (A ∩ B) to P (B), where, in this example,

P (A ∩ B) = P (both rejected) = P (rr) = 0.01.

Thus
P (A ∩ B) 0.01
P (A|B) = = = 0.5.
P (B) 0.02

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 63/96 HANOI – 2022 63 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.2 Multiplicative Rule

Multiplicative Rule

Theorem 7 (Multiplicative Rule)

If in an experiment the events A and B can both occur, then

P (A ∩ B) = P (A)P (B|A), provided P (A) > 0 (19)

Note
Since the events A ∩ B and B ∩ A are equivalent, it follows from Theorem 7 that we can also write

P (A ∩ B) = P (B ∩ A) = P (B)P (A|B), provided P (B) > 0 (20)

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 64/96 HANOI – 2022 64 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.2 Multiplicative Rule

Multiplicative Rule

Example 31

One bag contains 4 white balls and 3 black balls, and a second bag contains 3 white balls and 5 black balls.
One ball is drawn from the first bag and placed unseen in the second bag. What is the probability that a
ball now drawn from the second bag is black?

Solution. Let B1 , B2 , and W1 represent, respectively, the drawing of a black ball from bag 1, a black ball from
bag 2, and a white ball from bag 1. We are interested in the union of the mutually exclusive events B1 ∩ B2 and
W1 ∩ B2 . The various possibilities and their probabilities are illustrated in Figure 9. Now

P [(B1 ∩ B2 ) + (W1 ∩ B2 )] = P [B1 ∩ B2 ] + P [W1 ∩ B2 ] = P [B1 ]P [B2 |B1 ] + P [W1 ]P [B2 |W1 ]
 3  6   4  5  38
= + = .
7 9 7 9 63

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 65/96 HANOI – 2022 65 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.2 Multiplicative Rule

Multiplicative Rule
Figure 9

Figure 9: Tree diagram for Example 31

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 66/96 HANOI – 2022 66 / 96
 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.2 Multiplicative Rule

Multiplicative Rule

Theorem 8 (Multiplicative Rule)

If, in an experiment, the events A1 , A2 , . . . , An can occur, then

P (A1 ∩ A2 ∩ · · · ∩ An ) = P (A1 )P (A2 |A1 )P (A3 |A1 A2 ) . . . P (An |A1 ∩ A2 ∩ · · · ∩ An−1 ) (21)

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 67/96 HANOI – 2022 67 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.2 Multiplicative Rule

Multiplicative Rule

Example 32
Three cards are drawn in succession, without replacement, from an ordinary deck of playing cards. Find
the probability that the event A1 ∩ A2 ∩ A3 occurs, where A1 is the event that the first card is a red ace,
A2 is the event that the second card is a 10 or a jack, and A3 is the event that the third card is greater
than 3 but less than 7.

Solution. First we define the events A1 : “the first card is a red ace,” A2 : “the second card is a 10 or a jack,” A3 :
“the third card is greater than 3 but less than 7.” Now
2 8 12
P (A1 ) = , P (A2 |A1 ) = , P (A3 |A1 A2 ) =
52 51 50
and hence, by Theorem 8,

P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 |A1 )P (A3 |A1 ∩ A2 )

 2  8  12  8
= = .
52 51 50 5525

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 68/96 HANOI – 2022 68 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.3 Independence

Independence (two events)

Definition 18 (Independence (two events))

Two events A and B are independent if and only if

P (B|A) = P (B) or P (A|B) = P (A) (22)

assuming the existences of the conditional probabilities. Otherwise, A and B are dependent.

Theorem 9 (Independence (two events))

Two events A and B are independent if and only if

P (A ∩ B) = P (A)P (B) (23)

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 69/96 HANOI – 2022 69 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.3 Independence

Independence (two events)

Example 33
Toss two coins and observe the outcome. Define these events: A “Head (H) on the first coin”, B “Tail (T )
on the second coin”. Are events A and B independent?

Solution. From previous examples, you know that S = {HH, HT, T H, T T }. Use these four simple events to
find
1 1 1
P (A) = , P (B) = , P (A ∩ B) = .
2 2 4
Since
1
P (A)P (B) = P (A ∩ B) = ,
4
and the two events must be independent.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 70/96 HANOI – 2022 70 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.3 Independence

Independence (three events)

Definition 19 (Independence (three events))

A1 , A2 , and A3 are independent if and only if

(a) A1 and A2 are independent,
(b) A2 and A3 are independent,
(c) A1 and A3 are independent,
(d) P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 )P (A3 ).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 71/96 HANOI – 2022 71 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.3 Independence

Independence (three events)

Example 34
1
In an experiment with equiprobable outcomes, the event space is S = {1, 2, 3, 4}. P (s) = for all s ∈ S.
4
Are the events A1 = {1, 3, 4}, A2 = {2, 3, 4}, and A3 = ∅ independent?

Solution. These three sets satisfy the final condition of Definition 19 because A1 ∩ A2 ∩ A3 = ∅, and

P (A1 ∩ A2 ∩ A3 ) = P (A1 )P (A2 )P (A3 ) = 0.

However, A1 and A2 are not independent because, with all outcomes equiprobable,
1 3 3
P (A1 ∩ A2 ) = P ({3, 4}) = 6= P (A1 )P (A2 ) = × .
2 4 4
Hence the three events are not dependent.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 72/96 HANOI – 2022 72 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.3 Independence

Independence (more than three events)

Definition 20 (Independence (more than three events))

If n > 3, the sets A1 , A2 , . . . , An are independent if and only if

(a) every set of n − 1 sets taken from A1 , A2 , . . . , An is independent,
(b) P (A1 ∩ A2 ∩ · · · ∩ An ) = P (A1 )P (A2 ) · · · P (An ).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 73/96 HANOI – 2022 73 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.3 Independence

Independence (more than three events)

Remark 4
This definition and Example 34 show us that when n > 3 it is a complex matter to determine whether or
not a set of n events is independent. On the other hand, if we know that a set is independent, it is a simple
matter to determine the probability of the intersection of any subset of the events. Just multiply the
probabilities of the events in the subset.

Remark 5 (What’s the difference between mutually exclusive and independent events?)
1 When two events are mutually exclusive or disjoint, they cannot both happen when the experiment is
performed. Once the event B has occurred, event A cannot occur, so that P (A|B) = 0, or vice versa.
The occurrence of event B certainly affects the probability that event A can occur. Therefore,
mutually exclusive events must be dependent.
2 When two events are mutually exclusive or disjoint,
P (A ∩ B) = 0 and P (A ∪ B) = P (A) + P (B).
3 When two events are independent,
P (A ∩ B) = P (A)P (B) and P (A ∪ B) = P (A) + P (B) − P (A)P (B).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 74/96 HANOI – 2022 74 / 96

 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES 1.4.3 Independence

Conditional Probability, Additive Rules and Multiplicative Rule

Practice Test 5
In an experiment, A, B, and C are three independent events such that P (A) = 0.6, P (B) = 0.7, and
P (C) = 0.8. Determine the probability that exactly one of these three events will occur.

Practice Test 6
Let A and B be arbitrary events and P (A) = 0.4, P (B) = 0.6, and P (A|B) = 0.5. Find the probability of
(A ∪ B)?

Practice Test 7
Suppose that n guests check their hats when they arrive at a restaurant and that these hats are returned
to them in random order when they leave. Let pn denote the probability that no guest will receive the
proper hat. What is p6 ?

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 75/96 HANOI – 2022 75 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM

Content
1 1.1 SAMPLE SPACES AND EVENTS
1.1.1 Random Experiments
1.1.2. Sample Space
1.1.3. Event
1.1.4 Event Relations
1.1.5 Counting Techniques
2 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY
1.2.1 Probability of an Event. Equally Likely Outcomes
1.2.2 Theoretical Probability. Empirical Probability
1.2.3 Axioms of Probability
3 1.3 ADDITION RULES
4 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES
1.4.1 Conditional Probability
1.4.2 Multiplicative Rule
1.4.3 Independence
5 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM
1.5.1 Total Probability Rule
1.5.2 Bayes’ Theorem
6 1.6 BERNOULLI TRIAL FORMULARS
Nguyễn
1.6.1ThịBernoulli
Thu Thủy (SAMI-HUST) ProSta-CHAP1
trial/Independent Trials 76/96 HANOI – 2022 76 / 96
 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.1 Total Probability Rule

Total Probability Rule

Theorem 10 (Total Probability Rule (two events))

For any events A and B,

P (A) = P (A ∩ B) + P (A ∩ B C ) = P (B)P (A|B) + P (B C )P (A|B C ) (24)

Theorem 11 (Total Probability Rule (multiple events))

For an event space (or partition) {B1 , B2 , . . . , Bm } with P (Bi ) > 0 for all i and an event A, the
probability of the event A can be expressed as
m
X
P (A) = P (Bi )P (A|Bi ) (25)
i=1

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 77/96 HANOI – 2022 77 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.1 Total Probability Rule

Total Probability Rule

Figure 10: Partitioning the sample space S

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 78/96 HANOI – 2022 78 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.1 Total Probability Rule

Total Probability Rule

Proof Theorem 11
Consider the Venn diagram of Figure 10. The event A is seen to be the union of the mutually exclusive events
(B1 ∩ A), (B2 ∩ A), . . . , (Bm ∩ A) that is,

A = (B1 ∩ A) ∪ (B2 ∩ A) ∪ · · · ∪ (Bm ∩ A).

Using Corollary 3 and Theorem 7, we have

m
X  Xm
P (A) = P (Bi ∩ A) = P (Bi )P (A|Bi ).
i=1 i=1

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 79/96 HANOI – 2022 79 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.1 Total Probability Rule

Total Probability Rule

Example 35

A company has three machines B1 , B2 , and B3 for making 1 kΩ resistors. It has been observed that 80%
of resistors produced by B1 are within 50 of the nominal value. Machine B2 produces 90% of resistors
within 50 of the nominal value. The percentage for machine B3 is 60%. Each hour, machine B1 produces
3000 resistors, B2 produces 4000 resistors, and B3 produces 3000 resistors. All of the resistors are mixed
together at random in one bin and packed for shipment. What is the probability that the company ships a
resistor that is within 50 of the nominal value?

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 80/96 HANOI – 2022 80 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.1 Total Probability Rule

Total Probability Rule

Example 35 Solution
Let A = “resistor is within 50 of the nominal value”. Using the resistor accuracy information to formulate a
probability model, we write

P (A|B1 ) = 0.8, P (A|B2 ) = 0.9, P (A|B3 ) = 0.6.

The production figures state that 3000 + 4000 + 3000 = 10,000 resistors per hour are produced. The fraction
from machine B1 is P (B1 ) = 3000/10000 = 0.3. Similarly, P (B2 ) = 0.4 and P (B3 ) = 0.3. Now it is a simple
matter to apply the law of total probability to find the accuracy probability for all resistors shipped by the
company:

P (A) = P (B1 )P (A|B1 ) + P (B2 )P (A|B2 ) + P (B3 )P (A|B3 )

= 0.8 × 0.3 + 0.9 × 0.4 + 0.6 × 0.3 = 0.78.

For the whole factory, 78% of resistors are within 50 of the nominal value.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 81/96 HANOI – 2022 81 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.1 Total Probability Rule

Total Probability Rule

Practice Test 8
In a certain assembly plant, three machines, B1, B2, and B3, make 30%, 45%, and 25%, respectively, of the
products. It is known from past experience that 2%, 3%, and 2% of the products made by each machine,
respectively, are defective. Now, suppose that a finished product is randomly selected. What is the
probability that it is non-defective?

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 82/96 HANOI – 2022 82 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.2 Bayes’ Theorem

Bayes’ Theorem
Bayes’ Rule
From the definition of conditional probability,

P (AB) = P (A)P (B|A) = P (A)P (B|A).

Now, considering the second and last terms in the expression above, we can write

P (B)P (A|B)
P (B|A) = for P (A) > 0 (26)
P (A)

Theorem 12 (Bayes’ Theorem)

Let {B1 , B2 , . . . , Bm } be an event space with prior probabilities P (B1 ), P (B2 ), . . . , P (Bm ). If an event A
occurs such that P (A) 6= 0, the posterior probability of Bi given A is the conditional probability

P (Bi )P (A|Bi )
P (Bi |A) = Pm , i = 1, 2, . . . , m (27)
i=1 P (Bi )P (A|Bi )

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 83/96 HANOI – 2022 83 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.2 Bayes’ Theorem

Bayes’ Theorem

Proof Theorem 12
P (A ∩ Bi )
P (Bi |A) = (by the definition of conditional probability).
P (A)
P (A ∩ Bi ) = P (Bi )P (A|Bi ) (by the multiplicative rule).
P (A) = P (B1 )P (A|B1 ) + · · · + P (Bm )P (A|Bm ) (by total probability).

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 84/96 HANOI – 2022 84 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.2 Bayes’ Theorem

Bayes’ Theorem

Example 36

In Example 35 about a shipment of resistors from the factory, we learned that: the probability that a
resistor is from machine B3 is P (B3 ) = 0.3; the probability that a resistor is acceptable, i.e., within 50 of
the nominal value, is P (A) = 0.78; given that a resistor is from machine B3 , the conditional probability
that it is acceptable is P (A|B3 ) = 0.6. What is the probability that an acceptable resistor comes from
machine B3 ?

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 85/96 HANOI – 2022 85 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.2 Bayes’ Theorem

Bayes’ Theorem

Example 36 Solution
Now we are given the event A that a resistor is within 50 of the nominal value, and we need to find P (B3 |A).
Using Bayes’ theorem, we have
P (B3 )P (A|B3 )
P (B3 |A) = .
P (A)
Since all of the quantities we need are given in the problem description, our answer is
0.3 × 0.6
P (B3 |A) = = 0.23.
0.78
Similarly we obtain P (B1 |A) = 0.31 and P (B2 |A) = 0.46. Of all resistors within 50 of the nominal value, only
23% come from machine B3 (even though this machine produces 30% of all resistors). Machine B1 produces
31% of the resistors that meet the 50 criterion and machine B2 produces 46% of them.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 86/96 HANOI – 2022 86 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.2 Bayes’ Theorem

Bayes’ Theorem

Practice Test 9
A manufacturing firm employs three analytical plans for the design and development of a particular
product. For cost reasons, all three are used at varying times. In fact, plans 1, 2, and 3 are used for 30%,
20%, and 50% of the products, respectively. The defect rate is different for the three procedures as follows:

P (D|B1 ) = 0.01, P (D|B2 ) = 0.03, P (D|B3 ) = 0.02,

where P (D|Bi ) is the probability of a defective product, given plan Bi , i = 1, 2, 3. If a random product was
observed and found to be defective, which plan was most likely used and thus responsible?

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 87/96 HANOI – 2022 87 / 96

 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM 1.5.2 Bayes’ Theorem

Bayes’ Theorem

Practice Test 10
Police plan to enforce speed limits by using radar traps at four different locations within the city limits.
The radar traps at each of the locations L1 , L2 , L3 , and L4 will be operated 20%, 30%, 20%, and 30% of
the time. If a person who is speeding on her way to work has probabilities of 0.2, 0.1, 0.3, and 0.2,
respectively, of passing through these locations, (a) what is the probability that she will receive a speeding
ticket? (b) If the person received a speeding ticket on her way to work, what is the probability that she
passed through the radar trap located at L2 ?

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 88/96 HANOI – 2022 88 / 96

 1.6 BERNOULLI TRIAL FORMULARS

Content
1 1.1 SAMPLE SPACES AND EVENTS
1.1.1 Random Experiments
1.1.2. Sample Space
1.1.3. Event
1.1.4 Event Relations
1.1.5 Counting Techniques
2 1.2 INTERPRETATIONS AND AXIOMS OF PROBABILITY
1.2.1 Probability of an Event. Equally Likely Outcomes
1.2.2 Theoretical Probability. Empirical Probability
1.2.3 Axioms of Probability
3 1.3 ADDITION RULES
4 1.4 CONDITIONAL PROBABILITY. MULTIPLICATION RULES
1.4.1 Conditional Probability
1.4.2 Multiplicative Rule
1.4.3 Independence
5 1.5 TOTAL PROBABILITY RULES. BAYES’ THEOREM
1.5.1 Total Probability Rule
1.5.2 Bayes’ Theorem
6 1.6 BERNOULLI TRIAL FORMULARS
Nguyễn
1.6.1ThịBernoulli
Thu Thủy (SAMI-HUST) ProSta-CHAP1
trial/Independent Trials 89/96 HANOI – 2022 89 / 96
 1.6 BERNOULLI TRIAL FORMULARS 1.6.1 Bernoulli trial/Independent Trials

Bernoulli trial/Independent Trials

Definition 21 (Bernoulli trial)
In the theory of probability and statistics, a Bernoulli trial (or binomial trial) is a random experiment with
exactly two possible outcomes, “success” and “failure”, in which the probability of success is the same every
time the experiment is conducted.

Example 37
Examples of Bernoulli trials include:
1 Flipping a coin. In this context, obverse “heads” conventionally denotes success and reverse “tails”
denotes failure. A fair coin has the probability of success 0.5 by definition. In this case there are
exactly two possible outcomes.
2 Rolling a die, where a six is “success” and everything else a “failure”. In this case there are six possible
outcomes, and the event is a six; the complementary event “not a six” corresponds to the other five
possible outcomes.
3 In conducting a political opinion poll, choosing a voter at random to ascertain whether that voter will
vote “yes” in an upcoming referendum.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 90/96 HANOI – 2022 90 / 96

 1.6 BERNOULLI TRIAL FORMULARS 1.6.2 Bernoulli Trial Formulas

Bernoulli Trial Formulas

Introduction (Bernoulli Trial Calculator)

We start with a simple subexperiment in which there are two outcomes: a success occurs with probability p;
otherwise, a failure occurs with probability 1 − p. The results of all trials of the subexperiment are mutually
independent. An outcome of the complete experiment is a sequence of successes and failures denoted by a
sequence of ones and zeroes. For example, 10101 . . . is an alternating sequence of successes and failures. Let A
denote the event k successes and n − k failures in n trials. To find P (A) = Pn (k), we consider an example.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 91/96 HANOI – 2022 91 / 96

 1.6 BERNOULLI TRIAL FORMULARS 1.6.2 Bernoulli Trial Formulas

Bernoulli Trial Formulas

Example 38

What is the probability P5 (3) of three successes and two failures in five Bernoulli trials with success
probability p.

Solution. To find P5 (3), we observe that the outcomes with three successes in five trials are 11100, 11010,
11001, 10110, 10101, 10011, 01110, 01101, 01011, and 00111. We note that the probability of each outcome
is a product of five probabilities, each related to one subexperiment. In outcomes with three successes,
three of the probabilities are p and the other two are 1 − p. Therefore each outcome with three successes
has probability p3 (1 − p)2 .
We know that the number of such sequences is C53 . To find P5 (3), we add up the probabilities associated
with the 10 outcomes with 3 successes, yielding

P5 (3) = (C53 )(p3 )(1 − p)5−3 .

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 92/96 HANOI – 2022 92 / 96

 1.6 BERNOULLI TRIAL FORMULARS 1.6.2 Bernoulli Trial Formulas

Bernoulli Trial Formulas

Theorem 13 (Bernoulli Trial Calculator)

The probability of k successes and n − k failures in n Becnoulli trials is

Pn (k) = Cnk pk (1 − p)n−k = Cnn−k (1 − p)n−k pk (28)

where a success occurs with probability p and a failure occurs with probability 1 − p.

Example 39

In Example 35, we found that a randomly tested resistor was acceptable with probability P (A) = 0.78. If
we randomly test 100 resistors, what is the probability of Tk , the event that k resistors test acceptable?

Solution. Testing each resistor is a Bernoulli trial with a success occurring when a resistor is acceptable.
Thus for 0 ≤ k ≤ 100,
k
P (Tk ) = P100 (k) = C100 (0.78)k (1 − 0.78)100−k .

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 93/96 HANOI – 2022 93 / 96

 1.6 BERNOULLI TRIAL FORMULARS 1.6.2 Bernoulli Trial Formulas

Bernoulli Trial Formulas

Example 40

To communicate one bit of information reliably, cellular phones transmit the same binary symbol five
times. Thus the information “zero” is transmitted as 00000 and “one” is 11111. The receiver detects the
correct information if three or more binary symbols are received correctly. What is the information error
probability P (E), if the binary symbol error probability is q = 0.1?

Solution. In this case, we have five trials corresponding to the five times the binary symbol is sent. On each
trial, a success occurs when a binary symbol is received correctly. The probability of a success is
p = 1 − q = 0.9. The error event E occurs when the number of successes is strictly less than three:

P (E) = P5 (0) + P5 (1) + P5 (2)

= C50 (0.9)0 (0.1)5 + C51 (0.9)1 (0.1)4 + C52 (0.9)2 (0.1)3 = 0.00856.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 94/96 HANOI – 2022 94 / 96

 1.6 BERNOULLI TRIAL FORMULARS 1.6.2 Bernoulli Trial Formulas

Bernoulli Trial Formulas

Remark 6
Bernoulli trial computation can only done under the following circumstances.
2 outcomes only
When there only only 2 possibile outcomes, most of the time expressed as success or failure. This can
represent many different results such as heads or tails, win or lose, go or don’t go. It can only be done
when there are exactly 2 outcomes. Most of the times it will be expressed as success or failure.
Each trial must be independent
Each trial (each time the event occurs) must be independent of each other. This means that the
events are completely independent; they do not depend on the previous trial or the trial after. A
classic example is heads or tails. Each flip is independent of all others.
Probability of success is the same for each trial
The probability of the event occurring or there being success in the desired outcome must be the
same. For example, for each flip of a coin, there is always a 50% success rate of getting a heads. In
other words, the probability of an event occurring must be the same, not different, for each trial.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 95/96 HANOI – 2022 95 / 96

 1.6 BERNOULLI TRIAL FORMULARS 1.6.2 Bernoulli Trial Formulas

Bernoulli Trial Formulas

Practice Test 11
If 10 percent of the balls in a certain box are red, and if 20 balls are selected from the box at random, with
replacement, (a) what is the probability that more than three red balls will be obtained? (b) Also find the
probability that exactly two red balls will be obtained.

Practice Test 12
If 10 percent of the balls in a certain box are red, and if 20 balls are selected from the box at random, with
replacement, (a) what is the probability that more than three red balls will be obtained? (b) Also find the
probability that exactly two red balls will be obtained.

Nguyễn Thị Thu Thủy (SAMI-HUST) ProSta-CHAP1 96/96 HANOI – 2022 96 / 96