Probability

Daijoon Hyun
Sejong University
Spring 2024
Outline
• Sample space
• Events
• Counting sample points
• Probability of an event
• Additive rules
• Conditional probability
• Bayes’ rule

2
Introduction
• Probability and statistics theory
− Constitute a branch of mathematics for dealing with uncertainty

• Probability theory
− Provides a basis for the science of statistical inference from data

3
Probability
• Experiment
− Any process that generates a set of data

• Sample space (S)

− The set of all possible outcomes of a statistical experiment
− Each outcome in a sample space: element, sample point

4
Sample Space
• Example
− 1 coin tossing (H: head, T: tail)
• S = {T, H}  2 elements

− 2 coins tossing
• S = {TT, TH, HT, HH}  4 elements

− 1 coin tossing first; if outcome is H, toss it again; otherwise, roll

a die
• S = {HH, HT, T1, T2, T3, T4, T5, T6}
 8 elements

5
Sample Space
• Example
− Three items are selected at random from a manufacturing
process
• Each item is inspected and classified defective (D) or non-defective (N)
• S = {DDD, DDN, DND, DNN, NDD, NDN, NND, NNN}

6
Sample Space
• Representation of sample space
− Sample space with a large or infinite number of sample points
• Statement: S = {x| x is a city with a population over 1 million}
• Rule method: S = {(x, y)| }

7
Events
• An event, which collects outcomes of particular interest,
is a subset of the sample space S

• Example
1. One dice tossing
• S = {1, 2, 3, 4, 5, 6}
• Event A: the outcome is divisible by 3
• A=

2. Selection of three items at random (D, N)

• S = {DDD, DDN, DND, DNN, NDD, NDN, NND, NNN}
• Event B: the number of defectives is greater than 1
• B=

8
Events
• Example
− A sample space is the life in years of a certain electronic
components
•
− Event A that the component fails before the end of the fifth year
•

• Null set ( )
− S or an event contains no elements at all
• B ={x | x is an even factor of 7} =

9
Events
• Complement (A’) of an event A with respect to S is the
subset of all elements of S that are not in A
1. Select one card from am ordinary deck of 52 playing cards
• R is the event that a red card is selected
• R’ =

2. S = {book, cell phone, mp3, paper, stationery, laptop}

• A = {book, stationery, laptop, paper}
• A’ =

10
Events
• Intersection ( of two events A and B is the event
containing all elements that are common to A and B
− In tossing a dice, let A be the event that an even number occurs
− B is the event that a number greater than 3
• A= B=
• =

• Two events A and B are mutually exclusive, or disjoint, if

(
− A = {a, e, i, o, u}, B = {l, r, s, t}
• =

11
Events
• Union ( of two events A and B is the event
containing all the elements that belong to A or B or both

1. A = {a, b, c}, B = {b, c, d, e}

• A B=

2. M = {x|3 < x < 9} and N = {y|5 < y < 12}

• M N=

12
Events
• Venn diagrams
− Illustrate logical relation between sets
ex) Randomly select a card from an ordinary deck of 52 cards
• A: card is red / B: card is J, Q, or K of diamond / C: card is Ace

13
Exercises
1) Sample space
− Four students are selected at random and classified as male (M)
or female (F), list the elements of the sample space S1
− Define a second sample space S2 where the elements represent
the number of females selected

2) Events
− List the elements of the sets corresponding to the following
events:
• S = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}, A = {0, 2, 4, 6, 8}, B = {1, 3, 5, 7, 9},
C = {2, 3, 4, 5}, D = {1, 6, 7}

14
Counting Sample Points
• We can solve many probability problem by counting the
number of points
− Roll a dice twice: 36 elements in S
• Event A: double
• Event B: Sum is 4

15
Counting Sample Points
• Multiplication rule

− Elect a class president and a treasurer out of 22 members

− Roll a dice 4 times sequentially

16
Counting Sample Points
• Multiplication rule
− Form 5-letter words using 26 English alphabet characters:
• Characters can be repeated
• The words do not have to be meaningful

− Construct variable names of length 3 using a letter, a number,

and a letter (e.g., A2C):

− Assemble a computer by selecting parts:

• A chip from 2 brands, a hard disk from 4 stores, a memory from 3 stores,
and an accessory bundle from 5 stores

17
Counting Sample Points
• Permutation is an arrangement of all or part of a set of
objects
− List all possible sequence of a, b, and c:

− In general, n distinct objects can be arranged in

ways

18
Counting Sample Points
• Example
− The number of permutations that are possible by taking two
letters out of 4 letters (a, b, c, and d)
• ab, ac, ad, ba, bc, bd, ca, cb, cd, da, db, dc
• By multiplication rule, (4)(3) = 12

• In general, n distinct objects taken r at a time can be

arranged in

19
Counting Sample Points
• Example
1. Three awards (research, teaching, and service) will be given to
a class of 25 graduate students
• Each student can receive at most one award

2. A president and a treasurer are to be chosen from 50

students. How many different choices are possible?
a. There are no restrictions
b. A will serve only if he is president
c. B and C will serve together or not at all
d. D and E will not serve together?

20
Counting Sample Points
• Circular permutations
− Permutations that occur by arranging objects in a circle
− By considering one person in a fixed position and arranging the
others

A B C
= =
B C C A A B
A C B
= =
C B B A A C 21
Counting Sample Points
• Permutations with non-distinct objects

− Form three letters by using 1 a and 2 x

• Permutations of a, b, and c: {abc, acb, bac, bca, cab, cba}
• If b and c are changed to x: {axx, axx, xax, xxa, xax, xxa}
!
• {axx, xxa, xax}:
! !

− Form four letters by using 2 x and 2 y

!
• {xxyy, xyxy, yxxy, yyxx, xyyx, yxyx}:
! !

22
Counting Sample Points
• Example
− 10 players standing in a row in a college football team
• 1 freshman, 2 sophomores, 4 juniors, and 3 seniors
• Only their class level will be distinguished
1 2 2 3 3 3 3 4 4 4

23
Counting Sample Points
• Example
− Find the number of paths with minimum distance
• a: horizontal move
• b: vertical move
− Think about the number of permutations of three a and two b!!
B

A
a

24
Counting Sample Points
• Partitioning a set of n objects into r subsets (called cells)
− Intersection of every possible pair of subsets is
− Union of all subsets is the original set
− Order of the elements within a cell is not important

− Consider a set {a, e, i, o, u}

• Possible partitions into two cells, in which 1st cell contains 4 elements
and 2nd cell 1 element
{(e, i, o, u), (a)}, {(a, i, o, u), (e)} , {(a, e, o, u), (i)} , {(a, e, i, u), (o)}, {(a, e, i, o), (u)},

25
Counting Sample Points
• Extension of partitioning

• Example
− How many ways can 7 graduate students be assigned to 1 triple
and 2 double hotel rooms during a conference?
Triple Double1 Double2

1 2 3 4 5 6 7

26
Counting Sample Points
• Combination
− Ways of selecting r objects from n without regard to order
− A combination is a partition with two cells:
• P1 with r objects & P2 with (n-r) objects

27
Counting Sample Points
• Example
1. How many ways are there that a boy selects 3 from 10 arcade
games and 2 from 5 sports games?

2. How many different letter arrangements can be made from

the letters in the word STATISTICS ?

28
Probability of an Event
• Probability of an event

• Example
− Roll a dice twice: 36 elements in S
• Event A: double

• Event B: Sum is 4

29
Probability of an Event
• Example: roll a dice
− An even number is twice as likely to occur as an odd number
• If E is the event that a number less than 4 occurs on a single toss of the
die, find P(E)?

• Let A be the event that an even number turns up and let B be the event
that a number divisible by 3 occurs
• Find P(A∪B) and P(A∩B)?

30
Probability of an Event
• Probability of an event for equally likely outcomes

• Example
− In a poker hand consisting of 5 cards, find the probability of
holding 2 aces and 3 jacks

31
Additive Rule
• Additive rule

32
Additive Rule
• Additive rule for mutually exclusive events

• Example: buying a new car

− The probabilities of selecting colors: Green, White, Red, Blue
• P(G) = 0.09, P(W) = 0.15, P(R) = 0.21, P(B) = 0.23
− What is the probability that a buyer will purchase a car that
comes in one of those colors?
33
Additive Rule
• Use of complementary events

• Example
− The probabilities that an automobile mechanic will service
• P(3) = 0.12, P(4) = 0.19, P(5) = 0.28, P(6) = 0.24, P(7) = 0.10, P(8 or more)
= 0.07
− What is the probability that he will service at least 5 cars on his
next day at work?

34
Conditional Probability
• Conditional probability
− Probability problem under certain restrictions
• B is an event that a person get diabetes

− P(B|A) is “the probability that B occurs given that A occurs”

− Example: roll a dice
• The die is constructed so that the even numbers are twice as likely to
occur as the odd numbers
• Let A be an event resulting in a number greater than 3
• Let B be an event getting a perfect square (n2)
• P(B|A) =

35
Conditional Probability
• Conditional probability

• Example
− One of these individuals is to be selected at random
− What is P(M|E)?
• M: a man is chosen
• E: the one chosen is employed

36
Conditional Probability
• Example
− For a regular scheduled flight,
• Probability that it departs on time is P(D) = 0.83
• Probability that it arrives on time is P(A) = 0.82
• Probability that it departs and arrives on me is P(D∩A) = 0.78
− Find the probability that a plane
1) Arrives on time, given that it departed on time,
2) Departed on time, given that it has arrived on time
3) Arrives on time, given that it did not depart on time

37
Conditional Probability
• Independent events

• Example
− 2 cards are drawn in succession with replacement
• A: the first card is an ace
• B: the second card is a spade
• P(B|A) =
• P(B) =

38
Conditional Probability
• Product rule or multiplicative rule

• Example
− One ball is drawn from the first bag and placed unseen in the
second bag
• B1: drawing of a black ball from bag 1
• W1: drawing of a white ball from bag 1
• B2: drawing of a black ball from bag 2
− What is the probability that a ball now drawn from the second
bag is black?

39
Conditional Probability
• Independent events

• Example
− For a fire engine and an ambulance,
• Probability that the fire engine is available when needed is 0.98
• Probability that the ambulance is available when called is 0.92
− In burning building, find the probability that both the ambulance
and the fire engine will be available
• Assume that they operate independently

40
Conditional Probability
• Example
− An electrical system works if components A and B work and
either of the components C or D works
− Find the probability that
1) The entire system works
2) The component C does not work, given that the entire system works
• Assume that the four components work independently

41
Conditional Probability
• Extension of product rule

• Example
− Three cards are drawn in succession, without replacement
− Find the probability that the event A1∩A2∩A3 occurs
• A1: the first card is a red ace
• A2: the second card is a 10 or a jack
• A3: the third card is greater than 3 but less than 7

42
Exercises
• 200 adults are classified by sex and their education attained
• If a person is picked at random from this group, find the
probability that
1) the person is a male, given that the person has a secondary
education

2) the person does not have a college degree, given that the
person is a female

43
Exercises
• For a given circuit system,
1) What is the probability that the system works?
2) Given that the system works, what is the probability that the
component A is not working?
• Assume the components fail independently

44
Bayes’ Rule
• Example: image recognition
− Classify Sea bass and Salmon
• B1: Sea bass is observed
• B2: Salmon is observed
• A: skin brightness is a specific value x
− Find P(B1|A) and P(B2|A)?
• Known parameters: P(A|B1), P(A|B2), P(B1), P(B2)

45
Bayes’ Rule
• Bayes’ rule is the foundation of Bayesian inference

• Example
− 36 of those employed and 12 of those unemployed are
members of the Rotary Club
− Event A: the individual selected is a member of the Rotary Club
− P(A) =

46
Bayes’ Rule
− Tree diagram

'

47
Bayes’ Rule
• Total probability (rule of elimination)

48
 - Events
Bayes’ Rule A: the product is defective
B1: the product is made by machine B1
B2: the product is made by machine B2
• Example B3: the product is made by machine B3
− In a certain assembly plant, three machines, B1, B2,and B3, make
30%, 45%, and 25% of the products
• Defective ratio: 2% (B1), 3% (B2), and 2% (B3)
− When a product is randomly selected, what is the probability
that it is defective?

49
Bayes’ Rule
− Tree diagram

50
Bayes’ Rule
• Bayes’ rule

51
 - Events
Bayes’ Rule A: the product is defective
B1: the product is made by machine B1
B2: the product is made by machine B2
• Example B3: the product is made by machine B3
− In a certain assembly plant, three machines, B1, B2,and B3, make
30%, 45%, and 25% of the products
• Defective ratio: 2% (B1), 3% (B2), and 2% (B3)
− Suppose that a product is randomly selected and is defective
− What is the probability that this product is made by B3?

− P(B3|A) =

52
Bayes’ Rule
• Example:
− A manufacturing firm employs three analytical plans for the
development of a particular product
− Plans 1, 2, and 3 are used for 30%, 20%, and 50% of the products
− The defect rate is different for the three plans as follows:
• P(D|P1)=0.01, P(D|P2)=0.03, P(D|P3)=0.02
− If a random product was observed and found to be defective,
which plan was most likely used?

− P(P1|D) =

− P(P2|D) =

− P(P3|D) =
53
Exercises
• Three identical relay stations
− The number of malfunctions reported by each station and the
causes

− A malfunction is reported and it is found to be caused by other

human errors
− What is the probability that it came from station C?
• E: a malfunction by other human errors
• A: station A / B: station B / C: station C

54
Exercises
• How are the total number of ways to get 4 spades, 6
diamonds, 1 club, and 2 hearts?

• If the probability is 0.1 that a person will make a mistake on

his income tax return, find the probability
1) Four unrelated persons (P1, P2, P3, and P4) each make a mistake
2) P1 and P2 both make mistakes, and P3 and P4 do not make a
mistake

55
Exercises
• Rain forcast
− Events: F = rain is forcast, R = it actually rains
− The probability of raining
• P(R|F) = 0.75, P(R|F’) = 0.15
− The probability of rain forecast
• P(F) = 0.2
− Find the probability of wrong forecast using tree diagram?

56