Part II

Chapter 2: Probability

1
 Learning Objectives

1. Understand and describe sample spaces and events

2. Interpret probabilities and calculate probabilities of events
3. Use permutations and combinations to count outcomes
4. Calculate the probabilities of joint events
5. Interpret and calculate conditional probabilities
6. Determine independence and use independence to calculate
probabilities
7. Understand Bayes’ theorem and when to use it
8. Understand random variables

2
2-2 Interpretations and Axioms of Probability

Introduction

Relative frequency of corrupted pulses sent over a

communications channel.
3
 Interpretations of Probability

Equally Likely Outcomes

4
 2-4. Conditional Probability
Definition

P (B | A): means the probability that event B occurs given that A has
occurred.

Example:
Bag containing three white balls and two red balls:

5
Conditional Prob: Example Tree Diagram
Machines A and B make components, which are then placed on a
conveyor belt. Of those made by machine A, 93% are acceptable. Of those
made by machine B, 95% are acceptable. Machine A makes 60% of the
components and machine B makes the rest.

Find the probability that a component selected at random from the conveyor
belt is
a) made by machine A
b) made by machine A and acceptable
c) made by machine B and acceptable
d) made by machine B and unacceptable
e) acceptable
f) made by machine A given it is acceptable

6
7
 Conditional Probability (2)
Example: In a manufacturing process, 10% of parts contain visible surface
flaws (F) and 25% of these parts are defective (D). 5% of parts without surface
flaws are defective.
D: Defective
F: Surface Flaw

P(D|F) = 25%

P(D|F’) = 5%

Example: The table below shows an example of 400 parts classified by surface flaws
and functionality defective.

P(D|F) = 10/40 = 0.25

P(D|F’) = 18/360 = 0.05

8
 Conditional Probability (3)
Previous Example (continue):
𝑃𝑃(𝐹𝐹) = 40/400 and 𝑃𝑃(𝐷𝐷) = 28/400

𝑷𝑷(𝑭𝑭|𝑫𝑫) = 𝑃𝑃(𝐹𝐹∩𝐷𝐷)/𝑃𝑃(𝐷𝐷) = 𝟏𝟏𝟏𝟏/𝟐𝟐𝟐𝟐

𝑷𝑷(𝑫𝑫|𝑭𝑭) = 𝑃𝑃(𝐷𝐷∩𝐹𝐹)/𝑃𝑃(𝐹𝐹) = (10/400)/(40/400) = 𝟏𝟏𝟏𝟏/𝟒𝟒𝟒𝟒
𝑃𝑃 𝐷𝐷 = 𝑃𝑃 𝐷𝐷 ∩ 𝐹𝐹 + 𝑃𝑃 𝐷𝐷 ∩ 𝐹𝐹 ′ = 𝑃𝑃 𝐷𝐷/𝐹𝐹 𝑃𝑃 𝐹𝐹 + 𝑃𝑃(𝐷𝐷/𝐹𝐹𝐹)𝑃𝑃(𝐹𝐹𝐹)
𝑃𝑃(𝐷𝐷) = (10/40)(40/400) + (18/360)(360/400) = 28/400

9
 Compound Probability
In general if A and B are two events, the compound probability of both A and
B event occurring is given by
P(A ∩ B) = P(A) P(B | A) (I)

Note that if A and B are independent events the from Equation (I)
P(A ∩ B) = P(A)P(B) ⇒ P(B | A) = P(B)

Example: Bag containing three white balls and two red balls

Example: An unbiased die, marked 1 to 6 is rolled twice. Find the probability

of (a) rolling two sixes
(b) the second throw being a six, given that the first throw is a six
(c) getting a score of ten from the two throws
(d) throwing at least one six (answer 11/36)
(e) throwing exactly one six (answer 10/36)

10
11
 2-5. Multiplication and Total Probability
Rules
Multiplication Rule

 What is the conditional probability that a part from

tool 2 is selected randomly (without replacement),
given that a part from tool 1 is selected first?
10 parts
 Let E1 denote the event that the first part from
from tool 1
tool 1, and E2 denote the event that the second
part is from tool 2.
40 parts
P(E2|E1) = 40/49 from tool 2

12
 Multiplication and Total Probability Rules

 Let E denote the outcomes with the first part from tool 1 and the
second part from tool 2.
 Find P(E)?

P(E2|E1) = 40/49 (from previous slide)

P(E1) = 10/50

P(E) represents the outcome of selecting part of tool 1 and then followed
by the outcome of the conditional probability of selecting a part of tool 2
given that a part of tool 1 is already selected.

𝑷𝑷(𝑬𝑬) = 𝑷𝑷(𝑬𝑬𝑬𝑬) 𝑷𝑷(𝑬𝑬𝑬𝑬|𝑬𝑬𝑬𝑬) = (𝟏𝟏𝟏𝟏/𝟓𝟓𝟓𝟓) (𝟒𝟒𝟒𝟒/𝟒𝟒𝟒𝟒) = 𝟖𝟖/𝟒𝟒𝟒𝟒 = 𝑷𝑷(𝑬𝑬𝑬𝑬 ∩ 𝑬𝑬𝑬𝑬)

13
 Total Probability Rule (Multiple Events)

 A collection of sets E1, E2, … Ek such that

E1 ∪ E2 ∪ …… ∪ Ek = S is said to be exhaustive
 Assume E1, E2, … Ek are k mutually exclusive and exhaustive.
Then
P( B) = P( B ∩ E1 ) + P( B ∩ E2 ) + ... + P( B ∩ Ek )
= P( B | E1 ) ⋅ P( E1 ) + P( B | E2 ) ⋅ P( E2 ) + ... + P( B | Ek ) ⋅ P( Ek )

14
 Example 17: Semiconductor Failures-1

 Continuing the discussion of contamination during chip

manufacture, find the probability of failure
Probability Level of Probability
of Failure Contamination of Level
0.100 High 0.2
0.010 Medium 0.3
0.001 Low 0.5

15
 Example 17: Semiconductor Failures-2

 Let F denote the event that a chip fails

 Let H denote the event that a chip is exposed to high levels of
contamination
 Let M denote the event that a chip is exposed to medium levels of
contamination
 Let L denote the event that a chip is exposed to low levels of
contamination.

 Using Total Probability Rule

P(F) = P(F | H)P(H) + P(F | M)P(M) + P(F | L)P(L)
= (0.1)(0.2) + (0.01)(0.3) + (0.001)(0.5)
= 0.0235

16
 Multiplication and Total Probability Rules
Total Probability Rule:
Sometimes the probability of an event is given under each of
several conditions.

Partitioning an event into two

Partitioning an event into several
mutually exclusive subsets.
mutually exclusive subsets.
17
 Multiplication and Total Probability Rules

Total Probability Rule (two events)

18
 Multiplication and Total Probability Rules

Example:

19
 2-5 Multiplication and Total Probability
Rules

Total Probability Rule (multiple events)

20
 2-6. Independence

 The concept of independence is an important relationship

between events and it will be used throughout the course.
 Independence relationship depends on the probability model used
for the random experiment.

Example:
–The circuit fails if one of the devices fails.
–The probability that each device functions is shown in the graph.
–Assume that devices fail independently.
–What is the probability that the circuit operates?

21
 Independence
Example (cont.):
L R
 Let L and R denote the events that the left and right
devices operate, respectively. 0.8 0.9
 The probability that the circuit operates is:
𝑃𝑃 𝐿𝐿 𝑎𝑎𝑎𝑎𝑎𝑎 𝑅𝑅 = 𝑃𝑃 𝐿𝐿∩𝑅𝑅 = 𝑃𝑃 𝐿𝐿 𝑃𝑃 𝑅𝑅
𝑃𝑃 𝐿𝐿∩𝑅𝑅 = 0.8 0.9 = 0.72

T
0.95
Example: a
0.95
b
– The circuit operates if at least one device functions
– What is the probability that the circuit operates? B

22
 Independence
Example (cont.):
 Let T and B denote the events that the top and bottom devices operate,
respectively.
 The probability that the circuit operates is:
𝑃𝑃(𝑇𝑇 𝑜𝑜𝑜𝑜 𝐵𝐵) = 1 − 𝑃𝑃[(𝑇𝑇 𝑜𝑜𝑜𝑜 𝐵𝐵)’] = 1 − 𝑃𝑃[𝑇𝑇𝑇 𝑎𝑎𝑎𝑎𝑎𝑎 𝐵𝐵𝐵]
= 1 − 𝑃𝑃[𝑇𝑇𝑇] ∗ 𝑃𝑃[𝐵𝐵𝐵]

P[(T or B)’] =𝑃𝑃 𝑇𝑇 ∪ 𝐵𝐵 = probability of both of them are not functioning

𝑃𝑃 𝑇𝑇 ∪ 𝐵𝐵 = 𝑃𝑃 𝑇𝑇 ∩ 𝐵𝐵 = probability (top device not functioning) x

probability(bottom device not functioning)

𝑃𝑃 𝑇𝑇 ∪ 𝐵𝐵 = 𝑃𝑃 𝑇𝑇 ∩ 𝐵𝐵 = 𝑃𝑃 𝑇𝑇 × 𝑃𝑃 𝐵𝐵 = 1 − 𝑃𝑃 𝑇𝑇 × 1 − 𝑃𝑃 𝐵𝐵
= (1 − 0.95) (1 − 0.95) = (0.05)2
=
𝑷𝑷(𝑻𝑻 𝒐𝒐𝒐𝒐 𝑩𝑩) = 𝟏𝟏 − (𝟎𝟎. 𝟎𝟎𝟎𝟎) 𝟐𝟐 𝟎𝟎. 𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗
23
 Independence
Definition (two events)

24
 Independence

Definition (multiple events)

25
 Independence
Example:
 The circuit below operates if there is a path of functional devices from
left to right.
 What is the probability that the circuit operates?

 Partition the circuit into three

columns

 The probability that the circuit operates is equal to

(1-0.13) (1-0.052)(0.99) = 0.987

26
 2-7. Bayes’ Theorem
 Remember what we mentioned about total probability rules that
sometimes information is given in terms of conditional probability.

 But after a random experiment generates an outcome, we may be

interested in the probability that a condition was present (e.g. high
contamination) given an outcome (e.g. a semiconductor failure).

 In other words; you would like to know the chance that the failure is
due to the high contamination condition.

27
 Bayes’ Theorem (cont.)
 Thomas Bayes addressed this issue in 1700 and developed the Bayes’
theorem.
 Example: recall the example solved on slide 12:

Now we want to know

the probability of
P(H|F)?
i.e., a failure happened
and the H contamination
Condition exist.

28
 Bayes’ Theorem

𝑃𝑃 𝐹𝐹 𝐻𝐻 𝑃𝑃 𝐻𝐻
𝑃𝑃 𝐻𝐻 𝐹𝐹 = Multiplication Rule slide # 2-15
𝑃𝑃(𝐹𝐹)

P(F) was estimated on previous slide, P(F) = 0.024

0.10 × 0.20
𝑃𝑃 𝐻𝐻 𝐹𝐹 = = 0.83
0.024

29
 Bayes’ Theorem
Bayes’ Theorem

There are many conditions that could cause the outcome B (E1, E2, …, EK).
All of these conditions are mutually exclusive. What is the chance that the
current outcome has been caused by the E1 condition.

30
Example: Bayesian Network:
Bayesian networks are used on the web sites of high technology
manufacturers to allow customers to quickly diagnose problems with
products.
 A printer manufacturer obtained the following probabilities from a
database of test results.
 Printer failures are associated with three types of problems: HW, SW
and Others (such as connectors) with probabilities of 0.1, 0.6 and 0.3
respectively.
 The probability of a printer failure given HW problem is 0.9, given SW
problem is 0.3 and given any other of problem is 0.5.
 If a customer enters the manufacturer's web site to diagnose a
printer failure, what is the most likely cause of the problem??

31
Example 2.38: Bayesian Network (2)
 Let H, S, and O denote a HW , SW and other problems respectively, and let F
denote a printer failure.
𝑃𝑃 𝐹𝐹\𝐻𝐻 𝑃𝑃 𝐻𝐻
 Baye’s Theorem : 𝑃𝑃 𝐻𝐻\F =
𝑃𝑃(𝐹𝐹)
 The most likely cause of the problem is the one that corresponds to the
largest of 𝑃𝑃 𝐻𝐻\F , 𝑃𝑃 𝑆𝑆\F and 𝑃𝑃 𝑂𝑂\F
 Baye’s theorem Denominator:
𝑃𝑃 𝐹𝐹 = 𝑃𝑃 F\H 𝑃𝑃 𝐻𝐻 + 𝑃𝑃 F\S 𝑃𝑃 𝑆𝑆 + 𝑃𝑃 F\O 𝑃𝑃 𝑂𝑂
= 0.9 0.1 + 0.3 0.6 + 0.5 0.3 = 0.42
Then
𝑃𝑃 𝐹𝐹\𝐻𝐻 𝑃𝑃 𝐻𝐻 0.9 0.1
𝑃𝑃 𝐻𝐻\F = = = 0.21
𝑃𝑃 𝐹𝐹 0.42
𝑃𝑃 𝐹𝐹\S 𝑃𝑃 𝑆𝑆 0.3 0.6
𝑃𝑃 𝑆𝑆\F = = = 0.43
𝑃𝑃 𝐹𝐹 0.42
𝑃𝑃 𝐹𝐹\O 𝑃𝑃 𝑂𝑂 0.5 0.3
𝑃𝑃 𝑂𝑂\F = = = 0.36
𝑃𝑃 𝐹𝐹 0.42
 Notice: 𝑃𝑃 𝐻𝐻\F + 𝑃𝑃 𝑆𝑆\F + 𝑃𝑃 𝑂𝑂\F = 1 cause the 3 types of problems is
responsible for failure.
 Because 𝑷𝑷 𝑺𝑺\𝑭𝑭 is the largest, the most cause of the problem is the SW
category

32
 Permutations and Combinations
Another way to determine the number of outcomes of an event is to use the permutation
rules. An arrangement of n distinct objects in a specific order is called a permutation.

EXAMPLE: Art dealer has 3 paintings, say A, B, and C, to arrange in a row on a wall.
There would be 6 distinct ways to display the paintings. They are: ABC BAC CAB ACB
BCA CBA
The total number of different ways can be found using the fundamental counting rule.
There are 3 ways to select the first object, 2 ways to select the second object, and 1 way to
select the third object. Hence, there are 3 × 2 × 1 = 6 different ways to arrange three
objects in a row on a shelf.

Another way to solve this kind of problem is to use permutations. The number of
permutations of n different objects using all the objects is n! = n×(n-1)×(n-2)…2×1

EXAMPLE: In how many different ways can 6 people be arranged in a row for a
photograph?
SOLUTION: This is a permutation of 6 objects. Hence 6! = 5 × 4 × 3 × 2 = 720 ways.

33
Permutations

34
 Permutations of Subsets

Select a set of 4 marbles out of the given 8

marbles.

How many sets you can select?

marbles of different
colors
35
Permutations of Subsets: Example 2-10

Different
components
Holes

36
Permutations of Similar Objects

37
Permutations Similar Objects: Example 2-11

4!
2! 2!

d d n n 2 drilling operations then 2

notching operations
d n d n
(d d ), (n n) d n n d

n d d n

n d n d

n n d d

38
Permutations of Similar Objects: Example 2-12

9!
4! 3! 2!

One of the labels

39
 Counting–Subset Permutations and an
example

 The number of permutations of subsets of r objects taken from a set of n

𝑛𝑛!
distinct object is : 𝑃𝑃𝑟𝑟𝑛𝑛 = = 𝑛𝑛 𝑛𝑛 − 1 𝑛𝑛 − 2 … (𝑛𝑛 − 𝑟𝑟 + 1)
𝑛𝑛−𝑟𝑟 !
 Select 2 out of 4 different peppers ordered and without replacement

 What is the set of sequences?

{rg, gr, ry, yr, ro, or, go, og, gy, yg, oy, yo}

 The number of permutations of subsets of 2 objects taken from a set of 4

4!
distinct object is : 𝑃𝑃24 = = 12
4−2 !

40
 Combinations
Combination is a selection of r distinct objects from n distinct objects

Example: Suppose two letters are selected from the four letters, A, B, C, and D.
The different permutations are shown on the left and the different combinations
are shown on the right.

Permutations: Combinations:
AB BA CA DA AB BC
AC BC CB DB AC BD
AD BD CD DC AD CD

Example: There are three routes, A, B and C, joining two computers. In how
many ways can two routes can be chosen from A, B and C.

Example: Number of triangle to select 3 points from 5.

41
 Combinations (cont.)

Example:

Select a set of 5 marbles out of the given 8

marbles.
How many sets you can select?

marbles

42
Combinations: Example 2-13

8! 8!
C 8,5 = =
(8 − 5)! 5! 5! 3!

43
 Permutations and Combinations
Notice the difference between Permutations and Combinations:

 In Permutations, the elements are different, therefore the order

(or sequence) of elements is important.

 In Combinations, the elements are similar hence the order (or

sequence) is not important.

 You can look at the Combinations as a special case of

“Permutations of similar Objects” that has only two sets:
 First set is the selected (r) elements, and
 the second set is Not selected (n-r) elements.

44
 Tutorial
A computer system uses passwords that are exactly seven characters
and each character is one of the 26 letters (a-z) or 10 integers (0-9).
Uppercase letters are not used. Every character is used only once.

1. How many passwords are possible

2. If a password consists of exactly 6 letters and one number, how
many passwords are possible?
3. If a password consists of 5 letters and two numbers, how many
passwords are possible?
4. What is the probability to guess a password when you are given
all its 7 characters?
5. Answer Q1-Q4, assuming that character can be used more than
once.
 Tutorial
A computer system uses passwords that are exactly seven
characters and each character is one of the 26 letters (a-z) or
10 integers (0-9). Uppercase letters are not used.

1. How many passwords are possible

36 35 *34 *33 *32 *31 *30

this equivalent to find the permutations of 7 items taken

from a set of 36 (26+10) different items

36!
𝑃𝑃(36, 7) = = 42072307200
(36−7=29)!
47
 Tutorial
2. If a password consists of exactly 6 letters and one number,
how many passwords are possible?
a. For a particular layout (the number is the last digit)
How many password can be generated?
L L L L L L #

26! 10!
P(26,6)*P(10,1)=
26−6=20 ! 10−1=9 !
= 26 ∗ 25 ∗ 24 ∗ 23 ∗ 22 ∗ 21 ∗ 10=1.657.656.000

b. How many possible layout ? (where to put the number)

7
intuitively its 7, using the rule it is selection of 1 item out 7 : 1
=7

 Total possible passwords :

7
P(26,6)*P(10,1)=7*1657656000=11.603.592.000
1
49
 Tutorial
3. If a password consists of 5 letters and two numbers, how many
passwords are possible?
a. for a particular layout (the numbers are the last two digit)
How many password can be generated?
L L L L L # #

26! 10!
P(26,5)*P(10,2)=
26−5=21 ! 10−2=8 !
= 26 ∗ 25 ∗ 24 ∗ 23 ∗ 22 ∗ 10*9 =

b. but how many possible layout? (possible ways to place the two numbers)
it is equivalent to a selection of 2 item out 7 (combination of 2 out of 7): 72 = 21

 Total possible passwords :

7
P(26,5)*P(10,2)=14.918.904.000
2
51
 Tutorial
A computer system uses passwords that are exactly seven
characters and each character is one of the 26 letters (a-z) or 10
integers (0-9). Uppercase letters are not used.

4. What is the probability to guess a password when you are

given all its 7 characters?
There is only one chance out the number of possible passwords to
guess the passwords.
number of possible passwords is the arrangement of 7 characters
out of 7 characters, 𝑃𝑃 7,7 = 5040;
1 1 1
𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 = = =
𝑃𝑃 7,7 7! 5040
Tutorial 1

53
 Tutorial 1 (cont.)
6. The lifespans L of 2500 components were measured and the results were
recorded in the table below
Lifespan(hours) Number of components
0 ≤ L < 200 76
200 ≤ L < 300 293
300 ≤ L < 400 574
400 ≤ L < 500 1211
500 ≤ L < 600 346

a) Calculate the probability that a component has a lifespan of over 400 hours
b) Calculate the probability that a component has a life span of less than 300
Hours
c) Calculate the probability that a component that lasts for 300 hours will continue
to last for at least 500 hours.

7. Two people, A and B, rolled an unbiased die. The first to toss a 6 wins
Dhs10. Find A’s expected winnings if A goes first.

54