CHAPTER 1

Introduction
 Sample Space and Events
Basic definitions
• Random experiment:
Any activity whose outcome is not known in advance
with certainty each time it is repeated.
– Flipping a coin
– Tossing a die
– Product conformity
– Number of defectives in a sample
– Demand for a product
– Life of a product
Dr. Shokri Z. Selim 2
 Sample Space and Events
Basic definitions
• Sample Space, S: The set of all possible outcomes.
– Flipping a coin
– Tossing a die
– Product conformity
– Number of defectives in a sample
– Demand for a product
– Life of a product

Dr. Shokri Z. Selim 3

 Sample Space and Events
Basic definitions
• Event, A: A subset of the sample space.
– Flipping a coin
– Tossing a die
– Product conformity
– Number of defectives in a sample
– Demand for a product
– Life of a product

Dr. Shokri Z. Selim 4

 Sets
• Set
• Subset E  F if and only if, if x  E , then x  F
• Equality: E  F if and only if E  F and F  E
• Intersection: E ∩ F is the set of elements that
belong to both E and F.

Dr. Shokri Z. Selim 5

 Sets
• Union: E ⋃ F is the set of elements that belong
to either E or F or both.

Dr. Shokri Z. Selim 6

 Sets
• A and B are Mutually Exclusive if: A ∩ B = Φ
• The Complement of E, Ec is the set of all
elements that are not in E.

Dr. Shokri Z. Selim 7

 Sets

(Ec⋂G)⋃F

Dr. Shokri Z. Selim 8

 Axioms of Probability
• Let S be the sample space of a random
experiment.
• Suppose that to each event A  S, a number
denoted by P(A), is associated with A.
• If P satisfies the following axioms, then it is
called a probability and the number P(A) is
said to be the probability of A.

Dr. Shokri Z. Selim 9

 Axioms of Probability
• P(A) ≥ 0 for any event A.
• P(S) = 1 ( S is the sample space. )
• If A ∩ B = Φ then P(A ⋃ B ) = P(A)+P(B)
• If A1, A2,…, A∞, is a set of mutually exclusive
events (Ai ∩ Aj = Φ for all i and j), then
P(A1 ⋃ A2 … A∞ ) = P(A1)+P(A2)+ … +P(A∞)

Dr. Shokri Z. Selim 10

 Elementary Properties of Probability
• P(Ac ) = 1 - P(A)
• P(Φ ) = 0
• P(A) ≤ 1
• P(A ⋃ B ) = P(A)+P(B) - P(A ∩ B )
• P(A1 ⋃ A2 … A n ) = P(A1)+P(A2)+ … +P(A n)

Dr. Shokri Z. Selim 11

 Probability of a Partitioned Event
• Suppose events A1, A2,… A n partition event A
Then
P(A ) = P(A1)+P(A2)+ … +P(A n)

Dr. Shokri Z. Selim 12

 Probability of union of events
• Consider events A, B, and C
Then
P(A ⋃ B ⋃ C ) = P(A)+P(B)+P(C)
– P(A⋂B) – P(A⋂C) –
P(B⋂C)
+ P(A⋂B ⋂C)
• Important result:
P(A ⋃ B ⋃ C ) ≤ P(A) + P(B) + P(C)
Dr. Shokri Z. Selim 13
 The set difference
• A – B = { x : x  A and x  B }
• P(A – B)?
A = (A – B) + (A∩B)

disjoint

P(A) =P(A – B) + P(A∩B)

Dr. Shokri Z. Selim 14

 P[A⋃B] = P[A]+ P[B]- P[A⋂B]

• A⋃B = (A – B) + B

disjoint

A⋃B = [ A – (A∩B) ] + B

Dr. Shokri Z. Selim 15

 Conditional Probability

P  A B
P  B | A  , P  A  0
P  A
P  A  B   P  B | A P  A 
 P  A | B  P B 
P  A  B C  D   P D | A  B C  P C | A  B  P B | A P  A

Dr. Shokri Z. Selim 16

 P[B|A] versus P[B]
Outcome Prob Outcome Prob Outcome Prob
O1 0.2 O1 0.2 O1 0.8
O2 0.3 O2 0.6 O2 0.2
O3 0.5 O3 0.2 O3 0.0
Event Prob Event Prob Event Prob
A={O1, O2} 0.5 A={O1, O2} 0.8 A={O1, O2} 1
B={O2, O3} 0.8 B={O2} 0.6 B={O2, O3} 0.2
P(A∩B) = 0.3 P(A∩B) = 0.6 P(A∩B) = 0.2
P(B|A) = 0.6 P(B|A) = 0.75 P(B|A) = 0.2
Dr. Shokri Z. Selim 17
 Independence
Events A and B are independent if any of the
following equivalent statements is true:

P(A|B) = P(A)
P(B|A) = P(B)
P(A⋂B) = P(A)P(B)

Dr. Shokri Z. Selim 18

 Exercise
Disks of polycarbonate plastic from a supplier are
analyzed for scratch and shock resistance. The results from 100 disks are
summarized as follows:

Let A denote the event that a disk has high shock resistance, and let B denote the event that a disk has
high scratch resistance. Are events A and B independent?

Dr. Shokri Z. Selim 19

 Independence

Independent events are events where knowing the outcome of one doesn't change
the probability of the other

Dr. Shokri Z. Selim 20

 Example
The circuit operates only if there is a path of
functional devices from left to right.
The probability that each device functions is
shown on the graph.
The devices fail independently.
Find the probability that the circuit operates.

Dr. Shokri Z. Selim 21

 Solution

• Let T denote the event that the top device

operates
• Let B denote the event that the bottom device
operates
Then
P( T or B) = P( T ) + P ( B ) - P( T and B)
= P( T ) + P ( B ) - P( T )P( B)
= 0.95*2 - 0.95*0.95 = 0.9975
Dr. Shokri Z. Selim 22
 Another Approach
P( T or B) = 1 – P ( T′ and B′)
= 1 – P(T′)P(B′)
= 1 – 0.05*0.05 = 0.9975

Dr. Shokri Z. Selim 23

 ?

Dr. Shokri Z. Selim 24

 Independent Events
P[A∩B] = P[A] P[B] or P[A|B] = P[A]
If one of the following pairs of events is
independent then all the others are also
independent:
{A and B }
{A and Bc}
{Ac and B }
{Ac and Bc}.

Dr. Shokri Z. Selim 25

 jointly(mutually) Independent
Events
Events A1, A2, … , An are mutually (jointly)
independent if and only if the following relations hold for
all combinations of indices where 1 ≤ i < j < k < … ≤ n

P  Ai  A j   P  Ai  P  A j 
P  Ai  A j  Ak   P  Ai  P  A j  P  Ak 

P  A1  A2   An   P  A1  P  A2  P  An 
Dr. Shokri Z. Selim 26
 Jointly independent events

Source: Wikipedia
27
 Pairwise independent but not jointly
(mutually) independent

Source: Wikipedia
28
 Independent events

E1 and E2 are independent and

E2 and E3 are independent
True or false:
E1 and E3 are independent

Dr. Shokri Z. Selim 29

 Independent events
Outcome Prob Event Prob Event Prob
O1 0.4 A={O1, O2} 0.8 P(A∩B)=P(O2) 0.4
O2 0.4 B={O2, O3} 0.5 P(A∩C)=P(O1) 0.4
O3 0.1 C={O1, O3} 0.5 P(B∩C)=P(O3) 0.1
O4 0.1
A and B are independent
A and C are independent
However B and C are dependent

Dr. Shokri Z. Selim 30

 Independent events
for E1, E2 and E3
Pr[E1⋂ E2... ⋂ En] = Pr[E1] Pr[E2] … Pr[En]

True or false:

All pairs of events are independent

Dr. Shokri Z. Selim 31

 Independent events
Outcome Prob Event Prob
O1 0.7857 A={O1, O2} 0.7957
O2 0.01 B={O2, O3} 0.11
O3 0.1 C={O2, O4} 0.1143
O4 0.1043
P(A∩B∩C) = P(O2) = 0.01 = P(A)P(B)P(C)
P(A∩B) = P(O2) = 0.01 ≠ P(A)P(B)=0.0875
P(A∩C) = P(O2) = 0.01 ≠ P(A)P(C)=0.0909
P(B∩C) = P(O2) = 0.01 ≠ P(B)P(C)=0.0126

Dr. Shokri Z. Selim 32

 Posterior Probabilities
Law of Total Probability
S  {1, 2,3, 4,5, 6}
 S  A1    An and Ai : mutually exclusive
 B  ( A1  B )    ( An  B ) and ( Ai  B ) : mutually exclusive
 P ( B )  P ( A1  B )    P ( An  B )
 P ( A1 ) P ( B | A1 )    P ( An ) P ( B | An )

Dr. Shokri Z. Selim 33

 Law of Total Probability

• Law of Total Probability

If A1 , A 2 ,..., An is a partition of a sample space, then the probability of an
event B can be obtained from the probabilities P ( Ai )
and P( B | Ai ) using the formula
P ( B )  P ( A1 ) P ( B | A1 )    P ( An ) P ( B | An )

Dr. Shokri Z. Selim 34

 Example

Machine A B C
% of total qA qB qC
production
Prob(defective) pA pB pC

P(defective part) = ?

Dr. Shokri Z. Selim 35

Example

Dr. Shokri Z. Selim 36

 Example 2

In an experiment; E1, E2 and E3 are MEE.

The experiment is repeated until E1 or E2 occurs
Find P(E1 occurs before E2 )

Let A be the event E1 occurs

B be the event E2 occurs
C be the event E3 occurs

Dr. Shokri Z. Selim 37

Example 3

Dr. Shokri Z. Selim 38

 Example 3
P ( A before B )  P ( E1 )  1  P ( E 2 )  0
 P( E3 )  P( A before B)

P ( E1 ) P ( E1 )
P ( A before B )  =
1 - P( E3 ) P( E1 )  P( E2 )

If P( E3 )  0; P( A before B )  P ( E1 )

Dr. Shokri Z. Selim 39

 Law of Total Probability
• Example: Car Warranties
If A1 , A2 , A3 , and A4 are, respectively, the events that a car is assembled in
plants I, II, III, and IV, then they provide a partition of the sample space,
and the probabilities P ( Ai ) are the supply proportions of the four plants.

B = { a claim is made }
= the claim prob. for the four individual plants

P( B)  P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )  P( A3 ) P( B | A3 )  P( A4 ) P( B | A4 )
 (0.20  0.05)  (0.24  0.11)  (0.25  0.03)  (0.31 0.08)
 0.0687

Dr. Shokri Z. Selim 40

 Thomas Bayes (1701-1761)

Concerned with integrating

“prior knowledge” into
calculations of probability

Dr. Shokri Z. Selim 41

 Bayes’ Theorem
• Information is often presented in terms of
conditional probabilities
• Conditional probability = probability of an event
(failure) given a condition ( high or low
contamination
• An outcome is generated form a random
experiment >>>>>>>
What is the probability of that a condition
was present ( high contamination) given an
outcome ( failure)
Dr. Shokri Z. Selim 42
 Bayes’ Theorem

P( A  B )
P( A | B ) 
P( B )
P ( B | A) P ( A)
 for P ( B )  0
P( B )

Dr. Shokri Z. Selim 43

 Example 2-36 page 55

What is the conditional probability a high level of

contamination was present when a failure
occurred is to be determined.
Dr. Shokri Z. Selim 44
 Bayes’ Theorem
If E1 , E2 ,..., Ek are mutually exclusive and exhaustive
and B is any event,
P( B  E1 )
P( E 1| B )  for P( B )  0
P( B )
P( B | E1 ) P( E1 )

P( B | E1 ) P( E1 )  P( B | E2 ) P( E2 )  ...  P ( B | Ek ) P( Ek )

Dr. Shokri Z. Selim 45

 Example 32
In the flu season 3 viruses are known to cause flu.
70% of the population carry virus 1,
5% of them become sick,
20% carry virus 2,
20% of them become sick,
10% carry virus 3,
80% of them become sick.
If a person is sick which virus is likely to be the
cause?

Dr. Shokri Z. Selim 46

 Solution
70% carry virus 1 5% of them become sick
20% carry virus 2 20% of them become sick
10% carry virus 3 80% of them become sick

Vi = Event person carries virus i, i=1,2,3

S = Event person is sick
P(V1) = 0.7, P(V2) = 0.2, P(V3) = 0.1
P(S|V1) = 0.05, P(S|V2) = 0.2, P(S|V3) = 0.8

Dr. Shokri Z. Selim 47

 Solution
Vi = Event person carries virus i, i=1,2,3
S = Event person is sick
P(V1) = 0.7, P(V2) = 0.2, P(V3) = 0.1
P(S|V1) = 0.05, P(S|V2) = 0.2, P(S|V3) = 0.8
P ( S V1 )
P (V 1| S ) 
P( S )
P ( S | V1 ) P (V1 )

P ( S | V1 ) P (V1 )  P ( S | V2 ) P (V2 )  P ( S | V3 ) P (V3 )

Dr. Shokri Z. Selim 48

 Solution
70% carry virus 1 5% of them become sick
20% carry virus 2 20% of them become sick
10% carry virus 3 80% of them become sick
Bayes’ formula gives:
P( virus 1 | person is sick) = 0.226
P( virus 2 | person is sick) = 0.258
P( virus 3 | person is sick) = 0.516
Is this answer intuitive?

Dr. Shokri Z. Selim 49

 Solution
70% carry virus 1 5% of them become sick
20% carry virus 2 20% of them become sick
10% carry virus 3 80% of them become sick

Even though virus 1 is more common

and virus 3 is least common, the latter
is likely to be the cause of sickness.

Dr. Shokri Z. Selim 50

 Example 33

Part is Part is not

defective defective
0.0001 0.9999
Examinations
0.99 0.05
says defective
Examinations
says not defective 0.01 0.95

If a part is claimed defective, what is the probability

that it is really defective?
Dr. Shokri Z. Selim 51
 Solution
• Let D denote the event that a part is defective.
• Let S denote the event that inspection Part is Part is not
claims part is defective. defective defective
0.0001 0.9999
• Find P(D|S) ? Examinations 0.99 0.05
says defective
Examinations
P( D  S )
P( D | S )  says not
defective
0.01 0.95
P( S )
P( S | D) P( D)

P( S | D) P( D)  P( S | D c ) P( D c )
0.99 * 0.0001

0.99 * 0.0001  0.05 * 0.9999
Dr. Shokri Z. Selim 52
 Solution Using Trees

P( S | D ) P( D )
P( D | S ) 
P( S | D ) P( D )  P( S | D c ) P( D c )
0.99 * 0.0001
  0.001976
0.99 * 0.0001  0Z. Selim
Dr. Shokri .05 * 0.9999 53
 Example 34
Printer failures are associated with hardware, software, or other problems
with probabilities 0.1, 0.6, and 0.3 respectively.

P(printer failure due to hardware problem) = 0.9

P(printer failure due to software problem) = 0.2
P(printer failure due to other problems) = 0.5

If a printer fails what is the probability it is caused by H/W?

Dr. Shokri Z. Selim 54

 Solution
Let H denote the event that the printer has a H/W problem.
Let S denote the event that the printer has a S/W problem.
Let O denote the event that the printer has other problems.
Let F denote the event that the printer fails.

P( HW  F )
P( HW | F ) 
P( F )
P( HW  F )

P( HW  F )  P( SW  F )  P(O  F )
P( F | HW ). P( HW )

( F | HW ). P ( HW )  P( F | SW ). P( SW )  P( F | O ). P(O )
P(HW) = 0.1, P(SW) = 0.6, P(O) = 0.3
P(F|HW) = 0.9, Dr.P(F|SW) = 0.7, P(F|O) = 0.5
Shokri Z. Selim 55
Solution

P(H|F)=1/4
P(S|F)=1/3
P(O|F)=5/12

Dr. Shokri Z. Selim 56

 Summary
 P( Ai ) and P( B | Ai )  P( Ai | B)  ?
 P( A1 ), , P( An ) : the prior probabilities
 P( A1 | B), , P( An | B) : the posterior probabilities
P( Ai  B ) P ( Ai ) P ( B | Ai ) P ( Ai ) P ( B | Ai )
 P ( Ai | B )    n
P( B) P( B)
 P(A ) P( B | A )
j 1
j j

• Bayes’ Theorem
If A1 , A 2 ,..., An is a partition of a sample space, then the posterior
probabilities of the event Ai conditional on an event B can be obtained
from the probabilities P ( Ai ) andP( B | Ai ) using the formula

P ( Ai ) P ( B | Ai )
P( Ai | B)  n

 P(A ) P( B | A )
j 1
j j

Dr. Shokri Z. Selim 57

 Additional Example
• Example: Car Warranties
- The prior probabilities
P( plant I )  0.20, P( plant II )  0.24
P( plant III )  0.25, P( plant IV )  0.31
- If a claim is made on the warranty of the car, how does this change these
probabilities?
P ( plant I ) P (claim | plant I ) 0.20  0.05
P ( plant I | claim)    0.146
P (claim) 0.0687
P ( plant II ) P (claim | plant II ) 0.24  0.11
P ( plant II | claim)    0.384
P (claim) 0.0687
P ( plant III ) P (claim | plant III ) 0.25  0.03
P ( plant III | claim)    0.109
P (claim) 0.0687
P ( plant IV ) P (claim | plant IV ) 0.31 0.08
P ( plant IV | claim)    0.361
P (claim) 0.0687

Dr. Shokri Z. Selim 58

 Additional Example cont.

- No claim is made on the warranty

P ( plant I ) P (no claim | plant I )

P ( plant I | no claim) 
P (no claim)
0.20  0.95
  0.204
0.9313
P ( plant II ) P (no claim | plant II )
P ( plant II | no claim) 
P (no claim)
0.24  0.89
  0.229
0.9313
P ( plant III ) P (no claim | plant III )
P ( plant III | no claim) 
P (no claim)
0.25  0.97
  0.261
0.9313
P ( plant IV ) P (no claim | plant IV )
P ( plant IV | no claim) 
P (no claim)
0.31 0.92
  0.306
0.9313
Dr. Shokri Z. Selim 59
 Compound Experiments
• Consider two random experiments Ea, and Eb.
• Sample space of Ea, Sa={ a1, a2, … am}
• Sample space of Eb, Sb={ b1, b2, … bn}
• A compound experiment is Ea×Eb.

Dr. Shokri Z. Selim 60

 Compound Experiments
• The sample space of Ea×Eb is the product set S.
• For discrete outcomes
 a1b1 a1b2  a1bn 
a b a b  a b 
 2 1 2 2 2 n
S  Sa  Sb   
     
 amb1 amb2  ambn 
• For continuous outcomes
S  Sa  Sb  ( a, b) : a  Sa and b  Sb 
Dr. Shokri Z. Selim 61
 Example

Two coins and three dice are tossed.

The compound experiment: E1×E2×E3×E4×E5
S = {H, T}×{H, T}×
{1,2,3,4,5,6}×{1,2,3,4,5,6}×{1,2,3,4,5,6}
There are 22× 63 outcomes

Dr. Shokri Z. Selim 62