CIVL 7012/8012

Basic Laws and Axioms of

Probability
Why are we studying probability and
statistics?

• How can we quantify risks of decisions

based on samples from a population?
• How should samples be selected to support
good decisions?
• How do we design an experiment so that we
obtain the information we need?
• How can we model a problem that has
‘noise’?
Random Experiment
The goal is to understand, quantify and model the
variation affecting a physical system’s behavior. The model
is used to analyze and predict the physical system’s
behavior as system inputs affect system outputs. The
predictions are verified through experimentation with the
physical system.
Random Experiment
A random experiment can result in different
outcomes every time it is repeated, even though the
experiment is always repeated in the same manner.
Random Experiment
• Ex. Call center
Sample Spaces
• The set of all possible outcomes of a random
experiment is called the sample space, S.
• S is discrete if it consists of a finite or
countable infinite set of outcomes.
• S is continuous if it contains an interval of
real numbers.
Sample Space Defined By A Tree
Diagram
Example 2-2: Messages are classified as on-time(o)
or late(l). Classify the next 3 messages.
S = {ooo, ool, olo, oll, loo, lol, llo, lll}
Example 2-1: Defining Sample Spaces
• Randomly select a camera and record the recycle time
of a flash. S = R+ = {x | x > 0}, the positive real numbers.
• Suppose it is known that all recycle times are between
1.5 and 5 seconds. Then
S = {x | 1.5 < x < 5} is continuous.
• It is known that the recycle time has only three
values(low, medium or high). Then S = {low, medium,
high} is discrete.
• Does the camera conform to minimum recycle time
specifications?
S = {yes, no} is discrete.
Basic Laws and Axioms of Probability
DEFINITIONS

• Experiment – any action or process that generates

observations (e.g. flipping a coin)
• Trial – a single instance of an experiment (one flip of the coin)
• Outcome – the observation resulting from a trial (“heads”)
• Sample Space – the set of all possible outcomes of an
experiment (“heads” or “tails”) (may be discrete or
continuous)
• Event – a collection of one or more outcomes that share some
common trait
• Mutually Exclusive Events – events (sets) that have no
outcomes in common.
• Independent Events – events whose probability of occurrence
are unrelated
• Null Set or Impossible Event – an empty set in the sample
space
Events are Sets of Outcomes
• An event (E) is a subset of the sample space of
a random experiment.
• Event combinations
– The Union of two events consists of all outcomes
that are contained in one event or the other,
denoted as E1  E2.
– The Intersection of two events consists of all
outcomes that are contained in one event and the
other, denoted as E1  E2.
– The Complement of an event is the set of outcomes
in the sample space that are not contained in the
event, denoted as E.
Example 2-4 Continuous Events
Measurements of the thickness of a part are
modeled with the sample space: S = R+.
Let E1 = {x | 10 ≤ x < 12},
Let E2 = {x | 11 < x < 15}

– Then E1  E2 = {x | 10 ≤ x < 15}

– Then E1  E2 = {x | 11 < x < 12}
– Then E1 = {x | 0 < x < 10 or x ≥ 12}
– Then E1  E2 = {x | 12 ≤ x < 15}
Example 2-3 Discrete Events
Suppose that the recycle times of two cameras are recorded. Consider
only whether or not the cameras conform to the manufacturing
specifications. We abbreviate yes and no as y and n. The sample space
is S = {yy, yn, ny, nn}.

Suppose, E1 denotes an event that at least one camera conforms to

specifications, then E1 = {yy, yn, ny}
Suppose, E2 denotes an event that no camera conforms to
specifications, then E2 = {nn}
Suppose, E3 denotes an event that at least one camera does not
conform.
then E3 = {yn, ny, nn},
– Then E1  E3 = S
– Then E1  E3 = {yn, ny}
– Then E1 = {nn}
Venn Diagrams
Events A & B contain their respective outcomes. The
shaded regions indicate the event relation of each
diagram.

Event A in sample space S.

Sec 2-1.3 Events

Mutually Exclusive Events
• Events A and B are mutually exclusive because they
share no common outcomes.
• The occurrence of one event precludes the occurrence
of the other.
• Symbolically, A  B = Ø

Sec 2-1.3
Events
Complement

A  A
“outcomes in S not contained in A”
Mutually Exclusive Events - Laws
• Commutative law (event order is unimportant):
– A  B = B  A and A  B = B  A

• Distributive law (like in algebra):

– (A  B)  C = (A  C)  (B  C)
– (A  B)  C = (A  C)  (B  C)

• Associative law (like in algebra):

– (A  B)  C = A  (B  C)
– (A  B)  C = A  (B  C)
Mutually Exclusive Events - Laws
• DeMorgan’s law:
– (A  B) = A  B The complement of the union is
the intersection of the complements.
– (A  B) = A  B The complement of the
intersection is the union of the complements.

• Complement law:
(A) = A.
Counting Techniques
• There are three special rules, or counting
techniques, used to determine the number of
outcomes in events.
• They are :
1. Multiplication rule
2. Permutation rule
3. Combination rule
• Each has its special purpose that must be applied
properly – the right tool for the right job.
Counting – Multiplication Rule
• Multiplication rule:
– Let an operation consist of k steps and there are
• n1 ways of completing step 1,
• n2 ways of completing step 2, … and
• nk ways of completing step k.
– Then, the total number of ways to perform k steps
is:
• n1 · n2 · … · nk
Example 2-5 - Web Site Design
• In the design for a website, we can choose to
use among:
– 4 colors,
– 3 fonts, and
– 3 positions for an image.
How many designs are possible?
• Answer via the multiplication rule: 4 · 3 · 3 =
36
Counting – Permutation Rule
• A permutation is a unique sequence of distinct
items.
• If S = {a, b, c}, then there are 6 permutations
– Namely: abc, acb, bac, bca, cab, cba (order
matters)
• Number of permutations for a set of n items is
n!
• n! = n·(n-1)·(n-2)·…·2·1
• 7! = 7·6·5·4·3·2·1 = 5,040 = FACT(7) in Excel
• By definition: 0! = 1
Counting–Subset Permutations and an example
• For a sequence of r items from a set of n items:
n!
P  n(n  1)(n  2)...(n  r  1) 
n

(n  r )!
r

• Example 2-6: Printed Circuit Board

• A printed circuit board has eight different locations in which a
component can be placed. If four different components are to be
placed on the board, how many designs are possible?
• Answer: Order is important, so use the permutation formula with n
= 8, r = 4.
8! 8  7  6  5  4!
P4 
8
  8  7  6  5  1,680
(8  4)! 4!
Counting - Similar Item Permutations
• Used for counting the sequences when some
items are identical.
• The number of permutations of:
n = n1 + n2 + … + nr items of which
n1, n2, …., nr are identical.
n!
is calculated as:
n1 ! n2 ! ... nr !
Example 2-7: Hospital Schedule
• In a hospital, an operating room needs to schedule three
knee surgeries and two hip surgeries in a day. The knee
surgery is denoted as k and the hip as h.
– How many sequences are there?
Since there are 2 identical hip surgeries and 3 identical knee
surgeries, then
5! 5  4  3!
  10
2!3! 2 1  3!

– What is the set of sequences?

{kkkhh, kkhkh, kkhhk, khkkh, khkhk, khhkk, hkkkh, hkkhk,
hkhkk, hhkkk}
Counting – Combination Rule
• A combination is a selection of r items from a set of n
where order does not matter.
• If S = {a, b, c}, n =3, then
– If r = 3, there is 1 combination, namely: abc
– If r = 2, there are 3 combinations, namely ab, ac, and
bc
• # of permutations ≥ # of combinations
• Since order does not matter with combinations, we
are dividing the # of permutations by r!, where r! is the
# of arrangements of r elements.
n n!
C  
n

 r  r !(n  r )!
r
Example 2-8: Sampling w/o Replacement-1

• A bin of 50 parts contains 3 defectives and 47

non-defective parts. A sample of 6 parts is
selected from the 50 without replacement. How
many samples of size 6 contain 2 defective
parts?
• First, how many ways are there for selecting 2
parts from the 3 defective parts?
3!
C 
3
 3 different ways
2!1!
2

3 = COMBIN(3,2)
Example 2-8: Sampling w/o Replacement-2
• Now, how many ways are there for selecting 4
parts from the 47 non-defective parts?
47! 47  46  45  44  43!
C 47
   178,365 different ways
4! 43! 4  3  2 1  43!
4

178,365 = COMBIN(47,4)
Example 2-8: Sampling w/o Replacement-3
• Now, how many ways are there to obtain:
– 2 from 3 defectives, and
– 4 from 47 non-defectives?
C23C447  3178,365  535,095 different ways

535,095 = COMBIN(3,2)*COMBIN(47,4)
Definition of Probability
When conducting an experiment, the probability of
obtaining a specific outcome can be defined from
its relative frequency of occurrence:

n
P(A)  lim  
N   N 

Example: coin toss

Basic Axioms of Probability
• Let S be a sample space. Then P(S) = 1. 


• For any event A, 0  P( A)  1 .

• If A and B are mutually exclusive events, then

P( A  B)  P( A)  P( B). More generally, if
A1 , A2 ,..... are mutually exclusive events, then
P( A1  A2  ....)  P( A1 )  P( A2 )  ...
Probability
• Probability is the likelihood or chance that
a particular outcome or event from a
random experiment will occur.
• Probability is a number in the [0,1] interval.
• A probability of:
– 1 means certainty
– 0 means impossibility
Types of Probability
• Subjective probability is a “degree of belief.”

Example: “There is a 50% chance that I’ll

study tonight.”

• Relative frequency probability is based on how often an

event occurs over a very large sample space.
n( A)
lim
Example: n→ ∞ n
Probability Based on Equally-Likely Outcomes

• Whenever a sample space consists of N possible

outcomes that are equally likely, the probability of
each outcome is 1/N.
• Example: In a batch of 100 diodes, 1 is laser diode.
A diode is randomly selected from the batch.
Random means each diode has an equal chance of
being selected. The probability of choosing the laser
diode is 1/100 or 0.01, because each outcome in the
sample space is equally likely.
Probability of an Event
• For a discrete sample space, the probability
of an event E, denoted by P(E), equals the
sum of the probabilities of the outcomes in E.

• The discrete sample space may be:

– A finite set of outcomes
– A countably infinite set of outcomes.
Example 2-9: Probabilities of Events
• A random experiment has a sample space
{a,b,c,d}. These outcomes are not equally-likely;
their probabilities are: 0.1, 0.3, 0.5, 0.1.
• Let Event A = {a,b}, B = {b,c,d}, and C = {d}
– P(A) = 0.1 + 0.3 = 0.4
– P(B) = 0.3 + 0.5 + 0.1 = 0.9
– P(C) = 0.1
– P(A ) = 0.6 and P(B ) = 0.1 and P(C ) = 0.9
– Since event A B = {b}, then P(AB) = 0.3
– Since event A B = {a,b,c,d}, then P(AB) = 1.0
– Since event AC = {null}, then P(AC ) = 0
Basic Axioms of Probability
 For any event A, P(A’) = 1 – P(A).

 Let  denote the empty set. Then P(  ) = 0.

 If A is an event, and A = {O1, O2, …, On}, then P(A) =

P(O1) + P(O2) +….+ P(On).

 Addition Rule (for when A and B are not mutually

exclusive):
P( A  B)  P( A)  P( B)  P( A  B)
Basic Axioms of Probability

P(A1  A2  A3 )  P(A1)  P(A2 )  P(A3 )  P(A1  A2 )  P(A1  A3 )  P(A2  A3 )  P(A1  A2  A3 )

Addition Rules
• Joint events are generated by applying
basic set operations to individual events,
specifically:
– Unions of events, A  B
– Intersections of events, A  B
– Complements of events, A
• Probabilities of joint events can often be
determined from the probabilities of the
individual events that comprise them.
Example 2-10: Semiconductor Wafers
A wafer is randomly selected from a batch that is
classified by contamination and location.
– Let H be the event of high concentrations of
contaminants. Then P(H) = 358/940.
– Let C be the event of the wafer being located at the
center of a sputtering tool. Then P(C) = 626/940.
– P(HC) = 112/940 Location of Tool
Contamination Total
Center Edge
Low 514 68 582
High 112 246 358
Total 626 314 940

– P(HC) = P(H) + P(C)  P(HC)

= (358 + 626  112)/940
This is the addition rule.
4
0

Probability of a Union
• For any two events A and B, the probability of
union is given by:
P( A  B)  P( A)  P( B)  P( A  B)

• If events A and B are mutually exclusive, then

P( A  B)   ,
and therefore:
P( A  B )  P ( A)  P ( B )
Addition Rule: 3 or More Events
P( A  B  C )  P( A)  P( B)  P(C )  P( A  B)
 P( A  C )  P( B  C )  P( A  B  C )

Note the alternating signs.

If a collection of events Ei are pairwise mutually exclusive;
that is Ei  E j   , for all i, j
k
Then : P( E1 E 2  ...  Ek )  ∑ P( Ei )
i 1
Conditional Probability
The probability of A occurring given that B has already occurred:

P A  B 
P A | B  
P B 

The probability of occurrence of the intersection of two sets:

P( A  B)  P  A | B   P  B  or P(B|A)P(A)

P  A | B   P  A “Independent events”
If two events are independent, the probability of occurrence of the
intersection reduces to:

P( A  B )  P( A)  P( B ) “The Multiplication Rule”

Conditional Probability
Conditional Probability
• P(B | A) is the probability of event B
occurring, given that event A has already
occurred.
• A communications channel has an error
rate of 1 per 1000 bits transmitted. Errors
are rare, but do tend to occur in bursts. If a
bit is in error, the probability that the next
bit is also in error is greater than 1/1000.
Conditional Probability Rule
• The conditional probability of an event B given
an event A, denoted as P(B | A), is:
P(B | A) = P(AB) / P(A) for P(A) > 0.
• From a relative frequency perspective of n
equally likely outcomes:
– P(A) = (number of outcomes in A) / n
– P(A  B) = (number of outcomes in A  B) / n
– P(B | A) = number of outcomes in A  B / number of
outcomes in A
Example 2-11
There are 4 probabilities conditioned on flaws in the
below table.

P( F )  40 400 and P( D)  28 400

P( D | F )  P( D I F ) P( F )  10
400
40
400  10
40

P  D ' | F   P  D 'I F  P  F   30
400
40
400  30
40

P  D | F '  P  D I F ' P  F '  18

400
360
400  360
18

P  D ' | F '  P  D 'I F '  P  F '   342

400
360
400  342
360

Sec 2-4 Conditional Probability

 Random Samples
• Random means each item is equally likely to be
chosen. If more than one item is sampled,
random means that every sampling outcome is
equally likely.
– 2 items are taken from S = {a,b,c} without
replacement.
– Ordered sample space: S = {ab,ac,bc,ba,ca,cb}
– Unordered sample space: S = {ab,ac,bc}
Example 2-12 : Sampling Without Enumeration
• A batch of 50 parts contains 10 made by Tool 1 and 40 made by Tool 2. If
2 parts are selected randomly*,

a) What is the probability that the 2nd part came from Tool 2, given
that the 1st part came from Tool 1?

– P(E1)= P(1st part came from Tool 1) = 10/50

– P(E2 | E1) = P(2nd part came from Tool 2 given that 1st part came from Tool 1)
= 40/49

b) What is the probability that the 1st part came from Tool 1 and the
2nd part came from Tool 2?

– P(E1∩E2) = P(1st part came from Tool 1 and 2nd part came from Tool 2)
= (10/50)∙(40/49) = 8/49

*Selected randomly implies that at each step of the sample, the items
remain in the batch are equally likely to be selected.
Multiplication Rule
• The conditional probability can be rewritten to
generalize a multiplication rule.

P(AB) = P(B|A)·P(A) = P(A|B)·P(B)

• The last expression is obtained by exchanging

the roles of A and B.
5
0

Example 2-13: Machining Stages

The probability that a part made in the 1st stage of
a machining operation meets specifications is 0.90.
The probability that it meets specifications in the
2nd stage, given that met specifications in the first
stage is 0.95.
What is the probability that both stages meet
specifications?

• Let A and B denote the events that the part has

met1st and 2nd stage specifications, respectively.
• P(AB) = P(B | A)·P(A) = 0.95·0.90 = 0.855
5
1

Two Mutually Exclusive Subsets

• A and A are mutually
exclusive.
• AB and A  B are
mutually exclusive
• B = (A  B)  (A  B)

Total Probability Rule

For any two events A and B
Total Probability Rule (Multiple Events)
• A collection of sets E1, E2, … Ek such that
E1 E2  ……  Ek = S is said to be exhaustive.
• Assume E1, E2, … Ek are k mutually exclusive and exhaustive.
Then
P( B)  P( B  E1 )  P( B  E2 )  ...  P( B  Ek )
 P( B | E1 )  P( E1 )  P( B | E2 )  P( E2 )  ...  P( B | Ek )  P( Ek )
Example 2-14: Semiconductor
Contamination
Information about product failure based on chip manufacturing
process contamination is given below. Find the probability of
failure.

Probability Level of Probability

of Failure Contamination of Level
0.1 High 0.2
0.005 Not High 0.8

Let F denote the event that the product fails.

Let H denote the event that the chip is exposed to high
contamination during manufacture. Then

− P(F | H) = 0.100 and P(H) = 0.2, so P(F  H) = 0.02

− P(F | H ) = 0.005 and P(H ) = 0.8, so P(F  H ) = 0.004
− P(F) = P(F  H) + P(F  H ) (Using Total Probability rule)
= 0.020 + 0.004 = 0.024
Example 2-15: Semiconductor Failures-1
Probability Level of Probability
of Failure Contamination of Level
Continuing the discussion 0.100 High 0.2
of contamination during 0.010 Medium 0.3
0.001 Low 0.5
chip manufacture, find the
probability of failure.
5
5

Example 2-15: Semiconductor Failures-2

• Let F denote the event that a chip fails
• Let H denote the event that a chip is exposed to
high levels of contamination
• Let M denote the event that a chip is exposed to
medium levels of contamination
• Let L denote the event that a chip is exposed to
low levels of contamination.
• Using Total Probability Rule,
P(F) = P(F | H)P(H) + P(F | M)P(M) + P(F | L)P(L)
= (0.1)(0.2) + (0.01)(0.3) + (0.001)(0.5)
= 0.0235
5
6

Event Independence
• Two events are independent if any one of the
following equivalent statements is true:
1. P(A | B) = P(A)
2. P(B | A) = P(B)
3. P(AB) = P(A)·P(B)
• This means that occurrence of one event has
no impact on the probability of occurrence of
the other event.
Example 2-16: Flaws and Functions
Table 1 provides an example of 400 parts classified by surface flaws
and as (functionally) defective. Suppose that the situation is different
and follows Table 2. Let F denote the event that the part has surface
flaws. Let D denote the event that the part is defective. The data
shows whether the events are independent.
TABLE 1 Parts Classified TABLE 2 Parts Classified (data chg'd)
Surface Flaws Surface Flaws
Defective Yes (F ) No (F' ) Total Defective Yes (F ) No (F' ) Total
Yes (D ) 10 18 28 Yes (D ) 2 18 20
No (D' ) 30 342 372 No (D' ) 38 342 380
Total 40 360 400 Total 40 360 400

P (D |F ) = 10/40 = 0.25 P (D |F ) = 2/40 = 0.05

P (D ) = 28/400 = 0.10 P (D ) = 20/400 = 0.05
not same same
Events D & F are dependent Events D & F are independent
5
8

Independence with Multiple Events

The events E1, E2, … , Ek are independent if
and only if, for any subset of these events:

P(Ei1Ei2  … ,  Eik) = P(Ei1)·P(Ei2)·…·P(Eik)

Sec 2-6 Independence
59

Example 2-17: Semiconductor Wafers

Assume the probability that a wafer contains a large particle of contamination is
0.01 and that the wafers are independent; that is, the probability that a wafer
contains a large particle does not depend on the characteristics of any of the
other wafers. If 15 wafers are analyzed, what is the probability that no large
particles are found?

Solution:

Let Ei denote the event that the ith wafer contains no large particles,
i = 1, 2, …,15.

Then , P(Ei) = 0.99.

The required probability is P(E1  E2 … E15).
From the assumption of independence,

P(E1  E2 … E15) = P(E1)·P(E2)·…·P(E15)

= (0.99)15
= 0.86.
6
0

Bayes’ Theorem
• Thomas Bayes (1702-1761) was an
English mathematician and Presbyterian
minister.
• His idea was that we observe conditional
probabilities through prior information.
• Bayes’ theorem states that,
6
3

Example 2-18
The conditional probability that a high level of contamination was present
when a failure occurred is to be determined. The information from
Example 2-14 is summarized here.

Probability Level of Probability

of Failure Contamination of Level
0.1 High 0.2
0.005 Not High 0.8
Solution:
Let F denote the event that the product fails, and let H denote the event
that the chip is exposed to high levels of contamination. The requested
probability is P(F). P( F | H )  P( H ) 0.10  0.20
P( H | F )    0.83
P( F ) 0.024
P( F )  P( F | H )  P( H )  P( F | H ' )  P( H ' )
 0.1 0.2  0.005  0.8  0.024
Bayes Theorem with Total Probability
If E1, E2, … Ek are k mutually exclusive and
exhaustive events and B is any event,

P  B | E1  P  E1 
P  E1 | B  
P  B | E1  P  E1   P  B | E2  P  E2   ...  P  B | Ek  P  Ek 

where P(B) > 0

Note : Numerator expression is always one of

the terms in the sum of the denominator.
Example 2-19: Bayesian Network
A printer manufacturer obtained the following three types of printer
failure probabilities. Hardware P(H) = 0.3, software P(S) = 0.6, and
other P(O) = 0.1. Also, P(F | H) = 0.9, P(F | S) = 0.2, and P(F | O) =
0.5.
If a failure occurs, determine if it’s most likely due to hardware,
software, or other.
P( F )  P( F | H ) P( H )  P( F | S ) P( S )  P( F | O) P (O)
 0.9(0.1)  0.2(0.6)  0.5(0.3)  0.36
P ( F | H )  P ( H ) 0.9  0.1
P( H | F )    0.250
P( F ) 0.36
P ( F | S )  P ( S ) 0.2  0.6
P( S | F )    0.333
P( F ) 0.36
P ( F | O )  P (O) 0.5  0.3
P(O | F )    0.417
P( F ) 0.36
Note that the conditionals given failure add to 1. Because P(O | F) is
largest, the most likely cause of the problem is in the other category.