Session 5

Basic Probability and Baye’s

Theorem
Measure of uncertainty
Methods of Assigning Probabilities

• Classical method of assigning probability

(rules and laws)
• Relative frequency of occurrence
(cumulated historical data)
• Subjective Probability (personal intuition or
reasoning)
 Classical Probability

• Number of outcomes leading

to the event divided by the P( E )  ne
total number of outcomes N
possible Where :
• Each outcome is equally likely
N  total number of outcomes
• Determined a priori -- before
performing the experiment n e
 number of outcomes in E
• Applicable to games of chance
• Objective -- everyone correctly
using the method assigns an
identical probability
 Relative Frequency Probability
• Based on historical
data
• Computed after P( E )  n
e

performing the N
experiment Where :
• Number of times an N  total number of trials
event occurred divided
by the number of trials n e
 number of outcomes
• Objective -- everyone producing E
correctly using the
method assigns an
identical probability
 Subjective Probability
• Comes from a person’s intuition or
reasoning
• Subjective -- different individuals may
(correctly) assign different numeric
probabilities to the same event
• Degree of belief
• Useful for unique (single-trial) experiments
– New product introduction
– Initial public offering of common stock
– Site selection decisions
– Sporting events
 Structure of Probability
• Experiment
• Event
• Elementary Events
• Sample Space
• Unions and Intersections
• Mutually Exclusive Events
• Independent Events
• Collectively Exhaustive Events
• Complementary Events
 Experiment
• Experiment: a process that produces outcomes
– More than one possible outcome
– Only one outcome per trial
• Trial: one repetition of the process
• Elementary Event: cannot be decomposed or
broken down into other events
• Event: an outcome of an experiment
– May be an elementary event, or
– May be an aggregate of elementary events
– Usually represented by an uppercase letter, e.g.,
A, E1
 Sample Space
• The set of all elementary events for an
experiment
• Methods for describing a sample space
– roster or listing
– tree diagram
– Venn diagram
 Union of Sets
• The union of two sets contains an instance
of each element of the two sets.
X  1,4,7,9 X Y
Y  2,3,4,5,6
X  Y  1,2,3,4,5,6,7,9
C   IBM , DEC , Apple XY
F   Apple, Grape, Lime
C  F   IBM , DEC , Apple, Grape, Lime
 Intersection of Sets
• The intersection of two sets contains only
those element common to the two sets.

X  1,4,7,9 X Y
Y  2,3,4,5,6
X  Y   4

C  IBM , DEC , Apple  XY

F  Apple , Grape , Lime 
C  F  Apple 
 Mutually Exclusive Events
• Events with no
common outcomes P( X  Y )  0
• Occurrence of one
event precludes the
occurrence of the
other event X Y

C   IBM , DEC , Apple X  1,7,9

F   Grape, Lime Y  2,3,4,5,6
CF   X Y   
 Independent Events
• Occurrence of one event does not affect the
occurrence or nonoccurrence of the other
event
• The conditional probability of X given Y is
equal to the marginal probability of X.
• The conditional probability of Y given X is
equal to the marginal probability of Y.
P( X| Y )  P( X ) and P(Y | X )  P(Y )
 Complementary Events

• All elementary events not in the event ‘A’

are in its complementary event.

P(SampleSpace)  1
Sample
Space
A A P( A)  1  P( A)
 Four Types of Probability
• Marginal Probability
• Union Probability
• Joint Probability
• Conditional Probability
 Four Types of Probability

Marginal Union Joint Conditional

P( X ) P( X  Y ) P( X  Y ) P( X| Y )
The probability The probability The probability The probability
of X occurring of X or Y of X and Y of X occurring
occurring occurring given that Y
has occurred

X X Y X Y
Y
 General Law of Addition

P( X  Y )  P( X )  P(Y )  P( X  Y )

X Y
 Special Law of Addition

If X and Y are mutually exclusive,

P( X  Y )  P( X )  P(Y )

Y
X
 Computing Conditional
Probabilities
• A conditional probability is the probability of
one event, given that another event has
occurred:
P(A and B) The conditional
P(A | B)  probability of A given
P(B) that B has occurred

P(A and B) The conditional

P(B | A)  probability of B given
P(A) that A has occurred

Where P(A and B) = joint probability of A and B

P(A) = marginal probability of A
P(B) = marginal probability of B
 Probabilities Using Contingency
Table

Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)
A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probabilities Marginal (Simple) Probabilities

 Office Design Problem: a survey was conducted in which the
workers were asked which changes in office design would increase
productivity?

Probability Matrix/Contingency Table

Increase
Storage Space
Yes No Total
Noise Yes .56 .14 .70
Reduction No .11 .19 .30
Total .67 .33 1.00
Q. What office design changes would increase productivity, what is the
probability that this person would select reducing noise or more storage
space?

Increase
Storage Space
Yes No Total
Noise Yes .56 .14 .70
Reduction No .11 .19 .30
Total .67 .33 1.00

P( N  S )  P( N )  P( S )  P( N  S )
.70.67 .56
.81
 Demonstration Problem: Distribution of employee according to
gender in an organization::What is the prob that the employee is
either in a technical or a clerical position?

Type of Gender
Position Male Female Total
Managerial 8 3 11
Professional 31 13 44
Technical 52 17 69
Clerical 9 22 31
Total 100 55 155

P( T  C )  P( T )  P(C )
69 31
 
155 155
.645
 Demonstration Problem: Distribution of employee according to
gender in an organization::What is the prob that the employee is
either in a profession or a clerical position?

Type of Gender
Position Male Female Total
Managerial 8 3 11
Professional 31 13 44
Technical 52 17 69
Clerical 9 22 31
Total 100 55 155

P( P  C)  P( P)  P(C)
44 31
 
155 155
.484
A company has 140 employees, of which
30 are supervisors, 80 of the employees
are married and 20% of the married
employees are supervisors.

If a company employee is randomly

selected, what is the probability that the
employee is married and is a supervisor?
 Multiplication Rule

P( X  Y )  P( X )  P(Y | X )  P(Y )  P( X| Y )
80
P( M )   0. 5714
140
P( S| M )  0. 20
P ( M  S )  P ( M )  P ( S| M )
 ( 0. 5714 )( 0. 20 )  0.1143
 Special Law of Multiplication
for Independent Events
• General Law
P( X  Y )  P( X )  P(Y | X )  P(Y )  P( X| Y )
• Special Law
If events X and Y are independent,
P( X )  P( X | Y ), and P(Y )  P(Y | X ).
Consequently,
P( X  Y )  P( X )  P( Y )
 Law of Conditional Probability
• The conditional probability of X given Y is
the joint probability of X and Y divided by
the marginal probability of Y.

P( X  Y ) P(Y | X )  P( X )
P( X| Y )  
P(Y ) P(Y )
 Independent Events
• If X and Y are independent events, the
occurrence of Y does not affect the
probability of X occurring.
• If X and Y are independent events, the
occurrence of X does not affect the
probability of Y occurring.
If X and Y are independent events,
P( X | Y )  P( X ), and
P(Y | X )  P(Y ).
 Class Exercise
• According to a survey the number of business
failures in a country is 83,384.
• The construction industry accounted for
10,867 of these business failures. The Southern
states accounted for 8010 of the business
failures.
• Suppose that 1258 of all business failures were
construction business located in southern
states.
• A failed business is randomly selected from
the list of business failures
 Class Exercise...contd
• What is the probability that the business is located in
the southern states?
• What is the probability that the business is in the
construction industry or located in the southern
states?
• That the business is in the construction industry if it is
known that the business is located in the southern
states?
• That the business is located in the southern states if it
is known that the business is a construction business?
• That the business is not located in the south states if it
is known that the business is not a construction
business?
• Given the business is a construction budiness, what is
the probability that the business is not located in teh
southern states?
 Bayes’ Rule

• An extension to the conditional law of

probabilities
• Enables revision of original probabilities
with new information

P(Y | Xi ) P( Xi )
P( Xi| Y ) 
P(Y | X 1) P( X 1)  P(Y | X 2 ) P( X 2 )  P(Y | Xn ) P( Xn )
 Bayes’ Rule
• A particular type of printer ribbon is
produced by only two companies. Alamo
and South Jersy products.
• Suppose Alamo produces 65% and South
Jersy produces 35%.
• 80% of the robbons are defective, produced
by Alamo
• 12% are defective produced by South Jersy.
• A customer purchases a new ribbon.
What is the prob, Alamo produced that
ribbon?
What is the prob South Jersy produced it?
 Bayes’ Rule…contd
• If the ribbon is tested, and found defective,

• What is the prob Alamo produced it?

• What is the prob South Jersy produced it?
 Revision of Probabilities
with Bayes' Rule: Ribbon Problem
P( Alamo)  0. 65
P( SouthJersey)  0. 35
P( d | Alamo)  0. 08
P( d | SouthJersey)  0.12
P( d | Alamo)  P( Alamo)
P( Alamo| d ) 
P( d | Alamo)  P( Alamo)  P( d | SouthJersey)  P( SouthJersey)
( 0. 08)( 0. 65)
  0. 553
( 0. 08)( 0. 65)  ( 0.12 )( 0. 35)
P( d | SouthJersey)  P( SouthJersey)
P( SouthJersey| d ) 
P( d | Alamo)  P( Alamo)  P( d | SouthJersey)  P( SouthJersey)
( 0.12 )( 0. 35)
  0. 447
( 0. 08)( 0. 65)  ( 0.12 )( 0. 35)
 Revision of Probabilities
with Bayes’ Rule: Ribbon Problem

Prior Conditional Joint Revised

Probability Probability Probability Probability

Event P( Ei ) P(d| Ei ) P(Ei  d) P( Ei| d )

Alamo 0.65 0.08 0.052 0.052
0.094

=0.553

South Jersey 0.35 0.12 0.042 0.042

0.094
0.094
=0.447