Probability

Mirza Amin ul Haq

Probability
The calculated likelihood that a given event
will occur

Methods of Determining Probability

Empirical
Experimental observation
Example Process control

Theoretical
Uses known elements
Example Coin toss, die rolling

Subjective
Assumptions
Example I think that . . .

Probability Components
Experiment
An activity with observable results

Sample Space
A set of all possible outcomes

Event
A subset of a sample space

Outcome / Sample Point

The result of an experiment

Probability
What is the probability of a tossed coin
landing heads up?
Experiment

Sample Space
Event
Outcome

Probability Tree

Probability
A way of communicating the belief that an
event will occur.

Expressed as a number between 0 and 1

fraction, percent, decimal, odds
Total probability of all possible events totals 1

Relative Frequency
The number of times an event will occur
divided by the number of opportunities

nx
fx
N

f x = Relative frequency of outcome x

n x = Number of events with outcome x

= Total number of events

Expressed as a number between 0 and 1

fraction, percent, decimal, odds
Total frequency of all possible events totals 1

Probability
What is the probability of a tossed coin
landing heads up?
How many desirable
outcomes? 1
How many possible
outcomes? 2

Fx
Px
Fa

1
P
2

Probability Tree

.5 50%

What is the probability of the coin

landing tails up?

Probability
What is the probability of tossing a coin
twice and it landing heads up both times?
HH

How many desirable

outcomes? 1
HT

How many possible

outcomes? 4
TH

Fx
Px
Fa

1
P .25 25%
4

TT

Probability

3rd

What is the probability of tossing

a coin three times and it landing
1
heads up exactly two times?
How many desirable
outcomes? 3
st

How many possible

outcomes? 8

2nd

HHH

HHT

HTH

HTT

THH
THT

Fx
3
Px
P
.375 37.5%
8
Fa

TTH

TTT

Law of Large Numbers

The more trials that are conducted, the closer
the results become to the theoretical probability
Trial 1: Toss a single coin 5 times
H,T,H,H,T
P = .600 = 60%
Trial 2: Toss a single coin 500 times
H,H,H,T,T,H,T,T,T
P = .502 = 50.2%
Theoretical Probability = .5 = 50%

Probability
AND (Multiplication)
Independent events occurring simultaneously
Product of individual probabilities
If events A and B are independent, then the
probability of A and B occurring is:

P = P(A)P(B)

Probability
AND (Multiplication)
What is the probability of rolling a 4 on a single die?
How many desirable outcomes? 1
1
P4
6
How many possible outcomes? 6
What is the probability of rolling a 1 on a single die?
How many desirable outcomes? 1
1
P1
6
How many possible outcomes? 6

What is the probability of rolling a 4 and then a

1 using two dice?
P = (P4 )(P1 )

1 1
=
6 6

.0278 2.78%
36

Probability
OR (Addition)
Independent events occurring individually
Sum of individual probabilities
If events A and B are mutually exclusive, then
the probability of A or B occurring is:

P = P(A) + P(B)

Probability
OR (Addition)
What is the probability of rolling a 4 on a single die?
How many desirable outcomes? 1
1
P4
6
How many possible outcomes? 6
What is the probability of rolling a 1 on a single die?
How many desirable outcomes? 1
1
P1
6
How many possible outcomes? 6

What is the probability of rolling a 4 or a 1 on a

single die?
P ( P4 ) ( P1 )

1 1

6 6

2
.3333 33.33%
6

Probability
NOT
Independent event not occurring
1 minus the probability of occurrence

P = 1 - P(A)
What is the probability of not rolling a 1 on a die?

1
P 1 P1 1
6

5
.8333 83.33%
6

Probability
Two cards are dealt from a shuffled deck.
What is the probability that the first card is an
ace and the second card is a face card or a
ten?
How many cards are in a deck? 52
How many aces are in a deck? 4

How many face cards are in deck? 12

How many tens are in a deck? 4

Probability
What is the probability that the first card is an ace?
4
1
.0769 7.69%
52 13
Since the first card was NOT a face, what is the
probability that the second card is a face card?
12 4

.2353 23.53%
51 17
Since the first card was NOT a ten, what is the
probability that the second card is a ten?

4
.0784 7.84%
51

Probability
Two cards are dealt from a shuffled deck.
What is the probability that the first card is an
ace and the second card is a face card or a
ten?
P = PA (PF + P10 )
1 4 4
= +
13 17 51

1 12 4
= +
13 51 51

1 16
=
13 51

.0241 2.41%
If the first card is an ace, what is the
probability that the second card is a
face card or a ten? 31.37%

Business Topics
So, in my example here I had a coin flip. But this is a class
in business. We might talk about did a sale occur or not.
Maybe we will be interested in the amount of sales. The
universe of business examples is large.
We will typically talk about one or two or three variables and
each variable has basic outcomes. Remember the basic
outcomes are events. But events can be more complex. As
an example of a more complex event we might be interested
in the event when a die is rolled the result is an even
number. Or maybe when we look at store sales on a day
(like at each Wal-Mart) we are interested in the outcome of
sales being more than $1,000,000.
20

Contingency Table
A contingency table is a particular way to view a sample
space. Say we talk to 100 customers who just made a
purchase in a store and not only do we note the gender of
the customer we ask if they paid cash or used a credit card.
The responses are summarized on the next screen

21

Contingency Table
Payment Method

Gender

Cash Credit Card Total

Female

12

15

Male

17

68

85

20

80

100

Total

So, each of the 100 people observed had to be put in a

gender category and had to be given a payment method.
If you divide each number in the table by the grand total
(here 100 and this represents the total number of people
observed) the table is then called a joint probability table.
Lets do this and see what results on the next 22
screen.

Marginal Probabilities
The total column and the total row are called marginal
probabilities because the are written in the margins of the
table. But, note that adding across the Female row gives a
total of .15. This is the P(F), or the probability that you would
select a female when talking to someone involved in the
study.
The other marginal probabilities are
P(M) = .85

P(Cash) = .2 and P(Credit Card) = .8

23

Joint Probability
Gender
Female (A)
Male (A)
Total

Payment Method
Cash (B)
CC (B)
Total
P(A and B) P(A and B) P(A)
P(A and B) P(A and B) P(A)
P(B)
P(B)
1.0

From the example I have labeled the gender A and A and

the payment method B and B and then I have filled the
table out in definitional form.

24

Joint Probability Table

Payment Method

Gender

Cash Credit Card Total

Female

.03

.12

.15

Male

.17

.68

.85

.20

.80

Total

25

Marginal Probability
Note in the joint probability table on the previous screen
(which is a contingency table that has been modified by
dividing all numbers by the grand total!) that
1) In any row the marginal probability is the sum of the joint
probabilities in that row, and

2) In any column the marginal probability is the sum of the

joint probabilities in that column.
(Also note that the sum of the probability of complements
equals 1 > for example P(A) + P(Ac) = 1

26

Union of Events the General

Addition Rule
Sometimes we want to ask a question about the probability of A
or B, written P(A or B) = P(A B).
By the general addition rule
P(A or B) = P(A) + P(B) P(A and B).

In our example we have P(A or B) = .15 + 0.20 0.03 = 0.32.

Lets think about this some more. How many are Female? 3 +
12 = 15! How many paid cash? 3 + 17 = 20! But 3 of these
were in both A and B. So, when we ask a question about A or
B we want to include all that are A or B, but we only want to
include them once. If they are both we subtract out the
intersection because it was included in both the row and
27
column total.

Venn Diagram
A Venn diagram, named in honor of Mr. Venn, is another
way to present the sample space for two variables.

B
A

This area represents the intersection

of A and B.

The rectangle here

represents the sample
space. On one variable we
have event A and that takes
up the space represented
by circle A. Ignoring circle
B, all the rest of the
rectangle is Ac (the
complement of A). A similar
interpretation holds for B.
28

Final Comment
When we look at P(A and B) the intersection of A and B both A and B have to have occurred to have a non-zero
value here.
When we look at P(A or B) the union of A and B - either A
or B have to have occurred to have a non-zero value here.
But, many times both did occur and if there is an overlap
(intersection) of the two we have to subtract out the
overlap.

29

Conditional Probability

30

Conditional Probability
As we have seen, P(A) refers to the probability that event A will occur. A new
idea is that P(A|B) refers to the probability that A will occur but with the
understanding that B has already occurred and we know it. So, we say the
probability of A given B. The given B part means that it is known that B has
occurred.
By definition
P(A|B) = P(A and B)/P(B).
Similarly

P(B|A) = P(A and B)/P(A).

Note P(A and B) = P(B and A)

31

Now we have by definition

P(A|B) = P(A and B)/P(B).
In this definition, B has already occurred. The P(B) is the denominator of
P(A|B) and is thus the base of the conditional probability. The intersection of A
and B is in the numerator. Since B has occurred, the only way A can have
occurred is if there is an overlap of A and B. So we have the ratio
probability of overlap/probability of known event.
Lets turn to the example from above. The joint probability table is repeated on
the next slide.

32

Joint Probability Table

Payment Method

Gender

Cash Credit Card Total

Female

.03

.12

.15

Male

.17

.68

.85

.20

.80

Total

Lets say you saw someone pay cash for a purchase, but
your view was blocked as to the gender of the person. The
P(femalecash) = .03/.20 = .15
33

Joint Probability Table

Payment Method

Gender

Cash Credit Card Total

Female

.03

.12

.15

Male

.17

.68

.85

.20

.80

Total

Lets say you saw someone a female leave the store having
made a purchase, but your view was blocked as to the type
of payment. The P(cashfemale) = .03/.15 = .20
34

Independent Events
Events A and B are said to be independent if
P(A|B) = P(A) or P(B|A) = P(B).
In the example we have been using P(Female) = .15, and P(Female|Cash) = .15.

What is going on here? Well, in this example it turns out that the proportion of
females in the study is .15 and when you look at those who paid cash the
proportion of females is still .15. So, knowing that the person paid cash doesnt
change your view of the likelihood that the person is female.

But, in some case (not here) having information about B gives a different view
about A. When P(A|B) P(A) we say events A and B are dependent events.
Similarly, when P(B|A) P(B) events A and B are dependent.

35

Does a coin have a memory? In other words, does a coin remember how many
times it has come up heads and will thus come up tails if it came up heads a lot
lately? Say A is heads on the third flip, B is heads on the first two flips. Is heads
on the third flip influenced by the first two heads. No, coins have no memory!
Thus A and B are independent. (Note I am not concerned here about the
probability of getting three heads!)

Have you ever heard the saying, Pink sky in the morning, sailors take warning,
pink sky at night sailors delight. I just heard about it recently. Apparently it is a
rule of thumb about rain. Pink sky in the morning would serve as a warning for
rain that day. If A is rain in the day and B is pink sky in the morning, then it seems
that the P(A|B) P(A) and thus the probability of rain is influenced by morning sky
color (color is really just an indicator of conditions).

36

Lets think about one more example. If you watched a football team all year
you could use the empirical approach to find the probability that it will throw a
pass on a given play. Say P(pass)=0.4. This means the probability it will
pass on a given play is 0.4.
But, if there are 5 minutes left in the game and the team is down 14 points the
team will want to pass more. SO, P(pass|down 14 with 5 minutes left) = 0.75,
for example. This means the probability of a pass depends on the score and
time remaining!

37

I have used some examples to give you a feel about when events are
independent and when they are dependent.
By simple equation manipulation we change the conditional probability
definition to the rule called the multiplication law or rule for the intersection
of events:

P(A and B) = P(B)P(AB)

or P(A and B) = P(A)P(BA) .

Note the given part shows up in the other term.

Now this rule simplifies if A and B are independent. The conditional
probabilities revert to regular probabilities. We would then have

P(A and B) = P(B)P(A) = P(A)P(B).

Does this hold in our running example? Sure it does!

38

Say, as a new example, we have A and B with P(A)=.5, P(B)=.6 and P(A and B)
=.4
Then
a. P(AB) = .4/.6 = .667
b. P(BA) = .4/.5 = .8
c. A and B are not independent because we do NOT have P(AB) = P(A), or
P(BA) = P(B).
Say, as another example, we have A and B with P(A)=.3 and P(B)=.4 and here we
will say A and B are mutually exclusive. This means P(A and B) = 0 (in a
Venn Diagram A and B have no overlap), then
a. P(AB) = 0/.4 = 0

Here A and B are not independent.

39

Y
Y1
X

X1

P(X1 and Y1)

X2

P(X2 and Y1)

Totals

P(Y1)

Y2

Totals

P(X1 and Y2) P(X1)

P(X2 and Y2)

P(X2)

P(Y2)

1.00

Here I put the joint probability table again in general terms. Question X has
mutually exclusive and collectively exhaustive events X1 and X2. For Y we have
a similar set-up. Note here each has only two responses, but what we will see
below would apply if there are more than 2 responses.

Lets review some of the probability rules we just went through and then we will
add one more rule.

40

Inside the joint probability table we find joint probabilities (like P(X1 and Y1)
and in the margins we find the marginal probabilities (like P(X1)).
Marginal Probability Rule
P(X1) = P(X1 and Y1) + P(X1 and Y2)
General Addition Rule
P(X1 or Y1) = P(X1) + P(Y1) P(X1 and Y1)
Conditional Probability
P(X1|Y1) = P(X1 and Y1)/P(Y1)
Multiplication Rule
P(X1 and Y1) = P(X1|Y1)P(Y1)

41

The new part is to view the marginal probability rule as taking each part and
use the multiplication rule. So,
Marginal Probability Rule
P(X1) = P(X1 and Y1) + P(X1 and Y2)
=P(X1|Y1)P(Y1) + P(X1|Y2)P(Y2)
Where Y1 and Y2 are mutually exclusive and collectively exhaustive.

42