Probability

Dr. Adrija Bhattacharya

Department of CSE
Heritage Institute of Technology
 Phenomena

Non-deterministic
Deterministic
Deterministic Phenomena
• There exists a mathematical model that allows
“perfect” prediction the phenomena’s
outcome.
• Many examples exist in Physics, Chemistry
(the exact sciences).
Non-deterministic Phenomena
• No mathematical model exists that allows
“perfect” prediction the phenomena’s
outcome.
Non-deterministic Phenomena
• may be divided into two groups.
Random phenomena
– Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.

Haphazard phenomena
– unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
 Phenomena

Non-deterministic
Deterministic
Haphazard phenomena
– unpredictable outcomes, but no long-run,
exhibition of statistical regularity in the
outcomes.
– Do such phenomena exist?
– Will any non-deterministic phenomena exhibit
long-run statistical regularity eventually?
Random phenomena
– Unable to predict the outcomes, but in the long-
run, the outcomes exhibit statistical regularity.

Examples
1. Tossing a coin – outcomes S ={Head, Tail}
Unable to predict on each toss whether is Head or
Tail.
In the long run can predict that 50% of the time
heads will occur and 50% of the time tails will occur
2. Rolling a die – outcomes
S ={ , , , , , }

Unable to predict outcome but in the long run can

one can determine that each outcome will occur 1/6
of the time.
Use symmetry. Each side is the same. One side
should not occur more frequently than another side
in the long run. If the die is not balanced this may
not be true.
Definitions
 The sample Space, S
The sample space, S, for a random phenomena
is the set of all possible outcomes.
Examples
1. Tossing a coin – outcomes S ={Head, Tail}

2. Rolling a die – outcomes

S ={ , , , , , }

={1, 2, 3, 4, 5, 6}
 Trial and Events
• Consider an experiment which, though
repeated under essentially identical
conditions, does not give unique results but
may result in any one of the several possible
outcomes.
• The experiment is known as a trial
• The outcomes are known as events or casts.
 An Event , E
The event, E, is any subset of the sample space,
S. i.e. any set of outcomes (not necessarily all
outcomes) of the random phenomena
Venn
S diagram
E
The event, E, is said to have occurred if after the
outcome has been observed the outcome lies in E.
Often called as Favourable Events or Cases

S
E
 For example

• (i) Throwing of a die is a trial and getting l(or 2

or 3, ... or 6) is an event
• (ii) Tossing of a coin is a trial and getting head
(H) or tail (T) is an event
• (iii) Drawing two cards from a pack of well-
shuffled· cards is a.trial.and getting a king and
a queen are events.
Examples

1. Rolling a die – outcomes

S ={ , , , , , }
={1, 2, 3, 4, 5, 6}
Examples

1. Rolling a die – outcomes

S ={ , , , , , }
={1, 2, 3, 4, 5, 6}

E = the event that an even number is

rolled
= {2, 4, 6}
={ , , }
 Exhaustive Events
• The total number of possible outcomes in any
trial is known as exhaustive events or cases.
 For example

• In tossing of a coin there are two exhaustive

cases, viz., head and. Tail (the possibilty of the
coin standing on an edge being ignored).
• In throwing of a die, there are six, Exhaustive
cases since anyone of the 6 faces 1,2, ... ,6 may
come uppermost.
 Shout Please
• In drawing two cards from a pack of cards the
Exhaustive number of cases is………..
 Shout again Please
• In rolling of two dice, the exhaustive number
of cases is
 Finally
• In general in throwing of n dice the exhaustive
number of cases is 6^n.
 Special Events
The Null Event, The empty event - f

f = { } = the event that contains no outcomes

The Entire Event, The Sample Space - S
S = the event that contains all outcomes

The empty event, f , never occurs.

The entire event, S, always occurs.
 Set operations on Events
Union
Let A and B be two events, then the union of A
and B is the event (denoted by AB) defined by:
A  B = {e| e belongs to A or e belongs to B}

AB

A B
The event A  B occurs if the event A occurs or
the event and B occurs .

AB

A B
 Intersection

Let A and B be two events, then the intersection

of A and B is the event (denoted by AB) defined
by:
A  B = {e| e belongs to A and e belongs to B}

AB

A B
The event A  B occurs if the event A occurs and
the event and B occurs .

AB

A B
 Complement

Let A be any event, then the complement of A

(denoted by A ) defined by:

A = {e| e does not belongs to A}

A
A
The event A occurs if the event A does not
occur

A
A
In problems you will recognize that you are
working with:

1. Union if you see the word or,

2. Intersection if you see the word and,
3. Complement if you see the word not.
Definition: mutually exclusive
Two events A and B are called mutually
exclusive if:
A B =f

A B
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.

A B
Probability
Definition: probability of an Event E.
Suppose that the sample space S = {o1, o2, o3, …
oN} has a finite number, N, of oucomes.
Also each of the outcomes is equally likely
(because of symmetry).
Then for any event E

n(E) n(E) no. of outcomes in E

PE= = =
n(S ) N total no. of outcomes
Note : the symbol n ( A ) = no. of elements of A
 Thus this definition of P[E], i.e.

n(E) n(E) no. of outcomes in E

PE= = =
n(S ) N total no. of outcomes

Applies only to the special case when

1. The sample space has a finite no.of outcomes, and
2. Each outcome is equi-probable
If this is not true a more general definition of
probability is required.
Rules of Probability
 Rule The additive rule
(Mutually exclusive events)

P[A  B] = P[A] + P[B]

i.e.
P[A or B] = P[A] + P[B]

if A  B = f
(A and B mutually exclusive)
If two events A and B are are mutually
exclusive then:
1. They have no outcomes in common.
They can’t occur at the same time. The outcome
of the random experiment can not belong to both
A and B.

A B
 For example
• (i) In throwing a die all the 6 faces‘ numbered
1 to 6 are mutually exclusive
• since if anyone of these faces comes, ,the
possibility of others, in the same trial, is ruled
out
• (ii) Similarly in tossing a coin the events head
and tail are mutually exclusive.
P[A  B] = P[A] + P[B]
i.e.
P[A or B] = P[A] + P[B]

A B
 Equally likely events
• Outcomes of a trial are set to be equally likely
if taking into consideration all the 'relevant
evidence there is no reason to expect one in
preference-to the others.
 Example
• (i) In tossing an unbiased or uniforn coin, head
or tail are “Equally likely events”
• (a) In throwing an unbiased die, all the six
faces are equally likely to come.
 Rule The additive rule
(In general)

P[A  B] = P[A] + P[B] – P[A  B]

or
P[A or B] = P[A] + P[B] – P[A and B]
Logic A B
A B

A B

When P[A] is added to P[B] the outcome in A  B

are counted twice
hence
P[A  B] = P[A] + P[B] – P[A  B]
 P  A  B  = P  A + P  B  − P  A  B 

Example:
Saskatoon and Moncton are two of the cities competing
for the World university games. (There are also many
others). The organizers are narrowing the competition to
the final 5 cities.
There is a 20% chance that Saskatoon will be amongst
the final 5. There is a 35% chance that Moncton will be
amongst the final 5 and an 8% chance that both
Saskatoon and Moncton will be amongst the final 5.
What is the probability that Saskatoon or Moncton will
be amongst the final 5.
Solution:
Let A = the event that Saskatoon is amongst the final 5.
Let B = the event that Moncton is amongst the final 5.
Given P[A] = 0.20, P[B] = 0.35, and P[A  B] = 0.08
What is P[A  B]?
Note: “and” ≡ , “or” ≡  .
P  A  B = P  A + P  B − P  A  B
= 0.20 + 0.35 − 0.08 = 0.47
 Rule for complements

2. P  A  = 1 − P  A

or
P  not A = 1 − P  A
 Complement
Let A be any event, then the complement of A
(denoted by A ) defined by:

A = {e| e does not belongs to A}

A
A
The event A occurs if the event A does not
occur

A
A
Logic:
A and A are mutually exclusive.
and S = A  A

A
A

thus 1 = P  S  = P  A + P  A 
and P  A  = 1 − P  A
 Exercises
• What is the chance that leap year selected at
random will contain 53 Sundays?
 Exercises
• What is the chance that leap year selected at random will
contain 53 Sundays?
In a leap year (which consists of 366 days) there'are 52
complete weeks and 2 days over. The following are the
Possible combinations for these two 'over'days:
(i) Sunday and Monday, (ii) Monday and Tuesday, (iii) Tuesday
and Wednesday, (iv) Wednesday and Thursday, (v) Thursday
and Friday, (vi) Friday and Saturday, and (vii) Saturday and
Sunday.
• In order that it leap year selected at random should contain
53 Sundays, one of the two 'over" days must be Sunday. Since
out of the above 7 possibilities, 2 ((i)and (vii)) are favorable
to this event,
• Required Probability = 2/7
 Exercises 2
• A bag contains 3 red, 6 white and 7 blue balls,
What is the probability that two balls drawn
are white and blue?
 Exercises 3
• (a) Two cards are drawn at random from a well-
shuffled pack of 52 cards, Show that the chance
of drawing two aces is 1/221,
• (b) From a {kick of 52 cards, three are drawn at
random, Find the chance that they are a king, a
queen and a knave,
• (c) Four cards are drawn from a pack of cards,
Find the probability that
– (i) all are diamond,
– (ii) there is one card of each suit, and
– (iii) there are two spades and two hearts,
 Exercises 4
• Seven cards are drawn at random from a pack
of 52 cards. What is the probability that 4 will
be red and 3 black?
 Exercises 5
• The sum of two non-negative quantities is
equal to 2n. Find the chance that their product
is not less than 3/4th times their greatest
product.
Conditional Probability
 Conditional Probability
• Frequently before observing the outcome of a
random experiment you are given information
regarding the outcome
• How should this information be used in prediction of
the outcome.
• Namely, how should probabilities be adjusted to take
into account this information
• Usually the information is given in the following
form: You are told that the outcome belongs to a
given event. (i.e. you are told that a certain event has
occurred)
 Definition
Suppose that we are interested in computing the
probability of event A and we have been told
event B has occurred.
Then the conditional probability of A given B is
defined to be: P  A B  = P  A  B  if P  B   0
  P  B
where P(B I A) represents the conditional
probability of occurrence of B when the
event A has already happened and P(A I B) is the
conditional probability of happening of A. given
that B has already happened.
 Rationale:
If we’re told that event B has occurred then the sample
space is restricted to B.
The probability within B has to be normalized, This is
achieved by dividing by P[B]
The event A can now only occur if the outcome is in of
A ∩ B. Hence the new probability of A is:

A
P  A  B B
P  A B  =
P  B A∩B
 An Example
The academy awards is soon to be shown.
For a specific married couple the probability that
the husband watches the show is 80%, the
probability that his wife watches the show is
65%, while the probability that they both watch
the show is 60%.
If the husband is watching the show, what is the
probability that his wife is also watching the
show
 Solution:
The academy awards is soon to be shown.
Let B = the event that the husband watches the show
P[B]= 0.80
Let A = the event that his wife watches the show
P[A]= 0.65 and P[A ∩ B]= 0.60

P  A  B 0.60
P  A B  = = = 0.75
P  B 0.80
Independence
 Definition

Two events A and B are called independent if

P  A  B  = P  A P  B 
Note if P  B   0 and P  A  0 then
P  A  B P  A P  B 
P  A B  = = = P  A
P  B P  B
P  A  B P  A P  B 
and P  B A = = = P  B
P  A P  A
Thus in the case of independence the conditional probability of
an event is not affected by the knowledge of the other event
 Difference between independence
and mutually exclusive

mutually exclusive
Two mutually exclusive events are independent only in
the special case where
P  A = 0 and P  B  = 0. (also P  A  B  = 0
Mutually exclusive events are
A highly dependent otherwise. A
B
and B cannot occur
simultaneously. If one event
occurs the other event does not
occur.
Independent events
P  A  B  = P  A P  B 

P  A  B P  A
or = P  A =
P  B P S 
S

A B
The ratio of the probability of the
A B set A within B is the same as the
ratio of the probability of the set
A within the entire sample S.
The multiplicative rule of probability

 P  A P  B A if P  A  0
P  A  B = 
 P  B  P  A B  if P  B   0

and
P  A  B  = P  A P  B 

if A and B are independent.

 Example-Exercise
• What is the probability of drawing a 7 of clubs,
then a 4 of hearts without replacement?
• What is the probability of drawing a 7 of
clubs, then a 4 of hearts without
replacement? “given that”

• P(draw a 4 of hearts 7 of
• P(draw 7 of clubs)
clubs was drawn)

• = 1 1
•
52 51
• Notice how conditional probability “shrinks
the sample space”.
 Calculating Conditional Probabilities
Consider the two-way table. Define events
E: the grade comes from an EPS course, and
L: the grade is lower than a B.

Total
6300
1600
2100
Total 3392 2952 3656 10000

Find P(L)
P(L) = 3656 / 10000 = 0.3656
Find P(E | L)
P(E | L) = 800 / 3656 = 0.2188
Find P(L | E)
P(L| E) = 800 / 1600 = 0.5000
Multiplication Rule for Independent
Events Revisited
Multiplication Rule for Independent events:
P(A and B) = P(A) * P(B)
Independent – the occurrence of one event has no effect on the probability
of the occurrence of another event.
Example: A survey by the American Automobile Association (AAA) revealed
that 60 percent of its members made airline reservations last year. Two
members are selected at random. What is the probability both made airline
reservations last year?
P(R1 and R2) = P(R1)*P(R2) = (0.6)*(0.6) = .36

72
 General Multiplication Rule

Use when events are Dependent.

P(A and B) = P(A) * P(B|A)
For two events A and B, the joint probability that both
events will happen is found by multiplying the probability
event A will happen by the conditional probability of event
B occurring.

73
In a recent study it was found that the
probability that a randomly selected student is
a girl is .51 and is a girl and plays sports is .10.
If the student is female, what is the probability
that she plays sports?

P(S and F) .1
P(S|F) = = = .1961
P(F) .51
The probability that a randomly selected
student plays sports if (given that) they are
male is .31. What is the probability that the
student is male and plays sports if the
probability that they are male is .49?

P(S and M) x
P(S|M) = .31 =
P(M) .49
x = .1519
 Checking for Independence
• The probability of a student in this course getting an
A is 0.35; the probability of being female is 0.60. If
P(A and Female) = 0.15, are the events “getting an A”
and “being a female” independent?
• Check to see if P(A)•P(Female) = P(A and Female)
• (0.35)(0.60) ≠0.15
• Thus, the events “getting and A” and “being a
female” are not independent.
 Exercise
• If two dice are thrown, what is the probability
that the sum is
• (a) greater thon 8, and (b) neither 7 nor II?
 Exercise
• A card is drawn from a well-shuffled pack 0/
playing cards.
• What is the probability that it is either a spade
or an ace?
 Exercise
• An urn contains 4 tickets numbered 1, 2, 3 and
4 and another contains 6 Tickets numbered 2,
4, 6, 7, 8, 9. If one 0/ the two urns is chosen at
random and a ticket is drawn at random from
the chosen urn, find the probabilities that the
ticket drawn bears the number (i) 2 or4, (ii) 3,
(iii) 1or 9
Summary of the Rules of
Probability
 The additive rule
P[A  B] = P[A] + P[B] – P[A  B]

and
P[A  B] = P[A] + P[B] if A  B = f
 The Rule for complements
for any event E

P  E  = 1 − P  E 
Conditional probability

P  A  B
P  A B  =
P  B
 The multiplicative rule of probability

 P  A P  B A if P  A  0

P  A  B = 

 P  B  P 
 A B 
 if P  B   0
and

P  A  B = P  A P  B
if A and B are independent.

This is the definition of independent