Prob

Probability and Random Variables

The math, the computation, and examples.

Prof. Dr. Asad Ali

Department of Applied Mathematics and Statistics

Institute of Space Technology
Islamabad, Pakistan

1 / 128
Prob

Probability

Chapter 6: Probability

95 / 128
Prob

Probability

Probability, in our daily language, means the chance of happening of some event. For example,
you might have heard sentences like,
What’s the chance that Pakistan will win the toss in today’s match?
What are the chances of rain tomorrow?
What’s the chance that Mr X will win in next elections?
These are just statements, often we need a numerical quantification of these statements, here
comes the probability theory.
In statistics, probability is just a quantitative measure of uncertainty or a measure of the degree
of belief in a particular statement.
Some basic terms and concepts are introduced as following.
Experiment and Random Experiment: An experiment means a planned activity, which
yields some outcome or result. A single performance of an experiment is called a trial.
A random experiment is one whose results can not be predicted in advance, even if it’s repeated
under the same conditions. For example, tossing a coin or a die. Three desirable properties of a
random experiment are:
1 The experiment should be repeatable, practically or theoretically, any number of times.
2 The experiment always has two or more possible outcomes. An experiment that has only one possible
outcome is not a random experiment.
3 The outcome of each repetition is unpredictable.
96 / 128
Prob

Probability
Sample space: A set, denoted by ‘S’, of all possible outcomes of a random experiment is called
sample space. It can either be real or theoretical. Sample space is same as a universal set in a sets
theory problem. The elements of sample space are called sample points. For example when a
single coin is tossed once, the possible outcomes are a head (H) or a tail (T ), so the sample space is
S = {H, T }. Similarly, when two coins are tossed once (or a single coin is tossed twice), the sample
space consists of four possible outcomes S = {HH, HT, T H, T T }. Similarly, when a six faced die is
thrown once, the sample space is S = {1, 2, 3, 4, 5, 6}, and when thrown twice all possible outcomes
are:
 

 (1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6)  
(2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6) 

 

 

(3, 1) (3, 2) (3, 3) (3, 4) (3, 5) (3, 6)
 
S=

 (4, 1) (4, 2) (4, 3) (4, 4) (4, 5) (4.6) 

 (5, 1) (5, 2) (5, 3) (5, 4) (5, 5) (5, 6) 

 

 
(6, 1) (6, 2) (6, 3) (6, 4) (6, 5) (6, 6)
 

Event: Event is an individual outcome or a set of outcomes of our interest, of a random experiment
or a trail. Event is just a subset of the sample space. For example, in the experiment of tossing two
coins we may be interested, in finding the probability of a T T , that’s both coins lands with their
tail sides up, so E = {T T }. In rolling a pair of dice one may be interested in finding the probability
of two or more outcomes, e.g. (1,1) or (4,4) or (6, 6) appears, so here E = {(1, 1), (4, 4), (6, 6)}. One
may be interested, in finding the probability, e.g. that the sum of dots on both dice is 8, in which
case E = {(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)}.
97 / 128
Prob

Probability
Types of events:
¶ Mutually exclusive events: Two events A and B are said to be mutually exclusive if they can not
occur at the same time. For example, in a single toss of a coin, H and T are mutually exclusive as
their simultaneous occurrence is impossible. In simple words the intersection of mutually exclusive
events is an empty set.
· Exhaustive events: When the union of all the mutually exclusive events is the entire sample
space, we call them collectively exhaustive events, as they exhaust the whole sample space. In a
single coin tossing experiment, H and T are exhaustive events as together they comprise the entire
sample space, S = {H, T }.
¸ Equally likely events: Equally likely events are those that have an equal chance of occurrence at
each trial. For example, when a fair (balanced) coin is tossed, both H and T are equally likely to
occur in repeated trails with a probability of occurrence 1/2 for each.
How to count sample points in a sample space?
Before listing all the possible sample points, we must know as to how many are they, in order to
avoid missing/double counting any. Depending on the nature of the experiment, there are several
ways to calculate the number of all possible sample points in a sample space. The most commonly
used methods are:
Multiplication Rule: If an experiment consists of two sub-experiments such that one has n1 outcomes
and other has n2 outcomes, then the combined sample space consists of n1 ×n2 outcomes. For examples,
if a die and a coin are thrown at the same time the number of possible outcomes is 2 × 6 = 12.
Similarly, if an experiment has K possible outcomes in a single trial then in n trails it will have K n
possible outcomes. For example, if a coin is tossed twice, we have 2 × 2 = 22 = 4 possible outcomes.
Similarly, if the die is thrown twice, we have 6 × 6 = 62 = 36 possible outcomes.
98 / 128
Prob

Probability

Permutation Rule: A permutation is an “arrangement” of n objects in a “specific order”.

The arrangement of n objects in a specific order using r objects at a time is called a permutation
of n objects taking r objects at a time. It is written as n Pr , and the formula is:

n n!
Pr =
(n − r)!

“The combination to the safe was 472”. Now we do care about the order. “724” would not
work, nor would “247”. It has to be exactly 4-7-2.
Combination Rule: A “selection” of distinct objects “without any regard to order” is called
a combination. The number of combinations of r objects selected from n objects is denoted n Cr
and is given by the formula: !
n n n!
Cr = =
r (n − r)!r!

“My fruit salad is a combination of apples, grapes and bananas”. We don’t care what order the
fruits are in, they could also be ”bananas, grapes and apples” or “grapes, apples and bananas”,
it’s the same fruit salad.

99 / 128
Prob

Probability

Example 22.
A student club consists of 6 students of which a three person committee consisting of a president, a
secretary, and a treasure is to be chosen. In how many ways we can choose this committee?
Solution:
Here we see that the order: president, secretary, treasure, is of critical importance and the positions are
clearly distinguishable from each other.
Of the 6 students, a president can be chosen in 6 possible ways. From the rest of 5 students, a secretary
can be chosen in 5 five possible ways, and similarly a treasure can be chosen in 4 possible ways from the
rest of 4 students. So, we have overall 6 × 5 × 4 = 120 possible ways. Now lets do it using the prescribed
formula: n Pr .
Here n = 6 and r = 3, so

n n!
Pr =
(n − r)!
6 6! 6! 6 × 5 × 4 ×
3!
P3 = = = = 120 ways
(6 − 3)! 3! 3!


100 / 128
Prob

Probability

Example 23.
Now suppose a student has 3 friends among these 6 students and wants them to be elected at these
positions but does not care which friend is elected at which position. That’s the order does not matter.
In how many ways 3 students can be chosen out of 6 students?
Solution:
Here we see that the order: president, secretary, treasure, is of no importance.
Here we use combination rule: n Cr . Here again n = 6 and r = 3, so

n n!
Cr =
(n − r)!r!
6 6! 6!
C3 = =
(6 − 3)!3! 3!3!
6 × 5 × 4 × 3!
=
3!3!
3!

=
6 × 5 × 4 × 
3!
= 20 ways
3!
 3!

101 / 128
Prob

Probability

Definitions of Probability
Classical Probability:
If a random experiment can produce n mutually exclusive and equally likely outcomes, and if
m out to these outcomes are considered favorable to the occurrence of a certain event A, then the
m
probability of the event A, denoted by P (A), is defined as the ratio . We can write,
n
m Number of outcomes in A
P (A) = =
n Number of total outcomes in S
It’s the simplest definition of probability. However, this definition is not applicable if the assumption
of equally likely does not hold.
Empirical or relative frequency probability:
If a random experiment is repeated a large number of times, say n, under identical conditions and
if an event A is observed to occur m times, then the probability of A is defined as,
m
P (A) = lim
n→∞ n
m
As n → ∞, the ratio becomes stable at its numerical value. Indeed “identical conditions” is
n
rather a hard assumption and can’t be satisfied in most cases.

102 / 128
Prob

Probability

Axiomatic Probability:
This type of probability is based on certain axioms (a self-evident truth that requires no proof).
These axioms are:
For any event A, 0 ≤ P (A) ≤ 1; the probability of any thing lies between 0 and 1. Negative probability
has no meanings.
Probability can not be greater than one, because it’s just the proportion of a small portion (a subset) in
the entire total of sample space. Similarly it can not be negative, as it’s just a comparison of frequencies
rather than some measurement.
P (S) = 1; the probability of the entire sample space is 1.
If A and B are mutually exclusive events then P (A ∪ B) = P (A) + P (B).
If they are not independent i.e. they interact, then you will have to deduct probability of that interaction
from their union, that’s P (A ∪ B) = P (A) + P (B) − P (A ∩ B). We will learn about this at later slides.
Subjective Probability:
Subjective probability is based on personal judgment. In subjective probability, a person or group
makes an educated guess at the chance that an event will occur. This guess is based on the person’s
experience and evaluation of a solution. Subjective probability is applied when all the above approve
approaches can not be applied. However, a biased person can lead us to quite a wrong conclusion,
also, two or more persons can arrive at different probabilities, these are the negative points of this
approach.

103 / 128
Prob

Probability

Examples
Lets start from simple examples:
¶ When we toss a single coin we either get a Head (H) or a Tail (T ), so the sample space is S = {H, T }.
Lets denote the event of the occurrence of a head by A, that’s A = {H}, so the probability of A, i.e.
getting an H is,

number of favorable outcomes

P (A) =
number of all possible outcomes
number of elements in A
or =
number of elements in S
n(A)
=
n(S)
1
so P (A) =
2
So, probability is nothing more than just the proportion of the outcomes of interest in total possible
outcomes.

104 / 128
Prob

Probability

· A coin is tossed twice, what is the probability that

i) a head appears on both?
ii) at least one head occurs?
Solution:
The sample space for this experiment consists of 22 = 4 possible outcomes
S = {HH, HT, T H, T T }.
i) Let A denote the event that a head appears on both, then A = {HH}.
Thus the probability of A is
n(A)
P (A) =
n(S)
1
=
4

ii) Let B denotes that at least one head occurs. By at least one head we mean one or more heads. Now
look at the sample space, we see that out of all possible 4 outcomes, 3 outcomes contain at least one
head: HH, HT , T H. So,
n(B) 3
P (B) = =
n(S) 4

105 / 128
Prob

Probability

¸ Three coins are tossed at once, what is the probability that

i) a tail appears on all?
ii) at least two heads occur?
Solution:
The sample space for this experiment consists of 23 = 8 possible outcomes
S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }.
i) Let A denote the event that a tail appears on all, then A = {T T T }.
Thus the probability of A is
n(A)
P (A) =
n(S)
1
=
8

ii) Let B denotes that at least two heads occur. Look at the sample space, we see that the outcomes of
our interest are B = {HHH, HHT, HT H, T HH}. That’s out of all possible 8 outcomes there are 4
outcomes that contains at least two heads. So,
n(B) 4
P (B) = =
n(S) 8

106 / 128
Prob

Probability

¹ Two fair dice are thrown, what is the probability that a sum of 8 or more dots occurs (i.e. the sum
of dots on both dice is at least 8.)?
Solution:
A die has 6 faces, so K = 6. In this experiment two dice are thrown so n = 2 (equivalent to
throwing a single die twice) therefore the sample space for this experiment has K n = 62 = 36
outcomes:
 
 (1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6) 
(2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6)

 


 

(3, 1) (3, 2) (3, 3) (3, 4) (3, 5) (3, 6)
 
S=

 (4, 1) (4, 2) (4, 3) (4, 4) (4, 5) (4, 6) 

 (5, 1) (5, 2) (5, 3) (5, 4) (5, 5) (5, 6)

 

 

(6, 1) (6, 2) (6, 3) (6, 4) (6, 5) (6, 6)
Let A represent the event that the sum of dots is at least 8. Looking at the sample space we see
that,
 
 (1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6) 
(2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6)8

 


 

(3, 1) (3, 2) (3, 3) (3, 4) (3, 5)8 (3, 6)9
 
S=

 (4, 1) (4, 2) (4, 3) (4, 4)8 (4, 5)9 (4, 6)10 

 (5, 1) (5, 2) (5, 3)8 (5, 4)9 (5, 5)10 (5, 6)11

 

 

(6, 1) (6, 2)8 (6, 3)9 (6, 4)10 (6, 5)11 (6, 6)12
Thus we have,  
(2, 6) (3, 5) (3, 6) (4, 4) (4, 5) (4.6) (5, 3) (5, 4)
A=
(5, 5) (5, 6) (6, 2) (6, 3) (6, 4) (6, 5) (6, 6)
n(A) 15 5
So the probability of A is P (A) = = =
n(S) 36 12 107 / 128
Prob

Probability

Laws of probability
There are some probability rules, that make a probability problem easier to handle. Sometimes the
probabilities of certain events are known to us and we can use them to calculate the probability of
another event. Some events can be written as the union or intersection of two or more events and
sometimes an event can be split into two or more events. For example,
If a die is thrown twice, we may be interested in finding the probability of getting a total of 7 or 11.

⇒ Now here we have two outcomes 7 or 11, if either one occurs we consider it our favorite outcome.

⇒ We can consider them as separate events that’s let A denotes the event that a sum of 7 occurs and let
B be the event that a sum of 11 occurs.

If a die is thrown twice, we may be interested in finding the probability of getting a 3 on one die and
5 on the other die. (beware here, we are not talking about a total or sum of dots)

⇒ Here again we have two outcomes 3 and 5, if both occurs we consider it our favorite outcome.

⇒ Again we can consider them as separate events that’s let A denotes the event that 3 occurs and let B
be the event that 5 occurs.

108 / 128
Prob

Probability

Additive Law or (simply) Addition Law: This law has two forms:
¶ If A and B are any two events then
P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
¶ If A and B are two mutually exclusive events then

: P (∅) = 0


P (A ∪ B) = P (A) + P (B) −  ∩
P (A B)
P (A ∪ B) = P (A) + P (B)
∵ the intersection of mutually exclusive events is an empty set.

Sample Space A∪B= A + B - A∩B Sample Space A∪B= A + B

A A∩B B A B

109 / 128
Prob

Probability

Example 24.
Consider a sample space S = {1, 2, 3, 4, 5, 6, 9, 15}. A number is chosen at random, what is the probability
that the number is an even number or a prime number? (prime number: a natural number greater than
1, has no divisor other than itself and 1.)
Solution:
1
Here each sample point in the sample space has a probability of . Let A be event of an even number so
8
A = {2, 4, 6} and B be the event of a prime number so B = {2, 3, 5}. We need P (A or B) = P (A ∪ B).
Look at the Venn diagram below, we see that 2 is common in A and B.
Here P (A) = 38 and P (B) = 38 . Lets pretend that there is nothing
in common b/w A and B, i.e. A ∩ B = ∅. So, P (A ∪ B) =
P (A) + P (B) = 38 + 38 = 86 . Now lets be realistic, the outcome 2 is Sample Space A∪B= A + B - A∩B
both even and prime, so it’s represented twice i.e. its probability is
included in both P (A) and P (B). To calculate the correct P (A∪B), Evens Primes
1
we need to deduct one P (2) = from it. In which case,
8
4, 6 2 3, 5
3 3 1 5
P (A ∪ B) = + − = P (A) + P (B) − P (A ∩ B) =
8 8 8 8
S={2, 3, 4, 5, 6}
Which is the first form of the law of addition in probability, as
mentioned above.

110 / 128
Prob

Probability

Example 25. Another approach

Consider a sample space S = {1, 2, 3, 4, 5, 6, 9, 15}. A number is chosen at random, what is the probability
that the number is an even number or a prime number? (prime number: a natural number greater than
1, has no divisor other than itself and 1.)
Solution:
1
Here again each sample point in the sample space has a probability of . Let A = {2, 4, 6} and
8
B = {2, 3, 5} be just two events, and that there be nothing mentioned about even or prime numbers.
3 3
We know that P (A) = 8 and P (B) = 8 . We need P (A or B) = P (A ∪ B).
Again, lets pretend that there is nothing in common b/w A and B,
i.e. A ∩ B = ∅. So,
P (A or B) = P (A ∪ B) = P (A) + P (B) = 83 + 38 = 68 .
Now lets be realistic, the outcome 2 is represented twice i.e. its Sample Space A∪B= A + B - A∩B
probability is included in both P (A) and P (B). To calculate the
1 Evens Primes
correct P (A ∪ B), we need to deduct one P (2) = from it (in union
8
of two sets you write every common element only once). In which 4, 6 2 3, 5
case,
3 3 1 5
P (A ∪ B) = + − = P (A) + P (B) − P (A ∩ B) = S={2, 3, 4, 5, 6}
8 8 8 8
Which is the first form of the law of addition in probability, as
mentioned above. 111 / 128
Prob

Probability
Example 26. Yet another approach
Consider a sample space S = {1, 2, 3, 4, 5, 6, 9, 15}. A number is chosen at random, what is the probability
that the number is an even number or a prime number? (prime number: a natural number greater than 1, has no
divisor other than itself and 1.)
Solution:
Now just forget everything about prime or even numbers and the laws of probability. Suppose, our
sample space S = {1, 2, 3, 4, 5, 6, 9, 15} is just an ordinary universal set of sets theory for this particular
case. Also, we are given that A = {2, 4, 6} and B = {2, 3, 5} are any two subsets of S. The union of A
and B is A ∪ B = {2, 3, 4, 5, 6}.
What is the probability of AU B with respect to S?
No of elements in A ∪ B
P (A ∪ B) =
No of elements in S
5 Sample Space A∪B= A + B - A∩B
=
8
Evens Primes
Which is same as we calculated in above slides. It sounds like this
4, 6 2 3, 5
approach is easier than the earlier ones, however, as I told you before,
sometimes we are given incomplete information in which case we will
need these rules to find the probability of a particular event. S={2, 3, 4, 5, 6}
What if the sample space was not known to us? Or that we were
only given P (A), P (B) and/or P (A ∩ B) and we were asked to find
P (A ∪ B)? Here we would have been needing these rules...
112 / 128
Prob

Probability

Example 27.
A card is drawn at random from a pack of 52 playing cards. What is the probability that the card is
face card or a club ♣ card.
Solution:
We know that there are 12 face cards (the cards with faces of King, Queen and Jack) and 13 cards for
each suit (♦, ♣, ♥, ♠).

113 / 128
Prob

Probability

Example 27. Cont.

A card is drawn at random from a pack of 52 playing cards. What is the probability that the card is a
face card or a club ♣ card.
Solution: cont.
We know that there are 12 face cards (the cards with faces of King, Queen and Jack) and 13 cards for
each suit (diamond ♦, club ♣, heart ♥, spade ♠).

Now let A represents the event that the card is a ♣, and B be the event that the card is a face card.
12
Since there are 12 face cards out of 52 cards, therefore, P (A) = . Similarly there are 13 cards in a
52
13
club suit therefore, P (B) = . We need the probability of A or B, that’s P (A ∪ B)
52
12 13 25
P (A ∪ B) = P (A) + P (B) = + =
52 52 52
Done?
Not yet!
We know that there are 3 face cards in a club suit too. They were not only counted in face cards’
probability but also in club suit’s probability. We have counted the three face cards of club suit twice.
We need to deduct this probability from the above probability.

114 / 128
Prob

Probability

Example 27. Cont.

A card is drawn at random from a pack of 52 playing cards. What is the probability that the card is a
face card or a club ♣ card.
Solution : cont.
So, we need P (A ∩ B).
We have A ∩ B = {3 face cards}, so
3
P (A ∩ B) =
52
Thus,

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
12 13 3
= + −
52 52 52
22
=
52
That’s it.

115 / 128
Prob

Probability

Example 28.
Now lets a card is drawn at random from a pack of 52 playing cards. What is the probability that the
card is a diamond ♦ card or a club ♣ card.
Solution:
13
Let A be the event that the card is ♦ and B be the event that the card is ♣. Thus, P (A) = and
52
13
P (B) = . We know that there is nothing common in these two events. So, we need to apply the
52
simple law of addition,

P (A ∪ B) = P (A) + P (B)
13 13
= +
52 52
26
=
52
1
=
2
That’s it.

116 / 128
Prob

Probability

Conditional Probability
We often have some information available before finding the probability of a favorable event. For
example,

An experiment consists of rolling a die once. Let A be the event that a 6 occurs. When the die
was thrown, we were told that anything greater than 4 (i.e, 5, 6) has occurred. Let B be the event
that anything greater than 4 (i.e, 5, 6) has occurred. We know that the probability of every face
1 1
of the die is , thus, P (A) = . Now suppose that the die is rolled and we are told that the event
6 6
B has occurred. This leaves only two possible outcomes: 5 and 6. Based on this information, the
1
probability of A becomes ,
2
1
The probability of A given that B has already occurred =
2
What we have done is that we just conditioned the probability of our favorable event on available
information. This is called conditional probability.

117 / 128
Prob

Probability
The mathematical formula that helps in calculating such conditional probabilities is defined as: for
any two events A and B in a sample space S with P (B) > 0, the probability of event A given that
event B has already occurred, written as P (A|B) (here the vertical bar “|” means “given that”), is
given by,
P (A ∩ B)
P (A|B) = , provided P (B) > 0
P (B)
If P (B) = 0, the probability becomes undefined.
Now lets solve the above example with this formula.
Using the same notations/assignments the first thing that becomes clear is that
Number of outcomes in B
P (B) =
Number of outcomes in S
n{5, 6} 2 1
= = =
n{1, 2, 3, 4, 5, 6} 6 3

Now is there anything common in A and B? Yes, there is one outcome {6} in common. So
1
P (A ∩ B) = , thus using the formula we have,
6
1
P (A ∩ B) 6 3 1
P (A|B) = = 1 = =
P (B) 3
6 2

118 / 128
Prob

Probability
Example 29.
We toss a fair coin three successive times. We wish to find the conditional probability that more heads
occurs than tails, GIVEN that the first toss resulted in a head.
Solution:
Let A and B are the events where
A = {more heads than tails come up}, B = {1st toss is a head}.
The sample space for this experiment is,
S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }.
The first head occurs at B = {HHH, HHT, HT H, HT T }, so its probability is
4
P (B) =
8
Now the event A consists of the four elements HHH, HHT, HT H, T HH, so, event A ∩ B consists of
the three elements HHH, HHT, HT H so its probability is
3
P (A ∩ B) =
8
Thus, the conditional probability is
3
P (A ∩ B) 8 3
P (A|B) = = 4 =
P (B) 8
4
119 / 128
Prob

Probability

Example 30. another approach

We toss a fair coin three successive times. We wish to find the conditional probability that more heads
occurs than tails, GIVEN that the first toss resulted in a head.
Solution:
Let A and B are the events
A = {more heads than tails come up }, B = {1st toss is a head}.
The sample space for this experiment is,
S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T }.
Since the first head occurs at B = {HHH, HHT, HT H, HT T }, so its now our sample space; that
is now our sample space is reduced to B. Now looking at this reduced sample space we see that the
outcomes of our interest are HHH, HHT, HT H, where the number of heads is more than that of tails.
We can express this set as A|B = {HHH, HHT, HT H}. So the probability of more heads than tails
GIVEN that a head has occured at first toss is,

n(A|B)
P (A|B) =
n(B)
3
=
4
120 / 128
Prob

Probability
Example 31. A man tosses two fair dice. What is the conditional probability that the sum of the two
dice is 6, given that two different numbers appear.
Solution: For two dice, the sample space is 
 (1, 1)2 (1, 2)3 (1, 3)4 (1, 4)5 (1, 5)6 (1, 6)7 
(2, 1)3 (2, 2)4 (2, 3)5 (2, 4)6 (2, 5)7 (2, 6)8

 


 

(3, 1)4 (3, 2)5 (3, 3)6 (3, 4)7 (3, 5)8 (3, 6)9
 
S=

 (4, 1)5 (4, 2)6 (4, 3)7 (4, 4)8 (4, 5)9 (4, 6)10 

 (5, 1)6 (5, 2)7 (5, 3)8 (5, 4)9 (5, 5)10 (5, 6)11

 

 
(6, 1)7 (6, 2)8 (6, 3)9 (6, 4)10 (6, 5)11 (6, 6)12


Let A denotes that the sum is 6, thus

n(A) 5
A = {(1, 5)6 , (2, 4)6 , (3, 3)6 , (4, 2)6 , (5, 1)6 } =⇒ P (A) = =
n(S) 36
Let B denotes that two different numbers appear, thus
n(B) 30 5
B = {all the outcomes of S except the those on diagonal} =⇒ P (B) = = =
n(S) 36 6
4 1
Now A ∩ B = {(1, 5)6 , (2, 4)6 , (4, 2)6 , (5, 1)6 } =⇒ P (A ∩ B) = = ,
36 9
thus
1
P (A ∩ B) 2
P (A|B) = = 95 =
P (B) 6
15
Which is the required conditional probability of getting a sum of 6 at two dice, given that different
numbers appear on them.
121 / 128
Prob

Probability
Note: The probability that an event does not occur is 1 minus the probability that the event does
occur. If an event occurs in (say) 70% of all trials, it fails to occur in the other 30%. The probability
that an event occurs and the probability that it does not occur always add to 100%, or 1.
The complement of any event A is the event that A does not occur, written as Ac (or A0 ) . The
complement rule states that
P (Ac ) = 1 − P (A)
Multiplication law
This is another look of conditional probability. Using the previous law of conditional probability
we could find the probability of a particular event (say A) given another event (say B) has already
occurred. Using this current rule, we can calculate the probability of both the events occurring at
the same time that’s (A ∩ B). Thus, we can also write the conditional probability rule as,
P (A ∩ B)
P (A|B) = =⇒ P (A ∩ B) = P (A|B)P (B) = P (B|A)P (A)
P (B)
By this rule we can find the probability of the simultaneous occurrence of two or more events. In
the addition rule we used OR and here we use AND. For example in the case of addition rule we
said “what is the probability that A occurs or B occurs”. Whereas here we say that “what is the
probability that A and B occur”. We also notice that here we use the definition of conditional
probability because in most experiments the events occur in such a way that the occurrence of one
event affects the occurrence of other events. That’s two or more events defined on the same sample
space usually affect the occurrence of each other. Lets see it in an example on next slide.
122 / 128
Prob

Probability
Example 32. Two items are drawn at random from a box, which contains 10 items 4 of which are
defective. What is the probability that one item is defective and the other is good.
Solution: Lets assume that the first item drawn is good and the second is defective. Let A denotes that
a good item is drawn and B denotes that a defective is selected.
We know that out of 10 items there, 6 good items so the probability of A is
6
P (A) =
10
Now since one good item is selected, we are left with 9 items in the box, the probability of selecting a
defective item (event B) given that a good item (event A) has already been selected is now a conditional
probability, P (B|A). Which is,
4
P (B|A) =
9
Hence the probability that one item is good and the other is defective is given as,

P (B ∩ A) = P (A)P (B|A)
6 4
= ×
10 9
4
=
15

123 / 128
Prob

Probability
Independent and dependent events.
Two events A and B are said to be independent if and only if,
P (B|A) = P (B) or P (A|B) = P (A)
Otherwise A and B are dependent.
Thus we can say that two events A and B are said to be independent if and only if,
P (A ∩ B) = P (A)P (B)
Therefore the probability that two independent events will both occur, we simply find the product
of their individual probabilities.
Example 33. A town has two fire engines operating independently. The probability that a specific fire
engine is available when needed is 0.96. Find the following probabilities:
1 Probability that neither is available when needed.

2 Probability that both are available when needed.

Solution: Let A and B denote the availability of the two fire engines. Thus Ā and B̄ are events of non
availability of the fire engines. We have P (A) = P (B) = 0.96 and hence P (Ā) = P (B̄) = 0.04
(∵ Ā = 1 − A ⇒ P (Ā) = 1 − P (A)).
P (neither is available) = P (Ā ∩ B̄) = P (Ā)P (B̄) = 0.04 × 0.04 = 0.0016
P (both are available) = P (A ∩ B) = P (A)P (B) = 0.96 × 0.96 = 0.9216

124 / 128
Prob

Probability

Bayes’ Rule: This is a rule which revolutionized the theory of statistics. Before learning it, lets
learn about the total probability first.
Total Probability Theorem
Let A1 , A2 , A3 ,...,An be mutually exclusive and exhaustive events that form a partition of the sample
space. Assume that P (Ai ) > 0, for all i. Then for any event B, we have

P (B) = P (A1 ∩ B) + · · · + P (An ∩ B)

= P (A1 )P (B|A1 ) + · · · + P (An )P (B|An )
Xn
= P (Ai )P (B|Ai )
i=1

125 / 128
Prob

Probability

Now the Bayes’ rule is defined as,

Total Probability Theorem

Let A1 , A2 , A3 ,...,An be mutually exclusive and exhaustive events that form a partition of the sample
space. Assume that P (Ai ) > 0, for all i. Then for any event B such that P (B) > 0, we have

P (Ai )P (B|Ai )
P (Ai |B) =
P (B)
P (Ai )P (B|Ai )
=
P (A1 )P (B|A1 ) + · · · + P (An )P (B|An )
P (Ai )P (B|Ai )
= Pn
i=1 P (Ai )P (B|Ai )

Bayes’ rule enables us to combine our own belief about certain event Ai with the observed event
B. Here P (Ai ) are called a priori probabilities, whereas P (Ai |B) are called the a posteriori proba-
bilities. A prior probability is an initial probability value originally obtained before any additional
information is obtained. A posterior probability is a probability value that has been revised by
using additional information that is later obtained.

126 / 128
Prob

Probability

Example 34. An aircraft emergency locator transmitter (ELT) is a device designed to transmit a signal
in the case of a crash. The Altigauge Manufacturing Company makes 80% of the ELTs, the Bryant
Company makes 15% of them, and the Chartair Company makes the other 5%. The ELTs made by
Altigauge have a 4% rate of defects, the Bryant ELTs have a 6% rate of defects, and the Chartair ELTs
have a 9% rate of defects.
¬ If an ELT is randomly selected from the general population of all ELTs, find the probability that it
was made by the Altigauge Manufacturing Company.
­ If a randomly selected ELT is then tested and is found to be defective, find the probability that it
was made by the Altigauge Manufacturing Company.
Solution:
Lets use the following notation:
A = ELT manufactured by Altigauge
B = ELT manufactured by Bryant
C = ELT manufactured by Chartair
D = ELT is defective
D̄ = ELT is not defective (or it is good)
Now
¬ If an ELT is randomly selected from the general population of all ELTs, the probability that it was
made by Altigauge is 0.8 (because Altigauge manufactures 80% of them).

127 / 128
Prob

Probability

­ If we now have the additional information that the ELT was tested and was found to be defective,
we want to revise the probability from part ¬ in the light of the new information. We want to find
the value of P (A|D), which is the probability that the ELT was made by the Altigauge company
given that it is defective. Based on the given information, we know these probabilities:

P (A) = 0.80 because Altigauge makes 80% of the ELTs

P (B) = 0.15 because Bryant makes 15% of the ELTs
P (C) = 0.05 because Chartair makes 5% of the ELTs
P (D|A) = 0.04 because 4% of the Altigauge ELTs are defective
P (D|B) = 0.06 because 6% of the Bryant ELTs are defective
P (D|C) = 0.09 because 9% of the Chartair ELTs are defective

Here is Bayes’ theorem extended to include three events corresponding to the selection of ELTs from
the three manufacturers (A, B, C):
P (A)P (D|A)
P (A|D) =
P (A)P (D|A) + P (B)P (D|B) + P (C)P (D|C)
0.80 × 0.04
=
0.80 × 0.04 + 0.15 × 0.6 + 0.05 × 0.09
= 0.0703
128 / 128