Chapter2

PROBABILITY
2.1 Introduction
Probability theory is a branch of mathematics that has been developed to deal with
uncertainty. The theory of probability has its origin in the games of chance related to
gambling such as tossing of a coin, throwing of a die, drawing cards from a pack of cards
etc. Starting with games of chance, probability has become one of the basic tools of
statistics. The knowledge of probability theory makes it possible to interpret statistical
results, since many statistical procedures involve conclusions based on samples.
The concept of probability is frequently encountered in everyday communication. For
example, we may hear a physician say that a patient has a 50-50 chance of surviving a
certain operation. Another physician may say that he is 95 percent certain that a patient has a
particular disease. Thus we are accustomed to measuring the probability of the occurrence of
some event by a number between zero and one. The more likely the event, the closer the
number is to one; and the more unlikely the event, the closer the number is to zero. An event
that cannot occur has a probability of zero, and an event that is certain to occur has a
probability of one.

2.2 Basic Definitions.

We begin our development of probability by defining some basic key words.
(A) Experiment
The term experiment refers to the process of obtaining an observed result of some
phenomenon, and a performance of an experiment, is called a trial of the experiment. An
observed result, on a trial of the experiment, is called an outcome. This terminology is rather
general, our interest will be in situations, called random experiments, where there is
uncertainty about which outcome will occur when the experiment is performed. By a random
experiment we will mean any procedure that:
1- all possible outcomes can be completely defined in advance.
2- can be repeated, theoretically, any number of times under identical conditions.
3- any performance of the experiment results in an outcome that is not known to certain
occur in advance.
For example, a coin is tossed, assuming that the coin does not land on the side, there are two
1
possible outcomes of the experiment: heads (denoted by H) and tails (denoted by T). On any
performance of this experiment one does not know what the outcome will be. The coin can be
tossed as many times as desired.

(B) Sample Space

The Sample Space, denoted by S, associated with a random experiment is the set of all
possible outcomes of that experiment. Note that one and only one of the possible outcomes
will occur on any given trial of the experiment.

Example 2.1
An experiment consists of selecting two students, and the observed gender of each
student is of interest. The set of all possible outcomes may be represented by the sample
space;
S = { MM , MF , FM , FF }
If we are interested in the total number of males obtained, an appropriate sample space could
then be written as
S ={ 0 , 1 , 2 }
Thus, different sample spaces may be appropriated for the same experiment, depending on
the characteristic of interest.
In selecting three students, the sample space consists of 8 outcomes,
S = { MMM , MMF , MFM , FMM , MFF , FMF , FFM , FFF }

First selection Second selection Third selection

M MMM
M
F MMF
M
M MFM
F
F MFF
M M FMM
F F FMF
F M FFM
F FFF

Note that, in selecting n students, the sample space consists of 2n outcomes.

2 S
(C) Event
An event A is a collection of some of the possible
outcomes of the random experiment.
In the language of the set theory, A is a subset of the A
sample space S i.e.
AS

In particular, we may regard each individual outcome as an event; since each outcome is a
subset of S. (These are sometimes called elementary events). Also, the null set  (i.e. the set
which consists of no outcomes) and the complete sample space S are each special cases of
subsets of S, and thus both  and S may be regarded as events.  is known as the
impossible event and S as the certain event.
For example, in tossing a die, S ={1, 2, …, 6}, we might consider the following as typical
events;
A1 : Score is 1, i.e. {1}
A2 : Score is even, i.e. {2,4,6}
A3 : Score is less than 5, i.e. {1,2,3,4}.

(D) Occurrence of an event.

We say that an event A had occurred if the outcome of the experiment belongs to the
set A. For example, if the outcome in throwing a die was a score 6, then, as described above,
A2 occurred, but A1 did not occur and A3 did not occur.
S

(E) Union of Events.

Given two events A & B, the union of A & B, written
A B
AB
is defined as the set of outcomes which belong to either A
or B or both. Thus, the event (A  B) occurs whenever A or
B occurs or both occur. AB

3
(F) Intersection of events.
S
The intersection of A &B written as A  B is defined
as the set of outcomes which belong to both A and B. The
A B
set (A  B) occurs when both A & B occur.

AB

(G) Mutually exclusive Events S

Two events A and B are called mutually exclusive if
both A and B cannot occur in the same time i.e. if
A B
AB= 
so that A and B are disjoint.
A  B= 

(H) Complimentary event

Given an event A, the complimentary, A (or A/), is defined as A
that subset of S which do not belong to A. Thus the event A occurs
A
whenever A does not occur and vice versa.

(I) Equally Likely Events.

If no outcome of a random experiment is any more likely to occur than any other,
The outcomes are said to be equally likely.

2.3 Classical Probability

If a random experiment can result in N mutually exclusive and equally likely
outcomes, n(A) of which corresponds to the occurrence of some event A, then the
probability that the event A will occur, denoted by P(A), will defined as the ratio n(A)/N,
symbolically.
n(A) number of outcomes belong to A
P(A) = = (1.1)
N total number of outcomes belong to S

The relative frequency approach to probability depends on the repeatability of some

4
process and the ability to count the number of repetitions, as well as the number of times that
some event of interest occurs. We may define the probability of observing some
characteristic, A, of an event as follows:
If some process is repeated a large number of times, N, and if some resulting event
with the characteristic, A, occurs n times, the relative frequency of occurrence of A, n/N, will
be approximately equal to the probability of A.

Example 2.2
Throw an unbiased coin three times and observe the sequence of heads and tails. Here the
sample space is the collection of all possible sequences,
S = { HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}
Since the outcomes are equally likely and mutually exclusive then the probability of each
outcome is 1/8. Let A be the event that two or more heads appear consecutively, and B that
all the tosses are the same. Then
A = { HHH, HHT, THH } and B = { HHH, TTT }

Therefore
n( A) 3 n(B) 2 1
P(A) = = , P (B) = = =
N 8 N 8 4
n( AB) 1
P(AB)= = P ( { HHH } ) =
N 8
and
n( A B ) 1
P ( A  B )= = P ( { HHH , HHT , THH , TTT } ) =
N 2
Example 2.3
There are 15 balls, numbered from 1 to 15 in a bag. If a person selects one at random, what is
the probability that the number printed on the ball will be
i- a prime number greater than 5.
ii- an odd number less than 11.
Solution
Let A1 = { a prime number greater than 5 }
A2 = { An odd number less than 9 }
then
A1 = {7, 11,13}, A2 = {1, 3, 5, 7}, N = 15 , n( A1) = 3 , n(A2) = 4
5
 n ( A1 ) 3 1 n ( A 2) 4
 P( A1 ) = = = and P ( A 2 ) = =
N 15 5 N 15
Question: Find P(A1  A2) and P(A1  A2) ?

Example 2.4
A fair die is tossed twice. What is the probability that the sum of the upturned faces is 9?
Solution
The sample space for this experiment is
S = { (i , j) : i = 1, 2, ...,6 ; j = 1, 2, ...,6 }
Since the die is fair (unbiased), each of the 36 possible outcomes would be equally
likely to occur. If A represents the event that the sum of the upturned faces is 9 then,
A = {(3, 6) , (4, 5) , (5, 4) , (6, 3)}
hence,
P(A) = 4/36 = 1/9

2.4 Axiomatic Approach to Probability

For a given experiment, S denotes the sample space, a set real function that associates
a real value P(A) with each event A is called a probability function (or a probability
measure), and P(A) is called the probability of A, if the following properties are satisfied:

Axiom I. P(A) ≥ 0, for every event A  S.

Axiom II. P(S) = 1,

Axiom III. If A and B are two mutually exclusive events of S, then

P(A  B) = P(A) + P(B)

These axioms all seem to agree with our intuitive concept of probability and these few
axioms are sufficient to allow a mathematical structure to be developed.
Note, if A and B are two mutually exclusive events of S, then
P (A  B) = P (A) + P (B)

6
Example 2.5
Three horses A, B and C are in a race. A is twice as likely to win as B and B is twice
as likely to win as C. What is their respective probabilities of winning, i.e. P(A), P(B) and
P(C)?
Solution
Let P(C) = p,
since B is twice as likely to win as C, then
P(B) = 2p,
and since A is twice as likely to win as B, then
P(C) = 4p
Now the sum of the probabilities must be one (by property II), hence
P(A) + P(B) + P(C) = 1,
i.e.
4p + 2p + p = 1
Therefore, p=1/7, and accordingly
P(A) = 4p = 4/7, P(B) = 2p =2/7 and P(C) = p = 1/7.

Example 2.6
A die is loaded in such a way that each odd number is twice as likely to occur as each
even number. Find P(G), where G is the event that a number greater than 3 occurs on a
single roll of the die.
Solution
The sample space is S = {1, 2, 3, 4, 5, 6}. Hence, if we assign probability p to each
even number and probability 2p to each odd number, we find that
2p + p + 2p + p + 2p + p = 9 p = 1
1
in accordance with axiom II. It follows that p =
9
1 2 1 4
P (G) = + + =
9 9 9 9
If a sample space is accountably infinite, probabilities will have to be assigned to the
individual outcomes by means of a mathematical rule, preferably by means of a formula or
equation.

7
2.5 Some Rules of Probability
Based on the three axioms of probability, we can derive many other rules which have
important applications. Among them, the next rules are immediate consequences of the
axioms.
1. P (  ) = 0
2. P(A ) = 1 - P(A)
3. If A and B are events in S and A  B, then P(A)  P(B) ⇒ 0  P(A)  1.
4. If A and B are any two events in S, then the probability of occurrence of A and non-
occurrence of B is given by
P( A∩ B ) = P( A \ B ) = P(A) - P(A ∩ B)
5. If A and B are any two events in a sample space S, then
P(A  B) = P(A) + P(B) - P(A∩B)

6. Probability of neither A nor B occurs is

________
P(A B ) = P(A B) = 1 − P (A B)

Example 2.7
In a certain population of women 4% have had breast cancer, 20% are smokers and
3% are smokers and have had breast cancer. A woman is selected at random from the
population. What is the probability that:
a- She has had breast cancer or smokes?
b- She has had breast cancer and she is not a smoker?
Solution
If C is the event "she has breast cancer" and M the event "she is a smoker ", then we
have P(C) = 0.04, P(M) = 0.20 and P(C∩M) = 0.03.
a- P (C M) = P(C) + P(M) - P(C M) = 0.04 + 0.20 - 0.03 = 0.21
b- P( C∩ M ) = P(C) - P(C∩M) = 0.01

Example 2.7
The probability that a student passes mathematics is 0.75, and passes English is 0.85. If
the probability of passing at least one course is 0.9, what is the probability that he will pass
8
both courses?
Solution
If M is the event "passing mathematics" and E the event "passing English", then by
transposing the terms in the Additive Rule, we have
𝑃(𝑀 ∪ 𝐸) = 𝑃(𝑀) + 𝑃(𝐸) − 𝑃(𝑀 ∩ 𝐸)
0.9 = 0.75 + 0.85 − 𝑃(𝑀 ∩ 𝐸)
From which the probability that a student will pass both courses is
P (M ⋂ E) = 0.7
Questions:
what is the probability that he will fail both courses?
what is the probability that he will pass one and only one course?

2.6 Conditional Probability

A major objective of probability modeling is to determine how likely it is that an
event A will occur when a certain experiment is performed. However, there are numerous
cases in which the probability assigned to A will be affected by knowledge of the occurrence
or nonoccurrence of another event B. In such an example we will use the terminology
"conditional probability of A given B", and the notation P(A|B) will be used to distinguish
between this new concept and ordinary probability P(A).
As an illustration, consider the event B of getting a 4 when a fair die is tossed. Based
1
on the sample space S = {1,2,3,4,5,6} the probability of B occurring is . Now suppose that
6
it is known that the toss of the die resulted in a number greater than 3. We are now dealing
with a reduced sample space A= {4,5,6}, which is a subset of S. Relative to the sample space
A, we find that the probability that B occurs is 1/3. This example illustrates that events may
have different probabilities when considered relative different sample spaces, and we would
write
1
P(B|A )= ,
3
which can also be written
1 1 / 6 P(A B)
P(B | A) = = =
3 3/6 P (A)
where P(A∩B) and P(A) are found from the original sample space S. In other words, a
9
conditional probability relative to a subspace A of S may be calculated directly from the
sample space S itself.

Definition
If A and B are any two events in S and P(A) > 0, the conditional probability of B given
A is
P(A B)
P(B | A) = (2.3)
P(A)
We may illustrate the concept of conditional probability by referring the table in the
following example.

Example 2.8
A box contains 100 microchips, some of which were produced by factory 1 and the rest by
factory 2. Some of the microchips are defective and some are good (non defective). An
experiment consists of choosing one microchip at random from the box and testing
whether it is good or defective. Let
A be the event "obtaining a defective microchip"; consequently,
B be the event "the microchip was produced by factory 1" then
A is the event "obtaining a good microchip".
B is the event "the microchip was produced by factory 2".
Table 2.1 gives the number of microchips in each category.

Table 2.1 Numbers of defective and nondefective

microchips from two different factories
B B Totals

A 15 5 20
A 45 35 80

Totals 60 40 100

The probability of obtaining a defective microchip is

n ( A ) 20
P( A)= = = 0.20
n ( S ) 100
Now suppose that each microchip has a number stamped on it that identifies which

10
factory produced it. Thus, prior to testing whether it is defective, it can be determined
whether B has occurred (produced by factory 1) or B/ has occurred affects likelihood that a
defective microchip is selected, and the use of conditional probability is appropriate. For
example, if the event B has occurred, then the only microchips we should consider are
those in the first column of Table 2.1, and the total number is n(B) = 60. Furthermore, the
only defective chips to consider are those in both the first column and the first row, and
the total number is n(AB) = 15. Thus, the conditional probability of A given B is
n ( A  B ) 15
P( A/ B)= = = 0.25
n(B) 60
Notice that if we divide both the numerator and denominator by
n(S) = 100, we can express conditional probability in terms of some ordinary unconditional
probabilities,
n( AB)/n(S) P( AB)
P( A/ B)= =
n(B)/n(S) P(B)

Example 2.9
The probability that a student, selected at random from certain college, will pass
Mathematics I is 0.8 and will pass in both Mathematics I and Mathematics II is 0.5. What is
the probability that he will pass Mathematics II if it is known that he had passed
Mathematics I?
Solution
If M1 is the event "passing Mathematics I " and M2 the event "passing Mathematics II
", then
𝑃(𝑀1 ∩ 𝑀2) 0.5
𝑃(𝑀2|𝑀1) = = = 0.625
𝑃(𝑀1) 0.8

Multiplication Rule
If we multiply the expressions on both sides of (1.3) by P(A), we obtain the following
multiplication rule: if A and B are two events in S, then
P (A B) = P (A) P(B | A) = P (B) P (A | B) (2.4)

11
Example 2.10
If we randomly pick two television tubes in succession from a shipment of 240
television tubes of which 15 are defective, what is the probability that they will both be
defective?
Solution
If we assume equal probabilities for each selection (which is what we mean by
"randomly" picking the tubes), the probability that the first tube will be defective is
15
P(D1 ) =
240
and the probability that the second tube will be defective given that the first tube is defective
is
14
P(D2 | D1 ) =
239
Thus, the probability that both tubes will be defective is
15 14 7
P (D1  D2 ) = P (D1 ) P (D2 | D1 ) = . =
240 239 1 , 912
This assumes that we are sampling without replacement, namely, that the first tube is not
replaced before the second tube is selected.
Theorem 1.7 can easily be generalized so that it applies to more than two events; for
instance, for three events we have
P( A  B  C ) = P( A ) . P( B / A ) . P( C / A  B )
Example 2.11
A box of fuses contains 20 fuses, of which 5 are defective. If 3 of the fuses are
selected at random and removed from the box in succession without replacement, what is the
probability that all three fuses are defective?
Solution
If D1 is the event that the first fuse is defective, D2 is the event that the second fuse is
defective, and D3 is the event that the third fuse is defective, then
5 4 3
P ( D1 ) = , P ( D2 | D1 ) = , P ( D3 | D2 D1 ) =
20 19 18
and substitution into the formula yields

12
 5 4 3 1
P( D1∩D2∩ D3 ) = P( D1 D2 D3 ) = . . = .
20 19 18 114
2.7 Independent Events
Informally speaking, two events A and B are said to be independent if the occurrence
or nonoccurrence of either one does not affect the probability of the occurrence of the other.
Symbolically, two events A and B are independent if
P(B /A) = P(B ) and P (A/B) = P(A)
and it can be shown that either of these equalities implies the other when both of the
conditional probabilities exist, namely, when neither P(A) nor P(B) equals zero.
Now, if we substitute P(B) for P(B/A) into formula (1.3), we get
P(A B) = P(A) P(B | A)

= P(A)P (B)
and this is used as the formal definition of independence.

Definition
Two events A and B are independent iff
P(A  B ) = P (A ) P ( B )

otherwise, A and B are dependent.

It can be shown that if A and B are events such that P(A) > 0 and P(B) > 0, then A and
B are independent iff either of the following holds
P ( B / A ) = P ( B ) and P ( A / B ) = P ( A )

Example 2.12
Suppose A and B are events such that P(A) = 0.2, P(B) = 0.6 and P(A or B) = 0.68.
Are A and B independent events?
Solution
Note that P(A∩B) = P(A) + P(B) - P(A∪B)= 0.2 + 0.6 – 0.68 = 0.12
P(A) P(B) = (0.2)(0.6) = 0.12
Since P(A∩B) = P(A) P(B), we conclude that the events A and B are independent.

13
Example 2.13
A fair coin is tossed three times. If A is the event that a head occurs on each of the
first two tosses, B is the event that a tail occurs on the third toss, and C is the event that
exactly two tails occur in the tree tosses, show that
a- events A and B are independent;
b- events B and C are dependent.
Solution
Since the coin is fair then the eight possible outcomes, HHH, HHT, HTH, THH,
HTT, THT, TTH, and TTT, are equally likely, and
A = {HHH, HHT}
B = {HHT, HTT, THT, TTT}
C = {HTT, THT, TTH}
A  B = {HHT}
B  C = {HTT, THT}
the assumption that the eight possible outcomes are all equiprobable yields
P(A)=1/4, P(B)=1/2, P(C)=3/8, P(AB)=1/8 and P(BC)= 1/4.
a- Since P(A). P(B) = 1/4.1/2 = P(A  B), the events A and B are independent.
c- Since P(B). P(C) = 1/2 .3/8  P(B  C), the events B and C are not independent.
In connection with the definition of independence, given above, it can be shown that if A
and B are independent, then so are A and , and B, and and . For instance:

Definition
Events A1, A2, ...., Ak are independent iff the probability of the intersection of any 2, 3, ...
or k of these events equals the product of their respective probabilities.
For three events A, B, and C, for example, independence requires that;
P(A ⋂B) = P(A) . P(B)
P(A ⋂C) = P(A) . P(C)
P(B ⋂C) = P(B) . P(C)
and
P(A ⋂B ⋂C) = P(A) . P(B) . P(C)

14
It is of interest to note that three or more events can be pairwise independent without
being independent.
A common example of dependent events occurs in connection with repeated sampling
without replacement. In example 2.11 we considered the results of drawing fuses in
succession. Suppose that, instead, the outcome of the first fuse is tested and then the fuse is
replaced in the box before the second draw is made and the second fuse is replaced in the
box before the third draw is made. This type of sampling is referred to as sampling with
replacement, and it would reasonable to assume that the draws are independent trials. In
this case the probability that all three fuses are defective is
𝟓 𝟓 𝟓 𝟏
𝐏(𝐀 ∩ 𝐁 ∩ 𝐂) = 𝐏(𝑨)𝐏(𝑩)𝐏(𝐂) = . . =
𝟐𝟎 𝟐𝟎 𝟐𝟎 𝟔𝟒
2.8 Total Probability and Bayes' Theorem

There are many situations where the outcome of an experiment depends on what
happens in various intermediate stages. Suppose, in general the intermediate stage permits
k different alternatives (whose occurrence is denoted by B1, B2, ..., Bk). The events B1, B2,
..., Bk, in this case, constitute a partition of the sample space S (i.e. the B's are pairwise
mutually exclusive and their union equals S ). To find the probability of occurrence of an
event A that can occurs with one of the B's events, the following theorem is required ,
sometimes called the law or the rule of total probability.

Theorem 2.8: Total Probability

If the events B1, B2, ..., Bk constitute a partition of the sample space S and P(Bi) > 0 for
i = 1, 2, ..., k, then for any event A in S

Proof

15
Since the events B1, B2, ..., Bk are exhaustive (i.e their union equals S), then

Since the events B1, B2, ..., Bk are mutually exclusive, then A ⋂ B1 , A ⋂ B2 , ..., A ⋂ Bk are
also mutually exclusive. It follows that;

and the theorem results from applying (2.4) to each term in this summation.
Example 2.14
The members of a consulting firm rent cars from three rental agencies: 60% from agency 1,
30 % from agency 2, and 10 % from agency 3. If 9% of the cars from agency 1 need a
tune-up, 20 % of the cars from agency 2 need a tune-up, and 6% of the cars from agency 3
need a tune-up, what is the probability that a rental car delivered to the firm will need a
tune-up?
Solution
If A is the event that the car needs a tune-up, and B1, B2, and B3 are the events that the car
comes from rental agencies 1, 2, or 3, we have
P(B1) = 0.60, P(B2) = 0.30, P(B3) = 0.10, and
P(A|B1) = 0.09, P(A|B2) = 0.20, and P(A|B3) = 0.06.
Substituting these values into the formula of Theorem 2.8, we get
P(A) = (0.60)(0.09) + (0.30)(0.20) + (0.10)(0.06) = 0.12
Thus, 12% of all the rental cars delivered to this firm will need a tune-up.

16
Tree diagram

It is sometimes helpful to illustrate this result with a tree diagram given in figure 2.1. The
probability associated with branch Bi is P(Bi), and the probability associated with each
branch labeled A is a conditional probability P(A|Bi ), which may be different depending
on which branch, Bi , it follows. In order for A to occur, it must occur jointly with one and
only one of the events Bi .
With reference to the preceding example, suppose that we are interested in the following
question: If a rental car delivered to the consulting firm needs a tune-up, what is the
probability that it came from rental agency 2? To answer questions of this kind, we need
the following theorem, called Bayes' theorem:
Theorem 2.9 : Bayes' Formula
If B1 , B2 , ..., Bk constitute a partition of the sample space S and P(Bi) > 0 for i = 1, 2,
..., k, then for any event A in S such that P(A) > 0,

In words, the probability that event A was reached via the rah branch of the tree
diagram of Figure 2.1, given that it was reached via one of its k branches, is the ratio of the
probability associated with the rah branch to the sum of the probabilities associated with all
k branches of the tree.
Proof

17
Writing in accordance with the definition of conditional probability, we have only to
substitute P( Br ) . P( A | Br ) for P( A ∩ Br ) and the formula (2.5) for P( A ).

Example 2.15
With reference to Example 2.14, if a rental car delivered to the consulting firm needs a
tune-up, what is the probability that it came from rental agency 2?
Solution
Substituting the probabilities given in Example 2.14 into the formula of Theorem 2.9, we
get

Observe that although only 30% of the cars delivered to the firm come from agency 2, 50%
of those requiring a tune-up come from that agency.
Example 2.16
We are given three similar boxes of microchips as follows:
Box B1 contains 20 microchips of which 5 are defective,
Box B2 contains 35 microchips of which 7 are defective,
Box B3 contains 40 microchips of which 5 are defective.
A box selected at random, then a microchip is selected at random from the box. If the
component obtained is defective, find the probability that it came from box2.

18