Chapter 2

PROBABILITY
2.1 Introduction
The theory of probability had its origin in gambling and games of chance in mid-
eighteenth-century. The theory thus developed for "heads or tails" or "red or black" soon
found applications in situations where the outcomes were "boy or girl", " life or death ",
"pass or fail" and scholars began to apply probability theory to actuarial problems and
some aspects of the social sciences. Later, probability and statistics were introduced into
physics by L. Boltzmann, J. Gibbs, and J. Maxwell, and in this century they have found
applications in all phases of human endeavor which in some way involve an element of
uncertainty or risk. The names which are connected most prominently with the growth of
probability and mathematical statistics in the first half of this century are those A. N.
Kolmogorov, R. A. Fisher, J. Neyman, E. S. Pearson, and A. Wald.
The concept of probability is frequently encountered in everyday communication.
For example, we may hear a physician say that a patient has a 50-50 chance of surviving a
certain operation. Another physician may say that he is 95 percent certain that a patient
has a particular disease. Thus we are accustomed to measuring the probability of the
occurrence of some event by a number between zero and one. The more likely the event,
the closer the number is to one; and the more unlikely the event, the closer the number is
to zero. An event that cannot occur has a probability of zero, and an event that is certain
to occur has a probability of one.

2.2 Basic Definitions.

We begin our development of probability by defining some basic key words.
(A) Experiment
The term experiment refers to the process of obtaining an observed result of some
phenomenon, and a performance of an experiment, is called a trial of the experiment. An
observed result, on a trial of the experiment, is called an outcome. This terminology is
rather general; our interest will be in situations, called random experiments, where there
is uncertainty about which outcome will occur when the experiment is performed. By a
random experiment we will mean any procedure that:
1- all possible outcomes can be completely defined in advance.
2- can be repeated, theoretically, any number of times under identical conditions.

- 23 -
 3- any performance of the experiment results in an outcome that is not known to
certain occur in advance.
For example, a coin is tossed, assuming that the coin does not land on the side, there
are two possible outcomes of the experiment: heads (denoted by H) and tails (denoted by
T). On any performance of this experiment one does not know what the outcome will be.
The coin can be tossed as many times as desired.

(B) Sample Space

The Sample Space, denoted by S, associated with a random experiment is the set of
all possible outcomes of that experiment. Note that one and only one of the possible
outcomes will occur on any given trial of the experiment.

Example 2.1
An experiment consists of tossing two coins, and the observed face of each coin is
of interest. The set of all possible outcomes may be represented by the sample space;
S = { HH , HT , TH , TT }
An alternate way of representing such a sample space is to list all possible ordered pairs of
the numbers 1 and 0,
S ={ (1,1) , (1, 0 ) , ( 0 ,1) , ( 0 , 0 ) }
where, for example, (1,0) indicates that the first coin landed heads up and the second coin
landed tails up. If we are interested in the total number of heads obtained from the two
coins, an appropriate sample space could then be written as
S ={ 0 , 1 , 2 }
Thus, different sample spaces may be appropriated for the same experiment, depending on
the characteristic of interest.
In tossing the coin three times (or 3 coins), the sample space consists of 8 outcomes,
S = { HHH , HHT , HTH , THH , HTT , THT , TTH , TTT }
Note that, in tossing the coin n times the sample space consists of 2n outcomes.

Example 2.2
If a coin is tossed repeatedly until a head occurs, then the natural sample space is S
= {H, TH, TTH ...}. If one is interested in the number of tosses required to obtain a head,
then a possible sample space for this experiment would be the set of all positive integers,
S* = {1, 2, 3 ...},
and the outcomes would correspond directly to the number of tosses required to obtain the
first head. It will be shown in the next chapter that an outcome corresponding to a

- 24 -
sequence of tosses in which a head is never obtained need not be included in the sample
space.

Example 2.3
A light bulb is placed in service and the time of operation until it burns out is
measured. At least conceptually, the sample space for this experiment can be taken to be
the set of nonnegative real numbers,
S = { t : 0  t < }
Note that if the actual failure time could be measured only to the nearest hour, then
the sample space for the actual observed failure time would be the set of nonnegative
integers,
S* = { 0, 1, 2, 3, ... }.
Even though S* may be the observable sample space, one might prefer to describe the
properties and behavior of light bulbs in terms of the conceptual sample space S. In cases
of this type, the discreetness imposed by measurement limitations is sufficiently
negligible that it can be ignored, and both the measured response and the conceptual
response can be discussed relative to the conceptual sample space S.
A sample S space is said to be finite if it consists of a finite number of outcomes,
say
S = {e1 , e2 , ..., eN }
and it is said to be countably infinite if its outcomes can be put into a one
correspondence with the positive integers, say
S = {e1 , e2 , ... }.
Definition
If a sample space S is either finite or countably infinite, then it is called a discrete
sample space.
A set that is either finite or countably infinite is also said to be countable. This is
the case in the first two examples. It is also true for the last example when failure times are
recorded to the nearest hour, but not for the conceptual sample space. Since the conceptual
space involves outcomes that may assume any value in some interval of real numbers (i.e.,
the set of nonnegative real numbers), it could be termed a continuous sample space, and it
provides an example where a discrete sample space is not an appropriate model.
If the sample space consists of a continuum, such as all the points of a line segment
or all the points in a plane (i.e. an uncountable infinite number of outcomes), it is said to
be continuous. Continuous sample spaces arise in practice whenever the outcomes of

- 25 -
 experiments are measurements of physical properties such as temperature, speed, pressure,
length, that are measured on continuous scales.

(C) Event
An event A is a collection of some of the possible outcomes of the random experiment.
In the language of the set theory, A is a subset of the sample space S i.e.
AS
S
In particular, we may regard each individual outcome as an
event; since each outcome is a subset of S. (These are A
sometimes called elementary events). Also, the null set 
(i.e. the set which consists of no outcomes) and the complete
sample space S are each special cases of subsets of S, and
thus both  and S may be regarded as events.  is known as
the impossible event and S as the certain event.
For example, in tossing a die, we might consider the following as typical events;
A1 : Score is 1, i.e. {1}
A2 : Score is even, i.e. {2,4,6}
A3 : Score is less than 5, i.e. {1,2,3,4}.

(D) Occurrence of an event.

We say that an event A had occurred if the outcome of the experiment belongs to the
set A. For example, if the outcome in throwing a die was a score 6, then, as described above,
A2 occurred, but A1 did not occur and A3 did not occur.
S
(E) Union of Events.
Given two events A & B, the union of A & B, written
A B
AB
is defined as the set of outcomes which belong to either A
AB
or B or both. Thus, the event (A  B) occurs whenever A or
B occurs or both occur.

- 26 -
(F) Intersection of events. S
The intersection of A & B written as A  B is defined
as the set of outcomes which belong to both A and B. The
set (A  B) occurs when both A & B occur. A B

AB

S
(G) Mutually exclusive Events
Two events A and B are called mutually exclusive if
both A and B cannot occur in the same time i.e. if A B
AB= 
so that A and B are disjoint.
A  B= 

(H) Complimentary event

Given an event A, the complimentary, A A (or A/ ), is A
defined as that subset of S which do not belong to A. Thus the A
event A occurs whenever A does not occur and vice versa.

(I) Equally Likely Events.

If no outcome of a random experiment is any more likely to occur than any other,
The outcomes are said to be equally likely.

2.3 Classical Probability

Until fairly recently, probability was thought of by statisticians and mathematicians
only as an objective phenomenon derived from objective processes. The concept of
objective probability may be categorized further under the headings of:
1- Classical, or a priori, probability and
2- The relative frequency, or a posteriori, concept of probability.
We may define probability in the classical sense as follows:
If a random experiment can result in N mutually exclusive and equally likely

- 27 -
outcomes, n(A) of which corresponds to the occurrence of some event A, then the
probability that the event A will occur, denoted by P(A), will defined as the ratio n(A)/N,
symbolically.
n(A) number of outcomes belong to A
P(A) = = (2.1)
N total number of outcomes belong to S

The relative frequency approach to probability depends on the repeatability of some

process and the ability to count the number of repetitions, as well as the number of times that
some event of interest occurs. We may define the probability of observing some
characteristic, A, of an event as follows:
If some process is repeated a large number of times, N, and if some resulting event
with the characteristic, A, occurs n times, the relative frequency of occurrence of A, n/N, will
be approximately equal to the probability of A.
For instance, consider the fair die. If our experiment consists of throwing a fair
(unbiased) die, then there are 6 equally likely outcomes. We decide that the outcomes are
equally likely by reasoning that if the die fair and a face (a number) is turned up at random
there is certainly no reason to expect any face to be more likely to be turned up than any
other. On any throw of a die, just one face and not other will turned up, so we conclude that
the outcomes are mutually exclusive.

Example 2.4
Throw an unbiased coin three times and observe the sequence of heads and tails. Here
the sample space is the collection of all possible sequences,
S = { HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}
Since the outcomes are equally likely and mutually exclusive then the probability of each
outcome is 1/8. Let A be the event that two or more heads appear consecutively, and B that
all the tosses are the same. Then
A = { HHH, HHT, THH } and B = { HHH, TTT }
Therefore
n( A) 3 n(B) 2 1
P(A)= = , P(B)= = =
N 8 N 8 4

- 28 -
 n(AB) 1
P(AB)= = P ( { HHH } ) = , and
N 8

n(AB) 1
P(AB)= = P ( { HHH , HHT , THH , TTT } ) =
N 4
Example 2.5
There are 15 balls, numbered from 1 to 15 in a bag. If a person selects one at
random, what is the probability that the number printed on the ball will be
i- a prime number greater than 5.
ii- an odd number less than 11.
Solution
Let A1 = { a prime number greater than 5 }
A2 = { an odd number less than 9 }
then
A1 = { 7, 11, 13 } , A2 = { 1, 3, 5, 7 }, N = 15 , n( A1 ) = 3 , n( A2 ) = 4
n ( A1 ) 3 1 n ( A2 ) 4
 P ( A1 ) = = = and P ( A2 ) = =
N 15 5 N 15
Find P(A1  A2) and P(A1  A2) ?

Example 2.6
A fair die is tossed twice. What is the probability that the sum of the upturned faces is 9?
Solution
The sample space for this experiment is
S = { (i , j) : i = 1, 2, ...,6 ; j = 1, 2, ...,6 }
Since the die is fair (unbiased), each of the 36 possible outcomes would be equally
likely to occur. If A represents the event that the sum of the upturned faces is 9 then,
A = {(3, 6) , (4, 5) , (5, 4) , (6, 3)}
hence,
P(A) = 4/36 = 1/9

2.4 Axiomatic Approach To Probability

For a given experiment, S denotes the sample space, a set real function that associates

- 29 -
a real value P(A) with each event A is called a probability function (or a probability
measure), and P(A) is called the probability of A, if the following properties are satisfied:

Axiom I. P(A) ≥ 0, for every event A  S.

Axiom II. P(S) = 1,

Axiom III. If A1, A2, A3, .... , is a finite or infinite sequence of mutually exclusive
event of S, then

P(A1  A2  A3 ...) = P(A1) + P(A2) + P(A3) + ...

These axioms all seem to agree with our intuitive concept of probability and these few
axioms are sufficient to allow a mathematical structure to be developed.
Note, if A and B are two mutually exclusive events of S, then
P(A  B) = P(A) + P(B)

Example 2.7
Three horses A, B and C are in a race. A is twice as likely to win as B and B is twice
as likely to win as C. What is their respective probabilities of winning, i.e. P(A), P(B) and
P(C)?
Solution
Let P(C) = p,
since B is twice as likely to win as C, then
P(B) = 2p,
and since A is twice as likely to win as B, then
P(A) = 4p
Now the sum of the probabilities must be one (by property II), hence
P(A) + P(B) + P(C) = 1,
i.e.
4p + 2p + p = 1
Therefore, p=1/7, and accordingly
P(A) = 4p = 4/7, P(B) = 2p =2/7 and P(C) = p = 1/7.

- 30 -
Probability in Discrete Spaces
If A is an event in a discrete sample space S, then P(A) equals the sum of the
probabilities of the individual outcomes comprising A.
In particular if the sample space is given by
S = { e1 , e2 , e3 ,...}
and we suppose that to each elementary event {ei} we assign a real number pi, so that P({ei})
= pi such that;

pi  0 for all i ; and  pi = 1 (2.2)

i

Then the probability of an event A is given by

P ( A ) =  pi
i : e iA

It is understood , with this notation, that the summation is taken over all indices i such that ei
is an outcome in A.

Example 2.8
A die is loaded in such a way that each odd number is twice as likely to occur as each
even number. Find P(G), where G is the event that a number greater than 3 occurs on a
single roll of the die.
Solution
The sample space is S = {1, 2, 3, 4, 5, 6}. Hence, if we assign probability p to each
even number and probability 2p to each odd number, we find that
2p + p + 2p + p + 2p + p = 9 p = 1
1
in accordance with axiom II. It follows that p =
9
1 2 1 4
P( G ) = + + =
9 9 9 9
If a sample space is accountably infinite, probabilities will have to be assigned to the
individual outcomes by means of a mathematical rule, preferably by means of a formula or
equation.
Example 2.9
If, for a given experiment, e1 , e2 , e3 ,...., is an infinite sequence of outcomes, verify
that

- 31 -
 i
1
P ( { ei } ) =   for i = 1 , 2 , 3 , ....
 2
is, indeed, a probability measure.
Solution
Since the probabilities are all positive, it remains to be shown that P(S) = 1. Getting
1 1 1 1
P( S ) = + + + +
2 4 8 16
and making use of the formula for the sum of the terms of an infinite geometric progression,
we find that
1/2
P( S ) = =1
1 - 1/2
If ei is the event that a person flipping a balanced coin will get a tail for the first time on the
ith flip of the coin, then the appropriate probability is
i
1
P ( { ei } ) =   for i = 1 , 2 , 3 , ....
 2
Thus, the probability that the first tail will come on the third, fourth, or fifth flip of the coin
is
3 4 5
1 1 1 7
  +   +   =
 2  2  2 32
and the probability that the first tail will come on an odd-numbered flop of the coin is
1 3 5
1 1 1 1/2 2
  +   +   += =
 2  2  2 1 - 1/4 3
Here again we made use of the formula for the sum of the terms of an infinite geometric
progression.

2.5 Some Rules of Probability

Based on the three axioms of probability, we can derive many other rules which have
important applications. Among them, the next four theorems are immediate consequences of
the axioms.
Theorem 2.1
For any sample space S, p(  ) = 0
Proof

- 32 -
 Since S and φ are mutually exclusive and S   = S in accordance with definition of
the empty set φ, it follows that
P( S )=P( S   )
= P ( S ) + P (  ) ( by Axiom III )
and, hence, that P(  ) = 0.

Theorem 2.2
If A and A/ are complementary events in a sample space S, then
P(A ) = 1 - P(A)
Proof
In the second and third steps of the proof that follows, we make use of the definition
of a complement, according to which A and A/ are mutually exclusive and A  A/ = S.
Thus, we write
1= P ( S ) ( by Axiom II )
=P ( A  A )
=P ( A )+P( A ) ( by Axiom III )
and it follows that P( A ) = 1 - P(A).

Theorem 2.3
If A and B are events in S and A  B, then P(A)  P(B).
Proof
Since A  B, we can write
B=A  ( A  B)
as can easily be verified by means of a Venn diagram. Then, since A and A/B are mutually
exclusive, we get
P ( B ) = P ( A ) + P ( A  B ) ( by Axiom III )
P ( A ) ( by Axiom I )
In words, this theorem states that if A is a subset of B, then P(A) cannot be greater than
P(B).
Theorem 2.4
For any event A in S, 0  P(A)  1 .
Proof

- 33 -
 Using Theorem 2.3 and the fact that
 AS
for any event A in S, we have
P ()  P ( A )  P ( S )
Then, P(φ) = 0 and P (S) = 1 leads to the result that
0  P (A)  1

Theorem 2.5
If A and B are any two events in S, then the probability of occurrence of A and non-
occurrence of B is given by
P( A  B ) = P( A \ B ) = P(A) - P(A  B)
Proof
The approach will be to express the event A as unions of mutually exclusive events.
From set properties we have
A = ( A  B )  ( A  B)
It follows that A  B and A  B are mutually exclusive, so that
P(A) = P(A  B ) + P(A  B)
Hence the result.
Theorem 2.6
If A and B are any two events in a sample space S, then
P(A  B) = P(A) + P(B) - P(A  B)
Proof
Expressing the event A  B as union of mutually exclusive events as
AB=(A B)B
It follows that A  B/ and B are mutually exclusive, so that
P(A  B) = P(A  B ) + P(B)
= [P(A) - P(A  B)] + P(B)
= P(A) + P(B) - P(A  B)

Repeatedly using the formula of theorem 2.6, we can generalize this addition rule so
that it will apply to any number of events. For instance, for three events we get,
P(A  B  C) = P(A) + P(B) + P(C) - P(A  B) - P(A  C)

- 34 -
 - P(A  C) + P(A  B  C)

Example 2.10
The probability that Jaillan Gabr passes mathematics is 0.75, and passes English is
0.85. If the probability of passing at least one course is 0.9, what is the probability that
Jaillan will pass both courses?

Solution
If M is the event "passing mathematics" and E the event "passing English", then by
transposing the terms in the Additive Rule, ) we have
P( M E )=P( M )+P( E )-P( M E)
0.9 = 0.75 + 0.85 - P ( M  E )
From which the probability that Jaillan will pass both courses is
P(M  E) = 0.7
Questions: what is the probability that Jaillan will fail both courses?
what is the probability that Jaillan will pass one and only one course?

2.6 Conditional Probability

A major objective of probability modeling is to determine how likely it is that an
event A will occur when a certain experiment is performed. However, there are numerous
cases in which the probability assigned to A will be affected by knowledge of the occurrence
or nonoccurrence of another event B. In such an example we will use the terminology
"conditional probability of A given B", and the notation P(A/B) will be used to distinguish
between this new concept and ordinary probability P(A).
As an illustration, consider the event B of getting a 4 when a fair die is tossed. Based
1
on the sample space S = {1,2,3,4,5,6} the probability of B occurring is . Now suppose that
6
it is known that the toss of the die resulted in a number greater than 3. We are now dealing
with a reduced sample space A= {4,5,6}, which is a subset of S. Relative to the sample space
A, we find that the probability that B occurs is 1/3. This example illustrates that events may
have different probabilities when considered relative different sample spaces, and we would
write

- 35 -
 1
P(B/ A)= ,
3
which can also be written
1 1/ 6 P( AB)
P(B/ A)= = =
3 3/6 P(A)
where P(A  B) and P(A) are found from the original sample space S. In other words, a
conditional probability relative to a subspace A of S may be calculated directly from the
sample space S itself.
Definition
If A and B are any two events in S and P(A) > 0, the conditional probability of B given A is
P( A  B )
P( B/ A )= (2.3)
P( A )

Example 2.11
A box contains 100 microchips, some of which were produced by factory 1 and the
rest by factory 2. Some of the microchips are defective and some are good (non-defective).
An experiment consists of choosing one microchip at random from the box and testing
whether it is good or defective. Let

A be the event "obtaining a defective microchip"; consequently, and

B be the event "the microchip was produced by factory 1" , then
A is the event "obtaining a good microchip".
B is the event "the microchip was produced by factory 2".

- 36 -
 B B Totals

A 15 5 20
A 45 35 80

Totals 60 40 100

Table 2.1 Numbers of defective and nondefective

microchips from two different factories
Table 2.1 gives the number of microchips in each category.
The probability of obtaining a defective microchip is
n ( A ) 20
P(A)= = = 0.20
n ( S ) 100
Now suppose that each microchip has a number stamped on it that identifies which
factory produced it. Thus, prior to testing whether it is defective, it can be determined
whether B has occurred (produced by factory 1) or B/ has occurred affects likelihood that a
defective microchip is selected, and the use of conditional probability is appropriate. For
example, if the event B has occurred, then the only microchips we should consider are
those in the first column of Table 2.1, and the total number is n(B) = 60. Furthermore, the
only defective chips to consider are those in both the first column and the first row, and the
total number is n(AB) = 15. Thus, the conditional probability of A given B is
n ( A  B ) 15
P(A/B)= = = 0.25
n(B) 60
Notice that if we divide both the numerator and denominator by
n(S) = 100, we can express conditional probability in terms of some ordinary unconditional
probabilities,
n( AB)/n(S) P( AB)
P(A/B)= =
n(B)/n(S) P(B)

Example 2.12
The probability that a student, selected at random from certain college, will pass
economics is 0.8 and will pass in both economics and religion is 0.5. What is the probability
that he will pass religion if it is known that he had passed economics?
Solution

- 37 -
 If E is the event "passing economics" and R the event "passing religion", then
P ( E  R ) 0.5 5
P(R / E)= = = = 0.625
P(E) 0.8 8
Multiplication Rule
If we multiply the expressions on both sides of (2.3) by P(A), we obtain the following
multiplication rule
Theorem 2.7
If A and B are two events in S, then
P( A  B )=P( A ) P( B/ A )=P( B ) P( A/ B ) (2.4)

Example 2.13
If we randomly pick two television tubes in succession from a shipment of 240
television tubes of which 15 are defective, what is the probability that they will both
defective?
Solution
If we assume equal probabilities for each selection (which is what we mean by
"randomly" picking the tubes), the probability that the first tube will be defective is
15
240
and the probability that the second tube will be defective given that the first tube is defective is
14
239
Thus, the probability that both tubes will be defective is
15 14 7
. =
240 239 1 , 912
This assumes that we are sampling without replacement, namely, that the first tube is not
replaced before the second tube is selected.
Theorem 2.7 can easily be generalized so that it applies to more than two events; for
instance, for three events we have
Corollary:
If A, B, and C are any three events in a sample space S such that P(A  B) > 0, then

- 38 -
 P( A  B  C ) = P( A ) . P( B / A ) . P( C / A  B )
Proof
Writing A  B  C as ( A  B )  C and using the formula of Theorem 2.7 twice,
we get
P( A  B  C ) = P[( A  B )  C]
= P( A  B ) . P( C / A  B]
= P(A).P(B/A).P(C/AB)

Example 2.14
A box of fuses contains 20 fuses, of which 5 are defective. If 3 of the fuses are
selected at random and removed from the box in succession without replacement, what is the
probability that all three fuses are defective?
Solution
If D1 is the event that the first fuse is defective, D2 is the event that the second fuse is
defective, and D3 is the event that the third fuse is defective, then
5 4 3
P ( D1 ) = , P( D 2 / D1 ) = , P( D 3 / D 2  D1 ) =
20 19 18
and substitution into the formula yields
5 4 3 1
P( D1  D2  D3 ) = . . =
20 19 18 114
Further generalization of Theorem 2.7 and its corollary to k events is straightforward,
and the resulting formula can be proved by mathematical induction.

2.7 Total Probability and Bayes' Theorem

There are many situations where the outcome of an experiment depends on what
happens in various intermediate stages. Suppose, in general the intermediate stage permits k
different alternatives (whose occurrence is denoted by B1, B2, ..., Bk). The events B1, B2, ...,
Bk, in this case, constitute a partition of the sample space S (i.e. the B's are pairwise
mutually exclusive and their union equals S ). To find the probability of occurrence of an
event A that can occurs with one of the B's events, the following theorem is required ,
sometimes called the law or the rule of total probability.
Theorem 2.8 : Total Probability

- 39 -
 If the events B1, B2, ..., Bk constitute a partition of the sample space S and P(Bi) > 0 for
i = 1, 2, ..., k, then for any event A in S
k
P ( A ) =  P ( Bi ) . P ( A/ Bi ) (2.5)
i =1

Proof
S
Since the events B1, B2, ..., Bk are exhaustive (i.e
B1
their union equals S), then
A
A = A  S = (A  B1)  (A  B2)  ...  (A  Bk)
Since the events B1, B2, ..., Bk are mutually exclusive,
then A  B1 , A  B2 , ..., A  Bk are also mutually B2 Bk
exclusive. It follows that;
k
P ( A ) =  P (A  B i )
i =1

and the theorem results from applying (2.4) to each term in this summation.

Example 2.15
The members of a consulting firm rent cars from three rental agencies: 60% from
agency 1, 30 % from agency 2, and 10 % from agency 3. If 9% of the cars from agency 1
need a tune-up, 20 % of the cars from agency 2 need a tune-up, and 6% of the cars from
agency 3 need a tune-up, what is the probability that a rental car delivered to the firm will
need a tune-up?
Solution
If A is the event that the car needs a tune-up, and B1, B2, and B3 are the events that the
car comes from rental agencies 1, 2, or 3, we have
P(B1) = 0.60, P(B2) = 0.30, P(B3) = 0.10, and
P(A/B1) = 0.09, P(A/B2) = 0.20, and P(A/B3) = 0.06.
Substituting these values into the formula of Theorem 2.8, we get
P(A) = (0.60)(0.09) + (0.30)(0.20) + (0.10)(0.06) = 0.12
Thus, 12% of all the rental cars delivered to this firm will need a tune-up.

Tree diagram
It is sometimes helpful to illustrate this result with a tree diagram given in figure 2.1.

- 40 -
The probability associated with branch Bi is P(Bi), and the probability associated with each
branch labeled A is a conditional probability P(A/Bi ), which may be different depending
on which branch, Bi , it follows. In order for A to occur, it must occur jointly with one and
only one of the events Bi .

B1 A

B2 A

∶ ∶ ∶

BK A
With reference to the preceding example, suppose that we are interested in the
following question: If a rental car delivered to the consulting firm needs a tune-up, what is
the probability that it came from rental agency 2? To answer questions of this kind, we need
the following theorem, called Bayes' theorem:

Theorem 2.9 : Bayes' Formula

If B1 , B2 , ..., Bk constitute a partition of the sample space S and P(Bi) > 0 for i = 1, 2,
..., k, then for any event A in S such that P(A) > 0,
P ( Br ) . P ( A / Br )
P ( Br / A ) = k for r = 1, 2, ..., k (2.6)
 P ( Bi ) . P ( A / Bi )
i =1

In words, the probability that event A was reached via the rah branch of the tree
diagram of Figure 2.1, given that it was reached via one of its k branches, is the ratio of the
probability associated with the rah branch to the sum of the probabilities associated with all
k branches of the tree.
Proof
Writing in accordance with the definition of conditional probability, we have only to
substitute P( Br ) . P( A / Br ) for P( A  Br ) and the formula (2.5) for P( A ).
P ( A  Br )
P ( Br / A ) =
P(A)

- 41 -
Example 2.16
With reference to Example 2.15, if a rental car delivered to the consulting firm needs a
tune-up, what is the probability that it came from rental agency 2?
Solution
Substituting the probabilities given in Example 2.15 into the formula of Theorem 2.9,
we get
( 0.30 ) ( 0.20 )
P ( B2 / A ) =
( 0.60 ) ( 0.09 ) + ( 0.30 )( 0.20 ) + ( 0.10 ) ( 0.06 )
0.060
= = 0.5
0.120
Observe that although only 30% of the cars delivered to the firm come from agency 2,
50% of those requiring a tune-up come from that agency.

Example 2.17
We are given three similar boxes of microchips as follows:
Box B1 contains 20 microchips of which 5 are defective,
Box B2 contains 35 microchips of which 7 are defective, and
Box B3 contains 40 microchips of which 5 are defective.
A box selected at random, then a microchip is selected at random from the box. If the
component obtained is defective, find the probability that it came from box 2.
Solution
Defining the events B1, B2 and B3 to be respectively choosing "box 1", "box 2" and
"box 3". Let A be the event "obtaining a defective microchip". Thus, we have
P(B1) = P(B2) = P(B3 ) = 1/3

Box Microchip
5/20 A
B1
15/20 .A
1/3
1/3 7/35 A
. B2
28/35 A

- 42 -
 1/3 5/40 A
B3
35/40 A
Therefore;
 1   5   1   7   1   5  23
P ( A )=   +   +   =
 3   20   3   35   3   40  120
Now applying Bayes' formula (2.6), with k=3 and r=2 to find the probability that the
component came from box 2 given that it is defective as follows
1 7 
  
P( B 2 ) P(A / B2 )  3   35  8
P( B 2 /A) = = = = 0.348
P(A) 23 23
120
The tree diagram, given above, describes this process and gives the probability of each
branch of the tree.

2.8 Independent Events

Informally speaking, two events A and B are said to be independent if the occurrence
or nonoccurrence of either one does not affect the probability of the occurrence of the other.
Symbolically, two events A and B are independent if
P(B /A) = P(B ) and P (A/B) = P(A)
and it can be shown that either of these equalities implies the other when both of the
conditional probabilities exist, namely, when neither P(A) nor P(B) equals zero.
Now, if we substitute P(B) for P(B/A) into formula (2.3), we get
P (A  B ) = P (A) P(B /A) = P(A) P (B)
and this is used as the formal definition of independence.

Definition
Two events A and B are independent iff
P( AB)=P( A ) P( B )

otherwise, A and B are dependent.

- 43 -
 It can be shown that if A and B are events such that P(A) > 0 and P(B) > 0, then A and
B are independent iff either of the following holds
P ( B / A ) = P ( B ) and P ( A / B ) = P ( A )
Example 2.18
A fair coin is tossed three times. If A is the event that a head occurs on each of the
first two tosses, B is the event that a tail occurs on the third toss, and C is the event that
exactly two tails occur in the tree tosses, show that
a- events A and B are independent;
b- events B and C are dependent.
Solution
Since the coin is fair then the eight possible outcomes, HHH, HHT, HTH, THH,
HTT, THT, TTH, and TTT, are equally likely, and
A = {HHH, HHT}
B = {HHT, HTT, THT, TTT}
C = {HTT, THT, TTH}
A  B = {HHT}
B  C = {HTT, THT}
the assumption that the eight possible outcomes are all equiprobable yields
P(A) = 1/4 , P(B) = 1/2 , P(C) = 3/8, P(A  B) = 1/8 and P(B  C) = 1/4.
a- Since P(A). P(B) = 1/4.1/2 = P(A  B), the events A and B are independent.
b- Since P(B). P(C) = 1/2 .3/8  P(B  C), the events B and C are not independent.
In connection with the definition of independence, given above, it can be shown that if
A and B are independent, then so are A and B , A and B, and A and B . For instance:
Theorem 2.10
If A and B are independent, then A and B are also independent.
Proof
Since A = ( A  B)  (A  B ), A  B and A  B are mutually exclusive, and A
and B are independent by assumption, we have
P(A) = P[(A  B)  (A  B )]
= P(A  B) + (A  B )
= P(A). P(B) + P(A  B )

- 44 -
It follows that
P(A  B ) = P(A) - P(A). P(B)
= P(A). [1 - P(B)]
= P(A). P( B )
and hence that A and B are independent.
The reader is asked to show that if A and B are independent, then
A/ and B are independent and so are A/ and B/, and if A and B are dependent, then A and B/
are dependent.
To extend the concept of independence to more than two events, let us make the
following definition.

Definition
Events A1, A2, ...., Ak are independent iff the probability of the intersection of any 2, 3,
... or k of these events equals the product of their respective probabilities.
For three events A, B, and C, for example, independence requires that;
P(A  B) = P(A) . P(B)
P(A  C) = P(A) . P(C)
P(B  C) = P(B) . P(C)
and
P(A  B  C) = P(A) . P(B) . P(C)
It is of interest to note that three or more events can be pairwise independent
without being independent.
A common example of dependent events occurs in connection with repeated
sampling without replacement. In example 2.13 we considered the results of drawing fuses
in succession. Suppose that, instead, the outcome of the first fuse is tested and then the fuse
is replaced in the box before the second draw is made and the second fuse is replaced in the
box before the third draw is made. This type of sampling is referred to as sampling with
replacement, and it would reasonable to assume that the draws are independent trials. In this
case the probability that all three fuses are defective is
5 5 5 1
P( A  B  C ) = P(A) . P(B) . P(C) = . . =
20 20 20 64

- 45 -
2.9 Methods of Enumeration
In many experiments with finite sample space, such as games of chance, it may be
reasonable to assume that all the possible outcomes are equally likely. In that case, a realistic
probability model should result by following the classical approach and taking the
probability of an event A to be P(A) = n(A)/N. Counting the number of ways in which an
event may occur can be tedious problem in complicated experiments. In such cases some
techniques of combinatorial analysis are used, which could be called enumeration methods
of counting.

Multiplication Rule.
If one operation can be performed in n1 different ways and a second operation can be
performed in n2 different ways,..., and finally a k-th operation can be performed in nk
different ways, then the number of ways all these k operations can be carried out is the
product
n1 n2 . . . nk
Thus if there are N possible outcomes of each of r trials of an experiment, then there are Nr
outcomes in the sample space.

Example 2.19

SHIRT TIE
T1
S1 T2
T3
T4

T1
S2 T2
T3
T4

If a man has 2 shirts and 4 ties then he has

2x4 = 8 ways
of choosing a shirt and then a tie. The tree diagram of this example is (S for shirt and T for
tie)

- 46 -
Example 2.20
How many 4 digit numbers can be formed with the 10 digits 0, 1, 2,..., 9 if
(a) repetitions are allowed (b) repetitions are not allowed
(c) the last digit must be zero and repetitions are not allowed.
Solution
(a) The first digit can be any one of 9 digits (all digits except zero, since zero is not allowed)
the second, third and fourth digit can be any one of 10. Then 9.10.10.10 = 9000 numbers
can be formed.

(b) The first digit can be any one of 9 digits (any one but 0), the second digit can be any one
of 9(any but that used for the first digit), the third digit can be any one of 8 (any but
those used for the first two digits) and the fourth digit can be any one of 7 (any but those
used for the first three digits). Then
9.9.8.7 = 4536 numbers can be formed.
(c) The first digit can be chosen in 9 ways, the second in 8 ways and the third in 7 ways.
Then 9.8.7 = 504 numbers can be formed.

Permutations
Suppose that we are given n distinct objects and wish to arrange r of these objects in a
line. Then the number of different arrangements, or permutations as they are often called, is
given by
n
n!
Pr = n ( n - 1 ) ( n - 2 ) ... ( n - r + 1 ) = ( n - r ) !
n
In the particular case whose r = n, Pr becomes
n

Pn = n ( n - 1 ) ... 3 x 2 x 1 = n!
which is called n factorial.
Example 2.21
The number of different arrangements or permutations consisting of 3 letters each
which can be formed from the 7 letters A,B,C,D,E,F,G is

- 47 -
 7
7! 7!
P3 = ( 7 - 3) ! = 4 ! = 7x6x5 = 210 .
Suppose that a set consists of n objects of which n1 are similar of one type (i.e.
indistinguishable from each other), n2 are similar of a second type,...,nk are similar of a k-th
type. Here of course,
n = n1 + n2 +...+ nk.
Then the number of different permutations of the objects is
n n!
P n1, n 2 ,...,nk =
n1 ! n 2 ! ... nk !

Example 2.22
The number of different permutations of the 11 letters of the word MISSISSIPPI,
which consists of 1 M, 4 I's, 4 S's and 2 P's is
11!
= 3450 .
1! 4! 4! 2!
Example 2.23
It is required to seat 5 men and 4 women in a row so that the women occupy the even
places. How many such arrangements are possible?
Solution
The men may be seated in 5! ways and the women in 4! ways. Each arrangement of
the men may be associated with each arrangement of the women. Hence,
Number of arrangements = 5! x 4! = 2880

Example 2.24
Four different mathematics books, 6 different physics books, and two different
chemistry books are to be arranged on a shelf. How many different arrangements are
possible if
(a) the book in each particular subject must all stand together,
(b) only the math books must stand together?
Solution
(a) The Math. Books can be arranged among themselves in 4! ways, the physics books in

- 48 -
 6! ways, the chemistry books in 2! ways, and the three groups in 3! ways. Thus the
number of arrangements
= 4! 6! 2! 3! = 207360.
(b) Consider the four Mathematics Books as one big book. Then we have 9 books which
can be arranged in 9! ways. In all of these ways the Mathematics Books are together.
But the Math. Books can be arranged among themselves in 4! ways. Hence the number
of arrangements = 9! 4! = 8709120.

Combinations
In a permutation we are interested in the order of arrangement of the objects. Thus A,
B, C is a different permutation from B, C, A. In many problems, however, we are interested
only in selecting or choosing objects without regard to order? Such selections are called
combinations. For example ABC and BCA are the same combination.
The total number of combinations of r objects selected from n different objects (also
n
n
called the combinations of n things taken r at a time) is denoted by C or   , and is
r  r
given by
n n n!
  = C =
 r r r!(n - r)!

It can also be written as

n

 n  n ( n - 1 ) ... ( n - r + 1 ) Pr
  = =
 r r ! r!
It is easy to show that
n  n 
  =  

 r n - r
Example 2.25
The number of ways in which 3 cards can be chosen or selected from a total of 8
different cards is
 8 8 x 7 x 6
  = = 56 .
 3 3!

- 49 -
Example 2.26
Out of 5 mathematicians and 7 physicists, a committee consisting of 2 mathematicians
and 3 physicists is to be formed. In how many ways this be done if
(a) Any mathematician and any physicist can be included.
(b) One particular physicist must be in the committee.
(c) Tow particular mathematicians cannot be on the committee.
Solution
 5
(a) 2 math. out of 5 can be selected in   ways, and 3 physicists out of 7 can be selected
 2
 7  5  7
in   ways. Hence the total number of possible selections is     = 10 x 35 = 350 .
 3  2  3
 5
(b) 2 math. out of 5 can be selected in   ways, and 2 physicists out of 6 can be selected in
 2
 6  5  6 
  ways. Hence the total number of possible selections is    = 150 .
 2  2  2 
 3
(c) 2 math. out of 3 can be selected in   ways, and 3 physicists out of 7 can be selected in
 2
 7  3  7 
  ways. Hence the total number of possible selections is    = 105 .
 3  2  3 

2.10 Probability Using Enumeration Methods

In the previous section our concern focused on counting the number of ways a given
operation, or sequence of operations, could be performed. Here we want to couple those
enumeration results with the notion of probability. In a combinatorial setting, making the
transition from an enumeration to a probability is easy. If there are N ways to perform a
certain operation and a total of n(A) of those satisfy an event A, then P(A) as defined by
equation (2.1), is the ratio n(A)/N. This assumes, of course, that all possible outcomes are
equally likely.

Example 2.27

- 50 -
 A box contains 8 red, 3 white and 9 blue balls. If 3 balls are drawn at random without
replacement, determine the probability that
(a) all 3 are red, (b) all 3 are white, (c) 2 are red and 1 is white,
(d) at least 1 is white, (e) 1 of each color is drawn.
Solution
 8
 
=  3 =
number of selections of 3 out of 8 red 14
(a) Required probability = .
number of selections of 3 out of 20  20  285
 
 3
 3
 
=  3 =
number of selections of 3 out of 3 white balls 1
(b) Required probability =
number of selections of 3 out of 20 balls  20  1140
 
 3
 
(c) P(2 are red and 1 is white) =

 8  3
   
=  2   1 =
number of selections of 2 out of 8 red and 1 out of 3 white balls 7
number of selections of 3 out of 20 balls  20  95
 
 3
 
 17 
 
 3  23
(d) P(at least one is white) = 1 - P(none is white) = 1 -   = .
 20  57
 
 3
 
 8  3  9
     
(e) P(one of each colour is drawn) =  1   1   1  = 8 . 3 . 9 .3! = 18
 20  20 . 19 . 18 95
 
 3
 

- 51 -
 EXERCISES

[1] Of 50 patients on the third floor of a hospital, 35 are female and 12 are over 70 years of
age. Among those over 70 years of age, eight are female. How many of the 50 patients
are female or not over 70 years of age? (Use a venn diagram to help you answer the
question).

[2] Set C consists of the citizens of a certain town who voted "yes" for water
fluoridation. Set D consists of the citizens of the same town who have preschool
children. Define:
(a) C  D 5 (b) C  D 6 (c) C  D 7

[3] Let A, B and C be three arbitrary events. Find expressions for the following events that
of A, B and C.
a- Only A occurs, b- All three events occur, c- Non occurs,
d- At least one occurs, e- Not more than 2 occur, g- One and only one occurs.

[4] Let S = { e1 , e2 , e3 , e4 } be a sample space and let P be probability function defined on S,

a- Find P(e1) and P(e2) if P(e3) = P(e4) = ¼ and P(e1) = 2 P(e2).
b- Find P(e1) if P({e2 , e3}) = 2/3 , P({ e3 , e4 }) = ½ and P(e2) = 1/3.

[5] An integer between 1 and 100 is selected at random, find the probability of getting a
perfect square if
a- all integers are equally likely to be selected,
b- all integers between 1 and 50 are twice as likely to occur as the rest.

[6] A coin is weighted so that the probability that a head appears is twice as likely as tail.
Find the probability that we get head.

[7] Three students A, B and C are in a swimming race. A and B have the same probability of
winning and each is twice as likely to win as C.
a- What is the probability that A does not win?
b- What is the probability that B or C wins?

[8] In a certain population of women 4% have had breast cancer, 20% are smokers and 3%
are smokers and have had breast cancer. A woman is selected at random from the
population. What is the probability that:

- 52 -
 a- she has had breast cancer or smokes?
b- she has had breast cancer and she is not a smoker ?

[9] From a box containing 5 black balls and 3 green balls, 2 balls are drawn in succession,
the first ball being replaced in the box before the second draw is made.
a- What is the probability that both balls are the same color?
b- What is the probability that each color is represented?

[10] Solve problem [9] in case of drawing the second ball without replacing the first one.

[11] A coin is biased so that a head is twice as likely to occur as tail. If the coin is tossed
twice, what is the probability of getting:
a- exactly 2 tails, b- a tail and a head?

[12] The probability that a student, selected at random from a certain college, passes
mathematics is 0.8, the probability that passes English is 0.6 and the probability of
passing at least one of them is 0.9.
a- What is the probability that he failed both Math. and English ?
b- What is the probability that he failed Math. only?
c- If the student passed Math., what is the probability that he also passes English ?
d- If the student failed Math., what is the probability that he also
failed English ?

[13] If the three event A, B and C are independent with P(A)=1/4 , P(B) = 1/3 and P(C) =
1/5, find P(A  B  C).

[14] In a high school graduating class of 100 students, 54 studied mathematics, 69 studied
English and 35 studied both mathematics and English. If one of these students is
selected at random, find the probability that:
a- the student took mathematics or English,
b- the student did not take either of these subjects,
c- the student took English but not mathematics.

[15] A factory has three machines X, Y and Z produce plastic gears. The output of machine X
is twice as the output of machine Y and the output of machine Y is triple as the output of
machine Z. The percentages of defective output of these machines are respectively 4%,
2% and 5%. A gear is selected at random and is found to be defective, find the probability
that it was produced by machine Y

- 53 -
[16] A and B play 12 games of chess of which 6 are won by A, 4 are won by B, and 2 end in
a tie. They agree to play a tournament consisting of 3 games. Find the probability that
a- A wins all the three games, b- two games end in a tie.
c- A and B wins alternately, d- B wins at least one game.

[17] Let A and B events with P(A) = ½, P(B) = 1/3 and P(A  B) = 7/12. Find P(A/B),
P(A/ B ), P( A /B) and P( A / B ).

[18] Given P(A) = 0.5 and P(AUB) = 0.8 , find P(B) if:
a) A and B are mutually exclusive b) A and B are independent
c) P(A/B)=0.4. d) A is a subset of B.

[19] show that if P(B|A) = P(B) and P(B)  0, then P(A|B) = P(A).

[20] Let A and B events with P(A) = ¼, P(A  B) = 1/3 and P(B) = p.
a- Find p if A and B are mutually exclusive.
b- Find p if A and B are independent.
c- Find p if A is a subset of B.

[21] On a fighter plane 3 single-round shots are fired. The probability of success for the first
shot is 0.4, the second 0.5 and the third 0.7. It is sufficient to fire 3 shots in order to
destroy the plane. There is a probability of 0.2 for one successful shot and 0.6 for two
successful shots to destroy the plane. After these 3 shots the plane is destroyed, what
is the probability that it is destroyed by only one successful shot?

[22] At an electronic plant, it is known from past experience that the probability is 0.84 that
a new worker who has attended the company's training program will meet the
production quota, and that the corresponding probability is 0.49 for a new worker who
has not attended the company's training program. If 40% of all new workers attend the
training program, what is the probability that a new worker will meet the production
quota?

[23] When an experiment is performed, one and only one of the events A1, A2 or A3 will
occur. Find P(A1), P(A2), and P(A3) under each of the following assumptions:
a- P(A1) = P(A2) = P(A3). b- P(A1) = P(A2) and P(A3) = 1/2.
c- P(A1) = 2 P(A2) = 3 P(A3).

- 54 -
[24] A,B, and C are events such that P(A) = 1/3, P(B) = 1/4, and
P(C)= 1/5. Find P(ABC) under each of the following assumptions:
(a) If A, B, and C are mutually exclusive.
(b) If A, B, and C are independent.
[25] The probability that three men hit a target are respectively ¼, ½ and ¾. Each shoots
once at the target.
a- Find the probability that exactly one of them hits the target.
b- If only one hit the target, what is the probability that it was the first man?

[26] How many 4 digit numbers can be formed with the 10 digits 0,1,...,9 if:
a- repetitions are allowed,b- repetitions are not allowed,
c- the last digit must be zero and repetitions are not allowed,
d- it is an even number and repetitions are not allowed.

[27] It is required to seat 4 boys and 4 girls in a row. How many arrangements are possible if:
i- The boys and girls seat alternately.
ii- Only the girls have to seat together.

[28] A student is to answer 7 out of 10 questions on an examination. How many choices has
he, if he must answer at least 3 of the first 5 questions?

[29] Five red marbles, 2 white marbles, and 3 blue marbles are arranged in a row. If all the
marbles of the same color are not distinguishable from each other, how many different
arrangements are possible?

[30] In how many ways can a group of 10 people be divided into

a- two groups consisting of 7 and 3 People
b- three groups consisting of 5,3 and 2 people.

[31] Multiple choices:

1. If the knowledge that an event A has occurred implies that a second event B cannot occur,
the events A and B are said to be
a- independent b- mutually exclusive c- A contains B d- non of the above.
2. If P(A)=0.4, P(B)=0.5 and A & B are disjoint, then the probability of occurrence of both
A and B is:
a- 0.0 b- 0.1 c- 0.2 d- 0.9

- 55 -
3. If P(A)=0.4, P(B)=0.45 and P (A  B ) = 0.5 8then P(AB) is :
a- 0.84 b- 0.9 c- 0.95 d- non of the above.
4. If P(A) = ½, P(B) = 1/3 and P(AB) = 2/3 then the events A and B are:
a- independent b- mutually exclusive c- dependent. d- non of the above.
5. If P ( A ) = 0.6 and P ( A  B ) = 0.5 then the probability of neither A nor B occurs is:
a- 5/6 b- 0.1 c- 0.05 d- non of the above.
6. If P(A)=P(B)=0.5 and P(AB)=0.75, then A and B are:
a. Independent b. Mutually exclusive c. dependent d. A  B
7. If P(A/B)=1, then A & B are:
a. independent b. B  A c. A  B d. non of the above.

8. If P ( A / B ) = 1 , then A & B are:

a. independent b. B  A c. A  B d. Mutually exclusive
9. If P (B / A ) = 0 , then A & B are:
a. independent b. B  A c. A  B d. Mutually exclusive
10. Event A occurs with probability 0.4., the conditional probability that A occurs given that
B occurs is 0.5, while the conditional probability that A occurs given that B does not
occur is 0.2. What is the conditional probability that B occurs given that A occurs?
a- 5/6 b- 4/7 c- 5/8 d-Cannot be determined from the information given

<+><+><+><+><+><+><+><+>

- 56 -