CHAPTER THREE

3. ELEMENTARY PROBABILITY
3.1 Introduction

Probability theory is the foundation upon which the logic of inference is built. It helps us to
cope up with uncertainty. In general, probability is the chance of an outcome of an
experiment. It is the measure of how likely an outcome is to occur.

3.2 Deterministic and non-deterministic models

Deterministic models

Deterministic models always return the same result any time they are called with a specific
set of input values. In deterministic models the condition under which an experiment is
performed determines the outcome of the experiment.
A mathematically deterministic model is a representation y = f(x) that allows you to make
predictions of y based on x.
E.g. y = 2+3X - 4 X 2 . We can predict that if x = 3, then y = -25.
This type of model is "deterministic" because y is completely determined if you know x. In
such models, a given input will always produce the same output.
A deterministic event always has the same outcome and is predictable 100% of the time. E.g.
Distance traveled = time * velocity

Non-deterministic/ probability model

Nondeterministic models may return different results each time even if the input values that
they access remain the same. i.e. In nondeterministic models we cannot determine the
outcome of the experiment even if we know condition under which an experiment is
performed. A probability model is a representation y ~ p(y) not "y = p(y)". The notation "y ~
p(y)" specifically means that y is generated at random from a probability distribution whose
mathematical form is p(y). This model also allows you to make "what-if" predictions as to
the value of y, but, unlike the deterministic model, it does not allow you to say precisely what
the value of y will be. A probabilistic event is an event for which the exact outcome is not
predictable 100% of the time.

1
DMU, Department of Statistics
 3.3 Review of set theory
A set is a well-defined collection of objects. Sets are usually denoted by capital letters A, B
etc. Examples
1. A= { 1 , 2, 3 , 4 } describes the set consisting of positive integers 1, 2, 3 and 4.
2. A={ X :0 ≤ X ≤1 } ⇒A consisting of all real numbers between 0 and 1 inclusive or the
set of all X’s where X is a real number between 0 and 1, inclusive. The individual objects
making up the collection are called members or elements of A.
Examples: 1∈A and 6∉A
Empty set or Null set: - the set containing no members. i.e. {} or ϕ ⇒ impossible event.
Example: A is the set of all real numbers X satisfying the equation X 2 +1=0. A= ϕ
Union: Let A and B be any two subsets of a universal set, S. Then, the union of the Sets A
and B is the set of all elements in S that are in at least one of the sets A or B, it is denoted by
A∪B.
Intersection: Let A and B be any two subsets of universal set, S. Then the intersection of the
sets A and B is the set of all elements in S that are in both sets A and B, and is denoted by A
∩B.
De Morgan’s rules: for any two A and B events
 ( A ∩ B )c = A c ∪ B c
 ( A ∪ B )c = Ac ∩B c
Generally to know about the probability of an event, there must be know the concept of set
theory about the notation of set probability.
If A and B are events then the following conditions are true.
 If A and B are disjoint events then A∩B¿ ∅
 If at least one of the events occurs ,then it means that A∪B
 Both the events are occurs, then it means that A∩B
 If neither event A nor event B occurs, then it means that A ∩ B
 Only event A occurs, then it means that A ∩ B
 If exactly one of the events occurs, then it means that ( A ∩ B)∪(A ∩B)
 If not more than one of the events A or B occurs, then it means that
( A ∩ B)∪ (A ∩B) ∪( A ∩ B)

2
DMU, Department of Statistics
Exercise:

1. Let A, B, and C be three events associated with an experiment. Express the following
verbal statements in set notation.
a) At least one of the events occurs. f) At least two events occur.
b) Exactly one of the events occurs. g) All three events occur.
c) Exactly two of the events occur. h) None occurs.
d) Only A occurs. i) At most one occurs.
e) Both A and B but not C occurs. j) At most two occurs.

3.4 Random experiments, Sample space and events

Experiment: Any process of observation or measurement or any process which generates

well defined outcome.
Probability Experiment (Random Experiment): It is an experiment that can be repeated
any number of times under similar conditions and it is possible to enumerate the total number
of outcomes without predicting an individual outcome.
Example: If a fair coin is tossed three times, it is possible to enumerate all possible eight
sequences of head (H) and tail (T). But it is not possible to predict which sequence will occur
at any occasion.
Outcome: The result of a single trial of a random experiment
Sample Space(S): Set of all possible outcomes of a probability experiment.
Example: Sample space of a trial conducted by three tossing of a coin is
S= {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
Sample space can be
 Countable (finite or infinite)
1. S={0, 1, 2, 3, …}
2. S={ … -3, -2, -1, 0, 1, 2, 3, …}

 Uncountable

a) S=(0, ) c) S=(–, )
b) S=[5,6] d) S=[0, 1]*(0, ) = {(x, y): 0≤ x≤ 1; y>0
Event (Sample Point): It is a subset of sample space. It is a statement about one or more
outcomes of a random experiment. It is denoted by capital letter A, B, C - - -.

3
DMU, Department of Statistics
 For example, in the event, that there are exactly two heads in three tossing of a coin, it
would consist of three points HTH, HHT and THH.
Equally Likely Events: Events which have the same chance of occurring.
Complement of an Event: the complement of an event A means non- occurrence of A and is
' c
denoted by A or A or { Ā ¿ , contains those points of the sample space which don’t belong to A.
Elementary (simple) Event: an event having only a single element or sample point.
Mutually Exclusive (Disjoint) Events: Two events which cannot happen at the same time.
Independent Events: Two events are said to be independent if the occurrence of one does
not affect the probability of the other occurring.
Dependent Events: Two events are dependent if the first event affects the outcome or
occurrence of the second event in a way the probability is changed.
3.5 Counting Techniques
In order to determine a probability of certain event, first it is mandatory to know the number
of possibility that this event going to occur as well as the number of possible outcome of an
experiment that results this event. If we perform an experiment repeatedly to solve such type
of problems we are going to use different mathematical technique which is known to be
counting techniques.
That is in order to judge what is probable, we have to know what is possible. So, we have to
know
 The number of elements of an event
 The number of elements of the sample space
 In order to determine the number of outcomes, one can use several rules of counting
 The addition rule  The permutation rule
 The multiplication rule  The combination rule

I. Addition Rule:

Suppose that for an experiment the 1st procedure is designated by 1, can be performed n1
ways, the 2nd procedure is designated by 2, can be performed n2ways … the K t h procedure is
designated by K can be performed n k ways and each procedure/stps cannot be performed
together, then the total number of possibility of the experiment can be performed (
n1 +n 2+ n3+ …+n k ¿ ways
i.e. n ( A∨B )=n ( A ) +n ( B )−n( A ∩ B) , n ( A ∩ B )=0
 To list the outcomes of the sequence of events, tree diagram is used.

4
DMU, Department of Statistics
Example:
1. Supposes we are planning a trip to some place. If there are 3 bus routes & 2 train
routes that we can take, then there are 3+2=5 different routes that we can take.
2. A student goes to the nearest restaurant to have a breakfast. He can take tea, coffee,
or milk with bread, cake and sandwich. How many possibilities does he have?
Solutions:
 He can take tea with Bread, Cake or Sandwich. i.e. has 3 possibilities.
 He can take Coffee with Bread, Cake or Sandwich. i.e. has 3 possibilities.
 He can take Milk with Bread, Cake or Sandwich. i.e. has 3 possibilities.
So, there are nine Possibilities

II. Multiplication rule

If an operation consists of k steps and the 1st step can be done in n1 ways, the 2nd step can be
done in n2 ways (regardless of how the 1st step was performed), the k th step can be done in nk
ways, (regardless of how the preceding steps were performed), then the entire operation can
be performed in n1· n2·… · nk ways.
Example:
1. How many different ways can seven true or false questions be answered?
Solution: if there are 7 different true or false questions to be answered, we can answer the
first question in n1 = 2 ways, the second question in n2 = 2 ways, … , and the last (7th)
question can answered in n7 =2 ways. Therefore, we can answer in n 1*n2* … * n7=
2*2*2*2*2*2*2 = 128 ways.
2. The digits 0, 1, 2, 3, and 4 are to be used in 4 digit identification card. How many
different cards are possible if
i. Repetitions are permitted.
ii. Repetitions are not permitted.
Solutions a)
1st digit 2nd digit 3rd digit 4th digit
5 5 5 5
There are four steps
1. Selecting the 1st digit, this can be made in 5 ways.
2. Selecting the 2nd digit, this can be made in 5 ways.
3. Selecting the 3rd digit, this can be made in 5 ways.
4. Selecting the 4th digit, this can be made in 5 ways.
So there are 5*5*5*5= 625 different cards are possible

5
DMU, Department of Statistics
b)
1st digit 2nd digit 3rd digit 4th digit
5 4 3 2

There are four steps

1. Selecting the 1st digit, this can be made in 5 ways.
2. Selecting the 2nd digit, this can be made in 4 ways.
3. Selecting the 3rd digit, this can be made in 3 ways.
4. Selecting the 4th digit, this can be made in 2 ways.
So there are 5*4*3*2 =120 different cards are possible

III. Permutation
An arrangement of objects with attention given to order of arrangement is called
permutation. The number of permutation of n different objects taken r at a time is obtained
by:
n!
n Pr = for r=0 , 1, 2, ⋯, n
(n−r)!
Permutation Rule:
a) The number of permutations of n objects taken all together is n!
n! n!
n Pn = = =n!
i.e. n!= n*(n-1)*(n-2)*…*3*2*1 = (n−n)! 0!
Note: By definition 0! = 1
b) The arrangement of n distinct objects in a specific order using r objects at a time is called
the permutation of n objects taken r objects at a time. It is written as nPr and the formula is
n!
n Pr =
(n−r)!
c) The number of distinct permutation of n objects in which n 1 are alike, n2 are alike, ...,nk are
alike is
n!
n1 !. n2 !.⋯. nk ! forn=n1 + n2 +…+n k

Example:
1. Find number of permutations of the letters in the word ‘‘statistics’’.
2. Suppose we have a letters A, B, C, D
a) How many permutations are there taking all the four?
b) How many permutations are there two letters at a time?

6
DMU, Department of Statistics
 3. How many different permutations can be made from the letters in the word
“CORRECTION”?
Solution:
1. There are 3s’s, 3t’s, 1a’s, 2i’s and 1c’s. i.e. n1=3 ,n 2=3,n3 =1,n 4=2∧n5=1
10 !
Therefore = 50,400.
3! 3 ! 1 ! 2 ! 1!
2. a) Here n there are four distinct object
There are 4 !=24permutations
4!
4 P2 = =12 permutations
( 4−2)!
b)

3. Here n=10 of which 2 are C, 2 are O, 2 are R, 1 E, 1 T, 1 I AND 1 N

10 !
=453600 permutations
2! 2 ! 2 ! 1! 1 ! 1 ! 1!
IV. Combination
A selection of objects considered without regard to order in which they occur is called
Combination. The number of combination of n different objects taking r of them at a time is

n C r= ( nr )= r !(n−r
n!
)! , forr=0,1,2,⋯,n .
Example:
a) Given the letters A, B, C, and D list the permutation and combination for selecting
two letters.
Solution:
Permutation Combination
AB BA CA DA AB BC
AC BC CB DB AC BD
AD BD CD DC AD DC
Note that in permutation AB is different from BA but in combination AB is the same as BA.
b) In a club containing 7 members a committee of 3 people is to be formed. In how
many ways can the committee be formed?

: 7C3 =
n C r= ( nr )= r !(n−r
n!
7
)! 
C =( )=
3
7 7!
3 3!(7−3 )! = 35
Solution

7
DMU, Department of Statistics
 c) Among 15 clocks there are two defectives .In how many ways can an inspector chose
three of the clocks for inspection so that:
a) There is no restriction.
b) None of the defective clock is included.
c) Only one of the defective clocks is included.
d) Two of the defective clock is included.
Solution: a)n=15 of which 2 are defective and 13 are non-defective.
r = 3, If there is no restriction select three clocks from 15 clocks and this can be done in:

(nr)= ( n−r ) ! r ! = 12! 3 ! =455 ways

n! 15 !

b) This is equivalent to zero defective and three non-defectives, which can be done in:

(20)(133)=286 ways
c) This is equivalent to one defective and two non-defectives, which can be done in:

(21)(132)=156 ways
d) This is equivalent to two defective and one non defective, which can be done in:

(22)(131)=13 ways
Exercise: Out of 5 Mathematician and 7 Statistician a committee consisting of 2
Mathematician and 3 Statistician is to be formed. In how many ways this can be done if
a) There is no restriction
b) One particular Statistician should be included
c) Two particular Mathematicians cannot be included on the committee.

3.6 Definitions of probability

Definition: Probability is a numerical measure of the chance or likelihood that a particular

event will occur & it lies in the range from 0-1, inclusive. Probability is a building block of
inferential statistics.
Definition: Let E be an experiment. Let S be a sample space associated with E. With each
event A in S we associate a real number designated by P (A) and called the probability of A.

Approaches to measuring Probability

There are four different conceptual approaches to study probability theory. These are
 The classical approach  The frequencies approach

8
DMU, Department of Statistics
  The axiomatic approach  The subjective approach

1. Classical approach:
Definition: If there are n equally likely outcomes of an experiment, and out of the n
outcomes event A occur only k times the probability of the event A is denoted by P (A) is
defined as
¿ n(A) k
P(A) = Number of outcomes favorable ¿ event A Total number of outcomes = =
n(S) n
Note: Classical approach of measuring probability fails to answer for the following
conditions:
 If total number of outcomes is infinite or if it is not possible to enumerate all elements of
the sample space.
 If each outcome is not equally likely.
Example: Compute
a) The probability of having two boys & one girl is a three child family using the
classical method, assuming boys & girls are equally likely.
b) Using (a) compute the probability of having three boys in a three-child family.
c) Using (a) compute the probability of having three girls in a three –child family.
d) Using (a) compute the probability of having two girls & one boy in three child
family.
Solution

The sample space S or the experiment is

S= {BBB, BBG, BGB, BGG, GBB, GBG, GGB, GGG}
So n(S)=8
a) For the event A= ''two boys & a girl'' = {BBG,BGB,GBB} , we have n(A)=3,Since the
outcome are equally likely , the probability of A is P(A)= n(A)/n(S)=3/8 =0.375
b) Compute the probability of having three boys in a three-child family. For the event B=
''three boys'' = {BBB} , we have n(B)=1,Since the outcome are equally likely , the
probability of B is P(B)= n(B)/n(S)=1/8 = 0.125.
c) Compute the probability of having three girls in a three –child family. For the event C=
''three girls'' = {GGG} , we have n(C)=1,Since the outcome are equally likely , the
probability of C is P(C)= n(C)/n(S)=1/8 = 0.125

9
DMU, Department of Statistics
d) Compute the probability of having two girls & one boy in three child family.
For the event D= ''two girls & one boy'' = {BGG, GBG,GGB}, we have n(A)=3,Since
the outcome are equally likely, the probability of D is P(D)= n(D)/n(S)=3/8 =0.375.

Exercise: A fair die is tossed once. What is the probability of getting

a) Number 4? c) An even number?

b) An odd number? d) Number 8?

2. The Frequentist Approach (Empirical Probability)

This approach to probability is based on relative frequencies.

Definition: Suppose we do again and again a certain experiment n times and let A be an
event of the experiment and let k be the number of times that event A occurs. Therefore the
probability of the event A happening in the long run is given by:
Number of ×event A has occured k
P ( A )= =
Total number of observations n
In other words given a frequency distribution, the probability of an event (A) being
Frequency of class A
in a given class is P(A) =
Total frequency∈t h e distribution
Example: The national center for health statistics reported that of every 539 deaths in recent
years, 24 resulted that from automobile accident, 182 from cancer, and 353 from other
disease. What is the probability that particular death is due to an automobile accident?
Solution
P (automobile) = death due to automobile /total death =24/539 = 0.445
The probability that particular death is due to an automobile accident is 0.445.

3. The axiomatic approach

Let E be a random experiment and S be a sample space associated with E. With each event A
a real number called the probability of A satisfies the following properties called axioms of
probability or postulates of probability.
1. 0≤ P( A)≤ 1
2. P(S) =1, S is the sure/certain event.

10
DMU, Department of Statistics
3. If A1 and A2 are mutually exclusive events, the probability that one or the other occur
equals the sum of the two probabilities. i. e. P (A1∪A2) =P(A1)+P(A2)
n
Similarly P(A1∪A2∪ . . . An) = P(A1)+P(A2) +. . . +P(An) = ∑ A i
i=1

4. P (A') =1-P (A)

5. P (ø) =0, ø is the impossible event.

4. Subjective Approach

It is always based on some prior body of knowledge. Hence subjective measures of

uncertainty are always conditional on this prior knowledge. The subjective approach accepts
unreservedly that different people (even experts) may have vastly different beliefs about the
uncertainty of the same event.
Example: Abebe’s belief about the chances of Ethiopia Buna club winning the FA Cup this
year may be very different from Daniel's. Abebe, using only his knowledge of the current
team and past achievements may rate the chances at 30%. Daniel, on the other hand, may rate
the chances as 10% based on some inside knowledge he has about key players having to be
sold in the next two months.

3.7. Derived theorems of probability

Rule l: let A be an event and A' be the complement of A with respect to a given sample space
of an experiment, then P(A')=1-P(A)
Proof: let S be a sample space S=AUA' and, A and A' are mutually exclusive
A∩A' = ø
P(S) = P (AUA') = P (A') + P (A) and P(S) = 1
1= P (A') + P (A) => P (A') = 1-P (A)
Rule 2: let A and B are events of a sample space S, then
P (A' ∩ B) = P (B) - P (A ∩ B)
Proof: B =S ∩ B = (AUA') ∩ B = (A∩ B) U (A'∩ B)
If A∩B ≠ ø , then P(B) =P (A∩ B) +P (A' ∩ B)
P (A' ∩ B) = P(B) – P(A ∩ B).
Rule 3: Suppose A and B are two events of a sample space, then
P(AUB) = P(A) + P(B) – P(A ∩ B)
Proof:
(AUB) = AU(A' ∩ B), A and A' ∩ B are disjoint sets

11
DMU, Department of Statistics
∴P(AU B) = p(A) + p(A' ∩ B) . . . .*
But we have already proved that P (A’ n B) = P (B) – P (A ∩ B)
Put this in equation *
P(A U B) = P(A) + P (B) – P (A ∩ B)
Example: A fair die is thrown twice. Calculate the probability that the sum of spots on the
face of the die that turn up is divisible by 2 or 3.
Solution
S={(1,1),(1,2),(1,3),(1,4),(1,5),(1,6),(2,1),(2,2),(2,3),(2,4),(2,5),(2,6),(3,1),(3,2),(3,3),(3,4),
(3,5),(3,6),(4,1),(4,2),(4,3),(4,4),(4,5),(4,6),(5,1),(5,2),(5,3),(5,4),(5,4),(5,5),(5,6),(6,1),(6,2),
(6,3),(6,4),(6,5),(6,6)}
This sample space has 6*6 =36 elements let A be the event that the sum of the spots on the
die is divisible by 2 and B be the event that the sum of the spots on the die is divisible by
three, then
A = {(1,1), (1,3), (1,5), (2,2), (2,4), (2,6), (3,1), (3,3), (3,5), (4,2), (4,4), (4,6), (5,1), (5,3),
(5,5), (6,2), (6,4), (6,6)}
B = {(1,2), (1,5), (2,1), (2,4), (3,3), (3,6), (4,2), (4,5), (5,1), (5,4), (6,3), (6,6)}
A∩B = {(1, 5), (2,4), (3,3), (4,2), (5,1), (6,6)}
P (A or B) = P (A U B) = P (A) +P (B) – P (A∩B) = 18/36 + 12/36 -6/36 = 24/36 = 2/3
Exercise:
1. If two dice are thrown, what is the probability that the sum is
a) Greater than 8?
b) Neither 7 nor 11?
2. An urn contains 8 white balls and 2 green balls. A sample of three balls is selected at
random. What is the probability that the sample contains at least one green ball?
3. A box contains 12 light bulbs of which 5 are defective. All the bulbs look alike and have
equal probability of being chosen. Three bulbs are picked up at random. What is the
probability that at least 2 are defective?
4. A problem in Mathematics is given to three students, whose chances of solving it are
1
∧1
1 3 respectively. What is the probability that problem will be solved?
,
2 4

12
DMU, Department of Statistics
 2
∧3
5. The probabilities of A, B and C solving a problem are 1 7 respectively. If all the
,
2 8
three try to solve the problem simultaneously, find the probability that exactly one of
them will solve it.
6. A husband and wife appear in an interview for two vacancies in the same department.
1
∧1
The probability of husband's selection 7 that of wife's selection. What is the
5
probability that
A. Only one of them will be selected?
B. Both of them will be selected?
C. None of them will be selected?
D. At least one of them will be selected?

13
DMU, Department of Statistics