1.2.

7 Probability and Axioms of Probability

Probability is a numerical measure of the likelihood of the event. It is a number that we

attach to an event, say the event that we'll get over an inch of rain tomorrow, which reflects
the likelihood that we will get this much rain.

Probability of an event A is denoted by P(A) or Pr(A). Mathematically, a probability function

P[ . ] is a set function with domain A (event space) and counterdomain the interval [0,1]
which satisfies the following axioms;

Axiom 1: P(A) ≥ 0 for every event A ϵ A

Axiom 2: P*Ω+ = 1

Axiom 3: (a) If A1, A2 , … , Ak is a finite collection of mutually exclusive events in A, then

P (A1  A2  …  Ak ) = ∑ ( )
(b) If A1, A2 , … is an infinite collection of mutually exclusive events in A, then
P (A1  A2  … ) = ∑ ( )

This definition of probability is a mathematical definition. It tells us which set functions can
be called probability functions, it does not tell us what values the probability function P[ . ]
assigns to a given event A.

1.3 Interpretations of Probability

Statisticians have come up with various interpretations of probability each having its own
strongnesses and weaknesses. Two widely used interpretations will be discussed here.

1.3.1 Relative Frequency Interpretation

Consider an experiment that can be repeatedly performed in an identical and independent

fashion. Let E be an event and freq(E) be the number of replications that event E occurs if
the experiment is performed k times. Then the ratio freq(E)/k is called the relative
frequency of event E in a sequence of k replications. Empirical evidence, based on the
results of many such sequences of repeatable experiments, indicates that the relative
frequency stabilizes as k is increased as shown in the below figure. Hence, the probability of
an event is considered as equal to the limiting relative frequency of that event.
( )
P(E) =

Therefore, the probability of an event E related to a random experiment can be interpreted

as the ‘approximate proportion of times that E occurs if we repeat the experiment very large
number of times’.

9
Suppose a student tosses two fair dice and he is interested in the probability that the sum of
the two faces is equal to 6. Let E be the event that player gets two numbers adding up to 6
on the first roll. The below figure shows the relative frequency of getting two numbers
adding up to 6. For generating this, the dice were rolled 1000 times

It is observed that there is a lot of variation in the relative frequencies for a few
experiments. As he tosses the dice more times, the relative frequencies appear to stabilize.
One could guess from the graph that the true probability that he gets a sum equal to 6 is
just under 0.15. In fact, the actual probability is equal to 0.139.

For example, the quality control engineer of a bulb manufacturing company says that the
probability of a randomly selected bulb being defective is 0.08. It means that if we examine
a very large number of products from the production line, the proportion of defective items
will be approximately 8%.

1.3.2 Subjective Interpretation

The relative frequency notion of probability is useful when the process of interest, say
tossing a coin, can be repeated many times under similar conditions. But we wish to deal
with uncertainty of events from processes that will occur a single time. For example, you
may be interested in knowing the probability of

 You getting an A in this class

 Sri Lanka winning the next cricket world cup
 President being reelected

The answer to the above questions will be subjective probabilities. Each value is an
individual’s degree of belief in the occurrence of an event.

10
Determining probability

There are several ways of determining probability of events. Some are;

 Classical method
 Empirical method
 Subjective method
 Using probability models

1.4 Classical Probability and Empirical Probability

1.4.1 Classical Probability

In the classical method, probability of an event is calculated based on the sample space. Let
E be an event of the sample space Ω. Suppose that the sample space is finite and all the
outcomes on the sample space are equally likely. Then the probability of event E is given by,

( )
P(E) = ==
( )

For calculating probability using the classical probability method knowledge on counting
techniques is helpful.

Summary of Counting Procedures

 In a sequence of n events in which the first one had k1 possibilities, the second event
has k2 possibilities, … then the nth event has kn possibilities, the total possibilities of
the sequence is k1× k2×... kn

 Number of ways of permuting (ordering) a set of n distinct objects = n!

And the number of permutations of n objects in which n1 are alike, n2 are alike,… nk are alike
(n = n1 + n2 + … + nk)

 Number arrangements of k objects from n objects.

Ordered Un-ordered
Without
replacement
With
replacement

Activity 03

Example 15 : Consider problems from 1 to 8. Calculate the probabilities using classical

method when possible.

11
1.4.2 Empirical Probability (Relative Frequency)

The relative frequency notion of probability as discussed earlier in 1.4.1 is not practically
applicable directly because in order to determine the probability we need to do the
experiment ‘infinitely many times’. However, as we can see from figure 1.5, the relative
frequency is approximately equal to when t the limit when k is large enough. Therefore we
can use the relative frequency notion to obtain an ‘estimate’ for the probability of the event
E when k is large.

( )
P(E) =

1.5 Conditional Probability

In application of probability theory to practical problems to real life problems, we

sometimes come across situations like such and such has happened, now what is the
probability of something else will happen. For example, in an experiment to record the life
time of a light bulb, one might be interested in the probability that the probability that the
bulb will last 100 hours given that it has already lasted for 24 hours.

Definition

Let A and B be two events in A in given probability space ( , A, P( . )). The conditional
probability of event A given event B, denoted by P[A|B], is defined by

P[A|B] = if B>0
( )

Activity 04

Example 16: Consider an experiment of tossing a coin for two times. first coing is unbuiased
and second coin has a probability of having head eaquals to 0.6.

I. What is the probability of haveing Head given that head has occured.
II. What is the probability of having atleast one Head given that head has occured.

1.5.1 Independent Events

Two events A and B are said to be independent, if the occurrence of A does not depend on
the occurrence of B.

A and B are independent  P(A|B) = P(A) and P(B|A) = P(B)

12
1.7 Probability Rules

Probability rules are used for calculating the probabilities of events that are expressed in
terms of other events with set operations.

1.7.1 Complement Rule

Let E be an event, then

P(Ec) = 1- P(E)

1.7.2 Addition Rule

Let A and B be two events. Then, P(AB) = P(A) + P(B) – P(AB). If A and B are mutually
exclusive, then

P(AB) = P(A) + P(B)

1.7.3 Multiplication Rule

Let A and B be two events. Then,

P(AB) = P(B|A) P(A) = P(A|B) P(B)

A and B are independent  P(A|B) = P(A) and P(B|A) = P(B)

A and B are independent  P(AB) = P(A) P(B)

1.7.4 The law of Total Probability

Let E1, E2, … , En be mutually exclusive and exhaustive events. Then, for any event A,

P(A) = P(A E1) + P(A E2) + … + P(A E n)

= P(A | E1) P(E1) + P(A | E2) P(E2) + … + P(A | En) P(En)

13
1.7.5 Bayes’ Theorem

Let E1, E2, … , En be mutually exclusive and exhaustive events and A be any event.

Then for i = 1,2, …, n

( )
P( Ei| A) =
( )

( ) ( )
=
( )

Example 16:

Two groups of candidates are competing for the position of the board of Directors of a
company. The probabilities that the first and the second groups will win are 0.6 and 0.4
respectively. If the first group wins, the probability of introducing a new product is 0.4 and
corresponding probability if the second group wins is 0.75.

(i) What is the probability that the new product will be introduced?

(ii) lf the product was introduced what is the probability that the first group won?

Example 17:

You are off to soccer, and want to be the Goalkeeper, but that depends who is the Coach
today:
with Coach Sam the probability of being Goalkeeper is 0.5
with Coach Alex the probability of being Goalkeeper is 0.3

Sam is Coach more often ... about 6 out of every 10 games (a probability of 0.6).
So, what is the probability you will be a Goalkeeper today?

Example 18:

A biased coin (with probability of obtaining a Head equal to (p > 0) is tossed repeatedly and
independently until the first head is observed. Compute the probability that the first head
appears at an even numbered toss.

14