PROBABILITY THEORY

Probability theory is an important branch of statistics. Probability is the chance that something likely to
happen. It is attempted to measure uncertainty. Literally Probability means the chance of happening.

The Probability of a given event is the numerical value given to the likelihood of the occurrence of that
event. It is a number lying between 0 and 1. Zero is for an event which cannot occur and 1 for an event
certain to occur. When the occurrence of an event is uncertain, the probability a number between 0 and
1. For e.g., when a coin is tossed, the event of getting a head is uncertain so, its probability is neither 0
nor 1, but between 0 and 1.

Random Experiment

Experiments which can be repeated any number of times under uniform conditions and outcomes of
which vary irregularly in an unpredictable manner. In a random experiment, all the possible outcomes
are known in advance but, none of the outcomes can be predicted with certainty.

For e.g., the tossing of a coin is a random experiment, because it has two outcomes: head and tail. But,
we are uncertain whether head turns up or tail.

Features of a Random Experiment

1. It has more than one outcome.

2. The results are unpredictable.
3. It is repeatable.

Sample Point

Every elementary outcome of a random experiment is called sample point of that experiment. E.g.,
When a coin is tossed, getting a head is a sample point. When a die is thrown getting a 4 is a sample
point.

Sample Space

A set of all possible outcomes from an experiment is called sample space and it is denoted as S

E.g., A coin is tossed, Sample space = {H, T}

Two coins are tossed, S = {HH, HT, TH, TT}

A die is tossed, S = {1, 2, 3, 4, 5, 6}

Simple and Compound Events

Any possible outcome of a random experiment is called an event. Events could be either simple or
Compound. An event which cannot be further split is called Simple event. I.e., Simple event corresponds
to a single possible outcome.

E.g., 1) two coins are tossed, getting 2 heads is a simple event.

2) A die is thrown, getting 1 is a simple event.

3) Two dice are thrown, getting a sum of 2 is a simple event.

The event which may be further split up is called compound event. The joint occurrence of two or more
simple events is called compound event.

E.g., A die is thrown, getting an even number is a compound event because it may be 2 or 4 or 6.

Two coins are tossed, getting 1 head is compound because it may be (H, T) or (T, H).

Two dice are thrown, getting a sum of 3 is a compound because, it may be (1, 2) or (2, 1).

Sure Event and Impossible Event

An event is sure to occur is called sure event. E.g., When a die is thrown, getting a number less than 7
is a sure event.

Getting a white ball from the bag containing all white balls is a sure event.

Therefore, a sure event is the set containing all sample points of random experiment, i.e., sure event is
the sample space itself.

An event impossible to occur is called impossible event.

E.g., When a die is thrown, getting a number greater than 7 is an impossible event.

Getting a white ball from a bag containing all red balls is an impossible event.

Uncertain Event

An event is said to be uncertain if its happening is neither sure nor impossible.

E.g., When a die is thrown, getting a number less than 4 is an uncertain event.

Getting a white ball from a bag containing red and white balls.
Equally Likely Events

Two or more events are said to be equally likely if they have equal chances to occur and none of them
is expected to occur more often than the other.

E.g., when a coin is tossed, getting a head, and getting a tail are equally likely event.

When a die is thrown, getting 1 and getting 2 are equally likely events.

Mutually Exclusive Events

Two events are said to be mutually exclusive if the occurrence one prevents that of the other.

I.e., Two mutually exclusive events cannot occur simultaneously in the same trail. They have no sample
points in common. I.e., Two events A and B are mutually exclusive the A∩ B = ∅

E.g., when a coin is tossed, getting tail, and getting head are mutually exclusive

When a die is thrown, getting 1 and getting 2 are mutually exclusive events.

Exhaustive Events

A set of events is said to be exhaustive when it includes all possible outcomes of the random
experiment. I.e., Union of events is equal to sample space. Events A, B and C together from exhaustive if
A∪B∪C=S

E.g., When a coin is tossed, then events of getting tail and getting head are form exhaustive events.

When a die is thrown, events of getting 1, 2,3,4,5 and 6 together form exhaustive events.

Mutually Exclusive Events and Exhaustive Events

Mutually Exclusive Events and Exhaustive Events are events which are Mutually Exclusive, and their
union is sample space. A, B and C are events such that A∩ B = ∅, B∩ C = ∅ , A∩ C = ∅ and A ∪ B ∪ C = S

Then A, B C are Mutually Exclusive Events and Exhaustive Events.

E.g., In tossing two coins, let A = getting no head = {T, T), B = getting 1 head = {HT, TH} and C = getting 2
head = {H, H} Then A, B, C are mutually exclusive and exhaustive events.

Independent Events

Two or more events are said to be independent if the occurrence of one event does not affect the other.

E.g., two coins tossed together, the outcome of one do not depend on that of the other.

Two balls are drawn from a bag one after another with replacement, then the second draw does not
depend on the first draw.
Dependent Events

Two or more events are said to be dependent if the occurrence of one depends on that of the other.

E.g., two balls are drawn from a bag one after another without replacement, then the second draw
depends on the first draw.

Complement of Event

̅ or AI
If A is an event, then the event does not happen is called complement of A and is denoted by A
or AC

E.g., In tossing a coin, A is getting head then A1 is getting tail

Union of two events (Event A or B)

A and B are two events, then their union is the event which happens if either A or B happens i.e. at
least one of A and B happens and it is denoted by A ∪ B or A + B

E.g., A draws a king; B be drawing a spade then A ∪ B be drawing a king or spade.

Intersection of Events (Event A and B)

If A and B are two events, then their intersection is the event which happens if both A and B happen
together, and it is denoted by A ∩ B or AB

E.g., A draws a king; B be drawing a spade then A ∩ B be drawing a spade king card.

Probability

Approaches to Probability

There are four different approaches to probability. They are:

1. Mathematical or Classical Probability

2. Relative Frequency Theory of probability
3. Axiomatic approach probability
4. Personalistic or Subjective probability

Mathematical or Classical Probability

This is an earliest approach to the theory of Probability. It is defined as the ratio of the number of
favorable cases to the total number of equally likely cases. i.e.
 No.of favourable cases
Probability of an event A = P (A) =
Total no.of all possible cases

This approach assumes that various outcomes of an event are equally likely, so the Probability of their
happening is also equal. So, the theory determines the Probability of an event has taken place. So, this
approach is also called a 'priori' Probability.

According to classical theory

0
P (an impossible event) = =0
𝑛
𝑛
P (save event) = =1
𝑛
P (an uncertain event) is lies between 0 and 1.

So, probability of an event lies between zero and one.

Limitations of Classical Approach

• The total number of outcomes in a random experiment must be finite and known in advance.
• This method assumes that all outcomes are symmetrical or equally likely. So, this method is not
applicable for asymmetrical outcomes.
• The term equally likely means equi-probable. It is not possible to find whether outcomes are
equally likely, without knowing how to calculate the probability.
• This concept is limited in its application because there are many situations in which the various
outcomes cannot be regarded as equally likely. E.g., if we want to predict whether it will rain the
next day, the two possibilities, rain or no rain cannot be taken as equally likely always.

Relative Frequency Theory of probability

It is a method of estimating probability from experimental results. So, it is called empirical or statistical
definition of probability. If an experiment is performed repeatedly under identical conditions, then the
limiting value of the ratio of the number of times the event occurs to the number of trials is called
relative frequency or frequency ratio event. It may be observed that even though the outcomes of
individual repetitions vary irregularly, when we consider larger and larger numbers of repetitions, the
frequency ratio shows a tendency to become more stable and appear to approach a fixed number. The
tendency to stability of frequency ratios of events is an empirical fact common to all random
experiments. This phenomenon is known as statistical regularity. The number to which the frequency
ratio approaches as the number of repetitions increase is called the probability of the events 'A', usually
written as P (A)

f
P (A) = where n is the number of repetitions of the experiment and f is the number of times event
n
'A' happens.
For example, if a coin tossed 10 times and suppose head appears 6 times, then the frequency ratio of
getting head is 6/10 which is not exactly 1/2. But if we toss the coin a greater number of times, the
frequency ratio will approach nearer and nearer to 1/2. Therefore, the probability of getting head is 1/2.

Limitations of Relative Frequency Approach

• The computation of probability requires repetition of experiment many times which reduces its
practical utility.
• Even if the experiment conducted under essentially identical conditions, the result obtained by
two persons from large number of trials may not be the same.
• It is sometimes impossible to repeat the trails under same conditions.
• The trials must be real, not imaginary.

Axiomatic approach to probability

The axiomatic probability includes the concept of both the classical and empirical definitions of
probability.

Let ‘S’ be the sample space of a random experiment and ‘A’ be an event, then the probability of that
event A is expressed as p (A). The following are the axioms for P (A)

1. P(A) ≥ 0, i.e., probability is nonnegative, and it lies between 0 and 1

2. P(S) = 1, i.e., total probability = 1
3. P (A ∪B) = P (A) + P(B) if A and B are mutually exclusive
4. P (A ∩B) = P (A). P (B)

Limitations of Axiomatic Approach

• Axioms are the statements believed to be true, without questioning validity.

• This method has no attempt to define the term probability as such, but certain rules are set to
calculate probability

Subjective Probability

A probability which is calculated based on personal experiences is called subjective probability. It is also
known as Personalistic theory of probability. Such probabilities are calculated based on opinion of some
experts.

E.g., Probability of closing a company within two years, Probability of earning profits 10 years by a firm
etc.

Limitations of Subjective Approach

• In this approach probabilities are calculated purely based on opinion of some experts
• It lacks objectivity, it is highly flexible and unreliable.
 Properties of Probability
1. Probability of an event lies between 0 and 1. i.e., 0 ≤ P(A) ≤ 1.
2. let 'S' denotes the sample space then P(S)
Probability of null or impossible event = 0 Probability of uncertainty
event P (A) which is 0 < P (A) < 1.
3. A and B are two mutually exclusive events then P (A ∪ B) = P(A) + P(B)
4. A and B are mutually exclusive and exhaustive events so A ∩ B = ø and
A ∪ B =S

Addition Theorem of Probability

1. A and B are any two events then

P (A∪B) = P (A) + P (B) - P (A ∩B)

2. A and B are two mutually exclusively events, then P (A ∪B) = P (A) + P (B).

Conditional probability

Probability of an event A given that B has happened is called the conditional probability of A given B
and denoted by P (A/B).

P(A ∩ B)
P (A/B) =
P(B)

Independence of Two Events

Two events A and B are said to be independent if the occurrence of one does not affect the
occurrence of the other.

If A and B are two events such that P (A/B) = P (A) and P (A/B) =P (B), then A and B are
independent events.

Condition for independence of two events A and B is P (A ∩B) = P (A). P (B)

Multiplication Theorem of Probability

1. If A and B are any two events, then P (A ∩B) = P (A) P (B/A)

= P (B) P (A/B)

2. If A and B are two independent events then P (A ∩B) = P (A) P (B).

Inverse Probability

Suppose an event has happened because of the several causes, then we must find out the probability of
a particular cause which really affected the event to happen. Problem of this type are called inverse
probability. Bayes’ theorem is based upon inverse probability.
Bayes’ Theorem:

If an event A can happen only if one or the other of a set of mutually exclusive events B1, B2 .... Bn
happen, then

P(Bk ) P(A/Bk )
P (Bk /A) =
P(B1 ) P(A/B1 ) +P(B2 ) P(A/B2 ) P(A/B2) …….+ P(Bn )P(A/(Bn )