Probability: Definition, Foundations, and

Applications

Table of Contents

1. Introduction to Probability

2. Definitions and Interpretations

o 2.1 Classical Probability

o 2.2 Frequentist Probability

o 2.3 Axiomatic Definition of Probability

3. Mathematical Foundations

o 3.1 Sample Space and Events

o 3.2 Kolmogorov’s Axioms

o 3.3 Key Theorems (Addition Rule, Conditional Probability)

4. Six Detailed Examples

5. MCQs
1. Introduction to Probability

Probability is the mathematical language of uncertainty. It provides tools

to model randomness, predict outcomes, and make informed decisions in
fields as diverse as quantum physics, finance, artificial intelligence, and
social sciences. At its core, probability answers the question: How likely is
an event to occur?

2. Definitions and Interpretations:

2.1 Classical (A Priori) Probability:

Definition:
Classical probability assumes a finite sample space S where all outcomes
are equally likely. The probability of an event A is:

( Number of outcomes favourable ¿ A)

P(A)=
(Total number of outcomes∈ S)

Conditions:

 Outcomes must be mutually exclusive and exhaustive.

 Requires prior knowledge of all possible outcomes (e.g., fair dice,

shuffled cards).

Example:
Rolling a fair 6-sided die:

 Sample space: S = {1,2,3,4,5,6}.

 Probability of rolling a prime number:

3
P(Prime)= =0.5
6

2.2 Frequentist (Empirical) Probability

Definition:
Frequentist probability defines likelihood based on observed
frequencies from repeated experiments:
 (Number of × A)
P ( A )=lim ⁡( n→ ∞ )
(n)

where n is the total number of trials.

Conditions:

 Requires large-scale repetition of experiments.

 Used when outcomes are not equally likely (e.g., weather

forecasting).

Example:
Quality control in manufacturing:

 A factory produces 10,000 widgets, with 120 defective units.

 Empirical probability of a defective widget:

120
P(Defective)= =0.012
10000

2.3 Axiomatic Definition of Probability

Definition:
The axiomatic definition, formalized by Kolmogorov, provides
the mathematical rules that all probability systems must follow,
regardless of interpretation (classical, Bayesian, etc.). It revolves around
three non-negotiable axioms:

P(A) = Personal confidence (0 ≤ P(A) ≤ 1)

1. Non-Negativity:
P(A)≥0 for any event A.
Probability can’t be negative.

P(A) >= 0 for any event of A

2. Unitarity (Total Probability Rule):

P(S)=1, where S is the sample space.
The probability of "something happening" is 100%.

P(S) = 1 , where S is the sample space.

3. Countable Additivity (σ-Additivity):

If events A1,A2,… are mutually exclusive, then:
 ∞
P(¿ i=1 ¿ ∞❑ A i )=∑ ❑ P( Ai ) .This means that for a countable
i=1

number of disjoint events, the probability of their union is

the sum of their individual probabilities.

3. Mathematical Foundations:

3.1 Sample Space and Events:

 Sample Space (S): The set of all possible outcomes (e.g., S =

{H ,T} for a coin toss).

 Event: A subset of S (e.g., “heads” in a coin toss).

 Mutually Exclusive Events: Events that cannot occur

simultaneously (e.g., rolling a 1 and 6 on a die).

3.2 Kolmogorov’s Axioms:

The axioms are listed above.

3.3 Key Theorems:

 Addition Rule:

P(A∪B) = P(A) + P(B) − P(A∩B)

 Conditional Probability:

P( A ∩ B)
P( A ∣ B)=
P(B)

4. Six Detailed Examples

Example 1: Classical Probability

Scenario: Drawing a card from a standard 52-card deck.

Problem: Probability of drawing a Queen or a Heart.
Solution:

 Total outcomes: 52.

 Favorable outcomes:

o Queens: 4.

o Hearts: 13.

o Queen of Hearts (overlap): 1.

  Using the addition rule:

P(Queen ∪ Heart) = (4 + 13 – 1) / 52 = 4 / 13 = 0.3077

Example 2: Frequentist Probability

Scenario: Clinical trial for a new drug.

 500 patients received the drug; 400 showed improvement.

 500 patients received a placebo; 100 showed improvement.

Problem: What is the empirical probability that the drug causes

improvement?
Solution:

 Improvement with drug: 400500 = 0.8.

 Improvement with placebo: 100500 = 0.2.

 Relative probability increase:

P(Improvement | Drug) − P(Improvement | Placebo) = 0.8 − 0.2 = 0.6

(60%)

Example 3: Subjective Probability

Scenario: Predicting the outcome of a presidential election.

Problem: A political analyst assigns probabilities to Candidate X winning
based on polls and socioeconomic factors.
Solution:

 Analyst’s subjective probabilities:

o Win in State A: P = 0.65.

o Win in State B: P = 0.45.

 Assuming independence (simplified):

P(Win both) = 0.65 × 0.45 = 0.2925(29.25%)

Example 4:

A fair dice is thrown. Find the probability of getting (i) an even number, (ii)
a perfect square, and (iii) an integer greater than or equal to 3.

Solution:
When a dice is thrown, the sample space S is given by

S = {1, 2, 3, 4, 5, 6}, n(S) = 6

(i) Let A be the event of getting an even number.

A = {2, 4, 6), n(A) = 3

P(A) = n(A) / n(S) = 3 / 6 = 1 / 2

(ii) Let B be the event of getting a perfect square.

B = (1, 4}, n(B) = 2

P(B) = n(B) / n(S) = 2 / 6 = 1 / 3

(iii) Let C be the event of getting an integer greater than or equal to 3.

C = {3,4,5,6), n(C) = 4

P(C) = N(C) / n(S) = 4 / 6 = 2 / 3

Example 5:

What is the probability that a leap year selected at random will

have 53 Sundays?

Solution:

A leap year has 366 days, i.e., 52 weeks and 2 days. These 2 days can
occur in the following possible ways:

(i) Monday and Tuesday

(ii) Tuesday and Wednesday

(iii) Wednesday and Thursday

(iv) Thursday and Friday

(v) Friday and Saturday

(vi) Saturday and Sunday

(vii) Sunday and Monday

Number of exhaustive cases n = 7

Number of favourable cases m = 2

Let A be the event of getting 53 Sundays in a leap year.

P(A) = m / n = 2 / 7
Example 6:

Four letters of the word 'THURSDAY' are arranged in all possible

ways.

Find the probability that the word formed is 'HURT'.

Solution:

Total number of letters in the word 'THURSDAY' = 8

Four letters from 8 letters can be arranged in 8P4 ways.

n(S) = 8P4 = 1680

Let A be the event that the word formed is 'HURT'. The word 'HURT' can be
formed

in one way only.

N(A) = 1

P(A) = n(A) / n(S) = 1 / 1680

5.

Question: 1
Which statement best represents the classical (a priori) definition of
probability?

A. Probability is defined as the long-run relative frequency of an event

occurring over many repeated trials.
B. Probability is defined as the ratio of the number of favorable outcomes
to the total number of equally likely outcomes in a finite sample space.
C. Probability is a personal measure of belief or confidence that an event
will occur.
D. Probability is defined as a function that assigns numbers to events in a
σ-algebra according to Kolmogorov’s axioms.

Correct Answer: B

Question: 2
Which of the following definitions of probability is based on the idea that
probability is the limit of the relative frequency of an event in a large
number of trials?
A. Classical probability
B. Frequentist (empirical) probability
C. Subjective probability
D. Axiomatic probability

Correct Answer: B

Question: 3
According to Kolmogorov’s axiomatic definition of probability, which of the
following properties must any probability measure P satisfy?

A. For any two events A and B, P(A∪ B) = P(A) + P(B)

B. P(S) = 0, where S is the sample space.
C. P(A) ≥ 0 for every event A in the σ-algebra over the sample space S.
D. P(A) is given by the limit of the frequency of A occurring in repeated
independent trials.

Correct Answer: C

Question: 4
Which statement best describes the subjective interpretation of
probability?

A. Probability is defined as the ratio of favorable outcomes to the total

number of equally likely outcomes.
B. Probability is determined by the long-run relative frequency of an event
in repeated trials.
C. Probability is a personal measure of belief that reflects an individual's
degree of confidence in an event's occurrence.
D. Probability is rigorously defined solely by Kolmogorov’s axioms without
any reference to personal judgment.

Correct Answer: C

Question: 5
Which of the following is a key limitation of the classical (a priori)
definition of probability?

A. It assumes that all outcomes are equally likely, which is not always
valid in real-world situations.
B. It is based solely on experimental data gathered over a long period.
C. It incorporates subjective opinions, making it less mathematically
rigorous.
D. It is only applicable to continuous probability distributions.

Correct Answer: A