Introduction to

Probability
and Statistics

Syllabus
Introduction to Probability and Statistics
Course ID:MA2203

Lecture-2
Course Teacher: Dr. Manas Ranjan Tripathy

Department of Mathematics
National Institute of Techonology, Rourkela

. . . . . . . . . . . . . . . . . . . .
1/14
Introduction to
Probability
and Statistics
Continue...
Statistical or Empirical Probability (VON MISES): If an
Syllabus experiment is performed repeatedly under essentially ho-
mogeneous and identical conditions, then the limiting
value of the ratio of the number of times the event occurs
to the number of trials, as the number of trials becomes
indefinitely large, is called the probability of happening
of the event, it is being assumed that the limit is finite
and unique. Mathematically, if for n trials an event A
happens m times, then
m
P(A) = lim .
n→∞ n
J. E. Kerrich conducted a coin tossing experiment with
10 sets of 1000 tosses ech during his confinement in
World war-II. The number of heads found were: 502,
511, 529, 504, 476, 507, 520, 504, 529. The probability
of getting a head in tossing a coin once is computed
using the above definition, 5, 079/10, 000 = 0.5079 ≈
. . . . . . . . . . . . . . . . . . . .
0.5. 2/14
Introduction to
Probability
and Statistics
Continue...

Syllabus We also call the probability as the counterpart of relative

frequency. If we denote f (A) as the absolute frequency
of an event A. Then Relative frequency of A is denoted
by
f (A)
frel (A) = .
n
We can observe that, 0 ≤ frel (A) ≤ 1, frel (S) = 1. For
two mutually
∪ exclusive events, say A and B we have,
frel (A B) = frel (A) + frel (B). So, having all these prior
information regarding the probability, we will try to gen-
eralize the definition in such a way that, it should include
all the previous definitions, and might be the best and
most practical. This we call as Axiomatic definition of
Probability, which we will discuss in the next.

. . . . . . . . . . . . . . . . . . . .
3/14
Introduction to
Probability
and Statistics
Axiomatic Definition

Syllabus Let S be a given sample space, and S be a σ-field on it.

Then probability P is defined as a set function P : S →
[0, 1], which satisfies the following axioms.
1 For every event A ∈ S, 0 ≤ P(A) ≤ 1.
2 The entire sample space has the probability, P(S) = 1.
3 For mutually exclusive events A and B,
∪
P(A B) = P(A) + P(B).

The above axiom (3) can be extended to countable num-

ber of mutually exclusive events, say A1 , A2 , . . . that is,
∪ ∪
P(A1 A2 . . . ) = P(A1 ) + P(A2 ) + . . . .

. . . . . . . . . . . . . . . . . . . .
4/14
Introduction to
Probability
and Statistics
Basic Results on Probability

Syllabus 1 Probability of the impossible event is zero, that is P(∅) = 0.

∪ the ∅ does not contain any elements, hence
Observe that,
we have S ∅ = S. These ∪ two sets are disjoint. Using the
third axiom, we have P(S ∅) = P(S) = P(S) + P(∅), which
implies that P(∅) = 0.
Note: P(A) = 0, does not imply that A is necessarily an empty
set. In practice, probability ‘0’ is assigned to the events which
are so rare that they happen only once in a life time. For
example, in a random tossing of a coin, the event that the
coin will stand erect on its edge, is assigned the probability
‘0’.
2 Probability of the complementary event Ac of A is obtained as
P(Ac ) = 1 − P(A). To∪prove this, observe that, A and Ac are
c
disjoint events
∪ c and A A = S. Hence using axiom (3), we
have P(A A ) = P(S) = 1, this implies P(A) + P(Ac ) = 1,
and the result.

. . . . . . . . . . . . . . . . . . . .
5/14
Introduction to
Probability
and Statistics
Some More Results
∩
Syllabus 3 For any
∩ two events A and B, we have ∩ (i) P(Ac B) = P(B) −
P(A B). (ii) If B ⊂ A, then (a) P(A B c ) = P(A)∩− P(B),
(b) P(B) ≤ P(A). To prove (ii), the events ∪ B∩and A B c are
c
mutually
∩ exclusive, hence P(A) =∩P(B (A B )) = P(B) +
P(A ∩ B c ), this implies that P(A B c ) = P(A) − P(B). Also
P(A B c ) ≥ 0 which implies that P(A) ≥ P(B).
4 Addition Theorem∪ of Probability: For events A ∩
and B in a sam-
ple space S, P(A B) = P(A) ∩ + P(B)
∩ − P(A B). ∩
Proof: The events A − (A B), A ∪ B, and B − (A B) are
mutually exclusive and∪its union is A B.∩Hence applying ∩ ax-
iom (3), we∩ have P(A B) = P(A − (A∪ B)) + P(A B) +
P(B − (A B)) = P(A) + P(B) − P(A B). This proves the
theorem.
Ex. Prove this theorem without using axioms of probability. Hint:
Use set theory approach.

. . . . . . . . . . . . . . . . . . . .
6/14
Introduction to
Probability
and Statistics
Boole’s Inequality
For n events A1 , A2 , . . . , An , we have
Syllabus
∩
n ∑
n
(i) P( Ai ) ≥ P(Ai ) − (n − 1).
i=1 i=1
∪n ∑
n
(ii) P( Ai ) ≤ P(Ai ).
i=1 i=1

Proof:(i) This can be proved by the method of mathematical

∪
induction. Verify the result
∩ for n = 2 that is, P(A1 ∩ A2 ) =
P(A1 ) + P(A2 ) − P(A1 A2 ) ≤ 1, this implies P(A1 A2 ) ≥
P(A1 ) + P(A2 ) − 1. The result is true for n = 2. Assume that
the result holds true for n = k , that is

∩
k ∑
k
P( Ai ) ≥ P(Ai ) − (k − 1).
i=1 i=1

To prove that the result is true for n = k + 1.

. . . . . . . . . . . . . . . . . . . .
7/14
Introduction to
Probability
and Statistics
Continue...

Syllabus

k∩
+1 ∩
k ∩
L.H.S : P( Ai ) = P(( Ai ) Ak+1 )
i=1 i=1
∩
k
≥ P( Ai ) + P(Ak +1 ) − 1
i=1
∑
k
≥ P(Ai ) − (k − 1) + P(Ak +1 ) − 1
i=1
∑
k +1
= P(Ai ) − k : R.H.S
i=1

. . . . . . . . . . . . . . . . . . . .
8/14
Introduction to
Probability
and Statistics
Continue.....
Proof (ii): Now applying the above inequality, with the events
Syllabus
Ac1 , Ac2 , . . . , Acn , we have

∩ ∑
n
P( Ai ) ≥
c
P(Aci ) − (n − 1)
i=1
= 1 − P(A1 ) + · · · + 1 − P(An ) − (n − 1)
∑n
= 1− P(Ai ).
i=1

Hence, we have
∑
n ∩
P(Ai ) ≥ 1 − P( Aci )
i=1
∪
= 1 − P([ Ai ]c )
∪
= P( Ai ).
. . . . . . . . . . . . . . . . . . . .
9/14
Introduction to
Probability
and Statistics
Bonferroni’s Inequality

Syllabus Given n events, A1 , A2 , . . . , An , we have

∑
n ∪
n ∑
n ∑ ∩
P(Ai ) ≥ P( Ai ) ≥ P(Ai ) − P(Ai Aj ).
i=1 i=1 i=1 1≤i≤j≤n

Proof: This can be proved by the method of mathematical

induction. Check that, the result is true for n = 3.

∪
3 ∑
3 ∑ ∩ ∩
3
P( ) = P(Ai ) − P(Ai Aj ) + P( Ai )
i=1 i=1 1≤i<j≤3 i=1

∑
3 ∑ ∩
≥ P(Ai ) − P(Ai Aj ).
i=1 1≤i<j≤3

Let the result is true for n = k . To prove the result for

n = k + 1.

. . . . . . . . . . . . . . . . . . . .
10/14
Introduction to
Probability
and Statistics
Proof Continue...

Syllabus
k∪
+1 ∪
k ∪
P( Ai ) = P( Ai Ak +1 )
i=1 i=1
∪
k ∪
k ∩
= P( Ai ) + P(Ak +1 ) − P[( Ai ) Ak +1 ]
i=1 i=1
∪
k ∪
k ∩
= P( Ai ) + P(Ak +1 ) − P[ (Ai Ak+1 )]
i=1 i=1
∑
k ∑ ∩
≥ { P(Ai ) − P(Ai Aj )}
i=1 1≤i<j≤n

∪
k ∩
+P(Ak +1 ) − P{ (Ai Ak +1 )}.
i=1

. . . . . . . . . . . . . . . . . . . .
11/14
Introduction to
Probability
and Statistics
Continue....

Syllabus
From Boole’s inequality, we have

∪
k ∩ ∑
k ∩
P( (Ai Ak+1 )) ≤ P(Ai Ak +1 )
i=1 i=1

Using this, we get

k∪
+1 ∑
k+1 ∑ ∩ ∑
k ∩
P( Ai ) ≥ P(Ai ) − P(Ai Aj ) − P(Ai Ak +1 )
i=1 i=1 1≤i<j≤n i=1

∑
k+1 ∑ ∩
= P(Ai ) − P(Ai Aj ).
i=1 1≤i<j≤k +1

. . . . . . . . . . . . . . . . . . . .
12/14