CS203: Introduction to Probability

Problem Set 1 Axioms, inclusion-exclusion principle, conditional probability

Summary. Axiomatically, a probability function is defined by specifying three things. (1) A sample
space S (which may be finite, countably infinite or uncountably infinite of any cardinality), (2) a set of
events whose probabilities are of interest called S, and (3) a probability function P (·) that maps each
event E ∈ S to a value in the interval [0, 1]. Each event of interest is a subset of the sample space S,
thus, S is a subset of the powerset of S. The set of events S is a σ-field; it is closed under countably
infinite unions and is closed under complement. In other words, (1) if (En )∞ n=1 is any sequence of
∞
events from S, then ∪n=1 En is also present in S, and (2) if E ∈ S, then, E c ∈ S. Some simple
consequences of the definition of σ-field are given below. For any event E ∈ S, E c ∈ S and hence,
S = E ∪ E c ∈ S and φ = S c ∈ S. If (En )n≥1 is a sequence of events from S, then, so is (Enc )n≥1 and
therefore, ∪n≥1 Enc = (∩n≥1 En )c is in S. Complementing, ∩n≥1 En ∈ S. Hence, a σ-field is also closed
under finite or countably infinite intersections.
The probability function satisfies the following axioms. (1) 0 ≤ P (E) ≤ 1, for every E ∈ S. (2)
P (S) = 1. (3) If (En )n≥1 is a finite or a countably infinite sequence of events from S and they are
mutually exclusive (meaning that for all pairs of indices m and n, Em ∩ En = φ), then,
∞ ∞
!
[ X
P En = P (En ) .
n=1 n=1

Notation for intersection. We will denote the probability of intersection of events E1 , . . . , En , . . . ,

P (E1 ∩ · · · ∩ En ∩ · · · ) as P (E1 E2 · · · En · · · ). Thus, P (E1 ∩ E2 ) is denoted as P (E1 E2 ), etc.
The principle of inclusion and exclusion is a very useful and general principle to obtain the probability
of the union of a finite sequence of events E1 , . . . , En .
n n
!
[ X X X X
P Ei = P (Ei )− P (Ei Ej )+ P (Ei Ej Ek )−· · ·+(−1)k+1 P (Ei1 Ei2 · · · Eik )
i=1 i=1 i<j i<j<k i1 <i2 <...<ik

+ · · · + (−1)n+1 P (E1 E2 · · · En ) .
Notes. 1. Proof of the inclusion-exclusion principle may be done using induction (this is the general
argument, prove this!) and using a non-inductive argument based on the idea of indivisible events
(works for finite or countably infinite sample spaces, prove this!). 2. The summation of terms in the
RHS is an alternatingS sum in the sense that the summation upto odd k terms for k = 1, 3, . . . is an
bound on P ( ni=1 Ei ) and the summation upto even k terms for k = 2, 4, . . . is a lower bound
upper S
on P ( ni=1 Ei ). (Prove this!). This property is used to sometimes get bounds on approximating the
LHS by truncating the RHS upto k terms. In particular, we have the Boole’s inequality, also called the
union bound, namely, P (∪nk=1 Ek ) ≤ nk=1 P (Ek ). 3. In general, the inclusion-exclusion principle does
P
not apply to the case of P (∪∞n=1 En ).

The following two properties of the probability function are called continuity properties.

1. Let (En )∞
n=1 be a sequence of events from S such that E1 ⊂ E2 ⊂ · · · ⊂ En ⊂ · · · . Then,
∞ n
! !
[ [
P En = P lim Ei = lim P (En ) .
n→∞ n→∞
n=1 i=1

1
 2. Let (En )∞
n=1 be a sequence of events from S such that E1 ⊃ E2 ⊃ · · · ⊃ En ⊃ · · · . Then,

∞ n
! !
\ \
P En = P lim Ei = lim P (En ) .
n→∞ n→∞
n=1 i=1

Exercises

Axioms and Inclusion-Exclusion principle

1. If the sample space is C = C1 ∪ C2 and if P (C1 ) = 0.8 and P (C2 ) = 0.5, find P (C1 ∩ C2 ).

2. RIf the sample space is C = {c : −∞ < c < ∞} and if C ⊂ C is a set for which the integral
−|x| dx exists, find the value of α for which the integrand becomes a probability set function.
C αe
(The distribution is called the two-sided exponential distribution.)

3. Let C1 , C2 be subsets of the sample space C. Show that

P (C1 ∩ C2 ) ≤ P (C1 ) ≤ P (C1 ∪ C2 ) ≤ P (C1 ) + P (C2 ) .

4. Let C1 , C2 and C3 be three mutually disjoint subsets of the sample space C. Find P ((C1 ∪ C2 ) ∩ C3 )
and P (C1c ∪ C2c ).

5. Let C1 , . . . , Ck be k events from the sample space C. Show that

P (C1 ∪ C2 ∪ · · · ∪ Ck ) = 1 − P (C1c ∩ · · · ∩ Ckc ) .

That is, the probability that at least one of the events occur is one minus the probability that
none of them occur.

6. Prove the following. Let Cj ’s be events from the σ-field.

P (C1 ∩ C2 ) = 1 − P (C1c ) − P (C2c ) + P (C1c ∩ C2c )

P (C1 ∩ C2 ∩ C3 ) = 1 − q1 + q2 − q3

where,

q1 = P (C1c ) + P (C2c ) + P (C3c )

q2 = P (C1c ∩ C2c ) + P (C2c ∩ C3c ) + P (C3c ∩ C1c )
q3 = P (C1c ∩ C2c ∩ C3c ) .

7. Prove the principle of inclusion-exclusion using induction.

8. Suppose that the sample space S is either finite or is countably infinite. Let p(s) denote the
probability of the event {s}.
P
(a) Show from the axioms that for any subset E ⊂ S, P (E) = s∈E p(s). Thus, each element
of S may be viewed as atomic events, such that any general event is the union of (finite or
countably infinite and mutually exclusive) atomic events.

2
 (b) Using the above, design an alternative non-induction argument for the principle of inclusion-
exclusion. (Hint: Suppose a particular atomic event s appears in exactly k of the events
E
P1 , . . . , En . Show that the number of times s occurs in the r-wise intersection term
k
the binomial theorem (1 − 1)k = 0 to argue inclusion-


i1 <···<ir P (Ei1 · · · Eir ) is r . Use
exclusion).

9. Show the following property for inclusion-exclusion formula. Let E1 , E2 , . . . , En be events, and
p1 , p2 , . . . , pn are defined as follows: pj = sum of the probabilities of all possible intersections
involving j sets. in the notes,

P (E1 ∪ . . . ∪ En ) = p1 − p2 + · · · + (−1)n+1 pn

Show that for any 1 ≤ k ≤ n − 1,

 
P (E1 ∪ · · · ∪ En ) − p1 − p2 + · · · + (−1)k+1 pk ≤ pk+1 .

In particular, the error in Boole’s inequality is bounded by the second term, |P (E1 ∪ · · · ∪ En ) − p1 | ≤
p2 .

10. Suppose Ei is a sequence of events, i = 1, 2, . . . , such that P (Ei ) = 1. Show from first principles
that
∞
!
\
P Ei = 1 .
i=1

11. Consider a random experiment to choose a real number at random in the interval (0, 1) = {x ∈
R : 0 < x < 1}. To begin with we can define a probability function as follows. Let (a, b) ⊂ (0, 1)
be any open interval subset of the open interval (0, 1). Define P ((a, b)) = b − a, that is, the
probability of selecting a point from a sub-interval is directly proportional to the length of the
sub-interval. Show that for all a ∈ (0, 1), P ({a}) = 0. (Hint: Use continuity property of the
probability function).

Probability of events

1. A person has purchased 10 of 1000 tickets sold in a certain raffle. To determine the five prize
winners, five tickets are to be drawn at random and without replacement.

(a) Calculate the probability that this person wins (a) no prize. (b) wins at least one prize.
Define the sample space as all subsets of five tickets from the 1000 sold.
(b) Use conditional probability by viewing the experiment instead as a five step process. (1)
Draw a ticket randomly from the 1000 without replacement and keep it, (2) next draw a
ticket randomly from the 999 left and keep it with the first sample ticket, and so on till the
5th ticket is drawn. Use conditional probability to solve the first part of this question. (Hint:
Define Ej as the event that in the jth step of the process, the person’s ticket is not drawn.
Now appropriately define the event and decompose using multiplicative rule.)

2. There are five red chips and three blue chips in a bowl. The red chips are numbered 1,2,3,4,5
respectively and the blue chips are numbered 1,2,3, respectively. If two chips are drawn at random
and without replacement, find the probability that these chips either have the same number or
the same color.

3
3. In a lot of 50 light bulbs, there are 2 bad bulbs. An inspector examines five bulbs, which are
selected at random and without replacement.

(a) Find the probability of at least one defective bulb among the five.
(b) How many bulbs should be examined so that the probability of finding at least one bad bulb
exceeds 12 ?

4. There are n letters and n corresponding envelopes. If the letters are randomly placed in the
envelopes, calculate the following probabilities.

(a) There is at least one letter in its correct envelope. Show that for large n, this is ≈ 1 − e−1 .
(b) There are no letters placed in their correct envelopes. Show that for large n, this is ≈ e−1 .
(c) There are exactly k letters placed in their correct envelopes, for a given k = 1, 2, . . . , n.
(d) There are at least k letters placed in their correct envelope.

5. Four red balls and two blue balls are placed at random into two urns so that each urn contains
three balls. What is the probability of getting a blue ball if

(a) You select a ball at random from the first urn? (law of total probability).
(b) You select an urn at random and then select a ball from it at random? (law of total proba-
bility)
(c) You discard two balls from the second urn and select the last ball?

Remarks. 1. For part (c), does it matter if the second urn is replaced by the first urn in the
question. Does it matter if instead of the last ball, we choose the first ball, or choose the second
ball. For part (b) does it matter if the urn selected at random is replaced by the first urn (or
second urn).

6. Four fair dice are rolled and the four numbers shown are multiplied together. What is the proba-
bility that the product

(a) is divisible by 5?
(b) has last digit 5?

7. There are n letters of invitation addressed to world leaders. One of these letters is addressed to
U.S. President Trump and another one is addressed to Prime Minister Modi. These n letters are
arranged in a random sequence. What is the probability for each of the following events.

(a) PM Modi’s letter is the first letter in the sequence.

(b) PM Modi’s letter immediately follows President Trump’s letter.

8. Cup and Saucers. A tea set has 2n pairs of cup and saucers, where n pairs are red-colored and the
remaining n pairs are blue-colored. Suppose that all the cups are placed randomly on the saucers.
Once the cups have been placed (randomly) on the saucers, let xBB be the number of blue cups
that have been placed on blue saucers. Find the probability of the following events.

(a) xBB = 0. This event is equivalent to no cup is placed on a saucer with its color.
(b) xBB = n: all cups are placed on saucers of their color.

4
 (c) xBB = k, where, 0 ≤ k ≤ n. Exactly k red cups are placed on red saucers and k blue cups
are placed on blue saucers, the remaining n − k red cups are placed on blue saucers and
vice-versa.

9. Three players A, B and C take turns to roll a die in the order ABC, ABC, A . . . . The rule is that
a player drops out of the game immediately upon throwing a six. So the game stops when all
players have thrown a six.

(a) What is the sample space of the experiment?

(b) What is the probability that A is the first player to roll a six?
(c) What is the probability that A is the second player to roll a six?
(d) Suppose that the game stops when two players have rolled a six. What is the sample space?
Can you show that the game stops with probability 1.

10. Suppose we have a biased coin with probability of heads p and probability of tails 1 − p.

(a) Describe the sample space of independent coin toss sequences until the first heads is observed.
Let X be the number of tosses until the first heads is observed.
(b) Derive the probability mass function P (X = k), for k = 1, 2, . . . . (Geometric distribution).
(c) Let Y be the number of tosses till the second heads is observed. Describe the sample space
and the probability mass function P (Y = k), k = 2, 3, . . . .
(d) Let Zm be the number of coin tosses till the mth heads is observed. Describe the sample
space and the probability mass function P (Zm = k), k = m, m + 1, . . . . (Closely related to
Negative Binomial distribution).

11. A fair coin is tossed until we get exactly two heads.

(a) Describe the sample space.

(b) What is the probability that exactly n tosses are required, n ≥ 2.
(c) Show that as n → ∞, the probability that we get two heads consecutively tends to 1.

12. Murphy’s Law. A fair coin is tossed repeatedly. Let s denote any fixed sequence of heads and tails
of length r (this is the pattern string). Show that with probability 1 the sequence s will eventually
appear.

13. Two people A and B take successive turns in trying to put a basketball in the basket. Success
probabilities of A and B are 31 and 14 respectively. What is the probability that (a) A succeeds
before B? (b) B succeeds before A.

14. There are five red chips and three blue chips in a bowl. The red chips are numbered 1,2,3,4,5
respectively and the blue chips are numbered 1,2,3, respectively. If two chips are drawn at random
and without replacement, find the probability that these chips either have the same number or
the same color.

15. In a lot of 50 light bulbs, there are 2 bad bulbs. An inspector examines five bulbs, which are
selected at random and without replacement.

(a) Find the probability of at least one defective bulb among the five.
(b) How many bulbs should be examined so that the probability of finding at least one bad bulb
exceeds 12 ?

5
16. Three plants C1 , C2 and C3 respectively produce 10%,40% and 50% of a company’s output. C1
produces 1% of its products as defective, C2 has 3% of its products defective and C3 has 4% of its
products defective. One item that is selected at random from the company’s output is observed
to be defective. What is the posterior probability that this product came from plant C1 ? What
is the probability that a random selected output from the company is found defective.

17. Bowl 1 contains 6 red chips and 4 blue chips. Five of these 10 chips are selected at random and
without replacement and put in bowl 2, which was originally empty. One chip is drawn at random
from bowl 2. Given that this chip is blue, find the conditional probability that two red chips and
three blue chips are transferred from bowl 1 to bowl 2.

18. Each bag in a large box contains 25 tulip bulbs. It is known that 60% of the bags contain bulbs
for 5 red and 20 yellow tulips, while the remaining 40% of the bags contain bulbs for 15 red and
10 yellow tulips. A bag is selected at random and bulb taken at random from this bag is planted.

(a) What is the probability that it will be a yellow tulip?

(b) Given that it is yellow, what is the conditional probability it comes from a bag that contained
5 red and 20 yellow bulbs.

19. Monte Hall problem. A prize is placed behind one of three doors. You pick a door. Now Monty
Hall chooses one of the other two doors, opens it and shows to you that it is empty. He then gives
you the opportunity to keep your door or switch to the other unopened door. Should you stay
or switch? It will help to specify the sample space and the relevant events carefully. Thus write
Ω = {(ω1 , ω2 ) : ωi ∈ {1, 2, 3}}, where ω1 is where the prize is and ω2 is the door Monty opens.
(remark. Intuition suggests it doesnt matter. The correct answer is that you should switch. Prove
it.)

20. The probability that a person has a car is 14 . There are five persons and assume mutual indepen-
dence among the events of a person having a car. If it is known that at least one person has a
car, what is the probability that at least 3 persons have cars?

21. Let C1 and C2 be independent events. Show that the following pairs of events are also independent:
(a) C1 and C2c , (b) C1c and C2 and (c) C1c and C2c . Hint: In (a) write P (C1 ∩ C2c ) = P (C2c |
C1 )P (C1 ) = P (C1 )(1 − P (C2 | C1 )) = P (C1 )(1 − P (C2 )) = P (C1 )P (C2c ).

22. Say that C1 , C2 , . . . , Ck are independent events that have respective probabilities p1 , p2 , . . . , pk .
Argue that the probability of at least one of C1 , C2 , . . . , Ck occurs is equal to 1 − (1 − p1 )(1 −
p2 ) . . . (1 − pk ). Hint: Show and use the fact that (C1 ∪ C2 ∪ · · · ∪ Ck )c = C1c ∩ C2c ∩ . . . Ckc .

23. Polya’s Urn. An urn contains b blue balls and r green balls. A ball is drawn at random, its color is
noted and it is returned to the urn together with h further balls of the same color. This procedure
is repeated iteratively, indefinitely. Let B1 , B2 , . . . , Bn , · · · denote the event that the ball drawn
at respectively, the first iteration, second iteration, etc., the nth iteration, is blue. Likewise, define
the events R1 , R2 , . . . , Rn . . . that a red ball is drawn respectively, at the first iteration, second
iteration, etc., nth iteration. Answer the following questions.

(a) Find P (B1 ) , P (R1 ).

(b) Find P (B2 | B1 ) , P (B2 | R1 ).
(c) What is P (B2 )? Show that P (B2 ) = P (B1 ).
(d) Show that P (Bn ) = P (Bn−1 ). Hence argue that P (Bn ) = P (B1 ), for all n.

6
 (e) Show that P (B1 | B2 ) = P (B2 | B1 ). What is P (B1 | B3 )?
b+h
(f) Show that P (B1 | Bn ) = P (Bn | B1 ) = P (B2 | B1 ) = b+r+h .
(g) Show that for any j, k, P (Bi | Bj ) = P (Bj | Bi ).
(h) Find the probability that the first ball drawn is blue given that the following n drawn balls
are all blue. What is the limit of this probability as n → ∞?
(i) Show that in m + n successive iterations, the probability that m blue balls are followed by
n red balls is the same as the probability that n blue balls are followed by m red balls.
Generalize this result.

24. Balls and Bins problem. There are m balls and b bins. Each ball is thrown into one of the b bins
uniformly randomly and independent of each of the other balls. Let Ei denote the event that the
ith bin is empty, for i = 1, 2, . . . , b.

(a) What are P (Ei ), P (Eic )?

(b) Let i 6= j and 1 ≤ i, j ≤ b. What is the probability that ith bin and j th bin both are empty
(i 6= j)? (or, P (Ei ∩ Ej )?)
 
(c) What is the probability that both the bins i and j are not empty? (or, P Eic ∩ Ejc )?
S 
b
(d) What is the probability that at least one bin is empty? (or, P i=1 E i ). If we let p = 1b ,
then show that this probability equals the following expression:
b
!      
[ b m b m b+1 b
P Ei = (1 − p) − (1 − 2p) + . . . + (−1) (1 − bp)m
1 2 b
i=1
b
!
\
=1−P Eic .
i=1
T 
b c b!
[Note: It is obvious that if m < b, then LHS = 1. For m = b, show that P i=1 Ei = bb
which then gives a simple calculation for the LHS. ]
(e) Let i1 , i2 , . . . , ik be k distinct elements from {1, 2, . . . , b}. What is the probability that each
of the bins numbered i1 , i2 , . . . , ik are not empty? (or, P Eic1 ∩ Eic2 ∩ · · · ∩ Eick ). Let p = 1b .

Show that this probability is given by the expression

   
k m k
1− (1 − p) + (1 − 2p)m + · · · + (−1)k (1 − kp)m .
1 2

(f) * Suppose m = b. Show that each bin has at least c log b balls with probability at most b14 ,
for some positive constant c. In the analysis, you might have to use Sterling’s Approximation
m! ≤ ( m m
e) .

Remarks. The probability of events such as E1 , E1 ∩ E2 , Ei1 ∩ Ei2 ∩ ·
· · ∩ Eik are a bit easy to
m
calculate directly from first principles: for e.g., the last one is 1 − kb . Thus, the inclusion-
exclusion principle can be used to obtain the probability
 ofthe intersection of the complement of
some of these Ej ’s. For e.g., For part (c), write P Ei ∩ Ejc = 1 − P (Ei ∪ Ej ) and use inclusion-
c

exclusion principle.

7
 25. Binomial distribution. Consider a biased coin with P (H) = p and P (T ) = 1 − p. Consider an
experiment where it is tossed n times. Let X be the random variable that counts the number of
heads in the outcome of the experiment, 0 ≤ X ≤ n. Derive the pmf for X, pX (x), 0 ≤ x ≤ n.
1
26. Cauchy Distribution. The pdf fX (x) = π(1+x 2 ) , −∞ < x < ∞ defines the Cauchy distribution.
R∞
Show that this is indeed a probability density function by showing that −∞ fX (x)dx = 1. Another
way to define this distribution is as follows. Define a random variable X to have a uniform
distribution in the interval [− π2 , π2 ], so that fX (x) = π2 , for x in this interval, and zero otherwise.
Define Y = tan(X). Y has a Cauchy distribution.

Remark: This problem set is based on basic probability definitions. A general observation is that it
is always useful to have a clear understanding of the sample space of the random experiment and the
definition of probability in the problem setting, before we set out to calculate the probability of some
event of interest.