Probability Theory I

LECTURE 2

1 Sets and Events

1.1 What this Lecture is all About
The modern theory of probability is based on the concept of a set. Consequently,
it is natural to begin our discussion with some basic results of Set Theory. In
this unit, we define a set, random experiment, event, and event space. We also
discuss various laws pertaining to operations with sets and events, including
union, intersection, complement, DeMorgan’s laws, and limits of sequences of
events.
By the end of this Lecture, you should be able to:
1. list or describe the elements of a set or sample space;
2. explain what a random experiment is;

3. state the three properties of an event space;

4. distinguish between a set and an event;
5. state DeMorgan’s laws of sets and events;

6. describe mutually exclusive events;

7. define a monotone sequence of events;
8. find the limit of a monotone sequence of events.

1.2 Sets
1.2.1 Definition
A set is a collection of items or objects such as numbers or individuals.

1.2.2 Example
Describe the set of letters of the alphabet which are vowels.

1
1.2.3 Solution
The set of vowels is A = {a, e, i, o, u}.
The items or members of a set are called elements or points. For example,
a is an element of the set of vowels, i.e., a ∈ A. It is customary to denote a set
by a capital letter such as A, B, C, etc., and an element of a set by a lowercase
letter such as a, b, c, etc.
If an item, say a, is an element of a set A, we write a ∈ A, and if a is not an
element of the set A, we write a ∈ / A. If a and b both belong to the set A, we
write a, b ∈ A.
A set is well-defined if it is possible to determine whether or not an item
belongs to the set A. It should be clear that if the theory of sets is to be of any
use or meaning, it is necessary to consider sets that are well-defined. Thus, we
shall assume that the sets in question are well-defined.
A set can be defined by listing all the elements of the set (the Roster or
Tabular method) or by stating a property that defines the members of the set
(the Property or General-element method).

1.2.4 Example
Describe the set of all vowels using:
(i) The roster method.
(ii) The property method.

1.2.5 Solution
(i) A = {a, e, i, o, u}

(ii) A = {x : x is a vowel} (read ‘A is the set of all x such that x is a vowel’).

1.3 Some Useful Sets of Numbers

The following sets play an important role in all branches of Mathematics, par-
ticularly in Probability and Statistics:
1. N = {1, 2, 3, . . .}: the set of natural or counting numbers.
2. Z = {. . . , −2, −1, 0, 1, 2, . . .}: the set of all integers (i.e., whole numbers).

3. Q = {r : r = ab , a, b ∈ Z, b 6= 0}: the set of rational numbers.

4. R = {x : −∞ < x < ∞}: the set of real numbers.
You will encounter these sets from time to time. In the meantime, I will
concentrate on setting up the necessary basic machinery to handle sets and
subsets.

2
1.3.1 Definition
If every element of a set A belongs to a set B, we say that A is a subset of B
and write A ⊆ B. Two sets are equal if they contain exactly the same elements.

1.3.2 Example
Let Ω = {1, 2, 3, 4, 5, 6}, A = {1, 3, 5}, and B = {1, 3, 6}. Which of the following
statements is true?
(a) A ⊆ Ω.
(b) A = B.

1.3.3 Solution
(a) A ⊆ Ω is a true statement since all elements of A are in Ω.

(b) A 6= B since one element of the set A, namely 5, is not in B.

It is often convenient to represent subsets of the real line of the form:

E = {x : a ≤ x ≤ b}

by [a, b]. Similarly, subsets of the real line of the form [a, b), (a, b], or (a, b) can
be defined. Thus E = [0, 1) represents all real numbers x such that 0 ≤ x < 1.
In many practical applications, interest is in some subsets of a particular or
special set. To distinguish between the full set and the subsets of interest, we
have the following definition.

1.3.4 Definition
The universal set, usually denoted by Ω or U , is the set of all items or cases of
interest in an investigation.

1.3.5 Operations with Sets and Subsets

In Number Theory, the operations of addition, multiplication, etc., can be used
to obtain other quantities of interest such as sums and products. Analogous
operations can be defined for sets to form other sets of interest.

1.3.6 Definition
The set of all elements belonging to either A or B, denoted by A ∪ B, is called
the union of A and B.
For example, if A is a set of students taking Statistics and B is the set
of students taking Mathematics, then the union of A and B is the set of all
students taking Statistics and/or Mathematics. The set can be shown by means
of a diagram called a Venn diagram.

3
1.3.7 Example
If A = {1, 2, 3, 4} and B = {2, 4, 6, 8}, find A ∪ B.

1.3.8 Solution
The elements of the set B which are not in A are 6 and 8. Adding these to the
elements in A, we obtain the union of A and B:

A ∪ B = {1, 2, 3, 4, 6, 8}.

Another important operation with sets is that of taking intersections.

1.3.9 Definition
The set of all elements belonging to both A and B, denoted by A ∩ B, is called
the intersection of A and B.

1.3.10 Example
Let A = {1, 2, 3, 4} and B = {2, 4, 6, 8, 10}. Find A ∩ B.

1.3.11 Solution
The elements which are common to both sets are 2 and 4. Thus the intersection
of A and B is:
A ∩ B = {2, 4}.

1.3.12 Example
Let A = {2, 4, 6, 8, 10, 12} and the set B be defined by
B = {all positive integers less than or equal to 10 which are divisible by 3}. Find
A ∩ B.

1.3.13 Solution
The numbers in the set A which are divisible by 3 are 6 and 12. The number
in the set B which is divisible by 3 is 6. Hence the intersection of A and B is:

A ∩ B = {6}.

The intersection of A and B is shown by the shaded area in a Venn diagram.

A singleton is a set consisting of just one element. Thus in the above exam-
ple, A ∩ B = {6} is a singleton.

1.3.14 Definition
The set of all elements of a universal set not belonging to a set A, denoted by
Ac , is called the complement of A.

4
1.3.15 Example
Let Ω = {1, 2, . . . , 10} be the universal set, and let A = {1, 2}. Find the
complement of A.

1.3.16 Solution
The elements of the universal set which are not in A are {3, 4, 5, 6, 7, 8, 9, 10}.
Thus the complement of A is:

Ac = {3, 4, 5, 6, 7, 8, 9, 10}.

1.3.17 Example
Let the universal set be the real line, i.e., Ω = (−∞, ∞). Find the complement
of A in each of the following cases:
(i) A = (−∞, 0)
(ii) A = [0, 1] ∪ [2, 3]

1.3.18 Solution
The complement of A is given by:
(i) Ac = [0, ∞)
(ii) Ac = (−∞, 0) ∪ (1, 2) ∪ (3, ∞)
A Venn diagram to show the complement of a set A can be helpful.

1.3.19 Activity
(a) Let A be the set of vowels and Ω be the set of all letters of the alphabet.
Find:
(i) A ∩ {a, b, c}
(ii) A ∪ {a, b, c}
(iii) The complement of A.
(b) Let Ω = {1, 2, . . . , 10} be the universal set. Evaluate:

{1, 4, 5} ∩ {2, 4, 6}c .

(c) Let Ω = (−∞, ∞) be the universal set. Find the complement of B =

[0, 2] ∩ (1, 4].

5
1.4 DeMorgan’s Rules
A set of rules that are often useful in determining the complement of a set
or in calculating the probability of the complement of a compound event is
DeMorgan’s laws of sets.

1.4.1 Theorem (DeMorgan’s Laws)

Let E1 , E2 , . . . , En ⊆ Ω. Then:
Sn c Tn
(i) ( i=1 Ei ) = i=1 Eic
Tn c Sn
(ii) ( i=1 Ei ) = i=1 Eic

1.4.2 Example
Let Ω = (−∞, ∞). Use DeMorgan’s rules to find the complement of the set A
in each of the following cases:
(i) A = [0, 2] ∩ (1, 4]
S∞
(ii) A = n=1 [0, n)

1.4.3 Solution
(i) Ac = [0, 2]c ∪(1, 4]c = (−∞, 0)∪(2, ∞)∪(−∞, 1]∪(4, ∞) = (−∞, 1]∪(2, ∞)
T∞ T∞
(ii) Ac = n=1 [0, n]c = n=1 {(−∞, 0) ∪ [n, ∞)} = ∅ It is also possible to
think of a set containing no elements at all.

1.4.4 Definition
The empty set, denoted by ∅ or {}, is a set with no elements in it.

1.4.5 Example
Let A = {0, 1, 2, . . .} be the set of all non-negative integers and B = {. . . , −2, −1}
be the set of all negative integers. Find A ∩ B.

1.4.6 Solution
Since no integer can be both negative and non-negative, it is clear that the
intersection of A and B has no elements in it. That is, A ∩ B = ∅.

6
1.4.7 Activity
(a) Let V be the set of vowels and Ω be the set of all letters of the alphabet.
Use DeMorgan’s rules to find:
(i) {V ∩ {a, b, c}}c
(ii) {V ∪ {a, b, c}}c
(b) Let Ω = (−∞, ∞) be the universal set. Use DeMorgan’s rules to evaluate
{[0, 2] ∩ (1, 4]}c .

1.5 Sample Spaces and Events

Although the outcome of an experiment may not be predicted with certainty, it is
possible to list or describe, in advance (i.e., before the experiment is performed),
the set of all possible outcomes of the experiment.

1.5.1 Definition
The set Ω of all possible outcomes of an experiment is called the sample space
for the experiment.

1.5.2 Example
Suppose that a coin is tossed once. Describe the sample space for the experi-
ment.

1.5.3 Solution
The sample space is Ω = {H, T }, where H stands for a Head and T for a Tail.

1.5.4 Example
Suppose that a die is rolled once. Describe the sample space for the experiment.

1.5.5 Solution
The sample space is Ω = {1, 2, 3, 4, 5, 6}, the set of numbers on the 6 faces of a
die.
As indicated earlier, the modern axiomatic approach to Probability is based
on the concept of a set. We now show how the concept of a set forms the basis
of Probability Theory.
In most practical applications, one is often interested in some subsets of a
universal or full set Ω. Loosely, an event is a subset of the sample space. We
now define formally a family or class of subsets of Ω called events, to which we
shall assign numbers or weights called probabilities.

7
1.5.6 Definition
Let Ω be a non-empty set. A class ξ of subsets of Ω is called an event space
or, more technically, a σ-algebra (read ”sigma algebra”), if it has the following
three properties:

1. Ω ∈ ξ
2. If E ∈ ξ, then E c ∈ ξ.
S∞
3. If E1 , E2 , . . . are in ξ for n = 1, 2, . . ., then n=1 En ∈ ξ
By definition, an event means a set in ξ. The statement “the event E has
occurred” means that the outcome of an experiment can be represented by a
point or element of E.

1.5.7 Example
Suppose that a coin is tossed once. Describe, using set notation:
(i) An event space for the experiment.

(ii) The event that a Head appears.

1.5.8 Solution
(i) The event space is ξ = {∅, {H}, {T }, {H, T }}.
(ii) The event E that a Head appears is E = {H}.

1.5.9 Example
Suppose that a die is rolled once. Describe, using set notation, the event that
an even number occurs.

1.5.10 Solution
The event that an even number occurs means that the outcome of the experiment
is one of the numbers 2, 4, or 6. Thus the event E is given by E = {2, 4, 6}.

1.5.11 Example
Suppose that two dice are rolled. Describe the sample space for the experiment
using:

(i) The roster notation.

(ii) The property method (i.e., general element set-builder notation).

8
1.5.12 Solution
(i) Ω = {(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (2, 1), (2, 2), (2, 3), (2, 4), (2, 5),
(2, 6), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), (3, 6), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5),
(4, 6), (5, 1), (5, 2), (5, 3), (5, 4), (5, 5), (5, 6), (6, 1), (6, 2), (6, 3), (6, 4), (6, 5),
(6, 6)}.
(ii)
Ω = {(x, y) : x = 1, . . . , 6; y = 1, . . . , 6}.

1.5.13 Example
Suppose that a coin is tossed until a head appears, and let the outcome be the
number of tosses required. Describe the sample space for the experiment.

1.5.14 Solution
The sample space is Ω = N = {1, 2, 3, . . .}, the set of natural numbers.

1.5.15 Example
An experiment consists of measuring the lifetime, in hours, of a bulb. Describe
the sample space for the experiment and the event that the lifetime exceeds 10
hours.

1.5.16 Solution
Ω = [0, ∞) = {t : 0 ≤ t < ∞}.
The event that the lifetime of a bulb exceeds 10 hours is given by:

(10, ∞) = {t : t > 10}.

1.6 Remark
In most cases of practical interest, the event space is the set of all subsets of a
given sample space. It is important to note, however, that not all subsets of a
sample space are events. Since we are interested in subsets that are events, we
shall not discuss subsets that are not events.
From the foregoing, it is clear that operations with events are just operations
with sets and subsets of a universal set. It is important to observe, however,
the physical or practical interpretation of the operations of union, intersection,
etc., when dealing with events. The interpretation is given below.
Let ξ = {E : E ⊆ Ω} be an event space and E and F be events in ξ. Then:

(i) The union E ∪ F of E and F represents the event that either E or F

occurs.
(ii) The intersection E ∩ F represents the event that both E and F occur.

9
(iii) The complement E 0 represents the event that E does not occur.
(iv) The event E 0 ∪ F 0 represents the event that neither E nor F occurs.
(v) The event E ∩ F 0 represents the event that E occurs and F does not.
Other complex or compound events involving two or more events can be inter-
preted in a similar manner.

1.6.1 Definition
Two events E and F are said to be mutually exclusive if E ∩ F = ∅ (i.e., if
their intersection is empty). In practice, to say that two events are mutually
exclusive means that they cannot occur simultaneously. For example, if a die is
rolled once, we let E = {1, 3, 5} be the event that an odd number appears and
F = {2, 4, 6} be the event that an even number appears. It is clear that E and
F are mutually exclusive.

1.6.2 Activity
1. Suppose that a coin is tossed three times. Describe the sample space for
this experiment. 2. Suppose that a die is rolled until a 6 appears. Let the
outcome be the number of rolls required. Describe the sample space for the
experiment. 3. Two numbers are selected at random from the interval (0, 1).
Let the outcome be the sum of the two numbers. Describe the sample space for
this experiment. 4. Let E and F be events. Is A = (E c ∪ F )c an event? If it is
an event, explain in words what the event represents.

1.7 Monotone Sequence of Events

1.7.1 Definition
(i) A sequence of events E1 , E2 , . . . is said to be monotone increasing if E1 ⊆
E2 ⊆ . . ..
(ii) Similarly, the sequence E1 , E2 , . . . is said to be monotone decreasing if
E1 ⊇ E2 ⊇ . . ..

1.7.2 Theorem
(i) Let E1 ⊆ E2 ⊆ . . . be an increasing sequence of events. Then the sequence
has a limit, and that limit is given by:
∞
[
lim En = En .
n→∞
n=1

(ii) A decreasing sequence of events E1 ⊇ E2 ⊇ . . . has a limit given by:

∞
\
lim En = En .
n→∞
n=1

10
1.7.3 Example
Let En = [0, n). Find the limit of {En }∞
n=1 .

1.7.4 Solution
It is clear that [0, 1) ⊆ [0, 2) ⊆ [0, 3) . . ., i.e., {En } is increasing. It follows that
the limit of {En } is given by:

lim En = [0, ∞).

n→∞

1.7.5 Example
Let En = [0, 1 + n1 ]. Find the limit of {En }∞
n=1 .

1.7.6 Solution
Since [0, 2] ⊇ [0, 1 + 12 ] ⊇ . . . is decreasing and 1
n → 0 as n → ∞, it follows that:
∞
\
lim En = En = [0, 1].
n→∞
n=1

1.7.7 Activity
T∞
1. Let En = [0, 1 + n1 ]. Find the limit of En . 2. Prove that Fn = m=n Em is
monotone. Is the sequence increasing or decreasing?

1.8 Summary of the Lecture

In this Lecture, we have discussed some basic results of Set Theory. We defined a
set, random experiment, event, and event space. We have also discussed various
laws pertaining to operations with sets and events, including union, intersection,
complement, DeMorgan’s laws, and limits of sequences of events.

1.9 Exercise
1 List the elements of each of the following sets:

(a) The set of integers between 1 and 10 that are divisible by 2.

(b) The set of integers between 1 and 10 that are divisible by 3.
(c) The set A = {x : x2 + x − 6 = 0}.
(d) The set B = {x : 2x − 4 = 0 and x < 5}.
(e) The set C = {x : 2x − 4 = 0 and x ≥ 5}.
2 Let U = (−∞, ∞) and A = {x : x2 ≤ 4} and C = [0, 1]. Find:
(a) A ∪ B

11
 (b) A ∪ C
(c) A ∩ B
(d) Ac ∩ B
(e) (Ac ∩ B)c .

3 Suppose that a coin is tossed twice. Describe the sample space for the
experiment.
4 Suppose that two dice are rolled once and let the outcome of the experi-
ment be the sum of the two numbers that appear:
(i) Describe the sample space for the experiment.
(ii) Let E be the event that the sum of the two numbers is 7. Describe
the event E using the roster method.
5 Suppose a coin is tossed until 2 heads appear. Let the outcome of the
experiment be the number of tosses required. Describe the sample space
for the experiment.
6 The maximum age in years that a robin (bird) can live is 1 year:
(i) Describe the sample space for the lifetime of a robin.
(ii) Let E be the event that a newly born robin dies within 3 months of
its birth. Describe the event E using set-builder notation.

7 Let En = [0, (1 − n1 )n]. Find the limit of En .

S∞
8 Is the sequence Fn = m=n Em increasing or decreasing?

12