ECE270: Handout 1

Basic Concepts of Probability Theory (Part I)

Outline:
1. a random experiment,
2. an experiment outcome,
3. sample space and its three different types,
4. events,
5. review of set theory, Venn diagrams, and DeMorgan’s laws.

F Random Experiment
• Random experiment is an experiment in which the outcome varies in a unpredictable fashion,
when the experiment is repeated under the same conditions.
• A random experiment is denoted by E and is specified by i) stating an experimental proce-
dure, ii) exactly what is (are) measured or observed.

EXAMPLE 1

– An information block of length 5 is transmitted repeatedly over a noisy channel until

an error-free block arrives at the receiver. Count the number of transmission required.
Count the number of errors in each transmitted block.
– Measure the lifetime of a given computer memory chip in a specified environment.
– Pick two numbers between 0 and 1.
– Pick a number x between 0 and 1 randomly and then pick a number y between x and 1.
– Flip a coin twice. Note the sequence of heads and tails. Note the number of tails.

F An Experiment Outcome
• The result of a random experiment is called outcome. An outcome is denoted by lowercase
letters x, y, t, ξ, .... When we perform a random experiment one and only one outcome occurs.
So outcomes are mutually exclusive, in the sense that they cannot occur simultaneously.
F Sample Space
• Random experiments do not result in the same outcome. We should determine the set of all
possible outcomes. The set of all possible outcomes is called sample space and denoted as S.
An outcome ξ is an element of the set S, i.e., by set theory notation we have ξ ∈ S.
• We can view the result of a random experiment as a random selection of a single outcome ξ
from the set S.
 EXAMPLE 2

• A hospital administrator codes incoming patients according to whether they have insurance
(coding 1 if they do and 0 if they don’t) and their condition (rated as good, fair, serious). Cod-
ing the patients is a random experiment with sample space S = {(0, g), (0, f ), (0, s), (1, g), (1, f ), (1, s)},
ξ = (1, f ) ∈ S is an outcome.

Exercise: What are the sample spaces S of the random experiments in Example 1?

F Different Types of Sample Space

• A sample space S can be i) finite, ii) countably infinite, iii) uncountably infinite.

• A sample space S is either discrete or continuous.

• A discrete sample space S is either finite or countably infinite. A continuous sample space S
is uncountably infinite.

EXAMPLE 3 (different types of S)

– finite S, e.g., S = {1, 2, 3, 4, cos(π/6), sin(π/3)}, 6 ∈ S, S = {T T, T H, HT, HH}, T H ∈

S
– countably infinite S, e.g., S = {0, 1, 2, 3, ...}, 270 ∈ S, S = {T, HT, HHT, HHHT, ...}, T H ∈
/
S, HHHHT ∈ S
– uncountably infinite S, e.g., S = {θ : 0 6 θ 6 2π}, π ∈ S, S = {(x, y) : 0 6 x 6
y 6 2π}, (5.4, π) ∈ S (typical examples are: S is the real line, an interval, the plane, a
region).

Exercise: Can you think of a random experiment corresponding to the sample spaces in
Example 3?

Exercise: Consider the sample spaces of the random experiments in Example 1. Determine
the type of each sample space (i.e., whether it is finite, countably infinite, or uncountably
infinite).

Page 2
 F Events

• An event is a set consisting of outcome(s) and is denoted by capital letters A, B, E, F, .... Any
event E is a subset of the sample space S, i.e., E ⊂ S (Recall: in set theory for two sets A
and B, we say A ⊂ B if every element of A also belongs to B).

• We say the event E happens if the outcome of the experiment ξ belongs to the set E.

• An event from a discrete sample space S (S is either finite or countably infinite) may consist
of a single outcome. Such an event is called elementary event.

• Two events of special interest are: i) certain event S, this event always occurs, ii) null or
impossible event Φ, this event includes no outcome and hence never occurs.
Exercise: Consider the the random experiments in Example 1. For each random experiment
specify i) the null event, ii) the certain event, iii) an event that is neither certain nor impos-
sible.

EXAMPLE 4
Let consider Example 2 again.
Suppose A is the event that the patient is in serious condition ⇒ A = {(0, s), (1, s)}, we have
A ⊂ S, (0, s) ∈ A, (0, f ) ∈
/ A.
Suppose B is the event that the patient is uninsured ⇒ B = {(0, g), (0, f ), (0, s)}, we have
B ⊂ S, (0, f ) ∈ B, (1, f ) ∈
/ B.
Suppose C is the event that the patient is uninsured and is in serious condition ⇒ C = {(0, s)},
we have (0, s) ∈ C (C is an elementary event).

F Review of Set Theory

• In probability theory we are interested in determining the probabilities of events. Since each
event is represented by a set we need to know the basics of set theory.

• For two events E and F the new event E ∪ F (so-called the union of E and F ) consists of all
outcomes that are either in E or in F or in both.

• For two events E and F the new event E ∩ F or EF (so-called the intersection of E and F )
consists of all outcomes that are in both E and F . If E ∩ F = Φ then E and F are said to
be mutually exclusive.

• For any event E we define the new event E c (so-called the complement of E). The event E c
consists of all outcomes in S that are not in E, i.e., E c occurs if and only if E does not occur.

• For any event E we have E ∪ E c = S and E ∩ E c = Φ.

• For any event E we have (E c )c = E.

• If E = S then E c = Φ. Also, if E = Φ then E c = S.

• For two events E and F we write E ⊂ F if all outcomes in E are also in F .

Page 3
• For two events E and F suppose we have E ⊂ F . Then the occurrence of E necessarily implies
the occurrence of F (why? suppose E occurs. This implies that the random experiment
outcome belongs to E. Therefore, the outcome belongs to F as well. Hence, F occurs.).

• For two events E and F if E ⊂ F and F ⊂ E then we have E = F , i.e., the two events are
equal.

EXAMPLE 5
Let consider Example 4 again. We have C = A ∩ B, {(1, s)} = A ∩ B c , C ⊂ B, C ⊂ A,
B = B ∪ C.

F Venn Diagrams

• When discussing sets, we use Venn diagram to show the relations among sets. S is represented
as consisting of all outcomes in a large rectangular, the events F, E, G are represented as
consisting of all the outcomes in given circles within the rectangular. Events of interest are
indicated by shading appropriate regions of the diagram.

• The operation of forming unions, intersections, and complements of events obey certain rules
(laws) similar to the rules of algebra. The rules are:

i) commutative laws E∪F =F ∪E and E∩F =F ∩E

ii) associative laws (E ∪ F ) ∪ G = E ∪ (F ∪ G) and (E ∩ F ) ∩ G = E ∩ (F ∩ G)
iii) distributive laws (E∪F )∩G = (E∩G)∪(F ∩G) and (E∩F )∪G = (E∪G)∩(F ∪G)

F DeMorgan’s Laws

• DeMorgan’s laws describes a useful relationship between the three basic operations of forming
unions, intersections, and complements:

a) (E ∪ F )c = E c ∩ F c b) (E ∩ F )c = E c ∪ F c

Proof of a): we show 1) (E ∪ F )c ⊂ E c ∩ F c and 2) E c ∩ F c ⊂ (E ∪ F )c . Putting 1) and 2)

together we conclude (E ∪ F )c = E c ∩ F c .

– suppose ξ ∈ (E ∪ F )c ⇒ ξ ∈ / E∪F ⇒ ξ ∈ / F ⇒ ξ ∈ E c and ξ ∈ F c ⇒ ξ ∈

/ E and ξ ∈
E c ∩ F c ⇒ (E ∪ F )c ⊂ E c ∩ F c
– suppose ξ ∈ E c ∩ F c ⇒ ξ ∈ E c and ξ ∈ F c ⇒ ξ ∈
/ E and ξ ∈
/ F ⇒ξ∈
/ E∪F ⇒ ξ ∈
(E ∪ F )c ⇒ E c ∩ F c ⊂ (E ∪ F )c .

Proof of b): we use rule a)

(E c ∪ F c )c = (E c )c ∩ (F c )c = E ∩ F . By taking complement we find (E ∩ F )c = E c ∪ F c .

Page 4