‫بسم هللا الرحمن الرحيم‬

Chapter 1
Introduction to Probability Theory
In this chapter, we explore the basics of probability theory. We start by introducing
basic terminology, and then proceed to study axioms of probability theory. This is
followed by a review of a collection of important probability laws.

1.1 Random Experiments

Probability theory is the study of random or non-deterministic experiments. A random
experiment is an experiment whose outcome cannot be predicted with certainty; that is;
it is subject to some element of chance.

1.1.1 Examples of Random Experiments

1- If a dice is tossed in the air, then it is certain that the die will come down showing
one of its faces, but it is not certain that a 6 will appear on this face.
2- In physics lab, you are sure that the accuracy of the avometer is 0.001 volts, but you
cannot in advance predict that the error in your next reading is 0.0002 volts. You can
only say that the error lies somewhere in the interval ]  0.0005,0.0005] .
3- In a voice communication network, the number of calls and their durations are
random.

1.1.2 Applications

Understanding probability theory serves as an important tool in decision making

problems and developing new designs or improving existing ones.
1- In reliability analysis, the lifetime of machines used in a particular power system is
non-deterministic. You need to have identical copies of these machines as spare items
to avoid system failure. The number of copies to be purchased is dependent on the
probability of failure of a given machine as well as the connections and dependencies
among the functionalities of various machines.
2- In design of communication networks, the random usage profile must be modeled
adequately in order to determine the required system architecture to meet service
requirements, such as minimizing losses and transmission time, with minimal cost.
Similarly, in the design of power distribution networks, probability theory plays an
important role.

1.2 Sample Space and Events

The following definitions will be used repeatedly throughout the course.

1- The set (S) of all possible outcomes of a random experiment is called the sample
space.
2- A set is countable if it is finite or if its elements can be arranged in the form of a
sequence (there is a one-to-one correspondence with natural numbers), in which case it
is said to be countably infinite; otherwise the set is uncountable. Intervals on the real
line are uncountable sets. The set of points inside the unit circle is another example.
3- The sample space is said to be discrete if the set S is a countable set (finite or
infinite). On the other hand, the sample is said to be continuous if the set S is an
uncountable set.
1
4- An event is a subset of the sample space S representing a collection of related
outcomes of a random experiment. The empty set  is used to represent impossible
events and S is sometimes referred to as the certain or sure event.

Events can be combined using various set operations:

 A  B : consists of all outcomes that are contained in either of two events (A or
B (or both), any of A, B).
 A  B : consists of all outcomes that are contained in both of the two events (A
and B).
 A c : consists of the set of outcomes in S that are not in the event.

5- Two mutually exclusive (disjoint) events A, B are events that cannot occur
simultaneously, that is, A  B   . The occurrence of event A implies that B did not
happen, similarly, the occurrence of event B implies that A did not happen. For
example, the appearance of a 6 on a rolled dice and the appearance of a 5 are mutually
exclusive events. A student studying a course without completing its pre-requisites are
mutually exclusive events.

Example 1: Determine the sample space for each of the following random experiments
and indicate the elements of the event specified.

(a) Tossing a coin two times, A "a head appears once".

This is a multi-stage experiment and the number of possible outcomes in each stage is
small (2), therefore, it is best demonstrated by a tree diagram.
H

H
T

T
Figure 1 Tossing a coin twice

The sample space S  {HH , TT , HT , TH} and the event A  {HT , TH} . The sample
space is discrete and finite. The number of its elements is 2 2 .

(b) In a continuous production line, testing products until the first defective item is
found, B "the defective product is found after at least three trials".

S  {D, GD, GGD , GGGD , } which is a discrete but infnite sample space.
D  {GGD , GGGD , }

(c) Measuring the duration of a phone call in minutes, C "the call lasts at most 30
seconds".

S  [0, [ , which is a continuous sample space

C  [0,0.5] min
2
(d) Measuring the duration of a phone call to the nearest minute, D "the call lasts
between 2 and 4 minutes inclusive".
S  {0,1,2, , } , which is a discrete sample space
D  {2,3,4}

(e) Selecting two items from a batch consisting of 1 defective item and 9 good items.

D G

G
Figure 2 Selection without replacement from a batch

S  {DG , GD , GG}

(f) Selecting two items from a batch consisting of three distinct items labeled {a,b,c},
assuming selection is with replacement.

S  {aa , ab, ac, ba , bb, bc, ca , cb, cc} (ordered outcomes)

1.3 Interpretations of Probability

Probability is used to quantify the likelihood, or chance, that an outcome of a random
experiment will occur. The likelihood of an outcome is quantified by assigning a
number from the interval [0,1] to the outcome.
Another common interpretation of probability is based on the conceptual model of
repeated replications of the random experiment. The probability of an outcome is
interpreted as the limiting value of the proportion, or relative frequency, of times the
outcome occurs in n repetitions of the random experiment as n increases beyond all
bounds. For example, if we assign a probability 0.2 to the outcome that there is a
corrupted pulse in a digital signal, we might interpret this assignment as implying that,
if we analyze many pulses, approximately 20% of them will be corrupted.

1.4 Axioms of Probability Theory

In probability theory, there are three axioms from which all probability laws can be
derived:
i) 0  P(A)  1 , where P(A) is used to denote the probability of the event A.
ii) P(S)  1 (the sum of probabilities of all possible outcomes is equal to 1)
iii) If A and B are disjoint events, then P(A  B)  P(A)  P(B) .

1.5 Important Probability Laws

In what follows, we state some of the important probability laws and give a sketch of
their proofs.
3
(i) P(A c )  1  P(A)
 A  A c  S and A  A c  
 P(A  A c )  P(A)  P(A c )  P(S)  1
(ii) For any two events A and B, P(A  Bc )  P(A)  P(A  B)
First, the event A can be written as the union of the two disjoint events A  B and
A  Bc
P(A  Bc )  P(A  B)  P(A)
(iii) For any two events A and B, P(A  B)  P(A)  P(B)  P(A  B)
First, A  B can be written as the union of three disjoint events (see Figure 3)
A  B  (A  Bc )  (B  A c )  (A  B)
 P(A  B)  P(A  Bc )  P(B  A c )  P(A  B)
and from part (ii) the law directly follows.

S
A B

Figure 3 Partition of A  B into three disjoint sets

(iv) For any two events A and B,

P(A c  Bc )  1  P(A  B) and P(A c  Bc )  1  P(A  B)
Both laws follow directly from De-Morgan's laws.
Thus, another formula for computing the probability of the union of two events is
P(A  B)  1  P(A c  Bc )
The generalization of the above formula in case of lots of possibilities or alternatives
turns out to be extremely useful as we will see later
P(A1  A 2  A 3    A k )  1  P(A1c  A 2 c    A k c )
Example 2: Given P(A)  0.4 , P(B)  0.5 and P(A  Bc )  0.2 , find P(A  B) .
P(A  Bc )  P(A)  P(A  B) , P(A  B)  0.4  0.2  0.2
P(A  B)  P(A)  P(B)  P(A  B)  0.4  0.5  0.2  0.7

1.6 Finite Equi-probable Sample Spaces

Frequently, the physical characteristics of an experiment suggest that the various
outcomes of the sample space be assigned equal probabilities. For a finite probability
space S, where each sample point (element) has the same probability is called an
equiprobable or uniform space. In particular, if S contains NS elements, then the
probability of each element is ( 1 / NS ). Furthermore, if an event A contains N A
elements, then its probability is
no. of elements in A N A
P( A )  
no. of elements in S N S

4
It is natural to assume that the sample space of the experiment under consideration is
equiprobable unless otherwise stated.

Example 3: Determine the probability of an even number or a prime number appearing

in the toss of a fair dice.
S  {1,2,3,4,5,6} (finite)

The word "fair" emphasizes the fact that the sample points all have the same probability.
1
P(1)  P(2)    P(6) 
6
The event A that an even number appears is given by
3 1
A  {2,4,6} → P(A )  
6 2
The event B that a prime number appears is given by
3 1
B  {2,3,5} → P( B)  
6 2
1
A  B  {2} and P(A  B) 
6
Therefore, the required probability can be computed as
5
P(A  B)  P(A)  P(B)  P(A  B) 
6
Alternatively, we can directly find the event A  B  {2,3,4,5,6} and thus arrive at the
same result.

1.7 Continuous Uniform Sample Spaces

A continuous sample space may be an interval on the real line, a region in plane, a
volume in space,..,etc. In a uniform continuous sample space, the probability of some
event A is expressed in terms of a ratio of some geometrical measurement.

length of interval of A area of region of A

P( A)  or P( A ) 
length of interval of S area of region of S
volume of A
or P(A) 
volume of S

Example 4: The thickness of a copper wire is uniformly distributed over the interval
[1,3] mm , what is the probability that the thickness of a randomly selected piece of wire
lies between 2 and 2.5mm?
Define the event A to be that the selected copper wire has thickness lying within the
interval ]2,2.5[
2.5  2 0.5
P( A )    0.25
3 1 2

1.8 Conclusions
In this chapter, we have studied the fundamental concepts and laws of probability
theory. It appears that techniques for determining the cardinality of a set without
enumerating its elements play an important role in the calculations of probability over
finite discrete sample spaces. This is the topic of the next chapter.