Probability, Statistics and Reliability (with MATLAB)

Ignatius Nyendah

Department of Eng. & Informatics

March 14, 2024

Course Outline

Basic Set Theory

Naive Definitions & Examples

Set Operations

Set Algebra

Probability Theory

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 2 / 135
Basic Set Theory

A branch of mathematics that deals with sets, their formal properties

and correspondences between them is known as Set Theory.
Some key concepts of probability such as sample space, events and
random experiments are defined from sets and the correspondences
between them

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 3 / 135
Definition
A set is a “collection" of well-defined objects called elements or members
of that set.
Sets are usually denoted with capital letters A, B, C , X , Y , etc. while
elements of the set are denoted with lower case letters
a, b, c, x , y , etc.. If x is an element or a member of set A, this can be
written by using set builder notation as x ∈ A otherwise x ∈/ A.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 4 / 135
Example
1. The set of natural numbers N = {0, 1, 2, . . . } whose members or
elements are 0, 1, 2, . . .
2. In set theory, there is a set known as the “empty" set, which means a
set with no elements and it is denoted by ∅. It is formally and logically
used in set operations. Hence, if x is any object then the statement x ∈ ∅
is logically false.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 5 / 135
Set Representation
Basically, a set can be represented in two different forms:
i. The “roster or list form" in which members of such a set are listed
without replacement between curly brackets and separated by
commas or semicolons.
ii. The “statement form" where elements of the set are represented by a
rule or a statement that is defined between curly brackets.

Example
i. A := {0, 2, 4, 6}
ii. A := { n ∈ N: n is even and n ≤ 6}

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 6 / 135
(Graphical Representation of sets)
A set can also be graphically represented by diagrams called the Venn
diagram as it has been shown below for some set A, B and C with their
correspondences.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 7 / 135
Definition
Let A be any set. We say the cardinality or size of A is defined as the
number of elements or members in A. This number is denoted by Card(A)
, #A or |A|. Also, we say that the set A is finite if its size is finite, thus if
#A ∈ N otherwise it is said to be infinite and we write it as #A = ∞.

Example
1. Since the empty set ∅ has no element, then clearly #∅ = 0.
2. The set of numbers on a six-sided dice in a roster form is
Ω = {1, 2, 3, 4, 5, 6}. Obviously #Ω = 6 ∈ N and hence Ω is finite.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 8 / 135
Definition
Let A and B be two sets. We say that A is a subset of B (or that A is
included in B) if every member of A is also a member of B. This is
denoted by A ⊂ B and on the contrary A ̸⊂ B.

Remark
For any set E , one always have the ∅ ⊂ E and E ⊂ E .

Exercise
If A and B are two set such that A ⊂ B, then show by contradiction that
#A ≤ #B.

Definition
Let A and B be two sets. We say that A and B are equal if and only if
A ⊂ B and B ⊂ A and it is denoted by A = B otherwise A ̸= B.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 9 / 135
Definition
A set is said countable if it is infinite and if one can enumerate its
members (without repetition) using an infinite subset of natural numbers.
Otherwise it is said uncountable. A set is said to be most countable if it is
finite or countable as above.

Example
1) The set Z := {. . . , −2, −1, 0, 1, 2, . . . } of integers is countable but this
is not so obvious to prove!
2) But clearly N is countable by definition.

Exercise
Consider the following sets: R, N, C, Z, D and Q.
1) Name and range them in the order of inclusion from the tiniest to the
biggest.
2) Tell which ones are countable or uncountable.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 10 / 135
 The complement of a set A, with respect to the universal set Ω, is
the set {x ∈ Ω|x ∈
/ A} of all the elements of Ω that do not belong to
A and is denoted by Ac .
Note that Ωc = ∅.
The union of two sets A and B is the set of all elements that belong
to A or B (or both) and is denoted by A ∪ B. The intersection of
two sets A and B is the set of all elements that belong to both A and
B and it is denoted by A ∩ B.
Thus

A ∪ B = {x |x ∈ A or x ∈ B},
A ∩ B = {x |x ∈ A and x ∈ B}.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 11 / 135
In some cases, we will have to consider the union or the intersection
of several, even infinitely many sets, defined in the obvious way. For
example, if for every positive integer n, we are given a set An , then

∞
[
An = A1 ∪ A2 ∪ · · · = {x |x ∈ An for some n}
n=1

and
∞
\
An = A1 ∩ A2 ∩ · · · = {x |x ∈ An for some n}.
n=1

Two sets are said to be disjoint if their intersection is empty. Also,

they are said to be disjoint if no two of them have a common
element. A collection of sets is said to be a partition of a set A if the
sets in the collection are disjoint and their union is A.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 12 / 135
Set Algebra

Set operations have several properties, which are elementary consequences

of the definitions. Some examples are:

A∪B =B∪A
A ∩ (B ∪ C ) = (A ∩ B) ∪ (A ∩ C )
(Ac )c = A
A∪Ω=Ω
A ∪ (B ∪ C ) = (A ∪ B) ∪ C
A ∪ (B ∩ C ) = (A ∪ B) ∩ (A ∪ C )
A ∩ Ac = ∅
A∩Ω=A

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 13 / 135
Two particularly useful properties are given by de Morgan’s Law which
states that
∞
[ c ∞
\
An = Acn
n=1 n=1

and

∞
\ c ∞
[
An = Acn .
n=1 n=1

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 14 / 135
Probability Theory

Probability theory is concerned with the study of random (or chance)

phenomena.
Such phenomena are characterized by the fact that their future
behaviour is not predictable in a deterministic fashion.
The role of probability theory is to analyze the behaviour of a system
or an algorithm assuming the given probability assignments and
distributions.
Probability theory provides a modeling framework for the
mathematical study of phenomena influenced by randomness.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 15 / 135
Basic Concepts of Probability
The most pertinent fundamental concepts applied in structural reliability
include:
Experiment
Random Experiment
Sample Space
Event
Definition
An experiment in the theory of probability is understood as the realization
of a certain set of conditions.

Definition
A random experiment is any real-life situation whereby one performs an
experiment or phenomenon and cannot predict the outcome(s) with
certainty.
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 16 / 135
Definition
The sample space (or the outcome space) for a random experiment is the
set, usually denoted by S or Ω , of all its possible outcomes.

Definition
An event is simply a collection of certain sample points, that is, a subset
of the sample space. Mathematically, if we consider any random
experiment with sample space Ω.

1. An event for that experiment is any subset A of Ω.

2. The set of all events is the “power set" P(Ω) := {A : A ⊂ Ω}.
3. We say that an event A occurs if the outcome of the experiment lies
in A.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 17 / 135
Class Activity

An experiment consists of recording the lawyer (by number) assigned to

the next three cases. The outcome 132 means lawyer 1 was assigned case
1, lawyer 3 was assigned case 2, and lawyer 2 was assigned case 3.
a. Find the probability that all three cases are assigned to different
lawyers.
b. Find the probability that lawyer 2 is not assigned to any of the three
cases.
c. Find the probability that lawyer 2 is assigned to at least one case.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 18 / 135
Probabilistic Models
A probabilistic model is a mathematical description of an uncertain
situation. The two main elements of probabilistic model include:
The sample space
Probability law

Figure: The main ingredients of a probabilistic model.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 19 / 135
Sequential Models
Many experiments have an inherently sequential character, such as for
example tossing a coin three times, or observing the value of a stock
on five successive days, or receiving eight successive digits at a
communication receiver. Therefore, it is often useful to describe the
experiment and the associated sample space by means of a tree-based
sequential description, as shown below in

Figure: A sample space of an experiment involving two rolls of a 4-sided die. Th

possible outcomes are all the ordered pairs of the form (i, j)
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 20 / 135
Probability Axioms

1. (Nonnegativity) P(A) ≥ 0 for every event A

2. (Normalization) The probability of the entire sample space Ω is
equal to 1, that is, P(Ω) = 1.
3. (Additivity) If A and B are two disjoint events, then the probability
of their union satisfies

P(A ∪ B) = P(A) + P(B) (1)

Furthermore, if the sample space has an infinite number of elements and

A1 , A2 , ... is a sequence of disjoint events, then the probability of their
union satisfies

P(A1 ∪ A2 ∪ . . . ) = P(A1 ) + P(A2 ) + · · · (2)

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 21 / 135
Discrete Models

There are many instances when we are faced with the analysis of discrete
data rather than continuous data.
Discrete Probability Law
Given a finite number of possible outcomes, the probability law is specified
by the probabilities of the events that consist of a single element. The
probability of any event {s1 , s2 , . . . , sn } is the sum of the individual
probabilities of its elements and it is given as
P({s1 , s2 , . . . , sn }) = P(s1 ) + P(s2 ) + · · · + P(sn )

Discrete Uniform Probability

number of element of A (n)
P(A) = number of possible outcomes in Ω (N)

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 22 / 135
Discrete Models Cont.

Example
Consider the experiment of rolling a pair of 4-sided dice . We assume the
dice are fair, and we interpret this assumption to mean that each of the
sixteen possible outcomes [pairs(i, j), withi.j = 1, 2.3, 4] has the same
1
probability of 16 . To calculate the probability of an event , we must count
the number of elements of the event and divide by 16 (the total number of
possible outcomes). Find the following
1. Write all elements of Ω

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 23 / 135
Continuous Models

Probabilistic models with continuous sample space differ from their

discrete counterpart in the sense that the probabilities of the
single-element may not be sufficient to characterize the probability law.
Some Properties of Probability Laws
Consider a probability law, and let A, B and C be events.
a) If A ⊂ B, then P(A) ≤ P(B).
b) P(A ∪ B) = P(A) + P(B) − P(A ∩ B).
c) P(A ∪ B) ≤ P(A) + P(B).
d) P(A ∪ B ∪ C ) = P(A) + P(Ac ∩ B) + P(Ac ∩ B c ∩ C )

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 24 / 135
The Fundamental Counting Principle

We use the Fundamental Counting Principle to determine the number of

possible outcomes in a given situation.
Definition
If you can choose one item from a group of M items and the second item
from a group of N items, then the total number of two- item choices is
M × N or M · N.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 25 / 135
Example
A representation of all possible Choice. This tree diagram shows that there
are 6 different outfits from 2 pairs of jeans and three T-shirts.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 26 / 135
Example
Trial The Greasy Spoon Restaurant offers 6 appetizers and 14 main
courses. In how many ways can a person order a two-course meal?

Example
Draw a tree diagram to represent the number of possible outfits from 2
pairs of jeans, 3 T-shirts and 2 pairs of sneakers.

Example
Next semester, you are planning to take three courses: math, English and
humanities. There are 8 sections of math, 5 of English, and 4 of
humanities that you find suitable. Assuming no scheduling conflicts, how
many different three-course schedules are possible

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 26 / 135
Permutations

Definition
Permutation is an ordered arrangement of items that occurs when:
No item is used more than once.
The order of arrangement makes a difference.

Example
1. You need to arrange seven of your favourite books along a small shelf.
How many different ways can you arrange the books, assuming that
the order of the books makes a difference to you?
2. Suppose a business owner has a choice of 5 locations in which to
establish her business. She decides to rank each location according to
certain criteria, such as price of the store and parking facilities. How
many different ways can she rank the 5 locations?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 27 / 135
A Formula for Permutations

The number of possible permutations of r items are taken from n items is:

n!
n Pr = (3)
(n − r )!

Worked Example
You and 19 of your friends have decided to form a business. The group
needs to choose three officers– a CEO, an operating manager, and a
treasurer. In how many ways can those offices be filled?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 28 / 135
Solution:
Your group is choosing r = 3 officers from a group of n = 20 people. The
order matters because each officer has different responsibilities. We use
the formula (3)
to have
20!
20 P3 =
(20 − 3)!
20!
=
17!
20 × 19 × 18 × 17!
=
17!
= 6840.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 29 / 135
Permutation of Duplicate Items

The number of permutations of n items, where p items are identical, q

items are identical, r items are identical, and so on, is given by
n!
(4)
p!q!r ! . . .

Worked Example
In how many distinct ways can the letters of the word MISSISSIPPI be
arranged?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 30 / 135
Solution:
The word contains 11 letters (n = 11) where four Is are identical (p = 4),
four Ss are identical (q = 4) and 2 Ps are identical (r = 2). The number
of distinct permutations are:
n! 11!
=
p!q!r ! 4!4!2!
11 · 10 · 9 · 8 · 7 · 6 · 5 · 4!
=
4! · 4 · 3 · 2 · 1 · 2 · 1
= 34, 650

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 31 / 135
Combinations

Definition
A selection of distinct objects without regard to order is called a
combination.
Notably, Permutation problems involve situations in which order matters
while Combination problems involve situations in which the order of items
makes no difference.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 32 / 135
A Formula for Combinations

The number of possible combinations if r items are taken from n items is:
!
n n!
n Cr = = , for 0 ≤ r ≤ n. (5)
r r !(n − r )!

Examples
1. How many three-person committees could be formed from 8 people?
2. Suppose a committee of 5 students is to be selected from 25
students. In how many ways can this be done?
3. A newspaper editor has received 8 books to review. He decides that
he can use 3 reviews in his newspaper. How many different ways can
these 3 reviews be selected?
4. How many combinations of 4 objects are there, taken 2 at a time?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 33 / 135
Quiz (20 mins)

1. Suppose that I want to purchase a tablet computer. I can choose

either a large or a small screen; a 64GB, 128GB, or 256GB storage
capacity, and a black or white cover. How many different options do I
have?
2. A newspaper editor has received 8 books to review. He decides that
he can use 3 reviews in his newspaper. How many different ways can
these 3 reviews be selected?
3. I choose 3 cards from the standard deck of cards. What is the
probability that these cards contain at least one ace?
4. In a club there are 7 women and 5 men. A committee of 3 women
and 2 men is to be chosen. How many different possibilities are there?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 34 / 135
Conditional Probability

Conditional probability provides us with a way to reason about the

outcome of an experiment, based on partial information.
Number of elements of A ∩ B
P(A|B) = (6)
Number of elements of B
In generalising the argument, we introduce the following definition of
conditional probability

P(A ∩ B)
P(A|B) = , P(B) > 0. (7)
P(B)

In order words, out of the total probability of elements of B, P(A|B) is the

fraction that is assigned to possible outcomes that also belong to A.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 35 / 135
 P(Ω ∩ B)
P(Ω|B) =
P(B)
P(B)
=
P(B)
= 1 (Nonnegativity and Normalization)

For any two events A1 and A2 such that the event B has occurred, the
conditional probability is given as

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 36 / 135

P(A1 ∪ A2 |B)
P A1 ∪ A2 =

P B
P(A1 ∪ A2 ∩ B)
=

P B


P (A1 ∩ B) ∪ (A2 ∩ B)
=

P B


P (A1 ∩ B) + (A2 ∩ B)
=

P B



P (A1 ∩ B) P (A2 ∩ B)
=
+

P B P B



= P A1 |B + P A2 |B (Additivity)

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 37 / 135
Properties of Conditional Probability

The conditional probability of an event A, given that an event B has

occurred with probability P(B) > 0, is defined as

P(A ∩ B)
P(A|B) = (8)
P(B)

and specifies a new conditional probability on the sample space.

Conditional Probability can also be viewed as a probability law in a
new universe B, because all of the conditional probabilities is
concentrated on B.
If the possible outcomes are finitely many and equally likely, the

number of element of A ∩ B
P A|B =
number of elements of B

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 38 / 135
Modeling Using Conditional Probability

When constructing probabilistic models for experiments that have a

sequential character, it is often natural and convenient to first specify
conditional probabilities and then


use them to determine unconditional
probabilities. The rule P A ∩ B = P(B)P(A|B), which is a restatement of
the definition of conditional probability, is often helpful in this process.
Example
Radar Detection:If an aircraft is present in a certain area, a radar
correctly registers its presence with probability 0.99. If it is not present,
the radar falsely registers an aircraft presence with probability 0.10. We
assume that an aircraft is present with probability 0.05. What is the
probability of false alarm (a false indication of aircraft presence), and the
probability of missed detection (nothing registers, even though an aircraft
is present)?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 39 / 135
A sequential representation of the sample space is appropriate here, as
shown below:

P(B|A) = 0.99

Aircraft Present
P(A) = 0.05

P(B c |A) = 0.01 Missed Detection

P(B|Ac ) = 0.10 False Alarm

P(Ac ) = 0.95
Aircraft not Present

P(B c |Ac ) = 0.90

Figure: Sequential description of the sample space for the radar detection problem.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 40 / 135
Let A and B be the events

A = { an aircraft is present },
B = { the radar registers an aircraft presence },

and consider also their complements

Ac = {an aircraft is not present},

B c = {the radar does not register an aircraft presence}

The desired probabilities of false alarm and missed detection are

P(false alarm) = P(Ac ∩ B) = P(Ac )P(B|Ac ) = 0.95 × 0.10 = 0.095

P(missed detection) = P(A ∩ B c ) = P(A)P(B c |A) = 0.05 × 0.01 = 0.0005.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 41 / 135
Example
I roll a fair dice twice and obtain two numbers α1 = result of the first roll
and α2 = result of the second roll. Given that I know α1 + α2 = 7, what is
the probability that α1 = 4 or α2 = 4 ?

Solution: Let A be the event that α1 = 4 or α2 = 4 and B be the event

that α1 + α2 = 7. We are interested in P(A|B), so we can use
P(A ∩ B)
P(A|B) = .
P(B)
We note that
A = {(4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6), (1, 4), (2, 4), (3, 4), (5, 4), (6, 4)},
B = {(6, 1), (5, 2), (4, 3), (3, 4), (2, 5), (1, 6)}A ∩ B
So, we have
2
36 1
P(A|B) = 6 =
36
3
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 42 / 135
Multiplication Rule
If we consider an experiment where we are dealing with an event B which
occurs if and only if each one of the several events A1 , . . . , An has
occurred, i.e. A = A1 ∩ A2 ∩ · · · ∩ An The occurrence of A is viewed as an
occurrence of A1 , followed by the occurrence of A2 , then of A3 ,etc and it
visualized as a path on the tree with n branches, corresponding to the
events A1 , . . . , An . The probability of A is given by the rule below
Multiplication Rule
Assuming that all of the conditioning events have positive probability,
we have

n
\   n−1
\ 
P Ai = P(A1 )P(A2 |A1 )P(A2 )P(A3 |A1 ∩ A2 ) · · · P An | Ai
i=1 i=1
(9)
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 43 / 135
Example
Three cards are drawn from an ordinary 52-card deck without replacement
(drawn cards are not placed back in the deck ) . Find the probability that
none of the three cards is a heart.
Solution:
First, we define the events Ai = { the ith card is not a heart }i = 1, 2, 3
We will calculate P(A1 ∩ A2 ∩ A3 ), the probability that none of the three
card is a heart, using the multiplication rule

P(A1 ∩ A2 ∩ A3 ) = P(A1 )P(A2 |A1 )P(A3 |A1 ∩ A2 )

We have
39
P(A1 ) =
52
Since there are 39 card that are not heart in the 52-card deck. Given that
the first card is not a heart, we are left with 51 cards. 38 of which are not
hearts, and
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 44 / 135
 38
P(A2 |A1 ) = .
51
Finally, given that the first two cards drawn are not hearts. there are 37
cards which are not hearts in the remaining 50 card deck. and

37
P(A3 |A1 ∩ A2 ) =
50
Therefore,
39 38 37
P(A1 ∩ A2 ∩ A3 ) = · ·
52 51 50

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 45 / 135
Example
A class consisting of 4 graduate and 12 undergraduate students is
randomly divided into four groups of 4. What is the probability that each
group includes a graduate student?

Solution:
We will calculate P(A3 ) using the multiplication rule

P(A3 ) = P(A1 ∩ A2 ∩ A3 ) = P(A1 )P(A2 |A1 )P(A3 |A1 ∩ A2 )

We have
12
P(A1 ) =
15
Since there are 12 student slots in groups other than the one of student 1,
and there are 15 students slots overall, excluding student 1. Similarly,
8
P(A2 |A1 ) =
14
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 46 / 135
since there are 8 student slots in groups other than those of students 1
and 2, and there are 14 student slots, excluding students 1 and 2. Also
4
P(A3 |A1 ∩ A2 ) =
13
since there are 4 student in groups other than those of students 1, 2 and
3, and there are 13 students slots, excluding student 1, 2 and 3. Thus, the
desired probability is

12 8 4
× × .
15 14 13

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 47 / 135
Total Probability Theorem
Total Probability Theorem
Let A1 , . . . , An be disjoint events that form a partition of the sample
space (each possible outcome is included in exactly one of the events
A1 , . . . , An ) and assume that P(Ai ) > 0, for all i. Then, for any
event B, we have

P(B) = P(A1 ∩ B) + · · · + P(An ∩ B)

= P(A1 )P(B|A1 ) + · · · + P(An )P(B|An ).

Example
You enter a chess tournament where your probability of winning a game is
0.3 against half the players (call them type 1). 0.4 against a quarter of the
players (call them type 2), and 0.5 against the remaining quarter of the
players (call them type 3). You play a game against a randomly chosen
opponent. What is the probability of winning?
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 48 / 135
Solution:
Let Ai be the event of playing with an opponent of type i. We have

P(A1 ) = 0.5, P(A2 ) = 0.25, and P(A3 ) = 0.25.

Also. Let B be the event of winning. We have

P(B|A1 ) = 0.3, P(B|A2 ) = 0.4, and P(B|A2 ) = 0.5

Thus, by the total probability theorem, the probability of winning is

P(B) = P(A1 )P(B|A1 ) + P(A2 )P(B|A2 ) + P(A3 )P(B|A3 )

= (0.5 × 0.3) + (0.25 × 0.4) + (0.25 × 0.5)
= 0.375

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 49 / 135
Example
Alice is taking a probability class and at the end of each week she can be
either up-to-date or she may have fallen behind. If she is up-to-date in a
given week, the probability that she will be up-to-date (or behind) in the
next week is 0.8 (or 0.2, respectively). If she is behind in a given week, the
probability that she will be up-to-date (or behind) in the next week is 0.6
(or 0.4, respectively). Alice is (by default) up-to-date when she starts the
class. What is the probability that she is up-to-date after three weeks?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 50 / 135
Bayes’ Rule

Bayes’ Rule
Let A1 , . . . , An be disjoint events that form a partition of the sample
space and assume that P(Ai ) > 0, for all i. Then, for any event B
such that P(B) > 0, we have

P(Ai )P(B|Ai )
P(Ai |B) =
P(B)
P(Ai )P(B|Ai )
=
P(A1 )P(B|A1 ) + · · · + P(An )P(B|An )

Note: The probability of an event is sometimes computed more easily if

we condition B on another event A and we basically write it as

P(B) = P(B|A).P(A) + P(B|Ac ).P(Ac )

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 51 / 135
Example
A test of a certain rare disease is assumed to be correct 95% of the time:
if a person has the disease, the test results are positive with probability
0.95, and if the person does not have the disease, the test results are
negative of probability 0.95. A random person drawn from a certain
population has probability 0.001 of having the disease. Given that the
person just test positive, what is the probability of having the disease?

Solution:
If A is the event that the person has the disease, and B is the event that
the test results are positive, the desire probability, P(A|B), is

P(A)P(B|A)
P(A|B) =
P(A)P(B|A) + P(Ac )P(B|Ac )
0.001 × 0.95
=
(0.001 × 0.95) + (0.999 × 0.05)
= 0.0187

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 52 / 135
Assignment

Suppose 1 in 100 persons has a certain disease. A test detects the disease
in 99% of diseased person. The test also “detect" the disease in 5% of
healthy persons. With what probability does a positive test diagnose the
disease?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 53 / 135
Independent Events

Independent Events
As we have introduced conditional probability to capture the partial
information that event B provides about event A. An interesting and
important special case arises when the occurrence of B provides no
such information and does not alter the probability that A has oc-
curred

P(A|B) = P(A)

When the above equality holds, we say that A is independent of B. Note

that by definition

P(A ∩ B)
P(A|B) =
P(B)

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 54 / 135
The above relation is equivalent to

P(A ∩ B) = P(A)P(B) (10)

Example
Two numbers are drawn at random from the set {1, 2, 3, 4}. If order is not
important then what is the sample space Ω? Define the following
functions on Ω:

X ({i, j}) = i + j, Y ({i, j}) = |i − j|.

Which of the following pairs of events are independent?

1. X= 5 and Y=2
2. X= 5 and Y=1

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 55 / 135
Conditional Independence Two events A and B are said to be con-
ditionally independent, given another event C with P(C ) > 0, if

P(A ∩ B|C ) = P(A|C )P(B|C ).

Independence of a Collection of Events We say that the events

A1 , A2 and An are independent if

 
\
P Ai  = Πi ∈ SP(Ai ), for every subset S of {1,2,3,. . . , n}
i∈S

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 56 / 135
For the case of three events A1 , A2 and A3 , independence amounts to
satisfying the four conditions

P(A1 ∩ A2 ) = P(A1 )P(A2 )

P(A1 ∩ A3 ) = P(A1 )P(A3 )
P(A2 ∩ A3 ) = P(A2 )P(A3 )
P(A1 ∩ A2 ∩ A3 ) = P(A1 )P(A2 )P(A3 )

The first three conditions simply asserts that any two events are
independent, a property know as pairwise independence. But the fourth
condition is also important and does not follow from the first three.
Conversely, the fourth condition does not imply the first three.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 57 / 135
Section Exercises

1 You purchase a certain product. The manual states that the lifetime
T of the product, defined as the amount of time (in years) the
product works properly until it breaks down, satisfies
t
P(T > t) = e − 5 , ∀t ≥ 0.

You purchase the product and use it for two years without any
problems. What is the probability that it breaks down in the third
year?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 58 / 135
Section Exercises Cont’d

1 In my town, it’s rainy one third of the days. Given that it is rainy,
there will be heavy traffic with probability 12 , and given that it is not
rainy, there will be heavy traffic with probability 41 . If it’s rainy and
there is heavy traffic, I arrive late for work with probability 12 . On the
other hand, the probability of being late is reduced to 18 if it is not
rainy and there is no heavy traffic. In other situations (rainy and no
traffic, not rainy and traffic) the probability of being late is 0.25. You
pick a random day.
a. What is the probability that it’s not raining and there is heavy traffic
and I am not late?
b. What is the probability that I am late?
c. Given that I arrived late at work, what is the probability that it rained
that day?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 59 / 135
Discrete Random Variables

Starting with a probabilistic Model of an experiment:

A random variable is a real-valued function of the outcome of the
experiment.
A function of a random variable defines another random variable.
We can associate with each random variable certain “averages" of
interest, such as the mean and the variance.
A random variable can be conditioned on an event or another
random variable.
There is a notion of independence of a random variable from an
event or from another random variable.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 60 / 135
Definition
A discrete random variable is a function X (s) from a finite or countably
infinite sample space Ω to the real numbers:

X (.) : Ω → R.

Given an experiment and the corresponding set of possible outcomes (the

sample space), a random variable associates a particular number with each
outcome. We refer to this number as the numerical value or simply the
value of the random variable. Mathematically, a random variable is a
real-valued function of the experimental outcome.
Example
Toss a coin 3 times in sequence. The sample space Ω is given a

Ω = {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }

and examples of random variables include the following

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 61 / 135
 X (s) = number of Heads in the sequence; e.g. X (HTH) = 2.
Y (s) = The index of the first Head; e.g. Y (TTH) = 3.
Note: In this example X (s) and Y (s) are actually integers. Value Ranges

of a random variable correspond to events in Ω.

Example
By considering the sample space

Ω = {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }

with

X (s) = the number of Heads

the value X (s) = 2 corresponds to the event {HHT , HTH, THH}, and the
values 1 < X (s) ≤ 3, corresponds to {HHH, HHT , HTH, THH}.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 62 / 135
Probability Mass Function PMF The most important way to char-
acterize a random variable is through the probabilities of the values
that it can take. For a discrete random variable X , these are cap-
tured by the probability mass function of X . In particular, if x is
any possible value of X , the probability mass of x denoted by pX (x )
is the probability of the event {X = x } consisting of all outcomes the
gives rise to a value of X equal to x:

pX (x ) = P({X = x })

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 63 / 135
Example
Let the experiment consist of tow independent tosses of a fair coin and let
X be the number of heads obtained. Then, the PMF of X is

1
 4 if x = 0 or x = 2


pX (x ) = 1
, if x = 1,
2

0, otherwise

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 64 / 135
Value-Ranges of a random variable correspond to events in Ω have a
probability. Thus Value-ranges have a probability.
Example
For the sample space

Ω = {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }

with

X (s) = the number of Heads,

we have
6
P(0 < X ≤ 2) = .
8

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 64 / 135
Trial: What are the values of

P(X ≤ −1), P(X ≤ 0), P(X ≤ 1) and P(X ≤ 3)

Note: We will use upper case characters to denote random variables, and
lower case characters to denote real numbers such as the numerical values
of a random variable.

X
pX (x ) = 1
x

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 65 / 135
Note: We also write pX (x ) to denote P(X = x ).
Example
For the sample space

Ω = {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }

with

X (s) = the number of Heads,

we have
1
pX (0) = P({TTT }) =
8
3
pX (1) = P({HTT , THT , TTH}) =
8
3
pX (2) = P({HHT , HTH, THH}) =
8
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 66 / 135
 1
pX (3) = P({HHH}) =
8
where

pX (0) + pX (1) + pX (2) + pX (3) = 1.(Why ?)

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 67 / 135
Example
Toss a coin until “Heads" occurs. The sample space Ω is countably
infinite, namely

Ω = {H, TH, TTH, TTTH, . . . }.

The random variable X is the number of rolls until “Head" occurs:

X (H) = 1, X (TH) = 2, X (TTH) = 3, . . .

Then
1 1 1
pX (1) = , pX (2) = , pX (3) = , . . . (why ?)
2 4 8

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 68 / 135
and, as should be the case,
∞ n
1
X X  
p(k) = lim p(k) = lim 1− =1
k=1
n→∞
k=1
n→∞ 2n

NOTE: The outcomes in Ω do not have equal probability!.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 69 / 135
Expectation
We define the expected value (also called the expectation or the mean)
of a random variable X , with PMF pX by
X
E[x ] = x pX x .
x

Example
Consider two independent coin tosses, each with a 34 probability of a head,
and let X be the number of heads obtained. This is a binomial random
variable with parameters n = 2 and p = 43 . Its PMF is
 
 2
1
, if k = 0


4





   
pX (k) = 2 41 3
4 , if k = 1


 2


3
 4 , if k = 2



Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 70 / 135
So the mean is
 2  2
1 1 3 3 24 3
 
E=0· +1· 2· · +2· = =
4 4 4 4 16 2

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 71 / 135
Variance

The variance var (X ) of a random variable is defined by

var (X ) = E[(X − E[X ])2 ]

and it can also be calculated as

X
var (X ) = (x − E[X ])2 pX (x )
x

it is always nonnegative. Its square root is denoted by σX and is called the

standard deviation.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 72 / 135
Properties of Mean and Variance
Given a random variable of the form
Y = aX + b
where a and b are scalars. Let us derive the mean and the variance of the
linear function Y . We have
X X X
E[Y ] = (ax + b)pX (x ) = a xpx (x ) + b pX (x ) = aE[X ] + b.
x x x
Furthermore,
X
var (Y ) = (ax + b − E[aX + b])2 pX (x )
x
X
= (ax + b − aE[X ] − b)2 pX (x )
x
X
2
=a (x − E[X ])2 pX (x )
x
2
= a var (X ).
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 73 / 135
Variance in Terms of Moments Expression

var (X ) = E[X 2 ] − (E[X ])2

This expression is verified as follows

X
var (X ) = (x − E[X ])2 pX (x )
x
X
= (x 2 − 2x E[X ] + (E[X ])2 )
x
X X X
= x 2 pX (x ) − 2E[X ] xpX (x ) + (E[X ])2 pX (x )
x x x
= E[X 2 ] − 2(E[X ])2 + (E[X ])2
= E[X 2 ] − (E[X ])2

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 74 / 135
Example
If the weather is good (which happens with probability 0.6). Alice walks
the 2 miles to class at a speed of V = 5 miles per hour, and otherwise
rides her motorcycle at a speed of V = 30 miles per hour. What is the
mean of the time T to get to class?

Solution:
To solve this problem we first derive the PMF of T
(
2
0.6 if t = 5 hours
pT (t) = 2
0.4 if t = 30 hours

and then the mean is calculated by

2 2 4
E[T ] = 0.6 · + 0.4 · = hours.
5 30 15

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 75 / 135
Exercises

1 Suppose we roll a fair die. If 1 or 2 is showing, let X = 3, if 3 or 4 is

showing, let X = 4, and if 5 or 6 is showing, let X = 10. What is
E(X )?
2 Suppose X = 0 with prob. 12 , 1 with probability 14 , 2 with probability
1 1
8 , and more generally n with 2n+1 . What is E(X )?
3 Suppose we roll a die and let X be the value that is showing. Find the
Var (X ).

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 76 / 135
Joint Distribution

The probability mass function and the probability distribution function

can also be functions of more than one variable.
Probability mass function (pmf) of a single discrete random variable X
specifies how much probability mass is placed on each possible X value.
Probabilistic models often involve several random variables. For example,

in a medical diagnosis context , the results of several tests may be

significant , or in a networking context, the workloads of several routers
may be of interest . All of these random variables are associated with the
same experiment , sample space, and probability law, and their values may
relate in interesting ways.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 77 / 135
Consider two discrete random variables X and Y associated with the same
experiment. The probabilities of the values that X and Y can take are
captured by joint PMF of X and Y denoted by pX ,Y (x , y ) i.e. it describes
how much probability mass is placed on each possible pair of values (x , y ).
In particular, if (x , y ) is a pair of possible values of X and Y , the
probability mass of (x , y ) is the probability of the event

pX ,Y (x , y ) = P(X = x , Y = y ).

Note that, we use the abbreviated notation P(X = x , Y = y ) instead of

the more precise notation

P({X = x } ∩ {Y = y }) or P(X = x and Y = y ).

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 78 / 135
We can define the range from X and Y as

RXY = {(x , y )|PXY (x , y ) > 0}.

In particular, if Rx = {x1 , x2 , . . . } and RY = {y1 , y2 , . . . }, then we can

always write

RXY ⊂ RX × RY = {(xi , yj )|xi ∈ RX , yj ∈ RY }.

For two discrete random variables X and Y , we have

X
PXY (xi , yj ) = 1
(xi ,yj )∈RXY

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 79 / 135
We can use the joint PMF to find P((X , Y ) ∈ A) for any set A ∈ R2 i.e.
the set of all pairs (x , y ). Specifically, we have
X
P ((X , Y ) ∈ A) = P(X ,Y ) (xi , yj )
(x ,y )∈A

The marginal PMFs of X and Y can be obtained from the joint

PMF, using the formulas
X X
pX (x ) = pX ,Y (x , y ) pY (y ) = pX ,Y (x , y )
y x

A function g(X , Y ) of X and Y defines another random variable, and

XX
E[g(X , Y )] = g(x , y )pX ,Y (x , y )
x y

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 80 / 135
Example
An insurance company has customers with both home and auto policy. For
each type of policy, a deductible amount must be specified.
For auto policy, choices are 100 and 250 ghc.
For home policy, choices are 0, 100, and 200 ghc.
Suppose a customer is selected at random. Let
X be his deductible on the auto policy
Y his deductible on the home policy.

All possible values for (X , Y ) are

(100, 0), (100, 100), (100, 200), (250, 0), (250, 100), (250, 200).
Suppose the joint pmf is given by the insurance company below;

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 81 / 135
 p(x , y ) [y /x ] 0 100 200
100 0.20 0.10 0.20
250 0.05 0.15 0.30

1 What is the probability that X = 100?

2 What is the probability that X = 250?
3 Deduce the marginal probability of X (focus only on X)
4 Deduce the marginal probability of Y.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 82 / 135
Example
Toss a coin 3 times in sequence. For the sample space

Ω = {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }

We let

X (s) = number of Heads, Y (s) = index of the first H(0 for TTT ).

Then we have the joint probability mass function

p(X ,Y ) (x , y ) = P(X = x , Y = y ).

For example

pX ,Y (2, 1) = P(X = x , Y = y )
= P(2 Heads, 1st toss is Heads)
1
=
4
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 83 / 135
Independent Random Variable

Two discrete random variables X (s) and Y (s) are independent if

P(X = x , Y = y ) = P(X = x ) · P(Y = y ), for all x and y .

Equivalently, these random variables are said to be independent if their

probability mass functions satisfy

pX ,Y (x , y ) = pX (x )pY (y ).

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 83 / 135
EXERCISE: Are these random variables X and Y independent?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 84 / 135
Expected Value of a Function

The expected value of a function of a random variable is

X
E[g(X )] ≡ g (xk ) pX (xk )
k

EXAMPLE : The pay-off of rolling a die is $k 2 , where k is the side facing

up. What should the entry fee be for the betting to break even?

SOLUTION : Here g(X ) = X 2 , and

6
X 1 1 6(6 + 1)(2 · 6 + 1) 91 ∼
E[g(X )] = k2 = = = $15.17
k=1
6 6 6 6

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 85 / 135
Example

Find the following

i. E[X ]
ii. E[Y ]
iii. E[X , Y ]

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 86 / 135
The expected value of a function of two random variables is
XX
E[g(X , Y )] ≡ g (xk , yℓ ) p(X ,Y ) (xk , yℓ ) .
k ℓ

1 1 1 5
E[X ] = 1 · +2· +3· =
2 3 6 3
2 1 1 3
E[Y ] = 1 · + 2 · + 3 · =
3 6 6 2
1 1 1
E[XY ] = 1 · + 2 · +3·
3 12 12
2 1 1
+2· +4· +6·
9 18 18
1 1 1 5
+3· +6· +9· =
9 36 36 2

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 87 / 135
Conditional Distribution

Reconsider the insurance example:

p(x , y ) [y /x ] 0 100 200

100 0.20 0.10 0.20
250 0.05 0.15 0.30

The conditional pmf of say Y given that X = 100 is:

P(Y = 0 ∩ X = 100)
P(Y = 0|X = 100) = = ............
P(X = 100)
P(Y = 100|X = 100) =...............................
P(Y = 200|X = 100) =......................................

What is the expected value of Y given that X = 100? ("Conditional

Expectation").

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 88 / 135
Variance and Standard Deviation
Let X have mean
µ = E[X ]
Then the variance of X is
h i
(xk − µ)2 p (xk )
X
Var(X ) ≡ E (X − µ)2 ≡
k

which is the average weighted square distance from the mean. We have
h i
Var(X ) = E X 2 − 2µX + µ2
h i
= E X 2 − 2µE[X ] + µ2
h i
= E X 2 − 2µ2 + µ2
h i
= E X 2 − µ2

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 89 / 135
The standard deviation of X is
q q q
σ(X ) ≡ var(X ) = E [(X − µ)2 ] = E [X 2 ] − µ2

which is the average weighted distance from the mean.

EXAMPLE: The variance of rolling a die is
6 
1
X 
2
var(X ) = k · − µ2
k=1
6
 2
1 6(6 + 1)(2 · 6 + 1) 7 35
= − = .
6 6 2 12
The standard deviation is
s
35 ∼
σ= = 1.70.
12

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 90 / 135
Covariance Between Two Discrete Random Variables

Suppose X and Y are discrete and defined on the same sample space.
Then the covariance Cov (X , Y ) between X and Y is defined by

Cov (X , Y ) =E((X − µx )(Y − µy ))

X
= (x − µx )(y − uy )PX ,Y (x , y )
x ,y

Remark
Cov (X , X ) = E((X − µx )2 ) = Var (X )

Cov (X , Y ) = E(XY ) − µx µy

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 91 / 135
Example
Suppose X and Y are discrete with joint pmf given by

y =0 y =1 y =2 y =3
1 2 1
x =0 8 8 8 0
1 2 1
x =1 0 8 8 8

Compute Cov (X , Y ).

Cov (X , Y ) =E(XY ) − µx µy

1
2, x = 0


PX (x ) = 1
2 , x =1


0, otherwise

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 92 / 135
 
1
 8 , y = 0, y = 3


PY (y ) = 3
8 , y = 1, 2


0, otherwise

1
E(X ) =
2
3
E(Y ) =
2
Students are required to complete the solution.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 93 / 135
Exercise

EXERCISE : Prove the following :

var(aX + b) = a2 var(X )
Cov(X , Y ) = Cov(Y , X )
Cov(cX , Y ) = c Cov(X , Y )
Cov(X , cY ) = c Cov(X , Y )
Cov(X + Y , Z ) = Cov(X , Z ) + Cov(Y , Z )
var(X + Y ) = Var(X ) + var(Y ) + 2 Cov(X , Y )

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 94 / 135
Covariance of Independent Variables
Definition
The covariance of two random variables X and Y is

Cov(X , Y ) = E[(X − E[X ])(Y − E[Y ])]

Property: If X and Y are independent then Cov(X , Y ) = 0.

Proof.
We have already shown ( with µX ≡ E[X ] and µY ≡ E [Y ] ) that

Cov(X , Y ) ≡ E [(X − µX ) (Y − µY )] = E[XY ] − E[X ]E[Y ]

and that if X and Y are independent then

E[XY ] = E[X ]E[Y ]

from which the result follows.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 95 / 135
Exercise

Show that
E[X ] = 2
E[Y ] = 8
E[X , Y ] = 16
Cov(X , Y ) = 0
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 96 / 135
Trial

Compute h i h i
E[X ], E[Y ], E X2 , E Y 2
E[XY ], var(X ), var(Y )
for

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 97 / 135
Special Discrete Random Variables

A Bernoulli trial has only two outcomes, with probability

P(X = 1) = p
P(X = 0) = 1 − p

e.g., tossing a coin, winning or losing a game, · · · . We have

E[X ] = 1 · p + 0 · (1 − p) = p
h i
E X 2 = 12 · p + 02 · (1 − p) = p
h i
var(X ) = E X 2 − E[X ]2 = p − p 2 = p(1 − p).

NOTE: If p is small then Var(X ) ∼

= p.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 98 / 135
Examples of Bernoulli Trials

1
- When p = 2 (e.g., for tossing a coin), we have

1 1
E[X ] = p = , Var(X ) = p(1 − p) = .
2 4
- When rolling a die, with outcome k, (1 ≤ k ≤ 6), let X (k) = 1 if the roll
resulted in a six, and X (k) = 0 if the roll did not result in a six. Then

1 5
E[X ] = p = , Var(X ) = p(1 − p) = .
6 36
- When p = 0.01, then

E[X ] = 0.01 , Var(X ) = 0.0099 ∼

= 0.01

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 99 / 135
The Binomial Random Variable

Perform a Bernoulli trial n times in sequence. Assume the individual trials

are independent. Given that an outome is 100011001010 (n = 12), can
we say that;
P(100011001010) = p 5 · (1 − p)7 .
if X be the number of "successes" (i.e. 1’s)?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 100 / 135
The Binomial Random Variable

Perform a Bernoulli trial n times in sequence. Assume the individual trials

are independent. Given that an outome is 100011001010 (n = 12), can
we say that;
P(100011001010) = p 5 · (1 − p)7 .
if X be the number of "successes" (i.e. 1’s)?
Recall that, a Bernoulli random variable has two possible outcomes: 0 or
1. When the experiment is repeated for a finite number of times, we
obtain a binomial distribution which is the sum of independent and
identically distributed Bernoulli random variables.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 100 / 135
Hence, the solution to our problem becomes;

X (100011001010) = 5.
!
12
P(X = 5) = · p 5 · (1 − p)7 .
5
In general, for k successes in a sequence of n trials, we have
!
n
P(X = k) = · p k · (1 − p)n−k , (0 ≤ k ≤ n)
k

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 101 / 135
Assignment

Show that from

!
n
P(X = k) = · p k · (1 − p)n−k ,
k

and !
n
P(X = k + 1) = · p k+1 · (1 − p)n−k−1 ,
k +1
it follows that
P(X = k + 1) = ck · P(X = k),
where
n−k p
ck = · .
k +1 1−p

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 102 / 135
NOTE : This recurrence formula is an efficient and stable algorithm to
compute the binomial probabilities :

P(X = 0) = (1 − p)n
P(X = k + 1) = ck · P(X = k), k = 0, 1, · · · , n − 1

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 103 / 135
Mean and Variance of the Binomial Random Variable

By definition, the mean of a Binomial random variable X is

n n
!
X X n
E[X ] = k · P(X = k) = k· p k (1 − p)n−k ,
k
k=0 k=0

which can be shown to equal np.

An easy way to see this is as follows: If in a sequence of n independent
Bernoulli trials we let Xk = the outcome of the k th Bernoulli trial,
(Xk = 0 or 1), then
X ≡ X1 + X2 + · · · + Xn ,
is the Binomial random variable that counts the “successes".

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 104 / 135
 X ≡ X1 + X2 + · · · + Xn
We know that

E[Xk ] = p (11)

So
E[X ] = E [X1 ] + E [X2 ] + · · · + E [Xn ] = np.
We already know that
h i
Var (Xk ) = E Xk2 − (E [Xk ])2 = p − p 2 = p(1 − p),

so, since the Xk are independent, we have

Var(X ) = Var (X1 ) + Var (X2 ) + · · · + Var (Xn ) = np(1 − p).

NOTE: If p is small then Var(X ) ∼

= np.
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 105 / 135
Examples

For 12 tosses of a coin, with Heads is success, we have so

1
n = 12 , p=
2
E[X ] = np = 6, Var(X ) = np(1 − p) = 3.
- For 12 rolls of a die, with six is success, we have
1
n = 12 , p=
6
so E[X ] = np = 2 , Var(X ) = np(1 − p) = 5/3

- If n = 500 and p = 0.01, then

E[X ] = np = 5 , Var(X ) = np(1 − p) = 4.95 ∼

= 5.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 106 / 135
Poisson Random Variable

The Poisson variable approximates the Binomial random variable :

!
k
P(X = k) =
n ∼ e −λ · λ
· p k · (1 − p)n−k =
k k!

when we take

λ = np (the average number of successes ).

This approximation is accurate if n is large and p small. Recall that for the
Binomial random variable E[X ] = np, and Var(X ) = np(1 − p) ∼ = np when
p is small. Indeed, for the Poisson random variable we will show that

P[X ] = λ and Var(X ) = λ.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 107 / 135
A stable and efficient way to compute the Poisson probability

λk
P(X = k) = e −λ · , k = 0, 1, 2, · · · ,
k!
λk+1
P(X = k + 1) = e −λ · ,
(k + 1)!
is to use the recurrence relation

P(X = 0) = e −λ ,
λ
P(X = k + 1) = · P(X = k), k = 0, 1, 2, · · · .
k +1
NOTE : Unlike the Binomial random variable, the Poisson random variable
can have an arbitrarily large integer value k.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 108 / 135
The Poisson random variable
λk
P(X = k) = e −λ · , k = 0, 1, 2, · · · ,
k!
has (as shown later) : E[X ] = λ and Var(X ) = λ. The Poisson
distribution function is
k k
λℓ λℓ
e −λ = e −λ
X X
F (k) = P(X ≤ k) =
ℓ=0
ℓ! ℓ=0
ℓ!

with, as should be the case,

∞
λℓ
lim F (k) = e −λ = e −λ e λ = 1.
X
k→∞
ℓ=0
ℓ!

( using the Taylor series from Calculus for e λ ).

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 109 / 135
The Poisson random variable
λk
P(X = k) = e −λ · , k = 0, 1, 2, · · · ,
k!
models the probability of k "successes" in a given "time" interval, when
the average number of successes is λ.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 110 / 135
Example
Suppose customers arrive at the rate of six per hour. The probability that
k customers arrive in a one-hour period is

60 ∼
P(k = 0) = e −6 · = 0.0024
0!
6 1
P(k = 1) = e −6 · ∼
= 0.0148
1!
6 2
P(k = 2) = e −6 · ∼
= 0.0446
2!
The probability that more than 2 customers arrive is

1 − (0.0024 + 0.0148 + 0.0446) ∼

= 0.938.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 111 / 135
In Class Exercise

Question 1

Defects in a certain wire occur at the rate of one per 10 meter. Assume
the defects have a Poisson distribution.
What is the probability that:
- a 20-meter wire has no defects?
- a 20-meter wire has at most 2 defects?

Question 2

Customers arrive at a counter at the rate of 8 per hour. Assume the

arrivals have a Poisson distribution.
What is the probability that:
- no customer arrives in 15 minutes?
- two customers arrive in a period of 30 minutes?

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 112 / 135
Group Project

You are required to make a presentation on the following distributions;

1 Geometric
2 Negative Binomial
3 Hypergeometric
4 Normal
5 Lognormal
6 Gamma
7 Beta
8 Weibull

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 113 / 135
Transforms (Moment Generating Function)

The transform provides us with an alternative representation of a

probability law. It is not particularly intuitive, but it is often convenient for
certain types of mathematical manipulations.
The transforms associated to a random variable X is a function MX (x ) of
a scalar parameter s, defined by

MX (s) = E[e sX ]

For discrete random variables

X
MX (s) = e sx pX (x )
x

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 114 / 135
By Taylor expansion of e tX about t = 0, we have
" #
h
tX
i t 2X 2 t 3X 3
ψ(t) ≡ E e = E 1 + tX + + + ···
2! 3!
t 2 h 2i t 3 h 3i
= 1 + tE[X ] + E X + E X + ···
2! 3!
It follows that
h i
ψ ′ (0) = E[X ] , ψ ′′ (0) = E X 2 ( Why ? )

This sometimes facilitates computing the mean and the variance µ = E[X ],
h i
Var(X ) = E X 2 − µ2

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 115 / 135
Application of Transforms
The Poisson mean and variance:
∞ ∞
h i λk
e tk e −λ
X X
tX tk
ψ(t) ≡ E e = e P(X = k) =
k=0 k=0
k!
∞
−λ (λe t )k t t −1)
= e −λ e λe = e λ(e
X
=e .
k=0
k!

Here t −1)
ψ ′ (t) = λe t e λ(e
h
2 i t −1)
ψ ′′ (t) = λ λ e t + e t e λ(e ( Check !)
so that
E[X ] = ψ ′ (0) = λ
h i
E X 2 = ψ ′′ (0) = λ(λ + 1) = λ2 + λ
h i
Var(X ) = E X 2 − E[X ]2 = λ
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 116 / 135
Continuous Random Variable
A random variable X is called continuous if there is a nonnegative function
fX , called the probability density function of X , or PDF for short, such that
Z
P(X ∈ B) = fX (x )dx ,
B

for every subset B of the real line. † In particular, the probability that the
value of X falls within an interval is
Z b
P(a ≤ X ≤ b) = fX (x )dx ,
a

and can be interpreted as the area under the graphR of the PDF (see Fig.
3.1). For any single value a, we have P(X = a) = aa fX (x )dx = 0. For
this reason, including or excluding the endpoints of an interval has no
effect on its probability:

P(a ≤ X ≤ b) = P(a < X < b) = P(a ≤ X < b) = P(a < X ≤ b).

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 117 / 135
C.R.V cont.

Note that to qualify as a PDF, a function fX must be nonnegative, i.e.,

fX (x ) ≥ 0 for every x , and must also have the normalization property
Z ∞
fX (x )dx = P(−∞ < X < ∞) = 1.
−∞

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 118 / 135
Expectation of C.R.V and its Properties
Let X be a continuous random variable with PDF fX . The expectation of
X is defined by Z ∞
E[X ] = xfX (x )dx
−∞
- The expected value rule for a function g(X ) has the form
Z ∞
E[g(X )] = g(x )fX (x )dx
−∞
- The variance of X is defined by
h i Z ∞
2
var(X ) = E (X − E[X ]) = (x − E[X ])2 fX (x )dx
−∞
- We have h i
0 ≤ var(X ) = E X 2 − (E[X ])2 .
- If Y = aX + b, where a and b are given scalars, then
E[Y ] = aE[X ] + b, var(Y ) = a2 var(X )
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 119 / 135
Cumulative Distribution Function

We have been dealing with discrete and continuous random variables in a

somewhat different manner using PMFs and PDFs, respectively. It would
be desirable to describe all kinds of random variables with a single
mathematical concept.
This is accomplished with the cumulative distribution function, or CDF for
short. The CDF of a random variable X is denoted by FX and provides the
probability P(X ≤ x ). In particular, for every x we have
P

 pX (k), if X is discrete
k≤x

FX (x ) = P(X ≤ x ) =


R x

−∞ fX (t)dt, if X is continuous.

Loosely speaking, the CDF FX (x ) "accumulates" probability "up to" the

value x .
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 120 / 135
The (cumulative) probability distribution function is defined as

FX (x ) ≡ P(X ≤ x ).

Thus
FX (b) − FX (a) ≡ P(a < X ≤ b).
We must have
FX (−∞) = 0 and FX (∞) = 1
i.e., and
lim FX (x ) = 0,
x →−∞
lim FX (x ) = 1.
x →∞

Also, FX (x ) is a non-decreasing function of x . (Why ?)

NOTE : All the above is the same as for discrete random variables!

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 121 / 135
In general, from with
f (x ) ≡ F ′ (x ),
F (−∞) = 0 and F (∞) = 1,
we have from Calculus the following basic identities:
Z ∞ Z ∞
f (x )dx = F ′ (x )dx = F (∞) − F (−∞) = 1
−∞ −∞
Z x
f (x )dx = F (x ) − F (−∞) = F (x ) = P(X ≤ x )
−∞
Z b
f (x )dx = F (b) − F (a) = P(a < X ≤ b)
a
Z a
f (x )dx = F (a) − F (a) = 0 = P(X = a)
a

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 122 / 135
Continuous Random Variable
The probability density function is the derivative of the probability
distribution function :
d
fX (x ) ≡ FX′ (x ) ≡ FX (x ).
dx

Examples
In the "pointer example"
 
x ≤0
0,

 0,

 x ≤0
′
FX (x ) = x , 0 < x ≤ 1 and fX (x ) = FX (x ) = 1, 0<x ≤1

 

1, 1 < x 0, 1<x

NOTATION : When it is clear what X is then we also write f (x ) for fX (x ),

and F (x ) for FX (x ).
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 123 / 135
In Class Exercise 1

Given that
( (
0, x ≤0 0, x ≤0
F (x ) = , f (x ) = ,
1 − e −x , x > 0 e −x , x > 0

verify that
P(0 < X ≤ 1) = F (1) − F (0) = F (1) = 1 − e −1 ∼
= 0.63,
P(X > 1) = 1 − F (1) = e −1 ∼
= 0.37,
P(1 < X ≤ 2) = F (2) − F (1) = e −1 − e −2 ∼
= 0.23.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 124 / 135
In Class Exercise 2
For positive integer n, consider the density functions
(
cx n (1 − x n ) , 0 ≤ x ≤ 1
fn (x ) =
0, otherwise

Determine the value of c in terms of n.

Draw the graph of fn (x ) for n = 1, 2, 4, 8, 16.
Determine the distribution function Fn (x ).
Draw the graph of Fn (x ) for n = 1, 2, 3, 4, 8, 16.
 
1
Determine P 0 ≤ X ≤ 2 in terms of n.
 
1
What happens to P 0 ≤ X ≤ 2 when n becomes large?
 
9
Determine P 10 ≤ X ≤ 1 in terms of n.
 
9
What happens to P 10 ≤ X ≤ 1 when n becomes large?
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 125 / 135
Joint Distribution

A joint probability density function fX ,Y (x , y ) must satisfy

Z ∞ Z ∞
fX ,Y (x , y )dxdy = 1 ( "Volume" = 1 ).
−∞ −∞

The corresponding joint probability distribution function is

Z y Z x
FX ,Y (x , y ) = P(X ≤ x , Y ≤ y ) = fX ,Y (x , y )dxdy .
−∞ −∞

∂ 2 FX ,Y (x ,y )
By Calculus we have ∂x ∂y = fX ,Y (x , y ). Also,
Z dZ b
P(a < X ≤ b, c < Y ≤ d) = fX ,Y (x , y )dxdy
c a

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 126 / 135
Joint Distribution
Example
If (
1 for x ∈ (0, 1] and y ∈ (0, 1],
fX ,Y (x , y ) =
0 otherwise
then, for x ∈ (0, 1] and y ∈ (0, 1],
Z yZ x
FX ,Y (x , y ) = P(X ≤ x , Y ≤ y ) = 1dxdy = xy .
0 0

Thus
FX ,Y (x , y ) = xy , for x ∈ (0, 1] and y ∈ (0, 1].

For example
1 1 1 1 1
   
P X ≤ ,Y ≤ = FX ,Y , = .
3 2 3 2 6
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 127 / 135
Joint Distribution

EXERCISE : (
(1 − e −x ) (1 − e −y ) for x ≥ 0
Let FX ,Y (x , y ) = and y ≥ 0,, -
0 otherwise
Verify that
(
∂2F e −x −y for x ≥ 0
fX ,Y (x , y ) = = and y ≥ 0
∂x ∂y 0 otherwise

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 128 / 135
Joint Distribution

FX ,Y (x , y ) = 1 − e −x 1 − e −y fX ,Y (x , y ) = e −x −y ,



, for x , y ≥ 0

Also verify the following :

F (0, 0) = 0 , F (∞, ∞) = 1,
R∞R∞
0 0 fX ,Y (x , y )dxdy = 1, (Why zero lower limits ?)
fX (x ) = 0∞ e −x −y dy = e −x ,
R

fY (y ) = 0∞ e −x −y dx = e −y .
R

fX ,Y (x , y ) = fX (x ) · fY (y ).

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 129 / 135
Joint Distribution

 
P(1 < x < ∞) = FX (∞) − FX (1) = 1 − 1 − e −1 = e −1 ∼
= 0.37
Z 1Z 2
P(1 < x ≤ 2, 0 < y ≤ 1) = e −x −y dxdy
0 1
  
= e −1
− e −2 1 − e −1 ∼
= 0.15

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 130 / 135
Independent Continuous Random Variable
Recall that two events E and F are independent if

P(EF ) = P(E )P(F ).

Continuous random variables X (s) and Y (s) are independent if

P (X ∈ IX , Y ∈ IY ) = P (X ∈ IX ) · P (Y ∈ IY ) ,

for all allowable sets IX and IY (typically intervals) of real numbers.

Equivalently, X (s) and Y (s) are independent if for all such sets IX and IY
the events
X −1 (IX ) and Y −1 (IY ) ,
are independent in the sample space S.

X −1 (IX ) ≡ {s ∈ S : X (s) ∈ IX }
NOTE :
Y −1 (IY ) ≡ {s ∈ S : Y (s) ∈ IY }

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 131 / 135
FACT : X (s) and Y (s) are independent if for all x and y
fX ,Y (x , y ) = fX (x ) · fY (y ).
Example:The random variables with density function
(
e −x −y for x ≥ 0
fX ,Y (x , y ) = and y ≥ 0,
0 otherwise ,

are independent because (by the preceding exercise)

fX ,Y (x , y ) = e −x −y = e −x · e −y = fX (x ) · fY (y ).
Note :
(
(1 − e −x ) (1 − e −y ) for x ≥ 0
FX ,Y (x , y ) = and y ≥ 0
0 otherwise

also satisfies (by the preceding exercise)

FX ,Y (x , y ) = FX (x ) · FY (y ).
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 132 / 135
Independent C.R.V
PROPERTY :
For independent continuous random variables X and Y we have
FX ,Y (x , y ) = FX (x ) · FY (y ), for all x , y .
Proof.
FX ,Y (x , y ) = P(X ≤ x , Y ≤ y )
Z x Z y
= fX ,Y (x , y )dydx
−∞ −∞
Z x Z y
= fX (x ) · fY (y )dydx (by independence)
−∞ −∞
Z x  Z y 
= fX (x ) · fY (y )dy dx
−∞ −∞
Z x  Z y 
= fX (x )dx · fY (y )dy
−∞ −∞
= FX (x ) · FY (y ).
Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 133 / 135
Independent C.R.V

Example
For the pointer experiment

0,

 x ≤0
fX (x ) = 1, 0<x ≤1


0, 1<x

we have
1
x2
Z ∞ Z 1
1
E [X ] = xfX (x )dx = xdx = =
−∞ 0 2 0
2
and
1
x3
Z ∞ Z 1
h
2
i
2 2 1
E X = x fX (x )dx = x dx = = .
−∞ 0 3 0
3

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 134 / 135
Joint Distribution

EXERCISE : Prove the following for continuous random variables:

E [aX ] = aE [X ],
E [aX + b] = aE [X ] + b,
E [X + Y ] = E [X ] + E [Y ],
and compare the proofs to those for discrete random variables.

Example A stick of length 1 is split at a randomly selected point X . (

Thus X is uniformly distributed in the interval [0, 1].) Determine the
expected length of the piece containing the point 1/3.

Ignatius Nyendah Probability, Statistics and Reliability (with MATLAB) March 14, 2024 135 / 135