Chapter Three

Elementary Probability
Deterministic and non-deterministic models
A model is an approximate representation of a physical situation. It attempts to explain
observed behavior using a set of simple and understandable rules. These rules can be used to
predict the outcome of experiments involving the given physical situation. A useful model
explains all relevant aspects of a given situation. Such models can be used instead of
experiments to answer questions regarding the given situation. Models therefore allow the
engineer to avoid the costs of experimentation, namely, labor, equipment, and time.
The two basic types of models are deterministic models and non-deterministic models
(probability models)
Deterministic models used when the observational phenomenon has measurable properties.
In this type of model conditions under which an experiment is carried out determine the exact
outcome of the experiment. The solution of a set of mathematical equations specifies the
exact outcome of the experiment. As a simple example, suppose we are measuring the flow
of current in a thin copper wire, our model for this phenomena might be Ohm’s low that:
Current =Voltage/Resistance
Or

We call this deterministic model because it assumes that the interaction between these
idealized components is completely described by this formula or laws.
If an experiment involving the measurement of a set of voltages is repeated a number of times
under the same conditions, this theory predicts that the observations will always be exactly
the same.
However, if we performed this measurement process more than once, perhaps at different
times, or even on different days, the observed current could differ slightly because of small
changes or variations in factors that are not completely controlled, such as changes in
ambient temperature, fluctuations in performance of the gauge, small impurities present at
different locations in the wire and drifts in the voltage source. Consequently, a more realistic
model of the observed current might be

Where is a term added to the model to account for the fact that the observed values of
current flow do not perfectly conform to the deterministic model .We can think of as a
term that includes the effects of all of the unmodeled sources of variability that affect this
system.

Review of set theory: sets, union, intersection, complementation, De Morgan’s rules

The term set refers to a well-defined collection of objects that share a certain property or
certain properties. The term “well-defined” here means that the set is described in such a way
that one can decide whether or not a given object belongs in the set. If is a set, then the
objects of the collection are called the elements or members of the set . If is an element
of the set , we write . If is not an element of the set , we write .
As a convention, we use capital letters to denote the names of sets and lowercase letters for
elements of a set.
Note that for each objects and each set , exactly one of or but not both must
be true.
Example a.The set of counting numbers less than ten.
b.The set of letters in the word “Addis Ababa.”
c. The set of all countries in Africa.
Union
The union of two events is the event that consists of all outcomes that are contained in either
of the two events. We denote the union as:

Example: Consider an experiment in which you select a molded plastic part, such as a
connector, and measure its thickness. The possible values for thickness depend on the
resolution of the measuring instrument, and they also depend on upper and lower bounds for
thickness. If the objective of the analysis is to consider only whether or not the parts conform
to the manufacturing specifications, either part may or may not conform. We abbreviate yes
and no as y and n. If the ordered pair yn indicates that the first connector conforms and the
second does not, the set of all outcomes can be represented by the four outcomes:

Suppose that the set of all outcomes for which at least one part conforms is denoted as E1.
Then,

. The event, in which both parts do not conform, denoted as , contains only
the single outcome, * + Other examples of events are * + , the null set, and
. If * +
Intersection
The intersection of two events is the event that consists of all outcomes that are contained in
both of the two events. We denote the intersection as: .
* +.

Complementation
The complement of an event in a sample space is the set of outcomes in the sample space that
are not in the event. We denote the complementation of the event .
* +

De Morgan’s rules
De Morgan’s laws imply that

Example: ( ) * + * +

Random experiments, sample space and events

Random Experiments
An experiment that can result in different outcomes, even though it is repeated in the same
manner every time, is called a random experiment.
Example: If we measure the current in a thin copper wire, we are conducting an experiment.
However, in day-to-day repetitions of the measurement the results can differ slightly because
of small variations in variables that are not controlled in our experiment, including changes in
ambient temperatures, slight variations in gauge and small impurities in the chemical
composition of the wire if different locations are selected, and current source drifts.

However, no matter how carefully our experiment is designed and conducted, the variation is
almost always present, and its magnitude can be large enough that the important conclusions
from our experiment are not obvious.

Sample space and events

To model and analyze a random experiment, we must understand the set of possible
outcomes from the experiment.
The set of all possible outcomes of a random experiment is called the sample space of the
experiment. The sample space is denoted as S.

Example: Consider an experiment in which you select a molded plastic part, such as a
connector, and measure its thickness. The possible values for thickness depend on the
resolution of the measuring instrument, and they also depend on upper and lower bounds for
thickness. However, it might be convenient to define the sample space as simply the positive

real line

because a negative value for thickness cannot occur. If it is known that all connectors will be
between 10 and 11 millimeters thick, the sample space could be

If the objective of the analysis is to consider only whether a particular part is low, medium, or
high for thickness, the sample space might be taken to be the set of three outcomes:

If the objective of the analysis is to consider only whether or not a particular part conforms to
the manufacturing specifications, the sample space might be simplified to the set of two

outcomes

If the objective of the analysis is to consider only whether or not the parts conform to the
manufacturing specifications, either part may or may not conform. We abbreviate yes and no
as y and n. If the ordered pair yn indicates that the first connector conforms and the second
does not, the sample space can be represented by the four outcomes:

In random experiments in which items are selected from a batch, we will indicate whether or
not a selected item is replaced before the next one is selected. For example, if the batch
consists of three items {a, b, c} and our experiment is to select two items without

replacement, the sample space can be represented as

If items are replaced before the next one is selected, the sampling is referred to as with

replacement. Then the possible ordered outcomes are

An event is a subset of the sample space of a random experiment.

Consider the sample space S= {yy, yn, ny,nn} in the above Example. Suppose that the set of
all outcomes for which at least one part conforms is denoted as E1. Then,
 Finite sample spaces and equally likely outcomes
Finite sample spaces
Sample space can be
 Countable ( finite or infinite)
 Uncountable.
Example: .What is the sample space for the following experiment
a) Toss a die one time.
b) Toss a coin two times.
c) A light bulb is manufactured. It is tested for its life length by time.
Solution
a) S={1,2,3,4,5,6} (finite)
b) S={(HH),(HT),(TH),(TT)} (finite)
c) S={t /t≥0} (Uncountable)
Equally Likely Events: Events which have the same chance of occurring.

Example: In the experiment of tossing a die one time the events 1, 2, 3, 4, 5, 6 have
the same chance of occurring, so they are equally likely events.
Counting techniques
As sample spaces become larger, complete enumeration is difficult. Instead, counts of the
number outcomes in the sample space and in various events are often used to analyze the
random experiment. These methods are referred to as counting techniques.
In order to determine the number of outcomes, one can use several rules of counting.
The multiplication rule, Permutation rule, Combination rule

The Multiplication Rule: If a choice consists of k steps of which the first can be made in n1
ways, the second can be made in n2 ways…, the kth can be made in nk ways, then the whole

choice can be made in (n1 * n2 * ........* nk ) ways.

Example: The digits 0, 1, 2, 3, and 4 are to be used in 4 digit identification card. How many
different cards are possible if
A).Repetitions are permitted. B).Repetitions are not permitted.
Solutions
a)
1st digit 2nd digit 3rd digit 4th digit
5 5 5 5
 There are four steps
1. Selecting the 1st digit, this can be made in 5 ways.
2. Selecting the 2nd digit, this can be made in 5 ways.
3. Selecting the 3rd digit, this can be made in 5 ways.
4. Selecting the 4th digit, this can be made in 5 ways.

 5 * 5 * 5 * 5  625 different cards are possible.

b)
1st digit 2nd digit 3rd digit 4th digit
5 4 3 2
There are four steps
1. Selecting the 1st digit, this can be made in 5 ways.
2. Selecting the 2nd digit, this can be made in 4 ways.
3. Selecting the 3rd digit, this can be made in 3 ways.
4. Selecting the 4th digit, this can be made in 2 ways.
 5*4*3*2=120 different cards are possible
Permutation: An arrangement of n objects in a specified order is called permutation of the
objects.
Permutation Rules:
1. The number of permutations of n distinct objects taken all together is n!
Where n! n * (n  1) * (n  2) * .....* 3 * 2 *1
2. The arrangement of n objects in a specified order using r objects at a time is called the

permutation of n objects taken r objects at a time. It is written as n Pr and the

n!
formula is n Pr 
( n  r )!
3. The number of permutations of n objects in which k1 are alike k2 are alike ---- etc is
n!
n Pr 
k1!*k 2 * ...* k n
Example:
1. Suppose we have a letters A,B, C, D
a) How many permutations are there taking all the four?
b) How many permutations are there two letters at a time?
2. How many different permutations can be made from the letters in the word
“CORRECTION”?
 Solutions:
1.
a)
Here n  4, there are four disnict object
 There are 4! 24 permutations.
Here n  4, r2
b) 4! 24
 There are 4 P2    12 permutations.
(4  2)! 2
2.
Here n  10
Of which 2 are C , 2 are O, 2 are R ,1E ,1T ,1I ,1N
 K1  2, k 2  2, k3  2, k 4  k5  k 6  k 7  1
U sin g the 3rd rule of permutatio n , there are
10!
 453600 permutatio ns.
2!*2!*2!*1!*1!*1!*1!
Combination
A selection of objects with-out regard to order is called combination.
Example: Given the letters A, B, C, and D list the permutation and combination for selecting two
letters.
Solutions:
Permutation Combination
AB BC
AB BA CA DA
AC BD
AC BC CB DB
AD DC
AD BD CD DC

Note that in permutation AB is different from BA. But in combination AB is the same as BA.
Combination Rule

The number of combinations of r objects selected from n objects is denoted by  n

n Cr or 
r
 
and is given by the formula:

 n n!

r  (n  r )!*r!
 
Examples:
1. In how many ways a committee of 5 people be chosen out of 9 people?
Solutions:
n9 , r 5
n n! 9!

r
  ( n  r )!*r!  4!*5!  126 ways
 
2. Among 15 clocks there are two defectives .In how many ways can an inspector chose
three of the clocks for inspection so that:
a) There is no restriction.
b) None of the defective clock is included.
c) Only one of the defective clocks is included.
d) Two of the defective clock is included.
Solutions:
n  15 of which 2 are defective and 13 are non  defective.
r 3

a) If there is no restriction select three clocks from 15 clocks and this can be done in
:
n  15 , r 3
n n! 15!

r
  ( n  r )!*r!  12!*3!  455 ways
 
b) None of the defective clocks is included.
This is equivalent to zero defective and three non-defectives, which can be done in:
 2  13 

 0 *
   286 ways.
  3
c) Only one of the defective clocks is included.
This is equivalent to one defective and two non-defectives, which can be done in:
 2  13 

1  *
   156 ways.
   2
d) Two of the defective clock is included.
This is equivalent to two defective and one non defective, which can be done in:
 2  13 

 2 *
   13 ways.
   3
Definitions of probability
Probability is used to quantify the likelihood, or chance, that an outcome of a random
experiment will occur. A probability 0 indicates an outcome will not occur. A probability of 1
indicates an outcome will occur with certainty.
Subjective probability definition is defining the probability based on degree of belief that
the outcome will occur. Different individuals will no doubt assign different probabilities to
the same outcomes.

The classical approach

This approach is used when:
- All outcomes are equally likely.
- Total number of outcome is finite, say N.
Definition: If a random experiment with N equally likely outcomes is conducted and out of
these NA outcomes are favourable to the event A, then the probability that event A occur

denoted P(A) is defined as:

N A No. of outcomes favourableto A n( A)
P( A)   
N Total numberof outcomes n( S )
Examples:
1. A fair die is tossed once. What is the probability of getting
A).Number 4? B) An odd number? C).An even number? D).Number 8?
Solutions: First identify the sample space, say S
S  1, 2, 3, 4, 5, 6
 N  n( S )  6
a) Let A be the event of number 4
A  4
 N A  n( A)  1
n( A)
P ( A)  1 6
n( S )
b) Let A be the event of odd numbers
A  1,3,5
 N A  n( A)  3
n( A)
P ( A)   3 6  0.5
n( S )
c) Let A be the event of even numbers
 A  2,4,6
 N A  n( A)  3
n( A)
P( A)   3 6  0.5
n( S )
d) Let A be the event of number 8
A Ø

 N A  n( A)  0
n( A)
P ( A)  0 60
n( S )
2. A box of 80 candles consists of 30 defective and 50 non defective candles. If 10 of
this candles are selected at random, what is the probability
A). All will be defective. B). 6 will be non-defective C). All will be non-
defective
Solutions:
 80 
Total selection  
 10 
  N  n( S )
 
a) Let A be the event that all will be defective.
 30   50 
Total way in which A occur    10 
*
 
  N A  n( A)
   0 
 30   50 

 10 
* 0 

 P ( A) 
n( A)
      0.00001825
n( S )  80 

 10 

 
b) Let A be the event that 6 will be non-defective.
 30   50 
Total way in which A occur    4 *
 
  N A  n( A)
   6 
 30   50 

 4 * 6 

 P ( A) 
n( A)
      0.265
n( S )  80 

 10 

 
c) Let A be the event that all will be non-defective.
  30   50 
Total way in which A occur    0 *
 
  N A  n( A)
   10 
 30   50 

 0 * 10 

 P ( A) 
n( A)
      0.00624
n( S )  80 

 10 

 
Exercises:
1. What is the probability that a waitress will refuse to serve alcoholic beverages to only
three minors if she randomly checks the I.D’s of five students from among ten
students of which four are not of legal age?
Short coming of the classical approach:
This approach is not applicable when:
- The total number of outcomes is infinite.
- Outcomes are not equally likely.
The Frequentist Approach
This is based on the relative frequencies of outcomes belonging to an event.
Definition: The probability of an event A is the proportion of outcomes favourable to A in the
long run when the experiment is repeated under same condition.
NA
P ( A)  lim
N  N
Example: If records show that 60 out of 100,000 bulbs produced are defective. What
is the probability of a newly produced bulb to be defective?
Solution: Let A be the event that the newly produced bulb is defective.
NA 60
P ( A)  lim   0.0006
N  N 100,000

Axiomatic Approach:
Let E be a random experiment and S be a sample space associated with E. With each event A a
real number called the probability of A satisfies the following properties called axioms of
probability or postulates of probability.

1. P( A)  0
2. P( S )  1, S is the sure event.
3. If A and B are mutually exclusive events, the probability that one or the other occur
equals the sum of the two probabilities. i. e.
P( A  B)  P( A)  P( B)
4. P( A' )  1  P( A)
5. 0  P( A)  1
6. P(ø) =0, ø is the impossible event.

In general p( A  B)  p( A)  p( B)  p( A  B)

Chapter Four
Conditional Probability and Independence
Conditional Probability
The conditional probability of an event B given an event A, denoted as ( ⁄ ), is

( )
( ⁄ ) ( ) .
( )

This definition can be understood in a special case in which all outcomes of a random
experiment are equally likely. If there are n total outcomes,

Example1: A day’s production of 850 manufactured parts contains 50 parts that do not meet
customer requirements. Two parts are selected randomly without replacement from the batch.
What is the probability that the second part is defective given that the first part is defective?
Solution: Let A denote the event that the first part selected is defective, and let B denote the
event that the second part selected is defective. The probability needed can be expressed
as ( ⁄ ). If the first part is defective, prior to selecting the second part, the batch contains
849 parts, of which 49 are defective, therefore
 ( ⁄ ) ⁄
Example2: Continuing the previous example, if three parts are selected at random, what is
the probability that the first two are defective and the third is not defective? This event can be
described in shorthand notation as simply P(ddn). We have

( )

The third term is obtained as follows. After the first two parts are selected, there are 848
remaining. Of the remaining parts, 800 are not defective. In this example, it is easy to obtain
the solution with a conditional probability for each selection.

Multiplication theorem, Total probability theorem and Bayes’ Theorem

Multiplication Rule
( ) ( ⁄ ) ( ) ( ⁄ ) ( )
The multiplication rule is useful for determining the probability of an event that depends on
other events.
Example: The probability that an automobile battery subject to high engine compartment
temperature suffers low charging current is 0.7. The probability that a battery is subject to
high engine compartment temperature is 0.05. Let C denote the event that a battery suffers
low charging current, and let T denote the event that a battery is subject to high engine
compartment temperature. The probability that a battery is subject to low charging current
and high engine compartment temperature is

( ) ( ⁄ ) ( )

Total probability theorem

For any events A and B,

( ) ( ) ( ) ( ⁄ ) ( ) ( ⁄ ) ( )

Example: suppose that in semiconductor manufacturing the probability is 0.10 that a chip
that is subjected to high levels of contamination during manufacturing causes a product
failure. The probability is 0.005 that a chip that is not subjected to high contamination levels
during manufacturing causes a product failure. In a particular production run, 20% of the
chips are subject to high levels of contamination. What is the probability that a product using
one of these chips fails?
Let F denote the event that the product fails, and let H denote the event that the chip is
exposed to high levels of contamination. The requested probability is P(F), and the
information provided can be represented as

which can be interpreted as just the weighted average of the two probabilities of failure.

The reasoning used to develop the above equation can be applied more generally. In the
development of the above equation we only used the two mutually exclusive A and .
However the fact that =S, the entire sample space, was important.

In general a collection of sets such that said to

be exhaustive.
Assume that are k mutually exclusive and exhaustive sets. Then
( ) ( ) ( ) ( )
( ⁄ ) ( ) ( ⁄ ) ( ) ( ⁄ ) ( )
Example: In a particular production run, 20% of the chips are subjected to high levels of
contamination, 30% to medium levels of contamination, and 50% to low levels of
contamination. What is the probability that a product using one of these chips fails?

Let H denote the event that a chip is exposed to high levels of contamination
M denote the event that a chip is exposed to medium levels of contamination
L denote the event that a chip is exposed to low levels of contamination Then,
( ) ( ⁄ ) ( ) ( ⁄ ) ( ) ( ⁄ ) ( )
=0.10(0.20)+0.01(0.30)+0.001(0.50)=0.0235
Bayes’ Theorem
In some examples, we do not have a complete table of information. We might know one
conditional probability but would like to calculate a different one. Consider the following that
Continuing with the semiconductor manufacturing example, assume the following
probabilities for product failure subject to levels of contamination in manufacturing:
In this problem, we might ask the following: If the semiconductor chip in the product fails,
what is the probability that the chip was exposed to high levels of contamination? From the
definition of conditional probability,
( ) ( ⁄ ) ( ) ( ) ( ⁄ ) ( )
Now considering the second and last terms in the expression above, we can write
( ⁄ ) ( )
( ⁄ ) ( )
( )
This is a useful result that enables us to solve for in terms of ( ⁄ ) in terms of ( ⁄ )
In general, if ( ) in the denominator of Equation 2-11 is written using the Total Probability
Rule in Equation 2-8, we obtain the following general result, which is known as
Bayes’Theorem.
If are k mutually exclusive and exhaustive events and B is any event,

( ⁄ ) ( )
( ⁄ ) ( )
( ) ( ) ( ) ( ) ( ) ( )

Example: We can answer the question posed at the start of this section as follows: The
probability requested can be expressed as ( ⁄ ). Then,

( ⁄ ) ( ) ( )
( ⁄ )
( )

Independent Events
In some cases, the conditional probability of might equal P(B). In this special case,
knowledge that the outcome of the experiment is in event A does not affect the probability
that the outcome is in event B.

Two events are independent if any one of the following equivalent statements is true:

1. ( ⁄ ) ( )
2. ( ⁄ ) ( )
3. ( ) ( ) ( )
Example: A day’s production of 850 manufactured parts contains 50 parts that do not meet
customer requirements. Two parts are selected at random, without replacement, from the
batch.
 a) Find the probability that the second part selected is defective
b) Find the probability that the second part selected is defective given that the first part is
defective.
Let A denote the event that the first part is defective, and let B denote the event that the
second part is defective. Indeed ( ⁄ ) and the ( )

When considering three or more events, we can extend the definition of independence with
the following general result.
The events are independent if and only if for any subset of these events
,
( ) ( ) ( ) ( )
Example: The following circuit operates only if there is a path of functional devices from left
to right. The probability that each device functions is shown on the graph. Assume that
devices fail independently. What is the probability that the circuit operates?

Let T and B denote the events that the top and bottom devices operate, respectively. There is
a path if at least one device operates. The probability that the circuit operates is
( ) ,( )- ( )
a simple formula for the solution can be derived from the complements and From the
independence assumption,
( ) ( ) ( ) ( )

( )
 Chapter Five
One-dimensional Random Variables
Random variable: definition and distribution function
Definition
A random variable is a function that assigns a real number to each outcome in the sample
space of a random experiment. A random variable is denoted by an uppercase letter such as
X. After an experiment is conducted, the measured value of the random variable is denoted
by a lowercase letter such as x=70 mill amperes.

Discrete random variables

A discrete random variable is a random variable with a finite (or countably infinite) range.
In other experiments, we might record a count such as the number of transmitted bits that are
received in error. Then the measurement is limited to integers. Or we might record that a
proportion such as 0.0042 of the 10,000 transmitted bits were received in error. Then the
measurement is fractional, but it is still limited to discrete points on the real line. Whenever
the measurement is limited to discrete points on the real line, the random variable is said to be
a discrete random variable.
Examples of discrete random variables: number of scratches on a surface, proportion of
defective parts among 1000 tested, number of transmitted bits received in error.

Continuous random variables

A continuous random variable is a random variable with an interval (either finite or infinite)
of real numbers for its range. The range of the random variable includes all values in an
interval of real numbers; that is, the range can be thought of as a continuum.

Sometimes a measurement (such as current in a copper wire or length of a machined part) can
assume any value in an interval of real numbers (at least theoretically). Then arbitrary
precision in the measurement is possible. Of course, in practice, we might round off to the
nearest tenth or hundredth of a unit. The random variable that represents this measurement is
said to be a continuous random variable.

Examples of continuous random variables: electrical current, length, pressure, temperature,

time, voltage, weight
Cumulative distribution function and its properties
The cumulative distribution function of X is denoted as F(x) and given by:

( ) ( ) ∑ ( ), for discrete random variable X

( ) ( ) ∫ ( ), for continuous random variable X

For a random variable X, ( ) satisfies the following properties.

1. ( ) ( ) ∑ ( ), for discrete random variable X

2. ( ) ( ) ∫ ( ), for continuous random variable X

3. ( )
4. If , then ( ) ( )