Introduction to Probability Theory

Chapter Outline
Basic Probability

❖ Definitions and basic rules of probability

❖ Conditional Probability, Independence and Bayes Theorem

We start our investigation of probability with some basic concepts and definitions.
We live in a world full of uncertainty and describe this in many different ways. ⮚ I
should be back by 10 pm
⮚ He is sure to be in the team
⮚ Sir will not take class tomorrow
Each of these phrases expresses a degree of uncertainty. To deal with uncertainty
mathematically you need to be able to express it as number. We can do this by using a scale
from 0 to 1 to describe the chance that something will happen. On this scale 0 represents a
situation which is impossible and 1 represents something that is certain to happen. This number
is called the probability that something will happen.
So, if P is a probability then0 ≤ P ≤ 1.

In probability we are always concerned with experiments in which the outcome is not
determined in advance.
For example:
Θ Rolling a 6 sided die;
Θ Dealing a hand of 5 cards from a deck of 52;
Θ Measuring the time to failure of a mechanical component.
Θ Measuring rainfall on a given day.
Θ Measuring CO2 emissions from a car or a factory.
Θ Measuring how many people use a rail link between 7am and 9am on a given day.

Some Basic Definitions

Random Experiment

Example
Tossing a coin, throwing dice etc. are all examples of random experiments.
 1
Introduction to Probability Theory

Sample Space

Example
Θ If we roll an unbiased (fair) 6-sided die, then the sample space is �� = {1,2,3,4,5,6}.
Θ If we toss a fair coin twice, and denote heads by H and tails by T then the sample space is
�� = (����, ����, ����, ����)

In all of these examples, the sample space is finite. In many examples, sample spaces are not
finite.
Θ If we toss a fair coin until heads comes up, then the sample space is
�� = {��, ����, ������, ��������, … … }.
This time the sample space is infinite. However, the sample space is still countable or
discrete.
Θ If we are measuring the emissions from cars in an NCT center and are interested in how
many cars we need to test before we find one that fails, then our sample space is again
countably infinite.
Θ If we are measuring the actual emissions in ��/���� then our sample space would be
continuous.
Note that the definition of a sample space depends on what we are interested in measuring. We
may only be interested in whether or not a car fails an emission test in which case the sample
space would be {��������, ��������}.
So, based on above examples

Discrete and Continuous Sample Space

Event
 2
Introduction to Probability Theory

Before defining some basic laws on how to combine events, we need to review some elementary
notions from set theory.

Events can be combined in a variety of ways corresponding to standard operations on sets. The
three basic operations are union, intersection and complement. If �� ������ �� are two
events, then

Θ Union �� ∪ �� is the event consisting of all outcomes that are either in E or F (or both).
Θ Intersection �� ∩ �� is the event consisting of all outcomes that are in both E and F. Θ
Complement ���� ���� ��′ is the event consisting of all outcomes not belonging to E.

Venn diagrams are a useful way to think about events and combinations of events:

Θ �� ∪ �� is the entire coloured region;

Θ �� ∩ �� is the purple shaded region;
Θ (�� ∪ ��)��is the unshaded region.
Mutually Exclusive Events
Two events �� ������ �� are said to be mutually exclusive if it is not possible that both
of them occur at the same time ��.��.�� ∩ �� = ��.

For example, consider the toss of a coin. Let �� be the event that the coin lands on heads and
�� be the event that the coin lands on tails. It follows that, in a single fair coin toss, events ��
������ �� are mutually exclusive.

3
Introduction to Probability Theory

Mutual exclusivity can be shown on a Venn diagram.

Θ �� ������ �� are mutually exclusive

Θ �� ������ �� are not mutually exclusive
Θ �� ������ �� are not mutually exclusive
Another word for mutually exclusive is disjoint.

Independent & Dependent Events

Two or more events are said to be independent if the occurrence of an event does not affect the
occurrence of the other. Otherwise, the events are said to be dependent.
For example,
Θ Suppose that we are tossing a coin twice. Let �� be the event that the first coin toss
lands on heads. In addition, let �� be the event that the second coin toss lands on
heads. Clearly the result of the second coin toss is no way affected by the result of the
first coin toss.
Θ The event of drawing a card twice out of 52 cards without replacement. In the first case,
we draw one card out of 52 cards and in the second case, we draw one card out of 51
cards. In this case, the second drawing is dependent on the first drawing. But, if the card
is replaced before the drawing of the second card, then the total cards remain the same
as 52. In that case the events are independent.

Tree Diagram
Sample spaces can also be described graphically with tree diagrams. Tree Diagrams, like Venn
Diagrams, provide a simple visual way of representing events and their associated probabilities.
They can be used to calculate more complicated probabilities. When a sample space can be
constructed in several steps or stages, we can represent each of the ��1 ways of completing the
first step as a branch of a tree. Each of the ways of completing the second step can be
represented as ��2 branches starting from the ends of the original branches, and so forth.
Example
An automobile manufacturer provides vehicles equipped with selected options. Each vehicle is
ordered
⮚ With or without an automatic transmission
⮚ With or without air conditioning
⮚ With one of three choices of a stereo system
⮚ With one of four exterior colors
If the sample space consists of the set of all possible vehicle types, what is the number of
outcomes in the sample space?

4
Introduction to Probability Theory

The sample space contains 48 outcomes. The tree diagram for the different types of vehicles is
displayed in Fig. 1

Fig. 1: Tree diagram for different types of vehicles with 48 outcomes in the sample space.
Example:
Automobile Configurations
Consider an extension of the automobile manufacturer illustration in the previous example in
which another vehicle option is the interior color. There are four choices of interior color: red,
black, blue, or brown. However,

⮚ With a red exterior, only a black or red interior can be chosen.

⮚ With a white exterior, any interior color can be chosen.
⮚ With a blue exterior, only a black, red, or blue interior can be chosen.
⮚ With a brown exterior, only a brown interior can be chosen.

In Fig. 1, there are 12 vehicle types with each exterior color, but the number of interior color
choices depends on the exterior color. As shown in Fig. 2, the tree diagram can be extended to
show that there are 120 different vehicle types in the sample space.
 Fig. 2: Tree diagram for different types of vehicles with interior colors.

5
Introduction to Probability Theory

Independent Events in Tree Diagrams

Consider the very simple case of flipping a coin twice – each toss is a separate event. If the
coin is fair, the probability of tossing heads and tails is clearly 0.5. In the second toss, these
probabilities still hold.

In this example, all sets of branches have identical probabilities. This is typical of a tree
diagram where the events are independent.

Classical Probability

If an experiment can result in any one of �� different equally likely outcomes, and if
exactly �� of these ooutcomes correspond to event ��, then the probability of event
��
�� is ��(��) = ��.

Essentially, in this case the probability of an event is the ratio of favorable outcomes to
������������ ����
all possible outcomes. i.e. ��(��) =
 ���������������� ������������������ ���� ���������� ��

���������� ������������ ���� ����������������

6
Introduction to Probability Theory

More Laws of Probability

The Additive Rule

If �� ������ �� are two events, then

��(�� ∪ ��) = ��(��) + ��(��) − ��(�� ∩ ��).

If �� ������ �� are two mutually exclusive events, then

��(�� ∪ ��) = ��(��) + ��(��).

If ��1,��2, … … , ���� are mutually exclusive events, then

��(��1 ∪ ��2 ∪ ⋯ ∪ ����) = ��(��1) + ��(��2) + ⋯ + ��(����).

For three events ��, �� ������ ��,

��(�� ∪ �� ∪ ��) = ��(��) + ��(��) + ��(��)
−��(�� ∩ ��) − ��(�� ∩ ��) − ��(�� ∩ ��) + ��(�� ∩ �� ∩ ��).

If �� ������ ��′ are complementary events, then ��(��) + ��(��′) = 1.

Conditional Probability
Conditional probability is a probability of an event under condition that some other (not
impossible) event has happened.

The conditional probability of ��, ���������� ��, denoted by

��(��|��), is defined by ��(��|��) =��(�� ∩ ��)

 ��(��), ���������������� ��(��) > 0.

Conditional Probability on Tree Diagrams

If the probabilities on the second set of branches were different, there is dependence on the
outcome of the first event. This is known as conditional probability. Consider the slightly
more complicated example of drawing balls from a bag without replacement. The bag starts off
with 4 red balls and 6 blue balls. Two balls are drawn from the bag one after the other.

We can see from this diagram, for

example, that the probability of
drawing one of each is:

4
��(����) + ��(����) = 15
4 8
+ 15 = 15

7
Introduction to Probability Theory

The Product Rule or the Multiplicative Rule

If in an experiment the events �� ������ �� can both occur, then

��(�� ∩ ��) = ��(��) ��(��|��), provided ��(��) > 0.

Two events �� ������ �� are independent if and only if

��(�� ∩ ��) = ��(��)��(��)

Theorem of Total Probability

If the events ��1,��2, … … ,���� constitute a partition of the sample space ��

such that ��(����) ≠ 0 for �� = 1,2, … … , ��, then for any event �� in ��,
��
= ∑��(����)
��(��) =
��(��|����)
∑��(���� ∩ ��) ��=1
��=1
��
Baye’s Rule

If the events ��1,��2, … … ,���� constitute a partition of the sample space ��

such that ��(����) ≠ 0 for �� = 1,2, … … , ��, then for any event �� in ��
such that ��(��) ≠ 0, ��(����|��) =��(���� ∩ ��)
��
��=1=��(����) ��(��|����)
∑ ��(���� ∩ ��)
��
��=1������ �� = 1,2,
Probability Rules
… , �� ∑ ��(����)
��(��|����)
(For simultaneous trials) Multiplicative Rule
Baye’s Rule
(For consecutive
(Probability from joint & trials)
conditional)
Additive Rule
mutually exclusive Events are dependent
Events are not mutually
exclusive
Events are independent
Events are

8
Introduction to Probability Theory

Problem 01

Solution:
 9
Introduction to Probability Theory

Problem 02
Solution:

Figure: Tree
Diagram

10
Introduction to Probability Theory

Problem 03

Solution:

Problem 04
 Figure 2.9: An electrical system
Solution:

11
Introduction to Probability Theory

Problem 05
Semiconductor Failures
Continuing with semiconductor manufacturing, assume the following probabilities for product
failure subject to levels of contamination in manufacturing:

Probability of Failure Level of Contamination

0.10 High
0.01 Medium
0.001 Low
In a particular production run, 20% of the chips are subjected to high levels of contamination,
30% to medium levels of contamination, and 50% to low levels of contamination. (i) What is
the probability that a product using one of these chips fails?
(ii) If a randomly chosen chip fails, what is the probability that it is subject to medium levels
of contamination?
Solution:
(i) Let, H denote the event that a chip is exposed to high levels of contamination
M denote the event that a chip is exposed to medium levels of contamination L
denote the event that a chip is exposed to low levels of contamination

Then,

The calculations are conveniently organized with the tree diagram in following Figure.

Figure: Tree Diagram

()() ( )( )0.1277
(ii)( )( )
| 0.30 0.01
|===PMF

PMF ∩ PMPFM
=
PF
() () 0.0235
PF

12
Introduction to Probability Theory

Problem 06

Solution:
 Figure: Tree diagram

13
Introduction to Probability Theory

Problem 07
With reference to previous Problem 06, if a product was chosen randomly and found to be
defective, what is the probability that it was made by machine B3?

Solution:
and then substituting the probabilities calculated in previous example, we have

Problem 08

Solution:

14
Introduction to Probability Theory

Problem 09

Solution:
Problem 10

Solution:

15
Introduction to Probability Theory

Problem 11
The complete tree diagram is shown in following figure.

Figure: Complete Tree Diagram

16
 Introduction to Probability Theory

Problem 12
A lot of 100 semiconductor chips contain 20 that are defective.
(a) Two are selected, at random, without replacement, from the lot. Determine the probability
that the second chip selected is defective.
(b) Three are selected, at random, without replacement, from the lot. Determine the
probability that all are defective.

Solution:
(a) Two are selected, at random, without replacement, from the lot. Now the probability that
the second chip selected is defective
P (2nd is defective) = P (1st is defective and 2nd is defective)
+ P (1st is good and 2nd is defective)
= P (2nd is defective/1st is defective) P (1st is defective)
+ P (2nd is defective/1st is good) P (1st is good)
= (19/99) × (20/100) + (20/99) × (80/100)
= 0.19999
(b) Three are selected, at random, without replacement, from the lot. Now the probability
that all are defective
P (3 are all defective) = (20/100) × (19/99) × (18/98)
= 0.00704
 17
Introduction to Probability Theory

Problem 13
Disks of polycarbonate plastic from a supplier are analyzed for scratch and shock resistance.
The results from 100 disks are summarized as follows:
Shock Resistance
High Low
Scratch Resistance High 70 9 Low 16 5

(a) If a disk is selected at random, what is the probability that its scratch resistance is high
and its shock resistance is high?
(b) If a disk is selected at random, what is the probability that its scratch resistance is high
or its shock resistance is high?
(c) For the data above in the previous question, let A denote the event that a disk has high
shock resistance, and let B denote the event that a disk has high scratch resistance.
Determine the following probabilities:
(i) P (A) (ii) P (B) (iii) P (A\B) (iv) P (B\A)
Solution:
(a) If a disk is selected at random, the probability that its scratch resistance is high and
its shock resistance is high is
70 / (70 + 9 + 16 + 5) = 0.7
(b) If a disk is selected at random, the probability that its scratch resistance is high or its shock
resistance is high is
(70 + 9 + 16) / (70 + 9 + 16 + 5) = 95 / 100 = 0.95

(c) (i) P(A) = (70+16) / 100 = 0.86

(ii) P(B) = (70+9) / 100 = 0.79
(iii) P(A\B) = P(AB) / P(B) = 70 / 79 = 0.88607
(iv) P(B\A) = P(AB) / P(A) = 70 / 86 = 0.81395

Problem 14
Power Station. Two power stations operate in parallel to provide electricity supply of a village
located in a recreational area. Electricity demand is subject to considerable weekly and seasonal
fluctuations. Each station has a capacity so that it can supply the demand 80 percent of the time
in case the other station fails. The probability of failure of each station is 10 percent, whereas
the probability that both stations fail is 3 percent. What is the probability that the village
demand will be satisfied?

Solution:
 P[F1] = 0.1 – 0.03 = 0.07, P[F2] = 0.1 – 0.03 = 0.07, P[F12 ] = 0.03 where
Fi ≡ failure of only station i and Fij ≡ both stations i and j fail.
The 2 stations are the independent because
P[F12] = 0.03 ≠ P[F1]×P[F2].
P[Village unsatisfied] = P[U]
= P[U∩F12] + P[U∩F1] + P[U∩F2]
= P[U|F12] P[F12] + P[U|F1] P[F1] + P[U|F2] P[F2]
= 1 × P [F12 ] + 0.2 × P[F1] + 0.2 ×P[F2]
= 0.03 + 0.2 × 0.07 + 0.2 × 0.07
= 0.058
P[Village satisfied] = P[S] = 1- P[U] = 1– 0.058 = 0.942

18
Introduction to Probability Theory

Problem 15
A company produces machine components which pass through an automatic testing machine.
5% of the components entering the testing machine are defective. However, the machine is not
entirely reliable. If a component is defective there is 4% probability that it will not be rejected.
If a component is not defective there is 7% probability that it will be rejected.
a) What fraction of all the components are rejected?
b) What fraction of the components rejected are actually not defective?
c) What fraction of those not rejected are defective?
Answer
Let D represent a defective component, and G a good component.
Let R represent a rejected component, and A an accepted component.
Part (a) can be answered directly using a tree diagram.

Now we can calculate the probabilities of the various combined events:

Pr [D ∩ R] = Pr [D] × Pr [R | D] = (0.05)(0.96) = 0.0480 Rejected
Pr [D ∩ A] = Pr [D] × Pr [A | D] = (0.05)(0.04) = 0.0020 Accepted
Pr [G ∩ R] = Pr [G] × Pr [R | G] = (0.95)(0.07) = 0.0665 Rejected
Pr [G ∩ A] = Pr [G] × Pr [A | G] = (0.95)(0.93) = 0.8835 Accepted
Total = 1.0000 (Check)
Because all possibilities have been considered and there is no overlap among them, we see that
the “rejected” area is composed of only two possibilities, so the probability of rejection is the
sum of the probabilities of two intersections. The same can be said of the “accepted” area. Then
Pr [R] = Pr [D ∩ R] + Pr [G ∩ R] = 0.0480 + 0.0665 = 0.1145 and Pr [A] = Pr [D ∩ A] + Pr [G
∩ A] = 0.0020 + 0.8835 = 0.8855
a) The fraction rejected will be the probability of rejection, i,e, 0.1145 or 11.45 %. Now we can
calculate the required quantities to answer parts (b) and (c) using conditional probabilities in
the opposite order, so in a sense applying them backwards.

b) Fraction of components rejected which are not defective

= probability that a component is good, given that it was rejected
= Pr [G | R] = Pr [G ∩R] / Pr (R) = 0.0665/0.1145 = 0.58 or 58%

c) Fraction of components passed which are actually defective

= probability that a component is defective, given that it was passed
= Pr [D | A] = Pr [D ∩A] / Pr (A) = 0.0020/0.8855 = 0.0023 or 0.23 %.
(Note that Pr [G | R] ≠ Pr [R | G], and Pr [D | A] ≠ Pr [A | D].)

19
Introduction to Probability Theory

Assignment
Problem 01: When testing 1000 soldered joints, 4 failed during a vibration test and 5 failed
due to having a high resistance. Determine the probability of a joint failing due to (a) vibration,
(b) high resistance,
(c) vibration or high resistance and
(d) vibration and high resistance.

Problem 02: 16 parts are examined for defects. It is found that 10 are good, 4 have minor
defects, and 2 have major defects. Two parts are chosen at random from the 16 without
replacement, that is, the first part chosen is not returned to the mix before the second part is
chosen. Notice, then, that there will be only 15 possible choices for the second
part. (a) What is the probability that both are good?
(b) What is the probability that exactly one part has a major defect?

Problem 03: A shipment of 17 radios includes 5 radios that are defective. The receiver
samples 6 radios at random. What is the probability that exactly 3 of the radios selected are
defective? Solve the problem
(a) Using a probability tree diagram
(b) Using permutations and combinations.

Problem 04: A batch of 1 kW fire elements contains 16 which are within a power tolerance
and 4 which are not. If 3 elements are selected at random from the batch, calculate the
probabilities that
(a) all three are within the power tolerance and
(b) two are within but one is not within the power tolerance.
[(a) 0.4912 (b)0.4211]

Problem 05: A box contains 14 40W lamps, 28 60W lamps and 58 25W lamps, all the lamps
being of the same shape and size. Three lamps are drawn at random from the box, first one, then
a second, then a third. Determine the probabilities of:
(a) getting one 25W, one 40W and one 60W lamp, with replacement,
(b) getting one 25W, one 40W and one 60W lamp without replacement, and (c) getting
either one 25W and two 40W or one 60W and two 40W lamps with replacement. [(a)
0.0227 (b) 0.0234 (c) 0.0169]
Problem 06: From a box containing 6 black balls and 4 green balls, 3 balls are drawn in
succession, each ball being replaced in the box before the next draw is made. What is the
probability that
(a) All 3 are the same color? (b) Each color is represented? (0.28) & (0.72)

Problem 07: A bag contains 3 red, 6 white and 7 blue balls. What is the probability that two
balls drawn are white and blue? (0.35)

Problem 08: A computer game has three levels and one of the objectives of every level is to
collect a diamond. The probability of a randomly chosen player collecting a diamond on the
first level is 4/5, the second level is 2/3, and the third level is ½. The events are independent.
(a) Draw a tree diagram collecting diamonds on the three levels of the game.
(b) Find the probability that a randomly chosen player
(i) Collect all three diamonds. (ii) Collect only one diamond. (0.267) & (0.233)

20
Introduction to Probability Theory

Problem 09: The turnout of spectators at a motor rally is dependent upon the weather. On a
rainy day the probability of a big turnout is 0.4, but if does not rain, the probability of a big
turnout increases to 0.9. The weather forecast gives a probability of 0.75 that it will rain on the
day of the race.
(a) Draw a tree diagram to represent the information.
(b) Find the probability that there is a big turnout and it rains. (0.3) (c) Find the
probability that there is a big turnout. (0.525)

Problem 10: It is known that of the articles produced by a factory, 20% come from Machine
A, 30% from Machine B, and 50% from Machine C. The percentages of satisfactory articles
among those produced are 95% for A, 85% for B, and 90% for C. An article is chosen at
random. (a) What is the probability that it is satisfactory?
(b) Assuming that the article is satisfactory, what is the probability that it was produced by
Machine A?

Problem 11: In a factory, machines A, B, and C produce electronic components. Machine A

produces 16% of the components, machine B produces 50% of the components and machine C
produces the rest. Some of the components are defective. Machine A produces 4%, machine B
produces 3% and machine C produces 7% defective components.
(a) Draw a tree diagram to represent this information.
(b) Find the probability that a selected component is produced by machine B and defective.
(c) Defected
(d) Given that a randomly selected component is defected, find the probability that was
produced by machine B. (b) (0.015) (c) (0.0452) (d) (0.332)

Problem 12: A company producing electric relays has three manufacturing plants producing
50, 30, and 20 percent, respectively of its products. Suppose that the probabilities that a relay
manufactured by these plants is defective are 0.02, 0.05, and 0.01 respectively.
(a) If a relay is selected at random from the output of the company, what is the probability
that it is defective? (0.027)
(b) If a relay selected at random is found to be defective, what is the probability that it was
manufactured by plant 2? (0.556)
Problem 13: Computer keyboard failures are due to faulty electrical connects (12%) or
mechanical defects (88%). Mechanical defects are related to loose keys (27%) or improper
assembly (73%). Electrical connect defects are caused by defective wires (35%), improper
connections (13%), or poorly welded wires (52%).
(a) Find the probability that a failure is due to loose keys.
(b) Find the probability that a failure is due to improperly connected or poorly welded wires.

Problem 14: In a box containing 120 similar transistors 70 are satisfactory, 37 give too high a
gain under normal operating conditions and the remainder give too low a gain. Calculate the
probability that when drawing two transistors in turn, at random, with replacement, of having
(a) two satisfactory,
(b) none with low gain,
(c) one with high gain and one satisfactory,
(d) one with low gain and none satisfactory.
Determine the probabilities in (a), (b) and (c) above if the transistors are drawn without
replacement.

21