Elementary Probability Theory

Chapter Outline
Basic Probability

❖ Definitions and Basic Rules of Probability

❖ Conditional Probability, Independence and Bayes Theorem

We start our investigation of probability with some basic concepts and definitions.
We live in a world full of uncertainty and describe this in many different ways.
➢ I should be back by 10 pm
➢ He is sure to be in the team
➢ Sir will not take class tomorrow
Each of these phrases expresses a degree of uncertainty. To deal with uncertainty mathematically
you need to be able to express it as number. We can do this by using a scale from 0 to 1 to
describe the chance that something will happen. On this scale 0 represents a situation which is
impossible and 1 represents something that is certain to happen. This number is called the
probability that something will happen.
So, if P is a probability then 0  P  1.

In probability we are always concerned with experiments in which the outcome is not
determined in advance.
For example:
 Rolling a 6 sided die;
 Dealing a hand of 5 cards from a deck of 52;
 Measuring the time to success of a computer program.
 Measuring rainfall on a given day.
 Measuring CO2 emissions from a car or a factory.
 Measuring how many people use a rail link between 7am and 9am on a given day.

Some Basic Definitions

Random Experiment

Example
Tossing a coin, throwing dice etc. are all examples of random experiments.

1
 Elementary Probability Theory

Sample Space

Example
 If we roll an unbiased (fair) 6-sided die, then the sample space is 𝑆 = {1,2,3,4,5,6}.
 If we toss a fair coin twice, and denote heads by H and tails by T then the sample space is
𝑆 = (𝐻𝐻, 𝑇𝐻, 𝐻𝑇, 𝑇𝑇)

In all of these examples, the sample space is finite. In many examples, sample spaces are not
finite.
 If we toss a fair coin until heads comes up, then the sample space is
𝑆 = {𝐻, 𝑇𝐻, 𝑇𝑇𝐻, 𝑇𝑇𝑇𝐻, … … }.
This time the sample space is infinite. However, the sample space is still countable or
discrete.
 If we are measuring the emissions from cars in an NCT center and are interested in how
many cars we need to test before we find one that fails, then our sample space is again
countably infinite.
 If we are measuring the actual emissions in 𝑔/𝑘𝑚 then our sample space would be
continuous.
Note that the definition of a sample space depends on what we are interested in measuring. We
may only be interested in whether or not a car fails an emission test in which case the sample
space would be
{𝑃𝑎𝑠𝑠, 𝑓𝑎𝑖𝑙}.
So, based on above examples

Discrete and Continuous Sample Space

Event

2
 Elementary Probability Theory

Before defining some basic laws on how to combine events, we need to review some elementary
notions from set theory.

Events can be combined in a variety of ways corresponding to standard operations on sets. The
three basic operations are union, intersection and complement. If 𝐸 𝑎𝑛𝑑 𝐹 are two events,
then

 Union: 𝑬 ∪ 𝑭 is the event consisting of all outcomes that are either in E or F (or both).
 Intersection: 𝑬 ∩ 𝑭 is the event consisting of all outcomes that are in both E and F.
 Complement: 𝑬𝒄 𝒐𝒓 𝑬′ is the event consisting of all outcomes not belonging to E.
Venn diagrams are a useful way to think about events and combinations of events:

 𝐸 ∪ 𝐹 is the entire coloured region;

 𝐸 ∩ 𝐹 is the purple shaded region;
 (𝐸 ∪ 𝐹)𝑐 is the unshaded region.

3
 Elementary Probability Theory

Mutually Exclusive Events

Two events 𝐸 𝑎𝑛𝑑 𝐹 are said to be mutually exclusive if it is not possible that both of them occur
at the same time 𝑖. 𝑒. 𝐸 ∩ 𝐹 = 𝜙.

For example, consider the toss of a coin. Let 𝐻 be the event that the coin lands on heads and 𝑇
be the event that the coin lands on tails. It follows that, in a single fair coin toss, events 𝐻 𝑎𝑛𝑑 𝑇
are mutually exclusive.

Mutual exclusivity can be shown on a Venn diagram.

 𝐸 𝑎𝑛𝑑 𝐹 are mutually exclusive

 𝐸 𝑎𝑛𝑑 𝐺 are not mutually exclusive
 𝐹 𝑎𝑛𝑑 𝐺 are not mutually exclusive
Another word for mutually exclusive is disjoint.

Measurements of the time needed to complete a computer program might be modeled with
the sample space 𝑆 = 𝑅 + , the set of positive real numbers. Let

4
 Elementary Probability Theory

Independent & Dependent Events

Two or more events are said to be independent if the occurrence of an event does not affect the
occurrence of the other. Otherwise, the events are said to be dependent.
For example,
 Suppose that we are tossing a coin twice. Let 𝐸 be the event that the first coin toss lands
on heads. In addition, let 𝐹 be the event that the second coin toss lands on heads. Clearly
the result of the second coin toss is no way affected by the result of the first coin toss.
 The event of drawing a card twice out of 52 cards without replacement. In the first case,
we draw one card out of 52 cards and in the second case, we draw one card out of 51
cards. In this case, the second drawing is dependent on the first drawing.
But, if the card is replaced before the drawing of the second card, then the total cards
remain the same as 52. In that case the events are independent.

Tree Diagram
Sample spaces can also be described graphically with tree diagrams. Tree Diagrams, like Venn
Diagrams, provide a simple visual way of representing events and their associated probabilities.
They can be used to calculate more complicated probabilities. When a sample space can be
constructed in several steps or stages, we can represent each of the 𝑛1 ways of completing the
first step as a branch of a tree. Each of the ways of completing the second step can be represented
as 𝑛2 branches starting from the ends of the original branches, and so forth.
Example
An automobile manufacturer provides vehicles equipped with selected options. Each vehicle is
ordered
➢ With or without an automatic transmission
➢ With or without air conditioning
➢ With one of three choices of a stereo system
➢ With one of four exterior colors
If the sample space consists of the set of all possible vehicle types, what is the number of
outcomes in the sample space?
The sample space contains 48 outcomes. The tree diagram for the different types of vehicles is
displayed in Fig. 1

Fig. 1: Tree diagram for different types of vehicles with 48 outcomes in the sample space.

5
 Elementary Probability Theory

Example:
Automobile Configurations
Consider an extension of the automobile manufacturer illustration in the previous example in
which another vehicle option is the interior color. There are four choices of interior color: red,
black, blue, or brown. However,

➢ With a red exterior, only a black or red interior can be chosen.

➢ With a white exterior, any interior color can be chosen.
➢ With a blue exterior, only a black, red, or blue interior can be chosen.
➢ With a brown exterior, only a brown interior can be chosen.

In Fig. 1, there are 12 vehicle types with each exterior color, but the number of interior color
choices depends on the exterior color. As shown in Fig. 2, the tree diagram can be extended to
show that there are 120 different vehicle types in the sample space.

Fig. 2: Tree diagram for different types of vehicles with interior colors.

Independent Events in Tree Diagrams

Consider the very simple case of flipping a coin twice – each toss is a separate event. If the
coin is fair, the probability of tossing heads and tails is clearly 0.5. In the second toss, these
probabilities still hold.

In this example, all sets of branches have identical probabilities. This is typical of a tree
diagram where the events are independent.

6
 Elementary Probability Theory

Classical Probability

If an experiment can result in any one of 𝑁 different equally likely outcomes, and if
exactly 𝑛 of these ooutcomes correspond to event 𝐴, then the probability of event 𝐴 is
𝑛
𝑃(𝐴) =
𝑁

Essentially, in this case the probability of an event is the ratio of favorable outcomes to
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑓𝑎𝑣𝑜𝑟𝑎𝑏𝑙𝑒 𝑡𝑜 𝑒𝑣𝑒𝑛𝑡 𝐴
all possible outcomes. i.e. 𝑃(𝐴) =
𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠

More Laws of Probability

The Additive Rule

If 𝐴 𝑎𝑛𝑑 𝐵 are two events, then

𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵)

If 𝐴 𝑎𝑛𝑑 𝐵 are two mutually exclusive events, then

𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵)

If 𝐴1 , 𝐴2 , … … , 𝐴𝑛 are mutually exclusive events, then

𝑃(𝐴1 ∪ 𝐴2 ∪ ⋯ ∪ 𝐴𝑛 ) = 𝑃(𝐴1 ) + 𝑃(𝐴2 ) + ⋯ + 𝑃(𝐴𝑛 )

For three events 𝐴, 𝐵 𝑎𝑛𝑑 𝐶,

𝑃(𝐴 ∪ 𝐵 ∪ 𝐶) = 𝑃(𝐴) + 𝑃(𝐵) + 𝑃(𝐶)
−𝑃(𝐴 ∩ 𝐵) − 𝑃(𝐵 ∩ 𝐶) − 𝑃(𝐶 ∩ 𝐴) + 𝑃(𝐴 ∩ 𝐵 ∩ 𝐶)

If 𝐴 𝑎𝑛𝑑 𝐴′ are complementary events, then 𝑃(𝐴) + 𝑃(𝐴′) = 1

7
 Elementary Probability Theory

Conditional Probability
Conditional probability is a probability of an event under condition that some other (not
impossible) event has happened.

The conditional probability of 𝐵, 𝑔𝑖𝑣𝑒𝑛 𝐴, denoted by 𝑃(𝐵|𝐴), is defined by

𝑃(𝐴 ∩ 𝐵)
𝑃(𝐵|𝐴) = , 𝑝𝑟𝑜𝑣𝑖𝑑𝑒𝑑 𝑃(𝐴) > 0
𝑃(𝐴)

Conditional Probability on Tree Diagrams

If the probabilities on the second set of branches were different, there is dependence on the
outcome of the first event. This is known as conditional probability.
Consider the slightly more complicated example of drawing balls from a bag without
replacement. The bag starts off with 4 red balls and 6 blue balls. Two balls are drawn from the
bag one after the other.

We can see from this diagram, for

example, that the probability of
drawing one of each is:
4 4 8
𝑃(𝑅𝐵) + 𝑃(𝐵𝑅) = + =
15 15 15

The Product Rule or the Multiplicative Rule

If in an experiment the events 𝐴 𝑎𝑛𝑑 𝐵 can both occur, then

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) 𝑃(𝐵|𝐴), provided 𝑃(𝐴) > 0.

Symbolically, two events 𝐴 𝑎𝑛𝑑 𝐵 are independent if 𝑃(𝐵|𝐴) = 𝑃(𝐵).

Now, substituting 𝑃(𝐵) for 𝑃(𝐵|𝐴) into the above formula yields another formula for the
Product rule or Multiplicative rule

Two events 𝐴 𝑎𝑛𝑑 𝐵 are independent if and only if

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴)𝑃(𝐵)

8
 Elementary Probability Theory

Theorem of Total Probability

If the events 𝐵1 , 𝐵2 , … … , 𝐵𝑘 constitute a partition of the sample space 𝑆 such that

𝑃(𝐵𝑖 ) ≠ 0 for 𝑖 = 1,2, … … , 𝑘, then for any event 𝐴 in 𝑆,
𝑘 𝑘

𝑃(𝐴) = ∑ 𝑃(𝐵𝑖 ∩ 𝐴) = ∑ 𝑃(𝐵𝑖 ) 𝑃(𝐴|𝐵𝑖 )

𝑖=1 𝑖=1

Baye’s Rule

If the events 𝐵1 , 𝐵2 , … … , 𝐵𝑘 constitute a partition of the sample space 𝑆 such that

𝑃(𝐵𝑖 ) ≠ 0 for 𝑖 = 1,2, … … , 𝑘, then for any event 𝐴 in 𝑆 such that 𝑃(𝐴) ≠ 0,
𝑃(𝐵𝑟 ∩ 𝐴) 𝑃(𝐵𝑟 ) 𝑃(𝐴|𝐵𝑟 )
𝑃(𝐵𝑟 |𝐴) = 𝑘 = 𝑘 𝑓𝑜𝑟 𝑟 = 1,2, … , 𝑘
∑𝑖=1 𝑃(𝐵𝑖 ∩ 𝐴) ∑𝑖=1 𝑃(𝐵𝑖 ) 𝑃(𝐴|𝐵𝑖 )

Probability Rules

Baye’s Rule
Additive Rule Multiplicative Rule
(Probability from joint &
(For simultaneous trials) (For consecutive trials)
conditional)

Events are Events are not Events are Events are

mutually exclusive mutually exclusive independent dependent

9
 Elementary Probability Theory

Problem 01

10
 Elementary Probability Theory

Problem 02

Solution:

Figure: Tree Diagram

11
 Elementary Probability Theory

Problem 03

Solution:

Problem 04
With reference to previous Problem 06, if a product was chosen randomly and found to
be defective, what is the probability that it was made by machine B3?

Solution:

and then substituting the probabilities calculated in previous example, we have

12
 Elementary Probability Theory

Problem 05

Solution:

Problem 06
A lot of 100 semiconductor chips contain 20 that are defective.
(a) Two are selected, at random, without replacement, from the lot. Determine the probability
that the second chip selected is defective.
(b) Three are selected, at random, without replacement, from the lot. Determine the
probability that all are defective.
Solution:
(a) Two are selected, at random, without replacement, from the lot. Now the probability that
the second chip selected is defective
P (2nd is defective) = P (1st is defective and 2nd is defective)
+ P (1st is good and 2nd is defective)
= P (2nd is defective/1st is defective) P (1st is defective)
+ P (2nd is defective/1st is good) P (1st is good)
= (19/99) × (20/100) + (20/99) × (80/100)
= 0.19999
(b) Three are selected, at random, without replacement, from the lot. Now the probability that
all are defective
P (3 are all defective) = (20/100) × (19/99) × (18/98) = 0.00704

13
 Elementary Probability Theory

Problem 07
A company produces machine components which pass through an automatic testing machine.
5% of the components entering the testing machine are defective. However, the machine is not
entirely reliable. If a component is defective there is 4% probability that it will not be rejected.
If a component is not defective there is 7% probability that it will be rejected.
a) What fraction of all the components are rejected?
b) What fraction of the components rejected are actually not defective?
c) What fraction of those not rejected are defective?
Answer
Let D represent a defective component, and G a good component.
Let R represent a rejected component, and A an accepted component.
Part (a) can be answered directly using a tree diagram.

Now we can calculate the probabilities of the various combined events:

Pr [D ∩ R] = Pr [D] × Pr [R | D] = (0.05) (0.96) = 0.0480 Rejected
Pr [D ∩ A] = Pr [D] × Pr [A | D] = (0.05) (0.04) = 0.0020 Accepted
Pr [G ∩ R] = Pr [G] × Pr [R | G] = (0.95) (0.07) = 0.0665 Rejected
Pr [G ∩ A] = Pr [G] × Pr [A | G] = (0.95) (0.93) = 0.8835 Accepted
Total = 1.0000 (Check)
Because all possibilities have been considered and there is no overlap among them, we see that
the “rejected” area is composed of only two possibilities, so the probability of rejection is the
sum of the probabilities of two intersections. The same can be said of the “accepted” area.
Then Pr [R] = Pr [D ∩ R] + Pr [G ∩ R] = 0.0480 + 0.0665 = 0.1145
and Pr [A] = Pr [D ∩ A] + Pr [G ∩ A] = 0.0020 + 0.8835 = 0.8855

a) The fraction rejected will be the probability of rejection, i,e, 0.1145 or 11.45 %.
Now we can calculate the required quantities to answer parts (b) and (c) using conditional
probabilities in the opposite order, so in a sense applying them backwards.

b) Fraction of components rejected which are not defective

= probability that a component is good, given that it was rejected
= Pr [G | R] = Pr [G ∩R] / Pr (R) = 0.0665/0.1145 = 0.58 or 58%

c) Fraction of components passed which are actually defective

= probability that a component is defective, given that it was passed
= Pr [D | A] = Pr [D ∩A] / Pr (A) = 0.0020/0.8855 = 0.0023 or 0.23 %.
(Note that Pr [G | R] ≠ Pr [R | G], and Pr [D | A] ≠ Pr [A | D].)

14
 Elementary Probability Theory

HomeWorks

Problem 01: 16 parts are examined for defects. It is found that 10 are good, 4 have minor defects,
and 2 have major defects. Two parts are chosen at random from the 16 without replacement, that
is, the first part chosen is not returned to the mix before the second part is chosen.
(a) What is the probability that both are good?
(b) What is the probability that exactly one part has a major defect?

Problem 02: A box contains 14 40W lamps, 28 60W lamps and 58 25W lamps, all the lamps
being of the same shape and size. Three lamps are drawn at random from the box, first one, then
a second, then a third. Determine the probabilities of:
(a) getting one 25W, one 40W and one 60W lamp, with replacement,
(b) getting one 25W, one 40W and one 60W lamp without replacement, and
(c) getting either one 25W and two 40W or one 60W and two 40W lamps with replacement.
(a) 0.0227 (b) 0.0234 (c) 0.0169

Problem 03: From a box containing 6 black balls and 4 green balls, 3 balls are drawn in
succession, each ball being replaced in the box before the next draw is made.
What is the probability that
(a) All 3 are the same color?
(b) Each color is represented?
(a) 0.28 (b) 0.72

Problem 04: Computer keyboard failures are due to faulty electrical connects (12%) or
mechanical defects (88%). Mechanical defects are related to lose keys (27%) or improper
assembly (73%). Electrical connect defects are caused by defective wires (35%), improper
connections (13%), or poorly welded wires (52%).
(a) Find the probability that a failure is due to lose keys.
(b) Find the probability that a failure is due to improperly connected or poorly welded wires.
(a) 0.2376 (b) 0.078

Problem 05: A computer game has three levels and one of the objectives of every level is to
collect a diamond. The probability of a randomly chosen player collecting a diamond on the first
level is 4/5, the second level is 2/3, and the third level is ½. The events are independent.
(a) Draw a tree diagram collecting diamonds on the three levels of the game.
(b) Find the probability that a randomly chosen player
(i) Collect all three diamonds. (ii) Collect only one diamond. (0.267) & (0.233)

Problem 06: It is known that of the articles produced by a factory, 20% come from Machine A,
30% from Machine B, and 50% from Machine C. The percentages of satisfactory articles among
those produced are 95% for A, 85% for B, and 90% for C. An article is chosen at random.
(a) What is the probability that it is satisfactory?
(b) Assuming that the article is satisfactory, what is the probability that it was produced by
Machine A?

15
 Elementary Probability Theory

Problem 07: In a factory, machines A, B, and C produce computer components. Machine A

produces 16% of the components, machine B produces 50% of the components and machine C
produces the rest. Some of the components are defective. Machine A produces 4%, machine B
produces 3% and machine C produces 7% defective components.
(a) Draw a tree diagram to represent this information.
(b) Find the probability that a selected component is produced by machine B and defective.
(c) Defected
(d) Given that a randomly selected component is defected, find the probability that was
produced by machine B.
(b) (0.015) (c) (0.0452) (d) (0.332)

Problem 08: A company producing electric relays has three manufacturing plants producing 50,
30, and 20 percent, respectively of its products. Suppose that the probabilities that a relay
manufactured by these plants is defective are 0.02, 0.05, and 0.01 respectively.
(a) If a relay is selected at random from the output of the company, what is the probability
that it is defective?
(b) If a relay selected at random is found to be defective, what is the probability that it was
manufactured by plant 2?
(a) 0.027 (b) 0.556

Problem 09: Incoming calls to a customer service center are classified as complaints (75% of
call) or requests for information (25% of calls). Of the complaints, 40% deal with computer
equipment that does not respond and 57% deal with incomplete software installation; and in the
remaining 3% of complaints the user has improperly followed the installation instructions. The
requests for information are evenly divided on technical questions (50%) and requests to
purchase more products (50%).
(a) What is the probability that an incoming call to the customer service center will be from
a customer who has not followed installation instructions properly?
(b) Find the probability that an incoming call is a request for purchasing more products.
(a) 0.0225 (b) 0.125

16