Probability & Statistics for Scientist and Engineers

Dr. M. M. Bhatti

College of Mathematics and Systems Science, Shandong University of Science and

Technology, Qingdao, China
[email protected], [email protected]

March 9, 2023

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 1 / 25

Outline

1 Introduction

2 Basic probability
Random Experiments
Sample spaces
Events
The concept of probability
The Axioms of Probability
Some Important Theorems on Probability
Assignment of Probabilities

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 2 / 25

Introduction

Books:
1 Bluman, A. G. (2009). Elementary statistics: A step by step approach.
New York;: McGraw-Hill Higher Education.
2 Walpole, R. E., Myers, R. H., Myers, S. L., & Keying, E. Y. (2013). Es-
sentials of Probability & Statistics for Engineers & Scientists: Pearson
New International Edition PDF eBook. Pearson Higher Ed.
Marks strategy:

Class participation 40%

Final paper 60%

Class participation: Assignments+Attendance+Mid-term exam.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 3 / 25

Random Experiments

We are all familiar with the importance of experiments in science and engi-
neering. Experimentation is useful to us because we can assume that if we
perform certain experiments under very nearly identical conditions, we will
arrive at results that are essentially the same. In these circumstances, we
are able to control the value of the variables that affect the outcome of the
experiment.
However, in some experiments, we are not able to ascertain or control the
value of certain variables so that the results will vary from one performance
of the experiment to the next even though most of the conditions are the
same. These experiments are described as random.
Example 1.1
If we toss a coin, the result of the experiment is that it will either come up
“tails,” symbolized by T (or 0), or “heads,” symbolized by H (or 1), i.e.,
one of the elements of the set {H, T } (or {0, 1}).

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 4 / 25

Random Experiments

Example 1.2
If we toss a die, the result of the experiment is that it will come up with
one of the numbers in the set {1, 2, 3, 4, 5, 6}.

Example 1.3
If we toss a coin twice, there are four results possible, as indicated by
{HH, HT, T H, T T }, i.e., both heads, heads on first and tails on second,
etc.

Example 1.4
If we are making bolts with a machine, the result of the experiment is that
some may be defective. Thus when a bolt is made, it will be a member of
the set { defective, nondefective }.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 5 / 25

Sample spaces

A set S that consists of all possible outcomes of a random experiment is

called a sample space, and each outcome is called a sample point. Often
there will be more than one sample space that can describe outcomes of
an experiment, but there is usually only one that will provide the most
information.
Example 1.5
If we toss a die, one sample space, or set of all possible outcomes, is given
by {1, 2, 3, 4, 5, 6} while another is {odd, even}. It is clear, however, that
the latter would not be adequate to determine, for example, whether an
outcome is divisible by 3.

N.B: It is often useful to portray a sample space graphically. In such cases

it is desirable to use numbers in place of letters whenever possible.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 6 / 25

Sample spaces

Example 1.6
If we toss a coin twice and use 0 to represent tails and 1 to represent
heads, the sample space (see Example 1.3) can be portrayed by points as
in Fig. 1-1 where, for example, (0, 1) represents tails on first toss and
heads on second toss, i.e., TH.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 7 / 25

Sample spaces

Important definitions:
If a sample space has a finite number of points, as in Example 1.7, it
is called a finite sample space.
If it has as many points as there are natural numbers 1, 2, 3, . . . , it is
called a countably infinite sample space.
If it has as many points as there are in some interval on the x−axis,
such as 0 ≤ x ≤ 1, it is called a noncountably infinite sample space.
A sample space that is finite or countably infinite is often called a
discrete sample space, while one that is noncountably infinite is
called a nondiscrete sample space.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 8 / 25

Events
An event is a subset A of the sample space S, i.e., it is a set of possible
outcomes. If the outcome of an experiment is an element of A, we say that
the event A has occurred. An event consisting of a single point of S is
often called a simple or elementary event.
Example 1.7
If we toss a coin twice, the event that only one head comes up is the
subset of the sample space that consists of points (0, 1) and (1, 0), as
indicated in Fig. 1-2.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 9 / 25

Events

As particular events, we have S itself, which is the sure or certain event

since an element of S must occur, and the empty set ∅, which is called the
impossible event because an element of ∅ cannot occur.
By using set operations on events in S, we can obtain other events in S.
For example, if A and B are events, then
A ∪ B is the event “either A or B or both.” A ∪ B is called the union
of A and B.
A ∩ B is the event “both A or B.” A ∩ B is called the intersection of
A and B.
A0 is the event “not A.” A is called the complement of A.
A − B = A ∩ B 0 is the event “A but not B.” In particular, A0 = S − A.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 10 / 25

Events

Important definition:
If the sets corresponding to events A and B are disjoint, i.e., A ∩
B = ∅, we often say that the events are mutually exclusive. This
means that they cannot both occur. We say that a collection of events
A1 , A2 , . . . , An is mutually exclusive if every pair in the collection is
mutually exclusive.
Furthermore, Mutually exclusive events are events that can not happen
at the same time. Examples include: right and left hand turns, even
and odd numbers on a die, winning and losing a game, or running and
walking.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 11 / 25

Events

Example 1.8
Referring to the experiment of tossing a coin twice, let A be the event at
least one head occurs and B the event “the second toss results in a tail.”
Then A = {HT, T H, HH}, B = {HT, T T }, and so we have

A ∪ B = {HT, T H, HH, T T } = S, A ∩ B = {HT },

A0 = T T , A − B = {T H, HH}

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 12 / 25

The concept of probability

In any random experiment there is always uncertainty as to whether a partic-

ular event will or will not occur. As a measure of the chance, or probability,
with which we can expect the event to occur, it is convenient to assign a
number between 0 and 1. If we are sure or certain that the event will occur,
we say that its probability is 100% or 1, but if we are sure that the event will
not occur, we say that its probability is zero. If, for example, the probability
is 1/4 we would say that there is a 25% chance it will occur and a 75%
chance that it will not occur. Equivalently, we can say that the odds against
its occurrence are 75% to 25%, or 3 to 1.
There are two important procedures by means of which we can estimate the
probability of an event.
Classical approach.
Frequency approach.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 13 / 25

The concept of probability

Classical approach: If an event can occur in h different ways out of

a total number of n possible ways, all of which are equally likely, then
the probability of the event is h/n.

Example 1.9
Suppose we want to know the probability that a head will turn up in a
single toss of a coin. Since there are two equally likely ways in which the
coin can come upnamely, heads and tails (assuming it does not roll away
or stand on its edge) and of these two ways a head can arise in only one
way, we reason that the required probability is 1/2. In arriving at this, we
assume that the coin is fair, i.e., not loaded in any way.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 14 / 25

The concept of probability

Frequency approach: If after n repetitions of an experiment, where

n is very large, an event is observed to occur in h of these, then
the probability of the event is h/n. This is also called the empirical
probability of the event.

Example 1.10
If we toss a coin 1000 times and find that it comes up heads 532 times, we
estimate the probability of a head coming up to be 532/1000 = 0.532.

Drawbacks: Both the classical and frequency approaches have serious

drawbacks, the first because the words “equally likely” are vague and
the second because the “large number” involved is vague. Because of
these difficulties, mathematicians have been led to an axiomatic approach
to probability.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 15 / 25

The Axioms of Probability
Suppose we have a sample space S. If S is discrete, all subsets correspond
to events and conversely, but if S is nondiscrete, only special subsets (called
measurable) correspond to events. To each event A in the class C of
events, we associate a real number P (A). Then P is called a probability
function, and P (A) the probability of the event A, if the following axioms
are satisfied.
Axiom 1
For every event A in the class C,

P (A) ≥ 0 (1)

Axiom 2
For the sure or certain event S in the class C,

P (S) = 1 (2)

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 16 / 25

The Axioms of Probability

Axiom 3
For any number of mutually exclusive events A1 , A2 , . . . , in the class C,

P (A1 ∪ A2 ∪ . . . ) = P (A1 ) + P (A2 ) + . . . (3)

In particular, for two mutually exclusive events A1 , A2

P (A1 ∪ A2 ) = P (A1 ) + P (A2 ) (4)

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 17 / 25

Some Important Theorems on Probability

From the above axioms we can now prove various theorems on probability
that are important in further work.
Theorem 1.1
If A1 ⊂ A2 , then P (A1 ) 5 P (A2 ) and P (A2 − A1 ) = P (A2 ) − P (A1 ).

Theorem 1.2
For every event A,

0 5 P (A) 5 1, (5)

i.e., a probability is between 0 and 1.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 18 / 25

Some Important Theorems on Probability

Theorem 1.3

P (∅) = 0 (6)

i.e., the impossible event has probability zero.

Theorem 1.4
If A0 is the complement of A, then

P (A0 ) = 1 − P (A) (7)

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 19 / 25

Some Important Theorems on Probability

Theorem 1.5
If A = A1 ∩ A2 ∩ · · · ∩ An where A1 , A2 , . . . , An are mutually exclusive
events, then

P (A) = P (A1 ) + P (A2 ) + · · · + P (An ) (8)

In particular, if A = S, the sample space, then

P (A1 ) + P (A2 ) + · · · + P (An ) = 1 (9)

Theorem 1.6
If A and B are any two events, then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B) (10)

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 20 / 25

Some Important Theorems on Probability

Theorem 1.6 (cont...)

More generally, if A1 , A2 , A3 are any three events, then

P (A1 ∪ A2 ∪ A3 ) = P (A1 ) + P (A2 ) + P (A3 )

− P (A1 ∩ A2 ) − P (A2 ∩ A3 ) − P (A3 ∩ A1 )
+ P (A1 ∩ A2 ∩ A3 ) (11)

Generalizations to n events can also be made.

Theorem 1.7
For any events A and B,

P (A) = P (A ∩ B) + P (A ∩ B 0 ) (12)

Generalizations to n events can also be made.

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 21 / 25

Some Important Theorems on Probability

Theorem 1.8
If an event A must result in the occurrence of one of the mutually
exclusive events A1 , A2 , . . . , An , then

P (A) = P (A ∩ A1 ) + P (A ∩ A2 ) + · · · + P (A ∩ An ) (13)

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 22 / 25

Assignment of Probabilities
If a sample space S consists of a finite number of outcomes a1 , a2 , . . . , an ,
then by Theorem 1.5,

P (A1 ) + P (A2 ) + · · · + P (An ) = 1 (14)

where A1 , A2 , . . . , An are elementary events given by Ai = {ai }. It follows

that we can arbitrarily choose any nonnegative numbers for the probabilities
of these simple events as long as (14) is satisfied. In particular, if we assume
equal probabilities for all simple events, then
1
P (Ak ) = , k = 1, 2, . . . , n (15)
n
and if A is any event made up of h such simple events, we have
h
P (A) = (16)
n
This is equivalent to the classical approach to probability.
Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 23 / 25
Assignment of Probabilities

Example 1.11
A single die is tossed once. Find the probability of a 2 or 5 turning up.

The sample space is S = {1, 2, 3, 4, 5, 6}. If we assign equal probabilities to

the sample points, i.e., if we assume that the die is fair, then
1
P (1) = P (2) = · · · = P (6) = (17)
6
The event that either 2 or 5 turns up is indicated by 2 ∪ 5. Therefore,
1 1 1
P (2 ∪ 5) = P (2) + P (5) = + = (18)
6 6 3

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 24 / 25

Thank you for your kind attention!

Dr. M. M. Bhatti (SDUST) Chapter 1: Lecture # 1 and 2 March 9, 2023 25 / 25