ENGR

ENGR 371
371 –– Summer
Summer 204
2024

Probability and Statistics for Engineers

Chapter 1
Instructor: Dr. Amar Sabih, PEng
.
Introduction
Course number: ENGR 371
Course Title: Probability and Statistics in Engineering
Term: Summer 2-2024

Course Instructor: Dr. Amar Sabih, PEng.

E-Mail : [email protected]
Office Hours By appointment- in person or via Zoom
Lecture Time: Tuesday and Thursday 10:15AM - 1:00PM
Location: FG C-070

.
 Course Description

 This is an introductory course in probability and statistics.

 It aims at teaching engineering students the fundamentals of the
probability and statistics theory with applications to various engineering
disciplines.
 Many examples related to real life engineering (probabilistic) problems
will be addressed.
 Probability theory, randomness, conditional probability, joint probability,
independence and probability distributions are covered.
 Data collection, sampling, confidence intervals, hypothesis formulation,
errors, estimation topics are given.

3
.
 Learning Objectives

 To provide the student with knowledge of probability and

statistics focusing on engineering applications.
 To introduce fundamental and practical statistical tools
and process improvement methods as they are used in
different application areas of manufacturing, services
and other sectors.
 For student to understand the importance of data and
statistics related to industrial, manufacturing and
general engineering.

4
.
Textbook
Text Book: Applied Statistics and Probability for Engineers, Montgomery and Runger, 6th
Edition, Wiley
(Newer editions are also accepted)
References: Probability and Statistics for Engineers, Miller and Freund, 9th Edition, Pearson

5
.
Evaluation Scheme

Whichever is better. Your grade will be automatically calculated under both schemes and
the higher one will be selected. No action on your part is needed.

QUIZZES: Four quizzes will be given and the best three will be counted for 15% of your
grade. The questions on the quizzes will be related to the suggested problems.

TERM PROJECT: The project will be a team project with 5 students per group. This will
count for 15% of your grade. See the project document for more details.

EXAMS: One midterm and one final exam will be given. All exams will be closed book
exam.

6
.
Syllabus
W Week Topics (Sections in 7th Ed.) Tests
1 SECTIONS TOPICS
Jul 2 2.1-2.5 Introduction, Sample Spaces, Events, Counting Techniques, Axioms of
Probability, Addition Rules, Conditional Probability
2 Jul 4 2.6-2.9 Multiplication and Total Probability Rules, Independence, Bayes’ Theorem,
Random Variables
Jul 9 3.1-3.5 Discrete Random Variables, Probability Distribution and Probability Mass Quiz 1
Functions, Cumulative Distribution Functions, Mean and variance of a Discrete
Random Variable, Discrete Uniform Distribution, Binomial Distribution
3 Jul 11 3.6-3.8 Geometric Distribution, Negative Binomial Distribution, Hypergeometric
Distribution, Poisson Distribution
4.1-4.2 Continuous Random Variables, Continuous Distribution Functions, Cumulative
Distribution Functions
Jul 16 4.3-4.7 Mean and Variance of a Continuous Random Variable, Continuous Uniform Quiz 2
Distribution, Normal Distribution, Normal Distribution, Normal Approximation for
Binomial and Poisson Distributions, Exponential Distribution
4 Jul 18 5.1-5.3 Two Random Variables, Joint Probability Distributions, Conditional Probability
Distributions, Independence, more than Two Random Variables,
Jul 23 5.4-5.6 Covariance and Correlation, Multinomial Distribution, Bivariate Normal
Distribution, Linear Functions of Random Variables
Jul 25 Midterm exam
5 Jul 30 6.1, 6.7 Numerical summaries of data, Probability plots Quiz 3
7.1-7.3 Point Estimation, Sampling Distributions, Central Limit Theorem, Unbiased
Estimators, Variance of a Point Estimator, Standard Error, Mean Squared Error
Aug 1 8.1 - 8.3, Confidence Intervals on the Mean of a Normal Distribution with Variance Known
8.5, 8.7 and Unknown, Confidence Intervals on the Variance and on the Standard
Deviation, Guidelines for Confidence Intervals, Tolerance, and prediction
intervals
6 Aug 6 9.1, 9.2 Hypothesis Testing, Quiz 4
Tests on the Mean of a Normal Distribution with Known Variance
Aug 8 9.2, 9.3, 9.4 Tests on the Mean of a Normal Distribution with Unknown Variance, Tests on the
Variance and Standard Deviation of a Normal Distribution
7
.
Syllabus

8
.
 The Role of Statistics
in Engineering
CHAPTER OUTLINE
1-1 The Engineering Method 1-3 Mechanistic & Empirical
and Statistical Thinking Models
1-2 Collecting Engineering 1-4 Probability & Probability
Data Models
1-2.1 Basic Principles
1-2.2 Retrospective Study
1-2.3 Observational Study
1-2.4 Designed Experiments
1-2.5 Observed Processes
Over Time
Chapter 1 Title and Outline
9
.
 Learning Objectives for Chapter 1
After careful study of this chapter, you should be able to do
the following:
1. Identify the role of statistics in engineering problem-solving
process.
2. How variability affects the data collected and used for engineering
decisions.
3. Differentiate between enumerative and analytical studies.
4. Discuss the different methods that engineers use to collect data.
5. Identify the advantages that designed experiments have in
comparison to the other methods of collecting engineering data.
6. Explain the differences between mechanistic models & empirical
models.
7. Discuss how probability and probability models are used in
engineering and science.

Chapter 1 Learning Objectives

10

.
 Sec 1-1

Engineers and Statistics

What Engineers Do?
An engineer is someone who solves problems of interest to society with the
efficient application of scientific principles by:
• Refining existing products or processes
• Designing new products or processes
` that meets customers’ needs

The Creative Process

The engineering, or scientific, method is the approach to formulating and solving these
problems. The steps in the engineering method are shown in the following figure:

Steps 2–4 are enclosed in a box, indicating that several cycles or iterations of these
steps may be required to obtain the final solution. Consequently, engineers must
know how to efficiently plan experiments, collect data, analyze and interpret the data
and understand how the observed data relate to the model they have proposed for
11
the problem under study. Statistics
.
 Sec 1-1
Statistics Supports The Creative Process
The field of statistics deals with the collection, presentation,
analysis, and use of data to:
• Make decisions
• Solve problems
• Design products and processes
It is the science of data.
• Because many aspects of engineering practice involve working
with data, obviously knowledge of statistics is just as important to
an engineer as are the other engineering sciences.

• Statistical techniques can be powerful aids in designing new

products and systems, improving existing designs, and designing,
developing, and improving production processes.

12

.
 Sec 1-1

Variability
• Statistical techniques are useful to describe and understand
variability.
• By variability, we mean successive observations of a system
or phenomenon do not produce exactly the same result.
• Statistics gives us a framework for describing this variability
and for learning about potential sources of variability in a
system.
• Statistics indicate which of these sources are the most
important or which have the greatest impact in the system
• Incorporating variability into decision-making processes is
called statistical thinking.
• Example:gasoline mileage performance of your car, city
versus highway, vehicle’s condition, etc.
13

.
 An Engineering Example of Variability Sec 1-1

• A nylon connector to be used in an automotive engine application. Eight

prototype units are produced, and their pull-off forces are measured (in
pounds): 12.6, 12.9, 13.4, 12.3, 13.6, 13.5, 12.6, 13.1.
• All of the prototypes does not have the same pull-off force. We can see the
variability in the above measurements as they exhibit variability.
• The dot diagram is a very useful plot for displaying a small body of data -
say up to about 20 observations.
• This plot allows us to see easily two features of the data; the location, or the
middle, and the scatter or variability.
• Since pull-off force varies or exhibits variability, it is a random variable.
• A random variable, X, can be model by
X=µ+ε
where µ is a constant and ε a random disturbance.

Average 13 lb

14

Figure 1-2 Dot diagram of. the pull-off force data.

 An Engineering Example of Variability Sec 1-1

• A thicker wall, 1⁄8 inch in thickness was considered as an alternative design

• Both samples are plotted as dot diagrams in Figure 1.3

There are some obvious questions to ask:

• How do we know that another sample of prototypes will not give different
results?
• Is a sample of eight prototypes adequate to give reliable results?
• If we use the test results obtained so far to conclude that increasing the wall
thickness increases the strength, what risks are associated with this
decision?

Average 13 lb
Average 13.4 lb

15
Figure 1-3 Dot diagram of pull-off force for two wall thicknesses
.
 Populations and Samples Sec 1-1

Another question: Is it possible to know the effect of the connector

thickness on the performance? How to know that?

• This can be done by testing a selected sample

(such as the eight connectors) from a population
(such as the connectors that will be in the products
that are sold to customers
• The analysis is referred to as statistical inference
• Usually, measurements are obtained from a
sample and generalized to a population
• Clearly, analysis based on measurements from a
sample to measurements on all objects can result
in errors (called sampling errors).
• However, if the sample is selected properly, Figure 1-4 Statistical inference
sampling errors risks can be quantified and an is one type of reasoning.
appropriate sample size can be determined
16

.
 Basic Methods of Collecting Data
Three basic methods of collecting data:
– A retrospective study using historical data
• Data collected in the past for other purposes. Or collect data for a
process with time (concentration of acetone an hourly test sample
of output product)
– An observational study
• Data, presently collected, by a passive observer.
– A designed experiment
• Data collected in response to process input changes intentionally.

Also, it is possible to consider the following method:

‒ Process over time
• Control charts etc.

Sec 1-2.1 Collecting Engineering Data 17

.
 Designed Experiments
Hypothesis Test
• A statement about a process behavior value.
• Compared to a claim about another process value.
• Data is gathered to support or refuse the claim.
One-sample hypothesis test:
Example: mean pull-off force of a 3⁄32-inch design= 12.75 lb
vs
mean pull-off force of a 3⁄32-inch design< 12.75 lb
Two-sample hypothesis test:
• Example: Null hypothesis: H0 : μ1 (3/32 ) − μ2 ( 1/8)= 0,
vs
• Alternative hypothesis: H1: μ1 (3/32 ) > μ2 ( 1/8).

•Chapter 9 presents techniques for this type of problem

Figure 1-3 Dot diagram of pull-off force for two wall thicknesses 18

.
Sec 1-2.4 Designed Experiments
 Designed Experiments
• Designed experiments offer a very powerful approach to
studying complex systems, such as the distillation
column.
• The output of this process is acetone concentration that
is varies with three input factors:
• Inputs (factors) are:
1. Reboil temperature
2. Condensate temperature
3. Reflux rate

• Output changes as the inputs are changed by

experimenter

• An experiment design which uses every possible

combination of the factor levels to form a basic
experiment with “k” different settings for the process.
• Generally, if there are k factors and each has two levels,
a factorial experimental design will require 2k runs.

. 19
 Designed Experiments
An experiment design which uses every possible combination
of the factor levels to form a basic experiment with “k”
different settings for the process. This type of experiment is
called a factorial experiment.
Example:

Consider a petroleum distillation column:

• Output is acetone concentration

• Inputs (factors) are:
1. Reboil temperature
2. Condensate temperature
3. Reflux rate

• Output changes as the inputs are

changed by experimenter.
20
Sec 1.2.4 Designed Experiments .
 Designed Experiments
• Designed experiments offer a very powerful approach to studying complex systems,
such as the distillation column. This process has three factors (the two temperatures
and the reflux rate) and we want to investigate the effect of these three factors on
output acetone concentration.
• Suppose that we use two levels, “high” and “low” (denoted +1 and −1, respectively)
• For each of the three factors. A very reasonable experiment design strategy uses
every possible combination of the factor levels to form a basic experiment with eight
different settings for the process(a factorial experimental design will require 2k runs,
k = 3, the 23 design requires 8 tests, as in Table 1.1). This type of experiment is
called a factorial experiment.
• An important advantage of factorial experiments is that they allow one to detect an
interaction between factors.

. 21
 Designed Experiments
• A & C : response concentration is poor when the reboil temperature is low,
regardless of the condensate temperature. Thus, the condensate
temperature has no effect when the reboil temperature is low
• D & B: when the reboil temperature is high, a high condensate temperature
generates a good response, but a low condensate temperature generates
a poor response
• That is, the condensate
temperature changes the
response when the reboil
temperature is high, i.e.,the
effect of condensate
temperature depends on the
setting of the reboil
temperature, and these two
factors are said to interact in
this case.
• If the D combinations of high and low reboil and condensate
temperatures were not tested, such an interaction would not be detected.
. 22
 An Experiment in Variation

W. Edwards Deming, a famous industrial statistician & contributor to the Japanese

quality revolution, conducted an illustrative experiment on process over-control or
tampering.
Let’s look at his apparatus and experimental procedure.
Marbles were dropped through a funnel onto a target and the location where the
marble struck the target was recorded.
Variation was caused by several factors:
Marble placement in funnel & release dynamics, vibration, air currents,
measurement errors.

Figure 1-10 Deming’s

Funnel experiment

Sec 1-2.5 Observing Processes Over Time 23

.
 Deming’s Experimental Procedure
• The funnel was aligned with the center of the
target. Marbles were dropped. The distance
from the strike point to the target center was
measured and recorded.

• Strategy 1: The funnel was not moved. Then

the process was repeated.

• Strategy 2: The funnel was moved an equal

distance in the opposite direction to
compensate for the error. He continued to
make this type of adjustment after each marble
was dropped. Then the process was repeated.

Sec 1-2.5 Observing Processes Over Time 24

.
 Deming’s Experimental Procedure
• When both strategies were completed, Deming noticed the
variability of the distance from the target for strategy 2 was
approximately twice as large than for strategy 1.
• The deviations from the target is increased due to the
adjustments to the funnel . Hence, adjustments to the funnel
do not decrease future errors. Instead, they tend to move the
funnel farther from the target.
• This experiment explains that the adjustments to a process
based on random disturbances can actually increase the
variation of the process. This is referred to as overcontrol or
tampering.

Sec 1-2.5 Observing Processes Over Time 25

.
 Conclusions from the Deming Experiment

• The lesson of the Deming experiment is that a process

should not be adjusted in response to random
variation, but only when a clear shift in the process
value becomes apparent.

• Then a process adjustment should be made to return the

process outputs to their normal values.

• To identify when the shift occurs, a control chart is

used. Output values, plotted over time along with the
outer limits of normal variation, pinpoint when the
process leaves normal values and should be adjusted.

Sec 1-2.5 Observing Processes Over Time 26

.
 How Is the Change Detected?
 A control chart is used to detect changes.
Its characteristics are:
• Time-oriented horizontal axis, e.g.,
hours.
• Variable-of-interest vertical axis, e.g., %
acetone.
 Long-term average is plotted as the Figure 1-8 The dot diagram illustrates variation
center-line. but does not identify the problem.
 Long-term usual variability is plotted
as an upper and lower control limit
around the long-term average.

 A sample of size n is taken and the

averages are plotted over time. If the
plotted points are between the control
limits, then the process is normal; if
20 samples x 91.5 g l)
not, it needs to be adjusted.

Figure 1-9 A time series plot of concentration

provides more information than the dot diagram
. 27
 How Is the Change Detected Graphically?
 The center line on the control chart is
just the average of the concentration
measurements 𝑥̅ for the first 20
samples

 When the process is stable, the upper

control limit and the lower control limit
are located 3 standard deviations of
the concentration values above and
below the center line.
 The control limits indicates that some
upset or disturbance has affected the
Figure 1-13 A control chart for the
process around sample 20 because all
chemical process concentration data.
of the following observations are below
Process steps out at hour 24 & 29.
the center line, and two of them actually
Shut down & adjust process.
fall below the lower control limit. This is
a very strong signal that corrective
action is required in this process.
28

. Sec 1-2.5 Observing Processes Over Time

 Use of Control Charts
Deming contrasted two purposes of control charts:
1. Enumerative studies:
Sometimes we collect data from a process to evaluate current production.
For example, we might sample and measure resistivity on three semiconductor
wafers selected from a lot and create a control chart and use it to for lot-by-lot
acceptance sampling.
2. Analytic studies:
When we use data from current production to evaluate future production. We
apply conclusions to a conceptual, future population (Real-time control of a
production process)
The use of control charts is a very
important application of statistics for
monitoring, controlling, and
improving a process.
The branch of statistics that makes
use of control charts is called
statistical process control, or SPC

29
FIGURE 1.14 Enumerative versus analytic study
.
Sec 1-2.5 Observing Processes Over Time
 Mechanistic and Empirical Models
A mechanistic model is built from our underlying knowledge of the
basic physical mechanism that relates several variables.
Example: Ohm’s Law (flow of current in a thin copper wire)
Current = Voltage/Resistance
or I = E/R
I = E/R + 
• where  is a term added to the model to account for the fact that the
observed values of current flow do not perfectly conform to the
mechanistic model.
• This can be caused by changes in ambient temperature, fluctuations in
performance of the gauge, small impurities present at different
locations in the wire, and drifts in the voltage source.
• We can think of  as a term that includes the effects of all unmodeled
sources of variability that affect this system.
• In the mechanistic model, the form of the function is known. 30

Sec 1-3 Mechanistic & Empirical Models .

 Mechanistic and Empirical Models

 An empirical model is built from our engineering and

scientific knowledge of the phenomenon but is not
directly developed from our theoretical or first-
principles understanding of the underlying mechanism.
 The form of the function is not known a priori.

Sec 1-3 Mechanistic & Empirical Models 31

.
 An Example of an Empirical Model

• In a semiconductor manufacturing plant, the finished

semiconductor is wire-bonded to a frame.
• In an observational study, the variables recorded were:
• Pull strength to break the bond (y)
• Wire length (x1)
• Die height (x2)

• The data recorded are shown on the next slide.

Sec 1-3 Mechanistic & Empirical Models 32

.
 An Example of an Empirical Model
Table 1-2 Wire Bond Pull Strength Data

Sec 1-3 Mechanistic & 33

Empirical Models
.
 An Example of an Empirical Model

where the “hat,” or circumflex, over pull strength indicates

that this is an estimated or predicted quality.

Applying this technique to the data in Table 1.2 results in

the following estimated regression relationship:

(1.7)
Sec 1-3 Mechanistic & Empirical Models

. 34
 An Example of an Empirical Model
Visualizing the Recorded Data
From examination of this plot, we see that pull strength increases as both
wire length and die height increase.

Figure 1-15 Three-dimensional plot of the pull strength (y), wire length (x1) and
die height (x2) data.
35
Sec 1-3 Mechanistic & Empirical Models .
 An Example of an Empirical Model
Visualizing the Resultant Model Using Regression Analysis
• Notice that the predicted values lie on a plane above the wire length–die height
space.
• From the plot of the data in Figure 1.15, this model does not appear unreasonable.
• The empirical model in Equation 1.7 could be used to predict values of pull
strength for various combinations of wire length and die height that are of interest.
• Essentially, an engineer could use the empirical model in exactly the same way as a
mechanistic model.

Figure 1-16 Plot of the predicted values (a plane) of pull strength

from the empirical regression model (Equation 1.7). 36

Sec 1-3 Mechanistic & Empirical Models .

 Models Can Also Reflect Uncertainty
• Probability models help quantify the risks involved in statistical
inference, that is, risks involved in decisions made every day.
• Probability provides the framework for the study and application of
statistics.
• Probability concepts will be introduced in Chapter 2.

Sec 1-4 Probability & Probability Models 37