Advanced Statistical Approaches to Quality

A. Ben Hamza

C ONCORDIA I NSTITUTE FOR I NFORMATION S YSTEMS E NGINEERING

Concordia University
2018
 C ONTENTS

T ITLE PAGE i

C ONTENTS ii

6 C HAPTER 6

I NTRODUCTION TO S TATISTICAL Q UALITY C ONTROL 1

1.1 W HAT IS Q UALITY C ONTROL ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 W HAT IS S TATISTICAL Q UALITY C ONTROL ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 A B RIEF H ISTORY OF Q UALITY C ONTROL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 W HAT IS S TATISTICAL P ROCESS C ONTROL ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.5 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

6 C HAPTER 6

D ESCRIPTIVE S TATISTICS 4
2.1 D ESCRIPTIVE S TATISTICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1.1 M EASURES OF C ENTRAL T ENDENCY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1.2 M EASURES OF S PREAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.3 G RAPHICAL R EPRESENTATION OF D ATA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 P ROBLEMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

6 C HAPTER 6

R ANDOM VARIABLES AND P ROBABILITY D ISTRIBUTIONS 18

3.1 P ROBABILITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.1.1 S AMPLE S PACE AND E VENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.1.2 A XIOMS OF P ROBABILITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 R ANDOM VARIABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3 D ISCRETE P ROBABILITY D ISTRIBUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3.1 B INOMIAL D ISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3.2 P OISSON D ISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.4 C ONTINUOUS P ROBABILITY D ISTRIBUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4.1 U NIFORM D ISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.4.2 N ORMAL D ISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.3 N ORMAL A PPROXIMATION TO THE B INOMIAL D ISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . 39
3.4.4 E XPONENTIAL D ISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.4.5 C HI -S QUARED D ISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.4.6 S TUDENT ’ S t- DISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.4.7 F- DISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.5 I NFERENTIAL S TATISTICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.5.1 R ANDOM S AMPLING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.5.2 C ENTRAL L IMIT T HEOREM FOR S AMPLE M EANS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

ii
 3.5.3 C ENTRAL L IMIT T HEOREM FOR S AMPLE P ROPORTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.6 M ULTIVARIATE D ISTRIBUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.6.1 C ONCEPTS FROM M ATRIX A LGEBRA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.6.2 B IVARIATE D ISTRIBUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.6.3 M ULTIVARIATE N ORMAL D ISTRIBUTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.7 P ROBLEMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.8 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

6 C HAPTER 6

C ONFIDENCE I NTERVALS AND H YPOTHESIS T ESTING 55

4.1 P OINT AND I NTERVAL E STIMATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.2 S AMPLING D ISTRIBUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.3 C ONFIDENCE I NTERVALS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4.3.1 C ONFIDENCE I NTERVAL ON THE P OPULATION M EAN WHEN THE VARIANCE IS K NOWN . . . . . . . . . . 59
4.3.2 C ONFIDENCE I NTERVAL ON THE P OPULATION M EAN WHEN THE VARIANCE IS U NKNOWN . . . . . . . . . 61
4.3.3 C ONFIDENCE I NTERVAL ON THE P OPULATION VARIANCE . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.3.4 C ONFIDENCE I NTERVAL ON THE P OPULATION P ROPORTION . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3.5 C ONFIDENCE I NTERVAL ON THE D IFFERENCE IN P OPULATIONS M EANS . . . . . . . . . . . . . . . . . . . 67
4.3.6 C ONFIDENCE I NTERVAL ON THE R ATIO OF VARIANCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.4 H YPOTHESIS T ESTING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.4.1 T ESTS ON THE M EAN OF A N ORMAL P OPULATION WITH K NOWN VARIANCE . . . . . . . . . . . . . . . . 73
4.4.2 T ESTS ON THE M EAN OF A N ORMAL P OPULATION WITH U NKNOWN VARIANCE . . . . . . . . . . . . . . 76
4.4.3 T ESTS ON THE VARIANCE OF A N ORMAL P OPULATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
4.4.4 T ESTS ON A P ROPORTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.4.5 T ESTS ON THE D IFFERENCE IN M EANS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.4.6 T ESTS ON THE E QUALITY OF VARIANCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.5 P ROBLEMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.6 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

6 C HAPTER 6

S TATISTICAL P ROCESS AND Q UALITY C ONTROL 91

5.1 S TATISTICAL P ROCESS C ONTROL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.1.1 H YPOTHESIS T ESTING AND C ONTROL C HARTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.1.2 R ULES FOR D ETERMINING O UT-O F -C ONTROL P OINTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2 C ONTROL C HARTS FOR VARIABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.2.1 C ONTROL C HARTS FOR THE M EAN AND R ANGE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.2.2 C ONTROL C HARTS FOR THE M EAN AND S TANDARD D EVIATION . . . . . . . . . . . . . . . . . . . . . . . 97
5.3 P ROCESS C APABILITY A NALYSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
5.4 I NDIVIDUAL C ONTROL C HARTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.5 C UMULATIVE S UM C ONTROL C HART (CUSUM- CHART ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.6 E XPONENTIALLY W EIGHTED M OVING AVERAGE C ONTROL C HART (EWMA- CHART ) . . . . . . . . . . . . . . . . 113
5.7 C ONTROL C HARTS FOR ATTRIBUTES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.7.1 C ONTROL C HART FOR P ROPORTION D EFECTIVE : p- CHART . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.7.2 C ONTROL C HART FOR N UMBER OF D EFECTIVES : np- CHART . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.7.3 C ONTROL C HART FOR C OUNT OF D EFECTS : c- CHART . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.7.4 C ONTROL C HART FOR D EFECTS PER U NIT (u- CHART ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.8 A CCEPTANCE S AMPLING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.8.1 S AMPLING P LAN C RITERIA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.8.2 O PERATING C HARACTERISTIC (OC) C URVES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.8.3 AVERAGE O UTGOING Q UALITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.8.4 AVERAGE T OTAL I NSPECTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.9 M ULTIVARIATE C ONTROL C HARTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.9.1 χ2 AND H OTELLING T 2 C ONTROL C HART . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.10 P RINCIPAL C OMPONENTS A NALYSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.10.1 PCA A LGORITHM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.10.2 PCA T HEORY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
5.10.3 S CREE P LOT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

iii
 5.10.4 B IPLOT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.10.5 PCA C ONTROL C HART . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
5.11 P ROBLEMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
5.12 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

6 C HAPTER 6

R EGRESSION AND A NALYSIS OF VARIANCE 154

6.1 L INEAR R EGRESSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
6.1.1 R ESIDUAL A NALYSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
6.1.2 C ONFIDENCE I NTERVALS FOR THE I NTERCEPT β0 AND S LOPE β1 . . . . . . . . . . . . . . . . . . . . . . . 161
6.1.3 H YPOTHESIS T ESTING ABOUT THE S LOPE β1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
6.1.4 A NALYSIS OF VARIANCE P ROCEDURE FOR T ESTING S IGNIFICANCE OF R EGRESSION . . . . . . . . . . . . . 163
6.1.5 C ORRELATION AND D ETERMINATION C OEFFICIENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
6.1.6 R EGRESSION A NALYSIS : A C OMPLETE E XAMPLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
6.2 M ULTIPLE R EGRESSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.2.1 S CATTERPLOT M ATRIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.2.2 R ESIDUAL A NALYSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
6.2.3 H YPOTHESIS T ESTS AND A NALYSIS OF VARIANCE IN M ULTIPLE L INEAR R EGRESSION . . . . . . . . . . . . 172
6.3 A NALYSIS OF VARIANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.3.1 O NE -WAY ANOVA: C OMPLETELY R ANDOMIZED D ESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.3.2 M ODEL FOR O NE -WAY ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.3.3 R ESIDUAL A NALYSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.3.4 H YPOTHESES IN O NE -WAY ANOVA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.3.5 T OTAL VARIABILITY D ECOMPOSITION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.3.6 E XPECTED VALUES OF THE VARIABILITIES AND T ESTING E QUALITY OF M EANS . . . . . . . . . . . . . . . 177
6.4 P ROBLEMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
6.5 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

6 C HAPTER 6

D ESIGN OF E XPERIMENTS 189

7.1 FACTORIAL E XPERIMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
7.1.1 H YPOTHESES IN T WO -WAY ANOVA AND T OTAL VARIABILITY D ECOMPOSITION . . . . . . . . . . . . . . 191
7.1.2 R ESIDUAL A NALYSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
7.1.3 T WO -WAY ANOVA WITH O NE R EPLICATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
7.2 H IGH -FACTOR E XPERIMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
7.3 2k FACTORIAL D ESIGNS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.3.1 22 FACTORIAL D ESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.3.2 A NALYSIS OF VARIANCE FOR 2k D ESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.3.3 S TATISTICAL I NFERENCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
7.3.4 R ESIDUAL A NALYSIS FOR FOR 22 DESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
7.3.5 23 FACTORIAL D ESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
7.3.6 R ESIDUAL A NALYSIS FOR 23 D ESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
7.4 P ROBLEMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
7.5 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

iv
 C HAPTER 1

I NTRODUCTION TO S TATISTICAL Q UALITY C ONTROL

Quality means doing it right when no one is

looking.

Henry Ford

1.1 W HAT IS Q UALITY C ONTROL ?

Quality can mean different things to different people. Quality is often used to signify excellence of a manufactured
product or a service received. From the manufacturing standpoint quality is simply conformance to specifications.
Quality is also defined as meeting the requirements of the customer. Exceeding customer expectations has been ex-
tremely effective in building a loyal customer base. It is estimated to be five to seven times more costly to attain a new
customer that it is to retain a current one, so it makes a lot of sense to go that extra step [3].
There are various definitions attributed to historical leaders in the field of quality. Juran described quality as “fitness
for use”.
Quality control is a process employed to ensure a certain level of quality in a product or service, which may include
whatever actions a business deems necessary to provide for the control and verification of certain characteristics of a
product or service. The basic goal of quality control is to ensure that the products, services, or processes provided meet
specific requirements and are dependable, satisfactory, and fiscally sound.
Essentially, quality control involves the examination of a product, service, or process for certain minimum levels of
quality. The goal of a quality control team is to identify products or services that do not meet a companys specified
standards of quality. If a problem is identified, the job of a quality control team or professional may involve stopping
production temporarily. Depending on the particular service or product, as well as the type of problem identified,
production or implementation may not cease entirely.
Usually, it is not the job of a quality control team or professional to correct quality issues. Typically, other individ-
uals are involved in the process of discovering the cause of quality issues and fixing them. Once such problems are
overcome, the product, service, or process continues production or implementation as usual.

1.2 W HAT IS S TATISTICAL Q UALITY C ONTROL ?

Statistical quality control (SQC) is a term used to describe the activities associated with ensuring that goods and services
satisfy customer needs. SQC uses statistical analysis based on measurements taken from a process or from a sample
of products or services, to make decisions regarding the quality of goods and services. The statistical methods of SQC
may be divided into two main categories: Statistical process control (SPC) and acceptance sampling. SPC refers to the
use of statistical methods to measure and control the performance of a process to ensure that the output meets customer
needs. Acceptance sampling is a methodology of taking samples from lots of materials or products and inspecting the
items to determine if the items meet customer requirements. SPC may be used to to help control almost all processes
that can measured or monitored to ensure that the process performs within limits.
Deming, a statistician who gained fame by helping Japanese companies improve quality after the World War II,
believed that quality and productivity increase as variability decreases and, because all things vary, statistical methods

1
of quality control must be used to measure and gain understanding of the causes of the variation. Many companies,
particularly those in the auto industry, have adopted Deming’s philosophy and approach to quality.
The causes of variations in a product quality characteristic may be broadly classified into two main categories: com-
mon causes of variation (variation due to the system itself) and special causes of variation (variation due to factors
external to the system).
• Chance or common causes: when only these causes of variations are present in a process, the process is considered
to be stable or in-control. Examples of such causes include atmospheric pressure or temperature changes in
the production area, worker fatigue, and fluctuations caused by hiring, training, and supervisory policies and
practices. Common causes of variation are the responsibility of management.
• Assignable or special causes: when these causes of variations are present in a process, variation will be excessive
and the process is considered to be unstable or out-of-control. Examples of such causes include tampering or
unnecessary adjusting the process when it is inherently stable, using a wrong tool or an incorrect procedure.
Special causes of variation are the responsibility of workers and engineers.

1.3 A B RIEF H ISTORY OF Q UALITY C ONTROL

The quality movement can trace its roots back to medieval Europe, where craftsmen began organizing into unions
called guilds in the late 13th century. Until the early 19th century, manufacturing in the industrialized world tended to
follow this craftsmanship model. The factory system, with its emphasis on product inspection, started in Great Britain
in the mid-1750s and grew into the Industrial Revolution in the early 1800s.
In the early 20th century, manufacturers began to include quality processes in quality practices. After the United
States entered World War II, quality became a critical component of the war effort: Bullets manufactured in one state,
for example, had to work consistently in rifles made in another. The armed forces initially inspected virtually every
unit of product; then to simplify and speed up this process without compromising safety, the military began to use
sampling techniques for inspection, aided by the publication of military-specification standards and training courses
in Walter Shewharts statistical process control techniques.
The birth of total quality in the United States came as a direct response to the quality revolution in Japan following
World War II. The Japanese welcomed the input of Americans Joseph M. Juran and W. Edwards Deming and rather than
concentrating on inspection, focused on improving all organizational processes through the people who used them. By
the 1970s, U.S. industrial sectors such as automobiles and electronics had been broadsided by Japans high-quality
competition. The U.S. response, emphasizing not only statistics but approaches that embraced the entire organization,
became known as total quality management (TQM). By the last decade of the 20th century, TQM was considered a fad
by many business leaders. But while the use of the term TQM has faded somewhat, particularly in the United States,
its practices continue.
In the few years since the turn of the century, the quality movement seems to have matured beyond Total Quality.
New quality systems have evolved from the foundations of Deming, Juran and the early Japanese practitioners of
quality, and quality has moved beyond manufacturing into service, healthcare, education and government sectors.

1.4 W HAT IS S TATISTICAL P ROCESS C ONTROL ?

A process is the transformation of a set of inputs, and may include customer services, productions systems, and ad-
ministration activities. In each area or function of an organization there will be many processes taking place. Each
process may be analyzed by an examination of the inputs and outputs. This will determine the action necessary to
improve quality [2]. In order to produce an output which meets the requirements of the customer, it is necessary to
define, monitor and control the inputs to the process.
Statistical process control (SPC) is a procedure in which data is collected, organized, analyzed, and interpreted so
that a process can be maintained at its present level of quality or improved to a higher level of quality. SPC requires that
the process be improved continuously by reducing its variability. SPC refers to a number of different methods for mon-
itoring and assessing the quality of manufactured goods. Combined with methods from the Design of Experiments,
SPC is used in programs that define, measure, analyze, improve, and control development and production processes.
These programs are often implemented using “Design for Six Sigma” methodologies.

2
1.5 REFERENCES

[1] D. C. Montgomery, Introduction to Statistical Quality Control, John Wiley & Sons, 2005.
[2] J. S. Oakland, Statistical Process Control, Butterworth-Heinmann, 2003.
[3] G. M. Smith, Statistical Process Control and Quality Improvement, Prentice Hall, 2003.

3
 C HAPTER 2

D ESCRIPTIVE S TATISTICS

Numerical quantities focus on expected values,

graphical summaries on unexpected values.

John Tukey

Statistics is the science of collecting, organizing, summarizing and analyzing data in order to draw conclusions. Data
are typically collected from a population, which is defined as a collection of units. Then, we define a sample as a subset
of units selected from a population. A central problem in statistics is to obtain information about a population from
a sample. Statistics is often divided into two branches: descriptive statistics and inferential statistics. Descriptive
statistics focus on the collection, analysis, presentation, and description of a set of data, whereas inferential statistics
focus on making decisions about a large set of data (i.e. population) from a subset of the data (i.e. sample). Both
descriptive and inferential statistics can be used to analyze the population and sample data. Numerical descriptive
measures computed from the sample data are often called statistics, while numerical descriptive measures of the pop-
ulation are called parameters. The population parameters are typically unknown. For example, the unknown mean µ
and variance σ2 of the population are usually estimated using sample statistics.

2.1 D ESCRIPTIVE S TATISTICS

One important use of descriptive statistics is to summarize a collection of data in a clear and understandable way. For
example, the manufacturing manager’s job is to present results effectively so that the appropriate action can be taken.
The most powerful tool for expressing the results of any study is the graph. By presenting a graph, the manufacturing
manager need not explain it in words. Graphical methods are better suited than numerical methods for identifying
patterns in the data. It is usually easier for people to grasp the results from a graph than from any other form of analysis.
Numerical approaches are more precise and objective. Since the numerical and graphical approaches compliment each
other, it is wise to use both.
Descriptive statistics are very important, as if we simply presented our raw data it would be hard to visualize what
the data was showing, especially if there was a lot of it. Descriptive statistics therefore allow us to present the data
in a more meaningful way which allows simpler interpretation of the data. Typically, there are two general types of
numerical descriptive measures computed from the sample data, namely measures of central tendency and measures
of spread. Suppose, then, that our data set is of the form x1 , . . . , x n , where each xi is a number.

2.1.1 M EASURES OF C ENTRAL T ENDENCY

The measures of central tendency or location are ways of describing the central position of a frequency distribution for
a group of data. We can describe this central position using a number of statistics, including the mean, median, and
mode. Because we almost always think of our data set as a sample, we will refer to these statistics as the sample mean,
sample median, and sample mode. Hence, we will use the terms sample and data set interchangeably when dealing
with descriptive statistics.
Let x1 , . . . , x n be a sample of n measurements or observations. Denote by x(1) , . . . , x ( n) the ordered sample, sorted in
ascending order, where x(1) are x( n) are the smallest and largest data values, respectively.

4
Definition 2.1
• The sample mean or arithmetic average x̄ is given by
n
1
x̄ =
n ∑ xi (2.1.1)
i =1

• The sample median, denoted by Q2 , is given by


 x ( k +1 ) if n = 2k + 1 (odd number)
Q 2 = x ( k ) + x ( k +1 ) (2.1.2)
 if n = 2k (even number)
2

• The sample mode is the value that occurs most frequently in a data set.

The sample median is the middle value in an ordered list of an odd number of observations. If the number is even, the
median is defined as the average of the two middle values.
It is important to note that when there are multiple values occurring equally frequently in the data set, then the mode
is chosen as the smallest of those values.

Example 2.1 The following data represent the number of defectives observed each day over a 15-day period for a manufacturing
process. Calculate the sample mean, sample median, and sample mode of the defect data.

Day 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Defects 10 10 6 12 6 9 16 20 11 10 11 11 9 12 11

Solution: The sample mean, sample median, and sample mode are given by

x̄ = 10.9333, Q2 = 11, mode = 11.

MATLAB code
>> X = [10 10 6 12 6 9 16 20 11 10 11 11 9 12 11];
>> mean(X)
>> median(X)
>> mode(X)

Python code
>> import numpy as np
>> from scipy.stats import mode
>> X = np.array([10, 10, 6, 12, 6, 9, 16, 20, 11, 10, 11, 11, 9, 12, 11])
>> Xbar = np.mean(X) #sample mean
>> Q2 = np.median(X) #sample median
>> print(’Xbar={0:.2f}, Q2={1:.2f}’.format(Xbar, Q2))
>> M = mode(X) #sample mode
>> print("The mode is {} with a count of {}".format(M.mode[0], M.count[0]))

2.1.2 M EASURES OF S PREAD

The measures of central tendency only locate the center of the data; they do not provide information on how the
data are spread. The measures of spread help us summarize how spread out the data are. To describe this spread, a
number of statistics are available to us, including the range, quartiles, variance and standard deviation. Variation is
very important in quality control because it determines the level of conformance of the production process to the set
standards. For instance, if we are manufacturing tires, an excessive variation in the depth of the treads of the tires
would imply a high rate of defective products.

5
Definition 2.2
• The sample range, R, is given by R = x( n) − x(1)

• The sample variance, s2 , is given by

n
1
s2 = ∑ (xi − x̄)2 (2.1.3)
n−1 i =1
√
• The sample standard deviation is the positive square root of the sample variance. That is, s = s2 .

The range is the simplest of all measures of variability. It is the difference between the highest and the lowest values
of a data set. Sample variances and standard deviations are used as estimators of a populations variance and standard
deviation. Using n − 1 in the denominator instead of n yields a better estimate of the population. It is worth pointing
out that the smaller the sample standard deviation, the closer the data are scattered around the mean. If the sample
standard deviation is zero, then all the data observed are equal to the mean.

Example 2.2 Calculate the sample range, sample variance, and sample standard deviation for the defect data given in Example 2.1.

Solution: The sample range, sample variance, and sample standard deviation for the defect data are

R = 14, s2 = 12.0667, s = 3.4737.

MATLAB code
>> X = [10 10 6 12 6 9 16 20 11 10 11 11 9 12 11];
>> range(X)
>> var(X)
>> std(X)

Python code
>> import numpy as np
>> from scipy.stats import mode
>> X = np.array([10, 10, 6, 12, 6, 9, 16, 20, 11, 10, 11, 11, 9, 12, 11])
>> R = np.ptp(X) #sample range: ptp (peak to peak)
>> print(’R={0:.2f}’.format(R))
>> var = np.var(X, ddof=1) #sample variance
>> s = np.std(X, ddof=1) #sample standard deviation
>> print(’var = {0:.2f}, s = {1:.2f}’.format(var, s))

2.1.3 G RAPHICAL R EPRESENTATION OF D ATA

When we use descriptive statistics it is useful to summarize our group of data using a combination of tabulated descrip-
tion or graphical description. Graphical representations can make data easy to interpret by just looking at graphs. Dot
plot, bar chart, histogram, stem-and-leaf graph, Pareto chart, box plot, and scatter plots are types of graphs commonly
used in statistics.

Dot plot:
A dot plot is a plot that displays a dot or a point for each value in a data set along a number line. If there are multiple
occurrences of a specific value, then the dots will be stacked vertically. The frequency or the frequency count for a data
value is the number of times the value occurs in the data set. A dot plot gives us, for example, information about how
far the data are scattered and where most of the observations are concentrated.

Example 2.3 The following data gives the number of defective motors received in 20 shipments

8 12 10 16 6 25 21 15 17 5 26 21 29 8 10 21 10 17 15 13

Construct the dot plot for this data.

6
 0 5 10 15 20 25 30

F IGURE 2.1: Dot plot for the data on defective motors that are received in 20 ship-
ments.

Solution: Figure 2.1 shows the dot plot for the on on defective motors. We see that 70% of the time, the number of
defective motors was between 8 and 21.

Example 2.4 Construct the dot plot for the data given in Example 2.1.

Solution: Figure 2.2 shows the dot plot for the defective data set. Observe that since there are multiple occurrences of
specific observations, the dots are stacked vertically. The number of dots represents the frequency count for a specific
value. For instance, the value of 10 occurred 3 times since there are 3 dots stacked above the value of 10.

4 6 8 10 12 14 16 18 20 22
F IGURE 2.2: Dot plot for the data on the number of defectives observed each day over
a 15-day period for a manufacturing process.

Bar chart:
A bar chart presents each category of an attribute type variable as a bar whose length is the frequency or percentage
of values falling into a category. Bar charts permit the visual comparison of data by displaying the magnitude of each
category as a vertical (or horizontal) bar.

Example 2.5 Display the bar chart for the data given in Example 2.1.

Solution: Figure 2.3 shows the bar chart for the data on the number of defectives observed each day over a 15-day
period for a manufacturing process.

MATLAB code
>> X = [10 10 6 12 6 9 16 20 11 10 11 11 9 12 11];
>> bar(X,’g’,’EdgeColor’,[1 0.5 0.5]);
>> xlabel(’Days’,’fontsize’,14,’fontname’,’times’);
>> ylabel(’Defects’,’fontsize’,14,’fontname’,’times’);

Python code
>> import numpy as np
>> import matplotlib.pyplot as plt
>> import seaborn as sns; sns.set()
>> X = np.array([10, 10, 6, 12, 6, 9, 16, 20, 11, 10, 11, 11, 9, 12, 11])
>> plt.bar(np.arange(len(X)), X)
>> plt.xlabel(’Days’)
>> plt.ylabel(’Defects’)

7
 20

18

16

14

12

Defects
10

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Days
F IGURE 2.3: Bar chart for the data on the number of defectives observed each day over
a 15-day period for a manufacturing process.

Histogram:
A histogram is a special bar chart for measurement data. Its purpose is to graphically summarize the distribution of
a data set. The construction of a histogram starts with the division of a frequency distribution into equal classes, and
then each class is represented by a vertical bar. A histogram gives an estimate of the shape of the distribution of the
population from which the sample was taken. Histograms and relative-frequency histograms provide effective visual
displays of data organized into frequency tables. In these graphs, we use bars to represent each class, where the width
of the bar is the class width. For histograms, the height of the bar is the class frequency, whereas for relative-frequency
histograms, the height of the bar is the relative frequency of that class. The relative frequency for any class is obtained
by dividing the frequency for that class by the total number of observations.

Frequency for class

Relative frequency = (2.1.4)
Total number of observations

Example 2.6 Construct the histogram for the data given in Example 2.1.
Solution: Figure 2.4 shows the relative frequency histogram for the data on the number of defectives observed each
day over a 15-day period for a manufacturing process.

MATLAB code
>> X = [10 10 6 12 6 9 16 20 11 10 11 11 9 12 11];
>> a = min(X):1:max(X);
>> ncount = histc(X,a);
>> relativefreq = ncount/length(X);
>> bar(a, relativefreq,1)
>> xlabel(’Defects’)
>> ylabel(’Relative frequency’)

Python code
>> import numpy as np
>> import matplotlib.pyplot as plt

8
 >> import seaborn as sns; sns.set()
>> X = np.array([10, 10, 6, 12, 6, 9, 16, 20, 11, 10, 11, 11, 9, 12, 11])
>> plt.hist(X, bins=len(X), density=True)
>> plt.xlabel(’Defects’)
>> plt.ylabel(’Relative frequency’)

0.25

0.2
Relative frequency

0.15

0.1

0.05

0
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Defects

F IGURE 2.4: Histogram for the data on the number of defectives observed each day
over a 15-day period for a manufacturing process.

Stem-and-Leaf Diagram:
A stem-and-leaf plot or diagram is a data plot that uses part of a data value as the stem to form groups or classes and
part of the data value as the leaf: A stem-and-leaf plot has an advantage over a grouped frequency distribution, since
a stem-and-leaf plot retains the actual data by showing them in graphic form. The stem-and-leaf diagram is essentially
composed of two parts: the stem, which is on the left side of the diagram, and the leaf on the right. It resembles a
histogram that has been turned on its side.
The first step in creating a stem-and-leaf diagram is to reorganize the data in ascending (or descending) order.

Example 2.7 Construct the stem-and-leaf diagram for the data given in Example 2.1.

Solution: Figure 2.5 shows the stem-and-leaf diagram for the data on the number of defectives observed each day over
a 15-day period for a manufacturing process. The ordered data are: 6, 6, 9, 9, 10, 10, 10, 11, 11, 11, 11, 12, 12, 16, 20. The
numbers that start with 0 (i.e. single digits) have 6, 6, 9 again, and 9 as the second digits. The numbers that start with
1 have 0, 0, 0, 1, 1, 1, 1, 2, 2, 6 as the second digits. Finally, the only number that starts with 2 has 0 as the second digit.
The stem-and-leaf diagram shows that most of the data are clustered between 10 and 16.
MATLAB code
>> X = [10 10 6 12 6 9 16 20 11 10 11 11 9 12 11];
>> sort(X)
ans =
6 6 9 9 10 10 10 11 11 11 11 12 12 16 20
>> StemLeafPlot(X)

0 | 6 6 9 9

9
 0 | 6 6 9 9
1 | 0 0 0 1 1 1 1 2 2 6
2 | 0

F IGURE 2.5: Stem-and-leaf diagram for the data on the number of defectives observed
each day over a 15-day period for a manufacturing process.

1 | 0 0 0 1 1 1 1 2 2 6
2 | 0

Pareto Chart:
A Pareto chart or diagram is a special type of bar chart in which the categories of an attribute type variable are shown
on the x-axis, the frequencies in each category (listed from largest to smallest frequency) are shown on the left side
y-axis, and the cumulative percentage of frequencies are shown on the right side y-axis. The key idea behind Pareto
chart is to separate the “vital few” from the “trivial many”. A Pareto chart is a type of bar chart in which the horizontal
axis represents categories of interest. When the bars are ordered from largest to smallest in terms of frequency counts
for the categories, a Pareto chart can help you determine which of the categories make up the critical few and which
are the insignificant many. A cumulative percentage line helps you judge the added contribution of each category.

Example 2.8 Construct the Pareto chart for the data given in Example 2.1.
Solution: Figure 2.6 shows the Pareto chart for the data on the number of defectives observed each day over a 15-day
period for a manufacturing process. Observe that the categories have been ordered from the highest frequency to the
lowest frequency.
MATLAB code
>> X = [10 10 6 12 6 9 16 20 11 10 11 11 9 12 11];
>> pareto(X)

160 98%

140 85%

120 73%

100 61%

80 49%

60 37%

40 24%

20 12%

0 0%
8 7 14 4 15 12 11 9 10 2

F IGURE 2.6: Pareto chart for the data on the number of defectives observed each day
over a 15-day period for a manufacturing process.

Box Plot:
The box plot (also called box-and-whisker plot) is an alternative graphical representation method to either the stem-
and-leaf plot or the histogram. Recall that the median Q2 (also called second quartile) divides the lower 50% of a

10
sample from its upper 50%. The first quartile, denoted Q1 , divides the bottom 25% of the sample from the top 75%. The
third quartile, denoted Q3 , divides the bottom 75% of the sample from the top 25%. These 3 quartiles, in conjunction
with the smallest and largest values of the sample, form the so-called five-number summary.
A box plot provides an excellent visual summary of many important aspects of a distribution, and it is a convenient
way of graphically depicting groups of numerical data through their five-number summaries: the smallest data value
(sample minimum x(1) ), first quartile (Q1), median (Q2 ), third quartile (Q3 ), and largest data value (sample maximum
x( n) ). A box plot may also indicate which observations, if any, might be considered outliers (marked with a ’+’ on the
graph), as shown in Figure 2.7. The advantage of using the box plot is that when it is used for multiple variables, not
only does it graphically show the variation between the variables but it also shows the variations within the ranges.
The purpose of a box plot is not only to show how the data are spread but also to make obvious the presence of
outliers, i.e. observations (or measurements) that are unusually large or small relative to the other values in a data set.
Outliers typically are attributable to one of the following causes:

• The measurement is observed, recorded, or entered into the computer incorrectly.

• The measurement comes from a different population.

• The measurement is correct, but represents a rare (chance) event.

To determine the presence of outliers, we first need to find the interquartile range (IQR), which is defined as: IQR =
Q3 − Q1 . The IQR measures the vertical distance of the box, i.e. the distance between the first and third quartiles.
An outlier is defined as any observation away from the closest quartile by more than 1.5(IQR), i.e. any measurement
smaller that the lower inner fence LIF = Q1 − 1.5(IQR) or greater than the upper inner fence U IF = Q3 + 1.5(IQR)
is considered an outlier. Thus, a box plot helps provide information on whether outliers exist in the data. The vertical
dashed lines are referred to as whiskers, and they extend to the most extreme observation inside the inner fences. If
one whisker is longer than the other, then the distribution of the data is probably skewed in the direction of the longer
whisker. As can be seen in Figure 2.7, the largest measurement inside the fence is the second-largest observation.
The box plot can be used to describe the shape of a data distribution by looking at the position of the median line
compared to Q1 and Q3 , the lower and upper ends of the box. If the median is close to the middle of the box, the
distribution is fairly symmetric, providing equal-sized intervals to contain the two middle quarters of the data. If the
median line is to the left of center, the distribution is skewed to the right; if the median is to the right of center, the
distribution is skewed to the left. Also, for most skewed distributions, the whisker on the skewed side of the box tends
to be longer than the whisker on the other side. If the data is a matrix, then there is one box plot per column.

outlier

F IGURE 2.7: Illustration of box plot.

Example 2.9 The following data give the number of persons who take the bus during the off-peak time schedule from Central
Station to South Shore in Montreal:

12 12 12 14 15 16 16 16 16 17 17 18 18 18 19 19 20 20 20 20
20 20 20 20 21 21 21 22 22 23 23 23 24 24 25 26 26 28 28 28

11
 (i) Calculate the mean, median, and mode for the data.
(ii) Determine the five-number summary for the data.
(iii) Construct the box plot for the data.

Solution: From the box plot plot in Figure 2.8, we can see that the data are symmetric and do not contain any outliers.
MATLAB code
>> X = [12 12 12 14 15 16 16 16 16 17 17 18 18 18 19 19 20 20 20 20 ...
20 20 20 20 21 21 21 22 22 23 23 23 24 24 25 26 26 28 28 28];
>> mean(X)
ans =
20
>> median(X)
ans =
20
>> mode(X)
ans =
20
>> fivenumbersummary(X)
ans =
min: 12
max: 28
Q1: 17
Q2: 20
Q3: 23
>> boxplot(X);

28

26

24
Number of persons

22

20

18

16

14

12
1

F IGURE 2.8: Box plot for the data on the number of persons who take the bus during
the off-peak time.

Example 2.10 Compute the five-number summary and construct the box plot for the data given in Example 2.1.
Solution: Figure 2.9 shows the box plot for or the data on the number of defectives observed each day over a 15-day
period for a manufacturing process. The interquartile range is IQR = Q3 − Q1 = 11.75 − 9.25 = 2.5. So any observation
smaller than Q1 − 1.5(IQR) = 5.5 or greater than Q3 + 1.5(IQR) = 15.5 is considered as an outlier. Therefore, the
observations 16 and 20 (i.e. the numbers of defectives observed on days 7 and 8) are outliers. Before removing the
outliers from the data set, we need to make a concerted effort to find the cause of the outliers. An investigation may
discover that these measurements were correctly recorded or they represent defects that correspond to exceptional

12
parts being manufactured. Notice also that the lower whisker is longer that upper one, indicating that the number of
defectives are negatively skewed.
MATLAB code
>> X = [10 10 6 12 6 9 16 20 11 10 11 11 9 12 11];
>> fivenumbersummary(X)
>> boxplot(X);
>> ylabel(’Defects’,’fontsize’,14,’fontname’,’times’);

20

18

16
Defects

14

12

10

6
1

F IGURE 2.9: Box plot for the data on the number of defectives observed each day over
a 15-day period for a manufacturing process.

Example 2.11 The following data represent the temperature of two rooms: one was built with metal door and window frames and
the other one without any metal.

With metal 52 81 83 79 89 89 98 96 98 99 95 99 99 99 101

Without metal 58 62 65 71 59 60 99 60 96 93 87 89 92 85 81

(i) Construct the box plot.

(ii) Is there a difference between the two rooms and the level of temperature variability within the two rooms?

Solution:

(i) Figure 2.10 shows the box plot for the data of the temperature of the rooms.

(ii) The graphs show that there is a large disparity between the two groups, and for the room with metal the heat
level is predominantly below the median. For the room with metal, the observation number 52 is an outlier.
For the room without metal, the temperatures are more evenly distributed, albeit most of the observations are
below the median. Notice that the upper whisker is relatively longer that lower one, indicating that the tempera-
tures are positively skewed.
MATLAB code
>> data = [ 52 81 83 79 89 89 98 96 98 99 95 99 99 99 101
58 62 65 71 59 60 99 60 96 93 87 89 92 85 81];
>> char = {’With metal’,’Without metal’};
>> boxplot(data’,char);

13
 100
95
90
85
80
75
70
65
60
55
50
With metal Without metal

F IGURE 2.10: Box plot for the data on the temperature of the rooms.

Normal Probability Plot:

The normal probability plot is a graphical technique for assessing whether or not a data set is approximately normally
distributed. The data are plotted against a theoretical normal distribution in such a way that the points should form
an approximate straight line. Departures from this straight line indicate departures from normality. The shape in the
normal probability plot can be described as follows:

• For a skewed distribution, we should see a systematic non-linear tendency.

• For a heavy tailed distribution, we should see the data clumping away from the line. This usually results in a
systematic deviation away from the line in a form of an S.

• For a normal distribution, we should expect to see a linear tendency. It can have weak deviations in the tail, but
the overall tendency is linear.

Example 2.12 Suppose that seventeen randomly selected workers at a detergent factory were tested for exposure to a Bacillus
subtillis enzyme by measuring the ratio of forced expiratory volume (FEV) to vital capacity (VC).

0.61 0.70 0.76 0.84 0.63 0.72 0.78 0.85 0.64 0.73 0.82 0.85 0.67 0.74 0.83 0.87 0.88

Is it reasonable to conclude that the FEV to VC (FEV/VC) ratio is normally distributed?

Solution: FEV is the maximum volume of air a person can exhale in one second; VC is the maximum volume of air
that a person can exhale after taking a deep breath. Figure 2.11 shows the normal probability plot for FEV/VC. The
plot has the sample data displayed with the plot symbol ’+’. Superimposed on the plot is a line joining the first and
third quartiles of each column of X (a robust linear fit of the sample order statistics.) This line is extrapolated out to the
ends of the sample to help evaluate the linearity of the data. The tendency appears to be linear, hence it is reasonable
to believe that FEV/VC is normally distributed.

MATLAB code
>> X = [.61 .70 .76 .84 .63 .72 .78 .85 .64 .73 .82 .85 .67 .74 .83 .87 .88];
>> normplot(X);

14
 Normal Probability Plot
0.98

0.95
0.90

0.75

Probability
0.50

0.25

0.10
0.05

0.02
0.65 0.7 0.75 0.8 0.85
Data

F IGURE 2.11: Normal probability plot for FEV/VC.

Scatter Plot:
In simple correlation and regression studies, data are collected on two quantitative variables to determine whether a
relationship exists between the two variables. Let x1 , . . . , x n and y1 , . . . , y n be two samples of size n, and denote by x̄,
s x , ȳ and sy their sample mean standard deviations, respectively. An important first step in studying the relationship
between the variables xi and yi is to graph the data. A scatter plot is a graph in which each plotted dot represents an
observed pair ( x i , yi ) of values (see Figure 2.12). The value of xi is plotted with respect to the horizontal axis, and the
value of yi is plotted with respect to the vertical axis. The scatter plot provides a visual impression of the nature of the
relation between the x and y values in a bivariate data set. The variable along the vertical axis is called the dependent
variable, and the variable along the horizontal axis is called the independent variable. Our visual impression of the
closeness of the scatter to a linear relation can be quantified by calculating the correlation coefficient. Figure 2.12 shows
the correspondence between the appearance of a scatter plot and the value of the correlation coefficient.

Definition 2.3
• The sample covariance is defined as
n
1
s xy = ∑ ( x − x̄ )(yi − ȳ)
n − 1 i =1 i
(2.1.5)

• The sample correlation coefficient is defined as

s xy
r xy = (2.1.6)
s x sy

The sample correlation coefficient measures the strength and direction of a relationship between two variables using
sample data. The range of the correlation coefficient is from -1 to l.

Example 2.13 The following data relate the high temperature (°F) reached on a given day and the number of cans of soft drinks
sold from a particular vending machine in front of a grocery store. Data were collected for 15 different days.

Temperature 70 75 80 90 93 98 72 75 75 80 90 95 98 91 98
Quantity 30 31 40 52 57 59 33 38 32 45 53 56 62 51 58

(i) Calculate the sample covariance and sample correlation coefficient.

(ii) Construct the scatter plot.

15
 y

y
y
x x x

(a) r xy = 0.95 (b) r xy = 0.20 (c) r xy = −0.95

F IGURE 2.12: Correspondence between the values of correlation coefficient and the
amount of scatter.

Solution:

(i) The sample covariance and sample correlation coefficient between temperature and quantity are given by

s xy = 116.2619 and r xy = 0.979.

(ii) Figure 2.13 shows the scatter plot for the soft drinks data. We can observe from the plot that the number of
cans of soft drinks sold increases as the temperature increases, and that there seems to be a linear trend for this
association.
MATLAB code
>> X = [70 75 80 90 93 98 72 75 75 80 90 95 98 91 98];
>> Y = [30 31 40 52 57 59 33 38 32 45 53 56 62 51 58];
>> cov(X,Y)
>> corrcoef(X,Y)
>> scatter(X,Y,’ko’,’MarkerFaceColor’,[.49 1 .63]); grid on;
>> xlabel(’Temperature’,’fontsize’,14,’fontname’,’times’);
>> ylabel(’Number of cans’,’fontsize’,14,’fontname’,’times’);

65

60

55
Number of cans

50

45

40

35

30
70 75 80 85 90 95 100
Temperature

F IGURE 2.13: Scatter plot for the soft drinks data.

16
2.2 P ROBLEMS

❶ An engineer wants to measure the bias in a pH meter. She uses the meter to measure the pH in 13 neutral
substances and obtains the following data:

6.90 7.00 7.03 7.01 6.97 7.00 6.95 7.00 6.99 7.04 6.97 7.07 7.04

a) Calculate the sample range and mean.

b) Calculate the sample variance and standard deviation.
c) Check the assumption of normality for the pH data using normplot and boxplot. Identify any outliers.

❷ An entrepreneur faces many bureaucratic and legal hurdles when starting a new business. The following data
shows the time, in days, to complete all of the procedures required to start a business in 25 developed countries:

23 4 29 67 44 47 24 40 23 116 44 33 27 60 46 61 11 23 62 31 44 77 14 65 42

It is also reported that the overall average time to start a business in all developed countries is 30 days.

a) Construct the dot plot.

b) Construct the stem-and-leaf diagram.
c) Calculate the sample range, mean, and median of the time-to-start data.
d) Calculate the first quartile, second quartile, and the interquartile range.
e) Calculate the sample variance and standard deviation.
f) Check the assumption of normality for the time-to-start data using normplot and boxplot. Identify any
outliers.

2.3 REFERENCES

[1] D.C. Montgomery, Introduction to Statistical Quality Control, Wiley, 6th Edition, 2009.
[2] I. Bass and B. Lawton, Lean Six Sigma using SigmaXL and Minitab, McGraw-Hill Professional, 1st Edition, 2009.

17
 C HAPTER 3

R ANDOM VARIABLES AND P ROBABILITY D ISTRIBUTIONS

Life is a school of probability.

Walter Bagehot

Probability theory is a branch of mathematics that deals with calculating the likelihood of a given event’s occurrence,
which is expressed as a number between zero and one. An event with a probability of zero can be considered an
impossibility, whereas an event with a probability of one can be considered a certainty. The fundamental ingredient of
probability theory is an experiment that can be repeated, at least hypothetically, under essentially identical conditions
and that may lead to different outcomes on different trials. An experiment is the process by which one observation
is obtained. An example of an experiment would be the sorting out of defective parts from a production line. A
probability distribution assigns a probability to each of the possible outcomes of a random experiment. A random
variable is a function that associates a unique numerical value with every outcome of an experiment. The value of the
random variable will vary from trial to trial as the experiment is repeated.

3.1 P ROBABILITY

3.1.1 S AMPLE S PACE AND E VENTS

Definition 3.1
• A random experiment is a procedure or an operation whose outcome is uncertain and cannot be predicted in advance.

• A sample space, denoted by S , is the set of all possible outcomes of a random experiment.
• An event is a subset of the sample space S . That is, an event is an outcome or a collection of outcomes of a random
experiment.

Example 3.1

• A classic example that is used often to illustrate concepts of probability theory, is the experiment of flipping a coin. The
outcomes of this experiment are head and tail. So, the sample space is S = { H, T }, where H represents the event ’head’ and
T represents the event ’tail’.

• Tossing two coins and observing the up face on each is random experiment with sample space S = { HH, HT, TH, TT } of
four events.

• Rolling a single die is a random experiment with sample space S = { , , , , , }. Rolling an even number
( , or ) is an event, and rolling an odd number ( , or ) is also an event.

18
3.1.2 A XIOMS OF P ROBABILITY
Given a random experiment and a sample space S , the objective of probability is to assign to each event A a number
P( A), called the probability of the event A., which will give a precise measure of the chance that A will occur. P( A) is
defined as the ratio of the number of ways event A can occur to the number of possible outcomes:

Number of ways event A can occur

P( A) = · (3.1.1)
Total number of possible outcomes (sample space)

A probability must obey the following rules or axioms:

• Axiom 1: For any event A, 0 ≤ P( A) ≤ 1

• Axiom 2: P(S) = 1
• Axiom 3: If A1 , A2 , . . . is a collection of mutually exclusive events then

P ( A1 ∪ A2 ∪ . . .) = ∑ P ( A i ) .
i

Example 3.2 • What is the probability of each outcome when a loonie is tossed?

• A number from 1 to 9 is chosen at random. What is the probability of choosing an odd number?

• What is the probability of choosing a vowel from the alphabet?

Solution:

• P(head) = 1/2 and P(tail) = 1/2

• P(odd) = 5/9

• P(vowel) = 5/26.

Example 3.3 Determine the probability of the following events when rolling a single die:

• Probability of rolling a 4

• Probability of rolling an odd number

• Probability of rolling an even number

• Probability of rolling a 7

Solution: Since the sample space is Ω = { , , , , , }, the total number of outcomes is equal to 6.

• There is only one number 4 on the die. Thus, P(4) = 1/6

• There are 3 odd numbers (1, 3, 5). Thus, P(odd number) = 3/6 = 1/2

• There are 3 even numbers (2, 4, 6). Thus, P(even number) = 3/6 = 1/2

• The die does not contain a number 7. Thus, P(7) = 0.

19
3.2 R ANDOM VARIABLES

In many situations, we are interested in numbers associated with the outcomes of a random experiment. For example,
testing cars from a production line, we are interested in variables such as average emissions, fuel consumption, or
acceleration time.
Definition 3.2 A random variable X : S → R is a function that maps every outcome in the sample space of a random
experiment to a real number.
A discrete random variable X is a random variable that has a finite number of possible values in a discrete set,
whereas a continuous random variable X is a random variable that takes its values in a interval of numbers. Random
variables are usually denoted by capital letters X, whereas the values of the variables are usually denoted by lower
case letters x.
The probability distribution of a random variable X tells us what the possible values of X are and how probabilities
are assigned to those values. Figure 3.1 displays an example of discrete and continuous distributions.

f (x) f (x)

f (x3 )
f (x2 )

f (x4 )

f (x1 )

x
x1 x2 x3 x4 x
(a) Discrete distribution (b) Continuous distribution
F IGURE 3.1: Probability distributions.

Definition 3.3 The probability distribution or probability mass function (pmf) of a discrete random variable X is defined for every
value x of X by f ( x ) = P( X = x ), and it lists the values and their probabilities:

Value of X x1 x2 x3 . . . xk
P( X = xi ) f ( x1 ) f ( x2 ) f ( x3 ) . . . f ( xk )

The conditions f ( x i ) ≥ 0 and ∑i f ( xi ) = 1 are required for any pmf.

• The cumulative distribution function (cdf) F ( x ) is defined by

F( x ) = P( X ≤ x ) = ∑ f ( xi ) (3.2.1)
xi ≤x

• The expected value of X, denoted by E( X ) or µ, is

E( X ) = µ = ∑ xi f ( xi ) (3.2.2)
i

• The variance of X, denoted by Var ( X ) or σ2 , is

Var ( X ) = σ2 = ∑(xi − µ)2 f (xi ) = E(X2 ) − [E(X )]2 (3.2.3)

i

20
For a discrete random variable X, we have:

P ( a < X ≤ b) = P ( X ≤ b) − P ( X ≤ a ) = F ( b) − F ( a ) (3.2.4)
P ( a ≤ X ≤ b) = P ( a < X ≤ b) + P ( X = a ) = F ( b) − F ( a ) + f ( a ) (3.2.5)
P ( a < X < b) = P ( a < X ≤ b) − P ( X = b) = F ( b) − F ( a ) − f ( b). (3.2.6)

Example 3.4 Let a random variable X denote the number of defective parts produced per eight-hour shift by a machine. Experience
shows that it produces between 0 and 4 (inclusive) with the following probabilities:
X=x 0 1 2 3 4
f ( x ) = P( X = x ) 0.10 0.40 0.25 0.20 0.05

(i) Calculate the mean and standard deviation of X.

(ii) Calculate the probability that X falls in the interval (µ − 2σ, µ + 2σ ).

Solution: The probability mass function is shown in Figure 3.2.

0.4

0.35

0.3

0.25
f (x)

0.2

0.15

0.1

0.05

0
0 0.5 1 1.5 2 2.5 3 3.5 4
x

F IGURE 3.2: Probability mass function for the number of defective parts produced per
eight-hour shift by the machine.

MATLAB code
>> x = 0:4;
>> fx = [0.1 0.4 0.25 0.2 0.05];
>> h=stem(x,fx,’LineWidth’,2); % Visualize the pmf
>> set(get(h,’BaseLine’),’LineStyle’,’:’)
>> set(h,’MarkerFaceColor’,[.9 .9 .9]); grid
>> xlabel(’x’,’fontsize’,14,’fontname’,’times’);
>> ylabel(’f(x)’,’fontsize’,14,’fontname’,’times’);

(i) The mean and standard deviation of X are given by

µ = ∑ xi f ( x i ) = (0)(0.1) + (1)(0.4) + (2)(0.25) + (3)(0.2) + (4)(0.05) = 1.70
i
r
σ= ∑(xi − µ)2 f (xi ) = 1.05
i

21
 (ii) Calculate the probability that X falls in the interval (µ − 2σ, µ + 2σ ) is given by

P(µ − 2σ ≤ X ≤ µ + 2σ ) = P(−0.4 ≤ X ≤ 3.8)

= P ( X = 0) + P ( X = 1) + P ( X = 2) + P ( X = 3)
= 0.95.
In other words, the machine produces the number of defective parts within 2 standard deviations of the mean
with probability 95%.

Definition 3.4 The probability distribution or probability density function (pdf) of a continuous random variable X is a nonneg-
ative function f ( x ) such that
Z b Z ∞
P ( a ≤ X ≤ b) = f ( x )dx and f ( x )dx = 1 (3.2.7)
a −∞

• The cumulative distribution function (cdf) F ( x ) is defined by

Z x
F( x ) = P( X ≤ x ) = f (y)dy (3.2.8)
−∞

• The expected value of X, denoted by E( X ) or µ, is

Z ∞
E( X ) = µ = x f ( x )dx (3.2.9)
−∞

• The variance of X, denoted by Var ( X ) or σ2 , is

Z ∞
Var ( X ) = σ2 = ( x − µ)2 f ( x )dx = E( X 2 ) − [ E( X )]2 (3.2.10)
−∞

For a continuous random variable X, probabilities correspond to areas under the curve f ( x ). Also, for any single
value a, we have P( X = a) = 0. In addition,

P ( a < X < b) = P ( a ≤ X < b) = P ( a < X ≤ b) = P ( a ≤ X ≤ b). (3.2.11)

Let a and b be two constants. The expected value and variance of a random variable X satisfy the following proper-
ties:

• E( aX + b) = aE( X ) + b
• Var ( aX + b) = a2 Var ( X )

Example 3.5 Let X denote the width in mm of metal pipes from an automated production line. If X has the probability density
function (
10 e−10( x −5.5) if x ≥ 5.5
f (x) =
0 if x < 5.5.
determine:
(i) P( X < 5.7)
(ii) P( X > 6)
(iii) P(5.6 < X ≤ 6)

22
 (iv) the cumulative distribution function F ( x )
Solution:
(i)
Z 5.7
P( X < 5.7) = 10 e−10( x −5.5)dx
5.5
5.7

= −e−10( x −5.5)
5.5
= 1 − e−2 = 0.8647

(ii)
Z ∞
P ( X > 6) = 10 e−10( x −5.5)dx
6
∞

= −e−10( x −5.5)
6
= e−5 = 0.0067

(iii)
Z 6
P(5.6 < X ≤ 6) = 10 e−10( x −5.5)dx
5.6
6

= −e−10( x −5.5)
5.6
= e−1 − e−5 = 0.3611

(iv) For x < 5.5, F ( x ) = 0. For x ≥ 5.5, we have

Z x
F(x) = 10 e−10( t−5.5)dt
5.5
x

= −e−10( x −5.5)
5.5
= 1 − e−10( x −5.5)

Example 3.6 Let X denote the time in milliseconds for a chemical reaction to complete. Assume that the cumulative distribution
function of X is given by (
1 − e−0.05x if x ≥ 0
F(x) =
0 if x < 0.

(i) What is the probability density function of X?

(ii) What is the probability that a reaction completes within 40 milliseconds?
Solution:
(i) The probability density function is given by
(
′ 0.05 e−0.05x if x ≥ 0
f (x) = F (x) =
0 if x < 0.

(ii) The probability that the reaction completes within 40 milliseconds is

P( X ≤ 40) = F (40) = 1 − e−2 = 0.8647.

Using MATLAB, type: ≫ disttool to display interactive plots of the pdfs and cdfs of various continuous and discrete
probability distributions, as shown in Figure 3.3.

23
 0.8

0.6

0.4

0.2

0
−8 −6 −4 −2 0 2 4 6 8

F IGURE 3.3: disttool: graphical interface for exploring the effects of changing parame-
ters on the plot of a cdf or pdf.

3.3 D ISCRETE P ROBABILITY D ISTRIBUTIONS

A distribution is said to be discrete if it is built on discrete random variables. All the possible outcomes when pulling
a card from a stack are finite because we know in advance how many cards are in the stack and how many are being
pulled. A random variable is said to be discrete when all the possible outcomes are countable.

3.3.1 B INOMIAL D ISTRIBUTION

The binomial distribution assumes an experiment with n identical trials, each trial having only two possible outcomes
considered as success or failure and each trial independent of the previous ones. The binomial distribution is probably
the most commonly used discrete distribution.

Definition 3.5 A random variable X is said to have a binomial distribution, denoted X ∼ bino (n, p), if its probability mass
function is  
n x
f ( x ) = P( X = x ) = p (1 − p ) n − x , (3.3.1)
x
where

n = number of trials
x = number of successes in n trials
p = probability of success on any one trial
 
n n!
=
x x!(n − x )!

The mean and variance of X ∼ bino (n, p) are E( X ) = np and Var ( X ) = np(1 − p). Moreover, the cdf is given by
x  
n i
F( x ) = P( X ≤ x ) = ∑ p (1 − p ) n − i . (3.3.2)
i =0
i

Figure 3.4 shows the pdf and cdf of the binomial distribution X ∼ bino (n, p) with parameters n = 10 and p = 0.2.

24
 1 p = 0 . 2 ; % P r o b a b i l i t y o f success f o r each t r i a l
2 n = 1 0 ; % Number o f t r i a l s
3 x = 0 : n ; % Outcomes
4 f x = p d f ( ' b i n o ' , x , n , p ) ; % pmf
5 stem ( x , fx , ' LineWidth ' , 2 ) % V i s u a l i z e th e pmf
6 figure ;
7 Fx = c d f ( ' b i n o ' , x , n , p ) ; % c d f
8 s t a i r s ( x , Fx , ' LineWidth ' , 2 ) ; g r i d % V i s u a l i z e th e c d f

0.35 1

0.9
0.3

0.8
0.25
0.7

0.2 0.6

F (x)
f(x)

0.15 0.5

0.4
0.1
0.3

0.05
0.2

0 0.1
0 2 4 6 8 10 0 2 4 6 8 10
x x
(a) (b)
F IGURE 3.4: Binomial distribution X ∼ bino (n, p) with n = 10 and p = 0.2: (a) pdf,
and (b) cdf.

Example 3.7 A biased coin is tossed 6 times. The probability of heads on any toss is 0.3. Let X denote the number of heads that
come up. Calculate:
(i) P( X = 2)

(ii) P( X = 3)
(iii) P(1 < X ≤ 5)
(iv) the mean and variance of X.
Solution: If we call heads a success then X follows a binomial distribution with parameters n = 6 and p = 0.3.

(i) P( X = 2) = (62)(0.3)2(0.7)4 = 0.3241

(ii) P( X = 3) = (63)(0.3)3(0.7)3 = 0.1852

(iii) P(1 < X ≤ 5) = P( X = 2) + P( X = 3) + P( X = 4) + P( X = 5) = 0.5791

(iv) E( X ) = np = (6)(0.3) = 1.8 and Var ( X ) = np(1 − p) = (6)(0.3)(0.7) = 1.26

These probabilities can be computed using MATLAB as follows:
MATLAB code
>> pdf(’bino’,2,6,.3)
ans =
0.3241
>> pdf(’bino’,3,6,.3)
ans =
0.1852
>> cdf(’bino’,5,6,.3)-cdf(’bino’,1,6,.3)

25
 ans =
0.5791

Example 3.8 The probability that a randomly selected technician will finish his or her project successfully is 0.8. Let X be the
number of technicians among a randomly selected group of 10 technicians who will finish their projects successfully.
(i) Plot the probability mass function of X.
(ii) Calculate the probability that exactly one technician will finish his/her project successfully.
(iii) Calculate the probability that at least three technicians will finish their project successfully.
(iv) Calculate the probability that at most five technicians will finish their project successfully.
(v) Calculate the probability that between four and six (inclusive) technicians will finish their project successfully.
(vi) Calculate the mean and standard deviation of X.
Solution: The random variable X follows a binomial distribution with parameters n = 10 and p = 0.8.
(i) The graphical representation of the probability distribution is shown in Figure 3.5.

0.35

0.3

0.25

0.2
f (x)

0.15

0.1

0.05

0
0 2 4 6 8 10
x

F IGURE 3.5: Probability mass function of X ∼ bino (n, p) with n = 10 and p = 0.8.

(ii) P( X = 1) = (10 1 9
1 )(0.8) (0.2) ≈ 0

(iii) P( X ≥ 3) = 1 − P( X < 3) = 1 − P( X ≤ 2) = 1 − ( P( X = 0) + P( X = 1) + P( X = 2)) = 0.9999

(iv) P( X ≤ 5) = P( X = 0) + P( X = 1) + P( X = 2) + P( X = 3) + P( X = 4) + P( X = 5) = 0.0328
(v) P(4 ≤ X ≤ 6) = P( X = 4) + P( X = 5) + P( X = 6) = 0.12
p p
(vi) µ = np = (10)(0.8) = 8 and σ = np(1 − p) = (10)(0.8)(0.2) = 1.2649

Example 3.9 The probability that the Food and Drug Administration (FDA) will approve a new drug is 0.6. Suppose that five
new drugs are submitted to FDA for its approval.
(i) Plot the probability mass function of X.
(ii) Calculate the probability that exactly three drugs are approved.
(iii) Calculate the probability that at most three drugs are approved.

26
 (iv) Calculate the probability that at least three drugs are approved.
(v) Calculate the probability that between two and four (inclusive) drugs are approved.
(vi) Calculate the mean and standard deviation of X.
Solution: Let X be the number of dugs among the five new drugs submitted to FDA will be approved. The random
variable X follows a binomial distribution with parameters n = 5 and p = 0.6.
(i) The graphical representation of the probability distribution is shown in Figure 3.6.

0.35

0.3

0.25

0.2
f (x)

0.15

0.1

0.05

0
0 1 2 3 4 5
x

F IGURE 3.6: Probability mass function of X ∼ bino (n, p) with n = 5 and p = 0.6.

(ii) P( X = 3) = (53)(0.6)3(0.4)2 = 0.3456

(iii) P( X ≤ 3) = P( X = 0) + P( X = 1) + P( X = 2) + P( X = 3) = 0.6630
(iv) P( X ≥ 3) = 1 − P( X < 3) = 1 − P( X ≤ 2) = 1 − ( P( X = 0) + P( X = 1) + P( X = 2)) = 0.6826
(v) P(2 ≤ X ≤ 4) = P( X = 2) + P( X = 3) + P( X = 4) = 0.8352
p p
(vi) µ = np = (5)(0.6) = 3 and σ = np(1 − p) = (5)(0.6)(0.4) = 1.0954

Example 3.10 Each item produced by a certain process is independently defective with probability 0.02.
(i) What is the probability that a batch of 60 such items contains 0 defective items?
(ii) What is the probability that a batch of 60 such items contains at most 1 defective item?
Solution: Let X denote the number of defective items in the batch. X follows a binomial distribution with parameters
n = 60 and p = 0.02
(i)  
60
P(0 defectives) = P( X = 0) = (0.02)0(1 − 0.02)60 = 0.2976
0
(ii) X follows a binomial distribution with parameters n = 60 and p = 0.02. Thus,
P(at most 1 defective) = P ( X ≤ 1)
= P ( X = 0) + P ( X = 1)
   
60 0 60 60
= (0.02) (0.98) + (0.02)1(0.98)59
0 1
= 0.6619.

27
Example 3.11 A machine produces soda bottles, and 95 percent of all bottles produced pass an audit. What is the probability of
having only 2 bottles that pass audit in a randomly selected sample of 10 bottles?.

Solution: Let X denote the number bottles that pass audit. X follows a binomial distribution with parameters n = 10
and p = 0.95. The probability of having only 2 bottles that pass audit in a randomly selected sample of 10 bottles is
given by  
10
P ( X = 2) = (0.95)2(1 − 0.95)8 ≈ 0.
2
In other words, the probability of having only two good bottles out of 10 is zero.

Example 3.12 Suppose each customer who visits a car dealership independently returns his or her questionnaire with probability
of 0.78. What is the probability that out of 64 customers there are at most 36 returns?.

Solution: Let X denote the number of returns. X follows a binomial distribution with parameters n = 64 and p = 0.78.
The probability that out of 64 customers there are at most 36 returns is given by

36  
64
P( X ≤ 36) = ∑ (0.78)i(0.22)64−i ≈ 0.00009.
i =0
i

3.3.2 P OISSON D ISTRIBUTION

In many practical situations we are interested in measuring how many times a certain event occurs in a specific time
interval or in a specific length or area, where the average of such an event occurring is known. For instance:

• the number of phone calls received at an exchange or call center in an hour;

• the number of customers arriving at a toll booth per day;

• the number of flaws on a length of cable;

• the number of cars passing using a stretch of road during a day.

The Poisson distribution plays a key role in modeling such problems. For instance, a Quality Control manager may
want to know the probability of finding a defective part on a manufactured circuit board.

Definition 3.6 A random variable X is said to have a Poisson distribution, denoted X ∼ poiss(λ), if its probability mass function
is
e−λ λ x
f (x) = , x = 0, 1, 2, . . . (3.3.3)
x!
where the parameter λ > 0 is the number of occurrences per unit time or space.
The mean and variance of X ∼ poiss(λ) are equal: E( X ) = Var ( X ) = λ

The expected value of a Poisson-distributed random variable is equal to the parameter λ and so is its variance. Fig-
ure 3.7 shows the pdf and cdf of the Poisson distribution X ∼ poiss(λ) for different values of λ.

1 lambda = 5 ; % parameter lambda

2 x = 0:40;
3 f x = p d f ( ' p o i s s ' , x , lambda ) ; % pmf
4 stem ( x , fx , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e pmf
5 figure ;
6 Fx = c d f ( ' p o i s s ' , x , lambda ) ; % c d f
7 s t a i r s ( x , Fx , ' LineWidth ' , 2 ) ; g r i d % V i s u a l i z e th e c d f

Example 3.13 The number of flaws in a fibre optic cable follows a Poisson distribution. The average number of flaws in 50m of
cable is 1.2.

(i) What is the probability of exactly three flaws in 150m of cable?

(ii) What is the probability of at least two flaws in 100m of cable?

28
 0.18 1
λ=5 λ=5
0.16 λ=10 0.9 λ=10
λ=15 λ=15
0.8
0.14

0.7
0.12
0.6
0.1

F (x)
f(x)

0.5
0.08
0.4
0.06
0.3

0.04
0.2

0.02 0.1

0 0
0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40
x x
(a) (b)
F IGURE 3.7: Poisson distribution X ∼ poiss(λ) for different values of λ: (a) pdf, and
(b) cdf.

(iii) What is the probability of exactly one flaw in the first 50m of cable and exactly one flaw in the second 50m of cable?
Solution:
(i) The mean number of flaws in 150m of cable is 3.6. So the probability of exactly three flaws in 150m of cable is

e−3.6 (3.6)3
P ( X = 3) = = 0.212
3!

(ii) The mean number of flaws in 100m of cable is (2)(1.2) = 2.4. Let X be the number of flaws in 100m of cable.

P ( X ≥ 2) = 1 − P ( X ≤ 1)


= 1 − P ( X = 0) + P ( X = 1)
 −2.4 
e (2.4)0 e−2.4 (2.4)1
= 1− +
0! 1!
= 1 − (0.0907 + 0.2177) = 0.6916

(iii) Now let X denote the number of flaws in a 50m section of cable. Then we know that

e−1.2 (1.2)1
P ( X = 1) = = 0.3614
1!
As X follows a Poisson distribution, the occurrence of flaws in the first and second 50m of cable are indepen-
dent. Thus the probability of exactly one flaw in the first 50m and exactly one flaw in the second 50m is:
(0.3614)(0.3614) = 0.1306.

Example 3.14 The number of breakdowns of a machine is a random variable having the Poisson distribution with λ = 2.2
breakdown per month.
(i) Plot the probability mass function of the distribution over a period of 12 months.
(ii) Calculate the probability that the machine will work during any given month with no breakdown.
(iii) Calculate the probability that the machine will work during any given month with one breakdown.
(iv) Calculate the probability that the machine will work during any given month with two breakdowns.
(v) Calculate the probability that the machine will work during any given month with at least two breakdowns.

29
 (vi) Calculate the probability that the machine will work with four breakdowns in two months.
(vii) Calculate the probability that the machine will work with five breakdowns in two and half months.
Solution: Let X denote the number of breakdowns per month. From the given information, X ∼ poiss(λ) with λ = 2.2.
(i) The probability mass function of the distribution over a period of 12 months is shown in Figure 3.8.

0.35

0.3

0.25

0.2
f (x)

0.15

0.1

0.05

0
0 2 4 6 8 10 12
x

F IGURE 3.8: Probability mass function of the distribution over a period of 12 months.

(ii) The probability that the machine will work during any given month with no breakdown is

e−2.2 (2.2)0
P ( X = 0) = = 0.1108
0!

(iii) The the probability that the machine will work during any given month with one breakdown is

e−2.2 (2.2)1
P ( X = 1) = = 0.2438
1!

(iv) The the probability that the machine will work during any given month with two breakdowns is

e−2.2 (2.2)2
P ( X = 2) = = 0.2681
2!

(v) The the probability that the machine will work during any given month with at least two breakdowns is

P( X ≥ 2) = 1 − P( X < 2) = 1 − ( P( X = 0) + P( X = 1)) = 0.6454

(vi) The mean number of breakdowns in two months is λ = (2)(2.2) = 4.4. The probability that the machine will
work with four breakdowns in two months is
e−4.4 (4.4)4
P ( X = 4) = = 0.1917
4!

(vii) The mean number of breakdowns in two and half months is λ = (2.5)(2.2) = 5.5. The probability that the
machine will work with five breakdowns in two and half months is
e−5.5 (5.5)5
P ( X = 5) = = 0.1714.
5!

30
Example 3.15 The number of visitors to a web server per minute follows a Poisson distribution. If the average number of visitors
per minute is 4, what is the probability that:

(i) There are two or fewer visitors in one minute?

(ii) There are exactly two visitors in 30 seconds?

Solution: The average number of visitors in a minute is λ = 4.

(i) The probability that there are two or fewer visitors in one minute is

P ( X ≤ 2) = P ( X = 0) + P ( X = 1) + P ( X = 2)
e −4 ( 4 ) 0 e −4 ( 4 ) 1 e −4 ( 4 ) 2
= +
0! 1! 2!
= 0.0183 + 0.0733 + 0.1465
= 0.2381

(ii) If the average number of visitors in 1 minute is 4, the average in 30 seconds is λ = 2. Thus, the probability that
there are exactly two visitors in 30 seconds is

e −2 ( 2 ) 2
P ( X = 2) = = 0.2707
2!

Example 3.16 A product failure has historically averaged 3.84 occurrences per day. What is the probability of 5 failures in a
randomly selected day?

Solution: The average failure occurrences per day is λ = 3.84. The probability of 5 failures in a randomly selected day
is given by
e−3.84 (3.84)5
P ( X = 5) = = 0.1495.
5!
MATLAB code
>> lambda = 3.84; x = 5;
>> P = pdf(’poiss’,x,lambda)
P =
0.1495

Example 3.17 It is believed that the number of bookings taken per hour at an online travel agency follows a Poisson distribution.
Past records indicate that the hourly number of bookings has a mean of 15 and a standard deviation of 2.5. Comment on the
suitability of the Poisson distribution for this example?

Solution: If the number of hourly bookings at this travel agent did follow a Poisson distribution, we would expect that
the mean and variance should be equal. However, in this case

E( X ) = 15, Var ( X ) = (2.5)2 = 6.25

This suggests that the Poisson distribution is not appropriate for this case.

3.4 C ONTINUOUS P ROBABILITY D ISTRIBUTIONS

Most experiments in business operations have sample spaces that do not contain a finite, countable number of simple
events. A distribution is said to be continuous when it is built on continuous random variables, which are variables that
can assume the infinitely many values corresponding to points on a line interval. An example of a random variable
would be the time it takes a production line to produce one item. In contrast to discrete variables, which have values
that are countable, the continuous variables values are measurements.

31
3.4.1 U NIFORM D ISTRIBUTION
The simplest continuous probability distribution is called uniform distribution, as shown in Figure 3.9. It provides a
model for continuous random variables that are evenly (or randomly) distributed over a certain interval.

Definition 3.7 A random variable X is said to have a uniform distribution, denoted X ∼ uni f ( a, b), on the interval [ a, b] if its
probability density function is 
 1 for x ∈ [ a, b]
f (x) = b − a (3.4.1)
0 otherwise.

a+b ( b − a )2
The mean and variance of X ∼ uni f ( a, b) are E( X ) = and Var ( X ) = . Moreover, the cdf is given by
2 12


 0 for x < a
x − a
F( x ) = P( X ≤ x ) = for x ∈ [ a, b) (3.4.2)

 b−a

1 for x ≥ b

All outcomes over the uniform distribution’s entire range are equally likely. To throw a die is an example of a uniform
distribution, where any number of 1, 2, . . . , 6 has an equal probability to show up. Figure 3.10 displays the pdf and cdf
of a uniform distribution with parameters a = 0 and b = 1.
f(x)

x
a b

F IGURE 3.9: Illustration of the uniform pdf on the interval [ a, b].

1 a = 0 ; % parameter a
2 b = 1 ; % parameter b
3 x = − 3: .0 1 : 3 ;
4 fx = pdf ( ' u n i f ' , x , a , b ) ; % pdf
5 p l o t ( x , fx , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e p d f
6 figure ;
7 Fx = c d f ( ' u n i f ' , x , a , b ) ; % cdf
8 p l o t ( x , Fx , ' LineWidth ' , 2 ) ; g r i d % V i s u a l i z e th e c d f

Example 3.18 Suppose a delay in starting production due to an unexpected mechanical failure is anywhere from 0 to 30 minutes.

(i) Calculate the probability that production will be delayed by less than 10 minutes.

(ii) Calculate the probability that production will be delayed by more than 10 minutes.

32
 1 1

0.9 0.9

0.8 0.8

0.7 0.7

0.6 0.6

F (x)
f(x)

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1

0 0
−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3
x x
(a) (b)
F IGURE 3.10: Uniform distribution X ∼ uni f ( a, b) with a = 0 and b = 1: (a) pdf; (b)
cdf.

(iii) Calculate the probability that production will be delayed between 12 and 22 minutes.

(iv) Calculate the mean and standard deviation of the distribution.

Solution: Let X denote the the time by which production will be delayed. From the given information, we can see that
X ∼ uni f ( a, b) with a = 0 and b = 30. Using the cdf of the uniform distribution, we have:
10 1
(i) P( X ≤ 10) = F (10) = 30 = 3

10 1 2
(ii) P( X ≥ 10) = 1 − F (10) = 1 − 30 = 1− 3 = 3

22 12 1
(iii) P(12 ≤ X ≤ 22) = F (22) − F (12) = 30 − 30 = 3

(iv) The mean and standard deviation of X are given by

a+b 0 + 30
µ= = = 15
2 2
and
b−a 30 − 0
σ= √ = √ = 8.6603.
12 12

The pdf and cdf of X are depicted in Figure 3.11.

Example 3.19 A quality control inspector for Tamuda Company, which manufactures aluminum water pipes, believes that the
product has varying lengths. Suppose the pipes turned out by one of the production lines of Tamuda Company can be modeled
by a uniform distribution over the interval 29.50 to 30.05 feet. Calculate the mean and standard deviation of the length of the
aluminum water pipe.

Solution: Let X denote the length of the aluminum water pipe. From the given information, we can see that the random
variable X is uniformly distributed over the interval (29.50, 30.05), i.e. X ∼ uni f (29.50, 30.05). The mean and standard
deviation of X are given by
30.05 + 29.50
µX = E( X ) = = 29.775 feet
2
and q
30.05 − 29.50
σX = Var ( X ) = √ = 0.1588 feet.
12

33
 0.035 1

0.9
0.03
0.8

0.025 0.7

0.6
0.02

F (x)
f (x)

0.5
0.015
0.4

0.01 0.3

0.2
0.005
0.1

0 0
−10 0 10 20 30 40 −10 0 10 20 30 40
x x
(a) (b)
F IGURE 3.11: X ∼ uni f ( a, b) with parameters a = 0 and b = 30: (a) pdf; (b) cdf.

3.4.2 N ORMAL D ISTRIBUTION

The normal distribution is probably the most important distribution in the study of quality control. It is often used to
describe the behavior of a process and make estimates and inferences concerning the process.

Definition 3.8 A random variable X is normally distributed with mean µ and variance σ2 , denoted X ∼ N (µ, σ2 ), if its proba-
bility density function is
( x − µ)2
1 −
f (x) = √ e 2σ2 , −∞ < x < ∞ (3.4.3)
2πσ
A normal distribution is symmetric and has a bell-shaped density curve with a single peak, as shown in Figure 3.12.
The mean µ is where the peak of the density occurs, and the standard deviation σ indicates the spread or girth of the bell
curve. Many common attributes and physical characteristics such as test scores, heights, weights, etc., tend to follow
a normal distribution. Also, errors in measurement or production processes can often be approximated by a normal
distribution. The graph of the normal distribution depends on two factors: the mean µ and the standard deviation σ.
The mean of the distribution determines the location of the center of the graph, and the standard deviation determines
the height and width of the graph. When the standard deviation is large, the curve is short and wide; when the standard
deviation is small, the curve is tall and narrow. All normal distributions look like a symmetric, bell-shaped curves. This
can be visualized using the MATLAB Probability Distribution Function Tool (>> disttool) shown in Figure 3.3.
In Fig. 3.13(left), the shaded area under the curve is equal to the probability P( a ≤ X ≤ b).
X−µ
If X ∼ N (µ, σ2 ), then the random variable Z = ∼ N (0, 1). The distribution of the random variable Z is called
σ
the standard normal distribution, and its cdf is denoted by Φ. The value Φ(z) = P( Z ≤ z) is the shaded area under
the standard norm curve as shown in Figure 3.13(right). This value is tabulated and is used to calculate probabilities
for any normal random variable. Since the normal pdf is symmetric about zero, it follows that Φ(−z) = 1 − Φ(z).
Figure 3.14 shows the pdf and cdf of the standard normal distribution.

1 mu = 0 ; % mean
2 sigma = 1 ; % s ta n d a r d d e v i a t i o n
3 z = − 3: . 1 : 3 ;
4 f z = p d f ( ' norm ' , z , mu, sigma ) ; % pmf
5 p l o t ( z , fz , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e p d f
6 figure ;
7 Fz = c d f ( ' norm ' , z , mu, sigma ) ; % c d f
8 p l o t ( z , Fz , ' LineWidth ' , 2 ) ; g r i d % V i s u a l i z e th e c d f

The characteristics of the normal probability distribution are shown in Figure 3.15, where

• 68.26% of values of a normal random variable are within +/- 1 standard deviation of its mean.

34
 0.8
µ = 0, σ = 0.5
µ = 0, σ =1
0.7
µ = 0, σ =3
µ = 2, σ = 0.8
0.6

0.5

f (x)
0.4

0.3

0.2

0.1

0
−5 0 5
x

F IGURE 3.12: Probability density functions of the normal distribution. The blue curve
is the standard normal distribution.

Φ(z)

x
a b 0 z

F IGURE 3.13: Left: Probability P( a ≤ X ≤ b), where X ∼ N (µ, σ2 ). Right: Illustration

of Φ(z).

• 95.44% of values of a normal random variable are within +/- 2 standard deviations of its mean.

• 99.72% of values of a normal random variable are within +/- 3 standard deviations of its mean.

These percentage values can be calculated using the standard normal distribution table, or simply using the following
MATLAB commands:
MATLAB code
>> cdf(’normal’,1,0,1)-cdf(’normal’,-1,0,1)
ans =
0.6827
>> cdf(’normal’,2,0,1)-cdf(’normal’,-2,0,1)
ans =
0.9545
>> cdf(’normal’,3,0,1)-cdf(’normal’,-3,0,1)
ans =
0.9973

35
 0.4 1

0.9
0.35
0.8
0.3
0.7
0.25
0.6

Φ(z)
f(z)

0.2 0.5

0.4
0.15
0.3
0.1
0.2
0.05
0.1

0 0
−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3
z z
(a) (b)
F IGURE 3.14: Standard normal distribution Z ∼ N (0, 1): (a) pdf; (b) cdf.

34.1% 34.1%

13.6% 13.6%

2.1% 2.1%

µ − 3σ µ − 2σ µ−σ µ µ+σ µ + 2σ µ + 3σ

F IGURE 3.15: Characteristics of the normal probability distribution.

For α ∈ (0, 1), denote by zα/2 the upper 100α/2 percentage point of the standard normal distribution as shown in
Figure 3.16. That is,
P( Z > zα/2 ) = 1 − P( Z ≤ zα/2 ) = 1 − Φ(zα/2 ) = α/2
Thus, 
α α
Φ(zα/2 ) = 1 − ⇒ zα/2 = Φ−1 1 − ,
2 2
where Φ−1 denotes the inverse cdf of Φ.
For α = 0.05, the value of zα/2 can be computed using MATLAB as follows:
MATLAB code
>> alpha = 0.05;
>> icdf(’normal’,1-alpha/2,0,1)
ans =
1.96

Similarly, the lower 100α/2 percentage point of the standard normal distribution is such that

α/2 = P( Z < −zα/2 ) = Φ(−zα/2 ).

36
 Z ∼ N (0, 1)

α/2 α/2

−zα/2 0 zα/2

F IGURE 3.16: Upper and lower 100α/2 percentage points of the standard normal dis-
tribution.

Six Sigma Methodology and Normal Distribution:

Six Sigma is a business management strategy that seeks to improve the quality of process outputs by identifying and
removing the causes of defects and minimizing variability in manufacturing and business processes. It is based on
the statistical concept of 6σ, measuring a process at only 3.4 defects per million opportunities (DPMO), where the
standard deviation σ is the measure of variation in the process. DPMO is a measure of process performance and it is
defined as
1,000,000 × number of defects
DPMO = (3.4.4)
number of units × number of opportunities per unit
where a defect is defined as a measurable quality characteristic of the process or its output that is not conforming to
specifications or customer requirements.
The term “six sigma process” comes from the notion that if one has six standard deviations between the process mean
and the nearest specification limit, as shown in Figure, practically no items will fail to meet specifications. This is based
on the calculation method employed in process capability studies, which will be covered in Chapter 5.

LSL = −6σ µ − 1.5σ µ µ + 1.5σ U SL = 6σ

−6 −5 −4 −3 −2 −1 0 1 2 3 4 5 6
F IGURE 3.17: Statistical assumptions of the Six Sigma model.

Given the process standard deviation σ, we may calculate the DPMO as follows:
  
kσ − 1.5σ
DPMO = 1000000 1 − Φ = 1000000[1 − Φ(k − 1.5)] (3.4.5)
σ

37
where the 1.5σ shift is subjective but some experts use this as conversion from long to short term performance estimates,
and k = 1, 2, . . . , 6. For example, k = 6 corresponds to 6σ, which yields DPMO = 1000000[1 − Φ(6 − 1.5)] ≈ 3.4. So
the 3.4 DPMO of a Six Sigma process in fact corresponds to 4.5σ, namely 6σ minus the 1.5σ shift introduced to account
for long-term variation

Example 3.20 Suppose a quality characteristic of a product is normally distributed with mean µ = 18 and standard deviation
σ = 1.5. The specification limits given by the customer are (15, 21). Determine what percentage of the product meets the
specifications set by the customer.
Solution: Let the random variable X denote the quality characteristic of interest. Then, X is normally distributed with
mean µ = 18 and standard deviation σ = 1.5. We are interested in finding the probability
 
15 − µ 21 − µ
P(15 ≤ X ≤ 21) = P ≤Z≤
σ σ
 
15 − 18 21 − 18
= P ≤Z≤
1.5 1.5
= P(−2 ≤ Z ≤ 2)
= Φ(2) − Φ(−2) = 2Φ(2) − 1 = 0.9545

That is, the percentage of product that will meet the specifications set by the customer is 95.44%.

Example 3.21 The weekly profits of a large group of stores are normally distributed with a mean of µ = 1200 and a standard
deviation of σ = 200. What is the percentage of the stores that make $1500 or more a week?
Solution: The percentage of the stores that make $1500 or more a week is given by the probability
   
1500 − µ 1500 − 1200
P( X ≥ 1500) = P Z ≥ =P Z≥ = P( Z ≥ 1.5) = 1 − Φ(1.5) = 0.0668
σ 200

That is, 6.68% of the stores make more than $1500 week.

Example 3.22 The mean number of defective parts that come from a production line is µ = 10.5 with a standard deviation of
σ = 2.5. What is the probability that the number of defective parts for a randomly selected sample will be less than 15?
Solution:    
15 − µ 15 − 10.5
P( X < 15) = P Z < =P Z< = P( Z < 1.8) = Φ(1.8) = 0.9641.
σ 2.5

Example 3.23 The actual volume of soup in 500ml jars follows a normal distribution with mean 500ml and variance 16ml. If X
denotes the actual volume of soup in a jar, calculate
(i) P( X > 496)
(ii) P( X < 498)
(iii) P(492 < X < 506)
Solution: From the given information, X ∼ N (µ, σ2 ) with µ = 500 and σ = 4.
(i)  
496 − 500
P( X > 496) = P Z > = P( Z > −1) = 1 − Φ(−1) = Φ(1) = 0.8413
4

(ii)  
498 − 500
P( X < 498) = P Z < = P( Z < −0.5) = Φ(−0.5) = 1 − Φ(0.5) = 0.3085
4

(iii)  
492 − 500 506 − 500
P(492 < X < 506) = P <Z< = Φ(1.5) − Φ(−2) = 0.9104
4 4

38
Example 3.24 In the previous example, suppose that the mean volume of soup in a jar is unknown but that the standard deviation
is 4. If only 3% of jars are to contain less than 492ml what should the mean volume of soup in a jar be?
Solution: We want the value of µ for which
 
492 − µ
P Z< = 0.03
4
From the standard normal distribution table, we have P( Z < −1.88) = 1 − Φ(1.88) = 0.03. Thus,
492 − µ
= −1.88 ⇒ µ = 492 + (4)(1.88) = 499.52.
4

3.4.3 N ORMAL A PPROXIMATION TO THE B INOMIAL D ISTRIBUTION

The normal distribution can be used to approximate binomial probabilities when there is a very large number of trials
and when both np and n(1 − p) are large. A rule of thumb is to use this approximation when both np ≥ 5 and
n(1 − p) ≥ 5. If both are greater than 5, then the approximation should be good.

Definition 3.9 Let X ∼ bino (n, p). If n is large relative to p, then the random variable

X − np
Z= p (3.4.6)
np(1 − p)

is approximately a standard normal random variable.

Figure 3.18 shows normal approximations to the binomial distribution bino (10, 0.5) and bino (30, 0.5).

0.35 0.16
bino(10,0.5) bino(30,0.5)
0.3 N(µ,σ2) 0.14 N(µ,σ2)

0.12
0.25

0.1
0.2
0.08
0.15
0.06

0.1
0.04

0.05 0.02

0 0
0 2 4 6 8 10 0 5 10 15 20 25 30

(a) (b)
F IGURE 3.18: Normal approximation to the binomial distribution: (a) bino (10, 0.5); (b)
bino (30, 0.5).

Continuity Correction for the Normal Approximation to the Binomial Distribution

To improve the accuracy of the approximation, we usually use a correction factor to take into account that the bino-
mial random variable is discrete while the normal is continuous. The basic idea is to treat the discrete value x as the
continuous interval from x − 0.5 to x + 0.5. Let X ∼ bino (n, p) and Z ∼ N (0, 1). Then,
! !
x + 0.5 − np x + 0.5 − np
P( X ≤ x ) ≈ P Z < p =Φ p (3.4.7)
np(1 − p) np(1 − p)
! !
x − 0.5 − np x − 0.5 − np
P( X ≥ x ) ≈ P Z > p = 1−Φ p (3.4.8)
np(1 − p) np(1 − p)

39
 ! !
x2 + 0.5 − np x1 − 0.5 − np
P ( x1 ≤ X ≤ x2 ) ≈ Φ p −Φ p (3.4.9)
np(1 − p) np(1 − p)

Example 3.25 12% of the memory cards made at a certain factory are defective. If a sample of 150 cards is selected randomly, use
the normal approximation to the binomial distribution to calculate the probability that the sample contains:

(i) at most 20 defective cards

(ii) between 15 and 23 defective cards

(iii) exactly 17 defective cards

(iv) at least 19 defective cards

Solution: From the given information, we have n = 150 and p = 0.12. Both np = 18 ≥ 5 and n(1 − p) = 132 ≥ 5 are
satisfied, so we can use the normal approximation to the binomial. Using the continuity correction yields

(i) ! !
20 + 0.5 − np 20 + 0.5 − 18
P( X ≤ 20) ≈ Φ p =Φ p = Φ(0.63) = 0.7357
np(1 − p) 18(1 − 0.12)

(ii) ! !
23 + 0.5 − 18 15 − 0.5 − 18
P(15 ≤ X ≤ 23) ≈ Φ p −Φ p = Φ(1.38) − Φ(−0.88) = 0.7268
18(1 − 0.12) 18(1 − 0.12)

(iii) ! !
17 + 0.5 − 18 17 − 0.5 − 18
P(17 ≤ X ≤ 17) ≈ Φ p −Φ p = Φ(−0.13) − Φ(−0.38) = 0.0963
18(1 − 0.12) 18(1 − 0.12)

(iv) !
19 − 0.5 − 18
P( X ≥ 19) ≈ 1 − Φ p = 1 − Φ(0.55) = 0.45.
18(1 − 0.12)

3.4.4 E XPONENTIAL D ISTRIBUTION

The exponential distribution is commonly used to model waiting times between occurrences of rare events, lifetimes of
electrical or mechanical devices. It is primarily used in reliability applications. The exponential distribution closely re-
sembles the Poisson distribution. The Poisson distribution is built on discrete random variables and describes random
occurrences over some intervals, whereas the exponential distribution is continuous and describes the time between
random occurrences. Examples of an exponential distribution are the time between machine breakdowns and the
waiting time in a line at a supermarket.

Definition 3.10 A random variable X is said to have an exponential distribution, denoted X ∼ ex p(λ), with positive parameter
λ > 0 if its probability density function is (
λe−λx if x ≥ 0
f (x) = (3.4.10)
0 otherwise.
1 1
The mean and variance of X ∼ ex p(λ) are E( X ) = and Var ( X ) = 2 . Moreover, the cdf is given by
λ λ

F ( x ) = P( X ≤ x ) = 1 − e−λx . (3.4.11)

The parameter λ of the exponential distribution is often called the rate parameter. The pdf and cdf of the exponential
distribution X ∼ ex p(λ) for various values of the rate parameter λ are shown in Figure 3.19.

40
 1 lambda = 0 . 5 ; % parameter
2 x = 0: .1 : 5 ;
3 f x = p d f ( ' exp ' , x , lambda ) ; % pdf
4 p l o t ( x , fx , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e p d f
5 figure ;
6 Fx = c d f ( ' exp ' , x , lambda ) ; % cdf
7 p l o t ( x , Fx , ' LineWidth ' , 2 ) ; g r i d % V i s u a l i z e th e c d f

2 1
λ=0.5
1.8 λ=1.0 0.9
λ=1.5
1.6 0.8

1.4 0.7

1.2 0.6

F (x)
f(x)

1 0.5

0.8 0.4

0.6 0.3

0.4 0.2
λ=0.5
0.2 0.1 λ=1.0
λ=1.5
0 0
0 1 2 3 4 5 0 1 2 3 4 5
x x
(a) (b)
F IGURE 3.19: Exponential distribution X ∼ ex p(λ) for various values of λ: (a) pdf; (b)
cdf.

Example 3.26 Jobs are sent to a printer at an average of 3 jobs per hour.
(i) What is the expected time between jobs?
(ii) What is the probability that the next job is sent within 5 minutes?
Solution: Job arrivals represent rare events, thus the time X between them is exponentially distributed with rate 3
jobs/hour i.e. λ = 3.
(i) E( X ) = 1/λ = 1/3 hours or 20 minutes
(ii) Using the same units (hours), we have 5 min=1/12 hours. Thus, the probability that the next job is sent within 5
minutes is
P( X < 1/12) = F (1/12) = 1 − e−(3)(1/12) = 0.2212.

Example 3.27 Suppose that the time in months between line stoppages on a production line follows an exponential distribution
with λ = 1/2.
(i) What is the probability that the time until the line stops again will be more than 15 months?
(ii) What is the probability that the time until the line stops again will be less than 20 months?
(iii) What is the probability that the time until the line stops again will be between 10 and 15 months?
(iv) Calculate the mean µ and standard deviation σ. Then, calculate the probability that the time until the line stops will be
between (µ − 3σ ) and (µ + 3σ ).
Solution: The time X between them line stoppages is exponentially distributed with λ = 1/2.
(i) The probability that the time until the line stops again will be more than 15 months is

P( X > 15) = e−15λ = e−(15)(1/2) = e−7.5 ≈ 0

41
 (ii) The probability that the time until the line stops again will be less than 20 months is

P( X < 20) = 1 − e−(20)(1/2) ≈ 1

(iii) The probability that the time until the line stops again will be between 10 and 15 months is

P(10 ≤ X ≤ 15) = F (15) − F (10) = e−(10)(1/2) − e−(15)(1/2) ≈ 0.0062

(iv) The mean and the standard deviation are given by µ = σ = 1/λ = 2. Thus, the probability that the time until the
line stops will be between (µ − 3σ ) and (µ + 3σ ) is

P(µ − 3σ ≤ X ≤ µ + 3σ ) = P(−4 ≤ X ≤ 8) = P(0 ≤ X ≤ 8) = F (8) = 1 − e−(8)(1/2) ≈ 1.

3.4.5 C HI -S QUARED D ISTRIBUTION

The chi-squared distribution with n degrees of freedom is the distribution of a sum of the squares of n independent
standard normal random variables. It is typically used in hypothesis testing and in determining confidence intervals.

Definition 3.11 Suppose that Z1 , Z2 , . . . , Zn ∼ N (0, 1). Then, the random variable X = Z12 + Z22 + . . . + Zn2 is said to have a
chi-squared distribution, denoted X ∼ χ2 (n), with n degrees of freedom. The density function of X is

 1
x ( n−2) /2e− x/2 if x ≥ 0,
f ( x ) = 2n/2 Γ(n/2) (3.4.12)

0 otherwise.
R∞
where the gamma function is defined by the integral Γ(ν) = 0 e−t tν−1 dt.
The mean and variance of X ∼ χ2 (n) are E( X ) = n and Var ( X ) = 2n

The pdf and cdf of the χ2 -distribution for various values of degrees of freedom n are shown in Figure 3.20. The
distribution is asymmetric and as the degrees of freedom increase, the χ2 -curve approaches a normal distribution. The
mean of the χ2 -distribution is the degrees of freedom and the standard deviation is twice the degrees of freedom. This
implies that the χ2 -distribution distribution is more spread out, with a peak farther to the right, for larger than for
smaller degrees of freedom.

1 n = 5 ; % degree o f freedom
2 x = 0: .1 :3 0 ;
3 p l o t ( x , p d f ( ' c h i 2 ' , x , n ) , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e p d f
4 figure ;
5 p l o t ( x , c d f ( ' c h i 2 ' , x , n ) , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e c d f

For α ∈ (0, 1), let χ2α,n be the percentage point of the χ2 distribution as shown in Figure 3.21. That is,

P( X > χ2α,n ) = 1 − P( X ≤ χ2α,n ) = 1 − F (χ2α,n ) = α

Thus, χ2α,n = F −1 (1 − α) where F −1 denotes the inverse cdf of the χ2 (n) distribution.
For α = 0.05 and n = 5, the value of χ2α,n is computed using:

MATLAB code
>> alpha = 0.05; n = 5;
>> icdf(’chi2’,1-alpha,n)
ans =
11.0705

42
 0.16 1
n=5
n = 10 0.9
0.14 n = 15
n = 20
0.8
0.12
0.7
0.1
0.6

F (x)
f(x)

0.08 0.5

0.4
0.06
0.3
0.04 n=5
0.2
n = 10
0.02 n = 15
0.1
n = 20
0 0
0 5 10 15 20 25 30 0 5 10 15 20 25 30
x x
(a) (b)
F IGURE 3.20: χ2 -distribution X ∼ χ2 (n) for various values of n: (a) pdf; (b) cdf.

χ2α,n

F IGURE 3.21: The percentage point of the χ2 -distribution.

3.4.6 S TUDENT ’ S t- DISTRIBUTION

The Student’s t-distribution (or simply, t-distribution) is a continuous probability distribution that is used to estimate
population parameters when the sample size is small and/or when the population variance is unknown. It is typically
used to develop hypothesis tests and confidence intervals.

Definition 3.12 Let Z ∼ N (0, 1) and Y ∼ χ2 (n) be independent random variables. Then, the random variable X = √Z is
Y/n
said to have a t distribution, denoted X ∼ t(n), with n degrees of freedom. The density function of X is
 
Γ n+ 1  −( n+1) /2
2 x2
f (x) = √ 1+ , −∞ < x < ∞ (3.4.13)
nπΓ(n/2) n
n
The mean and variance of X ∼ t(n) are E( X ) = 0 and Var ( X ) =
n−2
The pdf and cdf of the t-distribution for various values of degrees of freedom n are shown in Figure 3.22. As the
number of degrees of freedom grows, the t-distribution approaches the normal distribution (red plot) with mean 0 and
variance 1. Note that the degrees of freedom of the t-distribution can be a real number.

43
 1 n = 3 ; % degree o f freedom
2 x = − 4: . 1 : 4 ;
3 p l o t ( x , p d f ( ' t ' , x , n ) , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e p d f
4 figure ;
5 p l o t ( x , c d f ( ' t ' , x , n ) , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e c d f

0.4 1
N(0, 1)
0.35
t(30) 0.9
t(3)
t(1) 0.8
0.3
0.7
0.25
0.6

F (x)
f(x)

0.2 0.5

0.4
0.15
0.3 N(0, 1)
0.1
t(30)
0.2
0.05 t(3)
0.1
t(1)
0 0
−4 −3 −2 −1 0 1 2 3 4 −4 −3 −2 −1 0 1 2 3 4
x x
(a) (b)
F IGURE 3.22: t-distribution X ∼ t(n) for various values of n: (a) pdf; (b) cdf.

For α ∈ (0, 1), let tα,n be the percentage point of the t-distribution as shown in Figure 3.23. That is,

P( X > tα,n ) = 1 − P( X ≤ tα,n ) = 1 − F (tα,n ) = α

Thus, tα,n = F −1 (1 − α) where F −1 denotes the inverse cdf of the t distribution. From the symmetry of the t-distribution
about the origin, it follows that −tα,n = t1−α,n .
For α = 0.05 and n = 5, the value of tα,n is computed using:
MATLAB code
>> alpha = 0.05; n = 5;
>> icdf(’t’,1-alpha,n)
ans =
1.4759

3.4.7 F- DISTRIBUTION
The F-distribution is formed from the ratios of two chi-squared random variables, and it arises in the testing of whether
two observed samples have the same variance. This distribution is typically used to develop hypothesis tests and
confidence intervals. The most common application of the F-distribution is in standard tests of hypotheses in analysis
of variance and regression.

Definition 3.13 Let Y1 ∼ χ2 (n1 ) and Y2 ∼ χ2 (n2 ) be independent random variables distributed as chi-squared with n1 and n2
degrees of freedom, respectively. Then, the random variable X = Y 1 /n1
Y2 /n2 is said to have an F distribution, denoted X ∼ F ( n1 , n2 ),
with n1 and n2 degrees of freedom. The density function of X is
 
Γ n1 +
2
n2
(n1n1 /2 n2n2 /2 ) x (n1 −2)/2
f (x) =

, x>0 (3.4.14)
Γ n21 Γ n22 [n2 + n1 x ]( n1 +n2 ) /2

n2 n2 (2n2 + 2n1 − 4)
The mean and variance of X ∼ F (n1 , n2 ) are E( X ) = for n2 > 2 and Var ( X ) = 2 for n2 > 4
n2 − 2 n1 ( n2 − 2 ) 2 ( n2 − 4 )

44
 α α

−tα,n = t1−α,n tα,n

F IGURE 3.23: The percentage point of the t-distribution.

The pdf and cdf of the F-distribution for degrees of freedom n1 = 50 and n2 = 10 are shown in Figure 3.24. When
describing an F-distribution, the number of degrees of freedom associated with the standard deviation in the numerator
of the F random variable is always stated first. Thus, F (50, 10) would refer to an F-distribution with n1 = 50 and
n2 = 10 degrees of freedom; whereas F (10, 50) would refer to an F-distribution with n1 = 10 and n2 = 50 degrees of
freedom. Note that the curve represented by F (50, 10) would differ from the curve represented by F (10, 50).
The MATLAB function cdf(’F’,x,n1,n2) returns values of the cdf of any specified F distribution. For example, if
X ∼ F (2, 27), then P( X > 2.5) is computed as follows:
MATLAB code
>> x = 2.5; n1 = 2; n2 = 27;
>> 1-cdf(’F’,x,n1,n2)
ans =
0.1009

1 n1 = 5 0 ;
2 n2 = 1 0 ;
3 x = 0 : .0 1 : 5 ;
4 p l o t ( x , p d f ( ' F ' , x , n1 , n2 ) , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e p d f
5 figure ;
6 p l o t ( x , c d f ( ' F ' , x , n1 , n2 ) , ' LineWidth ' , 2 ) ; % V i s u a l i z e th e c d f

For α ∈ (0, 1), let Fα,n1 ,n2 be the percentage point of the F distribution as shown in Figure 3.25. That is,

P( X > Fα,n1 ,n2 ) = 1 − P( X ≤ Fα,n1 ,n2 ) = 1 − F ( Fα,n1 ,n2 ) = α

Thus, Fα,n1 ,n2 = F −1 (1 − α) where F −1 denotes the inverse cdf of the F distribution. For α = 0.05, n1 = 11, and n2 = 9,
the value of Fα,n1 ,n2 is computed using:
MATLAB code
>> alpha = 0.05; n1 = 11; n2 = 9;
>> icdf(’F’,1-alpha,n1,n2)
ans =
2.1171

45
 0.9 1

0.8 0.9

0.8
0.7
0.7
0.6
0.6
0.5

F (x)
f(x)

0.5
0.4
0.4
0.3
0.3
0.2
0.2

0.1 0.1

0 0
0 1 2 3 4 5 0 1 2 3 4 5
x x
(a) (b)
F IGURE 3.24: F-distribution F ∼ F (n1 , n2 ) with n1 = 50 and n2 = 10: (a) pdf; (b) cdf.

0 Fα,n1 ,n2

F IGURE 3.25: The percentage point of the F-distribution.

3.5 I NFERENTIAL S TATISTICS

Inferential statistics are used to draw inferences about a population from a sample. The term population refers to all
possible measurements or outcomes that are of interest to us in a particular study, while the term sample refers to a
portion of the population that is representative of the population from which it was selected, as shown in Figure 3.26.
Consider an experiment in which 10 subjects who performed a task after 24 hours of sleep deprivation scored 12 points
lower than 10 subjects who performed after a normal night’s sleep. Is the difference real or could it be due to chance?
How much larger could the real difference be than the 12 points found in the sample? These are the types of questions
answered by inferential statistics. There are two main methods used in inferential statistics: estimation and hypothesis
testing. In estimation, the sample is used to estimate a parameter and a confidence interval about the estimate is
constructed. In the most common use of hypothesis testing, a null hypothesis is put forward and it is determined
whether the data are strong enough to reject it. For the sleep deprivation study, the null hypothesis would be that sleep
deprivation has no effect on performance.

46
 Sample

F IGURE 3.26: Illustration of population vs. sample.

3.5.1 R ANDOM S AMPLING

Definition 3.14 A random sample X1 , . . . , Xn of size n is a collection of n random variables that are independent and identically
distributed (i.i.d).
• The sample mean is defined as
n
1
X=
n ∑ Xi (3.5.1)
i =1

• The sample variance is defined as

n
1
S2 = ∑ ( X − X )2 (3.5.2)
n − 1 i =1 i

• The sample standard deviation S is the positive square root of the sample variance

In descriptive statistics, note that the sample mean, for instance, was defined as an arithmetic average of a fixed set
of numbers. In inferential statistics, however, the sample mean is defined as the average of random variables. Thus, X
is also a random variable. Assume that X1 , . . . , Xn is a random sample from a population with mean µ and variance
σ2 , that is E( Xi ) = µ and var ( Xi ) = σ2 for i = 1, . . . , n. Then, the mean µ X and variance σX2 of the sample mean X are
given by
1 n 1
µ X = E( X ) = ∑ E( Xi ) = nµ = µ (3.5.3)
n i =1 n
and
n
1 1 σ2
σX2 = var ( X ) = ∑ var(Xi ) = nσ 2
= · (3.5.4)
n2 i =1
n2 n

Example 3.28 The GPAs of all students enrolled at a large university have an approximately normal distribution with mean of
3.02 and a standard deviation of 0.29. Let X the mean GPA of a random sample of 25 students selected from this university. Find
the mean and standard deviation of X.

47
Solution: From the given information, we have n = 25, µ = 3.02, and σ = 0.29.
The mean µ X and variance σX2 of X are
µ X = E( X ) = µ = 3.02
and
σ 0.29
σX = √ = √ = 0.058.
n 25

Definition 3.15 Let X1 , . . . , Xn and Y1 , . . . , Yn be two random samples of size n from a population with mean µ and variance σ2 ,
and denote by SX and SY their respective sample standard deviations.
n
1
• The sample covariance is defined as S2XY =
n − 1 i∑
( Xi − X )(Yi − Y )
=1

S2XY
• The sample correlation coefficient is defined as r XY =
SX SY

3.5.2 C ENTRAL L IMIT T HEOREM FOR S AMPLE M EANS

Theorem 3.16 Let X1 , . . . , Xn be a random sample from a population with mean µ and variance σ2 . Then for n large, the
standardized random variable
X−µ •
Z= √ ∼ N (0, 1) (3.5.5)
σ/ n
•
where ∼ means “is approximately distributed as.” Thus, the sample mean X has approximately a normal distribution with mean
√ •
µ X = µ and standard deviation σX = σ/ n, i.e. X ∼ N (µ, σ2 /n).

An important point of this result is that the variance of X decreases as the sample size increases. Basically for a large
sample size n, the central limit theorem states that the sample mean X from a population is approximately normally
distributed with a mean µ X equal to the population mean µ, even if the population is not normally distributed, and
that the variance σX2 of the sample mean is n times smaller than the population variance σ2 . But, how large is “large
enough”? As a rough rule of thumb, many statisticians say that a sample size of 30 is large enough. The random
x̄− µ
variable Z is referred to as a test statistic and its value z = σ/√n is called z-score.
Therefore, the Central Limit Theorem states that for sufficiently large sample sizes (n ≥ 30), regardless of the shape
of the population distribution, if samples of size n are randomly drawn from a population that has a mean µ and a
standard deviation σ, the samples’ means X are approximately normally distributed. If the populations are normally
distributed, then the samples’ means are normally distributed regardless of the sample sizes. The implication of this
theorem is that for sufficiently large populations, the normal distribution can be used to analyze samples drawn from
populations that are not normally distributed, or whose distribution characteristics are unknown.
Regardless of its shape, the sampling distribution of X always has a mean identical to the√mean of the sampled
population and a standard deviation equal to the population standard deviation σ divided by n. Consequently, the
spread of the distribution of sample means is considerably less than the spread of the sampled population.
Figure 3.27 displays the probability density function of the sample mean of 30 uniformly distributed random vari-
ables Xi ∼ uni f (0, 1), i = 1, . . . , 30, and the corresponding sampling distribution N (µ, σ2 /n) with µ = 1/2 and
σ2 = 1/12. That is, the sampling distribution of X is N (1/2, 1/360). Note how close the approximation to the normal
distribution is.

Example 3.29 Assume that the weights of all packages of a certain brand of chocolate cookies have a mean of 32 ounces and a
standard deviation of σ = 0.3 ounce. Find the probability that the mean weight of a random sample of 40 packages of this brand of
cookies will be between 31.8 and 31.9 ounces.

Solution: From the given information, we have n = 40, µ = 32, and σ = 0.3. Since the sample size n = 40 is large, we
can apply the central limit theorem for sample means to find P(31.8 ≤ X ≤ 31.9), which is given by
 
31.8 − 32 31.9 − 32
P(31.8 ≤ X ≤ 31.9) = P √ ≤Z≤ √ = P(−4.21 ≤ Z ≤ −2.11) = 0.0175.
0.3/ 40 0.3/ 40

48
 8 X̄ with n = 30
N (µ, σ 2 /n)
7

f(x)
4

0
0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7
x

F IGURE 3.27: Illustration of the central limit theorem using 30 uniformly generated
random variables.

Example 3.30 A tire manufacturer claims that its tires will last an average of 60,000 miles with a standard deviation of 3,000
miles. Sixty-four tires were placed on test and the average failure miles for these tires was recorded. What is the probability that
the average failure miles will be more than 59,500 miles?.

Solution: We have x̄ = 59500, µ = 60000, σ = 3000, and n = 64. Since the sample size n = 64 is large, we can apply the
central limit theorem for sample means to find P( X > 59, 500). The z-score is given by

x̄ − µ 59500 − 60000
z= √ = √ = −1.33
σ/ n 3000/ 64

Thus, P( X > 59500) = P( Z > −1.33) = 1 − Φ(−1.33) = Φ(1.33) = 0.9082.

3.5.3 C ENTRAL L IMIT T HEOREM FOR S AMPLE P ROPORTIONS

A sampling distribution of a sample proportion is a distribution obtained by using the proportions computed from
random samples of a specific size obtained from a population.

Suppose that random samples of size n are selected from a population (distribution) in which the true proportion of
the attribute of interest is p. Then, provided that np > 5 and n(1 − p) > 5, the sampling distribution
p of the sample
proportion p̂ is approximately normally distributed with mean µ = p and standard deviation p(1 − p)/n

p̂ − p •
Z= p ∼ N (0, 1). (3.5.6)
p(1 − p)/n

Example 3.31 It is estimated that approximately 53 percent of university students graduate in 4 years or less. This figure is
affected by the fact that more students are attending university on a part-time basis. If 500 students on a large campus are selected
at random, what is the probability that between 50 and 60 percent of them will graduate in 4 years or less?.
p
Solution: We have n = 500, p = 0.53, and p(1 − p)/n = 0.0223. Since np = 265 > 5 and n(1 − p) = 235 > 5,
we can apply the central limit theorem for sample proportions to find P(0.5 ≤ p̂ ≤ 0.6). The z-scores are z = (0.5 −
0.53)/0.0223 = −1.35 and z = (0.6 − 0.53)/0.0223 = 3.14. Thus,

P(0.5 ≤ p̂ ≤ 0.6) = P(−1.35 ≤ Z ≤ 3.14) = Φ(3.14) − Φ(−1.35) = 0.91.

49
Example 3.32 Almudaina Corporation manufactures USB drives. The machine that is used to make these USBs is known to
produce 6% defective USBs. The quality control inspector selects a sample of 100 USBs every week and inspect them for being
good or defective. If 8% or more of the USBs in the sample are defective, the process is stopped and the machine is readjusted. What
is the probability that based on a sample of 100 USBs, the process will be stopped to readjust the machine?.

Solution: From the given information, we have n = 100, and p = 0.06. Since np = 6 > 5 and n(1 − p) = 94 > 5, we
can apply the central limit theorem for sample proportions to find P( p̂ > 0.08), which is given by
!
0.08 − 0.06
P( p̂ > 0.08) = P Z> p = P( Z > 0.8422) = 1 − Φ(0.8422) = 0.1999.
(0.06)(1 − 0.06)/100

3.6 M ULTIVARIATE D ISTRIBUTIONS

There are many situations in which we are interested in how two or more random variables are related. We start with
a brief review of some important concepts from matrix theory.

3.6.1 C ONCEPTS FROM M ATRIX A LGEBRA

Matrices play a central role in multivariate statistics. A matrix is a rectangular array of elements arranged in rows and
columns. An m × n matrix with m rows and n columns is given by
 
a11 a1,2 ··· a1n
 a21 a2,2 ··· a2n 
 
A = ( aij ) =  . .. .. ..  (3.6.1)
 .. . . . 
am1 am2 · · · amn

where aij are the elements or entries of the matrix A.

A column vector is an m × 1 matrix that consists of a single column of m elements:

x1
 x2 
 
x= . 
 .. 
xm

whereas a row vector is an 1 × m matrix that consists of a single row of m elements:

x = ( x1 , x2 , . . . , x m ).

Square Matrix:
A square matrix is an n × n matrix with the same number of rows and columns.

Transpose of a Matrix:
Let A be m × n matrix. The transpose of A, denoted by A′ , is obtained by interchanging its rows and columns. The
order (or size) of A′ is n × m. For example, if
 
1 3 4
A=
7 0 1

then
 
1 7
A ′ = 3 0 .
4 1

50
Symmetric Matrix:
When A′ = A, the matrix is called symmetric. That is, a symmetric matrix is a square matrix, in that it has the same
number of rows as it has columns, and the off-diagonal elements are symmetric (i.e. aij = a ji for all i and j). For
example,
 
1 9 5
A =  9 2 −4 
5 −4 3
is a symmetric matrix.

Diagonal Matrix:
A special case of a symmetric matrix is the diagonal matrix, which is an n × n matrix whose non-diagonal entries are
zeros. A diagonal matrix is trivially symmetric. For example,
 
1 0 0
A = 0 − 2 0 
0 0 3

is a diagonal matrix.
An identity matrix is a diagonal matrix of order n which has 1’s on the diagonal and 0’s on the off-diagonal
 
1 0 ··· 0
0 1 · · · 0
 
In =  . . .. .
 .. .. . .. 
0 0 ··· 1

Matrix Inverse:
A n × n square matrix A is called invertible or non-singular if there exists a matrix B such that: AB = BA = In . The
matrix B is called the inverse of A and is denoted by A−1 . For example, the inverse of
 
1 2
A=
3 4

is given by
 
−1 −1 1
A =
1.5 −0.5
MATLAB code
>> A = [1 2; 3 4]
>> inv(A)
ans =
-2.0000 1.0000
1.5000 -0.5000

Eigenvalues and Eigenvectors of a Square Matrix:

An eigenvalue and eigenvector of an n × n square matrix A are a scalar λi and a nonzero vector x i that satisfy

Axi = λi xi i = 1, . . . , n (3.6.2)

which can expressed in matrix form as follows:

AV = VD (3.6.3)
where D is a diagonal matrix whose diagonal elements are the eigenvalues, and V is a matrix whose columns are the
eigenvectors. These matrices can be easily computed using the MATLAB function eig:

51
 MATLAB code
>> A = [1 2; 3 4]
>> [V,D]=eig(A)
V =
-0.8246 -0.4160
0.5658 -0.9094

D =
-0.3723 0
0 5.3723

In this output, the columns of V correspond

 to eigenvectors
 of A, with eigenvalues given on the  D. For
diagonal of 
−0.8246 −0.4160
instance, in our example, A has eigenvector with eigenvalue -0.3723, and eigenvector with
0.5658 −0.9094
eigenvalue 5.3723.

3.6.2 B IVARIATE D ISTRIBUTIONS

Let X and Y be two random variables, then the probability distributions may be summarized as follows:

Discrete case:

• The joint pmf of X and Y is defined as f XY ( x, y) = P( X = x, Y = y)

• The marginal pmfs of X and of Y are f X ( x ) = ∑y f ( x, y) and f Y (y) = ∑x f ( x, y)

Continuous case:
R∞ R∞
• The joint cdf of X and Y is defined as FXY ( x, y) = P( X ≤ x, Y ≤ y) = −∞ −∞ f ( x, y)dxdy, where f ( x, y) is
the joint pdf also called bivariate probability density.
R∞ R∞
• The marginal pdfs are f X ( x ) = −∞ f ( x, y)dy and f Y (y) = −∞ f ( x, y)dx

We say that the random variables X and Y are independent if for every pair of x and y values, the joint pmf or pdf is
equal to the product of the marginal pmfs or pdfs: f ( x, y) = f X ( x ) f Y (y). Otherwise, the random variables X and Y are
said to be dependent. Note that the independence property can be generalized to any number of random variables.
When two random variables are not independent, it is frequently of interest to measure how they are related to one
another. In other words, information about the value of one random variable helps determine the value of the other.
This information is often measured using the covariance or correlation between the two random variables.

Definition 3.17 The covariance between X and Y, denoted cov( X, Y ) or σXY , is defined as

cov( X, Y ) = σXY = E( X − µ X )E(Y − µY ) = E( XY ) − E( X )E(Y ) (3.6.4)

where µ X and µY are the expected values of X and Y, respectively.

The correlation coefficient between X and Y, denoted ρ( X, Y ), is defined as

cov( X, Y ) σ
ρ( X, Y ) = = XY (3.6.5)
σX σY σX σY

where σX and σY are the standard deviations of X and Y, respectively.

Note that the correlation coefficient ρ( X, Y ) remains unaffected by a change of units, and therefore it is dimensionless.
Also, it can be shown that −1 ≤ ρ ≤ 1. The correlation coefficient measures the linear relationship between X and
Y. The strongest possible relationship corresponds to ρ = 1, while the strongest negative relationship corresponds
to ρ = −1. When ρ = 0, the random variables X and Y are said to be uncorrelated, but not necessarily independent.
Two variables could be uncorrelated yet highly dependent because there is a strong nonlinear relationship. Any value
|ρ| < 1 indicates only that the relationship is not completely linear, but there may still be a very strong nonlinear
relationship.

52
3.6.3 M ULTIVARIATE N ORMAL D ISTRIBUTION
A multivariate random variable is a row-vector X = ( X1 , . . . , X p ) (or column-vector X = ( X1 , . . . , X p )′ ) whose p
components are scalar-valued random variables. The expected value or mean of the random vector X is equal to
E( X ) = µ = (E( X1 ), . . . , E( X p )), and the covariance matrix of X is a p × p symmetric non-negative definite matrix,
denoted Σ = (σij ), whose (i, j)-th element σij is the covariance between the i-th and the j-th random variables, that is
 
σ12 σ12 · · · σ1p
 σ21 σ22 · · · σ2p 
 
Σ= . .. .. ..  (3.6.6)
 .. . . . 
σp1 σp2 · · · σp2

Note that the diagonal element σii = σi2 represents the variance of the i-th random variable Xi , while the off-diagonal
element σi,j represents the covariance between the i-th and the j-th random variables Xi and X j , where i, j = 1, . . . , p.

Definition 3.18 A p-dimensional random vector X = ( X1 , . . . , X p ) is said to have a multivariate normal distribution, denoted
X ∼ N (µ, Σ), if the pdf of X is
 
1 1 ′ −1
f ( x) = exp − ( x − µ) Σ ( x − µ ) (3.6.7)
(2π ) p/2 |Σ|1/2 2

where |Σ| and Σ−1 are the determinant and inverse matrix of Σ, respectively.

In the 2-dimensional (bivariate) case, the pdf of a random vector ( X, Y ) is

" # !
1 1 ( x 1 − µ1 ) 2 ( x 2 − µ2 ) 2 2ρ( x1 − µ1 )2
f ( x1 , x2 ) = p exp − + − ,
2πσ1 σ2 1−ρ 2 2 ( 1 − ρ2 ) σ12 σ22 σ1 σ2

where ρ is the correlation coefficient between X1 and X2 . In this case,

 
σ12 ρσ1 σ2
µ = ( µ1 , µ2 ) and Σ= .
ρσ1 σ2 σ22

Figure 3.28 displays the surface and contour plots of a bivariate normal pdf with
 
1 1/3
µ = (1/2, −1/2) and Σ= .
1/3 1

0.15
1
f(x1 , x2 )

0.1
x2

0.05
−1

2 −2
3
2
0 1
0
−2 −1 −3
−2 −3 −2 −1 0 1 2 3
x2 −3 x1 x1

(a) (b)
F IGURE 3.28: Bivariate normal pdf: (a) surface plot, and (b) contour plot.

53
3.7 P ROBLEMS

❶ Let a random variable X denote the number of medication errors for a patient at a hospital. Experience shows
that it occurs between 0 and 3 (inclusive) with the following probabilities:

X=x 0 1 2 3
f ( x ) = P( X = x ) 0.90 0.07 0.02 0.01

(i) Plot the probability mass function.

(ii) Calculate the mean µ and variance σ2 of the random variable X.
(ii) Calculate the probability that X falls in the interval (µ − 6σ, µ + 6σ ).

❷ A machine produces soda bottles, and 98.5 percent of all bottles produced pass an audit. What is the probability
of having only 2 bottles that pass audit in a randomly selected sample of 7 bottles?
❸ A product failure has historically averaged 3.84 occurrences per day. What is the probability of 5 failures in a
randomly selected day?
❹ The average number of accidents occurring in a manufacturing plant over a period of one year is equal to two.
Find the probability that during any given year five accidents will occur.
❺ The mean daily milk production of a herd of Greenville cows has a normal distribution with µ = 70 pounds and
σ = 13 pounds.
a) What is the probability that the milk production for a cow chosen at random will be less than 60 pounds?
b) What is the probability that the milk production for a cow chosen at random will be greater than 90 pounds?
c) What is the probability that the milk production for a cow chosen at random will be between 60 pounds and
90 pounds?
❻ A large drug company has 100 potential new prescription drugs under clinical test. About 20% of all drugs that
reach this stage are eventually licensed for sale. What is the probability that at least 15 out of the 100 drugs will
be eventually licensed?
❼ The reliability of an electrical fuse is the probability that a fuse, chosen at random from production, will function
under its designed conditions. A random sample of 1000 fuses was tested and x = 27 defectives were observed.
Calculate the approximate probability of observing 27 or more defectives, assuming that the fuse reliability is
0.98.

3.8 REFERENCES

[1] D.C. Montgomery, Introduction to Statistical Quality Control, Wiley, 6th Edition, 2009.
[2] I. Bass and B. Lawton, Lean Six Sigma using SigmaXL and Minitab, McGraw-Hill Professional, 1st Edition, 2009.

54
 C HAPTER 4

C ONFIDENCE I NTERVALS AND H YPOTHESIS T ESTING

The only relevant test of the validity of a

hypothesis is comparison of its predictions with
experience.

Milton Friedman

A confidence interval is an interval that provides an estimated range of values which is likely to include, with a
certain level of confidence, an unknown population parameter of interest. This estimated range of values is calculated
from a given set of sample data. Confidence intervals are one way to represent how “good” an estimate is; the larger a
confidence interval for a particular estimate, the more caution is required when using the estimate. Hypothesis testing,
on the hand, is a method of drawing inferences about a population based on statistical evidence from a sample. A
statistical hypothesis is an assumption about a population parameter. This assumption may or may not be true. The
purpose of hypothesis testing is to assess the validity of a statistical hypothesis made about a population.

4.1 P OINT AND I NTERVAL E STIMATION

Point estimation involves the use of sample data to calculate a single value (statistic) which serves as an estimate of an
unknown population parameter, while interval estimation involves the use of sample data to calculate an interval of
possible (or probable) values of an unknown population parameter.
Let X1 , . . . , Xn be a random sample from a population with an unknown parameter θ. A point estimator of θ is the
statistic Θ̂ = ϕ( X1 , . . . , Xn ), where ϕ is a given function. A numerical value θ̂ of the statistic Θ̂, calculated from the
observed sample data X1 = x1 , . . . , Xn = xn , is called the point estimate of the parameter θ. For example, the sample
mean X is an estimator of a normal population mean µ, while the numerical value µ̂ = x̄ calculated from a observed
sample data is the point estimate of µ. A good estimator must satisfy three conditions:

• Unbiased: The expected value of the estimator must be equal to the parameter

• Consistent: The value of the estimator approaches the value of the parameter as the sample size increases

• Relatively Efficient: The estimator has the smallest variance of all estimators which could be used.

Definition 4.1 Assume that X1 , . . . , Xn is a random sample from a population with an unknown parameter θ. Then,
• The point estimator Θ̂ is an unbiased estimator for the parameter θ if E(Θ̂) = θ.
• The minimum variance unbiased estimator (MVUE) is an unbiased estimator that has lower variance than any other unbi-
ased estimator for all possible values of the parameter θ.
q
• The standard error of a point estimator Θ̂ is given by s.e.(Θ̂) = var (Θ̂ ).

A biased point estimator is an estimator such that E(Θ̂) = θ + bias. For example, both the sample mean X and
sample variance S2 are unbiased estimators for the population mean µ and the population variance σ2 , respectively, i.e.

55
E( X ) = µ and E(S2 ) = σ2. However, the sample standard deviation S is a biased estimator of the population standard
deviation σ, i.e. E(S) 6= σ.
The MVUE is the most efficient estimator. An efficient estimator Θ̂ will produce an estimate closer to the true
parameter µ, and it can be shown that the sample mean X is MVUE for the population mean µ. On the other hand, an
interval estimate refers to a range of values used to estimate the population parameter. Such an interval estimate is
obtained by making use of the probability distribution of the point estimator.

Example 4.1 In a sample of five measurements, the diameter of a sphere was recorded by a scientist as 6.33, 6.37, 6.36, 6.32, and
6.37 centimeters (cm).

(i) Determine an unbiased and efficient estimate of the population mean.

(ii) Determine an unbiased and efficient estimate of the population variance.

(iii) Give an unbiased and inefficient estimate of the population mean.

Solution: From the given information, the sample size is n = 5.

(i) The unbiased and efficient estimate of the the population mean is
n
1 1
X=
n ∑ Xi = 5 (6.33 + 6.37 + 6.36 + 6.32 + 6.37) = 6.35
i =1

(ii) The unbiased and efficient estimate of the population variance is

n
1
S2 = ∑ ( X − X )2 = 0.00055.
n − 1 i =1 i

(iii) The median Q2 is one example of an unbiased and inefficient estimate of the population mean. By ordering the
data, we obtain Q2 = 6.36.
MATLAB code
>> X = [6.33 6.37 6.36 6.32 6.37];
>> mean(X), var(X), median(X)

4.2 S AMPLING D ISTRIBUTIONS

Any function of the random sample observations or elements is called a statistic. The sample mean X, the sample
variance S2 , and the sample standard deviation S are examples of a statistic. Generally, a statistic is used to estimate
the value of a population parameter. For example, X is a statistic that serves as an estimate of the population mean µ.
The probability distribution of a statistic is called a sampling distribution.
Let X1 , . . . , Xn ∼ N (µ, σ2 ) be a random sample from a normal population with mean µ and variance σ2 . According
to the central limit theorem, the sampling distribution of X is N (µ, σ2/n). Therefore, the sampling distribution of the
statistic
X−µ •
Z= √ ∼ N (0, 1) (4.2.1)
σ/ n
approximately follows a standard normal distribution. Figure 4.1 displays probability density functions of the sam-
ple mean from a standard normal population for various sample sizes. The red curve for the sample size of n = 1
corresponds to N (0, 1).
It can be shown that the expected value of the sample variance is equal to the population variance, that is E(S2 ) = σ2 .
Moreover, sampling distribution of the statistic

S2
( n − 1) ∼ χ2 ( n − 1 ) (4.2.2)
σ2

has a χ2 -distribution with n − 1 degrees of freedom. That is, the sampling distribution of the sample variance is chi-
squared: S2 ∼ [σ2 /(n − 1)]χ2 (n − 1).

56
 2.5
n=1
n=2
2 n = 30

1.5

f(x)
1

0.5

0
−4 −3 −2 −1 0 1 2 3 4
x

F IGURE 4.1: Sampling pdfs of the sample mean X for different values of the sample
size.

Example 4.2 The time it takes a central processing unit to process a certain type of job is normally distributed with mean 20
seconds and standard deviation 3 seconds. If a sample of 15 such jobs is observed, what is the probability that the sample variance
will exceed 12?
Solution: From the given information, we have n = 15 and σ2 = 9. Thus,
 
S2 12
P(S2 > 12) = P (n − 1) 2 > (14)
σ 9
= P(χ2 (14) > 18.67)
= 1 − P(χ2 (14) ≤ 18.67) = 0.1779,
where χ2 (14) is a χ2 -distribution with n − 1 = 14 degrees of freedom.

MATLAB code
>> n = 15; sigma2 = 9; x = 12;
>> chisq = (n-1)*x/sigma2;
>> P = 1-cdf(’chi2’,chisq,n-1)

Example 4.3 The Bravo Widget Company claims that their widgets last 5 years, with a standard deviation of 1 year. Assume that
their claims are true. If you test a random sample of 9 Acme widgets, what is the probability that the standard deviation in your
sample will be less than 0.95 years?
Solution: From the given information, we have n = 9 and σ = 1. Thus,
 
S2 (0.95)2
P(S < 0.95) = P (n − 1) 2 < (8)
σ (1)2
= P(χ2 (8) < 7.22) = 0.4869,
where χ2 (8) is a χ2 -distribution with n − 1 = 8 degrees of freedom.

If in Eq. (4.2.1) we replace σ by the sample standard deviation S, then the sampling distribution of the statistic
X −µ
√
X−µ σ/ n N (0, 1)
t= √ = ∼ q ∼ t ( n − 1) (4.2.3)
S/ n S/σ 1 2
n −1 χ ( n − 1 )

57
has a t-distribution with n − 1 degrees of freedom.

Example 4.4 Eureka Corporation manufactures light bulbs. The CEO claims that an average Eureka light bulb lasts 300 days. A
researcher randomly selects 15 bulbs for testing. The sampled bulbs last an average of 290 days, with a standard deviation of 50
days. If the CEO’s claim were true, what is the probability that 15 randomly selected bulbs would have an average life of no more
than 290 days?
Solution: From the given information, we have n = 15, µ = 300, x̄ = 260 and s = 50. Thus,
 
X−µ 290 − 300
P( X ≤ 290) = P √ ≤ √
S/ n 50/ 15
= P(t(14) ≤ −0.7746) = 0.2257,
where t(14) is a t-distribution with n − 1 = 14 degrees of freedom.

MATLAB code
>> n = 15; mu = 300; xbar = 290; s = 50;
>> x = (xbar-mu)/(s/sqrt(n));
>> P = cdf(’t’,x,n-1)

Let X1 , . . . , Xn1 ∼ N (µ X , σX2 ) and Y1 , . . . , Yn2 ∼ N (µY , σY2 ) be two samples from two normal populations. Then, the
sampling distribution of the statistic
S2 /σ2
F = X2 X2 ∼ F (n1 − 1, n2 − 1) (4.2.4)
SY /σY
has an F-distribution with n1 − 1 and n2 − 1 degrees of freedom, where S2X and SY2 are the sample variances of the
corresponding populations.

Example 4.5 Consider two independent samples: the first of size 10 from a normal population having variance 4 and the second
of size 5 from a normal population having variance 2. Compute the probability that the sample variance from the second sample
exceeds the one from the first.
Solution: From the given information, we have n1 = 10, n2 = 5, σX2 = 4 and σY2 = 2. Thus,
!
2 2 S2X
P(SY > SX ) = P <1
SY2
!
S2X /σX2 σY2
= P < 2
SY2 /σY2 σX
= P( F (9, 4) < 1/2) = 0.1782,

where F (9, 4) is an F-distribution with n1 − 1 = 9 and n1 − 1 = 4 degrees of freedom.

MATLAB code
>> n1 = 10; n2 = 5; sigmax2 = 4; sigmay2 = 2;
>> x = sigmay2/sigmax2;
>> P = cdf(’F’,x,n1-1,n2-1)

4.3 C ONFIDENCE I NTERVALS

A confidence interval is an interval estimate with a specific confidence level, (1 − α)%, where α ∈ (0, 1). The confidence
coefficient, denoted 1 − α, is the probability that the interval estimate will contain the population parameter θ. More
specifically, we want to construct a 100(1 − α)% confidence interval ℓ ≤ θ ≤ u such that

P(ℓ ≤ θ ≤ u) = 1 − α (4.3.1)

58
where ℓ and u are called lower and upper confidence limits, respectively. These confidence limits are calculated from the
observed sample data.
When a one-sided specification is employed, then only a one-sided confidence limit is needed. In this case, a lower-
confidence interval on θ is ℓ ≤ θ such that P(ℓ ≤ θ ) = 1 − α and an upper-confidence interval on θ is θ ≤ u such that
P(θ ≤ u) = 1 − α. A typical value of the the confidence level (1 − α)% is 95%, which means that if all samples of the
same size were selected, 95% of them include the population parameter θ somewhere within the confidence interval,
and 5% would not.

4.3.1 C ONFIDENCE I NTERVAL ON THE P OPULATION M EAN WHEN THE VARIANCE IS K NOWN

Let X1 , . . . , Xn be a random sample of size n (≥ 30) from a normal population with an unknown parameter mean µ and
known parameter variance σ2 . Since the sample size is large, it follows from the central limit theorem that the statistic
X−µ
Z= √
σ/ n
∼ N (0, 1), and as illustrated in Figure 4.2 we can write

 
X−µ
P −zα/2 ≤ √ ≤ zα/2 = 1 − α,
σ/ n

where zα/2 is the upper 100α/2 percentage point of the standard normal distribution.
Rearranging the terms inside the parentheses yields

 
σ σ
P X − zα/2 √ ≤ µ ≤ X + zα/2 √ = 1−α
n n

X −µ
Z = √ ∼ N (0, 1)
σ/ n

1 −α

α/2 α/2

−zα/2 0 zα/2

F IGURE 4.2: Illustration of confidence interval on the normal mean when the variance
is known.

59
Definition 4.2 Let X1 , . . . , Xn be a random sample from an N (µ, σ2 ) distribution with an unknown parameter mean µ and
known parameter variance σ2 .

• A 100(1 − α)% confidence interval on µ is given by

σ σ
x̄ − zα/2 √ ≤ µ ≤ x̄ + zα/2 √ (4.3.2)
n n

• A 100(1 − α)% upper-confidence interval on µ is given by

σ
µ ≤ x̄ + zα √ (4.3.3)
n

• A 100(1 − α)% lower-confidence interval on µ is given by

σ
x̄ − zα √ ≤ µ (4.3.4)
n

where x̄ is the observed sample mean.

Example 4.6 A survey was conducted of companies that use solar panels as a primary source of electricity. The question that was
asked was this: How much of the electricity used in your company comes from the solar panels? A random sample of 55 responses
produced a mean of 45 megawatts. Suppose the population standard deviation for this question is 15.5 megawatts.
(i) Find the 95% confidence interval for the mean.
(ii) Find the 95% upper-confidence interval for the mean.
(iiii) Find the 95% lower-confidence interval for the mean.
Solution: From the given information, we have x̄ = 45, σ = 15.5, and n = 55.
For 95% confidence level, we have α = 1 − 0.95 = 0.05, zα/2 = z0.025 = 1.96, and zα = z0.05 = 1.64.
(i) The 95% confidence interval is given by
σ σ
x̄ − zα/2 √ ≤ µ ≤ x̄ + zα/2 √
n n
15.5 15.5
45 − 1.96 √ ≤ µ ≤ 45 + 1.96 √
55 55
40.9 ≤ µ ≤ 49.1
Thus, we can be 95% sure that the mean will be between 40.9 and 49.1 megawatts. In other words, the probability
for the mean to be between 40.9 and 49.1 will be 0.95.

P(40.9 ≤ µ ≤ 49.1) = 0.95.

MATLAB code
>> xbar = 45; sigma = 15.5; n = 55; alpha = 1-0.95;
>> zalpha2 = icdf(’norm’,1-alpha/2,0,1);
>> LC = xbar - zalpha2*sigma/sqrt(n);
>> UC = xbar + zalpha2*sigma/sqrt(n);

(ii) The 95% upper-confidence interval is given by

σ 15.5
µ ≤ x̄ + zα √ = 45 + 1.64 √ ⇒ µ ≤ 48.44
n 55

(iii) The 95% lower-confidence interval is given by

σ 15.5
x̄ − zα √ = 45 − 1.64 √ ≤ µ ⇒ 41.56 ≤ µ.
n 55

60
4.3.2 C ONFIDENCE I NTERVAL ON THE P OPULATION M EAN WHEN THE VARIANCE IS U NKNOWN
When the sample size is small, we cannot apply the central limit theorem. Thus, we either assume that the sampled
population is normally distributed, or we need to verify that the sample data is approximately normally distributed
using for example the normal probability plot or the box plot. Therefore, when the population is normal and sample
X −µ
size is small, the statistic T = S/√n ∼ t(n − 1) follows a t-distribution with n − 1 degrees of freedom. Hence, as
illustrated in Figure 4.3 we can write
 
X−µ
P −tα/2,n−1 ≤ √ ≤ tα/2,n−1 = 1 − α,
S/ n
where tα/2,n−1 is the upper 100α/2 percentage point of the t-distribution with n − 1 degrees of freedom. It follows that
 
S S
P X − tα/2,n−1 √ ≤ µ ≤ X + tα/2,n−1 √ = 1−α
n n

X−µ
T = √ ∼ t(n − 1)
S/ n

1−α

α/2 α/2

−tα/2,n−1 tα/2,n−1

F IGURE 4.3: Illustration of confidence interval on the normal mean when the variance
is unknown.

Definition 4.3 Let X1 , . . . , Xn be a random sample from a normal population with an unknown parameter variance σ2 .
• A 100(1 − α)% confidence interval on µ is given by
s s
x̄ − tα/2,n−1 √ ≤ µ ≤ x̄ + tα/2,n−1 √ (4.3.5)
n n

• A 100(1 − α)% upper-confidence interval on µ is given by

s
µ ≤ x̄ + tα,n−1 √ (4.3.6)
n

• A 100(1 − α)% lower-confidence interval on µ is given by

s
x̄ − tα,n−1 √ ≤ µ (4.3.7)
n

where x̄ and s are the observed sample mean and sample standard deviation, respectively.

61
Example 4.7 A random sample of size 25 of a certain kind of lightbulb yielded an average lifetime of 1875 hours and a standard
deviation of 100 hours. From past experience it is known that the lifetime of this kind of bulb is normally distributed. Find the a
99% confidence interval for the population mean.
Solution: From the information given, we have n = 25, x̄ = 1875, s = 100. For 99% confidence level, we have
α = 1 − 0.99 = 0.01. Also, the population is assumed to be normally distributed. The 99% confidence interval is given
by
s s
x̄ − tα/2,n−1 √ ≤ µ ≤ x̄ + tα/2,n−1 √
n n
100 100
1875 − 2.7969 √ ≤ µ ≤ 1875 + 2.7969 √
25 25
1819.1 ≤ µ ≤ 1930.9
We can be 99% sure that the mean will be between 1819.1 and 1930.9. In other words, the probability for the mean to
be between 1819.1 and 1930.9 will be 0.99. That is,

P(1819.1 ≤ µ ≤ 1930.9) = 0.99.

MATLAB code
>> n=25; xbar = 1875; s = 100; alpha = 1-0.99;
>> talpha2 = icdf(’t’,1-alpha/2,n-1)
>> LC = xbar - talpha2*s/sqrt(n)
>> UC = xbar + talpha2*s/sqrt(n)

Example 4.8 A manager of a car rental company wants to estimate the average number of times luxury cars would be rented a
month. She takes a random sample of 19 cars that produces the following number of times the cars are rented in a month:
3 7 12 5 9 13 2 8 6 14 6 1 2 3 2 5 11 13 5

(i) Check the assumption of normality for the number of times the cars are rented in a month.
(ii) Find the 95% confidence interval to estimate the average.
(iii) Find the 95% upper-confidence interval to estimate the average.
(iv) Find the 95% lower-confidence interval to estimate the average.

Solution: The sample size is n = 19. Thus, the sample mean and sample standard deviation of the data are: x̄ =
127/19 = 6.68 and s = 4.23. For 95% confidence level, we have α = 1 − 0.95 = 0.05, tα/2,n−1 = t0.025,18 = 2.101, and
tα,n−1 = t0.05,18 = 1.734.
(i) According to the normal probability plot shown in Figure 4.4, there does not seem to be a severe deviation from
normality for this data. This is evident by the fact that the data appears to fall along a straight line.
MATLAB code
>> X = [3 7 12 5 9 13 2 8 6 14 6 1 2 3 2 5 11 13 5];
>> normplot(X);

(ii) The 95% confidence interval is given by

s s
x̄ − tα/2,n−1 √ ≤ µ ≤ x̄ + tα/2,n−1 √
n n
4.23 4.23
6.68 − 2.101 √ ≤ µ ≤ 6.68 + 2.101 √
19 19
4.65 ≤ µ ≤ 8.72
We can be 95 percent sure that the mean will be between 4.65 and 8.72. In other words, the probability for the
mean to be between 4.65 and 8.72 will be 0.95.

P(4.65 ≤ µ ≤ 8.72) = 0.95.

62
 0.98
0.95
0.90

0.75

Probability
0.50

0.25

0.10
0.05
0.02
2 4 6 8 10 12 14
Data

F IGURE 4.4: Normal probability plot for the number of times the cars are rented in a
month.

MATLAB code
>> X = [3 7 12 5 9 13 2 8 6 14 6 1 2 3 2 5 11 13 5];
>> xbar = mean(X); s = std(X); n = 19; alpha = 1-0.95;
>> talpha2 = icdf(’t’,1-alpha/2,n-1)
>> LC = xbar - talpha2*s/sqrt(n)
>> UC = xbar + talpha2*s/sqrt(n)

(iii) The 95% upper-confidence interval is given by

s 4.23
µ ≤ x̄ + tα,n √ = 6.68 + 1.734 √ ⇒ µ ≤ 8.37
n 19

(iv) The 95% lower-confidence interval is given by

s 4.23
x̄ − tα,n−1 √ = 6.68 − 1.734 √ ≤ µ ⇒ 5 ≤ µ.
n 19

4.3.3 C ONFIDENCE I NTERVAL ON THE P OPULATION VARIANCE

In quality control, in most cases the objective of the auditor is not to find the mean of a population but rather to deter-
mine the level of variation of the output. For instance, they would want to know how much variation the production
process exhibits about the target to see what adjustments are needed to reach a defect-free process.
2
When the population is normally distributed, the statistic (n − 1) σS2 ∼ χ2 (n − 1) follows a χ2 (n − 1) distribution
with n − 1 degrees of freedom, and as illustrated in Figure 4.5 we can construct the the 100(1 − α)% confidence interval
for σ2 as follows  
2 ( n − 1 ) S2 2
P χ1−α/2,n−1 ≤ ≤ χα/2,n−1 = 1 − α,
σ2
where χ2α/2,n−1 is the upper 100α/2 percentage point of the χ2-distribution with n − 1 degrees of freedom. It follows
that !
( n − 1 ) S2 2 ( n − 1 ) S2
P ≤σ ≤ 2 = 1−α
χ2α/2,n−1 χ1−α/2,n−1

63
 S2
(n − 1) ∼ χ2 (n − 1)
σ2

α/2

1 −α
α/2

χ21−α/2,n−1 χ2α/2,n−1

F IGURE 4.5: Illustration of confidence interval on the normal variance.

Definition 4.4 Let X1 , . . . , Xn ∼ N (µ, σ2 ) be a random sample from a normal population with mean µ and variance σ2 .
• A 100(1 − α)% confidence interval on σ2 is given by

( n − 1 ) s2 ( n − 1 ) s2
2
≤ σ2 ≤ 2 (4.3.8)
χα/2,n−1 χ1−α/2,n−1

• A 100(1 − α)% upper-confidence interval on σ2 is given by

( n − 1 ) s2
σ2 ≤ (4.3.9)
χ21−α,n−1

• A 100(1 − α)% lower-confidence interval on σ2 is given by

( n − 1 ) s2
≤ σ2 (4.3.10)
χ2α,n−1

where s2 is the observed sample variance.

Example 4.9 A sample of 9 screws was taken out of a production line and the sizes of the diameters in millimeters are as follows:
13 13 12 12.55 12.99 12.89 12.88 12.97 12.99

(i) Find the 95% confidence interval to estimate the population variance.
(ii) Find the 95% upper-confidence interval to estimate the population variance.
(iii) Find the 95% lower-confidence interval to estimate the population variance.

Solution: The sample size is n = 9. Thus, the sample variance of the data is: s2 = 0.11. For 95% confidence level, we
have α = 1 − 0.95 = 0.05, χ2α/2,n−1 = χ20.025,8 = 17.53, χ21−α/2,n−1 = 2.18, χ2α,n−1 = 15.51, and χ21−α,n−1 = 2.73.

(i) The 95% confidence interval is given by

( n − 1 ) s2 2 ( n − 1 ) s2
≤ σ ≤ ⇒ 0.05 ≤ σ2 ≤ 0.41
χ2α/2,n−1 χ21−α/2,n−1

64
 Thus, we can be 95% sure that the population variance will be between 0.05 and 0.41. In other words, the proba-
bility for the population variance to be between 0.05 and 0.41 will be 0.95.

P(0.05 ≤ σ2 ≤ 0.41) = 0.95.

MATLAB code
>> X = [13 13 12 12.55 12.99 12.89 12.88 12.97 12.99];
>> n = length(X); s2 = var(X); alpha = 1-0.95;
>> LC = (n-1)*s2/icdf(’chi2’,1-alpha/2,n-1)
>> UC = (n-1)*s2/icdf(’chi2’,alpha/2,n-1)

(ii) The 95% upper-confidence interval is given by

( n − 1 ) s2
σ2 ≤ ⇒ σ2 ≤ 0.33
χ21−α,n−1

(iii) The 95% lower-confidence interval is given by

( n − 1 ) s2
≤ σ2 ⇒ 0.06 ≤ σ2
χ2α,n−1

Example 4.10 The time taken by a worker in a car manufacturing company to finish a paint job on a car is normally distributed
with mean µ and variance σ2 . A sample of 15 paint jobs is randomly selected and assigned to that worker, and the time taken by
the worker to finish the job is jotted down. These data yield a sample standard deviation of 2.5 hours.

(i) Find the 95% confidence interval to estimate the population standard deviation.

(ii) Find the 95% upper-confidence interval to estimate the population standard deviation.

(iii) Find the 95% lower-confidence interval to estimate the population standard deviation

Solution: From the given information, we have n = 15 and s = 2.5. For 95% confidence level, we have α = 1 − 0.95 =
0.05, χ2α/2,n−1 = χ20.025,14 = 26.1189, χ21−α/2,n−1 = 5.6287, χ2α,n−1 = 23.6848, and χ21−α,n−1 = 6.5706.

(i) The 95% confidence interval is given by

( n − 1 ) s2 2 ( n − 1 ) s2
≤ σ ≤ ⇒ 3.3501 ≤ σ2 ≤ 15.5453
χ2α/2,n−1 χ21−α/2,n−1

We can be 95% sure that the population variance will be between 3.3501 and 15.5453. Therefore, by taking the
square root of the lower and upper confidence limits for σ2 , the 95% confidence interval of the population stan-
dard deviation σ is (1.8303, 3.9427).

(ii) The 95% upper-confidence interval is given by

( n − 1 ) s2
σ2 ≤ ⇒ σ2 ≤ 13.3168
χ21−α,n−1

Thus, the 95% upper-confidence interval of the population standard deviation σ is (0, 3.6492).

(iii) The 95% lower-confidence interval is given by

( n − 1 ) s2
≤ σ2 ⇒ 3.6944 ≤ σ2
χ2α,n−1

Thus, the 95% lower-confidence interval of the population standard deviation σ is (1.9221, +∞).

65
4.3.4 C ONFIDENCE I NTERVAL ON THE P OPULATION P ROPORTION
Consider a population of items, each of which independently meets certain standards with some unknown probability
p, and suppose that a sample of size n was taken from this population. If X denote the number of the n items that meet
the standards, then X ∼ bino (n, p). Thus, for n large and both np ≥ 5 and n(1 − p) ≥ 5, it follows that

X − np •
p ∼ N (0, 1)
np(1 − p)
• p
where
p ∼ means “is approximately distributed as.” Since p̂ = X/n is a point estimate of p, it follows that n p̂(1 − p̂) ≈
np(1 − p) and
n p̂ − np
p ∼ N (0, 1)
n p̂(1 − p̂)
Thus, !
n p̂ − np
P −zα/2 ≤ p ≤ zα/2 ≈ 1−α
n p̂(1 − p̂)
or, equivalently,
 q q 
P p̂ − zα/2 p̂(1 − p̂)/n ≤ p ≤ p̂ + zα/2 p̂(1 − p̂)/n ≈ 1−α

Definition 4.5 If p̂ is the proportion of observations in a random sample of size n, then

• A 100(1 − α)% confidence interval on p is given by
q q
p̂ − zα/2 p̂(1 − p̂)/n ≤ p ≤ p̂ + zα/2 p̂(1 − p̂)/n (4.3.11)

• A 100(1 − α)% upper-confidence interval on p is given by

q
p ≤ p̂ + zα p̂(1 − p̂)/n (4.3.12)

• A 100(1 − α)% lower-confidence interval on p is given by

q
p̂ − zα p̂(1 − p̂)/n ≤ p (4.3.13)

Example 4.11 The fraction of defective integrated circuits produced in a photolithography process is being studied. A random
sample of 300 circuits is tested, revealing 13 defectives. Find a 95% two-sided confidence interval on the fraction of defective
circuits produced by this particular tool.

Solution: From the given information, we have n = 300 and x = 13. Thus, the point estimate of the proportion is
p̂ = x/n = 13/300 = 0.0433. Since both n p̂ = 13 ≥ 5 and n(1 − p̂) = 287 ≥ 5 are satisfied, p̂ is approximately normal.
For 95% confidence level, we have α = 1 − 0.95 = 0.05, and zα/2 = z0.025 = 1.96. Thus, the 95% confidence interval is
given by
q q
p̂ − zα/2 p̂(1 − p̂)/n ≤ p ≤ p̂ + zα/2 p̂(1 − p̂)/n ⇒ 0.02 ≤ p ≤ 0.07

We can be 95% sure that the fraction of defective circuits will be between 0.02 and 0.07. In other words, the probability
for the proportion to be between 0.02 and 0.07 will be 0.95.

MATLAB code
>> n = 300; x = 13; phat = x/n; alpha = 1-0.95;
>> zalpha2 = icdf(’norm’,1-alpha/2,0,1);
>> LC = phat - zalpha2*sqrt(phat*(1-phat)/n)
>> UC = phat + zalpha2*sqrt(phat*(1-phat)/n)

66
Example 4.12 A random sample of 400 computer chips is taken from a large lot of chips and 50 of them are found defective. Find
a 90% two-sided confidence interval for the proportion of defective chips contained in the lot.

Solution: From the given information, we have n = 400 and x = 50. Thus, the point estimate of the proportion of
defective chips contained in the lot is p̂ = x/n = 50/400 = 0.125. Since both n p̂ = 50 ≥ 5 and n(1 − p̂) = 350 ≥ 5 are
satisfied, p̂ is approximately normal. For 90% confidence level, we have α = 1 − 0.9 = 0.1, and zα/2 = z0.025 = 1.6449.
The 90% confidence interval is given by
q q
p̂ − zα/2 p̂(1 − p̂)/n ≤ p ≤ p̂ + zα/2 p̂(1 − p̂)/n ⇒ 0.0978 ≤ p ≤ 0.1522

We can be 90% sure that the proportion of defective chips contained in the lot will be between 0.0978 and 0.1522.

4.3.5 C ONFIDENCE I NTERVAL ON THE D IFFERENCE IN P OPULATIONS M EANS

Just as in the analysis of a single population, to estimate the difference between two populations the researcher would
draw samples from each population.
Assume that X1 , . . . , Xn1 ∼ N (µ1, σ12 ) and Y1 , . . . , Yn2 ∼ N (µ2 , σ22 ) are two independent samples from two indepen-
dent normal populations, as depicted in Figure 4.6. Let X, Y, S12 and S22 be the sample means and sample variances,
respectively. Then,
! ! !
σ12 σ22 σ12 σ22
X ∼ N µ1 , and X ∼ N µ2 , =⇒ X − Y ∼ N µ1 − µ2 , +
n1 n2 n1 n2

To construct the confidence interval on the difference in means µ1 − µ2 , we consider two particular cases:

F IGURE 4.6: Illustration of two independent distributions N (µ1 , σ12 ) and N (µ2 , σ22 ).

Case 1: If σ1 and σ2 are known, then the statistic

X − Y − ( µ1 − µ2 )
r ∼ N (0, 1)
σ12 σ22
n1 + n2

Thus,  
 X − Y − ( µ1 − µ2 ) 
P
−zα/2 ≤ r ≤ zα/2 
 = 1−α
σ12 σ22
n1 + n2

or, equivalently,
 s s 
σ12 σ2 σ12 σ22
P  X − Y − zα/2 + 2 ≤ µ1 − µ2 ≤ X − Y + zα/2 +  = 1−α
n1 n2 n1 n2

67
Definition 4.6 Let x̄ and ȳ be the observed sample means of independent random samples of sizes n1 and n2 from two independent
normal populations with known variances σ12 and σ12 , respectively. Then,
• A 100(1 − α)% confidence interval on µ1 − µ2 is given by
s s
σ12 σ22 σ12 σ2
x − y − zα/2 + ≤ µ1 − µ2 ≤ x − y + zα/2 + 2 (4.3.14)
n1 n2 n1 n2

• A 100(1 − α)% upper-confidence interval on σ2 is given by

s
σ12 σ2
µ1 − µ2 ≤ x − y + z α + 2 (4.3.15)
n1 n2

• A 100(1 − α)% lower-confidence interval on σ2 is given by

s
σ12 σ2
x − y − zα + 2 ≤ µ1 − µ2 (4.3.16)
n1 n2

where s2 is the observed sample variance.

Example 4.13 The variances of two populations 1 and 2 are 16 and 9, respectively. A sample of 25 items was taken from Popu-
lation 1 with a mean of 50, and a sample of 22 items was taken from Population 2 with a mean of 45. Construct a 95% two-sided
confidence interval for the difference in population means.

Solution: We have n1 = 25, n2 = 22, σ1 = 4, and σ2 = 3. For 95% confidence level, we have α = 1 − 0.95 = 0.05, and
zα/2 = z0.025 = 1.96. Thus, the 95% two-sided confidence for the difference in population means is given by
s s
σ12 σ2 σ12 σ2
x − y − zα/2 + 2 ≤ µ1 − µ2 ≤ x − y + zα/2 + 2 ⇒ 2.99 ≤ µ1 − µ2 ≤ 7.01.
n1 n2 n1 n2

Hence, the probability for the difference in population means to be between 2.99 and 7.01 will be 0.95.

Case 2: If σ1 and σ2 are unknown and equal to σ2, then

     
σ2 σ2 σ2 σ2
X ∼ N µ1 , and X ∼ N µ2 , =⇒ X − Y ∼ N µ1 − µ2 , +
n1 n2 n1 n2

and
S12 S22
( n1 − 1 ) ∼ χ2 ( n1 − 1 ) and ( n2 − 1 ) ∼ χ2 ( n2 − 1 )
σ2 σ2
S12 S22
=⇒ (n1 − 1) + ( n 2 − 1 ) ∼ χ2 ( n1 + n2 − 2 )
σ2 σ2
or, equivalently,
S2p
( n1 + n2 − 2 ) ∼ χ2 ( n1 + n2 − 2 )
σ2
(n1 − 1)S12 + (n2 − 1)S22
where S2p = is called the pooled variance.
n1 + n2 − 2
X − Y − ( µ1 − µ2 ) S2p
Since Z = q ∼ N (0, 1) and V = (n1 + n2 − 2) 2 ∼ χ2 (n1 + n2 − 2), it follows that
σ2 σ2 σ
n1 + n2

Z X − Y − ( µ1 − µ2 )
p = √ ∼ t ( n1 + n2 − 2 )
V/(n1 + n2 − 2) S p 1/n1 + 1/n2

68
Therefore, !
X − Y − ( µ1 − µ2 )
P −tα/2,n1 +n2 −2 ≤ √ ≤ tα/2,n1 +n2 −2 = 1−α
S p 1/n1 + 1/n2

Definition 4.7 Let x̄ and ȳ be the observed sample means of independent random samples of sizes n1 and n2 from two independent
normal populations with unknown variances σ12 = σ22 = σ2 , respectively. Then,
• A 100(1 − α)% confidence interval on µ1 − µ2 is given by
s s
1 1 1 1
x − y − tα/2,n1 +n2 −2 s p + ≤ µ1 − µ2 ≤ x − y + tα/2,n1 +n2 −2 s p + (4.3.17)
n1 n2 n1 n2

• A 100(1 − α)% upper-confidence interval on σ2 is given by

s
1 1
µ1 − µ2 ≤ x − y + tα,n1 +n2 −2 s p + (4.3.18)
n1 n2

• A 100(1 − α)% lower-confidence interval on σ2 is given by

s
1 1
x − y − tα,n1 +n2 −2 s p + ≤ µ1 − µ2 (4.3.19)
n1 n2

where s p is the observed pooled standard deviation.

Example 4.14 The variances of two populations are assumed to be equal. A sample of 15 items was taken from Population I with a
mean of 50 and a standard deviation of 3, and a sample of 19 items was taken from Population II with a mean of 47 and a standard
deviation of 2.

(i) Calculate the pooled sample variance.

(ii) Construct a 95% two-sided confidence for the difference between the two population means.

Solution: From the given information, we have n1 = 15, n2 = 19, x̄ = 50, ȳ = 47, S1 = 3, and S2 = 2.

(i) The value of the pooled sample variance is given by

(n1 − 1)S12 + (n2 − 1)S22 (15 − 1)(32 ) + (19 − 1)(22 )

s2p = = = 6.19.
n1 + n2 − 2 19 + 15 − 2

(ii) For 95% confidence level, we have α = 1 − 0.95 = 0.05, and tα/2,n1 +n2 −2 = t0.025,32 = 2.04. Thus, the 95%
two-sided confidence for the difference between the two population means is given by
s s
1 1 1 1
x − y − tα/2,n1 +n2 −2 s p + ≤ µ1 − µ2 ≤ x − y + tα/2,n1 +n2 −2 s p +
n1 n2 n1 n2

1.25 ≤ µ1 − µ2 ≤ 4.75.
Hence, a 95% two-sided confidence for µ1 − µ2 is (1.25, 4.75).

MATLAB code
>> n1 = 15; n2 = 19; alpha = 0.05;
>> talpha2 = icdf(’t’,1-alpha/2,n1+n2-2)

69
Example 4.15 A pharmaceutical company sets two machines to fill 15oz bottles with cough syrup. Two random samples of
n1 = 16 bottles from machine 1 and n2 = 12 bottles from machine 2 are selected. The two samples yield the following sample
statistics:
X = 15.24 S12 = 0.64

Y = 14.96 S22 = 0.36

(i) Calculate the pooled sample variance.

(ii) Construct a 95% two-sided confidence for the mean difference of the amount of cough syrup filled in bottles by the two
machines.

Solution: From the given information, we have n1 = 16, n2 = 12, x̄ = 15.24, ȳ = 14.96, S12 = 0.64, and S22 = 0.36.

(i) The value of the pooled sample variance is given by

(n1 − 1)S12 + (n2 − 1)S22 (16 − 1)(0.64) + (12 − 1)(0.36)

s2p = = = 0.5215
n1 + n2 − 2 16 + 12 − 2

(ii) For 95% confidence level, we have α = 1 − 0.95 = 0.05, and tα/2,n1 +n2 −2 = t0.025,26 = 2.0555. Thus, the 95%
two-sided confidence for the difference between the two population means is given by
s s
1 1 1 1
x − y − tα/2,n1 +n2 −2 s p + ≤ µ1 − µ2 ≤ x − y + tα/2,n1 +n2 −2 s p +
n1 n2 n1 n2

−0.2869 ≤ µ1 − µ2 ≤ 0.8469.

Hence, a 95% two-sided confidence for µ1 − µ2 is (−0.2869, 0.8469).

4.3.6 C ONFIDENCE I NTERVAL ON THE R ATIO OF VARIANCES

Assume that X1 , . . . , Xn1 ∼ N (µ1 , σ12 ) and Y1 , . . . , Yn2 ∼ N (µ2 , σ22 ) are two independent samples from two independent
normal populations. Let S12 and S22 be the sample variances, respectively. Then the statistic

S12 /σ12
∼ F (n1 − 1, n2 − 1)
S22 /σ22

Denote by f α/2,n1 −1,n2 −1 and f 1−α/2,n1 −1,n2 −1 the upper and lower α/2 percentage points of the F (n1 − 1, n2 − 1) distri-
bution, as shown in Figure 4.7.
Thus,
!
S12 /σ12
P f 1−α/2,n1 −1,n2 −1 ≤ 2 2 ≤ f α/2,n1 −1,n2 −1 = 1 − α
S2 /σ2

or, equivalently,
!
S2 σ2 S2
P f 1−α/2,n1 −1,n2 −1 12 ≤ 12 ≤ f α/2,n1 −1,n2 −1 12 = 1−α
S2 σ2 S2

70
 S 12 /σ12
∼ F (n1 − 1, n2 − 1)
S 22 /σ22

α/2

1−α

α/2

f1−α/2,n 1 −1,n 2 −1 fα/2,n 1 −1,n 2 −1

F IGURE 4.7: Illustration of confidence interval on the ratio of variances.

Definition 4.8 Let s21 and s22 be the observed sample variances of two independent random samples of sizes n1 and n2 from two
independent normal populations with unknown variances σ12 and σ22 , respectively. Then,

• A 100(1 − α)% confidence interval on σ12 /σ22 is given by

s21 σ2 s2
f 1−α/2,n1 −1,n2 −1 2
≤ 12 ≤ f α/2,n1 −1,n2 −1 21 (4.3.20)
s2 σ2 s2

• A 100(1 − α)% upper-confidence interval on σ12 /σ22 is given by

σ12 s21
≤ f α,n − 1,n − 1 (4.3.21)
σ22 1 2
s22

• A 100(1 − α)% lower-confidence interval on σ12 /σ22 is given by

s21 σ12
f 1−α,n1 −1,n2 −1 ≤ (4.3.22)
s22 σ22

Example 4.16 The variances of two populations are assumed to be equal. A sample of 15 items was taken from Population I with
a standard deviation of 3, and a sample of 19 items was taken from Population II with a standard deviation of 2. Construct a 95%
two-sided confidence for the ratio of variances.

Solution: We have n1 = 15, n2 = 19, S1 = 3, and S2 = 2. For 95% confidence level, we have α = 1 − 0.95 = 0.05,
f α/2,n1 −1,n2 −1 = f 0.025,14,18 = 2.70, and f 1−α/2,n1 −1,n2 −1 = 0.35. Thus, the 95% two-sided confidence for the ratio of
variances is given by
s21 σ12 s21 σ12
f 1−α/2,n1 −1,n2 −1 ≤ ≤ f α/2,n1 −1,n2 −1 2 ⇒ 0.78 ≤ ≤ 6.07.
s22 σ22 s2 σ22

MATLAB code
>> n1 = 15; n2 = 19; alpha = 0.05;
>> falpha1 = icdf(’f’,1-alpha/2,n1-1,n2-1) %f_{alpha/2,n1-1,n2-1}
>> falpha2 = icdf(’f’,alpha/2,n1-1,n2-1) %f_{1-alpha/2,n1-1,n2-1}

71
4.4 H YPOTHESIS T ESTING

A statistical hypothesis is a statement or claim about a set of parameters of one or more populations. It is called a
hypothesis because it is not known whether or not it is true. A Hypothesis test is the decision-making procedure about
the hypothesis. In statistics terms, the hypothesis that we try to establish is called an alternative hypothesis H1 , while
its contradiction is called a null hypothesis H0 .
Consider a population with unknown parameter θ. Basically, there are three ways to set up the null and alternatives
hypothesis:

1. Two-tailed hypothesis test

H0 : θ = θ0
(4.4.1)
H1 : θ 6 = θ0

2. Upper-tailed hypothesis test

H0 : θ = θ0
(4.4.2)
H1 : θ > θ0

3. Lower-tailed hypothesis test

H0 : θ = θ0
(4.4.3)
H1 : θ < θ0

The two-tailed test is also called two-sided test, whereas the upper-and lower-tests are also referred to as one-sided
tests.

Example 4.17 A manufacturer of a certain brand of rice cereal claims that the average saturated fat content does not exceed 1.5
grams per serving. State the null and alternative hypotheses to be used in testing this claim.
Solution: The manufacturers claim should be rejected only if µ is greater than 1.5 milligrams and should not be rejected
if µ is less than or equal to 1.5 milligrams. Thus, we test

H0 : µ = 1.5
H1 : µ > 1.5.

Example 4.18 A real estate agent claims that 60% of all private residences being built today are 3-bedroom homes. To test this
claim, a large sample of new residences is inspected; the proportion of these homes with 3 bedrooms is recorded and used as the test
statistic. State the null and alternative hypotheses to be used in this test.
Solution: If the test statistic were substantially higher or lower than p = 0.6, we would reject the agents claim. Hence,
we should make the hypotheses:

H0 : p = 0.6
H1 : p 6= 0.6

The alternative hypothesis implies a two-tailed test with the critical region divided equally in both tails of the distribu-
tion of the test statistic.

The goal of any hypothesis test is to make a decision; in particular, we will decide whether to reject the null hypoth-
esis in favor of the alternative hypothesis H1 . Although we would like to be able to always make a correct decision,
we must remember that the decision will be based on the sample data. When a test is done, there are four possible
outcomes, as summarized in Table 4.1.

72
 H0 is true H0 is false
Reject H0 Type I Error Correct Decision
Do not reject H0 Correct Decision Type II Error
TABLE 4.1: Possible outcomes for a hypothesis test.

From Table 4.1, we can observe that there are two ways of making a mistake when doing a hypothesis test. Thus, we
may make one of the following two types of errors:

Type I error: is the error of rejecting H0 when it is true. The probability of making a Type I error, denote by α, is given
by
α = P(Type I error) = P(Reject H0 when H0 is true)

Type II error: is the error of accepting H0 when it is false. The probability of making a Type II error, denote by β, is
given by
β = P(Type II error) = P(Accept H0 when H0 is false)

The Type I error and Type II error are related. A decrease in the probability of one generally results in an increase
in the probability of the other. The probability, α, is also called the significance level for the hypothesis test. If the
significance level is fixed, then the rejection of H0 is done with a fixed degree of confidence in the decision. Because we
specify the level of significance before performing the hypothesis test, we basically control the risk of making a Type I
error. Typical values for α are 0.1, 0.05, and 0.01. For example, if α = 0.1 for a test, and the null hypothesis is rejected,
then one will be 90% certain that this is the correct decision.
After the hypotheses are stated, the next step is to design the study. An appropriate statistical test will be selected,
the level of significance will be chosen, and a plan to conduct the study will be formulated. To make an inference for
the study, the statistical test and level of significance are used. Once the level of significance is selected, a critical value
for the appropriate test is selected from a table in the Appendix.

A hypothesis testing procedure consists of four main steps:

Step 1: Specify the null and alternative hypotheses, H0 and H1 , and the significance level, α
Step 2: Determine an appropriate test statistic and compute its value using the sample data
Step 3: Specify the rejection region
Step 4: Make the appropriate conclusion by deciding whether H0 should be rejected.

4.4.1 T ESTS ON THE M EAN OF A N ORMAL P OPULATION WITH K NOWN VARIANCE

Let X1 , . . . , Xn be a sample of size n from a N (µ, σ2 ) population with unknown mean µ and known variance σ2 . In
testing the population mean µ, there are three ways to structure the hypothesis test:

Two-tailed Upper-Tailed Lower-Tailed

H0 : µ = µ0 H0 : µ = µ0 H0 : µ = µ0
H1 : µ 6= µ0 H1 : µ > µ0 H1 : µ < µ0

Recall that the sample mean X ∼ N (µ, σ2 /n). Under the assumption that the null hypothesis is true (i.e. H0 : µ =
µ0 ), it follows that the test statistic
X − µ0
Z0 = √ ∼ N (0, 1)
σ/ n
has a standard normal distribution. Thus, this hypothesis test is called z-test.
A critical region or rejection region is the set of all values such that the null hypothesis is rejected. We can then
determine a critical region based on the computed test statistic.

73
 Let z0 be the numerical value, calculated from the sample, of the test statistic Z0 . Then, for a selected significance
level α, the critical regions are (see Figure 4.8).

Two-tailed Upper-Tailed Lower-Tailed

If z0 < −zα/2 or z0 > zα/2 If z0 > zα If z0 < −zα
reject H0 reject H0 reject H0

1 −α 1 −α 1 −α
α/2 α/2 α α

−zα/2 0 zα/2 0 zα −zα 0

(a) (b) (c)

F IGURE 4.8: Critical region for the z-test alternative hypothesis: (a) µ 6= µ0 ; (b) µ > µ0 ;
(c) µ < µ0 .

Usually, α is specified in advance before any samples are drawn so that results will not influence the choice for the
level of significance. To conclude a statistical test, we compare our α-value with the p-value, which is the probability
of observing the given sample result under the assumption that the null hypothesis is true. The p-value is computed
using sample data and the sampling distribution. If the p-value is less than the significance level α, then we reject the
null hypothesis. For example, if α = 0.05 and the p-value is 0.03, then we reject the null hypothesis. The converse is
not true. If the p-value is greater than α, then we have insufficient evidence to reject the null hypothesis.

• If p-value ≤ α, we reject the null hypothesis and say the data are statistically significant at the level α.
• If p-value > α, we do not reject the null hypothesis.

Denote by Φ(·) the cdf of the N (0, 1) distribution. Then, the z-test may be summarized as follows:

74
Hypotheses:
Two-tailed Upper-Tailed Lower-Tailed
H0 : µ = µ0 H0 : µ = µ0 H0 : µ = µ0
H1 : µ 6= µ0 H1 : µ > µ0 H1 : µ < µ0
X − µ0
Test Statistic (z-test): Z0 = √
σ/ n
Critical Regions:
Two-tailed Upper-Tailed Lower-Tailed
|z0 | > zα/2 z0 > z α z0 < − z α
p-values:
Two-tailed Upper-Tailed Lower-Tailed
2[1 − Φ(|z0 |)] 1 − Φ ( z0 ) Φ ( z0 )

Example 4.19 The CEO of a large financial corporation claims that the average distance that commuting employees travel to work
is 32 km. The commuting employees feel otherwise. A sample of 64 employees was randomly selected and yielded a mean of 35 km.
Assuming a population standard deviation of 5 km,

(i) Test the CEO’s claim at the 5% level of significance.

(ii) Calculate the p-value for this test.
(iii) Test the CEO’s claim using a confidence interval with a 95% confidence coefficient.

Solution: From the given information, we have n = 64, x̄ = 35, µ0 = 32, σ = 5, and α = 0.05.
(i) Step 1: This is a two-tailed test, since the employees feel that the CEO’s claim is not correct, but whether they
feel that the average distance is less than 32 km or more than 32 km is not specified. Thus,
H0 : µ = 32
H1 : µ 6= 32

Step 2: Since σ is known, the appropriate test statistic is the z-test and its value is given by
x̄ − µ0 35 − 32
z0 = √ = √ = 4.8
σ/ n 5/ 64
Step 3: The rejection region is |z0 | > zα/2 , where zα/2 = z0.025 = 1.96.
Step 4: Since |z0 | = 4.8 > zα/2 = 1.96, we reject the null hypothesis H0 . There is sufficient sample evidence
to refute the CEO’s claim. The sample evidence supports the employees’ claim that the average distance
commuting employees travel to work is not equal to 32 km at the 5 percent level of significance. That is,
there is a significant difference between the sample mean and the postulated value of the population mean
of 32 km.
(ii) The p-value is equal to 2[1 − Φ(|z0 |)] = 2[1 − Φ(4.8)] = 0, which is less than the significance level α = 0.05;
therefore we reject the null hypothesis.
(iii) A confidence interval with a 95% confidence coefficient implies α = 0.05. The 95% confidence interval is given by
σ σ
x̄ − zα/2 √ ≤ µ ≤ x̄ + zα/2 √
n n
5 5
35 − 1.96 √ ≤ µ ≤ 35 + 1.96 √
64 64
33.775 ≤ µ ≤ 36.225
This interval clearly does not contain 32, the value of µ under the null hypothesis. Thus, we reject the null
hypothesis.

75
Example 4.20 A random sample of 36 pieces of copper wire produced in a plant of a wire manufacturing company yields the mean
tensile strength of 950 psi. Suppose that population of tensile strengths of all copper wires produced in that plant are distributed
with mean µ and standard deviation σ = 120 psi. Test the statistical hypothesis:

H0 : µ = 980 versus H1 : µ < 980

at the 99% level of confidence. Then, calculate the p-value.

Solution: From the given information, we have n = 36, x̄ = 950, µ0 = 980, σ = 120, and α = 0.01.

Step 1: This is a lower-tailed test. Thus,

H0 : µ = 980
H1 : µ < 980

Step 2: Since σ is known, the appropriate test statistic is the z-test and its value is given by

x̄ − µ0 950 − 980
z0 = √ = √ = −1.5
σ/ n 120/ 36

Step 3: The rejection region is z0 < −zα , where zα = z0.01 = 2.3263.

Step 4: Since z0 = −1.5 ≮ −zα = −2.3263, we do not reject the null hypothesis H0 .

The p-value is equal to Φ(z0 ) = Φ(−1.5) = 1 − Φ(1.5) = 0.0668, which is greater than the significance level α = 0.01;
therefore, there is insufficient evidence to reject the null hypothesis.

4.4.2 T ESTS ON THE M EAN OF A N ORMAL P OPULATION WITH U NKNOWN VARIANCE

Let X1 , . . . , Xn be a sample of size n from a N (µ, σ2) population with unknown mean µ and unknown variance σ2 . In
testing the population mean µ, there are three ways to structure the hypothesis test:

Two-tailed Upper-Tailed Lower-Tailed

H0 : µ = µ0 H0 : µ = µ0 H0 : µ = µ0
H1 : µ 6= µ0 H1 : µ > µ0 H1 : µ < µ0

Under the assumption that the null hypothesis is true (i.e. H0 : µ = µ0 ), it follows that the test statistic

X − µ0
T0 = √ ∼ t ( n − 1)
S/ n

has a t-distribution with n − 1 degrees of freedom. Thus, this hypothesis test is called t-test. When the population
standard deviation σ is not known, we typically use the t-test either (i) when the sample size is large (i.e. n ≥ 30) or
(ii) when the sample size is small (i.e. n < 30) and the population from which the sample is selected is approximately
normal.
A critical region or rejection region is the set of all values such that the null hypothesis is rejected. Let t0 be the
numerical value, calculated from the sample, of the test statistic T0 . Then, for a selected significance level α, the critical
regions are (see Figure 4.9):

Two-tailed Upper-Tailed Lower-Tailed

If t0 < −tα/2,n−1 or t0 > tα/2,n−1 If t0 > tα,n−1 If t0 < −tα,n−1
reject H0 reject H0 reject H0

Denote by F (·) the cdf of the t(n − 1) distribution. Then, the t-test may be summarized as follows:

76
 1 −α 1 −α 1 −α

α/2 α/2 α α

−tα/2,n−1 tα/2,n−1 tα,n−1 tα,n−1

(a) (b) (c)

F IGURE 4.9: Critical region for the t-test alternative hypothesis: (a) µ 6= µ0 ; (b) µ > µ0 ;
(c) µ < µ0 .

Hypotheses:
Two-tailed Upper-Tailed Lower-Tailed
H0 : µ = µ0 H0 : µ = µ0 H0 : µ = µ0
H1 : µ 6= µ0 H1 : µ > µ0 H1 : µ < µ0
X − µ0
Test Statistic (t-test): T0 = √
S/ n
Critical Regions:
Two-tailed Upper-Tailed Lower-Tailed
|t0 | > tα/2,n−1 t0 > tα,n−1 t0 < −tα,n−1
p-values:
Two-tailed Upper-Tailed Lower-Tailed
2[1 − F (|t0 |)] 1 − F ( t0 ) F ( t0 )

Example 4.21 The Atlas Electric Institute has published figures on the number of kilowatt hours used annually by various home
appliances. It is claimed that a vacuum cleaner uses an average of 46 kilowatt hours per year. If a random sample of 12 homes
included in a planned study indicates that vacuum cleaners use an average of 42 kilowatt hours per year with a standard deviation
of 11.9 kilowatt hours, does this suggest at the 0.05 level of significance that vacuum cleaners use, on average, less than 46 kilowatt
hours annually? Then, calculate the p-value. Assume the population of kilowatt hours to be normal.

Solution: From the information given, we have n = 12, x̄ = 42, µ0 = 46, s = 11.9, and α = 0.05.

Step 1: The hypotheses are:

H0 : µ = 46
H1 : µ < 46

Step 2: Since σ is unknown and the population of kilowatt hours is assumed to be normally distributed, the appro-
priate test statistic is the t-test and its value is given by

x̄ − µ0 42 − 46
t0 = √ = √ = −1.1644
s/ n 11.9/ 12

Step 3: The rejection region is t0 < −tα,n−1 , where tα,n−1 = t0.05,11 = 1.7959.

Step 4: Since t0 = −1.16 ≮ −tα,n−1 = −1.7959, we do not reject the null hypothesis H0 . We conclude that the average
number of kilowatt hours used annually by home vacuum cleaners is not significantly less than 46.

77
The p-value is equal to F (t0 ) = F (−1.1644) = 0.1344, which is greater than the significance level α = 0.05; therefore
we have insufficient evidence to reject the null hypothesis.

Example 4.22 Grand Auto Corporation produces auto batteries. The company claims that its top-of-the-line Never Die batteries
are good, on average, for at least 65 months. A consumer protection agency tested 45 such batteries to check this claim. It found
that the mean life of these 45 batteries is 63.4 months and the standard deviation is 3 months. Using 1% significance level, can you
conclude that the company’s claim is true?. Then, calculate the p-value. Assume the population of life of batteries to be normal.
Solution: From the given information, we have n = 45, x̄ = 63.4, µ0 = 65, s = 3, and α = 0.01.
Step 1: This is a lower-tailed test:

H0 : µ ≥ 65 (The mean life of batteries is at least 65 months )

H1 : µ < 65 (The mean life of batteries is less than 65 months)

Step 2: Since σ is unknown and n ≥ 30, the appropriate test statistic is the t-test and its value is given by

x̄ − µ0 63.4 − 65
t0 = √ = √ = −3.5777
s/ n 3/ 45

Step 3: The rejection region is t0 < −tα,n−1 , where tα,n−1 = t0.01,44 = 2.4141.
Step 4: Since t0 = −3.5777 < −tα,n−1 = −2.4141, we reject the null hypothesis H0 . We conclude that the mean life of
such batteries is less than 65 months.
The p-value is equal to F (t0 ) = F (−3.5777) = 0, which is less than the significance level α = 0.01; therefore we reject
the null hypothesis.

Example 4.23 A tool assembling company believes that a worker should take no more than 30 minutes to assemble a particular
tool. A sample of 16 workers who assembled that tool showed that the average time was 33 minutes with a sample standard
deviation of 6 minutes. Test at the 5% level od significance if the data provide sufficient evidence to indicate the validity of the
company’s belief. Then, calculate the p-value. Assume that the assembly times are normally distributed.
Solution: From the information given, we have n = 16, x̄ = 33, µ0 = 30, s = 6, and α = 0.05.
Step 1: This is a upper-tailed test:

H0 : µ ≤ 30 (The mean assembly time is no more than 30 minutes)

H1 : µ > 30 (The mean assembly time is more than 30 minutes)

Step 2: Since σ is unknown and the assembly times are assumed to be normally distributed, the appropriate test
statistic is the t-test and its value is given by

x̄ − µ0 33 − 30
t0 = √ = √ =2
s/ n 6/ 16

Step 3: The rejection region is t0 > tα,n−1 , where tα,n−1 = t0.05,15 = 1.7531.
Step 4: Since t0 = 2 > tα,n−1 = 1.7531, we reject the null hypothesis H0 . We conclude that the mean assembly time is
more than 30 minutes.
The p-value is equal to F (t0 ) = 1 − F (2) = 0.032, which is less than the significance level α = 0.05; therefore we reject
the null hypothesis.

Example 4.24 A city health department wishes to determine if the mean bacteria count per unit volume of water at a lake beach
is within the safety level of 200. A researcher collected 10 water samples of unit volume and found the bacteria counts to be

78
 175 190 205 193 184 207 204 193 196 180

(i) Check the assumption of normality for the bacteria counts.

(ii) Do the data strongly indicate that there is no cause for concern at 5% level of significance?

Solution: From the given information, we have n = 10, x̄ = 192.7, s = 10.812, µ0 = 200, and α = 0.05.
(i) Because the sample size is small, we must be willing to assume that the population distribution of bacteria counts
is normally distributed. As shown in Figure 4.10, the normal probability plot and boxplot indicate that the mea-
surements constitute a sample from a normal population. The normal probability plot appears to be reasonably
straight. Although the boxplot is not perfectly symmetric, it is not too skewed and there are no outliers.
MATLAB code
>> X = [175 190 205 193 184 207 204 193 196 180];
>> subplot(1,2,1); normplot(X); subplot(1,2,2); boxplot(X);

0.95

0.90
205

0.75 200
Probability

Bacteria count
195
0.50

190
0.25
185

0.10
180
0.05

175
175 180 185 190 195 200 205
Data 1

F IGURE 4.10: Normal probability and box plots for the bacteria counts.

(ii) Step 1: Let µ denote the current (population) mean bacteria count per unit volume of water. Then, the statement
“no cause for concern” translates to µ < 200, and the researcher is seeking strong evidence in support of this
hypothesis. So the formulation of the null and alternative hypotheses should be
H0 : µ = 200
H1 : µ < 200

Step 2: Since σ is unknown, the appropriate test statistic is the t-test and its value is given by
x̄ − µ0 192.7 − 200
t0 = √ = √ = −2.1351
s/ n 10.812/ 10
Step 3: The rejection region is t0 < −tα,n−1 , where tα,n−1 = t0.05,9 = 1.8331.
Step 4: Since t0 = −2.1351 < −tα,n−1 = −1.8331, we reject the null hypothesis H0 . On the basis of the data
obtained from these 10 measurements, there does seem to be strong evidence that the true mean is within
the safety level.
The p-value is equal to F (t0 ) = F (−2.1351) = 0.0308, which is less than the significance level α = 0.05; therefore
we reject the null hypothesis. There is strong evidence that the mean bacteria count is within the safety level.

MATLAB code
>> X = [175 190 205 193 184 207 204 193 196 180];
>> mu0 = 200; alpha = 0.05;
>> [h,p,ci,stats]=ttest(X,mu0,alpha,’left’)

79
4.4.3 T ESTS ON THE VARIANCE OF A N ORMAL P OPULATION
Let X1 , . . . , Xn be a sample of size n from a N (µ, σ2 ) population with unknown variance σ2 . In testing the population
variance σ2 , there are three ways to structure the hypothesis test:

Two-tailed Upper-Tailed Lower-Tailed

H0 : σ2 = σ02 H0 : σ2 = σ02 H0 : σ2 = σ02
H1 : σ2 6= σ02 H1 : σ2 > σ02 H1 : σ2 < σ02

Under the assumption that the null hypothesis is true (i.e. H0 : σ2 = σ02 ), it follows that the test statistic
( n − 1 ) S2
X02 = ∼ χ2 ( n − 1 )
σ02
has a χ2 -distribution with n − 1 degrees of freedom. Thus, this hypothesis test is called χ2 -test.
Let χ20 be the numerical value, calculated from the sample, of the test statistic X02 . Then, for a selected significance
level α, the critical regions are:

Two-tailed Upper-Tailed Lower-Tailed

If χ20 < χ21−α/2,n−1 or χ20 > χ2α/2,n−1 If χ20 > χ2α,n−1 If χ20 < χ21−α,n−1
reject H0 reject H0 reject H0

Then, the χ2 -test may be summarized as follows:

Hypotheses:
Two-tailed Upper-Tailed Lower-Tailed
H0 : σ2 = σ02 H0 : σ2 = σ02 H0 : σ2 = σ02
H1 : σ2 6= σ02 H1 : σ2 > σ02 H1 : σ2 < σ02
( n − 1 ) S2
Test Statistic: χ20 =
σ02
Critical Regions:
Two-tailed Upper-Tailed Lower-Tailed
χ20 < χ21−α/2,n−1 or χ20 > χ2α/2,n−1 χ20 > χ2α,n−1 χ20 < χ21−α,n−1
p-values:
Two-tailed Upper-Tailed Lower-Tailed
2 min( P(χ2α,n−1 < χ20 ), 1 − P(χ2α,n−1 < χ20 )) P(χ20 > χ2α,n−1 ) P(χ20 < χ21−α,n−1 )

Example 4.25 A manufacturer of car batteries claims that the life of the companys batteries is approximately normally distributed
with a standard deviation equal to 0.9 year. If a random sample of 10 of these batteries has a standard deviation of 1.2 years, do you
think that σ > 0.9 year? Then, calculate the p-value. Use a 0.05 level of significance.
Solution: From the given information and data, we have n = 10, s2 = (1.2)2 = 1.44, σ0 = 0.9, and α = 0.05.
Step 1: This is an upper-tailed test, that is
H0 : σ2 = 0.81
H1 : σ2 > 0.81

Step 2: The appropriate test statistic is the χ2 -test and its value is given by
( n − 1 ) s2 (10 − 1)(1.2)2
χ20 = 2
= = 16
σ0 (0.9)2

80
 Step 3: The rejection region is χ20 > χ2α,n−1, where χ2α,n−1 = χ20.05,9 = 16.919.

Step 4: Since χ20 = 16 > χ2α,n−1 = 16.919, we do not reject the null hypothesis H0 .

The p-value is equal to 1 − F (χ20 ) = 1 − F (16) = 0.0669, where F is the cdf of the χ2 -distribution with 9 degrees of
freedom. Since the p-value is greater than the significance level α = 0.05, we have insufficient evidence to reject the
null hypothesis.

Example 4.26 The production manager of a light bulb manufacturer believes that the lifespan of the 14W bulb with light output
of 800 lumens is 6000 hours. A random sample of 25 bulbs produced the sample mean of 6180 hours and sample standard deviation
of 178 hours. Test at the 5% level of significance that the population standard deviation is less than 200 hours. Then, calculate the
p-value for the test. Assume that the lifespan of these bulbs is normally distributed.
Solution: From the given information and data, we have n = 25, s = 178, σ0 = 200, and α = 0.05. The lifespan of the
the light bulbs is assumed to be normally distributed.
Step 1: This is a lower-tailed test, that is

H0 : σ2 = (200)2
H1 : σ2 < (200)2

or equivalently

H0 : σ = 200
H1 : σ < 200

Step 2: The appropriate test statistic is the χ2 -test and its value is given by

( n − 1 ) s2 (25 − 1)(178)2
χ20 = 2
= = 19.0104
σ0 (200)2

Step 3: The rejection region is χ20 < χ21−α,n−1, where χ21−α,n−1 = χ21−0.05,24 = 13.8484.

Step 4: Since χ20 = 19.0104 ≮ χ21−α,n−1 = 13.8484, we do not reject the null hypothesis H0 . The supervisor can
conclude that the standard deviation of the lifespan of a light bulb is 200.
The p-value is equal to F (χ20 ) = F (19.0104) = 0.2486, which is less than the significance level α = 0.05; therefore, there
is insufficient evidence to reject the null hypothesis.

4.4.4 T ESTS ON A P ROPORTION

Consider a population of items, each of which independently meets certain standards with some unknown probability
p, and suppose that a sample of size n was taken from this population. In testing the proportion p, there are three ways
to structure the hypothesis test:

Two-tailed Upper-Tailed Lower-Tailed

H0 : p = p0 H0 : p = p0 H0 : p = p0
H1 : p 6= p0 H1 : p > p0 H1 : p < p0

If X denote the number of the n items that meet the standards, then X ∼ bino (n, p). Thus, for n large and under the
assumption that the null hypothesis is true (i.e. H0 : p = p0), it follows that the test statistic
X − np0
Z0 = p ∼ N (0, 1)
np0 (1 − p0 )

has a approximate standard normal distribution.

81
 Let z0 be the numerical value, calculated from the sample, of the test statistic Z0 . Then, for a selected significance
level α, the critical regions are (see Figure)

Two-tailed Upper-Tailed Lower-Tailed

If z0 < −zα/2 or z0 > zα/2 If z0 > zα If z0 < −zα
reject H0 reject H0 reject H0

Denote by Φ(·) the cdf of the N (0, 1) distribution. Then, the proportion test may be summarized as follows:

Hypotheses:
Two-tailed Upper-Tailed Lower-Tailed
H0 : p = p0 H0 : p = p0 H0 : p = p0
H1 : p 6= p0 H1 : p > p0 H1 : p < p0
X − np0
Test Statistic: Z0 = p
np0 (1 − p0 )
Critical Regions:
Two-tailed Upper-Tailed Lower-Tailed
|z0 | > zα/2 z0 > z α z0 < − z α
p-values:
Two-tailed Upper-Tailed Lower-Tailed
2[1 − Φ(|z0 |)] 1 − Φ ( z0 ) Φ ( z0 )

Example 4.27 A builder claims that heat pumps are installed in 70% of all homes being constructed today in the city of Granada,
Spain. Would you agree with this claim if a random survey of new homes in this city showed that 8 out of 15 had heat pumps
installed? Then, calculate the p-value. Use a 0.10 level of significance.
Solution: From the given information, we have n = 15, p0 = 0.7, x = 8, and α = 0.10.
Step 1: This is a two-tailed test on proportion:

H0 : p = 0.7
H1 : p 6= 0.7

Step 2: The appropriate test statistic is the z-test and its value is given by

x − np0 8 − (15)(0.7)
z0 = p = p = −1.4086
np0 (1 − p0 ) (15)(0.7)(1 − 0.7)

Step 3: The rejection region is |z0 | > zα/2 , where zα/2 = z0.05 = 1.6449.
Step 4: Since |z0 | = 1.4086 ≯ zα/2 = 1.6449, we do not reject the null hypothesis H0 . We conclude that there is
insufficient reason to doubt the builders claim.
The p-value is equal to 2[1 − Φ(|z0 |)] = 2[1 − Φ(1.4086)] = 0.1590, which is greater than the significance level α = 0.05;
therefore we have insufficient evidence to reject the null hypothesis.

Example 4.28 A commonly prescribed drug for relieving nervous tension is believed to be only 60% effective. Experimental
results with a new drug administered to a random sample of 100 adults who were suffering from nervous tension show that 70
received relief. Is this sufficient evidence to conclude that the new drug is superior to the one commonly prescribed? Then, calculate
the p-value. Use a 0.05 level of significance.
Solution: From the given information, we have n = 100, p0 = 0.6, x = 70, and α = 0.05.

82
 Step 1: This is an upper-tailed test on proportion:

H0 : p = 0.6
H1 : p > 0.6

Step 2: The appropriate test statistic is the z-test and its value is given by

x − np0 70 − (100)(0.6)
z0 = p = p = 2.0412
np0 (1 − p0 ) (100)(0.6)(1 − 0.6)

Step 3: The rejection region is z0 > zα , where zα = z0.05 = 1.6449.

Step 4: Since z0 = 2.0412 > zα = 1.6449, we reject the null hypothesis H0 . We conclude that the new drug is superior.

The p-value is equal to 1 − Φ(z0 ) = 1 − Φ(2.0412) = 0.0206, which is smaller than the significance level α = 0.05;
therefore we reject the null hypothesis.

Example 4.29 Direct Mailing Company sells computers and computer parts by mail. The company claims that at least 90% of
all orders are mailed within 72 hours after they are received. The quality control department at the company often takes samples
to check if this claim is valid. A recently taken sample of 150 orders showed that 129 of them were mailed within 72 hours. Using
2.5% significance level, do you think the company’s claim is true?

Solution: From the given information, we have n = 150, p0 = 0.9, x = 129, and α = 0.025.

Step 1: This is an upper-tailed test on proportion:

H0 : p ≥ 0.9 (The company’s claim is true)

H1 : p < 0.9 (The company’s claim is false)

Step 2: The appropriate test statistic is the z-test and its value is given by

x − np0 129 − (150)(0.9)

z0 = p = p = −1.6330
np0 (1 − p0 ) (150)(0.9)(1 − 0.9)

Step 3: The rejection region is z0 < −zα , where zα = z0.025 = 1.96.

Step 4: Since z0 = −1.6330 ≮ −zα = −1.96, we do not reject the null hypothesis H0 . We conclude that the company’s
claim is true.

The p-value is equal to Φ(z0 ) = Φ(−1.6330) = 1 − Φ(1.6330) = 0.0512, which is greater than the significance level
α = 0.025; therefore we have insufficient evidence to reject the null hypothesis.

4.4.5 T ESTS ON THE D IFFERENCE IN M EANS

Assume that X1 , . . . , Xn1 ∼ N (µ1 , σ12 ) and Y1 , . . . , Yn2 ∼ N (µ2 , σ22 ) are two independent samples from two independent
normal populations. Let X, Y, S12 and S22 be the sample means and sample variances, respectively. In testing the mean
difference µ1 − µ2 = ∆0 , there are three ways to structure the hypothesis test:

Two-tailed Upper-Tailed Lower-Tailed

H0 : µ1 − µ2 = ∆0 H0 : µ1 − µ2 = ∆0 H0 : µ1 − µ2 = ∆0
H1 : µ1 − µ2 6= ∆0 H1 : µ1 − µ2 > ∆0 H1 : µ1 − µ2 < ∆0

83
◮ Case I) When σ1 and σ2 are known:
Under the assumption that the null hypothesis is true (i.e. H0 : µ1 − µ2 = ∆0 ), the test statistic

X − Y − ( µ1 − µ2 ) X − Y − ∆0
Z0 = s = s ∼ N (0, 1)
σ12 σ22 σ12 σ22
+ +
n1 n2 n1 n2
has a standard normal distribution.
Let z0 be the numerical value, calculated from the sample, of the test statistic Z0 . Then, for a selected significance
level α, the critical regions are

Two-tailed Upper-Tailed Lower-Tailed

If z0 < −zα/2 or z0 > zα/2 If z0 > zα If z0 < −zα
reject H0 reject H0 reject H0

and the z-test for the difference in means may be summarized as follows:

Hypotheses:
Two-tailed Upper-Tailed Lower-Tailed
H0 : µ1 − µ2 = ∆0 H0 : µ1 − µ2 = ∆0 H0 : µ1 − µ2 = ∆0
H1 : µ1 − µ2 6= ∆0 H1 : µ1 − µ2 > ∆0 H1 : µ1 − µ2 < ∆0
X − Y − ∆0
Test Statistic (z-test): Z0 = s
σ12 σ2
+ 2
n1 n2
Critical Regions:
Two-tailed Upper-Tailed Lower-Tailed
|z0 | > zα/2 z0 > z α z0 < − z α
p-values:
Two-tailed Upper-Tailed Lower-Tailed
2[1 − Φ(|z0 |)] 1 − Φ ( z0 ) Φ ( z0 )

Example 4.30 A random sample of size nl = 36 selected from a normal distribution with standard deviation σ1 = 4 has a mean
x̄ = 75. A second random sample of size n2 = 25 selected from a different normal distribution with a standard deviation σ2 = 6
has a mean ȳ = 85. Is there a significant difference between the population means at the 5 percent level of significance?. Then,
calculate the p-value.
Solution: From the given information, we have n1 = 36, n2 = 25, x̄ = 75, ȳ = 85, ∆0 = 0, σ1 = 4, σ2 = 6, and α = 0.05.
Step 1: Since we want to determine whether there is a difference between the population means, this will be a two-
tailed test. Hence,
H0 : µ1 = µ2
H1 : µ1 6 = µ2

Step 2: Since σ1 and σ2 are known, the appropriate test statistic is the z-test and its value is given by
x̄ − ȳ − ∆0 75 − 85
z0 = s = r = −7.2846
σ12 2
σ2 16 36
+ +
n1 n2 36 25

Step 3: The rejection region is |z0 | > zα/2 , where zα/2 = z0.025 = 1.96.

84
 Step 4: Since |z0 | = 7.2844 > zα/2 = 1.96, we reject the null hypothesis H0 . We can conclude that the means are
significantly different from each other.
The p-value is equal to 2[1 − Φ(|z0 |)] = 2[1 − Φ(7.2846)] = 0, which is less than the significance level α = 0.05;
therefore we reject the null hypothesis.

◮ Case II) When σ1 and σ2 are unknown and equal to σ2 :

Under the assumption that the null hypothesis is true (i.e. H0 : µ1 − µ2 = ∆0 ), the test statistic

X − Y − ( µ1 − µ2 ) X − Y − ∆0
T0 = √ = √ ∼ t ( n1 + n2 − 2 )
S p 1/n1 + 1/n2 S p 1/n1 + 1/n2

(n1 − 1)S12 + (n2 − 1)S22

has a t-distribution with n1 + n2 − 2 degrees of freedom, and S2p = is the pooled variance.
n1 + n2 − 2
Let t0 be the numerical value, calculated from the sample, of the test statistic T0 . Then, for a selected significance
level α, the critical regions are

Two-tailed Upper-Tailed Lower-Tailed

If t0 < −tα/2,,n1 +n2 −2 or t0 > tα/2,,n1 +n2 −2 If t0 > tα,,n1 +n2 −2 If t0 < −tα,,n1 +n2 −2
reject H0 reject H0 reject H0

Denote by F (·) the cdf of the t(n1 + n2 − 2) distribution. Then, the t-test for the difference in means may be summa-
rized as follows:

Hypotheses:
Two-tailed Upper-Tailed Lower-Tailed
H0 : µ1 − µ2 = ∆0 H0 : µ1 − µ2 = ∆0 H0 : µ1 − µ2 = ∆0
H1 : µ1 − µ2 6= ∆0 H1 : µ1 − µ2 > ∆0 H1 : µ1 − µ2 < ∆0
X − Y − ∆0
Test Statistic (t-test): T0 = √
S p 1/n1 + 1/n2
Critical Regions:
Two-tailed Upper-Tailed Lower-Tailed
|t0 | > tα/2,n1 +n2 −2 t0 > tα,n1 +n2 −2 t0 < −tα,n1 +n2 −2
p-values:
Two-tailed Upper-Tailed Lower-Tailed
2[1 − F (|t0 |)] 1 − F ( t0 ) F ( t0 )

Example 4.31 An experiment was performed to compare the abrasive wear of two different laminated materials X and Y. Twelve
pieces of material X were tested by exposing each piece to a machine measuring wear. Ten pieces of material Y were similarly tested.
In each case, the depth of wear was observed. The samples of material X gave an average (coded) wear of 85 units with a sample
standard deviation of 4, while the samples of material Y gave an average of 81 with a sample standard deviation of 5. Can we
conclude at the 0.05 level of significance that the abrasive wear of material X exceeds that of material Y by more than 2 units?
Assume the populations to be approximately normal with equal variances.
Solution: From the given information, we have n1 = 12, n2 = 10, x̄ = 85, ȳ = 81, ∆0 = 2, s1 = 4, s2 = 5, and α = 0.05.
Step 1: Let µ1 and µ2 represent the population means of the abrasive wear for material X and material Y, respectively.
This is an upper-tailed test. Thus,

H0 : µ1 − µ2 = 2
H1 : µ1 − µ2 > 2

85
 Step 2: Since the standard deviations of the populations are unknown, the appropriate test statistic is the t-test and its
value is given by
x̄ − ȳ − ∆0 85 − 81 − 2
t0 = √ = √ = 1.0432
s p 1/n1 + 1/n2 4.4777 1/12 + 1/10
where the pooled variance is

(n1 − 1)s21 + (n2 − 1)s22 (12 − 1)(4)2 + (10 − 1)(5)2

s2p = = = 20.05
n1 + n2 − 2 12 + 10 − 2

Step 3: The rejection region is t0 > tα,n1 +n2 −2 , where tα,n1 +n2 −2 = t0.05,20 = 1.7247.

Step 4: Since t0 = 1.0432 ≯ tα,n1 +n2 −2 = 1.7247, we do not reject the null hypothesis H0 . We are unable to conclude
that the abrasive wear of material X exceeds that of material Y by more than 2 units.

The p-value is equal to 1 − F (t0 ) = 1 − F (1.0432) = 0.1547, which is greater than the significance level α = 0.05;
therefore we have insufficient evidence to reject the null hypothesis.

Example 4.32 One process of making green gasoline, not just a gasoline additive, takes biomass in the form of sucrose and converts
it into gasoline using catalytic reactions. This research is still at the pilot plant stage. At one step in a pilot plant process, the
product volume (liters) consists of carbon chains of length 3. Nine runs were made with each of two catalysts and the product
volumes measured:
catalyst 1: 1.86 2.05 2.06 1.88 1.75 1.64 1.86 1.75 2.13
catalyst 2: 0.32 1.32 0.93 0.84 0.55 0.84 0.37 0.52 0.34
Is the mean yield with catalyst 1 more than 0.80 liters higher than the yield with catalyst 2? Test with α = 0.05.

Solution: From the given information and data, we have n1 = 9, n2 = 9, x̄ = 1.8867, ȳ = 0.67, ∆0 = 0.80, s1 = 0.1642,
s2 = 0.3366, and α = 0.05.
As shown in Figure 4.11, the normal probability plots and box plot indicate that the measurements constitute a sample
from a normal population.

Catalyst 1
0.95
Catalyst 2 2.2
0.90
2

1.8
0.75
1.6
Product volume
Probability

0.50 1.4

1.2

0.25 1

0.8
0.10
0.6
0.05
0.4
0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0.2
Data Catalyst 1 Catalyst 2

F IGURE 4.11: Normal probability and box plots for the product volumes.

Step 1: Let µ1 and µ2 represent the population means for catalyst 1 and catalyst 2, respectively. This is an upper-tailed
test. Thus,

H0 : µ1 − µ2 = 0.80
H1 : µ1 − µ2 > 0.80

86
 Step 2: Since the standard deviations of the populations are unknown, the appropriate test statistic is the t-test and its
value is given by
x̄ − ȳ − ∆0 1.8867 − 0.67 − 0.80
t0 = √ = √ = 3.3381
s p 1/n1 + 1/n2 0.2648 1/9 + 1/9

where the pooled variance is

(n1 − 1)s21 + (n2 − 1)s22 (9 − 1)(0.1642)2 + (9 − 1)(0.3366)2

s2p = = = 0.0701
n1 + n2 − 2 9+9−2

Step 3: The rejection region is t0 > tα,n1 +n2 −2 , where tα,n1 +n2 −2 = t0.05,16 = 1.7459.

Step 4: Since t0 = 3.3381 > tα,n1 +n2 −2 = 1.7459, we reject the null hypothesis H0 . We conclude that the mean product
volume from catalyst 1 is more than 0.80 liters higher than catalyst 2.

The p-value is equal to 1 − F (t0 ) = 1 − F (3.3381) = 0.0021, which is smaller than the significance level α = 0.05;
therefore we reject the null hypothesis.

4.4.6 T ESTS ON THE E QUALITY OF VARIANCES

The quality of any process depends on the amount of variability present in the process, which we measure in terms of
the variance of the quality characteristic. For example, if we have to choose between two similar processes, we would
prefer the one with smaller variance. Any process with smaller variance is more dependable and more predictable. In
fact, one of the most important criteria used to improve the quality of a process or to achieve 6σ quality is to reduce the
variance of the quality characteristic in the process. In practice, comparing the variances of two processes is common.
Assume that X1 , . . . , Xn1 ∼ N (µ1, σ12 ) and Y1 , . . . , Yn2 ∼ N (µ2 , σ22 ) are two independent samples from two indepen-
dent normal populations. Let S12 and S22 be the sample variances. In testing the equality of variances σ12 = σ22 , there are
three ways to structure the hypothesis test:

Two-tailed Upper-Tailed Lower-Tailed

H0 : σ12 = σ22 H0 : σ12 = σ22 H0 : σ12 = σ22
H1 : σ12 6= σ22 H1 : σ12 > σ22 H1 : σ12 < σ22

Under the assumption that the null hypothesis is true (i.e. H0 : σ12 = σ22 ), the test statistic

S12 /σ12 S12

F0 = = ∼ F (n1 − 1, n2 − 1)
S22 /σ22 S22

Let f 0 be the numerical value, calculated from the sample, of the test statistic F0 . Then, for a selected significance
level α, the critical regions are

Two-tailed Upper-Tailed Lower-Tailed

If f 0 < f 1−α/2,n1 −1,n2 −1 or f 0 > f α/2,n1 −1,n2 −1 If f 0 > f α,n1 −1,n2 −1 If f 0 < f 1−α,n1 −1,n2 −1
reject H0 reject H0 reject H0

and the f -test for the equality of variances may be summarized as follows:

87
Hypotheses:
Two-tailed Upper-Tailed Lower-Tailed
H0 : σ12 = σ22 H0 : σ12 = σ22 H0 : σ12 = σ22
H1 : σ12 6= σ22 H1 : σ12 > σ22 H1 : σ12 < σ22
S12
Test Statistic ( f -test): F0 =
S22
Critical Regions:
Two-tailed Upper-Tailed Lower-Tailed
f 0 > f α/2,n1 −1,n2 −1 or f 0 < f 1−α/2,n1 −1,n2 −1 f 0 > f α,n1 −1,n2 −1 f 0 < f 1−α,n1 −1,n2 −1

Example 4.33 In testing for the difference in the abrasive wear of the two materials in the previous example, we assumed that the
two unknown population variances were equal. Were we justified in making this assumption? Use a 0.10 level of significance.
Solution: From the given information, we have n1 = 12, n2 = 10, s1 = 4, s2 = 5, and α = 0.10.
Step 1: Let σ12 and σ22 represent the population variances of the abrasive wear for material X and material Y, respec-
tively. This is a two-tailed test. Thus,
H0 : σ12 = σ22
H1 : σ12 6= σ22

Step 2: Since the standard deviations of the populations are unknown, the appropriate test statistic is the f -test and
its value is given by
s2
f 0 = 21 = 0.64
s2
Step 3: The rejection region is f 0 > f α/2,n1 −1,n2 −1 or f 0 < f 1−α/2,n1 −1,n2 −1 , where f α/2,n1 −1,n2 −1 = 3.1025 and
f 1−α/2,n1 −1,n2 −1 = 0.3453.
Step 4: Since f 0 = 0.64 ≯ f α/2,n1 −1,n2 −1 = 3.1025, we do not reject the null hypothesis H0 . We conclude that there is
insufficient evidence that the variances differ.

Example 4.34 Suppose the following is the sample summary of samples from two independent processes:
n1 = 21 S12 = 24.6
n2 = 16 S22 = 16.4
We assume that the quality characteristics of the two processes are normally distributed N (µ1, σ12 ) and N (µ2 , σ22 ), respectively.
Test at the 5% level of significance the hypothesis H0 : σ12 = σ22 versus H1 : σ12 6= σ22 , and find the p-value for the test.
Solution: From the given information, we have n1 = 21, n2 = 16, s21 = 24.6, s22 = 16.4, and α = 0.05.
Step 1: This is a two-tailed test. Thus,
H0 : σ12 = σ22
H1 : σ12 6= σ22

Step 2: Since the standard deviations of the populations are unknown, the appropriate test statistic is the f -test and
its value is given by
s2
f 0 = 12 = 1.50
s2
Step 3: The rejection region is f 0 > f α/2,n1 −1,n2 −1 or f 0 < f 1−α/2,n1 −1,n2 −1 , where f α/2,n1 −1,n2 −1 = 2.7559 and
f 1−α/2,n1 −1,n2 −1 = 0.3886.
Step 4: Since f 0 = 1.50 ≯ f α/2,n1 −1,n2 −1 = 2.7559 and f 0 = 1.50 ≮ f 1−α/2,n1 −1,n2 −1 = 0.3886, we do not reject the null
hypothesis H0 . We conclude that there is insufficient evidence that the variances differ.

88
4.5 P ROBLEMS

❶ The total weight of a filled tire can dramatically affect the performance and safety of an automobile. Some trans-
portation officials argue that mechanics should check the tire weights of every vehicle as part of an annual inspec-
tion. Suppose the weight of a 185/60/14 filled tire is normally distributed with standard deviation 1.25 pounds.
In a random sample of 15 filled tires, the sample mean weight was 18.75 pounds. Find a 95% confidence interval
for the true mean weight of 185/60/14 tires.

❷ An electro Pneumatic hammer has an advertised impact force of 2.2 joules. In a random sample of 23 hammers,
the impact force for each tool was carefully measured (in joules), and the resulting data are as follows:

2.16 1.69 2.30 2.08 1.72 2.17 2.25 2.06 2.00 2.29 2.15 2.49
2.12 2.17 1.93 2.39 2.22 2.26 2.14 1.92 2.06 2.09 2.08

a) Check the assumption of normality for the impact force data.

b) Find the 95% two-sided confidence interval for the true mean impact force for this type of pneumatic ham-
mer.
c) Using the confidence interval constructed in part b), is there any evidence to suggest that the true mean
impact force is different from 2.2 joules as advertised? Justify your answer.

❸ Adobeware dishes are made from clay and are fired, or exposed to heat, in a large kiln. Large fluctuations in the
kiln temperature can cause cracks, bumps, or other flaws (and increase cost). With the kiln set at 800°C, a random
sample of 19 temperature measurements (in °C) was obtained. The sample variance was 17.55.

a) Find the 95% two-sided confidence interval for the true population variance in temperature of the kiln when
it is set to 800°C. Assume that the underlying distribution is normal.
b) Quality control engineers have determined that the maximum variance in temperature during firing should
be 16°C. Using the confidence interval constructed in part a), is there any evidence to suggest that the true
temperature variance is greater than 16°C? Justify your answer.

❹ A successful company usually has high brand name and logo recognition among consumers. For example, Coco-
Cola products are available to 98% of all people in the world, and therefore may have the highest logo recognition
on any company. A software firm developing a product would like to estimate the proportion of people who
recognize the Linux penguin logo. Of the 952 randomly selected consumers surveyed, 132 could identify the
product associated with the penguin.

a) Is the distribution of the sample proportion, p̂, approximately normal? Justify your answer.
b) Find the 95% two-sided confidence interval for the true proportion of consumers who recognize the Linux
penguin.
c) The company will market a Linux version of their new software if the true proportion of people who rec-
ognize the logo is greater that 0.10. Is there any evidence to suggest that the true proportion of people who
recognize the logo is greater than 0.10? Justify your answer.

❺ An engineer wants to measure the bias in a pH meter. She uses the meter to measure the pH in 15 neutral
substances (pH = 7.0) and obtains the following data:

7.04 7.0 7.03 7.01 6.97 7.00 6.95 7.00 6.99 7.04 6.97 7.07 7.04 6.97 7.08

a) Check the assumption of normality for the pH meter data.

b) Is there sufficient evidence to support the claim that the pH meter is not correctly calibrated at the 5% level
of significance
c) Find the 95% two-sided confidence interval to estimate the mean. Comment on your result.

89
 ❻ A quality control supervisor in a cannery knows that the exact amount each can contains will vary, since there are
certain uncontrollable factors that affect the amount of fill. Suppose regulatory agencies specify that the standard
deviation of the amount of fill should be less that 0.1 ounce. The quality control supervisor sampled 10 cans and
measured the amount of fill in each. The resulting data measurements are:

7.96, 7.90, 7.98, 8.01, 7.97, 7.96, 8.03, 8.02, 8.04, 8.02

Does this information, at the 0.05 level of significance, provide sufficient evidence to indicate that the standard
deviation of the fill measurements is less than 0.1 ounce? Then, calculate the p-value.
❼ The management of a luxurious hotel is concerned with increasing the return rate for hotel guests. One aspect of
first impressions by guests relates to the time it takes to deliver the guest’s luggage to the room after check-in
to the hotel. A random sample of 20 deliveries on a particular day were selected in Wing A of the hotel and a
random sample of 20 deliveries were selected in Wing B.
Wing A: 10.70, 9.89, 11.83, 9.04, 9.37, 11.68, 8.36, 9.76, 13.67, 8.96, 9.51, 10.85, 10.57, 11.06, 8.91, 11.79, 10.59, 9.13, 12.37, 9.91
Wing B: 7.20, 6.68, 9.29, 8.95, 6.61, 8.53, 8.92, 7.95, 7.57, 6.38, 8.89, 10.03, 9.30, 5.28, 9.23, 9.25, 8.44, 6.57, 10.61, 6.77

a) Is the normality assumption of the data satisfied? Justify your answer.

b) Was there a difference in the mean delivery time in the two wings on the hotel?. Test with α = 0.05.
c) Determine whether the variance in luggage delivery time is the same for Wing A and Wing B of the hotel at
the α = 0.05 level of significance.
d) Assume that delivery times to Wing A and Wing B are two independent normal populations with unknown
variances σ12 and σ22 , respectively. Construct a 90% two-sided confidence interval on the ratio of the two
standard deviations σ1 /σ2 . Comment on your result.

4.6 REFERENCES

[1] D.C. Montgomery, Introduction to Statistical Quality Control, Wiley, 6th Edition, 2009.
[2] I. Bass and B. Lawton, Lean Six Sigma using SigmaXL and Minitab, McGraw-Hill Professional, 1st Edition, 2009.
[3] B.C. Gupta and H.F. Walker, Applied statistics for the Six Sigma Green Belt, ASQ Quality Press, 2004.

90
 C HAPTER 5

S TATISTICAL P ROCESS AND Q UALITY C ONTROL

Almost all quality improvement comes via

simplification of design, manufacturing,, layout,
processes, and procedures.

Tom Peters

Statistical quality control (SQC) is a term used to describe the activities associated with ensuring that goods and
services satisfy customer needs. SQC uses statistical analysis based on measurements taken from a process or from a
sample of products or services, to make decisions regarding the quality of goods and services. The statistical methods
of SQC may be divided into two main categories: Statistical process control (SPC) and acceptance sampling. SPC refers
to the use of statistical methods to measure and control the performance of a process to ensure that the output meets
customer needs. Acceptance sampling is a methodology of taking samples from lots of materials or products and
inspecting the items to determine if the items meet customer requirements. A process may include customer services,
productions systems, and administration activities. SPC may be used to help control almost all processes that can be
measured or monitored to ensure that the process performs within limits.

5.1 S TATISTICAL P ROCESS C ONTROL

Statistical process control allows engineers to understand and monitor process variation through control charts. The
causes of process variation in a product quality characteristic may be broadly classified into two main categories:
common causes of variation (variation due to the system itself) and assignable causes of variation (variation due to
factors external to the system).
The concept of control charts was first introduced by Walter A. Shewhart of Bell Telephone Laboratories during
the 1920’s. For this reason, statistical control charts are also known as Shewhart control charts. A control chart is
a graphical method to quickly spot assignable cause of variation of a process. Variation is present in any process;
deciding when the variation is natural and when it needs correction is the key to quality control. A control chart
displays a quality characteristic that has been measured or computed from a sample versus the sample number or
time. The sample values to be used in a quality control effort are divided into subgroups with a sample representing
a subgroup. A control chart contains a center line (CL) that represents the average value of the quality characteristic
when the process is in control. Two other horizontal lines, called the upper control limit (UCL) and the lower control
limit (LCL), are also shown on the chart. These control limits are chosen so that if the process is in control, nearly all
of the sample points will fall between them. In general, as long as the points plot within the control limits, the process
is assumed to be in-control, and no action is necessary. However, a point that plots outside of the control limits is
interpreted as evidence that the process is out-of-control, and investigation and corrective action are required to find
and eliminate the assignable cause or causes responsible for this behavior. The sample points on the control chart are
usually connected with straight-line segments so that it is easier to visualize how the sequence of points has evolved
over time. Figure 5.1 illustrates the concept of a control chart, where the process is found to be out of control due to the
sample number 15 which falls outside the control limits.

91
 1.1 UCL

1.05

Sample statistic
1
CL

0.95

0.9
LCL
15

0.85
2 4 6 8 10 12 14 16 18 20
Sample number

F IGURE 5.1: Illustration of a control chart.

5.1.1 H YPOTHESIS T ESTING AND C ONTROL C HARTS

There is a close connection between control charts and hypothesis testing. A control chart may be formulated as
hypothesis test:

H0 : process is in-control
(5.1.1)
H1 : process is out-of-control

Control limits are established to control the probability of making the error of concluding that the process is out of
control when in fact it is not. This corresponds to the probability of making a Type I error if we were testing the null
hypothesis that the process is in control. On the other hand, we must be attentive to the error of not finding the process
out of control when in fact it is (Type II error). Thus, the choice of control limits is similar to the choice of a critical
region. When a point plots within the control limits, the null hypothesis is not rejected; and when a point plots outside
the control limits, the null hypothesis is rejected.

Definition 5.1 Let Θ be a sample statistic that measures some quality characteristic of interest, with mean µΘ and standard
deviation σΘ . The upper control limit, center line, and lower control limit are given by

UCL = µΘ + 3σΘ
CL = µΘ (5.1.2)
LCL = µΘ − 3σΘ

The 3σΘ limits imply that there is a probability of only 0.0026 of a sample statistic to fall outside the control limits if
the process is in-control.
Control chart are broadly classified into control charts for variables and control charts for attributes. Variable control
charts are used for quality characteristics that are measured on a continuous scale such as length, temperature, weight,
and time. Attribute control charts are used for quality characteristics in discrete (count) data, such as number of
defects. Attribute control charts are further divided into two main classes: attributes control charts for defective units,
and attribute control charts for defects per unit.

92
5.1.2 R ULES FOR D ETERMINING O UT-O F -C ONTROL P OINTS
The control chart is an important tool for distinguishing between the common causes of variation that are due to the
process and special causes of variation that are not due to the process. Only management can change the process. One
of the main goals of using a control chart is to determine when the process is out-of-control so that necessary actions
may be taken. The simplest rule for detecting the presence of an assignable (or special) cause of variation is one or
more plotted points falling outside the control limits UCL and LCL. Assignable causes are special causes of variation
that are ordinarily not part of the process, and should be corrected as warranted. Common causes, on the other hand,
are inherent in the design of the system and reflect the typical variation to be expected. An unstable (or out-of-control)
process exhibits variation due to both assignable and common causes. Improvement can be achieved by identifying
and removing the assignable cause(s). A stable process is one that exhibits only common-cause variation, and can be
improved only by changing the design of the process. Attempts to make adjustments to a stable process, which is
called tampering, results in more variation in the quality of the output. Control charts are used to detect the occurrence
of assignable causes affecting the quality of process output. Figure depicts a control chart in which the area between
UCL and LCL is subdivided into bands, each of which is 1σΘ wide.

UCL
Sample Statistic (Quality Characteristic)

µΘ + 2σΘ

µΘ + σΘ

CL

µΘ − σΘ

µΘ − 2σΘ

LCL

Sample Number
F IGURE 5.2: Illustration of control chart bands, each of which is 1σΘ wide.

The rules for determining out-of-control points in a control chart may be summarized in five main rules (refereed to as
Western Electric rules). That is, a process is considered out-of-control (unstable) if:

Rule 1: A point falls outside the upper and lower control limits, i.e. above UCL or below LCL
Rule 2: Two out of three consecutive points fall above µΘ + 2σΘ or below µΘ − 2σΘ
Rule 3: Four out of five consecutive points fall above µΘ + 1σΘ or below µΘ − 1σΘ
Rule 4: Eight or more consecutive points fall above µΘ or below µΘ
Rule 5: Eight or more consecutive points move upward (increasing) or downward (decreasing) in value.

5.2 C ONTROL C HARTS FOR VARIABLES

Control charts for variables are used to study a process when a characteristic is a measurement; for example, temper-
ature, cost, revenue, processing time, area, and waiting time. Variable charts are typically used in pairs. One chart

93
studies the variation in a process, and the other chart studies the variation in the process mean. A chart that studies
the process variability must be examined before the chart that studies the process mean. This is due to the fact that the
chart that studies the process mean assumes that the process variability is stable over time. One of the most commonly
used pairs of charts is the X-chart and the R-chart. Another pair is the X-chart and the s-chart. In this section, we
discuss in detail these two pairs of charts.

5.2.1 C ONTROL C HARTS FOR THE M EAN AND R ANGE

Control Chart for the Mean (X-chart):
An X-chart is a control chart plotting the sample means vs the sample number. Denote by µ and σ the process mean
and standard deviation, respectively. √
Considering the sample mean X as the sample statistic Θ yields µ X = µ and σX = σ/ n. When the parameters
µ and σ are unknown, we usually estimate them on the basis of preliminary samples (subgroups), taken when the
process is thought to be in control. Suppose m preliminary samples are available, each of size n. Denote by X i and Ri
the sample mean and range of the i-th sample, respectively. An unbiased estimator of µ is obtained by averaging m
sample (subgroup) means when the process is in control
m
1
CL = X =
m ∑ Xi (5.2.1)
i =1

where X (pronounced “X-double-bar”) is the average of the X i ’s.

On the other hand, it can be shown that the mean and standard deviation of relative range W = R/d2 are µW = d2
and σW = d3 , where d2 and d3 are constants that depend on the sample size n. That is, the relative range is an unbiased
estimator of σ. Since R = Wσ, it follows that µ R = d2 σ and σR = d3 σ. An unbiased estimator of µ R is the average of
the sample ranges given by
1 m
m i∑
R= Ri (5.2.2)
=1
where Ri = X( n),i − X(1),i is the range of the ith-sample (subgroup). Here X( n),i and X(1),i denote the largest and
smallest observations, respectively, in the sample.
Thus, an unbiased estimator of σ is
R
σ̂ = (5.2.3)
d2
√
The upper and control limits are located at a distance of 3σX = 3σ/ n above and below the center line. An estimator
of this distance is given by
√ 3( R/d2 ) 3
3σ̂/ n = √ = √ R = A2 R (5.2.4)
n d2 n
where A2 is a constant that depends on the sample size n (see Appendix Table V).

Control chart for the mean (X-chart):

The upper control limit, center line, and lower control limit of the X-chart are given by

UCL = x̄¯ + A2 r̄
CL = x̄¯ (5.2.5)
LCL = x̄¯ − A2 r̄

Example 5.1 A control chart for X is to be set up for an important quality characteristic. The sample size is n = 4, and x̄ and r
are computed for each of 25 preliminary samples. The summary data are
25 25
∑ x̄i = 7657 ∑ r̄i = 1180
i =1 i =1

(i) Find the control limits for X-chart

94
 (ii) Assuming the process is in control, estimate the process mean and standard deviation.

Solution: From the given information, the number of samples is m = 25 and the sample size is n = 4.

(i) The grand mean and the average range are given by
m m
1 7657 1 1180
x̄¯ =
m ∑ x̄i = 25
= 306.28 r̄ =
m ∑ ri = 25
= 47.20
i =1 i =1

The value of A2 for samples of size 4 is A2 = 0.729. Therefore, the control limits of the X-chart are

UCL = x̄¯ + A2 r̄ = 306.28 + (0.729)(47.20) = 340.69

CL = x̄¯ = 306.28
LCL = x̄¯ − A2 r̄ = 306.28 + (0.729)(47.20) = 271.87

(ii) The estimates of the process mean and standard deviation are

r̄ 47.20
µ̂ = x̄¯ = 306.28 σ̂ = = = 22.92
d2 2.059

Control Chart for the Range (R-chart):

In quality control, we want to control not only the mean value of some quality characteristic but also its variability. A
range chart (or simply R-chart) is a control chart plotting the sample ranges vs the sample number, and it monitors the
variation of a quality characteristic. Considering the sample range R as the statistic Θ yields µ R = d2 σ and σR = d3 σ,
where d2 and d3 are constants (see Appendix Table V) that depend on the sample size n. An unbiased estimator of µ R
is the mean R of the ranges of m samples
1 m
m i∑
CL = R = Ri
=1

The upper and control limits are located at a distance of 3σR = 3d3 σ above and below the center line. Since R/d2 as
an unbiased estimator of σ, it follows that the upper and control limits may be expressed as
 
d3 d3
UCL = R + 3 R = 1 + 3 R = D4 R (5.2.6)
d2 d2

and  
d d
LCL = R − 3 3 R = 1−3 3 R = D3 R (5.2.7)
d2 d2
where D3 and D4 are constants that depend on the sample size n (see Appendix Table V).

Control chart for the range (R-chart):

The upper control limit, center line, and lower control limit for the R-chart are given by

UCL = D4 r̄
CL = r̄ (5.2.8)
LCL = D3 r̄

The R-chart highlights the changes in the process variability and shows better results when analyzed in conjunction
with the X-chart. It is likely that a sample with the same mean may not reveal a shift in the process at all. Thus, it is
necessary to analyze both the X-and R-chart together to decide whether the process is in-control or out-of-control.

Example 5.2 Samples of size 5 are collected from a process every hour. After 30 samples have been collected, we calculate the
value of the average range r̄ = 2.5. Find the control limits for the R-chart.

95
Solution: From the given information, the number of samples is m = 30 and the sample size is n = 5. From Appendix
Table V, the values of D3 and D4 are D3 = 0 and D4 = 2.115. Thus, the upper control limit, center line, and lower
control limit for the R-chart are given by

UCL = D4 r̄ = (2.114)(2.5) = 5.29

CL = r̄ = 2.5
LCL = D3 r̄ = 0

Example 5.3 The data provided in Table 5.1 have been obtained by measuring four consecutive units on a assembly line every 30
minutes until the 20 subgroups (samples) are obtained. Each subgroup has 5 observations. Construct the X- and R-charts. Is the
process under statistical control? Explain.

TABLE 5.1: Assembly Data in Subgroups (Samples) Obtained at Regular Intervals.

Sample Data
Number X1 X2 X3 X4 X5
1 29 51 75 62 42
2 97 73 75 99 56
3 46 60 68 76 57
4 40 61 66 41 59
5 66 76 70 76 52
6 58 61 45 41 78
7 53 38 71 57 42
8 48 71 37 62 60
9 86 65 68 72 74
10 77 55 43 50 63
11 54 50 41 66 46
12 58 48 49 87 59
13 83 57 35 25 52
14 87 43 57 39 47
15 76 48 76 77 80
16 51 32 53 52 54
17 47 48 65 56 47
18 70 45 60 73 44
19 42 67 78 95 59
20 39 82 54 35 32

Solution: The upper control limit, center line, and lower control limit for the R-chart are

UCL = D4 r̄ = 74.219
CL = r̄ = 35.10
LCL = D3 r̄ = 0
where for a sample of size n = 5, Appendix Table V gives D3 = 0 and D4 = 2.114.

1 l o a d assembly.mat %l o a d assembly data

2 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' ,{ ' r ' , ' xbar ' } , ' sigma ' , ' range ' ) ; %R− and Xbar−c h a r t s

First we examine the R-chart for signs of special variation. The R-chart is shown in Figure 5.3(a), where all samples
appear to be in-control. None of the points on the R-chart is outside the control limits, and there are no other signals
indicating a lack of control. Thus, there are no indications of special sources of variation on the R-chart. In other words,
only common-cause variation appears to exist. In this case, we can proceed further to calculate the upper control limit,
center line, and lower control limit for the X-chart

UCL = x̄¯ + A2 r̄ = 78.926

CL = x̄¯ = 58.680
LCL = x̄¯ − A2 r̄ = 38.434

96
where the value of A2 for a sample of size n = 5 is A2 = 0.577.
The X-chart is shown in Figure 5.3(b), where the sample number 2 appears to be out-of-control. Further investigation
is warranted to determine the source(s) of this special variation. Thus, the X-chart can be interpreted without concern
that the observed variability in the sample means could be associated with a lack of control of process variability.
Therefore, the sample number 2 is discarded when remedial actions have been taken to remove special causes, and
then new limits are calculated using the remaining 24 samples (i.e. samples 1 and 3 to 25). These limits are referred to
as revised control limits.

80
80 2
UCL
UCL
70 75

60 70

65
Sample Range

50

Sample Mean
60
40 CL
CL
55
30
50
20
45
10
40
LCL
0 LCL
35
2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20
Sample Number Sample Number
(a) R-chart (b) X-chart
F IGURE 5.3: R- and X-charts for assembly data.

With sample 2 deleted, the revised mean range and grand mean are:

x̄¯ = 57.56 r̄ = 34.69

The revised control limits for the new R-chart are

UCL = D4 r̄ = 73.340
CL = r̄ = 34.684
LCL = D3 r̄ = 0

and the revised control limits for the new X-chart are

UCL = x̄¯ + A2 r̄ = 77.564

CL = x̄¯ = 57.558
LCL = x̄¯ − A2 r̄ = 37.551

The new X- and R-charts are shown in Figure 5.4. Notice now that all the points fall within the limits, indicating that
the process may be stable.

5.2.2 C ONTROL C HARTS FOR THE M EAN AND S TANDARD D EVIATION

It is customary to examine the control chart for standard deviation first and verify that the all the plotted points fall
within the control limits, and then proceed to constructing the X-chart. In fact, the concept of bringing the process
variability under control first and then proceeding to control the mean does make a lot of sense. This is due to the fact
that without controlling the process variability, it is almost impossible to bring the process mean under control.

Control Chart for the Standard Deviation (S-chart):

As the sample size n increases, the range becomes increasingly less efficient as a measure of variability. This is the
case because the range ignores all information between the two most extreme values (minimum and maximum sample

97
 80 80
UCL
UCL
70 75

60 70

65
Sample Range

50

Sample Mean
60
40 CL
CL 55
30
50
20
45
10
40
LCL
0 LCL
35
2 4 6 8 10 12 14 16 18 2 4 6 8 10 12 14 16 18
Sample Number Sample Number
(a) R-chart (b) X-chart
F IGURE 5.4: Revised R- and X-charts for assembly data.

values). A standard deviation control chart (or simply S-chart) is sensitive to changes in variation in the measurement
process, and it preferable for larger sample sizes (n ≥ 10). The S-chart plots the sample standard deviations vs the
sample number. Suppose m preliminary samples are available, each of size n. Denote by Si the sample standard
deviation of the i-th sample. An unbiased estimator of µS is the mean S of the sample standard deviations of m samples
m
1
CL = S =
m ∑ Si (5.2.9)
i =1

The upper and lower control q

limits are located at a distance of 3σS above and below the center line. It can be shown that
E(S) = µS = c4 σ and σS = σ 1 − c24 , where c4 is a constant that depends on the sample size n (see Appendix Table V).
q
Thus, S/c4 is an unbiased estimator of σ, which in turn implies that (S/c4 ) 1 − c24 is an estimator of σS . Therefore, the
upper and lower control limits may be expressed as
q  q 
S 3
UCL = S + 3 1 − c24 = 1+ 1 − c24 S = B4 S (5.2.10)
c4 c4

and  
q q
S 3
LCL = S − 3 1 − c24 = 1− 1 − c24 S = B3 S (5.2.11)
c4 c4
where B3 and B4 are constants that depend on the sample size n (see Appendix Table V).

Control chart for standard deviation (S-chart):

The upper control limit, center line, and lower control limit of the S-chart are given by

UCL = B4 s̄
CL = s̄ (5.2.12)
LCL = B3 s̄

Example 5.4 Containers are produced by a process where the volume of the containers is subject to quality control. Twenty-five
samples of size 5 each were used to establish the quality control parameters, and the sum of the sample standard deviations is

25
∑ si = 0.903
i =1

98
 (i) Find the control limits for the S-chart
(ii) Assuming the process is in control, estimate the process standard deviation.
Solution: From the given information, the number of samples is m = 25 and the sample size is n = 5. Thus, Appendix
Table V gives B3 = 0, B4 = 2.089, and c4 = 0.940.
(i) The average of the sample standard deviations is
m
1 0.903
s̄ =
m ∑ si = 25
= 0.0361
i =1

Therefore, the control limits for the S-chart are

UCL = B4 s̄ = (2.089)(0.0361) = 0.0754

CL = s̄ = 0.0361
LCL = B3 s̄ = 0

(ii) The process standard deviation can be estimated as follows

s̄ 0.0361
σ̂ = = = 0.0384.
c4 0.940

Control Chart for the Mean (from s̄):

We can now write the parameters of the corresponding X-chart involving the use of the sample standard deviation.
Let us assume that the sample standard deviation S and the sample mean X are available from the base preliminary
sample. Since S/c4 as an unbiased estimator of σ, the upper and control limits of the X-chart may also be written as

S
UCL = X + 3 √ = X + A3 S (5.2.13)
c4 n

and
S
LCL = X − 3 √ = X − A3 S (5.2.14)
c4 n
where A3 is a constant that depends on the sample size n (see Appendix Table V).

Control chart for mean (X-chart from s̄):

The upper control limit, center line, and lower control limit of the X-chart (from s̄) are given by

UCL = x̄¯ + A3 s̄
CL = x̄¯ (5.2.15)
LCL = x̄¯ − A3 s̄

Example 5.5 Using the data of Table 5.1, construct the S- and the X-charts.
Solution: The upper control limit, center line, and lower control limit for the S-chart are given by

UCL = B4 s̄ = 29.502
CL = s̄ = 14.122
LCL = B3 s̄ = 0

where B3 = 0 and B4 = 2.089 for samples of size n = 5.

Similar to the analysis that we performed in the case of R-chart, we first examine the S-chart for signs of special
variation. The S-chart is given in Figure 5.5(a), which shows that all plotted points fall within the control limits and
there is no evidence of any special pattern. Thus, we may conclude that the only variation present in the process is

99
due to common causes. In this case, we can proceed further to calculate the upper control limit, center line, and lower
control limit for the X-chart:

UCL = x̄¯ + A3 s̄ = 78.837

CL = x̄¯ = 58.680
LCL = x̄¯ − A3 s̄ = 38.523

where A3 = 1.427 for samples of size n = 5.

1 l o a d assembly.mat %l o a d assembly data

2 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' ,{ ' s ' , ' xbar ' } , ' sigma ' , ' s t d ' ) ; %S− and Xbar−c h a r t s

The X-chart is shown in Figure 5.5(b), where the sample number 2 is out-of-control. This indicate that the process
may not be under control and there are some special causes present that are affecting the process mean. Thus, a thor-
ough investigation should be launched to find the special causes, and appropriate action should be taken to eliminate
these special causes before we proceed to recalculate the control limits for the ongoing process.

30 UCL 80 2
UCL

75
25
Sample Standard Deviation

70

20
65
Sample Mean

60
15 CL
CL
55

10
50

45
5

40
LCL
0 LCL
35
2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20
Sample Number Sample Number
(a) s-chart (b) X-chart
F IGURE 5.5: S- and X-charts for assembly data.

With sample 2 deleted, the new upper control limit, center line, and lower control limit for the S-chart are given by

UCL = B4 s̄ = 29.072
CL = s̄ = 13.917
LCL = B3 s̄ = 0

The new upper control limit, center line, and lower control limit for the X-chart are given by

UCL = x̄¯ + A3 s̄ = 77.421

CL = x̄¯ = 57.558
LCL = x̄¯ − A3 s̄ = 37.694

The new X- and S-charts are shown in Figure 5.6. Notice now that all the points fall within the limits, indicating that
the process may be stable.

Example 5.6 A component part for a jet aircraft engine is manufactured by an investment casting process. The vane opening on
this casting is an important functional parameter of the part. Table 5.2 presents 20 samples of five parts each. The values given in
the table have been coded by using the last three digits of the dimension; that is, 31.6 should be 0.50316 inch.
(i) Estimate the process mean and standard deviation.

100
 30 30
UCL UCL

25 25
Sample Standard Deviation

Sample Standard Deviation

20 20

15 15
CL CL

10 10

5 5

0 LCL 0 LCL
2 4 6 8 10 12 14 16 18 2 4 6 8 10 12 14 16 18
Sample Number Sample Number
(a) S-chart (b) X-chart
F IGURE 5.6: Revised S- and X-charts for assembly data.

TABLE 5.2: Vane-Opening Data.

Sample Data
Number X1 X2 X3 X4 X5
1 33 29 31 32 33
2 33 31 35 37 31
3 35 37 33 34 36
4 30 31 33 34 33
5 33 34 35 33 34
6 38 37 39 40 38
7 30 31 32 34 31
8 29 39 38 39 39
9 28 33 35 36 43
10 38 33 32 35 32
11 28 30 28 32 31
12 31 35 35 35 34
13 27 32 34 35 37
14 33 33 35 37 36
15 35 37 32 35 39
16 33 33 27 31 30
17 35 34 34 30 32
18 32 33 30 30 33
19 25 27 34 27 28
20 35 35 36 33 30

(ii) Construct the R- and the X-charts.

(iii) Construct the S- and the X-charts.

Solution: Table 5.3 shows the vane-opening data with extra columns displaying the sample means, sample ranges,
and sample standard deviations. The grand mean, average range, and average standard deviation are also listed at the
bottom of the table. For a sample size of n = 5, we have D3 = 0, D4 = 2.114, A2 = 0.577, B3 = 0, B4 = 2.089, and
A3 = 1.427.
(i) Using Table 5.3, the process mean and standard deviation can be estimated as follows

R 5.8
X = 33.32 and σ̂ = = = 2.4936
d2 2.326
where d2 = 2.326 for samples of size n = 5.

101
 TABLE 5.3: Vane-Opening Data with Sample Means, Ranges, and Standard Devia-
tions.
Sample Data
Number X1 X2 X3 X4 X5 Xi Ri Si
1 33 29 31 32 33 31.60 4 1.67
2 33 31 35 37 31 33.40 6 2.61
3 35 37 33 34 36 35.00 4 1.58
4 30 31 33 34 33 32.20 4 1.64
5 33 34 35 33 34 33.80 2 0.84
6 30 31 32 34 31 31.60 4 1.52
7 38 33 32 35 32 34.00 6 2.55
8 31 35 35 35 34 34.00 4 1.73
9 27 32 34 35 37 33.00 10 3.81
10 33 33 35 37 36 34.80 4 1.79
11 35 37 32 35 39 35.60 7 2.61
12 33 33 27 31 30 30.80 6 2.49
13 35 34 34 30 32 33.00 5 2.00
14 32 33 30 30 33 31.60 3 1.52
15 35 35 36 33 30 33.80 6 2.39
16 33 33 27 31 30 30.80 6 2.49
17 35 34 34 30 32 33.00 5 2.00
18 32 33 30 30 33 31.60 3 1.52
19 25 27 34 27 28 28.20 9 3.42
20 35 35 36 33 30 33.80 6 2.39
X = 33.32 R = 5.8 S = 2.345

(ii) The upper control limit, center line, and lower control limit of the R-chart are
UCL = D4r̄ = 12.27
CL = r̄ = 5.8
LCL = D3r̄ = 0

The R-chart is analyzed first to determine if it is stable. Figure 5.7(a) shows that there is an out-of-control point on
the R-chart at sample (subgroup) 9. Assuming that the out-of-control point at sample 9 has an assignable cause,
it can be discarded from the data.
The X-chart can now be analyzed. The control limits of the X-chart are
UCL = x̄¯ + A2 r̄ = 36.67
CL = x̄¯ = 33.32
LCL = x̄¯ − A2 r̄ = 29.97

Figure 5.3(b) shows that there are out-of-control points at samples 6, 8, 11, and 19. Assuming assignable cause,
we can discard these samples from the data.
MATLAB code
>> load vaneopening.mat; %load vane-opening data
>> [stats,plotdata]=controlchart(X,’chart’,{’r’,’xbar’},’sigma’,’range’); %r & Xbar charts

Thus, if the out-of-control points at samples 6, 8, 9, 11, and 19 are discarded, then the new control limits are
calculated using the remaining 15 samples. The revised mean range and grand mean are the given by
x̄¯ = 33.21 r̄ = 5
The revised control limits for the new R-chart are
UCL = D4r̄ = 10.57
CL = r̄ = 5
LCL = D3r̄ = 0

102
 15 9 40

6
UCL 38

8 UCL
10 36
Sample Range

Sample Mean
34
CL
CL
5 32

30 11 LCL

0 LCL 19
28
2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20
Sample Number Sample Number
(a) R-chart (b) X-chart
F IGURE 5.7: R- and X-charts for the vane-opening data.

and the revised control limits for the new X-chart are

UCL = x̄¯ + A2 r̄ = 36.10

CL = x̄¯ = 33.21
LCL = x̄¯ − A2 r̄ = 30.33

The new X- and R-charts are shown in Figure 5.8. Notice now that all the points fall within the limits, indicating
that the process may be stable.

12 37

UCL UCL
10 36

35
8
Sample Range

Sample Mean

34
6

CL CL
33
4
32

2
31

0 LCL LCL
30
2 4 6 8 10 12 14 2 4 6 8 10 12 14
Sample Number Sample Number
(a) R-chart (b) X-chart
F IGURE 5.8: Revised R- and X-charts for the vane-opening data.

(iii) The upper control limit, center line, and lower control limit of the S-chart are given by

UCL = B4 s̄ = 4.899
CL = s̄ = 2.345
LCL = B3 s̄ = 0

103
 The upper control limit, center line, and lower control limit of the X-chart are given by

UCL = x̄¯ + A3 s̄ = 36.67

CL = x̄¯ = 33.32
LCL = x̄¯ − A3 s̄ = 29.97

We first examine the S-chart for signs of special variation. The S-chart are shown in Figure 5.9(a), where samples
6, 8, 11, and 19 are out-of-control. The X-chart is shown in Figure 5.9(b), where the sample number 9 is out-of-
control.

6 40

9
6
5 UCL 38
Sample Standard Deviation

8 UCL
4 36

Sample Mean
3 34
CL
CL
2 32

1 30 LCL
11

0 LCL 19
28
2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20
Sample Number Sample Number
(a) S-chart (b) X-chart
F IGURE 5.9: S- and X-charts for vane-opening data.

MATLAB code
>> load vaneopening.mat; %load vane-opening data
>> [stats,plotdata]=controlchart(X,’chart’,{’s’,’xbar’},’sigma’,’std’); %s & Xbar charts

With samples 6, 8, 9, 11, and 19 deleted, the new upper control limit, center line, and lower control limit of the
S-chart are given by

UCL = B4 s̄ = 4.281
CL = s̄ = 2.049
LCL = B3 s̄ = 0

and the new upper control limit, center line, and lower control limit of the X-chart are given by

UCL = x̄¯ + A3 s̄ = 36.14

CL = x̄¯ = 33.21
LCL = x̄¯ − A3 s̄ = 30.29

As shown in Figure 5.10, the remaining plotted points on the S- and X-charts indicate a stable process.

5.3 P ROCESS C APABILITY A NALYSIS

Process capability is the long-term performance level of the process after it has been brought under statistical control.
In other words, process capability is the range over which the natural variation of the process occurs as determined by
the system of common causes. A process may be in statistical control, but due to a high level of variation may not be

104
 4.5 37
UCL
4 UCL
36
3.5
Sample Standard Deviation

35
3

Sample Mean
2.5 34

2 CL CL
33
1.5
32
1

0.5 31

0 LCL LCL
30
2 4 6 8 10 12 14 2 4 6 8 10 12 14
Sample Number Sample Number
(a) s-chart (b) X-chart
F IGURE 5.10: Revised S- and X-charts for vane-opening data.

capable of producing output that is acceptable to customers. A process capability study assumes the system is stable
and that the data are normally distributed. A process is stable when only normal random variation is present.
A process capability analysis is simply the comparison of the distribution of a process output with the product
tolerances. Control charts limits can be compared by specification limits to determine the process capability. These
specification limits are often set by customers, management, and/or product designers. Moreover, specification limits
are usually two-sided, with upper specification limit (USL) and lower specification limit (LSL); or can be one-sided,
with either USL or LSL. Knowing the capability of your processes, we can specify better the quality performance
requirements for new machines, parts and processes. The capability of a process centered on the desired mean can
be measured using the process capability potential C p , which is defined as the ratio between the specification spread
(USL − LSL) and the process spread (6σ ):

Specification spread USL − LSL

Cp = = (5.3.1)
Process spread 6σ

where σ is the process standard deviation. The idea is illustrated graphically in Figure 5.11. Note that the specification
spread is the performance spread acceptable to customers, management, and/or product designers.
Let X be the process quality characteristic that we want to monitor. The performance of the process with respect to the
specification limits USL and LSL are defined as follows:

P( X > USL) = Percentage of nonconforming produced by the process at the upper end.

P( X < LSL) = Percentage of nonconforming produced by the process at the lower end.
Thus, the total percentage of nonconforming produced by the process is defined as

P( X < USL or X > USL) = 1 − P( LSL < X < USL) (5.3.2)

Other capability indices that are frequently used in process capability analysis include:

• Upper capability index: C pU = (USL − µ)/(3σ )

• Lower capability index: C pL = (µ − LSL)/(3σ )

• Process capability index: C pk = min(C pU , C pL )

These four measures of process capability quantify the degree to which your process produces output that meets the
customer’s specification, and can be used effectively to summarize process capability information in a convenient
unitless system. Calculating the process capability measures requires knowledge of the process mean and standard
deviation, µ and σ, which are usually estimated from data collected from the process. Assume m preliminary samples

105
 LSL USL

Specification spread: USL−LSL

Process spread: 6σ

µ − 3σ µ µ + 3σ

F IGURE 5.11: Specification spread vs. process spread.

TABLE 5.4: Process Capability Indices with σ̂ = R/d2 .

Index Estimated Equation
Cp (USL − LSL)/(6σ̂ )
C pU (USL − X )/(3σ̂)
C pL ( X − LSL)/(3σ̂ )
C pk min(C pU , C pL )

(with equal sample size n) are available, then µ̂ = X is the grand mean and σ̂ = R/d2 , where R is the mean of sample
ranges. Table 5.4 shows a summary of the process capability measures. Note that C pk ≤ C p , and are equal when X is at
target.
The lower and upper capability indices C pL and C pU are used when only one direction from the mean is important.
The process capability index, C pk , measures the distance of the process average X from the closest specification. In
other words, unlike C p , the index C pk takes process location into account. Moreover, C pk can be calculated in situations
where there is only one specification limit. Thus, C pk can be used in place of the other three capability measures. Three
possible cases can be considered:

Case 1: If C pk < 1, the process in not capable of consistently producing product within the specifications. The process
produces more than 2700 non-conforming units per million. It is impossible for the current process to meet
specifications even when it is in statistical control. If the specifications are realistic, an effort must be immediately
made to improve the process (i.e. reduce variation) to the point where it is capable of producing consistently
within specifications.

Case 2: If C pk ≥ 1.33, the process in highly capable and produces less than 64 non-conforming units per million. C pk
values of 1.33 or greater are considered to be industry benchmarks.

Case 3: If 1 ≤ C pk < 1.33, the process in barely capable and produces more than 64 but less than 2700 non-conforming
units per million. This process has a spread just about equal to specification width. It should be noted that if the
process mean moves to the left or the right, a significant portion of product will start falling outside one of the
specification limits. This process must be closely monitored.

Example 5.7 A pharmaceutical company carried out a process capability study on the weight of tablets produced and showed that
the process was in-control with a process mean X = 2504 mg and a mean range R = 91 mg from samples of size n = 4. Compute
the process capability indices for the specifications limits 2800 mg and 2200 mg, and interpret your result.

106
Solution: From the given information, we have X = 2504, R = 91, LSL = 2200, USL = 2800, and n = 4. For a sample
of size n = 4, Appendix Table V gives d2 = 2.059. Thus,

R 91
σ̂ = = = 44.1962
d2 2.059
USL − LSL
Cp = = 2.263
6σ̂
USL − X
C pU = = 2.232
3σ̂
X − LSL
C pL = = 2.293
3σ̂
C pk = min(C pU , C pL ) = 2.232

Since C pk = 2.232 ≥ 1.33, the process is highly capable. Moreover, C pU is smaller that C pL , which indicates that the
process is skewed more to the high side. Thus, the process is highly capable of meetings the requirements but not
centered. Some corrective action may have to be taken to centralize the process.

MATLAB code
n = 4; d2 = 2.059; %sample size equal 4
LSL = 2200; USL = 2800;
Xbarbar = 2504; Rbar = 91;
sigma = Rbar/d2;
Cp = (USL - LSL)/(6*sigma);
Cpl = (Xbarbar-LSL)/(3*sigma);
Cpu = (USL - Xbarbar)/(3*sigma);
Cpk = min(Cpl,Cpu);
fprintf(’Process Capability Indices: Cp=%.3f, Cpl=%.3f, Cpu=%.3f, ...
Cpk=%.3f\n’,Cp,Cpl,Cpu,Cpk);

Example 5.8 Assume that the vane-opening data (Example 5.6) are normally distributed and that the specifications are 34 ± 6.5.

(i) Determine the process capability index.

(ii) What proportion of the product will not meet specifications?

Solution: From the given information, we have LSL = 27.5 and USL = 40.5. From the revised R- and X-charts in
Example 5.6, we have X = 33.21 and R = 5. Thus, the estimated process standard deviation is σ̂ = R/d2 = 5/2.326 =
2.1496.

(i) The process capability indices are given by

USL − LSL
Cp = = 1.0079
6σ̂
USL − X
C pU = = 1.1299
3σ̂
X − LSL
C pL = = 0.8859
3σ̂
C pk = min(C pU , C pL ) = 0.8859

The C pk value is quite low (C pk < 1), which indicates that the process in not capable of consistently producing
product within the specification. Since C pL is smaller than C pU , the process is skewed more to the low side and
reflects a relatively poor capability in meeting the low side of the design specification.

107
 (ii) The total percentage of nonconforming produced by the process is

Percentage of Nonconforming = 1 − P( LSL < X < USL)

!
LSL − X X−X USL − X
= 1−P < <
σ̂ σ̂ σ̂
= 1 − P(−2.6578 < Z < 3.3898)
= 1 − (Φ(3.3898) − Φ(−2.6578)) = 0.0043

where Φ(·) is the cdf of the standard normal distribution N (0, 1). Thus, the proportion of product not meeting
specifications is 0.43%, which is quite low. Although we found that the process is in control, it is however not ca-
pable of meeting the stated specifications. In this case, common causes must be found for process improvement.

Example 5.9 A dimension has specifications of 2.125 ± 0.005. Data from the process indicate that the distribution is normally
distributed, and the X- and R-charts indicate that the process is stable. The control chart used a sample of size five and it is found
that X = 2.1261 and R = 0.0055. Determine the fraction of the manufactured product that will have this particular dimension
outside the specification limits.

Solution: From the given information, we have LSL = 2.120 and USL = 2.130, X = 2.1261, and R = 0.0055. For
a sample of size n = 5, Appendix Table V gives d2 = 2.326. Thus, the estimated process standard deviation is σ̂ =
R/d2 = 0.0055/2.326 = 0.00236.
The total percentage of nonconforming produced by the process is

Percentage of Nonconforming = 1 − P( LSL < X < USL)

!
LSL − X X−X USL − X
= 1−P < <
σ̂ σ̂ σ̂
= 1 − P(−2.58 < Z < 1.65)
= 1 − (Φ(1.65) − Φ(−2.58)) = 0.0544

where Φ(·) is the cdf of the standard normal distribution N (0, 1). Thus, approximately 5.44% of the products will fall
outside specification for this quality characteristic.

5.4 I NDIVIDUAL C ONTROL C HARTS

Some situations exist in which the sample consists of a single observation, that is, the sample size is equal to 1. A
sample of size one can occur when production is very slow or costly, and it is impractical to allow the sample size to
be greater than one. The individual control charts, I- and MR-charts, for variable data are appropriate for this type of
situation. The I-chart (also called X-chart) serves the same function as the X-chart except that now X is the value of the
individual measurement. Assuming that X ∼ N (µ x , σx2 ), the control limits of the I-chart are then given by

UCL = µ x + 3σx
CL = µ x (5.4.1)
LCL = µ x − 3σx

Since µ x and σx are unknown, they need to be estimated. Suppose m preliminary observations (samples) are avail-
able, each of size one. Then the process mean µ x can be estimated as the average of the individual measurements
µ̂ x = x̄ = (1/m) ∑m i =1 x i .
Since only individual measurements are available, the moving ranges MRi = | Xi − Xi−1 |, i = 2, . . . , m between
two successive samples Xi−1 and Xi need to be calculated to estimate the process variability (standard deviation) σ as
follows:
MR
σ̂ = (5.4.2)
d2

108
where
m
1
MR = ∑
m − 1 i =2
MRi (5.4.3)

Note that since the data are taken as pairs { Xi−1 , Xi } to calculate the moving ranges MRi = | Xi − Xi−1 |, the value of d2
is then equal to 1.128. Notice that division is done by m − 1 since only m − 1 moving range values are calculated (there
is no moving range for subgroup 1), where m is the number of observations.

Control chart for individuals (I-chart):

The upper control limit, center line, and lower control limit for the I-chart are given by

mr mr
UCL = x̄ + 3 = x̄ + 3
d2 1.128
CL = x̄ (5.4.4)
mr mr
LCL = x̄ − 3 = x̄ − 3
d2 1.128

Example 5.10 A company is manufacturing high precision tubes. Quality Control department is interested to determine if the
production process in under control. For simplicity, we assume that from each batch is measured the outer diameter of a random
selected tube. Measured data are given in Table 5.5. Construct the I-chart.

TABLE 5.5: Outer diameter of tubes.

Sample 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Diameter 99.82 99.63 99.89 99.45 100.03 99.76 100.23 99.81 99.91 100.12 100.05 99.78 100.01 100.04 99.95

Solution: The upper control limit, center line, and lower control limit for the I-chart are given by
mr 0.2593
UCL = x̄ + 3 = 99.8987 + 3 = 100.588
d2 1.128
CL = x̄ = 99.8987
mr 0.2593
LCL = x̄ − 3 = 99.8987 − 3 = 99.2093
d2 1.128

1 X=[99 .8 2 99 .6 3 99 .8 9 99 .4 5 100 .0 3 99 .7 6 100 .2 3 99 .8 1 . . .

2 99 .9 1 100 .1 2 100 .0 5 99 .7 8 100 .0 1 100 .0 4 99 .9 5 ] ; %o u t e r d i a m e te r data
3 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' , ' i ' , ' w i d th ' , 2 ) ; %p l o t I −c h a r t
4 f p r i n t f ( ' C o n t r o l l i m i t s f o r i −c h a r t : UCL=%g , CL=%g , LCL=%g\n ' , . . .
5 plotdata.ucl (1) , plotdata.cl (1) , p lotdata .lcl (1) ) ;

The I-chart is shown in Figure 5.12, where all samples appear to be in-control. Thus, the process of producing the tubes
is considered to be in statistical control.

On the other hand, the MR-chart is used to monitor the process variability. it can be shown that µ MR and σMR can
be estimated as
d
µ̂ MR = MR and σ̂MR = 3 MR (5.4.5)
d2

Control chart for moving ranges (MR-chart):

The upper control limit, center line, and lower control limit for the MR-chart are given by

UCL = D4 mr = 3.267 mr
CL = mr (5.4.6)
LCL = D3 mr = 0

109
 100.8

100.6 UCL

100.4

Individual Value
100.2

100
CL
99.8

99.6

99.4

99.2 LCL
2 4 6 8 10 12 14
Sample Number

F IGURE 5.12: I-chart for outer diameter data.

It is important to note that the moving range control chart can not be interpreted in the same way as the R chart
presented earlier, with respect to patterns or trends. Patterns or trends identified on the moving range chart do not
necessarily indicate that the process is out of control. The moving ranges MRi = | Xi − Xi1 | are correlated. There is a
natural dependency between successive MRi values.

Example 5.11 Using the data of Table 5.5, construct the MR-chart.

Solution: The upper control limit, center line, and lower control limit for the MR-chart are given by

UCL = D4 mr = (3.267)(0.2593) = 0.8471

CL = mr = 0.2593
LCL = D3 mr = 0

1 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' , ' mr ' , ' w i d th ' , 2 ) ; %p l o t MR− c h a r t

2 f p r i n t f ( ' C o n t r o l l i m i t s f o r mr− c h a r t : UCL=%g , CL=%g , LCL=%g\n ' , . . .
3 plotdata.ucl (2) , plotdata.cl (2) , p lotdata .lcl (2) ) ;

The mr-chart is shown in Figure 5.13, where all samples appear to be in-control.

Example 5.12 Packages of a particular instant dry food are filled by a machine and weighed. The weights (in ounces) for 15
successive packages have been collected and are displayed in Table 5.6. Construct the I- and MR-charts.

TABLE 5.6: Weights for dry food packages.

Bottle 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Weight 19.85 19.92 19.93 19.26 20.36 19.96 19.87 19.80 20.40 19.98 20.17 19.81 20.21 19.64 20.15

Solution: The moving ranges are calculated using MRi = | Xi − Xi−1 |. To illustrate, consider the first moving range at
subgroup 2:
MR2 = | X2 − X1 | = 19.92 − 19.85 = 0.07

The remaining moving ranges are calculated accordingly and are given in Table 5.7.

110
 0.9
UCL
0.8

0.7

0.6

Moving Range
0.5

0.4

0.3
CL
0.2

0.1

0 LCL
2 4 6 8 10 12 14
Sample Number

F IGURE 5.13: MR-chart for outer diameter data.

TABLE 5.7: Weights for dry food packages with moving ranges.
Bottle 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Weight 19.85 19.92 19.93 19.26 20.36 19.96 19.87 19.80 20.40 19.98 20.17 19.81 20.21 19.64 20.15
Moving Range - 0.07 0.01 0.67 1.10 0.40 0.09 0.07 0.60 0.42 0.19 0.36 0.40 0.57 0.51

The upper control limit, center line, and lower control limit for the I-chart are given by

mr 0.39
UCL = x̄ + 3 = 19.954 + 3 = 20.9909
d2 1.128
CL = x̄ = 19.954
mr 0.39
LCL = x̄ − 3 = 19.954 − 3 = 18.9171
d2 1.128

The upper control limit, center line, and lower control limit for the MR-chart are given by

UCL = D4 mr = (3.267)(0.39) = 1.27395

CL = mr = 0.39
LCL = D3 mr = 0

The I- and MR-charts are displayed in Figure 5.14, which shows that all samples fall within the control limits. Thus,
the process appears to be in statistical control.

5.5 C UMULATIVE S UM C ONTROL C HART (CUSUM- CHART )

Although Shewhart charts with 3σ limits can quickly detect large process changes, they are ineffective for small, sus-
tained process changes (for example, changes smaller than 1.5σ). An alterative control chart has been developed to
detect small shifts in the process mean is the so-called cumulative sum (CUSUM) control chart (CUSUM-chart), which
is more sensitive to small shifts in the process because it is based on not only the current observation, but also the most
recent past observations. Moreover, a CUSUM-chart is especially effective with samples of size n = 1.
Suppose m preliminary samples of size n ≥ 1 are available. Then, the CUSUM-chart plots the cumulative sums Ci of
deviations of the observations from some target mean µ0
" #
i i −1
Ci = ∑ (x̄ j − µ0 ) = ∑ (x̄ j − µ0 ) + ( x̄i − µ0 ) = Ci−1 + ( x̄i − µ0 ), i = 1, . . . , m (5.5.1)
j =1 j =1

111
 21 UCL 1.4

UCL
1.2
20.5
1
Individual Value

Moving Range
20 CL 0.8

0.6
19.5

0.4 CL

19
LCL 0.2

0 LCL
18.5
2 4 6 8 10 12 14 2 4 6 8 10 12 14
Sample Number Sample Number

F IGURE 5.14: I- and MR-charts for package weights.

against the subgroup (sample i), where x̄ j is the mean of the j-th sample (j = 1, . . . , i).
As long as the process remains “in control” at the target mean µ0 , the cumulative sums, Ci , will be approximately
zero. Otherwise, if the process shifts away from the target mean µ0 , then Ci become increasingly large in absolute
value. The tabular CUSUM for monitoring the process mean involves two statistics, Ci+ and Ci− , defined as

Ci+ = max[0, x̄i − (µ0 + K ) + Ci+−1 ]

(5.5.2)
Ci− = max[0, (µ0 − K ) − x̄i + Ci−−1 ]

where

• Ci+ is the accumulation of deviations above the target mean, with initial value C0+ = 0

• Ci− is the accumulation of deviations below the target mean, with initial value C0− = 0

• K is called the reference value given by K = |µ1 − µ0 |/2, where µ1 is the out-of-control mean that we are interested
in detecting.

A deviation from the target that is larger than K increases either the one-sided upper CUSUM Ci+ or the one-sided
lower CUSUM Ci− . If the out-of-control mean µ1 is unknown, there are methods for determining the value K. In this
√
situation we can let K = kσx̄ , where k is some constant chosen so that a particular shift is detected, and σx̄ = σ/ n with
σ denoting the process standard deviation. For example, say a shift from target of 1 standard deviation is important
to detect (i.e., detect whether the target has shifted to µ0 + 1σx̄ or µ0 − 1σx̄ , then k = 1, and K = 1σx̄ ). If the process
standard deviation is not known, it must be estimated from the data provided. The two-sided CUSUM-chart plots the
values of Ci+ and Cii for each sample i. A control limit violation (out-of-control) occurs when either Ci+ or Cii exceeds a
specified control limit (or threshold) H = hσx̄ , where h is typically equal to 5.

√ of CUSUM limits calculation, it is preferable to use the standardized value yi = ( x̄i − µ0 )/σx̄ =
For simplicity
( x̄i − µ0 )/(σ/ n) of the variable xi . Since the value of σ is unknown, it is usually estimated as σ̂ = MR/d2 for
individual observations, and as σ̂ = R/d2 for samples of size n > 1.

112
Control chart for standardized cumulative sums (CUSUM-chart):
The one-sided upper and lower CUSUMs of the standardized CUSUM-chart are given by

Ci+ = max[0, yi − k + Ci+−1 ]

(5.5.3)
Ci− = max[0, −k − yi + Ci−−1 ]

The upper control limit, center line, and lower control limit of the CUSUM-chart are given by

UCL = h
CL = 0 (5.5.4)
LCL = −h

Example 5.13 The data given in Table 5.8 are average readings from a process, taken every hour. (Read the observations down,
from left). The target value for the mean is µ0 = 160.

TABLE 5.8: Process readings.

159.0480 160.2766 160.3368 162.0092 162.2135
156.8969 159.2432 157.7590 159.2716 164.2690
159.1034 160.2861 160.8015 160.2469 159.7530
156.8976 163.2101 162.4781 162.5760 158.2998
161.8597 162.6982 162.8001 161.5964 158.6755
161.8039 159.1031 157.9376 160.6725 159.1114

1. Estimate the process standard deviation

2. Set up and apply a tabular CUSUM for this process, using standardized values h = 5 and k = 0.5. Interpret this chart.
Solution:
1. Since the data are individual observations, an estimate the process standard deviation is then given by

MR 2.0112
σ̂ = = = 1.7830
d2 1.128

1 X=[159 .0480 156 .8969 159 .1034 156 .8976 161 .8597 161 .8039 160 .2766 159 .2432 . . .
2 160 .2861 163 .2101 162 .6982 159 .1031 160 .3368 157 .7590 160 .8015 162 .4781 . . .
3 162 .8001 157 .9376 162 .0092 159 .2716 160 .2469 162 .5760 161 .5964 160 .6725 . . .
4 162 .2135 164 .2690 159 .7530 158 .2998 158 .6755 159 .1114 ] ;
5 mu0 = 160; %t a r g e t v a l u e f o r th e mean
6 MR = s l i d e f u n ( @range , 2 , X ) ; %moving range
7 MR2bar = mean(MR( 2 : end ) ) ;
8 d2 = 1 .128 ; %from Table i n th e Appendix
9 sigmahat = MR2bar / d2 ; %e s t i m a t e o f th e process s ta n d a r d d e v i a t i o n
10 h = 5;
11 k = 0 .5 ;
12 [ Cplus , Cminus ] = cusumchart ( X , mu0 , sigmahat , h , k ) ; %p l o t CUSUM−c h a r t

2. The CUSUM-chart is shown in Figure 5.15, where the process appears to be in-control. So there does not appear
to have been a shift of 0.5σ from the target value of 160.

5.6 E XPONENTIALLY W EIGHTED M OVING AVERAGE C ONTROL C HART (EWMA- CHART )

Another alterative control chart that is generally used for detecting small shifts in the process mean is the so-called ex-
ponentially weighted moving average (EWMA) chart, and it plots weighted moving average values. Like the CUSUM-
chart, the EWMA-chart is also preferred when the samples are of size n = 1 (i.e. individual measurements). Suppose

113
 Standardized Cumulative Sum Control Chart
5 UCL

Standardized Cumulative Sum

3

0 CL

−1

−2

−3

−4

−5 LCL
0 5 10 15 20 25 30
Sample Number

F IGURE 5.15: CUSUM-chart for process readings.

m preliminary samples of size n ≥ 1 are available. The EWMA statistic is defined by

zi = λ x̄i + (1 − λ)zi−1, i = 1, . . . , m (5.6.1)

where
• the weighting factor, 0 < λ ≤ 1, determines the depth of memory for the EWMA
• x̄i is the most current observation (i.e. average of the sample at time i)
• zi−1 is the previous EWMA statistic in which the initial value z0 is equal to the process target mean µ0 . If µ0 is
unknown, then z0 = x̄.
The EWMA-chart plots zi against the sample i. Let σ be the process standard deviation (i.e. σ = σx̄ ). It can be shown
that for large i, the standard deviation of zi is given by
r r
λ σ λ
σzi = σx̄ [1 − (1 − λ)2i ] ≈ √ , as i → ∞ (5.6.2)
2−λ n 2−λ

Note that large i means that EWMA control chart has been running for several time periods.

Control chart for the exponentially weighted moving average (EWMA-chart):

The upper control limit, center line, and lower control limit of the EWMA-chart are given by
r
σ̂ λ
UCL = µ0 + L √
n 2−λ
CL = µ0 (5.6.3)
r
σ̂ λ
LCL = µ0 − L √
n 2−λ

where L is the width of the control limits. For individual observations, we use σ̂ = MR/d2 ; and for samples of size
n > 1, we use σ̂ = R/d2 .

The values of the parameters L and λ can have a considerable impact on the performance of the chart. The parameter
λ determines the rate at which past (historical) data enter into the calculation of the EWMA statistic. A value of λ = 1

114
implies that only the most recent observation influences the EWMA. Thus, a large value of λ gives more weight to
recent data and less weight to historical data, while a small value of λ gives more weight to historical data. Although
the choice of λ and L is arbitrary. In practice, the values 0.05 ≤ λ ≤ 0.25 and 2.6 ≤ L ≤ 3 work well. Note that L = 3
matches other control charts, but it may be necessary to reduce L slightly for small values of λ.

Example 5.14 Using the data of Table 5.8:

1. Apply an EWMA control chart to these data using λ = 0.15 and L = 2.7
Solution: The EWMA-chart is shown in Figure 5.16, where the process is out-of-control at sample number 26.

1 lambda = 0 .1 5 ;
2 L = 2.7 ;
3 ewmachart ( X , lambda , L , mu0 , sigmahat ) ; %p l o t EWMA− c h a r t

161.5 26
UCL

161

160.5
EMWA

160 CL

159.5

159

LCL
5 10 15 20 25 30
Sample Number

F IGURE 5.16: EWMA-chart for process readings.

Example 5.15 The concentration of a chemical product is measured by taking four samples from each batch of material. The aver-
age concentration of these measurements is shown for the last 20 batches in in Table 5.9. Assume the target value of concentration
for this process is 100.

TABLE 5.9: Concentration Measurements.

Batch Concentration Batch Concentration
1 104.50 11 95.40
2 99.90 12 94.50
3 106.70 13 104.50
4 105.20 14 99.70
5 94.80 15 97.70
6 94.60 16 97.00
7 104.40 17 95.80
8 99.40 18 97.40
9 100.30 19 99.00
10 100.30 20 102.60

(i) Estimate the process standard deviation

115
 (ii) Set up and apply a tabular CUSUM for this process, using standardized values h = 5 and k = 0.5. Does the process appear
to be in control at the target?

(iii) Apply an EWMA control chart to these data using λ = 0.1 and L = 2.7. Interpret this chart.

Solution:

(i) Since the data are individual observations, an estimate the process standard deviation is then given by

MR 3.71
σ̂ = = = 2.29
d2 1.128

(ii) The CUSUM-chart is given in Figure 5.17(a), which shows that the process is in-control.

(iii) The EWMA-chart is given in Figure 5.17(b), which shows that the process is in-control.

Standardized Cumulative Sum Control Chart

5 UCL 102 UCL

4
101.5
Standardized Cumulative Sum

3
101
2
100.5
1 EMWA
0 CL 100 CL

−1
99.5
−2
99
−3
98.5
−4

−5 LCL 98 LCL
0 5 10 15 20 2 4 6 8 10 12 14 16 18 20
Sample Number Sample Number
(a) (b)
F IGURE 5.17: CUSUM- and EWMA-charts for concentration measurements.

Example 5.16 Packages of a particular instant dry food are filled by a machine and weighed. The weights (in ounces) for 24
successive packages have been collected and are displayed in Table 5.10. Assume the target mean weight for this process is 20
ounces.

TABLE 5.10: Weights for dry food packages.

Package Weight Package Weight
1 20.26 13 20.30
2 19.97 14 19.77
3 19.76 15 20.40
4 19.72 16 19.98
5 19.69 17 19.91
6 19.85 18 20.18
7 19.96 19 20.08
8 20.03 20 20.05
9 20.06 21 20.20
10 19.71 22 19.90
11 19.68 23 19.95
12 19.94 24 20.12

(i) Estimate the process standard deviation

116
 (ii) Set up and apply a tabular CUSUM for this process, using standardized values h = 5 and k = 0.5. Does the process appear
to be in control at the target?
(iii) Apply an EWMA control chart to these data using λ = 0.1 and L = 2.7. Interpret this chart.
Solution:
(i) Since the data are individual observations, an estimate the process standard deviation is then given by

MR 0.20
σ̂ = = = 0.18
d2 1.128

(ii) The CUSUM-chart is given in Figure 5.18(a), which shows that the process is in-control. Thus, there does not
appear to have been a shift of 0.5σ from the target value of 20 ounces.
(iii) The EWMA-chart is given in Figure 5.18(b), which shows that the process is in-control since all EWMA statistics
fall within the control limits.

Standardized Cumulative Sum Control Chart

5 UCL UCL
20.1
4
20.08
Standardized Cumulative Sum

3
20.06
2 20.04

1 20.02
EMWA

0 CL 20 CL

−1 19.98

19.96
−2
19.94
−3
19.92
−4
19.9
−5 LCL LCL
0 5 10 15 20 25 5 10 15 20
Sample Number Sample Number

F IGURE 5.18: CUSUM- and EWMA-charts for dry food packages.

5.7 C ONTROL C HARTS FOR ATTRIBUTES

In quality control, a defective quality characteristic is called a defect or non-conformance, whereas a unit that has at
least one defect is called a defective or nonconforming unit. In other words, a defective unit may have more than one
defect.

5.7.1 C ONTROL C HART FOR P ROPORTION D EFECTIVE : p- CHART

The proportion or fraction defective in a population is defined as the ratio of the number of defective items in the
population to the total number of items in that population. The goal of the fraction nonconforming control chart is
to monitor the proportion of defective units (fraction defective) for a process of interest using the data collected over
m samples (subgroups) each of size n. The p-chart plots the sample proportions (fraction defectives) vs the sample
number. If D is the number of units that are defective in a random sample of size n, then D ∼ bino (n, p). The sample
proportion defective P̂ = D/n is the ratio of the numberpof defective units in the sample, D, to the sample size n. The
mean and standard deviation of P̂ are µ P̂ = p and σP̂ = p(1 − p)/n, respectively.
Since p is unknown, it must be estimated from the available data. Suppose m preliminary samples are available,
each of size n. If Di is the number of defectives in the i-th sample, then the fraction defective of the i-th sample is
P̂i = Di /n, i = 1, . . . , m. An unbiased estimator of µ P̂ is the mean P of the fraction defectives of m samples,
m
1 1 m
CL = P =
m ∑ P̂i = mn i∑
Di
i =1 =1

117
 p
Since σP̂ = p(1 − p)/n, the upper and control limits may be expressed as
s
P (1 − P )
UCL = P + 3
n

and s
P(1 − P)
LCL = P + 3
n

Control chart for proportion defective (p-chart):

The upper control limit, center line, and lower control limit of the p-chart are given by
r
p̄(1 − p̄)
UCL = p̄ + 3
n
CL = p̄ (5.7.1)
r
p̄(1 − p̄)
LCL = p̄ − 3
n

Example 5.17 A quality control inspector wishes to construct a fraction-defective control chart for a light bulb production line.
Packages containing 1000 light bulbs are randomly selected, and all 1000 bulbs are light-tested. The results of the tests are given
in Table 5.11. Construct and plot a p-chart.

TABLE 5.11: Number of defectives observed in samples of 1000 light bulbs.

Sample 1 2 3 4 5 6 7 8 9 10 11 12 13
Number of Defectives 9 12 13 12 11 9 7 0 12 8 9 7 11

Solution: The center line CL is given by

∑ (Number of Defectives) 120

CL = p̄ = = = 0.0092
(13)(1000) 13000

The upper and lower control limits are given by

r
p̄(1 − p̄)
UCL = p̄ + 3 = 0.0183
n
r
p̄(1 − p̄)
LCL = p̄ − 3 = 0.0001
n

1 X = [ 9 12 13 12 11 9 7 0 12 8 9 7 1 1 ] ; % Number o f d e f e c t i v e s
2 n = 1000; %t o t a l number o f i n s p e c te d i te m s
3 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' , ' p ' , ' u n i t ' , n ) ; %p l o t p− c h a r t
4 f p r i n t f ( ' C o n t r o l l i m i t s f o r p−c h a r t : UCL=%g , CL=%g , LCL=%g\n ' , . . .
5 plotdata.ucl (1) , plotdata.cl (1) , p lotdata .lcl (1) ) ;

The p-chart, shown in Figure 5.19, indicates that the sample number 8 is outside the control limits.

Example 5.18 When a coupon redemption process is in control, then a maximum of 3% of the rebates are done incorrectly, for a
maximum acceptable proportion of errors of 0.03. For 20 sequential samples of 100 coupon redemptions each, an audit reveals that
the number of errors found in the rational subgroup samples are given in Table 5.12. Construct and plot a p-chart.

118
 −3
x 10
20

18 UCL

16

Proportion Defective
14

12

10
CL
8

2
8
0 LCL
2 4 6 8 10 12
Sample Number

F IGURE 5.19: p-chart for lightbulb data.

TABLE 5.12: Number of errors observed in samples of 100 coupon redemptions.

Sample 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Number of Errors 2 2 3 6 1 3 6 4 7 2 5 6 3 2 4 5 3 8 1 4

Solution: The center line CL is given by

∑ (Number of Errors) 77
CL = p̄ = = = 0.0385
(20)(100) 2000
The upper and lower control limits are given by
r
p̄(1 − p̄)
= 0.09622 UCL = p̄ + 3
n
r
p̄(1 − p̄)
LCL = p̄ − 3 =0
n
The p-chart is shown in Figure 5.20. As can be observed in the figure, only common-cause variation is included in the
chart, and the process appears to be stable.

0.1
UCL
0.09

0.08
Proportion Defective

0.07

0.06

0.05

0.04 CL

0.03

0.02

0.01

0 LCL
2 4 6 8 10 12 14 16 18 20
Sample Number

F IGURE 5.20: p-chart for the coupon data.

119
5.7.2 C ONTROL C HART FOR N UMBER OF D EFECTIVES : np- CHART
The np control chart is a slight variation of the p-chart, except now the actual number of defective items Di are plotted
on the control chart against the sample number, where i = 1, . . . , m. The control limits are based on the number of
defective units instead of the fraction defective. The np-chart and the p-chart will give the same resulting information.
That is, if the p-chart indicates an out-of-control situation for a process, then the np-chart for the same data will also
signal out-of-control. For the np-chart the average fraction defective is estimated as
m
1
P=
mn ∑ Di
i =1

Control chart for number of defectives (np-chart):

The upper control limit, center line, and lower control limit of the np-chart are given by
q
UCL = n p̄ + 3 n p̄(1 − p̄)
CL = n p̄ (5.7.2)
q
LCL = n p̄ − 3 n p̄(1 − p̄)

Example 5.19 Using the data in Table 5.11, construct and plot an np-chart.

Solution: The center line CL is given by

CL = n p̄ = 9.23077
The upper and lower control limits are given by
q
UCL = n p̄ + 3 n p̄(1 − p̄) = 18.3033
q
LCL = p̄ − 3 n p̄(1 − p̄) = 0.15828

1 X = [ 9 12 13 12 11 9 7 0 12 8 9 7 1 1 ] ; % Number o f d e f e c t i v e s
2 n = 1000; %t o t a l number o f i n s p e c te d i te m s
3 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' , ' np ' , ' u n i t ' , n ) ; %p l o t np− c h a r t
4 f p r i n t f ( ' C o n t r o l l i m i t s f o r np− c h a r t : UCL=%g , CL=%g , LCL=%g\n ' , . . .
5 plotdata.ucl (1) , plotdata.cl (1) , p lotdata .lcl (1) ) ;

The np-chart, shown in Figure 5.21, indicates that the sample number 8 is outside the control limits.

Example 5.20 For each of the 15 days, a number of magnets used in electric relays are inspected and the number of defectives
is recorded. The total number of magnets tested is 15,000. The results are given in Table 5.20. Construct and plot the p-and
np-charts.

TABLE 5.13: Number of defectives observed in samples of 1000 magnets.

Day Number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Number of Defectives 201 152 190 214 186 193 183 225 168 182 174 182 182 169 187

Solution: Since the total number of defectives in 15 days is 15000, the average sample size is n = 15000/15 = 1000.

• Control limits for the p-chart: UCL = 0.22277, CL = 0.185867, LCL = 0.148963

• Control limits for the np-chart: UCL = 222.77, CL = 185.867, LCL = 148.963

120
 20

18 UCL

16

Number of Defectives
14

12

10
CL
8

2
8
0 LCL
2 4 6 8 10 12
Sample Number

F IGURE 5.21: p-chart for lightbulb data.

A close examination of the control charts, shown in Figure 5.22, reveals that for the 8th week, the sample is above the
upper control limit. This indicates a significantly high percentage defective, which implies that there is an assignable
cause on the manufacturing process. Such cases may also result due to a lapse from the inspection department or the
sample size being quite different from the average used to calculate the control limits.

1 X = [201 152 190 214 186 193 183 225 168 182 174 182 182 169 1 8 7 ] ; % Number o f d e f e c t i v e s
2 n = 1000; %t o t a l number o f i n s p e c te d i te m s
3 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' , ' p ' , ' u n i t ' , n ) ; %p l o t p− c h a r t
4 f p r i n t f ( ' C o n t r o l l i m i t s f o r p−c h a r t : UCL=%g , CL=%g , LCL=%g\n ' , . . .
5 plotdata.ucl (1) , plotdata.cl (1) , p lotdata .lcl (1) ) ;
6 figure ;
7 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' , ' np ' , ' u n i t ' , n ) ; %p l o t np− c h a r t
8 f p r i n t f ( ' C o n t r o l l i m i t s f o r np− c h a r t : UCL=%g , CL=%g , LCL=%g\n ' , . . .
9 plotdata.ucl (1) , plotdata.cl (1) , p lotdata .lcl (1) ) ;

0.23 230
8 8
UCL UCL
0.22 220

0.21 210
Number of Defectives
Proportion Defective

0.2 200

0.19 190
CL CL
0.18 180

0.17 170

0.16 160

0.15 LCL 150 LCL

140
2 4 6 8 10 12 14 2 4 6 8 10 12 14
Sample Number Sample Number
(a) (b)
F IGURE 5.22: (a) p-chart, and (b) np-chart for magnet data.

121
5.7.3 C ONTROL C HART FOR C OUNT OF D EFECTS : c- CHART
The c-chart plots the numbers of defects per item. Assume that the number of defects X in a given inspection unit
follows a Poisson
√ distribution poiss(c), with parameter c. Then, the mean and standard deviation of X are µ X = c
and σX = c, respectively. Since c is unknown, it must be estimated from the available data. Suppose m preliminary
samples are available, each of size n. If Xi is the number of defects per sample, then an estimator of the number of
defects over the entire data set is given by
∑ m Xi
C = i =1
m

Control chart for count of defects (c-chart):

The upper control limit, center line, and lower control limit of the c-chart are given by
√
UCL = c̄ + 3 c̄
CL = c̄ (5.7.3)
√
LCL = c̄ − 3 c̄

Example 5.21 The number of noticeable defects found by quality control inspectors in a randomly selected 1-square-meter spec-
imen of woolen fabric from a certain loom is recorded each hour for a period of 20 hours. The results are shown in Table 5.14.
Construct and plot a c-chart to monitor the textile production process.

TABLE 5.14: Number of defects observed in specimens of woolen fabric over 20 con-
secutive hours.
Hour 1 2 3 4 5 6 7 8 9 10
Number of Defects 11 14 10 8 3 9 10 2 5 6
Hour 11 12 13 14 15 16 17 18 19 20
Number of Defects 12 3 4 5 6 8 11 8 7 9

Solution: The center line CL is the mean number of defects per square meter of woolen fabric given by

∑ (Number of Defects) 151

CL = c̄ = = = 7.55
20 20
The upper and lower control limits are given by
√
UCL = c̄ + 3 c̄ = 15.7932
√
LCL = c̄ − 3 c̄ = −0.69

1 X = [1 1 14 10 8 3 9 10 2 5 6 12 3 4 5 6 8 11 8 7 9 ] ; % Number o f d e f e c t s
2 n = 2 0 ; %s i z e o f i n s p e c te d u n i t s ( every 20 hours ) : sample s i z e
3 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' , ' c ' , ' u n i t ' , n ) ; %p l o t c− c h a r t
4 f p r i n t f ( ' C o n t r o l l i m i t s f o r c−c h a r t : UCL=%g , CL=%g , LCL=%g\n ' , . . .
5 plotdata.ucl (1) , plotdata.cl (1) , p lotdata .lcl (1) ) ;

Since a negative number of defects cannot be observed, the LCL value is set to 0. The c-chart, shown in Figure 5.23,
indicate that the process is in control.

Example 5.22 Samples of fabric from a textile mill, each 100 m2 , are selected, and the number of occurrences of foreign matter are
recorded. Data for 25 samples are shown in Table 5.15. Construct and plot a c-chart for the number of nonconformities.
Solution: The center line CL is the mean number of nonconformities
∑ (Number of Nonconformities ) 151
CL = c̄ = = = 7.56
100 25

122
 16 UCL

14

12

Count of Defects
10

8
CL

0 LCL
2 4 6 8 10 12 14 16 18 20
Sample Number

F IGURE 5.23: c-chart for woolen fabric data.

TABLE 5.15: Foreign Matter Data.

Sample Number 1 2 3 4 5 6 7 8 9 10 11 12 13
Nonconformities 5 4 7 6 8 5 6 5 16 10 9 7 8
Sample Number 14 15 16 17 18 19 20 21 22 23 24 25
Nonconformities 11 9 5 7 6 10 8 9 9 7 5 7

The upper and lower control limits for the c-chart are given by
√
UCL = c̄ + 3 c̄ = 15.8086
√
LCL = c̄ − 3 c̄ = 0

The c-chart is displayed in Figure 5.24, which indicates that the sample number 9 is out-of-control.

16 9 UCL

14
Count of Nonconformities

12

10

8
CL

0 LCL

5 10 15 20 25
Sample Number

F IGURE 5.24: c-chart for foreign matter data.

Assuming special causes for the out-of-control point (sample 9 is deleted), the revised centerline and control limits for

123
the c-chart are

CL = c̄ = 7.20833
√
UCL = c̄ + 3 c̄ = 15.2628
√
LCL = c̄ − 3 c̄ = 0

The revised c-chart is given in Figure 5.25, which shows that all the remaining points fall within the control limits.

16
UCL
14
Count of Nonconformities

12

10

8
CL
6

0 LCL
5 10 15 20
Sample Number

F IGURE 5.25: Revised c-chart for foreign matter data.

5.7.4 C ONTROL C HART FOR D EFECTS PER U NIT (u- CHART )

The u-chart monitors the average number of defects. Like the c-chart, the control limits in the u-chart are computed
based on the Poisson distribution. Suppose m preliminary samples are available, each of size n. In this chart we plot
the rate of defects Ui = Xi /n, which is the number of defects Xi per sample divided by the number n (sample size) of
units inspected. Thus, an estimator of the average number of defects Ui = Xi /n over m samples is given by

∑m
i = 1 Ui
U=
m

Unlike the c-chart, the u-chart does not require a constant number of units, and it can be used, for example, when the
samples are of different sizes.

Control chart for defects per unit (u-chart):

The upper control limit, center line, and lower control limit of the u-chart are given by
r
ū
UCL = ū + 3
n
CL = ū (5.7.4)
r
ū
LCL = ū − 3
n

Example 5.23 Using the data of Table 5.14, construct and plot the u-chart.

124
Solution: The upper control limit, center line, and lower control limit of the u-chart are given by
r r
ū 0.3775
UCL = ū + 3 = 0.3775 + 3 = 0.7897
n 20
CL = ū = 0.3775
r r
ū 0.3775
LCL = ū − 3 = 0.3775 − 3 = −0.0347
n 20
Since a negative number of defects per unit cannot be observed, the LCL value is set to 0. The u-chart, shown in
Figure 5.26, indicate that the process is in control.

1 X = [1 1 14 10 8 3 9 10 2 5 6 12 3 4 5 6 8 11 8 7 9 ] ; % Number o f d e f e c t s
2 n = 2 0 ; %s i z e o f i n s p e c te d u n i t s ( every 20 hours ) : sample s i z e
3 [ s t a t s , p l o t d a t a ]= c o n t r o l c h a r t ( X , ' c h a r t ' , ' u ' , ' u n i t ' , n ) ; %p l o t u− c h a r t
4 f p r i n t f ( ' C o n t r o l l i m i t s f o r u−c h a r t : UCL=%g , CL=%g , LCL=%g\n ' , . . .
5 plotdata.ucl (1) , plotdata.cl (1) , p lotdata .lcl (1) ) ;

0.8 UCL

0.7

0.6
Defects per Unit

0.5

0.4
CL

0.3

0.2

0.1

0 LCL
2 4 6 8 10 12 14 16 18 20
Sample Number

F IGURE 5.26: u-chart for woolen fabric data.

5.8 A CCEPTANCE S AMPLING

Acceptance sampling is a methodology commonly used in quality control and improvement to determine whether to
accept or reject a particular lot or batch of products before shipped to customers. This is done by divising a sampling
plan that sets the product acceptability criteria. A 100% inspection does not guarantee 100% compliance and is too time
consuming and costly. Rather than evaluating all items, a specified sample is taken, inspected or tested, and a decision
is made about accepting or rejecting the entire production lot. There are two major classifications of acceptance plans:
by attributes and by variables. When the decision to accept or reject a lot based on classification of the items as either
defective (conforming) or nondefective (nonconforming), the sampling plan is called inspection by attributes. The lot
is accepted if no more than an allowable number of defective items are found. The attribute case is the most common
for acceptance sampling, and will be assumed for the rest of this section. A sampling plan based on one sample is
known as a single sampling plan, while sampling plans based on two or more successively drawn samples are known
as double or multiple sampling plans. Selection of an acceptance sampling plan will depend on the nature of the
inspection test and the data produced.
An acceptance sampling plan works in the following way. A fixed number n of items is sampled from each lot of size
N, carefully inspected, and each item is judged to be either defective or nondefective. If the number d of defectives in

125
the sample is less than or equal to prespecified acceptance number c, the lot is accepted. Otherwise, the lot is rejected.
Thus, to design a sampling plan we need to know how many items n to sample and how many defectives items c in
that sample are enough to convince us that the lot is unacceptable.
In quality control, there is some relationship between the lot size N and the sample size n because the probability
distribution for the number d of defectives in a sample of n items from a lot will depend on the lot size N. For example,
a good sampling plan will provide for effective decisions with a sample of 10% or less of the lot size. If N is large and
n is small relative to N, then the probability distribution the number d of defectives follows a binomial distribution
 
n k
P(d = k) = p (1 − p ) n − k , k = 0, 1, . . . , n
k

where p denotes the lot fraction defective (true proportion nonconforming).

Thus, for a sampling plan with sample size n and acceptance number c, the probability of accepting a lot with lot
fraction defective p is given by

c c  
n k
Pa ( p) = P(Accept lot) = P(d ≤ c) = ∑ P(d = k) = ∑ p (1 − p ) n − k (5.8.1)
k =0 k =0
k

5.8.1 S AMPLING P LAN C RITERIA

In order to design a sampling plan, four values must be known and are typically determined from past experience,
engineering estimate, and/or management decisions:

1. Acceptable Quality Level (AQL). This is the maximum fraction defective, p1 , considered acceptable off the pro-
ducer’s line, and it is generally in the order of 1-2%. That is, Pa ( p1 ) = Pa ( AQL) should be large, typically near
0.95.

2. Producer’s Risk, α. This is the probability of rejecting a lot that is within acceptable quality level (AQL). That is,
the producer’s risk is the probability of rejecting H0 : p = p1 when H0 is true. Thus, α = 1 − Pa ( AQL) is the
probability of a Type I error. That is, the producer’s risk is the probability that a lot containing an acceptable
quality level is rejected.

3. Lot Tolerance Percent Defective (LTPD. This is the largest lot fraction defective, p2 , that can be tolerated by a con-
sumer. The LTPD has a low probability of acceptance.

4. Consumer’s Risk, β. This is the probability that a “bad” lot containing a greater number of defects than the LTPD
limit will be accepted. Thus, β = Pa ( LTPD ) is the probability of making a Type II error.

Example 5.24 A manufacturer of USB drives ships a particular USB in lots of 500 each. The acceptance sampling plan used prior
to shipment is based on a sample size n = 10 and acceptance number c = 1.

1. Find the producer’s risk if AQL = 0.05

2. Find the consumer’s risk if LTPD = 0.2

Solution: The lot size N = 500 is much larger than the sample size n = 10. Thus, the probability of lot acceptance is
given by

1      
10 k 10 10
Pa ( p) = ∑ p (1 − p)10−k = (1 − p)10 + p(1 − p)9 = (1 − p)10 + 10p(1 − p)9
k =0
k 0 1

1. For p = 0.05, we have Pa (0.05) = 0.914. Thus, the producer’s risk is α = 1 − Pa ( AQL) = 1 − Pa (0.05) =
1 − 0.914 = 0.086. That is, the producer will reject 8.6% of the lots, even if the lot fraction defective is as small as
0.05.

2. The consumer’s risk is β = Pa ( LTPD ) = Pa (0.2) = 0.376.

126
 1 c =1; % acceptance number
2 n =10; % sample s i z e
3 AQL = 0 .0 5 ; % a c c e p ta b l e q u a l i t y l e v e l
4 alpha = 1− c d f ( ' b i n o ' , c , n , AQL) ; % producer ' s r i s k
5 f p r i n t f ( ' Producer Risk = %. 3 f \n ' , alpha )
6 LTPD = 0 . 2 ; % l o t t o l e r a n c e p e r c e n t d e f e c t i v e
7 beta = c d f ( ' b i n o ' , c , n , LTPD) ; % consumer ' s r i s k
8 f p r i n t f ( ' Consumer Risk = %. 3 f \n ' , beta )

5.8.2 O PERATING C HARACTERISTIC (OC) C URVES

The operating characteristic (OC) curve is an excellent graphical tool for evaluating quality sampling plans. The OC
curve depicts the plot of the probability of lot acceptance Pa versus lot fraction defective p. The OC curve for Ex-
ample 5.24 is shown in Figure 5.27. Note that as the lot fraction defective increases, the probability of lot acceptance
decreases until it reaches 0.

1 c =1; % acceptance number

2 n =10; % sample s i z e
3 p =0:0 .0 1 : 1 ; % l o t f r a c t i o n d e f e c t i v e
4 Pa= c d f ( ' b i n o ' , c , n , p ) ; % p r o b a b i l i t y o f l o t acceptance
5 p l o t ( p , Pa ) ; % p l o t OC curve

1
Producer Risk
0.9

0.8

0.7

0.6
Pa

0.5

0.4

0.3

0.2
Consumer Risk

0.1

0 p
0 0.1 0.2 0.3 0.4 0.5
AQL LTPD

F IGURE 5.27: OC curve for Example 5.24.

In general, the steeper the OC curve the better the protection for both consumer and producer. In fact, the ideal OC
curve would be a parallel line to the y-axis at the AQL value as shown in Figure 5.28.
When sample sizes are increased, the OC curve becomes steeper as shown in Figure 5.29(b). In the same vein, when
the acceptance number is decreased, the curve gets steeper. Moreover, changing the acceptance number does not
significantly change the OC curve as shown in Figure 5.29(c).

5.8.3 AVERAGE O UTGOING Q UALITY

The average outgoing quality (AOQ) is the expected average quality of outgoing products for a given value of incoming
product quality. The AOQ curve can be used to evaluate a sampling plan by showing the average quality accepted by

127
 1

0.9

0.8

0.7

0.6

Pa
0.5

0.4

0.3

0.2

0.1

0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
p

F IGURE 5.28: Ideal OC curve (100% inspection).

1 1
n = 25, c = 1 n = 10, c = 0
0.9 0.9
n = 50, c = 2 n = 10, c = 1
0.8 n = 100, c = 4 0.8 n = 10, c = 2
0.7 0.7

0.6 0.6
Pa

Pa

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1

0 0
0 0.05 0.1 0.15 0.2 0.25 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
p p
(a) (b)
F IGURE 5.29: (a) OC curves for different sample sizes; (b) OC curves for the same
sample sizes.

the consumer for a given fraction defective.

 n
AOQ = 1 − p Pa (5.8.2)
N
where N is the lot size and n is the sample size. Note that when N ≫ n, then AOQ ≈ p Pa . The AOQ curve for
Example 5.24 is shown in Figure 5.30.

1 c =1; % acceptance number

2 n =10; % sample s i z e
3 N=500; % l o t s i z e
4 p =0:0 .0 1 : 1 ; % l o t f r a c t i o n d e f e c t i v e
5 Pa= c d f ( ' b i n o ' , c , n , p ) ; % p r o b a b i l i t y o f l o t acceptance
6 AOQ = (1 − n /N) * p . * Pa ; % average o u tg o i n g q u a l i t y
7 p l o t ( p ,AOQ) ; % p l o t AOQ curve

128
 0.09
Average Outgoing Quality (AOQ)
0.08

0.07

0.06

0.05

AOQ
0.04

0.03

0.02

0.01

0
0 0.2 0.4 0.6 0.8 1
p

F IGURE 5.30: AOQ curve for Example 5.24.

The AOQ curve initially increases as more defectives are produced, more are released. As more and more lots
are rejected, 100% inspections become more common and the AOQ curve starts to decrease as a result. The average
outgoing quality limit (AOQL) is simply the maximum value on the AOQ curve, and represents the maximum possible
fraction defective for the sampling plan.

5.8.4 AVERAGE T OTAL I NSPECTION

In practice, a rejected lot usually means that the lot is to follow a particular routine, to be screened, repaired, corrected,
or even rejected, or perhaps accepted after argument or waiver of specifications. In screening, the rest of the lot is 100%
inspection. Defective items are supposedly removed, and what is left of the lot is supposedly perfect. But the question
that rises is: What is the total amount of inspection when rejected lots are screened? If all lots contain zero defectives,
no lot will be rejected. If all items are defective, all lots will be inspected, and the amount to be inspected is N. Finally,
if the lot quality is 0 < p < 1, the average amount of inspection per lot will vary between the sample size n, and the lot
size N.
The average total inspection (ATI) is the average total number of items inspected per lot of size N. Let the quality of
the lot be p and the probability of lot acceptance be Pa , then the ATI per lot is

ATI = n + (1 − Pa )( N − n) (5.8.3)

The ATI curve for Example 5.24 is shown in Figure 5.31.

1 c =1; % acceptance number

2 n =10; % sample s i z e
3 N=500; % l o t s i z e
4 p =0:0 .0 1 : 1 ; % l o t f r a c t i o n d e f e c t i v e
5 Pa= c d f ( ' b i n o ' , c , n , p ) ; % p r o b a b i l i t y o f l o t acceptance
6 ATI = n +(N−n ) * (1 − Pa ) ; % average t o t a l i n s p e c t i o n
7 p l o t ( p , ATI ) ; % p l o t ATI curve

5.9 M ULTIVARIATE C ONTROL C HARTS

Many manufacturing and service businesses use univariate statistical control charts to monitor the performance of
their processes [1]. However, in most processes, there are more than one measurement process to monitor [2], and it is
increasingly difficult to determine the root cause of defects if multiple process variables exhibit faults or process devia-
tions at the same moment in time. Moreover, most processes are highly correlated, particularly for assembly operations
and chemical processes [2, 3]. Univariate control charts not only lead to frequent adjustments of the process but also

129
 500

450

400

350

300

AT I
250

200

150

100

50

0
0 0.2 0.4 0.6 0.8 1
p

F IGURE 5.31: AOQ curve for Example 5.24.

do not account for the correlation information between the measurement processes [3]. Multivariate quality control
methods overcome these limitations by monitoring the interactions of several process variables simultaneously and
also by determining hidden factors using dimensionality-reduction techniques [2]. The use of multivariate statistical
process control is also facilitated by the proliferation of sensor data that is typically complex, high-dimensional and
generally correlated. Multivariate charts are used to detect shifts in the mean or the relationship (covariance) between
several related parameters.
In recent years, several multivariate statistical process control techniques have been proposed to analyze and monitor
multivariate data [5, 4, 6]. With multivariate quality control charts, it is possible to have well-defined control limits,
while taking in consideration the cross-correlation between the variables. In addition, these multivariate charts may be
used to analyze the stability of the processes without the complication of simultaneously monitoring several univariate
control charts [2].
Mapping a multivariate situation as a univariate may lead to results where processes might seem to be in control
when in fact they are not and vice-versa, as illustrated in Fig. 5.32 which depicts the result of modelling two highly-
correlated variables as independent. The ellipse defines a region where the process is operating under normal operating
conditions. Any observation falling outside the ellipse is identified as a fault. If the variables were, however, modeled
as independent, then the control region would be defined between the rectangle. As can be seen in Fig. 5.32, some out-
of-control observations would be misidentified, indicating that the correlation structure between the variables should
be taken into account in order to accurately characterize the behavior of multivariate industrial environments [2].

5.9.1 χ2 AND H OTELLING T 2 C ONTROL C HART

In many industrial applications, the output of a process is characterized by p variables that are measured simulta-
neously. Independent variables can be charted individually, but if the variables are correlated, a multivariate chart
is needed to determine whether the process is in control. Generally, the univariate process variables make a ran-
dom vector, x = ( X1 , X2 , . . . , X p )′ . That is, the process has p quality characteristics X1 , X2 , . . . , X p that we are inter-
ested to monitor. Suppose the random vector x follows a multivariate normal distribution N (µ, Σ) with mean vector
µ = (µ1 , µ2 , . . . , µ p )′ and a covariance matrix Σ. The statistic given by

χ2 = n( x̄ − µ)′ Σ−1 ( x̄ − µ) (5.9.1)

follows a χ2 ( p) distribution with p degrees of freedom, where x̄ is the sample mean for each of the p quality character-
istics from a sample of size n, µ is the vector of in-control means for each quality characteristic, and Σ−1 is the inverse
of the covariance matrix. The upper control limit of the χ2 control chart is given by UCL = χ2α,p , where α is a given
significance level.
Since µ and Σ are unknown in practice, we usually estimate them on the basis of preliminary samples (subgroups),
taken when the process is thought to be in control. Suppose m preliminary samples are available, each of size n. The
Hotelling’s T 2 control chart is the most common monitoring technique for multivariate data, and it can be thought of

130
 x2

Normal range for x2

Normal range for x1 x1

out−of−control observations
in−control observations

F IGURE 5.32: Motivation behind using multivariate control charts.

as the multivariate counterpart of the Shewhart x̄-chart. The T 2 statistic is given by

T 2 = n( x̄ − µ̂)′ Σ
b −1 ( x̄ − x̄¯ ) (5.9.2)

b is the estimated covariance

where x̄ is the vector of sample means, µ̂ is the estimated vector of in-control means, and Σ
matrix for the quality characteristics when the process is in control.
b consider the vector of sample means given by
To find µ̂ and Σ,
 
x̄1k
 x̄2k 
 
x̄k =  ..  k = 1, 2, . . . , m (5.9.3)
 . 
x̄ pk

where (
n
1 j = 1, 2, . . . , p
x̄ jk =
n ∑ xijk k = 1, 2, . . . , m
(5.9.4)
i =1

is the sample mean of the jth quality characteristic for the kth sample, and xijk is the ith observation on the jth quality
characteristic in the kth sample.
The sample variances for the jth quality characteristic in the kth sample are given by
(
n  2
1 j = 1, 2, . . . , p
s2jk =
n − 1 i∑
xijk − x̄ jk (5.9.5)
=1 k = 1, 2, . . . , m

The sample covariance between the jth and hth quality characteristics in the kth sample is given by
(
n  
2 1 k = 1, 2, . . . , m
s jhk = ∑
n − 1 i =1
xijk − x̄ jk ( xihk − x̄hk ) ,
j 6= h
(5.9.6)

The target mean of each quality characteristic for m samples is given by

m
1
x̄¯ j =
m ∑ x̄ jk , j = 1, 2, . . . , p (5.9.7)
k =1

131
and the averaged sample variance and covariance are given by
m
1
s̄2j = ∑ s2jk j = 1, 2, . . . , p (5.9.8)
m k =1

and
m
1
s̄ jh =
m ∑ s jhk j 6= h (5.9.9)
k =1
b can be estimated by the mean vector x̄¯ and the average covariance matrix C, respectively, as follows:
Thus, µ̂ and Σ
   
x̄¯1 s̄21 s̄12 · · · s̄1p
 x̄¯2   s̄12 s̄22 · · · s̄2p 
   
x̄¯ =  ..  and C= . .. .. ..  (5.9.10)
 .   .. . . . 
x̄¯ p s̄1p s̄2p · · · s̄2p

The Hotelling’s T 2 statistic for the ith subgroup is given by

Tk2 = n( x̄k − x̄¯ )′ C −1 ( x̄k − x̄¯ ), k = 1, 2, . . . , m (5.9.11)

• For phase I, the upper control limit is

p(m − 1)(n − 1)
UCL = Fα,p,mn−m− p+1 (5.9.12)
mn − m − p + 1

• For phase II, the upper control limit is

p(m + 1)(n − 1)
UCL = Fα,p,mn−m− p+1 (5.9.13)
mn − m − p + 1

Example 5.25 Table 5.16 shows bivariate data for two quality characteristics X1 and X2 for 25 samples, each of size 5. Plot the
Hotelling’s T 2 control chart. Assume α = 0.05.
Solution: We have m = 25, n = 5, and p = 2. The summary statistics are shown in Table 5.17.
For phase I, the upper control limit is

p(m − 1)(n − 1)
UCL = Fα,p,mn−m− p+1 = 1.9394 F0.05,2,99 = (1.9394)(3.0882) = 5.9893
mn − m − p + 1

where F0.05,2,99 can be computed using MATLAB as follows: ≫ icdf(’F’,1-0.05,2,99). The Hotelling’s T 2 control chart is
shown in Figure 5.33, where the samples 5 and 18 appears to be out-of-control. However, all the samples in the X-charts
for each quality characteristic appear to be in-control as shown in Figure 5.34. Thus, the process is out-of-control.

5.10 P RINCIPAL C OMPONENTS A NALYSIS

Principal components analysis (PCA) is an explanatory technique to learn about data sets. The objective of PCA to
reduce the dimensionality of the data set while retaining as much as possible the variation in the data set. Principal
components (PCs) are linear transformations of the original set of variables, and are uncorrelated and ordered so
that the first few components carry most of the variation in the original data set. The first PC has the geometric
interpretation that it is a new coordinate axis that maximizes the variation of the projections of the data points on the
new coordinate axis. The general idea of PCA is as follows: if we have a set of moderately or strongly correlated
variables (i.e. the variables share much common information), it may then be possible to construct new variables that
are combinations of these variables that account for much of the original information contained in the data. The output

132
 Sample Quality Characteristic
Number k X1 X2
1 65 69 79 66 62 33 41 36 36 42
2 64 71 72 73 72 35 37 37 37 39
3 78 75 73 59 60 40 34 39 32 38
4 63 66 74 69 76 34 34 39 34 38
5 87 81 71 62 67 37 33 39 36 38
6 76 72 80 67 75 34 38 33 36 38
7 64 63 60 75 61 34 38 32 34 37
8 66 65 68 85 75 34 35 41 33 42
9 73 81 78 65 67 39 39 39 39 36
10 75 64 69 73 68 37 34 38 37 34
11 78 61 74 65 67 38 41 42 41 36
12 72 78 56 70 74 35 31 43 39 41
13 61 72 72 91 83 37 36 35 34 39
14 70 87 78 70 76 40 39 37 30 41
15 57 67 51 64 69 33 37 38 37 36
16 74 74 67 77 63 35 37 37 36 37
17 74 69 59 76 70 40 35 38 39 32
18 70 72 61 57 61 42 43 38 40 40
19 62 56 74 64 68 34 36 40 36 37
20 79 71 78 65 77 38 39 29 37 37
21 77 75 67 67 68 33 41 37 35 40
22 63 69 65 70 79 39 36 36 41 36
23 58 66 83 65 65 37 41 36 37 38
24 77 69 68 72 73 34 38 37 35 35
25 74 52 72 73 72 39 33 39 36 35
TABLE 5.16: Bivariate data for two quality characteristics.

12
18

10

8
Hotelling T2

15

6 UCL

0
0 5 10 15 20 25
Sample Number

F IGURE 5.33: T 2 control chart for bivariate data.

of PCA consists of coefficients that define the linear combinations used to obtain the new variables (PC loadings) and
the new variables (PCs) themselves. Examining the PC loadings and plotting the PCs can aid in data interpretation,
particularly with higher dimensional data.

Let x1 , x2 , . . . , xn be n observation vectors on a random vector x = ( X1 , X2 , . . . , X p )′ with p quality characteristics,

133
 Sample Summary Statistics
Number k x̄1k x̄2k s21k s22k s12k Tk2
1 68.20 37.60 42.70 14.30 -5.90 0.55
2 70.40 37.00 13.30 2.00 4.00 0.02
3 69.00 36.60 78.50 11.80 14.50 0.17
4 69.60 35.80 29.30 6.20 11.90 0.88
5 73.60 36.60 104.80 5.30 -7.45 1.43
6 74.00 35.80 23.50 5.20 -6.00 2.54
7 64.60 35.00 36.30 6.00 -2.00 5.24
8 71.80 37.00 69.70 17.50 -5.25 0.35
9 72.80 38.40 47.20 1.80 4.35 2.12
10 69.80 36.00 18.70 3.50 5.50 0.60
11 69.00 39.60 47.50 6.30 -3.00 4.52
12 70.00 37.80 70.00 23.20 -30.00 0.44
13 75.80 36.20 132.70 3.70 -4.95 3.81
14 76.20 37.40 49.20 19.30 11.40 4.02
15 61.60 36.20 55.80 3.70 0.10 7.19
16 71.00 36.40 33.50 0.80 -3.00 0.32
17 69.60 36.80 43.30 10.70 3.65 0.03
18 64.20 40.60 41.70 3.80 10.35 11.71
19 64.80 36.60 45.20 4.80 11.40 2.66
20 74.00 36.00 35.00 16.00 -8.25 2.26
21 70.80 37.20 23.20 11.20 -2.20 0.11
22 69.20 37.60 38.20 5.30 -3.65 0.31
23 67.40 37.80 86.30 3.70 -5.90 1.07
24 71.80 35.80 12.70 2.70 -5.30 1.23
25 68.60 36.40 86.80 6.80 18.20 0.37
TABLE 5.17: Summary Statistics for the bivariate data.

where

 
xi1
 xi2 
 
xi =  .. 
 . 
xip

is a p-dimensional column vector, which represents the vector of quality characteristic means. All n observation vectors
x1 , x2 , . . . , xn on p process variables can be transposed to row vectors and placed in a matrix X, of dimension n × p,
called data matrix or data set, as follows:

   
x1′ x11 x12 ··· x1j ··· x1p

 x2′ 

 x21
 x22 ··· x2j ··· x2p 
 ..   .. .. .. .. 
 .   . . . . 
X=


 =
 xi1
 (5.10.1)
 x′i   xi2 ··· xij ··· xip 
 ..   . .. .. .. 
 .   .. . . . 
x′n xn1 xn2 ··· xnj · · · xnp

An individual column of X corresponds to that data collected on a particular variable, while an individual row (obser-
vation) refers to the data collected on a particular individual or object for all variables. The value of the jth variable for
the ith observation x′i = ( xi1, . . . , x ij , . . . , x ip ) is xij , which is the element of the ith row and jth column of X.

134
 80 UCL 41
UCL
78
40
76
39
74
Sample Mean

Sample Mean
72 38

70 CL 37 CL

68
36
66
35
64

62 34

60 LCL LCL
33
5 10 15 20 25 5 10 15 20 25
Sample Number Sample Number
(a) (b)
F IGURE 5.34: X-chart for each quality characteristic of the bivariate data: (a) X-chart
for X1 . (b) X-chart for X2 .

• The sample mean vector x̄ is a p-dimensional vector given by

 
x̄1
 x̄2 
 
n  .. 
1 1  . 
x̄ = ∑ xi = X ′ 1 = 


 (5.10.2)
n i =1 n  x̄ j 
 .. 
 . 
x̄ p

where 1 = (1, 1, . . . , 1)′ is a n-dimensional column vector of all 1s, X ′ denotes the transpose of X, and x̄ j is the
mean of the jth variable. That is, x̄ j is the mean of the jth column of the data matrix X.
• The sample covariance matrix S = (sij ) is a p × p symmetric matrix given by
 
s11 s12 ··· s1i ··· s1p
 s12 s22 ··· s2i ··· s2p 
 
 .. .. .. .. 
 . . . . 
S=
 s1i
 (5.10.3)
 s2i ··· sii ··· sip 
 . .. .. .. 
 .. . . . 
s1p s2p ··· sip · · · s pp

where sii = s2i is the sample variance of the ith variable

n
1
sii = s2i = ∑ ( x − x̄i )2 (5.10.4)
n − 1 k=1 ki

and sij is the sample covariance of the ith and jth variables
n
1
sij = ∑ ( x − x̄i )( xkj − x̄ j ).
n − 1 k=1 ki
(5.10.5)

The sample covariance matrix S = (sij ) can be written as

1
S= X ′ HX (5.10.6)
n−1

135
 where H = I − J/n is a called centering matrix, and J = 11′ is an n × n matrix of all 1s. Note that a centered data
matrix Y is obtained by multiplying the centering matrix H with the data matrix X, that is Y = HX.
• The sample correlation matrix R = (rij ) is a symmetric p × p matrix given by
 
1 r12 · · · r1p
 r12 1 · · · r2p 
 
R= . .. . ..  (5.10.7)
 .. . . . . 
r1p r2p ··· 1

where rij is the sample correlation between ith and jth variables
sij sij
rij = √ = . (5.10.8)
sii s jj si s j

Note that if D = diag(s1 , s2 , . . . , s p ) is a p × p diagonal matrix of sample standard deviations, then the covariance
and correlation matrices can be written as S = DRD and R = D −1 SD −1 , where D −1 = diag(1/s1 , 1/s2 , . . . , 1/s p )
is the inverse matrix of D.

5.10.1 PCA A LGORITHM

Given a data matrix X, the PCA algorithm consists of four main steps:
Step 1: Compute the centered data matrix Y = HX by subtracting off-column means.
Step 2: Compute the p × p covariance matrix S of the centered data matrix as follows
1
S= Y′Y (5.10.9)
n−1

Step 3: Compute the eigenvectors and eigenvalues of S using eigen-decomposition

p
S = AΛA′ = ∑ λ j a j a′j (5.10.10)
j =1

where
• A = ( a1 , a2 , . . . , a p ) is a p × p orthogonal matrix (A′ A = I) whose columns a j = ( a j1 , a j2 , . . . , a jp )′ are the
eigenvectors of S such that a′j a j = 1, j = 1, . . . , p. The eigenvectors tell us a direction or linear combination
of existing data. The first eigenvector a1 defines a variable with the most variation in the data matrix. The
eigenvectors are the principal component (PC) coefficients, also known as loadings. That is, each eigenvector
a j contains coefficients for the jth PC. These eigenvectors are in order of decreasing component variance.
• Λ = diag(λ1 , λ2 , . . . , λ p ) is a p × p diagonal matrix whose elements are the eigenvalues of S arranged in
decreasing order, i.e. λ1 ≥ λ2 ≥ · · · ≥ λ p . The eigenvalues tell us about the amount of variation in the data
matrix, that is the variance in a particular direction. The eigenvalues measure the importance of the PCs.
Step 4: Compute the transformed data matrix Z = YA of size n × p
 
z11 z12 ··· z1j ··· z1p
 z21 z22 ··· z2j ··· z2p 
 
 .. .. .. .. 
′ ′ ′ ′
 . . . . 

Z = ( z1 , z2 , . . . , z i , . . . , z p ) =   (5.10.11)
z
 i1 z i2 ··· zij ··· zip 
 . .. .. .. 
 .. . . . 
zn1 zn2 ··· znj · · · znp
which contains the coordinates of the original data in the new coordinate system defined by the PCs. The rows of
Z correspond to observations zi = A′ ( xi − x̄), while its columns correspond to PC scores. The eigenvalues are the
variances of the columns of the PCs. Thus, the first PC score accounts for as much of the variability in the data as
possible, and each succeeding component score accounts for as much of the remaining variability as possible.

136
To apply PCA to the data matrix X, we use the MATLAB function pca as follows:
>> [A,Z,lambda,Tsquare] = pca(X)
where A is the eigenvector matrix, Z is the transformed data matrix, lambda is a p-dimensional vector of eigenvalues
i.e. Λ = diag(lambda) = diag(λ1 , λ2 , . . . , λ p ), and Tsquare is Hotelling’s T 2 , a statistical measure of the multivariate
distance of each observation from the center of the data set. Tsquare = ( T12 , T22 , . . . , Tp2 ) is p-dimensional vector whose
values are given by
Ti2 = ( xi − x̄)′ S−1 ( xi − x̄) (5.10.12)
Using the fact that S−1 = AΛ−1 A′ and zi = A′ ( x i − x̄), it follows that

Ti2 = ( xi − x̄)′ AΛ−1 A′ ( xi − x̄) = z′i Λ−1 zi (5.10.13)

Standardizing the data is often preferable when the variables are in different units or when the variance of the
different columns is substantial. In this case, we use pca(zscore(X)) instead of pca(X). That is, PCA is performed on the
correlation matrix instead of the covariance.

Hotelling Control Chart:

The Hotelling control chart is a multivariate extension of the X-chart that does take the correlation into account. The
plotted points on this chart are given by the Hotelling’s statistic Ti2 for individual observations:

Ti2 = ( xi − x̄)′ S−1 ( xi − x̄)

• For phase I, the upper control limit is

( n − 1)2
UCL = Bα,p/2,( n− p−1) /2 (5.10.14)
n
where B(·) the inverse cdf of the Beta distribution and α is the significance level (typically set to 0.05 or 0.01).
• For phase II, the upper control limit is

p(n + 1)(n − 1)
UCL = Fα,p,n− p (5.10.15)
n( n − p )

Example 5.26 The phosphorus content data set contains 18 observations, where each observation has 3 variables:
• X1: Inorganic phosphorus
• X2: Organic phosphorus
• X3: Plant phosphorus
This data set studies the effect of organic and inorganic phosphorus in the soil in comparison with the phosphorus content of the
corn grown. Start by loading the data in phosphorus.mat:
>> load phosphorus.mat;
>> whos
Name Size Bytes Class Attributes
Description 7x72 1008 char
X 18x3 432 double
observations 18x2 72 char
variables 3x2 12 char

1. Display the box plot for the phosphorus data matrix.

137
 2. Compute the mean vector x, covariance S, and correlation R.
3. Plot the second PC score vs. the first PC score.
4. Plot the Hotelling’s T 2 control chart.
Solution:

1 l o a d phosphorus.mat ;
2 [ n , p ]= s i z e ( X ) ; %s i z e o f data m a t r i x
3 xbar = mean( X ) ; %mean v e c t o r
4 S = cov (X ) ; %c o v a r i a n c e m a t r i x
5 R = c o r r (X ) ; %c o r r e l a t i o n m a t r i x
6 b o x p l o t (X ) ;
7 [ A , Z , lambda , Tsquare ]= pca ( X ) ; %perform PCA on data m a t r i x u s i n g c o v a r i a n c e
8 % PC2 score v s . PC1 score
9 s c a t t e r ( Z ( : , 1 ) ,Z ( : , 2 ) , 3 , ' o ' , ' MarkerFaceColor ' , [ .4 9 1 .6 3 ] , ' LineWidth ' , 1 ) ;

1. The side-by-side box plots for the data is shown in Figure 5.35. We can see that the third column of the data (i.e.
plant phosphorus) contains an outlier.

160

140

120

100

80

60

40

20

0
X1 X2 X3

F IGURE 5.35: Boxplot for the phosphorus content data.

2. The mean vector x̄, covariance S, and correlation R are

     
11.94 103.12 63.86 190.09 1.00 0.46 0.69
x̄ =  42.11  S =  63.86 185.63 130.38  R =  0.46 1.00 0.35 
81.28 190.09 130.38 728.80 0.69 0.35 1.00
Note that R can also be obtained from S using the MATLAB command corrcov as follows:

>> [R,sigma]=corrcov(S);

where sigma is the vector of sample standard deviations.

3. The plot the second PC score vs. the first PC score is shown in Figure 5.36. The labels displayed in Figure 5.36(b)
represent the observation numbers. Notice that the sample number 17 is an outlier.
4. The plot of the Hotelling’s T 2 control chart is displayed in Figure 5.37, which shows that the sample numbers 6
and 17 appear to be out-of-control.

1 %P l o t H o t e l l i n g ' s Tˆ 2 c o n t r o l c h a r t
2 alpha =0 .0 5 ;
3 UCL = ( ( n − 1) ˆ 2 / n ) * i c d f ( ' beta ' ,1 − alpha , p / 2 , ( n−p − 1) / 2 ) ;
4 p l o t ( Tsquare , ' bo− ' , ' MarkerFaceColor ' , [ .4 9 1 .6 3 ] , ' MarkerSize ' , 2 ) ;

138
 25 25 10
6
20 20

15 15
1
13 15
10 10
18
5 5
PC2 score

PC2 score
7 12
0 0 16
14
4 11
−5 −5 17

−10 −10 5
2 8
−15 −15 9

−20 −20 3

−25 −25
−40 −20 0 20 40 60 80 100 −40 −20 0 20 40 60 80 100
PC1 score PC1 score
(a) unlabeled plot (b) labeled plot
F IGURE 5.36: PC2 vs. PC1 score for the phosphorus content data.

12
17

10

8
Hotelling T2

6
UCL
6

0
0 2 4 6 8 10 12 14 16 18
Sample Number

F IGURE 5.37: Hotelling’s T 2 control chart for the phosphorus content data.

5.10.2 PCA T HEORY

Let x = ( X1 , X2 , . . . , X p )′ denote the original variables measured on the objects or individuals. In principal component
analysis, the p original variables are transformed into linear combinations of uncorrelated variables (PCs) Z1 , Z2 , . . . , Z p
such that the jth PC Zj is given by Zj = a ′j x, where a j = ( a1j , . . . , a pj )′ is the jth eigenvector of S. That is,

Zj = a1j X1 + a2j X2 + · · · + a pj X p (5.10.16)

Since Sa j = λ j a j , it follows that var ( Zj ) = λ j . Thus, the first PC Z1 has the largest variance, while the last PC Z p has the
smallest. Moreover, the PCs are uncorrelated, that is corr ( Zj , Zk ) = 0 for j 6= k, which results from the orthogonality of
the eigenvectors a′j ak = 0.
If x is standardized to have zero mean and unit variance for each Xi , then the PCs Zj are correlated with the original
variables Xi such that
q
corr ( Zj , Xi ) = aij λ j (5.10.17)

Denote by z = ( Z1 , Z2 , . . . , Z p )′ the p-dimensional vector of PCs. Then, the correlation matrix between the PCs and the

139
the original variables is given by
C = corr (z, x) = AΛ1/2 (5.10.18)
√ √ p
where A and Λ1/2
= diag( λ1 , λ2 , . . . , λ p ) are obtained by performing PCA on the correlation matrix, that is by
applying PCA on the standardized data matrix using pca(zscore(X)).

Example 5.27 Using the phosphorus content data

1. Determine the first and second PCs

2. Display the scatter plot of PC2 coefficients vs. PC1 coefficients, and label the points

3. Compute the correlation matrix between the PCs and the original variables

Solution

1. Using the columns of the eigenvector matrix

 
0.27 0.16 0.95
A =  0.22 0.95 −0.22  ,
0.94 −0.27 −0.22

the first PC is then given by

Z1 = 0.27 X1 + 0.22 X2 + 0.94 X3
and the second PC is given by
Z2 = 0.16 X1 + 0.95 X2 − 0.27 X3

2. The scatter plot of PC2 coefficients vs. PC1 coefficients is shown in Figure 5.38. This plot helps understand which
variables have a similar involvement within PCs. As can be seen in Figure 5.38, the variables X1 and X2 are
located on the left of the plot, while the variable X3 is located on the bottom right. This is consistent with the
values of the coefficients of PC1 and PC2.

1 s c a t t e r ( A ( : , 1 ) ,A ( : , 2 ) , 3 , ' o ' , ' MarkerFaceColor ' , [ .4 9 1 .6 3 ] , ' LineWidth ' , 1 ) ;

2 gname ( v a r i a b l e s ) ; %press th e E n te r o r Escape key t o s to p l a b e l i n g .

1
X2

0.8

0.6
PC2 coefficient

0.4

0.2 X1

−0.2
X3

−0.4
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
PC1 coefficient

F IGURE 5.38: Coefficients of PC2 vs. PC1 for the phosphorus content data.

3. The correlation matrix between the PCs and the original variables is given by

140
 1 [ Ac , Zc , lambdac , Tsquarec ]= pca ( zscore ( X ) ) ; %PCA on s t a n d a r d i z e d data
2 C = Ac * s q r t ( d i a g ( lambdac ) ) ; %component c o r r e l a t i o n m a t r i x

 
0.90 −0.19 0.40
C =  0.70 0.71 −0.09 
0.85 −0.39 −0.35
Note that all the three variables are highly correlated with the first PC. We can also see that the organic phospho-
rus (X2 ) is highly correlated with PC2.

5.10.3 S CREE P LOT

The percentage of variance accounted for by the jth PC is called explained variance, and it is given by

λj
ℓj = p × 100%, j = 1, . . . , p. (5.10.19)
∑ j =1 λ j

After computing the eigenvectors, we sort them by their corresponding eigenvalues and then we pick the d principal
components with the largest eigenvalues. The other components will be discarded. The natural question that arises is:
How do we select the value of d? Well, the proportion of variance retained by mapping down from p to d dimensions
can be found as the normalized sum of the d largest eigenvalues

d ∑dj=1 λ j
υ= ∑ ℓj = ∑ p × 100%. (5.10.20)
j =1 j =1 λ j

In many applications, d is chosen such that a relatively high percentage, say 70 − 95% of explained variance is retained.
The remaining variance is assumed to be due to noise. The number of principal components d (d << p) can also be
determined using the scree graph, which is the plot of the explained variance against the component number k. The
optimal number k is usually selected as the one where the kink (elbow) in the curve appears.

Example 5.28 Using the phosphorus content data

1. Compute the explained variance

2. Plot the explained variance vs. the number of PCs.

3. What would be the lowest-dimensional space to represent the phosphorus content data?

Solution

1 expvar =100 * lambda / sum ( lambda ) ;%p e r c e n t o f th e t o t a l v a r i a b i l i t y e x p l a i n e d by each PC.

2 p l o t ( expvar , ' ko− ' , ' MarkerFaceColor ' , [ .4 9 1 .6 3 ] , ' LineWidth ' , 1 ) ; %Scree p l o t
3 p a r e to ( expvar ) ; %Pareto p l o t

1. The explained variance by the three PCs are: ℓ1 = 80.04%, ℓ2 = 15.65%, and ℓ3 = 4.31%. Notice that PC1 and
PC2 combined account for 95.69% of the variance in the data.

2. The scree and pareto plots of the explained variance vs. the number of PCs are shown in Figure 5.39.

3. Based on the explained variance by both PC1 and PC2 and also from the scree and Pareto plots, it can be deduced
that the lowest-dimensional space to represent the phosphorus content data corresponds to d = 2.

141
 90

80 90 90%

80 80%
70
Explained Variance %

Explained Variance %
70 70%
60
60 60%
50
50 50%
40
40 40%
30
30 30%
20
20 20%

10 10 10%

0 0 0%
1 1.5 2 2.5 3 1 2
Number of Principal Components Number of Principal Components
(a) Scree plot (b) Pareto plot
F IGURE 5.39: Scree and Pareto plots for the phosphorus content data.

5.10.4 B IPLOT
The plot of the principal component coefficients is a graphical display of the variables, while the plot of the principal
component scores is a graphical display of the observations. The biplot was originally proposed by Gabriel (1971) as a
graphical tool that allows information on both observations and variables of a data matrix to be displayed graphically,
and hence the “bi” in the name. Observations are displayed as points while variables are displayed as vectors. The
biplot helps visualize both the principal component coefficients for each variable and the principal component scores
for each observation in a single plot. Each of the p variables is represented in the biplot by a vector, and the direction
and length of the vector indicates how each variable contributes to the two principal components in the biplot, as
shown in Figure 5.40(a). The axes in the biplot represent the principal components (columns of eigenvector matrix
A), and the observed variables (rows of A) are represented as vectors. A biplot allows us to visualize the magnitude
and sign of each variable’s contribution to the first two or three principal components, and how each observation is
represented in terms of those components. Each of the n observations is represented in the biplot by a point, and their
locations indicate the score of each observation for the two principal components in the plot. For example, points near
the left edge of this plot have the lowest scores for the first principal component.
A 2D biplot displays the first two PCs, i.e. PC2 vs. PC1, while a 3D biplot displays the first 3 PCs, i.e., PC1, PC2, and
PC3 as shown in Figure 5.40(b). It is usually difficult to visualize a 3D biplot on a 2D plane, but rotating 3D biplot can
be very useful when the first two PCs do not explain most of the variance in the data. The axes in the biplot represent
the PCs, and the observed variables are represented as vectors.

Example 5.29 Using the phosphorus content data

1. Display the 2D biplot of PC2 vs. PC1

2. Display the 3D biplot of PC1, PC2, and PC3

Solution

1 b i p l o t ( A ( : , 1 : 2 ) , ' Scores ' ,Z ( : , 1 : 2 ) , ' VarLabels ' , v a r i a b l e s ) ; %2D b i p l o t

2 b i p l o t ( A ( : , 1 : 3 ) , ' Scores ' ,Z ( : , 1 : 3 ) , ' VarLabels ' , v a r i a b l e s ) %3D b i p l o t

1. The 2D biplot of PC2 vs. PC1 is shown in Figure 5.40(a). The first principal component, represented in this biplot
by the horizontal axis, has positive coefficients for all 3 variables. That corresponds to the 3 vectors directed
into the right half of the plot. The second principal component, represented by the vertical axis, has 2 positive
coefficients for the variables X1 and X2 , and 1 negative coefficient for the variable X3 . That corresponds to vectors
directed into the top and bottom halves of the plot, respectively. This indicates that this component distinguishes
between observations that have high values for the first set of variables and low for the second, and observations

142
 that have the opposite. Each of the 18 observations (rows of scores) is represented in this plot by a point, and their
locations indicate the score of each observation for the two principal components in the plot. For example, points
near the left edge of this plot have the lowest scores for the first principal component. The angles of between
the vectors representing the variables and the PCs indicate the contribution of the variable to the PCs. A narrow
angle indicates that the variable plays a major role in the PC. For example, plant phosphorus (X3 ) is important in
the first PC, while organic phosphorus (X2 ) is important in the second PC.
2. The 3D biplot of PC1, PC2, and PC3 is shown in Figure 5.40(b).

1 X2

0.8

0.6 1 X1

0.4
0.5
0.2 X1 X2

PC3
PC2

0
0 X3

−0.2 −0.5
X3
−0.4
−1
−0.6 1
0.5 1
−0.8
0 0.5
0
−1 −0.5
−0.5
−1 −0.5 0 0.5 1 −1 −1
PC1 PC2 PC1
(a) 2D biplot (b) 3D biplot
F IGURE 5.40: 2D and 3D biplots for the phosphorus content data.

5.10.5 PCA C ONTROL C HART

q
Recall that the mean and variance of the jth principal component Zj are µ Zj = 0 and var ( Zj ) = λ j . That is, σZj = λj .

PCA Control chart for jth PC:

The upper control limit, center line, and lower control limit of the jth PC are given by
q
UCL = 3 λ j
CL = 0 (5.10.21)
q
LCL = −3 λj

The control chart for the first PC of the phosphorus content data is depicted in Figure 5.41, which shows that the
sample number 17 is out-of-control.

Example 5.30 The European Jobs data are the percentage employed in different industries in European countries during 1979.
The job categories are agriculture, mining, manufacturing, power supplies, construction, service industries, finance, social and
personal services, and transportation and communications. It is important to note that these data were collected during the Cold
War. The European Jobs data set contains 26 observations (countries), where each observation has 9 variables:
• X1 : agriculture (Agr)
• X2 : mining (Min)
• X3 : manufacturing (Man)

143
 100
17 UCL
80

60

40

20
Z1
0 CL

−20

−40

−60

−80
LCL
−100
0 2 4 6 8 10 12 14 16 18
Sample Number

F IGURE 5.41: PC1 control chart for the phosphorus content data.

• X4 : power supply industries (PS)

• X5 : construction (Con)
• X6 : service industries (SI)
• X7 : finance (Fin)
• X8 : social and personal services (SPS)
• X9 : transport and communications (TC)
To load the data set into the MATLAB workspace, type:
>> load europeanjobs.mat
>> whos
Name Size Bytes Class Attributes
X 26x9 1872 double
countries 26x14 728 char
description 15x96 2880 char
variables 9x3 54 char

1. Display the correlation matrix of the data.

2. Display the scatterplot matrix of the data.
3. Plot the second PC score vs. the first PC score.
4. Determine the first and second PCs
5. Display the scatter plot of PC2 coefficients vs. PC1 coefficients, and label the points
6. Compute the explained variance, and plot it against the number of PCs. What would be the lowest-dimensional space to
represent the phosphorus content data?
7. Display the 2D biplot of PC2 vs. PC1. Then, display the 3D biplot of PC1, PC2, and PC3
8. Plot the Hotelling and first PC control charts. Identify the out-of-control points

144
Solution:
MATLAB code
>> load europeanjobs.mat
>> whos
Name Size Bytes Class Attributes
X 26x9 1872 double
countries 26x1 1970 cell
description 15x96 2880 char
variables 1x9 588 cell
>> [n,p]=size(X); %size of data matrix
>> R = corrcoef(X); %correlation matrix
>> plotmatrix(X);
>> boxplot(X);
>> [A,Z,lambda,Tsquare]=pca(X); %perform PCA on data matrix using covariance
>> scatter(Z(:,1),Z(:,2),3,’o’,’MarkerFaceColor’,[.49 1 .63],’LineWidth’,1);

The data matrix X for the European jobs data is a 26 × 9 matrix given by

Agr Min Man PS Con SI Fin SPS TC

 
Belgium 3.3 0.9 27.6 0.9 8.2 19.1 6.2 26.6 7.2
Denmark  9.2 0.1 21.8 0.6 8.3 14.6 6.5 32.2 7.1 
 
France  10.8 0.8 27.5 0.9 8.9 16.8 6.0 22.6 5.7 
 
W. Germany  6.7 1.3 35.8 0.9 7.3 14.4 5.0 22.3 6.1 
 
Ireland  23.2 1.0 20.7 1.3 7.5 16.8 2.8 20.8 6.1 
 
Italy  15.9 0.6 27.6 0.5 10.0 18.1 1.6 20.1 5.7 
 
Luxembourg  7.7 3.1 30.8 0.8 9.2 18.5 4.6 19.2 6.2 
 
Netherlands  6.3 0.1 22.5 1.0 9.9 18.0 6.8 28.5 6.8 
 
United Kingdom 
 2.7 1.4 30.2 1.4 6.9 16.9 5.7 28.3 6.4 

Austria  12.7 1.1 30.2 1.4 9.0 16.8 4.9 16.8 7.0 
 
Finland  13.0 0.4 25.9 1.3 7.4 14.7 5.5 24.3 7.6 
 
Greece  41.4 0.6 17.6 0.6 8.1 11.5 2.4 11.0 6.7 
 
Norway  9.0 0.5 22.4 0.8 8.6 16.9 4.7 27.6 9.4 
X=  .
Portugal  27.8 0.3 24.5 0.6 8.4 13.3 2.7 16.7 5.7 
 
Spain  22.9 0.8 28.5 0.7 11.5 9.7 8.5 11.8 5.5 
 
Sweden  6.1 0.4 25.9 0.8 7.2 14.4 6.0 32.4 6.8 
 
Switzerland  7.7 0.2 37.8 0.8 9.5 17.5 5.3 15.4 5.7 
 
Turkey  66.8 0.7 7.9 0.1 2.8 5.2 1.1 11.9 3.2 
 
Bulgaria  23.6 1.9 32.3 0.6 7.9 8.0 0.7 18.2 6.7 
 
Czechoslovakia 
 16.5 2.9 35.5 1.2 8.7 9.2 0.9 17.9 7.0 

E. Germany  4.2 2.9 41.2 1.3 7.6 11.2 1.2 22.1 8.4 
 
Hungary  21.7 3.1 29.6 1.9 8.2 9.4 0.9 17.2 8.0 
 
Poland  31.1 2.5 25.7 0.9 8.4 7.5 0.9 16.1 6.9 
 
Rumania  34.7 2.1 30.1 0.6 8.7 5.9 1.3 11.7 5.0 
 
USSR  23.7 1.4 25.8 0.6 9.2 6.1 0.5 23.6 9.3 
Yugoslavia 48.7 1.5 16.8 1.1 4.9 6.4 11.3 5.3 4.0

1. The correlation matrix of the data is shown in Figure 5.42. From this correlation matrix plot we can see that the
percentage of people employed in agriculture is negatively correlated with virtually of the other employment
areas indicating a contrast between industrial and agricultural economies. We also see that the percent of people
employed in manufacturing is positively correlated with employment areas which are required support manu-
facturing such as power supply, mining, construction and transportation. Other interesting relationships between
these variables are also evident.
MATLAB code
>> R = corrcoef(X); %correlation matrix
>> imagesc(R);
>> set(gca,’XTick’,1:p); set(gca,’YTick’,1:p);

145
 F IGURE 5.42: Correlation matrix of the European jobs data.

>> set(gca,’XTickLabel’,variables); set(gca,’YTickLabel’,variables);

>> axis([0 p+1 0 p+1]); grid; colorbar;

2. When interpreting correlations it is important to visualize the bivariate relationships between all pairs of vari-
ables. This can be achieved by looking at a scatterplot matrix, which is shown in Figure 5.43

3. The plot the second PC score vs. the first PC score is shown in Figure 5.44. The labels displayed in Figure 5.44(b)
represent the names of the countries.
MATLAB code
>> scatter(Z(:,1),Z(:,2),15,’ko’,’MarkerFaceColor’,[.49 1 .63],’LineWidth’,1);
>> xlabel(’PC1 score’,’fontsize’,14,’fontname’,’times’);
>> ylabel(’PC2 score’,’fontsize’,14,’fontname’,’times’);
>> gname(countries); %press Enter or Escape key to stop labeling.

4. Using the columns of the eigenvector matrix

 
0.8918 −0.0068 0.1185 −0.0968 −0.1800 0.1526 −0.0916 −0.0687 0.3354
 0.0019 0.0923 0.0794 −0.0102 0.0011 −0.4564 0.7665 −0.2905 0.3240 
 −0.2713 0.7703 0.1847 −0.0104 −0.3360 0.2009 −0.1620 −0.0741 0.3375 
 
 −0.0084 0.0120 −0.0068 0.0181 0.0025 −0.2309 0.0629 0.9092 0.3399 
 
A=
 −0.0496 0.0690 −0.0773 −0.0829 0.7243 0.5584 0.1943 0.0045 0.3253 

 −0.1918 −0.2344 −0.5796 −0.6076 −0.2659 0.0216 −0.0879 −0.1044 0.3367 
 
 −0.0311 −0.1301 −0.4700 0.7812 −0.1211 0.0553 −0.0800 −0.1228 0.3344 
 −0.2980 −0.5668 0.5977 0.0483 −0.2359 0.2479 −0.0045 −0.0521 0.3324

−0.0454 −0.0099 0.1594 −0.0378 0.4349 −0.5459 −0.5675 −0.2238 0.3342
the first PC is then given by

Z1 = 0.89 X1 − 0.27 X3 − 0.192 X6 − 0.298 X8 = 0.89 Agr − 0.27 Man − 0.192 SI − 0.298 SPS

146
 100
50 Agr
0
4
2 Min
0
40
20 Man
0
2
1 PS
0
20
10 Con
0
20
10 SI
0
20
10 Fin
0
40
20 SPS
0
10
5 TC
0
0 501000 2 4 0 2040 0 1 2 0 10 200 10 200 10 200 20 400 5 10
F IGURE 5.43: Scatterplot matrix of the European jobs data.

15 15

E. Germany
10 10 Switzerland
Czechoslovakia Rumania
Bulgaria
Spain
5 5 W. Germany Hungary
Austria Poland
PC2 score

PC2 score

Luxembourg
Yugoslavia
0 0 Italy USSR Portugal
France Greece
United Kingdom
Finland
−5 −5 Belgium Ireland
Sweden
Norway Turkey
Netherlands
−10 −10
Denmark

−15 −15
−20 −10 0 10 20 30 40 50 60 −20 −10 0 10 20 30 40 50 60
PC1 score PC1 score
(a) unlabeled plot (b) labeled plot
F IGURE 5.44: PC2 vs. PC1 score for the European jobs data.

We can see that the first PC is essentially a contrast between agriculture and industrial/urban employment ar-
eas. This is evidenced by the positive coefficient for agriculture and the negative coefficients for manufacturing,
service industries, and social and personal services.
The second PC is given by

Z2 = 0.77 X3 − 0.234 X6 − 0.13 X7 − 0.567 X8 = 0.77 Man − 0.234 SI − 0.13 Fin − 0.567 SPS

We can see that the second PC appears to be a contrast between manufacturing and non-industrial areas such as
service industries and finance. This is evidenced by the positive coefficient for manufacturing and the negative
coefficients for service industries, finance, and social and personal services.
5. The scatter plot of PC2 coefficients vs. PC1 coefficients is shown in Figure 5.45. This plot helps understand which
variables have a similar involvement within PCs. As can be seen in Figure 5.45, the variables Agr is located on

147
 the right of the plot, while the other variables are located on the left of the plot. This is consistent with the values
of the coefficients of PC1 and PC2.
MATLAB code
>> scatter(A(:,1),A(:,2),3,’o’,’MarkerFaceColor’,[.49 1 .63],’LineWidth’,1);
>> gname(variables); %press Enter or Escape key to stop labeling.

0.8

Min
0.6

0.4 Man
PC2 coefficient

PS
0.2 TC

Con Agr
0

−0.2 SPS
SI
−0.4 Fin

−0.6
−0.4 −0.2 0 0.2 0.4 0.6 0.8 1
PC1 coefficient

F IGURE 5.45: Coefficients of PC2 vs. PC1 for the European jobs data.

6. The explained variance by the three PCs are: ℓ1 = 81.58%, ℓ2 = 11.75%, and ℓ3 = 4.09%. Notice that PC1 and PC2
combined account for 93.33% of the variance in the data. The scree and Pareto plots of the explained variance
vs. the number of PCs are shown in Figure 5.46. Based on the explained variance by both PC1 and PC2 and also
from the scree and Pareto plots, it can be deduced that the lowest-dimensional space to represent the European
jobs data corresponds to d = 2.
7. The 2D biplot of PC2 vs. PC1 is shown in Figure 5.47(a). The axes in the biplot represent the principal components
(columns of A), and the observed variables (rows of A) are represented as vectors. Each observation (row of Z)
is represented as a point in the biplot. From Figure 5.47(a), we can see that the first principal component has 1
positive coefficient for the first variable Agr and 3 negative coefficients for the variables Man, SI, and SPS. That
corresponds to 1 vector directed into the right half of the plot, and 3 vectors directed into the left half of the
plot, respectively. The second principal component, represented by the vertical axis, has 1 positive coefficient
for the variable Man, and 3 negative coefficients for the variables SI, FIN, and SPS. That corresponds to vectors
directed into the top and bottom halves of the plot, respectively. This indicates that this component distinguishes
between observations that have high values for the first set of variables and low for the second, and observations
that have the opposite. Each of the 26 countries is represented in this plot by a red point, and their locations
indicate the score of each observation for the two principal components in the plot. For example, points near the
left edge of this plot have the lowest scores for the first principal component. The variables are represented by
rays extending out from the plot origin. Rays that tend to point in the same direction represent variables that
are positively correlated. For example we can see that the rays for manufacturing, mining, and power supply
all point in the same direction, indicating the positive correlation of these employment areas with one another.
The ray for agriculture is fairly isolated indicating its weak positive correlation and more often times negative
correlation with the other employment areas. The cases are plotted in accordance with their scores on the first
three PCs. We can see that Turkey and Yugoslavia both extend far to the right in the direction of ray for the
agriculture variable. This indicates that these countries have a larger percentage of the workforce employed in
agriculture in comparison to the other countries in this data set. We can also see that Norway has a relatively
large percentage of its workforce employed in the social and service areas. The 3D biplot of PC1, PC2, and PC3 is
shown in Figure 5.47(b)

148
 90 100 100%

80

70 80 80%
Explained Variance %

Explained Variance %
60
60 60%
50

40
40 40%
30

20 20 20%
10

0 0 0%
1 2 3 4 5 6 7 8 9 1 2 3
Number of Principal Components Number of Principal Components
(a) Scree plot (b) Pareto plot
F IGURE 5.46: Scree and Pareto plots for the European jobs data.
MATLAB code
>> expvar=100*variance/sum(variance);%percent of the total variability explained by each PC.
>> plot(expvar,’ko-’,’MarkerFaceColor’,[.49 1 .63],’LineWidth’,1);
>> figure;
>> pareto(expvar);

0.8 Man

0.6

0.4
0.5
Component 3
Component 2

0.2 Man Agr

Min
Con SPS
PS
TC Agr 0 TC
Min
0 PS
Con
Fin
−0.2 SI −0.5
Fin
−0.4 SI
SPS
−0.6
0.5
−0.8 0 0.5
0
−0.5 −0.5
−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8
Component 1 Component 2 Component 1

(a) 2D biplot (b) 3D biplot

F IGURE 5.47: 2D and 3D biplots for the European jobs data.

MATLAB code
biplot(A(:,1:2),’Scores’,Z(:,1:2),’VarLabels’,variables)
figure; biplot(A(:,1:3),’Scores’,Z(:,1:3),’VarLabels’,variables)

8. The Hotelling and first PC charts are displayed in Figure 5.48. The Hotelling chart indicates that the samples 7
(Luxembourg), 18 (Yugoslavia), and 26 (Turkey) are out-of-control. All the plotted points on the first PC chart are
within the control limits.
MATLAB code
>> alpha = 0.05;
>> [outliers, h] = tsquarechart(X,alpha); %T^2 chart
>> figure;

149
 >> k=1;
>> [outliers, h] = pcachart(X,k); %1st PC control chart

20 60
26 UCL
18 18
7
40
16
UCL
14 20
Hotelling T2

12

Z1
0 CL
10

8 −20

6
−40
4
LCL
2 −60
0 5 10 15 20 25 30 0 5 10 15 20 25 30
Sample Number Sample Number

F IGURE 5.48: Hotelling and PCA charts for the European jobs data.

5.11 P ROBLEMS

❶ The management of a bank has embarked on a program of statistical process control and has decided to use
variable control charts to study the waiting time of customers during the peak noon to 1 p.m. lunch hour to
detect special causes of variation. Four customers are selected during the one-hour period; the first customer to
enter the bank every 15 minutes. Each set of four measurements makes up a subgroup (sample). Table 5.18 lists
the waiting time (operationally defined as the time from when the customer enters the line until he or she begins
to be served by the teller) for 20 days.

TABLE 5.18: Waiting Time for Customers at a Bank.

Sample Data
Number X1 X2 X3 X4
1 7.2 8.4 7.9 4.9
2 5.6 8.7 3.3 4.2
3 5.5 7.3 3.2 6.0
4 4.4 8.0 5.4 7.4
5 9.7 4.6 4.8 5.8
6 8.3 8.9 9.1 6.2
7 4.7 6.6 5.3 5.8
8 8.8 5.5 8.4 6.9
9 5.7 4.7 4.1 4.6
10 3.7 4.0 3.0 5.2
11 2.6 3.9 5.2 4.8
12 4.6 2.7 6.3 3.4
13 4.9 6.2 7.8 8.7
14 7.1 6.3 8.2 5.5
15 7.1 5.8 6.9 7.0
16 6.7 6.9 7.0 9.4
17 5.5 6.3 3.2 4.9
18 4.9 5.1 3.2 7.6
19 7.2 8.0 4.1 5.9
20 6.1 3.4 7.2 5.9

150
 a) Construct a table that shows the waiting time data with extra columns displaying the sample means and
sample ranges.
b) Estimate the process mean and standard deviation.
c) Construct the R- and the X-charts. Identify the out-of-control points using all Western Electric rules. If
necessary, revise your control limits, assuming that any samples that violate Western Electric rules can de
discarded.

❷ A sample data set called parts.mat in the MATLAB Statistics Toolbox contains measurements on newly ma-
chined parts, taken at one hour intervals for 36 hours. Each row of the runout matrix contains the measurements
for 4 parts chosen at random. The values indicate, in thousandths of an inch, the amount the part radius differs
from the target radius. To load the data set into the MATLAB workspace, type:

>> load parts

>> whos
Name Size Bytes Class Attributes
runout 36x4 1152 double

(i) Construct the R- and the X-charts. Identify the out-of-control points. If necessary, revise your control limits,
assuming that any samples that plot outside the control limits can de discarded.
(ii) Assuming the process is in control, estimate the process mean and standard deviation.
(iii) Construct the s- and the X-charts. Identify the out-of-control points. If necessary, revise your control limits,
assuming that any samples that plot outside the control limits can de discarded.

❸ Table 5.19 presents the weights, in ounces, for a sequence of 15 rational subgroup samples of potato chips, with
n = 4 for each sample. Assume that the specifications are 14 ± 1.37.

TABLE 5.19: Potato chip Data.

Sample Package Weights (oz)
Number X1 X2 X3 X4
1 15.01 14.98 15.16 14.80
2 15.09 15.14 15.08 15.03
3 15.04 15.10 14.93 15.13
4 14.90 15.03 14.94 14.92
5 15.04 15.05 15.08 14.98
6 14.96 14.81 14.96 14.91
7 15.01 15.10 14.90 15.03
8 14.71 14.92 14.77 14.95
9 14.81 14.80 14.64 14.95
10 15.03 14.89 14.99 15.03
11 15.16 14.91 14.95 14.83
12 14.92 15.05 15.01 15.02
13 15.06 15.03 14.95 15.02
14 14.99 15.14 15.04 15.11
15 14.94 15.08 14.90 15.17

(i) Construct the R- and the X-charts. Is the process under statistical control? Explain.
(ii) Construct the s- and the X-charts. Is the process under statistical control? Explain.
(iii) Assuming that the package weights are normally distributed, calculate the process capability index and the
proportion of the product that will not meet specifications.
(iv) Comment on the ability of the process to produce items that meet specifications?

❹ The diameter of holes is measured in consecutive order by an automatic sensor. The results of measuring 25 holes
are given in in Table 5.20. Assume the target diameter is 10 millimeters.

(i) Estimate the process standard deviation

151
 TABLE 5.20: Diameter measurements.
Sample Diameter Sample Diameter
1 9.94 14 9.99
2 9.93 15 10.12
3 10.09 16 9.81
4 9.98 17 9.73
5 10.11 18 10.14
6 9.99 19 9.96
7 10.11 20 10.06
8 9.84 21 10.11
9 9.82 22 9.95
10 10.38 23 9.92
11 9.99 24 10.09
12 10.41 25 9.85
13 10.36

(ii) Set up and apply a tabular cusum for this process, using standardized values h = 5 and k = 0.5. Does the
process appear to be operating in a state of statistical control at the desired target level?
(iii) Apply an EWMA control chart to these data using λ = 0.4 and L = 3. Interpret this chart.
❺ The wafer dataset (wafer.mat) is based on one found in “Statistical case studies for industrial process improve-
ment” by Czitrom and Spagon. It consists of oxide layer thickness measured in 9 locations on each of 116 semi-
conductor wafers. The measurements were taken by position on the wafer as shown in Figure 5.49. Note that the
first is in the center, the next 4 halfway out, and the last 4 on the edge. To load the data set into the MATLAB

6 9
2

3 1 5

4
7 8

F IGURE 5.49: Layout of wafer data.

workspace, type:

>> load wafer.mat

>> whos
Name Size Bytes Class Attributes
X 116x9 8352 double
description 5x89 890 char
variables 9x2 36 char

a) Display the correlation matrix of the data.

b) Display the side-by-side boxplots of the data. Comment on the plots.
c) Plot the second PC score vs. the first PC score. Comment on the plot.
d) Determine the first and second PCs.
e) Display the scatter plot of PC2 coefficients vs. PC1 coefficients, and label the points. Comment on the plot.
f) Compute the explained variance, and plot it against the number of PCs. What would be the lowest-
dimensional space to represent the phosphorus content data?
g) Display the 2D biplot of PC2 vs. PC1. Then, display the 3D biplot of PC1, PC2, and PC3. Comment on the
plots.
h) Plot the Hotelling and first PC control charts. Identify the out-of-control points.

152
5.12 REFERENCES

[1] D. C. Montgomery, Introduction to Statistical Quality Control, John Wiley & Sons, 6th edition, 2009.
[2] K. Yang and J. Trewn, Multivariate Statistical Process Control in Quality Management, Mc-Graw Hill Professional,
2004.
[3] K.H. Chen, D.S. Boning, and R.E. Welch, “Multivariate statistical process control and signature analysis using
eigenfactor detection methods,” Proc. Symposium on the Interface of Computer Science and Statistics, Costa Mesa, CA,
June 2001.
[4] N.D. Tracy, J.C. Young, and R.L. Mason, “Multivariate quality control charts for individual observations,” Journal
of Quality Technology, vol. 24, no. 22, pp. 88-95, 1992.

[5] J.A. Vargas, “Robust estimation in multivariate control charts for individual observations,” Journal of Quality Tech-
nology, vol. 35, no. 4, pp. 367-376, 2003.
[6] J.H. Sullivan and W.H. Woodall, “A comparison of multivariate control charts for individual observations,” Journal
of Quality Technology, vol. 28, no. 24, pp. 398-408, 1996.
[7] I.T. Jolliffe, Principal Component Analysis, New York: Springer, 1986.
[8] T.F. Cox, An Introduction to Multivariate Data Analysis, Hodder Arnold, 2005.

153
 C HAPTER 6

R EGRESSION AND A NALYSIS OF VARIANCE

The analysis of variance is not a mathematical

theorem, but rather a convenient method of
arranging the arithmetic.

Ronald Fisher

Regression analysis is a statistical tool for the investigation of relationships between variables. The goal of regression
analysis is to determine the values of parameters for a function that cause the function to best fit a given set of data
observations. Regression analysis is very useful in quality control because you can determine how much to change if
you have a quality issue. The simplest type of regression analysis is linear regression, which is a method for predicting
the value of a dependent variable, based on the value of one or more independent variables. For example, in the
relationship between age and weight of a cow during a specific phase of production, age is the independent variable
and weight is the dependent variable. As the cows age increases, its weight will also increase.
On the other hand, Analysis of Variance (ANOVA) is a method for testing differences among two or more means
by analyzing variance. An ANOVA conducted on an experimental design in which there is only one factor is called
a one-way ANOVA. If an experiment has two factors, then the ANOVA is called a two-way ANOVA. ANOVA is used in
the hope of showing that there is a difference between distribution means. For example, several groups of patients,
all suffering from high blood pressure, are submitted to several new treatments (one treatment for each group). These
treatments are expected to have different efficacies, and it is hoped that some of them will turn out to be particularly
effective. After the treatments, ANOVA will be used on blood pressure measurements, and it is hoped that it will reject
the hypothesis that all treatments are equally effective (or ineffective).

6.1 L INEAR R EGRESSION

Regression analysis concerns the study of relationships between quantitative variables with the object of identifying,
estimating, and validating the relationship. The estimated relationship can then be used to predict one variable from
the value of the other variable(s). Suppose we have n pairs of observations ( xi , yi ). Plotting a scatter plot is an important
preliminary step prior to undertaking a formal statistical analysis of the relationship between the variables x and y. The
variable x is called independent or predictor variable, while the variable y is called dependent or response variable.
The scatter plot shows whether or not there is a linear (straight-line) relation between the dependent and independent
variables. For example, the scatter plot of the car plant data given in Table 6.1 is shown in Figure 6.1 and it reveals that
the relationship between x and y is approximately linear; that is, the points seem to cluster around a straight line.

Example 6.1 The manager of a car plant wishes to investigate how the plant’s electricity usage depends upon the plant’s produc-
tion. A sample of 12-month data recorded from January to December of the previous year is selected. The production x (in million
dollars) and the electricity usage y (in million kWh) for each month are listed in Table 6.1. Construct a scatter plot of the data.
Solution:
MATLAB code
>> x = [4.51 3.58 4.31 5.06 5.64 4.99 5.29 5.83 4.70 5.61 4.90 4.20]’;
>> y = [2.48 2.26 2.47 2.77 2.99 3.05 3.18 3.46 3.03 3.26 2.67 2.53]’;
>> scatter(x,y,’ko’,’MarkerFaceColor’,[.49 1 .63]);

154
 TABLE 6.1: Car plant data.
Observation Production x Electricity usage y
January 4.51 2.48
February 3.58 2.26
March 4.31 2.47
April 5.06 2.77
May 5.64 2.99
June 4.99 3.05
July 5.29 3.18
August 5.83 3.46
September 4.70 3.03
October 5.61 3.26
November 4.90 2.67
December 4.20 2.53

The output scatter plot is displayed in Figure 6.1.

Since a linear relation is the simplest relationship to handle mathematically, we present the details of the statistical
regression analysis for this case. In simple linear regression, we assume that each observed value yi of the response
variable Yi can be described as follows:

Linear Regression Model:

yi = β0 + β1 xi + ε i , i = 1, . . . , n (6.1.1)
where
• yi is the dependent variable that we wish to predict or explain
• xi is the independent variable used to predict or explain y i
• β 0 and β 1 are unknown parameters, intercept and slope of the line, respectively, also called regression coefficients.
• ε i ∼ N (0, σ2) is a random error with unknown variance

3.5

3
y

2.5

2
3.5 4 4.5 5 5.5 6
x

F IGURE 6.1: Scatter plot of the car plant data.

155
 The term linear is used because Eq. (6.1.1) is a linear function of the unknown parameters β 0 and β 1 . The simple
linear regression model given by Eq. (6.1.1) may be written in matrix form as

y = Xβ + ε (6.1.2)

where      
y1 1 x1 ε1
 y2   1 x2     ε2 
    β0  
y= ..  X= .. ..  β= ε= ..  (6.1.3)
 .   . .  β1  . 
yn 1 xn εn
Since the random error ε i ∼ N (0, σ2), it follows that the values yi are observations from the independent random
variables Yi ∼ N ( β 0 + β 1 xi , σ2 ). Using the least-square method to fit the simple linear regression model (6.1.1), the
unknown parameters β 0 and β 1 can be found by minimizing the sum of squared deviations
n n
2
L= ∑ ε2i = ∑ yi − ( β0 + β1 xi ) (6.1.4)
i =1 i =1

Thus, the least squares estimators β̂ 0 and β̂ 1 that minimize L must satisfy
n
∂L
= −2 ∑ (yi − β̂ 0 − β̂ 1 xi ) = 0 (6.1.5)
∂β 0 i =1
n
∂L
= −2 ∑ (yi − β̂ 0 − β̂ 1 xi ) xi = 0 (6.1.6)
∂β 0 i =1

Simplifying these two equations yields to the least squares normal equations given by

 n n 

n β̂ 0 + β̂ 1 ∑ xi = ∑ yi
 
 β̂ 0 = ȳ − β̂ 1 x̄
i =1 i =1 =⇒ ∑n x y − n x̄ ȳ Sxy


n n
2
n

 β̂ 1 = i=n 1 i 2 i =
 0∑ i
 β̂ x + β̂ 1∑ i
x = ∑ i i
y x x
∑ i =1 i − n x̄ 2 S xx
i =1 i =1 i =1

where
n n
Sxx = ∑ (xi − x̄ )2 and Sxy = ∑ (xi − x̄ )(yi − ȳ) (6.1.7)
i =1 i =1

Using the identity ∑ni=1 ( xi − x̄ ) = 0, we can also write Sxx and Sxy as
n n
Sxx = ∑ xi (xi − x̄ ) and Sxy = ∑ yi (xi − x̄). (6.1.8)
i =1 i =1

Therefore, the estimated or fitted value of yi is given by the prediction or regression equation ŷi = β̂ 0 + β̂ 1 xi , where β̂ 0
and β̂ 1 are estimators of β 0 and β 1 , respectively.

Least-squares estimates:
Sxy
Slope: β̂ 1 =
Sxx
y-intercept: β̂ 0 = ȳ − β̂ 1 x̄
Regression line: ŷ = β̂ 0 + β̂ 1 x

Example 6.2 Using the data of Table 6.1,

1. Compute the least-squares estimate β̂ 0 and β̂ 1

156
 2. Find the regression line and plot it on the scatter plot.
Solution: We treat the production as the predictor variable, x, and electricity usage as the response variable, y.

1 n = 1 2 ; % Sample s i z e
2 x = [ 4 .5 1 3 .5 8 4 .3 1 5 .0 6 5 .6 4 4 .9 9 5 .2 9 5 .8 3 4 .7 0 5 .6 1 4 .9 0 4 .2 0 ] ' ;
3 y = [ 2 .4 8 2 .2 6 2 .4 7 2 .7 7 2 .9 9 3 .0 5 3 .1 8 3 .4 6 3 .0 3 3 .2 6 2 .6 7 2 .5 3 ] ' ;
4 s c a t t e r ( x , y , ' ko ' , ' MarkerFaceColor ' , [ .4 9 1 .6 3 ] ) ;
5 X = [ ones ( s i z e ( x ) ) x ] ;
6 b = regress ( y ,X) ;
7 b e ta 0 h a t = b ( 1 ) ; % E s ti m a te o f i n t e r c e p t
8 b e ta 1 h a t = b ( 2 ) ; % E s ti m a te o f s l o p e
9
10 % Sums o f squares
11 Sxx = ( n − 1) * v a r ( x ) ;
12 Syy = ( n − 1) * v a r ( y ) ;
13 Sxy = b e ta 1 h a t * Sxx ;
14 SSr = b e ta 1 h a t * Sxy ; % R e s i d u a l sum o f squares
15 SSt = Syy ; % T o t a l sum o f squares
16
17 y h a t = b e ta 0 h a t + b e ta 1 h a t * x ; % Regression l i n e
18 h o l d on ; p l o t ( x , yhat , ' r ' ) ;

1. The returned values of the estimated parameters intercept and slope are β̂ 0 = 0.409 and β̂ 1 = 0.499, respectively.
2. The estimated regression line is ŷ = 0.409 + 0.499x. The value of β̂ 1 means that for each increase of 1 unit in
x, the mean value of y is estimated to increase by 0.499. In other words, the positive slope value implies that
the estimated mean electricity usage increases by 0.499 million kWh for each additional million of production.
Figure 6.2 shows a scatter plot of the data and the graph of the estimated regression line (in red color).

3.5
ŷ = 0.41 + 0.50x

3
y

2.5

2
3.5 4 4.5 5 5.5 6
x

F IGURE 6.2: Scatter plot and regression line of the car plant data.

Example 6.3 Show that β̂ 0 and β̂ 1 are unbiased estimators of β 0 and β 1

Solution: Since E(yi ) = β 0 + β 1 xi and E(Sxy ) = ∑ni=1 E(yi )( xi − x̄ ), it follows that E(Sxy ) = β 1 Sxx . Thus, E( β̂ 1 ) = β 1 .
Also, E(ȳ) = β 0 + β 1 x̄ implies that E( β̂ 0 ) = β 0 . Therefore, β̂ 0 and β̂ 1 are unbiased estimators of β 0 and β 1 .

Example 6.4 Find the variances of β 0 and β 1

157
Solution: Since var (yi ) = σ2 and var (Sxy ) = ∑ni=1 var (yi )( xi − x̄ )2 , it follows that var (Sxy ) = σ2 Sxx . Thus,
var ( β̂ 1 ) = σ2 /Sxx . Similarly, it can be shown that var ( β̂ 0 ) = σ2 (1/n + x̄2 /Sxx ). Therefore, β̂ 1 ∼ N ( β 1, σ2 /Sxx )
and β̂ 0 ∼ N ( β 0 , σ2 (1/n + x̄2 /Sxx )).

6.1.1 R ESIDUAL A NALYSIS

In order to check the regression model assumptions, we need to compute and examine the normality of the residuals
ei = y i − ŷi , where yi is the observed value and ŷi is the predicted (fitted) value. The residuals are used to evaluate the
goodness of fit of the regression (also called least square) line. The calculated predicted values and residuals for the car
plant data are shown in Table 6.2.

TABLE 6.2: Predicted values and residuals for car plant data.
Production x Electricity usage y Predicted value ŷ Residual e = y − ŷ
4.51 2.48 2.6588 -0.1788
3.58 2.26 2.1949 0.0651
4.31 2.47 2.5590 -0.0890
5.06 2.77 2.9331 -0.1631
5.64 2.99 3.2225 -0.2325
4.99 3.05 2.8982 0.1518
5.29 3.18 3.0479 0.1321
5.83 3.46 3.3172 0.1428
4.70 3.03 2.7535 0.2765
5.61 3.26 3.2075 0.0525
4.90 2.67 2.8533 -0.1833
4.20 2.53 2.5041 0.0259

The first assumption we consider is that the random errors ε i are normally distributed. A normal probability plot of
the residuals (or standardized residuals zi = (yi − ŷi )/σ̂) will give an indication of whether or not the assumption of
normality of the random errors is appropriate. Recall that a normal probability plot is found by plotting the quantiles
of the observed sample against the corresponding quantiles of a standard normal distribution N (0, 1). If the normal
probability plot shows a straight line, it is reasonable to assume that the observed sample comes from a normal distri-
bution. If, on the other hand, the points deviate from a straight line, there is statistical evidence against the assumption
that the random errors are an independent sample from a normal distribution.
The normal probability plot for the car plant data, displayed in Figure 6.3, shows a relatively linear pattern. The fact
that the points in the lower and upper extremes of the normal probability plot do not deviate significantly from the
straight-line pattern indicates the normality assumption of the errors is not violated. That is, we can quite reasonably
conclude that the normal distribution provides a good model for the residuals.

1 % Normal p r o b a b i l i t y p l o t o f r e s i d u a l s
2 r e s = y−y h a t ; % R e s i d u a l s
3 n o r m p l o t ( r e s ) ; % D i s p l a y normal p r o b a b i l i t y p l o t

The second assumption we consider is that the random errors ε i have zero mean and same variance σ2 . A residual
plot, which is a scatter plot of the residuals ei against the predicted (fitted) values ŷi or against the independent variable
x, can be used to check this assumption. More precisely, if the second assumption is satisfied then we would expect
the residuals to vary randomly around zero and we would also expect the spread of the residuals to be about the same
throughout the residual plot. That is, if there is no violation in assumptions, this scatter plot should look like a band
around zero with randomly distributed points and no discernible pattern. There should be no relation between the
residuals and fitted values (or independent variables).
The residual plot against the predicted values for the car plant data, shown in Figure 6.4, indicates that the points
seem to be fluctuating randomly around zero. The residual plot against the independent variable is also displayed
in Figure 6.4. Although there is widespread scatter in the residual plots, there is no apparent pattern or relationship
between the residuals and ŷ (or x). The residuals appear to be evenly spread above and below zero. Thus, the residual

158
 0.95

0.90

0.75

Probability
0.50

0.25

0.10

0.05

−0.2 −0.15 −0.1 −0.05 0 0.05 0.1 0.15 0.2 0.25

Residuals

F IGURE 6.3: Normal probability plot of the residuals for the car plant data.

plots do not suggest violations of the assumptions of zero means and same variance of the random errors. Therefore,
the linear regression fits the data adequately.

1 % Residual p l o t ( r e si d u a l s vs. f i t t e d values )

2 r e s = y−y h a t ; % R e s i d u a l s
3 s c a t t e r ( yhat , y−yhat , ' ko ' ) ; % D i s p l a y r e s i d u a l p l o t
4 h = r e f l i n e ( 0 ) ; s e t ( h , ' l i n e s t y l e ' , '− ' , ' c o l o r ' , ' r ' ) ; % H o r i z o n t a l zero − l i n e

0.4

0.3

0.2
Residuals, e = y − ŷ

0.1

−0.1

−0.2

−0.3

−0.4
2 2.5 3 3.5
Predicted values, ŷ

F IGURE 6.4: Residual plot against the predicted values for the car plant data.

1 % Residual p l o t ( r e si d u a l s vs. f i t t e d values )

159
 2 r e s = y−y h a t ; % R e s i d u a l s
3 s c a t t e r ( x , y−yhat , ' ko ' ) ; % D i s p l a y r e s i d u a l p l o t
4 h = r e f l i n e ( 0 ) ; s e t ( h , ' l i n e s t y l e ' , '− ' , ' c o l o r ' , ' r ' ) ; % H o r i z o n t a l zero − l i n e

0.4

0.3

0.2

Residuals, e = y − ŷ
0.1

−0.1

−0.2

−0.3

−0.4
3.5 4 4.5 5 5.5 6
Production, x

F IGURE 6.5: Residual plot against the independent variable for the car plant data.

Therefore, in order to perform statistical inference on the regression, it must be the case that the residuals are distributed
with constant error variance. This requirement is easily verified through normal probability and residual plots.

Example 6.5 Let SSE = ∑ni=1 e2i be the sum of residuals. Show that SSE /(n − 2) is an unbiased estimator of σ2.

Solution: Denote by S 2 = SSE /(n − 2). Since (n − 2)S 2 /σ2 ∼ χ2 (n − 2), it follows that E((n − 2)S 2 /σ2 ) = n − 2.
Thus, E(S 2 ) = σ2 . That is, σ̂2 = SSE /(n − 2). This estimate of σ2 has n − 2 degrees of freedom (2 is subtracted from n
because two unknown parameters, β 0 and β 1 , are estimated).

Standard errors of β 0 and β 1 :

s  
1 x̄2
s.e.( β̂ 0 ) = σ̂2 + (6.1.9)
n Sxx
s
σ̂2
s.e.( β̂ 1 ) = (6.1.10)
Sxx

Example 6.6 Using the data of Table 6.1,

1. Compute the estimate of σ2
2. Compute the standard errors s.e.( β̂ 0 ) and s.e.( β̂ 1 )
Solution:

1 % SSe and v a r i a n c e e s t i m a t e
2 SSe = SSt − SSr ;
3 sigma2hat = SSe / ( n − 2) ;

160
 4
5 % Standard e r r o r s o f b e ta 0 h a t and b e ta 1 h a t
6 s e b e ta 0 h a t = s q r t ( sigma2hat * ( 1 / n + mean ( x ) ˆ 2 / Sxx ) ) ;
7 s e b e ta 1 h a t = s q r t ( sigma2hat / Sxx ) ;

1. The returned value is σ̂2 = 0.030. The estimated value of standard deviation σ̂ = 0.1732 implies that most of the
observed electricity usage (y) values will fall within approximately 2σ̂ = 0.3464 million kWh of their respective
predicted values.

2. The returned standard errors are s.e.( β̂ 0 ) = 0.386 and s.e.( β̂ 1 ) = 0.078352.

6.1.2 C ONFIDENCE I NTERVALS FOR THE I NTERCEPT β0 AND S LOPE β 1

It can be shown that the statistics

β̂ 0 − β 0 β̂ 1 − β 1
∼ t ( n − 2) and ∼ t ( n − 2) (6.1.11)
s.e.( β̂ 0 ) s.e.( β̂ 1 )

follow a t distribution with n − 2 degrees of freedom, where n is the number of observations.

Confidence Intervals for the Slope β 1 :

• A 100(1 − α)% confidence interval on β 1 is given by

β̂ 1 − tα/2,n−2 s.e.( β̂ 1 ) ≤ β 1 ≤ β̂ 1 + tα/2,n−2 s.e.( β̂ 1 )

• A 100(1 − α)% upper-confidence interval on β 1 is given by

β 1 ≤ β̂ 1 + tα,n−2 s.e.( β̂ 1 )

• A 100(1 − α)% lower-confidence interval on β 1 is given by

β̂ 1 − tα,n−2 s.e.( β̂ 1 ) ≤ β 1

Similarly, the confidence intervals for the intercept can be obtained by replacing β 1 and β̂ 1 with β 0 and β̂ 0 , respec-
tively, in the above box.

Example 6.7 Using the data of Table 6.1, find a 95% confidence interval for the true slope of the line. Interpret your result.

Solution:

1 % 95% c o n fi d e n c e i n t e r v a l
2 alpha = 0 .0 5 ; % S i g n i f i c a n c e l e v e l
3 t a l p h a = i c d f ( ' t ' ,1 − alpha / 2 , n − 2) ; %t { alpha / 2 , n−2}
4 %Confidence I n t e r v a l s L i m i t s
5 l c l = beta1hat − t a l p h a * s e b e ta 1 h a t ; % Lower−c o n fi d e n c e l i m i t
6 u c l = b e ta 1 h a t+ t a l p h a * s e b e ta 1 h a t ; % upper−c o n fi d e n c e l i m i t

100(1 − α)% = 95% implies that α = 0.05. Thus, the 95% confidence interval for the slope β 1 is

( β̂ 1 ± tα/2,n−2 s.e.( β̂ 1 ) = (0.499 ± (2.228139)(0.078352)) = (0.324252, 0.673409)

Thus, 0.324252 ≤ β 1 ≤ 0.673409. That is, on average, the monthly electricity usage increases by an amount between
0.32 and 0.67 for every extra monthly production.

161
6.1.3 H YPOTHESIS T ESTING ABOUT THE S LOPE β 1
When the residuals are normally distributed with constant error variance, we can perform inference on the linear
regression equation. We will now test the hypothesis of no linear relation between the predictor and the response
variable. In testing the population mean µ, there are three ways to structure the hypothesis test:

Two-tailed Upper-Tailed Lower-Tailed

H0 : β 1 = 0 H0 : β 1 = 0 H0 : β 1 = 0
H1 : β 1 6= 0 H1 : β 1 > 0 H1 : β 1 < 0

These hypotheses relate to the significance of regression. Failure to reject the null hypothesis is equivalent to concluding
that there is no linear relationship between x and y. On the other hand, rejection of the null hypothesis implies that x
is of value in explaining the variability in y. In the two-tailed test, we are testing the claim that a linear relation exists
between two variables without regard to the sign of the slope. In the upper-tailed test, we are testing that the claim
that the slope of the true regression line is positive. In the lower-tailed test, we are testing that the claim that the slope
of the true regression line is negative.
Under the assumption that the null hypothesis is true (i.e. H0 : β 1 = 0), the test statistic
β̂ 1
T0 = ∼ t ( n − 2) (6.1.12)
s.e.( β̂ 1 )
follows a t distribution with n − 2 degrees of freedom, where n is the number of observations.
Recall that a critical region or rejection region is the set of all values such that the null hypothesis is rejected. Let t0
be the numerical value, calculated from the sample data, of the test statistic T0 . Then, for a selected significance level
α, the critical regions are

Two-tailed Upper-Tailed Lower-Tailed

If t0 < −tα/2,n−2 or t0 > tα/2,n−2 If t0 > tα,n−2 If t0 < −tα,n−2
reject H0 reject H0 reject H0

Denote by F (·) the cdf of the t(n − 2) distribution. Then, the t-test about the slope β 1 may be summarized as follows:

Hypotheses:
Two-tailed Upper-Tailed Lower-Tailed
H0 : β 1 = 0 H0 : β 1 = 0 H0 : β 1 = 0
H1 : β 1 6= 0 H1 : β 1 > 0 H1 : β 1 < 0
β̂ 1
Test Statistic (t-test): T0 =
s.e.( β̂ 1 )
Critical Regions:
Two-tailed Upper-Tailed Lower-Tailed
|t0 | > tα/2,n−2 t0 > tα,n−2 t0 < −tα,n−2
P-values:
Two-tailed Upper-Tailed Lower-Tailed
2[1 − F (|t0 |)] 1 − F ( t0 ) F ( t0 )

Example 6.8 Using the data of Table 6.1,

1. Conduct a test to determine if the true slope of the line differs from 0. Use α = 0.05
2. Conduct a test to determine whether y is positively related to x. Use α = 0.05
Solution: Since the residuals are normally distributed with constant error variance, we can perform statistical inference
on the regression.

162
 1 alpha = 0 .0 5 ; % S i g n i f i c a n c e l e v e l
2 t a l p h a = i c d f ( ' t ' ,1 − alpha / 2 , n − 2) ; %t { alpha / 2 , n−2}
3 t 0 = b e ta 1 h a t / s e b e ta 1 h a t ; % Value o f T0 s t a t i s t i c
4 pvalue = 2 * (1 − c d f ( ' t ' , t0 , n − 2) ) ; % p−v a l u e

1. We are testing the claim that there is no linear relation between production and electricity usage.

Step 1: We test the hypotheses (two-tailed)

H0 : β1 = 0
H1 : β 1 6= 0

Step 2: The value of the t-test is given by

β̂ 1
t0 = = 6.366551
s.e.( β̂ 1 )

Step 3: The rejection region is |t0 | > tα/2,n−2, where tα/2,n−2 = t0.025,10 = 2.228139.
Step 4: Since |t0 | = 6.366551 > tα/2,n−2 = 2.228139, we reject the null hypothesis H0 .

Thus, the null hypothesis H0 : β 1 = 0 should be rejected. The same conclusion can be drawn by computing the
p-value = 2[1 − F (|t0 |)] = 0.000082, which is much smaller than α = 0.05.

2. We are testing the claim that the slope of the true regression is positive.

Step 1: We test the hypotheses (upper-tailed)

H0 : β1 = 0
H1 : β1 > 0

Step 2: The value of the t-test is given by

β̂ 1
t0 = = 6.366551
s.e.( β̂ 1 )

Step 3: The rejection region is t0 > tα,n−2, where tα,n−2 = t0.05,10 = 1.8125.
Step 4: Since t0 = 6.366551 > tα,n−2 = 1.8125, we reject the null hypothesis H0 .

Thus, the null hypothesis H0 : β 1 = 0 should be rejected.

6.1.4 A NALYSIS OF VARIANCE P ROCEDURE FOR T ESTING S IGNIFICANCE OF R EGRESSION

The total variability SST = ∑ni=1 (yi − ȳ)2 (called total sum of squares) in the dependent variable y can be partitioned into
the variability explained by the regression line SSR and the variability SSE (error sum of squares) about the regression
line as follows:
SST = SSR + SSE (6.1.13)
where SSR = ∑ni=1 (ŷi − ȳ)2 = β̂ 1 Sxy = S2xy /Sxx is referred to as the regression sum of squares, and it represents the
variation in y that is accounted for by regression on x.

Example 6.9 Show that the expected value of the regression sum of squares is E(SSR ) = σ2 + β21 Sxx

Solution: Since E(S2xy ) = var (Sxy ) + ( E(Sxy ))2 = σ2 Sxx + β21 S2xx , it follows that E(SSR ) = E(S2xy )/Sxx = σ2 + β21 Sxx .

Note that SST , SSR and SSE have n − 1, 1, and n − 2 degrees of freedom, respectively. Since SSE /σ2 ∼ χ2 (n − 2) and
SSR /σ2 ∼ χ2 (1), it follow that if the null hypothesis H0 : β 1 = 0 is true then E(SSR ) = σ2 , and the statistic

163
 SSR /1 MSR
F0 = = ∼ F (1, n − 2) (6.1.14)
SSE /(n − 2) MSE

where MSR = SSR /1 and MSE = SSE /(n − 2) are referred to as mean square regression and mean square error,
respectively.
Let f 0 is the numerical value, calculated from the data, of the test statistic F0 and f α,1,n−2 is the upper-percentage
points of the F-distribution F (1, n − 2). If f 0 > f α,1,n−2 then H0 : β 1 = 0 would be rejected.
This analysis of variance (ANOVA) for testing significance of regression is usually represented in tabular form, called
ANOVA table, as shown in Table 6.3. The ANOVA table for a linear regression gives the sum of squares, degrees of
freedom, mean squares for regression and error, and the value of the ANOVA F statistic.

TABLE 6.3: Analysis of Variance for Testing Significance of Regression.

Source of Variation Sum of Squares Degrees of Freedom Mean Square F0
MSR
Regression SSR 1 MSR
MSE
Error SSE n−2 MSE
Total SST n−1

Example 6.10 Using the data of Table 6.1, test the hypothesis H0 : β 1 = 0 versus H1 : β 1 6= 0 using the analysis of variance
procedure with α = 0.05.

Solution: The ANOVA for testing the null hypothesis H0 : β 1 = 0 versus the alternative hypothesis H1 : β 1 6= 0 is
summarized in Table 6.4. We want to test the significance of regression with α = 0.05.

Step 1: We test the hypotheses

H0 : There is no significant linear relationship

H1 : There is a significant linear relationship

or equivalently

H0 : β1 = 0
H1 : β 1 6= 0

Step 2: The value of the test statistic is given by

MSR
f0 = = 40.533
MSE

Step 3: The rejection region is f 0 > f α,1,n−2, where f α,1,n−2 = f 0.05,1,10 = 4.9646.

Step 4: Since f 0 = 40.533 > f α,1,n−2 = 4.9646, we reject the null hypothesis H0 .

Thus, we conclude that the monthly electricity usage of the plant is linearly related to its monthly production.

TABLE 6.4: ANOVA table for testing significance of regression on car plant data.
Source of Variation Sum of Squares Degrees of Freedom Mean Square F0
Regression 1.212 1 1.212 40.533
Error 0.299 10 0.030
Total 1.511 11

164
6.1.5 C ORRELATION AND D ETERMINATION C OEFFICIENTS
The correlation coefficient, r, is given by

Sxy
r= p (6.1.15)
Sxx Syy

and it measures the strength of the linear relationship between the variables y and x. The value of r is always between
-1 and 1. The sign of r is the same as the sign of β̂ 1 . A value of r ≈ 0 implies little or no linear relationship between y and
x, while a value of r ≈ 1 or r ≈ −1 indicates a strong positive or negative linear relationship, respectively. A positive
correlation coefficient (r > 0) indicates a positive slope, while a negative correlation coefficient (r < 0) indicates a
negative slope for the regression line.
The proportion of the total variability accounted for by the regression line is the coefficient of determination, r2 , given
by

SSR SS
r2 = = 1− E (6.1.16)
SST SST

which takes a value between 0 and 1.

For example, the coefficient of determination for the car plant data is r2 = 0.802, meaning that 80.2% of the sample
variation in electricity usage values can be explained by using the production to predict the electricity usage in the
straight-line model. The correlation coefficient is r = 0.8955 because the sign of β̂ 1 is positive. Thus, the relationship
between production and electricity usage is linear with a positive slope.

6.1.6 R EGRESSION A NALYSIS : A C OMPLETE E XAMPLE

Example 6.11 A random sample of eight drivers insured with a company and having similar auto insurance policies was selected.
Table 6.5 lists their driving experiences (in years) and monthly auto insurance premiums (in dollars).

TABLE 6.5: Auto insurance data.

Driving Monthly Auto
Experience Insurance Premium
5 64
2 87
12 50
9 71
15 44
6 56
25 42
16 60

1. Does the insurance premium depend on the driving experience or does the driving experience depend on the insurance
premium? Do you expect a positive or negative relationship between these two variables.

2. Find the least-squares regression line by choosing appropriate dependent and independent variables based on your answer in
part 1.

3. Interpret the meaning of the values β̂ 0 and β̂ 1 calculated in part 2.

4. Display the scatter plot and the regression line.

5. Calculate the correlation and determination coefficients, and explain what they mean.

6. Predict the monthly auto insurance premium for a driver with 10 years of driving experience.

165
 7. Calculate the standard errors

8. Display the normal probability plot. Does the normality assumption seem reasonable?

9. Display the residual plot. Does the constant variance assumption seem reasonable?

10. Construct a 90% confidence interval for β 1

11. Test at the 5% significance level whether β 1 is negative.

12. Conduct a t test (concerning β 1 ) for a significant regression. Use a significance level of 0.05.

13. Construct the ANOVA table and conduct an F-test for a significant regression. Use a significance level of 0.05.

Solution:

1. Intuitively, we expect the insurance premium to depend on driving experience. Thus, the insurance premium is
a dependent variable and driving experience is an independent variable in the regression model. A new driver is
considered a high risk by insurance companies, and has to pay a higher premium for auto insurance. Therefore,
the insurance premium is expected to decrease with an increase in the years of driving experience. Thats is, we
expect a negative relationship between these two variables.

2. The least-squares regression line is ŷ = 76.66 − 1.55x, where β̂ 0 = 76.66 and β̂ 1 = −1.55.

3. The value β̂ 0 = 76.66 gives the value of ŷ for x = 0; that is, it gives the monthly auto insurance premium for a
driver with no driving experience. The value β̂ 1 = −1.55 gives the change in ŷ due to a change of one unit in
x, indicating that, on average, for every extra year of driving experience, the monthly auto insurance premium
decreases by $1.55.

4. The scatter plot and the regression line are shown in Figure 6.6. As expected, we can see a negative relationship
between the dependent and independent variables.

90
ŷ = 76.66 − 1.55x
85

80

75

70

65
y

60

55

50

45

40
0 5 10 15 20 25
x

F IGURE 6.6: Scatter plot and regression line for the auto insurance data.

5. The value of r = −0.77 indicates that the driving experience and the monthly auto insurance premium are
negatively related. The value of r2 = 0.59 states that 59% of the total variation in insurance premiums is explained
by years of driving experience and 41% is not.

6. Using the regression line, we can predict the monthly auto insurance premium for a driver with x = 10 years of
driving experience as
ŷ = 76.66 − 1.55x = 76.66 − (1.55)(10) = $61.18

166
 7. The estimated value of standard deviation is σ̂ = 10.3199. Thus, the standard errors are s.e.( β̂ 0 ) = 6.961 and
s.e.( β̂ 1 ) = 0.52698.
8. A normal probability plot of the auto insurance data is provided in Figure 6.7. The plot is roughly linear and all
the data values lie within the bounds of the normal probability plot, indicating that the data are roughly normal.

0.95

0.90

0.75
Probability

0.50

0.25

0.10

0.05

−10 −5 0 5 10
Residuals

F IGURE 6.7: Normal probability plot of the residuals for the auto insurance data.

9. The residual plot for the auto insurance data, shown in Figure 6.8, indicate that the points seem to be randomly
dispersed around zero. That is, a linear regression model is appropriate for the data. Thus, the residual plot does
not suggest violations of the assumptions of zero means and same variance of the random errors.

15

10

5
e = y − ŷ

−5

−10

−15
35 40 45 50 55 60 65 70 75
ŷ

F IGURE 6.8: Residual plot for the auto insurance data.

10. 100(1 − α)% = 90% implies that α = 0.10. Thus, the 90% confidence interval for the slope β 1 is
( β̂ 1 ± tα/2,n−2 s.e.( β̂ 1 ) = (−2.57, −0.52)
Thus, −2.571 ≤ β 1 ≤ −0.523. That is, on average, the monthly auto insurance premium of a driver decreases by
an amount between $0.52 and $2.57 for every extra year of driving experience.

167
11. Since the residuals are normally distributed with constant error variance, we can perform statistical inference on
the regression. We are testing whether β 1 is negative at a significance level α = 0.05.

Step 1: We test the hypotheses (lower-tailed)

H0 : β1 = 0
H1 : β1 < 0

Step 2: The value of the t-test is given by

β̂ 1
t0 = = −2.93671
s.e.( β̂ 1 )

Step 3: The rejection region is t0 < −tα,n−2 , where tα,n−2 = t0.05,6 = 1.94318.
Step 4: Since t0 = −2.93671 < −tα,n−2 = −1.94318, we reject the null hypothesis H0 .

Thus, we reject the null hypothesis H0 : β 1 = 0 and conclude that β 1 < 0. That is, the monthly auto insurance
premium decreases with an increase in years of driving experience.

12. The t test (concerning β 1 ) for a significant regression is two-tailed. We are testing for significance of regression at
a significance level α = 0.05.

Step 1: We test the hypotheses (two-tailed)

H0 : β1 = 0
H1 : β 1 6= 0

Step 2: The value of the t-test is given by

β̂ 1
t0 = = −2.93671
s.e.( β̂ 1 )

Step 3: The rejection region is |t0 | > tα/2,n−2, where tα/2,n−2 = t0.025,6 = 2.446912.
Step 4: Since |t0 | = 2.93671 > tα/2,n−2 = 2.446912, we reject the null hypothesis H0 .

We conclude that there is evidence to suggest that β 1 6= 0, and the regression is therefore significant.

13. The ANOVA for testing significance of linear regression for the auto insurance data is summarized in Table 6.6.

TABLE 6.6: ANOVA table for simple linear regression using auto insurance data.
Source of Variation Sum of Squares Degrees of Freedom Mean Square F0
Regression 918.493 1 918.493 8.624
Error 639.007 6 106.501
Total 1557.500 7

We want to test the significance of regression using ANOVA with α = 0.05.

Step 1: We test the hypotheses

H0 : There is no significant linear relationship

H1 : There is a significant linear relationship

or equivalently

H0 : β1 = 0
H1 : β 1 6= 0

168
 Step 2: The value of the test statistic is given by

MSR
f0 = = 8.624
MSE

Step 3: The rejection region is f 0 > f α,1,n−2, where f α,1,n−2 = f 0.05,1,6 = 5.9874.
Step 4: Since f 0 = 8.624 > f α,1,n−2 = 5.9874, we reject the null hypothesis H0 .

Thus, we conclude that the monthly auto insurance premium is linearly related to driving experience.

6.2 M ULTIPLE R EGRESSION

A multiple regression model is a probabilistic model that includes more than one independent variable. It is valuable
for quantifying the impact of various simultaneous influences upon a single dependent variable. This section provides
a brief introduction to multiple regression analysis. In multiple regression analysis, we assume that each observed
value yi of the response variable Yi can be described as follows:

Multiple Linear Regression Model:

yi = β 0 + β 1 xi1 + β 2 xi2 + . . . + β k xik + ε i , i = 1, . . . , n (6.2.1)

where
• yi is the dependent variable
• x1 , x2 , . . . , x k are the independent variables
• β 0 , β 1 , . . . , β k are unknown parameters
• ε i ∼ N (0, σ2) is a random error with unknown variance

Thus, the multiple regression model uses two or more independent variables (x i ) to predict the value of a dependent
variable (yi ). The observed data for the multiple regression model can be represented by the format shown in Table 6.7,
where xij is the j-th observation or level of variable x j .

TABLE 6.7: Data for Multiple Linear Regression.

y x1 x2 ... xk
y1 x11 x12 ... x1k
y2 x21 x22 ... x2k
.. .. .. .. ..
. . . . .
yn xn1 xn2 ... xnk

The multiple regression model can be written in matrix form as

y = Xβ + ε (6.2.2)

where        
y1 1 x11 x12 ... x1k β0 ε1
 y2   1 x21 x22 ... x2k   β1   ε2 
       
y= ..  X= .. .. .. ..  β= ..  ε= .. 
 .   . . . .   .   . 
yn 1 xn1 xn2 ... x nk βk εn

Denote by X ′ the transpose of X, and ( X ′ X )−1 the inverse of X ′ X. Then, the vector β of unknown parameters can be
estimated using the least-square estimation:

169
Least-squares estimates:
Regression parameters: β̂ = ( X ′ X )−1 X ′ y

Regression line: ŷ = X β̂

Note that X ′ X is a matrix of size (k + 1) × (k + 1).

Example 6.12 A study was performed on the systolic blood pressure (SBP) y and its relationship to x1 = weight in pounds and
x2 = age in a class of males of approximately the same height. From 13 subjects preselected according to weight and age, the data
in Table 6.8 were obtained.

TABLE 6.8: Systolic blood pressure data.

Systolic blood pressure Weight Age
y x1 x2
120 152 50
141 183 20
124 171 20
126 165 30
117 158 30
129 161 50
123 149 60
125 158 50
132 170 40
123 153 55
132 164 40
155 190 40
147 185 20

1. Compute the least-squares estimate β̂

2. Find the regression equation and plot it on the scatter plot
3. Predict the systolic blood pressure of a 37-year old male who weighs 180 pounds.
Solution: We treat the weight and age as the predictor variables, x1 and x2 , and systolic blood pressure as the response
variable, y. The number of observations is n = 13. Thus, the multiple linear regression model is
yi = β 0 + β 1 xi1 + β 2 xi2 + ε i , i = 1, . . . , 13
MATLAB code
>> y = [120 141 124 126 117 129 123 125 132 123 132 155 147]’;
>> x1 = [152 183 171 165 158 161 149 158 170 153 164 190 185]’;
>> x2 = [50 20 20 30 30 50 60 50 40 55 40 40 20]’;
>> X = [ones(size(x1)) x1 x2];
>> b = regress(y,X)
b =
-65.0997
1.0771
0.4254
>> beta0hat = b(1); beta1hat = b(2); beta2hat = b(3);
>> x1fit = min(x1):3:max(x1);
>> x2fit = min(x2):3:max(x2);
>> [X1FIT,X2FIT] = meshgrid(x1fit,x2fit);
>> YFIT = beta0hat + beta1hat*X1FIT + beta2hat*X2FIT;
>> surf(X1FIT,X2FIT,YFIT);
>> hold on; scatter3(x1,x2,y,’filled’)

170
 1. The returned values are β̂ 0 = −65.0997 and β̂ 1 = 1.0771, and β̂ 2 = 0.4254. That is, the least-squares estimate β̂ is
given by  
−65.0997
β̂ =  1.0771 
0.4254
The value of the estimated intercept, β̂ 0 , represents the mean value of y when both x1 and x2 equal 0. Because the
systolic blood pressure cannot be less than 0, the intercept has no practical interpretation. The value of β̂ 1 means
that for each increase of 1 unit in x1 , the mean value of y is estimated to increase by 1.0771, holding constant the
effect of x2 . In other words, holding constant the age of a person, for each increase of 1 pound in the weight, the
fitted model predicts that the expected systolic blood pressure is estimated to increase by 1.0771. Similarly, the
value of β̂ 2 means that for each increase of 1 unit in x2 , the mean value of y is estimated to increase by 0.4254,
holding constant the effect of x1 . In other words, holding constant the weight of a person, for each increase of
1 year in the age, the fitted model predicts that the expected systolic blood pressure is estimated to increase by
0.4254.
2. The fitted regression equation is given by ŷ = −65.0997 + 1.0771x1 + 0.4254x2, and it describes a plane in the 3D
space of y, x1 , and x2 as shown in Figure 6.9(a). Note that the predicted values lie on the plane. The contour plot
of the regression model is also shown in Figure 6.9(b). Notice that the contour lines in this plot are straight lines.
This fitted regression equation can be used to predict systolic blood pressure for pairs of values of the regressor
variables weight (x1 ) and age (x2 ).
3. For x1 = 180 and x2 = 37, the fitted value is

ŷ = −65.0997 + 1.0771x1 + 0.4254x2

= −65.0997 + 1.0771(180) + 0.4254(37)
≈ 145

55

50

45

40
x2

35

30

25

20
150 155 160 165 170 175 180 185
x1
(a) (b)
F IGURE 6.9: Systolic blood pressure data: (a) Scatter plot and regression plane; (b)
Contour plot.

6.2.1 S CATTERPLOT M ATRIX

A scatterplot matrix is a graphical representation for visualizing the multivariable data in Table 6.8 as a matrix of
two-dimensional scatter plots. The scatterplot matrix displays scatterplots of all possible pairs of variables or columns
of the matrix data. Histograms of each variable or column are shown along the diagonal of the scatterplot matrix.
Figure 6.10 shows the scatterplot matrix of the systolic blood pressure data, where each of the three variables taking
the role of x-variable and y-variable against all the others. For example, the top panel (row) displays from left to right
the individuals scatterplots of y against x1 and y against x2 , respectively.

171
 160 160

140 140
SBP
120 120

100 100

200 200

180 180
Weight
160 160

140 140

60 60
50 50
40 40 Age
30 30
20 20
100 120 140 160 140 160 180 200 20 40 60 100 120 140 160 140 160 180 200 20 40 60

F IGURE 6.10: Scatterplot matrix of the systolic blood pressure data.

MATLAB code
>> y = [120 141 124 126 117 129 123 125 132 123 132 155 147]’;
>> x1 = [152 183 171 165 158 161 149 158 170 153 164 190 185]’;
>> x2 = [50 20 20 30 30 50 60 50 40 55 40 40 20]’;
>> data = [y x1 x2];
>> plotmatrix(data);

6.2.2 R ESIDUAL A NALYSIS

In order to check the regression model assumptions, we need to compute and examine the normality of the residuals
ei = yi − ŷi , where yi is the observed value and ŷi is the predicted (fitted) value. Thus, the n × 1 vector of residuals is
given by
e = y − ŷ = y − X β̂ (6.2.3)
The normal probability and residual plots for the the systolic blood pressure data are shown in Figure 6.11, which indi-
cate that the residuals are normally distributed with constant error variance. Thus, the regression model assumptions
are valid. Notice the relatively linear appearance of the normal probability plot and the relative scatter of the residuals
versus fitted values.

6.2.3 H YPOTHESIS T ESTS AND A NALYSIS OF VARIANCE IN M ULTIPLE L INEAR R EGRESSION

The test for significance of regression is a test to determine whether a linear relationship exists between the response
variable y and a subset of the regressor variables x1 , x2 , . . . , x k . The appropriate hypotheses are

H0 : β 1 = β 2 = · · · = β k = 0
(6.2.4)
H1 : At least one β i 6= 0

Rejection of H0 : β 1 = β 1 = · · · = β k = 0 implies that at least one of the regressor variables x1 , x2 , . . . , x k contributes

significantly to the model.
The test for significance of regression is a generalization of the procedure used in simple linear regression. The total
variability (i.e. total sum of squares) SST = ∑ni=1 (yi − ȳ)2 in the dependent variable y can be partitioned into the
variability explained by the regression line SSR (sum of squares due to regression) and the variability SSE (sum of
squares due to error) about the regression line as follows:

SST = SSR + SSE (6.2.5)

172
 0.98 5

0.95 4

0.90
3

0.75 2
Probability

e = y − ŷ
1
0.50
0

0.25 −1

−2
0.10

0.05 −3

0.02 −4
−3 −2 −1 0 1 2 3 4 115 120 125 130 135 140 145 150 155 160
Residuals ŷ
(a) (b)
F IGURE 6.11: Systolic blood pressure data: (a) Normal probability plot; (b) Residual
plot.

where
n n n
′
SST = ∑ (yi − ȳ)2 = y′ y − nȳ2, SSR = ∑ (ŷi − ȳ)2 = β̂ X ′ y − nȳ2 , SSE = ∑ e2i = e′ e
i =1 i =1 i =1

If the null hypothesis H0 : β 1 = β 1 = · · · = β k = 0 is true, then SSR /σ2 ∼ χ2 ( k ) and SSE /σ2 ∼ χ2 (n − k − 1).
Moreover, the test statistic is given by

SSR /k MSR
F0 = = ∼ F (k, n − k − 1) (6.2.6)
SSE /(n − k − 1) MSE

where MSR = SSR /k and MSE = SSE /(n − k − 1) are the mean square regression and mean square error, respectively.
Let f 0 is the numerical value, calculated from the data, of the test statistic F0 and f α,k,n−k−1 is the upper-percentage
points of the F-distribution F (k, n − k − 1). If f 0 > f α,k,n−k−1 then H0 : β 1 = β 1 = · · · = β k = 0 would be rejected.
It can be shown that an unbiased estimator of the variance σ2 of the error term in the multiple linear regression model
is given by

SSE
σ̂2 = = MSE (6.2.7)
n−k−1

Notice that the estimate of σ2 has n − k − 1 = n − (k + 1) degrees of freedom (k + 1 is subtracted from n because
k + 1 unknown parameters, β 0 , β 1 , . . . , β k , are estimated).
The analysis of variance (ANOVA) for testing significance of multiple regression is usually represented in tabular
form (ANOVA table), as shown in Table 6.9.

TABLE 6.9: Analysis of Variance for Testing Significance of Regression in Multiple

Linear Regression.
Source of Variation Sum of Squares Degrees of Freedom Mean Square F0
MSR
Regression SSR k MSR
MSE
Error SSE n−k−1 MSE
Total SST n−1

173
Example 6.13 Using the data of Table 6.8, test for significance of regression using the analysis of variance procedure with the
level of significance α = 0.05.
Solution: The ANOVA for testing significance of multiple regression for the systolic blood pressure data is summarized
in Table 6.10. We want to test the significance of regression with α = 0.05.

TABLE 6.10: ANOVA table for the systolic blood pressure data.
Source of Variation Sum of Squares Degrees of Freedom Mean Square F0
Regression 1423.838 2 711.919 113.126
Error 62.931 10 6.293
Total 1486.769 12

Step 1: We test the hypotheses

H0 : β1 = β2 = 0
H1 : At least one β i 6= 0

Step 2: The value of the test statistic is given by

MSR
f0 = = 113.126
MSE

Step 3: The rejection region is f 0 > f α,k,n−k−1, where f α,k,n−k−1 = f 0.05,2,10 = 4.1028.
Step 4: Since f 0 = 113.126 > f α,k,n−k−1 = 4.1028, we reject the null hypothesis H0 .
Thus, we conclude that systolic blood pressure is linearly related to either weight or age, or both.

Table 6.10 shows that the estimate of σ2 for the systolic blood pressure regression model is σ̂2 = MSE = 6.293. Also,
the coefficient of determination R2 for the systolic blood pressure data is given by

SSR SS
R2 = = 1 − E = 0.958
SST SST
Thus, 95.8% of the total variation has been explained by the regression model.
It is worth pointing out that the F-statistic is related to the coefficient of determination R2 by the formula:

R2 /k
F0 = (6.2.8)
(1 − R2 ) / ( n − k − 1 )

Therefore, R2 is large implies F0 is large, and vice versa.

6.3 A NALYSIS OF VARIANCE

In Chapter 4 we used sampling theory to test the significance of differences µ1 − µ2 between two sampling means µ1
and µ2 . In many situations there is a need to test the significance of differences between more than two sampling means
or, equivalently, to test the null hypothesis that more that two sample means are equal. Analysis of Variance (ANOVA)
is a statistical method that allows us to analyze and interpret observations from several populations by partitioning the
total variation in a data set according to the sources of variation that are present.
One-way analysis of variance (one-way ANOVA) is a technique used to compare means of two or more groups of
data using the F distribution. This technique can be used only for numerical data. The one-way ANOVA tests the
null hypothesis that samples in two or more groups are drawn from the same population, and it is conducted on an
experimental design in which there is only one factor (independent variable). The purpose of one-way ANOVA is to
find out whether data from several groups have a common mean. That is, to determine whether the groups are actually
different in the measured characteristic.

174
6.3.1 O NE -WAY ANOVA: C OMPLETELY R ANDOMIZED D ESIGN
In this section we consider a one-factor completely randomized design (experiment), where observations or measure-
ments are obtained for a independent random samples (called treatments or groups) of n measurements. By “com-
pletely randomized”, we mean that the participants have been randomly assigned to one of the unique levels of the
factor. The observed data for the response measurements can be represented by the format shown in Table 6.11, where
yij is the j-th observation on treatment i. The summary statistics
n
yi. = ∑ yij i = 1, . . . , a (Total of the observations under the i-th treatment)
j =1
ȳi. = yi. /n (Mean of the observations under the i-th treatment.)
a n
y.. = ∑ ∑ yij (Grand total of all observations)
i =1 j =1
ȳ.. = y.. /an (Grand mean of all observations)

also appear in the last two columns of the table.

TABLE 6.11: Typical Data for a Single-Factor Experiment.

Treatment Observations Totals Averages
1 y11 y12 . . . y1n y1. ȳ1.
2 y21 y22 . . . y2n y2. ȳ2.
.. .. .. .. .. .. ..
. . . . . . .
a y a1 y a2 ... y an y a. ȳ a.
y.. ȳ..

6.3.2 M ODEL FOR O NE -WAY ANOVA

To implement a formal statistical test for no difference among treatment effects, we need to have a population model
for the experiment. To this end, we assume that the response measurements with the ith treatment constitute a random
sample from a normal population N (µi , σ2 ) with a mean µi and a common variance σ2 . The samples are assumed to
be mutually independent. Thus, comparing a treatments can be described by the statistical model
(
i = 1, 2, . . . , a
Yij = µi + ε ij (6.3.1)
j = 1, 2, . . . , n

or, equivalently, (
i = 1, 2, . . . , a
Yij = µ + τi + ε ij (6.3.2)
j = 1, 2, . . . , n
where µi = µ + τi is the mean of the i-th treatment, µ is the overall mean common to all treatments, and τi is the i-th
treatment effect such that ∑ia=1 τi = 0. The errors ε ij ∼ N (0, σ2) are i.i.d., and Yij is a random variable denoting the
(i, j)-th observation yij . It can be shown that the least squares estimate of µi is µ̂i = ȳi. , which implies that the estimated
or fitted value of yij is ŷij = ȳi. . Thus, the estimate of the i-th treatment effect is given by

τ̂i = µ̂i − µ̂ = ȳi. − ȳ.. for i = 1, 2, . . . , a (6.3.3)

In other words, τ̂i is the difference between the mean of the observations under the i-th treatment and the grand mean
of all observations.

6.3.3 R ESIDUAL A NALYSIS

In order to check the regression model assumptions, we need to compute and examine the residuals

175
 eij = yij − ŷij = yij − ȳi. (6.3.4)

where yij is the observed value and ȳi. is the predicted (fitted) value. The normal probability and residual plots are
used to examine the assumptions underlying the ANOVA, and will also show whether there are any serious outliers.
The normality assumption of the errors ε ij can be checked by constructing a normal probability plot of the residuals,
while the assumption of equal variances of the errors ε ij at each factor level can be checked by plotting the residuals
eij against the factor levels and compare the spread in the residuals. It is also useful to plot the residuals eij against the
fitted values ŷij = ȳi..

• Normal probability plot: if the observations come from a normal distribution, then the points in the plot fall along
a straight line. If the scatter plot is nonlinear, then there is evidence to suggest that the data did not come from a
normal distribution.

• Residual plot: if the fitted model is appropriate for the data then there will be no apparent pattern in the plot; that
is there will only be random variation around 0. However, if the fitted model is not appropriate, there will be a
clear relationship between the factor levels or the fitted values and the residuals.

6.3.4 H YPOTHESES IN O NE -WAY ANOVA

The one-way ANOVA evaluates the hypothesis that the samples all have the same mean against the alternative that
the means are not all the same. It is assumed that the a populations are independent and normally distributed with
means µ1 , µ2 , . . . , µ a and common variance σ2 . The null hypothesis that no difference exists among the a means µi and
the alternative hypothesis is that not all the µi ’s are equal can be formulated as a one-sided hypothesis test:

H0 : µ1 = µ2 = . . . = µ a = µ
(6.3.5)
H1 : At least one µi 6= µ

or, equivalently,

H0 : τ1 = τ2 = . . . = τa = 0
(6.3.6)
H1 : At least one τi 6= 0

If H0 is true, then the observations will all have the same normal distribution N (µ, σ2 ), implying that there is no
significant difference between the treatments.

6.3.5 T OTAL VARIABILITY D ECOMPOSITION

The total variability in the data is described by the total sum of squares
a n
SST = ∑ ∑ (yij − ȳ.. )2 (6.3.7)
i =1 j =1

and can be partitioned, using ANOVA, into the within-treatments variability SSE and the between-treatments variabil-
ity SSTreatments , as follows:
a n a n a
∑ ∑ (yij − ȳ.. )2 = ∑ ∑ (yij − ȳi. )2 + n ∑ (ȳi. − ȳ.. )2 (6.3.8)
i =1 j =1 i =1 j =1 i =1
| {z } | {z } | {z }
Total (SS T ) Within (SS E ) Between (SSTreatments )

Note that SST has an − 1 degrees of freedom and SSTreatments has a − 1 degrees of freedom. Thus, the error sum of squares
SSE has a(n − 1). That is,

SST = SSE + SSTreatments (6.3.9)

|{z} |{z} | {z }
an −1 degrees of freedom a ( n −1) degrees of freedom a −1 degrees of freedom

The sums of squares SST , SSTreatments , and SSE can be computed efficiently using the following simplified formulas:

176
 a n
y2
SST = ∑ ∑ y2ij − an..
i =1 j =1
a y2i. y2 (6.3.10)
SSTreatments = ∑ − ..
i =1
n an
SSE = SST − SSTreatments

Note that in practice, the sums of squares SST and SSTreatments should be calculated or computed before the error sum of
squares SSE .

6.3.6 E XPECTED VALUES OF THE VARIABILITIES AND T ESTING E QUALITY OF M EANS

It can be shown that the expected values of SSTreatments and SSE are given by:

• E(SSTreatments ) = ( a − 1)σ2 + n ∑ia=1 τi2

• E(SSE ) = a(n − 1)σ2

Moreover,

SSE
• The error mean square MSE = is an unbiased estimator of σ2
a ( n − 1)

• If H0 is true, then the mean square for treatments MSTreatments = SSTreatments /( a − 1) is an unbiased estimator of σ2
SSTreatments /( a − 1) MSTreatments
• If H0 is true, then F0 = = ∼ F ( a − 1, a(n − 1))
SSE /a(n − 1) MSE

It is customary to present the decomposition of the sum of squares and the degrees of freedom in a tabular form
(ANOVA table), as shown in Table 6.12. It is important to point out that at an α-significance level, we would reject H0
if f 0 > f α,a−1,a( n−1), where f 0 is the numerical value, calculated from the data, of the test statistic F0 and f α,a−1,a( n−1)
is the upper-percentage points of the distribution F ( a − 1, a(n − 1)). The computed value of f 0 is usually presented in
the last column of the ANOVA table. We can also reject H0 if p-value = 1 − F ( f 0 ) < α, where F (·) is the cdf of the
F-distribution F ( a − 1, a(n − 1)).

TABLE 6.12: ANOVA table for one-factor experiment.

Source of Variation Sum of Squares Degrees of Freedom Mean Square F0
MSTreatments
Treatments SSTreatments a−1 MSTreatments
MSE
Error SSE a ( n − 1) MSE
Total SST an − 1

Example 6.14 A manufacturer of paper used for making grocery bags is interested in improving the tensile strength of the product.
Product engineering thinks that tensile strength is a function of the hardwood concentration in the pulp and that the range of
hardwood concentrations of practical interest is between 5 and 20%. A team of engineers responsible for the study decides to
investigate four levels of hardwood concentration: 5%, 10%, 15%, and 20%. They decide to make up six test specimens at each
concentration level, using a pilot plant. All 24 specimens are tested on a laboratory tensile tester, in random order. The data from
this experiment are shown in Table 6.13.

177
 (i) Calculate the totals and averages for each treatment (i.e. each hardwood concentration level).
(ii) Estimate the treatment effects for the four hardwood concentration levels.
(iii) Construct the ANOVA table.
(iv) Is there sufficient evidence to indicate a difference in hardwood concentrations affect the mean tensile strength of the paper?
Test using α = 0.01.
(v) Analyze the residuals and comment on model adequacy.

TABLE 6.13: Tensile Strength of Paper (psi).

Hardwood Observations
Concentration 1 2 3 4 5 6
5% 7 8 15 11 9 10
10% 12 17 13 18 19 15
15% 14 18 19 17 16 18
20% 19 25 22 23 18 20

Solution: The experiment involves a single factor, hardwood concentration, at four levels. Thus, we have a completely
randomized design with a = 4 treatments.
(i) The totals and averages for each treatment (i.e. hardwood concentration) are shown in Table 6.14, which presents
the data for the tensile strength of paper (psi) with totals and averages. From the four sample means, we observe
that the mean for the 20% hardwood concentration level is considerably larger than the means for the other
hardwood concentration levels. This can also be observed in the side-by-side boxplots in Figure 6.12. Thus, the
team of engineers need to determine whether the four population means differ.

TABLE 6.14: Tensile Strength of Paper (psi) with totals and averages.
Hardwood Observations
Concentration 1 2 3 4 5 6 Totals (yi. ) Averages (ȳi.)
5% 7 8 15 11 9 10 60 10.00
10% 12 17 13 18 19 15 94 15.67
15% 14 18 19 17 16 18 102 17.00
20% 19 25 22 23 18 20 127 21.17
y.. = 383 ȳ.. = 15.96

(ii) The estimate of the i-th treatment effect is τ̂i = ȳi. − ȳ.. for i = 1, 2, . . . , a. Thus,
τ̂1 = 10 − 15.96 = −5.96; τ̂2 = 15.67 − 15.96 = −0.29; τ̂3 = 17 − 15.96 = 1.04; τ̂4 = 21.17 − 15.96 = 5.21.

(iii) The output results of ANOVA for the tensile strength data are exhibited in Table 6.15.

1 data = [ 7 8 15 11 9 10
2 12 17 13 18 19 15
3 14 18 19 17 16 18
4 19 25 22 23 18 2 0 ] ;
5 [ p , t a b l e , s t a t s ] = anova1 ( data ' ) ; % Performs a one−way ANOVA

(iv) Let µ1 , µ2 , µ3 , and µ4 represent the mean tensile strengths for hardwood concentration levels 5%, 10%, 15%, and
20%, respectively. We want to test the equality of the 4 treatment means µ1 , µ2 , µ3 , µ4 with α = 0.01.
Step 1: We test the hypotheses
H0 : µ1 = µ2 = µ3 = µ4 = µ
H1 : At least one µi 6= µ
where µ is the overall mean common to all treatments.

178
 24

22

20

Tensile strength (psi)

18

16

14

12

10

5% 10% 15% 20%

Hardwood concentration

F IGURE 6.12: Boxplots for the tensile strength data.

TABLE 6.15: ANOVA table for the tensile strength data.

Step 2: The value of the test statistic is given by

MSTreatments
f0 = = 19.61
MSE

Step 3: The rejection region is f 0 > f α,a−1,a( n−1), where f α,a−1,a( n−1) = f 0.01,3,20 = 4.9382.
Step 4: Since f 0 = 19.61 > f α,a−1,a( n−1) = 4.9382, we reject the null hypothesis H0 .
Thus, we reject the null hypothesis of equality of the mean tensile strengths for the four hardwood concentration
levels of treatment, and we conclude that the hardwood concentration in the pulp significantly affects the mean
strength of the paper. We can arrive at the same conclusion by noting that p-value = 1 − F ( f 0 ) = 1 − F (19.61) =
3.59258 × 10−6 < α = 0.01, where F (·) is the cdf of the F-distribution F (3, 20).
(v) The calculated residual values for the tensile strength experiment are shown in Table 6.16, and the residual plots
are depicted in Figure 6.13. The data of normal probability plot appear fairly linear, suggesting that no reason
to doubt the normality assumption. The residual plots against the factor levels and fitted values exhibit random
scatter around 0.

1 >> data = [ 7 8 15 11 9 10
2 12 17 13 18 19 15
3 14 18 19 17 16 18
4 19 25 22 23 18 2 0 ] ;
5 >> r e s = bsxfun ( @minus , data , mean ( data , 2 ) ) ; %r e s i d u a l s
6 >> n o r m p l o t ( r e s ( : ) ) ;
7 >> f i g u r e ;
8 >> l e v e l s = [ 5 10 15 2 0 ] ' ;
9 >> x1 = repmat ( l e v e l s , n , 1 ) ; y = r e s ( : ) ;
10 >> s c a t t e r ( x1 , y , 3 , ' ko ' , ' MarkerFaceColor ' , [ .4 9 1 .6 3 ] ) ;
11 >> h o l d on ; h = r e f l i n e ( 0 ) ; s e t ( h , ' l i n e s t y l e ' , '− ' , ' c o l o r ' , ' r ' , ' LineWidth ' , 1 ) ;
12 >> h o l d o f f ; g r i d o f f ;
13 >> f i g u r e ;
14 >> y i b a r = mean ( data , 2 ) ; x2 = repmat ( y i b a r , n , 1 ) ;

179
 15 >> s c a t t e r ( x2 , y , 3 , ' ko ' , ' MarkerFaceColor ' , [ .4 9 1 .6 3 ] ) ;
16 >> h o l d on ; h = r e f l i n e ( 0 ) ; s e t ( h , ' l i n e s t y l e ' , '− ' , ' c o l o r ' , ' r ' , ' LineWidth ' , 1 ) ;
17 >> h o l d o f f ; g r i d o f f ;

TABLE 6.16: Residual values for the tensile strength experiment.

Hardwood Concentration Residuals
5% -3.00 -2.00 5.00 1.00 -1.00 0.00
10% -3.67 1.33 -2.67 2.33 3.33 -0.67
15% -3.00 1.00 2.00 0.00 -1.00 1.00
20% -2.17 3.83 0.83 1.83 -3.17 -1.17

0.98

0.95 4

0.90
2
0.75

Residuals
Probability

0
0.50

0.25 −2

0.10
−4
0.05

0.02
−6
−3 −2 −1 0 1 2 3 4 5 5 10 15 20
Residuals Factor levels
(a) (b)

2
Residuals

−2

−4

−6
10 12 14 16 18 20 22
ŷ

(c)
F IGURE 6.13: Residual analysis of the tensile strength experiment.

Example 6.15 An experiment was conducted to compare the wearing qualities of three types of paints when subjected to the
abrasive action of a slowly rotating cloth-surfaced wheel. Ten paint specimens were tested for each paint type, and the number of
hours until visible abrasion was apparent was recorder for each specimen. The data are shown in Table 6.17.

(i) Is there sufficient evidence to indicate a difference in mean time until abrasion is visibly evident for the three paint types?
Test using α = 0.05.

180
 (ii) Analyze the residuals and comment on model adequacy.

TABLE 6.17: Data for the paint experiment.

Paint Observations
Type 1 2 3 4 5 6 7 8 9 10
1 148 76 393 520 236 134 55 166 415 153
2 513 264 433 94 535 327 214 135 280 304
3 335 643 216 536 128 723 258 380 594 465

Solution: The experiment involves a single factor, paint type, at three levels. Thus, we have a completely randomized
design with a = 3 treatments. Table 6.18 displays the data for the paint experiment with totals and averages.

TABLE 6.18: Data for the paint experiment with totals and averages.
Paint Observations
Type 1 2 3 4 5 6 7 8 9 10 Totals (yi.) Averages (ȳi.)
1 148 76 393 520 236 134 55 166 415 153 2296 229.60
2 513 264 433 94 535 327 214 135 280 304 3099 309.90
3 335 643 216 536 128 723 258 380 594 465 4278 427.80
y.. = 9673 ȳ.. = 322.43

The output results of ANOVA for the paint data are presented in Table 6.19.
MATLAB code
data = [148 76 393 520 236 134 55 166 415 153
513 264 433 94 535 327 214 135 280 304
335 643 216 536 128 723 258 380 594 465];
[p,table,stats] = anova1(data’); % Performs a one-way ANOVA

TABLE 6.19: ANOVA for the paint type experiment.

(i) Let µ1 , µ2 , and µ3 represent the mean abrasion times for paint types 1, 2, and 3, respectively.
We want to test the equality of the 3 treatment means µ1 , µ2 , µ3 with α = 0.05.

Step 1: We test the hypotheses

H0 : µ1 = µ2 = µ3 = µ
H1 : At least one µi 6= µ

where µ is the overall mean common to all treatments.

Step 2: The value of the test statistic is given by

MSTreatments
f0 = = 3.48
MSE

Step 3: The rejection region is f 0 > f α,a−1,a( n−1), where f α,a−1,a( n−1) = f 0.05,2,27 = 3.35413.

181
 Step 4: Since f 0 = 3.48 > f α,a−1,a( n−1) = 3.35413, we reject the null hypothesis H0 .

Thus, we conclude that the mean time to visible abrasion differs for at least two of the three paint types. We can
arrive at the same conclusion by noting that p-value = 1 − F (3.48) = 0.0452 < α = 0.05.

(ii) The calculated residual values for the paint type experiment are shown in Table 6.20. Figure 6.14 displays the
residual plots for the paint type experiment. These plots do not reveal any model inadequacy or unusual problem
with the assumptions.

TABLE 6.20: Residual values for the paint experiment.

Paint Residuals
Type
1 -81.60 -153.60 163.40 290.40 6.40 -95.60 -174.60 -63.60 185.40 -76.60
2 203.10 -45.90 123.10 -215.90 225.10 17.10 -95.90 -174.90 -29.90 -5.90
3 -92.80 215.20 -211.80 108.20 -299.80 295.20 -169.80 -47.80 166.20 37.20

300
0.99
0.98

0.95 200

0.90
100
0.75 Residuals
Probability

0
0.50

0.25 −100

0.10
−200
0.05

0.02
0.01 −300
−300 −200 −100 0 100 200 300 1 2 3
Residuals Factor levels
(a) (b)

300

200

100
Residuals

−100

−200

−300
200 250 300 350 400 450
ŷ

(c)
F IGURE 6.14: Residual analysis of the paint type experiment.

Example 6.16 Suppose in an industrial experiment that an engineer is interested in how the mean absorption of moisture in
concrete varies among 5 different concrete aggregates. The samples are exposed to moisture for 48 hours. It is decided that 6

182
samples are to be tested for each aggregate, requiring a total of 30 samples to be tested. The data from this experiment are presented
in Table 6.21.

(i) Is there sufficient evidence to indicate a difference in concrete aggregate affect the mean absorption of moisture in concrete?
Draw comparative boxplots and perform an analysis of variance. Test using α = 0.05.

(ii) Analyze the residuals and comment on model adequacy.

TABLE 6.21: Absorption of Moisture in Concrete Aggregates.

Aggregate Absorption of Moisture
1 551 457 450 731 499 632
2 595 580 508 583 633 517
3 639 615 511 573 648 677
4 417 449 517 438 415 555
5 563 631 522 613 656 679

Solution: The experiment involves a single factor, aggregate, at five levels. Thus, we have a completely randomized
design with a = 5 treatments. A box plot for each aggregate is shown in Figure 6.15. From these side-by-side box plots
it is evident that the absorption is not the same for all aggregates. In fact, it appears as if aggregate 4 stands out from
the rest. The output results of ANOVA for the absorption of moisture data are presented in Table 6.22.
MATLAB code
>> data = [551 457 450 731 499 632
595 580 508 583 633 517
639 615 511 573 648 677
417 449 517 438 415 555
563 631 522 613 656 679];
>> [a,n] = size(data);
>> boxplot(data’);
>> [p,table,stats] = anova1(data’);

700

650
Absorption of Moisture

600

550

500

450

400

1 2 3 4 5
Aggregate

F IGURE 6.15: Box plots for the absorption of moisture in concrete aggregates.

(i) Let µ1 , µ2 , µ3 , and µ4 represent the mean absorption of moisture in concrete for aggregate 1, 2, 3, 4, 5, respectively.

183
 TABLE 6.22: ANOVA for the absorption of moisture data.

We want to test the equality of the 5 treatment means µ1 , µ2 , µ3 , µ4 , µ5 with α = 0.05. From the given information,
we have a = 5 and n = 6.
Step 1: We test the hypotheses

H0 : µ1 = µ2 = µ3 = µ4 = µ5 = µ
H1 : At least one µi 6= µ

where µ is the overall mean common to all treatments.

Step 2: The value of the test statistic is given by

MSTreatments
f0 = = 4.3
MSE

Step 3: The rejection region is f 0 > f α,a−1,a( n−1), where f α,a−1,a( n−1) = f 0.01,4,25 = 4.17742.
Step 4: Since f 0 = 4.3 > f α,a−1,a( n−1) = 4.17742, we reject the null hypothesis H0 .
Thus, we conclude that the aggregare in concrete significantly affects the absorption of moisture in concrete. We
can arrive at the same conclusion by noting that p-value = 1 − F (4.3) = 0.0088 < α = 0.05.
(ii) The residual plots are depicted in Figure 6.16. The data of normal probability plot appear fairly linear, suggesting
that no reason to doubt the normality assumption. The residual plots against the factor levels and fitted values
exhibit random scatter around 0.

184
 150
0.99
0.98
100
0.95
0.90
50
0.75

Residuals
Probability

0
0.50

0.25 −50

0.10
0.05 −100

0.02
0.01 −150
−100 −50 0 50 100 150 1 2 3 4 5
Residuals Factor levels
(a) (b)

200

150

100

50
Residuals

−50

−100

−150

−200
460 480 500 520 540 560 580 600 620
ŷ

(c)
F IGURE 6.16: Residual analysis of the absorption of moisture experiment.

185
6.4 P ROBLEMS

❶ Every Saturday, Montreal Gazette publishes in its HomeFront section a random sampler of recent real-estate
transactions for houses and condominiums in Montreal. Data collected from a sample of 15 transactions are
shown in Table 6.23, where x is the asking price (in thousand dollars) and y is the selling price (in thousand
dollars) of the property.

TABLE 6.23: Montreal-area real-estate transactions data.

Asking Price 419 239 279 430 669 399 598 339 395 294 295 575 228 170 349
Selling Price 400 200 265 399 671 393 580 315 363 290 277 520 223 165 342

a) Does the selling price depend on the asking price or does the asking price depend on the selling price? Do
you expect a positive or negative relationship between these two variables.
b) Find the least-squares regression line by choosing appropriate dependent and independent variables based
on your answer in part a).
c) Interpret the meaning of the values β̂ 0 and β̂ 1 calculated in part b).
d) Display the scatter plot and the regression line.
e) Calculate the correlation and determination coefficients, and explain what they mean.
f) Predict the selling price for a house with an asking price of $350K.
g) Calculate the standard errors
h) Display the normal probability plot. Does the normality assumption seem reasonable?
i) Display the residual plot. Does the constant variance assumption seem reasonable?
j) Find a 95% confidence interval estimate on the slope.
k) Conduct a test to determine if the true slope of the line differs from 0 using α = 0.05. What is the P-value for
this test?
l) Test at the 5% significance level whether β 1 is positive. What is the P-value for this test?

❷ How do real estate agents decide on the asking price for a newly listed condominium? A computer database in
a small community contains the listed selling price y (in thousands of dollars), the amount of living area x1 (in
hundreds of square feet), and the numbers of floors x2 , bedrooms x3 , and bathrooms x4 , for n = 15 randomly
selected condos currently on the market. The data are shown in Table 6.24.

TABLE 6.24: Data on 15 condominiums.

List Price, y Living Area, x1 Floors, x2 Bedrooms, x3 Bathrooms, x4
169.0 6 1 2 1
218.5 10 1 2 2
216.5 10 1 3 2
225.0 11 1 3 2
229.9 13 1 3 1.7
235.0 13 2 3 2.5
239.9 13 1 3 2
247.9 17 2 3 2.5
260.0 19 2 3 2
269.9 18 1 3 2
234.9 13 1 4 2
255.0 18 1 4 2
269.9 17 2 4 3
294.5 20 2 4 3
309.9 21 2 4 3

a) Fit the multiple linear regression model to these data.

b) Display the normal probability.

186
 c) Display the residual plot against the fitted values.
d) Display the residual plots against each of the independent variables.
e) Predict the selling price of a 900 square feet condo with one floor, two bedrooms and one bath.
f) Test the significance of regression using the ANOVA procedure with α = 0.05.
g) Calculate the coefficient of determination, and explain what is means.
❸ The compressive strength of concrete is being studied, and four different mixing techniques are being investi-
gated. Table 6.25 presents the collected data.
a) Calculate the totals and averages for each mixing technique.
b) Estimate the treatment effects for the four mixing techniques.
c) Does mixing technique affect compressive strength of the concrete? Draw comparative box plots and per-
form an analysis of variance. Use α = 0.05.
d) Analyze the residuals and comment on model adequacy.

TABLE 6.25: Data on compressive strength of concrete.

Mixing Technique Compressive Strength (psi)
1 3129 3000 2865 2890
2 3200 3300 2975 3150
3 2800 2900 2985 3050
4 2600 2700 2600 2765

❹ A firm wishes to compare four programs for training workers to perform a certain manual task. Twenty new
employees are randomly assigned to the training programs, with 5 in each program. At the end of the training
period, a test is conducted to see how quickly trainees can perform the task. The number of times the task is
performed per minute is recorded for each trainee, with the results presented in Table 6.26.
a) Calculate the totals and averages for each program.
b) Estimate the treatment effects for the four programs.
c) Using a 5% significance level, determine whether the treatments differ in their effectiveness.
d) Draw comparative box plots and perform an analysis of variance. Use α = 0.05.
e) Analyze the residuals and comment on model adequacy.

TABLE 6.26: Number of times the task is performed per minute.

Observations
Program 1 2 3 4 5
1 9 12 14 11 13
2 10 6 9 9 10
3 12 14 11 13 11
4 9 8 11 7 8

❺ An economist wants to test whether mean housing prices are the same regardless of which of three air-pollution
levels typically prevails. A random sample of house purchases in three areas yields the price data presented in
Table 6.27.
a) Calculate the totals and averages for each pollution level.
b) Estimate the treatment effects for the four pollution levels.
c) Using a 2.5% significance level, test whether housing prices differ by level of pollution.
d) Draw comparative box plots and perform an analysis of variance. Use α = 0.05.
e) Analyze the residuals and comment on model adequacy.
f) Draw the box plot of all the residuals and comment on your plot.

187
 TABLE 6.27: Mean housing prices (in thousands of dollars).
Pollution Observations
Level 1 2 3 4 5
Low 120 68 40 95 83
Medium 61 59 110 75 80
High 40 55 73 45 64

6.5 REFERENCES

[1] D.C. Montgomery, Introduction to Statistical Quality Control, Wiley, 6th Edition, 2009.

[2] I. Bass and B. Lawton, Lean Six Sigma using SigmaXL and Minitab, McGraw-Hill Professional, 1st Edition, 2009.

188
 C HAPTER 7

D ESIGN OF E XPERIMENTS

All life is an experiment. The more experiments

you make, the better.

Ralph Waldo Emerson

Experiments with two or more factors are encountered frequently. A factorial experiment is one that investigates the
effects of two or more independent variables (factors) on a single dependent variable (response). By a factor, we mean
a discrete variable used to classify experimental units, such as temperature, time, or pressure that may be varied from
trial to trial. Traditionally, experiments are designed to determine the effect of one variable (factor) upon one response.
Factorial design involves two or more factors in a single experiment, and can reduce the number of experiments one
has to perform by studying multiple factors simultaneously. Moreover, it can be used to find both main effects (from
each independent factor) and interaction effects (when both factors must be used to explain the outcome).

7.1 FACTORIAL E XPERIMENTS

Modeling real world phenomena often requires more than just one factor. The analysis of variance (ANOVA) can
be extended to handle the two-factor factorial experiment. Unlike one-way analysis of variance (one-way ANOVA)
tests which measure significant effects of one factor only, two-way analysis of variance (two-way ANOVA) tests (also
called two-factor ANOVA) measure the effects of two factors simultaneously. For example, an experiment might be
defined by two parameters, such as treatment and time point. One-way ANOVA tests would be able to assess only the
treatment effect or the time effect. Two-way ANOVA, on the other hand, would not only be able to assess both time
and treatment in the same test, but also whether there is an interaction between the parameters.
We consider a two-factor completely randomized factorial design, where there are two factors that each has two or
more levels. By “completely randomized”, we mean that the participants have been randomly assigned to one of the
unique levels of the factors. Let A and B be two fixed factors, with a levels of factor A and b levels of factor B. The
two factors are crossed so that there are ab treatment combinations, and a total of abn experimental units are randomly
allocated to the ab treatments with n per experiment. The n observations obtained under each treatment combination
are called replicates; that is, the experiment has n replicates, and each replicate contains all ab treatment (or factor)
combinations. This two-factor factorial design that has equal number of replicates is called balanced two-way layout,
as shown in Table 7.1, where yijk is the observation in the (i, j)-th cell for the k-th replicate, i = 1, 2, . . . , a, j = 1, 2 . . . , b,
and k = 1, 2, . . . , n.
Let Yijk be a random variable denoting the observation in the (i, j)-th cell for the k-th replicate. Thus, the observa-
tions in a completely randomized design with a two-factor factorial treatment structure and n > 1 replicates may be
described by the linear statistical model



i = 1, 2, . . . , a
yijk = µ + τi + β j + (τβ)ij + ε ijk j = 1, 2, . . . , b (7.1.1)


k = 1, 2, . . . , n

189
 TABLE 7.1: Data layout for a two-factor factorial experiment.
Factor A Factor B levels
Levels 1 2 ... j ... b
1 y111, . . . , y11n y121, . . . , y12n . . . y1j1 , . . . , y1jn ... y1b1, . . . , y1bn
2 y211, . . . , y21n y221, . . . , y22n . . . y2j1 , . . . , y2jn ... y2b1, . . . , y2bn
.. .. .. .. .. .. ..
. . . . . . .
i yi11, . . . , y i1n yi21, . . . , y i2n . . . yij1, . . . , y ijn ... yib1, . . . , y ibn
.. .. .. .. .. .. ..
. . . . . . .
a y a11, . . . , y a1n y a21, . . . , y a2n . . . y aj1 , . . . , y ajn ... y ab1, . . . , y abn

where the terms of the model are defined as follows:

• yijk is the observed (response variable) value in the (i, j)-th cell for the k-th replicate. That is, yijk is the response
from the k-th experimental unit receiving the i-th level of factor A and the j-th level of factor B
• µ is the overall mean effect (gran mean)
• τi is the effect of the i-th level of factor A
• β j is the effect of the j-th level of factor B
• (τβ)ij is the effect of the interaction between factor A and factor B

• ε ijk ∼ N (0, σ2) is a random error.

Assuming a fixed-effects model, the τi , β j , and (τβ)ij satisfy the following conditions

a b a b
∑ τi = 0, ∑ β j = 0, ∑ (τβ)ij = 0, ∑ (τβ)ij = 0.
i =1 j =1 i =1 j =1

Using the dot notation, let y... denote the grand total of all the observations, ȳi.. denote the mean of the observations
taken at the i-th level of factor A; ȳ.j. denote the mean of the observations taken at the j-th level of factor B; ȳij. denote
the mean of the observations in the (i, j)-th cell of Table 7.1; and ȳ... denote the grand mean of all the observations.

a b n
y... = ∑ ∑ ∑ yijk
i =1 j =1 k =1
b n
1
ȳi.. =
bn ∑ ∑ yijk
j =1 k =1
a n
1
ȳ.j. =
an ∑ ∑ yijk (7.1.2)
i =1 k =1
n
1
ȳij. =
n ∑ yijk
k =1
a b n
1
ȳ... =
abn ∑ ∑ ∑ yijk
i =1 j =1 k =1

It can be shown that the least squares estimates of parameters of the statistical model for a two-factor factorial experi-
ment with interaction given by Eq. (7.1.1) are

µ̂ = ȳ... ; τ̂i = ȳi.. − ȳ... ; β̂ j = ȳ.j. − ȳ... ; ([

τβ)ij = ȳij. − ȳi.. − ȳ.j. + ȳ... (7.1.3)

190
Thus, the fitted model for a balanced two-factor factorial experiment is given by

ŷijk = µ̂ + τ̂i + β̂ j + ([
τβ)ij = ȳij. (7.1.4)

That is, the (i, j)-th cell sample mean ȳij. is the fitted value for all observations in the (i, j)-th cell. A plot of the mean
response ȳij. versus the level of factor A for different levels of B is called interaction plot, and it is used to check the
presence of interactions between the factors A and B. The interaction plot gives a pictorial view of the tendency in the
data to show the effect of changing one factor as one moves from one level to another of a second factor.

7.1.1 H YPOTHESES IN T WO -WAY ANOVA AND T OTAL VARIABILITY D ECOMPOSITION

We are interested in testing the hypotheses of no main effect for factor A, no main effect for B, and no AB interaction
effect:

1. Hypotheses of no main effect for factor A

H0 : τ1 = τ2 = . . . = τa = 0
(7.1.5)
H1 : At least one τi 6= 0

2. Hypotheses of no main effect for factor B

H0 : β 1 = β 2 = . . . = β b = 0
(7.1.6)
H1 : At least one β j 6= 0

3. Hypotheses of no AB interaction

H0 : (τβ)11 = (τβ)12 = . . . = (τβ) ab = 0

(7.1.7)
H1 : At least one (τβ)ij 6= 0

As in the previous Chapter, the ANOVA tests these hypotheses by partitioning the total variability (total sum of
squares) SST in the data into the sum of squares SS A of the factor A, the sum of squares SSB of the factor B, the sum of
squares SS AB of the interaction AB, and the error sum of squares SSE as follows:

SST = SS A + SSB + SS AB + SSE (7.1.8)

where

a b n a b n
... y2
SST = ∑ ∑ ∑ (yijk − ȳ... )2 = ∑ ∑ ∑ y2ijk − abn
i =1 j =1 k =1 i =1 j =1 k =1
a
SS A = bn ∑ (ȳi.. − ȳ... )2
i =1
b
SSB = an ∑ (ȳ.j. − ȳ... )2 (7.1.9)
j =1
a b
SS AB = n ∑ ∑ (ȳij. − ȳi.. − ȳ.j. + ȳ... )2
i =1 j =1
a b n
SSE = ∑ ∑ ∑ (yijk − ȳij.)2 = SST − SS A − SSB − SS AB
i =1 j =1 k =1

191
Note that SST has abn − 1 degrees of freedom (d.f.), SS A has a − 1 d.f, SSB has b − 1 d.f, and SS AB has ( a − 1)(b − 1)
d.f.. Thus, the error sum of squares SSE has ab(n − 1) d.f. That is,

SST = SS A + SSB + SS AB + SSE (7.1.10)

|{z} |{z} |{z} | {z } |{z}
abn −1 d.f. a −1 d.f. b−1 d.f. ( a −1)( b−1) d.f ab( n −1) d.f.

It is worth pointing out that SS A measures the main effect of factor A, SSB measures the main effect of factor B, SS AB
measures the interaction effect of factors A and B, and SSE represents the variability in the yijk ’s not accounted for by
the main effects and interaction effects.
The mean squares are defined as the sum of squares divided by their degrees of freedom:
SS A SSB SS AB SSE
MS A = , MSB = , MS AB = , MSE = .
a−1 b−1 ( a − 1)(b − 1) ab(n − 1)
Thus, in the case of a 2-way ANOVA, the total variability is divided up into four components: variability among
the levels of each of the two factors, variability due to interaction of the two factors, and variability within cells (error
variability). Three separate statistical tests are performed (based on the F-statistic), comparing each of the first three
sources of variability (variability due to first factor, variability due to second factor, and variability due to interaction) to
the error variability. In each test, the resulting F-statistic value or p-value allows us to determine whether that specific
effect is significant.
The test statistics for testing the hypotheses of no main effect for factor A, no main effect for B, and no AB interaction
effect are computed by dividing the mean square and the mean error sum of square:
1. Hypotheses of no main effect for factor A

H0 : τ1 = τ2 = . . . = τa = 0
H1 : At least one τi 6= 0

If H0 is true, then F0 = MS A /MSE ∼ F ( a − 1, ab(n − 1)). At an α significance level, the null hypothesis would be
rejected if the numerical value f 0 of F0 is greater than f α,a−1,ab( n−1).
2. Hypotheses of no main effect for factor B

H0 : β1 = β2 = . . . = β b = 0
H1 : At least one β j 6= 0

If H0 is true, then F0 = MSB /MSE ∼ F (b − 1, ab(n − 1)). At an α significance level, the null hypothesis would be
rejected if the numerical value f 0 of F0 is greater than f α,b−1,ab( n−1).
3. Hypotheses of no AB interaction

H0 : (τβ)11 = (τβ)12 = . . . = (τβ) ab = 0

H1 : At least one (τβ)ij 6= 0

If H0 is true, then F0 = MS AB /MSE ∼ F (( a − 1)(b − 1), ab(n − 1)). At an α significance level, the null hypothesis
would be rejected if the numerical value f 0 of F0 is greater than f α,( a−1)( b−1),ab(n−1).
It is customary to present the decomposition of the sum of squares and the degrees of freedom in a tabular form
(ANOVA table), as shown in Table 7.2

Example 7.1 An experiment was conducted to determine the effects of four different pesticides on the yield of fruit from different
varieties ( B1 , B2 , B3 ) of a citrus tree. Eight trees from each variety were randomly selected from an orchard. The four pesticides
were then randomly assigned to two trees of each variety and applications were made according recommended levels. Yields of fruit
(in bushels per tree) were obtained after the test period. The data for this experiment are given in Table 7.3.
(i) Write an appropriate model for this experiment
(ii) Construct a plot of the treatment means (i.e. interaction plot).
(iii) Set up an analysis of variance table and conduct the appropriate F-tests on main effects and interactions using a significance
level α = 0.05

192
 TABLE 7.2: ANOVA table for a two-factor experiment.
Source of Sum of Degrees of Mean F0
Variation Squares Freedom Square
SS A MS A
A SS A a−1 MS A =
a−1 MSE
SSB MSB
B SSB b−1 MSB =
b−1 MSE
SS AB MS AB
Interaction SS AB ( a − 1)(b − 1) MS AB =
( a − 1)(b − 1) MSE
SSE
Error SSE ab(n − 1) MSE =
ab(n − 1)
Total SST abn − 1

TABLE 7.3: Data for the two-factor factorial experiment of fruit tree yield.
Pesticide Variety (Factor B)
(Factor A) 1 2 3
1 49 39 55 41 66 68
2 50 55 67 58 85 92
3 43 38 53 42 69 62
4 53 48 85 73 85 99

Solution: From the given information, the experiment is a completely randomized factorial experiment with factor, A,
pesticides, having a = 4 levels and factor, B, variety, having b = 3 levels. There are n = 2 replicates of the 12 factor-level
combination of the two factors.
(i) The model for this experiment is given by


i = 1, 2, 3, 4
yijk = µ + τi + β j + (τβ)ij + ε ijk j = 1, 2, 3 (7.1.11)


k = 1, 2
where µ is the overall mean yield per tree, τi is the effect of the i-th level of pesticide, β j is the effect of the j-th
level of variety of citrus tree, and (τβ)ij is the interaction effect of the i-th level of pesticide with the j-th level of
variety of citrus tree.
(ii) Figure 7.1 depicts the interaction plot for fruit yield experiment. From this plot, we can observe a lack of interac-
tion. We also observe that the differences in mean yields among the three varieties of citrus trees remain nearly
constant across the four pesticide levels. That is, the three lines for the three varieties are nearly parallel lines and
hence the interaction between the levels of variety and pesticide is not significant.
(iii) The analysis of variance is shown in the ANOVA Table 7.4.
MATLAB code
>> n=2; %number of replicates
>> a = 4; %factor A has a levels
>> b = 3; %factor B has b levels
>> data = [49 39 55 41 66 68
50 55 67 58 85 92
43 38 53 42 69 62
53 48 85 73 85 99];
>> y = data’; %transpose the data
>> [p,table,stats] = anova2(y,n);

Because the interaction is not significant as shown in the interaction plot, we can next test the main effects of the
two factors. These tests separately examine the differences among the levels of variety and the levels of pesticides.
Then, we test the interaction between the different pesticides and the different varieties. The appropriate F tests
on main effects and interactions using a significance level α = 0.05 are as follows:

193
 Level 1, Factor B
100
Level 2, Factor B
Level 3, Factor B
90

Mean Response, ȳij.

80

70

60

50

40

Level 1 Level 2 Level 3 Level 4

Factor A

F IGURE 7.1: Interaction plot for fruit yield experiment.

TABLE 7.4: ANOVA table for fruit yield experiment.

– F-test for factor A

Step 1: We test the hypotheses of no main effect for factor A (pesticides):

H0 : τ1 = τ2 = τ3 = τ4 = 0
H1 : At least one τi 6= 0

Step 2: The value of the test statistic is given by

MS A
f0 = = 17.56
MSE
Step 3: The rejection region is f 0 > f α,a−1,ab( n−1), where f α,a−1,ab( n−1) = f 0.05,3,12 = 3.4903.
Step 4: Since f 0 = 17.56 > f α,a−1,ab( n−1) = 3.4903, we reject the null hypothesis H0 . Thus, we have sufficient
evidence to indicate a significant difference in the mean yields among the four pesticide levels.
– F-test for factor B
Step 1: We test the hypotheses of no main effect for factor B (varieties):

H0 : β 1 = β 2 = β 3 = 0
H1 : At least one β j 6= 0

Step 2: The value of the test statistic is given by

MSB
f0 = = 47.24
MSE
Step 3: The rejection region is f 0 > f α,b−1,ab( n−1), where f α,b−1,ab( n−1) = f 0.05,2,12 = 3.8853.

194
 Step 4: Since f 0 = 47.24 > f α,b−1,ab( n−1) = 3.8853, we reject the null hypothesis H0 . Thus, we have sufficient
evidence to indicate a significant difference in the mean yields among the three varieties of citrus trees.
– F-test for interaction AB

Step 1: We test the hypotheses

H0 : (τβ)11 = (τβ)12 = . . . = (τβ)43 = 0

H1 : At least one (τβ)ij 6= 0

Step 2: The value of the test statistic is given by

MS AB
f0 = = 1.8
MSE

Step 3: The rejection region is f 0 > f α,( a−1)( b−1),ab(n−1), where f α,( a−1)( b−1),ab( n−1) = f 0.05,6,12 = 2.9961.
Step 4: Since f 0 = 1.8 ≯ f α,( a−1)( b−1),ab(n−1) = 2.9961, we do not reject the null hypothesis H0 . Thus, we have
insufficient evidence to indicate and interaction between pesticide levels and variety of trees levels.

7.1.2 R ESIDUAL A NALYSIS

The residuals are simply the differences between the data values and the corresponding cell sample means:

eijk = yijk − ŷijk = y ijk − ȳij. (7.1.12)

The normal probability plot is used to check the normality assumption of the errors ε ijk ∼ N (0, σ2), while the residual
plot is used to check the constant variance assumption. In terms of residuals, the error sum of squares can be written
as
a b n a b n
SSE = ∑ ∑ ∑ (yijk − ȳij.)2 = ∑ ∑ ∑ e2ijk (7.1.13)
i =1 j =1 k =1 i =1 j =1 k =1

Figure 7.2 shows the residual plots for fruit yield experiment. The data of normal probability plot appear fairly linear,
suggesting that no reason to doubt the normality assumption. Also, the residual plots against the fitted values as well
as against the factor levels of A and B exhibit random scatter around 0. Thus, the model assumptions are valid.

Example 7.2 Aircraft primer paints are applied to aluminum surfaces by two methods: dipping and spraying. The purpose of
the primer is to improve paint adhesion, and some parts can be primed using either application method. The process engineering
group responsible for this operation is interested in learning whether three different primers differ in their adhesion properties. A
factorial experiment was performed to investigate the effect of paint primer type and application method on paint adhesion. For
each combination of primer type and application method, three specimens were painted, then a finish paint was applied, and the
adhesion force was measured. The data from this experiment are given in Table 7.5.

(i) Write an appropriate statistical model for this experiment

(ii) Set up an analysis of variance table and conduct the appropriate F-tests on main effects and interactions using a significance
level α = 0.05

(iii) Construct the interaction plot.

(iv) Perform the residual analysis.

Solution: From the given information, the experiment is a completely randomized factorial experiment with factor, A,
primer type, having a = 3 levels and factor, B, application method, having b = 2 levels. There are n = 3 replicates of
the 6 factor-level combination of the two factors.

195
 8
0.98

0.95 6

0.90
4

0.75
2
Probability

Residuals
0.50 0

−2
0.25

−4
0.10

0.05 −6
0.02
−8
−6 −4 −2 0 2 4 6 40 50 60 70 80 90 100
Residuals Predicted value
(a) (b)
8 8

6 6

4 4

2 2
Residuals

Residuals
0 0

−2 −2

−4 −4

−6 −6

−8 −8
1 2 3 4 1 2 3
A B
(c) (d)
F IGURE 7.2: Residual analysis for fruit yield experiment: (a) Normal probability plot;
(b) Residual plot against fitted values; (c) Residual plot against factor A; (d) Residual
plot against factor B.

TABLE 7.5: Data for the two-factor factorial experiment of adhesion force.
Primer Application Method
Type Dipping Spraying
1 4.0, 4.5, 4.3 5.4, 4.9, 5.6
2 5.6, 4.9, 5.4 5.8, 6.1, 6.3
3 3.8, 3.7, 4.0 5.5, 5.0, 5.0

(i) The model for this experiment is given by



i = 1, 2, 3
yijk = µ + τi + β j + (τβ)ij + ε ijk j = 1, 2 (7.1.14)


k = 1, 2, 3
where µ is the overall mean adhesion force, τi is the effect of the i-th level of primer type, β j is the effect of the
j-th level of application method, and (τβ)ij is the interaction effect of the i-th level of primer type with the j-th
level of application method.
(ii) The MATLAB output for the analysis of variance table is given in the Table 7.4. The appropriate F-tests on main
effects and interactions using a significance level α = 0.05 are as follows:

196
 – F-test for factor A
Step 1: We test the hypotheses of no main effect for factor A (primer type):
H0 : τ1 = τ2 = τ3 = 0
H1 : At least one τi 6= 0
Step 2: The value of the test statistic is given by
MS A
f0 = = 27.86
MSE
Step 3: The rejection region is f 0 > f α,a−1,ab( n−1), where f α,a−1,ab( n−1) = f 0.05,2,12 = 3.8853.
Step 4: Since f 0 = 27.86 > f α,a−1,ab( n−1) = 3.8853, we reject the null hypothesis H0 . Thus, we have sufficient
evidence to indicate a significant difference in the mean adhesion force among the three levels of primer
type.
– F-test for factor B
Step 1: We test the hypotheses of no main effect for factor B (application method):
H0 : β 1 = β 2 = 0
H1 : At least one β j 6= 0
Step 2: The value of the test statistic is given by
MSB
f0 = = 59.7
MSE
Step 3: The rejection region is f 0 > f α,b−1,ab( n−1), where f α,b−1,ab( n−1) = f 0.05,1,12 = 4.7472.
Step 4: Since f 0 = 59.7 > f α,b−1,ab( n−1) = 4.7472, we reject the null hypothesis H0 . Thus, we have suffi-
cient evidence to indicate a significant difference in the mean adhesion force among the two application
methods.
– F-test for interaction AB

Step 1: We test the hypotheses

H0 : (τβ)11 = (τβ)12 = . . . = (τβ)32 = 0
H1 : At least one (τβ)ij 6= 0

Step 2: The value of the test statistic is given by

MS AB
f0 = = 1.47
MSE
Step 3: The rejection region is f 0 > f α,( a−1)( b−1),ab(n−1), where f α,( a−1)( b−1),ab( n−1) = f 0.05,2,12 = 3.8853.
Step 4: Since f 0 = 1.47 ≯ f α,( a−1)( b−1),ab(n−1) = 3.8853, we do not reject the null hypothesis H0 . Thus, we
have insufficient evidence to indicate and interaction between levels primer type and levels of application
method.

These conclusions can also be drawn from the last column of the ANOVA table by noticing that the p-values for
the two test statistics on the main effects are considerably smaller than α = 0.05, while the p-value for the test
statistic on the interaction is greater than α = 0.05.
(iii) Figure 7.3 depicts the interaction plot for adhesion force experiment. From this plot, we can observe a lack of
interaction because the lines are nearly parallel. Thus, the interaction between the levels of primer type and
application method is not significant. Furthermore, since a large response indicates greater adhesion force, we
conclude that spraying is the best application method and that primer type 2 is most effective.
(iv) The residuals are presented in Table 7.7. Figure 7.4 shows the residual plots for adhesion force experiment. The
data of normal probability plot appear fairly linear, suggesting that no reason to doubt the normality assumption.
Also, the residual plots against the fitted values as well as against the factor levels of A and B exhibit random
scatter around 0. Thus, the model assumptions are valid.

197
 TABLE 7.6: ANOVA table for for adhesion force experiment.

6.5
Dipping
Spraying
6
Mean Response, ȳij.

5.5

4.5

3.5
Level 1 Level 2 Level 3
Primer type

F IGURE 7.3: Interaction plot for adhesion force experiment.

TABLE 7.7: Residuals for the two-factor factorial experiment of adhesion force.
Primer Application Method
Type Dipping Spraying
1 -0.2667, 0.2333, 0.0333 0.1000, -0.4000, 0.3000
2 0.3000, -0.4000, 0.1000 -0.2667, 0.0333, 0.2333
3 -0.0333, -0.1333, 0.1667 0.3333, -0.1667, -0.1667

7.1.3 T WO -WAY ANOVA WITH O NE R EPLICATE

Two-way tests can also be analyzed on data with only one replicate (n = 1) per treatment combination. However,
the interaction between the factors cannot be tested. In other words, it is not possible to test the null hypothesis of no
interaction, but we can test the null hypotheses regarding main effects as long as we assume no interaction. Two-way
ANOVA table without replication is shown in Table 7.8.

Example 7.3 A building contractor employs three construction engineers, E1 , E2 , and E3 , to estimate and bid on jobs. To de-
termine whether one tends to be a more conservative (or liberal) estimator than the others, the contractor selects four projected
construction jobs and has each estimator independently estimate the cost (in dollars per square foot) of each job. The data for this
experiment are given in Table 7.9. Set up an analysis of variance table and conduct the appropriate F-tests on main effects using a
significance level α = 0.05

Solution: From the given information, the experiment is a completely randomized factorial experiment with factor, A,
estimator, having a = 3 levels and factor, B, construction job, having b = 4 levels. There is n = 1 replicate of the 12

198
 0.5
0.98
0.4
0.95

0.90 0.3

0.2
0.75
Probability

0.1

Residuals
0.50 0

−0.1
0.25
−0.2

0.10 −0.3
0.05
−0.4
0.02
−0.5
−0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 3.5 4 4.5 5 5.5 6 6.5
Residuals Predicted value
(a) (b)
0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1
Residuals

Residuals
0 0

−0.1 −0.1

−0.2 −0.2

−0.3 −0.3

−0.4 −0.4

−0.5 −0.5
1 2 3 1 2
A B
(c) (d)
F IGURE 7.4: Residual analysis for adhesion force experiment: (a) Normal probabil-
ity plot; (b) Residual plot against fitted values; (c) Residual plot against factor A; (d)
Residual plot against factor B.

TABLE 7.8: ANOVA table for a two-factor experiment with one replicate.
Source of Sum of Degrees of Mean F0
Variation Squares Freedom Square
SS A MS A
A SS A a−1 MS A =
a−1 MSE
SSB MSB
B SSB b−1 MSB =
b−1 MSE
SSE
Error SSE ( a − 1)(b − 1) MSE =
ab − 1
Total SST ab − 1

factor-level combination of the two factors. The analysis of variance is shown in the ANOVA Table 7.10.
We test the main effects of the two factors. These tests separately examine the differences among the levels of esti-
mator and the levels of construction job. The appropriate F tests on main effects and interactions using a significance
level α = 0.05 are as follows:

• F-test for factor A

199
 TABLE 7.9: Data for the two-factor factorial experiment of the building contractor.
Estimator Construction Job (Factor B)
(Factor A) 1 2 3 4
E1 35.10 34.50 29.25 31.60
E2 37.45 34.60 33.10 34.40
E3 36.30 35.10 32.45 32.90

TABLE 7.10: ANOVA table for the building contractor experiment.

Step 1: We test the hypotheses of no main effect for factor A (pesticides):

H0 : τ1 = τ2 = τ3 = 0
H1 : At least one τi 6= 0
Step 2: The value of the test statistic is given by
MS A
f0 = = 7.2
MSE
Step 3: The rejection region is f 0 > f α,a−1,( a−1)( b−1), where f α,a−1,( a−1)( b−1) = f 0.05,2,6 = 5.1433.
Step 4: Since f 0 = 7.2 > f α,a−1,( a−1)( b−1) = 5.1433, we reject the null hypothesis H0 . Thus, we have sufficient
evidence to indicate a significant difference in the mean among the three estimators.
• F-test for factor B
Step 1: We test the hypotheses of no main effect for factor B (varieties):
H0 : β 1 = β 2 = β 3 = β 4 = 0
H1 : At least one β j 6= 0
Step 2: The value of the test statistic is given by
MSB
f0 = = 16.61
MSE
Step 3: The rejection region is f 0 > f α,b−1,( a−1)( b−1), where f α,b−1,( a−1)( b−1) = f 0.05,3,6 = 4.7571.
Step 4: Since f 0 = 47.24 > f α,b−1,( a−1)( b−1) = 4.7571, we reject the null hypothesis H0 . Thus, we have sufficient
evidence to indicate a significant difference in the mean among the four construction jobs.

7.2 H IGH -FACTOR E XPERIMENTS

The two-factor experiments can be generalized to more than 2 factors. However, each additional factor adds a layer of
complexity to the analysis. For example, in the case of three-factor experiment there are a levels of factor A, b levels of
factor B, and c levels of factor C. The statistical model for a three-factor experiment can be written as follows:


 i = 1, 2, . . . , a

 j = 1, 2, . . . , b
yijkl = µ + τi + β j + γk + (τβ)ij + (τγ)ik + ( βγ) jk + (τβγ)ijk + ε ijkl (7.2.1)

 k = 1, 2, . . . , c


l = 1, 2, . . . , n
where τi , β j , and γk are the main effects; (τβ)ij , (τγ)ik , and ( βγ) jk are the two-interaction effects that have the same
interpretation as in the case of the two-factor experiment; and (τβγ)ijk is the three-factor interaction effect.

200
7.3 2k FACTORIAL D ESIGNS

In this section, we focus on experimental designs in which the experimental plan calls for the study of the effect on
a response of k factors, each at two levels. These are commonly known as 2k factorial experiments. For example, 22
(two-level) factorial experiments are factorial experiments in which each factor is investigated at only two levels.
The factorial experiments, where all combination of the levels of the factors are run, are usually referred to as full
factorial experiments. Full factorial two-level experiments are also referred to as 2k designs where k denotes the number
of factors being investigated in the experiment. These designs are referred to as two-level full factorial designs. A full
factorial two level design with k factors requires 2k runs for a single replicate. For example, a two-level experiment with
three factors will require 23 = 8 runs. The choice of the two levels of factors used in two level experiments depends on
the factor - some factors naturally have two levels. For example, if gender is a factor, then male and female are the two
levels.
Assume we have n observations yij , i = 1, . . . , 2k ; j = 1, . . . , n that are made at each of each of the 2k runs, and denote
by ȳi = (∑nj yij )/n the sample mean of the data of each run, as shown in Table 7.11. The grand mean, ȳ¯, is given by

2k 2k n
1 1
ȳ¯ =
2k
∑ ȳi = 2k n
∑ ∑ yij (7.3.1)
i =1 i =1 j =1

TABLE 7.11: Data Layout for a 2k Design.

Data
Run Rep 1 . . . Rep n Averages
1 y11 ... y1n ȳ1
2 y21 ... y2n ȳ2
3 y31 ... y3n ȳ3
.. .. .. .. ..
. . . . .
2k y2k 1 ... y2k n ȳ2k

7.3.1 22 FACTORIAL D ESIGN

The simplest of the two-level factorial experiments is the 22 design, where two factors A and factor B are investigated
at two levels. A single replicate of this design will require four runs (22 = 4). The geometry of the 22 design can
be represented using a square with the four treatment combinations lying at the four corners, as shown in Figure 7.5
(right). Note that the design can be represented geometrically as a square with the runs forming the corners of the
square. Figure 7.5 shows the 4 runs in a tabular format often called the test matrix. The signs “-” and “+” denote the
low and high levels of a factor, respectively. Figure 7.5 (left) shows the treatments for this design, where letters are used
to represent the treatments. For example, (1) represents the treatment combination where all factors involved are at the
low level or the level represented by −1; a represents the treatment combination where factor A is at the high level,
while the remaining factors (in this case, factor B) are at the low level. Similarly, b represents the treatment combination
where factor B is at the high level, while factor A is at the low level and ab represents the treatment combination where
factors A and B are at the high level. The letters (1), a, b, and ab represent the totals of all n observations taken at these
design points; that is n observations are made at each of the 22 = 4 treatment combinations (runs). Denote these n
observations by y ij , i = 1, . . . , 4; j = 1, . . . , n, as shown in Table 7.12. The averages, ȳi = (∑nj=1 yij )/n, i = 1, . . . 4, are
also shown in the last column of the table.
The design matrix, displayed in Figure 7.5 (center), shows that the sum of the terms resulting from the product of
any two columns of the design matrix is zero. That is, the 22 design is an orthogonal design. In fact all 2k designs are
orthogonal designs. Column I represents the intercept term. Columns A and B represent the respective factor settings.
Column AB represents the interaction and is the product of columns A and B. Note that with the exception of identity
column I, every column has an equal number of + and signs.
The effects investigated by the 22 design are the main effects A and B and the two-factor interaction AB. The main
effect of A (resp. B) is defined as the difference in the mean response between the high level of A (resp. B), averaged

201
 TABLE 7.12: Data Layout for a 22 Design.
Treatment Factor Data
Combination A B Rep 1 . . . Rep n Averages
(1) - - y11 ... y1n ȳ1
a + - y21 ... y2n ȳ2
b - + y31 ... y3n ȳ3
ab + + y41 ... y4n ȳ4

Design Geometry
+
(b) (ab)

Test Matrix Design Matrix

Run A B I A B AB
 
(1) - - + − − +

B
a + -  + + − − 
 
b - +  + − + − 
ab + + + + + +

(1) (a)
−
− +
A

F IGURE 7.5: 22 design.

over the levels of B (resp. A), while the main effect of the interaction AB is defined as the difference between the mean
effect of A at the high level of B and at the low level of B. Thus, Effect A , EffectB , and EffectAB are estimated as follows:

a + ab b + (1) 1 Contrast A
EffectA = A = ȳ A+ − ȳ A− = − = ( a + ab − b − (1)) =
2n 2n 2n 2n
b + ab a + (1) 1 ContrastB
EffectB = B = ȳ B+ − ȳ B− = − = (b + ab − a − (1)) = (7.3.2)
2n 2n 2n 2n
ab + (1) a + b 1 Contrast AB
EffectAB = AB = − = ( ab + (1) − a − b) =
2n 2n 2n 2n

where the contrast of a factor or interaction is obtained by point-wise multiplication of the treatment combinations in
the first column of Table 7.13 by the signs in the corresponding factor or interaction column.
Similarly, the sums of squares can be estimated as follows:

( a + ab − b − (1))2 (Contrast A )2
SS A = =
4n 4n
(b + ab − a − (1))2 (ContrastB )2 (7.3.3)
SSB = =
4n 4n
( ab + (1) − a − b)2 (Contrast AB )2
SS AB = =
4n 4n

In general, for any 2k design with n replicates, the effect estimates and the sum of squares for any effect or interaction
are computed as follows:

202
 TABLE 7.13: Signs for Contrasts in the 22 Design.
Treatment Factorial Effect
Combination A B AB
(1) - - +
a + - -
b - + -
ab + + +

Contrast
Effect =
n2k−1
(7.3.4)
(Contrast)2
SS = = n(Effect)2 2k−2
n2k

Example 7.4 Consider the data in Table 7.14 for a two-factor factorial experiment, where the data numbers denote the averages of
the cells.

TABLE 7.14: Factorial experiment without interaction.

Factor A Factor B
Blow Bhigh
Alow 15 25
Ahigh 40 50

1. Draw the design geometry

2. Find the main effects of A, B, and AB

3. Draw the interaction plot.

Solution:

1. The design geometry is shown in Figure 7.6.

+
25 50
B

15 40
−

− +
A

F IGURE 7.6: Design geometry.

203
 2. The main effects of A, B, and AB are
40 + 50 15 + 25
A= − = 25
2 2
25 + 50 15 + 40
B= − = 10
2 2
15 + 50 25 + 40
AB = − =0
2 2

3. The interaction plot, shown in Figure 7.7, reveals that there is no interaction (parallel lines) between the factors A
and B.

55 55
B− A−
50 B+ 50 A+

45 45
Average Response

Average Response
40 40

35 35

30 30

25 25

20 20

15 15

10 10
− + − +
A B

F IGURE 7.7: Interaction plots.

MATLAB code
>> y = [15 40 25 50]; %average response
>> M = fracfact(’b a ba’); % order: A B AB
>> interactionplot(y,{M(:,1) M(:,2)},’varnames’,{’A’,’B’})

Example 7.5 Consider the data in Table 7.15 for a two-factor factorial experiment, where the data numbers denote the averages of
the cells.

TABLE 7.15: Factorial experiment without interaction.

Factor A Factor B
Blow Bhigh
Alow 15 25
Ahigh 40 0

1. Draw the design geometry

2. Find the main effects of A, B, and AB

3. Draw the interaction plot.

Solution:

1. The design geometry is shown in Figure 7.8.

204
 +
25 0

B
15 40
−

− +
A

F IGURE 7.8: Design geometry.

2. The main effects of A, B, and AB are

40 + 0 15 + 25
A= − =0
2 2
25 + 0 15 + 40
B= − = −15
2 2
15 + 0 25 + 40
AB = − = −25
2 2

3. The interaction plot, shown in Figure 7.9, reveals that there is interaction between the factors A and B.

45 45
B− A−
40 B+ 40 A+

35 35
Average Response

Average Response

30 30

25 25

20 20

15 15

10 10

5 5

0 0

− + − +
A B

F IGURE 7.9: Interaction plots.

MATLAB code
>> y = [15 40 25 0]; %average response
>> M = fracfact(’b a ba’); % order: A B AB
>> interactionplot(y,{M(:,1) M(:,2)},’varnames’,{’A’,’B’})

7.3.2 A NALYSIS OF VARIANCE FOR 2k D ESIGN

The total sum of squares, which has 2k n − 1 degrees of freedom, is given by

205
 2k n
SST = ∑ ∑ (yij − ȳ¯)2 (7.3.5)
i =1 j =1

where ȳ¯ is the grand mean of all observations. Thus, the error sum of squares, which has 2k n − 1 degrees of freedom,
is obtained by subtraction

SSE = SST − SS A − SSB − SS AB . (7.3.6)

The ANOVA table for the two-factor (2k with k = 2) factorial design is shown in Table 7.16.

TABLE 7.16: ANOVA table for a two-factor (k = 2) factorial experiment.

Source of Sum of Degrees of Mean F0
Variation Squares Freedom Square
MS A
A SS A 1 MS A = SS A
MSE
MSB
B SSB 1 MSB = SSB
MSE
MS AB
Interaction SS AB 1 MS AB = SS AB
MSE
SSE
Error SSE 2k ( n − 1) MSE = k
2 ( n − 1)
Total SST 2k n − 1

7.3.3 S TATISTICAL I NFERENCE

It can be shown that variance of the effect estimate for a 2k design is given by
 
Contrast σ2
var (Effect) = var = (7.3.7)
n2k−1 n2k−2

Since the variance σ2 can be estimated by σ̂2 = MSE , the standard error of the effect is then given by
r
σ̂ MSE
s.e. (Effect) = √ = (7.3.8)
n2 k − 2 n2k−2
Thus, a 100(1 − α)% confidence interval for an effect is given by
r
σ̂ MSE
Effect ± tα/2,2k ( n−1) s.e. (Effect) = Effect ± tα/2,2k ( n−1) √ = Effect ± tα/2,2k (n−1) (7.3.9)
n2k−2 n2k−2
The significance of an effect can be tested using the t-statistic
√
Effect (Effect) n2k−2
tratio = = (7.3.10)
s.e. (Effect) σ̂
or equivalently the F-statistic
(Effect)2 (n2k−2 ) MSEffect
f ratio = t2ratio = 2
= (7.3.11)
σ̂ MSE
An effect is considered significant at level α if

|tratio | > tα/2,2k (n−1) or equivalently if f ratio > f α,1,2k ( n−1) (7.3.12)

206
7.3.4 R ESIDUAL A NALYSIS FOR FOR 22 DESIGN

The regression model used to obtain the predicted values is

y = β 0 + β 1 x1 + β 2 x2 + β 12 x1 x2 + ε (7.3.13)

where x1 represents factor A, x2 represents factor B, and x12 represents the interaction AB. The quantities x1 and x2 are
sometimes referred to as contrast coefficients and are given by
( (
−1 if A is low −1 if B is low
x1 = and x2 =
+1 if A is high +1 if B is high.

The regression coefficients β 0 , β 1 , β 2 , and β 12 are estimated as follows:

A B AB
β̂ 0 = ȳ¯ β̂ 1 = β̂ 2 = β̂ 12 =
2 2 2
Thus, the residuals are defined as eij = yij − ŷij , where the fitted values ŷij are obtained using the prediction (regression)
equation

Regression Equation for 22 Design:

A B AB
ŷ = ȳ¯ + x1 + x2 + x1 x2 (7.3.14)
2 2 2

From Eq. (7.3.12), an effect (or factor) in a 22 design is considered significant at a level α if f ratio > f α,1,4( n−1). Thus,
any factor that is not significant is excluded from the regression equation. For example, if the interaction AB is not
significant, i.e. f ratio = MS AB /MSE ≯ f α,1,4( n−1), then the regression equation becomes ŷ = ȳ¯ + ( A/2) x1 + ( B/2) x2.
Similar to the design matrix, we define the contrast matrix X for a 22 design as:
 
+1 −1 −1 +1
 +1 +1 −1 −1 
X=  +1 −1 +1 −1 


+1 +1 +1 +1

Using multiple regression analysis, the regression line may be estimated as ŷ = X β̂, where β̂ = ( X ′ X )−1 X ′ ȳ and
ȳ = (ȳ1 ȳ2 ȳ3 ȳ4 )′ is the column vector of averages.

1 k = 2 ; % 2ˆ2 design
2 M = f r a c f a c t ( ' b a ab ' ) ; % note th e c o r r e c t o r d e r
3 ybar = mean( data , 2 ) ; % column v e c t o r o f averages ( data o f s i z e 4xn )
4 X = [ ones ( 2 ˆ k , 1 ) M] ; % c o n t r a s t m a t r i x f o r f u l l model
5 b e ta h a t = r e g r e s s ( ybar , X ) ; % r e g r e s s i o n c o e f f i c i e n t s
6 y h a t = X * b e ta h a t ; % f i t t e d v a l u e s

The residuals can be used to check the normality assumption of the error by displaying the normal probability plot,
and also to check the constant error variance by displaying the residual plot against the fitted values or factors.

Example 7.6 A router is used to cut locating notches on a printed circuit board. The vibration level at the surface of the board as it
is cut is considered to be a major source of dimensional variation in the notches. Two factors are thought to influence vibration: bit
size (A) and cutting speed (B). Two bit sizes (1/16 and 1/8 inch) and two speeds (40 and 90 rpm) are selected, and four boards are
cut at each set of conditions shown below. The response variable is vibration measured as a resultant vector of three accelerometers
(x, y, and z) on each test circuit board. The resulting data are given in Table 7.17.
(i) Draw the design geometry and the interaction plot. Comment on the interaction. What levels of bit size and speed would
you recommend for routine operation?
(ii) Calculate the estimate of all factorial effects and the sums of squares by the contrast method.
(iii) Perform an analysis of variance.

207
 TABLE 7.17: Data for the router experiment.
Treatment Factor Data
Combination A B Rep 1 Rep 2 Rep 3 Rep 4
(1) - - 18.2 18.9 12.9 14.4
a + - 27.2 24.0 22.4 22.5
b - + 15.9 14.5 15.1 14.2
ab + + 41.0 43.9 36.3 39.9

(iv) Find the regression equation at a significance level α = 0.05.

(v) Construct a normal probability plot of the residuals, and plot the residuals versus the predicted vibration level. Interpret
these plots.

(vi) Find a 95% confidence interval for the interaction.

Solution: This is a 2k factorial experiment with k = 2 and n = 4. The data for the router experiment with averages in
shown in Table 7.18.

TABLE 7.18: Data for the router experiment with averages.

Treatment Factor Data
Combination A B Rep 1 Rep 2 Rep 3 Rep 4 Averages
(1) - - 18.2 18.9 12.9 14.4 16.10
a + - 27.2 24.0 22.4 22.5 24.02
b - + 15.9 14.5 15.1 14.2 14.93
ab + + 41.0 43.9 36.3 39.9 40.27

(i) Figure 7.10 depicts the design geometry and the interaction plot for the router experiment. From the interaction
plot, we can observe that there is interaction. Note that the large positive effect of speed occurs primarily when
bit size is at the high level.

45
B−
B+
(14.93) (40.27) 40
+
(b) (ab)
35
Vibration cps

30
B

25

20

(1) (a)
− 15
(16.10) (24.02)

10
− + − +
A A

F IGURE 7.10: Router experiment. Left: design geometry; Right: interaction plot.

To reduce the vibration, use the smaller bit. Once the small bit is specified, either speed will work equally well,
because the slope of the curve relating vibration to speed for the small tip is approximately zero. The process is
robust to speed changes if the small bit is used.

208
(ii) The effect estimates are given by
Contrast A 1
A= = [−(1) + a − b + ab] = 16.64
n2k−1 2n
ContrastB 1
B= = [−(1) − a + b + ab] = 7.54
n2k−1 2n
Contrast AB 1
AB = k − 1
= [(1) − a − b + ab] = 8.71
n2 2n
Using the formula SS = n(Effect)2 2k−2 for the sum of squares, we have

SS A = nA2 2k−2 = (4)(16.64)2 = 1107.23

SSB = nB2 2k−2 = (4)(7.54)2 = 227.26
SS AB = n( AB)2 2k−2 = (4)(8.71)2 = 303.63
2k n 4 4
SST = ∑ ∑ (yij − ȳ¯)2 = ∑ ∑ (yij − ȳ¯)2 = 1709.83
i =1 j =1 i =1 j =1

where ȳ¯ is the grand mean of all observations yij , that is

2k n 4 4
1 1
ȳ¯ =
2k n
∑ ∑ yij = 16 ∑ ∑ yij = 23.83
i =1 j =1 i =1 j =1

Thus, the error sum of squares is

SSE = SSTotal − SS A − SSB − SS AB = 71.72

(ii) The MATLAB output for the analysis of variance table is given in Table 7.19.

TABLE 7.19: ANOVA table for the router experiment.

1 k = 2 ; n = 4 ; %two− l e v e l experiment w i t h 4 r e p l i c a t e s
2 %data from th e r o u t e r experiment
3 y = [1 8 . 2 18 . 9 12 . 9 14 . 4
4 27 . 2 24 . 0 22 . 4 22 . 5
5 15 . 9 14 . 5 15 . 1 14 . 2
6 41 . 0 43 . 9 36 . 3 39 . 9 ] ;
7 %Use f r a c f a c t t o f i n d th e design m a t r i x ( please note th e r e v e r s e o r d e r )
8 M = f r a c f a c t ( ' b a ba ' ) ; % o r d e r : A B AB
9 R = repmat (M, n , 1 ) ; %r e p l i c a t e o f design m a t r i x
10 g1 = R ( : , 1 ) ; g2 = R ( : , 2 ) ;
11 [ p , t a b l e , s t a t s ] = anovan ( y ( : ) , {g1 g2 } , ' varnames ' ,{ ' B i t S i z e ' ' Speed ' } , ' model ' , ' f u l l ' ) ;

(iv) At a significance level α = 0.05, all the values of the F-test statistics for factor A, factor B, and interaction AB are
greater than f α,1,4( n−1) = f 0.05,1,12 = 4.7472 as shown in ANOVA Table 7.19; that is, f ratio > f α,1,4( n−1) = 4.7472.
Thus, the regression equation is given by
A B AB
ŷ = ȳ¯ + x1 + x2 + x1 x2 = 23.83 + 8.32x1 + 3.77x2 + 4.36x1 x2
2 2 2

209
 For example, for x1 = −1 and x2 = −1 the value of ŷ is equal to 16.10.
(v) Figure 7.11 shows the residual plots for the router experiment. The data of normal probability plot appear fairly
linear, suggesting that no reason to doubt the normality assumption. Also, the residual plots against the fitted val-
ues as well as against the factor levels of A and B exhibit random scatter around 0. Thus, the model assumptions
are valid. There is nothing unusual about the residual plots.

0.98
4
0.95
3
0.90
2
0.75
1
Probability

Residuals
0.50 0

−1
0.25
−2
0.10
−3
0.05
−4
0.02
−4 −3 −2 −1 0 1 2 3 10 15 20 25 30 35 40 45
Residuals Predicted value
(a) (b)
4 4

3 3

2 2

1 1
Residuals

Residuals

0 0

−1 −1

−2 −2

−3 −3

−4 −4
−1 1 −1 1
A B
(c) (d)
F IGURE 7.11: Residual analysis for the router experiment: (a) Normal probability plot;
(b) Residual plot against fitted values; (c) Residual plot against factor A (bit size); (d)
Residual plot against factor B (speed).

(vi) The 95% confidence interval for the interaction AB is given by

r r
MSE 5.98
AB ± tα/2,2k ( n−1) = 8.71 ± t0.025,12 = 8.71 ± 2.66
n2k−2 4
Thus, the interaction effect is between 6.05 and 11.37.

7.3.5 23 FACTORIAL D ESIGN

The 23 design is a two-level factorial experiment design with three factors A, B and C. This design tests three (k = 3)
main effects, A, B and C; three ((2k ) = (32) = 3) two-factor interaction effects, AB, BC, AC; and one ((3k ) = (33) = 1)
three-factor interaction effect, ABC. The design requires 23 = 8 runs per replicate. The eight treatment combinations
corresponding to these runs are (1), a, b, ab, c, ac, bc and abc, which represent the totals of all n observations taken at

210
these design points; that is n observations are made at each of the 23 = 8 treatment combinations. The data layout for
a 23 design is shown in Table 7.20, where n observations by yij , i = 1, . . . , 8; j = 1, . . . , n are made for each of the 23 = 8
runs.

TABLE 7.20: Data Layout for a 23 Design.

Treatment Factor Data
Combination A B C Rep 1 . . . Rep n Averages
(1) - - - y11 ... y1n ȳ1
a + - - y21 ... y2n ȳ2
b - + - y31 ... y3n ȳ3
ab + + - y41 ... y4n ȳ4
c + - + y51 ... y5n ȳ5
ac + - + y61 ... y6n ȳ6
bc + + + y71 ... y7n ȳ7
abc + + + y81 ... y8n ȳ8

The geometry of the 23 design can be represented using a using a cube with the eight treatment combinations lying
at the eight corners, as shown in Figure 7.12 (right). The contrasts of the factors A, B, C and the interactions AB, AC,

Design Geometry
Test Matrix Design Matrix
Run A B C I A B AB C AC BC ABC
 
(1) - - - + − − + − + + −
a + - -  + + − − − − + + 
 
b - + -  + − + − − + − + 
 
ab + + -  + + + + − − − − 
 
c - - +  + − − + + − − + 
 
ac + - +  + + − − + + − − 
 
bc - + +  + − + − + − + − 
abc + + + + + + + + + + +

F IGURE 7.12: 23 design.

BC, and ABC can be calculated using Table 7.13. Similar to the 22 design, the contrast of a factor or interaction is
obtained by point-wise multiplication of the treatment combinations in the first column of Table 7.13 by the signs in
the corresponding factor or interaction column. That is,

Contrast A = −(1) + a − b + ab − c + ac − bc + abc

Contrast B = −(1) − a + b + ab − c − ac + bc + abc
Contrast AB = (1) − a − b + ab + c − ac − bc + abc
ContrastC = −(1) − a − b − ab + c + ac + bc + abc
Contrast AC = (1) − a + b − ab − c + ac − bc + abc
ContrastBC = (1) + a − b − ab − c − ac + bc + abc
Contrast ABC = −(1) + a + b − ab + c − ac − bc + abc

Using Eq. (7.3.4), the effect estimates and the sum of squares for any effect or interaction are computed as follows:

211
 TABLE 7.21: Signs for Contrasts in the 23 Design.
Treatment Factorial Effect
Combination A B AB C AC BC ABC
(1) - - + - + + -
a + - - - - + +
b - + - - + - +
ab + + + - - - -
c - - + + - - +
ac + - - + + - -
bc - + - + - + -
abc + + + + + + +

Effect and Sum of Squares for 23 Design:

Contrast
Effect =
4n
(7.3.15)
(Contrast)2
SS = = 2n(Effect)2
8n

7.3.6 R ESIDUAL A NALYSIS FOR 23 D ESIGN

The residuals are defined as eij = yij − ŷij , where the fitted values ŷij are obtained using the prediction (regression)
equation

Regression Equation for 23 Design:

A B AB C AC BC ABC
ŷ = ȳ¯ + x + x2 + x x2 + x3 + x x3 + x2 x3 + x x2 x3 (7.3.16)
2 1 2 2 1 2 2 1 2 2 1

with contrast coefficients

( ( (
−1 if A is low −1 if B is low −1 if C is low
x1 = x2 = and x3 =
+1 if A is high +1 if B is high +1 if C is high.

Similar to the design matrix, we define the contrast matrix X for a 23 design as follows:
 
+1 −1 −1 +1 −1 +1 +1 −1
 +1 +1 −1 −1 −1 −1 +1 +1 
 
 +1 −1 +1 −1 −1 +1 −1 +1 
 
 +1 +1 +1 +1 −1 −1 −1 −1 
X=  +1 −1 −1 +1 +1 −1 −1 +1 

 
 +1 +1 −1 −1 +1 +1 −1 −1 
 
 +1 −1 +1 −1 +1 −1 +1 −1 
+1 +1 +1 +1 +1 +1 +1 +1

Using multiple regression analysis, the regression line may be estimated as ŷ = X β̂, where β̂ = ( X ′ X )−1 X ′ ȳ and
ȳ = (ȳ1 ȳ2 ȳ3 ȳ4 ȳ5 ȳ6 ȳ7 ȳ8 )′ is the column vector of averages.

1 k = 3 ; % 2ˆ3 design
2 M = f r a c f a c t ( ' c b cb a ac ab abc ' ) ; % note th e c o r r e c t o r d e r
3 ybar = sum ( data , 2 ) ; % column v e c t o r o f averages ( data o f s i z e 8xn )
4 X = [ ones ( 2 ˆ k , 1 ) M] ; % c o n t r a s t m a t r i x f o r f u l l model
5 b e ta h a t = r e g r e s s ( ybar , X ) ; % r e g r e s s i o n c o e f f i c i e n t s
6 y h a t = X * b e ta h a t ; % f i t t e d v a l u e s

212
 The residuals can be used to check the normality assumption of the error by displaying the normal probability plot,
and also to check the constant error variance by displaying the residual plot against the fitted values or factors.

Example 7.7 A quality engineer is studying the surface roughness of a part produced in a metal-cutting operation. Three factors,
feed rate (A), depth of cut (B), and tool angle (C), are of interest. All three factors have been assigned two levels, and two replicates
of a factorial design are run. The coded surface roughness data are given in Table 7.22.

TABLE 7.22: Data for the surface roughness experiment.

Treatment Factor Data
Combination A B C Rep 1 Rep 2
(1) - - - 9 7
a + - - 10 12
b - + - 9 11
ab + + - 12 15
c - - + 11 10
ac + - + 10 13
bc - + + 10 8
abc + + + 16 14

(i) Calculate the estimate of all factorial effects and the sums of squares by the contrast method.

(ii) Perform an analysis of variance.

(iii) Find the regression equation at a significance level α = 0.10.

(iv) Construct a normal probability plot of the residuals, and display the residuals plots. Comment on these plots.

Solution: This is a 23 factorial design in the factors feed rate (A), depth of cut (B), and tool angle (C), with n = 2
replicates. The data for the surface roughness experiment with averages are shown in Table 7.23.

TABLE 7.23: Data for the surface roughness experiment with averages.
Treatment Factor Data
Combination A B C Rep 1 Rep 2 Averages
(1) - - - 9 7 8.00
a + - - 10 12 11.00
b - + - 9 11 10.00
ab + + - 12 15 13.50
c - - + 11 10 10.50
ac + - + 10 13 11.50
bc - + + 10 8 9.00
abc + + + 16 14 15.00

213
(i) The effect estimates are given by
Contrast A 1
A= = (−(1) + a − b + ab − c + ac − bc + abc) = 3.38
n2k−1 4n
ContrastB 1
B= k − 1
= (−(1) − a + b + ab − c − ac + bc + abc) = 1.63
n2 4n
ContrastC 1
C= k − 1
= (−(1) − a − b − ab + c + ac + bc + abc) = 0.88
n2 4n
Contrast AB 1
AB = = ((1) − a − b + ab + c − ac − bc + abc) = 1.37
n2k−1 4n
Contrast AC 1
AC = k − 1
= ((1) − a + b − ab − c + ac − bc + abc) = 0.12
n2 4n
ContrastBC 1
BC = k − 1
= ((1) + a − b − ab − c − ac + bc + abc) = −0.63
n2 4n
Contrast ABC 1
ABC = = (−(1) + a + b − ab + c − ac − bc + abc) = 1.13
n2k−1 4n
Using the formula SS = n(Effect)2 2k−2 for the sum of squares, we have

SS A = nA2 2k−2 = 45.56

SSB = nB2 2k−2 = 10.56
SSC = nC2 2k−2 = 3.06
SS AB = n( AB)2 2k−2 = 7.56
SS AC = n( AC )2 2k−2 = 0.06
SSBC = n( BC )2 2k−2 = 1.56
SS ABC = n( ABC )2 2k−2 = 5.06
2k n 8 2
SST = ∑ ∑ (yij − ȳ¯)2 = ∑ ∑ (yij − ȳ¯)2 = 92.94
i =1 j =1 i =1 j =1

where ȳ¯ is the grand mean of all observations yij , that is

2k n 8 4
1 1
ȳ¯ =
2k n
∑ ∑ yij = 16 ∑ ∑ yij = 11.06
i =1 j =1 i =1 j =1

Thus, the error sum of squares is

SSE = SSTotal − SS A − SSB − SSC − SS AB − SS AC − SSBC − SS ABC = 19.50

(ii) The MATLAB output for the analysis of variance table is given in Table 7.24.

1 k = 3 ; n = 2 ; %th r e e − l e v e l experiment w i t h 2 r e p l i c a t e s
2 %data from th e roughness experiment
3 y = [9 7
4 10 12
5 9 11
6 12 15
7 11 10
8 10 13
9 10 8
10 16 1 4 ] ;
11 %Use f r a c f a c t t o f i n d th e design m a t r i x ( Note th e r e v e r s e o r d e r )
12 M = f r a c f a c t ( ' c b cb a ac ab abc ' ) ; %o r d e r : A B AB C AC BC ABC
13 R = repmat (M, n , 1 ) ; %r e p l i c a t e o f design m a t r i x
14 g1 = R ( : , 1 ) ; g2 = R ( : , 2 ) ; g3 = R ( : , 4 ) ; %f a c t o r s A , B , and C
15 [ p , t a b l e , s t a t s ] = anovan ( y ( : ) , {g1 g2 g3 } , ' varnames ' ,{ ' A ' ' B ' 'C ' } , ' model ' , ' f u l l ' ) ;

214
 TABLE 7.24: ANOVA table for the surface roughness 23 factorial experiment.

(iv) The F-ratios for all three main effects and the interactions are shown in ANOVA Table 7.24. At a significance level
α = 0.10, the critical value for each of these F-ratios is f α,1,2k ( n−1) = f 0.10,1,8 = 3.46. Notice that there is a strong
main effect of feed rate (A), since its F-ratio is well into the critical region. However, there is some indication of
an effect due to the depth of cut (B). That is, for both factors A and B, we have f ratio > f α,1,2k ( n−1) = 3.46. Thus,
the regression equation is given by

A B
ŷ = ȳ¯ + x + x2 = 11.06 + 1.69x1 + 0.81x2
2 1 2
For example, for x1 = −1 and x2 = −1 the value of ŷ is equal to 9.25.

(v) Figure 7.13 shows the normal probability plot for the surface roughness experiment. The data appears to lie
approximately along a straight line, suggesting that there is no reason to doubt the normality assumption. Fig-
ure 7.14 shows the the residual plots against the fitted values as well as against the factor levels of A, B, and C
exhibit random scatter around 0. Thus, the model assumptions are valid. There is nothing unusual about the
residual plots.

0.98

0.95
0.90

0.75
Probability

0.50

0.25

0.10
0.05

0.02
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
Residuals

F IGURE 7.13: Normal probability plot for the surface roughness experiment.

Example 7.8 In the manufacturing industry, the quality of surface finish of graded lumber is an important characteristic. Three
factors are to be tested, each at two levels, for their impact on the surface finish. Factor A is the type of wood: oak (level -1) or pine
(level 1). Factor B is the rate of feed: 2 m/min (level -1) or 4 m/min (level 1). Factor C is the depth of cut: 1 mm (level -1) and 3 mm
(level 1). For each treatment combination, three replications are carried out using a completely randomized design. The resulting
surface finish data are given in Table 7.25.

(i) Calculate the main effects and the interaction effects

215
 2.5 2.5
2 2
1.5 1.5
1 1
0.5 0.5
Residuals

Residuals
0 0
−0.5 −0.5
−1 −1
−1.5 −1.5
−2 −2
−2.5 −2.5
8 9 10 11 12 13 14 −1 1
Predicted value A
(a) (b)
2.5 2.5

2 2

1.5 1.5

1 1

0.5
Residuals

0.5
Residuals
0 0

−0.5 −0.5

−1 −1

−1.5 −1.5

−2 −2

−2.5 −2.5
−1 1 −1 1
B C
(c) (d)
F IGURE 7.14: Residual analysis for the surface roughness experiment: (a) Residual
plot against fitted values; (b) Residual plot against factor A; (c) Residual plot against
factor B; (d) Residual plot against factor C.

(ii) Calculate the sums of squares of each of the main effects and interaction effects.
(iii) Display the interaction plot, and perform an analysis of variance.
(iv) Find the regression equation at a significance level α = 0.05.
(v) Construct a normal probability plot of the residuals, and display the residuals plots. Comment on these plots.
Solution: This is a 23 factorial design in the factors feed rate (A), depth of cut (B), and tool angle (C), with n = 2
replicates. The data for the surface finish experiment with averages are shown in Table 7.26.
(i) Using the formula, Effect = Contrast/(n2k−1 ), the main effects and interaction effects are:
A = −2.667, B = 3.333, C = 18.833, AB = −8.833, AC = −3.333, BC = 1.333, ABC = −3.500

(ii) Using the formula, SS = n(Effect)2 2k−2 for the sum of squares, we have
SS A = 42.667, SSB = 66.667, SSC = 2128.167, SS AB = 468.167, SS AC = 66.667, SSBC = 10.667, SS ABC = 73.5

(iii) The interaction plot for the surface finish experiment in displayed in Figure 7.15. Notice that the interaction
between A and B is the most significant. The MATLAB output for the analysis of variance table is given in
Table 7.27.

216
 TABLE 7.25: Data for the surface finish experiment.
Treatment Factor Data
Combination A B C Rep 1 Rep 2 Rep 3
(1) - - - 6 8 9
a + - - 10 16 15
b - + - 18 12 15
ab + + - 12 9 10
c - - + 20 26 29
ac + - + 34 28 32
bc - + + 36 44 46
abc + + + 25 22 24

TABLE 7.26: Data for the surface finish experiment with averages.
Treatment Factor Data
Combination A B C Rep 1 Rep 2 Rep 3 Averages
(1) - - - 6 8 9 7.67
a + - - 10 16 15 13.67
b - + - 18 12 15 15.00
ab + + - 12 9 10 10.33
c - - + 20 26 29 25.00
ac + - + 34 28 32 31.33
bc - + + 36 44 46 42.00
abc + + + 25 22 24 23.67

−1 1 −1 1

30 A = −1
A=1
25
A 20
15
10
30 30 B = −1
B=1
25 25
20
B 20
15 15
10 10
30 C = −1
C=1
25
20 C
15
10
−1 1 −1 1

F IGURE 7.15: Interaction plot for the surface finish experiment.

1 k = 3 ; n = 3 ; %th r e e − l e v e l experiment w i t h 3 r e p l i c a t e s
2 %data from th e roughness experiment
3 y = [6 8 9
4 10 16 15
5 18 12 15
6 12 9 10

217
 TABLE 7.27: ANOVA table for the surface finish 23 factorial experiment.

7 20 26 29
8 34 28 32
9 36 44 46
10 25 22 2 4 ] ;
11 %Use f r a c f a c t t o f i n d th e design m a t r i x ( Note th e r e v e r s e o r d e r )
12 M = f r a c f a c t ( ' c b cb a ac ab abc ' ) ; %o r d e r : A B AB C AC BC ABC
13 R = repmat (M, n , 1 ) ; %r e p l i c a t e o f design m a t r i x
14 g1 = R ( : , 1 ) ; g2 = R ( : , 2 ) ; g3 = R ( : , 4 ) ; %f a c t o r s A , B , and C
15 [ p , t a b l e , s t a t s ] = anovan ( y ( : ) , {g1 g2 g3 } , ' varnames ' ,{ ' A ' ' B ' 'C ' } , ' model ' , ' f u l l ' ) ;
16 f i g u r e ; i n t e r a c t i o n p l o t ( y ( : ) ,{ g1 g2 g3 } , ' varnames ' ,{ ' A ' ' B ' 'C ' } ) ; %I n t e r a c t i o n p l o t

(iv) The F-ratios for all main effects and the interactions are shown in ANOVA Table 7.27. At a significance level α =
0.05, the critical value for each of these F-ratios is f α,1,2k ( n−1) = f 0.05,1,16 = 4.494. From the ANOVA table, it can be
seen that the F-ratio of B, C, AB, AC, and ABC are greater than the critical value, i.e. f ratio > f α,1,2k ( n−1) = 4.494.
Thus, the regression equation is given by

A B AB C AC BC ABC
ŷ = ȳ¯ + x1 + x2 + x1 x2 + x3 + x1 x3 + x2 x3 + x x2 x3
2 2 2 2 2 2 2 1
= 21.088 − 1.333x1 + 1.667x2 − 4.417x1 x2 + 9.417x3 − 1.667x1 x3 + 0.667x2 x3 − 1.750x1 x2 x3

Notice that the factors A and C are also included in the regression model to preserve the hierarchy of the model.
For example, for x1 = −1 and x2 = −1 the value of ŷ is equal to 7.667.
(v) Figure 7.16 shows the normal probability plot for the surface finish experiment. The data appears to lie approx-
imately along a straight line, suggesting that there is no reason to doubt the normality assumption. Figure 7.17
shows the the residual plots against the fitted values as well as against the factor levels of A, B, and C exhibit
random scatter around 0. Thus, the model assumptions are valid. There is nothing unusual about the residual
plots.

218
 0.98

0.95

0.90

0.75

Probability
0.50

0.25

0.10

0.05

0.02

−6 −4 −2 0 2 4
Residuals

F IGURE 7.16: Normal probability plot for the surface finish experiment.

4
4

2
2
Residuals

Residuals

0 0

−2 −2

−4
−4

−6
−6
5 10 15 20 25 30 35 40 45 −1 1
Predicted value A
(a) (b)
6 6

4 4

2 2
Residuals

Residuals

0 0

−2 −2

−4 −4

−6 −6

−1 1 −1 1
B C
(c) (d)
F IGURE 7.17: Residual analysis for the surface finish experiment: (a) Residual plot
against fitted values; (b) Residual plot against factor A; (c) Residual plot against factor
B; (d) Residual plot against factor C.

219
7.4 P ROBLEMS

❶ In an experiment conducted to determine which of 3 different missile systems is preferable, the propellant burn-
ing rate for 24 static firings was measured. Four different propellant types were used. The experiment yielded
duplicate observations of burning rates at each combination of the treatments. The data from this experiment are
given in Table 7.28.

a) Write an appropriate statistical model for this experiment

b) Set up an analysis of variance table and conduct the appropriate F-tests on main effects and interactions
using a significance level α = 0.05
c) Construct the interaction plot and draw conclusions.
d) Perform the residual analysis and draw conclusions.

TABLE 7.28: Data for the two-factor factorial experiment of propellant burning rates.
Missile Propellant Type
System b1 b2 b3 b4
a1 34.0, 32.7 30.1, 32.8 29.8, 26.7 29.0, 28.9
a2 32.0, 33.2 30.2, 29.8 28.7, 28.1 27.6, 27.8
a3 28.4, 29.3 27.3, 28.9 29.7, 27.3 28.8, 29.1

❷ An electrical engineer is investigating a plasma etching process used in semiconductor manufacturing. It is of

interest to study the effects of two factors, the C2 F6 gas flow rate (A) and the power applied to the cathode (B).
The response is the etch rate. Each factor is run at 3 levels, and 2 experimental runs on etch rate are made for each
of the 9 combinations. The setup is that of a completely randomized design. The data from this experiment are
given in Table 7.29.

a) Write an appropriate statistical model for this experiment

b) Set up an analysis of variance table and conduct the appropriate F-tests on main effects and interactions
using a significance level α = 0.05
c) Construct the interaction plot and draw conclusions.
d) Perform the residual analysis and draw conclusions.

TABLE 7.29: Data for the two-factor factorial experiment of plasma etching process.
C2 F6 Flow Power Supplied
Rate 1 2 3
1 288, 360 488, 465 670, 720
2 385, 411 482, 521 692, 724
3 488, 462 595, 612 761, 801

❸ A manufacturer wishes to determine the effectiveness of four types of machines (A, B, C, and D) in the production
of bolts. To accomplish this, the numbers of defective bolts produced by each machine in the days of a given week
are obtained for each of two shifts; the results are shown in Table 7.30.

a) Write an appropriate statistical model for this experiment

b) Set up an analysis of variance table and conduct the appropriate F-tests on main effects and interactions
using a significance level α = 0.05
c) Construct the interaction plot and draw conclusions.
d) Perform the residual analysis and draw conclusions.

220
 TABLE 7.30: Data for the two-factor factorial experiment of number of defective bolts.
Machine First Shift Second Shift
Type Mon. Tue. Wed. Thu. Fri. Mon. Tue. Wed. Thu. Fri.
A 6 4 5 5 4 5 7 4 6 8
B 10 8 7 7 9 7 9 12 8 8
C 7 5 6 5 9 9 7 5 4 6
D 8 4 6 5 5 5 7 9 7 10

❹ A basic processing step in the integrated circuit manufacturing industry is to grow an epitaxial layer on polished
silicon wafers. The wafers are mounted on a susceptor and positioned inside a bell jar. Chemical vapors are
introduced through nozzles near the top of the jar. The susceptor is rotated, and heat is applied. These conditions
are maintained until the epitaxial layer is thick enough. Table 7.31 presents the results of a 22 factorial design with
n = 4 replicates using the factors A = deposition time and B = arsenic flow rate. The two levels of deposition
time are − = short and + = long, and the two levels of arsenic flow rate are − = 55% and + = 59%. The
response variable is epitaxial layer thickness (µm).

TABLE 7.31: Data for the epitaxial process experiment.

Treatment Factor Data
Combination A B Rep 1 Rep 2 Rep 3 Rep 4
(1) - - 14.037 14.165 13.972 13.907
a + - 14.821 14.757 14.843 14.878
b - + 13.880 13.860 14.032 13.914
ab + + 14.888 14.921 14.415 14.932

a) Draw the design geometry and the interaction plot. Comment on the interaction.
b) Calculate the estimate of all factorial effects and the sums of squares by the contrast method.
c) Perform an analysis of variance.
d) Find the regression equation at a significance level α = 0.05.
e) Construct a normal probability plot of the residuals, and plot the residuals versus the predicted value as
well as versus the main factors. Interpret these plots.
f) Find a 95% confidence interval for the interaction.
❺ Consider an investigation into the effect of the concentration of the reactant and the amount of the catalyst on
the conversion (yield) in a chemical process. The objective of the experiment was to determine if adjustments to
either of these two factors would increase the yield. The factor reactant concentration, denoted by A, has two
levels, 15 and 25 percent. The other factor catalyst, denoted by B, has also two levels, 2 and 1 pounds. The
experiment is replicated three times. The order in which the runs are made is random. The response variable
data are given in Table 7.31.

TABLE 7.32: Data for the catalyst experiment.

Treatment Factor Data
Combination A B Rep 1 Rep 2 Rep 3
(1) - - 25 28 27
a + - 36 32 32
b - + 18 19 23
ab + + 29 30 33

a) Draw the design geometry and the interaction plot. Comment on the interaction.
b) Calculate the main effects and the interaction effects
c) Calculate the sums of squares of each of the main effects and interaction effects.

221
 d) Perform an analysis of variance.
e) Find the regression equation at a significance level α = 0.05.
f) Calculate the residuals using the regression model at α = 0.05. Then, display all the residuals in one single
boxplot. Are there any outliers?
g) Construct a normal probability plot of the residuals, and plot the residuals versus the predicted value as
well as versus the main factors. Interpret these plots.
h) Find a 95% confidence interval for each main factor.
i) Find a 95% confidence interval for the interaction.
❻ It is important to study the effect of the concentration of the reactant and the feed rate on the viscosity of the
product from a chemical process. Let the reactant concentration be factor A, at levels 15% and 25%. Let the feed
rate be factor B, with levels 20 lb/hr and 30 lb/hr. The experiment involves two experimental runs at each of the
four combinations (− = low and + = high). The viscosity readings are shown in Figure 7.18, i.e. the data given
inside the square or in the table.

(b) (ab)
+
132 149
137 152

Treatment Factor Data

Combination A B Rep 1 Rep 2
(1) - - 145 147
B

a + - 154 150
b - + 132 137
145 154 ab + + 149 152
147 150
−
(1) (a)

− A +

F IGURE 7.18: Data for the viscosity experiment.

a) Draw the design geometry and the interaction plot. Comment on the interaction.
b) Calculate the main effects and the interaction effects
c) Calculate the sums of squares of each of the main effects and interaction effects.
d) Perform an analysis of variance.
e) Find the regression equation at a significance level α = 0.05.
f) Construct a normal probability plot of the residuals, and plot the residuals versus the predicted value as
well as versus the main factors. Interpret these plots.
g) Find a 95% confidence interval for each main factor.
h) Find a 95% confidence interval for the interaction.
❼ In an experiment conducted to study a particular filtering system for coal, a coagulant was added to a solution
in a tank containing coal and sludge, which was then placed in a recirculation system in order that the coal could
be washed. Three factors were varied in the experimental process:
Factor A: percent solids circulated initially in the overflow
Factor B: flow rate of the polymer
Factor C: pH of the tank
The amount of solids in the underflow of the cleansing system determines how clean the coal has become. Two
levels of each factor were used and two experimental runs were made for each of the 23 = 8 treatment combi-
nations. The response measurements in percent solids by weight in the underflow of the circulation system are
given in Table 7.33.

222
 TABLE 7.33: Data for the coal experiment.
Treatment Factor Data
Combination A B C Rep 1 Rep 2
(1) - - - 4.65 5.81
a + - - 21.42 21.35
b - + - 12.66 12.56
ab + + - 18.27 16.62
c - - + 7.93 7.88
ac + - + 13.18 12.87
bc - + + 6.51 6.26
abc + + + 18.23 17.83

(i) Calculate the estimate of all factorial effects and the sums of squares by the contrast method.
(ii) Perform an analysis of variance.
(iii) Find the regression equation at a significance level α = 0.10.
(iv) Construct a normal probability plot of the residuals, and plot the residuals versus the predicted value as
well as versus the main factors. Interpret these plots.
❽ In the search for lower-pollution synthetic fuel, researchers are experimenting with three different factors, each
controlled at two levels, for the processing of such a fuel. Factor A is the concentration of corn extract at 5% and
10%, factor B is the concentration of an ethylene-based compound at 15% and 25%, and factor C is the distillation
temperature at 120°C and 150°C . The levels of undesirable emission of the fuel are are given in Table 7.34 for three
replications of each treatment; each level is randomly assigned to a treatment. The larger the level of emission,
the worse the impact on the environment.

TABLE 7.34: Data for the emission level experiment.

Treatment Factor Data
Combination A B C Rep 1 Rep 2 Rep 3
(1) - - - 30 24 26
a + - - 18 22 24
b - + - 30 32 25
ab + + - 43 47 41
c - - + 28 24 22
ac + - + 54 49 46
bc - + + 58 48 50
abc + + + 24 20 22

a) Calculate the main effects and the interaction effects

b) Calculate the sums of squares of each of the main effects and interaction effects.
c) Display the interaction plot, and perform an analysis of variance.
d) Find the regression equation at 5% level of significance. Which factors are significant?
e) Construct a normal probability plot of the residuals, and plot the residuals versus the predicted value as
well as versus the main factors. Interpret these plots.

7.5 REFERENCES

[1] D.C. Montgomery, Introduction to Statistical Quality Control, Wiley, 6th Edition, 2009.
[2] I. Bass and B. Lawton, Lean Six Sigma using SigmaXL and Minitab, McGraw-Hill Professional, 1st Edition, 2009.

223
TABLE I: C UMULATIVE S TANDARD N ORMAL D ISTRIBUTION

z 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.0 0.50000 0.50399 0.50798 0.51197 0.51595 0.51994 0.53292 0.52790 0.53188 0.53586
0.1 0.53983 0.54380 0.54776 0.55172 0.55576 0.55962 0.56356 0.56749 0.57142 0.57535
0.2 0.57926 0.58317 0.58706 0.59095 0.59484 0.59871 0.60257 0.60642 0.61026 0.61409
0.3 0.61791 0.62172 0.62552 0.62930 0.63307 0.63683 0.64058 0.64431 0.64803 0.65173
0.4 0.65542 0.65910 0.66276 0.66640 0.67003 0.67365 0.67724 0.68082 0.68439 0.68793
0.5 0.69146 0.69497 0.69847 0.70194 0.70540 0.70884 0.71226 0.71566 0.71904 0.72240
0.6 0.72575 0.72907 0.73237 0.73565 0.73891 0.74215 0.74537 0.74857 0.75175 0.75490
0.7 0.75804 0.76115 0.76424 0.76731 0.77035 0.77337 0.77637 0.77935 0.78231 0.78524
0.8 0.78814 0.79103 0.79389 0.79673 0.79955 0.80234 0.80511 0.80785 0.81057 0.81327
0.9 0.81594 0.81859 0.82121 0.82381 0.82639 0.82894 0.83147 0.83398 0.83646 0.83891
1.0 0.84135 0.84375 0.84614 0.84850 0.85083 0.85314 0.85543 0.85769 0.85993 0.86214
1.1 0.86433 0.86650 0.86864 0.87076 0.87286 0.87493 0.87698 0.87900 0.88100 0.88298
1.2 0.88493 0.88686 0.88877 0.89065 0.89251 0.89435 0.89616 0.89796 0.89973 0.90148
1.3 0.90320 0.90490 0.90658 0.90824 0.90988 0.91149 0.91309 0.91466 0.91621 0.91774
1.4 0.91924 0.92073 0.92220 0.92364 0.92507 0.92647 0.92785 0.92922 0.93056 0.93189
1.5 0.93319 0.93448 0.93574 0.93699 0.93822 0.93943 0.94062 0.94179 0.94295 0.94408
1.6 0.94520 0.94630 0.94738 0.94845 0.94950 0.95053 0.95154 0.95254 0.95352 0.95449
1.7 0.95544 0.95637 0.95728 0.95818 0.95907 0.95994 0.96080 0.96164 0.96246 0.96327
1.8 0.96407 0.96485 0.96562 0.96637 0.96712 0.96784 0.96856 0.96926 0.96995 0.97062
1.9 0.97128 0.97193 0.97257 0.97320 0.97381 0.97441 0.97500 0.97558 0.97615 0.97671
2.0 0.97725 0.97778 0.97831 0.97882 0.97933 0.97982 0.98030 0.98077 0.98124 0.98169
2.1 0.98214 0.98257 0.98300 0.98341 0.98382 0.98422 0.98461 0.98500 0.98537 0.98574
2.2 0.98610 0.98645 0.98679 0.98713 0.98745 0.98778 0.98809 0.98840 0.98870 0.98899
2.3 0.98928 0.98956 0.98983 0.99010 0.99036 0.99061 0.99086 0.99111 0.99134 0.99158
2.4 0.99180 0.99202 0.99224 0.99245 0.99266 0.99286 0.99305 0.99324 0.99343 0.99361
2.5 0.99379 0.99396 0.99413 0.99430 0.99446 0.99461 0.99477 0.99491 0.99506 0.99520
2.6 0.99534 0.99547 0.99560 0.99573 0.99586 0.99598 0.99609 0.99621 0.99632 0.99643
2.7 0.99653 0.99664 0.99674 0.99683 0.99693 0.99702 0.99711 0.99720 0.99728 0.99736
2.8 0.99745 0.99752 0.99760 0.99767 0.99774 0.99781 0.99788 0.99795 0.99801 0.99807
2.9 0.99813 0.99819 0.99825 0.99831 0.99836 0.99841 0.99846 0.99851 0.99856 0.99860
3.0 0.99865 0.99869 0.99874 0.99878 0.99882 0.99886 0.99889 0.99893 0.99896 0.99900
3.1 0.99903 0.99906 0.99910 0.99913 0.99916 0.99918 0.99921 0.99924 0.99926 0.99929
3.2 0.99931 0.99934 0.99936 0.99938 0.99940 0.99942 0.99944 0.99946 0.99948 0.99950
3.3 0.99952 0.99953 0.99955 0.99957 0.99958 0.99960 0.99961 0.99962 0.99964 0.99965
3.4 0.99966 0.99967 0.99969 0.99970 0.99971 0.99972 0.99973 0.99974 0.99975 0.99976
3.5 0.99977 0.99978 0.99978 0.99979 0.99980 0.99981 0.99982 0.99982 0.99983 0.99984
3.6 0.99984 0.99985 0.99985 0.99986 0.99986 0.99987 0.99987 0.99988 0.99988 0.99989
3.7 0.99989 0.99990 0.99990 0.99990 0.99991 0.99991 0.99992 0.99992 0.99992 0.99992
3.8 0.99993 0.99993 0.99993 0.99994 0.99994 0.99994 0.99994 0.99995 0.99995 0.99995
3.9 0.99995 0.99995 0.99996 0.99996 0.99996 0.99996 0.99996 0.99996 0.99997 0.99997

224
 TABLE II: P ERCENTAGE P OINTS χ2α,n

α
n 0.99 0.975 0.95 0.90 0.50 0.10 0.05 0.025 0.01
1 0.0002 0.001 0.004 0.02 0.45 2.71 3.84 5.02 6.63
2 0.02 0.05 0.10 0.21 1.39 4.61 5.99 7.38 9.21
3 0.11 0.22 0.35 0.58 2.37 6.25 7.81 9.35 11.34
4 0.30 0.48 0.71 1.06 3.36 7.78 9.49 11.14 13.28
5 0.55 0.83 1.15 1.61 4.35 9.24 11.07 12.83 15.09
6 0.87 1.24 1.64 2.20 5.35 10.64 12.59 14.45 16.81
7 1.24 1.69 2.17 2.83 6.35 12.02 14.07 16.01 18.48
8 1.65 2.18 2.73 3.49 7.34 13.36 15.51 17.53 20.09
9 2.09 2.70 3.33 4.17 8.34 14.68 16.92 19.02 21.67
10 2.56 3.24 3.94 4.87 9.34 15.99 18.31 20.48 23.21
11 3.05 3.81 4.57 5.58 10.34 17.28 19.68 21.92 24.72
12 3.57 4.40 5.23 6.30 11.34 18.55 21.03 23.34 26.22
13 4.11 5.01 5.89 7.04 12.34 19.81 22.36 24.74 27.69
14 4.66 5.62 6.57 7.79 13.34 21.06 23.68 26.12 29.14
15 5.23 6.26 7.26 8.55 14.34 22.31 25.00 27.49 30.58
16 5.81 6.90 7.96 9.31 15.34 23.54 26.30 28.85 32.00
17 6.41 7.56 8.67 10.09 16.34 24.77 27.59 30.19 33.41
18 7.01 8.23 9.39 10.86 17.34 25.99 28.87 31.53 34.81
19 7.63 8.90 10.12 11.65 18.34 27.20 30.14 32.85 36.19
20 8.26 9.59 10.85 12.44 19.34 28.41 31.41 34.17 37.57
21 8.90 10.28 11.59 13.24 20.34 29.62 32.67 35.48 38.93
22 9.54 10.98 12.34 14.04 21.34 30.81 33.92 36.78 40.29
23 10.20 11.69 13.09 14.85 22.34 32.01 35.17 38.08 41.64
24 10.86 12.40 13.85 15.66 23.34 33.20 36.42 39.36 42.98
25 11.52 13.11 14.61 16.47 24.34 34.38 37.65 40.65 44.31
26 12.20 13.84 15.38 17.29 25.34 35.56 38.89 41.92 45.64
27 12.88 14.57 16.15 18.11 26.34 36.74 40.11 43.19 46.96
28 13.56 15.30 16.93 18.94 27.34 37.92 41.34 44.46 48.28
29 14.26 16.04 17.71 19.77 28.34 39.09 42.56 45.72 49.59
30 14.95 16.78 18.49 20.60 29.34 40.26 43.77 46.98 50.89
40 22.16 24.42 26.51 29.05 39.34 51.81 55.76 59.34 63.69
50 29.71 32.35 34.76 37.69 49.33 63.17 67.50 71.42 76.15
60 37.48 40.47 43.19 46.46 59.33 74.40 79.08 83.30 88.38
70 45.44 48.75 51.74 55.33 69.33 85.53 90.53 95.02 100.43
80 53.54 57.15 60.39 64.28 79.33 96.58 101.88 106.63 112.33
90 61.75 65.64 69.13 73.29 89.33 107.57 113.15 118.14 124.12
100 70.06 74.22 77.93 82.36 99.33 118.50 124.34 129.56 135.81

225
 TABLE III: P ERCENTAGE P OINTS tα,n

α
n 0.25 0.10 0.05 0.025 0.01 0.0833 0.00625 0.005
1 1.000 3.078 6.314 12.706 31.821 38.204 50.923 63.657
2 0.816 1.886 2.920 4.303 6.965 7.649 8.860 9.925
3 0.765 1.638 2.353 3.182 4.541 4.857 5.392 5.841
4 0.741 1.533 2.132 2.776 3.747 3.961 4.315 4.604
5 0.727 1.476 2.015 2.571 3.365 3.534 3.810 4.032
6 0.718 1.440 1.943 2.447 3.143 3.287 3.521 3.707
7 0.711 1.415 1.895 2.365 2.998 3.128 3.335 3.499
8 0.706 1.397 1.860 2.306 2.896 3.016 3.206 3.355
9 0.703 1.383 1.833 2.262 2.821 2.933 3.111 3.250
10 0.700 1.372 1.812 2.228 2.764 2.870 3.038 3.169
11 0.697 1.363 1.796 2.201 2.718 2.820 2.981 3.106
12 0.695 1.356 1.782 2.179 2.681 2.779 2.934 3.055
13 0.694 1.350 1.771 2.160 2.650 2.746 2.896 3.012
14 0.692 1.345 1.761 2.145 2.624 2.718 2.864 2.977
15 0.691 1.341 1.753 2.131 2.602 2.694 2.837 2.947
16 0.690 1.337 1.746 2.120 2.583 2.673 2.813 2.921
17 0.689 1.333 1.740 2.110 2.567 2.655 2.793 2.898
18 0.688 1.330 1.734 2.101 2.552 2.639 2.775 2.878
19 0.688 1.328 1.729 2.093 2.539 2.625 2.759 2.861
20 0.687 1.325 1.725 2.086 2.528 2.613 2.744 2.845
21 0.686 1.323 1.721 2.080 2.518 2.601 2.732 2.831
22 0.686 1.321 1.717 2.074 2.508 2.591 2.720 2.819
23 0.685 1.319 1.714 2.069 2.500 2.582 2.710 2.807
24 0.685 1.318 1.711 2.064 2.492 2.574 2.700 2.797
25 0.684 1.316 1.708 2.060 2.485 2.566 2.692 2.787
26 0.684 1.315 1.706 2.056 2.479 2.559 2.684 2.779
27 0.684 1.314 1.703 2.052 2.473 2.552 2.676 2.771
28 0.683 1.313 1.701 2.048 2.467 2.546 2.669 2.763
29 0.683 1.311 1.699 2.045 2.462 2.541 2.663 2.756
30 0.683 1.310 1.697 2.042 2.457 2.536 2.657 2.750
40 0.681 1.303 1.684 2.021 2.423 2.499 2.616 2.704
60 0.679 1.296 1.671 2.000 2.390 2.463 2.575 2.660
120 0.677 1.289 1.658 1.980 2.358 2.428 2.536 2.617
∞ 0.674 1.282 1.645 1.960 2.326 2.394 2.498 2.576

226
 TABLE IV: P ERCENTAGE P OINTS f α,n1 ,n2 FOR α = 0.05
n1
n2 1 2 3 4 5 6 7 8 9 10 12 15 20 25 30 40 60
1 161.50 199.50 215.70 224.60 230.20 234.00 236.80 238.90 240.50 241.90 243.90 246.00 248.00 249.30 250.10 251.10 252.20
2 18.51 19.00 19.16 19.25 19.30 19.33 19.35 19.37 19.38 19.40 19.41 19.43 19.45 19.46 19.46 19.47 19.48
3 10.13 9.55 9.28 9.12 9.01 8.94 8.89 8.85 8.81 8.79 8.74 8.70 8.66 8.63 8.62 8.59 8.57
4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 6.00 5.96 5.91 5.86 5.80 5.77 5.75 5.72 5.69
5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 4.77 4.74 4.68 4.62 4.56 4.52 4.50 4.46 4.43
6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 4.10 4.06 4.00 3.94 3.87 3.83 3.81 3.77 3.74
7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 3.68 3.64 3.57 3.51 3.44 3.40 3.38 3.34 3.30
8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 3.39 3.35 3.28 3.22 3.15 3.11 3.08 3.04 3.01
9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 3.18 3.14 3.07 3.01 2.94 2.89 2.86 2.83 2.79
10 4.96 4.10 3.71 3.48 3.33 3.22 3.14 3.07 3.02 2.98 2.91 2.85 2.77 2.73 2.70 2.66 2.62
11 4.84 3.98 3.59 3.36 3.20 3.09 3.01 2.95 2.90 2.85 2.79 2.72 2.65 2.60 2.57 2.53 2.49
12 4.75 3.89 3.49 3.26 3.11 3.00 2.91 2.85 2.80 2.75 2.69 2.62 2.54 2.50 2.47 2.43 2.38
13 4.67 3.81 3.41 3.18 3.03 2.92 2.83 2.77 2.71 2.67 2.60 2.53 2.46 2.41 2.38 2.34 2.30
14 4.60 3.74 3.34 3.11 2.96 2.85 2.76 2.70 2.65 2.60 2.53 2.46 2.39 2.34 2.31 2.27 2.22
15 4.54 3.68 3.29 3.06 2.90 2.79 2.71 2.64 2.59 2.54 2.48 2.40 2.33 2.28 2.25 2.20 2.16
227

16 4.49 3.63 3.24 3.01 2.85 2.74 2.66 2.59 2.54 2.49 2.42 2.35 2.28 2.23 2.19 2.15 2.11
17 4.45 3.59 3.20 2.96 2.81 2.70 2.61 2.55 2.49 2.45 2.38 2.31 2.23 2.18 2.15 2.10 2.06
18 4.41 3.55 3.16 2.93 2.77 2.66 2.58 2.51 2.46 2.41 2.34 2.27 2.19 2.14 2.11 2.06 2.02
19 4.38 3.52 3.13 2.90 2.74 2.63 2.54 2.48 2.42 2.38 2.31 2.23 2.16 2.11 2.07 2.03 1.98
20 4.35 3.49 3.10 2.87 2.71 2.60 2.51 2.45 2.39 2.35 2.28 2.20 2.12 2.07 2.04 1.99 1.95
21 4.32 3.47 3.07 2.84 2.68 2.57 2.49 2.42 2.37 2.32 2.25 2.18 2.10 2.05 2.01 1.96 1.92
22 4.30 3.44 3.05 2.82 2.66 2.55 2.46 2.40 2.34 2.30 2.23 2.15 2.07 2.02 1.98 1.94 1.89
23 4.28 3.42 3.03 2.80 2.64 2.53 2.44 2.37 2.32 2.27 2.20 2.13 2.05 2.00 1.96 1.91 1.86
24 4.26 3.40 3.01 2.78 2.62 2.51 2.42 2.36 2.30 2.25 2.18 2.11 2.03 1.97 1.94 1.89 1.84
25 4.24 3.39 2.99 2.76 2.60 2.49 2.40 2.34 2.28 2.24 2.16 2.09 2.01 1.96 1.92 1.87 1.82
26 4.23 3.37 2.98 2.74 2.59 2.47 2.39 2.32 2.27 2.22 2.15 2.07 1.99 1.94 1.90 1.85 1.80
27 4.21 3.35 2.96 2.73 2.57 2.46 2.37 2.31 2.25 2.20 2.13 2.06 1.97 1.92 1.88 1.84 1.79
28 4.20 3.34 2.95 2.71 2.56 2.45 2.36 2.29 2.24 2.19 2.12 2.04 1.96 1.91 1.87 1.82 1.77
29 4.18 3.33 2.93 2.70 2.55 2.43 2.35 2.28 2.22 2.18 2.10 2.03 1.94 1.89 1.85 1.81 1.75
30 4.17 3.32 2.92 2.69 2.53 2.42 2.33 2.27 2.21 2.16 2.09 2.01 1.93 1.88 1.84 1.79 1.74
40 4.08 3.23 2.84 2.61 2.45 2.34 2.25 2.18 2.12 2.08 2.00 1.92 1.84 1.78 1.74 1.69 1.64
60 4.00 3.15 2.76 2.53 2.37 2.25 2.17 2.10 2.04 1.99 1.92 1.84 1.75 1.69 1.65 1.59 1.53
120 3.92 3.07 2.68 2.45 2.29 2.18 2.09 2.02 1.96 1.91 1.83 1.75 1.66 1.60 1.55 1.50 1.43
∞ 3.84 3.00 2.61 2.37 2.21 2.10 2.01 1.94 1.88 1.83 1.75 1.67 1.57 1.51 1.46 1.39 1.32
 TABLE V: C ONTROL C HART FACTORS

XBAR control chart S control chart R control chart

40 6 15

38 5

36 4 10

34 3

32 2 5

30 1

0 0
28
2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20

Factors for Control Limits

Sample X-chart S-chart R-chart
Number (n) A2 A3 c4 B3 B4 d2 d3 D3 D4
2 1.880 2.659 0.798 0.000 3.267 1.128 0.853 0.000 3.267
3 1.023 1.954 0.886 0.000 2.568 1.693 0.888 0.000 2.575
4 0.729 1.628 0.921 0.000 2.266 2.059 0.880 0.000 2.282
5 0.577 1.427 0.940 0.000 2.089 2.326 0.864 0.000 2.114
6 0.483 1.287 0.952 0.030 1.970 2.534 0.848 0.000 2.004
7 0.419 1.182 0.959 0.118 1.882 2.704 0.833 0.076 1.924
8 0.373 1.099 0.965 0.185 1.815 2.847 0.820 0.136 1.864
9 0.337 1.032 0.969 0.239 1.761 2.970 0.808 0.184 1.816
10 0.308 0.975 0.973 0.284 1.716 3.078 0.797 0.223 1.777
11 0.285 0.927 0.975 0.321 1.679 3.173 0.787 0.256 1.744
12 0.266 0.886 0.978 0.354 1.646 3.258 0.778 0.283 1.717
13 0.249 0.850 0.979 0.382 1.618 3.336 0.770 0.307 1.693
14 0.235 0.817 0.981 0.406 1.594 3.407 0.763 0.328 1.672
15 0.223 0.789 0.982 0.428 1.572 3.472 0.756 0.347 1.653
16 0.212 0.763 0.984 0.448 1.552 3.532 0.750 0.363 1.637
17 0.203 0.739 0.985 0.466 1.534 3.588 0.744 0.378 1.622
18 0.194 0.718 0.985 0.482 1.518 3.640 0.739 0.391 1.609
19 0.187 0.698 0.986 0.497 1.503 3.689 0.733 0.404 1.596
20 0.180 0.680 0.987 0.510 1.490 3.735 0.729 0.415 1.585
21 0.173 0.663 0.988 0.523 1.477 3.778 0.724 0.425 1.575
22 0.167 0.647 0.988 0.534 1.466 3.819 0.720 0.435 1.565
23 0.162 0.633 0.989 0.545 1.455 3.858 0.716 0.443 1.557
24 0.157 0.619 0.989 0.555 1.445 3.895 0.712 0.452 1.548
25 0.153 0.606 0.990 0.565 1.435 3.931 0.708 0.459 1.541

228