Journal of Quality Technology

A Quarterly Journal of Methods, Applications and Related Topics

ISSN: 0022-4065 (Print) 2575-6230 (Online) Journal homepage: https://www.tandfonline.com/loi/ujqt20

Multivariate SPC Methods for Process and Product

Monitoring

Theodora Kourti & John F. MacGregor

To cite this article: Theodora Kourti & John F. MacGregor (1996) Multivariate SPC Methods
for Process and Product Monitoring, Journal of Quality Technology, 28:4, 409-428, DOI:
10.1080/00224065.1996.11979699

To link to this article: https://doi.org/10.1080/00224065.1996.11979699

Published online: 21 Feb 2018.

Submit your article to this journal

Article views: 8

View related articles

Citing articles: 46 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ujqt20
Multivariate SPC Methods for Process
and Product Monitoring
THEODORA KOURTI and JOHN F. MacGREGOR
McMaster University, Hamilton, ON L8S 4L7, Canada

Statistical process control methods for monitoring processes with multivariate measurements in both
the product quality variable space and the process variable space are considered. Traditional multivariate
control charts based on X2 and T 2 statistics are shown to be very effective for detecting events when the
multivariate space is not too large or ill-conditioned. Methods for detecting the variable(s) contributing
to the out-of-control signal of the multivariate chart are suggested. Newer approaches based on principal
component analysis and partial least squares are able to handle large ill-conditioned measurement spaces;
they also provide diagnostics which can point to possible assignable causes for the event. The methods
are illustrated on a simulated process of a high pressure low density polyethylene reactor, and examples of
their application to a variety of industrial processes are referenced.

Introduction Neither the process variables nor the product

quality variables are independent of one another.
'IT''HE objective of statistical process control (SPC) Only a few underlying events are driving a process
11 is to monitor the performance of a process over at any time, and measurements on all these variables
time in order to detect any unusual events that may are simply different reflections of the same underlying
occur. By finding assignable causes for them, im- events. Multivariate methods that treat all the data
provements in the process and in the product quality simultaneously can extract information on the direc-
can be achieved by eliminating the causes or improv- tionality of the process variations, that is on how all
ing the process or its operating procedures. Most the variables are behaving relative to one another.
SPC procedures currently in practice are based on When special events occur in processes they affect
charting only a small number of variables, usually not only the magnitude of the variation in variables
the final product quality variables, y. These ap- but also their relationship to each other (i.e., the di-
proaches are often inadequate for modern process in- rection of variation). These events are often difficult
dustries where massive amounts of data are being to detect by looking only at the magnitude of each
collected continually on perhaps hundreds of process process variable independently because the signal-
variables, x. Measurements on temperatures, pres- to-noise ratio is very low in each variable. Multi-
sure, flowrates etc., are available every few seconds. variate methods can extract confirming information
However, final product quality variables, y, such as from observations on many variables and can reduce
polymer molecular weights or melt index, cut points the noise levels through averaging.
in distillations, etc., are usually available on a much
less frequent basis. In any effective scheme for mon-
In this paper we examine the application of some
itoring and diagnosing operating performance, both
traditional multivariate statistical process control
process and product data should be used to extract
charts to monitoring process variables as well as fi-
information.
nal product quality variables. We present some new
developments, based on contribution plots, for de-
Dr. Kourti is a Senior Research Engineer in the McMaster tecting the variable(s) that caused an out-of-control
Advanced Control Consortium, in the Chemical Engineering
signal in multivariate charts. We then relate the tra-
Department.
ditional approaches for monitoring and diagnosis to
Dr. MacGregor is a Professor in the Chemical Engineering approaches based on multivariate statistical projec-
Department and in the McMaster Advanced Control Consor- tion methods such as principal component analysis
tium. He is a Member of ASQC. (PCA) and partial least squares (PLS). Both of the

Vol. 28, No.4, October 1996 409 Journal of Quality Technology

410 THEODORA KOURTI AND JOHN F. MacGREGOR

above approaches are applied for monitoring and di- When the in-control covariance matrix ~ is not
agnosis of a simulated case of a high pressure low known and must be estimated from a limited amount
density polyethylene (LDPE) reactor. The particu- of data, it is more appropriate to plot Hotelling's T 2
lar simulation has been chosen because it can serve statistic given by
as a common example to all the methods discussed T2 = (z - 1£)'S(z - 1£) (2)
in this paper. A large number of applications of the
projection methods to monitoring a variety of indus- where S is an estimate of E, An upper control limit
trial processes (from the petrochemical to polymer TJCL is then obtained based on the F distribution
and steel industry) are referenced in the text, so that and will depend upon the degrees of freedom avail-
readers may find examples closer to their interests. able for the estimate S (Alt (1985), Tracy, Young,
and Mason (1992), Wierda (1994a)). For a single
Traditional Multivariate new n x 1 multivariate observation vector and an es-
SPC Approaches timate S based on m past multivariate observations
2 (m 2 -l)n
The SPC approach for process monitoring, cur- TU CL = mm-n
( ) Fo(n, m - n) (3)
rently in practice in several industries, is to chart
a small number of variables, usually the final prod- where Fo(n,m-n) is the upper 1000% critical point
uct quality variables (y), and examine them one of the F distribution with (n, m - n) degrees of free-
at a time (Shewhart, cumulative sum (CUSUM) dom.
or exponentially-weighted moving average (EWMA)
charts). However, when the quality of a product is Alternatively, other types of multivariate charts,
defined by more than one property, all the proper- such as Multivariate CUSUM and Multivariate
ties should be studied collectively. Multivariate SPC EWMA charts may be used (Lowry, Woodall,
charts developed for this purpose have been based on Champ, and Rigdon (1992), Sparks (1992), Wierda
the chi-squared statistic or on Hotelling's T 2 statistic (1994a)).
(Hotelling (1947), Alt (1985), Jackson (1980, 1991), The T 2 in (2) can also be expressed in terms of the
Montgomery (1996)).
principal components of the multinormal variables
Given a vector of measurements z on n normally (Mardia, Kent, and Bibby (1979), Jackson (1991))
distributed variables, with an in-control covariance n t2 n t2
matrix ~, one can test whether the current mean of T 2 = ~ ....E.. = ~ ....!!.2 (4)
LJ A LJ 8
the multivariate process is at its population mean 1£ a=l a a=l a
by computing the statistic where Aa , a = 1,2, ... , n, are the eigenvalues of S,
X2 = (z -1£)'~-l(Z -1£). (1) and t; are the scores from the principal component
This statistic will be distributed as a central chi- transformation. 8~ is the variance of t a (the variances
squared distribution with n degrees of freedom if the of the principal components are the eigenvalues of S).
mean is equal to 1£. A multivariate chi-squared chart Each score t a can be expressed as
n
can be constructed by plotting X2 versus time with
an upper control limit (UCL) given by Xz',n, where t; = p~(z -1£) = LPa,j(Zj - J.Lj) (5)
j=l
o is an appropriate significance level for performing
the test (e.g., 0 = 0.01 or 0.05). The X2 statistic where Pa is the eigenvector of S corresponding to
in (1) represents the directed or weighted distance Aa, and Pa,j, Zj, J.Lj are elements of the corresponding
(Mahalanobis distance) of any point from 1£. vectors, associated with the jth variable. Therefore a
T 2 statistic on the initial set of variables is equivalent
For two variables Zl, Z2, if we set (1) equal to Xz, n to a T 2 statistic on the scores when all the principal
the solution on Zl, Z2-space is an ellipse centered ~t components are utilized.
1£. All the points on the ellipse have the same Ma-
halanobis distance, Xz, n' In this case, instead of us- A Chemical Process
ing the chi-squared ch~rt, one can plot the variables
against each other and check if the point falls within We consider here the first two zones of a multi-
the elliptical control region, with the same degree of zone tubular reactor for the production of LDPE. A
confidence-the perimeter of the ellipse corresponds review on LDPE processes can be found in Kiparis-
to the UCL of the chart (Shewhart (1931), Jackson sides, Veros, and MacGregor (1993). Details on the
(1956)). simulation can be found in Veros, Papadakis, and

Journal of Quality Technology Vol. 28, No.4, October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 411

Kiparissides (1993). The schematic of the reactor is TABLE 1. Variables in the LOPE Reactor
shown in Figure 1. Ethylene and a solvent are fed
together with the initiator to the first section. Ad- Process Variables
ditional injections of ethylene, solvent, and initiators Tm a x l T m ax 2 Maximum Temperature of the
are made between the sections. Each section is jack- Reactor Mixture (K)
eted and cooled by a cooling medium flowing counter- Tout I T out 2 Outlet Temperature of the
currently. The operating conditions in the reactor in- Reactor Mixture (K)
fluence the molecular properties of the polymer pro- Tcin l Tcin 2 Inlet Temperature of the Coolant (K)
duced (weight and number average molecular weights ZI Z2 Position of T maxI and T max2 in the
and long and short chain branching), and these in Reactor (% of Reactor Length)
turn affect the behavior of the polymer in its final Fi l F i 2 Flowrate of the Initiators (g/s)
application. The productivity variable of interest is Fs I F s 2 Flow of the Solvent (% of Ethylene)
the conversion per pass. None of the product prop- Tin Inlet Temperature of the
erties are measured on-line, and in practice many of Feed Mixture (K)
them are either not measured at all or are measured Press Pressure of the Reactor (atm)
infrequently. However, many on-line process mea-
surements x, such as the temperature profile down Product Quality Variables
the reactor, the coolant temperatures and the sol-
CONV Cumulative Conversion of Monomer
vent and initiator flowrates, are available on a fre-
MW n Number Average Molecular Weight
quent basis. The five product variables y of interest
are listed in Table 1, together with the process mea- MW w Weight Average Molecular Weight
surements x assumed to be available for this study. LCB Long Chain Branching/lOOO atom C
Although the entire temperature profile is usually SCB Short Chain Branching/lOOO atom C
available for each reactor section, we use a common
Indices 1 and 2 in the variables refer to
industrial practice of summarizing the profile in each
zones 1 and 2.
section by its inlet, maximum, and outlet tempera-
tures together with the position of the maximum. In
this paper we assume that the process and quality data are available in steady-state conditions. More
details on the choice of the variables for monitor-
ing this process can be found in MacGregor, Jaeckle,
Tcin1 Tcin2 Kiparissides, and Koutoudi (1994).
We will examine the behavior of various monitor-
ing tools in several cases, where problems developed
in different parts of the reactor:
Case A: The problem appears in zone 1, and zone 2
mayor may not be affected.
Case B: The problem appears in zone 2, and zone 1
was not affected.
Case C: The problem appears simultaneously in
Press
both zones.
Two very common industrial problems are consid-
Temperature
ered:
Tmax2 1. An increase of the impurity content of the feed.
Tmax1
Impurities in ethylene (concentrations at the parts
per million level, ppm) inhibit the polymerization
process.
2. Fouling of the reactor walls by sticky polymer
which impedes heat transfer and cooling of the
Location z
reactor.
Both impurities and fouling cannot be measured di-
FIGURE 1. Schematic of a Two Zone LOPE Reactor rectly. When these problems occur they affect several
with a Typical Temperature Profile. process variables and eventually the product quality.

Vol. 28, No.4, October 1996 Journal of Quality Technology

412 THEODORA KOURTI AND JOHN F. MacGREGOR

Their existence can therefore be detected by the ef- ties. Had we used univariate charts, only points 24,
fect they have on the process and quality variables. 25, 44, and 61-63 would have been detected as out
This effect is not a simple shift of mean of one or more of control; points 23, 42, 43, and 60 would have been
variables; both the magnitudes and the relationships missed. (The Bonferroni type of limits, discussed
of the variables to each other (i.e., direction of pro- below, have been used for the univariate charts.)
cess variability) will change.
Once a problem (or an unusual event) has been
In the following examples the type of problem that detected by the multivariate chart it is important to
occurred (i.e., assignable cause) will be determined diagnose an assignable cause for it. This involves two
from the group of process variables that are affected. steps: first find which variable(s) contributed to the
out-of-control signal and second determine what hap-
Monitoring and Diagnosis pened in the process to upset the behavior of these
in the Product Space variables.

Traditional multivariate charts and new develop- Isolation of Variables Responsible

ments on detecting the variables contributing to the for the Out-of-Control Signal
out-of-control signal, are first discussed for monitor-
ing product properties only. Figure 2 gives the T 2 Multivariate monitoring charts can detect an un-
chart for the five measured polymer properties (listed usual event but do not provide a reason for it. It
in Table 1) that define the quality of the LDPE prod- is difficult to determine which one of the n variables
uct. The unnumbered observations in this chart were or which subset of them is responsible for an out-of-
obtained by simulating normal operating conditions; control signal. Wierda (1994a) lists several attempts
the numbered observations correspond to simulated reported in the literature to solve this issue. We com-
problematic operation caused by increasing levels of ment on a few of these approaches, since most others
impurities in the ethylene feed in the reactor. The are variations of them.
dashed line (- - -) in the T 2 charts corresponds to the
99% limit and the dotted line (...) to the 95% limit. Murphy (1987) used a procedure similar to dis-
Observations 22-25 show the T 2 response when the criminant analysis, in which the original set of vari-
impurity levels increase in the feed of zone 1 only ables n is divided into subsets, and he tried to detect
(case A), 41-44 when impurities enter zone 2 only the subset nl of the variables that is responsible for
(case B), and 60-63 when impurities enter both zones the out-of-control signal. Usually there is no a priori
(case C). Notice that the presence of impurities in the knowledge of how to select subsets, but several sug-
ethylene feed had an effect on the product, and that gestions are summarized by Wierda (1994a). This
this effect could be detected in all cases, by following procedure is lengthy, especially if several variables
the T 2 calculated from the product quality proper- are involved.

IMPURITIES: MonitoringY Space Wierda (1994b) discusses using the "step-down"

140
(I)
procedure instead of the Hotelling's T 2 chart. The
63
5 120
procedure requires ordering the variables in subsets.

~
It checks for a shift in the mean of a subset given that
100 the previous subset was acceptable. The procedure
case C requires prior knowledge on the distribution of the
>< so
2 means of the variables. Variables whose mean values

~
caseB
case A
rarely shift are put first in the order. A review of the
60
step-down procedure can also be found in Marden
CI
s and Perlman (1990). Other approaches to detection
and diagnosis of multivariate quality control data
'Eo< based on regression adjustments have been proposed
(Hawkins (1993), Wade and Woodall (1993), Zhang
70
(1985)). They rely upon plotting various residuals
OBSERVATIONS of quality variables after trying to remove the effects
of other out-of-control variables by a regression. Re-
FIGURE 2. T 2 Chart on the Quality Variables. Distur- cently Mason, Tracy, and Young (1995) suggested
bances are due to Impurities. a method for decomposing T 2 and showed how the

Journal of Quality Technology Vol. 28. No.4. October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 413

procedure suggested by Murphy and the step-down guidelines to help determine large errors. Any vari-
approach are subsets of their method. They demon- ables close to these limits should be investigated. In
strate its applications for cases where three to five what follows we give several suggestions for detecting
quality properties are involved. All of these proce- the variables that contributed to the out-of-control
dures may not be practical when a large number of signal.
variables are involved, or when there is no clear hi-
erarchical structure (ordering). Bar Plots of Normalized Errors
of the Variables
In 1980, a diagnostic approach was introduced by
With this approach n univariate plots are being
Jackson, in which if an unusual event was detected
by the T 2 chart, univariate charts for both the nor- replaced with one graph. When an observation Xk
malized peA scores (tal Sa) and the variables were falls out of limits in the multivariate chart, the nor-
consulted to diagnose the cause of the alarm. If there malized error for each variable j is calculated as
are only a few variables involved, then by looking at (Xk,j - /-Lj)/sj,
their univariate charts one could detect a gross error.
When the scores have some physical significance then and these normalized errors are plotted on a common
a group of variables out of control may affect only a graph. The variables with the highest normalized de-
specific principal component and from physical inter- viation(s) for a given observation can be detected at
pretation of the component one could diagnose the a glance. Fuchs and Benjamini (1994) have recently
problem. The idea of monitoring scores was criti- proposed multivariate profile charts where both the
cized as being of little value if there is no physical T 2 statistic and the bar plots of the normalized vari-
significance to the scores; since each component is a ables are displayed in a very efficient way for visual
linear combination of the original variables it is not inspection. Figure 3a shows the normalized errors
always easy to interpret the results. However, this is for the five quality variables for point 25 of Figure 2.
no longer the case if one goes one step further and When the value of the normalized variable exceeds
utilizes the contribution plots, discussed below. ±3, the variable plots out of the ±3<7 in a univariate
chart. For this example, MW w plots out of ±3<7. Fig-
Alt (1985) suggested using univariate "Shewhart- ure 3b shows the normalized errors plot for point 23
like" charts for the individual variables but with which was detected out of control in the T 2 chart of
Bonferroni-type limits (Le., replace 0:/2 with 0:/(2n) Figure 2. Notice that no normalized error is above 3.
in the control limit calculation), to check if it is pos- The reason that normalized error plots cannot detect
sible to isolate the variables that have caused the the problem for point 23 is that the error in MW w
problems. This is a method of opening up the limits is still small, and the variables are highly correlated.
to get the desired type I error. They are conservative Using score plots, described below, will overcome this
bounds yielding a type I error of at most 0:. These and generally detect the problem much more readily.
bounds would handle the type I error problem for in-
dependent variables but not for correlated variables. Bar Plots of Normalized Scores
Alt (1985) also emphasized that all the diagnostic This approach is preferable for correlated vari-
approaches should follow the T 2 chart and not re- ables. The normalized scores (tal Sa) of Xk are plot-
place it. Jackson (1991) modified his 1980 procedure ted on a common graph, and normalized scores with
and utilized univariate score plots with Bonferroni high values are detected. Since the scores are inde-
bounds for diagnosis; since scores are independent pendent, Bonferroni-type limits could also be used
these bounds solve the type I error problem. as guides for "high values". Once the score(s) that
contribute to the out of limit T 2 signal are detected,
Many of the above diagnostic procedures have
contribution plots (discussed below) can be used to
been concerned with providing correct type I error
find the quality variables, numerically responsible for
limits. However, once a multivariate chart has de-
the signal.
tected an event, there is no longer a need to be con-
cerned with precise control limits which control the For point 23 of Figure 2 the normalized scores are
type I error (0:) on the univariate charts. A devi- shown in Figure 4a. Notice that scores 2 and 5 have
ation at the chosen level of significance has already relatively high values, in spite of the fact that the
been detected. Diagnostic univariate plots are only individual normalized quality variables (Figure 3b)
used to help decide on the variables which are causing are not exceptionally large. Again, the limits for the
the deviation. Therefore limits at 3<7 are just rough normalized scores are roughly used as guides here,

Vol. 28, No.4, October 1996 Journal of Quality Technology

414 THEODORA KOURTI AND JOHN F. MacGREGOR

Impurities,Y space, nonna!.error for point 25 Impurities, Scores in Product, Score Norm. Error, Point 23
2

lt1
N I
MWn
~ 0
1

I
Q
§
I

-2
CONV LCB
8CB
I
I

3
~ -3 MWw

~
Z
4 2

-5
4 5 2 4

QUALITY VARIABLES COMPONENTS IN Y SPACE

(a)
(a)
Impurities, Y space,normal, errorfor poinl23

Impurities. contribution of Y vuriables on Score5. poiDI23

«> 9000
N 0.5
MWn
~
It"l
~8000
0
o 7000
o~ ~.5 8CB U
CI)
..
~ LCB 6000
~ -1 CONV ~ MWw
~
5000

~
-1.5

-2
~ 4000

3000

I
o MWw
Z -2.5 2000

1000
-3
4 u
QUALITY VARIABLES
o
2 4
(b) QUALITY VARIABLES
FIGURE 3. Normalized Errors for the Quality Variables, (b)
for Observations of Figure 2, (a) for Point 25 and (b) for
Impurities. contribution of Y variables on Score 2, point 23
Point 23. 0.03

C'l
but not for precise control of type I error; 2.7a would ~ 0.02 LCB
have been the value equivalent to Bonferroni limits 8 0.0 I
for 95% confidence on type I error for five variables CI)

(Alt (1985)). ~ SCB

Z 0
Jackson (1991) reports a somewhat similar pro- o
cedure suggested to him by R. Thomas (Burroughs ~ -0.0 I
MWn
Corp.) during a short course: "represent the value of
Hotelling's T 2 of an observation in a histogram form !Z; ~.02 MWw
by plotting the individual contributions of the scores o
U
squared (tal sa)2-stack bar graph-to indicate the ~.03
4
nature of the out-of-control situations". Here, by in- QUALITY VARIABLES
stead using the normalized scores talSa, Bonferroni
(C)
limits can be used on the chart as rough guidelines
for detecting large talSa values.
FIGURE 4. Fault Diagnosis using Scores in Quality
Variable Contributions to Individual Scores Space. (a) Normalized Scores for Point 23 of Figure 2,
(b) Variable Contributions to Score 5, and (c) Variable Con-
One set of diagnostic tools, recently suggested by tributions to Score 2.

Journal of Quality Technology Vol. 28, No.4, October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 415

Miller, Swanson, and Heckler (1993) and MacGregor normalized score (t a /sa )2
et al. (1994) are contribution plots on scores. A con-
tribution plot indicates how each variable involved in conta,j = t~Pa,j(Zj - /Lj) = ~a Pa,j(Zj - /Lj) (7a)
Sa Aa
the calculation of that score contributes to it. Contri-
bution plots can be constructed for normalized scores ii, conta,j is set equal to zero if it is negative (i.e.,
with high values; the variables on this plot that ap- its sign is opposite to the value of the score t a ) .
pear to have the largest contributions to it, should 2. Calculate the total contribution of variable Zj
be investigated.
K
From (5), the contribution of each variable Zj to
CONTj = ~)conta,j). (7b)
the score of the principal component a is
a=l

(6) Using this procedure on the scores 2 and 5 of Figure

4a, one would have obtained MW w as the responsible
The variable contribution plots for scores 5 and 2 variable for the out of limits signal of point 23 in
of Figure 4a are shown in Figures 4b and 4c, re- Figure 3. Another example where this procedure is
spectively. Variable MW w is mainly responsible for applied is shown later in Figures 7a and 7b.
score 5; variables MW wand LCB have high values Notice from (4) and (5) that T 2 can be written
for score 2. However score 2 has a negative value as a sum of the contributions of all variables over all
(see Figure 4a); it is this value (squared) that con- the scores
tributes to the calculation of the T 2 • Therefore vari-
ables with negative contributions are more significant
in driving this value high; positive contributions will
only make it smaller. Here we suggest that variables
with high contributions but with the same sign as the
score should be investigated. For the case of score 2,
high negative contributions are important and that
leads to variable MW ui- (From simulations of this It has been suggested (Nomikos (1995)) that a
particular process, indeed, we confirm that once im- graph with the contributions of each variable to T 2
purities keep building up it is variable MW w that could be constructed; these contributions could be
will be mostly affected.) positive or negative. However, from our experience,
The variable contribution plots together with the determining the variables responsible for the out-of-
score plots is a powerful tool of detecting variables control signal from such a graph is ambiguous. The
contributing to the out-of-control signal. Notice that discriminating ability of the approach described by
although T 2 signaled for point 23, the univariate (7a) and (7b) is more powerful; it unravels the vari-
charts on the quality variables could not detect the able(s) responsible for an increase in both the value
problem with MW ur- However the combination of of T 2 and the values of the high normalized scores.
the score plots and the variable contribution plots (The approach in (7) is not equivalent to plotting
did. all the contributions from (8) and keeping the ones
with positive values; rather to plotting contributions
Overall Average Variable Contributions in (8) with the summation over the high scores only
Very often more than one score may have high and keeping the positive values.)
values. Therefore it is suggested here that an "overall General Diagnostic Procedure
average contribution" per variable is calculated, over
all the scores with high values. The approach for diagnosing the variable(s) re-
sponsible for an out-of-control signal in a multivari-
For each one of the normalized scores with high ate T 2 chart, initially suggested by Jackson (1980)
values (say above 2.5) calculate the variable contribu- could be extended and utilized as follows:
tion, but keep only the contributions with the same 1. Out-of-limit signal detected: T 2 value for obser-
sign as the score. Sum the values over all the K high vation Xk, above T6cL (see (3)).
scores, K :$ n as shown below: 2. Check normalized scores of observation Xk: find
1. Repeat for all the K high scores K :$ n. scores with high values. (Bonferroni limits could
i. Calculate contribution of a variable Zj in the be used on the score charts as rough guides; if Q is

Vol. 28, No.4, October 1996 Journal of Quality Technology

416 THEODORA KOURTI AND JOHN F. MacGREGOR

the significance level for T 2 , use a I n for the score process data may contain information about these
charts). events.
3. Calculate the variable contributions for these high Finally, even if product quality measurements are
scores (see (6) and (7)); investigate variables with frequently available, monitoring the process may fa-
high contributions. cilitate assigning causes for an event. When monitor-
With this approach one does not necessarily have ing product quality only, even if we determine which
to plot the normalized scores and the variable con- quality variable was "unusual" and contributed to
tributions and consult graphs. The whole procedure the out-of-control signal, it may still be difficult to
can be easily programmed in a computer. In step determine what went wrong in the process. For ex-
2, the individual scores can be checked against some ample, in the LDPE process by following the product
prespecified limits, and in step 3 the variable contri- variables it was determined that for point 23, MW w
butions can then be calculated for the high scores. is the major contributor to the out-of-control signal.
However, there may be several reasons (combinations
With this procedure there is no need to interpret of process conditions) that might have caused molec-
the scores for physical significance; their role is to ular weight to change. Therefore although we have
facilitate the calculation of variables contributing to answered the question "Which variable is responsible
the large value of T 2 • In general this approach will for the out-of-control signal?" , it is not always easy to
point to a variable or a group of variables that con- answer the question "What process conditions have
tribute numerically to the out-of-control signal; these caused this?" Monitoring the process would bring us
variables and any variables highly correlated with closer to the answer.
them should be investigated in order to assign causes.
In what follows we show how to utilize traditional
Monitoring and Diagnosing approaches and extend their application to monitor-
in the Process Space ing process variables. The values of the process vari-
ables collected from historical data and correspond-
Traditionally the product quality properties are ing to conditions that produced acceptable product
charted to determine if the process is in a state of are used as a training set to calculate the mean and
"statistical control". However, product quality data covariance matrix. Then each new set of process
may not be available frequently, but only every few variables is checked against the "in-control" condi-
hours. If a problem develops in the process, out-of- tions. By looking at the process as well as the quality
control product is being produced and will go unno- variables we are truly doing statistical process con-
ticed at least until the results from the laboratory trol (SPC), as opposed to statistical quality control
become available. In the LDPE example none of the (SQC).
product properties are available on-line. However,
many on-line process measurements such as the tem- Figure 5 shows the response of the T 2 calculated
perature profile down the reactor, the coolant tem- for the fourteen process variables in the LDPE reac-
peratures, and the solvent and initiator flowrates are tor (see Table 1) for the same simulated operating
available on a frequent basis. The signature of any conditions as those of Figure 2. Observations 22-25
special events or faults occurring in the process that show the T 2 values when increasing levels of impu-
will eventually affect the product should also appear rities enter zone 1 (case A); 41-44 when impurities
in these process data (x). Therefore monitoring the enter zone 2 only (case B), and observations 60-63
process may be preferable. when impurities enter simultaneously in both zones
(case C). Notice that, for all the cases, the upset in
There are several other reasons why monitoring the process that eventually led to the upset in the
the process is advantageous. Sometimes, only a few product quality was detected by monitoring the T 2
properties of the product are measured, but these are statistic of the process variables only. In fact the up-
not sufficient to define entirely the product quality. set is detected earlier (Le., at a lower impurity con-
For example, only the relative viscosity (RV) may centration) by monitoring the process; notice that
be charted in nylon production, although there are points 22 and 41 with small amounts of impurities
other properties (amine end groups) that affect the are detected as problematic in the process, although
dye properties of the product. If process problems in the product they are around or even below the
that affect the amine groups occur, they will not be 95% limit. Furthermore the T 2 statistic on the pro-
detected by following RV only. In these cases the cess variables detects the problem on-line and one

Journal of Quality Technology Vol. 28, No.4, October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 417

IMPURITIES:MonitoringX Space. 14 Variables

combination of variables appears to have problems,
rn is that something has affected the extent of the reac-

i:
~soo
44 case C
tion. The problem (assignable cause) could be either
in a decrease in the efficiency of the initiator or in
an increase in the level of impurities in the reactor.
rn coseB The contribution plots may not explicitly reveal the
eo
~300
400
case A
25
2 cause of the event, but they point to the group of
process variables that are no longer consistent with
l:I.. normal operating conditions, and thereby focus the
~200 4
attention of the operators or the engineers and allow
~IOO 3 2 J~ them to use process knowledge to deduce possible
99.0%
o ."••••••" 1
•••••" .••"" ""."....." ..." •." ...." ..""".."..."..
causes.
o w w ~ ~ ~ 00 70
OBSERVATIONS Blocking the Process Variables

FIGURE 5. T 2 Chart on the Process Variables. Distur- When dealing with several sections of a unit or
bances are due to Impurities. even several units, one may have to deal with many
process variables. In these cases it may make sense
to break the process variables into logical sections
does not have to wait for the infrequent laboratory within which the variables are highly correlated but
tests on the quality properties. Once a problem (or
between which there is much less correlation, and
an unusual event) has been detected by the multivari- examine the T 2 response for the variables of each
ate chart on the process variables one can diagnose
section separately. These sections could correspond
an assignable cause for it. The procedures described
to physical units, or even sections of a unit.
earlier for identifying the variables contributing to
an out-of-control signal can be applied for multivari- In the LDPE example, since the process consists of
ate charts on process variables. Once we determine two zones, the process variables can be separated into
the process variables responsible for an out-of-control two groups and one may monitor the T 2 response for
signal it is much easier to diagnose sources for the each zone separately. The inlet temperature of the
process deviation, as illustrated below. feed mixtures in each zone and the pressure are com-
mon in both of the sections, hence these two vari-
The normalized errors for all fourteen process vari- ables were included in both groups, such that each
ables for point 23 of Figure 5 are plotted in Figure zone is described by eight process variables (see Ta-
6. Two variables seem to have large errors: the lo- ble 1). The objective of this variable separation is to
cation of the hot spot in zone 1, ZI, has moved fur- isolate and more easily diagnose a problem when it
ther up the reactor and the maximum temperature, occurs. Each chart gives the picture of the operation
T maxI has decreased. Because there are strong cor-
relations among the process variables, plots of nor- Impurities, X space. normal.error forpoint23
4
malized errors may not be sufficient to detect all the i l
variables with "unusual behavior". Figure 7a shows
the normalized scores for point 23. The variable con-
tributions for scores 1, 2, 3, and 5 which lie beyond
the 30' limits are plotted in Figure 7b. The max-
1
imum temperature in the reactor in the first zone
Tmax l . and the location of the hot spot in that zone,
ZI appear to be variables with important contribu- ~
n n '--L.....J '--'----J

tions, together with the temperature of the coolant 1

in the reactor, Tc-.inI, and the outlet temperature of TMAX1

the reactor mixture, T out l ' The combination of the L-J
normalized scores and the variable contribution plots 3
o 2 4 6 10 12 14 16
uncovered two more variables with "unusual" behav-
PROCESS VARIABLES
ior than those detected by the normalized errors of
the variables. From knowledge of the process and FIGURE 6. Normalized Errors for the Process Variables
polymerization histories the explanation when this for Point 23 of Figure 5.

Vol. 28, No.4, October 1996 Journal of Quality Technology

418 THEODORA KOURTI AND JOHN F. MacGREGOR

Impurities, Scorecontributions, Poinl23

IO.---~-~-~--~-~-~--~------, zone 1, for these observations. However, the effect
of the impurities is not expected to propagate from
~
zone to zone since they are consumed in the section
~ 6
which they enter (zone 1) and have no effect on zone
2 as it is correctly shown on the graph.

~ 4
4
Once an out-of-control signal is detected in the
CI) 2 chart of the blocked variables one can proceed with

~
9
diagnosis as discussed earlier. For zone 1 and ob-
Oh-,-j-----L--'---+-r;,------,dY---'--+-t1T12F=!.!c::!:1r------j servation 23, the variables contributions (not shown
here) for the high normalized scores (scores calcu-
~ -2
7
lated for variables of zone 1 only) show that variables
~ I. "2- ......•.•..........•.••••••••••...........•••••.....••••••••.
-4L.._~-~--~-~-~---,':-----,~_..::' Tm a x 1 : Zl, TC-inb and Tout! are mainly responsible for
o 4 10 12 14 16
COMPONENTS IN X SPACE
the out-of-control signal.

(a) By separating the variables into groups we were

able to clearly isolate the section with the problem.
Impurities. 0lntributi0ll orxvariables on _1,2,3" poinI23 1MPU1UTIES: Monitoring X s_ olZONB I. 8 Varilblao

TMAX1 caseC

caseA

2 4 10 12 14 16

PROCESS VARIABLES 70
OBSERVATIONS
(b)
(a)
FIGURE 7. Fault Diagnosis using Scores in Process
IMPURrrII!S: MonilOrina X SpaceofZONB 2, 8 Variables
Space. (a) Normalized Scores for Point 23 of Figure 5,
44
63
and (b) Variable Contributions to all High Scores.

history for each individual zone. Figure 8a shows coseC

the T 2 chart for zone 1 only, for the same simulated
conditions of Figures 2 and 5. Notice that the chart caseS

indicates that zone 1 appears to have problems for

points 22-25 and 60-63. That is, for cases A and
C, indeed a disturbance (impurities in the ethylene
feed) went through zone 1 in both cases. For points
41-44 (case B) no abnormality is detected for zone
1. During this time impurities entered zone 2 but
there was no effect on the process variables of zone 1 70

(their values should remain within normal operating OBSERVATIONS

range), and therefore the T 2 calculated from the pro- (b)
cess variables of zone 1 should remain within limits.
Figure 8b shows the T 2 chart for zone 2. Notice that FIGURE 8. T 2 Charts on Blocks of Process Variables
the chart indicates that there is nothing unusual for in the LOPE Reactor. Disturbances are due to Impurities,
points 22-25 in zone 2. Impurities had entered in (a) Zone 1 and (b) Zone 2.

Journal of Quality Technology Vol. 28, No.4, October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 419

Of course there are processes in which a problem in observations correspond to normal operating condi-
one section may cause operational problems in subse- tions. Notice from Figure 9a, that for this particular
quent sections. In such cases the T 2 chart approach case, fouling in zone 1 had a significant effect on the
should be utilized to detect the problem in the ear- product properties; fouling in zone 2 had a moderate
liest (first) section in which it appears. This is illus- effect (brings T 2 above the 95% limits but below the
trated with the following example, where the distur- 99%). The T 2 chart on all the process variables (Fig-
bance is an increasing level of fouling of the reactor ure 9b) detected the problems for all cases (including
walls. case B, where it shows that observations 42-46 are
out of the normal operating region, although they
Figures 9a-9d give the T 2 charts on the product had a moderate effect on the product). Monitoring
variables y and on the process variables x for both zone 1 only (Figure 9c) indicates correctly that there
zones, as well as the T 2 charts for zone 1 and zone were problems in observations 22-26 and 62-66. Fig-
2, for a different type of disturbance. In this exam- ure 9d indicates that zone 2 experienced disturbances
ple, observations 22-26 simulate process conditions in all three cases. Even though fouling of the walls
when fouling of the wall occurs in zone 1 (case A); was not observed in zone 2 for case A, the correla-
for observations 42-46 fouling occurs in zone 2 (case tions of the variables in zone 2 are slightly affected by
B); and for observations 62-66 fouling occurs simul- the disturbance that occurred in zone 1 (Le., fouling
taneously in both zones (case C). The unnumbered effects are carried through). However, by monitoring

FOULING: Monitoring Y Space FOULING: Monitoring X Space. 14 Variables

3000
V,)
V,) 66
~ 140 W

~:
~ 120

~ 100
> cueC
>< V,)
~ 80
~I~
:;!
::> 60 ~ 46
CI 8: 1000
~ cue A
40
~
~ soo
~
99.0%
70 00 10 20 70
OBSERVATIONS
(a)
FOULING: Monitoring X Spaceof ZONEI, 8 Variables FOULING: Monitoring X Spaceof ZONE2, 8 Variable.
N 900

N
~ BOO

rn 700
~ 600

~
4
66
cueC
soo cueC
S

> 400
cueB
caseA V,)
V,)

~
300

100. 2 cue B 2
8: 200 cue A

Z 100
"~~'!!~"""'" L 42>46·····.. •• ""~1. . 0
o •• ~,g~ ••••.
00 10 20 30 40 so 60 70 ~ o 10 70

OBSERVATIONS OBSERVATIONS
(e) (d)

FIGURE 9. Process Monitoring in the LDPE Reactor, using T 2 Charts. The Disturbance is Fouling. (a) T 2 Chart on
Quality Variables, (b) T 2 Chart on Process Variables, (c) T 2 Chart on Blocked Process Variables, Zone 1, and (d) T 2 Chart
on Blocked Process Variables, Zone 2.

Vol. 28, No.4, October 1996 Journal of Quality Technology

420 THEODORA KOURTI AND JOHN F. MacGREGOR

the sections separately we were able to detect the In the PCA approach a set of data Z is collected
earliest section of the unit (zone 1) that had prob- when the process was known to operate "in control" .
lems. This set can be either process (Z = X) or product
quality (Z = Y) variables. The method is equivalent
Blocking the variables and monitoring a separate to performing principal component transformation to
T 2 per block (unit) is a good approach when there are the original set of data Z (m x n) and keeping only the
not significant correlations between the variables of A (out of n) principal components with the highest
the different blocks. If there are correlations between variation. This way the original data set is projected
two or three blocks (i.e., recycles), an independent to an orthogonal space, which, in general, is of a
T 2 per block will still give a good indication of the lower dimension (I.e., A « n)
performance of the block; but an overall T 2 (global
A n
T 2 ) that includes the variables from all the blocks is
needed to monitor the performance of the plant (l.e., Z= L taP~ + L taP~ = Z + E. (9)
a=l a=A+l
to detect inconsistent behavior among blocks).
The location of this A-dimensional space with re-
Projection Methods spect to the original coordinates is given by the load-
ings p.,, which are the eigenvectors, corresponding to
A problem with utilizing traditional X2 and T 2 ap- the A largest eigenvalues of the covariance matrix of
proaches is that they may not be practical for very the original data set Z. Z indicates the estimate of Z
high-dimensional systems with collinearities. Many based on the first A principal components. The value
highly collinear variables lead to ill-conditioned data of A can be determined by cross-validation (Wold
which implies that X2 or T 2 becomes very sensitive to (1978)). The location of the projection of an obser-
deviations of the components corresponding to small vation onto the A-dimensional space is given by its
eigenvalues. The inversion of the covariance matrix score t«, which are linear combinations of the origi-
(needed for the calculation of X2 or T 2 ) may be a nal variables. The squared perpendicular distance of
problem. Breaking up the process into sections may an observation z, from the projection space, called
resolve the problem of dealing with large systems; squared prediction error (SPE) , gives a measure of
but this is not always feasible as discussed earlier how close the observation is to this A-dimensional
in systems with correlated units. A common proce- space:
dure for reducing the dimensionality of the variable n n
space is the use of projection methods like principal SPEZ i = L e~j = L(Zij - Z;j)2 (10)
component analysis (PCA) or partial least squares j=l j=l
(PLS). Another problem with using the T2 approach where eij is an element of the i t h row of matrix E
(either in the x- or y-space) is that it cannot easily and Z,j is calculated from the PCA model with A
handle missing data. If a sensor fails, or an analysis components. SPE measures the contribution of the
on one or more of the variables is messed up, then components that have not been utilized.
the whole set of good measurements on all the other
variables cannot be used. However the available good From data collected when the process is "in con-
measurements might still be a sufficient indication of trol" , the loadings corresponding to the normal oper-
the performance for that observation. Multivariate ating region are determined. For each new observa-
projection methods handle missing data more easily tion, Znew, the scores and the SPE are calculated and
and utilize the remaining information to test for an checked against "in-control" operation. For monitor-
out-of-control situation. ing the scores, T 2 charts based on the first A princi-
pal components are constructed:
The basic concepts and algorithms of PCA and
PLS (Geladi and Kowalski (1986)) and their use (11)
in multivariate monitoring of process operating per-
formance (Wise and Ricker (1989), Wise (1991),
An upper control limit can be calculated from (3)
Kresta, MacGregor, and Marlin (1991), Skagerberg,
with (A, m - A) degrees of freedom.
MacGregor, and Kiparissides (1992), MacGregor et
al. (1994), Nomikos and MacGregor (1994, 1995), For the SPE a chart is also constructed,
MacGregor and Kourti (1995)) have been extensively n
presented in the literature. A brief description of the SPEz n e w = L(Znew,j - Znew,j)2 (12)
methods is given below. j=l

Journal of Quality Technology Vol. 28, No.4, October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 421

where The use of the first A principal components with

a=A T 2 is discussed by Fuchs and Benjamini (1994). How-
znew = L tnew,a Pa ever they use a full-rank T 2 chart to detect out-of-
a=1 control observations and then examine normalized
is computed from the reference PCA model. Up- scores of the first A principal components to diagnose
per control limits for the SPE statistic can be com- the problem; they rely on the physical interpretation
puted using approximate results for the distribution of the principal components for an out-of-control sig-
of quadratic forms (Jackson (1991), Nomikos and nal and not on the contribution plots discussed be-
MacGregor (1995)). This is also often referred to low.
as a Q statistic (Jackson (1991)). The PCA approach for monitoring process vari-
The above procedure can be used for monitoring ables is used when product quality data are not avail-
either process variables (Z = X) or product quality able in the historical data set. In the PLS approach
variables (Z = V). Both SPE and Tl are necessary for monitoring process variables the model is devel-
for monitoring. The statistic on the scores checks if oped from historical data sets with measurements
a new observation remains within the normal operat- from both the process, X, and the quality, Y, vari-
ing region in the projection space, and SPE checks if ables obtained during in-control operation. PLS si-
the distance of the new observation from the projec- multaneously reduces the dimensions of X and Y,
tion space is within acceptable limits. If a totally new to find latent vectors for X that not only explain the
variation in the process data, but that variation in X
type of special event occurs which was not present
which is most predictive of the product quality data,
in the reference data used to develop the in-control
Y. This is accomplished by working on the sample
PCA model, then new components will appear and
covariance X'YY'X matrix. In the most common
the new observations will move away from the plane.
version of PLS (Hoskuldsson (1988)), the first PLS
Such new events can be detected by plotting the SPE.
latent variable tl = w~ x is that linear combination
One of the advantages of using Tl rather the tra- of the x variables that maximizes the covariance be-
ditional T 2 in cases when the variables are highly tween it and Y. The first PLS loading vector WI is
correlated and ~ is ill-conditioned, is illustrated here. the first eigenvector of the sample covariance ma-
Note that the traditional Hotelling T 2 in (4) can be trix X'YY'X. Once the scores tl = XWI for the
written as: first component have been computed (i.e., the values
of that latent variable for the n observations) the
A 2 n 2 n 2
columns of X are regressed on t l to give a regression
~ ~.
a a 2
T2 = '""
~ S2t + '""
~ s2t = T A + '"" s2 (13)
or loading vector
a=1 a a=A+I a a=A+I a
X't l
By scaling each t; by the reciprocal of its variance PI = t~tl
each term plays an equal role in the computation of
T 2 irrespective of the amount of variance it explains and the X matrix is deflated to give residuals
in the Z matrix. When the number of variables n is
X 2 = X - tIP~.
large, ~ is often singular and cannot be inverted, nor
can all the eigenvectors be obtained. Even if ~ can Y is also deflated as
be inverted, the last components a = A + 1, ... , n Y 2 = Y - tIq~
in (13) explain very little of the variance of Z and
generally represent random noise. By dividing these where ql is obtained by regressing the columns of Y
ta's by their very small variances, slight deviations on tl,
in these ta's (sometimes even due to round-off error) , t~Y
which have almost no effect on Z will be magnified ql=t~tl'
and lead to an out-of-control signal in T 2 • Therefore The second latent variable is then computed as t2 =
Tl based on the first A cross-validated components w~x where W2 is the first eigenvector of X~ Y 2Y~X2
provides a test for deviations in the variables that and so on. As in PCA, both the score vectors
are of greatest importance to the variance of Z. The (t l, t2,"') and the loading vectors (WI, W2,"') are
NIPALS algorithm can calculate sequentially the first orthogonal. For large ill-conditioned data sets, it is
A principal components without calculating all the usually convenient to calculate the PLS latent vari-
eigenvalues (Geladi and Kowalski (1986)). ables sequentially via the NIPALS algorithm (Geladi

Vol. 28, No.4, October 1996 Journal of Quality Technology

422 THEODORA KOURTI AND JOHN F. MacGREGOR

and Kowalski (1986)) and to stop based on cross- Any periods containing variations arising from spe-
validation criteria. An advantage of the NIPALS cial events that one would like to detect in the future
algorithm is that it handles missing data (i.e., ob- are omitted at this stage. The choice of this refer-
servation vectors from which some variable measure- ence set is critical to the successful application of the
ments are missing) in a very simple and straightfor- procedure, as discussed by Kresta, MacGregor, and
ward way (Kresta, MacGregor, and Marlin (1991), Marlin (1991).
Nelson, MacGregor, and Taylor (1996)).
A PLS model was built to relate the fourteen pro-
In the modelling stage, Pa's, W a 's, and qa's are cess variables, X, with the five quality variables, Y,
calculated sequentially as discussed. In the monitor- under normal operating conditions of the LDPE re-
ing stage, for each new observation, X new , the ta's are actor. Three latent variables could explain 90% in
calculated (using Pa's, wa's, and qa's) sequentially the Y. The 99% limits for the scores and the SPE
by deflating X new each time the score for a compo- were defined; then the monitoring scheme was tested
nent has been calculated. The multivariate control for simulated cases of fouling. Figure 10 shows the
chart is again a Tl chart on the first A latent vari- behavior of the process on a projection of the pro-
ables, t«, a = 1, ... , A; the difference with PCA is cess variables on the plane defined by the two la-
that these latent variables have been computed uti- tent variables tz and h. The unnumbered asterisks
lizing information from both the X and Y. There- denote normal operation. The numbered ones show
fore when using PLS, Tl monitors the variation in the process behavior under fouling. The ellipses show
the process variables which is more relevant to the confidence limits (99% solid line, 95% dotted line).
product quality variables (Kresta, MacGregor, and Fouling occurred for observations 22-26, 42-46, and
Marlin (1991)). A chart on the squared prediction 62-66, as described earlier for Figures 9a-9d. Dur-
x
error in the x-space, SPEx ne w (12), where new is now ing normal operation the projected points move ran-
computed from the reference PLS model, is used to domly within the limits. Notice, however, that dur-
detect if the covariance structure of the process has ing an "unusual" event the direction of the process
changed. A change in the pattern needs investiga- (i.e., the relationship among the variables) is chang-
tion because it means that the process conditions ing; different events (i.e., increasing fouling at differ-
are now different than the time that the model was ent sections of a unit) have different direction. The
developed. This change mayor may not have effects Tj chart for the three latent variables is shown in
on the product. Control limits for the Tl and SPEx Figure Ll a and the SPE chart in Figure Ll b. No-
charts are chosen in the same manner as previously tice that by monitoring the Tj chart on the scores
discussed. In both PCA and PLS the scores t; are and the SPE simultaneously, one could detect the
linear combinations of the process variables. If the problematic observations in all cases, utilizing infor-
process variables are normally distributed then the mation collected from only the x-space (i.e., process
scores are also normally distributed. When the pro- variables) .
cess variables are not normally distributed, the scores
(being linear combinations of these variables) are ex- 4.----~-~==~-==--~--__,

pected to be closer to a multinormal distribution by ~ -----------------;--:-------

2 -- ---&/--. .: . -"-,
the central limit theorem. ,/ ~._- __...zz .
0 "
.~,. .. .-.-.".-
.. :"' .. ,
.. -....
.. "\,
,,/
Monitoring and Diagnosis utilizing \',.......... /

---------------~;--~_.----_//
-2
Scores and SPE
,
An essential part of SPC is to establish control ~ -4

charts to detect special events as they occur, and -6 ~3

to diagnose possible causes for them while the in-
formation is fresh. The philosophy applied in devel- -8 \6.t
<.~
oping multivariate SPC procedures based on projec- -10
tion methods, is the same as that used for the tradi-
-12
tional univariate or multivariate charts. An appro- -6 -4 -2 o 2 4 6
priate reference set is chosen which defines the nor-
mal or "in-control" operating conditions for a partic-
ular process. In other words, a PCA or PLS model FIGURE 10. Process Monitoring in the LDPE Reactor,
is built based on data collected from various peri- using PLS. The Disturbance is Fouling. Projection on the
ods of plant operation when performance was good. t2 - t3 Plane.

Journal of Quality Technology Vol. 28, No.4, October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 423

FOULING: MOnitoring S!'Ores fromPLSon 14X Variable<

120 culated from the corresponding ta's of the reference
46 set; notice that in PLS the sa's are related to the
5
rI)
100 eigenvalues of X'YY'X.) Contribution plots on the
~ SPE (a bar plot of each term in the summation of
0 80 caseB (12)) indicate how each variable contributes to the
U
rI)
SPE for a specific observation. When an observation
~ 60
caseC
has a value of the SPEx n e w beyond the control limit,
Z the variables on this plot that appear to have larger
0
40 contributions than usual (and variables highly corre-
~
lated to them) could be investigated as responsible
C03<A
20 for the deviation. An example is shown in Figure
12a which gives the contribution plot for observation
0
0 70
26 of Figure llb. Recall that this observation cor-
OBSERVATIONS
responds to fouling in zone 1. The contribution plot
shows that the major process variables contributing
(a) to the SPEX 2 6 are the coolant inlet and the reacting
mixture outlet temperatures of zone 1 (as one might
expect in a fouled reactor). The contribution plot
FOULING: Monitoring SPE fromPLSon 14X Variable<
1200 for the SPE for point 66 (corresponding to fouling
66
in both zones) indicates (Figure 12b) that the inlet
1000 5 coolant and reacting mixture outlet temperatures for
both zones were responsible, consistent with what is
800 caseC expected.
g: 600 Multi-Block PLS
rI)

When a large number of variables is included in

400
the x-space, the monitoring and diagnosing charts
caseA
discussed in the previous sections may be diffi-
200
cult to interpret. The combined use of multi-block
9~,Q~
PLS (MB-PLS) and contribution plots may facilitate
0
0 10 20 70 this task. In the MB-PLS approach, sets of pro-
OBSERVATIONS cess variables X are broken into meaningful blocks
Xj , X 2 , · · · ; usually each block corresponds to a pro-
(b) cess unit or a section of a unit. The principles be-
FIGURE 11. Process Monitoring in the LOPE Reactor, hind multi-block data analysis methods and their al-
using PLS. The Disturbance is Fouling. (a) T~ Chart on 3 gorithms can be found in Wold (1982) and Wangen
PLS Scores, and (b) SPE Chart. and Kowalski (1988). Multi-block PLS is not equiv-
alent to performing PLS on each block separately;
all the blocks are considered together. MacGregor
Once a deviation from the normal operating region
et al. (1994) discuss an application of MB-PLS to
has been detected on-line, an assignable cause for this
process monitoring and diagnosing for the LDPE re-
deviation must be found and corrected. One set of di-
actor. In this case each block corresponds to one
agnostic tools are the contribution plots (on the SPE
reactor zone. Plots of tl vs t2 and SPE, obtained for
or on the scores) discussed by Miller, Swanson, and
each block of the process, were utilized to detect an
Heckler (1993) and MacGregor et al. (1994). Con-
abnormal event in the zone in which it occurred; then
tributions to PCA scores have been discussed earlier
contribution plots were successfully used to assign
(equations 4 or 5); the contribution of a variable x j to
causes for it. The following example demonstrates a
the PLS scores is given as Wa,j(xnew,(a-l),j), where
W a is the weight at component a and xnew,(a-l) is
monitoring procedure utilizing a chart of T1 calcu-
lated from the scores and the SPE obtained from the
the value of X new after deflation from the previous
MB-PLS model.
a -1 components, with xnew,O = X new -p,. The total
contribution to all scores can again be computed as Figures 13a and 13b give respectively the Ti
chart
in (7). (The value of Sa, a = 1,2, ... ,A, can be cal- (calculated from scores tl, t2 and t3 of block 1) and

Vol. 28, No.4, October 1996 Journal of Quality Technology

424 THEODORA KOURTI AND JOHN F. MacGREGOR

FOULING: 14X Variables; SPEcontribution in X space: polnl26

250 that the diagnostics of zone 2, in this case, do not
~ r-- show a problem for zone 2 for observations 22-26;
~
Q.;
200
recall that when a traditional multivariate chart on
the process variables was used for this zone (Figure
9d) it showed a problem for these observations; MB-
~
en ISO ~
PLS successfully isolated the effect from zone 1, and
~ TC_IN1 showed clearly that in 22-26 the problem occurs only

~
in zone 1. Utilizing the contribution plots for fault di-
100 agnosis (not shown here) has revealed that important
variables for the fouling cases were the coolant inlet
~
TOl.JTl
so temperature and the reacting mixture outlet temper-

U 0
2 4 6
Dr! 10 12 14
ature, in the corresponding zones.
Although the monitoring and diagnosis proce-
PROCESS VARIABLES dures based on MB-PLS and PLS gave comparable
results for this system with only fourteen variables,
(a) MB-PLS offers an advantage when bigger systems
with tens or hundreds of variables are involved.

Discussion on the Projection

Approaches
FOULING: 14X Variables: SPEc:ontribution in X space:polnl66
400
10
10
Control charts developed from the multivariate
~ 350 -. statistical projection methods (i.e., score charts
TC_IN1 (tll ta, t3,"') and SPE charts) have usually been
~ 300 r=:t based on a minimum of assumptions. The only es-
IE 250
TOUT! sential one is that the PCA or PLS model has been
(I)
I built on a representative set of data that captures
~200 all common cause variations. The statistical varia-
~ ISO
I.JT
C_IN2
tion about this model that captures "in-control" be-
TO havior, provides a reference distribution from which
-m
;2
~ 100 one can obtain control limits. Neither the assump-
~ SO
tions of normality, nor independence are needed if
U 0
Il--,...- - ~ one has enough data. (See for example the his-
2 4 6 10 12 14 torical reference distribution concepts discussed in
PROCESS VARIABLES Chapters 2 and 3 of the text by Box, Hunter, and
Hunter (1978).) The construction of control limits
(b)
for the multivariate charts, based on reference distri-
FIGURE 12. Process Monitoring in the LDPE Reactor, butions obtained from historical data, are discussed
using PLS. The Disturbance is Fouling. (a) Contribution of in Nomikos and MacGregor (1995).
Process Variables to SPE, Point 26, and (b) Contribution
of Process Variables to SPE, Point 66. If one wished to detect a very specific type of devi-
ation, then more powerful charts (e.g., the "cuscore
the SPE chart for block 1 (corresponding to zone 1) charts" of Box and Ramirez (1991» could be devised,
for the same simulated process conditions of Figures that are based on the specific nature of this devia-
9a and 9b (i.e., fouling in different sections). Notice tion. However, to construct a whole set of charts
that by monitoring block 1 (zone 1, only) problems based on statistical tests for specific types of special
are only detected for observations 22-26 and 62-66 events would assume that one has a priori knowledge
when zone 1 was affected by fouling. Observations of all such events, and how they will affect the dis-
42-46 show a normal operation for this zone. Fig- tribution of the variables. Then one would have to
ures 13c and 13d give the Tl and the SPE charts assess not only the ability of each chart to detect the
for block 2 (zone 2), for the same simulated foul- event for which it is specifically designed, but also
ing cases. Notice that problems were detected in assess its robustness to false signals when any of the
zone 2 for observations 42-46 and 62-66. Notice also other events occur.

Journal of Quality Technology Vol. 28, No.4, October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 425

FOULING: MonilOrioC s...... fromMBJ'LS: ZONEI FOULING: MBJ'LS. MonilOriac SPBfor 8 Variabl..: ZONEI
35,---~-~--~-~--~-~---, SOC>
26 66

~
5 4SO 26 66
30 5
400
caseA caseC
~ 25 3SO

~
caseA caseC
8
fI)
20
300

2SO
~ 15 iE
fI) 200 13
E!l -.~,Q~.. ----.--.-.---.- -..•----..•..•.•...•........-....-.-.--..•.•.•.••..
~ 10 ISO

~ 2 100
caseB
~ SO
.. ~,Q.'! ······v······· .."'"..4-~ •••••••• - ••••••• •••• -.--.
70 \ W 20 ~ ~ SO ro w
OBSERVATIONS OBSERVATIONS
(a) (bl
FOULING: MonilOriac s...... fromMBJ'LS: ZONE2 FOULING: MBJ'LS. MonilOrioC SPBforCVariabl..: ZONE2
46 66
N
4SO 5
~
5
66
20 5
N
caseB caseC N 3SO

~"
v, .. Jl.~J!.~ _.._ .
~:
caseB caseC

~ 10 3 3 iE
fI) 200
'1'
~
ISO

Z 5 100
o 50 caseA
~ o ..~,Q.'! ~.... .. . 1.. 1. .
70 o W 20 ~ ~ SO 00 W

OBSERVATIONS OBSERVATIONS
(el (d)

FIGURE 13. Process Monitoring in the LDPE Reactor, using MB-PLS. The Disturbance is Fouling. (a) T~ Chart on 3
MB-PLS Scores, Zone 1, (b) SPE Chart, Zone 1, (c) T~ Chart on 3 MB-PLS Scores, Zone 2, and (d) SPE Chart, Zone 2.

A typical multivariable process having hundreds tions about the nature of the specific event, they
of variables, may have dozens of special events must be supplemented with diagnostic procedures
which might occur, most of them affecting the mean such as contribution plots. The contribution plots
and covariance structure of the variables in an un- may not explicitly reveal the cause of the event, but
known way. For example, special events arising from they point to the group of process variables that are no
changes in impurity concentrations and fouling in the longer consistent with normal operating conditions,
LDPE reactor did not simply affect a few process and thereby focus the attention of the operators or
variable means. Rather, they simultaneously affected the engineers and allow them to use process knowl-
the mean and the covariance structure of variables in edge to deduce possible causes.
ways that were not known a priori.
Concluding Remarks
For this reason we believe that using multivariate
control charts which simply test whether the behav- Multivariate SPC monitoring methods can be ap-
ior of future data is consistent with past "in-control" plied to process variables as well as final quality vari-
behavior (captured by a PCA or PLS model) is a ables. When the number of process variables is not
preferable approach when dealing with large multi- too great, traditional control charts based on the X2
variate processes. However, when a deviation is de- and T 2 statistics can be very effective for the early
tected, since the charts involve no specific assump- detection of special events. When the dimension of

Vol. 28, No.4, October 1996 Journal of Quality Technology

426 THEODORA KOURTI AND JOHN F. MacGREGOR

the problem becomes large, charts based on mul- applications of the projection methods to the mon-
tivariate statistical projection methods (PCA and itoring and analysis of processes from the pharma-
PLS) and their multi-block versions are generally ceutical industry, the semiconductor industry, and
much more effective. These latter methods can eas- the tire and rubber and other chemical industries.
ily handle systems with many collinear and redun-
dant variables and can accommodate missing obser- Acknowledgments
vations in a straightforward fashion (Nelson, Mac-
The authors wish to thank the nineteen compa-
Gregor, and Taylor (1996)).
nies of the McMaster Advanced Control Consortium
Furthermore by monitoring the process variables for their financial support and their active involve-
as well as the quality variables, the diagnosis of ment in joint research projects, which provide the
assignable causes is much easier. By interrogating opportunity to test new research ideas in industry.
the underlying multivariable model for the in-control
process, contribution plots for the process variables References
are easily constructed. Although these plots will not ALT, F. B. (1985). "Multivariate Quality Control" in Encyclo-
unequivocally reveal the assignable cause(s) they will pedia of Statistical Sciences, Vol. 6 edited by N. L. Johnson
at least point to that group of process variables that and S. Kotz. John Wiley & Sons, New York, NY.
contributed most to the detected out-of-control sit- Box, G. E. P.; HUNTER, W. G.; and HUNTER, J. S. (1978).
uation, thereby aiding the engineer or operator in Statistics for Experimenters. John Wiley & Sons, New York,
their search for assignable causes. The inherent con- NY.
dition in all the methods discussed here is that mul- Box, G. E. P. and RAMIREZ, J. (1991). "Cumulative Score
tivariate SPC charts (either traditional or based on Charts" . Report No. 58, Center for Quality and Productivity
Improvement, University of Wisconsin, Madison, WI.
projection methods) of process variables or product
DAYAL, B.; MACGREGOR, J. F. ; TAYLOR, P. A.; KILDAW,
quality variables will detect only those unusual pro-
R.; and MARCIKIC, S. (1994). "Application of Feedforward
cess events that will affect the variables that have
Neural Networks and Partial Least Squares Regression for
been included in the chart. Modelling Kappa Number in a Continuous Kamyr Digester".
There has been a significant amount of work done Pulp and Paper Canada 95, pp. 26-32.
on monitoring continuous processes using projection FUCHS, C. and BENJAMINI, Y. (1994). "Multivariate Profile
Charts for Statistical Process Control". Technometrics 36,
methods (Kourti, Lee, and MacGregor (1996)). In-
pp. 182-195.
dustrial applications include Catalytic Cracking in
GELADI, P. and KOWALSKI, B. R. (1986). "Partial Least
Petroleum Refining (Slama (1991)), Applications in Squares Regression: A Tutorial". Analytica Chimica Acta
Mineral Processing (Hodouin, MacGregor, Hou, and 185, pp. 1-17.
Franklin (1993); Tano, Samskog, Garde, and Skager- HAWKINS, D. M. (1993). "Regression Adjustment for Variables
berg (1993); and Tano, Samskog, and Andreasson in Multivariate Quality Control". Journal of Quality Tech-
(1995)), a Ceramic Melter (Wise and Ricker (1989)), nology 25, pp. 170-182.
a Pulp Digester (Dayal, MacGregor, Taylor, Kildaw, HODOUIN, D.; MACGREGOR, J. F.; Hou, M.; and FRANKLIN,
and Marcikic (1994)), Photographic Paper Manufac- M. (1993). "Multivariate Statistical Analysis of Mineral Pro-
turing (Miller, Swanson, and Heckler (1993)), and cessing Plant Data". CIM Bulletin, Mineral Processing 86,
pp.23-34.
Cold Rolling and Annealing in the Steel Industry
HOSKULDSSON, A. (1988). "PLS regression Methods". Journal
(Vaculik (1994, 1995)). Simulation studies have been
of Chemometrics 2, pp. 211-228.
performed on a Distillation Column and a Fluidized
HOTELLING , H. (1947). "Multivariate Quality Control" in
Bed Rector for Hydrocracking (Kresta, MacGregor, Techniques of Statistical Analysis edited by Eisenhart, Has-
and Marlin (1991)), and on a Low Density Polyethy- tay, and Wallis. McGraw Hill, New York, NY.
lene Reactor (Skagerberg, MacGregor, and Kiparis- JACKSON, J. E. (1956). "Quality Control Methods for Two
sides (1992), MacGregor et al. (1994)). Related Variables". Industrial Quality Control 12(7), pp. 4-
8.
The above ideas have also been extended to moni-
JACKSON, J. E. (1980). "Principal Components and Factor
toring time varying batch processes by Nomikos and Analysis: Part I - Principal Components" . Journal of Quality
MacGregor (1994,1995); Kourti, Nomikos, and Mac- Technology 12, pp. 201-213.
Gregor (1995); and Kourti, Lee, and MacGregor JACKSON, J. E. (1991). A User's Guide to Principal Compo-
(1996), with several applications on polymerization nents. John Wiley & Sons, New York, NY.
reactors and other processes. Finally, unpublished KIPARISSIDES, C.; VEROS, G.; and MACGREGOR, J. F. (1993).
work (where the authors have been involved) includes "Mathematical Modelling, Optimization and Quality Con-

Journal of Quality Technology Vol. 28, No.4, October 1996

 MULTIVARIATE SPC METHODS FOR PROCESS AND PRODUCT MONITORING 427

trol of High Pressure Ethylene Polymerization Reactors". SHEWHART, W. A. (1931). Economic Control Quality of Man-
Journal of Macromolecular Science, Reviews in Macro- ufactured Product. D. Van Nostrand Co., Inc., New York,
molecular Chemistry and Physics C33 (4), pp. 437-527. NY.
KOURTI, T.; LEE, J.; and MACGREGOR, J. F. (1996). "Expe- SKAGERBERG, B.; MACGREGOR, J. F.; and KIPARISSIDES, C.
riences with Industrial Applications on Projection Methods (1992). "Multivariate Data Analysis Applied to Low Den-
for Multivariate Statistical Process Control". Computers in sity Polyethylene Reactors". Chemometrics and Intelligent
Chemical Engineering 20 Supplementary, pp. S745-S750. Laboratory Systems 14, pp. 341-356.
KOURTI, T.; NOMIKOS, P.; and MACGREGOR, J. F. (1995). SLAMA, C. F. (1991). "Analysis of Industrial FCCU data us-
"Analysis, Monitoring and Fault Diagnosis of Batch Pro- ing PCA and PLS". M.Eng. Thesis, McMaster University,
cesses Using Multi-block and Multiway PLS". Journal of Hamilton, ON, Canada.
Process Control 5, pp. 277-284. SPARKS, R. S. (1992). "Quality Control with Multivariate
KRESTA, J.; MACGREGOR, J. F.; and MARLIN, T. E. (1991). Data". Australian Journal of Statistics 34, pp. 375-390.
"Multivariate Statistical Monitoring of Process Operating TANO, K.; SAMSKOG, P.-O.; and ANDREASSON, B. (1995).
Performance". Canadian Journal of Chemical Engineering "Multivariate Modelling and On-Line Data Presentation for
69, pp. 35-47. Process Monitoring at LKAB". Presented at the Interna-
LOWRY, C. A.; WOODALL, W. H.; CHAMP, C. W.; and RIG- tional Federation of Automatic Control Symposium on Au-
DON, S. E. (1992). "A Multivariate Exponentially Weighted tomation in Mining Mineral and Metal Processing. Sun City,
Moving Average Control Chart". Technometrics 34, pp. 46- South Africa.
53. TANO, K.; SAMSKOG, P.-O.; GARDE J.-C.; and SKAGERBERG,
MACGREGOR, J. F.; JAECKLE, C.; KIPARISSIDES, C.; and B. (1993). "Partial Least Squares Modelling of Process Data
KOUTOUDI, M. (1994). "Process Monitoring and Diagnosis at LKAB: Predicting Chemical Assays in Iron Ore for Pro-
by Multi-Block PLS Methods". Journal of the American In- cess Control". Presented at the International Symposium on
stitute of Chemical Engineers 40, pp. 826-838. the Application of Computers and Operations Research in
MACGREGOR, J. F. and KOURTI, T. (1995). "Statistical Pro- the Mineral Industries. Canadian Institute of Mining, Met-
cess Control of Multivariate Processes". Control Engineering allurgy and Petroleum, Montreal, PQ, Canada.
Practice 3, pp. 403-414. TRACY, N. D.; YOUNG, J. C.; and MASON, R. L. (1992). "Mul-
MARDEN, J. I. and PERLMAN, M. D. (1990). "On the Inad- tivariate Control Charts for Individual Observations". Jour-
missibility of Step-Down Procedures for the Hotelling T 2 nal of Quality Technology 24, pp. 88-95.
Problem". Annals of Statistics 18, pp. 172-190. VACULIK, V. (1994). "Multivariate Data Analysis Using
MARDIA, K. V.; KENT, J. T.; and BIBBY, J. M. (1979). Mul- PCAjPLS with Extensions to Performance Monitoring at
tivariate Analysis. Academic Press Ltd., San Diego, CA. Dofasco". Presented at the Advanced Modelling and Con-
MASON, R. L.; TRACY, N. D.; and YOUNG, J. C. (1995). "De- trol Seminar by the Association of Iron and Steel Engineers,
composition of T 2 for Multivariate Control Chart Interpre- Cleveland, OH.
tation". Journal of Quality Technology 27, pp. 99-108. VACULIK, V. (1995). "Applications of Multivariate Projection
MILLER, P.; SWANSON, R. E.; and HECKLER, C. F. (1993). Methods in the Steel Industry". M.S. Thesis, Chemical En-
"Contribution Plots: The Missing Link in Multivariate Qual- gineering Department, McMaster University, Hamilton, ON,
ity Control" . Presented at Annual Fall Technical Conference Canada.
of the American Society for Quality Control (Milwaukee, VEROS, G.; PAPADAKIS, M.; and KIPARISSIDES, C. (1993).
WI) and the American Statistical Association (Alexandria, "Mathematical Modeling of High Pressure Tubular LDPE
VA). Copolymerization Reactors". Polymer Reaction Engineering
MONTGOMERY, D. C. (1996). Introduction to Statistical Qual- 3, pp. 427-460.
ity Control, 3rd ed. John Wiley & Sons, New York, NY. WADE, M. R. and WOODALL, W. H. (1993). "A Review and
MURPHY, B. J. (1987). "Selecting Out Of Control Variables Analysis of Cause Selecting Charts". Journal of Quality
with the T 2 Multivariate Control Procedure". The Statisti- Technology 25, pp. 161-169.
cian 36, pp. 571-583. WANGEN, L. E. and KOWALSKI, B. (1988). "A Multi-block
NELSON, P.; MACGREGOR, J. F.; and TAYLOR, P. A. (1996). Partial Least Squares Algorithm for Investigating Complex
"Missing Data Methods in PCA and PLS: Score Calculations Chemical Systems". Journal of Chemometrics 3, pp. 3-20.
with Incomplete Observations". Chemometrics and Intelli- WIERDA, S. J. (1994a). "Multivariate Statistical Process
gent Laboratory Systems 35 (in press). Control-Recent Results and Directions for Future Re-
NOMIKOS, P. (1995). Private Communication. search". Statistica Neerlandica 48, pp. 147-168.
No MIKOS , P. and MACGREGOR, J. F. (1994). "Monitoring WIERDA, S. J. (1994b). Multivariate Statistical Process Con-
of Batch Processes using Multi-way Principal Component trol. Groningen Theses in Economics, Management and Or-
Analysis". Journal of the American Institute of Chemical ganization. Wolters-Noordhoff, Groningen, The Netherlands,
Engineers 40, pp. 1361-1375. pp.95-124.
NOMIKOS, P. and MACGREGOR, J. F. (1995). "Multivariate WISE, B. M. (1991). "Adapting Multivariate Analysis for
SPC charts for Monitoring Batch Processes". Technometrics Monitoring and Modelling of Dynamic Systems" . Ph.D. Dis-
37, pp. 41-59. sertation, University of Washington, Seattle, WA.

Vol. 28, No.4, October 1996 Journal of Quality Technology

428 THEODORA KOURTI AND JOHN F. MacGREGOR

WISE, B. M. and RICKER, N. L. (1989). "Feedback Strategies ber of Components in Factor and Principal Component Mod-
in Multiple Sensor Systems". American Institute of Chem- els". Technometrics 20, pp. 397-405.
ical Engineers Symposium Series on Process Sensing 85, ZHANG, G. X. (1985). "Cause Selecting Control Charts-A
pp. 19-23. New Type of Quality Control Charts". The QR Journal 12,
WOLD, H. (1982). "Soft Modelling. The Basic Design and pp.221-225.
Some Extensions" in Systems under Indirect Observation
Vol. 2 edited by K. G. Joreskog and H. Wold. North Hol- Key Words: Multivariate Analysis, Multivariate
land, Amsterdam, Chapter l. Control Charts, Principal Components, Statistical
WOLD, S. (1978). "Cross-Validatory Estimation of the Num- Process Control.

Journal of Quality Technology Vol. 28, No.4, October 1996