The Statstcian (1999) 48, Part2. pp 159-177 Experimental design for product and process design and development Dougtas C. Montgomery Arizona State University, Tempe, USA [Recenwed September 1998, Revised December 1258] 1._ Introduction and background Experiments are performed by investigators in virtually all fickts of inquiry, Generally, expen ments ars used in studying or evaluating the performance of systems The system under study ‘can be represented by the general model of Fig. I. We visualize the system as.a combination of components, materials. people. equipment. processes and other resources that function collect cl to wansform 3 set of inputs into outputs described by one of more response Variables There are p= & +r carubles that potentially affect the performance of the system. Vanables x... 1 are controllable variables that can be adjusted to- and held at specific target levels, whereas the vanables <5, are either diffeult 0 control or uncontrollable in the field performance of the system, although they may be controlled for performing a specific experiment In many industrial eavironments. the system of Fig. | is 4 productica Process ora product. and the objectives of the experiment may include the following: {a}, to determine which vanables are most influential on the responscts ). (b) te determine where to set the influcntial x so that the response or fesponses are almost always near theit desired target values, (6) to determine where to set the influential 13 so that the variability 10 the response(s) is shall or reipemdeme Uiglan €! Montgoeicsy, Cepariment of Inkaaiat Engencering, Anta State meray cane US E-mas doug monigomery@asu ch, © 1999 Royal Stasatest Seeaty 0039-262650448 159Uncontollable Facies Fig. 1. General moet of a system (4) to determine where to set the influential 1 so that the effects of the uncontrollable ‘anables on the response(s) are small. Starstically designed expenments are extremely important in these activities. This paper discusses some aspects of how statistical design is used in developing new processes of improving the performance of existing processes. Designed experiments also play a. major role in engineering design activities. where new products are designed and developed and existing prodacts are ‘modified to enhance their features, haractenstics o¢ operation. 2. Aperspective on the development of statistical design ‘There have been four eras in the modem development of statistical experimental design. The agricultural era was led by the piomeering work of R.A. Fisher in the 1920s and 1930s. Fisher recognized that the analysis of data from systems (in this case agncultural systems) was often hampered by flaws in the way the experiment that generated the data had been performed. By ‘interacting with scientists and researchers in many ficlds be developed the insights that led to the three basic principles of experimental design: randomization, replication and blocking. Fisher systematically introduced statistical thinking and principles mto designing expertmental investiga tions, including the factorial design concepi, the basic building-block of much subsequent work in the field Although applications of statistical design in industnal settings certainly began in the 1930s the second or industrial era was catalysed by the development of response surface methodology (RSM) by Box and Wilson (1951), They recognized and exploited the fact that many andustnal ‘experiments are fundamentally different from their agricultural counterparts in two ways (a) the response vanable can usually be observed (nearly) immediately and (b) the experimenter can quickly earn crucial information from s small group of nuns that ca be used to plan the next experiment Over the next 30 years, RSM and other design techmiques spread throughout the chemical and the process industnes, although mostly in rescarch and development work, The application ofExperimental Design = 161 statisheal design at the plant or manufacturing process level was sill not extremely widespread, Some of the reasons for this include inadequate training in haste statistical concepts and methods for engincers and other proves specialists, and the lack of computing and software resources to support the application of statistically designed experiments. ‘The incteasing interest of Western industry in quality improvement that began in the late 1970s ushered in the third era of statistical design. The work of Genichi Taguchi (Tagucht and Wu, 1980; Kackar, 1985, Taguchi, 1987, 1991) had i significant effect on expanding the interest im and tase of designed experiments. Taguchi advocated using designed experiments for What he termed ubust parameter design, of (a) making processes msenaitive to ctivitonmental factors or other factors that are difficult to control, (by making products insensitive to: variation transmitted from components and fe) finding levels of the process vanables that force the mean to a desifed valuc while sumul- taneously reducing variability around this value ‘Taguchi suggested highly frachonated factonal designs and other orthogonal arrays along with some novel statistical methods to salve these problems. The resulting methodology generated much discussion and controversy, Part of the controversy arose because Taguchi's methodology was advocated in the West initially (and primanly) by entreprencurs. and the underlying statistical science had not been adequately peer reviewed. By the late 1980s the results of peer review indicated that. although Taguchi's engincering concepts and objectives were well founded, there ‘were substantial problems with his experimental strategy and methods of data analysis. For specific details of these issues, see Box (198%), Box cf al (1988), Hunter (1985, 1989), Mont- gomery (1992. 1997), Mycrs and Montgomery (1945) and Pagnaticllo and Ramberg (1992), Many ‘of these concems are also summaned in the extensive panel discussion in the May 1992 issue of Technometrics (see Nair (1992) There were two positive outcomes of the Taguchi controversy. First, designed experiments ‘became more widely used in the discrete parts industnes. including aulomotise and acrospace manufacturing. ¢lecttonics and semiconductors, and many other industries that had previously made litthe use of the techmique. Second. the fourth era of statistical design began. This era has included @ renewed interest in statistical design by both researchers and practitioners. and the development of alternatives to Taguchi's technical methods that allow his engineering concepts to be carried into practice efficiently and effectively. Some of these alternatives will be discussed and illustrated an subsequent sections. 3. A strategy for product and process development A fundamental approach te process and product design and development consias of three phases characterization. control and optimization. Chanicieration as the process of discovering the specific process variables that are responsible for the variability in the system's output responses. Systems (product and processes) are often described by many variables, particularly in the early stages of design and development work, when the level of scientific and engineering knowledge may be low Thus identifying the most important factors carly is cntical 10 successful eventual development of the system. Without proper characterization, a conaderable amount of guesswork about which sanables are important and the effect of Vanous factors on the responses of inter= est will typically occur. This contributes to long development lead times, missed deadlines for releasing products and process certification. and subsequent loss of competitive advantage Factonal and fractional factorial designs.are very useful in characterizing systems. In particular,1620. C. Montgomery the 2! and 2* ‘designs, often augmented with centre points (see Box ef al. (1978) and. Montgomery (1997) are typical choices. Statisticians usually refer Yo many of the astivibes that are performed during characterization as factor sercening. The use of two-level factors wath centre points allows both the efficicat estimation of effects and protection agaist pure quadratic ‘curvature. Control refers to process stability, ¢ 10 obtaining a consistent performance from the system ‘One of Taguchi's important contnbutions was his observation that some process vanables may affect the mean of the response variable, whereas others may affect the response sanance. These are typically referred to as location effects and dispersion effects respectively. Taguchi's approach to this problem was based on the use of signal-to-noise ratios. Although thts approach was flawed (sce Box (1988), Bor ef al. (1988), Montgomery (1997) and Myers and Montgomery (1995), 3t spurred much research and the development of several useful altematives, including the estimation of dispersion effects through the analysis of residuals from conventional factorial and fractional designs, and separate modelling of the mean and variance. Some use ful references ate Carroll and Ruppert (1988), Bartlett and Kendall (1946), Box and Meyer (1986), Myers and Montgomery (4993), Vining and Schaub (1996) and Vining and Bohn (1997), Optimization refers to manipulating the most important process variables to levels of setings that result in the best obtainable set of operating conditions forthe system. In mast industries, 11s ‘not unusual to have several system or process outputs that need to be jointly optimized. For cxample, in a chemical process, it may be necessary to consider responses such as provess yickL product motecular weight, viscosity. concentration and environmental parameters ("green manu- facturing considerations). This will usually require the simultaneous optimization of these (often conflicting) responses. RSM is an extremely useful framework for modelling and optimizing systems. For an introduction to RSM. sce Box and Draper (1987), Myers and Montgomery (1995) and Khuri and Comell (1996). Considerable effort has been devoted to developing response surface alternatives to Taguchs’s robust design. For details of the approaches and cxamples, refer to Myers (1991), Myers ef al (1992), Lucas (1994) and Myers and Montgomery (1995). Mixture experiments. a vananion of RSM. are also highly useful for robust product development and formulation The characterization control optimization strategy places a strong emphasis on sequential experimentation, in which we move from a state of relatively Tow knowledge about the system to a state of advanced knowledge. The objectives of determining which factors produce important cffcets. the role that each factor plays ir driving the mean andlor vanance of each response and the determination of optimum conditions are mest easily and efficiently accomplished through a sens of small interrelated experiments, Taguchi's approach to robust parameter design typically used relatively large comprehensase experiments. This strategy 1s often ineffective in practice, Ihecause at the earliest stages of a development problem the engincers and scientisis simply do riot now enough to plan a good comprehensive experiment, Furthermore. as pointed out by Coleman and Montgomery (193), large comprehensive experiments are difficull to complete successfully as planned 4, Making products and processes robust ‘The original robust parameter design strateyy’ proposed by Taguchi consists of first classifying the p system variables as k controllable variables and r noise variables (the uncontrollable variables 18 Fig 1). Then an #, * & “control (or inner) array” is selected for the controllable varables. As the name suggests. the control array vontains the settings of the controllable variables to be used in the experiment. Each row of this array consists of settings for the s for a particular expertmentalExperimental Design 163 run. Then ann. | vector of the first-order cocfticients associated with the control lable factor, the 4X ( main B contains coefficients for the controllable factor interactions and pure quadratic terms involving these vanables, 1s an 1 vector of the firsteorder noise factor coctticients and the A r. there will be a line or a plane representing the region of minimum. process variance. When pr there will be 3 single potnt that wall result in minimum sanance. If p<. then (x) = @ may not have a solution. See the discussion in Myers atnd Montgomery (1995). We illustrate these ideas with a second example, Consider a situation where there are two controllable vanables and three noise vanables. The combined array desige used by the expert: menters 1s shown in Table 2, The design uscd is. a 23-run variation of a central composite design. and it will support the response mode! (3), The fitted response model is $= 3037 — L9By, ~ 4ABK; + 2.608) + 218 + 2878, 5 0.274)2; + ORK, 2, + DSK 2 + 2 al ‘The mean and variance models are E,f atx, 2)) = 30.37 — 2.924, — 4.13%, + 206005 + 2.182;Expenmenta/Desgn = 169 Table2. Combined array exponmont with two controllable ver: ables and thine nose vanables “ Loy 1.00 100-100 108 too 100 100. Loa hoa how Loo joo io ho Joo 10 100 6.00) om 000 0.00 000 om oon and Sts (x. zy} = 19-26 + 3.208, + 124Se: + 750x +8520 + 22a where we have substituted parameter estimates into equations (4) and (5), and as in the previous example assumed that a) = 1. Figs 6 and 7 present contour plots of the process mean and yanance response surfaces generated from these models. Tn thas problem it i$ desirable to keep the process mean below 30, From an inspection of Figs and 7 it is clear that some trade-off will be necessary if we wish to make the process variance 1007 — —< 30 ose> ] 30 000 ™~ | x KX Ne = —_ 0.50} jx, 38 21.00 | SSS. Sa 100 050 0.00 080) 1 OD ca Fig. 6. Response surtace for the process mean170 (0.C Montgomery Fig.7, Response surface for the process variance small. Since there are only two controllable variables. a logical way to accomplish this trade-off 1s to overlay the contours of constant mean response and constant variance, as shown in Fig, 8. This plot shows the contours for which the process mean is less than of equal to 30 and the process vanance is less than or cqual to 25 The region bounded by these contours would represeat a typical operating region of low mean response and low process variance 7. Some benefits of statistical design to the engineering design and development community Statistically designed experiments play a significant role in the indusinal world, Among theit mowt important uses are in the design and development of new products and production peovesses Tneineers scientists and other technical process specialists are the peamary wsts of d=sienel teperments in these applications, The effective use of statistical design im process development can resull int iy ~ Drive-in time 1o0 080 0.00 (0.50 X, © Implant dose Fie 8 Crveray of the contours win process enn 30 of ess anevanarce 25 oresExpermeniai Desgn 171 (2) mproned process performance and yield (b) reduced vanability and closer conformance to larget requirements for critical product Properties. (€) reduced development lea times for new processes and (d)_ lower overall manufacturing costs Statisucal design can also play a major role an engineering design activities, where new products are developed and existing products are improved Some important applications of ‘atistical design in product design include (a) the evaluation and comparison of basic design configurations. (b) the evaluation of material alternatives, (6) the formulation of products. (4) the determination of key product design parameters that affect product field performance, (€) the selection of design parameters so that the product will work well under a wide variety of field conditions, of so that it will be insensitive to potential sources of sarability trans- mitted from processing variables that are difficult to control, and (1) the evaluation of product reliability and life performance. Many of these applications can be thought of as a variation of robust design. We have reviewed some aspects of the robust design problem, beginning with the original contributions of Taguchi. and pointed out that response surface methods provide a more logical and straightforward framework in which to implement his engineering concepts. 8 Concluding remarks As noted earlier. interest in and application of Taguchi’s robust parameter design methodology in the early 1980s ushered in the third era in the use of statistical design. The eventual peer review of his work. and othet events led to the beginning of the fourth era in the late 1980s. This fourth era hhas seen the use of statistical design techniques expand dramatically. both in the breadth of industries using the techniques and in the eatent to Which engineering. scientific afd operating ‘Personnel are familiar with the approach. Several things have fostered this expanded use of ‘statistical design, including the growing availability of good and inexpensive computer software supporting applications and the incorporation of desizned experiments as a formal part of the [Process characterization, For example, the semiconductor industry has been a leader in this regard ‘The development of effective and efficient methods for robust design based on separate mean and \ariance modelling and the response model approach has been a highlight of the fourth era. ‘Many important problems. remain, however, so these aspects of statistical design will continue to ‘be an active area of research. For example, information (such as we have for standard response surface designs) that would allow an experimenter to select easily: an appropriate design for the combined array 1s incomplete. Modified central composite designs, such as we illucrated, and computer-generated alphabetically optimal designs are logical candidates. but the properties and Performance of these designs have not been thoroughly studied. There is also a need for tolerance and prediction intervals based om the response model approach. There will continue to be expanded research and development in design optimality. and its applications. Mixture designs, mixture designs with process vanables. non-standard models and jother design problems involving constraints on process variables and irregular regions are obvious applications for alphabetically optimal designs. Another area of potential application for these techniques is experiments with deterministic computer models. These computer models are used172 DG Montgomery sn mechanical and electrical design problems. Design problems involving these models often have a large number of factors and many responses. Many of the analysis and modelling ssues for these types of experiments are fertile areas for research As experimental design is more widely applied, situations where the underlying statistical assumptions are not satisfied will be more frequently encountered. For example. in the semicon- ductor industry 1 1s relatively common to use defects and yield as response vanables. Further- more, many clectnical parameters are stricily positive vanables, often with many values near 0), 30: the assumption of normality 1s probably snappropnate. The generalized linear model ws a logical framework in which to analyse these types of experiment. The generalized linear model has been widely used an the biopharmaccutical field, It 4s now starting to find application in more general odustnal experiments: for more information and some interesting examples, sce Grego (1992). McCullagh and Nelder (1989), Myers and Montgomery (1997), Hamada and Nelder (1997) and Engel and Hucle (1996), Methods for handling multiple responses will continue to be of interest both to practitioners and to researchers. Since most experiments involve several responses, the need for improved analysis, model building and optimization methodology is apparent, Most of the current methods for multiple-response optimization assume that the responses are independent, which 1s obviously unrealistic. The response model approach produces two responses (the and yanance) for ach original response, and since these rexponse surfaces are derived from the same model they are not independent. There are-numerous approaches 10 multiple-response optimization, incorpor- ating techniques such as goal programming, genetic algorithms and simulated annealing. These techniques are widely used in the operations research ficld, but they fave not been adopted to any significant degree by the statistical design community Acknowledgements This research was supported by the National Science Foundation Industry/University Cooperative Research Center for Quality & Reliability Engineering at Arizona State University. The author wishes to thank Julia C O°Neill, Dan Grove and Martin Gibson for providing very helpful comments on an earlier drafi of this paper. Reterences: Alaniet, MoS and Kendall, DG (1940) The watsical ama of varunceetcogenety ant the logarithms rans formation JR Sturt Se 8,128 180, Bon, G. EP (1988) Signal-to-noee rates, porformance ericra sad wansleemstons (with discon, Tectonics, Mi 140. Bon, G. EP. Brgzand S. and Fung. CA. (1988) An explanatia and crtnqur of Trpuste’ coutrinatves to quality ccppmeering. Qual Reliah Ememe dat 4.128 131 Ban. GE Pasd Draper, NR (IGN) Emerica! Made! Auch and Reyponse Surfaces New York Wiley Bon. G EB. Hier, WG and Hunter. (19°R) Sans jor Experimenters Now York Wik lon, GFP and Meyer, D (1986) An analysis of unveplicated rastiona fasterials Zeviomenyess 28 11 18 Box GE. P and Wilkin KB (1951) On the caperimental staininem of opciniem oedeom JR State Sac 13. 14s Bymc. DM and Tagucly,§ (1987) The Tagucls appweach to parameter det “Ghat! Poe 38, Dice 18 24 (Carell, 1 aed Reuppen.D 198K) Trawsfirmation sad Mergers me Regorstiom New Ver Chaprcan sel Hal Coleman. 0 E and Montpemen, DC (1990 A xystemanc approach o planmung for a deupaed sahingnal experince (wich discustnon) Tevhmamecricn 38.1 27 Engel J amd Huele, AF 11996) A pencralized linear meeting approsch Wo tbe dewgen Frc temenics 346 178 Grego.) M (1991) Generalized linear models anal proces sana J Qu! Tecan), 26.248 298 Hamala ME set Noble} A. (V0F Groselize neat mks fr quality segrnemen xpsrnents Qt Tr, TD? jot‘Comments on the Paper by Montgomery 173 Maeer, 1S. (1948) Stansncal devign appliod to product deviga J Chaat Techmol 17.210 221 (1934) Let'all beware the Laon square (hail Emyng. 1483 408 Kackat. RN (1989) OfTlene quality contol, parametes deugn, at the Taguch methot J Quai! Fev. 17, 178 ti Khon A Hand Comell.J A (1M) Reipomie Surfaces Designs anct Anstises, Zed edn New York Dckher Locan JM (1994) How to acheeve a robust process using respons surface rcthodology J Qual Techen! 3h 248. 320 McCullagh, P amd Nekder, JA: 1LOXDE Gemenulized Linca Mestete, 2nd evn. New York: Chapman and Hatt AMonigumery, 1 C (1990) Using fractional (actors designs for tbust process developenent Qa! ingme, 3 143208 The ane of sistisical process control and dewga of cupetiments in proact and process rmpecmement HE Toms 2.4 17 (1997) Desogn dial Anal ads of Experiments, #th eda New York Wiley Myen RHE 1511) Respome surface methodology in quality umprineinient. Comms Sanat Theory Mesh 20,487 476 Ayes, RIL, Khun AT and Virung. GG. 11972) Response surface alternatives to the Tapiect robust parameter desig agpraach tor Stunae i, 1511S Myer. i Hand Montgomery, D.C (19S) Reyrinve Surface Methodaders. Privest und Product Optamramen iam ‘Deevigend Expermran New York Wiley 11997) A natoral on generalired fmear model: J Qual Tavhno! .29, aie, VN (ed )(1992) Taguchi's parameter deugn a pate! discussion Fry Aeametties, 34, 127— 16h Pigmnetio, Jo 1 J and Rambcry. JS (1992) Top ten tnumphs and tragudicy of Genict Taguchs Qo! Exeny, 4. on-2s Stoemabe AC, Toa KL and Wa CF J (1591 Fechmametrics 33. 315 $20 Taguchi, G17) Srstem oy Eaperimental Design Emgmeering Methods bw Optimsy Quality dnd Minimaze Cost Sew York: Uaipod Kemet Intemational {1991 frente tom 19 Qasliry Exigineering. New York: Unipubs Kins international Tagects, G amt Wu Y 1198) feerliction co: Offtime Chaulity Control. Nagoya Central Japan Qual Conmrel ‘Ausocutoa Ving. GG and Bobs, LL 11997) Respome wrfaces for the mcan sad samance wimg » bemparametns approach Qut Teche 30,202 291 Vannes. G G. and Schaub, B (1986) Expenmental downs for estimating both mean and vanance funesions J Quo! Fecimod 2813S 147 Economcal cxpermeniation mithads for robust design ‘Comments on the paper by Montgomery Dan Grane (Jadepemdent Consultant, Farchar) This rsa competent survey of where we currently stand in industnal experimentation, with many uscfal insights. However, from the perspective of a consultant working mosily with engineers, there are several issues and problems which tum up regularly in practice but do not receive attention here Faartamce motetting Thete 15 now quite an impressive literature’ on variance modelling im relation to robust engineering design The paper gives details of onc type of model, denved by taking.a fixed effects model such as ‘and allowing the noise variables (the <3} to vary randomly. giving equation (3). Engel and ) generalized equation (3) by allawing ote be a function of the controllable factors x, ertber Vina gencralized linear model of by using the adeas in Chan and Mak (1995). ‘(Other generalizations of equation (S) are cleatly feasible, for example to cater for » multilevel error saructure, bat before we are earned away with ou! enthusiasm for modelling we nced to ask how often the nosse factors can sensibly by thought of as varying randomly “in the real woeld”. Characteristics of System components in mass-production, and noixy signals from one subsystem fo another, arc obvious candndstes for this type of model. For others, such as climate, this formalization rs not useful and if se ‘nant fo reduce the vanation in output we must work directly with the svatem sersetitees. measured by the change m response when = is vaned (for f= Leo... F ‘Diese * ruduestiness porinlenis uch the to enginces @ Dymame robustness 1 Taguchi's name for an engineering scenario in which the aim 1s 10 ene robust relationship between an input and an output, In thts context robustness means that the relationshop remains almoxt constant for different scthings of the noise factors, Miller and Wu (1906) reviewed Taguch'’s appmach and proposed an alternative method of achicemy the sante goal, Therr method statistically speaking. similar in sprit bat lex daunting im detail thaa the proposals it chapecr 12 of Grove and Davis (1992), The Istter discussion links the modelling to Taguchi's peneral enginecring: philosoptiy174 Comments on the Paper by Montgomery Because of its relative complenity, and the huge experiments which i can gree tise-to, thes is an area of robust design where large potential gaans are to be had fram a sequential approach to experimentation. combined with the systematic use of engincenng knowledge, The losses from an unthinking application of sagnal-to-nonse rafios can be correspondingly large. Twas sorry nat 10 soe this topic mentioned an Doug Montgomery's survey because many of the engincers whom [alk 10 find Taguchi's ‘dynamic’ formulation very appealing Response surface methodulig aad accunucy in predicnon Many engineers are very demanding when i comes to predicting accuracy! Sometimes the classic sccoml-order model docs a suffickendly goad yob for them, but somctumes it docs not. even aflet response of Getor transformation, of the inclusion of sclected higher order interaction terms (as 1a’ Cornell and Montgomery (1996) The well-known textebooks on response surface methodology have little else to offer, and this papce docs not try to go beyond the staridard assumptions ‘One posubility ny two-s2age modelling. If the response 1s particularty ill behaved in one dimension, we fila relatively complicated model in just that dimension, identify key features of this “bocal” fit and model them as a function of other factors, This “repeated measures’ idea has been apphied very succeuafully to a 1 (1998). Related ideas appear in Bisgaard and Steinberg (1997 some or all of the firstestage features have a direct engineering interpretation In the application of Halliday cr al. (1998), this techmigue also helped 10 solve the problem of strange- shaped engineering constraints on the factor spare. because the design of the data collection for the local modelling could be separated from the design in the remazning dimensions. Another posssbility that stays fairiy close to standand response surface methodology 1s the piecewise fitnng of (say) a pair of quadratic models swith a smoothish jon. This docs not nccesanly have to involve non-linear fiting because the position of the knot or join may be well established by experience Rum order In engineering experiments, 4 random run onder is rarely feasible. and there is considerable interest in the following simple peoblem: given 3 set of factor combination, in which ordet should they be run, taking account of cost and of possible trends and step changes im the response? Prowndes! thal we have a tentative model of the trend or the step changes (block effects), we eat (a) expand the V-matnx with one or more columns that correspond to the trend of block parameters and {b) rearrange the runs to “minimize” the cortelanons between these new colurans and the ongral columns representing factor effects, subject to cost constraints ‘There are several norms that can be used tn this minimization’, melding \(@) the maximum correlation. pioneered by Draper and Stoneman (1968), and (b). the determinant of 17. pioneered by Joner and Campbell (1976) As these rather old tefcrences show, the nevessary theory has been around for a whale. but very few commercial packages offer to choose a run onder: The Taguchi comonmverey The controversy about Taguchi methods as addressed ii the paper, but the Aitforical tone suggesting that the controversy #5 over —does nat fit my experience. In same firms at least. the argument over signal-to-nose ratios 18 rumbling on Dou Montgomery is ight to pot some of the blame on the centreprencunial manner in which Taguchh’s ideas have been disseminated. but we also nce to recognize the belecf among many enguncers that conventional state obscure and difficult, They ate then told 1m Tagucht-sisle training that they can stop worrying about statistics because all that they need to dos to. sclect the appropriate xignal-to-noise ratio fir their type of data. It ix not xurproung that many of them find thrs approach very attractive IC Lam nght, statisticians must accept a share of the blame. Although we would all like to see more Matistice taught 10 engincers, many of them do encounter a course of module of some kind the unfortunate thing is how many of them look back and say that they could mot sce the point of mt Ht has bern saad many times before, bul we must put more emphasis on data collection, data analysand modelling. and much less oa formal inference, * Interox tions sand ruteasimens Finally. | am puzzled by the author's comment on Fig. 4. This diagram makes it cleat that if we assumeComments on the Paper by Montgomery 178 ly close 10 0, Which will completely eliminate confirmed by the subsequent modelling eveteise “smooth” factor effects there must be a setting of the effect of =). This Martin Gibson ( Morons Hitchiv Thave the following comments on the pape. (2), There 1s no mention of the ideal function highlighted by Grove and Davis (1992) (also mot en the references), Experimenters nced 10. find respome measures for what they (ot customers) want father than what they do not want. This nceds to be made more explicit (b) Taguchi provided a family network of designs bul, as Diamond showed, Hadamased mates have done much the same thing fe) Doug mentions characterization, control and optimization and 1 still wonder Whether these phases can be connected to Tapuch’s system-—parametet tolerance principles (d) The concluding remarks do not expand oa the epsrational rescarch techniques of goal program» ming, genetic algorithms and simulated annealing. | only have a scant appreciation of these and ss necessary to snform the reader of the paper about their suitability and capabuiny. 1 would Bane Ithed to have scen some simple examples. fe} Other areas of research would include mixtures inside fractional factonals vers fractional factorials inside mixtures. Which is the preferred way forward? 4) What happens in simultaneous optimization of categorical and continuous variables? 4g) Another method i the analysis of means for process optimuabon Who i usmg it? What new research his been going on? ftv What about the random strategy approach (cf. Diamond) for computer simulation applicalions” ‘Overall the paper has potential and 1s mteresting though it would be enhanced with answers to these points, ‘Tim Davis (Fond Hirke, Cologne) Within the Ford Motor Company, we work with a generalization of Taguchi's on ginal three categories of noise Factors. as follows: 1) picce-to-pioce variability due to mass-production; (2), product changes over time and mileage, such as wear: (3) customer usage pattems and duty cycles (4) extemal environmental effects, such as climate and road conditions: ($8) internal cnvironmental effects (unwanted effects fom one component to another}, such as \ibeation and beat transfer, and noisy signals (called “inputs” in Montgomery's Fig. 1 lt ts not clear to me whether all these types of noise fit into the modelling framework proposed by Montgomery Nevertheless would be a mastake to rely entirely om parameter design to achieve robust designs, emportant though thes methodology ts. Several robusiness stratepics arc available to the design engineer to deal with nose factors. among them the follenving (AY change the technology of the product (¢ ¢ change from a mechamcal to an electtonee speed- ometer) to achieve better perfor. (B) make the current design assumpt to the noise factors wdentificd, through 18) discovery of eontrel-by noise Factor interactions (paramctet design) ot (a1) streagthenmg the current des gr (C) change the magnitude of the noise factor: (D) insert a compensation device for the notse factor (¢ ga heat shield), (E) disguise the bem from the end-user ‘The successful achievement of a robust design involves combsning strategies (Ab (E) to deal with the identified noses (1). (5) Withits Ford, we call this approach "noise factor management” For example. in an automobile, # #5 extremely important fo minimize unwanted cross-talk between components (nore factor (Si) by. good package design (stratcey (C1) rather than by relying on parameter despa (strategy (BX) of compensation devices (strategy (DD) applied to the components The first case-study presented by Montgomery 1s interesting from the nowe factor management poral of view. ance one of the noise factors identified, +; (temperature), turned out to have po effect oa the sens1o Or lg 176 Comments on the Paper by Montgamery fesponse within the range over which 1 was investigated, This is a major discovery? | could easily imagine a situation Where the next round of cxpermeiits would be aimed at secing how far temperature could be allowed to vary before it had an effect: this mht result in no longer roquinng an expensive cootrol mechanism for temperature The imesiment thus saved could be diveried inio a bettcr control mechansam for the other nove factor =) (oxide thicknicss) to reduce even further the variability of the fexpomse T hope to publish 2 more detitled discussion of noise factor management in the near future. Author's reply T would like to thank the discussants for thet interesting comments. They hive adentiied scveral important issues that time and space prohibited me from presenting formally im the paper. and | shall take thes opportunity to offer a bnef reply te same off their remarks I belicve that the charactenation control optimization paradigm is an effective alternative frame- work to the Taguchi approach of system -parameter-tolerance denign. It logically promotes sequential sxpenmentabon by separating ihe objectives of system charactenzation (in which we attempe to isolate the important factors first) and optimization (where we take the best advantage of this icformaticn) The Taguchi approsch always seemed to confuse these objectives of experimentation, of to attempt 10 accomplish both objectives in a single ey The variance modelling aproach illustrated in the paper can be extended to-various categories or types ‘of porse vanables. All that 1s required is that the noise vanables Be controllable for the purponcx of the expenment, and that enough be known about the variability of a noise factor so that is standard deviauion can be estimated. This could be done from a rough idea of the range over which the nowse factor wall vary. There have been some extensions of this idea: for example. Pledger (1996) showed how observable but uncontrollable noise factors miay be incorporated in the general respame model framework, Wolfinger and Tobias (1998) have presented methodology for simultancoudy modelling three components of a general mixed model formulation of the robust desizn problem: location (fixed) effects. dispersion effects and random effects Both Dan Grove and Martin Gibson comment on dynamic robusiness problems. ami, of course, experments where the response is 3 performance curve are widely encountered m practice. My feeling has always been that the Taguchi approach of creating 3 “dynamic” signal-to-noise ratio is generally flrecd just as the usual “stanc” signal-to-noise ratios are problematic in mast applications. For example, the static smaller-is-best and larger-is-bes! signal-to-noise ratios actually confound location and disper jon, and torn out to be better measures of location in most cases. Furthermore, it is obvious on inspection of these quantities that they ate not proper signal-to-notse ratios, because they are not dimensionless Consider the dynamic signal-to-noise ratio SN = 10logy(f/o¥ where /! ts usually called the sensitivity measure and the nesponse of interest 115 connected to a signal factor Mf through the relationship vs AM +e and « has standard deviation 0. Obviously, this signal-to-noise ratio t not dimensionless. and 4 its usefulness as a wadely applicable summary of the performance characteristics of mterest in the response vanable is. in my mind questionable. Indeed. Lunani ef al. (1997) have recently shown that Taguchi's dynam signal-to-noise ratios only appropriate when the vanance of the fesponse is proportional to the square of the scrtivity measure This is analogous to the limilztion of the usual static signal-to-noise rato for the nominalis-best case, which is eeally enly appropriate when the canance 18 proportional te the square of the mean (sec Box (19N%)), Thus, when 4 performance curve ts of imetest. ry preferenee id be an analyats along the lines of the proposals in Bisgaard and Steinberg (1997) guchi hax always emphasized the nced to design experiments to reduce greatly or to elmmate interactions, and the signal-to-neise ratios are often claimed to assist im this, along with other tech- magus such as the use of sliding factor levels, the proper choice of a response vanable and so forth. Famhermoce, Taguchs hay und that, of we cannot design an capenment by using these tools such that addinity (or absence of interaction) is attained, then we do not understand the engmeering system sufficently well to conduct efficicnt experimentation, This argument to me is analogous 1 one that claims that we should not inc a control chart on a process because the process 1% wo vanable to use19 cf Qs Comments on the Paper by Montgomery 177 Saishical process control! Now there may be some systenis an which the underlying engineering scicee '» sufficiently well understood that wee could design the experiment to obtarn the desired additivily. uch as felatively simple mechanscal systems where Newton’ laws directly and oftcn fairly obviously apply However, there are many systems in the senvisonshuctor, chemical and biotechnology ficlds where tbe underlying mechanisms are much more obscure and less well understood Its just these types of system where the use of small sequential expenments and the characterization contrel-optamt tation philos- phy wall havea high pay-off s Tim Davis points out, many strategies are available to the enginecr to achieve robustness I classify them into two general categories: statistical strategies, of which the vanance modelling techeaque resented in my paper ts an cvample, and technological stratewies, such as selecting a different materal altermatne. of changing some aspect of the design, such as the mechanical-to-clectranic speedometer example Robust desiga ts 20 important problem to both the engineering and the statistics communities. Many important aspect of this problem have not been well studied and for which good asy-t0-apphy, standard solution methodologies are not avalable. | hope that the discussants and 1 have stimulated others to work in thts challenging and useful arca. References in the comments Besgzand. Sand Stenberg. DM. (1997) The deegn an analyen of 24° xs prowpe cuperumcets Teclmomeines. 3. Bon. G EP (1888) Sipnalownus ration, performance enters and transformations (th docassee),Teeknometnis, ML 1, (Chas Lat MAL TK (1995) A repression approach for ditconcrang nll taration around a tags Ay) Sour ¥077 Corset JA. and Mectzomer D.C (1996) Interaction mele 2s abtematres to -order polynomials J Qa Ferhat WL 103-176 Draper. NR ant Stoneman. DM (1968) Factor changes and imear tends in cightrim tmosievel factor! designs Tec tmomezrns, 18, 301-310 Grove. DM mat Danse TP (1992) Engineering Quali & Experimental Besogn, Londo: Loagman Hoticks. T. Lwramce, AJ and Dinix T (18 Engine-mappang experiments: 4 teo-sare approach Technomerics $8, 130-136 Jouocr, BL at Campbell. (1979) Dewgeang experuments hen run order i mpeetant. Tevomeans. 1249) Lissa ML. Nar VN and Wasserman G'S.11997) Graphical metbous for rabunt dewgm = th dynam sbaraciersoos. (Qa! Teche 29.327 338 Mlle. A aed Wis CF (29%) Parameter dewegn for wpmal-rewpomse systems.» diferent look at Tap parameter deugn Satur Se 12213 . Plasper, M (1998) Observable uncwenliabic facoryin pursmeerdewen I Quul Technol, 28,153. 162 Woltnger, RD. and Tobax RD (1998) Jom extimanos of locabon, dspenioe, andl andes fests sm bus design Tecbnomans,4@, 62-71