ITERATIVE DESIGN OF EXPERIMENT: IMPLICATION OF GENETIC

ALGORITHM TO KRIGING MODELLING METHOD.

Hoang K.H1. and Laksh S.2

Engineering Drive 4, Singapore 117576

ABSTRACT

A MATLAB program was written in this work for the purpose of design of experiment for batch
experiments. We present the program that based on previous data, iteratively searching for a
new set of experimental points, updating the model and searching for the optimal points. The
stochastic method genetic algorithm is used to solve the multidimensional unconstrained non-
linear of maximizing likelihood function posed by Kriging method. Genetic algorithm is also
used as a searching tool for the maximal of 'expected improvement' which is the criterion to
select additional sample points. The program is tested and shows promising result.

INTRODUCTION

In any manufacturing industry, where processes are highly complex, it has been more difficult to
develop models for global optimization. Under that condition, it is desirable to have a design of
experiment (DOE) scheme that can gather most unbiased data from least number of experiments
conducted, which can eventually lead to confirmation of optimal operating condition. To
visualize the relation between input and output, either time-consuming hard-to-build
mathematical model or experimental-based statistical model will be used. By far, the most
popular statistical model is the traditional response methodology (RSM), which employs the
second or third order polynomial and least square regression technique. While this methodology
provided an easy estimation, it has rather limited capability to solve non-linear system
accurately. Higher order polynomial can be used to solve the problem, but require high number
of experiment to estimate all the coefficients. Recently, Kriging model, which has been applied
in geostatistics, has gained popularity in this field, by providing a flexible structure that not only
can be fitted accurately into problems that seems complex and unnatural, but also providing
uncertainty for regions. The limited of this technique is the difficulty of getting the right
program to fit the model and additional effort to use the model compare to normal polynomial
model. LecturerIn this work, we adopted the framework suggested by Jones D.R. (‘A Taxonomy
of global optimization methods based on Response surfaces’,2001) and implemented the
genetic algorithm as optimization solver to develop into iterative method.

____________________________________________________
1
Student ,2Senior Lecturer
ITERATIVE METHOD

Kriging modelling

Kriging models origin in mining geostatistical application involving spatially and temporally
correlated data. Kriging combines a global model plus local deviation:

(1)

With f(x) is known polynomial, or in this program, is a constant, while is realization of a

(2)

where R is the correlation matrix where R(xi,,xj) is the correlation of between any 2 sample xi, xj.
There are several correlation functions, which is chosen by user. In this work, we use the
Gaussian correlation function in the form of

(3)

The parameter of this model are ( ) which is estimated by solving the maximization of
likelihood function:

(4)

Where both R and ƒ are function of θ. The predicted value of on x* and deviation are

(5)

) (6)

(7)

While any value of will create interpolative Kriging model, best fit model is created by
solving multidimensional unconstrained nonlinear problem of maximizing likelihood function.

Expected improvement

In this study, we adopted the concept of expected improvement (EI). Instead of directly
searching for the optimal, EI will increase as the predicted response is smaller than the best
value obtained and/or the variance is large enough. Thus, for the next iteration, we use genetic
algorithm to search for next experiment point that have highest EI value. If all the EI value is
close to zeros, the iteration may be terminated as there is nearly no improvement can be found.
Improvement of point x* is

(8)

Where ybest is the minimum value has been found. Since we defined y* is Gaussian distributed,
with mean and variance , the expected improvement is calculated as

(9)

Where (.) is the standard normal cumulative distribution function and is the standard
normal density function.

Genetic Algorithm

Genetic algorithms (GAs) are stochastic global search and optimization methods that mimic the
metaphor of natural biological evolution. GAs operate on a population of potential solutions
applying the principle of survival of the fittest to produce successively better approximations to
a solution. At each generation of a GA, a new set of approximations is created by the process of
selecting individuals according to their level of fitness in the problem domain and reproducing
them using operators borrowed from natural genetics. This process leads to the evolution of
populations of individuals that are better suited to their environment than the individuals from
which they were created, just as in natural adaptation.

Iterative method

No

Ye
 Figure 1: Flow chart for iterative design of experiment

The method starts with initial set of data, which can be previous experiments, or process
database, or a new set of experiment. The data collected is process by Kriging modelling and
genetic algorithm to search for best fitted θ , thus, the best model. Due to the nature of genetic
algorithm and depend on the level of non-linearity of the responses, the modelling process can
be done several times to get the best possible results from likelihood function. From the
estimated parameter, Kriging model will be constructed, and genetic algorithm acts as EI search
engine to find the next best point. Again, as the nature of genetic algorithm, this process can be
done several times, so the best results can be found and further distilled through fuzzy c-means
to the desired number of experiments.

CASE STUDY

The test function used for the method is Himmeblau’s function:

This function is a multi optimal function ( 4 minimal and 1 local maxima). In the test case, the
initial data set is 52 designs, which may be used for normal response surface methodology. The
next best EI points are located by GA, and then distilled by fuzzy c-means to 1 to 4 points. The
iteration continue to 6th batch, with 40 points tested where next best EI were 10-6

CONCLUSION

In this work, a methodology for design of experiment and search for global optimization was
demonstrated. We adopted Kriging modelling and Genetic Algorithm to provide the unbiased
program that has the capability of generating fairly accurate model, of searching for the next
experiment with criteria of best ‘expected improvement’ to further explore the regions and
search for optimal. The case study implied that the proposed method is effective, and still, there
are improvement to be done for this method.

REFERENCES

A.J. Chipperfield, P. J. Fleming, The MATLAB Genetic Algorithm Toolbox, retrieve from
mathwork.com
Jones, D.R, A Taxonomy of Global Optimization Methods Based on Response Surfaces in
Journal of Global Optimization,2001
Jones, D.R, Schonlau,M, Welch,W.J, Efficient Global Optimization of Expensive Black-Box
Functions in Journal of Global Optimization,1998
Junghui Chen, David Shan Hill Wong,Shi-Shang Jang, Seng-Lu Yang Product and Process
Development Using Artificial Neural-Network Model and Information Analysis in AIChE
Journal, April 1998
Zhang,G., Olsen,M.M.,Block,D.E, New Experimental Design Method for Highly
Nonlinear and Dimensional Processes, in WileyInterscience, June 2007