Robustness evaluation in analytical methods optimized using experimental

designs

Sergio L.C. Ferreira, Adriana O. Caires, Thaise da S. Borges, Ariana

M.D.S. Lima, Laiana O.B. Silva, Walter N.L. dos Santos

PII: S0026-265X(16)30359-9
DOI: doi: 10.1016/j.microc.2016.12.004
Reference: MICROC 2632

To appear in: Microchemical Journal

Received date: 14 September 2016

Revised date: 30 November 2016
Accepted date: 1 December 2016

Please cite this article as: Sergio L.C. Ferreira, Adriana O. Caires, Thaise da S. Borges,
Ariana M.D.S. Lima, Laiana O.B. Silva, Walter N.L. dos Santos, Robustness evaluation in
analytical methods optimized using experimental designs, Microchemical Journal (2016),
doi: 10.1016/j.microc.2016.12.004

This is a PDF file of an unedited manuscript that has been accepted for publication.
As a service to our customers we are providing this early version of the manuscript.
The manuscript will undergo copyediting, typesetting, and review of the resulting proof
before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that
apply to the journal pertain.
 ACCEPTED MANUSCRIPT

Robustness evaluation in analytical methods optimized using experimental designs

T
Sergio L.C. Ferreira1,2*, Adriana O. Caires1,2, Thaise da S. Borges3,

IP
Ariana M.D.S. Lima1,2, Laiana O.B. Silva2,4, Walter N.L. dos Santos2,4

R
SC
NU
1- Universidade Federal da Bahia, Instituto de Química, Grupo de Pesquisa em Química e
Quimiometria, Campus Ondina, 40170-115, Salvador, Bahia, Brazil
MA
2- Instituto Nacional de Ciência e Tecnologia, INCT, de Energia e Ambiente, Universidade
Federal da Bahia, 40170-115, Salvador, Bahia, Brazil
D
TE

3- Universidade Federal da Bahia, Escola Politécnica, 40210-630, Salvador, Bahia, Brazil

P
CE

4- Universidade do Estado da Bahia, Departamento de Ciências Exatas e da Terra, 41150-000,

Salvador, Bahia, Brazil.
AC

*Correspondence author:

Sergio L. C. Ferreira
E-mail: [email protected]

FAX: + 55 71 32836831

1
 ACCEPTED MANUSCRIPT

Abstract

Robustness has been defined as being the capacity of an analytical procedure to

produce unbiased results when small changes in the experimental conditions are made
voluntarily. The robustness tests are important because sometimes the analyst needs to

T
modify one or a few chemical and/or instrumental parameters of a procedure, and this can

IP
only be done if these changes do not affect the method's accuracy considering the

R
previously optimized results. The two-level full factorial designs would be the most

SC
recommended for robustness evaluations of analytical methods. However, for operational
convenience, the fractional factorial designs are more often used. The Plackett-Burman

NU
designs have been one of the most employed in robustness evaluations. Some examples of
robustness test applications in some analytical methods employing different chemometric
MA
tools are given. A discussion about the terms robustness and ruggedness is made. Seventy
five references are cited.
D
TE

Keywords: Robustness; Ruggedness; Factorial design; Plackett-Burman designs;

Experimental designs; Curvature test; Pareto chart.
P
CE
AC

2
 ACCEPTED MANUSCRIPT

1. Robustness

The optimization step of the analytical methods has often been performed utilizing
multivariate techniques [1-3]. The two-level full factorial design allows a preliminary
evaluation of the factors by the development of linear models [4-7]. However, the critical

T
conditions of the factors have been achieved using the surface response methodologies such

IP
as: Doehlert matrix [8-10], composite central [11-13], Box-Behnken [14-16] and three-level

R
factorial designs [17-20], which allow the obtaining of quadratic models. All of these tools

SC
have their advantages and drawbacks and provide better results than the univariate
optimization methodology. However, robustness studies are required to assess the

NU
vulnerability of the factors in the analytical methods optimized by using the chemometric
tools.
MA
Robustness is a parameter that has been evaluated in validation studies of analytical
methods, which has been defined as the "capacity of an analytical procedure to produce
D

unbiased results in the presence of small changes in the experimental conditions" [21].
Another concept defines that "robustness would characterize the behavior of the analytical
TE

method when experimental variables inherent to the analytical process are slightly
P

modified" [22]. A third definition proposes that this analytical parameter is "the capacity of
CE

an analytical procedure to remain unaffected by small but deliberately introduced variations

in method parameters and provides an indication of its reliability during normal usage" [23].
AC

Robustness and ruggedness are frequently confused and interrelated. However, there is a
distinction between these two analytical parameters. The US Pharmacopeia defined
ruggedness as being "the degree of reproducibility of test results obtained by the analysis of
the same sample under a variety of normal test conditions, such as different laboratories,
different analysts, different instruments, different lots of reagents, different elapsed assay
times, different assay temperatures, different days, etc." [24, 25]. On the other hand, the
International Union of Pure and Applied Chemistry (IUPAC) defined that "The ruggedness of
an analytical method is the resistance to change in the results produced by an analytical
method when minor deviations are made from the experimental conditions described in the
procedure" [26].

3
 ACCEPTED MANUSCRIPT

1.1. Steps developed in a robustness study

During the robustness evaluation of an analytical method, the following steps are
performed:
a) Selection of factors and their levels

T
During the optimization of an analytical method, the critical values of the factors, also

IP
known as nominal values, are obtained generally by employing quadratic (process variables)

R
and quadratic or cubic models (mixture designs). On the other hand, the robustness

SC
evaluation of the analytical methods generally involves faster experimental designs and
most of these are established by linear models. A factor is considered robust for a method
when this, as well as all their interactions, is not statistically significant for the experimental

NU
dominion defined in the tests. Then, robustness studies are frequently required, considering
that the experimental dominions of the factors are established symmetrically in relation to
MA
the nominal values achieved by optimization. These factors are modified in short
experimental dominions to ensure that the interactions between these can be negligible.
D

Often, the ranges (experimental dominion) for the quantitative factors such as concentration
TE

of solutions, flow rate, extraction volume, reagent concentration, preconcentration time and
others are defined in percentage terms. However, robustness studies for the pH factor
P

involve unit or fraction variations of pH.

CE

b) Choice of the ideal experimental design for robustness evaluations

AC

Several chemometric tools have been employed for robustness assessments.

Therefore, two-level full factorial and fractional factorial designs, Plackett-Burman matrix
and others designs, including some response surface methodologies, may be used in
robustness tests. Each one of these has their advantages and drawbacks. The number of
factors can influence directly the choice of the ideal design. The two-level full factorial one
has several advantages, however if the number of factors is large, this design is very
inappropriate. The Plackett-Burman design is one of the most used because it allows the
evaluation of many factors with a minimum of experiments. Details, advantages and
limitations of these designs are described later as a topic of this work.

4
 ACCEPTED MANUSCRIPT

c) Development of the experimental work

After choosing the appropriate experimental design for the robustness test, the
experiments should be performed in a random order. Sometimes, for practical reasons, one
or more factors are blocked, but the others are randomized. Situations that require

T
experiments in more than one day also can be performed. In these cases, the experiments

IP
are established in a blocked form.

R
During the developments of the experiments, it is desirable to perform replicate

SC
experiments under conditions defined as nominal values. These replicates must be made at
the beginning and at the end of the robustness test experiments, since the results of these
experiments are employed to check the performance at the beginning and at the end of the

NU
experiments. Also the replicate results are used to determine the experimental error, which
is used to estimate the significance of the factors and their interactions.
MA
In the robustness studies, one or several responses are quantified. However, the effects of
the factors and their interactions are calculated considering only one response. This way,
D

studies involving more than one response are established using a multiple response
TE

function. There are various strategies to calculate a multiple response. The desirability
function D, which was proposed by Derringer and Suich in 1980, has been the most
P

employed in experimental designs [27].

CE

d) Calculation and the significance characterization of the effects

AC

The estimated effects of the factors are calculated by the following equation:

EA = {ΣY(at positive level)/(N/2)} - {ΣY(at negative level)/( N/2)} Equation 1

in which A represents the factors studied or the dummy variables established in the
experimental designs employed in the robustness study, and N=2k, in which k is the factor
number [24].
The factor effects should be written as confidence interval (CI), as the equation:

CI = Effect ± (Seffect)x(t) Equation 2

where Seffect is the standard deviation of the effects and their interactions, and t is the
Student coefficient, which is determined by the degree of freedom of the replicates used for
calculation of the estimated variance of the experimental error. The effect of a factor,

5
 ACCEPTED MANUSCRIPT

therefore, is considered significant when this confidence interval does not include the 0
value [24].

Often, in robustness studies, replicates of the central point (nominal values) are employed
for the determination of the standard deviation of the effects. So, this statistical parameter,

T
Seffect, is calculated with the following equation:

IP
Seffect = (4S2/N)1/2, Equation 3

R
SC
where S2 is the estimated variance of the experimental error, N is the number of different
experiments, which is determined by N = 2k, and k is the factor number.

NU
The estimated variance of the experimental error is determined by the equation:

S2 = Σ (xi - xo)2/(rn-1),
MA
Equation 4

with xi as the individual values of the responses obtained, xo the average of the values of
these responses and (rn-1) the degree of freedom, considering rn as the number of
D

replicates performed during the experiments [24].

TE

The Pareto chart is one of most employed graphic options to show the significance of the
P

factors and their interactions in experimental designs. Generally, horizontal bars are
CE

presented to evidence the importance of the factors. In this, the effects are presented as
"standardized effects" (SE), which are calculated by the ratio between the "effect values"
AC

and their standard deviation obtained during the optimization. Also a vertical line is drawn
to show the factors and interactions that are significant. This line on the x-axis corresponds
to the critical value of the Student's t distribution, which is determined considering the
degree of freedom involved in the calculation of the standard deviation of the effects and a
confidence level established in the work [28]. Generally, the confidence level established in
Analytical chemistry is 95%.

SE = Effect value/Seffect Equation 5

6
 ACCEPTED MANUSCRIPT

1.2. Robustness - Practical aspects

In practical terms, robustness studies are required considering that during the
development of analytical procedures, the operators have questions to which the answers
can only be obtained by the determination of this parameter of method validation. Here are

T
some practical examples:

R IP
a) The optimization of a procedure proposed for the determination of palladium in several

SC
samples by dispersive liquid-liquid micro extraction atomic absorption spectrometry
recommended the use of 1.0 mL of (pH 5.0) acetate buffer with analytical concentration of

NU
acetate of 0.20 M [29]. In this case, can an acetate buffer with pH 4.75 be used? Can the
analytical concentration of buffer solution be changed from 0.20 M to 0.50 M?
MA
b) An online preconcentration system proposed, for the determination of mercury in
biological and sea-water samples, the use of 120 s preconcentration time and a sample flow
rate of 4.0 mL min−1 [30]. Can the sample flow rate be increased from 4.0 to 5.0 mL min-1 ?
D

May the preconcentration time be increased from 120 to 150 min? How do these changes
TE

affect the accuracy of the method?

P

c) During the determination of lead in several samples employing electrothermal atomic

CE

absorption spectrometry (ET AAS), the instrumental parameters were established as:
pyrolysis and atomization temperatures of 900 and 2000oC, respectively [31]. Can the
AC

pyrolysis temperature be increased from 900 to 1000oC? Can the atomization temperature
be decreased by 10% of the recommended value by general procedure?

d) An extraction procedure of molybdenum(VI) ions from acid solution using polyurethane

foam recommends the use of 100 mg of sorbent, 24 mL of 4.0 M potassium thiocyanate
solution and an extraction time of 16 min [32]. Can the potassium thiocyanate solution
volume be changed from 24 to 20 mL? Can the extraction time be changed from 16 to 20
min?

All of these questions can be widely answered by employing robustness tests using
several chemometric tools.

7
 ACCEPTED MANUSCRIPT

1.3. Robustness - Theoretical aspects

The efficiency of a robustness test using multivariate methodology is ensured if the

T
following criteria are observed during the development of the experiments:

IP
a) All the effects of the main factors and their interactions are not significant for the
established experimental dominions.

R
b) The models generated do not present lack of fit.

SC
c) The curvature test is another tool that allows the evaluation of the robustness of the
factors when the two-level full factorial design is performed. In this case, if the factors are

NU
robust, the curvature obtained in the linear model generated is not significant.
The curvature is calculated using the equation:
MA
Curvature = RFD – RCP, Equation 6
where RED is the average of the responses obtained from experiments achieved by the
D

factorial design, and RCP is the average of the responses obtained for the central point. The
curvature can only be calculated if these averages expressed as confidence interval at 95%
TE

level are different. If these means are statistically equal, then the curvature is zero and the
P

method is robust considering the established experimental dominions [24, 28].

CE

1.4. Chemometric tools employed in robustness tests

AC

1.4.1. Use of two-level full factorial designs for the evaluation of robustness
The two-level full factorial design is the most appropriate chemometric tool to
evaluate the robustness of analytical methods, because it allows determining the
significance of all main effects and their interactions without confounding between them.
Moreover, this design allows the development of the curvature test, which can also provide
more information about robustness of the analytical method studied. In recent times,
although laborious, this design has often been employed for robustness evaluation of
analytical methods [14, 33-46].

8
 ACCEPTED MANUSCRIPT

Application 1
An automatic on-line pre-concentration system using a knotted reactor optimized by
employing the Box Behnken design was proposed for the determination of lead in drinking
water. During the validation studies, the robustness of this method for the pH factors and

T
buffer solution concentration were determined by using a two-level full factorial design.

IP
Table 1 shows the coded and real values of these factors and the response (absorbance),

R
which was measured by flame atomic absorption spectrometry [38].

SC
Table 1

NU
The calculation of the standard deviation and the confidence intervals of the factors and
their interpretations are given in Table 2.
MA
Table 2

In this table, it can be seen that the two factors and their interactions are not significant for
D

the established experimental dominions. So, these results prove that this method is robust
TE

for the pH range of 8.80 ± 0.50 and buffer solution concentration of 0.024 ± 0.004 M.
P

Table 2 also shows that the SE values obtained are lower than the critical value of tStudent for
CE

the degree of freedom of 2 and confidence level of 95%. A Pareto chart, such as Figure 1,
constructed using the standardized effects of the factors, also proved that the two factors
AC

and the interaction do not have statistical significance considering the experimental
dominions that were studied.
The curvature test shown in equation 6 was also applied in the data from Table 1.
Then, the averages of the responses found by factorial design and the by the central points
expressed as confidence interval are: 0.0327 ± 0.0029 and 0.0324± 0.0060, respectively.
These results demonstrated that there is no statistical difference between these averages.
Thus, the curvature test also showed that this method is robust for pH and buffer solution
concentration in the ranges studied.
Figure 1
Application 2
During the validation studies of a method proposed for the determination of lead in
fruit juices employing slurry sampling, the robustness of the factors pyrolysis and

9
 ACCEPTED MANUSCRIPT

atomization temperatures and nitric acid concentration was investigated. Table 3 shows the
coded and real values of these factors and the triplicates of the central points, which are the
nominal values of the method optimized previously by a Box Behnken design.
Table 3

T
The data obtained were evaluated and some statistical parameters were achieved: Firstly,

IP
the effects of the factors were calculated as the equation 1. Afterward, the estimated

R
variance of the experimental error was obtained by the equation 4, considering that xi are

SC
the three replicates of the central point, xo the average of these replicates, and the degree
of freedom is 2. In addition, the standard deviation of the effects was calculated using the
equation 3. Finally, the standardized effects were obtained by the ratio between the effect

NU
values and the standard deviation of the effects (equation 5). All of these results are
presented in Table 4.
MA
Table 4
In this, it can be seen that all the effects of the factors expressed as confidence interval
D

include the value 0. Therefore, the effects of the factors and their interactions are not
TE

significant. All the results obtained for the standardized factors are lower than 4.3027, which
is the limit for the significance of the factors.
P
CE

All of these results demonstrated that the factors pyrolysis and atomization temperatures
and nitric acid concentration are robust in the ranges of 905 ± 90 oC, 1940 ± 194 oC and 1.30
AC

± 0.30 M, respectively.

1.4.2. Employment of Plackett-Burman designs in robustness tests

The Plackett-Burman matrix is one of the most employed experimental designs for
robustness evaluation of analytical methods [47-50]. It assumes that the interactions can be
completely ignored and only the main effects are calculated with a reduced number of
experiments. Its great advantage is the possibility of evaluating many factors with a low
number of experiments [24, 47, 51-53]. This tool establishes a design for N-1 factors in N
experiments, being that N must be multiple of 4. When the number of experiments required
by the design is not a multiple of 4, imaginary factors denoted as “dummy factors” can be

10
 ACCEPTED MANUSCRIPT

added to complete the design [54]. The dummy factors are defined as being virtual variables,
in which the variation from one level to another does not represent a physical change.

Table 5 shows Plackett–Burman designs, which can be used at the convenience of the
operator. Additionally, the smallest number of factors that can be studied is three. So, a

T
Plackett–Burman design for the evaluation of three factors requires the inclusion of four

IP
dummy factors, and eight experiments should be performed [47]. On the other hand, if

R
there are four factors, three dummy factors should be included and eight runs are necessary.

SC
Table 5

NU
These designs allow estimating the standard error of the main effects, also considering the
dummy factor effects. All the designs must have at least three dummy factors, and the
MA
significance of the main factors are determined with more efficiency [47]. The designs for
N=8 runs and seven factors and N=12 runs and eleven factors are shown in Tables 6 and 7
[24].
D
TE

Table 6 and Table 7

P

Bendicho et al evaluated the robustness of sample preparation methods proposed

CE

for the determination of arsenic in seafood samples by employing a Plackett-Burman design

[55]. Furlanetto et al proposed a method of employing capillary electrophoresis for
AC

rufloxacin hydrochloride determination in coated tablets. The robustness was tested using
an eight-run Plackett-Burman matrix [56]. Plackett-Burman design was employed for the
robustness assessment of a method using the Comprehensive Two-Dimensional Gas
Chromatography [57]. The robustness of a FIA-SIA method proposed for the evaluation of
total antioxidant capacity of wines was performed using a Plackett-Burman design [58]. A
method developed for the determination of isoflavone aglycones by employing liquid
chromatography, used Plackett-Burman design for robustness evaluation [59].

11
 ACCEPTED MANUSCRIPT

1.4.3. Employment of Taguchi matrices in robustness tests

The Taguchi methodology has also been employed often in robustness studies [60-
62]. It generally proposes the use of a two-level fractional factorial designs, which form
orthogonal matrices that are designated as L4, L8, L12, L16 and L32, in which the numeric

T
index corresponds to the number of experiments that must be performed. The L4 array is a

IP
23-1 fractional factorial design, which the levels of the third variable are defined by the

R
generating relation I = 123. The L8 matrix is a 27-4 fractional factorial design with the

SC
generating relations of: I = -124, I = -135, I = -236 and I = 1237. The L12 Taguchi matrix is
quite different from the Plackett Burman design proposed to evaluate 11 factors with 12

NU
trials. However, these two designs are orthogonal and should show the same results if all
interaction effects are negligible [63].
MA

Other chemometric tools employed for robustness evaluation include: Fractional factorial
D

design [64-66], Box-Behnken design [67-70] and Central composite design [71-75].
P TE
CE
AC

Conclusions

The robustness is a parameter that has often been required in validation studies of
analytical methods.
The two-level full factorial design is the most efficient chemometric tool for
robustness evaluation; however, it is inappropriate when the number of factors is high.
The Plackett-Burman matrix is the most recommended design and most employed
for robustness studies when the number of factors is high.
This work concluded that most of the authors have defined that robustness and
ruggedness are synonymous terms. However, only the US Pharmacopoeia has made a
distinction between these terms, defining ruggedness as being the evaluation of the results
achieved during the analysis of the same sample employing an established method, but

12
 ACCEPTED MANUSCRIPT

using different equipments, different analysts, different days, different lots of reagent,
different laboratories and etc. On other hand, several authors have conceptualized
robustness as being the capacity of an analytical method to produce unbiased results when
the experimental factors are slightly modified voluntarily.

T
R IP
Acknowledgements

SC
Authors are grateful to the Conselho Nacional de Desenvolvimento Científico e

NU
Tecnológico (CNPq), to the Fundação de Amparo à Pesquisa do Estado da Bahia (FAPESB)
and to the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) for
MA
providing grants and fellowships and for financial support.
D
P TE
CE
AC

13
 ACCEPTED MANUSCRIPT

References

[1] C.G. Novaes, M.A. Bezerra, E.G.P. da Silva, A.M.P. dos Santos, I.L.D. Romao, J.H.S. Neto, A review
of multivariate designs applied to the optimization of methods based on inductively coupled plasma
optical emission spectrometry (ICP OES), Microchemical Journal, 128 (2016) 331-346.
[2] L.V. Candioti, M.M. De Zan, M.S. Camara, H.C. Goicoechea, Experimental design and multiple

T
response optimization. Using the desirability function in analytical methods development, Talanta,
124 (2014) 123-138.

IP
[3] C.G. Novaes, S.L.C. Ferreira, J.H.S. Neto, F.A. de Santana, L.A. Portugal, H.C. Goicoechea, A
Multiple Response Function for Optimization of Analytical Strategies Involving Multi-elemental

R
Determination, Current Analytical Chemistry, 12 (2016) 94-101.
[4] K.M. Diniz, C.R.T. Tarley, Speciation analysis of chromium in water samples through sequential

SC
combination of dispersive magnetic solid phase extraction using mesoporous amino-functionalized
Fe3O4/SiO2 nanoparticles and cloud point extraction, Microchemical Journal, 123 (2015) 185-195.
[5] B. Parodi, A. Londonio, G. Polla, P. Smichowski, On-line solid phase extraction of bismuth by FI-

NU
HG-AAS using L-proline immobilised on carbon nanotubes combined with factorial design,
Microchemical Journal, 122 (2015) 70-75.
[6] Z. Jemmali, A. Chartier, C. Dufresne, C. Elfakir, Optimization of the derivatization protocol of
MA
pentacyclic triterpenes prior to their gas chromatography-mass spectrometry analysis in plant
extracts, Talanta, 147 (2016) 35-43.
[7] I.G. Ampatzopoulos, M.G. Kouskoura, C.K. Markopoulou, Coupling Classical Analytical Methods
with Modeling in Routine Analysis Tests - Merits and Drawbacks, Current Analytical Chemistry, 11
(2015) 244-256.
D

[8] K.L.T. Rodrigues, A.L. Sanson, A.D. Quaresma, R.D. Gomes, G.A. da Silva, R. Afonso, Chemometric
approach to optimize the operational parameters of ESI for the determination of contaminants of
TE

emerging concern in aqueous matrices by LC-IT-TOF-HRMS, Microchemical Journal, 117 (2014) 242-
249.
[9] E. Heidarizadi, R. Tabaraki, Simultaneous spectrophotometric determination of synthetic dyes in
P

food samples after cloud point extraction using multiple response optimizations, Talanta, 148 (2016)
CE

237-246.
[10] L. dos Santos, Q.O. dos Santos, I. Moreno, C.G. Novaes, M.J.S. dos Santos, M.A. Bezerra,
Multivariate Optimization of a Simultaneous Cloud Point Extraction Procedure of Cd, Cu and Ni from
Sediments Samples and Determination by ICP OES, Journal of the Brazilian Chemical Society, 27
AC

(2016) 745-752.
[11] M. Di Carro, F. Ardini, E. Magi, Multivariate optimization of headspace solid-phase
microextraction followed by gas chromatography-mass spectrometry for the determination of
methylpyrazines in cocoa liquors, Microchemical Journal, 121 (2015) 172-177.
[12] A.M.H. Benavides, P.J. Martin-Alvarez, L. Vazquez, G. Reglero, F.J. Senorans, E. Ibanez,
Optimization of Countercurrent Supercritical Fluid Extraction of Minor Components from Olive Oil,
Current Analytical Chemistry, 10 (2014) 78-85.
[13] B. Hashemi, M. Shamsipur, A. Barati, Dispersive Liquid-Liquid Microextraction Based on
Solidification of Floating Organic Drop with Central Composite Design for the Determination of
Nitrophenols Using High-Performance Liquid Chromatography, Journal of the Brazilian Chemical
Society, 26 (2015) 2046-2053.
[14] N.R. Wingert, M.A.G. Nunes, A.T. Barden, P. Gomes, E.I. Muller, E.M.M. Flores, M. Steppe, Ultra-
performance LC-ESI/Q-TOF MS for the Rapid Analysis of Rivaroxaban: Method Validation Using
Experimental Design for Robustness Evaluation, Current Analytical Chemistry, 11 (2015) 124-129.
[15] M. Fattahi, R. Rahimi, Optimization of Extraction Parameters of Phenolic Antioxidants from
Leaves of Capparis spinosa Using Response Surface Methodology, Food Analytical Methods, 9 (2016)
2321-2334.

14
 ACCEPTED MANUSCRIPT

[16] O. Sayar, F. Aboufazeli, H. Zhad, O. Sadeghi, M. Karimi, E. Najafi, Optimization of Solid-phase

Extraction by Experimental Design Methodology for Determination of Lead Ions Using Graphene
Modified Nano-sheets as a Novel Sorbent, Current Analytical Chemistry, 10 (2014) 512-521.
[17] T. Huffer, X.L. Osorio, M.A. Jochmann, B. Schilling, T.C. Schmidt, Multi-walled carbon nanotubes
as sorptive material for solventless in-tube microextraction (ITEX2)-a factorial design study, Analytical
and Bioanalytical Chemistry, 405 (2013) 8387-8395.

T
[18] T. Rakic, I. Kasagic-Vujanovic, M. Jovanovic, B. Jancic-Stojanovic, D. Ivanovic, Comparison of full
factorial design, central composite design, and Box-Behnken design in chromatographic method

IP
development for the determination of fluconazole and its impurities, Analytical Letters, 47 (2014)
1334-1347.

R
[19] A.A. D'Archivio, M.A. Maggi, C. Marinelli, F. Ruggieri, F. Stecca, Optimisation of temperature-
programmed gas chromatographic separation of organochloride pesticides by response surface

SC
methodology, Journal of Chromatography A, 1423 (2015) 149-157.
[20] S.O. Souza, D.V.L. Avila, A.S.C. Monteiro, C.A.B. Garcia, J.P.H. Alves, T.A. Maranhao, R.G.O.
Araujo, Simultaneous Determination of Thorium and Uranium in Mineral Fertilizers by Inductively

NU
Coupled Plasma Optical Emission Spectrometry, Journal of the Brazilian Chemical Society, 27 (2016)
799-806.
[21] J.M. Bosque-Sendra, M. Nechar, L.C. Rodriguez, Decision protocol for checking robustness with
previous outlier detection in the validation of analytical methods, Fresenius Journal of Analytical
MA
Chemistry, 365 (1999) 480-488.
[22] L. Cuadros-Rodriguez, R. Romero, J.M. Bosque-Sendra, The role of the robustness/ruggedness
and inertia studies in research and development of analytical processes, Critical Reviews in Analytical
Chemistry, 35 (2005) 57-69.
D

[23] ICH Harmonised tripartite guideline validation of analytical procedures: text and methodology
Q2(R1), 2005.
TE

[24] B.M.G.V. D.L. Massart, L.M.C. Buydens, S. de Jong, P.J. Lewi, J. Smeyers-Verbeke, Handbook of
Chemometrics and Qualimetrics: Part A, Third ed., Elsevier Science B. V., 2003.
[25] E. Aguilera, R. Lucena, S. Cardenas, M. Valcarcel, E. Trullols, I. Ruisanchez, Robustness in
P

qualitative analysis: a practical approach, Trac-Trends in Analytical Chemistry, 25 (2006) 621-627.

[26] M. Thompson, S.L.R. Ellison, R. Wood, Harmonized guidelines for single-laboratory validation of
CE

methods of analysis - (IUPAC technical report), Pure and Applied Chemistry, 74 (2002) 835-855.
[27] G. Derringer, R. Suich, Simultaneous-optimization of several response variables, Journal of
Quality Technology, 12 (1980) 214-219.
AC

[28] S.L.C. Ferreira, Introdução às técnicas de planejamento de experimentos, 1a. ed., Editora vento
Leste, Salvador, Brasil, 2015.
[29] M. Pouyan, G. Bagherian, N. Goudarzi, Determination of ultra-trace palladium (II) in water, soil,
and food samples by dispersive liquid-liquid microextraction-atomic absorption spectrometry using
2-mercaptobenzimidazole as a complexing agent, Microchemical Journal, 127 (2016) 46-51.
[30] E.V. Alonso, M.D.M. Guerrero, P. Cueto, J.B. Benitez, J.M.C. Pavon, A.G. de Torres, Development
of an on-line solid phase extraction method based on new functionalized magnetic nanoparticles.
Use in the determination of mercury in biological and sea-water samples, Talanta, 153 (2016) 228-
239.
[31] A.M.S. Lima, D. Araujo, S.L.C. Ferreira, Use of tartaric acid-citric acid-sucrose as chemical
modifier for the determination of lead in several matrices employing ET AAS, Microchemical Journal,
126 (2016) 368-372.
[32] S.L.C. Ferreira, H.C. dos Santos, M.S. Fernandes, M.S. de Carvalho, Application of Doehlert matrix
and factorial designs in optimization of experimental variables associated with preconcentration and
determination of molybdenum in sea-water by inductively coupled plasma optical emission
spectrometry, Journal of Analytical Atomic Spectrometry, 17 (2002) 115-120.
[33] S. Furlanetto, S. Orlandini, B. Pasquini, C. Caprini, P. Mura, S. Pinzauti, Fast analysis of
glibenclamide and its impurities: quality by design framework in capillary electrophoresis method
development, Analytical and Bioanalytical Chemistry, 407 (2015) 7637-7646.

15
 ACCEPTED MANUSCRIPT

[34] H.I. Mokhtar, R.A. Abdel-Salam, G.M. Haddad, Development of a fast high performance liquid
chromatographic screening system for eight antidiabetic drugs by an improved methodology of in-
silico robustness simulation, Journal of Chromatography A, 1399 (2015) 32-44.
[35] L. Moreno-Vilet, J. Bonnin-Paris, S. Bostyn, M.A. Ruiz-Cabrera, M. Moscosa-Santillan, Assessment
of sugars separation from a model carbohydrates solution by nanofiltration using a design of
experiments (DoE) methodology, Separation and Purification Technology, 131 (2014) 84-93.

T
[36] A.P. Christ, M.S. Machado, P. Rosa, C.F. Codevilla, C.M.B. Rolim, A.I.H. Adams, Development and
validation of a stability-indication LC-UV method for determination of daptomycin injectable form

IP
and kinetic study in alkaline medium, Analytical Methods, 6 (2014) 1242-1247.
[37] M.Z. Milenkovic, V.A. Marinkovic, P.S. Sibinovic, R.M. Palic, D.M. Milenovic, An HPLC method for

R
the determination of digoxin in dissolution samples, Journal of the Serbian Chemical Society, 75
(2010) 1583-1594.

SC
[38] A.S. Souza, G.C. Brandao, W.N.L. dos Santos, V.A. Lemos, E.M. Ganzarolli, R.E. Bruns, S.L.C.
Ferreira, Automatic on-line pre-concentration system using a knotted reactor for the FAAS
determination of lead in drinking water, Journal of Hazardous Materials, 141 (2007) 540-545.

NU
[39] K. Jankowski, J. Giersz, M. Paprocka, Improved determination of iodine by sequential
(photo)chemical vapor generation and pneumatic nebulization in the programmable temperature
spray chamber and inductively coupled plasma optical emission spectrometry, Microchemical
Journal, 113 (2014) 17-22.
MA
[40] R. Ragonese, M. Mulholland, J. Kalman, Full and fractionated experimental designs for
robustness testing in the high-performance liquid chromatographic analysis of codeine phosphate,
pseudoephedrine hydrochloride and chlorpheniramine maleate in a pharmaceutical preparation,
Journal of Chromatography A, 870 (2000) 45-51.
D

[41] H.K. Chepkwony, A. Vermaelen, E. Roets, J. Hoogmartens, Development and validation of an

reversed-phase liquid chromatographic method for analysis of spiramycin and related substances,
TE

Chromatographia, 54 (2001) 51-56.

[42] F.N. Kamau, H.K. Chepkwony, J.K. Ngugi, D. Debremaeker, E. Roets, J. Hoogmartens, Isocratic
liquid chromatographic method for the analysis of azithromycin and its structurally related,
P

substances in bulk samples, Journal of Chromatographic Science, 40 (2002) 529-533.

[43] B.M. DeBorba, J.S. Rohrer, L. Bhattacharyya, Development and validation of an assay for citric
CE

acid/citrate and phosphate in pharmaceutical dosage forms using ion chromatography with
suppressed conductivity detection, Journal of Pharmaceutical and Biomedical Analysis, 36 (2004)
517-524.
AC

[44] G. Savic, M. Zecevic, B. Jocic, L. Zivanovic, Validation of an HPLC method for the determination of
valdecoxib and its degradation product: a mixture of alpha- and beta-n-lactosyl sulfonamide
anomers, Chromatographia, 66 (2007) 29-35.
[45] W.N.L. dos Santos, J.V.S. Santos, L.O.B. Silva, A.S. Araujo, V.A. Lemos, M. Miro, S.L.C. Ferreira,
On-line simultaneous pre-concentration procedure for the determination of cadmium and lead in
drinking water employing sequential multi-element flame atomic absorption spectrometry,
International Journal of Environmental Analytical Chemistry, 91 (2011) 1425-1435.
[46] W.N.L. dos Santos, A.F. Silva Junior, L.O.B. Silva, B.R.S. Santos, D.L.F. Da Silva, Multivariate
optimization of a digestion procedure for bismuth determination in urine using continuous flow
hydride generation and atomic fluorescence spectrometry, Microchemical Journal, 130 (2017) 147-
152.
[47] Y. Vander Heyden, A. Nijhuis, J. Smeyers-Verbeke, B.G.M. Vandeginste, D.L. Massart, Guidance
for robustness/ruggedness tests in method validation, Journal of Pharmaceutical and Biomedical
Analysis, 24 (2001) 723-753.
[48] R.M. Bernardi, F.B. D'Avila, V. Todeschini, P.E. Froehlich, A.M. Bergold, A comparative study on
the analytical performance of a charged aerosol detector and an ultraviolet detector for the RP-LC
analysis of dabigatran etexilate in capsules, Analytical Methods, 5 (2013) 4777-4784.

16
 ACCEPTED MANUSCRIPT

[49] C.S. Paim, F. Fuhr, V. Todeschini, M. Steppe, E.E.S. Schapoval, Simultaneous analysis of
gemifloxacin mesylate and its main synthetic impurity by an optimized capillary zone electrophoretic
method, Analytical Methods, 6 (2014) 1657-1665.
[50] M. Elkarbane, M.A. Al-Kamarany, H. Bouchafra, M. Azougagh, Y. Cherrah, P. Hubert, Y. Vander
Heyden, A. Bouklouze, Total Error-Based Validation Including the Experimental Design-Based
Robustness Evaluation of a Stability-Indicating Method for the Simultaneous Quantification of

T
Hydrochlorothiazide and Valsartan in Tablet Formulations, Acta Chromatographica, 27 (2015) 195-
214.

IP
[51] M. Maskovic, B. Jancic-Stojanovic, A. Malenovic, D. Ivanovic, M. Medenica, Assessment of Liquid
Chromatographic Method Robustness by Use of Plackett-Burman Design, Acta Chromatographica, 22

R
(2010) 281-296.
[52] S. Farsimadan, N. Goudarzi, M.A. Chamjangali, G. Bagherian, Optimization of ultrasound-assisted

SC
dispersive liquid-liquid microextraction based on solidification of floating organic droplets by
experimental design methodologies for determination of three anti-anxiety drugs in human serum
and urine samples by high performance liquid chromatography, Microchemical Journal, 128 (2016)

NU
47-54.
[53] I. Gaubeur, M.A. Aguirre, N. Kovachev, M. Hidalgo, A. Canals, Dispersive liquid-liquid
microextraction combined with laser-induced breakdown spectrometry and inductively coupled
plasma optical emission spectrometry to elemental analysis, Microchemical Journal, 121 (2015) 219-
MA
226.
[54] D.C. Montgomery, Design and Analysis of Experiments, Eighth ed., John Wiley & Sons, Inc.,
2013.
[55] C. Santos, F. Alava-Moreno, I. Lavilla, C. Bendicho, Total As in seafood as determined by
D

transverse heated electrothermal atomic absorption spectrometry-longitudinal Zeeman background

correction: An evaluation of automated ultrasonic slurry sampling, ultrasound-assisted extraction
TE

and microwave-assisted digestion methods, Journal of Analytical Atomic Spectrometry, 15 (2000)

987-994.
[56] S. Furlanetto, S. Orlandini, E. La Porta, S. Coran, S. Pinzauti, Optimization and validation of a CZE
P

method for rufloxacin hydrochloride determination in coated tablets, Journal of Pharmaceutical and
Biomedical Analysis, 28 (2002) 1161-1171.
CE

[57] L.F. Oliveira, S. Braga, P.R. Filgueiras, F. Augusto, R.J. Poppi, Assessment of robustness on
analysis using headspace solid-phase microextraction and comprehensive two-dimensional gas
chromatography through experimental designs, Talanta, 129 (2014) 303-308.
AC

[58] D. Karampelas, A. Economou, A. Calokerinos, A novel hybrid flow-injection/sequential-injection

methodology for the rapid evaluation of the total antioxidant capacity of wines using inhibition of the
alkaline luminol-potassium permanganate chemiluminescent reaction, Microchemical Journal, 118
(2015) 223-230.
[59] M.C. Nemitz, F.K.J. Yatsu, J. Bidone, L.S. Koester, V.L. Bassani, C.V. Garcia, A.S.L. Mendez, G.L.
von Poser, H.F. Teixeira, A versatile, stability-indicating and high-throughput ultra-fast liquid
chromatography method for the determination of isoflavone aglycones in soybeans, topical
formulations, and permeation assays, Talanta, 134 (2015) 183-193.
[60] M.J. Cozar-Bernal, A.M. Rabasco, M.L. Gonzalez-Rodriguez, Development and validation of a high
performance chromatographic method for determining sumatriptan in niosomes, Journal of
Pharmaceutical and Biomedical Analysis, 72 (2013) 251-260.
[61] T. Mohammadi, A. Moheb, M. Sadrzadeh, A. Razmi, Separation of copper ions by electrodialysis
using Taguchi experimental design, Desalination, 169 (2004) 21-31.
[62] V.A.L. Wortel, W.G. Hansen, S.C.C. Wiedemann, Optimising multivariate calibration by
robustness criteria, Journal of near Infrared Spectroscopy, 9 (2001) 141-151.
[63] S. Bruns R.E. , I.S., de Barros Neto B. , Statistical Design - Chemometrics, Elsevier B.V.,
Amsterdam, 2006.

17
 ACCEPTED MANUSCRIPT

[64] A. Tumpa, T. Miladinovic, T. Rakic, A. Stajic, B. Jancic-Stojanovic, Quality by Design

Determination of Diclofenac Potassium and its Impurities by High-Performance Liquid
Chromatography, Analytical Letters, 49 (2016) 445-457.
[65] M.L. Oca, L. Rubio, M.C. Ortiz, L.A. Sarabia, I. Garcia, Robustness testing in the determination of
seven drugs in animal muscle by liquid chromatography-tandem mass spectrometry, Chemometrics
and Intelligent Laboratory Systems, 151 (2016) 172-180.

T
[66] I. Kasagic-Vujanovic, B. Jancic-Stojanovic, T. Rakic, D. Ivanovic, Design of Experiments in
Optimization and Validation of a Hydrophilic Interaction Liquid Chromatography Method for

IP
Determination of Amlodipine Besylate and Bisoprolol Fumarate, Journal of Liquid Chromatography &
Related Technologies, 38 (2015) 919-928.

R
[67] M. Ferreira, L.L. Chaves, S.A.C. Lima, S. Reis, Optimization of nanostructured lipid carriers loaded
with methotrexate: A tool for inflammatory and cancer therapy, International Journal of

SC
Pharmaceutics, 492 (2015) 65-72.
[68] S.S. Panda, V. Bera, S. Beg, S.K. Sahu, Ultrafast Liquid Chromatographic Method Development
and its Validation for Quantification of Telaprevir in Pharmaceutical Dosage Form by Using Quality by

NU
Design Approach, Journal of Chromatographic Science, 53 (2015) 1193-1202.
[69] W. Ahmad, S.M.A. Zaidi, M. Mujeeb, S.H. Ansari, S. Ahmad, HPLC and HPTLC Methods by Design
for Quantitative Characterization and in vitro Anti-oxidant Activity of Polyherbal Formulation
Containing Rheum emodi, Journal of Chromatographic Science, 52 (2014) 911-918.
MA
[70] M.S. Hasnain, S. Rao, M.K. Singh, N. Vig, A. Gupta, A. Ansari, P. Sen, P. Joshi, S.A. Ansari,
Development and validation of LC-MS/MS method for the quantitation of lenalidomide in human
plasma using Box-Behnken experimental design, Analyst, 138 (2013) 1581-1588.
[71] T.B. Solanki, P.A. Shah, K.G. Patel, Central composite design for validation of hptlc method for
D

simultaneous estimation of olmesartan medoxomil, amlodipine besylate and hydrochlorothiazide in

tablets, Indian Journal of Pharmaceutical Sciences, 76 (2014) 179-187.
TE

[72] J. Acevska, G. Stefkov, N. Nakov, R. Petkovska, L. Ugrinova, S. Kulevanova, A. Dimitrovska,

Chemometric approach for the development, optimization and validation of hilic methods used for
the determination of alkaloids from poppy straw, Macedonian Journal of Chemistry and Chemical
P

Engineering, 33 (2014) 73-83.

[73] A. Porel, Y. Sanyal, A. Kundu, Simultaneous HPLC determination of 22 components of essential
CE

oils; method robustness with experimental design, Indian Journal of Pharmaceutical Sciences, 76
(2014) 19-30.
[74] A. Oliva, J.B. Farina, M. Llabres, Development and validation of an UPLC method for
AC

determination of content uniformity in low-dose solid drugs products using the design space
approach, Talanta, 115 (2013) 490-499.
[75] R. Petkovska, C. Cornett, A. Dimitrovska, Experimental design approach for the development and
validation of an enantiospecific RP-HPLC method for simultaneous determination of clopidogrel and
related compounds, Macedonian Journal of Chemistry and Chemical Engineering, 27 (2008) 53-64.

18
 ACCEPTED MANUSCRIPT

T
IP
R
SC
NU
MA
D
PTE
CE
AC

19
 ACCEPTED MANUSCRIPT

T
IP
R
SC
NU
MA
D
PTE
CE
AC

20
 ACCEPTED MANUSCRIPT

Table 1. Evaluation of the robustness for pH and buffer solution concentration

Experiment pH Buffer solution concentration Absorbance

(M)
1 8.30 (-1) 0.020 (-1) 0.0314

T
IP
2 9.30 (1) 0.020 (-1) 0.0318
3 8.30 (-1) 0.028 (1) 0.0323

R
4 9.30 (1) 0.028 (1) 0.0353

SC
CP 8.80 (0) 0.024 (0) 0.0297
CP 8.80 (0) 0.024 (0) 0.0340

NU
CP 8.80 (0) 0.024 (0) 0.0335
MA
D
P TE
CE
AC

21
 ACCEPTED MANUSCRIPT

Table 2. Determination of statistical parameters of the robustness study

Parameter pH BC pH x BC
Effect = (0.0318 + 0.0353 - = (0.0323 + 0.0353- = (0.0314 + 0.0353 -
0.0314 - 0.0323)/2 0.0314- 0.0318)/2 0.0318 - 0.0323)/2
= 0.0017 = 0.0022 = 0.0013

T
Estimated Triplicates of the central point: 0.0297; 0.0340; 0.0335.

IP
variance of
the Average = 0.0324

R
experimental
error (S2) S2 = {(0.0297 - 0.0324)2 + (0.0340 - 0.0324)2 + (0.0335 - 0.0324)2}/(3-1)

SC
and S2 = 5.53.10-6

NU
The standard SEffect = (4S2/N)1/2 = ((4x5.53.10-6)/4)1/2 = 0.0024
deviation of
the effects
MA
(SEffect)
Confidence 0.0017 ± 0.0024x4.3027 0.0022 ± 0.0013 ± 0.0024x4.3027
interval at 0.0017 ± 0.0103 0.0024x4.3027 0.0013 ± 0.0103
95% level - 0.0086 to 0.0120* 0.0022 ± 0.0103 -0.0090 to 0.0116*
D

-0.0081 to 0.0125*
Standardized
TE

effects (SE) SE = Effect/ SEffect

P

SEpH = 0.0017/0.0024 SEBC = 0.0022/0.0024 SEpH = 0.0013/0.0024

CE

SEpH = 0.71** SEBC = 0.92** SEpH = 0.54**

*- This interval includes the value zero. So, the effect of this factor is not significant.
AC

**- This value is lower than 4.3027, which is the tStudent value for 2 degree of freedom and
confidence limit of 95%.

22
 ACCEPTED MANUSCRIPT

Table 3. Evaluation of the robustness of factors during determination of lead by ETAAS

Experiment [HNO3] Atomization Pyrolysis Absorbance

(M) temperature temperature
(oC) (oC)

T
1 1.00 (-1) 1746 (-1) 815 (-1) 0.05905

IP
2 1.60 (1) 1746 (-1) 815 (-1) 0.05789

R
3 1.00 (-1) 2134 (1) 815 (-1) 0.05896

SC
4 1.60 (1) 2134 (1) 815 (-1) 0.05577

5 1.00 (-1) 1746 (-1) 995 (1) 0.05909

NU
6 1.60 (1) 1746 (-1) 995 (1) 0.05478

7 1.00 (-1) 2134 (1) 995 (1) 0.05879

MA
8 1.60 (1) 2134 (1) 995 (1) 0.05752

CP 1.30 (0) 1940 (0) 905 (0) 0.05802

CP 1.30 (0) 1940 (0) 905 (0) 0.05709

D

CP 1.30 (0) 1940 (0) 905 (0) 0.05401

P TE
CE
AC

23
 ACCEPTED MANUSCRIPT

Table 4. Calculations of the effects of the two-level full factorial design

Effect Effects as confidence Standardized

interval effects
[HNO3] -0.002483 -0.002483 ± 0.006368 - 1.67

T
Atomization t. 0.000057 0.000057 ± 0.006368 0.04

IP
Pyrolysis t. -0.000373 -0.000373 ± 0.006368 - 0.25
[HNO3] x Atom t. 0.000252 0.000252 ± 0.006368 0.17

R
[HNO3] x Pyro t. -0.000307 -0.000307 ± 0.006368 - 0.21

SC
Atom t x Pyro t. 0.001163 0.001163 ± 0.006368 0.79
The estimated variance of the experimental error = 4.40.10-6

The standard deviation of the effects = 0.00148

NU
MA
D
P TE
CE
AC

24
 ACCEPTED MANUSCRIPT

Table 5. The recommended Plackett - Burman designs [47]

Number of factors Selected design Number of dummy Number of

factors experiments (N)

T
3-4 P-B design 7 factors 4-3 8

IP
5-8 P-B design 11 factors 6-3 12

R
9 - 12 P-B design 15 factors 6-3 16

SC
13 - 16 P-B design 19 factors 6-3 20

NU
17 - 20 P-B design 23 factors 6-3 24
MA
P-B - Plackett-Burman
D
P TE
CE
AC

25
 ACCEPTED MANUSCRIPT

Table 6. Plackett-Burman design for 7 factors (N=8) [50]

Exp. A B C D E F G

1 + + + - + - -

T
2 - + + + - + -

IP
3 - - + + + - +

R
SC
4 + - - + + + -
5 - + - - + + +
6 + - +
NU - - + +
MA
7 + + - + - - +
8 - - - - - - -
D
P TE
CE
AC

26
 ACCEPTED MANUSCRIPT

Table 7. Plackett-Burman design for 11 factors (N=12) [47]

Exp. A B C D E F G H I J K

T
1 + + - + + + - - - + -

IP
2 - + + - + + + - - - +

R
SC
3 + - + + - + + + - - -
4 - + - + + - + + + - -
5 - - + - +
NU+ - + + + -
MA
6 - - - + - + + - + + +
7 + - - - + - + + - + +
D

8 + + - - - + - + + - +
TE

9 + + + - - - + - + + -
P

10 - + + + - - - + - + +
CE

11 + - + + + - - - + - +
AC

12 - - - - - - - - - - -

27
 ACCEPTED MANUSCRIPT

Highlights

Several robustness concepts are cited and discussed.

Advantages and limitations of the main tools used for robustness tests are discussed.

T
Examples of robustness tests in analytical methods are presented.

IP
Seventy references are cited.

R
SC
NU
MA
D
P TE
CE
AC

28