Front. Chem. Sci. Eng.

https://doi.org/10.1007/s11705-021-2071-9

RESEARCH ARTICLE

Synergistic optimization framework for the process synthesis

and design of biorefineries

Nikolaus I. Vollmer1, Resul Al2, Krist V. Gernaey1, Gürkan Sin (✉)1

1 Process and Systems Engineering (PROSYS) Research Center, Department of Chemical and Biochemical Engineering,
Technical University of Denmark, 2800 Kgs. Lyngby, Denmark
2 Novo Nordisk A/S, 2880 Bagsværd, Denmark

© Higher Education Press 2021

Abstract The conceptual process design of novel 1 Introduction

bioprocesses in biorefinery setups is an important task,
which remains yet challenging due to several limitations. The global necessity of novel process solutions to meet the
We propose a novel framework incorporating super- demands of a growing society and find adequate solutions,
structure optimization and simulation-based optimization matching the increasing need for sustainable process
synergistically. In this context, several approaches for solutions, is high [1]. In general, biotechnological process
superstructure optimization based on different surrogate solutions are deemed to be conceptually more sustainable
models can be deployed. By means of a case study, the [2]. Recent developments in synthetic biology allow for
framework is introduced and validated, and the different producing a vast palette of biofuels, chemicals, foods, and
superstructure optimization approaches are benchmarked. even pharmaceutical ingredients in cell factories. The
The results indicate that even though surrogate-based global biofoundry initiative aims at accelerating the
optimization approaches alleviate the underlying computa- acquisition of knowledge, the integration of data, and the
tional issues, there remains a potential issue regarding their development of new cell factories [3,4].
validation. The development of appropriate surrogate On the other side, the implementation of biorefinery
models, comprising the selection of surrogate type, concepts that incorporate these chemicals’ sustainable
sampling type, and size for training and cross-validation production from sustainable feedstocks, as, e.g., lignocel-
sets, are essential factors. Regarding this aspect, satisfac- lulosic biomass or other residues or dedicated crop plants is
tory validation metrics do not ensure a successful outcome dramatically low. Up to the current date, a total number of
from its embedded use in an optimization problem. less than 100 active lignocellulosic biorefineries are
Furthermore, the framework’s synergistic effects by operated around the world [5]. This is mainly due to
sequentially performing superstructure optimization to these biorefinery concepts’ critical economic robustness,
determine candidate process topologies and simulation- as the operative margins for most products are narrow and
based optimization to consolidate the process design under chemical processes are competitive [6]. Despite several
uncertainty offer an alternative and promising approach. ideas of designing these biorefineries and improving their
These findings invite for a critical assessment of surrogate- economic robustness by mass and heat integration or
based optimization approaches and point out the necessity producing several products simultaneously in a so-called
of benchmarking to ensure consistency and quality of multi-product biorefinery, a conceptual design strategy that
optimized solutions. significantly promotes the implementation of these bior-
efineries is still an active area of research [7–10]. Current
Keywords biotechnology, surrogate modelling, super- research directions span from the integration and expan-
structure optimization, simulation-based optimization, pro- sion of established approaches [11,12], over the use of
cess design novel approaches and models [13,14], up to the inclusion
of further economic, environmental, and sustainability
factors in particular [15–19], as well as the accommodation
of specificities for fermentation-based processes [20]. The
Received December 17, 2020; accepted May 10, 2021 vast majority of these studies primarily follow three
E-mail: [email protected] approaches for conceptual process design that are briefly
2 Front. Chem. Sci. Eng.

introduced in the following paragraphs. synthesis and the potential of SBO for detailed design
Classically, process design for biorefineries is performed optimization. The framework itself capitalizes biotechno-
by so-called hierarchical decomposition and involves logical knowledge to guide decisions in a bottom-up
domain knowledge from different expert areas to yield a strategy for both SSO and SBO, which finally yields a
consolidated solution [21]. However, this methodology has consolidated process design.
several shortcomings as it usually involves several The remainder of the paper is structured as follows: The
iterations, which results in a long idea-to-process time, used surrogate models for the superstructure are introduced
and does not necessarily yield an optimal solution for the in section 2.1 and the methods of performing SSO in
process design, especially considering that biotechnologi- section 2.2, and the SBO in section 2.3. The detailed
cal processes suffer from issues with the scale-up from components of the proposed framework and the applied
laboratory scale to production scale [3,22]. workflow are introduced in section 2.5. For the application
A computational and more conceptual approach to of the framework, a case study is introduced in section 3.1.
process design is superstructure optimization (SSO) [23]. The corresponding high-fidelity models for the specific
Out of all possible process options provided in a super- case study are introduced in appendix A.1 (cf. ESM),
structure, the optimal configuration is chosen through including the corresponding methodology for uncertainty
mathematical optimization. In comparison to the previous and sensitivity analysis in appendix A.2 (cf. ESM). The
alternative, the resulting process design is globally evaluation of the framework’s SSO step is presented in
optimal. However, the methodology is initially limited section 3.2 and the SBO step in section 3.3. Conclusions
by the number of alternatives that are included in the and an outlook on future research work are given in
superstructure. Furthermore, the resulting optimization section 4.
problem can become very large even though a considerable
number of possible solutions can be infeasible a priori.
Also, the use of high-fidelity models is limited in this 2 Theoretical background
approach as this can be computationally challenging to
solve. Lastly, the incorporation of uncertainty increases the 2.1 Surrogate models
complexity of formulating the optimization problem
through stochastic or robust optimization [24]. SSO has Surrogate models describe various types of models, which
been successfully performed for several chemical produc- mimic the behavior of original first-principle models. The
tion processes; however, the design of biotechnological primary reason for developing and using a surrogate
processes comprises several new challenges and uncer- model, described by McBride and Sundmacher, is the
tainties which are difficult to address [25]. reduction of computational costs for model evaluations,
A second, more recent approach to process design is which is an inherent part of the SSO, as explained in
simulation-based optimization (SBO). The methodology section 2.2. This comes at the cost of a certain error in the
builds upon the evaluation of process simulations where surrogate model’s predictions due to imperfect resem-
high-fidelity models are used [26]. Furthermore, uncer- blance. The applied technique for creating a surrogate
tainties can be easily included in a Monte Carlo-based model highly depends on the computational constraints
approach [27]. Several proposed frameworks utilize regarding the original and the surrogate model, as well as
machine learning surrogate models as, e.g., stochastic the type of application of the surrogate model [29]. The
kriging (SK), coupled with a Bayesian optimization simplest model types are linear, piecewise linear, or
approach to iteratively improve the objective function polynomial functions, which are fitted to the original
results, leading to an optimal process design [27,28]. data [30,31]. However, the majority of the used surrogate
Despite this approach being able to incorporate complex models are based on machine learning techniques, among
physiological models of cell factories and processes, it is others radial basis functions, Gaussian process regressors,
heavily limited by available computational power. The and at the higher end artificial neural networks (ANNs) and
number of simulations can get very high for several even deep neural networks [29,32]. The following
alternative process configurations, which constrains this subsections describe first the development procedure for
approach severely to a small design space. these model types and subsequently four different
After elucidating the challenges within the process surrogate model types which are used in this framework,
design of novel bioprocesses in biorefinery setups, it one based on algebraic equations, one piecewise linear
becomes evident that novel solutions are required, which model, and two machine learning models.
incorporate knowledge from different fields and approach The development procedure for surrogate models —
the problem in an interdisciplinary manner. Hence, we independent of the model type is similar and commonly
propose a novel synergistic optimization framework for the starts with sampling the model input space by, e.g., Latin
conceptual process design of biorefineries, based on a hypercube (LHS) or Sobol sampling. To ensure a space-
hybrid approach integrating surrogate-based SSO with filling sampling for a sufficient interpolative quality of the
SBO. It harnesses both the power of the SSO for process model, the choice of an adequate sampling technique is
 Nikolaus I. Vollmer et al. Synergistic optimization framework 3

crucial [33]. Subsequently, the surrogate model is fitted to a point p. Furthermore, an important property of an
part of the data, dependent on the chosen validation n-simplex
is it being an affine and hence convex space,
strategy. which allows any point x* inside the simplex to be
A standard method, which is also applied in this described as a linear combination of the vertices with the
framework, is called k -fold cross-validation: the sampled coefficients αi < 1:
dataset is divided into k equally sized subsets. In a routine, Xnþ1 Xnþ1
each subset is once used to validate the surrogate model, x* ¼ i¼1
α i $vi ¼ α $px 8 vi , pxi 2
:
i¼1 i i
(1)
and the remaining k – 1 subsets are used for the fitting of
the model. The total validation error is calculated by the Furthermore, the functional relationship of the surrogate
average error of the k validation folds [34]. model f is defined as:
 n
ℝ ↕ ↓ℝr
2.1.1 Automated learning of algebraic models f : : (2)
x↦z
The automated learning for algebraic models (ALAMO) The set of real numbers is denoted as ℝ. Given this, the
toolbox was developed to create algebraic models for conclusion for the surrogate model is that any point z* can
applications like SSO [35,36]. As described in Wilson’s also be described by a linear combination of elements pz of
and Sahinidis’ work, the ALAMO toolbox first fits simple, the point p by Eq. (1), which is equivalent to the concept of
algebraic models, consisting of several nonlinear terms and regression. In conclusion, the triangulation of the input
their linear combinations to the points in the input space. space T X with the functional relationship to the output
Then, by employing derivative-free optimization, the best space Z ¼ f ðT X Þ represents the surrogate model based on
suitable combination of terms is determined through error the described regression concept.
maximization sampling. Further constraints on the model Reconsidering the one-dimensional case, a 1-simplex,
outputs can be imposed, and the possibility to perform also called edge, is nothing else than a line segment; the
adaptive sampling to increase the number of points in the one 1-simplicial complex corresponds to the set of
input space at specific locations [35]. Especially for composed line segments, where each point on any line
process design purposes, the ALAMO toolbox has been segment can be described as a linear combination of the
applied widely as a surrogate modeling technique with two vertices or endpoints of the line segment it lies on.
promising results [22,37,38]. Lastly, the functional relationship describes the functional
value of the point of the line segment by a linear
2.1.2 Delaunay triangulation regression (DTR) combination of the vertices’ functional values.
A prominent type of triangulation is the so-called
The second surrogate model is based on the concept of Delaunay triangulation, which imposes the criterion that
regression by a piecewise linear functional relationship. In the circumcircle or its higher-dimensional equivalent, its
the case of a one-dimensional input space, the piecewise n-hypersphere of the vertices of the simplex cannot contain
linear functional relationship is a set of composed line another vertex [39]. There exist published algorithms to
segments. For a general definition of this piecewise linear create a DTR for a given set of points [40,41].
functional relationship, be P
ℝnr a set of points in the
n  r -dimensional space and a subset of real numbers. All 2.1.3 Gaussian process regression (GPR)
elements p 2 P are considered vectors in the Euclidean
vector space over ℝnr and consist of two elements The third surrogate model is based on the concept of
p ¼ fpx ,pz g. All elements px and pz are respectively regression by a stochastic process — the eponymous GPR
elements of the sets Px
P and Pz
P, both subsets of P. and particular kernel functions to determine parameters of
Furthermore, be X
ℝn the input space and Z
ℝr the the following functional relationship:
output space of our functional relationship with all  n
elements x 2 X and z 2 Z and P
X  Z. ℝ ↕ ↓ℝ
f : , (3)
Defining the input space X as n -manifold with x↦z
boundary, the triangulation T X of X is a homogeneous
with the expression, as explained by Al et al.:
simplicial k -complex Σ, being homeomorphic to X and
k ¼ n, referring to that the S consists only of n -simplices. z ¼ GP ðxÞ þ
2GP $F ðx,ωÞ, GP ðxÞ ¼ $βðxÞ, (4)
An n -simplex
is the special form of an n -polytope and
consists of exactly n þ 1 vertices v, so fv1 ,:::,vnþ1 g 
where, GP ðxÞ denotes the mean value of the Gaussian
2 Σ. For illustrative purposes, for n ¼ 2 the 2-simplex is a process and
2GP its variance. For the mean value GP ðxÞ, 
triangle. Hence the n -simplex is the equivalent to a triangle are estimated parameters from the input data, as well as the
in any dimension n. At this point, we define the set of variance
2GP and βðxÞ relate to a group of basis functions.
vertices V and every vertex v 2 V as the element px of the Furthermore, F ðx,ωÞ describes a zero mean unit variance
4 Front. Chem. Sci. Eng.

stochastic process. The correlation or kernel functions ω tions is a state-task-network, in which each unit operation
are used to correlate any point in the input space x* with model forms a task in the network, the flows are described
existing points x [42]. There are various available kernel as states, and both are connected via nodes that correspond
functions; hence, the reader is referred to the book by to mixers or splitters. This composition highly resembles
Rasmussen [43]. A couple of remarks on GPR as a an actual process flowsheet, with the main difference being
surrogate model shall be made: it is very well capable of that the nodes represent binary decisions on whether a
displaying highly nonfunctional relationships, and the process path exists [50]. For the interested reader, a
amount of data necessary to obtain a good surrogate model description of other existing superstructure formulations
is relatively low and favorably low-dimensional [29]. GPR can be found in a recent review by Mencarelli et al. [51].
surrogate models have seen a wide range of applications in As described by Chen and Grossman, the resulting
machine learning, particularly also in different process optimization problem is a mixed-integer program (MIP) of
design tasks [44–46]. the form:
8
> min z ¼ f ðx,yÞ
2.1.4 ANN >
>
>
< s:t: hðx,yÞ ¼ 0
MIP : , (5)
The last type of surrogate model is the class of ANNs. >
> gðx,yÞ£0
>
>
ANNs are used in a plethora of applications, especially :
complex machine learning tasks as image and voice x 2 X , y 2 ½0,1
recognition, natural language processing, and artificial
intelligence. Al et al. give the following description: in with a defined objective function z ¼ f ðx,yÞ, referring to a
general, neural networks consist of at least three layers, one certain metric, e.g., product purity, a key performance
input, one hidden, and one output layer. This case is indicator or sustainability measures, and subject to equality
considered as a shallow neural network and, in particular, a and inequality constraints hðx,yÞ and gðx,yÞ, describing
multi-layer perceptron. Each layer contains a certain physical constraints, system and equipment specifications
number of nodes that relate their inputs to their outputs and their limits, as well as other process constraints, e.g.,
with a so-called transfer or activation function of a specific product purity. The continuous variables x 2 X denote
type. When several hidden layers are added to the network, process variables as states, mass and energy flows, and
it is referred to as a deep neural network [42]. Due to their design parameters, all within a specific input space X  ℝ.
flexible architecture and possibilities in terms of compos- The binary variables y denote the mentioned decisions on
ing and learning a network, neural networks are an ideal the existence of equipment or process paths. Depending on
candidate for the use as a surrogate model in process the underlying physical system, both the objective function
design tasks and have seen a widespread application in the f and the constraints can be nonlinear; hence the
area of process systems engineering [29,47–49]. optimization problem is a mixed-integer nonlinear pro-
gram (MINLP).
2.2 SSO The objective function evaluation can be theoretically
calculated based on the unit operation models and
SSO as a method for process synthesis and design is a additional equations. However, the complexity of the
computational method based on mathematical optimiza- formulation and the computational cost of evaluating high-
tion. Following Chen and Grossmann’s elaboration, SSO fidelity models commonly makes the solution of the
involves three steps, namely (1) the definition of a set of optimization problem by currently available solvers
process alternatives in a superstructure representation, (2) complicated or even intractable [49,51]. Consequently,
the formulation of the corresponding optimization pro- there have been various research efforts in surmounting
blem, and (3) solving the optimization problem with an these hurdles in SSO, e.g., linearization of the objective
adequate solver in order to obtain the optimal process function, surrogate model-assisted SSO, or decomposition
design. The primary limitations remain firstly with the algorithms [50,51]. Especially the capacity of surrogate
initial definition of the design space, implying that only model-assisted SSO alleviating the computational burden
solutions can be found that are initially considered; has been exploited in various studies [49,52,53]. This
secondly, these limitations are also fueled by the assump- allows for an elegant solution to integrate complex high-
tions taken for the models and the capabilities of the fidelity models from different platforms indirectly via their
employed solver for the optimization problem [23]. surrogates into a simple superstructure formulation,
Moreover, a capital restriction is the inability to account compared to extensive equation-based approaches as,
for uncertainty in deterministic optimization approaches e.g., generalized disjunctive programming [49,51].
such as SSO [24]. In the following section, four different superstructure
The superstructure itself can be postulated in various formulations with respectively one of the introduced
ways. One of the most common superstructure formula- surrogate models in section 2.1 are introduced. The
 Nikolaus I. Vollmer et al. Synergistic optimization framework 5

benchmark of all four approaches is part of sections 3.2.2 given functional relationship Eq. (1) for T X , we can rewrite
and 3.2.3. Eq. (2) in the following way:
X
ym,s $xd ¼ i2I
ym,s $αm,s,i $pxm,d,i
2.2.1 Surrogate-assisted MINLP SP

The first option under investigation are ALAMO surrogate 8 d 2 IX , s 2 IS , m 2 IM , (8)

models in an MINLP and no reformulation of the actual
X
optimization problem. The resulting algebraic model ym,s $zd ¼ ym,s $αm,s,i $pzm,d,i
i 2 ISP
equations from the ALAMO surrogate models can be
introduced as the function for the objective function f ðx,yÞ
and the equality constraints hðx,yÞ in the MINLP 8 d 2 IZ , s 2 IS , m 2 IM : (9)
formulation described in Eq. (5), the problem can be Furthermore, due to the convex property of the simplex,
solved with an adequate solver. As the ALAMO surrogates the following equation considering the linear combination
are fitted from flowsheet simulations, a binary variable ym coefficients αm,s,i , which are treated as variables by the
for all flowsheet options m 2 IM is introduced, as well as optimizer, must be fulfilled:
an SOS1 constraint, indicating that the optimal solution X
can only lie in one flowsheet: αm,s,i ¼ 1 8 s 2 IS , m 2 IM : (10)
i2I
X SP

m2I
ym ¼ 1: (6) Since an MILP only allows the formulation of linear
M
equations, a Big-M notation for Eqs. (8) and (9) is
introduced with the three variables , φ and ψ replacing the
2.2.2 Surrogate-assisted mixed-integer linear program three products ym,s $xd , ym,s $αm,s,i and ym,s $zd . The follow-
(MILP) ing are the four equations for the product ym,s,d $xd :

The second option for the SSO suggested here involves

developing a superstructure based on DTR surrogate
models and the reformulation of the underlying MINLP
into an MILP.
For the set of possible process design configurations M
with an index set IM ¼ f1,2,:::,M g, a certain number of
flowsheet simulations are performed for each configuration
m 2 M : A DTR surrogate model is then fitted for each
configuration m 2 M from a set of sample points Pm,x In these equations, M denotes so-called Big-M para-
over the input space X
ℝn and a set of simulation points meters; their values are set to the upper bound of the
Pm,z over the output space Z
ℝr . Every sample point pxm respective variable they represent. Similarly, also the Big-
and respectively pzm equally is a set of the type pxm ¼ M notation for the second product ym,s $αm,s,i is written:
fpxm,1 ,pxm,2 ,:::,pxm,n g and pz ¼ fpzm,1 ,pzm,2 ,:::,pzm,r g with every
pjm,d being a scalar value. Besides, two index sets IX ¼
f1,2,:::,ng
ℕ and IZ ¼ f1,2,:::,rg
ℕ for the dimension
of the input and output space are defined and an index set
IP for set Px and Pz as jPx j ¼ jPz j. The created triangulation
T X yields a set of simplices S, fulfilling the definitions
stated in section 2.1.2. Be IS ¼ f1,2,:::,jSjg
ℕ the index
set of S, then each simplex
m,s again is a specific set of
points pxm,d,i with i 2 ISP ¼ fi jpxm,d,i 2
m,s 8d 2 IX g  IP In this case, the Big-M parameter is equal to 1 and hence
that are all part of the simplex. Lastly, a binary variable ym,s left out of the equations. Lastly, also the Big-M notation for
is introduced, which relates to whether a point x is part of a the third product ym,s $zd is listed here:
simplex
2 S, in the flowsheet configuration m 2 M .
The MILP is postulated as follows: The objective
function is the minimum value of one element of the output
variable z:
min zk , (7)
with zk 2 z and k£r. Furthermore, for any input variable x
on the boundary ∂X of X and output variable z under the
6 Front. Chem. Sci. Eng.

Equations (11)–(22) now allow to express Eqs. (8) and MILP with Eq. (7). The series of NLP can now be solved
(9) without the multiplication of variables in the following sequentially for each NLP performing being solved s times
way: with the different initial points x0 2 X0 by using a suitable
X optimization algorithm.
m,s,d ¼ i2I
φm,s,d,i $pxm,d,i 8 d 2 IX , s 2 IS , m 2 IM ,
SP
2.3 SBO
(23)
X All the presented SSO approaches in section 2.2 are
ψ m,s,d ¼ i 2 ISP
φm,s,d,i $pzm,d,i 8 d 2 IZ , s 2 IS , m 2 IM : deterministic approaches to optimization, relating to the
(24) found optimum, not incorporating any kind of uncertainty.
In contrast to that, the process design of a novel chemical
To satisfy Eqs. (23) and (24), there exists exactly one or biochemical process inherently represents a significant
simplex in one flowsheet configuration in which the point x uncertainty by itself [56]. Due to the development of
is located, and all other simplices in this flowsheet and all computational power and the ever-more growing use of
other flowsheets do not contain the point. Hence, an SOS1 simulation software, making the computational tractability
constraint is added to the postulation in order to express of complex systems possible, the concept of SBO or, in
this: particular, stochastic simulation optimization has seen
X X increasing interest over the past years. This concept allows
m2I s2I
ym,s ¼ 1: (25)
M S to incorporate stochastic considerations as uncertainty into
With Eqs. (7) and (10), the MILP is now well defined an optimization formulation and solve them, given
and can be solved with a suitable optimization algorithm. sufficient computational capacities [26–28]. In contrast to
Alternatively, further constraints on other elements of the the prior presented approaches, the system to be optimized
output variables can be imposed, referring to other process does not need to follow a particular mathematical structure,
metrics, e.g., economic indicators. For different types of as the systems are commonly treated as black boxes [27].
triangulations, this MILP has been postulated in similar Drawbacks of SBO are that the ability to find an
ways by Misener et al. [54,55]. optimum to a given optimization problem with SBO is
heavily constrained by the computational tractability,
which can be easily exceeded by excessively high
2.2.3 Surrogate-assisted series of nonlinear programs
computational costs for simulation evaluations, a high
(NLP)
dimensionality of the problem, or the description of
multiple objectives and constraints [26]. Furthermore, as
The third option for the SSO suggested by the authors
information about derivatives in black-box systems is not
involves developing a superstructure based on GPR or
readily available, the proof of global optimality for the
ANN surrogate models and the reformulation of the
obtained solution by SBO remains challenging [57].
underlying MINLP into a series of NLP.
Among the several approaches for performing SBO, a
To remove the integer variables y from the problem as
surrogate model-based method with SK surrogate models
stated in Eq. (5), which are introduced for different
will be elucidated in this section, as their use is favorable
flowsheet options, the series of NLP has to involve an NLP
for computationally expensive simulations [44]. The
for each of these flowsheet options. For the set of possible
interested reader is referred to a comprehensive summary
process design configurations M with an index set
of the benefits and drawbacks of several other approaches
IM ¼ f1,2,:::,M g, a certain number of flowsheet simula-
by Amaran et al. [57]. Besides the mentioned benefits of
tions are performed for each configuration m 2 M : For
SBO, a surrogate-based approach gives into the structure
each flowsheet configuration m 2 M a GPR or ANN
of the search space and location of a possible global
surrogate is then fitted from a set of sample points Pm,x over
optimum [58]. SK as surrogate model type is an extended
the input space X
ℝn and a set of simulation points Pm,zOBJ
variant to the presented GPR in section 2.1.3. For the
for the metric to be evaluated in the objective function individual differences, the reader is referred to the original
over the output space Z OBJ
ℝ. This procedure is contributions [28,59].
performed for each of the metrics j to be considered as Following the framework described by Al et al., the SK
CON
constraint with a set of simulation points Pm,z,j over the surrogate is described with the following functional
respective output space Zj CON

ℝ. The boundaries for the relationship for any input point x 2 ℝn , closely related to
input variables are set to the bounds of the input space ∂X Eq. (4):
and a set of initial points in the input space X0 2 X are z ＝  ＋ εðxÞ ＋ LðxÞ, (26)
declared, in order to assure global optimality by perform-
ing a multi-start optimization with an amount of jX0 j ¼ s. where  is now a constant term referring to the mean value
The objective function is equally formulated as for the of the prediction and ε(x) and LðxÞ representing extrinsic
 Nikolaus I. Vollmer et al. Synergistic optimization framework 7

and intrinsic uncertainty in the prediction. Extrinsic with the presented solver in section 2.3 and will thus be
uncertainty describes the uncertainty of the surrogate used as such in the scope of this study.
model with regards to the high-fidelity model due to an
imperfect representation of the input space X
ℝn by the
set of sampling points PX . Intrinsic uncertainty represents 2.5 Framework S3O
uncertainty in the original model with regards to the
original physical system it represents. The model is fitted As mentioned, the conceptual design of novel bioprocesses
to the initial set of sampling points of the input space PX in a biorefinery setup is a highly complex challenge. They
and simulation points PZ in the output space Z 2 ℝ. arise from the difficulty of utilizing sustainable feedstocks
The performed optimization works as an evolutionary as lignocellulosic biomass, which requires additional unit
program, where an adaptive search with infill optimization operations in the upstream process, the problematic nature
under a given infill criterion is performed. With an initially of microorganisms of showing scale-dependent dynamics
small set of sample points, the infill optimization directs due to the intricate interactions in the regulation within the
the search towards more promising areas in the search transcriptional and metabolic network, and the arduous
space by adding new sample points to the set PX . In this conception of the downstream process, which has to
iterative procedure, the SK model is updated and hence account for the specifications in the upstream process
improved. The iterations are terminated upon reaching a while being constrained by economic limitations. While all
specific criterion, and the simulation results from each step three introduced process design strategies can be utilized,
can be investigated, and the optimum can be determined the resulting process design can face several hurdles
from the set of sample points of the last iteration i* [27]: deriving from a lack of conceptuality, intrinsic limitations
in their feasibility, or disproportionate computational
min z ¼ min Pi,*x : (27) burdens. To surmount these hurdles, the proposed frame-
Different infill criteria can be used for this task, and the work aims to leverage synergies in applying all proposed
reader is referred to the literature for an overview [27]. strategies in a hybrid manner, where the benefits of each
methodology are harnessed to expedite the global task of
2.4 Optimization under uncertainty designing a process conceptually.
It comprises three sequential steps, namely (1) the
Process design inherently involves various sources of selection of product sets, substrates, and operations, (2)
uncertainty. These uncertainties can be accommodated in SSO for determining candidate process topologies, and (3)
the optimization; however, this always implies a certain simulation optimization for consolidating an optimal
imparity in the calculated objective function, a so-called process design. It is illustrated in Fig. 1.
“price of robustness” [60]. Several factors influence this
tradeoff: Grossmann et al. mention the availability of 2.5.1 Selection of product sets, substrates, and operations
information on the type of uncertainty, as well as general
data on the uncertainty, the way of hedging against the The overarching idea in the framework is to “begin with
uncertainty, a tremendous computational burden, as well as the end in mind” [64,65]. Applying this principle in a
difficult tractability of the results for this [24]. biorefinery context, the first thing to be defined is the set of
Established practices in performing optimization under products. Due to the critical economic viability of
uncertainty in connection with mathematical programming biorefinery concepts, it is of utmost importance to choose
are robust optimization, chance-constraint programming, an appropriate portfolio of products, which exploits the
or stochastic programming, which are investigated for available substrate to the maximally possible extent and
several decades and recently received attention by the use potentially maximizes the economic key performance
of data-driven modeling techniques as, e.g., deep and indicators of the biorefinery. Once the set of products is
reinforcement learning in order to alleviate certain short- defined, the feedstock for the biorefinery has to be chosen
comings of the classical mathematical programming accordingly, where a feedstock candidate should contain
approaches [61]. However, for increased tractability of reasonable amounts of the respective substrate that is
the optimization problem, simulation-based approaches needed to produce the desired set of products.
that use high-fidelity simulation models and a metamodel- Based on both a defined set of products and a feedstock,
ing approach seem to be a more viable solution [24,62]. all potential process candidates’ necessary unit operations
Here, Monte Carlo methods, as introduced in section can be defined. As an integral part of this framework, this
A.2.1, represent a straightforward approach to include the step is heavily influenced by domain knowledge, which
optimization under uncertainty in SBO and allow a simple allows for a bottom-up assessment of the possible
statistical quantification of the objective and the constraint alternatives reducing the workload in this step immensely.
values. This has been performed successfully for similar With the defined number of alternatives for unit operations
process design tasks [27,63]. This procedure is integrated and process routes, a mechanistic model is developed for
8 Front. Chem. Sci. Eng.

Fig. 1 Illustration of the proposed framework S3O with its three stages: (1) selection of products and models, (2) SSO, and (3) SBO, as
well as the employed software and toolboxes.

each unit operation, incorporating the domain knowledge facilitated. The candidate topologies, which are prone to be
in the form of mathematical equations to describe the the best process design, are subjected to simulation
underlying physical, chemical, and biological phenomena. optimization under uncertainty to ultimately yield a
consolidated process design. Possible uncertainties that
can be included span from uncertainties in the technical
2.5.2 SSO domain, e.g., scale-up issues for fermentation reactors or
fluctuations in impurities that are separated in the down-
With all developed models and the set of all process stream processing, over the operational domain, e.g.,
alternatives, a superstructure is composed. By the prior varying product and feedstock prices and supply, up to the
selection of possible process alternatives through expert computational domain, e.g., uncertainties in model para-
knowledge and ultimately subjected to economic con- meters, error propagation properties of models and
siderations, the size of the superstructure is a priori heavily uncertainties in design parameters, as already mentioned
reduced and hence faster to solve. Classically, by in section A.2. Constraints from the prior stage can be
performing SSO, the result is a deterministically optimal added, as well as additional constraints. The entire
process design. A second paramount benefit of SSO is the workflow of the implemented framework is shown in
possibility to account for nontrivial design decisions, Fig. 2.
deriving from the nature of binary decisions in the process For the SSO, it is crucial to assess which option delivers
design. However, this does not allow directly to incorpo- the best results under the given objective of determining a
rate possible uncertainties, which is why the SSO in this small set of candidate process topologies. The constraints
framework serves as a selection tool for not only one here are that (1) the methodology should be able to pick
deterministic optimal process design but rather several candidates that turn out to be optimal, (2) delivers results
candidate process topologies. Furthermore, additional consistently with respect to differently shaped design
constraints regarding the operability of the process, spaces and flowsheet alternatives, (3) the SSO can be
product quality constraints, or others can be included in performed within a reasonable amount of time with
this stage. reasonable computational resources, (4) the results are
consistent with each other, e.g., considering different
sample sizes for the input space and (5) should yield
2.5.3 Simulation optimization solutions which are close to the theoretical underlying
global optimum of the original flowsheet model. In the
By selecting several candidate process topologies through following section 3.2, all proposed options are evaluated
SSO, the number of candidates subjected to simulation and benchmarked regarding these criteria to define which
optimization is again reduced and hence computationally methodology to choose ultimately.
 Nikolaus I. Vollmer et al. Synergistic optimization framework 9

Fig. 2 Workflow of the proposed framework S3O indicating the tasks in the three stages and its intermediate and the final results.

3 Case study and results 3.1 Selection of product sets, substrate and operations

To demonstrate the feasibility and capabilities of the As the first and main produced chemical in the biorefinery
introduced framework, a relevant industrial case study is concept in the case study, xylitol is selected. Xylitol is a
selected. After a concise description of the case study and sugar substitute with a similar taste to sucrose and
its relevance to industry, the application of the three steps manifold beneficial health properties, as around 40%
of the framework and the results are described and fewer calories than sucrose, a very low glycemic index,
analyzed. which makes it very suitable for diabetic nutrition, and
10 Front. Chem. Sci. Eng.

anti-cariogenic properties [66]. It has been determined as fraction can be either used in a combustion process to
one of the top 12 chemicals to be produced in a biorefinery provide heat for steam generation in order to integrate heat
concept by the US Department of Energy already in 2004, over the different process trains in the biorefinery or can be
maintained that status in 2010, and is still attracting further converted in a pyrolysis process into sustainable
researcher’s interest in order to facilitate a biotechnological aviation fuels, for which there is a high demand with
production in cell factories [67–71]. Nevertheless, and higher economic margins than for common biofuels as,
despite a market volume estimate of 1 Bio USD and more e.g., bioethanol [77–79]. The downstream process for both
in 2022, there are only a few companies worldwide, e.g., the cellulosic and the hemicellulosic process train can
DuPont Nutrition Biosciences, producing xylitol in a involve a classic setup with an evaporation unit and
chemical process from wood biomass or corn in a chemical following crystallization units or involve alternative
conversion process [70,72]. technologies as membrane separation; both approaches
Proposed process designs in literature are scarce; are applied commercially and have been investigated for
however, those who approach the idea of a process design their use in biorefinery downstream processes for both
for the biotechnological production of xylitol either xylitol and succinic acid [53,68,76]. As possible feedstock,
conclude that the chemical production process is econom- wheat straw with a high hemicellulosic content is chosen.
ically more safe and promising or that xylitol can increase A potential superstructure formulation for this base-case
the added value of a biorefinery with other principal process design is illustrated in Fig. 3.
products, as for example ethanol [73–75]. Hence, the It becomes evident that by following the concept of
process design of a biotechnological production process beginning with the end in mind and rigorously applying
for xylitol is a perfectly suitable case study for demonstrat- expert knowledge in a bottom-up composition approach
ing the proposed framework. for the superstructure, the initial search space is kept
Beginning with the end in mind, according to the comparatively small, which expedites all the following
framework, a suitable set of products is supposed to be steps in the workflow.
chosen in the first step of the workflow. As xylitol is However, in order to analyze the results of the
favorably produced from the hemicellulosic fraction of application of this framework more tractable and thus
lignocellulosic biomass, the product set can involve a accessible to the reader, we reduce the superstructure of
product for the cellulosic fraction of the biomass and one this base case design to a smaller subset of only the xylitol
for the lignin fraction. For the cellulosic fraction, succinic production train with a limited amount of unit operations.
acid is considered as a product. It is also one of the top 12 Once the framework is validated, the whole superstructure,
chemicals to be produced in a biorefinery setup and has as in Fig. 3, can be processed nonetheless. The reduced
potential as a platform chemical, which makes it attractive superstructure involves six unit operations, namely a
for several industrial branches, which benefits the biomass pretreatment unit operated as dilute acid pretreat-
economic resilience of the biorefinery [67,76]. The lignin ment (PT), an upconcentration unit (UCH), a fermentation

Fig. 3 Illustration of the entire initial bottom-up composed superstructure for the base-case process design of the introduced case study
with a hemicellulose, a cellulose, and a lignin process train; the reduced superstructure which will serve as the base case in this study is
marked in bold.
 Nikolaus I. Vollmer et al. Synergistic optimization framework 11

unit operated as batch fermentation (FX), an evaporation section A.1.4 and subsequently validated and calibrated
unit (EX), a first crystallization unit operated as cooling with proprietary experimental data. The considered opera-
crystallization (CX1) and a second crystallization unit tional variables are the crystallization time tCX1,2 in h, the
operated as antisolvent crystallization (CX2). Out of these flowrate of coolant for CX1 FC,CX1 in kg$s–1 and the
six unit operations, three are compulsory (PT, FX, CX1), flowrate of antisolvent for CX2 FAS,CX2 in kg$s–1 as well as
and three are optional (UCH, EX, CX2), which results in the cooling temperature TC,CX1 in °C for CX1. The
eight binary decisions or eight different flowsheet alter- considered model outputs for this study are the mass of
natives. These are listed in Table 1 together with their produced xylitol MXyo in kg, the concentration of the
configuration ID (cID), which will be used as an identifier inhibitory compounds 5-hydroxymethylfurfural and acetic
throughout the section. acid in the final process stage C5HMF ,CAac in g$L–1 as well
Table 1 Overview of all flowsheet options with their respective cID as a CO2 ratio φ in kg$kg–1 indicating how much CO2 is
and the units composing the flowsheet.
produced per kilogram of xylitol by the generation of
cID Flowsheet
steam to provide heat in the pretreatment and the
evaporation units. The considered uncertainty in the third
1 PT-UCH-FX-EX-CX1-CX2
step of the framework is the composition of the feedstock.
2 PT-UCH-FX-EX-CX1 All models are implemented in MATLAB. The evapora-
3 PT-UCH-FX-CX1-CX2 tion model is interfaced with a COM interface to
4 PT-UCH-FX-CX1 MATLAB. All models are assessed regarding their
robustness by means of Monte Carlo-based uncertainty
5 PT-FX-EX-CX1-CX2
and sensitivity analysis as described in sections A.2.1 and
6 PT-FX-EX-CX1
A.2.2. Hence, the full design space for the case study is set
7 PT-FX-CX1-CX2 up by factorial selection in SPDlab, while excluding
8 PT-FX-CX1 infeasible options a priori. All model implementations are
available through the S3O GitHub repository [82].

With eight flowsheet options as listed in Table 1, the 3.2 SSO

following step 2 of the framework could also be solved by
a purely enumeration-based approach instead of formulat- 3.2.1 Flowsheet sensitivity analysis
ing an SSO problem. However, to rigorously search
through the entire design space for globally optimal In order to perform the design space exploration, all
solutions and to better account for nontrivial design operational variables for each flowsheet are considered as
decisions while accelerating calculation times, the SSO input variables. With the functionalities of SPDlab, N =
approach is the favorable option. The small problem size is 2000 samples with LHS sampling for each flowsheet (cIDs
chosen as mentioned to enhance the tractability in the 1–8) are simulated [83]. The flowsheet samples are used as
scope of this manuscript. input for the flowsheet sensitivity analysis by using the
For all the unit operations, mechanistic models are easyGSA toolbox in MATLAB [42] to determine which of
developed: the PT model is set up as described in section the operational variables are most important regarding the
A.1.1 while considering the same components as Prunescu model output and should hence be considered as variables
et al. [80]. The data for the calibration of the model derives in the optimization problem. The design space exploration
from proprietary experiments. The operational variables results are illustrated in Fig. 4 in violin plots for each
considered in the pretreatment model are the pretreatment model output for each relevant flowsheet option. The
temperature TPT in °C, the pretreatment time tPT in min and results of the ANN-assisted flowsheet sensitivity analysis
the acid concentration acid in wt-% to the original are illustrated in a heatmap in Fig. 5. All scripts and
biomass. The fermentation model (FX) is set up as implementations regarding the flowsheet sensitivity ana-
described in section A.1.2, while using data from batch lysis are available through the S3O GitHub repository [82].
experiments from Tochampa et al. [81]. The considered As a first major result, the flowsheets with the cIDs 3, 4,
operational variables for the fermentation model are the 7, and 8 turn out to be infeasible with the given design
fermentation time tFX in h as well as the inoculum space, as no set of input variables results in produced
concentration inoc in g$g–1 broth. The evaporation model xylitol. Furthermore, it becomes evident that there are
(UCH, EX) is set up as described in section A.1.3 in the major differences between the flowsheets’ design spaces
ASPEN Plus process simulation software while using the with cIDs 1, 2, 5, and 6. Where cID 6 seems to have an
DIPPR property database and the NRTL equation of state. evenly distributed number of points regarding the mass of
The considered operational variable is the vapor fraction xylitol produced, for cID 6, most of the sets of input
for either evaporation vEX or upconcentration vUCH . The variables turn out to be infeasible with only a small feasible
crystallization model (CX1, CX2) is set up as described in fraction. Regarding the inhibitory compounds, especially
12 Front. Chem. Sci. Eng.

Fig. 4 Violin plots of the results from the design space exploration of flowsheets (a) cID 1, (b) cID 2, (c) cID 5, and (d) cID 6 with the
outputs: the mass of produced xylitol (upper left), the concentration of 5-hydroymethylfurfural in the final stage (upper right), the
concentration of acetic acid in the final stage (lower left) and the CO2 ratio (lower right).

Fig. 5 Heatmap of the total sensitivity indices (STi ) calculated with the easyGSA toolbox by using ANN surrogates for all flowsheet
options and all operational variables.

cIDs 2 and 6, as well as cID 5 to minor extents, seem to operational variables seem vital despite a different order,
produce significant amounts of inhibitors. This allows for and also similar variables are insensitive concerning the
the conclusion that both inhibitory compounds should be output. Hence, for cIDs 1, 5, and 6, we define TPT , acid,
constrained to a certain level in the optimization problem inoc, tFX and vEX and for cID 2, we define acid, in
formulation to ensure product quality. oc, tFX , vEX and vUCH as the variables to be considered in
Regarding the flowsheet sensitivity analysis, a more the optimization problem. All simulations consider an
uniform picture emerges. For all four flowsheets, similar initial amount of lignocellulosic biomass of M ¼ 1000 kg.
 Nikolaus I. Vollmer et al. Synergistic optimization framework 13

In conclusion, the design space exploration helped order to obtain the validation metrics. All scripts and
reduce the initial search problem by 50%, and the implementations regarding the surrogate performance
flowsheet sensitivity analysis can be used as a tool for analysis are available through the S3O GitHub repository
identifying crucial variables for SSO. [82].
The calculated validation metrics are the coefficient of
determination R2 and the root mean square error (RMSE)
3.2.2 Surrogate model performance assessment
for the whole dataset, as well as the testing and the training
dataset, dependent on the applied validation method.
For the surrogate model development and validation, N =
Figure 6 shows the parity plots for both N = 500 and N
500 and N = 1000 flowsheet simulations with LHS
= 1000 flowsheet samples for each surrogate model
sampling are performed to compare their performance with
predicting the amount of produced xylitol. For the
a different number of sampling points. The flowsheets
ALAMO surrogate models, the parity plots show a
simulations are again performed with the SPDlab func-
relatively high variance in the model and the tendency to
tionalities. All four surrogate model types are fitted and
underpredict higher values for the amount of xylitol.
cross-validated with these sampling points, as explained in
Secondly, the GPR surrogates show almost a perfect fit,
section 2.1. The ALAMO surrogates are fitted within the
apart from cID 6, where the accuracy is slightly lower than
ALAMO software. For cross-validation, 20% of the input
for the other three flowsheets. Lastly, the ANN surrogates
samples are split off, and ALAMO is given this fraction as
show equally good fits to the data, however, with higher
a validation dataset to calculate validation metrics
variances than the GPR models and a decrease in the
internally. The DTR surrogates are all created with the
prediction quality for cID 6. This can be attributed to the
Delaunay functionality in Python’s scipy package, utiliz-
low number of feasible sampling points in the design space
ing the QHull algorithm [41]. For their cross-validation,
of cID 6, which is diametric to both surrogate model
due to the model’s inexistent extrapolative capabilities, the
methodologies as they rely heavily on the amount of
boundary points, which form the convex hull of the design
provided input data. The DTR surrogates are not shown as
space, are added to the sampling points. By random
parity plots as the simulation points become an inherent
selection, a fraction of 20% of the sample points is split off
part of the model, which makes the coefficient of
to calculate the validation metrics for the training and
testing dataset. This procedure is repeated ten times to determination R2 ¼ 1 by definition and a parity plot thus
calculate the range of the cross-validation metrics. dispensable.
The GPR surrogates are fitted in MATLAB while For all illustrated models in Fig. 6, the cross-validation
utilizing the statistics and machine learning toolbox. For metrics for the full, the testing, and the training data set for
each dataset and each output, an internal routine to flowsheet cID 1 for the output variable being the amount of
optimize hyperparameters, e.g., the kernel functions in produced xylitol are listed in Table 2.
MATLAB, is used with default settings to fit the GPR The cross-validation metrics for all other flowsheet cIDs
model to the data. Subsequently, a 5-fold cross-validation, and the other output variables are listed in the supporting
as described in section 2.1, is performed to obtain the material. For the ALAMO surrogate models, the impres-
validation metrics. The ANN surrogate is equally fitted in sion from the parity plots is confirmed by average values
MATLAB while using the deep learning toolbox. For each for R2 between 0.6 and 0.8 and RMSE values, which are
dataset and each output, a grid-search algorithm as significantly high. As described earlier, the R2 and RMSE
implemented in the easyGSA toolbox is employed to values for the DTR surrogate model are immanently 0 or 1
optimize several parameters, e.g., the number of nodes in for the full and the training data set, but for the testing
the hidden layer to fit the ANN model to the data. dataset, it becomes obvious that the quality of fit for unseen
Afterward, also a 5-fold cross-validation is performed in data is insufficient, expressed by R2 values around 0.6 and

Table 2 Cross-validation metrics of all surrogate models for flowsheet option cID 1 for both N = 500 and N = 1000 samples for the output variable
being the amount of produced xylitol.
ALAMO DTR GPR ANN
Model
N = 500 N = 1000 N = 500 N = 1000 N = 500 N = 1000 N = 500 N = 1000
R2 0.822 0.765 1 1 1 1 0.997 0.994
RMSE 5.27 6.29 0 0 0.007 0.017 0.597 0.922
R2train 0.817 0.762 1 1 0.997 1 0.997 0.994
R2test 0.722 0.724 0.487 0.642 0.933 0.952 0.895 0.956
RMSEtrain 5.35 6.31 0 0 0.423 0.121 0.674 0.944
RMSEtest 6.54 6.99 8.802 7.677 2.945 2.66 4.002 2.535
14 Front. Chem. Sci. Eng.

Fig. 6 Parity plots of the ALAMO, the GPR, and the ANN surrogate models for all flowsheets (ALAMO: (a) cID 1, (d) cID 2, (g) cID 5,
and (j) cID 6; GPR: (b) cID 1, (e) cID 2, (h) cID 5, and (k) cID 6; ANN: (c) cID 1, (f) cID 2, (i) cID 5, and (l) cID 6) indicating the predicted
outputs over the simulated outputs for N = 500 (dark blue, blue, turquoise) and N = 1000 samples (green, bright green, yellow).

RMSE values up to almost 10. Equally, for the GPR and with N = 1000 instead of N = 500 samples, which is in
the ANN surrogate, the validation metrics confirm the agreement with the described properties in section 2.1. The
parity plots’ results; both model types show excellent ALAMO surrogates do not show a consistent improvement
validation scores even for the testing data sets. As a general of the validation metrics with increasing sample size,
trend, it is also to denote that for the DTR, the GPR, and which is relatable to the difficulty of fitting a given number
the ANN surrogates, the model quality overall increases of algebraic terms to datasets of increasing size.
 Nikolaus I. Vollmer et al. Synergistic optimization framework 15

Overall, it is to state that both machine learning linear functional relationships as given with Eq. (2).
surrogate models show the best validation metrics and Equations (6) through (25) are implemented in PYOMO.
both ALAMO and especially the DTR surrogates reveal Equally for the MILP, the four flowsheets are solved
insufficient predictive abilities for unseen data points in the separately, which reduces the dimensionality of the binary
test dataset. variable. However, the resulting simplified MILP yields
the same solution; hence the simplification is valid. For N =
500 flowsheet samples the resulting optimization problem
3.2.3 SSO results
results in around 1000000 continuous and 50000 integer
variables and double the number for N = 1000 flowsheet
In order to define the underlying optimization problem to
samples. All problems are solved to global optimality.
the SSO properly, the following objective function and
For both the GPR and the ANN surrogates, the
constraints are introduced:
MATLAB solver fmincon with the sequential quadratic
8
>
> max MXyo ¼ f ðx,yÞ programming algorithm is chosen. The formulation of the
>
> problem is equally given with Eq. (28), with f , g1 , g2 and
>
>
>
> s:t: CHmf ¼ g1 ðx,yÞ£0:5 g$L – 1 g3 being functional relationships as given with Eq. (3)
<
MINLP : CAac ¼ g2 ðx,yÞ£0:5 g$L – 1 : (28) while excluding the binary variable y. The surrogates are
>
> passed to the solver together with a multi-start option,
>
>
>
> φ ¼ g3 ðx,yÞ³0,1 indicating that the optimization problem should be solved
>
>
: for q ¼ 1000 times in order to ensure global optimality.
x 2 X , y 2 ½0,1
For flowsheets where the optimization problem yields a
The operational variables for the solver to choose are high fraction of all the multi-start solutions converging, the
acid, inoc, tFX ,vEX for all flowsheet cIDs, TPT for flow- results are within the same magnitude as the other
sheet cIDs 1, 2, and 6 and vUCH for flowsheet cID 2. In flowsheets’ results. With cID 2 and N = 1000 samples
order to solve the underlying optimization to each SSO, and also for cID 6 and the ANN, however, the observed
suitable solvers have to be chosen. For the ALAMO solutions diverge, which is also reflected in the small
surrogates and the resulting MINLP, the BARON solver is number of multi-starts converging. All scripts and
chosen [84]. All algebraic equations for each flowsheet implementations regarding the SSO are available through
option are instantiated in PYOMO. However, due to the the S3O GitHub repository [82].
reduced design space of four flowsheet options, the binary All results from the SSO for all four flowsheets for N =
variable is removed, and the four flowsheets are solved 500 samples are listed in Table 3 until Table 6. Results
separately to accelerate the calculation process. This from the SSO of flowsheet cID 6 with N = 500 samples
converts the MINLP given as in Eq. (28) into four NLPs; with all surrogate models and their respective solvers. All
however, the results do not differ as the underlying tables indicate the predicted objective function values and
optimization problem does not change. Hence this constraints (opt) and the results from the corresponding
simplification is valid. All optimization problems are validation simulation with the original flowsheet for the
solved with the default solver settings. If a solution is same conditions (val) — according to optimal operational
found, the problem is solved to global optimality; conditions and their lower (lb) and upper (ub) bounds.
otherwise, the optimization problem is found infeasible. For the MINLP problems utilizing ALAMO surrogates
For the DTR surrogates and the resulting MILP, the and the BARON solver, it becomes clear that the low
Gurobi solver is chosen. The formulation of the problem is surrogate quality, expressed by the poor performance
equally given with Eq. (28), with f , g1 , g2 and g3 being metrics, heavily affects the feasibility and the quality of the
Table 3 Results from the SSO of flowsheet cID 1 with N = 500 samples with all surrogate models and their respective solvers.
cID 1-500 ub lb ALAMO/BARON DTR/Gurobi GPR/fmincon ANN/fmincon
opt val opt val opt val opt val
TPT 173 195 179.581 184.31 187.74 177.24
Acid 0.5 2 0.672 1.456 1.337 2.000
Inoc 0.5 3 3.000 1.523 1.497 1.191
tFX 8 16 47.938 43.207 42.656 47.727
vEX 0.99 0.998 0.995 0.996 0.998 0.998
MXyo 59.852 0.000 49.094 48.964 54.083 43.410 85.240 45.682
CHmf 0.5 0.000 0.028 0.058 0.060 0.034 0.006 0.020 0.007
CAac 0.5 0.002 0.004 0.002 0.002 0.001 0.000 0.001 0.000
φ 0.1 0.140 0.000 0.118 0.117 0.116 0.100 0.100 0.114
16 Front. Chem. Sci. Eng.

Table 4 Results from the SSO of flowsheet cID 2 with N = 500 samples with all surrogate models and their respective solvers.
cID 2-500 ub lb ALAMO/BARON DTR/Gurobi GPR/fmincon ANN/fmincon
opt val opt val opt val opt val
Acid 0.5 2 0.685 0.762 0.874 2.000
Inoc 0.5 3 2.998 1.493 0.500 2.736
tFX 12 48 47.999 46.427 40.493 12.490
vEX 0.99 0.998 0.996 0.997 0.990 0.998
vUCH 0.4 0.6 0.512 0.520 0.585 0.423
MXyo 53.004 0.000 50.017 51.123 13.315 0.078 4.410 11.938
CHmf 0.5 0.500 3.935 0.500 0.451 1.830 0.779 1.467 0.136
CAac 0.5 0.000 0.289 0.022 0.021 0.082 0.048 0.057 0.008
φ 0.1 0.123 0.000 0.117 0.120 0.028 0.000 0.037 0.028

Table 5 Results from the SSO of flowsheet cID 5 with N = 500 samples with all surrogate models and their respective solvers.
cID 5-500 ub lb ALAMO/BARON DTR/Gurobi GPR/fmincon ANN/fmincon
opt val opt val opt val opt val
TPT 173 195 193.69 185.50 186.6
Acid 0.5 2 0.776 1.188 1.127
inoc 0.5 3 2.315 0.963 0.782
tFX 8 16 28.415 45.079 48.000
vEX 0.99 0.998 0.993 0.998 0.998
MXyo Infeasible 47.915 47.86 54.829 48.12 67.400 46.86
CHmf 0.5 0.152 0.152 0.057 0.038 0.044 0.022
CAac 0.5 0.012 0.126 0.003 0.002 0.002 0.001
φ 0.1 0.118 0.118 0.132 0.123 0.168 0.117

Table 6 Results from the SSO of flowsheet cID 6 with N = 500 samples with all surrogate models and their respective solvers.
cID 6-500 ub lb ALAMO/BARON DTR/Gurobi GPR/fmincon ANN/fmincon
opt val opt val opt val opt val
TPT 173 195 184.00 184.00 191.04
Acid 0.5 2 0.960 1.265 1.609
inoc 0.5 3 2.536 2.507 0.976
tFX 8 16 23.221 22.465 45.041
vEX 0.99 0.998 0.998 0.998 0.997
MXyo Infeasible 43.688 43.95 47.727 43.58 0.016 26.35
CHmf 0.5 0.373 0.347 0.500 0.322 11.321 4.627
CAac 0.5 0.022 0.021 0.014 0.018 0.243 0.134
φ 0.1 0.112 0.112 0.101 0.111 0.000 0.066

optimization results. For this option, none of the obtained all flowsheets for all sample sizes. However, it becomes
solutions were satisfactory. However, for the MILP clear that for all flowsheets, the theoretical underlying
problems utilizing the DTR surrogates, all solutions global optimum is not reached. This is attributable to the
show a very close agreement between the predicted number of sampling points provided as the DTR surrogates
solutions and their validations, despite the insufficient are strictly interpolative and can only predict according to
validation metrics. Furthermore, the problem is solved for the provided input samples.
 Nikolaus I. Vollmer et al. Synergistic optimization framework 17

For the GPR and the ANN surrogates, for N ¼ 500 simulations are added (cross).
samples, the results are mostly not in agreement with the Again, it becomes clear that the most consistent
validation simulations despite the good validation metrics. combination of the surrogate model, optimization problem,
For N ¼ 1000 samples, however, the results from the and solver is the choice of DTR surrogates despite its
optimization improve and converge with the validation validation metrics, as overall, the consistency is highest.
results. This is explained by the general trend of machine For both the GPR and the ANN, it is visible that the models
learning models to show an improved prediction when predict higher objective function values, but the validation
provided with larger amounts of training data. Overall it simulations are less in agreement than with the DTR
shows that the GPR-assisted NLP predicts more consis- surrogates. For the ALAMO surrogates, it becomes
tency for different flowsheets and sampling sizes. In apparent that their performance in the given optimization
contrast to that, the ANN assisted NLP predicts more problem is impaired. This can potentially be attributed to
inconsistently, depending on the sampling size and the underprediction surrogate models, as seen in Fig. 6,
flowsheet, but with successful predictions, the predicted which do not allow for an optimal solution under the given
values for the objective function are higher than the ones constraints.
predicted by the GPR and thus closer to what would Overall, it is to point out that after analyzing the quality
correspond to the global optimum for the rigorous of the surrogate models and the results from the SSO, the
flowsheet. indication regarding the quality between the different
The illustration Fig. 7 indicates both the prediction of surrogate models is ambiguous. However, regarding the
each surrogate model for each flowsheet and sample size as underlying case study, it becomes apparent that flowsheet
the center of the circle and the root mean square error of the cID 1 shows the best objective function values for both
testing data set of the surrogate model as the radius of the sample sizes; hence it should be subjected to investigation
circle. Furthermore, the results from the validation in the third step of the framework. Both cID 2 and cID 5

Fig. 7 Bubble plot for the visualization of the consistency metrics of the different superstructure modeling approaches, the center of each
sphere indicating the predicted value in the optimization problem, the radius of the sphere being the RMSE of the testing dataset in the
cross-validation, and the cross/saltire indicating the respective validation simulation for (a) cID 1, (b) cID 2, (c) cID 5 and (d) cID 6 for
respectively N = 500 samples (blue, cross) and N = 1000 samples (yellow, saltire).
18 Front. Chem. Sci. Eng.

show very similar objective function values and constraint flowsheet option with the highest number of possible
values, which is why both should be equally subjected to unit operations to maximize the product yield. Both cID 2
investigation in the third step of the framework. and cID 5 seem to perform equally well, despite their
difference being once performing an upconcentration
3.3 SBO results before the fermentation and once performing two crystal-
lization steps in the downstream processing. Overall, the
All flowsheets considered candidate process topologies yield for all flowsheet options is comparatively low. This
from step two of the framework are now subjected to SBO indicates the possible use of alternative unit operations and
using the MOSKopt solver [27], utilizing SK surrogate operation modes in the initial superstructure to increase the
models. As uncertain input, the wheat straw composition is amount of final product.
chosen to vary around 5% by the nominal value. The Secondly, it is noteworthy that for all the three
composition of the feedstock is highly dependent on flowsheets, the predicted optimum for each respectively,
climate effects as well as geological conditions of the despite being subjected to uncertainty in the feedstock, is
fields, amongst others, which lead to varying compositions globally higher than by any of the introduced SSO
[85]. Logically, a varying feedstock composition influ- alternatives. Figure 8 indicates for each cID how much
ences the product yields, explaining the importance of the objective function value improved quantitatively over
critically assessing this uncertainty. The initially assumed the 100 iterations when providing the operational condi-
composition is 31.3% hemicellulose, 42.7% cellulose, and tions found to be optimal in step 2.
the residual as lignin; the uncertainty is supposed to be In conclusion, the performed SBO based on the SSO
uniformly distributed. For the SBO, twenty-five initial results from step 2 appears to be an excellent combinatorial
samples and each sample with 100 Monte Carlo samples solution leveraging synergistic effects between screening a
are chosen. The optimization criterion for the underlying multitude of alternatives and thoroughly designing few
Bayesian Optimization is the multi-constraint FEI, as alternatives.
explained in Al et al.’s work [27]. The solver hedges
against uncertainty with the simulation’s mean values and
performs 75 iterations, which results in 100 total calcula- 4 Conclusions and future research
tion steps. The starting points are chosen to be the optimal
results for each flowsheet from step two of the framework. 4.1 Conclusions
All scripts and implementations regarding the SBO are
available through the S3O GitHub repository [82]. The In this paper, we presented a framework to expedite the
results from the three SBO runs are listed in the following conceptual process design of novel bioprocesses in
Table 7 (opt) together with the corresponding validation biorefinery setups by leveraging synergistic effects from
simulations (val). applying expert knowledge and combining SSO and SBO
Firstly, the results from the SSO are confirmed, referring in a hybrid manner. In this process, four different SSO
to that cID 1 appears to be the optimal flowsheet in the alternatives are investigated and benchmarked. The
given design space, including a pretreatment unit, an proposed bottom-up approach to compose the initial
upconcentration unit, a fermentation unit, an evaporation superstructure by beginning with the end in mind and
unit, and two crystallization units. This is also the first selecting a product set, a feedstock, and subsequently,
Table 7 Results from the SBO for all candidate process topologies with the MOSKopt solver, using 25 initial sampling points, 75 iterations, the
mean value as hedge against uncertainty, and the multi-constraint FEI criterion
ub lb cID 1 cID 2 cID 5
opt val opt val opt val
TPT 173 195 195.000 195
Acid 0.5 2 0.715 0.984 0.879
Inoc 0.5 3 1.611 3.000 3
tFX 8 16 44.994 30.367 24.271
vEX 0.99 0.998 0.997 0.998 0.998
vUCH 0.4 0.6 0.400
MXyo 56.310 56.736 53.760 54.224 53.96 54.17
CHmf 0.5 0.045 0.032 0.492 0.471 0.062 0.061
CAac 0.5 0.001 0.001 0.019 0.020 0.002 0.002
φ 0.1 0.119 0.125 0.127 0.129 0.128 0.128
 Nikolaus I. Vollmer et al. Synergistic optimization framework 19

Fig. 8 Visualization of the improvement of the objective function value over the iterations in the SBO with the MOSKopt solver for
flowsheet (a) cID 1, (b) cID 2, and (c) cID 5.

the processing units reduce the initial maximum size of the coefficient of determination and the root mean square error,
superstructure heavily. are solitarily not a good indicator of the surrogate model’s
The second step of the proposed framework is based on quality in the subsequent optimization application. For
using SSO to solve the process synthesis problem under- example, the GPR and ANN surrogates suffer from several
lying the superstructure and determine several candidate shortcomings in the optimization problem, whereas their
process topologies. Four different alternatives of solving validation metrics are superior. Opposed to this, the DTR
the optimization problem by surrogate model-assisted SSO surrogates’ validation metrics are inferior compared to the
are proposed: the use of ALAMO surrogate models in an GPR and the ANN, but the optimization results are
MINLP, the use of DTR surrogate models in an MILP, and consistent and qualitatively better. Hence, we recommend
the use of either GPR or ANN surrogate models in a series a critical assessment of the quality of a chosen surrogate
of NLP with a multi-start solution strategy. The latter three model and its validation in the respective optimization
options serve the purpose of predicting several candidate problem. We also point out that the combination of DTR
process topologies. A thorough benchmark of all four surrogates in an MILP has shown promising results in
alternatives reveals that all surrogate models have different using the proposed framework in this particular case study.
shortcomings when applied in the framework. A general Regarding the size of the resulting superstructure
property of surrogate models is the shortcoming in terms of formulations for larger design cases, it is to state that
adhering to mass balance restrictions. Furthermore, they do also the computational burden will consequently increase
not rely on physical constraints but rather on an alternative but can be alleviated in several ways. For example, the
mathematical formulation and the quantity of provided necessary flowsheet simulations can be expedited by using
data to fit the behavior of the underlying high-fidelity both a parallelization approach to evaluate bigger numbers
model accurately. of flowsheet options and utilizing cloud or cluster
In consequence, the predictions of surrogate models computing solutions to accelerate the speed of a single
inherently deviate to a certain extent. Hence, a combined simulation. Furthermore, by decreasing the initial size of
validation of both surrogate performance and optimization the superstructure as in step 1 of the framework and solely
results is essential to benchmark the proposed alternatives. focusing on several candidate process topologies in step 3,
In other words, the mere selection of a surrogate model the overall computational burden is reduced optimally. It
based on convincing cross-validation metrics is not a can also facilitate the solution of larger problems than the
guarantee to obtain a consistent and qualitatively good presented one.
result from the resulting optimization problem in which the Lastly, the third step of the framework involves the SBO
surrogate model is used. We have empirically presented in of all candidate process topologies from step 2. The used
this study that cross-validation metrics alone, i.e., the solver MOSKopt can improve the objective function value
20 Front. Chem. Sci. Eng.

consistently for all subjected candidates under a given Acknowledgements The authors would like to express their gratitude to the
uncertainty scenario and yields a consolidated optimal Novo Nordisk Foundation (Grant No. NNF17SA0031362) for funding the
Fermentation-Based Biomanufacturing Initiative of which this project is a
process design as the final result of the framework. part.
Overall, it remains to conclude that our proposed
framework was validated successfully. It considerably Electronic Supplementary Material Supplementary material is available
facilitates and accelerates the conceptual process design of in the online version of this article at https://dx.doi.org/10.1007/s11705-021-
novel bioprocesses in biorefinery setups, as shown with an 2071-9 and is accessible for authorized users.
example of a xylitol production process in a biorefinery.
References
4.2 Future research
1. United Nations. Transforming our world: the 2030 agenda for
Despite the impaired validation metrics, the proposed sustainable development, 2015
framework with the DTR surrogate models performed best 2. Ubando A T, Felix C B, Chen W H. Biorefineries in circular
in the optimization task and is hence the suggested bioeconomy: a comprehensive review. Bioresource Technology,
alternative in this case. However, to improve the validation 2020, 299: 122585
metrics and improve the predictions in terms of global 3. Straathof A J J, Wahl S A, Benjamin K R, Takors R, Wierckx N,
optimality, a first option would be to base the framework Noorman H J. Grand research challenges for sustainable industrial
on adaptive sampling strategies instead of a static sampling biotechnology. Trends in Biotechnology, 2019, 37(10): 1042–1050
strategy as Latin hypercube sampling [86–88]. Alterna- 4. Hillson N, Caddick M, Cai Y, Carrasco J A, Chang M W, Curach N
tively, algorithms that reduce the mesh’s original size by C, Bell D J, Feuvre R L, Friedman D C, Fu X, et al. Building a
iteratively removing edges of the triangulation can be global alliance of biofoundries. Nature Communications, 2019, 10
employed [89]. The improvement for the triangulation and (1): 1038–1041
subsequently the MILP is a reduced number of simplices 5. Hassan S S, Williams G A, Jaiswal A K. Lignocellulosic
by having fewer sampling points in regions, where a linear biorefineries in Europe: current state and prospects. Trends in
interpolation with a lower error is possible, and increasing Biotechnology, 2019, 37(3): 231–234
the number of sampling points in regions, where the linear 6. Hassan S S, Williams G A, Jaiswal A K. Moving towards the second
interpolation causes a greater error and improving the generation of lignocellulosic biorefineries in the EU: drivers,
prediction in the vicinity of the global optimum. This has challenges, and opportunities. Renewable & Sustainable Energy
been proposed in the literature by Chen et al. for the case of Reviews, 2019, 101: 590–599
a triangulation model but also as assistance for improving 7. Moncada B J, Aristizábal M V, Cardona A C A. Design strategies
the fit of other machine learning models as GPRs, which for sustainable biorefineries. Biochemical Engineering Journal,
can be equally used for this optimization task, as shown in 2016, 116: 122–134
this work [90,91]. 8. Chaturvedi T, Torres A I, Stephanopoulos G, Thomsen M H,
Overall, we acknowledge that the presented results here Schmidt J E. Developing process designs for biorefineries-
are specific to the selected case study. Therefore, the definitions, categories, and unit operations. Energies, 2020, 13(6):
application in further case studies is necessary in order to 1493
consolidate these conclusions further. Indeed, we believe 9. Kokossis A C, Yang A. On the use of systems technologies and a
there are concrete empirical observations, which indicate systematic approach for the synthesis and the design of future
that the validation issue of emerging surrogate-based/ biorefineries. Computers & Chemical Engineering, 2010, 34(9):
machine learning-based optimization approaches needs to 1397–1405
be more critically analyzed and assessed to ensure their 10. Chemmangattuvalappil N G, Ng D K S, Ng L Y, Ooi J, Chong J W,
appropriate use in further process systems engineering Eden M R. A review of process systems engineering (PSE) tools for
applications. the design of ionic liquids and integrated biorefineries. Processes
From a more holistic perspective, the use of this (Basel, Switzerland), 2020, 8(12): 1–29
framework is not only limited to process design but can 11. Tey S Y, Wong S S, Lam J A, Ong N Q X, Foo D C Y, Ng D K S.
equally be applied for the optimization of whole value Extended hierarchical decomposition approach for the synthesis of
chains, involving the choice of different feedstocks, plant biorefinery processes. Chemical Engineering Research & Design,
locations, and logistic constraints. Lastly, especially for the 2021, 166: 40–54
used fermentation models, the black-box kinetics can be 12. Clauser N M, Felissia F E, Area M C, Vallejos M E. A framework
replaced by genome-scale metabolic models in order to for the design and analysis of integrated multi-product biorefineries
perform also cell factory optimization, which can con- from agricultural and forestry wastes. Renewable & Sustainable
tribute to the further expedition of designing bioprocesses Energy Reviews, 2021, 139: 110687
and promote the transition towards a bio-based and circular 13. Mountraki A D, Benjelloun-Mlayah B, Kokossis A C. A surrogate
economy as instigated by the 2030 Sustainability Agenda modeling approach for the development of biorefineries. Frontiers in
of the United Nations. Chemical Engineering, 2020, 2: 12
 Nikolaus I. Vollmer et al. Synergistic optimization framework 21

14. Pyrgakis K A, Kokossis A C. A total site synthesis approach for the 30. Friedman M. Multivariate adaptive regression splines. Annals of
selection, integration and planning of multiple-feedstock biorefi- Statistics, 1991, 19(1): 1–67
neries. Computers & Chemical Engineering, 2019, 122: 326–355 31. Sudret B. Global sensitivity analysis using polynomial chaos
15. Meramo-Hurtado S I, González-Delgado Á D. Biorefinery synthesis expansions. Reliability Engineering & System Safety, 2008, 93(7):
and design using sustainability parameters and hierarchical/3D 964–979
multi-objective optimization. Journal of Cleaner Production, 2019, 32. Williams B A, Cremaschi S. Surrogate model selection for design
240: 118134 space approximation and surrogatebased optimization. Computer-
16. Galanopoulos C, Giuliano A, Barletta D, Zondervan E. An Aided Chemical Engineering, 2019, 47: 353–358
integrated methodology for the economic and environmental 33. Janssen H. Monte-Carlo based uncertainty analysis: sampling
assessment of a biorefinery supply chain. Chemical Engineering efficiency and sampling convergence. Reliability Engineering &
Research & Design, 2020, 160: 199–215 System Safety, 2013, 109: 123–132
17. Ulonska K, König A, Klatt M, Mitsos A, Viell J. Optimization of 34. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical
multiproduct biorefinery processes under consideration of biomass Learning: Data Mining, Inference, and Prediction. 2nd ed. New
supply chain management and market developments. Industrial & York: Springer, 2009
Engineering Chemistry Research, 2018, 57(20): 6980–6991 35. Wilson Z T, Sahinidis N V. The ALAMO approach to machine
18. Aristizábal-Marulanda V, Cardona Alzate C A. Methods for learning. Computers & Chemical Engineering, 2017, 106: 785–795
designing and assessing biorefineries. Biofuels, Bioproducts & 36. Cozad A, Sahinidis N V, Miller D C. Learning surrogate models for
Biorefining, 2019, 13(3): 789–808 simulation-based optimization. AIChE Journal. American Institute
19. Meramo-Hurtado S I, González-Delgado Á D. Process synthesis, of Chemical Engineers, 2014, 60(6): 2211–2227
analysis, and optimization methodologies toward chemical process 37. Eslick J C, Ng B, Gao Q, Tong C H, Sahinidis N V, Miller D C. A
sustainability. Industrial & Engineering Chemistry Research, 2021, framework for optimization and quantification of uncertainty and
60(11): 4193–4217 sensitivity for developing carbon capture systems. Energy Procedia,
20. Darkwah K, Knutson B L, Seay J R. A Perspective on challenges 2014, 63: 1055–1063
and prospects for applying process systems engineering tools to 38. Miller D C, Siirola J D, Agarwal D, Burgard A P, Lee A, Eslick J C,
fermentation-based biorefineries. ACS Sustainable Chemistry & Nicholson B, Laird C, Biegler L T, Bhattacharyya D, Sahinidis N V,
Engineering, 2018, 6(3): 2829–2844 Grossmann I E, Gounaris C E, Gunter D. Next generation multi-
21. Biegler L T, Grossmann I E, Westerberg A W. Systematic Methods scale process systems engineering framework. Computer-Aided
for Chemical Process design. 1st ed. London: Pearson, 1997 Chemical Engineering, 2018, 44: 2209–2214
22. Yuan Z, Eden M R. Superstructure optimization of integrated fast 39. Delaunay B. On the empty sphere. Journal of Physics and Radium.
pyrolysis-gasification for production of liquid fuels and propylene. 1934, 12(7): 793–800 (in French)
AIChE Journal. American Institute of Chemical Engineers, 2016, 62 40. Žalik B. An efficient sweep-line Delaunay triangulation algorithm.
(9): 3155–3176 CAD Computer Aided Design, 2005, 37(10): 1027–1038
23. Chen Q, Grossmann I E. Recent developments and challenges in 41. Barber C B, Dobkin D P, Huhdanpaa H. The quickhull algorithm for
optimization-based process synthesis. Annual Review of Chemical convex hulls. ACM Transactions on Mathematical Software, 1996,
and Biomolecular Engineering, 2017, 8(1): 249–283 22(4): 469–483
24. Grossmann I E, Apap R M, Calfa B A, García-Herreros P, Zhang Q. 42. Al R, Behera C R, Zubov A, Gernaey K V, Sin G. Meta-modeling
Recent advances in mathematical programming techniques for the based efficient global sensitivity analysis for wastewater treatment
optimization of process systems under uncertainty. Computers & plants—an application to the BSM2 model. Computers & Chemical
Chemical Engineering, 2016, 91: 3–14 Engineering, 2019, 127: 233–246
25. Koutinas M, Kiparissides A, Pistikopoulos E N, Mantalaris A. 43. Rasmussen C E. Gaussian processes in machine learning. In:
Bioprocess systems engineering: transferring traditional process Bousquet O, von Luxburg U, Rätsch G, eds. Lecture Notes in
engineering principles to industrial biotechnology. Computational Computer Science (including subseries Lecture Notes in Artificial
and Structural Biotechnology Journal, 2012, 3(4): e201210022 Intelligence and Lecture Notes in Bioinformatics). Berlin, Heidel-
26. Bhosekar A, Ierapetritou M. Advances in surrogate based modeling, berg: Springer Verlag, 2004, 63–71
feasibility analysis, and optimization: a review. Computers & 44. Boukouvala F, Ierapetritou M G. Feasibility analysis of black-box
Chemical Engineering, 2018, 108: 250–267 processes using an adaptive sampling kriging-based method.
27. Al R, Behera C R, Gernaey K V, Sin G. Stochastic simulation-based Computers & Chemical Engineering, 2012, 36(1): 358–368
superstructure optimization framework for process synthesis and 45. Caballero J A, Grossmann I E. An algorithm for the use of surrogate
design under uncertainty. Computers & Chemical Engineering, models in modular flowsheet optimization. AIChE Journal.
2020, 143: 107118 American Institute of Chemical Engineers, 2008, 54(10): 2633–
28. Wang Z, Ierapetritou M. Constrained optimization of black-box 2650
stochastic systems using a novel feasibility enhanced kriging-based 46. Davis E, Ierapetritou M. A kriging based method for the solution of
method. Computers & Chemical Engineering, 2018, 118: 210–223 mixed-integer nonlinear programs containing black-box functions.
29. McBride K, Sundmacher K. Overview of surrogate modeling in Journal of Global Optimization, 2009, 43(2-3): 191–205
chemical process engineering. Chemieingenieurtechnik (Wein- 47. Hwangbo S, Al R, Sin G. An integrated framework for plant data-
heim), 2019, 91(3): 228–239 driven process modeling using deep-learning with Monte-Carlo
22 Front. Chem. Sci. Eng.

simulations. Computers & Chemical Engineering, 2020, 143: adventure. Chemical Engineering Science, 2017, 170: 677–693
107071 66. Da Silva S S, Chandel A K. D-Xylitol: Fermentative Production,
48. Schweidtmann A M, Mitsos A. Deterministic global optimization Application and Commercialization. Berlin Heidelberg: Springer-
with artificial neural networks embedded. Journal of Optimization Verlag, 2012
Theory and Applications, 2019, 180(3): 925–948 67. Choi S, Song C W, Shin J H, Lee S Y. Biorefineries for the
49. Henao C A, Maravelias C T. Surrogate-based superstructure production of top building block chemicals and their derivatives.
optimization framework. AIChE Journal. American Institute of Metabolic Engineering, 2015, 28: 223–239
Chemical Engineers, 2011, 57(5): 1216–1232 68. de Albuquerque T L, da Silva I J, de MacEdo G R, Rocha M V P.
50. Yeomans H, Grossmann I E. A systematic modeling framework of Biotechnological production of xylitol from lignocellulosic wastes:
superstructure optimization in process synthesis. Computers & a review. Process Biochemistry, 2014, 49(11): 1779–1789
Chemical Engineering, 1999, 23(6): 709–731 69. Venkateswar Rao L, Goli J K, Gentela J, Koti S. Bioconversion of
51. Mencarelli L, Chen Q, Pagot A, Grossmann I E. A review on lignocellulosic biomass to xylitol: an overview. Bioresource
superstructure optimization approaches in process system engineer- Technology, 2016, 213: 299–310
ing. Computers & Chemical Engineering, 2020, 136: 106808 70. Dasgupta D, Bandhu S, Adhikari D K, Ghosh D. Challenges and
52. Huster W R, Schweidtmann A M, Lüthje J T, Mitsos A. prospects of xylitol production with whole cell bio-catalysis: a
Deterministic global superstructure-based optimization of an review. Microbiological Research, 2017, 197: 9–21
organic Rankine cycle. Computers & Chemical Engineering, 71. Felipe Hernández-Pérez A, de Arruda P V, Sene L, da Silva S S,
2020, 141: 106996 Kumar Chandel A, de Almeida Felipe M G. Xylitol bioproduction:
53. Jones M, Forero-Hernandez H, Zubov A, Sarup B, Sin G. state-of-the-art, industrial paradigm shift, and opportunities for
Superstructure optimization of oleochemical processes with surro- integrated biorefineries. Critical Reviews in Biotechnology, 2019,
gate models. Computer-Aided Chemical Engineering, 2018, 44: 39(7): 924–943
277–282 72. Delgado Arcaño Y, Valmaña García O D, Mandelli D, Carvalho W
54. Misener R, Floudas C A. Piecewise-linear approximations of A, Magalhães Pontes L A. Xylitol: a review on the progress and
multidimensional functions. Journal of Optimization Theory and challenges of its production by chemical route. Catalysis Today,
Applications, 2010, 145(1): 120–147 2020, 344: 2–14
55. Misener R, Gounaris C E, Floudas C A. Global optimization of gas 73. Mountraki A D, Koutsospyros K R, Mlayah B B, Kokossis A C.
lifting operations: a comparative study of piecewise linear Selection of biorefinery routes: the case of xylitol and its integration
formulations. Industrial & Engineering Chemistry Research, 2009, with an organosolv process. Waste and Biomass Valorization, 2017,
48(13): 6098–6104 8(7): 2283–2300
56. Pistikopoulos E N. Uncertainty in process design and operations. 74. Franceschin G, Sudiro M, Ingram T, Smirnova I, Brunner G,
Computers & Chemical Engineering, 1995, 19(Suppl 1): 553–563 Bertucco A. Conversion of rye straw into fuel and xylitol: a
57. Amaran S, Sahinidis N V, Sharda B, Bury S J. Simulation technical and economical assessment based on experimental data.
optimization: a review of algorithms and applications. 4OR, 2014, Chemical Engineering Research & Design, 2011, 89(6): 631–640
12(4): 301–333 75. Giuliano A, Barletta D, De Bari I, Poletto M. Techno-economic
58. Fu M C, Price C C, Zhu J, Hillier F S. Handbook of Simulation assessment of a lignocellulosic biorefinery co-producing ethanol
Optimization Associate Series Editor. New York: Springer, 2015 and xylitol or furfural. Computer-Aided Chemical Engineering,
59. Ankenman B, Nelson B L, Staum J. Stochastic kriging for 2018, 43: 585–590
simulation metamodeling. Operations Research, 2010, 58(2): 371– 76. Mancini E, Mansouri S S, Gernaey K V, Luo J, Pinelo M. From
382 second generation feed-stocks to innovative fermentation and
60. Bertsimas D, Sim M. The price of robustness. Operations Research, downstream techniques for succinic acid production. Critical
2004, 52(1): 35–53 Reviews in Environmental Science and Technology, 2020, 50(18):
61. Ning C, You F. Optimization under uncertainty in the era of big data 1829–1873
and deep learning: when machine learning meets mathematical 77. Ragauskas A J, Beckham G T, Biddy M J, Chandra R, Chen F,
programming. Computers & Chemical Engineering, 2019, 125: Davis M F, Davison B H, Dixon R A, Gilna P, Keller M, Langan P,
434–448 Naskar A K, Saddler J N, Tschaplinski T J, Tuskan G A, Wyman C
62. Hüllen G, Zhai J, Kim S H, Sinha A, Realff M J, Boukouvala F. E. Lignin valorization: improving lignin processing in the
Managing uncertainty in data-driven simulation-based optimization. biorefinery. Science, 2014, 344(6185): 1246843
Computers & Chemical Engineering, 2020, 136: 106519 78. Ponnusamy V K, Nguyen D D, Dharmaraja J, Shobana S, Banu J R,
63. Marques C M, Moniz S, de Sousa J P, Barbosa-Póvoa A P. A Saratale R G, Chang S W, Kumar G. A review on lignin structure,
simulation-optimization approach to integrate process design and pretreatments, fermentation reactions and biorefinery potential.
planning decisions under technical and market uncertainties: a case Bioresource Technology, 2019, 271: 462–472
from the chemical-pharmaceutical industry. Computers & Chemical 79. Wang W C, Tao L. Bio-jet fuel conversion technologies. Renewable
Engineering, 2017, 106: 796–813 & Sustainable Energy Reviews, 2016, 53: 801–822
64. Crater J S, Lievense J C. Scale-up of industrial microbial processes. 80. Prunescu R M, Blanke M, Jakobsen J G, Sin G. Dynamic modeling
FEMS Microbiology Letters, 2018, 365(13): 138 and validation of a biomass hydrothermal pretreatment process—a
65. Noorman H J, Heijnen J J. Biochemical engineering’s grand demonstration scale study. AIChE Journal. American Institute of
 Nikolaus I. Vollmer et al. Synergistic optimization framework 23

Chemical Engineers, 2015, 61(12): 4235–4250 Chemical Engineering, 2014, 68: 220–232
81. Tochampa W, Sirisansaneeyakul S, Vanichsriratana W, Srinopha- 87. Garud S S, Karimi I A, Kraft M. Smart sampling algorithm for
kun P, Bakker H H C, Chisti Y. A model of xylitol production by the surrogate model development. Computers & Chemical Engineering,
yeast Candida mogii. Bioprocess and Biosystems Engineering, 2017, 96: 103–114
2005, 28(3): 175–183 88. Garud S S, Karimi I A, Brownbridge G P E, Kraft M. Evaluating
82. S3O GitHub Repository. 2021, 10.5281/zenodo.5017353 smart sampling for constructing multidimensional surrogate models.
83. Al R, Behera C R, Gernaey K V, Sin G. Towards development of a Computers & Chemical Engineering, 2018, 108: 276–288
decision support tool for conceptual design of wastewater treatment 89. Obermeier A, Vollmer N, Windmeier C, Esche E, Repke J U.
plants using stochastic simulation optimization. Computer-Aided Generation of linear-based surrogate models from non-linear
Chemical Engineering, 2019, 46: 325–330 functional relationships for use in scheduling formulation. Compu-
84. Kılınç M R, Sahinidis N V. Exploiting integrality in the global ters & Chemical Engineering, 2021, 146: 107203
optimization of mixed-integer nonlinear programming problems 90. Chen Y, Goetsch P, Hoque M A, Lu J, Tarkoma S. d-Simplexed:
with BARON. Optimization Methods & Software, 2018, 33(3): adaptive delaunay triangulation for performance modeling and
540–562 prediction on big data analytics. IEEE Transactions on Big Data,
85. Vassilev S V, Baxter D, Andersen L K, Vassileva C G, Morgan T J. 2019, in press
An overview of the organic and inorganic phase composition of 91. Jiang P, Zhang Y, Zhou Q, Shao X, Hu J, Shu L. An adaptive
biomass. Fuel, 2012, 94: 1–33 sampling strategy for kriging metamodel based on Delaunay
86. Eason J, Cremaschi S. Adaptive sequential sampling for surrogate triangulation and TOPSIS. Applied Intelligence, 2018, 48(6):
model generation with artificial neural networks. Computers & 1644–1645