Expert Systems With Applications 242 (2024) 122797

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Robust optimization of geoenergy production using data-driven deep

recurrent auto-encoder and fully-connected neural network proxy
Cong Xiao a,b ,∗, Shicheng Zhang a,b , Yuanlin Hu c , Xiaolong Gu d , Xingfang Ma a,b , Tong Zhou e ,
Jianbing Jin f
a
Key Laboratory of Petroleum Engineering, Ministry of Education, China University of Petroleum, Beijing, China
b College of Petroleum Engineering, China University of Petroleum, Beijing, China
c Qinghai Oilfield Downhole Operation Company, Gansu, China
d Karamay Gas Co., Ltd, Xinjiang, China
e
Research Institute of Petroleum Exploration and Production, SINOPEC, Beijing, China
f
Jiangsu Key Laboratory of Atmospheric Environment Monitoring and Pollution Control, Jiangsu Collaborative Innovation Center of Atmospheric Environment and
Equipment Technology, School of Environmental Science and Engineering, University of Information Science and Technology, Nanjing, Jiangsu, China

ARTICLE INFO ABSTRACT

Keywords: Robust and efficient optimization of post-history well production schedule under history-matched geomodel
Geoenergy recovery known as closed-loop production management is crucial to achieve reasonable and reliable decision-making
Production forecasting for subsurface geoenergy resources (e.g., oil/gas resources, geothermal, natural gas hydrate, etc.) recovery
Particle-swarm optimization
in petroleum industry. The procedure generally consists of a cycling implementation of geomodel history
Deep-learning surrogate
matching and production schedule optimization through repeatedly simulating high-fidelity reservoir models
Robust optimization
and therefore needs almost infeasible computation cost particularly for field-scale oil production scenario.
Through hybridizing deep recurrent neural network and particle-swarm optimization algorithm, we present a
novel data-driven computation framework for robust post-history production optimization without explicitly
performing history matching. In the proposed framework, we develop a hybrid neural network proxy model,
which combines recurrent autoencoder (RAE) and fully-connected neural network (FCNN), referred to as
RAE-FCNN. The RAE structure can extract latent information from time-series history data, and the FCNN
structure can map the combination of extracted latent variables by RAE and post-history well-control sequence
to net present value (NPV). The trained RAE-FCNN proxy model can predict NPV values corresponding to
the history measurements and user-specific post-history well controls. In the robust optimization framework,
the averaged NPV over an ensemble of noisy history measurements are regarded as the objective function
while the post-history well-controls are chosen as the decision variables. The proposed surrogate-based
optimization framework has been demonstrated on two waterflooding reservoir production cases, e.g., a 2D
synthetic Gaussian model and a 3D benchmaRk channelized EGG model.The decision variables considered
in optimization are the injection rates with linear bound constraints at the injectors. In comparison with
the optimization results by high-fidelity reservoir models, solely 1000 high-fidelity reservoir models for both
cases are sufficient to train accurate proxy model and achieve satisfactory post-history production optimization
results. Our proposed deep-learning-based optimization method can effectively make full use of history data to
predict post-history oil well production without the need of computationally expensive history matching step,
which enables us to achieve quick decision-making and risk evaluation for close-loop waterflooding reservoir
production management and optimization.

1. Introduction timization methods to improve the cumulative oil production or

economic net present value of the oilfield (Capolei, Völcker, Frydendall,
Subsurface geoenergy energy resources (e.g., oil/gas resources, & Jorgensen, 2012; Zhao, Zhang, Cao, Kang, & Xie, 2019). Reservoir
geothermal, natural gas hydrate, etc.) production optimization gen- simulation is an important part of production optimization, and its
erally demands numerous reservoir simulations and numerical op-

∗ Corresponding author.
E-mail addresses: [email protected] (C. Xiao), [email protected] (S. Zhang), [email protected] (Y. Hu), [email protected] (X. Gu),
[email protected] (X. Ma), [email protected] (T. Zhou), [email protected] (J. Jin).

https://doi.org/10.1016/j.eswa.2023.122797
Received 5 June 2023; Received in revised form 19 November 2023; Accepted 28 November 2023
Available online 30 November 2023
0957-4174/© 2023 Elsevier Ltd. All rights reserved.
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

simulation results are used to evaluate the objective functions. The robust production optimization is generally performed based
However, evaluation of the objective functions needs to run the nu- on an ensemble of history-matched geomodels. The reservoir his-
merical simulations, which greatly affects the efficiency of well control tory matching is resolved by adjoint/gradient-based optimization or
optimization. Although the traditional optimization method can also ensemble-based data assimilation methods (Emerick & Reynolds, 2012,
achieve good results, when dealing with complex and large-scale reser- 2013), during which a large number of high-fidelity model simulations
voir models, the optimization convergence speed is slow, and the are required and thus leads to intensive computation-cost, especially
workload is particularly cumbersome. for the field-scale applications (Bukshtynov, Volkov, Durlofsky, & Aziz,
In general, derivative-free optimization and derivative/gradient- 2015). To accelerate the reservoir history matching step or production
based optimization are two of the most popular approaches for max- optimization step, many deep-learning surrogate models have been
imizing waterflooding production (Goda & Sato, 2013, 2014; Ma, Kim, recently proposed in the literature (Kim, Lee, & Choe, 2021; Lu,
Volkov and Durlofsky, 2022; Zeng, Nie, & Yang, 2021). The gradient Forouzanfar, & Reynolds, 2017). For example, Li et al. (2022), Ma,
computation of defined objective function with respect to the well Zhang, Wang et al. (2022) and Ma, Zhang, Zhang et al. (2022) proposed
controls is crucial for gradient-based optimization algorithms. The gra- surrogate-based history matching approaches based on the recurrent
dient can be computed using either numerical methods (deterministic neural network. The convolutional neural network is used to extract
gradient, e.g., finite different, or stochastic gradient, e.g., SPSA (Pouladi spatial features of the geomodels, and these compressed features are
et al., 2020) and EnOpt (Fonseca, Leeuwenburgh, Van den Hof, & then fed to recurrent neural network for predicting time-series well
Jansen, 2015)) or analytical adjoint methods. In contrast, derivative- responses, e.g., oil/water/gas production rate. This trained RNN-based
free methods treat simulators as black boxes and rely on advanced proxy model is integrated with any optimizers, e.g., particle-swarm
evolutionary optimizers, e.g., particle swarm optimization, differential optimization, differential evolution and ensemble-based data assim-
evolutionary and genetic algorithm (Chen, Zhang, Zhao, & Liu, 2023; ilation, to perform reservoir history matching. Kim and Durlofsky
Du et al., 2023; Xiao, Lin, Leeuwenburgh, & Heemink, 2022). (2021) proposed a deep-learning-based well-control optimization using
The derivative-free optimization and derivative/gradient-based op- a recurrent neural network model. The recurrent neural network takes
timization methods demand us to repeatedly run numerous reservoir the time-series well controls as inputs and predicts time-series well
simulations. To mitigate the intensive computation cost, a large number responses. The proxy model is then combined with particle-swarm
of surrogate models has been employed to accelerate the production optimizer to resolve the well control optimization with nonlinear
optimization problems (Atadeger, Onur, Sheth, & Banerjee, 2023; Chen constraints.
et al., 2023; Du et al., 2023; Kuk, Stopa, Kuk, Janiga, & Wojnarowski, The aforementioned recurrent neural network proxies are solely
2021; Li, Wang, Dong, Wang and Qu, 2022; Wang, 2001; Xiao et al., used to handle history matching or production optimization that the
2022). For example, Xiong, Kim, and Fu (2020) proposed a Gaussian- geomodels or well-controls are assumed to be known beforehand and
process regression surrogate model with dimension reduction method. hence are not suitable for close-loop production optimization problems.
Through compressing the original high-dimensional decision variables Recently, the use of recurrent neural network to speedup close-loop
into low dimension subspace, an accurate proxy model can be ob- production optimization also has been preliminarily investigated. For
tained by a limited number training samples. Chen, Zhang, Xue, Zhang, example, Wang, Chang, Kong, and Zhang (2023) proposed a learning
and Yang (2019) proposed a proxy-assisted collaborative group op- surrogate model to address closed-loop optimization of geothermal
timization algorithm, in which two proxy models are constructed to reservoir development. Different from the previous deep-learning sur-
synergistically find the global optimum. Chen et al. (2020) developed rogate models for solely history matching or production optimization,
a novel hierarchical proxy-based PSO optimization framework that both the geomodels and well controls are regarded as the inputs.
combines a global-screening step and a local-searching step. Chen et al. The convolutional autoencoder–decoder and recurrent neural network
(2021) proposed a similar hierarchical surrogate-assisted evolutionary are employed to extract the spatial geomodel features and tempo-
optimization algorithm with differential evolutionary as the optimizer. ral well-control features, respectively. This hybrid recurrent neural
Surrogate-assisted evolutionary algorithms, especially the local–global network can predict the time-varying well responses corresponding
hybrid hierarchical proxy-based evolutionary algorithms, should be to different combinations of geomodel and well control sequences.
very promising in accelerating reservoir production optimization ap- In the closed-loop optimization framework, the history matching and
plications (Xue et al., 2022; Zhong et al., 2022). In addition, the production optimization steps are continuously performed based on
kernel-based machine-learning-based methods (such as Support-Vector the proxy-based iterative ensemble smoother and differential evolu-
regression or Gaussian-Process regression) also have been employed tionary algorithm. Similarly, Kim and Durlofsky (2023) also presented
to assist the production optimization problems in the community of a deep-learning proxy for robust optimization under geomodel un-
petroleum engineering (Almasov, Nguyen, & Onur, 2022; Almasov & certainties. Strictly speaking, their approach does not actually realize
Onur, 2021, 2022; Guo et al., 2018; Guo & Reynolds, 2018; Liu & the close-loop optimization. They assume that the geomodels have
Reynolds, 2021). been obtained by the conventional history matching step. And then
The above optimization algorithms are mainly used to resolve the the history-matched geomodels and well controls are regarded as the
deterministic production optimization problem, where the effects of inputs of hybrid convolutional-recurrent neural network. The trained
history measurements are not considered or the geomodels provided by deep-learning proxy is integrated with particle swarm optimization to
the history matching step are assumed to be deterministic. Hence, these optimize the well controls correspondingly.
methods cannot consider the all steps of the closed-loop production Although both the reservoir history matching and production op-
optimization including the history matching steps. In general, incorpo- timization have been simultaneously accelerated by the above deep-
rating geomodel uncertainties into production optimization procedure learning proxy models, the inherent issue related to the history match-
is referred to as the robust production optimization (Hou, Zhou, Zhang, ing remains. Some reparameterization techniques such as
Kang, & Xie, 2015). Both deterministic and robust optimization are Hough-transformation (Yao, Chang, Li, & Zhang, 2018), level-set
based on the geomodels obtained by history-matching step, which can (Chang, Zhang, & Lu, 2010), optimization-based principle compo-
be referred as to closed-loop production optimization. This process nent analysis (Vo & Durlofsky, 2015, 2016), and convolutional-neural-
consists of alternative conduction of simulation-based history matching network based principle component analysis (Liu, Sun, & Durlofsky,
and optimization procedure in a cycling manner, which results in 2018), have been proposed to characterize complex geological features
almost infeasible and computationally intensive optimization of well- such as non-Gaussian facies models (Chen, Gao, Ramirez, Vink, Gi-
controls accurately and efficiently in practical applications (Hou et al., rardi, et al., 2015), multi-Gaussian channelized models, and discrete
2015; Silva, Emerick, Couto, & Alves, 2017). fracture systems (Lu & Zhang, 2015), obtaining accurate geomodels

2
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

through history matching remains a challenging task. The poor qual- assimilation algorithm, e.g., ensemble smoother with multiple data as-
ity of history-matched geomodels can result in unreliable production similation. The implementation details related to the data assimilation
optimization. algorithm can be referred to Emerick and Reynolds (2012, 2013). For
It is urgent to efficiently perform close-loop production optimization the second step, the well controls are optimized based on the history-
without actually performing history matching procedure. Recently, the matched geomodels. We should emphasize that to simplify the notation
data-space inversion (DSI) and its variations can predict post-history the 𝐦 denotes the history-matched geomodel in the following parts.
well responses without performing history matching (Lima, Emerick, The aim of production optimization problem is to maximize the net
& Ortiz, 2020; Sun & Durlofsky, 2017). Sun and Durlofsky (2019) present value (NPV) through adjusting the well controls. Considering
and Sun, Hui, and Durlofsky (2017) applied DSI approaches to quan- the aforementioned oil-water two-phase system, the NPV is defined as
tify post-history well production for complex fractured reservoirs and 𝑁𝑇 𝑁𝑃 𝑁𝐼
∑ ∑ ∑
carbon dioxide storage projects. Liu, Rao, Zhao, Xu, and Gong (2021) 𝑁𝑃 𝑉 (𝐔) = 𝑛
[ (𝑟𝑜 𝑞𝑜,𝑖 (𝐮𝑛 , 𝐦) − 𝑟𝑤𝑝 𝑞𝑤𝑝,𝑖
𝑛
(𝐮𝑛 , 𝐦)) − 𝑛
𝑟𝑤𝑖 𝑞𝑤𝑖,𝑗 (𝐮𝑛 , 𝐦)]
employed DSI to predict post-history saturation and pressure fields. 𝑛=1 𝑖=1 𝑗=1
Subsequently, Jiang, Sun, and Durlofsky (2020) proposed a modified 𝛥𝑡𝑛
× (2)
version of DSI (e.g., data-space inversion with varying control) to (1 + 𝑏)𝑡𝑛
predict post-history production dynamic corresponding to specific well-
where, 𝐔 = [𝐮1 , … , 𝐮𝑛 , … , 𝐮𝑁𝑇 ]𝑇 is a vectorized well controls for all
controls during close-loop reservoir management. Recently, Xiao et al.
wells. 𝐘 = [𝐲1 , … , 𝐲𝑛 , … , 𝐲𝑁𝑇 ]𝑇 . 𝑁 𝑃 and 𝑁 𝐼 denote the total number of
(2023) proposed a generic data-driven post-history production forecast- 𝑛 and 𝑞 𝑛 , respectively, denote
producers and injectors, respectively. 𝑞𝑜,𝑖 𝑤𝑝,𝑖
ing framework using a hybrid recurrent neural network proxy through
combining recurrent autoencoder with long short-term memory neural the oil and water production rate (STB/day) of the 𝑖th producers at the
𝑛 , respectively, denote the water injection rate (STB/day)
timestep 𝑛. 𝑞𝑤𝑖,𝑗
network.
of the 𝑗th injectors at the timestep 𝑛. 𝑞𝑜,𝑖 𝑛 and 𝑞 𝑛 𝑛
This paper presented an efficient alternative that directly imple- 𝑤𝑝,𝑖 and 𝑞𝑤𝑖,𝑗 are the
ments the production forecast and optimization of well-controls with- similar notations as the 𝐲𝑛 , e.g., see Eq. (1).
out the explicit implementation of history matching process. Specifi- Comparing to the deterministic production optimization mentioned
cally, we propose an efficient and robust data-driven computational above, to account for geological uncertainties during optimization, it
framework for post-history production forecasting and optimization is recommended to use a robust production optimization method that
through hybridizing deep recurrent neural network and particle-swarm assesses the objective function’s average value across a range of noisy
optimization algorithm. The remainder of this paper is organized as measurements. This can be achieved by defining the final objective
follows. Formula of the deterministic and/or robust waterflooding pro- function as the average net present value (NPV) across 𝑁𝑒 posterior
duction optimizations is described in Section 2. Section 3 introduces geomodels.
a novel hybrid recurrent neural network and proxy-based PSO algo- [
𝑁𝑒 𝑁𝑇 𝑁𝑃
1 ∑ ∑∑
rithm for post-history production forecasting and optimization. Sec- 𝐸𝑁𝑃 𝑉 (𝐔) = [ (𝑟 𝑞 𝑛 (𝐮𝑛 , 𝐦𝑘 ) − 𝑟𝑤𝑝 𝑞𝑤𝑝,𝑖
𝑛
(𝐮𝑛 , 𝐦𝑘 ))
𝑁𝑒 𝑘=1 𝑛=1 𝑖=1 𝑜 𝑜,𝑖
tion 4 discusses and evaluates some numerical results of synthetic
𝑁𝐼
]
and benchmark waterflooding petroleum production cases using our ∑ 𝛥𝑡𝑛
𝑛
considered optimization algorithms. Finally, Section 5 summarizes our − 𝑟𝑤𝑖 𝑞𝑤𝑖,𝑗 (𝐮𝑛 , 𝐦𝑘 )] (3)
𝑗=1
(1 + 𝑏)𝑡𝑛
contributions and discusses the future work.
where, 𝐦𝑘 represents the 𝑘th posterior or history-matched geomodel
2. Formula of closed-loop waterflooding production optimization from the ensemble-based history matching step.
In general, the well-controls cannot be adjusted infinitely (Yao,
In the community of petroleum engineering, after the primary pro- Wei, Zhang, & Jiang, 2012). The deterministic and/or robust pro-
duction stage, injecting water into the formation through injection duction optimizations are usually subject to some constraints. The
wells can supplement formation energy, thereby improving crude oil bound-constrained production optimization can be represented as
recovery rate. The optimization of production system can extract more
crude oil from the formation by adjusting the injection and produc- 𝐔 = arg max 𝑁𝑃 𝑉 (𝐔) 𝑜𝑟 arg max 𝐸𝑁𝑃 𝑉 (𝐔)
𝐔 𝐔
tion measures of injection wells and production wells. Water injection [𝐪1𝑜 , … , 𝐪𝑛𝑜 , … ; 𝐪1𝑤𝑝 , … , 𝐪𝑛𝑤𝑝 , … ; 𝐪1𝑤𝑖 , … , 𝐪𝑛𝑤𝑖 , …] = 𝑓 (𝐔, 𝐦),
displacement of crude oil is a process of oil-water two-phase flow, the
dynamic equations for a two-phase oil-water system can be referred 𝑛 = 1, … , 𝑁𝑇
to Mattax and Dalton (1990). For the sake of simplicity, we will provide 𝐔 = [𝐮1 , 𝐮2 , … , 𝐮𝑛 , … , 𝐮𝑁𝑇 ]
a direct and explicit formula that describes the response of a well as
𝑢𝑙𝑜𝑤 ⩽ 𝑢𝑛 ⩽ 𝑢𝑢𝑝 , 𝑛 = 1, … , 𝑁𝑇 (4)
a function of its well controls. A nonlinear operator that represents
the mathematical relationship between the well controls 𝐔 and the where, the scalar 𝑢𝑙𝑜𝑤 and 𝑢𝑢𝑝 represent the upper and lower bounds
simulated well response 𝐘 is as follows for the well controls, respectively. 𝑢𝑛 denotes one element of vector for
well controls 𝐮𝑛 .
𝐘 = 𝑓 (𝐔, 𝐦) (1)
The generic closed-loop waterflooding production optimization pro-
where, 𝐔 = [𝐮1 , 𝐮2 , . . . , 𝐮𝑛 , . . . , 𝐮𝑁𝑇 ] and 𝐘 = [𝐲1 , … , 𝐲𝑛 , … , 𝐲𝑁𝑇 ]. 𝐮𝑛 cedure generally performs numerous high-fidelity reservoir simula-
∈ 𝑅𝑁𝑢 represents the well-control sequence, e.g., bottom-hole pressure tions and therefore presents intensive computation cost particularly for
and injection/production rate. 𝐲𝑛 ∈ 𝑅𝑁𝑑 denotes the liquid rate for field-scale applications. Besides, the difficulty related to the accurate
producers and water injection rate for injectors. 𝑁𝑑 is the number of calibration of complex geomodels still remains. The issues regarding
simulated results, e.g., oil/water rate or water injection rate at each the computation efficiency and geomodel calibration accuracy moti-
timestep. 𝑁𝑢 is the number of well-controls at each timestep. 𝑁𝑇 is vate us to develop an alternative closed-loop production optimization
the number of simulation steps. 𝐦 denotes the geomodel parameters, strategy that performs the well controls optimization without actually
e.g., rock permeability and porosity. performing history matching step.
As noted earlier, the closed-loop waterflooding production opti-
mization consists of two continuous steps: reservoir history matching 3. Methodology
and well-control optimization. For the first step, the geomodel pa-
rameters 𝐦 are updated through assimilating history measurements. In this section, the process of utilizing a deep learning model to
This step generally can be performed by use of ensemble-based data speedup the optimization of waterflooding production is presented.

3
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

This process encompasses the architecture of the neural network, the

̂ = 𝜱𝐹 𝐶𝑁𝑁 (𝝍 ℎ , 𝐮𝑓 , 𝜽𝑓 𝑐𝑛𝑛 )
𝐹 𝐶𝑁𝑁 ∶ 𝐍𝐏𝐕 (9)
generation of training samples, the training of the network, and the
implementation of the proxy-based production optimization algorithm. where, 𝝍 ℎ represents the compressed latent variables for history data
The conventional approach to history matching using model-based
𝐝ℎ . 𝐝̂ℎ is the reconstructed history data by decoder part of RAE struc-
techniques involves using history measurements to generate posterior ̂ denotes the predicted net present value by FCNN structure.
ture. 𝐍𝐏𝐕
geomodels, which are then used to predict post-history well production.
The 𝐘 represents a vector of the time-series well production responses 𝜽𝑒𝑛𝑐 , 𝜽𝑓 𝑐𝑛𝑛 , and 𝜽𝑑𝑒𝑐 denote all trainable network parameters for the
corresponding to one specific reservoir simulation. This vector contains encoder, FCNN, and decoder units, respectively.
the history prediction 𝐘ℎ and post-history prediction 𝐘𝑓 , that is, 𝐘 = Our proposed RAE-FCNN is a type of hybrid recurrent convolutional
[𝐘ℎ , 𝐘𝑓 ]. neural network model. To satisfy our purpose, we apply RAE-FCNN
The results of reservoir simulation are influenced by the history proxy model to address post-history forecast and well-control optimiza-
well-controls and post-history well-controls (e.g., 𝐔 = [𝐔ℎ , 𝐔𝑓 ]), such tion. A schematic diagram of the RAE-FCNN architecture is presented
as the pressure at the bottom of the well and/or the rate of injection, in Fig. 1. As shown in Fig. 1, the proposed RAE-FCNN surrogate
as well as the geomodels 𝐦. These can be expressed in the following model consists of a RAE-based time-series feature compression module
manner. and a FCNN-based prediction module, which will be described in the
𝐘𝑖 = 𝑓 (𝐦𝑖 , 𝐔𝑖 ), 𝑖 = 1, 2, … , 𝑁𝑠 (5) following parts.
The use of recurrent convolutional neural network proxy to acceler-
where, 𝑁𝑠 is the number of geomodel realizations. The well controls
𝐔𝑖 and geomodel 𝐦𝑖 (i = 1,2, . . . , 𝑁𝑠 ) can be generated randomly. ate reservoir flow simulations can be found in the literature (Atadeger,
In general, the predetermined statistical distributions are assumed for Sheth, Vera, Banerjee, & Onur, 2022; Xiao, Leeuwenburgh, Lin, &
both well controls and geomodels. Note that the history well-controls Heemink, 2021; Xiao et al., 2022). For example, Jin, Liu, and Durlofsky
should be specific beforehand and only the post-history well-controls (2020) proposed a data-driven autoregressive surrogate model (embed
can be varying corresponding to different geomodel realizations, that to control-based methods, e.g., E2C) for emulating reservoir simulation
is, 𝐔𝑖 = [𝐔ℎ , 𝐔𝑖𝑓 ]). For instance, the 𝐔𝑖 can be generated from a problem. Tang, Liu, and Durlofsky (2020) developed a combination
uniform distribution. Furthermore, the well controls typically satisfy of auto-encoder and RNN-type convolutional long short term memory
some constraints, e.g., see Eq. (4). To obtain an ensemble of prediction recurrent network (convLSTM) to simulate the dynamic of subsurface
results 𝐘𝑖 , forward simulations are run for the (𝐦𝑖 , 𝐔𝑖 ) pairs. Also note flow simulation process. Both E2C-based methods and RNN can be
that the well-control 𝐔ℎ for history production stage should be specific. used to address time-series prediction problems. In the community of
Given the consideration of well control variance, the complete dataset,
petroleum engineering, the E2C method takes the pressure, saturation
which includes well controls, historical predictions, and post-history
and well controls at the current step as inputs to predict the saturation
predictions, can be expressed as follows
and pressure for the next step, which can be further used to calculate
𝐘𝑖 = [𝐔ℎ , 𝐔𝑖𝑓 , 𝐘𝑖ℎ , 𝐘𝑖𝑓 ], 𝑖 = 1, 2, … , 𝑁𝑠 (6) the well response. The RNN-type proxies solely consider the time-series
The succeeding sections will provide comprehensive explanations well-controls as inputs and then directly predict well responses, e.g., oil
on how to efficiently perform well-control optimization using a hybrid and water rate. The RNN is not explicitly necessary to predict pressure
deep recurrent auto-encoder and fully-connected neural network proxy and phase saturation. In general, the RNN should be faster than that
model. The proposed neural network proxy model serves as a nonlinear of E2C method especially for the field-scale application models due to
enhancement to consider the variability of well controls. the nature of model-physics non-intrusion. However, the E2C method
should be more physical consistency due to the predictions of saturation
3.1. Hybrid recurrent auto-encoder and fully-connected neural network and pressure.
proxy: RAE-FCNN Long Short-Term Memory (LSTM) Neural Network. The LSTM is
generally used to handle time-series data. as a variant of conventional
For the conventional closed-loop production optimization, given the
recurrent neural network, the LSTM has been widely used for various
specific history well-controls 𝐮ℎ ∈ 𝑅𝑛𝑢ℎ , the observed data is used to
update the geomodel parameters 𝐦. Based on the updated geomodel, application domains, such as natural language processing (Qing-Dao-
the post-history well-controls 𝐮𝑓 ∈ 𝑅𝑛𝑢𝑓 are optimized by maximizing Er-Ji, Su, & Liu, 2020; Xiao, Chang, Zhang, & Liu, 2020), computer
the NPV values. We can generally assume that 𝐝 denotes a forecast of vision (Yang, Singh, Tavakkoli, Amiri, & Rai, 2020), and atmosphere
simulating reservoir model, this vector constitutes the prediction data dynamic prediction (Inapakurthi, Miriyala, & Mitra, 2021; Sani, 2020).
𝐝ℎ ∈ 𝑅𝑛𝑑ℎ from the history production stage with time sequence 𝑡ℎ As shown in Fig. 2, the LSTM cell includes three gate-control units and
and the prediction 𝑁𝑃 𝑉 value at the post-history production stage one cell state. The input vector 𝐱𝑡 , the short-term hidden state 𝐡𝑡 , and
with time sequence 𝑡𝑓 . In general, both 𝐝ℎ and 𝑁𝑃 𝑉 should satisfy the long-term cell state 𝐜𝑡 are fed to the LSTM cell to generate the
nonlinear functional relationships with the geomodel parameters 𝐦, output 𝐲𝑡 . The vectors 𝐠𝑡 , 𝐟𝑡 , 𝐢𝑡 , and 𝐨𝑡 can be as follows:
history well-controls 𝐮ℎ and post-history well-controls 𝐮𝑓 .
In this study, we propose a hybrid recurrent neural network sur- 𝐠𝑡 = 𝑣(𝐖𝑇𝑥𝑔 𝐱𝑡 + 𝐖𝑇ℎ𝑔 𝐡𝑡−1 + 𝐛𝑔 )
rogate, e.g., RAE-FCNN, to predict 𝐍𝐏𝐕 values given the history data 𝐟𝑡 = 𝜎(𝐖𝑇𝑥𝑓 𝐱𝑡 + 𝐖𝑇ℎ𝑓 𝐡𝑡−1 + 𝐛𝑓 )
𝐝ℎ and post-history well control 𝐮𝑓 . The general formula of RAE-FCNN
model can be presented as follows, 𝐢𝑡 = 𝜎(𝐖𝑇𝑥𝑖 𝐱𝑡 + 𝐖𝑇ℎ𝑖 𝐡𝑡−1 + 𝐛𝑖 )

𝑁𝑃 𝑉 = 𝐟𝑅𝐴𝐸−𝐹 𝐶𝑁𝑁 (𝐝ℎ , 𝐮𝑓 ) (7) 𝐨𝑡 = 𝜎(𝐖𝑇𝑥𝑜 𝐱𝑡 + 𝐖𝑇ℎ𝑜 𝐡𝑡−1 + 𝐛𝑜 ) (10)

where, 𝑣 denotes the nonlinear function, 𝜎 is a 𝑆𝑖𝑔𝑚𝑜𝑖𝑑 activation

3.2. Description of RAE-FCNN structure
function. The matrices 𝐖𝑥𝑔 , 𝐖𝑥𝑓 , 𝐖𝑥𝑖 and 𝐖𝑥𝑜 are the weight matrices
The RAE-FCNN is comprised of two parts, including the recurrent of four neural network layers that process the input 𝐱𝑡 . The matrices
autoencoder (RAE) and fully-connected neural network (FCNN). The 𝐖ℎ𝑔 , 𝐖ℎ𝑓 , 𝐖ℎ𝑖 and 𝐖ℎ𝑜 are the weight matrices of four neural network
RAE structure can extract time-series information of history data and layers that process the short-term state 𝐡𝑡 . The vectors 𝐛𝑔 , 𝐛𝑓 , 𝐛𝑖 and
the FCNN structure can approximate extracted time-series information 𝐛𝑜 are the bias coefficients for the four layers.
and post-history well-control sequence to NPV values. The RAE and In addition, the output gate processes the updated long-term state
FCNN structure can be presented as follows: 𝐜𝑡 and the output vector 𝐨𝑡 to produce the updated short-term state 𝐡𝑡 .
𝑅𝐴𝐸 ∶ 𝝍 ℎ = 𝜱𝑒𝑛𝑐 (𝐝ℎ , 𝜽𝑒𝑛𝑐 ), 𝐝̂ℎ = 𝜱𝑑𝑒𝑐 (𝝍 ℎ , 𝜽𝑑𝑒𝑐 ) (8) Finally, we can process the updated short-term state 𝐡𝑡 using a fully

4
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 1. Architecture of the proposed RAE-FCNN surrogate model.

Fig. 2. The cell structure of LSTM structure.

connected layer of linear neurons to generate the final output vector encoder part extracts the low-dimensional latent information from the
𝐲𝑡 . All these steps can be presented as follows original time-series history data. The LSTM will regard the history data
𝑑ℎ𝑡 (e.g., oil/water/gas rate and bottom-hole pressure) as the input at
𝐜𝑡 = 𝐟𝑡 ⊗ 𝐜𝑡−1 + 𝐢𝑡 ⊗ 𝐠𝑡
each time step 𝑡 = 1, 2, . . . , 𝑡ℎ . The time-series hidden states 𝐡𝑡 , 𝑡 =
𝐡𝑡 = 𝐨𝑡 ⊗ 𝑢(𝐜𝑡 ) 1, 2, . . . , 𝑡ℎ , produced by the LSTM layers are further compressed to
𝐲𝑡 = 𝐖𝑇𝑜𝑢𝑡 𝐡𝑡 + 𝐛𝑜𝑢𝑡 (11) the latent space 𝐳 ∈ 𝑅𝑛𝑧 by a fully-connected layer. For the decoder
part, a fully-connected layer is firstly used to achieve the time-series
where, ⊗ denotes an element-wise multiplication, 𝐖𝑜𝑢𝑟 and 𝐛𝑜𝑢𝑡 are the hidden state 𝐡𝑡 , 𝑡 = 1, 2, . . . , 𝑡ℎ , which can be further fed to LSTM
weight matrices and the bias terms for the output layer, respectively. layers for mapping the latent variables 𝐳 back to the original time-series
𝐑𝐀𝐄-𝐛𝐚𝐬𝐞𝐝 𝐭𝐢𝐦𝐞-𝐬𝐞𝐫𝐢𝐞𝐬 𝐟 𝐞𝐚𝐭𝐮𝐫𝐞 𝐜𝐨𝐦𝐩𝐫𝐞𝐬𝐬𝐢𝐨𝐧 𝐦𝐨𝐝𝐮𝐥𝐞 The recurrent history data 𝑑ℎ . The structure of recurrent autoencoder has been shown
autoencoder referred to as RAE has been widely used for time-series in Fig. 3(a).
data representation and compression in the community of computer 𝐅𝐂𝐍𝐍-𝐛𝐚𝐬𝐞𝐝 𝐩𝐫𝐞𝐝𝐢𝐜𝐭𝐢𝐨𝐧 𝐦𝐨𝐝𝐮𝐥𝐞 The FCNN-based prediction module
vision. In the community of petroleum engineering, Lima et al. (2020) architecture is shown in Fig. 3(b). A dense layer is initially used to
employed RAE to compress the time-series well data (e.g., compress the post-history well controls for each time step 𝑡 = 1, 2, . . . ,
oil/water/gas rate and bottom-hole pressure). Fig. 3(a) illustrates the 𝑡𝑓 , into a latent vector 𝑢𝑙 . This latent vector 𝑢𝑙 is then combined with the
RAE structure. In general, the autoencoder structure is mostly used latent space variable 𝐳 achieved from the encoder part of RAE module.
to compress the imaging-type spatial features, while the recurrent The combined vector [𝐳, 𝑢𝑙 ] will be fed to a new dense layer, e.g., see
autoencoder can be used to compress the time-series well data. The the green rectangle in Fig. 3(b), to produce the NPV values 𝐍𝐏𝐕.

5
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 3. Illustration of RAE-based time-series feature compression module and FCNN-based prediction module.

3.3. Dataset preparation and network training 2019) and the 𝐴𝑑𝑎𝑚 optimizer (Li, Hariri-Ardebili, Deng, Wei, & Cao,
2023) is utilized for training the neural network model. The entire
Through simulating high-fidelity models corresponding to a number training process is carried out in an open source high-performance ma-
of well-controls (i.e., 𝐮ℎ and 𝐮𝑓 ) and geomodel parameters 𝐦, we can chine learning package. The dataset is divided into training, validation,
obtain a set of samples to train the RAE-FCNN proxy model. These and testing samples to effectively train the machine-learning models.
simulated results can be further re-arranged as {𝐦𝑖 , 𝐮ℎ , 𝐮𝑖𝑓 ); 𝐍𝐏𝐕𝑖 } The 𝐴𝑑𝑎𝑚 stochastic optimization algorithm is used to optimize the
for the 𝑖th training sample (𝑖 = 1, …, 𝑁𝑠 ). The loss function integrating model parameters with the training data. Hyper-parameter optimiza-
the RAE and FCNN can be expressed as: tion is achieved using the Bayesian optimization algorithm (Nguyen,
𝐿𝑜𝑠𝑠 = 𝐿𝑟𝑎𝑒 (𝜽𝑒𝑛𝑐 , 𝜽𝑑𝑒𝑐 ) + 𝐿𝑓 𝑐𝑛𝑛 (𝜽𝑓 𝑐𝑛𝑛 ) (12) 2019) and validation samples to obtain the optimal machine-learning
model structure. Finally, the model error can be measured using the
where testing sample.
𝑁𝑠
1 ∑ 𝑖
𝐿𝑟𝑎𝑒 (𝜽𝑒𝑛𝑐 , 𝜽𝑑𝑒𝑐 ) = ‖𝐝 − 𝜱𝑑𝑒𝑐 (𝜱𝑒𝑛𝑐 (𝐝𝑖ℎ , 𝜽𝑒𝑛𝑐 ), 𝜽𝑑𝑒𝑐 )‖22
𝑁𝑠 𝑖=1 ℎ 3.4. RAE-FCNN proxy-based forecast and optimization under geological
𝑁𝑠 uncertainty
1 ∑
𝐿𝑓 𝑐𝑛𝑛 (𝜽𝑓 𝑐𝑛𝑛 ) = ‖𝐍𝐏𝐕𝑖 − 𝜱𝐹 𝐶𝑁𝑁 (𝝍 𝑖ℎ , 𝐮𝑖𝑓 , 𝜽𝑓 𝑐𝑛𝑛 )‖22 (13)
𝑁𝑠 𝑖=1 The prediction of post-history well response can be obtained given
The RAE and FCNN are trained together in this hybrid deep-learning the history measurements and post-history well-controls, e.g., 𝐝𝑓 =
architecture. The network is constructed using 𝑃 𝑦𝑇 𝑜𝑟𝑐ℎ (Paszke et al., 𝐟𝑅𝐴𝐸−𝐹 𝐶𝑁𝑁 (𝐝ℎ , 𝐮𝑓 ). This online prediction is computationally efficient

6
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

as it does not rely on high-fidelity model simulation. In practice, some the second case is a 3D benchmark EGG model with non-Gaussian-
noises generally impact the observed data. The randomized maximum distributed geomodel parameters (Jansen, 2011). The high-fidelity
likelihood method can be utilized to measure the uncertainty caused model (HFM) simulations are performed based on a free open-source
by measurement errors. software Open Porous Media (𝑂𝑃 𝑀) for reservoir modeling and simu-
lation (Rasmussen et al., 2021). The basic-settings for running reservoir
𝐍𝐏𝐕𝑖 = 𝐟𝑅𝐴𝐸−𝐹 𝐶𝑁𝑁 (𝐝𝑜𝑏𝑠 + 𝐞𝑖 , 𝐮𝑓 ), 𝑖 = 1, 2, … , 𝑁𝑒 (14) simulations are given in Table 1.
𝑖
where, 𝐞𝑖 and 𝐍𝐏𝐕 represents the 𝑖th measurement noise vector and
corresponding post-history NPV value prediction, respectively. 4.1. Description of reservoir models
We can further combine RAE-FCNN proxy model and the PSO
algorithm to address closed-loop waterflooding reservoir production In these two cases, the uncertainty of geomodels parameters 𝐦 is
optimization. The proposed data-driven post-history production pre- characterized by rock permeability. The logarithmic permeability is
diction and optimization framework is comprised of three modules. assumed to satisfy log-Gaussian distribution. We generate Gaussian-
Firstly, a predetermined quantity of geomodels, such as 1000, are distributed realizations of logarithmic permeability using the Stanford
stochastically generated at predetermined statistical distributions. The Geostatistical Modeling Software (𝑆𝐺𝑒𝑀𝑆) (Remy, 2005). Through
specifying the mean ln(𝐦) and variance 𝜎𝑙𝑛𝐾 2 , respectively, the loga-
corresponding same amount of post-history well-controls are also sam-
pled from preset bounds [𝑢𝑙𝑜𝑤 , 𝑢𝑢𝑝 ], e.g., see Eq. (4), while the history rithmic permeability fields can be presented as follows
well-controls should be specific for all geomodels. A set of random one- (𝑥 −𝑥 ) 2
(𝑦 −𝑦 ) 2
−[ 1 2 2 + 1 2 2 ]
to-one combination of geomodel and well-control is used to generate 2 𝜂 𝜂
𝐂𝑙𝑛𝐾 (𝐱1 , 𝐱2 ) = 𝜎𝑙𝑛𝐾 𝑒 𝑥 𝑦 (15)
dataset (including history and post-history oil/water production rate)
where, 𝜂𝑥 and 𝜂𝑦 denote the correlation length of the random loga-
by running high-fidelity reservoir model simulations. The NPV values
rithmic permeability field at spatial 𝑥 and 𝑦 direction, respectively.
are then computed for each combination of geomodel and well control
𝑥𝑖 and 𝑦𝑖 denotes the coordinates of 𝑖th gridblock. The logarithmic
correspondingly, e.g., see Eq. (2). Secondly, the training and accuracy
permeability field obtained from the 𝑆𝐺𝑒𝑀𝑆 are presented in Fig. 5
assessment of RAE-FCNN surrogate model can be performed based
Figs. 5 and 6 display several realizations of geomodels for the
on the training samples and testing samples, respectively. The hyper-
synthetic 2D reservoir model and 3D benchmark EGG model, respec-
parameters are optimized based on the validation samples. Thirdly,
tively. The details referring to these two reservoir models can be found
the trained RAE-FCNN proxy model is combined with PSO algorithm
at (Xiao et al., 2023). The logarithmic permeability field is spatially
to optimize the post-history well controls. We should note that al-
heterogeneous and has significant geological uncertainty. For these two
though a heuristic optimizer such as PSO algorithm is chosen as the
models, the post-history injection rates of all four injectors are the
optimizer, the proxy-based optimization method is suitable for a wide
decision variables. For the synthetic 2D reservoir model, the injection
range of optimization algorithms including differential evolutionary
rates are optimized within the constraints [𝐮min , 𝐮max ] (i.e., 𝐮min =
(DE), genetic algorithm (GA), ensemble-based optimizer (e.g., EnOpt)
0 m3 /day and 𝐮max = 100 m3 /day). Controls of four injectors are
and even gradient-based optimizers that can rapidly construct exact
optimized at 1800 days and result in totally 40 decision variables.
gradients from the RAE-FCNN proxy by using auto-differentiation, as
For the 3D benchmark EGG model, the injection rates are optimized
done in our previous work (Xiao et al., 2022). We have proposed
within the constraints [𝐮min , 𝐮max ] (i.e., 𝐮min = 0 m3 /day and 𝐮max =
an efficient surrogate-assisted deterministic inversion framework to
79.5 m3 /day). Controls of eight injectors are optimized at 2400 days
primarily explore the possibility of applying deep neural network sur-
and result in totally 216 decision variables.
rogate to approximate the gradient of large-scale history matching by
In some realistic cases where we often consider bottomhole pres-
using auto-differentiation. The previous work has demonstrated the
sures (BHPs) at the producers as a part of decision variables or well
ability of using DL proxy to proceed the gradient-based optimization.
controls. In these two case-studies, we do not consider any well controls
We provide a surrogate model to fully replace the original high-fidelity
at the producers. The problem we consider might be not very com-
reservoir models. The optimization time almost can be ignored due to
plicated optimization problem and thus we do not need data hungry
the proxy model. Overall, we mainly focus on the accuracy of proposed
network for the closed loop production optimization. Although we only
neural network proxy model, any optimizers should be suitable for the
consider the injection rate of the injectors as the decision variables,
proxy-based production optimization theoretically.
we should note that the proposed methodology can be easily extended
The implementation of the proxy-based optimization framework
to optimize the BHPs at the producers as well. If the BHPs optimiza-
has been shown in Fig. 4. The RAE-FCNN proxy is trained at the
tion will make the problem more complex, we should improve the
initial offline stage (the right-downward box in the figure). Once this
deep-learning structure or provide more training samples.
is successfully completed, we perform a number of PSO runs using the
RAE-FCNN-based proxy in place of the high-fidelity model simulator.
4.2. Quality assessment of RAE-FCNN proxy models
The NPV values then can be predicted for each particle at each iter-
ation. Multiple PSO runs are performed with different initial swarms
Taking a two layers LSTM as an example, Table 2 shows an il-
to account for the stochastic nature of the algorithm. The RAE-FCNN
lustration of RAE-FCNN architecture. In this study, 1000 high-fidelity
proxy-based optimization, if implemented, enable the PSO algorithm to
models are simulated to prepare the training samples for RAE-FCNN
generate average 𝑁𝑃 𝑉 values for every population without requiring
proxy models. The relative error (𝑅𝐸), the coefficient of determination
high-fidelity reservoir model simulations. This makes the combination
(R2 ) and absolute error (𝐴𝐸) over 𝑁𝑡𝑒𝑠𝑡 testing samples regarded as an
of the PSO algorithm and RAE-FCNN proxy model an effective and reli-
indicator to assess the proxy quality are given as follows
able framework for data-driven optimization of waterflooding reservoir
∑𝑁𝑡𝑒𝑠𝑡 𝑁𝑃 𝑉 𝑖 −𝑁𝑃̂ 𝑉
𝑖
production.
𝑖=1 𝑁𝑃 𝑉 𝑖
𝑅𝐸 = (16)
𝑁𝑡𝑒𝑠𝑡
4. Case-study and results discussion
∑𝑁𝑡𝑒𝑠𝑡 𝑖 2
𝑖=1
(𝑁𝑃 𝑉 𝑖 − 𝑁𝑃̂ 𝑉 )
2
𝑅 =1− (17)
In this part, two waterflooding production cases are designed to ∑𝑁𝑡𝑒𝑠𝑡 𝑖 2
(𝑁𝑃 𝑉 𝑖 − 𝑁𝑃̄ 𝑉 )
demonstrate the performance of our proposed RAE-FCNN proxy-based 𝑖=1
∑𝑁𝑡𝑒𝑠𝑡 𝑖
production optimization workflow. The first case is a synthetic reservoir
𝑖=1
|𝑁𝑃 𝑉 𝑖 − 𝑁𝑃̂ 𝑉 |
model with Gaussian-distributed geomodel parameters, by contrast, 𝐴𝐸 = (18)
𝑁𝑡𝑒𝑠𝑡

7
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 4. Architecture of the proposed proxy-based PSO algorithm for waterflooding production optimization.

Table 1
Basic-setting parameters of the 2D Gaussian model and 3D Egg model using OPM-𝑓 𝑙𝑜𝑤 simulator.
Parameters Gaussian model Egg model
Dimension 40 × 120 × 1 60 × 60 × 7
Number of wells 6 producers, 4 injectors 4 producers, 8 injectors
Controlled wells 4 injectors 8 injectors
Number of time steps, 𝑁 120 120
Number of control times, 𝑇𝑤 10 27
Number of input variables, 𝑁u 40 216
Lower bound on input values, 𝐮min (m3 /day) 0 0
Upper bound on input values, 𝐮max (m3 /day) 100 79.5
prod
Oil production profit, 𝑟𝑜 ($/m3 ) 252 126
prod
Water production cost, 𝑟𝑤𝑝 ($/m3 ) 60 19
inj
Water injection cost, 𝑟𝑤𝑖 ($/m3 ) 30 6
Yearly interest rate, (𝑏𝑤 ) 0 0

Table 2 Table 3
Illustration of RAE-FCNN architecture. Taking a two layers LSTM as an Hyper parameter optimization results of RAE-FCNN model for example 1.
example here. Hype-parameters Ranges Example 1 Example 2
Unit Layer Input size Output size
Hidden state size, 𝑛ℎ𝑠 [32,64,128,256] 128 256
Input (𝑡ℎ , 𝑛𝑑ℎ ) – Number of layers, 𝑛𝑙 [1,2,3,4] 2 3
LSTM layer 1 for encoder (𝑡ℎ , 𝑛𝑑ℎ ) (𝑡ℎ , 𝑛ℎ𝑠 ) Learning rate, 𝑙𝑟 [1 × 10−2 ,1 × 10−3 ,1 × 10−4 ] 1 × 10−3 1 × 10−3
RAE LSTM layer 2 for encoder (𝑡ℎ , 𝑛ℎ𝑠 ) (𝑡ℎ , 𝑛ℎ𝑠 )
𝑀𝑅𝐸 – 0.068 0.055
LSTM layer 1 for decoder (𝑡ℎ , 𝑛ℎ𝑠 ) (𝑡ℎ , 𝑛ℎ𝑠 )
LSTM layer 2 for decoder (𝑡ℎ , 𝑛ℎ𝑠 ) (𝑡ℎ , 𝑛𝑑ℎ )
Input (1, 𝑛ℎ𝑠 × 𝑡𝑓 ) –
Sigmod (1, 𝑛ℎ𝑠 × 𝑡𝑓 ) (1, 𝑛ℎ𝑠 × 𝑡𝑓 ) model and 3D EGG model, respectively. The RAE-FCNN proxy model
FCNN FCNN layer 1 (1, 𝑛ℎ𝑠 × 𝑡𝑓 ) (1, 𝑛𝑢𝑓 )
parameterized by optimal hyper-parameters yield low loss functions.
FCNN layer 2 (1, 𝑛ℎ𝑠 + 𝑛𝑢𝑓 ) (1,1)
Sigmod (1,1) (1,1) The overfitting is a very important indicator for ensuring the predictive
capacity of the proxy model. As we can see from Fig. 7, the persistent
declines of both training and testing loss functions demonstrate that the
𝑖 overfitting does not occur as the training proceeds
where, 𝑁𝑃 𝑉 𝑖 and 𝑁𝑃̂ 𝑉 separately denote the NPV values predicted
𝑖 Figs. 8 and 9 show the relative error (𝑅𝐸), the coefficient of
from HFM and RAE-FCNN proxy model for the testing sample 𝑖. 𝑁𝑃̄ 𝑉 determination (R2 ) and absolute error (𝐴𝐸) over all testing samples
𝑖
is the average value of 𝑁𝑃 𝑉 . with respect to the training sample size for these two reservoir models,
For the first step, the hyper-parameters are optimized based on the respectively. In general, the smaller the 𝑅𝐸 and 𝐴𝐸 values, the better
validation samples. Table 3 displays the optimized hyper-parameters the predictability of the RAE-FCNN proxy model. The accuracy of the
for these two models. Fig. 7(a) and (b) depict the changes of loga- RAE-FCNN proxy model improves as the number of training samples. In
rithmic loss functions as the training proceeds for the 2D Gaussian these two case-studies, we can obtain high R2 values, e.g., about 95%

8
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 5. Illustration of well placement and three random permeability realizations for the 2D synthetic Gaussian model.

Fig. 6. Illustration of well placement and three random permeability realizations for the 3D EGG model.

corresponding to 800 training samples. As we can see from Fig. 9, the In general, the level of measurement noise has influences on the
median 𝑅𝐸 values are reduced from 12% to 7% for the 2D Gaussian post-history predictions obtained from the RAE-FCNN proxy model. In
model, by contrast, it has been reduced from 7% to 4% for the 3D this study, we set the level of measurement noise to be 10%, 20%
EGG model. Figs. 8(d) and 9(d) further illustrate the post-history NPV and 30%. Figs. 10 and 11 show the ensemble of prior and posterior
NPV predictions as the different level of measurement noise for the 2D
predictions obtained from the RAE-FCNN proxy models and high-
Gaussian model and 3D EGG model, respectively. The green, blue and
fidelity models. It can be seen that the HFM simulation results have
red lines denote the prior, posterior and reference NPV values, respec-
a good agreement with RAE-FCNN proxy predictions over 100 testing tively. It can be found that the uncertainty of post-history NPV values
samples. The RAE-FCNN proxy model can accurately forecast the post- is significantly reduced by considering the history measurements. The
history NPV values with a limited number of training samples (e.g., 𝑁𝑠 spread of NPV values becomes narrower and closer to the reference
= 800), especially for the synthetic 2D Gaussian model. NPV values (the red lines) as the level of measurement noise decreases.

9
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 7. Evolution of logarithmic loss functions with respect to the hype-parameters. The first and second rows are corresponding to the 2D Gaussian model and 3D EGG model,
respectively.

Fig. 8. The relative error (𝑅𝐸), the coefficient of determination (R2 ) and absolute error (𝐴𝐸) over all testing samples with respect to the training sample size for synthetic 2D
Gaussian model.

The simulation of a 2D synthetic Gaussian model and 3D EGG model Proxy-PSO does not involve any additional high-fidelity model simu-
are about 15 and 80 s, respectively, while the simulation of RAE-FCNN lation, the Proxy-PSO algorithm comes to convergence until the NPV
proxy model takes about 0.1 s on GPU card. The RAE-FCNN proxy values do not increase significantly. To test our method, we repeatedly
model demonstrates superior computation efficiency compared to the perform the optimization procedure for different initial populations at
high-fidelity model simulation. 10 times. The 20% observation errors are used to generate the history
measurements.
4.3. Production optimization results
4.4. Example 1

In this section, these two waterflooding reservoir models are used to We perform the deterministic optimization without considering ge-
verify our proposed surrogate-based optimization method. Specifically, omodel uncertainty, which can be realized by directly input the history
the NPV value at the post-history production period is the optimized measurement to the RAE-FCNN proxy. The evolution of objective func-
objective. The hype-parameter settings, including population size, con- tion values during the optimization process is presented in the first row
striction factor, two acceleration coefficients are 50, 0.729, 1.5 and of Fig. 12 (the plot on the left top corner — NPV vs iterations), and
2.0 for PSO algorithm used our proposed proxy-based PSO algorithm the optimization results of post-history well-controls for four injectors
(referred to as Proxy-PSO). As a comparison, PSO algorithm with high- are shown in the second row of Fig. 12. In this case-study, the RAE-
fidelity model (referred to as HFM-PSO) is also performed. Since the FCNN proxy mode is trained with 800 samples. To assess the quality

10
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 9. The relative error (𝑅𝐸), the coefficient of determination (R2 ) and absolute error (𝐴𝐸) over all testing samples with respect to the training sample size for synthetic 3D
Gaussian model.

Fig. 10. Post-history NPV values under measurement noise by the RAE-FCNN proxy model for 2D Gaussian model.

of the optimization results, we also compute the NPV by running the The NPV values from RAE-FCNN proxy and HFM are slightly distinctive
high-fidelity model simulations with the optimal well control settings due to the approximation errors, which might be further reduced by
obtained from the Proxy-PSO correspondingly, e.g., see the pink star. adding more training samples. Alternatively, another approach for

11
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 11. Post-history NPV values under measurement noise by the RAE-FCNN proxy model for 3D channelized EGG model.

handling this might be to retrain the RAE-FCNN proxy model after measurements are generated by adding 20% observation errors. These
some number of iterations to improve accuracy in the current region history noisy measurements and post-history well controls are fed to
of search space. This type of retraining could be repeated as necessary. RAE-FCNN to predict the NPV values correspondingly. The evolution of
After performing our proposed Proxy-PSO algorithm, the NPV value has ensemble of NPV values in the robust optimization process is presented
increased from 3.75 × 106 $to 6.06 × 106 $. It can be seen that the in Fig. 14. As a comparison, another 100 random history data that are
optimized case obtains significantly higher NPV than the initial case, different from the history measurements are regarded as the inputs.
demonstrating that the Proxy-PSO optimization can effectively identify The blue and red lines represent the evolutions of NPV values with
the optimal well-control settings for waterflooding reservoir produc- and without considering history measurements, respectively. It can be
tion. We also should note that the entire optimization procedure solely observed that the uncertainty of NPV values has been largely reduced
cost about 50 s, which are much faster than the case by simulating when considering the history measurements, indicated by the narrow
the high-fidelity reservoir models directly. Therefore, the optimization spread of NPV values. The initial and optimized cumulative oil and
efficiency can be significantly improved by using RAE-FCNN proxy water production, water injection profiles are also illustrated in Fig. 14.
models. Without considering the history measurements, the optimal cumulative
In the case-study, the water injection rates for all four injectors are oil production is substantially overestimated and thus causes high
the optimized variables. It can be found that the final cumulative water development risk. These results demonstrate that our proposed Proxy-
injection before and after the optimization is almost consistent, while PSO algorithm contributes to an efficient and reliable robust production
the optimized cumulative oil and water productions are improved and optimization.
reduced, respectively. This result reveals that given total amount of
injection water, optimizing the water injection rates can significantly 4.5. Example 2
improve the economic profit. As shown in Fig. 12, the water should
be injected in a way that decreases first and then increases for all four In this part, the proposed Proxy-PSO algorithm is further verified
injectors. This type of water injection strategy can improve the water- on a benchmark 3D EGG model. We also first perform the deterministic
sweeping effects. The remaining oil saturation distributions before and optimization corresponding to the specific history measurements. The
after the optimization are presented in Fig. 13. This figure is generated evolution of objective function values in the optimization process is
by running the high-fidelity model simulations with the initial and presented in the first row of Fig. 15, and the controls of the best
optimal well-control settings. The degree of water breakthrough for solution found among the 10 runs for different wells are shown in the
producer P2 and P5 is decreased. In addition, the amount of remaining second row of Fig. 15. We also compute the NPV values by running the
oil around the producer P4 and P6 also reduces as the optimization high-fidelity model simulations with the optimal well-control settings
proceeds. All these results verify the reliability of the optimized well correspondingly, e.g., see the pink star. As the Proxy-PSO algorithm
controls by Proxy-PSO algorithm. proceeds, the NPV value has increased from negative economic profit
We further investigate the robust optimization over 100 posterior −8.42 × 105 $to 6.15 × 106 $. It can be concluded that the optimized
geomodels. Since our proposed RAE-FCNN proxy model do not directly results obtain significantly higher NPV than the initial case. The Proxy-
incorporate the geomodels as the inputs. Instead, we compute the PSO optimization can effectively and efficiently search the optimal
averaged objective function over the ensemble of perturbed history well-control schedules for waterflooding reservoir production.
measurements for implicitly considering geomodel uncertainty in ro- We also should note that the entire optimization procedure solely
bust optimization process. That is, 100 different realizations of noisy cost about 60 s, which are much faster than the case by simulating the

12
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 12. Evolution of objective function values in the optimization process and optimized water injection sequences for Example 1. The pink star denote the NPV values computed
by HFM simulation.

Table 4 Example 1, all production metrics, including optimized cumulative

The mean, best, worst, and standard deviations results of all 10 runs corresponding to oil and water production, cumulative water injection are decrease
different training samples for example 2.
compared with the initial model. Specifically, the cumulative water
Method Training size NPV, × 106 $
production and cumulative water injection have reduced by about 2
Initial Best Mean Worst Standard deviation times and 3 times, respectively, by contrast, the cumulative oil pro-
100 −0.842 5.97 5.69 5.50 0.15 duction has reduced slightly. These results indicate that controlling the
300 −0.842 6.11 5.96 5.77 0.11 water production and water injection are crucial to achieve high eco-
Proxy-PSO
500 −0.842 6.12 5.99 5.81 0.08
800 −0.842 6.67 6.12 5.88 0.14
nomic profits. It can be seen from Fig. 15 that the water injection rate
for almost all eight injectors should be decreased after the Proxy-PSO
HFM-PSO 1000 −0.842 5.68 5.24 4.95 0.18
algorithm. This type of water injection strategy can effectively decrease
the degree of water breakthrough for the producers. The remaining oil
saturation distributions before and after the optimization are presented
in Fig. 16. The reduction of water breakthrough is optimized at the
expense of high remaining oil saturation.
high-fidelity reservoir models directly. The computational efficiency
In the traditional close-loop production optimization, the produc-
evaluation in this study is based on the total number of original
high-fidelity model simulations, as the time required for implementing tion optimization is performed based on the updated geomodels ob-
Proxy-PSO is insignificant compared to running HFM. The simulations tained by history matching. The ensemble-based data assimilation
of 1000 prior models are the most demanding part of our defined (e.g., ES-MDA) and optimization algorithms (,e.g., EnOpt and StoSAG)
Proxy-PSO optimization workflow. Although conventional close-loop are generally adopted to resolve these two steps. Due to the compu-
optimization with high-fidelity simulation models was not performed, tation issue, we do not explicitly perform the traditional close-loop
it is evident that Proxy-PSO is highly efficient for practical applications, production optimization procedure as a comparison in this study. To
particularly in uncertainty quantification and production optimization. verify the proposed proxy-based optimization method, the posterior
Furthermore, the amount of saved computation time increases as the model is assumed to be perfectly known and fully identical with the
model complexity. reference model, and then HFM-PSO algorithm is used to optimize the
In this benchmark 3D EGG model, the water injection rates for all post-history well controls by high-fidelity model simulations. Fig. 17
eight injectors are the optimized variables. Different from the previous shows the comparisons of objective function values in the optimization

13
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 13. The remaining oil saturation distributions before and after the waterflooding production optimization for Example 1.

Fig. 14. Evolution of ensemble of NPV values with and without considering history measurements in the robust optimization process for Example 1.

process for Proxy-PSO and HFM-PSO algorithms. The dots represents of high-fidelity model simulations. Note that the NPV values shown in
the optimized NPV values for Proxy-PSO with 100, 300, 500 and 800 Fig. 17 and Table 4 are computed by running HFM simulations. It can
training samples, respectively. The blue lines denotes the evolution be observed that the optimized NPV values progressively improve as
of NPV values as the HFM-PSO algorithm proceeds. We specify 1000 the number of training samples. Comparing to the Proxy-PSO, the HFM-
HFM simulations for the HFM-PSO algorithm. Table 4 shows the mean, PSO has not fully come to convergence at 1000 HFM runs, as a result,
best, worst, and standard deviations results of all 10 independent runs. our proposed Proxy-PSO can achieve higher NPV values than that of
The results are given in terms of the obtained NPV and the number HFM-PSO even when solely 100 training samples are used. We also

14
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 15. Evolution of objective function values in the optimization process and optimized water injection sequences for Example 2. The pink star denote the NPV values computed
by HFM simulation.

should emphasize that these 10 independent PSO runs for HFM-PSO 4.6. Additional remarks
requires 10 000 HFM simulations, by contrast, the HFM simulations
solely demand at the offline RAE-FCNN training stage, the online While the proposed method has its advantages, there are also some
prediction and production optimization does not involve additional limitations worth considering. In the current study, we do not consider
HFM runs. nonlinear state constraints (such as injection bottomhole pressures
We further perform the robust optimization over 100 posterior at the injectors in case injection rates are well controls, field water
geomodels. The evolution of ensemble of NPV values in the robust injection rates, and/or field production rate) in production optimization
optimization process is presented in Fig. 18. The blue and red lines applications. In the proposed deep recurrent auto-encoder and fully-
represent the evolution of NPV values with and without considering connected neural network proxy, the trained RAE-FCNN proxy model
history measurements, respectively. It can be observed that considering can directly predict scalar NPV value corresponding to the history
the history measurements leads to a substantial uncertainty reduc- measurements and user-specific post-history well controls. We do not
tion of the NPV values. Comparing to the Example 1, the history predict time-series well responses, such as injection bottomhole pres-
measurements contribute significantly to the uncertainty reduction. sures at the injectors in case injection rates are well controls, field
These results demonstrate that our proposed Proxy-PSO algorithm is water injection rates, and/or field production rate. It should be worthy
particularly useful to perform uncertainty quantification. of considering these constraints while optimizing the well controls. To

15
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 16. The remaining oil saturation distributions before and after the waterflooding production optimization for Example 2.

The RAE-FCNN proxy model used in our approach may require

a significant amount of training data to achieve the desired level of
accuracy. It is often difficult to determine the exact number of training
samples needed, and thus, it may be necessary to invest considerable
resources into training the proxy model. Additionally, the conventional
use of a proxy model in an optimization workflow involves training the
proxy in an offline stage, followed by using it for function evaluations in
the online optimization process. However, this approach can be further
improved by incorporating an adaptive or dynamic proxy modeling
strategy, where the proxy model is updated periodically during the
optimization process. This could be a valuable area of research for
future studies.
We should emphasize that the above adaptive or dynamic proxy-
based optimization procedure to retrain the proxy model are relatively
simple to perform for the conventional optimization problems with-
out closed-loop behavior. We only need to re-sample the training
points around the current region of search space for well controls. By
contrast, in our proposed proxy-based closed-loop optimization, the
history predictions and post-history well controls are the inputs of our
Fig. 17. Evolution of objective function values in the optimization process for
proposed RNN-type proxy model and hence needed to be re-sampled
Proxy-PSO and HFM-PSO algorithms.
simultaneously. Since the history predictions depend on the geomodels,
which are uncertain beforehand, the sampling of history predictions
will become problematic. We are trying to resolve this extrapolation
incorporate these constraints, we can combine recurrent autoencoder
issue in a near-future study.
and long short-term memory neural network (Xiao et al., 2023). The
long short-term memory neural network can be used to predict the
post-history well responses, e.g., oil/water rate and water injection 5. Conclusions
rate. The production optimization with constraints should be a very
promising research direction in a future study (Atadeger et al., 2023; Production optimization and management of geoenergy resources
Kim & Durlofsky, 2021; Nasir, Yu, & Sepehrnoori, 2020; Nasir et al., under geomodel uncertainty generally demands implementation of his-
2020). tory matching and production steps, however, the solutions of these
The main goal of this study is to perform accurate and efficient two steps requires intensive computation-cost by running numerous
closed-loop production optimization for the waterflooding production high-fidelity reservoir model simulations. This work presents a novel
process using proposed data-driven optimization method by replacing RNN-proxy-assisted evolutionary algorithm aiming at accelerating well-
a high-fidelity simulator with a RNN-type proxy model. In the results control optimization problem. We construct a RNN-proxy through com-
reported in our study, we also find that the optimal NPVs found in bining recurrent autoencoder and full-connected neural network. With-
proxy-based optimization are not as high as those from high-fidelity out explicitly performing history matching step, the history data can
simulation-based optimization. This discrepancy appears to result from be indirectly used to guide the post-history production optimization
error in the proxy estimates. This error might have a larger impact and thus achieve more reliable well-control settings. The performance
at later PSO iterations, in which the best solution has evolved away of the RNN-proxy-based optimization approach has been initially as-
from the region defined by the original training runs. One approach for sessed through a synthetic 2D Gaussian model and a benchmark 3D
handling this would be to retrain the RNN-based proxy after some num- channelized EGG model for practical production optimization demon-
ber of iterations to improve accuracy in the current region of search stration. By comparing our proposed approach with the traditional
space. This type of retraining could be repeated as necessary (Kim & method that solely relies on high-fidelity reservoir simulations, we have
Durlofsky, 2021; Ma, Kim et al., 2022). demonstrated that our surrogate-based optimization approach is highly

16
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Fig. 18. Evolution of ensemble of NPV values with and without considering history measurements in the robust optimization process for Example 2.

effective for achieving desirable well-control settings. The use of RAE- Declaration of competing interest
FCNN proxy model expedites the optimization process and rapidly ob-
tain improved NPV values with significantly fewer high-fidelity model The authors declare that they have no known competing finan-
simulations within a short time-frame. cial interests or personal relationships that could have appeared to
In the current work, we do not explicitly perform comparisons influence the work reported in this paper.
between proposed RNN-proxy-assisted evolutionary algorithm and tra-
Data availability
ditional close-loop production optimization procedure that demands
continuous implementation of history matching and production opti-
Data will be made available on request.
mization. We mainly focus on single-loop optimization under geomodel
uncertainty. The extension of the RNN proxy model to address multi- Acknowledgments
cycling closed-loop production optimization should be more suitable
to practical applications. The main issue will be whether we can This work is supported by Science Foundation of China University of
repeatedly employ the RNN proxies pre-trained at previous loops to Petroleum, Beijing, China (No. 2462021BJRC005) and National Natural
continuously optimize the well controls at the next loop and how to Science Foundation of China (No. 52304055). Computing resources are
guarantee its accuracy as the optimization proceeds. The hybrid use of provided by Department of Petroleum Engineering at China University
transfer-learning and multi-fidelity training data that low-fidelity data of Petroleum, Beijing.
and high-fidelity data are generated by pre-trained deep-learning surro-
gate at the previous loop and full-physic model simulations respectively References
should be very promising. In this study, our proposed data-driven close-
Almasov, A., Nguyen, Q., & Onur, M. (2022). Nonlinearly constrained life-cycle
loop optimization workflow actually has not been compared with the production optimization with a least-squares support-vector regression proxy. In
conventional version, where the reservoir history matching and pro- ECMOR 2022, Vol. 2022 (pp. 1–29). European Association of Geoscientists &
duction optimization are performed. The accurate characterization of Engineers.
Almasov, A., & Onur, M. (2021). Life-cycle optimization of the carbon dioxide huff-n-
non-Gaussian geomodel (e.g., the 3D Channelized EGG model) should puff process in an unconventional oil reservoir using least-squares support vector
be very challenging, our approach should outperform the conventional and Gaussian process regression proxies. SPE Journal, 26(04), 1914–1945.
simulation-based close-loop optimization. The comparisons are planned Almasov, A., & Onur, M. (2022). Life-cycle production optimization of the CO2-
water-alternating-gas injection process using least-squares support-vector regression
to be conducted in a future study.
(LS-SVR) Proxy. In SPE annual technical conference and exhibition. OnePetro.
Atadeger, A., Onur, M., Sheth, S., & Banerjee, R. (2023). Deep learning-based and
kernel-based proxy models for nonlinearly constrained life-cycle production opti-
CRediT authorship contribution statement mization. In SPE reservoir characterisation and simulation conference and exhibition.
OnePetro.
Atadeger, A., Sheth, S., Vera, G., Banerjee, R., & Onur, M. (2022). Deep learning-based
Cong Xiao: Software modelling, Data Formal analysis, Writing. proxy models to simulate subsurface flow of three-dimensional reservoir systems.
Shicheng Zhang: Supervision, Funding acquisition, Conceptualization. In ECMOR 2022, Vol. 2022 (pp. 1–32). European Association of Geoscientists &
Engineers.
Xingfang Ma: Supervision, Funding acquisition, Conceptualization.
Bukshtynov, V., Volkov, O., Durlofsky, L. J., & Aziz, K. (2015). Comprehensive
Tong Zhou: Investigation, Reviewing. Jianbing Jin: Project adminis- framework for gradient-based optimization in closed-loop reservoir management.
tration, Software, Validation. Computational Geosciences, 19(4), 877–897.

17
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Capolei, A., Völcker, C., Frydendall, J., & Jorgensen, J. B. (2012). Oil reservoir pro- Li, X., Xiao, K., Li, X., Yu, C., Fan, D., & Sun, Z. (2022). A well rate prediction method
duction optimization using single shooting and ESDIRK methods. IFAC Proceedings based on LSTM algorithm considering manual operations. Journal of Petroleum
Volumes, 45(8), 286–291. Science and Engineering, 210, Article 110047.
Chang, H., Zhang, D., & Lu, Z. (2010). History matching of facies distribution with Lima, M. M., Emerick, A. A., & Ortiz, C. E. (2020). Data-space inversion with ensemble
the EnKF and level set parameterization. Journal of Computational Physics, 229(20), smoother. Computational Geosciences, 24, 1179–1200.
8011–8030. Liu, D., Rao, X., Zhao, H., Xu, Y.-F., & Gong, R.-X. (2021). An improved data space
Chen, C., Gao, G., Ramirez, B. A., Vink, J. C., Girardi, A. M., et al. (2015). Assisted inversion method to predict reservoir state fields via observed production data.
history matching of channelized models using pluri-principal component analysis. Petroleum Science, 18(4), 1127–1142.
In SPE reservoir simulation symposium. Society of Petroleum Engineers. Liu, Z., & Reynolds, A. C. (2021). Gradient-enhanced support vector regression for
Chen, G., Li, Y., Zhang, K., Xue, X., Wang, J., Luo, Q., et al. (2021). Efficient robust life-cycle production optimization with nonlinear-state constraints. SPE
hierarchical surrogate-assisted differential evolution for high-dimensional expensive Journal, 26(04), 1590–1613.
optimization. Information Sciences, 542, 228–246. Liu, Y., Sun, W., & Durlofsky, L. J. (2018). A deep-learning-based geological pa-
Chen, G., Zhang, K., Xue, X., Zhang, L., & Yang, Y. (2019). Surrogate-assisted rameterization for history matching complex models. Mathematical Geosciences,
evolutionary algorithm with dimensionality reduction method for water flooding 1–42.
production optimization. Journal of Petroleum Science and Engineering, 185, Article Lu, R., Forouzanfar, F., & Reynolds, A. C. (2017). An efficient adaptive algorithm
106633. for robust control optimization using StoSAG. Journal of Petroleum Science and
Chen, G., Zhang, K., Zhang, L., Xue, X., Ji, D., Yao, C., et al. (2020). Global and Engineering, 159, 314–330.
local surrogate-model-assisted differential evolution for waterflooding production Lu, L., & Zhang, D. (2015). Assisted history matching for fractured reservoirs by use
optimization. SPE Journal, 25(01), 105–118. of hough-transform-based parameterization. Spe Journal, 20(05), 0942–0961.
Chen, Y.-T., Zhang, D.-X., Zhao, Q., & Liu, D.-X. (2023). Interpretable machine learning Ma, Z., Kim, Y. D., Volkov, O., & Durlofsky, L. J. (2022). Optimization of subsurface
optimization (InterOpt) for operational parameters: A case study of highly-efficient flow operations using a dynamic proxy strategy. Mathematical Geosciences, 54(8),
shale gas development. Petroleum Science, 20(3), 1788–1805. http://dx.doi.org/10. 1261–1287.
1016/j.petsci.2022.12.017, URL: https://www.sciencedirect.com/science/article/ Ma, X., Zhang, K., Wang, J., Yao, C., Yang, Y., Sun, H., et al. (2022). An Efficient
pii/S1995822622003429. Spatial-Temporal Convolution Recurrent Neural Network Surrogate Model for
Du, S.-Y., Zhao, X.-G., Xie, C.-Y., Zhu, J.-W., Wang, J.-L., Yang, J.-S., et al. (2023). History Matching. SPE Journal, 27(02), 1160–1175. http://dx.doi.org/10.2118/
Data-driven production optimization using particle swarm algorithm based on 208604-PA.
the ensemble-learning proxy model. Petroleum Science, 20(5), 2951–2966. http: Ma, X., Zhang, K., Zhang, J., Wang, Y., Zhang, L., Liu, P., et al. (2022). A novel
//dx.doi.org/10.1016/j.petsci.2023.04.001, URL: https://www.sciencedirect.com/ hybrid recurrent convolutional network for surrogate modeling of history matching
science/article/pii/S1995822623000961. and uncertainty quantification. Journal of Petroleum Science and Engineering, 210,
Emerick, A. A., & Reynolds, A. C. (2012). History matching time-lapse seismic data 110109.
using the ensemble Kalman filter with multiple data assimilations. Computational Mattax, C. C., & Dalton, R. L. (1990). Reservoir simulation (includes associated papers
Geosciences, 16(3), 639–659. 21606 and 21620). Journal of Petroleum Technology, 42(06), 692–695.
Emerick, A. A., & Reynolds, A. C. (2013). Ensemble smoother with multiple data Nasir, Y., Yu, W., & Sepehrnoori, K. (2020). Hybrid derivative-free technique and
assimilation. Computers & Geosciences, 55, 3–15. effective machine learning surrogate for nonlinear constrained well placement and
production optimization. Journal of Petroleum Science and Engineering, 186, Article
Fonseca, R. M., Leeuwenburgh, O., Van den Hof, P. M., & Jansen, J. D. (2015). Im-
106726.
proving the ensemble-optimization method through covariance-matrix adaptation.
Nguyen, V. (2019). Bayesian optimization for accelerating hyper-parameter tuning.
SPE Journal, 20(01), 155–168.
In 2019 IEEE second international conference on artificial intelligence and knowledge
Goda, T., & Sato, K. (2013). Global optimization of injection well placement toward
engineering (AIKE) (pp. 302–305). IEEE.
higher safety of CO2 geological storage. Energy Procedia, 37, 4583–4590.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., et al. (2019).
Goda, T., & Sato, K. (2014). History matching with iterative latin hypercube samplings
PyTorch: An imperative style, high-performance deep learning library. In Advances
and parameterization of reservoir heterogeneity. Journal of Petroleum Science and
in neural information processing systems (pp. 8024–8035).
Engineering, 114, 61–73.
Pouladi, B., Karkevandi-Talkhooncheh, A., Sharifi, M., Gerami, S., Nourmohammad, A.,
Guo, Z., Chen, C., Gao, G., Cao, R., Li, R., & Liu, C. (2018). Integration of support vector
& Vahidi, A. (2020). Enhancement of SPSA algorithm performance using reservoir
regression with distributed Gauss-Newton optimization method and its applications
quality maps: Application to coupled well placement and control optimization
to the uncertainty assessment of unconventional assets. SPE Reservoir Evaluation &
problems. Journal of Petroleum Science and Engineering, 189, Article 106984.
Engineering, 21(04), 1007–1026.
Qing-Dao-Er-Ji, R., Su, Y. L., & Liu, W. W. (2020). Research on the LSTM mongolian
Guo, Z., & Reynolds, A. C. (2018). Robust life-cycle production optimization with a
and Chinese machine translation based on morpheme encoding. Neural Computing
support-vector-regression proxy. SPE Journal, 23(06), 2409–2427.
and Applications, 32(1), 41–49.
Hou, J., Zhou, K., Zhang, X. S., Kang, X. D., & Xie, H. (2015). A review of closed-loop
Rasmussen, A. F., Sandve, T. H., Bao, K., Lauser, A., Hove, J., Skaflestad, B., et al.
reservoir management. Petroleum Science, 12(001), 114–128.
(2021). The open porous media flow reservoir simulator. Computers & Mathematics
Inapakurthi, R. K., Miriyala, S. S., & Mitra, K. (2021). Deep learning based dynamic with Applications, 81, 159–185.
behavior modelling and prediction of particulate matter in air. Chemical Engineering Remy, N. (2005). S-GeMS: the stanford geostatistical modeling software: a tool for new
Journal, 426, Article 131221. algorithms development. In Geostatistics Banff 2004 (pp. 865–871). Springer.
Jansen, J. (2011). Adjoint-based optimization of multi-phase flow through porous Sani, S. H. (2020). Short-term and long-term air quality forecasting technique using
media–a review. Computers & Fluids, 46(1), 40–51. stacked LSTM. In ICCIP 2020: 2020 the 6th international conference on communication
Jiang, S., Sun, W., & Durlofsky, L. J. (2020). A data-space inversion procedure for and information processing.
well control optimization and closed-loop reservoir management. Computational Silva, V., Emerick, A. A., Couto, P., & Alves, J. (2017). History matching and
Geosciences, 24, 361–379. production optimization under uncertainties – Application of closed-loop reservoir
Jin, Z. L., Liu, Y., & Durlofsky, L. J. (2020). Deep-learning-based surrogate model for management. Journal of Petroleum Science and Engineering, 157, 860–874.
reservoir simulation with time-varying well controls. Journal of Petroleum Science Sun, W., & Durlofsky, L. J. (2017). A new data-space inversion procedure for efficient
and Engineering, 192, Article 107273. uncertainty quantification in subsurface flow problems. Mathematical Geosciences,
Kim, Y. D., & Durlofsky, L. J. (2021). A recurrent neural network–based proxy model for 49, 679–715.
well-control optimization with nonlinear output constraints. SPE Journal, 26(04), Sun, W., & Durlofsky, L. J. (2019). Data-space approaches for uncertainty quantification
1837–1857. of CO 2 plume location in geological carbon storage. Advances in Water Resources,
Kim, Y. D., & Durlofsky, L. J. (2023). Convolutional–recurrent neural network proxy 123, 234–255.
for robust optimization and closed-loop reservoir management. Computational Sun, W., Hui, M. H., & Durlofsky, L. J. (2017). Production forecasting and uncertainty
Geosciences, 27(2), 179–202. quantification for naturally fractured reservoirs using a new data-space inversion
Kim, J., Lee, K., & Choe, J. (2021). Efficient and robust optimization for well patterns procedure. Computational Geosciences, 21(5–6), 1443–1458.
using a PSO algorithm with a CNN-based proxy model. Journal of Petroleum Science Tang, M., Liu, Y., & Durlofsky, L. J. (2020). A deep-learning-based surrogate model for
and Engineering, (1), Article 109088. data assimilation in dynamic subsurface flow problems. Journal of Computational
Kuk, E., Stopa, J., Kuk, M., Janiga, D., & Wojnarowski, P. (2021). Petroleum reservoir Physics, 413, Article 109456.
control optimization with the use of the auto-adaptive decision trees. Energies, Vo, H. X., & Durlofsky, L. J. (2015). Data assimilation and uncertainty assessment for
14(18), 5702. complex geological models using a new PCA-based parameterization. Computational
Li, Y., Hariri-Ardebili, M. A., Deng, T., Wei, Q., & Cao, M. (2023). A surrogate- Geosciences, 19(4), 747–767.
assisted stochastic optimization inversion algorithm: Parameter identification of Vo, H. X., & Durlofsky, L. J. (2016). Regularized kernel PCA for the efficient
dams. Advanced Engineering Informatics, 55, Article 101853. parameterization of complex geological models. Journal of Computational Physics,
Li, W., Wang, L., Dong, Z., Wang, R., & Qu, B. (2022). Reservoir production prediction 322, 859–881.
with optimized artificial neural network and time series approaches. Journal of Wang, L. (2001). Intelligent optimization algorithms with applications. beijing tsinghua
Petroleum Science and Engineering, 215, Article 110586. university press.

18
C. Xiao et al. Expert Systems With Applications 242 (2024) 122797

Wang, N., Chang, H., Kong, X.-Z., & Zhang, D. (2023). Deep learning based closed- Xue, X., Chen, G., Zhang, K., Zhang, L., Zhao, X., Song, L., et al. (2022). A divide-and-
loop well control optimization of geothermal reservoir with uncertain permeability. conquer optimization paradigm for waterflooding production optimization. Journal
Renewable Energy, 211, 379–394. of Petroleum Science and Engineering, 211, Article 110050.
Xiao, Q., Chang, X., Zhang, X., & Liu, X. (2020). Multi-information spatial–temporal Yang, R., Singh, S. K., Tavakkoli, M., Amiri, N., & Rai, R. (2020). CNN-LSTM
LSTM fusion continuous sign language neural machine translation. IEEE Access, 8, deep learning architecture for computer vision-based modal frequency detection.
216718–216728. Mechanical Systems and Signal Processing, 144, Article 106885.
Xiao, C., Leeuwenburgh, O., Lin, H.-X., & Heemink, A. (2021). Conditioning of Yao, M., Chang, H., Li, X., & Zhang, D. (2018). Tuning fractures with dynamic data.
deep-learning surrogate models to image data with application to reservoir Water Resources Research, 54(2), 680–707.
characterization. Knowledge-Based Systems, 220, Article 106956. Yao, J., Wei, S. L., Zhang, K., & Jiang, J. K. (2012). Constrained reservoir production
Xiao, C., Lin, H.-X., Leeuwenburgh, O., & Heemink, A. (2022). Surrogate-assisted optimization. Zhongguo Shiyou Daxue Xuebao, 36(2), 125–129+135.
inversion for large-scale history matching: Comparative study between projection- Zeng, Y., Nie, Y., & Yang, Y. (2021). Gradient-based production optimization in
based reduced-order modeling and deep neural network. Journal of Petroleum reservoir development. IOP Conference Series: Earth and Environmental Science,
Science and Engineering, 208, Article 109287. 632(2), Article 022014, (6pp).
Xiao, C., Zhang, S., Ma, X., Zhou, T., Hou, T., & Chen, F. (2023). Robust production Zhao, H., Zhang, Y., Cao, L., Kang, X., & Xie, X. (2019). Constrained short-term
forecast and uncertainty quantification for waterflooding reservoir using hybrid and long-term multi-objective production optimization using general stochastic
recurrent auto-encoder and long short-term memory neural network. Journal of approximation algorithm. Cluster Computing, 22, 6267–6281.
Petroleum Science and Engineering, 230, Article 212252. Zhong, C., Zhang, K., Xue, X., Qi, J., Zhang, L., Yan, X., et al. (2022). Historical
Xiong, H., Kim, C., & Fu, J. (2020). A data-driven approach to forecasting production Window-Enhanced Transfer Gaussian Process for Production Optimization. SPE
with applications to multiple shale plays. In SPE improved oil recovery conference. Journal, 1–18. http://dx.doi.org/10.2118/209813-PA.

19