An uncertainty-aware Digital Shadow for underground multimodal CO₂ storage monitoring

Geological carbon storage (GCS) constitutes capturing carbon dioxide (CO₂) emissions from industrial processes followed by storage in deep geological formations, such as depleted oil and gas reservoirs, deep saline aquifers, or coal seams (Page et al. 2020). Because of its ability to scale (Ringrose 2020, 2023), this net-negative CO₂-emission technology is widely considered as one of the most promising technologies to help mitigate the effects of anthropogenic CO₂ greenhouse gas emissions, which are major contributors to global warming (Bui et al. 2018; IPCC special report 2005). According to the Intergovernmental Panel on Climate Change (IPCC, Panel on Climate Change) (2018)), it is imperative to contain atmospheric warming to $<2\degree$C by the year 2100 to avoid serious consequences of global warming. To achieve this goal, scalable net-negative emission technologies that can remove tens of gigatonnes of CO₂ per year (IPCC special report 2018) are needed in addition to reducing emissions. These needs call for a transition towards increased use of renewable energy in combination with the global deployment of GCS technology to store significant amounts of supercritical CO₂ into geological formations to mitigate the effects of climate change (Metz et al. 2005; Orr Jr 2009). When supercritical CO₂—captured at anthropogenic sources (fossil-fuel power plants, steel, cement, and chemical industries), is stored securely and permanently in underground geological storage reservoirs, it can be scaled to mitigate upto $20\%$ of the total CO₂ emission problem (Ketzer, Iglesias, and Einloft 2012). This work is a first step towards an uncertainty-aware Digital Twin aimed at optimizing underground operations while mitigating risks.

Our main technical objective is directed towards monitoring CO₂-storage projects, so that assurances can be made whether storage proceeds as expected and whether CO₂ plumes remain within the storage complex. These assurances are referred to as Conformance and Containment by the regulators and operators of GCS (Ringrose 2020, 2023). Even though the proposed methodology could be applied to other settings, our focus is on CO₂ storage in offshore saline aquifers in a marine setup as illustrated in figure 1. Compared to storage projects on land, this choice for marine settings circumvents potential complications and risks associated with future liability and geomechanically-induced pressure changes that may trigger earthquakes (Zoback and Gorelick 2012; He et al. 2011; Raza et al. 2016). While this approach simplifies the complexity of underground CO₂ storage projects to some degree, simulations for the state of CO₂ plumes are on balance more challenging than conventional reservoir simulations because the fluid-flow properties are less well-known. Besides, the process of displacing brine with supercritical CO₂ is intricate, taxing multi-phase flow simulators (Ringrose 2020, 2023). To illustrate the possible impact of lack of knowledge on the reservoir properties (the permeability), figure 1 also includes a juxtaposition between multi-phase flow simulations for two different realizations of permeability fields drawn from a stochastic baseline established from active-source surface seismic data (Yin, Orozco, et al. 2024). While both realizations for the permeability are consistent with this baseline, they yield vastly different outcomes where in one case the CO₂ plume remains well within (figure 1 bottom-left) the storage complex (denoted by the white dashed line in figure 1) whereas the other simulation obviously breaches Containment (figure 1 bottom-right), due to the presence of an “unknown” high-permeability streak. By maximally leveraging information derived from multimodal time-lapse monitoring data, we plan to guide uncertain reservoir simulations, so outlying “unpredicted” geological events, such as the one caused by the high-permeability streak, can be accounted for.

As stated eloquently by Ringrose (2020); Ringrose (2023), CO₂-storage projects can not rely on static reservoir models alone (Ringrose and Bentley 2016). Instead, reservoir models and simulations need to be updated continuously from multimodal sources (e.g. active-source surface seismic and/or pressure/saturation measurements at wells) of time-lapse data. By bringing this insight together with the latest developments in generative Artificial AI, which includes the sub-discipline of Simulation-Based Inference (SBI, Cranmer, Brehmer, and Louppe (2020)), where conditional neural networks are trained to act as posterior density estimators, and nonlinear ensemble filtering (Spantini, Baptista, and Marzouk 2022), a Digital Shadow (Asch 2022) is coined as a framework for monitoring underground CO₂-storage projects. Digital Shadows differ from Digital Twins (Asch 2022) in the sense that they integrate information on the state of a physical object or system — e.g. the temporal behavior of CO₂-injection induced changes in pressure/saturation for GCS — by capturing information unidirectionally from real-time, possibly multimodal, observed time-lapse data without feedback loops. While our framework is designed to allow for future optimization, control, and decision-making to mitigate risks of GCS that would turn our Shadow into a Twin, our main focus in this work is to introduce a scalable machine-learning assisted data-assimilation framework (Asch 2022; Spantini, Baptista, and Marzouk 2022). To be relevant to realistic CO₂-storage projects, use is made of scalable state-of-the-art reservoir simulation, wave-equation based seismic inversion (Virieux and Operto 2009; Baysal, Kosloff, and Sherwood 1983), ensemble Bayesian filtering (Asch 2022), and the advanced machine-learning technique of neural posterior density estimation (Cranmer, Brehmer, and Louppe 2020; Radev et al. 2020), to capture uncertainties due to our lack of knowledge on the reservoir properties, the permeability field, in a principled way. Because the envisioned Digital Shadow is aimed at monitoring CO₂-injection projects over many decades, its performance is validated in silico—i.e. via a well-designed computer-simulation study. As with the development of climate models, this approach is taken by necessity because suitable time-lapse datasets spanning over long periods of time are not yet available.

The central approach taken in this work finds its basis in ensemble Kalman Filtering (EnKF, Evensen (1994); Asch, Bocquet, and Nodet (2016); Carrassi et al. (2018)), a widely employed and scalable Bayesian data-assimilation (DA) technique. Here, DA refers to a statistical approach where indirect and/or sparse real-world time-lapse measurements are integrated into a model that describes the dynamics of the hidden state of CO₂ plumes. While other time-lapse data-collection modalities exist, including controlled-source electromagnetics (Ayani, Grana, and Liu 2020; Grana, Liu, and Ayani 2021), gravity (Appriou and Bonneville 2022), history matching from data collected at wells (G. Zhang, Lu, and Zhu 2014; Strandli, Mehnert, and Benson 2014), and even tomography with muons (Gluyas et al. 2019), we employ active-source time-lapse surface-seismic surveys (D. E. Lumley 2001) in combination with data collected at wells. Seismic surveys are known for their relatively high attainable resolution, sensitivity to small CO₂-induced changes in seismic rock properties, and spatial coverage in both the vertical and horizontal coordinate directions (Ringrose 2020, 2023).

By including crucial multi-phase fluid-flow simulations (Nordbotten and Celia 2011) for the state, and by exchanging the extended Kalman filter (Asch, Bocquet, and Nodet 2016; Asch 2022) for better scalable EnKF (Bruer et al. 2024) type of approach, our work extends early seismic monitoring work by C. Huang and Zhu (2020), where full-waveform inversion (FWI, Virieux and Operto (2009)) is combined with DA. Our choice for a scalable ensemble filtering type of approach (Herrmann 2023; Gahlot et al. 2023; Spantini, Baptista, and Marzouk 2022) is supported by recent work of Bruer et al. (2024), who conducted a careful study on the computational complexity of realistic CO₂ projects. This work is aimed at further improving several aspects of ensemble-filter approaches including (i) incorporation of stochasticity in the state dynamics by treating the permeability distribution as a random variable newly drawn after each time-lapse timestep; (ii) removal of reliance on limiting Gaussian assumptions and linearizations of the observation operator; (iii) replacement of the crucial linear Kalman correction step by a learned nonlinear mapping similar to the coupling approach recently proposed by Spantini, Baptista, and Marzouk (2022); and (iv) extending the approach to multimodal time-lapse data.

To meet these additional complexities and evident nonlinearities, we develop a new neural sequential Bayesian inference framework based on Variational Inference (Jordan et al. 1998) with Conditional Normalizing Flows (CNFs, Rezende and Mohamed (2015); Papamakarios et al. (2021); Radev et al. (2020)). As in the work by Spantini, Baptista, and Marzouk (2022), nonlinearities in both the state and observation operators are accounted for by a prior-to-posterior mapping through the latent space of a nonlinear transform, given by a CNF in this work. However, by allowing the permeability field, which emerges in the state for the fluid flow, to appear as a stochastic nuisance parameter, our proposed formulation sets itself apart from existing work. By using the fact that CNFs implicitly marginalize over nuisance parameters during training (Cranmer, Brehmer, and Louppe 2020; Radev et al. 2020), stochasticity due to the lack of knowledge on the permeability field is factored into the uncertainty captured by our neural approximation for the posterior distribution of the CO₂ plume’s state.

Finally, the presented approach provides an alternative to “pure” machine-learning approaches, where neural networks are either trained on pairs saturation/seismic shot records (Um et al. 2023), on reservoir models and saturation plumes (Tang et al. 2021), or on “time-lapse movies” (X. Huang, Wang, and Alkhalifah 2024), to produce estimates for the dynamic state with uncertainties based on Mont-Carlo dropout. Aside from the argument that this type of uncertainty quantification may not be Bayesian (Folgoc et al. 2021), our proposed approach is solidly anchored by the fields of DA, where high-fidelity simulations for the state and observations are used to recursively create forecast ensembles, and by SBI (Cranmer, Brehmer, and Louppe 2020; Radev et al. 2020), where advantage is taken of the important concept of summary statistics (Orozco et al. 2023; Orozco, Siahkoohi, et al. 2024) to simplify the complex relationship between the state (saturation/pressure) and seismic observations (shot records). Instead of relying on the network to provide the uncertainty, our deployment of CNFs aims to learn the Bayesian posterior distribution directly that includes both aleatoric, by adding additive noise to the observed time-lapse data, and epistemic uncertainty, by treating the permeability field as a stochastic nuisance parameter. While our approach can be extended to include network-related epistemic uncertainty (Qu, Araya-Polo, and Demanet 2024), we leave this to future work to keep our approach relatively simple.

The Digital Shadow for underground CO₂-storage monitoring is aimed to demonstrate how learned nonlinear Bayesian ensemble filtering techniques can be used to characterize the state from multimodal time-lapse monitoring data. Our work is organized as follows. In Section 2, we start by introducing the main ingredients of nonlinear ensemble filtering, followed in Section 3 by an exposition on how techniques from amortized Bayesian inference (Cranmer, Brehmer, and Louppe 2020; Radev et al. 2020) can be used to derive our nonlinear extension of ensemble filtering based on CNFs. After specifying algorithms for Sequential Simulation-based Bayesian Inference and Forecast, Training, and Analysis, which includes an introduction of the important aspect of summary statistics, we proceed by illustrating the “lifecycle” of the proposed Digital Shadow, which includes the important prior-to-posterior mapping that corrects the predictions for the state from time-lapse observations. Next, we outline a detailed procedure to validate the proposed Digital Shadow prototype in a realistic off-shore setting representative of the South-Western North Sea. In addition to detailing this setting, which includes descriptions of the probabilistic baseline, ground-truth simulations, and monitoring with the Digital Shadow, Section 4 also introduces certain performance metrics. Guided by these metrics, a case study is presented in Section 5 with an emphasis on examining the advantages of working with multimodal data. This case study is followed by Section 6 in which the Digital Shadow is put in a broader context, followed by conclusions in Section 7.

While the presented framework is designed to be general, the focus will be on geophysical monitoring of Geological Carbon Storage (GCS) aimed at keeping track of the dynamics of CO₂ plumes via a combination of multi-phase flow simulations and time-lapse multimodal geophysical measurements, consisting of sparse localized direct measurements of the saturation and pressure perturbations (shown in figure 12) collected within boreholes of (monitoring) wells and indirect active-source seismic measurements, collected at the surface. In this scenario, the time-varying state-dynamics are measured at discrete time instants, $k=1\cdots K$, and consists of the time-varying spatial distribution for the CO₂ saturation, $\mathbf{x}_{k}[S_{\mathrm{CO_{2}}}]$, and induced pressure perturbation, $\mathbf{x}_{k}[\delta p]$, which can be expressed as follows:

In this expression for the uncertain nonlinear dynamics, the possibly time-dependent dynamics operator, $\mathcal{M}_{k}$, represents multi-phase fluid-flow simulations that transition the state from the previous timestep, $t_{k-1}$, to the next timestep, $t_{k}$, for all $K$ time-lapse timesteps. The dynamics operator, $\mathcal{M}_{k}$, which implements the underlying continuous dynamics between times $t_{k-1}$ and $t_{k}$, may itself be time-dependent because of induced chemical and/or rock mechanical changes. Because the spatial distribution of the permeability field, collected in the vector $\boldsymbol{\kappa}$, is highly heterogeneous (Ringrose 2020, 2023), unknown, and possibly time-dependent, it is treated as a stochastic nuisance parameter (Radev et al. 2020). Without loss of generality, we assume samples for the spatial gridded permeability field to be drawn from the time-invariant probability distribution—i.e, $\boldsymbol{\kappa}\sim p(\boldsymbol{\kappa})$ where $p(\boldsymbol{\kappa})$ is assumed to be independent of the state and time-lapse observations. The dynamics operator takes samples from the spatial distribution for the subsurface permeability, $\boldsymbol{\kappa}$, and the CO₂ saturation and reservoir pressure perturbations, collected at the previous timestep $k-1$ in the vector, $\mathbf{x}_{k-1}$, as input and produces samples for the CO₂ saturation and reservoir pressure perturbations at the next timestep, $\mathbf{x}_{k}$, as output. While this procedure offers access to samples from the transition density, the transition density itself can not be evaluated because of the presence of the stochastic nuisance parameter, $\boldsymbol{\kappa}$. To leading order, the nonlinear dynamics consists of solutions of Darcy’s equation and mass conservation laws (Pruess and Nordbotten 2011; Li et al. 2020). Despite the fact that the dynamics can be modeled accurately with reservoir simulations (Krogstad et al. 2015; Settgast et al. 2018; Rasmussen et al. 2021; Stacey and Williams 2017), relying on these simulations alone is unfeasible because of a lack of knowledge on the permeability.

To improve estimates for the state, CO₂ plumes are monitored by collecting discrete time-lapse well (Freifeld et al. 2009) and surface-seismic data (D. Lumley 2010). We model these multimodal time-lapse datasets as

where the vector, $\mathbf{y}_{k}$, represents the observed multimodal data at timestep $k$. To add realism, the seismic data is corrupted by the noise term, $\boldsymbol{\epsilon}_{k}$, sampled by filtering realizations of standard zero-centered Gaussian noise with a bandwidth-limited seismic source-time function. This filtered noise is normalized to attain a preset signal-to-noise ratio before it is added to the seismic shot records prior to reverse-time migration. The shot records themselves are generated with 2D acoustic simulations with the seismic properties (velocity and density) linked to the CO₂ saturation through the patchy saturation model¹¹1To arrive at this acquisition geometry, use was made of reciprocity—i.e., the computational experiments are calculated for $16$ sources, reducing the computational costs. (Avseth, Mukerji, and Mavko 2010). The vector, $\mathbf{\bar{m}_{k}}=(\mathbf{\bar{v}},\mathbf{\bar{\rho}})$ with $\mathbf{\bar{v}}$ and $\mathbf{\bar{\rho}}$ the reference velocity and density, represents a time-dependent seismic reference model from which the migration-velocity model for imaging will be derived and with respect to which the time-lapse differences in the seismic shot records will be calculated. According to these definitions, the observation operator, $\mathcal{H}_{k}$, models multimodal, direct and indirect, time-lapse monitoring data, collected at the (monitoring) wells and migrated from surface-seismic shot records. While this observation operator may also depend on time, e.g. because the acquisition geometry changes, we assume it, like the dynamics operator, to be time-invariant, and drop the time index $k$ from the notation.

Due to sparsely sampled receiver and well locations, and the non-trivial null-space of the modeling operators, the above measurement operator is ill-posed, which renders monitoring the CO₂ dynamics challenging, a situation that is worsened by the intrinsic stochasticity of the dynamics. To overcome these challenges, we introduce a nonlinear Bayesian filtering framework where the posterior distribution for the state, conditioned on the collected multimodal time-lapse data, is approximated by Variational Inference (Jordan et al. 1999; David M. Blei and McAuliffe 2017) with machine learning (Rezende and Mohamed 2015; Siahkoohi et al. 2023b; Orozco, Siahkoohi, et al. 2024), neural posterior density estimation to be precise. With this approach, which combines the latest techniques from sequential Bayesian inference and amortized neural posterior density estimation (Papamakarios et al. 2021), we aim to quantify the uncertainty of the CO₂ plumes in a principled manner.

Because both the dynamics and observation operators are nonlinear, we begin by defining a general probabilistic model for the state and observations in terms of the following conditional distributions:

and

for $k=0\cdots K$, representing, respectively, the transition density, defined by equation 1, and the data likelihood, given by equation 2. These conditional distributions hold because the above monitoring problem (cf. equations 1 and 2) is Markovian, which implies that the likelihood does not depend on the time-lapse history—i.e., $\left(\mathbf{y}_{k}\perp\mathbf{y}_{1:k-1}\right)\mid\mathbf{x}_{k}$ with the shorthand, $\mathbf{y}_{1:k}=\left\{\mathbf{y}_{1},\ldots,\mathbf{y}_{k}\right\}$, so that $p(\mathbf{y}_{k}\mid\mathbf{x}_{k},\mathbf{y}_{1:k-1})=p(\mathbf{y}_{k}\mid\mathbf{x}_{k})$. This orthogonality assumption holds as long as the nuisance parameters are independent with respect to the state (Rubio, Marzouk, and Parno 2023). The aim of Bayesian filtering is to recursively compute the posterior distribution of the state at time, $k$, conditioned on the complete history of collected time-lapse data—i.e., compute $p(\mathbf{x}_{k}\mid\mathbf{y}_{1:k})$. There exists a rich literature on statistical inference for dynamical systems (McGoff, Mukherjee, and Pillai 2015) aimed to accomplish this goal, which entails the following three steps (Asch 2022):

Initialization step: Choose the prior distribution for the state $p(\mathbf{x}_{0})$.

Prediction step: Compute the predictive distribution for the state derived from the Chapman-Kolmogorov equation yielding (Tatsis, Dertimanis, and Chatzi 2022)

This step uses equation 1 to advance samples of the distribution $p(\mathbf{x}_{k-1}\mid\mathbf{y}_{1:k-1})$ to samples of $p(\mathbf{x}_{k}\mid\mathbf{y}_{1:k-1})$.

Analysis step: Apply Bayes’ rule

with $p(\mathbf{y}_{k}|\mathbf{x}_{k})$ and $p(\mathbf{x}_{k}\mid\mathbf{y}_{1:k-1})$ acting as the “likelihood” and “prior”. During this step, the posterior distribution is calculated by correcting the “prior” with the “likelihood”. To this end, equation 4 will be used to draw samples from the likelihood.

These three steps are general and make no assumptions on the linearity and Gaussianity of the distributions involved. The general solution of the above Bayesian filtering problem is far from trivial for the following reasons: (i) equations 5 and 6 only permit analytical solutions in cases where the state-dynamics and observation operators are linear and Gaussian distributed; (ii) the presence of the nuisance permeability parameter in equation 1 requires access to the marginal likelihood, which is computationally infeasible (Cranmer, Brehmer, and Louppe 2020; Kruse et al. 2021a); (iii) likewise, the marginalization in equation 5 is also not viable computationally; and finally (iv) computing the above conditional probabilities becomes intractable in situations where the state is high dimensional and the state-dynamics and observational operators are computationally expensive to evaluate.

To address these challenges, numerous data-assimilation techniques have been developed over the years (Asch, Bocquet, and Nodet 2016). The ensemble Kalman filter (EnKF) (Evensen 1994; Carrassi et al. 2018) features amongst the most advanced techniques and is widely used for large nonlinear state-space and observation models. During the prediction step of the data-assimilation recursions, EnKF propagates a set of $M$ particles from the previous timestep, $\left\{\mathbf{x}^{(1)}_{k-1},\ldots,\mathbf{x}^{(M)}_{k-1}\right\}$, to the next timestep by applying the state-space dynamics (equation 1) to each particle, followed by generating simulated observation samples, $\left\{\mathbf{y}^{(1)}_{k},\ldots,\mathbf{y}^{(M)}_{k}\right\}$, based on the observation model of equation 2. During the subsequent analysis step, EnKF uses the pairs, $\{\mathbf{x}_{k}^{(m)},\mathbf{y}_{k}^{(m)}\}_{m=1}^{M}$, to calculate empirical Monte-Carlo based linear updates to the forecast ensemble. While EnKF avoids the need for linearization, calculation of gradients, and is relatively straightforward to scale, it relies on linear updates that prevent it from solving the above nonlinear and non-Gaussian Bayesian filtering problem accurately, even when increasing the ensemble size (Spantini, Baptista, and Marzouk 2022).

Instead of relying on linear mappings, we consider the analysis step in equation 6 as a trainable prior-to-posterior transformation and employ techniques from amortized neural posterior density estimation with CNFs (Kruse et al. 2021b; Gahlot et al. 2023; Herrmann 2023) to carry out the analysis step, an approach also taken by Spantini, Baptista, and Marzouk (2022), who proposes a closely related but different coupling technique. Our proposed approach derives from simulation-based inference (Cranmer, Brehmer, and Louppe 2020; Radev et al. 2020) with invertible CNFs, which extends EnKF by allowing for learned nonlinear prior-to-posterior density transformations. CNFs represent a class of invertible neural networks that, dependent on a condition term, convert samples from a complicated distribution, through a series of nonlinear differentiable transformations, into samples from a simpler distribution, e.g. the standard Gaussian distribution. Instead of explicit evaluations of the corresponding densities, and similar to EnKF workflows, access is only needed to samples from the “likelihood” and “prior”, available through simulations with equations 1 and 2.

Recent developments in generative Artificial Intelligence (genAI), where neural networks are trained as surrogates for high-dimensional (conditional) distributions, are key to the development and success of our uncertainty-aware Digital Shadow, which relies on capturing the posterior distribution from simulated samples for the state and observations. While any amortized neural network implementation capable of producing samples conditioned on samples for different observations will work in our setting, and this includes conditional neural networks based on conditional score-based diffusion (Batzolis et al. 2021), CNFs have our preference because they allow us to quickly sample from the posterior, $\mathbf{x}\sim p(\mathbf{x}\mid\mathbf{y})$, and conduct density calculations—i.e., evaluate the posterior density $p(\mathbf{x}\mid\mathbf{y})$ and its derivatives with respect to the states, $\mathbf{x}$, and observations, $\mathbf{y}$. This latter feature is necessary when designing Digital Twins capable of uncertainty-informed decisions (Alsing, Edwards, and Wandelt 2023), control (Gahlot et al. 2024), and Bayesian experimental design (Orozco, Herrmann, and Chen 2024). While non-amortized approaches may be feasible, amortization of the neural estimate of the posterior distribution is essential because it allows us to marginalize over stochastic nuisance parameters (the permeability field, $\boldsymbol{\kappa}$, in our case) while capturing its uncertainty from the ensemble of dynamic states and associated time-lapse observations.

Specifically, our goal is to capture the posterior distribution from pairs, $\{\mathbf{x}^{(m)},\mathbf{y}^{(m)}\}_{m=1}^{M}$ (time-dependence was dropped for simplicity). To achieve this goal, we make use of Bayesian Variational Inference (Rezende and Mohamed 2015) in combination with the ability to carry out simulations of the state dynamics and observations (cf. equations 1 and 2), which are of the type (Radev et al. 2020)

where $\mathcal{G}$ is a generic simulator that depends on samples of the prior, $\mathbf{x}\sim p(\mathbf{x})$, and nuisance parameters, $\boldsymbol{\zeta}\sim p(\boldsymbol{\zeta}\mid\mathbf{x})$, possibly conditioned on samples of the prior. These simulated samples give us implicit access to the likelihood. Given $M$ sampled pairs, $\{\mathbf{x}^{(m)},\mathbf{y}^{(m)}\}_{m=1}^{M}$, simulation-based inference with CNFs proceeds by training an invertible neural network to approximate the posterior distribution, $p(\mathbf{x}\mid\mathbf{y})$, by minimizing the forward Kullback–Leibler (KL) divergence between the true and the approximate posterior for all possible data, $\mathbf{y}$. The training objective reads (Radev et al. 2020; Siahkoohi et al. 2023a)

To arrive at this objective involving samples from the joint distribution for $\mathbf{x}$ and $\mathbf{y}$, use was made of the law of total probability that allows us to rewrite the marginalization over observations, $\mathbf{y}$, in terms of the expectation over samples from the joint distribution. The term $p(\mathbf{x}\mid\mathbf{y})$ can be dropped because it does not depend on the parameters, $\boldsymbol{\phi}$. For more details, see Siahkoohi et al. (2023a). The above objective is used to approximate the true posterior distribution as accurately as possible, $p_{\boldsymbol{\phi}}(\mathbf{x}\mid\mathbf{y})\approx p(\mathbf{x}\mid\mathbf{y})$ for all possible $\mathbf{x}$ and $\mathbf{y}$, drawn from the joint distribution, $p(\mathbf{x},\mathbf{y})$. Following Radev et al. (2020), the approximate posterior, $p_{\boldsymbol{\phi}}(\mathbf{x}\mid\mathbf{y})$ is parameterized with a conditional Invertible Neural Network (cINN, Ardizzone et al. (2019a)), $f_{\boldsymbol{\phi}}$, which conditioned on, $\mathbf{y}$, defines an invertible mapping.

With the change-of-variable rule of probability

with $\mathbf{J}_{f_{\boldsymbol{\phi}}}$ the Jacobian of $f_{\boldsymbol{\phi}}$ evaluated at $\mathbf{x}$ and $\mathbf{y}$, the objective of maximizing the likelihood in equation 8 becomes

Given the simulated training pairs, $\{\mathbf{x}^{(m)},\mathbf{y}^{(m)}\}_{m=1}^{M}$, maximization of the expectations in equation 9 can be approximated by minimization of the sum:

During derivation of this training loss for CNFs, use was made of the property: $\mathcal{N}(0,\mathbf{I})\propto\exp(-(1/2)\|\mathbf{z}\|_{2}^{2})$. In the ideal situation where the estimated network weights, $\widehat{\boldsymbol{\phi}}$, attain a global minimum of the objective in equation 8, the latent distribution will be statistically independent of the conditioning data (Radev et al. 2020). In that case, samples from the posterior can be drawn based on the following equivalence:

According to this equivalence, sampling from the posterior corresponds to generating samples from the multidimensional standard Gaussian distribution, $\mathbf{z}\sim\mathcal{N}(0,\mathbf{I})$, followed by computing $f^{-1}_{\widehat{\boldsymbol{\phi}}}(\mathbf{z};\mathbf{y})$. In the next section, we will employ and adapt these training and inference phases for the less-than-ideal recursive training regime of data-assimilation problems where the ensemble sizes are relatively small for which the global minimum of equation 8 may not be attained.

After randomized initialization of $M$ particles of the CO₂ plume’s initial state, $\widehat{\mathbf{x}}^{(m)}_{0}\sim p(\mathbf{x}_{0})$ (line 2 of Algorithm 1 ), training pairs for the state (equation 1) and observations (equation 2), $\{\mathbf{x}_{k}^{(m)},\mathbf{y}_{k}^{(m)}\}_{m=1}^{M}$, for the next time step, are predicted by simulations with the state dynamics and detailed in the Forecast step of Algorithm 2 ). For notational convenience, we already introduced the symbol, $\widehat{\quad}$, to distinguish between predicted “digital states” and analyzed predictions (denoted by the $\widehat{\quad}$) for the state conditioned on the field observed data. During the Forecast step of Algorithm 2 , random samples of the stochastic permeability field and noise are drawn and used to draw samples from the transition density (line 3 of Algorithm 2 ), conditioned on analyzed samples, $\{\widehat{\mathbf{x}}_{k}^{(m)}\}_{m=1}^{M}$, from the previous time step. These predictions for the advanced state are subsequently employed to sample observations (line 4 of Algorithm 2 ), producing the simulated training ensemble, $\{\mathbf{x}_{k}^{(m)},\mathbf{y}_{k}^{(m)}\}_{m=1}^{M}$. These simulated pairs are utilized during the Training step (line 6 of Algorithm 1 ) where equation 10 is minimized with respect to the CNF’s network weights, yielding $\widehat{\boldsymbol{\phi}}_{k}$. Given these network weights, $M$ latent-space representations for each pair are calculated via

for $m=1\cdots M$ (Spantini, Baptista, and Marzouk 2022). Given observed field data, $\mathbf{y}_{k}^{\mathrm{obs}}$, collected at time $k$ (line 7 of Algorithm 1 ), these latent variables are used to correct the state for $m=1\cdots M$ via

In this expression, $\{\widehat{\mathbf{x}}_{k}^{(m)}\}_{m=1}^{M}$, corresponds for $k\geq 1$ to analyzed states, produced by the prior-to-posterior mapping (Algorithm 2 in Spantini, Baptista, and Marzouk (2022)). To offset the potential effects of working with a limited ensemble size, $M$, we choose to adapt this approach of producing samples from the latent space calculated from the forecast ensemble. According to Spantini, Baptista, and Marzouk (2022), this process mitigates the effects when a global minimum of equation 10 is not attained, implying that $p_{\widehat{\boldsymbol{\phi}}}(\mathbf{z})\perp\mathbf{x}$ may no longer hold. In that case, the latent output distribution is no longer guaranteed to be the same for any fixed, $\mathbf{y}$ (Radev et al. 2020). By producing latent samples that derive directly from the predicted state-observation pairs, this issue is resolved. When conditioned on the observed field data, $\mathbf{y}_{k}^{\mathrm{obs}}$, these latent samples improve the inference after running the CNF in the reverse direction (cf. equation 13). Algorithm 2 summarizes the three key phases, namely the Forecast, Training, and Analysis steps, which are used during the recursions in Algorithm 1 that approximate equations 5 and 6. During these recursions, for timesteps $k>1$, samples of the posterior distribution from the previous timestep, $\{\widehat{\mathbf{x}}^{(m)}_{k-1}\}_{m=1}^{M}$, serve as “priors” for the next timestep, producing, $\{\widehat{\mathbf{x}}^{(m)}_{k}\}_{m=1}^{M}$, which are conditioned on the observations, $\mathbf{y}_{k}^{\mathrm{obs}}$. This procedure corresponds to the nonlinear prior-to-posterior of Spantini, Baptista, and Marzouk (2022).

To illustrate the different stages of the Digital Shadow’s recursions, figure 2 is included to depict its “lifecycle” at the first timestep, $k=1$. The lifecycle entails the following stages. Samples for the initial state of the CO₂ saturation, denoted with a slight abuse of notation by $\widehat{\mathbf{x}}^{(m)}_{0}\sim p(\mathbf{x}_{0})$, are drawn first from the uniform distribution, $\widehat{\mathbf{x}}^{(m)}_{0}\sim\mathcal{U}(0.1,\,0.6)$ (figure 2 (top-left)). By simulating the state-dynamics equation 1 (figure 2 top-row middle), predictions for the state at the next timestep are sampled, $\mathbf{x}_{k}\sim p(\mathbf{x}_{k}|\widehat{\mathbf{x}}_{k-1})$. Given these predicted samples for the state, corresponding samples for the observations are simulated with equation 2 (figure 2 top-row right). These two types of simulations make up the Forecast step, which produces the predicted training ensemble, $\bigl{\{}\mathbf{x}_{k},\mathbf{y}_{k}\bigr{\}}_{m=1}^{M}$, consisting of $M$ samples drawn from the joint distribution, $\mathbf{y}_{k}\sim p(\mathbf{x}_{k},\mathbf{y}_{k})$. This predicted ensemble serves as input to the Training step (figure 2 bottom right) during which the weights of the Digital Shadow’s neural networks are trained by minimizing equation 9. This produces the optimized network weights, $\widehat{\boldsymbol{\phi}}_{k}$, at timestep, $k$. After collecting field observations, $\mathbf{y}_{k}^{\mathrm{obs}}$, posterior inference is conducted with the neural posterior density estimator, $p_{\widehat{\boldsymbol{\phi}}_{k}}(\widehat{\mathbf{x}}_{k}\mid\mathbf{y}_{k}^{\mathrm{obs}})$, conditioned on the observed field data, $\mathbf{y}_{k}^{\mathrm{obs}}$. To avoid finite sample size and training effects during the Analysis step (figure 2 lower left), a sample from the posterior is drawn by evaluating the CNF in the forward latent-space direction on the predicted pair, $(\mathbf{x}_{k},\mathbf{y}_{k})$, yielding $\mathbf{z}_{k}=f_{\widehat{\boldsymbol{\phi}}_{k}}(\mathbf{x}_{k};\mathbf{y}_{k})$. This operation is followed by running the CNF, conditioned on the observed field data, $\mathbf{y}_{k}^{\mathrm{obs}}$, in the reverse direction, yielding the sample $\widehat{\mathbf{x}}_{k}=f^{-1}_{\widehat{\boldsymbol{\phi}}_{k}}(\mathbf{z}_{k};\mathbf{y}_{k}^{\mathrm{obs}})$. By design, this sample approximates a sample from the true posterior, $\mathbf{x}_{k}\sim p(\mathbf{x}_{k}\mid\mathbf{y}_{k}^{\mathrm{obs}})$. At the next timestep, this sample becomes a sample for the “prior”—i.e., for $k\rightarrow k+1$, and the recursions proceed by repeating themselves for all, $K$, timesteps. By comparing figure 2’s sample for the plume prediction, $\mathbf{x}_{k}\sim p(\mathbf{x}_{k}|\widehat{\mathbf{x}}_{k-1})$, with the corresponding analyzed sample conditioned on the observed field data, $\widehat{\mathbf{x}}_{k}\sim p_{\widehat{\boldsymbol{\phi}}_{k}}(\widehat{\mathbf{x}}_{k}\mid\mathbf{y}_{k}^{\mathrm{obs}})$, it becomes apparent that the predicted state produced by the Forecast step undergoes a significant correction during the Analysis step. This mapping during the Analysis step between the predicted and inferred states—i.e., $\mathbf{x}_{k}\mapsto\widehat{\mathbf{x}}_{k}$, derives from the invertibility of CNFs and the implicit conditioning of the predictions on $\mathbf{y}^{\mathrm{obs}}_{k-1}$ (via $\widehat{\mathbf{x}}_{k-1}$) and explicit conditioning on $\mathbf{y}^{\mathrm{obs}}_{k}$. Despite the fact that the state at the previous timestep is conditioned on $\mathbf{y}^{\mathrm{obs}}_{k-1}$, the prediction varies considerably. This variability is caused by the unknown permeability field, which appears as a stochastic nuisance parameter in the dynamics (cf. equation 1), and explains the significant time-lapse data-informed correction performed by the Analysis step. The importance of the correction is nicely illustrated by the example in figure 2 where thanks to the correction the plume is extended significantly in the leftward direction, a feature completely missed by the predicted state.

By running through the Digital Shadow’s lifecycle $k$ times recursively for an ensemble consisting of $M$ particles, the Prediction and Analysis steps approximate samplings from $p(\mathbf{x}_{k}\mid\mathbf{y}^{\mathrm{obs}}_{1:k-1})$ and $p(\mathbf{x}_{k}\mid\mathbf{y}^{\mathrm{obs}}_{1:k})$ of equations 5 and 6. Because the proposed posterior density estimation scheme recurrently trains networks, it can be considered as an instance of neural sequential Bayesian inference, which forms the basis of our Digital Shadow. Before going into further details on how the simulations are performed, we introduce the important concept of summary statistics, which simplifies the observable-to-state mapping, renders typically non-uniform seismic data space uniform, and also learns to detect features that inform the posterior.

Despite the adequate expressiveness of CNF’s neural architecture based on cINNs (Ardizzone et al. 2019b), the mapping of CO₂ plume-induced acoustic changes to seismic shot records remains extremely challenging because of the large degrees of freedom of Earth models and seismic shot records (Cranmer, Brehmer, and Louppe 2020), which live in three dimensions and five dimensions for 2- and 3D- Earth models, respectively. This challenge is compounded by the relatively weak signal strength of time-lapse changes, the complexity of the time-lapse changes to seismic observations map, and the fact that raw seismic shot records are collected non-uniformly—i.e. different seismic surveys may have different numbers of shot records or shot records may vary in the number of traces (receivers), time samples, etc. To overcome these issues, we introduce a hybrid approach, consisting of physics-based summary statistics (Radev et al. 2020; Orozco et al. 2023), and a trainable U-net (Ronneberger, Fischer, and Brox 2015) that forms the conditional arm of the cINN (Ardizzone et al. 2019a). By mapping the shot records to the image domain, uniformity is assured because the discretization can be controlled in that domain. Moreover, the mapping from the image domain to the velocity model is much simpler, so the conditional arm of the cINN can dedicate itself to learning image-domain features that maximally inform the posterior while the network’s invertible arm can concentrate on learning how to produce samples of the posterior from realizations of standard Gaussian noise conditioned on these features. More importantly, Alsing, Wandelt, and Feeney (2018) also showed that under certain conditions the gradient of the score function (the $\log$-likelihood) compresses physical data optimally. Motivated by this work, Orozco et al. (2023) demonstrated that information in the original shot records can be preserved during pre-stack reverse-time migration (RTM) as long as the migration-velocity model is sufficiently accurate. Yin, Orozco, et al. (2024) showed that this condition can be relaxed in the WISE and WISER (Yin, Orozco, and Herrmann 2024) frameworks. The network weights for the U-net in the conditional arm are absorbed in the network weights, $\boldsymbol{\phi}$, of the CNFs, and trained during the minimization of the objective in equation 10.

To meet the challenges of imaging subtle time-lapse changes induced by the replacement of brine by supercritical CO₂, we calculate the residual with respect to a reference model, $\mathbf{\bar{m}}_{k}$. To avoid tomographic artifacts in the gradient, the residual is migrated with the inverse-scattering imaging condition (Stolk, Hoop, and Op’t Root 2009; Douma et al. 2010; Whitmore and Crawley 2012; P. Wang et al. 2024). This corresponds to computing sensitivities with respect to the acoustic impedance for a constant density. Given this choice, the observation operator linking the state of the CO₂ plume to migrated time-lapse seismic images is given by

In this expression, the linear operator $\Bigl{(}{\mathbf{J}_{\mathcal{F}[\mathbf{v}]}}\Bigr{|}_{{\mathbf{\widetilde{v}}_{k}}}\Bigr{)}_{\mathsf{ic}}$ corresponds the Jacobian of the forward wave operator, $\mathcal{F}[\mathbf{m}]$, calculated at $\mathbf{\widetilde{v}}_{k}$, which is a smoothed version of the reference velocity model, $\mathbf{\bar{v}_{k}}$. The symbol ^⊤ denotes the adjoint. Together with the reference densities, the reference wavespeeds are collected in the seismic reference model, $\bar{\mathbf{m}}_{k}$. The nonlinear forward model, $\mathcal{F}$, represents the active-source experiments part of the survey. Use of the inverse-scattering imaging condition is denoted by the subscript, $\mathsf{ic}$ in the definition of the reverse-time migration operator, $\Bigl{(}{\mathbf{J}_{\mathcal{F}[\mathbf{v}]}}\Bigr{|}_{{\mathbf{\widetilde{v}}_{k}}}\Bigr{)}^{\top}_{\mathsf{ic}}$. Finally, the vector, $\mathbf{d}_{k}$, collects the simulated shot records at time-lapse timestep, $k$, for all active-source experiments and is simulated with

In this nonlinear forward model at timestep, $k$, the CO₂saturation induced corrected time-lapse seismic medium properties—i.e., the wavespeed, $\mathbf{v}_{k}$, and the density, $\boldsymbol{\rho}_{k}$, are computed with respect to the baseline, $\mathbf{m}_{0}$, via $\mathbf{m}_{k}=\mathcal{R}({\mathbf{m}_{0},\mathbf{x}_{k}})$. This nonlinear operator, $\mathcal{R}(\cdot,\cdot)$, captures the rock physics, which models how changes in the state—i.e., the CO₂ saturation, map to changes in the seismic medium properties, $\mathbf{m}_{k}$. Because “observed field” seismic time-lapse data is assumed to come from the same forward model, equation 14 will also be used to in silico simulate $\mathbf{y}_{k}^{\mathrm{obs}}$, simply by replacing the simulated time-lapse shot records, $\mathbf{d}_{k}$, by the simulated observed time-lapse seismic shot records, $\mathbf{d}_{k}^{\mathrm{obs}}$, that stand for data collected in the field.

Given the exposition of the amortized Bayesian inference, the neural sequential simulation-based Bayesian inference, and physics-based summary statistics, we are now in a position to setup our Digital Shadow for GCS prior to conducting our case study.

To mimic a realistic CO₂ storage project where reservoir properties are not known precisely, a stochastic baseline for the permeability field is established. To this end, the velocity model depicted in figure 1 is used to generate noisy baseline seismic data (signal-to-noise ratio S/N $12\mathrm{dB}$). This seismic baseline dataset serves as input to full-waveform variational Inference via Subsurface Extensions (WISE (Yin, Orozco, et al. 2024)). Given a poor 1D migration-velocity model and noisy shot records, WISE draws samples from the posterior distribution for the velocity. These samples for the wavespeed conditioned on shot data are plotted in figure 3 (a) and are converted to the permeability fields plotted in figure 3 (b). This elementwise velocity-to-permeability conversion is done with the following nonlinear elementwise mapping: