Identification and estimation of causal effects using non-concurrent controls in platform trials

Michele Santacatterina Corresponding author: [email protected]. This article is based upon work supported by the National Science Foundation under Grant No 2306556 Federico Macchiavelli Giron Xinyi Zhang and Iván Díaz
Division of Biostatistics Department of Population Health
New York University School of Medicine
New York NY 10016

Abstract

Platform trials are multi-arm designs that simultaneously evaluate multiple treatments for a single disease within the same overall trial structure. Unlike traditional randomized controlled trials, they allow treatment arms to enter and exit the trial at distinct times while maintaining a control arm throughout. This control arm comprises both concurrent controls, where participants are randomized concurrently to either the treatment or control arm, and non-concurrent controls, who enter the trial when the treatment arm under study is unavailable. While flexible, platform trials introduce the challenge of using non-concurrent controls, raising questions about estimating treatment effects. Specifically, which estimands should be targeted? Under what assumptions can these estimands be identified and estimated? Are there any efficiency gains? In this paper, we discuss issues related to the identification and estimation assumptions of common choices of estimand. We conclude that the most robust strategy to increase efficiency without imposing unwarranted assumptions is to target the concurrent average treatment effect (cATE), the ATE among only concurrent units, using a covariate-adjusted doubly robust estimator. Our studies suggests that, for the purpose of obtaining efficiency gains, collecting important prognostic variables is more important than relying on non-concurrent controls. We also discuss the perils of targeting ATE due to an untestable extrapolation assumption that will often be invalid. We provide simulations illustrating our points and an application to the ACTT platform trial, resulting in a 20% improvement in precision.

Keywords: adaptive trials; causality; doubly robust; efficiency; estimand

1 Introduction

Platform trials are multi-arm designs that simultaneously evaluate multiple treatments for a single disease within the same overall trial structure (Woodcock & LaVange, 2017; Berry et al., 2015; Park et al., 2022). Unlike traditional randomized controlled trials, they allow treatment arms to enter and exit the trial at distinct times while maintaining a control arm throughout. These trials have been instrumental in assessing the efficacy of treatments across various therapeutic areas (Barker et al., 2009; Foltynie et al., 2023; Wells et al., 2012, among others) and gained traction during the COVID-19 pandemic (Hayward et al., 2021; Angus et al., 2020; Kalil et al., 2021, among others). For instance, the Adaptive COVID-19 Treatment Trial (ACTT) (Kalil et al., 2021) was a platform trial that investigated treatments for hospitalized adult patients with COVID-19 pneumonia. ACTT comprised of multiple stages, as depicted in Figure 1. In the initial stage (ACTT-1), the efficacy of remdesivir alone versus placebo was evaluated. Subsequently, in the second stage (ACTT-2), placebo was discontinued, and a new treatment, remdesivir plus baricitinib, was introduced while concurrently randomizing participants to remdesivir alone. Here, the remdesivir alone arm served as a shared arm between the ACTT-1 and ACTT-2 stages. The remdesivir alone arm is termed non-concurrent for remdesivir plus baricitinib during ACTT-1 and concurrent during ACTT-2. In this paper, we adhere to the terminology used in current literature (Bofill Roig et al., 2023; Lee & Wason, 2020) and designate the shared arm as control, irrespective of whether it is a placebo arm or an active control or an experimental treatment. Thus, we consistently use the terms concurrent and non-concurrent controls regardless of the nature of the shared arm.

Refer to caption — Figure 1: Adaptive COVID-19 Treatment Trial (ACTT) schema. Example of concurrent and non-concurrent arm

The central question revolves around the efficient utilization of non-concurrent controls to estimate treatment effects in platform trials. Specifically, what estimands should be targeted to evaluate the causal effect of a treatment versus a shared control? Under what assumptions can these estimands be identified and estimated? Does using non-concurrent controls lead to efficiency gains?

Addressing these questions requires careful consideration of how the timing of entry into the platform trial may introduce bias into the study results, which is referred to as “time drift”, “temporal drift” or “time trend”. Various methods have been proposed to control for it, including test-then-pool approaches (Viele et al., 2014), frequentist and Bayesian regression models (Lee & Wason, 2020; Sridhara et al., 2022; Bofill Roig et al., 2023; Saville et al., 2022), propensity-score-based methods (Yuan et al., 2019; Chen et al., 2020), and other approaches (Han et al., 2017; Collignon et al., 2020; Ibrahim & Chen, 2000; Neuenschwander et al., 2009; Banbeta et al., 2019; Gravestock et al., 2017; Bennett et al., 2021; Hobbs et al., 2011; Normington et al., 2020; Schmidli et al., 2020; Hupf et al., 2021; Jiang et al., 2023).

While these methods provide a statistical way to incorporate non-concurrent controls and control for the “temporal drift” bias, these approaches are “model-first”, meaning that they are focused on first providing a model for the outcome and then reverse engineering interpretations for the estimated parameters in terms of causal effects. This approach conflicts with the recently advocated estimand framework (FDA, 2021) and the International Council for Harmonisation (ICH) E9(R1) guidance (International Council for Harmonisation, 2017), where the causal target of interest is first identified based solely on scientific discussions, and then the optimal statistical estimation method for that target is deployed. Existing methods for the use of non-concurrent controls lack a formal framework for characterizing causal effects and their identifying conditions, which implies that interpreting the effect estimates from these procedures and making recommendations regarding clinical practice become challenging. These concerns are underscored in recent reviews (Collignon et al., 2022; Koenig et al., 2024) and are discussed in the FDA estimand framework (FDA, 2021) and the ICH E9(R1) guidance (International Council for Harmonisation, 2017).

In this paper, we discuss the use of non-concurrent controls, using the estimand framework to guide our discussion and choices. We propose to target the concurrent average treatment effect of treatment arm $k$ , $\mathsf{cATE}(k)$ , as an estimand of interest in platform trials. Specifically, $\mathsf{cATE}(k)$ is the marginal average difference in outcomes for individuals who receive treatment $k$ compared to those in the shared control group, among the concurrent population. We then provide assumptions for its non-parametric identification, and show that these assumptions are all feasible, in contrast to the assumptions required for identification of the average treatment effect, which are not testable. We develop several estimators for $\mathsf{cATE}(k)$ , including outcome regression, inverse probability weighting, and doubly robust estimators. We show that efficiency gains can be obtained by leveraging non-concurrent controls for estimators based on outcome regression under correct models specification. Interestingly, we also show that there are no asymptotic gains in efficiency when using non-concurrent controls with doubly robust estimators adjusted by time of entry into the trial when treatment availability is a deterministic function of entry time. However, we show that efficiency gains can be obtained when treatment availability is a stochastic function of entry time.

In randomized trials, efficiency gains may come from multiple sources. For instance, one can attempt to gain efficiency by increasing the sample size, as illustrated by the use of non-concurrent controls. Alternatively, precision may be increased through adjustment for prognostic variables (see Colantuoni & Rosenblum, 2015; Benkeser et al., 2021, among others). Prognostic variables are often incorporated though regression models, which can then be mapped into conditional or marginal effect estimates. However, it is important to remember that outcome models may lead to biased results under certain types of misspecification. It is therefore important to use doubly robust estimators for covariate adjustment. Doubly robust estimators are consistent when either the treatment assignment or the outcome model is correctly specified, a property we obtain “for free” in platform trials due to randomization. Therefore, a key takeaway when targeting $\mathsf{cATE}(k)$ in platform trials is to use a doubly robust estimator that prioritizes identifying strong prognostic baseline variables rather than relying on non-concurrent controls. The latter provide no efficiency benefit when using a robust estimator that does not rely on the ability to correctly specify the outcome regression mechanism when treatment availability is a deterministic function of entry time.

Finally, we further highlight the risks of targeting the average treatment effect using data for the entire duration of the trial including the non-concurrent period, due to its dependence on an untestable extrapolation assumption.

2 Notation and setup

For each of $i\in\{1,\ldots,n\}$ study participants, let $E_{i}$ denote the (random) entry time, after eligibility screening and consent, of a unit into the study, let $W_{i}$ denote a set of baseline variables, let $A_{i}$ denote the randomized treatment taking values $k=0,1,\ldots,K$ , where $0$ denotes the control arm and $1,\ldots,K$ denotes the treatments of interest. Let $V_{k,i}$ denote an indicator of whether arm $k$ was available at time $E_{i}$ , and define $V_{i}=(V_{0,i},\ldots,V_{K,i})$ . Let $Y_{i}$ denote a binary or numerical outcome measured at a fixed time after entry $E_{i}$ . The observed data is $D=(Z_{1},\ldots,Z_{n})$ , where $Z_{i}$ represents the data for the experimental unit $i$ , i.e., $Z_{i}=(E_{i},W_{i},V_{i},A_{i},Y_{i})\sim\mathsf{P}$ . We define $V_{0}=V_{1}=\cdots=V_{j}=1$ with probability one so that at least $j$ treatments plus control are available at the start of the trial. We also assume the data are ordered in time of study entry in the sense that $E_{1}\leq E_{2}\leq\cdots\leq E_{n}$ . Note that $\mathsf{P}(A_{i}=k\mid V_{i,k}=0)=0$ by design.

2.1 A structural causal model and associated DAG

To encapsulate the role of entry time and non-concurrent controls in platform trials, we posit the structural causal model and directed acyclic graph (DAG) (Pearl, 1995) represented in Figure 2, and its interpretation in terms of a non-parametric structural equation model in eq. (1) respectively.

Figure 2: DAG associated to the structural equation model in equation (1).

$\displaystyle E_{i}$	$\displaystyle=f_{E}(U_{E,i}),$
$\displaystyle W_{i}$	$\displaystyle=f_{W}(E_{i},U_{W,i}),$
$\displaystyle V_{k,i}$	$\displaystyle=f_{V_{k}}(E_{i},U_{V_{k},i}),$	(1)
$\displaystyle A_{i}$	$\displaystyle=f_{A}(V_{k,i},W_{i},U_{A,i}),$
$\displaystyle Y_{i}$	$\displaystyle=f_{Y}(A_{i},W_{i},E_{i},U_{Y,i}).$

We now discuss some important features of Model (1). Model (1) allows all variables to be dependent, directly or through other variables, on entry time $E_{i}$ , and therefore appropriately models temporal drifts. It also allows the treatment assignment $A_{i}$ to depend on the participant’s covariates $W_{i}$ , thus allowing study designs such as stratified randomization (Broglio, 2018). Model (1) also imposes some exclusion restrictions. First, the treatment assignment $A_{i}$ is not allowed to depend on the entry time $E_{i}$ other than through treatment availability $V_{k,i}$ . In other words, a participant entering the study at time $E_{i}$ can only be assigned to available treatments at that time, but the randomization probability of a treatment that is available for assignment does not vary in time. Second, the outcome for unit $i$ , $Y_{i}$ , is not allowed to depend on the availability of treatments $V_{k,i}$ , other than through the treatment actually given to unit $i$ , but it is allowed to directly depends on unit’s $i$ entry time. Third, the availability of treatments $V_{k,i}$ does not depend on covariates $W_{i}$ . The two last assumptions are reasonable assumptions since the treatments under evaluation do not often depend on trial data. In this paper, we assumed that covariates $W_{i}$ depend on entry time $E_{i}$ , and not the other way around. This assumption is reasonable because, in many platform trials, entry time does not depend on individual-level data. Furthermore, it can be shown that the results presented in the next sections also hold when entry time $E_{i}$ depends on $W_{i}$ . In Model (1), the functions $f_{E}$ , $f_{W}$ , and $f_{Y}$ are completely unknown, thus making the model non-parametric, while the treatment assignment function, $f_{A}$ , and treatment availability function, $f_{V_{k}}$ , are known by design. In addition, the random variables $U_{E,i}$ , $U_{W,i}$ , and $U_{Y,i}$ are unmeasured factors that impact the entry time, covariates, and outcomes, respectively. The random variables $U_{A,i}$ control the randomization probabilities and are known by design. The random variables $U_{V_{k}}$ represent all factors that determine the availability of treatments for subjects in the trial.

In the following section, under this model and its associated DAG, we define the concurrent average treatment effect as the causal estimand of interest and introduce its identification assumptions, aligning with the estimand framework advocated by the FDA (FDA, 2021).

3 Definition and identification of the concurrent average treatment effect

In this paper, we focus on endpoints measured at fixed time-points post-randomization. Additionally, we consider an intention-to-treat (ITT) analysis. Our results can be easily extended to binary endpoints. We define the concurrent average treatment effect of treatment $k$ against a shared control arm in terms of counterfactual variables (Pearl, 2010), $Y_{i}(k)=f_{Y}(k,V_{k,i},W_{i},E_{i},U_{Y,i})$ , where $k=0,\dots,K$ , that would have been observed in a hypothetical world where treatment $A_{i}=k$ had been given, i.e., $\mathsf{P}(A_{i}=k)=1$ . We first define it and then discuss their non-parametric identification.

Definition 1 (Conditional and marginal average treatment effect of treatment arm $k$ compared to shared control among concurrent population).

	$\displaystyle\mathsf{cCATE}(k,w,e)$	$\displaystyle=\mathsf{E}[Y(k)-Y(0)\mid W=w,E=e,V_{k}=1]$
	$\displaystyle\mathsf{cATE}(k)$	$\displaystyle=\mathsf{E}[\mathsf{cCATE}(k,W,E)\mid V_{k}=1].$

$\mathsf{cATE}(k)$ is the ITT-average treatment effect among only concurrent units, $V_{k}=1$ . $\mathsf{cCATE}(k,w,e)$ is its conditional versions, conditioning on baseline variables $W$ and entry time $E$ . We now provide assumptions to identify it.

3.1 Non-parametric identification

Non-parametric identification allows us to express the causal target quantity of interest in terms of the distribution of the observed data without relying on assumptions on the functional form of the distributions (Pearl, 1995). In order to discuss non-parametric identification of $\mathsf{cATE}(k)$ , we introduce the following assumptions:

A1weak A-ignorability.

Assume
$\mathsf{E}[Y(k)|W=w,E=e,V_{k}=1]=\mathsf{E}[Y(k)|A=k,W=w,E=e,V_{k}=1]$ .

A2Consistency.

Assume
$\mathsf{P}(Y(k)|A=k,W=w,E=e,V_{k}=1)=\mathsf{P}(Y|A=k,W=w,E=e,V_{k}=1)$ .

A3Positivity of treatment assignment mechanism among concurrent units.

Assume
$\mathsf{P}(A=k\mid W=w,E=e,V_{k}=1)>0$ for all $w$ and $e$ s.t. $V_{k}=1$ .

A4Positivity of shared arm assignment mechanism among all controls.

Assume
$\mathsf{P}(A=0\mid W=w,E=e)>0$ for all $w$ and $e$ .

A5Pooling concurrent and non-concurrent controls.

Assume
$\mathsf{E}(Y\mid A=0,W=w,E=e,V_{k}=1)=\mathsf{E}(Y\mid A=0,W=w,E=e)$ for all $e$ s.t. $V_{k}=1$ .

Assumption A1 is an untestable assumption, i.e., it is a function of counterfactuals which are unobservable; that state that once we control for $W$ , $E$ and $V_{k}$ , the counterfactual outcome under $k$ is independent from the treatment assignment. We expect this to hold by design because of randomization.

Assumption A2 is a standard causal inference assumption that states that under $V_{k}=v$ and once controlled for $W,E$ , the distribution of the observed outcome under $A=k$ is the same as that of the counterfactual outcome $Y(k)$ for all $k$ in $\{0,\ldots,K\}$ . This assumption is implied by the structural causal model (1). We expect this also to hold by design.

Assumption A3 states that once a treatment arm is available in the trial all covariate profiles $w,e$ have a positive probability of receiving such treatment. Similarly, Assumption A4 states that within the shared control arm, covariate profiles $w,e$ have a positive probability of receiving the control group (Note that assumption A4 is redundant given assumption A3 but helps clarifying our identification proofs). These two assumptions hold by design in platform trials.

Assumption A5 states that once we control for $W$ and $E$ , the conditional expectation of outcome $Y$ under control ( $A=0$ ) in the pooled concurrent and non-concurrent units ( $V_{k}=0$ , and $V_{k}=1$ – right-hand side of assumption A5) is the same as that among only concurrent units ( $V_{k}=1$ – left-hand side of assumption A5), for all values $e$ among the concurrent units. In other words, after conditioning on $W$ and $E$ , what is learned using all the pooled data can be used to predict conditional expectations under only concurrent. In addition, it is straightforward to see that these quantities depends only on observable data. For instance, we know by design that we have data for the left-hand side of assumption A5 for all $e$ such that $V_{k}=1$ and for the right-hand side, for all $e$ in the shared arm. Therefore assumption A5 can be tested as discussed in our practical guidelines in section 8.

Remark 1.

Under Model (1) with $V_{k}$ a deterministic function of $E$ , i.e., $V_{k,i}=\mathds{1}[{E_{i}>t_{k}}]$ , where $t_{k}$ is a positive scalar, assumption A5 holds always true at the population level and for the true conditional outcome expectations.

In contrast, when $V_{k,i}$ is a stochastic function of $E$ , i.e., $V_{k,i}=f_{V_{k}}(E_{i},U_{V_{k},i})$ , an assumption is needed since we do not know $U_{V_{k},i}$ . e.g., the unknown error can be different between the two expectations. In addition, while this remark is always true at the population level and for the true conditional outcome expectations, for given estimators, its validity depends on the correct model specification. For instance, if there are non-linearities in $W$ in the data, and we fit a linear models within the pooled dataset and within the concurrent dataset, these two linear regressions will not be equal because they will capture the projection of the true non-linear expectation onto linear models in different subset of the range of $W$ . This underscores the importance of using non-parametric models for these regressions if data are to be pooled.

We now provide an identification theorem, under assumptions A1-A5.

Theorem 1 (Identification of $\mathsf{cATE}$ in platform adaptive trials under Model (1)).

Assume Model (1) and assumptions A1-A3. Then we have:

The parameter $\mathsf{cCATE}(k,w,e)$ is identified as

\mathsf{E}(Y\mid A=k,W=w,E=e,V_{k}=1)-\mathsf{E}(Y\mid A=0,W=w,E=e,V_{k}=1),

(2)

Under assumptions A1-A5, $\mathsf{cCATE}(k,w,e)$ is also identified as

\mathsf{E}(Y\mid A=k,W=w,E=e,V_{k}=1)-\mathsf{E}(Y\mid A=0,W=w,E=e).

(3)

Furthermore, $\mathsf{cATE}(k)$ is identified by taking the average of the above expression for $\mathsf{cCATE}(k)$ over the distribution of $(W,E)$ conditional on $V_{k}=1$ .

Equivalent expressions based on weighting are provided in the appendix. A comparison between expressions (2) and (3) reveals why researchers have been historically motivated to use non-concurrent controls: they can be useful in estimating the outcome expectation for the controls, therefore potentially reducing the variance of the estimator.

In the next sections, we will show that, when $V_{k}$ is a deterministic function of $E$ , efficiency gains only bear out for plug-in estimators based on parametric regressions, which can be biased if the models are misspecified. Doubly robust estimators, which are always consistent by virtue of randomization, will not benefit from these efficiency gains. However, we will also show, that efficiency gains can be obtained when $V_{k}$ is a stochastic function of $E$ .

3.2 On the identification of the average treatment effect

In this paper, we propose targeting $\mathsf{cATE}(k)$ in platform trials. However, many researchers are familiar with another estimand, the average treatment effect, defined as the expected difference between treatment $k$ and control $0$ in the entire trial population ( $V_{k}=1$ and $V_{k}=0$ ). In formulas,

Definition 2 (Conditional and marginal average treatment effect of treatment arm $k$ compared to shared control).

	$\displaystyle\mathsf{CATE}(k,w,e)$	$\displaystyle=\mathsf{E}[Y(k)-Y(0)\mid W=w,E=e]$
	$\displaystyle\mathsf{ATE}(k)$	$\displaystyle=\mathsf{E}[\mathsf{CATE}(k,W,E)],$

where $\mathsf{CATE}(k,w,e)$ is the conditional version of $\mathsf{ATE}(k)$ , conditioning on baseline variables $W$ and entry time $E$ . The familiarity of $\mathsf{ATE}(k)$ is partly because, in standard randomized controlled trials, the common statistical model used to evaluate treatment effects is a linear model that regresses the outcome on the treatment group and baseline covariates. In such model, the canonical interpretation of the model coefficient for the treatment aligns with that of the $\mathsf{ATE}(k)$ . For instance, assuming the following linear model: $\mathsf{E}\left[Y|A,W,E\right]=g(A,W,E;\alpha)=\eta+\beta A+W\gamma_{w}+E% \gamma_{e}$ , we can show that,

	$\displaystyle\mathsf{ATE}(1)=\mathsf{E}[Y(1)-Y(0)]$	$\displaystyle=\mathsf{E}[\mathsf{E}[Y\|A=1,W,E]]-\mathsf{E}[\mathsf{E}[Y\|A=0,W,% E]]$
		$\displaystyle=\mathsf{E}[\eta+1\beta+W\gamma_{w}+E\gamma_{e}]-\mathsf{E}[\eta+% 0\beta+W\gamma_{w}+E\gamma_{e}]$
		$\displaystyle=\beta.$

We now explain why we believe that targeting $\mathsf{ATE}(k)$ in platform trials is dangerous. To do so, we start by introducing an additional assumption needed to identify $\mathsf{ATE}(k)$ in platform trials:

A6Extrapolation of outcome mechanism among the treated.

Assume
$\mathsf{E}(Y\mid A=k,W=w,E=e,V_{k}=1)=\mathsf{E}(Y\mid A=k,W=w,E=e)$ for all $e$ .

Assumption A5 and assumption A6 state that the outcome distribution among controls and treated are exchangeable between patients for whom treatment $k$ is available and those for whom it is not, respectively; given patients’ baseline variables and entry time. Note that assumption A5 and assumption A6 are similar in nature in that they assume exchangeability of the outcome mechanism for treatment and control arms. However, there is a fundamental difference between these assumptions that makes identification based on A5 more reliable than identification based on A6. Assumption A5 is a testable assumption since it is based on observed data. In addition, as aforementioned, assumption A5 is a statement that holds always true when $V_{k}$ is a determinist function of $E$ and its validity only depends on the correct model specification. Consequently, whether pooling data in a specific regression algorithm is appropriate can be empirically checked as shown in our practical guidelines.

On the other hand, assumption A6 is an identification assumption based on unobserved data, since it requires assuming that the conditional outcome expectation observed in patients who could hypothetically be randomized to treatment $k$ can be used to extrapolate to those who could not. In other words, A6 is an extrapolation assumption, since it assumes that the expected outcome under treatment $A=k$ in times $E=e$ and baseline variables $W=w$ of no treatment availability $V_{k}=0$ can be extrapolated from a model fit on times $E=e$ and baseline variables $W=w$ where the treatment was available. Consequently, assumption A6 cannot be empirically checked.

We know state and show in the appendix that identification of $\mathsf{ATE}(k)$ depends on the extrapolation assumption A6.

Theorem 2 (Identification of $\mathsf{ATE}(k)$ in platform adaptive trials under Model (1)).

Assume Model (1) and assumptions A1-A6. Then we have that $\mathsf{CATE}(k,w,e)$ is identified as (3). Consequently, $\mathsf{ATE}(k)$ is identified by taking the average of the above expression for $\mathsf{CATE}(k)$ over the marginal distribution of $(W,E)$ .

In summary, unlike $\mathsf{cATE}(k)$ , $\mathsf{ATE}(k)$ depends on an extrapolation assumption (A6) which can be risky. Firstly, this assumption cannot be tested. Secondly, it is often unrealistic for novel diseases with a rapidly changing pathology and clinical landscape.

4 Relation to analytical approaches common in the literature

Regression models are often used to estimate $\mathsf{E}(Y\mid A=k,E=e,V_{k}=1)$ or $\mathsf{E}(Y\mid A=k,W=w,E=e,V_{k}=1)$ and then used to extrapolate to units where $V_{k}=0$ , thus targeting $\mathsf{ATE}(k)$ as discussed in Section 3.2. Inferences are then made using the regression coefficient related to the treatment, whether within the frequentist (Lee & Wason, 2020; Bofill Roig, Krotka, Burman, Glimm, Gold & Hees, 2022) or Bayesian framework (Saville et al., 2022; Bofill Roig, König, Meyer & Posch, 2022; Ibrahim & Chen, 2000).

Matching techniques have been proposed to estimate the average treatment effect among the treated, $\mathsf{E}[Y(k)-Y(0)|A=k]$ . The idea is to balance covariates $W$ between concurrent and non-concurrent controls by using for instance a matching algorithm based on the propensity score (Yuan et al., 2019).

Bayesian methods have been proposed to include non-concurrent controls. The idea is to learn a prior of the parameter of interest using non-concurrent controls only. Then, this prior is combined with the concurrent control data via Bayes’ theorem. Meta-analytic priors (Schmidli et al., 2014) or elastic priors (Jiang et al., 2023) have been proposed. These methods assume an exchangeability assumption for the control parameters, which relates to A5. Other Bayesian methods have been proposed (Neuenschwander et al., 2009; Bennett et al., 2021; Wei et al., 2024, among others). These methods, however, do not allow for the use of baseline covariates $W$ and it is not clear what estimands they target.

In this paper, to estimate $\mathsf{cATE}(k)$ , we propose estimators based on outcome regression (OR) and inverse-probability-weighting (IPW), and doubly robust estimators.

5 Estimation of $\mathsf{cATE}(k)$

To build intuition, we start by introducing outcome regression (OR) and inverse probability weighting (IPW) estimators for $\mathsf{cATE}(k)$ considering $V_{k}$ a deterministic function of $E$ . Since OR and IPW estimators are not robust to model misspecification, we then propose doubly robust (DR) estimators. To simplify notation, the following sections assume there are only two treatment arms $k=1$ and $k=0$ . Furthermore, we assume that $V_{1}=\mathds{1}\{E>t\}$ for some time $t$ such that treatment $A=1$ is only available for patients who entered the trial after time $t$ .

5.1 Estimators based on parametric outcome regression

Based on the identification results presented in Theorem 1, eq. (2), the conditional mean $\mathsf{E}(Y\mid A=k,W=w,E=e,V_{1}=1)$ , where $a=\{0,1\}$ , can be modelled as

\mathsf{E}(Y\mid A=k,W=w,E=e,V_{1}=1)=\mu_{\text{oc}}(k,w,e,1;\beta_{a}),

where (oc) stands for only-concurrent. Based on the identification results presented in Theorem 1, eq. (3) , the conditional mean $\mathsf{E}(Y\mid A=0,W=w,E=e)$ , can be modelled as

\mathsf{E}(Y\mid A=0,W=w,E=e)=\mu_{\text{all}}(0,w,e;\alpha_{a}).

An estimate of $\beta_{a}$ and $\alpha_{a}$ can be then obtained by using maximum likelihood estimation, i.e., ordinary least squares, only among the concurrent controls $V_{k}=1$ for $\mu_{\text{oc}}(a,w,e,1;\beta_{a})$ and among all concurrent and non-concurrent controls when using $\mu_{\text{all}}(0,w,e;\alpha_{a})$ , i.e., among $A=0$ only. Let $\hat{\beta}_{a}$ and $\hat{\alpha}_{a}$ denote consistent estimators of $\beta_{a}$ and $\alpha_{a}$ , respectively. We then propose

	$\displaystyle\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$	$\displaystyle=\frac{\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1}\}\mu_{\text{oc}}(1,w% _{i},e_{i},1;\hat{\beta}_{1})}{\sum_{i=1}^{n}\mathds{1}{\{v_{k,i}=1}\}}$
		$\displaystyle-\frac{\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1}\}\mu_{\text{oc}}(0,w% _{i},e_{i},1;\hat{\beta}_{0})}{\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1}\}},$

as an outcome regression estimator for $\mathsf{cATE}(k)$ . Under Theorem 1, eq. (3), we propose the alternative outcome regression estimator for $\mathsf{cATE}(k)$ ,

	$\displaystyle\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}$	$\displaystyle=\frac{\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1}\}\mu_{\text{oc}}(1,w% _{i},e_{i},1;\hat{\beta}_{1})}{\sum_{i=1}^{n}\mathds{1}{\{v_{k,i}=1}\}}$
		$\displaystyle-\frac{\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1}\}\mu_{\text{all}}(0,% w_{i},e_{i};\hat{\alpha}_{0})}{\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1}\}}.$

Large sample properties.

We derived the asymptotic properties of $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$ , and $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}$ , using the approach of M-estimation (Boos & Stefanski, 2013, Chapter 7). Under regularity conditions (Boos & Stefanski, 2013, Section 7.2), $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$ , $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}$ , are consistent and asymptotically Normal, with asymptotic variance derived in the appendix.

5.2 An estimator based on parametric inverse probability weighting

Following standard procedures, we model the conditional probability of treatment assignment given $W$ and $E$ among only $V_{k}=1$ by using a logistic regression model,

\displaystyle\mathsf{E}(\mathds{1}{\{A=1}\}\mid W=w,E=e,V_{k}=1)

\displaystyle=\pi_{\text{oc}}(w,e,1;\eta)=\frac{\exp(\eta^{T}x)}{1+\exp(\eta^{% T}x)},

where $x=(w,e,1)$ . We obtain an estimate of $\eta$ by using maximum likelihood estimation, only among the concurrent controls $V_{k}=1$ . We then propose,

\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}}=\frac{\sum_{i=1}^{n}\gamma^{1}_{i% }\mathds{1}{\{V_{k,i}=1}\}y_{i}}{\sum_{i=1}^{n}\gamma^{1}_{i}\ }-\frac{\sum_{i% =1}^{n}\gamma^{0}_{i}\mathds{1}{\{V_{k,i}=1}\}y_{i}}{\sum_{i=1}^{n}\gamma^{0}_% {i}\ },

where $\gamma^{0}_{i}=\mathds{1}{\{A_{i}=0}\}/(1-\hat{\pi}_{\text{oc}})$ , $\gamma^{1}_{i}=\mathds{1}{\{A_{i}=1}\}/\hat{\pi}_{\text{oc}}$ , and $\hat{\pi}_{\text{oc}}=\pi_{\text{oc}}(w,e,1;\hat{\eta})$ for clarity.

Large sample properties.

We derived the asymptotic properties of $\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}}$ using the approach of M-estimation (Boos & Stefanski, 2013, Chapter 7). Under regularity conditions (Boos & Stefanski, 2013, Section 7.2), $\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}}$ , is consistent and asymptotically Normal, with asymptotic variance derived in the appendix.

5.3 Doubly robust estimators

Doubly robust (DR) estimators for average treatment effects provide consistent estimates by combining outcome regression and IPW. To derive DR estimators of $\mathsf{cATE}(k)$ , we follow standard practice of constructing them based on efficient influence functions (EIF)s (Bickel et al., 1993; Fisher & Kennedy, 2021; Hines et al., 2022; Kennedy, 2022). Influence functions are a core component of classical statistical theory. They aid in constructing estimators with desirable properties such as double robustness, asymptotic normality, and fast rates of convergence. Additionally, they enable the incorporation of machine learning algorithms while preserving valid statistical inferences and providing insights into statistical efficiency, i.e., the best performance for estimating an estimand. We provide efficiency considerations of the proposed estimators in section 6. The next theorem provide these EIFs,

Theorem 3.

The efficient influence function, $\varphi(Z,\mathsf{cATE}(k))$ , for $\mathsf{cATE}(k)$ under Model (1), is equal to

\frac{\mathds{1}{\{V_{k}=1\}}}{\mathsf{P}(V_{k}=1)}\bigg{[}\frac{2A-1}{\mathsf% {P}(A\mid W,E,V_{k}=1)}\{Y-\mathsf{E}(Y\mid A,W,E,V_{k}=1)\}\\ +\mathsf{E}(Y\mid A=1,W,E,V_{k}=1)-\mathsf{E}(Y\mid A=0,W,E,V_{k}=1)-\mathsf{% cATE}(k).\bigg{]}

(4)

The efficient influence function, $\varphi(Z,\mathsf{cATE}(k))$ , for $\mathsf{cATE}(k)$ under Model (1) and assuming A5, is equal to

\frac{\mathds{1}{\{V_{k}=1\}}}{\mathsf{P}(V_{k}=1)}\bigg{[}\frac{A}{\mathsf{P}% (A\mid V_{k}=1,W,E)}\{Y-\mathsf{E}(Y\mid A,W,E,V_{k}=1)\}\bigg{]}-\\ \frac{1-A}{\mathsf{P}(A\mid W,E)}\frac{\mathsf{P}(V_{k}=1\mid E,W)}{\mathsf{P}% (V_{k}=1)}\{Y-\mathsf{E}(Y\mid A,E,W)\}+\\ \frac{\mathds{1}{\{V_{k}=1\}}}{\mathsf{P}(V_{k}=1)}\Big{[}\mathsf{E}(Y\mid A=1% ,W,E,V_{k}=1)-\mathsf{E}(Y\mid A=0,W,E)\Big{]}-\mathsf{cATE}(k).

(5)

These influence functions suggest the following estimators,

	$\displaystyle\hat{\mathsf{cATE}}_{\text{DR}}^{\text{oc}}$	$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\bigg{(}\frac{\mathds{1}{\{V_{k,i}=1\}}% }{n^{-1}\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1\}}}\bigg{[}\frac{(2a_{i}-1)}{\pi_% {\text{oc}}(w_{i},e_{i},1)}\{y_{i}-\mu_{\text{oc}}(a_{i},w_{i},e_{i},1)\}$
		$\displaystyle+\mu_{\text{oc}}(1,w_{i},e_{i},1)-\mu_{\text{oc}}(0,w_{i},e_{i},1% )\bigg{]}\bigg{)}$
	$\displaystyle\hat{\mathsf{cATE}}_{\text{DR}}^{\text{all}}$	$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\bigg{(}\frac{\mathds{1}{\{V_{k,i}=1\}}% }{n^{-1}\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1\}}}\bigg{[}\frac{\mathds{1}{\{A_{% i}=1\}}}{\pi_{\text{oc}}(w_{i},e_{i},1)}\{y_{i}-\mu_{\text{oc}}(1,w_{i},e_{i},% 1)\}\bigg{]}$
		$\displaystyle+\frac{\mathds{1}{\{A_{i}=0\}}}{1-\pi_{\text{all}}(w_{i},e_{i})}% \frac{\nu(w_{i},e_{i})}{n^{-1}\sum_{i=1}^{n}\mathds{1}{\{V_{k,i}=1\}}}\{y_{i}-% \mu_{\text{all}}(0,w_{i},e_{i})\}$
		$\displaystyle+\frac{\mathds{1}{\{V_{k,i}=1\}}}{n^{-1}\sum_{i=1}^{n}\mathds{1}{% \{V_{k,i}=1\}}}\bigg{[}\mu_{\text{oc}}(1,w_{i},e_{i},1)-\mu_{\text{all}}(0,w_{% i},e_{i})\bigg{]}\bigg{)},$

where, $\pi_{\text{oc}}(w_{i},e_{i},1),\mu_{\text{oc}}(1,w_{i},e_{i},1),\mu_{\text{all% }}(0,w_{i},e_{i})$ , and $\pi_{\text{all}}(w_{i},e_{i})$ , can be estimated by using parametric and machine learning methods. Building on the results from the previous sections, we can leverage the linear regression models introduced earlier: $\mu_{\text{oc}}(0,w_{i},e_{i},1;\beta_{0})$ , and $\mu_{\text{all}}(0,w_{i},e_{i};\alpha_{0})$ as outcome models. Similarly, the logistic regression models $\pi_{\text{all}}(w,e;\eta)$ and $\nu(w,e;\xi)$ introduced previously for $\mathsf{P}[A=1\mid W=w,E=e]$ and $\mathsf{P}[V_{k}=1\mid W=w,E=e]$ , respectively, can also be employed.

Large sample properties.

Note that if $V_{k}$ is a deterministic function of $E$ , and therefore $\mathsf{P}(V_{k}=1\mid E,W)=\mathds{1}\{E>t\}$ , the two EIFs (4) and (5) are the same. In addition, the first influence function in Theorem 3 boils down to the standard influence function for the average treatment effect in the the $V_{k}=1$ population. Therefore, it inherits the standard analysis of the one-step estimator for average treatment effects as discussed in (Kennedy et al., 2021, Section 4.1). Similar analysis can be conducted when $V_{k}$ is a stochastic function of $E$ . In summary, if the outcome models are correctly specified, it can be shown that estimators of the form of $\hat{\mathsf{cATE}}_{\text{DR}}^{\text{oc}}$ and $\hat{\mathsf{cATE}}_{\text{DR}}^{\text{all}}$ are root-n consistent, asymptotically normal with asymptotically valid 95% confidence intervals given by the closed-form expressions $\hat{\mathsf{cATE}}_{\text{DR}}^{\text{oc}}\pm 1.96\sqrt{\hat{\text{var}}\{% \varphi(Z,\mathsf{cATE}(k))\}/n}$ and $\hat{\mathsf{cATE}}_{\text{DR}}^{\text{all}}\pm 1.96\sqrt{\hat{\text{var}}\{% \varphi(Z,\mathsf{cATE}(k))\}/n}$ , respectively, and efficient in the local asymptotic minimax sense. If the models are misspecified, the confidence intervals will be conservative (Kennedy, 2022).

Double robustness.

Similarly, if $V_{k}$ is a deterministic function of $E$ the two estimators boils down to the standard DR estimator for the average treatment effect in the $V_{k}=1$ population, they also inherit the same double robust property. This means that if either the outcome regression model ( $\mu_{\text{oc}}(0,w_{i},e_{i},1;\beta_{0})$ , $\mu_{\text{all}}(0,w_{i},e_{i};\alpha_{0})$ ) or the treatment assignment model ( $\pi_{\text{oc}}(w_{i},e_{i},1;\eta)$ , $\pi_{\text{all}}(w,e;\eta)$ ) is correctly specified (in a parametric sense), then the DR estimator is consistent, see section 4.2 of Kennedy (2022) for details. Recall that our proposed estimators are doubly robust due to randomization i.e. the treatment assignment mechanism is known by design. We provide some empirical result of this property in our simulations in section 7. In addition, it can be shown that if $V_{k}$ is a stochastic function of $E$ , the same double robustness property holds, provided the model for $V_{k}$ is correctly specified.

6 Efficiency considerations

Estimators based on outcome regression.

As shown in the appendix, the influence function of the conditional expectation under control, $\varphi(Z_{i},\hat{\mu}_{0})$ , depends on two components: the influence function of $\mu_{0}$ itself and that of the regression coefficients. Here, $\mu_{0}=\mathsf{E}[Y(0)|V_{k}=1]$ depending on the estimand under study. Therefore, a more precise estimation of the regression coefficients (the variance of the estimated regression coefficient is inversely proportional to the sample size), translate to a more precise estimation of $\mu_{0}$ and consequently of $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}$ compared to $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$ .

Doubly robust estimators.

As previously discussed, if $V_{k}$ is a deterministic function of $E$ the two EIFs presented in section 5.3 are the same. In this case, efficiency gains come solely from better fitting of the regression $E(Y\mid A=0,W,E,V_{k}=1)$ , which under assumption (A5) is equal to $E(Y\mid A=0,W,E,V_{k}=1)=E(Y\mid A=0,W,E)$ (because $V_{k}=\mathds{1}\{E>t\}$ ). In this case it becomes purely about getting this regression right, and these efficiency gains do not show up in the first order analysis of the estimator. In addition, as aforementioned, efficiency gains can be obtained by leveraging prognostic variables as discussed in Colantuoni & Rosenblum (2015); Benkeser et al. (2021). We show some empirical results in our simulations in section 7. Interestingly, efficiency gains can also be achieved when using doubly robust estimators under Model (1) when $V_{k}$ is a non deterministic function of $E$ as shown in our simulation setting. Under this scenario, we expect to see efficiency gains also when using an estimator based on inverse probability weighting solely.

7 Simulations

In this section we evaluate the performance of the proposed estimators with respect to, bias squared, variance, mean square error, and coverage of the 95% confidence interval, across levels of the percentage of concurrent controls, and model misspecification when estimating $\mathsf{cATE}(k)$ . We do not compare our proposed estimators with methods described in section 4, because it is not clear if they target $\mathsf{cATE}(k)$ .

Finally, our simulations aim to showcase the theoretical properties previously discussed, rather than evaluating them under complex real-world scenarios.

7.1 Setup

Aims

To evaluate the performance and gains in efficiency of our proposed estimators across levels of (1) percentage of concurrent controls (90% to 10%) and (2) model misspecification (correct outcome and treatment models; and misspecified outcome and correct treatment model) considering $V_{k}$ a deterministic function of $E$ . In addition, we also evaluate efficiency gains by comparing the estimated variance of the outcome regression (Section 5.1) and doubly robust (Section 5.3) estimators that only use concurrent data compared with those that use all data considering $V_{k}$ both a deterministic and a stochastic function of $E$ .

Data-generating mechanisms

We considered generating data from Model (1). Specifically, we considered a sample size of $n=1,000$ and for each subject $i=1,\dots,n$ , we simulated the following data:

Step 1. the entry time $E\sim\text{Norm}(0,1)$ and a baseline covariate $W=-\kappa_{1}+0.8E+\text{Norm}(0,1)$ , where $\kappa_{1}=n^{-1}\sum_{i=1}^{n}0.8E$ ;
Step 2. an indicator whether treatment $k$ was available at time $E$ , $V_{k}$ as a deterministic function of $E$ being less than a threshold describing the level of the percentage of concurrent controls;
Step 3. a binary treatment $A\sim\text{Bernoulli}(\pi(W))$ , where $\pi(W)=\left(1+\exp\left(-(-\kappa_{2}+0.8W)\right)\right)^{-1}$ and $\kappa_{2}=n^{-1}\sum_{i=1}^{n}0.8W$ when $V_{k}=1$ and $A=0$ , otherwise (participants for which treatment only control is available);
Step 4. two counterfactual outcomes, $Y(0)=0.8W+0.5E+\text{Norm}(0,1)$ , and $Y(k)=Y(0)+\Delta$ , with $\Delta=0.8$ , and the observed outcome $Y=AY(1)+(1-A)Y(0)$ . Since we consider a homogeneous treatment effect, $\Delta=\mathsf{cATE}(k)=0.8$ .

Estimands

The estimand of interest is $\mathsf{cATE}(k)$ .

Methods

For each dataset across levels of percentage of concurrent controls, and misspecification we used the methods summarized in Table 1.

Performance metrics

Bias squared, variance, mean square error (MSE), and coverage of the 95% confidence interval. In addition, we also considered the ratio of the estimated variances.

Scenarios

We considered levels of percentage of concurrent controls between 10% and 90% by 10%. Misspecified models were set to only include an intercept – not controlling for any covariates or entry time.

Table 1: Methods used in the estimation of

\mathsf{cATE}(k)

	Acronym
Method	$\mathsf{cATE}(k)$
Outcome regression using only concurrent data, ( $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$ , Section 5.1)	OR-oc
Outcome regression using all data, ( $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}$ , $\hat{\mathsf{ATE}}_{\text{OR}}$ , Section 5.1)	OR-ac
Weighting using only concurrent data ( $\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}}$ , Section 5.2)	IPW
Doubly robust using only concurrent data ( $\hat{\mathsf{cATE}}_{\text{DR}}^{\text{oc}}$ , Section 5.3)	DR-oc
Doubly robust using all data ( $\hat{\mathsf{cATE}}_{\text{DR}}^{\text{all}}$ , Section 5.3)	DR-ac

7.2 Results

7.2.1 Bias, variance, MSE, and coverage

Figure 3 and Figure 4, show bias squared, variance, MSE and coverage of the 95% confidence intervals in estimating $\mathsf{cATE}(k)$ across percentage of concurrent controls when models are correct and when only the treatment model is correctly specified, respectively. When both the outcome and the treatment models are correct (Figure 3), bias squared is negligible across levels of concurrent controls for all methods. Variance is shown to increase with decreasing levels of concurrent controls across all methods, with OR-ac being smaller than OR-oc, suggesting a gain in efficiency (more on this in the next section). Similar behavior can be seen for the MSE. Finally, all methods achieve desirable coverage levels. When the outcome model is misspecified (Figure 4), both estimators based on outcome regression show bias while maintaining a relatively small variance. MSE is consequently dominated by bias. DR estimators and the IPW estimator maintaine negligible levels of bias and relatively small variance.

7.2.2 Efficiency gains

Figure 5 shows the ratio of the estimated standard errors of DR-oc over DC-ac and OR-oc over OR-ac across levels of concurrent controls and misspecification.

The top panels of Figure 5 follow Model (1) where $V_{k}$ is a deterministic function of $E$ . As discussed in Section 6, estimators based on outcome regression that use all controls seem to have a gain in efficiency, while DR estimators did not under correct models.

The bottom panels of Figure 5 follow Model (1) where $V_{k}$ is a not a deterministic function of $E$ . Specifically, we generated $V_{k}\sim\text{Bernoulli}(\pi(E))$ , where $\pi(E)=\left(1+\exp\left(-(-\kappa_{4}+0.5E)\right)\right)^{-1}$ and $\kappa_{4}=\kappa_{3}+n^{-1}\sum_{i=1}^{n}0.5E$ , and $\kappa_{3}$ is the threshold discussed in Step 2 above. Under correct models (including the one for $V_{k}$ ), the entry time $E$ is a prognostic variable of $V_{k}$ and therefore improves efficiency compared to using only concurrent controls (Bottom panels of 5; DR-oc/DR-ac). These results suggest that efficiency gains can be obtained when using a doubly robust estimator under Model (1) when $V_{k}$ is not a deterministic function of $E$ and the model for $V_{k}$ is correct.

Summary of results.

Methods based on outcome regression improve efficiency when using non-concurrent controls. However, they introduce bias when misspecified. In contrast, doubly robust estimators provide consistent estimates with relatively small variance when either the treatment or outcome model is correctly specified. In addition, doubly robust estimator have the potential to additionally improve efficiency when $V_{k}$ is not a deterministic function of $E$ under Model (1).

8 Practical considerations

What estimand should we target?

In this paper, we introduce $\mathsf{cATE}(k)$ , whereas it not clear what estimand the current related literature targets. Given the more stringent and untestable assumptions needed to identify $\mathsf{ATE}(k)$ , we suggest targeting $\mathsf{cATE}(k)$ . Note that $\mathsf{ATE}(k)$ and $\mathsf{cATE}(k)$ will coincide under the assumption of homogeneous treatment effect. While this is true in theory, we would expect it not to hold in practical settings, leading to different results as showed in our case study in the next section.

Should we pool concurrent and non-concurrent controls? Testing assumption A5.

To evaluate assumption A5 under Model (1), we propose using the method introduced by Luedtke et al. (2019). Specifically, following the notation of the original paper, we suggest to set $R_{P}(o)\triangleq\mathsf{E}(Y\mid A,W,E,V_{k}=1)\ -\mathsf{E}(Y\mid A=0,W,E)$ and $S_{P}(o)\equiv 0$ , where $o$ is the observed data, and $R_{P}$ and $S_{p}$ are elements of the space of univariate bounded real-valued measurable functions defined on the the support of the distribution $P$ (Luedtke et al., 2019). We show an application of this testing procedure in our case study. Note that if $V_{k}$ is a deterministic function of $E$ as in our case study, this test is a test on the statistical models, i.e., if the projections on the linear model are different, and not a test on the true expectations. In other words, it is a test to evaluate model misspecifications and therefore guide analysts on the choice to either use or not non-concurrent controls.

Should we leverage prognostic baseline variables for additional precision?

Recent literature suggests that incorporating baseline prognostic variables can improve the precision of estimates (Colantuoni & Rosenblum, 2015). We propose following this approach by appropriately controlling for these variables in the analysis. This may explain the increased precision observed with DR-oc and OR-oc estimators compared to the naive estimator in our case study (presented in the next section), despite being computed within the concurrent population only.

What estimator should we use?

Under Model (1) and assuming $V_{k}$ is a deterministic function of $E$ , to estimate $\mathsf{cATE}(k)$ , we recommend using $\hat{\mathsf{cATE}}_{\text{DR}}^{\text{oc}}$ , the doubly robust estimator using only concurrent data (Section 5.3). We recommend this estimator because: 1) it has the same efficiency as the DR estimator that uses non-concurrent controls 2) it does not require any additional assumptions, thus better aligning with the FDA recommendations (FDA, 2021, Section A.5.1); 3) it accommodates covariates that, if prognostic, can be leveraged to improve efficiency (Colantuoni & Rosenblum, 2015); and 4) it is doubly robust, meaning that it is consistent when either the treatment assignment model or the outcome model is correctly specified, a property we obtain “for free” in platform trials due to randomization. Assuming $V_{k}$ is a stochastic function of $E$ and if an analyst chooses to leverage non-concurrent controls, then the DR-ac estimator is recommended.

Sample size calculation for a prospective trial with non-concurrent controls.

Our theoretical and methodological results suggest an efficiency gain when including non-concurrent control with estimators based on regression models. While these results are promising, we suggest to conduct standard sample size calculation as if the non-concurrent control data will not be available. At the analysis stage, precision can then be improved by using non-concurrent control data as previously described with the caveat that the outcome model must be correctly specified.

Multiple comparisons.

Our proposed methods enable the use of standard type I error control procedures, such as Bonferroni or Benjamini-Hochberg corrections, due to the validity of 95% confidence intervals, test statistics, and p-values (demonstrated in previous sections). This allows for straightforward application of these corrections in platform trials with, for instance, pre-planned interim analyses, and multiple primary endpoints.

Summary of practical guidelines.

Target $\mathsf{cATE}(k)$ as the estimand of interest. Use DR-oc to obtain consistent estimates of $\mathsf{cATE}(k)$ . To improve efficiency, focus on prognostic baseline covariates rather than relying on non-concurrent controls. If leveraging non-concurrent controls is of interest, use DR-ac. Finally, conduct sample size calculations without considering non-concurrent controls and conduct standard multiple comparisons adjustments.

9 The Adaptive COVID-19 Treatment Platform Trial

In this section, we apply our proposed estimators to estimate $\mathsf{cATE}$ using data from the Adaptive COVID-19 Treatment Trial (ACTT) (Kalil et al., 2021). This was a platform trial that investigated treatments for hospitalized adult patients with COVID-19 pneumonia. The trial comprised multiple stages, as illustrated in Figure 1. The initial phase, ACTT-1, involved the assessment of the effectiveness of remdesivir alone compared to placebo. Subsequently, in the second stage (ACTT-2), the placebo was phased out, and a novel treatment, combining remdesivir with baricitinib, was introduced. Simultaneously, participants were randomized to receive either remdesivir alone or the combination therapy of remdesivir and baricitinib. Data were accessed using the NIAID Clinical Trials Data Repository (https://data.niaid.nih.gov/). We have a Data User Agreement in place for its use.

Study population and endpoint of interest.

We considered the combined participants of ACCT-1 and ACTT-2 as our study population. We followed the inclusion and exclusion criteria of the original study. The final study population was comprised of 1,379 participants, 541 from ACTT-1 and 1,033 from ACTT-2. We considered the time to recovery in days as our enpoint of interest.

Treatments under study and targeted causal estimand.

We considered two treatment arms: remdesivir alone (which served as the shared control arm) and remdesivir plus baricitinib. Our target estimand was $\mathsf{cATE}(1)$ , where $1$ represents the remdesivir plus baricitinib arm and $0$ represents the remdesivir alone shared arm.

Baseline covariates.

We consider the following baseline covariates: age, sex assigned at birth (female, male), race (White, Black, Asian, Other: American Indian or Alaska Native, Native Hawaiian or Other Pacific Islander, Multiple), ethnicity (Hispanic or Latino, Not Hispanic or Latino), BMI, geographic region of study site (Asia, Europe, North America), disease severity stratum (mild, severe), and having any of these comorbidities: duration of symptoms, hypertension, coronary artery disease, congestive heart disease, chronic oxygen requirement, chronic respiratory disease, chronic liver disease, chronic kidney disease, diabetes type I, diabetes type II, obesity, cancer, immune deficiency, and asthma, in addition to the entry time which we normalized to be between 0 and 1.

Models setup.

We computed OR-oc, OR-ac, OR-ad, IPW, DR-oc and DR-ac (Table 1) by using linear and logistic regression models. We computed the naive estimator by taking the average difference in the endpoint between the two arms among only concurrent participants. Variances were obtained by using the sandwich estimator (for naive, OR-oc, OR-ac, IPW) and by taking the variance of the efficient influence function (for DR-oc and DR-ac). Wald 95% confidence intervals and Wald tests were constructed.

Results.

Table 2 shows the point estimate for $\mathsf{cATE}(1)$ , standard errors, 95% confidence intervals and p-values. The naive estimate of $\mathsf{cATE}(1)$ , resulted in a value of -1.33 with a standard error of 0.58. This suggest that baricitinib plus remdesivir was superior to remdesivir alone in reducing recovery time as in the original trial (Kalil et al., 2021). OR-oc, IPW, DR-oc and DR-ac improved precision while maintaining a similar point estimate. OR-ac improved precision the most (around 28% improvement compared with the naive estimator), however, it resulted in a different point estimate, -0.75 which led to a non significant result. This suggest that the outcome model used to obtain OR-ac might be misspecified and therefore that assumption A5 does not hold. Using an omnibus test as described in our practical guidelines in section 8 we obtained a p-value $\leq 0.001$ using both the variance and the eigenvalue approach (Luedtke et al., 2019), thus supporting rejecting assumption A5 and, therefore, suggests not using non-concurrent controls. In contrast, doubly robust estimators improved precision while maintaining a similar point estimate as the naive estimator. We believe the improved precision observed in OR-oc, IPW, and DR-oc compared to the naive estimator stems from appropriately adjusting for baseline variables, as discussed in Colantuoni & Rosenblum (2015), and in our previous section. Note that a conditional analysis, targeting $\mathsf{ATE}(1)$ , using a standard linear regression model regressing the outcome in the full population on treatment arms, entry time and baseline covariates led to a non significant point estimate of -0.46 (standard error equal to 0.52).

Table 2: Estimated

\mathsf{cATE}(1)

using the ACTT data.

Method	$\hat{\mathsf{cATE}}(1)$	SE	95% CI	p-value	Ratio
OR-oc	-1.29	0.47	(-2.21;-0.37)	$<$ 0.01	1.22
OR-ac	-0.75	0.45	(-1.63;0.13)	0.10	1.28
IPW	-1.28	0.47	(-2.20;-0.36)	$<$ 0.01	1.22
DR-oc	-1.30	0.47	(-2.22;-0.38)	$<$ 0.01	1.22
DR-ac	-1.30	0.47	(-2.22;-0.38)	$<$ 0.01	1.21
naive	-1.33	0.58	(-2.47;-0.19)	0.02	1.00

10 Conclusion

In this paper, we introduced identification results and estimation techniques to identify and estimate concurrent average treatment effects $\mathsf{cATE}(k)$ in the presence of non-concurrent control in platform trials. We argue that identifying and estimating $\mathsf{ATE}(k)$ relies on an extrapolation assumption that is both untestable and often too stringent, particularly in the context of platform trials, where multiple, potentially novel treatments or interventions are being evaluated and the outcome mechanism is poorly understood. Therefore, we advocate focusing primarily on $\mathsf{cATE}(k)$ , where assumptions can be tested. By focusing on $\mathsf{cATE}(k)$ rather than $\mathsf{ATE}(k)$ , we also open the door to leveraging non-parametric models based on machine and deep learning techniques for learning outcome and treatment assignment mechanisms under the proposed doubly robust estimators (Kennedy, 2022; Díaz, 2020; Hirshberg & Wager, 2021). In fact, while these methods can capture complex data relationships, potentially mitigating model misspecification, they may not be suitable for extrapolation. Furthermore, our proposed doubly robust estimator accommodates Bayesian techniques while retaining valid frequentist properties, as demonstrated in (Shin & Antonelli, 2023; Antonelli et al., 2022).

A key takeaway of this paper is to target $\mathsf{cATE}(k)$ in platform trials, and, under Model (1), to use a doubly robust estimator only among concurrent units, prioritizing the identification of strong prognostic baseline variables rather than relying on non-concurrent controls, especially when $V_{k}$ is a deterministic function of $E$ .

In this paper, we presented results that can be used for continuous and binary endpoints. Estimators can be constructed for time-to-event endpoints under the non-parametric causal model introduced in eq. (1).

Finally, in this paper, we demonstrate results assuming a structural equation model where treatment assignment may depend on baseline covariates; however, similar identification and estimation results can be obtained without baseline covariates.

References

(1)
Angus et al. (2020) Angus, D. C., Derde, L., Al-Beidh, F., Annane, D., Arabi, Y., Beane, A., van Bentum-Puijk, W., Berry, L., Bhimani, Z., Bonten, M. et al. (2020), ‘Effect of hydrocortisone on mortality and organ support in patients with severe covid-19: the remap-cap covid-19 corticosteroid domain randomized clinical trial’, Jama 324(13), 1317–1329.
Antonelli et al. (2022) Antonelli, J., Papadogeorgou, G. & Dominici, F. (2022), ‘Causal inference in high dimensions: a marriage between bayesian modeling and good frequentist properties’, Biometrics 78(1), 100–114.
Banbeta et al. (2019) Banbeta, A., Rosmalen, J., Dejardin, D. & Lesaffre, E. (2019), ‘Modified power prior with multiple historical trials for binary endpoints’, Stat Med. 38.
https://doi.org/10.1002/sim.8019
Barker et al. (2009) Barker, A., Sigman, C., Kelloff, G. J., Hylton, N., Berry, D. A. & Esserman, L. (2009), ‘I-spy 2: an adaptive breast cancer trial design in the setting of neoadjuvant chemotherapy’, Clinical Pharmacology & Therapeutics 86(1), 97–100.
Benkeser et al. (2021) Benkeser, D., Díaz, I., Luedtke, A., Segal, J., Scharfstein, D. & Rosenblum, M. (2021), ‘Improving precision and power in randomized trials for covid-19 treatments using covariate adjustment, for binary, ordinal, and time-to-event outcomes’, Biometrics 77(4), 1467–1481.
Bennett et al. (2021) Bennett, M., White, S., Best, N. & Mander, A. (2021), ‘A novel equivalence probability weighted power prior for using historical control data in an adaptive clinical trial design: A comparison to standard methods’, Pharm Stat. 20.
https://doi.org/10.1002/pst.2088
Berry et al. (2015) Berry, S. M., Connor, J. T. & Lewis, R. J. (2015), ‘The platform trial: an efficient strategy for evaluating multiple treatments’, Jama 313(16), 1619–1620.
Bickel et al. (1993) Bickel, P. J., Klaassen, C. A., Bickel, P. J., Ritov, Y., Klaassen, J., Wellner, J. A. & Ritov, Y. (1993), Efficient and adaptive estimation for semiparametric models, Vol. 4, Springer.
Bofill Roig et al. (2023) Bofill Roig, M., Burgwinkel, C., Garczarek, U., Koenig, F., Posch, M., Nguyen, Q. & Hees, K. (2023), ‘On the use of non-concurrent controls in platform trials: a scoping review’, Trials 24(1), 1–17.
Bofill Roig, König, Meyer & Posch (2022) Bofill Roig, M., König, F., Meyer, E. & Posch, M. (2022), ‘Commentary: Two approaches to analyze platform trials incorporating non-concurrent controls with a common assumption’, Clin Trials. 19.
https://doi.org/10.1177/17407745221112016
Bofill Roig, Krotka, Burman, Glimm, Gold & Hees (2022) Bofill Roig, M., Krotka, P., Burman, C. F., Glimm, E., Gold, S. M. & Hees, K. (2022), ‘On model-based time trend adjustments in platform trials with non-concurrent controls’, BMC Med Res Methodol. 22.
https://doi.org/10.1186/s12874-022-01683-w
Boos & Stefanski (2013) Boos, D. D. & Stefanski, L. A. (2013), Essential statistical inference: theory and methods, Vol. 591, Springer.
Broglio (2018) Broglio, K. (2018), ‘Randomization in clinical trials: permuted blocks and stratification’, Jama 319(21), 2223–2224.
Chen et al. (2020) Chen, W. C., Wang, C., Li, H., Lu, N., Tiwari, R. & Xu, Y. (2020), ‘Propensity score-integrated composite likelihood approach for augmenting the control arm of a randomized controlled trial by incorporating real-world data’, J Biopharm Stat. 30.
https://doi.org/10.1080/10543406.2020.1730877
Colantuoni & Rosenblum (2015) Colantuoni, E. & Rosenblum, M. (2015), ‘Leveraging prognostic baseline variables to gain precision in randomized trials’, Statistics in medicine 34(18), 2602–2617.
Collignon et al. (2022) Collignon, O., Schiel, A., Burman, C.-F., Rufibach, K., Posch, M. & Bretz, F. (2022), ‘Estimands and complex innovative designs’, Clinical Pharmacology & Therapeutics 112(6), 1183–1190.
Collignon et al. (2020) Collignon, O., Schritz, A., Senn, S. J. & Spezia, R. (2020), ‘Clustered allocation as a way of understanding historical controls: Components of variation and regulatory considerations’, Stat Methods Med Res. 29.
https://doi.org/10.1177/0962280219880213
Díaz (2020) Díaz, I. (2020), ‘Machine learning in the estimation of causal effects: targeted minimum loss-based estimation and double/debiased machine learning’, Biostatistics 21(2), 353–358.
FDA (2021) FDA (2021), ‘E9 (r1) statistical principles for clinical trials: addendum: estimands and sensitivity analysis in clinical trials’, Guidance for Industry .
https://shorturl.at/clJL5
Fisher & Kennedy (2021) Fisher, A. & Kennedy, E. H. (2021), ‘Visually communicating and teaching intuition for influence functions’, The American Statistician 75(2), 162–172.
Foltynie et al. (2023) Foltynie, T., Gandhi, S., Gonzalez-Robles, C., Zeissler, M.-L., Mills, G., Barker, R., Carpenter, J., Schrag, A., Schapira, A., Bandmann, O. et al. (2023), ‘Towards a multi-arm multi-stage platform trial of disease modifying approaches in parkinson’s disease’, Brain 146(7), 2717–2722.
Gravestock et al. (2017) Gravestock, I., Held, L. & consortium, C.-N. (2017), ‘Adaptive power priors with empirical bayes for clinical trials’, Pharmaceutical statistics 16(5), 349–360.
Han et al. (2017) Han, B., Zhan, J., John Zhong, Z., Liu, D. & Lindborg, S. (2017), ‘Covariate-adjusted borrowing of historical control data in randomized clinical trials’, Pharm Stat. 16.
https://doi.org/10.1002/pst.1815
Hayward et al. (2021) Hayward, G., Butler, C. C., Yu, L.-M., Saville, B. R., Berry, N., Dorward, J., Gbinigie, O., Van Hecke, O., Ogburn, E., Swayze, H. et al. (2021), ‘Platform randomised trial of interventions against covid-19 in older people (principle): protocol for a randomised, controlled, open-label, adaptive platform, trial of community treatment of covid-19 syndromic illness in people at higher risk’, BMJ open 11(6), e046799.
Hines et al. (2022) Hines, O., Dukes, O., Diaz-Ordaz, K. & Vansteelandt, S. (2022), ‘Demystifying statistical learning based on efficient influence functions’, The American Statistician 76(3), 292–304.
Hirshberg & Wager (2021) Hirshberg, D. A. & Wager, S. (2021), ‘Augmented minimax linear estimation’, The Annals of Statistics 49(6), 3206–3227.
Hobbs et al. (2011) Hobbs, B. P., Carlin, B. P., Mandrekar, S. J. & Sargent, D. J. (2011), ‘Hierarchical commensurate and power prior models for adaptive incorporation of historical information in clinical trials’, Biometrics. 67.
https://doi.org/10.1111/j.1541-0420.2011.01564.x
Hupf et al. (2021) Hupf, B., Bunn, V., Lin, J. & Dong, C. (2021), ‘Bayesian semiparametric meta-analytic-predictive prior for historical control borrowing in clinical trials’, Stat Med. 40.
https://doi.org/10.1002/sim.8970
Ibrahim & Chen (2000) Ibrahim, J. G. & Chen, M. H. (2000), ‘Power prior distributions for regression models’, Stat Sci. 15.
https://doi.org/10.1214/ss/1009212673
International Council for Harmonisation (2017) International Council for Harmonisation (2017), ‘Ich harmonised guideline e9 (r1): Estimands and sensitivity analysis in clinical trials’.
https://shorturl.at/arQT6
Jiang et al. (2023) Jiang, L., Nie, L. & Yuan, Y. (2023), ‘Elastic priors to dynamically borrow information from historical data in clinical trials’, Biometrics 79(1), 49–60.
Kalil et al. (2021) Kalil, A. C., Patterson, T. F., Mehta, A. K., Tomashek, K. M., Wolfe, C. R., Ghazaryan, V., Marconi, V. C., Ruiz-Palacios, G. M., Hsieh, L., Kline, S. et al. (2021), ‘Baricitinib plus remdesivir for hospitalized adults with covid-19’, New England Journal of Medicine 384(9), 795–807.
Kennedy (2022) Kennedy, E. H. (2022), ‘Semiparametric doubly robust targeted double machine learning: a review’, arXiv preprint arXiv:2203.06469 .
Kennedy et al. (2021) Kennedy, E. H., Balakrishnan, S. & Wasserman, L. (2021), ‘Semiparametric counterfactual density estimation’, arXiv preprint arXiv:2102.12034 .
Koenig et al. (2024) Koenig, F., Spiertz, C., Millar, D., Rodríguez-Navarro, S., Machín, N., Van Dessel, A., Genescà, J., Pericàs, J. M., Posch, M., Sánchez-Montalva, A. et al. (2024), ‘Current state-of-the-art and gaps in platform trials: 10 things you should know, insights from eu-pearl’, Eclinicalmedicine 67.
Lee & Wason (2020) Lee, K. M. & Wason, J. (2020), ‘Including non-concurrent control patients in the analysis of platform trials: is it worth it?’, BMC medical research methodology 20(1), 1–12.
Luedtke et al. (2019) Luedtke, A., Carone, M. & van der Laan, M. J. (2019), ‘An omnibus non-parametric test of equality in distribution for unknown functions’, Journal of the Royal Statistical Society Series B: Statistical Methodology 81(1), 75–99.
Neuenschwander et al. (2009) Neuenschwander, B., Branson, M. & Spiegelhalter, D. J. (2009), ‘A note on the power prior’, Stat Med. 28.
https://doi.org/10.1002/sim.3722
Normington et al. (2020) Normington, J., Zhu, J., Mattiello, F., Sarkar, S. & Carlin, B. (2020), ‘An efficient bayesian platform trial design for borrowing adaptively from historical control data in lymphoma’, Contemp Clin Trials. 89.
https://doi.org/10.1016/j.cct.2019.105890
Park et al. (2022) Park, J. J., Detry, M. A., Murthy, S., Guyatt, G. & Mills, E. J. (2022), ‘How to use and interpret the results of a platform trial: users’ guide to the medical literature’, Jama 327(1), 67–74.
Pearl (1995) Pearl, J. (1995), ‘Causal diagrams for empirical research’, Biometrika 82(4), 669–688.
Pearl (2010) Pearl, J. (2010), ‘An introduction to causal inference’, The International Journal of Biostatistics 6(2), 7.
Saville et al. (2022) Saville, B. R., Berry, D. A., Berry, N. S., Viele, K. & Berry, S. M. (2022), ‘The bayesian time machine: accounting for temporal drift in multi-arm platform trials’, Clinical Trials 19(5), 490–501.
Schmidli et al. (2014) Schmidli, H., Gsteiger, S., Roychoudhury, S., O’Hagan, A., Spiegelhalter, D. & Neuenschwander, B. (2014), ‘Robust meta-analytic-predictive priors in clinical trials with historical control information’, Biometrics 70(4), 1023–1032.
Schmidli et al. (2020) Schmidli, H., Häring, D. A., Thomas, M., Cassidy, A., Weber, S. & Bretz, F. (2020), ‘Beyond randomized clinical trials: Use of external controls’, Clin Pharmacol Ther. 107.
https://doi.org/10.1002/cpt.1723
Shin & Antonelli (2023) Shin, H. & Antonelli, J. (2023), ‘Improved inference for doubly robust estimators of heterogeneous treatment effects’, Biometrics 79(4), 3140–3152.
Sridhara et al. (2022) Sridhara, R., Marchenko, O., Jiang, Q., Pazdur, R., Posch, M., Berry, S., Theoret, M., Shen, Y. L., Gwise, T., Hess, L. et al. (2022), ‘Use of nonconcurrent common control in master protocols in oncology trials: report of an american statistical association biopharmaceutical section open forum discussion’, Statistics in Biopharmaceutical Research 14(3), 353–357.
Viele et al. (2014) Viele, K., Berry, S., Neuenschwander, B., Amzal, B., Chen, F. & Enas, N. (2014), ‘Use of historical control data for assessing treatment effects in clinical trials’, Pharm Stat. 13.
https://doi.org/10.1002/pst.1589
Wei et al. (2024) Wei, W., Blaha, O., Esserman, D., Zelterman, D., Kane, M., Liu, R. & Lin, J. (2024), ‘A bayesian platform trial design with hybrid control based on multisource exchangeability modelling’, Statistics in Medicine 43(12), 2439–2451.
Wells et al. (2012) Wells, A., Fisher, P., Myers, S., Wheatley, J., Patel, T. & Brewin, C. R. (2012), ‘Metacognitive therapy in treatment-resistant depression: A platform trial’, Behaviour research and therapy 50(6), 367–373.
Woodcock & LaVange (2017) Woodcock, J. & LaVange, L. M. (2017), ‘Master protocols to study multiple therapies, multiple diseases, or both’, New England Journal of Medicine 377(1), 62–70.
Yuan et al. (2019) Yuan, J., Liu, J., Zhu, R., Lu, Y. & Palm, U. (2019), ‘Design of randomized controlled confirmatory trials using historical control data to augment sample size for concurrent controls’, J Biopharm Stat. 29.
https://doi.org/10.1080/10543406.2018.1559853

SUPPLEMENTARY MATERIAL
Identification and estimation of causal effects using non-concurrent controls in platform trials
Michele Santacatterina, Federico Macchiavelli Giron, Xinyi Zhang, and Iván Díaz

Division of Biostatistics, Department of Population Health,

New York University School of Medicine,

New York, NY, 10016

Proofs and M-estimation details:: Theorem 1, 2, and 3 and M-estimation details for estimators based on outcome regression and parametric weighting.

Proofs and M-estimation details

Proof of Theorem 1

Figure 6: DAG associated to the structural equation model in equation (6).

$\displaystyle E_{i}$	$\displaystyle=f_{E}(U_{E,i}),$
$\displaystyle W_{i}$	$\displaystyle=f_{W}(E_{i},U_{W,i}),$
$\displaystyle V_{k,i}$	$\displaystyle=f_{V_{k}}(E_{i},U_{V}),$	(6)
$\displaystyle A_{i}$	$\displaystyle=f_{A}(V_{i},W_{i},U_{A,i}),$
$\displaystyle Y_{i}(k)$	$\displaystyle=f_{Y(k)}(k,W_{i},E_{i},U_{Y,i}).$

Since we are interested in the effect of $A$ on $Y$ , and in using non-concurrent controls, $V_{k}=0$ , we study paths from $A$ to $Y(k)$ and then apply d-separation. We start by studying paths from $A$ to $Y(k)$ .

	$\displaystyle A\leftarrow V_{k}\leftarrow E\rightarrow Y(k)$	$\displaystyle\{V_{k}\};\{E\};\{V_{k},E\}$
	$\displaystyle A\leftarrow V_{k}\leftarrow E\rightarrow W\rightarrow Y(k)$	$\displaystyle\{V_{k}\};\{E\};\{W\};\{V_{k},E\};\{V_{k},W\};\{E,W\};\{V_{k},W,E\}$
	$\displaystyle A\leftarrow W\rightarrow Y(k)$	$\displaystyle\{W\}$
	$\displaystyle A\leftarrow W\leftarrow E\rightarrow Y(k)$	$\displaystyle\{W\};\{E\};\{W,E\}.$

By applying d-separation, the set $\{V_{k},W,E\}$ conditionally block the path from $A$ to $Y(k)$ . This leads to the following assumptions:

A1weak A-ignorability.

Let $a=0,\ldots,K$ . Assume
$\mathsf{E}[Y(k)|W=w,E=e,V_{k}=v]=\mathsf{E}[Y(k)|A=k,W=w,E=e,V_{k}=v]$ .

A2Consistency.

Assume
$\mathsf{P}(Y(k)|A=k,W=w,E=e,V_{k}=v)=\mathsf{P}(Y|A=k,W=w,E=e,V_{k}=v)$ .

A3Positivity of treatment assignment mechanism among concurrent units.

Assume
$\mathsf{P}(A=k\mid W=w,E=e,V_{k}=1)>0$ for all $w$ and $e$ s.t. $V_{k}=1$ .

A4Positivity of shared arm assignment mechanism among all controls.

Assume
$\mathsf{P}(A=0\mid W=w,E=e)>0$ for all $w$ and $e$ .

A5Pooling concurrent and non-concurrent controls.

Assume
$\mathsf{E}(Y\mid A=0,W=w,E=e,V_{k}=1)=\mathsf{E}(Y\mid A=0,W=w,E=e)$ for all $e$ s.t. $V_{k}=1$ .

A6Conditional exchangeability of outcome mechanism among the treated.

Assume
$\mathsf{E}(Y\mid A=k,W=w,E=e,V_{k}=1)=\mathsf{E}(Y\mid A=k,W=w,E=e)$ for all $e$ .

Identification of concurrent ATE

Recall that

Definition 3 (Conditional and marginal average treatment effect of treatment arm $k$ compared to control among concurrent population).

	$\displaystyle\mathsf{cCATE}(k,w,e)$	$\displaystyle=\mathsf{E}[Y(k)-Y(0)\mid V_{k}=1,W=w,E=e]$
	$\displaystyle\mathsf{cATE}(k)$	$\displaystyle=\mathsf{E}[\mathsf{cCATE}(k,W,E)\mid V_{k}=1].$

Identification based on the G-formula.

Proof We start by showing it for treatment $k$ . We refer to $W=w,E=e$ as $W,E$ for clarity and (IE) as iterated expectation.

	$\displaystyle\mathsf{E}(Y(k)\mid V_{k}=1)$
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(k)\|V_{k}=1,W,E)\mid V_{k}=1)$	by (IE)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(k)\|A=k,V_{k}=1,W,E)\mid V_{k}=1)$	by (A1)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=k,V_{k}=1,W,E)\mid V_{k}=1)$	by (A2,A3)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}(\mathds{1}[V_{k}=1]% \mathsf{E}(Y\|A=k,V_{k}=1,W,E))$

We now show the proof under treatment 0.

	$\displaystyle\mathsf{E}(Y(0)\mid V_{k}=1)$
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(0)\|V_{k}=1,W,E)\mid V_{k}=1)$	by (IE)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(0)\|A=0,V_{k}=1,W,E)\mid V_{k}=1)$	by (A1)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=0,V_{k}=1,W,E)\mid V_{k}=1)$	by (A2,A4)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=0,W,E)\mid V_{k}=1)$	by (A5)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}(\mathds{1}[V_{k}=1]% \mathsf{E}(Y\|A=0,W,E))$

Consequently, under (A1)-(A4), $\mathsf{cCATE}(k,w,e)$ is non-parametrically identified as

\mathsf{E}(Y\mid A=k,V_{k}=1,W,E)-\mathsf{E}(Y\mid A=0,V_{k}=1,W,E).

(7)

In addition, under (A1)-(A5), $\mathsf{cCATE}(k,w,e)$ is identified as

\mathsf{E}(Y\mid A=k,V_{k}=1,W,E)-\mathsf{E}(Y\mid A=0,W,E).

(8)

∎

Identification based on weighting.

Proof We start by showing it for treatment $k$ . We refer to $W=w,E=e$ as $W,E$ for clarity, and (IE) as iterated expectation.

	$\displaystyle\mathsf{E}(Y(k)\mid V_{k}=1)$
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(k)\|V_{k}=1,W,E)\mid V_{k}=1)$	by (IE)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(k)\|A=k,V_{k}=1,W,E)\mid V_{k}=1)$	by (A1)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=k,V_{k}=1,W,E)\mid V_{k}=1)$	by (A2)
	$\displaystyle=\mathsf{E}\left(\mathsf{E}\left(\frac{\mathds{1}[A=k,V_{k}=1]Y}{% \mathsf{P}(A=k\|V_{k}=1,W,E)}\|W,E\right)\mid V_{k}=1\right)$	by (A3)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathds{1}[V_{k}=1]% \mathsf{E}\left(\frac{\mathds{1}[A=k,V_{k}=1]Y}{\mathsf{P}(A=k\|V_{k}=1,W,E)}\|W% ,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=k,V_{k}=1]Y}{\mathsf{P}(A=k\|V_{k}=1,W,E)}\|W,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=k]Y\mathsf{P}(V_{k}=1\|W,E)}{\mathsf{P}(A=k\|V_{k}=1,W,E)}\|W,% E\right)\right)$	by (IE)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\frac{\mathds{1}[A=% k]Y\mathsf{P}(V_{k}=1\|W,E)}{\mathsf{P}(A=k\|V_{k}=1,W,E)}\right)$

Note that if $V_{k}$ is deterministic, then $\mathsf{P}(V_{k}=1|W,E)=\mathds{1}[E>t]=\mathds{1}[V_{k}=1]$ and therefore

\displaystyle\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\frac{\mathds{1}[A=k% ]Y\mathsf{P}(V_{k}=1|W,E)}{\mathsf{P}(A=k|V_{k}=1,W,E)}\right)=\frac{1}{% \mathsf{P}(V_{k}=1)}\mathsf{E}\left(\frac{\mathds{1}[A=k]Y\mathds{1}[V_{k}=1]}% {\mathsf{P}(A=k|V_{k}=1,W,E)}\right).

We now show the proof under treatment 0.

	$\displaystyle\mathsf{E}(Y(0)\mid V_{k}=1)$
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(0)\|V_{k}=1,W,E)\mid V_{k}=1)$	by (IE)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(0)\|A=0,V_{k}=1,W,E)\mid V_{k}=1)$	by (A1)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=0,V_{k}=1,W,E)\mid V_{k}=1)$	by (A2)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=0,W,E)\mid V_{k}=1)$	by (A5)
	$\displaystyle=\mathsf{E}\left(\mathsf{E}\left(\frac{\mathds{1}[A=0]Y}{\mathsf{% P}(A=0\|W,E)}\|W,E\right)\mid V_{k}=1\right)$	by (A4)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathds{1}[V_{k}=1]% \mathsf{E}\left(\frac{\mathds{1}[A=0]Y}{\mathsf{P}(A=0\|W,E)}\|W,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}(\mathds{% 1}[V_{k}=1]\|W,E)\mathsf{E}\left(\frac{\mathds{1}[A=0]Y}{\mathsf{P}(A=0\|W,E)}\|W% ,E\right)\right)$	by (IE)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=0]Y\mathds{1}[V_{k}=1]}{\mathsf{P}(A=0\|W,E)}\|W,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=0]Y\mathsf{E}(\mathds{1}[V_{k}=1]\|W,E)}{\mathsf{P}(A=0\|W,E)% }\|W,E\right)\right)$	by (IE)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=0]Y\mathsf{P}(V_{k}=1\|W,E)}{\mathsf{P}(A=0\|W,E)}\|W,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\frac{\mathds{1}[A=% 0]Y\mathsf{P}(V_{k}=1\|W,E)}{\mathsf{P}(A=0\|W,E)}\right)$

Note that if $V_{k}$ is deterministic, then $\mathsf{P}(V_{k}=1|W,E)=\mathds{1}[E>t]=\mathds{1}[V_{k}=1]$ and therefore

\displaystyle\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\frac{\mathds{1}[A=0% ]Y\mathsf{P}(V_{k}=1|W,E)}{\mathsf{P}(A=0|W,E)}\right)=\frac{1}{\mathsf{P}(V_{% k}=1)}\mathsf{E}\left(\frac{\mathds{1}[A=0]Y\mathds{1}[V_{k}=1]}{\mathsf{P}(A=% 0|W,E)}\right).

∎

Proof of Theorem 2

Identification of ATE

Recall that

Definition 4 (Conditional and marginal average treatment effect of treatment arm $k$ compared to control).

	$\displaystyle\mathsf{CATE}(k,e,w)$	$\displaystyle=\mathsf{E}[Y(k)-Y(0)\mid W=w,E=e]$
	$\displaystyle\mathsf{ATE}(k)$	$\displaystyle=\mathsf{E}[\mathsf{CATE}(k,W,E)].$

Identification based on the G-formula.

Proof We show the proof for treatment $0$ . We refer to $W=w,E=e$ as $W,E$ for clarity.

$\displaystyle\mathsf{E}(Y(0))$	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(0)\|W,E))$	by (IE)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(0)\|A=0,W,E))$	by (A1)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=0,W,E))$	by (A2,A4)

The proof for treatment $k$ can be shown by following the steps for identifying $\mathsf{E}(Y(k)\mid V_{k}=1)$ in the section above and then assuming (A6) to be able to marginalize to concurrent and non-concurrent controls combined. Consequently, under (A1-A6), $\mathsf{CATE}(k,w,e)$ is identified as

\mathsf{E}(Y\mid A=k,W,E,V_{k}=1)-\mathsf{E}(Y\mid A=0,W,E).

(9)

∎

M-estimation details

We here provide detail on the M-estimation approach for obtaining asymptotic variances for outcome regression and weighted estimators. Recall that $Z_{i}$ represent the data for the experimental unit $i$ , i.e., $Z_{i}=(E_{i},W_{i},V_{k,i},A_{i},Y_{i})\sim\mathsf{P}$ and consider $X_{i}=(E_{i},W_{i})$ .

Outcome regression

$\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$

. This estimator consider only concurrent controls. Let’s define $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}=\mu_{k}-\mu_{0}$ where $\mu_{k}$ and $\mu_{0}$ are the mean outcomes under treatment $k$ and control in the only concurrent control population. We started by considering controls, $\theta_{0}=(\beta_{0},\mu_{0})$ and the following estimating equations

\displaystyle\sum_{i=1}^{n}h(Z_{i},\theta_{0})

\displaystyle=\sum_{i=1}^{n}\begin{pmatrix}h_{1}(Z_{i},\beta_{0})\\ h_{2}(Z_{i},\mu_{0})\end{pmatrix}=0

where $h_{1}(Z_{i},\beta_{0})=X_{i}^{\top}V_{i}(1-A_{i})(Y_{i}-X_{i}\beta_{0})$ and $h_{2}(X_{i},\mu_{0})=V_{i}(Z_{i}\beta_{0}-\mu_{0})$ are the score functions for the model of the conditional mean and the the marginal mean under control, respectively. We consider the following Jacobian matrix of the estimating equations,

	$\displaystyle\overline{\mathbf{G}}(\hat{\theta}_{0})$	$\displaystyle=-\frac{1}{n}\sum_{i=1}^{n}\frac{\partial h(Z_{i},\theta_{0})}{% \partial\theta_{0}^{\top}}\biggr{\rvert}_{\theta_{0}=\hat{\theta}_{0}}$
		$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\begin{pmatrix}\overline{\mathbf{G}}_{1% 1}&\mathbf{0}\\ \overline{\mathbf{G}}_{21}&\overline{\mathbf{G}}_{22}\end{pmatrix}$
		$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\begin{pmatrix}X_{i}^{\top}V_{i}(1-A_{i% })X_{i}&0\\ -V_{i}X_{i}&V_{i}\end{pmatrix}$

We then constructed the following influence functions

	$\displaystyle\varphi(Z_{i},\hat{\beta}_{0})$	$\displaystyle=\overline{\mathbf{G}}_{11}^{-1}h_{1}(Z_{i},\hat{\beta}_{0})$
	$\displaystyle\varphi(Z_{i},\hat{\mu}_{0})$	$\displaystyle=\overline{\mathbf{G}}_{22}^{-1}\left(h_{2}(Z_{i},\mu_{0})+(-% \overline{\mathbf{G}}_{21})\varphi(Z_{i},\hat{\beta}_{0})\right),$

where $\hat{\beta}_{0}$ where obtained by ordinary least squares. We conducted a similar analysis for $\theta_{k}=(\beta_{k},\mu_{k})$ . Finally, we obtained the variance of $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$ as,

\displaystyle\hat{V}(\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}})

\displaystyle=\frac{1}{n}\left(\frac{1}{n}\sum_{i=1}^{n}\varphi(Z_{i},\hat{% \mathsf{cATE}}_{\text{OR}}^{\text{oc}})\varphi(Z_{i},\hat{\mathsf{cATE}}_{% \text{OR}}^{\text{oc}})^{\top}\right),

where $\varphi(Z_{i},\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}})=\varphi(Z_{i},\hat{% \mu}_{k})-\varphi(Z_{i},\hat{\mu}_{0})$ .

$\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}$

. This estimator consider both concurrent and non concurrent controls when estimating $\mathsf{E}(Y\mid A=0,W=w,E=e)$ . Hence, the analysis for $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}$ looks the same as that for $\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$ only changing the estimating equation for $\mathsf{E}(Y\mid A=0,W=w,E=e)$ , i.e., $h_{1}(Z_{i},\beta_{0})=Z_{i}^{\top}(1-A_{i})(Y_{i}-Z_{i}\beta_{0})$ , while the conditional mean of the outcome among the treated remains computed within only concurrent, i.e., $\mathsf{E}(Y\mid A=k,V_{k}=1,W=w,E=e)$ . Specifically, we started by considering controls, $\theta_{0}=(\alpha_{0},\mu_{0})$ and the following estimating equations

\displaystyle\sum_{i=1}^{n}h(Z_{i},\theta_{0})

\displaystyle=\sum_{i=1}^{n}\begin{pmatrix}h_{1}(Z_{i},\alpha_{0})\\ h_{2}(Z_{i},\mu_{0})\end{pmatrix}=0

where $h_{1}(Z_{i},\alpha_{0})=X_{i}^{\top}(1-A_{i})(Y_{i}-X_{i}\alpha_{0})$ and $h_{2}(Z_{i},\mu_{0})=V_{i}(X_{i}\alpha_{0}-\mu_{0})$ are the score functions for the model of the conditional mean and the the marginal mean under control, respectively. While for the treated units we considered, $\theta_{k}=(\beta_{k},\mu_{k})$ and the following estimating equations

\displaystyle\sum_{i=1}^{n}h(Z_{i},\theta_{k})

\displaystyle=\sum_{i=1}^{n}\begin{pmatrix}h_{1}(Z_{i},\beta_{k})\\ h_{2}(Z_{i},\mu_{k})\end{pmatrix}=0

where $h_{1}(Z_{i},\beta_{k})=X_{i}^{\top}V_{i}A_{i}(Y_{i}-X_{i}\beta_{k})$ and $h_{2}(Z_{i},\mu_{k})=V_{i}(X_{i}\beta_{k}-\mu_{k})$ . Derivation of the Jacobian matrix of the estimating equations is similar to the above.

Parametric inverse probability weighting

$\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}}$

. This estimator consider only concurrent controls. Let’s define $\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}}=\mu_{k}-\mu_{0}$ . We started by considering controls, $\theta=(\eta,\mu_{0},\mu_{1})$ and the following estimating equations

\displaystyle\sum_{i=1}^{n}h(Z_{i},\theta)

\displaystyle=\sum_{i=1}^{n}\begin{pmatrix}h_{1}(Z_{i},\eta)\\ h_{2}(Z_{i},\mu_{0})\\ h_{3}(Z_{i},\mu_{1})\end{pmatrix}=0

where $h_{1}(Z_{i},\eta)=X_{i}^{\top}V_{i}(A_{i}-\pi)$ and $h_{2}(Z_{i},\mu_{0})=V_{i}(\gamma^{0}_{i}Y_{i}-\mu_{0})$ , $h_{3}(Z_{i},\mu_{1})=V_{i}(\gamma^{1}_{i}Y_{i}-\mu_{1})$ are the score functions for the model of the conditional probability and the marginal mean under control and treatment, respectively, and where $\gamma^{0}_{i}=\mathds{1}\{{A_{i}=0}\}/(1-\pi_{i})$ , $\gamma^{k}_{i}=\mathds{1}\{{A_{i}=k}\}/(\pi_{i})$ , and $\pi_{i}=\frac{\exp(X_{i}^{\top}\eta)}{1+\exp(X_{i}^{\top}\eta)}$ . We consider the following Jacobian matrix of the estimating equations,

	$\displaystyle\overline{\mathbf{G}}(\hat{\theta}_{0})$	$\displaystyle=-\frac{1}{n}\sum_{i=1}^{n}\frac{\partial h(Z_{i},\theta_{0})}{% \partial\theta_{0}^{\top}}\biggr{\rvert}_{\theta=\hat{\theta}}$
		$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\begin{pmatrix}\overline{\mathbf{G}}_{1% 1}&\mathbf{0}&\mathbf{0}\\ \overline{\mathbf{G}}_{21}&\overline{\mathbf{G}}_{22}&\mathbf{0}\\ \overline{\mathbf{G}}_{31}&\mathbf{0}&\overline{\mathbf{G}}_{33}\\ \end{pmatrix}$
		$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\begin{pmatrix}X_{i}^{\top}V_{i}\frac{% \exp(X_{i}^{\top}\eta)}{1+\exp(X_{i}^{\top}\eta)^{2}}X_{i}&0&0\\ (1-A_{i})V_{i}X_{i}Y_{i}\exp(X_{i}^{\top}\eta)&V_{i}&0\\ A_{i}V_{i}X_{i}Y_{i}\exp(-X_{i}^{\top}\eta)&0&V_{i}\\ \end{pmatrix}$

We then constructed the following influence functions

	$\displaystyle\varphi(Z_{i},\hat{\eta})$	$\displaystyle=\overline{\mathbf{G}}_{11}^{-1}h_{1}(Z_{i},\hat{\eta}),$
	$\displaystyle\varphi(Z_{i},\hat{\mu}_{0})$	$\displaystyle=\overline{\mathbf{G}}_{22}^{-1}\left(h_{2}(Z_{i},\mu_{0})+(-% \overline{\mathbf{G}}_{21})\varphi(Z_{i},\hat{\eta})\right),$
	$\displaystyle\varphi(Z_{i},\hat{\mu}_{k})$	$\displaystyle=\overline{\mathbf{G}}_{33}^{-1}\left(h_{3}(Z_{i},\mu_{k})+(-% \overline{\mathbf{G}}_{31})\varphi(Z_{i},\hat{\eta})\right)$

where $\hat{\eta}$ where obtained by ordinary least squares. Finally, we obtained the variance of $\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}}$ as,

\displaystyle\hat{V}(\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}})

\displaystyle=\frac{1}{n}\left(\frac{1}{n}\sum_{i=1}^{n}\varphi(Z_{i},\hat{% \mathsf{cATE}}_{\text{IPW}}^{\text{oc}})\varphi(Z_{i},\hat{\mathsf{cATE}}_{% \text{IPW}}^{\text{oc}})^{\top}\right),

where $\varphi(Z_{i},\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}})=\varphi(Z_{i},\hat% {\mu}_{k})-\varphi(Z_{i},\hat{\mu}_{0})$ .

Proof of Theorem 3

We start by introducing some notation. We introduce an operator $\mathsf{IF}:\psi\rightarrow L_{2}(\mathds{P})$ , where $\mathds{P}$ is a probability distribution assumed to lie in some nonparametric model $\mathcal{P}$ , that maps functionals $\psi:\mathcal{P}\rightarrow\mathds{R}$ to their influence function $\varphi(z)\in L_{2}(\mathds{P})$ and where $z$ is our observed data. Recall the following building blocks:

(bb1)

the influence function of $\mu(x)=\mathsf{E}[Y|X=x]$ is $\mathsf{IF}(\mu(x))=\frac{\mathds{1}[X=x]}{\mathsf{P}(X=x)}(Y-\mathsf{E}[Y|X=x])$
(bb2)

the influence function of $\mathsf{p}(x)=\mathsf{P}(X=x)$ is $\mathsf{IF}(p(x))=(\mathds{1}[X=x]-p(x))$
(bb3)

$\mathsf{IF}(\psi_{1}\psi_{2})=\mathsf{IF}(\psi_{1})\psi_{2}+\psi_{1}\mathsf{IF% }(\psi_{2})$ (product rule)
(bb4)

$\mathsf{IF}(f(\psi))=\psi^{\prime}\mathsf{IF}(\psi)$ (chain rule)
(bb5)

$\mathsf{P}(A,B,C)=\mathsf{P}(A|B,C)\mathsf{P}(B,C)=\mathsf{P}(A|B,C)\mathsf{P}% (B|C)\mathsf{P}(C)$
(bb6)

$\sum_{x}\mathds{1}[A=k,X=x]=\mathds{1}[A=k]$ .

Finally, recall that the parameter of interest (under the aforementioned identification assumption) is

\displaystyle\mathsf{cATE}(k)

\displaystyle=\mathsf{E}[\mathsf{E}[Y|A=k,W-w,E=e,V_{k}=1]-\mathsf{E}[Y|A=0,W-% w,E=e,V_{k}=1]\mid V_{k}=1].

while in the nonparametric model that assumes (A5) is

\displaystyle\mathsf{cATE}(k)

\displaystyle=\mathsf{E}[\mathsf{E}[Y|A=k,W-w,E=e,V_{k}=1]-\mathsf{E}[Y|A=0,W-% w,E=e]\mid V_{k}=1].

Theorem 3, eq. (4).

We define $(X=x)=(W=w,E=e)$ and pretend that the data is discrete. Recall that under discrete data

	$\displaystyle\mathsf{E}[\mathsf{E}[Y\|A=1,X=x,V_{k}=1]\mid V_{k}=1]$	$\displaystyle=\frac{\mathsf{E}[\mathds{1}[V_{k}=1]\mathsf{E}[Y\|A=1,X=x,V_{k}=1% ]]}{P(V_{k}=1)}$
		$\displaystyle=\frac{\sum_{x}\mathds{1}[V_{k}=1]\mathsf{E}[Y\|A=1,X=x,V_{k}=1]P(% X=x)}{P(V_{k}=1)}$
		$\displaystyle=\frac{\sum_{x}\mathds{1}[V_{k}=1]\mu_{\text{oc}}(1,x,1)p(x)}{P(V% _{k}=1)}$
		$\displaystyle=\frac{\psi^{1}_{num}}{\psi_{den}}=\psi^{1}.$

We now analyze the influence function of $\psi^{1}_{num}$ ,

	$\displaystyle\varphi(Z,\psi^{1}_{num})\equiv\mathsf{IF}\{\psi^{1}_{num}\}$
	$\displaystyle=\mathsf{IF}\{\sum_{x}\mathds{1}[V_{k}=1]\mu_{\text{oc}}(1,x,1)p(% x)\}$
	$\displaystyle=\mathds{1}[V_{k}=1]\sum_{x}\left[\mathsf{IF}\{\mu_{\text{oc}}(1,% x,1)\}p(x)+\mu(1,x,1)\mathsf{IF}\{p(x)\}\right]$	by (bb3)
	$\displaystyle=\mathds{1}[V_{k}=1]\sum_{x}\left[\left(\frac{\mathds{1}[A=1,X=x,% V_{k}=1]}{p(1,x,1)}\{Y-\mu_{\text{oc}}(1,x,1)\}\right)\right.p(x)$
	$\displaystyle+\mu_{\text{oc}}(1,x,1)\left(\mathds{1}[X=x]-p(x)\right)\left.\right]$	by (bb1,bb2)
	$\displaystyle=\mathds{1}[V_{k}=1]\sum_{x}\left[\left(\frac{\mathds{1}[A=1,X=x,% V_{k}=1]}{\mathsf{P}(A=1\mid X=x,V_{k}=1)\mathsf{P}(V_{k}=1\|X=x)}\{Y-\mu_{% \text{oc}}(1,x,1)\}\right)\right.$
	$\displaystyle+\mu_{\text{oc}}(1,x,1)\mathds{1}[X=x]-\mu_{\text{oc}}(1,x,1)p(x)% \left.\right]$	by (bb5)
	$\displaystyle=\mathds{1}[V_{k}=1]\left[\left(\frac{\mathds{1}[A=1]}{\mathsf{P}% (A=1\mid X=x,V_{k}=1)}\{Y-\mu_{\text{oc}}(1,x,1)\}\right)+\mu_{\text{oc}}(1,x,% 1)\right]-\psi^{1}_{num}$	by (bb6)

where in the last equality we also used the fact that $\mathsf{P}(V_{k}=1|X=x)=1$ under $\mathds{1}[V_{k}=1]$ , and $\psi^{1}_{num}=\sum_{x}\mathds{1}[V_{k}=1]\mu_{\text{oc}}(1,x,1)p(x)$ . Analogously we can compute the influence function of $\psi^{0}_{num}$ ,

	$\displaystyle\varphi(Z,\psi^{0}_{num})\equiv\mathsf{IF}\{\psi^{0}_{num}\}$
	$\displaystyle=\mathsf{IF}\{\sum_{x}\mathds{1}[V_{k}=1]\mu_{\text{oc}}(0,x,1)p(% x)\}$
	$\displaystyle=\mathds{1}[V_{k}=1]\left[\left(\frac{\mathds{1}[A=0]}{\mathsf{P}% (A=0\mid X=x,V_{k}=1)}\{Y-\mu_{\text{oc}}(0,x,1)\}\right)+\mu_{\text{oc}}(0,x,% 1)\right]-\psi^{0}_{num}$

We no compute the influence function of $\psi_{den}$ ,

\displaystyle\varphi(Z,\psi_{den})\equiv\mathsf{IF}\{\psi_{den}\}=\mathds{1}[V% _{k}=1]-\psi_{den}

We no consider the influence function of $\frac{\psi^{1}_{num}}{\psi_{den}}=\psi^{1}$ ,

	$\displaystyle\varphi(Z,\psi^{1})\equiv\mathsf{IF}\{\psi^{1}\}=\frac{\mathsf{IF% }\{\psi^{1}_{num}\}}{\psi_{den}}-\frac{\psi^{1}_{num}}{\psi_{den}}\frac{% \mathsf{IF}\{\psi_{den}\}}{\psi_{den}}$
	$\displaystyle=\frac{1}{\psi_{den}}\left[\mathsf{IF}\{\psi^{1}_{num}\}-\frac{% \psi^{1}_{num}}{\psi_{den}}\mathsf{IF}\{\psi_{den}\}\right]$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\left[\left(\mathds{1}[V_{k}=1]% \left[\left(\frac{\mathds{1}[A=1]}{\mathsf{P}(A=1\mid X=x,V_{k}=1)}\{Y-\mu_{% \text{oc}}(1,x,1)\}\right)+\mu_{\text{oc}}(1,x,1)\right]-\psi^{1}_{num}\right)\right.$
	$\displaystyle-\left.\frac{\psi^{1}_{num}}{\psi_{den}}(\mathds{1}[V_{k}=1]-\psi% _{den})\right]$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\left[\left(\mathds{1}[V_{k}=1]% \left[\left(\frac{\mathds{1}[A=1]}{\mathsf{P}(A=1\mid X=x,V_{k}=1)}\{Y-\mu_{% \text{oc}}(1,x,1)\}\right)+\mu_{\text{oc}}(1,x,1)\right]-\psi^{1}_{num}\right)\right.$
	$\displaystyle-\left.\frac{\psi^{1}_{num}}{\psi_{den}}\mathds{1}[V_{k}=1]+\frac% {\psi^{1}_{num}}{\psi_{den}}\psi_{den}\right]$
	$\displaystyle=\frac{\mathds{1}[V_{k}=1]}{\mathsf{P}(V_{k}=1)}\left[\frac{% \mathds{1}[A=1]}{\mathsf{P}(A=1\mid X=x,V_{k}=1)}\{Y-\mu_{\text{oc}}(1,x,1)\}+% \mu_{\text{oc}}(1,x,1)-\psi^{1}\right]$

We can now combine $\frac{\psi^{1}_{num}-\psi^{0}_{num}}{\psi_{den}}=\mathsf{cATE}(k)$ to obtain

	$\displaystyle\varphi(Z,\mathsf{cATE}(k))\equiv\mathsf{IF}\{\mathsf{cATE}(k)\}$	$\displaystyle=\frac{\mathds{1}{\{V_{k}=1\}}}{\mathsf{P}(V_{k}=1)}\bigg{[}\frac% {2A-1}{\mathsf{P}(A\mid W,E,V_{k}=1)}\{Y-\mathsf{E}(Y\mid A,W,E,V_{k}=1)\}$
		$\displaystyle+\mathsf{E}(Y\mid A=1,W,E,V_{k}=1)-\mathsf{E}(Y\mid A=0,W,E,V_{k}% =1)-\mathsf{cATE}(k).\bigg{]}$

Theorem 3, eq. (5).

Under assumption (A5), we now target (among controls),

	$\displaystyle\mathsf{E}[\mathsf{E}[Y\|A=0,X=x]\mid V_{k}=1]$	$\displaystyle=\frac{\mathsf{E}[\mathds{1}[V_{k}=1]\mathsf{E}[Y\|A=0,X=x]]}{P(V_% {k}=1)}$
		$\displaystyle=\frac{\mathsf{E}[\mathsf{E}[\mathds{1}[V_{k}=1]\mid X=x]\mathsf{% E}[Y\|A=0,X=x]]}{P(V_{k}=1)}$
		$\displaystyle=\frac{\sum_{x}\mathsf{P}(V_{k}=1\mid X=x)\mathsf{E}[Y\|A=0,X=x]P(% X=x)}{P(V_{k}=1)}$
		$\displaystyle=\frac{\sum_{x}\nu(w_{i},e_{i})\mu_{\text{all}}(1,x)p(x)}{P(V_{k}% =1)}$
		$\displaystyle=\frac{\psi^{0}_{num}}{\psi_{den}}=\psi^{0}.$

The influence function of $\psi^{0}_{num}$ is

	$\displaystyle\varphi(Z,\psi^{0}_{num})\equiv\mathsf{IF}\{\psi^{0}_{num}\}$
	$\displaystyle=\sum_{x}\left[\frac{\mathds{1}[X=x]}{\mathsf{P}(X=x)}(\mathds{1}% [V_{k}=1]-\nu(x))\mu_{\text{all}}(0,x)p(x)\right.$
	$\displaystyle+\nu(x)\left(\frac{\mathds{1}[A=0,X=x]}{\mathsf{P}(A=0\mid X=x)% \mathsf{P}(X=x)}\{Y-\mu_{\text{all}}(0,x)\}\right)p(x)$
	$\displaystyle+\nu(x)\mu_{\text{all}}(0,x)(\mathds{1}[X=x]-p(x))\left.\frac{}{}\right]$
	$\displaystyle=\sum_{x}\left[\frac{\mathds{1}[X=x]\mathds{1}[V_{k}=1]}{\mathsf{% P}(X=x)}\mu_{\text{all}}(0,x)p(x)-\frac{\mathds{1}[X=x]}{\mathsf{P}(X=x)}\nu(x% )\mu_{\text{all}}(0,x)p(x)\right.$
	$\displaystyle+\nu(x)\left(\frac{\mathds{1}[A=0,X=x]}{\mathsf{P}(A=0\mid X=x)% \mathsf{P}(X=x)}\{Y-\mu_{\text{all}}(0,x)\}\right)p(x)$
	$\displaystyle+\mathds{1}[X=x]\nu(x)\mu_{\text{all}}(0,x)-\nu(x)\mu_{\text{all}% }(0,x)p(x))\left.\frac{}{}\right]$
	$\displaystyle=\mathds{1}[V_{k}=1]\mu_{\text{all}}(0,x)+\nu(x)\left(\frac{% \mathds{1}[A=0]}{\mathsf{P}(A=0\mid X=x)}\{Y-\mu_{\text{all}}(0,x)\}\right)-% \psi^{0}_{num}$

As shown before, we can then compute the influence function of $\frac{\psi^{0}_{num}}{\psi_{den}}$ , and finally of $\mathsf{cATE}(k)$ under assumption (A5), leading to,

	$\displaystyle\varphi(Z,\mathsf{cATE}(k))\equiv\mathsf{IF}\{\mathsf{cATE}(k)\}$	$\displaystyle=\frac{\mathds{1}{\{V_{k}=1\}}}{\mathsf{P}(V_{k}=1)}\bigg{[}\frac% {A}{\mathsf{P}(A\mid V_{k}=1,W,E)}\{Y-\mathsf{E}(Y\mid A,W,E,V_{k}=1)\}\bigg{]}$
		$\displaystyle-\frac{1-A}{\mathsf{P}(A\mid W,E)}\frac{\mathsf{P}(V_{k}=1\mid E,% W)}{\mathsf{P}(V_{k}=1)}\{Y-\mathsf{E}(Y\mid A,E,W)\}$
		$\displaystyle+\frac{\mathds{1}{\{V_{k}=1\}}}{\mathsf{P}(V_{k}=1)}\Big{[}% \mathsf{E}(Y\mid A=1,W,E,V_{k}=1)-\mathsf{E}(Y\mid A=0,W,E)-\mathsf{cATE}(k)% \Big{]}$

	$\displaystyle\mathsf{E}(Y(k)\mid V_{k}=1)$
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(k)\|V_{k}=1,W,E)\mid V_{k}=1)$	by (IE)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(k)\|A=k,V_{k}=1,W,E)\mid V_{k}=1)$	by (A1)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=k,V_{k}=1,W,E)\mid V_{k}=1)$	by (A2)
	$\displaystyle=\mathsf{E}\left(\mathsf{E}\left(\frac{\mathds{1}[A=k,V_{k}=1]Y}{% \mathsf{P}(A=k\|V_{k}=1,W,E)}\|W,E\right)\mid V_{k}=1\right)$	by (A3)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathds{1}[V_{k}=1]% \mathsf{E}\left(\frac{\mathds{1}[A=k,V_{k}=1]Y}{\mathsf{P}(A=k\|V_{k}=1,W,E)}\|W% ,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=k,V_{k}=1]Y}{\mathsf{P}(A=k\|V_{k}=1,W,E)}\|W,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=k]Y\mathsf{P}(V_{k}=1\|W,E)}{\mathsf{P}(A=k\|V_{k}=1,W,E)}\|W,% E\right)\right)$	by (IE)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\frac{\mathds{1}[A=% k]Y\mathsf{P}(V_{k}=1\|W,E)}{\mathsf{P}(A=k\|V_{k}=1,W,E)}\right)$

	$\displaystyle\mathsf{E}(Y(0)\mid V_{k}=1)$
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(0)\|V_{k}=1,W,E)\mid V_{k}=1)$	by (IE)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y(0)\|A=0,V_{k}=1,W,E)\mid V_{k}=1)$	by (A1)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=0,V_{k}=1,W,E)\mid V_{k}=1)$	by (A2)
	$\displaystyle=\mathsf{E}(\mathsf{E}(Y\|A=0,W,E)\mid V_{k}=1)$	by (A5)
	$\displaystyle=\mathsf{E}\left(\mathsf{E}\left(\frac{\mathds{1}[A=0]Y}{\mathsf{% P}(A=0\|W,E)}\|W,E\right)\mid V_{k}=1\right)$	by (A4)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathds{1}[V_{k}=1]% \mathsf{E}\left(\frac{\mathds{1}[A=0]Y}{\mathsf{P}(A=0\|W,E)}\|W,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}(\mathds{% 1}[V_{k}=1]\|W,E)\mathsf{E}\left(\frac{\mathds{1}[A=0]Y}{\mathsf{P}(A=0\|W,E)}\|W% ,E\right)\right)$	by (IE)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=0]Y\mathds{1}[V_{k}=1]}{\mathsf{P}(A=0\|W,E)}\|W,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=0]Y\mathsf{E}(\mathds{1}[V_{k}=1]\|W,E)}{\mathsf{P}(A=0\|W,E)% }\|W,E\right)\right)$	by (IE)
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\mathsf{E}\left(% \frac{\mathds{1}[A=0]Y\mathsf{P}(V_{k}=1\|W,E)}{\mathsf{P}(A=0\|W,E)}\|W,E\right)\right)$
	$\displaystyle=\frac{1}{\mathsf{P}(V_{k}=1)}\mathsf{E}\left(\frac{\mathds{1}[A=% 0]Y\mathsf{P}(V_{k}=1\|W,E)}{\mathsf{P}(A=0\|W,E)}\right)$

	$\displaystyle\mathsf{E}[\mathsf{E}[Y\|A=1,X=x,V_{k}=1]\mid V_{k}=1]$	$\displaystyle=\frac{\mathsf{E}[\mathds{1}[V_{k}=1]\mathsf{E}[Y\|A=1,X=x,V_{k}=1% ]]}{P(V_{k}=1)}$
		$\displaystyle=\frac{\sum_{x}\mathds{1}[V_{k}=1]\mathsf{E}[Y\|A=1,X=x,V_{k}=1]P(% X=x)}{P(V_{k}=1)}$
		$\displaystyle=\frac{\sum_{x}\mathds{1}[V_{k}=1]\mu_{\text{oc}}(1,x,1)p(x)}{P(V% _{k}=1)}$
		$\displaystyle=\frac{\psi^{1}_{num}}{\psi_{den}}=\psi^{1}.$

	$\displaystyle\mathsf{E}[\mathsf{E}[Y\|A=0,X=x]\mid V_{k}=1]$	$\displaystyle=\frac{\mathsf{E}[\mathds{1}[V_{k}=1]\mathsf{E}[Y\|A=0,X=x]]}{P(V_% {k}=1)}$
		$\displaystyle=\frac{\mathsf{E}[\mathsf{E}[\mathds{1}[V_{k}=1]\mid X=x]\mathsf{% E}[Y\|A=0,X=x]]}{P(V_{k}=1)}$
		$\displaystyle=\frac{\sum_{x}\mathsf{P}(V_{k}=1\mid X=x)\mathsf{E}[Y\|A=0,X=x]P(% X=x)}{P(V_{k}=1)}$
		$\displaystyle=\frac{\sum_{x}\nu(w_{i},e_{i})\mu_{\text{all}}(1,x)p(x)}{P(V_{k}% =1)}$
		$\displaystyle=\frac{\psi^{0}_{num}}{\psi_{den}}=\psi^{0}.$

Identification and estimation of causal effects using non-concurrent controls in platform trials

Abstract

1 Introduction

2 Notation and setup

2.1 A structural causal model and associated DAG

3 Definition and identification of the concurrent average treatment effect

Definition 1 (Conditional and marginal average treatment effect of treatment arm k𝑘kitalic_k compared to shared control among concurrent population).

3.1 Non-parametric identification

A1weak A-ignorability.

A2Consistency.

A3Positivity of treatment assignment mechanism among concurrent units.

A4Positivity of shared arm assignment mechanism among all controls.

A5Pooling concurrent and non-concurrent controls.

Remark 1.

Theorem 1 (Identification of 𝖼𝖠𝖳𝖤𝖼𝖠𝖳𝖤\mathsf{cATE}sansserif_cATE in platform adaptive trials under Model (1)).

3.2 On the identification of the average treatment effect

Definition 2 (Conditional and marginal average treatment effect of treatment arm k𝑘kitalic_k compared to shared control).

A6Extrapolation of outcome mechanism among the treated.

Theorem 2 (Identification of 𝖠𝖳𝖤⁢(k)𝖠𝖳𝖤𝑘\mathsf{ATE}(k)sansserif_ATE ( italic_k ) in platform adaptive trials under Model (1)).

4 Relation to analytical approaches common in the literature

5 Estimation of 𝖼𝖠𝖳𝖤⁢(k)𝖼𝖠𝖳𝖤𝑘\mathsf{cATE}(k)sansserif_cATE ( italic_k )

5.1 Estimators based on parametric outcome regression

Large sample properties.

5.2 An estimator based on parametric inverse probability weighting

Large sample properties.

5.3 Doubly robust estimators

Theorem 3.

Large sample properties.

Double robustness.

6 Efficiency considerations

Estimators based on outcome regression.

Doubly robust estimators.

7 Simulations

7.1 Setup

Aims

Data-generating mechanisms

Estimands

Methods

Performance metrics

Scenarios

7.2 Results

7.2.1 Bias, variance, MSE, and coverage

7.2.2 Efficiency gains

Summary of results.

8 Practical considerations

What estimand should we target?

Should we pool concurrent and non-concurrent controls? Testing assumption A5.

Should we leverage prognostic baseline variables for additional precision?

What estimator should we use?

Sample size calculation for a prospective trial with non-concurrent controls.

Multiple comparisons.

Summary of practical guidelines.

9 The Adaptive COVID-19 Treatment Platform Trial

Study population and endpoint of interest.

Treatments under study and targeted causal estimand.

Baseline covariates.

Models setup.

Results.

10 Conclusion

References

Proofs and M-estimation details

Proof of Theorem 1

A1weak A-ignorability.

A2Consistency.

A3Positivity of treatment assignment mechanism among concurrent units.

A4Positivity of shared arm assignment mechanism among all controls.

A5Pooling concurrent and non-concurrent controls.

A6Conditional exchangeability of outcome mechanism among the treated.

Identification of concurrent ATE

Definition 3 (Conditional and marginal average treatment effect of treatment arm k𝑘kitalic_k compared to control among concurrent population).

Identification based on the G-formula.

Identification based on weighting.

Proof of Theorem 2

Identification of ATE

Definition 4 (Conditional and marginal average treatment effect of treatment arm k𝑘kitalic_k compared to control).

Identification based on the G-formula.

M-estimation details

Outcome regression

𝖼𝖠𝖳𝖤^ORocsuperscriptsubscript^𝖼𝖠𝖳𝖤ORoc\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}over^ start_ARG sansserif_cATE end_ARG start_POSTSUBSCRIPT OR end_POSTSUBSCRIPT start_POSTSUPERSCRIPT oc end_POSTSUPERSCRIPT

𝖼𝖠𝖳𝖤^ORallsuperscriptsubscript^𝖼𝖠𝖳𝖤ORall\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}over^ start_ARG sansserif_cATE end_ARG start_POSTSUBSCRIPT OR end_POSTSUBSCRIPT start_POSTSUPERSCRIPT all end_POSTSUPERSCRIPT

Definition 1 (Conditional and marginal average treatment effect of treatment arm $k$ compared to shared control among concurrent population).

Theorem 1 (Identification of $\mathsf{cATE}$ in platform adaptive trials under Model (1)).

Definition 2 (Conditional and marginal average treatment effect of treatment arm $k$ compared to shared control).

Theorem 2 (Identification of $\mathsf{ATE}(k)$ in platform adaptive trials under Model (1)).

5 Estimation of $\mathsf{cATE}(k)$

Definition 3 (Conditional and marginal average treatment effect of treatment arm $k$ compared to control among concurrent population).

Definition 4 (Conditional and marginal average treatment effect of treatment arm $k$ compared to control).

$\hat{\mathsf{cATE}}_{\text{OR}}^{\text{oc}}$

$\hat{\mathsf{cATE}}_{\text{OR}}^{\text{all}}$

$\hat{\mathsf{cATE}}_{\text{IPW}}^{\text{oc}}$