Causal Inference in the Closed-Loop: Marginal Structural Models for Sequential Excursion Effects

We thank the reviewer for their positive feedback on our work. The reviewer raises a very interesting methodological/modeling question, and identifies a notational issue, both of which we address below.

1. “The largest weakness I see here is both experimental results uses linear or generalized linear model for $m$. I see the advantages of well-developed generalized linear model in experimental sciences for good interpretation and statistical guarantees, though would be curious to find out how the authors see the method would work beyond linear models.”

• The projection parameter specification, as laid out in Section 3, is entirely generic, and remains well-specified even for more complicated models $m$ (e.g., random forests, kernel-based methods, neural networks). However, as you rightly point out, we adopt linear or generalized linear models (GLMs) as our working models in both simulations and the applied study. For the questions of scientific interest in the applied optogenetics study, we felt that the target causal effects (e.g., dose response effects and effect dissipation, pooled across time) could be well approximated using GLMs.

• The theory applies for a large class of working mean models $m$, and does not rely on any distributional/likelihood-based assumptions. Beyond their familiarity to most readers, the technical reason for opting for GLMs in our applied examples is—as you again correctly identify—for statistical guarantees (e.g., convergence properties and inferential tools). In particular, Theorem 3.5 stipulates that $m$ must be “Donsker” in $\beta$: GLMs automatically satisfy this technical requirement (see, e.g., Examples 19.7 and 19.8 in van der Vaart (2000)). Whether a model class is “Donsker” is a non-trivial query regarding its “complexity” or “size”, but can be investigated using machinery and results from empirical process theory (van der Vaart & Wellner (1996)). The situation is indeed more complicated for more flexible models, and this Donsker requirement is not always satisfied. However, we note that it is satisfied for some formulations of random forests (see, e.g., Theorem 3.2 in Scornet (2016)) and kernel estimators (see, e.g., Theorems 3.7 and 3.13 in Beutner & Zähle (2023)). In our revised Discussion section, we will discuss this issue in more detail, and note that future research will focus on applying random forests and kernel estimators (when they satisfy the Donsker requirement), and the more challenging problem of studying the statistical properties of flexible model specifications when the Donsker requirement is not satisfied.

2. “Proposition 3.4 $Z_i$ is introduced but the proposition only uses $Z$ throughout. would be good to specify the relationship between $Z_i$ and $Z$.”

• We thank the reviewer for noticing this notational issue. As we mention in the beginning of Section 2, we adopt a common statistical shorthand in that “we often suppress [subject-specific] indices to reduce notational burden”. In this case, $Z$ represents a generic observation (i.e., from an arbitrary subject), whereas $Z_i$ represents that same data, but specifically from subject $i$. In the revision, we will specifically reiterate in Proposition 3.4 that we omit the subject-specific index for clarity.

References:

• van der Vaart AW, Wellner JA. Weak convergence. Springer New York; 1996.
• van der Vaart AW. Asymptotic statistics. Cambridge University Press; 2000 Jun 19.
• Scornet E. On the asymptotics of random forests. Journal of Multivariate Analysis. 2016 Apr 1;146:72-83.
• Beutner E, Zähle H. Donsker results for the empirical process indexed by functions of locally bounded variation and applications to the smoothed empirical process. Bernoulli. 2023 Feb;29(1):205-28.

We thank the reviewer for their helpful questions, comments and feedback on our work. We believe that the proposed revisions and clarifications in line with the responses below will improve the strength of the paper.

1. “The focus of this paper centers on an application study that refines causal effect estimation within a specialized causal graph, akin to Figure 1E, rather than introducing broad methodological advancements in causal inference.”

• We used the causal graph in Figure 1E to provide an illustration, which may have suggested limited applicability. To clarify, the proposed methods are applicable in all sequentially randomized settings, going far beyond the optogenetics studies that motivated our work. In particular, treatment at each time point may depend on the complete history of observed variables, $H_{t}$ (as opposed to just $X_t$ as in the illustrative causal graph). In the revision, we will emphasize that this is an illustrative causal graph, but elaborate on the generality of the methodology to causal effect estimation within a far more general set of causal graphs. The methodological advancement is the introduction of specific estimands that are valid in the presence of positivity violations, paired with powerful statistical approaches. Such positivity violations occur regularly across many application areas beyond neuroscience, for instance in mobile health studies where active treatment may be systematically withheld for ethical reasons (e.g., see discussion of treatment “availability” in Section 4 in Boruvka et al. (2018; JASA)).

2. “[...] This setup closely relates to estimating causal effects in temporal sequences. It is unclear what is the difference/gap between the proposed framework and the existing temporal causal inference methods. I wonder if the authors could discuss this in more detail. [...]”

• Please see global “Author Rebuttal” for a detailed response: the major challenges in this case are the large number of timepoints (for which traditional MSMs are unstable) and the inherent positivity violations in the design (where no existing methods can handle treatment sequences longer than length one). Further, existing machine learning-based causal methodology (RMSNs, CRNs, CTs) rely on positivity, and do not provide statistical guarantees or inferential tools (e.g., confidence intervals).

3. “I think it would be better if the authors explicitly introduce the problem settings in Section 2 rather than mention it together with relevant literature…”

• We thank the reviewer for this suggestion and agree with their point. In the revision, we will split Section 2 into two sections. The first will be a less technical summary of existing methods, highlighting the gaps. The second will then explicitly introduce the problem and notation.

4. “The authors should consider adding more baselines in experiments. In this paper, the authors did not use any baselines, and thus it is unclear whether the proposed framework outperforms existing methods.”

• Please see global “Author Rebuttal” for a detailed response. Briefly, our paper introduces new target causal estimands. Consequently, no existing estimators could be used to compare against our approach. That said, we will add a simulation in the appendix (see Figure 1 in attached 1-page PDF) to show how the proposed methods uncover effects when standard “macro” approaches yield null results (see Point 6 below).

5. “The settings and evaluation metrics of the experiments need to be revised.”

• We thank the reviewer for their suggestion; we believe this criticism reflects our choice to present our findings graphically as opposed to numerically (e.g., in tables). In the revision, we will provide the numerical bias and coverage results in a table, and display mean-squared error (MSE) of the proposed estimator across the simulation settings (see Table 1 in attached 1-page PDF).

6. “[...] so only estimating macro effects can be biased and inaccurate. However, the authors did not conduct any simulation studies to demonstrate the advantage of the proposed method. ”

• In Appendix 2, we show mathematically that macro effects may be misleading. Specifically, we demonstrate that, even if the magnitude of local causal effects are large, the macro effects can be exactly zero. In the revision, we will include a simulation that demonstrates this more concretely: macro approaches will show null results, whereas the proposed methods will uncover more nuanced non-null effects (see Figure 1 in attached 1-page PDF for preliminary results).

7. “The current presentation of experimental results primarily relies on descriptive analysis, and I think the authors should use some quantitative metrics to evaluate the performance.”

• See response to Point 5 above: we will provide explicit numerical tables (in addition to our figures) to demonstrate bias, MSE, and coverage properties (see Tables 1-3 in attached 1-page PDF).

8. “I wonder how is $\Delta$ determined and How would the value of $\Delta$ affects the sequential excursion effects?”

• The number of intervention timepoints in the counterfactual outcome of interest ($\Delta$) determines the nature of the effects being estimated. However, 2-3 intervention timepoints are often sufficient to capture a rich collection of sequential excursion effects. We found that, in our application, conclusions were stable across a range of $\Delta$ values (e.g., in Section 4.2, we mention that “[w]e fit comparable models for sequences as long as $\Delta=7$ and found results were similar across $\Delta$ values.”). In practice, the value of $\Delta$ may be constrained, as the variance of parameter estimates can grow if $\Delta$ is set too large (likely for the same reason that the parameters of non-history-restricted MSMs grows prohibitively large as the number of timepoints increases). In our revision, we will emphasize that $\Delta$ should be chosen on the basis of subject matter knowledge.

Thank you for the detailed response and the additional numerical experimental results. I have raised my score by 1.

We thank the reviewer for their feedback and questions. Most of the major concerns are, we believe, a result of misunderstandings of the goals and claims of our work. We hope that our revisions will make them clearer, and address the questions below:

1. "Some claims are not clear, which makes the paper unreadable, especially for those who are not familiar on biologic and optogenetics."

• The reviewers unanimously felt that a strength of our work is the importance of the methods for applications in neuroscience, thus we feel we should maintain some domain-specific language. We make a significant effort to provide the necessary background to motivate our methods. Moreover, the problem setup and counterfactual notation we use follow the conventions of the sequentially randomized experiments causal inference literature (e.g., MSMs, HR-MSMs) as well as much of the causal machine learning research (e.g., RMSNs, CRNs, CTs). Thus, no domain-specific knowledge should be necessary to understand the key methodological and theoretical development (Sections 2, 3, 4.1). Our contributions to the field of counterfactual-based causal inference are general and extend well-established statistical theory.

2. "I am kind of confused for the definition and notation of treatment, what's the difference of G and A? I thought they are the same but the description in intro confuses me. optogenetic manipulation is not about the laser and no-laser."

• We are sympathetic that the biology may be unfamiliar to some readers and, in our revision, we will further emphasize the distinction between group ($G$) and laser ($A$). The group, $G$, and laser stimulation, $A$, are indeed very different. Optogenetics studies often include randomly assigned groups of “light-sensitive” animals ($G = 1$), and light-insensitive negative controls ($G = 0$). Animals in the $G = 1$ group express a protein that causes their neurons to be activated in response to laser stimulation, whereas negative control animals ($G = 0$) do not express this protein, and thus do not exhibit this response to the laser. Animals in both groups can receive the laser treatment ($A_t = 1$), or not ($A_t = 0$), on each trial, $t$, but the application of the laser will not influence the neuronal activity in the control group ($G=0$) . This is a fundamental distinction, which we make a significant effort to explain in the paper. Indeed, on line 32 we state: “Experiments often include both treatment ($G=1$) and negative-control ($G=0$) groups, with animals assigned randomly to each. While the laser is often applied on a random subset of trials in both groups, only treatment animals express the protein that enables the laser to trigger the target neural response. The control group thus controls for “off-target” effects such as the laser heating the brain…”

3. "The experimental evaluation is weak. Indeed there are only one baseline method, i.e., HC to compare with the proposed estimator,"

• See global “Author Rebuttal for a detailed response. A primary contribution of our work is the introduction of new target estimands (i.e., they allow us to ask new causal questions that previous methods cannot): there are no existing estimators against which we can compare our proposed methods.

• We believe that the reviewer’s description of our empirical evaluation as “weak” may be a result of misunderstanding. For example, “HC” is not the “only one baseline method.” Rather “HC” is a family of variance estimators that can be used to estimate uncertainty for our point estimators, i.e., they can be used as components of our proposed framework. We describe in the Results Section 4.1 that: “the sample size-adjusted HC3-based CIs achieve 95% coverage. All CIs achieve 95% coverage when n is large, showing that we can conduct valid inference in all settings.” Moreover, in the Figure 2 caption, where the “HC” results are presented, we describe that the results show “[t]he coverage of 95% CIs constructed using one of three established robust variance estimators and our robust variance estimator. The nominal coverage is reached for either large $n$ or large $T$ for all estimators.”

• As we emphasize in the global response, we evaluated our methods across a wide range of simulation settings (see Tables 1-3 in the attached 1-page PDF with updated numerical results), and a very thorough application for which the results were confirmed and commended by the original neuroscientists who authored the Nature paper.

4. "There are many sequential models (e.g., Causal Transformer, CRN, RMSN ,etc) which can handle this kind of time-dependent causal inference task."

• As we detail in the global response, we omitted these methods because they are not applicable in our setting. In our revision, we will expand our literature review to include them, and explain the reasons why they are not suitable for this problem:

“While RMSNs (Lim et al., 2018), CRNs (Bica et al., 2020) and CTs (Melnychuk et al., 2022) can be used to estimate causal effects over time, these methods have important limitations: (a) these methods all target the effects of static (i.e., fixed) treatment sequences and depend heavily a positivity assumption, and thus cannot be applied in closed-loop designs; (b) the target estimands for these methods are always conditional on complete history of covariates/outcomes before treatment initiation, $H_{t - \Delta + 1}$, whereas our estimands may depend on all history variables if desired, but may be averaged over some or all of these variables (often yielding greater statistical power) depending on the user’s preference; and (c) unlike for our methods, there are no rigorous statistical guarantees (e.g., consistency, asymptotic normality) that have been proven for RMSNs, CRNs or CTs, and correspondingly, no inferential tools (e.g., confidence intervals) developed for uncertainty quantification.”