An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation

We thank reviewer for their constructive feedback and comments. We have addressed their concerns and questions below.

W1: Independent Causal Mechanism (ICM) Principle

We find the reviewer’s comment about relation of our model to the ICM principle by Peters et al. [1] quite interesting, especially since cyclic SEMs have been around for quite a bit longer than the ICM principle itself, with roots in econometrics from as early as 1940s.[2] Despite this, we are not aware of any explicit reference to the principle in modern literature on cyclic SEMs, including in works by Peters et al.[1][2][3] Nevertheless, we can observe the following.

Similarity — Structural equations as ICMs: Our SEM formulation in Eq. (1) is a direct instantiation of the ICM principle as described by Peters et al.[1] The two equations represent the autonomous mechanisms, and their independence is captured by the mutual independence of the exogenous noise terms $N_X, N_Y$. The simultaneity in our model is not a violation of ICM, but rather the equilibrium state resulting from the interaction of these two independent mechanisms. Assuming the existence of this equilibrium is a statement about the scope of systems under analysis, and not about the nature of the mechanisms themselves. Indeed, surgically changing $\tau$ to some $\tau’$, for example, does not in itself alter $f$ and vice versa. And precisely because of ICM this may or may not make the system unstable depending on the nature of $\tau’$, but in our setting Theorem 4 (Appendix G.7) shows that soft interventions induced by outcome-invariant DA are always stable.

Difference — Markov factors as ICMs: We speculate that this is the main motivation for reviewer’s comment about ICMs. Indeed, in most settings, ICMs are discussed in terms of more familiar Markov factorization of joint distribution of the system since such a factorization always exists in DAGs. However, such a factorization generally does not exist for cyclic models, which is precisely why they are almost exclusively represented as simultaneous structural equations.

We thank the reviewer for this very insightful and thought-provoking question, and agree that this connection to ICMs should be made explicit in the manuscript. To address this in the preliminaries Section 2 we will make clear that our model's structural equations are a direct instantiation of the ICM principle and point to a more detailed commentary in Appendix B as an appropriate place to elaborate nuances since it is exclusively dedicated to discuss cyclic SEMs.

Success and failure modes w.r.t. ICM

Given the above clarification about structural equations being a more appropriate choice for representing ICMs in cyclic models, we believe our manuscript already addresses the reviewer’s wishes of success/failure modes with respect to ICMs $\tau$, $f$, etc. We provide a recap (and an additional comment) for completeness.

The task is to improve estimation of $f$ which corresponds to the ICM of the outcome $Y$. And we do this by changing/ intervening on the ICM $\tau$ of the treatment $X$ while keeping all other ICMs in the system unchanged. This is done via DA, and it is successful/unsuccessful in the following cases.

1. Outcome-invariant DA: Observation 1 shows that outcome-invariant DA is equivalent to surgically changing the treatment ICM $\tau$ to $G\tau$ while keeping all other ICMs in the system unaffected. As such, outcome-invariance complies with the ICM principle.
   1. Success — perturbing spurious features: Theorems 2 shows that this reduces confounding bias when the DA aligns with spurious features.
   2. Benign failure — no spurious features perturbed: Theorems 2 also shows that this does not reduce confounding bias when the DA is orthogonal to spurious features. Fortunately however, this does not perform worse than ERM, and so the failure is rather benign compared to the following case.
2. Malignant failure(?) — outcome-invariance violation: Although outside the scope of our current work, we conjecture that a violation of outcome-invariance is equivalent to a violation of the ICM principle itself. Since outcome-invariance is necessary for DA equivalence to IVL, a violation would mean that the DA not only affects $X$ but also $Y$ so that it is now a confounder. We believe this to be true because violations of outcome invariance in DA has been shown to bias estimates in works that don’t necessarily study causality [6]. Although this raises many interesting questions in itself, we defer a further formal investigation to future works in order to keep the current submission focused.

Scope of applicability

Given the above classification, it is natural, as the reviewer points out, to ask whether the conditions required for our method’s success are reasonable. In the next section, we argue that they are—particularly when viewed through the appropriate lens of instrumental variables (IVs).

W3 & Limitations: On Untestable Assumptions

We agree with the reviewer that the untestability of assumptions is a central concern, and appreciate the opportunity to clarify this point more directly in the main text.

1. IVs Are the Right Benchmark

We believe IVs are the appropriate lens for judging whether our assumptions are "too strong." Our IV-like variables (IVLs) are a direct generalization of IVs—designed to be more available by relaxing the outcome relevance condition—while still aiming to reduce confounding. This generalization naturally inherits the core limitations of IVs: assumptions like exclusion restriction and unconfoundedness are not testable from observational data and must be justified via domain knowledge.

Similarly, outcome-invariant data augmentation (DA), which we show is equivalent to IVL by construction (Observation 2), carries the same interpretability and limitations. In this light, we argue it is inappropriate to expect our method to be free from the very limitations of the framework it generalizes.

$\overbrace{ \underbrace{\text{outcome-invariance}}\_{\text{popular model for DA}} + \underbrace{\text{spurious targets} }\_{\text{benign failure if violated}} }^{\text{un-testable DA assumptions}} \iff \overbrace{\text{un-confoundedness}+\text{exclusion}}^{\text{un-testable IV/IVL assumptions}}$

2. Clarifying Assumptions

• Outcome Invariance: The central assumption is outcome-invariant DA—analogous to exclusion restriction and un-confoundedness in IVs. While untestable, this is a common and accepted modeling choice in the DA and invariance literature [6–8].
• Perturbing Spurious Features: This condition is not a new limitation but a direct analogue of the exclusion principle—an IV must enter $Y$ via a spurious feature of $X$ to identify the corresponding confounded component of $f$.
• Joint Diagonalizability: The reviewer raises a valid concern about Assumptions 1 and 2 behind our theoretical results. These are only required for our use of the Relative Squared Error (RSE) metric, which—as argued by Janzing [9]—is more relevant for causal estimation. Our core results on confounding bias reduction still hold under the standard risk-based estimation error $\mathbb{E}^{\mathcal{M}}[\ell(f(X), \hat{h}(X))]$, without requiring Assumptions 1 and 2.

Acknowledging that these clarifications could be better emphasized, we will move and expand the limitations discussion from the Appendix into the main paper (as Section 7), directly addressing the concerns around untestable assumptions and theoretical scope.

W2: On Empirical Improvements

We acknowledge your observation that on Colored-MNIST, the additional benefit of DA+IVL over DA+ERM is modest (although, arguably still significant). We would like to offer two points of context:

1. DA+ERM's performance is a key result: A central contribution of our paper is to provide a causal interpretation for why outcome-invariant DA works: it acts as a soft intervention. The fact that the simple DA+ERM baseline is already so effective is strong empirical validation of this core thesis. The performance of DA+ERM should therefore be seen as a strength of our framework, not a weakness of the DA+IVL extension.
2. Contextualizing the DA+IVL improvement: While the additional gain from DA+IVL is modest on this specific task, it is still competitive with or better than several established OOD generalization methods (as seen in Figure 5). Furthermore, on the Optical Device dataset, our proposed DA+IVLcc is the only method that provides a significant performance gain over both the ERM and DA+ERM baselines, demonstrating its value in other settings.

We hope these clarifications have addressed your concerns. We thank you again for your insightful review, which helped us identify key areas for improvement in the presentation of our work. We are confident that by incorporating these points, we can make the paper's contributions and limitations clearer to the reader.

[1]: J. Peters et al. Elements of Causal Inference. MIT Press, 2017.

[2]: C. F. Christ. “The Cowles Commission's Contributions to Econometrics at Chicago, 1939–1955.” J. Econ. Lit., 1994.

[3]: S. Bongers et al. “Foundations of Structural Causal Models with Cycles and Latent Variables.” Ann. Statist., 2021.

[4]: J. M. Mooij et al. “On Causal Discovery with Cyclic Additive Noise Models.” NeurIPS, 2011.

[5]: D. Rothenhäusler et al. “BACKSHIFT: Learning Causal Cyclic Graphs from Unknown Shift Interventions.” NeurIPS, 2015.

[6]: S. Chen et al. A group-theoretic framework for data augmentation. JMLR, 2020.

[7]: C. Lyle et al. On the benefits of invariance in neural networks. arXiv, 2020.

[8]: H. Shao et al. A theory of PAC learnability under transformation invariances. NeurIPS, 2022.

[9]: D. Janzing and B. Schölkopf. Detecting confounding in multivariate linear models via spectral analysis. J. Causal Inference, 2018.

We thank Reviewer MsXV for their constructive feedback. The reviewer raises crucial points, including about the value of our work in a non-identifiable setting, which we are happy to clarify.

Response to Limited Applicability

Our work's significance is best understood through the following lens:

1. Practical Value: A Safe, "No-Regret" Improvement: Our primary response is grounded in Theorem 3: under the standard DA assumption of outcome-invariance, our approach is guaranteed to perform no worse than standard ERM. This provides a "no-regret" choice for practitioners. They gain a potential reduction in bias if the data augmentation targets spurious features, but risk nothing compared to the ERM baseline.
2. Broader Significance (1): Addressing the Upstream Problem for Robustness: Reducing confounding bias, even in non-identifiable settings, is not a niche goal but the upstream problem for OOD generalization and robustness [4]. A model that fails on shifted distributions often does so because it learned spurious correlations (i.e., confounding). Our work, in line with the goals of causal regularization [10-12], tackles this root cause directly. By formalizing why DA can help (as a soft intervention) and when (by perturbing spurious features), we provide a principled path toward building inherently more robust models.
3. Broader Significance (2): Downstream Implications for:
   1. Partial Identifiability: Lower confounding bias in the data (e.g., via DA) can result in tighter bounds in partial identification [9].
   2. Sensitivity Analysis: Lower confounding bias means reduced sensitivity to confounding, leading to more reliable conclusions from subsequent sensitivity analyses.

The value is in repurposing the already existing, cheap tool of DA to provide a safe and causally-grounded improvement over the ERM baseline, with direct positive implications for the above downstream tasks. We will clarify and explicitly mention applicability in section 1 of our draft.

Theoretical Results, Assumptions, and Metrics

The reviewer's concerns about the theoretical results, assumptions, and the RSE metric are interconnected, and we address them together.

• RSE is a direct ATE Error Metric. There seems to be a slight misunderstanding here. The Relative Squared Error (RSE) is not a surrogate for the average treatment effect (ATE) error; it is itself a valid ATE estimation error [33,10]. Furthermore, Janzing argues that its structural nature makes RSE more relevant for causal statements than the more familiar risk based estimation error (EE) $\mathbb{E}^{\mathcal{M}}[ \ell( f(X), \hat{h}(X) ) ]$ [33,12], which also motivates our choice. The (un-normalized) RSE is simply $\|\hat{h}-f\|^2$, which directly measures the squared error of the estimate $\hat{h}$ from the true causal function $f$ (which equals the ATE in our model). The normalization is only done to make the metric more interpretable and meaningful for our experiments. We used the term "surrogate for confounding bias" because any choice of estimation error also contains statistical bias, but we see now how this was unclear. We will revise the phrasing to "causal estimation error" in the paper to avoid this confusion.
• Strong Assumptions are for the Stricter Metric. Our strongest assumptions (Assumptions 1 & 2, regarding joint diagonalizability) are a direct consequence of using the stricter, structural RSE metric. We would like to emphasize that our core results on bias reduction also hold for the standard risk-based EE, without requiring Assumptions 1 and 2. We will make this clarification about choice of metric vs. generality of results explicit in the revised manuscript. We believe this directly addresses the concerns about the limited insightfulness of our results and choice of estimation error. Should the reviewer still find it helpful, we are also prepared to add experimental results using EE to the appendix , as our core claims hold for that metric as well.
• On the Practicality of Assumptions. To avoid redundancy, we provide a more detailed discussion on limitation of assumptions in our response to Reviewer CVxW. Please refer to the same.

Empirical Evaluation

• Weakness: Benefits on real-world data are questionable. We respectfully disagree with this characterization. Our results in Fig. 5 show a consistent and often significant improvement.
  • On the Optical Device dataset, our proposed DA+IVL\textsubscript{CC} is the only method that meaningfully outperforms both the ERM and DA+ERM baselines.
  • The reviewer mentions that DA+IVL improvement over DA+ERM is modest (though, we argue, still significant) in Colored-MNIST. We point out that this is still competitive with or better than established OOD generalization benchmarks. Also, we urge the reviewer that DA+IVL should not distract from giving credit on the undeniable performance of DA+ERM baseline itself, as DA as soft intervention is also a central contribution of our framework.
• Weakness: Linearity assumption in the optical device dataset. This is a fair point. We followed the precedent set by [33] who introduced this dataset and also analyzed it under a linearity assumption [10]. While the true mechanism may or may not be linear, this setup allows for a controlled experiment and direct comparison with prior work.

Addressing Specific Questions

1. Comparison with causal regularization methods.

   [12] involves a purely theoretical analysis for using \ell_2 to improve causal estimation in purely observational (X, Y) setting.

   [10] considers also considers purely observational (X, Y) but under special, linear modelling assumptions so that now estimating | f | is possible, even if f isn’t. This information is then used to develop a cross-validation scheme.

   In [11] authors consider data from 2 observational (X, Y) sources to develop a croos-validation scheme for \ell_2 regularization based on stability of regression parameters across the 2 datasets.

   Regarding incorporating these into our experiments, we thank the reviewer for highlighting this area of improvement. Due to time limitations, however, we couldn't include additional experimental results in this rebuttal. Nevetheless, we do plan to complete and incorporate these into the camera-ready revision.

2. Significance and meaning of $\alpha$ in $\mathcal{I}\_{\alpha}$ — the degree of OOD robustness. In Theorem 1, the set $\mathcal{I}\_{\alpha}\subseteq\operatorname{span}(\mathbf{K}^\top)$ represents the interventions over which the IVL estimate has robust, or optimal ‘worst-case’ predictive performance. The parameter $\alpha$ can be interpreted as a ‘knob’ that determines how OOD the interventions in this set would be. As $\alpha\rightarrow 0$, then $\lim_{\alpha\rightarrow 0}\mathcal{I}\_\alpha = \operatorname{span}(\mathbf{K}^\top)$ such that now our estimates would have robust predictions over arbitrarily large interventions of the form $\operatorname{do}(\tilde{Z}\coloneqq\xi), \;\xi\in\operatorname{span}(\mathbf{K}^\top)$. So, essentially, $\alpha$ determines the trade-off IVL makes between in-distribution risk (i.e., ERM) vs. OOD treatment-intervention risk (i.e., IV regression).

3. On justification and notation of Observation 1

   Justification of Observation 1. Theorem 4 (Appendix G.7) shows that for SEM $\mathcal{A}$ in Eq. (7),

   $X^{\mathcal{A};\operatorname{do}(\tau\coloneqq G\tau)} = GX^{\mathcal{A}}, \qquad Y^{\mathcal{A};\operatorname{do}(\tau\coloneqq G\tau)} = Y^{\mathcal{A}}.\qquad(\dagger)$

   So it immediately follows that $P\_{X, Y, G, C}^{\mathcal{A};\operatorname{do}(\tau\coloneqq G\tau)} = P\_{GX, Y, G, C}^{\mathcal{A}}$ given that $G$ and $C$ are exogenous.

   Intuition for Observation 1. The intuition behind Eq. ($\dagger$) and, by extension, Observation 1 is that the transformation $GX$ does not change $Y$ by definition of outcome-invariance, despite cycles.

   Clarification of notation for $G\tau$. We

   • Definition: $G\tau$ is simply the composition $G\circ\tau$. Given that we have

   $\tau: \mathcal{Y}\times\mathcal{C}\times\mathcal{N}\_X\rightarrow \mathcal{X}, \qquad \delta:\mathcal{X}\times\mathcal{G}\rightarrow \mathcal{X}$,

   then using $\mathbf{g}\mathbf{x}\coloneqq \delta(\mathbf{x}, \mathbf{g})$ as shorthand for group action, $G\tau$ is simply

   $G\tau(\cdot) = G\circ\tau(\cdot) = \delta( \tau(\cdot), G ).$

   • Form in Example 2: For Example 2, $\tau$ has the block-matrix form

   $\tau(\cdot) = \mathbf{T}(\cdot) ,\qquad \mathbf{T}\coloneqq[\mathbf{0}\_{m} \quad \mathbf{E}^\top \quad\sigma\mathbf{I}\_{m\times m}]$

   so that now for any tuple $(Y, C, N\_X)$ we have

   $\tau(Y, C, N\_X) = \mathbf{T} \begin{bmatrix} Y \\ C \\ N\_X \end{bmatrix}$.

   Finally, given the definition $GX\coloneqq X + \gamma\mathbf{K}^\top G$, we have

   $G\tau(\cdot) = G\circ\tau(\cdot) = \mathbf{T}(\cdot) + \gamma\mathbf{K}^\top G.$

Response to Limitations

We fully agree with the reviewer and will explicitly emphasize this context in the camera-ready version. Our goal is not to claim full identification, but to provide a principled step forward from ERM, but the residual bias still necessitates caution. Crucially, we will frame this as the reality of the problem setting, where our method still offers a robust and provable improvement over standard ERM.

We appreciate the reviewer’s feedback and hope our response addresses their concerns. We are confident the clarifications will strengthen the manuscript and better highlight its practical relevance. We also note that the reviewer’s comments and criticisms focus primarily on identifiability, whereas others reviewed our work from a broader perspective.

References: See manuscript.

Thank you very much for your positive feedback and for increasing your score. We really appreciate your detailed and constructive suggestions.

We agree with all of your recommendations, and many of them seem to overlap with other reviewers’ suggestions as well. We think they are good additions to the manuscript and as such, we will incorporate them into the camera-ready version as follows:

1. Broader Significance: We will explicitly discuss broader significance in Section 1, expanding on the points mentioned in our rebuttal.
2. Assumptions and Metrics: When introducing RSE in Section 3, we will explicitly contrast it with EE to better motivate our choice. Following this, we will mention that Assumptions 1 and 2 are required for RSE, but not for corresponding results with EE. This will ensure readers understand the generality of our findings and also give us the opportunity to discuss (in Appendix A) the nuanced difference between these two metrics that Janzing points out, which we believe is actually quite interesting!
3. Non-linear Experiments: Experiments with non-linear models for the Optical Device dataset would definitely be insightful. We will try a few simple approaches (polynomial features, non-linear kernel SVMs, shallow NNs, etc) and report the results in Appendix F, with a summary in the main text to highlight their implications.
4. Causal Regularization Baselines: As promised, we will work to include causal regularization baselines in our experiments. These will make our suite of baselines more complete and meaningful.
5. Limitations Section: Absolutely, this seems to be the most pressing request by all reviewers — the Limitation section will be moved from Appendix E to the main paper and expanded to include (i) contextualisation of assumptions, (ii) explicitly caution about residual error still being arbitrarily large.
6. Notation/Nomenclature in Appendix: We completely understand the frustration with notation. Unfortunately, this is inevitable since we have to juggle multiple SEMs, their intervention forms, examples and overloading to bridge the topics of causality and DA. A notation/nomenclature section would certainly help! We will definitely include it.

These suggestions were quite helpful for polishing the draft, and we once again express our gratitude for your time and feedback.

Thanks!

We sincerely thank the reviewer for their positive assessment and insightful, constructive feedback. We are encouraged that they found our approach novel, and the core ideas sound and valuable. The reviewer's questions and suggestions for improvement are appreciated and will help us improve the clarity and impact of our work.

We have structured our response to first address the main weaknesses identified (intuition, comparisons, and practical assumptions), followed by answers to the specific clarification questions.

On Weaknesses and General Improvements

1. Intuition for Bias Mitigation and Additive Noise Restriction (Weakness 3 & Additive Noise Restriction Question)

Thank you for the suggestion. We agree that a more direct intuitive explanation would be helpful.

The core intuition behind why the outcome-invariant data augmentation (DA) induced soft intervention helps reduce confounding bias is that it injects new randomness into the treatment $X$ that is, by construction, independent of the hidden confounder $C$. This process effectively "dilutes" the spurious correlation between $X$ and $Y$ that is mediated by $C$. By training on this augmented data, the model is discouraged from relying on the spurious path and is guided toward learning the true causal relationship, thereby mitigating confounding bias.

Another, albeit more `technical' intuition is to look at it from the perspective of the data processing inequality $I(GX;C)\leq I(X;C)$ applied to the Markov chain $C\rightarrow X\rightarrow GX$, so that $GX$ now carries less information about the confounder $C$ than the original treatment $X$, unless $GX$ preserves information about $C$ (this is precisely the condition of perturbing spurious features in our linear settings). This perspective also portrays that our method is not merely limited to additive noise, and applies to more general transformations $G$.

We will add a similar explanation at the end of Section 4.1 to better frame our theoretical result and also emphasize that it is more general than our simple additive noise setting would suggest.

2. Availability of Symmetries and Discussion of Limitations (Weakness 1 & Limitations)

We thank the reviewer for this crucial point. We agree that assuming known structural symmetries is a strong requirement and they are not universally available. Our work builds on a key insight inspired by this very challenge:

The widespread use of an outcome-invariant model for data augmentation in the literature seems to suggest that in many application domains, such as vision, natural language processing, etc., practitioners already possess and leverage this type of symmetry knowledge, even if implicitly.

And the motivation behind our work is to show that these existing, domain-justified symmetries can be repurposed as a principled tool to mitigate confounding bias.

Therefore, our framework is most valuable in application domains where such data augmentations are already established. More generally, and as with any non-experimental causal method like Instrumental Variables, justifying these symmetries ultimately relies on domain knowledge. This is a fundamental characteristic of the problem space, rather than a unique limitation of our approach.

We discuss this and other limitations of our method in Appendix E.4, but will be moving that section into the main paper with more explicit discussions of the same for better transparency.

We also make some additional comments on symmetry availability under Reviewer CVxW's thread that you may find interesting -- see under "Additional Reflections on untestable validity of IV vs. outcome-invariant DA".

3. Clarity of Comparison to Other Methods (Weakness 2)

We appreciate the reviewer's call for more transparency here. Our rationale for using DA parameters ($G$) as inputs for baseline domain generalization methods (e.g., V-REx, DRO, IRM, etc.) is to ensure a fair comparison. The fundamental challenge we address is reducing confounding bias given only observational data $(X, Y)$ and knowledge of some symmetries via valid augmentations. For a fair comparison, all methods must be restricted to this same information. Therefore, we use the DA parameter $G$ as a proxy for the "environments" or "domains" that these other methods typically require as input. The reason this is still a conceptually valid comparison is because by virtue of being domain generalizaiton methods, they are essentially solving a robust loss of a similar form as in Corollary 1. This gives us direct and correct controls to compare our DA+IVL method against.

We will explicitly state this rationale in Section 6 to make the experimental design clearer.

Responses to Specific Questions

• On subsuming cyclic relations under "confounding": We appreciate the nuance. Our use of “confounding bias” is the same as Pearl’s definition of the term [5] $\text{Confounding Bias}\coloneqq \mathbb{E}[Y\mid X] - \mathbb{E}[Y\mid\operatorname{do}(X)]$ . Which is to say, any bias that causes the observational to deviate from the interventional. In our context, this would also include simultaneity. Such a general framing also follows the tradition in econometrics [14, 39] where omitted-variables (i.e. confounders) are lumped together with simultaneity as sources of endogeneity (statistical dependence between the treatment $X$ and the residual $Y - f(X)$). We will add a footnote in Section 2.1 to clarify our use of the term.
• Clarification of line 141: This sentence describes the two parts of our IVL risk. The IV risk term ensures the solution lies in a set that contains the true causal function $f$ (i.e., where "$f$ is present") (assuming $\alpha=0$). Since this set may not be a single point, the ERM risk term acts as a regularizer, selecting a solution from this set that also has good predictive performance. We will rephrase lines 140-142 for better clarity.
• Clarification of line 280: "Between empirical versions" was imprecise phrasing. It refers to the transformed variables $X'$ and $Y'$ defined just below in the text. We perform a closed-form OLS regression of the empirical transform $Y'$ on $X'$. We will revise this to be more precise.
• Clarification of line 335 ("regression simulates an intervention"): By simulation we essentially mean is equivalent to:
  1. Applying DA is equivalent to a soft intervention on the treatment mechanism (Observation 1).
  2. And DA+IVL regression is equivalent to a worst-case DA from a given set (Corollary 1). In this sense, the DA+IVL regression procedure finds a solution for a worst-case intervention scenario encoded by the DA. We will clarify this two-step logic in Section 4.2.
• Confusing sentence in the Abstract: Thank you for highlighting this. We will revise the sentence for clarity. A proposed revision is: "We introduce the concept of IV-like (IVL) regression, which leverages sources of treatment randomization even when they are irrelevant to the outcome. We show that this approach can still improve predictive performance across interventions and reduce confounding bias."
• Arrow in Table 1: This is a good point to clarify. The arrow indicates that moving down the column leads to methods that yield lower confounding bias. "Worst-case or adversarial DA" refers to a method that optimizes for a solution robust to the worst possible augmentation. In our setting, this robustness forces the model to ignore spurious correlations, thereby yielding a better causal estimate with less bias. We will add a note to the table's caption to make this interpretation explicit.

Thank you once again for your thorough and valuable review. We are confident that by incorporating these changes, we can significantly improve the paper.

References:

Please see the manuscript.

We’re grateful for your positive review and happy that you liked the IV vs. IVL spin. Also, thank you for pointing out parts that could use more clarification.

We would like to extend our sincerest thanks to the reviewer for their thoughtful and encouraging feedback. We are particularly pleased that they found the core idea original and compelling, and appreciated the connection we draw between data augmentation and soft interventions. The reviewer's summary of our work's strengths and weaknesses is fair and very helpful. We will address the identified weaknesses and questions below.

On the Practicality of Finding Suitable Data Augmentations and the Untestability of Assumptions

This is the reviewer's main concern, and we agree it is a crucial point. The reviewer correctly draws a parallel between the difficulty of finding a suitable data augmentation and the challenge of satisfying the properties for valid Instrumental Variables (IV). Both, as the reviewer notes, must often "lie purely on assumption."

We believe the most appropriate yardstick for evaluating our framework is indeed that of IVs. Our proposed IV-like (IVL) variables are a direct generalization of IVs, designed to exploit a specific trade-off:

• Availability vs. Identifiability: By relaxing the "outcome relevance" property of a standard IV, IVLs become easier to find and are more broadly available. The price for this increased availability is that IVL regression can no longer guarantee the identification of the causal effect. However, as we show, it can still be instrumental in reducing confounding bias.
• Inherited Limitations: Consequently, the limitations of IVLs regarding testability and reliance on prior knowledge are the same as those for standard IVs. The validity of an IVL (e.g., exclusion restriction, un-confoundedness) is not empirically testable from observational data alone and must be justified by domain expertise—the same standard applied to IVs for decades.
• Outcome invariance: Outcome-invariant data augmentation (DA), by virtue of equivalence with IVL by construction, also inherits these properties. The outcome-invariance assumption itself now becomes the central, untestable assumption, analogous to exclusion restriction and un-confoundedness. And while we acknowledge that this is a seemingly strong assumption, we point out that it is a fairly common modeling choice in the data augmentation and invariance literature.[1-10]
• Perturbing spurious features: The condition of IVLs and/or DA perturbing spurious features of the treatment $X$ is also not a new limitation introduced by our framework. In fact, this condition is just a re-framing of the exclusion principle in IVs and is also necessary for identifiability in IV regression. It doesn’t pop up simply because of our IVL relaxation. Indeed, if an IV does not enter $Y$ via spurious features of $X$, then the respective components of the causal function $f$ cannot be identified. Analogously, if no spurious features are perturbed, then IVLs cannot help mitigate confounding bias. The same holds for outcome-invariant DA by virtue of their equivalence with IVL.

$\overbrace{ \underbrace{\text{outcome-invariance}}\_{\text{popular model for DA}} + \underbrace{\text{spurious targets} }\_{\text{benign failure if violated}} }^{\text{un-testable DA assumptions}} \iff \overbrace{\text{un-confoundedness}+\text{exclusion}}^{\text{un-testable IV/IVL assumptions}}$

Ultimately, our work does not propose a "free lunch." Instead, it formalizes a trade-off that may be quite practical in many settings where valid IVs are scarce, but plausible outcome-invariances (i.e., data augmentations) are abundant. Consequently, we believe it would be inappropriate to expect IVLs and/or outcome-invariant DA not to come with the same limitations as IVs other than the one trade-off made — availability vs. identifiability.

In the camera-ready version, we will make sure to emphasize that these untestabe assumptions are inherent to the difficulty of causal estimation under hidden confounding, and not a limitation of our method per se.

Additional Reflections on untestable validity of IV vs. outcome-invariant DA: A majority of reviewers voiced concerns over the untestable assumption on the validity of the IV and outcome-invariant DA. While this is a fair and valid criticism, it’s worth pointing out the key distinction between IV and DA. The classic IV relies on premise that the IV induces a “natural” variation on the treatment. The researchers must therefore "look for” these variables passively and then need to justify why they act as a valid IV. In contrast, DA is typically designed by the researchers who have control over how the transformations are applied to the data. This flexibility can make DA outcome-invariant “by design”. In fact, it’s quite intuitive to distinguish between DA that are outcome-invariant and those that are not. For example, any approaches that use the labeled data to learn DA cannot be outcome-invariant.

In my opinion, this distinction makes outcome-invariant DA less susceptible to the question of validity compared to the classical IV.

On the Restricted Setting of the Theoretical Results

The reviewer notes that our theoretical results, particularly Theorem 1, operate in a more restricted setting than the general framing of the paper. We thank the reviewer for their understanding that this is reasonable given other strengths of our work.

We wish to clarify that the primary reason for Assumptions 1 and 2 is our choice of evaluation metric: the Relative Squared Error (RSE) by Janzing.[11] Janzing argues that the structural nature of RSE makes it a more relevant estimation error for causal settings [13], which also motivates our choice. Nevertheless, there is no generally accepted metric for causal estimation since quantifying causal influence is itself a non-trivial problem [13, 14]. So if we instead use the more familiar risk-based estimation error $\mathbb{E}^{\mathcal{M}}[ \ell( f(X), \hat{h}(X) ) ]$, we can in fact drop Assumptions 1 and 2 altogether from our results.

We therefore urge the reviewers to see these assumptions as a requirement for a more appropriate measure of causal estimation error rather than a necessary crutch. And we shall explicitly provide the same disclaimer in the camera ready version of our manuscript as well.

Question: Examining Violations of DA Assumption(s?)

We interpret this question as asking about the robustness of our method to violations of the core outcome-invariance assumption. Our paper mainly studies the case of exact invariance to make the connection with causal inference more obvious and tractable. And although exact invariance has precedence as a modeling assumption in DA [1-10], we understand the hesitancy to commit to this in practice. And naturally, this opens the door to thinking about approximate notions of invariance.[1, 6]

As such, we have been looking to incorporate the same in our framework. Preliminary results seem to suggest that our main theorems can indeed be extended to this more general setting. We plan to further develop this line of work in a journal version. This allows us to present a more focused and self-contained contribution here, while laying a clear path for future work.

We hope these clarifications have addressed the reviewer's concerns. We once again thank them for their positive and constructive review, which has helped us to polish our thinking and positioning of this work.

[1]: S. Chen et al. A group-theoretic framework for data augmentation. JMLR, 2020.

[2]: C. Lyle et al. On the benefits of invariance in neural networks. arXiv, 2020.

[3]: H. Shao et al. A theory of PAC learnability under transformation invariances. NeurIPS, 2022.

[4]: A. Fawzi and P. Frossard. Manitest: Are classifiers really invariant? BMVC, 2015.

[5]: Y. Dubois et al. Lossy compression for lossless prediction. NeurIPS, 2021.

[6]: M. Petrache and S. Trivedi. Approximation-generalization trade-offs under (approximate) group equivariance. NeurIPS, 2023.

[7]: O. Montasser et al. Transformation-invariant learning and theoretical guarantees for OOD generalization. NeurIPS, 2024.

[8]: D. Romero and S. Lohit. Learning partial equivariances from data. NeurIPS, 2022.

[9]: S. Zhu et al. Understanding the generalization benefit of model invariance from a data perspective. NeurIPS, 2021.

[10]: S. Wong et al. Understanding data augmentation for classification: When to warp? DICTA, 2016.

[11]: D. Janzing. Causal regularization. NeurIPS, 2019.

[12]: C. Vankadara et al. Interpolation and regularization for causal learning. NeurIPS, 2022.

[13]: D. Janzing and B. Schölkopf. Detecting confounding in multivariate linear models via spectral analysis. J. Causal Inference, 2018.

[14]: D. Janzing et al. Quantifying causal influences. Ann. Stat., 2013.

Thank you so much for your very encouraging feedback and enthusiasm for our work!