Causality is Invariance Across Heterogeneous Units

Dear Reviewer rhSD, Thank you for your review and the comments you provided. While we appreciate your recognition of the importance of our work’s theme, we would like to address some of the weaknesses and questions you raised.

1. Abstract Revision: We have rewritten the abstract to better focus on the core proposals of our paper, providing a clearer entry point into our research.
2. Invariance in Causal Modules Literature: Regarding the literature on invariant causal modules, we have reviewed relevant papers to situate our work within this context. However, we respectfully note that our approach, while acknowledging the broader literature, introduces a unique perspective, particularly in terms of utilizing neural network structures based on DiscoSCM for counterfactual predictions.
3. Counterfactual Predictions Literature: We respectfully disagree with the suggestion that our approach is not novel. While estimating counterfactual predictions is indeed not a new concept, our method, predicated on the DiscoSCM framework, provides a distinct and practical approach to counterfactual estimation in Layer 3. We have thoroughly reviewed and cited the relevant literature, including Pearl's work, to position our contribution within this larger context.
4. Representing Discrete Models: Concerning the representation of discrete models using degenerate Gaussian distributions, we have reviewed the suggested paper by Andrews et al. and others. However, our approach diverges significantly in methodology and application, as outlined in our paper.
5. Empirical Example and Results: We understand your concern regarding the absence of empirical results in the main text. We will consider integrating key findings from the supplemental material into the main text in our revised manuscript.
6. Focus of Section 5.1: We acknowledge your suggestion to restructure the discussion in Section 5.1. In our revised paper, we will aim to present these considerations in a more coherent and focused manner earlier in the text.

In conclusion, we believe our paper provides a valuable and novel contribution to the field of causal inference. The criticisms raised, while noted, do not detract from the fundamental strengths and innovations of our work. We hope that our thorough revisions and clarifications will help in better understanding the significance of our research. Thank you once again for your review.

Dear Reviewer XwiY,

Thank you for your thoughtful feedback on our manuscript. We appreciate the opportunity to address the concerns you raised regarding our model's approach and its conceptual contributions.

1. Necessity of Decoupling Individual Semantics: A key aspect of our model is the decoupling of individual semantics from the exogenous variable $U = u$. This approach is fundamental for the practical implementation of our individual/population-level counterfactual algorithms. Without this decoupling, we would be compelled to rely on Pearl's algorithms, necessitating complete knowledge of structural equations and the practical computation of $P(u|e)$, rendering the process intractable. This intractability has been a major hindrance in current literature for practically computing counterfactuals. Our model represents the first practical and reasonable approach to Layer 3 valuations, to the best of our knowledge, even allowing for heterogeneous counterfactual estimation.

2. Response to Specific Questions:

   • Running Example and Grammatical Issues: We are grateful for your observation regarding the issues in our running example and grammar. These have been corrected in the revised version of our manuscript.
   • "Reducible Pocket" Metaphor: The term "reducible pocket" is indeed a concept borrowed from Stephen Wolfram, who claimed finding pockets of reducibility is the story of science. We use this metaphor to describe the predictive capabilities of our DiscoModel on modeling complex underlying causal mechanisms.

We believe these clarifications and revisions will address your concerns and enhance the overall quality and understanding of our paper.

Thank you once again for your constructive feedback.

Dear Reviewer YHxQ,

Thank you for your valuable feedback on our manuscript. We are grateful for the opportunity to clarify and address the concerns you raised regarding our model's assumptions, the randomness of the causal representation variable, the focus on identifiability results, and notation clarity.

1. Clarification of Model Assumptions: We appreciate your concern about the complexity of our model's assumptions. However, there seems to be a misunderstanding in your interpretation. Our model's foundational assumption is that the outcome $Y_i$ is independent of the treatment $X_i$ given the treatment assignment $T_i$ and the causal representation $z_i$ for any individual $i$. This "unit causal representation assumption" is now explicitly stated in Section 2.1 of our revised manuscript. It is crucial for overcoming computational difficulties in calculating $P(i|e)$. The problem of calculating the probability of selecting a particular user from billions faces numerical overflow challenges, which are alleviated by computing the probability density $P(z_i|e)$ of a high-dimensional Gaussian distribution sample. This assumption is vital and commonly employed in various fields, akin to user embedding in recommendation systems [Bonner, Vasile 2018] or token embedding in language models, from a computational aspect.

2. Randomness of Causal Representation Variable: In our model, we denote the posterior of the unit selection variable $S$ as $P(\cdot|e)$, and its corresponding variable as $S(e)$. The randomness in the causal representation variable $z_{S(e)}$ arises from the selection of different units $i$. The DiscoModel's unique approach decouples individual semantics from the exogenous variable $U = u$, using the unit selection variable $S$ for individual-level computations. This enables our model to practically answer Layer 3 queries. We provide a more detailed explanation in Section 2.1 and the appendix of our revised version, comparing to the role of $U = u$ in determining both the selection of individual $i$ and the exogenous uncertainty in SCM.

3. Identifiability of Causal Representations: We argue that prioritizing identifiability results regarding causal representations is not the primary concern in causal research. Instead, the focus is typically on the identification of causal estimands, often involving the conversion of causal quantities (e.g. $P(y|do(x), c)$) into expressions that only contain conditional probabilities. The consistency assumption in traditional models bridges potential outcomes with conditional probabilities learnable from data, aiding in identification. Our model's distributional-consistency assumption and independent potential noise assumption serve a similar role, ensuring robust identification.

4. Notational Clarity: We acknowledge your feedback on the clarity and consistency of notations. We have conducted a thorough revision of the manuscript to ensure all notations are formally defined and consistently used. This revision aims to enhance the paper's accessibility and understanding for both readers and reviewers.

We trust these clarifications address your concerns and effectively highlight the robustness and novelty of our DiscoModel. We look forward to your reassessment of our manuscript.

Reference: S Bonner, F Vasile. "Causal Embeddings for Recommendation." Proceedings of the 12th

Dear Reviewer thPo,

Thank you for your insightful comments and constructive feedback on our manuscript. We have undertaken a thorough revision to address your concerns, focusing on enhancing the clarity and understanding of our paper.

1. Notational Clarity and Individual Semantics: To avoid confusion between the randomness associated with selecting an individual and the exogenous uncertainty, we have revised our notation, using $i$ to represent an individual, instead of $u$. This change underscores a key point of our research: the decoupling of individual semantics from the exogenous variable $U = u$.

2. Unit Causal Representation Assumption: We have explicitly articulated the assumption of unit causal representation in our revised manuscript. This assumption is crucial for addressing the computational complexity of $P(i|e)$ and forms the bedrock of our approach.

3. Rigorous Formulation of Equations and Concepts:We have provided a more rigorous description of the derivations of key equations, such as Equations (2) and (8). Moreover, we have included a more detailed introduction to DiscoSCM in the appendix, encompassing definitions and conclusions related to Independent Potential Noise DiscoSCM. This should enhance the technical clarity and rigor of our paper.

4. Addressing Concerns about DiscoSCM and do-calculus: In response to your query about what DiscoSCM offers beyond the classical do-calculus of Pearl, we clarify that the Layer 1 and 2 valuations under SCM and DiscoSCM are mathematically equivalent. The do-calculus is designed to convert $P(y|do(x), c)$ expressions into those using only conditional probabilities, thus not encompassing Layer 3 valuations. The significant contribution of DiscoSCM lies precisely in this layer. The differences and unique aspects of DiscoSCM, especially in handling Layer 3 queries, are conceptually illustrated through our running example.

We believe that these revisions and clarifications address your concerns and significantly enhance the manuscript's quality. We appreciate your feedback, which has been instrumental in refining our work.

Thank you once again for your review.