Identifiable Latent Causal Content for Domain Adaptation under Latent Covariate Shift

Q1: I would suggest to frame the graph as a set of assumptions, rather than as another "view" of the MSDA problem. This would make the limitations of the work clearer.

R1: Thank you for your thoughtful feedback. We agree that presenting our graphical representation in this manner would enhance clarity regarding the limitations of our work. We have added a Section A.8 in the revised version, including assumptions (i)-(iii) in proposition 4.2, and most importantly, assumptions on the proposed causal graph as formulated in Eqs. (1)-(3), to understand our assumptions for identifying $\mathbf{z}_c$. For examples, we assume two-parameter exponential family members for latent noise variables $\mathbf{n}$ as defined in Eq. (1), considering informative latent noise variables $n_i$ can be automatically separated from noise by an estimating model (see [1]). However, in real applications, the distribution of $\mathbf{n}$ could be arbitrary. In this context, assumption Eq. (1) may only serve as an approximation for the true distribution of $\mathbf{n}$. More details please see Section A.8.

[1] Sorrenson, Peter, Carsten Rother, and Ullrich Köthe. "Disentanglement by nonlinear ica with general incompressible-flow networks (gin)." arXiv preprint arXiv:2001.04872 (2020).

Q2: This could include the clarification on how this graph relates to typical causal and anti-causal tasks.

R2: Thank you for your valuable suggestion. Regarding causal or anti-causal tasks, we recognize that the complexities of specific tasks may require a more tailored causal graph structure. Nevertheless, we posit that these tasks might benefit from the proposed causal graph. The link between the proposed causal graph and causal or anti-causal tasks could be attributed to the deliberate modeling of latent causal variables within our framework. To illustrate, let us explore its potential application in segmentation tasks, where interpreting graph nodes as distinct regions may be reasonable. In this scenario, justifying the absence of direct connections between nodes may be conceivable; instead, connections should be conceptualized within high-level latent variables, as highlighted in the proposed causal graph. More details can be found in Section A.9 in the revised version.

Q3: Please add a discussion and mention the limitations of the work

R3: Thank you once again for your insightful suggestions. The most challenging assumption in our model necessitates the presence of substantial changes in latent noise variables, as they play a crucial role in identifiability. However, in practical applications, a prevalent challenge arises from the scarcity of extensive source domain datasets that adequately capture such substantial changes. In this context, the assumptions related to identifiability in our results may not be satisfied, and thus, the performance of the proposed method may benefit from the introduction of certain regularizations. One potential regularization approach involves leveraging the independence among the noise variables, and further details can be found in Appendix A.10.

Q4: Related to that of Alabdulmohsin et al., 2023

R4: Thank you once again for your valuable suggestions. We have incorporated the referenced work into our related work section. We think that each method possesses its own set of advantages and disadvantages. For instance, proxy-based methods offer the advantage of relaxing model assumptions, but they need a high-quality proxy. In contrast, our approach eliminates the need for a proxy variable but entails making stronger assumptions within the model.

Q5: please refrain from using superlatives

R5: Thank you for your valuable suggestions. Due to the constraints of the rebuttal time frame, our priority is to address more complex reviews. We will thoroughly revise inappropriate presentations in the final version. Your understanding is greatly appreciated.

Q1: This paper is highly incremental...its causal graph, learning method, and identifiability analysis are very similar to existing works [1, 2]. The only difference may lie in the addition modeling of the causal generation from zc to zs. However, this assumption may not be widely applicable. Particularly, in the Terra Incognita data and PACS data considered, the authors failed to elaborate why (this is important, since without this edge, the causal graph is the same as [1]). Besides, the identifiability analysis is also a simple application of [3].

R1, Part A: We strongly disagree the above.

Primarily, we wish to emphasize that the fundamental essence of our contribution is encapsulated within the proposed paradigm. Within this framework, each element of our proposed causal graph, learning method, and identifiability analysis has been meticulously crafted to address the nuances of the entirely novel problem paradigm we have introduced. This deliberate tailoring ensures that our methodologies are not merely incremental but intricately designed to suit the distinctive challenges posed by this innovative problem context. Regrettably, we observe a substantial oversight by the reviewer in recognizing the paramount importance of this foundational aspect.

Regarding to the proposed causal graph. In the context of causal graph analysis, a commonly accepted assumption asserts that the domain index causally affects latent variables, and subsequently, these latent variables causally influence the observed data denoted as X. Rooted in this foundational premise, the disparities among causal models becomes evident in the nuanced modeling of the latent space. Notably, our approach to modeling the latent space markedly diverges from that outlined in references [1,2]. As pinpointed, the core distinction lies in the modeling of the causal generation from zc to zs. This critical divergence encapsulates a pivotal aspect of our model, reinforcing the compelling rationale behind our unique representation of latent causal relationships within the system.

Concerning learning methods and identifiability analysis, the divergences in latent causal models underscore the imperative for distinct approaches. Each latent causal model necessitates a specialized learning method, tailored to capture the intricacies of its unique latent space. Furthermore, disparities in model structures and assumptions mandate a nuanced identifiability analysis, essential for ensuring robust and precise inference of causal relationships within the specified framework. In essence, the selection of a latent causal model not only shapes the learning strategy but also underscores the critical need for a meticulous identifiability assessment, reinforcing the foundation for reliable causal inference.

We firmly disagree with the assertion that 'our assumption may not be widely applicable.

Both assumptions—whether latent content variables cause latent style variables or by using a confounder, are inherently reasonable. It is challenging to definitively argue against the widespread applicability of our assumption that 'latent content variables cause latent style variables'. A robust method for validation involves a thorough assessment of the distinct advantages associated with each assumption within real-world applications, coupled with its recognition within the community.

For real-world applications, 'latent content variables cause latent style variables,' we present compelling experimental results for real-world dataset, showcasing a robust performance. For recognition within the community: Our assumption, latent content variables cause latent style variables, aligns with established works in the domain adaptation/generalization field [L1,L2,L3], as well as recent advancements in self-supervised learning [L4,L5].

Dear Reviewer 4yH5,

We greatly appreciate your feedback. Could you kindly verify if the provided clarification addresses your concerns, particularly regarding contributions?

Allow us to emphasize our contributions anew. The core of our work is embedded within the proposed paradigm. Under this framework, every component of our suggested causal graph, learning method, and identifiability analysis has been intricately fashioned to navigate the intricacies of the entirely novel problem paradigm we've presented. This intentional customization guarantees that our methodologies transcend mere incremental advancements, as they are intricately fashioned to align with the distinctive challenges posed by this innovative problem context.

If there are lingering uncertainties or if additional clarification is necessary, please don't hesitate to notify us. We understand the time constraints you face and truly appreciate your thoughtful consideration. Your reassessment plays a crucial role in advancing our work, and we stand prepared to offer any further clarification needed.

Best regards,

Authors

Q1: The contributions of this paper are limited. While this paper introduces a different paradigm, it appears to still be a latent representation disentangle mechanism from a causal perspective.

R1: We appreciate your time and careful consideration of our work, as reflected in your insightful review. However, we would like to express a disagreement with the notion that the contributions of our paper are limited. We believe our work significantly advances the field, particularly in the context of Latent Covariate Shift, and we would like to address the specific concerns raised.

New Problem Setting - Latent Covariate Shift: We introduce a novel paradigm, latent covariate shift, which deviates from the commonly-used Conditional Shift. This paradigm shift represents a substantial contribution to the field, offering a fresh perspective on addressing covariate shift issues.

Intricate Latent Causal Generative Model: Our paper presents a sophisticated latent causal generative model, introducing latent content variables and latent style variables. This model provides a nuanced understanding of latent covariate shift mechanisms, offering a more comprehensive and realistic representation.

Theoretical Analysis of Identifiability: We conduct a thorough analysis of identifiability in the proposed causal model, presenting both a complete non-identifiability result and a partial identifiability result for latent content variables. This theoretical foundation adds depth to our work and contributes valuable insights to the research community.

New Method for Latent Covariate Shift: Guided by our theoretical results, we design a new method specifically tailored for latent covariate shift. The proposed method is a practical application of our theoretical findings, showcasing the applicability and relevance of our work.

Verification of Advantages: Through rigorous experimentation, we validate the advantages of latent covariate shift and the proposed method. Our empirical results provide concrete evidence of the effectiveness and superiority of our approach.

In summary, the introduction of a new paradigm, theoretical analysis, method design, and empirical verification collectively demonstrate the significance and impact of our work on addressing latent covariate shift challenges.

Furthermore, we wish to emphasize that the fundamental essence of our contribution is encapsulated within the proposed paradigm. Within this framework, each element of our proposed causal graph, learning method, and identifiability analysis has been meticulously crafted to address the nuances of the entirely novel problem paradigm we have introduced. This deliberate tailoring ensures that our methodologies are not merely incremental but intricately designed to suit the distinctive challenges posed by this innovative problem context. Regrettably, we observe a substantial oversight by the reviewer in recognizing the paramount importance of this foundational aspect.

Q2: However, the feasibility of their latent variable modeling approach relies on conditions made in Proposition 4.2, which are not adequately explained and ensured against potential violations

R2: We appreciate your time and careful review again. The main assumptions, encompassing i to iii in Proposition 4.2, stem from the realm of nonlinear ICA. Over recent years, nonlinear ICA has found applications in diverse real-world scenarios, such as domain adaptation [1], image generation and translation [2], domain generalization [3]. Consequently, we assert that these three assumptions have been substantiated through practical applications. We have add a new Section A.8 for understanding these assumptions.

[1] Kong, Lingjing, et al. "Partial Identifiability for Domain Adaptation." ICML 2022.

[2] Xie, Shaoan, et al. "Multi-domain image generation and translation with identifiability guarantees." ICLR 2022.

[3] Wang, Xinyi, et al. "Causal balancing for domain generalization." arXiv preprint arXiv:2206.05263 (2022).

Dear Reviewer Corp,

Your feedback is invaluable to us. Could you kindly confirm whether the clarification we provided adequately addresses your concerns? If any uncertainties persist or if additional clarification is required, please donot hesitate to inform us. We genuinely appreciate your time, recognizing that you are undoubtedly busy. Your reconsideration is crucial for the progress of our work, and we are more than willing to provide any further clarification.

Q1: The justification of graph 1(c), $\mathbf{z}_c$ causes $\mathbf{z}_s$...Provide a motivating example.. The difference of $\mathbf{z}_c$ causes $\mathbf{z}_s$..

R1: Thank you for your time and efforts in enhancing this work. For clarifying these questions, we would like first further clarify the notations, including the domain index $\mathbf{u}$, latent content variables $\mathbf{z}_c$, latent style variable $\mathbf{z}_s$, label $\mathbf{y}$, and observation $\mathbf{x}$.

In domain adaptation, latent content variables, latent style variables, domain index, and labels play distinct roles in capturing and aligning information across different domains. Let us consider an example in the context of animal data, were collected from different locations:

Domain index $\mathbf{u}$: it denotes an indicator of the source domain from which the data originates, e.g., locations index).

Latent content variables $\mathbf{z}_c$: this denotes unobservable factors that represent the essential content or intrinsic characteristics of the object. For example: latent content variables encompass features such as the species, or specific morphological characteristics that are inherent to each type of animal.

Latent style variables $\mathbf{z}_s$: These variables aim to capture aspects of the data that are not specific to the intrinsic features, but style-related variations in the data. For example: latent style variables could represent variations in background, lighting conditions, environmental context.

Observation $\mathbf{x}$: The actual image data, including both content and style, including all visual information captured by camera.

Labels $\mathbf{y}$: it represents ground truth labels indicating the class or category of each instance. Example: a label refers to the assigned category or class that indicates the identity or type of each animal in the dataset.

Let us illustrate Figure 1 (c) with an example in the context of animal data. The relationship between $\mathbf{u}$ causing $\mathbf{x}$ is modeled to capture the fact that input in domain adaptation changes across different domains (locations).

Now, considering why $\mathbf{u}$ causes $\mathbf{z}_c$, let us explore how the unique characteristics and environmental factors associated with each location influence the latent content variables for zebras. For instance, the shades and patterns of the stripes on zebras vary based on the specific location, and certain adaptations to the local ecosystem can be reflected in $\mathbf{z}_c$.

Moving on to why $\mathbf{z}_c$ causes $\mathbf{y}$, $\mathbf{z}_c$ essentially capture features relevant to predicting $\mathbf{y}$ in a classification task. For a more detailed explanation, please refer to the paragraph 'zc causes y:' in the main paper.

$\mathbf{z}_c$ influence observations $\mathbf{x}$ because they serve as a conceptual representation of the underlying, unobservable factors that contribute to the observed data.

Finally, let us provide a example why $\mathbf{z}_c$ causes $\mathbf{z}_s$ in Figure 2 (a). An apt example is that different animals (corresponding to $\mathbf{z}_c$) often choose different environments (corresponding to $\mathbf{z}_s$) where the basic needs of the animals to survive are met.

Thus far, we have provided comprehensive explanations for all variables and causal directions depicted in Figure 1 (c) and Figure 2 (a). These causal directions not only elucidate the internal relationships within the models but also indicate how we effectively model the changes in various distributions across domains ($\mathbf{u}$). These considerations underscore the holistic nature of our approach, wherein the causal relationships guide our representation of the dynamic shifts in distributions across different domains.

For when $p(\mathbf{y}∣\mathbf{z}_c)$ remains invariant: it aligns with a fundamental principle in causality, specifically the notion of independent causal mechanisms. This principle posits that the conditional distribution of $p(\mathbf{y}∣\mathbf{z}_c)$ remains stable and unaffected by external factors. In the context of domain adaptation, assuming the invariance of $p(\mathbf{y}∣\mathbf{z}_c)$across domains is a common practice. The invariance in $p(\mathbf{y}∣\mathbf{z}_c)$ is crucial for ensuring the model's ability to make consistent and robust predictions in the target domain. This assumption implies that the predictive relationship between the label $\mathbf{y}$ and $\mathbf{z}_c$ remains unchanged despite variations in data distributions across different domains. This stability facilitates a seamless transition from training on source domains to making reliable predictions in the target domain, contributing to the model's adaptability and generalization.

Thanks for addressing my comments. I would like to clarify my points. If $g_c$ and $g_x$ are invertible functions, then it seems to me that one can merge $z_c$ and $n_c$ ($z_s$ and $n_s$) for the adaptation task. What is the intuition/technical reason to introduce additional variables? Will we still be able to identify the graph without them?

Additionally, I would suggest adding a remark in Proposition 4.2 to discuss why $p(y\mid z_c)$ is identifiable.

Thanks for the nice explanation for Q1. Connecting to the previous question, can the authors also give an example of $n_c$ and $n_s$ in this case? Besides, I have slightly mixed feelings about the explanation for $z_c\rightarrow z_s$. Would it also be the case that due to natural selection, the environment would cause animals with certain characteristics to survive? Although my point is not to overemphasize the justification, it might strengthen the motivation of the paper if there is an example that demonstrates a clear $z_c\rightarrow z_s$. A bidirectional edge case is also good, but I am not sure if this would break the analysis or not.

Regarding to "bidirectional edge case is also good". We acknowledge and do not refute this perspective. Indeed, existing literature has explored various ways to model the relationship between latent content variables $\mathbf{z}\_{c}$ and $\mathbf{z}\_{s}$, for example, bidirectional edge $\mathbf{z}\_{c} - \mathbf{z}\_{s}$, or a confounder $\mathbf{c} \rightarrow \mathbf{z}\_{s}, \mathbf{c} \rightarrow \mathbf{z}\_{c}$, or $\mathbf{z}\_{c} \rightarrow \mathbf{z}\_{s}$. We acknowledge the complexity of definitively arguing for the superiority of one assumption over another. In our opinion, the choice among these three assumptions is inherently reasonable, contingent upon the specific application scenarios under consideration, as we asserted in the related work. In the context of our study, particularly focused on animal classification tasks, we express a preference for the $\mathbf{z}\_{c} \rightarrow \mathbf{z}\_{s}$. This preference also aligns with the findings and methodologies of exsting works, including references [L1,L2,L3,L4,L5], which have demonstrated its effectiveness in capturing and explaining the underlying dynamics of similar systems.

[L1] Gong, Mingming, et al. "Domain adaptation with conditional transferable components." International conference on machine learning. PMLR, 2016.

[L2] Stojanov, Petar, et al. "Data-driven approach to multiple-source domain adaptation." The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 2019.

[L3] Mahajan, Divyat, Shruti Tople, and Amit Sharma. "Domain generalization using causal matching." International Conference on Machine Learning. PMLR, 2021.

[L4] Von Kügelgen, Julius, et al. "Self-supervised learning with data augmentations provably isolates content from style." Advances in neural information processing systems 34 (2021): 16451-16467.

[L5] Daunhawer, Imant, et al. "Identifiability results for multimodal contrastive learning." arXiv preprint arXiv:2303.09166 (2023).

We hope that these clarifications further address any concerns you may have. Should you have additional questions or concerns, we welcome further discussion. We sincerely appreciate the time and effort you dedicated to reviewing our work.

Dear Reviewer YFyZ,

Thank you sincerely for the further discussion. We have offered additional clarification. Your feedback holds immense value for us.

Could you please verify if the clarification we offered sufficiently resolves your concerns? If any uncertainties linger or if you require further elucidation, please inform us without hesitation.

We understand the demands on your time and genuinely appreciate your consideration. Your reconsideration is pivotal for advancing our work, and we are fully committed to furnishing any additional clarification you may need.

Best,