Adaptive Invariant Representation Learning for Non-stationary Domain Generalization

Thanks for your review. We would like to address your concerns as follows.

Q1. Regarding Theorem 4.5.

We clarify that the usage of Rademacher complexity is not our main contribution. Rademacher complexity is one conventional way to relate the learning bound with finite data setting (in practice we only have access to finite data). It has been used to construct learning bounds for many domain adaptation/generalization settings [1,2,3]. The key contribution in Theorem 4.5 is to construct a learning bound based on the non-stationary mechanism $M^{\ast}$ which provides the guarantee for the proposed algorithm.

We also clarify that the assumption about bounded loss function is common in domain adaptation/generalization literature [1,4,5]. This assumption is mild and can be easily satisfied by many loss functions. For example, cross-entropy loss can be bounded by $C$ modifying the softmax output from $\left(p\_1, p\_2, \cdots. p\_{|\mathcal{Y}|} \right)$ to $\left(\hat{p}\_1, \hat{p}\_2, \cdots. \hat{p}\_{|\mathcal{Y}|} \right)$ where $\hat{p}\_i = p\_i (1 - \exp(-C)|\mathcal{Y}|) + \exp(-C)$.

$K$ is a constant that depends on statistical distance $\mathcal{D}$ used measures the distance between two distributions. In particular, $K = \frac{1}{\sqrt{2}}$ when $\mathcal{D}$ is KL-divergence and $K = \sqrt{2}$ when $\mathcal{D}$ is JS-divergence. The details of $K$ are already given in the proof of Lemma A.1 in Appendix A. Based on your suggestion, we revised Theorem 4.5 to include this information.

Q2. Regarding Proposition 4.6 and reweighting method.

Could you provide more details for your argument “Secondly, Proposition 4.6 is also trivial”? We note that this proposition is not trivial and the goal of this one is to help us to connect Theorem 4.5 (which suggests us finding the mapping $m\_{t-1}^{\ast}: \mathcal{X} \times \mathcal{Y} \rightarrow \mathcal{X} \times \mathcal{Y}$ such that distance of the joint distributions $P^{X,Y}$ between two domains $D\_t$ and $D\_t^{\ast} = m\_{t-1} \sharp D\_{t-1}$ is minimized $\forall t \in [2,T]$) to our 2-stage algorithm shown in Remark 4.7 (which minimizes (1) distance of label distribution $P(Y)$ by importance weighting and (2) distance of label-conditional feature distribution $P(X|Y)$ by invariant representation learning).

We also clarify that importance weighting method is used to minimize the distance of label distribution $P(Y)$ between two domains $D\_t$ and $D\_t^{\ast}$ rather than achieving label-conditional invariance representation learning which is equivalent to minimizing the distance of label-conditional distribution $P(X|Y)$.

Q3. Regarding invariant property.

As suggested by Proposition 4.6, after conducting importance weighting, we need to find the sequence of mappings in the input space $\mathcal{X}$ that can help to minimize the distance of label-conditional distribution $P(X|Y)$ between 2 consecutive source domains. However, finding such mappings in input space may not be trivial due to the complexity of input space. We alleviate this issue by finding the sequence of mappings in the representation space $\mathcal{Z}$ such that each $m\_t^{\ast}$ can be replaced by $f\_t^{\ast}: \mathcal{X} \rightarrow \mathcal{Z}$ and $g\_t^{\ast}: \mathcal{X} \rightarrow \mathcal{Z}$. In essence, we find $f\_t^{\ast}$ and $g\_t^{\ast}$ such that the distance of the label-conditional distribution $P^{Z|Y}$ between two consecutive source domains is minimized. This objective is equivalent to finding label-conditional invariant representation between two consecutive source domains. (i.e., $P^{Z|Y}$ is invariant across 2 domains $D\_t$, $D\_{t+1}$ is equivalent to $\mathcal{D}\left( P^{Z|Y}\_{D\_t}, P^{Z|Y}\_{D\_{t+1}} \right) = 0$).

Dear reviewer VfLk,

Thank you for your time helping review and improve the paper. We hope our response has addressed all your concerns. Since we are approaching the end of the discussion period, please let us know if you have any other questions, and we are happy to discuss more. If you’re satisfied with our response, we sincerely hope you could reconsider the rating.

Best regards,

Authors

Thanks for your review. We would like to address your concerns as follows.

Q1. Regarding standard deviation.

Thanks for your suggestion. We’ve revised Table 1 to include standard deviation calculated from results of 5 different random seeds.

Q2. Regarding scalability.

We clarify that the design of our method allows it to scale well with larger datasets or more complex network architectures. In particular, compared to the base model (ERM), our method (AIRL) has only one additional Transformer and LSTM layers, incurring only minimal overhead in terms of computation. Note that we only need to use Transformer and LSTM layers during training. In the inference stage, the predictions are made by the fixed representation $g$ mapping and the classifier $h$ pre-generated by LSTM which then results in a similar inference time with ERM. Compared to existing works for non-stationary domain generalization, our method required fewer computational resources. It’s because of our effective design to capture non-stationary patterns. Specifically, LSSAE and DRAIN have more complex architectures and objective functions resulting in much more training time than our method. While DPNET has slightly better training time than ours, this model requires storing previous data to make predictions and is not generalized to multiple target domains. To further support our claim, the average training times (i.e., seconds) of these methods for different datasets are shown in the table below.

Q3. Regarding section 5 writing.

Thanks for your suggestion. Could you provide more details about what thing we should do to improve clarification for section 5? We will revise this section based on your comments in our final version.

Dear reviewer vTu8,

Thank you for your time helping review and improve the paper. We hope our response has addressed all your concerns. Since we are approaching the end of the discussion period, please let us know if you have any other questions, and we are happy to discuss more. If you’re satisfied with our response, we sincerely hope you could reconsider the rating.

Best regards,

Authors

Thanks for your review. We would like to address your concerns as follows.

Q1. Regarding problem setting and invariant learning.

We consider DG in non-stationary environment where domains evolve along specific direction (e.g., time, space). This setting is different from conventional DG setting considered in existing methods (including IRM). In essence, conventional DG assumes domains are sampled from a stationary environment and target domains lie on or are near the mixture of source domains. In addition, evaluation protocol is also different between non-stationary and conventional DG. Models designed for conventional DG follow “leave-one-out” evaluation (i.e., one domain is target while the remaining domains are source) because the goal is to generalize to arbitrary domains near the mixture of source domains. Meanwhile, in non-stationary DG, models are trained on previous domains and evaluated on future domains in domain sequences. We also note that dataset (colored MNIST) used in IRM paper is different from the one (rotated MNIST) used in our work.

Under the assumption that target domains lie on or are near the mixture of source domains, existing works that enforce invariant representations across all source domains can help to generalize the model to target domains [1,2,3]. However, this assumption may not hold in non-stationary DG where the target domains may be far from the mixture of source domains resulting in the failure of the existing methods. In non-stationary DG, the key is to capture non-stationary patterns from the sequence of source domains which can help us to estimate the target domains. Our proposed method captures these patterns in the representation space via learning the sequence of invariant representation space in which each space is for a pair of consecutive source domains. Note that, this method is different from the existing works that learn a general invariant representation space for all source domains.

Q2. Regarding the usage domain index and order.

We clarify that both our method and all currently existing baselines for non-stationary DG rely on the utilization of domain order information. This information is necessary to capture non-stationary patterns over the source domains, thereby playing a pivotal role in facilitating models to generalize to target domains in the context of non-stationary DG. It is noteworthy that in numerous applications in computer vision [4], natural language processing [5], and healthcare [6], this information naturally emerges and does not necessitate additional efforts for collection.

Q3. Regarding Definition 4.1.

We clarify that the “generalizability” term $\Theta\left( \widehat{M} \right)$ defined in Definition 4.1 is small does not imply $\widehat{M}$ is a good mechanism because the errors of $\widehat{M}$ on both source and target domains may be large while $\Theta\left( \widehat{M} \right)$ is small. Instead, $\Theta\left(\widehat{M}\right)$ measures how consistent in terms of performance of mechanism $\widehat{M}$ on capturing non-stationary patterns on source and target domains. To avoid ambiguation, we’ve revised Definition 4.1 by replacing the term “generalizability” with “non-stationary consistency”.

We also note that rather than working with an arbitrary $\widehat{M}$, we focus on $M^{\ast} = \arg\min\_{\widehat{M} \in \mathcal{M}} \frac{1}{T-1} \sum\_{t=2} \mathcal{D} \left( P^{X,Y}\_{D_t}, P^{X,Y}\_{\widehat{D}\_t} \right)$ in our analysis (Theorem 4.5). In practice, we can achieve small error of $M^{\ast}$ on source domains during training. Then, under Assumption 4.3 for $\Theta\left( M^{\ast} \right)$, $M^{\ast}$ is guaranteed to achieve small error on target domain.

Q4. Regarding Assumption 4.3.

We clarify that Assumption 4.3 about $\Theta\left( M^{\ast} \right)$ is reasonable. If Assumption 4.3 does not hold, it means that non-stationary patterns underlying target domain are very different from the patterns estimated from the source domains, making the task to generalize to target domain infeasible. It is noteworthy that we do not mention Assumption 4.3 is equivalent to “assuming that there exists a pattern in the hypothesis space” because it only applied for optimal mechanism $M^{\ast}$. We only mentioned that Assumption 4.3 implies that assumption. We’ve revised our writing to avoid this ambiguity.

Q5. Regarding section 5 writing.

Thanks for your suggestion. Could you provide more details about what thing we should do to improve clarification for section 5? We will revise this section based on your comments in our final version.

Dear reviewer 6RJQ,

Thank you for your time helping review and improve the paper. We hope our response has addressed all your concerns. Since we are approaching the end of the discussion period, please let us know if you have any other questions, and we are happy to discuss more. If you’re satisfied with our response, we sincerely hope you could reconsider the rating.

Best regards,

Authors

Thanks for your review. We would like to address your concerns as follows.

Q1. Regarding definition of generalizability term $\Theta\left(\widehat{M}\right)$.

To clarify, $\Theta\left(\widehat{M}\right)$ is defined as the difference between two terms: error of $\widehat{M}$ on sequence of source domains $\frac{1}{T-1}\sum\_{t=2}\mathcal{D}\left( P^{X,Y}\_{D\_t},P^{X,Y}\_{\widehat{D}\_t}\right)$ and error of $\widehat{M}$ on target domain $\mathcal{D}\left(P^{X,Y}\_{D\_{T+1}},P^{X,Y}\_{\widehat{D}\_{T+1}}\right)$. In other words, $\Theta\left(\widehat{M}\right)$ measures how consistent in terms of performance of mechanism $\widehat{M}$ on capturing non-stationary patterns on source and target domains, and the term “generalizability” used in Definition 4.1 implies it. Clearly, good $\widehat{M}$ makes $\Theta\left(\widehat{M}\right)$ small but $\Theta\left( \widehat{M} \right)$ does not imply $\widehat{M}$ can capture the evolving on target domain because $\widehat{M}$ may have large error on both source and target domains. Thanks for your comment, we’ve revised Definition 4.1 by replacing the term “generalizability” with “non-stationary consistency” to avoid ambiguity.

Q2. Regarding Assumption 4.3.

We clarify that Assumption 4.3 implies error of $M^{\ast}$ on target domain is small. Note that Assumption 4.3 holds only for the optimal mechanism $M^{\ast} = argmin\_{\widehat{M} \in \mathcal{M}} \frac{1}{T-1} \sum\_{t=2} \mathcal{D} \left( P^{X,Y}\_{D_t}, P^{X,Y}\_{\widehat{D}\_t} \right)$ instead of arbitrary $\widehat{M}$. During training, we can feasibly achieve small source error for $M^{\ast}$ (i.e., $\frac{1}{T-1} \sum\_{t=2} \mathcal{D} \left( P^{X,Y}\_{D\_t}, P^{X,Y}\_{D^{\ast}\_t} \right)$ by using high-capacity hypothesis class (e.g., neural network). This fact combining with Assumption 4.3 ($\left \| \mathcal{D} \left( P^{X,Y}\_{D\_{T+1}}, P^{X,Y}\_{ D^{\ast}\_{T+1}} \right) - \frac{1}{T-1} \sum\_{t=2} \mathcal{D} \left( P^{X,Y}\_{D\_t}, P^{X,Y}\_{D^{\ast}\_t} \right) \right| < \epsilon$) implies that error of $M^{\ast}$ on target domain $\mathcal{D} \left( P^{X,Y}\_{D\_{T+1}}, P^{X,Y}\_{ D^{\ast}\_{T+1}} \right)$ is small.

Q3. Regarding uncertainty metric to Table 1.

Thanks for your suggestion. We’ve revised Table 1 to include standard deviation calculated from results of 5 different random seeds.

Q4. Regarding the setting the proposed approach outperforms conventional DG methods.

To clarify, our method is designed for the setting where data evolves along a specific direction (e.g., space, time), and by taking non-stationary environment into consideration, our method achieves better performance compared to existing DG methods in this setting. In essence, existing DG methods often enforce invariant representations across all domains, and it has been shown that when target domains lie on or are near the mixture of source domains, enforcing invariant representations across all source domains can help to generalize the model to target domains [1,2,3]. However, this assumption may not hold in non-stationary DG where target domains may be far from the mixture of source domains resulting in the failure of existing methods.

To further support this argument, we conduct an experiment on rotated RMNIST dataset with DANN [4] – a model that learns invariant representations across all domains. Specifically, we create 5 domains by rotating images by 0, 15, 30, 45, and 60 degrees, respectively, and follow leave-one-out evaluation (i.e., one domain is target while remaining domains are source). Clearly, the setting where target domain is images rotated by 0 or 60 degrees can be considered as non-stationary DG while other settings can be considered as conventional DG. The performances of DANN with different target domains are shown in the following table. As we can see, the accuracy drops significantly when target domain is images rotated by 0 or 60 degrees. This result demonstrates that conventional DG methods are not suitable for non-stationary DG.

Q5. Regarding FMoV dataset.

Thanks for pointing out the great resource related to our research. We will explore this dataset in future works.

Q6. Regarding Table 1 presentation.

Thanks for your suggestion. We’ve revised Table 1 to distinguish different approaches.

References

[1] Albuquerque, Isabela, et al. "Generalizing to unseen domains via distribution matching."

[2] Sicilia, Anthony, Xingchen Zhao, and Seong Jae Hwang. "Domain adversarial neural networks for domain generalization: When it works and how to improve."

[3] Phung, Trung, et al. "On learning domain-invariant representations for transfer learning with multiple sources."

[4] Ganin, Yaroslav, et al. "Domain-adversarial training of neural networks."

Dear reviewer Uk8r,

Thank you for your time helping review and improve the paper. We hope our response has addressed all your concerns. Since we are approaching the end of the discussion period, please let us know if you have any other questions, and we are happy to discuss more. If you’re satisfied with our response, we sincerely hope you could reconsider the rating.

Best regards,

Authors