Rectifying Group Irregularities in Explanations for Distribution Shift

Thank you for reading our paper and for the review. Our response is below:

Requirement of group annotations

In Section 4.1 of our initial submission, we briefly mention that our method can apply to scenarios without group labels and we included an experiment in Appendix E. We find that using KMeans to determine groups results in improving the feasibility and robustness of explanations similar to when we use prespecified groups.

Extension to image/language data as naive

Regarding the point that the extension to image/language data was naive, we want to emphasize that the way of transforming image and language data into interpretable features in our method is just one way of generating human-understandable shift explanations. In the updated paper, we further expanded our framework by considering two additional methods of generating shift explanations for image and language data. These two methods are the following:

1. Deriving shift explanations in a pre-trained model’s embedding space and decoding the embedding into human-understandable explanations using a pre-trained vec2text [Morris et al, 2023] model.
2. Using concepts (see [Koh et al, ICML 2020]) as the interpretable features for generating shift explanations for image data.

We performed additional experiments on language and image data with these two methods and included the results in Appendix H. The results again demonstrate the benefits of GSE in rectifying group irregularities and enhancing feasibility and robustness in generating shift explanations. In addition, to the best of our knowledge, other advanced techniques for generating shift explanations for image and language data do not yet exist. The only other work on shift explanations is from Kulinski et al. [Kulinski et al, ICML 2023] which also considered shift explanations over bag-of-word features for language.

Finally, we emphasize that instead of proposing new methods for extracting interpretable features from different data modalities, we wanted to show that group irregularities exist when using shift explanations over different modalities and that our method is able to alleviate this issue. We hope that further research can use our framework to propose more sophisticated shift explanation methods for images and data.

[Morris et al, 2023]: Morris, John X., et al. "Text embeddings reveal (almost) as much as text." arXiv preprint arXiv:2310.06816 (2023).

[Kulinski et al, ICML 2023]: Sean Kulinski and David I Inouye. Towards explaining distribution shifts. In International Conference on Machine Learning, pp. 17931–17952. PMLR, 2023.

[Koh et al, ICML 2020] Koh, Pang Wei, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. "Concept bottleneck models." In International conference on machine learning, pp. 5338-5348. PMLR, 2020.

Thank you for reading our paper and for the helpful feedback. We respond to your comments and questions below:

Discussion of DRO

Thank you for the suggestion to discuss DRO, we added a short discussion to Section 5. Our formulation of the worst-group loss is a form of DRO for learning distribution shift explanations. While DRO is often used for reducing spurious correlations in a discriminative model, we use it for finding shift explanations that can alleviate the group irregularity issue and enhance feasibility and robustness at the same time. In our response to reviewer XuMW, we demonstrate that improving the feasibility of the shift explanations can be tied to mitigating spurious correlations. Furthermore, leveraging worst-group loss can not only contribute to reducing spurious correlations but also yield the additional benefit of enhancing the robustness of the generated shift explanations. To substantiate these claims, we further provide theoretical analysis (Section 3.5 in the revision) and comprehensive empirical evaluations in the paper.

Connection to Wasserstein DRO

Regarding the connection between our approach and Wasserstein DRO, despite the necessity of optimizing Wasserstein distance in both our setting and prior studies such as [Kuhn, Daniel, et al], we address an orthogonal problem from them. In those prior studies, Wasserstein DRO is a replacement for ERM which optimizes a risk over all distributions within a small Wasserstein ball for learning a more robust model. On the other hand, as pointed out in [Kulinski et al, ICML 2023], the goal of optimizing Wasserstein distance in our setting is for generating explanations for understanding how distribution shift occurs. Plus, different from [Kulinski et al, ICML 2023], our primary focus is on investigating the group irregularity issues in shift explanations learned by optimizing reduction in Wasserstein distance. We added some more discussion concerning this point in Section 5 in the updated paper.

[Kulinski et al, ICML 2023]: Sean Kulinski and David I Inouye. Towards explaining distribution shifts. In International Conference on Machine Learning, pp. 17931–17952. PMLR, 2023.

Dear authors,

Reviewer 3qo2 raised concerns about the text examples for group membership. Would you be able to use another example to address the concerns?

Thanks, Your AC

Thank you for the review! We respond to your points below:

Modeling the problem using shifts between conditional distributions

We model the problem as explaining shifts between distributions which also respect subgroups. A subgroup is a conditional distribution; for instance, a group can be the source distribution conditioned on age being over 30. We do not use the term conditional distribution since the term “group” is more common in fairness and robustness literature [Dwork et al, ITCS 2012; Sagawa et al, ICLR 2020].

Statements lacking references

We believe that everything mentioned was in fact cited. Everything mentioned in the abstract is cited in the beginning of the introduction. We are not aware of any ICLR papers with citations in the abstract. We also cite the Adult dataset in the introduction (last paragraph of page 1) before it is mentioned in the caption of Figure 1. The percentages at the end of page 3 explicitly refer to the explanation shown in Figure 3a which is cited in the same sentence with the percentage. If there are other places where you believe we are lacking references, please let us know and we will be happy to update the paper.

Confusing and imprecise sentences

The sentences mentioned are all well formed and accurate for their context. We updated the paper to further clarify these sentences or removed them if they were not necessary. The “map a male close to a female…” example appears in the introduction and explains a phenomenon at a high level, but it is still an accurate description. Since this caused confusion, we decided to remove it. For contributions 2-4, we clarified their differences in the paper, we explained “quality” in the caption of Figure 2, and we added a sentence to clarify the setting of Figure 3b.

Theorem

We removed all numbers and explained the parameters in Theorem 1 based on your suggestion. By removing the numbers, we generalized the Theorem. Also, as suggested by reviewer XuMW, we further provided additional theorems to prove that our method can enhance the feasibility and robustness of shift explanations.

Generalization to image data

Regarding the generalization to image data remaining a pure language problem, we only introduce such a translation of image data to language data to produce interpretable explanations, but our method does not depend on this featurization technique. In addition, even though we are using language features, this does not mean that image properties are not represented. Language can capture many image-specific properties such as brightness, perspective, and style which are also used by the text-to-image model. For instance, in Figure 5 of the original submission, the generated caption includes “zoo photography” which is an image style feature

Appendix G of the initial submission also includes experiments where we learn an explanation directly over image pixels to show that our method is not dependent on this featurization step, but the resulting explanations are not interpretable. We have modified Section 3.4.2 to reference these additional experiments in Appendix H. We hope that future work proposes more sophisticated interpretable image featurization techniques so we can use them within our framework to get better image shift explanations.

Choice of feasible and infeasible features

Regarding why we considered the sex feature unfeasible and age feasible, we made this choice based on existing work on feasibility of explanations [Poyiadzi et al, AAAI 20] where sex was treated as immutable and age was partially mutable. Therefore, we allowed age to be modified and deemed the sex feature as infeasible.

We also want to address the ethics issue. It was not our intention to make anyone feel targetted or discriminated against and we apologize if that happened. The choice of sex as infeasible comes from prior work on feasible explanations [Poyiadzi et al, AAAI 2020; Mothilal et al, FAT* 2020]. We do not consider sex as generally infeasible, but we acknowledge that there are cases where a user may not want an explanation to include the sex feature. For instance, [Salimi et al, ECAI 2023] conducted a user study on existing explanation techniques and a user commented that “it is highly unethical to suggest someone changes their sex.” The choice of infeasible features is entirely dependent on a user’s intentions, and we chose to use sex as one example. Instead of sex as an infeasible feature, we can use race or age. In our revision, we currently replaced the example with one using race as an infeasible feature, but we also included an example with age as infeasible in Appendix F, and if the reviewer prefers age then we can swap the examples.

Conditional distributions: the potential readers of the manuscript will have background from other areas as well and explaining once that "group" and "conditional density" can be used interchangeably will improve the readability significantly.

References: It is the authors' responsibility to be accurate with citations. To this end, I commented examples where I found that citations were too inaccurately used. It is not sufficient to have a reference somewhere that could be the same statement as the first sentence in the abstract (note that I did not insist on adding a reference there but rather to repeat the sentence in the introduction with citation). Regarding the percentages (p. 3), it is not relevant that they are in the figure - it must be said where these numbers are from (= reference or own experiment).

I appreciate the improved formulations in various places. However, I still don't get the understanding between contributions 2 and 3. Contribution 4 is just about the necessary experiments to confirm contribution 2.

The new formulation of the theorem is more relevant.

Image-based generalization: Where does the proposed pipeline include image features ("not interpretable")? The hard part of image representation is the extraction of a descriptive text; however, there are also features that cannot be described by words. A proper image-based generalization should also be in the position to handle those non-textual features. To this end: consider including a brief version of the "raw pixel" experiment, including a short description and a comparison to the text-based version in the main paper.

Regarding the feasible features: maybe a clarification had been sufficient to avoid a potential issue here; also not sure that the wording "race" is more appropriate in that respect.

Thank you for taking the time to read and provide feedback on our paper! Our response is below:

Regarding the novelty of the proposed method

We appreciate the pointer to prior work [1] which tackles subpopulation shifts with worst-group loss. The scope of our paper, however, is not to solve the classical subpopulation shift issue as in [1] where model performance on subpopulations is the primary focus. Instead, our goal is to reveal that poor performance on subgroups occurs in distribution shift explanations and propose to leverage worst-group loss to alleviate this issue. In our paper, instead of just extending the optimization of overall PE to the worst-group PE, we further provide theoretical analysis and extensive empirical evidence to demonstrate that optimizing worst-group PE can enhance the feasibility and robustness of distribution shift explanations. To our knowledge, this has not been explored by any prior works. We have expanded on the comparison with DRO in Section 5 and highlighted our theoretical analysis in Section 3.5 in the updated paper.

Use of “feasibility” and “robustness” in the introduction and motivation

Thank you for the feedback on clarifying the discussion of feasibility and robustness. We added additional details to the high-level description of feasibility in Section 2.1 based on your feedback. Intuitively, feasibility measures the percentage of source samples that are modified in an actionable way by the explanation. For instance, if the explanation modifies the race feature (which we may consider as unactionable) for half of the source samples, then the feasibility is 0.5. Robustness measures the degree to which the shift explanation changes with respect to small perturbations to the source distribution. If there are other confusions caused by our high level definitions in Section 2.1, we will be happy to modify the text to make it clearer.

We also admit that the formal definitions of “feasibility” and “robustness” given in Section 3.4 in the original submission came too late. To address this issue, we move Section 3.4 right after Section 3.1 so that the reader can immediately get familiar with these definitions after the motivating examples.

Clarity of Figure 1a and 3a

Based on the feedback from reviewer 3qo2, we changed the example in Figure 1 to no longer use the sex feature. Figure 1a is meant to illustrate how the Vanilla explanation can change the proportions of Black and White people, shown as black and white figures respectively, while the GSE explanation is depicted as keeping the proportion the same. Figure 1a is a comparison of Vanilla and GSE explanations on the global level and we have added a subcaption to clarify this. The other part of the comparison happens with the annotated “Reduction in Wasserstein distance” in Figure 1a where the GSE explanation is shown to have a total reduction in Wasserstein distance very similar to the reduction for the Black subpopulation while the Vanilla explanation has a larger gap between these two reductions. We explain this at the bottom of page 1.

Theoretical justification of GSE improving feasibility and robustness

We added two additional theorems to analyze robustness and feasibility in a 1D setting in Appendix J. Theorem 2 shows that robustness of a regular explanation scales with a parameter of the target distribution, which is worse than the robustness of a worst-group explanation which is 0. Theorem 3 shows that the feasibility of a regular explanation can be 0 while the feasibility of a worst-group explanation is 1. We have limited the theoretical analysis to the 1D setting since we wanted to validate that group irregularities occur even in the simplest 1D case and that worst-group optimization can mitigate the problem. More comprehensive theoretical analysis beyond 1D settings is left for future work.

Connection to domain generalization and adaptation

We added a short comparison of domain generalization and adaptation with shift explanations to the related work. Both domain generalization and adaptation are methods for learning models while shift explanations are concerned with explaining how data shifts when distribution shifts happen.