Counterfactual Image Generation for adversarially robust and interpretable Classifiers

Thank you for your review. We appreciate the time and effort you have put into reading our work and communicating your assessment.

We would like to address the points mentioned in "Weaknesses":

1. We acknowledge that being limited to binary classification may seem like a limitation. However, this setting covers a wide range of problems in medical health diagnosis, structural health monitoring, and other tasks involving anomaly detection. Furthermore, we argue that visual explanations of class predictions are most informative when compared against a “background” class, as, for example, directly comparing an image of a table against an image of a chair may not yield the best visual explanation of what a chair is. As such, our framework would be applicable to multi-class problems when formulating them as several, separate binary classification tasks where each class is either present or absent from the image in question.

2. We acknowledge the importance of this point. In our experiments, although there was some variability in the perceived quality of counterfactual image generations across different runs, the classification performance remained mostly consistent. Our primary focus was on demonstrating the utility of counterfactual image generation for image classifiers, rather than on classification performance per se. Hence, we prioritized showcasing other metrics in the limited space available. Additionally, given the short timeframe for rebuttal and the substantial training time required (12 to 24 hours for larger models), systematically re-training all models to collect comprehensive multi-seed data was not feasible.

3. The Hybrid discriminator is introduced in Section 3.1.2. We added a sentence explicitly stating the name that will be used for the hybrid architecture in the revised version.

4. We indeed combined the classifier and discriminator into a single model, as it has outputs for estimating the “fakeness” of an image as well as one for the class prediction. The confusion may come from the fact that we added the auxiliary task of classifying the images to the generator as well. In our framework, both the generator and discriminator make predictions for the class.

5. We acknowledge the absence of a comparative analysis with other counterfactual approaches in our paper. Our challenge was the scarcity of comparable counterfactual image generation methods that reported performance metrics aligning with our focus, such as saliency map evaluation against segmentation labels. Additionally, re-implementing existing methods with sufficient fidelity for an accurate performance comparison proved to be complex. As such, we focused on demonstrating the efficacy of our method within the constraints of available comparative data and reproducibility standards.

Concerning the questions:

1. They do indeed both have good classification performances. If efficiency is important, then the discriminator is the better choice due to its smaller architecture size. If the saliency maps are always necessary, then we would suggest using only the generator. However, the one would obtain the best performing model by combining the generator's and discriminator's prediction as kind of ensemble.
2. There is no limit to the number of cycles that could be used until hardware limitations (memory) are reached. We experimented with values higher than 1, but the minor improvements did not justify the longer training times.
3. You are of course correct. This mishap must have occurred during last-minute updates to the Figure's legend in an effort to make all legends consistent. We corrected the mistake and uploaded the new Figure.

Thank you for your review. We appreciate the time and effort you have put into reading our work and communicating your assessment.

We would like to address the points mentioned in "Weaknesses":

1. There was indeed important information missing. We added the values used for the perturbation bound, norm, and codebase used for the attack. We selected a rather small L_inf bound of 0.1 because we are interested in small, imperceptible modifications of the image.

2. The generator’s objective is to create an image belonging to the opposite class compared to the original image and is as such trained to produce counterfactuals. The information about the semantics of the two classes is provided by the gradients flowing through the discriminator. Indeed, this information may be imperfect when the discriminator focuses on non-class-relevant features, like radiologist annotations in medical images. In such scenarios, our generator might include these annotations in healthy samples, creating images that are not necessarily counterfactuals in real-world terms, but still counterfactuals relative to the dataset and the discriminator’s predictions. The evaluation on segmentation masks yields a quantitative metric for the produced counterfactual samples and can reveal such “Clever Hans” situations.

3. Indeed, other models could be excellent at generating counterfactual images as well. However, the counterfactual image generation per se is only a subtask in our study, which aims at merging two lines of work: one focusing on using GANs for creating counterfactual images and one using GANs for creating adversarially robust classifiers. As such, our models are not comparable to diffusion models or VAEs.

Concerning the question on other GAN paradigms: our method could be used in combination with other GAN variants that aim at improving training stability and output quality, such as StyleGAN. We adopted the CycleGAN approach because it is an intuitive addition and has already been proposed for counterfactual image generation: Charachon et al., “Leveraging conditional generative models in a general explanation framework of classifier decisions”.

Thank you for your review. We appreciate the time and effort you have put into reading our work and communicating your assessment.

We would like to address the points mentioned in "Weaknesses":

1.2. Our method is designed for binary image classification tasks, which involve two distinct classes. In this context, the 'opposing class' refers to the alternative category that an image does not currently belong to. The aim is to achieve minimal alterations to the original image to transition it convincingly into this opposite class.

1.3. Our critique of current explainability methods using GANs is that they miss an opportunity for dual improvement. By fixing the classifier's weights during the adversarial generation phase, these methods manage to highlight a classifier’s proneness to small perturbations but do not offer a solution for how to overcome this limitation. Our proposed approach addresses this gap.
We acknowledge that this paragraph lacked some clarity and revised it in the new version.

1.4. Unlike pre-trained generative models, our methodology creates targeted augmentations in an online fashion during training. The generator in our framework is designed to produce minimally altered images that are misclassified by the discriminator in terms of both realism and class attribution. As the generator and discriminator are trained together in a min-max game, these augmented samples progressively improve. This leads to increasingly refined counterfactual samples.

1.5. This is probably related to a previous question: in this work, we focused on binary image classification. The second important aspect of our work is the fact that we combine the classifier and discriminator in a single network. It must predict the class of the image and whether the image was generated or not. The generator, on the other hand, must create images that the discriminator (i) misclassifies (attributes to the wrong class) and (ii) fails to detect that it was generated (outputs a high probability that the image belongs to the real distribution although it was generated).

1.6. Again, this probably becomes more clear when narrowing down the problem to a binary classification task.

2.1. We use the terminology of a of research leveraging counterfactual images as explainability for neural image classifiers:
CounteRGAN: Generating Realistic Counterfactuals with Residual Generative Adversarial Nets, Daniel Nemirovsky et al.
GANterfactual—Counterfactual Explanations for Medical Non-experts Using Generative Adversarial Learning, Silvan Mertes et al.
EXPLAINING IMAGE CLASSIFIERS BY COUNTERFACTUAL GENERATION, Chang et al. We included additional citations of related work in the area of counterfactual image generation as additional information for the readers.

3.1. ACGANs indeed share similarities with our method in integrating the discriminator and classifier. However, our approach differs significantly, primarily in our generator's design, which is unconditioned, as opposed to the class-conditioned generator in ACGANs. This distinction is crucial for generating explanations in our framework, where we compare the original image directly with its counterfactual generated by G. To achieve something similar with ACGANs, one would have to obtain the latent representation z of the original image, generate a new image conditioned on class 0, and another on class 1, and then compare these two generated images. In this scenario, one would be comparing two generated images, where important features from the original image may have been lost or modified. ACGANs were not designed to solve the same problem that we are targeting.

3.2. We acknowledge that being limited to binary classification may seem like a limitation. However, this setting covers a large range of problems in medical health diagnosis, structural health monitoring, and other tasks involving anomaly detection. Furthermore, we argue that visual explanations of class predictions are most informative when compared against a “background” class, as, for example, directly comparing an image of a table against an image of a chair may not yield the best visual explanation of what a chair is. As such, our framework would be applicable to multi-class problems when formulating them as several, separate binary classification tasks where each class is either present or absent from the image in question.

Thank you for your review. We appreciate the time and effort you have put into reading our work and communicating your assessment.

We would like to address the points mentioned in "Weaknesses":

1. Recent studies demonstrate a converging relationship between adversarial robustness and model interpretability in neural networks. For instance, [1, 2] show that adversarially trained networks' loss gradients align with features that are intuitively salient to humans. These two works are mentioned in our related work section.
   An advantage of our framework compared to a two-stage approach is efficiency: we create data augmentation, adversarial samples, and saliency maps in a single training run. Furthermore, the saliency maps in our method are actually a byproduct of our adversarial objective, which encourages the classifier to focus on perceptually relevant features. Using GANs for improving classifier robustness has been extensively studied (c.f. our related work section) - we made tweaks to those methods (including weight sharing between classifier and discriminator, regularization between original and generated image, etc.) which allow us to simultaneously leverage the adversarial examples as visual explanations. This is a clear benefit over current attribution methods and frameworks for adversarial robustness.
   [1] Robustness May Be at Odds with Accuracy, Tsipras et al.
   [2] Adversarial Explanations for Understanding Image Classification Decisions and Improved Neural Network Robustness, Woods et al.

2. We acknowledge two main points of complexity: (i) the generator’s classification head, and (ii) the role of cycle consistency loss.
   (i) The generator's classification head is crucial as our generator is unconditioned, a notable deviation from similar models. This classification head infuses direct class information into the generator, enhancing its performance under the multi-task learning paradigm. While we were not able to include detailed ablation studies due to the page limit, we plan to publish the codebase for reproducing our results, which may shed more light on the impact of different hyperparameters on the final model performance.
   (ii) The cycle consistency loss, as discussed by Charachon et al. in “Leveraging conditional generative models in a general explanation framework of classifier decisions” (c.f. citation in Section 3.2.2), is proven effective in related contexts. It contributes to the overall stability and expressive capability of the generator, ensuring more reliable and coherent outputs.

3. The non-adversarially trained models in our study were developed using standard supervised learning techniques. Due to space constraints, we couldn't include all training details in the paper, but we intend to release our complete codebase post-review, which will offer comprehensive insights into our implementation.
   The “D” in Section 4.5. refers to the “Hybrid D”, where “Hybrid” specifies the architecture of the discriminator used in this experiment. Following your suggestion, we also mentioned the discriminator’s architecture in the Section referring to the Figure in question. Also, we appreciate your feedback on consistency in naming. There was a wrong notation in the legend of Figure 3, where we noted D_fake instead of p_fake. We have revised Figure 3 and its legend to align with the notation established in Section 3.2.1.

4. Given that our generator is unconditioned and receives updates through gradients from the discriminator, it's reasonable to infer that it minimally alters features critical to the discriminator's classification. The generator's primary guidance, apart from an auxiliary objective indicating the original image's class, comes from its goal to change the discriminator’s class and “fakeness” predictions. Therefore, the differences highlighted between 'x' and G(x) are indeed indicative of the regions the discriminator focuses on for its decision-making.

5. Our research primarily focuses on applications such as structural health monitoring or medical diagnosis. In these scenarios, the classes typically involve 'healthy' or 'undamaged' versus 'unhealthy' or 'damaged' states. In such contexts, our method's saliency maps are effectively comparable to segmentation labels. While our approach could extend to diverse class pairs like 'dogs' and 'cars,' the resulting saliency maps would differ and aren't intended for semantic segmentation. The aim of our study is to showcase the effectiveness of our method in highlighting regions relevant to the classification process, using cases where these regions align with segmentation maps for clear demonstration.

We have also adapted the word damaged - in real-damaged vs real-undamaged - to a less ambiguous formulation. Regarding the cross entropy, we have double-checked our notation.