Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution

W1(a): Thanks to the reviewer for the suggestion. We would like to clarify that this view is one of the motivations of our paper. In lines 194-197, we explained that the attribution-based OOD detection method such as GAIA is developed based on the baseline selection of x'=0. GAIA checks OOD samples by counting the distribution of non-zero attribution gradients, so the highly OOD nature represented by non-zero attribution gradients has been proven. However, using the black image with x=0 as the baseline will make it difficult to retain the original semantic information of the sample when attributing, and it is easily disturbed by noise, which will cause deviations when counting non-zero attribution gradients, reducing the accuracy of OOD detection. Adversarial samples can retain the semantic information of samples while introducing minimal perturbations, so we designed an adversarial attribution-based OOD detection method by layer splitting and attribution integration, corresponding to the pseudo code of lines 409-414. It is worth mentioning that since our method is a whole, we did not split the algorithm for verification, and extensive experiments, especially the performance on the ImageNet dataset, have proved the effectiveness of our algorithm, that is, it can more accurately count the distribution of non-zero attribution gradients and thus improve the OOD detection effect.

W1(b): Thanks to the reviewer for the suggestion. We would like to clarify that the first two contributions summarized in the introduction are iterative, and the first contribution (Introducing adversarial examples in OOD detection) paves the way for the second contribution (Layer-splitting technology and integration using adversarial attribution). They are a whole, which can be seen from our pseudo code. So the ablation cannot be done separately.

W2: The purpose of adversarial attacks is to detect the stability around the input sample points, especially in the case of noise. As we mentioned in the response to Weakness 1 (a), GAIA uses x=0 as the baseline. For different datasets, this choice is often complex and ad-hoc, and x=0 is not suitable for all datasets. Adversarial attacks can use existing samples to generate adversarial samples with semantic similarity to the input point samples through a small number of iterations, which is more suitable for different datasets and is not affected by differences in dataset distribution.

W3: We would like to clarify that this task is beyond the scope of OOD detection and belongs to a different setting. In fact, the definitions of adversarial samples and OOD samples are also different, as can be seen in lines 143-157 and 210-234. The current methods for distinguishing adversarial examples from clean examples are common in adversarial defense, which is not relevant to the task of this article. OOD detection is more focused on identifying out-of-distribution data, that is, detecting whether the test sample comes from outside the distribution of the training data, so as to avoid the model from making unreliable predictions on unfamiliar inputs. Adversarial sample detection targets adversarial samples generated by adversarial attacks that are disguised as within the distribution. We believe that the question raised by the reviewer may be feasible in theory, but it needs to be further verified in combination with the nature of the adversarial attack.

W4: Thanks to the reviewer for the insight into our paper, we agree that an ideal robust system should be able to handle all inputs, including adversarial samples, and not just reject adversarial samples. However, OOD detection is still indispensable in safety-critical applications such as autonomous driving, medical care, finance and other fields. As an additional safety mechanism, OOD detection ensures that inputs that differ significantly from the training distribution do not lead to potentially dangerous decisions. Although rejection-based adversarial defense methods are being revisited, OOD detection can be used as an auxiliary mechanism to identify unknown distributions and help achieve robust processing. Therefore, we believe that OOD detection can still play a significant role in scenarios where system reliability, security, and adaptability to novel data are crucial.

W5: Thanks to the reviewer's suggestion, we provide a time consumption comparison between our method and other baselines, with the evaluation indicator FPS, that is, the number of image frames generated per second. It can be seen that on the ImageNet dataset, although our method is slightly slower than GAIA, it has achieved a significant performance improvement. For Rankfeat, our method can not only achieve faster running efficiency but also perform more accurate OOD detection. Therefore, we believe that the running cost is an acceptable price.

Thanks for the responses. Some of my concerns (W4, W5) have been addressed, but I still have concerns (W1, W2, W3) following:

For response 1:
Thanks for the further explanation, and I have noted the related descriptions in the original manuscript. However, my concern is that the existing experiments do not effectively support the claims in the paper, and some key experimental results are missing. Hence, in my previous review, I would like to see more experiments specifically considering non-zero gradient behaviors and ablation studies. Also, I believe that the current architecture allows for ablation studies, such as using a) a baseline model without any changes, b) a model without layer-splitting technology, and c) the final proposed model. If there is any error in my understanding, please point it out.

For response 2&3:
To better discuss, I further summarize my two questions: First, the model is trained with adversarial examples but is not applied to tasks with adversarial attacks. Instead, it is used for OOD detection in traditional, non-attack scenarios.
A natural question arises: Why use adversarial examples for training? Is it feasible to train with noisy examples (not adversarial perturbations, but common corruptions)? And what are the differences between the two? The second question is, why has not the proposed model trained with adversarial examples been applied to adversarial detection tasks, e.g., treating adversarial examples as a special case of OOD?
Your responses partially addressed my questions. However, the original manuscript introduces the concept of adversarial examples in many places, yet the final goal is completely unrelated to adversarial tasks. I believe this somewhat reduces the readability of the paper.

In summary, the W2&3 are not the main reason for my borderline rejection of the paper, but I strongly recommend that the authors clarify these two concepts and their corresponding analyses more clearly in the paper. More importantly, if the authors can further provide the experiments mentioned in W1 and more discussion, I am willing to increase my score.

W1: We thank the reviewer for the valuable comment. We would like to clarify that the insignificant improvement of our method on CIFAR100 does not mean limited effect, but means that our method can achieve the same or even slightly better performance than GAIA on small datasets. Besides, we would like to emphasize that our approach demonstrates significant improvements on the larger-scale ImageNet dataset. This distinction highlights the strength of our method in addressing the challenges of OOD detection in large-scale environments, which is a critical focus of our work. We also want to clarify that current OOD detection metrics such as FPR95 and AUROC have already achieved promising performance across many benchmark datasets. However, our approach prioritizes robustness and reliability in large-scale scenarios like ImageNet, where these challenges become more pronounced.

W2: We thank the reviewer for the valuable comment. We hope that the following clarifications can alleviate the reviewer's doubts. The x and y in Figure 2 represent the attribution gradient values ​​and frequency values ​​in the frequency histogram, respectively. The GAIA frequency histogram above represents the attribution gradient distribution before using GAIA for OOD detection. It can be seen that it is difficult to distinguish which value of the attribution gradient represents the OOD sample. After performing GAIA, as shown in the frequency histogram below, the OOD samples represented by non-zero attribution gradients can be distinguished. For our method, by performing multiple adversarial attacks to analyze the feature distribution shifts from ID adversarial samples to OOD input samples, we can progressively identify high-confidence non-zero gradients, thereby obtaining the true explanation pattern representations denoted by the shaded regions.

W3: Thank you for the reviewer's valuable suggestion. We would like to clarify that the first two contributions summarized in the introduction are iterative, and the first contribution paves the way for the second contribution. They are a whole, which can be seen from our pseudo code. So the ablation cannot be done separately.

Thank you for your in-depth review and feedback on our work. Regarding the limited improvement in AUROC you mentioned, we would like to explain via the perspective of consistency of performance improvement with a horizontal view:

Although our performance improvement is limited in absolute terms compared to the baseline, our experimental results show that our method has achieved consistent improvement in comparison with each baseline method. This comprehensive and consistent improvement fully demonstrates the robustness of our method. Especially on complex large-scale datasets such as ImageNet, while it is very difficult to maintain such consistent improvement across various benchmarking methods, our results technically reflect its capability to be a universal and robust method across the board.

Dear reviewer rQF8,

Thank you again for your thoughtful feedback on our work and for giving us the opportunity to address your comments. In our previous rebuttal, we gave examples to illustrate the importance of OOD detection in real-world tasks such as finance, network security, and medical diagnosis. In addition, to specifically illustrate the effectiveness of our method, we conducted experiments on the iNaturalist dataset and saved the detected OOD samples in the provided anonymous link.

iNaturalist, as a dataset containing 10,000 animal and plant samples, we can detect about 100 more OOD samples than the SOTA baseline GAIA, and the performance gap will be larger if it is extended to a larger dataset. Interestingly, we found that these OOD samples mainly belong to the "plant" category. This phenomenon shows that our method has excellent performance in tasks such as agricultural anomaly monitoring. Since it is necessary to accurately distinguish between crop and non-crop plants (such as weeds or diseased plants), the detected OOD samples may be unknown diseases, abnormal weeds, or unregistered plants, providing support for precision agriculture. Expanding to environmental monitoring and disaster warning tasks, detecting abnormal vegetation phenomena in the environment can also provide early warning signals, such as the impact of environmental pollution and climate change on plant communities.

In addition to the comparative experiments, we also conducted ablation experiments on the first two contributions of our method, namely the adversarial module and the layer splitting module, to verify the effectiveness of our method.

Table 1. OOD Detection Method: Ours (Non-adversarial + layer splitting)

Table 2. OOD Detection Method: Ours (Adversarial + Non-layer splitting)

Table 3. Detection Method: Ours (Non-adversarial + Non-layer splitting)

It can be seen that compared with Table 1 of Non-adversarial + layer splitting, Table 3 (Non-adversarial + Non-layer splitting) achieved a performance reduction of 16.32% and 8.05% on FPR95 and AUROC, respectively, and performed worse on each dataset, which shows the effectiveness of the layer splitting module. Compared with Table 2 of adversarial + Non-layer splitting, Table 3 (Non-adversarial + Non-layer splitting) achieved a performance reduction of 10.37% and 5.17% on FPR95 and AUROC, respectively, and performed worse on other datasets except the FPR95 indicator of Textures, which strongly shows the effectiveness of the adversarial module.

We sincerely hope that our above response has adequately addressed the reviewer’s concerns. We would be truly grateful if the reviewer could consider the possibility of a score adjustment.

Dear Reviewer rQF8,

Thanks for rasing the socre and your feedback is of great significance to our work! We promise to include the ablation study in the revised manuscript. Regarding the out-of-distribution (OOD) detection task for plants, we are happy to share the relevant experimental data and provide further explanations.

Firstly, we selected the ImageNet dataset as the in-distribution (ID) dataset. To perform the plant OOD detection task, we chose the iNaturalist dataset as the OOD dataset. Specifically, we manually selected 110 plant categories from the iNaturalist dataset and randomly sampled 10,000 images for these categories. The related experimental codes and results can be found in our anonymous GitHub repository (https://anonymous.4open.science/r/S-I-F6F7/additional_OOD_samples/). The entire work is fully open-source.

The detected OOD samples are stored in the "additional_OOD_samples" folder. Interestingly, the OOD samples detected from the iNaturalist dataset are all plant species that are not present in the ImageNet dataset, demonstrating the effectiveness of our method in plant OOD detection tasks. This result can be further explained by the dataset characteristics: ImageNet is a general-purpose image classification dataset, while iNaturalist focuses on biodiversity and covers a wide range of fine-grained species. For certain visually similar plant categories, such as the Violet category in ImageNet and the Viola sororia category in iNaturalist, conventional OOD detection methods like MSP, ODIN, and GradNorm struggle to distinguish them and often classify OOD samples as ID categories. However, our method can accurately distinguish fine-grained plant species, as evidenced in the additional_OOD_samples folder.

We provide the following example for the reviewers’ reference:

Detection of angiosperms: The following link shows an OOD sample detected from the iNaturalist dataset, belonging to the Lotus corniculatus category (https://anonymous.4open.science/r/S-I-F6F7/additional_OOD_samples/b51107eaf0608d345a265f623c776706.jpg). The corresponding ID sample from the ImageNet dataset, belonging to the Rapeseed category, is available at this link: (https://anonymous.4open.science/r/S-I-F6F7/ImageNet_rapeseed/1.png).

By comparing these two images, it becomes evident that the visual differences between them are minimal, making it challenging for humans to distinguish visually similar plant species. However, our method effectively identifies fine-grained plant species, which has significant real-world implications, such as discovering new plant species or protecting endangered plant populations.

Dear Reviewer rQF8

We hope this message finds you well. We are writing to kindly follow up on our recent response to your review comments regarding the experimental details. As the rebuttal deadline for ICLR is today, we wanted to ensure that you had the opportunity to review the additional information we provided.

We deeply value your constructive feedback, and we believe the details we shared address your concerns and could enhance your assessment of our work. Please let us know if there are any further clarifications or additional information you would need from us before the deadline.

Thank you for your time and effort in reviewing our submission.

Best regards,

Authors of Submission 9432

Dear reviewer rQF8,

Thank you for your reply. Under the experimental setup we described, we conducted a comprehensive evaluation of various methods in terms of the FPR95 and AUROC metrics. The results are summarized in the table below:

As shown in the table, our method achieves the best performance across both metrics, with the lowest FPR95 (28.59) and the highest AUROC (93.67). Compared to the strongest baseline, GAIA, our method reduces FPR95 by 0.90 and improves AUROC by 0.16. These results demonstrate the effectiveness of our approach in the plant OOD detection task, showcasing its potential for practical applications.For traditional methods such as MSP, ODIN, and GradNorm, our method demonstrates more significant improvements in both FPR95 and AUROC metrics. Additionally, we have provided an anonymous link containing one OOD image from the iNaturalist dataset and one ID image from the ImageNet dataset. The experimental results show that our method can effectively identify the finer-grained OOD category from a set of visually similar plant images and correctly classify it as the OOD sample.

We hope these results address your concerns and further validate the contributions of our work. If this resolves your concerns, we would kindly request that you could consider raising the score.

W1: We appreciate the reviewer’s observation regarding the performance. While it is true that our method achieves comparable results to GAIA on CIFAR-100, we would like to emphasize that our approach demonstrates significant improvements on the larger-scale ImageNet dataset. This distinction highlights the strength of our method in addressing the challenges of OOD detection in large-scale environments, which is a critical focus of our work. We also want to clarify that current OOD detection metrics such as FPR95 and AUROC have already achieved promising performance across many benchmark datasets. However, our approach prioritizes robustness and reliability in large-scale scenarios like ImageNet, where these challenges become more pronounced. In addition, we would like to clarify that the arguments pointed out by the reviewers, such as lines 202 and 311, are not our assumptions, but in fact they are based on evidence. For example, for line 202, we first explain at the beginning of the paragraph that gradient attribution-based OOD detection methods like GAIA are based on baseline selection with x'=0 (this is pointed out in GAIA), and the current attribution methods that use adversarial samples as baseline selection (such as Pan et al., 2021; Zhu et al., 2024b;a, which we cited) have been shown to achieve SOTA attribution performance. And the baseline selection of x'=0 in GAIA does not take into account the scenario of adversarial perturbations. We are the first to apply the concept of adversarial attribution to OOD detection. This is one of our contributions and we have added the correct reference in line 205.

For line 311, this is our logical reasoning based on the fact that OOD samples will be given overconfident predictions (see the description of Figure 2 in GAIA), not our assumption. We have cited GAIA in line 269 and explained the argument that "OOD samples typically exhibit overconfident predictions". Since the label of the adversarial sample belongs to the label set of the input sample, if the input sample is an ID sample, according to the definition of the OOD sample, the adversarial sample is obviously still an ID sample. Therefore, we believe in Proof 2 that $f (x_{adv} ; \theta)$ will not show overly confidence. As for the reviewer's statement that "Proof 2 appears to be based on the assumption that $x_{adv}$ of $x_{out}$ will exhibit overly confidence.", it is obvious that if the input sample is an OOD sample, according to this logic, $f (x_{adv} ; \theta)$ will show overly confidence. In short, this is our logical reasoning based on the viewpoint that GAIA has proved, not a temporary assumption we made.

Q1: As we answered in Weakness, GAIA has already explained the argument that OOD samples will be given Overconfident Prediction in Figure 2 of the paper. In addition, GAIA also cited the work [1] and [2] in related work to show the argument that "Networks tend to display overconfident softmax scores when predicting OOD inputs". We would like to clarify that we need this proven argument as the basis for our proof, and we have cited this argument in line 269. The concept of "overly confident" does not require additional definition, and its definition or not will not affect the reasoning of our paper.

[1] Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 427–436, 2015.

[2] Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 41–50, 2019.

Q2: The OOD score τ in the pseudocode is the score that we need the algorithm to return to determine whether the input sample is an OOD sample. This score is what we need in the end. The higher the score, the higher the possibility that the input sample is OOD. Therefore, it only needs to be returned at the end of the pseudocode, just like the confidence score representing the predicted category will be returned at the end of the binary classification task.

Q3: We would like to clarify that Eq. 1 is only an explanation of the ID and OOD sample classification method in our problem definition. Here we refer to the problem definition in GAIA and compare the ID and OOD sample classification to a binary classification task. Ω(X) represents the classification method, such as GAIA or Rankfeat baselines, and ξ is the threshold of the binary classification task, which can be adjusted by the developer. Therefore, the discussion here is just to give readers who are not familiar with or do not understand the OOD detection task a mathematical understanding. Whether Ω(X) and ξ are mentioned later does not affect the introduction of our method.

Dear Reviewer bTVY,

We hope this message finds you well. We are writing to kindly follow up regarding our rebuttal to your comments on our submission.

We have made our best effort to address the concerns you raised, and we believe our responses provide clarity and additional insights that may help in your evaluation. As the rebuttal phase concludes today, we wanted to check if there is any additional information or clarification we could provide to further assist your review.

Thank you very much for your time and effort in reviewing our work. We sincerely appreciate your feedback and consideration.

Best regards,

Authors of Submission 9432

W1: We thank the reviewer for the valuable comments. We would like to clarify that the insignificant improvement of our method on CIFAR100 does not mean limited effect, but means that our method can achieve the same or even slightly better performance than GAIA on small datasets. Besides, we would like to emphasize that our approach demonstrates significant improvements on the larger-scale ImageNet dataset. This distinction highlights the strength of our method in addressing the challenges of OOD detection in large-scale environments, which is a critical focus of our work. We also want to clarify that current OOD detection metrics such as FPR95 and AUROC have already achieved promising performance across many benchmark datasets. However, our approach prioritizes robustness and reliability in large-scale scenarios like ImageNet, where these challenges become more pronounced.

W2: We thank the reviewer for the valuable comments. In fact, we want to clarify that each model can be layered, which is not a problem and will not affect its scalability. In addition, when GAIA calculates the gradient, it also calculates the corresponding gradient for each layer. Our layered operation is not much more complicated than GAIA. We are applicable to all scenarios where GAIA is applicable, and the gradients of these layers can also be split.

Q1: We thank the reviewer for the valuable comments. We would like to clarify that the introduction of adversarial attack in this paper does not affect AI security. The purpose of introducing adversarial samples is to create a data point that deviates a lot from the current input point to improve the accuracy of OOD detection and increase the adaptability of the model when it is perturbed by adversarial perturbations. In addition, according to the definition of OOD samples, adversarial samples are still ID samples, so no additional OOD samples will be generated to pollute the dataset.

Dear Reviewer 8iu4,

Thank you for your thoughtful review and the valuable feedback on our submission. We greatly appreciate the time and effort you’ve dedicated to evaluating our work.

We have provided detailed responses to address your comments and would be grateful if you could kindly consider them before the rebuttal phase ends today. Please let us know if there is anything further we can clarify！Thank you again for your support and consideration.

Best regards,

Authors of Submission 9432