Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution

Theoretical Claims:

We appreciate the reviewer's suggestions. Our mathematical proofs are all grounded in theoretical foundations. Regarding the reviewer's concern about "OOD samples being overconfident in prediction," we have cited (Nguyen et al., 2015; Hein et al., 2019) on lines 161-164 in our manuscript to support this claim. This perspective is also accepted by GAIA, as illustrated in Figure 1 of the GAIA paper. We hope our response has addressed the reviewer's concerns.

Weaknesses:

1. We appreciate the reviewer’s suggestion. Indeed, replacing "noise" with "perturbation" better reflects the deliberately manipulated nature of adversarial attacks. Regarding Equation 3, it can be extended as an untargeted BIM (Basic Iterative Method) attack, meaning that the adversarial example's label differs from the original label and does not need to be manipulated to a specific category. Notably, the difference between BIM and FGSM lies in the iterative process: FGSM generates adversarial examples with a single-step update, whereas BIM performs multiple iterations and projects the perturbations within a predefined bound to control the magnitude of the disturbance. To help readers understand the concept of adversarial attacks (especially BIM), we cited Kurakin et al. (2018) [1] as a reference on line 190. But regarding the understanding of Equation 3, we find the reviewer’s suggestion very constructive, and we will cite [1] at Equation 3 to further support the explanation.

[1] Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018). Adversarial examples in the physical world. In Artificial intelligence safety and security (pp. 99-112). Chapman and Hall/CRC.

2. We appreciate the reviewer’s suggestion. For the adversarial attack hyperparameters in Equation 3, we used a learning rate $\eta$ of 0.001 and a fixed number of attack step $T=2$. For the parameter $\epsilon$ in Equation 4, since it is jointly controlled by the learning rate and the attack step, and we have fixed the number of attack step in our experiments, we conduct an ablation study on the hyperparameter learning rate $\eta$ to investigate the impact of different perturbations on our method. For details on the ablation experiment, please refer to Q3 in Questions For Authors.

3. Please refer to Q3 in Questions For Authors

4. We referred to the definition of OOD detection in GAIA [2] to propose our Equation 1. Indeed, we highly appreciate the reviewer’s constructive suggestions. As an open-source and comprehensive OOD detection framework, the OpenOOD has made significant contributions to OOD research. We will refine our manuscript by incorporating the definition of OOD detection at Equation 1 in OpenOOD.

[2] Chen, J., Li, J., Qu, X., Wang, J., Wan, J., & Xiao, J. (2023). Gaia: Delving into gradient-based attribution abnormality for out-of-distribution detection. Advances in Neural Information Processing Systems, 36, 79946-79958.

Other Comments Or Suggestions:

1&2. We sincerely thank the reviewer for pointing out the errors, and we have addressed this issue.

Questions For Authors:

Q1. For the selection of post-hoc detectors, we referred to the baselines set in GAIA, which includes gradient-based OOD detection methods such as GAIA and GradNorm, as well as feature representation-based methods like Rankfeat and React, and output-based methods such as MSP and ODIN. Regarding the inclusion of additional comparison methods, we are open to the reviewer’s suggestions and are willing to compare our method with the latest 2024 baseline "SCALE", the results are in the link: https://anonymous.4open.science/r/S-I-F6F7/rebuttal/additional_exp_result.md. It can be seen that our method has better performance than SCALE on each OOD dataset. Using CIFAR100 as the benchmark and a learning rate of 0.001, our method shows a 65.32% improvement in FPR95 and a 28.37% improvement in AUROC compared to SCALE.

Q2. In fact, this attack is essentially a BIM attack. Please refer to our response in Weaknesses 1, which we hope will resolve the reviewer's concerns.

Q3. We appreciate the reviewer's suggestions and are happy to conduct an ablation study on the learning rate to explore the impact of different perturbation magnitudes on our method. In the Table at link: https://anonymous.4open.science/r/S-I-F6F7/rebuttal/additional_exp_result.md, we vary the learning rate in the range {0.0005, 0.001, 0.0015, 0.002}. It can be seen that our method consistently achieves high AUROC and low FPR95 scores across all OOD datasets under different settings. The performance remains stable, with only marginal fluctuations observed. This demonstrates that our method is robust to learning rate selection and can generalize well without requiring sensitive hyperparameter tuning.

We sincerely hope that our response has addressed the reviewer's concerns. We would be truly grateful if the reviewer would consider adjusting the score accordingly.

We thank the reviewer for the valuable suggestions. We provide the following responses to the questions in “Questions For Authors”.

Q1: We would like to clarify that the insignificant improvement of our method on CIFAR100 does not mean limited effect, but means that our method can achieve the same or even slightly better performance than GAIA on small datasets. Besides, we would like to emphasize that our approach demonstrates significant improvements on the larger-scale ImageNet dataset. This distinction highlights the strength of our method in addressing the challenges of OOD detection in large-scale environments, which is a critical focus of our work. We also want to clarify that current OOD detection metrics such as FPR95 and AUROC have already achieved promising performance across many benchmark datasets. However, our approach prioritizes robustness and reliability in large-scale scenarios like ImageNet, where these challenges become more pronounced.

We are pleased to highlight the significance of our method through the practical task of plant OOD detection. Firstly, we selected the ImageNet dataset as the in-distribution (ID) dataset. To perform the plant OOD detection task, we chose the iNaturalist dataset as the OOD dataset. Specifically, we manually selected 110 plant categories from the iNaturalist dataset and randomly sampled 10,000 images for these categories. The related experimental codes and results can be found in our anonymous GitHub repository (https://anonymous.4open.science/r/S-I-F6F7/additional_OOD_samples/). The entire work is fully open-source.

The detected OOD samples are stored in the "additional_OOD_samples" folder. Interestingly, the OOD samples detected from the iNaturalist dataset are all plant species that are not present in the ImageNet dataset, demonstrating the effectiveness of our method in plant OOD detection tasks. This result can be further explained by the dataset characteristics: ImageNet is a general-purpose image classification dataset, while iNaturalist focuses on biodiversity and covers a wide range of fine-grained species. For certain visually similar plant categories, such as the Violet category in ImageNet and the Viola sororia category in iNaturalist, conventional OOD detection methods like MSP, ODIN, and GradNorm struggle to distinguish them and often classify OOD samples as ID categories. However, our method can accurately distinguish fine-grained plant species, as evidenced in the additional_OOD_samples folder. We provide the following example for the reviewers’ reference:

Detection of angiosperms: The following link shows an OOD sample detected from the iNaturalist dataset, belonging to the Lotus corniculatus category (https://anonymous.4open.science/r/S-I-F6F7/additional_OOD_samples/b51107eaf0608d345a265f623c776706.jpg). The corresponding ID sample from the ImageNet dataset, belonging to the Rapeseed category, is available at this link: (https://anonymous.4open.science/r/S-I-F6F7/ImageNet_rapeseed/1.png). By comparing these two images, it becomes evident that the visual differences between them are minimal, making it challenging for humans to distinguish visually similar plant species. However, our method effectively identifies fine-grained plant species, which has significant real-world implications, such as discovering new plant species or protecting endangered plant populations.

Regarding the inclusion of additional comparison methods, we are open to the reviewer’s suggestions and are willing to compare our method with the latest 2024 baseline "SCALE". It can be seen that our method has significantly better performance than SCALE on each OOD dataset. For more details, please refer to Q1 in Questions For Authors at Reviewer 5PTw

Q2: We sincerely express that we have provided ablation experiments of the adversarial attack module and layer splitting module in Part B of the supplementary material. By comparing Table 4 and Table 5, it can be proved that adversarial attack module plays an important role. Similarly, by comparing the ablation experiments in Table 3 and Table 5, it can be proved that the layer splitting module also plays an important role. We hope our response can address the reviewer's concerns.

Q3: We kindly acknowledge that we have provided a comparison of computational costs in Part C of the Supplementary Materials. We measure the efficiency of our methods using the frame per second metric in Table 6. It can be observed that on the ImageNet dataset, our method is slightly slower than GAIA. However, from the perspective of the main experimental metrics, FPR95 and AUROC, on the ImageNet dataset, our method achieves significant performance improvement. Compared to the computationally similar RankFeat method, our approach not only achieves faster runtime efficiency but also provides more accurate OOD detection. Therefore, we believe the computational cost of our algorithm is an acceptable trade-off.

Other Strengths And Weaknesses

Weakness1：We thank the reviewer for the valuable comment. We would like to clarify that the insignificant improvement of our method on CIFAR100 does not mean limited effect, but means that our method can achieve the same or even slightly better performance than GAIA on small datasets. Besides, we would like to emphasize that our approach demonstrates significant improvements on the larger-scale ImageNet dataset. This distinction highlights the strength of our method in addressing the challenges of OOD detection in large-scale environments, which is a critical focus of our work. We also want to clarify that current OOD detection metrics such as FPR95 and AUROC have already achieved promising performance across many benchmark datasets. However, our approach prioritizes robustness and reliability in large-scale scenarios like ImageNet, where these challenges become more pronounced.

We are pleased to highlight the significance of our method through the practical task of plant OOD detection. For details, please see our response to Reviewer SPNg in Q1.

Weakness2：Thanks to the reviewer for the suggestion. We would like to clarify that this view is one of the motivations of our paper. In lines 194-197, we explained that the attribution-based OOD detection method such as GAIA is developed based on the baseline selection of $x'=0$. GAIA checks OOD samples by counting the distribution of non-zero attribution gradients, so the highly OOD nature represented by non-zero attribution gradients has been proven. However, using the black image with x=0 as the baseline will make it difficult to retain the original semantic information of the sample when attributing, and it is easily disturbed by noise, which will cause deviations when counting non-zero attribution gradients, reducing the accuracy of OOD detection. Adversarial samples can retain the semantic information of samples while introducing minimal perturbations, so we designed an adversarial attribution-based OOD detection method by layer splitting and attribution integration, corresponding to the pseudo code of lines 409-414. Since non-zero gradient represents a high confidence OOD sample probability (as expressed by GAIA), it is obvious that the accuracy of calculating non-zero gradient will significantly affect the accuracy of OOD detection. This is the starting point for designing our method, and the performance improvement in extensive experiments (especially on large-scale ImageNet) has proven that our method can more accurately calculate the attribution gradient (i.e., we can obtain more accurate non-zero and zero gradient distribution) to obtain better OOD detection performance.

Other Comments Or Suggestions

For the FPR numbers in brackets, we hope that the following explanation can help the reviewer’s understanding. Referring to Table 1, we can see that the backbone models are ResNet34 and WRN40, respectively. Therefore, the FPR numbers in brackets represent the results on the WRN40 model. We promise to revise the text to avoid ambiguity.

Questions For Authors

Q1: Our method can be easily extended to other gradient-based OOD detection methods. By introducing adversarial samples as a baseline, the gradient calculation can be stabilized and noise interference can be reduced. This strategy can be applied to other gradient-based methods (such as GradNorm, GAIA, etc.) to enhance the distinguishability of gradient patterns by generating adversarial samples, thereby improving detection robustness. In addition, layer splitting can alleviate the problem of gradient explosion or cumulative error in deep networks by analyzing the gradient sensitivity of different layers in a hierarchical manner. This module can be independently integrated into other methods, such as decomposing by layer when computing gradient importance and optimizing the detection logic by leveraging the characteristics of different layers (e.g., low-level textures and high-level semantics).

Q2: Regarding the inclusion of additional comparison methods, we are open to the reviewer’s suggestions and are willing to compare our method with the latest 2024 baseline "SCALE" [1], the results are in the link: https://anonymous.4open.science/r/S-I-F6F7/rebuttal/additional_exp_result.md. It can be seen that our method has better performance than SCALE on each OOD dataset. Using CIFAR100 as the benchmark and a learning rate of 0.001, our method shows a 65.32% improvement in FPR95 and a 28.37% improvement in AUROC compared to SCALE.

[1] Xu, K., Chen, R., Franchi, G., & Yao, A. (2023). Scaling for training time and post-hoc out-of-distribution detection enhancement. arXiv preprint arXiv:2310.00227.

Q3: Please see the clarifications in Weakness 2.

We sincerely hope that our response has addressed the reviewer's concerns. We would be truly grateful if the reviewer would consider adjusting the score accordingly.

Claims And Evidence:

1. We sincerely state that in Section B of the supplementary materials, we have provided ablation experiments for the adversarial attack module and the layer splitting module. By comparing Table 4 and Table 5, it can be demonstrated that the adversarial attack module plays a significant role in our method.

2. We would like to point out that the statement, "different network layers focus on different types of features (textures, semantics, etc.)" is both intuitive and theoretically supported, as demonstrated in numerous studies [1][2]. GAIA assumes that all intermediate layers have the same influence on the feature maps, which contradicts the hierarchical feature distribution inherent in neural networks. Compared to GAIA's uniform treatment of all layers during attribution, our proposed layer splitting theory offers an innovative contribution by more precisely capturing the feature sensitivity of different layers. Finally, the ablation experiments shown in Table 3 and Table 5 of the supplementary materials already demonstrate that the layer splitting module plays a crucial role in our method.

[1] Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. ECCV 2014.

[2] Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., & Lipson, H. (2015). Understanding neural networks through deep visualization. ICML Workshop on Deep Learning.

3. We would like to point out that comparing the ablation experiments with and without integration does not align with the principles of discrete path integration. According to the formula in line 282: $A=L\left(f\left(x_T\right)\right)-L\left(f\left(x_0\right)\right)=\sum_{i=0}^{T-1} \frac{\partial L\left(f\left(x_{i}\right)\right)}{\partial x_i}\left(x_{i+1}-x_i\right)$, the attribution result $A$ is expressed as the cumulative sum of gradient changes along each point on the path. If layer-by-layer integration is not used and the attribution is only calculated based on the changes between the start and end points, the gradient variations along the path are ignored. This results in an inaccurate attribution gradient distribution, making it impossible to perform the OOD detection task properly, let alone improve detection accuracy.

4. We sincerely state that in Section C of the supplementary materials, we have provided a comparison of computational costs. We use the frames per second (FPS) metric in Table 6 to evaluate the efficiency of the methods. We believe the computational cost of our algorithm is an acceptable trade-off.

Methods And Evaluation Criteria

Please refer to point 4 in the Claims and Evidence section for the efficiency analysis. We are also pleased to highlight the significance of our method through the practical task of plant OOD detection. For details, please see our response to Reviewer SPNg in Q1.

Theoretical Claims

We would like to clarify that we have provided extensive mathematical proofs as the theoretical foundation for our method (including but not limited to the claims and proofs of Theorem 1 and Theorem 2). We use Equation 3 to ensure that the adversarial perturbation consistently enhances feature sensitivity. Equations 9 and 10 formally derive the layer splitting strategy from a mathematical perspective.

Experimental Designs Or Analyses

The analyses of time efficiency, ablation studies, and real-world OOD cases can be found in the above rebuttals.

Relation To Broader Scientific Literature

We would like to clarify that energy-based OOD scoring has already been used as a baseline for comparison in our paper on line 371, and our method demonstrates significantly better performance. As for Mahalanobis distance-based scoring, since it is an earlier work and it seems that no clear citation link has been provided by the reviewer, we have not included it as a comparison target. We compare our method with the latest 2024 baseline "SCALE". It can be seen that our method has better performance than SCALE on each OOD dataset. For more details, please refer to Q1 in Questions For Authors at Reviewer 5PTw

Essential References Not Discussed

We appreciate the reviewer's addition; however, it seems that no clear citation link of the two references has been provided. If the reviewer could supply the link, we would be happy to discuss these works.

Other Strengths And Weaknesses

The efficiency analysis, comparison experiments, and ablation studies can be found in the above rebuttals.

Other Comments Or Suggestions

We appreciate the reviewer’s suggestions, and we will improve the presentation of the formulas and figures to enhance the clarity of the paper.

Questions For Authors

The time efficiency analysis, comparison experiments, and real-world OOD case analysis can be found in the above rebuttals.

We sincerely hope that our response has addressed the reviewer's concerns. We would be truly grateful if the reviewer would consider adjusting the score accordingly.