Denoising Diffusion Path: Attribution Noise Reduction with An Auxiliary Diffusion Model

Thank you for appreciating the novelty of the proposed method and better quantitative results compared to existing methods. We would like to address your concerns as follows:

Weakness 1: Presentation in Section 3

Thank you for pointing out these issues. We will provide a more detailed description of integrated gradient (IG) in Section 3 so that it will be more friendly to those readers who are unfamiliar with this area. Specifically, the $x’$ denotes a baseline image, i.e., starting point of the integration path, which can be a black image or a noisy image.

Weakness 2: Lack of introduction of classifier-guided diffusion and whether the method is applicable when no predefined category is given

Thank you for the suggestion. In the current version of the manuscript, we have mentioned the classifier-guided diffusion model [1] in Section 2. To enhance the readability, we will add a brief introduction of the classifier-guided diffusion model in Section 3 in the future version as below: “The classifier-guided reverse process conditioned the $p_{\theta} (x_{t}|x_{t+1})$ on class label $y$, then deriving a sampling strategy of $x_{t-1} \sim \mathcal{N} (\mu + s \Sigma \nabla_{x_{t}} \log p_{\phi}(y|x), \Sigma)$. In this paper, we let the $\mu = \mu_{\theta} (x)$ to ensure the intermediate images are centered at the target image.”

For the second question, when there are no pre-defined categories for target images, the DDPath can also work because the pre-trained diffusion models and classifiers are fixed, and any target image can obtain classification results from the final Softmax output and corresponding saliency maps through back-propagation. However, the only concern is that we cannot guarantee the correctness of the classification result and whether this classification result matches well with the saliency map.

[1] Diffusion Models Beat GANs on Image Synthesis

We sincerely appreciate your time and valuable feedback. We would be grateful to know if our response is satisfactory or if there are any additional points we should address. Looking forward to hear from you.

Thank you for appreciating the interesting idea of the proposed DDPath and comprehensive experiments. We would like to address your concerns as follows:

Weakness 1: About details of the Algorithm 1

First, the input timestep is $t \in [0, \ldots, T-1]$, and $t$ also denotes the steps along the path.

Second, we would like to clarify that the $x’$ is the baseline, and the $x_{t}$ are sampled by the pre-trained diffusion model. Hence, the sampled images $x_{t}$ are aligned with the noise scheduling process, and they consist of the attribution path.

Weakness 2: About the diffusion model

We apologize for the confusion. Here, we clarify the motivation for using a diffusion model as follows. Intuitively, we intend to investigate the denoising power of diffusion models in reducing explanation noise. However, the unconditional diffusion models cannot guarantee that the finally sampled image is consistent with a target image to be explained, furthermore, unconditionally sampled images have NO class or semantic information. Consequently, we applied the classifier-guided diffusion model to generate step images within the attribution path. Specifically, the target image $x$ is an RGB image rather than a noisy image, distinct from the noisy images typically associated with diffusion models, as shown in Algorithm 1. Our implementation focuses on the Reverse process to sample step images, and as outlined in line 8 of Algorithm 1, the sampling mean incorporates class probability $p_{\phi}(y|x_{t})$, aligning our approach with the generative nature of diffusion model.

Question 1: Clarification on diffusion model

Please refer to the responses of Weakness 1 and Weakness 2 for clarification on the diffusion model.

Question 2: Time-complexity comparison

Thanks for pointing out this issue. For traditional attribution methods like IG, the time complexity is $\mathcal{O}(n)$, where $n$ denotes the number of forward-back processes with respect to the target model. When combined with DDPath, for example, the DDPath-IG, the time-complexity increases due to the reverse sampling process of classifier-guided diffusion models, i.e., $\mathcal{O}(m)$, where $m$ is the number of the guided reverse sampling process. Hence, within one attribution step, the time complexity of DDPath-IG is $\mathcal{O}(m) + \mathcal{O}(n)$. In Table R1, we compare the attribution time of IG and DDPath-IG, due to the simple sampling strategy we use, the DDPath-IG requires more time. However, the proposed DDPath has the potential to reduce sampling steps (or the path length) by combining more efficient diffusion sampling strategies with fewer sampling steps. Then, the DDPath will also be more efficient in practice. This is what we are working on for the next version of DDPath.

Table R1 Comparison of attribution time for one image using VGG-19 as the target model

Question 3: About denoising task weights in DTR

Thanks for your valuable comment. The DTR is an interesting idea of creating separate information pathways within a single diffusion model by selectively activating different subsets of channels for each task. Per your concern, we evaluated our DDPath by setting $\rho = 1 - (\frac{t}{T})^{\alpha}$ and $\kappa = (\frac{t}{T})^{\alpha}$ with both $\alpha = 0.5$ and $\alpha = 2$; and we note that the linear scaling used in this paper is equal to that of $\alpha = 1$.

As shown in Table R2, we can see that the DDPath-IG surpasses the baseline IG among different $\alpha$ values, indicating the effectiveness of our DDPath. When $\alpha=2$, the weight of the mean term decreases slowly at the early step, ensuring better preservation of the main object in the images. Besides, the weight of the class-related variance term increases fast at higher steps, enabling better preservation of discriminative information and object details, and this is consistent with the mechanism of task weights in [1]. In contrast, when $\alpha=0.5$, the variance weight increases fast at early steps while the noises are still severe, hence, the class-related information can be affected by the noises while influencing the classification results and attribution qualities. The setting of $\alpha=1$ is a trade-off in our experiments.

For visualizations of different settings, we showcased the corresponding saliency maps in Figure A3 in the attached PDF file. This experiment helps us complete the study of the scaling scheme.

Table R2 Scaling with different $\alpha$ values using VGG-19 target model

[1] Park et al., Denoising Task Routing for Diffusion Models, ICLR 2024.

Thank you for appreciating the smart problem framing, clear method explanation, and thorough testing. We would like to address your concerns as follows:

W 1: Computational cost

We acknowledge that the current DDPath might be computationally inefficient. However, we argue that the consistent quantitative performance gains can offset its increased computational cost. As demonstrated in Figure 4, DDPath exhibits superior denoising capabilities with more sampling steps, a characteristic NOT observed in other methods. Also, we agree with the reviewer that improving the computation efficiency is important in future studies, e.g., combining Consistency models [ICML 2023] and one-step diffusion [CVPR 2024].

In addition, we emphasize that this paper primarily explores the alternative path formulation based on advanced diffusion models, hoping it can inspire more exciting studies on diffusion-based DNN attribution.

W 2: Marginal quantitative improvements

First, while the DDPath did not achieve a substantial performance gain over existing SOTA methods, it consistently demonstrated competitive improvements. In previous studies, the Score-CAM (Ins. 38.6) exhibited a 2.9% increase in insertion AUC when compared to the baseline Grad-CAM (Ins. 35.7) [CVPR 2020]; the CGC [CVPR 2022] improved the Insertion score to 52.16 against the baseline with 48.60 (3.56% improvements). In our paper, the DDPath-IG surpassed the IG, BlurIG, and GIG by 4.2%, 3.1%, and 2.3%, respectively. Additionally, from cases in Figure 3, we can see the superiority of DDPath in terms of Insertion and Deletion curves, demonstrating that DDPath captures more discriminative class-aware information. Hence, these findings suggest that DDPath represents a promising potential for future advancements in DNN attribution.

Second, this paper is the first exploration of integrating diffusion models into DNN attribution, which has shown considerable effectiveness, and practically, since the DDPath can be fused with forward and backward processes of DNNs, it potentially provides new perspectives for explainable training and trustworthy AI.

W 3: Unconvincing visual results

We clarify that we aim to mitigate the noise in saliency maps to ensure that the model's predictions are solely driven by relevant features. This is critical to XAI. In Figure 3, we can see less noise obtained by DDPath, especially in discriminative regions like the heads and wings of birds, or the legs of lobster. In Figures 6 and 7 in the Appendix, we can see more significant visual results, especially those images containing multiple objects.

W 4: Limited to classification models

Application tasks: Current popular studies in XAI, especially the path-based methods, focused on image classification task, so as to this paper. Theoretically, the DDPath does not support the regression or detection tasks due to the step images are generated by classifier-guided diffusion model without regression or detection guidance.

Reliance on pre-trained diffusion models: Recall that our motivation is to make intermediate images along the path to correlate with the original data distribution and then reduce the prediction bias and explanation noise. Since the original diffusion model was trained with no conditions, we applied the classifier-guided diffusion model to make the intermediate images within the path have class information. Fortunately, the diffusion model in [1] was trained on the ImageNet dataset, facilitating our interpretation study on ImageNet. Indeed, for the interpretation of other domains, it is better to have a classifier-guided diffusion model pre-trained on those domains.

Q 1: Image variation and semantics

Taking integrated gradient (IG), a typical attribution method, as an example, the intermediate images are generated by independently injecting image intensities into a black baseline image. This generation process has no correlation with semantic information. For DDPath, in line 8 of Algorithm 1, the sampling mean involves the conditional probability $p_{\phi}(y | x_{t})$, which incorporates class information into generated intermediate images.

Q 2: Improvement in attribution-based applications

One of the typical attribution-based applications is weakly-supervised localization, i.e., localizing objects in the images with only class labels rather than bounding boxes. The pointing game in section 5.4 is also an application that verifies the performance of localizing objects through saliency maps, where the annotated bounding boxes in MS COCO are used as ground-truth labels. The results in Table 2 (of the paper) show improvements obtained by DDPath.

Q 3: Diffusion model size

To investigate the diffusion model size, we apply the released diffusion models by [1]. Note that these diffusion models of different sizes correspond to different image resolutions, including $64\times 64$, $128 \times 128$, $256 \times 256$, and $512 \times 512$. The visualization results can be found in Figure A1 in the attached PDF file. We can see that larger models generated larger resolution of saliency maps, and they illustrate more fine-grained details.

Q 4: Attribution for adversarial samples

Thanks for your concern. We applied two approaches to generate adversarial samples, one is the Fast Gradient Sign Attack (FGSM) described by Goodfellow et. al [2], and the other is adding simple Gaussian noise. We compared the results of IG and DDPath-IG in terms of Insertion and Deletion values, these results and the saliency maps are shown in Figure A2 in the attached PDF file. It is interesting that the IG generated saliency maps with degraded quality, while the DDPath-IG are more robust to adversarial samples (FGSM and Gaussian). Additionally, the Insertion and Deletion values of DDPath-IG are superior to IG.

[1] Diffusion Models Beat GANs on Image Synthesis, NeurIPS 2021.

[2] Explaining and Harnessing Adversarial Examples, ICLR 2015.