Training for Stable Explanation for Free

Thanks for your detailed and constructive reviews.

To Weakness 1: The paper’s discussion of explanation robustness is focused on adversarial robustness. However, the explanation robustness can also be affected by other factors, such as distributional shifts [1, 2]. It would be beneficial to discuss the relationship and difference between the proposed method and these methods.

1. We believe that adversarial robustness (AR) and distributional shifts (DS) as highlighted in [1,2] are distinct because of the “perturbation budget” $\epsilon$ between the original and perturbed inputs: $\epsilon$ is intentionally small in AR for stealthiness, but relatively large in DS due to the more substantial variations inherent in out-of-distribution (OOD) samples, making DS and AR incomparable.

Specifically, the goal of AR is to maintain the explanations unchanged even with neglectable input perturbations, aiming for consistency in explanations. In contrast, DS ensures reasonable predictions (and explanations) for OOD samples, where a substantial deviation from the in-distribution samples is expected, naturally leading to different explanations and rankings.

For example, two images of birds (e.g., Figure 2 from [1]), although from the same class, differ significantly in shape and location, leading to varied feature importance across the images. This variability is typical in DS scenarios. Conversely, in AR cases, the visual and explanatory differences are minimal, thereby requiring the explanation rankings to remain consistent.

2. Despite the fundamental differences of $\epsilon$ between AR and DS, the concepts and properties of thickness and R2ET, as detailed in Section 4, are not specified to AR, but broadly applicable to different robustness scenarios. Thus, we can view them in the DS frameworks:

Specifically, the variable $\mathcal{D}$ in Definition 4.1 and 4.2 (definition of thickness) denotes the (unknown) distribution of sample $x'$. The thickness can be defined under DS framework by viewing $x\in\mathcal{X}$ as in-distribution samples, and $x'\in\mathcal{D}$ as out-of-distribution samples. Furthermore, Propositions 4.4 – 4.7 do not rely on specific assumptions about the distribution $\mathcal{D}$ and merely limit the perturbation budget to $\epsilon$. Thus, these propositions hold even if $\mathcal{D}$ represents the out-of-distributions. This adaptability indicates the universal applicability of thickness and R2ET, making them address both AR and DS challenges effectively.

To Weakness 2: While the paper tests various data modalities, the focus is small-scale datasets, such as MNIST. Could you discuss the potential limitations of the R2ET method when applied to real-world, large-scale datasets? An evaluation of the scalability of the proposed method is beneficial.

We want to point out that we also include ROCT image dataset from real-world medical applications (where robust explanation is of critical importance) in our experiments. The dataset consists of images having 771x514 pixels on average, making it comparable in scale to well-known datasets such as ImageNet (469x387), MS COCO (640x480), and CelebA (178x218).

Regarding the scalability of the R2ET method, we analyze the computational efficiency of R2ET, highlighting that it requires only two additional backward propagations. This makes R2ET considerably more efficient (20x faster) than many adversarial training (AT) methods, as well as [1] and [2]. More discussions are disclosed in Lines 810-818. This efficiency suggests that R2ET’s approach to generating robust explanations is scalable to larger, more complex datasets while maintaining computational feasibility.

To Limitation: The discussion of limitations in the paper could be more comprehensive. While the authors mention general applicability issues, they do not delve into specific limitations that might affect the deployment of their method in real-world settings, particularly in scenarios with highly variable and noisy data.

We recognize that the limitations discussed could be expanded to better address scenarios that involve highly variable and noisy data, which fall outside the “small perturbation” scenarios. While the theoretical analyses in Sections 4-5 will still hold across varying distributions and perturbations, affirming that our approach is theoretically sound under unknown perturbations. It points to a valuable direction for future research: testing and potentially adjusting R2ET to ensure robustness and reliability in more diverse and challenging environments.

Thanks for your detailed and constructive reviews.

To Weakness 1: The thickness concept has been used for prediction robustness.

Their fundamental meanings differ: the thickness for prediction robustness measures the distance between two decision boundaries, while explanation thickness qualifies the expected gaps between two features' importance. Furthermore, different from the “boundary thickness” for prediction robustness, our explanation thickness is inherently more complicated to optimize due to the nature of gradient-based explanations being defined as first-order derivative relevant terms. Thus, we try to speed up the computation and avoid adversarial training.

To Weakness 2: Eq. (6) involves evaluating the Hessian matrix and that can be quite time consuming, compared to WD that does not need the Hessian matrix.

While Eq. (6) requires calculating the Hessian norm, we address the potential computation burden by the finite difference. It allows us to estimate the Hessian norm effectively with only two additional backward propagations. Thus, compared to Vanilla and WD, R2ET remains competitive in terms of computations. R2ET is much more time-saving comparing many existing baselines, such as about 20x faster than adversarial training. More discussions concerning time complexity are in Lines 810-818.

To Question 1: In Figure 2, why the correlation indicates that thickness is a better metric of explanation robustness? More details are needed.

The left subfigure demonstrates a high correlation between the sample’s thickness and the attacker’s required budget to manipulate the rankings. This high correlation signifies that samples with greater thickness demand a larger attack budget for a successful manipulation, thereby justifying that thickness serves as a metric for evaluating explanation robustness.

Conversely, the right subfigure shows a low correlation between the Hessian norm of samples and the required attack budget. It indicates that the difficulty of manipulating explanations remains similar across samples, regardless of whether they have a large or small Hessian norm. This low correlation suggests that the Hessian norm is not a reliable indicator of explanation robustness.

To Question 2: When applying R2ET to other explanation methods, do you need to modify the R2ET algorithm? In other words, how easy it is to apply R2ET to a broader spectrum of explanation methods?

R2ET is general to a range of gradient-based explanation methods—including Gradient (Grad), Gradient*Input, SmoothGrad, and Integrated Gradients (IG)—without any modifications given theoretical analysis (see Appendix A.1.2) and experimental outcomes (Section 6).

In the future, we aim to extend R2ET to a broader spectrum of explanation methods beyond gradient-based ones. Given the objective of R2ET that directly maximizes the gaps between feature importances and minimizes the Hessian norm, R2ET should also perform effectively with other explanation methods.

I will keep my score after reviewing other referees' comments and the manuscript.

Thanks for your detailed and constructive reviews. Due to the character limitation of rebuttal, we have to make the response concise and simple. Feel free to discuss them if more questions/concerns.

To W1:

We agree that P@k may not be the “best” and “direct” metric for capturing ranking stability, and believe that the study of “best” evaluation measurement may be explored in future work, which potentially calls for psychology experts or human-involved surveys.

We evaluate the explanation ranking robustness by P@k because 1) P@k is widely used to evaluate the explanation robustness in existing works [13, 86, 72, 32, 81, 68, 23]. 2) P@k better aligns with human perception. As shown in Figure 1, only the top few features (without their importance score nor relative rankings) will be delivered to end-users in credit card applications.

Furthermore, as discussed in Lines 997-1000, while there are various metrics for ranking similarity—such as cosine similarity, Pearson correlation, and the one in [103]—these can be sensitive to changes in the ordering of less important features, which might not reflect human perception accurately as well. Because they often consider the entire rankings, potentially distorting their usefulness in scenarios where only the most critical features are relevant to decision-making.

To W2:

While we include an analytical discussion on the time complexity of R2ET in the Appendix, we acknowledge that we did not present empirical data comparing the actual training times. Because the training time is influenced by several factors such as convergence rates, hardware specifications, and learning rates. Rather, we compare R2ET with baselines using similar analytical training resources (number of backward propagations), except Exact-H and SSR which are much more costly. Besides, we agree that demonstrating the computational efficiency of R2ET during explanation time could significantly strengthen our argument, as R2ET adds no cost to the existing gradient-based explanation methods.

To W3:

In the revised version, we will clarify that the theoretical analysis applies to gradient (saliency) explanations, as initially introduced in the preliminary. Although the primary results are on saliency explanations, we extend our work by showing that R2ET could generalize to other mentioned explanation methods, such as Grad*Inp, IG and SmoothGrad, by maximizing their lower bounds (Appendix A.1.2). The experimental results further indicate R2ET could fit more explanation methods.

To C1:

Thanks a lot. We agree with your comments, and will partially modify our writings logics in the revised version according to your suggestions.

To C2:

We report “thickness” in Figure 2, Table 3, and Table 5, to show the strong correlation between thickness and robustness. The gaps shown in Figure 8 are not exactly the same as the thickness. We will include more figures for R2ET, and it is supposed to see a significant gap between the $k$-th and $(k+1)$-th features, aligning with R2ET's objectives.

To "small things":

For small thing 1: We agree that humans may resolve higher-level structures and discuss such high-resolution images in the “concept-based” explanation cases, where top-k sub-regions may be of more importance than others. For others, we will modify our writing and make them clearer.

To Q1:

Please refer to the response above.

To Q2:

The connection is the perturbation budget $\epsilon$. Intuitively, Eq. (5) shows that a L-locally Lipschitz model’s local behavior under an $\epsilon$ manipulation can be approximated using $\epsilon$ and its high-order derivative information (such as Hessian). A large $\epsilon$ (for a stronger manipulation) leads to a rigorous restriction to the model (e.g., smaller change rates). Based on Eq. (5), R2ET in Eq. (6) “integrates” $\epsilon$ into $\lambda_2$. On the other hand, Eq. (7) aligns with two objective functions with $\epsilon$ as shown in Lines 629-634.

To Q3:

We have justified the use of P@k in response to W1.

As shown in Table 3, the model with the highest thickness leads to the best P@k. Thus, we attribute the super performance of R2ET\H to their higher thickness than R2ET. In other words, R2ET does not always achieve the best performance may because of the potential mismatch between R2ET’s optimization goal and thickness. Although we agree that there may be a mismatch between P@k and “real ranking consistency”. The later mismatch will be one of our future research directions.

To Q4:

Our analysis and methods in Sections 4 and 5 are independent of the signs of gradients. In other words, the statements and proofs will hold in the general case where saliency does or does not have a sign. In the experiments, we follow existing works to take the magnitude (absolute values) of the gradient.

To Q5:

We study the “worst-case” for attackers in Sec. 5, which provides a guarantee for a successful attack on the explanation rankings. In practice, the attacker could be limited to its attack (perturbation) budget and computation resources (number of attack iterations). Thus, the guaranteed successful attack is defined by the upper bound of the attack iterations.

To limitation:

We will show the assumptions more explicitly as one of the limitations, and discuss the (positive) societal impacts in the revised version.

To Weakness 1: The authors first motivation is that explanations within a small L-p norm can flip all of the rankings and have significantly different "top" features ...

1. There is a misunderstanding of what the L-p norm is measuring and let us clarify that. The “small L-p norm” is not used to measure the magnitude of the explanation itself, but to qualify the distance between an original and its perturbed counterpart. Our study is driven by the observations (Fig. 1) that a “slightly manipulated” explanation (with a small L-p distance to the original explanation) can lead to extremely different feature rankings from the original ranking, altering human perception and decision-making processes. Thus, we study ranking-based metrics aligning with human perception of salient features.

2. The above response is irrelevant to the magnitude of the original explanation (e.g., no matter whether the gradient’s L-p norm is large or small). As detailed in Proposition 4.5, changes in rankings depend on the joint effect of the differences in gradient values and the Hessian matrix, rather than the value of the L-p norm.

3. In our experiments, the classifiers are not necessary to be (extremely) flat since we impose no such constraints and assumptions on the explanations, and R2ET has shown to be effective generally.

To Weakness 2: The authors state that attacking L-p robustness leads to an arms race...

1. Theoretically, Proposition 4.5 delineates that the defender, e.g., R2ET, with proper objective function preserves the explanation rankings for all types of attackers with a given perturbation budget, and thus mitigates the arms races. Specifically, the proposition shows that the attackers cannot alter the models’ explanation ranking if their perturbation budgets is below the critical threshold (RHS of the inequality). The only chance to manipulate the ranking is to increase the attack budgets. On the other hand, the defenders, e.g., R2ET, increase the critical threshold to prevent any types of attackers to mitigate the arms races. In the case of Hessian norm being nearly zero (the denominator goes to zero), it becomes practically impossible for attackers to manipulate rankings successfully. Notice that the proposition is regardless of the “scale” of the models.

2. Experimentally, Table 1 and Appendix B.2.3 report explanation robustness facing attackers with different objectives and different attack strategies, respectively. For all types of attackers, R2ET typically showcases superior robustness under various attacks, indicating the mitigation of the arms races. The datasets and models involved in the experiments are significantly “larger” than those in [89], e.g., the largest feature dimension studied in [89] is 2k in [89] compared to 396k in ours.

In sum, we will clarify these statements more explicitly in the revised version.

PS. Our proposed approach focuses on ranking-based metrics for evaluating explanation robustness, and contrasts to the L-p norm-based metrics used in [89].

To Weakness 3: I think the authors should introduce some toy experiments...

Essentially, R2ET enhances explanation ranking robustness because it maximizes the gaps among features and minimizes Hessian norm, rather than minimizing the magnitude of the gradient. Intuitively, wider gaps (larger absolute distances) indicate more significant distinctions in feature importance. The Hessian norm implies the change rate of the feature importance, and a smaller Hessian norm makes it harder for attackers (or any perturbations) to change the model behavior.

Our empirical evidence shows that the key is not the magnitudes of gradients, but their differences. Specifically, in Lines 935-943 (Figures 8 and Table 7) of the appendix, we take the top-2 case of models in Bank and COMPAS in Figure 8 and Table 7 as an example: the model for Bank showcases a smaller average gradient magnitude (approximately 0.08) yet achieves greater top-2 robustness compared to the one for COMPAS, which has a larger gradient magnitude (around 0.3). It indicates that our approach’s benefits extend beyond conditions where gradient magnitudes are minimal.

To address your suggestion regarding toy experiments, which could indeed further validate and clarify the benefits of R2ET. We will consider adding these experiments in the revised version.

To Question: It is not impossible that my main three points in the weaknesses section stem from a misunderstanding. Do the authors think that I have missed something or misrepresented any of their points?

The most significant misunderstanding is the role of L-p norm. We do not focus on the norm of explanation, e.g., $|I(x)|$, nor small norm explanations. Instead, we investigate whether the L-p distance between the original explanation and its perturbed counterpart, e.g., $|I(x)-I(x’)|$, can serve as a suitable explanation robustness measurement.

The reviewer seems to misunderstand that we study and restrict the L-p norm of explanation, and doubts the motivation for studying gradient-based explanations with a small L-p norm. However, as Figure 1 shows, our core argument is that even a seemingly minor change (evaluated by L-p norm) in explanations could lead to extremely different explanation rankings (compared to their original explanation ranking). It underscores the necessity for a novel ranking-based approach to assess explanation robustness. The magnitude of the gradient is not our main focus, and will only marginally impact our theoretical and experimental conclusions.

We would appreciate it if the reviewer could diminish the misunderstanding and re-read the manuscript concerning our response. It would be even better if the reviewer could give more feedback and raise the scores.