Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability

Thank you for the comprehensive review and the constructive feedback.

W1. The qualitative examples in Figure 3 do not clearly demonstrate the superiority of PA-LRP over the AttnLRP baseline. The visualizations appear very similar, which weakens the visual impact of the proposed method.

Our most important result is on attention-level PE, such as Rotary PE (RoPE). In this setting, the PE is integrated at each attention layer, making it significantly more impactful in both the forward and backward passes. In this regime, our empirical results are substantially better, as shown by the large performance gap between our method and AttnLRP in Tables 3 and 5, and by the qualitative examples in Figure 7 (for example, note the blue distracting words attributed by AttnLRP, such as “West,” “Harry,” and “Reyn”). Additionally, our theoretical findings are more relevant in this setting. For example, Theorem 3.3 proves that previously proposed LRP methods are not faithful when positional features are involved. It is also important to highlight that attention-level PEs are widely popular in both LLMs (Qwen, LLama, Pythia and others) and vision models like SAM2 (see line 108 in our paper).

W2 & Q2. Although Tables 1, 2, and 3 report improved quantitative results, the experimental setup lacks detail. In particular, it is unclear how many samples were used in the evaluation, how they were selected, and whether any variance or statistical significance was assessed. The absence of error bars or multiple runs raises concerns about the robustness of the reported improvements…. Q2. For the perturbation and segmentation tests (Tables 1–3), how many samples were included in the evaluation? Were the results obtained from the full test set or a selected subset? Did you run multiple trials, and if so, could you report any measures of variance or statistical signifiance?

We thank the reviewer for the suggestion regarding statistical significance, which we address with dedicated experiments. To ensure the robustness of our results, we report variance across runs and conduct paired t-tests to support statistical significance. The results will appear in the camera-ready version, and we add them to all of the relevant experiments (Tables 1-5). Please see Tables A and B below for perturbation tests in NLP. We demonstrate that the variance is consistently low, and paired t-tests yield small p-values with t-scores aligned with performance trends.

Table A: Table 3 from our paper (perturbations in NLP under the finetuning regime), with added statistical information. The P-values are obtained using paired t-tests compared to our method.

---

Table B: Table 5 from our paper (perturbations in NLP in zero-shot settings), with added statistical information.

Regarding the experimental setup, please note that the XAI experiments are deterministic, so the seed has no impact. This methodology was also not employed in previous work [1–3], including papers accepted to ICML and CVPR. Regarding the number of examples, we thank the reviewer for this comment. The information was mistakenly removed from the paper before submission. We will add the number of examples used to each table. For better clarity, we report the number of examples used in each experiment here: For ARC-Easy, we use 5,200 samples; for Wiki-text, 5,000 examples; and for IMDB, 5,000 examples.

Q1 In Figure 3, the explainability heatmaps generated by PA-LRP and AttnLRP appear visually similar. Could you elaborate on the qualitative differences that PA-LRP captures and explain why these differences might not be visually apparent despite the claimed improvements?

We want to clarify that the impact of PE on attribution is less significant for input-level PE (as used in ViT) than for attention-level PE (primarily employed in LLMs). Therefore, the differences are expected to be relatively minor in ViT. That said, we believe Figure 3 (see analysis in lines 302–323) clearly describes the differences between the explainability heatmaps. For example, see line 310:

The attributed signal derived solely from positional-associated relevance captures unique relationships, exhibiting clearer spatial and structural patterns. In particular, relevance is distributed across the entire object, especially in the snake, bird, and shark examples. In contrast, the baseline method, which does not propagate relevance through PEs, produces a sparser pattern that does not focus on the entire object but instead is highly selective to specific patches. One possible explanation is that positional-associated relevance better captures concepts related to position, structure, order, and 321 broader regions within the image.

W.1. The premise that LRP based explainable methods are most effective model specific XAI method.

While we acknowledge that no single XAI method can be definitively labeled as universally superior, we maintain that LRP shows the most promise for our specific application. Although this conclusion arises from the results of the Attention LRP paper, to support this position, we conducted comparative experiments against established methods including IG, SHAP, and [Leeman at al], which consistently demonstrated LRP's superior performance in our evaluation metrics. Please see Tables A and B mentioned in the response for reviewer jDLx.

W.2.1. Simply appends the positional relevance to that of Attn LRP.

We respectfully disagree that our work is incremental. In the LRP literature, each paper focuses on improving different aspects. For example, [1], presented at ICML, focuses on softmax and LayerNorm, while attnLRP [2], also presented at ICML, focuses on attention layers. Our method revisits the concept of attributing PEs and propagating relevance through PE layers, and demonstrates both empirical and theoretical importance, especially for LLMs such as LLaMA3. We consider this a major advancement, as XAI tools for such models are extremely important for the community.

W.2.2. The interactions between the attention layers and tokens needs to be explained.

Regarding the comment “The interactions between the attention layers and tokens need to be explained,” could you please elaborate? In Section 2.1, we discuss how attention is computed using RoPE and ALiBi, and in Section 3.2, we explain how relevance should be propagated through the RoPE and attention layers.

W.3. In quantitative analysis, positive perturbation tests for DeIT variants shows Attn LRP performing better for Base and Tiny models. How does the removal of relevant pixels changes the positional understanding, eg: it is understandable that removing relevant pixels does changes the prediction performance but how does the relevance is distributed in context of the positional encodings and propagation rules.

The reviewer seems to ask what would be the effect of removing relevant pixels for prediction on the distribution of positional and semantic attribution maps. The question should be asked backwards, how do the positional and semantic attribution maps differ in terms of the order of the most relevant pixels.

The conclusion should be drawn backwards, how does positional and semantic understanding (e.g. attribution maps) affect the resulting order of which pixels are most relevant for classification.

We refer the reviewer to our qualitative results section (line 296), where we discuss the difference between positional and semantic attributions. In general, for a few cases, our approach demonstrates a minor decrease in positive perturbation performance while achieving substantial improvement in negative perturbation performance. These results mainly show that positional attribution facilitates identifying truly important features, but at the cost of giving higher relevancy scores to additional tokens, which may have been critical for the computational flow (for example for storing global information regarding the prediction), but are not by themselves crucial for prediction.

W4. The evaluation is mostly dependent on the perturbation and segmentation based tests. Human based tests might have helped. The quantitative test are mostly acts as proxies.

We respectfully disagree with the reviewer, as there is no definitive and reliable way to evaluate XAI performance using human-based testing.

Q1. One of the questions that arise are the situations where positional embeddings are less informative in the predictive performance for tasks such as classification

Our results strongly suggest that for most situations, positional embeddings are crucial for high-quality XAI performance. PE information is already thoroughly integrated throughout the model's representations and mixed with semantic representations. Attempting to isolate specific conditions where PE attribution might or might not be beneficial goes beyond the paper's scope.

---

References:

[1] XAI for Transformers: Better Explanations through Conservative Propagation. Ali et al. ICML 2022

[2] AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers. Achtibat et al. ICML 24

[3] Transformer Interpretability Beyond Attention Visualization. Chefer et al. CPVR 2021

We thank the reviewer for their detailed comments and suggestions and provide responses to the raised concerns below.

W.1. Limited Scope.

We respectfully disagree with this statement. Transformer explainability, especially for LLMs, is a crucial emerging discipline, with LRP-based methods being at the center of this domain. Moreover, Attn-LRP [1] shows that LRP-based XAI methods consistently outperform other XAI methods such as KernelSHAP, IG, and others. We underline that at least two dozen papers proposing LRP techniques for transformers have been previously published at top-tier conferences, such as ICML and ICLR. Notable examples include [1-3].

To establish LRP's superiority in XAI performance, we conduct a series of experiments using the baselines suggested by the reviewer (Leemann et al, IG, SHAP). The results, presented in Tables A and B, mentioned in the response to reviewer jDLx,, show that our method consistently outperforms the baselines. It is demonstrated that methods which are not LRP-based consistently show inferior performance across tasks. Moreover, our method consistently outperforms Attn-LRP, with no single case in which Attn-LRP achieves the best score. In particular, our approach improves the AU-MSE score in the generation scenario by 14.5% for LLaMA 2–7B, 10.6% for LLaMA 2–7B Quantized, and 51.41% for Tiny-LLaMA.

Additional rows for Table 3 from our paper (perturbations in NLP under the finetuning regime), including variance values (shown after the ± symbol). The table reports paired t-test results comparing each baseline to our method, such that each element in the pair the comparison is made for the classification accuracy, for each sample, across all perturbation steps. Results are based on 5,000 randomly sampled reviews from the IMDB test set. Please see Table A in response to reviewer jDLx.

Additional rows for Table 5 from our paper (perturbations in NLP under the zero-shot regime), including variance values (shown after the ± symbol). For multiple-choice question setting, we utilize the ARC-easy dataset, containing 5200 samples. For next token prediction task setting, we use the Wiki dataset, containing 5000 uniformly selected samples . Please see Table B in the response for reviewer jDLx.

We also note that SHAP is also significantly more computationally involved due to their model-agnostic nature, which requires surrogate modeling and repeated inference, resulting in a total runtime increase of approximately 5 times more in our experiments.

W.2. Motivation for specific rules LRP.

While exploring optimal rules for PE-LRP through theoretical analysis or grid search could further improve our empirical results, we deliberately chose not to perform hyperparameter tuning for our PE-LRP rules in order to neutralize the impact of hyperparameter search.

In our experiments, we followed the same guidelines reported as optimal in AttnLRP, specifically a combination of the $\epsilon$-rule and the $\gamma$-rule. We agree with the reviewer that a broader discussion of alternative relevance propagation rules is warranted. To address this concern, we now include quantitative results on perturbation tests in vision tasks using the alpha-beta rule, demonstrating the significant performance degradation it causes in our setup, which is based on Attn-LRP.

Table C: The table demonstrates that using the settings presented in AttnLRP (epsilon, gamma) provides the optimal performance for both methods. We underline that while the alpha-beta rule manages to achieve higher performance for negative perturbation for our method, it is accompanied by a drastic tradeoff with positive perturbations, which implies that the attribution map is coupled with high level of noise.

W.3. … distinction between positional and semantic importance …

We do not claim to fully disentangle positional relevance from semantic relevance. Rather, our aim is to highlight the existence of a positional relevance component—one that has been largely overlooked but is shown through our analysis to be crucial for achieving strong XAI performance. We first want to clarify that in our experiments, we included a separate setting ("PE Only") as an ablation to better understand and isolate the contribution of position-associated relevance, and in practice, both positional and semantic relevance were computed through the same LRP backward path.

Secondly, we believe that the reviewer oversimplifies the problem. For example, in attention-level PE, such as RoPE (popular in LLMs like LLaMA and Qwen), PE is integrated into all attention layers, and thus cannot be treated as part of the input. Additionally, this type of PE is relative rather than absolute, so simply redefining the input is not feasible. Moreover, without dedicated LRP rules, relevance conservation will be violated, as PE components involve many nonlinearities (for example, RoPE includes trigonometric functions). Our work addresses all these challenges with both theoretical and empirical analysis.

W.4. Some confusion about notation.

Equation 3 presents the input space as defined in previous work, while Equation 4 introduces our reformulation. The intuition is simple: previous methods define the input space using only the word embeddings (a list of vectors in $\mathbb{R}^D$). We argue that this is not precise for Transformers, as PE is also part of the input. In short, a transformer with PE operates over ordered sequences rather than unordered sets, so defining the input space without position does not accurately reflect the problem. Therefore, the input space should be defined as token-position pairs, consisting of both the semantic information representing the words (as in prior work) and the positional information, which we explicitly include in our reformulation ($\forall k \in [K], j \in [L]: P_{j,k}$).

W.5. No measures of dispersion.

We thank the reviewer for the suggestion, which we address with dedicated experiments. To ensure the robustness of our results, we report variance across runs and conduct paired t-tests to support statistical significance. The results will appear in the camera-ready version. See examples below for perturbation tests in NLP. We demonstrate that the variance is consistently low, and paired t-tests yield small p-values with t-scores aligned with performance trends.

Please see Tables A and B mentioned in the response for reviewer jDLx

Q.1: … What … input for plain LRP? Did you backpropagate to E_i or E_i + P_k ...?

We first want to clarify that our main contribution focuses on attention-level PE, not input-level PE. Attention-level PE is widely used in modern LLMs and has a strong impact on the behavior of each attention layer. In this setting, our results are the strongest, and our theoretical analysis stands out (see Lemma 3.3).

Following this perspective, our contribution is to define LRP rules that enable relevance to be propagated through PE layers such as RoPE, ALiBi, and input-level PE.

For input-level PE, which is the simpler case and not our main contribution, we define the input space as the concatenation of $E_i$​ and $P_k$​. We then propagate relevance to both components and sum the positive values along the corresponding feature dimensions.

Q.2: Why can't we just use LRP from the sum E_I + P_K (for standard additive embeddings)?

First, the main part of our work, which is more complex, novel, and effective: our PE-LRP rules for attention-level PE used in LLMs. For input-level PE, simply applying LRP to the sum $E_i + P_k$​ can lead to a loss of relevance, as negative positional relevance may override positive semantic relevance at each coordinate, and vice versa. More importantly, since LRP is designed to propagate relevance all the way to the input space, stopping relevance propagation at the path embeddings in ViT models results in ignoring the pixel-space and producing explanations based only on the patch-space (and ignoring the entire computation of the patch embedding). This contradicts the core principle of LRP, which requires relevance to be propagated to the actual input.

We are grateful for the reviewer’s feedback.

W.1 The empirical experiments are partly inconclusive, given that for some metrics, the baseline wins, and for others, the improvements seem to be marginal.

We respectfully disagree with the reviewer’s assessment and maintain that our empirical results clearly demonstrate the importance of attributing positional components for achieving high XAI performance. In the NLP domain, using attention-level PE, our method consistently outperforms Attn-LRP across all model capacities and evaluation settings. Notably, on the IMDB classification task (Table 3), our approach improves the AU-MSE score in the generation scenario by a substantial margin, ranging from 14.5% to 51.41%.

While we occasionally observe only marginal improvements on certain metrics, these cases are consistently accompanied by significant gains on complementary metrics for the same model capacity. In some perturbation experiments for vision tasks, our method exhibits a slight decrease in positive perturbation performance but consistently achieves substantial improvements in negative perturbation performance—highlighting its overall advantage.

W.1. & Q.1.1 there are no error bars or number of trials reported.

Thank you for pointing out this shortcoming. We will add the number of runs, SD, and paired t-tests to all experiments. Reassuringly, all differences we pointed out in the paper are significant, verified using the paired t-tests. We report p<0.03 figures for our experiments, mostly with p values lower than 10^-4. See, for example, Tables A,B below, which provide the analysis for Tables 3,5 in the paper, respectively.

W.2. … argues that AttnLRP is not conservative unless positional embeddings are attributed. I do not understand why …

The conservation property is fundamental to LRP's interpretability guarantees, ensuring attribution magnitudes remain proportional to the function’s output, and is highly regarded as a crucial factor for achieving faithful explanations [1, 2, 3]. The conservation property by itself is not a consideration for the distinction between considering and not considering positional embeddings, but rather a theoretical justification to explain improvement for empirical results.

W.3. The work claims to present some theoretical evidence that the omission of positional embeddings in attribution gives low fidelity. However, this is not proven, but only an extreme-case example is provided. The claim is also not consistent with the empirical results, which indicate that in some cases, fidelity can be higher when omitting the attribution of positional embeddings.

We respectfully disagree with the claim that our fidelity argument is not proven. It is supported by a counterexample, a well-established and effective proof technique, and follows the same standard used in prior work in the domain (e.g., Chefer, Lemma 3; AttnLRP, Appendix A.2.1). We also do not consider this proof to fall under the category of an “extreme case.” Rather, it reflects the simplest and most controlled setting, demonstrating that even in such basic scenarios, fidelity issues can arise.

We would like to emphasize that employing simplifications to derive theoretical insights is a widely accepted practice in deep learning research. In particular, theoretical analyses of faithfulness in LRP methods (especially in the context of Transformers) are exceptionally rare and inherently difficult to formalize. From this perspective, we view Lemma 3.3 as a meaningful and technically valuable contribution.

With regard to consistency with empirical results, we emphasize that our theoretical claims on faithfulness are solely focused on attention-level PE, which is widely used in LLMs. In the experiments we conducted with attention-level PE (see Tables 3 and 5), the results are highly consistent with our theory. For example, when incorporating our PE-LRP rules into LLaMA models, our approach improves the AU-MSE score in the generation scenario by 14.5% for LLaMA 2-7B, 10.6% for LLaMA 2-7B Quantized, and 51.41% for Tiny-LLaMA. Notably, AttnLRP never outperforms both of our methods (PE-LRP and PE only) in any setting (see Tables 3 and 5) in the attention-level regime.

Q.1.2. .. there might seem to be some cases in which there is no clear benefit from explicitly attributing positional embeddings. ..These points make the results somewhat inconclusive

We clarify that our main results are not inconclusive. Our PE-LRP rules are particularly effective when incorporated in the attention-level, and in those settings (see Tables 3 and 5 in our paper), our method consistently outperforms Attn-LRP, with no single case in which Attn-LRP achieves the best score.

Q.1.3. These points make the results somewhat inconclusive, as the work does not provide any analysis as to when the proposed approach should be used, and when it should not. The qualitative results also do not seem to give a clear difference…

With regard to determining when and why to propagate PE is necessary, our empirical results demonstrate that PE attribution is crucial for explanation quality across the board - the 'when' is always for models with attention-level PE (our NLP experiments) and almost always for Vision, and the 'why' is that PE information is already thoroughly integrated throughout the model's representations and therefore must be attributed. Attempting to isolate specific conditions where PE attribution might or might not be beneficial goes beyond the paper's scope.

Q.2. For Table 1, the work states that the proposed approach lags behind for the positive perturbation, but performs better in the negative perturbation. This seems to suggest that for some setups, it is better to not regard the positional embedding. What are the implications of this observation? How significant are these results?

The reviewer misinterprets the mixed positive/negative perturbation results reported for vision classification task (Table 1). For a few cases, our approach showed a minor decrease in positive perturbation performance while achieving substantial improvement in negative perturbation performance. This trade-off is favorable - the slight positive perturbation decrease is outweighed by the significant negative perturbation gains, representing an overall improvement in explanation fidelity. These cases do not support the reviewer's broader claim about settings where disregarding positional embeddings is beneficial; rather, they demonstrate our method's effectiveness. These results mainly show that our approach is better at identifying truly important features, but it may have some noise in its ranking.

Additionally, it is important to note that our work does not perform hyperparameter tuning for the LRP hyperparameters or the selection of LRP rules, and thus our setup is likely not optimal and could be readily improved through such procedures.

Q.3. In Table 3, PE seems to outperform both AttnLRP as well as AttnLRP+PE on Tiny LLaMa. However, adding AttnLRP+PE does not seem to improve significantly over AttnLRP. Do you have any idea why this could be the case?

One possible explanation is that in Table 3, when evaluating XAI performance for sentiment analysis, the model relies heavily on opening statements and closing conclusions to make its prediction. As such, most of the prediction is attributed to the positional components of tokens at the beginning and at the end of the prompt.

Q.4.1. I am sceptical of how useful Lemma 3.1 is...

For Lemma 3.1, referring to input-level PE, such as in vision transformers, the image patches undergo convolution (or linear layers) first, then PEs are added to these patch embeddings before entering the transformer layers. This means PE is not simply "part of the input by design" - it's a separate additive component that AttnLRP treats as transparent. Moreover, while some methods choose to focus only on attributing the sum of positional and semantic relevance, without propagating relevance through the entire model as LRP is designed to do, this decision is unjustified and purely heuristic. Our approach provides a clear rationale for this design choice and defines how positional encodings should be treated.

Q.4.2. However, I do not find this conservativity argument meaningful for understanding the distinction between handling and not handling positional embeddings.

Assume, for example, that in a ViT model, patch embeddings (denoted by E) are extracted via a convolutional layer, and then PEs (P) are added to form Z=E+P, which is then passed into the Transformer. If we aim to propagate relevance back to the input space of the model at the pixel level (what LRP is designed for), the existing methods typically propagate relevance only through E, not through P. As a result, the relevance R(E) is propagated through the convolution back to the input, but any positional relevance R(P) is lost.

Additionally, for input-level PE, simply applying LRP to the sum positional and semantical relevance​ can lead to a loss of relevance, as negative positional relevance may override positive semantic relevance at each coordinate, and vice versa.

Q.4.3. .. do not think that Lemma 3.2 is very useful, as Eq. 9 is per definition conservative.

While there are specific cases (due to a combination of architecture, particular LRP rules, and the unjustified decision to stop relevance propagation before reaching the input space) where ignoring PE does not violate conservation, there are also clear cases where conservation is violated, such as the example discussed in our response to Q.4.2. In general, when relevance is propagated all the way to the input space rather than stopping at the point where PE is added, ignoring PE leads to a violation of conservation. This is critical, as propagating relevance to the final input space is a fundamental principle of LRP.