Intervening Anchor Token: Decoding Strategy in Alleviating Hallucinations for MLLMs

Q1: Some of the color schemes in the figures, such as Figures 4 and 5, are prone to causing confusion. It is recommended to use colors with higher contrast and more distinct differentiation.

A1: I apologize for any confusion caused by the color schemes in the figures. We have refined the figures by adopting a color scheme with higher contrast and more distinct differentiation to enhance clarity in the final version.

Q2: The paper begins its analysis with the phenomena of token information flow and token information aggregation, gradually developing a method to mitigate hallucinations. However, it does not yet discuss the underlying causes or triggers of these phenomena, relying only on empirical analysis. Nevertheless, the analytical approach used in the paper is still worth learning from.

A2: Thank you for your insightful suggestion! We posit that the primary triggers of token information flow and token information aggregation in MLLMs stem from the intrinsic properties of the attention mechanism. Specifically, in the shallow layers, the model aggregates critical information from the input sequence into a small subset of tokens, referred to as Anchor Tokens, through high attention weights. In the deeper layers, these tokens form localized information hubs, often at the expense of overlooking other contextual information. Our observation aligns with the findings reported in [1].

Furthermore, we hypothesize that this phenomenon fundamentally traces back to the configuration of the QK parameters within the attention weight matrix, which plays a pivotal role in modulating attention distributions. Specifically, the eigenspectrum of the generated matrix $\mathrm{W}$ and variations in $\operatorname{tr}\left(\mathrm{W}^2\right)$ significantly influence the information propagation patterns across layers, leading to either token localization or uniformity.

This analysis is elaborated in Section 3, titled “When are tokens localized or uniform?” Through theoretical derivations and empirical evidence, we demonstrate that token localization closely ties to the distribution of the eigenspectrum of $\mathrm{W}$. This finding not only provides a novel perspective for understanding the localized properties of the attention mechanism but also establishes a theoretical foundation for addressing hallucination issues in MLLMs.

Q3: The paper provides limited explanation of the proposed method. Although the method is simple (just one single formula), a smoother transition from the theoretical derivation is needed, along with a clear description of its application scenarios.

A3: I apologize for any confusion caused by our initial description. To provide a clearer connection between the theoretical analysis and the proposed method, as well as to better explain its application scenarios, we have added a transitional paragraph in the revised version:

“The previous analysis demonstrates that anchor tokens significantly impact the expressive capability and hallucination phenomena of MLLMs through the eigenspectrum properties of the attention weight matrix. Specifically, moderate propagation of anchor tokens enhances the model's expressivity, while over-propagation leads to hallucinations. These findings highlight that controlling the eigenspectrum of the query-key weight matrix ($\mathrm{W}$) can effectively regulate the propagation intensity of anchor tokens. Compared to intervening in $\mathrm{W} = \mathrm{W} _ {\mathrm{QK}} {\Sigma}$, we choose to directly adjust the eigenspectrum of $\mathrm{W} _ {\mathrm{QK}}$. This is because $\mathrm{W}$ introduces the covariance matrix ${\Sigma}$, which remains unchanged during inference, making it sufficient to adjust $\mathrm{W} _ {\mathrm{QK}}$ to flexibly control attention propagation. Hence, merely intervening the $\mathrm{W}_{\mathrm{QK}}$ helps the model avoid triggers for hallucinations.”

Q1: The introduction mentions summary tokens but does not define them, leaving unclear the distinction between summary tokens and anchor tokens.

A1: Thank you for your valuable feedback. We provide clear definitions for both below.

Summary Tokens are a specific type of text token in LLMs that primarily serve to aggregate critical information from a sequence during generation and provide global guidance for subsequent token generation. They reflect the "Aggregation Pattern" inherent to LLMs, enabling the model to synthesize global context for generating coherent outputs. However, excessive reliance on the global information provided by summary tokens may cause the model to overlook original contextual or visual modality inputs, leading to hallucinations.

Anchor Tokens are key tokens with high propagation probability in the attention mechanism, especially in multimodal tasks, where they emphasize information interaction between multimodal tokens. However, the over-propagation of anchor tokens can lead to an overemphasis on localized information, causing the generated content to disproportionately focus on specific objects or concepts while neglecting broader contextual cues. This imbalance in attention distribution can contribute to the emergence of hallucinations.

---

Q2: There are some typos in the text that need to be checked: Line 301: "Conversly" → "Conversely" Line 333: "7,3" → "7.3"

A2: We apologize for the confusion caused by the typo. We have carefully reviewed the text and made the necessary corrections in the revised manuscript.

We hope that our response sufficiently addresses your comments. If you have any further questions or concerns, please feel free to reach out to us for further discussion.

Q3: It would be interesting to study the application of TAME to some layers rather than all layers through experiments.

A3: Thank you for your insightful suggestion. As analyzed in Figure 1 and Appendix A (main paper), early, middle, and late layers exhibit significant differences in token propagation patterns. Early layers ($l_{0}$) primarily focus on extracting low-level features and token initialization, middle layers ($l_{16}$) emphasize multi-modal alignment and feature aggregation, while late layers ($l_{31}$) are responsible for high-level reasoning and final output generation. These differences play a critical role in the model's behavior and error generation, especially in hallucination-prone scenarios.

We evaluate TAME by applying it to different combinations of layers ($l_{0}$, $l_{16}$, and $l_{31}$), as shown in the table below. In this experiment, the baseline is OPERA [1] (Exp I). The results from Exp II-IV demonstrate the incremental impact of TAME when applied to individual layers: Early layers ($l_{0}$, Exp II): Applying TAME at this stage slightly reduces hallucinations by refining token initialization and propagation, but its overall impact is limited due to the lack of deeper-layer information.Middle layers ($l_{16}$, Exp III): Incorporating TAME at this stage significantly improves multi-modal alignment, optimizing feature integration and enhancing response generation accuracy.Late layers ($l_{31}$, Exp IV): Applying TAME here substantially reduces errors, as this stage directly influences final reasoning and output generation. The results from Exp V-VIII further demonstrate the cumulative effect of applying TAME to multiple layers: $l_{0} + l_{16} + l_{31}$ (Exp VIII): Applying TAME across all layers achieves the best overall performance, with the fewest hallucinations and the highest accuracy across all tasks.

This phenomenon highlights the importance of holistically optimizing token propagation across the model. Early layers provide foundational improvements, middle layers optimize multi-modal representations, and late layers ensure high-level reasoning accuracy. The experiments suggest that integrating TAME across more layers significantly reduces error-prone responses, as it dynamically mitigates over-propagation at different stages of the model.

Table2: Ablation study of applying TAME on different layers. $l_{0}$: layer 0; $l_{16}$: layer 16; $l_{31}$: layer 31

References:

[1] OPERA: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation.

Q1: Although this paper provides an additional theoretical framework, the impact of anchor tokens on LVLM hallucination was first proposed by OPERA [1].

A1: Thank you for your suggestion. OPERA [1] is a relatively complex approach to reducing hallucinations, focusing primarily on anchor tokens in the textual modality without fully considering the deeper implications of multimodal interactions in MLLMs. While OPERA effectively reduces hallucinations through a penalty mechanism targeting anchor token contributions, it incurs significant computational overhead, increasing inference time by 2-3 times due to the additional processing required for each token.

In contrast, our proposed method, TAME, offers an efficient solution that outperforms current state-of-the-art approaches. TAME not only accounts for the interactions between multimodal tokens but also addresses the limitations in existing methods regarding the understanding of anchor tokens' roles in multimodal tasks. Specifically, the dual nature of anchor tokens, which can act as both a "blessing" and a "curse," has not been sufficiently clarified. Our work focuses on two key research questions: (Q1) Under what conditions do tokens exhibit localized or uniform distributions? (Q2) How do anchor tokens influence the generation of hallucinations?

Moreover, TAME mitigates the over-propagation of anchor tokens by dynamically adjusting the eigenspectrum variance of the attention weight matrix. This lightweight approach significantly improves hallucination reduction without introducing additional inference costs. Our work provides a novel and theoretically grounded framework for addressing hallucination problems in MLLMs, demonstrating its efficiency and robustness across practical tasks.

Q2: TAME primarily addresses object hallucinations but may not be as effective for other types of hallucinations, such as incorrect attributes or relations (e.g., limited performance improvement on the MME benchmark). Experiments on more types of hallucination should be conducted.

A2: Thank you for your insightful comments. The table below presents our experimental results on the hallucination subset of the MME dataset, which includes object-level hallucinations and attribute-level hallucinations. TAME, as a plug-and-play decoding strategy, delivers significant performance improvements across methods by dynamically adjusting the eigenspectrum variance of the attention weight matrix. This mitigates anchor token over-propagation and enhances model performance in object existence, count estimation, position alignment, and color consistency.

Table 1: Evaluation results on the hallucination subset of MME. Object-level: Existence and Count. Attribute-level: Position and Color. max-tokens=512.

---

Q6: I feel the proposed method can be used for any LLM-based approach. Why does the title claim it is only for multimodal LLMs?

A6: Thank you for your question. We extend our experiments to single-modal LLMs. As shown in the table below, we conduct evaluations on the Wikitext-103 [6] and MiniPile [7] datasets to assess the scalability and consistency of TAME's impact. TAME was integrated as a plug-and-play enhancement across three distinct model configurations, including BLOOM (LLaMA architecture with ALiBi) [8] and OpenLLaMA [9].

The results show that TAME consistently improves perplexity (PPL) across all architectures and parameter sizes, demonstrating its effectiveness in enhancing model performance. These findings further support that TAME is a generalizable mechanism, suitable for both multimodal and single-modal LLMs, broadening its applicability.

Table 2:Perplexity (PPL) comparison on WikiText-103 and MiniPile datasets using BLOOM and OpenLLaMA architectures with and without TAME across varying parameter sizes.

---

References:

[1] Dense passage retrieval for open-domain question answering.

[2] SURf: Teaching Large Vision-Language Models to Selectively Utilize Retrieved Information.

[3] RLHF-V: Towards trustworthy MLLMs via behavior alignment from fine-grained correctional human feedback.

[4] Calibrated Self-Rewarding Vision Language Models.

[5] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning.

[6] Pointer sentinel mixture models.

[7] The minipile challenge for data-efficient language models.

[8] The power of scale for parameter-efficient prompt tuning.

[9] LLaMA: Open and efficient foundation language models.

Table 1: Comparison of methods on different benchmarks. SEEDᴵ refers to SEED image evaluation, VisWizⱽ refers to image evaluation for VisWiz VQA, POPEᴿ refers to POPE Random, MMEᴼ refers to MME Object-level Hallucination Existence Count.

---

Q1: I hope the paper could explore additional approaches to reduce hallucinations, such as Retrieval-Augmented Generation (RAG), Reinforcement Learning from Human Feedback (RLHF), improvements in factuality metrics, and high-quality, data-based instruction tuning.

Q2: The paper currently uses a few simple evaluation sets like MS COCO. I would suggest incorporating more challenging benchmarks for a more rigorous assessment.

A1 and A2: Thank you for your insightful suggestion. We conduct a rigorous evaluation of VCD, OPERA, and RAG (Vanilla-RAG [1], SURf-RAG [2]) as well as RLHF (RLHF-V [3], CSR [4]) on six popular MLLM benchmarks and four additional ones, as shown in the table below.

For details, SEED-Bench consists of 19k multiple choice questions with human annotations, while spanning 12 evaluation dimensions. GQA incorporates a novel evaluation metrics suite focused on consistency, grounding, and plausibility, establishing a rigorous standard for assessing in vision-language tasks. Vizwiz examines certain perception capabilities, like knowledge and relation. MME contains 14 meticulously designed subtasks that challenge the models' interpretative and analytical skills. MMBench examines LVLMs on general perception capabilities using a wide range of tasks. POPE is an assessment methodology designed to scrutinize object hallucination in LVLMs.

Vanilla-RAG concatenates the Top-N image-caption pairs from the database, which have the highest CLIP score similarity to the test image, before appending the questions and images for the LVLMs to respond. SURf-RAG is a self-refinement framework that teaches LVLMs to selectively utilize retrieved information. RLHF-V collects fine-grained paragraph-level corrections from humans on hallucinations and performs dense direct preference optimization using human feedback. CSR enables the model to self-improve by iteratively generating candidate responses, evaluating the reward for each response, and curating preference data for fine-tuning.

For the RAG and RLHF methods, our approach seamlessly integrates as a plug-and-play module within their frameworks, without incurring extra inference time. By combining RAG with TAME, external knowledge retrieval enhances the accuracy of generated outputs, ensuring better alignment with the input context. Similarly, when integrated with RLHF, our method leverages human feedback to guide the model in producing outputs that are more faithful to the actual content. Experimental results demonstrate that our approach delivers comprehensive improvements across both frameworks, further validating its effectiveness.

We conduct additional experiments to evaluate improvements in factuality metrics and the effectiveness of data-driven instruction tuning on model performance. Our evaluations focus on benchmarks such as GQA, Vizwiz, MME, and POPE, which test real-world knowledge QA and multimodal understanding tasks, as shown in the table below. The results demonstrate that instruction tuning with LRV-Instruction-finetuned [5] moderately enhances performance by leveraging high-quality image-text pairs for task-specific fine-tuning, improving the model's alignment with real-world knowledge. However, integrating TAME further amplifies these gains by dynamically mitigating hallucinations and strengthening factual alignment, resulting in significant improvements across all benchmarks. This combination achieves a higher degree of correctness and consistency in generated outputs, validating that TAME effectively complements instruction tuning as a plug-and-play enhancement, improving both accuracy and robustness in real-world multimodal tasks.