Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models

Weakness: On Model Scope and Mitigation Contributions

We thank the reviewer for their time and valuable feedback. We are glad that the importance of hallucination analysis in reasoning models has been recognized. Below we address the two concerns raised regarding generalizability and the lack (W1) of a mitigation technique (W2 in Q2).

1. On model selection and the annotation pipeline

As we noted in our response to Reviewer z9Q8’s W3, we chose DeepSeek-R1 for a combination of practical and empirical reasons: the model exhibits a high hallucination rate, low inference cost, and strong recognition within the open-source community. These characteristics made DeepSeek-R1 a suitable foundation for controlled, large-scale analysis of reasoning errors in CoT-style outputs.

To ensure annotation quality and consistency, we adopted a rigorous, multi-stage pipeline that combines GPT-4o-assisted tagging with human verification. In particular, as described in Appendix C.2, we defined the following categories with precise criteria:

• Wrong Reasoning This refers to a sentence or group of sentences responsible for “drawing conclusions or summarizing” within the reasoning chain, but which ultimately arrives at a judgment or answer that is clearly inconsistent with the facts. In simple terms, the model continues reasoning based on an incorrect premise and incorrectly accepts it.
• External Incorrect Knowledge This refers to a sentence or group of sentences in which the model references or builds upon external knowledge introduced directly or indirectly by the user input (i.e., information not contained in the model’s internal knowledge base or the relevant RFC document). These statements contain factual errors because the model accepts, incorporates, or elaborates on user-supplied information that is itself incorrect or misleading. In short, the model incorrectly relies on “imported” knowledge provided through the prompt.
• Internal Incorrect Knowledge This refers to fact-based content produced by the model that stems from its own internal knowledge, not prompted or introduced by the user. The model treats this information as objective truth, often presenting it with confidence, but it is factually incorrect when checked against the authoritative RFC document. In short, it reflects mislearned or misremembered knowledge from the model’s prior training or internal reasoning.
• Unreasonable Assumptions This refers to unsupported, disconnected assumptions raised by the model in its reasoning, often introduced with conditional language such as “if…” or “suppose…”. These assumptions lack justification from the context or facts, leading to a flawed logical foundation from the outset.
• Self-queries This refers to rhetorical or reflective questions posed by the model to itself during reasoning, often to explore or test new ideas. These typically end in question marks or include phrases like “let me think,” “could it be,” or “wait…,” guiding the model’s next steps.

Following GPT-4o–based annotation, we manually sampled 10% of the annotated dataset to refine the labeling schema, correcting edge cases and ambiguous boundaries between categories. The final annotation pipeline used in the study is the result of multiple iterations of refinement and validation. (See Q1 for generalizability of cross-model analysis.)

Question: On Generality, Prescriptiveness, and Resource Considerations

We thank the reviewer for their thoughtful feedback and critical suggestions. We agree that understanding whether metacognitive drift and hallucination patterns generalize across models is essential (Q1), and that a path toward practical mitigation (Q2) would strengthen the contribution. We further discuss the cost overhead of different methods. (Q3)

1. On generality beyond DeepSeek-R1

We agree that validating the generality of metacognitive drift across models is important. While DeepSeek-R1 served as our main analysis target due to its high hallucination incidence, affordability, and widespread usage, we also conducted additional tests on other models during the study's early phase.

Specifically, we probed Claude and Qwen using the same Type I and Type II setups (see detailed tables in our response to Reviewer z9Q8’s W3), and found that hallucination behaviors, particularly persistence and early error propagation, were also evident in these models. For example, Claude exhibited a hallucination rate of 67.8% in Type I and Qwen reached 94.4%. These findings suggest that the phenomena we report are not unique to DeepSeek-R1 but reflect shared limitations in multi-step reasoning across model families.

We hope this supports the broader relevance of our observations.

2. On actionable impact and mitigation

We appreciate the reviewer’s suggestion regarding mitigation. Although our current work does not directly propose or test a new hallucination reduction method, we believe it makes important foundational contributions toward that goal.

Our analysis reveals that many hallucinations stem from metacognitive failures. The model continues reasoning under false premises with unjustified confidence. In Section 3.4 and Table 3, we show that many existing detection techniques fail precisely because they ignore the dynamic nature of confidence evolution across multi-step reasoning.

This insight provides a new perspective for designing mitigation strategies. Rather than relying on static thresholds or isolated claim-level signals, future methods could focus on detecting suspicious confidence trajectories (e.g., abrupt increases or inconsistent shifts) or on interrupting reflection cycles that reinforce errors.

In this sense, our work offers a conceptual and analytical basis for intervention, and we view mitigation as a natural extension. Several follow-up efforts are underway to explore this direction.

3. On cost–benefit analysis of different baselines

We thank the reviewer for raising this important question. Below we offer a clearer breakdown of the computational cost.

To simplify the analysis, we normalize all LLM calls to a single average time unit, $T$, representing the time cost of one inference pass through a large language model (e.g., DeepSeek-R1 or GPT-4o). Other variables are defined as:

• $S$: Total number of sentences to be evaluated.

• $C_{\text{avg}}$: Average number of claims extracted per sentence (average 1.8).

• $Q$: Number of question variants generated per claim (typically 3).

• $M$: Number of times each question is re-answered (typically 3).

• $N$: Number of self-check samples per original CoT (typically 20).

• $T_{cla}$: Inference time for a BERT-like classifier, where $T_{cla}$ $\ll T$.

• $n$: Lightweight post-inference operations (e.g., attention/statistics), where $n \ll T$.

• Semantic Entropy

  Semantic Entropy decomposes sentences into atomic claims, generates new questions per claim, and then samples answers multiple times.

  $\mathrm{Time}\approx S \cdot (T + C_{\mathrm{avg}}\cdot Q \cdot M \cdot T) = S \cdot (T + 1.8\times3\times3 \cdot T) = S \cdot (T + 16.2 \cdot T) = 17.2ST$

• CCP (Claim Consistency via Prediction)

  CCP uses the same decomposed claims and evaluates token-level prediction confidence with semantic grouping via an NLI model.

  $\mathrm{Time}\approx S \cdot C_{\text{avg}} \cdot T = S \cdot 1.8 \cdot T = 1.8ST$

• SelfCheckGPT

  This method generates N responses (e.g., 20) and compares each with the original CoT trace using NLI-based sentence matching.

  $\mathrm{Time}\approx 20T$

• Medium-Cost Methods

  These include Logit Entropy, Attention Strength, and Spectral Entropy. Each method requires one full inference and a lightweight internal analysis.

  $\mathrm{Time}\approx T$

• HDM2 model

  HDM2 model applies a fine-tuned BERT classifier directly on the full CoT output:

  $\mathrm{Time}\approx T_{\text{cla}}\quad(\text{where } T_{\text{cla}} \ll T)$

  Inference is highly efficient—typically sub-second per sample on GPU.

We are greatly encouraged by the reviewer’s positive feedback. During the rebuttal process, we have worked to address the concerns raised as thoroughly as possible, providing additional data and clarifications where needed. We look forward to continued discussion.

Weakness: On Writing problem, Figure1 and robustness

1. On Writing problems and Figure 1 (W1, W2 and W4)

We thank the reviewer for their detailed feedback. We understand that parts of the paper (especially the early modeling and Figure 1) may have caused confusion due to the density of new terms and abstractions. While we aimed to balance technical rigor and brevity, we now see that some concepts were introduced too quickly or without sufficient grounding for first-time readers. We apologize for the resulting difficulty.

In the revision, we will clarify and more explicitly contextualize key concepts in the introduction and Section 2, including hallucination types, reasoning claims, and reflective interventions, as we recognize that some readers may find the current exposition insufficiently connected. We will also revise Figure 1, its caption, and related text to ensure that each component is clearly introduced with appropriate context, examples, and cross-references.

To help clarify now, we provide a more detailed breakdown of Figure 1, which is meant to offer an overview of our analysis framework:

Figure 1(a) presents a comparison between the reasoning-phase knowledge domain and the training-phase knowledge domain [1], as a foundation for defining two distinct types of hallucination.

• The upper part is the knowledge state during reasoning, i.e., what the model appears to “know” or retrieve when solving a specific task. This internal knowledge state may include:

  • Known: Correct facts from prior training,

  • Unknown concepts,

  • Incorrect beliefs, e.g., “the Sun is blue.”

• The lower part of the figure represents what the model was actually exposed to during training. This includes only a partial and sometimes distorted subset of real-world knowledge. In other words, the model's training data does not cover all facts:

  • Some facts were never included in the training set (Type II hallucination).
  • Some were present but learned only partially or with uncertainty (Type I hallucination).

Figure 1(b) illustrates our reasoning graph [2] [3], which represents how the model constructs its Chain-of-Thought (CoT) during multi-step reasoning.

• Each node represents a claim made by the model at a certain step (e.g., a fact, sub-conclusion, or logical assertion).
• The number next to each node indicates the order of the reasoning step.
• During the reasoning process, new knowledge may be introduced, either from the model itself (internal knowledge) or from the input prompt (external knowledge). If incorrect knowledge enters the graph, it can be carried forward to later steps and mislead the final answer.
• What makes this dangerous is that CoT often wraps the reasoning in fluent logic, making these errors look reasonable even when they are wrong.

Figure 1(c) gives an example of how incorrect knowledge introduced early can lead to hallucinated conclusions through reasoning.

• In this case, an incorrect claim (marked as $ck_1$) is injected at Step 2.
• As the reasoning continues, more knowledge is added at later steps. However, since the early claim is flawed, it silently influences downstream steps.
• At Step 7, the model performs a reflection [4], revisiting and modifying its earlier claim $c_1$ to a new version $c_4$, leading to a final conclusion that is logically consistent but factually wrong.

2. On Modelling robustness (W3)

We would like to clarify that robustness is not the main focus of this work. Our method is specifically aimed at improving the interpretability of the CoT process, and all evaluations are aligned with that objective. While robustness is an important aspect in general, our work does not make claims in that direction.

Regarding the point about deeper-level methods, while we acknowledge that some existing approaches do aim to provide interpretability at the level of model architecture or internal representations, we are not aware of prior work that directly addresses or explains hallucinations in the CoT process in the way our method attempts to. Our approach is motivated by the need for more targeted analysis of intermediate reasoning steps, which we believe remains relatively underexplored.

Finally, our modeling assumptions are not speculative, but are grounded in prior literature and supported by multiple references cited in the main paper and in this response. These provide theoretical and empirical motivation for our formulation.

Questions: On Confidence and Equation 3

1. What is confidence and what does it mean in a metacognitive sense? (Q1.1)

In our context, metacognition [5-7] refers to the model’s awareness of whether it knows something. Confidence captures how certain the model believes it knows something, regardless of whether that belief is actually correct.

For example, if the model says “the sun is red” and is fully certain about it, it is showing high confidence. What matters here is not the truth of the claim, but that the model believes it knows the answer with certainty. This reflects its internal metacognitive state.

Conversely, when the model faces knowledge it hasn’t fully learned or is uncertain about, it often gives more hesitant responses—indicating low confidence. These claims are more likely to be revised or overturned during the reasoning process.

Importantly, our modeling emphasizes that confidence is not static: it can change as the reasoning unfolds. In multi-step CoT reasoning, the model’s belief in a given claim may be weakened, reinforced, or even reversed based on new knowledge introduced later. Our work focuses on how these confidence shifts happen, and how they relate to hallucination formation and propagation.

2. How should Equation (3) be interpreted? (Q1.2)

Equation (3) defines how the model's confidence in a new claim $c_q$ changes relative to a previous claim $c_p$ during the reflection process. Specifically, it quantifies:

• Here, $\text{conf}(c_q)$ and $\text{conf}(c_p)$ represent the model's internal confidence in corresponding claims $c_q$ and $c_p$, respectively.

• The change in confidence depends on two sources:

  1. Internal feedback, captured by $f(c_{p-1}, c_q)$, which reflects how strongly the current reasoning step $c_{p-1}$ supports or contradicts $c_q$. We model this as a function of $c_{p-1}$ rather than $c_p$, because of the correspondence between $c_p$ and $c_q$. Thus, the feedback should be determined based on the prior state before the confidence update (i.e., $c_{p-1}$) and the new claim $c_q$.
  2. Prompt alignment [8], modeled by $g(c_q, \text{prompt})$, which reflects how well the new claim $c_q$ semantically aligns with the original user instruction or prompt context.

• The parameter $\alpha \in [0, 1]$ controls the weighting between these two influences.

In simple terms, this equation states that the model will become more confident in a new claim $c_q$ if it is both (a) supported by its own recent reasoning history, and (b) aligned with what the user originally asked. If either of these is weak, confidence may decrease. This formulation gives us a concrete way to track how belief in a claim strengthens or weakens during reflection.

3. Considered other metacognitive confidence modeling methods? (Q2)

Yes. As we explain in Section 3.4, many hallucination detection methods [9] today are in fact trying to estimate how confident the model is about its own statements. In other words, they are testing whether the model is metacognitively “sure” of what it’s saying.

However, the key issue is that most existing methods rely on fixed rules, confidence heuristics, or surface-level signals. They do not deeply involve semantic understanding, and they do not touch the internal model reasoning process. As a result, these methods often cannot give reliable confidence estimates, especially in knowledge-intensive tasks. Our experiments show that in these cases, they perform poorly (see Table 5).

[1] Physics of Language Models: Part 3.1. ICML 2024

[2] On the Biology of a Large Language Model, Transformer Circuits, 2025.

[3] Physics of Language Models: Part 2.1. ICLR 2025.

[4] Mirror: Multiple-perspective Self-Reflection Method for Knowledge-rich Reasoning. ACL2024.

[5] Large language models lack essential metacognition for reliable medical reasoning. Nature communications, 2025.

[6] Decoupling Metacognition from Cognition. AAAI 2025.

[7] Language models (mostly) know what they know. 2022.

[8] From Yes-Men to Truth-Tellers. ICML 2025

[9] Detecting hallucinations in large language models using semantic entropy. Nature 2025.

We sincerely thank Reviewer BZYH for their careful reading and constructive feedback. We understand the core concern lies in the clarity of Figure 1 and Section 2, where several key concepts were introduced too quickly, potentially limiting accessibility. This was partly due to space constraints, as we prioritized presenting broad experimental insights. In the revision, we will improve the exposition and readability of these sections while maintaining technical depth.

We are encouraged that the other three reviewers recognized the strong motivation and extensive experimental validation of our work. In this response, we have addressed Reviewer BZYH’s main concerns in detail and clarified the points that were previously unclear. We hope the core value of this paper can be seen in the empirical observations and insights provided in Section 3, which we believe offer useful perspectives for future research on hallucination behavior. We would sincerely appreciate it if the reviewer could reconsider their assessment in light of these contributions.

I thank the authors for their thorough review. I appreciate their future effort in building the concepts in the paper "from the ground up", and clarying some of the questions I have posed. In light of their comment, I am raising my score to 3.

Weakness: On Clarity, Section 2.3, Terminology, and Generalizability

We sincerely thank the reviewer for the constructive and thoughtful comments. Below, we address the concerns (weaknesses) regarding writing clarity (W1 and W2), conceptual complexity, Section 2.3 implementation detail (W1), and the choice of our controlled knowledge domain (RFC documents) (W3).

1. On writing clarity, missing references, and Section 2.3

We apologize for the confusion caused by the missing citations at lines 105 and 121 (W1). These were formatting oversights and we will correct them in the revised version. More broadly, we understand that Section 2.3 may feel too abstract in its current form. While it is intended to provide a conceptual abstraction of our reflection framework, we agree that it lacks sufficient detail to be easily interpreted by new readers.

In the revised version, we will focus on the following two key improvements:

• Proper references and clearer explanations, such as metacognitive drift, prompt-aligned bias, and flawed reflection. These terms will be explicitly defined at first use and connected to relevant prior work or intuitive examples to aid reader understanding.
• Modeling assumptions and procedures, including how claim-level confidence is computed from model signals, and how confidence propagates across reasoning steps during reflection. This includes elaborating on the conceptual role of the $g(c_q, \text{prompt})$ term and clarifying the structural function of the reflection framework introduced in Section 2.3.

We would also like to clarify that we do not aim to provide a fully-automated or deployable implementation of Equations (2) and (3). Instead, our goal is to propose a conceptual decomposition that separates metacognitive confidence from alignment-based influence. This serves as a foundation for future analysis and improvements in LLM reasoning.

2. On Figure 1 and Section 2 alignment

We agree that Figure 1 currently lacks clear linkage to Section 2 (W2) and may confuse readers due to undefined terms and unreferenced notations. In the revised paper, we will introduce all figure terms in the main text with clear definitions, including prompt-aligned bias, hallucination types, and reasoning steps. (Detailed explanation of Fig. 1 in BZYH W1).

3. On use of RFCs and generalizability

We would like to clarify our motivation for choosing the RFC corpus (W3). This was not an simplification, but a deliberate design choice to create a controlled knowledge environment for analyzing hallucination formation.

• RFC documents offer clear, modular, and factual content with minimal ambiguity, which makes it possible to precisely annotate ground truth, track knowledge injection points, and isolate the causes of reasoning errors.

• Compared to open-domain sources, RFCs have well-defined standards and low risk of semantic noise or conflicting data, making them ideal for identifying when the model, rather than the data, is the source of hallucination.

It’s important to note that our focus is on a model’s loyalty to its learned knowledge, not necessarily to real-world facts. In other words, if a model hallucinates due to incorrect training data, we do not count that as a metacognitive failure of the model. This is precisely why we restrict the knowledge source to RFCs, so we can minimize data ambiguity and focus on the model’s internal reasoning behavior.

That said, we agree that real-world reasoning mistakes often stem from a mixture of factors: hallucinations, corrupted training signals, poor alignment, etc. We view our current setup as a necessary first step for disentangling these effects, and we are already planning to extend our methodology to more complex, noisy, and multi-domain settings.

Question: On Reasoning Drift, Type II Control, and Trajectory Tracing

We thank the reviewer for raising these valuable questions, which target key aspects of our methodology and analysis depth. The concerns primarily relate to the interpretation of reasoning length (Q1), the handling of Type II hallucination controls and corresponding evaluation (Q2), and the transparency of how reasoning trajectories are extracted and analyzed (Q3).

Q1. On the correlation between claim length and uncertainty

Our Obs. 2 is not based on chain length alone, but rather on the dynamic reflection behavior that occurs when the model faces low confidence in its intermediate claims. Specifically, when hallucinations occur, the model often performs more reflections, attempting to adjust its metacognitive confidence before either rejecting or fully accepting the hallucinated claim. This reflective process naturally increases the CoT length.

To validate this point, we conducted an analysis on the Type I dataset (as detailed in Appendix B.2), where each sample was generated using 5 independent runs. We grouped samples by the number of hallucinations observed and calculated the average CoT length (number of claims) for each group:

The data show a clear positive correlation between hallucination frequency and CoT length, supporting our obs. 2 that uncertainty-driven reflection leads to longer reasoning chains. While task complexity might also contribute, the strong trend observed here indicates that the increase in length is closely tied to hallucination-related metacognitive drift.

Q2. On missing analysis of Type II control and Part C

Thank you for pointing this out. The reviewer is correct that the Type II control group was not explicitly included in the main analysis of Table 2. This omission was primarily due to its limited usable sample size. In particular, prompt-aligned bias tends to be consistently triggered, making it difficult to collect clean, non-hallucinated contrast cases in this setting.

Nonetheless, we agree that the absence of detailed discussion may make Observation 3 feel speculative. In fact, upon closer inspection of Part C (Type II), we observed that the model unexpectedly generated an average of 5.25 incorrect internal knowledge units, which is quite close to the 6.73 observed in Type I, despite their different sources of hallucination (misleading prompt vs. knowledge absence).

Moreover, these internally hallucinated facts in Type II traces exhibited similar propagation patterns to Type I, such as 50% adoption, 40% correction, and 10% rejection. This suggests that the model is not just copying the misleading information from the prompt, but also generates additional incorrect internal knowledge on its own, which likely plays a key role in the final wrong answer. This behavioral pattern is the core empirical support for Obs. 3.

Q3. On how reasoning trajectories in Fig. 2 are selected and traced

Thank you for raising this question. Strictly speaking, the examples shown in Figure 2 are neither curated hand-picked cases nor representative samples selected via statistical procedures.

Each case in Figure 2 was randomly drawn from the set of CoT traces corresponding to the three types, with the goal of illustrating structural differences in trajectory behavior. For visualization purposes, we manually shortened the full reasoning trace, as many original CoTs contain more than 40 claims (as shown in Table 2A). In the case of (b) Control: Error Reject, we applied a very minimal manual adjustment to the CoT trace. Specifically, we shifted the error rejection slightly earlier than it actually occurred, so that the correction dynamics could be more clearly shown within a readable length.

We acknowledge this intervention and will clearly state it in the figure caption and appendix. These examples are meant to be illustrative, not evaluative, and do not affect our quantitative analyses.

We also appreciate the reviewer’s attention to the reliability of our CoT trajectory annotation process. As noted, identifying structural elements such as dropped paths or conditioned branches from model-generated CoT traces is inherently non-trivial, since these are implicit in free-form natural language.

Due to space limitations, we were unable to include full examples. Below we provide concrete, real cases illustrating how such structures are identified:

Conditioned Branches:
Case1:(Unreasonable assumption)
If a node doesn't validate the UDP checksum, maybe an attacker could tamper with the UDP payload (which might include the HMAC) without the checksum catching it.
Case2:
Wait, maybe the process is that the HMAC validation is done after the checksum check. If the checksum is invalid, the packet is dropped before HMAC validation. So if a node doesn't check the checksum, it might process a packet with a modified HMAC...

Dropped Paths:
Case1:(Dropping claim)
I believe RFC 3866 is related to email headers... If that's the case, then it might be updating an older RFC that dealt with similar subject matter.
==drop==
Alternatively, maybe I can think of other RFCs in the same category, like SPF, DKIM, DMARC.
Case2:
Alternatively, maybe I can think of the authors. Let's see. The authors of RFC 3866 are J. Ott and D. Mutz. 
==drop==
Wait, the title is \"Extensible Message Format for Message Disposition Notifications.\" No, that's RFC 8098.  
==drop==
Wait, no.\n\nWait, I'm getting confused. Let me try to recall. Another approach: RFC 3866 is titled \"Common Profile for Instant Messaging (CPIM)\". No, that's RFC 3860. 

We are sincerely encouraged by the reviewer’s remark that they enjoyed reading the paper. Throughout the review process, we have strived to provide the most faithful and transparent explanations possible within the available space, in the hope of addressing all concerns raised. We would warmly welcome further discussion or reconsideration of the decision.

The reviewer thanks for the detailed responses. My concerns are enougly addressed. Considering the contribution of this paper (hallucination in reasoning process), the reviewer leans to BA.

Weakness&Question: On Confidence Modeling, Prompt Bias, Model Scope, and Structure

We thank the reviewer for their careful reading and thoughtful comments. The suggestions regarding the empirical grounding of our theoretical model (W1), the interpretation of prompt-aligned bias (W2 and Q1), the generalizability of findings (W3), and the presentation of experimental setup (W4 and Q2) are all well taken. Below we address each point in turn and outline how we plan to revise the manuscript accordingly.

1. On bridging theoretical confidence modeling (Eq. 2/3) with empirical validation

We appreciate the reviewer’s recognition of the confidence update model in Section 2.3. The confidence update model in Section 2.3 is designed to help describe how confidence may evolve during multi-step reasoning. We acknowledge that in the current version, we do not quantitatively estimate each variable in Equations (2) and (3).

However, as discussed in Section 3.4, many existing hallucination detection methods (such as entropy-based uncertainty, logit margins, and self-consistency) can be viewed as practical ways to approximate the dynamic changes in model confidence, especially for individual claims. While we do not directly compare our formulation with these methods, they share a similar motivation: estimating whether the model “believes it knows” something, and to what extent.

We believe the main limitation of these methods is that they focus on single-point confidence estimation for individual claims, rather than modeling the change in confidence ($\Delta {\text{conf}}$ ) across the entire reasoning trajectory. In long-form CoT reasoning, it is not just the confidence of a specific step that matters, but how that confidence emerges and evolves as the reasoning progresses.

As shown in Appendix E.3, even after applying smoothing techniques, the confidence signals across CoT steps often show sudden jumps up and down, rather than smooth or consistent changes. This supports our view that hallucination is not purely a local phenomenon, but often emerges from global inconsistency or failure to maintain coherent belief across steps. Capturing this dynamic behavior is what our confidence update model aims to describe.

2. On whether prompt-aligned bias reflects true alignment or knowledge deficit

We sincerely appreciate this insightful observation. In fact, we considered this exact alternative explanation during our analysis. Many of the insights in our paper, including Observations I and III, stem from carefully distinguishing between what the model learned and how it chooses to reason given that knowledge.

To investigate whether the model’s acceptance of false premises is simply due to a knowledge deficit, we conducted an additional experiment: We constructed a balanced set of 500 factually correct statements and 500 factually incorrect statements (the same source pool used for Type II hallucination generation) and asked the model to judge their factual correctness using a neutral prompt.

The results are shown below:

This shows that the model is fully capable of recognizing most of these facts as true or false.

Moreover, in the Type II (Unseen or Incorrect) cases selected for analysis, we did not observe any signs of the model expressing uncertainty or epistemic hesitation about the injected incorrect information (in answer). The model confidently accepted and followed the external incorrect knowledge, despite clearly “knowing better” in isolation.

Taken together, this evidence supports our interpretation that the model’s behavior is not simply caused by a lack of knowledge. Instead, we argue that it reflects a prompt-aligned bias, where the model over-prioritizes consistency with the input prompt, even at the expense of factual correctness.

3. On generalizability beyond DeepSeek-R1

We thank the reviewer for raising this important point. Due to time constraints, we were unable to perform a full-scale replication of our framework across multiple model families. However, before selecting DeepSeek-R1 for our main experiments, we conducted an extensive preliminary survey of hallucination behavior across several major reasoning-capable LLMs.

In selecting a model, we considered three factors:

1. Frequency of hallucination phenomena (especially CoT-induced),
2. API accessibility and cost, and
3. Relevance and adoption within the current open-source LLM ecosystem.

DeepSeek-R1 was ultimately chosen because it offered a high hallucination rate, low API cost, and strong community recognition, making it well-suited for large-scale controlled experimentation.

To support this decision, we include two tables below summarizing hallucination behavior observed in other models during our preliminary evaluations, all using the same Type I / Type II setup.

Accept rate (i.e., percentage of 5-time queries where hallucination occurred ≥ 4 times):

Note: DeepSeek’s values are calculated across all outputs, consistent with Table 1 in the main paper. Claude and Qwen were evaluated on subsets filtered through DeepSeek’s outputs, leading to different sample distributions.

Hallucination rate across all responses (i.e., proportion of hallucinated answers among all responses):

Unfortunately, we were unable to include GPT-o3 in this comparison, as its inference costs exceeded our available resources at the time of the study.

We view our current work as establishing the methodology and analysis tools, and we plan to apply the same framework to additional models in future work. We believe the core phenomena we observe—such as error propagation, reflection failure, and prompt-aligned drift—are not specific to DeepSeek-R1 but reflect broader behaviors in CoT-style reasoning across LLMs.

4. On clearer pointer to experimental setup in Section 3

Thank you for this helpful suggestion. We agree that readers would benefit from a clearer connection between the experimental results in Section 3 and the dataset/setup details provided in Appendix B.

In the revised version, we will add a brief summary paragraph at the beginning of Section 3 to outline:

• The dataset construction process, including prompt formulation and hallucination control conditions (Types I and II).
• The model sampling and annotation protocol.
• Key evaluation metrics used throughout the analysis.

We will also include an explicit forward reference to Appendix B for readers who wish to explore the experimental configuration and data labeling procedures in more depth.

We believe this change will significantly improve the clarity and flow of the paper.

We are deeply encouraged by the reviewer’s belief that our paper could be valuable to the community. We have done our best to address the raised concerns, supplementing our response with further data and analysis. We hope this can invite further discussion and reconsideration of the rating, as we believe the work may offer useful perspectives for ongoing research in this area.