Thank you for taking the time to review our paper and for your insightful comments. Please find our responses to your questions below.

Q1. Computational overhead compared to the baseline approaches.

A1. We thank the reviewer for highlighting the importance of evaluating the computational overhead of our method. To this end, we conducted experiments comparing the average detection time per input with baseline methods. We randomly sampled 100 prompts (average length: 30 tokens) and measured detection time across the benchmark models. The results show that our method introduces moderate overhead, remaining within the same order of magnitude as the model’s inference time. The table below summarizes the detection time (in seconds) and the corresponding inference time for reference.

On average, for 7B–8B models (Llama-2-7B, Llama-3.1, Mistral), our method requires around 3.5 seconds per input, compared to 7.5 seconds for the baseline lie detector and 0.05 seconds for RepE. For Llama-2-13B, it takes 7.7 seconds, while the lie detector baseline requires 13.2 seconds and RepE 0.08 seconds. For backdoor detection, the ONION baseline works as a pre-processing step and takes approximately 0.3 seconds per input across all models.

While RepE is the fastest due to its single-pass extraction, it sacrifices detection reliability, as discussed in the paper. Similarly, ONION performs poorly in backdoor detection. In contrast, our method is more efficient than the baseline lie detector while achieving stronger detection performance.

In summary, our method introduces moderate overhead and strikes a balance between efficiency and effectiveness. We have included a detailed breakdown of runtime performance across model sizes in the appendix.

Q2. Could the authors give their opinion on the ability of a white-box adversary to bypass their detection method?

A2. We thank the reviewer for raising this important point regarding the robustness of our method against white-box adversaries. We acknowledge that adversarial detection is particularly challenging under adaptive attacks and appreciate the opportunity to clarify the threat model in our work. While a white-box adversary could theoretically attempt to bypass our detection by minimizing causal signals, such attacks are highly non-trivial in practice for two reasons:

Layer-level causal effects in our method are discrete values based on separate interventions. This process produces non-differentiable outputs and discrete shifts in behavior. As a result, the causal signals they generate are not amenable to standard gradient-based optimization techniques. This makes it challenging even for a white-box adversary to perform targeted manipulation.

Token-level causal effects are based on statistical aggregation across calculated distances between intervened attention heads and the original one. This complex calculation process makes them inherently noisy and non-smooth. As a result, an adversarial would need to account for a wide range of token-level variations, as well as attention heads changes. This would significantly complicate the optimization process potentially adopted by an adaptive attacker.

We have added a discussion to clarify that our experiments were conducted to detect adversaries who are not aware of the defense mechanism, and a discussion on the difficulty of conducting adaptive attack.

Comments:

C1.: remove inconsistency in notation regarding definitions.

A1. Thank you for pointing out the inconsistency in our submission, and we have revised accordingly

C2. section three presents very simple concepts in an overly complicated way.

A2. We thank the reviewer for the comment. Our intention was to provide a formal foundation for our causal analysis, while acknowledging the practical compromises needed for token and layer level analysis. We acknowledge the reviewer’s concern that parts of Section 3 may have introduced unnecessary complexity and redundancy. In response, we have streamlined the definitions and moved some of them to the appendix.

C3. Overall the figures and tables take a lot of space.

A3. We will revisit the figures/tables and move them to appendix in case space is required (to clarify all issues during the rebuttal).

Other response: We acknowledge the readability issue in Figure 3 and have revised it with larger fonts for better visibility. Additionally, we have expanded the captions of tables and figures to make them more informative. For example, the caption of Figure 2 now clarifies that it illustrates combined causal effects at the prompt and layer levels for a truth and lie response.

Thank you for taking the time to review our paper and for your insightful comments. Please find our responses to your questions below.

Q1. Causal map visualization showing both input tokens and layers.

A1. We thank the reviewer for their careful observation regarding the presentation of the causal map in Figure 2 and sorry for the confusion. In our work, the term “causal map” refers to the combined causal effects computed across two dimensions including 1) token-level causal effects, which correspond to contribution of the input (embedding); 2) layer-level causal effects, which correspond to contribution of the transformer layers. We chose the term as it resembles a “heat map” over the input and the internal layers.

Q2. The “causal effect” definition and concerns about downstream effects after intervention.

A2. We appreciate the reviewer for raising this thoughtful concern regarding the formal definition of causal effect in our work. As discussed in Section 3.1, a proper causality analysis would require a formal model such as structured causal model, and systematic intervention (as shown in equation 3). Such analysis is, however, often impractical as it is computationally intractable. What we adopted in this work is a lightweight and practical approach that is referred to in prior works [1, 2] as causal mediation analysis (CMA). In CMA, causal effects are approximated by comparing outcomes between the normal execution and an abnormal execution. In our context, the normal execution refers to the LLM’s standard output and the abnormal execution corresponds to the output when we apply targeted interventions, i.e., modifying a specific token or skipping a transformer layer. The causal effect is then approximated by measuring the difference between the two executions. We acknowledge the reviewer’s concern that interventions like changing a token or skipping a layer may introduce cascading effects throughout the model. Nonetheless, we show that such poor approximation of the actual causal effect is already effective for detecting LLM misbehavior. We have added more clarification on our causality analysis method and its limitation in the revised submission to clarify this issue.

We also appreciate the insightful comments on the downstream impact of adding instruction. This is in fact a question on the definition of misbehavior. Indeed, our definition of whether a model misbehaves (e.g., whether it is generating toxic responses) does not depend on the prompt. In other words, our method will detect misbehavior itself (such as generating toxic responses) even if the model is instructed to do so (such as with the instruction “answer the following question with a lie”). Such a choice could be considered reasonable if we aim to detect jailbreak attacks such as DAN that aims to induce toxic responses with “legit” excuses. Having said that, we agree that further research on detecting misbehavior considering the context is an extremely interesting one.

Reference:

[1] Locating and Editing Factual Associations in GPT (NeurIPS 2022)

[2] Investigating Gender Bias in Language Models Using Causal Mediation Analysis (NeurIPS 2020)

Thank you for taking the time to review our paper and for your insightful comments. Please find our responses to your questions below.

Q1. API access scenarios.

A1. We thank the reviewer for raising this important and practical question regarding the applicability of our method. Our method is primarily designed for users who train/finetune their own models and would like to monitor and diagnose their model when serving their customers. It assumes white-box access to the transformer layer outputs to compute token-level or layer-level causal effects. We do agree that this limits applicability to commercial black-box LLMs. This limitation can be potentially addressed through two ways:

1. Approximate token-level causal effect based on output distribution: In our method, we rely on logits different before and after intervention to compute the casual effect. If the logits are not available, we can replace it with token embedding differences or semantic difference before and after the intervention. Note that this wouldn't work for approximating the layer-wise causal effect.
2. Shadow model approximation: We would train a shadow model based on existing open-source LLMs and use the layer-wise casual effect of the shadow model to approximate the commercial one. We will explore both directions in future work and look forward to extending our method to broader settings when transformer internals are not accessible.

Q2. Resource costs of executing this method in LLMs of different sizes.

A2. We thank the reviewer for highlighting the importance of discussing the resource overhead of our method. To answer the question, we have run experiments to evaluate the overhead of our method.

The efficiency of our method depends primarily on the input length and the number of models layers. Our experiments with an A100 server on the models studied in the submission show that layer-level causal effect computation takes about 0.08 seconds per layer, and token-level computation averages 0.04 seconds per token. Note that analyzing the casual effect of a layer or token takes much less time than generating the response since we only need to generate the first token to conduct the analysis.

To evaluate the overall time overhead, we run the models with and without our method and compare the time. For models with 31 layers, such as Llama-2-7b, with our method, the models take around 6.89 seconds (3.82 seconds is spent on running our method) per input to generate the outputs, whereas without our method, the models take about 3.07 seconds to complete inference process. For models such as Llama-2-13b with 40 layers, it takes around 14.41 seconds (7.73 seconds is spent on running our method) per input with our method, and 6.68 seconds without our method. We remark that this overhead can be significantly reduced if we run our analysis and the original model in parallel, i.e., the overall time becomes 3.82 seconds and 7.73 seconds. The details of detection time are shown in the table below. Specifically, we randomly sampled 100 prompts with an average length of 30 tokens and measured the average detection time and inference time per input across all benchmark models.

In terms of memory overhead, if we run our method and the generation sequentially, there is little overhead; if we run them concurrently, because we need to create an additional model when analyzing the causal effect for each layer and each token (whilst the original model is executing to generate the response), the memory consumption doubles.

We have added a new subsection to discuss the resource costs of our method under the appendix of our revised submission.

Thank you for taking the time to review our paper and for your insightful comments. Please find our responses to your questions below.

Q1. Concerns on LLM is induced to misbehavior (e.g., lie) with instruction.

A1. We thank the reviewer for the careful evaluation and for raising an important question about the definition of misbehavior. In our work, we consider the model to misbehave regardless of whether the behavior is prompted. That is, our detection focuses on the output behavior itself (e.g., generating toxic content), not the intent of the prompt. This design aligns with real-world scenarios like jailbreak attacks (e.g., DAN) that induce harmful outputs through seemingly legitimate instructions. Having said that, we agree that further research on detecting misbehavior considering the context is an extremely interesting one.

Q2. “Empirical results” to support the configuration.

A2. We thank the reviewer for highlighting the need for more empirical evidence to support our design choices and appreciate the opportunity to clarify how these configurations were determined. In response, we have conducted additional experiments on each task and included the results under the appendix in the revised submission.

Our design choices were motivated by both prior research and the need to balance detection accuracy and efficiency. Specifically, we chose to use attention head outputs instead of hidden states for two main reasons. First, prior studies [1, 2] show that attention layers effectively encapsulate the model’s stored knowledge, including harmful or inappropriate content—making attention heads a more direct and sensitive signal for detecting misbehavior. Second, attention head outputs are considerably smaller than hidden states, reducing computational overhead. To further balance detection accuracy and efficiency, we adopt a sparse sampling strategy. For layer selection, we sample three representative layers from the early, middle, and late stages of the model, reflecting the progressive evolution of internal representations [3]. This also loosely mirrors how humans process information, from perception to reasoning. For each selected layer, we sample three attention heads to ensure functional diversity without incurring unnecessary overhead. We have clarified these design choices in the revised submission.

In our empirical evaluation, we compared our method against two baselines to assess the effectiveness of our sparse sampling strategy. The first baseline applies LLMScan with all layers, sampling three attention heads per layer, following our standard head selection. The second baseline uses all attention heads but only from the three selected layers. We conducted experiments on four detection tasks, each with one representative dataset. The results, shown below (token-level detection accuracy), demonstrate that our sampling strategy achieves comparable or better performance while avoiding unnecessary noise:

This confirms that selecting representative layers and heads is sufficient to capture the key causal signals for effective detection.

Reference:

[1] Locating and Editing Factual Associations in GPT (NeurIPS 2022)

[2] On the Role of Attention Heads in Large Language Model Safety (ICLR 2025)

[3] Transformer Feed-Forward Layers Are Key-Value Memories (EMNLP 2021)

Q3. About the understanding of Table 2: toxicity detection.

A3. We thank the reviewer for their careful reading and for pointing out the potential confusion regarding the bold values in Table 2. At table 2, we compare the performance of our method based on token-level causal effect only and layer-level causal effect only, on the same task and the same models. We highlighted the better results. For example, for toxicity detection tasks, the token-level performance for the Mistral model is 1.0 accuracy. Since the accuracy of the layer-level analysis is only 0.98, we highlighted the token-level performance for this case. We have revised the table caption to clarify this.

Q4. Application scenario: real-time detection or retrospective analysis?

A4. We thank the reviewer for raising this question regarding the application scenario. Our method is primarily designed for real-time detection, as it analyzes the LLM’s internal behavior during inference to enable on-the-fly detection of potential misbehavior. We have added a detailed discussion on the intended usage and the computational overhead of runtime detection in the appendix (see our replies to Reviewer 8BQP-Question 2, and Reviewer RtXP-Question 1 for details).

Others

Thanks for pointing out the typos. We fixed all of them in our revised submission.