Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs

Dear reviewer, thank you for your recognition of our work, your careful review of our paper, and your valuable feedback on the ICRT method. Below are our responses to your comments, as well as our plans for future work.

(I) Regarding the Causal Link Between LLM Vulnerabilities and Human Cognitive Heuristics:

We have supplemented detailed ablation experiments to verify the contribution of the intent recognition and concept decomposition modules to the attack success rate. The experimental results show that after removing these components, the attack success rate significantly decreases (e.g., on GPT-4-0613, after removing intent recognition and/or concept decomposition, the ASR drops from 96% to 0% or close to 0%). These data support our argument at the behavioral level, showing that by leveraging the "simplicity effect" and "relevance bias," we can effectively reduce the risk of triggering security detection.

$\dagger$: Intent Rec. + Concept Dec. could not generate human language prompt.

Our jailbreak success criteria are as follows: the attack is considered successful only when it passes through GPT-4 and is manually verified, ensuring that the task is fully bypassed. Additionally, in future research, we will introduce more detailed internal mechanism analyses to further verify whether LLMs exhibit behavior similar to the human "simplicity effect" when processing prompts with low complexity decomposition. We believe these supplementary analyses will provide stronger causal evidence for our theory.

(II) Regarding Reproducibility and Preventing Misuse, we will take the following measures:

1. Code Release and Documentation: We commit to releasing a secure, filtered version of the code and detailed experimental documentation after the paper is published, ensuring that other researchers can reproduce our results in a controlled environment.

2. Security Filtering Mechanisms: When releasing the code, we will integrate necessary security measures and filters to prevent the code from being used for malicious purposes. For example, we will include security warnings for generated outputs and embed an automatic review mechanism in the code.

3. Controlled Access and Collaboration Mechanism: We plan to collaborate with trusted third-party platforms or organizations to establish controlled access, ensuring that researchers use our tools within a secure and regulated framework, while preventing misuse for malicious attacks. At the same time, we will report our successful findings to LLM vendors to encourage them to strengthen their defense mechanisms and jointly advance the security of LLMs.

We take reproducibility and security issues very seriously and will continue to improve these measures in our future work to ensure that our research advances academic progress while minimizing the potential for misuse.

Once again, thank you for your recognition of our work and your constructive suggestions. We will further improve the experimental design and theoretical discussions, aiming to enhance the scientific and practical value of our method, as well as ensure the reproducibility and security of our research results. We look forward to receiving more guidance and support in future communications.

Dear reviewer, we greatly appreciate your feedback, which has helped improve our work. Below are our responses to your suggestions and concerns.

(I) Analysis of Failure Cases:

To better understand failure cases and analyze the model's thought process, we conducted experiments with some of the latest models that possess deeper reasoning capabilities and analyzed their reactions to malicious prompts. Specifically, we delved into the thought process of these models to further uncover the reasons for the failures. Below are the comparison results with other advanced attack methods:

Example of a Failed Attempt to Promote Violence:

First, when the model receives the request, it identifies that the request contains potential harmful content. Deepseek’s thought process is: "This request involves harmful or sensitive content, which could lead to social unrest or violent behavior. Processing this request may result in serious consequences." Next, the model performs a risk assessment, thinking: "Generating such content may lead to real-world violence or harm, so it must be avoided." Subsequently, the model activates its security mechanisms based on its ethical guidelines and content safety standards, refusing to continue processing the request. The model’s thought process is: "According to my ethical guidelines, I cannot continue with this request. I must prevent generating any potentially harmful output." Finally, the model outputs a rejection message: "Sorry, I cannot answer this question" or "This request violates my usage policy," and believes: "I have detected the harmful nature of this prompt and must refuse to generate it to protect users and adhere to ethical standards."

Failure Analysis: The failure of the model is primarily attributed to its robust security mechanisms and deep reasoning capabilities. When malicious prompts approach the model's boundaries, the model successfully identifies and intercepts these requests through its built-in defense mechanisms. Its deep reasoning not only evaluates the content of the text but also considers the long-term impact of the request, preventing potential societal harm. Additionally, Deepseek has undergone adversarial training, which enhances its robustness, making attack methods unable to successfully breach its defenses.

(II) Improvement of Ranking Evaluation Metrics

To overcome the traditional binary approach that simply judges whether a jailbreak attack is successful or not, we introduce a ranking-based evaluation method. First, we generate jailbreak texts using different attack methods and select those that successfully bypass the security mechanisms as input for evaluation. Then, we perform pairwise comparisons of these texts using multiple LLMs to assess which text in each pair is more harmful, thus accumulating a large amount of pairwise comparison data. Finally, we apply Elo Ranking, HodgeRank, and Rank Centrality algorithms to this data to generate global harm rankings for each attack method (Figure 4 ).

The advantage of this method is that it not only evaluates whether the attack is successful but also analyzes the harm of each text, providing a more comprehensive assessment of the attack’s effectiveness.

Meanwhile, to further demonstrate the excellent performance of our method in generating harmful jailbreak texts, we added comparisons with the most advanced attack methods.

(III) Comparison of Efficiency for Cognitive Decomposition Technique:

We greatly appreciate this suggestion and below is the efficiency comparison, showing the time and number of searches required to generate a query for each method:

Thank you once again for your valuable feedback and suggestions. We look forward to your continued guidance and are happy to provide any additional information if needed.

Dear reviewer, thank you for your review and valuable feedback on our work. Below are our responses to your comments.

(I) Regarding the Limited Novelty: We appreciate the reviewer’s insightful comments. ICRT introduces significant innovations that advance the field, specifically:

1. Difference with [1]: The most significant difference between ICRT and [1] is the way the attacker interacts with the target model. [1] is a multi-round method, which queries each decomposed object individually to the target model to test if the defense mechanism can be triggered. Meanwhile, ICRT is a single-round method, where each decomposed sub-concept is not individually queried by the victim. Reducing the number of interactions effectively reduces the likelihood of an attack being detected. Moreover, our framework goes beyond traditional step-by-step decomposition methods [1] by incorporating cognitive biases such as the "simplicity effect" and "relevance bias" into the attack design. This cognitive bias-based attack optimization approach is novel and has not been widely explored in the literature.

2. Difference with [2]: [2] (similar to [6] of Reviewer HEDd) uses malicious intent as the purpose of the role to deceive the victim through role-playing. However, role-playing is just one of the options for modifying the jailbreak prompt to be closer to human language in ICRT. We can also integrate the proposed method with hypothetical discussions or virtual background creation, adapting it to different contexts and models. The success of ICRT is not entirely dependent on role-playing alone. We conducted ablation experiments, and the results indicate that concept decomposition without multi-interaction plays a crucial role in the success of ICRT.

3. Evaluation Framework: We propose a comprehensive evaluation framework that goes beyond simply measuring success or failure. Instead, our framework quantitatively assesses the harm of generated text through ranking aggregation. Specifically, we perform pairwise comparisons of outputs to capture subtle differences in harmfulness, thereby providing a more detailed and objective metric for evaluating risk.

$\dagger$: Intent Rec. + Concept Dec. could not generate human language prompt.

(II) Regarding Verification on SOTA LLMs:

We appreciate the reviewer’s feedback on the validation of SOTA LLMs in our study. In our supplementary experiments, we not only validated the effectiveness of ICRT on models released in 2023 but also included tests on GPT-O1 and deepseek series models, which represent the latest advancements in LLMs. Below are the experimental results comparing ICRT with other recent jailbreak methods on these models:

This demonstrates that ICRT is effective in bypassing the security mechanisms of these newly released models.

(III) Regarding Prompt Sensitivity Analysis:

In our approach, prompts play a crucial role, so we conducted several experiments to analyze the sensitivity of prompts to ensure the robustness of our attack strategy. First, to analyze the impact of different prompt designs on the attack effectiveness, we performed ablation experiments. As shown in (I), the results illustrate the effect of variations in prompt structure and content on the attack success rate. The ablation experiments indicate that intent recognition and concept decomposition are essential for improving the attack success rate.

We also conducted multiple jailbreak attempts, performing 15 independent attack trials for different models to evaluate the impact of the prompts on the jailbreak success rate. The results are as follows:

We hope these experiments and analyses address the reviewer’s concerns regarding prompt sensitivity and provide further support for the effectiveness of our method.

(IV) Regarding Citations of Related Work: We greatly appreciate this suggestion and will include these relevant references in the revised version to ensure our research is more comprehensively aligned with the current advancements.

We thank the reviewer for their valuable feedback on our paper. If the reviewer has any further questions or suggestions, we would be happy to provide more details and data.

Dear reviewer, thank you for your in-depth evaluation and valuable feedback on our paper. Below is our detailed response to the comments and suggestions you provided.

(I) Regarding the Limited Novelty: We appreciate the reviewer’s keen observation regarding the similarities between our method and existing work ([5-6]) in attack decomposition and role-playing prompts. ICRT achieves significant innovations on several key points and advances the field. Specifically:

1. Difference with [5]: The most significant difference between ICRT and [5] is the way the attacker interacts with the target model. [5] is a multi-round method, which queries each decomposed object individually to the target model to test if the defense mechanism can be triggered. Meanwhile, ICRT is a single-round method, where each decomposed sub-concept is not individually queried by the victim. Reducing the number of interactions effectively reduces the likelihood of an attack being detected.

2. Difference with [6]: [6] use of malicious intent as the purpose of the role to deceive the victim through role-playing. However, role-playing is just one of the options for modifying the jailbreak prompt to be closer to human language in ICRT. We can also integrate the proposed method with hypothetical discussions or virtual background creation, adapting it to different contexts and models. The success of ICRT is not entirely dependent on role-playing alone. We conducted ablation experiments (see IV), and the results indicate that concept decomposition without multi-interaction plays a crucial role in the success of ICRT.

3. Evaluation Framework: We propose a comprehensive evaluation framework that goes beyond simply measuring success or failure. Instead, our framework quantitatively assesses the harm of generated text through ranking aggregation. Specifically, we perform pairwise comparisons of outputs to capture subtle differences in harmfulness, thereby providing a more detailed and objective metric for evaluating risk.

(II) Regarding the Obsolescence of Baselines: We have supplemented our experiments with a comparison against the jailbreak attack methods released at the end of 2023 and in 2024, and added experiments on new large models. ICRT gets the best result on DeepSeek and ChatGPT-O1.

(III) Regarding the impact of adversarial training on ranking consistency: We appreciate the reviewer’s question about adversarial training and ranking consistency. We have addressed this in our experiments, and here is our response:

1. Adversarial Training and Ranking Consistency: Your question about whether adversarial training affects the consistency of rankings across models is insightful. Adversarial training affects the success rate of jailbreaks, but our ranking method is applied after a successful jailbreak, using the same criteria. In other words, the ranking is based on a set of already successful jailbreak texts, and pairwise comparisons are made based on their harmfulness, ensuring that there is no inconsistency before and after adversarial training.

2. Pairwise Harmfulness Comparison and Global Ranking: We conducted 1260 pairwise comparisons, where multiple LLMs voted on which of two jailbreak texts from different methods was more harmful. The comparison results were then aggregated using an algorithm to obtain a ranking of the attack methods based on their harmfulness (see Figure 4).

3. Further Experimental Validation: We compared the harmfulness of jailbreak texts from the four baseline methods, the results are as follows:

As shown in the table, ICRT outperforms the others on all three metrics, further validating our method’s advantage in generating harmful jailbreak texts.

(IV) Regarding Ablation Studies: The experiments have been conducted, and the results are as follows.

$\dagger$: Intent Rec. + Concept Dec. could not generate human language prompt.

Thank you again for your recognition and suggestions. If you have any further questions or need clarification, please feel free to reach out. We will keep improving our research and look forward to your continued guidance.