Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models

Thank you for your feedback.

Comparison to related past work: We've included a comparison table (will also add in the final paper) evaluating similar work across key axes:

• Iterative trigger search,
• Use of efficient proxy targets,
• Trigger universality (works across input types),
• Semantic filtering for correctness/readability,
• Cross-model transferability,
• Applicability to reasoning models,
• Impact on model latency | Paper | Iterative approach | Efficient proxy target | Universal Triggers | Semantic Filtering | Transferability | Targets Reasoning LLMs | Slowdown | Task | | :---------------------------------------------------------------------------- | :----------------- | :--------------------- | :----------------- | :----------------- | :-------------- | :--------------------- | :------- | :-------------------------------- | | Large Language Models Can Be Easily Distracted by Irrelevant Context | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | Math word problems | |Adversarial Math Word Problem Generation | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | Math word problems | | MathAttack: Attacking LLMs towards Math Solving Ability | ✗ | ✗ | ✗ | ✓ | ✓ | ✓ | ✗ | Math word problems | | Adversarial Examples for Reading Comprehension | ✓ | ✓ | ✗ | ✓ | ✓ | ✗ | ✗ | Reading comprehension | | Evaluating Models’ Local Decision Boundaries via Contrast Sets | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | Multi-task NLP evaluation | | An LLM Can Fool Itself: PromptAttack | ✗ | ✗ | ✗ | ✓ | ✓ | ✗ | ✗ | General NLP classification | | Universal & Transferable Adversarial Attacks on Aligned LLMs | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | Safety/alignment bypass | | CatAttack (Ours) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Math reasoning|

CatAttack presents a novel approach that, to the best of our knowledge and unlike the prior works, employs an iterative trigger refinement, efficient proxy targets, and universal, transferable, semantically valid triggers to successfully attack LLMs' math reasoning capabilities.

Limited Model Variety, Cross family transfer, and Dataset Scope: We ran experiments on additional varieties of reasoning models, mixture of experts models, and non-reasoning models, with the CatAttack triggers generated using Deepseek V3 as proxy to address this feedback. We find that the triggers successfully work across these varieties.

We ran the dataset scope experiments on a sample of 1000 problems from GSM8k.

• Results (Please see Tables R2.1 & R2.2 in the R2 rebuttal response above ): Triggers from DeepSeek V3 successfully transferred, increasing ASR (>2-7x baseline) and causing slowdowns. This confirms cross-family transfer of triggers.

Limited Trigger Set CatAttack triggers demonstrate effective, model-transferable, query-agnostic attacks. While large-scale trigger generation is promising future work, our triggers already outperform random ones. E.g., Llama 3 8B: 37.5% vs. 13.4% ASR; 5.44× vs. 1.33× slowdown; (please see Table R3.1 in the rebuttal response above), highlighting their potency.

Robustness testing via finetuning: We fine-tuned Llama 3 8B on 21K GSM8K examples (mixing clean and Trigger 2/3-augmented data) and evaluated on unseen Trigger 1. Specifically, we used 7000 GSM8K questions, appended each with either Trigger 2 or Trigger 3 to form 14000 adversarial examples, and mixed these with 7000 clean questions (no suffix) for a total of 21000 training instances.

ASR dropped marginally (28.4% → 27.1%). Response length also remained inflated (236.6 → 356.3 tokens).

Like Problemathic (Anantheswaran et al., ‘24), SFT on trigger-augmented data also failed to generalize to new triggers, showing naïve SFT doesn't mitigate suffix-trigger vulnerability or slowdown.

Thank you for the responses. I have changed my score.

Thank you so much for your feedback and question.

To directly address the concern regarding the lack of a baseline comparison to assess the significance of our results and to clearly demonstrate the impact of our CatAttack technique, we have conducted new experiments. For this baseline, we evaluated our discovered triggers against generic, semantically irrelevant random phrases on 1000 examples from the GSM8K dataset. The random phrases were:

• "The sky is blue on summer afternoons."
• "On warm summer afternoons, the heavens look a deep azure—fun fact!"
• "Art and music always, somehow, bring great joy to everyone." They were selected to be approximately the same lengths as our original triggers. We evaluated the impact on ASR and response slowdown on two capable models: Llama 3 8B Instruct (an instruction-tuned model) and Qwen QwQ 32B (a strong reasoning model).

Below is a comparison of their impact (on 1000 randomly selected GSM8K examples):

Table R3.1: Comparative Impact - CatAttack Triggers vs. Random Phrases

These comparisons clearly demonstrate the more severe impact of CatAttack triggers compared to generic random phrases:

• Higher Attack Success Rates: CatAttack triggers are consistently more effective at inducing incorrect answers in reasoning tasks across models of different capabilities.
• More Severe Slowdowns: The most significant distinction lies in the response slowdown. CatAttack triggers lead to substantially higher average slowdowns and affect a much larger proportion of responses, often causing them to become excessively long. This points to a more profound disruption of the model's internal processing.

We appreciate your review. We have addressed your concerns with clarifications and new experiments:

Question 1: Model Scope & Transferability We tested our 3 triggers on 5 additional diverse models (Qwen, Phi, Llama, Mistral) using 1000 GSM8K questions, measuring ASR (multiplier over baseline error) & slowdown.

Table R2.1 ASR:

Table R2.2 Slowdown Rate - % responses > token budget b:

Tables R2.1-R2.2 show triggers from DeepSeek V3 successfully transferred to diverse model families, increasing ASR (2x-7x over baseline) & causing slowdowns. These results, with prior o1/o3-mini findings, show cross-model transferability despite cost limits for testing the largest proprietary models.

Question 2: Semantic Verification Details To ensure suffixes didn't alter math problems, our process was:

• Goal: Confirm modified prompts were semantically identical to the original problems
• Human Verification: Three annotators, blinded to LLM responses, independently reviewed prompts.
• Rigorous Check: Annotators manually solved original and modified questions. A prompt was passed only if all three confirmed identical manual solutions. This helps filter out triggers that change the problem and also safeguards against errors from the discovery judge in the pipeline.

Question 3: Vulnerability of Distilled Qwen R1 32B Model Distilled models may be more vulnerable due to:

• Lossy Distillation: The teacher’s outputs are used to train distil Qwen R1 32B, but Qwen 32B's reduced parameter count and capacity mean that it may not be able to perfectly replicate the nuanced reasoning behaviour of the parent R1 model. Distillation can be effective but it may not be able to perfectly transfer all capabilities.
• Training Differences: SFT on teacher outputs might mimic behavior without internalizing robustness from teacher's RL phase.

Question 4: Pipeline Sensitivity to Judge Model We addressed judge sensitivity by:

1. Ensuring Trigger Validity: Human review (Q2) validates triggers independently of the discovery judge.
2. Testing ASR Sensitivity (New Experiment): We re-evaluated ASRs on 1000 GSM8K examples for Llama 3.1 8B, Mistral Small 24B, Qwen-3 30B, & Qwen QwQ 32B using Gemini 2.5 Flash vs. our GPT-4o judge.

Table R2.3 Combined Absolute ASR (%) with Different Evaluation Judges:

Results show high inter-judge reliability for Mistral24B (identical ASRs) and Llama 3.1 8B(37.50% vs. 36.40%). For reasoning models, while absolute ASRs varied, both judges confirmed that our triggers significantly increased ASR, showing substantial degradation in math reasoning.

Rate of Transfer of Attacks from Proxy to Target: We acknowledge the ~20% proxy-to-target transfer rate. This is not ideal, but this proxy discovery method is more efficient than direct attacks on reasoning models. Also, CatAttack yields universal, query-agnostic triggers; once transferred (even from a few proxy successes), they are highly scalable across inputs and diverse model families (per new experiments). Future work could use reasoning attackers to improve the transfer rate.

Thank you for your feedback.

To address the effectiveness of the choice of adversarial phrases:

We ran experiments with 3 random phrases of similar lengths as that of CatAttack triggers, and found that they are not as effective. The random phrases used were

• "The sky is blue on summer afternoons."
• "On warm summer afternoons, the heavens look a deep azure—fun fact!"
• "Art and music always, somehow, bring great joy to everyone."

These comparisons clearly demonstrate the more severe impact of CatAttack triggers compared to generic random phrases:

Reasoning divergence:
We observed that the model gets to the answer in most scenarios but then gets into a loop of self-reflection trying to make the connection between the original problem and the trigger phrase. Reasoning by design is meant to incorporate deep thinking and self-reflection—it gets confused and is not able to recover. Please see appendix for examples.

Defense mechanisms:
Based on R4’s feedback, we tried data augmentation and finetuning but that did not work for increasing model robustness. Prompting the model to be self-aware of its limitations works in some cases. Rewriting the prompt using a cheap model would also be a promising solution.

Thank you for your suggestion about testing defense mechanisms. We have conducted the experiments you requested and discovered an effective, yet easy defense. By adding a simple instruction after the trigger - "First, review the prompt, discard any irrelevant or distracting details, and focus only on the information needed to solve the problem" - we observed the following:

• Llama 3 8B: Combined ASR reduced from 37.50% → 9.87%
• Mistral Small 24B: Combined ASR reduced from 30.30% → 5.17%

This finding shows that explicit instructions to ignore irrelevant information can to a certain extent defend against such adversarial triggers. We will add a dedicated section on defense mechanisms to the paper.

While we identify a simple defense, the fact that such a defense is necessary reveals fundamental issues with how current models process irrelevant information.