Antidistillation Sampling

Thank you for your thorough and constructive review. We're encouraged by your positive assessment of our work's practical relevance, empirical rigor, and technical soundness. We appreciate your recognition that this addresses a real challenge for LLM providers and your acknowledgment of our extensive evaluation. We'd like to address your specific concerns and questions.

W1: Impact on traces.

We agree that our procedure affects generated traces—this is indeed the core principle of our algorithm, and we appreciate your observation. However, we'd like to clarify several important points:

Moderate λ preserves interpretability: For practical deployment scenarios with modest performance degradation (~5% in Figure 9), traces remain highly interpretable while providing meaningful antidistillation protection. The first example trace in our paper demonstrates this balance effectively.

Proof-of-concept nature: This work represents the first approach to address controlled distillation poisoning—a problem setting that was previously unexplored. While we agree it would be ideal to have traces indistinguishable from non-poisoned ones, establishing that such poisoning is even possible represents a significant conceptual advance.

Future research directions: We're optimistic about improvements in subsequent work. The dramatic difference between our antidistillation curves and temperature sampling curves (visible across all our tradeoff plots) suggests substantial room for refinement. Similar to how early watermarking techniques were easily detectable but improved over time, we anticipate future work will develop more sophisticated poisoning methods that better preserve trace quality.

W2: How to pick the penalty parameter?

We respectfully suggest that having a tunable hyperparameter is actually an advantage rather than a limitation:

Adaptive security posture: Model providers can dynamically adjust λ based on their assessment of distillation risk, content sensitivity, or user behavior patterns. For instance, they might apply stronger protection for sensitive domains (hacking, military applications, biological warfare, coercion, proprietary business analysis, etc.) while using lighter protection for general queries.

Predictable tradeoff curves: Our experiments demonstrate that the utility-distillability tradeoff follows fairly predictable patterns across tasks. One can accurately estimate the full curve from just a few evaluation points, enabling intelligent λ selection without exhaustive tuning.

Task-agnostic approaches: Future work could use a comprehensive capability set to precompute proxy gradients, then select λ based on this unified representation rather than task-specific tuning. Additionally, small auxiliary models could automatically choose appropriate λ values based on conversation context and estimated security risk.

Q1: Long-Context Settings (AIME 2025, 32K+ tokens)

Intuitively, we suspect that antidistillation will work better in longer context settings, as there is more room to include the "adversarial example" in the nuanced choice of tokens. However, due to our academic compute constraints, we could not investigate benchmarks requiring very long context reasoning. We did try AIME 2024, but the benchmark is significantly compromised (particularly for Qwen models), and the results so high-variance that we felt they had no scientific value. The size of the benchmark is very small as well.

Additionally, in the long context setting, we note that one does not have to use ADS on the entire reasoning trace. It could be used only in critical parts, or fluctuated off and on as necessary. That is, $\lambda$ could be chosen based on context and content, and in some cases $\lambda = 0$.

Q2: Verifree Compatibility

Given that Verifree was published after our submission deadline, we cannot provide definitive experimental validation. However, we note that our method operates primarily through sampling modifications rather than log probability manipulation. While approaches that rely on solution log probabilities might potentially be adapted to work with our poisoned traces, this would require dedicated research to establish effectiveness and represents an interesting direction for future work.

Q3: Model Scale Interactions

We've conducted additional experiments specifically addressing proxy-student size mismatches:

Cross-scale robustness: Using a Qwen2.5-1.5B proxy with Llama-3.2-3B student on GSM8K, we observe strong antidistillation effects across λ values from 0.0674 to 0.811, with student accuracy dropping from 59.88% to 4.599% as teacher accuracy decreases from 91.81% to 19.77%. In this setting, we also have a somewhat more favorable latency tradeoff (840s vs 417s).

Reverse configuration: Using a Qwen2.5-3B proxy with Llama-3.2-1B student demonstrates similarly robust poisoning effects, with student accuracy dropping from 30.85% to 7.527% as λ increases from 0.0811 to 0.191.

These results demonstrate that our method tolerates reasonable size disparities between proxy and student models, addressing practical deployment scenarios where model owners lack precise knowledge of potential student architectures.

Qwen2.5-1.5B Proxy student and Llama-3.2-3B Student on GSM8k

Qwen2.5-3B Proxy student and Llama-3.2-1B student on GSM8k

Q4: Human Study on Utility

We absolutely agree that human evaluation would strengthen our analysis of practical utility. Unfortunately, conducting a rigorous human study within the rebuttal timeframe isn't feasible. However, we view this as a crucial component of future work—systematically evaluating whether users can distinguish between standard and lightly poisoned traces (e.g., at 5% performance degradation levels) would provide valuable validation of real-world applicability.

Broader Impact

We're gratified by your assessment that this work addresses a practically important challenge. The security implications of unrestricted model distillation have received insufficient attention in the literature, and we believe establishing the feasibility of controlled poisoning represents an important step toward more secure foundation model deployment.

While this initial work doesn't solve all practical concerns, it demonstrates that the binary choice between "release traces" and "withhold traces" is false—there exists a middle ground where partial protection is possible. This opens up new research directions and provides foundation model developers with previously unavailable options for protecting their intellectual property while maintaining service quality.

Conclusion

We're grateful for your thorough review and particularly appreciate your recognition of this work's practical importance and technical soundness. We believe our responses have effectively addressed your concerns:

1. Solution quality remains acceptable at practical operating points, with clear improvement trajectories for future work
2. Hyperparameter selection offers valuable adaptive control, with predictable tradeoff patterns enabling practical deployment
3. Cross-scale generalization is empirically demonstrated through our additional experiments
4. Long-context applicability appears promising based on theoretical considerations, though resource constraints limited full evaluation

Given our comprehensive responses and the additional experimental validation we've provided, we respectfully hope you'll consider increasing your rating. We believe this work represents a solid contribution that will catalyze important future research in foundation model protection—an increasingly critical area as these models become more valuable and ubiquitous.

Thank you again for your constructive engagement with our work.

Thank you for your thoughtful review of our work on antidistillation sampling. We appreciate your engagement with the technical details and practical considerations. We'd like to address each of your concerns systematically.

W1: Hyperparameters are task dependent.

We respectfully disagree that task-dependent hyperparameters represent a fundamental limitation. Rather, we view this as a feature, not a bug--providing model providers with precise control over their security posture.

Model providers can adaptively choose λ based on conversation content or context, applying stronger antidistillation for sensitive domains (e.g., military applications, private data analysis) while using lighter protection for general queries.

Our experiments show that the tradeoff curve behavior is quite predictable—one can accurately estimate the full curve with far fewer evaluation points than we plotted, enabling intelligent λ selection.

We outline clear paths forward, including using MMLU performance to predict optimal hyperparameter ranges across tasks, and developing classifiers to select appropriate λ values automatically.

This tunability mirrors established security practices--differential privacy requires choosing ε and δ, watermarking requires choosing strength parameters, and adversarial robustness requires choosing perturbation magnitudes. The security community has successfully operationalized such parameterized defenses.

W2: Cost-benefit analysis.

This is the first work in what we hope becomes a productive research direction, and we don't expect to solve all practical concerns in one paper. However, several factors make the tradeoff more favorable than initially apparent:

Selective deployment: As mentioned above, ADS need not be applied universally. Model providers can activate it only when they suspect distillation attempts or for particularly sensitive content, serving as a form of adaptive throttling.

Computational overhead: In the very worst case, our sampling method makes inference $3\times$ more expensive in exchange for protecting models from distillation attacks. This is still something model providers care about, and future work can further reduce this overhead. In the more realistic case, when using smaller proxy models with large teacher models, the proxy computation cost becomes a smaller fraction of the total inference budget. We collected some timing information for our experiments with the 3b proxy. In the setting of GSM8k trace generation, AD sampling took 1008s while plain sampling took 428s -- that is, our method is just a bit more than two times slower than normal sampling. We expect future research to cut this overhead down even further.

Security-critical applications: Many deployment scenarios exist where security justifies performance costs—consider hacking, military applications, biological warfare, coercion, etc., where protecting model capabilities outweighs latency concerns.

It was NOT obvious that our method would work as well as it does, given the relatively complicated technical machinery under the hood. Our work is the first to show that anti-distillation is even possible; currently, there is no way to really protect against distillation attacks without holding back the traces or explicitly blocking adversarial accounts. Our results are statistically significant, in that our method significantly and obviously changes the shape of the tradeoff curve; this makes us optimistic that there is room for more methods in this direction. We hope our work will stimulate further research in this area to try and improve upon our results, leading to a more practical application of our method.

W3: Model owners could not release traces.

In principle, we agree; a model owner could decide not to release traces. However, as things stand, model owners (e.g., OpenAI, Anthropic, Google, etc.) do release traces, and have not documented any plans to stop this practice, meaning that distillation from these models will remain possible for the foreseeable future. On the other hand, if our paper results in changing the practice of frontier model providers, we would consider this a significant impact of our work. It's also worth pointing out that companies probably do want to release traces, because not doing so puts them at a disadvantage relative to other model providers: it's harder to get feedback, billing becomes less transparent since users wouldn't get all the tokens they used, users like to understand why certain decisions were made (especially in, e.g, software engineering) -- so hiding traces could lead to backlash. We hope for transparency and protection at the same time.

To answer your question more specifically, we agree that it would be more desirable if our algorithm induced traces that are indistinguishable from non-poisoned traces. However, we strongly emphasize that our algorithm is the first to address this problem setting. In this way, an exciting direction for future research would be to render poisoned traces indistinguishable from non-poisoned ones. However, in our view, the fact that our algorithm did not address every possible concern does not make it unworthy of publication.

Also please note: as seen in Fig. 9, for relatively small amounts of antidistillation (around ~5% degradation), the traces are not heavily impacted. You can see this in the first example trace listed in the paper.

Q1: Sensitivity to proxy/attacker capability differences.

We've conducted additional experiments specifically to address this concern:

Cross-size robustness: Using a Qwen2.5-1.5B proxy with Llama-3.2-3B student on GSM8K, we observe robust antidistillation effects across λ values from 0.0674 to 0.811, with student accuracy dropping from 59.88% to 4.599% as teacher accuracy decreases from 91.81% to 19.77%. In this setting, we also have a somewhat more favorable latency tradeoff (840s vs 417s).

Reverse configuration: Using a Qwen2.5-3B proxy with Llama-3.2-1B student shows similarly strong poisoning effects, with student accuracy dropping from 30.85% to 7.527% as λ increases.

These results demonstrate that our method tolerates reasonable size mismatches between proxy and student models, addressing practical deployment scenarios where exact architectural knowledge is unavailable.

Qwen2.5-1.5B Proxy student and Llama-3.2-3B Student on GSM8k

Qwen2.5-3B Proxy student and Llama-3.2-1B student on GSM8k

Broader Perspective

Resource constraints: As academics, we cannot afford to run experiments at the scale of frontier models (100B+ parameters, million-token contexts). However, we've demonstrated the core principle across multiple benchmarks, architectures, and scales within our resource constraints. The research community often advances through such incremental contributions that industry can then scale.

Statistical significance: Our results show clear, statistically significant degradation in student performance across multiple settings, confirming that antidistillation is not only possible but practically achievable.

Research impact: Even if this specific implementation isn't immediately production-ready, demonstrating that controlled distillation poisoning is feasible represents a significant step forward in understanding model security. We hope this work catalyzes further research toward more efficient and practical solutions.

We believe the evidence strongly supports the value of this research direction, even as we acknowledge areas for future improvement. The field advances through such foundational contributions that establish feasibility and point toward promising research directions.

Conclusion

We sincerely appreciate the time and effort you've invested in reviewing our work. We believe our responses demonstrate that:

1. Task-dependent hyperparameters provide valuable adaptive security control rather than representing a fundamental limitation
2. Cost-benefit tradeoffs are reasonable for security-critical applications, with clear paths for improvement in future work
3. Reasoning trace quality remains acceptable at practical operating points, with competitive dynamics preventing simple trace withholding

Most importantly, we've provided additional experimental evidence showing robust cross-architecture and cross-scale generalization, directly addressing your questions about capability gaps and larger student models.

While we acknowledge this is foundational work with room for improvement, we respectfully submit that demonstrating the feasibility of controlled distillation poisoning--previously thought impossible--represents a significant contribution to model security. The dramatic performance differences between our method and temperature sampling across multiple benchmarks provide strong evidence of practical utility.

Given our thorough responses to your concerns and the additional experimental validation we've provided, we hope you'll consider revising your assessment. We believe this work merits acceptance as an important first step in protecting foundation model intellectual property while maintaining service quality.

Thanks for your review and constructive comments. We're glad that you find our paper has clear practical significance, is exceptionally well-written, and features logical technical derivation. We provide additional experimental evidence and discussion to address your concerns.

Concerns regarding originality

You raise an important question about our algorithmic contribution. We believe our work introduces several genuinely novel insights that extend well beyond "skillful synthesis" of existing techniques.. As far as we are aware, no prior algorithms address the setting we consider, and our derivation produces a truly novel solution to an important problem.

It's worth noting that every algorithm builds upon existing ideas. One could argue that the Fast Fourier Transform isn't innovative since it applies divide-and-conquer to the Discrete Fourier Transform, yet we rightfully consider FFT a fundamental breakthrough. The key insight—recognizing recursive structure—was neither obvious nor trivial to discover.

Consider controlled decoding, which you mentioned in your review. Existing methods typically steer generation toward attributes like sentiment or safety using auxiliary models. In contrast, our objective centers on simulated distillation training steps of a proxy model to poison traces for unknown student models. While our method can be seen as controlled decoding, this problem formulation and solution approach are, to our knowledge, completely novel.

Two innovations deserve particular emphasis. First, no prior work demonstrated that perturbations computed for one architecture would transfer effectively to completely different target architectures. This cross-architecture generalization was theoretically uncertain and empirically unproven. Second, our gradient precomputation technique solves a fundamental computational challenge by deriving a method to precompute gradients once rather than computing them at every decoding step, making the approach computationally tractable through careful finite-difference approximation.

The method's effectiveness was far from guaranteed. We made several strong assumptions: that proxy models would generalize to unknown students, that single-step gradient approximations would suffice for complex optimization landscapes, and that our finite-difference approach would preserve essential gradient information. The fact that these assumptions hold together--producing the dramatic shift in trade-off curves we demonstrate--represents a surprising empirical phenomenon that advances our understanding of model vulnerabilities and defenses.

This is, at minimum, a fascinating discovery that opens new research directions. The clear effectiveness of our approach suggests that even better methods may exist, making this an exciting area for future investigation.

Practicality

You correctly identify computational overhead as a key concern. In the very worst case, our sampling method makes inference $3\times$ more expensive in exchange for protecting models from distillation attacks. This is still something model providers care about, and future work can further reduce this overhead. In the more realistic case, when using smaller proxy models with large teacher models, the proxy computation cost becomes a smaller fraction of the total inference budget. Our method proves particularly practical in the regime of ~5% reductions in teacher performance (Figure 9), where the traces are also more comprehensible (see the first trace listed in the paper).

We collected some timing information for our experiments with the 3b proxy. In the setting of GSM8k trace generation, AD sampling took 1008s while plain sampling took 428s -- that is, our method is just a bit more than two times slower than normal sampling. We expect future research can cut this down.

Our new experiments demonstrate effectiveness across different proxy-student size combinations. A Qwen2.5-1.5B proxy successfully poisons Llama-3.2-3B student distillation, suggesting that using smaller proxies can significantly reduce computational overhead while preserving protection efficacy.

Importantly, antidistillation sampling enables selective deployment rather than universal application. Model providers can activate the method through the tunable λ parameter when detecting suspicious usage patterns or serving particularly sensitive content. This targeted approach eliminates overhead during normal operation while providing protection when most needed--essentially functioning as an intelligent throttling mechanism.

We view this as establishing a new research direction rather than providing the final solution. Future work could explore training teachers to inherently produce antidistillation traces or developing specialized heads for efficient poisoning, potentially eliminating the computational overhead entirely.

Scope of evaluation

Your points about experimental breadth are well-taken. Within academic compute constraints, we focused on demonstrating our core hypothesis: that antidistillation sampling can effectively poison distillation attempts. Our cross-architecture results (Qwen teacher/proxy with Llama student) address the most critical practical question--whether the method works when attackers use different architectures than the defender anticipates.

We have reasonable evidence that our proxy setup works across architectures -- as you pointed out, this is the main result we show in the paper. Qwen and Llama represent genuinely different architectures trained on distinct datasets, providing meaningful evidence for generalization. We believe there is generally representation similarity across architectures; they're trained on similar data with similar objectives, ultimately using stacks of similar layers. Even GRU or SSM models can be seen as riffs on linear attention. It has long been known that adversarial examples often transfer across models [1, 2]. The representational similarity across models has also been noted [3].

While we acknowledge that broader evaluation would strengthen our claims, we believe our evidence sufficiently demonstrates the method's viability to warrant community attention and follow-up research.

Passive threat model

The adaptive adversary question represents an excellent direction for future research. Our work establishes the fundamental feasibility of trace poisoning--a necessary first step before studying arms race dynamics. Starting with the simplest possible setting and solution has revealed promising initial results that justify deeper investigation.

We expect adaptive defenses will emerge, potentially through statistical anomaly detection or robust training techniques. However, our method's dynamic nature--using gradients from a hidden proxy model with secret evaluation loss--creates a moving target similar to stream ciphers in cryptography. This fundamental property should make detection and countermeasures significantly more challenging than static watermarking approaches.

The cat-and-mouse game between poisoning and detection methods will likely drive important innovations in model security. Our contribution provides a concrete foundation for this research direction.

Generalization evidence

This paper was quite expensive for us as academics. We can't afford to run all these experiments. We have shown that our idea works and presented it clearly. The next obvious question is whether it generalizes beyond ~10b models and for much longer traces. We simply cannot do experiments at this scale; what recourse do we realistically have? See above.

We've conducted a few additional experiments to address cross-size generalization concerns. Using a Qwen2.5-1.5B proxy with Llama-3.2-3B student on GSM8k, we observe strong antidistillation effects. The reverse configuration (Qwen2.5-3B proxy with Llama-3.2-1B student) shows similarly robust poisoning effects.

These results demonstrate that our method tolerates reasonable size mismatches between proxy and student models, addressing practical deployment scenarios where exact architectural knowledge is unavailable. In the setting where we use a smaller proxy, we also have a somewhat more favorable latency tradeoff (840s vs 417s).

Qwen2.5-1.5B Proxy student and Llama-3.2-3B Student on GSM8k

Qwen2.5-3B Proxy student and Llama-3.2-1B student on GSM8k

Summary

We agree there's substantial room for future work and hope to catalyze research in this important area. This represents the first paper addressing antidistillation through sampling strategies, and our results demonstrate clear feasibility with quantifiable trade-offs. The dramatic difference between our trade-off curves and temperature sampling baselines indicates we've identified a genuinely effective approach to an important security problem.

While no first paper in a new area achieves perfect generality or eliminates all limitations, our contribution establishes both the problem's tractability and a concrete solution that others can build upon. We hope to start the scientific conversation by presenting it at this venue.

[1] The Space of Transferable Adversarial Examples. Tramer et al.

[2] Explaining and Harnessing Adversarial Examples. Goodfellow et al.

[3] The Platonic Representation Hypothesis. Huh et al.

Dear Reviewer,

The deadline for the author-reviewer discussion is approaching (Aug 8, 11.59pm AoE). Please read carefully the authors' rebuttal and engage in meaningful discussion.

Thank you, Your AC