TokenSwap: A Lightweight Method to Disrupt Memorized Sequences in LLMs

We thank the reviewer for their strong support of our paper and constructive feedback. We appreciate the reviewer's recognition of the important problem TokenSwap addresses, its strong empirical results, and its practical applicability.

W1: The captions of the tables are inadequate

Thank you for this feedback. In the revised version, we will expand the table captions to clearly describe the models used, experimental setup, and key details needed to understand the results.

Nit 1: On line 267, a citation is broken

Thank you for pointing this out. We will fix the citation in the revised version.

Extension 1: Does speculative decoding reduce memorization?

Thank you for this interesting question. We test speculative decoding on the Pile memorized dataset using Pythia-6.9B as the main model and Pythia-70M as the auxiliary model for both TokenSwap and speculative decoding. For speculative decoding, we set $\gamma$ as 5.

Pile-Memorized Dataset Results (with speculative decoding):

Speculative decoding improves over standard decoding but still shows $4 \times$ higher exact match rate than TokenSwap and higher memorization on approximate metrics (ROUGE-L, Levenshtein). We hypothesize this occurs because: (1) the large model selects tokens from the small model based on its own likelihood, preserving memorization potential, and (2) the large model frequently generates tokens especially when the small model produces low-likelihood candidates.

We thank the reviewer for their valuable feedback and positive assessment of our paper. We appreciate the reviewer recognizing our method's practicality, strong empirical results, and clarity of presentation.

W1: Limited applicability due to fixed swap-token set (e.g., no code or multilingual tokens)

Thank you for this feedback. We acknowledge this limitation in Section 5. We are excited about future work extending our approach to code and multilingual settings. The key challenge for domains like code is whether auxiliary models can provide syntactically correct but different alternatives, similar to what we achieve in natural language.

While we focus on natural language, a recent study [1] found that 76% of employee LLM use cases involved natural language while only 8% involved coding. Though specific to one organization, this suggests natural language covers a substantial fraction of practical use cases.

W2: Risk of performance degradation for certain queries such as "Please only revise the format of my text."

Thank you for this question. We do not expect TokenSwap to degrade performance in tasks requiring minimal content changes, such as reformatting. In such cases, both the primary and auxiliary models would likely agree on the next token since the text to be repeated is already in the context. Additionally, transformers are known to excel at text repetition through mechanisms like induction heads [2].

To test this empirically, we created 50 markdown-formatted sequences and prompted LLaMA-3-8B to convert them to plain text. We compared standard decoding with TokenSwap (using SmolLM-135M as the auxiliary model). In all 50 cases, the outputs were identical, confirming no performance degradation for such formatting tasks.

W3: TokenSwap may not be robust to certain adaptive attacks (e.g., character-by-character prompting)

We appreciate this important point and will add it to our limitations. TokenSwap is designed for users who want to avoid accidentally reproducing memorized content due to copyright and plagiarism risks. Our method provides a practical, low-cost solution for such users. While determined users could potentially bypass our method through adaptive attacks, we expect most users would not attempt this given the legal and ethical risks of verbatim reproduction.

Q1: In Table 2, what are the 0.1% regurgitated examples for WritingPrompts? Do they have no token overlap with the swap set?

They do have token overlap with the swap set. However, in rare cases, the auxiliary model and main model agree on the same token as most likely at all positions where TokenSwap could intervene. When both models predict identical tokens throughout a sequence, no swapping occurs, allowing potential regurgitation.

We emphasize that for the WritingPrompts task we reduce regurgitated examples from 83.4% to 0.1% without any access to the training data, demonstrating the effectiveness of our approach in spite of these rare cases.

[1]Commonwealth of Pennsylvania, 2025. Lessons from Pennsylvania’s Generative AI Pilot with ChatGPT (OpenAI Pilot Report 2025). Pennsylvania Office of Administration.

[2]Olsson, Catherine, et al. "In-context learning and induction heads." arXiv preprint arXiv:2209.11895 (2022).

We thank the reviewer for their constructive feedback and insightful comments. We appreciate the reviewer's recognition of TokenSwap's practical advantages, strong empirical results, and clear presentation. Below, we address each of the concerns raised and provide clarifications.

W1: Although TokenSwap weakens assumptions (no access to training data or model weights), it still needs access to logits. Therefore its real-world impact is limited.

We acknowledge that some models do not provide logit access and will include this in our limitations. However, several factors support TokenSwap's practical impact:

Data constraints: Models like Llama/DeepSeek provide logits but lack publicly available training data. Unlike pretraining, unlearning or other methods like MemFree and CP-Fuse, we do not require access to training data or any subset of it, making our method accessible to such models.

Resource barriers: While model weights may be available, most users lack the infrastructure (dozens of GPUs, ~1TB memory) and cannot afford retraining costs. This makes post-hoc methods valuable for end users.

Complementary benefits: Table 4 shows TokenSwap provides additional gains even when combined with pre-training methods. Since no memorization mitigation is perfect, even developers with resources for pre-training methods benefit from TokenSwap.

Flexibility: As a post-hoc approach, TokenSwap can be easily enabled or disabled based on the specific use case.

W2: MemFree does not require querying the entire dataset during inference as mentioned in Table 1

We thank the reviewer for pointing this out. We will correct our description of MemFree's runtime optimizations and revise Table 1 accordingly. However, the core limitation remains: MemFree requires access to the complete training corpus (or its duplicated portions). Except for research models like Pythia, production models (Llama, Mistral, DeepSeek) do not release their training data, making MemFree inaccessible for users of these models. TokenSwap operates without requiring training data or model weights, making it significantly more practical. We will clarify our wording about MemFree's runtime and emphasize that our main advantage lies in this data accessibility difference.

W3: EMR alone can be gamed so please include Fractional Exact Match Rate.

We thank the reviewer for this question. Since EMR ignores cases where the output differs very slightly (changing or inserting a single token), we already include two widely used metrics to measure approximate memorization: ROUGE-L and Levenshtein distance [1,2,3]. These cannot be gamed by changing, inserting or deleting a small number of tokens.

ROUGE-L measures the longest common subsequence between generated and reference text, capturing non-contiguous matches.

Levenshtein measures the minimum edit distance (insertions, deletions, substitutions) needed to transform one text into another.

FER measures the fraction of tokens that are identical at the same position [4]. However, FER can be gamed by simple insertions, whereas ROUGE-L and Levenshtein are robust to such manipulations.

For example:

• Reference text: "The American musician and satirist Tom Lehrer has died at the age of 97"
• Generated text: "American musician and satirist Tom Lehrer has died at the age of 97"

For simplicity, consider every word as a token. Then, we have, FER = 0 (due to position shift), but ROUGE-L = 0.93 (13/14 tokens matched).

For completeness, we calculate FER in addition to the current metrics. We present the FER results for the tasks on the real datasets. We will add the FER numbers to the final version of the paper.

LeetCode Dataset (Llama):

Pile-Memorized Dataset (Pythia):

We observe that TokenSwap has the lowest FER across both tasks compared to Standard generation and CP-Fuse.

W4: Llama evaluation is unconvincing since the training data is unknown. Consider evaluation on OLMo.

We evaluate TokenSwap on OLMo-2-13B, which is trained on the open Dolma dataset. Since Dolma contains 3 trillion tokens making exhaustive memorization search impractical, we focus on Wikipedia as a known subset. We sample 5000 random Wikipedia sequences and prompt OLMo-2-13B with 50 token prefixes to generate the next 50 tokens. We select those with ROUGE-L > 0.9 with the ground-truth to identify sequences memorized verbatim or near-verbatim.

OLMo-2-13B Results:

For CP-Fuse, we use OLMo-2-7B as the second model.

TokenSwap completely eliminates exact verbatim generation and reduces approximate-verbatim generation significantly (ROUGE-L: 0.38 vs 0.95, Levenstein 0.50 vs 0.07, FER reduced by more than 3× compared to standard generation).

We chose LeetCode for Llama because prior work demonstrates that these models generate LeetCode problems verbatim on prefix probing [3]. While training data is unavailable, this approach follows established methodology, Carlini et al. [5] demonstrated GPT-2 memorization without training data access.

W5 & Lim2: TokenSwap requires the auxiliary model to share the same tokens for the token subset

We thank the reviewer for this question. Our token subset consists of 110 high-frequency function words (e.g., the, of, and, to). These are consistently represented as single tokens across all major subword tokenization families, including Byte-Pair Encoding (BPE), WordPiece, Unigram Language Model, and Byte-Level BPE. We verify this empirically across Pythia, LLaMA-3, OLMo, and GPT-2 tokenizers. We will add this discussion to Appendix C.3.

W6: Performance metric should be merged into the main table.

We appreciate this suggestion. In the extreme-memorization stress test, models are fine-tuned for 50 epochs to deliberately induce memorization, creating an artificial situation where standard generation actually has a significantly worse cross-entropy loss than TokenSwap. We will merge the performance metrics into Table 2 with clear labeling to explain this result and provide proper context for readers.

W7a: MemFree seems to have better overall performance in Table 2

Our results indicate that while MemFree ensures zero exact matches by design, TokenSwap achieves better mitigation in near (or approximate) verbatim generation. For example, the ROUGE-L and Levenshtein scores in Table 2 are:

WritingPrompts:

MathAbstracts:

This table shows TokenSwap reduces near-verbatim generation more effectively. Further, TokenSwap requires no training data access, unlike MemFree.

W7b: MemFree does not perform poorly on approximate memorization in Table 2.

Thank you for this suggestion. We will change our wording in Section 4.1 when discussing the results of Table 2 with respect to MemFree.

W8 and W9: Table formatting and missing citation

Thank you for pointing this out. We will reorganize table columns and fix the broken citation on line 267.

Q1: In line 244, why do you claim that the evaluation of commonsense reasoning is not available for base models?

We misstated this. Base models can be evaluated on commonsense reasoning benchmarks. We meant that MT-Bench (conversational evaluation) requires instruction-following capabilities not available in base Pythia models. We will clarify this distinction in the revision.

Lim 1: TokenSwap only shows its effectiveness for natural language content.

We already acknowledge this limitation in Section 5 and identify it as future work. For domains like code generation, the key challenge is whether auxiliary models can provide grammatically (or syntactically) correct but different alternatives like in natural language. In code, next tokens for syntax elements may be more deterministic, reducing disruption effectiveness. Future work should explore whether this approach extends to code or if alternative strategies (e.g., main model for planning, auxiliary for low-level implementation) would be more effective.

[1] Abad, Javier, et al. "Copyright-protected language generation via adaptive model fusion." International Conference on Learning Representations, 2025.

[2] Hans, Abhimanyu, et al. "Be like a goldfish, don't memorize! mitigating memorization in generative llms." Advances in Neural Information Processing Systems, 2024

[3] Karamolegkou, Antonia, et al. "Copyright violations and large language models." Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

[4] Ozdayi, Mustafa Safa, et al. "Controlling the extraction of memorized data from large language models via prompt-tuning." Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL): Short Papers, 2023

[5] Carlini, Nicholas, et al. "Extracting training data from large language models." 30th USENIX Security Symposium (USENIX Security 21), 2021.