BiMark: Unbiased Multilayer Watermarking for Large Language Models

Q1:Computational cost

R1: For a single-layer reweighting, the method requires first calculating the probability of a vocabulary subset, then obtaining scaling factors based on these original probabilities, and finally adjusting the probability of candidate tokens. The complexity of this operation is $O(|\mathcal{V}|)$.

For multi-layer setting, the operation is iterative, making the computation cost grow linearly with the increase in the number of layers, leading to a total complexity of $O(|d\mathcal{V}|)$.

The table below shows the runtime for BiMark during a single inference pass in our experiments:

The experimental result shows BiMark can benefit significantly from large batch size.

Q2: Type-II error of unbiased multi-layer reweighting.

R2: Although it is challenging to provide a close-form expression of type-II error rate of multi-layer watermarking at present since the current layer's probability depends on the reweighted distribution of the previous layer, the analytical idea is the same as single-layer reweighting. Specifically, the detectability of multi-layer watermarking gradually decreases as unbiased reweighting reduces distribution entropy across layers.

Intuitively, our method iteratively adjusts probability on tokens through unbiased reweighting based on our red-green vocabulary partitions. In a simplified case with two probabilities $p$ and $1-p$, each layer of reweighting transfers some probability $\Delta$ between these groups with fair chance, creating a detectable pattern.

According to type-II error rate analysis of our single-layer unbiased reweighting, higher entropy distributions allow higher expected green ratio in watermarked text, improving detectability. However, this process also inherently reduces entropy because entropy $(-\sum_x p(x)\log p(x))$ is concave, and our reweighting creates a more "peaked" distribution in expectation.

As a result, with each successive reweighting layer, the entropy of the reweighted distribution decreases in expectation, leading to diminishing marginal contributions to detectability.

This analysis explains our empirical observation that detection performance does not increase monotonically with the number of layers. Deeper layers operate on increasingly low-entropy distributions, offering proportionally smaller gains in overall watermark strength.

Q3: Experimental comparison.

R3: We would like to clarify our experimental choices:

For zero-bit case, BiMark was compared with Soft-Red-List (it only supports zero-bit watermarking). The labelling of Soft-Red-List as "KGW" in the figures will be corrected for consistency in the revised version.

For multi-bit case, BiMark was compared with MPAC. MPAC is not available for the zero-bit scenario since it is specifically designed for multi-bit watermarking tasks and is built upon Soft-Red-List.

GINSEW $1$ is not available as it is intended for a different purpose—specifically, protecting text-generation models from theft through distillation rather than detecting watermarks in individual text segments.

$1$ Protecting language generation models via invisible watermarking.

Q4: Ablation study

R4: In Fig.5(c), we set the number of reweighting layers to $d=50$. We apologize that the legend does not clearly reflect this setting. We chose this large value to more clearly demonstrate how scaling factor affects watermark detectability when the number of layers is fixed. While the relationship between scaling factor and detectability holds for any value of $d$, the effects are more obvious with larger $d$ values, making the trends easier to visualize.

Q5: Potential vulnerability to attack

R5: We have additionally evaluated the vulnerability of our method under paraphrasing attack, as shown in R3 to Reviewer kPVR. Our current work focuses primarily on the watermarking mechanism itself. We acknowledge the importance of attack resistance, and we will investigate broader attack scenarios and conduct vulnerability analysis as part of our future work.

Q6: Code release

R6: Thank you for suggestions. We plan to release the code implementation at the time of the final decision announcement.

Q7:Concerns about representation

R7: Thank you for your suggestions. We will enhance our paper by: (1) providing more detailed experimental analysis (2) redesigning Figs. 1 and 6 to better illustrate BiMark's mechanisms; and (3) improving overall presentation with consistent terminology and clearer explanations throughout the paper.

Q1: Sophisticated adversarial attacks

R1: We conducted advanced paraphrasing attacks and reported experimental results in our response to Reviewer kPVR in R3.

Q2: Inference-time benchmarks

R2: We analyzed the computational cost and give experimental time cost of BiMark. Please refer to our response to Reviewer jksy in R1.

Q3: Non-monotonic performance as the number of layers increases

R3: The non-monotonic relationship between layer count and detection performance can be explained through our analytical framework:

• Initial enhancement: Adding layers initially strengthens detection by creating more statistical patterns across independent vocabulary partitions.
• Entropy reduction effect: As shown in our theoretical analysis (R2 of Our response to Reviewer jksy), each layer of unbiased reweighting reduces the expected entropy of the probability distribution.
• Diminishing benefits mechanism: When too many layers are applied, the probability distribution becomes increasingly concentrated, causing later layers to operate on highly skewed distributions. This creates a situation where green lists in later layers may have extremely low probability, making generated tokens unlikely to fall into these green lists and adding noise to the detection process.

We observed this phenomenon more prominently with larger scaling factors, as they cause more dramatic probability redistribution at each layer, accelerating both the initial enhancement and subsequent diminishing benefits.

Q4: Meaning of symbols

R4: $M$ stores absolute counts for detection. During detecting watermark, a ratio of green counts is calculated based on the absolute counts in $M$. Though there is noise caused by randomness of text generation, for certain length of watermarked text, the green ratio of watermarked text will significantly exceed 1/2 and that of non-watermarked text will be around 1/2.

Q1: Computational cost of multi-layer reweighting

R1: Please refer to our response to Reviewer jksy in R1.

Q2: Robustness experiments

R2: We conducted advanced paraphrasing attacks to our watermarked text. The detectability of BiMark under paraphrasing shows desirable robustness. Analysis reveals multi-layer watermarking provides fine-grained watermark evidence to keep the green ratio of watermarked text steadily deviating from 1/2 after paraphrasing. For more details, please refer to our response to Reviewer kPVR in R3.

Q3: Terminology consistency

R3: Thank you for pointing out the inconsistencies in our terminology. We will correct these issues and thoroughly revise the paper to avoid other potential issues.

Q4: Scaling factor selection

R4: For selecting scaling factors, we recommend adjusting based on the application scenario:

• For better text quality: use smaller scaling factors with more layers.
• For computational efficiency: use larger scaling factors with fewer layers.

The underline insight behind these selection lies in entropy reduction effect of multi-layer reweighting. For details, please refer to our response to Reviewer jksy in R2.

Q1:Perplexity

R1: We reported perplexity in our experiments because it is one of the most commonly used metrics for evaluating the quality of generated text. We will revise the relevant statement in the final paper.

Q2:Challenge of unbiased watermark and model-agnostic detection

R2: We agree with you that prior work such as DipMark and SynthID has addressed the model-agnostic requirement. However, our work uniquely aims to integrate three critical properties—text quality preservation, model-agnostic detection, and message embedding capacity—into a single cohesive framework, a task that remains challenging in the current research landscape. We will revise our statement accordingly to prevent confusion.

Q3:Baseline methods and evaluation criteria

R3: Additional comparative evaluations with $1$ are conducted, as presented in the following table. Note that $2$ is not suitable as a baseline due to its significant inefficiency. Previous studies have confirmed its high computational cost—for example, embedding a 32-bit message requires approximately 29,000 seconds $1$ .

The table below reports the bit match rates of multi-bit watermark detection experiments on 8-bit, 16-bit and 32-bit message embedding and extraction, where Length indicates the number of tokens in the watermarked text being detected.

Compared with $1$ , BiMark is an unbiased watermarking method, and the Error Correction Code component of $1$ has the potential to integrate with BiMark to further improve multi-bit watermark performance.

Additional comparative evaluations with DiPmark $4$ are conducted. We exclude comparison with Gamma-Reweight $5$ as it is not model-agnostic detectable, thus beyond the scope of the application scenario of our study. Additional evaluations on robustness are conducted through the advanced paraphraser model Dipper $6$ . The table below reports TPR@1%FPR of watermark detection on watermarked text with and without paraphrasing, where (20,0) indicates parameters lex_diversity and order_diversity of Dipper are 20 and 0.

$1$ Provably Robust Multibit Watermarking for AI-generated Text via Error Correction Code.

$2$ Three bricks to consolidate watermarks for large language models.

$3$ Advancing Beyond Identification: Multi-bit Watermark for Large Language Models.

$4$ A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models.

$5$ Unbiased Watermark for Large Language Models.

$6$ Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defence.

$7$ A Watermark for Large Language Models.

$8$ Scalable watermarking for identifying large language model outputs.

Q4: Robustness analysis and evaluation

R4: A primary robustness evaluation has been provided in Section 5.1 Table 3 in our paper. Additional evaluations are conducted to test BiMark's resilience against paraphrasing attacks, as outlined in R3.

Q5:Comparison with DiPmark and Gamma-Reweight

R5: