Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text

We thank the reviewer for their valuable time and effort in providing this constructive review. We used collective feedback from reviewers and conducted additional experiments with results in Figure 3 (per-token detection performance), Figure 4 (open-source generation detection), Figure 9 (detecting different prompting strategies), and Figure 17 (ablation). Below is our response with a point-by-point note for the questions/weaknesses listed.

Firstly, we highlight that the “cross-perplexity” definition is a novel concept in NLP to the best of our knowledge. Its usage in perplexity-detectors for normalization is motivated in section 3.3, resulting in a significant increase in performance by addressing the perplexity-only-detectors limitations (as shown in Figure 17 in Appendix A5). The definition and use of x-PPL came about after viewing the perplexity-only-detector (and its limitation) from a specific lens as mentioned in “Capybara problem” (section 3.2) important for motivating this work.

Based on the feedback, we have updated the draft highlighting our changes with blue text.

1. Usage of TPR @ low-FPR across experiments

• In the metrics section, we discuss how detectors’ performance should be evaluated under a low-false-positive regime (which can be realized by many metrics) since type 1 errors are significantly more costly than type 2 in downstream tasks. We also argue that typically reported metrics like standalone AUC values, as in other baselines, don’t translate into real-life performance sufficiency (as shown in Table 2: AUCs are uncorrelated with TPR at low FPR). Below is our rationale for metrics in figures:
  • Figure 1 reports TPR @ 0.01% FPR and Figure 2 reports F1-Score on the identical dataset as Figure 1 (ChatGPT benchmark by [1])
  • Figure 3 is updated to have TPR @ 0.01% FPR at the varying sizes of samples (# of tokens)
  • Figure 4 reports TPR observed at different FPR levels (on log scale) for LLaMA-2-13B generations over 3 datasets
  • Figure 5, 6, and 7 reports how Binoculars maintain high precision (with varying recall) over multiple generators, languages and domains respectively.
  • Figure 9's dataset contains only machine-generated samples and hence only reports an error rate.

2. The abstract claim of detecting news documents Binoculars detect 95% of synthetic samples at a false positive rate of 0.01%

• This figure is from the first column in Figure 1 (News, ChatGPT dataset [1]). We updated the abstract to replace the previously rounded number to 94.92%.

3. Document size impact figure only contains the Student dataset

• We updated Figure 3 to include all 3 datasets from [1] and also added another baseline method (GPTZero).

4. Performance comparison with state-of-the-art and using the same methods should be used for comparison

• Our emphasis is directed toward baselines that function within post hoc, out-of-domain (zero-order), and black-box detection scenarios. We use state-of-the-art Ghostbuster (Verma et al., 2023), the commercially deployed GPTZero, and DetectGPT to compare detection performance over various datasets in Section 4.
• Performance Benchmark: As per above, on all benchmark datasets in section 4, we use these baselines (which include the ChatGPT benchmark dataset competing baseline in [1]). To align our experiments better, we have updated the figure as mentioned above in this note. We thank you for your feedback on this.
• Reliability: In Section 5, we evaluate the reliability of Binoculars in various settings that constitute edge cases and interesting behaviors of our detector. With the exception of Figure 9 (modified prompting strategies), we report these numbers as absolute to aide understanding of Binoculars specifically.

5. Methods in Figure 6 are not mentioned or discussed in the text

• Thank you for this feedback. We have added a paragraph in section 5.1 for these methods.

6. The selection of the methods used to compare with is not motivated.

• We updated the draft to motivate our choice of baselines (also mentioned in the previous point) (section 2). To reiterate, we focus on evaluating baselines designed for post hoc, out-of-domain (zero-order), and black-box detection scenarios. which includes state-of-the-art Ghostbusters (Verma et al., 2023), the commercially deployed GPTZero, and DetectGPT.

7. Ablation study to ascertain the contribution of using the proposed detection score vs. simple perplexity

• Due to the page limit we have this study in Figure 12 + Table 3 in Appendix A. We compare PPL, xPPL, and Binoculars (Falcon and LLaMA variants) performance over the ChatGPT dataset from 1]. We see how Binoculars perform better than its components to detect machine text.

Dear Reviewer CMmZ,

As the discussion period ends, we would like to kindly ask for your reply to our remarks. We feel we have addressed your concerns and we look forward to further discussions and guidance. Please let us know if there is anything else we can do or other questions we can answer.

We thank the reviewer for their valuable time and effort in providing this constructive review. We are glad that you found our paper well-written, and easy to follow. We used collective feedback from reviewers and conducted additional experiments with results in Figure 3 (per-token detection performance), Figure 4 (open-source generation detection), Figure 9 (detecting different prompting strategies), and Figure 17 (ablation). Below is our response with a point-by-point note for the questions/weaknesses listed.

1. In section 2, it could be clearer how the proposed method relates to other PPL-based ones

• One difference between Binoculars and some other perplexity-based detectors is in the number of forward passes required to compute a discriminative score. For example, DetectGPT is built on a similar motivation -- that perplexity alone is insufficient for classification -- but to overcome the weakness in the signal, DetectGPT utilizes the perplexity of several perturbations to the input. In addition to the overhead of generating perturbations, it also requires computing many perplexities and thus it can be far more computationally expensive. So there may be various ways to extract meaningful information from perplexity, ours is less computationally demanding than some others.

2. In Table 2, we see that the performance can vary a lot depending on the chosen LLM for scoring: what makes a good LLM for the method? Why other choices reach lower TPR?

• This is more because of how stringent TPR @ 0.01% FPR as a performance metric is than instability of performance. We updated the table 2 to include TPR @ 0.1% FPR. F1-Score and AUC. As you may see, the performances are "stable" through the lens of F1-Score and AUC - which are fine metrics to use when the costs of type 1 and 2 errors are uniform.
• However, in problems with the severely skewed cost of error towards FP (like detection), metrics like (vanilla) F1-score and AUC don’t provide downstream safety in the deployment of these systems.
• We also note (in section 4.2) “that AUC scores are often uncorrelated with TRP@FPR when the FPR is below 1%.” (as seen in the updated Table 2).
• Moreover, we see saturation of performance using Binoculars (at least in its current format) irrespective of the scorers used. However, with the surge in open-source LMs, we do not claim an exhaustive search of scoring models to be used.

3. What would happen if the LM (evaluated or chosen for scoring) is less similar, e.g. different training set, different kind of model, etc.

• Please note, by our x-PPL definition, we have an implicit constraint of having to use two models to share a tokenizer, and most available open-source models have a high degree of training set overlap (eg. Falcon, Llama families, etc.).
• The cross-perplexity measures the degree to which two models' next-token distributions overlap. We suspect cross-ppl from two models trained on dissimilar datasets would provide more noise than the signal we want to detect (see updated text section 3.3 for motivation of x-PPL).
• The similar behavior of the transformers model is what we depend on for cross-ppl and Binoculars score definition. Thank you for mentioning this important point. We expanded the definition section to mention the impact of different training sets and will be happy to provide any more information required.
• In the ablation experiment (in Figure 17), we see how x-PPL and PPl alone aren't enough for reliable detection performance.

4. How would the training set of the LLMs (mostly written by humans I guess) be detected

• Training samples are a super-set of memorized strings in Section 5.3 (i.e. strings that were part of training but may or may not be re-generated by LLM).
• For a perplexity-based detector, memorized/famous text is more difficult to classify in comparison to samples from larger training sets (since the famous text is expected to be in the training set multiple times).
• Also, the classification of such samples is domain-specific since both humans and machines can be deemed to have produced it in different domains (eg. plagiarism detection v/s removal of LLM-generated text from a training corpus).
• With this understanding and performance on memorized text, we believe that our detector would score samples from the training set towards the human side of the threshold and should be leveraged accordingly in downstream tasks.

Dear Reviewer M521,

As the discussion period ends, we would like to kindly ask for your reply to our remarks. We feel we have addressed your concerns and we look forward to further discussions and guidance. Please let us know if there is anything else we can do or other questions we can answer.

We thank the reviewer for their valuable time and effort in providing this constructive review. Based on the feedback, we have updated the draft highlighting our changes with blue text. We used collective feedback from reviewers and conducted additional experiments with results in Figure 3 (per-token detection performance), Figure 4 (open-source generation detection), Figure 9 (detecting different prompting strategies), and Figure 17 (ablation). Below is our response with a point-by-point note for the questions/weaknesses listed.

1. Only Ghostbusters used as a baseline in open-source language models text detection (LLaMA-2-13B)

• We add baseline experiments in Figure 4. The performances on Falcon-7B generations can be found in Figure 13 in Appendix A3.

2. Only models from Falcon and Llama-2 families used for computing x-PPL

• Thanks for noting this. This is because of the following:
• Implicit constraint of x-PPL computation which requires two models to have identical tokenizer (we mention in section 3.1)
• As per our intuition (and motivation, section 3.3), our detector's performance is dependent on having two similar (by training set) models for computing x-PPL and would only cause a marginal change in performance across different scoring model pairs. We show this in the updated Table 2 in Appendix A1, how F1-Score and AUC metrics (which weigh type 1 and type 2 error equally) are stable across multiple pairs of scoring models only resulting in a marginal change in the performance of the detector. Please note, that we do not claim our scoring models search to be exhaustive.

3. The reason behind the effectiveness of Binoculars

• In the updated draft, we develop on motivation behind our detector (use of x-PPL) in section 3.3: "Language models are known for producing low-perplexity text relative to humans and thus a perplexity threshold classifier makes for an obvious detecting scheme. However, in the LLM era, the generated text may exhibit a high perplexity score in the absence of the prompt (“Capybara Problem” in Table 1). To calibrate for prompts that yield high-perplexity generation, we use cross-perplexity introduced in Equation (3) as a normalizing factor that roughly encodes the perplexity level of next-token predictions from two models."
• We employ this cross-perplexity, which encodes the degree with which two models' next-token distributions overlap, as our normalizing mechanism to solve the "Capybara Problem." The similar behavior of the transformers model is what we depend on for the x-PPL and Binoculars score definition. Thank you for mentioning this important point. We expanded the definition section to mention the impact of different training sets and will be happy to provide any more information required.

To summarize, we present a novel detector that is able to maintain high efficacy in a false-positive regime under various domains/datasets/baselines, using only open-source components to detect in zero-shot settings. Broadly, we highlight the need to evaluate detectors in terms of low-false positive metrics and showcase how our method is performative in this regime (note: in principle, by lowering the threshold even further we can virtually eliminate any false positives). We further study our detector’s reliability in settings that constitute edge cases to understand interesting behaviors, abilities, and limitations of our detector.

We are happy to take on any follow-up questions or other feedback.

Again, we thank you for your time and contribution to this key area of secure and safe machine learning in the generative AI era.

Dear Reviewer iAnY,

As the discussion period ends, we would like to kindly ask for your reply to our remarks. We feel we have addressed your concerns and we look forward to further discussions and guidance. Please let us know if there is anything else we can do or other questions we can answer.

9. Use of scoring models trained on different domains and other aspects of Binoculars.

• Please note, by our x-PPL definition, we have an implicit constraint of having to use two models to share a tokenizer, and most available open-source models have a high degree of training set overlap (eg. Falcon, Llama family).
• The cross-perplexity measures the degree to which two models' next-token distributions overlap. We suspect cross-ppl from two models trained on dissimilar datasets would provide more noise than the signal we want to detect (see updated text section 3.3 for motivation of x-PPL). The similar behavior of the transformers model is what we depend on for cross-ppl and Binoculars score definition. Thank you for mentioning this important point. We expanded the definition section to mention the impact of different training sets and will be happy to provide any more information required.
• In the ablation experiment (in Figure 17), we see how x-PPL and PPl alone aren't enough for reliable detection performance.
• We are happy to answer any further or follow-up questions on this.

10. When showing FPR/TPR plots, how are these plots generated? by changing the detection threshold? I do not understand why you say 0.01% FPR threshold is "The smallest threshold we can comprehensively evaluate with our limited compute resources."

• Yes, for each method we move the threshold to achieve 0.01% FPR and report the TPR figures to compare performance in low false positive regime.
• The FPR rate we can achieve on a log scale is a function of the number of samples. For all our experiments, we use a balanced dataset (50-50 human and LM samples). At 0.01% FPR, we would need 10k samples per class, and at 0.001% we would need 100k samples per class.

To summarize, we present a novel detector that is able to maintain high efficacy in a false-positive regime under various domains/datasets/baselines, using only open-source components to detect in zero-shot settings. Broadly, we highlight the need to evaluate detectors in terms of low-false positive metrics and showcase how our method is performative in this regime (note: in principle, by lowering the threshold even further we can virtually eliminate any false positives). We further study our detector’s reliability in settings that constitute edge cases to understand interesting behaviors, abilities, and limitations of our detector.

We are happy to take on any follow-up questions or other feedback.

Again, we thank our reviewers for their time and contribution to this key area of secure and safe machine learning in the generative AI era.

We thank the reviewer for their valuable time and effort in providing this constructive review. Based on the feedback, we have updated the draft highlighting our changes with blue text. Driven by your feedback we have conducted additional experiments with results in Figure 3 (per-token detection performance), Figure 4 (open-source generation detection), Figure 9 (detecting different prompting strategies), and Figure 17 (ablation). Below is our response with a point-by-point note for the questions/weaknesses listed.

1. Zero-shot Detection Claim.

• We would like to clarify that we do not optimize the threshold for the target task (in all cases/experiments), but use a global threshold obtained from Binoculars. As mentioned in section 4.3: since our global threshold is tuned on a ChatGPT benchmark dataset (News, Creative Writing, Essay) by [1] (Ghostbusters method), we make an exception "to be sure that we meet the OOD criteria, we do not include the ChatGPT datasets in the threshold determination, and only use samples from CC News, CNN, and PubMed (generated via LLaMA and Falcon) to determine threshold to do predictions/evaluation on ChatGPT benchmark dataset (Figure 1, 2, & 3). For open-source datasets (CC News, CNN, and PubMed), we report AUC plots for all methods that are threshold-invariant for all methods (Figure 4). With this OOD threshold, we demonstrate that reliably detects texts generated by different LMs and from different domains.

2. Roberta, LR GLTR, Stylistic, Nela not explained.

• Thank you for pointing out, that we have added a note in the M4 Datasets note in section 5.1

3. Choice of baselines and across experiments

• In this work, our emphasis is directed toward baselines that function within post hoc, out-of-domain (zero-order), and black-box detection scenarios. We use state-of-the-art Ghostbusters [1], GPTZero, and DetectGPT to compare detection performance over various datasets in section 4.
• In section 5, we evaluate the reliability of Binoculars in various settings that constitute edge cases and interesting behaviors of our detector (multi-domain, languages, non-native English text, etc.)
• We updated the draft to mention our choice of baselines and rationale towards the end of Section 2.

4. Methodology Results of M4 Datasets [2]

• From Wang et al., we use ChatGPT vs Human performance table. To aid OOD comparison between Binoculars and other baselines, we use the mean of OOD-reported performances by Wang et al. For each baseline, for the "arXiv" domain for example, we use the mean of performance when the respective baseline is trained on {Wikipedia, WikiHow, Reddit, ELI5, PeerRead}.
• We update the caption Figure 7 to clarify this.

5. Absence baseline comparison on M4 dataset

• Rationale: As mentioned above, in Section 4 we use baseline methods to compare our performance with established zero-shot, OOD domain detectors on various known benchmark datasets. Our intent for M4 datasets is to study our detector's performance in multiple domains, languages, and text generators and report non-relative claims.
• Constraint: Unlike other feedback, we could not run other methods on M4 datasets which consist of 39 datasets with 3k samples each due to sizable constraints: a) Ghostbusters require approx $1150 OpenAI credits to get log-probs for 46 million tokens (512 per sample), b) computational constraint for DetectGPT which requires computing multiple perturbations.
• Thus, we move M4 datasets experiments to Section 5 (reliability).

6. Baselines in Orca Dataset

• We run other methods to append baselines in Figure 9 to show our method is more robust than other methods.

7. Inconsistency/typos in formulation/notation.

• We thank you for the detailed feedback and have updated to fix all pointers.

8. Intuition on x-PPL, its interpretation, and its effect on the detection method

• Lower/Upper Bound: Binoculars score are ratios of two cross-entropy measures (both being positive) and is a positive number
• Impact of using identical models for scoring: We perform this experiment and report the findings in Figure 17 in Appendix A5. We observe using Falcon-7B as $M_1$ and $M_2$ comes close to Binocular's performance.

[1] Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting Text Ghost- written by Large Language Models. arxiv:2305.15047[cs], May 2023. doi: 10.48550/arXiv.2305. 15047. URL http://arxiv.org/abs/2305.15047. [2] Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Alham Fikri Aji, and Preslav Nakov. M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection. arxiv:2305.14902[cs], May 2023. doi: 10.48550/arXiv.2305.14902. URL http://arxiv.org/abs/2305.14902