Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection

We appreciate your detailed and thoughtful feedback, particularly your acknowledgement of the value of the task setting! Your concern W1 has been carefully examined and addressed in the revised version. We hope this resolves your issue. Please let us know if you have any further questions.

W1: Comparing to this proprietary and estimation approach, why not use open-source large model such as LLaMA as the scoring model? This paper did not mention models like LLaMA at all. If authors can give me a convincing answer or add some comparison experiment, I'll consider raise score.

Reply: Please refer to the reply to Common Concern 2.

Q1: what's the difference between source model and scoring model.

Reply: We have updated the paper for better clarity. The text is generated using the source model, and the scoring model is used to create a detection metric. In our tests, we utilize a consistent scoring model to identify text generated by different source models.

Q2: Figure1: I think the notation p(x|x) is weird.

Reply: We have amended the paper to eliminate any confusion. The notation simply adheres to the format of the conditional probability function p(\tilde{x}|x) and considers p(x)= p(x|x) as a particular instance of the function.

Q3: Line293: What do you mean by "generate corresponding text from source LLMs for each sample"? Do you mean generate text with the same prefix?

Reply: Indeed, you are correct. We have made revisions to the paper for better clarity.

We are profoundly grateful for your acknowledgment of our work, which will undoubtedly inspire our ongoing research!

We appreciate your feedback. Your concern about W3 have been taken into consideration and addressed. However, we believe that your issues with W1 and W2 may be due to unfamiliarity with the recent studies about zero-shot machine-generated text detection. We hope our explanation will dispel your uncertainties. If you have additional questions, please don't hesitate to let us know.

W1: The evaluations are conducted on limited datasets (150 pairs), potentially restricting the findings’ generalizability across more diverse or larger data sources.

Reply: Please refer to the reply to Common Concern 1.

W2: While PDE is an effective enhancement, it primarily focuses on density estimation and integration with existing detection methods rather than introducing a fundamentally new detection approach.

Reply: PDE facilitates the application of white-box techniques on proprietary models, eliminating the previous restriction that only black-box methods could utilize proprietary models for detection. In terms of its impact, we believe PDE holds greater significance than a solitary detection method, as it allows for the integration of white-box methods with the most recent LLMs.

W3: it may be better to use bigger or newer open models (like Qwen2.5-1.5B or Llama3) than to use PDE to proprietary models.

Reply: Please refer to the reply to Common Concern 2.

Remarks: Small remarks

Reply: We have revised the paper for the descriptions and equations.

Thank you for your valuable input. We have addressed your concern regarding W2. However, we suspect that your concerns with W1 and W3 might be due to a lack of familiarity with the task of identifying machine-generated text using zero-shot techniques. We hope our response will clarify your doubts. Please let us know if you have any further queries.

W1: The amount of machine-generated texts in the datasets in Table 2 (which is supposed to contain the main results) is extremely small.

Reply: Please refer to the reply to Common Concern 1.

W2: The choice of baseline open LLM (i.e. GPT-Neo-2.7B) for applying methods like Fast-Detect and Likelihood seems unreasonable because GPT-Neo-2.7B is weak compared to the newer open LLMs of comparable size.

Reply: Please refer to the replay to Common Concern 2.

W3: AUROC scores in the Tables seem to be very high both for the best baseline and for the best method suggested by the authors.

Reply: It is commonplace for zero-shot detection techniques to achieve AUROC scores exceeding 0.90. This can be seen in recent research like DetectGPT by Mitchell et al. (ICML 2023), Fast-DetectGPT by Bao et al. (ICLR 2024), DNA-GPT by Yang et al. (ICLR 2024), and DLAD by Zeng et al. (NeurIPS 2024).

Q1: Line 138-140, Equation 2: why do you call this d(x,pθ) value "curvature"? It doesn't seem to have a meaning of a curvature. Maybe "skewness" would be better word?

Reply: The idea of "probability curvature" was introduced by DetectGPT (Mitchell et al., ICML 2023), which is a measure of the difference between the log-likelihoods of the revised and original text. Machine-generated text typically has a positive curvature because its rephrased versions usually have lower log-likelihoods. However, human-written text often has a curvature close to zero because its rephrased versions can have either higher or lower log-likelihoods. Fast-DetectGPT adopts this idea but substitutes the "probability function" with the "conditional probability function". We mention Fast-DetectGPT here to demonstrate how PDE can be applied to it.

Q2: Only AUROC is reported as a metric for comparison in the Tables. It would be interesting to see, what are the accuracy scores for all methods (for some fixed threshold)?

Reply: We also present the True Positive Rate against the False Positive Rate in Figure 6, enabling a clear comparison of the effectiveness of various methods at different thresholds. Furthermore, we believe that ACC is not a suitable metric for detecting machine-generated text as it depends on the proportion of positive to negative samples. Depending on the situation, the quantity of positive samples may be significantly greater or lesser than the negative samples, leading to inflated ACC scores even when using a detector that guesses randomly.

Q3: Are there any differences between "Likelihood" method and perplexity scores?

Reply: Essentially, they are equivalent for zero-shot detection tasks. The likelihood represents the average log likelihood, calculated as "1/N ∑ log p(x_j | x_<j)", while the perplexity score is simply the exponential of the negative average log likelihood, expressed as "exp(-average log likelihood)". We will explain this further.

Remarks: Minor remarks about the presentation

Reply: We will revise the paper to improve the presentation. Specifically, we will replace log p(x|x) with its equal expression log p(x), where log p(x|x) is just an expression in form of the conditional probability function. We will replace the color with a softer red. For appendix D.1 and E, we will consider putting them into the main text if space allows.

Thank you for your answers, paper revision and new experiments. It was interesting and surprising that Neo-2.7B worked better as a backbone for your method than newer models.

However, I find important to continue the discussion of the following point:

Q2: Only AUROC is reported as a metric for comparison in the Tables. It would be interesting to see, what are the accuracy scores for all methods (for some fixed threshold)?

Reply: We also present the True Positive Rate against the False Positive Rate in Figure 6, enabling a clear comparison of the effectiveness of various methods at different thresholds. Furthermore, we believe that ACC is not a suitable metric for detecting machine-generated text as it depends on the proportion of positive to negative samples. Depending on the situation, the quantity of positive samples may be significantly greater or lesser than the negative samples, leading to inflated ACC scores even when using a detector that guesses randomly.

I think I need to elaborate my thoughts about AUROC and accuracy in more details. When the metrics are very close to each other, it becomes difficult to draw clear conclusions about the relative performance of the methods. And, as I already told in my review, there are several methods with high AUROC in your work.

In the same time, accuracy is more discrete and more illustrative metric than AUROC. E.g. if you have 300 examples in your dataset, and the accuracy for a method A is bigger than the accuracy for a method B by 1%, then method A correctly classified three more examples compared to Method B (in general, the set of correctly identified examples can be different for two methods though). I asked to see the accuracy (for some fixed threshold) because it would be interesting to see exactly how many more examples your method classified correctly in each case.

I completely agree that accuracy is not suitable metric when one is using unbalanced datasets. However, your datasets appear to be perfectly balanced (with an equal number of natural and generated samples), so in this case, accuracy can serve as a meaningful complement to the AUROC metric.

I’m sorry that I don’t have the opportunity to address the other points right now, but I will respond to them later as well.

We deeply appreciate your thorough explanation regarding the issue. We have updated the paper to incorporate a comparison of ACC in Table 6.

Despite ACC not being commonly employed by recent zero-shot detection studies, such as DetectGPT (Mitchell et al., ICML 2023), Fast-DetectGPT (Bao et al., ICLR 2024), DNA-GPT (Yang et al., ICLR 2024), FourierGPT (Yang et al., EMNLP 2024), and DLAD (Zeng et al., NeurIPS 2024), we think your argument is valid and will provide readers an intuitive understanding about the detection accuracy. Here is a part of the comparison on ChatGPT generations from Table 6.

|                             | Best Threshold per Dataset        | Best Threshold across Datasets    | Diff    |
| Method                      | XSum   | Writing| PubMed | Avg.   | XSum   | Writing| PubMed | Avg.   | Avg.    |
| Fast-Detect (GPT-J/Neo-2.7) | 0.9600 | 0.9633 | 0.8267 | 0.9167 | 0.9400 | 0.9367 | 0.7567 | 0.8778 | -0.0389 |
| Fast-Detect (Phi2-2.7B)     | 0.7467 | 0.6967 | 0.7600 | 0.7344 | 0.6767 | 0.6967 | 0.7467 | 0.7067 | -0.0277 |
| Fast-Detect (Llama3-8B)     | 0.8000 | 0.7700 | 0.7267 | 0.7656 | 0.6900 | 0.7667 | 0.6567 | 0.7044 | -0.0612 |
| DNA-GPT (GPT-3.5)           | 0.8750 | 0.8592 | 0.8833 | 0.8725 | 0.8108 | 0.5986 | 0.7467 | 0.7187 | -0.1538 |
| PDE (Fast-Detect, GPT-3.5)  | 0.9633 | 0.9967 | 0.9200 | 0.9600 | 0.9033 | 0.9967 | 0.9167 | 0.9389 | -0.0211 |

We also assess the ACCs utilizing the optimal threshold across datasets and source models, yielding results that are nearly identical (deviation less than 0.006) to the best threshold across datasets. As the table shows, PDE consistently produces higher ACCs than the baselines. Most crucially, PDE offers the most stable metric across datasets and source models, implying that it can identify texts from a variety of domains and source models using a fixed threshold.

Additionally, we have included a section “Appendix E.1 Robustness across Source Models and Domains” and a corresponding summary in Section 3.5 to account for an out-of-domain setup for both source models and text domains. The findings reveal that PDE maintains the highest ACCs across all the source models and domains using a threshold from out-of-domain datasets.

Dear Reviewer, we have diligently attended to your previously raised concerns. Would you mind providing us with any additional feedback you might have?

Since our last conversation, we have included additional experimental outcomes we believe sufficiently address your concerns.

Fist, we introduce Mix3 (450 pairs), which is a blend of XSum, Writing, and PubMed, and Mix6 (900 pairs), which is an integration of six languages. These datasets provide diversity in both domain and language, thereby achieving the required scale and variety. We have updated the average AUROCs (Avg.) with the AUROCs on the individual dataset, as shown in Table 1, 2, 3, 4, and 5. The adjusted scores do not change the final conclusion or discussion.

Next, we have included an out-of-domain scenario and provided the ACCs for both textual domains and source models, as indicated in Table 8 and 9. PDE has yielded the highest ACCs in all these settings.

Last, we have cited "AI-generated text boundary detection with RoFT, Kushnareva et. al, 2024" as pertinent literature. We relocated the appendix section titled "Necessity of PDE" into the main body of the text. For the appendix section "Robustness Under Paraphrasing Attack," we included a summary in Section 3.5, while keeping the detailed content in the appendix due to space constraints.

Sorry for the delay. I checked the updates in your paper, re-read your discussion with me and with other reviewers, and "Reply to Two Common Concerns". After it, I have the following thoughts.

I appreciate the new experiments, new accuracy results, new Appendix E.1 Robustness across Source Models and Domains, the explanations about the number of examples, and answers to my questions that helped me to clarify some parts of your paper. Two of my major concerns were addressed with these additions. Due to this, I update my main score from 3 to 5 and adjust other scores accordingly.

I especially appreciated your answer to the question about curvature. It stimulated me to re-read again the method description in the original paper about DetectGPT and get acquainted with the newer FastDetectGPT paper that introduced some formalisms that you use.

Nevertheless, I would like to point out some other things that I didn't comment on the previous time due to lack of time.

First:

we suspect that your concerns with W1 and W3 might be due to a lack of familiarity with the task of identifying machine-generated text using zero-shot techniques

I found your comment regarding my familiarity with the task dismissive, as it implies a lack of understanding on my part, and it felt more like a personal attack than a constructive critique. I would appreciate it if you could refrain from making comments like that in the future.

Second:

The following concern still remains unaddressed:

AUROC scores in the Tables seem to be very high both for the best baseline and for the best method suggested by the authors. It means that the task statement is too easy. It would be good to add more complicated data, where all the methods will struggle, and the difference between methods will become more clear.

Your answer was the following:

It is commonplace for zero-shot detection techniques to achieve AUROC scores exceeding 0.90. This can be seen in recent research like DetectGPT by Mitchell et al. (ICML 2023), Fast-DetectGPT by Bao et al. (ICLR 2024), DNA-GPT by Yang et al. (ICLR 2024), and DLAD by Zeng et al. (NeurIPS 2024).

You are right that many papers about artificial text detection use simple datasets that allow us to gain high scores. But if they all do it, it doesn't mean that it is a good thing to do.

I know that the discussion phase is close to its end, so it is too late to ask for a new experiment for this particular paper. However, I recommend you try more complicated datasets in your next research. For a start, I would suggest trying the dataset from the paper "RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors" by Dugan et al.

I suggest this not because I want you to cite this paper (I'm not even an author of it, and my scores don't depend on citations anyway) but because I think that researchers in the field should move closer to real-life scenarios, which are usually much more harsh than very simple datasets that you see in most papers on the topic. No detector from the papers you listed will gain 0.9 AUC in real-life scenarios where users of generative models are much more cunning and have much more diverse behavior, than generations in the simple datasets everyone uses. That "RAID" benchmark looks like the most close to some real-life thing. I suppose that datasets from "SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection" by Wang et.al are also "more-or-less okay" option for the future research, yet are still a bit too simple.

We are grateful for your feedback, particularly for acknowledging our revisions. Thank you for being candid about your thoughts on our potentially inappropriate wording, which will certainly aid us in crafting better responses in the future. We also value your thorough explanation and suggestions regarding experiments on more challenging datasets such as RAID, and we will take this into account in our next revision. However, we argue that our current experiments have already covered the perspectives provided by RAID and adequately support our claim as follows.

You are right that many papers about artificial text detection use simple datasets that allow us to gain high scores. But if they all do it, it doesn't mean that it is a good thing to do.

Replay: We agree that testing with more challenging datasets can highlight the advancement made by the method; there are always more datasets to explore. However, we would like to argue that the current experiments are sufficient to justify the novelty and contribution of PDE — specifically, "enabling white-box methods to use proprietary models," which was previously unachievable.

First, more challenging dataset like RAID combines domain coverage, model coverage, and adversarial coverage, except for multilingual coverage. Our current experiments already address these aspects: the domain-diverse dataset for domain coverage, the five source models for model coverage, the paraphrasing attack for adversarial coverage, and the language-diverse dataset for multilingual coverage.

Second, the primary aim of this study is to show the feasibility and potential of enabling white-box methods to use proprietary LLMs, not necessarily to outdo all other techniques in every situation. We argue that this potential is well demonstrated through its effectiveness across multiple white-box methods and the significant improvements over versions using open-source models.

We appreciate your comprehensive feedback! It appears there may be a few fundamental misconceptions about our study. We trust our response will clarify these points for you. Please let us know if you have any further queries.

W1: The novelty of the proposed method compared to `vanilla’ Fast-DetectGPT is somewhat ambiguous.

Reply: The novelty of PDE does not rely on a comparison with vanilla Fast-DetectGPT, as PDE is a versatile method that can be applied to other white-box methods, such as Entropy, Rank, and LogRank. We have shown that PDE is an adaptable method that has facilitated the implementation of white-box methods in proprietary models for the FIRST time, which is the source of its novelty. In section 2, we merely use Fast-DetectGPT as an example to show how to use PDE with white-box methods, and we do not assert that its uniqueness is dependent on this comparison.

W2: Datasets used for experiments are small: 150 pairs of human-written/AI-generated texts each.

Reply: Please refer to the reply to Common Concern 1.

W3: In particular, there are concerns regarding the generalization capabilities of the proposed. Also, given data is not large enough to properly train the baseline methods (RoBERTa).

Reply: We do NOT engage in any model training. The benefit of zero-shot detection methods is their generalization capacity, which is possible because we utilize pre-existing LLMs as the scoring model. These LLMs have demonstrated strong generalization abilities across various domains and languages. Our tests across different datasets, source models, and languages have verified these capacities. Furthermore, the RoBERTa baseline was trained by OpenAI as described in section 3.1. Again, it's important to note that we do not conduct any model training.

W4: AUROC is the only quality metric reported. For reference, one of the common practices for Zero-shot AI-content detector evaluation is to set a threshold (by fixing FP-rate) on some 'training’ data (i.e., the probability of falsely declaring human-written text as AI-generated) and then report metrics (TPR/TNR/detection accuracy/etc.) on the `evaluation’ data.

Reply: This is NOT true. We present a comparison of TPR against FPR in Figure 6, which allows us to clearly evaluate the performance of different methods under different FPR conditions.

W5: It is unclear how good this method would generalize across different styles of texts. As it follows from the text, all reported experiments were performed only in “in-domain” setup. Datasets used in this work are very small and style-specific (e.g., scientific-like PubMed and free-form fictional Writing).

Reply: This is the first time I've come across the idea of an "in-domain" setup for zero-shot detection techniques. It's a bit confusing since we don't actually train any model with an "in-domain" dataset. Nevertheless, given that different datasets might have varying metric distributions, we also assess these methods by testing them on a combined dataset of XSum, Writing, and PubMed (integrating the three datasets into one) and present the TPR/FPR in Figure 6.

Dear Reviewer, we have diligently attended to your previously raised concerns. Would you mind providing us with any additional feedback you might have?

Since our previous discussion, we have added more experimental results that we think adequately address your questions.

Initially, we present Mix3 (450 pairs), a combination of XSum, Writing, and PubMed, and Mix6 (900 pairs), a fusion of six languages. This results in datasets that are diverse in terms of domain and language, thereby fulfilling the desired scale and diversity. We've updated the average AUROCs (Avg.) with the AUROCs on the single dataset, as evidenced in Table 1, 2, 3, 4, and 5. The revised scores do not alter the conclusion or the discussion.

Secondly, we've incorporated an out-of-domain setting for both text domains and source models, as displayed in Table 8 and 9. Across these settings, PDE has produced the highest ACCs.

Lastly, we've omitted weaker baselines such as RoBERTa, since our claims are not dependent on comparisons with them. We also discuss the possible implications for non-native writers and reference to the related work in the Broader Impact section 3.5.

Great thanks to Reviewer 32Jz for the mention. We just realize that the "out-of-domain" setting is typically emphasized by recent Binoculars (Hans et al., 2024), which we had missed during our literature review. We have now incorporated this research into our work and have added a section titled “Appendix E.1 Robustness across Source Models and Domains”, and a corresponding recap in Section 3.5, to speak to this "out-of-domain" configuration for both text domains and source models. Our results indicate that PDE maintains the highest ACCs across the source models and domains using a threshold from out-of-domain datasets.

Additionally, we have also updated the references to the open-source models Phi-2, Qwen2.5, and Llama3, which were not included in the previous version.