PaLD: Detection of Text Partially Written by Large Language Models

We appreciate the reviewer's comments and recognizing the novelty of the mixed-text LLM detection setting. We hope that the following responses help resolve some of the concerns of the reviewer.

Response to weaknesses:

W1. This paper assumes that the segmentation occurs at the sentence-level, i.e., a minimal segment would be a single semantically meaningful sentence. This is a natural choice to make, as sentences are a natural way to segment text in human languages, and are thus a likely method humans may edit LLM written texts. It is also natural to use a sentence as the minimal unit of which a piece of text may be considered all LLM or all human. Otherwise, it may be extremely difficult to define what unit of text is considered all LLM or all human. Even at the sentence-level, an "all LLM" text may share words with the "all human" counterpart. This should not mean that only the shared words are "human" and the words in between the shared words of the "all LLM" text are considered LLM-written. Nevertheless, PaLD-TI does assume a predefined segmentation, which means that if the segmentation is misaligned with the ground-truth, then PaLD-TI will not be able to achieve perfect accuracy at the character-level. We note that PaLD-PE (percentage estimation) is agnostic to any specific segmentation.

W2. Regarding additional experiments, as mentioned in the Initial Common Response, we are currently evaluating several new domains from the RoFT and RAID datasets, and will add these results to the manuscript shortly.

Response to questions:

Q1. A threshold for the $T$-score is not necessary for PaLD; the only requirement is to compute $T$-scores. In PaLD-PE, predictions are made by sampling from the posterior of $P(T(X)|\Delta)$. In PaLD-TI, we simply use the $T$-score to compute our objective function. Regarding ablation studies, Sec. 4.3 explores different $T$-scores, and different distributions to model the $P(T(X)|\Delta)$ conditional distribution for PaLD-PE. As you suggested, we will additionally add an ablation study on the choice of bandwidth for when the KDE is used to model this conditional distribution. Below, we report the effect of the bandwidth ($h$) on PaLD-PE's performance on the WP dataset. As mentioned in the appendix, we use Scott's rule to automatically set the bandwidth. Below, we additionally report the average bandwidths chosen by Scott's rule over all the $\phi_k$'s in Sec. 3.2, with one standard deviation. As shown, PaLD-PE is not too sensitive to the choice of bandwidth, and Scott's rule is able to determine a bandwidth resulting in accurate estimates of the LLM fraction.

MAE of point estimates from PaLD-PE on the WP dataset.

Q2. The reviewer's suggestion is great as it is argued in LLM detection that the False Positive Rate (FPR) must be kept low [1], and poses a greater risk than False Negative Rate (FNR). To measure the impacts of both the FNR and FPR, we further report the segment-wise F1-score for the text identification task in the tables below. We also report the F1-score of FastDetectGPT applied segment-wise, which was the strongest baseline in terms of accuracy in Table 3. As shown, PaLD-TI significantly outperforms it in terms of F1-score as well, demonstrating a good balance between precision and recall compared to baselines. In summary, the PaLD-TI outperforms FastDetectGPT not only in its accuracy (Table 3), but also in the F1-score (Table below).

LLM text identification (segment-wise F1-score).

We plan to update the manuscript with these results. Percentage estimation is a regression task, so misclassifications are not applicable.

References

[1] Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2024). Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36.

We appreciate the reviewer's comments and recognition of our work. We hope the following responses help alleviate some of the reviewer's concerns.

Response to weaknesses:

W1. We agree that scaling the method is a relatively minor issue compared to the text identification task, and left for future work. Please refer to the section on complexity of PaLD-TI and greedy algorithm in the Initial Common Response regarding this topic.

W2. Thank you for the suggestion. We are currently evaluating PaLD when it is faced with distribution shifts such as perturbations at inference time, and will update the manuscript shortly.

W3. Regarding additional experiments, as mentioned in the Initial Common Response, we are currently evaluating several new domains from the RoFT and RAID datasets, and will add these results to the manuscript shortly.

W4. Thank you for your suggestion. We will update the related works section with a few papers related to partial LLM-written text, and reduce the section on watermarking. To summarize, most such works detect the boundary when a text goes from being human-written to LLM-written. This means that each text only has two segments, where the first is human-written, and the second is LLM-written. The RoFT dataset [1], containing human-written sentences completed by GPT2, was created to evaluate how humans detect the boundary. [2, 3, 4] provide RoBERTa or Transformer-based models for solving the task in an automated fashion. [5] evaluates several approaches and finds that perplexity-based approaches are more robust for boundary detection. Our proposed framework, PaLD, is more general, as it encompasses settings where any segment of a multi-segment text could be human or LLM-written, does not require white-box access to an LLM, and can be used with both supervised $T$-scores as well zero-shot $T$-scores.

References:

[1] Liam Dugan, Daphne Ippolito, Arun Kirubarajan, and Chris Callison-Burch. RoFT: A tool for evaluating human detection of machine-generated text. In Qun Liu and David Schlangen (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 189–196, Online, October 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-demos.25.

[2] Joseph Cutler, Liam Dugan, Shreya Havaldar, and Adam Stein. Automatic detection of hybrid human-machine text boundaries, 2021.

[3] Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. All that’s ‘human’ is not gold: Evaluating human evaluation of generated text. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 7282–7296, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.acl-long.565

[4] Zijie Zeng, Lele Sha, Yuheng Li, Kaixun Yang, Dragan Gaševi´c, and Guanliang Chen. Towards auto- matic boundary detection for human-ai hybrid essay in education. arXiv preprint arXiv:2307.12267, 2023.

[5] Laida Kushnareva, Tatiana Gaintseva, Dmitry Abulkhanov, Kristian Kuznetsov, German Magai, Eduard Tulchinskii, Serguei Barannikov, Sergey Nikolenko, and Irina Piontkovskaya. Boundary detection in mixed AI-human texts. In First Conference on Language Modeling, 2024

Thank you for taking the time to read the paper and provide valuable feedback! Please find our responses to your comments in the following point-by-point response.

Response to weaknesses:

W1. Regarding clarity of the concepts and mathematical notations, we have addressed these in the responses to your questions below. We will also update the manuscript to reflect these changes. If there is still confusion, we are happy to help clarify specific concepts that are unclear to the reviewer.

W2. Regarding additional experiments, as mentioned in the Initial Common Response, we are currently evaluating several new domains from the RoFT and RAID datasets, and will add these results to the manuscript shortly. Regarding distributional assumptions, there is (i) the mixed-text distributional assumption described around line 191, and there is also (ii) the conditional distribution of the $T$-scores given LLM fraction $\Delta$ that PaLD-PE uses, described in Sec. 3.2. For (i), this assumption of the underlying mixed-text data distribution simply says that each segment is either human or LLM, which is the only assumption that PaLD-TI makes. Note that this assumption is a novel assumption, and a contribution of our work. The mixture ratio $\eta$ is not assumed to be known. PaLD-PE in fact does not make any segment-level assumptions on the text $X$. For PaLD-TI, assuming each segment is either human or LLM is a general assumption that encompasses many scenarios of interest, such as those described in the introduction (e.g., LLMs editing parts of human-writtten articles, or humans refining a draft written by an LLM). For (ii), PaLD-PE fits a model to the $P(T(X)|\Delta)$ conditional distribution and assumes a Beta prior on $\Delta$, which is used to make inferences on the LLM fraction $\Delta$. Our ablation studies in Sec. 4.3 demonstrate that these distributional assumptions do not make a significant difference in performance. While a distribution shift in $P(T(X)|\Delta)$ could presumably occur due to different texts encountered, our cross-domain experiments in Sec. 4.2 demonstrate how a $P(T(X)|\Delta)$ distribution fit in one text domain still yields accurate predictions of the LLM fraction on another text domain.

Response to questions:

Q1. We thank the reviewer for pointing out this confusion. Classic LLM text detection methods, such as (Fast)DetectGPT and Ghostbuster, are designed to perform a binary classification on a paragraph which is either all LLM-generated or entirely human-written. However, in the proposed mixed-text setting, the size of the segment that are LLM-generated or human-written is usually shorter than a paragraph, e.g., only a sentence in a 10-sentence paragraph. This often causes a degradation of the performance of those classic LLM detection methods, as explained in line 94-95 and evidenced in Table 2 and 3. We will improve the wording and add more explanations to clarity in the manuscript.

Q2. Thanks for the suggestion, and we will add references to Bayesian frameworks [1] and nonparameteric models in NLP [2] in the introduction in the updated manuscript shortly. Please also feel free to suggest any reference that is appropriate.

Q3. In line 112, $\mathcal{X}$ is meant to be the sample space of texts that can be randomly drawn, such as all finite-length strings pertaining to a dictionary. Specifically, $\mathcal{X} = \mathcal{D}^*$, where $\mathcal{D}$ is a dictionary of characters, such as all ASCII symbols. We will update the manuscript accordingly to clarify this.

Q4. In this work, we use sentences as segments for the running examples as well as the experiments, so the total number of segments would be the number of sentences of the text. This is a natural choice to make, as sentences are a natural way to segment text in human languages, and are thus a likely method humans may edit LLM written texts. As mentioned in Sec. 5, we leave a more general segmentation for future work.

Q5. Regarding $\eta$ and $\delta$, please see the referenced footnote for an example at the beginning of Sec. 3. $\eta$ represents the fraction of segments that are LLM-written, and $\delta$ is the induced fraction of characters that are LLM-written. Given an $\eta$, a text $X$ is generated from the mixed-text distribution, and then $\delta$ can be calculated from $X$. For the 4-segment text at the beginning of Sec. 3, $\eta$ = 0.5 since LLM generates two sentences. However, there are 109 out of 197 LLM-generated characters, and therefore $\delta = 109/197 \approx 0.553$.

Q6. In Eq. (1), $\epsilon$ is not a hyperparameter, but rather a parameter of the normalized quantile slope set so that $1-\epsilon$ of the probability under the distribution of $T(X)$ is considered. In this way, it is a cutoff for the quantiles of $T(X)$, otherwise an infinite range of quantiles would need to be considered (i.e., $Q(\delta, p)=-\infty$ when $p=0$ and $Q(\delta, p)=\infty$ when $p=1$). Thus, we only consider the quantiles in the range of $p \in (\epsilon/2, 1-\epsilon/2)$ when computing $\sigma$. If $\epsilon$ is set too small, then many (potentially infinite) quantiles in the tails of the distribution (which are less significant) will need to be considered, as mentioned above. If $\epsilon$ is set too large, then not enough of the $T(X)$ distribution will be considered, and the estimated quantile slope may not reflect the full distribution of $T(X)$. In practice, when $\sigma$ is estimated, we set $\epsilon = 0.05$, so that 95% of the quantiles of $T(X)$ are considered, and 5% of the less relevant tails are ignored.

Q7. The $T$-scores of all the combinations of segments will be considered, as implied by Eq. (4). Note that in this example each segment is a sentence. All subsets $S \subseteq \lbrace 1,\dots,n\rbrace$ are considered to maximize the $T$-score difference. Since we disregard $S = \emptyset$ and $S^\complement = \emptyset$ as mentioned in Sec. 3.3, the example in Fig. 6 shows all the subsets of $\lbrace 1,2,3\rbrace$ except $\emptyset$ and $\lbrace 1,2,3\rbrace$, which happen to be 1- and 2-sentence combinations. If the text has $k$ sentences, then PaLD-TI will consider 1- to $k-1$-sentence combinations.

Q8. There are no theoretical guarantees on the approximation error that the greedy algorithm yields for our optimization problem in Eq. (4), as the objective function is unlikely to be submodular in $S$. Despite this, we experimentally show in Table 3 that PaLD-TI with greedy algorithm outperforms all other baselines by at least 9% in terms of segment accuracy, with PaLD-TI with exact solver outperforming the baselines by at least 17%. Please refer to the Initial Common Response for a more in-depth discussion regarding the complexity-performance tradeoff of PaLD-TI and greedy algorithm.

Q9. The WP and Yelp datasets are from different domains and have different collection processes. They contain rather different text samples. WP's texts contain human-written short stories, and Yelp contains human-written reviews of restaurants, businesses, etc. This was in fact mentioned at the beginning of Sec. 4. In Table 9 (appendix), we show text samples (human and mixed-text), where the first two rows are examples from Yelp, and bottom two rows are examples from WP.

Q10. Regarding additional experiments, as mentioned in the Initial Common Response, we are currently evaluating several new domains from the RoFT and RAID datasets, and will add these results to the manuscript shortly. Regarding more challenging cases such as human mimicking LLM styles, we are not aware of any current datasets that could support this experiment currently. We appreciate the suggestion and have mentioned these in the final remarks.

References:

[1] Shay Cohen. Bayesian Analysis in Natural Language Processing. Springer International Publishing, 2019.

[2] Narges Sharif-Razavian and Andreas Zollmann. An overview of nonparametric bayesian models and applications to natural language processing, 2008.

Thank you for taking the time to read the paper and provide valuable feedback! Please find our responses to your comments in the following point-by-point response.

Response to weaknesses:

W1. Thank you for bringing the potential distribution mismatch to our attention. In fact, we choose to include the experimental results with RoBERTa trained on paragraph-level texts since the RoBERTa base model is pretrained on longer texts. Using sentence-level text is too short for the RoBERTa architecture. For the sake of completeness, we provide the RoBERTa classifier baselines trained on segment-length texts rather than paragraph-level texts, and report the resulting performance for both percentage estimation (PE) and and text identification (TI) tasks in the tables below. When compared to the baselines trained on paragraph-level data, their performance is actually worse (note that both training on paragraph-level and sentence-level do not perform well compared to other methods), despite the potentially better distributional alignment. We plan to update the manuscript with these results.

Percentage estimation (MAE); RoBERTa classifiers trained on segment-level data.

LLM text identification (segment-wise accuracy); RoBERTa classifiers trained on segment-level data.

W2. Thank you for the suggestion. We have re-ran the DetectGPT and FastDetectGPT segment-wise baselines by setting the threshold to maximize the difference between TPR and FPR on a segment-level validation set. We show the difference by setting threshold at the paragraph level vs. sentence level in the tables below (it will be updated in Tab.~2, 3 in the upcoming revised version). There is an improvement in their performance, albeit around $<0.03$ in MAE for the percentage estimation task, and $<1\%$ for the segment-accuracy in the text identification task.

Percentage estimation (MAE):

Text identification (segment accuracy):

W3. Thank you for your suggestions. Regarding unfairness of comparing zero-shot with supervised methods, in our paper, we are not restricting comparisons to only zero-shot methods, thus it is fair to compare against baselines that contain both zero-shot and supervised methods in Table 3. If one wishes to only compare among zero-shot methods, we indeed already have a zero-shot version of PaLD-TI, with PaLD-TI results using unsupervised T-score in the ablation study in Table 8, which reports PaLD-TI performance using a variety of T-scores. Comparing Table 3 and Table 8, we see that for each T-score used, PaLD-TI improves the text identification performance compared to the T-score segment-wise baseline; this improvement is most drastic for the RoBERTa, RoBERTa-LN, and DetectGPT T-scores, the last of which is zero-shot.

W4. Regarding additional experiments, as mentioned in the Initial Common Response, we are currently evaluating several new domains from the RoFT and RAID datasets that you suggested, and will add these results to the manuscript shortly.

W5. For the sentence-level mask-and-fill data generation, non-instruction-tuned LLMs do not perform well. For example, we found that Llama-3-8b without instruction tuning is not able to generate meaningful fills. As an example, when Llama-3-8b is prompted with our mask-and-fill prompt (Appendix A.1),

What sentences should go in the 3 [MASK] locations of the following text? Only provide exactly one sentence per [MASK] location. Only provide the sentences as a numbered list with 3 sentences total. 
[MASK]. I got a taylor ham, egg, and cheese and a short stack of blueberry pancakes (I simply couldn’t decide between sweet or savory, plus leftovers are never a bad thing). The egg sandwich was fantastic. I was nervous when they said they didn’t have kaiser rolls so I went with a hamburger roll. It was an excellent decision. [MASK]. They layered the cheese on the sandwich, which to me is a must for a true egg sandwich. [MASK].

The response is:

1. The blueberry pancakes were good, but not as good as the egg sandwich. 
2. The egg sandwich was fantastic. 
3. They layered the cheese on the sandwich, which to me is a must for a true egg sandwich.

Similarly, we show an example using the Llama-3-8b-instruct:

1. The service was friendly and attentive, which is always a plus.
2. The egg sandwich was a bit messy, but that's just part of the charm.
3. The blueberry pancakes were fluffy and flavorful, with a perfect balance of sweet and tangy.

Note that for the non-instruction-tuned Llama-3 model, fills 2 and 3 are copied and repeated from the original human-written text. Thus, to consider more LLMs for infilling, we are currently applying other instruction-tuned LLMs for evaluation of PaLD (see Table 21 for an example with the same prompt with GPT-4o), and will update the response soon. In addition, the new RoFT dataset contains human-LLM mixed text, where the LLM is GPT2-XL, which will provide another example for mixed text generated by another LLM other than GPT-4o.

W6. Regarding the combinatorial search for PaLD-TI and other efficient methods, please refer to the Initial Common Response.

Response to questions:

Q1. The notation in the manuscript is correct. $X$ is the entire random mixed text, comprised of many segments. The $i$-th segment, $X_i$, is generated from the mixture distribution.

Q2. Thank you for pointing out the typo, we will fix it in the updated manuscript.

Q3. Actually, the fractions of the texts in the training split and testing split do match. Note that the target fractions for the testing split are 0.25, 0.5, and 0.75, but as described in Appendix A.1, this is not actually the true LLM text percentages, but rather the target fractions, which determine the number of masked sentences for the mask-and-fill procedure. Approximately 0.25, 0.5, and 0.75 of the sentences are masked out and filled by GPT-4o, then the true LLM fraction $\delta$ values are computed post-hoc. The true fractions depend on the text content and thus vary. We found that this procedure yielded similar distributions of $\delta$ over $[0,1]$ compared to setting the target fractions to 0.1, 0.2, ..., 0.9 as done for the training split. To provide concrete evidence for this, we compute a histogram of the $\delta$ values on the Yelp dataset for both train and test splits:

Frequency of $\delta$ occuring in interval bins, on Yelp dataset:

Thus, despite the ``target'' fractions differing between the train and test split, the true fractions actually match. In the upcoming revision, we will make this point clearer. Thank you for bringing it to our attention.

Q4. We would like to emphasize that there can be a lot of noise in the estimation of the quantile slope, as it may depend on several sources of randomness such as the text distribution as well as the stochasticity of the $T$-scores (which is true for DetectGPT, Ghostbuster, and FastDetectGPT). Therefore, the quantile slope is only a proxy for the distinguishability of different $\delta$ values. As the quantile slopes for DetectGPT, Ghostbuster, and FastDetectGPT are the three smallest, and all around a similar value, their correlation with the resulting segment accuracy may be more influenced by randomness compared to the correlation between the higher quantile slope $T$-scores (e.g., RoBERTa) and their resulting segment accuracies. Further analysis on a good proxy, and also what properties a good detector should have, should be considered as an interesting future direction.

Other Experimental Additions

In addition to the evaluations on new LLM data and new domains, we have updated the manuscript with the following:

• Reviewer DuTY: A comparison on the distribution of the ground-truth LLM fraction $\delta$ between the training and testing split of Yelp, demonstrating that although the target fractions differ, the actual character-level LLM fraction distribution is the same. This is shown in Appendix A.1 (Table 9).
• Reviewer DuTY: An ablation study showing the ineffectivness of finetuning the RoBERTa segment-wise baselines on segment-level data, shown in Appendix A.3 (Tables 10, 11).
• Reviewer DuTY: Updated results on DetectGPT and FastDetectGPT baselines, where the threshold chosen is now based on segment-level validation data. This is shown throughout the results section (Tables 2 and 3), and a direct comparison between thresholding based on segment-level vs. paragrpah-level data is shown in Appendix A.3 (Tables 19, 20).
• Reviewer DuTY: Results on how PaLD performs on text consisting of all-LLM or no-LLM segments in Appendix A.3 (Tables 16, 17).
• Reviewer 1FyB: An added section in the related works discussing partial LLM-written text, namely LLM boundary detection
• Reviewer 1FyB: Evaluation of PaLD when subject to paraphrasing attacks, shown in Sec. 4.3 (Table 7).
• Reviewer qnTJ: An ablation study on the KDE bandwidth for PaLD-PE, shown in Appendix A.3 (Table 13).
• Reviewer qnTJ: Results on segment-level $F_1$ scores for LLM text identification in Appendix A.3, providing more context on misclassifications (Table 12).
• Reviewer qnTJ: A result demonstrating the effect of segmentation mismatch in Appendix A.3. (Table 18).
• Reviewers DUtY and 3hmG: Minor typos or clarification throughout the text
• Reviewers DUtY, 3hmG and 1FyB: Discussion on practical runtimes of PaLD-TI with exact and greedy solver (Section 4.1).

We hope that these new updates help alleviate the reviewers' concerns about our work.