Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection

Thank you for your insightful review. To address your concern, we would like to provide the response to your reviews:

W1: Unvalidated Assumption of LLM Homogeneity: a central assumption is that LLM-generated texts form a compact, learnable distribution, unlike human texts. However, this assumption is neither empirically validated nor tested across different models or domains. If LLM texts are multimodal or diverse, this could possibly make method worse

Thank you for your suggestion. As NeurIPS does not permit the inclusion of PDFs or anonymous external links in the rebuttal this year, we apologize in advance that we are unable to provide annotated figures or visual clarifications. We hope our written explanations are sufficiently clear, and we are happy to incorporate this additional clarifications in the final version. For this question, we can provide the inter- and intra-LLM and human text cosine distance across diverse domains:

We compute the average cosine distance on the Deepfake dataset, which contains domains such as cmv, xsum, eli5 and so on, and LLMs including GPT-3.5-turbo, GLM-130B, Bloom-7B and so on. The embedding backbone we use is RoBERTa pretrained with contrastive loss. The results show that the Intra-distance of LLM is smaller than the human while the inter-distance between human and LLM is relatively large, supporting our OOD hypothesis that LLM text forms a coherent in-distribution cluster, while human-written texts—spanning varied styles and domains—act as natural OOD examples.. We hope this can address your concern.

W2: Lack of Ablation: the proposed framework combines the OOD loss with a supervised contrastive loss. However, the paper does not ablate the contribution of this component. It is unclear whether the performance gains are because of the OOD formulation itself or primarily from the contrastive learning

Thank you for your suggestion. We will the ablation of each loss component:

These results highlight the critical role of the OOD loss. Even without contrastive learning, the OOD loss alone substantially improves performance, especially in terms of FPR-95%, indicating its strong discriminative power. While contrastive learning offers additional benefits, the OOD loss is the primary driver of robustness and separation between distributions.

W3: Old Datasets: most of the datasets contains relatively old models, which generations could be very easily detected

Thank you for the helpful comment. At the time of our submission, RAID was already the most up-to-date benchmark available. To further address your concern, we split the RAID test set to isolate newer models such as GPT-4, ChatGPT, and LLaMA-70B-Chat, and additionally evaluated our method on a recent benchmark, EvoBench [1], using the XSum dataset with GPT-4o outputs. The results are as follows:

These results show that our method maintains strong performance even on outputs from state-of-the-art models, demonstrating its effectiveness beyond older-generation LLMs.

Q1: Your approach relies on score-based decisions (e.g., DeepSVDD radius, HRN sigmoid, energy scores), but the paper does not describe how inference thresholds are selected. Are these thresholds fixed, learned, or tuned on validation data? Are they consistent across datasets or adjusted per domain? Please clarify this, and ideally include ablation or robustness analysis over threshold choices.

Thank you for the thoughtful question. We would like to clarify that our evaluation metrics—ROC-AUC, PR-AUC, and FPR at 95% TPR—are all threshold-independent or use standardized criteria for threshold selection. Specifically, ROC-AUC and PR-AUC measure performance across all possible decision thresholds, providing a holistic view of the model's discriminative ability. The FPR at 95% TPR metric, while threshold-based, uses a fixed target TPR to compute the corresponding FPR, ensuring consistency and comparability across models and datasets. Therefore, our method does not require the manual tuning or learning of specific thresholds during inference, and the reported results are not sensitive to any particular threshold choice.

Q2: The proposed framework combines the OOD loss with a supervised contrastive loss. However, your ablation study (Table 3) only compares different OOD losses against binary classification, but does not isolate the effect of the contrastive term. Could you include an ablation where and separately to assess how much of the gain comes from contrastive learning? This would help assess whether the gains are due to the OOD formulation itself or enhanced representation learning from contrastive supervision.

Thank you for your question. We have already provided the ablation study of each loss component in W2. Please have a check. Hope this could address your concern.

Q3: Can you quantify somehow intra-class variance of LLM texts across different domains or models? It would be better to show statistically that LLM texts are indeed statistically coherent and form a compact in-distribution set. This would strengthen the argument for using a one-class formulation and help validate the theoretical motivation.

Yes, we have already provided the intra and inter distance in W1. Please have a check.

Q4: Most datasets used in evaluation include outputs from relatively early-generation LLMs. Do you plan to test on newer model outputs, or maybe select a subset from data which contains only newer models?

Yes, we have provided the results in W3. Please have a check.

[1] Yu, Xiao, et al. "EvoBench: Towards Real-world LLM-Generated Text Detection Benchmarking for Evolving Large Language Models." Findings of the Association for Computational Linguistics: ACL 2025. 2025.

We hope our responses adequately address all your concerns. If you have any further questions or feedback, we would be glad to continue the discussion during the discussion phase.

Thank you for your insightful follow-up questions.

You are absolutely right in noting that the distinct clustering of LLM- and human-generated texts suggests the feasibility of separating them using a simple hyperplane. In fact, as shown in Table 3, even a basic classification head achieves reasonable performance across all benchmarks (e.g., AUROC of 89.0 on DeepFake). However, we emphasize that merely relying on a classification head may limit generalization, especially in scenarios such as higher-quality LLMs and adversarial paraphrasing (as you can see, the performance of the single classification head drops on the RAID dataset, which includes various attacked data), where the boundary between LLM and human texts becomes more subtle. Therefore, instead of learning to discriminate human versus LLM texts directly, our method focuses on learning a compact and consistent representation of LLM outputs, treating diverse human-written texts as OOD due to the difficulty of modeling true human distribution. Moreover, when computing the distance, we have already applied the L2-normalization to the output representation.

We hope this can address your concerns. Thanks a lot.

Thank you for your detailed review and comment. To address your concern, we would like to provide the response to your reviews:

W1: One-class/OOD ideas have a long history in NLP anomaly detection; related work could cite and empirically compare to prior one-class text detectors.

We believe there is OOD idea in previous NLP anomaly detection for classification, which is not applied in LLM detection. However, we would like to reiterate that we are the first to apply out-of-distribution detection methods to LLM detection. Moreover, we validate our method empirically and theoretically. Specifically, we have demonstrated the effectiveness of the algorithm on a range of classic OOD methods such as DeepSVDD, Energy-based and HRN OOD detection methods. Our method achieves SoTA performance on diverse LLM detection tasks including multilingual and attacked text settings. Meanwhile, the generalizability is also validated.

We would like to kindly ask the reviewer to provide the specific citation of prior one-class text detectors. We are happy to provide the comparison in LLM detection settings.

W2: The χ²-divergence in Theorem 2 is not empirically estimated, so the link between theory and practice remains qualitative.

Thank you for your suggestion. We would like to provide the empirical $\chi^2$-divergence estimation of our setting:

We evaluate the $\chi^2$-divergence on Deepfake dataset, which contains multiple domains and source LLMs. We use the full human set as the true distribution and sample subsets as the fitted distribution. We apply the RoBERTa model to compute the embedding. The results show that while some domains such as squad and xsum obtain low $\chi^2$-divergence, some other domains like sci_gen and eli5 gain significantly high divergence. Moreover, the average divergence is also large (9.0491), demonstrating the gap between the fitted and true human distribution.

We would like to point out that in Theorem 2, we unfortunately have a typo: the last term for the loss of $\hat{D}$ should be $0.9 q_H (D+1) \cdot \Delta$ instead of $0.9 q_H (D-1) \cdot \Delta$. We corrected this typo in the appendix and our final version. Thus even with smaller values of the $\chi^2$-divergence (e.g. D<1), the potential difference in loss is still significant.

W3: Encoder choice fixed to RoBERTa. How would results change with more recent lightweight or multilingual encoders?

Thank you for your suggestion. In fact, we already included the results of different backbones in Table 6 in the appendix including BERT, RoBERTa and FLAN-T5. Following your suggestion, we provide results of more efficient backbone E5-small with 33M parameters and Multilingual-E5-base:

The model is trained with the identical setting on DeepFake and DeepSVDD. Both efficient and multilingual backbones achieve comparable results, indicating the robustness of our method for backbone choice.

W4: How sensitive are DeepSVDD radius or energy margins to training-set size?

We clarify that in our DeepSVDD experiments, we only use simple one-class setting which does not rely on the radius. For the energy margins and training-set size, we provide the sensitivity analysis experiment on DeepFake dataset:

We can see that the model performance is robust with different energy margin and training set size.

We hope our responses adequately address all your concerns. If you have any further questions or feedback, we would be glad to continue the discussion during the discussion phase.

Thank you for your concrete comment and suggestion. To address your concern, we would like to provide the response to your reviews:

W1: While the authors provide experiments supporting their method, some of the intuition provided could benefit from some experiments that demonstrate the validity of the assumptions. For example, the OOD framework and modeling the LLM text distribution as "in-distribution" could benefit from some plots demonstrating that this is indeed the case in practice. My understanding is that Figure 1 is not a visualization of text embeddings on real data. If a figure demonstrating, say, the text embeddings of LLM texts and different human texts (say, coming from different domains or writing patterns) was shown, this would help support the OOD hypothesis. This could help clarify whether textual domain shift or human/LLM writing is the primary cause for a distribution shift. Right now it seems the authors argue that all LLM text across various domains can be viewed as in-distribution, whereas human text distributions are more varied.

As NeurIPS does not permit the inclusion of PDFs or anonymous external links in the rebuttal this year, we apologize in advance that we are unable to provide annotated figures or visual (such as t-SNE of embeddings) clarifications. We hope our written explanations are sufficiently clear, and we are happy to incorporate this additional clarification in the final version. For this question, we can provide the inter- and intra-LLM and human text cosine distance across diverse domains:

We compute the average cosine distance on the Deepfake dataset, which contains domains such as cmv, xsum, eli5 and so on, and LLMs including GPT-3.5-turbo, GLM-130B, Bloom-7B and so on. The embedding backbone we use is RoBERTa pretrained with contrastive loss. The results show that the Intra-distance of LLM is smaller than the human while the inter-distance between human and LLM is relatively large, supporting our OOD hypothesis that LLM text forms a coherent in-distribution cluster, while human-written texts—spanning varied styles and domains—act as natural OOD examples. We hope this can address your concern.

W2: Building on the previous point, I think some experiments regarding domain vs. human/LLM should be provided. My understanding is that the LLM-written parts of the training splits of the datasets are used to train the models. But if further splits across domains could be provided, that would be helpful to understand the method.

Yes, thanks for the suggestion, we provide the results of model trained within specific domain:

These strong per-domain performances indicate that the model effectively distinguishes LLM vs. human text within individual domains, further supporting the validity of our OOD formulation.

W3: Is there a zero-shot variant based on the OOD framework that could be developed? The training of such detectors can present obstacles to future times when stronger and higher-performing LLMs are developed

We would like to first thank the reviewer for providing such an insightful and constructive proposal. It is indeed pretty interesting to extend our out-of-distribution detection framework to zero-shot methods. However, it is non-trivial and challenging to simply apply our current framework to zero-shot methods. The reason is that the current OOD detection methods generally require supervised (one-class) training, which is not compatible with current metric-based zero-shot methods. Nevertheless, we believe the idea behind our method, namely considering the human-generated texts as OOD data, can help to design novel statistic metrics for zero-shot methods, for example, help to find the best metric threshold in the zero-shot methods.

W4: A brief comparison of training/inference time or parameter counts versus baselines would be helpful to understand the performance/complexity tradeoffs, which is another practical concern when deploying such detectors.

Thank you for you suggestion. We provide the training and inference cost comparison of our method and baselines:

The time of training and inference is evaluated on single 4090 GPU. The backbone we use is unsup-simcse-roberta-base. For the training time, the training time per epoch is similar to the DeTeCtive baseline since we share the same backbone model and the difference is the loss objective. However, our method leads to faster convergence, which reduces the overall training cost compared with DeTeCtive. For the inference, our method is also the fastest since FastDetectGPT requires LLM inference (such as GPT-Neo-2.7B), which needs 10 $\times$ more computation and DeTeCtive needs database construction and KNN for searching, which brings extra inference time.

W5: Recent focus has shifted to edited LLM texts, or partial human/LLM text. This is a more accurate representation of how LLM texts are used in practice. It would be interesting to see how the method would work on such data.

Thanks for raising this up. We want to clarify that the test set RAID used in our experiments has already included the attacked data including paraphrase, insert paragraphs and so on. Moreover, we would like to provide the results comparison with baselines on test set with only attacked text from RAID:

Our method achieves a substantially higher AUC_ROC and significantly lower FPR-95% compared to the baseline, indicating strong robustness on edited or partially human-generated text.

We hope our responses adequately address all your concerns. If you have any further questions or feedback, we would be glad to continue the discussion during the discussion phase.

Thank you for your detailed review. To address your concern, we would like to provide the response to your reviews:

W1: No empirical or theoretical evidence is provided about a difference in choice from existing work. That is setting human generated text as OOD and not machine generated text as OOD.

Thanks for the suggestion. We provide the result of setting machine-generated text as OOD:

The results are trained on DeepSVDD and Deepfake dataset without contrastive loss. The strong performance of human generated text as OOD indicates that modeling human text as OOD is a valid and effective formulation.

W2: While author formulate problem as single class, from equation 6 and equation 10 i.e. L_contrastive loss it seems author use both human and machine generated text during training. Since the loss weights α and β are set as 1 in main experiments, the contrastive loss (binary classification loss) also play equal role in training. Hence, it would be good to understand performance of method with β set to 0 to truly understand if the gains are from single class modeling or augmenting contrastive loss with another loss that constraints intra-class embeddings for machine generated text. In supplement Figure 3, author show metrics remaining same for different ratio of alpha:beta however adding ablation for β =0 and α =0 will help.

We will provide the results when $\alpha$ and $\beta$ is set to 0:

Without OOD loss ($\beta$ = 0), the AUC_ROC is degraded to 88.6 while with only OOD loss, the AUC_ROC achieves 93.7 and the FPR at 95% obtains 33.7, demonstrating the importance of OOD loss.

Q1: some theoretical/empirical justification of considering human generated text as OOD v/s machine generated text as OOD will help

Thank you for your question. We have already provided the results comparison of human-generated text as OOD and machine-generated text as OOD in W1. Please have a check.

Q2: benchmarking gain from adding OOD loss to contrastive loss or removing contrastive loss completely can inform better if the gains is from single class modeling or improving contrastive loss with another loss term that constraints intra-class embeddings for machine generated text?

We have provided the results of different loss choice in W2. Please have a check.

We hope our responses adequately address all your concerns. If you have any further questions or feedback, we would be glad to continue the discussion during the discussion phase.

I thank the authors for running additional experiments to address my concerns. My concerns regarding human as OOD v/s machine as OOD and role of OOD loss in performance has been resolved. I am more confident about my positive recommendation now. Thanks