OSTAR: Optimized Statistical Text-classifier with Adversarial Resistance

Weakness 1: Thank you for your observation; when implementing text rewriting, we perform full-document perturbations and have specified in the appendix that our perturbations include sentence-level attacks such as Sentence Reversal: inverting word order and Sentence Repetition: duplicating clauses/phrases—we maintain our adversarial selections and testing demonstrate certain universality.

Weakness 2: Thank you for your observation; we have adopted multiple word-level attacks here including Word Merging, KeyBoard Typos, Character WordCase, Spelling Errors etc. as shown below, and can additionally incorporate BERT-based adjective replacement attacks if necessary.

Weakness 3:

Thank you for your feedback! All statistical feature dimensions in our proposed method demonstrate utility, as evidenced by the ablation studies on feature dimensions in the table below. The results reveal that, for these specific features, our designed approach represents the optimal combination identified thus far. Nevertheless, our primary goal is not to establish a definitive framework for constructing statistical features, but rather to propose a unified methodology that integrates statistical feature-based analysis with classifier-based techniques to enhance AI-generated text detection. We hope this work inspires further exploration into more effective statistical modeling approaches, motivating researchers to advance this field by combining statistical and classification-based strategies.

Question 1: Thank you for your question;！In related studies we have incorporated alternative textual quality assessment methods as statistical features—such as evaluation metrics spanning literary sophistication and textual sentiment domains—and integrating these into our framework constitutes an effective feature selection strategy where valid features should further enhance performance when added to the system.

Question 1: Thank you for your question. Regarding defense against multiple adversarial attacks, a viable solution involves incorporating combinations of various attack methods into the training data during model development to enhance the model's ability to recognize such complex scenarios. As for the short-text detection challenge, we can introduce additional statistical or contextual features independent of text length. These improvements would significantly enhance the model’s performance in complex attack scenarios and short-text situations.

Question 2: Thank you for your question! We've detailed adversarial attack metrics in Appendix E. Below is the enhanced specification table:

Question 3:

​ Thank you for your question! The contrastive learning strategy does indeed significantly increase training time and hardware resource requirements. Here we provide a comparison between training without contrastive learning and training with it: specifically, VRAM usage increases by approximately 50%, while training time rises by roughly 1.5 times.

Question 4:

Thank you for your question; maintaining balanced proportions between human texts and machine-generated texts during training is crucial as severe data imbalance significantly undermines model performance: excessive machine texts cause the model to overlearn mechanical patterns, generating outputs lacking human linguistic diversity and naturalness while degrading recognition of human text features and lowering human text recall rates; such imbalance distorts learning objectives, creating biases toward dominant data sources and ultimately impairing generalization capabilities and overall performance in text generation, content detection, or hybrid text processing tasks.

Weakness 1: Our method comprises MDSP feature extraction, statistical features, and the MFCL optimization strategy. Key hyperparameters include MDSP's sliding window size (adjusts feature scope; small sizes limit text-variation reflection, large sizes introduce noise) and MFCL's contrastive learning weight (balances robustness against attacks vs. primary task focus). Experiments validating these parameters are tabulated below. Regarding the sliding window size, a window size of 3 achieved optimal balance (accuracy ACC: 91.94%, F1-score: 92.36%), enabling sufficient context feature capture while avoiding interference from distant text noise. Excessively small windows lacked contextual awareness, whereas larger windows (size 5) introduced irrelevant signals and imposed stringent text length requirements, performing inadequately on shorter texts. This sensitivity experiment demonstrated that a window size of 3 achieves the most robust feature extraction by optimizing local context modeling.

Regarding the sensitivity of the contrastive learning weight, this parameter exhibits high sensitivity. Performance improves significantly within the 0.05 to 0.02 range, but declines when the weight falls below 0.02. The final selection of 0.02 is based on the simultaneous peaking of both metrics at this value (Pert=81.27, Para=81.46), representing the optimal balance between the effect of contrastive learning and primary task performance.

Regarding further implementation details, we provide a table of attack impact rates here; if the paper is accepted for publication, we will include it in the main text to enhance the clarity of implementation details.

Weakness 2 : Thank you for pointing out. ​L_Para​ refers to the contrastive learning loss function for paraphrase attacks, while ​L_Pert​ refers to the contrastive learning loss function for perturbation attacks. The calculation of both aligns with the two types of data generated during the "Adversarial Data Preparation" stage mentioned earlier. Together with their corresponding coefficients ​λ₁​ and ​λ₂, they constitute the MFCL loss function. In the original text, we described: “MFCL can be divided into Paraphrase Contrastive Learning (Para-contrast) and Perturbation Contrastive Learning (Pert-contrast)” and “λ₁ and λ₂ denote the weighting coefficients for L_Para (paraphrase loss) and L_Pert (perturbation loss), respectively.” ​Issue:​​ The original description lacks sufficient correspondence with the "Adversarial Data Preparation" section. ​Corrected version​ (to be implemented if accepted for publication): “MFCL can be divided into Paraphrase Contrastive Learning and Perturbation Contrastive Learning, which generate two loss functions: L_Para and L_Pert, respectively.” and “λ₁ and λ₂ denote the weighting coefficients for L_Para and L_Pert, respectively.”

Weakness 3: Thank you for your feedback. While leveraging contrastive learning to enhance model robustness is indeed a mainstream approach, our work introduces a core innovation: for the first time in AI-generated text detection, we propose a novel adversarial attack taxonomy fundamentally based on whether the generative source is altered. This foundational classification led to our pioneering MFCL optimization strategy—a specialized contrastive learning paradigm tailored for complex adversarial scenarios in this field. By differentially addressing distinct attack types, MFCL significantly strengthens detection robustness against intense adversarial perturbations. We contend that this deep integration of domain-specific attack mechanisms into the contrastive learning framework constitutes a substantive innovation for the domain.

Weakness 4: Thank you very much for pointing this out. We have corrected both issues and performed another double-check of the entire manuscript. Should our paper be accepted, these errors will not appear in the printed version.

Question 1: Thank you for your question. Based on the discriminability and stability metrics after adversarial attacks in Appendix A, Syntax Features contribute most significantly. We conducted ablation studies of statistical features on a small-scale dataset (GML130B for deepfake), with results shown below. Although improvement levels vary, all four-dimensional features enhance performance. This method primarily serves as an attempt to integrate statistical and fine-tuning approaches, and we plan to incorporate more statistical features to improve generalizability in future work.

Question 2: We sampled 500 HC3 points and tested OSTAR under unseen single and combined attacks. Its robustness varies by adversarial intensity: OSTAR's adversarially-trained features transfer well to unseen single attacks. Even against novel BERT-generated attacks (e.g., rewriting 20% sentences), it retains high accuracy. F1-score dips only ~2-3% versus known attacks, achieving 94.03—confirming stable semantic attack comprehension. For unseen combined attacks (perturbation→paraphrase or paraphrase→perturbation with random perturbations), statistically optimized features struggle with complex multi-step sequences. Contrastive learning remains robust, scoring F1 of 92.83 and 89.21 respectively—acceptable degradation versus seen attacks. Enhanced training coverage or advanced learning strategies could further improve performance.

Question 3: ​ ​ Thank you for your question. We believe directly employing large language models for machine-generated text detection is unreliable, as these models weren't designed for such tasks. Their vulnerability under attacks is substantial, as evidenced in the table below. We tested two cutting-edge large models on 1,000 DeepFake samples under paraphrasing attacks, yielding unacceptable results. Moreover, classification outcomes lack interpretability, and training such models for enhanced robustness proves challenging. The prompt： "You are a text classifier. Your task is to determine if the following text was generated by a machine or written by a human. "     "Respond with ONLY the single word 'machine' if the text is machine-generated, or 'human' if it is human-written. "     "Do not include any explanations, punctuation, or additional text. "     "Example responses:\nmachine\nhuman\n"

Meta-Llama-3-1-8B-Instruct Classification Report

DeepSeek-R1-Distill-Qwen-7B

However, utilizing large models for feature extraction remains viable. For instance, we're currently exploring their use to derive statistical features related to text evaluation metrics (commonly applied in other text processing methods) as foundations for model training.

Answer 1: Our "intrinsic invariant features" refer to the universal characteristics of machine-generated text—specifically, those properties that maintain relative stability even after text undergoes rewriting or perturbation by another text generator. While validation set design partially addresses the issue of "capturing superficial correlative patterns in training data" (since practitioners adjust training methods—such as epochs, learning rates, and regularization—based on validation performance), this remains fundamentally a training strategy. Merely tuning hyperparameters without structurally enhancing the network's capability yields limited improvement. Therefore, we explicitly extracted statistically stable features for identifying AI-generated text and implemented the MFCL contrastive learning strategy to capture the intrinsic relationships between attacked texts and original texts. This approach significantly boosts the model's capacity to recognize and interpret inherent characteristics of AI-generated content. Moreover, optimizing training strategies via validation sets and enhancing model performance through architecture improvements represent two distinct pathways for capturing intrinsic task features: one evaluates whether the training process achieves intrinsic feature extraction, while the other expands the model’s feature extraction capability by enriching architecture and feature categories. These methods are mutually non-conflicting and can be applied concurrently. In our training, we indeed leveraged validation sets to optimize model performance.

Answer 2: In Figure 3 and Appendix A, we provide the average differences between human and machine features under the MDSP method across three datasets, along with the post-attack distributions of both human and machine texts from these datasets. These results demonstrate that our proposed method captures intrinsic patterns of machine-generated text, effectively distinguishing human text from machine text—even when facing various attacks. Though statistical feature ranges shift in both human and machine texts after attacks, their distributions remain analogous; simultaneously, distinctions between the two text types persist. Through further training, these patterns become capturable as intrinsic regularities. Therefore, we assert that our original paper provides experimental evidence confirming our method's extraction of intrinsic patterns. We can supplement additional experimental analyses in the appendix to help readers better understand how our method captures inherent distinctions between AI-generated text and human text.

Answer 3: In fact, current methods perform well on the original datasets but exhibit poor performance on attacked datasets, as shown in the Para. and Pert. columns of Table 2. Here, we present results of existing methods on the CheckGPT dataset from Table 2, which demonstrate significant room for improvement in performance. Therefore, we believe that advancing this metric under adversarial conditions holds meaningful significance for the progress of the field.

Answer 4: Figure 1a provides background on the AI-generated text detection task in adversarial settings. We included this figure to give readers a clearer understanding of our task. The current description of Figure 1a in the main text is not precise enough. Originally, we introduced it as follows: “As shown in Figure 1(a), in real-world scenarios, MGT texts are often attacked to evade detection, critically challenging the robustness of existing detection methods[6, 7, 8].” We will revise it to: “In real-world scenarios, users frequently employ adversarial tactics to evade detection when attempting to bypass machine-generated text detectors. This poses significant challenges to the robustness of existing detection methods [6, 7, 8]. To explain how these adversarial tactics work, which can help clarify our detection task, Figure 1(a) provides an introduction to the process regarding the robust MGT detection task background.”

In Figure 1b, our method and existing methods based on statistical features and classifiers are already connected to some extent (using color) and show differentiation (with clear procedural differences). However, we can add more explicit annotations in the future to highlight our innovation.

Answer 5: Thank you for your correction. Experimentally, OSTAR indeed performs best on para and pert datasets. However, its slight degradation in performance on the original data—caused by the necessary incorporation of contrastive and adversarial learning—explains the differences shown in the table. We will further revise the manuscript and double-check the content to eliminate any oversights or basic errors.

Answer 6: We apologize for not providing the code link. Since including any external links is prohibited during the review process, it was not included in the manuscript. The code and relevant links will be made accessible online in the camera-ready version upon acceptance.