DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning

We sincerely thank the reviewer 1CCF for the thoughtful and detailed comments. Below, we provide point-by-point responses to each of the concerns.

---

Weakness1: While it appears from the resutls that we can detect collaboration on datasets different from those in training, the process of creating these simulated test cases is so similar to that in the training that it is difficult to argue that the approach can be applied in new scenrios [...]

Thank you for your positive feedback. In our generalization analysis, we did not solely rely on our own dataset, RealBench, but also incorporated challenging datasets like DetectRL, Beemo (Table 2), and HART (Table 3), which are recently published. Notably, HART contains data generated by real human-AI collaboration. The three levels of testing proposed by HART are particularly relevant to real-world complex application scenarios.

The motivation for our "human-AI collaboration detection" task inherently encompasses a wide range of scenarios and is highly challenging. While existing research and published datasets may not yet cover all real-world collaboration types, the methodology we propose is a general detection method with multi-class hierarchical modeling ability and task adaptability. We believe our approach has the potential to be extended to more complex scenarios.

To better align with realistic scenarios involving more than two authors, we have expanded the HART dataset to include three authors and conducted the following experiments:

We used models not seen during training, such as gemma-3-12b and deepseekv2, along with new prompts to introduce a third author for text editing in the HART dataset. We designed experiments to explore the impact of different editor sequences on the final text.

DETree is trained on RealBench and evaluated using the development set as the retrieval database. First, we investigated the influence of the first author on the final text, while keeping the second and third authors constant. The results show that when the third author is introduced, the model's ability to distinguish the first author is reduced. However, the model can still effectively differentiate between the two text types.

We also examined the influence of the middle editor on the final text. The results show a significant decline in detection accuracy when the third author is introduced.

Finally, the third editor is easily distinguishable.

The initial creator of the text sets the general direction and ideas, so these features are difficult to mask through editing. However, the middle editor's features, which are based on rephrasing the initial text, are easily masked by the third editor, leading to a significant decrease in detection accuracy.

This demonstrates that the DETree's encoder is capable of capturing fine-grained, generalizable features. Even when three authors are introduced into the test text (while the model was only exposed to two authors during training), and the third author's model, and prompt are unseen, the encoder can still extract key features to distinguish between different authors.

Weakness2&Q1: Theorem 3.1 is not stated as a theorem. The sentence "categories with closer LCA are expected to be more similar in the embedding space." is problematic because it is unclear over whay is the epxectation. The word "should" in the second line is confusing. Can you remove it? In Line 247 you refer to the "constraint defined in Theorem 3.1". What constraint? do you mean "condition"?

We acknowledge that Theorem 3.1 was not clearly stated as a theorem, and we will revise it to a more formal mathematical expression. Specifically:

If in the tree $\mathcal{T}$, for any leaf class X corresponding to node c, the following holds:

$\mathbb{E}\bigl[\mathrm{sim}(X, Y)\bigr] > \mathbb{E}\bigl[\mathrm{sim}(X, Z)\bigr], \quad \forall\, 0 \le i < j \le d_c,\; Y \in H_c^{(i)},\; Z \in H_c^{(j)}.$

Then, we can equivalently deduce that for any leaf classes X, Y, Z, the following inequality holds:

$\mathbb{E}\bigl[\mathrm{sim}(X, Y)\bigr] > \mathbb{E}\bigl[\mathrm{sim}(X, Z)\bigr], \quad \text{if } d_{\mathrm{LCA}(X, Y)} > d_{\mathrm{LCA}(X, Z)}.$

Intuitively, we aim to optimize the second set of inequalities, but it is difficult to optimize directly. Therefore, we transform it into the equivalent first set of inequalities for optimization.

We would appreciate your feedback on whether this form is appropriate, or if presenting it as a theorem may not be necessary.

In line 247, the "constraint" refers to the inequality constraint defined by Equation 2 in the original paper. We will update the wording accordingly to avoid confusion and ensure clarity.

Weakness3: In Retrieval-Based Few-Shots adaptation, it is unclear what you mean by "superficial features" and why their "ability to generalize across domains" is limited. Of course, perforamcne bottlneck due to differences between train and test cases is the fundamental challenge in learning. It is unclear how the approach taken here is different than standard training? Also, do you allow few-shots adaptation in competing methods?

We thank the reviewer for pointing this out. The term "superficial features" is indeed imprecise. We refer to features learned by supervised binary classification methods, which are optimized solely for binary discrimination and may capture spurious but separable features in the training data, leading to poor generalization in OOD settings. We performed dimensionality reduction on features extracted by MAGE (a supervised binary method) on the Beemo dataset (OOD) and observed significant distributional overlap, indicating a lack of effective separation. This is what we mean by the "limit ability to generalize across domains." We will include the corresponding visualization in the final version.

In contrast, DETree captures intrinsic affinities across arbitrary categories, guiding the model to learn the fine-grained differences and relationships between categories. For example, even when texts are generated by different models from the same prompt, or when the same model generates texts through paraphrasing, polishing, or continuation, we aim for the model to understand the inherent relationships and distinctions between them. Thus, even with a small number of support samples, our method still achieves robust source matching and classification.

In Table 2 (right), we report the performance of all models with few-shot capability under the same support set; in Table 2 (left), we report the zero-shot results of all models without such capability under the same test setting.

Weakness4: Related works: It is worth noting that the problem of deecting collaboration between authors existed before the AI era, and some methods were developed there [...]

Indeed, author collaboration detection has been an active area of research prior to the AI era. We will incorporate discussions on text stylometry and author attribution in the related works section of the final version. Our approach, however, specifically focuses on detecting a wide range of human-AI collaborative text patterns in the context of the AI generation era. While we acknowledge the interpretability advantages of stylometry methods, our approach demonstrates superior generalization and robustness, offering a valuable complement to traditional methods.

Q2 Please explain if you allow few-shot adaptation when evaluating competing methods. This is crucial to understanding whether the improvement is the result of the proposed hierarchical contrastive methodology.

In Table 1, we provide multiple sets of supervised experimental results, including training individually on each dataset and training with different priors on RealBench, which clearly demonstrates the performance improvements attributed to our hierarchical contrastive learning method.

To ensure fairness, in Table 2, for models that support few-shot adaptation, we report their performance under the same support set. For models that do not support few-shot adaptation, we also provide results under the same testing environment without few-shot adaptation.

We sincerely thank the reviewer UQj1 for the thoughtful and detailed comments. Below, we provide point-by-point responses to each of the concerns.

---

Weakness1: One concern is the manual intervention used in shaping the HAT. The authors acknowledge that purely agglomerative clustering can yield "unnatural" splits, so they employ an editable top-down reorganization using task-specific prior knowledge (Appendix B). In practice, they experimented with three hand-crafted priors (hybrid text closer to AI, closer to human, or separate) and chose the one yielding best performance. While this analysis is interesting, it does mean the final tree is partly guided by human assumptions rather than learned entirely from data.

As mentioned, the use of manual intervention in constructing the HAT is guided by task-specific prior knowledge to make slight adjustments, enhancing the model's fit for the Human-AI Collaborative Texts detection task. However, it is important to note that the tree structure is primarily driven by the similarity matrix learned in the first stage of the model. We believe that one of the key strengths of DETree lies in its ability to balance data adaptiveness with task flexibility, allowing for moderate optimization that aligns with the task while preserving the underlying hierarchical relationships from the data.

Weakness2: Two-stage training on a 16M-sample corpus plus k-NN retrieval at inference raises memory/latency concerns despite prototype compression.

During the training phase, we use a RoBERTa-large encoding model with approximately 0.5B parameters. Given the large dataset size, training can be completed in about 40 hours using 8×RTX4090 GPUs.

In the inference phase, we utilize a smaller, non-autoregressive encoding model, which is faster compared to methods that calculate text probability distributions using large language models. When we compress the database to 10k samples for k-NN retrieval classification, the computational overhead is equivalent to attaching a 10k-sized classification head (~10M parameters) after the encoder.

Binoculars and Fast_DetectGPT run inference on two RTX 4090 GPUs, while RADAR, MAGE, and DETree run inference on a single RTX 4090 GPU. We tested inference speed with 1k samples and report the average inference time per sample.

Weakness3: The paper focuses on texts generated through one-step human-AI collaboration, e.g. a human draft then one AI edit, or one AI generates then one human edit, etc. It mentions not covering scenarios with more than two "authors" due to data scarcity. This is a reasonable limitation, but it means very complex yet realistic writing processes (iterative back-and-forth edits or multiple AIs) are out of scope.

Thank you for highlighting this limitation. Due to resource constraints, we have not previously explored scenarios involving more than two "authors." However, when sufficient data is available, our method can be directly extended to handle more complex writing processes, such as iterative back-and-forth edits or collaborations involving multiple AIs. To better align with realistic scenarios involving more than two authors, we have expanded the HART dataset to include three authors and conducted the following experiments:

We used models not seen during training, such as gemma-3-12b and deepseekv2, along with new prompts to introduce a third author for text editing in the HART dataset. We designed experiments to explore the impact of different editor sequences on the final text.

DETree is trained on RealBench and evaluated using the development set as the retrieval database. First, we investigated the influence of the first author on the final text, while keeping the second and third authors constant. The results show that when the third author is introduced, the model's ability to distinguish the first author is reduced. However, the model can still effectively differentiate between the two text types.

We also examined the influence of the middle editor on the final text. The results show a significant decline in detection accuracy when the third author is introduced.

Finally, the third editor is easily distinguishable.

The initial creator of the text sets the general direction and ideas, so these features are difficult to mask through editing. However, the middle editor's features, which are based on rephrasing the initial text, are easily masked by the third editor, leading to a significant decrease in detection accuracy.

This demonstrates that the DETree's encoder is capable of capturing fine-grained, generalizable features. Even when three authors are introduced into the test text (while the model was only exposed to two authors during training), and the third author's model, and prompt are unseen, the encoder can still extract key features to distinguish between different authors.

Q1: It would be nice to show how “human-like” the automatically generated edits are. For example: measuring the stylistic difference between AI-polished text and human-polished text (perhaps using a language model evaluator or some linguistic metrics).

We supplement analysis by using language model evaluation metrics (Log-Perplexity, Log-Rank) on the HART dataset to measure the stylistic similarity between AI-polished and human-polished texts. Specifically, for the two types of texts, we use these evaluation metrics as the basis for classification. If the two types of texts cannot be separated effectively (i.e., low classification accuracy), it indicates that the distributions of the texts are highly consistent under the given metric.

HART uses the commonly used model GPT-Neo-2.7B to provide the conditional probability of each token. Log-Perplexity is calculated as follows:

$\text{Log-Perplexity}=-\frac{1}{N}\sum_{i=1}^{N}\log p(w_i\mid w_{<i})$

Log-Rank measures the position of the correct token in the predicted ranking. For each token, we record its rank $r_i$ in the probability-sorted list, then take the logarithm and compute the average:

$\text{Log‑Rank}=\frac{1}{t}\sum_{i=1}^{t}\log r_{\theta}\bigl(x_i\mid x_{<i}\bigr)$

We report the classification metric TPR@5% FPR. The detailed description of the HART dataset can be found in lines 749-761 of the paper. The differences between AI-polished and human-polished texts using these metrics are visible in Level-2 of the HART dataset.

Q2: For the most part the paper is clearly written, but a few points could use clarification. The hierarchical prior assumptions are introduced in text and Figure 7, but the main paper only briefly references “prior1/2/3” in results without deep explanation. Readers must dig into Appendix B to understand them. Also, some implementation details, such as the exact temperature used in TSCL or how the KNN search is accelerated (if at all), are not obvious in the main text. The authors could consider adding them as footnotes.

Regarding the hierarchical prior assumptions, we will provide a more detailed explanation of these in the final version to ensure that readers can better understand our assumptions. Additionally, we will include further details on the temperature (0.05) used in TSCL and the Faiss-GPU acceleration of the K-NN search, providing clear explanations as footnotes.

Q3: Lots of prior work on detection uses TPR@1% FPR as the metric, as keeping the number of false positives low is critical. Could the authors compute and report some results with this metric instead of TPR@5% FPR?

Since the HART paper uses TPR@5% FPR as the comparison metric, we adopted TPR@5% FPR for consistency in our evaluation. We have now included the performance of DETree with different priors on the HART dataset using TPR@1% FPR.

We sincerely thank the reviewer DGKn for the thoughtful and detailed comments. Below, we provide point-by-point responses to each of the concerns.

---

Weakness1: [quality] RealBench is largely synthetic (paraphrase/translation of existing corpora); the authors do not quantify annotation noise or genre imbalance;

Thank you for your valuable feedback regarding annotation noise and genre imbalance. Our annotation process is based on publicly available datasets and is programmatically generated, ensuring determinism and the absence of additional subjective bias. Therefore, it theoretically does not introduce new annotation noise. In the final version, we will include a detailed explanation of the annotation quality in the original datasets and further examine whether any noise analysis has already been conducted. Additionally, we will provide a breakdown of the genre distribution in RealBench in the appendix.

Weakness2: [originality] the approach uses well‑known building blocks (contrastive learning, agglomerative clustering, k‑NN), so novelty is incremental;

We appreciate the reviewer's comments on the originality of our approach. Our method models the hierarchical relationships between different sources in human-AI collaborative texts using the Hierarchical Affinity Tree(HAT) structure. Building on this, we introduce Tree-structured Contrastive Loss (TSCL), which innovatively applies contrastive learning to the modeling of HAT structures, aligning the embedding space with the HAT structure. This method provides a general and interpretable solution for handling scalable author categories and can be conveniently applied to other hierarchical tasks. Additionally, we propose a K-Means-based compression algorithm and a retrieval-based few-shot method, which significantly enhance both classification speed and accuracy. Our approach demonstrates state-of-the-art performance in AI text detection across in-domain, out-of-domain, and hybrid text scenarios. It offers an effective detection solution in the context of the rapid emergence of new LLMs.

Weakness3: [significance] RealBench is also used as the retrieval database at test time, so in‑distribution and some OOD scores may be optimistic.

We would like to provide some additional clarifications regarding the experimental setup:

First, in Table 1, we present multiple sets of supervised training results, including training individually on each dataset and training with different priors on RealBench.

Second, although DTree in Table 2 uses RealBench as the training set, it is tested on multiple retrieval databases, including RealBench, MAGE, M4, and RAID. The results show that even without using RealBench as the retrieval database, the models perform well in OOD environments. Additionally, we conducted few-shot experiments, reporting performance on the same support set for models that support few-shot learning, and providing results without few-shot support for other models to ensure fair comparisons.

Finally, in Table 3, we trained the model solely on RealBench and evaluated it according to the original test setup from the HART dataset, where the retrieval database was not the RealBench training set but the dev split of HART.

Q1: How many RealBench samples are verbatim or near‑duplicate across training/validation/test? Please provide a de‑duplication analysis.

Thank you for your reminder. We performed a duplication analysis on the texts in RealBench using a hash algorithm. There are no duplicate texts in the original datasets- MAGE, M4, RAID, OUTFOX, or in their corresponding augmented versions. However, in the TuringBench dataset, 663 texts are identical between the training and test sets, and 311 texts are identical between the training and validation sets. A small number of duplicates were also found in the augmented data derived from TuringBench.

Q2: Did you test DETree with an entirely unseen LLM that does not appear in RealBench?

LLaMA 3.1‑70B and GPT‑4o in Beemo, Claude and PaLM‑2 in DetectRL, GPT‑4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, LLaMA 3.3‑70B‑Instruct, and Qwen 2.5‑72B‑Instruct in HART are all models not seen in RealBench.

Q3: What is the exact setting of the Silhouette‑based subtree edit heuristic (threshold, max depth)? An ablation varying these parameters would clarify its importance.

Thank you for your insightful comment. In the construction of HAT subtrees, we control the degree of node splitting using a termination threshold. A smaller threshold results in deeper trees. To assess the impact of tree depth on model performance, we tested three different termination thresholds (0.3, 0.25, and 0.2), corresponding to maximum tree depths of 3, 6, and 9, respectively. The model was trained on RealBench and evaluated on MAGE.

The experimental results show that DETree performs well across different tree depths. Specifically, when the tree depth is shallow, fine-grained category information is lost, leading to a performance drop. As the tree depth increases, model performance improves, reaching an optimal level at a depth of 6. Further increasing the depth only results in a slight decrease in performance. The results reported in the paper correspond to the experiment with a maximum tree depth of 6.

We will include additional ablation studies in the final version to further clarify this aspect.

Q4: Could TSCL be combined with a lightweight classifier (e.g., logistic regression on frozen embeddings) to remove the dependency on k‑NN?

With TSCL training, the fine-grained features learned by DETree are capable of distinguishing between various text categories. As a result, any classifier can be used for classification, and this can be combined with our K-Means Based Database Compression method to further reduce training overhead.

While we chose to use k-NN for classification due to its ability to fit complex decision boundaries and to avoid introducing additional training overhead for the few-shot method.

The following experimental results report that DETree was trained on the MAGE training set, with the K-Means compression method used to reduce the training set to 10K samples for use with traditional classifiers, and tested on the MAGE test set.

Limitations:

The authors mention the scarcity of >2‑author collaborations and rare domains, but should also discuss: 1.the synthetic nature and potential style bias of RealBench; 2. relatively heavy GPU and memory footprintl; 3. how easy it would be for sophisticated attackers to fine‑tune their style to evade tree‑aware detectors (in the context of the detection-generation arms race).

As for the synthetic nature and potential style bias of RealBench, we employed different paraphrasing models and prompts during both the training and testing phases of RealBench. Additionally, we introduced multiple external datasets with significant differences in style and generation methods (such as MAGE, M4, DetectRL, Beemo, HART, etc.) during evaluation to ensure diversity and generalization.

During the training phase, we use a RoBERTa-large encoding model with approximately 0.5B parameters. Given the large dataset size, training can be completed in about 40 hours using 8×RTX4090 GPUs.

In the inference phase, when we compress the database to 10k samples for k-NN retrieval classification, the computational overhead is equivalent to attaching a 10k-sized classification head (approximately 10M parameters) after the encoder, with minimal memory cost.

Regarding the concern about sophisticated attackers fine-tuning their style to evade tree-aware detectors, this adversarial issue has not been addressed in the current work. We acknowledge the importance of this challenge and will discuss it in the limitations and future work sections of the final version.

Paper Formatting Concerns:

The text on figures 3, 10, 12-15 is tiny to the point of unreadability (especially in the Appendix). Consider breaking them up into more figures to increase the font size. Minor issues with text consistency, e.g,. capiltalized "in Equation 3" on l.174 but immediately "in equation 4" on l.175.

Thank you for the valuable comments on formatting. We will address the above issues in the final version.

We sincerely thank the reviewer U1pr for the thoughtful and detailed comments. Below, we provide point-by-point responses to each of the concerns.

---

Weakness:Certain feature definitions, especially around how local and global perplexity are computed and normalized, could be further clarified clearer mathematical notation or concrete examples.

Thank you for pointing out the lack of clarity in the expression. You mentioned that "local and global perplexity are computed and normalized, and could be further clarified with clearer mathematical notation or concrete examples." However, we did not use the terms "local and global perplexity" in the paper. To avoid any misunderstandings, we would appreciate further clarification on which specific part you are referring to, so we can make the necessary improvements and additions.

Q1: how does DETree behave when model-generated text has been post-edited by humans or blends human and AI content (i.e., in mixed authorship settings)?

In mixed authorship settings, we investigate the behavior of DETree in the Hybrid Text Detection section of the paper, based on the HART dataset. HART categorizes samples based on the extent of AI involvement in text generation into four types: human-written (Type 0), human content rephrased by AI (Type 1), AI generated content humanized through various methods (including LLM paraphrasing, human editing, and commercial tools) (Type 2), and fully AI-generated content (Type 3). Based on this labeling scheme, HART defines three detection task levels: Level 1 detects AI involvement (Type 0 vs. Type 1/2/3), Level 2 detects whether the core content is AI-generated (Type 0/1 vs. Type 2/3), and Level 3 detects whether the text is fully generated by AI (Type 0/1/2 vs. Type 3).

DETree is trained on RealBench and evaluated strictly according to the testing setup of the HART dataset. As shown in Table 3 of the paper, the model performs significantly better than the results reported in the original paper under this setup.

To better align with realistic scenarios involving more than two authors, we have expanded the HART dataset to include three authors and conducted the following experiments:

We used models not seen during training, such as gemma-3-12b and deepseekv2, along with new prompts to introduce a third author for text editing in the HART dataset. We designed experiments to explore the impact of different editor sequences on the final text.

DETree is trained on RealBench and evaluated using the development set as the retrieval database. First, we investigated the influence of the first author on the final text, while keeping the second and third authors constant. The results show that when the third author is introduced, the model's ability to distinguish the first author is reduced. However, the model can still effectively differentiate between the two text types.

Finally, the third editor is easily distinguishable.

The initial creator of the text sets the general direction and ideas, so these features are difficult to mask through editing. However, the middle editor's features, which are based on rephrasing the initial text, are easily masked by the third editor, leading to a significant decrease in detection accuracy.

This demonstrates that the DETree's encoder is capable of capturing fine-grained, generalizable features. Even when three authors are introduced into the test text (while the model was only exposed to two authors during training), and the third author's model, and prompt are unseen, the encoder can still extract key features to distinguish between different authors.

Q2: How much does each category of features—local perplexity, global perplexity, embedding similarity—contribute to the model’s performance?

In our approach, embedding similarity plays a central role in computing the class similarity matrix, constructing the HAT structure, and performing the final retrieval classification. As for local perplexity and global perplexity, these features were not used in the paper.

Q3: could you compare DETree to other lightweight or interpretable models such as logistic regression, rule-based classifiers, or shallow decision forests?

In our experiments, we compared DETree with rule-based classifiers designed using perplexity and its variants, such as LRR, Glimpse, Fast-Detect, and Binoculars. These classifiers are lightweight and interpretable. As shown in Tables 1, 2, and 3, DETree outperforms these models in both in-domain, out-of-domain, and hybrid text test environments.

Q4: How do you balance the depth and complexity of the decision tree against interpretability? Does a deeper tree improve performance substantially, or introduce interpretability costs?

Thank you for your insightful comment. In the construction of HAT subtrees, we control the degree of node splitting using a termination threshold. A smaller threshold results in deeper trees. To assess the impact of tree depth on model performance, we tested three different termination thresholds (0.3, 0.25, and 0.2), corresponding to maximum tree depths of 3, 6, and 9, respectively. The model was trained on RealBench and evaluated on MAGE.

The experimental results show that DETree performs well across different tree depths. Specifically, when the tree depth is shallow, fine-grained category information is lost, leading to a performance drop. As the tree depth increases, model performance improves, reaching an optimal level at a depth of 6. Further increasing the depth only results in a slight decrease in performance. The results reported in the paper correspond to the experiment with a maximum tree depth of 6.

We will include additional ablation studies in the final version to further clarify this aspect.