MOSAIC: Multiple Observers Spotting AI Content, a Robust Approach to Machine-Generated Text Detection

Thank you for your time and remarks. You pointed out fair issues that we are currently working on. Here are our answers to your questions.

The computational cost of running multiple language models for detection is significant but not thoroughly addressed. [...]

The goal of the paper is to introduce a new theory-grounded method for combining models and how it helps in detecting machine-generated texts. It is currently on the slow side, needing about 10 seconds per document. Runtime optimization is planned for future work. Here is a quick description of our current system’s shortcomings and how it could be improved : We run the texts one by one with our LLMs. The models are loaded onto different GPUs, so we move the logits onto the same device to perform the Blahut-Arimoto, perplexity and cross-entropy operations, and then return our score. In this current setup, there is a lot of room for improvement. For instance, moving the logits to the same device creates a bottleneck. Furthermore, while the calculations are performed on one GPU, the other ones are inactive. A better way to do it would be to calculate all the logits for all texts, store them on different GPUs, then perform our calculations in parallel. Even better would be to have all the models loaded onto the same GPU (using quantized or distilled versions), so moving the logits would be unnecessary. These are ideas we considered but have not implemented here because it is not the focus of our work.

The method struggles with detecting texts from fine-tuned models, [...]. The authors could explore incorporating domain-adapted models into the ensemble to address this limitation.

Including domain-adapted models is a very appealing idea, that we could readily integrate in our framework as they would share a common tokenizer. An in-depth investigation of this issue is left for future work. We believe however that this issue is partly addressed when looking at how having the Tower models in our ensemble improves our performances on foreign languages (over Binoculars, using Falcons). Same for MOSAIC-5, adding Phi-3-4k-Instruct improves our detection AUC on the ChatGPT datasets (Reddit, Reuter and Essay), another instruct model.

The paper's treatment of sampling methods' impact on detection is limited[...].

That is not the goal of our paper, such an exploration can be found in the RAID paper https://aclanthology.org/2024.acl-long.674, we have the results of our method on the whole test set of RAID. We will add a table in the next version of this paper giving more information about how adding sampling and repetition penalty impact our performance.

The choice of baseline models in the ensemble appears somewhat arbitrary. The authors could provide clearer criteria for selecting which models to include, especially since they show that adding weak models (like unigram) can be detrimental to performance.

The choice is indeed arbitrary, as our goal is mostly to study the combination method. An ideal combination would cover many languages and domains, and adding fine-tuned models would be a great idea in that regard. However, that was not the goal here nor do we have the resources to run a completely thorough study.

Have you explored methods to reduce the computational overhead of running multiple models, such as model distillation or selective invocation of ensemble members based on confidence thresholds?

We have not explored such methods in this work. Model distillation is a current project of ours. A current bottleneck of our method is that we need all our logits on the same device to perform our operations (Blahut-Arimoto, perplexity and cross-entropy), moving them all represents an important time loss. Model distillation (and/or quantization) would reduce the size of our models, allowing us to load them onto the same GPU, completely eliminating the need to move the logits.

Thank you for your positive review, we have provided further explanations on the points you have raised and are able to answer more if need be.

Questions: Lines 54-64, could you please define (or further explain) the term 'robustness' in a machine-generated text detection problem? “Robustness” refers to the ability to detect machine-generated text across various languages, domains and generators. In Figure 1, the caption should provide an explanation of the pipeline's operation. It needs to define μ and describe how the Logits are transformed into features of the mixture model.

The description of the pipeline is provided line 206 along with Proposition 1, should we refer to it directly in the caption ?

Line 95, does the circle operator o in the definition of the output space y represent a concatenating operation?

Yes, it represents token concatenation.

Section 3.1 mentions that the experiments involve some imbalanced datasets. Could you please discuss whether these imbalances are tolerable, or specified rebalance techniques are required?

Imbalanced datasets refers to datasets containing more generated than human texts. It can be an issue depending on the metric used, it is not really an issue in our case as there is no training hence no overfitting. It is important to specify what a “True positive” is (usually a machine-generated text detected as such) in the case of TPR@5%FPR though, as this score is affected by imbalances, unlike AUC.

Section 4.1 mentions the use of Blahut-Arimoto weights. Briefly introducing how are these weights calculated, and what specific role they play in adjusting the contributions of individual models within the ensemble might be helpful for readers.

The role of the Blahut-Arimoto weights $\mu$ is defined in our Proposition 1 page 206. As we simply use the Blahut-Arimoto algorithm, a full explanation is given in the Section B of the Appendix.

Further discussion is needed on how the choice of tokenization and preprocessing methods affects detection accuracy. Can the proposed method be adapted to incorporate other detection models? Are there any limitations on the types of models that can be integrated?

The choice of tokenization is dictated by the models we use, as they must share the same tokenization of our documents in order to operate at the token-level. There is no other preprocessing done to the texts. Our combination method can be applied to any arbitrary number of models as long as they share a tokenizer. This is the only limitation there is concerning integration and something we are working on.

Thank you for your insight and remarks, you point out the need for better explanations as to our experiment settings in the paper, we have answered below and will incorporate this feedback in the revisions.

My first main criticism for the paper is that I am slightly incredulous of the results and the ensemble used. I feel like for a fair comparison, the authors should use the same suite of models as used in the baseline models. [...] My second main criticism is that the improvement of MOSAIC over other methods is not so salient. [...] Aside from M4, MOSAIC-4 doesn’t do better than the baselines on RAID and Ghostbusters. A minor point related to this: consider merging table 2 and 3? It would be easier for readers to compare the results.

As briefly mentioned in Section 4.3, MOSAIC needs an ensemble of models that use the same tokenizer. This is why we separated tables 2 and 3. Table 2 uses the default models proposed by each method (GPT2 for DetectGPT and FastDetectGPT, Falcon-7b and its instruct version for Binoculars), whereas Table 3 displays results using only the 4 models in our ensemble. In this table, Best single-model refers to FastDetectGPT using Tower-13b (the best 1-model solution out of the 12 mentioned in lines 363-367). Best two-model refers to Binoculars method using TowerBase-7b as the detector, and Llama-2-7b as the auxiliary model, selected out of the 12 possible permutations of 2 models out of 4. The choice of the best combination was made on the CC_news subset (l 368). We feel that this was a fair way of comparing methods without having to deal also with the difference in underlying models. Instead of running MOSAIC with the other models, we ran all other methods (DetectGPT, FastDetectGPT, Binocular, MOSAIC) with a fixed set of models.

My third main criticism is that this method lacks testing against adversarial attacks. [...]

We ran both MOSAIC-4 and MOSAIC-5 (with Phi-3-4k-instruct) on the whole RAID test set, which includes various types of adversarial attacks. Attacks that make the text differ a lot from the model’s probability distribution are very effective, especially synonyms replacement as they make generated texts more surprising while not having the same effect on human texts (that’s the idea behind DetectGPT). On average over all perturbations, we lose about 5 points on TPR@5%FPR on the RAID test dataset (No attacks gives 0.752 and 0.745 for 4 and 5). We will include these findings in the revised version of the paper. Coupled with a detailed analysis of the results depending on each adversarial attack in the appendix.

I don’t necessarily agree with the statement in the abstract: “Most approaches evaluate an input document by a well-chosen detector LLM, assuming that low-perplexity scores reliably signal machine-made content.”

In this statement, we most wanted to refer to unsupervised detection methods. We will make this more precise in the final version. Other, non PPL based techniques are discussed in the related work. Watermarking is different as in this case texts are generated with the intent of being discovered, as we also briefly mention it in our Related Work.

What is the run-time of this algorithm compared to the baselines?

Our algorithm processes each text in 10 seconds on V100 GPUs. Runtime optimization is planned for future work. Here is a quick description of our current system’s shortcomings and how it could be improved : We run the texts one by one with our LLMs. The models are loaded onto different GPUs, so we move the logits onto the same device to perform the Blahut-Arimoto, perplexity and cross-entropy operations, and then return our score. In this current setup, there is a lot of room for improvement. For instance, moving the logits to the same device creates a bottleneck. Furthermore, while the calculations are performed on one GPU, the other ones are inactive. A better way to do it would be to calculate all the logits for all texts, store them on different GPUs, then perform our calculations in parallel. Even better would be to have all the models loaded onto the same GPU (using quantized or distilled versions), so moving the logits would be unnecessary. These are ideas we considered but have not implemented here because it is not the focus of our work.

Dear authors,

Thank you for your clarification. I think ensemble is cool, but it has to serve a useful purpose. If your main claim is that this method helps you search for the optimal detectors, then I think you are saying that certain detectors may not work well on certain domains, and instead of bothering to find optimal detectors for each domain, MOSAIC can automatically look for optimal detectors.

However, to what extent is this a valid assumption? It seems that the original binocular is already pretty decent across various domains, unless you can provide additional robustness, I don't see great values although this is a cool ensemble you are doing.

And additionally, in your abstract, you claimed that your method increased the robustness (L21-23), so that's why I took it as a method paper and looked for evidences that your method is significantly stronger than the baselines. If you want to reclaim that the main contribution is to provide optimization, then you need to do significant rewriting of the paper.

Regarding the performance drop, I was referring to the drop on Ghostbusters. Before it was 0.993 0.996 0.990 (L350), and after using your method it becomes 0.677 0.663 0.481 (L383). It is arguably caused by the change in base models, but it is also unclear whether the improvement on M4 is also caused by the multilinguality of Tower.

I hope this clarifies. I am always happy to discuss more!

Thank you for your comments, you are highlighting the fact that our work needs further clarifications, we have tried to address all your concerns down below and are open to discussion. In particular, we would like to emphasize that our results are very consistent with those of Hans et al (2024), as we discuss in detail below.

LN058: Why is there a requirement for time-varying mixture models? Shouldn't the language models used remain consistent regardless >of the text source?

For each token, the context varies and so does the probability distribution of each of our models. This is why our proposal studies a combination mechanism that is able to adapt its weights at each time-step. The benefits of this extra degree of freedom are assessed experimentally.

LN059: Can you provide an intuitive explanation as to why we need to minimize the worst-case encoding size?

Large Language Models are trained to minimize perplexity, which, when viewed through the lens of information theory, corresponds to reducing encoding size. Indeed, the negative log of a probability corresponds to the encoding length because of the connection to information theory. Specifically, the optimal code length for a symbol (in bits) is −log_⁡2(p), where p is the symbol's probability. This is derived from Shannon's entropy, which quantifies the average information content. A less likely event (lower p) requires more bits to encode, as it carries more information. Our hypothesis is that focusing on minimizing worst-case perplexity (averaged across a set of known LLMs) could enhance the robustness of detection. This approach is expected to reduce the perplexities of machine-generated samples more significantly than those of human-written texts.

Figure 1: Does \miu represent the probability assigned to each token?

$\mu$ represents the weights assigned to each model’s distribution. As can be seen in proposition 1, our optimal distribution $q^{\star}$ is a context-dependent mixture (weighted sum) of each detector model probability distributions.

LN332: The statement might not be accurate. In my understanding, FastDetectGPT and DetectGPT, despite their similar names, differ substantially in their principles and performance. [...]

You are right, we could be more precise. FastDetectGPT does not generate tokens but uses the model’s logits to compute changes in log-probability. The detection method relies on the conditional probability curvature, a variation of the probability curvature defined in the DetectGPT paper.

Table 2: The results significantly differ from the findings presented in Figure 3 of the Binoculars report (Hans et al., 2024)[...].

Figure 3 of the Binoculars report displays True Positive Rate versus False Positive Rate on a log-scale. We instead, report the AUC, that is the area under the TPR versus FPR curve taken on a linear scale, which may have caused the confusion you are referring to. Furthermore, the test datasets are not exactly similar as we used all of Binoculars data, including both Llama-2-13b and Falcon-7b as generators whereas Figure 3 in (Hans et al, 2024) seems to only use the Llama-2-13b generated data. Nevertheless, our results still remain highly consistent with (Hans et al, 2024) as we also report detection scores close to 99.5 for the Binocular method on the Binocular dataset. The other results are not completely identical, as the underlying models behind the other baselines differ: we used the default detector models in the github code of DetectGPT and FastDetectGPT (GPT2), whereas the Binoculars paper used Llama-2-13b for DetectGPT and GPT-Neo with GPT-J for FastDetectGPT. All the results reported Table 3 use the same set of models for a fair comparison, and do not directly compare with (Hans et al, 2024) who use a different pair of detectors.

Table 3: MOSAIC-4 (avg) only outperforms in 1 out of 12 datasets, and MOSAIC-4 (unif) only outperforms in 3 out of 12 datasets. Does this suggest that the method achieves robustness at the cost of detection accuracy?

We do not claim that our method is the best performing one in all cases but that is is effective on average (Table 3), i.e. robust to changes in the generator model. Our goal is to provide a robust technique that dispenses with the need to search for the best detector model(s).

Table 4: MOSAIC-4 (avg) shows good detection accuracies after incorporating the generator LLM. Does this imply that the method still has issues with brittleness and requires revision when new generators become available?

In Table 4, our results on the original Binoculars dataset are actually better than when we regenerate the same texts, using the same protocol but with a Llama-2 model. Thus having the generator model itself does not appear particularly advantageous. In fact, the availability of new generator models does not change anything for our method, which can seamlessly accommodate a growing number of detectors (assuming they share a same tokenizer).

Thank you for your review, and the references you provided as we did not know of these papers. We tried to address your concerns and answer your questions below.

Weaknesses: the paper itself is not self-contained as the algorithm is not properly explained and one needs to refer to Appendix B for those unfamiliar with it (most of the readers?)

Blahut-Arimoto algorithm is not a contribution of this paper, but rather a mere tool that we use to solve the optimization in eq. (4). Indeed, we simply applied it to a family of models to obtain $q^{\star}$, the probability distribution maximising the mutual information between our model’s logits.

the presentation of the final results is weak (3 tables) and one needs to navigate across tables in different pages (Table 2 & 3) to see the difference. A graph would be more informative

We separated Tables 2 and 3 because the models used by the methods are not the same. In Table 2 we report the baseline results using the default models used in the original papers. In Table 3 we selected the best result of each baseline using the 4 models in our ensemble, ie the best Fast-DetectGPT (out of 4 possibilities) and the best Binoculars combination (out of all the permutations of 2 out of 4 choices, so $P(4, 2) = 12$). We judged it was necessary to separate these tables as having one big table would introduce confusion between methods and the detector models used. Table 4 displays different results on different data so adding it would be hard as well. Would having Tables 2 and 3 on the same page make it easier to understand ? If so we can do that in the final version. Since many results are very close (up to the 3rd decimal), such differences would be hard to spot if we used graphs.

while hard to interpret (see 2) ), it seems that the proposed method often underperforms existing methods, while being computationally more expensive (??). The interpretability features of looking into the respective weights, as well as the robustness is still compelling, but if this is not reflected in the final performance it would be important to explain

As noted in the paper (line 375), we do not claim the superiority of our approach for each dataset, but its reliability on average. Our ensemble method eliminates the need to optimise the choice of detector model. For example, using the default models (Falcon-7b and its instruct version) with Binoculars leads to average results (0.686 AUC) on the Arabic M4 data, while the “best two-model” combination of Binoculars (Tower-7b and Llama-2-7b) greatly improves this result (0.897 AUC). Such hyperparameter search is unnecessary in our case and has been the motivation behind our research.

Questions: one would expect that that more iterations for Blahut–Arimoto, the higher the final performance. Is this the case? A graph with performance over number of iterations would be very informative

We use Blahut-Arimoto in order to obtain the $\mu$ coefficients deciding $q^{\star}$ our optimal distribution. As the problem is concave, the algorithm rapidly converges to the optimal solution, our stopping condition is either 1000 iterations (rarely happens) or when the weights improve by less than 1e-6 after an iteration. A difference of 1e-6 in weights barely impacts performance.

most of the numbers seem very close. Do the algorithms all detect the same documents? Your algorithm in particular is different than the others as it relies on a panel of judges, could you provide some error analysis?

This is a good idea that we will look into. As all methods use a variation of perplexity we would expect the texts detected to be roughly the same but this deserves further exploration. When looking into this issue, we decided upon a threshold using the same methodology as Binoculars and computed confusion matrices for some datasets. There are only a few texts where the thresholds do not agree, indicating that these methods mostly identify the same texts. For our RAID extract :

For Binoculars Pubmed dataset :

For Binoculars CNN dataset :

For Binoculars CC_News dataset :

We would like to thank the reviewers for their time and responses. We have now posted a revised version of the paper providing clarifications as well as additional information and experimental results as was requested. Notably, we have added a new section related to computational complexity, and results concerning (a) robustness with respect to adversarial attacks; (b) robustness with respect to changes of generation parameters. All these are in the appendix. Changes are highlighted in red in the revised text. We hope these modifications will help you better appreciate this work and its implications.