Humans vs ChatGPT: Uncovering the Non-trivial Distinctions by Evaluating Parallel Responses

The overall study seems to be just a mix-and-match of whatever tool is available for the author/s that will be able to give some form of analysis without proposing any novel approach to shed new insights in the growing field of discriminating between machine and human-generated texts. In addition, most of the analysis is weak or unmotivated. (For example, what are non-trivial features? Why is there a need for analysis of stopwords use? In what way does the use of stopwords shed light on human and machine-generated text?) Moreover, there is no comparison of existing works in literature including already established classification systems such as DetectGPT, GPT-Zero, etc with the trained models from the paper. These limitations in the paper severely affect the quality of the work and its inclusion for ICLR.

Another demerit of the paper is the lack of concrete discussion of the data used. The author/s claim that the data is novel but provides very limited information on how it was collected particularly on important details such as language, statistical characteristic (how long, how many words), context of where the comments or questions are about, genre or domain, etc. Moreover, the author/s seem to use the collected data for querying ChatGPT but other prompt-ready QA datasets have already been constructed. Why did the authors not use or combine this instead?

Improvement of how the motivation of the paper is pieced together, especially in the Introduction section, should be strictly addressed to increase readability and in order for readers to appreciate the purpose of the study (see comments on structuring below).

Minor comments on structuring and flow of discussion that need to be addressed:

The introduction section needs to be improved as it does not sound technical/scientific. Vague and overgeneralized claims with no basis should be removed particularly in the first sentence “Large Language Models (LLMs) have been propping up ever since OpenAI revealed ChatGPT”. LLMs have already been an active research even before ChatGPT. The sentence “ChatGPT-generated content has made its way into all aspects of life.” is also vague and too highfaluting. The introduction of ChatGPT had some impact, yes, but you need to ground this technical with figures instead and not with these types of statements.

There is no ChatGPT-4. Do you mean GPT-4? ChatGPT was packaged on its own using the GPT-3.5 Turbo model while GPT-4 is a separate model that was only integrated into the ChatGPT system as another option.

1. My main concern is that the paper does not provide in-depth and significant features or differences between human-generated text and ChatGPT-generated text. This leads readers to find it difficult to obtain sufficient information from the paper.

2. Some findings are not rigorous. For example, although the author asserts the differences on the words based on Figure 6, using only two examples and a lesser-used metric "WORD NEIGHBORHOODS" is not sufficient to adequately demonstrate this point.

3. The experiment is done on various metrics, unveiling whether human and ChatGPT texts are significantly different on the corresponding factors. However, the authors do not delve deeper to explain why these kinds of similarity or difference happen.

4. In section 4.1, the author classify the documents with machine learning models. The usage of BERT is not explained clearly enough. Does BERT embeddings are simply used for classification without other machine learning models? If that, which kind of classifier is used?

• The motivation of this study is unclear. Even texts generated by two humans (depending on their geography/ dialects/ etc.) would have some differences and similarities. Having a clearly defined objective can help better understand both the design of experiments and the significance of results. For instance, if the goal was to evaluate whether an LLM can answer like human for stack overflow questions (or any specific domain), performing these analyses on fine-tuned LLM (or few-shot/ incontext examples) can give more useful insights.

• the paper needs significant scientific rigor in the analysis to highlight the similarities and differences in how humans vs SoTA LLMs generate text. There a lot of similar work out there (even the two comparative studies the authors point out have similar analysis and rigorous insights on experiments and results)
• It is not clear if Fig 1 uses single or combined human and chatgpt corpus categories. The figure does not add value to the main content of the paper (could perhaps go in appendix).
• It is interesting to see that YA-D3 vs YA-H have the least in common in terms of semantic textual similarity (Fig 4) - this could have been an interesting topic to investigate further, do some human evaluations, etc.
• Fig 5 uses T-SNE projections; some experiments suggest that using UMAP visualization might show better projections - also, there are many advanced methods (such as Deep Aligned Clustering, for example) that can be used to further cluster the semantic embeddings to get a better understanding of topics and overlap between the two datasets.
• Can’t draw much insights from Fig 6 and it needs more explanation (what are the colors here?)