Text2NKG: Fine-Grained N-ary Relation Extraction for N-ary relational Knowledge Graph Construction

Weaknesses:

1. While the extension of the HyperRED dataset is a valuable contribution, the paper could benefit from additional experimentation on a wider range of datasets, especially in diverse domains. This would further validate the framework's performance and applicability across various knowledge graph construction scenarios.

2. The paper does not provide a detailed discussion for the computational resources required to implement Text2NKG. This information is crucial for practitioners and researchers to understand the feasibility and potential resource constraints associated with adopting the framework.

3. The paper does not extensively discuss potential overfitting issues. Since fine-grained relation extraction involves complex pattern recognition, it is essential to address how Text2NKG generalizes to new, unseen data, especially in real-world, noisy text environments.

4. While Text2NKG demonstrates substantial improvements over the previous state-of-the-art model, CubeRE, a more comprehensive comparison with a wider range of alternative approaches would provide a more complete assessment of its relative strengths and weaknesses.

5. The paper does not explicitly address potential biases that may arise during the relation extraction process. Natural language data can contain biases that may be inadvertently learned by the model, which could impact the quality of the extracted relations.

6. The technical details are not described clearly and somehow seems lack of novelty.

Overall, the workload of this paper is not particularly large, and the complexity is not that high. The experiments and diagrams mentioned in the article are not so sufficient. More experimental materials can be provided to form a strong support for the experimental results. The article lacks discussion on the limitations of the model and could briefly describe its potential defects and areas for future improvement. An overview of Text2NKG described in Figure 3 should be a very important diagram for the article, but the architecture of the entire model seems to be not regular enough, and there are not many annotations in the diagram, such as solid tokens "[S] and [/S]" for entity delimitation and annotations for levitated markers "[L] and [/L]". In METHODOLOGY, there is a lack of detailed description of the working principle of the linear classifier FNN. In addition, the introduction in the OUTPUT MERGING part is too general. It is best to describe the novel method proposed in this article in detail so that readers can better understand it. In the ablation study, three important components are suddenly mentioned, including data augmentation (DA), null-label weight hyperparameter (α), and hetero-ordered merging (HM). However, the previous METHODOLOGY chapter does not seem to emphasize the importance of the first two components. The HyperRED dataset is the only existing dataset for extracting nary relations with annotated extracted entities. How can the generalization ability of the model be demonstrated? The structure in INTRODUCTION is slightly confusing, such as whether the second paragraph can be considered in the following chapter, and then whether the third paragraph should appear in related work. Some of the content in the article is duplicated, resulting in the compression of the substantive content. For example, in RELATED WORK "An n-ary relational knowledge graph (NKG) consists of n-ary relational facts, which contain n entities (n ≥ 2)...... because we cannot distinguish which binary relations are combined to represent the n-ary relational fact in the whole KG. "is repeated in the INTRODUCTION, and some places can be pointed out briefly. There are some minor errors in the grammar and format of the paper, such as "course-grained" should be "coarse-grained", and "imput" in "where t = | X | is the imput token length". In addition, it is best to add parentheses to the citations in the text, otherwise the content will look confusing and difficult to read.

1. The paper is not organized clearly, which is not friendly for understanding. Specifically, there are lack the more details and essential discussion about extracting 3-ary facts as an atomic unit.
2. The algorithms in this paper are not explained in detail, which may cause the fusion for readers. Moreover, some variable symbols in algorithms are not interpreted such as the TRAINING STRATEGY.

1. I believe it should be coarse-grained?
2. The quality of the writing in this paper leaves room for improvement. Normally, the introduction should briefly outline the motivation behind the research and explain the methodology. However, this paper's introduction seems to be excessively brief when it comes to motivation and methodology. Additionally, in related work section, the phrase "we propose..." has been reiterated twice. This excessive emphasis on their framework's function seems pointlessly repetitive.
3. Noticing only two referenced works from 2023 (ChatGPT and GPT-4) raises questions regarding the absence of related papers from the same year. One could question if no other papers published in 2023 bear relevance to their work.
4. The range of experiments conducted appears to be limited, with insufficient diversity in datasets and methods being compared.
5. The paper seems to have a lackluster impact on the specific field it targets.

1. Technical novelty is limited. The model's approach is rather conventional, involving the extraction of 3-ary facts through existing methods and their subsequent merging to create higher-order models.
2. The paper's representation of the methodology falls short. There are notable deficiencies in explaining crucial technical aspects. E.g., the process of merging 3-ary facts to form higher-order facts lacks clarity.
3. The experimental evaluation is not sufficient. The study only utilizes a single dataset, HyperRED, and does not include comparisons with diverse datasets such as JF17K, FB-AUTO, and M-FB15K.