Reflection on Knowledge Graph for Large Language Models Reasoning

Thank you for your valuable comments. We will explain your concerns point by point.

Comment 1:

What are the reflection capabilities explicitly referred to in lines 72 to 73?

Reply:

Reflection capabilities specifically refer to the comprehensive ability of large language models to handle complex knowledge scenarios by leveraging multi-task collaboration for deep analysis and decoupling of input content, semantic understanding, noise detection, and implicit reasoning. We approach this from two perspectives.

Multi-task Collaborative Capabilities: In our framework of Decoupling-Exploration-Refinement-Reconstruction-Inference, reflection capabilities are demonstrated through the dynamic collaboration and deep reasoning of the model across various tasks.

• In the Query Decoupling phase, the LLM breaks down multi-dimensional complex questions into single-hop atomic problems, reducing problem complexity and improving the precision of knowledge matching.
• In the Subgraph Retrieval phase, the LLM leverages its reasoning capabilities to search for knowledge in the knowledge graph relevant to each subquery.
• In the Knowledge Refinement phase, the model filters and assigns weights to evidence, identifying noisy information and prioritizing knowledge that better supports the query.
• In the Knowledge Reconstruction phase, the model leverages implicit reasoning to reorganize triples into natural language information that better aligns with the context. This collaborative mechanism across tasks significantly enhances the overall reasoning depth and robustness of the model.

Deep Knowledge Reasoning Ability: Through our knowledge-driven multi-task instruction fine-tuning method, the model goes beyond shallow understanding and generation of input content, acquiring reflection capabilities based on knowledge graphs. Specifically, after training, the model can deeply evaluate the plausibility of information, actively identify and correct potential erroneous knowledge. This capability surpasses the traditional generative mode of LLMs, enabling the model to perform deep reasoning in complex knowledge scenarios.

Comment 2:

How is the corresponding entity subset Esub collected as mentioned in Section 3.1? How to make sure the decoupled entities can be grounded to the corresponding KGs? If the subsets are provided in the dataset like FactKG, how is that method been adapted to dataset like WebQSP?

Reply:

For the FactKG dataset, since the entity set is already provided, we leverage the entity set during the "Query Decoupling" stage to assist the LLM in efficiently performing question decoupling. In contrast, the WebQSP dataset has lower question complexity and fewer related entities compared to FactKG.

Therefore, we designate the topic entity as the sole member of the entity set, serving as the starting point for the first sub-query to train the LLM's ability to predict the number of hops. Testing results show that the LLM achieves a hop number prediction accuracy of exceeding 97%, highlighting its high effectiveness in this task.

Thank you for your valuable comments. We will explain your concerns point by point.

Comment 1：

The broad adaptability of RefKG in other knowledge-intensive tasks or domains.

Reply:

Our method, RefKG, focuses on question answering and fact verification, unifying these two tasks within a single modeling framework. This dedicated focus allows the framework to achieve optimal performance in these core tasks. However, other knowledge-intensive tasks, such as information extraction, knowledge graph construction, and knowledge graph completion, may require different optimization directions or modules, which are beyond the scope of our current research.

We conducted experiments on both general-purpose and domain-specific datasets. FactKG and WebQSP utilize DBpedia and Wikidata, two large-scale knowledge graphs with extremely broad knowledge coverage, demonstrating RefKG's adaptability to complex question-answering tasks across multiple domains. Meanwhile, MetaQA, based on the MovieQA knowledge graph in the movie knowledge domain, further validates RefKG's exceptional performance in domain-specific tasks.

We have summarized the knowledge graphs used in the three benchmark datasets, the corresponding knowledge domains, the number of triples required per query, the hop numbers of the queries, and the best performance of our method on these datasets, as detailed in the table below:

It is worth emphasizing that our method incorporates knowledge-driven multi-task training, focusing on enhancing the model's ability to retrieve, refine, and apply knowledge, rather than limiting it to a specific knowledge domain. The capabilities learned by the model not only demonstrate broad generalization across various general-purpose domains but also support the plug-and-play integration of domain-specific knowledge graphs, enabling efficient performance on specialized tasks.

Comment 2：

What steps were taken to identify and mitigate potential biases in the LLM-generated corpus used for multi-task tuning?

Reply:

We have developed specific evaluation and filtering methods to monitor the quality of the generated corpus, as described in Section 3.4.1, "Quality Control".

For the "Query Decoupling" task, let $E$ represent the entity set of the original sentence, and $E_{div,i}$ denote the entity set for each sub-query after decoupling. The criteria for considering the decoupling results as high-quality are as follows:

(a) $E_{div} \neq \emptyset$.

(b) $E = \bigcup_{i=1}^{H} e_{div,i}$.

(c) If $|E_{\text{div}}| > 1$, then $\forall e_{div,i} \in E_{\text{div}}, e_{div,i} \subsetneqq E$. If $|E_{\text{div}}| = 1$, then $E_{\text{div}} = E$.

For the knowledge reconstruction task, let $E$ represent the set of all entities in the evidence triples and $R$ represent the set of all relations in the evidence triples. If the reconstruction results fully contain $E$ and $R$, completeness is considered ensured. Additionally, by jointly reasoning with the textual evidence and the query, if the correct answer is obtained, correctness is considered ensured. Data that satisfies both completeness and correctness is regarded as high-quality.

Additionally, the corpus we generate focuses on training models to utilize knowledge appropriately and does not involve content related to safety, ethics, or politics that may carry potential biases.

We greatly appreciate you taking the time to review our work and provide valuable feedback. As the discussion phase between the authors and reviewers draws to a close, we would like to take this opportunity to further clarify our responses.

In the latest version of the PDF, we have made several modifications and additions, including:

• Corrected tyops and unclear expressions;
• Added the quantification and analysis of noise in the overall process, further clarifying the impact of noise on model performance and providing a detailed explanation of the effectiveness of our noise control strategy;
• Added an additional comparative experiment, which thoroughly analyzes the rationale behind our method design and training effectiveness, further validating the superiority of the method in complex tasks;
• Introduced a quantitative analysis of inference time, demonstrating the efficiency of the model in large-scale applications.

It is worth emphasizing that current approaches often introduce additional noise in the pipeline process of knowledge retrieval and reasoning, leading to the accumulation of errors, impeding LLMs from effectively combining the external knowledge in answering complex multi-hop questions. To this end, our method is specifically designed to enhance the reasoning capabilities of LLMs through reflective engagement with knowledge graphs, while effectively controlling the noise. We believe this method has tremendous potential to advance the field. Our main contributions are:

• We introduce a knowledge-driven multi-task instruction fine-tuning method, which enables the model to effectively complete the full process of decoupling, exploration, refinement, reconstruction, and reasoning within a knowledge graph through multi-task collaboration. Multi-task fine-tuning allows the model to share learned features and representations across different tasks.

• We trained an expert scoring model based on LLM. This module can identify and filter out triples that do not support question answering, effectively controlling the introduction of noise. This significantly enhances the accuracy of reasoning tasks across various knowledge scenarios. Additionally, this module improves the interpretability and efficiency of knowledge-based question-answering tasks.

We believe this approach has great potential to advance the development of this field. Additionally, we have made appropriate revisions based on the feedback provided by the reviewers. We sincerely hope that our paper will be accepted.

Once again, we sincerely thank you for your involvement and thoughtful feedback!

Thank you for your valuable comments. We will explain your concerns point by point.

Comment 1:

How do you ensure the reliability of this decomposition step, especially for queries with complex semantic dependencies?

Reply:

It is important to clarify that Table 7 presents the results of our traceability analysis on 100 error cases from the FactKG dataset. Due to the large number of entities and the high complexity of the questions in the FactKG dataset, the "Query Decoupling" task in the first step is relatively more challenging. Nevertheless, the accuracy of this task still exceeds 90% on the FactKG dataset, reaches over 97% on the WebQSP dataset, and is nearly 100% on the Meta QA dataset.

First and foremost, introducing the "Query Decoupling" step is essential. This step was designed to address challenges in semantic understanding and multi-hop reasoning for complex queries by decoupling intricate questions into manageable sub-queries. Attempting to retrieve and reason over a complete complex query in a single step often fails to produce accurate results, especially in the case of three-hop problems. While the decoupling process may introduce some errors, the proportion of such errors is controllable, and the benefits it brings in most cases significantly outweigh any potential drawbacks.

Secondly, we apply strict quality control measures to the generated training data to minimize the possibility of introducing errors, as described in Section 3.4.1, "Quality Control." We have developed specific evaluation methods to ensure the quality of the generated data. Specifically, let $E$ represent the entity set of the original sentence, and $E_{div,i}$ denote the entity set for each sub-query after decoupling. The criteria for considering the decoupling results as high-quality are as follows:

(a) $E_{div} \neq \emptyset$.

(b) $E = \bigcup_{i=1}^{H} e_{div,i}$.

(c) If $|E_{\text{div}}| > 1$, then $\forall e_{div,i} \in E_{\text{div}}, e_{div,i} \subsetneqq E$. If $|E_{\text{div}}| = 1$, then $E_{\text{div}} = E$.

Overall, we aim for all entities in the original entity set to be reasonably and accurately assigned to each entity subset, ensuring that every query is appropriately decoupling.

Comment 2:

Case analysis of scoring and filtering evidence triples using the expert model during the knowledge refinement stage.

Reply:

The expert model scores the evidence triples based on the query. In this step, conflicting knowledge with the query can be filtered out, while knowledge supporting the query is retained.

For example, using data from FactKG:

{
  "question": "Yes, Agra Airport is located in India where the leader is Narendra Modi.",
  "types": [["coll:model", "num2", "multi claim"]],
  "entity": ["India", "Agra_Airport", "Narendra_Modi"],
  "Label": [true],
  "triplet_evidence": [
      ["Agra_Airport", "location", "India"],
      ["India", "leader", "Narendra_Modi"],
      ["Agra_Airport", "location", "Uttar_Pradesh"]
  ]
}

The expert model scores the evidence triples, retaining those more relevant to the query, such as (Agra_Airport, location, India) and (India, leader, Narendra_Modi), while discarding those that do not directly support answering the query, such as (Agra_Airport, location, Uttar_Pradesh).

{
  "qid": 5517,
  "question": Abdul Taib Mahmud was born in the Kingdom of Sarawak and he was succeeded by Abdul Rahman Ya'kub.",
  "types": [["written", "num2", "multi claim"]],
  "entity": ["Abdul_Rahman_Ya'kub", "Kingdom_of_Sarawak", "Abdul_Taib_Mahmud"],
  "Label": [true],
  "used_all_relations": ["leader", "birthPlace", "placeOfBirth", "leader", "birthPlace", "placeOfBirth"],
  "total_evidence": [
      ["Abdul_Taib_Mahmud", "birthPlace", "Kingdom_of_Sarawak"],
      ["Abdul_Taib_Mahmud", "successor", "Abdul_Rahman_Ya'kub"],
      ["Abdul_Rahman_Ya'kub", "birthPlace", "Kingdom_of_Sarawak"],
      ["Abdul_Taib_Mahmud", "children", "Sulaiman_Abdul_Rahman_Taib"],
    ]
  ]
}

The expert model scores the triples and selects (Abdul_Taib_Mahmud, birthPlace, Kingdom_of_Sarawak) and (Abdul_Taib_Mahmud, successor, Abdul_Rahman_Ya'kub), while irrelevant noisy triples are filtered out to prevent interference with the reasoning results.

Thank you for your valuable comments. We will explain your concerns point by point.

Comment 1:

The proposed method’s novelty may be limited.

Reply:

One of our major contributions is the design of a knowledge-driven multi-task instruction tuning method, which enables the model to effectively complete the full process of decoupling, exploration, refinement, reconstruction, and reasoning within a knowledge graph through multi-task collaboration. Multi-task fine-tuning allows the model to share learned features and representations across different tasks. This not only improves training efficiency and generalization capabilities but also enables the model to transfer knowledge gained from one task to others.

Unlike traditional LLMs that primarily perform shallow understanding and generation of input content, our method endows the model with reflection capabilities based on knowledge graphs. Specifically, after training, the model can deeply evaluate the plausibility of information, actively identify and correct potential errors in knowledge. This capability surpasses traditional generative paradigms, enabling the model to perform deep reasoning in complex knowledge scenarios.

Another significant contribution is that we trained an expert scoring model based on LLM. This module can identify and filter out triples that do not support question answering, effectively controlling the introduction of noise. This significantly enhances the accuracy of reasoning tasks across various knowledge scenarios. Additionally, this module improves the interpretability and efficiency of knowledge-based question-answering tasks.

Comment 2:

The computational cost poses scalability challenges, as well as the average number of LLM calls.

Reply:

Firstly, our method RefKG employs a process of decoupling, retrieval, refinement, and reasoning to enable LLMs to engage in deep thought on knowledge graphs. By invoking the LLM multiple times to accomplish various tasks, this approach is not redundant but essential for fully tapping into the LLM's potential for deep understanding and utilization of knowledge, ensuring both accuracy of results and effective use of knowledge in a way that is irreplaceable.

KAPING[1] and KB-BINDER[2] make only a few calls to the large language model (LLM), including just one instance. We conducted a comparison with KAPING on WebQSP (wikidata) as shown in the following graph:

The results indicate that the approach of making only a few calls to the LLM fails to fully exploit the potential of the LLM to solve complex problems, thus not achieving optimal performance.

Secondly, from a scalability perspective, RefKG employs knowledge-driven multitask instruction fine-tuning on LLMs, allowing a single LLM to exhibit multiple capabilities. With a one-time training and deployment, it can flexibly handle calls for various tasks. This approach not only conserves resources but also maintains the method's transferability and scalability.

Finally, we conducted a detailed quantification of the scale and difficulty of different tasks, as well as the average number of times RefKG invoked LLMs and the inference speed. We randomly selected 100 samples from each dataset for experimentation, and the results are shown in the table below:

Across the three datasets, the average number of LLM invocations was 4.8, and the average total inference time was 2.1 seconds.

[1] Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering

[2] Few-shot In-context Learning for Knowledge Base Question Answering