BioKGBench: A Knowledge Graph Checking Benchmark of AI Agent for Biomedical Science

Dear reviewer dk1Q,

Thank you for your comments. I would like to address your concerns point by point.

Weakness 1: Lack cost analysis of BKGAgent.

Thank you for your suggestion. We have conducted a cost analysis based on agent histories, calculating the average tokens per task, as shown in the table below. AutoGPT, despite being a single-agent system, has higher costs than AutoGen (dual-agent) due to its use of the ReAct algorithm. Our BKGAgent incurs higher costs due to its three-agent framework and the need for detailed tool usage instructions in prompts. This aligns with our discussion in the manuscript (lines 509–522).

Weakness 2 & Question 3: The generalizability of proposed pipeline to other domains

Our work specifically focuses on the biomedical domain rather than building a general agent. When designing BKGAgent, we carefully considered the unique characteristics of biomedical knowledge and data, enabling it to process KGs, databases, and research papers effectively. Our pipeline is generalizable to most biomedical KGs, as they typically adopt a property graph model. Additionally, the generalizability of BKGAgent can be further enhanced by extending its specialized tools to accommodate other domain-specific requirements.

Weakness 3 & Question 4: Minor writing issues with the use of the word "propose".

Thank you for your suggestion. We will replace the word "propose" with "adopt" in the revised PDF.

Question 1: How the labels are made for SCV task? Any human agreement tests on the labeling?

The SCV dataset is reconstructed from well-known, expert-annotated datasets (lines 249-251). Additionally, we performed a secondary verification using Qwen1.5-72B to ensure the claims are consistent and free from conflicts.

Question 2: Do the authors have a vision on the difference between proposed agent method and the existing agent methods? How the difference can impact on the performance on BioKGBench?

Yes, we discuss the differences and their impact primarily in Section 4.2:

• Atomic capabilities: In Table 6, we compare the atomic abilities of BKGAgent with existing agents, highlighting that many existing agents cannot query KGs or access external knowledge bases, which makes them inadequate for tasks like KGCheck (lines 445–447).
• General abilities and specialized tools: In the Agent Comparison section (lines 458–470), we emphasize that existing agents lack powerful tools compared to BKGAgent, resulting in poor performance. This highlights the importance of integrating general capabilities, such as coding, with specialized tools to improve agent effectiveness.
• Numerical differences: In lines 509–522, we analyze the impact of increasing the number of agents, concluding that while it slightly improves performance, it also raises communication costs and complexities, which can offset the benefits.

Question 5: Classification process and scoring criteria for SCV task.

As mentioned in lines 246-247, LLMs perform scientific claim verification using a Retrieval-Augmented Generation (RAG) approach. As detailed in Figure 3 and Appendix C.3, we fix the RAG pipeline with state-of-the-art embedding models and rerankers, feeding retrieval results into the prompts of LLMs, which then classify the claims as "support," "refute," or "nei." We evaluate LLM outputs using Accuracy, Right Quotes, and Error Rate (see Table 4, lines 363-367).

We hope our response has adequately addressed your concerns. We appreciate the time and effort you have invested in reviewing our work.

Best regards,

Paper 7569 Authors

Dear reviewer X1bo,

Thank you for your comments. I would like to address your concerns point by point.

Weakness 2: Discuss the relationship between KGQA and KBQA and the feasibility of using KBQA methods in the KGQA and KGCheck tasks.

Here, we highlight the key differences between KBQA and our KGQA task:

• Different Input: KBQA datasets, such as CWQ, WebQSP, and GrailQA, provide the key entity in each question as part of the task input. In contrast, our KGQA task takes only the raw question as input, requiring LLMs not only to select appropriate tools based on context but also to perform Named Entity Recognition (NER) and relationship matching to derive the tool's parameters. Therefore, our task is more challenging and better suited for evaluating LLMs.

• Different KG Structures: Works like Think-on-graph utilize knowledge bases such as Freebase, Wikidata, and DBpedia, which are based on RDF (Resource Description Framework) representations. In contrast, the vast majority of knowledge graphs in the biomedical domain, such as CKG, are built using property graph model. These two types of knowledge graphs differ significantly in terms of data storage models and query languages:

  • Data Models: RDF enforces a strict triple-based format <subject, predicate, object> for representing data, whereas property graph model stores data in the form of nodes (entities) and edges (relationships), where both nodes and edges can have attributes stored as key-value pairs. This fundamental difference leads to distinct querying strategies.
  • Query Languages: RDF-based knowledge graphs rely on SPARQL as the query language, whereas the latter uses Cypher.

These differences make it infeasible to directly benchmark KBQA methods like Think-on-graph on our KGQA task.

Weakness 4: Explaination of the metrics for the process.

We apologize for the lack of detailed explanation of the process metrics. In the revised PDF, we will include the following clarifications and add the prompts used in the appendix:

• Understanding: whether the agent clearly understood the task and the purpose of the given tool.
• Reasoning: whether the agent arrived at the final answer through sufficient evidence and reasoning, rather than simply providing random answers or guessing.
• Efficiency: whether the agent efficiently solved the problem without unnecessary discussion on unrelated topics.
• KG Process: whether the agent queried the knowledge graph during the task.
• Information Retrieval: whether the agent retrieved information from external knowledge sources in some way during the check.

As mentioned in lines 452-457, we employ Qwen2-72B to score the agent's performance across these five criteria, where the model provides a simple "yes" or "no" response for each criterion.

Question 1: The proportion of different reasoning types (one-hop, multi-hop, conjunction) involved in the KGCheck task.

The KGCheck task focuses on atomic checks, which are classified into node-level and triple-level checks (as detailed in Table 5 and Lines 286–297). Since one-hop, multi-hop, and conjunction reasoning ultimately reduce to triple-level checks, we do not categorize them separately. For triple-level checks, we classify them into two types:

• Triple-existing: Ensures the accuracy of relationships between connected nodes (25.8%).
• Triple-potential: Assesses whether a relationship exists between unconnected nodes (29.8%).

Question 2: The recall rate of errors detected by BKGAgent on the KGCheck task.

As shown in the table below, our BKGAgent achieves a 20% higher recall rate of errors (i.e., the ability to correctly identify errors in the KG) than the second-best agent, demonstrating its strong performance.

Weakness 1: The effectiveness of using three agents.

While our BKGAgent shows only a slight performance improvement over “AutoGen w/ our tools” in terms of EM, it achieves a 20% higher recall rate of errors (i.e., the ability to correctly identify errors in the KG) than the second-best system, demonstrating its strong performance.

Additionally, regarding the slight EM improvement, as a benchmark study, we also explored how varying the number of agents impacts performance to derive meaningful insights. As discussed in lines 509–522:

• Increasing the number of agents raises communication costs, limiting significant performance gains.
• Effective planning algorithms have a more substantial impact than the number of agents, as shown in Figure 5 (6.3% vs. 0.9%).

Dear reviewer pqUW,

Thank you for your comments. I would like to address your concerns point by point.

Weakness 2: Limited applicability of the benchmark.

The KGCheck task combines two atomic tasks—KGQA and SCV. LLM agents are required to identify errors in large-scale KGs by processing both structured data (such as knowledge graphs) and unstructured data (such as literature). These are exactly the types of data typically generated by biomedical research. KGCheck evaluates the agent’s ability to comprehend and handle biomedical knowledge from heterogeneous sources, which we believe is highly valuable in this context and offers insights into how LLM agents can be used to detect and correct errors in large-scale datasets.

Weakness 3&5: Scalability concerns for larger knowledge graphs and limited generalizability to other KGs.

Our approach is scalable in biomedical domain. Our approach is specifically designed for property graph models, which are widely adopted in the construction of biomedical KGs, such as BioKG, PharmKG, PrimeKG, etc. This characteristic makes our approach inherently scalable to other property graph-based knowledge graphs in the biomedical domain. We chose the CKG because it is one of the most authoritative and comprehensive biomedical KGs (line 197-206). As shown in following table, CKG's scale and complexity are larger than many other well-known KGs. This ensures that our experiments are conducted on a representative and sufficiently complex graph. Given that property graph models dominate the biomedical domain and that we have validated our approach on a large and complex graph like CKG, we believe our method is scalable and well-suited for biomedical knowledge graphs.

Table. Comparison of different biomedical knowledge graphs.

Weakness 4: Concerns over using an outdated knowledge graph.

As noted in lines 298–300, the CKG used in our study is built using database data from May 2024, not 2020. To ensure consistent benchmarking, we fix the KG data's timeframe, and the dataset labels are updated annually with the latest database releases to maintain relevance and accuracy.

Question 2: Difference references of CKG.

Thank you for pointing this out, and we apologize for this minor writing issue. We will address it in the revised PDF and ensure the references are corrected.

[1] Walsh, Brian, Sameh K. Mohamed, and Vít Nováček. "Biokg: A knowledge graph for relational learning on biological data." Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020.

[2] Zheng, Shuangjia, et al. "PharmKG: a dedicated knowledge graph benchmark for bomedical data mining." Briefings in bioinformatics 22.4 (2021): bbaa344.

[3] Chandak, Payal, Kexin Huang, and Marinka Zitnik. "Building a knowledge graph to enable precision medicine." Scientific Data 10.1 (2023): 67.

[4] Santos, Alberto, et al. "A knowledge graph to interpret clinical proteomics data." Nature biotechnology 40.5 (2022): 692-702.

Weakness 1: Lack comparison with other RAG-based baselines.

It is important to clarify that the SCV task is not designed to evaluate different RAG methods but rather to evaluate LLMs as agents within a plug-and-play RAG pipeline, using fixed SOTA embedding models and rerankers. As shown in Figure 9:

• Step 1: Input claims and the corpus are embedded using a fixed embedding model and stored in a vector database.
• Step 2: A reranker retrieves the top-k relevant bibliographies.
• Step 3: The claim and retrieved bibliographies are fed to the LLM via a prompt template, and the LLM generates conclusions with logical quotations. In this step, we substitute and compare different LLMs to evaluate their performance.

In conclusion, RAG serves as a tool for LLMs in our setup, not the primary subject of evaluation. Our study aligns with AgentBench [1] setup, aiming to assess LLMs' abilities in the biomedical domain, including tool usage, terminology comprehension, reasoning, rather than benchmarking RAG implementations.

Question 1: Difference between KGQA and STaRK.

• Setup: STaRK evaluates RAG methods using various embedding models and rerankers, some of which are outdated now. In contrast, RAG serves as a tool for LLMs in our setup. We fix the RAG pipeline with state-of-the-art components (Appendix C.3, Figure 9) and use it to feed retrieval results into the prompts of LLMs, allowing the LLM to perform claim verification. Our primary focus is to evaluate the LLM agent's capabilities, such as tool usage, biomedical knowledge understanding, and reasoning.
• Tasks: STaRK emphasizes retrieval, while our work goes further by requiring LLMs to comprehend biomedical knowledge and reasoning to perform claim verification and make discoveries.

We hope our response has adequately addressed your concerns. We appreciate the time and effort you have invested in reviewing our work.

[1] Liu, Xiao, et al. "Agentbench: Evaluating llms as agents." arXiv preprint arXiv:2308.03688 (2023).

• Weakness 4: Missing details.

  For SCV, does the model have access to the whole corpus with 5664 articles, or it only have access to the particular articles?

  Thank you for your comment. To clarify, the LLMs have access to the whole corpus (the abstracts of 5,664 articles) when performing the claim verification task. We will add this clarification in the revised PDF.

  If this task is about global claims, how to handle the potential conflicts between claims?

  As shown in Table 4, the claims are derived from the experimental results in the papers. The SCV dataset is reconstructed from well-known, expert-annotated datasets (lines 249-251). Additionally, we performed a secondary verification using Qwen1.5-72B to ensure the claims are consistent and free from conflicts. Therefore, we do not expect any conflicts between the claims.

  What is the additional information the Qwen model used for evaluation has access to?

  As mentioned in lines 454-456 of the paper, we provide the Qwen2-72B model with a prompt that includes several scoring demonstrations, the histories of the agents to be evaluated, and the evaluation criteria (five aspects).

  Does the framework support unseen nodes on the KG?

  Our BKGAgent does support unseen nodes on the KG. However, our task only requires the agents to handle existing nodes in the KG.

  For the baseline agent without access to the "our tools", how do they access the KG and literature pool?

  For the baseline agents, we provide prompts detailing how to query the KG (e.g., URL, username, password) and include instructions to verify findings using reliable external literature and databases.

We hope our response has adequately addressed your concerns. We appreciate the time and effort you have invested in reviewing our work.

• Weakness 2: Comparison with works of the KG-guided QA setting.

  Existing works in the KG-guided QA setting are KBQA (Knowledge Base Question Answering). Here, we highlight the key differences between KBQA and our KGQA task:

  • Different Input: KBQA datasets, such as CWQ, WebQSP, and GrailQA, provide the key entity in each question as part of the task input. In contrast, our KGQA task takes only the raw question as input, requiring LLMs not only to select appropriate tools based on context but also to perform Named Entity Recognition (NER) and relationship matching to derive the tool's parameters. Therefore, our task is more challenging and better suited for evaluating LLMs.
  • Different KG Structures: Works like Think-on-graph utilize knowledge bases such as Freebase, Wikidata, and DBpedia, which are based on RDF (Resource Description Framework) representations. In contrast, the vast majority of knowledge graphs in the biomedical domain, such as CKG, are built using property graph model. These two types of knowledge graphs differ significantly in terms of data storage models and query languages:
    • Data Models: RDF enforces a strict triple-based format <subject, predicate, object> for representing data, whereas property graph model stores data in the form of nodes (entities) and edges (relationships), where both nodes and edges can have attributes stored as key-value pairs. This fundamental difference leads to distinct querying strategies.
    • Query Languages: RDF-based knowledge graphs rely on SPARQL as the query language, whereas the latter uses Cypher.

  These differences make it infeasible to directly benchmark KBQA methods like Think-on-graph on our KGQA task.

• Weakness 3: Soundness of the evaluation metrics.

  • To clarify (also see lines 293-297 and Table 5), an existing relationship refers to a relationship that is already represented by an edge between two nodes in the KG. In this case, we check whether the known relationship between the two entities is correct. A potential relationship, on the other hand, refers to the absence of an edge between two nodes in the KG, which suggests that no relationship exists between the entities. In this case, we examine whether a relationship actually exists between these two nodes.
  • In our task, an agent verifies KG triples using reliable external sources (as ground truth). If the agent finds a contradiction between the KG and external literature or databases, we label this as "refute". For example, if an agent identifies a relationship between two unconnected entities based on reliable papers, it indicates an error in the KG, which we define as "refute".

Dear reviewer 1J1A,

Thank you for your comments. I would like to address your concerns point by point.

• Weakness 1: Novelty as a benchmark work.

  When evaluating the LLMs only (Table 7), the LLMs do not have access to the KG or literature database, making the task a straightforward task.

  As stated in the paper (lines 234-235 and 246-247), the LLMs do have access to the knowledge graph (KG) and literature database during the evaluation. This follows the interactive evaluation of LLM-as-Agent setup from AgentBench [1]. Specifically, for KGQA, we provide each LLM with the same tools via identical prompts. For SCV, we develop a plug-and-play retrieval-augmented generation (RAG) pipeline for the LLMs. These two tasks systematically evaluate an LLM's abilities in the biomedical domain, including tool usage, terminology understanding, reasoning, and more.

  When evaluating the agents (Table 8), the assumption is that the tool design is not part of the agent design.

  We would like to clarify that we did not assume tools are separate from the agent system; rather, tools are a critical part of the system. In Table 8, we evaluate both the existing agent with our tools and the vanilla agent without our tools. Our results show that our tools enhance performance, while also demonstrating that a general agent, relying solely on coding abilities, cannot effectively handle specialized tasks in professional domains.

  The agent benchmark task is not designed to be result-oriented

  Each task in our benchmark has been designed with specific result-oriented evaluation metrics, as outlined in Table 1 of the paper. These metrics were carefully selected to quantitatively assess the performance of the agents based on the results they produce, ensuring a clear measure of success for each task.

[1] Liu, Xiao, et al. "Agentbench: Evaluating llms as agents." arXiv preprint arXiv:2308.03688 (2023).