GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation

W1/Q1: Scalability to large KGs

We agree that handling large KGs is essential for real-world applications. To support large-scale KGs, inspired by LLM training, we implement a Single Program Multiple Data (SPMD)-style message-passing mechanism that enables distributed training and inference across multiple GPUs.

Specifically, we partition the full KG into balanced subgraphs using the METIS algorithm, with each device storing only a subset of the graph in memory. During message passing, each device first aggregates information locally and then exchanges messages with other devices to finalize the node embedding updates. This design allows GFM-RAG to scale effectively by leveraging more hardware.

In addition, instead of performing message passing over the entire KG, we can only conduct it on the localized k-hop subgraph around the query. This significantly reduces computational and memory costs, further enhancing scalability on large KGs.

We will clarify this strategy in the revised version and add a discussion in the appendix.

W2/Q2: Use of DistMult and limitations

We appreciate the reviewer highlighting the limitations of DistMult in modeling antisymmetric or inverse relations. We chose DistMult as the message function for its simplicity and computational efficiency, as it has no training parameters and is easy to compute. That said, we agree that more expressive models like RotatE $1$ or QuatE $2$ could potentially enhance performance. In preliminary experiments, we found DistMult to offer a strong tradeoff between efficiency and effectiveness in our retrieval-focused setting, but exploring alternative KGE models in our GFM is a valuable direction that we plan to pursue in future work.

Q3: Clarification of Eq. 19 (Path interpretation)

We thank the reviewer for pointing this out. In Eq. 19, the top-k paths are selected based on the product of gradient scores over triples forming the path, which approximates the contribution of that path to the final prediction via the chain rule. This allows us to identify influential multi-hop reasoning chains and interpret the model’s behavior. We will clarify this in the main text and provide more details in the Appendix.

W3/Q4: Comparison with G-Rretriever and SubgraphRAG

We thank the reviewer for highlighting recent related work. In response, we have conducted additional experiments comparing GFM-RAG with SubgraphRAG and G-Retriever across three benchmark datasets. The results are shown below:

Comparison with SubgraphRAG and G-retriever.

As the results show, GFM-RAG consistently outperforms both SubgraphRAG and G-Retriever across all datasets and recall settings. While SubgraphRAG and G-Retriever both employ GNN-based reasoning to enhance retrieval, they are primarily tailored to well-constructed KGs with domain-specific fine-tuning, and often operate on shallow, fixed subgraphs. Their architectures typically involve small GNNs with limited reasoning depth and generalization ability.

In contrast, GFM-RAG introduces a generalizable graph foundation model trained over diverse KGs and designed to operate on noisy, incomplete, and large-scale knowledge graphs. This allows GFM-RAG to capture transferable multi-hop reasoning patterns and maintain robust performance across domains without domain-specific adaptation.

We will incorporate these comparisons and insights into the final version to enhance our empirical and methodological contributions.

W1: Novelty of pipeline components

We understand the reviewer’s concern that individual modules in GFM-RAG may appear conventional. However, the core novelty lies in the integration and scaling of these components into a unified graph-based foundation model designed for retrieval-augmented generation (RAG).

To our knowledge, GFM-RAG is the first scalable foundation model for graph-based retrieval, enabling single-step multi-hop reasoning and adhering to neural scaling laws, with strong zero-shot generalization across domains. These contributions go beyond reusing standard modules, aiming to elevate the RAG paradigm using GNN-enabled foundation models and pretraining strategies.

W2: Architecture and training method ablation

We appreciate the reviewer’s suggestion to provide a deeper analysis of the architectural and training design choices. We have conducted a comprehensive set of ablation studies, with results summarized in Appendix E:

• In Appendix E.1, we compare different sentence embedding models used in GFM-RAG.

• In Appendix E.2, we evaluate the effectiveness of the two training tasks (KG completion and document retrieval) and their impact on downstream performance.

• In Appendix E.3, we examine the sensitivity of GFM-RAG to different task loss weightings.

• In Appendix E.5, we analyze the impact of model parameter size on performance.

To further investigate architectural design, we varied the number of GNN layers from 1 to 8 while keeping the hidden dimension fixed (512), and evaluated model performance across all datasets. The results are shown below:

Ablation study of different GNN layers.

We observe that performance generally improves with deeper GNN layers, which we attribute to both the increased model sizes and the ability to capture more complex multi-hop associations. This trend aligns with the neural scaling laws observed in foundation models, where larger parameter counts typically yield better generalization.

Interestingly, we find that performance peaks around 4 layers in some cases. As discussed in Appendix A and Section 4.8, GFM-RAG is designed to capture logical associations from KGs through multi-hop message passing. However, since the maximum number of reasoning hops required by our datasets is 4, additional layers beyond this offer limited benefit—likely due to the absence of higher-hop training signals. This finding supports our hypothesis that GFM-RAG effectively learns query-relevant multi-hop reasoning paths, and that deeper architectures may not improve performance without datasets requiring more complex reasoning.

In summary, these results demonstrate the effectiveness and interpretability of the proposed GNN-based architecture, and confirm that both model capacity and logical expressibility contribute to GFM-RAG’s strong performance. We recognize the potential of other architectural designs and aim to explore them in the future, inspiring the community to do the same.

W3: Performance gap with HotpotQA leaderboard

We thank the reviewer for pointing out the discrepancy with the leaderboard performance. As stated in Section 4.1 and Appendix B, to ensure fair comparison with recent methods such as HippoRAG and IRCoT, we adopt the same experimental setup that uses the same 1,000 randomly sampled HotpotQA subsets from the validation set, rather than the full test set. This setting is widely adopted in prior LLM-based multi-hop QA works to reduce computational costs associated with LLM-based generation while enabling reproducible and meaningful benchmarking.

Therefore, the performance reported in Table 2 is not directly comparable to the leaderboard numbers based on the full test set. We will clarify this in the final version and note this distinction more explicitly to avoid confusion.

Q1: Why use GNNs as the backbone for GFM?

We appreciate this excellent question. Currently, graph foundation models (GFMs) are still an emerging area without a fixed architectural definition. Our decision to use GNNs is grounded in both theory and practicality:

• Scalability: Transformer-based graph models are often computationally prohibitive for large graphs. GNNs scale more gracefully with graph size.

• Logical Reasoning: As discussed in Appendix A, GNNs possess strong logical expressiveness, making them well-suited for multi-hop and compositional reasoning, which is essential for retrieval tasks. Case studies can be found in Section 4.8.

Our implementation (8M parameters, 6-layer GNN) is a starting point toward scaling graph-based foundation models efficiently. We view this as a foundation for future architectural exploration. In our response to W2, we also analyze the impact of different GNN layers.

Q2: Generalization to evolving domains and KGs

We agree that rapidly evolving KGs present challenges. However, GFM-RAG is trained on diverse and large-scale KG corpora, enabling it to generalize to new or unseen domains and KGs without retraining.

As demonstrated in Figure 3, without retraining, GFM-RAG performs consistently well across seven out-of-domain datasets, all of which involve previously unseen domains and KGs. These results demonstrate that GFM-RAG is not overfitted to a fixed KG schema or domain but instead learns generalizable reasoning capabilities. This allows GFM-RAG to adapt to new and evolving KGs without requiring frequent retraining.

W1: Without fine-tuning, the performance of GFM-RAG is poor and does not support zero-shot QA retrieval. This suggests training directly on the target dataset may be more reasonable.

Clarification of the term “fine-tuning”

We appreciate the reviewer’s concerns and would like to clarify that the term “fine-tuning” in Table 10 refers to supervised document retrieval fine-tuning (Section 3.2.2)—not fine-tuning on downstream QA datasets.

As detailed in Section 3.2, GFM-RAG follows a two-stage training process:

1. Self-supervised KG Completion Pretraining (stage 1): GFM is trained to predict masked relations and entities from incomplete KGs, fostering robust multi-hop reasoning and tolerance to KG noise.
2. Supervised Document Retrieval Fine-tuning (stage 2): Analogous to instruction tuning in LLMs, this stage trains GFM-RAG on diverse retrieval instances to learn to understand users’ queries and better capture the relevance between questions and knowledge for retrieval.

We acknowledge the terminology may have caused confusion and will revise the paper to consistently use “supervised fine-tuning” for Stage 2 and “domain-specific fine-tuning” for adaptation to individual target datasets.

Table 10 Clarification:

The experiments are conducted on three multi-hop QA datasets (HotpotQA, Musique, 2Wiki). Removing supervised fine-tuning significantly reduces retrieval accuracy, underscoring its necessity for effective query relevance modeling. While skipping KG pretraining has limited impact on retrieval performance, it enhances the model's ability for other tasks, such as KG completion (Table 11), and improves overall robustness in handling incomplete KGs. A detailed explanation is also provided in our response to Q3.

Zero-shot Generalizability

Section 4.6 and Figure 3 demonstrate GFM-RAG’s strong zero-shot retrieval performance across seven unseen domain-specific datasets, surpassing HippoRAG and LightRAG. Table 13 and Appendix E.4 further show that optional domain-specific fine-tuning can further enhance performance when target data is available:

Comparison between the zero-shot performance and the performance of fine-tuning on target domains.

Is training directly on the target dataset more effective?

To address this, we compare GFM-RAG with methods trained directly on the target multi-hop datasets. Specifically, we compare with the following baselines:

• HippoRAG, a strong, non-training-required baseline.
• SubgraphRAG [1] and G-retriever [2], two GNN-enhanced retrieval methods that are trained from scratch on the target datasets.
• GFM-RAG (target), the proposed GFM retriever that is trained from scratch on the target dataset.
• GFM-RAG (all), the proposed GFM retriever that is trained on multiple datasets (HotpotQA, MuSiQue, and 2Wiki).

Performance of directly training on target datasets.

The results show that even when directly fine-tuned on target datasets, GFM-RAG (target) outperforms SubgraphRAG and G-retriever, demonstrating the superiority of the GFM architecture and the effectiveness of two-stage training. Additionally, GFM-RAG (all) performs competitively across all datasets, confirming its ability to leverage diverse KGs and develop generalizable reasoning skills across domains.

We will revise the terminology and include these expanded analyses in the final version.

[1] Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation. ICLR 25

[2] G-retriever: Retrieval-augmented generation for textual graph understanding and question answering NeurIPS 24

W2: Impact of GNN depth

We conducted an ablation varying the number of GNN layers (1–8) while fixing the hidden dimension (512). The results are shown below:

Ablation study of different GNN layers.

Performance improves with deeper GNNs due to larger size and better multi-hop reasoning, echoing neural scaling laws. Yet, performance sometimes peaks at 4 layers—matching the 4-hop max in training data—suggesting GFM-RAG captures relevant multi-hop paths, with deeper layers offering little gain without more complex signals. Detailed analysis can be found in our response to Reviewer 9ZFX W2.

W3: Addressing KG noise and incompleteness.

We appreciate the reviewer’s concern, which aligns with our use of GNNs for their robustness to KG incompleteness. Real-world KGs are inherently noisy. Unlike structure-only methods like HippoRAG, we adopt GNN and enhance its robustness through self-supervised KGC pretraining: masking relations and predicting links to foster multi-hop reasoning in incomplete settings. Table 11 confirms its effectiveness in handling incomplete graphs.

Experiment results on KG completion

Q2: Ablation Study of Training Datasets

We want to confirm that GFM-RAG is trained on multiple datasets (HotpotQA, MuSiQue, 2Wiki), but we ensure strict separation between training and evaluation data, with no overlap in documents or entities, thus eliminating any risk of information leakage.

The motivation for this setup is rooted in the foundation model paradigm. Similar to how LLMs are trained on diverse corpora to learn generalizable linguistic patterns, GFM-RAG aims to learn universal graph reasoning capabilities from diverse KG-indexes. This design enables strong zero-shot transfer to new domains and datasets, eliminating the need for retraining or fine-tuning on every downstream domain. Results in Fig. 3 and our response to W1 demonstrate the strong generalizability of GFM-RAG on out-of-domain datasets without retraining.

We further conducted ablation studies where GFM-RAG is trained separately on each dataset, and we report performance across all three benchmarks. Results are shown below:

Ablation study of GFM-RAG trained on each dataset.

These results show that GFM-RAG not only performs well on the trained datasets, but also generalizes well to other datasets. More importantly, the model trained on multi-domain datasets performs competitively across all datasets—validating its ability to generalize effectively across domains and benefit from training on diverse KGs by learning generalizable reasoning ability across domains.

We will include this ablation study and further clarify the design rationale in the final version of the paper.

Q3: Inconsistency between Table 10 and Table 13.

Table 10 examines the impact of two-stage training strategies, while Table 13 evaluates zero-shot generalizability and domain-specific transferability; their findings are not contradictory.

Table 10 Clarification:

“Fine-tune” and “pre-train” refer to “Supervised Document Retrieval Fine-tuning” and “Self-supervised KG Completion Pre-training,” respectively, as explained in our response to W1.

Removing supervised fine-tuning significantly reduces retrieval accuracy, highlighting its importance for capturing query-document relevance. In contrast, removing self-supervised pretraining has a smaller effect on retrieval but improves generalization to tasks beyond retrieval and handling incomplete KGs.

Table 13 Clarification:

The “fine-tune” in Table 13 denotes the “fine-tuning on domain-specific QA datasets”.

Table 13 compares the zero-shot performance of HippoRAG, LightRAG, and GFM-RAG across seven domain-specific test sets without additional training. GFM-RAG, trained on multi-hop QA datasets, shows strong retrieval performance across all unseen datasets, demonstrating robust zero-shot generalizability.

When fine-tuned on these seven domain-specific training sets, GFM-RAG’s performance further improves, underscoring its strong transferability when target training data is available.

W1.1: Lack of ablation study on ent-document inverted index

We sincerely thank the reviewer for highlighting this point. We have conducted a detailed ablation study to compare several alternative inverted index strategies:

• IDF+Top-T Pred: Our proposed method (Eq. 14–16), which maps the top-T entities predicted by GFM to documents using inverse document frequency (IDF)-weighted scores.
• IDF+All Pred: Use all predicted entities from GFM and weights them by IDF (w/o Eq 14)
• Top-T Pred: Use only the top-T predicted entities without applying IDF weighting (w/o Eq 15).
• All Pred: Use all entity predictions and directly map to document scores (w/o Eq 14 and 15).

Ablation study of different inverted indices.

The results show that the proposed IDF + Top-k Pred performs the best. This indicates that the inverted index is a crucial component of GFM-RAG, which serves as a bridge between structured reasoning over KGs and the unstructured documents required by LLMs, necessitating a careful design.

We acknowledge the reviewer’s concern regarding potential alternatives, and as a promising future direction, we plan to explore end-to-end models that can jointly reason over structured and unstructured knowledge without relying on an explicit inverted index.

W1.2: Inference time comparison

We conducted an empirical analysis comparing GFM-RAG, GraphRAG, and LightRAG in terms of both retrieval latency (Time in seconds) and retrieval quality (Recall@5). The results are shown below:

Inference time comparison.

The results clearly demonstrate that GFM-RAG outperforms both LightRAG and GraphRAG in retrieval accuracy while also being significantly more efficient in terms of inference time. Despite performing the inverted index, GFM-RAG achieves superior latency because the reasoning is executed in a single GNN forward pass over a pre-constructed knowledge graph index.

In addition, GFM-RAG also benefits from the quick index process during retrieval, as it does not construct a traditional vector database to store documents and entities:

Indexing time comparison.

We will include these results in the final version to improve clarity and better highlight the advantages of GFM-RAG.

W2.1: Ablation study of training datasets

We thank the reviewer for raising this valuable point. We want to confirm that GFM-RAG is trained on multiple datasets (HotpotQA, MuSiQue, and 2Wiki), but we ensure strict separation between training and evaluation data, with no overlap in documents or entities, thus eliminating any risk of information leakage.

The motivation for this setup is rooted in the foundation model paradigm. Similar to how LLMs are trained on diverse corpora to learn generalizable linguistic patterns, GFM-RAG aims to learn universal graph reasoning capabilities from diverse KG-indexes. This design enables strong zero-shot transfer to new domains and datasets, eliminating the need for retraining or fine-tuning on every downstream domain. Results in Fig. 3 demonstrate the strong generalizability of GFM-RAG on out-of-domain datasets without retraining, compared to GraphRAG and LightRAG.

We further conducted ablation studies where GFM-RAG is trained separately on each dataset, and we report performance across all three benchmarks. Results are shown below:

Ablation study of GFM-RAG trained on each dataset.

These results show that GFM-RAG not only performs well on the trained datasets, but also generalizes well to other datasets. More importantly, the model trained on multi-domain datasets performs competitively across all datasets—validating its ability to generalize effectively across domains and benefit from training on diverse KGs by learning generalizable reasoning ability across domains.

We will include this ablation study and further clarify the design rationale in the final version of the paper.

W2.2: Token cost for index construction

In Appendix E.7, we report the token cost during index construction. Specifically, we find that constructing the KG-index requires approximately 4.8 M tokens per 1,000 documents, which costs around $2.6 using GPT-4o-mini. LightRAG and GraphRAG cost 5.7 M tokens and 7.6 M tokens, respectively. Compared to other methods, GFM-RAG is more cost-effective as it does not require generating community-level summaries.

Token cost comparison for index construction