W1:

Thank you for pointing out this critical issue. We acknowledge that our initial draft did not comprehensively discuss its performance.

This was primarily because, considering that the task types currently processable by GFMs are limited and well-defined, we initially judged that utilizing language models for classification among a small number of limited tasks is a relatively mature and reliable component.

To more precisely address your concerns, we manually annotated 1,000 node classification instances from the Computers dataset for validation within the limited time available, and tested using the text-embedding-ada-002 model. The results demonstrate excellent classifier performance: the average latency for classifying a single query is only 3.54 milliseconds and the classification accuracy reached 100%.

To further address your concern about the "impact of its classification errors," we deliberately assigned incorrect classifications to these 1,000 samples (classifying them as edge-level or graph-level tasks) for experimentation. The experimental results are shown in the following table:

This further confirms our initial judgment: for the limited task types that current GFMs can handle, LLM-based classifiers are already very mature and reliable. Even with certain classification errors, the overall system maintains good robustness.

W2

We thank the reviewer for this very pertinent critique. We have urgently added a supplementary experiment. The results are summarized as follows:

To achieve similar or even better performance levels, RAG4GFM only needs a small amount of time to complete indexing and retrieval construction for new knowledge, with computational resource consumption throughout the entire process being far lower than GraphLoRA.

W3

Thank you for raising concerns about the noise issue. This is indeed a critical consideration for RAG-type methods.

As this is the first work to establish a RAG paradigm in the GFM domain with graph data scenarios, we aimed to pursue a relatively general and flexible preliminary framework, and therefore did not specifically propose a dedicated module for identifying introduced noise.

Nevertheless, we have designed several noise-filtering information screening mechanisms in the RAG4GFM framework, primarily in three aspects:

• Task-Aware Reranking: Before fusion, our retriever reranks retrieved candidates based on comprehensive relevance to the query (Equation 3). This ensures that the most relevant graph knowledge is prioritized.
• Attention-based Fusion: In the graph fusion module, we dynamically allocate weights $\alpha_i$ to each retrieved graph through attention mechanisms. Even if a low-quality graph is retrieved, it will receive minimal attention weight, minimizing its impact on the final results.
• Threshold Filtering: The thresholds $\gamma$ and $\delta$ serve to thoroughly eliminate low-relevance information.

We have provided a preliminary case study in Appendix C.3 for your intuitive understanding.

Q1

Thank you for this insightful question about our feature engineering details. Our theoretical motivation for concatenating LAPPE with node in-degree and out-degree stems from the important principle of global-local information complementarity in modern graph representation learning. Existing research[1,2] shows that standard Laplacian positional encoding primarily contains global structural information rather than local structural information, limiting its performance in graph learning tasks requiring local structures. Specifically, LAPPE captures global positional information by embedding graphs into Euclidean space through eigendecomposition of the graph Laplacian[3], while in-degree and out-degree serve as simple local centrality measures reflecting node connectivity within neighborhoods[4]. The classic Graphormer[5] model adopts similar ideas, combining centrality encoding (based on node degrees) and spatial encoding to capture orthogonal and complementary structural information with excellent performance.

To quantitatively validate our design choice, we conducted a detailed ablation study.The results are summarized below:

This experiment provides strong evidence that these two feature types provide complementary information, thus justifying our design choice.

[1] Hoang, Thi Linh, and Viet Cuong Ta. "Balancing structure and position information in graph transformer network with a learnable node embedding." Expert Systems with Applications 238 (2024): 122096.

[2] Cantürk, Semih, et al. "Graph positional and structural encoder." arXiv preprint arXiv:2307.07107 (2023).

[3] Flores, Alejandro. Laplacian Positional Encoding. afloresep.github.io, 8 Oct. 2024, https://afloresep.github.io/posts/2024/10/laplacian_positional_encoding/. Accessed 31 July 2025.

[4] Metcalf, Leigh, and William Casey. "Chapter 5 - Graph Theory." Cybersecurity and Applied Mathematics, edited by Leigh Metcalf and William Casey, Syngress, 2016, pp. 67–94. https://doi.org/10.1016/B978-0-12-804452-0.00005-1. Accessed 31 July 2025.

[5] Ying, Chengxuan, et al. Do Transformers Really Perform Bad for Graph Representation? arXiv, 2021, https://arxiv.org/abs/2106.05234. Accessed 31 July 2025.

Q2

This is an excellent suggestion, and we acknowledge that we did not give this aspect sufficient attention in our initial experimental design.

Following your recommendation, we have added the parameter sensitivity analysis you suggested.The results are summarized in the table below:

The experimental results indicates that RAG4GFM has a relatively low dependency on meticulous parameter tuning, highlighting its practical utility.

We will add these detailed sensitivity analysis results, including validation on different datasets and guidelines for parameter selection, to the appendix of our revised manuscript.

Q3

Thank you for your extremely meticulous review, and we sincerely apologize for this oversight! The inconsistency you observed is indeed due to differences in evaluation settings, which we failed to clearly explain in our initial draft, and we deeply regret this confusion. The ablation study results in Figure 3 (88.5%) were conducted during the early stages of our research using a randomly selected subset (20% of the data) from the Computers dataset for rapid validation experiments, aimed at quickly assessing the relative importance of each component. In contrast, the results in Table 2 (79.56%) represent formal experiments conducted on the complete dataset following standard evaluation protocols. We recognize that this inconsistency seriously undermines the credibility and reproducibility of our work. Such methodological discrepancies are unacceptable in rigorous academic research, and we take full responsibility for this oversight. To rectify this critical issue, we have completely re-run all ablation experiments on the full Computers dataset using identical experimental protocols and evaluation metrics as those used for Table 2. The corrected ablation study results on the complete dataset are as follows:

The corrected results still clearly demonstrate the importance of the RAG framework, graph fusion module, and graph indexing components, supporting our core conclusions.

W1

Thank you for this very constructive suggestion. In our initial draft, due to the massive computational overhead of conducting multiple experimental runs across 7 different GFM backbone models and various tasks, we were unable to include error bars. Based on your recommendation, we have urgently completed supplementary experiments. We selected three GFM models (HiGPT, GLEM, AnyGraph) and re-ran experiments three times on two datasets (Computers, History), calculating standard deviations for performance metrics. The results are shown in the table below:

From the experimental results, the average performance across multiple runs is highly consistent with the single-run results reported in our initial draft, demonstrating the stability and reproducibility of our experimental results. Furthermore, all experiments maintain small standard deviations, indicating that our proposed RAG4GFM framework exhibits stable performance.

W2

Thank you very much for this insightful observation. RAG4GFM's performance being influenced by the capabilities of underlying GFMs and LLMs is indeed an inherent challenge that any RAG system faces. However, we believe this dependency is effectively mitigated in our framework.

First, the "retrieval-augmentation" paradigm of RAG can substantially suppress potential biases or hallucinations that underlying models might generate based solely on their internal parametric knowledge, by providing authentic and verifiable external data for the model's reasoning process.

Second, we employ a modular design that ensures our framework can be combined with different GFMs or LLMs without being bound to specific models. This means that as more powerful GFMs and LLMs emerge in the future, our framework can directly integrate them and benefit from their performance improvements.

Finally, we conducted extensive experiments in our paper: to demonstrate the universality of our framework, we selected up to seven GFMs with diverse architectures for comprehensive experiments. The results consistently show that RAG4GFM brings performance improvements to all these models, proving the broad effectiveness of our approach.

W3

Thank you for your valuable suggestion regarding theoretical depth. As the first work to systematically introduce the RAG paradigm to the GFM domain, our primary research focus was on the framework's innovative design and its extensive empirical validation. While providing a complete, novel theoretical proof for RAG4GFM on short notice is challenging, the effectiveness of our framework is highly consistent with existing theoretical analyses in the broader RAG research field.

Alignment with Existing RAG Theory

1. On Knowledge Integration: Recent theoretical work models RAG systems as cascaded information channels, where each component (query encoding, retrieval, context integration, generation) acts as a distinct information-theoretic channel with a measurable capacity[1]. Our Multi-level Graph Indexing is designed precisely to maximize the information capacity of these channels. Furthermore, our attention-based feature fusion mechanism adheres to the theoretical principle of "context-adaptive weighting" rather than simple information concatenation. A key innovation of our work, the fusion of topological structure, extends this integration theory from one-dimensional text to high-dimensional graph structures, ensuring that the graph's connectivity patterns are effectively absorbed and utilized as a core form of knowledge.

2. On Enhancing Reasoning Fidelity: A significant body of recent research suggests that a root cause of hallucination in LLMs is the high uncertainty (or entropy) in their generation process when faced with knowledge-intensive tasks[2]. By providing concrete, relevant external knowledge to condition the generative model, RAG dramatically reduces the model's predictive uncertainty (i.e., lowers complexity or perplexity) for a given task[3]. This reduction in uncertainty, in turn, greatly improves the fidelity and reliability of the output. RAG has also been theorized as an effective means of augmenting a model with non-parametric memory, efficiently decoupling the model's learned general reasoning abilities (parametric memory) from specific, updatable facts in an external knowledge base (non-parametric memory).

Existing Theoretical Underpinnings

Our current framework design also incorporates components with established theoretical support:

• The Multi-level Graph Indexing module's use of the HNSW algorithm comes with a theoretical complexity guarantee of $O(\log_2 N)$.
• The Laplacian Positional Encoding (LAPPE) we use is grounded in solid spectral graph theory and effectively captures the structural information of nodes.

Future Plan

Following your suggestion, we will add a new discussion chapter in the appendix of our revised manuscript. This section will delve deeper into the connection between our graph fusion mechanism and attention theory. It will also elaborate from an information-theoretic perspective on why "evidence-based reasoning" effectively reduces model uncertainty and, thus, hallucination. We hope this will partially address your expectations regarding theoretical depth.

[1] Zhao, Siyun, et al. "Retrieval augmented generation (rag) and beyond: A comprehensive survey on how to make your llms use external data more wisely." arXiv preprint arXiv:2409.14924 (2024).

[2] Gao, Yunfan, et al. "Retrieval-augmented generation for large language models: A survey." arXiv preprint arXiv:2312.10997 2.1 (2023).

[3] Shuster, Kurt, et al. "Retrieval augmentation reduces hallucination in conversation." arXiv preprint arXiv:2104.07567 (2021).

W4:

We sincerely thank the reviewer for pointing out these typographical and grammatical errors. We are very grateful for your meticulous and rigorous review and sincerely apologize for these oversights.

We promise that in the final version, all the errors you have listed (e.g., "Retriver" -> "Retriever") will be corrected. Furthermore, we will conduct a thorough professional proofreading of the entire manuscript to ensure the accuracy and professionalism of the language and to improve the overall readability of the paper.

W1

Thank you for your candid criticism. While our work builds on existing concepts, its core innovation is the first systematic adaptation of the RAG paradigm to graph-structured data and GFMs. Our framework provides an end-to-end solution to three fundamental challenges that prior methods do not address:

Unified Graph Indexing: Standard text RAG is insufficient for GFM scenarios involving hybrid queries of text and structure. We designed a unified multi-dimensional index that simultaneously encodes and indexes both textual semantics and graph topology.

Task-Aware Retrieval: We developed adaptive retrieval strategies with task-aware reranking mechanisms that dynamically select the optimal indices to support diverse GFM downstream tasks.

Graph Knowledge Integration: Simple text concatenation is ineffective for complex graph data. We pioneered a novel two-stage Graph Fusion Enhancement module that integrates semantic features via attention and topological structures via sparse matrix operations. To our knowledge, this graph-native fusion approach is a novel contribution to both RAG and GFM literature.

Integrating these solutions into a coherent RAG framework tailored for GFMs is a pioneering contribution that establishes a new research direction.

W2&Q4

We apologize for the unclear presentation in Section 3.3. Since images cannot be added, we provide a more intuitive explanation here.

The Graph Fusion Enhancement module aims to seamlessly integrate retrieved high-quality graph knowledge into the original query graph through two key steps:

Step 1: Attention-based Feature Fusion: Equations (4)-(6) compute attention weights $\alpha_i$ to measure similarity between the $i$-th retrieved graph and query graph, then use these weights to perform weighted averaging of node features $\mathbf{h}_u$ from retrieved graphs. The threshold function $\mathbf{I}_{{\alpha_i>\gamma}}$ filters low-quality results, ultimately fusing high-quality external features into the original node features $\mathbf{h}_v$ to obtain enhanced representations $\mathbf{h}'^{N}_v$, $\mathbf{h}'_{\text{SRC}^E}$, and $\mathbf{h}'_{\text{DST}^E}$. Equations (7)-(8) handle graph-level tasks, where $\beta_v^i$ computes attention weights between query graph global features $\mathbf{h}_q^g$ and retrieved graph global features $\mathbf{h}_g^i$ via SOFTMAX.

Step 2: Topological Structure Enhancement: Equation (9) performs structural fusion between the original query graph's adjacency matrix $\mathbf{A}$ and retrieved graphs' adjacency matrices $\mathbf{A}_i$. Each $\mathbf{A}_i$ is weighted by the same attention weight $\alpha_i$, with threshold function $\mathbb{I}[\alpha_i > \gamma]$ ensuring only high-quality structural information participates in fusion, forming the enhanced adjacency matrix $\mathbf{A}'$. This mechanism emphasizes highly relevant external knowledge through dynamic weight allocation while reducing noise through threshold filtering, achieving dual enhancement of features and structure.

W3&Q1

We thank you for pointing out this critical omission in our experimental description, and we sincerely apologize for the lack of clarity.

The graphs mentioned in Figure 1, such as "Code Graph," "Wikipedia Graph," and "Cite Network," are not the external knowledge bases used directly in our RAG experiments.

For our actual RAG experiments, to ensure rigor, reproducibility, and fair comparison, we adopted the following standardized method for constructing the external knowledge base:

For each specific downstream task, the corresponding external knowledge base is constructed exclusively from the training set of that task's dataset.

For example, when performing the node classification task on the Computers dataset, we build our multi-level graph index using all nodes, features, and connections present in the training split of that dataset. During the inference phase, when the model needs to make a prediction for a node in the test set, it retrieves information from this knowledge base built from the training set. This design ensures that the knowledge retrieved by RAG is highly relevant to the task while strictly avoiding any data leakage from the test set.

W4&Q3

Thank you for your very reasonable request for a more in-depth ablation study. To better understand the contribution of each component, we have added the more fine-grained ablation experiments you suggested.

Specifically, we selected two GFM backbones (AnyGraph, HiGPT) and conducted node classification experiments on two datasets (Computers, History). We compared the performance of several variants, focusing on two key areas: the indexing components and the graph fusion mechanisms. The results are presented below:

Analysis of Results

1. Ablation on Indexing Components: Crucially, the combination of Text Semantics and Structural indices (part of our full model) achieves the best performance (79.56%), a 3.22 percentage point improvement over the best single index, proving the effectiveness of fusing multimodal information at the retrieval stage.

2. Ablation on Graph Fusion Components:

• The Dual Fusion of Structure and Features (our full model, 79.56%) achieves the optimal performance, demonstrating that the two fusion mechanisms provide complementary enhancements.

Q2

Thank you for this valuable suggestion, which would greatly enhance the interpretability of our work. We have already included a detailed case study on link prediction in Appendix C.3 of our supplementary materials.

However, we recognize that while this case study describes the overall process, it could be more explicit in showing how retrieved subgraphs specifically influence predictions. Therefore, we supplement with the following detailed explanation:

Retrieval Impact: RAG4GFM discovered a key usage pattern where a high-end ultrabook (ThinkPad X1 Carbon) connects to both portable accessories (wireless mouse) and desktop peripherals (multi-port hub), providing critical structural evidence for the target product pair.

Fusion Process: These pattern-exhibiting subgraphs received high attention weights (α=0.79), integrating their structural information into the original SpectrePro X15 and PowerStation node representations, enabling the GFM to understand user habits of switching between mobile and desktop setups.

Prediction Enhancement: The enhanced representations increased the GFM's link prediction probability from 0.15 to 0.82, successfully identifying that these seemingly different products serve the same user group in different contexts.

We commit to adding a simplified illustrative diagram in the final version to provide readers with an at-a-glance view of how knowledge is retrieved, integrated, and impacts the model's decision-making process.

Q5

We sincerely apologize for the incomplete code package and the lack of clear instructions.

Our intention was to decouple our RAG4GFM framework from the replaceable GFM backbones. We were concerned that bundling the GFM code with our own might obscure our core contribution from the reviewers' perspective. Therefore, we only included a note in the README file instructing users to place the downloaded GFM code into a "GFM/" directory.

As we are now in the rebuttal phase, we are unable to add links or update the Supplementary Material. Instead, we offer a more detailed guide to running our code below:

Setup and Execution Guide

1. Place GFM Source Code: Place the source code of your chosen GFM backbone (e.g., AnyGraph, GraphGPT) into a folder named GFM/ located in the project's root directory.
2. Configure the Model: In the main configuration file, set the gfm_model parameter to the name of the chosen model (e.g., 'AnyGraph').
3. Run the Pipeline: Execute the main script by running python model/rag4gfm_pipeline.py.

L1

We thank the reviewer and agree that our discussion of limitations was too generic. We will completely rewrite this section to be more specific.

The revised "Conclusion and Future Work" section will address:

• Scalability Challenges: We will discuss the challenges of applying RAG4GFM to billion-node graphs and real-time systems, and explore solutions like DiskANN and asynchronous fusion.

• Knowledge Reliability: We will address the need for future confidence-scoring or fact-checking modules to handle noisy or outdated "dirty data" from the retrieval source.

• Broader Explorations: We will outline future research directions, including:

  • Utilizing "negative" or "contrastive knowledge" to refine decision boundaries.
  • Extending our framework to multimodal graphs that include images.

W1

1. Thank you for your valuable suggestion. We acknowledge that while HNSW provides efficient $O(\log_2 N)$ complexity, there remains room for optimization in specific scenarios.
2. We emphasize that the RAG4GFM framework is designed to be modular. This means our proposed Multi-level Graph Indexing module can be flexibly replaced with more targeted indexing structures. Beyond HNSW, our envisioned solutions include:
   • Ultra-large scale dataset solutions: For scenarios where indices cannot be fully loaded into memory, we can integrate DiskANN [1]. This approach is specifically designed for efficient ANN search on disk media such as SSDs, addressing HNSW's over-reliance on memory through optimized graph structures and I/O strategies.
   • Balanced and composite solutions: We can employ composite indices represented by IVFFlat [2]. Compared to HNSW, IVFFlat typically offers faster construction speed and lower memory consumption. While slightly inferior in peak query speed, it provides an excellent choice for applications requiring balance between resources and performance. Furthermore, IVFFlat can be combined with Product Quantization (PQ) [3] to further compress vectors, achieving higher memory efficiency.
3. In this work, we chose HNSW for its proven universality and excellent performance across various vector retrieval tasks, which is appropriate for building a foundation framework with broad applicability.
4. We will include discussion of different indexing structures in the revised version of our paper and identify this as a promising direction for future exploration.

[1] Jayaram Subramanya, Suhas, et al. "Diskann: Fast accurate billion-point nearest neighbor search on a single node." Advances in neural information processing Systems 32 (2019).

[2] Johnson, Jeff, Matthijs Douze, and Hervé Jégou. "Billion-scale similarity search with GPUs." IEEE Transactions on Big Data 7.3 (2019): 535-547.

[3] Jegou, Herve, Matthijs Douze, and Cordelia Schmid. "Product quantization for nearest neighbor search." IEEE transactions on pattern analysis and machine intelligence 33.1 (2010): 117-128.

W2

This is an excellent direction for future development. In the current work, our core objective is to establish a general RAG paradigm for GFMs, addressing the fundamental challenges they face when processing graph-structured data. Therefore, we have not considered other modalities such as images.

Our framework already possesses preliminary capabilities for handling multimodal data. For instance, our multi-level indexing mechanism inherently fuses semantic features and structural features of graph data.

Regarding the integration of visual information suggested by the reviewer, our framework can incorporate a visual encoder to integrate visual features and align them with graph embeddings and text embeddings in a unified semantic space for hybrid indexing. Additionally, our designed attention-based fusion enhancement mechanism can naturally accommodate additional modalities. Therefore, overall, our current framework can naturally support image data through the addition of a visual encoder.

We greatly appreciate your suggestion and will explicitly identify in the "Future Work" section of the revised version that extending RAG4GFM to broader multimodal scenarios represents an important research direction.

W3

Thank you for raising this important practical application consideration.

As foundational work aimed at validating the feasibility of a new paradigm, our current primary objective is to ensure the framework's generality and effectiveness. We have incorporated efficiency considerations into our design. As mentioned in Sections 3.1, 3.3, and Appendix A.1 of our paper, the current model primarily ensures baseline performance through efficient HNSW indexing and sparse matrix operations. Nevertheless, we acknowledge that in-depth optimization research for real-time response scenarios is indeed insufficient. Based on your suggestion, we have explored several possible implementations:

Retrieval Phase Optimization:

• Hardware acceleration and index compression: Fully leverage GPU parallel computing and employ techniques such as Product Quantization to compress indices, reducing computational load and I/O overhead during retrieval.
• Result caching and prefetching: For high-frequency query patterns, establish an LRU cache layer to directly return cached retrieval results.

Fusion Phase Optimization:

• Model quantization and distillation: Quantize features and model parameters in the graph fusion module.
• Asynchronous fusion and updates: Design the knowledge retrieval and fusion processes as background asynchronous tasks.

We will thoroughly explore these real-time optimization strategies in future work and explicitly identify this important research direction in the conclusion section of the revised version.

Q1

1. This is an excellent technical question. We apologize for not clearly explaining the weight design in the indexing construction and retrieval phases, and we deeply regret this oversight.

2. First, during the indexing construction phase, our core principle is to preserve all dimensional information without loss. Therefore, we do not apply any pre-weighting or fusion to text, structural, edge, and graph-level features. Instead, we construct independent, parallel indexing spaces for each feature type, thereby completely preserving their respective information.

3. Second, the "importance" of features is dynamically and adaptively manifested in the subsequent retrieval phase based on specific tasks, without requiring adjustment of feature weights for different types of graph data. Our task-aware retriever first determines the query type, then decides which indexing space(s) to retrieve from. For example, a node classification task will simultaneously query both text semantic indices and structural indices, then integrate the two sets of results through the RRF algorithm, which implicitly balances the contributions of different features according to task requirements.

Q2

Thank you for this very practical question.

Our current approach employs a LLM-based classifier (based on the text-embedding-ada-002 model) that determines retrieval strategies by computing semantic similarity between user queries and predefined task templates (such as node classification, link prediction, graph classification, etc.).

We acknowledge insufficient discussion of its performance in our draft. Considering that current GFM task types are limited and well-defined, we judged LLM-based classification as mature and reliable, focusing research efforts on more challenging graph knowledge retrieval and fusion aspects. Given that the capability boundaries of current GFMs are primarily concentrated on these limited standard graph learning tasks, when users pose queries beyond this scope, even if the classifier works perfectly, the underlying GFM itself cannot provide effective solutions. Therefore, we adopt a strategy of directly prompting users with "task not supported."

To more precisely address your concerns, we manually annotated 1,000 node classification instances from the Computers dataset for validation within the limited time available, and tested using the text-embedding-ada-002 model. The results demonstrate excellent classifier performance: the average latency for classifying a single query is only 3.54 milliseconds, and the classification accuracy on these 1,000 data instances reached 100%.

Q3

Your question touches upon one of the core motivations of our research. We need to honestly acknowledge that, as foundational work introducing the RAG paradigm to the GFM domain for the first time, our current primary objective is to validate the feasibility and general effectiveness of this new paradigm. Therefore, we have indeed given insufficient consideration to dynamic update scenarios in actual production environments.

While we have not specifically designed streaming graph experiments, our current experimental design simulates "knowledge update" scenarios to some extent. We deliberately selected datasets (such as the TAG benchmark) that were not used during GFM pre-training for testing. This process essentially simulates the core idea of models rapidly adapting and solving problems through external retrieval when encountering entirely new knowledge environments. The results in our paper demonstrate the effectiveness of RAG4GFM under this simulated scenario. Additionally, we supplemented with comparative experiments on knowledge update efficiency, comparing RAG4GFM with parameter-efficient fine-tuning methods such as GraphLoRA on the same tasks. The results show that our method achieves similar performance while demonstrating faster knowledge update speed and significantly reduced memory consumption, which indirectly proves the advantages of our framework in rapid knowledge updates.

However, we agree that we currently lack experiments on true streaming or dynamic graph scenarios. This is a critical direction that we must explore in-depth in our future work. We will explicitly state this limitation in the conclusion of our revised manuscript.