RAGraph: A General Retrieval-Augmented Graph Learning Framework

Thanks for your insightful comments. We address your concerns and answer your questions below.

W1: Why RAGraph works on par with RAG in NLP/CV, and how “generation” works. What pattern was learned from retrieved graph pattern?

A1: In NLP, RAG enhances the generation of LLM by retrieving relevant information via prompts. Similarly, in RAGraph, we enhance downstream graph learning by integrating information from retrieved toy graphs. Using these toy graphs with shared patterns assists the model inference. In our framework, the "generation" involves the retrieval-enhanced Graph Prompt: Toy Graph Intra Propagate & Query-Toy-Graph Inter Propagate to propagate retrieved knowledge (X and Y) into query graph. To illustrate, we analyze this from both experiment and theory.

1. Experiment 1: We perform a case study to illustrate how "generation" works by displaying specific instances of node vectors. Due to word restriction, please refer to the global response A2 and Rebuttal PDF Figure 1.

2. Experiment 2: In traditional GNN tasks, GCN, GAT, and GIN typically expand their receptive fields through stacked message-passing layers or neighborhood subgraph sampling for inference. Patterns learned in these contexts are often localized within the constrained receptive field. In contrast, in RAGraph, we observe that subgraphs sharing similar patterns often exhibit properties more aligned with downstream tasks. These subgraphs provide richer information for inference compared to simply enlarging receptive fields. As shown in Main Text Tables 1 and 2, Figure 3, RAGraph's strategy of incorporating toy graphs significantly outperforms baselines.

3. Theory 1: Furthermore, we provide a theoretical justification of retrieval augmentation in GNNs (see Appendix B.4). From an information-theoretic perspective, introducing RAG knowledge into GNNs enhances the mutual information between input features X and output labels Y, such that $I(X, RAG; Y) \geq I(X; Y)$, thereby improving the performance on downstream tasks. This is aligned with the information theory of RAG in NLP [1].

4. Theory 2: Recent studies [2] [3] also suggest the generalization error diminishes with an increase in the node number of the graph in Theorem 1.1 [4]: the generalization error between the expected loss $R_{exp}(\Theta)=\mathbb{E}_{(x,y)\sim \mu_G}[\mathcal{L}(\Theta(x), y)]$ and

empirical loss $R_{emp}(\Theta)=\frac{1}{m}\sum_{i=1}^m[\mathcal{L}(\Theta(x^i), y^i)]$ are supper bounded: $|R_{exp}(\Theta)-R_{emp}(\Theta)| \leq \sqrt{\frac{C}{m}q(N)}$, where C represents the model complexity (e.g., parameters), m denotes the training set size, and $q(N) = \mathbb{E}_{N\sim v} [N^{-\frac{1}{D+1}}]$ depends on the average graph size N (node num) with v is the graph size distribution and D is the metric-measure space dimension. In RAGraph, retrieving similar toy graphs significantly increases the number of graph nodes (via Query-Toy-Graph Inter Propagate, linking toy graph nodes to query graph), significantly augmenting N while reducing q(N). Consequently, upper bound of generalization error decreases, promoting smoother graph learning convergence and enhancing pattern learning.

[1] Generalization Analysis of Message Passing Neural Networks on Large Random Graphs. In NeurIPS 2022.

[2] Wasserstein Barycenter Matching for Graph Size Generalization of Message Passing Neural Networks. In ICML 2023.

[3] An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation. In ACL 2024

W2: Lack motivation to construct toy database and retrieve.

A2: The motivation behind initially constructing the toy graph database is to identify similar knowledge patterns in graphs (where the toy graph serves as a repository of such patterns, including X and Y). Therefore, establishing such a database is necessary to store potential candidate sets for downstream tasks. However, during the construction of toy graphs, if the toy graph size is too large, it may introduce excessive noise and adversely affect model; conversely, if it is too small, the introduced knowledge may be insufficient. Thus, we employ a method of chunking the resource graph given hop $k$. Moreover, to better store long-tail knowledge and simulate real world scenarios, we also introduce inverse importance sampling and leverage augmentation to expand toy graph database.

In retrieval process, the motivation is to comprehensively retrieve knowledge X and Y that can enhance downstream tasks, and we evaluate similarity across four dimensions: time, structure, environment, and semantic relevance in Appendix B.3.

Thank you for your suggestions, and we will incorporate these clarifications into the final version.

Q1: Why do we need Toy Graphs Augmentation.

A3: We have provided an analysis and ablation study to illustrate the importance of augmentations. Due to word restrictions, please refer to global response A1 and Rebuttal PDF Table 1.

Q2: Why noise-based graph prompting is required？

A4: The necessity arises because ensuring the high quality of the graph vector base is challenging, which heavily depends on the quality of external knowledge. In RAGraph, we introduce Noise-based Graph Prompting Tuning (outlined in Section 4.3.3) to address this challenge. Due to word restriction, please refer to global response A3.

Q3: If RAGraph still has the zero-short inference ability.

A5: Your suggestion is very insightful. To assess whether fine-tuned model maintains zero-shot inference capability when tested without knowledge, we conducted experiments comparing PRODIGY and RAGraph:

Results in Rebuttal PDF Table 5 indicate that the performance without knowledge shows a decrease compared to injected knowledge due to the absence of knowledge. However, compared to PRODIGY, RAGraph is more robust and we argue that the trained model still retains its zero-shot knowledge inference capability.

Thank you for your insightful comments. We address your concerns and answer your questions below.

W1: The diversity of datasets needs to be improved. For link prediction, the paper is evaluated on mostly e-commerce data, where knowledge graph tasks should also be considered. For node classification, both homophilic datasets e.g. Cora, Arxiv; and heterophilic datasets should be evaluated.

A1: Thank you very much for your suggestions.

• For the link prediction dataset: we have incorporated the link prediction results on the temporal knowledge graph ICEWS [1] with the backbone SPA [5].

• For the homophilic node classification datasets: we have introduced the OGBN-Arxiv [2] and Cora [3] datasets for the homophilic setting with backbone GCN.

• For the heterophilic node classification dataset: we have included the OGBN-MAG [4] graph, which contains 244,160,499 nodes and 1,728,364,232 edges with backbone R-GCN.

The experimental results of the RAGraph and baselines in Table 6 of the Rebuttal PDF demonstrate the superiority of the RAGraph, aligning with our analysis and conclusions presented in Section 5 Experiment.

[1] https://www.lockheedmartin.com/en-us/capabilities/research-labs/advanced-technology-labs/icews.html

[2] https://ogb.stanford.edu/docs/nodeprop/#ogbn-arxiv.

[3] https://paperswithcode.com/dataset/cora.

[4] Microsoft Academic Graph: when experts are not enough. Quantitative Science Studies. (2020). https://ogb.stanford.edu/docs/lsc/mag240m/.

[5] Search to Pass Messages for Temporal Knowledge Graph Completion. In ACL 2022.

W2: Lack of large-scale experiments.

A2: Thanks for your suggestions! We have conducted node classification (OGBN-MAG) tasks on large-scale graphs as detailed in A1 to W1 and shown in Table 6 of the Rebuttal PDF.

We will add these experiment results to the final version. In addition, regarding the reproducibility of the experiment, we have also updated the evaluation code for this sub-test in anonymous github.

W3: The model seems complicated with a big hyperparameter search space.

A3: Thank you for raising this question, indeed, the challenge of dealing with a large search space is inherent and formidable in most deep-learning models. In our RAGraph, for those sensitive hyperparameters $k$ and $topK$, we have presented in Figure 3. For less sensitive hyperparameters, we observed optimal performance across a broad spectrum of values and employed Bayesian optimization using Optuna (https://github.com/optuna/optuna) to address hyperparameter tuning.

To mitigate the difficulty of manually fine-tuning hyperparameters for each dataset, we adopted a Bayesian optimization technique that concurrently optimizes hyperparameters across multiple datasets on the validation set [6] [7]. This approach automates and streamlines the hyperparameter tuning process across various datasets in RAGraph, eliminating the need for individual dataset-specific fine-tuning, and enhancing both generalization capability and resource utilization.

[6] Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical Bayesian optimization of machine learning algorithms. In NeurIPS.

[7] Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & Freitas, N. (2016). Taking the human out of the loop: A review of Bayesian optimization. In IEEE Transactions on Power Delivery.

I've carefully read the rebuttal and appreciate the authors' efforts in addressing the weaknesses I identified.

Regarding W1, while I acknowledge the additional experiments, since academic graphs like MAG are known to be homophilic, the experiments still predominantly reflect a homophilic setting. For W2, I appreciate the inclusion of additional experiments, and the results are indeed convincing. However, for W3, the authors did not directly address my concern.

Given that the major concerns are still valid and my scores are already positive. I'm keeping my original score.

Dear Reviewer sivC,

Thank you for your careful reading of our rebuttal and for providing constructive feedback. We greatly appreciate the time and effort you have invested in evaluating our manuscript.

For Weakness 3, we apologize for not providing a direct response in our initial rebuttal. We have now prepared a more detailed analysis.

Firstly, the extensive hyperparameter search space in RAGraph is not merely a complexity for its own sake; it also facilitates the integration of more dimensions of information, such as $w_1\sim w_4$ balancing the importance of different similarities and $\gamma$ weighing the significance of task-specific output vectors versus hidden embeddings. This, in turn, enhances RAGraph's performance across a range of tasks, including node-level, edge-level, and graph-level scenarios. By accommodating a comprehensive search space, our model can effectively capture the intricate relationships between nodes and edges, leading to superior performance across diverse tasks.

Secondly, aside from those sensitive parameters $k, topK$ which we have already supplemented in sensitivity experiments in Figure 3, and apart from the augmentation number $K=50$, we also attempt to remove $\alpha, \lambda, \gamma, w_1\sim w_4$, and conduct additional experiments to assess the impact of these hyperparameters, denoted as "w/o hyperparameters":

The results demonstrate that even without these hyperparameters, RAGraph still exhibits strong performance, with only a slight decrease compared to the tuned settings. This robustness underscores the strength of our model’s design, highlighting its ability to maintain high performance across varying conditions. Consequently, these hyperparameters perform well under a broad range of search spaces. This also further validates that when employing optimization algorithms like Bayesian optimization, RAGraph does not require as extensive a search space compared to traditional method like grid search.

Thirdly, the challenge of dealing with a large search space is inherent and formidable in most deep-learning models. In order to expedite the process of finding the most suitable hyperparameters while reducing search space, we utilized Optuna [2], an advanced hyperparameter optimization framework. Optuna employs an efficient sampling algorithm that significantly reduces the size of the search space by intelligently selecting candidate hyperparameters based on their potential impact on model performance. Unlike grid search or random search, Optuna dynamically prunes unpromising trials, focusing on the most promising regions of the search space, thus accelerating the overall training and improving the efficiency of our experiments.

Lastly, to empirically validate the effectiveness of Optuna, we conducted experiments using ACM dataset [1] and set hyperparameter $\alpha, \lambda, \gamma, w_1\sim w_4$ between range [0,1]. We compared the results of Grid Search and Optuna under the same settings. For Grid Search, with each parameter having a search step size of 2, we explored a total of 2^7 combinations. With the HAN backbone taking nearly 1.2 seconds per epoch and running for 10 epochs, the total time for Grid Search was [27 minutes and 39 seconds]. In contrast, using Optuna with 100 trials under the same settings, the total time was reduced to [4 minutes and 28 seconds]. This empirical analysis demonstrates that Optuna significantly reduces the search space of RAGraph and accelerates the hyperparameter optimization process.

We hope these additional points will further clarify your questions. If you have any further questions, please do not hesitate to contact us. We are committed to improving the manuscript and ensuring that all concerns are adequately addressed. Thank you again for your valuable suggestions and for maintaining a positive assessment of our work.

[1] Wang X, Ji H, Shi C, Wang B, Ye Y, Cui P, Yu PS. Heterogeneous graph attention network. In WWW 2019.

[2] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD.

Best regards,

Authors of the paper 8566

Dear Reviewer sivC,

Thank you for your careful reading of our rebuttal and for providing constructive feedback. We greatly appreciate the time and effort you have invested in evaluating our manuscript.

Regarding Weakness 1, it appears there may have been a misunderstanding.

We adhere to the definition of a heterophilic graph as outlined in the HAN [1] in Introduction <Heterogeneity of graph>, which describes it as "a heterogeneous graph is a special kind of information network containing either multiple types of entities or multiple types of links". In our rebuttal experiments, the Microsoft Academic Graph (MAG) [2] contains four types of entities: papers, authors, institutions, and fields of study, as well as four types of relationships that connect these entities: an author is "affiliated with" an institution, an author "writes" a paper, a paper "cites" another paper, and a paper "has a topic of" a field of study. Thus, the MAG dataset fulfills the criteria for heterophilic settings.

Additionally, in our experiments, we utilized dynamic bipartite graphs such as TAOBAO, KOUBEI, and AMAZON, which also contain two types of entities and are classified as heterogeneous graphs. Furthermore, we also conducted experiments on a more heterogeneous knowledge graph, ICEWS, as part of our rebuttal.

Moreover, we included additional experiments using datasets with more widely recognized heterogeneous graphs, as employed in HAN [1], specifically IMDB (three types of entities and two types of relationships) and ACM (three types of entities and two types of relationships). Specifically, we strictly follow the experiment configuration of HAN and leverage HAN as backbone model:

The results from these datasets also demonstrate the effectiveness of our approach. We will add these experiment results to the final version.

[1] Wang X, Ji H, Shi C, Wang B, Ye Y, Cui P, Yu PS. Heterogeneous graph attention network. In WWW 2019.

[2] Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. Microsoft academic graph: When experts are not enough. In Quantitative Science Studies 2020.

Best regards,

Authors of the paper 8566

Thanks for your insightful comments. We address your concerns and answer the questions below.

W1 & Q1: Unclear of the contribution of (a) inverse importance sampling, (b) augmentation, (c) four similarities, and (d) fusion method to performance improvement.

A1: We conducted four ablation experiments on both node and graph classification tasks with settings in Appendix C.4 with the following variants by removing: (a) inverse important sampling strategy (wo IIS), (b) augmentation (wo AUG), (c) any one of the four similarities (wo Time, wo Structure, wo Environment, wo Semantic); and (d) Only use X (mask the task-specific output vector, w X) and only use Y (mask the decoder, w Y):

(a) The adoption of Inverse Importance Sampling strategy is crucial. In RAGraph, subgraphs are sampled as toy graphs, where nodes with higher degrees (non-long-tail knowledge, extensively learned and embedded into GNN parameters) are more frequently included in subgraphs due to their extensive connections with neighbors, resulting in higher frequency in toy graph base [1]. Conversely, nodes with low degrees (long-tail knowledge), are more important but ignored. To mitigate this issue, we propose this by prioritizing nodes with lower degrees to capture long-tail knowledge when sampling.

The ablation results in Rebuttal PDF Table 2 show that w (with) IIS significantly outperforms wo (without) IIS.

(b) Furthermore, regarding the rationale for conducting augmentation, due to word restriction, please refer to global response A1, and ablation result is in PDF Table 1.

(c) In practical applications, the four similarities all contribute to performance improvement and we state the significance as follows:

• Time information is crucial to predict future states or trends [2] via node history, i.e. in social networks, analyzing historical user interaction aids in predicting future behaviors.

• Structure pertains to how nodes are interconnected and overall graph topology, vital for capturing similar graph structure patterns [3]. In transportation networks, factories are always located on the outer ring of the city, sharing similar structural connectivity, aiding in the discovery of spatio-temporal patterns [4].

• Sharing similar neighborhoods is essential for evaluating node similarity and correlation. In recommendations, shared purchase histories between users and products indicate potential interests, akin to collaborative filtering [5].

• Semantic information measures similarity based on features [6]. In knowledge graphs, identifying relevant subgraphs to query nodes enhances retrieval accuracy based on semantic similarity.

The ablation results in Rebuttal PDF Table 3 indicate that each type of similarity has a positive impact.

(d) Fusion and decoder here represent one of the core contributions of RAGraph:

• Overall Task Perspective: For same tasks, decoder can be directly employed to obtain outputs. For different tasks, decoder can be masked and utilize pre-computed embeddings without training or be tuned to better adapt. This underscores our primary contribution, where decoder functions as a versatile "plug-and-play" and "tune-free" component.

• Integral Fusion Strategy: Fusion Strategy facilitates concurrent information propagation from toy graphs X (hidden embeddings) and Y (task-specific output vector) to query graph, aligning with our secondary contribution.

Experimental results in Rebuttal PDF Table 4 show the effect brought by fusion is enormous, and any deficiency in X / Y has an impact on model performance.

[1] Walking in Facebook: A case study of unbiased sampling of OSNs. In SIGCOMM 2010.

[2] GraphPro: Graph Pre-training and Prompt Learning for Recommendation. In WWW 2024.

[3] Position-aware Graph Neural Networks. In ICML 2019.

[4] Spatiotemporal Multi-Graph Convolution Network for Ride-Hailing Demand Forecasting. In AAAI 2019.

[5] A Survey of Collaborative Filtering Techniques. In AAAI 2009.

[6] PRODIGY: Enabling In-context Learning Over Graphs. In NeurIPS 2023.

W2 & Q2: Unclear how to set hyperparameters.

A2: For those sensitive $k$ and $topK$, we have presented and analysed in Figure 3. For less sensitive hyperparameters, we observed optimal performance across a broad spectrum of values. In our study, for less sensitive parameters, we employed Bayesian optimization using Optuna (https://github.com/optuna/optuna) to achieve hyperparameter tuning.

• Bayesian Optimization: To mitigate the difficulty of manually fine-tuning hyperparameters for each dataset, we adopted a Bayesian optimization technique that concurrently optimizes hyperparameters across multiple datasets on the validation set [7] [8]. This approach automates and streamlines the hyperparameter tuning process across various datasets in RAGraph, eliminating the need for individual dataset-specific fine-tuning, and enhancing both generalization capability and resource utilization. We apologize for our oversight of the paper and will supplement details into the main text.

• Hyperparameter Settings: The hyperparameter configuration in our study is detailed as follows: $k$ is set to 2, $topK$ is set to 5, $\alpha=\lambda=\gamma=0.5, K=50, w_1=w_2=w_3=0.1, w_4=0.6$, which can be found in Appendix C.4.

[7] Practical Bayesian optimization of machine learning algorithms. In NeurIPS 2012.

[8] Taking the human out of the loop: A review of Bayesian optimization. In TPD 2016.

W3: Two inaccurate writings.

A3: Thanks for pointing out this question. We will correct the two inaccurate writings in final version by replacing (1) "all" to "almost all", (2) "inferior" to "better". Additionally, we have thoroughly reviewed the manuscript for any other potential typographical errors to ensure accuracy.

Q4: Qualitative analyses of toy graphs retrieving.

A4: Due to word restriction, please refer to the global response A2 and the Rebuttal PDF Figure 1.

Dear Reviewer 3qQ8,

We would like to express our gratitude for taking the time to review our paper and provide us with valuable comments. We understand that you have busy schedules and apologize for any inconvenience caused by our urging letter.

With only 1 day left in the discussion period, we hope to receive feedback from the reviewers: did our response address your concerns, and what can we do to further improve our score?

Thank you for your consideration, and we look forward to receiving your feedback soon.

Best regards,

Authors of the paper 8566

Thank you for your insightful comments. We address your concerns and answer your questions below.

W1: The difficulty to construct and maintain high-quality and diverse graph vector base for different tasks.

A1: We acknowledge the challenge you mentioned of constructing and maintaining high-quality graph vector bases tailored to diverse tasks. In RAGraph, our toy graph base largely leverages significant prior research datasets in pre-trained GNNs [1] [2] [3] [4], which are trained on meticulously curated graph datasets and cover diverse domains, such as biology, chemistry, medicine recommendation tasks, etc. For example, the PROTEINS dataset [5], derived from cryo-electron microscopy and X-ray crystallography, and the ENZYMES dataset [6], based on EC enzyme classification, are meticulously annotated by medical experts.

Moreover, to address inherent challenges in data quality, we introduce Noise-based Graph Prompting Tuning (Section 4.3.3). This method involves fine-tuning the model with artificially introduced noisy toy graphs (Inner-Toy-Graph Noise & Toy-Graph Noise), inspired by noise-tuning techniques in NLP [7] [8] [9]. Our approach enhances the model's robustness against real-world retrieval noise, as evidenced by superior performance compared to traditional tuning methods (in Main Text Tables 1 and 2). This approach reduces the stringent requirement for an exceptionally high-quality graph vector base, thereby ensuring robust performance across various tasks within our RAGraph, and significantly mitigating data quality impacts.

Lastly, to verify the diversity of applications for RAGraph, we also conducted experiments on the time-series encyclopedia TGB Wiki, the paper-cited datasets Arxiv and Cora, and large-scale graphs MAG in Rebuttal PDF Table 6.

The experimental results demonstrate that RAGraph can be applied in diverse real-world cases. In addition, the effect of noise-based fine-tuning is better than that of direct fine-tuning, which also proves the effectiveness of our NFT approach, effectively addressing the inherent challenge in data quality.

[1] Liu, Z., Yu, X., Fang, Y., & Zhang, X. 2023. GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks. In WWW.

[2] Xia, L., Kao, B., & Huang, C. (2024). OpenGraph: Towards Open Graph Foundation Models. Retrieved from arXiv preprint arXiv.

[3] Yu, X., Zhou, C., Fang, Y., & Zhang, X. (2023). MultiGPrompt for Multi-Task Pre-Training and Prompting on Graphs. In WWW.

[4] Huang, Q., Ren, H., Chen, P., Kržmanc, G., Zeng, D., Liang, P., & Leskovec, J. (2023). PRODIGY: Enabling In-context Learning Over Graphs. In NeurIPS.

[5] Borgwardt, K. M., Ong, C. S., Schönauer, S., Vishwanathan, S. V. N., Smola, A. J., & Kriegel, H.-P. (2005). Predicting protein function through graph kernels. In Bioinformatics.

[6] Wang, S., Dong, Y., Huang, X., Chen, C., & Li, J. (2022). FAITH: Few-shot graph classification with hierarchical task graphs. In IJCAI.

[7] Yoran, O., Wolfson, T., Ram, O., & Berant, J. (2024). Making Retrieval-Augmented Language Models Robust to Irrelevant Context. In ICLR.

[8] Fang, F., Bai, Y., Ni, S., Yang, M., Chen, X., & Xu, R. (2024). Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training. In ACL.

[9] Cuconasu, F., Trappolini, G., Siciliano, F., Filice, S., Campagnano, C., Maarek, Y., Tonellotto, N., & Silvestri, F. (2024). The Power of Noise: Redefining Retrieval for RAG Systems. In arxiv.

Dear Reviewer va3W,

As the discussion phase is ending tomorrow, we extend our gratitude once more for your valuable and insightful comments!

We have provided careful and detailed responses to all your questions. It would be greatly appreciated if you could kindly let us know whether we have answered all your questions. Please also kindly let us know if you have any further questions, and we would like to try our best to resolve them before the deadline.

Best regards,

Authors of the paper 8566

Dear Reviewer va3W,

Thank you for your careful reading of our rebuttal and for providing constructive feedback. We are delighted to know that your main concerns have been resolved! We greatly appreciate the time and effort you have invested in evaluating our manuscript.

Regarding your remaining concern, we believe there may have been a misunderstanding. In our experiments, we utilized a diverse set of datasets, including academic datasets such as PROTEINS, BZR, Cora, IMDB, ACM and large-scale datasets like MAG, supplemented by Rebuttal. In addition to these, we employed several industrial graph datasets, including TAOBAO (from the largest online shopping platform in China, https://ali-home.alibaba.com/en-US/about-alibaba), KOUBEI (from the large consumption platform, https://www.koubei.com/), and AMAZON (from the largest online shopping platform in USA, https://www.amazon.com/).

On large-scale dynamic recommendation graph data—specifically TAOBAO, KOUBEI, and AMAZON—our performance significantly outperforms that of GraphPro [1]. GraphPro has already been successfully deployed on a large-scale online platform for dynamic streaming data (as referenced in Section 4.5 of GraphPro), where it has achieved notable improvements in CTR prediction performance. In contrast to GraphPro, our algorithm not only delivers superior performance but also offers faster inference speed.

To prove this point, we conducted training and inference on 204,168 nodes in the dynamic TAOBAO dataset, and calculated their time consumed under the same settings:

As illustrated in the Table, this advantage stems largely from RAGraph's k-hop subgraphs—query graphs—for inference, whereas GraphPro requires inference over the entire graph. Furthermore, RAGraph's capability to directly retrieve historical evaluation data without the need for model fine-tuning enables it to achieve superior results, as demonstrated by the comparison between GraphPro/NF and Vanilla/NF in Table 2 in the main Paper.

Therefore, in terms of performance and efficiency, our model outperforms GraphPro which has been deployed online on these three large-scale dynamic industrial graph datasets In addition, our toy graph base can also be cached to accelerate deployment in the industrial area.

Given the proven success of GraphPro in online deployment, we are optimistic about the future application of RAGraph in the industrial GNN field, i.e., in the recommendation system. We also anticipate that RAGraph could potentially achieve notable results in RAG within NLP, leveraging the potential success of large graph models.

At last, please also kindly let us know if you have any further questions or what can we do to further improve our score? And we would like to try our best to resolve them before the deadline.

[1] GraphPro: Graph Pre-training and Prompt Learning for Recommendation. In WWW 2024.

Best regards,

Authors of the paper 8566