Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
The first attempt to fully utilize the potential of data to empower directed graph learning through data-centric machine learning.
摘要
评审与讨论
This paper proposes a general data-centric directed graph online knowledge distillation framework called EDEN. The framework achieves data-centric machine learning, guided by the proposed hierarchical encoding theory for the graph-structured data. The paper conducts experiments to validate the efficacy of the proposed method.
给作者的问题
N/A
论据与证据
It is not clear why (at least should give some examples or references) "real-world digraphs commonly exhibit hierarchical structure" (LHS of line 206, line 207).
方法与评估标准
The method seems valid but requires lightweight adaptation due to scalability issues. There are only two dataests for link prediction though, which may not be sufficient (but acceptable).
理论论述
The claims seem correct, but I did not check carefully.
实验设计与分析
The designs and analyses seem valid.
补充材料
I roughly glanced through it.
与现有文献的关系
There seems to be good relations to existing literature, and with novel contributions.
遗漏的重要参考文献
N/A
其他优缺点
The paper has somewhat too long an abstract, and it is not friendly to non-experts in the specific field.
其他意见或建议
N/A
Q1: Claims And Evidence
We sincerely apologize for the insufficient explanation in our initial submission, which may have caused confusion. We kindly ask you to refer to our response to Reviewer RNnj Q2, where we provided an example of the hierarchical structure of directed graphs in the real world using citation networks. We also elaborated on our motivation for employing trees as a hierarchical data structure for data organization and learning, which is inspired by other relevant references.
If possible, we will include this real-world example of hierarchical structures in directed graphs in Sec 1 and the appendix of the revised submission, along with additional references, to provide a clearer presentation.
We hope that the above responses address your concerns and enhance your confidence in our manuscript.
Q2: Methods And Evaluation Criteria
We sincerely appreciate your valuable insights, and please allow us to offer the following justifications:
(1) Scalability Issues and Lightweight Alternative: Your concerns about scalability issues regarding our proposed EDEN are valid, and as a matter of fact, we have recognized it as the main bottleneck that requires our future efforts to optimize it further based on the lightweight alternative of EDEN in Sec 3.4. We kindly refer you to our response to Reviewer RNnj Q1 & Q2, which includes the theoretical motivation and implementation details of the current lightweight alternative, as well as the discussion on potential future efficiency improvements and the revision plans in the new submission.
(2) Probably Missed Link-level Test: In response to your concerns regarding the absence of results for the link prediction task, we apologize for the lack of a direct indication in the main text, which may have led to your omission. In fact, we have already provided additional link-level evaluations for all datasets in Appendix A.14 of the initial submission. We fully intend to address this in the subsequent editing process.
Once again, we thank your valuable comment and hope our response can address your concern.
Q3: Other Strengths And Weaknesses
Thank you for your comments. We acknowledge that the current abstract may be more extensive than typical abstracts. Our aim was to provide detailed explanations to help readers understand the intended task and the design of EDEN. However, we recognize the need for improvements and will refine the abstract during the editing process to enhance clarity and conciseness while ensuring intuitive interpretations of EDEN’s core contributions for future readers.
This paper introduces entropy-driven digraph knowledge distillation (EDEN), a data-centric framework for representation learning on directed graphs. EDEN addresses the limitations of current directed Graph Neural Networks by leveraging directed structural measurements to construct a hierarchical knowledge tree (HKT), refined using mutual information of node features. This enables topology and profile knowledge transfer through a tree layer-wise distillation process. For downstream tasks like node classification and link prediction, EDEN employs a random-walk based technique. Experiments show that EDEN, as a hot-and-plug module, improves the performance of existing methods.
给作者的问题
See "Experimental Designs Or Analyses"
论据与证据
The motivation for the proposed method lacks sufficient clarity. The claim that existing methods "fail to fully explore the potential correlations between directed topology and node profiles at the data level" is not adequately substantiated. Illustrating how these limitations impact downstream tasks like node classification or link prediction would provide a clearer rationale for the necessity of the proposed method.
方法与评估标准
The paper suffers from a lack of necessary details, which hinder readability and comprehension. Many technical definitions, such as the structural measurements (Eqs 1-3), are presented without intuitive explanations. These shortcomings make the paper challenging to grasp and weaken its overall accessibility.
To comprehensively assess the scalability and efficiency of EDEN, it is crucial to include larger datasets in the evaluation, particularly to test its performance on large-scale graphs.
理论论述
N/A
实验设计与分析
Are there any latest methodologies from 2024 that could serve as baselines, and if so, should they be included to ensure a more comprehensive and up-to-date evaluation?
补充材料
I recommend including the commands to re-implement the experimental results in the README, with the hyperparameters for all experiments.
与现有文献的关系
The topic is quite relevant.
遗漏的重要参考文献
N/A
其他优缺点
See all sections above.
其他意见或建议
See all sections above.
Due to the word limit imposed by the new regulations of ICML 2025 rebuttal, we have not provided detailed references, but we will gladly supply them in our subsequent discussions if needed.
Q1: Claims And Evidence
We sincerely apologize for any concerns that may have arisen. We kindly refer you to our response to Reviewer RNnj Q1, which addresses similar issues. Based on this, we provide the following clarifications and outline our revision plans, hoping to further alleviate your concerns.
(1) Important Directed Edges: Recent studies [LoG 2024, ICDE 2024, WWW 2025] have shown that considering edge directionality provides a novel approach to addressing the heterogeneity issue. These studies conducted extensive empirical analyses, highlighting the significant impact of extracting directed graph knowledge to enhance downstream task performance.
(2) Our Contribution: Although they are effective, these methods vary in data knowledge extraction and lack a unified framework. Therefore, we propose EDEN, which aims to achieve more efficient data-centric graph learning and further enhance downstream task performance (i.e., fully explore the potential correlations between directed topology and node profiles at the data level). Based on this, inspired by your insightful comments, we have conducted additional experiments to further demonstrate the improvements of our framework on existing studies.
| Models | Empire | Rating | ||||||
|---|---|---|---|---|---|---|---|---|
| Node-C | Existence | Direction | Link-C | Node-C | Existence | Direction | Link-C | |
| ADPA | 79.3±0.4 | 67.3±0.5 | 54.2±0.4 | 59.0±0.5 | 44.8±0.5 | 77.8±0.4 | 83.7±0.3 | 64.5±0.5 |
| ADPA+EDEN | 81.7±0.4 | 68.6±0.4 | 55.3±0.6 | 60.7±0.3 | 46.6±0.3 | 79.4±0.5 | 85.2±0.2 | 66.0±0.6 |
| MAP++ | 79.5±0.3 | 67.6±0.6 | 54.7±0.5 | 59.8±0.4 | 45.4±0.5 | 78.5±0.3 | 84.4±0.3 | 65.6±0.4 |
| (MAP++)+EDEN | 81.4±0.6 | 69.0±0.5 | 56.4±0.4 | 61.2±0.4 | 47.5±0.4 | 79.8±0.3 | 85.6±0.4 | 66.8±0.5 |
We apologize if our initial presentation caused any misunderstandings. In the revised submission, we will reorganize the motivation, references, and experimental results based on the above clarification for a clearer presentation.
Q2: Methods And Evaluation Criteria
Please allow us to offer the following explanations and revision plans, which we trust will effectively address your concerns and enhance your confidence in our manuscript.
Enhancing the Readability
We plan to add intuitive explanations and background about formulas and introduce a table that includes mathematical symbol definitions. Taking Eq. (1–3) as examples, we will provide more background on the concept of Shannon entropy and graph mining.
Scalability Test and Performance
We fully agree on the importance of enhancing EDEN's scalability. However, as the first data-centric digraph learning framework, some complexity is inevitably introduced. We kindly ask for your understanding in this regard. That being said, we introduce a lightweight alternative in Sec 3.4, which has been evaluated on the million-scale Arxiv and WikiTalk and renders it comparable to or even superior to the best baseline (Table 1 and Figure 3). We also plan to provide a more detailed discussion on potential efficiency optimizations, as elaborated in our responses to Reviewer RNnj Q1.
Q3: Experimental Designs Or Analyses
Thanks for your valuable suggestions. We add ADPA (ICDE 2024) and MAP (WWW 2025) as additional baselines to the already existing Dir-GNN (LoG 2024) and HoloNet (ICLR 2024) as follows:
| Node-level | CoraML | CiteSeer | WikiCS | Tolokers | Arxiv |
|---|---|---|---|---|---|
| ADPA | 83.2±0.6 | 64.4±0.6 | 80.2±0.5 | 79.5±0.3 | 67.8±0.5 |
| MAP++ | 83.5±0.4 | 64.7±0.7 | 79.8±0.4 | 80.1±0.3 | 68.2±0.4 |
| EDEN | 84.6±0.5 | 65.8±0.6 | 81.4±0.3 | 81.3±0.2 | 69.7±0.3 |
| Link-level | Slashdot | WikiTalk | ||||
|---|---|---|---|---|---|---|
| Existence (AUC) | Direction (AP) | Link-C (ACC) | Existence (AUC) | Direction (AP) | Link-C (ACC) | |
| ADPA | 90.9±0.1 | 92.6±0.1 | 86.0±0.1 | 94.8±0.2 | 90.8±0.1 | 90.6±0.1 |
| MAP++ | 91.2±0.0 | 92.7±0.1 | 86.4±0.1 | 94.7±0.1 | 90.6±0.1 | 90.5±0.0 |
| EDEN | 91.8±0.1 | 93.1±0.0 | 87.1±0.2 | 95.4±0.1 | 91.7±0.1 | 91.0±0.1 |
Q4: Supplementary Material
Thank you for your notification, we will include implementation details in the README to guide any interested researchers to implement our method.
Thank you for the rebuttal. I still have the following concerns:
Q1: What does "data-centric" mean in your response to Reviewer RNnj Q1?
Q2(1): The revision plan does not convince me that the revised paper will be sufficiently readable. Overall, the motivation and method sections lack necessary explanations and clear relationships. For example, simply introducing the concept of Shannon entropy does not aid in understanding Eq. (1). You should provide a detailed explanation of why topology uncertainty is relevant to your motivation.
Q2(2): When you refer to "EDEN" in Table 1, are you indicating the lightweight version? Although the authors discuss methods to accelerate EDEN in Section 3.4, these methods seem rough, and it is unclear whether they will impact the model's performance.
Q3: Many graph embedding methods perform well on directed graphs, such as:
[1] Scalable Graph Embeddings via Sparse Transpose Proximities.
[2] ELTRA: An Embedding Method based on Learning-to-Rank to Preserve Asymmetric Information in Directed Graphs.
I recommend that the authors conduct a comprehensive comparison with SOTA graph embedding methods.
Minor New Concerns:
The method "NSTE" is cited alongside the paper "Directed Graph Auto-Encoders". Is this a mistake?
We sincerely appreciate your thorough review and apologize for any confusion. Please allow us to provide the following supplementary clarification, which we hope will strengthen your confidence in our work.
Q1
Why and What is Data-centric Graph ML
Conventional model-centric approaches prioritize GNN design complexity while neglecting the data itself. Therefore, we advocate treating graphs as knowledge sources, where higher-order graph patterns—e.g., homophily, heterophily, motifs, and communities—can provide topology- and semantic-aware insights in model design and computation.
Existing but Insufficient Studies
- Dir-GNN [LoG 2024] – Utilizes directed edges to design directed message passing augmented with additional trainable parameters.
- ADPA [ICDE 2024] – Utilizes directed neighborhood connectivity to obtain graph operators. These operators are used to obtain directed propagated messages and design trainable message aggregators.
- MAP [WWW 2025] – Utilizes node degrees and directed label patterns to improve complex-domain message passing by optimizing the Magnetic Laplacian for each edge (determines the strength of edge directionality).
Q2(1):
We acknowledge your concerns and recognize that this point involves an abstract concept that may not be immediately intuitive. Nonetheless, we are committed to presenting it clearly and accessible. To that end, we offer the following clarification, along with further revision plans.
Topology Uncertainty and Our Motivation
The core of our work is grounded in the observation that real-world relational data often contain structural noise from dynamic evolution—e.g., spurious edges and missing links—resulting in topology uncertainty [Nature Physics 2012].
We quantify this uncertainty using structural entropy, defined as the information needed to describe the graph via random walks and communities [TOIT 2016]. By minimizing this metric, we construct HKT to reorganize node affiliations, thereby effectively denoising the graph.
Revision Plan
(1) In the appendix, we will formalize topology uncertainty through structural entropy and highlight that minimizing structural entropy reduces graph noise and reveals data knowledge. Moreover, we will provide a real-world case study (citation network).
(2) In Sec 3.1, we will emphasize topology uncertainty is relevant to our motivation and justify structural entropy minimization can filter graph noise while revealing data knowledge by HKT.
(3) In Sec 3.1-3.2, we will refine the neural mutual information estimator to emphasize its capacity for serving a dual role as both a predictive module and a further graph denoising mechanism driven by HKT.
Q2(2):
Experimental Results
All reported results are obtained by lightweight EDEN, ensuring fair comparison.
Revision Plan for Lightweight EDEN Implementation Details
Para. 1: We will provide a detailed algorithm based on a Monte Carlo approach in the appendix, along with additional theoretical motivation and implementation details (Reviewer RNnj Q2).
Para. 2: Based on Eq. (7–9), we will formalize the equation of class-specific prototypes and provide an intuitive explanation in the appendix.
Para. 3: We will expand the description of the computation-friendly directed graph learning function and provide its formal equations.
Performance Impact
In response to your valuable comments, we have conducted additional experiments as follows:
| Node-C | Tolokers (Acc) | Tolokers (Time) | Rating (Acc) | Rating (Time) |
|---|---|---|---|---|
| EDEN (Ori.) | 82.1±0.3 | 240.8±6.5 | 46.8±0.3 | 132.3±4.9 |
| EDEN (Light.) | 81.3±0.2 | 72.1±3.6 | 46.3±0.4 | 57.6±2.2 |
Q3
Motivated by your constructive suggestion, we have conducted the following experiments.
| CoraML (N-C) | CoraML (L-C) | WikiCS (N-C) | WikiCS (L-C) | Arxiv (N-C) | Arxiv (L-C) | |
|---|---|---|---|---|---|---|
| STRAP [KDD 2019] | 80.8±0.3 | 71.3±0.4 | 77.5±0.4 | 78.7±0.4 | 66.9±0.5 | 75.4±0.4 |
| ELTRA [CIKM 2023] | 82.2±0.6 | 72.2±0.5 | 78.6±0.4 | 80.4±0.5 | 67.6±0.6 | 76.9±0.5 |
| PSNE [CIKM 2024] | 81.7±0.4 | 72.8±0.3 | 78.2±0.2 | 79.8±0.4 | 68.3±0.4 | 77.8±0.4 |
| EDEN | 84.6±0.5 | 75.2±0.5 | 81.4±0.3 | 83.5±0.2 | 69.7±0.3 | 80.2±0.2 |
Graph embedding methods do not explicitly leverage label supervision, resulting in lower performance compared to the semi-supervised baselines. However, their effectiveness under limited label settings underscores their utility. Therefore, we will provide a detailed comparison in the revised submission.
Minor New Concerns
We apologize for the confusion. This cited paper presents an autoencoder framework, differing from the semi-supervised paradigm. Thus, we only apply its encoder component, as described in Section "Neural Source and Target Encodings" (NSTE) in the original paper.
This paper focuses on data-efficient representation learning for directed graphs and presents a novel online knowledge distillation framework based on a hierarchical tree structure. Leveraging this framework, the authors introduce EDEN, a new method that can be employed as a plug-and-play module to improve performance of existing directed graph neural networks or as an entirely new neural network architecture. The paper elucidates the novelty of the proposed knowledge distillation framework for graph learning and the effectiveness of the EDEN method through detailed textual presentations, charts, and relevant theoretical proofs, in a manner that is both reader-friendly and comprehensible. Moreover, extensive experiments have been conducted to validate the practicality of the proposed method.
给作者的问题
None.
论据与证据
This work explicitly highlights in Section 1: Introduction that existing directed graph neural network methods fail to fully leverage the rich knowledge embedded in data, thereby imposing a limited upper bound on performance. Building on this observation, the authors conduct an in-depth analysis of the issue. In Section 2: Preliminaries, they define knowledge from two dimensions—semantic and topological—using a hierarchical tree structure. This forms the basis for constructing a novel online knowledge distillation framework that paves the way for data-efficient graph learning. The evidence supporting the framework is meticulously presented and thoroughly substantiated in Section 3: Methodology, Section 4: Experiments, and Appendix.
方法与评估标准
For the Methods, the authors provide visual representations through Figure 1 and Figure 2, which facilitate reader comprehension. Additionally, they offer a granular exposition of the methods, relying on the formal problem definitions, equations, presentations, and theoretical proofs detailed in the Preliminaries and Methodology sections. Regarding the Evaluation Criteria, the authors elaborate on the experimental settings in the Experiments section, as demonstrated in Appendix A.10–A.13.
理论论述
In this paper, the authors identify relevant theoretical foundations and conduct a rigorous theoretical analysis. For instance, in the Preliminaries section, the concept of the hierarchical knowledge tree, inspired by hierarchical coding systems, is introduced. The related literature is comprehensively cited, and a clear problem definition is provided for ease of understanding. Furthermore, in the Methodology section, the authors perform a detailed theoretical analysis of the hierarchical online data knowledge distillation process from the perspective of neural mutual information estimation, thereby demonstrating its feasibility.
实验设计与分析
The experimental designs and analyses in this work are both comprehensive and sound. Specifically, the performance evaluations across 14 datasets of varying scales and four downstream tasks demonstrate the superiority of this work. Building on this foundation, the ablation studies and sensitivity analyses are sufficiently thorough, providing robust evidence for the contributions of each module. Additionally, the authors leverage the appendix to offer supplementary convergence analyses, further enhancing the thoroughness and completeness of the experimental work.
补充材料
The authors provide the code in the supplementary materials, which further completes this work.
与现有文献的关系
This work, taking directed graph representation learning as an example, proposes a universal data-efficient online knowledge distillation framework for graph learning. This is an inspiring attempt. Subsequent research in related fields can build upon the hierarchical knowledge tree defined by the authors to make more attempts and promote the vigorous development of data-centric graph learning.
遗漏的重要参考文献
None.
其他优缺点
Strengths:
-
Novel and Well-Motivated: The paper focuses on data-efficient graph learning, an issue of critical importance for the future development of the graph learning community. Building on this motivation, the authors introduce a novel framework for online data distillation using a hierarchical tree structure. This approach is highly intuitive and natural, leveraging the inherent properties of tree structures in a way that aligns well with human understanding. I believe this framework will be highly beneficial for future research in this area.
-
Complete Methodology and Robust Theory: The authors provide a detailed exposition of the proposed online data distillation framework and the new method built upon it. The presentation is clear and easy to follow, making the complex concepts accessible to readers. The methodology is further enriched with clear definitions and thorough theoretical analysis, enhancing the interpretability and credibility of the proposed approach.
-
Significant and Comprehensive Results: As a general method for data-efficient graph representation learning, the proposed EDEN method demonstrates substantial performance improvements when used as a plug-and-play module for existing methods. Additionally, it achieves state-of-the-art performance when employed as a standalone neural network architecture. The authors conduct extensive experiments across multiple datasets, downstream tasks, and backbone models, providing solid evidence of the method's effectiveness. The experimental results are robust and comprehensive, supporting the claims made in the paper.
-
Well-Presented: The authors make excellent use of figures and clear presentations to simplify complex concepts, making the paper easy to understand. I appreciate the effort made by the authors to ensure that the content is well-organized and clearly communicated.
Weaknesses:
- Enhanced Explanations for Accessibility: The authors provide detailed theoretical analysis in the methodology section, which is commendable. However, understanding these complex formulas may still be challenging for readers who are not well-versed in the field. Could the authors consider offering more intuitive descriptions or visual aids to lower the barrier to entry for a broader audience?
- Detailed Discussion of Future Directions: While the authors acknowledge the current limitations of their work, the discussion is somewhat brief. A more detailed exploration of promising future research directions would be valuable. Specifically, outlining potential avenues for achieving more data-efficient graph learning could provide valuable insights for the community.
- Thorough Review Needed: The authors are encouraged to conduct a comprehensive review of the entire manuscript to avoid potential typographical errors and minor mistakes. Ensuring the accuracy and clarity of the text will enhance the overall quality of the submission.
其他意见或建议
None.
W1: Enhanced Explanations for Accessibility
We appreciate your in-depth feedback and acknowledge that certain sections, particularly the formula interpretations, require background knowledge, which may pose challenges for readers. To improve readability, we will incorporate more intuitive explanations in the revision process, as suggested, ensuring greater accessibility for our audience.
Specifically, we plan to include the collection of annotations used for formulas in a tabular presentation to guide our readers in the main document. We will also read through the manuscript and add more intuitive explanations to display the functions and meaning of each formula to make them easy to comprehend.
We thank you again for your valuable comments to enhance the readability of our work.
W2: Detailed Discussion of Future Directions
We sincerely appreciate your suggestions and acknowledge that some content may be brief due to our focus on presenting methodological details within the space constraints of the main document.
We will carry the obligation to provide more valuable insights for further development of this field, and we plan to provide additional information either within the main document or in the appendix, with clear references for accessibility for readers.
Specifically, we intend to open a new discussion in the appendix for potential proposals to further enhance the efficiency of EDEN with theoretical support and valuable related works for inspiring interested researchers. We also plan to add a clear indication in Sec 3.4 to guide our readers to this particular section in the appendix. Thank you!
W3: Thorough Review Needed
Thank you for your notification. We will thoroughly review the manuscript to eliminate errors and ensure both accuracy and clarity of the content. More importantly, we will ensure that each formula is well interpreted for our readers with clearer annotations in the table for their convenience.
I commend the authors for their detailed revision plan, which addresses my concerns. After carefully reviewing other reviewers' feedback, I recommend expanding the discussion on computational efficiency optimizations and research motivations in the revised manuscript. The machine learning field is shifting from model-centric to data-centric approaches, and this work could provide valuable insights. Accordingly, I have raised my score and voted for acceptance.
Thank you very much for your positive feedback. Rest assured that we will meticulously follow your suggestions to enhance further the presentation of our paper in the revised version and address any potential concerns. We are pleased to know that our work has been recognized for its value in highlighting data-centric approaches within the graph ML community. Your endorsement means a great deal to us.
The author proposes a complex but effective method called EDEN, which tailored for the directed graph, specifically, it frist build a coarse-gradined Hierarchical Knowledge Tree (HKT), then, it refine the HKT with knowledge flow in the HKT. The method is widely adopted in the 14 graphs and the results valid the method's effectiveness.
给作者的问题
See above.
论据与证据
Claim 1: The author primarily discusses the limitations of Graph Neural Networks (GNNs) on undirected graphs, highlighting issues such as suboptimal data representations and inefficient learning at the model level.
Claim 2: Additionally, the author argues that existing methods fail to fully capture the potential correlations between directed graph topology and node profiles at the data level. To address this, the proposed approach enhances existing methods by incorporating directed graph knowledge.
Evidence 1: The author demonstrates that directed graphs provide richer and more diverse knowledge in terms of topology and node profiles, supporting this claim with case studies.
Evidence 2: Extensive empirical studies are conducted in the experimental section to validate the proposed method.
方法与评估标准
Yes.
理论论述
No, since I lack the background knowledge, I may not be able to check all the details of derivation.
实验设计与分析
Yes.
补充材料
Yes.
与现有文献的关系
I still question whether such computationally expensive operations are truly necessary.
遗漏的重要参考文献
I am not sure.
其他优缺点
My main concern is the neccessary of such complex tree design and refine processes, and finally use the "Monte Carlo" method to enhance the efficiency.
- Could you give a real case that represents the so-called digraph data knowledge? Does it really exists?
其他意见或建议
- The figure 2 is hard to distinguish the words and what you want to express, and also the Figure 1 (quite small)
伦理审查问题
No.
Due to the word limit imposed by the new regulations of ICML 2025 rebuttal, we have not provided detailed references, but we will gladly supply them in our subsequent discussions if needed.
Q1: Relation To Broader Scientific Literature
I sincerely apologize for any confusion that may have been caused. Please allow us to clarify our motivations and methodology.
Motivation
Recent studies have shown that considering the directionality of edges provides a novel perspective to address the long-standing heterogeneity challenges in graph ML and achieves impressive performance. These studies collectively emphasize the necessity of extracting implicit data knowledge from directed graphs to drive the model’s learning
- Dir-GNN [LoG 2024] facilitates homophilous aggregation by directed edges.
- ADPA [ICDE 2024] derives high-order propagation operators from directed neighboring connectivity.
- MAP [WWW 2025] optimizes the strength of directed edges by node degrees and directed label patterns.
Despite their effectiveness, the data-centric concept remains unformulated, and optimization perspectives vary. Therefore, we aim to propose a unified framework, which leverages Tree as the carriers of data knowledge and achieves learning. Inspired by Reviewer zpYj Q3, we have conducted additional experiments, which illustrate the superiority of our framework.
Complexity
We acknowledge that the current EDEN may limit scalability. However, this is somewhat inevitable in the early stages of establishing a unified framework. We kindly ask for your understanding in this regard. That being said, we have made practical efforts to simplify EDEN. In Sec 3.4, we introduced a lightweight EDEN, derived from three orthogonal perspectives, and successfully applied it to million-level Arxiv and WikiTalk. Table 1 and Figure 3 indicate that our method demonstrates a certain efficiency advantage.
We also recognize the need for further efficiency improvements. If possible, we plan to further discuss promising optimization directions in the appendix. For instance, we will explore the possibility of integrating the HKT construction with graph partitioning techniques to enable parallel processing.
We trust that the above response will address your concerns and enhance your confidence in our manuscript. If you require additional clarification, please do not hesitate to contact me.
Q2: Other Strengths And Weaknesses
Why Tree
(1) Graph Mining: We adopt the tree structure for its intuitiveness in revealing hierarchical knowledge within structured data. This concept has been theoretically substantiated in prior studies [STOC 1988, TOIT 2016, ICML 2024]. We kindly suggest that you refer to Sec 2.2 for a detailed elaboration.
(2) Graph Learning: In some recent studies [NeurIPS 2021, NeurIPS 2024, ICLR 2024], researchers have abstracted the core of GNN as the tree-based message passing. They have theoretically demonstrated its effectiveness.
Based on these insightful studies, the motivation for adopting a tree-based framework is its versatility and significant potential for data-centric graph learning. Additionally, we optimize the efficiency of this framework in Sec 3.4, rendering it comparable to or even superior to the best baseline shown in Sec 4.
Why Monte Carlo
This technique has recently been widely employed in tree-based numerical computation [SIGMOD 2023, WWW 2024]. In this paper, we construct HKT by minimizing structural entropy via a greedy algorithm, which iteratively selects optimal sets for leaf Combining and Lifting. By integrating the Monte Carlo method, we approximate solutions through random sampling, avoiding exhaustive enumeration of all branches for subsequent greedy selection.
Real Case
(1) Research Perspective: As previously mentioned in our response to Q1, the data knowledge derived from directed graphs provides key insights that improve model computation and yield superior performance.
(2) Practical Perspective: Taking citation networks with interdisciplinary (e.g., AI4Science) as an example. In this context, at the first (leaf) level of HKT, we use edge directions to model intra- and inter-field citations. Based on this, HKT endows us with the ability to organize hierarchical knowledge. Specifically, at higher levels of the tree, we can interpret this as progressively more abstract concepts, such as research groups, institutes, and universities. Explicitly representing these concepts can be regarded as revealing data knowledge and providing insights to enhance tree-based graph learning.
We trust that our response effectively addresses any potential concerns. Additionally, we plan to enrich Sec 1, 2.2, and 3.4, and the appendix, with more detailed motivation, background, and intuitive interpretations of the technologies.
Q3: Other Comments Or Suggestions
We sincerely appreciate your suggestion and plan to reformat the figure to a vertical layout to enhance clarity.
This paper introduces EDEN, an entropy-driven approach for data-centric directed graph learning. The framework constructs a Hierarchical Knowledge Tree (HKT) to distill structural and semantic knowledge from directed graphs, serving as a plug-and-play module to enhance existing graph neural networks. EDEN shows strong performance improvements on various directed graph datasets and tasks.
Strengths:
- Novelty and Clear Motivation: It introduces a novel online knowledge distillation framework for directed graphs, leveraging hierarchical knowledge trees and mutual information to enhance data efficiency.
- Complete Methodology and Theory: The paper provides detailed theoretical analysis and clear definitions, making the approach interpretable and credible.
- Comprehensive Results: Extensive experiments on diverse datasets and tasks show strong empirical improvements.
Weaknesses:
- Complexity and Scalability Concerns: While a lightweight version exists, the full framework’s computational cost may limit large-scale applicability.
- Readability and Clarity: Some technical sections lack intuitive explanations, making the method harder to follow.
The work is significant in the field of data-centric graph learning, offering a new perspective for directed graph learning. Key concerns of reviews included clarity of motivation, baseline comparisons, and scalability, which the authors addressed in rebuttal with additional experiments and explanations.