GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning

We thank the reviewer for the positive and constructive feedback. We are encouraged that the reviewer acknowledges the novelty and thoroughness of our method, the rigor of our ablation studies, and the balance between theoretical grounding and empirical validation. Below, we address the raised concerns in detail.

---

Q1: The paper includes too many techniques, making the presentation feel unfocused and blurring the line between original contributions and existing works.

A1: We thank the reviewer for this insightful comment. GRAVER indeed designs multiple components, but all are organized around a unified goal: building and reusing a transferable graph vocabulary to support robust graph foundation models. To avoid possible confusion, we clarify the core contributions of GRAVER as follows:

• Generative Graph Vocabulary Construction. GRAVER proposes a new paradigm where class-specific subgraph patterns are extracted and disentangled into semantic primitives, forming a reusable vocabulary of graph tokens. This vocabulary captures transferable structural and semantic features across domains.
• Vocabulary-Guided Prompt Augmentation. Rather than relying on raw node features or full-graph encoding, GRAVER constructs prompts by selectively assembling vocabulary tokens based on task-specific prototypes. This design improves sample efficiency and enhances generalization in few-shot settings.
• Hierarchical Routing with MoE-CoE. To adaptively reuse vocabulary tokens across domains and classes, GRAVER introduces a two-level routing mechanism. The first level selects domain-relevant experts (MoE), and the second level refines the token composition within each class (CoE), enhancing both robustness and specificity.

Other components, such as feature enhancement or retrieval strategies, serve as auxiliary modules to support this vocabulary-centric workflow and are aligned with prior literature, which also have been properly cited. In the revision, we will emphasize the above core contributions more clearly in the method overview and contribution summary.

---

Q2: The paper lacks discussion of traditional few-shot graph learning methods.

A2: Thank the reviewer for the thoughtful suggestion. We emphasize that GRAVER targets a different goal:

• Few-shot fine-tuning is a standard setting in GFM literature. We adopt the few-shot fine-tuning setting following recent GFM studies, where the goal is to assess cross-domain generalization with limited task-specific supervision. This is a widely used protocol for evaluating foundation models’ adaptability in both vision and language domains, and increasingly in graph learning.
• GRAVER is fundamentally distinct from traditional few-shot graph learning. The objective of few-shot graph learning works is to design models specifically optimized for task-level generalization under few-shot constraints, often via episodic training or meta-learning. In contrast, GRAVER is not designed for task-level few-shot learning, but rather as a foundation model that can generalize across graph domains and tasks, with few-shot fine-tuning merely serving as a test modality.
• From the GFM perspective, zero-shot generalization is the ultimate goal. Especially with the integration of textual and LLM-enhanced knowledge, the focus of GFM is shifting toward zero-shot adaptability across unseen domains. In this context, few-shot fine-tuning serves as a middle ground to evaluate adaptability under minimal supervision, but is not the design target of GRAVER.

Also, we acknowledge that the current draft lacks an explicit discussion of traditional few-shot graph learning works, we will include the following subsection in the revised manuscript:

Related Work: Few-shot Graph Learning. (Additional)
Few-shot graph learning addresses the challenge of learning on graphs with extremely limited labeled data per class, and has been widely studied in both node classification and graph classification. Inspired by meta-learning successes in vision and text domains [1], early studies adapted meta-learning to graphs but encountered unique challenges due to their relational and non-Euclidean structure. To address this, researchers developed graph-specific meta-learning frameworks based on Graph Neural Networks (GNNs) for fast adaptation to new classes with minimal supervision [1].
Few-shot node classification. Early methods combined GNNs with optimization-based meta-learning. Meta-GNN [1] incorporated Model-Agnostic Meta-Learning (MAML), training on simulated tasks to learn transferable initializations for novel classes. Metric-based alternatives such as Graph Prototypical Networks (GPN) [2] created class prototypes in embedding space to classify query nodes efficiently. G-Meta [3] enhanced this by building local subgraphs and using second-order optimization to capture task-specific structure. Meta-GPS [4] introduced learnable scaling and shifting transformations to refine prototype representations, further improving task adaptability under a MAML framework. To cope with limited task diversity, SMILE [5] proposed dual-level mixup within and across tasks to synthesize new nodes and even novel tasks, thereby alleviating overfitting and improving generalization.
Few-shot graph classification. Optimization-based methods have proven effective. Adaptive-Step Graph Meta-Learner [6] dynamically adjusted the gradient steps in the inner loop through a trainable controller, capturing discriminative substructures of unseen classes more effectively. FAITH [7] modeled inter-task relationships via a hierarchical task graph, organizing tasks, classes, and graphs into a three-level structure, and selecting related tasks using a loss-guided sampling strategy. These methods demonstrated strong performance, particularly when labeled training graphs were sparse.
Recent advances have also explored unsupervised strategies. SMART [8] eliminates meta-learning entirely, pre-training a GNN on large-scale unlabeled graphs using self-supervised objectives and a novel graph mixup technique. During few-shot adaptation, it applies prompt tuning by freezing the encoder and updating a small set of task-specific parameters. This lightweight adaptation yields strong results, often surpassing supervised meta-learners, and highlights the promise of leveraging large unlabeled datasets for few-shot graph learning.

---

Q3: The pre-training datasets are relatively small. Evaluating on larger graphs with hundreds of millions of nodes would better demonstrate real-world applicability.

A3: We appreciate the reviewer’s suggestion regarding scalability. We agree that scaling graph foundation models (GFMs) to massive real-world graphs is an important direction. However, we intend to clarify several contextual points:

• GFM research is still in its formative stage. The community has not yet converged on standardized benchmarks, architecture formations, or evaluation protocols for GFMs. Current efforts primarily focus on understanding generalization across domains and tasks, especially in few-shot and zero-shot settings. To our knowledge, no existing GFM studies have evaluated models at the scale of hundreds of millions of nodes.
• Our dataset scale already surpasses prior GFM works. Compared to recent GFM literatures [9-13], our pre-training corpus reaches over 250,000 nodes and 3.5 million edges across multiple graph datasets. This places GRAVER among the largest-scale settings used for GFM pre-training in existing academic works.
• Scalability remains a core challenge under constrained resources. As detailed in Appendix E.3, our experiments were conducted on an 80GB A100 GPU. Scaling to graphs in the hundreds of millions is beyond the scope of current infrastructure, and remains a critical open challenge in graph learning community. That said, we view this as a vital future direction for GFM deployment in real-world applications.

We will make these clarifications explicit in the revised manuscript.

---

We sincerely thank the reviewer for the constructive comments. We have clarified the core contributions, distinguished our setting from traditional few-shot methods, and discussed scalability concerns. We hope these responses address the concerns and would greatly appreciate a score revision!

---

Refs:
[1] Meta-GNN: On few-shot node classification in graph meta-learning, CIKM 2019.
[2] Graph prototypical networks for few-shot learning on attributed networks, CIKM 2020.
[3] Graph meta learning via local subgraphs, NeurIPS 2020.
[4] Few-shot node classification on attributed networks with graph meta-learning, SIGIR 2022.
[5] Dual-level mixup for graph few-shot learning with fewer tasks, WWW 2025.
[6] Adaptive-step graph meta-learner for few-shot graph classification, CIKM 2020.
[7] FAITH: Few-shot graph classification with hierarchical task graphs, IJCAI 2022.
[8] A simple but effective approach for unsupervised few-shot graph classification, WWW 2024.
[9] ZeroG: Investigating cross-dataset zero-shot transferability in graphs, KDD 2024.
[10] One for All: Towards training one graph model for all classification tasks, ICLR 2024.
[11] How much can transfer? BRIDGE: Bounded multi-domain graph foundation model with generalization guarantees, ICML 2025.
[12] AutoGFM: Automated graph foundation model with adaptive architecture customization, ICML 2025.
[13] Multi-domain graph foundation models: Robust knowledge transfer via topology alignment, ICML 2025.

We sincerely thank the reviewer for recognizing the importance of fine-tuning instability in GFMs, and for appreciating our use of ego‑graph disentanglement, graphon modeling, and both theoretical and empirical validations. Our detailed response is as follows.

---

Q1: The multi-component design of GRAVER may hinder reproducibility and real-world deployment.

A1: We acknowledge the reviewer’s concern and clarify that GRAVER is structured as a modular and reproducible pipeline. Each component is lightweight, well-isolated, and intentionally designed to support extensibility:

• LLM-enhanced features are derived using open-source toolchains. we retrieve raw node-associated texts and process them via pre-trained models. No fine-tuning or training of the LLM is required, ensuring ease of use.
• Ego-graph disentanglement is a non-learnable and deterministic decomposition process that extracts $K$ structural factors per node via topological similarity and structural entropy, without any training or tuning overhead.
• Graphon estimation is conducted on a small number of factor-aligned subgraphs using empirical neighborhood averaging, making it stateless and reusable. No complex optimization or learning is involved.
• MoE-CoE routing involves only two small learnable vectors per task, applied during fine-tuning. The design is deliberately simple and plug-in compatible with transformer decoders.

Furthermore, we emphasize that all pre-training and vocabulary generation are conducted offline, making fine-tuning fast, memory-efficient, and readily deployable in practice. Our released code includes full scripts and default settings for all modules to ensure reproducibility.

---

Q2: The generalizability of vocabularies across domains is unclear.

A2: Thank you for raising this important point. Our vocabularies are domain-specific during pre-training, but become reusable across tasks within that domain during fine-tuning. This design aligns with GRAVER's modular "pretrain-then-finetune" paradigm. Each domain undergoes preprocessing to construct a vocabulary using disentangled ego-graphs and graphon modeling. Once generated, these vocabulary tokens are shared across all tasks in that domain, including zero-shot and cross-dataset scenarios, as demonstrated throughout our experimental settings (Section 5.1~5.4). This approach inherently validates that our vocabularies, though initially domain-specific, possess strong reusability and generalization capacity across diverse downstream tasks.

Furthermore, to confirm that the routing network indeed utilizes domain-specific vocabularies effectively, we illustrate the routing weights of MoE layer under cross-dataset and cross-domain settings. Results:

In the cross-dataset setting, the routing weights concentrate on Cora (0.44) and PubMed (0.41) (datasets with similar structures), while assigning much lower weights to ogbn-Home and Wiki-CS. This demonstrates how the routing network adaptively prioritizes vocabularies from structurally aligned sources.

In contrast, in the cross-domain case targeting ogbn-Home, a structurally distinct dataset, the routing becomes more balanced across all domains (0.19~0.36), indicating no clear structural match. This confirms that GRAVER's routing mechanism can automatically adjust based on the structural divergence between source and target domains, drawing knowledge from multiple diverse sources when structural alignment is weak.

The results show that MoE adaptively selects vocabulary tokens most relevant to the target domain or task, demonstrating that GRAVER not only supports vocabulary reuse, but also dynamically mitigates potential mismatch through informed routing. We will clarify this distinction more explicitly in the revised version.

---

Q3: Theoretical bounds in the paper remain abstract and are not clearly connected to model design or practical interpretation.

A3: We appreciate this feedback and agree that theoretical insights are most valuable when they inform practical design. In GRAVER, the propositions serve as theoretical justifications for key architectural decisions:

• Proposition 1 provides a bound on semantic discrepancy across vocabulary tokens, showing that the bound is governed by the number of disentangled semantic factors ($K$) and the mutual information regularizer $\mathcal{R}_{\text{MI}}$. This directly motivates the use of semantic information preservation (SIP) during pretraining to ensure well-separated vocabulary channels, which improves transferability. The corresponding empirical observation: removing SIP leads to a performance drop, aligns with this bound.
• Proposition 2 provides convergence guarantees for the graphon-based generation of vocabulary tokens, ensuring that the learned distributions faithfully capture ego-graph structures. This supports the use of graphon modeling over more black-box neural generators (e.g., VAEs or diffusion), especially under low-data regimes.

To make these links clearer, we have now added direct references from our model design choices back to the propositions in the appendix. While the bounds may appear abstract, they serve to constrain the design space and ensure that each component: disentanglement, generation, routing, has theoretical grounding, guiding our practical implementation.

---

Q4: Why use graphons instead of neural generative models (GraphVAEs, diffusion, autoregressive)? Any comparisons in terms of transferability and fidelity?

A4: We chose graphons as the generative backbone for vocabulary construction due to their nonparametric, data-efficient nature and theoretical interpretability. Unlike GraphVAEs or diffusion-based generators, which require large training sets to generalize well and often involve high variance in generation, graphons allow explicit modeling of graph distributional structure with provable convergence (see Proposition 2). This makes them particularly well-suited for few-shot and zero-shot settings, where structural prior needs to be distilled from limited examples.

While we did not perform exhaustive comparisons with all generative baselines, we emphasize that GRAVER’s vocabulary is designed not for raw graph generation, but for producing task-adaptive, transferable subgraphs under domain shift. Graphon-based generation gives us explicit control over structural smoothness and heterogeneity, aligning well with our routing and prompt integration mechanisms.

---

Q5: Are vocabularies general or domain-specific? Can vocabularies generated in one domain generalize to structurally different ones?

A5: Please refer to our response to Q2.

In short, the vocabularies in GRAVER are fundamentally domain-specific, as they are constructed from disentangled ego-graphs and graphons fitted to the local structures observed in each domain during pre-training. However, once pre-trained, these vocabularies are reusable across tasks and datasets within the same domain, supporting efficient few-shot adaptation.

Importantly, all our cross-dataset and cross-domain experiments (Section 5.2) are conducted under the vocabulary reuse setting: we do not re-train or regenerate vocabularies in the target domain. This empirically verifies that the pre-trained vocabularies, though learned within source domains, retain sufficient generality to support downstream adaptation in unseen domains.

The additional results in A2 show that the model dynamically reweights vocabulary channels depending on the domain and task, indicating effective domain-aware vocabulary reuse. While we agree that cross-domain reuse metrics are an interesting direction, our current results already demonstrate strong generalization behavior, suggesting that the learned vocabularies, while structurally grounded in source domains, are transferable enough to support adaptation to distinct domains.

---

Q6: What happens if the vocabulary is misaligned or noisy? Can the MoE-CoE mechanism effectively filter out mismatched subgraphs?

A6: GRAVER is explicitly designed to handle vocabulary mismatch through its hierarchical routing architecture. The MoE layer operates at the inter-domain level, selecting relevant vocabulary sets based on the target domain’s global characteristics, while the CoE layer works intra-domain, selecting the most semantically aligned subgraphs within a vocabulary set. This two-level gating mechanism helps the model dynamically downweight irrelevant or noisy patterns, even when vocabulary and target domain structures are not perfectly aligned.

In essence, while vocabulary mismatch may introduce noise, the MoE-CoE mechanism provides resilience through selective routing, ensuring that only the most compatible augmentations contribute to downstream prediction.

---

We sincerely thank the reviewer for the thoughtful and constructive feedback. We hope our responses have addressed all concerns.

We sincerely thank the reviewer for the positive and detailed feedback. We are especially encouraged by the recognition of our augmentation strategy, the theoretical depth behind ego-graph disentanglement and vocabulary generation, and the systematic evaluation. Below, we address each of the concerns raised.

---

Q1: Risk of semantic-structural mismatch due to using LLM for semantic alignment while structure is modeled independently.

A1: We appreciate the reviewer’s thoughtful observation. GRAVER explicitly addresses this issue through both the design of the semantic enhancement pipeline and the separation of routing versus content generation:

• LLM-enhanced semantics are strictly derived from retrieved raw texts. As shown in the illustrative example (Figure 2), our LLM is applied to raw titles and abstracts retrieved for each node to generate class-aware but semantically consistent descriptions. We do not use hallucinated or externally constructed semantics. This ensures that enhanced text does not drift from the original node meaning, avoiding the risk of introducing misleading semantics.
• Textual semantics guide vocabulary selection, not structural generation. GRAVER’s vocabulary is generated entirely from disentangled ego-graphs via graphon modeling (Section 4.1), independent of any semantic signal. The LLM-enhanced embeddings are only used to compute attention over these pre-generated subgraphs (Section 4.2). This clear decoupling ensures that even if semantic descriptions are noisy, the underlying graph augmentations remain structurally valid and transferable.
• Fallback routing ensures robustness in low-quality text scenarios. In cases where the semantic input is missing, ambiguous, or misleading, GRAVER falls back to CoE routing, which leverages intra-class structural patterns for vocabulary selection. This fallback mechanism is empirically validated in Appendix D.3.4, where routing performance remains stable under partial or ablated semantic inputs.

Taken together, GRAVER is robust to semantic-structural mismatch by: (1) grounding semantic enhancement in original text, (2) separating routing from graph construction, and (3) using structure-only fallback when necessary. We will make these points clearer in Section 4.2 and include the illustrative example to aid understanding.

---

Q2: Missing pre-training cost analysis. While fine-tuning is efficient, the cost of multi-domain pre-training and vocabulary construction is unclear.

A2: We appreciate the reviewer’s concern. While GRAVER focuses on improving fine-tuning efficiency, we agree that understanding pre-training complexity is important, and we clarify the following:

• We have explicitly analyzed the computational complexity of pre-training. In Appendix B, we provide a detailed breakdown of the time and space complexity for both the pre-training stage (Algorithm 1) and fine-tuning stage (Algorithm 2). The pre-training involves subgraph disentanglement and conditional graphon fitting, both of which are lightweight and performed once per source domain. The complexity scales with the number of source domains and the size of tokenized subgraphs, which remain small.
• Efficiency at pre-training is not the core objective of GRAVER. Like other Graph Foundation Models [1-5] and large models for languages and vision, the community has reached a consensus that the pre-training phase is a one-time cost. Our focus is on designing a modular framework that enables robust and efficient fine-tuning, which is the real bottleneck in the real-world scenarios.
• Practicality comes from reusable vocabulary and high adaptation efficiency. Once pre-trained, GRAVER reuses a shared vocabulary across domains and tasks. This allows rapid downstream tuning, as demonstrated in our speedup results (Section 5.4, Figure 6). In settings with numerous or evolving tasks, this efficiency gain is far more critical than saving pre-training time.

We will clarify this perspective and move the complexity analysis reference to the main text for better visibility.

---

Q3: Lack of measurement of structural heterogeneity across domains. The paper does not clarify whether domains differ significantly in structure.

A3: We appreciate the reviewer’s thoughtful comment. GRAVER is designed precisely to address the challenge of structural heterogeneity across domains, which is a core motivation for its generative vocabulary design and hierarchical routing mechanism.

• Clear domain-level structural differences exist across datasets. We categorize datasets into three domains: Citation, Co-purchase, and Web Link. Although this categorization may appear intuitive, we reinforce it using measurable structural statistics, including average degree, node homophily, and class count. These are reported below (also detailed in Table D.1 and Appendix D):

These statistics show clear structural divergence:

(1) Citation Domain: tend to be sparser, with lower average degree and moderate homophily.

(2) Co-purchase Domain: highly dense, with higher degrees and homophily, and reflect different linking principles (co-buying patterns rather than citations).

(3) Web Link Domain: its edges are based on hyperlink co-occurrence, and the structural connectivity is medium-dense with flatter hierarchy.

• Structural heterogeneity is central to the motivation of GRAVER. Unlike approaches that rely on matching structures across domains, GRAVER embraces heterogeneity by using generative vocabulary subgraphs as structure-agnostic, transferable prompts. This avoids brittle assumptions about topology alignment. Our hierarchical routing (MoE + CoE) learns to dynamically select suitable vocabularies depending on local task context, instead of forcing hard alignment between structurally divergent graphs.
• GRAVER’s empirical success confirms its robustness to structural gap. Our results in cross-domain and zero-shot settings (e.g., PubMed → ogbn-Tech, or WikiCS → ogbn-arXiv) demonstrate that the model can transfer knowledge across domains with substantial topological discrepancy. This confirms that the design does not exploit structural similarity, but rather overcomes topological mismatch through prompt generalization and adaptation.

We hope this response clarifies the structural heterogeneity across domains and strengthens the justification for the design and evaluation protocol of GRAVER.

---

Q4: Potential concern about semantic-structural mismatch when using LLM-enhanced features.

A4: Thank you for raising this concern. GRAVER is explicitly designed to mitigate semantic-structural mismatch through a decoupled architecture: we retrieve raw node-associated texts and encode them with a pre-trained LLM to ensure semantic prompts are faithful to node attributes, while structural augmentation is separately modeled via disentangled ego-graphs. The MoE-CoE routing maintains this separation by handling semantic and structural cues independently. As shown in Section 5.5 (Figure 7), GRAVER remains robust under weak textual alignment and benefits further from high-quality semantics, confirming the effectiveness of this design.

---

Q5: How the MoE-CoE routing mechanism responds to structural heterogeneity across domains.

A5: GRAVER is designed to explicitly address domain-level structural heterogeneity through its hierarchical routing mechanism: the MoE layer assigns weights across domains, while the CoE layer routes over subgraph prototypes within each domain. To evaluate how routing behavior changes with structural divergence, we illustrate the routing weights of MoE layer under cross-dataset and cross-domain settings. Results:

In the cross-dataset setting, the routing weights concentrate on Cora (0.44) and PubMed (0.41) (datasets with similar structures), while assigning much lower weights to ogbn-Home and Wiki-CS. This reflects MoE’s preference for structurally aligned sources.

In contrast, in the cross-domain case targeting ogbn-Home, a structurally distinct dataset, the routing becomes more balanced across all domains (0.19~0.36), indicating no clear structural match. This suggests GRAVER aggregates knowledge from diverse sources to support adaptation when structural alignment is weak.

Overall, the results verify that GRAVER’s MoE routing mechanism adapts to structural heterogeneity by emphasizing aligned sources or diversifying when necessary.

---

We sincerely thank the reviewer again for the thoughtful feedback. We hope our clarifications have addressed all concerns.

---

Refs:
[1] ZeroG: Investigating cross-dataset zero-shot transferability in graphs, KDD 2024.
[2] One for All: Towards training one graph model for all classification tasks, ICLR 2024.
[3] How much can transfer? BRIDGE: Bounded multi-domain graph foundation model with generalization guarantees, ICML 2025.
[4] AutoGFM: Automated graph foundation model with adaptive architecture customization, ICML 2025.
[5] Multi-domain graph foundation models: Robust knowledge transfer via topology alignment, ICML 2025.

We sincerely thank the reviewer for their appreciation of our paper’s clear structure, well-grounded motivation, and especially the novelty of the graphon-based vocabulary generation. Our detailed responses are as follows.

---

Q1: The performance improvements may mainly stem from LLM-enhanced features.

A1: We thank the reviewer for raising this important concern.

• Feature-enhancement was applied equally to all baselines. To ensure a fair comparison, we reconfirm that we used the same LLM-based node feature enhancement (as described in Eq. (1), lines 120~126) consistently across all datasets evaluated on all baselines. This ensures that performance gains are attributable to GRAVER’s design, not differences in feature processing.
• Using LLM-enhanced features is a widely-adopted practice in recent GFM studies. Recent GFM works [1-3] (such as ZeroG [1], cited in line 125 where the LLM-enhanced tricks are introduced) regularly use LLMs to enhance node features on text-attributed graphs. These works differ in how LLMs are integrated, but all rely on LLMs’ generalization strength. In our case, this enhancement serves as a standard input processing step, not our major contribution.
• Retrieval of original texts contributes more significantly than LLM enhancement. Our ablation in Section 5.5 (Figure 7) separates the effects of raw-text retrieval and LLM enhancement. Results show that retrieving original node texts contributes more to performance, particularly for zero-shot adaptation. This highlights the importance of recovering full semantic context (often lost in original features obtained by, e.g., bag-of-words) and aligns with our goal of enabling robust cross-domain GFM.
• Additional ablation experiment (GRAVER without any textual enhancement). To further isolate GRAVER’s contribution, we introduce a new ablation using the original, non-enhanced node features (i.e., no text or LLM). Under this strict setting (1-shot node classification), GRAVER still outperforms strong baselines, confirming the effectiveness of its generative vocabulary and hierarchical routing modules. Results:

---

Q2: The MoE-CoE routing mechanism may be functionally simple, despite being presented with complex naming.

A2: Thank you for raising this insightful point. While MoE-CoE involves softmax-based weighting, it differs from that in several key ways:

• Selective Sparsity vs. Uniform Aggregation. Unlike typical softmax fusion that softly aggregates all inputs, MoE-CoE is designed to enforce selective sparsity. Through domain- and class-specific projections ($\mathbf{W}_ {\text{M}}$, $\mathbf{W}_{\text{C}}$ in Eq. (14)), it suppresses irrelevant sources, resulting in targeted and meaningful composition rather than uniform blending.
• Adaptive Routing Structure vs. Static Weighting. Traditional softmax fusion often uses static attention over fixed candidates. In contrast, MoE-CoE dynamically computes routing weights conditioned on each target support instance, enabling fine-grained, task-aware augmentation.
• Hierarchical Decision Logic vs. Single-layer Fusion. MoE-CoE explicitly separates domain-level and class-level routing, which makes the overall decision process more interpretable and modular, especially important when scaling to larger, noisier, or multi-task graph settings.

We chose the term “MoE-CoE” to reflect this structured design and to emphasize GRAVER’s role as a flexible framework, not just a standalone model. We will clarify these distinctions to avoid any misunderstanding.

---

Q3: Proposition 1 does not clearly support the claim that the upper bound of semantic discrepancy is governed by a specific vocabulary combination.

A3: We sincerely appreciate this insightful comment. Indeed, at first glance, the final form of the upper bound (Eq. (7) in Proposition 1) appears to depend solely on the number of disentangled factors ($K$). However, this simplified form is the final theoretical conclusion after several derivation steps. Importantly, during the derivation (Proof in Appendix C.2~C.3), the mutual information regularization term ($\mathcal{R}_{\text{MI}}$, Eq. (6)), induced by the Semantic Independence Promotion (SIP), explicitly appears and constrains the intermediate upper bound (Eq. (7), [a]).

Specifically, our key theoretical insight is that by constraining $\mathcal{R}_ {\text{MI}}$ during pre-training, the semantic discrepancy ($\Delta$) can be effectively bounded. We do not directly optimize this simplified upper bound during training. Instead, we incorporate the mutual information regularizer $\mathcal{R}_ {\text{MI}}$ directly into our pre-training objective (Eq. (8)), implicitly controlling the semantic discrepancy between vocabularies. In other words, the simplified form of the upper bound (depending only on $K$) is precisely as have already explicitly controlled vocabulary independence through SIP ($\mathcal{R}_{\text{MI}}$).

---

Q4: The logical connection between “disentangled ego-graphs” and “independent, minimal semantic units” is unclear.

A4: In Section 4.1, we explicitly decompose each node’s neighborhood (ego-graph) into multiple disentangled subgraphs, each capturing a distinct semantic aspect. Proposition 1 then theoretically demonstrates that if these subgraphs (or vocabularies) are semantically independent (each vocabulary encodes clearly distinct semantics), the overall semantic representation becomes more stable and robust to minor feature variations.

Therefore, the phrase “independent and minimal semantic units” emphasizes that each disentangled subgraph ideally encodes a unique, non-redundant piece of semantic information. Minimizing semantic redundancy between these vocabularies (enforced by SIP and its mutual-information regularizer $\mathcal{R}_ {\text{MI}}$) is crucial for effectively bounding the semantic discrepancy and facilitating stable knowledge transfer across tasks and domains.

---

Q5: Empirical results suggest Proposition 1’s theoretical claims might not align with observed practical outcomes.

A5: We appreciate this critical observation. There indeed appears to be a misunderstanding stemming from the simplified final expression of the bound in Proposition 1 (Eq. (7)), which depends only on the number of disentangled factors. If taken literally, reducing $K$ would minimize the semantic discrepancy, which contradicts the empirical observation that increasing semantic diversity (i.e., larger $K$) improves performance.

We clarify this misunderstanding as follows: the derived bound (Eq. (7)) explicitly relies on the assumption that semantic independence across vocabularies (enforced via SIP and the regularizer $\mathcal{R}_{\text{MI}}$) has already been achieved during pre-training (we also explain this in A3). Therefore, Proposition 1 is not intended as a direct optimization target (i.e., reducing $K$), but rather as theoretical justification for why controlling semantic redundancy (via SIP) stabilizes the semantic representation.

In practice, moderate semantic diversity (larger $K$) indeed proves beneficial by capturing richer semantics, as demonstrated by our empirical observations. Thus, there is no contradiction: Proposition 1 provides theoretical backing for controlling semantic independence through $\mathcal{R}_\text{MI}$, rather than advocating directly reducing $K$ (given a certain $K$, the upper bound of $\Delta$ is confirmed).

---

Q6: The reason for splitting “ogbn-product” into “tech” and “home” is unclear, and the renaming causes confusion about the dataset’s origin.

A6: We appreciate the reviewer highlighting this confusion. The ogbn-Tech and ogbn-Home datasets used in our experiments are split from the original ogbn-Products dataset, following the same protocol used in related work ZeroG [1].

Due to space constraints, we briefly referenced this in the main text, and we provide full details in Appendix D.1. As described on line 928, we created the two subsets to be non-overlapping in class space, which allows us to simulate realistic in-domain adaptation under class-level shifts, a setting that better reflects generalization within a single structural distribution.

In our revision, we will explicitly clarify in the main text to avoid confusion and ensure dataset provenance is transparent.

---

Q7: The contributions may feel overstated due to dense mathematical and technical presentation, despite the paper’s novelty and detail.

A7: We sincerely thank the reviewer for acknowledging the novelty of our ideas and the level of detail in the paper. We are especially encouraged by the time and effort taken to carefully engage with both the theoretical and empirical aspects of our work.

That said, we understand the concern regarding potential over-packaging of technical content. Our intention was to provide clear theoretical grounding for key design choices, not to overstate their complexity. We will be mindful in refining the presentation to ensure that the contributions are communicated with clarity and appropriate emphasis.

---

We hope our responses have clarified the theoretical and empirical intentions behind our design. If the above points address your concerns, we would greatly appreciate a reconsideration of your score!

---

Refs:
[1] ZeroG: Investigating cross-dataset zero-shot transferability in graphs, KDD 2024.
[2] One for All: Towards training one graph model for all classification tasks, ICLR 2024.
[3] Exploring the potential of large language models (LLMs) in learning on graphs, KDD 2024.