Where Graph Meets Heterogeneity: Multi-View Collaborative Graph Experts

W1/Q1. Complexity concerns

Response: Our solution introduces a minor additional overhead but does not lead to a considerable increase in overall complexity [1, 2]. MoE has been widely adopted in LLMs precisely because it allows for the expansion of the model capacity while maintaining inference efficiency through sparse expert activates [3, 4]. In this work, we provide a detailed theoretical complexity analysis in Appendix B. When employing $E$ experts, the computational complexity of the gating mechanism within a single MvCGE layer is $\mathcal{O}(NDE+|\mathcal{E}|D)$, while that of each hierarchical graph expert (e.g., GCN) is $\mathcal{O}(ND^2+|\mathcal{E}|D)$. In practice, $E$ is typically not much larger than the $D$, so it will not be a bottleneck. The most time-consuming operation remains the graph convolution ($\mathcal{O}(|\mathcal{E}|D)$). It is the gating mechanism that allows each sample to activate only a small subset of $K$ experts, even with dozens of graph experts. In addition, for spectral GNNs such as GCN, we can decouple the filter computation from the learnable parameters [5, 6], substantially reducing the number of filtering operations required. Empirically, Appendix A reports the results of MvCGE on two large-scale graphs, and here we further provide efficiency comparisons on three datasets to offer a clearer perspective:

Response: We evaluated MvCGE under two settings: (1) 3 experts with each sample selecting the top-1 expert, and (2) 6 experts with each sample selecting the top-2 experts. The experiments were conducted on single-view graphs. Note that GCN processes only a single view, whereas MvCGE processes two views simultaneously. Nevertheless, MvCGE did not introduce significant computational overhead, which is consistent with our theoretical analysis and other empirical results.

W2. Multiple hyperparameters tuning

Response: Although the proposed model does require tuning of four hyperparameters, in practice, we fixed the number of experts to 6 and the number of selected experts to 2, and still achieved strong performance across all datasets. Furthermore, as demonstrated in the experiments shown in Fig. 6, it is not necessary to finely tune these two hyperparameters within a reasonable range.

W3. Regarding equations and definitions

Response: We apologize for any inconvenience caused by the complex notation and formulas. We will carefully consider your suggestions and systematically adjust the organization and presentation of the text to facilitate readers’ understanding of our method.

[1] Towards understanding the mixture-of-experts layer in deep learning. NeurIPS 2023.
[2] Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ICLR 2017.
[3] Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. JMLR 2022.
[4] Scaling vision with sparse mixture of experts. NeurIPS 2021.
[5] Semi-Supervised Classification with Graph Convolutional Networks. ICLR 2017.
[6] Convolutional neural networks on graphs with chebyshev approximation, revisited. NeurIPS 2022.

We sincerely thank the reviewer for their careful reading and recognition of our work. Below, we provide point-by-point responses to address your concerns.

W1/Q1. Discussion on homophily/heterophily

Response: In this work, we did not explicitly discuss the issue of homophily or heterophily. However, spectral GNNs are closely related to these properties [1, 2]. In homophilic graphs, neighboring nodes tend to share similar attributes or labels, implying that the energy is concentrated in the low-frequency components of the Laplacian spectrum [3, 4]. Conversely, heterophilic graphs exhibit substantial variations between nodes, leading to more high-frequency components. In multi-view graphs, despite sharing the same node set, different views may exhibit significant differences in homophily and heterophily, resulting in distinct spectral distributions. This heterogeneity renders single graph convolutions/filters inadequate for unified processing, which aligns with our theoretical motivation. For instance, the YELP dataset demonstrates substantial inter-view divergence, which explains why MvCGE achieves notably superior performance on it. We will strengthen the discussion of these aspects in the Method and Experiment sections.

W2/Q2. Further discussions on theoretical analysis

Response: Proposition 1 and Corollary 1 primarily serve as the theoretical motivation to expose the inherent limitations of shared graph filters in multi-view graph learning. These insights motivate us to explore collaboration graph filter and propose our solution: By designing a team of specialized graph filters under the MoE framework, MvCGE enables: (1) View-Specific Filter Selection: Each view dynamically activates the most suitable filter combination through graph-aware routing; (2)Collaborative Spectral Processing: Distinct filters specialize in handling complementary frequency bands across views. Therefore, the total error can be decomposed and now depends on the optimal combination that can be chosen for each view from all graph experts, circumventing the error growth bound in Proposition 1. We will reconsider the placement and presentation of these theoretical analyses in future versions.

W3. Graph discrepancy loss clarification

Response: In MvCGE, each graph expert is assigned a set of samples and generates node representations for them, and we expect these graph experts to learn different latent spaces during the training process, and thus need to measure the distribution differences between the learned representations among the graph experts. We build on research related to graph isomorphism computation [5] by introducing the readout function combined with graph experts to obtain graph embeddings, which are essentially global representations that encode graph structure. And the distribution of graph embeddings is used to measure the difference between graph experts. Therefore, we first perform a readout of the representations learned by each graph expert for a certain view by $\mathbf{z} = \mathcal{P}(\mathbf{Z})$. Subsequently, using $Q$ to represent the combination of the readout function and the graph expert, we treat the set of all these embeddings (across different views) as samples from the distribution $\mathbb{P}\_{Q}$. Hence, $\mathbb{P}\_{Q}$ is the empirical distribution over the graph-level embeddings ${\mathbf{z}}$. Whereas different graph experts have different distributions, denoting as $\mathbb{P}\_{Q}$ and $\mathbb{P}\_{Q'}$. Finally, we measure how different the distributions are by using the powerful Maximum Mean Discrepancy (MMD) with an RBF kernel. For a certain view, we compute the node embeddings via graph expert $m$, apply the readout function $\mathcal{P}$, and obtain a single embedding $\mathbf{z}$. Doing this for each view in the dataset yields a set of $\mathbf{z}$, which we treat as samples from $\mathbb{P}\_{Q}$, and $\mathbb{P}\_{Q}$ is simply the empirical distribution of these embeddings.

[1] Graph neural networks for graphs with heterophily: A survey. arXiv preprint arXiv:2202.07082 (2022).
[2] Addressing heterophily in graph anomaly detection: A perspective of graph spectrum. WWW 2023.
[3] Revisiting heterophily for graph neural networks. NeurIPS 2022.
[4] Beyond low-frequency information in graph convolutional networks. AAAI 2021.
[5] How powerful are graph neural networks? ICLR 2018.

We appreciate your thorough review and insightful comments. Below we address every weakness (W) and question (Q) in detail, and we explain the concrete additions that will appear in our revised manuscript.

W1. Reproducibility

Response: To facilitate online access without the need to download additional files, we did not include the source code directly in the supplementary materials. Instead, we have provided an anonymous link for convenient viewing. We would like to clarify that the anonymous project link to the source code, as well as the implementation platform details, are both already included in Appendix A.2. We believe these materials sufficiently demonstrate the effectiveness of the proposed MvCGE and ensure the reproducibility of the experiments presented in this paper.

W2/Q1. Details about the method

Response: We apologize for any inconvenience this may have caused. To ensure precise expression, we adopted a relatively complex notation system. In future versions, we will adjust the organization of the main text and appendices by simplifying the notation as much as possible in the main body, moving certain technical details to the appendix, and providing comprehensive explanations for each symbol. This will help balance readability and accuracy. Here we provide a step-by-step description of the routing function:

W3. Lack of GNN-MoE-based literature

Response: As discussed in the related work section, the MoE [1] architecture has been widely adopted in many modern large language models (LLMs) [2], yet its potential in graph learning remains largely underexplored. A few recent studies have proposed MoE-based methods for graph learning, but these primarily focus on single-view graphs [3, 4, 5]. In contrast, the MoE architecture is inherently well-suited for modeling complex multi-view graphs, which represents a clear research gap. Consequently, we were unable to identify directly comparable baselines in the literature. In this work, we adopt the classical single-view GMoE [3] as a baseline for comparison. We conduct the comparison on both single- and multi-view datasets:

As shown by the experimental results, GMoE demonstrates competitive performance on both types of datasets. However, it consistently falls short of MvCGE, further indicating that while the MoE architecture holds significant potential in multi-view graph scenarios, existing frameworks are insufficient to fully exploit this advantage.

W4. Limited theoretical analysis

Response: Although we do not provide a direct theoretical analysis of the proposed method, we do offer theoretical justification for its rationality. Specifically, a formal lower-bound analysis of single-filter transferability across views is presented in Appendix C. This analysis theoretically demonstrates that a single graph filter cannot adapt to multiple graph structures with distinct frequency characteristics, whereas a combination of multiple filters does not suffer from this limitation. Building upon this insight, we considered the balance between consistency and complementarity, which motivated the design of the MvCGE framework.

Q2. Regarding Equation (13)

Response: The section containing Equation (13) introduces the graph discrepancy loss, which measures the distributional differences between any pair of graph experts within each layer of MvCGE. Therefore, for the sake of clarity and conciseness, we omit the superscript $(l)$ and use $f_{\Theta_m}$ and $f_{\Theta_{m'}}$ to denote any two experts within the same layer when describing this loss.

[1] Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ICLR 2017.
[2] Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024).
[3] Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling. NeurIPS 2023.
[4] Mixture of experts for node classification. ICMR 2025.
[5] Fair graph representation learning via diverse mixture-of-experts. WWW 2023.

We appreciate your constructive feedback and your recognition of our work. Many of your comments provide valuable insights that deserve further exploration. In the following, we systematically address each weakness (W) and question (Q) raised, and we would like to clarify several possible and potential misunderstandings.

W1. Confusion about $\mathcal{L}\_{\mathrm{dsp}}$

Response: We regret any confusion that may have occurred and would like to provide some clarifications. As mentioned at lines 239-240, the Maximum Mean Discrepancy (MMD) $\mathcal{M}^2_{\mathcal{H}}(\mathbb{P}\_{\mathcal{Q}}, \mathbb{P}\_{\mathcal{Q}'})$ [1] is adopted to quantify the discrepancy between different graph experts. Then Eqn. (18) gives the MMD-based loss function:

Therefore, minimizing $\mathcal{L}\_{\mathrm{dsp}}$ maximizes the differences among graph filters, which is consistent with our original design that promotes MvCGE to capture complementary information. We will add explicit statements like "minimizing the total loss maximizes MMD, thereby increasing inter-expert diversity" to avoid potential misunderstandings.

W2/Q1. Complexity concerns

Response: Our solution introduces a minor additional overhead but does not lead to a considerable increase in overall complexity [2, 3]. MoE has been widely adopted in LLMs precisely because it allows for the expansion of the model capacity while maintaining inference efficiency through sparse expert activates [4, 5]. In this work, we provide a detailed theoretical complexity analysis in Appendix B. When employing $E$ experts, the computational complexity of the gating mechanism within a single MvCGE layer is $\mathcal{O}(NDE+|\mathcal{E}|D)$, while that of each hierarchical graph expert (e.g., GCN) is $\mathcal{O}(ND^2+|\mathcal{E}|D)$. In practice, $E$ is typically not much larger than the $D$, so it will not be a bottleneck. The most time-consuming operation remains the graph convolution ($\mathcal{O}(|\mathcal{E}|D)$). It is the gating mechanism that allows each sample to activate only a small subset of $K$ experts, even with dozens of graph experts. In addition, for spectral GNNs such as GCN, we can decouple the filter computation from the learnable parameters [6, 7], substantially reducing the number of filtering operations required. Empirically, Appendix A reports the results of MvCGE on two large-scale graphs, and here we further provide efficiency comparisons on three datasets to offer a clearer perspective:

We evaluated MvCGE under two settings: (1) 3 experts with each sample selecting the top-1 expert, and (2) 6 experts with each sample selecting the top-2 experts. The experiments were conducted on single-view graphs. Note that GCN processes only a single view, whereas MvCGE processes two views simultaneously. Nevertheless, MvCGE did not introduce significant computational overhead, which is consistent with our theoretical analysis and other empirical results.

W3/Q6. Generalization to other tasks

Response: In this work, we primarily validate the effectiveness of our model on the common graph learning task, semi-supervised node classification, rather than positioning MvCGE as a foundation model. As we demonstrated, it is straightforward to adapt the model to some graph tasks by simply replacing $\mathcal{L}\_{\text{task}}$. We have conducted additional experiments on link prediction and graph-level classification, as shown below:

For the graph-level tasks, we followed the experimental settings of [8], while for the link prediction experiments, we adopted the settings from [9]. As can be seen, MvCGE continues to demonstrate strong performance on these tasks. We will consider evaluating the model in even more scenarios in future work. Thank you for your valuable suggestion.

W4/Q5. Reproducibility

Response: We clarify that we have included the anonymous repository link and hardware details in Appendix A.2, which ensures reproducibility of our work. Here we further provide the detailed settings to address your concern:

• For single-view graphs, we generate an additional feature-similarity $k$NN graph ($k=10$, Euclidean distance) as the second view.
• For the number of experts/Top-K, we fix them as $E=6$ and $K=2$ for Table 1 and 2, and analyze the impact of their choice in our paper (refer to Fig. 6), and we will add clear statements of the exact $k$NN threshold and other tuning grids to an Implementation Details subsection.
• Regarding the sensitivity for $\alpha$ and $\beta$, we have analyzed in the main body of our submission (refer to Fig. 5). As can be observed, performance is stable in $\pm1$ order-of-magnitude around the chosen $\alpha$ and $\beta$. We will revise our paper to include more detailed settings for reproducibility.

Q2. View definition & expert design

Response: For real multi-view datasets we use the provided edge sets. For single-view datasets we add a feature-similarity kNN graph (see above). All experts share the same base spectral GNN layer (e.g., GCN or ChebNet ). Diversity is imposed mainly by the mechanism of MoE and our proposed loss $\mathcal{L}\_{\text{dsp}}$.

Q3. Expert specialisation

Response: After training, it is indeed observed that different experts tend to focus on samples from a specific view. This forms the unique inductive bias of MvCGE, enabling the preservation of view-specific information. Regarding the substructures you have mentioned, since MvCGE is built upon graph spectral theory, different experts are able to learn distinct filter parameters, thereby capturing homophilic/heterophilic patterns. We have conducted visualization experiments on expert specialization and routing. Due to the NeurIPS 2025 rebuttal policy, we will include detailed discussion and related results in future versions.

Q4. Robustness to noise

Response: Owing to the presence of the multi-view aggregation module, MvCGE is theoretically capable of assigning low weights or even blocking those views that result in poor training loss (when using gating-based aggregation). As a result, the framework exhibits a certain degree of tolerance to noise. We conduct an experiment on ACM for a better understanding:

Here, we construct a noisy view by randomly generating an adjacency matrix and test MvCGE's performance under different noise rates. We will include additional experiments on robustness to further discuss this aspect.

[1] A kernel two-sample test. JMLR 2012.
[2] Towards understanding the mixture-of-experts layer in deep learning. NeurIPS 2023.
[3] Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ICLR 2017.
[4] Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. JMLR 2022.
[5] Scaling vision with sparse mixture of experts. NeurIPS 2021.
[6] Semi-Supervised Classification with Graph Convolutional Networks. ICLR 2017.
[7] Convolutional neural networks on graphs with chebyshev approximation, revisited. NeurIPS 2022.
[8] How powerful are graph neural networks? ICLR 2018.
[9] Evaluating graph neural networks for link prediction: Current pitfalls and new benchmarking. NeurIPS 2023.

Thank you very much for your valuable suggestions. We will carefully revise our manuscript according to your comments and are ready to respond to any further questions you may have. We sincerely appreciate your time.