How Particle System Theory Enhances Hypergraph Message Passing

Q1: Dataset Type (Cora, Citeseer, Pubmed).

We appreciate the reviewer's vigilance regarding data types. The reviewer is correct that Cora, Citeseer, and Pubmed were originally introduced as graph datasets. However, in hypergraph representation learning research, it is standard practice to convert these citation networks into hypergraphs to capture higher-order relationships. The key transformation lies in defining the hyperedges. Specifically, following the methodology of ED-HNN [1] and common practice in the field, for co-citation networks (Cora, Citeseer, Pubmed), all documents cited by a given document are connected by a hyperedge. As previously stated, ED-HNN, which our work builds upon, explicitly follows this established methodology. This conversion allows for the evaluation of hypergraph models on established benchmarks.

Q2: Performance improvements.

Our model, HAMP, demonstrates superior performance through consistent gains and stability across key benchmarks. As detailed in Table 1, HAMP achieves strong results on all nine datasets, including a notable 3% improvement on both the Senate and Walmart datasets.

More importantly, HAMP-II shows exceptional robustness as network depth increases. Figure 2 illustrates that while competing methods suffer from significant performance degradation at greater depths, HAMP-II consistently outperforms them. This highlights HAMP's ability to capture complex representations while maintaining stability, establishing it as a valuable and reliable framework for deep Hypergraph Neural Networks (HNNs).

Q3: Extra advantage of using deep HAMP-II.

We will clarify HAMP-II's depth behavior from both theoretical and empirical perspectives. As our theoretical analysis in the article shows, based on the particle system perspective, HAMP enables stable information propagation through deeper layers. From an empirical standpoint, HAMP-II achieves rapid convergence on locally structured datasets (e.g., Cora, CiteSeer) with shallow architectures (2-3 layers), demonstrating parameter efficiency. However, for datasets exhibiting long-range dependencies (e.g., Congress, Walmart), deep architectures are essential to capture multi-hop interactions. As depicted in Figure 2, on the Congress dataset, a 10-layer HAMP-II model improves accuracy over its 2-layer counterpart, whereas ED-HNN degrades due to over-smoothing. This demonstrates HAMP-II's unique capacity to leverage depth for complex topologies without performance collapse.

Q4: The performance of the model on particle system datasets.

Exploring HAMP within the context of particle systems is indeed an interesting direction, as the continuous dynamics of such systems naturally align with our HAMP framework. Our current evaluation, however, focuses on discrete hypergraph benchmarks to facilitate direct comparison with the existing hypergraph neural network literature.

Q5: Definition of bi-flocking. Thank you for this insightful question, it has revealed an important conceptual inaccuracy in our original terminology. This prompted us to replace all instances of ''bi-flocking'' with the more precise term ''bi-cluster flocking'' adopted from [2]. Next, we will elaborate on the concept of bi-cluster flocking, which is characterized by the following two conditions:

• Intra-cluster cohesion: bounded position differences and velocity alignment within each sub-ensemble.
• Inter-cluster separation: diverging positions between sub-ensembles.

References:

[1] Wang, et al. Equivariant hypergraph diffusion neural operators. In ICLR, 2023.

[2] Fang, et al. Emergent behaviors of the Cucker-Smale ensemble under attractive-repulsive couplings and rayleigh frictions. Mathematical Models and Methods in Applied Sciences, 29(07):1349–1385, 2019.

We really appreciate you for the valuable insights. All concerns are addressed below point by point.

Q1: Scalability of HAMP.

As mentioned in the appendix A and C.3, we analyze the computational complexity of one layer in HAMP-I/II. This aligns with efficient ED-HNN. We show Time-Memory tradeoff analysis of different methods on Walmart dataset ($|\mathcal{V}| \approx 88k$, $|\mathcal{E}| \approx 69k$). Results (Figure 5) show: HAMP-I/II use 20–26% less memory than ED-HNN. Runtime is near-identical to ED-HNN ($\Delta t \approx 0.1$s) and faster than HDS$^{ode}$.

Q2: Broader application scenarios.

Our current experiments focus on hypergraph node classification to ensure a fair comparison with state-of-the-art methods like ED-HNN and AllDeepSets. This approach also serves to validate our core message-passing mechanism on established hypergraph benchmarks. While this work centers on hypergraph node classification, we recognize the potential of HAMP for other learning tasks. Future work will explore extending its versatility to hyperedge prediction, implemented by scoring pairwise node representations, and to hypergraph classification, using a global pooling mechanism. These extensions will further demonstrate the robustness and adaptability of the HAMP framework.

Q3: The main advantages of HAMP compared to HDS$^{ode}$.

As discussed in our related work, HDS$^{ode}$ adopts control-diffusion ODEs to model hypergraph dynamic systems. In contrast, we design hypergraph message passing from the perspective of particle system theory, which inherently alleviates the over-smoothing problem. Previous methods predominantly neglect a key physical insight: message passing is inherently a particle-level phenomenon, involving local, interactive dynamics that cannot be fully captured by pure diffusion models. Our core innovation lies in modeling hyperedges as dynamic interaction fields between node-level particles, where class-dependent equilibrium distributions in the feature space are achieved through particle dynamics equations incorporating attractive-repulsive forces and Allen-Cahn potential regularization.

We appreciate your recognition of the writing and motivation of our work. Please find our detailed responses to your comments below.

Q1: Fundamental differentiation from classical message passing.

While Equations 4 and 5 can be expressed in a message passing form, the key novelty of HAMP lies in introducing a particle system perspective to message passing: we view message passing as the result of a physically inspired interacting particle system. This perspective enables several conceptual and practical advances beyond standard message passing schemes.

First, traditional message passing typically aggregates information based on local neighborhood similarity and implicitly assumes homophily. In contrast, the proposed particle system formulation explicitly models both attractive and repulsive interactions between nodes, allowing it to handle heterophilic structures where neighbors may belong to different classes. The repulsive force mechanism is especially effective in promoting class separation, as demonstrated in our experiments on heterophilic datasets.

Second, a known side effect of repulsion is feature explosion in deep networks. To address this, we introduce the Allen-Cahn regularization term, inspired by phase transition dynamics, which stabilizes the system by imposing a double-well potential. This term promotes polarization of node features while keeping them bounded, allowing for deeper and more stable message passing.

Third, our formulation provides a physically grounded justification for several behaviors observed in hypergraph networks, including over-smoothing. We prove that the particle-driven dynamics maintain a positive lower bound on the Dirichlet energy, analytically mitigating over-smoothing—a known limitation of diffusion-based message passing.

Q2: How to implement the repulsive force.

The interaction force in our particle system, $f_\beta(\mathbf{x}_i, \mathbf{x}_j, e)$, integrates both attraction and repulsion to model complex interactions. The attractive component operates via a two-phase hypergraph diffusion process, similar to ED-HNN, that propagates features from nodes to hyperedges and back. Crucially, during the second (hyperedge-to-node) phase, we introduce a repulsive force. This force acts as a predefined prior to prevent homogenization---the uniform averaging of features across a hyperedge---by leveraging dataset-specific information. While effective, we note that a fully data-driven approach to modeling this repulsive force could further enhance the model's expressiveness, and we leave this for future work.

Q3: Figure 3 without the error bars.

Thank you for this valuable suggestion regarding statistical rigor. We agree completely on its importance. For Figure 3, we found that adding error bars directly to the main plot created significant visual clutter, especially in the deeper layers ($>$25) where the critical performance divergence occurs. This overlap obscured the figure's primary insight. However, the main takeaway is visually unambiguous: our models with stochastic components maintain $>$60% accuracy at layer 35, while the noise-free versions collapse to 50%. This substantial performance gap demonstrates the enhanced stability our method provides. To fully address your point while maintaining the figure's clarity, we have now included a version of Figure 3 with complete error bars in the supplementary material. We believe this approach provides the requested statistical validation without compromising the interpretability of the main plot.

Q4: Functional distinction and necessity of the double-well potential.

As shown in Table 2, our framework features a crucial functional distinction between its two core components. The repulsion term, $f_\beta$, governs pairwise node interactions within hyperedges, primarily enforcing feature separation. In contrast, the Allen-Cahn potential, $f_d$, operates on individual node features, creating a double-well energy potential that preserves their intrinsic stability throughout the learning dynamics.

This second mechanism ($f_d$) is particularly impactful in heterophilic environments like the Senate and House datasets, as it directly counters the homogenization of node features during aggregation. While simpler monotonic potentials could provide basic stability, they lack the double-well's capacity to induce meta-stable phase separationl, which actively promotes class-aware feature clustering.

This dual-component design explains the performance variations observed: some datasets benefit more from the robust feature preservation offered by the Allen-Cahn potential, while more interaction-centric datasets rely heavily on the pairwise repulsion from $f_\beta$. Our framework intentionally unifies these mechanisms to adeptly handle such diverse hypergraph topologies.

Thanks for your meticulous observation and insightful comments, we are deeply encouraged. All concerns are addressed below point by point.

Q1: How to model $f_\beta(\mathbf{x}_i, \mathbf{x}_j, e)$.

The interaction force in our particle system, $f_\beta(\mathbf{x}_i, \mathbf{x}_j, e)$, integrates both attraction and repulsion. The attractive component is modeled similarly to ED-HNN [1], using a two-phase hypergraph diffusion process that propagates features first from nodes to hyperedges, and then from hyperedges back to nodes.

Crucially, during the second phase (hyperedge-to-node), we introduce a repulsive force as a prior. This repulsion prevents homogenization across all nodes in a hyperedge, i.e., the uniform averaging of features. By leveraging prior information from the dataset to disrupt this averaging, the model can capture far more complex interactions.

Further details on the formulation and implementation of $f_\beta$ are provided in Appendix A.

Q2: Performance improvements.

Our model, HAMP, demonstrates superior performance through consistent gains and stability across key benchmarks. As detailed in Table 1, HAMP achieves strong results on all nine datasets, including a notable 3% improvement on both the Senate and Walmart datasets.

More importantly, HAMP-II shows exceptional robustness as network depth increases. Figure 2 illustrates that while competing methods suffer from significant performance degradation at greater depths, HAMP-II consistently outperforms them. This highlights HAMP's ability to capture complex representations while maintaining stability, establishing it as a valuable and reliable framework for deep Hypergraph Neural Networks (HNNs).

Q3: Definition of homophilic and heterophilic hypergraphs.

To formalize the concepts of homophily and heterophily in hypergraphs, we adopt the Class-Edge (CE) homophily metric $H_{CE}$, as proposed in [2] and used by ED-GNN [1]. This metric quantifies the consistency of class labels among nodes within the same hyperedge: $H_{CE} = \frac{1}{|\mathcal{E}|} \sum_{e \in \mathcal{E}} \frac{ \max_{c \in \mathcal{C}} |\{ v \in e \mid y_v = c \}| }{ |e| }.$ Here, $\mathcal{E}$ is the set of hyperedges, $\mathcal{C}$ is the set of node classes, and $y_v$ is the class label for node $v$. A higher $H_{CE}$ score indicates that hyperedges tend to connect nodes of the same class.

Following established criteria [1], we classify a hypergraph as homophilic if $H_{CE} > 0.7$ and heterophilic if $H_{CE} \leq 0.7$.

Q4: Other learning tasks.

Our current experiments focus on hypergraph node classification to ensure a fair comparison with state-of-the-art methods like ED-HNN and AllDeepSets. This approach also serves to validate our core message-passing mechanism on established hypergraph benchmarks. While this work centers on hypergraph node classification, we recognize the potential of HAMP for other learning tasks. Future work will explore extending its versatility to hyperedge prediction, implemented by scoring pairwise node representations, and to hypergraph classification, using a global pooling mechanism. These extensions will further demonstrate the robustness and adaptability of the HAMP framework.

References:

[1] Wang, et al. Equivariant hypergraph diffusion neural operators. In ICLR, 2023.

[2] Pei, et al. Geom-GCN: Geometric graph convolutional networks. In ICLR, 2020.