Port-Hamiltonian Architectural Bias for Long-Range Propagation in Deep Graph Networks

We sincerely thank the Reviewer for the consistently positive feedback and the valuable insights provided on our work. In particular, we appreciate the Reviewer’s recognition of the very clear motivation behind our approach, as well as its originality and cleverness. Additionally, we are grateful for the acknowledgment of our strong theoretical results, our superb experimental setup, and the consistently favorable results demonstrated. We are pleased to address each comment in detail, following the original order.

Regarding theoretical insights in the full port-Hamiltonian case

We appreciate the Reviewer’s effort in improving the quality of our work. In this regard, we included the Reviewer’s suggestion in Appendix B.7, where we derived new additional theoretical results on the effects of the port-Hamiltonian components on information propagation. Specifically, we observe that the sensitivity matrix can be linearly decomposed into the conservative and dissipative terms. Under the assumption that the driving forces and their derivatives are bounded, the self-influence of a node after one update can be constrained within fixed upper and lower bounds, which are mediated (among the other) by the step-size $\epsilon$. Additionally, we demonstrate that a similar upper bound applies to the influence between neighboring nodes. These results indicate that, under mild assumptions, the port-Hamiltonian components theoretically support long-range propagation.

Regarding the explanation of Theorem 2.3

We thank the Reviewer for carefully reading our manuscript and helping us to further improve on the presentation of our work. We acknowledge that the wording “retains its complete past” may be ambiguous and leaves room for interpretation about how “strong” the past influences the final representation. However, since the sensitivity matrix does not vanish, we believe that the information can not be discarded. Moreover, as our upper bound on the self-node BSM in Appendix A.1 shows, the further the node representation lies in the past, the higher is its influence on the final state to achieve the energy conservation regime. We are happy to further discuss with the Reviewer on this regard, since we do not fully grasp the intended meaning of the “imply something stronger” comment.

Regarding the explanations of the performance gains coming from driving forces

We thank the Reviewer again for pointing us to opportunities to further improve our work. Dampening and external forces have great interpretations stemming from physical phenomena, like a pendulum that is suffering friction losses but at the same time is externally moved around in space. In our scenario, we believe that the learned driving forces act as an adaptive filter mechanism that filters noisy information in the embedding evolution, facilitating the learning of relevant information, thus resulting in improved performance on the downstream task. However, visualizing the filtering mechanism poses a significant challenge because we don’t have explicit knowledge of what constitutes "relevant" versus "noisy" information. Similarly, visualizing the high-dimensional node dynamic trajectories is also not trivial, since the 2D representation may not reflect the actual non-linear behavior in the latent space.

Regarding minor typos

We thank the Reviewer for thorough reading of our work. We corrected them in the revised paper.

Regarding classical GCN in Theorem 2.4

We thank the Reviewer for the comment, since our goal is to provide a general framework which can be used with many different aggregation schemes. Considering the difference between our vanilla neighborhood aggregation and the degree-normalized GCN sum, we find that the constants $\frac{1}{\sqrt{d_u}\sqrt{d_v}}$ can be injected as part of a weighted adjacency matrix into our proof. For the ease of presentation, we opt for omitting these kinds of scaling factors in the proof.

Regarding controlling the upper bound Appendix Th. A.1

We thank the Reviewer for the question. In our experiments, we tested two aggregations schemes, i.e., the one implemented in Eq. 6 and the classical GCN aggregation. Although Theorem A.1 theoretically indicates a potential increase in the sensitivity measure, we did not observe this issue with either aggregation method during our experiments. Furthermore, we found that incorporating the GCN aggregation scheme did not consistently lead to improved performance. Hence, we did not see the necessity to include norm-constraining regularizers on the weights during training in our experiments. We recommend practitioners to treat the aggregation scheme as a hyperparameter to be selected via model selection.

Regarding the interpretability of a trained PH-DGN

We thank the Reviewer for the comment. Our experimental suite was designed to deliberately demonstrate the long-range capabilities of our approach and, as such, we did not focus on examining the interpretability of the trained model. Interpreting the dynamics of the model is inherently challenging due to the lack of ground truth for what constitutes the "true" flow of information across the graph, especially on tasks like predicting 3D properties of peptides. Without this reference, it becomes difficult to disentangle how the model processes and propagates information or to verify whether it aligns with any hypothesized dynamics. This challenge is further amplified by the fact that, in our experiments, the driving forces are modeled using complex neural networks, making it difficult to deduce clear intuitions about how these forces operate within the model. These limitations highlight the need for future work to explore the interpretability of the methods.

We thank the Reviewer for the extensive feedback on our manuscript and for acknowledging that we carefully explained the knowledge behind our method and that we theoretically show and empirically validate the usefulness in the long-range regime. Below, we address each of your comments, for which we are grateful. We found them to be helpful to further improve the quality of our paper, and we hope that you are satisfied with our response. We hope that in light of our clarifications and modifications to the paper, you will consider revising your score.
In particular, following the Reviewer’s suggestions, we highlight how the revised version of the paper now contains new additional theoretical guarantees on long-range propagation that also accounts for the presence of non-conservative forces, as well as new experiments to prove the usefulness of the fully conservative PH-DGN (and, of course, additional clarifications to all Reviewer’s questions).

1) Regarding the theoretical guarantees for the general PH-DGN

We appreciate the Reviewer’s effort in improving the quality of our work. We note that the main goal of our paper is to reconcile under a single framework strong theoretical guarantees of conservation, for non-dissipative long-range propagation, with non-conservative behaviors, to potentially improve the performance on the downstream task. Indeed, without driving forces, the final ability of the system to model all complex nonlinear dynamics is restricted in real-world scenarios, as empirically shown in Table 2. To further accommodate the Reviewer’s suggestion, we derived some additional theoretical results on the effects of the port-Hamiltonian components on information propagation in Appendix B.7. In particular, we note that the sensitivity matrix can be linearly decomposed into the conservative and dissipative terms. Assuming that the driving forces and their derivatives are bounded, the self-influence of a node after one update can be constrained within fixed upper and lower bounds which are mediated (among the other) by the step-size $\epsilon$. Additionally, we demonstrate that a similar upper bound applies to the influence between neighboring nodes. These results indicate that, under mild assumptions, the port-Hamiltonian components theoretically support long-range propagation.

2) Regarding Tables 2 and 5

We thank the Reviewer for the feedback. Both Tables 2 and 5 contain results for the LRGB benchmark. Due to submission length constraints, we decided to report in the main text (i.e., Table 2) only a selection of all the considered baselines while in the appendix (i.e., Table 5) we report the full list of baselines. In our revised paper, we have improved the clarity of both tables by better specifying which result comes from which paper, and whether or not the model uses positional/structural encodings. We are open to other suggestions to improve the clarity of the tables.

3) Regarding the usefulness of PH-DGN$_C$ on the real dataset

We appreciate the Reviewer’s comment. First, we would like to highlight that, given our PH-DGN is designed through the lens of differential equations, the most relevant baselines for comparison are DE-DGN models. As shown in Table 2, PH-DGN$_C$ achieves superior performance compared to all DE-DGN baselines. Furthermore, PH-DGN$_C$ outperforms global graph transformers and multi-hop approaches across both tasks, and ranks as the third-best overall model on the peptide-func dataset. Overall, PH-DGN$_C$ outperforms 12 out of 13 baselines on peptide-func and 7 out of 13 baselines on the other task. In general, our intuition is that PH-DGN$_C$ may have improved utility for tasks that require preserving all information, such as propagating all node labels over the graph or counting the number of nodes with a given label. To provide empirical support for this intuition, we have included in the revised manuscript (Appendix D.4) an ablation study on the Minesweeper task from (Platonov et al. (2023)) and on the graph transfer task. For your convenience, we report the Minesweeper results in the table below. In the Minesweeper task (i.e., a counting-based task), PH-DGN shows no advantage in deviating from a purely conservative regime. Notably, our PH-DGN$_C$ achieves a new state-of-the-art performance of 98.45 on this benchmark. These results suggest that model selection should determine the optimal use of port components depending on the specific data setting.

Platonov et al. A critical look at the evaluation of GNNs under heterophily: Are we really making progress?. ICLR 2023

Now we are happy to clarify on each question, following original ordering:

- Regarding the term “anti-derivative”

We use the term “anti-derivative” to refer to a differentiable function $F$ whose derivative is equal to the original function $f$. Therefore, the anti-derivative of an activation function is the function $F(x)$ whose derivative leads to the original activation function, i.e., $F^\prime(x) = \sigma (x)$.

- Regarding the norm of the sensitivity

Since $[\partial x_u(T)/ \partial x_u(T-t)]$ is an $\mathbb{R}^{d\times d}$ matrix, our statements are valid for all sub-multiplicative matrix norms, e.g., p-norm and Frobenius norm

- Regarding upper bound sensitivity

We thank the Reviewer for the insightful comment. To clarify, Theorem 2.3 provides a lower bound for the influence of the same node $u$, while Theorem 2.4 establishes an upper bound for the influence between two different nodes, $u$ and $v$, both derived using our proposed model. The study of upper bounds for interactions between different nodes was recently proposed by Topping et al. (2022) and Di Giovanni et al. (2023) as a means to characterize the long-range propagation problem and it is currently a widely accepted means to characterize the long-range propagation problem within the community of deep learning for graphs. We agree with the Reviewer that an upper bound may not fully reflect the actual long-range capability of the model. However, we believe that, since information cannot vanish and our PH-DGN$_C$ can theoretically propagate more information, our PH-DGN$_C$ is theoretically more effective in long-range propagation. This claim is supported by our experimental results presented in Section 3. Specifically, our model (with the bigger upper bound) achieves better results on all long-range tasks than MPNN models (with a smaller upper bound, as shown in Topping et al. 2022 and Di Giovanni et al. 2023). Therefore, we believe that our theoretical bounds reflect the practical sensitivity of PH-DGN, demonstrating its superior capability to propagate information over long ranges compared to previous state-of-the-art models.

Topping et al. Understanding over-squashing and bottlenecks on graphs via curvature. ICLR 2022

Di Giovanni et al. On over-squashing in message passing neural networks: the impact of width, depth, and topology. ICML 2023

- Regarding the graph transfer task

We refer the Reviewer to Appendix C.2 for deeper insights on the data and the task of the graph transfer experiment. We built this experiment based on the graph transfer task proposed by Di Giovanni et al. (2023). In simple words, it is an information-exchange task where we measure how far the information can travel within the graph. The source node is assigned with feature “1”, the target with feature “0”, and intermediate nodes with a random feature sampled from a uniform distribution in [0, 0.5). Then, we implemented a supervised task (specifically, a regression problem) that measures how much of the source node information has reached the target node. On large graphs, the more information reaches the target node, the more effective the model is in the long-range regime. We clarified this in the revised manuscript.

Di Giovanni et al. On over-squashing in message passing neural networks: the impact of width, depth, and topology. ICML 2023

We now move to the minor comments, following original ordering. Again, we thank the Reviewer for working with us in improving the quality of our work.

- Regarding non-dissipativeness

The Reviewer is correct that we designed our PH-DGN as a non-dissipative system in the sense of “energy-preserving”. Specifically, we built our PH-DGN on top of recent literature on non-dissipative dynamical systems (Haber et al. (2017); Bo et al. (2019); Gravina et al. (2023)), which show that a non-dissipative behavior allows capturing long-term (in the case of time series) and long-range (in the graph domain) dependencies. In a broader sense, the energy of the system can be associated to the node information, since it is linked to the node embedding sensitivity. We believe that our PH-DGN with driving forces achieves better performance on the real-world LRGB because the driving forces act as an adaptive filter mechanism that filters noisy information, facilitating the learning of relevant information. As discussed in the newly introduced Appendix D.4, there are scenarios where a purely non-dissipative approach is more beneficial, as it ensures no loss of information. Although we believe there is a strong correlation between long-range and non-dissipative, we acknowledge the source of misunderstanding and rephrased the sentence in the revised draft.

Haber et al. Stable architectures for deep neural networks. Invers Problems 2017

Bo et al. AntisymmetricRNN: A dynamical system view on recurrent neural networks. ICLR 2019

Gravina et al. Anti-Symmetric DGN: a stable architecture for Deep Graph Networks. ICLR 2023

- Regarding positional encoding in the text

The positional/structural encoding refers only to MPNN baselines, while our model does not rely on such an approach. We clarified this aspect in the revised manuscript (Table 2 and Section 3.4).

- Regarding Tables 2 and 5

We thank the Reviewer for the effort in improving the quality of our work. We clarified this aspect in the revised manuscript.

- Regarding the comparison clarification

Again, we thank the Reviewer for its effort and we refer them to the revised manuscript, which now contains a deeper clarification on the usefulness of the proposed model with respect to existing methods.

- Regarding Adam citation

We included the citations to our employed optimization strategies in the revised manuscript

- Regarding n. layer typo

We corrected the typo in the revised manuscript. Thank you.

- Regarding Table 6

We thank the Reviewer for the comment. Appendix Table 6 serves as a high-level comparison with related works on Hamiltonian inspired DGNs. We opted to report our framework as a single row in the table for simplicity reasons, since in our PH-DGN the driving forces can be turned on and off depending on the specific needs of the problem. Indeed, from a high-level perspective, the conservative approach could be seen as a subset of the full port-Hamiltonian approach, explaining why in the single row scenario we marked the Hamiltonian conservation column. Following the Reviewer’s suggestion, we revised Table 6 to clarify that driving forces do not allow for purely Hamiltonian conservation.

1) Regarding the theoretical guarantees for the general PH-DGN

To further accommodate the Reviewer's suggestion, we derived some additional theoretical results on the effects of the port-Hamiltonian components on information propagation in Appendix B.7. [...] These results indicate that, under mild assumptions, the port-Hamiltonian components theoretically support long-range propagation.

I thank the authors for providing additional theoretical results on PH-DGNs with dissipative components. From Theorem B.1--B.3, I understand that when we introduce the small dissipative components, the perturbation of the BSM is small as well, guaranteeing the non-vanishing gradient and long-range propagation.

2) Regarding Tables 2 and 5

In our revised paper, we have improved the clarity of both tables by better specifying which result comes from which paper, and whether or not the model uses positional/structural encodings.

I thank the authors for updating the tables. The update improves their clarity. I have several clarifications:

1. Does Re-evaluated in Table 5 mean that the results of GCNII+PE/SE and DRew-GCN+PE/SE are re-evaluated by the authors and not cited from Tönshoff et al (2023)?
2. If 1 is true, could you let me know why the authors re-evaluated these models?

In addition, I have the following suggestions for further improvement. However, since this is a matter of taste, it is OK that the authos opts not to adopt them:

1. While the order of Table 2 is Tönshoffetal -> Multi-hop (Gutteridge) -> Transformers (Gutteridge), that of Table 5 is Multi-hop (Gutteridge) -> Transformers (Gutteridge) -> Tönshoff. Tables 2 and 5 should have the same order of sections.
2. I think the section title of Tönshoffetal in Table 5 should be MPNNs rather than Re-evaluated because not all models are re-evaluated and other section titles are model names (e.g., Multi-hop DGNs)

3) Regarding the usefulness of PH-DGN$_C$ on the real dataset

For $\mathrm{PH-GDN}_{\mathrm{C}}$, which has theoretical guarantees, the prediction performance on the real dataset (long-range graph benchmark) does not outperform existing methods, and hence its practical usefulness is limited.

I thank the authors for responding to my concerns about the practical usefulness of $\mathrm{PH-GDN}_{\mathrm{C}}$

We appreciate the Reviewer's comment. First, we would like to highlight that, given our PH-DGN is designed through the lens of differential equations, the most relevant baselines for comparison are DE-DGN models.

I think the design of the numerical experiment should reflect the messages the authors want to claim. From the practitioners' point of view, one of the most important criteria for selecting models is prediction performances. For them, the origin of the model architecture, especially whether the architecture is derived from differential equations, is not important. If the authors emphasize the comparison between PH-DGN with other DE-DGN models, I want to clarify what what the authors want to claim by this comparison.

Furthermore, PH-DGN$_C$ outperforms global graph transformers and multi-hop approaches across both tasks and ranks as the third-best overall model on the peptide-func dataset. Overall, PH-DGN$_C$ outperforms 12 out of 13 baselines on peptide-func and 7 out of 13 baselines on the other task.

As in the previous discussion, I want to clarify what the authors want to claim by these comparisons.

To provide empirical support for this intuition, we have included in the revised manuscript (Appendix D.4) an ablation study on the Minesweeper task from (Platonov et al. (2023)) and on the graph transfer task.

I want to clarify the experiment setting of this task, more specifically, the following part:

The model selection is split into two stages. First, the best purely conservative PH-DGN (i.e., PH-DGN$_C$) is selected from a grid with total number of layers L∈{1,5,8}, embedding dimension d∈{256,512}, step size ϵ∈{0.1,1.0} according to the validation ROC-AUC. Then, the best selected configuration is tested with different driving forces.

Given training and validation datasets, I wonder how the learnable parameters in models for dampening and external forces are used for choosing the hyperparameters of PH-DGN$_C$.

---

Regarding the term "anti-derivative"

I thank the authors for the explanation. I did not know the primitive integral is also called the anti-derivative. I am sorry that I should have checked it by myself.

Regarding the norm of the sensitivity

Since $[\partial x_u(T)/ \partial x_u(T-t)]$ is an $\mathbb{R}^{d\times d}$ matrix, our statements are valid for all sub-multiplicative matrix norms, e.g., p-norm and Frobenius norm

I understand that the proof is valid for any sub-multiplicative norm. I suggest explicitly writing that the norm is any sub-multiplicative to the statement.

I thank the authors for the further responses to my questions.

Regarding the potential risk of overfitting in this protocol, we kindly ask the Reviewer to elaborate further on their argument.

First, I realize that the Dampening and the External forcing do not have hyperparameters, which I overlooked in the last comment. I agree with the authors in that we do not have to choose hyperparameters of these components using the validation dataset.

Still, I think that there is a possibility that the performance of PH-DGN$_C$ could be underestimated by the authors' protocol. In order to search the best hyperparameters from the set of possible hyperparameters (which we denote by $\Theta$), we need to search all possible spaces of learnable parameters for each hyperparameter $\theta \in \Theta$ (using the training dataset), then choose the best hyperparameter (using the validation dataset). However, in the authors' protocol, since we only search the part of learnable parameters to choose the hyperparameter, we could fail to find the best model.

---

For example, for simplicity, suppose we only have two learnable parameters --- $w$ for PH-DGN$_C$ and $w'$ for the Dampening and the External forcing, and one hyperparameter $\theta$ (shared by PH-DGN$_C$ and PH-DGN), which takes only two values $\theta=0, 1$.

For PH-DGN$_C$, we assume:

• When we fix $\theta=0$, the model achieves the best performance $p_0$ at $w=a_0$,
• When we fix $\theta=1$, the model achieves the best performance $p_1$ at $w=a_1$,

where $p_0 > p_1$.

For PH-DGN, we assume:

• When we fix $\theta=0$, the model achieves the best performance $q_0$ at $(w, w') = (a_0, b_0)$,
• When we fix $\theta=1$, the model achieves the best performance $q_1$ at $(w, w') = (a_1, b_1)$,

where $q_0 < q_1$

Then, the best-performing PH-DGN is $(w, w') = (a_1, b_1)$ and $\theta=1$, which achieves $q_1$. However, if we follow the authors' protocol, we first choose $\theta=0$ because we have $p_0 > p_1$, then choose the learnable parameter $(w, w') = (a_0, b_0)$ to get the sub-optimal PH-DGN, which achieves $q_0$.

We thank the Reviewer for the quick response to our message. We noticed that the response may contain several typos, which made some parts challenging to interpret. Below, we provide our reply based on our best understanding of the concerns raised.

As for the under-estimation of the general PH-DGN (i.e., the one with driving forces) in the ablation study in Appendix D.4, we want to emphasize again, that the goal of this study was not to optimize performance but rather to investigate how different driving forces contribute to solve the task under a constant starting point, i.e., the hyperparameters selected for PH-DGN$_C$. We do acknowledge that there may be a possible configuration of shared hyperparameters that could lead to better performances on the validation set for the general PH-DGN. However, while we agree that this is crucial for optimization purposes, it is outside the scope of our ablation, which is to show the effect of singular forces to the same purely conservative regime in the Minesweeper task. ​​Moreover, it seems from the comment that the Reviewer may believe we are not retraining the general PH-DGN after selecting the hyperparameters for PH-DGN$_C$​. However, as we have previously explained, this is not the case. In fact, we retrain the model to optimize its learnable parameters using the selected shared hyperparameters to ensure a fair evaluation.

Finally, since this is an ablation study and optimizing performance is not critical for its purpose, we believe the Reviewer’s concerns, while valid, may not significantly impact the evaluation of our contributions. In light of this, we kindly encourage the Reviewer to consider this context when assessing their score.

Regarding the relation between the nature of the task with the balance between conservative and dissipative parts

We appreciate the Reviewer's constructive comment. In general, we believe that hyperparameter tuning should be performed to select the right balance between conservative components and driving forces components to achieve the best performance. Nevertheless, our intuition from our experiments in Section 3 and Appendix D.4 is that a purely conservative PH-DGN may have improved utility for tasks that require preserving all information. This is the case of propagating all node labels over the graph, or counting the number of nodes with a given label. Differently, driving forces, by acting as adaptive filters, tend to enhance performance on real-world tasks characterized by high noise levels.

Regarding additional experiments on real Hamiltonian dynamics simulation

We thank the Reviewer for the suggestions to demonstrate which other application fields could benefit from our work. Simulating or learning dynamical behavior stemming from physical or chemistry-based generation processes is a very interesting field of application, explored in seminal works on physics-inspired neural networks, which explicitly aim to learn a specific Hamiltonian dynamic based on observations. We designed our model with the goal of improving the long-range capabilities of DGNs, employing port-Hamiltonian systems theory to compute information-rich embeddings that do not (necessarily) mimic (quantum)physical behavior but rather include long-range dynamics. Therefore, we aimed to transform the evolution of the port-Hamiltonian system dynamics into the unfolding of iterative applications of information propagation in DGNs, thereby introducing non-dissipative properties into the DGN’s design. With this general aim in mind, we believe that our approach is broadly applicable to a variety of graph-based tasks where long-range dependencies are critical, going beyond strictly physical systems. Nevertheless, approaching molecular dynamics simulations or other problems governed by physical Hamiltonian dynamics is an exciting direction for future research, as it could provide advantages even in such domains.

We thank the Reviewer for highlighting the solid, general, and sound nature of our approach, as well as the improved efficiency over previous methods. We are also grateful for recognizing the clarity of our work and its practical value, especially on long-range propagation. Lastly, we thank the Reviewer for the constructive and positive comments on our manuscript. We will reply to each weakness and question in original ordering. In particular, aside from the requested clarifications, following up the Reviewer’s suggestion, we have added a novel experiment on a recent benchmark that allows appreciating the impact of the different dissipative components in our approach.

Regarding existing ideas

The Reviewer raises an important comment. As correctly noted by the Reviewer, our PH-DGN belongs to the family of DE-DGNs, thus it builds upon previous works on message passing and neural-ODEs. However, PH-DGN offers a mathematically grounded approach by leveraging port-Hamiltonian dynamical systems to balance non-dissipative long-range propagation and non-conservative behaviors, which to the best of our knowledge has never been done before. We then verified this property on a comprehensive experimental suite. Although our PH-DGN builds upon known theory (which however has been proposed and used in a totally different context), we believe that our model provides a more general and efficient solution designed to tackle the problem of long-range propagation in graphs, which represents a significant challenge for message passing models. Moreover, in contrast to Hamiltonian-based approaches, our PH-DGN is capable of deviating from purely conservative trajectories when needed by the downstream task. In other words, it can employ driving forces as an adaptive filter mechanism that filters noisy information, facilitating the learning of relevant long-range information. We believe that these considerations together with our theoretical and experimental results shed light on the novelty of our approach.

Regarding the derivation of the discretization scheme

We appreciate the Reviewer’s effort to further improve the clarity of our work. Due to the submission page limit, we deliberately opt to omit the details of the final discretized model in the main text. All necessary steps together with background information on symplectic integration schemes are presented in Appendix A.3. Specifically, Eq. 11 provides the discretization step for our PH-DGN without driving forces. We explicitly rewritten Eq. 11 by decomposing it into its p and q components in Eqs 12 and 13. The explicit discretization steps for our PH-DGN with driving forces are given in Eqs. 13 and 14. We are happy to discuss further if specific steps of the discretization are still unclear.

Regarding the structure of W and V

We thank the Reviewer for the comment, and we are happy to elaborate on this aspect, since we believe it is a crucial step in the derivation of the discrete model. Following the Hamiltonian formalism, we recall that the global state $x$ can be decomposed into the two components $p$ and $q$. Therefore, imposing a block diagonal structure for $W$ and $V$ is required only theoretically in the global formulation of our model (i.e., Eq. 11) to ensure the separation of components in the explicit integration scheme. Looking at the explicit scheme (i.e., Eqs. 12 and 13 or Eqs. 13 and 14), each block in the original $W$ and $V$ are $\mathbb{R}^{d/2\times d/2}$ matrices with no constraints on the structure. Thus, $W_p$, $W_q$, $V_p$, and $V_q$ can be considered as independent standard weight matrices. We made this point more clear in the revised manuscript.

Regarding why the port-Hamiltonian framework is more appropriate than simpler alternatives

We thank the Reviewer for the comment. As evidenced by our experimental results and previous literature on long-range propagation (Dwivedi et al. (2022); Di Giovanni et al (2023)) standard MPNN-based models, which can be considered as the simplest DGN models, are insensitive to information contained at distant nodes. To counteract this limitation, recent literature introduced global attention mechanisms or rewiring techniques. While effective, these approaches significantly increase the complexity of information propagation due to the use of denser graph shift operators. In contrast, our PH-DGN achieves state-of-the-art performance without relying on such additional techniques, thus, providing a lightweight model that inherently supports long-range propagation by design.

References:

Dwivedi et al. Long Range Graph Benchmark. NeurIPS 2022.

Di Giovanni et al. On over-squashing in message passing neural networks: the impact of width, depth, and topology. ICML 2023

Regarding the ablation studies

We appreciate the Reviewer’s constructive feedback. To address their suggestion and enhance the quality of our work, we have included an additional benchmark and an ablation study in Appendix D.4 that examines the impact of different dissipative components on the Minesweeper task from Platonov et al. (2023). While the results indicate that certain driving components perform better than others for this task, we recommend performing model selection to identify the optimal components based on the specific data setting. For convenience, we present the results for this task in the table below.

Platonov et al. A critical look at the evaluation of GNNs under heterophily: Are we really making progress?. ICLR 2023

Regarding hyperparameter choices

As detailed in Appendix C, our experiments adhered to the established procedures for each task to ensure fair evaluation and reproducibility. For hyperparameters specific to our PH-DGN, such as the step size $\epsilon$ and the number of layers, we selected values from a thorough and reasonable range, taking into account factors like the average graph diameter in the training set. Lastly, we highlight that we conducted a thorough model selection over a comprehensive grid to minimize the risk of suboptimal performance. We have included this discussion in the revised manuscript. Thank you.

Regarding the problems tackled by PH-DGN

The main objective of our work is to design the information flow within a graph as a solution of a port-Hamiltonian system to mitigate the challenge of long-range propagation in DGNs. Throughout Section 2, we provide theoretical statements to support the claim that our PH-DGN can effectively learn and propagate long-range dependencies between nodes. Afterward, in Section 3 we empirically support our theoretical findings by evaluating our PH-DGN on the graph transfer task, graph property prediction and the long-range benchmark, all specifically designed to test the model's capabilities in the long-range regime. In summary, we believe that our method is optimal for those problems that require the exploitation of long-range dependencies to be effectively solved. As an example, PH-DGN is beneficial to solve shortest-path-based problems, e.g., compute the diameter of a graph (see Section 3.3), or molecular tasks in which far away nodes interact to determine the overall function of the molecule (see Section 3.4). Furthermore, as emerged by our experiments in Section 3.4 and in the newly added Appendix D.4, the use of driving forces can lead to better performance on real-world tasks. The driving forces act as an adaptive filter mechanism that filters noisy information. Meanwhile, a purely conservative approach (i.e., without driving forces) can have improved utility for tasks that require preserving all information, like the Minesweeper task.