Structural Inference with Dynamics Encoding and Partial Correlation Coefficients

We are grateful to the reviewer for their encouraging and insightful feedback. It is heartening to hear that the novelty of the SIDEC concept and its broad applicability were well-received, alongside the paper's organized presentation of both theory and implementation. The acknowledgment of SIDEC's superior performance compared to baseline models is also greatly appreciated.

Regarding the concerns raised by the reviewer:

W. While the paper is commendable overall, one potential weakness lies in the absence of an evaluation on real-world data. The authors acknowledge this in the "Limitations" section, citing the challenges of collecting such data. It would be beneficial to include plans for addressing this in future work.

Thank you for highlighting this issue. While we did add experiments on three real-world datasets in response to Reviewer 5DvY, these datasets proved problematic due to their unreliable and imprecise adjacency matrices, which compromised our ability to fully showcase SIDEC's strengths and performance.

We used the PEMS03, PEMS04, and PEMS07 datasets [1] from California's Caltrans Performance Measurement System (PeMS) [2]. All these data are aggregated into 5-minutes windows, which means there are 288 points in the traffic flow for one day. The adjacency matrix of the nodes is constructed by road network distance with a thresholded Gaussian kernel [1]. The summary of these datasets can be found in the following table:

We resampled the data such that constructing 49 time steps of points for each trajectory, and obtained 534, 346, and 547 trajectories, respectively. We then split all datasets with ratio 8:2:2 into training sets, validation sets and test sets, and run SIDEC on all of them. We report the average AUROC values (in %) of ten runs in the following table:

As shown in the table, despite the noise and missing data, and the inaccuracy in the adjacency matrices provided by the dataset, SIDEC manages to perform well and can deal with the challenges. It is notable that the adjacency matrices in these dataset just connect the sensors if thy are on the same road, but they ignore the existence of paths or alleys that may also connect the sensors. Because of this, when compared with the experimental results discussed in the original submission, the performance of SIDEC decreases massively.

We've added these findings to the Appendix D.4 of our revised manuscript. We believe this additional real-world data testing not only addresses your concerns but also significantly strengthens the validity and applicability of our research. We plan to address this by collecting more reliable gene regulatory network data through in-lab experiments, although this is a time-intensive process.

---

Q1. The choice of using partial correlation as the final step in reconstructing the graph structure merits further clarification.

Many thanks for the comment. We selected the partial correlation coefficient (PCOR) as it has shown superior performance in StructInfer [3] benchmarks, especially for one-dimensional trajectories. We believe that with effective dimension extraction and dynamic embedding, PCOR can be adapted for broader applications, including directed graphs and multi-dimensional nodes. This was a key factor in developing SIDEC.

---

Q2. A suggestion for refining the paper layout is to consider rearranging Algorithms 1 and 2 before presenting Algorithms 3 and 4 in the appendix for better logical flow.

Thank you for this suggestion. We have revised the manuscript accordingly to enhance readability and logical flow.

We hope our responses satisfactorily address the reviewer's concerns, and we extend our gratitude for their valuable contributions toward improving our paper. To facilitate easy identification, sections and paragraphs that have been revised in the manuscript are highlighted in blue.

---

References

[1] C. Song, Y. Lin, S. Guo and H. Wan.Spatial- Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting. AAAI, volume 34, 914–921.

[2] C. Chen, K. Petty, A. Skabardonis, P. Varaiya and Z. Jia. Freeway performance measurement system: mining loop detector data. Transportation Research Record 1748(1):96–102.

[3] Authors Anonymous. Benchmarking structural inference methods for dynamical interacting systems. https://structinfer.github.io/, 2023.

Q1. SIDEC includes PCOR, and some PROR models are also mentioned in the Structural inference with correlations and partial correlations section. Why are there no comparative experiments of these models included in the experimental section?

Thank you for this question. The decision to exclude a direct comparative analysis of PROR models in the main experimental section was made due to the inherent limitations of these models. Specifically, correlation-based structural inference methods are typically constrained to graph reconstruction of undirected graphs with one-dimensional node features, significantly limiting their application scope.

To address this, we conducted a thorough evaluation of various correlation-based methods, as an addition to the experimental results demonstrated in Appendix D.2. We evaluate the baselines and SIDEC on LN_NS datasets, which is characterized by one-dimensional node feature and undirected underlying interaction graph. We add these methods for evaluation:

• Person Correlation (P-cor) [9]
• Distance Correlation (D-cor) [10]
• Low-order Partial Correlation (LPCOR) [11]
• Semi-partial Correlation Coefficients (SPCOR) [12]

Additionally, we incorporated PCOR and SIDEC results into this analysis for a comprehensive comparison. Our results are presented in the following table:

The results indicate that both P-cor and D-cor show subpar performance across all datasets. This can be attributed to their limitation of considering only pairwise correlations and overlooking the influence of other potentially impactful nodes. LPCOR, while better, still falls short as it calculates coefficients conditional on a subset rather than all other variables.

Interestingly, SPCOR and PCOR demonstrate comparable performance, with PCOR slightly outperforming SPCOR. This can be attributed to SPCOR's reliance on semi-partial correlation calculations, which may limit its effectiveness in identifying connections between nodes with larger degrees. Despite the variations in graph structure across the datasets, neither SPCOR nor PCOR outperforms SIDEC.

We believe these findings reinforce our approach of integrating VDE within SIDEC, which significantly enhances the overall performance of structural inference. We have updated our manuscript to include these comprehensive results in Appendix D.2, offering a broader context and validation for our methodological choices.

---

Q2. Judging from the results of the ablation experiment, the optimal size of the time window is 1. It seems that the existence of the time window does not fully improve the performance of the model. Does this mean that it is not necessary to use VDB?

Thank you for highlighting the significance of the optimal time window size in our ablation experiments. The choice of the time window $\tau$, is indeed a critical aspect of our model, as it determines the scale at which the dynamics of the system are analyzed. The Markovian assumption inherent in our model implies that knowing the state of the system at time $\tau$ should suffice to predict its future state at time $t+\tau$, without the need to consider its earlier history [13].

In the case of the StructInfer [14] datasets, which were generated with smaller time intervals and subsequently subsampled, a $\tau$ value of 1 suggests that the system behaves Markovian over these intervals. This means that past states do not significantly influence future states beyond this time window.

However, it's important to note that there are scenarios where:

1. The system might not strictly adhere to Markovian dynamics, and our observations are closely spaced such that immediate past states influence future states.
2. The data are directly sampled from simulations at minimal time intervals.

In these scenarios, a $\tau$ value larger than 1 might be more appropriate. Determining the optimal value of $\tau$ in such cases is a complex task and requires further investigation, which we plan to undertake in future research.

---

W2. Experiment Datasets are all artificially generated and may not fully reflect the complexity and diversity of the real world. Therefore, the experimental results of this article may have certain biases and limitations, and further testing and evaluation on more real data sets are required.

We would like to emphasis that obtaining comprehensive, real-world data for structural inference poses significant challenges, both in terms of cost and time, which is mentioned in the section of limitation in the submission. Often, real-world data lack complete information on node features or reliable knowledge of underlying interaction graphs, as seen in gene regulatory networks [15] and scene graphs [4]. In other cases, such as transaction networks [5] and road maps [6], data may not cover all nodes in the system.

Despite these challenges, we agree with the necessity of testing our method, SIDEC, on real-world data. To this end, we have conducted additional experiments using three widely recognized public traffic network datasets: PEMS03, PEMS04, and PEMS07 [7]. These datasets, derived from the California Caltrans Performance Measurement System (PeMS) [8], comprise data aggregated into 5-minute intervals. The adjacency matrix of the nodes is constructed by road network distance with a thresholded Gaussian kernel [7]. The following table summarizes these datasets:

We resampled the data such that constructing 49 time steps of points for each trajectory, and obtained 534, 346, and 547 trajectories, respectively. We then split all datasets with ratio 8:2:2 into training sets, validation sets and test sets, and run SIDEC on all of them. Our results, presented below, demonstrate SIDEC's effective performance despite challenges such as noise, missing data, and adjacency matrix inaccuracies. It's important to note that these datasets' adjacency matrices only connect sensors on the same road, omitting alternative connecting paths, which could impact results.

As shown in the table, despite the noise and missing data, and the inaccuracy in the adjacency matrices provided by the dataset, SIDEC manages to perform well and can deal with the challenges. It is notable that the adjacency matrices in these datasets just connect the sensors if they are on the same road, but they ignore the existence of paths or alleys that may also connect the sensors. Because of this, when compared with the experimental results discussed in the original submission, the performance of SIDEC is impacted. We've added these findings to Appendix D.4 of our revised manuscript. We believe this additional real-world data testing not only addresses your concerns but also significantly strengthens the validity and applicability of our research.

---

W3. This method combines VDB and PCOR, but the paper does not give the impact on training time after the combination.

We appreciate your insightful observation regarding the necessity of addressing the training time in our methodology. In response to your query, it's important to clarify that PCOR does not require training and, therefore, does not directly influence the training duration of the variational dynamics encoder (VDE). Our method, SIDEC, integrates structural inference through two main components: training of VDE and subsequent inference utilizing both VDE and PCOR. Through our experimental observations, we found that the training phase of VDE is the most time-consuming aspect, accounting for approximately 98% of the total computation time. This pattern is consistent across all our experiments. We revised the manuscript to add this description in Appendix A.7.

---

We sincerely thank you, Reviewer 5DvY, for your motivating and positive feedback on our work. We appreciate your recognition of the novelty and clarity of our paper and are encouraged by your acknowledgment of SIDEC's performance. Below, we address your concerns in detail:

W1. This paper mainly designs and analyzes the SIDEC framework based on information bottleneck theory and partial correlation coefficient, but it does not give clear mathematical proofs and theoretical guarantees to explain how they influence each other.

Thank you for your insightful feedback. We acknowledge the reviewer's concern regarding the lack of detailed mathematical proofs and theoretical guarantees in our paper, specifically in the context of the SIDEC framework's reliance on information bottleneck theory and partial correlation coefficient (PCOR).

Our proposed framework, SIDEC, utilizes the variational dynamics encoder (VDE) [1], a variant of the variational autoencoder (VAE), which adheres to the variational information bottleneck (VIB) [2] principle. Specifically, the VDE in our framework seeks to optimize the encoded representation $\mathbf{z}^t_i$ to be informative about the future state $v^{t+\tau}_i$ while minimizing dependency on the current state $v^t_i$, as dictated by the information bottleneck theory [3]. This process effectively compresses the representation of input trajectories, ensuring that the dynamics of each node are captured over a defined time window $\tau$.

Upon obtaining the dynamics embeddings, we utilize PCOR to infer the graph structure. PCOR's efficacy in structural inference for one-dimensional data [15], underpins its integration into our framework. The interplay between VDE and PCOR within SIDEC is thus crucial; the former captures dynamic node embeddings, while the latter leverages these embeddings for structural inference.

We recognize that a comprehensive mathematical proof elaborating on this interplay is absent from our current manuscript. The decision to defer a detailed proof to future work stems from the complexity of the theoretical underpinnings, which require a more extensive treatment than our current scope allows. Future work will focus on providing these rigorous mathematical validations, particularly elucidating the mutual influence of VDE and PCOR within the SIDEC framework.

We appreciate this opportunity to clarify our approach and its theoretical foundations. We believe that these additional details, along with our commitment to future work, reinforce the contributions of our paper and address the concerns raised. We will integrate this refined explanation into the Appendix of our manuscript to ensure our approach's theoretical aspects are both transparent and robust.

---

Dear Reviewer 5DvY,

We hope you are doing well. As the deadline for the author-reviewer discussion phase is nearing, we would like to courteously inquire if our rebuttal has effectively addressed the concerns you raised in your review of our paper.

Your insightful feedback, especially regarding experiments with real-world data, the comparison of correlation-based methods, and the assessment of time window lengths, has been invaluable in refining our work. We have endeavored to address each of these points meticulously in our rebuttal and are eager to know if our responses and subsequent revisions have met your expectations and clarified the issues you pointed out.

We recognize the demands on your time and deeply appreciate any further feedback you can offer on our rebuttal. Your expertise is not only vital for the review process but also enriches our ongoing learning and growth in this field.

Thank you once again for your time and invaluable insights. We eagerly await your response.

Best regards, Authors

Q5. What does "level of added Gaussian noise" mean? Are you changing the mean of the noise? Please be clear.

The StructInfer dataset [13], which we utilized in our study, includes trajectories with Gaussian noise added at various levels. As detailed in the dataset's documentation, the authors introduced this noise to the generated trajectories to simulate real-world conditions more accurately. The modified node features, represented as $\tilde{v}_i^t$, are calculated using the formula: $\tilde{v}_i^t = v_i^t + \zeta \cdot 0.02 \cdot \Delta$, where $\zeta$ is a standard Gaussian random variable ($\zeta \sim \mathcal{N}(0,1)$), $v_i^t$ denotes the original feature vector of node $i$ at time $t$, and $\Delta$ represents the designated noise level. The noise levels applied range from 1 to 5 across all the original trajectories.

This addition of Gaussian noise is an important aspect of our experimental data, and we have updated our paper to include a comprehensive description of this process in Appendix B.

We extend our gratitude to Reviewer Fg5F for the thorough, constructive, and insightful feedback. It is gratifying to know that our work has piqued your interest. We sincerely hope that our responses adequately address your concerns and contribute to enhancing the quality of our paper. To facilitate easy identification, sections and paragraphs that have been revised in the manuscript are highlighted in blue.

---

References

[1] C. X. Hernández, H. K. Wayment-Steele, M. M. Sultan, B. E. Husic, and V. S. Pande. Variational encoding of complex dynamics. Physical Review E, 97(6):062412, 2018.

[2] A. A. Alemi, I. Fischer, J. V. Dillon, and Kevin Murphy. Deep variational information bottleneck. In Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.

[3] A. Wang and J. Pang. Iterative structural inference of directed graphs. In Advances in Neural Information Processing Systems 35 (NeurIPS), 2022.

[4] A. Wang and J. Pang. Active learning based structural inference. In Proceedings of the 40th International Conference on Machine Learning (ICML), pages 36224-36245. PMLR, 2023.

[5] A. Wang, T. P. Tong and J. Pang. Effective and efficient structural inference with reservoir computing. In Proceedings of the 40th International Conference on Machine Learning (ICML), pages 36391-36410. PMLR, 2023.

[6] M. Irwin and Z. Wang. Dynamic Systems Modeling, pp. 1–12. John Wiley & Sons, Ltd, 2017.

[7] T. Kipf, E. Fetaya, K.-C. Wang, M. Welling, and R. Zemel. Neural relational inference for interacting systems. In Proceedings of the 35th International Conference on Machine Learning (ICML), pages 2688–2697. PMLR, 2018.

[8] S. Löwe, D. Madras, R. Z. Shilling, and M. Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. In Proceedings of the 1st Conference on Causal Learning and Reasoning (CLeaR), pages 509–525. PMLR, 2022.

[9] S. Chen, J. Wang, and G. Li. Neural relational inference with efficient message passing mechanisms. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), pages 7055–7063, 2021.

[10] F. Noé and F. Nuske. A variational approach to modeling slow processes in stochastic dynamical systems. Multiscale Modeling & Simulation, 11(2):635–655, 2013.

[11] C.F. Li, G.C. Guo, and J. Piilo. Non-Markovian quantum dynamics: What does it mean? Europhysics Letters, 127(5), p.50001, 2019.

[12] S. M. Smith, K. L. Miller, G. Salimi-Khorshidi, M. Webster, C. F. Beckmann, T. E. Nichols, J. D. Ramsey, and M. W. Woolrich. Network modelling methods for FMRI. Neuroimage, 54(2): 875–891, 2011.

[13] Authors Anonymous. Benchmarking structural inference methods for dynamical interacting systems. https://structinfer.github.io/, 2023.

Q1. (pg. 4) You write that $\tau$ represents the length of the time window required for the system's dynamics to be considered Markovian. What do you precisely mean by this? Why would the time window affect whether or not the system is Markovian?

In the context of the paper, when it is mentioned that $\tau$ represents the length of the time window required for the system's dynamics to be considered Markovian, it is referring to the concept of Markovianity in continuous-time Markov processes [10]. In these processes, the future state of the system is determined solely by its current state and not by how it arrived in that state. This property is essential to simplify the modeling and analysis of complex dynamical systems.

The time window $\tau$ is crucial because it sets the scale at which the dynamics are examined. For a system to be considered Markovian, the choice of $\tau$ should be such that the memory of past states does not significantly influence the future states beyond this time window. In practical terms, this means that if you know the state of the system at time $\tau$, then this information is sufficient to predict its state at time $t+\tau$ without needing to know its earlier history.

The impact of the time window $\tau$ on the Markovian nature of the system can be understood as follows:

• If $\tau$ is too short, the system's evolution may still be influenced by its immediate past, not adhering to the Markovian property.
• If $\tau$ is appropriately chosen, it allows for the dynamics of the system to 'forget' its past states, thus adhering more closely to the Markovian assumption. This is critical for the effective application of Markov models and for simplifying the analysis of the system's dynamics. This could be the case of analyzing open quantum system which somehow expresses its non-Markovian characteristic [11].

Therefore, the correct selection of $\tau$ is vital to ensure that the system behaves in a Markovian manner, which is a fundamental assumption for the modeling techniques discussed in the paper. We added Appendix A.4 to state it.

---

Q2. What is the dimension of the embedding of node-level dynamics produced by the VDE? And what is the effect of the dimension on the performance of the algorithm?

In our original submission (Section 4.1), we detailed that the embedding dimension for node-level dynamics produced by the VDE is one. Specifically, for a given node feature $v_i^t$ (representing node $i$ at time $t$), the corresponding embedding $\mathbf{z}^t_i$ generated by the VDE also has a dimension of one.

This dimensionality is a critical aspect of our methodology, as the use of the partial correlation coefficient (PCOR) requires the input dimension for each node at every time step to be exactly one. This requirement presents a limitation in directly applying PCOR for structural inference with nodes that have multiple features at any time step. In such cases, our method is constrained by the necessity to process each feature individually due to the single-dimensional nature of the embeddings.

---

We thank Reviewer Fg5F for the constructive feedback and are pleased that our SIDEC framework's novelty and significance have been recognized. We address each of your concerns below to provide further clarity and detail.

W1. There are some missing details about key elements of the procedure. How are the dynamics modeled, and how does the adjacency matrix $A$ influence the dynamics? How is $A$ estimated from the embedded representations? It's hinted that this is done by the partial correlations formula, but it doesn't seem to be explicitly stated.

We would like to thank the reviewer for the constructive comments and pointing out the missing details. Here are our answers:

• How are the dynamics modeled?

  As mentioned in the second paragraph of Section 4.1 in our original submission, we use the Variational Dynamics Encoder (VDE) [1], guided by the Variational Information Bottleneck (VIB) principle [2], to model dynamics. The VDE operates on the principle:

  $\mathbf{z}^t_i = \arg\min_{\mathbf{z}^t_i} I(\mathbf{z}^t_i;v^t_i)-\mathfrak{u}\cdot I(\mathbf{z}^t_i; v^{t+\tau}_i),$ where $\mathbf{z}^t_i$ is the latent variable for node $i$ at time $t$, and $\mathfrak{u}$ is the Lagrangian multiplier that balances the trade-off between sufficiency and minimality. $v^{t}_i$ and $v^{t+\tau}_i$ represent the feature of node $i$ at time $t$ and $t+\tau$, respectively. In this work, we denote $\tau$ as the length of time window. The latent variable $z^r_i$ is utilized to model the dynamics of node $i$ at time $t$ with a refer of time window $\tau$. This approach minimizes the mutual information between the latent variable $\mathbf{z}^t_i$ (for node $i$ at time $t$) and the feature vector $v^t_i$, considering a time window $\tau$. In short, this method optimizes the trade-off between capturing relevant information and minimizing redundancy.

• How does the adjacency matrix $A$ influence the dynamics?

  According to previous works on structural inference [3, 4, 5], based on the perspective of dynamical systems [6], the adjacency matrix has a crucial influence on the node dynamics, that is, the node features evolve in a observable time period $0 \leq t \leq T$, is based on the following formulation:

  $V^{t+1} \leftarrow \\{V^0, V^1, ..., V^t; \mathbf{A}\\},$ where $V^t$ denotes the all node features at time $t$, and $\mathbf{A}$ represents the adjacency matrix of the graph, which represents the structure of interaction between nodes in the dynamical system. With the widely-adopted Markovian assumption on dynamical systems, we can further simplify the previous formulation into: $V^{t+1} \leftarrow \\{V^t;\mathbf{A}\\},$ where the future features only depend on the present features and the structure of the system. We may further instantiate the formulation above, and with a scope on every node: $v_i^{t+1} = v_i^t+ \Delta \cdot \sum_ {j \in \mathcal{U}_i}f\Big(||v_i, v_j ||\_{\alpha}\Big) ,$ where $\Delta$ denotes a time interval, $\mathcal{U}\_i$ is the set of nodes connected with node $i$, and $f(\cdot)$ is the state-transition function deriving to dynamics caused by the edge from node $j$ to $i$, and $||\cdot , \cdot||\_\alpha$ denotes the $\alpha$-distance. In this formulation, $\mathcal{U}\_i$ is actually derived from the adjacency matrix $\mathbf{A}$. So in all three formulations, we can see that adjacency matrix $\mathbf{A}$ plays an important role in the evolving of the system, it determines which nodes have interaction and thus make the whole system to be dynamic.

• How is $A$ estimated from the embedded representations?

  We acknowledge the oversight in our initial manuscript. The matrix $\mathbf{P}$, mentioned in Section 4.3, serves as the reconstructed adjacency matrix $\mathbf{A}$. We have revised Section 4.3 in the manuscript to explicitly state this.

---

Dear Reviewer Fg5F,

We hope you are doing well. As the deadline for the author-reviewer discussion phase is nearing, we would like to courteously inquire if our rebuttal has effectively addressed the concerns you raised in your review of our paper.

Your insightful feedback, especially regarding the setup of the proposed method, the scalability of SIDEC, and the datasets, has been invaluable in refining our work. We have endeavored to address each of these points meticulously in our rebuttal and are eager to know if our responses and subsequent revisions have met your expectations and clarified the issues you pointed out.

We recognize the demands on your time and deeply appreciate any further feedback you can offer on our rebuttal. Your expertise is not only vital for the review process but also enriches our ongoing learning and growth in this field.

Thank you once again for your time and invaluable insights. We eagerly await your response.

Best regards, Authors

Dear Reviewer Fg5F,

We deeply appreciate your acknowledgment of our detailed responses and are encouraged by your decision to increase the evaluation score. Thank you for your valuable feedback and support throughout this review process.

Regarding your suggestion on W1, we agree that explicitly including the equation $v_i^{t + 1} = v_i^t + \Delta \cdot \sum_{j \in \mathcal{U}i} f( || v_i, v_j ||_\alpha)$ in the manuscript will provide clarity on the model we assume. We have incorporated this equation into Section 3 to ensure the model's framework is clearly understood.

Once again, we extend our gratitude for your constructive feedback and for recognizing the enhancements made to our manuscript. Your insights have been instrumental in refining our work.

Warm regards, Authors

Dear Reviewer Fg5F,

We extend our sincere thanks for your constructive feedback and insights, which have been instrumental in refining our paper. Please find below our detailed responses to the issues raised in your last response.

W1. In response to your suggestion, we would like to further elucidate the role of of the adjacency matrix $\mathbf{A}$ in our model's dynamics. The influence of $\mathbf{A}$ can be understood through the following formulation: for node $i$ in the system, its feature at time $t+1$ is given by $v_i^{t+1} = v_i^t+ \Delta \cdot \sum_ {j \in \mathcal{U}_i}f\Big(||v_i, v_j ||\_{\alpha}\Big) ,$ where $\Delta$ denotes a time interval, $\mathcal{U}\_i$ is the set of nodes connected with node $i$, and $f(\cdot)$ is the state-transition function deriving to dynamics caused by the edge from node $j$ to $i$, and $||\cdot , \cdot||\_\alpha$ denotes the $\alpha$-distance. Importantly, $\mathcal{U}\_i$ is derived from the adjacency matrix $\mathbf{A}$, highlighting its pivotal role in determining the interactions between nodes and, consequently, the system's dynamics.

To illustrate this, let us consider the "Springs" dataset, used in our experiments. For node $i$ in this system, its 2-dimensional velocity vector at time $t$, denoted as $x_i'(t)$ is derived from the integration of its 2-dimensional accerleration vector $x_i''(t)$. This is calculated using the formula: $m_i\cdot x''i(t) = \sum_{j\in\mathcal{N}_i}-k\cdot\big(x_i(t)-x_j(t)\big).$ Here, $m_i$ represents the mass of the node, $k$ is the spring constant, and $\mathcal{N}_i$ is the set of neighboring nodes with connections to node $i$, which is determined directly by the adjacency matrix $\mathbf{A}$. We hope this explanation successfully clarifies the integral role of the adjacency matrix $\mathbf{A}$ in defining the dynamics within our model.

---

W2. We appreciate your recommendation to conduct a more straightforward comparison. We have gathered experimental data to compare the performance of SIDEC with full node trajectories (SIDEC(*Full)) and single-node trajectories. The table below presents our findings:

The results indicate that SIDEC(*Full) is consistently outperformed by SIDEC using single-node trajectories. Additionally, using full trajectories leads to scalability issues when facing graphs with more than 150 nodes. It also complicates the identification of latent variables corresponding to specific nodes. These findings justified our choice to focus on single-node trajectories in the VDE stage for efficiency and efficacy. We have included these results in a new section, Appendix A.5, in the revised manuscript.

---

Q1. We concur with your observation regarding the continuous nature of real processes. While the system may not be precisely Markovian, selecting an appropriate time window length can effectively limit dependencies on past states after sampling individual time steps. We have incorporated this consideration into our methodology.

We are deeply grateful for your comments and suggestions, which have significantly contributed to the advancement of our research. We hope our responses have adequately addressed your concerns.

Warm regards, Authors