P-Align: Self-Alignment in Physical Dynamical System Modeling

1. While the method is well-motivated, a more in-depth analysis is needed to explain why improving physics consistency of the training samples leads to better prediction performance.

• A. Although the authors mention several metrics that can serve as physics-aware rewards, they lack implementation details, such as which metrics are applied to each dataset and how they are computed.
• B. A quantitative and qualitative comparison between the original data and the augmented data in terms of physics-aware metrics is necessary to demonstrate the effectiveness of the self-alignment process.
• C. Since the paper lacks training details (the number of alignment iterations and the training steps in each iteration), there remains a possibility that the performance improvements result from increased training schedule. To better understand the impact of physics-consistent training samples, two additional experiments should be provided: (i) a comparison with baselines trained under the same computational budget, and (ii) training models from scratch with samples at varying levels of physics consistency, which can be obtained from previous runs of P-Align and categorized based on the physics-aware metrics.

2. While P-Align is compared with other plug-in enhancement methods, none of these methods utilize the physics-aware metrics. To provide a more comprehensive analysis, it would be beneficial to compare P-Align with approaches that incorporate such metrics directly into their loss functions. Although these methods may face optimization challenges, as suggested in the Introduction (Line 53), experimental evidence is necessary to substantiate this claim.

1. There are lots of typos in paper:

• Line 18, the usage of "to provides" needs to be "to provide"
• Line 26, Dynamical systems "provides" -> Dynamical systems "provide"
• In the part of framework overview, "identify" -> "identifies", "find" -> "finds", "add" -> "adds", "use" -> "uses"
• Line 168, "a embedding space" -> "an embedding space"
• Line 180, "sample" -> "samples", $E\phi$ -> $E_\phi$
• Line 184, "we assumes" -> "we assume"
• Line 235, "scores is" -> "scores are", "such us" -> "such as"
• Line 269, "making ... our selection target" -> "making ... as our selection target"

These are very low-level mistakes, which should not occur in a good paper.

2. In line 170, $\mathcal{X}_t\in\mathbb{R}^{C \times H\times W}$, you need to introduce the meaning of the three dimensions. Does it stand for an image of the input? If so, what is the physical consistency and priors of image dynamics? I think you selected datasets are not applicable to your model. Those datasets, such as fluid modeled with graphs, may be better.
3. There are no ablation studies to verify the effectiveness of the proposed four modules.
4. The conclusion that better training data have lower generalization error upper bound is trivial. The proofs in the paper are unnecessary.

1. Many of the implementation details are not clear or missing, see my question 1,2,3

2. The method requires prior knowledge of physics constraints, which is not necessarily available in real-world scenarios. Also, this could be an unfair comparison with vanilla baseline, since the latter could be easily improved by adding a regularization term to penalize the physics violation.

3. My major concern is the problem setting in this paper. For dynamical/physics system forecasting specially with these PDE like inputs, using multiple past observation to predict the future is a commonly used and very effective way to improve the forecasting. While this paper seems not using these basics from dynamical systems, therefore the experiments could be less legit. see my question 4.

1. The authors' writing needs further improvement. There are many typos, such as "provides" in line 26, "anchor" in line 182, and "of in" in line 468. Additionally, the authors should reorganize their notation, especially in the "self-discovery" of Section 4.2, where the explanation of Z^{m}_{t} is difficult to understand. Furthermore, the descriptions of equations 6 and 11 seem contradictory.

2. In the authors' framework, the choice of physical metrics appears to play a significant role. Why did the authors select these metrics? Have they considered choosing other metrics?

3. Equations 25 and 26 do not seem to rigorously derive the conclusion of equation 27. Can the authors provide a more rigorous proof?

4. I noticed that the results for the TaxiBJ+ and SEVIR datasets in Table 1 do not match those reported in the original Earthfarsser paper. Can the authors explain the reason for this discrepancy?

5. The baseline methods compared by the authors do not seem to include the methods proposed in the original paper for DRS and FireSys. In this paper's baselines, the newer method MmvP, which is used for macro-motion trajectory prediction, appears unsuitable for physical scenarios like DRS and FireSys. Have the authors considered validating some more recent methods for modeling physical dynamics, such as HOPE [1] and PGODE [2]?

[1] Hope: High-order graph ode for modeling interacting dynamics, ICML 2023.

[2] PGODE: Towards High-quality System Dynamics Modeling, ICML 2024.