HelmSim: Learning Helmholtz Dynamics for Interpretable Fluid Simulation

Many thanks to Reviewer rfHe for providing a detailed review and insightful questions.

Q1: Omitted all the classic works from Computer Graphics.

Many thanks for the reviewer recommending these papers to us. But after carefully reading these papers, we regretly find that none of these papers can be applied to our task. Our paper predicts the future solely based on the observed physics quantities, and the ground truth velocity is unaccessible.

Considering these papers are partially related to our topic, we add a new paragraph naming as computer graphic for fluid simulation in the $\underline{\text{related work section of revised paper}}$ to discuss these CG-based works. Here are some key points.

Q2: The paper focuses on extremely short prediction horizons.

As we stated in Q1, we attempt to predict the future fluid solely based on one observed physical quantity, which may be different from your conventional understanding of the fluid simulation task.

Following the previous paper, we establish our experiment as follows:

Thus, we believe that our experiment setting is well-established and can well examine the model's effectiveness.

Reference: [1] Li, Zongyi, et al. "Fourier neural operator for parametric partial differential equations." ICLR 2021.

[2] Hao, Zhongkai, et al. "Gnot: A general neural operator transformer for operator learning." ICML 2023.

[3] Wang, Rui, et al. "Incorporating symmetry into deep dynamics models for improved generalization." ICLR 2021.

[4] Deng, Yitong, et al. "Learning Vortex Dynamics for Fluid Inference and Prediction." ICLR 2023.

Q3: Why the Helmholtz decomposition should be enforced throughout the network? Have the authors tried adding only a single layer at the end?

Note that the multiscale property is one of the foundation features of fluid. Thus, we attempt to capture the complex dynamics in multiple scales. Following the reviewer's suggestion, we also experiment with the case that only Helmholtz decomposition to the end layer as follows. We can find that our proposed multiscale learning is essential.

Q4: An ablation for comparable parameter sizes of different models.

Firstly, we want to emphasize that all the baselines are compared with their official configurations in their paper, which means FNO is proposed as a lightweight model.

To ensure a fair comparison, we enlarge the FNO parameter and compare it with HelmSim as follows. We can find that, even with sufficient model parameter, FNO still performs worse than HelmSim.

Q5: An ablation for HelmSim of different parameter sizes.

We here add results to compare the sensitivity to the number of parameters on the $64\times 64$ Navier-Stokes dataset. Considering GPU memory and input channels, we report the results of changing the channels of deep representations to half or twice the original channels. The results show that the original parameter number is an efficient and accurate setting.

We sincerely thank the reviewer for providing valuable suggestions. However, we think that there might exist some misunderstandings of reviewer in our experiment setting or scope. During the rebuttal period, we made a great effort to clarify our implementation details and ensure a fair and comprehensive comparison. We hope the reviewer can re-examine the scope of our paper.

Dear Reviewer,

We kindly remind you that it has been 2 days since we posted our rebuttal and there are only 3 days left in the reviewer-author discussion period. Please let us know if our response has addressed your concerns.

Following your suggestion, we have answered your concerns and improved the paper in the following aspects:

• We dived into 6 papers you recommended but regretly found that none of these papers can be applied to our task. To clarify the differences in experiment settings, we add a new paragraph in the related work section to discuss the CG-based works.
• We also clarify the experiment settings of our paper by listing previous papers that we followed.
• We added ablation of different parameter numbers, multiscale HelmDynamic Block to show our efficient and accurate design.

All of these results have been included in the $\underline{\text{revised paper highlighted in blue}}$.

Thanks again for your valuable review. We are looking forward to your reply and are happy to answer any future questions.

Dear Reviewer,

Thanks again for your valuable and constructive review, which helps us clarify the relation with CG-based methods and improve the experiment with detailed ablations. All the updates have been included in the $\underline{\text{revised paper highlighted in blue}}$.

We kindly remind you that the reviewer-author discussion phase will end in a few hours. May we know if our response addresses your main concerns? After that, we will not have a chance to respond to your comments.

Sincere thanks for your dedication! We are looking forward to your feedback.

Dear Reviewer,

We kindly remind you that the 12-day reviewer-author discussion only left 1 hour.

In the rebuttal, we have answered your concerns by clarifying the relation between HelmSim and CG-based methods, providing new ablation studies on model parameter and multiscale design. All the updates have been included in the $\underline{\text{revised paper}}$, including 8 pages of new results and analysis.

Many thanks for your dedication in reviewing paper, which helps us a lot in imporving our work. Would you please kindly let us know if our response addresses your main concerns? We eagerly await your reply.

Many thanks to Reviewer 2nz9 for providing the insightful review and comments.

Q1: It is unclear what the observed state is at all.

Following the reviewer's suggestion, we have added the observed state information in $\underline{\text{Appendix A.1 of revised paper}}$.

Q2: Are these compared over networks trained on various random seeds, or how was this single value chosen?

Sorry for this missing information. We repeated all the experiments three times with seeds randomly selected from 0 to 1000. The presented results are the mean value of three repeated experiments.

Experimentally, the standard deviations of relative L2 are smaller than 0.001 for Navier-Stokes, Bounded N-S and Sea temperature and smaller than 0.003 for Spreading Ink. For scientific rigor, we keep four decimal places for all results. We have included this in $\underline{\text{Appendix A.2 and A.3 of revised paper}}$.

Q3: More details about boundary condition inclusion.

As we presented in $\underline{\text{Eq. (4) of original submission}}$, we add the boundary condition to the correlation calculation process. A detailed description is also added in $\underline{\text{Appendix A.2 of revised paper}}$.

Q4: Typo of the last sentence on page 4.

Thanks for the suggestion. We have corrected this typo in the $\underline{\text{revised paper}}$.

Q5: Explanation of Figure 9: Efficiency comparison.

Sorry for the misleading y-axis label. The y-axis is Relative L2, which means that a smaller value refers to better performance. We have rephrased this figure in the $\underline{\text{revised paper}}$.

Q6: FNO with roughly 9x less parameters to train on compared to HelmSim. Compare HelmSim to FNO with similar model size.

Firstly, we want to emphasize that all the baselines are compared with their official configurations in their paper, which means FNO is proposed as a lightweight model.

Following the reviewer's suggestion, we enlarge the FNO parameter and compare it with HelmSim as follows. We can find that, even with sufficient model parameters, FNO still performs worse than HelmSim.

Thank you to the authors for the extensive replies to all the reviewers, and for grouping the concerns into specific categories.

Thanks for including so much extra detailed information in the appendix, that will ensure reproducibility and clarity of the paper. Indeed, the comparison with FNO is very much appreciated, and now clearly shows the improvement of the proposed HelmSim approach. The additional section to 3D is perhaps slightly weak, since as the authors mentioned, it is a larger challenge within the community, and it would have been a great addition to see realized within this paper.

On the topic of how directly learning the velocity "overwhelms" the model, the way this could be understood is that we craft a simpler representation where the solution is easier to learn. Which goes into engineering features for neural network representations. In this case it is interpretable, yet it remains unclear if this decomposition-based representation is either the easiest to learn or the most physically interpretable. Where one would rather use dimensionality reduction techniques for the first, and potentially directly learn velocities and pressures for the latter. This is only a small comment to add, and perhaps a bigger problem challenge to tackle in the future.

One concern still remains, and perhaps has been amplified by the extra experiments, that the improvement feels somewhat incremental. Same with learning velocity directly, where it is a slight trade-off of runtime/memory for accuracy. It would be great if the model outperforms all baselines by a small amount, but excels in specific scenarios. This one specific scenario is what is missing to make this an excellent paper as opposed to a good paper.

However, the answers from the authors, in addition to the extensive benchmarks, have proven satisfactory and convinced me to improve my score. Thank you once again for clarifying a lot of the points,

We would like to sincerely thank Reviewer Mqos for providing valuable feedback.

Section 1: Source of Improvement and Ablation Study

Q1: Whether the Helmholtz decomposition is the primary source of the observed performance improvement?

We added new ablations on with or without Helmholtz Dynamics and with or without multihead, here are the results:

It is obsevered that the improvement brought by learning Helmholtz Dynamics is much more significant than the improvement brought by multihead design.

Q2: The ablation studies need to be explained more comprehensively with sufficient details.

Following the reviewer's suggestion, we have rephrased the $\underline{\text{Appendix B of revised paper}}$ with newly added implementation details for each ablation.

Q3: Why Vortex cannot be applied to other datasets besides Spreading ink dataset?

Vortex models multiple vortex trajectories as a function of time. Since different video sequences have inherently different vortex trajectories, we need to re-train Vortex to fit every video sequence. However, the other three benchmarks, for example, the Navier-Stokes dataset, have more than 1000 different video sequences. It means that we need to train 1000+ Vortex models for these benchmarks, which is unacceptable. But as per your request, we still implement this experiment in $\underline{\text{Table 15 of revised paper}}$, where we train one vortex model on one single video sequence and generalize it to others. Here are part of the results. We can find that Vortex degenerates a lot.

Section 2: Interpretability Claim

Q4: Does the Helmholtz decomposition significantly improve interpretability compared to baseline models?

As we keep emphasizing in defining "Helmholtz dynamics" in $\underline{\text{Section 3.1}}$, HelmSim not only learns the velocity field but also estimates the potential and stream functions of fluid. We believe that this design surpasses other baselines in learning "physically interpretable evidence", not just the velocity fields.

Q5: In Figure 4, the model predicts non-zero velocity outside the boundary.

Sorry for this misleading visualization. Since the Spreading Ink dataset is only within the circle boundary, we only compute the loss within the boundary. Thus, the part outside of the boundary is without restriction.

We have added the implementation details in the $\underline{\text{Appendix A of revised paper}}$.

Section 3: Multiscale modeling

Q6: The aggregation operation after "Integration" needs further clarification.

We conduct this operation following U-Net. As per the reviewer's suggestion, we have included details in the $\underline{\text{Appendix A.2 of revised paper}}$.

We would like to sincerely thank Reviewer eKS7 for providing a detailed review and insightful suggestions.

Q1: How will you generalize the proposed method for 3D cases?

According to the official formalization of Helmholtz decomposition ${\mathbf{F}}(\mathbf{r}) = \nabla\Phi(\mathbf{r}) + \nabla \times {\mathbf{A}}(\mathbf{r}), \mathbf{r} \in \mathbb{V}$, the stream function for a 3D fluid field is a 3D vector ${\mathbf{A}}(\mathbf{r}) = (\mathbf{A}_x(\mathbf{r}), \mathbf{A}_y(\mathbf{r}), \mathbf{A}_z(\mathbf{r}))$. In 2D cases, we set the the velocity component on the $z$-axis to zero, that is $\mathbf{A}_x(\mathbf{r})=\mathbf{A}_y(\mathbf{r})=0$. To extend HelmSim to 3D fluid, the key point is to change the HelmDynamic block into learning 3D potential and stream functions $\widehat{\Phi}\in\mathbb{R}^{1\times D \times H\times W}$ and $\widehat{\mathbf{A}}\in\mathbb{R}^{3\times D \times H\times W}$, where $D$ is the additional depth dimension of 3D fluid. Then, following the Helmholtz decomposition presented in $\underline{\text{Eq. (1) of revised paper}}$, we can easily obtain the inferred 3D vector velocity field, thereby enabling HelmSim to achieve the velocity-aware 3D fluid prediction.

Q2: What are the Reynold numbers for the cases? How is your method's performance on turbulence?

Sorry for this missing information. The Reynold number for the Navier-Stokes dataset is about $10^4$, and for the Bounded N-S dataset is about 300.

To better evaluate the performance of the turbulence dataset, we experimented with a turbulence dataset provided by TF-Net [1]. We strictly followed its setting and reported the RMSE for predicting future 20 timesteps. In this dataset, Helmsim still perfroms best. These results have also been included in $\underline{\text{Appendix D of revised paper}}$.

Reference:

[1] Wang, Rui, et al. "Towards physics-informed deep learning for turbulent flow prediction." KDD. 2020.

Q3: Missing movies for the predictions to check for temporal coherence.

Following the reviewer's suggestion, we have provided the prediction movies in the $\underline{\text{revised supplementary material}}$, where HelmSim achieves favorable temporal coherence.

Dear Reviewer,

Many thanks for your valuable and detailed review.

Following your suggestion, we have answered your concerns and improved the paper in the following aspects:

• We have added 1 new dataset (Turbulence) for all 7 models including TF-Net (KDD 2020) to demonstrate our performance on turbulence.
• We recalled and rephrased the ablations provided in the original submission to demonstrate the effectiveness of our design for boundary condition, multiscale and multihead.
• We clarified the generalization direction of HelmSim to 3D fluid and highlighted that most of the recent neural fluid solvers only focus on the 2D fluid simulation, such as our baselines FNO, LSM, MWT, U-NO, and Vortex. 3D fluid simulation is a quite challenging problem, and we leave it to future work.
• We added movies of predictions in supplementary materials for intuitive comparison.

In total, we have added more than 60 new experiments. All of these results have been included in the $\underline{\text{revised paper}}$.

We kindly remind you that the reviewer-author discussion phase will end in 24 hours. After that, we may not have a chance to respond to your comments.

Sincere thanks for your dedication! We are looking forward to your reply.