DeepLag: Discovering Deep Lagrangian Dynamics for Intuitive Fluid Prediction

Special thanks to Reviewer Kve3 for their detailed review and insightful suggestions, your dedication to evaluating our work despite your busy schedule is greatly appreciated.

Weakness1: About the detailed content selected to represent in the main body of the paper.

Thank you for your valuable feedback. We acknowledge that the current presentation may pose difficulties for readers seeking detailed technical insights, especially regarding the experimental setup, dataset descriptions, and model architecture. Due to space constraints, our initial focus was on presenting the core ideas and results, with specific implementation details relegated to the appendix. We will address this in our revisions by integrating a comprehensive description of the experimental setup, dataset characteristics, and detailed model architecture directly into the main body of the paper.

Weakness2: Comparison with a Lagrange-only model.

Thank you for your comment. We appreciate your suggestion regarding including a comparison with a Lagrange-only model alongside existing field-based prediction models for a more comprehensive experimental evaluation, however, we have the following reasons not to do so:

Firstly, it's important to note that existing benchmarks $^{[1,2]}$ and real-world observational datasets, such as the Ocean Current dataset by ECMWF, predominantly represent Eulerian perspectives due to their ease of collection, representation, and storage.

Secondly, in practical applications like ocean studies, precisely tracking nearly infinite particles is often impractical. Therefore, there is a critical need for deep learning models that can effectively and efficiently extract Lagrangian information from Eulerian data in an unsupervised manner.

Thirdly, adhering to established conventions and baseline settings allows for a fair and transparent comparison among different models. As outlined in $\underline{\text{Lines 77-78}}$ of our paper, our research is focused on Eulerian data for fluid prediction, aligning with practical applications and existing benchmarks.

Regarding the comparison with a Lagrange-only model, it's important to clarify that such models operate purely on Lagrangian data inputs and outputs. This does not align with the specific problem statement and datasets used in our study, making it challenging to conduct a fair comparison.

In conclusion, while we appreciate the suggestion, our experimental design focuses on evaluating field-based prediction models within the context of Eulerian data. This approach ensures consistency and relevance to practical applications and existing benchmarks.

---

[1] Takamoto et al., "PDEBench: An Extensive Benchmark for Scientific Machine Learning", NeurIPS D&B 2022 [2] Gupta and Brandstetter, "Towards Multi-spatiotemporal-scale Generalized PDE Modeling", 2022

Q1: Comparison with one particle-based deep-learning model

Thank you for your inquiry. We appreciate your suggestion regarding comparing our approach with a Lagrange-Only model.

As outlined in Weakness2 of our paper, the decision not to include a comparison with a Lagrange-Only model stems from several considerations. Primarily, our research focuses on leveraging both Eulerian and Lagrangian perspectives within a unified framework, termed the Euler-Lagrange paradigm, currently on Eulerian data. This dual perspective allows for a more comprehensive understanding and prediction of fluid dynamics, which is the primary contribution and focus of our study.

However, we acknowledge the merit of your suggestion to include comparisons with models that solely operate within the Lagrangian domain. In response to your suggestion, we have contemplated a dual approach where Eulerian information supplements purely Lagrangian particle data for prediction tasks. This idea for proving the effectiveness of the Euler-Lagrange paradigm on Lagrangian data, while relevant and insightful, falls beyond the scope of the current paper but will be considered for future work. We appreciate your valuable input and will duly note it for further exploration.

Special thanks to Reviewer PRfM for their detailed review and insightful suggestions.

strength2: On the algorithmic complexity brought by the multi-scale design.

Please recall $\underline{\text{Table 6 in Appendix A.1}}$ that the number of tracking particles at each scale has an exponential decrease w.r.t the index of that scale, resulting in adding more coarser scales only bringing about a minor fraction of extra computing, while the overall complexity doesn't vary too much. For quantitative results, please refer to Weakness3 below.

Weakness1: Justification for certain statements.

Thank you for carefully reviewing our paper and sorry for the controversial expressions, however, you can rest assured that every word of this paper is written by ourselves not LLM.

• Regarding 'curse of dimensionality', we acknowledge your concern. While the curse of dimensionality itself refers to specific challenges in high-dimensional data spaces and may not directly cause high computational costs in fluid dynamics, we understand that computational challenges in fluid dynamics primarily stem from the complex numerical methods required to solve PDEs over large and intricate grids. We will revise the text to better clarify this distinction.

• Regarding the statement of CFL condition. We agree with you that the CFL condition is essential for ensuring numerical stability and accuracy in fluid dynamics simulations, regardless of the method employed. What we are trying to say is that Lagrangian approaches for Eulerian fluid prediction like Semi-Lagrangian method offer more flexibility in time step sizes (e.g. adaptive time stepping) than Eulerian methods. We will remove this claim for scientific rigor.

Weakness2: Reorganize the related works.

In the previous paper, we follow the survey$^{[1]}$, where 'Neural Solvers' indicates Physic-Informed Neural Networks (PINNs) and 'Neural Operators' refers to the models that map the input function space to the output space, such as FNOs.

Following your suggestion, we would rephrase category names to 'Classical ML methods', 'Physic-Informed Neural Networks (PINNs)' and 'Neural Operators'.

---

[1] Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications, arXiv 2022

Weakness3: More ablations.

As per your request, we conducted further ablations, yielding the following results:

Regarding (a) the number of scales, increasing the number of scales generally improves performance up to a point, after which diminishing returns are observed. As for (b) overall model size, increasing the latent dimension tends to improve performance, but too many parameters can be unnecessary for the model, as seen in the ablations.

Additionally, in response to your concern about comparing U-Net with a similar parameter count or runtime to DeepLag, please refer to $\underline{\text{Appendix E}}$. U-Net consistently exhibits significantly larger parameter counts across all datasets compared to DeepLag, rendering such experiments unnecessary. Furthermore, experiments scaling U-Net to match DeepLag's runtime show:

In conclusion, augmenting U-Net with additional convolutional layers and increasing the latent dimension shows that too many parameters can overwhelm U-Net, indicating a scalability limitation.

Q1: Elaboration on Figure 7 and Table 10.

Thank you for your inquiry regarding the discrepancy between $\underline{\text{Figure 7}}$ and $\underline{\text{Lines 345-346}}$, as well as $\underline{\text{Table 10}}$. Please note that $\underline{\text{Figure 7}}$ depicts memory usage, while $\underline{\text{Table 10}}$ and $\underline{\text{Lines 345-346}}$ refer to parameter counts.

Thus, the confusion arises from the difference in metrics presented. Specifically, U-Net is characterized by a larger number of parameters but a smaller memory footprint, as indicated in our findings. We acknowledge the need for clarity in distinguishing these metrics and will ensure that this distinction is clearly labeled in our revised manuscript.

Limitations: About Occam's Razor, practical interest and writing advice.

Many thanks for your valuable suggestion. Firstly, we want to highlight several properties of our model that correspond to your mentioned question:

(a) About implementation effort: DeepLag can be implemented solely based on Pytorch modules, which may not be as hard as you thought.

(b) About hyperparameter: The primary consideration is the total number of tracking particles in the finest scale. Ablations in $\underline{\text{Table 5 in Section 4.4}}$ demonstrate that performance generally improves with an increased number of particles. Therefore, you can decide this up to your computation resource.

(c) About efficiency: We have to say that DeepLag is slower than U-Net. However, the performance improvement brought by DeepLag can be acknowledged in some applications, such as wind tunnel test, which prefers high precision to efficiency.

In addition, the learned particle trajectory can be visualized to help research understand fluid dynamics.

Following your suggestion, we will extend the Conclusion to discuss more about strengths/applications and limitations/future work. One promising direction is to speed up DeepLag with advanced attention mechanisms, such as FlashAttention.

We would like to thank Reviewer PBwE for the detailed review and insightful suggestions.

Weakness1: Explanation of the performance difference between datasets. "Just wondering what is the reason between this discrepancy? Is the proposed method better than in short-term or long-term tasks?"

Thank you for your thorough review of our paper. The observed performance differences between the Bounded Navier-Stokes and Ocean Current datasets stem from the diversity in dataset characteristics rather than any weaknesses in our model.

The Bounded Navier-Stokes dataset represents a smaller scale with irregular variations, whereas the Ocean Current dataset spans larger scales with periodicity and oceanic averages. Therefore, for ocean data, achieving accurate predictions on average state can also yield excellent results over the long term.

Weakness2: About the boundary condition processing in the framework. "How does the proposed method handle the complex boundaries? e.g., how to impose different boundary conditions into the framework?"

Actually, DeepLag is very flexible at adapting to various complex boundary conditions. As depicted in $\underline{\text{Figure 9 in Appendix G}}$, whether the boundary is known or unknown, simple or intricate, the dynamic sampling module which is optimized with the pointwise variance of vorticity could recognize the boundary area, wavefront, wake and far-field zone from the features contained in the input distribution. By the way, for datasets whose boundary is known (like the pillars in Bounded Navier-Stokes), we could concatenate the boundary mask as a new channel into model input as a guide to enhance the performance of DeepLag and the baselines.

To verify the generalizing ability on the new domain of DeepLag, we ran a zero-shot test with the old model checkpoint on a newly generated Bounded N-S dataset which has a different number, position and size of obstacles. DeepLag still has a ~7% promotion w.r.t the best baseline, U-Net, on relative L2 (DeepLag: 0.203, U-Net: 0.217). The visual comparison ($\underline{\text{Figure 2 in Global Response PDF}}$) between the two models further shows that DeepLag adaptively generalizes well on new domains and handles complex boundaries well.

Sincerely thank Reviewer rCcn for the detailed review and insightful suggestions.

Weakness1: On the difference between DeepLag and FluidNet [Tompson et al, ICML 2017].

Thanks for your recommendation on related work and rigorous questions. We will cite FluidNet in the revised paper. However, we have to say that FluidNet is distinct from DeepLag:

The key design of FluidNet is replacing one step of classical method with a learned module implemented by a surrogate CNN for accumulation. However, in the Lagrangian perspective, it still employs the traditional numerical approach (Maccormack method).

In contrast, DeepLag represents an approach that integrates both Eulerian and Lagrangian within a pure deep learning framework. This allows for more generalized modeling of Lagrangian dynamics across multiple elements, demonstrating its capability as a fully deep learning-based solution capable of end-to-end autonomous learning.

We will revise the claim you concern to "explicitly combines Eulerian and Lagrangian in a deep learning framework" for scientific rigor.

Weakness2: Comparison between DeepLag and U-Net under similar running time.

As per your request and following $\underline{\text{Figure 7}}$, we conducted experiments on the 3D Smoke dataset that scaled up U-Net to make a comparison between DeepLag and U-Net under similar running time. Concretely, we add more conv layers into the standard U-Net and increase the latent dimension. The results show that too many parameters overwhelm U-Net, indicating that it has a shortcoming in scalability.

Q1: Visualization of the particle movements of the Bounded Navier-Stokes dataset.

Based on your request, we have visualized the particle movements on the Bounded Navier-Stokes dataset in $\underline{\text{Figure 1 in PDF of Global Response}}$. Given the dataset's complex and frequent motion patterns, we have plotted particle offsets between consecutive frames, which effectively reflect instantaneous particle movements.

As depicted, DeepLag still can learn intuitive and reasonable motion. However, there is a slight decrease in overall quality compared to Ocean Current, which may be because of lacking standard physical quantities. However, it is worth noticing that DeepLag performs best in this dataset, which verifies the benefits of learned Lagrangian dynamics in improving prediction.

Q2: Velocity field extraction from particle movements of the Bounded N-S dataset.

Extracting dense velocity fields from the particle perspective in the Bounded N-S dataset is challenging due to the sparse nature of particles, which is necessary to reduce computational complexity. However, key pathways of complex motion regions can be identified through adaptive sampling, which already proves valuable for understanding fluid dynamics.

Q3: Is the EuToLag attention an overkill?

In theory, relying solely on local velocity fields for particle updates seems sufficient. However, in practical fluid dynamics, the exact velocity of fluid motion is often unknown, and due to the incompressibility of fluids, disturbances from distant regions can influence local dynamics. Therefore, incorporating global attention mechanisms ensures comprehensive modeling, addressing scenarios where distant influences play a role. Take Ocean Current as an instance, the test set Relative L2 increases to 0.0264 from 0.0250 after removing the EuToLag attention (at the 20th training epoch).

Moreover, the EuToLag attention introduces minimal computational overhead ($O(n)$) and slightly increases training time (from 1030s/epoch to 1150s/epoch on Bounded N-S), while yielding noticeable performance improvements. Thus, we believe that EuToLag attention is not 'overkill'.

Q4: About the positional embedding.

We concatenate two additional channels to the input (three for 3D Smoke), representing normalized (x, y) coordinates (or (x, y, z) for 3D Smoke).

Q5: Generalization on new domains.

To verify the generalizing ability of DeepLag, we ran a zero-shot test with the old model checkpoint on a newly generated Bounded N-S dataset which has a different number, position and size of obstacles.

DeepLag still has a ~7% promotion w.r.t the best baseline, U-Net, on relative L2 (DeepLag: 0.203, U-Net: 0.217). The visual comparison ($\underline{\text{Figure 2 in Global Response PDF}}$) between the two models further shows that DeepLag adaptively generalizes well on new domains.

Limitations: On the scalability to larger domains of the fully-connected layer in the dynamic sampling module.

Sorry for the vague description. The "fully-connected layer" in $\underline{\text{Line 151}}$ refers to channel dimension MLP rather than spatial. Therefore, the quadratical complexity for a fully-connected layer doesn't exist.

Further, we trained a new model on a 256*256 Bounded N-S (4x larger domain) as requested, which still has a ~17% promotion w.r.t U-Net on relative L2 (DeepLag: 0.051, U-Net: 0.060), the comparison below also shows that the increase of the time complexity is minor, underscoring the scalability of DeepLag.

Many thanks to Reviewer UohQ for the detailed review and suggestions.

Weakness1: The model’s reliance on extensive hyperparameter tuning for particle tracking could pose challenges.

The only hyperparameter needed for tuning is the total number of the tracking particles in the finest scale. Please recall $\underline{\text{Table 6 in Appendix A.1}}$ that the number of tracking particles at each scale has an exponential decrease w.r.t the index of that scale. The ablation in $\underline{\text{Table 5}}$ shows that the performance always tends to incline as we simply increase this number (which means our DeepLag is scalable), therefore the hyperparameter choice depends more on the limitation of computing resources rather than blind searching.

Weakness2: The particle tracking mechanism struggles to maintain focus on particles near the domain borders.

We respectfully point out that our model works well on boundary particles. $\underline{\text{Figure 9 in Appendix G}}$ shows that DeepLag is apt to sample more points in the complex-dynamics area, which is near and behind the column obstacles. Plus, the red boxes in $\underline{\text{Figure 10,14 in Appendix H and Figure 16 in Appendix I}}$ indicate that DeepLag is good at capturing the fine details in the wake zone where turbulence and vortex form.

Weakness3: The paper contains a few grammatical errors.

Thank you for your thorough review of our paper and for bringing to our attention the grammatical errors that may detract from its professional presentation. We sincerely apologize for these inaccuracies and have already taken steps to correct them. Specifically, we will address errors such as the article misusage in $\underline{\text{Line 3 of paper}}$ ("an Eulerian perspective" corrected to "the Eulerian perspective") and the preposition error in $\underline{\text{Line 70 of paper}}$ ("tracking a certain particle with initial position $\mathbf{s}_0$ with its displacement $\mathbf{d} = \mathbf{d}(\mathbf{s}_0, \mathit{t})$" corrected to "tracking a particular particle from initial position $\mathbf{s}_0$ by its displacement $\mathbf{d} = \mathbf{d}(\mathbf{s}_0, \mathit{t})$"). These corrections are crucial for maintaining the professional standard of our presentation. Moving forward, we will conduct a comprehensive proofreading to ensure that all grammatical errors are identified and rectified.

Q1: About the difference between the two parts of Fig. 1

As indicated in the figure caption, Fig. 1 presents two distinct visualizations. The left part depicts the trajectories of Lagrangian particles overlaid on the mean state observed from the Eulerian perspective. In contrast, the right part displays successive Eulerian frames with scattered positions of tracked particles. These visualizations align with the description provided in $\underline{\text{Lines 42-43 of paper}}$, where the trajectories of fluid motion are more visibly represented through the dynamic Lagrangian view compared to the density variations observed in static Eulerian grids. This underscores our motivation to incorporate and study Lagrangian dynamics.

Q2: About the supervision over the Lagrangian space.

There is no supervision signal in the Lagrangian space due to the following reasons. Firstly, the widely used large-scale and fine-grained data in hydromechanics is all Eulerian since they're relatively easier to collect, represent and store. Secondly, it's impractical to track almost infinite particles precisely in real-world applications like ocean study, so there is a need for deep models to unsupervised extract Lagrangian information from Eulerian data effectively and efficiently. Thirdly, following the convention and the setting of baselines allow us a fair and clear comparison.

Q3: The result of interchanging the positions of the LagToEu and EuToLag blocks.

We conducted experiments as per your request, swapping the positions of the EuToLag and LagToEu blocks and validating their effects on both 2D (Bounded Navier-Stokes) and 3D (3D Smoke) datasets. The experimental results indicate that there was minimal change in performance, suggesting that the flow of information between Eulerian and Lagrangian perspectives is bidirectional. This insensitivity to the order in which information is transferred between the two perspectives underscores the robustness of our approach. These findings further support our statement in $\underline{\text{Line 73}}$ of the manuscript that "Two perspectives are constitutionally equivalent," which is substantiated both theoretically ($\underline{\text{Eq. (1) and (2) in Section 2.1}}$​) and intuitively.

Q4: Explanation on the ablation of the EuToLag attention.

The relatively small promotion of EuToLag attention can be interpreted as follows.

Firstly, EuToLag attention itself adds minimal computational overhead ($O(n)$) and only marginally increases the training time (from 1030s/epoch to 1150s/epoch). The primary computational load lies in the LagToEu module, which processes dense Eulerian data using patch embedding and patch recovery. Therefore, EuToLag attention does not significantly impact efficiency.

Secondly, in scenarios where precision demands capturing global information for effectiveness, integrating EuToLag attention remains justified. In this paper, we mainly focus on effective Eulerian-Lagrangian collaborative modeling and efficiency optimization is a promising future work.

Q5: Positions of the cylinders in the Bounded Navier-Stokes Benchmark.

The positions of the cylinders are fixed, but the initial condition varies in different samples, which can simulate a scenario like bridge pillars in a torrential river.