Symmetric Basis Convolutions for Learning Lagrangian Fluid Mechanics

We appreciate your attention to the computational aspects of our work and have revised the paper to include a discussion of computational performance and a three-dimensional test case. In this response, we will provide additional context to your specific questions and weaknesses mentioned. Furthermore, we have uploaded a new revision of the paper, including the feedback of all reviewers with changes highlighted in purple.

Regarding three-dimensional systems: We have added an additional three-dimensional test case in Appendix Sec. C.5, analogous to test case I in Sec. C.1.1 and Appendix Sec. C.1.2, see Section 4.1 of the main paper for a discussion of the results. This additional test case demonstrates how insights from one and two dimensions, e.g., regarding using symmetric and antisymmetric bases, also transfer to three dimensions and how our proposed SFBC approach is also more resilient to non-ideal training setups.

Regarding the intermediate Matrix B: The size of the intermediate matrix is certainly a concern and is a potential limitation even in two spatial dimensions for larger particle counts as the size scales with n x u x v x w (for n edges and three spatial dimensions using u,v and w weights per dimension). However, in our backend, which will be published alongside the paper, we perform this computation in a batched format by splitting the set of edges into separate batches of size b. This limits the size of the intermediate matrix to b x u x v x w. However, this would still require storing potentially large matrices for backpropagation, which we avoid by recomputing the components of the outer products using a custom backprop operation. Accordingly, we only need to store the relative positions and input features for backpropagation, which allows us to train our network, even in three dimensions, using very low memory requirements, e.g., less than 4GByte. Note that some implementations of this Einsum operation can perform the operation without storing explicit intermediates, e.g., Nvidia’s Cutlass library. We have expanded the discussion Appendix Sec. A.1 to better reflect this.

Regarding computational evaluations: We have added a runtime performance evaluation in Appendix Sec. C.6, which evaluates a broad set of hyperparameter choices, including basis functions, coordinate mappings, and network architectures. In this evaluation, we found that, on average, the performance difference between different basis functions is very low, i.e., less than 7% between methods (see Figure 38), with a similar scaling of computational costs between methods based on the number of basis terms, i.e., O(n^d) for d dimensions and n basis terms (see Figure 36). We also found a very low overhead for coordinate mappings and window functions. Overall, the performance overhead of using SFBC over LinCConv is small for the networks we evaluated. We briefly discuss these results in Section 4 of the main paper.

Furthermore, we trained our networks with a fixed number of iterations per training and, thus, a comparable overall computational cost. Based on our initialization tests, see Appendix Sec. C.3.5, we consider this training strategy to perform well for our given tasks. Naturally, network initialization substantially impacts training, and we have made best efforts to provide all networks with equally good initializations. However, we see improved methods for initialization of the different convolutional architectures as an important component to be improved in future work. We have highlighted this as a limitation in the Conclusions.

We appreciate the feedback provided and have revised the paper accordingly. In this response, we will address the questions posited and the weaknesses mentioned in your review. Furthermore, we have uploaded a new revision of the paper, including the feedback of all reviewers with changes highlighted in purple.

Regarding temporal unrolling behavior: We apologize for the omission and agree that it is a good idea to provide sequences that show the temporal behavior of the different methods and will include examples alongside the source code and data repositories upon publication. You can find an example of the temporal unrolling behavior for LinCConv with and without a window function and an example of our SFBC approach with and without a window function in the supplementary material. These gifs clearly highlight the difference in temporal behavior, i.e., how our Fourier-based approach yields a significantly lower and more temporally stable behavior than LinCConv

Regarding the outer product: Using an outer product can certainly be a limitation in performance as the number of weights of a filter scales with n^d (n basis terms, d dimensions); however, our approach shares this scaling behavior with regular convolutions and with prior work, e.g., Fey et al. [2018] and Ummenhofer et al. [2019]. Based on prior work and our own results in 3D, see Appendix Sec. C.5, this is not yet a limitation in 3D simulations, which are commonly used in fluid mechanics and other simulation problems. It could, however, become a limiting factor for convolutions in higher dimensions.

Regarding computational overhead: We have included a runtime evaluation in one, two, and three dimensions for different architectures and hyperparameters in Appendix Sec. C.6 (Figures 36 and 38) . This evaluation shows that for most basis term counts, especially in three dimensions, the computational requirements are very similar between the different basis functions, i.e., they are within 7% of each other. For one dimensional networks, we were able to observe a worse performance of our proposed approach and the Chebyshev basis, as the other basis functions are limited in computational performance by other parts of the network, but only for 16 and 32 base terms. For larger networks, we also observed the expected scaling of O(n^d), with n being the number of basis terms and for d dimensions. We added a reference to these results in Section 4.

Regarding differentiation from prior work: Fey's method and similarly LinCConv by Ummenhofer both used a separable basis, with the latter not being aware that their Linear Interpolation is separable. We expanded this concept to a generic formulation of separable bases and showed how having a generic framework to construct continuous convolution layers. This allows for readily choosing the best basis function for a specific task by investigating bases with different inherent properties, e.g., symmetries and smoothness.

Note that with our framework a much broader set of convolutional filters can be explored that are, for example, rotationally symmetric instead of just axisymmetric. Such a filter can be constructed, e.g., by using a spherical coordinate mapping with a single constant filter for the angular component and a different basis, e.g., a Chebyshev base, for the radial components. We have found promising initial results in this direction, but this opens up a much broader hyperparameter space that is beyond our current scope and computational resources. We have added this as an outlook in the Conclusions in Sec. 5. As an example of using such a combination consider the following example of using a combination of a linear basis for the radial component and a cubic spline basis for the angular component: https://i.imgur.com/iaZCnxy.png (uploaded as an anonymous user so double blind reviewing is preserved properly) .

Regarding Limitations: We have revised the conclusion section of the paper to highlight the limitations of our proposed method. These limitations include (a) investigating other physical systems and simulations outside of SPH, (b) exploring more complex architectures based on continuous convolutions, and (c) a lack of exploring a large space of hyperparameter choices that seem potentially relevant for future work, e.g., using dissimilar bases within a single network. Our aim was to provide a framework and a thorough investigation of how continuous convolution kernels can be built, and correspondingly, these topics are very interesting directions for follow-up work. We plan to explore some of these avenues and hope that our submission and source code will likewise inspire follow-up work by others.

We appreciate your evaluation of our submission, highlighting some important aspects we have addressed in the revised manuscript. In addition to the explicit questions, this response also addresses the choice of the datasets mentioned as a weakness. Furthermore, we have uploaded a new revision of the paper, including the feedback of all reviewers with changes highlighted in purple.

Regarding the datasets: We agree that the Liquid3D and Waterramps datasets are useful datasets for machine learning in general, but we chose to generate our own datasets built on a more numerically quantifiable approach. For example, consider the Liquid3D dataset with random shapes colliding in random directions and the error being evaluated using particle distances. While this evaluation is useful from a data science perspective, it is not obvious how the error being evaluated relates to the actual fluid mechanics or the viewer's perception. However, when constructing the datasets, primarily for test cases I, II, and IV, we intentionally chose settings with narrow scopes that have clearly defined and expected behavior. Instead of evaluating particle distances, we can directly evaluate the resulting flow field regarding density, velocity and divergence. We have added this explanation to Appendix B.

Regarding the basis questions: When talking about anti-symmetry beyond one-dimensional cases, we refer to antisymmetry with respect to the origin, i.e., the classical physical antisymmetry of f(\bf{x}) = -f(-\bf{x}). The basis functions we use are defined over the domain [-1,1]^d, i.e., the unit cube in 3D, where the relative distance of particles, i.e., the edge distances, are first scaled inversely by the particle support radius and then mapped using an optional coordinate mapping, analogous to the work of Ummenhofer et al. [2019] regarding the orthogonality of the combined basis, we have included proof of this property in Appendix A.1.

Regarding the DMCF approach: Prior to submission, we consulted with the original authors of the DMCF approach and decided to implement their approach as described in the paper based on these discussions. The DMCF approach used the Open3D backend for their convolutions and placed the antisymmetric constraint on top of this backend. Accordingly, this required several workarounds, i.e., the explicit mirroring of weights creates a disconnect between computed gradients, which assume the weights to be real, and the actual gradients. For our implementation, we used our own pytorch-geometric-based backend that will be published alongside the paper, and, accordingly, we could implement the antisymmetry requirement directly in the backend. In our implementation, the learned filter is still constrained to be antisymmetric (see Eq, (44)), and all weights contribute to the learned filter, whereas the original DMCF implementation was constrained to be antisymmetric but only half of all weights contributed to the learned filter. We have clarified this in Section 3.

Regarding the Fourier basis: We did use the bases directly. For an explicit equation of SFB, see Eq. (53) in Appendix Sec. A.4. We believe that the results for the purely antisymmetric and symmetric terms in 1D (and now 3D) show that the combined basis can already learn purely antisymmetric and symmetric problems with the current learning setup, albeit at a higher error than a purely even or odd basis shown in Figure 2 (which is the expected behavior). Furthermore, the odd Fourier basis only utilizes antisymmetric terms in 1D and is significantly smoother than the DMCF approach, resulting in the lower observed error in Figure 2.

Ummenhofer et al. [2019] - Benjamin Ummenhofer, Lukas Prantl, Nils Thuerey, and Vladlen Koltun. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Representations, 2019

We appreciate the feedback provided and have revised our paper to address the weaknesses mentioned. While there were no specific questions, we would still like to address and clarify the weaknesses you pointed out. Furthermore, we have uploaded a new revision of the paper, including the feedback of all reviewers with changes highlighted in purple.

Regarding theoretical foundations: We have added a short proof that shows that the separable basis construction of our work preserves orthogonality, i.e., if two orthogonal bases b_i and b_j are chosen, then the resulting two-dimensional basis is orthogonal in 2D, and analogously in 3D, see Appendix sec. A.1.

Regarding the evaluations: We have added an additional test case that is constructed analogously to test case I but performs the evaluations in 3D instead of 1D, which highlights how some insights, e.g., how symmetric bases help with learning even symmetric functions, remain consistent across dimensions, see Section 4.1 in the main paper and Appendix C.5.

Regarding more diverse physical systems: Our paper currently uses a compressible, an explicit weakly compressible, and an implicit incompressible simulation approach with very different requirements for numerical solvers and their respective stability. This provides a broad basis of evaluation for our network, albeit admittedly limited to particle-based hydrodynamics. We believe that exploring other physical systems and areas of application will be an exciting and promising direction for future work.