Geometry-Informed Neural Networks

We thank the reviewer for their positive feedback. We would merely like to answer the question concerning the extension of the method to constraints that require running a physics solver. Conceptually, this is straightforward by adopting a solver-in-the-loop [1], which we already do with the persistent-homology (PH) constraint. Loosely speaking, such a solver maps an input geometry to a differentiable scalar objective by discretizing the domain and solving an algebraic problem. This high-level procedure is the same for a PDE solver. Our framework can reuse PDE solvers from classical (differentiable) topology optimization, e.g., [2].

We are currently developing the use of a PDE solver with the proposed framework in a follow-up work. Despite the conceptual similarity, there are technical differences; for example, PDE solvers are generally more expensive, require more careful discretization, and have more complicated loss landscapes, which is the reason we chose to omit this discussion in this submission.

[1] Um, K., Brand, R., Fei, Y., Holl, P., and Thuerey, N. Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers. Advances in Neural Information Processing Systems, 2020.

[2] Jia, Y., Wang, C., and Zhang, X. S. FEniTop: a simple FEniCSx implementation for 2D and 3D topology optimization supporting parallel computing. Structural and Multidisciplinary Optimization, August 2024.

We thank the reviewer for the detailed and balanced review. In the following, we address the main questions.

Q1

The reviewer writes that while the experiments look promising, their scope is insufficient and, in particular, only two studied problems are relevant to the core research question. We believe the first four problems do support the central theme by isolating two key aspects: shape optimization with NN representations (Plateau and mirror) and training a generative model through a diversity constraint (physics and obstacle). We invite the reviewer to view these tasks as ablations of solution multiplicity and shape representations. The reviewer might also find relevant the rebuttal pdf (linked below) and the discussion with reviewer 8unU, where we provide a matrix overview of all problems and tasks and discuss their variety. However, we do agree that we can strengthen the existing problems with more ablations, which we do under Q3.

Q2

While the reviewer is right that our method does not come with convergence guarantees, this is the case for most adjacent methods in machine learning, physics-informed learning, and topology optimization. The convergence is a complex interplay between the model, the problem, and the optimizer, we do not foresee that such guarantees are easily obtained for the investigated non-linear differential losses on moving domains.

Concerning the desired distribution, our best mental model is the Boltzmann distribution induced by the objective function over the feasible set. As described in Section 2.3, this distribution is over a function space which prohibits the direct use of Boltzmann generators (BGs) and motivates the use of an explicit diversity term that plays the role of the entropy term in BGs. Since this is an unexplored setting, we also call for further investigation under limitations and future work.

Q3

We have performed additional studies ablating each constraint for three tasks (obstacle, wheel, bracket) with the metrics collected in the rebuttal pdf. The majority of these results quantitatively confirm the obvious roles of the constraints (e.g., interface losses improve interface metrics, etc.). The diversity ablation reduces the diversity in the bracket and wheel solutions roughly five and ten-fold, respectively. Notably, diversity and smoothness losses are competing (less diversity allows for lower smoothness). The reviewer might also be interested in how these ablations impact the computational speed discussed in the response to reviewer 9JbR.

We also include the ablation of the augmented Lagrangian. For the simpler problems, the constraint satisfaction is similar. For the more complex bracket problem with diversity and smoothness terms, we find that ALM finds more diverse shapes with more latent structure. This is best summarized visually (this 3x3 plot of ALM ablation can be juxtaposed with the 5x5 plot in Figure 6): while the overall structure of the shape is learnt, we see almost no interesting diversity or latent structure, and there are floaters and spurious surface features. We believe this is because ALM helps provide a stronger training signal throughout the training by adaptively increasing the weight of each loss. For some losses, the Lagrangian weight changes over two orders of magnitude.

Q4

We thank the reviewer for the additional references.

Other comments

The reviewer writes that the “system is fragile and hard-to-tune”. We must briefly comment that the wheel and bracket experiments share the exact same setup. While we chose simpler models for simpler tasks, all experiments (including the generative PINN task which is very sensitive to initialization) can be repeated with WIRE. However, in under-determined problems (those without objective, specifically, obstacle), the inductive bias of the model impacts the identified solutions, so the reviewer is right that there is a setup-dependence in this sense. Correspondingly, the low-frequency biased softplus MLP provides smooth and visually pleasing solutions, motivating such a choice. The reviewer might appreciate the following plot illustrating the role of inductive biases and diversity in connection to Q3. We are open to adding this discussion and experiments to the manuscript if the reviewer suggests this would strengthen our work.

Lastly, the reviewer asks how we could incorporate ground-truth data. Conceptually, this is simple by adding a supervision loss. Using auto-decoder style training, the model should learn to organize the ground-truths in the latent space (imagine placing ground-truth shapes in Figure 6). Alternatively, the latent positions of the ground-truths can be enforced with a specific interpolation in mind (e.g., vertices of a hypercube). We believe this is a very exciting future direction.

We thank the reviewer for their detailed review. In the following, we will address the questions.

Sharp features

The method is not limited to smooth surfaces. The smoothness of the surface is inherited from the NN used to represent the shapes. The smoothness properties of MLPs are governed by the activation function. We use at least $C^2$ activations to have twice continuously differentiable surfaces, for which the Hessian and, hence, curvatures are well-defined. If this is not required, sharp features can be easily represented with, e.g., ReLU MLPs. We will also add that the log-surface-strain objective promotes sharp features that can be approximately represented with $C^2$ NNs, such as WIRE (see top row in Figure 1 or this image for a close-up).

Impact of different constraints

Per the reviewer’s inquiry, we have done a more comprehensive constraint ablation study collecting the time per iteration and metrics. The timings reported below are for the 3D jet engine bracket problem, which has the highest number and complexity of constraints. The other experiments show similar behaviour, so we omit these for brevity but can include them upon request.

Most constraints have little effect on the time per iteration. Of the total runtime, the surface strain takes ~10% (due to the Hessian) and the PH solver ~75% (expensive multi-processed CPU task). To pre-empt a potential confusion – the runtime increases when ablating the eikonal constraint as it destroys the geometric regularity of the implicit function, hindering efficient surface point sampling that usually takes ~15% of total time. We use an A100-SXM GPU, and the peak memory usage for a batch of 9 shapes does not exceed 16 GB.

These two losses also have a strong impact on the iterations needed. As we describe around line 430, adding the smoothness loss increases the number of iterations roughly two-fold, and adding the diversity increases it roughly five-fold. The rebuttal pdf also collects the impact of these ablations on the metrics.

Overall, the biggest effect is not so much the number of constraints, but rather losses that are ill-conditioned as described in “Limitations and Future Work”. This conditioning issue is also known in the PINN literature, and its mitigation is an open and active research topic.

Additional examples

We would first like to clarify a potential misunderstanding that “almost all qualitative examples in the paper use one particular set of constraints (Figures 3 and 6)”. Figures 1, 3, and 6 illustrate the same jet engine bracket problem for presentation continuity. However, we also solve five other problems illustrated in Figures 4 and 5 and discussed in Section 4.2. We have prepared a visual overview of the problem-constraint matrix in the rebuttal pdf. If this does not answer the reviewer’s question for “additional qualitative examples beyond Figure 3”, we kindly ask for clarification.

Failure cases

A few failure cases are discussed in the Appendix of the submission. For example, Figure 10 shows the bracket trained with softplus, SIREN, and WIRE, demonstrating the importance of the model’s inductive bias. Around line 329, we also discuss the impact of the surface sampling strategy, which should ensure that point samples are distributed uniformly and that high-curvature regions are not missed. We observed that naive sampling strategies lack these properties and lead to ridge artifacts that contain all the curvature but are missed during the optimization. We provide a basic illustration here. We discuss another failure mode around line 415: the emergence of latent space structure is sensitive to the diversity constraint.

One could also view the ablations as failure cases. Appendix B.2 and the additional ablation study now contain more examples. We are glad to produce additional figures illustrating these failure modes if the reviewer suggests this would further strengthen the submission.

We thank the reviewer for their considerate review. In the following, we address the three weaknesses.

Task diversity

While we agree that more experiments are almost always better, we would like to present a matrix of problems and constraints that illustrates the variety of considered settings, available in the rebuttal pdf, together with an additional ablation study. Not captured by this matrix are also the differences in domain dimension, shape representation, and problem symmetries (e.g., the jet engine bracket has no symmetries, making this a very general problem). Since this is an unexplored research setting, unfortunately, there does not exist a catalogue of standard problems, so defining each new problem in addition to the main research question is a considerable effort. However, we agree this is necessary to develop this research direction further and hope this is addressed in future work.

Baselines

While the reviewer and we seem to agree that a fully satisfactory comparison to a baseline is difficult to produce, we have followed the reviewer’s advice and implemented at least partially comparable baselines.

Topology optimization

As suggested, we consider classical TO, specifically FeniTop [1] which implements the standard SIMP method with a popular FEM solver. We define a TO problem that is as similar as possible, applying a diagonal force to the top cylindrical pin interface and allowing a 7% volume fraction in the same design region. The other interfaces are fixed in the same way. The shape compliance is minimized for 400 iterations on a 104x172x60 FEM grid (taking 190 min on a 32 core CPU to give a sense of runtime, although a fair timing comparison requires a more nuanced discussion). The produced shape is visualized here.

Then, we compute the surface strain (the objective we use) for this TO shape and, conversely, the compliance for a GINN shape (Section 4.2; illustrated in the penultimate column of Figure 3). Unsurprisingly, both shapes perform best at the objective they were optimized for while also satisfying the constraints up to the relevant precision. At the very least, this serves as a sanity check and confirmation of the constraint satisfaction.

Human-expert dataset

A unique aspect of GINN is the data-free shape generative aspect. Comparison to classical TO is trivial since it is inherently limited to a single solution, and its diversity would be 0. Instead, we use the simJEB [2] dataset to give some intuitive estimate on the diversity of the produced results. The dataset is due to a design challenge on a related problem described in Section 4.2. The shapes in the dataset are produced by human experts, many of whom also used topology optimization. To compute the diversity metric, we sample 196 clean shapes from the simJEB dataset and 14x14 equidistant samples from the 2d latent space generative GINN model.

Even though these sets are not directly comparable as they optimize for different objectives, this comparison indicates that GINNs can produce diversity on the same and larger magnitude as a dataset that required an estimated collective effort of 14 expert human years [2].

Computational cost

We thank the reviewer for highlighting the runtimes, which made us realize that we have reported an outdated runtime in the submission. The up-to-date runtimes on an A100-SXM GPU are 30 min for 10k iterations of a single-shape model and 4.5h for 50k iterations for the generative model, not 72h, as mistakenly reported in the submission. This large discrepancy was due to previously logging plots at every training iteration.

Since reviewer 9JbR also asks about runtimes, we invite the reviewer to read our other rebuttal, where we report a more comprehensive constraint ablation study. In brief, the two difficult losses are surface strain and diversity, both in terms of runtime and iterations needed.

Respecting the character limit, we are happy to continue with a more detailed discussion upon the reviewer's request.

[1] Jia, Y., Wang, C., and Zhang, X. S. FEniTop: a simple FEniCSx implementation for 2D and 3D topology optimization supporting parallel computing. Structural and Multidisciplinary Optimization, August 2024.

[2] Whalen et al., SimJEB: Simulated Jet Engine Bracket Dataset, Computer Graphics Forum 2021