DEBOSH: Deep Bayesian Shape Optimization

Only one equation in manuscript - indicates lack of soundness and novelty.

Do not agree that use of Reentrant GNN with Bayesian optimization is sufficient for ICLR level.

Where are failure cases illustrated and discussed ?

The proposed approach in this paper seems reasonable, but I still have some concerns.

The main issue is that the current design changes are entirely optimized by the network adaptively. While this may indeed yield a higher metric on the designed acquisition function, I am not entirely certain if this metric truly reflects real-world conditions. On one hand, the optimization direction may not necessarily align with the expectations of industrial design (the metric itself may only capture one aspect, while there may be other factors that need to be considered but cannot be formulated). On the other hand, since this metric is also estimated, its accuracy itself deserves further exploration.

Additionally, I have some doubts about step 4 of the method. I am unsure whether incorporating noisy data as supplementary training data would bring benefits or introduce additional noise.

First, the paper is really dense and quite hard to read. I would suggest some rewriting to make it less compact.

Then, I am not convinced by the approach taken by the authors w.r.t the output of the surrogate model. Here, the surrogate model learns to predict both the surface pressure and the metric value. Even though this metric is computed from approximate pressure values with another neural network, it still breaks the physical integration between the pressure field and the different forces. To foster generalization, the GNN should capture the physical phenomenon through the pressure and other volume fields if necessary, and compute the metrics from them through integration. This leads in a trustful and physically-based metric, which is not the case with this method. It is claimed and showed in the appendix that using the integration of the pressure values has a higher MAE for the considered metrics. This is not surprising as regressing a single value is less challenging than predicting the pressure on the whole surface. However, if a surrogate model were to correctly predict the pressure on the surface, then the drag or lift-to-drag would be actually meaningful.

Besides, the reentrant GNN does not provide uncertainties on the predictions, but only a proxy for it. This is a key and crucial difference with the other methods. Techniques such as Deep Ensembles and MC-Dropout provide approximate Bayesian inference in deep gaussian process, while the reentrant GNN does not. If we take twice the same shape, the output will give the same prediction both times.

As a consequence, the motivation behind the reentrant GNN is not transparent to me. It seems to be a rapid and not ingenious solution with a wrong tool. I tend to agree on the receptive field analysis. A reentrant GNN with $k$-iterations of depth $d$ should have the same receptive field as a simple GNN with depth $d*k$, and a higher receptive field than a simple GNN with the same depth. Still, in Figure 5 it seems that it converges most of the time in 1 step for in-distribution shapes, and thus the gain in performance seems marginal. The same can be deducted from the comparison between reentrant GNN and GNN ensembles which are both very close in prediction performance.

• The framework is limited in the sense that it bounds itself to a set of pre-existing shapes in the dataset. In fact, the main question that is addressed is how one can find the most promising shapes to be passed to physical simulator. This is a less strong achievement than a full-fledged BO which could 'synthesis' new shapes.

• The above point begs the question of how the cost of training and optimization are amortized during later inference stages. At least in the case of 1500 airfoils, I feel the brute-force approach of simulating all of them would have been comparable in computational cost to the proposed approach.

• I think the current validation could have been focused on the key idea: physically-based uncertainty estimation. For example, showing that both the physical simulation and the GNN simulate the long-range dependencies similarly (according to Fig. 2).

• The choice of OOD selection based on the final aerodynamic performance seems not to be well grounded. In general, the dataset, at least for airfoils, seems to be pretty uniform (not diverse). This is inferable from the low-dimensional latent space (3D) and also visually from Fig. 3 where the in-distribution and OOD samples don't differ a lot.

• When the computational cost of the proposed method is compared to GNN ensembles, the fact that ensembling is trivially parallelizable should have been discussed.