Many thanks to Reviewer oAjG for your instructive reviews.

Q1: It's not clear what the goal exactly is. The problem might be something else entirely: the distribution over output physical states.

(1) Our goal is to local-adaptively control each point’s state distribution.

As stated in lines 210-218, neither overly peaky nor smooth weights are desirable. As shown in Fig.3(c) and stated in its caption, points in slow varying regions should have sharp distribution, while those in fast varying regions with higher uncertainty expect a relatively smooth distribution.

(2) We respectfully disagree with “the problem might be something else."

We appreciate the reviewer’s careful observation. However, as shown in Figure 13 of Appendix (overly uniform assignments) and Figure 8 left of Appendix (Collapsed into two states), both cases illustrate unreasonable point-wise state assignments, failing to meet our goal above.

These cases reveal an underlying issue: the lack of adaptive point-wise assignment, rather than the "distribution over output physical states." We believe our work directly addresses this core limitation of Transolver.

(3) New experiments on annealing.

As per your request, we conduct ablation studies (see Q2(1) of Reviewer dnan) with annealing strategies. This further verifies the statements above.

Q2: Why spread some input features over several states, not others?

Again, our goal is point-wise adaptive distributions, not just peaky ones. A more uniform distribution is more suitable, especially for fast-changing regions with higher uncertainty.

Q3: Why use sampling? Is the sampling seed fixed? Stability

As shown in Eq.(5) and Algo 1, we don't perform explicit sampling during training or inference. Instead, we generate new state weights using Gumbel-softmax, which enables the model to explore more diverse state assignment distribution. With a fixed seed at the beginning, each training step yields different distributions for the same element. This will not affect the training stability. Please see: https://anonymous.4open.science/r/ICML_rebuttal_3035-FD1C

Q4: The contributions are quite minor, cannot transfer. Multi-GPU optimization is nice, but a bit of an implementation detail.

As you mentioned, Transolver++ is specific to Transolver, which is why we name it "Transolver++," not others. However, the considered questions and design principle of our paper are valuable to this community. Here is the summary.

(1) Q1 can demonstrate insights behind local adaptive mechanisms.

(2) Parallelism framework stems from in-depth investigation.

The "multi-GPU optimization" design stems from our in-depth insights into Transolver and PDE-solving tasks, not only "implementation detail," where we carefully investigated which representation should be parallel computed or communicated. Without these efforts, Transolver++ cannot achieve the lowest communication overhead and presents linear scalability now.

It is also the first efficient parallelism framework in neural operators for PDE solving, which can serve as a starting point for exploring model-system co-design in solving PDEs. Thus, the contribution of the parallelism framework should not be underestimated.

(3) Outstanding performance on industrial tasks can advance research to real applications. A new dataset of AirCraft will be made public.

Q5: New datasets analysis is a bit misleading, baselines are on patches and stitched.

Thanks for your detailed review. We will highlight our training setting in the revision.

(1) We have to conduct patch-wise training, otherwise all prior methods are inapplicable.

As noted in Table 2's footnote, prior methods cannot handle million-scale data and require patch-wise training. Since this is the only way (we can figure out) to measure other models on DrivAerNet++, we believe this comparison exactly shows their capability in large-scale tasks.

This unaligned setting highlights the advantage of Transolver++ in large-scale tasks (parallelism framework is our key contribution), instead of being considered meaningless.

(2) Comparison under the same setting

We also provide results of aligned settings in six standard benchmarks and AirCraft. We will further clarify this in the revision. We also trained Transolver++ under the patch setting, which is still the best compared to other models.

Q6: Confusing/vague terms and phrases.

Many thanks. "Mesh points" are common in PDE-solving and refer to vertices. We will carefully revise other terms in the revision.

Many thanks to Reviewer VjMs for your detailed and instructive review.

Q1: How generalizable is the proposed method across different geometries, particularly in experiments involving car designs from the DrivAerNet++ dataset? Could you provide more details on the statistical distribution of design parameters for car shapes in the training and test splits?

Thank you for your constructive question, which helps to improve our work and analysis. We follow the sampling ratios provided in Figure 8 of the original DrivAerNet++ paper, using equal numbers of samples for both WW and WWC (wheels open detailed/closed):

Training and testing share the same distribution, with no car-type shift between splits.

Transolver++ shows consistently strong performance across all categories, confirming its robustness and generalization. A summary will be added in the revised version.

Q2: UPT is a conceptually similar method and is not mentioned in the related work. How does this Transolver-based method differentiate itself from UPT and its follow-up NeuralDEM? Have the authors experimented with UPT, or do they have a high-level understanding of why one method might outperform the other?

Thank you for your valuable suggestion. It prompted us to reflect on the distinction between Transolver-based methods and UPT, representing important advances in neural solvers for scientific problems.

(1) Conceptual Difference

First, we note that Transolver (ICML 2024) predates UPT (NeurIPS 2024).

Second, the modeling paradigms differ: UPT decouples the encoder and decoder, enabling flexible querying across arbitrary scales, while Transolver-based models are fully end-to-end, enabling direct supervision on physical states and achieving stronger performance on fixed meshes (e.g., AirCraft).

Third, UPT compresses geometry via supernodes into a global latent without explicit state assignments. NeuralDEM builds on this with primary quantity modeling and scalar control. In contrast, Transolver++ learns soft point-to-state assignments, enabling interpretable adaptive assignments based on shared physical behavior.

Fourth, UPT evolves dynamics entirely in the global latent space. Transolver-based models apply slice-deslice at every layer, supporting progressive refinement of physical states and stronger geometry-physics coupling, with enhanced interpretability and control.

(2) Empirical Comparison

As our focus was on end-to-end architectures, UPT was not included in our original experiments. During the rebuttal period, we trained UPT on the AirCraft dataset.

We observed that UPT has ~10× more parameters than Transolver++ (14.3M vs. 1.7M), which may lead to slower convergence. Additionally, its lack of explicit physical modeling for nodes may limit its ability to capture fine-grained structures. As a result, UPT underperforms compared to Transolver++ in this setting.

We sincerely apologize for not including UPT in the related work section earlier. A detailed comparison and proper citation will be added in the revised version.

Q3: Is this sampling performed only during training, or does it also occur during inference?

As shown in Eq.(5) and Algo 1, stochasticity occurs in both training and inference, which enables the model to explore a more diverse state assignment distribution. Also, we provide train log plots at https://anonymous.4open.science/r/ICML_rebuttal_3035-FD1C to show the stability of Transolver++.

Q4: What does the variable f represent?

As we stated in Lines 267-270 right column of main text, we have stated that "Also, we found that ..., where x is used to generate slice weights, and f is combined with weights to generate physical states." In all, f represents geometric features while x, as other features, is used to generate weights in the github implementation of Transolver. We will make this clearer in the revised paper. Notably, this design significantly reduces computational complexity and enables the first practical application of deep learning to large-scale industrial simulations.

Many thanks to Reviewer Ma1w for your invaluable suggestions.

Q1: Lack of baseline range on large geometries benchmarks, such as UPT, GraphViT, and PointTransformer. Also, compare with imperfect baseline traditional solvers.

(1) New baselines.

Thank you for your constructive and helpful suggestions. Due to the limited rebuttal time, we were only able to carefully tune and compare several models on the AirCraft dataset, as summarized below. We will include these baselines along with brief model descriptions in the revised version. Notably, our model consistently achieves the best performance under this setting, demonstrating its effectiveness and robustness across strong baselines.

(2) Imperfect baseline traditional solvers

For completeness, we also used a different traditional PDE solver (UnsCFD) to re-simulate the AirCraft data (smallest mesh scale in all three datasets). Each case took 5–6 hours on average, requiring 3–4 days to complete the full test set. In contrast, our neural model trains in under 10 hours and infers each case in less than 1 second, offering a significant efficiency advantage.

Moreover, as shown below, Transolver++ significantly outperforms UnsCFD in accuracy, showing its potential as a practical surrogate for large-scale simulations. These results will be included in the appendix of the updated version.

Q2: No error bars in Table2.

Thank you for pointing this out, and we sincerely apologize for the oversight. All experiments for Transolver and Transolver++ were run at least three times. We provide the error bars (mean ± std) for Table 2 below and will include them in the revised version for clarity.

Q3: Is it possible that the method improves a single model, but the analysis is strictly limited to that particular model? In particular, perhaps the strongest contribution of the paper is the slice reparameterization. I am not sure how much knowledge it adds to the community.

Thanks for your detailed and instructive comments. As you mentioned, Transolver++ is specific to Transolver, which is why we name it "Transolver++," not others. However, the considered questions and design principles of our paper are valuable to this community. Here is the summarization.

We would like to further clarify our contributions to this work. Our contributions lie in the following aspects.

1. A local adaptive mechanism to better learn the physical states.

The idea of learning hidden physical states is not specific to our model, which is widely shared in this community as a means of compression without explicit modeling. Our contribution lies in improving how such states are learned. The underlying principle is general and can be extended to other models with similar goals.

2. An end-to-end model with highly parallel implementation

To our best knowledge, Transolver++ is the first to validate end-to-end deep models and verify an effective parallelism framework for industrial datasets with million-scale geometries, which is meaningful to the community. Though tailored to Transolver, it provides insights into scalable architecture design and low overhead parallelism, which is ensuring the smallest communicating representation.

It is also the first efficient parallelism framework in neural operators for PDE solving, which can serve as a starting point for exploring model-system co-design in solving PDEs. Thus, the contribution of the parallelism framework should not be underestimated.

3. Outstanding performance on industrial datasets

Our model consistently achieves SOTA results on both standard benchmarks and industrial datasets. As shown above, we further validated other models on the AirCraft dataset and our model on the newly-released DrivAerML (Ashton et al., CoRR 2024, 800 million mesh points per sample) datasets below.

This outstanding performance firmly demonstrates the potential of advancing neural PDE solvers to real applications, which is a strong encouragement to the community.

Many thanks to Reviewer dnan for providing a detailed and in-depth review.

Q1: Reclarification of the concept of "Physical States," particularly in early layers.

Thank you for your rigorous review.

(1) "Physical states" is from Transolver (Wu et al., 2024)

In Transolver, "physical states" are defined as physical internal-consistent representations associated with specific geometric regions (e.g., windshield or sunroof of a driving car). Extensive visualizations in their work demonstrate its ability to learn such states. Besides, after confirming with its authors that all the state visualizations in Transolver are based on the first layer representation, we think this concept is reasonable for early layers.

Since Transolver++ is an upgraded version of Transolver, we would like to keep this concept to ensure consistency.

(2) New experiments

To address your concern, in DrivAerNet++, we randomly sample 100 points across various regions and calculate JS divergence between first-layer slice weights among different areas. We can find that the slice weights present clear internal consistency and external inconsistency in different regions.

Thus, slicing operation is not a trivial weighted sum. It can ensure the model to capture physical internal-consistent representations.

(3) Representation learning perspective

Note that all model parameters will be simultaneously updated during training. In Transformer-based models, residual connections blur the distinction between early and ending layers, as gradients from the final loss can directly affect the first layer. Therefore, we cannot simply assume that early-layer representations are less related to physics.

Considering visualizations of Transolver and concept consistency, we think "physical state" an appropriate term. It also clarifies our paper's core idea in "learning distinguishable states".

Q2: Reclarification of the novelty and significance of our work.

To our best knowledge, Transolver++ is the first to design and validate deep models on million-scale industrial datasets and to verify an effective parallelism framework for PDEs, which is meaningful to this community in paving the way to practical neural solvers for real-world applications.

Our technical contributions are summarized as follows.

(1) Local adaptive mechanism is better in learning physical states

Beyond only considering oversmoothing, our approach tries to enable local-adaptive for each point, where learned distributions are expected to be uniform for high-uncertainty areas and peaky for high-confident areas.

To further demonstrate that our design is more than avoiding oversmoothing, we also provide ablation of "global annealing" temperature (a well-established way for oversmoothing attention). Results below confirm the effectiveness of our method. This also delivers the insight that local-adaptive representations are crucial in PDE solving.

(2) Highly parallel framework to enable million-scale geometries

While Eq (5) seems straightforward, it stems from our in-depth insights for Transolver and PDE-solving tasks, where we comprehensively investigated which representation should be parallel computed or communicated. With these efforts, Transolver++ successfully achieves the lowest communication overhead and presents linear scalability.

It is also the first efficient parallelism framework in neural operators for PDE solving, which serves as a starting point for exploring model-system co-design in solving PDEs. Thus, the contribution of the parallelism framework should not be underestimated.

(3) Outstanding performance on industrial tasks

Our model achieves 13% and 20% promotion on standard benchmarks and industrial tasks. The new AirCraft dataset will also be made public to facilitate future research. More results are in Q1 and Q3 of Reviewer Ma1w.

Q3: How does Gumbel-softmax affect the deslice process?

As shown in Eq.(5) and Algo 1, we generate new slice weights using Gumbel-Softmax rather than sampling a specific state, which allows the generated weights to be directly used for deslice.

Q4: f is not explained anywhere in the paper?

As stated in Lines 267-270, we have stated that "Also, we found that ..., where x is used to generate slice weights, and f is combined with weights to form physical states." f represents geometric feature. We will highlight this in the revision.