Fengbo: a Clifford Neural Operator pipeline for 3D PDEs in Computational Fluid Dynamics

We wish to thank the reviewer for their constructive feedback and for praising the direction of the paper and its underlying theory. We address their points below:

1. Extension to other Dimensions

Although the paper is focused on a 3D problem, the approach itself is not. It is quite the opposite, as Clifford Algebra (CA) Networks are renowned for being applicable to problems of any dimension. This was noted by reviewer E7Y6 and demonstrated in several papers, e.g. here, where the same ResNet was used for both 2D and 3D Maxwell’s Equations.

Fengbo can be easily recast in 2D by moving layers from 3D to 2D and employing $G(2,0,0)$ instead of $G(3,0,0)$. Since 3D flow estimation is more challenging than 2D, we restricted the paper’s scope to the 3D case, as in Li et al., 2024. Applying the same ideas to a 2D problem would be immediate, especially due to CA's inherent flexibility. We have added the section "Pipeline Generalisability" in Appendix C and Tables 5, 6 to clarify this.

2. Accuracy & Complexity

Fengbo offers several benefits:

• Joint Estimation
  We estimate pressure and velocity fields jointly, treating them as coupled and geometrically meaningful quantities. This was highlighted as a promising future direction by GINO's authors in their rebuttal:

  “While in this work we only considered the pressure field so Clifford may not be helpful, it will be interesting to explore using Clifford-FNO […] to address complex 3D geometry when modeling more fields in the future”
  See here.

The joint estimation implies:

• Physical Consistency
  Pressure and velocity fields are inherently coupled through Eqs. 1–2. Joint estimation captures this coupling and enables physically consistent predictions, with knowledge of one field improving the other.

• Higher Efficiency
  A unified representation for both fields eliminates the need for a more complex model. This improves efficiency while maintaining accuracy, even for coarse discretisation (Appendix D).

• Low Computational Complexity
  Fengbo uses a single network to manage both fields, with complexity depending on the chosen discretisation. This significantly lowers computational costs. We have expanded Table 4 and added Appendix E to stress this.

Its accuracy is consistently higher or competitive with other methods, which often fall short in one or more of the areas above. We ought to disagree on the level of complexity of the approach. Fengbo is, in fact, extremely simple at the network level. More specifically, we wish to emphasise:

• 60% fewer parameters of the SOTA Neural Operator (NO) model.
• Only 1 NO against 3 in GINO.
• just convolutions beside the FNO.
• No learnt embedding of point clouds onto regular grids (Geo-FNO, GNO)
• No computationally expensive attention layers (see Appendix E).
• Robust performance despite simple discretisation, unlike Geo-FNO, ONO, OFormer.
• No latent space, ensuring fully interpretable outputs.

3. Contribution over past CA Nets

Fengbo introduces key advancements over other CA Nets:

• Extending the Clifford NO code to support any 3D problem beyond Maxwell’s equations, as it was implemented by Brandstetter et al., 2023, which handled only vectors and bivectors.
• Introducing an approach to map geometrical quantities to physical quantities, surpassing prior works limited to physics-to-physics or geometry-to-geometry mappings (Brandstetter, Ruhe, Pepe).
• Novel modeling of 3D unstructured meshes CFD data in as multivectors defined over a 3D regular grid.
• Original Chi-shaped architecture to perform precise operations based on our modeling choices.
• Solving a 3D fluid dynamics problem, whereas past CA Nets focused on 2D flows.

4. Parameter Specifications

The parameter counts for other models were not disclosed by their authors. However, GINO has approximately 100M parameters, as confirmed by its authors. Accordingly, we have moved the discussion on parameter count from Table 2 into the main body of the manuscript and expanded the discussion on model size in Appendix D.

5. Memory

The reviewer correctly noted that 42M parameters is modest, especially for a 3D setting. The bottleneck arises from processing 6D tensors (of size $B \times C \times M \times M \times M \times D$, where $B$ = batch size, $C$= channels, $M$ = resolution, and $D$ = dimensionality from CA) via 3D convolutions, which are inherently computationally expensive.

We hope to have thoroughly addressed the concerns raised, particularly those regarding Fengbo’s generalisability to other dimensions and its novelty over previous CA Nets. We trust that our clarifications and the additional evidence provided effectively resolve the identified weaknesses, and we sincerely hope they encourage you to reconsider our work more favorably.

We wish to genuinely thank the reviewer for reading our rebuttal and revised manuscript carefully, and increasing their score from 3 to a 5. We also really appreciate the time taken to engage in constructive discussion.

Despite the reviewer score being final or not, we would still like to comment for the sake of clarity and for full transparency:

1. Complexity

We now understand better the complexity issue of the model raised by the reviewer. At theoretical level, Clifford Networks are fundamentally different than other approaches. At implementation level, however, there is virtually no difference between a Clifford Neural Operator and a Neural Operator, or, more generally, between a Clifford Neural Network and a classical one: setting up a Clifford NO is not any different than setting up a standard NO.

The only practical difference, at network level, is the embedding of data within an algebra. This translates into one line of code. For example:

x = TensorToGeometric(x, grade = 1, algebra = [1, 1, 1])

Embeds tensor x as a grade-1 multivector (i.e. a vector) in the $G(3,0,0)$ algebra: this tells the network explicitly to treat x as a vector in that algebra. Since there is very little practical difference, extending our approach to other PDEs or to datasets which include transients would be just as hard/easy to do with our Clifford architecture than to do with a classical one.

The reviewer is not wrong when mentioning that Clifford Algebra might not appear intuitive to work with, and a basic understanding on it is indeed a prerequisite to work with such architectures. With Clifford Algebra we are moving off the beaten track, but we see this as a strength rather than a weakness.

2. White Box Model

For full clarity: by white box model we mean two things: 1) our approach works exclusively in 3D Euclidean space and 2) that outputs of the layers in our pipeline are interpretable by design. For example, if we assign to scalar components the meaning of pressure field (as we do in the second half of the architecture) they cannot not be interpreted as such. This is because quantities remain consistent within the same algebra.

To the best of our knowledge, no other pipeline that estimates 2 or 3D flows shows such properties. In fact, it is often quite the opposite, since many tend to work in latent space instead, such as GINO, GNO or Geo-FNO.

3. Original Claims

We trust that the original claims the reviewer is referring to are those pointed out by gPtN, on the potential overlook of limitations, and by TsLb, on the handling irregular point clouds.

We believe the reviewers raised valid points and we addressed them in full transparency of the review process. About the point raised by reviewer gPtN, we did notice some additional limitations that could be highlighted, for example the computational inefficiency of the Clifford Algebra libraries currently available. About the points raised by reviewer TsLb, we conceded that claiming the ability to handle irregular point clouds is conceptually different than discretising them and still operating on regular grids with convolutional layers and FNO.

We view these points as subtle albeit important details that, however, do not affect the overall significance and merits of our work. On the contrary, we consider these clarifications—an integral part of a healthy review process—not only necessary but also valuable, as they have helped to further highlight the strengths of our manuscript.

We wish to state this clearly: we have no reservations about acknowledging imperfections in our work, taking the reviewers' comments into account and making corrections when needed. We have addressed with complete integrity and transparency any claim that reviewers found overstated and that we agreed with upon rebuttal.

We wish to thank the reviewer for acknowledging the merits of our pipeline, its geometric consistency, and interpretability. We address the points raised below:

Weaknesses:

1. Transformers comparison

   We recognize the importance of contextualizing our work within the broader literature, including fundamental contributions from the transformer literature. We have addressed this as follows:

   • We added three paragraphs in the Related Work section for a better contextualization, two of them on Transformer-based methods. We highlighted notable architectures, including the Transolver, as well as their merits and limitations in terms of computational complexity, which is generally quadratic or sub-quadratic.
   • We compared our pipeline to 6 additional models in Table 2, including transformer-based models, doubling the reported results from 6 to 12 models.
   • We studied 7 more models in terms of complexity, discretization convergence, and ability to handle irregular geometries in Table 4, doubling the number of models from 7 to 14 and including transformer-based models such as Galerkin and Transolver.
   • We better contextualized the results in Section 4.3 by comparing Fengbo to other methods, including Geo-FNO (neural operator), ONO, and OFormer (transformer). These are complex architectures that have been shown to fail on large unstructured meshes like those of the ShapeNet Car dataset. Fengbo, however, still performs exceptionally well with coarse discretization and a simple architecture (as opposed to learned mappings from irregular to regular grids, such as Geo-FNO, or computationally intensive approaches such as ONO and OFormer). This remains true even at very coarse discretization levels, as shown in Appendix A.
   • We added Appendix E, where we perform a detailed comparison of Fengbo's complexity with the current transformer-based state-of-the-art, the Transolver. This includes: Fig. 14: A comparison of complexity in the 3D case; Fig. 15 and Table 9: A comparison on the 2D case.

These results demonstrate that Fengbo operates at two orders of magnitude lower complexity, even when comparing our 3D with Transolver's 2D, while yielding only slightly higher errors. We are optimistic that these additions have increased the quality of the manuscript, strengthened the case for Fengbo's merits, and better contextualized it with respect to current state-of-the-art models such as the Transolver.

2. On model's size:

Since many models do not include their corresponding number of parameters, we moved the discussion on the model size from Table 2 to in-text only. A thorough discussion on model sizes, both in terms of trainable parameters and memory usage (MB), has been added to the ablations in Appendix D.

3. On different data representations

We added Tables 5 and 6 and the subsection Pipeline Generalizability in Appendix C - Implementation Details. Since this paper focuses on 3D CFD, we aligned our approach with the representations used by the only two known 3D datasets on this problem: ShapeNet Car and Ahmed Body, both of which employ unstructured meshes. For reference:

• GINO (Li et al., 2023) also uses these two datasets.
• Transolver's experimental setup includes eight datasets, of which only one is 3D (ShapeNet Car).

The data representations in Transolver—point clouds, structured meshes, and regular grids—are primarily applied to 2D datasets. However, Fengbo's preprocessing pipeline is specifically tailored to handle 3D data through the following steps:

• (a) Sampling irregular point clouds from unstructured meshes.
• (b) Discretizing the points into a regular grid of resolution $M$.
• (c) Embedding the data into a multivector form.

Such steps effectively encompass all three data representations above, and we could deal with them through a combination of the steps above:

• Irregular Point clouds: Steps (b) and (c) convert them into a regular grid and embed them in multivector form.
• Structured meshes: Steps (a), (b), and (c) sample points from the mesh, discretize them, and embed them.
• Regular grids: Step (c) directly embeds them into a multivector form (as already demonstrated in the literature).

This is reported in Table 6.

Furthermore, as Fengbo is built on Clifford Algebra, it inherently generalizes to problems in any dimension and within any spatial representation. This adaptability ensures that Fengbo is well-suited to a wide variety of data structures and applications.

4. Typos

   We resolved the typos in the text and in Eq. (11)-(12).

We hope that our responses and revisions address the points raised in a satisfactory manner, as we are confident they have strengthened the contributions of our work. We genuinely hope these points inspire the reviewer to view our work more positively.

We would like to thank the reviewer for praising our writing and the level of detail of the results, as well as for their insightful comments. We address them below:

1. Contributions over Clifford Algebra (CA) Networks

Fengbo is not merely an application paper of past works on a new problem (i.e. 3D flows estimation), since it presents several important contributions over previous CA Nets. These include:

• Extension to general 3D
  We extended the Clifford Neural Operator (NO) code to support any 3D problem, and not just Maxwell’s equations as originally designed. This enables the processing of full-grade multivectors and hence its extendability.

• Geometry-to-Physics Mapping
  Fengbo maps geometrical quantities to physical ones, while prior works were limited to physics-to-physics or geometry-to-geometry mappings (Brandstetter, Ruhe, Pepe).

• Novel Data Representation
  We introduced an original way of modeling large-scale 3D CFD unstructured meshes as multivectors defined over 3D voxels. Unstructured meshes were not considered in past CA nets papers.

• Novel Structure
  Fengbo has an original structure:

  • Parallel blocks to operate on $N_g$ geometrical quantities.
  • Sum of contributions and mixing through the FNO.
  • Splitting into $N_p$ parallel blocks for each physical parameter to estimate.

  This design is based on our modeling choices and has been chosen to perform precise operations (Table 1).

2. Addressing PointNet Methods

We agree that we have overlooked some classes of methods. To better contextualize our work, we have added, among several others, also PointNet and PointNet++-based models to our discussion, as suggested by the reviewer. Specifically:

• These methods are addressed in Section 2 (Related Work).
• Their performances when estimating pressure and velocity are included in Table 2.
• Their complexity, ability to deal with irregular grids, and discretisation convergence properties are discussed in Table 4.

3. Handling Irregular Grids

The reviewer is correct in stating that we convert irregular grids to Cartesian grids, much like CNNs and FNOs. When we claim that Fengbo handles irregular grids in Table 4, we mean that our pipeline can learn from irregular point clouds accurately and efficiently through discretisation. However, this approach is conceptually different from models like PointNet or MeshGraph Net, which handle irregular grids by design. We understand the reviewer’s concern and have modified Table 4 for Fengbo accordingly.

4. Geo-FNO Distinction

The reviewer is correct that the FNO can handle irregular geometries if geometric transfer is employed. However, that would be a Geo-FNO architecture, which is considered a distinct model. The FNO itself still requires a regular grid due to the FFT. We made this distinction clear in Table 4.

Note also that Geo-FNO learns the mapping from irregular to regular grids but fails on the ShapeNet Car dataset. Fengbo, on the other hand, performs competitively even with a simple deterministic discretisation. This is possible thanks to the coupling of geometrical and physical quantities within the unified CA framework and the consequent inductive bias. This is a fundamental point for the success of our approach (see comment above)

Questions

1. The discrepancy between training and testing errors, while counterintuitive, is not uncommon. We believe this arises from the effect of regularisation, which penalises the model’s performance during training to prevent overfitting. This is a likely issue here, given the training set includes only a few hundred instances.

2. We understand the reviewer’s concern about mixing different loss terms, as they involve different "units" of measurement. We included the $L_1$ norm after empirically observing its regularising effect, which helped reduce overfitting on the small training set.

• The relative $L_2$ norm is essentially an $L_2$ with normalisation factor. It is particularly sensitive to larger errors because it squares the deviations, emphasizing outliers and ensuring stability.
• The absolute $L_1$ norm is less sensitive to outliers and more robust to noise, as it penalises errors linearly. This property encourages sparsity and smoother solutions.

By combining these two components, the loss function balances sensitivity to large deviations with robustness to smaller-scale variations and noise. This is conceptually similar to elastic net regularisation, which combines $L_1$ and $L_2$ penalties on a model’s parameters.

We trust that our responses have helped in making a stronger case for Fengbo’s contributions, especially regarding its novelty and advancements over existing CA Networks. We appreciate that the reviewer’s comments have helped us better contextualise Fengbo within the existing methods and improve the quality of our work.

We wish to thank the reviewer for the detailed analysis of our work, and for acknowledging its generalisability to multidimensional fields. We address the points raised below:

Weaknesses:

1. We tested our pipeline on the two datasets available for 3D CFD simulations, namely ShapeNet Car and Ahmed Body, which are simulations in air at steady state. This is consistent with the literature on 3D flows (see Li 2024, Wu 2024).

2. That is valid, but we also emphasised the following key advantages:

• Coupling between pressure and velocity (crucial, more below).
• No need for ad hoc modules to handle irregular grids, which might still fail (Geo-FNO).
• Low computational complexity
• Low number of parameters

The accuracy reported is higher than most and comparable to all other methods, which often fall short in one or more of the areas above.

3. We showed in Appendix D how even at very coarse discretisation $M = 40$ we still outperform more sophisticated models such as GNO and Geo-FNO.

Questions:

1. Velocity normals

ShapeNet Car datasets include two meshes, one outlining the surface of the car, over which the pressure field is defined, and one outlining the volume surrounding the car, over which the velocity field is defined. The bivector component is missing in $V$ since no information about the normals on the mesh of the velocity field has been provided, as opposed to $P$. We agree that, if it were to be made available, it would likely correspond to an improvement in the performance.

2. Interpretability

We agree that the interpretability remains mostly qualitative in our work, as we have not introduced a quantitative analysis of intermediate layers nor operated on them directly. We are aware of authors who exploited the interpretability of Clifford Layers to perform specific operations on them, most notably:

• This paper on 3D point clouds,
• This paper on camera poses,

which, however, perform rigid body motion on geometrical quantities. Operating on intermediate pressure and velocity fields, in a way that respects the physics of the problem, is more complex, and would require different operations beyond simple neural layers. We leave this aspect for future research, but we maintain that end-to-end interpretability, albeit so far qualitative, remains both feasible and an important contribution of our work.

3. MeshGraph Net

We have now included MeshGraph Net in Table 2, reporting the relative $L_2$ norm on pressure and velocity fields for the ShapeNet Car dataset, as well as in Table 4, discussing its computational complexity and properties.

4. Adaptability of other models for velocity estimation:

The reviewer is correct in stating that other models can be adapted to estimate velocity, also jointly with pressure. However, our joint estimation does more than having multiple outputs at once: we embed pressure and velocity (as well as all the other inputs) in a mathematical space that treats them as geometrically meaningful quantities within the unified framework of Clifford Algebra (CA). This is a fundamental element in our pipeline, since it provides valuable inductive bias that keeps estimations consistent with each other, and the reason behind the robust performances with a simple pipeline.

Authors of GINO already emphasized the potential of CA in their rebuttal :

“While in this work we only considered the pressure field so Clifford may not be helpful, it will be interesting to explore using Clifford-FNO […] to address complex 3D geometry when modeling more fields in the future.”

We are the first to do so with an approach that pairs novel modeling choices (unstructured meshes as 3D discrete volumes of multivectors) with an original pipeline in a geometry-to-phyics setting. Thus, while most networks could be adapted to handle multiple fields, we believe only CA Networks can do so in a way that explicitly couples the physics to the geometry of the problem and keeps predictions consistent.

We are aware that the reviewer is confident in their judgment of our work. Nonetheless, we hope that our rebuttal has made a stronger case for the contributions of our pipeline, particularly after clarifying the implications of joint estimation within the CA framework. These contributions are not just limited to a reduction in the number of parameters, but also include:

• physically and geometrically consistent joint estimation of quantities.
• competitive performances.
• low computational complexity
• robustness to discretisation
• original data modelling and network design

We hope that addressing the points made by the reviewer could lead to an overall more positive outlook on our work.

We wish to thank the reviewer for the time they took to read our manuscript, for the useful comments they made, and for defining our approach fresh and creative. We address the weaknesses below:

1. Benchmarking

We agree that benchmarking could be more comprehensive. We decided to expand it as follows:

1. We added three paragraphs in the Related Work section for better contextualisation in the literature, discussing PointNet methods, Transformers and GNOT.

2. We compared our pipeline to 6 more models in Table 2, doubling the reported results from 6 to 12 models.

3. We added results on the relative $L_2$ norm for velocity, in addition to pressure, for 8 models in Table 2.

4. We studied 7 more models in terms of complexity, discretisation convergence, and the ability to handle irregular geometries in Table 4, doubling the number of models from 7 to 14.

5. Added references to Tables 2 and 4

6. Contextualised the results obtained in Section 4.3 by comparing them to other methods: Geo-FNO, ONO, and OFormer are complex architectures that fail on large unstructured meshes such as ShapeNet Car. Geo-FNO, for example, employs a learned mapping from irregular to regular meshes. Nonetheless, Fengbo still performs exceptionally well, even with a coarse, deterministic discretisation and a simple architecture. This robustness holds even under harsh discretisation levels, as demonstrated in Appendix D. We believe this robustness stems from the joint estimation of different physical quantities embedded in the same mathematical space, providing an inductive bias that ensures the predicted pressure and velocity fields remain coupled and consistent.

7. We added Appendix E, in which we compare Fengbo’s complexity with the current transformer-based state-of-the-art, i.e., the Transolver, for 2 and 3D cases. We demonstrate we operate at 2 orders of magnitude lower complexity than the Transolver while yielding only slightly higher error.

2. Generalisability and Scalability

We agree that we can make a stronger argument about Fengbo’s Generalisability and Scalability. For this reason:

1. We added a discussion on Generalisability in Appendix C, including Tables 5 and 6. Clifford Algebra Networks are well-known for their adaptability to any dimension. As a result, Fengbo could be readily transposed to 2D, as well as extended to any other PDE in 2 and 3D space. An example is shown here, where the same Clifford ResNet is used for both 2D and 3D Maxwell’s Equations. Projecting Fengbo down to 2D is not only possible but also straightforward, especially thanks to CA.

2. To demonstrate the Scalability of our approach, we extended Appendix D and increased ablations from 1 to 4. Beside ablations on grid size $M$, we added:

• Impact of hidden channels in the 3D Clifford FNO block for both datasets.
• Impact of the number of Clifford FNO blocks for both datasets.
• Impact of Fourier modes in the FNO for both datasets.
• Effect of these variables on the number of parameters and model size.

Showing how performances improve as the model scales.

3. Limitations

We agree that some limitations might have been overlooked. Hence, we extended the Limitations subsection. Besides the computational bottleneck that arises from 3D convolutions (which prevents testing larger versions of Fengbo), we also discussed:

1. The intrinsic limitation of CA Networks, which are not optimised for use with underlying tensor structures.
2. The hypothesis that Fengbo may not perform as well on datasets that are not as rich as ShapeNet Car and Ahmed Body, since we believe full-grade multivectors play a crucial role in the success of this approach.

Answers Questions

• 4. Please compare with more recently developed methods:
  Addressed in Benchmarking.

• 5. Dataset selection:
  As the focus of the paper is 3D PDEs for CFD, we employed the only two known comprehensive datasets on the topic: ShapeNet Car and Ahmed Body.

  This dataset choice aligns with prior literature, such as:

  • This paper, which also focuses on 3D PDEs and employs the same two datasets.
  • This paper, which takes a broader scope but uses only one 3D dataset (ShapeNet Car) out of eight studied.

Although limited to two, we are confident these datasets are sufficiently comprehensive, since:

• ShapeNet Car accounts for variability and complexity of shape.
• Ahmed Body accounts for multiple CFD simulation parameters.

We hope these additions clarify most, if not all, doubts raised by the reviewer. We are confident that the feedback provided has helped us further substantiate the strengths and contributions of Fengbo, and the advantages that Clifford Algebra brings to 3D flow estimation and, more broadly, to the NO literature.