Bridging Equivariant GNNs and Spherical CNNs for Structured Physical Domains

Thank you for your review. We are working on a revision to incorporate your feedback, but we address your comments and questions here as well.

“It's not very clear to me how the 3D shapes are mapped to the radar signal in the Radar dataset. Similarly, why is the output signal defined over a sphere rather than over the mesh surface in the Drag Prediction dataset?”

For the radar dataset, the 3D shape is processed as a point cloud and encoded using a GNN. This is then decoded to a spherical radar signal.

To clarify, for all three applications, we are not projecting the mesh onto the sphere. Rather, we are modeling the directional dependency of each output (radar signal, drag prediction, control policy) as a spherical (angle-dependent) function. These are inherently global functions defined over the sphere of directions, not over the local mesh surface. In the radar setting, the radar signature is dependent on the sensor viewing angle on the object, in the drag prediction task, the drag value depends on the direction of the airflow impacting the object. In the control policy learning task, the policy value depends on the agent’s motion angle. If our explanation is still unclear, we would be happy to explain further.

“Why using global averge pooling? … this choice still feels like it might discard a lot of important spatial information. Isn't there some better choice?”

We use global pooling to aggregate spatial features across the mesh, enabling the model to produce outputs that depend on global geometry and direction. This is related to our answer for the previous question, the spherical output signal is actually a global signal, and thus pooling across the mesh is appropriate. While global average pooling is effective in this context, we also experimented with other pooling strategies, such as top-k pooling, and found similar results. That said, we agree that more expressive alternatives like attention-based pooling could better preserve spatial structure, and we are interested in exploring such methods in future work.

“By training on lower resolution data, isn't the model simply going to learn only the low frequencies component of the signals? In this sense, the zero-shot super-resolution is not much more interesting than interpolation …”

While training on lower-resolution data does limit the signal to lower frequency components, our model's ability to perform zero-shot super-resolution stems from more than just interpolation. Because the spatial sample locations are randomized across the different meshes (due to the different mesh geometries and orientations), achieving good performance requires the model to learn the correlation between low and high frequency components across the dataset. This is only possible when there is consistent structure in the signals that the model can exploit, such as those arising from similar geometry or alignment across samples.

We additionally note that this has been seen in prior works like Fourier Neural Operators[1], where models trained on coarse data, PDEs in their case, can generalize to finer resolutions by learning global patterns. We also agree that this could have been made clearer and will revise the text to emphasize these details in the paper.

[1] Z. Li et al., Fourier Neural Operator for Parametric PDEs, ICLR 2021.

“Fig 4 is a bit hard to read.”

Thank you for the suggestion. We will improve the readability of Fig. 4 by increasing the size in the final version. We will also revise the caption to provide more context and better highlight the key takeaways from the figure.

“Tab 2: could you report the standard deviation over the 50 initializations?”

Instead of reporting the standard deviation across the 50 initializations, we follow the common practice in policy learning literature of reporting the mean performance of the 50 initializations over the final 10 evaluation rollouts. This reflects the stability of the final policy and aligns with established conventions in the field. We acknowledge that additional variance metrics (e.g.,standard deviation) could provide further insight and will consider including them in future work.

“Tab. 3: why is the NE-G2S much faster than G2S?”

The difference in runtime between G2S and NE-G2S in Table 3 is primarily due to the increased computational overhead of equivariant architectures. Specifically, SO(3)-equivariant models like G2S rely on spectral convolutions and spherical harmonics, which are inherently more expensive than the standard operations used in non-equivariant models such as NE-G2S. That being said we find that the increased performance of equivariant G2S to out-way the minute (6ms) increase in inference time.

“TSNL: … because the resulting signal has unbounded frequency, a Fourier Transform with finite samples can not be exact and will always introduce some artifacts, thereby breaking the exact equivariance of this layer. This is in contrast with what the authors claim in lines 727 and 730.”

Thank you for pointing this out. You are correct in that applying a Fourier Transform with finite samples can introduce minor artifacts, which in turn lead to a small equivariance error in the TSNL layer. While this is typically negligible in practice, we agree that the claim in lines 727 and 730 should be updated accordingly. We will revise the text to clarify this point and will additionally quantify the equivariance error introduced by TSNL in our revised version.

Additionally, we note that this approach is common in many equivariant architectures such as the Spherical Fourier Neural Operator [1], Spherical CNN [2], and Gauge Equivariant Mesh CNNs [3]. Specifically in [2] they measure this equivariance error and find that it is $< 2e-6$ (see Sec 5.1 of [2]).

[1] Bonev, Boris, et al. "Spherical fourier neural operators: Learning stable dynamics on the sphere." International conference on machine learning. PMLR, 2023.

[2] Cohen, Taco S., et al. "Spherical CNNs." International Conference on Learning Representations. 2018.

[3] De Haan, Pim, et al. "Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs." International Conference on Learning Representations, 2021.

Thank you for your review. We are working on a revision to incorporate your feedback, but we address your comments and questions here as well.

“A closely related architectural baseline is missing: [A] uses a spherical encoder and SO(3)-equivariant CNNs [15], with a similar motivation of operating in the frequency domain rather than the spatial domain.”

Thank you for bringing this work to our attention, we will add it to our related work section and explain its relationship to our method. Indeed, [A] shares key architectural similarities with G2S, including the use of spherical representations and SO(3)-equivariant convolutions in the frequency domain, which support continuity, equivariance, and data efficiency. However, there are two important distinctions. First, [A] begins from 2D image inputs, using a ResNet backbone followed by projection onto the sphere, whereas G2S is designed for 3D geometric inputs and operates directly in the spherical frequency domain from the outset. Second, [A] targets 3D pose estimation and thus operates effectively at a fixed low bandwidth ($L_{max} = 5$), while G2S must adapt $L_{max}$ to the demands of each domain, ranging from 5 in our policy learning tasks to 40 in radar prediction. Therefore, we employ various techniques such as a frequency upsampling strategy to scale output resolution efficiently.

“The use of point-wise MLPs in TSNL raises questions about whether end-to-end SO(3)-equivariance is preserved.”

The TSNL is equivariant up to the sampling error introduced by the inverse Fourier transform which we use to convert from frequency domain features to real-space signals on a fixed spherical grid. Because we apply this MLP pointwise to each of these points it is SO(3)-equivariant by construction. While it’s true that performing a Fourier transform with finite samples can introduce minor artifacts, these result in negligible equivariance error in practice. We note that this approach is common in many equivariant architectures such as the Spherical Fourier Neural Operator [1], Spherical CNN [2], and Gauge Equivariant Mesh CNNs [3]. Specifically in [2] they measure this equivariance error and find that it is $< 2e-6$ (see Sec 5.1 of [2]).

[1] Bonev, Boris, et al. "Spherical fourier neural operators: Learning stable dynamics on the sphere." International conference on machine learning. PMLR, 2023.

[2] Cohen, Taco S., et al. "Spherical CNNs." International Conference on Learning Representations. 2018.

[3] De Haan, Pim, et al. "Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs." International Conference on Learning Representations, 2021.

“While the use of SO(3)-equivariant models is well-motivated, the rationale for spherical representation and frequency-domain approach remains vague.”

We appreciate this comment and agree that the rationale for using spherical representations and the frequency domain deserves further clarification. Many of the signals we consider (e.g., radar and drag) are global in nature but direction-dependent. Representing them as functions on the sphere allows us to model this directional dependence explicitly, without tying the representation to the surface geometry of the 3D object. Figure 5c illustrates how G2Sphere models these directional signals by learning Fourier coefficients enabling continuous signal reconstruction via spherical harmonics. This approach is more efficient than spatial methods that either predict dense grids (5a) or require repeated coordinate queries (5b), as the cost of prediction is decoupled from the number of output points. Modeling in the frequency domain also provides a strong structural prior that helps avoid overfitting to noisy or irregular signals. We observe this in the drag prediction experiments (Fig. 4), where spatial baselines tend to overfit, while G2S is biased toward smoother solutions due to its harmonic representation. Additionally, it provides a clean and scalable way to maintain SO(3) equivariance, resulting in an efficient end-to-end equivariant architecture. To demonstrate this, we do include competitive spatial-domain baselines such as transformers in the mesh-to-sphere experiments, and IBC and diffusion models in the policy learning tasks.

“Although SH frequency is increased to 40, the paper does not isolate how this impacts precision.”

We do include a study in Appendix H on the effect of frequency on precision. As shown in Table 4, increasing $L_{max}$ leads to a clear reduction in error (0.26 MSE reduction from $L=5$ to $L=40$), while Figure 6 provides qualitative evidence of improved predictions, e.g. critical specular radar effects are completely missing at lower $L$ but become observable as $L$ increases. However, as noted in Figure 7, higher frequencies can lead to overfitting in low-data regimes. Additionally, we observe that compute and memory usage increase substantially with larger $L_{max}$. We agree that analyzing the trade-off between SH frequency and performance is critical to understanding the role of $L_{max}$​ and we will move this study to the main body.

“The paper mentions sample efficiency in the introduction, but does not provide supporting experiments.”

The mention of sample efficiency in the introduction is supported by the policy learning experiments (Table 2) and the generalization tests on the drag domain (Fig 4). In all of our policy learning experiments we are in relatively low data regime settings with only 100 demos for each dataset. Therefore, when we see both the equivariant and non-equivariant versions of G2S outperforming the baselines, the primary reason for this is their increased sample efficiency. For example, in the push-T task, the non-equivariant G2S achieves a success rate of 0.97, the equivariant version reaches 1.0, while the diffusion model scores 0.95. The differences are even more pronounced in the random push-T variant, where non-equivariant G2S achieves 0.74, equivariant G2S reaches 0.92, and diffusion drops to 0.71. These gains in sample efficiency are primarily due to the combination of equivariance, which allows the model to better generalize under rotations, and the frequency modeling, which provides regularization, stability, and avoids overfitting. This overfitting in particular can be seen in the generalization on the drag domain (Fig 4). That being said, we acknowledge that additional ablations on dataset sizes would further strengthen this claim, and we plan to include this to provide a more comprehensive analysis of sample efficiency.

“Sections 5 and Appendix G contain overlapping content and overly verbose explanations, which at times read like LLM-generated text. A more concise and well-structured narrative would greatly enhance clarity.“

We thank the reviewer for the valuable feedback. While Section 5 contains some overlap with Appendix G, this redundancy was intentional to ensure the reproducibility of our experiments. Section 5 provides a high-level overview of key model parameters and important details, while Appendix G includes a more comprehensive set of specifications and additional information necessary for fully replicating the experiments. This design ensures that readers have both a concise summary in the main text and an exhaustive reference in the appendix for reproducibility. However, we acknowledge that there might have been too much overlap and will take care to streamline the content to reduce this. Additionally we will do the following to address the other points brought up:

• Separation of reused and new modules: We will revise Section 4 to more clearly differentiate between components adopted from prior work and those introduced as novel contributions in this paper.
• Linking motivation and experiments: We will strengthen the connection between the motivation and the experiments by revising both the introduction and experimental sections to clearly articulate the purpose of each experiment and how it addresses our core research questions.
• Cohesive flow: We will revise the transition from the methodology section to the experiments to ensure a more cohesive flow. In particular, we will add forward references from Section 4 to the relevant experiments in Section 5. For example, highlighting how the frequency upsampling technique introduced in Section 4.2 directly contributes to the improved high-resolution outputs shown in Figure 3. This will help clarify how specific design choices lead to observed performance gains.

“The paper critiques HEALPix spatial grids (L29–30) for overfitting to sample locations.“

While HEALPix is cited as an example of a possible grid for spherical output signals, we do not explicitly compare to it in this work. However, we evaluate several representative gridded methods that similarly predict or rely on regularly sampled points over the sphere. For example, our transformer baseline in the radar domain predicts a fixed grid of uniformly spaced outputs. In the drag prediction and policy learning tasks, implicit models such as IBC and the implicit transformer are conditioned on coordinate queries sampled from a fixed grid, which the model must learn to reconstruct.

As we stated in the introduction using fixed spatial grids, whether for direct prediction or coordinate conditioning, commonly leads to overfitting to the sample locations. This behavior is clearly visible in Figure 4 (drag prediction), where the implicit transformer baseline overfits and in Figure 8 (policy learning), where IBC shows severe overfitting. We will revise the paper to clarify that this overfitting tendency is characteristic of the broader class of gridded methods, of which HEALPix, our transformer baseline, and coordinate-conditioned implicit models like IBC are all representative.

Thank you for your review. We are working on a revision to incorporate your feedback, but we address your comments and questions here as well.

“It might be valuable to provide a discussion on the efficiency of the method. In particular, it would be helpful to clarify the maximum frequency (L_{max}) applied in this work. As is well-known in the literature, when L_{max} is high, the SO(3) equivariant networks can become computationally demanding, and it would be insightful to understand how this is handled in your setting.”

We agree that the computational costs of SO(3)-equivariant networks with high $L_{max}$ is an important consideration. This is especially true of equivariant GNNs where maintaining high $L_{max}$ across a large number of nodes quickly becomes computationally prohibitive. In contrast, this is less of an issue with spherical models as we only have a single node, e.g. the sphere itself. Therefore, in our GNN encoder we use smaller $L_{max}$, for example in our mesh-to-sphere experiments we use $L_{max}=5$ for all layers, before raising the $L_{max}$ in the spherical decoder layers. By combining this lower $L_{max}$ encoder architecture and the frequency upsampling technique (Sec 4.2) in our decoder we avoid incurring the large computational costs that make high $L$ GNNs intractable.

As a rough rule of thumb, we found frequency upsampling to be necessary when $L_{\max} > 10$. This typically corresponds to domains requiring high-fidelity predictions, such as mesh-to-radar, where we push $L_{\max}$ to the upper limits of what our compute allows. We include ablations showing the relationship between $L_{max}$ and output fidelity in Appendix H. In lower-complexity domains such as policy learning, we instead use small fixed values ($L_{max} \leq 5$) to encourage generalization and avoid overfitting.

“How does your approach compare to standard, strong point cloud or mesh baselines, e.g. PointNet++ and RepSurf, that are not SO(3)-equivariant?”

Thanks for bringing these stronger mesh and point cloud baseline encoders to our attention. While we were unable to run these baselines within the current rebuttal timeline, our paper does include competitive non-equivariant baselines (e.g. transformers in the supervised learning settings and diffusion models in the policy learning domain) that serve as points of comparison.These models serve as reasonable baselines for our domains and help highlight the benefits of incorporating equivariance and frequency-based modeling. If additional results become available before the end of the discussion period, we will report and include them.

“...the paper could emphasize the generated dataset which is also part of the contribution of this paper. And it may be helpful to emphasize the importance of these generated datasets and how they can contribute to real-world applications.”

We agree that the generated datasets represent a valuable contribution of this work and are planning on releasing them alongside our code. The domains we consider in this work are generally lacking in publicly available datasets and we hope that our data can lay the foundations of a standardized and reusable testbed for future methods targeting spherical signal learning from 3D data. Additionally, the scale of these datasets enables the broader application of ML models in this space which has historically lagged behind other more mainstream applications such as text, image, and audio.

“How did you handle the distance in the 3D meshes when performing the Fourier transformation? I understand that applying the Fourier transformation on the S2 space is natural. However, when working with mesh data, defining the origin of the system requires transforming the mesh points by considering their distances to the origin. In this case, I believe applying spherical harmonics alone is not sufficient. How did you address the handling of these distances?”

You’re absolutely correct that the SH alone are not enough to encode this type of 3D geometric data as they only capture the angular ($S^2$) information. To incorporate the radial (distance) information, we use a radial network which transforms pairwise distances into feature embeddings. This design parallels approaches like Equiformer [1], which similarly uses Gaussian-based radial basis functions transformed via learnable MLPs to encode edge distance information in their equivariant attention layers.

[1] Liao, Yi-Lun, and Tess Smidt. "Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs." The Eleventh International Conference on Learning Representations.

“In Figure 2, it is said it will use "grid of pre-computed harmonic basis functions". Does this mean that each channel will be corresponding to a certain fixed distance?”

This question relates to the decoder stage of our pipeline, where we transform features from Fourier space back to real space. The phrase “grid of precomputed harmonic basis functions” refers to our use of a fixed sampling grid on the sphere, over which we evaluate the inverse spherical Fourier transform to recover real-space signals. These basis functions are precomputed purely for efficiency, e.g. to avoid recomputing SH values during every forward pass, and are not related to encoding different radial distances. To clarify: no, the channels do not correspond to fixed distances, they represent frequency components on the sphere, not spatial or radial positions.

For example, suppose $L_{\max} = 10$. Then there are 45 total frequency components (since we have $\sum_{l=0}^{10} (2l + 1) = 121$ modes, but due to conjugate symmetry for real-valued signals, we only need to store the 45 complex coefficients). If we are modeling a function $f: S^2 \rightarrow \mathbb{R}^{50}$, the frequency-domain representation would be a tensor of shape [B, 45, 50], where B is the batch size. After applying the inverse Fourier transform on a set of, say, 10,000 sampled points on the sphere, the real-space signal is recovered as a tensor of shape [B, 10,000, 50], representing the function values at each of those 10K spherical coordinates.

Thank you for your effort in providing the rebuttal and the new experiments regarding PointNet++. These address my main concerns.

Thank you for your review. We are working on a revision to incorporate your feedback, but we address your comments and questions here as well.

“The proposed method consists of certain existing algorithms and is short of a systematic theory.”

Our primary goal in this work is not to create a systematic theory but to address a lack of methods for mapping 3D geometric inputs (e.g. point clouds and meshes) to spherical signals, despite the fact that this type of mapping is found across many different domains. Our proposed method is general enough to demonstrate strong performance across a range of diverse domains.

“There are some English presentation and grammar errors in the paper.”

Thank you for pointing this out. We will carefully revise the paper to improve clarity and grammatical correctness.

“There are several hyperparameters for implementing the proposed method, what are the best values for a task given the dataset?”

G2Sphere was trained using the Adam optimizer with a learning rate of 1e-4 and a decay rate of 0.95. In general, we use the largest batch size available to us given the compute required for the domain. For example, when using large meshes, such as in the radar domain, we use a smaller batch size of 4. In the policy learning domains, where the geometric input is smaller, we can use a larger batch size of 64. Additional hyperparameters, including the number of layers in both the encoder and decoder, the layer-wise feature dimensions, and the choice of maximum SH frequency , $L_{max}$ are detailed in Appendix G. For completeness, the appendix also provides the hyperparameter settings used for all baseline methods.