On the Fourier analysis in the SO(3) space : the EquiLoPO Network

On Computational Efficiency and Practical Applications We have included a complexity analysis in the revised manuscript. The computational complexity of our convolution operation is $O(N^3 L_{\text{in}}^3 L_{\text{out}}^3 D_{\text{in}} D_{\text{out}})$, where $N$ is the spatial dimension, $L_{\text{in}}$ and $L_{\text{out}}$ are the maximum expansion order of input and output data respectively, and $D_{\text{in}}$ and $D_{\text{out}}$ are the number of input and output channels. We compare this with traditional 3D convolutions, which have complexity $O(N^3 D_{\text{in}} D_{\text{out}})$. On standard hardware (e.g., GPUs), our model requires more computation and memory, but we provide benchmark results showing that training and inference times are still practical for moderate-sized datasets. We also discuss optimization techniques to improve efficiency.

On foundation models While foundation models like DINOV2 are powerful, they may not explicitly handle rotational equivariance, which is crucial in domains like medical imaging, molecular modeling, and physics simulations. Specifically to DINOV2, it has incorporated many different pieces, starting from very careful training data curation, to distillation and DL tricks, without any specific novelty in any individual component.

On the other hand, our method provides several innovations, e.g., a way to incorporate rotational symmetry directly into the network architecture, a practical activation in the Fourier domain, etc, potentially improving performance on tasks where such symmetry is present. We believe that our approach offers complementary benefits and can be integrated with or enhance foundation models in specific applications.

On Activation Function Details We chose local activation functions with trainable polynomial coefficients to introduce nonlinearity while preserving equivariance. The polynomial approximation allows us to analytically compute the activation in the Fourier (Wigner) space. We selected a quadratic polynomial (degree 2) as a balance between approximation accuracy and computational efficiency. Higher-degree polynomials would increase computational complexity significantly. We also experimented with fixed-coefficient polynomials and other activation strategies, but the trainable polynomial provided the best performance.

On adaptive coefficients The adaptive coefficient approach is beneficial when the distribution of feature values varies significantly across different layers or datasets. It allows the activation function to adjust to the statistical properties of the input data dynamically. This is particularly useful in cases with high variability or when the data mean is shifted, improving the activation approximation and overall model performance.

On the Impact of Batch Normalization on 6D Data Implementing batch normalization for high-dimensional 6D data in our network was actually straightforward. Batch normalization operates by normalizing data along all dimensions except the feature (channel) dimension, which involves computing the mean and variance over the batch and spatial dimensions.

In our case, since we represent data in the $\text{SE}(3)$ space using Wigner $D$-matrix coefficients, we can compute the mean and variance directly from these coefficients. The properties of the Wigner representations allow for efficient calculation of statistical moments needed for normalization.

Importantly, this normalization does not affect the equivariance properties of the network. Since all coefficients of each function are divided by the same normalization factor (the standard deviation) and only zero-coefficient is shifted the mean, the transformation properties under rotations remain unchanged.

As a result, we did not encounter any unique challenges or issues when adapting batch normalization to our 6D data. The standard batch normalization procedure could be applied effectively, and tuning the parameters followed the usual practices without requiring special considerations.

On augmentation in 3D Thank you for bringing this to our attention. You are correct that the computational cost of applying rotations to data is negligible compared to the overall cost of training a neural network. When we mentioned "increased computational demand," we did not mean the overhead of performing the rotations themselves.

Our point is that achieving rotational invariance through data augmentation requires the network to see training examples in many different orientations. This necessitates generating a much larger augmented dataset to cover the space of possible rotations adequately. Even with dynamic data augmentation—applying rotations on-the-fly during training—the network must process a more diverse set of inputs, which leads to longer training times and increased computational load.

On classes of convolution Thank you for your comment. We appreciate the opportunity to clarify the distinction.

While both group convolutional networks and steerable networks are types of group convolutional networks aiming for equivariance, they differ in how they implement convolutions and handle rotations:

• Group Convolutional Networks: These networks perform convolutions over both translations and rotations, effectively operating on the group space (e.g., SE(3) for 3D rotations and translations). This involves using 6D filters (3D spatial dimensions + 3D rotations) and convolving over all combinations of positions and orientations. Due to computational constraints, they often work with discrete rotations, approximating the continuous rotation group with a finite set of angles.

• Steerable Networks: These networks perform standard translational convolutions over the spatial domain but design their filters to be equivariant under rotations. The filters are constructed using irreducible representations, transforming in a specific way under rotations. This approach avoids the need to convolve over rotations explicitly and doesn't require 6D filters.

Our method bridges these approaches:

• We perform convolutions over the continuous rotation group SO(3) like group convolutional networks but avoid discretization by working analytically in the spectral domain using irreducible representations (Wigner D-matrices).

• Unlike some steerable networks that constrain filters to specific forms to ensure equivariance, our filters are fully trainable and unconstrained (within the band-limit set by the maximum expansion degree).

By clarifying this, we aim to highlight how our method combines the strengths of both approaches: capturing full roto-translational symmetries without relying on discrete rotations, and allowing flexible, expressive filters without strict constraints.

On orientational pooling Orientation pooling refers to the operation of aggregating features over the rotational dimension to produce rotation-invariant representations.

Section 3.1 Thank you, we have moved the introduction of key notation from the appendix to the main text to improve readability and ensure that readers have all necessary information upfront.

Our method differs in that we use irreducible representations (Wigner-D matrices) as the Fourier basis for achieving analytical equivariance in the continuous SO(3) group, and we introduce a local activation function in the SO(3) space. This allows for unconstrained filters and avoids reducing the architecture to a steerable network. We have clarified these differences in the manuscript.

On direct or semi-direct product structure Thank you for your remark. We changed this part in the manuscript.

On nonlinearities and compact representations We agree that this statement could be misleading and even questionable. And it would not hold true for all types of data. Our intention was to suggest that for data of hierarchical structure, activation functions serve to learn compact dictionaries of features. Here we can cite classic literature, e.g., Goodfellow et al.'s "Deep Learning" book. But again, this would not be true for other types of data.

On “pointwise” and “local” terms Yes, in our context, "pointwise" and "local" refer to operations applied independently at each point in space (or rotation). We have clarified this terminology in the manuscript to avoid confusion.

On Table 1 We have added additional information on the memory requirements and running time during inference. Please see some extraction of the information below, (Memory, FLOPS, and Time consumption for EquiLoPOResNet-18, ILPOResNet-18 and ResNet-18):

Thanks for addressing my questions. Please read my below thoughts.

On augmentation in 3D I'm not sure I agree with your position. Indeed the network must process a more diverse set of inputs, which leads to longer training times and increased computational load, but imposing equivariance also does just that. You either augment the filters with extra transformed copies of the original, in essence adding extra dimensions to the activations, or you use a steerable basis, which also adds extra dimensions to the activations. This is evident in the updated Table 1. I don't think this detracts from the central message of the paper, I just think the statement is not balanced. Equivariance requires more FLOPS, but the upside is a model that does not violate physics and basic imaging properties.

Group convolution and steerable convolution Thanks for adding this section. This distinction is common among the ML community.

Spectral domain convolutions One thing I do not understand is that you say your filters are unconstrained because you operate in the spectral/harmonic domain. I can see this in Equation 9. Indeed you can transform the filters back to the physical domain by instantiating $S_{k_1, k_2, k_3, k_4}^{l_1, l_2}(\vec{r}_0)$. If you do this you see that S, which is a representation of your filter, is a linear combination of spherical harmonics, the coefficients being a mixture of the learnable weights w and Clebsch-Gordan coefficients. In this sense, the representation of the filters in the physical domain is indeed constrained, and as far as I can tell, not different from steerable convolutions.

Nit: line 238 "Eq ??" is incorrectly rendered

Just to say, thank you once again for the rebuttal. I see you have put in a lot of work updating the manuscript and answering everyone's issues.

Response to Reviewer

Thank you for your thoughtful review and appreciation of our work. We address your concerns below.

Weaknesses:

1. The introduction does not detail the research background and related literature, and the research motivation is not sufficient; it only mentions the advantages of combining the two.

   Response: We appreciate your feedback. In the revised manuscript, we have enriched the beginning by moving a significant part of the motivation and related work from the appendix to the main text. This provides a more comprehensive background and clarifies the motivation behind our approach.

2. There is a lack of comparison with the latest literature.

   Response: We have updated the related work section to include comparisons with recent advancements in equivariant neural networks. This highlights how our method relates to the latest literature and underscores its novelty.

3. The detailed computational cost of different ResNet-style architectures should be given, including training time, number of model parameters, time and space complexity.

   Response: Thank you for this suggestion. We have added additional information to the table with computational analysis to the appendix of the manuscript. The table below summarizes the memory usage, GFLOPs, time and space complexities, and inference time per batch of 32 samples for each model.

   Table: Computational Costs for EquiLoPOResNet-18, ILPONet-18, and ResNet-18

   Model	Memory (GB)	GFLOPs	Time Complexity	Space Complexity	Inference Time per Batch (seconds)
   ResNet-18	2.47	35.52	$O(B N^3 D_{\text{in}} D_{\text{out}})$	$O(B N^3 D_{\text{out}})$	0.03
   ILPONet-18	7.14	19.58	$O(B N^3 L^3 D_{\text{in}} D_{\text{out}})$	$O(B N^3 L^3 D_{\text{out}})$	0.30
   EquiLoPO (local activation)	27.02	68.44	$O(B N^3 L_{\text{in}}^3 L_{\text{out}}^3 D_{\text{in}} D_{\text{out}})$	$O(B N^3 L_{\text{out}}^3 D_{\text{out}})$	2.49
   EquiLoPO (global activation)	30.05	53.75	$O(B N^3 L_{\text{in}}^3 L_{\text{out}}^3 D_{\text{in}} D_{\text{out}})$	$O(B N^3 L_{\text{out}}^3 D_{\text{out}})$	2.13

   Where:

   • $B$: Batch size
   • $N$: Spatial dimension (e.g., image size)
   • $D_{\text{in}}$, $D_{\text{out}}$: Input and output feature dimensions
   • $L$, $L_{\text{in}}$, $L_{\text{out}}$: Expansion degrees in the Wigner D-matrix representations

We hope that these revisions address your concerns. Thank you again for your valuable feedback, which has helped us improve our manuscript.

On the Degree of equivariance Thank you for your question. Our model achieves analytical rotational equivariance by design, as all operations are inherently rotationally invariant or equivariant. To confirm our implementation, we tested the model with 90-degree rotations and observed discrepancies in predictions on the order of $10^{-8}$ to $10^{-9}$, which are attributable to numerical precision limits. These results confirmed the correctness of our implementation, and we did not report them in the paper since equivariance was an expected property of our design.

On the best model selection Selecting the best model depends on various factors, including the specific characteristics of the dataset, such as the presence of rotational symmetries, class imbalance, and the complexity of the task. Based on our experiments, local activation functions with adaptive coefficients generally provide strong performance across different datasets. However, the optimal choice may vary.

We recommend that end users consider the nature of their data and perform validation experiments to determine which activation function and model configuration yield the best performance for their specific application.

On the 6D feature map The computational complexity of our convolution operation is $O(N^3 L_{\text{in}}^3 L_{\text{out}}^3 D_{\text{in}} D_{\text{out}})$, where $N$ is the spatial dimension, $L_{\text{in}}$ and $L_{\text{out}}$ are the maximum expansion orders of input and output data respectively, and $D_{\text{in}}$ and $D_{\text{out}}$ are the number of input and output channels. We compare this with traditional 3D convolutions, which have complexity $O(N^3 D_{\text{in}} D_{\text{out}})$. Fortunately, our model requires much smaller number of channels. EquiLoPONet uses fewer parameters then the classical resnet architecture but needs 2 times more flops.

On the Appendix Thank you for your feedback. Due to the page limitations and formatting guidelines, we had to place some of the detailed explanations, experimental settings, and architectural details in the appendix. Our intention was to present the core contributions, main ideas, and essential results in the main paper, while providing supplementary material in the appendix for readers interested in the technical details.

We acknowledge that this can make it challenging for readers to fully grasp all aspects of our methodology. We have moved related work and essential explanations back to the main text.

On Figure 1 We understand that the figure may not fully convey the construction of 6D feature maps. Our primary goal with Figure 1 was to illustrate the importance of local equivariance and to compare two different approaches:

1. 3D to 3D (Translations Only): In this approach, feature maps contain probabilities of features at each point in space, focusing solely on translations.

2. 3D to 6D (Translations + Rotations): Here, at each point in space, we have not just a single feature value but a distribution over rotations, capturing both translational and rotational information.

We acknowledge that visualizing 6D data in a 2D figure is inherently challenging. While it is difficult to depict the full 6D feature maps, Figure 1 aims to conceptually demonstrate the difference between these two approaches and highlight how incorporating rotations allows the network to capture richer, more informative features that are locally equivariant. We will try to draft a supplementary Figure with 6D feature maps before the deadline.

On explanations in Sec 3 We have revised Section 3 to ensure consistent and clear notation. Variables are now uniquely defined to avoid confusion. We have moved key equations from the appendix into the main text, providing necessary derivations and explanations to make the paper self-contained. We have reformatted all equations to comply with standard conventions and the paper's formatting guidelines, enhancing readability.

On the analysis Thank you for your feedback. In our work, we adhered to the conventional architecture of convolutional neural networks (CNNs), utilizing the same sequence of operations—including convolution, activation, pooling, and normalization layers—as is standard practice. Our primary aim was to isolate and evaluate the impact of the proposed activation functions within this familiar framework.

Because we maintained standard settings for normalization, pooling, and convolution parameters, we did not conduct ablation studies on these components. We focused our analysis on the activation functions since they represent our main contribution and the key variable in our experiments. By keeping other factors constant, we ensured that any changes in performance could be attributed to the activation functions themselves.

On evaluation metrics Regarding evaluation metrics, we used accuracy and AUC-ROC as proposed by the authors of the dataset. These metrics are widely accepted and facilitate fair comparisons with existing methods.

On the best architecture rules Thank you for pointing this out. The different configurations in Table 1 correspond to various activation functions and settings we explored to evaluate their impact on performance. The optimal configuration may vary depending on the specific dataset and task. We recommend that users consider the characteristics of their dataset and possibly perform validation experiments to select the model configuration that yields the best performance for their specific application. We acknowledge that providing more guidance on model selection would be helpful, and we will aim to include such recommendations in future work.

The reported metrics were computed using a single random seed. We acknowledge that reporting results over multiple seeds would provide a more robust evaluation by accounting for variability due to random initialization. We plan to perform multiple runs and report mean and standard deviation in the final version.

On grammar We have thoroughly proofread the manuscript and corrected all grammatical and typographical errors to enhance clarity.

On the context and introduction: We agree that providing context and situating our method within the broader equivariant deep learning literature is important. In the revised manuscript, we have moved the related work section from the appendix into the main text.

On the equivariant convolution layer: You are correct that equivariant convolution layers are well-established and fall within the steerable class of group CNNs. Our contribution, however, lies in extending group convolutions to operate analytically over the continuous SO(3) group, rather than over discrete or finite rotation groups. This enables true rotational equivariance in 3D space without relying on approximations or discretizations. Additionally, our method bridges the gap between traditional group convolutional networks and steerable convolutional networks by combining the strengths of both approaches. Specifically, we allow for unconstrained trainable filters (as in group convolutions) while achieving analytical equivariance (as in steerable convolutions).

On Fourier o ReLU o InverseFourier: We have considered inverse Fourier-based activation functions, where one transforms the data back to real space, applies a nonlinear activation like ReLU, and then transforms back to the Fourier domain. However, our goal is to achieve analytical equivariance throughout the network. Transitioning to real space introduces discretization and sampling, which can break analytical equivariance due to the finite resolution of the spatial grid. The ReLU function applied in real space results in an output with infinite bandwidth in the Fourier domain, meaning it requires an infinite number of Fourier coefficients to represent it exactly. Practically, we have to truncate this to a finite number of coefficients, leading to an approximation that depends on the sampling scheme and can introduce errors. This compromises the equivariance properties because the activation's output in the Fourier domain will depend on the specific set of sampled points used during the inverse and forward transforms.

Nonetheless, we acknowledge that an explicit comparison could provide additional insights. If time permits, before the deadline, we will conduct extra experiments.

On the classification: Our intention was to highlight two distinct approaches: 1) Group convolution over roto-translations (SE(3)) ; 2) Steerable convolution over translations with irreducible representations. We acknowledge that both methods fall under the broader category of group convolutional networks, and the distinction may have been misleading. We have revised the manuscript to clarify this classification, emphasizing the difference between performing convolutions over the group itself (SE(3)) versus over the base space (R^3) with filters transforming under group representations. Our method bridges these approaches by performing group convolutions over continuous SO(3) without imposing constraints on the filters, thus combining the benefits of both methods.

On the explicit conclusions: We have added take-away messages to the conclusion.

On 3D point clouds: Our current method is specifically designed for regular volumetric data, and extending it to irregular data such as 3D point clouds would require significant modifications. We have acknowledged this limitation in the manuscript. In future work, we plan to explore how our approach could be adapted to point cloud data and compared with benchmarks like e3nn.

On the equations for group convolutions: Thank you! We have reviewed and corrected the equations for group convolutions. We now clearly state when convolution or correlation is being used and have updated the notation accordingly. We have also standardized the notation throughout the manuscript so that the integration measure is consistently placed at the beginning of the integral expressions.

On the filter constraints: Our method allows for trainable filters without additional constraints beyond those imposed by the finite expansion order (i.e., resolution limits). While parametrizing kernels in the Wigner-D basis ensures equivariance, we do not restrict the filter shapes to specific forms as in steerable networks.

On canonicalization: Actually, we were referring to a different concept. In some domains, it is possible to extract a natural orientation and align the data accordingly—a process sometimes referred to as canonicalization. Examples include aligning images of houses so that the baseline aligns with the horizon, or defining local coordinate frames around residues in proteins. These methods leverage inherent structural or contextual cues to establish a consistent orientation across the dataset, thereby reducing variability due to rotations.

On citations and local activation: We have added the suggested citation of Andrearczyk2020 and Kaba2023. We've also added Bekkers2024 and Weiler2019 when discussing the locality of the activation.

We thank the referee for the thoughtful feedback and for giving us the opportunity to clarify our contributions. We apologize for any confusion in our previous response. To clarify, we see a distinction between two main approaches in the literature for achieving rotational equivariance:

1. Group Convolutions (Roto-Translational Convolution):
   These methods perform convolutions over the entire group of rotations and translations, specifically the SE(3) group in 3D space. This involves convolving over both spatial positions and orientations (roto-translations). Examples include methods that use group convolutions over discrete subgroups of SE(3), such as "3D G-CNNs for Pulmonary Nodule Detection" by Winkels and Cohen (2018), and "CubeNet: Equivariance to 3D Rotation and Translation" by Worrall and Welling (2018). While these approaches achieve equivariance, they often rely on discretization of the rotation group and lack analytical equivariance with respect to continuous rotational space.

2. Steerable Networks (Translational Convolution with Equivariant Filters):
   These methods perform standard convolutions over spatial positions (translations) but use filters that are designed to be equivariant under rotations. The filters are constructed using irreducible representations and are constrained to lie within specific subspaces to ensure equivariance, as seen in works like "Steerable CNNs" by Cohen and Welling (2017), "3D Steerable CNNs" by Weiler et al. (2018), and "Tensor Field Networks" by Thomas et al. (2018). This often involves using predetermined kernel bases or imposing harmonic constraints on the filters.

Our method aims to bridge the gap between these two approaches by combining their strengths:

• Analytical Equivariance over Continuous SO(3):
  Like steerable networks, we achieve analytical rotational equivariance over the continuous SO(3) group. We operate in the spectral domain using Wigner $D$-matrix representations, allowing us to avoid discretization and approximations associated with sampling the rotation group.

• Unconstrained Trainable Filters:
  Unlike steerable networks that constrain filters to specific harmonic subspaces or predefined bases, our method allows for unconstrained, fully trainable filters (within the band-limit imposed by the maximum expansion degree). This means the filters can learn any function expressible in the chosen basis, providing greater flexibility and expressiveness.

In essence, our method performs group convolutions over continuous rotations (SO(3)) without imposing strict constraints on the filters beyond the necessary band-limit.

Regarding canonicalization:

Yes, we are essentially referring to the same concept. Thank you again for your valuable feedback.

Thank you for your thoughtful remarks and for sharing your perspective. We apologize if our previous explanations were not sufficiently clear, and we appreciate the opportunity to further clarify this point.

In steerable networks, the convolution operation in the 3D case is expressed as:

where constraints are imposed on $p_{k_1 k_2}^{l_1 l_2}(r_1)$ to ensure equivariance. These constraints lead to a parametrization of the kernels within a subspace spanned by irreducible representations (irreps), forming a complete basis for equivariant operations.

In our work, we consider the operation:

Here, we do not impose explicit constraints on $w$ because this operation remains equivariant regardless of the specific form of $w$. This is a key distinction: while steerable methods constrain the kernel $p$ to ensure equivariance, our method naturally achieves equivariance through the structure of the convolution operation without additional constraints on $w$.

By performing a Fourier-Wigner decomposition of the rotational component, we arrive at:

where $S$ depends on the Wigner coefficients of $w$. At this point, the formulation aligns with the steerable framework because $f^{l}$ represents irreps in $\mathrm{SO}(3)$, and one could derive the full basis for $S$ using the theory of steerable networks.

We elaborate on the relation between our method and steerable networks in Appendix A of our paper. In this appendix, we demonstrate how our convolution operation can be connected to the steerable convolution. In a particular case they are equivalent.

The Core Differences:

1. Filter Definition: While the mathematical foundations overlap, the way filters are defined and utilized in our method differs from traditional steerable networks.

2. Interpretability: Our filters retain interpretability, as they can be transformed back into real space, allowing us to observe directly how the distribution of rotations maps to outputs.

In summary, although the underlying mathematical structures are very similar, the operational framework and filter design in our method provide advantages in flexibility and interpretability. We believe these distinctions are significant and contribute to the novelty of our approach.

We hope this clarification addresses your concerns. Thank you again for engaging deeply with our work and prompting us to refine our explanations.