Affine Steerable Equivariant Layer for Canonicalization of Neural Networks

• W1. A proper comparison against fully equivariant networks for classification (e.g., InvarPDEs-Net and InvarLayer from [1]) is not included in the experiments. Including these baselines would strengthen the paper, especially since canonicalization may benefit from pre-training in non-invariant domains (e.g., ImageNet-1k in this paper) unlike fully equivariant networks.
• W2. It seems possible to use the moving frame in Definition 8 (or the equivariant matrix in Theorem 10) directly for canonicalization without multiplication by type-0 features (Eq. (10)), as this coincides with the definition of equivariant frames in frame averaging [8]. Adding this as a baseline would strengthen the results, as it can be understood as ablating trainability from canonicalization.
• W3. It was unclear to me how the definition and property of moving frames given in Definition 8 and Theorem 9 logically leads to the construction of equivariants using relative equivariants given in Section 2.3.
• W4. The paper demonstrates a single application of the proposed layer as the last layer of a canonicalization network, which extracts 2-channel type-1 features from type-0 features, while all other layers of the network map between type-0 features [1]. This misses (1) feature maps of type >1, (2) layers taking non-scalar features as input, and (3) layers mapping between non-scalar features, all of which are possible within the proposed framework. Testing (a subset) of these would strengthen the paper. A straightforward way is to modify InvarPDEs-Net and InvarLayer from [1] to have non-scalar hidden features, analogous to steerable networks in [2].

Minor typo

• Line 13: InvarLayer -> InvarLayer (Li et al. 2024)

[1] Li et al. Affine equivariant networks based on differential invariants (2024)

[2] Cohen and Welling, Steerable CNNs (2017)

[8] Puny et al. Frame averaging for invariant and equivariant network design (2021)

1. The paper conducts experiments solely on MNIST datasets and skips existing methods like [1] for comparison.

2. The contribution does not seem significant and the experiments section is limited.

3. The motivation for using the proposed approach given the already existing methods that conduct canonicalization, and steerable layers, is unclear to me after going over the paper.

4. Definition 4, Proposition 5 as well as Theorem 7 can be improved in terms of formalizing as well as readability of equations.

[1] Implicit Convolutional Kernels for Steerable CNNs, Zhdanov et al.

1. Experiment Design: Using invariant classification to demonstrate the effectiveness of the proposed EquivarLayer for canonicalization is somewhat indirect. The true strengths or limitations of EquivarLayer are obscured by the downstream ResNet-50 prediction network, yet five pages are dedicated to deriving the EquivarLayer. Since canonicalization is a downstream task, its accuracy depends significantly on the model’s ability to preserve information—a quality typically evaluated by learning equivariant features, performing invariant classification, or computing equivariant error. Key questions remain, such as whether EquivarLayer improves upon InvarLayer (as it is a generalization) and how it compares to other steerable methods achieving subgroups of affine group equivariance.
2. Experiment Setup and Comparability: The experiment setup limits comparability with other methods. The train/test split is not provided, and the scale factor range (0.8 to 1.2 or 1.6) differs from that of other scale-equivariant papers, which often use a range of [0.3, 1]. Additionally, key scale-equivariant works, such as [1], are not cited. These omissions make it challenging to compare this model with existing equivariant CNNs, especially since no equivariant network baselines are included in the experimental section.
3. Clarity in Derivations: The derivations following Eq. (16) in Section 2.3 are unclear. The notation for symbols like $u_x, u_y$ is not adequately defined, and it was only by referring to Li et al. (2024) that I understood they represent gradients. While this is a minor issue, a more significant concern is the lack of discussion around $\alpha$ from Eq. (14). Instead, the paper provides an example of an equivariant matrix (Eq. 17) without explaining the derivation of $\alpha$. Additionally, one of the advantages of equivariance is its ability to preserve information. While the proposed method is computationally efficient, it may sacrifice some information, as it limits gradients to second order and relies on differential invariants for local features. It remains unclear if the gradient order relates to information preservation. The derivations also do not clarify why only first- and second-order gradients are used or how the equivariant matrix is derived.

[1] Sosnovik, Ivan, Michał Szmaja, and Arnold Smeulders. "Scale-Equivariant Steerable Networks." International Conference on Learning Representations.

I will categorize the issues I find with the paper into three categories: (1) Claimed novelty; (2) Lack of presentation of concrete examples given the level of generality claimed and (3) experimental validation. I want to note from the onset that I am open to raising my score, and the degree to which I am penalizing (lack of) novelty here is very much based on what the authors themselves claim.

• (1) The paper uses language that supports the notion where a lot of the concepts are novel (in the sense of new mathematical objects) rather than, their application in this form in the context of machine learning being novel.
• Considering the framework presented up to section 2.3, I would like to highlight obviously the work of Olver himself in [4] where the same framework of constructing equivariant maps using moving frames based on his previous work is summarized (ending up with the abstract operator of eq. (13)). Directly related is the work of Sangalli et. al which also uses the prolongation of the group action on the jet space for invariantization (based on Olver's work) in the context of image and volumetric data [1-3]. From [1] appendix B, the general framework is directly applicable taking X = $\mathbb{R}^2$, $Y = \mathbb{R}^{c}$, $Z = \mathbb{R}^{c^{'}}$, we arrive at the same principle where an equivariant operator $\mathcal{F} \to \mathcal{F}^{'}$ on the function spaces is defined implicitly once a G-invariant operator on the product space $X \times Y$ is available ($g \cdot (x, u) = (gx, \rho(g)u) = (gx, u)$ ($\rho = \rho_0$), corresponding to $g \cdot f(x) = f(g^{-1}x)$). In this work the notation is less clear and it appears that $Y = \mathcal{F}$ indicating a higher order operator, however the differential invariants are always (correctly) constructed with the same $x \in X$ argument i.e. $\hat{\mathcal{I}}(f)(x) = I(x, f(x))$ (we make use of differential invariants dependent on $x$). It should be highlighted then that the 'equivariant' $\mathcal{E}: X \times Y \to Z$ presented here is a $G$-equivariant map on the product space respecting $\mathcal{E} \circ \rho(g) = \rho^{'}(g) \circ \mathcal{E}$ , and would correspond to the case where $\rho \neq \rho_0$ ((16) in [1]). This is indeed a different group action and the authors seek to effectively work with induced representations which also acts on the codomain $L_{\rho}(g)(f)(x) = \rho(g)f(g^{-1}x)$, however it should not be presented as a departure from this framework if one is simply changing the representation used.
• The authors should also try and clarify further how their framework differs from [6] in its usage of the sup-normalized differential invariants. Are the polynomial relative invariants of [6] and the relative invariants here the same? I would also highlight [5] and [7] which also deal with differential invariants of the affine group of the plane, where the zero division errors are treated with a different formulation (see [5] (34) and (35)).
• Regarding (2), I would have liked to see more concrete examples of the framework being employed in the context where the additional flexibility in terms of input/output representations is valuable e.g. beyond image data, see for example [9]. I am not concerned with SOTA results, simply a validation that the increased generality allows the use of the differential invariants for a larger set of applications and modalities. It would also be useful for practitioners if more concrete examples of the realized (beyond theoretical) operators is presented.
• Regarding (3), it would be useful to quantify more clearly the relationship between the amount of data augmentation used and the size of the dataset for each experiment. More importantly, what would be very useful here is to have a thorough presentation of the time/space complexity of each layer employing this framework, see e.g. [1] appendix D or [6], with the inclusion of standard (3-channel) image data.

[1] "Moving frame net: SE(3)-equivariant network for volumes", Sangalli et. al, 2023 [2] "Differential invariants for SE(2)-equivariant networks", Sangalli et. al, 2023 [3] "Equivariant Deep Learning Based on Scale-Spaces and Moving Frames", Sangalli 2023 [4] "Using Moving Frames to Construct Equivariant Maps", Peter Olvar (March 2024) [5] "Affine Differential Invariants for Invariant Feature Point Detection", Tuznik et. al 2018 [6] "Affine Equivariant Networks Based on Differential Invariants", Li et. al, 2024 [7] "Affine invariant detection: edge maps, anisotropic diffusion, and active contours", Olver et.al 1999 [8] "On Relative Invariants", Fels & Olver 1997 [9] "Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data" Finzi et. al 2020