Learning Rotation-Invariant Representation using Rotation-Equivariant CNNs

• The GIE method relies on rotation-equivariant properties, requiring GCNN backbones, which may limit the method’s applicability in scenarios where GCNNs are impractical.
• While the study includes datasets with varying rotation properties, a broader analysis of the model’s effectiveness on datasets with more complex real-world invariances could strengthen its claims.
• The experiments are mainly restricted to classifications. It would be better validate the method on the applications with rotation variance issues, such as object detection and key point matching.
• Following the last point, the image level global rotation is not very common in real-world scenarios. So the experiments on imag classification cannot fully demonstrate the benefits of the method. As shown in the experiments, the proposed method cannot bring improvement for the original dataset NR dataset. It implies the image-level rotation is not common in the real world. Differently, the rotation issue can be common in the local features. And the rotation invariant representation can be validated better on these related applications.

1. Effectiveness of SSL and GIE with E2CNN: A primary concern is that an E2CNN can potentially be trained without SSL or GIE and still achieve robust rotation invariance. If this is the case, the need for incorporating SSL and GIE to improve E2CNN becomes unclear. While Tables 1 and 2 show that SSL+GIE+E2CNN outperforms SSL+E2CNN, it’s worth noting that E2CNN is inherently equivariant to the p4-group, which may indicate a limitation in the number of orientation samples. The experiment in the appendix, comparing the p4-group and p8-group, raises questions, as the authors have matched GPU memory usage instead of parameter count—an approach that deviates from common practice in equivariant network studies, which typically prioritize matching parameter counts. Since matched parameter counts are more relevant for fair comparisons, this paper should provide evidence that SSL+GIE+E2CNN is superior to SSL+E2CNN, and to E2CNN alone, with matched parameter counts for p16- and even p32-groups. I think that matched parameter count is more important than matched GPU memory consumption in the long run as GPU hardware is evolving fast.

I tend to change score based on the answer to this question.

2. Motivation and Benchmark Choices: Some motivations in Figure 1 are debatable. While humans may struggle with reading rotated text, recognizing rotated objects generally relies more on shape than orientation, so rotation is less problematic for object recognition without canonicalization. Furthermore, E2CNN has achieved over 99% accuracy on rotated handwritten digits in MNIST, effectively distinguishing between rotated 6 and 9, likely due to shape differences inherent in handwriting styles. Despite the relevance of rotated MNIST as a benchmark in rotation-equivariant literature, this dataset is absent from the paper’s evaluations. MNIST offers a zero-background advantage, which is ideal for testing rotation without cropping, so including it could strengthen the results and make us easy to compare the results in equivariant literature.

3. Parameter Comparison in Results: The SSL+GIE+E2CNN models consistently have more parameters than SSL+E2CNN due to the additional equivariance predictor, which makes the results less convincing. For instance, in Table 1, SimSiam+GIE+E2CNN+EqvPred (76.54%) uses 12.83M parameters, while SimSiam+GIE+E2CNN+group pooling (73.26%) has 11.14M parameters. This additional 15% in parameters only yields a 4% accuracy increase, which is not particularly impressive. It would strengthen the paper if the authors could demonstrate that SSL+GIE+E2CNN performs better than SSL+E2CNN with fewer or equal parameters.

1. The technical novelty is limited as the authors have used existing rotation equivariant network and trained it to generate rotation equivariant only.
2. The approach has high requirements. It requires specific CNN backbone network. It is not compatible with Vision Transformers.
3. The approach is compared with just a few baselines: SIMCLR and SimSiam.
4. The performance of the approach is lower in some cases when dataset is not rotated. This shows its limited use cases (only for rotation invariant datasets).
5. Basic RotNet approach is not compared with the E(2)-CNN.

The main weakness of this work is that, after reading the paper multiple times, it is still unclear to me what the contribution of the paper is over existing work. I will outline this in detail as follows:

• positioning relative to existing work. As mentioned in the related work section, there already exists multiple works that learn robust equivariant and invariant representations in an unsupervised manner. However, the authors do not outline what the differences are between this work and existing works. The objective of the related work section should be to position your work relative to others. Not to list what exists.

  To complement my previous statement, papers such as [1], which propose a very similar method to learn invariant and equivariant representations, are not cited, let alone discussed. In fact, the formulation of [1] is more general, and considered wider, more complex groups than this paper. The contribution of the paper over existing work remains uncertain.

• Components and analyses already done in previous work. One of the core components of the method is the group attentioning operation the authors introduce. However, this has been proposed previously over 4 years ago in [2, 3]. See, for example, how Fig. 3 in [2] summarizes Eqs. (1-8).

  Note in addition that [3] explicitly mentions the problem with squared images (Appx. C).

• repeated analyses and conclusions. I consider many of the statements included / derived in this paper to be well-known from literature. For example: Line 316 - … we evaluate the linear class. acc. on both the NR and R datasets, which included images rotated in four directions. As shown GIE achieves highest accuracy on the R dataset. (Similar in lines 360 and 367). Results in Figure 4. -> Please note that this is a direct effect of using G-CNNs. This has been known and proved for many years. For example, note how Fig 4 clearly resembles analyses done over 5 years ago by [4] (see Fig 4 in [4] ). It is unclear what the new insights are. Analyses similar to those in Fig 5 were done in [1] as well. In fact, similar analyses have been done previously based on class-dependent statistics (not only dataset level statistics) in [5].

An additional concern relates to how the method is not comparing its results with previous related works, e.g., [1].