EKAN: Equivariant Kolmogorov-Arnold Networks

1. The technical contributions presented in the paper lack novelty, as the core techniques have been previously explored in the literature.
2. Although EKAN is compared against MLP, KAN, and EMLP, the effectiveness of the model could be further validated by including comparisons with more recent equivariant architectures or domain-specific models.
3. A discussion on the efficiency of the KAN-based model would be valuable, particularly for application-oriented readers. Given the current trend where researchers may prioritize data-driven approaches over constraining models with equivariant functions, such a comparison could highlight EKAN's practical utility [1].

[1] Abramson, J., Adler, J., Dunger, J. et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature 630, 493–500 (2024). https://doi.org/10.1038/s41586-024-07487-w

1. The novelty and depth of this work appear somewhat limited, as the methodology seems relatively straightforward by extending existing techniques from MLP to EMLP (Finzi et al., 2021) to construct gated basis functions and equivariant linear weights. It would strengthen the paper if the authors provided deeper insights into the intrinsic challenges or unique aspects of building EKAN from KAN, beyond the application of existing equivariant techniques.
2. Certain numerical tests lack fair benchmarking. For instance, more variations in the width and depth of MLP and EMLP should be considered to ensure comparability. Additionally, the results in Table 3 suggest that the performance of EMLP or EKAN may depend on balancing model size with training set size. However, model sizes for (E)MLP and (E)KAN are not equivalent across tests. It would be helpful if the authors presented results for MLP and EMLP with model sizes under 50K, as well as for KAN and EKAN with sizes over 100K, to provide a more balanced and thorough comparison. More general, the authors could test all models with 3-4 different parameter counts ranging from 30K to 150K.

Theoretical part: Main drawback of this paper is the ambiguous and unclear description of theoretical parts, which makes it very hard to follow the paper. The main cause I feel is that the distinction of the definition and property is not clear and/or some mathematical terminologies are not introduced properly. The followings are the umbiguous/unclear, but not exhaustive, parts of the paper:

Section 3

• The sentence starting ‘In general’ in line 139 is neither trivial nor understandable. What is the assumption on "the" vector space $U$ and how is it associated with the matrix group?

Section 4

• Assuming the definition of (10), for example, $T(-1, -1)$ is also allowed -- What does this notation mean?.

• As I mentioned above, how the decomposition (10) is associated to the matrix group is unclear, so I do not see how this expression helps the later discussion.

• Descriptions of lines 253-256 are very vague. For example, why is the input/output feature does not lie with in U_{I}/U_{o}? What is the necessity of “align with gated basis functions?” Why does adding a gate scalar help to obtain the actual input/output space? The expression $U_{I}/U_{o}$ is also extremely misleading, as this could also mean the quotient space of $U_{I}$.

• Section 4.2 is very hard to follow. For example, lines 280-284 are very difficult to understand. ‘’For the non-scalar term $v_{I, a}$, we apply the basis functions …$ I do not see any mathematical formulation for this, and it is hard to see whether this is the definition or the property derived from some equations. Line 281, “For the scalar term, …, which is equivalent to applying basis functions element-wise.” This description also does not make sense, since I do not think two mathematical terms which are supposed to be equivalent are not introduced already.

• While the paper reads “$U_{m}$ can be written as …”at line 284, I do not know the original definition of $U_{m}$, and I am not sure if the equations (13, 14) hold. I could not check the validity of the proof of Theorem 1.

• Unfortunately, I cannot follow the rest of the theoretical claims due to the lack of my understanding for the above questions.

Another question for the motivation of the paper is that: Why don't we use frame averaging for KAN model, but rather put hard equivariance to existing KAN model?

Experiment part: I also have some concerns on the setting of experiments.

• N-body simulations is a relatively small, while representative, physical system among other types of scientific simulations. The number of parameters for MLP and EMLP is relatively high compared to EKAN, and I suspect that this advantage might come from the difference in the number parameters. Would it be possible to change the number of parameters/layers of MLP/EMLP/EKAN to see how the number of parameters have an impact on the test accuracy?
• I think the choice of baselines is not exhaustive. For example, Steerable E(3)-GNN [1] can be applied to N-body experiments and Top Quark Tagging. Also, Clifford Group Equivariant Neural Networks [2] could be another strong baseline. While I understand that the focus is rather on the side of comparison to (E)MLP, I still think the authors should at least mention other baselines (and hopefully include those baselines in the experiments) since all the scenarios in the experiments are in scientific domain and those models are shown to be very effective to solve tasks in (some of) those experiments.

Overall, I feel the paper needs profound revision in the writing, so the paper is more self-contained and readers could follow the main idea of the paper much more comfortably.

Typo

• Line 814, Adan optimizer

[1] Brandstetter et.al., "Geometric and Physical Quantities improve E(3) Equivariant Message Passing," ICLR 2022.

[2] Ruhe et.al., "Clifford Group Equivariant Neural Networks," NeurIPS 2023

• The paper completely neglects any literature from 2021 onwards. This is close to scientific misconduct.
• Comparisons to steerable methods are missing. This is of (theoretical) interest since EMLP / EKAN operate by linear combination of scaled subspaces, whereas steerable methods in principle are more expressive - at the cost of being computationally slow.
• The top quark tagging problem has been done in many different papers in the last 2 years with extremely great results. All of this is completely ignored in the paper. It is thus really hard to judge the results.

The reviewer doesn't consider it their duty to list all literature, both for general comparison, but also for experiments on the top tagging experiment.